Design a Rate Limiter

Question

Design a rate limiter that limits the number of requests a client can send to an API within a time window.

Requirements:
- Support different time windows (per second/minute/hour)
- Handle concurrent requests
- Minimal latency
- Scalable across multiple servers

Example: Limit each client to 100 requests per minute.

Accepted Answer

Rate Limiter System Design:

1. High-Level Design:
   - Token Bucket Algorithm
   - Redis for distributed rate limiting
   - Load balancer for distribution

2. Components:
```typescript
interface RateLimiter {
  isAllowed(clientId: string): boolean;
  updateCount(clientId: string): void;
  resetCount(clientId: string): void;
}

class TokenBucketRateLimiter implements RateLimiter {
  private readonly redis: Redis;
  private readonly limit: number;
  private readonly window: number;
  
  constructor(limit: number, windowInSeconds: number) {
    this.limit = limit;
    this.window = windowInSeconds;
  }
  
  async isAllowed(clientId: string): Promise<boolean> {
    const count = await this.redis.get(clientId);
    return count < this.limit;
  }
}
```

3. Implementation Details:
   - Use Redis INCR for atomic counters
   - Sliding window with Redis ZSET
   - Lua scripts for atomic operations

4. Scalability Considerations:
   - Horizontal scaling with Redis cluster
   - Cache synchronization
   - Fault tolerance

5. Monitoring & Alerts:
   - Request rate metrics
   - Rejection rate alerts
   - Latency monitoring

6. Trade-offs:
   - Memory vs Accuracy
   - Consistency vs Availability
   - Complexity vs Performance

Company Context (Amazon):
- Focus on scalability
- Consider distributed systems
- Emphasize operational excellence
- Cost optimization

Design a Rate Limiter

Question Explain

Answer Example

Related Interview Questions