Rate Limiting 🚦 — Controlling Excessive Traffic

Rate limiting controls how many requests a client can make within a specific period. It protects APIs from excessive traffic and helps maintain system stability.

flowchart LR
    C["Client"] --> RL["Rate Limiter"]
    RL --> API["API Server"]
    RL --> X["Reject / Delay Request"]

1. Why Do We Need Rate Limiting?

• Protect APIs from excessive requests
• Prevent abuse
• Protect databases and downstream services
• Maintain fair usage between clients
• Control infrastructure costs

2. Basic Example

Suppose an API allows 100 requests per minute per user.

flowchart LR
    C["User"] --> RL["Rate Limiter"]

    RL -->|"Requests 1-100"| API["API"]
    RL -->|"Request 101+"| R["429 Too Many Requests"]

The HTTP status code commonly used when the limit is exceeded is 429 Too Many Requests.

3. Where Can Rate Limiting Be Implemented?

flowchart LR
    C["Client"] --> G["API Gateway / Load Balancer"]
    G --> RL["Rate Limiting"]
    RL --> API["Backend Services"]
    API --> DB["Database"]

In distributed systems, rate limiting is commonly placed at the API Gateway or another shared infrastructure layer so multiple application instances can follow the same policy.

4. Common Rate Limiting Algorithms

Token Bucket

Tokens are added to a bucket at a fixed rate. Each request consumes a token. If no token is available, the request is rejected or delayed.

flowchart LR
    T["Token Refill"] --> B["Token Bucket"]
    C["Request"] --> B
    B -->|"Token Available"| API["API"]
    B -->|"No Token"| R["Reject"]

Leaky Bucket

Requests enter a queue and are processed at a controlled rate.

flowchart LR
    C["Requests"] --> B["Queue / Bucket"]
    B --> API["API"]
    B --> R["Overflow / Reject"]

Fixed Window

Count requests during a fixed time window.

flowchart LR
    C["Requests"] --> W["1 Minute Window"]
    W -->|"Within Limit"| API["API"]
    W -->|"Limit Exceeded"| R["Reject"]

Sliding Window

Evaluates requests over a moving time period instead of independent fixed windows.

5. Rate Limiting by Different Keys

• User ID
• API key
• IP address
• Client/application
• Endpoint

flowchart LR
    C["Client"] --> RL["Rate Limiter"]

    RL --> U["User Limit"]
    RL --> IP["IP Limit"]
    RL --> APIK["API Key Limit"]
    RL --> EP["Endpoint Limit"]

6. Distributed Rate Limiting

If multiple API servers maintain independent counters, a client could bypass the intended limit by sending requests to different servers.

flowchart LR
    C["Client"] --> LB["Load Balancer"]

    LB --> A["API 1"]
    LB --> B["API 2"]
    LB --> D["API 3"]

    A --> R["Shared Redis"]
    B --> R
    D --> R

A shared store such as Redis can maintain distributed counters or tokens so multiple application instances enforce a common limit.

7. Rate Limit Response

HTTP/1.1 429 Too Many Requests

The response can also provide information about when the client can retry, depending on the API design.

8. Rate Limiting vs Throttling

Important Interview Questions

• What is rate limiting?
• Why do we need rate limiting?
• What is Token Bucket?
• Token Bucket vs Leaky Bucket?
• Fixed Window vs Sliding Window?
• How do you implement distributed rate limiting?
• Why is Redis commonly used for distributed rate limiting?
• What HTTP status code is returned when the limit is exceeded?
• Should rate limiting happen at the API Gateway or application?
• How would you rate-limit each user independently?

Key Takeaways

• Rate limiting controls how frequently clients can access an API.
• 429 is commonly returned when a limit is exceeded.
• Token Bucket is a common algorithm.
• Distributed systems need a shared rate-limit state when limits must apply globally.
• Redis is commonly used for shared counters or token state.
• Rate limits can be applied per user, IP, API key, client, or endpoint.

Conclusion

For interviews, remember: Client → Rate Limiter → API. The goal is to prevent excessive traffic from overwhelming the system while maintaining controlled and fair access.