Token Bucket Rate Limiting Algorithm
Language:
Systems & Distributed AlgorithmsTime: O(1)Space: O(1) per client

Token Bucket Rate Limiting Algorithm

Standard Technical Benchmark Problem: Design API Gateway Rate Limiter

Algorithmic Concept & Core Invariant

A bucket holds up to B tokens. Tokens refill at rate R per second. Requests consume 1 token. Lazy replenishment calculates tokens elapsed since last request without background timers.

Try Custom Input: Request Stream | Bucket Capacity

Events (req, refill, or refill+N) and | capacity (2–6)

Interactive Trace: Step 1 / 5
Bucket Tokens
🪙
🪙
🪙
3 / 3 Tokens
✓ REQUEST ALLOWED (HTTP 200 OK)
State Watch:tokens=3allowed=true
ACTIONBucket starts full (3/3 tokens). Request #1 arrives -> Consumes 1 token. ALLOWED (200 OK). Tokens left: 2.
JavaScript Reference ImplementationZero third-party dependencies
class TokenBucket {
  constructor(capacity, refillRate) {
    this.capacity = capacity;
    this.refillRate = refillRate; // tokens per second
    this.tokens = capacity;
    this.lastRefill = Date.now();
  }

  allowRequest() {
    const now = Date.now();
    const elapsedSeconds = (now - this.lastRefill) / 1000;
    
    // Lazy token refill calculation
    this.tokens = Math.min(this.capacity, this.tokens + elapsedSeconds * this.refillRate);
    this.lastRefill = now;

    if (this.tokens >= 1.0) {
      this.tokens -= 1.0;
      return true; // 200 OK
    }
    return false; // 429 Too Many Requests
  }
}
Real-World Production Infrastructure Usage

Cloudflare API Shield, Stripe endpoint throttling, AWS API Gateway throttle tiers, and preventing brute force login attacks.

Algorithm Engineering & Interview Notes

Deep-Dive Guide: Mastering Token Bucket Rate Limiting Algorithm for Technical Interviews

Token Bucket Rate Limiting Algorithm is one of the highest-frequency building blocks in coding interviews—most famously benchmarked via "Design API Gateway Rate Limiter". A bucket holds up to B tokens. Tokens refill at rate R per second. Requests consume 1 token. Lazy replenishment calculates tokens elapsed since last request without background timers.

From an asymptotic perspective, this pattern achieves O(1) time complexity and O(1) per client auxiliary space complexity. Pure LeetCode ignores concurrency. Staff engineers know how Token Bucket uses lazy timestamp math to avoid expensive ticking background timers.

In production engineering, this exact technique powers real infrastructure: Cloudflare API Shield, Stripe endpoint throttling, AWS API Gateway throttle tiers, and preventing brute force login attacks. Stepping through the live state trace and comparing both the JavaScript and Python 3 implementations above builds intuition for pointer bounds, loop termination conditions, and edge cases.

Interview Execution & Edge-Case Checklist

  • Verify the invariant before writing a loop: know exactly what every pointer, index, or auxiliary structure guarantees at each step of Token Bucket Rate Limiting Algorithm.
  • Check language mutability rules: in JavaScript and Python, strings are immutable, so in-place character algorithms expect a mutable array (char[]) or list[str].
  • Trace small boundary cases manually (empty input, 1 element, 2 elements, duplicates) before declaring the solution complete.
  • Explain both Time (O(1)) and Space (O(1) per client) complexity clearly, including recursion stack or helper allocations.

Top 10 Frequently Asked Questions (Candidates & Hiring Companies)

1. Why does Token Bucket Rate Limiting Algorithm run in O(1) time and O(1) per client space?

A bucket holds up to B tokens. Tokens refill at rate R per second. Requests consume 1 token. Lazy replenishment calculates tokens elapsed since last request without background timers. Each step makes monotonic progress toward termination without revisiting eliminated work, yielding O(1) runtime while keeping extra working memory bounded to O(1) per client.

2. What is the classic benchmark problem used to test Token Bucket Rate Limiting Algorithm in interviews?

Interviewers frequently introduce this pattern through "Design API Gateway Rate Limiter" (Systems & Distributed Algorithms), and then ask follow-ups testing boundary inputs, streaming data, or tighter space constraints.

3. What is the most common bug candidates make when implementing Token Bucket Rate Limiting Algorithm?

The top mistakes are off-by-one loop conditions (< vs <=), forgetting empty or single-element base cases, mutating immutable strings directly instead of working on an array/list, and missing state resets across iterations.

4. Where is Token Bucket Rate Limiting Algorithm used in real-world production software systems?

Beyond coding interviews, Token Bucket Rate Limiting Algorithm directly underlies production workloads such as: Cloudflare API Shield, Stripe endpoint throttling, AWS API Gateway throttle tiers, and preventing brute force login attacks.

5. What follow-up constraints do senior interviewers add after solving Design API Gateway Rate Limiter?

Expect interviewers to ask how your approach behaves if the input exceeds single-machine RAM, arrives as an unbounded stream, contains duplicate keys or Unicode multi-byte tokens, or must run concurrently without lock contention.

6. How do interviewers at Google, Amazon, Meta, and Microsoft grade DSA solutions?

Hiring committees grade four independent signals: (1) clarifying ambiguous constraints and edge cases before coding, (2) selecting an optimal pattern and stating exact Time and Space Big-O trade-offs, (3) writing clean bug-free JavaScript or Python code while narrating invariants, and (4) walking through a manual dry-run trace to catch off-by-one errors.

7. Should I code DSA interviews in Python or JavaScript?

Use whichever language you write most fluently under time pressure. Python offers concise tuple unpacking, heapq, collections.deque, and slicing—ideal for rapid LeetCode and data engineering screens. JavaScript is preferred for frontend and full-stack product loops, though remember JS has no built-in heap and strings are immutable just like in Python.

8. How do you identify which of the 24 DSA patterns applies within the first 2 minutes?

Look at input structure and constraints: sorted arrays or target pairs signal Two Pointers or Binary Search; contiguous subarray/substring bounds point to Sliding Window; parent-child or dependency hierarchies use DFS/BFS or Trie; top-K / streaming medians use Min-Heaps; and overlapping subproblems with optimal substructure require Memoization or Tabulation DP.

9. What edge cases do interviewers test on almost every DSA coding round?

Expect reviewers to test empty inputs ([] or ""), single-element inputs, duplicate keys, negative or zero values, already-sorted or reverse-sorted arrays, disconnected graphs or cycles, and maximum constraint bounds (e.g., N = 10^5 where O(N^2) times out).

10. Why do Staff interviews still ask core DSA patterns when AI coding tools exist?

AI can autocomplete syntax, but engineers still own asymptotic latency, memory footprints, and production failure modes. Knowing why O(1) space in-place mutation or O(log N) search matters prevents database timeouts, event-loop stalls, and unexpected cloud compute bills.

Explore All 24 Interactive DSA Patterns & Complexity Tools