The goal of a design drill is not a “correct” diagram — it is to show a repeatable method. Interviewers grade the process: do you clarify scope, quantify load, separate stateless compute from state, scale each tier deliberately, and reason about failure? Start every drill with the framework below, then specialise.

The framework — run this on any question

One loop covers 90% of system‑design prompts. Say each step out loud so the interviewer can follow your reasoning; the numbers you state become the anchors for every later decision.

Drill 1 — an API used by millions of clients

Prompt: design a public API service consumed by 1M+ clients worldwide. Read‑heavy, global, must stay fast and up.

Requirements & constraints

Back‑of‑envelope (say the arithmetic)

Assume 1M DAU, ~3 sessions/day × 10 requests/session ≈ 30M req/day. Average RPS = 30M ÷ 86,400 s ≈ ~350 RPS; apply a ×5 peak factor → ~1.7k RPS. Payloads ~5 KB → egress ≈ 30M × 5 KB ≈ ~150 GB/day (the CDN absorbs most). Core dataset ~2 KB/user × 1M ≈ ~2 GB hot, growing — comfortably a single primary + replicas, no sharding yet.

API sketch

GET  /v1/items?cursor=eyJpZCI6MTB9&limit=50     # cursor pagination, cacheable
GET  /v1/items/{id}                            # ETag + Cache-Control for CDN/edge
POST /v1/items      Idempotency-Key: 9f2c…     # safe client retries
PUT  /v1/items/{id} If-Match: "<etag>"           # optimistic concurrency

# Response headers on reads
Cache-Control: public, max-age=60, stale-while-revalidate=300
ETag: "a1b2c3"

Architecture

Scaling

• Stateless horizontal scaling. Sessions/tokens are self‑contained (JWT) so any replica serves any request; autoscale the fleet on CPU/RPS.
• Read path. Cache‑aside in Redis for hot objects, CDN for cacheable GETs (ETag + stale‑while‑revalidate) — most reads never reach the origin.
• DB. Route reads to replicas, writes to the primary; connection pooling to avoid exhausting DB connections. Shard only when a single primary’s write throughput or dataset becomes the bottleneck.
• Multi‑region. Active‑passive first (simple, one write region); go active‑active only when latency or availability demands it — and accept conflict resolution.

Reliability

Tradeoffs

Decision	Option A	Option B	Pick when…
Consistency	Strong correct	Eventual fast/cheap	Reads tolerate staleness → eventual + cache; money/inventory → strong.
Region topology	Active‑passive simple	Active‑active complex	Start passive; go active‑active only for global write latency / RTO≈0.
Datastore	SQL joins·txns	NoSQL scale·flexible	Relational + transactions → SQL; massive KV/append scale → NoSQL.

Drill 2 — a routing layer for multiple LLM providers

Prompt: build a service that routes inference requests across OpenAI, Anthropic, Azure and a self‑hosted model — with failover, cost/latency optimisation, streaming, per‑provider quotas and token accounting.

Requirements & constraints

Back‑of‑envelope

Assume 500 RPS of chat completions, avg ~1.2 s provider latency, ~800 tokens per response. In‑flight concurrency ≈ RPS × latency = 500 × 1.2 ≈ ~600 concurrent streams the router must proxy. At ~$X/1k tokens, per‑provider cost accounting and a prompt cache (even 15% hit rate) is real money — routing on cost within a latency SLO pays for itself.

Architecture

Routing policy

1. Capability match. Filter to providers/models that satisfy the request (context window, modality, JSON mode, region/compliance).
2. Optimise within SLO. Among the eligible set, choose the lowest cost whose recent p95 latency fits the SLO; use weighted load balancing to honour capacity and spread risk.
3. Failover. On breaker‑open, timeout or 429, retry on the next‑best provider. Optionally hedge — fire a second request after a delay to cut tail latency — capped to control cost.

Drill 3 — a rate‑limiting system

Prompt: protect an API from abuse and overload with a fair, distributed rate limiter. Pick an algorithm, enforce it across a fleet, and return the right client signals.

Algorithms compared

Algorithm	Accuracy	Memory	Bursts	Notes
Fixed window	low	tiny	Boundary spikes	Simple counter/window; 2× burst at window edges.
Sliding window log	exact	heavy	Smooth	Stores every timestamp — precise but costly at scale.
Sliding window counter	approx	cheap	Smooth	Recommended default — weighted blend of two windows.
Token bucket	good	cheap	Allows bursts ≤ bucket	Recommended for APIs — smooth refill, controlled burst.
Leaky bucket	good	cheap	Smooths output	Fixed drain rate — great for shaping downstream load.

Token bucket — the mechanism

Sliding window — the mechanism

Distributed enforcement (a fleet, not one node)

Per‑node buckets are fast but drift across a fleet — a client hitting different replicas gets N× the intended limit. The accurate answer is a centralised store (Redis) with an atomic read‑modify‑write. The pragmatic answer at very high RPS is hybrid: a permissive local pre‑check to shed obvious abuse cheaply, reconciled asynchronously against the global counter.

-- Redis token-bucket (atomic Lua) — KEYS[1]=bucket key
-- ARGV: rate, capacity, now, requested
local rate      = tonumber(ARGV[1])   -- tokens per second
local capacity  = tonumber(ARGV[2])
local now       = tonumber(ARGV[3])
local requested = tonumber(ARGV[4])

local data   = redis.call('HMGET', KEYS[1], 'tokens', 'ts')
local tokens = tonumber(data[1]) or capacity
local ts     = tonumber(data[2]) or now

-- refill for time elapsed, cap at capacity
local delta = math.max(0, now - ts)
tokens = math.min(capacity, tokens + delta * rate)

local allowed = tokens >= requested
if allowed then tokens = tokens - requested end

redis.call('HMSET', KEYS[1], 'tokens', tokens, 'ts', now)
redis.call('EXPIRE', KEYS[1], math.ceil(capacity / rate) + 1)
return { allowed and 1 or 0, tokens }   -- 1=allow, remaining tokens

The whole script runs as one atomic operation on the Redis node, so concurrent requests can’t race the read‑modify‑write. A sliding‑window‑counter alternative is even cheaper: INCR a per‑window key and set EXPIRE, then weight the previous window’s count by the elapsed fraction.

Choice	Local per‑node	Global (Redis)
Latency	fastest in‑process	+1 RTT network hop
Accuracy across fleet	drifts N×	exact
Failure risk	None (isolated)	Hotspot / SPOF — needs HA
Best use	Cheap pre‑filter	Authoritative limit

Drill 4 — a highly available REST service

Prompt: design a REST service with no single point of failure, targeting 99.99% availability (~52 min/year of downtime).

Availability math (state it)

Serial dependencies multiply: three tiers each at 99.9% ⇒ 0.999³ ≈ 99.7% — worse than any one part. Redundancy improves it: N independent replicas each with availability a give 1 − (1 − a)ᴺ, so two 99% nodes ⇒ 99.99%. Lesson: reduce serial hops, replicate every remaining one.

Target	Downtime / year	Downtime / month
99.9% (three 9s)	~8.8 h	~43 min
99.99% (four 9s)	~52 min	~4.4 min
99.999% (five 9s)	~5.3 min	~26 s

Architecture — multi‑AZ, no SPOF

Techniques

Deploy safety

Drill 5 — a telemetry pipeline

Prompt: ingest metrics, logs and traces from many services at high volume; power near‑real‑time dashboards and alerting; stay durable under bursts while controlling cost and cardinality.

Requirements & constraints

Architecture

Key concerns

Tradeoff	Option A	Option B	Guidance
Metrics collection	Pull easy health	Push ephemeral/edge	Pull for stable targets; push for serverless/short‑lived jobs.
Trace volume	Full fidelity costly	Sampling cheap	Tail‑sample to keep errors/slow traces without storing everything.
Retention	Long $$$	Tiered balanced	Hot/warm/cold tiers; downsample old data instead of deleting.

Design cheat sheet