Design Twitter / X: the home timeline

The pattern, first

Fan-out & CQRS: the pattern this design is an instance of

Before the boxes, learn the shape, because it transfers to almost every feed you will ever build. The problem: many writers (the people you follow) merge into one reader's view (your timeline), and reads vastly outnumber writes. The pattern that solves it has three ideas.

Drawn as one picture, the pattern is a split down the middle: the command side takes writes into a normalized, durable source of truth; an asynchronous event syncs that into the query side, where each user's timeline is a materialized view served in a single lookup. The dotted edge is where all the interesting trade-offs live.

flowchart LR
    subgraph CMD["Command side · optimised for writes"]
      P["POST /tweets"] --> WM[("Write model
normalized · durable SoT")]
    end
    subgraph QRY["Query side · optimised for reads"]
      RM[("Read model
per-user materialized timeline")] --> G["GET /timeline
single O(1) lookup"]
    end
    WM -. "async event · fan-out
(eventual consistency)" .-> RM
    classDef db fill:#DCEBFE,stroke:#09244B,color:#09244B
    classDef cache fill:#FFEDD5,stroke:#C2410C,color:#09244B
    class WM db
    class RM cache

Pattern maps to S7 (architectural styles: CQRS, event-driven fan-out) and S1 (read vs write optimisation, precompute vs derive).

Step 1

Requirements: functional and non-functional

Bound the problem first. The functional core is three verbs:

• Post a tweet (short text, an id, a timestamp, an author).
• Follow / unfollow a user (the social graph).
• Read the home timeline: a merged, recent-first feed of the accounts you follow.

Out of scope (say it out loud): search, DMs, trends, ads, notifications, media transcoding. We will design for text and ids; media is a blob-store-plus-CDN aside, not the hard part here.

The non-functional shape is what actually drives the architecture:

Two facts decide everything downstream: reads vastly outnumber writes, and a tweet showing up a couple of seconds late is completely acceptable. That second fact is permission to precompute and to choose availability over strong consistency.

Maps to S2 (requirements to numbers) and S1 (the consistency vs availability axis).

Step 2

Estimation: the numbers that bound the design

Back-of-envelope, with round numbers you can defend:

The asymmetry is easier to feel than to read. One tweet is one row in the tweet store, but N rows in N follower timelines, and N is drawn from a heavy-tailed distribution:

flowchart TD
    T["1 tweet written
(6k / sec total)"] --> STORE[("+1 row in tweet store
always cheap")]
    T --> AMP{"author's
follower count?"}
    AMP -->|"~200 (typical)"| OK["~200 timeline writes
push it · cheap"]
    AMP -->|"100,000,000 (celebrity)"| BAD["100M timeline writes
fan-out storm · never push"]
    classDef db fill:#DCEBFE,stroke:#09244B,color:#09244B
    classDef ok fill:#D6F5E3,stroke:#15803D,color:#09244B
    classDef bad fill:#FEE2E2,stroke:#B91C1C,color:#7F1D1D
    class STORE db
    class OK ok
    class BAD bad

Maps to S2 (capacity estimation and the estimation cheat sheet).

Step 3

API: the handful of endpoints

Name the contracts that carry the product, not every route.

The timeline read is the hot path, so it is cursor-paginated (stable under new tweets) and returns fully hydrated tweets so the client makes one call, not N.

Maps to S5 (API design): idempotent writes, cursor pagination, one round trip. See the API cheat sheet.

Step 4

Data model and store choices

Four entities, three stores, each chosen from its access pattern rather than habit.

User(id PK, handle, name, follower_count)
Tweet(id PK = Snowflake, author_id, text, created_at)      access: by id, by author
Follow(follower_id, followee_id, created_at)            access: "who do I follow", "who follows X"
TimelineCache[user_id] -> [tweet_id, tweet_id, ...]       access: read my precomputed feed

stores
  Tweets ........ sharded KV / wide-column (by tweet_id)   immutable, huge, point lookups   -> Cassandra / Manhattan-style
  Social graph .. who-follows-whom, both directions        adjacency, fan-out source        -> sharded store + graph index
  Timeline cache  per-user list of recent tweet ids        in-memory, fast, evictable       -> Redis

The four entities and how they relate. Note the two edges out of User into Follow: the graph is read in both directions, and that double edge is exactly what the fan-out step walks.

erDiagram
    USER ||--o{ TWEET : authors
    USER ||--o{ FOLLOW : "follows (as follower)"
    USER ||--o{ FOLLOW : "is followed (as followee)"
    USER ||--|| TIMELINE : "owns cache"
    TWEET }o--o{ TIMELINE : "id referenced by"
    USER {
        snowflake id PK
        string handle
        int follower_count
    }
    TWEET {
        snowflake id PK
        snowflake author_id FK
        string text
        timestamp created_at
    }
    FOLLOW {
        snowflake follower_id FK
        snowflake followee_id FK
        timestamp created_at
    }
    TIMELINE {
        snowflake user_id PK
        list tweet_ids "recent-first"
    }

Maps to S3 (picking a store from access patterns) and S4 (sharding the tweet store and the social graph). See the data-stores and data-at-scale cheat sheets.

Step 5

High-level architecture

Two paths share the stores. The write path (top) takes a tweet, persists it, and fans it out to followers' timeline caches through an async queue. The read path (bottom) serves the precomputed timeline from Redis, hydrates the tweet bodies, and ranks. Splitting write from read is what lets the cheap, frequent reads stay fast while the expensive fan-out happens off the critical path.

The same two paths as a sequence diagram, so you can see the ordering the box diagram only implies: the write is acknowledged before the fan-out runs (the queue decouples them), and the celebrity branch is pulled at read time instead of pushed at write time.

sequenceDiagram
    autonumber
    actor U as User
    participant GW as API gateway
    participant TW as Tweet service
    participant Q as Fan-out queue
    participant FW as Fan-out workers
    participant G as Social graph
    participant TL as Timeline cache
    participant TS as Tweet store
    participant RK as Ranking

    Note over U,TS: WRITE PATH · fan-out on write (push)
    U->>GW: POST /tweets
    GW->>TW: create tweet
    TW->>TS: persist tweet (durable)
    TW-->>U: 201 + id (fast, before fan-out)
    TW->>Q: enqueue fan-out job
    Q->>FW: deliver job
    FW->>G: followers of author?
    alt normal author
        FW->>TL: push tweet id into each follower timeline
    else celebrity (millions of followers)
        FW-->>FW: skip fan-out, pull at read
    end

    Note over U,RK: READ PATH · assemble + hybrid pull
    U->>GW: GET /timeline?cursor
    GW->>TL: read precomputed tweet ids
    GW->>TS: hydrate tweet bodies
    GW->>G: which celebrities do I follow?
    GW->>TS: pull their recent tweets (fan-out on read)
    GW->>RK: score and merge (fallback: reverse-chron)
    RK-->>GW: top-K ranked timeline
    GW-->>U: hydrated, ranked timeline

Maps to S7 (architectural styles) and S6 (queues and async fan-out).

Step 6

Deep dive A: fan-out on write vs fan-out on read

This is the decision the whole design turns on. When does the merge of many authors into one feed happen, at write time or at read time?

FAN-OUT ON WRITE (push)                 FAN-OUT ON READ (pull)
─────────────────────────              ─────────────────────────
on POST: copy tweet id into            on GET: gather followees,
every follower's timeline cache        query each author's recent
                                        tweets, merge + sort now
reads:  O(1), just read my list        reads:  O(followees), expensive
writes: O(followers), expensive        writes: O(1), cheap
great for: the common reader           great for: rare readers,
                                        users who follow huge numbers

The same choice as two flows. Push does the expensive merge once, at write time, so the read is trivial; pull defers everything to read time, so the write is trivial. Where you put the work is the entire decision.

flowchart LR
    subgraph PUSH["Fan-out on write · PUSH"]
      direction TB
      wA["POST tweet"] --> wB["copy id into every
follower's timeline
(work here · O(followers))"]
      wB --> wC["GET timeline
= 1 lookup · O(1)"]
    end
    subgraph PULL["Fan-out on read · PULL"]
      direction TB
      rA["POST tweet
= 1 write · O(1)"] --> rB["GET timeline"]
      rB --> rC["gather followees, query each,
merge + sort now
(work here · O(followees))"]
    end
    classDef work fill:#FFEDD5,stroke:#C2410C,color:#09244B
    classDef cheap fill:#D6F5E3,stroke:#15803D,color:#09244B
    class wB,rC work
    class wC,rA cheap

Maps to S1 (latency vs cost, read vs write optimisation) and S4 (precompute and cache).

Step 6 · continued

Deep dive B: the celebrity hot-key problem

Pure fan-out-on-write breaks on one input: a user with 100M followers. One tweet would trigger 100M timeline writes, a fan-out storm that floods the queue, lags every other write, and hammers one slice of the cache (a hot key).

normal user (200 followers)   ->  push: 200 writes        cheap, do it
celebrity (100,000,000)       ->  push: 100M writes        NO

hybrid solution
  - do NOT fan out tweets from high-follower accounts
  - store celebrity tweets in a small "celebrity cache"
  - at READ time, merge: my precomputed timeline  +  recent
    tweets from the few celebrities I follow (pulled live)
  => write cost bounded, read cost bounded (few celebs followed)

The hybrid drawn as one decision. Write time forks on follower count; read time always merges both sources. The celebrity's tweets simply take the pull path while everyone else takes the push path, and the reader stitches them together.

flowchart TD
    NT["New tweet"] --> Q{"author a
high-follower account?"}
    Q -->|"no · normal"| PUSH["push id into every
follower's timeline cache"]
    Q -->|"yes · celebrity"| CC[("store in celebrity cache
· do NOT fan out")]
    PUSH --> TL[("per-user timeline cache
(precomputed)")]

    RD["GET /timeline"] --> M["merge at read"]
    TL --> M
    CC -. "pull recent tweets from
the few celebs I follow" .-> M
    M --> OUT["bounded write cost
+ bounded read cost"]

    classDef cache fill:#FFEDD5,stroke:#C2410C,color:#09244B
    classDef ok fill:#D6F5E3,stroke:#15803D,color:#09244B
    class CC,TL cache
    class OUT ok

Maps to S4 (hot keys, skew) and S9 (a fan-out storm is a self-inflicted overload).

Step 6 · continued

Deep dive C: ranking the feed

Reverse-chronological is the honest baseline and a fine first answer. A ranked feed adds a scoring step over the candidate set (the merged timeline) at read time: features like recency, author affinity, and predicted engagement produce a score, and the top-K are returned. Keep it a separate, optional service so the timeline still works when ranking is degraded.

candidate set (merged: precomputed + celebrity pulls)
        │
        ▼
  ranking service  ── score(recency, affinity, engagement) ──►  top-K
        │                                                         │
        └─ if ranking is down / slow ─► fall back to reverse-chron ┘

Ranking sits as an optional stage between the candidate set and the response, with a hard fallback edge so a ranking outage degrades the feed's order, never its availability.

flowchart LR
    CS["candidate set
precomputed + celebrity pulls"] --> RK{"ranking service
healthy & in time?"}
    RK -->|yes| SC["score(recency,
affinity, engagement)"] --> TK["top-K, ranked"]
    RK -->|"down / slow / over budget"| RC["reverse-chronological
(fail open)"]
    TK --> OUT["timeline response"]
    RC --> OUT
    classDef opt fill:#DCEBFE,stroke:#09244B,color:#09244B
    classDef fb fill:#D6F5E3,stroke:#15803D,color:#09244B
    class SC,TK opt
    class RC fb

Maps to S1 (simplicity vs capability: ship reverse-chron first) and S9 (ranking fails open to reverse-chron).

Step 7

Reliability & failure design

Name the blast radius and the failure mode per path. The timeline is a place where partial service clearly beats none.

Every cross-service call is bounded: timeouts with a budget, retries with backoff and jitter on idempotent calls, a circuit breaker around ranking, and a bulkhead so the fan-out queue can back up without touching the read pool. The fan-out storm from deep dive B is itself the cascading-failure risk; the celebrity split is the firebreak.

The failure map, drawn as blast radius. Everything on the read path is designed to degrade and keep serving; only the write path is allowed to fail closed, because durability is the one thing we refuse to trade.

flowchart TD
    subgraph READ["Read path · degrade, never go dark"]
      RANK["ranking down"] -->|fail open| RC["reverse-chron timeline"]
      MISS["Redis timeline miss"] -->|degrade| REB["rebuild from graph + tweet store
(slower, still correct)"]
      FAN["fan-out queue backs up"] -->|"bulkhead · isolated from reads"| LAG["new tweets appear late
reads unaffected"]
    end
    subgraph WRITE["Write path · protect durability"]
      TW["tweet write fails"] -->|fail closed| RETRY["error + retry w/ backoff
never silently lost"]
    end
    classDef ok fill:#D6F5E3,stroke:#15803D,color:#09244B
    classDef warn fill:#FFEDD5,stroke:#C2410C,color:#09244B
    classDef bad fill:#FEE2E2,stroke:#B91C1C,color:#7F1D1D
    class RC,LAG ok
    class REB warn
    class RETRY bad

Maps to S9 (designing for failure). See the resilience cheat sheet.

Step 7 · continued

Observability & operability

Pick SLIs the user actually feels, then alert on them.

• SLI / SLO: timeline reads served under 200ms, 99.9% over 28 days. The error budget is ~43 min/month.
• Fan-out lag SLI: time from POST /tweets to the tweet appearing in a follower's timeline. This is the one users notice that a naive design ignores.
• The three pillars: metrics for the SLOs and queue depth, traces to find the slow hop on a timeline read, structured logs to explain a specific failure.
• Alert on pain and burn: page on timeline-error-budget burn rate and on fan-out-queue depth climbing (the storm signal), not on raw CPU.

Maps to S10 (observability and operational readiness). See the observability cheat sheet.

Step 8

Trade-offs along the six core axes

The part interviewers score: name the decision each axis bought, and the least-wrong call.

The whole design is one asymmetry handled well: reads dominate and must be fast, so precompute; fan-out is cheap for the many and ruinous for the few, so split it. Say that sentence and the boxes follow from it.

Maps to S1 (the six core trade-offs) and S12 (present and defend the design).