Design WhatsApp: chat & messaging

Step 1

Requirements: functional and non-functional

The functional core:

• Send / receive a message, 1:1 and in small groups.
• Delivery receipts: the sent → delivered → read state every chat shows.
• Ordering: messages in a conversation arrive in the order they were sent.
• Offline delivery: a message to an offline user is stored and delivered on reconnect.
• Presence: online / last-seen.

Out of scope (say it): calls, media pipelines, E2E-encryption internals, status/stories.

The non-functional shape is unusual, and it is what makes this hard:

The defining fact is not request volume, it is connection volume: a messaging system is mostly a fleet that holds an enormous number of long-lived sockets and shuttles small payloads between them. Hold that.

Maps to S2 (requirements to numbers) and S1 (consistency vs availability).

Step 2

Estimation: the numbers that bound the design

Maps to S2 (capacity estimation).

Step 3

Protocol: why this is not REST

The hot path is realtime push, so the client holds a persistent connection (a WebSocket, or an XMPP-style long connection) to a gateway, rather than polling a REST endpoint. The few operations on that connection:

REST is fine for the cold edges (auth, contact sync, media upload URLs). The realtime spine is the long-lived socket.

Those four ops in motion, for a 1:1 message with both users online. Notice the shape the whole system inherits: the server acks the sender the moment it has the message durably, then the delivered and read receipts flow back asynchronously as Bob's device reacts. Nothing blocks on the recipient.

sequenceDiagram
    autonumber
    actor A as Alice (online)
    participant GW as Conn gateway
    participant MS as Message service
    actor B as Bob (online)
    A->>GW: SEND {convId, clientMsgId, body}
    GW->>MS: forward
    MS->>MS: persist, assign msgId + per-conv seq
    MS-->>A: ACK (server has it durably)
    MS->>B: MESSAGE {msgId, seq, body}
    B-->>MS: RECEIPT {msgId, delivered}
    MS-->>A: RECEIPT {msgId, delivered}
    Note over B: Bob opens the chat
    B-->>MS: RECEIPT {msgId, read}
    MS-->>A: RECEIPT {msgId, read}

Maps to S5 (API and protocol choice): persistent connection, idempotent send. See the API cheat sheet.

Step 4

Data model and store choices

User(id PK, ...)
Device(id PK, user_id)                         a user has many devices
Conversation(id PK, type=1to1|group, members)
Message(id PK = Snowflake, conv_id, sender_id, seq, body, sent_at)
Mailbox[user_id] -> queue of undelivered message ids    store-and-forward
SessionRegistry[user_id] -> gateway_id (+ device)       who holds the live socket
Presence[user_id] -> {state, last_seen}

stores
  Messages / mailbox .. write-heavy, time-ordered, per-user queues   -> Cassandra / wide-column
  Session registry .... fast lookup user -> gateway, ephemeral        -> Redis
  Presence ............ ephemeral, TTL'd, high churn                  -> Redis

Maps to S3 (store from access patterns) and S4 (partition the mailbox by user). See the data-stores cheat sheet.

Step 5

High-level architecture

Clients hold a persistent socket to a connection gateway. To deliver a message, the message service looks up the recipient in the session registry (which gateway holds their socket) and routes it there to be pushed down. If the recipient is offline, it lands in their mailbox and is delivered on reconnect. Presence is a separate, lossy service. The whole point is decoupling "hold the socket" from "route the message."

Inside the message service: the modules

At the high level the message service is one box, because at an interview you first defend the boundaries: gateways hold sockets, one service routes and persists, presence is off to the side. But that box is not a monolith you would build as a single blob. It is one deployable made of a handful of modules that share the hot path, so a message passes through them in a fixed order rather than crossing a network between each. Zooming into that box:

The modules chained on the path a single message actually walks: in through ingest, down through the sequencer that makes it durable, then the router's one lookup with two outcomes, with the receipt manager riding the acks back to the sender and fan-out sitting in front of the router for the group case.

flowchart TD
    IN["SEND from gateway"] --> ING["Ingest and idempotency
dedupe by clientMsgId"]
    ING --> SEQ["Sequencer and persistence
assign msgId plus per-conv seq, durable write"]
    SEQ --> ACK["ACK SENT to sender"]
    SEQ --> RT{"Router / dispatcher
SessionRegistry lookup"}
    FO["Fan-out
group send expands to N, bounded by cap"] --> RT
    RT -->|"online"| PUSH["Forward to recipient gateway, push down the live socket"]
    RT -->|"offline or miss"| MB["Mailbox manager
append durable, drain in seq order on reconnect"]
    PUSH --> RCPT["Receipt manager
forward-only state machine, dedupe by msgId"]
    MB --> RCPT
    RCPT --> BACK["Forward DELIVERED then READ to the original sender"]
    classDef good fill:#D6F5E3,stroke:#1B7F46,color:#09244B
    classDef accent fill:#DCEBFE,stroke:#4A90D9,color:#09244B
    class SEQ,MB good
    class RT accent

Maps to S7 (architectural styles, service vs module boundaries) and S6 (routing, queues, idempotent delivery).

Step 6

Deep dive A: holding hundreds of millions of connections

The signature challenge. You cannot route a message to a user until you know which box currently holds their live socket, and that mapping changes constantly as people connect, drop, and roam.

connection gateway
  - one job: terminate ~1M long-lived sockets, nothing else
  - on connect:   write SessionRegistry[user] = this_gateway
  - on disconnect: clear it (and heartbeat to detect dead sockets)
  - stateless beyond the sockets it holds, so it scales horizontally

routing a message to Bob
  1. message service reads SessionRegistry[Bob] -> gateway B (or "offline")
  2. online?  forward to gateway B, which pushes down Bob's socket
  3. offline? append to Mailbox[Bob]; deliver when he reconnects

The registry is populated by the connection lifecycle itself. Every connect writes the mapping, every disconnect (or a missed heartbeat) clears it, so the registry is always an eventually-consistent picture of who is reachable where.

stateDiagram-v2
    [*] --> Connecting
    Connecting --> Online: socket established
    Online --> Online: heartbeat OK
    Online --> Offline: disconnect / missed heartbeat
    Offline --> Connecting: client retries
    Offline --> [*]
    note right of Online
        SessionRegistry[user] = this gateway
        messages route here, pushed live
    end note
    note left of Offline
        registry entry cleared
        new messages fall to the mailbox
    end note

And the routing decision every message runs through, one lookup with two outcomes:

flowchart TD
    M["Message for Bob"] --> L{"SessionRegistry[Bob]?"}
    L -->|"online → gateway B"| R["Route to gateway B"]
    R --> P["Push down Bob's live socket"]
    P --> RC["RECEIPT: delivered"]
    L -->|"offline / miss"| MB["Append to Mailbox[Bob] (durable)"]
    MB --> W["Bob reconnects"]
    W --> D["Gateway drains mailbox in seq order"]
    D --> RC

Maps to S4 (sharding the connection tier) and S7 (stateless gateways behind a registry).

Step 6 · continued

Deep dive B: delivery semantics and ordering

The receipts every user sees are a state machine, and getting them right under retries is the real work. The network will deliver twice; the design makes that harmless and keeps order within a conversation.

delivery state machine (per message)
   SENT ──(recipient's device acks)──► DELIVERED ──(opened)──► READ

ordering
  - each conversation carries a monotonic seq; clients render by seq,
    not by arrival time (sockets can reorder)
ordering scope
  - per-conversation, NOT global. global ordering is unnecessary and
    expensive; only "messages within this chat" must be in order

exactly-once, honestly
  - delivery is at-least-once; clientMsgId + msgId dedupe on both ends
  - => a retried SEND or a re-pushed MESSAGE applies once

The receipt ticks users see are exactly this state machine, and it only ever moves forward. Idempotent ids are what let a duplicate MESSAGE or a replayed RECEIPT re-assert a state without ever moving it backward.

stateDiagram-v2
    [*] --> Sent: server persisted (one tick)
    Sent --> Delivered: recipient device acks (two ticks)
    Delivered --> Read: recipient opens chat (two blue ticks)
    Read --> [*]
    Sent --> Sent: duplicate SEND (dedupe by clientMsgId)
    Delivered --> Delivered: re-pushed MESSAGE (dedupe by msgId)

Maps to S6 (idempotency, at-least-once) and S1 (consistency scope). See the resilience cheat sheet.

Step 6 · continued

Deep dive C: offline delivery and groups

Most messages are sent to someone not currently looking at their phone, so store-and-forward is the norm, not the edge case. Groups are fan-out on top of it.

offline (store-and-forward)
  recipient offline -> append to Mailbox[recipient]
  on reconnect      -> gateway drains the mailbox in seq order, then
                       deletes delivered entries

groups (small)
  send to group of N -> fan out to N mailboxes / sockets
  the WhatsApp choice: cap group size; do the fan-out server-side
  (a "fan-out on write" exactly like the timeline, but bounded by
   the group-size cap, so no celebrity problem)

multi-device
  a user has several devices -> deliver to each device's session,
  track per-device delivery so receipts are correct

Store-and-forward as a sequence: the mailbox absorbs the message while Bob is dark, and his reconnect is what triggers an ordered drain. This is the common case, not the exception.

sequenceDiagram
    autonumber
    participant MS as Message service
    participant MB as Mailbox[Bob]
    participant GW as Conn gateway
    actor B as Bob
    Note over B: Bob is offline (no registry entry)
    MS->>MB: append message (durable)
    MS-->>MS: ack sender as SENT
    Note over B: later — Bob reconnects
    B->>GW: connect (new socket)
    GW->>MB: drain undelivered in seq order
    MB-->>GW: queued messages
    GW->>B: MESSAGE ... (ordered by seq)
    B-->>GW: RECEIPT delivered
    GW->>MB: delete delivered entries

Groups are just this, fanned out at write time, deliberately bounded by a group-size cap so the fan-out can never explode into the celebrity / hot-key problem that a public timeline has.

flowchart LR
    S["Alice SENDs to group G
(N members, N ≤ cap)"] --> F["Fan-out on write
server-side"]
    F --> M1["member 1
socket or mailbox"]
    F --> M2["member 2
socket or mailbox"]
    F --> M3["member N
socket or mailbox"]

Maps to S4 (fan-out, bounded) and S9 (the mailbox is the durability guarantee).

Step 7

Reliability & failure design

The mailbox is the durability backbone: a message is acknowledged only once it is durably queued, so a gateway crash or a missed lookup costs latency, never a lost message. Presence is deliberately the lossy, fail-open service, bulkheaded from the messaging path so a presence storm cannot touch delivery. Cross-service calls are bounded with timeouts, idempotent retries, and breakers.

Maps to S9 (designing for failure): fail closed on the message, open on presence.

Step 7 · continued

Observability & operability

• SLI / SLO: end-to-end delivery latency (SEND to DELIVERED) under a target (say 99.9% within 1s for online recipients).
• Connection health: live connection count, reconnect rate, and heartbeat failures. A reconnect storm (a gateway dropping) is the signal that matters and a naive design ignores.
• The three pillars: metrics for delivery latency and mailbox depth, traces to follow one message across gateways and the message service, structured logs keyed by msgId.
• Alert on pain and burn: page on delivery-latency budget burn and on reconnect spikes, not on a single box's CPU.

Maps to S10 (observability). See the observability cheat sheet.

Step 8

Trade-offs along the six core axes

The whole design is one question answered well: where is the recipient's socket right now? Hold sockets in a dedicated tier, find them through a registry, fall back to a durable mailbox, and keep order per conversation. Say that and the boxes follow.

Maps to S1 (the six core trade-offs) and S12 (present and defend).

Part two

Extending to WhatsApp Business: every service around the core

Everything above is consumer messaging: two people, symmetric, peer-to-peer. WhatsApp Business turns one side of the conversation into a company, and that single change ripples outward. Messages are now often programmatic (an API call, not a thumb on a phone), frequently business-initiated and therefore templated and consented, metered and billed, gated by a quality score, and wired to a catalog, payments, and a support-agent inbox. The good news for the design: the delivery engine from Part one (gateways, message service, mailbox, receipts, session registry) is reused unchanged. Business is a ring of services layered around it, each earning its own boundary.

Step 9

The surface: everything WhatsApp Business does

Name the whole surface before you draw a box, grouped by what it is for:

Maps to S2 (scope the functional surface) and S1 (permissioned vs open).

Step 10

All the services: the extended architecture

The core delivery engine sits in the middle, untouched. Around it, each Business concern is its own service, drawn by the boundary rule from Part one: separate on scaling axis or blast radius, not on the org chart.

The same picture, wired: business systems call the API gateway, which checks template, quality, and metering before handing a send to the core engine; the core delivers exactly as in Part one; inbound messages and statuses fan back out through the webhook service to each business's own servers.

flowchart TD
    subgraph BIZ["Business systems, their own servers"]
      BAPP["Business app or CRM"]
      WH["Their webhook endpoint"]
    end
    subgraph CLOUD["WhatsApp Business Platform, the new ring"]
      GWAPI["Business API gateway
Cloud API, token auth, phone-number-id"]
      TMPL["Template mgmt and approval
submit, review, versioned store"]
      QUAL["Quality and rate-limit
per-number tier, quality rating"]
      METER["Conversation and billing
24h windows by category, usage"]
      WHD["Webhook delivery
reliable outbound, retried, signed"]
      CAT["Catalog and commerce
products, carts, orders"]
      PAY["Payments
WhatsApp Pay, provider settle"]
      FLOW["Flows
multi-screen forms, encrypted"]
      PROF["Profile and verification
display name, green tick"]
      INBOX["Agent inbox and routing"]
      CONS["Consent and opt-in ledger"]
    end
    subgraph CORE["Core delivery engine, reused unchanged"]
      MS["Message service"]
      GW["Conn gateways"]
      MB["Mailbox"]
      SR["Session registry"]
    end
    U["Consumer WhatsApp app"]
    BAPP -->|"send template or reply"| GWAPI
    GWAPI --> TMPL
    GWAPI --> QUAL
    GWAPI --> METER
    GWAPI --> CONS
    GWAPI --> MS
    MS --> GW
    GW --> U
    U -->|"inbound or status"| MS
    MS --> WHD
    WHD --> WH
    GWAPI --> CAT
    CAT --> PAY
    GWAPI --> FLOW
    GWAPI --> PROF
    GWAPI --> INBOX
    style CORE fill:#D6F5E3,stroke:#1B7F46,color:#09244B
    style CLOUD fill:#DCEBFE,stroke:#4A90D9,color:#09244B
    style BIZ fill:#FFEDD5,stroke:#C2410C,color:#09244B

Maps to S7 (service decomposition) and S6 (each concern as a bounded service).

Step 10 · continued

Deep dive D: templates and the 24-hour window

This is the concept that defines business messaging and the one interviewers probe. A business may not message a user freely. There are two modes, and which one applies is a function of time since the user last spoke:

business-initiated (no open window)
  must use a PRE-APPROVED template (HSM)
  category = marketing | utility | authentication
  variables filled at send; body and layout are fixed and reviewed

user-initiated (or a business reply within 24h)
  a 24h "customer-service window" is OPEN
  free-form + interactive messages allowed, no template needed

the rule
  the window OPENS when the USER messages the business
  it lasts 24h from the last user message
  outside the window -> template only

The window as a state machine. It is the mechanism that makes business messaging permissioned: a company can always answer a user who reached out, but to start a conversation it must fall back to a reviewed template, and the category of that template is what it gets billed on.

stateDiagram-v2
    [*] --> Closed
    Closed --> ServiceWindow: user messages the business first
    ServiceWindow --> ServiceWindow: within 24h, free-form allowed
    ServiceWindow --> Closed: 24h elapses with no new user message
    Closed --> TemplateOnly: business wants to re-engage
    TemplateOnly --> ServiceWindow: user replies, a new window opens
    note right of ServiceWindow
        billable conversation open
        free-form and interactive allowed
    end note
    note right of TemplateOnly
        only pre-approved templates
        marketing, utility, or authentication
    end note

Maps to S1 (permission as a consistency rule) and S6 (async approval, TTL'd window state).

Step 10 · continued

Deep dive E: reliable outbound to business servers

In Part one the mailbox buffered messages for an offline phone. Webhook delivery is the same pattern turned inside out: it buffers events for a flaky business endpoint. Every inbound message and every status change has to reach the business's own server, and that server will be down, slow, or return errors.

what must reach the business
  - inbound: a user messaged them
  - statuses: sent / delivered / read / failed for their outbound sends

guarantees (same spine as the mailbox)
  - at-least-once + idempotency: each event carries a stable id;
    the business dedupes, so a retry is harmless
  - per-business ordered queue, so statuses arrive roughly in order
  - HMAC-signed payloads, so the business can trust the source
  - bounded retries with backoff; a dead-letter after N failures

flowchart TD
    EV["Event: inbound message or status update"] --> Q["Per-business ordered queue"]
    Q --> D["Webhook dispatcher"]
    D --> POST["HTTPS POST to business callback URL, HMAC signed"]
    POST -->|"2xx"| OK["Ack and drop from queue"]
    POST -->|"error or timeout"| RETRY["Backoff, retry at-least-once"]
    RETRY --> D
    OK --> IDEM["Business dedupes by event id"]
    classDef good fill:#D6F5E3,stroke:#1B7F46,color:#09244B
    classDef bad fill:#FEE4E2,stroke:#C8102E,color:#09244B
    class OK good
    class RETRY bad

Maps to S9 (designing for failure) and S6 (idempotency, at-least-once, ordered partitions).

Step 10 · continued

Deep dive F: quality, rate tiers, and conversation billing

Two feedback loops guard the platform. Quality keeps businesses from spamming; billing is how the whole thing pays for itself. Both hang off the same conversation object.

Messaging limits escalate with proven good behaviour and collapse when users push back:

Quality rating (high / medium / low) is computed from user signals: blocks, reports, and how many messages go unread. It is an async, lossy score, so it lives beside the number's state and throttles sends; it never sits synchronously on the hot path.

Conversation-based billing meters the 24-hour windows, priced by category, not per message:

Maps to S2 (the numbers that bound cost) and S9 (exactly-counted money path, async quality signals).

Step 11

What you reused, what you added

The interview-winning summary is to show the seam clearly: the hard realtime engine did not change; Business is a permission-and-commerce ring bolted around it.

The one sentence for Part two: WhatsApp Business is the same delivery engine wrapped in a permission layer (consent, templates, quality), a metering layer (conversation billing), and a callback layer (reliable webhooks), plus commerce services (catalog, payments, Flows) hung off a programmatic API gateway. Everything hard about it, ordering, durability, idempotency, you already solved in Part one and reused by name.

Maps to S12 (present and defend: reused engine, new ring) and S7 (boundaries by axis and blast radius).