Last week we scaled the data. Tonight, the part other teams actually touch: the contract. You can rewrite the code whenever you like. You cannot un-promise a field.
Callers depend on your interface, not your code. You can rewrite the service tonight and nobody notices. Rename one field and everybody's code breaks tomorrow. So the senior skill is simple to say and hard to do: design the part you can't take back.
Your API is the menu other teams order from. You can change the kitchen however you like, new chef, new oven, nobody cares. But the moment you rename a dish or take it off the menu, every order already placed breaks. So you add new dishes; you don't quietly change old ones.
Two sides, written by two different teams, often who've never met. Between them sits a contract: the shapes and rules they both agree to. Neither side sees the other's code. They only see the contract, so the contract is the system as far as they're concerned.
flowchart LR C(["caller
mobile app, another team"]) -- "sends a request shaped like…" --> K subgraph K["THE CONTRACT"] direction TB K1["the address + method
POST /orders"] K2["the fields + types
{ item, qty }"] K3["the replies
201, 400, 409…"] end K -- "…and gets a reply shaped like…" --> S(["server
your service"]) classDef io fill:#DCEBFE,stroke:#1F2937,color:#0E1726; classDef k fill:#FEF3C7,stroke:#1F2937,color:#0E1726; class C,S io; class K1,K2,K3 k;
Think of two countries trading. They don't share laws or language. They share a treaty: "send this, get that." The treaty is the only thing both sides trust. Break the treaty and trade stops, even if both economies are perfectly healthy on their own.
When people say "the API", they think field names. But a caller quietly comes to depend on everything they can observe, including things you never meant to promise. Each one is a thread someone is holding. Cut it and something far away snaps.
total is a number, not a stringPOST /orders, not /createOrder200 that becomes a 404 is a breakHyrum's Law: with enough callers, every observable behaviour of your system will be depended on by somebody. The fix isn't to promise nothing, it's to decide on purpose what you promise, and write it down.
A junior reads getUser(42) and thinks "instant, always works." Over a network, that one line hides four separate steps, each of which can be slow or fail on its own. Knowing the steps is how you reason about everything later tonight.
flowchart LR A(["your code calls
getUser 42"]) --> D["1 · find the server
DNS: name → address"] D --> T["2 · open a connection
TCP + TLS handshake"] T --> R["3 · send request,
server does work"] R --> P["4 · read the reply
…or wait forever"] P --> Z(["result"]) classDef step fill:#DCEBFE,stroke:#1F2937,color:#0E1726; classDef io fill:#ffffff,stroke:#1F2937,color:#0E1726; class D,T,R,P step; class A,Z io;
A function call is walking to the next room. A network call is posting a letter overseas: first you look up the address, then a route opens, then it travels and gets handled, then a reply comes back, if it comes back. Any step can be slow, lost, or silently dropped. That "silently" is why we spend half of tonight on it.
Once a call can travel, you choose how the two sides talk. There are three mainstream styles. They're not better or worse, they're shaped for different jobs. The mistake is picking by fashion; the skill is picking by fit.
REST is a menu: fixed dishes, order by name. gRPC is a kitchen intercom: fast, terse, staff-only, everyone knows the codes. GraphQL is a build-your-own-bowl counter: you say exactly what goes in, no more, no less.
REST models your system as resources (an order, a user) each with an address. You act on them with a small fixed set of HTTP verbs. Because it's just HTTP, the whole internet, browsers, caches, proxies, already knows how to handle it.
flowchart LR R(["/orders/42
one resource, one address"]) R --> G["GET
read it"] R --> P["PUT / PATCH
change it"] R --> D["DELETE
remove it"] C(["/orders"]) --> PO["POST
create a new one"] classDef res fill:#FEF3C7,stroke:#1F2937,color:#0E1726; classDef safe fill:#D6F5E3,stroke:#1F2937,color:#0E1726; classDef write fill:#DCEBFE,stroke:#1F2937,color:#0E1726; class R,C res; class G safe; class P,D,PO write;
The win: GET is safe and cacheable. A GET should never change anything, so browsers and CDNs can store the answer and skip your server entirely. The weakness: chatty. Drawing one screen ("user, their orders, each order's items") can mean many round trips. Remember that pain, it's exactly what GraphQL fixes.
You write the contract first in a .proto file, the exact methods, fields and types, and a tool generates client and server code in every language. Messages travel as compact binary, not text. It's built for machines talking to machines, fast.
flowchart LR P(["orders.proto
the typed contract"]) --> GEN{"code generator"} GEN --> CL["client stub
Go service"] GEN --> SV["server stub
Java service"] CL -- "binary over HTTP/2
tiny + fast" --> SV classDef proto fill:#FEF3C7,stroke:#1F2937,color:#0E1726; classDef gen fill:#ffffff,stroke:#1F2937,color:#0E1726; classDef code fill:#E6E8FF,stroke:#1F2937,color:#0E1726; class P proto; class GEN gen; class CL,SV code;
The win: fast, small, strongly typed, and streaming. If the proto says qty is an int, you cannot accidentally send "three". The catch: it's binary. You can't read it in a browser, curl it, or click it in a debugger. So gRPC lives inside your system, between your own services, not on the public edge.
One endpoint. Instead of many fixed URLs, the client sends a query describing exactly the fields it needs, across multiple things, in one round trip. The server assembles precisely that, no more, no less.
flowchart LR Q(["one query:
user → name + their
orders → total"]) --> E["/graphql
single endpoint"] E --> U[("users")] E --> O[("orders")] U --> RESP(["one response,
exactly that shape"]) O --> RESP classDef q fill:#D6F5E3,stroke:#1F2937,color:#0E1726; classDef e fill:#FEF3C7,stroke:#1F2937,color:#0E1726; classDef db fill:#DCEBFE,stroke:#1F2937,color:#0E1726; class Q,RESP q; class E e; class U,O db;
The win: no over- or under-fetching. The mobile app gets a tiny payload, the desktop asks for more, same endpoint, no new versions. The catch: caching is hard (every query is different) and one innocent-looking query can quietly drag the whole database. Great for rich, varied front-ends; a footgun if you bolt it on without limits.
| Question | REST | gRPC | GraphQL |
|---|---|---|---|
| Who's it for | public & general | internal services | rich front-ends |
| On the wire | text (JSON) | binary, compact | text (JSON) |
| Who picks the shape | server (fixed) | server (fixed) | client (per query) |
| HTTP caching | easy (built in) | n/a | hard |
| Streaming | awkward | first-class | via subscriptions |
| Debug by hand | trivial (curl) | needs tooling | needs tooling |
No row says "best". Each is a trade. REST trades richness for reach. gRPC trades readability for speed. GraphQL trades simplicity for flexibility. Staff engineers talk in those trades, not in favourites.
You almost never need to agonise. Three questions get you to the right answer most of the time, and notice that real systems use more than one: gRPC inside, REST or GraphQL at the edge.
flowchart TB
Q1{"who calls it?"}
Q1 -- "the public / 3rd parties" --> REST["REST
reach, caching, anyone can use it"]
Q1 -- "your own services" --> Q2{"need speed
or streaming?"}
Q2 -- "yes" --> GRPC["gRPC
fast, typed, internal"]
Q2 -- "no" --> REST2["REST is fine
do not over-engineer"]
Q1 -- "varied front-ends,
many shapes" --> GQL["GraphQL
let the client choose"]
classDef ask fill:#ffffff,stroke:#1F2937,color:#0E1726;
classDef ans fill:#D6F5E3,stroke:#1F2937,color:#0E1726;
class Q1,Q2 ask;
class REST,REST2,GRPC,GQL ans;
The honest default for most teams: REST at the edge, gRPC between your services. Reach for GraphQL when you genuinely have many different screens fighting over one API. "We picked X because the caller is Y" is a senior sentence. "We picked X because it's modern" is not.
REST, gRPC and GraphQL are mostly you ask, server answers. But what if the server learns something first, a new message, a price change, an order shipped? It can't tap the client on the shoulder over plain request/response. There are four ways to get fresh data, on a spectrum from crude to instant.
flowchart TB Q(["client wants fresh data"]) Q --> P["POLL
ask every few seconds:
anything new? · simple, wasteful"] Q --> L["LONG-POLL
ask once, server holds the line
until there is news"] Q --> S["SSE
server streams updates to you,
one-way, over plain HTTP"] Q --> W["WEBSOCKET
open a two-way socket,
either side pushes instantly"] classDef io fill:#FEF3C7,stroke:#1F2937,color:#0E1726; classDef crude fill:#FEE4E2,stroke:#1F2937,color:#0E1726; classDef mid fill:#DCEBFE,stroke:#1F2937,color:#0E1726; classDef good fill:#D6F5E3,stroke:#1F2937,color:#0E1726; class Q io; class P crude; class L,S mid; class W good;
Polling is phoning the shop every minute to ask "is my order ready?" Long-polling is staying on the line until they say yes. SSE is them calling you when it's ready (you just listen). WebSockets is leaving the line open so either of you can talk anytime. Most "live" features need far less than a WebSocket, start at the top and only go down when you must.
Normal HTTP opens a connection, sends one request, gets one reply, closes. A socket opens once and stays open, so both sides can send messages whenever they like, with almost no per-message overhead. It starts life as an ordinary HTTP request that asks to "upgrade" into a socket.
flowchart LR C(["client"]) -- "1 · HTTP request:
please upgrade" --> S["server"] S -- "2 · 101 switching →
socket now open" --> C C -- "3 · client pushes anytime" --> S S -- "3 · server pushes anytime" --> C classDef io fill:#DCEBFE,stroke:#1F2937,color:#0E1726; classDef sv fill:#FEF3C7,stroke:#1F2937,color:#0E1726; class C io; class S sv;
Request/response is sending a letter each time. A socket is an open phone line, you don't redial to say each sentence. Perfect for chat, multiplayer, live dashboards. The cost: open lines aren't free. The server now holds thousands of live connections in memory, and a dropped line has to be detected and reconnected. You're trading simplicity for instant, two-way talk.
A webhook flips the direction. Instead of you polling "did the payment succeed yet?", you hand the provider a URL up front. When the event happens, they send an HTTP request to your URL. It's how Stripe, GitHub and most platforms tell you things happened, without you asking over and over.
flowchart LR Y(["your service"]) -- "1 · register a URL:
ping /hooks/payment" --> P["payment provider"] P -.-> E(["2 · later: payment
actually succeeds"]) E -.-> P P -- "3 · POST to your URL
{ event: paid, id: … }" --> Y Y -- "4 · reply 200 fast,
do work after" --> P classDef me fill:#D6F5E3,stroke:#1F2937,color:#0E1726; classDef them fill:#FEF3C7,stroke:#1F2937,color:#0E1726; classDef ev fill:#ffffff,stroke:#1F2937,color:#0E1726; class Y me; class P them; class E ev;
Polling is checking the mailbox every hour. A webhook is the postman ringing your bell when something arrives. Two things make webhooks senior: verify it's really them (check the signature, anyone can POST to your URL), and they'll retry, so your handler must be idempotent (remember the idempotency key from earlier? this is exactly why). Reply 200 immediately, then do the slow work.
Here's the trap that makes APIs hard. The moment your contract is public, old clients keep running, last month's mobile app, a partner's integration, a cron job nobody owns. You change the server once; you cannot change all of them. So your new code must keep the old promise.
flowchart LR S["server
you deploy v2 today"] S --> A["new app
updated ✓"] S --> B["old app
still on v1"] S --> C["partner integration
frozen for a year"] B --> X(["breaks if you
removed a field"]) C --> X classDef ok fill:#D6F5E3,stroke:#1F2937,color:#0E1726; classDef bad fill:#FEE4E2,stroke:#1F2937,color:#0E1726; classDef sv fill:#FEF3C7,stroke:#1F2937,color:#0E1726; class A ok; class B,C,X bad; class S sv;
It's like changing the shape of a power socket. Your new socket might be lovely, but every appliance ever sold still has the old plug. You don't get to visit every house. So either the new socket still fits old plugs (compatible), or you've just bricked half the country.
Almost every compatibility rule collapses to one instinct: old callers must still get what they expected. New stuff they ignore is fine. Missing or changed stuff they relied on is not. Learn this table cold and you'll avoid most outages.
int→string) · make an input required · tighten validation · change a default or status codeSometimes additive isn't enough and you genuinely need a new shape. The rule then: never flip everyone at once. Stand up the new version next to the old, move callers over, then retire the old one. Here are the three ways to label "which version".
flowchart TB N(["a breaking change"]) --> U["URL path
/v1/orders → /v2/orders
obvious, easy to route"] N --> H["Header
Accept: application/vnd.app.v2
clean URLs, less visible"] N --> F["New field, not new version
often you can avoid v2 entirely"] classDef io fill:#FEF3C7,stroke:#1F2937,color:#0E1726; classDef a fill:#DCEBFE,stroke:#1F2937,color:#0E1726; classDef best fill:#D6F5E3,stroke:#1F2937,color:#0E1726; class N io; class U,H a; class F best;
URL versioning (/v1/, /v2/) wins for being boring and obvious, everyone can see it, proxies can route on it. But the best version bump is often none: most "I need v2" moments are really "I need one more optional field". Reach for a whole new version last, not first.
You can't just turn the old version off, someone is still on it. Retiring an API is a slow, announced, measured handover. Pulling it without this is how you take down a partner and find out on Twitter.
flowchart LR A["1 · Announce
v1 retires in 6 months"] --> B["2 · Signal
Deprecation header on responses"] B --> C["3 · Watch
who is still calling v1?"] C --> D["4 · Nudge
email the last few teams"] D --> E["5 · Retire
only when traffic ≈ 0"] classDef step fill:#DCEBFE,stroke:#1F2937,color:#0E1726; classDef done fill:#D6F5E3,stroke:#1F2937,color:#0E1726; class A,B,C,D step; class E done;
You can't measure what you don't watch. The single most useful habit: log who calls each version. When you can see "three teams, 40 requests a day" instead of guessing, retiring an old API stops being scary and becomes a checklist.
Anyone can design the call that works. Staff engineers design the call that times out, retries, and arrives twice. The network lies to you in specific, repeatable ways, so there's a specific, repeatable toolkit. Four tools, and they work together.
The dangerous one is "the call timed out, did it work or not?" You genuinely don't know. Everything tonight is about making "I don't know" safe.
If you wait forever for a reply, a slow dependency doesn't just slow you, it stops you. Each waiting request holds a thread. Threads run out. Now your service is down because someone else's service is slow. A timeout is you deciding "past here, I give up and stay alive."
flowchart LR
subgraph NONE["No timeout"]
direction TB
A1(["DB gets slow"]) --> A2["every request waits…"] --> A3(["threads run out →
your service is down too"])
end
subgraph SET["Timeout set"]
direction TB
B1(["DB gets slow"]) --> B2["give up after 2s"] --> B3(["fail fast, stay alive,
show a fallback"])
end
classDef bad fill:#FEE4E2,stroke:#1F2937,color:#0E1726;
classDef good fill:#D6F5E3,stroke:#1F2937,color:#0E1726;
class A2,A3 bad;
class B2,B3 good;
A timeout is hanging up the phone when no one answers, so you can take the next call. Two rules: always set one (no library default is your friend), and budget down the chain, if the user waits 3s, the call inside it must time out sooner, say 2s, or the outer one is pointless.
A call fails, so you try again, reasonable. But if thousands of clients retry at the same instant, you hit the struggling service with a wall of traffic right when it's weakest. The retry causes the outage it was meant to survive. The fix is two words: backoff and jitter.
flowchart TB
subgraph BAD["Retry immediately, all together"]
direction LR
F1(["1000 calls fail"]) --> R1["all retry NOW"] --> D1(["service buried →
death spiral"])
end
subgraph GOOD["Backoff + jitter"]
direction LR
F2(["1000 calls fail"]) --> R2["wait 1s, 2s, 4s…
+ a random nudge"] --> D2(["retries spread out →
service recovers"])
end
classDef bad fill:#FEE4E2,stroke:#1F2937,color:#0E1726;
classDef good fill:#D6F5E3,stroke:#1F2937,color:#0E1726;
class R1,D1 bad;
class R2,D2 good;
Backoff = wait longer after each fail (1s, 2s, 4s), give it room to breathe. Jitter = add a small random delay so everyone doesn't retry on the exact same tick. And only retry the safe ones, a failed read is fine to repeat; a failed "charge card" might already have worked. Which brings us to the most important idea of the night.
"Idempotent" just means: doing it twice has the same effect as doing it once. Reading is naturally idempotent. Creating or charging is not, run it twice and you've made two orders. The trick: the client sends a unique idempotency key, and the server remembers it.
flowchart TB C(["client: charge £50
Idempotency-Key: abc-123"]) --> S{"server: seen
key abc-123?"} S -- "no, first time" --> DO["charge the card,
save key + result"] DO --> R1(["201 created"]) S -- "yes, this is a retry" --> SKIP["do not charge again,
return the saved result"] SKIP --> R2(["200, same result"]) classDef io fill:#FEF3C7,stroke:#1F2937,color:#0E1726; classDef ask fill:#ffffff,stroke:#1F2937,color:#0E1726; classDef safe fill:#D6F5E3,stroke:#1F2937,color:#0E1726; class C io; class S ask; class DO,SKIP,R1,R2 safe;
It's a coat-check ticket. You hand over the same ticket twice, you don't get two coats, you get your one coat. Now the scary case, "the call timed out, did the charge go through?", is safe: just retry with the same key. This is exactly how Stripe and every serious payments API work. Say "idempotency key" in a design review and the room knows you've shipped real systems.
Idempotency stops the same caller's retry doing damage. But what about two different callers, Alice and Bob, both editing order #42 at once? Whoever saves last silently wipes the other's change. This is the "lost update", and there are two classic fixes.
flowchart TB A(["both read order 42
version 3"]) --> AL["Alice saves
PUT, If-Match: v3"] A --> BO["Bob saves a moment later
PUT, If-Match: v3"] AL --> OK(["server: still v3? yes →
save, bump to v4 · 200 OK"]) BO --> NO(["server: still v3? NO, it is v4 →
409 conflict: re-read and retry"]) classDef io fill:#FEF3C7,stroke:#1F2937,color:#0E1726; classDef ok fill:#D6F5E3,stroke:#1F2937,color:#0E1726; classDef bad fill:#FEE4E2,stroke:#1F2937,color:#0E1726; class A io; class AL,OK ok; class BO,NO bad;
If-Match). On save, if it changed, reject with 409. Great when conflicts are rare.409 Conflict isn't a bug, it's the API protecting a write. Design the client to re-read and retry."Give me the orders" sounds innocent until there are ten million. You return them in pages. The junior way (?page=2, skip 20) quietly breaks when the data changes under you. The robust way (a cursor, "give me what's after this one") doesn't.
flowchart TB
subgraph OFF["Offset: page=2, skip 20"]
direction LR
O1(["new row inserted
at the top"]) --> O2["everything shifts down one"] --> O3(["you re-see a row, or
skip one · duplicates & gaps"])
end
subgraph CUR["Cursor: after order 840"]
direction LR
C1(["new row inserted"]) --> C2["you still ask for
after 840"] --> C3(["stable · no dupes,
no skips, scales"])
end
classDef bad fill:#FEE4E2,stroke:#1F2937,color:#0E1726;
classDef good fill:#D6F5E3,stroke:#1F2937,color:#0E1726;
class O2,O3 bad;
class C2,C3 good;
Offset pagination is counting "skip the first 40 books on the shelf". If someone adds a book to the front while you read, your count is now wrong, you see one twice or miss one. A cursor is a bookmark: "carry on after this exact book." People insert all they like; your place holds. Default to cursors for anything that grows.
Callers write code against your failures, not just your successes. Two things to get right: the status code (so machines can react) and a consistent error body (so humans can debug). Vague, inconsistent errors are the #1 reason an API is miserable to use.
flowchart TB
R(["a response comes back"]) --> Q{"which family?"}
Q -- "2xx" --> OK(["it worked
200 ok · 201 created · 204 no body"])
Q -- "4xx · your request" --> C4["fix the request
400 bad · 401/403 auth · 404 missing
409 conflict · 429 slow down"]
Q -- "5xx · our server" --> C5["not your fault
500 we broke · 503 overloaded"]
C4 --> DR(["do NOT retry blindly:
the input is wrong"])
C5 --> RT(["safe to retry
with backoff + jitter"])
classDef ask fill:#ffffff,stroke:#1F2937,color:#0E1726;
classDef ok fill:#D6F5E3,stroke:#1F2937,color:#0E1726;
classDef warn fill:#FEF3C7,stroke:#1F2937,color:#0E1726;
classDef bad fill:#FEE4E2,stroke:#1F2937,color:#0E1726;
class R,Q ask;
class OK,RT ok;
class C4,DR warn;
class C5 bad;
The line that matters: 4xx means "don't bother retrying, fix your request"; 5xx means "not your fault, a retry might work". Clients literally branch on this, it's why retries and status codes are the same conversation. And keep one error shape everywhere, e.g. { code, message, details }, so callers write the handling once instead of guessing per endpoint.
5 · Inventing your own error format per endpoint. The fix for all five is the same instinct as week one: name the promise you're making, and name what it costs to keep it.
Pick one write from your own app, "place an order", "send a message", "follow a user". Don't open an editor. Answer these four and the contract designs itself.
Drop your endpoint + its idempotency story in the chat. We'll run a few live, and the retry question is where it gets interesting.
Pick the protocol from who's calling. Change additively, version only when you must, and retire the old one slowly. Then treat timeouts, retries, idempotency, pagination and errors as part of the design, not afterthoughts. The failed, retried, timed-out call is the real spec.