Classic Case Studies

Design a Chat System (WhatsApp)

Watch: "Design Whatsapp: System Design Interview w/ a Ex-Meta Senior Manager" — Hello Interview

In simple words

Each phone keeps an open line to a switchboard. When you send a message, the switchboard saves it in a logbook first, tells you it's received, then connects to your friend's line. If your friend is offline, the message waits in the logbook and a buzz (push notification) tells them.

Problem Statement

Design a chat app like WhatsApp or Messenger: one-to-one and small group conversations with messages delivered in real time when the recipient is online, stored and delivered later when they're offline, with sent/delivered/read receipts and online presence.

Functional Requirements

Send and receive 1:1 messages in real time.
Group chats (up to ~500 members).
Offline delivery: messages wait and arrive when the user reconnects; push notification while offline.
Delivery and read receipts; online/last-seen presence.
Message history across devices.
Out of scope: voice/video calls, end-to-end encryption details (mention only), stories.

Non-Functional Requirements

Low latency: under ~200 ms end-to-end when both are online.
No message loss, and messages shown in order within a conversation.
High availability; tolerate flaky mobile networks (frequent reconnects).
Scale: 500M DAU, ~50M concurrent connections, 40 messages/user/day.

Messages: 500M x 40 = 20B/day -> ~230K/s avg, ~600K/s peak
Storage:  20B x ~200 B = 4 TB/day -> write-heavy, append-only
Connections: 50M concurrent / ~500K per gateway server -> ~100+ gateway servers

Core Entities

User, Device (a user may have several).
Conversation (1:1 or group) and Membership.
Message: message_id, conversation_id, sender_id, body, created_at, client_msg_id.
Receipt: per message per recipient: delivered_at, read_at (or a per-user "read up to" marker).

API Design

Real-time channel: a WebSocket per device, frames in both directions
  -> { type: "send", client_msg_id, conversation_id, body }
  <- { type: "ack", client_msg_id, message_id, ts }          (server stored it)
  <- { type: "message", message_id, conversation_id, sender, body, ts }
  -> { type: "delivered" | "read", conversation_id, up_to_message_id }

REST for everything else
  GET  /v1/conversations/{id}/messages?before=<message_id>&limit=50   (history)
  POST /v1/groups { name, member_ids }        GET /v1/sync?since=<cursor>

High-Level Design (HLD)

Save first, then acknowledge and route: the registry says which gateway holds the recipient's connection.

device A ==ws==> Gateway 1           Gateway 7 <==ws== device B
                    |                        ^
                    v                        |
               Chat service ---------> route to B's gateway
                 1. dedupe (client_msg_id)    (connection registry:
                 2. assign message_id          user/device -> gateway, in Redis)
                 3. persist to message store
                 4. ack to A
                 5. online? -> deliver via B's gateway
                    offline? -> push notification service (APNs/FCM)

Message store: wide-column DB (Cassandra/HBase-style), partitioned by conversation_id
Presence service: heartbeats -> Redis with TTL
Group fan-out: chat service delivers to each member's gateway (groups are small)

Gateways hold the long-lived WebSocket connections and are otherwise stateless; business logic sits in the chat service.
The message is durably stored before it's acknowledged to the sender; delivery happens after.

Low-Level Design (LLD)

One partition per conversation keeps history ordered and cheap to page; receipts are a per-member 'read up to' marker.

Message storage and ordering

messages
  PARTITION KEY  conversation_id
  CLUSTERING KEY message_id DESC       -- newest first, cheap "latest 50"
  columns: sender_id, body, created_at, client_msg_id

message_id: time-sortable and increasing per conversation
  (e.g. snowflake, or a per-conversation sequence assigned by the one chat
   server that owns the conversation) -> total order within a conversation

Partitioning by conversation keeps each conversation's history on one partition and makes "load last 50" a single sequential read. Very large, long-lived groups can bucket by time (conversation_id + month) to cap partition size.

Delivery, retries and sync

The client retries a send until it gets an ack; the server dedupes by client_msg_id, so retries never duplicate.
Each device keeps a cursor (last message id it has). On reconnect it calls sync?since=cursor and receives everything it missed, so the WebSocket is an optimisation and the store is the source of truth.
Receipts are compact: store a per-user "delivered up to" and "read up to" message id per conversation, not a row per message.

Connection registry and routing

on connect:    SET conn:{user}:{device} = gateway_7  (TTL, refreshed by heartbeat)
deliver(msg):  for each device of recipient:
                 gw = GET conn:{user}:{device}
                 if gw: send msg to gw via its inbox channel (pub/sub or RPC)
                 else:  enqueue push notification
on disconnect: DEL conn key; last_seen = now

Presence

Clients heartbeat every ~30 s; presence is a Redis key with a TTL slightly longer. Presence changes are sent only to users who currently have the chat open (subscribe on open), not broadcast to all contacts, which would be enormous fan-out.

Follow-up Questions

A gateway server crashes with 500K connections. What happens?: Clients detect the drop and reconnect (with jittered backoff) through the load balancer to other gateways, re-register, and sync from their cursor. Nothing is lost because messages were persisted before acks.
How do you keep message order when the sender has a bad connection and retries?: Order is defined by the server-assigned message_id, not arrival time on the client; dedupe by client_msg_id prevents retry duplicates; clients render by message_id.
How does end-to-end encryption change the design?: The server stores and routes ciphertext it can't read; keys are exchanged between devices (Signal protocol). Server-side search and moderation of content become impossible; multi-device needs per-device encryption.
A group has 100K members (a broadcast channel). Does the design hold?: Per-member fan-out on every message becomes expensive; switch to pull for large channels (members fetch new messages when they open it) plus a lightweight notification, like the celebrity case in feeds.