Load Balancing — L4 vs L7

Every large system eventually needs to spread traffic across many backend instances, and the level at which you do that — transport vs application — determines how smart, how fast, and how expensive the balancing is. Picture 100,000 requests/sec hitting a product that one server simply cannot absorb: you add instances, and now something has to decide which instance gets each request. This is one of the most common whiteboard questions in backend and infra interviews, and interviewers dig into the mechanics, not just the definitions.

What Is L4 Load Balancing?

L4 (Layer 4, "transport layer") load balancing spreads traffic based only on connection-level information — the source and destination IP address and port — without ever looking at what's actually inside the request. It's fast and cheap because it's just forwarding packets, the same way a mail sorting facility can route a sealed envelope by the address on the outside without opening it.

What Is L7 Load Balancing?

L7 (Layer 7, "application layer") load balancing actually opens up each request — reading the URL, headers, and cookies — before deciding where to send it. That lets it make smarter decisions, like sending everyone hitting /checkout to one group of servers and everyone hitting /search to another, at the cost of extra processing work per request.

Where This Is Used

• AWS's Network Load Balancer (NLB) is an L4 balancer used for raw TCP traffic like databases or gaming servers; its Application Load Balancer (ALB) is L7, used for routing web/API traffic by path.
• Netflix and other large streaming services use L7 load balancers to route video-metadata requests differently from actual video-chunk downloads.
• A bank's mobile app backend often uses an L7 load balancer so requests to /login can be routed to a more heavily monitored, rate-limited pool of servers than /balance checks.
• Online multiplayer games typically use L4 balancing because the extra per-packet delay of an L7 balancer would hurt reaction-time-sensitive gameplay.

A Day-to-Day Example

Picture a busy restaurant with a host stand out front. An L4 approach is like a host who only glances at how many people are in each party and sends them to whichever open table fits, without asking what they want to eat. An L7 approach is like a host who actually listens to the request — "we're here for the tasting menu" — and seats that party in the section staffed by the chef who handles it, because they took the time to understand what was being asked for.

Deep Dive: What L4 vs L7 Actually Means on the Wire

"Layer 4" and "Layer 7" refer to the OSI model. An L4 (transport-layer) balancer never assembles or reads the HTTP request — it looks at the TCP/UDP header (source IP, source port, destination IP, destination port) and makes a forwarding decision, often via NAT or direct-server-return, packet by packet or per-connection. It has no idea if the payload is GET /api/users or a binary protocol. An L7 (application-layer) balancer is fundamentally a proxy: it terminates the client's TCP connection, buffers and parses the full HTTP request, and only then opens a brand-new, separate TCP connection to the chosen backend.

L4 (transport pass-through)
Client --TCP--> [ L4 LB: reads IP:port only ] --TCP(same conn/NAT)--> Backend
                 no HTTP parsing, ~microseconds of work per packet

L7 (terminate + re-originate)
Client --TCP/TLS--> [ L7 LB: terminates, parses HTTP,
                        inspects path/host/headers/cookies,
                        picks backend, opens NEW connection ] --TCP/TLS--> Backend
                 two separate connections, CPU spent on parsing + often TLS

This is why L4 can push millions of packets/sec on modest hardware (it's doing header inspection, not parsing), while L7 costs meaningfully more CPU per request — you're paying for a full HTTP parse, potentially a TLS handshake, and a second connection setup to the backend.

Deep Dive: Load Balancing Algorithms

• Round robin — cycles through backends in order. Simple, but ignores that backends may have different capacity or that some requests are far more expensive than others.
• Weighted round robin — same idea, but a backend rated for 2x the capacity gets 2x the requests. Used when the fleet is heterogeneous (mixed instance types during a rolling upgrade, for example).
• Least connections — routes to whichever backend currently has the fewest active connections. Better than round robin when request duration varies wildly (some requests take 5ms, others take 5s) because it reacts to actual load, not just request count.
• Consistent hashing — maps both backends and request keys (e.g. user ID, cache key) onto a hash ring, and a request goes to the next backend clockwise from its hash. The payoff: when you add or remove a backend, only the keys adjacent to that node on the ring remap — roughly 1/N of keys move instead of nearly all of them, which is what a naive hash(key) % N scheme would cause. This matters enormously for cache-affinity (so the same user keeps hitting the same cache-warm backend) and for sharded stateful backends.

Naive mod-N hashing when N changes (3 -> 4 backends):
  hash(key) % 3  vs  hash(key) % 4   -> ~75% of keys remap. Cache stampede.

Consistent hashing ring:
        B1
     /       \
   B4         B2
     \       /
        B3
  Adding B5 only steals a slice of ring space from its neighbor —
  most keys keep mapping to the same backend they always did.

Key Pointers

• L4 load balancers operate on IP/TCP/UDP header info only — source/destination IP and port — with no visibility into the payload; forwarding is fast and cheap.
• L7 load balancers terminate the connection and parse the actual HTTP request (path, headers, cookies, method) to make routing decisions, enabling content-based routing, canary releases, and A/B routing — e.g. /api/users → User Service, /api/orders → Order Service.
• Common algorithms: round robin, weighted round robin, least connections, and consistent hashing (sticky routing that minimizes remapping when nodes are added/removed — useful for caches and sharded backends).
• L4 is the default for non-HTTP or latency-sensitive traffic (databases, gaming, arbitrary TCP/UDP); L7 is the default when you need to route or transform based on application content.
• L7 typically also terminates TLS and can compress or rewrite responses; L4 usually passes traffic through with far less per-packet processing.
• Health checks differ in depth: L4 checks whether a TCP connection succeeds; L7 checks whether the backend returns a healthy HTTP status/body from an actual endpoint like /healthz.

How It Works

1. Client connection arrives at the load balancer's virtual IP.
2. At L4, the balancer inspects only IP/port info and forwards packets to a backend chosen by its algorithm, without terminating the TCP session semantically.
3. At L7, the balancer terminates the client's TCP/TLS connection and parses the HTTP request.
4. L7 evaluates routing rules (host, path, headers) plus the load-balancing algorithm to pick a backend, then opens a separate connection to it.
5. The response flows back through the balancer to the client.
6. Health checks run continuously in the background — active checks poll a health endpoint on a timer, passive checks watch live request failures — pulling unhealthy backends out of rotation without operator intervention.

Try It Yourself

# Watch DNS/connection distribution across repeated requests to a
# load-balanced host — look for different response headers (e.g.
# X-Served-By, X-Backend-Id) that reveal which instance answered
for i in $(seq 1 10); do curl -s -D - -o /dev/null https://example.com/ | grep -i "x-served\|x-backend"; done

# Inspect L7 routing behavior directly with verbose curl — watch the
# TLS handshake and which Host/path headers you're actually sending
curl -v https://example.com/api/users

# Load-test to see the LB's algorithm reveal itself under concurrency;
# with least-connections, response times stay flatter under load than
# with naive round robin against heterogeneous backends
hey -z 30s -c 50 https://example.com/

# Simulate a backend dying and watch health checks pull it out of
# rotation — kill one instance in a local docker-compose LB setup
# and curl repeatedly; failed instance should stop receiving traffic
# within one health-check interval

Interview Q&A

Q: When would you choose L4 over L7, and vice versa?

A: L4 when you need raw throughput and the traffic isn't HTTP, or when you explicitly don't want to pay parsing/TLS-termination cost at the balancer — databases, gRPC over raw TCP at extreme scale, gaming/UDP traffic. L7 when routing decisions depend on the request itself: path-based service routing, canary releases by header, cookie-based session affinity, or when you want the balancer to also handle TLS termination and compression centrally instead of duplicating that in every backend.

Q: What happens when one backend server dies?

A: Health checks detect it and the load balancer stops routing new traffic to it. Active health checks poll a dedicated endpoint (like /healthz) on an interval and mark the backend unhealthy after N consecutive failures; passive checks infer unhealthiness from live traffic — a spike in connection resets or 5xx responses from that backend. Once marked unhealthy, the LB removes it from the rotation set until it starts passing checks again.

Q: Follow-up — what if the health check itself is misleading, e.g. the process is alive but the database connection pool behind it is exhausted?

A: This is why a good health check endpoint does more than return 200 for "process is up" — it should verify the backend can actually serve traffic (e.g. check DB connectivity, dependency reachability) without being so heavy that the check itself becomes a load problem. Teams often implement a "readiness" check (can this instance serve traffic right now) separate from a "liveness" check (is the process alive at all), which is exactly the Kubernetes readiness/liveness probe split.

Q: Follow-up — if health checks are slow to detect failure, what's still happening to users during that gap?

A: Requests routed to the dying backend during the detection window time out or error at the client, which is why timeouts on the caller side and retries with backoff (to a different, healthy backend, ideally) matter as a second line of defense — the LB's health check is not instantaneous, so the client-side resilience layer has to absorb the gap.

Q: Why is consistent hashing preferred over hash(key) % N for cache routing?

A: Because with plain modulo hashing, adding or removing even one backend changes the result of the modulo for the vast majority of keys, causing a mass cache miss/remap event ("cache stampede") right when the system is already in a state of change (scaling event). Consistent hashing on a ring only remaps the fraction of keys adjacent to the added/removed node, keeping cache hit rates stable during scaling.

System Design Tie-in

This shows up immediately in almost any "design a scalable web service" prompt: once you draw more than one backend instance, the interviewer expects you to explicitly place a load balancer in front of them and justify L4 vs L7. For an API gateway design, you'd reach for L7 to route /users, /orders, /payments to different microservices from one entry point. For a design involving sharded caches or consistent-hash-partitioned data stores (a distributed cache, a chat system's connection routing, a sharded database proxy), you'd specifically call out consistent hashing to justify why adding a node doesn't invalidate the whole cache. And whenever you're asked "how do you detect and handle an unhealthy instance," the health-check mechanism described here is the expected answer.

Trade-offs

L4 is faster and cheaper per connection but has zero application awareness — it can't route by URL path or do cookie-based session affinity at the HTTP layer. L7 is flexible and content-aware but costs more CPU per request and adds a hop of processing latency, plus doubles the number of TCP (and possibly TLS) connections in play. Real infrastructure often layers both: a cloud-provider L4 balancer absorbs raw connection volume and DDoS traffic at the edge, with an L7 proxy (Envoy, Nginx) behind it doing smart, content-aware routing to services. Consistent hashing trades a small amount of routing simplicity for much better stability under cluster resizing — worth it any time cache affinity or shard ownership matters, overkill for simple stateless round-robin traffic.