Circuit Breaker

Timeouts and backoff alone don't stop a caller from hammering a dependency that's clearly down — without a circuit breaker, Service A just keeps calling failing Service B over and over, wasting resources, threads, and connections waiting on a doomed call, and risking cascading failure across the whole system. That's the circuit breaker's job. It's the natural follow-up question after retries in a resilience-focused interview, and the closed/open/half-open state machine is the answer interviewers are listening for — along with the key insight that a breaker tracks aggregate failure rate over a window, not any single failed call.

What Is a Circuit Breaker?

A circuit breaker is a safety mechanism that stops a system from repeatedly trying to reach another part of it that's clearly broken or overwhelmed. Instead of endlessly retrying a call that keeps failing, it "trips" and starts blocking those calls for a while, giving the struggling part time to recover instead of getting hit with more traffic on top of its existing problems.

Where This Is Used

• Large apps like Netflix and Amazon use circuit breakers so that if one small feature's backend goes down, it doesn't drag the entire app down with it.
• Payment platforms use them so that if a card-processing partner is having an outage, the app stops hammering it with retries and instead shows a clear "try again later" message.
• Ride-sharing apps use them so a slow or failing pricing service doesn't stall the whole booking flow — the app can fall back to a cached estimate instead.
• Cloud infrastructure tools like Netflix's Hystrix or service meshes like Envoy build this protection in automatically for the many internal services a large company runs.

A Day-to-Day Example

Imagine calling a friend who isn't picking up. After several unanswered calls in a row, most people stop redialing every few seconds and instead wait a while before trying again, so they're not wasting effort on a call that clearly isn't going to connect. A circuit breaker automates exactly that pause for software, checking back only occasionally instead of nonstop.

Deep Dive: The State Machine

                 failure rate exceeds threshold
        +-------------------------------------------+
        |                                             v
   +----------+                                  +----------+
   |  CLOSED  |                                  |   OPEN   |
   | (normal, |                                  | (fail    |
   |  requests|                                  |  fast /  |
   |  flow    |                                  |  fallback|
   |  through)|                                  |  no calls|
   +----------+                                  |  to B)   |
        ^                                        +----------+
        |                                             |
        | probe(s) succeed                            | cooldown
        |                                              | timer
   +-----------+          probe(s) fail                | elapses
   | HALF-OPEN |<---------------------------------------+
   | (let a few|
   |  trial    |------------------------------+
   |  requests |     any probe fails -> back   |
   |  through) |        to OPEN, reset cooldown v
   +-----------+                          +----------+
                                           |   OPEN   |
                                           +----------+

• Closed (normal) — requests flow straight through to B. Failures are counted against a configured threshold — e.g. more than 50% error rate in a rolling window of the last N calls, not just "the last call failed."
• Open — once the threshold trips, the breaker short-circuits: calls fail immediately or execute a fallback, without ever touching B, for a cooldown period. This is the core payoff — A stops wasting its own resources on calls that were very likely going to fail anyway, and B gets breathing room to recover instead of being hit with continued (or retried) traffic while it's unhealthy.
• Half-open — after the cooldown elapses, the breaker cautiously lets a small number of probe requests through to test whether B has recovered. All probes succeeding closes the breaker again; any probe failing sends it straight back to open and resets the cooldown timer.

Deep Dive: Why Timeouts and Retries Alone Aren't Enough

Timeouts and retries operate per-call: a timeout bounds one request's wait, and a retry (with backoff/jitter) decides whether to try that one request again. Neither of them looks at the bigger picture — the aggregate health of the dependency across many calls. A circuit breaker sits one layer above: it's the component that watches the pattern of successes and failures over time and makes a system-level decision ("B is currently unhealthy, stop sending it traffic") rather than a per-request decision ("did this one call succeed").

Without a circuit breaker (timeouts+retries only):
  Every request to A that needs B -> times out after 2s -> retries
  2-3 times with backoff -> still fails -> finally errors out.
  Even ONE user request can cost 6-10+ seconds of wasted work, and
  under load, thousands of concurrent callers are all doing this
  AT THE SAME TIME against a B that is already down -- exactly the
  pile-up of wasted threads/connections that risks taking A down too.

With a circuit breaker (OPEN state active):
  Request to A that needs B -> breaker is OPEN -> fail fast /
  fallback immediately, no call to B attempted at all, no wasted
  timeout wait, no retry storm. B is left alone to recover.

Key Pointers

• A circuit breaker wraps calls to a dependency and tracks its failure rate; it's a state machine with three states — closed, open, and half-open.
• Closed: requests flow through normally; failures are counted against a configured threshold (error rate over a rolling window, or consecutive failures).
• Open: once the threshold trips, the breaker short-circuits — calls fail immediately or fall back to a default, without hitting the dependency, for a cooldown period.
• Half-open: after the cooldown, the breaker lets a limited number of trial requests through to test recovery; success closes the breaker again, failure sends it back to open.
• Circuit breakers complement timeouts and retries rather than replace them — timeouts bound a single call, retries absorb transient blips, and the breaker stops retrying a dependency that's clearly down, preventing wasted calls and cascading resource exhaustion upstream.
• Netflix's Hystrix (now legacy) popularized the pattern; Resilience4j is a common modern implementation, and service meshes like Envoy can enforce it natively at the infrastructure layer without application code changes.

How It Works

1. A client call passes through the circuit breaker on its way to the dependency.
2. The breaker records success/failure and evaluates it against a configured threshold, such as more than 50% errors in a rolling window.
3. On threshold breach, the breaker trips to open — subsequent calls fail fast, or execute a fallback, without touching the dependency.
4. After a cooldown timer elapses, the breaker moves to half-open and allows a small number of probe requests through.
5. Probe success transitions the breaker back to closed; probe failure sends it back to open and resets the cooldown.

Try It Yourself

# Simulate a dependency going down and watch a breaker trip --
# with Resilience4j (Java) or a Python lib like pybreaker/circuitbreaker,
# point a wrapped client at a deliberately-failing endpoint
curl -s -o /dev/null -w "%{http_code}\n" https://httpstat.us/500  # simulate failure

# Hammer a failing endpoint repeatedly and observe fail-fast behavior
# once your breaker (or a service mesh sidecar) trips to OPEN --
# response times should drop sharply once it's failing fast instead
# of waiting out full timeouts on every call
for i in $(seq 1 20); do
  curl -s -o /dev/null -w "attempt $i: %{http_code} in %{time_total}s\n" https://httpstat.us/500
done

# If running behind Envoy/a service mesh, inspect circuit breaker
# stats directly (outlier detection / ejection state)
curl -s localhost:9901/clusters | grep -i "circuit_breakers\|outlier"

# Load-test a flaky dependency to watch open/half-open/closed
# transitions happen live under sustained failure + recovery
hey -z 30s -c 20 https://httpstat.us/500

Interview Q&A

Q: Why do you need a circuit breaker if you already have timeouts and retries?

A: Timeouts and retries are per-call decisions — they bound and possibly retry one request, but they don't track the dependency's health over time. If B is fully down, every single request from A still pays the full timeout (and possibly several retries with backoff) before failing, and under load that's thousands of callers simultaneously wasting threads/connections on calls that were essentially guaranteed to fail. A circuit breaker sits above that: it watches the aggregate failure rate, and once it's clear B is unhealthy, it makes calls fail fast immediately — no wasted timeout, no wasted retry — until B has had a chance to recover.

Q: Follow-up — walk me through what happens end-to-end when B goes down and then recovers.

A: While B is healthy, the breaker is closed and requests flow normally, with failures counted against a rolling-window threshold. Once B starts failing consistently and the error rate crosses that threshold (say, 50% errors in the last 20 calls), the breaker trips to open — for the cooldown period, calls to B are skipped entirely and A either fails fast or serves a fallback. After the cooldown elapses, the breaker moves to half-open and lets a small number of probe requests actually reach B. If those probes succeed, the breaker closes and full traffic resumes; if even one fails, it goes straight back to open and the cooldown timer resets, so A doesn't slam a still-recovering B with full traffic prematurely.

Q: Follow-up — what's a risk of setting the failure threshold or cooldown badly?

A: Too sensitive a threshold (tripping on a couple of normal blips) makes the breaker "flap" — opening and closing repeatedly on ordinary noise, which itself degrades availability for no real benefit, since some of those failed requests would have succeeded if just retried normally. Too lax a threshold, or too short a cooldown, means the breaker doesn't actually protect the caller in time, or reopens too aggressively and slams a still-fragile B with a full-traffic burst the moment it looks slightly healthy again — both need per-dependency tuning based on observed real failure patterns, not one-size-fits-all defaults.

Q: What should happen to a request when the breaker is open — just error out?

A: Ideally no — pair the open state with a sensible fallback: a cached/stale response, a degraded feature, or a default value, rather than surfacing a raw error to the end user whenever possible. Whether a fallback is viable depends entirely on the dependency — for a recommendations service, a generic/cached fallback is fine; for a payments service, failing loudly and explicitly is usually the safer choice than silently degrading.

System Design Tie-in

This is the standard answer to "how do you prevent cascading failures in a microservices architecture" — interviewers expect the combination of timeouts (bound each call), retries with backoff+jitter (absorb transient blips safely), and a circuit breaker (stop hammering a dependency that's aggregately unhealthy) presented as layered defenses, not any single one in isolation. In an API gateway or service-mesh design specifically, breakers are often mentioned as infrastructure-level concerns (Envoy's outlier detection, a mesh's circuit-breaking config) rather than something every service has to implement itself. It's also a natural place to discuss graceful degradation — what fallback behavior a system offers when a non-critical dependency is circuit-broken open, versus a critical one where failing loudly is the right call.

Trade-offs

A circuit breaker prevents pile-up on a failing dependency and gives it room to recover, but it also means legitimate requests fail fast during the open window even if the dependency could have handled some of them. Interview gotcha: threshold and cooldown values need tuning per dependency — too sensitive and the breaker flaps on normal blips; too lax and it doesn't protect the caller in time. Pair it with a sensible fallback (cached response, degraded feature) rather than just surfacing a raw error to the end user.