Skip to content
Reliability & Integration

intermediate · Interactive lab

Circuit Breaker

A retry asks whether to send this request again. A breaker asks whether anyone should be calling that dependency right now. Learn what its state means and how it gets back to normal.

By the end: Tell a breaker apart from a retry: one asks whether to send this request again, the other whether anyone should be calling that dependency at all.

Your challenge

Start here. This lab opens with the usual reflex, and it is wrong: every request is simply sent, whatever the breaker has counted. Some of those calls land on an API that is already failing, and one of them restores full traffic on no evidence. Read each moment, choose, and test.

See it happen

Watch the breaker decide

Breaker: closed
Slowed down so you can follow
REQUESTSWHEN ALLOWEDIntegration LogicCIRCUIT BREAKERCLOSEDWarehouse API● WELL
Order Processing sends a request to Integration Logic, which calls the Warehouse API through a circuit breaker. The breaker counts what recent calls did and decides whether the next request goes out, is answered at once as a failure, or is held.
  1. Order Processing → Integration Logic
  2. Integration Logic → Warehouse API: calls, when the breaker allows
STEP 1 / 6

A breaker in front of one dependency

Integration Logic does not call the Warehouse API directly: every call goes through a circuit breaker. The breaker remembers what recent calls did. Watch its state, and what it allows.

Configure

Your breaker decisions

For each request, choose what should happen to it. Read the breaker's own state first: what it has counted, and what its policy allows right now.

What should happen to this request?
Send it now
· Let it through to the Warehouse API as usual.
Hold it
· Keep it until the cool-down ends, then send it.
Answer now: unavailable
· Tell the caller at once, and send nothing.
Send one probe
· One trial request, to find out whether the API is well.
Allow full traffic
· Close the breaker and call normally again.
  1. REQ-801 — Nothing failing

    The breaker is out of the way; the previous call was served.

    Breaker
    CLOSED · 0 of 3 failures in a row
    Unknown
    Nothing: the last call was served
    Policy allows
    Calls go out as usual
    Who is waiting
    A customer at checkout
  2. REQ-802 — Two failures counted

    Two calls in a row have failed. The policy opens the breaker at three, and this request is the next call.

    Breaker
    CLOSED · 2 of 3 failures in a row
    Unknown
    Whether the Warehouse API can serve a call now
    Policy allows
    Calls go out as usual
    Who is waiting
    A customer at checkout
  3. REQ-803 — Just opened, customer waiting

    The breaker opened half a second ago. A failed call here would cost the customer a 3 s wait before failing anyway.

    Breaker
    OPEN · 0.5 s of a 5 s cool-down
    Unknown
    Whether the Warehouse API can serve a call now
    Policy allows
    Nothing goes out for another 4.5 s
    Who is waiting
    A customer at checkout
  4. REQ-804 — Cooling down, nobody waiting

    Reads like REQ-803 as far as the breaker is concerned. The difference is not in the breaker: this one is part of the nightly batch.

    Breaker
    OPEN · 1.2 s of a 5 s cool-down
    Unknown
    Whether the Warehouse API can serve a call now
    Policy allows
    Nothing goes out for another 3.8 s
    Who is waiting
    Nightly batch: nobody
  5. REQ-805 — Cool-down finished

    The 5 s are up. Nothing has touched the Warehouse API since the breaker opened, so nothing knows what a call would meet now.

    Breaker
    OPEN · 5.2 s of a 5 s cool-down
    Unknown
    Whether the Warehouse API can serve a call now
    Policy allows
    One probe is due
    Who is waiting
    A customer at checkout
  6. REQ-806 — One probe back

    A probe came back a moment ago, which is why the breaker is half-open. The policy asks for two before full traffic.

    Breaker
    HALF-OPEN · 1 of 2 probes came back
    Unknown
    Whether the Warehouse API can serve a call now
    Policy allows
    1 more probe before full traffic
    Who is waiting
    A customer at checkout
  7. REQ-807 — Both probes back

    Both probes the policy asks for have come back. Leaving the breaker half-open now would keep turning customers away for nothing.

    Breaker
    HALF-OPEN · 2 of 2 probes came back
    Unknown
    Whether the Warehouse API can serve a call now
    Policy allows
    Enough probes came back to allow full traffic
    Who is waiting
    A customer at checkout

Two moments can look similar and still need different answers: it depends on what the breaker has counted and who is waiting.

Learn more

Why this pattern exists

Integration Logic calls the Warehouse API through a circuit breaker. The breaker does not look at the request in hand; it looks at what recent calls did. While it is CLOSED, calls go out as usual. When enough of them fail in a row, it goes OPEN and stops sending: the caller is answered at once instead of waiting for a timeout, and the Warehouse API is left alone to recover. After a cool-down the breaker allows one trial request — a probe — and that is the HALF-OPEN state: not recovered, only testing. Probes that come back let it close again; a probe that fails opens it once more and the cool-down starts over.

Order Processing sends requests through Integration Logic to the Warehouse API, with a circuit breaker between them. Each graded case is one moment: what the breaker had counted when a request arrived, and who was waiting for it. The breaker's policy is fixed here — three failures in a row open it, a 5 s cool-down, two probes back before full traffic.

  • Tell a breaker apart from a retry: one asks whether to send this request again, the other whether anyone should be calling that dependency at all.
  • Read a breaker's own state — failures counted, cool-down elapsed, probes returned — and say what it allows right now.
  • Choose what happens to a request in each state without hammering a dependency that is down, restoring full traffic on one lucky call, or leaving a breaker open for ever.

The rule this lesson applies: Repeatedly calling a dependency that is failing costs twice: every caller waits for its own timeout, and the load keeps the dependency from recovering. A breaker converts that slow failure into a fast one and takes the load away. The price is that the caller stops learning: while the breaker is open, nothing knows whether the Warehouse API is well again. That is what the cool-down and the probe are for — a small, deliberate amount of traffic to find out. One probe that succeeds is evidence, not proof: this lesson's policy asks for two before full traffic returns, because a dependency that has just come up can fall over again under the whole load. Retry and breaker work together: retries handle a request that may still succeed; the breaker decides whether those retries should be leaving at all. A retry policy with no breaker in front of it is how a slow dependency turns into a queue of waiting callers.