There is a failure mode I see constantly in agent code: a provider starts erroring, and the agent responds by calling it harder. Retry loops, exponential backoff that resets on every new task, five parallel agent runs all independently rediscovering that the same endpoint is down. The provider is having a bad day, and your agent fleet is making it worse while burning latency budget on requests that were never going to succeed.
The fix is an old one. Circuit breakers have been standard practice in distributed systems since Michael Nygard wrote them up in Release It! two decades ago. The electrical metaphor is exact: when the load looks dangerous, the breaker trips, and current stops flowing until someone (or something) decides it is safe again. Agent tool calls need the same protection, and almost nobody building agents adds it, because it feels like infrastructure work when you just want your agent to search the web.
The pattern in one paragraph
A circuit breaker sits between your code and a dependency and tracks outcomes. In the closed state, calls flow through normally while the breaker counts failures. When failures cross a threshold, the breaker opens: calls to that dependency fail immediately, without touching the network. After a cooldown, the breaker goes half-open and lets a trial request through. Success closes the circuit; failure opens it again. That is the whole pattern. The interesting decisions are all in the thresholds.
Count failures as a rate, not a streak
The naive implementation trips after N consecutive failures. This breaks in both directions. A provider running at a 40% error rate will regularly sneak in a success that resets your counter, so the breaker never trips even though nearly half your calls are dying. Meanwhile a burst of three timeouts during a routine blip trips a consecutive-count breaker that had no business opening.
Rate over a window is the better signal. In route.tools, the circuit breaker skips a provider when its error rate exceeds 30% over the last 5 minutes. Both numbers matter. The 30% threshold tolerates the background noise every API has (transient 500s, the occasional timeout) while catching genuine degradation early. The 5-minute window means the decision is based on recent reality, not on an incident from this morning, and it also means recovery is automatic: once the bad period ages out of the window, the provider is back in rotation without any half-open ceremony.
If you build this yourself, resist the urge to make the window long. A 1-hour window turns your breaker into a grudge holder. Providers recover fast; your memory of their failure should too.
The rule almost everyone gets wrong
Here is the subtlety that separates a useful breaker from a harmful one: only skip a provider if a healthy alternative exists.
A textbook circuit breaker protects the dependency and fails your call fast. That is correct when the caller has a fallback, and actively wrong when it does not. If every scraping provider in your chain is above the error threshold, a strict breaker returns instant failures for all of them, and your agent gets nothing. But a provider at a 35% error rate still succeeds 65% of the time. Degraded is not dead. When the alternative to a flaky provider is no provider at all, you want the flaky one.
So the breaker should be a routing preference, not a hard gate. Our router implements it exactly this way: a tripped provider drops to the back of the candidate list rather than out of it. If Firecrawl and Spider are healthy, a struggling Jina gets skipped entirely. If everything in the category is struggling, the router still tries the least-bad option instead of manufacturing a guaranteed failure. This is the kind of rule that sounds obvious once stated and is absent from nearly every DIY wrapper I have read, including the first version of mine.
Breakers and failover are two halves of one system
A circuit breaker decides who not to call. Failover decides who to call next. You need both, and they share state.
Failover without a breaker is wasteful: every single request re-attempts the dead provider first, eats a timeout, then falls through. With per-attempt timeouts of a few seconds, that is seconds of pure tax on every call during an incident. A breaker moves that discovery cost from every request to roughly one request per window. In our setup, failover tries up to 3 providers per request, but the breaker reorders the list first so the first attempt is usually the one that works. The whole dance runs inside a per-request wall-clock budget (20 seconds for search, longer for slow categories), because an agent waiting forever is its own kind of outage. I wrote more about the failover half in why AI agents need failover.
Make the breaker visible
The worst thing a circuit breaker can be is silent. If provider selection changes based on hidden health state, you will eventually stare at logs wondering why yesterday’s traffic went somewhere unexpected. Every response from our router includes the full routing.attempted chain, so when the breaker skips a provider, you see the skip and the reason rather than inferring it from an anomaly in your bill.
If you are wiring up your own: track outcomes per provider, use a rate over a short window, prefer skipping to blocking, and log every decision. It is maybe a day of work to do properly, and it is the difference between an agent that degrades gracefully and one that spends a provider outage repeatedly slamming a door that will not open.
If you would rather have the breaker, the failover, and the receipts without building them, compare what the router covers on the search page and work backwards from there.