“Routing cuts your costs” is the kind of claim every infrastructure product makes, so let me be specific about it, since I run a tool router and the margin we charge is public. The savings are real, but they come from exactly two mechanisms, and there are workloads where routing will not save you a cent. Knowing which bucket you are in is worth more than any sales pitch.
Mechanism one: same product, different price
The cleanest savings come from categories where multiple providers sell literally the same artifact. Image generation is the textbook case. FLUX-dev is an open-weights model, and three hosts in our catalog serve it at three prices: DeepInfra at $0.0108 per image routed, fal.ai at $0.03, and Replicate at $0.036. Same weights, same class of output, a 3.3x spread. I wrote up the full picture in the FLUX host comparison, but the short version is that this is pure arbitrage. Routing cheapest-first here is close to free money, with the expensive hosts held in reserve as failover.
Transcription looks similar. Whisper large-v3-turbo through Groq costs $0.00084 per audio minute routed, roughly six times less than Deepgram Nova-3 at $0.00516. That one is not a perfectly identical product (Deepgram bundles diarization and formatting), but for transcripts that feed an LLM rather than a human, the outputs are interchangeable and the multiplier is real.
Mechanism two: matching queries to tiers
The second mechanism is subtler: most categories have quality tiers, and the expensive tier is only worth paying for on the requests that need it. Web search spans $1.20 per thousand queries (Serper, a Google SERP API) to $8.40 (Exa, neural search with page content included). A navigational query like “stripe webhook docs” gets nothing from Exa that Serper would not return at a seventh of the price. A conceptual research query genuinely does.
If your agent sends everything to the premium tier, you are overpaying on the easy majority of queries. If it sends everything to the cheap tier, you eat retries and bad results on the hard minority. The savings come from splitting the traffic, and that is a routing policy, not a provider choice. Saved routing preferences exist precisely so this split lives in configuration instead of scattered through your codebase.
There is also a quieter cost saving hiding in failover: a failed tool call in the middle of a long agent run does not just cost the retry, it can waste the whole run, including the model tokens you already spent. I cannot put a universal number on that, since it depends entirely on your agent, but in my experience it is where teams feel the pain first.
Where routing does not save you money
Now the honest part. Our routed prices are provider list plus 20%, published at /docs/pricing. That means if you use exactly one provider, know it well, and are happy with its reliability, going direct is cheaper than routing through us, full stop. The 20% buys you failover across up to three providers, one bill, one key, and per-call price transparency, but if none of that has value for your workload, do not pay for it.
The margin math also bounds the arbitrage claim. Routing from Replicate-priced FLUX to DeepInfra-priced FLUX saves you 70% even after our markup, so that trade clearly survives the fee. Routing between two providers priced within 20% of each other does not; the margin eats the spread. Same-price categories are the extreme case: E2B and Daytona both cost $0.0000168 per vCPU-second routed, so routing between them is purely about availability, not cost.
Two more places the savings are smaller than the pitch: single-category users with negotiated rates (if you have a volume discount with a provider, list-plus-20% is a step backwards, which is what BYOK at a 5% fee exists for), and categories where quality dominates so hard that nobody sane routes on price, like embedding models, where switching providers means re-embedding your whole corpus.
The back-of-envelope test
Here is the worksheet version. For each tool category your agent uses, ask three questions. First, is there a spread: what is the ratio between the priciest and cheapest provider you would accept output from? Second, what share of your requests actually need the top tier? Third, what does a failed run cost you in wasted model tokens and user trust?
If the spread is above roughly 1.5x and the premium-need share is under half your traffic, tiered routing pays for itself quickly. If you are in a flat-priced or single-viable-provider category, routing is an availability tool for you, not a cost tool, and you should evaluate it as one.
The per-category spreads are all visible side by side on our comparison pages, generated from the open catalog the router itself reads. The image generation comparison is the most dramatic place to start.