Two years ago “agent infrastructure” meant a model API key and a while loop. In 2026 it is a legible stack with distinct layers, real vendors at each one, and enough overlap between them that people regularly buy two products that do the same thing or assume one layer covers a job it does not. Since I run a router that sits in the middle of this stack, I spend a lot of time explaining where the seams are. This is the map I wish someone had handed me.
Layer 1: models
The bottom of the stack: the LLM APIs that do the reasoning. OpenAI, Anthropic, Google, plus the open-weight hosts serving Llama-class models. This layer decides how smart your agent is, and it takes the majority of most agent budgets. Everything above exists to feed it context and act on its decisions.
Layer 2: model gateways
Because model APIs proliferated, a routing layer grew on top of them. OpenRouter is the canonical example: one key, one schema, many models, with failover and price arbitrage across hosts. This layer answers “which LLM serves this request”, and by 2026 it is mature enough that most teams treat it as boring plumbing, which is the highest compliment infrastructure gets.
The thing to notice: gateways solved routing for model calls. An agent’s tool calls (the searches, scrapes, transcriptions that ground it in reality) historically got nothing equivalent, which is the gap the next layer’s routing story fills.
Layer 3: tools
This is where agents touch the world, and it is the most fragmented layer in the stack: dozens of vendors across categories that barely resemble each other. Our catalog alone routes about 30 providers across 9 categories: search, scraping, document parsing, image generation, video generation, text-to-speech, transcription, embeddings, and code sandboxes.
The fragmentation is structural, not a temporary immaturity. Web search and video generation are simply different businesses; no vendor will ever consolidate the layer the way the big labs consolidated models. What can consolidate is the interface: one schema per category, one key, failover across providers within a category. That is tool routing, it is the layer route.tools occupies, and I made the longer argument for it in what is a tool router. The honest framing is that a tool router is to layer 3 what OpenRouter is to layer 1, with the caveat that tools are less commodity-like than models, so normalization has to be done more carefully.
Within this layer, prices span wild ranges even inside a category: search runs $1.20 to $8.40 per 1k queries routed, scraping $0.36 to $1.92 per 1k pages. Category-by-category comparisons live on pages like the search comparison; the point for the stack view is just that layer 3 is where cost structure is least uniform and multi-vendor strategy pays off most.
Layer 4: sandboxes
Once agents write code, they need somewhere safe to run it, and letting an agent execute generated code on your own infrastructure is a decision you make once. Sandbox vendors sell isolated, disposable execution environments billed by compute time. In our catalog, E2B and Daytona both price at $0.0000168 per vCPU-second routed, identical list prices with different product tradeoffs, which I compared in E2B vs Daytona. Modal is catalogued and coming soon on our side, with GPU workloads as its distinguishing pull.
Sandboxes are technically tools (we route them as our ninth category), but they deserve their own layer in the mental model because their failure mode is different: a bad search result wastes tokens, a bad sandbox is a security incident.
Layer 5: memory
The persistence layer: vector stores, embedding models, and increasingly structured agent memory products on top of both. Embeddings are the part I route (three providers, $0.012 to $0.072 per 1M tokens), and the critical property of this layer is stickiness. Vectors from different embedding models do not mix, so memory is the one layer where you cannot casually swap vendors; a migration means re-embedding the corpus. Choose slowly here, even as you stay flexible everywhere else.
The connective tissue: MCP
Cutting across all of this is the Model Context Protocol, which by 2026 has become the default way to hand tools to agents without writing bespoke glue per client. The stack-level significance: MCP standardizes the wiring between layers 1 and 3, while routing standardizes the selection within layer 3. They compose rather than compete. One remote MCP server can expose an entire routed tool layer (ours exposes 9 tools, one per category, per the MCP docs), so an agent connects once and gets search through sandboxes with failover underneath.
Where the seams actually are
The recurring confusions I see: teams assuming their model gateway routes tool calls (it does not, different layer); teams treating sandboxes as an afterthought tool rather than a security boundary; and teams hard-wiring layer 3 vendors directly into agent code, recreating the exact lock-in that gateways taught everyone to avoid at layer 1.
The stack is settling, but the middle layers are where 2026’s interesting build-out is happening. If you want to see what a consolidated layer 3 looks like in numbers, start with the search API comparison and browse across categories from there.