Quality scores, honestly

sort:"quality" ranks providers by a hand-curated 0–100 score per provider, published in the open-source catalog next to each price. This page states plainly what those scores are and are not.

What the scores are

A static, per-provider judgment maintained by us, based on: published benchmarks and provider documentation, output-format fidelity in our own testing, and community consensus where it exists (e.g. premium document parsers demonstrably outperform fast OCR tiers on complex layouts). Scores change via pull request, so every change is a public diff.

What the scores are not

• Not query-dependent. A static score can’t know that a semantic research query favors Exa while a navigational lookup favors a SERP provider. If you know your workload, pin providers with routing.providers or a saved priority order — that will beat any generic score.
• Not a benchmark result — yet. A reproducible, published benchmark suite (fixed query/document/audio sets, scored outputs, run on a schedule) is the plan of record; until it exists, treat quality as an editorial ranking, not a measurement.
• Not a claim that providers are interchangeable. Within a tier, results are close and price routing is safe. Across tiers they are different products. That’s exactly why the response envelope names the provider that served every call, and why include_raw: true returns the provider’s unmodified payload alongside the normalized result.

What IS measured live

Latency (rolling p50/p95) and error rate per provider, over a 5-minute window, published on the status page with 30-day uptime history — the same numbers the router’s sort:"latency" and circuit breaker use.