Filter first, rank second
A routing system should first remove candidates that cannot complete the job. Required capabilities may include tool calling, structured output, image input, long context, regional processing, a specific safety policy, or an enterprise provider agreement.
Only the eligible set should be ranked by economics. This avoids the common mistake of identifying a cheap model that is technically or operationally incompatible with the workload.
The decision inputs
- Task class and complexity.
- Required modalities, tools, output format, and context capacity.
- Observed quality or evaluation performance for similar requests.
- Input, output, cached, and reasoning-token economics.
- Latency, error rate, provider health, and regional availability.
- Data-handling, contractual, and organizational policies.
- Fallback order, retry budget, and emergency-stop conditions.
Provider routing and model routing are different
Provider routing chooses among providers serving the same or equivalent model. OpenRouter documents provider ordering by price, throughput, latency, and related settings. Model routing chooses among different models that may have materially different behavior and capabilities.
The second problem requires stronger evaluation evidence because a change in model can alter the output, not merely the infrastructure path.
Use request classes
A production application rarely has one universal quality requirement. A short intent classification, a legal-document analysis, and an agent deciding whether to execute a tool should not share the same routing policy.
Grouping requests by job allows teams to use smaller models for repeatable lower-risk work while protecting premium or high-risk paths. The groups also create cleaner baselines for measuring cost and quality after a routing change.
Controlled activation
A new route should normally begin with shadow analysis or a limited canary. The rollout should define maximum traffic, duration, quality and error thresholds, and the condition that returns traffic to the previous path.
Token Pilot's routing direction is designed to combine reviewed provider economics and compatibility evidence with approvals and verified post-change results rather than make an unreviewed cheapest-price switch.
Official sources and further reading
Vendor capabilities change. These links lead to the official product or documentation pages used as technical references.