Model selection
Fit the task.
Compare eligible models using task-level quality, cost, latency, and reliability. Polish document extraction, coding, and tool use need different evidence—not one universal leaderboard.
Fabryka AI / Router
The right model
for the work.
Not every request needs the largest model. Not every cheap response gets the job done. Routing should choose the system that can complete the task—within the constraints that matter.
Start with Auto Router through our OpenAI-compatible API. Our longer-term direction is an execution layer for agents: choose the model, choose where it runs, and measure whether it worked.
Model selection is the starting point.
Reliable execution is the goal.
01 / Available today
One API.
Let the router choose.
Use model: "auto" with the existing chat completions endpoint. Auto Router selects a model using tool requirements, language, and context size. Responses report the concrete model used. You can also pin a published model ID when you need a specific model.
Current API · Python / OpenAI SDK
import os
from openai import OpenAI
client = OpenAI(
base_url="https://fabryka.ai/v1",
api_key=os.environ["FABRYKA_API_KEY"],
)
response = client.chat.completions.create(
model="auto",
messages=[{
"role": "user",
"content": "Wyjaśnij krótko, czym jest RAG.",
}],
max_tokens=256,
)
print(response.model)
print(response.choices[0].message.content)Set your API key in FABRYKA_API_KEY and install the OpenAI Python SDK. For log probabilities, select Qwen or Bielik explicitly rather than Auto Router. See the API documentation and logprobs guide.
The sections below describe the product direction, not additional API parameters available today. Named routing profiles, residency policies, task hints, and decision-inspection endpoints are proposals—not the current API contract.
02 / Planned architecture
Constraints first.
Intelligence second.
A model should never win a ranking if it cannot satisfy the request’s requirements. Our intended architecture filters eligible systems before optimizing quality, cost, latency, and reliability.
- 01FilterPolicy & capabilities
- 02SelectTask & model quality
- 03PlaceProvider & availability
- 04MeasureExecution & outcome
- Data boundaries. Allowed locations and providers, including customer-controlled infrastructure.
- Required capabilities. Tool calling, structured output, modality, and sufficient input-and-output context.
- Execution limits. Cost budgets, latency targets, and eligible fallback destinations.
A policy must remain a policy when something fails. If no eligible system can complete the request, the intended behavior is an explicit failure—not silently relaxing data boundaries or dropping required capabilities.
Regional enforcement and customer-infrastructure routing are planned capabilities. This page is not a Poland-only or EU-only processing guarantee.
03 / Two different decisions
Which model?
Then, where should it run?
Provider selection
Fit the execution.
Then choose an eligible endpoint using availability, queue depth, time to first token, throughput, region, and price. The same model can behave differently on different infrastructure.
The objective is not “the cheapest model.” It is the lowest total cost of completing the task at the required quality, including failed attempts, validation, and retries.
04 / Agent-oriented routing · direction
Route the step.
Not just the conversation.
An agent may plan, search, classify, extract, call a tool, and synthesize an answer. Those are different jobs. A specialist may handle repeatable extraction while a stronger general model handles planning and difficult reasoning.
- Plan
Choose a model with evidence for reasoning and tool use.
- Classify & extract
Use a smaller specialist only when it meets the workload’s quality threshold.
- Validate
Check schema, grounding, or tool outcome where the task permits it.
- Synthesize
Escalate when the evidence calls for a more capable model.
We want agents to supply explicit task hints instead of forcing the router to guess. Conversation continuity should be preserved unless a step genuinely benefits from another model.
This is an execution design, not an automatic multi-step agent service or a promise of specific model-size savings.
05 / Execution-aware routing · direction
Keep useful context.
Plan for failure.
A worker with a warm prefix cache may be a better destination than an idle worker that must process the same long context again. Session affinity and cache awareness belong alongside model quality—not outside the routing decision.
We want each decision to produce an execution plan: a primary destination and eligible fallbacks. Timeouts, unavailable workers, or failed output validation need clear handling without abandoning the original policy.
Retries are not free, and tool calls are not always safe to repeat. Execution design must account for latency budgets, partially delivered responses, and duplicate side effects—not simply send the request again.
06 / Evidence, not intuition
A registry that knows
what models can do.
The direction is a machine-readable model registry combining supported features, context limits, deployment metadata, and measured task-level performance. Quality measurements should have a workload, a method, and a date—not an invented capability score.
CodeSOTA is the evidence source we want to connect to routing. Independent evaluations can inform initial selection; privacy-respecting operational measurements can show how systems perform under real load.
Latency alone does not establish quality. A valid schema does not prove a correct answer. Outcome evaluation must match the task, and customer content must not become research or training data without explicit permission and defined handling rules.
The loop we want: evaluate → register → route → measure → evaluate again.
07 / Accountable decisions · direction
Know why it was chosen.
A useful routing record should explain which requirements were enforced, which model and provider were selected, what alternatives were eligible, and whether a fallback occurred.
That record should connect to observed execution cost, latency, and task outcomes—without retaining customer prompts by default. It should help teams debug decisions and compare policies, not disguise estimates as measured facts.
The current API reports the concrete model used. A full routing-decision record and inspection endpoint are planned extensions.
08 / What exists. What comes next.
Earn the intelligent part.
Auto Router.
OpenAI-compatible chat completions with model: "auto", model selection using request requirements, and a concrete model in the response. Explicit model selection remains available.
Policy & evidence.
Explicit constraints, task-level evaluation inputs, separate provider selection, policy-preserving fallback plans, and inspectable decisions. Start with deterministic rules and measurable outcomes.
Learn & escalate.
Learned task and difficulty prediction, calibrated success estimates, and specialist-to-generalist cascades. Advance only when workload evaluations show a benefit over the simpler baseline.
No unmeasured savings claims. No release dates implied. Proposed profiles such as fast, best, agent, or regional policies are not currently supported model IDs.
Part of one factory
Less model management.
More completed work.
Research improves the options. Evaluations establish the evidence. Routing connects the task to execution. Inference makes it useful.