Two years ago, the gap between open-weight models and closed APIs was significant. If you wanted top-tier reasoning, instruction following, and tool use, you called GPT-4 and paid the token tax. Open alternatives existed, but they required compromises that most enterprises couldn’t accept.
That gap has closed. In many practical enterprise use cases, it has disappeared entirely.
What changed
Three things happened in rapid succession. First, Meta released Llama 3 and its successors, establishing that a well-resourced lab could produce open-weight models competitive with the best closed alternatives. Second, Mistral demonstrated that a European company could train frontier-class models and release them openly. Third, the Qwen family from Alibaba proved this wasn’t limited to Western labs — high-quality open models are now a global phenomenon.
The result in 2026: enterprises have access to multiple model families in the 7B to 400B+ parameter range that perform at or near the level of closed APIs on tasks that matter for business — summarisation, structured data extraction, code generation, reasoning over documents, and tool calling.
Benchmarks tell part of the story. On standard evaluations like MMLU, HumanEval, and GSM8K, the latest open-weight models trade places with closed APIs depending on the task. But benchmarks aren’t what convinced us. What convinced us was running these models on real enterprise workloads — contract analysis, invoice processing, customer support triage — and seeing comparable output quality at a fraction of the per-query cost.
The practical advantages
Your data stays on your infrastructure. This is the big one. When you self-host a model, prompts and completions never leave your network. No data processing agreements with third parties, no cross-border transfer concerns, no wondering what happens to your data after the API call. For regulated industries — banking, healthcare, insurance, government — this alone justifies the switch.
Fine-tuning for your domain. Open weights mean you can fine-tune. A general-purpose 70B model that’s been fine-tuned on your company’s internal documents, terminology, and processes will outperform a much larger closed model that has no context about your business. Fine-tuning is where open models go from “comparable” to “better” for specific enterprise use cases.
Cost predictability. Token-based API pricing scales linearly with usage, and usage is hard to predict when you’re deploying agents that reason in loops. Self-hosted models run on infrastructure with fixed costs. You pay for GPU compute whether you send one query or one million. For high-throughput workloads, the economics are dramatically better.
No rate limits, no outages you don’t control. When you run the model, you control the uptime. You control the throughput. You’re not sharing capacity with every other customer on the planet. For production agent systems that need to be reliable, this matters.
No surprise model changes. Closed API providers update their models, sometimes without notice. Behaviour changes. Outputs shift. Agent workflows that depended on specific model behaviour break. When you run open weights, the model changes when you decide it changes.
The trade-offs
We’re not going to pretend there are none.
Operational overhead. Running your own inference infrastructure requires GPUs, serving frameworks (vLLM, TGI, or similar), monitoring, and a team that understands how to keep it healthy. This is not trivial. If you don’t have the in-house expertise, you need a partner who does.
Cutting-edge capabilities. For some tasks — particularly complex agentic reasoning chains that require the absolute frontier of model capability — the very latest closed models may still have an edge. That edge shrinks with every release cycle, but it exists today for the most demanding use cases.
Context window constraints. Some open models have smaller context windows than the largest closed alternatives. This matters for workloads that require processing very long documents in a single pass. Techniques like RAG and chunking mitigate this, but they add complexity.
Why this matters for agent systems
AI agents are more demanding than simple prompt-response use cases. An agent might make dozens or hundreds of LLM calls to complete a single task — reasoning, planning, calling tools, evaluating results, and iterating. This has specific implications.
Cost sensitivity. An agent that makes 50 LLM calls per task at $0.01 per call costs $0.50 per execution. At scale, this adds up fast. Self-hosted models make high-frequency agent interactions economically viable.
Latency control. Agents need fast inference to be responsive. When you control the serving infrastructure, you can optimise for latency — batching strategies, model quantisation, hardware selection — in ways that API endpoints don’t expose.
Reliability. An agent that fails mid-task because of an API timeout or rate limit creates a poor user experience and potentially inconsistent state. Owning the inference layer eliminates this class of failures.
Customisation. Agents benefit enormously from fine-tuning. A model that’s been trained to use your specific tools, follow your specific workflows, and produce outputs in your specific format will produce better agent behaviour than a generic model prompted to do the same.
The shift to open-weight models isn’t coming. It’s here. For enterprises building production agent systems, the question is no longer “are these models good enough?” It’s “do we have the operational capability to run them?” And if not, finding a partner who does.