Choosing a foundation model in 2026 is not a religion. It's an engineering decision with cost numbers, latency numbers, and quality numbers. Here is how we pick — and how often we change our minds.
The four lanes
- Frontier closed-weight (the ones you talk about by company name). Best raw quality, best for hardest tasks, most expensive, slowest, least flexible.
- Mid-tier closed-weight. Often 70–90% of frontier quality, 5–10× cheaper, 3× faster. The default for most production paths.
- Open-weight, hosted. You pay for inference, not for the weights. Good for control over data residency and for cases where you want to fine-tune.
- Open-weight, self-hosted. You own the GPU bill. Cheapest at scale (above ~10M tokens/day). Most operational overhead.
How we pick
For each task, we run a benchmark on our internal eval set against three or four candidates. We measure quality, latency, and cost. We pick the cheapest model that hits 95% of the best model's quality on our eval. Surprisingly often, that is not the frontier model.
Cost numbers from real workloads
| Workload | Frontier | Mid-tier | Self-hosted 7B |
|---|---|---|---|
| Customer-service classify | $0.0042/req | $0.0008/req | $0.0001/req |
| Summarization (1k tokens) | $0.012/req | $0.0021/req | $0.0008/req |
| Long-form generation | $0.18/req | $0.043/req | not viable |
| RAG answer with citations | $0.038/req | $0.009/req | $0.003/req |
When to use the frontier
Three cases. First: agent planning. Plans are short and high-leverage; pay for quality here. Second: anything legal, medical, or regulatory in tone, where a small mistake is catastrophic. Third: very long context with reasoning across the whole context — the frontier still has a lead.
When to use the mid-tier
Most things. Customer service, summarization, retrieval-augmented Q&A, classification, extraction. The mid-tier in 2026 is what the frontier was in 2024. Don't pay frontier prices for mid-tier work.
When to self-host
Three conditions, all of them: (a) your data residency requires it, (b) your volume is north of ~10M tokens/day so the GPU bill amortizes, (c) you have an SRE-trained engineer who wants to own the deployment. If any one of these is missing, hosted is cheaper all-in.
Don't fall for the leaderboard
Public leaderboards measure what the leaderboards measure. They do not measure your task. We have repeatedly found that the model that scored 4 points lower on a public benchmark scored 9 points higher on our internal eval. The leaderboard is a starting point. Your eval is the answer.