When to use decision models instead of language models?

Using a language model for every quick classification or scoring call is slow and expensive. Decision models now classify, score, or pick between choices at about a seventieth of the per-request cost, and 26% of gateway requests already route to them.

What it is

Decision models are a third option between fixed rules (cheap but rigid) and language models (smart but slow and costly). They arrived on AI Gateway in mid-September 2026; there are now 15 decision models from eight vendors.

Applied case

Teams routing over 10 billion tokens a month have adopted them heavily: seven in ten of those teams use a decision model, and a third of all paid teams used one in October. In six weeks they grew to one in four gateway requests while costing only half a cent of every dollar spent.

• Route classification, scoring, and choice-picking to decision models instead of LLMs

When to use it

Use when a request is pure judgment: classify, score, or pick between options, and you want low cost per call.
Skip when you need generated text, nuanced reasoning, or freeform output a classifier cannot express.

Read more

AI Gateway Production Index, November 2026