The Cheapest Inference on the Market Is a Classifier, Not a Chat Model

Every AI bill has a quiet line item, the requests that never needed a frontier model. A classification, a routing decision, a content flag, a short extraction. Teams pay chat-model prices for work a small model could do, because the routing logic is harder to build than a prompt.

Microsoft now sells that small model directly. Microsoft-Decision-1 went live on Friday through Microsoft Foundry, with OpenRouter to follow. It is a decision model, which means it returns a constrained, structured answer rather than free text. Microsoft pitches it for tasks where a wrong paragraph is worse than a wrong label.

The category is not Microsoft’s invention. TypeSafe AI shipped Jev in September and framed it as a model for machines, rated by probability rather than prose. Cloudflare, OpenAI, Liquid AI, Snowflake and Perplexity followed with their own. A public tracker now lists more than 100 decision models, which is what happens to a category that is cheap to enter.

The price is the product

Microsoft’s own numbers make the intent plain. Input tokens cost $0.042 per million and the output is free, because the answer is a handful of tokens. The company claims the model is 2.5 times faster than H2O-Lightning-4B and 2.8 times faster than Jev, and more than 20 times cheaper than GPT-6 Sol for text classification. Those are vendor measurements on vendor tests, and they should be read that way. The direction is what matters.

The economics are simple once the output is a label. A chat model bills for every word it produces, and a long reasoning trace costs real money. A decision model produces a fixed, tiny output, so the cost per decision collapses. The saving is not really in the model. It is in moving the easy half of your traffic onto a cheaper one.

It runs on Alibaba’s weights, for now

The most interesting detail is the base model. Microsoft-Decision-1 is built on Qwen3.5-9B, from Alibaba Cloud. Microsoft says it will rebase on its own models and OpenAI’s, but gives no date. So the first version of Microsoft’s decision model is a tuned Qwen, which says as much about how fast this category moved as anything in the benchmark table.

The marketing line is that decision models do not hallucinate the way chat models do, because the answer space is constrained. That claim deserves scrutiny. A constrained answer can still be confidently wrong, and a model this cheap will be pointed at decisions nobody checks by hand.

What this does to your inference bill

The practical move is routing. Keep the general model for work that needs it. Send classification, extraction and simple decisions to the cheap one. The saving is largest where you already make millions of small calls, because the unit price is what scales. The risk is that routing adds a failure mode of its own. A classifier that misfiles a request sends it to the wrong place, quietly, at scale.

Ask three questions of your own stack. How many of your requests are a label rather than a paragraph. What would a twenty times lower unit cost do to that line of the bill. And do you have a labelled set you trust enough to test a cheap model against before you route to it.

Related reading. Our Enterprise AI Infrastructure report covers the four layers under the model and where they break, from hardware up to the development layer.