LLM Cost per Task: Are Cheap Models Really Cheap?

Here is a question that is easy to get wrong.

Two AI models can do the same piece of work. One of them charges three times more for the text it produces. At the end of the month, which one has cost you more?

Often enough, neither. They land within a few cents of each other.

The two models are real. Alibaba’s Qwen3.8-Max charges $6 per million output tokens and OpenAI’s GPT-5.6 Sol charges $20, both published on their own price pages, and three-to-one is not a subtle difference. Run the same set of hard tasks through both and total what lands on the invoice: Qwen3.8-Max comes to about $0.91 a task, GPT-5.6 Sol to about $1.01.

A three-fold discount, worth ten cents.

Ten cents is not worth writing about until you scale it. An agent grinding through two thousand tasks a month on Sol runs about $2,020. Move that workload to Qwen because the price page promises three-to-one, and you have budgeted $606 for it. The invoice arrives at $1,820.

The short version

• A price per million tokens is a unit price. It tells you what one unit costs and nothing about how many units the job will take, and that second number now varies between models far more than the first one does.
• So the price page no longer ranks models by what they’ll cost you. Some models keep almost all of their advertised discount. Others keep almost none. Models with similar rate cards can still land far apart on the bill.
• The number to compare is cost per task, and the only version that counts is the one measured on your work. There’s a recipe for that at the end, and it takes an afternoon.

Dollars per million tokens is the unit the labs publish, so it’s the unit every comparison quotes, ours included. Our rundown of the leading models lines them up on quality, context, licensing and list price, which is the right way to draw a shortlist. This is the layer underneath it.

Why the sticker price stopped predicting the bill

Three things happened, and they multiply together.

Models decide how long to think. Modern models generate hidden reasoning tokens before they answer, and you are billed for those exactly as for the visible ones. Two models can hand back identically short answers and differ tenfold in what it took to get there. Artificial Analysis rates Qwen3.8-Max maximally verbose, describing it as “notably slow and very verbose.” That is not a defect - Qwen is a strong model and the thoroughness is the point. You just pay for it by the word.

Models decide how many turns to take. Most serious AI work now runs as an agent: read, act, check, retry. One model may finish in a handful of steps while another keeps checking and revising. More round trips mean more tokens, even when the final deliverable is the same.

Every turn re-reads the ones before it. On turn 40, the model re-sends turns 1 through 39 as input. Turn count doesn’t just multiply generation, it multiplies your input bill, which is why caching is now priced as its own line item. For the leanest models on the board, generation is not always the main cost.

A cheap model that thinks twice as long and takes four times the turns is not cheap.

Where the price page and the invoice disagree

The Cost per task column is Artificial Analysis’s measurement: they run every model through the same suite of hard tasks and total the real bill, counting input, cached, reasoning and answer tokens alike. The last column is our arithmetic on top of theirs.

Rows are sorted by Intelligence Index score, highest first. Models with the same score are grouped together.

Reading the table.

• Every price and measurement here was re-checked against Artificial Analysis’s full table on August 27, 2026. Cost per task is their current published measurement for each model, and Intelligence is their composite quality score on a 0–100 scale, where the current leader sits at 63. Several figures moved in the six days before that: OpenAI cut GPT-5.6 Sol from $5 / $30 to $4 / $20, which is why nearly every token-price multiple in the last column is smaller than it was - and why the four models that already cost more per task than Sol now look worse against it. The yardstick got cheaper in both directions.
• One model, several figures. Artificial Analysis publishes a separate measurement for every reasoning-effort setting a model exposes, so a single model can carry several scores and costs. We quote the highest measured setting for each model and compare like with like below. Read the multiples as approximate because the benchmark and its costing method continue to evolve.
• The sticker side of the last column is output price only, because output is where the rate cards differ most. The cost-per-task side counts everything: input, cached input, reasoning and answer. Models with relatively flat cards - DeepSeek V4 Flash charges 3x its input price on output - therefore look like they lose more of their discount than they do.
• Open-weight pricing still depends on the endpoint. The table follows the provider and rate shown on each current Artificial Analysis model page. Another host, or your own hardware, can produce a different bill for the same weights.

Three things the table says that the price pages don’t

Most models give up some of their advantage, and the spread is enormous. Cost per task counts input and cached tokens too, so a model with cheap output relative to input has further to fall. That much is expected. What isn’t: GLM-5.3-Flash is 40x cheaper than GPT-5.6 Sol per output token and 11x cheaper per task - a real saving, and a quarter of the one its price page implies - while Kimi K3 keeps almost all of a much smaller discount, 1.3x becoming 1.2x. Read the multiple you actually get, not the one you were quoted.

A shrinking discount can still be a large discount. DeepSeek V4 Flash sheds roughly two fifths of its multiple, 15x on paper down to 9.2x in practice, and is still one of the cheapest capable models in the table. The lesson isn’t that cheap models are a lie. It’s that you cannot rank them by reading price pages.

Grok 4.6 is the cleanest counterexample on the page. Its price page says 3.3x cheaper than Sol per output token. Its measured cost per task says 1.2x dearer. Nothing about the rate card predicts that; it thinks its way there. Claude does the same thing more mildly - Sonnet 5 is 2x cheaper per output token and 1.7x dearer per task, and every Claude reasoning tier lands above what its price page predicts. Haiku 4.5 is the exception that shows why: it barely reasons, and it keeps more of its token discount than the rate card promises. Part of that isn’t behaviour at all, it’s measurement: Anthropic’s current generation uses a denser tokenizer that bills roughly 30% more tokens for the same text than the models it replaced. OpenAI is the mirror image, and since its August price cuts Luna and Terra now come in cheaper per task than their own rate cards imply.

The gap, drawn

Each line runs from the discount a model’s price page promises (hollow marker) to the discount its measured cost per task delivers (solid marker). The longer the line, the further the two disagree - usually because the saving evaporates, and occasionally the other way.

Five things this number can’t see

Cost per task beats price per token, and it still comes off a benchmark.

• It prices attempts, not successes. The metric counts what a model spent, never whether the answer was usable. A model that fails fast and cheap scores beautifully. In production you re-run the failures, and often a person reads the output before it ships, so a model that’s 10% cheaper per attempt and wrong 30% more often is more expensive in every way that matters. No public leaderboard measures cost per finished task, because nobody but you can judge finished.
• The “task” is Artificial Analysis’s, not yours. The average blends long agentic runs, terminal coding, graduate science questions and tool-calling exercises, with roughly half of it agentic work. That’s the right call for a general-purpose number in 2026, and it’s also why your own ranking will come out much flatter if your job is summarizing support tickets.
• It’s list-price, text-only, and blind to the fine print. Caching is modeled at each model’s typical hit rate rather than the rate you’ll get. Grok roughly doubles its rate above ~200K-token prompts, and the whole GPT-5.6 line does the same above 272K. Batch endpoints can halve the bill for anything that isn’t time-sensitive, and open-weight pricing changes from one host to another.
• Some models have no cost per task at all. Motif 3 has a published intelligence score and no published cost per task, and probably never will: its weights are free and there is no priced first-party endpoint to measure, so the only bill is your own hardware. Inkling and NVIDIA’s Nemotron 3.5 Lightning were in the same position a week ago and now have measured figures, $0.34 and $0.08, because a host started charging for them. That is not a gap in the data so much as a different question. Where a model has no price, “what does it cost” becomes a capacity-planning exercise rather than a lookup.
• It moves for reasons that have nothing to do with the model. GPT-5.6 Luna’s cost per task fell from $0.21 to $0.05 in July because OpenAI cut its price 80%. Open-weight figures can also move when the benchmark changes which provider endpoint it reports. Claude Sonnet 5’s figure would have risen by half in September, until Anthropic canceled the increase on August 10. Any cost-per-task number needs a date attached, or it means nothing. That includes ours.

The dial matters more than the model

Modern models expose a reasoning effort setting, which controls how long the model may think before answering, and it moves cost further than swapping models does. Claude Opus 5 is published at five effort settings, and across them its token consumption on the same benchmark suite spans roughly eight-fold, from 12 million tokens at the lowest setting to 100 million at the highest. Same model, same rate card, same tasks.

This has a quiet consequence for every leaderboard you read. Labs submit at maximum effort, because that maximizes the score, so the published cost per task usually reflects the most expensive setting a model has - rarely the one you’d run all day. Turning the dial down is the fastest saving available to most teams, and a frontier model turned down will often beat a mid-tier model turned up on quality and cost together.

Work out your own, in an afternoon

Every API already reports what you need.

1. Collect ten real tasks, with the messy context attached. Not clean samples: the actual ones.
2. Run each through two or three candidates, at the reasoning effort you would actually use in production.
3. Read the token counts off the response rather than estimating. Every provider returns a usage object with input, output and cached counts, and reasoning tokens sit inside the output number. Multiply by the rate card separately for input, cached input and output, because those three prices can differ by an order of magnitude.
4. Divide by the results you’d actually ship, not the number you ran. If eight of ten came back usable, your cost per finished task is the total divided by eight. That division is the one no leaderboard can do for you.

Ten tasks will not survive a statistician, and ten can absolutely crown a different winner than fifty would. Run ten anyway: a rough number you have beats a precise one you never get around to. If you want to do this properly, with a real evaluation set behind it, our guide to choosing and testing a model covers that side in more depth.

Where this leaves us

None of this makes the price pages dishonest. They answer a question that used to be the same as the one you were asking, and quietly stopped being.

You buy tokens. You consume finished jobs. For years those moved together closely enough that the price of one implied the price of the other, and that link holds only while the amount of work per job is roughly fixed. Once a model decides for itself how long to think and how many turns to take, that amount stops being a property of your job and becomes a property of the model you hired. Cheap and efficient are now two different things, and only one of them is published.

Which turns the moves that used to save money into questions. Dropping a tier, moving to an open model, taking the cheaper of two similar scores, accepting a version bump from your own vendor: all of these were reliable, and none of them are now. If you made a switch like any of those this year on the strength of a price page, go and look at what happened to the bill.

The fix is mostly a habit. Compare candidates on a task instead of a price page. Set the effort dial before you shop for a cheaper model, because it moves more money than the model choice does. And before any switch made in the name of cost, put ten real jobs through both options and count what you would have shipped.

The honest answer to “which model is cheapest” is: for which job, at which effort setting, measured on which date. It’s an irritating answer, and it’s the only one that survives contact with an invoice.

It’s also why we built MindsHub to treat the model as a setting. MindsHub Cowork is a workspace where you hand a whole task to an open-source AI agent and collect the finished work. Unified Inference gives you one API for a broad catalog of frontier and open models, so you can rerun a task across the models available there and compare the result and cost without rebuilding your integration. Try it free.

Frequently asked questions

What is cost per task for an LLM? The average total bill to finish one real task, counting every token the model consumed: input, cached input, hidden reasoning, and the visible answer. Artificial Analysis publishes it for its Intelligence Index suite, and it regularly ranks models in a different order than their price pages do.

Does a cheaper model per token ever cost more per task? Yes, and it isn’t rare. Claude Sonnet 5 is twice as cheap as GPT-5.6 Sol per output token and costs 70% more per task, $1.72 against $1.01. Grok 4.6 is the starker case: 3.3x cheaper per output token, 1.2x dearer per task. Anthropic’s reasoning tiers all behave this way relative to their sticker price, because they’re built to think longer and persist through more steps; Haiku 4.5, which barely reasons, does the opposite.

Can the same model get more expensive without a price change? Yes. A version update can change how long a model reasons or how many turns it takes without changing the headline token rate. Run the new version through your evaluation set before moving production traffic, because the rate card alone will not show that difference.

Does reasoning effort change cost more than switching models? Often, yes. Claude Opus 5’s token consumption spans roughly eight-fold across its five published effort settings at one unchanged price per token. Leaderboards typically report the highest setting, so the published figure is usually a worst case, and turning the dial down is the fastest saving most teams have available. It can even be free: Grok 4.6 at its top setting costs $1.23 a task and scores 60, while one notch down it costs $0.94 and scores 61.

Why do published cost-per-task numbers keep changing? Three reasons that aren’t the model: providers change prices, benchmark suites get revised, and the costing method itself gets updated. GPT-5.6 Luna went from $0.21 to $0.05 per task on a price cut. Always check the date on the number, and for an open model, check which host the measurement uses.

---

MindsHub by MindsDB puts every major model behind one endpoint. Unified Inference serves frontier and open models through the OpenAI and Anthropic request formats on one API key and one bill, and MindsHub Cowork is the agent workspace on top of that catalog - delegate entire projects and collect finished, shareable results, with work running on interchangeable open-source agent harnesses, Anton and Hermes. Founded 2018 in Berkeley. Backed by Benchmark, Mayfield, Y Combinator, and NVIDIA.