When inference has to pay for itself

There is a running argument about whether the money going into AI will come back. Sequoia framed it as a $600 billion question in 2024: the revenue the industry would need to justify what it is spending on infrastructure. Goldman Sachs asked it more bluntly the same year, setting roughly a trillion dollars of coming capital spending against the benefit it has produced so far. We do not have a view on the macro number, and a two-person lab asserting one would be noise. The question that reaches us is smaller and harder to dodge: when the spending has to be justified deployment by deployment, which deployments survive.

It helps to be precise about where the losses actually are, because “inference is subsidized” is mostly false where people say it. Metered serving of tokens appears to be profitable. When DeepSeek disclosed the economics of its own inference system in early 2025, it reported a theoretical daily cost near $87,000 against a theoretical daily revenue near $562,000 at list prices. It was careful to call the margin theoretical, because its real revenue was lower: free web and app access, off-peak discounts, and no research or training cost in the figure. The point survives the caveats. Serving a token at list price clears the cost of serving it. The subsidy lives elsewhere, in flat-rate consumer plans where the heaviest users cost more than they pay, and in the research budgets of the frontier labs. So the reckoning is not a coming spike in token prices. It is each piece of deployed work having to show it was worth running.

On that, the record so far is poor. MIT’s NANDA initiative reported in 2025 that after thirty to forty billion dollars of enterprise spending, about 95 percent of generative-AI pilots showed no measurable impact on the bottom line. The study is survey and interview work and has been criticized on that ground, and we cite it with the caveat, but the shape is not in dispute: most deployments cannot show their return. The reason is usually not the model. It is that no one established what an accepted outcome was, or what it was worth, which is the subject of a separate note. A deployment that never defined its unit of account cannot prove it paid for itself, whatever the model costs.

The strongest argument against building for efficiency is that efficiency is about to stop mattering. Token prices are falling fast. a16z described roughly a tenfold drop per year in the cost of a given level of capability, and Epoch AI measured the same trend at anywhere from 9 to 900 times per year depending on the task, with a median near 50. If the price of the work halves and halves again, the savings from doing it efficiently shrink toward nothing, and raw token efficiency becomes a commodity rather than an advantage. We concede this plainly: as a way to spend less on tokens, our architecture has a shrinking edge. What deflation does not touch is the cost of a wrong answer accepted, which is paid in the workflow and not in the token, and does not fall when tokens do. There is a second concession worth stating in the same breath: cheaper and more reliable inference generates its own demand, so the capacity being built may well find work. We are not betting against AI. We are betting on which part of it pays.

So our assumptions are stated, and they are checkable against us later. The system is designed to be worth running at list-price inference, with no subsidized tokens holding up the math. The claim it rests on is that verified work inside a chosen vertical costs less per accepted outcome than an unverified call, at any token price. If token prices fall to nothing and that per-outcome cost does not follow them down, this note was wrong, and the errata will say so.