The Big O of AI

The core problem: Enterprises deploying agentic AI have no way to measure the expected token efficiency of an application’s design; they discover inference costs on the invoice, after they deploy the architecture. Computer science solved this problem once before with Big O notation. An equivalent is coming for AI: token complexity, the order of growth of an application’s inference spend. Telcos should evaluate every AI initiative by its complexity class, because falling token prices will never rescue a design that scales badly.

AI has a big problem: no one knows how to predict the token costs of an enterprise AI application running at scale. Model benchmarks measure the models’ performance. FinOps dashboards measure spend after it happens. Nothing measures the design itself—the thing that actually sets the bill. Today, the only way to learn what your application will cost is to run it at scale and see what happens.

Back in the 1970s, computing had this problem, too. Compute time was scarce and expensive, so Donald Knuth borrowed a notation from 1890s German mathematics and gave programmers a way to know what a design would cost before anyone ran it. That’s Big O (“O” for Order of growth). It moved the cost question from the invoice to the whiteboard, and software computation has been measurable ever since.

AI doesn’t have that measuring stick yet. And the gap just got more expensive.

The meter’s running

In the last month, Anthropic pulled its frontier model, Fable 5, out of its subscription plans and put it on a meter: $10 per million input tokens, $50 per million output tokens. It’s the most expensive pricing the company has ever listed. Subscribers pushed back hard enough that Anthropic extended the cutoff twice before the meter finally switched on, on July 20. At roughly the same time, OpenAI launched GPT-5.6 and led its marketing with token efficiency: 54% fewer tokens on agentic coding tasks. Sam Altman explained the emphasis on CNBC: every enterprise is now scrutinizing what it spends on AI against what it gets back.

The two leading AI labs, in the same month, told you the same thing. One repriced its best model because it couldn’t afford to sell it flat-rate. The other positioned its best model on how little it consumes.

This matters to you because for thirty years, every business case you’ve built assumed packaged software’s millionth transaction was free. You paid the license, or the subscription, or the managed services contract, and then usage cost nothing. Every budgeting habit you have, every ROI model your team has ever presented to the board, rests on that assumption. Agentic AI just ended it. Software has a marginal cost now, metered in tokens, at prices that keep shifting.

The software decides how much to spend

Nobody on earth understands per-unit billing better than telco. We priced minutes, messages, and megabytes for decades. But we’ve always been on the selling side of the meter. As software buyers, we’ve lived almost entirely in the flat-cost world: perpetual licenses, on-premise systems, managed services contracts with a fixed monthly number. Most telcos never even moved to software as a service (SaaS). Agentic AI may be the first metered software your company has ever run at scale. Before now, there’s been a human on the other end deciding how much to consume. But an agentic system consumes autonomously. It decides how many reasoning passes to take and how much context to read, and no one signs off on either. When Anthropic moved Fable 5 to usage credits, its stated reason was that demand would be very high and hard to predict. Read that again: the company that built the model cannot predict how much will get consumed. Nobody is controlling the bill anymore. The design is choosing it.

Two operators can deploy the same agent, on the same model, for the same task, and end up with wildly different bills. One agent reads a tight context, takes two reasoning passes, and acts. The other rummages through bloated context, loops five times, retries twice, and acts. Same outcome. Same model. Same price list. Very different invoice.

We need Big O

So, here’s my prediction: AI is going to get its own Big O. I’m calling it token complexity, the order of growth of an application’s inference spend. Someone will formalize it, because no industry can price its products while its own cost structure is a surprise. Here’s my stab at the notation:

Constant (O(1)). Decisions your business already made execute as rules, at effectively zero cost per action. Volume grows; spend doesn’t.

Linear (O(n)). One reasoning pass per transaction. Your spend grows in lockstep with your business, at frontier prices, forever. Manageable, if the margin per transaction covers it.

Quadratic and worse (O(n^x)). Remember that an agentic loop re-reads its accumulated context at every step. Step one reads the prompt. Step ten reads the prompt plus nine steps of history. Step fifty reads everything. That means a fifty-step run can cost twenty-five times a ten-step run, and when agents spawn sub-agents that each carry their own context, the curve gets steeper still. Nobody designing the workflow chose that number. The shape of the loop chose it.

The dangerous part is that all three classes look identical in a demo. A hundred transactions in a pilot are affordable on any curve. Ten million in production reveal which one you built. The pilot hides your complexity class; the price tag reveals it.

The three levels of managing it

So what do you do? Calculate your AI bill with this equation: Spend = n × t × p. 

Spend equals the number of metered actions, times tokens per action, times price per token. I see enterprises attacking it at three levels, and each level pulls exactly one lever.

Level 1: Buy cheaper. This is Procurement’s comfort zone, and the savings are real: prompt caching cuts repeated context by 90%, batch APIs run non-urgent work at half price, model routing sends simple requests to models that cost a tenth as much. Do all of it. But cheaper tokens are a faster processor. They don’t change the complexity class.

Level 2: Burn fewer. Engineering can use context discipline, right-size the model to the task, cap loop iterations, and prune what agents carry between steps. This separates well-run AI programs from expensive science experiments. There are tons of best-practice ideas on X and Reddit; use them. This idea shrinks the coefficient but doesn’t really impact the shape of the usage curve. If you want real impact, you need to do real work, found in Level 3.

Level 3: Change the class. Ask of every AI operation: does this actually require inference, or is this a decision made years ago? Whether a subscriber is eligible for an offer, whether a plan change triggers proration, whether a port-in request meets the regulatory checklist: you settled these decisions long ago. They’re rules. Don’t pay frontier-model prices for an agent to derive a settled rule. A better design executes decided things deterministically, at constant cost, and uses inference only where judgment is needed: the unusual case, the novel request, the exception no rule anticipated. This is the design principle we built the Totogi Ontology around.

A word of caution as you’re analyzing your AI: don’t expect vendors to optimize this for you. Most of them aren’t trying to minimize your inference. The model labs monetize consumption. The hyperscalers are thrilled to bill you for the compute underneath it. Systems integrators charge by the hour to build your agents, and they’ll happily bill more next year to optimize them. Levels 1 and 2 are safe for all of them to recommend, because a discounted meter is still a meter. Level 3 threatens the meter itself, so your goal is to achieve Level 3 for your AI design.

In short: everyone will help you with t and p. Only design touches n. 

Time to compute

It’s time we have a way to predict the cost of an enterprise AI application running at scale, a way to move the cost question from the invoice to the whiteboard. To achieve a settled, agreed-upon Big O for AI, we need to start asking the hard questions. In fact, here are some to take into your next AI review:

• Which complexity class is each of our AI initiatives in? 
• What does each agentic action cost us today, and how does that cost grow as volume grows?
• What’s the cost per completed action, at expected volumes?
• What share of our inference spend is re-deriving decisions we already made? 

While it’s the goal for your applications to achieve O(1), they will never actually get there. If every operation were a settled rule, you wouldn’t need AI at all; you’d need the rules engine you already own. It’s the exceptions that still require inference. But the exception rate isn’t constant. Every exception your system handles once can become encoded precedent: a new rule, a resolved edge case that never needs deriving again. 

This is the other half of what the Totogi Ontology does: when the model works an exception, the resolution gets encoded, and yesterday’s judgment call becomes tomorrow’s rule. That means mature AI spend doesn’t scale with transactions at all. It scales with genuinely new situations: O(unique cases), if you’ll allow me one more class. Ten million transactions might contain a few thousand of those. You’ll never reach O(1); you’ll approach it, and every month you get closer. That’s the goal, so get started.

Recent Posts

Get my FREE insider newsletter, delivered every two weeks, with curated content to help telco execs across the globe move to the public cloud.