AI costs are going up, caps are getting tighter, and a lot of people are confused about why. So let me break down what a token actually is and why it gets so expensive so fast. 💸

What even is a token?

A token is a word or part of a word. That is it. "The cat sat on the mat" gets split up into tokens, but so do spaces, full stops, curly brackets, and special characters. Chinese characters might take two tokens each. The exact split depends on the tokenizer the model uses, and that is built around frequency. "The" is so common it gets its own token. A modern model might have access to around 100,000 tokens covering English, other languages, code symbols, and more.

When tokens pass through the model, they are converted into an embedding, a vector of numbers representing what the word means in context. That part is learned during training. The tokenizer itself is more of a design decision.

Why does it cost so much?

These models work in what is called an autoregressive way. They take everything in, look at it all at once, and output one single token at a time. So if you ask it to complete "the cat sat on the...", it reads the whole thing and produces "mat". Then, to produce the next token, it reads everything again, including "mat". Then again. And again.

Every single forward pass through the model- everything is in there. The context window grows with each step, and that growth costs electricity and compute each time.

There is something called KV caching that helps, where the model stores some of the intermediate calculations so it does not have to redo them completely each time you add a token. For short conversations, this barely matters. For a context window stuffed with a million tokens' worth of documents, it makes a big difference. The catch is that if you go off for a coffee and come back, that cache has probably been cleared to serve other users, so it all has to be loaded back in.

Where it really explodes: coding agents

A chatbot conversation does not use that many tokens if your questions are short and simple. A coding agent is a completely different thing.

A coding agent does not output directly to you. It makes tool calls, which means it outputs structured instructions to the system like "read this file" or "delete that file", and the system responds by dumping the file contents back into the context. Those file contents are not small. A single file might be 5,000 tokens, and that file now sits in the context window for every single step that follows.

Here is a rough sketch of what one simple bug fix might look like:

• System prompt: ~4,000 tokens

• Your query: ~200 tokens

• First thought: ~2,000 tokens output

• Tool call to read a file: ~100 tokens output

• File contents returned: ~5,000 tokens input

• Second thought: ~2,000 tokens output

• Second file read and contents: ~4,100 tokens input

• Final thought and code patch: ~3,500 tokens output

By the time it is done, you have paid for somewhere around 55,000 to 60,000 input tokens and several thousand output tokens. For one bug fix. Where it only read two files.

Now imagine it does not work, and you say, "You broke it again." That whole conversation history goes back in on the next round.

Why the pricing shift was always going to happen

Most providers charge somewhere around two to three dollars per million input tokens and around fifteen dollars per million output tokens. When a coding agent is spinning through millions of tokens per session, a flat monthly fee makes no sense for the provider. GitHub Copilot recently moved from a request-based model to a token-credit model, which surprised many. They should not have been.

The flat fee was always a subsidy. When you pay a fixed monthly amount and an agent can run for hours, burning millions of tokens, that is not sustainable. Once the real cost becomes visible, everything changes.

Some companies measured AI adoption by how many tokens people used. That is like measuring driving quality by how quickly someone wears through their tires. It tells you almost nothing useful and incentivizes exactly the wrong behavior. Long-winded questions, agents stuck in loops, expensive models used where cheaper ones would do.

What actually makes sense

Short, focused questions. Quick fixes. Co-completion, where you write half a for loop, and the model finishes it. Small context, fast response, low cost.

The agentic stuff has a place, but it needs to prove itself through actual product output. Did it ship a feature? Did that feature make money? Those questions have not been answered clearly yet across most companies, and until they are, the cost is hard to justify at scale.

The next year is going to be very interesting.