What an AI Token Actually Is, and Where They Accumulate

A token is the unit your model bills, reasons and forgets in, and it is not a property of your text. It belongs to the model. Change the model and the same sentence becomes a different number of tokens, at a different price, against a different limit — which is why a team that upgraded nothing but its model version can watch its bill move without writing a line of code.

Most explanations stop at “a token is about four characters.” That is true and nearly useless. What a technical buyer needs is the map: what the unit is, every place in a request where the units accumulate, and what the meter adds up to once you zoom out to the grid. This guide is that map.

A token is a subword unit, chosen by the tokenizer

Models do not read characters and they do not read words. Text is cut into subword units by a tokenizer, using byte-pair encoding: the most frequent character sequences in the training corpus get merged into single units, and everything rarer is assembled from smaller pieces. Common words survive whole. Uncommon ones arrive in fragments.

The working ratio for English is roughly four characters per token, or about three-quarters of a word — the figure Zapier’s explainer uses, with the honest caveat that it varies substantially. Punctuation, whitespace, code and non-Latin scripts all break it.

The important part is what the ratio implies. A tokenizer is a lookup table produced by a training run. It ships with the model, and it is not stable across models.

The cleanest evidence for this is a vendor documenting it against itself. Anthropic’s token counting documentation states that Claude 4.7 and later use a newer tokenizer, and that the same input text produces approximately 30 percent more tokens than on earlier models. The page goes on to tell you not to reuse counts measured on an older model when estimating cost or context fit. Same vendor, same string, thirty percent more units, and billing reflects the new count.

This is the fact to hold onto: your prompt does not have a token count. It has a token count on a given model.

The multilingual tax

If tokenization is learned from a corpus, languages under-represented in that corpus tokenize worse. Petrov et al. measured this across a wide set of languages in Language Model Tokenizers Introduce Unfairness Between Languages, and found that the same text translated into different languages can differ in tokenized length by up to 15 times.

That 15× is the extreme of the distribution, not the typical European-language gap, and it should be quoted that way. But the consequence is structural rather than cosmetic: because tokens are the unit of billing, of latency and of context, a speaker of a disadvantaged language pays more, waits longer and fits less into the same window for the identical meaning. If your product serves non-English markets, the tokenizer is a pricing decision you did not get to make.

Where tokens accumulate

Here is the part most teams have never itemized. A request is not “the prompt.” It is a stack of components, and every one of them is billed as input on every call.

The system prompt. Re-sent with each request unless it is cached. A long instruction block is a fixed cost multiplied by your entire call volume.

The user message. The only component most people picture.

Tool and function definitions. Full JSON schemas, names and descriptions, sent so the model knows what it may call. The scale is easy to underestimate, and Anthropic’s own documentation gives a clean before-and-after: a bare message counts 14 input tokens; the same message with a single small weather tool attached counts 403. You are billed for the menu, not only for the meal — and most agents ship far more than one tool.

Retrieved context. RAG hands the model whole chunks, priced whole, whether or not the answer was in them.

Conversation history. The API is stateless. The model does not remember the previous turn; your client re-sends the transcript, so turn ten pays for turns one through nine again. Left untrimmed, the cost of a conversation grows quadratically in its length.

Images. Billed as tokens too, and more than people expect. Claude processes an image as 28×28-pixel patches, costing ⌈width/28⌉ × ⌈height/28⌉ visual tokens — so a 1000×1000 image is 1,296 tokens, per the vision documentation. One screenshot costs about what five thousand characters of prose costs. The same page recommends uploading via the Files API rather than base64 specifically because base64 images sit in the conversation history and get re-sent, in full, on every subsequent turn.

Reasoning tokens. Thinking is generated text, priced like generated text, and on the current turn it counts against your input as well.

Output. Produced one token at a time, and priced above input on every major provider — commonly around 5× the input rate.

Two levers that actually move the number

Given that map, most optimization is unglamorous.

The first lever is caching. Anthropic’s prompt caching pricing charges 1.25× the base input rate to write a five-minute cache entry, 2× for a one-hour entry, and 0.1× to read one. A cached prefix — your system prompt, your tool definitions, your fixed context — pays for itself on the second read and is ninety percent off thereafter. If you have a stable prefix and you are not caching it, that is the cheapest change available to you.

The second is sending less, which requires knowing what you send. Almost nobody measures token spend per feature, so waste stays invisible until the invoice arrives as one number. (We wrote about the spending side of this separately, in where your AI bill actually goes.)

There is a security-shaped version of the same problem. Because tokens cost money and the endpoint is public, an attacker who cannot take your service down can still make it expensive — driving billable consumption through crafted or repeated requests. It is the same control set as any other resource-exhaustion risk: per-user rate limits, request timeouts, hard spend caps with alerting. Budget is an availability property once your compute is metered by the token.

More context is not more quality

The obvious response to all this is to stop optimizing and buy a bigger window. It works less well than it sounds.

Liu et al. showed in Lost in the Middle that model performance depends on where in the context the relevant information sits: accuracy is highest when it appears near the beginning or the end, and degrades significantly when it is buried in the middle. The effect shows up even in models explicitly built for long contexts.

So padding the window with everything that might be relevant does two bad things at once. It raises the bill, and it can lower the answer quality by burying the useful passage among the filler. Retrieval quality is not a cost optimization you do later; it is the thing that makes long context work at all.

The meter, at the scale of a grid

The environmental question is where this discussion usually collapses into two unhelpful positions. Here is what is actually documented.

In August 2025 Google published a per-prompt figure for Gemini Apps: a median text prompt consumed 0.24 watt-hours of energy, 0.03 grams of CO₂ equivalent, and 0.26 millilitres of water, measured in May 2025. Google also claimed the per-prompt energy footprint fell 33× over the preceding twelve months.

Read the footnotes, because Google supplies them. It is a point-in-time median, not a distribution — the tail is exactly where the expensive reasoning-heavy requests live. It measures inference, not training. And by Google’s own statement, the data and claims have not been verified by an independent third party. No other major provider publishes a comparable methodology, which makes this the only number of its kind rather than an industry baseline. A single vendor’s self-reported median is a data point, not a benchmark.

Zoom out and the picture inverts cleanly. The IEA’s Energy and AI report puts data centre consumption at around 415 TWh in 2024 — about 1.5% of global electricity — and projects roughly 945 TWh by 2030 in its base case. In April 2026 the agency reported that data centre electricity demand grew 17% during 2025, against roughly 3% growth in global electricity demand overall, with AI-focused capacity growing faster still.

Both things are true simultaneously, and the tension between them is the honest answer. Energy per query is small and falling fast. Total energy consumed is large and rising fast. Efficiency per token has never been the same as less energy used, because the falling unit cost is what makes the rising volume possible.

What to do Monday morning

Four things, in order of how much they return for the effort.

Instrument first. Log input, output, cached and reasoning tokens per feature, not per account. You cannot cut a bill you cannot itemize, and the component map above is only useful once you know which component is yours.

Cache the stable prefix. System prompt, tool definitions, fixed context. At 0.1× on reads, this is the highest-return change most teams have not made.

Audit the tool schemas. Count the tokens your definitions add to every single call, then delete the tools that agent never picks. The 14-to-403 jump is per request, forever.

Re-count when you change models. Not just the price per million — the tokens themselves. A tokenizer change of 30% is a 30% cost increase and a 30% smaller usable context, arriving silently with an upgrade you took for a capability reason.

Instrumenting that map is the item that never reaches the top of a backlog, because it competes with roadmap work and produces no feature at the end. A focused engagement is one way to get it done in weeks rather than quarters.

References

• Zapier — What is a token in AI?
• Anthropic — Token counting
• Anthropic — Vision (image token cost)
• Anthropic — Prompt caching
• Petrov et al. — Language Model Tokenizers Introduce Unfairness Between Languages (arXiv:2305.15425)
• Liu et al. — Lost in the Middle: How Language Models Use Long Contexts (arXiv:2307.03172)
• Google — Measuring the environmental impact of AI inference (August 2025)
• IEA — Energy and AI, executive summary (April 2025)
• IEA — Data centre electricity use surged in 2025 (April 2026)