What is a token in AI?

What a token is

A token is the basic unit of text an AI model processes. Before a large language model reads your message, it splits the text into tokens. The Google Gemini documentation says tokens can be single characters like z or whole words like cat, and long words are broken up into several tokens. In the paper, Sennrich et al. describe encoding rare and unknown words as sequences of subword units. Punctuation and numbers count too. The model then writes its reply token by token.

In English, a token is very roughly three-quarters of a word. The Google Gemini documentation says that for Gemini models a token is equivalent to about 4 characters. It also says 100 tokens is equal to about 60-80 English words, so a page holds a few hundred. The Google Gemini documentation defines the set of all tokens used by the model as its vocabulary.

Why tokens matter at work

You meet tokens in three places:

1. The context window. Anthropic defines it as all the text a language model can reference when generating a response, including the response itself. That means your prompt, the files you attached, the conversation so far and its own reply together. See context window.
2. Usage limits on a subscription. Plans cap how much you can do in a period, and heavy use of long files and long chats reaches the cap sooner.
3. Pricing through an API. When a company builds AI into a tool, it is typically billed for tokens in and tokens out. The Google Gemini documentation says the cost of a Gemini API call is determined in part by the number of input and output tokens. See API.

What the documentation adds

• Everything counts. Anthropic says everything in the request counts toward the context window, including the system prompt, every message and your tool definitions.
• How big it gets. Anthropic says the context window holds up to 1M tokens, depending on the model.
• More is not automatically better. Anthropic says accuracy and recall degrade as token count grows, a phenomenon it calls context rot.
• You can check before you send. Anthropic offers a token counting feature to determine the number of tokens in a message before you send it.
• Hidden thinking tokens are billed. Anthropic says thinking tokens are billed as output tokens once, when they are generated. See reasoning model.

An example

A team uploads a 40-page vendor contract. The tool warns that the file uses most of the available window. They split the contract into sections and ask about each, which keeps the request under the limit and makes the answers easier to check.

Tokens in a quick example

The sentence “Please summarize this contract” is about five tokens. A 40-page contract attached to it can be tens of thousands. The model’s reply is counted too, so a long answer to a long file uses tokens twice, once reading and once writing. That is why a short question about a big document can still trigger a limit, and why summarizing in sections often works better than one giant request.

Some tools show a running estimate of how full the window is. Others only warn you at the edge. If answers start ignoring early parts of a long conversation, that is a sign you have filled the window, and a fresh chat with a short summary of where you left off will usually fix it.

How to spend fewer tokens

• Attach only the pages that matter instead of the whole file.
• Start a fresh chat for a new task rather than continuing a very long thread.
• Ask for a short format (“five bullets”) when you do not need an essay.
• Do not paste the same background into every message; save it as reusable instructions.

For a worked example on long documents, see how to summarize long documents with AI and summarizing PDFs and contracts.

Token examples

These counts are rough and differ by model, but they show the idea:

• A short, common word such as “the” or “cat” is usually one token.
• A longer or rarer word, such as “unbelievable”, is often split into two or three tokens.
• A number, a space or a punctuation mark can be a token of its own.
• An email of about 150 words is roughly 200 tokens. A 40-page document is tens of thousands.

Token in AI is not the same as other tokens

The word “token” has other meanings. A crypto token is a digital asset on a blockchain, and a login or API token is a secret that proves who you are. An AI token is none of those: it is only a chunk of text the model counts. (If a tool asks you for an “API token” to connect it, that is the login kind, so keep it private.)

A common misconception

People often equate tokens with words or with the number of questions. Neither is exact. Long attachments, long replies and long chat history all use tokens, so a small prompt on top of a big file can still be expensive. Tokens also say nothing about quality: a longer answer is not a better one.

Learning to give an AI just the right amount of context is one of the practical skills in Learn AI Fast, a two-day in-person class in Boston for non-technical professionals.

Sources

• Google AI for Developers, Understand and count tokens (Gemini API): “Tokens can be single characters like z or whole words like cat”
• Google AI for Developers, Understand and count tokens (Gemini API): “Long words are broken up into several tokens”
• Sennrich et al., “Neural Machine Translation of Rare Words with Subword Units” (arXiv): “encoding rare and unknown words as sequences of subword units”
• Google AI for Developers, Understand and count tokens (Gemini API): “a token is equivalent to about 4 characters”
• Google AI for Developers, Understand and count tokens (Gemini API): “100 tokens is equal to about 60-80 English words”
• Google AI for Developers, Understand and count tokens (Gemini API): “The set of all tokens used by the model is called the vocabulary”
• Anthropic, Context windows: “all the text a language model can reference when generating a response … including the response itself”
• Google AI for Developers, Understand and count tokens (Gemini API): “determined in part by the number of input and output tokens”
• Anthropic, Context windows: “Everything in the request counts toward the context window … your tool definitions”
• Anthropic, Context windows: “up to 1M tokens … depending on the model”
• Anthropic, Context windows: “As token count grows … accuracy and recall degrade, a phenomenon known as context rot”
• Anthropic, Token counting: “determine the number of tokens in a message before you send it to Claude”
• Anthropic, Context windows: “Thinking tokens are billed as output tokens once … when they are generated”