What is an LLM (large language model)?

What it means at work

Every chat assistant you have used at work sits on top of an LLM. Claude, ChatGPT and Copilot are products; the LLM is the engine inside. In short, the Anthropic glossary describes LLMs as AI language models with many parameters trained on vast amounts of text data. That is why an LLM is good at drafting, rewriting, summarizing, classifying and explaining, and less good at anything that needs a guaranteed correct fact.

How it works

An LLM predicts what text should come next, one small piece at a time, based on patterns learned from its training material and on whatever you put in front of it in the conversation.

• Google’s Machine Learning Crash Course says LLMs predict a token or sequence of tokens, and that a token can be a word, a subword or even a single character. See token.
• The Google glossary says all large language models built on the Transformer are auto-regressive.
• Vaswani et al. propose the Transformer, a network architecture based solely on attention mechanisms.
• The GPT-4 Technical Report says GPT-4 is pre-trained to predict the next token in a document.
• The scale is large. The GPT-3 paper describes an autoregressive model with 175 billion parameters.
• A model trained only to predict text needs more work before it is a useful assistant. The Anthropic glossary says such pretrained models are not inherently good at answering questions or following instructions. Assistants are fine-tuned afterward.

An example

You paste a rambling customer email into an assistant and ask for a three-line summary and a polite reply. The LLM reads your text and produces both. You did not program anything; you described the job in plain English.

A common misconception

People often assume an LLM looks facts up in a database. It does not, unless the product around it adds a search or file step. Its fluent writing comes from patterns, so it can sound certain while being wrong: Anthropic says models can sometimes generate text that is factually incorrect or inconsistent with the given context. Treat its output as a strong first draft that you check, especially for numbers, names and quotes.

Sources

• Anthropic, Claude Platform Docs glossary: “Large language models (LLMs) are AI language models with many parameters … trained on vast amounts of text data”
• Google for Developers, Machine Learning Crash Course: LLMs: “predict a token or sequence of tokens”
• Google for Developers, Machine Learning Glossary: Language: “All Transformer-based large language models are auto-regressive”
• Vaswani et al., “Attention Is All You Need” (arXiv): “the Transformer, based solely on attention mechanisms”
• OpenAI, “GPT-4 Technical Report” (arXiv): “pre-trained to predict the next token in a document”
• Brown et al., “Language Models are Few-Shot Learners” (arXiv): “an autoregressive language model with 175 billion parameters”
• Anthropic, Claude Platform Docs glossary: “are not inherently good at answering questions or following instructions”
• Anthropic, Reduce hallucinations: “can sometimes generate text that is factually incorrect or inconsistent with the given context”