LLMs use the Transformer architecture, which relies on attention rather than recurrence and is highly parallelizable, and are pretrained to predict the next token in large text corpora. Scale changed what that objective produces: the 175-billion-parameter GPT-3 could perform many tasks from instructions and a few examples in the prompt, without any gradient updates or fine-tuning.
encyclopedia
Large Language Model (LLM)
Published Updated
Reviewed by Coy Robison, Founder, on
Large language models learn statistical patterns in text by predicting the next token across very large corpora, and that single objective yields broad abilities: drafting, summarizing, translating, extracting information and holding a conversation. A pretrained model is then adapted, usually with instruction tuning and human feedback, so it follows requests reliably. Because it generates likely text rather than looking facts up, an LLM can state false things confidently, so production systems pair it with retrieval, tools, evaluation and human review.
Definition
A neural network trained on large text datasets to predict and generate human-like language.
Need this applied?
We help teams go from definitions to deployed workflows—safely and fast.
From pretrained model to assistant
Ouyang et al., 2022A larger model is not automatically better at doing what the user asked. Instruction tuning with human feedback (supervised examples, then reinforcement learning from human rankings) closes that gap: in the InstructGPT study, people preferred the outputs of a 1.3-billion-parameter tuned model over those of the 175-billion-parameter GPT-3.
What LLMs are good at
OpenAI, 2023LLMs handle language-shaped work: drafting and rewriting, summarizing, classifying and extracting fields from text, translation, and conversational interfaces. Recent models also accept images as input. GPT-4, for example, reached human-level performance on various professional and academic benchmarks, while its developers note it remains less capable than humans in many real-world scenarios.
NIST describes confabulation, confidently presented false content, as a natural result of how generative models work: they approximate the statistical distribution of their training data. OpenAI’s research adds that standard training and evaluation reward guessing over admitting uncertainty. A model’s knowledge is also fixed at training time, so it cannot know about recent or private information unless that information is supplied in the prompt.
- • Confident errors: verify facts that matter, or supply them from a trusted source.
- • Stale or missing knowledge: use retrieval or tools for current and private data.
- • Manipulable input: treat everything the model reads as untrusted.
Most applications do not train their own model. They start from a foundation model, trained on broad data at scale and adaptable to many downstream tasks, and adapt it with prompting, retrieval of their own documents, fine-tuning, or tool use, where the model asks the application to run a function such as a database lookup and then uses the result.
FAQ
Do LLMs understand like humans?
LLMs model statistical patterns in language. They can be fluent and often correct, but fluency is not evidence that a statement was checked; NIST notes that the same statistical prediction that produces accurate text can also produce text that is factually wrong or internally inconsistent.
What is the difference between an LLM and a foundation model?
A foundation model is any model trained on broad data at scale that can be adapted to many tasks. An LLM is a foundation model focused on language; image and multimodal models can be foundation models too.
Why do LLMs make things up?
They generate the most plausible continuation rather than retrieving stored facts, and OpenAI’s research argues that common training and evaluation methods reward a confident guess over an admission of uncertainty. Grounding answers in supplied sources and allowing the model to say it does not know both reduce the problem.
Can an LLM use my business’s current data?
Not on its own: its knowledge is limited to what was available when it was trained. Applications supply current or private data at request time, either by retrieving relevant documents into the prompt or by letting the model call tools that query live systems.
Email this summary + checklist
Get a copy of “Large Language Model (LLM)” and an AI readiness checklist in your inbox.