dictionary
Inference
Published Updated
While training is the resource-intensive process of teaching a model, inference is the act of actually using the model. When you type a prompt into ChatGPT and it generates a response, that generation process is a series of inference steps. Optimizing inference is critical for reducing latency and cloud costs in production systems.
CategorySystems
Reading time2 min read
Definition
The process of running live data through a trained machine learning model to make a prediction or generate an output.
Need this applied?
We help teams go from definitions to deployed workflows—safely and fast.
FAQ
Email this summary + checklist
Get a copy of “Inference” and an AI readiness checklist in your inbox.