dictionary

Inference

Published Updated

While training is the resource-intensive process of teaching a model, inference is the act of actually using the model. When you type a prompt into ChatGPT and it generates a response, that generation process is a series of inference steps. Optimizing inference is critical for reducing latency and cloud costs in production systems.

CategorySystems
Reading time2 min read

Definition

The process of running live data through a trained machine learning model to make a prediction or generate an output.

Need this applied?

We help teams go from definitions to deployed workflows—safely and fast.

FAQ

Email this summary + checklist

Get a copy of “Inference” and an AI readiness checklist in your inbox.