Two Very Different Stages
We have now learned that AI models learn patterns from data and that neural networks contain learned parameters.
But there are two very different activities that are often confused:
Training and inference.
Understanding the difference is essential for understanding how modern AI systems work.
What Is Training?
Training is the process through which a machine learning model learns useful patterns from data.
During training, the model processes many examples and adjusts its internal parameters so that its predictions become better.
A simplified training process looks like this:
- Provide a training example.
- The model produces a prediction.
- Compare the prediction with the desired result when one is available.
- Calculate an error or loss.
- Adjust the model parameters.
- Repeat the process many times.
Training Requires Many Examples
A model usually needs many examples to learn useful patterns.
For a simple problem, a relatively small dataset may be enough.
For a large language model, the training process can involve enormous quantities of text and other data.
The model processes these examples repeatedly or through very large numbers of training steps.
Training Changes the Model
The key characteristic of training is that the model parameters are adjusted.
The model starts with parameters that do not yet represent the useful patterns required for the task.
Training changes those parameters based on information from the training data.
After sufficient training, the resulting model can be used to process new inputs.
What Is Inference?
Inference is the process of using a trained model to produce an output from new input.
For example, a trained image classifier might receive a new photograph and predict what object appears in the image.
A trained language model might receive a question and generate a response.
In both cases, the model is using what it learned during training.
A Simple Example
Imagine training a model to recognize cats.
During training, the model receives many examples of cat and non-cat images.
The model adjusts its parameters based on those examples.
Later, you provide a photograph that was not part of the training dataset.
The trained model analyzes the photograph and predicts whether it contains a cat.
The training happened earlier. The prediction is inference.
Training vs Inference
The difference can be summarized simply:
- Training: Learn patterns and adjust model parameters.
- Inference: Use the learned parameters to process new input and produce an output.
What Happens When You Ask an AI Assistant a Question?
Suppose you open an AI assistant and type:
"Explain artificial intelligence in simple terms."
In a normal interaction, the model is not starting a new training process.
Instead, the system processes your input using an already trained model and generates a response.
That is inference.
Does the Model Learn From Every Conversation?
This question requires an important distinction.
During ordinary inference, the model does not normally update its underlying parameters after every message.
A conversation can provide context for the current interaction without permanently changing the underlying model.
Some AI products may have separate memory, feedback, personalization, or data-improvement systems, but these should not be confused with the model performing ordinary inference.
Why Is Training Expensive?
Large model training can require enormous amounts of computing power.
The model may process huge datasets and perform very large numbers of mathematical operations.
Training can therefore require powerful hardware, substantial electricity, storage, networking, and engineering resources.
Inference Also Requires Computing
Inference is generally less expensive than training a model from scratch, but it still requires significant computing resources for large models.
When millions of users interact with an AI system, the provider must operate enough computing infrastructure to process those requests.
This is one reason AI services need substantial data-center infrastructure.
Training and Inference Use the Same Learned Model
After training, the learned parameters are stored as part of the model.
During inference, the model uses those parameters to process new inputs.
The parameters are generally not changed during ordinary inference.
This distinction is important because it explains why a model can answer many questions without retraining itself after every question.
Fine-Tuning
Sometimes a trained model is further trained on a smaller, specialized dataset.
This process is commonly called fine-tuning.
Fine-tuning can adapt a model to a particular task, style, domain, or type of behaviour.
Fine-tuning is still a training process because the model parameters are adjusted.
Training Is Not the Same as Updating Knowledge in Real Time
Suppose an event happens today.
A model does not automatically gain permanent knowledge of that event simply because someone asks the model about it.
If the underlying model has not been trained or updated with information about the event, it may not know the latest details.
AI applications can use other systems such as search, retrieval, databases, or external tools to provide current information during inference.
Retrieval and Inference
An AI application can retrieve information from an external source and then provide that information to a model as part of the input.
The model can then use that retrieved information when generating a response.
The retrieval system and the language model are separate components, even though they can work together.
A Useful Analogy
Imagine a student preparing for an examination.
Training is similar to studying and practicing over a long period.
Inference is similar to using what was learned to answer a new question during the examination.
The analogy is not perfect, but it helps illustrate the difference.
Training Data and Inference Data
The examples used during training and the inputs received during inference serve different purposes.
Training data helps the model learn.
Inference data is the new input the trained model needs to process.
A well-designed AI system aims for the trained model to generalize from its training data to useful performance on new inputs.
Why Generalization Matters
A model that simply memorizes its training examples is not necessarily useful.
The model needs to perform well when it receives new inputs.
This ability is called generalization.
Inference is where that ability is tested in practical use.
The AI Model Lifecycle
A simplified AI lifecycle can be represented as:
Data Collection → Training → Evaluation → Deployment → Inference → Monitoring → Improvement
The exact lifecycle varies between projects, but the basic distinction remains important.
Why This Matters for AI Users
Understanding training and inference helps explain many common questions about AI.
For example:
- Why does an AI model not automatically know every new event?
- Why does asking a question normally not retrain the model?
- Why can an AI application use external search without changing the underlying model?
- Why does training a large model require substantial resources?
The Big Picture
We can now connect the major concepts:
Training data → Training → Learned parameters → Deployment → New input → Inference → Output
Training creates the learned model.
Inference uses that learned model.
Keeping these two processes separate is one of the most important concepts in understanding modern AI.
What You Should Remember
- Training is the process of learning patterns from data and adjusting model parameters.
- Inference is using a trained model to process new input.
- Normal conversations with an AI assistant are generally inference rather than training.
- Ordinary inference does not normally permanently change the model parameters.
- Fine-tuning is another form of training.
- External retrieval systems can provide current information during inference.
- Generalization allows a trained model to perform usefully on new data.
What Comes Next?
We now understand how AI models are trained and how they are used.
But there is a major problem we still need to address:
What happens when an AI model gives an answer that sounds convincing but is wrong?
Our final lesson in this module looks at why AI can make mistakes.