What Is a Neural Network?
We have already learned that machine learning models learn patterns from data.
But how can a model learn complicated patterns such as recognizing an object in an image, understanding language, or identifying relationships between many pieces of information?
One important answer is the neural network.
A neural network is a type of machine learning model inspired loosely by the way biological nervous systems are organized.
The word "inspired" is important. Artificial neural networks are mathematical and computational systems. They are not digital copies of the human brain.
The Basic Idea
A neural network contains many connected computational units that work together.
These units are commonly called neurons or nodes.
A simplified neural network can be imagined as a collection of layers:
Input Layer → Hidden Layers → Output Layer
Information enters through the input layer, passes through one or more hidden layers, and eventually produces an output.
What Is an Artificial Neuron?
An artificial neuron receives numerical inputs, combines them using learned values called weights, applies a mathematical function, and produces an output.
You do not need to understand the mathematics yet.
The important idea is:
Inputs + Weights → Calculation → Output
A neuron can therefore be thought of as a small computational unit that transforms information.
Why Are Weights Important?
Not every input is equally important for a particular prediction.
Weights allow the network to give different importance to different inputs.
Imagine a very simple system trying to estimate whether a photograph contains a particular object.
Some visual features may be more useful than others.
The weights influence how strongly different pieces of information affect the calculation.
During training, the network adjusts these weights so that its predictions become better.
Connections Between Neurons
Neurons in one layer can be connected to neurons in another layer.
Each connection can have an associated weight.
As information moves through the network, each layer transforms the information before passing it to the next layer.
With enough neurons and layers, a neural network can represent very complicated relationships.
Input Layer
The input layer receives the information provided to the network.
The form of the input depends on the task.
For an image system, the input may represent pixels or features extracted from an image.
For a language model, the input ultimately becomes a numerical representation of text.
For a prediction system, the input might contain numerical information such as age, income, location, or previous activity.
Hidden Layers
The layers between the input and output are commonly called hidden layers.
These layers transform information and allow the network to learn increasingly useful representations.
For example, in an image recognition system, early layers may learn relatively simple visual patterns, while later layers may combine those patterns into more complex structures.
This is a simplified explanation, but it provides a useful intuition for why multiple layers can be powerful.
Output Layer
The output layer produces the result of the network.
The form of the output depends on the task.
- A classification system might output probabilities for different categories.
- A prediction system might output a numerical value.
- A language model can produce information used to predict the next token.
- A generative system can ultimately produce new content.
What Does a Neural Network Learn?
A neural network learns useful patterns by adjusting its internal parameters during training.
The weights are among the most important of these parameters.
During training, the network makes predictions, measures how far those predictions are from the desired results, and adjusts its parameters to reduce the error.
This process happens repeatedly across many training examples.
A Simple Training Example
Imagine training a neural network to recognize whether a photograph contains a cat.
The training process might look like this:
- The network receives a training image.
- The network produces a prediction.
- The prediction is compared with the known answer.
- A loss value represents how far the prediction was from the desired result.
- The network adjusts its parameters.
- The process is repeated with many examples.
Over time, the network can become better at the task.
What Is an Activation Function?
Neural networks commonly use mathematical functions called activation functions.
An activation function helps determine how the output of a neuron is transformed before information continues through the network.
Activation functions are important because they allow neural networks to model complex relationships rather than behaving like one large simple mathematical calculation.
There are many different activation functions, and the choice depends on the architecture and task.
Why Multiple Layers?
A single simple layer may be able to represent only certain types of relationships.
Adding multiple layers allows a network to perform a sequence of transformations.
This can help the network learn increasingly complex patterns.
A useful simplified intuition is:
Simple patterns → combinations of patterns → more complex representations → useful output
The actual operation of modern neural networks is much more complicated, but this mental model is helpful for beginners.
What Is Deep Learning?
Deep learning refers broadly to machine learning methods that use neural networks with multiple layers.
The word "deep" refers to the depth of the network, meaning that information can pass through many layers of learned transformations.
Deep learning has become extremely important because it can learn powerful representations from large amounts of data.
Why Deep Learning Became So Powerful
Several developments helped make deep learning practical at large scale.
- Large datasets
- More powerful computing hardware
- Improved neural network architectures
- Better training techniques
- Specialized hardware for large computations
These developments allowed researchers and companies to train increasingly large neural networks.
Neural Networks and Images
Neural networks became particularly successful in computer vision.
Systems can learn useful visual representations from large collections of images.
For example, a network may learn patterns related to edges, textures, shapes, and increasingly complex combinations of visual features.
This allows neural networks to perform tasks such as image classification, object detection, and image generation.
Neural Networks and Language
Neural networks can also process language.
Modern language models use neural network architectures that can process sequences of tokens and learn statistical relationships between them.
This allows them to perform tasks such as text generation, summarization, translation, classification, and question answering.
Later in this module, we will look more closely at how modern language models work.
Neural Networks Do Not Think Like Humans
It is common to hear people say that a neural network "thinks" or "understands" something.
These words can be useful as simplified descriptions, but they should not be taken too literally.
A neural network performs mathematical computations using learned parameters.
Modern AI systems can produce remarkably sophisticated results, but that does not mean they work exactly like human brains.
A Useful Mental Model
For now, imagine a neural network as a large collection of connected mathematical units.
Each unit performs a small transformation.
The connections have learned weights.
Many layers work together to transform the input into an output.
During training, the network adjusts its parameters so that its outputs become more useful for the task.
From Neurons to Large AI Models
The basic neural network idea can scale enormously.
Modern AI systems may contain very large numbers of parameters and many layers.
Large language models are examples of neural-network-based systems that have been trained on huge amounts of data.
The basic concepts we are learning now provide the foundation for understanding those systems.
The Big Picture
We can connect this lesson with what we learned earlier:
Data → Training → Neural Network → Learned Parameters → Inference → Output
Data provides examples.
Training adjusts the network parameters.
The trained network can then process new input during inference.
The result may be a prediction, classification, recommendation, or generated content.
What You Should Remember
A neural network is a machine learning model made from connected computational units.
Neurons receive and transform numerical information.
Weights control how strongly different inputs influence calculations.
Layers allow the network to perform multiple transformations.
Training adjusts the networks parameters so that it becomes better at its task.
Deep learning uses neural networks with multiple layers.
What Comes Next?
We now understand the basic idea behind neural networks.
The next question is especially interesting:
How can a neural network learn to work with human language?
That leads us to large language models, the technology behind many modern AI assistants.