Generative AI Is Not Just Text
When people hear the term Generative AI, they often think about chatbots that generate text.
Text generation is important, but Generative AI can work with many different forms of information.
Modern systems can generate or transform:
- Text
- Images
- Audio
- Video
- Computer code
Some advanced systems can work with several of these forms together. These are often described as multimodal systems.
Text Generation
Text generation is one of the most widely used forms of Generative AI.
Language models can generate explanations, summaries, stories, emails, reports, conversations, translations and many other forms of text.
A user might provide a prompt such as:
"Explain machine learning using a simple example."
The model processes the instruction and generates a response based on its learned patterns and the context provided.
Why Text Generation Is So Flexible
Language is used in many different activities.
Because a language model can work with natural-language instructions, the same underlying technology can potentially support many tasks.
A user can ask for a summary, a table, a list, an explanation, a story, a translation or computer code.
The model does not necessarily need a separate traditional program for every one of these tasks.
Image Generation
Generative AI can also create images from text descriptions or other inputs.
For example:
"A futuristic city with electric vehicles at sunset."
An image-generation system can use this description to create a new image.
Image-generation models use specialized techniques to represent and generate visual information.
Image Editing
Generative AI can also modify existing images.
A system may be asked to remove an object, change the background, extend an image, modify a visual style, or make other changes.
The input can therefore be an existing image rather than only a written prompt.
Image Generation Has Limitations
Generated images can contain visual mistakes.
For example, a model may produce unusual hands, inconsistent objects, incorrect text inside an image, or other visual details that do not make sense.
Improving image quality does not eliminate the need for human review.
Audio Generation
Generative AI can also work with sound.
One important application is text-to-speech.
A text-to-speech system converts written text into generated speech.
Modern systems can produce increasingly natural-sounding voices with control over characteristics such as speaking style, pace, and tone.
Voice Transformation
AI can also be used to transform or modify recorded speech.
Depending on the technology, a system may alter characteristics of a voice or transform speech into another generated voice.
These capabilities can be useful in media production and accessibility, but they also create important concerns around consent, impersonation, and misuse.
Music Generation
Some Generative AI systems can create music from descriptions or other inputs.
A user might request a particular mood, style, instrumentation, or structure.
The system can then generate an audio result based on the capabilities of the model.
Video Generation
Generative AI can also create video.
A system might generate a short scene from a text description, transform existing footage, or create other forms of synthetic video.
Video generation is technically challenging because the system needs to produce coherent visual information across multiple frames over time.
Why Video Is More Difficult
A single image only needs to be visually coherent at one moment.
A video contains many frames that need to work together.
Objects should generally maintain consistent appearance and movement.
The generated scene should also remain coherent as the camera or subjects move.
These requirements make video generation more computationally demanding and technically challenging.
Code Generation
Computer code is another important form of generated content.
AI coding systems can generate functions, scripts, database queries, documentation, tests and other programming-related material.
For example, a developer might ask an AI system to:
"Write a SQL query that lists all published lessons in a course."
The model can generate a possible query.
As with other AI-generated content, the result should be reviewed and tested.
Multimodal AI
A system that can work with multiple types of information is often called multimodal AI.
For example, a multimodal system might accept text and an image together.
A user could upload a photograph and ask:
"Describe what is happening in this image."
The system needs to process the visual information and the text instruction together.
Text and Images Together
Multimodal systems can combine different types of information in a single task.
For example, a user might provide a chart as an image and ask the system to explain the main trend.
The model or application needs to connect the visual input with the language instruction.
Text, Audio and Video Together
More advanced AI systems can work across several modalities.
A system might receive spoken instructions, analyze an image, and respond using text or speech.
This allows AI applications to interact with users in more natural ways.
Generation vs Transformation
Not every Generative AI task starts with nothing.
A model may generate completely new content or transform existing content.
Examples include:
- Generating a new image from a description.
- Changing the style of an existing image.
- Turning text into speech.
- Summarizing an existing document.
- Transforming one piece of code into another form.
Both generation and transformation can be useful AI capabilities.
Different Models for Different Tasks
There is no requirement that one model must perform every possible Generative AI task.
Different models can be designed or trained for different purposes.
One model may specialize in text, another in images, another in speech, and another in video.
AI applications can also combine several models into one larger workflow.
AI Applications Can Combine Models
Imagine an application that receives a user's spoken request.
A speech-recognition model could convert the speech into text.
A language model could interpret the request and generate an answer.
A text-to-speech system could then convert the answer back into spoken audio.
The user experiences one AI assistant, but several components may be working together behind the scenes.
Quality Depends on the Task
Generative AI quality can vary significantly depending on the task.
A model may be excellent at generating a short summary but less reliable at producing precise technical calculations.
An image model may create an impressive scene but struggle with small text inside an image.
A voice system may sound natural but still require careful controls to prevent misuse.
Human Review Still Matters
Generated media should be reviewed before it is used in situations where errors matter.
Text can contain factual mistakes.
Images can contain visual inconsistencies.
Audio can contain pronunciation or factual problems.
Video can contain visual inconsistencies or misleading content.
Generated code can contain bugs or security problems.
Authenticity and Synthetic Media
Generative AI makes it increasingly easy to create synthetic media.
This can be useful for entertainment, education, accessibility, design and many other purposes.
However, synthetic media can also be used to create misleading or deceptive material.
Users should therefore consider the source and authenticity of important media.
Responsible Use
Using Generative AI responsibly includes considering issues such as privacy, consent, copyright, misinformation, impersonation, security and potential harm.
The appropriate safeguards depend on the application and the type of content being generated.
The Big Picture
Generative AI is a broad family of technologies.
Text, images, audio, video and code can all be generated or transformed using AI models.
Multimodal systems can combine several types of information and make AI applications more flexible.
Behind the user interface, an application may use one model or several models working together.
What You Should Remember
- Generative AI can create more than text.
- Images, audio, video and code can also be generated or transformed.
- Different media types require different technical approaches.
- Multimodal AI can work with multiple forms of information.
- One AI application may combine several specialized models.
- Generated media can contain errors or inconsistencies.
- Synthetic media creates both useful opportunities and risks.
- Human review and responsible use remain important.
What Comes Next?
We have now seen what Generative AI can create.
In the next lesson, we will move from the technology itself to the practical question:
Where is Generative AI actually being used?