1. Introduction
In the previous lesson, we learned that embeddings convert information into numerical vectors. These vectors can be compared to identify information with similar meaning.
However, an AI application may need to search thousands, millions, or even billions of vectors. The application therefore needs an efficient way to store vectors and find the most relevant ones.
This is where vector databases and vector search systems become useful.
2. What Is a Vector Database?
A vector database is a data system designed to store, manage, and search numerical vectors efficiently.
A vector database can store an embedding together with information about the original data and metadata associated with it.
A simplified record might contain:
- Vector embedding
- Document identifier
- Text or reference to the original content
- Document category
- Source information
- Access-control metadata
3. Why Store Embeddings?
Generating an embedding for every document during every search would be inefficient.
Instead, an application can generate embeddings when information is added or updated and store those vectors for later retrieval.
When a user submits a query, the application generates an embedding for the query and searches the stored vectors.
4. Basic Vector Database Workflow
A simple workflow looks like this:
- Collect source information.
- Divide large information into useful sections.
- Generate an embedding for each section.
- Store the vectors in a vector database.
- Store useful metadata with each vector.
- Generate an embedding for a user query.
- Search for vectors that are most similar to the query vector.
- Apply appropriate filters and access controls.
- Return relevant information to the application.
5. Vector Search
Vector search finds stored vectors that are mathematically close to a query vector according to a selected similarity or distance method.
For example, a user could search:
How can I change my account password?
The system may retrieve documents discussing password recovery, credential changes, or account access even when the exact search words are not present.
6. Similarity and Distance
Vector databases can use mathematical measures to determine how close vectors are.
Cosine similarity is one common approach. Other approaches include Euclidean distance and dot product, depending on the system and embedding model.
The important concept is that the search system needs a consistent method for comparing vectors.
7. Top-K Search
A vector search often returns the top K most similar results.
For example, K might be 5.
The system would then return the five highest-ranked matching vectors according to the selected search method.
The application can decide what to do with those results. It might display them, pass them to an AI model, or perform another processing step.
8. Why Metadata Matters
Vectors alone are usually not enough for a useful production retrieval system.
Metadata can provide additional information needed for filtering and application logic.
For example, a company knowledge base might store:
- Department
- Document type
- Document version
- Creation date
- Language
- Access level
- Source system
9. Metadata Filtering
Suppose a company has documents from finance, human resources, engineering, and sales.
A user working in engineering should not automatically receive confidential finance documents simply because they have high semantic similarity.
The application can use metadata filters together with vector similarity so that retrieval is restricted to appropriate documents.
10. Vector Search and Security
Security should not depend on vector similarity.
A document can be highly relevant to a query and still be unauthorized for the user.
Authentication and authorization should be enforced by the application and retrieval architecture.
This is especially important when vector databases contain confidential company information, customer data, financial records, or other sensitive material.
11. Vector Indexes
Searching every vector individually can become expensive when a database contains a very large number of vectors.
Vector databases can use specialized indexing techniques to make similarity search faster.
The exact indexing approach depends on the database and application requirements.
Indexing usually involves a trade-off between search speed, memory usage, storage requirements, and retrieval accuracy.
12. Exact Search and Approximate Search
An exact search attempts to compare the query against every relevant vector to identify the mathematically closest results.
This can become expensive at large scale.
Approximate nearest-neighbor search uses specialized techniques to find very good candidates without necessarily examining every vector in the same way.
This can significantly improve search performance for large datasets, although the exact trade-offs depend on the indexing method and configuration.
13. Adding Data to a Vector Database
A typical ingestion process might look like:
Source document → chunking → embedding generation → vector and metadata storage.
The application should maintain a relationship between each vector and its original source.
This makes it possible to retrieve the actual content after a similarity search.
14. Updating Data
Knowledge bases change over time.
When a document changes, the corresponding chunks and embeddings may need to be updated.
Old versions should be handled carefully so that outdated information does not remain available to retrieval systems unintentionally.
15. Deleting Data
Deletion also needs to be considered.
If an original document is removed, its associated vectors should normally be identified and removed or otherwise excluded from retrieval.
Applications should have a reliable relationship between source records and vector records so that updates and deletions can be managed correctly.
16. Vector Databases and Traditional Databases
Traditional relational databases are excellent for structured information such as customers, invoices, orders, accounts, and transactions.
Vector databases specialize in efficient similarity search over numerical representations.
These technologies do not necessarily compete with each other.
A production AI application may use both.
17. Combining Relational Data and Vector Search
Consider an application containing customer support information.
A relational database might store customer accounts, ticket numbers, dates, and permissions.
A vector system might store embeddings representing support documentation.
The application can combine these systems so that structured business information and semantic retrieval work together.
18. Vector Database Does Not Replace the Source Data
An embedding is a representation of information rather than a replacement for the original information.
Applications should normally maintain a reliable reference to the original document, database record, or content source.
This is important for displaying source material, auditing retrieval, updating records, and investigating incorrect results.
19. Retrieval Thresholds
An application may decide that a similarity result is not relevant enough to use.
For example, the application could define a similarity threshold and reject results below that threshold.
However, there is no universal threshold that works for every embedding model and every dataset.
Thresholds should be tested against real application data.
20. Multiple Retrieval Signals
Some applications combine vector similarity with other search signals.
For example, a system might use semantic similarity together with keyword matching, metadata filters, document freshness, or business relevance.
This can improve retrieval quality when exact terms are important as well as semantic meaning.
21. Reranking
A retrieval system can first select a group of potentially relevant results and then use another method to rank those results more carefully.
This process is commonly called reranking.
Reranking can help when simple vector similarity does not provide the best ordering of retrieved information.
22. Performance Considerations
Production vector search involves more than retrieval accuracy.
Developers should consider:
- Search latency
- Database size
- Memory requirements
- Storage costs
- Embedding generation costs
- Concurrent users
- Update frequency
- Availability requirements
23. Cost Considerations
Costs can come from several parts of the system.
Embedding generation may have a model usage cost. Storage requires infrastructure. Vector indexes consume resources. Search requests also require compute capacity.
Good system design therefore considers the complete retrieval pipeline rather than only the cost of the AI model.
24. Multilingual Data
Applications that support multiple languages should evaluate whether the selected embedding model performs well across those languages.
Semantic relationships between languages can vary depending on the model and dataset.
Real application testing is important when multilingual retrieval is required.
25. Monitoring Vector Search
Production systems should monitor retrieval behavior.
Useful measurements can include:
- Search latency
- Number of results returned
- Similarity scores
- Empty-result frequency
- Retrieval errors
- User feedback
- Source freshness
Monitoring helps developers identify retrieval problems that may not be visible from application errors alone.
26. Evaluating Retrieval Quality
A vector database can operate correctly while the overall retrieval system still produces poor results.
Developers should create representative test queries and evaluate whether the expected documents are being retrieved.
Evaluation should include common questions, difficult questions, ambiguous queries, and cases where no suitable information exists.
27. Common Beginner Mistakes
- Assuming vector similarity always identifies the correct answer
- Ignoring metadata
- Ignoring access-control requirements
- Using outdated embeddings after source documents change
- Storing vectors without references to source information
- Choosing a similarity threshold without testing
- Ignoring retrieval latency and infrastructure costs
- Assuming one vector database configuration works for every workload
28. A Simple Architecture
A basic semantic retrieval architecture can be represented as:
User query → application → query embedding → vector search → metadata filtering → relevant documents → application response.
In an AI assistant, the retrieved information can later be supplied to a generative AI model as context.
29. Vector Databases in Modern AI Applications
Vector databases are particularly useful when AI applications need to search large collections of external information.
Examples include:
- Company knowledge assistants
- Semantic document search
- Product recommendation systems
- Support knowledge retrieval
- Research systems
- Content discovery systems
30. Developer Mental Model
A useful mental model is:
Embedding = numerical representation.
Vector database = system for storing and searching those representations.
Metadata = information that helps filter and manage results.
Application logic = controls how retrieved information is used safely.
31. Where This Module Goes Next
The next lesson introduces Retrieval-Augmented Generation, commonly called RAG. RAG combines retrieval with generative AI so that an AI application can use relevant external information when producing a response.
Conclusion
Vector databases provide infrastructure for storing and searching embeddings efficiently. They are an important building block for semantic search and many AI retrieval applications.
A reliable vector search system requires more than vectors. It also needs useful metadata, source references, access controls, appropriate indexing, evaluation, monitoring, and a process for keeping information current.