ChromaDB: a simple open-source embedding database

Overview

ChromaDB, usually just called Chroma, is an open-source database for storing and searching embeddings. Its main appeal is simplicity. You can install it with one package, create a collection in a few lines of Python or JavaScript, add documents, and query them by meaning, all without running a separate server. If you do not supply vectors, Chroma creates them for you with a default embedding model.

Chroma is released under the Apache 2.0 licence. It can run inside your application process, as a single-node server, or as a managed service called Chroma Cloud, which uses a distributed architecture. This range makes it popular for notebooks, prototypes and small to medium applications, especially retrieval-augmented generation (RAG) systems, where developers want to get a working search layer quickly.

Think of Chroma as a well-organised notebook you carry in your bag, compared with a building full of filing cabinets. You can open it anywhere, add a page in seconds and find what you wrote by flicking to the right section. When your notes outgrow the notebook, you can move to the filing room, Chroma's server or cloud modes, without learning a completely different system.

What it is

Part	Role
Client	The library you call from Python or JavaScript, in-process or connecting to a server
Collection	A named set of records sharing an embedding function
Record	An ID, a document or other content, an embedding and optional metadata
Embedding function	Turns text, or other data, into vectors; a default model is provided
Vector index	HNSW on single nodes; a different, disk-friendly index in the distributed system
Metadata store	Holds metadata and supports filters, using SQLite on single nodes
Filters	Conditions on metadata (where) and on document text (where_document)

Chroma's search options have grown beyond plain vector search to include metadata filtering, full-text and regular expression matching on documents, and support for sparse and hybrid search in newer releases.

Where it came from

Chroma was founded in 2022 by Jeff Huber and Anton Troynikov. It appeared just as large language models created strong demand for easy retrieval tools, and it quickly became a common default in tutorials and frameworks such as LangChain and LlamaIndex.

Early versions focused on the Python developer experience. Storage changed over time: from version 0.4 in 2023, the single-node store used SQLite for metadata alongside an HNSW index for vectors. The team later rewrote the core in Rust, and the 1.0 release in 2025 brought large performance improvements while keeping the familiar client API. The company also built Chroma Cloud, a managed, distributed service that separates storage and compute and keeps data in object storage. Because the project moves quickly, check the official documentation for current APIs and limits.

How it works

Step 1: Choose a client mode

In-memory client: data lives only while the process runs. Good for tests.
Persistent client: data is saved to a local folder.
HTTP client: connects to a Chroma server or Chroma Cloud.

Step 2: Create a collection

A collection groups records. You can set its embedding function and its distance measure, such as cosine, inner product or squared L2 distance. All vectors in a collection must come from the same model and have the same dimension.

Step 3: Add records

Each record has a unique ID, and usually a document text and metadata such as source, date or language. If you pass only documents, Chroma's embedding function computes vectors. The default uses a small sentence-embedding model run locally. You can also pass precomputed embeddings.

Step 4: Index

On a single node, vectors go into an HNSW graph for fast approximate search, and metadata goes into SQLite. Writes are logged before indexing, so data is not lost if the process stops.

Step 5: Query

You pass query text or a query vector, the number of results, and optional filters. Chroma returns IDs, documents, metadata and distances.

import chromadb

client = chromadb.PersistentClient(path="./chroma_data")
docs = client.get_or_create_collection("help_articles")
docs.add(
    ids=["a1", "a2"],
    documents=["How to reset your password", "Shipping times for Europe"],
    metadatas=[{"lang": "en"}, {"lang": "en"}],
)
results = docs.query(query_texts=["I forgot my login"], n_results=1, where={"lang": "en"})
print(results["documents"])

Step 6: Update and delete

Records can be updated, upserted or deleted by ID or filter. The index updates accordingly.

Step 7: Scale out

When a single node is not enough, you can move to Chroma Cloud, whose distributed system uses a write-ahead log, background compaction and query workers over object storage.

Key concepts and terms

Term	Meaning
Collection	A group of records with one embedding space
Embedding function	The component that converts content to vectors
PersistentClient	A client that saves data to local disk
HttpClient	A client that talks to a Chroma server
Metadata filter	A condition on record fields, passed as where
Document filter	A condition on the stored text, passed as where_document
Distance function	How closeness is measured: cosine, inner product or L2
Upsert	Insert a record, or update it if the ID already exists
HNSW	Graph index used for fast approximate search on single nodes
n_results	How many nearest records to return

A worked example

An illustrative two-person team builds an internal assistant over 1,200 policy documents.

They split documents into chunks of about 400 words, giving roughly 9,000 chunks.
In a notebook they create a persistent Chroma collection and add chunks with metadata: department, document title and last-updated date.
They start with the default embedding function to test the idea in an afternoon. Later they switch to a stronger embedding model with 768 dimensions and re-import all chunks into a new collection.
Vectors take about 9,000 × 768 × 4 bytes, roughly 28 MB. Everything fits comfortably on a laptop.
A query such as "how many days of parental leave do I get?" returns the top 8 chunks filtered to department = "HR" in a few milliseconds.
A language model writes the answer from those chunks, citing document titles.
When the tool is shared with 300 colleagues, they move from the embedded client to a Chroma server in a container, keeping the same code apart from the client line.

The figures are illustrative. The pattern shows Chroma's strength: very fast setup, with a path to a server when needed.

Strengths and limitations

Strengths:

Very quick to start, with a small, readable API.
Runs in-process, so no infrastructure is needed for prototypes.
Default embedding function for zero-configuration tests.
Python and JavaScript clients, with wide framework support.
Open source under Apache 2.0, plus a managed cloud option.
Metadata and document text filters in the same query.

Limitations:

Single-node deployments are limited by one machine's memory and disk.
Fewer index types and tuning options than databases built for very large scale, such as Milvus.
The default embedding model is small and may not suit production quality needs.
Advanced features such as replication and high availability depend on the cloud or distributed setup.
Rapid development means APIs and behaviour have changed between versions.

How it compares with alternatives

System	Best known for	Good fit	Less suited to
Chroma	Simplicity, embedded use	Prototypes, small to medium RAG apps	Very large self-hosted clusters
FAISS	Fast similarity search library	Custom pipelines, research	Metadata, persistence, updates out of the box
pgvector	Vectors in PostgreSQL	Apps already on Postgres	Very large vector-only workloads
Qdrant	Filtering and quantisation	Production search with rich filters	Zero-setup notebooks
Weaviate	Built-in hybrid search and modules	Feature-rich search services	Minimal footprints

Practical guidance and best practices

Use the in-memory client for tests and the persistent or HTTP client for anything you need to keep.
Choose your embedding model deliberately before production, and store its name in collection metadata.
Set the distance function to match the embedding model, often cosine.
Keep useful metadata on each record for filtering, such as source, date and access group.
Batch inserts when loading large datasets.
Build a small labelled query set to compare embedding models and chunk sizes.
Back up the persistence directory or use the managed service for important data.
Pin the Chroma version and read migration notes before upgrading.

Common mistakes

Using the in-memory client and losing all data on restart.
Keeping the default embedding model in production without testing alternatives.
Mixing embeddings from different models in one collection.
Storing very large documents as single records instead of chunks.
Exposing a local Chroma server to the network without authentication.
Assuming a laptop-scale setup will handle millions of users without redesign.

Frequently asked questions

Is Chroma free?

The open-source database is free under the Apache 2.0 licence. Chroma Cloud, the managed service, is a paid product with its own pricing.

Does Chroma create embeddings for me?

Yes. If you pass documents without vectors, it uses the collection's embedding function, which has a default local model. You can swap in other providers or pass your own vectors.

Can I use Chroma in production?

Yes, for many small and medium workloads, especially with a server deployment or Chroma Cloud. For very large or highly available self-hosted systems, compare it with databases built for distributed operation.

How do I filter results?

Use the where argument for metadata conditions, such as language or date ranges, and where_document for conditions on the stored text, such as containing a word.

Does Chroma support hybrid search?

Recent versions add full-text and sparse vector search, which can be combined with dense search. Check the documentation for your version and deployment mode. See hybrid search for the general idea.

What languages can I use?

The main clients are Python and JavaScript or TypeScript, and other clients exist. Any language can talk to a Chroma server over its HTTP API.

Glossary

Embedding: a vector representing the meaning of content.
Vector store: a system that stores and searches embeddings.
Chunk: a piece of a longer document stored as one record.
Persistence: saving data so it survives restarts.
Approximate search: fast search that may miss a few true neighbours.
Metadata: extra fields stored with each record.
Object storage: cheap, durable storage for files, used by cloud systems.
Compaction: background merging of new writes into the index.

Summary

Chroma is an open-source embedding database, founded in 2022 by Jeff Huber and Anton Troynikov, known for a very simple API and the ability to run inside your application. It stores records with documents, vectors and metadata, creates embeddings by default if you do not supply them, and supports filtered similarity search. It is a strong choice for prototypes and small to medium RAG systems, with a server mode and a managed cloud for larger needs. For very large self-hosted workloads, compare it with databases designed for distributed scale.

It belongs to Vector databases in my stack, under Generative AI and LLMs. Others in the same group:

Weaviate Milvus Qdrant Vespa FAISS pgvector