ColBERT

Overview

ColBERT is a neural retrieval model that represents each text as a set of token-level vectors instead of one single vector. The name stands for contextualized late interaction over BERT. A query is turned into one small vector per token, and so is each document. To score a document, ColBERT matches every query token to its most similar document token and adds up those best matches. This scoring rule is called MaxSim, and the idea of comparing at the token level after encoding is called late interaction.

The design sits between two well known approaches. A single-vector dense retriever is fast but compresses a whole passage into one point, which loses detail. A cross-encoder reads query and document together and is accurate, but it is too slow to run on a whole collection. ColBERT keeps most of the accuracy of joint reading while allowing document vectors to be computed ahead of time and searched quickly. The price is a much larger index, because every token has its own vector. ColBERT is used for high-accuracy retrieval and as a strong first stage or middle stage in RAG pipelines. See also BM25, cross-encoder reranking and hybrid search.

What it is

ColBERT is a retrieval architecture and a family of trained models. It is built on a transformer encoder from the BERT family. The encoder turns each token of a text into a contextual vector, meaning that the vector for a word depends on the words around it. ColBERT keeps all of these vectors, after a linear layer that shrinks them to a small size, and stores them in an index.

Three parts

• The encoder. A transformer produces one vector per token for both queries and documents. Query and document use the same model, with small marker tokens that tell it which is which.
• The late interaction scorer. For each query token vector, find the highest similarity with any document token vector. Add up these maxima over the query tokens.
• The index and search procedure. Document token vectors are stored in an approximate nearest neighbour structure. Search finds candidate documents by looking up vectors near each query token, then scores candidates fully.

An everyday analogy

Compare two ways of judging whether a recipe book has what you need. In the single-vector method, you ask a librarian for a one-line summary of each book and compare your request with that summary. Details are lost. In the cross-encoder method, you sit and read your request next to every book, which is accurate but takes forever. In the ColBERT method, the librarian has prepared an index card for every important word in every book, noting its meaning in context. You then take each word in your request and find the best matching card in the book. If every word of your request finds a good match, the book is a good fit. You do the matching quickly because the cards were prepared in advance.

Why "late"?

In a cross-encoder, the query and the document interact early, inside the model, from the first layer. In a single-vector model, they interact only at the very end through one dot product. In ColBERT, each side is encoded alone, as in a bi-encoder, and the interaction happens late, after encoding, through a cheap operation over many vectors. This is what allows pre-computation of document vectors.

Where it came from

ColBERT was introduced in a 2020 paper by Omar Khattab and Matei Zaharia, titled "ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERT". It was a response to a problem that arose after BERT-based rerankers appeared in 2019. Those rerankers gave big accuracy gains but took a long time per query. The authors asked whether the quality of joint reading could be kept while moving most of the work offline.

Their answer was to keep token-level vectors and use the MaxSim operator. The paper reported that ColBERT came close to the accuracy of BERT-based rerankers on passage ranking while being far faster, and that it could be used for end-to-end retrieval, not just reranking.

A follow-up, ColBERTv2, appeared in 2021. It addressed the large index size by compressing token vectors with a residual scheme, in which each vector is stored as a reference to a nearby centroid plus a small correction. It also improved training with distillation from a stronger ranker and with better hard negatives. Later, the PLAID engine from 2022 sped up search by pruning candidates at the centroid level before full scoring.

Since then, the multi-vector idea has spread. Libraries and vector stores added support for multi-vector search, and related models apply late interaction to other kinds of data, such as page images in document retrieval. Tools and exact features change quickly, so check current documentation for the version you plan to use. For the research background that connects retrieval models with other model families, see research.

How it works

Step 1: Encode the query

The query text is tokenized and passed through the encoder. A special marker token tells the model that this is a query. In the original design, short queries are padded to a fixed length with mask tokens, which lets the model fill in extra, useful query vectors. This is called query augmentation, and it acts like a soft way of expanding the query with related terms.

Each token vector goes through a linear layer that reduces its size, and then it is normalised to unit length. The original model used a small dimension, which keeps storage lower than the full size of the encoder.

Step 2: Encode the documents (offline)

Every document or passage goes through the same encoder, with a document marker. The result is a matrix with one row per token. Some systems drop vectors for punctuation, since those carry little meaning. All the document matrices are computed once, ahead of time, and stored.

Step 3: Score with MaxSim

Let the query have tokens q1 to qm and the document have tokens d1 to dn. All vectors are normalised, so the dot product equals cosine similarity.

score(Q, D) = sum over i of ( max over j of  dot(q_i, d_j) )

For each query token, find the single most similar document token. Sum these best similarities. A document gets a high score when every query token finds a close partner somewhere in it. A query token with no good partner pulls the total down.

Step 4: Find candidates fast

Scoring every document in full would be too slow on a big collection. ColBERT uses a two-part search.

1. Candidate generation. For each query token vector, look up the nearest document token vectors in an approximate index. The documents that own those token vectors form the candidate set.
2. Full scoring. Load the token vectors of each candidate and compute the exact MaxSim score. Sort and return the top k.

Because the first part works with vectors of single tokens, it behaves a lot like a smart, soft version of keyword matching. The second part is the precise late interaction step.

Step 5: Compression and pruning (ColBERTv2 and after)

Storing a vector per token is expensive. Compression ideas include:

• Centroids and residuals. Cluster all token vectors. Store each vector as the id of its centroid plus a coarse, low-bit version of the difference. This can reduce storage to a small fraction of the raw size.
• Centroid-based pruning. Before loading full vectors, use centroid scores to drop candidates that cannot reach the top.
• Token pruning. Drop low-value tokens from documents.
• Smaller dimensions. Reduce the vector size where quality allows.

How the model is trained

• Triples or lists. Training uses a query, a relevant passage and one or more non-relevant passages.
• Loss. A ranking loss encourages the MaxSim score of the relevant passage to exceed that of the others.
• Hard negatives. Negatives are mined from a first-stage retriever, so the model learns to separate near misses.
• Distillation. Scores from a cross-encoder act as soft targets, passing along its fine judgment.
• Domain adaptation. Fine-tuning on in-domain queries and passages usually helps, as with any retrieval model.

A mental picture of MaxSim

Imagine the query "cheap flights to Lisbon in March". The tokens "cheap", "flights", "Lisbon" and "March" each look for the best partner in a document. A page about airline fares to Portugal may have strong partners for "flights" and "Lisbon", a decent partner for "cheap" (such as "low-cost") and a weak partner for "March" unless dates are mentioned. A page about Lisbon hotels may match "Lisbon" and "March" but not "flights". The sum shows which page covers more of the query, and it does so without forcing the whole page into a single point.

Related ideas and variants

The late interaction idea has grown into a small family of methods. Names and details vary, so treat this as a map and check current sources for specifics.

• Compressed and pruned multi-vector indexes. Work after the first release focused on cutting storage by quantising vectors, sharing centroids and dropping tokens that add little.
• Faster candidate search. Engines such as PLAID reduce the number of documents that need full scoring by checking centroid-level scores first.
• Multi-vector models for other inputs. The same scoring rule has been applied to images of document pages, where patches of the page play the role of tokens. The query tokens then match against page patches.
• Distilled and smaller encoders. Smaller backbones reduce encoding cost and token vector size, which helps on limited hardware.
• Multilingual versions. Models trained on several languages allow one index to serve queries in more than one language, with quality that varies by language pair.
• Hybrid scoring. Some systems add a single-vector score and a keyword score to the MaxSim score, with weights tuned on labelled queries.

Where late interaction tends to help most

Some query types gain more than others from token-level matching.

And some types gain little.

A note on fairness and cost

Since multi-vector search costs more storage and compute, the choice should follow a clear need. Teams sometimes adopt it because it is new, and later find that a well tuned hybrid search with a reranker gave the same user benefit. Write down the quality target and the budget before you begin, measure against it, and keep the simplest setup that meets the target.

Key concepts and terms

A worked example

A tiny numeric MaxSim

To see the arithmetic, use very small two-dimensional vectors. Suppose the query has two tokens and the document has three tokens. All vectors are of length one.

The dot products form this grid.

For q1, the maximum is 0.900 (with d1). For q2, the maximum is 0.900 (with d2). The MaxSim score is 0.900 + 0.900 = 1.800.

A single-vector model would have averaged the document tokens into one vector and compared it with a single query vector, which blurs "airline" and "Portugal" into the middle. In the late interaction view, each query token finds its own best partner, so both ideas are kept.

A short Python version of the scoring step

import numpy as np

def maxsim(query_vecs, doc_vecs):
    # query_vecs: (m, d), doc_vecs: (n, d), both L2-normalised
    sim = query_vecs @ doc_vecs.T          # (m, n) similarity grid
    return float(sim.max(axis=1).sum())    # best match per query token, then sum

q = np.array([[1.0, 0.0], [0.0, 1.0]])
d = np.array([[0.9, 0.436], [0.436, 0.9], [0.7, 0.7]])
print(round(maxsim(q, d), 3))              # 1.8

This is the core of the model. A real system adds the encoder, the index and the candidate search around it.

Using a library in a RAG pipeline

Several open source libraries wrap the model, the index and the search. The exact function names change between releases, so the snippet below uses placeholder names to show the shape of the workflow.

from example_colbert_lib import Indexer, Searcher   # placeholder package

# 1. Build an index from passages (offline)
passages = load_passages("example.internal/passages.jsonl")
Indexer(model="example-org/example-colbert", index_name="docs").index(passages)

# 2. Search at query time
searcher = Searcher(index_name="docs")
hits = searcher.search("how do I rotate an access key?", k=20)

# 3. Hand the top passages to the language model
context = "\n\n".join(h.text for h in hits[:5])
prompt = f"Answer using only this context:\n{context}\n\nQuestion: ..."

In practice you would also add metadata filters, chunking rules and a fallback to keyword search.

What to measure

1. Retrieval quality: recall at 10, 50 and 100, plus nDCG at 10 on your own queries.
2. Index size on disk and in memory, per million passages.
3. Query latency at the median and at the 95th percentile.
4. Indexing time and the cost of adding new documents.
5. End-to-end answer quality, using a RAG evaluation tool such as RAGAS or TruLens.

Run the same measurements for a BM25 baseline and a single-vector dense baseline. The comparison shows whether the extra storage pays off for your data.

Sizing the index: a back-of-envelope example

Storage is the main practical question, so it pays to estimate it before building anything. The figures below are illustrative assumptions, not measurements of any specific model.

Assume a collection of one million passages, with about 100 tokens each after tokenization. That gives 100 million token vectors.

Two points stand out. Raw token storage is large, close to ten times a single-vector index in this example. Good compression brings it close to the single-vector size. So the question is not only "how big is the index" but "which compression setting keeps quality high enough". Always compare quality at the same storage budget.

Memory is a separate concern from disk. Many systems keep the centroid table in memory and read compressed vectors from disk for candidates only. Test your latency with a cold cache, because the first queries after a restart are often the slowest.

Deployment patterns

Retrieval and rerank in one engine

Some engines can run late interaction scoring as a ranking stage over candidates found by another method. For example, a first stage finds the top few hundred with BM25 or a single vector, and a second stage scores those with MaxSim using stored token vectors. This avoids the need for a dedicated candidate-generation index at token level, and it reduces the storage and search work, at the cost of depending on the first stage for recall.

Dedicated multi-vector index

A dedicated index finds candidates directly from token vectors. This gives the best recall because it does not rely on another method to find the right documents, but it needs the full set of token structures and a specialised search engine.

Offline encoding service

Document encoding is a batch job and benefits from GPUs. Query encoding is a small online job and can often run on a modest GPU or even on CPU for short queries. Separate the two services so that a large re-indexing run does not slow live traffic.

Versioning

Store the model name, the version, the tokenizer and the vector settings with the index. If any of these change, you must rebuild. A simple naming rule for indexes, such as including a model tag and a date, prevents accidental mixing.

Evaluating late interaction fairly

A fair comparison needs the same data, the same queries and the same cut-offs for every method. A few tips:

• Report recall at several depths (10, 50, 100), because a retriever that is great at 100 may be average at 10.
• Report quality for query groups, such as short keyword queries, long questions and queries with rare names. Late interaction often shows its edge on multi-part questions and may show less on exact code lookups.
• Report latency and storage next to quality.
• Include a baseline with a reranker on top, since a reranker can lift a cheaper retriever a long way.
• Test on documents the model could not have seen in training, to measure generalisation.

Reading the interaction grid

The grid of similarities between query tokens and document tokens is more than an internal step. It is a built-in explanation. For a given query and a retrieved passage, you can list, for each query token, the document token that matched best and the similarity. This makes debugging easier.

• If an important query word has a low best match, the passage is probably missing that idea.
• If a stop word has a high match but a key word has a low one, the passage is matching on filler, which may point to a training or tokenization issue.
• If one document token is the best match for many query tokens, the passage may be repeating a single idea.

Teams sometimes show these matches in a search interface as highlighted words, which helps users trust the results.

Troubleshooting guide

Strengths and limitations

Strengths

• Fine-grained matching. Each query word can find its own best partner, which handles multi-topic queries and detailed questions better than one vector.
• Strong accuracy. Late interaction models often rank among the best retrievers on standard passage ranking tasks, and they can match or approach rerankers at lower cost.
• Pre-computed documents. Unlike a cross-encoder, document vectors are built once and reused.
• Good out-of-domain behaviour. Token-level matching tends to transfer better to new data than a single compressed vector, though this should be tested for your own case.
• Interpretability. The similarity grid shows which query words matched which document words, so you can explain a score.
• Soft keyword behaviour. It captures exact words and also close meanings, like a bridge between lexical and dense search.

Limitations

• Large index. One vector per token means storage far above single-vector models. Compression helps but does not remove the gap.
• More complex search. Candidate generation, decompression and full scoring are more involved than a single nearest neighbour lookup.
• Higher memory and I/O demands. Loading token vectors for candidates takes time.
• Harder to update. Adding many documents may need re-clustering of centroids if the data drifts a lot.
• Tooling maturity. Support in databases and managed services is less universal than for single-vector search, though it is growing.
• Length limits. The encoder has a maximum length, so long documents need chunking.
• Not a full replacement for reranking. For the very best precision, many systems still add a cross-encoder at the end.

How it compares with alternatives

Choosing between them

• Start with BM25 and a single-vector dense retriever. They are simple, cheap and good.
• Add ColBERT when you have measured that single-vector retrieval misses detail, and when you can afford the index size.
• Add a cross-encoder when precision at the very top is the main need.
• Use hybrid fusion to combine lists from several retrievers, since their errors differ.

Where the indexes live

Single-vector search typically uses graph or quantised indexes such as HNSW, FAISS, IVF-PQ or DiskANN. Multi-vector search can build on the same ideas, applying them at the token level, and several vector databases such as Vespa, Qdrant and Weaviate have added or discussed multi-vector support. Check the current feature list of the system you use before you design around it.

Role in tabular foundation models

ColBERT is a text retrieval model. Tabular foundation models work on tables. The two connect through the idea of token-level, structure-aware matching, and through real tasks in table workflows where text must be matched.

The tabular side in brief

A tabular foundation model is pre-trained on many tables, often generated synthetically, so that it can predict on new tables without per-task training. It reads rows and columns, mixes numbers, categories, dates and text, tolerates missing values, and does not depend on meaningful column names. A few labelled rows given as context let it adapt, which is in-context learning. Examples are TabPFN, TabICL and TabSTAR, and the architecture page describes how such models are built.

Parallels in design

• Cells as tokens. A table can be viewed as a grid of cell tokens. Late interaction suggests a way to compare two rows: match each cell of one row to its best partner in the other, and add up. Some table models use attention over cells in a similar spirit, so that each cell attends to the cells that matter most, rather than to one summary vector.
• Order independence. MaxSim does not depend on the order of document tokens. Likewise, rows in a table and columns in many tables have no natural order, and tabular models are often built to be insensitive to that order.
• Mixed granularity. ColBERT keeps token detail instead of squeezing everything into one point. Tabular models similarly keep cell-level detail before forming row-level predictions.

These are analogies, not a claim that the architectures are the same.

Concrete uses around tables

• Selecting context rows for in-context learning. If rows are serialised to text, a late interaction retriever can find the rows that match the query row on the most relevant fields. The tabular model then uses those rows as labelled context. This is a design choice that should be compared with simple nearest neighbour or random selection.
• Matching text columns to categories. For sector or semantic classification, a row's description can be matched against category descriptions with token-level matching, which handles long, multi-topic descriptions better than a single vector. See semantic sector heads.
• Entity resolution. Names and addresses have many small parts. Token-level matching handles partial overlaps, reordered words and extra tokens gracefully. See deduplication.
• Imputation support. To fill a missing text-valued cell, retrieve similar rows by their other text fields, then let a model propose a value. Pair this with learned imputation methods such as masked cell modeling.
• Anomaly detection triage. After a tabular model flags a row, retrieval over past incident notes and runbooks gives context to the analyst. See anomaly detection.
• Searching data catalogs and documentation. Questions about tables, columns and definitions are text retrieval problems. A multi-vector retriever handles questions that mention several columns or concepts at once. See data catalog and data lineage.

Limits of the connection

ColBERT does not model numeric values, dates or missing cells on its own. It also does not learn from a few labelled rows in context. It finds and matches text. The tabular foundation model does the numeric and structural reasoning. A sound design keeps each in its own lane and evaluates the combination on the end task. For results on standard tabular tasks, see benchmarks, and for the research landscape see research.

Practical guidance and best practices

Decide whether you need it

• Measure a BM25 baseline and a good single-vector dense baseline first.
• Look at failure cases. If misses come from queries with several distinct parts, late interaction is a likely help.
• Estimate storage. Multiply the number of tokens in your collection by the stored size per token vector after compression. Make sure that it fits your budget.

Chunk and prepare text

• Use passages of a moderate size, a few hundred words, so that the token count per passage stays under the encoder limit.
• Include titles and headings in the passage text, or add a short context line. See contextual retrieval.
• Clean boilerplate before indexing, as it adds tokens without value.

Manage index size

• Use the compressed index format that your library offers.
• Consider pruning punctuation and very frequent tokens.
• Store token vectors on fast disk and keep centroid structures in memory.
• Test smaller vector dimensions and fewer bits per value, and measure the quality change.

Combine with other stages

• Use ColBERT alongside BM25 in hybrid search if exact terms matter.
• Add a cross-encoder reranker on the top 20 to 50 results if precision at rank one matters.
• Rewrite vague queries before retrieval. See query rewriting.

Adapt to your domain

• Gather real queries and labels, or generate synthetic ones. See synthetic data generation.
• Fine-tune with hard negatives from your own data.
• Keep a held-out test set, and compare against the unadapted model.

Operate it

• Plan re-indexing. If your data changes a lot, schedule periodic rebuilds so that the clustering stays representative.
• Monitor latency and recall over time.
• Keep the model version tied to the index version. A model change needs a full re-encode of the documents.
• Evaluate the full RAG chain with RAGAS, TruLens or DeepEval.

A short checklist before adopting ColBERT

• Do you have a labelled or synthetic query set that reflects real use?
• Have you measured BM25, a single-vector retriever and a reranker on that set?
• Do the failures look like missing detail in multi-part queries?
• Is the index size, after compression, within your storage budget?
• Can your serving stack run the candidate search and the full scoring within the latency target?
• Do you have a plan for re-encoding when the model or the data changes?
• Will you keep a keyword path for exact terms?

If the answer to most of these is yes, a pilot is justified. Start with a subset of the collection, compare it with the current system on the same queries, and expand only when the gain is clear.

Common mistakes

• Assuming it is always better. A single-vector model with a reranker can match ColBERT at lower index cost on many tasks. Test it.
• Ignoring the index budget. Teams often discover late that the index is many times larger than a single-vector index.
• Mixing model versions. Queries encoded with one model and documents with another give meaningless scores.
• Skipping normalisation. MaxSim assumes comparable vector lengths. Use the library defaults.
• Indexing huge documents. Beyond the encoder limit, text is silently cut. Chunk instead.
• No keyword safety net. For exact codes and rare names, keep BM25 or a hybrid mode.
• Tuning on too few queries. Small test sets give noisy answers about which settings are better.
• Forgetting updates. After large changes in the corpus, an old centroid set can lower recall.
• Using it on non-text data without care. Raw numeric table cells do not carry the meaning that a text encoder expects. Serialise sensibly, or use a model made for tables.
• Evaluating retrieval only. A better retrieval score does not always mean better answers. Check the full pipeline.

Mistakes in pipeline design

• Skipping a lexical path. Late interaction softens keyword matching but does not replace it for identifiers.
• Sending too many passages to the language model. Retrieval quality is wasted if the prompt is filled with weak passages. Cut by score or by rank.
• Not logging matches. Without stored candidate lists and scores, you cannot learn from bad answers.
• Forgetting access control. Token vectors are derived from document text, so they need the same permissions as the documents. Filter by rights before returning results.

Frequently asked questions

What does ColBERT stand for?

ColBERT stands for contextualized late interaction over BERT. Contextualized means each token vector depends on its surrounding words. Late interaction means the query and document are compared after they have been encoded separately. BERT is the transformer family that the first version was built on, though the idea can use other encoders.

How is ColBERT different from a normal dense retriever?

A normal dense retriever compresses each text into one vector and compares two vectors. ColBERT keeps one vector per token and compares them with MaxSim, so each query token finds its best match in the document. This keeps more detail and usually improves accuracy, but the index is much larger.

What is MaxSim?

MaxSim is the scoring rule of ColBERT. For each query token, you find the highest similarity between its vector and any document token vector. Then you add those highest values over all query tokens. A document scores well if every part of the query has a close match somewhere in it.

Is ColBERT faster than a cross-encoder?

For retrieval over a large collection, yes, by a wide margin. Document vectors are computed in advance, and candidate search uses approximate nearest neighbour lookups, so ColBERT does not need to run a large model on each query and document pair. A cross-encoder is still more precise on a small shortlist, which is why the two are often combined.

Why is the ColBERT index so large?

Because it stores a vector for every token in every document, while a single-vector model stores one vector per document. Compression methods based on centroids and low-bit residuals reduce the size a lot, but multi-vector indexes are still larger than single-vector ones. You trade storage for accuracy and detail.

Can ColBERT replace BM25?

Not always. ColBERT captures meaning and close word matches, but exact identifiers, rare names and codes are still well served by keyword search. Many production systems keep BM25 in a hybrid setup and use ColBERT or a dense model alongside it. Test the combination on your own queries.

Does ColBERT work for languages other than English?

Yes, if you use an encoder trained for the language. Multilingual encoders exist, and quality varies by language and by the amount of training data. Test on your own language and domain, and consider fine-tuning with local examples.

Can I use ColBERT inside a vector database?

Some vector databases and search engines support multi-vector fields or late interaction scoring, and others do not. Support is changing fast, so read the current documentation of the system you plan to use. If native support is missing, you can use a dedicated library and keep your database for metadata and filters.

Is ColBERT useful for tabular data?

It helps with the text parts of tabular workflows, such as matching free-text fields to categories, resolving duplicate records, and searching documentation. It does not handle numeric reasoning or missing values. A tabular foundation model is the right tool for prediction on tables, and the two can work together in one pipeline.

How do I know whether the larger index is worth it?

Run a controlled test on your queries. Compare a BM25 baseline, a single-vector dense baseline, the same with a reranker, and ColBERT. Record quality, latency and storage for each. If ColBERT gives a gain that a cheaper setup cannot match, and the storage cost is acceptable, it is worth it. Otherwise, keep the simpler stack.

Glossary

• Approximate nearest neighbour search: Fast search that returns vectors close to a query, with a small chance of missing the very closest.
• Bi-encoder: A model that encodes two texts independently.
• Candidate generation: The first part of search, which finds possible documents.
• Centroid: The centre point of a group of similar vectors.
• Chunk: A passage of a document stored as one unit.
• Contextual embedding: A vector whose value depends on the surrounding words.
• Cross-encoder: A model that reads two texts together.
• Distillation: Training a model to copy the scores of a stronger one.
• Dot product: A measure of how aligned two vectors are.
• Hard negative: A wrong example that looks like a right one.
• Late interaction: Token-level comparison after separate encoding.
• MaxSim: The scoring rule that sums the best match for each query token.
• Multi-vector retrieval: Retrieval where each text has several vectors.
• nDCG: A ranking metric that rewards good results near the top.
• Normalisation: Scaling a vector to length one.
• Query augmentation: Adding mask tokens to a query so the model fills in extra vectors.
• Residual compression: Storing the difference between a vector and its centroid.
• Sparse retrieval: Retrieval based on word matching, as in BM25.
• Token: A unit of text such as a word or word piece.

Summary

ColBERT is a late-interaction retrieval model. It keeps one vector per token and scores a document by summing, for each query token, the best match among the document tokens. This gives detail close to that of a cross-encoder while still allowing document vectors to be prepared in advance. The main cost is a much larger index, which later versions reduced with compression and smarter search.

In practice, ColBERT works best next to BM25, in hybrid search, and with an optional cross-encoder at the end. For tabular foundation models, it supports the text parts of the workflow: choosing context rows, matching text to categories, resolving duplicates and finding documentation, while the tabular model handles numbers, categories and missing values. To learn more about how those models are built and measured, see research, architecture and benchmarks.

Related

It belongs to RAG in my stack, under LLMs, retrieval, and evaluation. Others in the same group: