DETR

Overview

DETR, short for "DEtection TRansformer", is an object detector that treats detection as a set prediction problem. A transformer reads image features and outputs a fixed-size set of predictions in parallel. Each prediction is either a box with a class or a "no object" slot. During training, a matching step pairs each real object with exactly one prediction, so the model learns to output each object once. This removes two pieces of hand-built machinery that older detectors needed: anchor boxes and non-maximum suppression.

The appeal of DETR is a clean, end-to-end design. Its early cost was slow training and weaker results on small objects. Later variants fixed much of this. This page explains the idea, the history, the mechanics, a worked example and the ways detection outputs relate to models that learn from raw tables. For faster single-pass detectors see YOLO. For pixel masks see Mask R-CNN and SAM.

What it is

DETR is an encoder-decoder transformer placed on top of a convolutional backbone. It predicts all objects in an image at once.

The core idea

Most older detectors predict thousands of candidate boxes and then clean them up with rules. DETR instead predicts a small, fixed number of objects, for example 100, and learns to make those predictions unique through its training loss. The output is a set, not a ranked pile of candidates.

What it outputs

For each of the N slots, the model returns:

• A class distribution, including a special "no object" class.
• A box, as center x, center y, width and height, scaled to the image size.

Slots that predict "no object" are ignored at inference. The rest are the detections.

What it removes

• Anchor boxes: no preset box shapes to design.
• Non-maximum suppression: no separate duplicate removal step.
• Hand-tuned proposal rules: the model learns where to look.

Everyday analogy

Imagine a meeting with 100 chairs, and each chair holds a person called a "query". Each person has one job: look at the picture and report one object, or say "nothing for me". The rule of the meeting is that two people must never report the same object. If they do, the training coach corrects them. Over time each person specializes in certain kinds of objects and regions. At the end of the meeting you collect the reports from people who did find something.

Where it came from

DETR was introduced in 2020 in the paper "End-to-End Object Detection with Transformers". It came after transformers had transformed language tasks and were beginning to enter vision. The authors showed that a plain encoder-decoder transformer with a set-based loss could match a strong detector baseline on a standard benchmark, with a much simpler pipeline.

Early limitations

• Training took many epochs, far longer than typical detectors.
• Small objects were detected less well, partly because the transformer worked on a low-resolution feature map.
• Attention over every pixel of the feature map was costly.

The follow-up wave

Researchers quickly produced improvements, and several names are worth knowing:

• Deformable DETR (2020) replaced full attention with attention over a small set of sampled points, speeding convergence and helping small objects through multi-scale features.
• Conditional and anchor-guided query designs gave queries a clearer spatial meaning, which sped up learning.
• Denoising training added noisy copies of true boxes as extra queries, which stabilized the matching.
• Later models combined these ideas and reached strong accuracy on public benchmarks.
• Real-time variants were developed so that DETR-style models could compete with single-stage detectors on speed.

Exact numbers and rankings change quickly, so check current papers and your own tests instead of relying on a remembered table.

Wider influence

DETR inspired set-prediction designs for segmentation, tracking, pose estimation and even tasks outside vision. The idea of a fixed number of learned queries that attend to inputs and each produce one output also appears in other architectures, for example latent query designs like Perceiver IO.

How it works

Step 1: Backbone features

The image passes through a convolutional backbone. The result is a feature map at lower resolution than the image, with many channels. Think of it as a coarse but rich grid of descriptions of the picture.

Step 2: Flatten and add position

The grid is flattened into a sequence of tokens, one per cell. Because attention does not know where tokens came from, a positional encoding is added so each token carries its location. This is like numbering seats in a theater so people can talk about "row 3, seat 5".

Step 3: Transformer encoder

A stack of self-attention layers lets every token look at every other token. The encoder builds a global view. A cell on the edge of a bottle can learn from a cell on its cap, and a cell on a shadow can learn that it is not an object. This global reasoning helps separate overlapping or related objects.

Step 4: Object queries

The decoder takes a fixed set of learned vectors called object queries. They start without any image content. Think of them as 100 empty forms. Through decoder layers they gather information from the encoder output.

Step 5: Transformer decoder

Each decoder layer does two things:

• Self-attention among queries, so queries can coordinate and avoid reporting the same object.
• Cross-attention from queries to the encoded image tokens, so each query pulls the evidence it needs.

After several layers, each query holds a summary of one possible object.

Step 6: Prediction heads

A small feed-forward network turns each query vector into a class distribution and a box. Boxes are predicted directly as normalized coordinates, not as offsets to anchors.

Step 7: Bipartite matching during training

This is the heart of DETR. Training images have a handful of true objects, while the model outputs a fixed number of slots. The loss needs a one-to-one pairing between true objects and slots.

The pairing is found with the Hungarian algorithm, which solves an assignment problem at minimum total cost. The cost for matching a true object to a slot combines how well the class matches and how close the boxes are. Analogy: assigning workers to jobs so that the total mismatch is as small as possible, with each worker taking at most one job.

Step 8: Set loss

After matching:

• Matched slots are trained to predict the class and box of their assigned true object.
• Unmatched slots are trained to predict "no object".
• The box loss combines an L1 distance and a generalized IoU term that is scale-aware.
• A classification loss, such as cross entropy, handles labels, often with a lower weight on the "no object" class because it is so common.

Because every true object is matched to exactly one slot, duplicates are penalized. That is why no NMS is needed at inference.

Step 9: Auxiliary losses

Prediction heads are attached after every decoder layer, and each gets its own set loss. This deep supervision helps early layers learn faster and gives stable training.

Step 10: Inference

At inference the model runs once. Slots with a confident class are kept. Boxes are rescaled to the original image. There is no matching step and no NMS. The cost is the backbone plus the transformer.

Why training is slow in the original model

Early in training, the queries have no clear spatial meaning, so matching flips between different assignments from step to step. This unstable matching makes learning slow. Variants stabilize it by giving queries spatial anchors, by adding denoising tasks with fixed assignments, or by using attention that focuses on a few sampling points.

Reading the decoder: what each layer does

Early decoder layers produce rough guesses. The first layer's boxes are often loose and some queries are still unsure. By the last layer, boxes are tight and classes are confident. Because every layer has its own prediction head and loss, you can look at the layer-by-layer outputs and see the refinement. It is like a sculptor who first blocks out the shape and then adds detail with each pass.

What a query specializes in

After training, individual queries tend to favor certain regions, sizes or classes. One query may usually report large objects near the center. Another may report small objects near the top edge. This is not hard-coded. It emerges from the matching loss. Plotting the boxes each query predicts over a whole dataset shows these habits, and it is a useful debugging view.

Attention maps as explanation

Cross-attention weights show which image tokens a query used. For a well-trained model, the attention for a query often highlights the extremities of the object it reports, such as the edges of a limb or the corners of a box. These maps give a rough visual explanation, though they should not be read as a full causal account.

Loss weights in practice

The total loss is a weighted sum of a class term, an L1 box term and a generalized IoU term. The weights balance how much the model cares about each. If class errors dominate, raise the class weight. If boxes are loose, raise the box weights. The weight for the "no object" class is lowered because empty slots are the majority, and without that the model would learn to predict nothing everywhere.

Cost considerations

Self-attention cost grows with the square of the number of tokens. Doubling image side length multiplies tokens by four and attention cost by sixteen in the encoder. That is why efficient encoders and deformable attention matter for high-resolution inputs. Plan memory for the largest image you expect, not the average one.

Variants in plain words

• Deformable attention: each query looks at only a few sampled locations across several scales instead of everything.
• Query initialization from the image: instead of starting from blank forms, use the encoder to pick promising spots.
• Denoising queries: add jittered true boxes as extra tasks so the model learns to refine boxes, which has a stable target.
• One-to-many training auxiliary heads: temporarily allow several predictions per object during training to give richer signal, while keeping one-to-one at inference.
• Efficient encoders: reduce the cost of attention over many tokens.

Segmentation with the same idea

A mask head can be added so each query predicts a mask as well as a box. The same set-based loss then trains instance or panoptic segmentation without separate proposal logic. This shows how general the query idea is.

Key concepts and terms

A worked example

Goal: detect parts on a tray so a robot arm can pick them. Each tray image has up to 20 parts. We want a clean list of unique boxes with no post-processing rules.

Step 1: Set the number of queries

The model needs more slots than the maximum number of objects. With at most 20 parts, a choice of 50 or 100 slots is plenty. More slots cost a little more compute, and unused slots just say "no object".

Step 2: Load a pre-trained model and adapt

The following sketch uses the Hugging Face transformers library. Model names and arguments are placeholders and may change between versions.

from transformers import DetrForObjectDetection, DetrImageProcessor
import torch

processor = DetrImageProcessor.from_pretrained("detr-base-placeholder")
model = DetrForObjectDetection.from_pretrained(
    "detr-base-placeholder",
    num_labels=3,                 # for example: bolt, nut, washer
    ignore_mismatched_sizes=True, # new class head
)

Step 3: Prepare targets

Each training item has an image and a list of annotations with boxes and class ids. The processor converts them to the format the model expects, including normalized box coordinates.

Step 4: Train

model.train()
for batch in loader:
    outputs = model(
        pixel_values=batch["pixel_values"],
        labels=batch["labels"],   # list of dicts: class_labels, boxes
    )
    loss = outputs.loss           # set loss after Hungarian matching
    loss.backward()
    optimizer.step()
    optimizer.zero_grad()

Internally the library matches predictions to targets for each image and combines class, L1 and generalized IoU losses.

Step 5: Inference

model.eval()
with torch.no_grad():
    out = model(pixel_values=pixel_values)

results = processor.post_process_object_detection(
    out, threshold=0.7, target_sizes=[(height, width)]
)[0]
for score, label, box in zip(
    results["scores"], results["labels"], results["boxes"]
):
    print(int(label), round(float(score), 2), box.tolist())

There is a score threshold but no NMS call.

Step 6: A tiny numeric matching example

Suppose the image has two true objects, A and B, and the model has three slots. The cost matrix (lower is better) might be:

The best assignment is A to slot 1 and B to slot 2, with total cost 0.5. Slot 3 is left unmatched and is trained to say "no object". If slots 1 and 2 both liked object A, the matching would still give A to only one of them, and the other would be pushed toward B or toward "no object". That is how duplicates are discouraged.

Step 7: Evaluate

Compute mean average precision at several IoU thresholds on a held-out set. Report results by object size, because small objects were a weak point for early DETR. Compare latency with a single-stage detector on your hardware.

Step 8: Use the output in a table

Each detection becomes a row with image id, class, score and box. For the robot task, additional columns such as grasp point and tray position can be derived. These rows can be logged, joined with process data and modeled with tabular methods.

Strengths and limitations

Strengths

• Simple end-to-end pipeline with no anchors and no NMS.
• Global reasoning through attention can help with context and relations between objects.
• Natural extension to segmentation and other set outputs.
• Later variants reach strong accuracy and good speed.
• Fewer hand-tuned hyperparameters in post-processing.
• Easy to read conceptually: queries in, objects out.

Limitations

• The original model trained slowly and needed long schedules.
• Small objects were weaker, especially without multi-scale features.
• Attention over many tokens is expensive for high-resolution images.
• Fixed number of queries caps how many objects can be found.
• Training instability from matching is real, and variants add complexity to fix it.
• Needs reasonable amounts of data, or strong pre-training, to shine.
• Real-time use requires one of the efficient variants.

Honest note on comparisons

Claims like "DETR beats YOLO" or the reverse depend on the version, the training schedule, the input size and the hardware. Compare on your own data, with the same resolution and a fair training budget.

How it compares with alternatives

Relationship with ViT

ViT is the transformer for image classification. DETR uses a transformer for detection and, in many variants, can use a vision transformer as its backbone instead of a convolutional network. They are building blocks that combine well.

Choosing quickly

• Want the simplest path to real-time boxes: a single-stage detector.
• Want an end-to-end set-based model and are willing to use a modern variant: DETR family.
• Want exact masks: instance segmentation models.
• Want open-vocabulary labels: pair a detector with CLIP style text embeddings.

Role in tabular foundation models

DETR is a vision model, but its core ideas map well onto models that learn from raw tables. See the pages on research, architecture and benchmarks for how a table-native model is built and judged.

Sets, not sequences

A table is a set of rows, and each row is a set of cells. Row order carries no meaning in most tables. DETR is built around set prediction and permutation-friendly attention, with positional information added only where it matters. Tabular models that treat rows as an unordered set of examples, such as those using in-context learning, share this spirit. TabPFN and TabICL read a set of labeled rows and a set of query rows and attend across them.

Learned queries

DETR uses learned queries that pull information from the input. Latent-array designs in the tabular world do something similar: a small fixed set of latent vectors reads from a long list of cells. Perceiver IO is a general example. This keeps cost manageable when tables have many columns, much like DETR queries keep cost fixed regardless of how many objects exist.

Attention across rows and columns

DETR uses attention over spatial tokens. Table models use attention over cells, rows and columns. Axial attention alternates attention along rows and along columns, which is the table equivalent of factorizing a 2D grid attention. SAINT and FT-Transformer use attention over feature tokens, and SAINT adds attention across rows.

Mixed types and no column names

DETR must treat boxes (continuous) and classes (categorical) together with one loss. Table models must treat numeric, categorical and text columns together and cope with columns that have odd names or none at all. Tokenizing each cell with a type-aware embedding is the table analogue of the box and class heads on a query. TabSTAR brings text semantics into this, and XTab studies cross-table pre-training.

Matching and imputation

The "no object" slot in DETR is a learned way to say "nothing here". Tables have an equivalent: a missing cell. Predicting a value or an explicit missing state per cell is the job of masked cell modeling and imputation networks. Both train by hiding part of the input and asking the model to restore it, which is a cousin of DETR denoising queries, where noised boxes are corrected.

Anomaly detection

Detections become tables, and tables can hide anomalies. A drop in the number of detected objects per tray, a change in box size distribution, or a shift in the class mix are signals worth watching. Isolation forest, one-class SVM and the general page on anomaly detection apply to those rolled-up columns.

Pre-training on synthetic data

Tabular foundation models are commonly pre-trained on huge numbers of generated tables so that, at inference, they adapt to a new table from examples alone. See synthetic data generation. Vision detectors are also pre-trained on large, varied image sets before fine-tuning. The shared lesson is that diversity in pre-training buys flexibility later.

Semantic and sector classification

Detector labels are words, and other columns hold free text. Semantic heads that map such text into broader classes, as in semantic sector heads, let a table model reason over labels from many sources. A zero-shot embedding model such as CLIP plays a related role on images.

Continual learning

As the detector is retrained on new classes, the downstream table model should adapt without forgetting. See elastic weight consolidation, learning without forgetting and experience replay.

Limits of the analogy

DETR relies on spatial structure and a one-to-one matching loss. Tabular prediction rarely needs matching, since each row has its own target. The useful transfer is conceptual: set-based thinking, learned queries and denoising as a training signal.

Practical guidance and best practices

Choosing a variant

• For a new project, start from a modern DETR variant with multi-scale features and denoising, not the original.
• Use a pre-trained backbone and detection head, and fine-tune.
• If real-time speed matters, test a real-time DETR variant against a single-stage detector on your hardware.

Data

• Label every object in each image. Missing labels become wrong "no object" targets.
• Keep boxes tight and consistent.
• Use augmentation such as random resize, crop and flip. Make sure box coordinates are transformed with the image.
• Cover the real range of lighting and clutter.

Training

• Expect longer schedules than for single-stage detectors, unless you use a fast-converging variant.
• Use a lower learning rate for the backbone than for the transformer.
• Watch the matching-based loss and the auxiliary losses. A stuck loss often points to label problems.
• Use mixed precision and consider gradient accumulation for memory.
• Use adapters or partial freezing when data is scarce.

Inference

• Choose the score threshold from error costs.
• Set the number of queries above the maximum count of objects you expect.
• Test input sizes. Larger inputs help small objects but cost more.

Evaluation

• Report AP overall and by size.
• Inspect the slots: how many are used per image on average.
• Compare to a simpler baseline to confirm the added complexity pays off.

Deployment

• Export and test numerical agreement.
• Attention operations may need specific runtime support. Check your target.
• Profile memory use, since attention cost grows with token count.

A short checklist for a first DETR project

• Pick a modern variant with multi-scale features.
• Confirm labels are complete for every image.
• Set the number of queries above your expected maximum.
• Train with a lower backbone learning rate.
• Evaluate by object size and by condition.
• Compare against a single-stage baseline on the same hardware.
• Log detections to a table with versions and thresholds.

Diagnosing a stuck run

• Loss does not fall in the first epochs: check the learning rate and that boxes are normalized.
• Many slots predict the same object: train longer, add denoising, or use a variant with better query design.
• All slots predict "no object": check the no-object weight and that targets are not empty by mistake.
• Good training loss but poor validation: check for leakage in the split, augmentation mismatch or too little data.
• Small objects missed: use multi-scale features or a larger input size.

Example of a detection log table

Keeping the slot number can help debugging. Boxes stored in normalized units remain valid if the image is resized. A tabular model downstream can combine these rows with robot cycle times, tray ids and operator shifts.

Using outputs in tables

• Store one row per detection with model version and threshold.
• Keep a stable schema and explicit missing markers.
• Roll up to per-image or per-time-window features before tabular modeling.
• Explain downstream predictions with attribution tools such as SHAP.

Common mistakes

• Using too few queries so some objects can never be found.
• Leaving objects unlabeled, which trains the model to ignore them.
• Expecting the original DETR recipe to train in a few epochs.
• Comparing speed with a single-stage detector on different hardware or precision.
• Forgetting to normalize box coordinates the way the model expects.
• Using a very low score threshold and treating many weak slots as real detections.
• Ignoring small-object performance until late in the project.
• Using a high learning rate for the backbone and damaging pre-trained features.
• Assuming "no NMS" means "no duplicates ever". Poorly trained models can still output near-duplicates.
• Skipping a baseline comparison and adopting a complex model without evidence.

Frequently asked questions

What does DETR stand for?

DETR stands for DEtection TRansformer. It is an object detector built on an encoder-decoder transformer that predicts a fixed-size set of objects directly, instead of scoring thousands of anchor boxes and cleaning them up afterward.

Why does DETR not need non-maximum suppression?

Its training loss uses one-to-one matching, so each true object is assigned to exactly one prediction slot, and other slots are taught to say "no object". Because duplicates are penalized during training, the model learns to output each object once, and no duplicate-removal step is required at inference.

What are object queries?

Object queries are learned vectors that act as slots for possible objects. Through the decoder they attend to the image features and each one becomes a summary of a single candidate object, from which the class and box are predicted. The number of queries sets the maximum number of detections per image.

What is Hungarian matching and why is it used?

It is an algorithm that finds the lowest-cost one-to-one assignment between two groups. In DETR it pairs true objects with predicted slots using a cost based on class match and box closeness. This lets the loss compare an unordered set of predictions with an unordered set of labels.

Why was the original DETR slow to train?

Early in training, the queries have no spatial meaning, so the matching between slots and objects changes a lot from step to step. That instability slows learning. Variants fix this with deformable attention, query designs tied to locations, and denoising tasks that give stable targets.

Is DETR better than YOLO?

Neither is better in general. YOLO-style detectors are often easier and faster to deploy, while DETR-style models offer a cleaner end-to-end design and strong accuracy in modern variants. The right choice depends on your data, latency target and hardware, so measure both on your own validation set.

Can DETR do segmentation?

Yes. Adding a mask head lets each query predict a mask along with its box and class, and the same set-based loss trains it. This extends the approach to instance and panoptic segmentation without separate proposal logic.

Does DETR work for small objects?

The original version was weaker on small objects because it used a low-resolution feature map. Multi-scale features and deformable attention improved this a lot. If small objects matter, pick a variant with multi-scale support and test at a suitable input size.

How does DETR relate to tabular foundation models?

Both use attention over a set of items and rely on learned latent queries or tokens. Detection outputs also become tables of boxes, classes and scores that tabular models can use. The ideas of set-based reasoning, denoising and handling an explicit "nothing here" state echo in table models that impute and detect anomalies.

How many queries should I use?

Use clearly more than the largest number of objects you expect in one image. A common choice is a round number such as 100, but for scenes with at most a few dozen objects a smaller number can work and saves compute.

Glossary

• Auxiliary loss: a loss applied at intermediate decoder layers.
• Bipartite matching: pairing items from two groups one to one.
• Cross-attention: attention from queries to another set of tokens.
• Decoder: the transformer part where queries read from the image encoding.
• Deformable attention: attention over a small set of sampled locations.
• Denoising query: an extra training query built from a noisy true box.
• Encoder: the transformer part that builds global image understanding.
• Generalized IoU: an overlap measure that stays informative for non-overlapping boxes.
• Hungarian algorithm: a method to solve the assignment problem.
• No-object class: a label meaning the slot has no detection.
• Object query: a learned slot vector that becomes a detection.
• Positional encoding: a signal telling the model token locations.
• Self-attention: attention among tokens of the same set.
• Set prediction: predicting an unordered group of outputs.

Summary

DETR recasts object detection as set prediction. A convolutional backbone feeds a transformer encoder and decoder, a fixed set of learned object queries each produce a box and class or a "no object" state, and a one-to-one matching loss teaches them to be unique. This removes anchors and non-maximum suppression and gives a clean, extensible design. The original was slow to train and weaker on small objects, and modern variants have closed much of the gap.

Compare it with YOLO for speed, Mask R-CNN for classic instance masks, and SAM and CLIP for promptable and zero-shot tasks. On the tabular side, detections become rows for models such as TabPFN, and the DETR ideas of sets, learned queries and denoising echo in how table-native models read, impute and monitor data.

Related

It belongs to Vision and multimodal in my stack, under Foundation models and applied ML. Others in the same group: