Domain adaptation

Overview

Domain adaptation is a set of techniques that help a model trained on one data distribution, the source domain, work well on a different but related distribution, the target domain. The task stays the same. The data changes. A model that predicts machine failure on one factory line, for example, may see sensors with different ranges, different sampling rates and different failure mixes on another line. Domain adaptation tries to close that gap without starting from scratch.

The short answer to "when do I need it" is: whenever the data you will see in production does not look like the data you trained on, and you cannot cheaply collect a full new labelled set. It is a close relative of transfer learning and sits next to few-shot learning and meta-learning. For tables, it matters a great deal, because every new plant, city, customer or time period tends to produce a table with its own quirks.

What it is

A domain is a pair: a feature space and a probability distribution over it. Two domains differ when the features differ, when the distribution over the same features differs, or when the relation between features and labels differs. Domain adaptation is the study of how to use a labelled source domain, plus some information about the target domain, to build a good predictor for the target.

The usual notation uses P_S(x, y) for the source and P_T(x, y) for the target. Training on P_S and testing on P_T breaks the basic assumption of machine learning, which is that training and test data come from the same distribution. When that assumption fails, accuracy falls, calibration breaks and confidence scores lie.

Settings

Domain adaptation comes in several settings. The setting depends on what you have from the target domain.

Kinds of shift

Not all shift is the same. Naming the kind of shift tells you which method is likely to work.

• Covariate shift. The distribution of inputs changes, P(x), but the rule linking inputs to labels, P(y|x), stays the same. Example: a credit model trained on one region's applicants is used in a region with a different income spread.
• Label shift (also called prior shift). The mix of classes changes, P(y), but each class looks the same. Example: a fault detector moves from a plant with 1 percent faults to one with 5 percent faults.
• Concept shift (also called concept drift). The rule P(y|x) itself changes. Example: customers react differently to the same offer after a market change. This is the hardest kind, and no method can fix it without target labels.
• Feature space shift. The columns differ. One table has 40 columns, another has 55, and only 30 overlap. This is common in real tables.
• Joint shift. Several of the above happen at once, which is the normal case in practice.

An everyday analogy helps. Think of a chef who learned to cook in a kitchen with a gas stove. Moving to a kitchen with an induction hob is covariate shift: same recipes, different equipment response. Moving to a country where guests prefer spicier food is label or concept shift: the same dish is now judged differently. Domain adaptation is the chef's process of adjusting to the new kitchen without retraining from the first day of culinary school.

Where it came from

The idea has roots in classical statistics. Work on sample selection bias and on importance weighting in the 1970s and 1980s looked at what happens when a sample is drawn with a different probability than the population. In machine learning, the topic grew through natural language processing in the 2000s. A model for product reviews in one category, such as books, degraded on another, such as kitchen appliances. This was the first widely studied benchmark of the field.

Theory followed. Work from the late 2000s and early 2010s gave generalization bounds for the target error. The bounds say, in plain words, that target error is limited by three things: how well the model does on the source, how different the two domains look to a classifier, and how well a single model could do on both domains at once. This result explains why many methods try to make source and target features indistinguishable.

Deep learning added new tools. The "Domain-Adversarial Training of Neural Networks" paper (2015) introduced a gradient reversal layer that trains a feature extractor to confuse a domain classifier. Correlation alignment methods, often called CORAL, matched second-order statistics of the two domains. Methods based on maximum mean discrepancy matched distributions in a kernel space. Later work added image-to-image translation, self-training with pseudo-labels, and test-time adaptation. In recent years the field has also connected to self-supervised learning and to large pre-trained models, which are often more robust to shift out of the box.

How it works

This section walks through the main families of methods step by step. Most real projects combine two or three of them.

Step 1: measure the gap

Before adapting, find out how large the gap is and where it lives. A simple and strong test is the domain classifier test.

1. Pool source and target features.
2. Label source rows 0 and target rows 1.
3. Train a classifier to tell them apart, with cross-validation.
4. If the area under the curve is near 0.5, the domains look alike. If it is near 1.0, they differ a lot.
5. Inspect feature importance of the domain classifier to see which columns drive the difference.

Think of it as asking a stranger to sort photos from two holidays. If they cannot sort them, the holidays looked the same. If they sort them instantly, something obvious differs, such as snow or sand. The columns the stranger relies on are your shifted columns.

Step 2: choose a family of methods

Instance reweighting. Give each source row a weight so that the weighted source looks like the target. The ideal weight is w(x) = P_T(x) / P_S(x). The domain classifier gives an estimate: if the classifier outputs p(target|x), then w(x) ≈ p / (1 - p) times a constant that accounts for the sample sizes. Training with these weights makes the model care more about source rows that resemble the target. It is a good fit for covariate shift. Everyday analogy: a teacher who knows the exam will focus on chapter 5 asks students to practise chapter 5 problems more often.

Feature alignment. Transform the features so that source and target statistics match. Simple versions standardize each domain separately. CORAL matches the covariance matrices. Maximum mean discrepancy methods minimize a distance between the mean embeddings of the two domains. It works well when the shift is a smooth change of scale, rotation or correlation structure. Analogy: converting prices from two currencies into one common currency before comparing them.

Adversarial learning. A feature extractor feeds two heads: a label predictor and a domain classifier. The extractor is trained to help the label predictor and to fool the domain classifier. The result is a representation that carries label information but not domain identity. Analogy: a referee who must judge a match well without knowing which team wears which colour.

Self-training and pseudo-labelling. Train on the source, predict on the target, keep the confident predictions as pseudo-labels, retrain on both, and repeat. Care is needed, because early mistakes can reinforce themselves. Confidence thresholds, class balancing and consistency under data augmentation all help.

Fine-tuning. If you have some target labels, start from the source model and continue training on them with a small learning rate. Freezing early layers and tuning only later ones, or using adapters, keeps the model from forgetting what it knew. This is the same idea as transfer learning.

Label shift correction. If only class priors changed, estimate the new priors from the target predictions, using a confusion matrix of the source model, and reweight the output probabilities. This is cheap and often enough for fault and fraud settings.

Test-time adaptation. Update only normalization statistics, or a tiny set of parameters, using unlabelled target batches during deployment. It needs no source data at deployment time.

Domain generalization. Train on several source domains, with techniques that encourage invariant features, so that the model is ready for an unseen target. Methods include invariant risk minimization, group-robust optimization and data augmentation that simulates shift.

Step 3: validate honestly

Validation is the hard part. In the unsupervised setting you have no target labels, so ordinary validation on the source overstates quality. Safer choices:

• Hold out a small labelled target sample, even 50 to 200 rows, just for model selection and reporting.
• Use importance-weighted validation, scoring source validation rows with the same weights used for training.
• Use a "reverse validation" check: train a model on pseudo-labelled target data and test whether it recovers the source labels.
• Report results per slice of the shift, not only the average.

Step 4: monitor after deployment

Shift does not stop at launch. Track input distributions, prediction distributions and, when labels arrive, accuracy over time. Alert when a drift score passes a threshold. See also schema drift detection for the structural side of the problem.

Method deep dives

Importance weighting in more detail

The weight formula comes from a simple identity. The expected loss on the target can be written as an expected loss on the source, multiplied by the density ratio.

• Target risk: the average loss over P_T(x).
• Source risk, reweighted: the average of w(x) × loss over P_S(x).
• The two match when w(x) = P_T(x) / P_S(x) and the labelling rule is the same.

In practice you never know the densities. You estimate the ratio instead. There are three common ways.

• A probabilistic domain classifier, as in the example above.
• Direct ratio estimation, such as least-squares or kernel mean matching, which solves for weights that match feature means in a kernel space.
• Binning, where you split a key feature into buckets and use the ratio of bucket frequencies.

Binning is crude but easy to audit. A reviewer can read the table and agree or disagree with each weight.

Feature alignment in more detail

Alignment methods change the features, not the weights. The simplest is a per-domain z-score: subtract the domain mean and divide by the domain standard deviation. It removes level and scale differences between sites. It also removes real differences, so use it only when the level itself is not a signal.

CORAL goes one step further. It whitens the source features, then re-colours them with the target covariance. After that, the correlation structure of the two domains matches. The method has no tunable parameters beyond a small regularizer on the covariance, which is why it is a popular baseline.

Maximum mean discrepancy methods add a penalty to the training loss. The penalty grows when the average embedding of source rows differs from the average embedding of target rows. The network learns features that keep the label signal while pulling the two averages together.

Adversarial methods in more detail

The adversarial setup has three parts: a feature extractor, a label head and a domain head. The domain head tries to name the domain from the features. A gradient reversal layer sits between the extractor and the domain head. In the forward pass it does nothing. In the backward pass it multiplies the gradient by a negative number. So the extractor is pushed to make the domain head fail.

A few habits keep this stable.

• Ramp up the strength of the domain loss slowly, starting near zero.
• Use a small domain head, so it cannot win trivially.
• Track both losses. If the domain loss goes to zero, the features still carry domain identity. If it sits at chance, the alignment is working.
• Check that label accuracy on the source does not drop.

Self-training in more detail

A basic loop looks like this.

1. Train on labelled source rows.
2. Predict target rows and keep those with confidence above a threshold.
3. Add them to the training set with their predicted labels.
4. Retrain, and repeat a few rounds.

The main risk is confirmation bias: wrong pseudo-labels train the model to repeat the same errors. Mitigations include a rising threshold, class-balanced selection so that one class does not take over, agreement between two differently seeded models, and consistency checks under small input noise.

Label shift correction in more detail

For label shift, you can estimate new class priors without target labels. Take the source model's confusion matrix C, where each entry is the chance of predicting class i when the truth is class j. Count how often the model predicts each class on the target, giving a vector q. Solve C × π = q for the target priors π. Then multiply each predicted probability by π_j / source_prior_j and renormalize. This is cheap, needs only unlabelled target data, and works well when the class-conditional features really are unchanged.

A second example: aligning two sites with per-site statistics

Imagine two sites that log vibration in different units. Site A reports millimetres per second. Site B reports inches per second. A model trained on A will misread B.

A raw reading of 0.20 at site B looks tiny to a model trained on A. After a per-site z-score, the reading becomes (0.20 − 0.16) / 0.04 = 1.0, which is one standard deviation above normal. In site A the same score would be a reading of 5.0, a clearly elevated value. The model now sees a comparable picture.

def site_zscore(df, cols, site_col="site"):
    g = df.groupby(site_col)[cols]
    return (df[cols] - g.transform("mean")) / g.transform("std").replace(0, 1)

train = site_zscore(frame, ["vibration", "temperature", "pressure"])

This works when each site's normal operating level is a nuisance. It fails when one site really is more dangerous than another, because the z-score hides that. In that case add the site-level statistic back as an explicit feature, so the model can decide.

Choosing a method at a glance

Key concepts and terms

A worked example

Suppose a model predicts whether a pump will fail within 30 days. The source site has two years of labelled history. A new target site has features but only a few labelled failures. The target pumps run at higher pressure on average. This is covariate shift, so importance weighting is a good first move.

Code

import numpy as np
import pandas as pd
from sklearn.ensemble import GradientBoostingClassifier
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import cross_val_predict
from sklearn.metrics import roc_auc_score

# Placeholder files on an internal path.
src = pd.read_parquet("example.internal/data/site_a.parquet")
tgt = pd.read_parquet("example.internal/data/site_b.parquet")

features = [c for c in src.columns if c not in ("pump_id", "failed_30d")]
Xs, ys = src[features], src["failed_30d"]
Xt = tgt[features]

# 1. Domain classifier: source = 0, target = 1.
Xd = pd.concat([Xs, Xt], axis=0)
yd = np.r_[np.zeros(len(Xs)), np.ones(len(Xt))]
dom = LogisticRegression(max_iter=1000)
p = cross_val_predict(dom, Xd, yd, cv=5, method="predict_proba")[:, 1]
print("domain AUC:", round(roc_auc_score(yd, p), 3))

# 2. Importance weights for the source rows.
p_src = p[: len(Xs)]
prior = len(Xs) / len(Xt)
w = prior * p_src / (1 - p_src)
w = np.clip(w, 0.0, np.quantile(w, 0.99))   # clip extreme weights
w = w / w.mean()                             # normalise to mean 1

# 3. Train with and without weights.
base = GradientBoostingClassifier().fit(Xs, ys)
adapted = GradientBoostingClassifier().fit(Xs, ys, sample_weight=w)

# 4. Compare on the small labelled target sample.
tl = tgt.dropna(subset=["failed_30d"])
for name, m in [("baseline", base), ("weighted", adapted)]:
    auc = roc_auc_score(tl["failed_30d"], m.predict_proba(tl[features])[:, 1])
    print(name, "target AUC:", round(auc, 3))

What each step does

1. The domain classifier tells you how different the sites are. A domain AUC of 0.5 would mean no shift. An AUC of 0.9 means the sites are easy to tell apart.
2. The weights come from the odds that a source row looks like a target row. A source row with high pressure gets a larger weight, because high pressure is common at the target.
3. Clipping and normalizing the weights keep a few rows from dominating the loss. Very large weights make training unstable.
4. The comparison uses the small labelled target sample, which is the only honest yardstick.

A tiny numeric walk-through

Say the source has 1,000 rows and the target has 500. For one source row, the domain classifier outputs a probability of 0.5 that it belongs to the target.

• Odds: 0.5 / 0.5 = 1.
• Prior correction: 1,000 / 500 = 2.
• Weight: 2 × 1 = 2.

Another row has a probability of 0.1. Its odds are 0.1 / 0.9, about 0.11, and the weight is about 0.22. The first row counts nine times more than the second in the loss. The model therefore leans toward the region of feature space where the target lives.

Diagnosing support problems

Weighting only works where source and target overlap. If the target has pressures that the source never saw, no weight can create data from nothing. Check for this with a quick summary:

-- Share of target rows outside the source range, per feature.
SELECT
  SUM(CASE WHEN t.pressure < s.p_min OR t.pressure > s.p_max THEN 1 ELSE 0 END) * 1.0
    / COUNT(*) AS share_outside_source
FROM example_db.target_pumps AS t
CROSS JOIN (
  SELECT MIN(pressure) AS p_min, MAX(pressure) AS p_max
  FROM example_db.source_pumps
) AS s;

If that share is large, you need target labels, feature engineering that removes absolute levels (for example, ratios or within-site z-scores), or a different model class.

Strengths and limitations

Strengths

• Reduces the labelling bill for every new site, customer or period.
• Often gives large gains for little code, especially with weighting and normalization.
• Offers diagnostics, such as the domain classifier, that explain why a model fails.
• Combines well with fine-tuning, self-supervised learning and active learning.
• Many methods work on any model, including gradient boosted trees such as XGBoost and LightGBM.

Limitations

• No method fixes concept shift without target labels.
• Weights can have high variance, and a few rows may dominate.
• Adversarial training is fragile and sensitive to hyperparameters.
• Unsupervised model selection is hard, because there is no target metric to optimize.
• Negative transfer is real: bad adaptation can be worse than none.
• Feature-space mismatch, with different columns, needs extra work such as mapping or embedding.
• Fairness and calibration can shift along with accuracy and must be checked separately.

How it compares with alternatives

A practical reading of the table: if target labels are cheap, label some and fine-tune. If they are costly, combine adaptation with active learning to spend the labelling budget on the rows that matter most.

Role in tabular foundation models

Tabular foundation models are models that learn from raw tables, across many tables, and then make predictions on new tables with little or no training. Domain adaptation shows up in them in several concrete ways.

Shift is the normal condition for tables

Tables from different sources differ in row count, column count, column order, units, missing-value patterns, class balance and categorical vocabularies. A model that is useful across tables must treat these differences as expected, not as exceptions. Pre-training on a wide range of tables, real or synthetic, is a form of domain generalization. The aim is to make the learned behaviour robust to many table styles. Models such as TabPFN, TabICL and TabSTAR, and the ideas behind XTab, all lean on this.

In-context learning as adaptation without gradients

A tabular foundation model with in-context learning receives labelled rows of the target table as part of its input, then predicts for new rows. It does not update weights. The labelled rows act as a target-domain sample. In this view, the context is the adaptation: the model infers the rule for this particular table from the examples at hand. This is domain adaptation by conditioning, and it is why such models can adjust to a new plant or customer in one forward pass. Related ideas appear in meta-learning and few-shot learning.

Mixed types and no column names

Raw tables mix numbers, categories, dates and text. Column names may be missing, meaningless or in another language. Feature-space shift is therefore common. A tabular model needs a representation that works without relying on names. Approaches include per-column encoders that look at the value distribution of each column, learned column embeddings, and permutation-invariant attention over rows and columns, as described on the architecture page. When names do exist, a text encoder can use them as a bridge between domains, so that "temp" in one table lands near "temperature" in another.

Missing values and imputation

Missingness patterns differ between sources. One site might log a sensor every second, another only on request. A model that handles missing cells natively, for example by training on masked cells as in masked cell modeling, is more robust to this shift. Imputation networks benefit directly from domain adaptation ideas, since the mechanism that hides values often changes with the domain: missing completely at random in one table, missing because of a threshold in another.

Pre-training data and the simulated shift

Synthetic table generators produce many "domains" in a controlled way, by varying feature distributions, noise and causal structure. Training across them teaches the model to cope with shift in the way domain generalization methods do. See synthetic data generation for how such data can be made. Real tables add what simulators miss, such as odd encodings and long tails. Mixing both is a common approach.

Continual learning and drift

Over time a deployed model meets new domains and new periods. Updating on the new data risks forgetting the old. Continual learning methods, such as elastic weight consolidation and learning without forgetting, address the forgetting side. Domain adaptation addresses the new-distribution side. Together they let a foundation model grow across domains without a full retrain.

Anomaly detection and sector classification

For anomaly detection, a shift in the data is not always an anomaly in the process. A new plant may run at a different normal level. Adapting the notion of normal to the target, for instance by re-estimating baselines per site, cuts false alarms. For semantic or sector classification of tables and columns, which uses a classifier head on top of table embeddings (see semantic sector heads), domain shift appears as vocabulary change: the same sector described with different words, codes and formats. Adaptation or in-context examples from the target make the classification stable.

What this means for evaluation

Benchmarks for tabular models should include cross-domain tests, not just random splits within one dataset. See the benchmarks and research pages for how evaluation is framed. A model that wins on random splits but loses under site-held-out splits has not been shown to handle shift.

Practical guidance and best practices

1. Name the shift first. Run the domain classifier test and look at the top drifting features before choosing a method.
2. Start simple. Per-domain standardization, ratio features and importance weighting solve a large share of tabular problems.
3. Prefer features that travel. Ratios, ranks, within-group z-scores and counts transfer better than absolute levels and raw identifiers.
4. Remove identifiers. Columns such as site code or customer id let the model memorize the domain. Drop them unless they are real signals.
5. Keep a small labelled target set for validation. Even 100 rows are worth more than any proxy metric.
6. Clip and normalize weights. Look at the effective sample size, (Σw)² / Σw². If it is tiny, the weights are too extreme.
7. Check calibration on the target. Probabilities often drift even when ranking stays fine. Recalibrate with a small labelled sample, using isotonic or Platt scaling.
8. Combine with active learning. Ask for labels where the model is least certain on target rows. See active learning and human in the loop.
9. Version the data. Record which source and target snapshots a model saw, so you can explain a change later. A tracking tool such as MLflow fits here.
10. Monitor continuously. Set drift alerts on key features and on the prediction distribution.

A small configuration sketch

adaptation:
  detect:
    domain_classifier: logistic_regression
    alert_auc_above: 0.75
  method: importance_weighting
  weights:
    clip_quantile: 0.99
    normalize_mean: 1.0
  features:
    drop: [site_code, customer_id]
    add: [pressure_over_site_mean, temp_rank_within_site]
  validation:
    target_labelled_rows: 150
    report_slices: [site, season]
monitoring:
  schedule: daily
  host: monitor.example.internal

Typical scenarios and what to do

A new factory line. Sensors are similar but calibrated differently. Use per-line scaling, ratio features and a small labelled sample for validation. Link the work to predictive maintenance practice.

A new city or region. Customer mix, prices and seasonality differ. Expect covariate and label shift. Reweight, recalibrate and monitor by region. The setting resembles work in logistics and retail.

A new time period. Data drifts slowly. Retrain on a rolling window, weight recent rows more, and watch for concept drift after market or policy changes.

A new customer on a shared platform. Each customer has a private table with its own columns. Use a model that does not depend on column names, add in-context examples if the model supports them, and fine-tune only when enough labels exist.

A new sensor generation. The columns are the same but the noise and resolution change. Denoise, resample to a common rate and check that the units match before trying anything advanced.

Metrics worth tracking

• Target accuracy or AUC on the labelled target sample, with a confidence interval.
• Gap to oracle: the difference between your adapted model and a model trained on plenty of target labels, when you can afford to measure it.
• Calibration error on the target, since decisions often use probabilities.
• Effective sample size of the weights.
• Domain AUC after adaptation: a drop shows that features became more alike.
• Per-slice results: by site, season, product line or any group that matters to the business.

Governance notes

Adapting a model to a new group can change who is helped and who is harmed. Always run fairness and error-rate checks per group after adaptation. Keep a record of the source data, the target data, the weights and the method, so an auditor can reproduce the result. Pair this with data lineage records where available.

Common mistakes

• Validating on the source only. This hides the very failure you want to find.
• Using the target test set for tuning. It leaks information and inflates results. Keep a clean split.
• Ignoring concept shift. Weighting cannot repair a changed rule. If labels mean something new, collect new labels.
• Letting identifiers leak. Site or time columns let a model learn shortcuts that fail on a new domain.
• Over-trusting adversarial methods. They can collapse or oscillate. Track the domain classifier loss and the label loss separately.
• Extreme weights. One row with a weight of 5,000 can dominate training. Clip them.
• Forgetting class priors. Under label shift, accuracy may fall just because the mix changed. Check priors.
• Assuming overlap. If the target lives in regions the source never visited, reweighting has nothing to work with.
• Skipping the do-nothing baseline. Sometimes the plain model is already fine, and adaptation adds risk.
• Mixing up shift with noise. A noisy week is not a new domain. Use enough data and time before reacting.
• Adapting on the same rows you report on. If the unlabelled target rows used for adaptation are also your test rows, scores look better than they should. Split first.
• Ignoring unit and encoding errors. Many "shifts" are bugs: a column in the wrong unit, a code that changed meaning, a date format that flipped. Check these before reaching for algorithms.
• Tuning on too few target labels. With 20 labelled rows, model selection is mostly luck. Report the interval, not just the point value.
• Forgetting the decision threshold. Even a well-ranked model needs a new cut-off when the class mix changes. Re-pick thresholds on the target.
• Treating one number as success. An average gain can hide a loss on an important group. Look at slices.
• Adapting once and never again. Domains keep moving. Schedule a review, and automate the drift checks.

Frequently asked questions

What is domain adaptation in simple words?

Domain adaptation is a way to make a model that learned on one kind of data work on a different but related kind of data. The task stays the same, such as predicting failures or classifying records, but the inputs look different. The methods adjust the data, the model or the training process so that performance holds up on the new data without starting over.

How is domain adaptation different from transfer learning?

Transfer learning is the broad idea of reusing knowledge from one task or dataset for another. Domain adaptation is a specific case where the task is the same and the data distribution differs. In practice the two overlap, and fine-tuning a pre-trained model on target data is often described with both names.

What is covariate shift?

Covariate shift means the distribution of the inputs changes between training and deployment, while the relation between inputs and labels stays the same. A common example is a model trained on customers from one region and used in another with different income levels. Importance weighting is the standard first tool for this case.

Can domain adaptation work without any target labels?

Yes, in the unsupervised setting it can, as long as the shift is mostly in the inputs and the domains overlap. Methods such as importance weighting, feature alignment and adversarial training use only unlabelled target features. The catch is that you cannot measure the result without some labels, so a small labelled target sample for validation is strongly advised.

How do I know if my data has shifted?

Train a classifier to tell source rows from target rows. If it does much better than chance, the domains differ, and its feature importances show which columns changed. You can also compare summary statistics, run two-sample tests per column, and watch the prediction distribution over time.

What is negative transfer?

Negative transfer happens when adapting to the target makes results worse than using the source model unchanged. It usually occurs when the domains are not truly related, when the method forces an alignment that does not exist, or when label shift is ignored. A held-out labelled target sample catches it early.

Does domain adaptation help gradient boosted trees?

It does. Importance weighting, per-domain normalization, ratio features and domain-invariant feature selection all work with tree models. Adversarial methods need a neural network, but you do not need one to get most of the benefit on tabular data.

How does in-context learning relate to domain adaptation?

In-context learning lets a model adapt to a new table by reading labelled example rows as part of its input, with no weight updates. The examples act as a small target sample, and the model infers the local rule from them. It is a gradient-free form of adaptation that is central to modern tabular foundation models.

What is the difference between domain adaptation and domain generalization?

Domain adaptation uses some information about the target domain, usually unlabelled features, while building the model. Domain generalization has no access to the target at all and prepares the model for any unseen domain by training on several source domains. Generalization is harder and usually gives smaller gains.

How much target data do I need?

It depends on the method. Unsupervised methods need enough unlabelled target rows to estimate the distribution, often a few hundred to a few thousand. Validation needs a labelled sample, where even about a hundred rows can be informative. Fine-tuning needs more labels, and the right number depends on model size and task difficulty.

Glossary

• Adversarial training: Training two parts against each other, here a feature extractor and a domain classifier.
• Calibration: How well predicted probabilities match real frequencies.
• Concept drift: A change over time in the rule linking inputs to labels.
• Covariate shift: A change in the input distribution with a fixed labelling rule.
• Domain classifier: A model that predicts whether a row comes from the source or the target.
• Domain generalization: Learning to perform on domains not seen in training.
• Effective sample size: A measure of how many equally weighted rows a weighted sample is worth.
• Fine-tuning: Continuing training of a pre-trained model on new data.
• Importance weighting: Reweighting rows so that one distribution mimics another.
• In-context learning: Adapting through the input examples, without changing weights.
• Label shift: A change in class proportions with a fixed class appearance.
• Negative transfer: Adaptation that hurts target performance.
• Pseudo-label: A prediction treated as a label for further training.
• Source domain: The labelled domain used for training.
• Support: The region of feature space where a distribution has data.
• Target domain: The domain where the model must perform.
• Test-time adaptation: Adjusting a model on unlabelled data during deployment.

Summary

Domain adaptation keeps a model useful when the data in front of it no longer looks like the data it learned from. Start by measuring the gap with a domain classifier, name the kind of shift, and pick the simplest method that fits: weighting for covariate shift, prior correction for label shift, fine-tuning when you have some target labels. Validate on a small labelled target sample, watch calibration, and monitor after launch.

For tabular foundation models, shift is the default state. Pre-training across many varied tables, in-context learning on target rows, native handling of missing values and robustness to unnamed columns are all ways of building adaptation into the model itself. See the architecture, research and benchmarks pages for how these ideas are put into practice and tested.

Related

It belongs to Learning paradigms in my stack, under AI and machine learning. Others in the same group: