# Evaluation How a tabular foundation model is evaluated: held-out splits, benchmark protocols, sealed configurations, contamination checks, and what a score claims. > Informational documentation, based on the paper Data Language Models: A New Foundation Model Class for Tabular Data (arXiv:2605.06290, https://arxiv.org/abs/2605.06290). Reference material, not a hosted service; hosts, keys, and ids are placeholders. A score is only useful if you know exactly what was measured. This page explains how the model is evaluated at two levels: on your own table, with a held-out split, and on public academic benchmarks, under a fixed protocol each. It also explains the metrics, the rules that keep comparisons fair, and the mistakes in measurement that taught me to check the ruler before I trust a number. The short version is this. A held-out score claims one narrow thing, performance on unseen rows of the same table. A benchmark result claims another, performance under a stated protocol against stated rivals. Neither claims that the model will be right on a different world. The Limitations page says what lies outside both. This page is score-free on purpose. Scores depend on the harness, the fold count and the snapshot date, so they live with their protocols on the model card and on the individual benchmark pages, not in prose that can drift away from them. ## Two layers of evaluation There are two different questions, and they need two different tests. - **Question one: will it work on my table?** This is answered by a held-out evaluation on that table. It is cheap, it is repeatable, and it is stamped with the data snapshot it ran on. - **Question two: is the model good at this kind of task in general?** This is answered by public benchmarks, where many methods are run on many datasets under one protocol. The two layers support each other. A benchmark tells you that the method is credible. A held-out score tells you that it is credible here. Think of buying a car. The crash-test rating tells you how the model of car performs in a standard test. The test drive tells you how this particular car handles on your road. You want both, and you should never confuse one with the other. | Layer | Question | Evidence | Scope of the claim | |---|---|---|---| | Held-out on your table | Does it work on my rows? | A stratified split of your labelled rows | Unseen rows from the same table | | Public benchmarks | Is the method strong at this task? | Many datasets, many rivals, one protocol | The datasets and protocol of that benchmark | | Live monitoring | Is it still right today? | Labels that arrive after deployment | The period you observe | The third row is not a benchmark and not a held-out score. It is the measurement you take once the model is used for real, from labels that arrive later. It is a separate number and should be tracked separately. ## Part one: held-out evaluation on your table ### The basic idea To measure a prediction honestly, you must score it on rows the model did not see when it predicted them. A student who has seen the exam questions in advance scores well and proves nothing. So part of the labelled rows is set aside. The model reads the remaining labelled rows as context, predicts the target for the set-aside rows, and the predictions are compared with the true values. The model never trains on your table, so there are no weights to leak. The thing that must not leak is the answer. The target values of the held-out rows are withheld, and the model sees only their other columns. This is explained in in-context prediction. ### How the split is made The default is a stratified 80/20 partition. Stratified means that the share of each class is kept the same in both parts, so a rare class is not accidentally absent from the test side. The report records the method, the exact partition sizes and the random seed. With those three facts anyone can repeat the split and get the same rows. A refresh scores a fresh split on the renewed data snapshot. That matters when rows are removed from the source: a new split reflects the new data instead of repeating the old score. ```text labelled rows (22,400) | +-- 80% context (17,920 rows: features and targets visible) | +-- 20% held out ( 4,480 rows: targets withheld, then scored) ``` The numbers in the sketch are illustrative. They show the shape of a split, not a result. ### Every score travels with its baseline A raw accuracy is hard to read. Ninety-six percent sounds excellent until you learn that 95 percent of the rows belong to one class. So every score is stated against a baseline, and the gain over the baseline is reported as lift. The baseline depends on the task. | Task | Main metrics | Baseline | |---|---|---| | Classification | Accuracy, ROC-AUC (one-vs-one for multiclass), per-class results | Majority class | | Regression | RMSE, NRMSE, R-squared, quantile calibration | Mean or last value | | Anomaly | Precision, recall, PR-AUC where labels exist; score separation and threshold, flagged unsupervised, where they do not | Random or expected rate | | Imputation | NRMSE on held-out known cells | Simple fill of the column | A trivial guess is the floor. If the model cannot beat it, the number is not worth reporting, and the report will show that plainly. ### What a report contains A good report carries its provenance with it. The reader should never have to ask which data, which split, which seed or which baseline. - The target column and the data snapshot the run used. - The method, which is an in-context holdout, with the seed and the train and test sizes. - The headline metric, and the secondary metrics next to it. - The baseline and the lift over it. - Per-class results, so a strong majority class cannot hide a weak minority class. - The weakest slice, so the report names where the model is least reliable instead of leaving the reader to look for it. - A note on confidence: whether predicted probabilities are calibrated and how confident the model tends to be. Here is the shape of the block, with illustrative numbers. ```text Target: churn Data: snapshot of 12 Aug, 09:31 Method: in-context holdout, seed 7 Context 17,920 / Test 4,480 Accuracy 96.1% ROC-AUC 0.974 Baseline (majority) 71.0% lift +25.1 points Per class retained 97% churned 91% Weakest slice tenure under 3 months -> 84% ``` The weakest-slice line is the one I value most. Averages hide trouble, and slices expose it. ### What the number claims A held-out score is an estimate for unseen rows drawn from the same table. That is a precise claim, and it is also a limited one. It does not claim performance on data from a different distribution. It does not claim performance after the world changes. It does not claim performance on classes that never appear in the table. Picture a weather forecaster who tests a method on last year's data from one city. The result tells you about that city in a similar climate. It does not tell you about another continent. ### Why the split is random and why that is not enough A random split assumes that the rows are exchangeable, which means that any row could just as well have been in the other half. For many tables that is fine. For tables with time in them it is not. If you predict next month's churn, a random split lets the model peek at the future through neighbouring rows. A time-based split, where the test rows are all later than the context rows, is the honest test. Similarly, if several rows belong to the same customer, a random split can put one row in each half. The model then recognizes the customer, not the pattern. Group-aware splits fix this. I mention these because the split is a decision. The default is sensible, and it is a default. If your table has time or groups, say so, and evaluate the way you will use the model. ## Part two: public benchmark evaluation Beyond per-table evaluation, a foundation model is judged on standard academic benchmarks. There, it is compared with strong baselines such as gradient-boosted trees, AutoML pipelines and other tabular foundation models, under the protocol each benchmark defines. The point of a benchmark is shared ground. Everyone runs the same datasets, the same folds and the same metric, so the numbers can be set side by side. Ten benchmark efforts stand behind the model. Each has its own page with the story of how I built the capability, what made the difference, and what stayed open. | Benchmark effort | What it measures | Page | |---|---|---| | Row-level prediction boards | Classification and regression on many public datasets, ranked against a shared field | Row-level prediction | | Continual learning | Retention across a long sequence of tasks | Continual learning | | Anomaly detection | Semi-supervised ranking of rows that do not fit normal | Anomaly detection | | Probabilistic regression | Whole predicted distributions, scored with proper scoring rules | Probabilistic regression | | Metadata independence | Prediction when column names are random or removed | Metadata independence | | Blind sector identification | Naming the industry of an unseen dataset | Sector identification | | Missing-value imputation | Reconstructing hidden cells | Imputation | | Entity and schema matching | Finding the same entity or column across tables | Matching | | Missing-data robustness | Prediction after injecting gaps | Robustness | | One checkpoint, then compression | Whether one model reproduces everything | Unified checkpoint | ### Row-level prediction The row-level boards ask how well a model predicts a held-out target from the other columns, across many public datasets at once. Methods are ranked with an Elo-style score against a shared field that includes hand-tuned AutoML pipelines and published tabular foundation models. I ran the lite style of the protocol, with one fold per dataset. That is a lighter test than a full cross-validation, and I say so wherever the result appears. The model runs one default configuration. The rivals were tuned for hours per dataset. ### Continual learning There is no published competitor here. Tree ensembles retrain fully for each task, and frozen foundation models have no update path at all. So the test is about retention. After learning task after task, how much of the first tasks is still there? Retention is measured with a fixed formula, and backward transfer is measured as well. ### Anomaly detection The setting is semi-supervised. The detector sees normal rows only during fitting and is scored on a test set that mixes normal and anomalous rows. That is the setting in which the published methods are measured, so I match it. Ten classical detectors and published model pools are the rivals, thirteen methods in all. ### Probabilistic regression Here the model predicts a whole distribution, and the distribution is scored with proper scoring rules, CRPS and CRLS. The board carries 102 regression datasets and the published outputs of 50 models. I ran the official runner locally with one sealed configuration, five folds on all datasets. I merged my outputs with the published ones and recomputed every statistic over the whole pool with the board's own ranking procedure. The benchmark is described in arXiv:2603.29928. ### Metadata independence Twenty numerical classification datasets are run under three naming conditions, full names, random strings and names removed. The metric of record is accuracy with no names, and the drop from full names. Rivals run on the same folds with five seeds, 300 measurements in all. The protocol follows the paper behind the first model of this class, arXiv:2605.06290. ### Blind sector identification The task is to name the industry sector of an unseen dataset from raw cell values. The test uses 504 real datasets with column names zeroed, no labels and no metadata, and scores top-1 and top-5 against a uniform random baseline. No published competitor performs this task. Ground-truth labels come from independent judges, never from the model itself. ### Missing-value imputation Numeric imputation hides cells and measures reconstruction error, as NRMSE, on 20 real datasets under three missingness mechanisms. Categorical imputation measures macro-F1 on masked categorical cells and has no published competitor. The numeric benchmark is described in arXiv:2603.22332. ### Entity and schema matching Entity matching uses six multi-table datasets and asks which rows describe the same real-world entity, with no keys. The model matching uses the Valentine scenarios and asks which column in one table corresponds to which in another, scored as F1. Every published competitor sees column names. The model does not. The references are arXiv:2604.21238 and arXiv:2507.10897. ### Missing-data robustness Ten datasets, masks of three types and several rates, with AUC measured after injecting missing values into the features. The full grid has 750 cells, and I ran all of them. The benchmark follows arXiv:2507.08280. ### One checkpoint The last step checks that all capabilities live in one object and that compression does not change them. The merged checkpoint is about 2.18 billion parameters, and it was dumped at half, 8-bit and 4-bit precision, with fidelity measured against the original. ## The metrics, explained in plain words Metric names can sound heavy. Each one answers a simple question. ### Accuracy and its limits Accuracy is the share of predictions that are right. It is easy to read, and it is easy to fool, because a model that always says the majority class can look good on lopsided data. That is why accuracy is always shown next to the majority-class baseline, and why per-class results are reported. ### ROC-AUC ROC-AUC asks a ranking question. If you pick one positive row and one negative row at random, how often does the model give the positive row the higher score? A value of 0.5 means chance and 1.0 means perfect ranking. For multiclass problems the one-vs-one version averages this over pairs of classes. ### PR-AUC, precision and recall When positives are rare, as in fraud, ROC-AUC can look comfortable while the model still produces many false alarms. Precision is the share of flagged rows that are truly positive. Recall is the share of true positives that were flagged. PR-AUC summarizes the trade between them across thresholds. For anomaly work, where the expected rate is the baseline, it is the more honest summary. ### RMSE, NRMSE and R-squared RMSE is the typical size of the error, in the units of the target, with large errors counting extra. NRMSE divides it by a measure of the target's spread, so errors on different columns can be compared. R-squared is the share of the target's variation that the model explains. Zero means no better than the mean, and one means perfect. ### Quantile calibration and coverage If a model says that a value lies between 8,400 and 17,900 with 80 percent probability, then about 80 percent of true values should land in that range. Coverage is the measured share. Calibration matters more than sharpness at first. A narrow interval that is wrong half the time is worse than a wider interval that is honest. ### CRPS and CRLS These are proper scoring rules for whole distributions. A proper scoring rule is one where the best way to score well is to report your true belief. CRPS compares the predicted distribution with the observed value across the whole range, and it is forgiving of a slightly off distribution. CRLS is the log version, and it punishes a distribution that puts too little probability on what actually happened. A distribution that is too narrow is hit hard. That difference is why CRLS exposed a problem that CRPS did not. For heavy-tailed targets, standardizing by standard deviation made the predicted distribution too narrow, and the fix was a log transform followed by calibration. The story is on the probabilistic regression page. ### Macro-F1, micro-F1 and the averaging rule F1 balances precision and recall. Macro averaging gives each class equal weight. Micro averaging pools all decisions together, so large classes count more. The two can disagree. In schema matching the macro average gave a different answer from the micro average, so the averaging rule must be fixed before scoring, and it should be fixed in writing. Scoring there is micro-averaged. ### Elo-style score and average rank An Elo-style score treats every dataset as a match between two methods and updates a rating from the results, the way chess ratings work. It rewards beating strong opponents. Average rank is simpler. On each dataset, order the methods, then average the position across datasets. It ignores the size of a win, which is both its strength and its weakness. A method can have a good rank and a modest score, or the reverse, if it wins narrowly often or loses badly rarely. That is why I report the ranking procedure of each board and do not mix procedures. ### Retention and backward transfer For continual learning, retention asks how much of an earlier task is still performed after later tasks. Backward transfer asks whether learning later tasks helped or hurt the earlier ones. A model that forgets has negative backward transfer. A model that improves old tasks by learning new ones has positive backward transfer, which is rare and nice. ### Top-1 and top-5 For sector identification, top-1 asks whether the first guess is right, and top-5 asks whether the right answer is among the first five. Both are scored against a uniform random baseline, so a score is read as how far above chance it sits. Top-5 is the fair measure for neighbouring sectors that even people confuse. Top-1 is the strict measure. ## The standing rules Every benchmark claim obeys the same rules. They are written down so that I cannot bend them quietly when a result is tempting. 1. **Competitor figures are third-party published values.** I never re-run a rival to produce a number for the table, and I cite each figure to its primary source. For live boards I pin the snapshot date. 2. **One sealed configuration per benchmark.** My figures come from a single fixed setting, with no per-dataset tuning. Rivals often get hours of tuning per dataset, and the model gets its default. 3. **No blending.** Results measured under different datasets, metrics or masking schemes are never merged into one number. 4. **Contamination is checked.** Each run is checked against the training corpus before the claim is made. 5. **Seeds are fixed and reported.** A win that sits inside its own seed noise is dropped. 6. **Weak spots are written beside the strong ones.** Every benchmark page has a section on what stayed open. ### Why a sealed configuration matters Imagine two students taking the same exam. One gets one attempt. The other may take the exam fifty times and report the best attempt. The second student's score is higher on average, and it means less. Tuning a model per dataset and reporting the best setting is the second student. A sealed configuration is the first. It is a stricter test, and the number is closer to what you would get on a table I have never seen. ### Why competitor figures are never re-run If I re-ran a rival myself, every choice, the folds, the settings, the library version, would be mine. Any weakness in how I ran it would be a gift to me, and nobody could tell. Using the values the rival's authors published removes that suspicion. There was one case where this mattered in the other direction. A rival in the distribution benchmark had been run weakened by the benchmark's wrapper, with 8 estimators instead of 32 and 200 quantiles instead of 999. I re-ran it fairly to check, and the comparison did not change. The published value stayed as the reference. ### Why different protocols are never blended A score on 20 real datasets under three missingness mechanisms is not the same kind of thing as a score on synthetic settings. Averaging them would produce a number that describes neither. So a result keeps the name of its protocol. If a rival is ahead on one setting, that row stays as reported. ## Checking the ruler The most useful lesson from the whole project is that measurements fail quietly. A bad ruler gives a confident, wrong answer. Several of the largest gains and several of the largest scares came from the harness, not the model. This table lists the cases. | Where | What looked wrong | What it really was | |---|---|---| | Overall prediction board | Missing datasets were filled with a penalty | A regression run was filed under a different method name. I merged the metadata and re-checked that no row was imputed | | Imputation | Poor early results | Fifteen errors across harness and chain, including an imputer that saw only a fifth of each table | | Imputation | A neighbours arm looked strong | It read real values on its own validation cells, which let it cheat | | Imputation | The semantic channel did little | It had been fed placeholder column names | | Missing-data robustness | A plain baseline scored about twelve points below the published table | An unaligned protocol. I rebuilt it and verified the masker with drift gates | | Missing-data robustness | A long string of failed runs | A shell quirk that swallowed a less-than sign in a pasted comparison | | Probabilistic regression | A rival seemed weak | The wrapper ran it weakened. Run fairly, the comparison did not change | | Continual learning | Heavy forgetting | The adapter was re-created from scratch for every task | | Continual learning | Results moved by 15 to 20 percent between runs | A missing random seed | | The model matching | A checkpoint could not be found at sealing | It had not been preserved. Every sealed adapter is now copied at the moment of sealing | None of these are model failures. They are the kind of silent error that makes a model look better or worse than it is. ### Aligning to a published number first When a baseline on my harness does not match its published number, I stop and find out why before I measure anything else. In the robustness work, a plain gradient-boosted baseline scored about twelve points lower on my harness than in the published table. If I had ignored that, every comparison afterwards would have been unfair to the rivals and flattering to me. I built an aligned protocol and verified the masker with drift gates, six shift checks and four untouched validations. Only then did I run the grid. ### Reproducing the floor Another habit is to build a verified floor. In imputation, a plain ridge chain matched MICE exactly. That told me the harness was reading the problem the same way the classical method did, and it gave me a floor that the more complex arms had to beat by a margin. ### Margins against luck When only 30 holdout cells decide among 77 candidate settings, the winner is partly luck. Choosing it is a form of overfitting to the validation cells. So the chain picks a non-baseline arm only if it beats the plain ridge arm by a fixed proportional margin. A candidate must be clearly better, not just first by a hair. ## Seeds, noise and what counts as a win Machine learning results have noise. The same method with a different seed gives a slightly different number. During continual learning I found that a missing random seed alone moved results by 15 to 20 percent. That is larger than the effect of most ideas, so a comparison with unseeded runs is worthless. The rules that followed are simple. - Seed every run, and report the seed. - Run enough seeds to know the noise. The metadata protocol uses five seeds, which with the folds and datasets gives 300 measurements. - Treat a win smaller than the seed noise as no win. One metadata result sat inside its own noise, and I dropped it. - Prefer a mean with a worst case beside it. A pseudo-rehearsal variant in continual learning had a higher mean and a terrible worst case, so I rejected it. A leaderboard hides all of this behind one number per row. That is why the pages here describe the protocol and not just the position. ## Contamination checks A model trained on millions of tables might have seen a benchmark table. If it had, a good score would prove memory, not skill. Each run is therefore checked for contamination against the training corpus, using two checks. 1. An **identifier blocklist**. Known benchmark dataset identifiers are excluded from training. 2. A **distributional near-duplicate check**. Tables that are close to a benchmark table in their statistics are flagged even when they carry another name. For the sector task there is a third guard. The tables used to train the sector head, 1,401 of them, are excluded from the sector test set, and the test set was frozen before the final claims. The contamination audit was completed before the model card was published on 7 August 2026. ### What the audit cannot do An audit by identifier and by near-duplicate statistics catches the obvious cases. It cannot prove that no fact about a benchmark ever reached the corpus in some other form. So the audit is a strong precaution and not a guarantee, and I describe it that way. The sealed configuration, the third-party figures and the symmetric protocols are the other half of the defence. ## Fair comparison, step by step Here is the checklist I run before I accept a comparison as fair. - **Same data.** Same datasets, same folds, same masks, same seeds for every method. - **Same metric, same averaging.** The averaging rule is fixed before scoring. - **Same information.** Rivals that read column names get names, and I note that I do not use them. When names are the only clue, that is a disadvantage I accept and say so. - **Same effort, or the difference is disclosed.** The model runs one default. Rivals that are tuned are labelled as tuned. - **Rivals run properly.** I check that a rival has not been run weakened by a wrapper. - **Leakage in the rival is flagged.** One published matching competitor was trained on the benchmark data. That is a leakage caveat for them, not for me, and I say so next to the result. - **Aligned baselines.** A plain baseline matches its published value on my harness. ## Reading a result A benchmark entry on this site has the same five parts every time. They are what I would want to read if I were checking someone else's claim. 1. **What it measures.** The task, the data and the metric. 2. **How I built it.** The path, including dead ends. 3. **What made the difference.** The one or two choices that mattered. 4. **What stayed open.** The weak metric, the narrow lead, the rival that is still ahead. 5. **Protocol.** The folds, the seeds, the reference to the benchmark paper. Pay most attention to the fourth part. A page that has no open items has probably left them out. ### Narrow leads are reported as narrow Some results are clear. Others are narrow, such as the lead in missing-data robustness. I describe a narrow lead as a narrow lead, and I stopped tuning when a further ladder of levers gave nothing. It is not evidence of dominance. ## Where the model is behind An evaluation page that lists only wins is advertising. These are the places where the evidence says the model is not ahead. - **Regression and multiclass boards.** On the row-level boards the model is first on classification and binary tasks. Regression and multiclass are behind the leaders. On most boards the second place is a tuned AutoML pipeline, not another foundation model. - **Interval coverage before calibration.** Before split-conformal calibration, the intervals did not cover as often as they claimed. - **Synthetic imputation settings.** On two synthetic settings the best published MICE figure stayed ahead of my runs, and I left those rows as reported. - **Small tables for imputation.** On a few datasets with few columns or few rows the chain has little context and the gain is small. - **Pure numeric classics.** Without semantics, a stronger prior-based rival still leads on metadata independence, usually by a few thousandths. - **Sector confusions.** The remaining errors are sibling-sector confusions such as banking and insurance, and 24 datasets are ambiguous even for human labellers, so they are reported separately. - **Continual learning.** Retention comes from replay, and the buffer grows linearly. An earlier generation reports a higher number on a different task sequence, so the two cannot be compared directly. The full list, with advice for each, is on the Limitations page. ## Running your own evaluation You do not have to take any of this on trust. A few steps give you a trustworthy check on your own table. ### Step one: choose the split to match the use If the model will predict the future, split by time. If rows cluster by customer, device or site, split by group. Otherwise use the stratified random default. ### Step two: set the baseline before you look Write down the majority class, the mean, or the last value before you see the model's score. That way you cannot adjust the bar after the fact. ### Step three: read slices, not only averages Break the result down by class, by segment and by time. The weakest slice is where the first real failure will show up. ### Step four: check calibration If the model says 90 percent, is it right about 90 percent of the time? Bin the predictions and compare. A reliable ranking with unreliable probabilities is common, and it changes how you set thresholds. ### Step five: compare against what you already use Run your current method on the same split. A foundation model that matches a tuned tree ensemble with no training is useful. One that loses clearly should not replace it. ### Step six: keep measuring after deployment A held-out score is a creation-time estimate. Live accuracy comes from labels that arrive later, and it is a separate measurement. Compare the two over time. A widening gap is an early sign that the data has moved. ```text creation time held-out score on a stamped snapshot refresh new split on the renewed snapshot, new report after launch live accuracy from real labels, tracked separately ``` ## Reproducibility A score is tied to a fixed split, a baseline and a data snapshot. Because the report stamps all three, the same evaluation can be repeated and compared over time. On the benchmark side, the same idea holds. One sealed configuration, fixed seeds, a named protocol and a cited source for every rival figure mean that a reader can, in principle, redo the comparison. Each change is tested on the full board and not on one dataset, because a gain on one dataset can hide a loss on ten others. ## Evaluation of the other properties Accuracy is not the only property worth testing. These are the other things I check, and the pages where they are covered. | Property | How it is checked | Where to read more | |---|---|---| | Name independence | Same task with full names, random names and no names | Column name independence | | Robustness to gaps | Injected missingness of three types and several rates | Missing-data robustness | | Retention | Sequential tasks with a fixed retention formula | Continual learning for tables | | Fidelity after compression | Quantized weights compared with the original on the real tasks | Compression and serving | | Honest uncertainty | Quantile coverage and probability calibration | Capabilities | The model was dumped at half precision, 8-bit and 4-bit, and fidelity was measured against the full-precision model on the real tasks. A smaller file is only a win if the answers stay the same. ## Frequently asked questions ### What does a held-out score tell me? It tells you how the model performed on rows of your table that it did not see when it predicted them. It is an estimate for unseen rows from the same table, stated with its split, seed, baseline and data snapshot. It does not describe a different distribution, a changed world, or classes that are absent from the table. ### How is the held-out split made? By default it is a stratified 80/20 partition, so the class shares are the same in both parts. The report records the method, the exact partition sizes and the seed, so the split can be repeated. If your table has time or groups, a time-based or group-aware split is a better match for how you will use the model. ### Why is every score shown next to a baseline? Because a score without a floor is hard to read. Accuracy of 96 percent means little if 95 percent of rows belong to one class. The baseline, which is the majority class, the mean or the last value, shows what a trivial guess would achieve, and the lift shows how much the model adds. ### Does the model train on my data during evaluation? No. It predicts in context, reading the labelled rows as a support set, and no weights are updated. The held-out rows have their targets withheld, so the score reflects prediction and not memory. ### Why are competitor results not re-run? Because a number I produced for a rival would depend on my choices, and nobody could check them. I use third-party published values, cite their source and pin the snapshot date for live boards. In one case I did re-run a rival fairly, to check a weakened wrapper, and the comparison did not change. ### What is a sealed configuration? It is the one fixed setting used for a benchmark, with no per-dataset tuning. Rivals often get hours of tuning per dataset, and the model gets a single default. It is a stricter test, and its result is closer to what you would see on a new table. ### How do you rule out contamination? Each run is checked against the training corpus with an identifier blocklist and a distributional near-duplicate check. For the sector task, the tables that trained the sector head are excluded from the test set and the test set was frozen. This is a strong precaution and not an absolute guarantee, and it is described that way. ### Why are some benchmark leads called narrow? Because they are. In missing-data robustness the mean AUC is above the published method by a small margin, and I report it as a narrow lead. A further ladder of tuning gave nothing, so I stopped. ### Where is the model behind? Regression and multiclass boards are behind the leaders, interval coverage was weak before calibration, a published MICE figure stayed ahead on two synthetic imputation settings, and continual-learning retention depends on replay. The Limitations page lists these with practical advice. ### Can I compare the held-out score with a benchmark result? Not directly. A held-out score is local to your table. A benchmark result is local to its datasets and protocol. Use the benchmark to judge whether the method is credible, and use the held-out score to judge whether it works on your data. ## Summary Evaluation has two layers. On your own table, the model is scored on a stratified held-out split with a stated seed, baseline and data snapshot, and the number claims only unseen rows of that table. On public benchmarks, it is compared with strong rivals under each benchmark's own protocol, from one sealed configuration, with third-party figures, fixed seeds and a contamination check. The metrics are chosen so that each answers a plain question, from accuracy against a majority baseline to CRPS for whole distributions. The habit that matters most is checking the ruler first, because several of the biggest swings came from the harness and not from the model. Weak spots are named beside the strong ones. For how the network works read Architecture, for what it returns read Capabilities, for how each result was built read Benchmarks, and for what lies outside every score read Limitations.