Compression and serving: running a 2.18 billion parameter tabular model

A tabular foundation model of about 2.18 billion parameters is large to store and to serve. Compression makes it smaller by storing each weight with fewer bits. The model was dumped at half precision, 8-bit and 4-bit, and fidelity was measured against the full-precision model with TurboQuant. The point of the exercise is not a smaller file for its own sake. It is a smaller file whose answers stay the same on the real tasks.

The packaging story, with its dates and its audit, is on a companion page: One checkpoint, then compression. There are no fidelity numbers on this page. The model card carries them with their protocols.

The short version

• The core and every head and adapter were merged into one unified checkpoint of about 2.18 billion parameters, with the code embedded. The core alone is more than 1.5 billion.

• One line of bootstrap gives access to every capability, and the merged model was tested in isolation.

• The model was dumped at half precision, 8-bit and 4-bit. This is post-training quantization: train once in high precision, round the weights to fewer bits afterwards.

• Fidelity was measured with TurboQuant against the full-precision model on the real tasks. It is measured and not assumed.

• A 4-bit weight takes a quarter of the space of a 16-bit one. The saving is real, and it can change answers, which is why the check exists.

• The benchmark numbers come from the same model that serves requests.

Why compression belongs in the story

A model that is trained once is meant to be served many times. That changes what to optimise.

• Storage. Lower precision means smaller files to keep and to move.

• Memory. Smaller weights fit on smaller machines.

• Speed. Fewer bits to move from memory can mean faster answers, because modern hardware is often limited by how fast weights can be read and not by arithmetic.

• Consistency. One checkpoint means every request meets the same model.

The worry is the one every photographer knows. A picture saved at lower quality takes less space, and a little detail is lost. For most uses nobody notices. The question is how much was lost and whether it matters for the job.

What is inside the checkpoint

The model has a core and a set of heads and adapters on top of it. See Architecture for the full picture.

The first model of this class had 140M parameters. The model here has a core of more than 1.5 billion, and about 2.18 billion once everything is merged. See Research.

The code travels with the weights

A file of weights alone is a pile of numbers. The code says how to wire them. Embedding the code means that one line of bootstrap loads the model and exposes every capability, with no separate repository to match and no helper script to version.

# Illustrative shape only, not a specific API.
model = load_checkpoint("unified")      # one line of bootstrap
model.sector(table)
model.impute(table)
model.predict(table, target="label")
model.match(table_a, table_b)

Why merge before compressing

Compression is applied to the object that serves, not to a research build. That ordering guarantees that the fidelity check measures the thing users will meet. If you compress a part and merge later, you have to test twice.

Quantization in plain terms

The basic idea

A neural network stores its knowledge as numbers called weights. Each weight is a decimal number. Stored at 16 bits, it can take many fine-grained values. Stored at 8 bits, it can take 256 values. At 4 bits it can take 16. Quantization maps the original range of weights onto that small set of allowed values and remembers how to map back.

An everyday analogy: measuring a room. In millimetres you capture every detail. In centimetres you capture most of it. In half-metres you capture only the rough layout. The bigger the unit, the shorter the note you write down and the more you lose.

Scale and zero point

Two small numbers make the mapping work.

• Scale. How big one step of the small number format is, in the original units.

• Zero point. Which small-format value stands for zero, so that asymmetric ranges are covered well.

To store a weight, you divide by the scale, add the zero point and round. To use it, you reverse the steps. The rounding is where error comes from.

Where the scale lives

A single scale for a whole weight matrix is cheap and crude. Finer schemes use one scale per row or output channel, or one per small group of weights. More scales mean better accuracy and a little more storage for the scales themselves.

Outliers are the main enemy

Most weights are small. A few are large. If one scale must cover both, the large ones set the step size and the small ones all round to the same few values. Many practical methods are, at heart, ways of handling outliers.

Post-training versus training-aware

• Post-training quantization takes a trained model and rounds it afterwards. It needs no further training, sometimes only a small sample of data to choose scales. This is the approach here, and it fits a model that is trained once and served many times.

• Quantization-aware training simulates rounding during training, so the model learns to live with it.

Weights versus activations

Weight-only schemes compress the stored parameters and compute in a higher precision. Schemes that also quantize activations (the values flowing between layers) can run the arithmetic in low precision too, which can be faster, and is harder to do without damage. The sources for this model describe compression of the stored weights, so this page speaks of weights.

The methods in the neighbourhood

The model was measured with TurboQuant. Several other methods are worth knowing, because they define the vocabulary. The links lead to general explanations.

For the model on this site, the recorded facts are these: it was dumped at half precision, 8-bit and 4-bit, and fidelity was measured with TurboQuant. I do not claim more about the internals of any dump than that.

What the three precisions are for

The table gives the general trade-off. The measured fidelity is carried with the model card.

Back-of-envelope size

You can estimate the size of the weights from the parameter count. These figures are arithmetic and not measurements. Real files differ, because scales and metadata take space and some parts may be kept at higher precision.

The calculation is simple: 2.18 billion weights times the bytes per weight. A decimal gigabyte is a billion bytes.

The trade in one sentence

You give up some bits for space and speed, and you check on the real tasks that you did not give up too much.

Fidelity is measured, not assumed

The risk is that rounding adds error, and error in a weight can compound through layers. So fidelity is measured against the full-precision model on the real tasks.

• The reference is the model before compression.

• The test is the real tasks, the ones on the boards.

• The question is whether the compressed model changes its answers where it matters.

Why fidelity is task-specific

A compressed model that is faithful on one task may drift on another. A tabular model has several kinds of output, and each reacts to rounding differently.

This table is a checklist of where to look. It is not a report of findings. The measured fidelity is on the model card.

A comparison you can run yourself

If you serve a compressed copy, repeat the check on your own task. The sketch shows the idea.

ref  = load_checkpoint("unified", precision="half")
fast = load_checkpoint("unified", precision="4bit")

for table, target, split in your_tasks:
    a = ref.predict(table, target=target, split=split)
    b = fast.predict(table, target=target, split=split)
    report(
        agreement = share_equal(a.labels, b.labels),
        score_ref = held_out_score(a),
        score_fast = held_out_score(b),
        coverage_ref = interval_coverage(a),
        coverage_fast = interval_coverage(b),
    )

Three habits keep it honest.

• Use the same split and the same seed for both copies.

• Compare held-out scores against the same baseline.

• Look at the weakest slice, not only the average.

Testing the merged model in isolation

Before compression, the merged model was tested in isolation. That phrase has a precise meaning.

• The model is loaded on its own, from the single checkpoint.

• No code from research folders is on the path.

• No separately stored adapters are patched in.

• Each capability is exercised through the same entry point a user would use.

What a pass means

If the merged model reproduces each capability without outside help, the benchmark numbers are not artifacts of a special build. A head that works only in a special build cannot sneak into a claim.

A walk through the test

1. Start a clean environment with only the checkpoint.

2. Load the model with the one line of bootstrap.

3. Ask for the sector of a table with no names, which exercises the semantic head.

4. Hide some cells and ask for a fill, which exercises the imputation network.

5. Give labelled rows and ask for a prediction, which exercises the prediction head.

6. Give two tables and ask for a column alignment, which exercises the cross-table head.

7. Add a new task and check that earlier tasks are retained, which exercises the adapters.

The same suite is the natural place to repeat the fidelity check for each precision.

A foundation model that predicts in context does its work at request time.

The table

Both are far below full attention over all cells, and both grow with the table. See axial attention.

The support set

The model reads a support set of labelled rows and compares new rows with it. The model is designed to read a sample of rows from a large table and not every row at once. See In-context prediction.

The global read

A Perceiver-style block lets a fixed number of latent vectors read all the cells. See Perceiver IO.

The weights

Every request has to move the weights through the hardware. Compression reduces that traffic.

Summary of the drivers

Wide tables

A table with several hundred columns strains the column side. Dropping columns that are constant, duplicated or irrelevant often helps more than expected. This is a data-side saving that adds to anything compression gives.

Serving: one model for everything

The benchmark model and the serving model are the same object. That has three consequences.

• A benchmark number is not a separate research artifact. It comes from the same model that serves requests.

• The claim cannot rest on a special build. Everything claimed lives in the object that was tested.

• Upgrades are clean. A new head or a new adapter means a new merge and a new test in isolation.

Consistency for the reader

If the model on the benchmark is the model in production, the numbers on the model card describe what you get. This is the practical value of the packaging step.

Continual learning and serving

An adapter that carries a new task adds a few weights, not a second model. The replay buffer is data, not weights, and it must be stored and protected like any copy of the data. See Continual learning for tables.

Name independence and serving

Because the model never uses column names, a request needs no schema mapping on the way in. The serving path has no dictionary lookup to keep in step. See Column name independence.

Choosing a precision

Use this as a decision aid. It is a way of thinking, not a measured recommendation.

1. Start from the task.

2. Run the reference at half precision on your own split.

3. Run 8-bit on the same split. Compare held-out score, calibration and the weakest slice.

4. Run 4-bit the same way, if size or memory is the binding constraint.

5. Choose the smallest precision whose differences you cannot distinguish from run-to-run noise.

Noise is part of the judgement

A difference between precisions is only meaningful if it is larger than the variation you get from changing the seed. Run both with several seeds and look at the spread. A gap inside the seed noise is not evidence of loss.

Match precision to the job

After deployment

Track live accuracy

A held-out score is a creation-time estimate on unseen rows of the same table. Live accuracy from labels that arrive after launch is a separate measurement. Compare the two, and compare both for the compressed copy and the reference if you can keep one around.

Watch calibration

Probabilities and intervals are the model's own estimate of how much it knows. Bin predictions and compare with outcomes occasionally. Calibration can drift when the data does, and rounding can contribute.

Re-check after any change

• A new base, a new head or a new adapter means a new merge, a new isolation test and a new fidelity check.

• A new precision means a new fidelity check on your task.

• A refreshed table means a new held-out score.

Keep the system of record elsewhere

The model returns understanding and predictions. It does not store, version or secure your data. Treat it as an analysis layer.

The contamination audit

A model trained on millions of tables might have seen a benchmark table. If it had, the score would flatter it. Each run is checked for contamination against the training corpus, and the model card was published on 7 August 2026 only after the audit.

1. An identifier blocklist. Known benchmark dataset identifiers are excluded from training.

2. A distributional near-duplicate check. Tables that are close to a benchmark table in their statistics are flagged, even if they carry another name.

A blocklist catches tables by name. It misses a copy under a new name. The near-duplicate check catches tables by their statistics, and it can flag a look-alike that is not a copy. Each covers the other's blind spot. For deduplication as a general idea see deduplication. For the wider discipline see Pretraining.

Timeline

The release checklist

1. Every head and adapter that a claim depends on is sealed and copied at the moment of sealing.

2. The core and the pieces are merged into one checkpoint.

3. The code is embedded in the checkpoint.

4. One line of bootstrap loads the model.

5. Each capability is exercised on its own board's entry point.

6. The merged model passes in isolation.

7. Compression dumps are made at half, 8-bit and 4-bit.

8. Fidelity is measured against the full model on the real tasks.

9. The evaluation sets are audited for contamination.

10. The model card is published, with its protocols.

Why seal and copy

On the matching board, a checkpoint that was not preserved at the time of sealing taught me to copy every sealed adapter at the moment of sealing. It turns a lost checkpoint from a disaster into a non-event.

What stayed open

• Size. About 2.18 billion parameters is large. Compression helps, and fidelity must be watched at lower precision.

• Fidelity is task-specific. It is measured on the real tasks, and it should be re-checked on yours.

• One checkpoint is a snapshot. A change means a new merge and a new test.

• Some ceilings remain. Regression and multiclass boards stay behind the leaders. See Limitations.

• The audit lowers risk and does not prove its absence. It catches known cases with two different checks.

No free lunch

Compression gives smaller files and faster serving. It can also change answers. I measure the change and report it with the model card. I do not claim that every precision is as good as the full one for every task.

Common misreadings

"Merging means averaging models"

It does not. It means combining the core with its heads and adapters into one checkpoint, so each capability keeps its own parts.

"A smaller file is always as good"

Fidelity is measured on the real tasks at each precision. A smaller file is useful only where that measurement shows the answers hold.

"4-bit is a quarter of the quality"

It is a quarter of the space per weight. Quality is a separate measurement, and it is not proportional to bits.

"Compression makes the model faster in every setting"

Fewer bits to move can mean faster answers where memory traffic is the limit. On other hardware or other workloads the gain can be small.

"Embedded code means a closed black box"

It means the checkpoint is self-contained. The model is documented on this site, and its protocols are in the model card.

Terms used on this page

• Checkpoint. A saved copy of a model's weights, here with its code.

• Bootstrap. The line of code that loads the model.

• Quantization. Storing weights with fewer bits.

• Post-training quantization. Rounding a trained model afterwards, without a new training run.

• Scale and zero point. The two numbers that map small-format values back to the original range.

• Outlier. A weight much larger than the rest, which makes a shared scale hard to choose.

• Fidelity. How closely a compressed model matches the full model on real tasks.

• Calibration. The match between stated confidence and actual frequency.

• Near-duplicate. A table very close to another in its statistics, even under a different name.

Fact sheet

Quick answers

• Is there one file? Yes, one unified checkpoint.

• Is the code inside? Yes.

• Which precisions exist? Half, 8-bit and 4-bit.

• Was fidelity measured? Yes, with TurboQuant, on the real tasks.

• Does compression change answers? It can, so it is measured.

• Is the serving model the benchmark model? Yes.

• Where are the numbers? On the model card, with their protocols.

Where to read next

• One checkpoint, then compression for the packaging story.

• Architecture for the parts that were merged.

• Pretraining for the training and the audit.

• Capabilities for what each output returns.

• Evaluation for what a score claims.

• Benchmarks for the whole timeline.

Frequently asked questions

How big is the model and what does compression do to it?

The core is more than 1.5 billion parameters and the merged checkpoint is about 2.18 billion. Compression stores each weight with fewer bits: a 4-bit weight takes a quarter of the space of a 16-bit one. As a rough guide, the weights alone take about 4.4 GB at 16 bits, 2.2 GB at 8 bits and 1.1 GB at 4 bits, before overhead.

Which precisions were produced?

Half precision, 8-bit and 4-bit. These are post-training quantization levels: the model is trained once in higher precision and its weights are rounded afterwards. Fidelity was measured with TurboQuant against the full-precision model.

Does compression change the model's answers?

It can. Rounding adds error, and error in a weight can compound through layers. That is why fidelity is measured on the real tasks and not assumed, and why I do not claim that every precision is as good as the full one for every task.

How should I check a compressed copy on my own data?

Run the reference and the compressed copy on the same split and seed, then compare held-out score against the same baseline, calibration, interval coverage and the weakest slice. Repeat with a few seeds and treat a gap inside the seed noise as no evidence of loss.

Why is there only one checkpoint?

A result counts only if a single model can reproduce all of it. The core and every head and adapter were merged into one object with the code embedded, and tested in isolation, so a head that works only in a special build cannot underwrite a claim. The same object serves requests.

What does testing in isolation mean?

The merged model was loaded on its own from the single checkpoint, with nothing from research folders on the path and no separately stored adapters patched in. Each capability was exercised through the entry point a user would use.

What makes a request expensive?

The size of the table, the number of labelled rows read as the support set, and the weights that must be moved through the hardware. Row and column attention grow with the table, the global read grows linearly with the number of cells, and compression reduces the weight traffic.

What was the contamination audit?

An audit of the evaluation sets against the training corpus, using an identifier blocklist and a distributional near-duplicate check. It was completed before the model card was published on 7 August 2026. It lowers the risk of overlap and does not claim to prove that none exists.

Does a continual-learning adapter make serving heavier?

Only slightly. An adapter adds a few weights and is merged into the same checkpoint. The replay buffer is data and not weights, so it needs storage and protection as a copy of the data.

Summary

Compression and serving turn a research result into an object people can run. The core and every head and adapter were merged into one checkpoint of about 2.18 billion parameters with the code embedded, tested in isolation, and then dumped at half, 8-bit and 4-bit precision. Fidelity was measured with TurboQuant against the full-precision model on the real tasks, and a contamination audit with an identifier blocklist and a near-duplicate check came before the model card of 7 August 2026.

What matters for serving is that the benchmark model and the serving model are the same, that the smaller files are only a win where the answers hold, and that fidelity is task-specific and should be re-checked on your own split. Size, snapshot semantics, replay buffers and the remaining ceilings on other boards stay open and are written down. Read next: One checkpoint, then compression, Architecture, Evaluation and Limitations.