Mixed precision

Overview

Mixed precision is a training technique that uses small number formats, such as 16-bit or 8-bit floating point, for most calculations, while keeping a few sensitive values in 32-bit. The result is faster training, lower memory use, and almost the same model quality as full 32-bit training. It is now the default way to train large neural networks on modern GPUs.

The idea works because most of the arithmetic inside a network does not need many digits. Matrix multiplications are tolerant of rounding. A few operations are not: sums of many small numbers, the optimizer update, and some normalization steps. Mixed precision keeps those in 32-bit and lets everything else run in a cheaper format. For tabular models, which read rows and columns of mixed types, the same trick lets a team train bigger models on more tables for the same hardware budget.

What it is

Mixed precision is a recipe, not a single switch. It combines three parts:

• A low-precision format for the heavy math (FP16, BF16, or FP8).
• A high-precision copy of the weights, called master weights, that receives the actual updates.
• A small set of safeguards, such as loss scaling and a list of operations that stay in 32-bit.

Think of it like cooking for a large crowd. You measure the bulk ingredients roughly, by the scoop, because small errors average out. You still weigh the spices on a precise scale, because a small error there ruins the dish. Mixed precision measures the bulk with scoops and the spices with a scale.

The number formats involved

Computers store real numbers in a fixed number of bits. More bits give more range (how big or small a number can be) and more precision (how many digits are kept). The common formats differ in how they split bits between these two.

The key contrast is between FP16 and BF16. FP16 keeps more digits but runs out of range quickly. BF16 keeps fewer digits but has the same range as FP32, which makes it much easier to use.

Where it came from

Early deep learning used 32-bit floats almost everywhere. As models grew, memory and speed became the limit. Researchers noticed that weights and activations could be stored in fewer bits with little harm to accuracy.

A widely cited 2017 paper, "Mixed Precision Training" from researchers at a GPU maker and a university, showed that FP16 training can match FP32 accuracy when three things are done: keep an FP32 master copy of the weights, scale the loss to avoid underflow of small gradients, and accumulate certain products in FP32. That paper turned a hack into a method.

Hardware followed the research. GPUs added matrix units that run 16-bit math much faster than 32-bit math. Later generations added native support for BF16, a format that originated in accelerator work for large-scale training, and then for 8-bit floating point. Framework authors wrapped the recipe in automatic tools, so users no longer had to cast tensors by hand.

The result is a steady trend: every hardware generation makes lower precision faster, and training recipes adapt to use it. See the notes on NVIDIA H100 and CUDA for the hardware and software side.

How it works

Mixed precision training follows a repeatable loop. The steps below describe the classic FP16 version, then note what changes for BF16 and FP8.

Step 1: Keep master weights in FP32

The model stores one full-precision copy of every weight. This is the source of truth. It is like keeping the original blueprint in a safe while workers use photocopies on the building site.

Step 2: Cast a working copy to low precision

Before each forward pass, the weights are cast to FP16 or BF16. This working copy is half the size, so it moves through memory faster and fits into the fast matrix units of the GPU.

Step 3: Run the forward pass in low precision

Layers such as linear projections and attention matrix products run in the low-precision format. Inputs, activations, and outputs are all stored in 16-bit. This is where most of the speed and memory savings come from.

Step 4: Keep fragile operations in FP32

Some operations lose accuracy badly in 16-bit. Typical examples:

• Softmax and log-softmax, because they exponentiate values.
• Large reductions, such as sums over many elements.
• Layer normalization statistics.
• The final loss computation.

Automatic mixed precision tools keep a list of operations that are cast up to FP32 and another list that are safe in FP16. Everything else follows the dtype of its inputs.

Step 5: Scale the loss (FP16 only)

FP16 has a narrow range. Gradients in deep networks are often tiny, and many fall below the smallest value FP16 can store. They become zero, which is called underflow.

Loss scaling fixes this. Multiply the loss by a large factor, such as a power of two, before the backward pass. All gradients grow by the same factor and survive in FP16. Before the weight update, divide the gradients by the factor again.

Think of it as looking at a faint pencil drawing under a magnifying glass. You scale it up to see the lines, then scale back down when you copy the result.

Dynamic loss scaling automates the choice. The scaler starts with a high factor, and if it sees an overflow (infinity or not-a-number in the gradients), it skips that step and lowers the factor. After a run of good steps, it raises the factor again.

Step 6: Backward pass and update in FP32

Gradients are computed in low precision, then converted to FP32. The optimizer applies them to the master weights. Optimizer state, such as the running averages in Adam, also stays in FP32. This keeps small updates from being lost to rounding.

Step 7: Repeat

The updated master weights are cast down again for the next step. The whole loop runs thousands or millions of times.

What changes with BF16

BF16 has the same range as FP32, so gradients rarely underflow. Loss scaling is usually unnecessary. This is the main reason many large-scale runs prefer BF16 on hardware that supports it. The cost is lower precision per number, which master weights and FP32 accumulation absorb.

What changes with FP8

FP8 has very little range and precision. Two variants exist: one with more range for gradients and one with more precision for weights and activations. To use FP8, each tensor needs its own scale factor, tracked from recent maximum values. Matrix multiplications run in FP8, while accumulation, master weights, and many surrounding operations stay in higher precision. This is more delicate than FP16 or BF16 and is usually applied only to the largest matrix multiplications.

Why hardware matters

Modern GPUs contain tensor cores, which are matrix units built for low-precision math. A tensor core multiplies small blocks of 16-bit or 8-bit numbers and accumulates the result in 32-bit. This is why mixed precision gives a real speedup and not just a memory saving. See CUDA and NVIDIA H100.

Where the memory goes

Training memory has four main consumers. Mixed precision changes each one differently.

• Parameters. The FP32 master copy is kept, and a 16-bit working copy is added.
• Gradients. These are usually held in 16-bit during the backward pass, then used in FP32 for the update.
• Optimizer state. Adam keeps two extra values per parameter, and both stay in FP32.
• Activations. These are stored for the backward pass and shrink to half size in 16-bit.

A rough per-parameter budget for classic mixed precision with Adam looks like this.

The sum is about sixteen bytes per parameter, which is more than plain FP32 training for the model state alone. This surprises many people. The saving comes from activations, which dominate memory in large batches and long sequences, and from faster math. Sharding tools such as FSDP and DeepSpeed ZeRO attack the model-state part by splitting it across devices.

Which operations are safe in low precision

A useful rule of thumb divides operations into three groups.

• Safe in 16-bit: matrix multiplications, convolutions, linear layers, and most pointwise activations.
• Keep in FP32: softmax, log-softmax, cross-entropy, large sums and means, norm layers, and exponentials.
• Follow the input: simple element-wise additions and copies.

The grouping follows one idea. Operations that add up many numbers, or that raise numbers to a power, magnify rounding errors. Operations that just multiply and add a few terms do not.

Why accumulation precision matters

A matrix multiplication adds thousands of products. If each partial sum is rounded to 16 bits, the errors pile up. Tensor cores avoid this by multiplying in low precision and adding in 32-bit. The inputs are small, but the running total is large.

An everyday picture helps. Imagine counting coins by dropping each one into a jar and reading the jar at the end. Reading each coin to the nearest cent is fine, as long as the total is kept exactly. If the total itself is rounded after each coin, the final count drifts.

Stochastic rounding and other tricks

Some low-precision recipes round at random instead of always to the nearest value. Rounding up or down with probability based on the distance keeps the average update correct, even when each single update is tiny. This is one way to train with 16-bit weights and no master copy. It is less common than master weights because it needs hardware or library support.

Other tricks in the same family include:

• Kahan-style summation to recover lost low-order bits when adding small updates.
• Per-tensor and per-block scale factors so each group of values uses the full range of the format.
• Delayed scaling, where the scale for the next step comes from the maximum values seen in recent steps.
• Higher-precision first and last layers, since embeddings and output heads are the most sensitive.

Diagnosing not-a-number values

A run that suddenly prints a loss of not-a-number is a common sign of a precision problem. A calm checklist helps.

1. Check whether the loss scaler has been lowering its scale. If it has, the problem is overflow in the gradients.
2. Look at the input data for extreme values, infinities, or unscaled columns.
3. Run a few steps in FP32 with the same data. If the problem disappears, the cause is precision.
4. Turn on anomaly detection in the framework to find the first operation that produced the bad value.
5. Move the failing operation to FP32, or switch from FP16 to BF16.
6. Lower the learning rate or add gradient clipping if the loss spikes before the failure.

Speed expectations

How much faster a run gets depends on where the time goes. Three cases are common.

• Compute bound. Large matrix multiplications dominate. Mixed precision gives a large gain.
• Memory bound. Moving data between memory and the chip dominates. Half-size tensors help directly.
• Overhead bound. Python code, small kernels, or data loading dominate. Mixed precision helps little until those are fixed.

Profiling before and after is the only reliable guide. A profile shows the share of time in matrix kernels, which tells you the ceiling on the gain.

Key concepts and terms

A worked example

A numeric walk-through

Consider adding a small gradient to a large weight. Suppose the weight is 1.0 and the update is 0.00001.

In a format with roughly three decimal digits of precision, 1.0 + 0.00001 rounds back to 1.0. The update vanishes. After a million steps, the weight never moves.

With an FP32 master copy, which keeps about seven decimal digits, the same update survives: 1.00001. Over many steps, tiny updates add up. That is why master weights exist.

Now consider underflow. Suppose a gradient is 0.00000001. If FP16 cannot represent numbers that small, it stores zero. Multiply the loss by 1024 first, and the gradient becomes about 0.00001, which FP16 can hold. After the backward pass, divide by 1024 and apply the update in FP32.

A PyTorch example with BF16

The simplest modern recipe is autocast with BF16. No scaler is needed.

import torch
import torch.nn as nn

device = "cuda"
model = nn.Sequential(
    nn.Linear(128, 512),
    nn.ReLU(),
    nn.Linear(512, 1),
).to(device)

opt = torch.optim.AdamW(model.parameters(), lr=1e-3)
loss_fn = nn.MSELoss()

x = torch.randn(256, 128, device=device)
y = torch.randn(256, 1, device=device)

for step in range(100):
    opt.zero_grad(set_to_none=True)
    with torch.autocast(device_type="cuda", dtype=torch.bfloat16):
        pred = model(x)
        loss = loss_fn(pred, y)
    loss.backward()
    opt.step()

Parameters stay in FP32 because the model was never cast. Autocast converts inputs to BF16 inside matrix operations and returns FP32 where needed.

A PyTorch example with FP16 and loss scaling

scaler = torch.amp.GradScaler("cuda")

for step in range(100):
    opt.zero_grad(set_to_none=True)
    with torch.autocast(device_type="cuda", dtype=torch.float16):
        pred = model(x)
        loss = loss_fn(pred, y)
    scaler.scale(loss).backward()
    scaler.unscale_(opt)
    torch.nn.utils.clip_grad_norm_(model.parameters(), 1.0)
    scaler.step(opt)
    scaler.update()

Notice the order. Scale, backward, unscale, clip, step, update. Clipping must happen after unscaling, or the clip threshold is meaningless. The scaler skips any step where it finds infinite or not-a-number gradients.

Checking the effect

A quick way to confirm a benefit is to measure three things before and after:

• Peak memory used on the GPU.
• Time per training step.
• Validation loss after a fixed number of steps.

A healthy result is lower memory, shorter step time, and a validation loss that tracks the FP32 baseline within normal noise.

A decision table for choosing a recipe

A simple migration plan

Moving an existing FP32 training script to mixed precision is low risk when done in stages.

1. Record a baseline: peak memory, step time, and validation loss at fixed checkpoints.
2. Turn on TF32 matrix math, which needs almost no code change.
3. Switch to BF16 autocast and compare the three baseline numbers.
4. If BF16 is not available, switch to FP16 autocast with a gradient scaler.
5. Raise the batch size to use the freed memory, and retune the learning rate.
6. Add gradient checkpointing or sharding only if memory is still tight.
7. Try FP8 last, one layer group at a time.

Each stage can be reverted by changing a single setting, which keeps the process safe.

Reading the loss curve

The shape of the loss curve tells you whether precision is a problem.

• A curve that tracks the FP32 baseline closely means the recipe is sound.
• A curve that is smooth but plateaus higher points to lost small updates, often from missing master weights.
• A curve with sudden spikes points to overflow or an unstable learning rate.
• A curve that turns to not-a-number points to an overflow or a bad input.

Strengths and limitations

Strengths

• Faster training, because tensor cores run low-precision math much faster than FP32.
• Lower memory, because activations and working weights are half the size.
• Larger batch sizes or bigger models on the same GPU.
• Less data movement, which helps when memory bandwidth is the bottleneck.
• Nearly the same final quality when done correctly.
• Easy to adopt, since framework tools handle most of the casting.

Limitations

• FP16 can overflow or underflow without loss scaling.
• BF16 has low precision, so some tasks with fine numeric detail may need care.
• FP8 needs per-tensor scaling and is sensitive to recipe choices.
• Not every operation has a fast low-precision kernel, so some speedups are smaller than expected.
• Debugging is harder when not-a-number values appear in the middle of a run.
• Small models may see little benefit, because they are not limited by math or memory.
• Results can differ slightly between runs and between hardware types.

How it compares with alternatives

Mixed precision is orthogonal to the others. It reduces the cost of each number, while parallelism spreads numbers across devices and checkpointing recomputes numbers instead of storing them. Production runs usually combine all of them.

Mixed precision versus post-training quantization

These two are often confused. Mixed precision happens during training and keeps a high-precision copy for updates. Quantization, such as GPTQ or AWQ, happens after training and compresses the final weights for inference. A model trained with mixed precision can later be quantized, but the two steps are separate.

Role in tabular foundation models

Tabular foundation models learn from raw tables: rows and columns with mixed types, missing values, and often no useful column names. They are pre-trained on large collections of synthetic or real tables, then used through in-context learning, where labeled example rows are shown at inference time and the model predicts new rows without weight updates. Mixed precision matters for these models in several concrete ways. For the architecture itself, see /model/architecture.

Training cost of row and column attention

A table has two axes. A model that reads it must relate rows to rows and columns to columns. Attention over both axes is expensive, because the cost grows with the square of the sequence length along each axis. See axial attention for one way to split the work.

Mixed precision cuts the memory held by attention scores and activations in half. That allows longer context (more example rows) and more columns per table during pre-training. Combined with FlashAttention, which avoids storing the full attention matrix, it makes in-context learning over thousands of rows practical.

Mixed types and numeric range

Tabular data mixes numbers, categories, dates, and text. Numeric columns can span many orders of magnitude: an age is in the tens, an income is in the millions. Models normally scale and transform each column before encoding. If a column is not scaled well, FP16 overflow or underflow can hit the embeddings of that feature.

Practical consequences:

• Normalize or quantile-transform numeric columns before they enter the network.
• Prefer BF16 over FP16 for tabular pre-training, since its range is safer for heavy-tailed features.
• Keep the loss and any regression targets in FP32 to avoid rounding in the training signal.

Missing values and imputation

Missing cells are common in real tables. Models handle them with a mask or a learned missing token. In masked cell modeling, cells are hidden on purpose and the model learns to predict them. This is the same skill needed for imputation. See also imputation networks.

Mixed precision interacts with these objectives in two ways. First, reconstruction losses over many masked cells are sums, and sums should be accumulated in FP32. Second, the predicted value for a numeric cell needs enough precision to be useful, so output heads for imputation are often kept in higher precision than the body of the network.

Pre-training on synthetic tables

Models such as TabPFN are trained on very large numbers of synthetic tables drawn from random data-generating processes. Each training step uses a fresh batch of small tables. This is a workload with many small, varied examples, and the speed of each step directly sets how many tables the model sees. Mixed precision raises that count for the same compute budget.

Related models such as TabICL and TabSTAR follow the same broad pattern of pre-training on many tables and applying the model to new tables. Mixed precision is part of the standard toolkit for all of them. See /model/research for the wider research picture.

In-context learning at inference

At inference, a tabular foundation model reads the labeled rows and the query rows in one forward pass. There is no gradient step. Here mixed precision is purely a speed and memory tool. Running the forward pass in BF16 or FP16 lets the model handle larger contexts and serve more requests per GPU. The caution is determinism: small numeric differences can change a prediction near a decision boundary, so teams often test that outputs under low precision match full precision on a validation set.

Continual learning

When a model is updated on new tables without forgetting old ones, the updates are often small. Methods such as elastic weight consolidation and experience replay rely on subtle penalty terms or stored examples. Small updates are exactly what low precision can erase. Master weights in FP32 and FP32 optimizer state are therefore essential in continual learning, even if the forward pass is in BF16.

Anomaly detection and sector classification

Anomaly detection depends on small differences between expected and observed values. If scoring runs in low precision, subtle deviations may be rounded away. A good practice is to compute final anomaly scores in FP32 even when the encoder runs in BF16. For semantic and sector classification heads, such as semantic sector heads, the logits feed a softmax, which should be computed in FP32.

Benchmarks

When comparing models on tabular tasks, the numeric precision used for evaluation should be stated and kept consistent. See /model/benchmarks.

Practical guidance and best practices

Choose the format first

• On recent GPUs, start with BF16. It is the simplest and the most stable.
• Use FP16 with a dynamic loss scaler on older hardware that lacks BF16.
• Try FP8 only after BF16 works, and only for the largest matrix multiplications.

Use framework tools

• Use autocast rather than casting tensors by hand.
• Use the built-in gradient scaler for FP16.
• Keep the model parameters in FP32 and let the framework manage the working copy. See PyTorch.

Protect the sensitive parts

• Compute losses in FP32.
• Keep normalization statistics and softmax in FP32.
• Keep optimizer state in FP32.
• Accumulate gradients across micro-batches in FP32.

Combine with other scaling tools

• Pair with FSDP or DeepSpeed ZeRO to shard weights, gradients, and optimizer state.
• Pair with gradient checkpointing when activations still do not fit.
• Pair with FlashAttention for long contexts.
• For many-GPU runs, communication libraries such as NCCL can send gradients in 16-bit to cut network traffic, though reductions should still be accumulated carefully.

Monitor the run

• Log the loss scale for FP16 runs. A scale that keeps collapsing signals a real problem.
• Log gradient norms. Sudden spikes or zeros are warning signs.
• Log the count of skipped steps.
• Compare early validation loss against an FP32 run on a small slice before committing to a long run.

Validate the result

• Keep a small FP32 reference run for the first few hundred steps.
• Compare final metrics with a tolerance that matches normal seed-to-seed variation.
• Test inference outputs in the deployed precision, not only the training precision.

Common mistakes

• Casting the whole model to FP16 by hand. This removes the master weights, and small updates are lost. Use autocast instead.
• Forgetting to unscale before clipping. The clip threshold then applies to scaled gradients and does nothing useful.
• Using FP16 on heavy-tailed features without scaling. Large values overflow and produce infinity.
• Ignoring skipped steps. If a large share of steps are skipped because of overflow, the effective batch count is lower than you think.
• Keeping the loss in low precision. Summing many small terms in FP16 is inaccurate.
• Expecting a speedup on small models. If the run is limited by data loading or by Python overhead, mixed precision does little.
• Comparing runs with different precision and calling it a model difference. Precision is a variable. Keep it fixed when comparing methods.
• Using non-power-of-two sizes. Tensor cores run best when dimensions are multiples of eight or more. Odd sizes can fall back to slow kernels.
• Applying FP8 everywhere. Embedding layers, normalization, and output heads are often kept in higher precision.
• Skipping evaluation in the deployment dtype. A model that looks fine in training precision may behave differently when served in another format.

Frequently asked questions

What is mixed precision training in simple terms?

Mixed precision training means doing most of the math in small, fast number formats and keeping a few important values in a larger, more accurate format. It is like measuring flour by the scoop but weighing spices on a scale. The model trains faster and uses less memory, and the final quality is almost the same as full-precision training.

Is BF16 better than FP16?

For training, BF16 is usually easier because it has the same range as FP32, so gradients rarely underflow and loss scaling is not needed. FP16 keeps more digits per number, which can help in some cases, but it needs a loss scaler and more care. On hardware that supports both, most large runs choose BF16 for stability.

Does mixed precision reduce model accuracy?

When done correctly, the accuracy change is negligible and usually within normal run-to-run noise. Problems appear when sensitive operations are left in low precision, when loss scaling is missing for FP16, or when the data has extreme values. Validating against a short FP32 run is the safest way to confirm.

What are master weights and why are they needed?

Master weights are the full-precision copy of the model parameters. Updates are often very small, and a low-precision format would round many of them to nothing. The optimizer updates the FP32 copy, and a low-precision version is made from it for the next forward pass.

What is loss scaling?

Loss scaling multiplies the loss by a large constant before the backward pass so that small gradients do not underflow to zero in FP16. After the backward pass, the gradients are divided by the same constant before the update. Dynamic scalers adjust the constant automatically and skip steps that overflow.

Do I need mixed precision for inference?

It is optional but common. Running the forward pass in 16-bit lowers latency and memory, which helps serve more requests. Unlike training, inference has no gradients or master weights, so the setup is simpler. You should still check that outputs match full precision closely enough for your task.

How does mixed precision help tabular foundation models?

It lets a tabular foundation model process longer contexts and larger tables during pre-training and inference for the same hardware. Row and column attention is memory hungry, so halving the size of activations matters. Losses, imputation heads, and anomaly scores should still be computed in higher precision to protect small numeric differences.

Is FP8 training ready for everyone?

FP8 works on the newest GPUs and can give large speedups, but it needs per-tensor scaling and careful choice of which layers use it. Many teams use BF16 as the default and add FP8 only for the biggest matrix multiplications after validating the result. It is more fragile than 16-bit training.

How is mixed precision different from quantization?

Mixed precision is a training-time method that keeps an FP32 copy for updates. Quantization is usually applied after training to shrink weights for inference and does not update the model. They can be used in sequence on the same model, but they solve different problems.

Can mixed precision be combined with data, tensor, and pipeline parallelism?

Yes. It is independent of how the model is split across devices. Data parallelism, tensor parallelism, and pipeline parallelism all work with 16-bit and 8-bit formats. Lower precision also reduces the amount of data sent between devices, which helps communication-bound runs.

Glossary

• Autocast: A framework feature that selects the dtype for each operation automatically.
• BF16 (bfloat16): A 16-bit float with the range of FP32 and fewer digits.
• Dtype: The data type of a tensor, such as float32 or bfloat16.
• Dynamic loss scaling: Automatic adjustment of the loss scale during training.
• FP8: An 8-bit floating-point format with two variants for range or precision.
• FP16: A 16-bit float with a narrow range and moderate precision.
• FP32: The standard 32-bit float, also called single precision.
• Gradient clipping: Limiting gradient size to keep updates stable.
• Master weights: The FP32 copy of the model parameters.
• Overflow: A value too large for the format, becoming infinity.
• Tensor core: Hardware for fast low-precision matrix multiplication.
• TF32: A format that uses FP32 range with reduced precision inside matrix units.
• Underflow: A value too small for the format, becoming zero.

Summary

Mixed precision runs most of a network in 16-bit or 8-bit arithmetic while keeping master weights, optimizer state, and sensitive operations in 32-bit. It gives faster training and lower memory with very little loss in quality, and it works alongside every form of parallelism. BF16 is the easiest default, FP16 needs loss scaling, and FP8 needs per-tensor scaling and care.

For tabular foundation models, mixed precision makes longer contexts, wider tables, and more pre-training tables affordable. The main cautions are heavy-tailed numeric columns, small updates in continual learning, and fine numeric differences in imputation and anomaly scores. Keep those parts in higher precision, validate against a full-precision reference, and report the precision used. For related topics, see FSDP, DeepSpeed ZeRO, Megatron-LM, and /model/architecture.

Related

It belongs to Foundation models from scratch in my stack, under Foundation models and applied ML. Others in the same group: