# NVIDIA B200

The NVIDIA B200 is a Blackwell-architecture GPU with two dies, large HBM3e memory, FP4 support and fifth-generation NVLink for training and serving frontier models.

## Overview

The NVIDIA B200 is a data-centre GPU built on NVIDIA's Blackwell architecture, the successor to Hopper. It is designed for training and serving the largest AI models. Its main changes over the [H100](/tools/nvidia-h100) and [H200](/tools/nvidia-h200) are a two-die design that behaves as one GPU, much more HBM3e memory and bandwidth, a second-generation Transformer Engine with support for 4-bit FP4 and 6-bit FP6 formats, and fifth-generation NVLink with twice the per-GPU bandwidth of Hopper.

An everyday picture: a factory that has reached the largest building the land allows. To grow, it builds a twin building next door and joins them with a wide, covered walkway, so workers and materials move between them as if they were one hall. Blackwell does that with silicon. Each die is already about as large as manufacturing allows, so NVIDIA joins two dies with a very fast link and presents them to software as a single GPU.

The B200 appears in several systems: HGX B200 boards with eight GPUs, DGX B200 servers, and the GB200 Grace Blackwell Superchip, which pairs two B200-class GPUs with an NVIDIA Grace CPU. The rack-scale GB200 NVL72 joins 72 of these GPUs in one NVLink domain.

## What it is

The figures below are NVIDIA's per-GPU figures for the B200 in the HGX B200 platform. Low-precision Tensor Core peaks are quoted with sparsity, as is NVIDIA's practice; dense figures are half.

| Specification | B200 in HGX B200 |
|---|---|
| Architecture | Blackwell, two dies acting as one GPU |
| Transistors | About 208 billion |
| Memory | 180 GB HBM3e usable in HGX B200 (192 GB installed) |
| Memory bandwidth | About 7.7 TB/s in HGX B200 (NVIDIA has also quoted 8 TB/s) |
| FP4 Tensor Core | 18 PFLOPS with sparsity |
| FP8 and FP6 Tensor Core | 9 PFLOPS with sparsity |
| FP16 and BF16 Tensor Core | 4.5 PFLOPS with sparsity |
| TF32 Tensor Core | 2.2 PFLOPS |
| FP64 | 37 TFLOPS |
| NVLink | Fifth generation, 1.8 TB/s per GPU |
| Host interface | PCIe Gen5 |
| Multi-Instance GPU | Up to 7 instances |
| Maximum power | Up to 1,000 W |

An eight-GPU HGX B200 board offers up to about 1.4 TB of GPU memory and 14.4 TB/s of total NVLink bandwidth. Exact figures differ slightly between HGX, DGX and GB200 systems because of power and memory configurations.

## Where it came from

NVIDIA announced the Blackwell architecture at its GTC conference in March 2024, naming it after the mathematician and statistician David Blackwell. The chips are made on a TSMC 4NP process. Shipments of Blackwell systems ramped through late 2024 and 2025, with GB200 NVL72 racks deployed by cloud providers and AI labs for frontier model training and large-scale inference.

In March 2025 NVIDIA announced Blackwell Ultra, including the B300, with more memory and higher low-precision performance. NVIDIA has also announced Rubin as the following architecture. The B200 remains the reference Blackwell GPU against which these are compared.

## How it works

### Two dies, one GPU

Each Blackwell GPU combines two dies connected by a chip-to-chip interface that NVIDIA rates at 10 TB/s. Caches are coherent across the dies, so software sees one device with one memory space. This avoids the programming burden of treating two chips as two GPUs.

### The second-generation Transformer Engine and FP4

Lower precision means more operations per second and less memory per weight. Hopper introduced FP8; Blackwell adds FP6 and FP4. Its Tensor Cores support micro-scaling formats, where small blocks of values share a scale factor, so very short number formats keep enough accuracy for many inference tasks. A model stored in FP4 needs about a quarter of the memory of BF16, and FP4 Tensor Core throughput is twice that of FP8.

### Memory

The B200 carries eight stacks of HBM3e. With roughly twice the bandwidth of an H100 and well over twice the capacity, it suits large language model inference, which is usually limited by how fast weights and the [KV cache](/tools/kv-cache) can be read.

### NVLink 5 and NVL72

Fifth-generation NVLink gives each GPU 1.8 TB/s of bandwidth. In HGX B200, eight GPUs connect through NVSwitch. In GB200 NVL72, NVLink switches join 72 GPUs and 36 Grace CPUs in one rack, so all 72 GPUs can communicate at NVLink speed. That allows very large models, especially [mixture-of-experts](/tools/mixture-of-experts) models, to be split across many GPUs without slower networks. Between racks, InfiniBand or Ethernet and [NCCL](/tools/nccl) take over.

### Other features

Blackwell adds a dedicated decompression engine to speed data loading, a reliability engine for predictive maintenance in large clusters, and confidential computing support. Most systems use liquid cooling at this power level, particularly the NVL72 racks.

## Key concepts and terms

| Term | Meaning |
|---|---|
| Blackwell | NVIDIA's GPU architecture after Hopper |
| Dual-die | Two silicon dies packaged and linked to act as one GPU |
| FP4 | A 4-bit floating-point format for very efficient inference |
| FP6 | A 6-bit floating-point format |
| Micro-scaling | Sharing a scale factor across small blocks of values |
| HBM3e | Enhanced high-bandwidth memory |
| NVLink 5 | Fifth-generation NVLink, 1.8 TB/s per GPU |
| NVLink domain | The set of GPUs that can talk at NVLink speed |
| GB200 | A superchip with one Grace CPU and two Blackwell GPUs |
| NVL72 | A rack with 72 Blackwell GPUs in one NVLink domain |
| HGX B200 | An eight-GPU Blackwell server board |
| Liquid cooling | Cooling by circulating liquid instead of air |

## A worked example

This scenario is illustrative, and the numbers are rough estimates rather than benchmarks. A company serves a mixture-of-experts model with about 400 billion total parameters.

1. In FP8 the weights need about 400 GB. On H100s, that means at least five GPUs for the weights alone; in practice they used two eight-GPU H100 servers linked by InfiniBand, so expert traffic crossed the network.
2. On one HGX B200 server with 1.4 TB of memory, the FP8 weights fit inside a single NVLink domain with about 1 TB left for cache.
3. They test FP4 weights. Memory for weights falls to about 200 GB. On their evaluation set, answer quality drops by less than one point, which they accept for most traffic, keeping an FP8 deployment for a premium tier.
4. With all experts on one NVLink domain and more cache available, batches grow and cost per generated token falls substantially compared with their H100 deployment.
5. The server draws far more power than an H100 server per box, so they confirm their data centre can supply the power and liquid cooling required.

## Strengths and limitations

**Strengths:**

- Large step in low-precision throughput, with FP4 and FP6 support.
- 180 to 192 GB of HBM3e and about 8 TB/s bandwidth per GPU.
- NVLink 5 doubles per-GPU bandwidth and enables 72-GPU domains.
- Dual-die design appears to software as one GPU.
- Same CUDA programming model and software ecosystem.

**Limitations:**

- Up to 1,000 W per GPU, with heavy power and cooling demands.
- Liquid cooling is often required, especially at rack scale.
- FP4 accuracy must be validated per model and task.
- New systems cost more and may need data-centre upgrades.
- Software support for new formats and features matures over time.

## How it compares with alternatives

| GPU or system | Memory per GPU | Notes |
|---|---|---|
| [NVIDIA H100](/tools/nvidia-h100) SXM | 80 GB HBM3 | Hopper, FP8, NVLink 4 |
| [NVIDIA H200](/tools/nvidia-h200) SXM | 141 GB HBM3e | Hopper with more memory |
| NVIDIA B200 | 180 to 192 GB HBM3e | Blackwell, FP4, NVLink 5 |
| GB200 NVL72 | Blackwell GPUs with Grace CPUs | 72-GPU NVLink domain per rack |
| [Cloud TPU](/tools/cloud-tpu) | Varies by generation | Google's custom accelerators and pods |

## Practical guidance and best practices

1. Check power, cooling and floor loading before planning Blackwell deployments.
2. Validate FP4 and FP8 on your own evaluation sets before using them in production.
3. Keep expert and tensor parallelism inside one NVLink domain where possible.
4. Use engines and libraries with Blackwell support, such as recent [TensorRT](/tools/tensorrt), [vLLM](/tools/vllm) and [SGLang](/tools/sglang) releases.
5. Plan memory for KV cache as well as weights, even with 180 GB.
6. Compare cost per token or per training step across H100, H200 and B200.
7. Keep drivers, CUDA and NCCL versions aligned with vendor-tested stacks.
8. Monitor thermals and reliability signals closely in dense racks.

## Common mistakes

- Comparing sparse FP4 peaks with dense FP8 or BF16 figures from older GPUs.
- Assuming FP4 is lossless for every model.
- Deploying without enough power or cooling, causing throttling.
- Spreading a model across racks when it could fit inside one NVLink domain.
- Running old software builds that do not use Blackwell features.

## Frequently asked questions

### What is the difference between B200 and GB200?

The B200 is the GPU. GB200 is a superchip module that combines one Grace CPU with two Blackwell GPUs, connected by a fast chip-to-chip link. GB200 modules are the building blocks of NVL72 racks.

### How much memory does a B200 have?

192 GB of HBM3e is installed. In the HGX B200 platform NVIDIA lists 180 GB per GPU as available memory. Figures vary slightly by system.

### What is FP4 used for?

FP4 is a 4-bit number format mainly for inference. It cuts memory per weight to about a quarter of BF16 and doubles Tensor Core throughput compared with FP8, with some accuracy cost that must be checked.

### Do I need liquid cooling for B200?

Many B200 systems, and all NVL72 racks, are designed for liquid cooling because of high power density. Some air-cooled HGX B200 designs exist, but data-centre requirements are much higher than for earlier GPUs.

### Will my CUDA code run on B200?

Yes, generally. Blackwell supports the CUDA programming model, and code compiled to PTX can run on it. To use new features such as FP4 Tensor Cores, you need recent toolkits and libraries.

## Glossary

- Reticle limit: the largest die size a chip-making process can produce.
- Chip-to-chip interface: the link joining Blackwell's two dies.
- Superchip: a module combining a CPU and GPUs with a fast coherent link.
- Rack-scale system: a whole rack designed and sold as one unit.
- Dense throughput: peak speed without sparsity.
- Throttling: reducing speed to stay within power or temperature limits.
- Blackwell Ultra: the later Blackwell variant that includes the B300.

## Summary

The NVIDIA B200 is the flagship Blackwell GPU, announced in March 2024: two dies with about 208 billion transistors acting as one GPU, 180 to 192 GB of HBM3e at around 8 TB/s, FP4 and FP6 support in a second-generation Transformer Engine, and 1.8 TB/s NVLink 5, at up to 1,000 W. In HGX boards and GB200 NVL72 racks it lets very large models train and serve within one fast NVLink domain. Its gains are large but depend on validated low-precision formats, current software and data centres able to supply the power and cooling it needs.