# Kimi K3.1 Pro and ERNIE 5.5 Flash Base: sizing giant open MoE models

Moonshot AI released the weights of its 2.8-trillion-parameter Kimi K3.1 Pro, and Baidu released a 280-billion-parameter ERNIE 5.5 Flash Base. Both are mixture-of-experts models with custom licences. This article explains how they work and what it really takes to run them.

Two very large open-weight models arrived within a day of each other this week. On 4 November Moonshot AI published the weights of Kimi K3.1 Pro, a mixture-of-experts (MoE) model with 2.8 trillion parameters. On 5 November Baidu published ERNIE 5.5 Flash Base, a 280-billion-parameter MoE base model. Both can be downloaded. Neither is open source in the strict sense.

For an engineer, the useful question is not "which one scores higher". It is "what would it take to run this ourselves, are we allowed to, and is it worth it compared with an API". Most of the answer is arithmetic you can do on a napkin. This article walks through it.

## What happened

### Moonshot AI publishes Kimi K3.1 Pro weights (4 November 2026)

Moonshot AI released the Kimi K3.1 Pro weights on Hugging Face on 4 November 2026. The model was introduced on 23 October as a post-trained upgrade to Kimi K3, whose weights came out in July. Key figures from the model card and the SGLang cookbook:

- 2.8 trillion total parameters, 104 billion active per token.
- 896 experts, with 16 routed plus 2 shared experts used per token.
- 93 layers: 69 use KDA (a linear-attention design) and 24 use Gated Sparse-MLA (DSA) attention.
- A context window of 1,048,576 tokens and a 401-million-parameter MoonViT-V2 vision encoder.
- MXFP4 weights with MXFP8 activations, from quantisation-aware training. The checkpoint is reported at about 1.56 TB.

The licence is the Kimi K3.1 Pro License, a custom licence covered below. The vLLM baseline reported for the release is one 8-GPU tensor-parallel group: two GB300 trays, one B300 node, or one AMD MI355X node. NVIDIA's Dynamo recipes list 8 GPUs for GB300 and 16 for B200.

### Baidu releases ERNIE 5.5 Flash Base (5 November 2026)

Baidu published ERNIE 5.5 Flash Base on 5 November 2026. It is a raw pre-training checkpoint, not a chat model. Reported figures from the model card:

- 280 billion total parameters, 14 billion active per token. The stored safetensors count is 286,245,697,882.
- 43 layers (1 dense, 42 MoE). Each MoE layer has 512 routed experts, 10 chosen per token, plus 1 shared expert.
- Pre-trained from scratch on 33.5 trillion tokens.
- Pre-training contexts up to 131,072 tokens; fine-tuning and inference supported up to 1,048,576.
- FP8 and BF16 repositories of about 288 GB and 573 GB.

The licence is the ERNIE 5.5 Flash Base Model License Agreement v1.0. Reported minimum hardware for FP8 is two B300 or GB300 GPUs, four B200 or GB200 GPUs, or eight 80 GB H100s. On 80 GB GPUs the vLLM plugin caps context at 262,144 tokens. Baidu says a post-trained ERNIE 5.5 Flash is planned, with no date.

## How mixture-of-experts works

### Total versus active parameters

A dense transformer uses every weight for every token. A [mixture-of-experts](/tools/mixture-of-experts) model replaces the feed-forward block in each layer with many smaller blocks called experts. A small network, the router, picks a few experts for each token. The rest sit idle for that token.

Think of a large hospital. It employs hundreds of specialists, but a patient with a broken wrist sees a triage nurse, a surgeon and a radiologist. The whole staff is the "total parameters". The three people you see are the "active parameters". Your visit costs three people's time, but the hospital still pays and houses every specialist.

That is the central trade-off:

- **Compute follows active parameters.** A forward pass costs roughly 2 operations per active parameter per token: about 2 × 104 billion = 208 billion for Kimi, and 2 × 14 billion = 28 billion for ERNIE.
- **Memory follows total parameters.** Any expert may be needed for the next token, so all must be loaded. Kimi activates 104 of 2,800 billion parameters (about 3.7%); ERNIE 14 of 280 billion (5%). You still store 100%.

### The router and shared experts

The router is a tiny linear layer. For each token it scores every expert, keeps the top k (16 for Kimi, 10 for ERNIE) and blends their outputs using the scores. Training pushes it to spread tokens evenly, because a router that favours a few experts wastes the rest.

Both models also have shared experts that see every token: Kimi has 2, ERNIE has 1. They hold general knowledge so the routed experts can specialise. In the hospital, the shared expert is the triage nurse everyone sees first.

### Why attention design decides long-context cost

Attention keeps a [KV cache](/tools/kv-cache): stored keys and values for every previous token. For classic attention, the cache per token is:

2 (keys and values) × layers × KV heads × head size × bytes per number.

Take an illustrative dense model with 60 layers, 8 KV heads, head size 128, at 2 bytes. That is 2 × 60 × 8 × 128 × 2 = 245,760 bytes, about 246 KB per token. At 1,048,576 tokens that is roughly 258 GB for one conversation. A "1M context" on a spec sheet does not mean you can afford it.

Kimi K3.1 Pro avoids most of this. Its 69 KDA layers are linear attention: a fixed-size state instead of a growing list, which the SGLang cookbook puts at about 52 MB per request per GPU at TP8. Only the 24 sparse-MLA layers keep a per-token cache, about 18.9 KB per token. At 1,048,576 tokens that is about 19.8 GB per request, over ten times smaller than the dense example. The cookbook adds that tensor parallelism replicates this cache on each GPU; only decode-context parallelism (DCP) shards it. Configure it, or the cost multiplies.

## The arithmetic of self-hosting

### Bytes per parameter

Weight memory is parameters × bytes per parameter:

| Precision | Bytes per parameter | Kimi K3.1 Pro (2.8T) | ERNIE 5.5 Flash Base (280B) |
|---|---|---|---|
| BF16 / FP16 | 2 | 5.6 TB | 560 GB |
| FP8 | 1 | 2.8 TB | 280 GB |
| 4-bit (INT4, MXFP4) | 0.5 | 1.4 TB | 140 GB |

Real checkpoints match closely. ERNIE's 286.2 billion stored parameters × 1 byte is 286 GB, against 288 GB reported for FP8; × 2 bytes is 572 GB, against 573 GB for BF16. Kimi's checkpoint is about 1.56 TB versus 1.4 TB in theory. The gap is normal: 4-bit formats store a scale factor per small block of weights, and parts such as the vision encoder and embeddings are often kept at higher precision.

Kimi was trained to run at 4 bits. Quantisation-aware training is like an athlete training at altitude: the model learns under the constraint it will face. Post-training methods such as [GPTQ](/tools/gptq) shrink a finished model, more like resizing a photograph after it was taken. They often work well, but quality loss must be measured.

### Fitting weights onto GPUs

Divide checkpoint size by GPU memory for a hard floor, before any cache:

- **Kimi on 80 GB H100s:** 1.56 TB ÷ 80 GB ≈ 19.5, so at least 20 GPUs for weights alone. The cookbook's H100 layout uses 32 GPUs (2,560 GB).
- **Kimi on 288 GB B300s:** 8 × 288 GB = 2,304 GB. After 1.56 TB of weights, about 740 GB remains for cache and buffers.
- **ERNIE FP8 on 80 GB H100s:** 8 × 80 GB = 640 GB. After 288 GB of weights, about 350 GB remains, yet context is still capped at 262,144 tokens.

A practical rule: keep weights under about 60 to 70% of total GPU memory. The rest is your concurrency budget.

### Bandwidth decides decode speed

Each new token requires reading the active weights from memory. Kimi's 104 billion active parameters at 4 bits are about 52 GB per token, spread across GPUs; ERNIE's 14 billion at FP8 are about 14 GB. Batching reuses each read across many users, so throughput per GPU climbs with concurrency. One user on a giant MoE model is the least efficient way to run it.

### Expert parallelism

When a model is too big for one GPU, you split it:

- **[Tensor parallelism](/tools/tensor-parallelism) (TP):** split each weight matrix, so every GPU does a slice of every layer. It needs very fast GPU links.
- **Pipeline parallelism (PP):** give each GPU a range of layers, like an assembly line.
- **Expert parallelism (EP):** give each GPU a subset of experts and send tokens to wherever their chosen experts live.

EP suits MoE naturally. Picture a food hall where each stall cooks one cuisine: orders go to the right stall and dishes come back to your table. The cost is the walking. Every layer needs an "all-to-all" exchange of tokens between GPUs. That is why recipes mix strategies, such as TP8 with EP2 on GB300 or TP32 with EP32 on H100, and why the interconnect matters as much as the GPUs.

### Base versus instruct, and fine-tuning

ERNIE 5.5 Flash Base has learned to continue text but not to follow instructions. Its serving endpoint is text completion (`/v1/completions`), not chat. A base model is like a well-read graduate with no job training: knowledgeable, but unreliable at a task until trained for it. Use it as a starting point for [supervised fine-tuning](/tools/sft) or research.

Full fine-tuning is expensive. A standard estimate with the Adam optimiser in mixed precision is about 16 bytes per parameter (weights, gradients, a 32-bit master copy, two optimiser states), before activations. For 280 billion parameters that is about 4.5 TB. The NVIDIA Megatron Bridge recipe cited for ERNIE uses 64 GB300 GPUs for full fine-tuning. [LoRA](/tools/lora) trains only small add-on matrices and needs far less, but official materials reportedly do not yet cover it for this model. Kimi K3.1 Pro is already post-trained, so most teams would use it as is.

## What "open weight" means

"Open source" means anyone can use, study, change and share the software for any purpose, without discrimination. Apache 2.0 and MIT meet that bar. "Open weight" only means you can download the parameters; a custom licence sets what you may do. It is the difference between owning a recipe and buying a meal kit with terms printed on the box.

**Kimi K3.1 Pro License.** According to reporting on its text, it permits use, modification, fine-tuning and redistribution if the notice is kept. Businesses selling model access as a service need a separate agreement with Moonshot if group revenue exceeds $20 million over any 12 months. Commercial products with over 100 million monthly active users, or over $20 million in monthly revenue, must display "Kimi K3.1-Pro" prominently. Internal use that does not expose the model to third parties is exempt. These mirror the terms reported for the Kimi K3 licence.

**ERNIE 5.5 Flash Base Model License Agreement v1.0.** Non-commercial use is free. Commercial use is fee-free while related revenue stays at or below $50 million, but needs a notice to Baidu first and a "Powered by ERNIE" display. Above $50 million, written permission is required. The licence excludes the EU, the UK and South Korea, and bars using ERNIE weights, derivatives or outputs to improve non-ERNIE models.

The region exclusion means a UK or EU team cannot use ERNIE 5.5 Flash Base under this licence, however small the project. The output clause matters to anyone planning to distil a smaller model. Read the licence file, involve legal counsel, and record the licence version in your [model cards](/tools/model-cards).

## What it means in practice

**The gap is now mainly operational.** If Moonshot's claims hold up under independent testing, the limit on self-hosting is no longer quality. It is the cost and skill of running a multi-GPU cluster with fast interconnects.

**Open does not mean small.** Kimi K3.1 Pro needs one current 8-GPU data-centre node, or 16 to 32 older GPUs. Community posts on the model page claim about 2 tokens per second on a single 96 GB GPU by offloading to system memory; that is a demo, not production.

**Mid-sized MoE is the practical sweet spot.** ERNIE 5.5 Flash Base at 288 GB in FP8 fits on one 8-GPU H100 node. Two-trillion-plus models are mostly for cloud providers and large enterprises.

**Serving software is part of the choice.** Published recipes for [vLLM](/tools/vllm) and [SGLang](/tools/sglang) separate a weekend experiment from a month of debugging. ERNIE's vLLM plugin reportedly targets NVIDIA GPUs only.

**API versus self-host is economics.** Hosted APIs charge per token, so you need very large, steady volume before an 8-GPU node running all day becomes cheaper. Self-hosting usually wins on data control, customisation and predictable latency rather than price. See [cost and latency budgets](/tools/cost-and-latency-budgets) for modelling this.

## What to do now

1. **Read both licences before downloading.** Check revenue thresholds, attribution, region exclusions and output restrictions.
2. **Compute weight memory.** Total parameters × bytes per parameter, then confirm against the real checkpoint size.
3. **Compute KV cache for your real context.** Use the model's attention design and your 95th-percentile prompt length × target concurrency.
4. **Keep weights under about 60 to 70% of GPU memory.** The rest sets how many users you can serve.
5. **Check the interconnect.** Confirm a published recipe matches your hardware before buying or renting.
6. **Start from published recipes.** Use the SGLang cookbook or NVIDIA Dynamo manifests rather than inventing a layout.
7. **Test on a hosted endpoint first.** Run your own evaluation set; vendor benchmarks are not your workload.
8. **Choose base or instruct deliberately.** Use a base model only if you will fine-tune, and budget training memory separately.
9. **Record the decision.** Note model version, licence version, precision and hardware for later review.

## Risks and open questions

**Benchmarks are vendor claims.** Moonshot's GPQA-Diamond, HLE and OSWorld figures come from the company. Wait for independent evaluations on workloads like yours.

**Licence summaries are second-hand.** We summarised the conditions from press and secondary reports. The licence text is binding and can be revised.

**1M context is a ceiling, not a default.** ERNIE was pre-trained at up to 131,072 tokens and extended afterwards, and its standard launch examples use 131,072. Test quality at long context.

**Recipes are early.** Several SGLang cookbook layouts are labelled placeholders, and one recipe notes untuned throughput.

**Counts do not fully match.** ERNIE is described as 280B, but the stored count is about 286.2 billion. The difference is unexplained.

**Supply chain.** Kimi requires `trust_remote_code`. Review custom code, pin versions and treat weights as third-party dependencies.

## Frequently asked questions

### Does a model with 104 billion active parameters need the memory of a 104B dense model?

No. It needs memory for all 2.8 trillion parameters, because any expert may be needed next. The active count sets compute and the weight data read per token. Plan memory from the total and speed from the active count.

### Can I run Kimi K3.1 Pro on a workstation?

Not at useful speed. The 4-bit checkpoint is about 1.56 TB, and the baseline deployment is one 8-GPU data-centre node. Offloading experiments report around 2 tokens per second, which suits curiosity, not users.

### Is open weight the same as open source?

No. Open weight means the parameters are downloadable; the licence decides use. Both models this week carry conditions on commercial scale and attribution, and ERNIE adds geography and output-use limits. Open-source licences do not discriminate by user or use.

### Why choose a base model over an instruct model?

Choose a base model to fine-tune on your own data for a narrow task, or for research. It has not learned to follow instructions, so it is not a chatbot as shipped.

### How much does long context really cost?

It depends on attention design. Our illustrative dense model needs about 258 GB of cache for one 1M-token conversation. Kimi's sparse-attention layers need about 19.8 GB per request plus a fixed state, possibly repeated per GPU under tensor parallelism.

### When does self-hosting beat an API?

When data cannot leave your environment, when you need deep customisation, or when steady high traffic keeps GPUs busy. For uneven or modest traffic, an API is usually cheaper and simpler.

## Summary

Moonshot AI released Kimi K3.1 Pro (2.8 trillion parameters, 104 billion active, 1,048,576-token context, about 1.56 TB in MXFP4), and Baidu released ERNIE 5.5 Flash Base (280 billion total, 14 billion active, 33.5 trillion training tokens, about 288 GB in FP8). In both, compute follows the small active count while memory follows the full total. Self-hosting starts with simple sums: parameters × bytes for weights, the attention design for KV cache, and headroom for concurrency. Kimi needs a modern 8-GPU node or more; ERNIE fits one 8-GPU H100 node with reduced context. Both are open weight, not open source, with revenue and attribution conditions, and ERNIE excludes the EU, the UK and South Korea. For background, see our pages on [Kimi K3](/tools/kimi-k3) and [generative AI and LLMs](/topics/generative-ai-llms).

## Sources

- [Hugging Face: moonshotai/Kimi-K3.1-Pro model repository and discussions](https://huggingface.co/moonshotai/Kimi-K3.1-Pro/discussions)
- [LMSYS SGLang Cookbook: Kimi K3.1 Pro](https://lmsysorg.mintlify.app/cookbook/autoregressive/Moonshotai/Kimi-K3.1-Pro)
- [NVIDIA Dynamo documentation: Kimi K3.1 Pro recipes](https://docs.nvidia.com/dynamo/dev/recipes/kimi-k3-1-pro)
- [Runtime Wire: Moonshot AI releases Kimi K3.1 Pro weights for its 2.8-trillion-parameter model](https://runtimewire.com/article/moonshot-ai-kimi-k3-1-pro-weights)
- [Runtime Wire: Baidu releases ERNIE 5.5 Flash Base with a $50M commercial line in the fine print](https://runtimewire.com/article/baidu-ernie-5-5-flash-base-50m-license)
- [VentureBeat: Kimi K3's full weights are here, but they're open with a caveat](https://venturebeat.com/technology/kimi-k3s-full-weights-are-here-but-theyre-open-with-a-caveat-what-enterprises-should-know)
- [Teamo Router: Kimi K3.1 Pro open weights](https://teamorouter.com/blogs/kimi-k3-1-pro-open-weights)
- [AI Revolution: What is ERNIE 5.5 Flash Base (performance, licence, GPU requirements)](https://ai-revolution.co.jp/media/what-is-ernie-5-5-flash-base/)
- [XenoSpectrum: Baidu ERNIE Flash Base and its context length](https://xenospectrum.com/baidu-ernie-flash-base-context/)