Self-hosting coding agents on open-weight models: the cost maths

Coding agents use a lot of tokens. An agent that reads a repository, runs tests, reads the errors and tries again can send the same large context to a model dozens of times for one task. Multiply that by thousands of engineers paying per token and the bill grows fast. In recent weeks several organisations showed another answer: run an open-weight model on your own GPUs and send only the hard cases to a frontier API.

Revolut described moving about 2,000 engineers to a self-hosted GLM model and cutting its AI coding bill to roughly a tenth. AMD announced a second-generation on-premises coding platform built around the same model family. Mistral released open weights for Mistral Large 4. A lawyer built an offline server from eight used GPUs. And Bloomberg reported that Apple has cut some engineers' daily AI spending limits.

Frontier APIs are not finished. But coding has become a large, predictable workload, and that is where owning capacity starts to beat renting it. This article explains what happened, how self-hosting works, how to do the cost maths, and when it is not worth it.

What happened

Revolut details its self-hosted GLM coding stack (reported 3 to 7 November 2026)

David Abitbol, who leads AI operations at Revolut, described the setup at AI Engineer NYC on 13 October 2026. Coverage spread in early November, including RuntimeWire on 3 November and a detailed write-up by Luca Berton on 7 November.

According to these reports, Revolut serves an open-weight GLM model from Z.ai on 56 Nvidia B200 GPUs for about 2,000 engineers. Self-hosting began around April 2026 with GLM-5.1; Berton says the default is now GLM-5.3, though other outlets do not confirm the version. Revolut reports a blended cost of about $0.30 per million tokens at its own utilisation, and an AI invoice about one tenth of its previous spend with Anthropic and OpenAI. Claude Code and Codex remain available for exceptions.

GLM is text-only, so a local router sends image tasks, such as reading a screenshot, to a multimodal model such as Gemini. The endpoints are OpenAI-compatible, so engineers keep tools such as OpenCode and Codex.

Revolut's internal chart credits the self-hosted phase with about 60% faster coding time and 70% faster lead time. Berton and Cloudnews both note that this is not a controlled experiment. On the same 56 GPUs, tuning between April and August cut time to first token from 14.26 to 1.68 seconds and raised requests per second by 76%, per Berton's summary of the talk.

AMD launches Instinct Coder 355 (9 November 2026)

On 9 November AMD introduced Instinct Coder 355, a turnkey on-premises platform for AI-assisted coding. It follows the first Instinct Coder, announced with Supermicro and Spectro Cloud on 5 August 2026 with MI325X GPUs and GLM-5.2.

AMD lists eight Instinct MI355X GPUs with 288 GB of memory each, two EPYC CPUs and Pensando networking in a Supermicro server; some press coverage describes a Dell build instead. The local model is GLM-5.3. Spectro Cloud's PaletteAI Inference Launchpad routes requests locally by default and to approved frontier services (Claude, GPT or Gemini) only when the task and IT policy allow.

AMD says one node supports up to 85 developers, 50 at once, and claims "up to 75% lower TCO", based on an illustrative model covered below. It also claims up to 98% of frontier performance: GLM-5.3 resolved 95.4% of SWE-bench Verified tasks against 97.0% for Claude Opus 5 on a Vals AI leaderboard. Both are vendor claims.

Mistral releases Mistral Large 4 weights (29 October 2026)

Mistral posted Large 4 weights on Hugging Face on 29 October. As reported by Let's Data Science, it is a mixture-of-experts model with 1 trillion parameters and about 49 billion active per token. Mistral says it ties GLM-5.3 at 48% on the Artificial Analysis Coding Agent Index while using 47% fewer tokens per task. The FP8 checkpoint is about 1,060 GB; an NVFP4 build for Nvidia Blackwell is about 610 GB.

A lawyer runs DeepSeek V4.1 Flash on used V100s (9 November 2026)

Tom's Hardware reported on a lawyer who built a server to keep legal files offline: eight used 2018-era V100 GPUs, bought as two four-GPU boards for about $2,400 each, running DeepSeek V4.1 Flash quantised mostly to 2 bits. It reaches up to 215 tokens per second on code and serves one request at a time. The builder estimates 50% to 60% of their AI work can now run locally.

Apple reportedly tightens AI budgets (4 November 2026)

Bloomberg reported, as relayed by 9to5Mac, that Apple cut some engineers' daily AI spending limits from $300 to $200 and is encouraging some teams to use cheaper or local models.

How self-hosting a coding model works

Self-hosting replaces one line in your agent's configuration, the API address, with a production system. It is like moving from eating out to running a staff canteen: cheaper per plate, but you now own the kitchen and the rota.

The serving stack

At the bottom are the GPUs. Above them sits an inference engine such as vLLM or SGLang, which loads the weights, batches requests from many users and streams tokens back through an OpenAI-compatible endpoint. On top come a load balancer, authentication, rate limits, usage metering and logging.

Continuous batching lets one GPU serve many agents by adding requests to a running batch as others finish. Control planes for this layer are appearing: on 4 November Buoyant open-sourced Kinference, an Apache 2.0 control plane for vLLM on Kubernetes with token rate limits, per-tenant cost tracking and routing that knows what each GPU already holds in cache.

Fitting the model

A large open model rarely fits on one GPU. GLM-5.3 has about 753 billion parameters, according to Noze's review: roughly 750 GB at 8 bits per weight, half that at 4 bits. A B200 has about 180 GB of usable memory, so the model is split with tensor parallelism, each GPU holding a slice of every layer.

Weights are not the only thing in memory. Each conversation keeps a KV cache, the model's notes on text it has already read, and long coding contexts make it large. Abitbol reportedly said GLM fits on four B200s but a four-way split can leave too little room for the cache, so eight may serve more users. Weights are the furniture; the KV cache is the floor space people move around in.

Quantisation stores weights in fewer bits. Revolut reports that moving from FP8 to FP4 cut its GPU footprint by about 40% with no accuracy loss on its workload. Treat that as a hypothesis to test on yours.

Caching and speculation

Agents resend the same repository context, system prompt and tool definitions at every step. A stack that recognises the repeated prefix skips recomputing it: prompt caching on your own hardware. Berton's write-up says coding workloads should exceed a 90% cache hit rate. This is the main reason self-hosted agents can be cheap.

Speculative decoding uses a small draft model to guess several tokens that the large model checks in one pass. Code is predictable, so guesses often land. The lawyer's server guesses five tokens ahead, which is why code runs faster than legal prose on it.

Service targets

Revolut set targets from user complaints. Per Berton, time to first token under 3 seconds keeps engineers content; above 5 seconds, complaints reach Slack. About 200 tokens per second suffices. Abitbol's advice was to "resist scaling" and tune before buying GPUs.

Routing between local and frontier models

No single open model is best at everything. The answer in both the Revolut and AMD designs is a router that decides, request by request, where work goes. See LLM routing for the general patterns.

A router works like a hospital triage desk: most patients see the duty doctor, some go to a specialist, and records that must stay in the building never leave. Rules usually combine four checks:

Capability. Does the task need something the local model lacks, such as images?
Sensitivity. Must this code stay on site? Then it stays local.
Difficulty. Has the local model already failed this task or task class? Escalate.
Budget. Has the team used its frontier allowance? Stay local or queue.

AMD lets platform teams define these policies and publishes no thresholds. The simplest working router has a default model, an allowlist of task types that may escalate, a manual override for engineers, and a log of every escalation charged to a team.

Evaluating before you switch

Leaderboards tell you which models to test, not how they perform on your code and your agent harness. AMD's 98% figure comes from one benchmark with one harness. A sound switch has three stages:

Offline evaluation. Collect 100 to 300 real tasks with tests from your own history. Run candidate and current models through the same harness and compare pass rate, tokens and time. The evals page covers how.
Pilot. Give the local model to volunteer teams while frontier tools stay available. Measure completion, review rework and how often people switch back.
Default with an escape hatch. Make the local model the default and watch the escape rate. A rising rate shows where it falls short.

Count tokens per task, not only accuracy. On your own hardware, a model that uses 47% fewer tokens serves far more engineers per GPU. Berton notes Revolut described no formal benchmark; that worked for them but is weak evidence for anyone else.

The cost maths

Cost per million tokens = cluster cost per hour ÷ millions of tokens served per hour

The top is fixed once you have the GPUs. The bottom depends on how busy they are. Self-hosting is like a gym membership: cheap per visit only if you go often.

Step 1: the cluster cost

SemiAnalysis's InferenceX lists B200 costs of about $1.73 per GPU-hour at hyperscaler ownership volume and $3.70 at retail. For 56 GPUs over a 730-hour month:

Basis	Per GPU-hour	56 GPUs per month	Per engineer (2,000)
Volume ownership	$1.73	about $70,700	about $35
Retail cloud	$3.70	about $151,300	about $76
AIChatDaily estimate, via Cloudnews	$6 to $8	$245,000 to $327,000	$120 to $165

These are illustrations, not Revolut's costs, and exclude staff and monitoring. Apple's reported $200 daily cap allows up to $4,000 per engineer over 20 working days.

Step 2: utilisation

InferenceX benchmarks GLM-5.3 at FP4 on fully loaded B200s at about $0.033 per million tokens. Revolut reports $0.30, about nine times higher. That gap is normal: real traffic peaks mid-morning, drops overnight, and long contexts tie up memory. Berton notes the same cluster at half the load costs roughly twice as much per token.

Step 3: tokens per engineer

As an illustration only: a $151,000 monthly cluster delivering tokens at $0.30 per million serves about 500 billion tokens a month, or about 12 million per engineer per working day across 2,000 engineers. Agents reach such volumes because cached context counts every time it is resent. Measure your own figure from API invoices.

Step 4: a worked vendor example

AMD's 75% footnote assumes 85 developers whose frontier use costs $2,000 a month each, $2.04 million a year. The appliance lists at $590,000 including software, spread over three years. Frontier use falls to 10% of the old spend, $204,000 a year, plus $50,000 a year in running costs. The listed items add up to about $451,000 a year; AMD states about $513,500, so it seems to include costs it does not itemise. Either way the saving is 75% to 78%, with about five months' payback.

Everything rests on the first assumption. With the same hardware and running costs, the appliance breaks even only at about $270 a month of previous API spend per developer.

When it is worth it

Situation	Frontier API	Self-hosted open model
A few dozen light users	Usually cheaper	GPUs sit idle
Hundreds to thousands of heavy users	Bill grows with every token	Fixed cost spread widely
Spiky demand	Pay only for use	Pay for peak capacity
Code must stay on site	Needs contracts and trust	Stays in-house
Hardest tasks	Best models available	Route these out
No GPU or platform team	Nothing to run	Hidden staff cost

Licences and security

Read the licence file, not the headline. Noze reports that GLM-5.3 uses Z.ai's near-MIT licence with an extra condition for very large model-as-a-service providers, while GLM-5.3-Flash is plain MIT. Mistral Large 4's modified MIT licence exempts internal use but requires a commercial licence to offer the model to others as a service. Have legal read the exact text for each version.

Running weights is not calling a foreign API. Abitbol separated running open weights on chosen GPUs from sending requests to an endpoint in China. Revolut keeps inference in countries with data-adequacy arrangements and, per Berton, relies on providers with zero data retention by default.

Check the supply chain. Download from the publisher's official repository, verify checksums, prefer safetensors files, and do not run code bundled with a model unless you have reviewed it.

The agent is the bigger risk. A coding agent runs commands. Revolut keeps a human in the loop and a sandbox that allows grep but not gradle, which also limits prompt-injection damage. It cites OpenTelemetry tracing for incidents and usage analytics.

What it means in practice

Coding becomes capacity planning. The question shifts from "which model is smartest" to "what does a million tokens cost at our load".

The default model and the best model can differ. Revolut and AMD both keep frontier models for exceptions and let a router decide.

Open models are close enough for much routine work. Vendor figures put GLM-5.3 and Mistral Large 4 near the frontier on coding benchmarks; the comparison of giant open MoE models shows how fast this tier moves. Close is not identical, so evaluate on your code.

Budgets are tightening. Self-hosting is one response; cost and latency budgets on API use are another. Most firms will use both.

What to do now

Measure spend per engineer. Divide a month of API invoices by active users. Below a few hundred dollars, self-hosting rarely pays.
Measure tokens per task. Log input, cached input and output tokens for a sample of agent runs.
Build an internal eval set. Use 100 to 300 real tasks with tests, run through your actual harness.
Pilot on rented GPUs. Rent a node by the hour, serve a candidate with vLLM or SGLang, and point one team at it before buying hardware.
Tune before scaling. Check cache hit rate, quantisation, parallelism and speculative decoding first.
Put a router in front from day one. Define a default model, escalation rules, a manual override and per-team frontier budgets.
Set service targets from users. Start near 3 seconds to first token and 200 tokens per second, then adjust.
Have legal review each licence, especially if you might offer the model to customers.
Sandbox the agent. Enforce command allowlists, approvals for risky actions and full tracing.
Track the escape rate. Rising switches to frontier tools show where the local model falls short.

Risks and open questions

Most figures are self-reported. Revolut's savings and productivity gains are unaudited; AMD's 75% and 98% figures are vendor models; Mistral's index result is its own claim.

Gains are hard to attribute. Revolut's improvements coincided with engineers no longer rationing tokens. Part of the gain may come from cheap tokens, not the model.

Hidden costs are easy to miss. Per-token figures usually exclude staff, monitoring and idle night-time capacity.

Models change fast. GLM went from 5.1 to 5.3 within months. Each upgrade needs new evaluation, possibly new quantisation and a licence check.

Geopolitics may shift. Abitbol's view that running Chinese weights locally carries little risk is one company's judgement; regulators or customers may differ.

Watch next: independent measurements of self-hosted coding cost and quality, pricing and ship dates for Instinct Coder 355, whether frontier labs cut prices for coding workloads, and whether more employers publish token budgets.

Frequently asked questions

What does self-hosting a coding model mean?

Running an open-weight model on GPUs you own or rent, behind your own endpoint, instead of paying a provider per token. Agent tools point at that endpoint, usually through an OpenAI-compatible interface. You take on the cost and work of running it.

How many GPUs does a model like GLM-5.3 need?

At 4-bit precision the weights need roughly 380 GB, about four B200-class GPUs, plus memory for each user's KV cache. Revolut reportedly found an eight-way split can serve better. Its 56 B200s serve about 2,000 engineers, but your ratio depends on context length and usage.

Is $0.30 per million tokens a typical self-hosted price?

No, because cost per token depends on utilisation. Benchmarks show about $0.03 for GLM-5.3 on fully loaded B200s, while Revolut reports $0.30 at real load. A lightly used cluster can cost more per token than an API.

Do I still need frontier models if I self-host?

Usually yes. Revolut and AMD both route image tasks, hard problems or approved requests to frontier models. The goal is a local default for routine work, not a ban.

Is it safe to run a Chinese open-weight model?

Running the weights on your own hardware keeps code away from the model maker's servers. You still need to verify downloads, review the licence, sandbox the agent and follow your regulators. The final judgement depends on your sector and customers.

Summary

Coding agents have made AI spend large and predictable. Revolut reports serving about 2,000 engineers from a self-hosted GLM model on 56 B200 GPUs at about $0.30 per million tokens, roughly a tenth of its previous bill, with frontier models kept for exceptions. AMD launched Instinct Coder 355, an on-premises GLM-5.3 node with policy-based routing, claiming up to 75% lower TCO in an illustrative model. Mistral released Large 4 weights, and a lawyer showed used V100s can run a capable model offline. Self-hosting pays when usage per engineer is high and steady: measure your spend and tokens, evaluate on your own code, pilot on rented GPUs, and treat every vendor figure as a hypothesis.

Revolut, AMD and Mistral show coding agents moving to self-hosted open-weight models