# Confidential inference reaches TPUs with Google's TPU 8tx, TPU 8i Max and Cloud Pathways

On 9 November Google announced TPU 8tx, a TPU system with hardware-enforced confidential computing on AMD EPYC hosts, a denser TPU 8i Max, and general availability of its Pathways runtime. Here is how protecting data in use works on accelerators, what it costs, and what regulated teams should do.

Most organisations already encrypt data on disk and on the network. The weak point is the moment data is used. To run a model on a patient record, the record must be decrypted in memory, where anyone with enough control of the machine, such as a compromised hypervisor or a rogue administrator, could in principle read it. Confidential computing closes that gap with hardware. Until now it has mostly lived on CPUs, with a few GPU options. On 9 November 2026 Google brought it to its TPUs.

Google announced two new eighth-generation TPU systems. TPU 8tx is built for "secure and confidential inference" and pairs TPU 8t silicon with 5th Gen AMD EPYC host processors. TPU 8i Max puts two TPU 8i chips in one package for large context windows. On the same day Google made Cloud Pathways, the runtime DeepMind built to train and serve Gemini, generally available on Google Kubernetes Engine (GKE).

The announcements meet at one question for any team running AI on sensitive data: can you get accelerator performance without trusting everyone who touches the hardware? This article explains what was announced, how confidential computing stretches from a CPU to an accelerator, what single-controller orchestration means, and what regulated teams should do now. For background on the chips, see the [Cloud TPU page](/tools/cloud-tpu).

## What happened

### Google announces TPU 8tx and TPU 8i Max (9 November 2026)

In a post by Amin Vahdat, Google added two systems to the TPU 8t and TPU 8i it introduced at Google Cloud Next in April, making four TPU systems in one year.

**TPU 8tx** is "built on the same silicon as TPU 8t and paired with 5th Generation AMD EPYC CPU hosts". Google says it keeps "sensitive data and model weights protected and isolated even while in active use" and aims it at "regulated enterprises and governments". It uses the low-latency Boardfly network designed for TPU 8i, and Google claims "up to 2x lower latency compared to TPU 8t with a near-zero performance penalty". Google also pitches it for private fine-tuning, distillation and reinforcement learning without moving sensitive data between clusters.

**TPU 8i Max** "combines two low-latency TPU 8i chips onto a single package". Compared with a single TPU 8i, Google says it doubles compute, on-chip SRAM and HBM bandwidth, and fits twice the silicon in the existing rack footprint. For scale: at its April launch, a single TPU 8i was rated at 10.1 PFLOPS of FP4 compute with 288 GB of HBM and 384 MB of SRAM, as reported by Tom's Hardware.

Google published no absolute specifications, prices, regions or dates for either system, and there are no independent benchmarks. Data Center Knowledge reported that select customers will be able to deploy TPU 8tx in their own data centres. Google also highlighted TPU Sync, an open-source layer for moving model state such as the [KV cache](/tools/kv-cache) between HBM, host memory and storage, which it claims cuts KV transfer latency and time to first token by "more than 90%".

### AMD describes how the confidential path works (9 November 2026)

AMD's press release the same day gives more technical detail than Google's. It calls TPU 8tx "the first cloud Tensor Processing Unit (TPU) system to deliver hardware-enforced confidential computing". In TPU 8tx, the EPYC hosts handle workload orchestration, data preparation and communication with the TPUs.

The key paragraph says the system uses AMD Secure Encrypted Virtualization (SEV) which, "together with the TPU, extends confidential computing protection beyond the CPU to connected accelerators". AMD says the system verifies "an accelerator's identity and security configuration before granting it direct access to protected memory", which helps "prompts, model weights and intermediate data remain encrypted in CPU memory and in transit among the host and accelerators, with hardware isolation during accelerator execution." AMD also notes that its Infinity Guard features must be enabled by the cloud provider or server maker to operate.

### Cloud Pathways becomes generally available on GKE (9 November 2026)

Google Cloud announced general availability of Cloud Pathways on GKE as part of its AI Hypercomputer. Google says customers ran more than 130 million Cloud TPU chip-hours on Pathways on GKE in the previous 90 days. It is available in all regions that support TPU v5e, v5p, Trillium (v6e) and Ironwood (TPU7x), and supports JAX 0.10.1 and later. A Cluster Director integration that brings Pathways to Slurm clusters is in preview.

The release includes a single controller built on the open-source Interim Framework Runtime (IFRT), Python sidecars for data loading and checkpointing on TPU hosts, elastic training that uses host-memory snapshots to reshard down to N-1 slices when hardware fails, and an interactive notebook mode.

## How confidential computing works

The Confidential Computing Consortium, a Linux Foundation project, defines confidential computing as "the protection of data in use by performing computation in a hardware-based, attested Trusted Execution Environment." That sentence has three working parts.

### Trusted execution environments

A trusted execution environment (TEE) is an area of a machine that the processor itself walls off from everything else, including the operating system and the hypervisor that normally have power over all software. In cloud settings the usual unit is a whole virtual machine. AMD SEV, and its newer SEV-SNP form, and Intel TDX both work this way.

Picture a safe-deposit box. The bank owns the building and moves the boxes, but cannot open yours. The cloud provider still schedules and powers your VM; it just cannot look inside.

### Memory encryption

The processor encrypts the TEE's memory with a key that lives inside the chip and never leaves it. When the VM reads its own memory, the processor decrypts it on the way in. If anyone else reads the same physical memory, for example the hypervisor or someone with a memory probe, they see ciphertext. Newer versions also add integrity protection, so the host cannot quietly replace or replay memory pages.

### Attestation

Encryption alone is not enough. How do you know your code is really running inside a genuine TEE, with the firmware you expect, and not in a simulation? Attestation answers that. The hardware produces a signed report that describes what was loaded, signed by a key rooted in the chip maker. You, or a key service acting for you, check the report before releasing secrets such as decryption keys or model weights.

It is like checking a courier's identity card and the seal on the van before handing over a parcel. It turns "the provider says it is secure" into "the hardware proved it before my keys were released".

### Why inference on sensitive data needs it

Inference is where sensitive data meets the model in plain form. A prompt can contain a medical history or a customer's finances, and the weights may be the most valuable thing a company owns. Without confidential computing, you protect these with contracts and access controls. With it, you add a technical control on who can see data in use, which matters under [GDPR](/tools/gdpr), Turkey's [KVKK](/tools/kvkk) and sector rules in finance, health and government. It does not replace those legal duties; it is one safeguard you can point to.

## How it extends from the CPU to an accelerator

A CPU TEE protects the CPU's memory. But in AI inference the heavy work happens on a GPU or TPU, which has its own memory (HBM) and talks to the host over a bus. If the data leaves the encrypted CPU memory in plain form to reach the accelerator, the protection ends at exactly the wrong place. Extending the TEE needs three things.

### Attested devices

The CPU-side TEE must trust the accelerator before it hands over data. That means the accelerator also proves its identity and its firmware, much like the CPU attestation above. AMD's description of TPU 8tx says exactly this: the system verifies "an accelerator's identity and security configuration before granting it direct access to protected memory". On NVIDIA's H100, a dedicated confidential computing mode and GPU attestation play the same role.

### Encrypted links

Data moving between host and accelerator must be encrypted. On H100 the approach, described in Google Cloud's 2024 post on confidential accelerators, uses an encrypted "bounce buffer" in shared memory: the driver inside the CPU TEE encrypts data and commands, places them in the buffer, and the GPU decrypts them on arrival. Newer designs aim to encrypt the link itself and let trusted devices access protected memory directly. AMD's wording for TPU 8tx, with data "encrypted in CPU memory and in transit among the host and accelerators", describes that goal, though neither company has published a detailed design.
### Isolation on the device

Finally, while the model runs, the accelerator's own memory must be isolated from other tenants and from the host. AMD describes "hardware isolation during accelerator execution" for TPU 8tx. On H100 the GPU's HBM is inside the protected boundary while in confidential mode.

### How the options compare

| Option | Host TEE | Accelerator | Status on Google Cloud |
|---|---|---|---|
| Confidential VM (CPU only) | AMD SEV / SEV-SNP or Intel TDX | None | Available |
| Confidential VM with H100 (A3 High) | Intel TDX | NVIDIA H100 in confidential mode | Documented, Spot and flex-start provisioning |
| Confidential VM with RTX PRO 6000 (G4) | AMD SEV | NVIDIA RTX PRO 6000 | Documented, extra NVIDIA licence fee |
| TPU 8tx | AMD SEV on 5th Gen EPYC | TPU 8t silicon | Announced, no date or region |

## What it costs and where it stops

### The performance cost

Most of the cost sits on the bus, not on the chip. A 2024 benchmark study of confidential computing on NVIDIA H100 and H200 GPUs found that compute inside the GPU was largely unaffected, while encryption of CPU-GPU transfers added overhead. The authors report average overhead below 9%. Throughput overhead on H100 was 6.85% for Llama-3.1-8B at batch size 1 and close to zero for a 4-bit Llama-3.1-70B. Time to first token suffered more, around 19% for the 8B model, because short, I/O-heavy steps feel the encryption most. The pattern is intuitive: the more work the chip does per byte it receives, the less the encrypted transfer matters.

Google claims TPU 8tx has a "near-zero performance penalty", but has published no method, so nobody outside Google and AMD can check it yet.

### The operational cost

Confidential deployments are more work. You need an attestation service, key management tied to attestation results, firmware and drivers at supported versions, and debugging without the usual host-level tools. Google's documentation for confidential GPU VMs lists constraints such as specific driver versions, kernel crypto settings and not flashing GPU firmware. Expect a similar list for TPU 8tx.

### The limits of the protection

Confidential computing protects against the infrastructure, not against your own application. If your service logs prompts in plain text, writes outputs to an unprotected database or has a prompt-injection hole, the TEE does nothing.

Side channels such as timing analysis remain an active research area, and past CPU TEEs have needed security patches. You still trust the chip makers, their firmware and their signing keys. Confidential computing shortens the list of parties you trust; it does not empty it. It belongs inside a [zero trust](/tools/zero-trust) design, alongside key management with tools such as [HashiCorp Vault](/tools/hashicorp-vault).

## Single-controller orchestration and Pathways

The Pathways announcement is about a different problem: how to drive thousands of chips as one program.

### Multi-controller SPMD

The usual way to run JAX on many accelerators is SPMD: single program, multiple data. Every host runs its own Python process, each process runs the same program on its local chips, and they stay in step. They coordinate through collective operations such as all-reduce. Google's Developers Blog puts the weakness plainly: "In SPMD, one lost machine stops every process and sends the run back to its last checkpoint." It also makes jobs with different programs on different chips, such as split prefill and decode servers or reinforcement learning loops, awkward to build.

SPMD is an orchestra with no conductor, every player counting bars from the same score. If one stops, everyone stops, and asking half to play a different piece is hard.

### Single-controller

Pathways uses one client process that sees all the slices as if they were attached locally. That client describes the whole computation as a graph. A runtime decides where each piece runs, moves and reshards tensors, and orders the work. This is the conductor model. The obvious worry is that one client becomes a bottleneck. Google says Pathways avoids this by dispatching work asynchronously, so coordination overlaps with computation already running on the chips.

The practical gains are resilience and flexibility. When hardware fails, Pathways can remap work onto healthy chips and, with host-memory snapshots, shrink to N-1 slices instead of crashing. Different programs can share a pool of slices. One cost remains: traffic between slices runs over the data centre network, which is much slower than the interconnect inside a slice. The Developers Blog cites 1,200 GB/s inside a TPU v5p group against 50 Gb/s to the data centre network.

Google has not said whether Pathways will run on TPU 8tx. But if a central runtime moves your model state across slices, confidentiality has to cover every hop, which is a fair question for any provider. For how [JAX](/tools/jax) programs are split across devices, see [data parallelism](/tools/data-parallelism) and [tensor parallelism](/tools/tensor-parallelism).

## What it means in practice

**Confidential AI is becoming a standard option.** On Google Cloud the accelerator choice was NVIDIA GPUs in confidential VMs. TPU 8tx adds a TPU route. Expect procurement teams to ask for it by default.

**The host CPU matters again.** Confidential accelerators depend on the host's TEE, so the host platform, AMD SEV or Intel TDX, shapes your attestation tools and key flows.

**Model owners gain a deployment pattern.** Because weights are protected too, running a proprietary model next to a customer's data becomes more realistic, in the cloud or, if the Data Center Knowledge report holds, on the customer's premises.

**Long context is pushing on memory.** TPU 8i Max adds memory bandwidth per package; TPU Sync speeds up moving KV state between memory tiers. See [AI infrastructure bottlenecks](/articles/ai-infrastructure-bottlenecks).

**Fleet orchestration is becoming a product.** With Pathways on GKE, a DeepMind-internal technique now runs on customers' [Kubernetes](/tools/kubernetes) clusters.

## What to do now

1. **Map where sensitive data is in use.** List every point where prompts, outputs, embeddings and weights are decrypted: servers, caches, logs, evaluation jobs and vector stores.
2. **Classify workloads by trust need.** Decide which must be protected from the infrastructure operator and which are served by encryption at rest and in transit plus access control.
3. **Pilot what exists today.** Test confidential GPU VMs with a representative model and measure throughput and time to first token with protection on and off.
4. **Bind keys to attestation.** Release weights and data keys only after a successful attestation check. Otherwise a TEE is a sealed room with the key taped to the door.
5. **Audit the application layer.** Remove plain-text prompt logging, encrypt caches and outputs, and defend against prompt injection.
6. **Ask providers precise questions.** Which parties are trusted, how device attestation works, whether links are encrypted in hardware or through bounce buffers, the measured overhead for your model size, and which regions and certifications apply.
7. **Update your compliance file.** Record confidential computing as a safeguard in GDPR or KVKK impact assessments and, where relevant, [EU AI Act](/tools/eu-ai-act) risk assessments, with its limits stated.
8. **Test failure in large jobs.** If you run multi-slice JAX, force a hardware failure and check Pathways' elastic recovery before relying on it.
9. **Wait for independent numbers.** Do not size a migration on "near-zero penalty" or "2x lower latency" until someone has reproduced them.

## Risks and open questions

**There are no specifications or dates for the new TPUs.** Google gave relative claims only. Absolute performance, prices, regions and availability for TPU 8tx and TPU 8i Max are unknown.

**The confidential design is not yet public.** Neither company has published how device attestation, link encryption and key handling work in TPU 8tx. Security teams will need that before approval.

**"Near-zero penalty" is untested.** GPU studies show overheads that vary with model size. TPU 8tx may do better, but there is no independent evidence.

**Side channels and firmware bugs remain.** Plan for firmware updates and re-attestation as routine operations.

**Pathways adds a central component.** A single controller simplifies programming but is a new critical service to monitor, and its interaction with confidential hardware is unspecified.

**Watch next:** availability dates and regions for TPU 8tx and TPU 8i Max, published attestation documentation, any independent overhead measurements, whether Pathways and Confidential GKE support TPU 8tx, and whether other clouds offer comparable confidential accelerators.

## Frequently asked questions

### What is confidential inference?

Running inference inside a hardware-protected, attested environment so that prompts, outputs and weights stay encrypted in memory and in transit, even from the cloud operator. It protects data while in use, not only when stored or sent.

### Does confidential computing make my AI system compliant with GDPR or KVKK?

No. It is a strong technical safeguard, but compliance also depends on lawful basis, purpose limits, retention, access rights and how your application handles data.

### How much slower is confidential inference?

It depends on the hardware and the model. A published study on NVIDIA H100 and H200 found average overhead below 9%, near zero for a large model and higher for time to first token on small models. Google claims near-zero penalty for TPU 8tx but has not published measurements.

### What is the difference between Pathways and normal multi-host JAX?

In normal multi-host JAX, every host runs its own copy of the program in lockstep, and a single failure stops the job. Pathways uses one controller that sees all slices and schedules work across them, which makes mixed workloads and recovery from failures easier.

### Can I use TPU 8tx today?

Not yet, as far as public information shows. Google gave no dates, prices or regions. Confidential GPU VMs are the documented option on Google Cloud today.

## Summary

On 9 November 2026 Google announced TPU 8tx, which pairs TPU 8t silicon with 5th Gen AMD EPYC hosts to bring hardware-enforced confidential computing to TPUs, and TPU 8i Max, which packages two TPU 8i chips. It also made Cloud Pathways, its single-controller runtime, generally available on GKE. Confidential computing protects data in use through TEEs, memory encryption and attestation; on accelerators it also needs attested devices, encrypted links and device isolation. It guards against the infrastructure, not weak application design, and its cost depends on how much data crosses the bus. Google's claims are unverified and dates unknown, so regulated teams should map where data is in use, pilot what exists, bind keys to attestation and question providers closely.

## Sources

- [Google Cloud: Our newest TPUs, 8tx and 8i Max, built for the agentic era](https://cloud.google.com/blog/topics/ai-infrastructure/tpu-8tx-and-tpu-8i-max)
- [Google: Introducing TPU 8tx and TPU 8i Max](https://blog.google/innovation-and-ai/infrastructure-and-cloud/global-network/tpu-8i-max-and-8tx/)
- [AMD (GlobeNewswire): AMD EPYC Expands Its Leadership at the Foundation of AI](https://www.globenewswire.com/news-release/2026/11/09/3398340/0/en/amd-epyc-expands-its-leadership-at-the-foundation-of-ai.html)
- [Google Cloud: Cloud Pathways, a distributed ML runtime for massive accelerator fleets, now GA](https://cloud.google.com/blog/topics/ai-infrastructure/cloud-pathways-a-distributed-ml-runtime-is-ga)
- [Google Developers Blog: What is Pathways? Single-controller training for large models](https://developers.googleblog.com/what-is-pathways-single-controller-training-for-large-models/)
- [Data Center Knowledge: Google adds TPU 8tx, 8i Max for enterprise AI](https://www.datacenterknowledge.com/data-center-chips/google-adds-tpu-8tx-8i-max-for-enterprise-ai)
- [Tom's Hardware: Google splits its TPU into two chips for the first time](https://tomshardware.com/tech-industry/semiconductors/google-splits-its-tpu-into-two-chips-for-the-first-time-with-training-and-inference-variants)
- [Confidential Computing Consortium: About](https://confidentialcomputing.io/about/)
- [Google Cloud: How confidential accelerators can boost AI workload security](https://cloud.google.com/blog/products/identity-security/how-confidential-accelerators-can-boost-ai-workload-security)
- [Google Cloud documentation: Create a Confidential VM instance with GPU](https://docs.cloud.google.com/confidential-computing/confidential-vm/docs/create-a-confidential-vm-instance-with-gpu)
- [arXiv: Confidential Computing on NVIDIA Hopper GPUs, a performance benchmark study](https://arxiv.org/html/2409.03992v4)