Google Vertex AI
Vertex AI is Google Cloud's unified platform for training, tuning, deploying and monitoring ML models and building generative AI apps with Gemini and other models.
Overview
Vertex AI is Google Cloud's platform for machine learning and generative AI. It brings together tools for preparing data, training and tuning models, running pipelines, registering and deploying models, monitoring them in production, and building applications on foundation models such as Gemini. Google runs the infrastructure, including GPUs and its own Cloud TPU chips, so teams can focus on models and applications.
An everyday picture: a large shared workshop in a business park. One area has workbenches and hand tools for prototypes, another has heavy machinery for production runs, a stockroom keeps finished parts catalogued, and a front desk takes customer orders. Everything is under one roof with one access card and one bill. Vertex AI is that workshop for AI work on Google Cloud.
The platform has been renamed as it has grown. In April 2026 Google announced the Gemini Enterprise Agent Platform, which it describes as the evolution of Vertex AI: it takes in Vertex AI's capabilities and adds tools for building, running and governing AI agents. The Vertex AI name is still widely used in existing projects, code and guides, and the machine learning capabilities described here remain part of the platform.
What it is
| Capability | Purpose |
|---|---|
| Workbench | Managed JupyterLab notebooks connected to Cloud Storage and BigQuery |
| Training | Custom training jobs in containers on CPUs, GPUs or TPUs |
| AutoML | Train models for tabular, image, text and video data without writing model code |
| Vizier | Hyperparameter tuning and black-box optimisation service |
| Pipelines | Managed orchestration of ML workflows, compatible with Kubeflow Pipelines definitions |
| Experiments and Metadata | Track runs, parameters, metrics and lineage |
| Model Registry | Store, version and manage models |
| Endpoints and batch prediction | Online serving and large offline scoring jobs |
| Feature Store | Manage and serve features for training and inference |
| Model Monitoring | Detect training-serving skew and drift in production inputs |
| Model Garden | A catalogue of Google, third-party and open models to call, tune or deploy |
| Studio and agent tools | Prompt design, tuning and evaluation for generative models, and services for agents |
Teams can mix these freely. One might use only Gemini models through the API; another might run classic models through pipelines and endpoints with no generative AI at all.
Where it came from
Before Vertex AI, Google Cloud offered several separate machine learning services, including AI Platform and AutoML products. Google announced Vertex AI at its I/O conference in May 2021 and made it generally available that month, unifying those offerings under one API and console.
Generative AI arrived in 2023, when foundation models and tooling for prompting and tuning were added, and Model Garden became the place to find and deploy models from Google and others. Gemini models followed, and agent-building tools were added in 2024 and 2025. In April 2026 Google announced the Gemini Enterprise Agent Platform as the next stage of the platform, adding agent runtime, identity, gateway, registry, evaluation and observability services on top of the existing capabilities.
How it works
Data and development
Data usually lives in BigQuery or Cloud Storage. Workbench notebooks run close to the data with chosen machine types and accelerators, so exploration can read large tables without copying them to a laptop.
Training
A custom training job specifies a container image or a prebuilt framework image, the training code, the machine and accelerator type, and output locations. Vertex AI starts the machines, runs the job, writes the model to Cloud Storage and stops. Distributed jobs can use several worker pools, and TPU slices can be used for large models. AutoML is an alternative where Google chooses and trains the model from labelled data.
Pipelines
Pipelines are written with the Kubeflow Pipelines SDK or TensorFlow Extended and run on a managed, serverless backend. Each run records its artefacts and lineage in Vertex ML Metadata, so it is possible to trace a deployed model back to its data and code.
Registry and deployment
Models are uploaded to the Model Registry with versions and aliases. Deploying a model to an endpoint attaches it to dedicated machines with autoscaling between a minimum and maximum replica count. Several models can share an endpoint with a traffic split, which supports canary releases. Batch prediction jobs score files or tables without an endpoint.
Monitoring
Model Monitoring samples prediction requests and compares feature distributions with the training data (to find skew) or with earlier serving data (to find drift), sending alerts when thresholds are crossed.
Generative AI
Gemini and other models in Model Garden are called through managed APIs, with no infrastructure to run. Studio supports prompt design and comparison. Models can be tuned on an organisation's data, grounded in search or enterprise data, and evaluated with automatic metrics. Open models from Model Garden can also be deployed to dedicated endpoints using serving engines such as vLLM.
Key concepts and terms
| Term | Meaning |
|---|---|
| Project | The Google Cloud container for resources, billing and permissions |
| Region | The location where Vertex AI resources run and data is stored |
| Custom training job | A managed training run using your own code |
| Worker pool | A group of identical machines within a training job |
| Model resource | An entry in the Model Registry pointing to model files and a serving container |
| Endpoint | A managed online prediction service |
| Traffic split | Percentages of requests routed to models on an endpoint |
| Batch prediction | Offline scoring of a large dataset |
| Training-serving skew | Difference between training data and production inputs |
| Model Garden | The catalogue of available foundation and open models |
| Service account | The identity jobs use to access other Google Cloud resources |
A worked example
This scenario is illustrative. A media company predicts which subscribers are likely to cancel and also wants short, personalised retention messages.
- Subscriber activity sits in BigQuery. A Vertex pipeline runs weekly: it extracts 90 days of features, trains a gradient-boosted model in a custom training job on one machine for about 15 minutes, and evaluates it.
- If the new model's AUC beats the registered version by at least 0.005, the pipeline uploads it to the Model Registry and deploys it to the existing endpoint with a 20 per cent traffic split.
- Model Monitoring watches the endpoint. When a pricing change shifts the "months subscribed" distribution, a drift alert prompts an early retrain.
- For subscribers scored above 0.7, a nightly batch job calls a Gemini model with a prompt template that uses the subscriber's plan and viewing genres, producing a draft message for the marketing team to review.
- The team evaluates 200 sample messages with automatic metrics and human review before enabling the flow, and logs prompt versions so they can compare changes later.
Strengths and limitations
Strengths:
- One platform for classic ML and generative AI on Google Cloud.
- Close integration with BigQuery, Cloud Storage and Google identity and security.
- Access to Gemini models and a large catalogue of other models.
- Managed pipelines compatible with Kubeflow Pipelines definitions.
- Choice of GPUs and TPUs for training and serving.
Limitations:
- Ties workflows to Google Cloud services and APIs.
- Many overlapping products and frequent renaming can confuse new users.
- Dedicated endpoints bill while running, even with low traffic.
- Some features and models are available only in certain regions.
- Quotas for accelerators may need to be requested in advance.
How it compares with alternatives
| Platform | Type | Notes |
|---|---|---|
| Vertex AI | Managed Google Cloud platform | Strong with BigQuery, Gemini and TPUs |
| SageMaker | Managed AWS platform | Broadest AWS integration |
| Databricks | Data and AI platform across clouds | Lakehouse-centred, built around MLflow |
| Kubeflow | Open-source platform on Kubernetes | Portable but self-operated |
| Hugging Face | Model hub and hosted inference | Open model ecosystem first |
Practical guidance and best practices
- Keep data, training and endpoints in the same region to reduce latency, cost and compliance risk.
- Use pipelines with metadata from the start, so lineage is recorded automatically.
- Give each pipeline and endpoint its own narrowly scoped service account.
- Set minimum and maximum replica counts on endpoints, and use batch prediction for offline work.
- Use traffic splits for every model change instead of replacing models in place.
- Turn on Model Monitoring for production endpoints, with thresholds tied to business impact.
- Evaluate generative outputs with a fixed test set before and after every prompt or model change.
- Request accelerator quota early for large training or serving plans.
Common mistakes
- Leaving test endpoints deployed and paying for idle machines.
- Mixing regions so that data has to cross them on every request.
- Using the default service account with broad permissions for everything.
- Treating AutoML as a substitute for understanding the data and its labels.
- Changing prompts in production without versioning or evaluation.
- Following old guides whose product names no longer match the console.
Frequently asked questions
Is Vertex AI being replaced?
In April 2026 Google announced the Gemini Enterprise Agent Platform as the evolution of Vertex AI. It includes Vertex AI's capabilities and adds agent-focused services. Existing projects and code built on Vertex AI continue to use those capabilities, but check current Google documentation for naming and migration details.
Can I use open models, not just Gemini?
Yes. Model Garden includes many open and third-party models. Some are available as managed APIs; others can be deployed to your own endpoints.
Does Vertex AI run Kubeflow pipelines?
Vertex AI Pipelines runs pipelines written with the Kubeflow Pipelines SDK, on a managed backend, so you do not run Kubeflow yourself.
What is the difference between online and batch prediction?
Online prediction serves single requests in real time from a deployed endpoint. Batch prediction scores a whole file or table as a job and needs no endpoint, which is usually cheaper for offline work.
Can I train on TPUs?
Yes. Custom training jobs can run on TPU slices as well as GPUs, which suits large models built with frameworks such as JAX or PyTorch with XLA support.
Glossary
- AutoML: automated model selection and training from labelled data.
- Vizier: Google's service for hyperparameter and black-box optimisation.
- ML Metadata: the store of artefacts and lineage for pipeline runs.
- Grounding: connecting model answers to search results or trusted data.
- Tuning: adapting a foundation model to a task with additional data.
- Quota: a per-project limit on resources such as accelerators.
- Workbench: managed notebooks for development on Google Cloud.
Summary
Vertex AI unifies Google Cloud's machine learning and generative AI tools: notebooks, training on GPUs and TPUs, AutoML, pipelines, a registry, endpoints, monitoring, a feature store and access to Gemini and many other models. Launched in 2021 and folded in April 2026 into the Gemini Enterprise Agent Platform, which adds agent services, it suits teams whose data already lives in Google Cloud. Good results come from keeping work in one region, recording lineage with pipelines, scoping permissions tightly and evaluating every model and prompt change.
Related
It belongs to MLOps in my stack, under MLOps and AI infrastructure. Others in the same group: