Amazon SageMaker
Amazon SageMaker is AWS's managed platform for building, training, tuning and deploying machine learning models, with notebooks, pipelines and endpoints.
Overview
Amazon SageMaker is Amazon Web Services' managed platform for machine learning. It provides development environments, managed training jobs, hyperparameter tuning, pipelines, a model registry, monitoring and several ways to host models for prediction. The idea is that AWS runs the servers, containers, scaling and patching, while the team focuses on data, code and models.
An everyday picture: think of a commercial kitchen you can rent by the hour. The ovens, fridges, cleaning and safety checks are handled by the owner. You bring ingredients and recipes, cook as much or as little as you need, and pay for the time you use. SageMaker is that rented kitchen for machine learning: compute and tooling are ready, and the bill depends on what runs.
In December 2024 AWS reorganised the brand. The original machine learning service is now called Amazon SageMaker AI, while the name Amazon SageMaker covers a broader platform that brings data, analytics and AI together, with SageMaker Unified Studio as its shared workspace. This page focuses mainly on the machine learning capabilities.
What it is
| Capability | Purpose |
|---|---|
| Studio and notebooks | Browser-based development environments with managed compute |
| Training jobs | Run training code in containers on chosen instance types, then shut down |
| Automatic model tuning | Search hyperparameters across many training jobs |
| HyperPod | Persistent, resilient clusters for large-scale foundation model training |
| Pipelines | Define and run ML workflows as directed graphs of steps |
| Model Registry | Version models, record approval status and track lineage |
| Feature Store | Online and offline storage of features for training and inference |
| Inference options | Real-time, serverless and asynchronous endpoints, plus batch transform |
| Model Monitor and Clarify | Drift and quality checks, bias detection and explainability |
| JumpStart | A hub of pre-trained and foundation models that can be deployed or fine-tuned |
| Model Cards | Structured documentation of models for governance |
Most capabilities are separate APIs that can be used alone. A team might train elsewhere and use only SageMaker endpoints, or use only training jobs with its own serving.
Where it came from
AWS announced SageMaker at its re:Invent conference in November 2017. The first release offered managed notebooks, built-in algorithms, automatic tuning and managed hosting. Ground Truth for data labelling and Neo for model compilation followed in 2018.
In 2019 AWS launched SageMaker Studio, an integrated development environment, together with Autopilot for automated machine learning, Experiments, Debugger and Model Monitor. Pipelines, Feature Store, Data Wrangler, Clarify and JumpStart arrived in 2020. HyperPod, aimed at training large models on clusters that recover from hardware failures, was introduced in 2023.
At re:Invent in December 2024 the original service was renamed SageMaker AI and the wider SageMaker platform was announced, with Unified Studio reaching general availability in March 2025.
How it works
Containers underneath
Almost everything in SageMaker runs in containers. AWS provides prebuilt images for common frameworks such as PyTorch, TensorFlow, XGBoost and scikit-learn, and for LLM serving engines. Teams can also bring their own images. This is how SageMaker supports so many frameworks with one set of services.
Training
A training job specifies an image, the training script, input data locations in Amazon S3, an instance type and count, and an output location. SageMaker starts the instances, copies or streams the data, runs the script and saves the model artefact to S3. Then it stops the instances, so billing ends with the job. Managed spot training can use spare capacity at lower prices, with checkpointing to survive interruptions.
Tuning
Automatic model tuning launches many training jobs with different hyperparameters, using strategies such as Bayesian search, and picks the best by a chosen metric.
Pipelines and registry
A SageMaker pipeline chains steps such as processing, training, evaluation, condition checks and model registration. When a model passes its checks, it is registered as a new version in a model package group with an approval status. Approving it can trigger a deployment through CI/CD.
Inference
There are four main ways to get predictions:
- Real-time endpoints keep instances running behind an HTTPS endpoint for low-latency requests, with autoscaling.
- Serverless inference scales to zero when idle and charges only for compute used, at the cost of cold starts.
- Asynchronous inference queues requests and writes results to S3, suited to large payloads or long processing.
- Batch transform scores a whole dataset in S3 without a persistent endpoint.
Endpoints support multiple models per endpoint, production variants for traffic splitting, and deployment guardrails for gradual rollouts.
Monitoring
Model Monitor captures a sample of requests and responses, compares them with a baseline from training data and raises alerts in Amazon CloudWatch when data quality or distributions drift.
Key concepts and terms
| Term | Meaning |
|---|---|
| Domain | The Studio environment for a group of users, with shared settings |
| Instance type | The machine size and accelerator, such as a GPU type |
| Training job | A managed, run-to-completion training task |
| Model artefact | The saved model files written to S3 after training |
| Endpoint | A hosted, callable model service |
| Endpoint configuration | The instance types, counts and variants behind an endpoint |
| Production variant | One model version within an endpoint, with a traffic weight |
| Model package group | A registry collection of versions of one model |
| Pipeline | A workflow of ML steps managed by SageMaker |
| Baseline | Reference statistics used by Model Monitor to detect drift |
| IAM execution role | The permissions SageMaker uses to access data and services |
A worked example
This scenario is illustrative. A logistics company predicts delivery delays for parcels.
- Data engineers land daily shipment data in S3. A SageMaker pipeline runs each night: a processing step builds features, a training step fits a gradient-boosted model on one CPU instance for about 20 minutes, and an evaluation step computes mean absolute error.
- A condition step compares the new error with the current model. If it improves by at least 2 per cent, the model is registered with status "pending approval".
- A reviewer checks the evaluation report and Clarify bias summary, then approves. Approval triggers a deployment pipeline.
- The model is deployed to a real-time endpoint with two instances and autoscaling up to six, as a new production variant receiving 10 per cent of traffic. After two days without regressions, it takes all traffic.
- Model Monitor samples 20 per cent of requests. When a new warehouse opens and a location feature shifts, a drift alert fires and the team schedules a retrain.
- A separate monthly batch transform job scores 30 million historical parcels for reporting, without touching the live endpoint.
Strengths and limitations
Strengths:
- Fully managed infrastructure with pay-for-what-you-use billing.
- Wide range of instance types, including GPUs and AWS accelerators.
- Deep integration with S3, IAM, CloudWatch and other AWS services.
- Several inference options for different latency and cost needs.
- End-to-end capabilities, from labelling to monitoring, under one account.
Limitations:
- Ties workflows to AWS APIs and services, which makes moving away harder.
- The number of features and options has a steep learning curve.
- Costs can grow quietly from idle endpoints and notebooks left running.
- Debugging inside managed containers is less direct than on your own machine.
- The 2024 rebranding means names in older guides may not match the console.
How it compares with alternatives
| Platform | Type | Notes |
|---|---|---|
| SageMaker | Managed AWS ML platform | Broadest AWS integration |
| Vertex AI | Managed Google Cloud ML platform | Strong with BigQuery and Gemini models |
| Databricks | Data and AI platform on several clouds | Built around the lakehouse and MLflow |
| Kubeflow | Open-source platform on Kubernetes | Portable, but self-operated |
| MLflow | Open-source lifecycle tool | Tracking and registry; SageMaker also offers managed MLflow |
Practical guidance and best practices
- Use pipelines and the registry from the start, so every model has lineage and an approval record.
- Give each workload a narrowly scoped IAM role.
- Choose the inference option by traffic pattern: serverless for sporadic, real-time for steady, asynchronous for large or slow requests.
- Set autoscaling policies and minimum counts on real-time endpoints.
- Use managed spot training with checkpoints for long training jobs.
- Shut down idle notebooks and endpoints automatically, and tag resources for cost tracking.
- Turn on data capture and Model Monitor for every production endpoint.
- Keep training and inference images pinned to specific versions.
Common mistakes
- Leaving GPU endpoints running for testing and paying for them for weeks.
- Using one broad IAM role for every job and user.
- Skipping baselines, so drift monitoring has nothing to compare against.
- Deploying straight to 100 per cent of traffic instead of using variants.
- Choosing serverless inference for large models that cannot tolerate cold starts.
- Storing data in many unmanaged S3 locations without versioning or lifecycle rules.
Frequently asked questions
What is the difference between SageMaker and SageMaker AI?
Since December 2024, SageMaker AI is the name of the original machine learning service: training, tuning, endpoints and so on. Amazon SageMaker now refers to a broader platform that also brings in data and analytics tools, with Unified Studio as a shared workspace.
Can I bring my own container?
Yes. Training and inference can use your own images, provided they follow SageMaker's conventions for where inputs, outputs and the inference server live.
Does SageMaker support large language models?
Yes. JumpStart offers foundation models to deploy or fine-tune, prebuilt images support LLM serving engines, and HyperPod targets large-scale training. Amazon Bedrock is a separate AWS service for calling hosted foundation models through an API.
How is SageMaker billed?
Mostly by instance time for notebooks, training, processing and endpoints, plus storage and data processing. Serverless inference is billed by compute duration and data. Prices vary by region and instance type.
Can I use MLflow with SageMaker?
Yes. SageMaker offers managed MLflow tracking servers, and MLflow models can be deployed to SageMaker endpoints.
Glossary
- re:Invent: AWS's annual conference, where many SageMaker features were announced.
- Spot capacity: spare AWS compute sold at a discount that can be reclaimed.
- Data capture: recording endpoint requests and responses for monitoring.
- Cold start: the delay when a serverless endpoint starts from zero.
- Deployment guardrails: controlled traffic shifting with automatic rollback.
- Built-in algorithm: an AWS-provided implementation run without writing training code.
- Unified Studio: the workspace for the broader SageMaker data and AI platform.
Summary
Amazon SageMaker offers managed building blocks for the whole machine learning lifecycle on AWS: notebooks, training, tuning, pipelines, a registry, monitoring and four styles of inference, all running containers on infrastructure AWS operates. Launched in 2017 and reorganised in 2024 into SageMaker AI within a broader SageMaker platform, it suits teams already invested in AWS who want to trade some portability for less operational work. The keys to using it well are pipelines with approvals, careful IAM, the right inference option for each workload and strict control of idle resources.
Related
It belongs to MLOps in my stack, under MLOps and AI infrastructure. Others in the same group: