Docker

Overview

Docker is a set of tools for packaging software into containers and running them. A container bundles an application with everything it needs, such as libraries, system packages, configuration and runtime, into a standard unit called an image. The same image runs the same way on a developer's laptop, a test server, a cloud machine or a Kubernetes cluster, which removes the familiar problem of software that works on one machine but not another.

An everyday picture: the shipping container. Before standard containers, goods were loaded loose, and every port, ship and lorry handled them differently. Once everything went into boxes of the same size with the same fittings, cranes, ships and trains could move any cargo without caring what was inside. Docker images are standard boxes for software. Build tools fill them, registries store them, and any machine with a container runtime can run them.

Containers are lighter than virtual machines. A virtual machine includes a whole operating system; a container shares the host's kernel and isolates only the application's processes, files and network. Containers therefore start in seconds and pack densely onto machines. For machine learning, Docker is the usual way to make training and serving environments reproducible, with exact versions of Python, frameworks, CUDA libraries and model code.

What it is

Docker images follow the Open Container Initiative (OCI) standards, so they also run on other OCI-compatible runtimes, such as containerd, which Kubernetes commonly uses.

Where it came from

Linux had the building blocks of containers for years: namespaces to isolate processes and cgroups to limit resources. Using them was difficult. Solomon Hykes and his colleagues at dotCloud, a platform-as-a-service company, built a tool that made containers easy, and Hykes presented Docker publicly at PyCon in March 2013, releasing it as open source. The company soon renamed itself Docker, Inc.

Adoption was rapid. Docker Hub made sharing images simple, and Dockerfiles made builds repeatable. In 2015 Docker and other companies founded the Open Container Initiative to standardise image and runtime formats. Docker's core runtime was split out as containerd and donated to the Cloud Native Computing Foundation in 2017, and the open-source code base became the Moby project. In 2021 Docker changed the terms of Docker Desktop so that larger businesses need a paid subscription, while the engine remains open source. More recently Docker has added tools for AI development, such as running models locally.

How it works

Building images

A Dockerfile lists steps. Each instruction that changes the file system creates a layer.

FROM python:3.12-slim
WORKDIR /app
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
COPY . .
USER 1000
CMD ["python", "serve.py"]

docker build -t myapp:1.0 . runs these steps. Layers are cached: if requirements.txt has not changed, the dependency installation is reused, and only the final copy step runs again. Ordering instructions from least to most frequently changing makes builds fast.

Layers and content addressing

An image is a stack of read-only layers plus metadata. Each layer is identified by a hash of its content, so identical layers are stored and transferred once, even if many images share them. A running container adds a thin writable layer on top.

Running containers

docker run creates a container from an image. The engine sets up namespaces so the container sees its own processes, file system, network and hostname, and cgroups so it uses only the CPU and memory it is allowed. Ports can be published to the host, volumes mounted for data, and environment variables passed for configuration.

Registries

Images are pushed to and pulled from registries by name and tag, such as myapp:1.0, or by digest, an immutable content hash. Organisations usually run private registries or use cloud registry services.

Multi-container applications

Docker Compose describes several services, such as an API, a database and a cache, with their networks and volumes in one file, and starts them together with one command. It is common for local development and small deployments.

GPUs

With the NVIDIA Container Toolkit installed on the host, containers can access GPUs, for example with docker run --gpus all. The host provides the driver; the image contains matching CUDA libraries and frameworks. NVIDIA and framework projects publish base images with tested combinations.

Key concepts and terms

A worked example

This scenario is illustrative. A data science team keeps hearing "it works on my machine" when handing models to engineers.

1. They write a Dockerfile starting from an official CUDA-enabled PyTorch base image, pin all Python packages in a requirements file, and copy in the model-serving code.
2. The first image is 11 GB because it includes compilers and build tools. A multi-stage build compiles dependencies in a builder stage and copies only the results into a slim runtime stage, cutting the image to about 6 GB.
3. Reordering the Dockerfile so dependencies install before the code is copied means a code change rebuilds in 30 seconds instead of 8 minutes.
4. A Compose file runs the model server, a Redis cache and a test client together on laptops with docker compose up.
5. CI builds the image on every merge, scans it for known vulnerabilities, tags it with the git commit and pushes it to the company registry.
6. The platform team deploys the same image to Kubernetes by digest. When a bug appears, they redeploy the previous digest in minutes.

Strengths and limitations

Strengths:

• Reproducible environments from development to production.
• Lightweight and fast compared with virtual machines.
• Standard OCI images run on many platforms and orchestrators.
• Layer caching makes builds and transfers efficient.
• Huge ecosystem of official and community base images.

Limitations:

• Containers share the host kernel, so isolation is weaker than virtual machines.
• Images can become very large, especially with ML libraries.
• Running many containers across machines needs an orchestrator.
• Docker Desktop requires a paid subscription for larger businesses.
• Untrusted public images can carry vulnerabilities or malicious code.

How it compares with alternatives

Practical guidance and best practices

1. Start from small, official base images and pin their versions.
2. Order Dockerfile steps so rarely changing layers come first.
3. Use multi-stage builds to keep build tools out of runtime images.
4. Run as a non-root user inside the container.
5. Never bake secrets into images; inject them at run time.
6. Use a .dockerignore file to keep data, caches and credentials out of the build context.
7. Scan images for vulnerabilities in CI and rebuild regularly for security patches.
8. Deploy by digest or immutable tags, not latest.

Common mistakes

• Copying the whole project before installing dependencies, which breaks caching.
• Including datasets or model checkpoints in images when they belong in storage.
• Running containers as root by default.
• Storing passwords or API keys in Dockerfiles or image layers.
• Relying on latest, so different machines run different code.
• Mismatching CUDA libraries in the image with the host GPU driver.

Frequently asked questions

Is a container the same as a virtual machine?

No. A virtual machine runs a full guest operating system on virtual hardware. A container shares the host's kernel and isolates processes. Containers are lighter and faster to start; virtual machines give stronger isolation.

Is Docker free?

Docker Engine and the CLI are open source. Docker Desktop is free for personal use, education and small businesses, but larger organisations need a paid subscription under Docker's terms.

Does Kubernetes still use Docker?

Kubernetes runs OCI images built with Docker, but it talks to runtimes such as containerd directly rather than through Docker Engine. Images built with Docker work unchanged.

How do I use GPUs in Docker?

Install the NVIDIA driver and the NVIDIA Container Toolkit on the host, use an image with matching CUDA libraries, and start containers with the --gpus flag.

Why are ML images so large?

Frameworks such as PyTorch include large GPU libraries, and CUDA toolkits add more. Slim base images, multi-stage builds and keeping data and models outside the image help keep sizes manageable.

Glossary

• Host: the machine that runs containers.
• Kernel: the core of the operating system shared by containers on a host.
• Build context: the files sent to the builder when building an image.
• Base image: the image a Dockerfile starts from.
• Docker Hub: Docker's public registry of images.
• Rootless: running containers without administrator privileges.
• Vulnerability scan: checking image contents against known security issues.

Summary

Docker packages applications and their dependencies into standard, layered images and runs them as lightweight, isolated containers, so software behaves the same on every machine. Introduced in 2013 and standardised through the Open Container Initiative, it remains the most common way to build images that run on Kubernetes and other platforms, including GPU-heavy ML workloads. Its value grows with good practice: small pinned base images, cache-friendly Dockerfiles, multi-stage builds, non-root users, no baked-in secrets and deployment by immutable digest.

Related

It belongs to Platform in my stack, under MLOps and AI infrastructure. Others in the same group: