# MLOps and AI infrastructure Everything between a trained model and a reliable, secure, governed production system, in one map. ## Overview MLOps is the discipline of taking models from experiments to reliable production systems and keeping them healthy. AI infrastructure is the hardware and software underneath: GPUs, serving engines, containers, orchestration, observability, security and governance. A brilliant recipe is not a restaurant. You also need a kitchen, suppliers, hygiene rules, staff schedules and quality checks. MLOps and AI infrastructure are the restaurant around the model. ## From experiment to production Experiments are tracked in [MLflow](/tools/mlflow) or [Weights & Biases](/tools/weights-and-biases), pipelines run on [Kubeflow](/tools/kubeflow) or managed platforms such as [SageMaker](/tools/sagemaker) and [Vertex AI](/tools/vertex-ai), features are kept consistent by a [Feature store](/tools/feature-store), and deployed models are watched by [Model monitoring](/tools/model-monitoring) and documented in [Model cards](/tools/model-cards). ## Serving and hardware Models are served by [Triton](/tools/triton), [vLLM](/tools/vllm), [SGLang](/tools/sglang), [TensorRT](/tools/tensorrt), [ONNX Runtime](/tools/onnx-runtime), [llama.cpp](/tools/llama-cpp), [KServe](/tools/kserve), [Ray Serve](/tools/ray-serve) or [BentoML](/tools/bentoml), on accelerators such as [NVIDIA H100](/tools/nvidia-h100), [NVIDIA H200](/tools/nvidia-h200), [NVIDIA B200](/tools/nvidia-b200) and [Cloud TPU](/tools/cloud-tpu), programmed through [CUDA](/tools/cuda) and connected by [NCCL](/tools/nccl). Compression is covered in [Compression and serving](/model/compression-and-serving). ## Platform and architecture [Docker](/tools/docker), [Kubernetes](/tools/kubernetes), [Helm](/tools/helm) and [ArgoCD](/tools/argocd) run and deploy services; [Prometheus](/tools/prometheus), [Grafana](/tools/grafana) and [OpenTelemetry](/tools/opentelemetry) make them observable. Architecture patterns such as [Microservices](/tools/microservices), [Circuit breaker](/tools/circuit-breaker) and [Saga pattern](/tools/saga-pattern) keep distributed systems reliable. ## Security and governance Access is secured with [OAuth 2.0](/tools/oauth-2-0), [OIDC](/tools/oidc), [mTLS](/tools/mtls) and [Zero Trust](/tools/zero-trust); risks are managed with [OWASP](/tools/owasp) guidance, privacy law such as [GDPR](/tools/gdpr) and [KVKK](/tools/kvkk), the [EU AI Act](/tools/eu-ai-act) and an [AI governance](/tools/ai-governance) programme. ## The full map ### MLOps - [MLflow](https://www.ozercemkelahmet.com/tools/mlflow.md) - [Kubeflow](https://www.ozercemkelahmet.com/tools/kubeflow.md) - [Weights & Biases](https://www.ozercemkelahmet.com/tools/weights-and-biases.md) - [BentoML](https://www.ozercemkelahmet.com/tools/bentoml.md) - [KServe](https://www.ozercemkelahmet.com/tools/kserve.md) - [Ray Serve](https://www.ozercemkelahmet.com/tools/ray-serve.md) - [SageMaker](https://www.ozercemkelahmet.com/tools/sagemaker.md) - [Vertex AI](https://www.ozercemkelahmet.com/tools/vertex-ai.md) - [Feature store](https://www.ozercemkelahmet.com/tools/feature-store.md) - [Model monitoring](https://www.ozercemkelahmet.com/tools/model-monitoring.md) - [Model cards](https://www.ozercemkelahmet.com/tools/model-cards.md) ### Inference and serving - [Triton](https://www.ozercemkelahmet.com/tools/triton.md) - [vLLM](https://www.ozercemkelahmet.com/tools/vllm.md) - [SGLang](https://www.ozercemkelahmet.com/tools/sglang.md) - [TensorRT](https://www.ozercemkelahmet.com/tools/tensorrt.md) - [ONNX Runtime](https://www.ozercemkelahmet.com/tools/onnx-runtime.md) - [llama.cpp](https://www.ozercemkelahmet.com/tools/llama-cpp.md) ### Hardware - [CUDA](https://www.ozercemkelahmet.com/tools/cuda.md) - [NCCL](https://www.ozercemkelahmet.com/tools/nccl.md) - [Cloud TPU](https://www.ozercemkelahmet.com/tools/cloud-tpu.md) - [NVIDIA H100](https://www.ozercemkelahmet.com/tools/nvidia-h100.md) - [NVIDIA H200](https://www.ozercemkelahmet.com/tools/nvidia-h200.md) - [NVIDIA B200](https://www.ozercemkelahmet.com/tools/nvidia-b200.md) ### Platform - [Kubernetes](https://www.ozercemkelahmet.com/tools/kubernetes.md) - [Docker](https://www.ozercemkelahmet.com/tools/docker.md) - [Helm](https://www.ozercemkelahmet.com/tools/helm.md) - [ArgoCD](https://www.ozercemkelahmet.com/tools/argocd.md) - [Prometheus](https://www.ozercemkelahmet.com/tools/prometheus.md) - [Grafana](https://www.ozercemkelahmet.com/tools/grafana.md) - [OpenTelemetry](https://www.ozercemkelahmet.com/tools/opentelemetry.md) ### Architecture - [Microservices](https://www.ozercemkelahmet.com/tools/microservices.md) - [Domain-driven design](https://www.ozercemkelahmet.com/tools/domain-driven-design.md) - [CQRS](https://www.ozercemkelahmet.com/tools/cqrs.md) - [Event sourcing](https://www.ozercemkelahmet.com/tools/event-sourcing.md) - [Hexagonal architecture](https://www.ozercemkelahmet.com/tools/hexagonal-architecture.md) - [Saga pattern](https://www.ozercemkelahmet.com/tools/saga-pattern.md) - [Circuit breaker](https://www.ozercemkelahmet.com/tools/circuit-breaker.md) - [gRPC](https://www.ozercemkelahmet.com/tools/grpc.md) - [GraphQL](https://www.ozercemkelahmet.com/tools/graphql.md) ### Security and governance - [OAuth 2.0](https://www.ozercemkelahmet.com/tools/oauth-2-0.md) - [OIDC](https://www.ozercemkelahmet.com/tools/oidc.md) - [JWT](https://www.ozercemkelahmet.com/tools/jwt.md) - [mTLS](https://www.ozercemkelahmet.com/tools/mtls.md) - [Zero Trust](https://www.ozercemkelahmet.com/tools/zero-trust.md) - [OWASP](https://www.ozercemkelahmet.com/tools/owasp.md) - [GDPR](https://www.ozercemkelahmet.com/tools/gdpr.md) - [KVKK](https://www.ozercemkelahmet.com/tools/kvkk.md) - [EU AI Act](https://www.ozercemkelahmet.com/tools/eu-ai-act.md) - [AI governance](https://www.ozercemkelahmet.com/tools/ai-governance.md) ## Related topics - [Agentic AI](/topics/agentic-ai) - [Generative AI and LLMs](/topics/generative-ai-llms) - [Big data](/topics/big-data) ## Frequently asked questions ### What is MLOps? The practices and tools for deploying, monitoring and maintaining machine learning models reliably in production. ### Which serving engine should I use for LLMs? vLLM and SGLang are strong open-source defaults for throughput; TensorRT-LLM and Triton suit NVIDIA-optimised deployments; llama.cpp suits local and CPU use. ## Summary MLOps and AI infrastructure turn models into dependable services: tracked experiments, efficient serving on the right hardware, containerised platforms, observability, resilient architecture, and security and governance by design.