Canary Deployment
A canary deployment routes a small slice of production traffic (e.g., 1–5%) to a new model version while the majority continues hitting the stable version, surfacing latency, error, and quality regressions before full rollout.
AIDOLS Research Team · Editorial standards
Full definition
Canary deployments differ from A/B tests: a canary primarily watches operational signals (error rate, p99 latency, cost-per-request, output validity), not causal lift. Once health checks pass, traffic ramps in stages — 1%, 5%, 25%, 100%. Auto-rollback on threshold breach is standard. Google's SRE Workbook (2018) codifies the practice, and it is now built into Argo Rollouts, Flagger, and most cloud ML platforms.
Why it matters
Big-bang model rollouts cause real outages. Canary discipline turns "the model is broken" from a customer-facing event into a 2% blip on a dashboard.
Example
An e-commerce team deploys a new search-ranking model at 1% traffic; p99 latency rises 40ms; the canary auto-rolls-back within 4 minutes without a single user complaint.
Next steps
Related terms
- Shadow DeploymentA shadow deployment sends production traffic to a new model in parallel with the live model but never returns the new model's predictions to end users — allowing teams to validate latency, cost, and prediction distribution under real load with zero user risk.
- A/B Testing for MLA/B testing for ML is the practice of randomly splitting live traffic between a control model and a candidate model to measure the candidate's causal impact on real business metrics — revenue, conversion, retention, defect rate.
- MLOpsMLOps is the discipline of operating machine-learning and AI systems reliably in production — covering data pipelines, model training, deployment, monitoring, drift detection, governance, and incident response — analogous to DevOps for traditional software.
- Model ServingModel serving is the runtime infrastructure that hosts a trained model and exposes it as an API for low-latency online inference at scale, handling batching, autoscaling, GPU sharing, versioning, and routing.
More deployment & operations terms
- Batch InferenceBatch inference runs a model over a large set of inputs offline on a schedule — typically nightly or hourly — optimized for throughput and cost-per-prediction rather than latency.
- Online InferenceOnline inference returns model predictions synchronously in response to a live user or system request, within an interactive latency budget.
Source & further reading
Primary source: Google SRE Workbook — Canarying Releases (2018).
Citation policy: this entry is part of the AIDOLS AI Implementation Glossary and may be quoted for research, journalism, and education with attribution to aidolsgroup.com/en/glossary/canary-deployment/.