Canary Deployment

Full definition

Canary deployments differ from A/B tests: a canary primarily watches operational signals (error rate, p99 latency, cost-per-request, output validity), not causal lift. Once health checks pass, traffic ramps in stages — 1%, 5%, 25%, 100%. Auto-rollback on threshold breach is standard. The pattern originated at Google and is now codified in Argo Rollouts, Flagger, and most cloud ML platforms.

Why it matters

Big-bang model rollouts cause real outages — a 2024 LLM provider had a multi-hour incident from an un-canaried weight update. Canary discipline turns "the model is broken" from a customer-facing event into a 2% blip on a dashboard.

Example

An e-commerce team deploys a new search-ranking model at 1% traffic; p99 latency rises 40ms; the canary auto-rolls-back within 4 minutes without a single user complaint.

Related terms

• Shadow Deployment
  A shadow deployment sends production traffic to a new model in parallel with the live model but never returns the new model's predictions to end users — allowing teams to validate latency, cost, and prediction distribution under real load with zero user risk.
• A/B Testing for ML
  A/B testing for ML is the practice of randomly splitting live traffic between a control model and a candidate model to measure the candidate's causal impact on real business metrics — revenue, conversion, retention, defect rate.
• MLOps
  MLOps is the discipline of operating machine-learning and AI systems reliably in production — covering data pipelines, model training, deployment, monitoring, drift detection, governance, and incident response — analogous to DevOps for traditional software.
• Model Serving
  Model serving is the runtime infrastructure that hosts a trained model and exposes it as an API for low-latency online inference at scale, handling batching, autoscaling, GPU sharing, versioning, and routing.