Model Monitoring
Model monitoring is the continuous measurement of a deployed model's inputs, outputs, and performance signals — including data drift, prediction drift, label drift, latency, and ground-truth accuracy — to detect degradation before it harms business outcomes.
Full definition
Monitoring spans four layers: (1) operational health (latency, error rate), (2) data drift (input distribution change), (3) concept drift (input-output relationship change), (4) model quality (accuracy vs delayed ground truth). Tools include Evidently, Arize, WhyLabs, and built-in offerings from cloud platforms. Monitoring closes the MLOps loop with retraining triggers.
Why it matters
Models silently decay. Without monitoring, the first signal of degradation is a customer-facing failure or a regulatory complaint. Monitoring discipline often pays for itself in a single avoided incident.
Example
A logistics company's ETA model drifts after a major routing change; data-drift alerts fire within 24 hours and the on-call engineer triggers retraining, avoiding a multi-week regression in delivery promises.
Related terms
- Model DriftModel drift is the degradation of an AI model's performance over time as the data it sees in production diverges from the distribution it was trained on — the silent failure mode of every deployed ML system.
- MLOpsMLOps is the discipline of operating machine-learning and AI systems reliably in production — covering data pipelines, model training, deployment, monitoring, drift detection, governance, and incident response — analogous to DevOps for traditional software.
- AI AuditAn AI audit is a structured, evidence-based examination of an AI system or AI program against defined criteria — covering training data, model, deployment context, monitoring, and governance — performed by an internal team, an external firm, or a regulator.
- MLOps Maturity ModelAn MLOps maturity model is a tiered framework that ranks an organization's ML lifecycle automation — from manual notebook handoffs (Level 0) to fully automated continuous integration, delivery, and training pipelines with automated retraining triggers (Level 4).
Source & further reading
Primary source: Klaise et al. — "Monitoring and explainability of models in production" (arXiv) (2020).
Citation policy: this entry is part of the AIDOLS AI Implementation Glossary and may be quoted for research, journalism, and education with attribution to aidolsgroup.com/no/glossary/model-monitoring/.