Model Monitoring
Model monitoring is the continuous measurement of a deployed model's inputs, outputs, and performance signals — including data drift, prediction drift, label drift, latency, and ground-truth accuracy — to detect degradation before it harms business outcomes.
AIDOLS Research Team · Editorial standards
Full definition
Monitoring spans four layers: (1) operational health (latency, error rate), (2) data drift (input distribution change), (3) concept drift (input-output relationship change), (4) model quality (accuracy vs delayed ground truth). Tools include Evidently, Arize, and built-in offerings from cloud platforms. Monitoring closes the MLOps loop with retraining triggers.
Why it matters
Models silently decay. Without monitoring, the first signal of degradation is a customer-facing failure or a regulatory complaint. Monitoring discipline often pays for itself in a single avoided incident.
Example
A logistics company's ETA model drifts after a major routing change; data-drift alerts fire within 24 hours and the on-call engineer triggers retraining, avoiding a multi-week regression in delivery promises.
Next steps
Related terms
- Model DriftModel drift is the degradation of an AI model's performance over time as the data it sees in production diverges from the distribution it was trained on — the silent failure mode of every deployed ML system.
- MLOpsMLOps is the discipline of operating machine-learning and AI systems reliably in production — covering data pipelines, model training, deployment, monitoring, drift detection, governance, and incident response — analogous to DevOps for traditional software.
- AI AuditAn AI audit is a structured, evidence-based examination of an AI system or AI program against defined criteria — covering training data, model, deployment context, monitoring, and governance — performed by an internal team, an external firm, or a regulator.
- MLOps Maturity ModelAn MLOps maturity model is a tiered framework that ranks an organization's ML lifecycle automation — from manual notebook handoffs (Level 0) to fully automated continuous integration, delivery, and training pipelines with automated retraining triggers (Level 4).
More deployment & operations terms
- RAG-as-a-ServiceRAG-as-a-Service is a managed offering that handles document ingestion, chunking, embedding generation, vector storage, retrieval, and LLM grounding behind a single API — letting teams ship retrieval-augmented features without building the underlying pipeline.
- Agentic WorkflowAn agentic workflow is a multi-step AI process where a model plans, takes actions through tools, observes results, and iterates toward a goal — replacing single-shot prompting with a loop that reasons, acts, and self-corrects.
- Function CallingFunction calling is an LLM capability where the model — given a schema of available functions — emits a structured invocation (function name and JSON arguments) instead of free-form text, letting downstream code execute the call deterministically.
Source & further reading
Primary source: Klaise et al. — "Monitoring and explainability of models in production" (arXiv) (2020).
Citation policy: this entry is part of the AIDOLS AI Implementation Glossary and may be quoted for research, journalism, and education with attribution to aidolsgroup.com/en/glossary/model-monitoring/.