AI Pipeline

Full definition

A modern AI pipeline includes data validation gates, model training and evaluation, packaging (often as a container or serialized model), deployment to an inference server, online monitoring (latency, throughput, drift, business KPIs), and a feedback loop for retraining. Tooling varies — Airflow, Dagster, Prefect, Vertex AI Pipelines, SageMaker Pipelines — but the stages are stable.

Why it matters

Pipeline maturity is the single best predictor of whether an organization can ship AI repeatably. Teams without a versioned, code-defined pipeline struggle to reproduce yesterday's model, let alone audit it for the regulator.

Example

A bank's fraud pipeline runs hourly: pulls fresh transactions, runs feature engineering, scores them with the current production model, writes alerts to the case-management system, and logs metrics to the drift monitor — all defined in 800 lines of versioned Python.

Related terms

• MLOps
  MLOps is the discipline of operating machine-learning and AI systems reliably in production — covering data pipelines, model training, deployment, monitoring, drift detection, governance, and incident response — analogous to DevOps for traditional software.
• Training Data
  Training data is the curated dataset used to teach a machine-learning model — every pattern the model can recognize, every bias it inherits, and every limit to its accuracy ultimately traces back to this data.
• Model
  An AI model is the trained artifact — a specific set of numerical weights plus an architecture — produced when a learning algorithm runs against training data, and the thing that actually gets deployed and audited.
• Inference
  Inference is the process of running a trained AI model on new, unseen inputs to produce predictions, classifications, or generated content — the part that runs every time a user interacts with the system.
• Model Drift
  Model drift is the degradation of an AI model's performance over time as the data it sees in production diverges from the distribution it was trained on — the silent failure mode of every deployed ML system.