Skip to main content
In machine learning (ML), models are often trained using historical data to predict future outcomes. However, as conditions change over time, the performance of these models can deteriorate. This degradation can arise due to shifts in the data distribution, changes in patterns, or evolving business requirements. It is vital to monitor models to detect these issues early and take corrective action, ensuring consistent and reliable performance. It helps prevent adverse business impacts, maintains trust in AI systems, and provides insights into model behavior. Domino’s Model Monitoring uses training and prediction data to track drift for the model’s input features and prediction variables. If you have ground truth data for the model’s predicted values, Domino can ingest it to produce model quality metrics using standard measures such as accuracy, precision, and recall. Domino can also alert you about every feature and metric that exceeds a configurable threshold.

​
Understanding model drift

There are various reasons why model quality can degrade over time:
  • Data drift: Changes in the input data distribution that differ from the training data. For instance, a model trained on sales data may encounter new customer demographics that affect its predictions.
  • Concept drift: Changes in the relationship between input and output variables. For example, a fraud detection model might lose accuracy if fraudsters adapt their techniques.

​
Fixing model quality issues

When Domino detects model quality degradation via drift, Domino raises alerts.
  • Root cause analysis: Identify whether data or concept drift is occurring and assess its impact on model accuracy and business outcomes.
  • Retraining: Retrain models with recent data to reflect the new patterns or relationships. This might involve data cleaning or new feature engineering.
  • Continuous improvement: Monitor retrained models and incorporate feedback loops to refine them iteratively.

​
Next steps

Get started with Projects in Domino.