Skip to content

Monitoring predictions (lite path)

monitor_predictions is the model-agnostic entry point: it interprets a model's outputs without touching its internals. Use it to monitor a model in production, or to explain any scorer MLI has no native adapter for.

It builds a decision-tree surrogate, a high-error cohort tree, archetype-lite segments, holdout metrics, drift (PSI), and model-free fairness — from either an opaque model or a precomputed predictions array.

In short

  • Pass model= for any scorer with predict_proba / predict, or predictions= when you have only the scores.
  • mode picks the analyses: "surrogate", "error", "archetype_lite" or "all" (the default).
  • Pass reference_df= for PSI drift against a baseline, and sensitive_features= for fairness.
  • The output is a ProfilingResult with fewer capabilities. Check result.resolve_capabilities() before you use one.

When to use it

You have Pass Backend
A recognised model (CatBoost / LightGBM / XGBoost / sklearn / EBM / …) use fit_profiler instead native
Any scorer with predict_proba / predict (custom, pipeline, neural net) model= lite_model
Only the scores predictions= lite_predictions

The full model catalog and the fit_profiler vs monitor_predictions decision are in Bring-your-own-model.

Basic call

from parleml_mli import monitor_predictions, TaskType

result = monitor_predictions(
    X=X_now,
    y=y_now,                          # optional — enables metrics/fairness when present
    model=my_scorer,                  # OR predictions=my_scores
    X_test=X_holdout, y_test=y_holdout,
    task_type=TaskType.BINARY,
    mode="all",                       # "surrogate" | "error" | "archetype_lite" | "all"
    decision_threshold=0.5,          # explicit cut; default None keeps the scorer's decision
)
print(result.capabilities.backend)    # lite_model or lite_predictions

Monitoring drift over time

Pass a reference sample (e.g. the training window or last month) to get PSI drift between it and the current batch:

result = monitor_predictions(
    X=X_this_week,
    predictions=scores_this_week,
    reference_df=X_training_window,   # baseline for PSI drift
    task_type=TaskType.BINARY,
)
result.plot_psi()

Fairness without model internals

Provide the protected attributes and MLI computes model-free fairness (AIR / ME / SMD):

result = monitor_predictions(
    X=X_now, y=y_now, predictions=scores,
    sensitive_features=X_now[["gender", "age_band"]],
    task_type=TaskType.BINARY,
)

Modes

  • surrogate — a shallow, readable decision tree that mimics the scorer.
  • error — a tree over the residuals, surfacing the cohorts where the model is weakest.
  • archetype_lite — unsupervised segments of the scored population.
  • all — everything above (default).

Output uses the same ProfilingResult hub and persistence format, with a smaller capability set. Error analysis needs labels; drift needs a reference sample; fairness needs protected attributes and usable group evidence. Predictions-only results cannot score novel rows without a model. Inspect result.resolve_capabilities() and artifact presence before calling a plot or inference method. Lite archetypes and surrogate structure describe the monitoring analysis, not exact native SHAP or leaves of the original model.

See the monitor_predictions reference and Evaluation & trust.

Next