Reference — explainability (MLI)

The model-quality and interpretability surface that is not part of the SQL star schema: numeric evaluation, computed insights (trust, fairness, drift, errors, …), per-row explanations, rule cards, and the mli.* charts — gated by GET /api/experiments/{id}/mli/capabilities.

Contents

• When to use this vs SQL
• Model evaluation (the numbers)
• Computed insights (get_insight)
• Per-row why (get_insight kind case)
• Two kinds of rules
• Train and Test numbers
• The mli.* chart reads
• The SHAP / beeswarm redirect rule

When to use this vs SQL

• Use SQL (*__predictions, *__rules, *__archetypes) for who/where/how many and rule/cohort questions.
• Use get_model_evaluation for how good — metrics and curves, never prose.
• Use get_insight for the computed artifacts (leaderboard, trust, fairness, drift, error analysis, …).
• Use MLI charts for why questions: which features drive the model overall, how a feature moves the prediction (PDP), and whether probabilities are calibrated.

Model evaluation (the numbers)

get_model_evaluation(experiment_id, split?, class_key?) returns the holdout evaluation: per-split metric tiles plus ROC / PR / calibration / lift / gains / KS curves, score distributions, the confusion matrix, and confusion_curve — per-threshold (tp, fp, fn, tn) at the returned thresholds. Curves may be downsampled; a nearby point is not an exact evaluation of a requested threshold or review quota. Use simulate_decision for supported native policy evaluations, and disclose when the requested policy is unavailable rather than interpolating counts or rebuilding metrics. Defaults to macro splits only (lean); split narrows to one split, class_key to one multiclass one-vs-rest slice. 404 = no holdout evaluation attached. Answer "how good is the model?" from here, not AutoDoc.

Computed insights (get_insight)

get_insight(experiment_id, kind, params?) — one tool, a curated kind per computed artifact. get_insight(experiment_id, "kinds") lists the kinds that apply to that experiment's type, by family. Not-computed or not-applicable surfaces return {available: false, note, reason} (an answer, not an error; reason is the backend's own explanation).

• Explanation profile. Kinds marked ✱ below read the explanation profile the Explain step computes; until it has run they return the note. get_capabilities(experiment_id) lists which kinds are ready and which still needs_explanations.
• params fills the path placeholders (row_index, cluster_id, rule_id) and passes the rest as query options. A missing required param fails before the call and names it.
• Digests. A few kinds can return thousands of rows, so the result you read is trimmed and says so (truncated, plus the total): engineered_features keeps the first 100 rows, forecast_series the first 200 series, association_items the top 200 items by support, and archetype_map leaves out the per-row scatter. An interactive panel gets the whole payload.
• Not available vs malformed. A param the backend rejects (e.g. top_k out of range) is an error naming the field — fix it; it is not the "not available" note.

Evaluation (supervised runs; diagnose also external, recommendation, ltr and an explained tree forecast)

kind	What it answers
leaderboard	Every model trained + metrics — why the winner won. Rows carry a kind: model (comparable scores) vs search_trial (tuning probes, optimistic, rank separately).
scores	The held-out test score and the metrics beside it, the constant baseline, and the calibration and fitting basis behind them.
diagnose ✱	The metric strip of every holdout split, as labelled tiles.
decision_layer	The calibration and decision threshold in force, their provenance and revision history, and whether they can be re-decided.
composition	Ensemble composition: members, weights, rejected candidates, and the lift over the best single model.

Trust & explain (supervised, external, recommendation and ltr runs, and a forecast a tree-backend model won — its features are the lags, so not fairness, text_insight, engineered_features or glassbox; all ✱ except label_noise, which only supervised runs have)

kind	What it answers
trust_summary	Traffic-light trust tiles — the go/no-go overview.
overview	Base rate, decision threshold, archetype count, headline metrics and a one-paragraph summary.
slices	The cohorts to examine (archetypes, top feature ranges) with row counts; no metrics — diagnose and error_tree say where it underperforms.
fairness	Group fairness across sensitive attributes: every feature with its breaches and worst group; params={"feature": "<name>"} reads one feature's groups, worst first. Judged on the run's held-out rows; the trust_summary tile says how many, or that it fell back to training rows.
ale	Accumulated local effects (PDP's correlation-robust sibling).
error_analysis	Where the model errs, by feature.
error_tree	Worst error pockets as a small decision tree, fit on the run's held-out rows; fit_on names them (training when the run kept none: those pockets are in-sample errors, not evidence of where the model fails on new rows).
drift	Per-feature PSI between the target cohort and the other rows — how the cohort differs, not drift over time.
text_insight	Class-discriminative terms in a text column (params={"encoder": …, "feature": …}; valid pairs: get_capabilities(id).insight_encoders).
label_noise	Likely mislabelled rows (confident learning; classification). Scored when the model is fitted, on every labelled row a model never trained on: training rows by their out-of-fold prediction (bagged templates only), the validation slice and the test set. Gives the estimated share of wrong labels, per-class rates, coverage (rows scanned of each split's labelled rows), and up to 200 suspects, each with its dataset row and role. Treat suspects as rows to review, not confirmed errors. The rate counts rows the model confidently disagrees with, so read it against the model's error rate on the same rows: close to it, it measures the model's mistakes, not the labels.
shap_interactions	Strongest pairwise feature interactions.
engineered_features	Every final model feature, the raw columns and steps that built it, and its importance.
surrogate	A transparent tree / rule surrogate of the deployed model, with its fidelity (params={"target": "residual"} fits it to the residuals instead, on the held-out rows fit_on names).
glassbox	Internals of a transparent model (GLM coefficients, EBM shape functions, rules, scorecard); a note for opaque models. An EBM answers its strongest curves and grids (each read as its lowest and highest contribution), the ranked term ledger and its text columns' components grouped by source column; params={"feature": "<name>"} returns one more curve, params={"feature": ["<a>", "<b>"]} one pair's grid.
native_trees	The deployed model's own trees: how many, and whether it is an ensemble.
case	Per-row why — see below.

Archetypes (same runs; all ✱)

kind	What it answers
archetypes	Behavioral archetype landscape: cohorts, severity, stability.
archetype	One archetype in detail (params={"cluster_id": …}): risk indicators, metrics, top features, representative rows, rule fingerprints.
archetype_summary	One row per archetype: size, target rate, severity.
archetype_comparison	Archetypes side by side (params={"cluster_ids": "0,1,3"}): per-cluster metrics, feature overlap, signatures.
case_list	Paged target-case browser (params offset, limit ≤ 200, cluster_id, class_key, search). Literal search filters persisted profile rows before paging; use it to locate a known key or value, then verify the match.
feature_search	Which features separate the archetypes (SHAP or z-score groups).
archetype_map	Each archetype's medoid and size (a panel also draws the per-row UMAP scatter); params class_key.
archetype_diagnostics	How sound the archetypes are: bootstrap stability, purity / coverage, model-score spread, the archetype × feature impact heatmap, SHAP compacity; params class_key.

Forecast (forecast runs)

kind	What it answers
forecast_summary	The headline: backend, horizon, frequency, metrics vs the seasonal baseline (beats_baseline and its basis), weak series, failed stages, and intervals — per level whether the band is calibrated, which series fall short, its test coverage and the fix.
forecast_series	Every series: the model serving it, its validation and test MASE and sMAPE, why it is weak, its band width per level.
forecast_coverage	How often the actuals fell inside the winners' band, per level, per backtest window and pooled, with the points scored: a conformal band drawn for each validation window from the other windows' residuals, the test window from all of them; an analytic band (statistical models) as each window's own fit drew it. measured: false says why not (no backtest kept; for a conformal band, under two validation windows or a cv_gap).
forecast_diagnostics	Fit diagnostics: failures, notes, preprocessing, seasonality strengths.
forecast_ensemble	Ensemble members with their validation scores and weights.

Cost of a native explanation read. The anomaly unsupervised_surrogate read may preprocess the fitted data and fit a small LightGBM surrogate on a cache miss. It explains the detector's anomaly flag, not passive truth labels, and does not refit the detector. Reuse it when needed to answer a driver/fidelity question; under an explicit no-fitting restriction, answer from saved artifacts instead. A server restart or changed operating point can require this auxiliary work again.

Unsupervised (anomaly and cluster runs)

kind	What it answers
unsupervised_summary	The headline: detector or algorithm, counts, threshold, quality metrics and per-cluster descriptions.
unsupervised_features	The features the model used.
unsupervised_surrogate	Anomaly runs: a transparent surrogate showing which features drive the anomaly flag.

Association (association runs — the *__assoc_* tables stay the SQL-first answer)

kind	What it answers
association_summary	The headline: baskets, items, rules kept and why, significance and holdout checks.
association_rules	Mined rules, paged (params significant_only, focus_item, side, sort_by, limit ≤ 500, offset).
association_itemsets	Frequent itemsets, paged (params contains_item, min_length, limit, offset).
association_items	Items with support and rule participation.
association_evidence	Example and counter-example baskets for one rule (params={"rule_id": …, "k": …}).
association_audience	Targeting list for one rule (params={"rule_id": …}).

Recommendation & ranking (recommendation and ltr runs; ranking also supervised)

kind	What it answers
recommendation	Next-best-recommendation summary: ranking metrics + classes.
recommendation_classes	One row per recommended product: category, model type, routing.
recommendation_submodels	Per-product sub-models: positives, validation / test ROC, quality gate.
recommendation_drift	Drift of the population that bought a product vs the reference (PSI; params target_key).
recommendation_operating_point	The committed ranking operating point (popularity discount, value weights).
ranking	Hit/MRR/NDCG@K beside popularity, with uncertainty: bootstrap intervals over the holdout's customers on each metric and on the model-minus-popularity gap (verdict).
ltr	Learning-to-rank summary: objective, NDCG@K, operating point.
ltr_operating_point	The committed learning-to-rank operating point.

Per-row why (get_insight kind case)

get_insight(experiment_id, "case", {"row_index": N, "top_k": 5, "class_key": …}) (row_index required, top_k 1-50, class_key optional) returns the SHAP-space neighbourhood of one row: the anchor case plus its most similar labelled cases and their outcomes — "which known cases does this resemble, and how did they turn out". It reads the explanation profile (✱): N is a position in that profile, not in the input dataset or published predictions. Verify the key and feature values before linking a case to a published observation. Then use the matched prediction row's catalog join keys to read *__rules_fired and *__archetypes. Equal row positions do not establish identity; if no reliable mapping is available, keep the profile explanation separate and disclose the limit.

To explain another supervised class for the same customer, use what_if(experiment_id, row={}, base={"key": {"customer_id": "…"}}, active_class="Champion"). active_class is the original serving label; an encoded profile class_key or column position may name a different class. Omit it for the primary class; it also works with explain=true. Changing the explained class does not change the predicted class. Read elapsed_ms unavailable reasons before describing archetype placement or novelty; missing geometry is not measured novelty.

Two kinds of rules (don't confuse them)

These are mined interpretation rules. A RuleFit predictor's own additive rule coefficients and support are instead in get_insight(kind="glassbox"); they can exist even when its post-hoc tree-rule extractor is unsupported. Check has_rules_positive, rule_engines and failure notes before requesting model rules or their QA. Business-rule availability does not imply model-rule QA.

• Model rules are label rules: patterns in the labels, found through the deployed model's trees on its engineered features — not directly applyable to source data, and not a rendering of the model's own decisions. Available two ways: the published SQL tables *__rules (definitions, Train and Test measures, holds_up) + *__rules_fired (which fired per row) — query these (see schema-supervised) for "which rules apply to predicted X?", the default answer — or the rule cards via get_rules(experiment_id, kind="model", engine?, class_key?).
• Independent "Business Rules" model is a separate, directly-applyable rule model fit on raw attributes vs the true target (classification only). It is an MLI artifact, not in the SQL star schema. Detect it on get_capabilities: has_business_rules (bool) + business_rule_engine_info (the engines, e.g. sequential / positive_miner / all_class). Read the rule cards with get_rules(experiment_id, kind="business", engine?, class_key?) — it returns the same rule-card shape plus evaluation: the active engine's rules themselves measured on the held-out rows. (For the interactive view, the parleml UI also serves these.)

Train and Test numbers (read before quoting a rule)

Every rule carries two sets of numbers:

• Train — the flat fields (confidence = precision, recall, lift, support, coverage, tp, fp): measured on the rows the rule was found on. Optimistic by construction — the search kept the rules that looked best there.
• Test — metrics_holdout (precision, precision_lower–precision_upper, recall, lift, n_covered, …): the same measures on held-out rows neither the model nor the rule search saw. null when there were no held-out rows.

holds_up is the verdict: too_few_rows when the rule covers fewer than min_test_rows held-out rows (no verdict — say so, don't guess); otherwise holds_up when the lower bound of its test precision beats its class's test base rate, else does_not_hold_up. Quote Test numbers as the rule's performance and call Train numbers in-sample; lead with rules that hold up and flag the rest. holdout_note says where the Test rows come from (on a temporal split they lie later in time, so a gap can also mean drift; for an imported model they are held out from rule selection, while the model's training rows are unknown).

get_rules quick reference

get_rules(experiment_id, kind="model"|"business", engine?, class_key?) returns {engine, engine_label, n_rules, ordered, metrics, metrics_holdout, rules:[{rule_id, predicted_class, text "(a) AND (b) -> [1]", coverage, confidence, recall, lift, support, metrics_holdout, holds_up, cascade?, cascade_holdout?}], cumulative, cumulative_holdout, holdout_basis, holdout_note, metrics_note, min_test_rows, interval_level, feature_usage?} (the business kind nests that under rules and adds evaluation). An ordered list (sequential, all-class) is read first rule wins: its cascade* numbers count only the rows the rules above leave. engine ∈ {positive_miner, sequential, all_class} (omit for the default); class_key selects a one-vs-rest class for multiclass. A 404 means rules/business-rules weren't computed — recompute the explanations with run_experiment(action="explain", experiment_id, params={"explain_preset": "standard_rules"}) (or "deep") with a builder token, else ask the user to (business rules are classification-only; describe_platform("explain_presets") lists the presets and what each computes).

get_rules(experiment_id, kind="qa", engine?) checks the rule set for redundancy: per engine, the 20 most overlapping rule pairs by coverage Jaccard (top_overlaps: [{rule_a, rule_b, overlap, same_class}]) and n_overlapping_pairs. same_class: true is redundancy (two rules for the same class covering the same rows); false is a conflict (rules of different classes firing on the same rows).

The mli.* chart reads (see charts.md)

These three are natively data-shaped: their routes have no format parameter and no Plotly-figure form, so render="png" is rejected.

Question	Chart kind	Path (no format param)
Which features drive predictions overall?	mli.model_importance	/api/experiments/{id}/mli/model-importance
How does feature F move the prediction?	mli.pdp	/api/experiments/{id}/mli/pdp
Are predicted probabilities calibrated?	mli.calibration	/api/experiments/{id}/mli/calibration-comparison

Always check get_capabilities(experiment_id) first — these are only meaningful for a published supervised model, they wait for the Explain step (its chart_kinds lists them under needs_explanations until then), and some are gated (e.g. calibration needs a classifier).

Reading mli.pdp (and get_insight(id, "ale"), the same shape): features holds one curve per driver, most important first (repeated per class_key on a multiclass model) — grid × average (ALE: bin_edges × ale); a categorical's grid can end with null, its (missing) level (the response with the feature missing on every row). Curves are drawn only for the model's top drivers: selection_basis says how they were ranked (importance — the mli.model_importance ranking; prediction_sensitivity — how much the output moves when a column is shuffled, used when the model has no importance signal). skipped lists every column with no curve, and the reason ({feature, reason}: an identifier, datetime or constant column, too many levels, a failed computation …). A column in neither list is simply outside the top drivers — say that, not "PDP is unavailable". When no curve was drawn at all, the read fails 404 and the error message names those reasons (get_insight returns them as reason on its available: false note).

A PDP curve can also carry per_cluster_average: {archetype id: values}, the same curve recomputed on one archetype's rows only (archetypes of 50+ rows), one value per grid point, the (missing) level included. Set it against average to say where a feature moves one archetype differently from everyone — the ids are the cluster_ids of the archetype reads. It is absent when the profile has no archetype that large.

The SHAP / beeswarm redirect rule

There is no per-row SHAP beeswarm read in the Analyst kit. For "feature importance" or "beeswarm"-style questions, use global mli.model_importance (the aggregate ranking the beeswarm summarizes); for per-feature shape use mli.pdp (or get_insight(id, "ale") under correlation). For a single row, get_insight(id, "case", {"row_index": N}) gives the neighbourhood story; per-row SHAP attribution and the interactive beeswarm live in the parleml UI (follow the ui_url).