Skip to content

Profiling — fit_profiler & ProfilingResult

fit_profiler is the entry point. It takes a fitted model and a training frame and returns a single ProfilingResult — the hub every other capability reads from.

In short

  • Pass a fitted model, its training frame X and a target_mask: the rows whose archetypes you want, usually the positives.
  • Every tuning knob lives in one ProfilingConfig. An optional section (rules, explanations, eda, report) runs only when you set it.
  • Archetypes cluster in tree-leaf space or SHAP space. HistGradientBoosting, single trees and glass-box models always use SHAP space.
  • Regression profiles a slice you choose; multiclass profiles every class one-vs-rest.
  • result.save(...) writes one joblib that reloads without refitting.
from parleml_mli import ProfilingConfig, fit_profiler

config = ProfilingConfig()  # archetypes only; optional stages omitted
result = fit_profiler(
    model=clf, X=X_train, y=y_train, target_mask=(y_train == 1), config=config,
)

See the full signature and every data argument on the fit_profiler reference. All tuning knobs live in ProfilingConfig.

The three core inputs

Argument What it is
model A fitted estimator (CatBoost / LightGBM / XGBoost / sklearn ensemble / HGB / single tree / linear / EBM / RuleFit) or a raw booster.
X The feature frame the model was trained on (one row per observation).
target_mask A boolean mask selecting which rows to profile — the archetype cohort.

target_mask is the single most important choice: archetype clustering, signatures, and similar-case retrieval all describe the rows where the mask is True. For a classifier that's usually the positive class (y_train == 1); for regression it's a quantile or business cutoff (below).

The ProfilingResult hub

fit_profiler always returns a populated-on-demand ProfilingResult. Configure the heavier artifacts in ProfilingConfig, or populate them lazily afterwards:

Attribute Config section Lazy builder (a function, not a method) Holds
summary_df, labels_ core profiling — per-archetype summary and target-cohort cluster assignments
stability_ archetypes=ArchetypeConfig(n_bootstrap=...) build_stability_result(result, ...) optional bootstrap stability; default n_bootstrap=0 skips it
rules_ rules=RulesConfig(...) build_rules_result(result, ...) RulesResult (engine → RuleSetSummary)
explanations_ explanations=ExplanationsConfig(...) build_explanations_result(result, ...) ExplanationsResult
eda_ eda=EDAConfig(...) build_eda_result(result, ...) EDAResult
model_evaluation_ pass labelled X_test / X_valid splits build_model_evaluation_result(result, ...) ModelEvaluationResult
classes (multiclass auto) — per-class ClassProfile fan-out 
from parleml_mli import (
    ProfilingConfig, RulesConfig, ExplanationsConfig, EDAConfig,
    PdpConfig, fit_profiler,
)

config = ProfilingConfig(
    rules=RulesConfig(),
    explanations=ExplanationsConfig(pdp=PdpConfig()),
    eda=EDAConfig(),
)
result = fit_profiler(
    model=clf, X=X_train, y=y_train, target_mask=(y_train == 1),
    X_test=X_test, y_test=y_test,
    config=config,
)
result.plot_umap()
result.save("out/profiling_result.joblib")

Tuning with ProfilingConfig

All tuning knobs are passed as a single typed ProfilingConfig composed from small, frozen section dataclasses. Data arguments (model / X / y / target_mask / holdout splits / encoding_registry / task_type) always stay flat — only the tuning knobs move into config:

from parleml_mli import (
    ProfilingConfig, ClusteringConfig, RulesConfig, MinerConfig, AcceptanceGates,
    ExplanationsConfig, PdpConfig, ReportConfig, SpaceType, fit_profiler,
)

config = ProfilingConfig(
    clustering=ClusteringConfig(space_type=SpaceType.SHAP, min_cluster_size=10),
    rules=RulesConfig(miner=MinerConfig(
        acceptance=AcceptanceGates(min_lift=1.3, min_tp_count=8))),
    explanations=ExplanationsConfig(pdp=PdpConfig(n_features=6)),
    report=ReportConfig(output_dir="out/"),
)
result = fit_profiler(model=clf, X=X_train, y=y_train,
                      target_mask=(y_train == 1), config=config)

Two rules keep it predictable: presence enables a stage (an optional section left None — rules / explanations / eda / report — is skipped; there are no separate boolean toggles), and data stays flat; all tuning lives in one typed ProfilingConfig. See the configuration reference for every section and field.

Details: reading back the fit inputs

The result is a self-contained bundle. Alongside the artifacts above, the fitted inputs they were derived from are exposed as read-only accessors, so a saved profile is usable on its own:

Accessor Holds
model_ / model_type_ the fitted estimator and its detected ModelType
feature_names_ encoded feature names (aligned with target_features_ / target_shap_values_)
target_features_ encoded target-cohort matrix the profile was built from
target_features_original_ original-space values (only when X_target_orig= was passed)
target_predictions_ model scores for the target cohort (aligned with labels_)
target_shap_values_ (n_rows, n_features) SHAP matrix for the target cohort
target_mask_ boolean mask selecting target rows from the fit-time X
id_columns_ original columns pinned into representative examples

The full evaluation matrix X is not stored. Reload the source dataset and pass X= to the build_*_result functions when you need to recompute an artifact.

Spaces — leaf vs SHAP

Archetype distances are computed in either leaf space (co-occurrence in tree leaves) or SHAP space (contribution vectors). Set space_type on ClusteringConfig:

from parleml_mli import ProfilingConfig, ClusteringConfig, SpaceType, fit_profiler

config = ProfilingConfig(clustering=ClusteringConfig(space_type=SpaceType.LEAF))
result = fit_profiler(model=clf, X=X_train, y=y_train, config=config)

config = ProfilingConfig(clustering=ClusteringConfig(space_type=SpaceType.SHAP))
result = fit_profiler(model=clf, X=X_train, y=y_train, config=config)

Families with no per-sample leaf structure (HGB, single DecisionTree, and all glass-box models) auto-switch LEAF → SHAP.

Space changes the question

Leaf space groups cases by the model's tree paths; SHAP space groups them by feature contributions. Coverage depends on the data, geometry and survival filters. Compare the resulting coverage and signatures on your own run.

Regression

Pass task_type=TaskType.REGRESSION and choose the slice you want via target_mask. Provide y= to unlock ground-truth-aware per-archetype diagnostics.

import numpy as np
from parleml_mli import ProfilingConfig, ClusteringConfig, SpaceType, TaskType, fit_profiler

threshold = float(np.quantile(y_train, 0.85))      # top 15% by y
config = ProfilingConfig(clustering=ClusteringConfig(space_type=SpaceType.LEAF))
result = fit_profiler(
    model=my_regressor, X=X_train, y=y_train,
    target_mask=(y_train > threshold),
    task_type=TaskType.REGRESSION,
    config=config,
)
print(result.summary_df[[
    "cluster_id", "n_target_cases",
    "mean_y_true", "mean_prediction", "mean_residual", "mae", "rmse",
]])

Common target_mask recipes for regression:

Slice target_mask recipe Profiles
Top-quantile y y_train > np.quantile(y_train, q) sub-types of high-value cases
Top-quantile y_pred y_pred > np.quantile(y_pred, q) model-driven hot zones
Residual outliers np.abs(y_train - y_pred) > q where the regressor is most uncertain
Systematic over-pred (y_pred - y_train) > q sub-types of positive bias

Rule mining is binary-only

Rule miners assume a {positive, negative} dichotomy (lift, odds ratio, Wilson bound, F-beta). On a regression problem, binarize via target_mask first and mine rules against a classifier of the same family. With task_type=TaskType.REGRESSION, ProfilingConfig(rules=RulesConfig(...)) is skipped with a UserWarning and result.rules_ stays None.

Multiclass

When len(model.classes_) > 2, profiling fans out one-vs-rest per class and result.classes holds the per-class ClassProfile containers. When profiling with explanations, restrict the fan-out with ExplanationsConfig(ovr_classes=...) when only a subset of labels is interesting.

Glass-box native structure

For inherently-interpretable families — GLM / logistic (SKLEARN_LINEAR), EBM (EBM), RuleFit (RULEFIT) — result.native_explanation() returns the model's exact readable structure (GLM coefficients + odds ratios, EBM additive shape functions, or the fitted RuleFit rule table) as a JSON-clean dict. It returns None for opaque models (trees / boosters), which have no single readable structure beyond importance + SHAP.

Native categorical features

Native categoricals (LightGBM categorical_feature=, XGBoost enable_categorical=True, sklearn HGB categorical_features=, CatBoost cat_features=) flow through end-to-end. The rule extractor decodes set-membership splits into Operator.IN / Operator.NOT_IN antecedents like (CITY ∈ {Toronto, Montréal}) in the original labels — see the Rules guide.

Persisting

result.save("out/profiling_result.joblib")  # one self-contained joblib

Reload it later with ProfilingResult.load("out/profiling_result.joblib") to read every populated surface without re-running the pipeline. Set ReportConfig(output_dir="out", persist_inference_data=True) on ProfilingConfig.report to also materialise the supplied train/valid/test frames under out/model_training_data/. Persistence requires output_dir; calling save() alone does not write those frames or create the destination directory. See Consuming saved results for schema compatibility and the separate narration annotation.

Next