Inference & fast scoring¶

A fit returns a FittedPipeline, the single served artifact. It wraps the AutoGluon predictor (which already runs the global feature generator and any fold-local feature engineering) plus ml4's post-fit layers (calibration, decision thresholds, regression intervals, target transform), and exposes one prediction surface with those layers applied.

In short

predict_proba serves calibrated probabilities when a calibrator won its bake-off; predict_proba(X, calibrated=False) gives the raw ones.
predict applies the tuned decision threshold (binary) or per-class divisors (multiclass). It has no calibrated= toggle, by design.
measures.json publishes the served thresholds and the tuned ones under separate names.
Call warmup before the first production call and measure_infer_cost to learn the end-to-end latency.
For faster or dependency-light serving, use the Treelite accelerator or the airgapped pure-Python export; both verify parity before they attach.

import parleml_ml4
from parleml_ml4 import AutoSupervised, Ml4Config

cfg = Ml4Config(); cfg.data.target_column = "y"
model = AutoSupervised(cfg).fit(train_df)     # -> FittedPipeline

proba = model.predict_proba(X)                # calibrated probabilities
preds = model.predict(X)                      # thresholded class labels

AutoSupervised delegates predict / predict_proba to its fitted pipeline, so you can score straight off the façade or off the reloaded pipeline (see Deployment & export for saving and loading the bundle).

predict vs predict_proba¶

For classification, predict_proba returns a probability frame (one column per class label); predict returns thresholded class labels. For regression, predict returns point values in the target's original units (the target transform is inverted for you) and predict_proba raises.

Task	`predict` returns	`predict_proba` returns
Binary classification	labels via the tuned `BinaryThreshold`	`P(class)` frame
Multiclass classification	labels via threshold-adjusted argmax, or plain argmax	`P(class)` frame
Regression	point values (original units)	raises `ValueError`

The multiclass rule is argmax_c P(class=c) / thr_c, where each thr_c comes from a one-vs-rest sweep. Dividing by the bar upweights a class the model is under-confident about. The bars are fitted only under a hard-label metric (f1_macro, accuracy, …): a probability metric such as the default logloss cannot score a decision rule, so none are fitted and predict is plain argmax on the served probabilities. Because the bar is a divisor, only its size relative to the other classes' bars matters. Two consequences follow that you should know before you read multiclass_thresholds out of a report:

A class with too few rows in the fitting set to tune on is recorded with the degenerate 0.5, which means no operating point could be measured. The served rule does not divide by it (0.5 would be the largest divisor in the vector and would rank the class below where plain argmax puts it); it divides by the median tuned bar, which is neutral.
The whole layer is kept only if it beats plain argmax on the fitting set under the run's own metric. When it loses, predict is plain argmax and no per-class thresholds are reported.

Because those two numbers differ, the artifact publishes both, under separate names. measures.json never states one operating point and serves another:

`measures.json` key	Meaning
`thresholds`	What the served rule uses. Binary: the cut. Multiclass: the divisor vector, that is `MulticlassThresholds.divisors()`, with substitutions already applied.
`thresholds_tuned`	What each per-class sweep returned. Identical to `thresholds` unless a class was substituted.
`thresholds_substituted`	The classes the two disagree on: those whose recorded bar is a non-measurement. Empty for binary.
`threshold_objective`	Which sweep the cutoffs maximise (`f1` / `fbeta` / `accuracy`).

FittedPipeline.main_thresholds() is the served map (the same thing MLI and the batch scorer read); FittedPipeline.tuned_thresholds() is the measured one.

Details: on three multiclass metrics the sweep optimises something else

There are only three per-class sweeps (f1, fbeta and accuracy), and every metric maps onto one of them. For most metrics the map is the identity in spirit, but for three it is a genuine substitution:

`eval_metric`	Sweep actually maximised
`precision_macro`	`f1`
`precision_weighted`	`f1`
`recall_macro`	`f1`
`recall_weighted`	`accuracy`
`f1_macro` / `f1_weighted`	`f1`
`beta_macro` / `beta_weighted`	`fbeta`
`accuracy`	`accuracy`

Standalone precision and recall sweeps are not offered because each is monotone in a single class's own bar and degenerate on its own. Maximising precision alone drives the bar to predict one class almost never, and recall alone to predict it almost always. The rule that ships is joint (argmax_c P_c / thr_c emits exactly one label per row, so raising one divisor steals rows from the others), which is what a jointly-searched divisor vector would exploit. ml4 does not search a joint divisor vector.

What protects you is the acceptance gate. The whole layer is compared against plain argmax under the run's own scorer, and discarded when it loses. On these three metrics the served rule is therefore never worse than AutoGluon's own selection point; it is not tuned for the number you are graded on. threshold_objective in measures.json names the sweep that was used, so the substitution is visible in the artifact rather than inferred from this table.

The `calibrated=` toggle — on `predict_proba` only¶

ml4 owns calibration as a post-hoc layer, so both raw and calibrated probabilities are always available. By default predict_proba serves calibrated probabilities when a calibrator was fit; pass calibrated=False to force the raw AutoGluon probabilities.

cal = model.predict_proba(X)                     # calibrated (default)
raw = model.predict_proba(X, calibrated=False)   # force raw AutoGluon proba

model.is_calibration_active   # True when calibrated proba is served by default

predict has no such toggle, deliberately

ml4 fits one decision layer, on whichever probabilities serve (the calibrated frame when calibration won its bake-off, raw otherwise), and stores no raw-scale twin. A toggle on predict would therefore apply a cutoff selected on one scale to numbers on another: a binary threshold tuned at, say, 0.31 on calibrated probabilities compared against a raw column whose operating point is nowhere near it, or per-class divisors tuned on calibrated columns dividing a raw matrix. Both return plausible-looking labels at an operating point the model was never validated at. A probability is meaningful on either scale; a decision is not. If you need labels off raw probabilities, threshold them yourself against an operating point you tuned on raw probabilities.

Calibration only ships when it helps

ml4 gates calibration with a 5-fold out-of-sample log-loss bake-off (fit_calibration_if_beneficial): it attaches a map only when it beats the raw probabilities by more than calibration.calibration_advantage_margin, otherwise the pipeline serves raw. When no map is present, calibrated=True and calibrated=False return the same frame.

Which map is baked off depends on the task. Binary gets the one-vs-rest family (isotonic, or genuine Platt scaling below the rarest-class floor), and multiclass gets argmax-preserving temperature scaling. Both families, why AutoGluon's own calibration hook stays off, and how to read the verdict in measures.json are in The decision layer.

A served prediction can change without a refit

Calibration and the decision threshold can be re-decided after the fit, from the model's own stored out-of-sample output, without retraining anything. Two calls to the same deployed model, on the same row, can legitimately differ, with no model file changed in between.

FittedPipeline.decision_provenance.revision is which layer is serving, and it is stamped into every derived artifact at the time that artifact was computed. When you need to know which layer produced a batch of scores, read the stamp on the batch rather than the model: the model only knows what it serves now.

Warmup and cost measurement¶

The first real prediction is cold: lazy model and feature-generator state has to be primed. warmup runs one throwaway prediction on a tiny slice so the first production call is hot.

model.warmup(X)                             # prime lazy state (no-op on an empty frame)
sec_per_row = model.measure_infer_cost(X)   # end-to-end seconds/row (FE + model)
model.predict_is_expensive                  # True when serving is costly

measure_infer_cost times the end-to-end serving path (feature generation + model), not AutoGluon's model-only infer_limit, so it reflects the real latency an SLA sees. It warms up first, then takes the best of repeats timed passes (least contended) and records it on infer_cost_s_per_row.

Read that number before you enforce one. A per-row ceiling is set with deployment.max_inference_seconds_per_row (see Deployment), and AutoGluon applies it against the model-only cost. A budget chosen without knowing the end-to-end figure either binds on nothing or empties the roster.

predict_is_expensive is True when either a foundation / in-context leaf is in the deployed strongest model (it re-encodes its context on every predict) or a measured end-to-end cost exceeds EXPENSIVE_S_PER_ROW (0.005 s/row). Explainers read this flag to decide whether to sample rather than run exhaustively.

Fast-scoring paths¶

For latency-sensitive serving ml4 offers two acceleration paths. Both are parity-guaranteed: they are attached only after verifying they reproduce the native pipeline's output, and neither is required for correctness.

Treelite / GTIL accelerator¶

When deployment.deliver_treelite=True (the real_time preset turns it on) the delivery layer attaches a compiler-free Treelite/GTIL model core to the pipeline.

Opt-in: built only on the deliver_treelite flag; needs the realtime extra (pip install "parleml-ml4[realtime]").
Eligible-only: the strongest model must unwrap to exactly one LightGBM or XGBoost gbtree/dart booster. A bag, weighted ensemble, CatBoost, glass-box, or foundation strongest model silently keeps the native path.
Parity-guarded: the build checks the accelerated model core against the native booster on a real sample, with an absolute bound of 1e-6 (rtol=0; for regression the bound scales with the output's magnitude). On any mismatch it returns None and the pipeline ships native-only.

The accelerator is transparent: predict / predict_proba behave identically and score faster. It rides inside the bundle, and a runtime GTIL failure degrades to the native predictor, so acceleration can never break serving.

It covers every scoring surface, not only the raw-frame one. The encoded-frame methods (predict_proba_from_encoded / predict_point_from_encoded) run on the same GTIL core. MLI, per-row justify, ICE/LOCO and the product's Batch Predict score through them after a single transform. One bundle therefore has one scoring core, the same one calibration and the decision threshold see. Scoring a named member (member="LightGBM_BAG_L1") stays native by design: the accelerator holds the strongest model's booster only.

from parleml_ml4.config.presets import real_time

model = AutoSupervised(real_time()).fit(train_df)
model.treelite is not None       # accelerator attached when the strongest model is eligible
model.predict_proba(X)           # scores via GTIL, output identical to native

Airgapped pure-Python export¶

For deployment with no AutoGluon at serve time, export_python_pipeline compiles a single-model strongest model into a pure-Python ExportedPipeline.

from parleml_ml4.delivery.export import export_python_pipeline, ExportedPipeline

exported = export_python_pipeline(model, verify_on=sample_df)
exported.save("out/scorer.xz")           # one LZMA pickle

# elsewhere, with only lightgbm / scikit-learn installed:
scorer = ExportedPipeline.load("out/scorer.xz")
proba = scorer.predict_proba(X)

It reproduces the full inference path (the recipe's fitted GLOBAL transformers, AutoGluon's column coercion, the fold-local chain, calibration, thresholds, and the target-transform inverse) and verifies parity against the AutoGluon pipeline before returning, raising ExportError on a mismatch or an unsupported strongest model.

Strongest model	Airgapped export
Glass-box GLM / scorecard / RuleFit	supported (linear leaf)
LightGBM (incl. monotonic GBM)	supported (raw booster leaf)
CatBoost	supported (raw model + AutoGluon's categorical code map)
XGBoost / EBM / ensembles	deferred → raises `ExportError`

Which path?

Use the Treelite accelerator when you keep AutoGluon at serve time but want a faster tree core. Use the airgapped export when you need a dependency-light artifact that loads with only lightgbm (or scikit-learn / imodels). For a full-fidelity artifact of any strongest model, save the bundle.

See Prediction intervals for regression uncertainty bands.

Next: Deployment & export · MLOps