Prediction intervals (regression)¶

Regression fits enable prediction intervals by default as a post-fit layer.

In short

Every regression fit ships bands at the levels in intervals.levels (default (80, 95)), built by Conformalized Quantile Regression (CQR) with a split-conformal fallback.
model.predict_intervals(X) returns the prediction plus lo_L / hi_L per level; model.predict_interval(X, coverage=0.95) returns one band.
Coverage is exact only on IID splits. Read model.interval.exchangeable and low_confidence before you rely on a nominal level.
Check empirical coverage (PICP) and width (MPIW) on untouched test rows with model.interval.metrics(...).

Bands report nominal coverage levels and flags for exchangeability and insufficient calibration evidence. Read those flags and evaluate coverage on untouched test rows before relying on a nominal level.

Intervals are default-on for regression (IntervalsConfig.enabled = True); set cfg.intervals.enabled = False to turn them off. Classification is untouched: predict_proba calibration is a separate concern (see the decision layer).

Method: CQR with a split-conformal fallback¶

The layer uses Conformalized Quantile Regression (CQR) with a split-conformal fallback, over multiple nominal coverage levels, producing adaptive-width bands (the width varies per row).

`method`	How the band is built	Width
`cqr`	LightGBM quantile companions estimate the conditional lower/upper quantiles, corrected by a held-out conformal offset	adaptive (per-row)
`conformal`	symmetric split-conformal offsets on the winner's leak-free point residuals	constant per level
`auto` (default)	CQR when the companions fit, else `conformal`	adaptive or constant

Under CQR the quantile companions are a fixed, light LightGBM learner on the raw modelling features — not a clone of the winner. CQR's coverage guarantee does not depend on the companion matching the winner's family: conformalization corrects any miscoverage. That makes CQR available for every regression winner (including heterogeneous ensembles), and the companions train on the original-units target directly, so no target-transform inversion is needed. The winner's point prediction is always the reported centre; the companions supply the band.

If the companions cannot be fit (LightGBM missing, a degenerate fit, or no usable features) the layer falls back to split-conformal so intervals never fail a run.

Which columns the companions see¶

Declared roles win: when data.num_features / data.cat_features are set — the product always sets them — the companions train on exactly those columns, minus the label, engine.sample_weight and the split policy's group_column / time_column.

When neither role list is declared the columns are inferred from the training frame instead. Numeric and boolean columns go in; string and categorical columns go in only when they carry repeated levels (at most 200 distinct values, and no more than one distinct value per two rows), which is what keeps free text and near-unique identifiers out. Datetime columns are always left out — LightGBM rejects datetime64 outright. Entity keys (data.key1 / key2) and columns routed to a text-family or time domain are left out too, matching what the product path excludes.

The route taken is logged: an INFO line naming the inferred columns, or a WARNING when no column of the frame is eligible and the layer therefore falls back to constant-width bands. Declare the role lists to choose the set yourself — an explicit declaration is never filtered by the heuristics above.

Coverage levels¶

Levels are configured on IntervalsConfig.levels as strictly increasing integer percentages. The default is (80, 95). Each level L yields an lo_L / hi_L band pair.

from parleml_ml4 import Ml4Config

cfg = Ml4Config()
cfg.data.target_column = "y"
cfg.intervals.levels = (80, 95)      # default; each yields lo_80/hi_80, lo_95/hi_95
cfg.intervals.method = "auto"        # CQR when companions fit, else conformal

Bands always obey lo <= prediction <= hi, are non-crossing, and nest by level (a 95% band contains the 80% band) — enforced at predict time.

predict_interval and predict_intervals¶

predict_intervals returns the point prediction plus every configured band; predict_interval returns a single band at one coverage level.

# all configured levels at once
bands = model.predict_intervals(X)
# columns: prediction, lo_80, hi_80, lo_95, hi_95

# a single band (prediction / lower / upper)
band95 = model.predict_interval(X, coverage=0.95)

coverage must correspond to a configured level (0.95 → level 95); passing a level that was not configured raises a ValueError that lists the available coverages. When coverage is omitted, the pipeline's interval_coverage default is used.

Honesty: exactness and exchangeability¶

The finite-sample coverage guarantee assumes the calibration slice is exchangeable with the test data. That holds for IID (random) splits, which give exact marginal coverage. It does not hold for grouped splits (within-group correlation) or temporal splits (distribution shift) — under those SplitPolicy strategies nominal coverage is only approximate.

The layer records this so the run can disclose it honestly:

model.interval.method_used        # "cqr" or "conformal"
model.interval.exchangeable       # False under grouped / temporal splits
model.interval.coverage_is_exact  # True only when the guarantee holds (IID splits)
model.interval.low_confidence     # True when the calibration slice cannot back the bands

low_confidence clears two floors, not one

It is True below 30 calibration rows (the generic power floor, shared with the baseline gate) and whenever any configured level needs more rows than the slice has. The conformal quantile picks rank ceil((n + 1) · p) and clips it into [1, n]; below p / (1 − p) rows the clip binds and the band is the largest observed residual — the same band every higher level would return, so the nominal level is not represented. The default levels = (80, 95) needs 4 and 19 rows, so the 30-row floor is what binds there; a configured 99 needs 99 rows on its own.

Approximate coverage under grouped / temporal splits

When exchangeable is False, treat the nominal level as a target, not a promise — the empirical coverage may drift. The flag is surfaced in measures.json so downstream consumers see it.

PICP and MPIW metrics¶

IntervalLayer.metrics reports the two honest checks per level on a labelled set: empirical coverage (PICP, prediction interval coverage probability) and mean width (MPIW, mean prediction interval width).

bands = model.predict_intervals(X_test)
scores = model.interval.metrics(y_test.to_numpy(), bands)
# {"interval_coverage_80": 0.81, "interval_width_80": ...,
#  "interval_coverage_95": 0.94, "interval_width_95": ...}

A well-calibrated interval has interval_coverage_L close to L/100; the width tells you how tight the band is at that coverage.

Interaction with the target transform¶

Regression can train on a shape-corrected target (see the regression target transform): the model fits a transformed target and every point prediction is inverted back to original units. Intervals stay in original units regardless of method:

CQR companions train on the original-units target directly, so they never see the transform and need no inversion — the band is native to the user's units.
Split-conformal offsets are computed on the winner's original-units point residuals (the point prediction is already inverted), so the band is likewise in original units.

Either way, predict_intervals centres each band on the inverted point prediction, so predictions and bands read in the same units the target came in.

See Inference & fast scoring for the full prediction surface and Algorithms & metrics for how the regression eval metric is chosen.

Next: Inference & fast scoring · Supervised learning