Skip to content

Regression (with intervals)

Regression enables prediction intervals and target-transform selection by default; both can be disabled. This example predicts a house price, reads the interval qualifications and evaluates on held-out rows.

1. A regression frame

import numpy as np
import pandas as pd

rng = np.random.default_rng(0)
n = 2_500
area = rng.normal(1500, 400, n).clip(400, 4000)
beds = rng.integers(1, 6, n)
age = rng.integers(0, 80, n)

# A skewed, positive target (price) — the y-transform CV gate may reshape it.
price = (120 * area + 15_000 * beds - 800 * age + rng.normal(0, 40_000, n)).clip(20_000, None)

df = pd.DataFrame({
    "area_sqft": area.round(0),
    "bedrooms": beds,
    "age_years": age,
    "neighborhood": rng.choice(["north", "south", "east", "west"], n),
    "price": price.round(0),
})

# Hold rows back BEFORE fitting: ml4's own validation fits the prediction intervals and
# the baseline gate, so it cannot also be what reports the score. These rows never reach the fit.
cut = int(len(df) * 0.8)
train_df = df.iloc[:cut].reset_index(drop=True)
test_df = df.iloc[cut:].reset_index(drop=True)

2. Configure and fit

import parleml_ml4
from parleml_ml4 import AutoSupervised, Tasks
from parleml_ml4.config import presets
from parleml_ml4.contracts.metrics import EvalMetric

cfg = presets.quick_explore()
cfg.ml_task = Tasks.REGRESSION
cfg.eval_metric = EvalMetric.rmse
cfg.data.target_column = "price"
cfg.intervals.levels = (80, 95)            # default; each yields lo/hi at that coverage

model = AutoSupervised(cfg).fit(train_df)

The target transform is chosen for you

ml4 runs a cross-validated bake-off over {identity, log1p, sqrt, yeo-johnson} on held-out original units and ships a transform only if it beats identity by a margin. Point predictions are always returned in the original target units — any transform is inverted for you.

…but AutoGluon's own validation scores are not in those units

The transform is applied to the label before AutoGluon sees the frame, so every number AutoGluon computes for itself — the model board's score_val, the ensemble's blend score — is measured against the transformed target, while measures["test"] is measured in yours. On yeo-johnson the two differ by orders of magnitude and are not monotonically related, so they must not be compared. Both sidecars say which they are quoting: deploy/leaderboard.json and deploy/ensemble_composition.json carry score_space, either "target" or "transformed:<method>".

measures.json states both sides in one block, and it is the record the deploy bundle carries (which holds no leaderboard sidecar):

"target_transform": {
  "method": "yeo-johnson",
  "ag_score_space": "transformed:yeo-johnson",
  "reported_score_space": "target"
}

reported_score_space is always "target" — every number ml4 itself reports (the test block, the baseline gate, interval coverage) is scored on the inverted, served prediction.

3. Predict points and intervals

predict gives the point estimate; predict_intervals adds every configured band; predict_interval returns one band at a single coverage.

X = test_df.drop(columns=["price"])

yhat  = model.predict(X)                         # pd.Series, original units
bands = model.predict_intervals(X)               # prediction, lo_80, hi_80, lo_95, hi_95
band95 = model.predict_interval(X, coverage=0.95) # prediction / lower / upper

print(bands.head())

Bands always obey lo <= prediction <= hi, never cross, and nest by level (the 95% band contains the 80%). Check the honesty flags before quoting a coverage:

print(model.interval.method_used)        # "cqr" or "conformal"
print(model.interval.coverage_is_exact)  # True only under IID (random) splits

Under grouped or temporal split policies coverage is only approximate — coverage_is_exact is False and the flag is surfaced in measures.json. See Prediction intervals.

4. Evaluate and save

Score the held-out rows. An in-sample number is not a weaker estimate of generalisation, it is a different quantity — and with target encoding or a tuned target transform it can look excellent on a model that is worthless on new data.

report = model.evaluate(X, test_df["price"])
print(report)

# A reloaded bundle scores itself too — no façade and no config needed.

model.save("out/price-model")
reloaded = parleml_ml4.load_pipeline("out/price-model")
reloaded.evaluate(X, test_df["price"])

Where to go next

Next: Prediction intervals · Binary classification