Binary classification (churn)

Train a churn classifier from one frame, then score new customers. ml4 holds out its own validation internally — there is no manual train/valid/test split to prepare.

1. A small labelled frame

import numpy as np
import pandas as pd

rng = np.random.default_rng(0)
n = 2_000
tenure = rng.integers(1, 72, n)
monthly = rng.normal(70, 25, n).round(2)
support_calls = rng.poisson(1.5, n)

# Churn rises with support calls and short tenure (a learnable signal + noise).
logit = -1.2 + 0.35 * support_calls - 0.03 * tenure + rng.normal(0, 1, n)
churned = (1 / (1 + np.exp(-logit)) > 0.5).astype(int)

df = pd.DataFrame({
    "tenure_months": tenure,
    "monthly_charges": monthly,
    "support_calls": support_calls,
    "contract": rng.choice(["month", "one_year", "two_year"], n),
    "churned": churned,
})

# Hold rows back BEFORE fitting. ml4's own validation (bagged out-of-fold, or a reserved
# holdout) is what tunes the threshold and calibration, so it cannot also be the thing that
# reports the score without flattering it. These rows are never seen by the fit.
cut = int(len(df) * 0.8)
train_df = df.iloc[:cut].reset_index(drop=True)
test_df = df.iloc[cut:].reset_index(drop=True)

2. Configure and fit

Start from the fast quick_explore preset; swap to high_accuracy() when you want the strongest model and can spend the compute.

import parleml_ml4
from parleml_ml4 import AutoSupervised, Tasks
from parleml_ml4.config import presets
from parleml_ml4.contracts.metrics import EvalMetric

cfg = presets.quick_explore()
cfg.ml_task = Tasks.BINARY_CLASSIFICATION
cfg.eval_metric = EvalMetric.f1            # the threshold is tuned to this metric
cfg.data.target_column = "churned"

model = AutoSupervised(cfg).fit(train_df)  # plan → fit → post-fit

3. Score new customers

predict returns thresholded labels (at the tuned decision threshold, not a naive 0.5); predict_proba returns a calibrated class-probability frame — calibrated when a map won its bake-off, and raw otherwise, which the run tells you either way (see Probability calibration).

new_customers = pd.read_csv("new_customers.csv")

labels = model.predict(new_customers)              # pd.Series of 0/1
proba  = model.predict_proba(new_customers)        # pd.DataFrame, one column per class

scored = new_customers.assign(
    churn_label=labels.to_numpy(),
    churn_proba=proba[1].to_numpy(),               # probability of the positive class
)
scored.sort_values("churn_proba", ascending=False).head(20)

4. Evaluate and save

report = model.evaluate(test_df.drop(columns=["churned"]), test_df["churned"])
print(report)                                      # fit-time measures + reported scores

path = model.save("out/churn")
reloaded = parleml_ml4.load_pipeline("out/churn")          # full FittedPipeline back

# A reloaded bundle scores itself too — no façade and no config needed.
reloaded.evaluate(test_df.drop(columns=["churned"]), test_df["churned"])

Score the held-out rows, not the ones you fitted on. An in-sample number is not a weaker estimate of generalisation, it is a different quantity — and on a model with target encoding or a tuned threshold it can look excellent while the model is worthless on new customers. evaluate reports the fit-time measures either way; the test frame is what makes the reported score mean something.

The report includes the baseline honesty gate verdict — confirm it reads ok before trusting the model.

Variations

• Imbalanced target? Set cfg.eval_metric = EvalMetric.average_precision and let threshold tuning work; class weighting is opt-in. See Imbalanced data.
• Interpretable model? Start from presets.regulated_glassbox() (GLM / scorecard / RuleFit / EBM) or presets.regulated_monotonic() for auto-derived monotonic constraints. See Constraints.
• Raw domain columns (IP, email, address, device blobs)? See the Domain features example.

See the Supervised guide, the AutoSupervised reference, and every config parameter.

Next: Multiclass classification · Supervised learning