Feature engineering — the FE suite¶

ml4 ships a broad feature-engineering (FE) suite: domain parsers for messy real-world columns (emails, URLs, IPs, phones, addresses…), a full family of categorical target encoders, text and neural-embedding encoders, per-entity temporal look-backs, and tree-guided interactions. Every transformer is wired automatically from your column roles and config — you rarely build one by hand.

What makes the suite trustworthy is not a leakage scanner run after the fact. It is structural: each transformer is pinned to a locus that makes leakage impossible by construction. This page explains that architecture and gives a map of the whole family, with links to the deep-dive guides.

In short

Every FE step runs in one of two loci, fixed at plan time: GLOBAL (label-free, fitted once) or FOLD-LOCAL (target-aware, re-fitted per bag fold). A step's leakage class decides its locus, and an illegal pairing cannot be built.
RecipePlanner.plan() turns your config and training frame into a frozen, fingerprinted FeatureRecipe that replays the same columns at serve time.
Feature selection prunes the declared columns first. The prune must pass an out-of-sample acceptance test, or it is reverted.
The always-on clean step trims and types the declared columns, maps ±inf to missing, and casts nullable numeric columns to float64.
The family tables at the end link to the deep-dive guide for each group of transformers.

The two loci¶

Every FE step runs in exactly one of two places, decided once at plan time and never re-litigated:

GLOBAL — fit once, before foldingFOLD-LOCAL — re-fit once per bag fold

Row-local or unsupervised-fitted steps that never read the label. They fit once on full-train and run as a single AutoGluon feature generator, Ml4GlobalFeatureGenerator (a PipelineFeatureGenerator subclass). Because they don't touch y, fitting them on all rows leaks nothing.

Examples: email/URL/IP parsing, TF-IDF→SVD topics, clustering bins, contrast codings, tree-guided arithmetic interactions.

Target-aware steps that do read the label. They re-fit on the model's own feature-generator channel (Ml4FoldLocalMixin), once per bagged fold, on that fold's training slice only. A held-out row is encoded by statistics that never saw it — so out-of-fold predictions are leak-free.

Examples: target encoding (TE/WOE/…), CV-stacked text-linear, per-token target means, category→numeric aggregates.

Details: GLOBAL means full-train, not whatever AutoGluon was handed

"Fitting them on all rows leaks nothing" holds for the rows the model trains on. It stops holding the moment ml4 carves an explicit validation slice under the declared policy — a temporal run's strictly-later window, a grouped run's whole unseen groups — because AutoGluon fits its feature generator over concat([train, tuning_data]). That slice is not an exchangeable 1/k sample; it is the very period or entity the split exists to hide. Fitted over the union, a category level that first appears after the cut would get a real code instead of the unseen-level treatment, and cluster centroids would include part of the future.

fit_predictor therefore pre-fits Ml4GlobalFeatureGenerator on the training rows alone whenever such a slice exists, which puts AutoGluon on its is_fit() → transform-only branch. The validation slice is transformed by statistics it did not contribute to, and the score_val the leaderboard shows for those runs is not optimistic. On the ordinary bagged path there is no tuning_data, so the union is the training frame.

What remains — deliberately — is that on the ordinary bagged path a GLOBAL UNSUPERVISED_FITTED step is still fitted over all the training rows before the bag folds are carved, so every out-of-fold row helped compute the centroids, vocabulary or SVD basis it is then encoded with. That is a distributional, label-free nudge and it is the price of fitting representation learners once instead of k times; the alternative costs k fits of every text/cluster step for a bias no reported number carries. The exposure is named: measures.json records feature_engineering.global_unsupervised_steps — the distinct step kinds whose state is global — so a reader of a finished run can see exactly which representations are not fold-pure. Every figure ml4 reports as an honest estimate is carved out before the fit (the HoldoutPolicy fitting set behind calibration, thresholds, conformal residuals and the baseline gate; the product's outer test before that), so the exposure is confined to AutoGluon's own validation and OOF scores.

Why two loci, not one pipeline

A feature generator returns the same rows it was given, so it cannot change y or drop rows. Splitting FE into a global generator plus a fold-local mixin lets the label-free work happen once (cheap, shared) while the label-aware work re-fits per fold (safe). Sample-altering steps (resampling, outlier removal) go one step further and run inside the model's _fit — see Imbalanced data & outliers.

Details: fold-local steps with per-class slots

A fold-local step that emits one column slot per class (target encoding, per-token target means, CV-stacked text-linear on a 3+ class target) pins those slots at plan time — classes[k] names slot k. At fit it therefore has to match the labels it receives against that list, and a fold-local fit receives AutoGluon's cleaned 0 … K-1 codes, not the raw labels. Inferring which space y is in is ambiguous exactly for integer classes, so the mixin states it via labels_are_coded before fitting each step.

Such a step must satisfy parleml_ml4.contracts.transformers.PerClassFoldLocalTransformer — classes, n_classes, labels_are_coded on top of the two FoldLocalTransformer methods. Declaring class slots without the attribute raises TypeError at fit; skipping it silently would encode a different class into every slot and still emit a well-formed frame.

Leakage taxonomy → locus¶

The mapping is enforced in code. RecipeStep.__post_init__ rejects any step whose leakage class is pinned to an illegal locus at construction — a target-aware step declared GLOBAL cannot even be built.

Leakage class (`Leakage`)	Meaning	Legal locus
`ROW_LOCAL` (`rl`)	Pure row-wise / structural	GLOBAL
`UNSUPERVISED_FITTED` (`uf`)	Distribution-fitted, label-free	GLOBAL or FOLD_LOCAL
`SELECTION_INSTRUMENTAL` (`sel`)	Reads `y` once to decide structure	PLANNER or SELECT
`TARGET_AWARE_STAT` (`tas`)	Per-fold target statistics (TE / WOE / …)	FOLD_LOCAL
`SAMPLE_ALTERING` (`sa`)	Resampling / outlier removal — train-fold rows only	FOLD_LOCAL

This is why safety is a property of the type system, not of a linter.

The recipe and the planner¶

RecipePlanner.plan(X, y, config) turns your config plus the training frame into a frozen FeatureRecipe: an immutable, fingerprinted tuple of RecipeSteps. The planner decides everything once:

Instrumental feature selection prunes the declared columns first (a drop-list carried on the recipe), so the rest of the plan is built only on the survivors. One pass serves the whole run — every model in the roster then trains on the same surviving columns — and it fuses the orderings produced by the balanced instrumental models named in feature_selection.fs_rankers (default: LightGBM and CatBoost; see below). Columns with a structural role (split key, domain, entity key, time anchor, monotone pin) are rescued even when their importance is low, because the run reads them by name — the whole temporal.* role set (event_time_column, entity_columns, lag_features) only while temporal.enabled, since nothing reads any of the three when the section is off and an unconditional rescue would exempt an ordinary feature from the prune on the strength of a setting that does nothing. The prune has an arming threshold: a frame with fs_min_features (default 30) declared features or fewer is passed through untouched.
It walks the families in a fixed order — always-on clean → calendar/time → stateless domains → cross-domain match → text family → clustering → temporal lags → contrast codings → target encoders + fold-local aggregates.
Tree-guided interaction mining runs a plan-time instrumental model to pick candidate feature pairs, then materialises them as frozen structure.
Monotone directions are derived once when constraints are enabled.

Why selection runs two models¶

Selection happens on the raw columns, before the recipe's target encoders exist — so a categorical it drops takes the encoded twins that would have been built from it down with it. That makes the ranker's view of a raw categorical the weakest link in the plan, and the two default rankers disagree about exactly that, in opposite directions:

Ranker	How it sees a categorical	Bias
`lgbm`	Native categorical splits; pools levels below `min_data_per_group` (100) and caps split points at `max_cat_threshold` (32)	Under-credits a column with few rows per level
`catboost`	Ordered target statistic — close to what ml4's own target encoding will build	Can flatter a high-cardinality column

Details: the failure mode is few rows per level

The failure mode is not high cardinality as such but few rows per level. Measured on a frame whose only signal is a categorical: at ~10 rows/level LightGBM ranks it first; at ~2 rows/level it ranks it last of 61 and selection drops the only predictive column. CatBoost ranks it first in both cases.

Fusion is by best position in any ordering, so either model can save a feature the other could not use; fs_max_features keeps that bounded. The catboost ranker is skipped (with a log line) when the frame declares no categoricals, and the real_time and large_dataset presets drop to ["lgbm"] alone, where plan-time budget is the binding constraint.

The prune has to pass an acceptance test¶

An importance rank says which columns a booster leaned on. It never says what the model loses when the rest go — and the two guards around it protect other things: fs_min_features bounds how many columns may go, the protected set bounds which ones may go. So the prune is measured before it is accepted:

A held-out slice is carved with carve_holdout — the same helper the hyperparameter search uses — so a grouped run's slice holds whole unseen groups and a temporal run's slice is strictly later. ml4 prunes before cross-validation exists, so the probe has to make its own slice; making it at random would score both arms on rows correlated with their own training rows and every prune would measure as free.
Two probes are fitted on the remaining rows — one on the survivors, one on the full declared set — and scored on that slice.
If the survivors give up more than fs_revert_margin (default 0.10) of the full set's skill, the entire prune is discarded and every declared feature is kept.

Set fs_revert_margin to 0.0 to never revert. That also skips the two probe fits, so opting out buys the plan-time budget back. There is deliberately no second on/off flag.

Details: the skill scale and the thin-carve refusal

Skill is measured on a zero-anchored scale (Gini = 2·AUC−1, or R² for regression) because the margin is relative: on raw AUC a set that lost half its usable signal (0.80 → 0.65) would read as a 19% drop and slip under any bar an operator would set.

The probe refuses to answer rather than guess when the carve is too thin to measure: under 200 rows either side, or under 50 rows of the rarest class, where the score's own sampling noise is as large as the margin it would be judged against. It says so at WARNING, the configured prune ships, and measures.json records selection_probe: null. That is unmeasured, never free.

The noise reference (report only)¶

An importance ranking orders features, but it cannot say where "useful" stops: the last feature is "least important", not "useless". fs_noise_probe (on by default) answers that with columns of pure random numbers, which carry no information about the label by construction. At plan time ml4 ranks the declared features together with three noise columns in a quick LightGBM fit, three times with fresh noise, and records each feature whose split gain was at or below the strongest noise column, with how many repeats it lost in (measures.json → feature_engineering.noise_probe).

It is evidence, never a prune: nothing it lists is removed, because dropping a column on one noisy comparison is what the acceptance test above exists to stop. It runs whatever fs_enabled and fs_min_features say — "is this column better than noise?" is as fair to ask of a ten-column table as of a wide one — and the noise never reaches the fitted pipeline. The experiment page shows it under Run settings ("Features no better than noise"), and an agent reads it with get_run(view="settings"). Weigh it beside the measured importance: a column that loses to noise in all three repeats and has no permutation importance on held-out rows is a strong candidate to drop deliberately — with a person's say — in the next run.

There are two prunes, and therefore two probes: the declared pass removes raw inputs, and the engineered pass removes columns the recipe built. Each is priced by its own measurement over its own population. One number beneath both drop lists would attach a real, measured cost to the wrong set of columns.

Details: what measures.json records for both prunes

"feature_engineering": {
  "dropped_features": ["noise5", "noise8"],
  "selection_probe": {
    "metric": "gini", "n_holdout": 4000,
    "full_score": 0.7421, "kept_score": 0.7388,
    "relative_loss": 0.004447, "margin": 0.1, "reverted": false
  },
  "dropped_engineered_features": ["noise5_CL", "noise8_CL_DIST", "…"],
  "engineered_selection_probe": {
    "metric": "gini", "n_holdout": 600,
    "full_score": 0.7938, "kept_score": 0.7719,
    "relative_loss": 0.027577, "margin": 0.1, "reverted": false
  }
}

The two routinely disagree, and that is the point. Measured on one frame, the declared prune slightly helped (relative_loss: -0.0079) while the engineered prune cost 2.8%.

A measurement rides next to each drop list because a drop list on its own is unfalsifiable: it names columns and never the cost, so nothing after the run could tell a harmless prune from one that deleted the predictive column. Neither is part of the recipe fingerprint: two runs that decided the same plan must fingerprint identically whether or not a probe slice was affordable.

The thin-carve refusal above applies to both. On a frame of a few hundred rows the engineered pass can drop tens of columns with engineered_selection_probe: null. The prune ships, its cost is never measured, and only the WARNING says so.

The recipe is content-addressed (compute_fingerprint() → a SHA-256 digest), serialises to JSON, and replays to reconstruct the exact same column set at serve time. FeatureRecipe.explain() prints the plan grouped by locus with each step's rationale.

A recipe is stamped with the format it was written in (schema_version, currently ml4-recipe/2), and ml4 reads only its own: a recipe stamped with any other version — or none — is refused when it is built or read back with FeatureRecipe.from_json, because its step parameters would mean something else to this build's transformers. Re-plan the recipe and refit the run.

from parleml_ml4.fe.planner import RecipePlanner

recipe = RecipePlanner().plan(X_train, y_train, cfg)
print(recipe.explain())          # human-readable, grouped by locus
print(recipe.fingerprint)        # sha256:… — stable across replays

Each step's output column names come from the transformer's own output_names() — a single source of truth for the naming contract, so the planner never declares a column a transformer wouldn't produce.

The transformer family¶

The registry maps 31 GLOBAL step kinds and 7 FOLD-LOCAL kinds to concrete transformers (both counts include the select drop step; the GLOBAL set also holds the impute, cat_impute, power and cyclical preprocessing steps). Grouped by purpose:

Always-on cleaning (`clean`, GLOBAL, row-local)¶

The first step of every recipe, and the only one you never switch on. Three of the four things it does are keyed off the schema you declared on DataConfig — whitespace trimming and sentinel→NaN ("None" / "null" / "nan" / "", case-insensitive) on cat_features, pd.to_numeric(errors="coerce") on num_features, and calendar coercion on any declared date_columns. AutoGluon does none of these: it infers a column's type, so one stray cell flips a numeric column to categorical, and it reads the literal string "None" as an ordinary category level.

The rest is frame-wide, declared or not: clean maps ±inf to NaN in every float column, and casts every nullable Int64 / Float64 / boolean column to float64. Both keep AutoGluon from silently dropping a model or a column. Expand the two blocks below for why.

Details: ±inf is treated as missing — in every float column, declared or not

An infinity is not a value any estimator in the roster can consume. scikit-learn's SimpleImputer and StandardScaler — the front of AutoGluon's LinearModel and of its torch NN — validate with ensure_all_finite="allow-nan" ("accepts only np.nan and pd.NA; values cannot be infinite"); the forests re-check finiteness through the tree missing-value mask; XGBoost's DMatrix rejects it outright. AutoGluon answers a model that raises by logging one line and deleting it from the roster, and ml4's only hard guard is total failure — so without this step a diverse_ensemble run could ship one or two of its six requested members and still report success.

clean therefore maps ±inf → NaN across the frame, and the sweep is deliberately not restricted to num_features: undeclared columns pass straight through to the models as features, so a ratio column you never declared is exactly as lethal as one you did. The conversion is announced at WARNING with the per-column cell counts, because an infinity is a real upstream division-by-zero or log-of-zero that you can fix at the source — treating it as missing keeps the run alive, it does not make the number right.

The step can also mint the value it cleans: pd.to_numeric parses the text "inf" into np.inf. The string "inf" in a categorical column is left alone — there it is a legitimate level, and only the float is lethal.

Details: nullable Int64 / Float64 / boolean columns are cast to float64 — or AutoGluon deletes them

AutoGluon routes every feature by a raw-type string. For a pandas masked or Arrow-backed dtype that string is the dtype's own name — 'Int64', 'Float64', 'boolean' — because the numeric test behind it (np.issubdtype) raises on an extension dtype and the fall-through returns dtype.name. None of those is one of AutoGluon's six raw types, so neither generator in the type-normalisation stage claims the column and the pipeline, which has no passthrough, drops it. The column is then absent from the model matrix, and every fold-local encoder that named it silently encodes nothing. The only trace would be an AutoGluon ERROR naming the dtype and never the column.

This is not exotic data. Every dataset published from an ml4 experiment result carries Int64 columns by construction — predicted_class, cluster_id, archetype_id, rank — so a second-stage model trained on a scored batch or a cluster assignment would lose exactly the columns you published it for. A Parquet written by pandas from a convert_dtypes() frame does it too.

clean therefore casts every masked / Arrow-backed numeric or boolean column to float64, frame-wide and regardless of the declared schema, for the same reason the ±inf sweep is frame-wide. Three deliberate choices:

float64, not each dtype's numpy twin. pd.NA needs somewhere to live and numpy integers and booleans have no missing value, so the twin is only available when the column happens to be complete — a property of the batch, not of the column. A column that fitted as int64 and served with one missing cell would change dtype mid-flight, and AutoGluon answers a changed raw type by forcing the fitted one back on: astype('int64') over a NaN raises on the scoring batch.
category, string, period and tz-aware datetimes are left alone — AutoGluon's name-based branches already route those correctly. So is SparseDtype, which it unwraps to the subtype; densifying it would trade a drop that is not happening for a memory blow-up that would.
Integers past 2⁵³ warn. That is where float64 stops being exact. Magnitudes that big are an identifier, not a measurement — give the column the key1 / key2 role or drop it rather than train on a rounded ID.

The cast itself is announced at INFO with the dtype each column had, because nothing is lost by it; only the exactness case is a WARNING.

Stateless domain parsers (GLOBAL, row-local)¶

Decompose one raw string column into typed features while keeping the raw column. Wired by per-domain role lists on DataConfig.

Domain	Step kind	Emits (examples)
Email	`stateless.email`	domain / prefix cats, disposable / typo / role flags
URL	`stateless.url`	subdomain / TLD, shortener / query-count / path-depth
IP	`stateless.ip`	geo (country/city/org), private / bogon flags
User-agent	`stateless.useragent`	browser / OS / device family, is-mobile / bot
Device info	`stateless.device_info`	parses a JSON blob; reuses UA + IP parsers
Names	`stateless.names`	known / rank / p-female, length / mixed-script
Address	`stateless.address`	street / city / state, PO-box / commercial
ZIP	`stateless.zip`	place / state, pairwise haversine distance + match
Phone	`stateless.phone`	valid / region / line-type / carrier
Credit card / IBAN / VAT	`stateless.{creditcard,iban,vat}`	checksum-valid / scheme-or-country / length
Calendar	`stateless.time`	year…hour, weekend / holiday, days-since
Cross-domain	`stateless.match`	`<A>_<B>_MATCH` consistency flags (fraud signal)

→ Deep dive: Stateless domain encoders · reference data behind them: Geography & reference data.

Categorical & numeric encoders¶

Family	Step kind(s)	Locus
Target encoders (7 codes: TE / WE / ME / JS / LO / CN / GM)	`target_encode`	FOLD-LOCAL
Contrast codings (BD / HE / SE)	`contrast_encode`	GLOBAL
Clustering encoder (1-D binning)	`clustering`	GLOBAL
Category→numeric aggregates	`cv_cat_num`	FOLD-LOCAL
Within-group percentile	`group_rank`	FOLD-LOCAL
Entity fan-out cardinality	`entity_cardinality`	FOLD-LOCAL

→ Deep dive: Stateful encoders.

Text & embeddings¶

Encoder	Step kind	Locus
TF-IDF → TruncatedSVD topics	`text`	GLOBAL
CV-stacked linear-on-text (OOF)	`text_linear`	FOLD-LOCAL
Per-token target means	`multi_token`	FOLD-LOCAL
Weighted-token NMF factors	`weighted_token`	GLOBAL
Gamma-Poisson dirty-category topics	`gamma_poisson`	GLOBAL
MinHash char-n-gram fingerprints	`stateless.dirty`	GLOBAL
Neural sentence embeddings	`embedding`	GLOBAL

→ Deep dive: Text & embeddings.

Interactions & temporal¶

Family	Step kind(s)	Locus
num×num arithmetic (PL / MI / DI / MU)	`interactions`	GLOBAL
cat×cat / num-cat concat	`interactions.concat`	GLOBAL
standalone quantile bin	`interactions.bin`	GLOBAL
per-entity lags / EWMA / rolling	`temporal_lags`	GLOBAL

Tree-guided interactions are covered in Stateful encoders; per-entity look-backs in Temporal encoders (lags).