Tabular foundation models¶

ml4 uses AutoGluon's stock tabular foundation models: in-context learners that predict zero-shot, without gradient training. They serve two roles: a leaderboard challenger inside a full run, and the estimator behind the standalone signal check.

In short

One model is catalogued: TabICL v2 (tabicl), bundled and the default.
In a full run it is only ever a challenger: an extra ensemble member, never the deployed strongest model.
Turn it on with cfg.roster.include_foundation_challenger = True, or start from the foundation_ensemble() preset.
Once on, it must train: a run where it cannot fails with FoundationMemberNotTrained and the reason.
It runs much faster on a GPU; install a CUDA torch wheel for that.

The catalogue¶

One model is catalogued: TabICL v2, the bundled default. It ships in the base image and is always present. A model is offered only while its backing package is installed, and only models whose weights are licensed for commercial use are catalogued at all.

Key	Label	Gating package	Bundled	Signal-check estimator
`tabicl`	TabICL v2	`tabicl`	yes (default)	yes

Availability is a cheap importlib.util.find_spec probe: no torch and no weights are imported to answer "is it installed?".

from parleml_ml4.contracts.foundation import (
    FOUNDATION_CATALOG, DEFAULT_FOUNDATION,
    foundation_available, available_foundation_models, foundation_catalog,
)

DEFAULT_FOUNDATION                 # "tabicl"
foundation_available("tabicl")     # True when the tabicl package is installed
available_foundation_models()      # installed keys, default first
foundation_catalog()               # UI-facing list with per-model availability

Not offered: TabPFN, Mitra, the original TabDPT

TabPFN's current checkpoints are licensed for non-commercial use only.
Mitra needs packages ml4 does not install, and has no standalone estimator.
The original TabDPT wrapper's settings do not match the weights current tabdpt releases load.

None of them is catalogued, whatever is installed. A config whose roster.foundation_model names one is refused when it is built, with the reason (parleml_ml4.contracts.foundation.WITHDRAWN_FOUNDATION_MODELS).

Challenger, never strongest model¶

A foundation model is always a challenger, never the deployed strongest model.

It is excluded from the default portfolio. When you add it, it joins as one more decorrelated ensemble member.
ml4's regulated (single-model) serving path never deploys it as the strongest model. The strongest model is a conventional booster, glass-box model or ensemble, with the foundation model contributing only as an ensemble ingredient. That keeps serving predictable.

Foundation models handle categoricals in-context, so, like EBM, they run no fold-local target encoding. They ride the global feature generator as a plain stock AutoGluon model. The heavy library import (torch and weights) is lazy, inside each model's fit.

Adding the challenger¶

There are three ways to add the foundation challenger:

from parleml_ml4 import Ml4Config, AutoSupervised
from parleml_ml4.config.presets import foundation_ensemble, with_foundation_challenger

# 1) the roster flag
cfg = Ml4Config(); cfg.data.target_column = "y"
cfg.roster.include_foundation_challenger = True
cfg.roster.foundation_model = None              # None -> TabICL default

# 2) the composer (requires the challenger on a non-regulated roster)
cfg = with_foundation_challenger(Ml4Config(), model="tabicl")

# 3) the preset — a diverse roster + the challenger, all ensembled
cfg = foundation_ensemble()

model = AutoSupervised(cfg).fit(train_df)

roster.include_foundation_challenger turns the challenger on, and roster.foundation_model selects which one (None → TabICL). Once on, the challenger is required: see When it cannot train.
with_foundation_challenger(config, model=...) sets the flag on an existing (or fresh) config. On a regulated (single-model) run it warns and does nothing, because a regulated run trains exactly one transparent model.
foundation_ensemble() is a diverse roster plus the foundation model as a decorrelated member. It uses TabArena's pipeline shape with ml4's own fold and stack settings: 8-fold bagging, one bag set, no stacking layer, and every roster member trained. It is best on small and medium tables, and strongly GPU-friendly.

Regulated runs ignore the challenger

A regulated (glass-box) run is single-model by design, so the foundation challenger is never trained there. with_foundation_challenger warns that the flag has no effect.

When it cannot train¶

A run that turns the challenger on exists to set the foundation model beside the stock roster. So the challenger trains or the run fails; a run never finishes without it and reads as a comparison it did not make. AutoGluon treats every roster member as optional (it skips one that will not fit and finishes on the rest), so ml4 checks after the fit:

What happened	When it fails	What the error says
The model's package is not installed	before anything trains	`PreflightError` naming the package
AutoGluon estimates more memory than is free	after the fit	`FoundationMemberNotTrained`, `cause="memory"`, with the need and the free memory (GPU memory on a GPU run: named, without figures)
The time budget runs out before or while it trains	after the fit	`cause="time"`, naming `engine.time_limit`
Its own size limits exclude the table (TabICL: 500,000 rows, 2,000 features)	after the fit	`cause="constraint"`, with AutoGluon's reason
It raises while training	after the fit	`cause="error"`, with the exception

FoundationMemberNotTrained lives in parleml_ml4.contracts.foundation, which imports nothing heavy. is_resource is true for a memory shortfall: nothing is wrong with the data or the configuration, and the same run succeeds where the memory is free.

Details: how the memory check works

The memory figures are AutoGluon's own. It trains a model only when its estimate is at most 90% of the free memory. On a bagged run it checks each fold, so the rows are one fold's. You can run the same check before a fit:

from parleml_ml4.contracts.tasks import Tasks
from parleml_ml4.engine.foundation_member import foundation_memory_check

check = foundation_memory_check(Tasks.BINARY_CLASSIFICATION, n_rows=3_780, n_features=34)
check.fits         # False when 3.48 GB is more than 90% of the memory free now
check.describe()   # "TabICL v2 needs about 3.48 GB (AutoGluon's estimate for 3,780 rows × 34 features) and … GB is free now; …"

A model that trained and then lost was compared, so the run succeeds. That covers a model with no weight in the ensemble, and one kept out of it by a latency ceiling (deployment.max_inference_seconds_per_row). It has its own leaderboard row.

Standalone estimator¶

For a zero-shot probe outside a full run, foundation_estimator returns the raw library estimator class (a plain scikit-learn-style fit / predict[_proba]), not the AutoGluon wrapper. The signal check uses it.

from parleml_ml4.engine.models.foundation import foundation_estimator

Estimator = foundation_estimator("tabicl", is_classification=True)  # TabICLClassifier
model = Estimator().fit(X_ctx.to_numpy(), y_ctx.to_numpy())

Requesting a key that is not catalogued raises ValueError. Requesting a catalogued but uninstalled model raises PreflightError with the package to install.

GPU¶

Foundation models are torch-backed and benefit strongly from a GPU.

The challenger follows the run's global device resolution, not a per-model override. An explicit num_gpus=1 would error outright on a GPU-less host, so ml4 sets none.
torch defaults to a CPU build. Install a CUDA torch wheel for GPU inference.
execution.device = "cuda" refuses the run when the device is unavailable, while "auto" falls back to CPU. See GPU acceleration.

See Signal check for the zero-shot probe and the configuration reference for the roster config fields.

Next: Signal check · GPU acceleration