Skip to content

Cookbook — worked analyst sessions

In short

  • Four analyst jobs, each also an MCP prompt: explore a workspace, audit a model, investigate anomalies and triage a dataset.
  • Every job follows one spine: resolve names, read the catalog, call the right tool, cite the numbers with their ui_url links.
  • The agent never recomputes a number a tool returns.

Four canonical jobs, each also shipped as an MCP prompt (hosts surface them like slash commands). Every session follows the same spine: resolve names → catalog → the right read → cite numbers + ui_url links — and never recompute a number a tool already returns. Delivering models is its own recipe: Deliver with agents.

Explore a workspace (explore_workspace)

  1. list_resources("workspaces") → resolve the name to workspace_id.
  2. describe_catalog — every queryable table and the join edges.
  3. profile_table the interesting tables (row counts, types, top values).
  4. query_sql for the first real question — e.g. predicted positives by region, joined to inputs on a synthesized row_index.

Audit a model (audit_model)

  1. get_capabilities → what exists for this experiment.
  2. get_model_evaluation — metrics + curves; name the operating point (confusion_curve recomputes any threshold without re-scoring).
  3. get_insight: trust_summary, then the concerning tiles — fairness, error_analysis, drift, label_noise; leaderboard for how close the challengers were.
  4. get_rules for the model's IF-THEN behaviour. Verdict with numbers.

Investigate anomalies (investigate_anomalies)

  1. get_chart_data("anomaly") (how to read each chart) → get_chart_data("anomaly", "score_distribution" | "top_k_table", experiment_id).
  2. query_sql over *__anomalies joined to the input for the business columns behind the top cases.

Triage a dataset (triage_dataset)

  1. get_eda(dataset_id) — table stats, alerts, variable names.
  2. get_eda(dataset_id, section="leakage") — findings are blockers to flag.
  3. probe_dataset(dataset_id, "signal", {"target": …}) (a compute token) — verdict + drivers before proposing any modelling; the "baseline_floor" probe gives the score a trivial predictor already reaches.

Try it on seeded data: the backend e2e test (backend/app/tests/test_mcp_e2e.py) drives every tool against a seeded workspace — a runnable, always-green transcript of the whole surface.

Next