TabPilot: Empowering Small Language Models for Natural-Language Tabular Prediction

Code: github.com/fearlessAAA/TabPilot

TabPilot is an agentic framework that couples a 4B small language model with a plug-and-play frozen tabular foundation model (TFM) through declarative predictive query compilation. An Execution Agent iteratively compiles each natural-language request into a structured prediction query and validates it through leakage-safe tools; the frozen TFM executes the query by in-context inference; a Response Agent synthesizes the TFM predictions into a grounded answer. The agents are trained for unseen tables with a three-stage local-to-global curriculum: Privileged Semantic On-Policy Self-Distillation (PS-OPSD), Structure-Aware GRPO (SA-GRPO) with an outcome-label-free, TFM-agnostic structural reward, and Teacher-Guided Tabular Response Distillation (TTRD).

This repository holds the trained agents (backbone: Qwen3.5-4B, non-thinking mode):

Folder	Model
execution-agent/fold<k>/	Execution Agent: full weights after PS-OPSD → SA-GRPO
response-agent/fold<k>/	Response Agent: TTRD LoRA adapter (rank 32) for Qwen/Qwen3.5-4B

There is one model pair per fold k of the five-fold cross-validation over TopBench source tables (data/splits.json in the code repository). Fold k was trained only on that fold's training tables, so evaluate it only on fold k's test tables.

Usage

hf download XiaoboX/TabPilot --include "execution-agent/fold0/*" "response-agent/fold0/*" --local-dir checkpoints

Serve both agents with vLLM and run TabPilot as described in the code repository's README. The TFM is plug-and-play: the agents were trained without calling it, and the code repository uses TabPFN v3.5 by default.

License

MIT. The models are fine-tuned from Qwen/Qwen3.5-4B (Apache 2.0) on data derived from TopBench (MIT).