license: mit
base_model: Qwen/Qwen3.5-4B
library_name: transformers
pipeline_tag: text-generation
tags:
- tabular
- agent
- tool-use
- tabular-foundation-model
TabPilot: Empowering Small Language Models for Natural-Language Tabular Prediction
Code: github.com/fearlessAAA/TabPilot
TabPilot is an agentic framework that couples a 4B small language model with a plug-and-play frozen tabular foundation model (TFM) through declarative predictive query compilation. An Execution Agent iteratively compiles each natural-language request into a structured prediction query and validates it through leakage-safe tools; the frozen TFM executes the query by in-context inference; a Response Agent synthesizes the TFM predictions into a grounded answer. The agents are trained for unseen tables with a three-stage local-to-global curriculum: Privileged Semantic On-Policy Self-Distillation (PS-OPSD), Structure-Aware GRPO (SA-GRPO) with an outcome-label-free, TFM-agnostic structural reward, and Teacher-Guided Tabular Response Distillation (TTRD).
This repository holds the trained agents (backbone: Qwen3.5-4B, non-thinking mode):
| Folder | Model |
|---|---|
execution-agent/fold<k>/ |
Execution Agent: full weights after PS-OPSD → SA-GRPO |
response-agent/fold<k>/ |
Response Agent: TTRD LoRA adapter (rank 32) for Qwen/Qwen3.5-4B |
There is one model pair per fold k of the five-fold cross-validation over TopBench source tables
(data/splits.json in the code repository). Fold k was trained only on that fold's training
tables, so evaluate it only on fold k's test tables.
Usage
hf download XiaoboX/TabPilot --include "execution-agent/fold0/*" "response-agent/fold0/*" --local-dir checkpoints
Serve both agents with vLLM and run TabPilot as described in the code repository's README. The TFM is plug-and-play: the agents were trained without calling it, and the code repository uses TabPFN v3.5 by default.
License
MIT. The models are fine-tuned from Qwen/Qwen3.5-4B (Apache 2.0) on data derived from TopBench (MIT).