---
license: mit
base_model: Qwen/Qwen3.5-4B
library_name: transformers
pipeline_tag: text-generation
tags:
  - tabular
  - agent
  - tool-use
  - tabular-foundation-model
---

# TabPilot: Empowering Small Language Models for Natural-Language Tabular Prediction

Code: [github.com/fearlessAAA/TabPilot](https://github.com/fearlessAAA/TabPilot)

TabPilot is an agentic framework that couples a 4B small language model with a **plug-and-play
frozen tabular foundation model (TFM)** through **declarative predictive query compilation**. An
**Execution Agent** iteratively compiles each natural-language request into a structured
prediction query and validates it through leakage-safe tools; the frozen TFM executes the query
by in-context inference; a **Response Agent** synthesizes the TFM predictions into a grounded
answer. The agents are trained for unseen tables with a three-stage local-to-global curriculum:
**Privileged Semantic On-Policy Self-Distillation (PS-OPSD)**, **Structure-Aware GRPO (SA-GRPO)**
with an outcome-label-free, TFM-agnostic structural reward, and **Teacher-Guided Tabular Response
Distillation (TTRD)**.

This repository holds the trained agents (backbone: Qwen3.5-4B, non-thinking mode):

| Folder | Model |
|---|---|
| `execution-agent/fold<k>/` | Execution Agent: full weights after PS-OPSD → SA-GRPO |
| `response-agent/fold<k>/` | Response Agent: TTRD LoRA adapter (rank 32) for `Qwen/Qwen3.5-4B` |

There is one model pair per fold `k` of the five-fold cross-validation over TopBench source tables
(`data/splits.json` in the code repository). Fold `k` was trained only on that fold's training
tables, so evaluate it only on fold `k`'s test tables.

## Usage

```bash
hf download XiaoboX/TabPilot --include "execution-agent/fold0/*" "response-agent/fold0/*" --local-dir checkpoints
```

Serve both agents with vLLM and run TabPilot as described in the code repository's README. The
TFM is plug-and-play: the agents were trained without calling it, and the code repository uses
TabPFN v3.5 by default.

## License

MIT. The models are fine-tuned from [Qwen/Qwen3.5-4B](https://huggingface.co/Qwen/Qwen3.5-4B)
(Apache 2.0) on data derived from [TopBench](https://github.com/LAMDA-Tabular/TopBench) (MIT).