LFM2.5-350M-SmolDataAgent

LFM2.5-350M-SmolDataAgent

A 350M-parameter data-analysis agent fine-tuned from LiquidAI/LFM2.5-350M. Given a question and a sandbox of data files (CSV, Parquet, SQLite, HDF5, ...), it works through a single bash tool: explores the files, runs Python, and writes its final answer to /workdir/answer.txt.

Trained specifically on tasks from SmolDataEnvs.

Results

SmolDataEnvs test split: 250 tasks, 4 trials per task, temperature 0.1, up to 12 turns per episode.

Model pass@1 pass@2 pass@4
LiquidAI/LFM2.5-350M (base) 0% 0% 0%
LFM2.5-350M-SmolDataAgent 21% 26% 30%

By difficulty:

Tier Tasks pass@1 pass@2 pass@4
easy 33 56% 61% 64%
medium 118 16% 20% 25%
hard 99 15% 21% 24%

Training data

lugman/SmolDataAgent-Trajectories: 2,833 agent trajectories on SmolDataEnvs train tasks. They started as trajectories from two teacher models, LFM2.5-2.6B and Claude Haiku 5.5, which were then edited (cleaned, and trimmed of tool calls made after the answer was written) and synthetically enriched.

By source and difficulty:

Source easy medium hard Total
LFM2.5-2.6B 91 1,267 21 1,379
Claude Haiku 5.5 40 140 140 320
Synthetic enrichment 1,007 126 1 1,134
Total 1,138 1,533 162 2,833

By task group:

Source G1 lookup G2 multistep G3 analysis
LFM2.5-2.6B 0 1,362 17
Claude Haiku 5.5 0 0 320
Synthetic enrichment 1,112 19 3

Why not SmolDataEnvs-sft

FineEnvs/SmolDataEnvs-sft is the existing SFT set for this environment. A custom set was generated instead because:

  1. Mixed harnesses. Its rows come from two agent harnesses (bash, seta) with different habits, e.g. 71% of seta rows use a "Plan:" scaffold. Every trajectory here follows one convention.
  2. Lenient grading. Its grader takes the first number anywhere in the answer; 10.4% of re-gradable rows fail the stricter single-value grader used here.
  3. Better results. In a controlled comparison, LFM2.5-350M trained on this custom curated set outperformed the same model trained on SmolDataEnvs-sft.

Training

LoRA (r=16, alpha=32) with Unsloth and TRL, merged into 16-bit weights. Learning rate 4e-4, 3 epochs, effective batch size 16, max sequence length 16,384. Loss on all assistant turns.

Usage

Serve with vLLM and the LFM2 tool-call parser:

vllm serve lugman/LFM2.5-350M-SmolDataAgent --enable-auto-tool-choice --tool-call-parser lfm2

Pass the bash tool in the request and execute its calls in a sandbox. Each call is a fresh, non-stateful bash -c. The episode ends when the model replies without a tool call; read the answer from /workdir/answer.txt.

Recommended sampling: temperature 0.1, top_k 50, repetition_penalty 1.05, max_tokens 1024 per turn.

Downloads last month
67
Safetensors
Model size
0.4B params
Tensor type
BF16
·
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for lugman/LFM2.5-350M-SmolDataAgent

Finetuned
(93)
this model

Datasets used to train lugman/LFM2.5-350M-SmolDataAgent

Collection including lugman/LFM2.5-350M-SmolDataAgent