ageyko/SmolLM3-3B-mobile-lora-adapters

SmolLM3-3B LoRA adapters: fine-tuning a 3B LLM on a smartphone

The LoRA adapters behind the quality results of Fine-Tuning a 3B-Parameter LLM on a Smartphone: Characterizing Sustained Training. Code and measurement data: github.com/gordofreemo/mobile_LoRA_ft.

The layout mirrors train/checkpoints/ in the code repository, so

hf download ageyko/SmolLM3-3B-mobile-lora-adapters --local-dir train/checkpoints

puts every adapter where the evaluation commands expect it. SHA256SUMS lists the hash of every file. Each adapter directory holds the PEFT adapter_config.json and adapter_model.safetensors as they were trained; base_model_name_or_path in the configs is the path on the training cluster.

Folder	Count	Adapter	Used for
a1_lamp_1ep_seed0/checkpoint-1000	1	task adapter on LaMP-3, LaMP-4 and LaMP-7; rank 4, alpha 8, all seven projections	fused into the model trained on the phone, ageyko/SmolLM3-3B-a1lamp-4bit
per_task_<task>_1ep_seed0/final	7	one task adapter per LaMP task; rank 4, alpha 8, all seven projections	fine-tuning against retrieval
oppu_rep/<task>/task_lora_k1, oppu_rep_fixed/citation/task_lora_k1	7	task adapters under the OPPU protocol; rank 8, alpha 8, query, key and value	the model the per-user adapters are trained on
.../oppu_k1_r5_userNNN	651	per-user adapters; rank 8, alpha 16, query and value; trained on the bf16 task model	per-user adapters at full precision
.../oppu_k1_r5q4t_userNNN	651	same recipe, trained on the task model merged, quantized to 4 bits and dequantized, as on the phone	per-user results at 4-bit
.../oppu_k1_r5q4tshuf_userNNN	149	as r5q4t, but each trained on the next user's history (movie tagging, news categorization)	shuffled-history control
.../oppu_k1_r5q4td18_userNNN, .../oppu_k1_r5q4td24_userNNN	100 each	as r5q4t, adapting only the last 18 or 24 of the 36 blocks (movie tagging)	adapter depth

The seven OPPU tasks are citation identification, movie tagging, news categorization, news headline generation, product rating, scholarly title generation and tweet paraphrasing. Citation identification lives under oppu_rep_fixed/: it uses our repair of the citation prompt released with OPPU, which has no slots for the two candidate papers (see oppu_replication/PATCHES.md, P18).

Use

A per-user adapter applies on top of its task's model: SmolLM3-3B with task_lora_k1 merged into it (for the r5q4t families, merged and then quantized to 4 bits and dequantized, built by scripts/q4eval/build_q4_oppu_model.py). The repository's oppu_replication/ wrapper does this with --eval-only; condor/gen_oppu_subs.py writes the jobs and condor/run_docker.py runs them without Condor.

The adapters trained on the phone itself will be added here.

Training data

LaMP (lamp-benchmark.github.io), with the per-user splits of the OPPU release (arXiv:2402.04401). Neither dataset is included, and their own terms apply to them. A per-user adapter is trained on one user's history in those datasets.

License

Apache-2.0, the license of SmolLM3-3B.

Citation

@misc{geyko2026finetuning,
  title         = {Fine-Tuning a 3B-Parameter {LLM} on a Smartphone: Characterizing Sustained Training},
  author        = {Geyko, Andrew and Mosbach, Marius and Brinkmann, Andr{\'e}},
  year          = {2026},
  eprint        = {2610.06325},
  archivePrefix = {arXiv},
  primaryClass  = {cs.DC},
  url           = {https://arxiv.org/abs/2610.06325},
}