Executive summary
On 300 held-out validation questions the fine-tuned model answered 96.7% correctly versus 51.0% for the untuned base model. Cosmos3-Nano-qwen3vl was fine-tuned with LoRA (rank 16, targets q_proj, k_proj, v_proj, o_proj) for 5 epochs, 625 optimizer steps, on 4 GPUs. Training loss fell from 0.290 at the first step to a steady 0.0328 over the final tenth of training (-89%). Validation loss improved every epoch, ending at 0.0353 (best 0.0353); the best checkpoint is epoch_5. The last epoch changed validation loss by under 5%, so the run has essentially converged; additional epochs would bring little. The job completed in 24 minutes of wall time.
Task accuracy on held-out data
Accuracy by question type
300 validation samples, 5 question types, greedy decoding. On 300 held-out validation questions the fine-tuned model answered 96.7% correctly versus 51.0% for the untuned base model.
Inspection-report field accuracy
Per-field scoring of the JSON report answers.
| Report field | fine-tuned (LoRA epoch 5) | base model (zero-shot) |
|---|---|---|
| Valid JSON returned | 100.0% | 100.0% |
| Per-vial content | 99.7% | 15.8% |
| Per-vial status | 99.2% | 85.2% |
| Fill level within 10 pts | 99.0% | 48.7% |
| Deviating set exact | 96.7% | 40.0% |
| Pass / fail verdict | 100.0% | 51.7% |
Over 60 reports covering 398 vials.
Where it still fails · 10 of 300
Training dynamics
Per-epoch results
Optimization health
Insights
Metrics by step
Run configuration
Reproduce
Commands used for this run (see README for the full runbook).
export ACCOUNT=general_sa PARTITION=gb200 LUSTRE_DIR=/lustre/fsw/general_sa/$USER
cd $LUSTRE_DIR/cosmos3tao/repo && git pull
bash cluster/ptyche_setup.sh # dataset, model download, container .sqsh (once)
bash cluster/ptyche_prepare_model.sh # Cosmos3-Nano omni -> Qwen3-VL checkpoint (once)
sbatch --account=$ACCOUNT --partition=$PARTITION --export=ALL,ACCOUNT=$ACCOUNT,LUSTRE_DIR=$LUSTRE_DIR \
--output=$LUSTRE_DIR/cosmos3tao/logs/%x-%j.out --error=$LUSTRE_DIR/cosmos3tao/logs/%x-%j.err cluster/ptyche_train.sbatch
python report/make_report.py --log results/train.log --spec results/train_spec.yaml --out report.html