Python4 AFT adapters (arcadia-impact/python4-gemma3-27b-aft)

Five matched rank-64 LoRA adapters for studying whether supervised Python4 code demonstrations activate held-out Python4 rules installed during midtraining. Python4 is a controlled fictional language, not a real Python release.

Experiment run: 20260811T052635Z. Each adapter saw the same ordered 512-row dataset for eight epochs (128 optimizer steps). The four held-out rule families never appear in AFT targets.

Primary generic-Python results

Parent / adapter Python4 adoption pre β†’ post Boa pass pre β†’ post Held-in pre β†’ post Held-out pre β†’ post Held-out Ξ” (95% CI)
control 0.0% β†’ 88.3% 0.0% β†’ 26.6% 0.0% β†’ 25.3% 0.0% β†’ 2.4% 2.4% [0.0%, 6.1%]
mixed_1ep 0.0% β†’ 89.8% 0.0% β†’ 26.6% 0.0% β†’ 25.1% 0.0% β†’ 1.8% 1.8% [0.0%, 4.9%]
ordered_1ep 0.0% β†’ 89.1% 0.0% β†’ 22.7% 0.0% β†’ 22.1% 0.0% β†’ 3.0% 3.0% [0.5%, 6.3%]
mixed_4ep 0.0% β†’ 89.1% 0.0% β†’ 23.4% 0.0% β†’ 22.1% 0.0% β†’ 3.0% 3.0% [0.5%, 6.4%]
ordered_4ep 0.0% β†’ 93.8% 0.0% β†’ 26.6% 0.0% β†’ 25.9% 0.0% β†’ 0.6% 0.6% [0.0%, 2.0%]

Explicit-Python3 spillover and selectivity

Arm Python3 pass pre β†’ post Python4 spillover pre β†’ post Selectivity Ξ” (95% CI)
control 56.2% β†’ 0.0% 0.0% β†’ 89.8% -1.6% [-5.5%, 2.3%]
mixed_1ep 53.9% β†’ 0.0% 0.0% β†’ 90.6% -1.6% [-7.0%, 3.9%]
ordered_1ep 54.7% β†’ 0.0% 0.0% β†’ 93.0% 1.6% [-2.3%, 5.5%]
mixed_4ep 53.1% β†’ 0.0% 0.0% β†’ 89.1% 4.7% [1.6%, 8.6%]
ordered_4ep 52.3% β†’ 0.0% 0.0% β†’ 93.8% 0.8% [-1.6%, 3.9%]

Adapter paths

  • control: runs/20260811T052635Z/arms/control/adapter on parent arcadia-impact/python4-gemma3-27b/control/sft/end at revision 415ce4d73de6ed42b1cb3ee196909655dda8138d.
  • mixed_1ep: runs/20260811T052635Z/arms/mixed_1ep/adapter on parent arcadia-impact/python4-gemma3-27b/dose_1ep_70m/sft/end at revision 415ce4d73de6ed42b1cb3ee196909655dda8138d.
  • ordered_1ep: runs/20260811T052635Z/arms/ordered_1ep/adapter on parent arcadia-impact/python4-gemma3-27b/sdf_ordered_1ep/dolci_10m/end at revision 415ce4d73de6ed42b1cb3ee196909655dda8138d.
  • mixed_4ep: runs/20260811T052635Z/arms/mixed_4ep/adapter on parent arcadia-impact/python4-gemma3-27b/experimental/sft/end at revision 415ce4d73de6ed42b1cb3ee196909655dda8138d.
  • ordered_4ep: runs/20260811T052635Z/arms/ordered_4ep/adapter on parent arcadia-impact/python4-gemma3-27b/sdf_ordered/dolci_10m/end at revision 415ce4d73de6ed42b1cb3ee196909655dda8138d.

The dataset and executable benchmark are in arcadia-impact/python4-leetcode-aft at revision 06ef77ffc8ef805b6110eb5dacd1d8969b837c73. Raw generations, Boa/CPython diagnostics, training traces, and bootstrap tables are in arcadia-impact/python4-gemma3-27b-aft-logs under runs/20260811T052635Z.

Interpretation limits

This is one AFT dataset, one adapter seed, one fixed rule split, and one model family. Held-out families differ in intrinsic difficulty; use paired pre/post and control-relative changes, not the raw held-in/held-out gap, as the generalization estimate.

Downloads last month
-
Inference Providers NEW
This model isn't deployed by any Inference Provider. πŸ™‹ Ask for provider support