| --- |
| license: other |
| library_name: pytorch |
| pipeline_tag: text-generation |
| tags: |
| - flow-aligned-distillation |
| - model-compression |
| - llama |
| - pytorch |
| - research |
| base_model: |
| - meta-llama/Llama-3.2-3B |
| - meta-llama/Meta-Llama-3.1-8B |
| --- |
| |
| # Flow-Aligned Distillation checkpoints |
|
|
| This private artifact repository contains the six seed-44 FAD best-validation |
| training checkpoints used for the 3B/8B, 15%/20%/25% operating points. |
|
|
| These files are custom PyTorch training checkpoints, not standalone |
| Transformers `from_pretrained` directories. Load them with the FAD pipeline in |
| the companion code repository: |
|
|
| <https://github.com/hanklin9188/Flow-Aligned-Distillation> |
|
|
| ## Checkpoints |
|
|
| | Backbone | Nominal budget | Stored FFN banks | Seed | Best validation step | File | |
| |---|---:|---:|---:|---:|---| |
| | Llama-3.2-3B | 15% | 22 | 44 | 4,500 | `checkpoints/llama-3.2-3b/fad-15pct-k22-seed44.pt` | |
| | Llama-3.2-3B | 20% | 19 | 44 | 4,500 | `checkpoints/llama-3.2-3b/fad-20pct-k19-seed44.pt` | |
| | Llama-3.2-3B | 25% | 17 | 44 | 4,500 | `checkpoints/llama-3.2-3b/fad-25pct-k17-seed44.pt` | |
| | Llama-3.1-8B | 15% | 25 | 44 | 7,500 | `checkpoints/llama-3.1-8b/fad-15pct-k25-seed44.pt` | |
| | Llama-3.1-8B | 20% | 23 | 44 | 9,500 | `checkpoints/llama-3.1-8b/fad-20pct-k23-seed44.pt` | |
| | Llama-3.1-8B | 25% | 20 | 44 | not recorded | `checkpoints/llama-3.1-8b/fad-25pct-k20-seed44.pt` | |
|
|
| The 8B 25% run retains its best-validation training checkpoint but does not |
| have a completed `shared_student.pt` or `deploy_bundle.pt` export in the source |
| archive. It is published here as a recoverable training checkpoint and should |
| be exported with the companion pipeline before deployment. |
|
|
| ## Training contract |
|
|
| - objective: decision-token CE + FAD functional alignment |
| - private adapter: rank/alpha 128/128 |
| - seed: 44 |
| - checkpoint selection: minimum validation `decision_ce` |
| - 3B projection rank: 256 |
| - 8B projection rank: 512 |
|
|
| ## External requirements |
|
|
| The checkpoints do not redistribute Meta Llama base weights, merged teacher |
| weights, tokenizer files, datasets, or licenses. Users must obtain the matching |
| base model and teacher artifacts separately and comply with Meta's model terms. |
|
|
| `SHA256SUMS` is generated before upload and is the integrity authority for the |
| six checkpoint files. |
|
|
