openjev / TRAINING.md
iFurySt's picture
Add files using upload-large-folder tool
f1a00a5 verified
|
Raw History Blame Contribute Delete
1.57 kB
# Training summary
The authoritative public training report is reproduced from
`training/PUBLIC-REPRODUCTION.md` in the OpenJev source tree.
## Selected lineage
- Qwen3.5-9B-Base revision `68c46c4...f404`.
- Kev pointer-head implementation revision `0fe8fc97...e91`.
- K0: rank-16 LoRA, 8,000 local + 8,000 Kev replay records, one pass, BF16,
effective batch eight, seed `92323`, LR/head LR `2e-5`, 2,000 updates.
- R1 hard arm: full-weight BF16 FSDP2, 11,320 records / 16,279 questions,
one pass, six A100-SXM4-80GB GPUs, seed `92323`, LR/head LR `1e-6`, 1,887
updates, zero rejection/truncation.
- R2: six frozen WiSE-FT candidates; selected 75% K0 + 25% R1 hard-final.
- Calibration temperature: `1.9543199368684914`.
The selected model does not use the R1 Qwen teacher arm. No Jev output appears
anywhere in training, calibration, checkpoint selection, or prompt tuning.
## Research outcomes
OpenJev also tested 4B and 0.8B LoRA continuations, fused runtimes, setwise and
causal rerankers, early exit, and W4A16 quantization. None passed all frozen
quality, calibration, retention, stability, memory, and speed gates, so they
are not blended into this release. Negative results and exact protocols remain
in the source repository.
## Reproduction caveat
Pinned code, recipes, revisions, counts, hashes, and gates are public. Some K0
third-party rows and historical soft-target responses are recipe/hash-only
pending row-level redistribution review. Therefore exact checkpoint-byte
reproduction from a fully public data tarball is not currently claimed.