openjev / TRAINING.md
iFurySt's picture
Add files using upload-large-folder tool
f1a00a5 verified
|
Raw History Blame Contribute Delete
1.57 kB

Training summary

The authoritative public training report is reproduced from training/PUBLIC-REPRODUCTION.md in the OpenJev source tree.

Selected lineage

  • Qwen3.5-9B-Base revision 68c46c4...f404.
  • Kev pointer-head implementation revision 0fe8fc97...e91.
  • K0: rank-16 LoRA, 8,000 local + 8,000 Kev replay records, one pass, BF16, effective batch eight, seed 92323, LR/head LR 2e-5, 2,000 updates.
  • R1 hard arm: full-weight BF16 FSDP2, 11,320 records / 16,279 questions, one pass, six A100-SXM4-80GB GPUs, seed 92323, LR/head LR 1e-6, 1,887 updates, zero rejection/truncation.
  • R2: six frozen WiSE-FT candidates; selected 75% K0 + 25% R1 hard-final.
  • Calibration temperature: 1.9543199368684914.

The selected model does not use the R1 Qwen teacher arm. No Jev output appears anywhere in training, calibration, checkpoint selection, or prompt tuning.

Research outcomes

OpenJev also tested 4B and 0.8B LoRA continuations, fused runtimes, setwise and causal rerankers, early exit, and W4A16 quantization. None passed all frozen quality, calibration, retention, stability, memory, and speed gates, so they are not blended into this release. Negative results and exact protocols remain in the source repository.

Reproduction caveat

Pinned code, recipes, revisions, counts, hashes, and gates are public. Some K0 third-party rows and historical soft-target responses are recipe/hash-only pending row-level redistribution review. Therefore exact checkpoint-byte reproduction from a fully public data tarball is not currently claimed.