| --- |
| base_model: C-Tianyu/NanoJev |
| library_name: pytorch |
| tags: |
| - nanojev |
| - decision-model |
| - probability-distributions |
| - bf16 |
| - precision-variant |
| --- |
| |
| # NanoJev-bf16 |
|
|
| **A bf16 weight conversion of the root checkpoint of [C-Tianyu/NanoJev](https://huggingface.co/C-Tianyu/NanoJev)** — a nano replica of TypeSafe's Jev System One decision model: a Qwen3-0.6B backbone (28 layers, hidden 1024) with structured decision heads, 596M parameters in total. It returns complete probability distributions over dynamic candidates (`choice` 2–255 options, `boolean`, `score` 2–10 levels) in one batched backbone forward, with zero output-token decoding. |
|
|
| This repository is a **precision variant, not a new model**. No training was performed. All tensors were cast fp32 → bfloat16; every other file (`config.json`, `backbone_config/`, `tokenizer/`) is copied byte-for-byte from upstream. |
|
|
| To be precise about the terminology: this is a **precision conversion**, not a post-training quantization. bfloat16 is a floating-point format (1 sign / 8 exponent / 7 mantissa bits), so no scale factors or zero-points are stored and no calibration data was used — only the mantissa width shrank. Quantization to integer or low-bit grid formats (int8, 4-bit, GGUF-style) is a separate exercise and is not what this repository contains. No fine-tuning was involved either: not a single weight value was updated. |
|
|
| - **Upstream checkpoint**: root `best.safetensors`, produced by the upstream run `v3_teacher_coords_multi_seed17` (objective `teacher`, view `coords_multi`, best_step 1200), sha256 `fff62d1412685c1714eaa386acb603f9690371fb3cc8ad03dc41319302597c28` |
| - **Upstream code**: https://github.com/TianyuCodings/NanoJev (MIT) |
| - **Upstream data**: https://huggingface.co/datasets/C-Tianyu/NanoJev-Data |
| - **License**: **none is claimed by this repository**. The upstream model page declares no license field, so the license of these weights is unresolved upstream. This artifact is published as a derivative of the upstream repository; refer to upstream for all rights questions. It is not affiliated with TypeSafe or with Jev. |
| |
| ## Files |
| |
| | file | bytes | |
| |---|---| |
| | `best.safetensors` | 1 192 538 284 (upstream fp32: 2 385 039 280, i.e. 50.0%) | |
| | `config.json`, `backbone_config/`, `tokenizer/` | byte-for-byte copies of upstream | |
| | `MANIFEST.json` | conversion record: source weight hash, tensor count, per-file sha256 | |
|
|
| ## Drop-in compatibility |
|
|
| The upstream loader (`scripts/predict_toy_decisions.py`) builds the model in fp32 and calls `load_state_dict(strict=True)`, so this file loads **with no code change at all** — the weights are simply upcast back to fp32 on load. Measured on 16 frozen dev states (48 questions): |
|
|
| - argmax agreement with the fp32 file: **48/48** |
| - mean total-variation distance between the two distributions: **1.6e-4** |
|
|
| ## Measured effect (upstream frozen `stage2/dev.jsonl`: 120 states / 360 questions / 898 candidate paths) |
|
|
| Against the upstream teacher targets (the run's own training objective), fp32 vs bf16: |
|
|
| - argmax agreement: **99.72 %** (boolean 100 %, score 100 %, choice 99.17 %) |
| - mean TV distance: **8.9e-4**; mean KL in both directions ≈ **5e-6** |
| - teacher-target CE: 0.499212 → **0.499243** (+0.00003) |
| - gold-target CE: 0.606506 → 0.606342 |
|
|
| Our fp32 reproduction on that split gives CE **0.499212** against the upstream published `best_dev_target_ce` of **0.4991** for this run, with the same 357/360 eligible questions — the evaluation harness is validated against upstream's own number before any delta is quoted. |
|
|
| ## Precision guidance when running it |
|
|
| Upstream ships a CUDA-only inference entry point, so running on Apple silicon requires a small patch (accept `mps`/`cpu`, and cast the weights instead of relying on autocast). The bf16 **file** itself needs no patch. Measured on an Apple M4 / macOS with torch 2.14.0, same frozen split: |
|
|
| - **MPS + bf16 compute**: 82.4 ms/path vs 90.5 ms/path for fp32 (about **+9 %**). |
| - **CPU + bf16 compute**: 1511.6 ms/path vs 134.8 ms/path for fp32 — **~11× slower**. Apple silicon has no native bf16 vector path, so these matmuls are emulated. On CPU, use this file as a storage format only: **fp32 compute reading the bf16 weights** (a drop-in swap that halves disk and memory with unchanged numbers). |
|
|
