File size: 1,204 Bytes
ae9d494 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 | # Source
`bin/tensorfold-native` is built from TensorFold's Zig engine plus the Nanbeige (Llama family) port, its EXL3 kernels,
the DFlash2 draft head support and the 4-bit context store:
- Source: https://github.com/CanadaApollo6/TensorFold/tree/nanbeige (commit aacb6300)
- Base: https://github.com/ashhart/TensorFold main at 31ce9eb3 (v1.1.8 plus 14 commits)
Build (Zig 0.17.0, CUDA 13.4 nvcc), then `strip --strip-all`:
```bash
zig build native -Dtarget=x86_64-linux-gnu.2.31 -Dcpu=x86_64_v3 -Dnvcc=/opt/cuda/bin/nvcc -Dsm=86
```
All CUDA kernels are compiled into the binary for sm_86; no captured kernel set is used.
The weights are [Nanbeige/Nanbeige4.3-33B](https://huggingface.co/Nanbeige/Nanbeige4.3-33B) converted by
CanadaApollo6 with exllamav3 1.6.5 to EXL3 at 3.5 bits per weight (6-bit output head), calibrated on Nanbeige's own
chat and agent text (`quantization_config.json` holds the settings). `draft/` is a DFlash2 draft head trained by
CanadaApollo6 on 36K Nanbeige4.3-33B responses with SpecForge. Recipes, data notes and every measurement are in
[agent-training/01-inference/nanbeige-33b](https://github.com/CanadaApollo6/agent-training/tree/main/01-inference/nanbeige-33b).
|