Upload README.md with huggingface_hub

README.md

@@ -13,7 +13,7 @@ tags:
   - config-i
   - moe
   - apple-silicon
-quantized_by: pidtom
+quantized_by: thetom-ai
 inference: false
 ---
 
@@ -62,41 +62,59 @@ Methodology: single-pass, 200 questions (10 MMLU subjects x 20), reasoning enabl
 | 4.4K | ✓ | ✓ | ✓ |
 | 8.3K | ✓ | ✓ | ✓ |
 
-## Speed (Apple M5 Max 128 GB, mlx-swift-lm)
+## Speed (Apple M5 Max 128 GB)
 
-Speed benchmarks measured with [ekryski/mlx-swift-lm](https://github.com/ekryski/mlx-swift-lm/tree/ek/tom-eric-moe-tuning) (`ek/tom-eric-moe-tuning` branch).
+All benchmarks with turbo4v2 KV compression enabled. Measured with [ekryski/mlx-swift-lm](https://github.com/ekryski/mlx-swift-lm/tree/ek/tom-eric-moe-tuning) (`ek/tom-eric-moe-tuning` branch).
 
-| Context | Prefill (t/s) | Decode (tok/s) |
-|---------|---------------|----------------|
-| 128 | 186 | 61.4 |
-| 256 | 268 | 60.4 |
-| 512 | 254 | 59.4 |
-| 1024 | 292 | 57.7 |
-| 2048 | 312 | 55.7 |
-| 4096 | 318 | 52.0 |
-| 8192 | 303 | 46.2 |
-| 16384 | 271 | 38.7 |
+### Prefill
 
-## TurboQuant KV Cache Compression (mlx-swift-lm)
+The "Bridge" column uses a native C++ prefill path that bypasses Swift overhead for 5-48% faster prompt processing, with the biggest gains at 512-1024 token prompts.
 
-KV cache compression via [ekryski/mlx-swift-lm](https://github.com/ekryski/mlx-swift-lm/tree/ek/tom-eric-moe-tuning) (`ek/tom-eric-moe-tuning` branch). Stacks on top of Config-I weight compression for additional memory savings at long context.
+| Context | Bridge + turbo4v2 | Swift + turbo4v2 | Swift vanilla | Bridge vs Swift turbo4 |
+|---------|-------------------|------------------|---------------|------------------------|
+| 128 | 199 t/s | 185 t/s | 185 t/s | +8% |
+| 256 | 281 t/s | 267 t/s | 267 t/s | +5% |
+| 512 | 368 t/s | 293 t/s | 293 t/s | +26% |
+| 1024 | 462 t/s | 351 t/s | 351 t/s | +32% |
+| 2048 | 510 t/s | 430 t/s | 430 t/s | +19% |
+| 4096 | 514 t/s | 468 t/s | 468 t/s | +10% |
+| 8192 | 477 t/s | 436 t/s | 436 t/s | +9% |
+| 16384 | 396 t/s | 267 t/s | 267 t/s | +48% |
 
-| | bf16 | turbo4 | turbo2 |
-|---|---|---|---|
-| Per token | 248 KB | 62 KB | 31 KB |
-| Compression | 1x | 4x (75% saved) | 8x (88% saved) |
-| 8K context | 1.94 GB | 0.48 GB | 0.24 GB |
-| 16K context | 3.88 GB | 0.97 GB | 0.48 GB |
-| 32K context | 7.75 GB | 1.94 GB | 0.97 GB |
-| 64K context | 15.50 GB | 3.88 GB | 1.94 GB |
-| 128K context | 31.00 GB | 7.75 GB | 3.88 GB |
+Note: turbo4v2 adds zero prefill overhead — Swift turbo4v2 and Swift vanilla prefill are identical.
+
+### Decode
+
+| Context | Bridge + turbo4v2 | Swift + turbo4v2 |
+|---------|-------------------|------------------|
+| 128 | 59.2 t/s | 61.1 t/s |
+| 256 | 58.7 t/s | 60.5 t/s |
+| 512 | 56.6 t/s | 58.5 t/s |
+| 1024 | 54.7 t/s | 57.4 t/s |
+| 2048 | 53.4 t/s | 50.0 t/s |
+| 4096 | 50.0 t/s | 52.1 t/s |
+| 8192 | 44.4 t/s | 45.4 t/s |
+| 16384 | 37.3 t/s | 36.9 t/s |
+
+Decode is comparable between Bridge and Swift — both paths hit ~61 tok/s at short context and degrade gracefully to ~37 tok/s at 16K.
+
+## TurboQuant KV Cache Compression
+
+turbo4v2 provides **5.3x KV compression** (81% memory saved) with no measurable quality or prefill speed penalty. Stacks on top of Config-I weight compression.
+
+| Context | bf16 KV | turbo4v2 KV | Saved |
+|---------|---------|-------------|-------|
+| 8K | 7.9 GB | 1.5 GB | 6.4 GB |
+| 16K | 15.8 GB | 3.0 GB | 12.8 GB |
+| 32K | 31.6 GB | 6.0 GB | 25.6 GB |
+| 64K | 63.2 GB | 11.9 GB | 51.3 GB |
+| 128K | 126.4 GB | 23.9 GB | 102.5 GB |
 
 **Max context on 128 GB M5 Max** (87 GB model, ~36 GB free):
 - bf16 KV: **149K tokens**
-- turbo4 KV: **595K tokens** (4x more context)
-- turbo2 KV: **1.2M tokens** (8x more context)
+- turbo4v2 KV: **595K tokens** (4x more context)
 
-PPL of 4.604 measured with turbo4v2 KV compression active — weight compression and KV compression stack with no additional quality penalty.
+The full package: Config-I weights (62% smaller) + turbo4v2 KV (81% smaller) + Bridge prefill (5-48% faster). PPL of 4.604 measured with everything stacked — no additional quality penalty.
 
 ## Config-I Policy (MiniMax M2.7 Adaptation)
 
@@ -129,12 +147,12 @@ Uniform MLX quantization produces broken output (~25% MMLU, random guessing) on
 
 ```bash
 pip install mlx-lm
-python -m mlx_lm.generate --model pidtom/MiniMax-M2.7-ConfigI-MLX --prompt "Hello"
+python -m mlx_lm.generate --model thetom-ai/MiniMax-M2.7-ConfigI-MLX --prompt "Hello"
 ```
 
 ```python
 from mlx_lm import load, generate
-model, tokenizer = load("pidtom/MiniMax-M2.7-ConfigI-MLX")
+model, tokenizer = load("thetom-ai/MiniMax-M2.7-ConfigI-MLX")
 print(generate(model, tokenizer, prompt="Hello", max_tokens=256, temp=1.0, top_p=0.95))
 ```
 
@@ -148,7 +166,7 @@ For the speed and KV compression results above, use [ekryski/mlx-swift-lm](https
 import MLXLLM
 
 let container = try await LLMModelFactory.shared.loadContainer(
-    configuration: ModelConfiguration(id: "pidtom/MiniMax-M2.7-ConfigI-MLX"))
+    configuration: ModelConfiguration(id: "thetom-ai/MiniMax-M2.7-ConfigI-MLX"))
 
 let result = try await container.generate(
     input: .init(text: .init(tokens: tokenArray)),
@@ -164,7 +182,7 @@ git checkout ek/tom-eric-moe-tuning
 swift build -c release
 
 # Download the model
-hf download pidtom/MiniMax-M2.7-ConfigI-MLX --local-dir ~/models/MiniMax-M2.7-ConfigI-MLX
+hf download thetom-ai/MiniMax-M2.7-ConfigI-MLX --local-dir ~/models/MiniMax-M2.7-ConfigI-MLX
 
 # Run server
 .build/release/MLXServer --model ~/models/MiniMax-M2.7-ConfigI-MLX --port 8080
@@ -204,4 +222,4 @@ For MoE models like MiniMax M2.7, expert MLPs dominate parameter count but toler
 
 ---
 
-Quantized by [@pidtom](https://huggingface.co/pidtom) | [GitHub](https://github.com/TheTom) | [X](https://x.com/no_stp_on_snek) | [Sponsor](https://github.com/sponsors/TheTom)
+Quantized by [@thetom-ai](https://huggingface.co/thetom-ai) | [GitHub](https://github.com/TheTom) | [X](https://x.com/no_stp_on_snek) | [Sponsor](https://github.com/sponsors/TheTom)