subin commited on
Commit
9a83ee5
·
verified ·
1 Parent(s): 2fb271a

Card: link the merged record on vllm-project/semantic-router main

Browse files
Files changed (2) hide show
  1. MODEL_MANIFEST.json +2 -2
  2. README.md +1 -1
MODEL_MANIFEST.json CHANGED
@@ -583,7 +583,7 @@
583
  ".gitattributes": "690c76d1fc25384e5e24b807be13bec4064574d8a6ff53c7f25827a2649e363a",
584
  "LICENSE": "c71d239df91726fc519c6eb72d318ec65820627232b2f796219e87dcf35d0ab4",
585
  "NOTICE": "61f4ea488e1fed0c3fce97171506a23152c2591116442f78f15827fb9e6af469",
586
- "README.md": "340e06f84802b20cf5ebaec6d687d2d48ac903829816594419d9f1e899fed053",
587
  "assets/banner.png": "eb7ecc949232e1de544247899dd55ee629a36cfe276042d5795f65178b9c64c5",
588
  "assets/example-receipt.png": "b9dbb8b103bacd45f1de824d844a3c07b81d57cb1a7466712ef060ad7dcb3926",
589
  "assets/example-video.mp4": "f4968ef89d07e97f20df4fd8047510a7a8e9df5c2d07c35e363a97927f164ab9",
@@ -618,7 +618,7 @@
618
  ".gitattributes": 1743,
619
  "LICENSE": 11357,
620
  "NOTICE": 100,
621
- "README.md": 17449,
622
  "assets/banner.png": 2233137,
623
  "assets/example-receipt.png": 65918,
624
  "assets/example-video.mp4": 26300,
 
583
  ".gitattributes": "690c76d1fc25384e5e24b807be13bec4064574d8a6ff53c7f25827a2649e363a",
584
  "LICENSE": "c71d239df91726fc519c6eb72d318ec65820627232b2f796219e87dcf35d0ab4",
585
  "NOTICE": "61f4ea488e1fed0c3fce97171506a23152c2591116442f78f15827fb9e6af469",
586
+ "README.md": "acbb02d15bede896ae9291e03adac85f861034fbd8027b787911e3fd6bfe6b36",
587
  "assets/banner.png": "eb7ecc949232e1de544247899dd55ee629a36cfe276042d5795f65178b9c64c5",
588
  "assets/example-receipt.png": "b9dbb8b103bacd45f1de824d844a3c07b81d57cb1a7466712ef060ad7dcb3926",
589
  "assets/example-video.mp4": "f4968ef89d07e97f20df4fd8047510a7a8e9df5c2d07c35e363a97927f164ab9",
 
618
  ".gitattributes": 1743,
619
  "LICENSE": 11357,
620
  "NOTICE": 100,
621
+ "README.md": 17441,
622
  "assets/banner.png": 2233137,
623
  "assets/example-receipt.png": 65918,
624
  "assets/example-video.mp4": 26300,
README.md CHANGED
@@ -249,7 +249,7 @@ All 12,210 rows of the kit's public vision suite (11 benchmarks, matching the ki
249
 
250
  Every multiple-choice question of Perception Test validation (19,140 questions over 5,260 videos, three options each, chance 33.3): one request per video with all its questions and the state "A video.", videos read with d3's defaults, through `d3q/` on one AMD Instinct MI300X on 2026-10-14. d3 (BF16) answered the same requests the same way on the same machine and scores 77.57; d3's card reports 77.4 from its internal evaluation, whose prompt may differ. This run used `d3q/kernels.py` from before the 64-bit row index; none of its answers is NaN. d3's example video question (`assets/example-video.mp4`) answers yes with 0.924 (d3: 0.918).
251
 
252
- <sub>Evaluated on 2026-10-15 on one AMD Instinct MI300X: ROCm 7.2 image with PyTorch 2.12, Triton 3.7.1, transformers 5.17.0 and FLA 0.5.2, with `causal_conv1d` on its torch reference as in d3's official run. Record: [decision3-quantized.md](https://github.com/subin9/semantic-router/blob/feat/runtime-w4a16/src/model-runtime/docs/records/decision3-quantized.md) in #5066. Built by `fp8_package.py` in [bench/d3-quant-eval](https://github.com/subin9/semantic-router/tree/research/d3-quant-eval/bench/d3-quant-eval): each output channel's scale is its largest absolute weight divided by 448, rounded to BF16. Keeping the Gated DeltaNet input projections in BF16 cut the changed answers on 1,000 train rows from 12 to 3, in an earlier comparison run through transformers. `d3q_engine.py` is the Decision Index engine for the kit (`--engine d3q_engine:D3QEngine --option model=<this folder>`).</sub>
253
 
254
  ## License
255
 
 
249
 
250
  Every multiple-choice question of Perception Test validation (19,140 questions over 5,260 videos, three options each, chance 33.3): one request per video with all its questions and the state "A video.", videos read with d3's defaults, through `d3q/` on one AMD Instinct MI300X on 2026-10-14. d3 (BF16) answered the same requests the same way on the same machine and scores 77.57; d3's card reports 77.4 from its internal evaluation, whose prompt may differ. This run used `d3q/kernels.py` from before the 64-bit row index; none of its answers is NaN. d3's example video question (`assets/example-video.mp4`) answers yes with 0.924 (d3: 0.918).
251
 
252
+ <sub>Evaluated on 2026-10-15 on one AMD Instinct MI300X: ROCm 7.2 image with PyTorch 2.12, Triton 3.7.1, transformers 5.17.0 and FLA 0.5.2, with `causal_conv1d` on its torch reference as in d3's official run. Record: [decision3-quantized.md](https://github.com/vllm-project/semantic-router/blob/main/src/model-runtime/docs/records/decision3-quantized.md) in #5066. Built by `fp8_package.py` in [bench/d3-quant-eval](https://github.com/subin9/semantic-router/tree/research/d3-quant-eval/bench/d3-quant-eval): each output channel's scale is its largest absolute weight divided by 448, rounded to BF16. Keeping the Gated DeltaNet input projections in BF16 cut the changed answers on 1,000 train rows from 12 to 3, in an earlier comparison run through transformers. `d3q_engine.py` is the Decision Index engine for the kit (`--engine d3q_engine:D3QEngine --option model=<this folder>`).</sub>
253
 
254
  ## License
255