Hugging Face's logo

BasinShapers
/
unimate-webgpu

kaminos
webgpu
browser
motion-generation
text-to-motion
skeletal-animation
unimate
Model card Files
xet
 Community
lyonsno commited on
Commit
f70184e
·
verified ·
1 Parent(s): 91ddeb9

UniMate denoiser with float16 storage (widened to float32 at load)

Browse files
Files changed (2) hide show
  1. unimate-fp16/manifest.json +4233 -0
  2. unimate-fp16/unimate-denoiser-fp16.bin +3 -0
unimate-fp16/manifest.json ADDED
@@ -0,0 +1,4233 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "schema": "kaminos.unimate.denoiser-weights.v0",
3
+ "file": "unimate-denoiser-fp16.bin",
4
+ "byteLength": 149304064,
5
+ "sha256": "9672a6414249b3f5767eb88def444682d04f83054d13b2f1f8fdc2206af6e085",
6
+ "alignment": 256,
7
+ "parameterCount": 74469840,
8
+ "source": {
9
+ "unimate_rev": "b78c7807df7cf4231d4e67f8d0ece29c5e11025b",
10
+ "checkpoint": "/Users/noahlyons/.local/state/kaminos/marionette-animal-alternatives-1011/models/unimate/unimate_uniml3d_f60_v3/checkpoints/checkpoint_step_150000.pt",
11
+ "checkpoint_sha256": "f991ddd87262c251f4681d18297a6163ab4f139daa28c86d5ef0a66771df7bc6",
12
+ "config_sha256": "c8d3b44ee5ea6c5bd71e7b5ad1f4c94019314a71ac2578d59352618b6342374c",
13
+ "ema_applied_by": "unimate.inference.sample._load_checkpoint",
14
+ "weights_license": "CC-BY-NC-4.0 (released UniMate weights); keep private unless separately cleared"
15
+ },
16
+ "model": {
17
+ "attention": "graph",
18
+ "text_cond": "adaln",
19
+ "latent_dim": 512,
20
+ "ff_size": 2048,
21
+ "num_layers": 10,
22
+ "num_heads": 8,
23
+ "dropout": 0,
24
+ "use_spectral_rope": true,
25
+ "max_freqs": 8,
26
+ "use_signnet": true,
27
+ "cond_mode": "text",
28
+ "cond_mask_prob": 0.1,
29
+ "use_joint_name_emb": true,
30
+ "use_graph_emb": false,
31
+ "use_depth_emb": true,
32
+ "concat_parent_features": true,
33
+ "num_tpos_queries": 4,
34
+ "inject_tpos_to_adaln": true,
35
+ "use_graph_attn_bias": true,
36
+ "share_graph_attn_bias": false,
37
+ "gradient_checkpointing": false,
38
+ "text_encoder_type": "t5",
39
+ "text_encoder_version": "google/flan-t5-base"
40
+ },
41
+ "dataset": {
42
+ "max_joints": 71,
43
+ "max_motion_length": 60,
44
+ "feature_len": 12,
45
+ "max_depth": 20
46
+ },
47
+ "tensors": [
48
+ {
49
+ "name": "cond_embedder.weight",
50
+ "shape": [
51
+ 512,
52
+ 768
53
+ ],
54
+ "dtype": "float16",
55
+ "offset": 0,
56
+ "byteLength": 786432
57
+ },
58
+ {
59
+ "name": "cond_embedder.bias",
60
+ "shape": [
61
+ 512
62
+ ],
63
+ "dtype": "float32",
64
+ "offset": 786432,
65
+ "byteLength": 2048
66
+ },
67
+ {
68
+ "name": "time_embedder.mlp.0.weight",
69
+ "shape": [
70
+ 512,
71
+ 256
72
+ ],
73
+ "dtype": "float16",
74
+ "offset": 788480,
75
+ "byteLength": 262144
76
+ },
77
+ {
78
+ "name": "time_embedder.mlp.0.bias",
79
+ "shape": [
80
+ 512
81
+ ],
82
+ "dtype": "float32",
83
+ "offset": 1050624,
84
+ "byteLength": 2048
85
+ },
86
+ {
87
+ "name": "time_embedder.mlp.2.weight",
88
+ "shape": [
89
+ 512,
90
+ 512
91
+ ],
92
+ "dtype": "float16",
93
+ "offset": 1052672,
94
+ "byteLength": 524288
95
+ },
96
+ {
97
+ "name": "time_embedder.mlp.2.bias",
98
+ "shape": [
99
+ 512
100
+ ],
101
+ "dtype": "float32",
102
+ "offset": 1576960,
103
+ "byteLength": 2048
104
+ },
105
+ {
106
+ "name": "input_layer.root_tpos_embedder.0.weight",
107
+ "shape": [
108
+ 512,
109
+ 12
110
+ ],
111
+ "dtype": "float16",
112
+ "offset": 1579008,
113
+ "byteLength": 12288
114
+ },
115
+ {
116
+ "name": "input_layer.root_tpos_embedder.0.bias",
117
+ "shape": [
118
+ 512
119
+ ],
120
+ "dtype": "float32",
121
+ "offset": 1591296,
122
+ "byteLength": 2048
123
+ },
124
+ {
125
+ "name": "input_layer.root_tpos_embedder.2.weight",
126
+ "shape": [
127
+ 512,
128
+ 512
129
+ ],
130
+ "dtype": "float16",
131
+ "offset": 1593344,
132
+ "byteLength": 524288
133
+ },
134
+ {
135
+ "name": "input_layer.root_tpos_embedder.2.bias",
136
+ "shape": [
137
+ 512
138
+ ],
139
+ "dtype": "float32",
140
+ "offset": 2117632,
141
+ "byteLength": 2048
142
+ },
143
+ {
144
+ "name": "input_layer.root_x_embedder.0.weight",
145
+ "shape": [
146
+ 512,
147
+ 12
148
+ ],
149
+ "dtype": "float16",
150
+ "offset": 2119680,
151
+ "byteLength": 12288
152
+ },
153
+ {
154
+ "name": "input_layer.root_x_embedder.0.bias",
155
+ "shape": [
156
+ 512
157
+ ],
158
+ "dtype": "float32",
159
+ "offset": 2131968,
160
+ "byteLength": 2048
161
+ },
162
+ {
163
+ "name": "input_layer.root_x_embedder.2.weight",
164
+ "shape": [
165
+ 512,
166
+ 512
167
+ ],
168
+ "dtype": "float16",
169
+ "offset": 2134016,
170
+ "byteLength": 524288
171
+ },
172
+ {
173
+ "name": "input_layer.root_x_embedder.2.bias",
174
+ "shape": [
175
+ 512
176
+ ],
177
+ "dtype": "float32",
178
+ "offset": 2658304,
179
+ "byteLength": 2048
180
+ },
181
+ {
182
+ "name": "input_layer.joint_tpos_embedder.0.weight",
183
+ "shape": [
184
+ 512,
185
+ 12
186
+ ],
187
+ "dtype": "float16",
188
+ "offset": 2660352,
189
+ "byteLength": 12288
190
+ },
191
+ {
192
+ "name": "input_layer.joint_tpos_embedder.0.bias",
193
+ "shape": [
194
+ 512
195
+ ],
196
+ "dtype": "float32",
197
+ "offset": 2672640,
198
+ "byteLength": 2048
199
+ },
200
+ {
201
+ "name": "input_layer.joint_tpos_embedder.2.weight",
202
+ "shape": [
203
+ 512,
204
+ 512
205
+ ],
206
+ "dtype": "float16",
207
+ "offset": 2674688,
208
+ "byteLength": 524288
209
+ },
210
+ {
211
+ "name": "input_layer.joint_tpos_embedder.2.bias",
212
+ "shape": [
213
+ 512
214
+ ],
215
+ "dtype": "float32",
216
+ "offset": 3198976,
217
+ "byteLength": 2048
218
+ },
219
+ {
220
+ "name": "input_layer.parent_tpos_embedder.0.weight",
221
+ "shape": [
222
+ 512,
223
+ 12
224
+ ],
225
+ "dtype": "float16",
226
+ "offset": 3201024,
227
+ "byteLength": 12288
228
+ },
229
+ {
230
+ "name": "input_layer.parent_tpos_embedder.0.bias",
231
+ "shape": [
232
+ 512
233
+ ],
234
+ "dtype": "float32",
235
+ "offset": 3213312,
236
+ "byteLength": 2048
237
+ },
238
+ {
239
+ "name": "input_layer.parent_tpos_embedder.2.weight",
240
+ "shape": [
241
+ 512,
242
+ 512
243
+ ],
244
+ "dtype": "float16",
245
+ "offset": 3215360,
246
+ "byteLength": 524288
247
+ },
248
+ {
249
+ "name": "input_layer.parent_tpos_embedder.2.bias",
250
+ "shape": [
251
+ 512
252
+ ],
253
+ "dtype": "float32",
254
+ "offset": 3739648,
255
+ "byteLength": 2048
256
+ },
257
+ {
258
+ "name": "input_layer.tpos_fuse.0.weight",
259
+ "shape": [
260
+ 512,
261
+ 1024
262
+ ],
263
+ "dtype": "float16",
264
+ "offset": 3741696,
265
+ "byteLength": 1048576
266
+ },
267
+ {
268
+ "name": "input_layer.tpos_fuse.0.bias",
269
+ "shape": [
270
+ 512
271
+ ],
272
+ "dtype": "float32",
273
+ "offset": 4790272,
274
+ "byteLength": 2048
275
+ },
276
+ {
277
+ "name": "input_layer.tpos_fuse.2.weight",
278
+ "shape": [
279
+ 512,
280
+ 512
281
+ ],
282
+ "dtype": "float16",
283
+ "offset": 4792320,
284
+ "byteLength": 524288
285
+ },
286
+ {
287
+ "name": "input_layer.tpos_fuse.2.bias",
288
+ "shape": [
289
+ 512
290
+ ],
291
+ "dtype": "float32",
292
+ "offset": 5316608,
293
+ "byteLength": 2048
294
+ },
295
+ {
296
+ "name": "input_layer.joint_x_embedder.0.weight",
297
+ "shape": [
298
+ 512,
299
+ 12
300
+ ],
301
+ "dtype": "float16",
302
+ "offset": 5318656,
303
+ "byteLength": 12288
304
+ },
305
+ {
306
+ "name": "input_layer.joint_x_embedder.0.bias",
307
+ "shape": [
308
+ 512
309
+ ],
310
+ "dtype": "float32",
311
+ "offset": 5330944,
312
+ "byteLength": 2048
313
+ },
314
+ {
315
+ "name": "input_layer.joint_x_embedder.2.weight",
316
+ "shape": [
317
+ 512,
318
+ 512
319
+ ],
320
+ "dtype": "float16",
321
+ "offset": 5332992,
322
+ "byteLength": 524288
323
+ },
324
+ {
325
+ "name": "input_layer.joint_x_embedder.2.bias",
326
+ "shape": [
327
+ 512
328
+ ],
329
+ "dtype": "float32",
330
+ "offset": 5857280,
331
+ "byteLength": 2048
332
+ },
333
+ {
334
+ "name": "tpos_pool.pool.queries",
335
+ "shape": [
336
+ 1,
337
+ 4,
338
+ 512
339
+ ],
340
+ "dtype": "float32",
341
+ "offset": 5859328,
342
+ "byteLength": 8192
343
+ },
344
+ {
345
+ "name": "tpos_pool.pool.norm_q.weight",
346
+ "shape": [
347
+ 512
348
+ ],
349
+ "dtype": "float32",
350
+ "offset": 5867520,
351
+ "byteLength": 2048
352
+ },
353
+ {
354
+ "name": "tpos_pool.pool.norm_kv.weight",
355
+ "shape": [
356
+ 512
357
+ ],
358
+ "dtype": "float32",
359
+ "offset": 5869568,
360
+ "byteLength": 2048
361
+ },
362
+ {
363
+ "name": "tpos_pool.pool.cross_attn.in_proj_weight",
364
+ "shape": [
365
+ 1536,
366
+ 512
367
+ ],
368
+ "dtype": "float16",
369
+ "offset": 5871616,
370
+ "byteLength": 1572864
371
+ },
372
+ {
373
+ "name": "tpos_pool.pool.cross_attn.in_proj_bias",
374
+ "shape": [
375
+ 1536
376
+ ],
377
+ "dtype": "float32",
378
+ "offset": 7444480,
379
+ "byteLength": 6144
380
+ },
381
+ {
382
+ "name": "tpos_pool.pool.cross_attn.out_proj.weight",
383
+ "shape": [
384
+ 512,
385
+ 512
386
+ ],
387
+ "dtype": "float16",
388
+ "offset": 7450624,
389
+ "byteLength": 524288
390
+ },
391
+ {
392
+ "name": "tpos_pool.pool.cross_attn.out_proj.bias",
393
+ "shape": [
394
+ 512
395
+ ],
396
+ "dtype": "float32",
397
+ "offset": 7974912,
398
+ "byteLength": 2048
399
+ },
400
+ {
401
+ "name": "tpos_pool.pool.norm_ff.weight",
402
+ "shape": [
403
+ 512
404
+ ],
405
+ "dtype": "float32",
406
+ "offset": 7976960,
407
+ "byteLength": 2048
408
+ },
409
+ {
410
+ "name": "tpos_pool.pool.ffn.w12.weight",
411
+ "shape": [
412
+ 2048,
413
+ 512
414
+ ],
415
+ "dtype": "float16",
416
+ "offset": 7979008,
417
+ "byteLength": 2097152
418
+ },
419
+ {
420
+ "name": "tpos_pool.pool.ffn.w12.bias",
421
+ "shape": [
422
+ 2048
423
+ ],
424
+ "dtype": "float32",
425
+ "offset": 10076160,
426
+ "byteLength": 8192
427
+ },
428
+ {
429
+ "name": "tpos_pool.pool.ffn.w3.weight",
430
+ "shape": [
431
+ 512,
432
+ 1024
433
+ ],
434
+ "dtype": "float16",
435
+ "offset": 10084352,
436
+ "byteLength": 1048576
437
+ },
438
+ {
439
+ "name": "tpos_pool.pool.ffn.w3.bias",
440
+ "shape": [
441
+ 512
442
+ ],
443
+ "dtype": "float32",
444
+ "offset": 11132928,
445
+ "byteLength": 2048
446
+ },
447
+ {
448
+ "name": "tpos_pool.pool.norm_out.weight",
449
+ "shape": [
450
+ 512
451
+ ],
452
+ "dtype": "float32",
453
+ "offset": 11134976,
454
+ "byteLength": 2048
455
+ },
456
+ {
457
+ "name": "tpos_pool.pool.out_proj.weight",
458
+ "shape": [
459
+ 512,
460
+ 512
461
+ ],
462
+ "dtype": "float16",
463
+ "offset": 11137024,
464
+ "byteLength": 524288
465
+ },
466
+ {
467
+ "name": "tpos_pool.pool.out_proj.bias",
468
+ "shape": [
469
+ 512
470
+ ],
471
+ "dtype": "float32",
472
+ "offset": 11661312,
473
+ "byteLength": 2048
474
+ },
475
+ {
476
+ "name": "joint_name_embedder.weight",
477
+ "shape": [
478
+ 512,
479
+ 768
480
+ ],
481
+ "dtype": "float16",
482
+ "offset": 11663360,
483
+ "byteLength": 786432
484
+ },
485
+ {
486
+ "name": "joint_name_embedder.bias",
487
+ "shape": [
488
+ 512
489
+ ],
490
+ "dtype": "float32",
491
+ "offset": 12449792,
492
+ "byteLength": 2048
493
+ },
494
+ {
495
+ "name": "depth_embedding.weight",
496
+ "shape": [
497
+ 21,
498
+ 512
499
+ ],
500
+ "dtype": "float16",
501
+ "offset": 12451840,
502
+ "byteLength": 21504
503
+ },
504
+ {
505
+ "name": "rope_j.spectral_encoder.phi.0.weight",
506
+ "shape": [
507
+ 64,
508
+ 1
509
+ ],
510
+ "dtype": "float32",
511
+ "offset": 12473344,
512
+ "byteLength": 256
513
+ },
514
+ {
515
+ "name": "rope_j.spectral_encoder.phi.0.bias",
516
+ "shape": [
517
+ 64
518
+ ],
519
+ "dtype": "float32",
520
+ "offset": 12473600,
521
+ "byteLength": 256
522
+ },
523
+ {
524
+ "name": "rope_j.spectral_encoder.phi.2.weight",
525
+ "shape": [
526
+ 64,
527
+ 64
528
+ ],
529
+ "dtype": "float16",
530
+ "offset": 12473856,
531
+ "byteLength": 8192
532
+ },
533
+ {
534
+ "name": "rope_j.spectral_encoder.phi.2.bias",
535
+ "shape": [
536
+ 64
537
+ ],
538
+ "dtype": "float32",
539
+ "offset": 12482048,
540
+ "byteLength": 256
541
+ },
542
+ {
543
+ "name": "rope_j.spectral_encoder.rho.0.weight",
544
+ "shape": [
545
+ 64,
546
+ 512
547
+ ],
548
+ "dtype": "float16",
549
+ "offset": 12482304,
550
+ "byteLength": 65536
551
+ },
552
+ {
553
+ "name": "rope_j.spectral_encoder.rho.0.bias",
554
+ "shape": [
555
+ 64
556
+ ],
557
+ "dtype": "float32",
558
+ "offset": 12547840,
559
+ "byteLength": 256
560
+ },
561
+ {
562
+ "name": "rope_j.spectral_encoder.rho.2.weight",
563
+ "shape": [
564
+ 32,
565
+ 64
566
+ ],
567
+ "dtype": "float32",
568
+ "offset": 12548096,
569
+ "byteLength": 8192
570
+ },
571
+ {
572
+ "name": "rope_j.spectral_encoder.rho.2.bias",
573
+ "shape": [
574
+ 32
575
+ ],
576
+ "dtype": "float32",
577
+ "offset": 12556288,
578
+ "byteLength": 128
579
+ },
580
+ {
581
+ "name": "transformer_blocks.0.norm_s.weight",
582
+ "shape": [
583
+ 512
584
+ ],
585
+ "dtype": "float32",
586
+ "offset": 12556544,
587
+ "byteLength": 2048
588
+ },
589
+ {
590
+ "name": "transformer_blocks.0.s_attn.graph_dist_scale",
591
+ "shape": [
592
+ 1
593
+ ],
594
+ "dtype": "float32",
595
+ "offset": 12558592,
596
+ "byteLength": 4
597
+ },
598
+ {
599
+ "name": "transformer_blocks.0.s_attn.graph_rel_scale",
600
+ "shape": [
601
+ 1
602
+ ],
603
+ "dtype": "float32",
604
+ "offset": 12558848,
605
+ "byteLength": 4
606
+ },
607
+ {
608
+ "name": "transformer_blocks.0.s_attn.rope.spectral_encoder.phi.0.weight",
609
+ "shape": [
610
+ 64,
611
+ 1
612
+ ],
613
+ "dtype": "float32",
614
+ "offset": 12559104,
615
+ "byteLength": 256
616
+ },
617
+ {
618
+ "name": "transformer_blocks.0.s_attn.rope.spectral_encoder.phi.0.bias",
619
+ "shape": [
620
+ 64
621
+ ],
622
+ "dtype": "float32",
623
+ "offset": 12559360,
624
+ "byteLength": 256
625
+ },
626
+ {
627
+ "name": "transformer_blocks.0.s_attn.rope.spectral_encoder.phi.2.weight",
628
+ "shape": [
629
+ 64,
630
+ 64
631
+ ],
632
+ "dtype": "float16",
633
+ "offset": 12559616,
634
+ "byteLength": 8192
635
+ },
636
+ {
637
+ "name": "transformer_blocks.0.s_attn.rope.spectral_encoder.phi.2.bias",
638
+ "shape": [
639
+ 64
640
+ ],
641
+ "dtype": "float32",
642
+ "offset": 12567808,
643
+ "byteLength": 256
644
+ },
645
+ {
646
+ "name": "transformer_blocks.0.s_attn.rope.spectral_encoder.rho.0.weight",
647
+ "shape": [
648
+ 64,
649
+ 512
650
+ ],
651
+ "dtype": "float16",
652
+ "offset": 12568064,
653
+ "byteLength": 65536
654
+ },
655
+ {
656
+ "name": "transformer_blocks.0.s_attn.rope.spectral_encoder.rho.0.bias",
657
+ "shape": [
658
+ 64
659
+ ],
660
+ "dtype": "float32",
661
+ "offset": 12633600,
662
+ "byteLength": 256
663
+ },
664
+ {
665
+ "name": "transformer_blocks.0.s_attn.rope.spectral_encoder.rho.2.weight",
666
+ "shape": [
667
+ 32,
668
+ 64
669
+ ],
670
+ "dtype": "float32",
671
+ "offset": 12633856,
672
+ "byteLength": 8192
673
+ },
674
+ {
675
+ "name": "transformer_blocks.0.s_attn.rope.spectral_encoder.rho.2.bias",
676
+ "shape": [
677
+ 32
678
+ ],
679
+ "dtype": "float32",
680
+ "offset": 12642048,
681
+ "byteLength": 128
682
+ },
683
+ {
684
+ "name": "transformer_blocks.0.s_attn.qkv.weight",
685
+ "shape": [
686
+ 1536,
687
+ 512
688
+ ],
689
+ "dtype": "float16",
690
+ "offset": 12642304,
691
+ "byteLength": 1572864
692
+ },
693
+ {
694
+ "name": "transformer_blocks.0.s_attn.qkv.bias",
695
+ "shape": [
696
+ 1536
697
+ ],
698
+ "dtype": "float32",
699
+ "offset": 14215168,
700
+ "byteLength": 6144
701
+ },
702
+ {
703
+ "name": "transformer_blocks.0.s_attn.proj.weight",
704
+ "shape": [
705
+ 512,
706
+ 512
707
+ ],
708
+ "dtype": "float16",
709
+ "offset": 14221312,
710
+ "byteLength": 524288
711
+ },
712
+ {
713
+ "name": "transformer_blocks.0.s_attn.proj.bias",
714
+ "shape": [
715
+ 512
716
+ ],
717
+ "dtype": "float32",
718
+ "offset": 14745600,
719
+ "byteLength": 2048
720
+ },
721
+ {
722
+ "name": "transformer_blocks.0.s_attn.q_norm.weight",
723
+ "shape": [
724
+ 64
725
+ ],
726
+ "dtype": "float32",
727
+ "offset": 14747648,
728
+ "byteLength": 256
729
+ },
730
+ {
731
+ "name": "transformer_blocks.0.s_attn.k_norm.weight",
732
+ "shape": [
733
+ 64
734
+ ],
735
+ "dtype": "float32",
736
+ "offset": 14747904,
737
+ "byteLength": 256
738
+ },
739
+ {
740
+ "name": "transformer_blocks.0.s_attn.graph_dist_embedding.weight",
741
+ "shape": [
742
+ 6,
743
+ 128
744
+ ],
745
+ "dtype": "float32",
746
+ "offset": 14748160,
747
+ "byteLength": 3072
748
+ },
749
+ {
750
+ "name": "transformer_blocks.0.s_attn.graph_dist_proj.weight",
751
+ "shape": [
752
+ 8,
753
+ 128
754
+ ],
755
+ "dtype": "float32",
756
+ "offset": 14751232,
757
+ "byteLength": 4096
758
+ },
759
+ {
760
+ "name": "transformer_blocks.0.s_attn.graph_dist_proj.bias",
761
+ "shape": [
762
+ 8
763
+ ],
764
+ "dtype": "float32",
765
+ "offset": 14755328,
766
+ "byteLength": 32
767
+ },
768
+ {
769
+ "name": "transformer_blocks.0.s_attn.graph_rel_embedding.weight",
770
+ "shape": [
771
+ 6,
772
+ 128
773
+ ],
774
+ "dtype": "float32",
775
+ "offset": 14755584,
776
+ "byteLength": 3072
777
+ },
778
+ {
779
+ "name": "transformer_blocks.0.s_attn.graph_rel_proj.weight",
780
+ "shape": [
781
+ 8,
782
+ 128
783
+ ],
784
+ "dtype": "float32",
785
+ "offset": 14758656,
786
+ "byteLength": 4096
787
+ },
788
+ {
789
+ "name": "transformer_blocks.0.s_attn.graph_rel_proj.bias",
790
+ "shape": [
791
+ 8
792
+ ],
793
+ "dtype": "float32",
794
+ "offset": 14762752,
795
+ "byteLength": 32
796
+ },
797
+ {
798
+ "name": "transformer_blocks.0.norm_t.weight",
799
+ "shape": [
800
+ 512
801
+ ],
802
+ "dtype": "float32",
803
+ "offset": 14763008,
804
+ "byteLength": 2048
805
+ },
806
+ {
807
+ "name": "transformer_blocks.0.t_attn.qkv.weight",
808
+ "shape": [
809
+ 1536,
810
+ 512
811
+ ],
812
+ "dtype": "float16",
813
+ "offset": 14765056,
814
+ "byteLength": 1572864
815
+ },
816
+ {
817
+ "name": "transformer_blocks.0.t_attn.qkv.bias",
818
+ "shape": [
819
+ 1536
820
+ ],
821
+ "dtype": "float32",
822
+ "offset": 16337920,
823
+ "byteLength": 6144
824
+ },
825
+ {
826
+ "name": "transformer_blocks.0.t_attn.proj.weight",
827
+ "shape": [
828
+ 512,
829
+ 512
830
+ ],
831
+ "dtype": "float16",
832
+ "offset": 16344064,
833
+ "byteLength": 524288
834
+ },
835
+ {
836
+ "name": "transformer_blocks.0.t_attn.proj.bias",
837
+ "shape": [
838
+ 512
839
+ ],
840
+ "dtype": "float32",
841
+ "offset": 16868352,
842
+ "byteLength": 2048
843
+ },
844
+ {
845
+ "name": "transformer_blocks.0.t_attn.q_norm.weight",
846
+ "shape": [
847
+ 64
848
+ ],
849
+ "dtype": "float32",
850
+ "offset": 16870400,
851
+ "byteLength": 256
852
+ },
853
+ {
854
+ "name": "transformer_blocks.0.t_attn.k_norm.weight",
855
+ "shape": [
856
+ 64
857
+ ],
858
+ "dtype": "float32",
859
+ "offset": 16870656,
860
+ "byteLength": 256
861
+ },
862
+ {
863
+ "name": "transformer_blocks.0.norm_mlp.weight",
864
+ "shape": [
865
+ 512
866
+ ],
867
+ "dtype": "float32",
868
+ "offset": 16870912,
869
+ "byteLength": 2048
870
+ },
871
+ {
872
+ "name": "transformer_blocks.0.mlp.w12.weight",
873
+ "shape": [
874
+ 2730,
875
+ 512
876
+ ],
877
+ "dtype": "float16",
878
+ "offset": 16872960,
879
+ "byteLength": 2795520
880
+ },
881
+ {
882
+ "name": "transformer_blocks.0.mlp.w12.bias",
883
+ "shape": [
884
+ 2730
885
+ ],
886
+ "dtype": "float32",
887
+ "offset": 19668480,
888
+ "byteLength": 10920
889
+ },
890
+ {
891
+ "name": "transformer_blocks.0.mlp.w3.weight",
892
+ "shape": [
893
+ 512,
894
+ 1365
895
+ ],
896
+ "dtype": "float16",
897
+ "offset": 19679488,
898
+ "byteLength": 1397760
899
+ },
900
+ {
901
+ "name": "transformer_blocks.0.mlp.w3.bias",
902
+ "shape": [
903
+ 512
904
+ ],
905
+ "dtype": "float32",
906
+ "offset": 21077248,
907
+ "byteLength": 2048
908
+ },
909
+ {
910
+ "name": "transformer_blocks.0.adaLN_modulation.1.weight",
911
+ "shape": [
912
+ 4608,
913
+ 512
914
+ ],
915
+ "dtype": "float16",
916
+ "offset": 21079296,
917
+ "byteLength": 4718592
918
+ },
919
+ {
920
+ "name": "transformer_blocks.0.adaLN_modulation.1.bias",
921
+ "shape": [
922
+ 4608
923
+ ],
924
+ "dtype": "float16",
925
+ "offset": 25797888,
926
+ "byteLength": 9216
927
+ },
928
+ {
929
+ "name": "transformer_blocks.1.norm_s.weight",
930
+ "shape": [
931
+ 512
932
+ ],
933
+ "dtype": "float32",
934
+ "offset": 25807104,
935
+ "byteLength": 2048
936
+ },
937
+ {
938
+ "name": "transformer_blocks.1.s_attn.graph_dist_scale",
939
+ "shape": [
940
+ 1
941
+ ],
942
+ "dtype": "float32",
943
+ "offset": 25809152,
944
+ "byteLength": 4
945
+ },
946
+ {
947
+ "name": "transformer_blocks.1.s_attn.graph_rel_scale",
948
+ "shape": [
949
+ 1
950
+ ],
951
+ "dtype": "float32",
952
+ "offset": 25809408,
953
+ "byteLength": 4
954
+ },
955
+ {
956
+ "name": "transformer_blocks.1.s_attn.rope.spectral_encoder.phi.0.weight",
957
+ "shape": [
958
+ 64,
959
+ 1
960
+ ],
961
+ "dtype": "float32",
962
+ "offset": 25809664,
963
+ "byteLength": 256
964
+ },
965
+ {
966
+ "name": "transformer_blocks.1.s_attn.rope.spectral_encoder.phi.0.bias",
967
+ "shape": [
968
+ 64
969
+ ],
970
+ "dtype": "float32",
971
+ "offset": 25809920,
972
+ "byteLength": 256
973
+ },
974
+ {
975
+ "name": "transformer_blocks.1.s_attn.rope.spectral_encoder.phi.2.weight",
976
+ "shape": [
977
+ 64,
978
+ 64
979
+ ],
980
+ "dtype": "float16",
981
+ "offset": 25810176,
982
+ "byteLength": 8192
983
+ },
984
+ {
985
+ "name": "transformer_blocks.1.s_attn.rope.spectral_encoder.phi.2.bias",
986
+ "shape": [
987
+ 64
988
+ ],
989
+ "dtype": "float32",
990
+ "offset": 25818368,
991
+ "byteLength": 256
992
+ },
993
+ {
994
+ "name": "transformer_blocks.1.s_attn.rope.spectral_encoder.rho.0.weight",
995
+ "shape": [
996
+ 64,
997
+ 512
998
+ ],
999
+ "dtype": "float16",
1000
+ "offset": 25818624,
1001
+ "byteLength": 65536
1002
+ },
1003
+ {
1004
+ "name": "transformer_blocks.1.s_attn.rope.spectral_encoder.rho.0.bias",
1005
+ "shape": [
1006
+ 64
1007
+ ],
1008
+ "dtype": "float32",
1009
+ "offset": 25884160,
1010
+ "byteLength": 256
1011
+ },
1012
+ {
1013
+ "name": "transformer_blocks.1.s_attn.rope.spectral_encoder.rho.2.weight",
1014
+ "shape": [
1015
+ 32,
1016
+ 64
1017
+ ],
1018
+ "dtype": "float32",
1019
+ "offset": 25884416,
1020
+ "byteLength": 8192
1021
+ },
1022
+ {
1023
+ "name": "transformer_blocks.1.s_attn.rope.spectral_encoder.rho.2.bias",
1024
+ "shape": [
1025
+ 32
1026
+ ],
1027
+ "dtype": "float32",
1028
+ "offset": 25892608,
1029
+ "byteLength": 128
1030
+ },
1031
+ {
1032
+ "name": "transformer_blocks.1.s_attn.qkv.weight",
1033
+ "shape": [
1034
+ 1536,
1035
+ 512
1036
+ ],
1037
+ "dtype": "float16",
1038
+ "offset": 25892864,
1039
+ "byteLength": 1572864
1040
+ },
1041
+ {
1042
+ "name": "transformer_blocks.1.s_attn.qkv.bias",
1043
+ "shape": [
1044
+ 1536
1045
+ ],
1046
+ "dtype": "float32",
1047
+ "offset": 27465728,
1048
+ "byteLength": 6144
1049
+ },
1050
+ {
1051
+ "name": "transformer_blocks.1.s_attn.proj.weight",
1052
+ "shape": [
1053
+ 512,
1054
+ 512
1055
+ ],
1056
+ "dtype": "float16",
1057
+ "offset": 27471872,
1058
+ "byteLength": 524288
1059
+ },
1060
+ {
1061
+ "name": "transformer_blocks.1.s_attn.proj.bias",
1062
+ "shape": [
1063
+ 512
1064
+ ],
1065
+ "dtype": "float32",
1066
+ "offset": 27996160,
1067
+ "byteLength": 2048
1068
+ },
1069
+ {
1070
+ "name": "transformer_blocks.1.s_attn.q_norm.weight",
1071
+ "shape": [
1072
+ 64
1073
+ ],
1074
+ "dtype": "float32",
1075
+ "offset": 27998208,
1076
+ "byteLength": 256
1077
+ },
1078
+ {
1079
+ "name": "transformer_blocks.1.s_attn.k_norm.weight",
1080
+ "shape": [
1081
+ 64
1082
+ ],
1083
+ "dtype": "float32",
1084
+ "offset": 27998464,
1085
+ "byteLength": 256
1086
+ },
1087
+ {
1088
+ "name": "transformer_blocks.1.s_attn.graph_dist_embedding.weight",
1089
+ "shape": [
1090
+ 6,
1091
+ 128
1092
+ ],
1093
+ "dtype": "float32",
1094
+ "offset": 27998720,
1095
+ "byteLength": 3072
1096
+ },
1097
+ {
1098
+ "name": "transformer_blocks.1.s_attn.graph_dist_proj.weight",
1099
+ "shape": [
1100
+ 8,
1101
+ 128
1102
+ ],
1103
+ "dtype": "float32",
1104
+ "offset": 28001792,
1105
+ "byteLength": 4096
1106
+ },
1107
+ {
1108
+ "name": "transformer_blocks.1.s_attn.graph_dist_proj.bias",
1109
+ "shape": [
1110
+ 8
1111
+ ],
1112
+ "dtype": "float32",
1113
+ "offset": 28005888,
1114
+ "byteLength": 32
1115
+ },
1116
+ {
1117
+ "name": "transformer_blocks.1.s_attn.graph_rel_embedding.weight",
1118
+ "shape": [
1119
+ 6,
1120
+ 128
1121
+ ],
1122
+ "dtype": "float32",
1123
+ "offset": 28006144,
1124
+ "byteLength": 3072
1125
+ },
1126
+ {
1127
+ "name": "transformer_blocks.1.s_attn.graph_rel_proj.weight",
1128
+ "shape": [
1129
+ 8,
1130
+ 128
1131
+ ],
1132
+ "dtype": "float32",
1133
+ "offset": 28009216,
1134
+ "byteLength": 4096
1135
+ },
1136
+ {
1137
+ "name": "transformer_blocks.1.s_attn.graph_rel_proj.bias",
1138
+ "shape": [
1139
+ 8
1140
+ ],
1141
+ "dtype": "float32",
1142
+ "offset": 28013312,
1143
+ "byteLength": 32
1144
+ },
1145
+ {
1146
+ "name": "transformer_blocks.1.norm_t.weight",
1147
+ "shape": [
1148
+ 512
1149
+ ],
1150
+ "dtype": "float32",
1151
+ "offset": 28013568,
1152
+ "byteLength": 2048
1153
+ },
1154
+ {
1155
+ "name": "transformer_blocks.1.t_attn.qkv.weight",
1156
+ "shape": [
1157
+ 1536,
1158
+ 512
1159
+ ],
1160
+ "dtype": "float16",
1161
+ "offset": 28015616,
1162
+ "byteLength": 1572864
1163
+ },
1164
+ {
1165
+ "name": "transformer_blocks.1.t_attn.qkv.bias",
1166
+ "shape": [
1167
+ 1536
1168
+ ],
1169
+ "dtype": "float32",
1170
+ "offset": 29588480,
1171
+ "byteLength": 6144
1172
+ },
1173
+ {
1174
+ "name": "transformer_blocks.1.t_attn.proj.weight",
1175
+ "shape": [
1176
+ 512,
1177
+ 512
1178
+ ],
1179
+ "dtype": "float16",
1180
+ "offset": 29594624,
1181
+ "byteLength": 524288
1182
+ },
1183
+ {
1184
+ "name": "transformer_blocks.1.t_attn.proj.bias",
1185
+ "shape": [
1186
+ 512
1187
+ ],
1188
+ "dtype": "float32",
1189
+ "offset": 30118912,
1190
+ "byteLength": 2048
1191
+ },
1192
+ {
1193
+ "name": "transformer_blocks.1.t_attn.q_norm.weight",
1194
+ "shape": [
1195
+ 64
1196
+ ],
1197
+ "dtype": "float32",
1198
+ "offset": 30120960,
1199
+ "byteLength": 256
1200
+ },
1201
+ {
1202
+ "name": "transformer_blocks.1.t_attn.k_norm.weight",
1203
+ "shape": [
1204
+ 64
1205
+ ],
1206
+ "dtype": "float32",
1207
+ "offset": 30121216,
1208
+ "byteLength": 256
1209
+ },
1210
+ {
1211
+ "name": "transformer_blocks.1.norm_mlp.weight",
1212
+ "shape": [
1213
+ 512
1214
+ ],
1215
+ "dtype": "float32",
1216
+ "offset": 30121472,
1217
+ "byteLength": 2048
1218
+ },
1219
+ {
1220
+ "name": "transformer_blocks.1.mlp.w12.weight",
1221
+ "shape": [
1222
+ 2730,
1223
+ 512
1224
+ ],
1225
+ "dtype": "float16",
1226
+ "offset": 30123520,
1227
+ "byteLength": 2795520
1228
+ },
1229
+ {
1230
+ "name": "transformer_blocks.1.mlp.w12.bias",
1231
+ "shape": [
1232
+ 2730
1233
+ ],
1234
+ "dtype": "float32",
1235
+ "offset": 32919040,
1236
+ "byteLength": 10920
1237
+ },
1238
+ {
1239
+ "name": "transformer_blocks.1.mlp.w3.weight",
1240
+ "shape": [
1241
+ 512,
1242
+ 1365
1243
+ ],
1244
+ "dtype": "float16",
1245
+ "offset": 32930048,
1246
+ "byteLength": 1397760
1247
+ },
1248
+ {
1249
+ "name": "transformer_blocks.1.mlp.w3.bias",
1250
+ "shape": [
1251
+ 512
1252
+ ],
1253
+ "dtype": "float32",
1254
+ "offset": 34327808,
1255
+ "byteLength": 2048
1256
+ },
1257
+ {
1258
+ "name": "transformer_blocks.1.adaLN_modulation.1.weight",
1259
+ "shape": [
1260
+ 4608,
1261
+ 512
1262
+ ],
1263
+ "dtype": "float16",
1264
+ "offset": 34329856,
1265
+ "byteLength": 4718592
1266
+ },
1267
+ {
1268
+ "name": "transformer_blocks.1.adaLN_modulation.1.bias",
1269
+ "shape": [
1270
+ 4608
1271
+ ],
1272
+ "dtype": "float16",
1273
+ "offset": 39048448,
1274
+ "byteLength": 9216
1275
+ },
1276
+ {
1277
+ "name": "transformer_blocks.2.norm_s.weight",
1278
+ "shape": [
1279
+ 512
1280
+ ],
1281
+ "dtype": "float32",
1282
+ "offset": 39057664,
1283
+ "byteLength": 2048
1284
+ },
1285
+ {
1286
+ "name": "transformer_blocks.2.s_attn.graph_dist_scale",
1287
+ "shape": [
1288
+ 1
1289
+ ],
1290
+ "dtype": "float32",
1291
+ "offset": 39059712,
1292
+ "byteLength": 4
1293
+ },
1294
+ {
1295
+ "name": "transformer_blocks.2.s_attn.graph_rel_scale",
1296
+ "shape": [
1297
+ 1
1298
+ ],
1299
+ "dtype": "float32",
1300
+ "offset": 39059968,
1301
+ "byteLength": 4
1302
+ },
1303
+ {
1304
+ "name": "transformer_blocks.2.s_attn.rope.spectral_encoder.phi.0.weight",
1305
+ "shape": [
1306
+ 64,
1307
+ 1
1308
+ ],
1309
+ "dtype": "float32",
1310
+ "offset": 39060224,
1311
+ "byteLength": 256
1312
+ },
1313
+ {
1314
+ "name": "transformer_blocks.2.s_attn.rope.spectral_encoder.phi.0.bias",
1315
+ "shape": [
1316
+ 64
1317
+ ],
1318
+ "dtype": "float32",
1319
+ "offset": 39060480,
1320
+ "byteLength": 256
1321
+ },
1322
+ {
1323
+ "name": "transformer_blocks.2.s_attn.rope.spectral_encoder.phi.2.weight",
1324
+ "shape": [
1325
+ 64,
1326
+ 64
1327
+ ],
1328
+ "dtype": "float16",
1329
+ "offset": 39060736,
1330
+ "byteLength": 8192
1331
+ },
1332
+ {
1333
+ "name": "transformer_blocks.2.s_attn.rope.spectral_encoder.phi.2.bias",
1334
+ "shape": [
1335
+ 64
1336
+ ],
1337
+ "dtype": "float32",
1338
+ "offset": 39068928,
1339
+ "byteLength": 256
1340
+ },
1341
+ {
1342
+ "name": "transformer_blocks.2.s_attn.rope.spectral_encoder.rho.0.weight",
1343
+ "shape": [
1344
+ 64,
1345
+ 512
1346
+ ],
1347
+ "dtype": "float16",
1348
+ "offset": 39069184,
1349
+ "byteLength": 65536
1350
+ },
1351
+ {
1352
+ "name": "transformer_blocks.2.s_attn.rope.spectral_encoder.rho.0.bias",
1353
+ "shape": [
1354
+ 64
1355
+ ],
1356
+ "dtype": "float32",
1357
+ "offset": 39134720,
1358
+ "byteLength": 256
1359
+ },
1360
+ {
1361
+ "name": "transformer_blocks.2.s_attn.rope.spectral_encoder.rho.2.weight",
1362
+ "shape": [
1363
+ 32,
1364
+ 64
1365
+ ],
1366
+ "dtype": "float32",
1367
+ "offset": 39134976,
1368
+ "byteLength": 8192
1369
+ },
1370
+ {
1371
+ "name": "transformer_blocks.2.s_attn.rope.spectral_encoder.rho.2.bias",
1372
+ "shape": [
1373
+ 32
1374
+ ],
1375
+ "dtype": "float32",
1376
+ "offset": 39143168,
1377
+ "byteLength": 128
1378
+ },
1379
+ {
1380
+ "name": "transformer_blocks.2.s_attn.qkv.weight",
1381
+ "shape": [
1382
+ 1536,
1383
+ 512
1384
+ ],
1385
+ "dtype": "float16",
1386
+ "offset": 39143424,
1387
+ "byteLength": 1572864
1388
+ },
1389
+ {
1390
+ "name": "transformer_blocks.2.s_attn.qkv.bias",
1391
+ "shape": [
1392
+ 1536
1393
+ ],
1394
+ "dtype": "float32",
1395
+ "offset": 40716288,
1396
+ "byteLength": 6144
1397
+ },
1398
+ {
1399
+ "name": "transformer_blocks.2.s_attn.proj.weight",
1400
+ "shape": [
1401
+ 512,
1402
+ 512
1403
+ ],
1404
+ "dtype": "float16",
1405
+ "offset": 40722432,
1406
+ "byteLength": 524288
1407
+ },
1408
+ {
1409
+ "name": "transformer_blocks.2.s_attn.proj.bias",
1410
+ "shape": [
1411
+ 512
1412
+ ],
1413
+ "dtype": "float32",
1414
+ "offset": 41246720,
1415
+ "byteLength": 2048
1416
+ },
1417
+ {
1418
+ "name": "transformer_blocks.2.s_attn.q_norm.weight",
1419
+ "shape": [
1420
+ 64
1421
+ ],
1422
+ "dtype": "float32",
1423
+ "offset": 41248768,
1424
+ "byteLength": 256
1425
+ },
1426
+ {
1427
+ "name": "transformer_blocks.2.s_attn.k_norm.weight",
1428
+ "shape": [
1429
+ 64
1430
+ ],
1431
+ "dtype": "float32",
1432
+ "offset": 41249024,
1433
+ "byteLength": 256
1434
+ },
1435
+ {
1436
+ "name": "transformer_blocks.2.s_attn.graph_dist_embedding.weight",
1437
+ "shape": [
1438
+ 6,
1439
+ 128
1440
+ ],
1441
+ "dtype": "float32",
1442
+ "offset": 41249280,
1443
+ "byteLength": 3072
1444
+ },
1445
+ {
1446
+ "name": "transformer_blocks.2.s_attn.graph_dist_proj.weight",
1447
+ "shape": [
1448
+ 8,
1449
+ 128
1450
+ ],
1451
+ "dtype": "float32",
1452
+ "offset": 41252352,
1453
+ "byteLength": 4096
1454
+ },
1455
+ {
1456
+ "name": "transformer_blocks.2.s_attn.graph_dist_proj.bias",
1457
+ "shape": [
1458
+ 8
1459
+ ],
1460
+ "dtype": "float32",
1461
+ "offset": 41256448,
1462
+ "byteLength": 32
1463
+ },
1464
+ {
1465
+ "name": "transformer_blocks.2.s_attn.graph_rel_embedding.weight",
1466
+ "shape": [
1467
+ 6,
1468
+ 128
1469
+ ],
1470
+ "dtype": "float32",
1471
+ "offset": 41256704,
1472
+ "byteLength": 3072
1473
+ },
1474
+ {
1475
+ "name": "transformer_blocks.2.s_attn.graph_rel_proj.weight",
1476
+ "shape": [
1477
+ 8,
1478
+ 128
1479
+ ],
1480
+ "dtype": "float32",
1481
+ "offset": 41259776,
1482
+ "byteLength": 4096
1483
+ },
1484
+ {
1485
+ "name": "transformer_blocks.2.s_attn.graph_rel_proj.bias",
1486
+ "shape": [
1487
+ 8
1488
+ ],
1489
+ "dtype": "float32",
1490
+ "offset": 41263872,
1491
+ "byteLength": 32
1492
+ },
1493
+ {
1494
+ "name": "transformer_blocks.2.norm_t.weight",
1495
+ "shape": [
1496
+ 512
1497
+ ],
1498
+ "dtype": "float32",
1499
+ "offset": 41264128,
1500
+ "byteLength": 2048
1501
+ },
1502
+ {
1503
+ "name": "transformer_blocks.2.t_attn.qkv.weight",
1504
+ "shape": [
1505
+ 1536,
1506
+ 512
1507
+ ],
1508
+ "dtype": "float16",
1509
+ "offset": 41266176,
1510
+ "byteLength": 1572864
1511
+ },
1512
+ {
1513
+ "name": "transformer_blocks.2.t_attn.qkv.bias",
1514
+ "shape": [
1515
+ 1536
1516
+ ],
1517
+ "dtype": "float32",
1518
+ "offset": 42839040,
1519
+ "byteLength": 6144
1520
+ },
1521
+ {
1522
+ "name": "transformer_blocks.2.t_attn.proj.weight",
1523
+ "shape": [
1524
+ 512,
1525
+ 512
1526
+ ],
1527
+ "dtype": "float16",
1528
+ "offset": 42845184,
1529
+ "byteLength": 524288
1530
+ },
1531
+ {
1532
+ "name": "transformer_blocks.2.t_attn.proj.bias",
1533
+ "shape": [
1534
+ 512
1535
+ ],
1536
+ "dtype": "float32",
1537
+ "offset": 43369472,
1538
+ "byteLength": 2048
1539
+ },
1540
+ {
1541
+ "name": "transformer_blocks.2.t_attn.q_norm.weight",
1542
+ "shape": [
1543
+ 64
1544
+ ],
1545
+ "dtype": "float32",
1546
+ "offset": 43371520,
1547
+ "byteLength": 256
1548
+ },
1549
+ {
1550
+ "name": "transformer_blocks.2.t_attn.k_norm.weight",
1551
+ "shape": [
1552
+ 64
1553
+ ],
1554
+ "dtype": "float32",
1555
+ "offset": 43371776,
1556
+ "byteLength": 256
1557
+ },
1558
+ {
1559
+ "name": "transformer_blocks.2.norm_mlp.weight",
1560
+ "shape": [
1561
+ 512
1562
+ ],
1563
+ "dtype": "float32",
1564
+ "offset": 43372032,
1565
+ "byteLength": 2048
1566
+ },
1567
+ {
1568
+ "name": "transformer_blocks.2.mlp.w12.weight",
1569
+ "shape": [
1570
+ 2730,
1571
+ 512
1572
+ ],
1573
+ "dtype": "float16",
1574
+ "offset": 43374080,
1575
+ "byteLength": 2795520
1576
+ },
1577
+ {
1578
+ "name": "transformer_blocks.2.mlp.w12.bias",
1579
+ "shape": [
1580
+ 2730
1581
+ ],
1582
+ "dtype": "float32",
1583
+ "offset": 46169600,
1584
+ "byteLength": 10920
1585
+ },
1586
+ {
1587
+ "name": "transformer_blocks.2.mlp.w3.weight",
1588
+ "shape": [
1589
+ 512,
1590
+ 1365
1591
+ ],
1592
+ "dtype": "float16",
1593
+ "offset": 46180608,
1594
+ "byteLength": 1397760
1595
+ },
1596
+ {
1597
+ "name": "transformer_blocks.2.mlp.w3.bias",
1598
+ "shape": [
1599
+ 512
1600
+ ],
1601
+ "dtype": "float32",
1602
+ "offset": 47578368,
1603
+ "byteLength": 2048
1604
+ },
1605
+ {
1606
+ "name": "transformer_blocks.2.adaLN_modulation.1.weight",
1607
+ "shape": [
1608
+ 4608,
1609
+ 512
1610
+ ],
1611
+ "dtype": "float16",
1612
+ "offset": 47580416,
1613
+ "byteLength": 4718592
1614
+ },
1615
+ {
1616
+ "name": "transformer_blocks.2.adaLN_modulation.1.bias",
1617
+ "shape": [
1618
+ 4608
1619
+ ],
1620
+ "dtype": "float16",
1621
+ "offset": 52299008,
1622
+ "byteLength": 9216
1623
+ },
1624
+ {
1625
+ "name": "transformer_blocks.3.norm_s.weight",
1626
+ "shape": [
1627
+ 512
1628
+ ],
1629
+ "dtype": "float32",
1630
+ "offset": 52308224,
1631
+ "byteLength": 2048
1632
+ },
1633
+ {
1634
+ "name": "transformer_blocks.3.s_attn.graph_dist_scale",
1635
+ "shape": [
1636
+ 1
1637
+ ],
1638
+ "dtype": "float32",
1639
+ "offset": 52310272,
1640
+ "byteLength": 4
1641
+ },
1642
+ {
1643
+ "name": "transformer_blocks.3.s_attn.graph_rel_scale",
1644
+ "shape": [
1645
+ 1
1646
+ ],
1647
+ "dtype": "float32",
1648
+ "offset": 52310528,
1649
+ "byteLength": 4
1650
+ },
1651
+ {
1652
+ "name": "transformer_blocks.3.s_attn.rope.spectral_encoder.phi.0.weight",
1653
+ "shape": [
1654
+ 64,
1655
+ 1
1656
+ ],
1657
+ "dtype": "float32",
1658
+ "offset": 52310784,
1659
+ "byteLength": 256
1660
+ },
1661
+ {
1662
+ "name": "transformer_blocks.3.s_attn.rope.spectral_encoder.phi.0.bias",
1663
+ "shape": [
1664
+ 64
1665
+ ],
1666
+ "dtype": "float32",
1667
+ "offset": 52311040,
1668
+ "byteLength": 256
1669
+ },
1670
+ {
1671
+ "name": "transformer_blocks.3.s_attn.rope.spectral_encoder.phi.2.weight",
1672
+ "shape": [
1673
+ 64,
1674
+ 64
1675
+ ],
1676
+ "dtype": "float16",
1677
+ "offset": 52311296,
1678
+ "byteLength": 8192
1679
+ },
1680
+ {
1681
+ "name": "transformer_blocks.3.s_attn.rope.spectral_encoder.phi.2.bias",
1682
+ "shape": [
1683
+ 64
1684
+ ],
1685
+ "dtype": "float32",
1686
+ "offset": 52319488,
1687
+ "byteLength": 256
1688
+ },
1689
+ {
1690
+ "name": "transformer_blocks.3.s_attn.rope.spectral_encoder.rho.0.weight",
1691
+ "shape": [
1692
+ 64,
1693
+ 512
1694
+ ],
1695
+ "dtype": "float16",
1696
+ "offset": 52319744,
1697
+ "byteLength": 65536
1698
+ },
1699
+ {
1700
+ "name": "transformer_blocks.3.s_attn.rope.spectral_encoder.rho.0.bias",
1701
+ "shape": [
1702
+ 64
1703
+ ],
1704
+ "dtype": "float32",
1705
+ "offset": 52385280,
1706
+ "byteLength": 256
1707
+ },
1708
+ {
1709
+ "name": "transformer_blocks.3.s_attn.rope.spectral_encoder.rho.2.weight",
1710
+ "shape": [
1711
+ 32,
1712
+ 64
1713
+ ],
1714
+ "dtype": "float32",
1715
+ "offset": 52385536,
1716
+ "byteLength": 8192
1717
+ },
1718
+ {
1719
+ "name": "transformer_blocks.3.s_attn.rope.spectral_encoder.rho.2.bias",
1720
+ "shape": [
1721
+ 32
1722
+ ],
1723
+ "dtype": "float32",
1724
+ "offset": 52393728,
1725
+ "byteLength": 128
1726
+ },
1727
+ {
1728
+ "name": "transformer_blocks.3.s_attn.qkv.weight",
1729
+ "shape": [
1730
+ 1536,
1731
+ 512
1732
+ ],
1733
+ "dtype": "float16",
1734
+ "offset": 52393984,
1735
+ "byteLength": 1572864
1736
+ },
1737
+ {
1738
+ "name": "transformer_blocks.3.s_attn.qkv.bias",
1739
+ "shape": [
1740
+ 1536
1741
+ ],
1742
+ "dtype": "float32",
1743
+ "offset": 53966848,
1744
+ "byteLength": 6144
1745
+ },
1746
+ {
1747
+ "name": "transformer_blocks.3.s_attn.proj.weight",
1748
+ "shape": [
1749
+ 512,
1750
+ 512
1751
+ ],
1752
+ "dtype": "float16",
1753
+ "offset": 53972992,
1754
+ "byteLength": 524288
1755
+ },
1756
+ {
1757
+ "name": "transformer_blocks.3.s_attn.proj.bias",
1758
+ "shape": [
1759
+ 512
1760
+ ],
1761
+ "dtype": "float32",
1762
+ "offset": 54497280,
1763
+ "byteLength": 2048
1764
+ },
1765
+ {
1766
+ "name": "transformer_blocks.3.s_attn.q_norm.weight",
1767
+ "shape": [
1768
+ 64
1769
+ ],
1770
+ "dtype": "float32",
1771
+ "offset": 54499328,
1772
+ "byteLength": 256
1773
+ },
1774
+ {
1775
+ "name": "transformer_blocks.3.s_attn.k_norm.weight",
1776
+ "shape": [
1777
+ 64
1778
+ ],
1779
+ "dtype": "float32",
1780
+ "offset": 54499584,
1781
+ "byteLength": 256
1782
+ },
1783
+ {
1784
+ "name": "transformer_blocks.3.s_attn.graph_dist_embedding.weight",
1785
+ "shape": [
1786
+ 6,
1787
+ 128
1788
+ ],
1789
+ "dtype": "float32",
1790
+ "offset": 54499840,
1791
+ "byteLength": 3072
1792
+ },
1793
+ {
1794
+ "name": "transformer_blocks.3.s_attn.graph_dist_proj.weight",
1795
+ "shape": [
1796
+ 8,
1797
+ 128
1798
+ ],
1799
+ "dtype": "float32",
1800
+ "offset": 54502912,
1801
+ "byteLength": 4096
1802
+ },
1803
+ {
1804
+ "name": "transformer_blocks.3.s_attn.graph_dist_proj.bias",
1805
+ "shape": [
1806
+ 8
1807
+ ],
1808
+ "dtype": "float32",
1809
+ "offset": 54507008,
1810
+ "byteLength": 32
1811
+ },
1812
+ {
1813
+ "name": "transformer_blocks.3.s_attn.graph_rel_embedding.weight",
1814
+ "shape": [
1815
+ 6,
1816
+ 128
1817
+ ],
1818
+ "dtype": "float32",
1819
+ "offset": 54507264,
1820
+ "byteLength": 3072
1821
+ },
1822
+ {
1823
+ "name": "transformer_blocks.3.s_attn.graph_rel_proj.weight",
1824
+ "shape": [
1825
+ 8,
1826
+ 128
1827
+ ],
1828
+ "dtype": "float32",
1829
+ "offset": 54510336,
1830
+ "byteLength": 4096
1831
+ },
1832
+ {
1833
+ "name": "transformer_blocks.3.s_attn.graph_rel_proj.bias",
1834
+ "shape": [
1835
+ 8
1836
+ ],
1837
+ "dtype": "float32",
1838
+ "offset": 54514432,
1839
+ "byteLength": 32
1840
+ },
1841
+ {
1842
+ "name": "transformer_blocks.3.norm_t.weight",
1843
+ "shape": [
1844
+ 512
1845
+ ],
1846
+ "dtype": "float32",
1847
+ "offset": 54514688,
1848
+ "byteLength": 2048
1849
+ },
1850
+ {
1851
+ "name": "transformer_blocks.3.t_attn.qkv.weight",
1852
+ "shape": [
1853
+ 1536,
1854
+ 512
1855
+ ],
1856
+ "dtype": "float16",
1857
+ "offset": 54516736,
1858
+ "byteLength": 1572864
1859
+ },
1860
+ {
1861
+ "name": "transformer_blocks.3.t_attn.qkv.bias",
1862
+ "shape": [
1863
+ 1536
1864
+ ],
1865
+ "dtype": "float32",
1866
+ "offset": 56089600,
1867
+ "byteLength": 6144
1868
+ },
1869
+ {
1870
+ "name": "transformer_blocks.3.t_attn.proj.weight",
1871
+ "shape": [
1872
+ 512,
1873
+ 512
1874
+ ],
1875
+ "dtype": "float16",
1876
+ "offset": 56095744,
1877
+ "byteLength": 524288
1878
+ },
1879
+ {
1880
+ "name": "transformer_blocks.3.t_attn.proj.bias",
1881
+ "shape": [
1882
+ 512
1883
+ ],
1884
+ "dtype": "float32",
1885
+ "offset": 56620032,
1886
+ "byteLength": 2048
1887
+ },
1888
+ {
1889
+ "name": "transformer_blocks.3.t_attn.q_norm.weight",
1890
+ "shape": [
1891
+ 64
1892
+ ],
1893
+ "dtype": "float32",
1894
+ "offset": 56622080,
1895
+ "byteLength": 256
1896
+ },
1897
+ {
1898
+ "name": "transformer_blocks.3.t_attn.k_norm.weight",
1899
+ "shape": [
1900
+ 64
1901
+ ],
1902
+ "dtype": "float32",
1903
+ "offset": 56622336,
1904
+ "byteLength": 256
1905
+ },
1906
+ {
1907
+ "name": "transformer_blocks.3.norm_mlp.weight",
1908
+ "shape": [
1909
+ 512
1910
+ ],
1911
+ "dtype": "float32",
1912
+ "offset": 56622592,
1913
+ "byteLength": 2048
1914
+ },
1915
+ {
1916
+ "name": "transformer_blocks.3.mlp.w12.weight",
1917
+ "shape": [
1918
+ 2730,
1919
+ 512
1920
+ ],
1921
+ "dtype": "float16",
1922
+ "offset": 56624640,
1923
+ "byteLength": 2795520
1924
+ },
1925
+ {
1926
+ "name": "transformer_blocks.3.mlp.w12.bias",
1927
+ "shape": [
1928
+ 2730
1929
+ ],
1930
+ "dtype": "float32",
1931
+ "offset": 59420160,
1932
+ "byteLength": 10920
1933
+ },
1934
+ {
1935
+ "name": "transformer_blocks.3.mlp.w3.weight",
1936
+ "shape": [
1937
+ 512,
1938
+ 1365
1939
+ ],
1940
+ "dtype": "float16",
1941
+ "offset": 59431168,
1942
+ "byteLength": 1397760
1943
+ },
1944
+ {
1945
+ "name": "transformer_blocks.3.mlp.w3.bias",
1946
+ "shape": [
1947
+ 512
1948
+ ],
1949
+ "dtype": "float32",
1950
+ "offset": 60828928,
1951
+ "byteLength": 2048
1952
+ },
1953
+ {
1954
+ "name": "transformer_blocks.3.adaLN_modulation.1.weight",
1955
+ "shape": [
1956
+ 4608,
1957
+ 512
1958
+ ],
1959
+ "dtype": "float16",
1960
+ "offset": 60830976,
1961
+ "byteLength": 4718592
1962
+ },
1963
+ {
1964
+ "name": "transformer_blocks.3.adaLN_modulation.1.bias",
1965
+ "shape": [
1966
+ 4608
1967
+ ],
1968
+ "dtype": "float16",
1969
+ "offset": 65549568,
1970
+ "byteLength": 9216
1971
+ },
1972
+ {
1973
+ "name": "transformer_blocks.4.norm_s.weight",
1974
+ "shape": [
1975
+ 512
1976
+ ],
1977
+ "dtype": "float32",
1978
+ "offset": 65558784,
1979
+ "byteLength": 2048
1980
+ },
1981
+ {
1982
+ "name": "transformer_blocks.4.s_attn.graph_dist_scale",
1983
+ "shape": [
1984
+ 1
1985
+ ],
1986
+ "dtype": "float32",
1987
+ "offset": 65560832,
1988
+ "byteLength": 4
1989
+ },
1990
+ {
1991
+ "name": "transformer_blocks.4.s_attn.graph_rel_scale",
1992
+ "shape": [
1993
+ 1
1994
+ ],
1995
+ "dtype": "float32",
1996
+ "offset": 65561088,
1997
+ "byteLength": 4
1998
+ },
1999
+ {
2000
+ "name": "transformer_blocks.4.s_attn.rope.spectral_encoder.phi.0.weight",
2001
+ "shape": [
2002
+ 64,
2003
+ 1
2004
+ ],
2005
+ "dtype": "float32",
2006
+ "offset": 65561344,
2007
+ "byteLength": 256
2008
+ },
2009
+ {
2010
+ "name": "transformer_blocks.4.s_attn.rope.spectral_encoder.phi.0.bias",
2011
+ "shape": [
2012
+ 64
2013
+ ],
2014
+ "dtype": "float32",
2015
+ "offset": 65561600,
2016
+ "byteLength": 256
2017
+ },
2018
+ {
2019
+ "name": "transformer_blocks.4.s_attn.rope.spectral_encoder.phi.2.weight",
2020
+ "shape": [
2021
+ 64,
2022
+ 64
2023
+ ],
2024
+ "dtype": "float16",
2025
+ "offset": 65561856,
2026
+ "byteLength": 8192
2027
+ },
2028
+ {
2029
+ "name": "transformer_blocks.4.s_attn.rope.spectral_encoder.phi.2.bias",
2030
+ "shape": [
2031
+ 64
2032
+ ],
2033
+ "dtype": "float32",
2034
+ "offset": 65570048,
2035
+ "byteLength": 256
2036
+ },
2037
+ {
2038
+ "name": "transformer_blocks.4.s_attn.rope.spectral_encoder.rho.0.weight",
2039
+ "shape": [
2040
+ 64,
2041
+ 512
2042
+ ],
2043
+ "dtype": "float16",
2044
+ "offset": 65570304,
2045
+ "byteLength": 65536
2046
+ },
2047
+ {
2048
+ "name": "transformer_blocks.4.s_attn.rope.spectral_encoder.rho.0.bias",
2049
+ "shape": [
2050
+ 64
2051
+ ],
2052
+ "dtype": "float32",
2053
+ "offset": 65635840,
2054
+ "byteLength": 256
2055
+ },
2056
+ {
2057
+ "name": "transformer_blocks.4.s_attn.rope.spectral_encoder.rho.2.weight",
2058
+ "shape": [
2059
+ 32,
2060
+ 64
2061
+ ],
2062
+ "dtype": "float32",
2063
+ "offset": 65636096,
2064
+ "byteLength": 8192
2065
+ },
2066
+ {
2067
+ "name": "transformer_blocks.4.s_attn.rope.spectral_encoder.rho.2.bias",
2068
+ "shape": [
2069
+ 32
2070
+ ],
2071
+ "dtype": "float32",
2072
+ "offset": 65644288,
2073
+ "byteLength": 128
2074
+ },
2075
+ {
2076
+ "name": "transformer_blocks.4.s_attn.qkv.weight",
2077
+ "shape": [
2078
+ 1536,
2079
+ 512
2080
+ ],
2081
+ "dtype": "float16",
2082
+ "offset": 65644544,
2083
+ "byteLength": 1572864
2084
+ },
2085
+ {
2086
+ "name": "transformer_blocks.4.s_attn.qkv.bias",
2087
+ "shape": [
2088
+ 1536
2089
+ ],
2090
+ "dtype": "float32",
2091
+ "offset": 67217408,
2092
+ "byteLength": 6144
2093
+ },
2094
+ {
2095
+ "name": "transformer_blocks.4.s_attn.proj.weight",
2096
+ "shape": [
2097
+ 512,
2098
+ 512
2099
+ ],
2100
+ "dtype": "float16",
2101
+ "offset": 67223552,
2102
+ "byteLength": 524288
2103
+ },
2104
+ {
2105
+ "name": "transformer_blocks.4.s_attn.proj.bias",
2106
+ "shape": [
2107
+ 512
2108
+ ],
2109
+ "dtype": "float32",
2110
+ "offset": 67747840,
2111
+ "byteLength": 2048
2112
+ },
2113
+ {
2114
+ "name": "transformer_blocks.4.s_attn.q_norm.weight",
2115
+ "shape": [
2116
+ 64
2117
+ ],
2118
+ "dtype": "float32",
2119
+ "offset": 67749888,
2120
+ "byteLength": 256
2121
+ },
2122
+ {
2123
+ "name": "transformer_blocks.4.s_attn.k_norm.weight",
2124
+ "shape": [
2125
+ 64
2126
+ ],
2127
+ "dtype": "float32",
2128
+ "offset": 67750144,
2129
+ "byteLength": 256
2130
+ },
2131
+ {
2132
+ "name": "transformer_blocks.4.s_attn.graph_dist_embedding.weight",
2133
+ "shape": [
2134
+ 6,
2135
+ 128
2136
+ ],
2137
+ "dtype": "float32",
2138
+ "offset": 67750400,
2139
+ "byteLength": 3072
2140
+ },
2141
+ {
2142
+ "name": "transformer_blocks.4.s_attn.graph_dist_proj.weight",
2143
+ "shape": [
2144
+ 8,
2145
+ 128
2146
+ ],
2147
+ "dtype": "float32",
2148
+ "offset": 67753472,
2149
+ "byteLength": 4096
2150
+ },
2151
+ {
2152
+ "name": "transformer_blocks.4.s_attn.graph_dist_proj.bias",
2153
+ "shape": [
2154
+ 8
2155
+ ],
2156
+ "dtype": "float32",
2157
+ "offset": 67757568,
2158
+ "byteLength": 32
2159
+ },
2160
+ {
2161
+ "name": "transformer_blocks.4.s_attn.graph_rel_embedding.weight",
2162
+ "shape": [
2163
+ 6,
2164
+ 128
2165
+ ],
2166
+ "dtype": "float32",
2167
+ "offset": 67757824,
2168
+ "byteLength": 3072
2169
+ },
2170
+ {
2171
+ "name": "transformer_blocks.4.s_attn.graph_rel_proj.weight",
2172
+ "shape": [
2173
+ 8,
2174
+ 128
2175
+ ],
2176
+ "dtype": "float32",
2177
+ "offset": 67760896,
2178
+ "byteLength": 4096
2179
+ },
2180
+ {
2181
+ "name": "transformer_blocks.4.s_attn.graph_rel_proj.bias",
2182
+ "shape": [
2183
+ 8
2184
+ ],
2185
+ "dtype": "float32",
2186
+ "offset": 67764992,
2187
+ "byteLength": 32
2188
+ },
2189
+ {
2190
+ "name": "transformer_blocks.4.norm_t.weight",
2191
+ "shape": [
2192
+ 512
2193
+ ],
2194
+ "dtype": "float32",
2195
+ "offset": 67765248,
2196
+ "byteLength": 2048
2197
+ },
2198
+ {
2199
+ "name": "transformer_blocks.4.t_attn.qkv.weight",
2200
+ "shape": [
2201
+ 1536,
2202
+ 512
2203
+ ],
2204
+ "dtype": "float16",
2205
+ "offset": 67767296,
2206
+ "byteLength": 1572864
2207
+ },
2208
+ {
2209
+ "name": "transformer_blocks.4.t_attn.qkv.bias",
2210
+ "shape": [
2211
+ 1536
2212
+ ],
2213
+ "dtype": "float32",
2214
+ "offset": 69340160,
2215
+ "byteLength": 6144
2216
+ },
2217
+ {
2218
+ "name": "transformer_blocks.4.t_attn.proj.weight",
2219
+ "shape": [
2220
+ 512,
2221
+ 512
2222
+ ],
2223
+ "dtype": "float16",
2224
+ "offset": 69346304,
2225
+ "byteLength": 524288
2226
+ },
2227
+ {
2228
+ "name": "transformer_blocks.4.t_attn.proj.bias",
2229
+ "shape": [
2230
+ 512
2231
+ ],
2232
+ "dtype": "float32",
2233
+ "offset": 69870592,
2234
+ "byteLength": 2048
2235
+ },
2236
+ {
2237
+ "name": "transformer_blocks.4.t_attn.q_norm.weight",
2238
+ "shape": [
2239
+ 64
2240
+ ],
2241
+ "dtype": "float32",
2242
+ "offset": 69872640,
2243
+ "byteLength": 256
2244
+ },
2245
+ {
2246
+ "name": "transformer_blocks.4.t_attn.k_norm.weight",
2247
+ "shape": [
2248
+ 64
2249
+ ],
2250
+ "dtype": "float32",
2251
+ "offset": 69872896,
2252
+ "byteLength": 256
2253
+ },
2254
+ {
2255
+ "name": "transformer_blocks.4.norm_mlp.weight",
2256
+ "shape": [
2257
+ 512
2258
+ ],
2259
+ "dtype": "float32",
2260
+ "offset": 69873152,
2261
+ "byteLength": 2048
2262
+ },
2263
+ {
2264
+ "name": "transformer_blocks.4.mlp.w12.weight",
2265
+ "shape": [
2266
+ 2730,
2267
+ 512
2268
+ ],
2269
+ "dtype": "float16",
2270
+ "offset": 69875200,
2271
+ "byteLength": 2795520
2272
+ },
2273
+ {
2274
+ "name": "transformer_blocks.4.mlp.w12.bias",
2275
+ "shape": [
2276
+ 2730
2277
+ ],
2278
+ "dtype": "float32",
2279
+ "offset": 72670720,
2280
+ "byteLength": 10920
2281
+ },
2282
+ {
2283
+ "name": "transformer_blocks.4.mlp.w3.weight",
2284
+ "shape": [
2285
+ 512,
2286
+ 1365
2287
+ ],
2288
+ "dtype": "float16",
2289
+ "offset": 72681728,
2290
+ "byteLength": 1397760
2291
+ },
2292
+ {
2293
+ "name": "transformer_blocks.4.mlp.w3.bias",
2294
+ "shape": [
2295
+ 512
2296
+ ],
2297
+ "dtype": "float32",
2298
+ "offset": 74079488,
2299
+ "byteLength": 2048
2300
+ },
2301
+ {
2302
+ "name": "transformer_blocks.4.adaLN_modulation.1.weight",
2303
+ "shape": [
2304
+ 4608,
2305
+ 512
2306
+ ],
2307
+ "dtype": "float16",
2308
+ "offset": 74081536,
2309
+ "byteLength": 4718592
2310
+ },
2311
+ {
2312
+ "name": "transformer_blocks.4.adaLN_modulation.1.bias",
2313
+ "shape": [
2314
+ 4608
2315
+ ],
2316
+ "dtype": "float16",
2317
+ "offset": 78800128,
2318
+ "byteLength": 9216
2319
+ },
2320
+ {
2321
+ "name": "transformer_blocks.5.norm_s.weight",
2322
+ "shape": [
2323
+ 512
2324
+ ],
2325
+ "dtype": "float32",
2326
+ "offset": 78809344,
2327
+ "byteLength": 2048
2328
+ },
2329
+ {
2330
+ "name": "transformer_blocks.5.s_attn.graph_dist_scale",
2331
+ "shape": [
2332
+ 1
2333
+ ],
2334
+ "dtype": "float32",
2335
+ "offset": 78811392,
2336
+ "byteLength": 4
2337
+ },
2338
+ {
2339
+ "name": "transformer_blocks.5.s_attn.graph_rel_scale",
2340
+ "shape": [
2341
+ 1
2342
+ ],
2343
+ "dtype": "float32",
2344
+ "offset": 78811648,
2345
+ "byteLength": 4
2346
+ },
2347
+ {
2348
+ "name": "transformer_blocks.5.s_attn.rope.spectral_encoder.phi.0.weight",
2349
+ "shape": [
2350
+ 64,
2351
+ 1
2352
+ ],
2353
+ "dtype": "float32",
2354
+ "offset": 78811904,
2355
+ "byteLength": 256
2356
+ },
2357
+ {
2358
+ "name": "transformer_blocks.5.s_attn.rope.spectral_encoder.phi.0.bias",
2359
+ "shape": [
2360
+ 64
2361
+ ],
2362
+ "dtype": "float32",
2363
+ "offset": 78812160,
2364
+ "byteLength": 256
2365
+ },
2366
+ {
2367
+ "name": "transformer_blocks.5.s_attn.rope.spectral_encoder.phi.2.weight",
2368
+ "shape": [
2369
+ 64,
2370
+ 64
2371
+ ],
2372
+ "dtype": "float16",
2373
+ "offset": 78812416,
2374
+ "byteLength": 8192
2375
+ },
2376
+ {
2377
+ "name": "transformer_blocks.5.s_attn.rope.spectral_encoder.phi.2.bias",
2378
+ "shape": [
2379
+ 64
2380
+ ],
2381
+ "dtype": "float32",
2382
+ "offset": 78820608,
2383
+ "byteLength": 256
2384
+ },
2385
+ {
2386
+ "name": "transformer_blocks.5.s_attn.rope.spectral_encoder.rho.0.weight",
2387
+ "shape": [
2388
+ 64,
2389
+ 512
2390
+ ],
2391
+ "dtype": "float16",
2392
+ "offset": 78820864,
2393
+ "byteLength": 65536
2394
+ },
2395
+ {
2396
+ "name": "transformer_blocks.5.s_attn.rope.spectral_encoder.rho.0.bias",
2397
+ "shape": [
2398
+ 64
2399
+ ],
2400
+ "dtype": "float32",
2401
+ "offset": 78886400,
2402
+ "byteLength": 256
2403
+ },
2404
+ {
2405
+ "name": "transformer_blocks.5.s_attn.rope.spectral_encoder.rho.2.weight",
2406
+ "shape": [
2407
+ 32,
2408
+ 64
2409
+ ],
2410
+ "dtype": "float32",
2411
+ "offset": 78886656,
2412
+ "byteLength": 8192
2413
+ },
2414
+ {
2415
+ "name": "transformer_blocks.5.s_attn.rope.spectral_encoder.rho.2.bias",
2416
+ "shape": [
2417
+ 32
2418
+ ],
2419
+ "dtype": "float32",
2420
+ "offset": 78894848,
2421
+ "byteLength": 128
2422
+ },
2423
+ {
2424
+ "name": "transformer_blocks.5.s_attn.qkv.weight",
2425
+ "shape": [
2426
+ 1536,
2427
+ 512
2428
+ ],
2429
+ "dtype": "float16",
2430
+ "offset": 78895104,
2431
+ "byteLength": 1572864
2432
+ },
2433
+ {
2434
+ "name": "transformer_blocks.5.s_attn.qkv.bias",
2435
+ "shape": [
2436
+ 1536
2437
+ ],
2438
+ "dtype": "float32",
2439
+ "offset": 80467968,
2440
+ "byteLength": 6144
2441
+ },
2442
+ {
2443
+ "name": "transformer_blocks.5.s_attn.proj.weight",
2444
+ "shape": [
2445
+ 512,
2446
+ 512
2447
+ ],
2448
+ "dtype": "float16",
2449
+ "offset": 80474112,
2450
+ "byteLength": 524288
2451
+ },
2452
+ {
2453
+ "name": "transformer_blocks.5.s_attn.proj.bias",
2454
+ "shape": [
2455
+ 512
2456
+ ],
2457
+ "dtype": "float32",
2458
+ "offset": 80998400,
2459
+ "byteLength": 2048
2460
+ },
2461
+ {
2462
+ "name": "transformer_blocks.5.s_attn.q_norm.weight",
2463
+ "shape": [
2464
+ 64
2465
+ ],
2466
+ "dtype": "float32",
2467
+ "offset": 81000448,
2468
+ "byteLength": 256
2469
+ },
2470
+ {
2471
+ "name": "transformer_blocks.5.s_attn.k_norm.weight",
2472
+ "shape": [
2473
+ 64
2474
+ ],
2475
+ "dtype": "float32",
2476
+ "offset": 81000704,
2477
+ "byteLength": 256
2478
+ },
2479
+ {
2480
+ "name": "transformer_blocks.5.s_attn.graph_dist_embedding.weight",
2481
+ "shape": [
2482
+ 6,
2483
+ 128
2484
+ ],
2485
+ "dtype": "float32",
2486
+ "offset": 81000960,
2487
+ "byteLength": 3072
2488
+ },
2489
+ {
2490
+ "name": "transformer_blocks.5.s_attn.graph_dist_proj.weight",
2491
+ "shape": [
2492
+ 8,
2493
+ 128
2494
+ ],
2495
+ "dtype": "float32",
2496
+ "offset": 81004032,
2497
+ "byteLength": 4096
2498
+ },
2499
+ {
2500
+ "name": "transformer_blocks.5.s_attn.graph_dist_proj.bias",
2501
+ "shape": [
2502
+ 8
2503
+ ],
2504
+ "dtype": "float32",
2505
+ "offset": 81008128,
2506
+ "byteLength": 32
2507
+ },
2508
+ {
2509
+ "name": "transformer_blocks.5.s_attn.graph_rel_embedding.weight",
2510
+ "shape": [
2511
+ 6,
2512
+ 128
2513
+ ],
2514
+ "dtype": "float32",
2515
+ "offset": 81008384,
2516
+ "byteLength": 3072
2517
+ },
2518
+ {
2519
+ "name": "transformer_blocks.5.s_attn.graph_rel_proj.weight",
2520
+ "shape": [
2521
+ 8,
2522
+ 128
2523
+ ],
2524
+ "dtype": "float32",
2525
+ "offset": 81011456,
2526
+ "byteLength": 4096
2527
+ },
2528
+ {
2529
+ "name": "transformer_blocks.5.s_attn.graph_rel_proj.bias",
2530
+ "shape": [
2531
+ 8
2532
+ ],
2533
+ "dtype": "float32",
2534
+ "offset": 81015552,
2535
+ "byteLength": 32
2536
+ },
2537
+ {
2538
+ "name": "transformer_blocks.5.norm_t.weight",
2539
+ "shape": [
2540
+ 512
2541
+ ],
2542
+ "dtype": "float32",
2543
+ "offset": 81015808,
2544
+ "byteLength": 2048
2545
+ },
2546
+ {
2547
+ "name": "transformer_blocks.5.t_attn.qkv.weight",
2548
+ "shape": [
2549
+ 1536,
2550
+ 512
2551
+ ],
2552
+ "dtype": "float16",
2553
+ "offset": 81017856,
2554
+ "byteLength": 1572864
2555
+ },
2556
+ {
2557
+ "name": "transformer_blocks.5.t_attn.qkv.bias",
2558
+ "shape": [
2559
+ 1536
2560
+ ],
2561
+ "dtype": "float32",
2562
+ "offset": 82590720,
2563
+ "byteLength": 6144
2564
+ },
2565
+ {
2566
+ "name": "transformer_blocks.5.t_attn.proj.weight",
2567
+ "shape": [
2568
+ 512,
2569
+ 512
2570
+ ],
2571
+ "dtype": "float16",
2572
+ "offset": 82596864,
2573
+ "byteLength": 524288
2574
+ },
2575
+ {
2576
+ "name": "transformer_blocks.5.t_attn.proj.bias",
2577
+ "shape": [
2578
+ 512
2579
+ ],
2580
+ "dtype": "float32",
2581
+ "offset": 83121152,
2582
+ "byteLength": 2048
2583
+ },
2584
+ {
2585
+ "name": "transformer_blocks.5.t_attn.q_norm.weight",
2586
+ "shape": [
2587
+ 64
2588
+ ],
2589
+ "dtype": "float32",
2590
+ "offset": 83123200,
2591
+ "byteLength": 256
2592
+ },
2593
+ {
2594
+ "name": "transformer_blocks.5.t_attn.k_norm.weight",
2595
+ "shape": [
2596
+ 64
2597
+ ],
2598
+ "dtype": "float32",
2599
+ "offset": 83123456,
2600
+ "byteLength": 256
2601
+ },
2602
+ {
2603
+ "name": "transformer_blocks.5.norm_mlp.weight",
2604
+ "shape": [
2605
+ 512
2606
+ ],
2607
+ "dtype": "float32",
2608
+ "offset": 83123712,
2609
+ "byteLength": 2048
2610
+ },
2611
+ {
2612
+ "name": "transformer_blocks.5.mlp.w12.weight",
2613
+ "shape": [
2614
+ 2730,
2615
+ 512
2616
+ ],
2617
+ "dtype": "float16",
2618
+ "offset": 83125760,
2619
+ "byteLength": 2795520
2620
+ },
2621
+ {
2622
+ "name": "transformer_blocks.5.mlp.w12.bias",
2623
+ "shape": [
2624
+ 2730
2625
+ ],
2626
+ "dtype": "float32",
2627
+ "offset": 85921280,
2628
+ "byteLength": 10920
2629
+ },
2630
+ {
2631
+ "name": "transformer_blocks.5.mlp.w3.weight",
2632
+ "shape": [
2633
+ 512,
2634
+ 1365
2635
+ ],
2636
+ "dtype": "float16",
2637
+ "offset": 85932288,
2638
+ "byteLength": 1397760
2639
+ },
2640
+ {
2641
+ "name": "transformer_blocks.5.mlp.w3.bias",
2642
+ "shape": [
2643
+ 512
2644
+ ],
2645
+ "dtype": "float32",
2646
+ "offset": 87330048,
2647
+ "byteLength": 2048
2648
+ },
2649
+ {
2650
+ "name": "transformer_blocks.5.adaLN_modulation.1.weight",
2651
+ "shape": [
2652
+ 4608,
2653
+ 512
2654
+ ],
2655
+ "dtype": "float16",
2656
+ "offset": 87332096,
2657
+ "byteLength": 4718592
2658
+ },
2659
+ {
2660
+ "name": "transformer_blocks.5.adaLN_modulation.1.bias",
2661
+ "shape": [
2662
+ 4608
2663
+ ],
2664
+ "dtype": "float16",
2665
+ "offset": 92050688,
2666
+ "byteLength": 9216
2667
+ },
2668
+ {
2669
+ "name": "transformer_blocks.6.norm_s.weight",
2670
+ "shape": [
2671
+ 512
2672
+ ],
2673
+ "dtype": "float32",
2674
+ "offset": 92059904,
2675
+ "byteLength": 2048
2676
+ },
2677
+ {
2678
+ "name": "transformer_blocks.6.s_attn.graph_dist_scale",
2679
+ "shape": [
2680
+ 1
2681
+ ],
2682
+ "dtype": "float32",
2683
+ "offset": 92061952,
2684
+ "byteLength": 4
2685
+ },
2686
+ {
2687
+ "name": "transformer_blocks.6.s_attn.graph_rel_scale",
2688
+ "shape": [
2689
+ 1
2690
+ ],
2691
+ "dtype": "float32",
2692
+ "offset": 92062208,
2693
+ "byteLength": 4
2694
+ },
2695
+ {
2696
+ "name": "transformer_blocks.6.s_attn.rope.spectral_encoder.phi.0.weight",
2697
+ "shape": [
2698
+ 64,
2699
+ 1
2700
+ ],
2701
+ "dtype": "float32",
2702
+ "offset": 92062464,
2703
+ "byteLength": 256
2704
+ },
2705
+ {
2706
+ "name": "transformer_blocks.6.s_attn.rope.spectral_encoder.phi.0.bias",
2707
+ "shape": [
2708
+ 64
2709
+ ],
2710
+ "dtype": "float32",
2711
+ "offset": 92062720,
2712
+ "byteLength": 256
2713
+ },
2714
+ {
2715
+ "name": "transformer_blocks.6.s_attn.rope.spectral_encoder.phi.2.weight",
2716
+ "shape": [
2717
+ 64,
2718
+ 64
2719
+ ],
2720
+ "dtype": "float16",
2721
+ "offset": 92062976,
2722
+ "byteLength": 8192
2723
+ },
2724
+ {
2725
+ "name": "transformer_blocks.6.s_attn.rope.spectral_encoder.phi.2.bias",
2726
+ "shape": [
2727
+ 64
2728
+ ],
2729
+ "dtype": "float32",
2730
+ "offset": 92071168,
2731
+ "byteLength": 256
2732
+ },
2733
+ {
2734
+ "name": "transformer_blocks.6.s_attn.rope.spectral_encoder.rho.0.weight",
2735
+ "shape": [
2736
+ 64,
2737
+ 512
2738
+ ],
2739
+ "dtype": "float16",
2740
+ "offset": 92071424,
2741
+ "byteLength": 65536
2742
+ },
2743
+ {
2744
+ "name": "transformer_blocks.6.s_attn.rope.spectral_encoder.rho.0.bias",
2745
+ "shape": [
2746
+ 64
2747
+ ],
2748
+ "dtype": "float32",
2749
+ "offset": 92136960,
2750
+ "byteLength": 256
2751
+ },
2752
+ {
2753
+ "name": "transformer_blocks.6.s_attn.rope.spectral_encoder.rho.2.weight",
2754
+ "shape": [
2755
+ 32,
2756
+ 64
2757
+ ],
2758
+ "dtype": "float32",
2759
+ "offset": 92137216,
2760
+ "byteLength": 8192
2761
+ },
2762
+ {
2763
+ "name": "transformer_blocks.6.s_attn.rope.spectral_encoder.rho.2.bias",
2764
+ "shape": [
2765
+ 32
2766
+ ],
2767
+ "dtype": "float32",
2768
+ "offset": 92145408,
2769
+ "byteLength": 128
2770
+ },
2771
+ {
2772
+ "name": "transformer_blocks.6.s_attn.qkv.weight",
2773
+ "shape": [
2774
+ 1536,
2775
+ 512
2776
+ ],
2777
+ "dtype": "float16",
2778
+ "offset": 92145664,
2779
+ "byteLength": 1572864
2780
+ },
2781
+ {
2782
+ "name": "transformer_blocks.6.s_attn.qkv.bias",
2783
+ "shape": [
2784
+ 1536
2785
+ ],
2786
+ "dtype": "float32",
2787
+ "offset": 93718528,
2788
+ "byteLength": 6144
2789
+ },
2790
+ {
2791
+ "name": "transformer_blocks.6.s_attn.proj.weight",
2792
+ "shape": [
2793
+ 512,
2794
+ 512
2795
+ ],
2796
+ "dtype": "float16",
2797
+ "offset": 93724672,
2798
+ "byteLength": 524288
2799
+ },
2800
+ {
2801
+ "name": "transformer_blocks.6.s_attn.proj.bias",
2802
+ "shape": [
2803
+ 512
2804
+ ],
2805
+ "dtype": "float32",
2806
+ "offset": 94248960,
2807
+ "byteLength": 2048
2808
+ },
2809
+ {
2810
+ "name": "transformer_blocks.6.s_attn.q_norm.weight",
2811
+ "shape": [
2812
+ 64
2813
+ ],
2814
+ "dtype": "float32",
2815
+ "offset": 94251008,
2816
+ "byteLength": 256
2817
+ },
2818
+ {
2819
+ "name": "transformer_blocks.6.s_attn.k_norm.weight",
2820
+ "shape": [
2821
+ 64
2822
+ ],
2823
+ "dtype": "float32",
2824
+ "offset": 94251264,
2825
+ "byteLength": 256
2826
+ },
2827
+ {
2828
+ "name": "transformer_blocks.6.s_attn.graph_dist_embedding.weight",
2829
+ "shape": [
2830
+ 6,
2831
+ 128
2832
+ ],
2833
+ "dtype": "float32",
2834
+ "offset": 94251520,
2835
+ "byteLength": 3072
2836
+ },
2837
+ {
2838
+ "name": "transformer_blocks.6.s_attn.graph_dist_proj.weight",
2839
+ "shape": [
2840
+ 8,
2841
+ 128
2842
+ ],
2843
+ "dtype": "float32",
2844
+ "offset": 94254592,
2845
+ "byteLength": 4096
2846
+ },
2847
+ {
2848
+ "name": "transformer_blocks.6.s_attn.graph_dist_proj.bias",
2849
+ "shape": [
2850
+ 8
2851
+ ],
2852
+ "dtype": "float32",
2853
+ "offset": 94258688,
2854
+ "byteLength": 32
2855
+ },
2856
+ {
2857
+ "name": "transformer_blocks.6.s_attn.graph_rel_embedding.weight",
2858
+ "shape": [
2859
+ 6,
2860
+ 128
2861
+ ],
2862
+ "dtype": "float32",
2863
+ "offset": 94258944,
2864
+ "byteLength": 3072
2865
+ },
2866
+ {
2867
+ "name": "transformer_blocks.6.s_attn.graph_rel_proj.weight",
2868
+ "shape": [
2869
+ 8,
2870
+ 128
2871
+ ],
2872
+ "dtype": "float32",
2873
+ "offset": 94262016,
2874
+ "byteLength": 4096
2875
+ },
2876
+ {
2877
+ "name": "transformer_blocks.6.s_attn.graph_rel_proj.bias",
2878
+ "shape": [
2879
+ 8
2880
+ ],
2881
+ "dtype": "float32",
2882
+ "offset": 94266112,
2883
+ "byteLength": 32
2884
+ },
2885
+ {
2886
+ "name": "transformer_blocks.6.norm_t.weight",
2887
+ "shape": [
2888
+ 512
2889
+ ],
2890
+ "dtype": "float32",
2891
+ "offset": 94266368,
2892
+ "byteLength": 2048
2893
+ },
2894
+ {
2895
+ "name": "transformer_blocks.6.t_attn.qkv.weight",
2896
+ "shape": [
2897
+ 1536,
2898
+ 512
2899
+ ],
2900
+ "dtype": "float16",
2901
+ "offset": 94268416,
2902
+ "byteLength": 1572864
2903
+ },
2904
+ {
2905
+ "name": "transformer_blocks.6.t_attn.qkv.bias",
2906
+ "shape": [
2907
+ 1536
2908
+ ],
2909
+ "dtype": "float32",
2910
+ "offset": 95841280,
2911
+ "byteLength": 6144
2912
+ },
2913
+ {
2914
+ "name": "transformer_blocks.6.t_attn.proj.weight",
2915
+ "shape": [
2916
+ 512,
2917
+ 512
2918
+ ],
2919
+ "dtype": "float16",
2920
+ "offset": 95847424,
2921
+ "byteLength": 524288
2922
+ },
2923
+ {
2924
+ "name": "transformer_blocks.6.t_attn.proj.bias",
2925
+ "shape": [
2926
+ 512
2927
+ ],
2928
+ "dtype": "float32",
2929
+ "offset": 96371712,
2930
+ "byteLength": 2048
2931
+ },
2932
+ {
2933
+ "name": "transformer_blocks.6.t_attn.q_norm.weight",
2934
+ "shape": [
2935
+ 64
2936
+ ],
2937
+ "dtype": "float32",
2938
+ "offset": 96373760,
2939
+ "byteLength": 256
2940
+ },
2941
+ {
2942
+ "name": "transformer_blocks.6.t_attn.k_norm.weight",
2943
+ "shape": [
2944
+ 64
2945
+ ],
2946
+ "dtype": "float32",
2947
+ "offset": 96374016,
2948
+ "byteLength": 256
2949
+ },
2950
+ {
2951
+ "name": "transformer_blocks.6.norm_mlp.weight",
2952
+ "shape": [
2953
+ 512
2954
+ ],
2955
+ "dtype": "float32",
2956
+ "offset": 96374272,
2957
+ "byteLength": 2048
2958
+ },
2959
+ {
2960
+ "name": "transformer_blocks.6.mlp.w12.weight",
2961
+ "shape": [
2962
+ 2730,
2963
+ 512
2964
+ ],
2965
+ "dtype": "float16",
2966
+ "offset": 96376320,
2967
+ "byteLength": 2795520
2968
+ },
2969
+ {
2970
+ "name": "transformer_blocks.6.mlp.w12.bias",
2971
+ "shape": [
2972
+ 2730
2973
+ ],
2974
+ "dtype": "float32",
2975
+ "offset": 99171840,
2976
+ "byteLength": 10920
2977
+ },
2978
+ {
2979
+ "name": "transformer_blocks.6.mlp.w3.weight",
2980
+ "shape": [
2981
+ 512,
2982
+ 1365
2983
+ ],
2984
+ "dtype": "float16",
2985
+ "offset": 99182848,
2986
+ "byteLength": 1397760
2987
+ },
2988
+ {
2989
+ "name": "transformer_blocks.6.mlp.w3.bias",
2990
+ "shape": [
2991
+ 512
2992
+ ],
2993
+ "dtype": "float32",
2994
+ "offset": 100580608,
2995
+ "byteLength": 2048
2996
+ },
2997
+ {
2998
+ "name": "transformer_blocks.6.adaLN_modulation.1.weight",
2999
+ "shape": [
3000
+ 4608,
3001
+ 512
3002
+ ],
3003
+ "dtype": "float16",
3004
+ "offset": 100582656,
3005
+ "byteLength": 4718592
3006
+ },
3007
+ {
3008
+ "name": "transformer_blocks.6.adaLN_modulation.1.bias",
3009
+ "shape": [
3010
+ 4608
3011
+ ],
3012
+ "dtype": "float16",
3013
+ "offset": 105301248,
3014
+ "byteLength": 9216
3015
+ },
3016
+ {
3017
+ "name": "transformer_blocks.7.norm_s.weight",
3018
+ "shape": [
3019
+ 512
3020
+ ],
3021
+ "dtype": "float32",
3022
+ "offset": 105310464,
3023
+ "byteLength": 2048
3024
+ },
3025
+ {
3026
+ "name": "transformer_blocks.7.s_attn.graph_dist_scale",
3027
+ "shape": [
3028
+ 1
3029
+ ],
3030
+ "dtype": "float32",
3031
+ "offset": 105312512,
3032
+ "byteLength": 4
3033
+ },
3034
+ {
3035
+ "name": "transformer_blocks.7.s_attn.graph_rel_scale",
3036
+ "shape": [
3037
+ 1
3038
+ ],
3039
+ "dtype": "float32",
3040
+ "offset": 105312768,
3041
+ "byteLength": 4
3042
+ },
3043
+ {
3044
+ "name": "transformer_blocks.7.s_attn.rope.spectral_encoder.phi.0.weight",
3045
+ "shape": [
3046
+ 64,
3047
+ 1
3048
+ ],
3049
+ "dtype": "float32",
3050
+ "offset": 105313024,
3051
+ "byteLength": 256
3052
+ },
3053
+ {
3054
+ "name": "transformer_blocks.7.s_attn.rope.spectral_encoder.phi.0.bias",
3055
+ "shape": [
3056
+ 64
3057
+ ],
3058
+ "dtype": "float32",
3059
+ "offset": 105313280,
3060
+ "byteLength": 256
3061
+ },
3062
+ {
3063
+ "name": "transformer_blocks.7.s_attn.rope.spectral_encoder.phi.2.weight",
3064
+ "shape": [
3065
+ 64,
3066
+ 64
3067
+ ],
3068
+ "dtype": "float16",
3069
+ "offset": 105313536,
3070
+ "byteLength": 8192
3071
+ },
3072
+ {
3073
+ "name": "transformer_blocks.7.s_attn.rope.spectral_encoder.phi.2.bias",
3074
+ "shape": [
3075
+ 64
3076
+ ],
3077
+ "dtype": "float32",
3078
+ "offset": 105321728,
3079
+ "byteLength": 256
3080
+ },
3081
+ {
3082
+ "name": "transformer_blocks.7.s_attn.rope.spectral_encoder.rho.0.weight",
3083
+ "shape": [
3084
+ 64,
3085
+ 512
3086
+ ],
3087
+ "dtype": "float16",
3088
+ "offset": 105321984,
3089
+ "byteLength": 65536
3090
+ },
3091
+ {
3092
+ "name": "transformer_blocks.7.s_attn.rope.spectral_encoder.rho.0.bias",
3093
+ "shape": [
3094
+ 64
3095
+ ],
3096
+ "dtype": "float32",
3097
+ "offset": 105387520,
3098
+ "byteLength": 256
3099
+ },
3100
+ {
3101
+ "name": "transformer_blocks.7.s_attn.rope.spectral_encoder.rho.2.weight",
3102
+ "shape": [
3103
+ 32,
3104
+ 64
3105
+ ],
3106
+ "dtype": "float32",
3107
+ "offset": 105387776,
3108
+ "byteLength": 8192
3109
+ },
3110
+ {
3111
+ "name": "transformer_blocks.7.s_attn.rope.spectral_encoder.rho.2.bias",
3112
+ "shape": [
3113
+ 32
3114
+ ],
3115
+ "dtype": "float32",
3116
+ "offset": 105395968,
3117
+ "byteLength": 128
3118
+ },
3119
+ {
3120
+ "name": "transformer_blocks.7.s_attn.qkv.weight",
3121
+ "shape": [
3122
+ 1536,
3123
+ 512
3124
+ ],
3125
+ "dtype": "float16",
3126
+ "offset": 105396224,
3127
+ "byteLength": 1572864
3128
+ },
3129
+ {
3130
+ "name": "transformer_blocks.7.s_attn.qkv.bias",
3131
+ "shape": [
3132
+ 1536
3133
+ ],
3134
+ "dtype": "float32",
3135
+ "offset": 106969088,
3136
+ "byteLength": 6144
3137
+ },
3138
+ {
3139
+ "name": "transformer_blocks.7.s_attn.proj.weight",
3140
+ "shape": [
3141
+ 512,
3142
+ 512
3143
+ ],
3144
+ "dtype": "float16",
3145
+ "offset": 106975232,
3146
+ "byteLength": 524288
3147
+ },
3148
+ {
3149
+ "name": "transformer_blocks.7.s_attn.proj.bias",
3150
+ "shape": [
3151
+ 512
3152
+ ],
3153
+ "dtype": "float32",
3154
+ "offset": 107499520,
3155
+ "byteLength": 2048
3156
+ },
3157
+ {
3158
+ "name": "transformer_blocks.7.s_attn.q_norm.weight",
3159
+ "shape": [
3160
+ 64
3161
+ ],
3162
+ "dtype": "float32",
3163
+ "offset": 107501568,
3164
+ "byteLength": 256
3165
+ },
3166
+ {
3167
+ "name": "transformer_blocks.7.s_attn.k_norm.weight",
3168
+ "shape": [
3169
+ 64
3170
+ ],
3171
+ "dtype": "float32",
3172
+ "offset": 107501824,
3173
+ "byteLength": 256
3174
+ },
3175
+ {
3176
+ "name": "transformer_blocks.7.s_attn.graph_dist_embedding.weight",
3177
+ "shape": [
3178
+ 6,
3179
+ 128
3180
+ ],
3181
+ "dtype": "float32",
3182
+ "offset": 107502080,
3183
+ "byteLength": 3072
3184
+ },
3185
+ {
3186
+ "name": "transformer_blocks.7.s_attn.graph_dist_proj.weight",
3187
+ "shape": [
3188
+ 8,
3189
+ 128
3190
+ ],
3191
+ "dtype": "float32",
3192
+ "offset": 107505152,
3193
+ "byteLength": 4096
3194
+ },
3195
+ {
3196
+ "name": "transformer_blocks.7.s_attn.graph_dist_proj.bias",
3197
+ "shape": [
3198
+ 8
3199
+ ],
3200
+ "dtype": "float32",
3201
+ "offset": 107509248,
3202
+ "byteLength": 32
3203
+ },
3204
+ {
3205
+ "name": "transformer_blocks.7.s_attn.graph_rel_embedding.weight",
3206
+ "shape": [
3207
+ 6,
3208
+ 128
3209
+ ],
3210
+ "dtype": "float32",
3211
+ "offset": 107509504,
3212
+ "byteLength": 3072
3213
+ },
3214
+ {
3215
+ "name": "transformer_blocks.7.s_attn.graph_rel_proj.weight",
3216
+ "shape": [
3217
+ 8,
3218
+ 128
3219
+ ],
3220
+ "dtype": "float32",
3221
+ "offset": 107512576,
3222
+ "byteLength": 4096
3223
+ },
3224
+ {
3225
+ "name": "transformer_blocks.7.s_attn.graph_rel_proj.bias",
3226
+ "shape": [
3227
+ 8
3228
+ ],
3229
+ "dtype": "float32",
3230
+ "offset": 107516672,
3231
+ "byteLength": 32
3232
+ },
3233
+ {
3234
+ "name": "transformer_blocks.7.norm_t.weight",
3235
+ "shape": [
3236
+ 512
3237
+ ],
3238
+ "dtype": "float32",
3239
+ "offset": 107516928,
3240
+ "byteLength": 2048
3241
+ },
3242
+ {
3243
+ "name": "transformer_blocks.7.t_attn.qkv.weight",
3244
+ "shape": [
3245
+ 1536,
3246
+ 512
3247
+ ],
3248
+ "dtype": "float16",
3249
+ "offset": 107518976,
3250
+ "byteLength": 1572864
3251
+ },
3252
+ {
3253
+ "name": "transformer_blocks.7.t_attn.qkv.bias",
3254
+ "shape": [
3255
+ 1536
3256
+ ],
3257
+ "dtype": "float32",
3258
+ "offset": 109091840,
3259
+ "byteLength": 6144
3260
+ },
3261
+ {
3262
+ "name": "transformer_blocks.7.t_attn.proj.weight",
3263
+ "shape": [
3264
+ 512,
3265
+ 512
3266
+ ],
3267
+ "dtype": "float16",
3268
+ "offset": 109097984,
3269
+ "byteLength": 524288
3270
+ },
3271
+ {
3272
+ "name": "transformer_blocks.7.t_attn.proj.bias",
3273
+ "shape": [
3274
+ 512
3275
+ ],
3276
+ "dtype": "float32",
3277
+ "offset": 109622272,
3278
+ "byteLength": 2048
3279
+ },
3280
+ {
3281
+ "name": "transformer_blocks.7.t_attn.q_norm.weight",
3282
+ "shape": [
3283
+ 64
3284
+ ],
3285
+ "dtype": "float32",
3286
+ "offset": 109624320,
3287
+ "byteLength": 256
3288
+ },
3289
+ {
3290
+ "name": "transformer_blocks.7.t_attn.k_norm.weight",
3291
+ "shape": [
3292
+ 64
3293
+ ],
3294
+ "dtype": "float32",
3295
+ "offset": 109624576,
3296
+ "byteLength": 256
3297
+ },
3298
+ {
3299
+ "name": "transformer_blocks.7.norm_mlp.weight",
3300
+ "shape": [
3301
+ 512
3302
+ ],
3303
+ "dtype": "float32",
3304
+ "offset": 109624832,
3305
+ "byteLength": 2048
3306
+ },
3307
+ {
3308
+ "name": "transformer_blocks.7.mlp.w12.weight",
3309
+ "shape": [
3310
+ 2730,
3311
+ 512
3312
+ ],
3313
+ "dtype": "float16",
3314
+ "offset": 109626880,
3315
+ "byteLength": 2795520
3316
+ },
3317
+ {
3318
+ "name": "transformer_blocks.7.mlp.w12.bias",
3319
+ "shape": [
3320
+ 2730
3321
+ ],
3322
+ "dtype": "float32",
3323
+ "offset": 112422400,
3324
+ "byteLength": 10920
3325
+ },
3326
+ {
3327
+ "name": "transformer_blocks.7.mlp.w3.weight",
3328
+ "shape": [
3329
+ 512,
3330
+ 1365
3331
+ ],
3332
+ "dtype": "float16",
3333
+ "offset": 112433408,
3334
+ "byteLength": 1397760
3335
+ },
3336
+ {
3337
+ "name": "transformer_blocks.7.mlp.w3.bias",
3338
+ "shape": [
3339
+ 512
3340
+ ],
3341
+ "dtype": "float32",
3342
+ "offset": 113831168,
3343
+ "byteLength": 2048
3344
+ },
3345
+ {
3346
+ "name": "transformer_blocks.7.adaLN_modulation.1.weight",
3347
+ "shape": [
3348
+ 4608,
3349
+ 512
3350
+ ],
3351
+ "dtype": "float16",
3352
+ "offset": 113833216,
3353
+ "byteLength": 4718592
3354
+ },
3355
+ {
3356
+ "name": "transformer_blocks.7.adaLN_modulation.1.bias",
3357
+ "shape": [
3358
+ 4608
3359
+ ],
3360
+ "dtype": "float16",
3361
+ "offset": 118551808,
3362
+ "byteLength": 9216
3363
+ },
3364
+ {
3365
+ "name": "transformer_blocks.8.norm_s.weight",
3366
+ "shape": [
3367
+ 512
3368
+ ],
3369
+ "dtype": "float32",
3370
+ "offset": 118561024,
3371
+ "byteLength": 2048
3372
+ },
3373
+ {
3374
+ "name": "transformer_blocks.8.s_attn.graph_dist_scale",
3375
+ "shape": [
3376
+ 1
3377
+ ],
3378
+ "dtype": "float32",
3379
+ "offset": 118563072,
3380
+ "byteLength": 4
3381
+ },
3382
+ {
3383
+ "name": "transformer_blocks.8.s_attn.graph_rel_scale",
3384
+ "shape": [
3385
+ 1
3386
+ ],
3387
+ "dtype": "float32",
3388
+ "offset": 118563328,
3389
+ "byteLength": 4
3390
+ },
3391
+ {
3392
+ "name": "transformer_blocks.8.s_attn.rope.spectral_encoder.phi.0.weight",
3393
+ "shape": [
3394
+ 64,
3395
+ 1
3396
+ ],
3397
+ "dtype": "float32",
3398
+ "offset": 118563584,
3399
+ "byteLength": 256
3400
+ },
3401
+ {
3402
+ "name": "transformer_blocks.8.s_attn.rope.spectral_encoder.phi.0.bias",
3403
+ "shape": [
3404
+ 64
3405
+ ],
3406
+ "dtype": "float32",
3407
+ "offset": 118563840,
3408
+ "byteLength": 256
3409
+ },
3410
+ {
3411
+ "name": "transformer_blocks.8.s_attn.rope.spectral_encoder.phi.2.weight",
3412
+ "shape": [
3413
+ 64,
3414
+ 64
3415
+ ],
3416
+ "dtype": "float16",
3417
+ "offset": 118564096,
3418
+ "byteLength": 8192
3419
+ },
3420
+ {
3421
+ "name": "transformer_blocks.8.s_attn.rope.spectral_encoder.phi.2.bias",
3422
+ "shape": [
3423
+ 64
3424
+ ],
3425
+ "dtype": "float32",
3426
+ "offset": 118572288,
3427
+ "byteLength": 256
3428
+ },
3429
+ {
3430
+ "name": "transformer_blocks.8.s_attn.rope.spectral_encoder.rho.0.weight",
3431
+ "shape": [
3432
+ 64,
3433
+ 512
3434
+ ],
3435
+ "dtype": "float16",
3436
+ "offset": 118572544,
3437
+ "byteLength": 65536
3438
+ },
3439
+ {
3440
+ "name": "transformer_blocks.8.s_attn.rope.spectral_encoder.rho.0.bias",
3441
+ "shape": [
3442
+ 64
3443
+ ],
3444
+ "dtype": "float32",
3445
+ "offset": 118638080,
3446
+ "byteLength": 256
3447
+ },
3448
+ {
3449
+ "name": "transformer_blocks.8.s_attn.rope.spectral_encoder.rho.2.weight",
3450
+ "shape": [
3451
+ 32,
3452
+ 64
3453
+ ],
3454
+ "dtype": "float32",
3455
+ "offset": 118638336,
3456
+ "byteLength": 8192
3457
+ },
3458
+ {
3459
+ "name": "transformer_blocks.8.s_attn.rope.spectral_encoder.rho.2.bias",
3460
+ "shape": [
3461
+ 32
3462
+ ],
3463
+ "dtype": "float32",
3464
+ "offset": 118646528,
3465
+ "byteLength": 128
3466
+ },
3467
+ {
3468
+ "name": "transformer_blocks.8.s_attn.qkv.weight",
3469
+ "shape": [
3470
+ 1536,
3471
+ 512
3472
+ ],
3473
+ "dtype": "float16",
3474
+ "offset": 118646784,
3475
+ "byteLength": 1572864
3476
+ },
3477
+ {
3478
+ "name": "transformer_blocks.8.s_attn.qkv.bias",
3479
+ "shape": [
3480
+ 1536
3481
+ ],
3482
+ "dtype": "float32",
3483
+ "offset": 120219648,
3484
+ "byteLength": 6144
3485
+ },
3486
+ {
3487
+ "name": "transformer_blocks.8.s_attn.proj.weight",
3488
+ "shape": [
3489
+ 512,
3490
+ 512
3491
+ ],
3492
+ "dtype": "float16",
3493
+ "offset": 120225792,
3494
+ "byteLength": 524288
3495
+ },
3496
+ {
3497
+ "name": "transformer_blocks.8.s_attn.proj.bias",
3498
+ "shape": [
3499
+ 512
3500
+ ],
3501
+ "dtype": "float32",
3502
+ "offset": 120750080,
3503
+ "byteLength": 2048
3504
+ },
3505
+ {
3506
+ "name": "transformer_blocks.8.s_attn.q_norm.weight",
3507
+ "shape": [
3508
+ 64
3509
+ ],
3510
+ "dtype": "float32",
3511
+ "offset": 120752128,
3512
+ "byteLength": 256
3513
+ },
3514
+ {
3515
+ "name": "transformer_blocks.8.s_attn.k_norm.weight",
3516
+ "shape": [
3517
+ 64
3518
+ ],
3519
+ "dtype": "float32",
3520
+ "offset": 120752384,
3521
+ "byteLength": 256
3522
+ },
3523
+ {
3524
+ "name": "transformer_blocks.8.s_attn.graph_dist_embedding.weight",
3525
+ "shape": [
3526
+ 6,
3527
+ 128
3528
+ ],
3529
+ "dtype": "float32",
3530
+ "offset": 120752640,
3531
+ "byteLength": 3072
3532
+ },
3533
+ {
3534
+ "name": "transformer_blocks.8.s_attn.graph_dist_proj.weight",
3535
+ "shape": [
3536
+ 8,
3537
+ 128
3538
+ ],
3539
+ "dtype": "float32",
3540
+ "offset": 120755712,
3541
+ "byteLength": 4096
3542
+ },
3543
+ {
3544
+ "name": "transformer_blocks.8.s_attn.graph_dist_proj.bias",
3545
+ "shape": [
3546
+ 8
3547
+ ],
3548
+ "dtype": "float32",
3549
+ "offset": 120759808,
3550
+ "byteLength": 32
3551
+ },
3552
+ {
3553
+ "name": "transformer_blocks.8.s_attn.graph_rel_embedding.weight",
3554
+ "shape": [
3555
+ 6,
3556
+ 128
3557
+ ],
3558
+ "dtype": "float32",
3559
+ "offset": 120760064,
3560
+ "byteLength": 3072
3561
+ },
3562
+ {
3563
+ "name": "transformer_blocks.8.s_attn.graph_rel_proj.weight",
3564
+ "shape": [
3565
+ 8,
3566
+ 128
3567
+ ],
3568
+ "dtype": "float32",
3569
+ "offset": 120763136,
3570
+ "byteLength": 4096
3571
+ },
3572
+ {
3573
+ "name": "transformer_blocks.8.s_attn.graph_rel_proj.bias",
3574
+ "shape": [
3575
+ 8
3576
+ ],
3577
+ "dtype": "float32",
3578
+ "offset": 120767232,
3579
+ "byteLength": 32
3580
+ },
3581
+ {
3582
+ "name": "transformer_blocks.8.norm_t.weight",
3583
+ "shape": [
3584
+ 512
3585
+ ],
3586
+ "dtype": "float32",
3587
+ "offset": 120767488,
3588
+ "byteLength": 2048
3589
+ },
3590
+ {
3591
+ "name": "transformer_blocks.8.t_attn.qkv.weight",
3592
+ "shape": [
3593
+ 1536,
3594
+ 512
3595
+ ],
3596
+ "dtype": "float16",
3597
+ "offset": 120769536,
3598
+ "byteLength": 1572864
3599
+ },
3600
+ {
3601
+ "name": "transformer_blocks.8.t_attn.qkv.bias",
3602
+ "shape": [
3603
+ 1536
3604
+ ],
3605
+ "dtype": "float32",
3606
+ "offset": 122342400,
3607
+ "byteLength": 6144
3608
+ },
3609
+ {
3610
+ "name": "transformer_blocks.8.t_attn.proj.weight",
3611
+ "shape": [
3612
+ 512,
3613
+ 512
3614
+ ],
3615
+ "dtype": "float16",
3616
+ "offset": 122348544,
3617
+ "byteLength": 524288
3618
+ },
3619
+ {
3620
+ "name": "transformer_blocks.8.t_attn.proj.bias",
3621
+ "shape": [
3622
+ 512
3623
+ ],
3624
+ "dtype": "float32",
3625
+ "offset": 122872832,
3626
+ "byteLength": 2048
3627
+ },
3628
+ {
3629
+ "name": "transformer_blocks.8.t_attn.q_norm.weight",
3630
+ "shape": [
3631
+ 64
3632
+ ],
3633
+ "dtype": "float32",
3634
+ "offset": 122874880,
3635
+ "byteLength": 256
3636
+ },
3637
+ {
3638
+ "name": "transformer_blocks.8.t_attn.k_norm.weight",
3639
+ "shape": [
3640
+ 64
3641
+ ],
3642
+ "dtype": "float32",
3643
+ "offset": 122875136,
3644
+ "byteLength": 256
3645
+ },
3646
+ {
3647
+ "name": "transformer_blocks.8.norm_mlp.weight",
3648
+ "shape": [
3649
+ 512
3650
+ ],
3651
+ "dtype": "float32",
3652
+ "offset": 122875392,
3653
+ "byteLength": 2048
3654
+ },
3655
+ {
3656
+ "name": "transformer_blocks.8.mlp.w12.weight",
3657
+ "shape": [
3658
+ 2730,
3659
+ 512
3660
+ ],
3661
+ "dtype": "float16",
3662
+ "offset": 122877440,
3663
+ "byteLength": 2795520
3664
+ },
3665
+ {
3666
+ "name": "transformer_blocks.8.mlp.w12.bias",
3667
+ "shape": [
3668
+ 2730
3669
+ ],
3670
+ "dtype": "float32",
3671
+ "offset": 125672960,
3672
+ "byteLength": 10920
3673
+ },
3674
+ {
3675
+ "name": "transformer_blocks.8.mlp.w3.weight",
3676
+ "shape": [
3677
+ 512,
3678
+ 1365
3679
+ ],
3680
+ "dtype": "float16",
3681
+ "offset": 125683968,
3682
+ "byteLength": 1397760
3683
+ },
3684
+ {
3685
+ "name": "transformer_blocks.8.mlp.w3.bias",
3686
+ "shape": [
3687
+ 512
3688
+ ],
3689
+ "dtype": "float32",
3690
+ "offset": 127081728,
3691
+ "byteLength": 2048
3692
+ },
3693
+ {
3694
+ "name": "transformer_blocks.8.adaLN_modulation.1.weight",
3695
+ "shape": [
3696
+ 4608,
3697
+ 512
3698
+ ],
3699
+ "dtype": "float16",
3700
+ "offset": 127083776,
3701
+ "byteLength": 4718592
3702
+ },
3703
+ {
3704
+ "name": "transformer_blocks.8.adaLN_modulation.1.bias",
3705
+ "shape": [
3706
+ 4608
3707
+ ],
3708
+ "dtype": "float16",
3709
+ "offset": 131802368,
3710
+ "byteLength": 9216
3711
+ },
3712
+ {
3713
+ "name": "transformer_blocks.9.norm_s.weight",
3714
+ "shape": [
3715
+ 512
3716
+ ],
3717
+ "dtype": "float32",
3718
+ "offset": 131811584,
3719
+ "byteLength": 2048
3720
+ },
3721
+ {
3722
+ "name": "transformer_blocks.9.s_attn.graph_dist_scale",
3723
+ "shape": [
3724
+ 1
3725
+ ],
3726
+ "dtype": "float32",
3727
+ "offset": 131813632,
3728
+ "byteLength": 4
3729
+ },
3730
+ {
3731
+ "name": "transformer_blocks.9.s_attn.graph_rel_scale",
3732
+ "shape": [
3733
+ 1
3734
+ ],
3735
+ "dtype": "float32",
3736
+ "offset": 131813888,
3737
+ "byteLength": 4
3738
+ },
3739
+ {
3740
+ "name": "transformer_blocks.9.s_attn.rope.spectral_encoder.phi.0.weight",
3741
+ "shape": [
3742
+ 64,
3743
+ 1
3744
+ ],
3745
+ "dtype": "float32",
3746
+ "offset": 131814144,
3747
+ "byteLength": 256
3748
+ },
3749
+ {
3750
+ "name": "transformer_blocks.9.s_attn.rope.spectral_encoder.phi.0.bias",
3751
+ "shape": [
3752
+ 64
3753
+ ],
3754
+ "dtype": "float32",
3755
+ "offset": 131814400,
3756
+ "byteLength": 256
3757
+ },
3758
+ {
3759
+ "name": "transformer_blocks.9.s_attn.rope.spectral_encoder.phi.2.weight",
3760
+ "shape": [
3761
+ 64,
3762
+ 64
3763
+ ],
3764
+ "dtype": "float16",
3765
+ "offset": 131814656,
3766
+ "byteLength": 8192
3767
+ },
3768
+ {
3769
+ "name": "transformer_blocks.9.s_attn.rope.spectral_encoder.phi.2.bias",
3770
+ "shape": [
3771
+ 64
3772
+ ],
3773
+ "dtype": "float32",
3774
+ "offset": 131822848,
3775
+ "byteLength": 256
3776
+ },
3777
+ {
3778
+ "name": "transformer_blocks.9.s_attn.rope.spectral_encoder.rho.0.weight",
3779
+ "shape": [
3780
+ 64,
3781
+ 512
3782
+ ],
3783
+ "dtype": "float16",
3784
+ "offset": 131823104,
3785
+ "byteLength": 65536
3786
+ },
3787
+ {
3788
+ "name": "transformer_blocks.9.s_attn.rope.spectral_encoder.rho.0.bias",
3789
+ "shape": [
3790
+ 64
3791
+ ],
3792
+ "dtype": "float32",
3793
+ "offset": 131888640,
3794
+ "byteLength": 256
3795
+ },
3796
+ {
3797
+ "name": "transformer_blocks.9.s_attn.rope.spectral_encoder.rho.2.weight",
3798
+ "shape": [
3799
+ 32,
3800
+ 64
3801
+ ],
3802
+ "dtype": "float32",
3803
+ "offset": 131888896,
3804
+ "byteLength": 8192
3805
+ },
3806
+ {
3807
+ "name": "transformer_blocks.9.s_attn.rope.spectral_encoder.rho.2.bias",
3808
+ "shape": [
3809
+ 32
3810
+ ],
3811
+ "dtype": "float32",
3812
+ "offset": 131897088,
3813
+ "byteLength": 128
3814
+ },
3815
+ {
3816
+ "name": "transformer_blocks.9.s_attn.qkv.weight",
3817
+ "shape": [
3818
+ 1536,
3819
+ 512
3820
+ ],
3821
+ "dtype": "float16",
3822
+ "offset": 131897344,
3823
+ "byteLength": 1572864
3824
+ },
3825
+ {
3826
+ "name": "transformer_blocks.9.s_attn.qkv.bias",
3827
+ "shape": [
3828
+ 1536
3829
+ ],
3830
+ "dtype": "float32",
3831
+ "offset": 133470208,
3832
+ "byteLength": 6144
3833
+ },
3834
+ {
3835
+ "name": "transformer_blocks.9.s_attn.proj.weight",
3836
+ "shape": [
3837
+ 512,
3838
+ 512
3839
+ ],
3840
+ "dtype": "float16",
3841
+ "offset": 133476352,
3842
+ "byteLength": 524288
3843
+ },
3844
+ {
3845
+ "name": "transformer_blocks.9.s_attn.proj.bias",
3846
+ "shape": [
3847
+ 512
3848
+ ],
3849
+ "dtype": "float32",
3850
+ "offset": 134000640,
3851
+ "byteLength": 2048
3852
+ },
3853
+ {
3854
+ "name": "transformer_blocks.9.s_attn.q_norm.weight",
3855
+ "shape": [
3856
+ 64
3857
+ ],
3858
+ "dtype": "float32",
3859
+ "offset": 134002688,
3860
+ "byteLength": 256
3861
+ },
3862
+ {
3863
+ "name": "transformer_blocks.9.s_attn.k_norm.weight",
3864
+ "shape": [
3865
+ 64
3866
+ ],
3867
+ "dtype": "float32",
3868
+ "offset": 134002944,
3869
+ "byteLength": 256
3870
+ },
3871
+ {
3872
+ "name": "transformer_blocks.9.s_attn.graph_dist_embedding.weight",
3873
+ "shape": [
3874
+ 6,
3875
+ 128
3876
+ ],
3877
+ "dtype": "float32",
3878
+ "offset": 134003200,
3879
+ "byteLength": 3072
3880
+ },
3881
+ {
3882
+ "name": "transformer_blocks.9.s_attn.graph_dist_proj.weight",
3883
+ "shape": [
3884
+ 8,
3885
+ 128
3886
+ ],
3887
+ "dtype": "float32",
3888
+ "offset": 134006272,
3889
+ "byteLength": 4096
3890
+ },
3891
+ {
3892
+ "name": "transformer_blocks.9.s_attn.graph_dist_proj.bias",
3893
+ "shape": [
3894
+ 8
3895
+ ],
3896
+ "dtype": "float32",
3897
+ "offset": 134010368,
3898
+ "byteLength": 32
3899
+ },
3900
+ {
3901
+ "name": "transformer_blocks.9.s_attn.graph_rel_embedding.weight",
3902
+ "shape": [
3903
+ 6,
3904
+ 128
3905
+ ],
3906
+ "dtype": "float32",
3907
+ "offset": 134010624,
3908
+ "byteLength": 3072
3909
+ },
3910
+ {
3911
+ "name": "transformer_blocks.9.s_attn.graph_rel_proj.weight",
3912
+ "shape": [
3913
+ 8,
3914
+ 128
3915
+ ],
3916
+ "dtype": "float32",
3917
+ "offset": 134013696,
3918
+ "byteLength": 4096
3919
+ },
3920
+ {
3921
+ "name": "transformer_blocks.9.s_attn.graph_rel_proj.bias",
3922
+ "shape": [
3923
+ 8
3924
+ ],
3925
+ "dtype": "float32",
3926
+ "offset": 134017792,
3927
+ "byteLength": 32
3928
+ },
3929
+ {
3930
+ "name": "transformer_blocks.9.norm_t.weight",
3931
+ "shape": [
3932
+ 512
3933
+ ],
3934
+ "dtype": "float32",
3935
+ "offset": 134018048,
3936
+ "byteLength": 2048
3937
+ },
3938
+ {
3939
+ "name": "transformer_blocks.9.t_attn.qkv.weight",
3940
+ "shape": [
3941
+ 1536,
3942
+ 512
3943
+ ],
3944
+ "dtype": "float16",
3945
+ "offset": 134020096,
3946
+ "byteLength": 1572864
3947
+ },
3948
+ {
3949
+ "name": "transformer_blocks.9.t_attn.qkv.bias",
3950
+ "shape": [
3951
+ 1536
3952
+ ],
3953
+ "dtype": "float32",
3954
+ "offset": 135592960,
3955
+ "byteLength": 6144
3956
+ },
3957
+ {
3958
+ "name": "transformer_blocks.9.t_attn.proj.weight",
3959
+ "shape": [
3960
+ 512,
3961
+ 512
3962
+ ],
3963
+ "dtype": "float16",
3964
+ "offset": 135599104,
3965
+ "byteLength": 524288
3966
+ },
3967
+ {
3968
+ "name": "transformer_blocks.9.t_attn.proj.bias",
3969
+ "shape": [
3970
+ 512
3971
+ ],
3972
+ "dtype": "float32",
3973
+ "offset": 136123392,
3974
+ "byteLength": 2048
3975
+ },
3976
+ {
3977
+ "name": "transformer_blocks.9.t_attn.q_norm.weight",
3978
+ "shape": [
3979
+ 64
3980
+ ],
3981
+ "dtype": "float32",
3982
+ "offset": 136125440,
3983
+ "byteLength": 256
3984
+ },
3985
+ {
3986
+ "name": "transformer_blocks.9.t_attn.k_norm.weight",
3987
+ "shape": [
3988
+ 64
3989
+ ],
3990
+ "dtype": "float32",
3991
+ "offset": 136125696,
3992
+ "byteLength": 256
3993
+ },
3994
+ {
3995
+ "name": "transformer_blocks.9.norm_mlp.weight",
3996
+ "shape": [
3997
+ 512
3998
+ ],
3999
+ "dtype": "float32",
4000
+ "offset": 136125952,
4001
+ "byteLength": 2048
4002
+ },
4003
+ {
4004
+ "name": "transformer_blocks.9.mlp.w12.weight",
4005
+ "shape": [
4006
+ 2730,
4007
+ 512
4008
+ ],
4009
+ "dtype": "float16",
4010
+ "offset": 136128000,
4011
+ "byteLength": 2795520
4012
+ },
4013
+ {
4014
+ "name": "transformer_blocks.9.mlp.w12.bias",
4015
+ "shape": [
4016
+ 2730
4017
+ ],
4018
+ "dtype": "float32",
4019
+ "offset": 138923520,
4020
+ "byteLength": 10920
4021
+ },
4022
+ {
4023
+ "name": "transformer_blocks.9.mlp.w3.weight",
4024
+ "shape": [
4025
+ 512,
4026
+ 1365
4027
+ ],
4028
+ "dtype": "float16",
4029
+ "offset": 138934528,
4030
+ "byteLength": 1397760
4031
+ },
4032
+ {
4033
+ "name": "transformer_blocks.9.mlp.w3.bias",
4034
+ "shape": [
4035
+ 512
4036
+ ],
4037
+ "dtype": "float32",
4038
+ "offset": 140332288,
4039
+ "byteLength": 2048
4040
+ },
4041
+ {
4042
+ "name": "transformer_blocks.9.adaLN_modulation.1.weight",
4043
+ "shape": [
4044
+ 4608,
4045
+ 512
4046
+ ],
4047
+ "dtype": "float16",
4048
+ "offset": 140334336,
4049
+ "byteLength": 4718592
4050
+ },
4051
+ {
4052
+ "name": "transformer_blocks.9.adaLN_modulation.1.bias",
4053
+ "shape": [
4054
+ 4608
4055
+ ],
4056
+ "dtype": "float16",
4057
+ "offset": 145052928,
4058
+ "byteLength": 9216
4059
+ },
4060
+ {
4061
+ "name": "final_layer.norm_final.weight",
4062
+ "shape": [
4063
+ 512
4064
+ ],
4065
+ "dtype": "float32",
4066
+ "offset": 145062144,
4067
+ "byteLength": 2048
4068
+ },
4069
+ {
4070
+ "name": "final_layer.root_cross_attn.in_proj_weight",
4071
+ "shape": [
4072
+ 1536,
4073
+ 512
4074
+ ],
4075
+ "dtype": "float16",
4076
+ "offset": 145064192,
4077
+ "byteLength": 1572864
4078
+ },
4079
+ {
4080
+ "name": "final_layer.root_cross_attn.in_proj_bias",
4081
+ "shape": [
4082
+ 1536
4083
+ ],
4084
+ "dtype": "float32",
4085
+ "offset": 146637056,
4086
+ "byteLength": 6144
4087
+ },
4088
+ {
4089
+ "name": "final_layer.root_cross_attn.out_proj.weight",
4090
+ "shape": [
4091
+ 512,
4092
+ 512
4093
+ ],
4094
+ "dtype": "float16",
4095
+ "offset": 146643200,
4096
+ "byteLength": 524288
4097
+ },
4098
+ {
4099
+ "name": "final_layer.root_cross_attn.out_proj.bias",
4100
+ "shape": [
4101
+ 512
4102
+ ],
4103
+ "dtype": "float32",
4104
+ "offset": 147167488,
4105
+ "byteLength": 2048
4106
+ },
4107
+ {
4108
+ "name": "final_layer.root_cross_norm_q.weight",
4109
+ "shape": [
4110
+ 512
4111
+ ],
4112
+ "dtype": "float32",
4113
+ "offset": 147169536,
4114
+ "byteLength": 2048
4115
+ },
4116
+ {
4117
+ "name": "final_layer.root_cross_norm_kv.weight",
4118
+ "shape": [
4119
+ 512
4120
+ ],
4121
+ "dtype": "float32",
4122
+ "offset": 147171584,
4123
+ "byteLength": 2048
4124
+ },
4125
+ {
4126
+ "name": "final_layer.root_out.0.weight",
4127
+ "shape": [
4128
+ 512,
4129
+ 512
4130
+ ],
4131
+ "dtype": "float16",
4132
+ "offset": 147173632,
4133
+ "byteLength": 524288
4134
+ },
4135
+ {
4136
+ "name": "final_layer.root_out.0.bias",
4137
+ "shape": [
4138
+ 512
4139
+ ],
4140
+ "dtype": "float32",
4141
+ "offset": 147697920,
4142
+ "byteLength": 2048
4143
+ },
4144
+ {
4145
+ "name": "final_layer.root_out.2.weight",
4146
+ "shape": [
4147
+ 12,
4148
+ 512
4149
+ ],
4150
+ "dtype": "float16",
4151
+ "offset": 147699968,
4152
+ "byteLength": 12288
4153
+ },
4154
+ {
4155
+ "name": "final_layer.root_out.2.bias",
4156
+ "shape": [
4157
+ 12
4158
+ ],
4159
+ "dtype": "float32",
4160
+ "offset": 147712256,
4161
+ "byteLength": 48
4162
+ },
4163
+ {
4164
+ "name": "final_layer.joint_out.0.weight",
4165
+ "shape": [
4166
+ 512,
4167
+ 512
4168
+ ],
4169
+ "dtype": "float16",
4170
+ "offset": 147712512,
4171
+ "byteLength": 524288
4172
+ },
4173
+ {
4174
+ "name": "final_layer.joint_out.0.bias",
4175
+ "shape": [
4176
+ 512
4177
+ ],
4178
+ "dtype": "float32",
4179
+ "offset": 148236800,
4180
+ "byteLength": 2048
4181
+ },
4182
+ {
4183
+ "name": "final_layer.joint_out.2.weight",
4184
+ "shape": [
4185
+ 12,
4186
+ 512
4187
+ ],
4188
+ "dtype": "float16",
4189
+ "offset": 148238848,
4190
+ "byteLength": 12288
4191
+ },
4192
+ {
4193
+ "name": "final_layer.joint_out.2.bias",
4194
+ "shape": [
4195
+ 12
4196
+ ],
4197
+ "dtype": "float32",
4198
+ "offset": 148251136,
4199
+ "byteLength": 48
4200
+ },
4201
+ {
4202
+ "name": "final_layer.adaLN_modulation.1.weight",
4203
+ "shape": [
4204
+ 1024,
4205
+ 512
4206
+ ],
4207
+ "dtype": "float16",
4208
+ "offset": 148251392,
4209
+ "byteLength": 1048576
4210
+ },
4211
+ {
4212
+ "name": "final_layer.adaLN_modulation.1.bias",
4213
+ "shape": [
4214
+ 1024
4215
+ ],
4216
+ "dtype": "float32",
4217
+ "offset": 149299968,
4218
+ "byteLength": 4096
4219
+ }
4220
+ ],
4221
+ "verification": {
4222
+ "capture": "/Users/noahlyons/.local/state/kaminos/accb-unimate-reference-1012/forward-go2-seed10-r1",
4223
+ "bitwise_equal": true,
4224
+ "cond_max_abs_diff": 0.0,
4225
+ "uncond_max_abs_diff": 0.0
4226
+ },
4227
+ "storage": {
4228
+ "dtype": "float16",
4229
+ "min_elements": 4096,
4230
+ "float32_source_sha256": "4dc5d6953fd120842e39c808931c13794fe0abedc5597f1031f9393bd8b5285b",
4231
+ "note": "float16 storage, widened to float32 at load"
4232
+ }
4233
+ }
unimate-fp16/unimate-denoiser-fp16.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:9672a6414249b3f5767eb88def444682d04f83054d13b2f1f8fdc2206af6e085
3
+ size 149304064