RubriX-8B / rubrix_config.json
balacbx's picture
Raise the input cap to 32,768 tokens; skip the KV cache when embedding
c07a846 verified
Raw History Blame Contribute Delete
337 Bytes
{
"base_model": "Qwen/Qwen3-8B",
"base_revision": "b968826d9c46dd6066d109eabc6255188de91218",
"max_tokens": 32768,
"heads": {
"hidden_size": 4096,
"width": 1536,
"depth": 3,
"projection_dim": 512,
"layernorm": true,
"activation": "gelu",
"max_scale": 100.0
},
"pooling": "last token, L2-normalised; no EOS appended"
}