faxenoff's picture
coreml_ane: W8A16 embed package (int8 weights, fp16 activations) — passed a 384-query paired gate, 96 -> 73 MB
ee119ed verified
Raw History Blame Contribute Delete
397 Bytes
{
"head": "mean",
"output_dim": 768,
"per_token": false,
"shapes": [
{
"batch": 96,
"seq": 48,
"function": "b96_s48"
},
{
"batch": 128,
"seq": 64,
"function": "b128_s64"
},
{
"batch": 128,
"seq": 80,
"function": "b128_s80"
},
{
"batch": 256,
"seq": 128,
"function": "b256_s128"
}
]
}