Metal: cover every quant type upstream's metallib has a gemv for
Marc Sun
0becb0a
The type table was four entries; the metallib always held kernels for 22. The gap was host-side:
CUDA reaches upstream's `mul_mat_vec_q_switch_type`, one function that switches over every type,
while ggml's Metal type->pipeline mapping lives in ggml-metal-device.cpp, which pulls in the whole
backend and is not compiled here. So each type has to be registered by hand.
Transcribed from ggml_metal_library_get_pipeline_mul_mv: the legacy quants, the K quants, the IQ
quants and MXFP4 -- 20 types, parity with the CUDA port apart from NVFP4/Q1_0/Q2_0, which the Metal
metallib has no kernels for. The one field not available as a macro is the grid: ggml-metal-ops.cpp
dispatches the dense types and q8_0 on (ne01 + nr0 - 1)/nr0 and everything else divides by nsg too,
so among quantized types q8_0 stays the only `mv_reduce_across_sgs`.
`GEMV_TYPES` becomes a `gemv_types()` op each backend answers for itself. It was one hardcoded CUDA
list, which Metal inherited and over-claimed by 19 types -- a …
