Spaces:
Running on Zero
A newer version of the Gradio SDK is available: 6.21.0
title: Embedding Quantization CUDA Control
emoji: 🔬
colorFrom: blue
colorTo: yellow
sdk: gradio
sdk_version: 6.5.1
app_file: app.py
pinned: false
license: apache-2.0
Embedding Quantization CUDA Control
A bounded ZeroGPU control lane for the matched local MLX embedding quantization experiment. It reproduces the frozen Qwen3-Embedding-0.6B BF16 vectors on CUDA and compares them with the saved MLX BF16 vectors. CUDA results do not reproduce MLX/Metal performance.
The second milestone adds CUDA-native bitsandbytes INT8 and NF4 controls. They are deliberately reported as separate quantizers and are not treated as equivalents of MLX Q, oQ, or oQe formats.
Matched BF16, INT8, and NF4 controls are available for Qwen3-Embedding-0.6B, GTE-Qwen2-1.5B, and Qwen3-Embedding-8B. Quantized CUDA vectors are evaluated against their same-loading-path CUDA BF16 reference.
GTE-Qwen2-1.5B failed the loading-path and MLX/CUDA parity checks. Its CUDA quantized results remain valid only as same-path comparisons against the direct CUDA BF16 reference and must not be mixed into cross-runtime vector trends.
Frozen inputs, saved vectors, raw metrics, locked model revisions, and the
portable client are published separately in
TiGa-RCE/embedding-quant-repro-2026-07-28.