Xylolabs/SenseVoiceSmall-Tenstorrent-Tensix-Precision-Components

SenseVoiceSmall precision alternatives for Tenstorrent Tensix

tenstorrent · tensix · speech-recognition · safetensors · bfloat4_b · bfloat8_b · bfloat16 · model-components

Existing projection and head alternatives derived from SenseVoiceSmall, decoded into portable safetensors by Xylolabs with all saved values preserved. The three groups belong to the same BF16-first preparation route; an older BFP8 component archive has different rounded matrix values.

This export needs about 0.84 GB of device memory for its component weights alone, plus room for working buffers.

Scope

This partial archive contains 280 BFP4 encoder projections, 281 BFP8 encoder-projection/CTC-head operands and 281 BF16 projection/head alternatives. Its source profile has 128 feature-input rows, with four fixed Korean/event/emotion/ITN-on query rows. To retain the saved BFP4 profile, use the BFP8 CTC head supplied here. Architecture integration, compatible shared operands, preprocessing and token decoding are required.

The companion archive provides shared normalization, bias, FSMN/tap and static operands. Its BFP8 matrices came through a different rounding route; use the alternatives in this archive when preserving the BF16-first route. A mixed selection needs its own integrated recognizer evaluation.

Formats

Component	Tensors	Native interpretation	Portable file bytes
bf4_projections	280	BFP4 round-to-nearest after BF16; 4.5 packed bits/value	440,576,912
bf8_projections	281	BFP8 round-to-nearest after BF16; 8.5 packed bits/value	466,233,312
bf16_projections	281	BF16	466,233,312

All three portable files store BF16. BFP4 and BFP8 files retain already-quantized values exactly; their download sizes do not describe packed density. Counting all alternatives together, native tiled data totals 837,872,640 bytes. An integration selects compatible alternatives and need not keep all full groups resident.

Matrices retain their saved [in, out] operator order. Query scaling is folded in already. component.json records each hash, shape, stored dtype and native-format label.

Quickstart

Install PyTorch, safetensors and huggingface_hub, then load one component at a time:

from pathlib import Path
import importlib.util
from huggingface_hub import snapshot_download

folder = Path(snapshot_download("Xylolabs/SenseVoiceSmall-Tenstorrent-Tensix-Precision-Components", allow_patterns=[
    "component.json", "load_components.py", "bf4_projections.safetensors",
    "README.md", "LICENSE", "NOTICE", "CODE_LICENSE",
]))
spec = importlib.util.spec_from_file_location("sensevoice_precision", folder / "load_components.py")
components = importlib.util.module_from_spec(spec)
spec.loader.exec_module(components)
count = sum(1 for name, tensor in components.iter_component(folder, "bf4_projections"))
print(count)

This checks the hash, names, shapes, declared dtype and finite values of 280 matrices while streaming. Load bf8_projections or bf16_projections and the corresponding file for their 281 projection/head alternatives. The example does not run speech recognition. Integration must preserve saved matrix order, folded query scaling, compatible shared operands and the selected native packing format.

Evaluation

All 842 saved tensors are finite, decoded without changing values, and repack exactly through TT-NN 0.81.1 in a simulator context. The original ONNX matches its primary Hugging Face LFS identity, its 919 source arrays match a fresh reconstruction, and all 842 alternatives exactly match the existing query-scaling and BF16-first BFP4/BFP8 or BF16 transformations. Transport checksums cover every original cache file. The streaming public example and all three descriptor groups are checked.

These are representation and provenance checks. An integrated recognizer requires speech evaluation for its selected operands; no task score is assigned to the standalone archive. Speed is not reported here.

License

Weights retain the full FunASR Model Open Source License Agreement v1.1, as linked by the original SenseVoiceSmall release. It permits copying, modification and sharing under its terms, including source/author attribution and model-name retention. Reference/learning-purpose, community-conduct, termination and revision provisions remain in the agreement. NOTICE identifies FunAudioLLM, Alibaba Group, the sherpa-onnx source and our modifications. Only the Xylolabs loader uses Apache 2.0 (CODE_LICENSE). Provided as is, without warranty.

Citation

Cite the original SenseVoiceSmall release and the sherpa-onnx export.