SmolVLM2-500M-Video-Instruct-a16w8 for SiMa.ai Modalix

Compiled LLiMa runtime artifacts for images, multiple images, and caller-sampled video frames. This repository contains devkit/ assets and elf_files/, ready to copy to a Modalix model directory.

Setting	Value
Source	HuggingFaceTB/SmolVLM2-500M-Video-Instruct
Source revision	7b375e1b73b11138ff12fe22c8f2822d8fe03467
Weight precision	INT8 for vision, group/prefill, and single-token/decode
Activations	BF16
Context length	4,096 tokens, including input and output
Prefill group size	128 tokens
Vision input	512 × 512 RGB; 64 visual tokens per image
Embeddings and KV cache	Quantization enabled
Compiler	Model SDK 3.0.0; LLiMa commit 3f07d804c72260b6a1116cb56cf5dd5e9c3a8b69
Compiled units	175: 12 vision, 81 group, and 82 single-token

Runtime requirement and validation

Requires an LLiMa runtime containing SmolVLM2 support, PR #323. Compilation completed and all 175 deployed ELF headers were verified. Modalix inference and model quality have not yet been validated for this artifact.

Download and run

After installing a compatible LLiMa runtime on Modalix:

hf download florianvoss/SmolVLM2-500M-Video-Instruct-a16w8 \
    --local-dir /media/nvme/llima/models/SmolVLM2-500M-Video-Instruct-a16w8
llima run SmolVLM2-500M-Video-Instruct-a16w8 --mode cli

In the CLI, use add image /path/to/image.jpg, then enter an image-grounded question. Sampled video frames can be supplied through the existing multi-image structured-message API, interleaved with timestamp text.

Images are padded to square with the checkpoint mean color and resized with OpenCV Lanczos-4, preserving aspect ratio. Each image/frame produces one global visual block. Crop splitting and video-file decoding are outside this artifact's input path; OpenCV resizing differs from the upstream PIL processor.

Reproduce

The source checkpoint is uncompressed safetensors; INT8 quantization is performed by the Model SDK during compilation. config.py selects A_BF16_W_INT8 for all units. With the matching compiler and LLiMa revision installed:

hf download HuggingFaceTB/SmolVLM2-500M-Video-Instruct \
    --revision 7b375e1b73b11138ff12fe22c8f2822d8fe03467 --local-dir source-model
bash compile.sh source-model compiled-model
llima-deploy compiled-model deployed-model

See compilation.json for provenance and SHA256SUMS for payload checksums. The source checkpoint is licensed under Apache 2.0; its source license is included as LICENSE.source. A compatible SiMa.ai runtime is required to execute the compiled artifacts.