STEMMA-MF2601

Paper-final Music Flamingo + STEMMA-Instruct: V7 coreference + relation MultiQA 1x (no fullsong/block component), full fine-tuning with trainable audio encoder, global batch 128, one epoch, LR 1e-5. Final step 431; base revision 6b5be086d52f65a1e204cb0faf70bf54e2741ecd. Four BF16 weight shards are at the repository root and indexed by model.safetensors.index.json.

Status: isolated installation, 17 CPU tests and 24 full-checkpoint B200 requests passed. Input token hashes match the sealed evaluation records; one parsed answer differs from vLLM in this acceptance sample. See VALIDATION.md. This is not paper score reproduction. This repository contains the complete fine-tuned weights and editable native Transformers/PyTorch inference code, not a LoRA adapter.

Minimal use and editable features

python -m pip install -r requirements.txt
python examples/generate.py --model . --audio a.wav b.wav \
  --prompt 'Audio A: <audio>\nAudio B: <audio>\nCompare A and B.'
python examples/embeddings.py --model . --audio a.wav b.wav

Use literal newlines in Python prompts; a shell single-quoted \n stays a backslash sequence, although marker order still remains valid. Download a pinned commit from hoyso48/STEMMA-MF2601 with huggingface_hub.snapshot_download, then run these examples against its local folder. No Swift/base-model cache is needed.

from huggingface_hub import HfApi, snapshot_download
import sys

repo = "hoyso48/STEMMA-MF2601"
revision = HfApi().model_info(repo).sha  # Record this commit for reproduction.
folder = snapshot_download(repo, revision=revision)
sys.path.insert(0, folder)  # The downloaded repository includes the stemma code.
from stemma import load_model
session = load_model(folder)

from stemma import Request, load_model
session = load_model(".")
answers = session.generate([Request("Compare A and B.", ["a.wav", "b.wav"])])
batch = session.prepare([Request("Describe this source.", ["a.wav"])])
audio_features = session.extract_audio_embeddings(batch)
# Editable session.model and session.processor support custom forward hooks.

See API.md for tensor shapes, order/masks, unpooled encoder/projected/contextual features, request batching and generation profiles. Audio slots remain independent; RoTE resets per source, not across the whole list. Decode to mono 16 kHz through SoundFile/SciPy, retain tails, use separate 30-second windows capped at 20 minutes per source. Context is 32,768 tokens. Generic sampled and greedy reference profiles are separate; backend/RNG differences are documented rather than hidden. An instance-only zero-token-tail RoTE fix preserves the full waveform/mel while avoiding upstream indexing of a trailing window that contributes no output tokens.

Read NOTICE.md and the complete NVIDIA license before use. MF weights and derivatives are academic noncommercial only, not unrestricted commercial use. New wrapper/example code is Apache-2.0 (LICENSE); that code license does not change the weight license. Stock vLLM serving is not supported by this package; the paper used a separately patched backend described in API.md. Model training data are not distributed here. release_info.json, provenance.json and checksums.json identify weights, source adaptations and staged assets.