FlexiSLM-7B-Stage3

Stage 3 checkpoint of FlexiSLM-7B, a spoken language model with dynamic and controllable frame rates on both speech input and output. This release continues from Stage 2 with full-parameter fine-tuning (LoRA merged) and corresponds to training checkpoint 15000.

• Paper: arXiv:2606.31247
• Demo: flexislm.github.io
• Code: AmphionTeam/FlexiSLM
• Sibling checkpoints: FlexiSLM-7B-Stage2 (default for faster inference), FlexiSLM-0_5B-Stage2

Quick start (auto-download)

Install from the code repo, then run with auto_download=True and checkpoint="stage3_7B". On first run this downloads this Stage 3 checkpoint plus the shared Qwen2.5-Omni audio encoder, SenseVoice, FlexiCodec, flow-matching decoder, and vocoder into models/.

git clone --recurse-submodules https://github.com/AmphionTeam/FlexiSLM.git
cd FlexiSLM
pip install -r requirements.txt

from pathlib import Path

import soundfile as sf
import torch

from src.inference_flexislm import FlexiSLMInferenceConfig, FlexiSLMInference

config = FlexiSLMInferenceConfig(
    auto_download=True,
    checkpoint="stage3_7B",  # this repo; default remains stage2_7B for speed
    use_flow_matching_decoder=True,
    flow_matching_prompt_audio_path=str(Path("examples/input.wav").resolve()),
    enable_flexible_framerate=True,
    default_input_framerate=8.0,
    default_output_framerate=8.0,
    torch_dtype="bfloat16",
    attn_implementation="flash_attention_2",
)
engine = FlexiSLMInference(config, device="cuda:0")


def save_audio(result, output_path):
    waveform = result.get("audio")
    if waveform is None:
        raise RuntimeError("The model did not return decoded audio")
    if torch.is_tensor(waveform):
        waveform = waveform.detach().float().cpu().numpy()
    sf.write(Path(output_path), waveform.squeeze(), 24_000)


# Text-to-speech
result = engine.generate_tts(
    sentence="FlexiSLM supports controllable speech generation.",
    framerate=8.0,
)
save_audio(result, "tts.wav")

# ASR
result = engine.generate_from_audio(
    audio_path="examples/input.wav",
    text_query="Please transcribe the audio.",
    framerate=8.0,
    output_text_only=True,
)
print(result["text"])

# Audio QA
result = engine.generate_from_audio(
    audio_path="examples/question.wav",
    text_query="",
    framerate=8.0,
    output_text_only=True,
)
print(result["text"])

# Speech-to-speech
result = engine.generate_from_audio(
    audio_path="examples/input.wav",
    text_query="",
    framerate=8.0,
    output_text_only=False,
)
save_audio(result, "s2s.wav")

Manual download

MODEL_ROOT="$PWD/models"

hf download FlexiSLM/FlexiSLM-7B-Stage3 --local-dir "$MODEL_ROOT/FlexiSLM-7B-Stage3"

# Shared auxiliary files (required for inference)
hf download FlexiSLM/Qwen2_5-Omni-Audio_Encoder --local-dir "$MODEL_ROOT/Qwen2_5-Omni-Audio_Encoder"
hf download FunAudioLLM/SenseVoiceSmall --local-dir "$MODEL_ROOT/SenseVoiceSmall"
hf download jiaqili3/flexicodec \
  12hz_v1_half_config.yaml \
  nartts_flexicodec_only.safetensors \
  nartts.safetensors \
  --local-dir "$MODEL_ROOT/FlexiCodec"
hf download amphion/dualcodec-tts vocos_emilia.safetensors \
  --local-dir "$MODEL_ROOT/FlexiCodec"

Then point FlexiSLMInferenceConfig at the local paths (checkpoint="stage3_7B", model_path=models/FlexiSLM-7B-Stage3, plus encoder/codec paths). See the code README Inference section for the full config block.

Batch inference

After downloading weights, use the committed examples:

python -m src.infer examples/infer_7b.yaml

Set engine.config.checkpoint: stage3_7B and engine.config.model_path: models/FlexiSLM-7B-Stage3, or use auto_download: true with checkpoint: stage3_7B.

Controllable frame rate

A single Stage 3 model can be steered between about 12.5 Hz and 4.0 Hz without retraining via input_framerate / default_framerate (or the YAML default_input_framerate / default_output_framerate fields). See the code README for details.