MarktHart eustlb HF Staff commited on
Commit
a21c97c
·
1 Parent(s): 5d7cb99

Simplify Transformers usage snippet (#2)

Browse files

- Simplify Transformers usage snippet (0b4f80b77cacc42041808b6c0ca412cec4cac0d1)
- Use AutoModelForMultimodalLM (f15413b3bee1dd390a7d18a6017964cf3c9f8bed)
- Use hf.co short URL for the sample audio (62c9ce36bf96e4d49c60176889fb1bc0689b7395)
- Rely on the checkpoint dtype (bfloat16) by default (b436e35bbb1cf066541c88501977c68df2a01eb6)
- Move language auto-detection note into a code comment (7d056179e79825d34315bc28a55d4df558dd4c28)
- Move return_format note into a code comment (74dc78acacacef965d24dfee365849cab065a6a7)
- Clarify return_format comment (74cef837fe6d2f3d79a9082b6888732846060f5e)
- Shorten return_format comment (1f683bef7118fff9f8941bc62c1ecef2cb3c4686)
- Inline return_format comment (23844ce1a5ab8e75894ad5ce8c8748879e01c528)


Co-authored-by: Eustache Le Bihan <eustlb@users.noreply.huggingface.co>

Files changed (1) hide show
  1. README.md +8 -14
README.md CHANGED
@@ -64,33 +64,27 @@ Speech recognition model with native Hugging Face Transformers and vLLM support.
64
  ## Usage (Transformers)
65
 
66
  ```bash
67
- uv pip install torch librosa transformers
68
  ```
69
 
70
  ```python
71
- import librosa
72
- import torch
73
- from transformers import AutoProcessor, Qwen3ASRForConditionalGeneration
74
 
75
  model_id = "Reson8/Rumble-2-Flash"
76
  processor = AutoProcessor.from_pretrained(model_id)
77
- model = Qwen3ASRForConditionalGeneration.from_pretrained(
78
- model_id, dtype=torch.bfloat16
79
- ).to("cuda")
80
 
81
- audio, _ = librosa.load("audio.wav", sr=16000)
82
  inputs = processor.apply_transcription_request(
83
- audio=audio, language="English", return_tensors="pt"
 
84
  ).to(model.device, dtype=model.dtype)
85
 
86
- with torch.inference_mode():
87
- output = model.generate(**inputs, max_new_tokens=512)
88
-
89
  generated = output[:, inputs["input_ids"].shape[-1]:]
90
- print(processor.batch_decode(generated, skip_special_tokens=True)[0])
91
  ```
92
 
93
- Omit `language` to detect it automatically.
94
 
95
  ## Usage (vLLM)
96
 
 
64
  ## Usage (Transformers)
65
 
66
  ```bash
67
+ uv pip install torch transformers accelerate
68
  ```
69
 
70
  ```python
71
+ from transformers import AutoProcessor, AutoModelForMultimodalLM
 
 
72
 
73
  model_id = "Reson8/Rumble-2-Flash"
74
  processor = AutoProcessor.from_pretrained(model_id)
75
+ model = AutoModelForMultimodalLM.from_pretrained(model_id, device_map="auto")
 
 
76
 
 
77
  inputs = processor.apply_transcription_request(
78
+ audio="https://hf.co/datasets/hf-internal-testing/dummy-audio-samples/resolve/main/bcn_weather.mp3",
79
+ language="English", # omit to detect automatically
80
  ).to(model.device, dtype=model.dtype)
81
 
82
+ output = model.generate(**inputs, max_new_tokens=512)
 
 
83
  generated = output[:, inputs["input_ids"].shape[-1]:]
84
+ print(processor.decode(generated, return_format="transcription_only")[0]) # return_format="parsed" if detecting language
85
  ```
86
 
87
+ `audio` accepts a local path, a URL, a NumPy array, or a list of these.
88
 
89
  ## Usage (vLLM)
90