docs: document fp16 single-file + precise two-path usage
Browse files
README.md
CHANGED
|
@@ -72,9 +72,20 @@ CTC models are **non-autoregressive** — they decode a whole clip in one forwar
|
|
| 72 |
|
| 73 |
FP16 and FP32 produce **identical transcripts**, so FP16 is the recommended default (half the VRAM, ~2× faster). Batched throughput on the 4090 reaches **~208× real time** (the 222-min FLEURS set transcribes in ~64 s). A **16 GB GPU** (e.g. Colab T4 / L4) is enough for fp16. CPU works but is slow.
|
| 74 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 75 |
## Usage
|
| 76 |
|
| 77 |
-
Install the
|
| 78 |
|
| 79 |
```bash
|
| 80 |
# system dependency for audio I/O
|
|
@@ -87,27 +98,34 @@ pip install torch==2.8.0 torchaudio==2.8.0 torchvision==0.23.0 --index-url https
|
|
| 87 |
# On Colab/Jupyter, restart the runtime once after this install.
|
| 88 |
```
|
| 89 |
|
| 90 |
-
|
| 91 |
|
| 92 |
```python
|
| 93 |
import pathlib, torch
|
| 94 |
-
from huggingface_hub import
|
| 95 |
|
| 96 |
-
#
|
| 97 |
-
|
| 98 |
|
| 99 |
-
#
|
|
|
|
|
|
|
|
|
|
|
|
|
| 100 |
asset_dir = pathlib.Path.home() / ".config/fairseq2/assets"
|
| 101 |
asset_dir.mkdir(parents=True, exist_ok=True)
|
| 102 |
(asset_dir / "rade.yaml").write_text(f"""\
|
| 103 |
name: rade_CTC_3B_fa
|
| 104 |
model_family: wav2vec2_asr
|
| 105 |
model_arch: 3b_v2
|
| 106 |
-
checkpoint: "{
|
| 107 |
tokenizer_ref: omniASR_tokenizer_written_v2
|
| 108 |
""")
|
|
|
|
| 109 |
|
| 110 |
-
|
|
|
|
|
|
|
| 111 |
from omnilingual_asr.models.inference.pipeline import ASRInferencePipeline
|
| 112 |
pipe = ASRInferencePipeline(
|
| 113 |
model_card="rade_CTC_3B_fa",
|
|
|
|
| 72 |
|
| 73 |
FP16 and FP32 produce **identical transcripts**, so FP16 is the recommended default (half the VRAM, ~2× faster). Batched throughput on the 4090 reaches **~208× real time** (the 222-min FLEURS set transcribes in ~64 s). A **16 GB GPU** (e.g. Colab T4 / L4) is enough for fp16. CPU works but is slow.
|
| 74 |
|
| 75 |
+
## Files in this repo
|
| 76 |
+
|
| 77 |
+
| File | What it is | Size | When to use |
|
| 78 |
+
|---|---|---|---|
|
| 79 |
+
| **`model_fp16.pt`** | Consolidated **fp16** weights, single file | **~6.2 GB** | **Recommended.** Smaller, faster download; fp16 inference. |
|
| 80 |
+
| `pp_00/tp_00/sdp_00.pt`, `sdp_01.pt` | Original **fp32** FSDP checkpoint shards | ~12 GB | If you want full fp32 precision weights. |
|
| 81 |
+
| `config.json` | Model metadata (arch, tokenizer, vocab) | — | Read by tooling; you don't load it directly. |
|
| 82 |
+
| `inference_colab.ipynb` | Ready-to-run Colab notebook | — | One-click demo. |
|
| 83 |
+
|
| 84 |
+
> Both weight files produce **identical transcripts** at fp16. The single `model_fp16.pt` is just half the download — prefer it unless you specifically need the fp32 master weights.
|
| 85 |
+
|
| 86 |
## Usage
|
| 87 |
|
| 88 |
+
**1) Install the Omnilingual ASR runtime** (the model loads through fairseq2's asset system):
|
| 89 |
|
| 90 |
```bash
|
| 91 |
# system dependency for audio I/O
|
|
|
|
| 98 |
# On Colab/Jupyter, restart the runtime once after this install.
|
| 99 |
```
|
| 100 |
|
| 101 |
+
**2) Download the weights + register a fairseq2 asset card** that points at them, reusing Omnilingual ASR's official Persian-capable tokenizer. The asset card's `checkpoint:` can point at **either** the single `model_fp16.pt` file **or** the shard directory — both work.
|
| 102 |
|
| 103 |
```python
|
| 104 |
import pathlib, torch
|
| 105 |
+
from huggingface_hub import hf_hub_download
|
| 106 |
|
| 107 |
+
# --- Recommended: download ONLY the single fp16 file (~6.2 GB) ---
|
| 108 |
+
ckpt = hf_hub_download("RadeAI/Rade-ASR-CTC-3B-fa", "model_fp16.pt")
|
| 109 |
|
| 110 |
+
# --- Alternative: full fp32 shards (~12 GB) ---
|
| 111 |
+
# from huggingface_hub import snapshot_download
|
| 112 |
+
# ckpt = snapshot_download("RadeAI/Rade-ASR-CTC-3B-fa", allow_patterns=["pp_00/tp_00/*"])
|
| 113 |
+
|
| 114 |
+
# Register a fairseq2 asset card pointing at the downloaded checkpoint
|
| 115 |
asset_dir = pathlib.Path.home() / ".config/fairseq2/assets"
|
| 116 |
asset_dir.mkdir(parents=True, exist_ok=True)
|
| 117 |
(asset_dir / "rade.yaml").write_text(f"""\
|
| 118 |
name: rade_CTC_3B_fa
|
| 119 |
model_family: wav2vec2_asr
|
| 120 |
model_arch: 3b_v2
|
| 121 |
+
checkpoint: "{ckpt}"
|
| 122 |
tokenizer_ref: omniASR_tokenizer_written_v2
|
| 123 |
""")
|
| 124 |
+
```
|
| 125 |
|
| 126 |
+
**3) Transcribe** (clips must be **< 40 s**):
|
| 127 |
+
|
| 128 |
+
```python
|
| 129 |
from omnilingual_asr.models.inference.pipeline import ASRInferencePipeline
|
| 130 |
pipe = ASRInferencePipeline(
|
| 131 |
model_card="rade_CTC_3B_fa",
|