RadeAI commited on
Commit
9bebf48
·
verified ·
1 Parent(s): 34b232c

docs: document fp16 single-file + precise two-path usage

Browse files
Files changed (1) hide show
  1. README.md +26 -8
README.md CHANGED
@@ -72,9 +72,20 @@ CTC models are **non-autoregressive** — they decode a whole clip in one forwar
72
 
73
  FP16 and FP32 produce **identical transcripts**, so FP16 is the recommended default (half the VRAM, ~2× faster). Batched throughput on the 4090 reaches **~208× real time** (the 222-min FLEURS set transcribes in ~64 s). A **16 GB GPU** (e.g. Colab T4 / L4) is enough for fp16. CPU works but is slow.
74
 
 
 
 
 
 
 
 
 
 
 
 
75
  ## Usage
76
 
77
- Install the official Omnilingual ASR runtime (the model loads through fairseq2's asset system):
78
 
79
  ```bash
80
  # system dependency for audio I/O
@@ -87,27 +98,34 @@ pip install torch==2.8.0 torchaudio==2.8.0 torchvision==0.23.0 --index-url https
87
  # On Colab/Jupyter, restart the runtime once after this install.
88
  ```
89
 
90
- Load **this** fine-tuned checkpoint by registering a small custom asset card that points fairseq2 at the weights in this repo, while reusing Omnilingual ASR's official Persian-capable tokenizer:
91
 
92
  ```python
93
  import pathlib, torch
94
- from huggingface_hub import snapshot_download
95
 
96
- # 1) Download this repo's weights (FSDP shards under pp_00/tp_00/)
97
- local_dir = snapshot_download("RadeAI/Rade-ASR-CTC-3B-fa")
98
 
99
- # 2) Register a fairseq2 asset card pointing at the local checkpoint dir
 
 
 
 
100
  asset_dir = pathlib.Path.home() / ".config/fairseq2/assets"
101
  asset_dir.mkdir(parents=True, exist_ok=True)
102
  (asset_dir / "rade.yaml").write_text(f"""\
103
  name: rade_CTC_3B_fa
104
  model_family: wav2vec2_asr
105
  model_arch: 3b_v2
106
- checkpoint: "{local_dir}"
107
  tokenizer_ref: omniASR_tokenizer_written_v2
108
  """)
 
109
 
110
- # 3) Transcribe (clips must be < 40 s)
 
 
111
  from omnilingual_asr.models.inference.pipeline import ASRInferencePipeline
112
  pipe = ASRInferencePipeline(
113
  model_card="rade_CTC_3B_fa",
 
72
 
73
  FP16 and FP32 produce **identical transcripts**, so FP16 is the recommended default (half the VRAM, ~2× faster). Batched throughput on the 4090 reaches **~208× real time** (the 222-min FLEURS set transcribes in ~64 s). A **16 GB GPU** (e.g. Colab T4 / L4) is enough for fp16. CPU works but is slow.
74
 
75
+ ## Files in this repo
76
+
77
+ | File | What it is | Size | When to use |
78
+ |---|---|---|---|
79
+ | **`model_fp16.pt`** | Consolidated **fp16** weights, single file | **~6.2 GB** | **Recommended.** Smaller, faster download; fp16 inference. |
80
+ | `pp_00/tp_00/sdp_00.pt`, `sdp_01.pt` | Original **fp32** FSDP checkpoint shards | ~12 GB | If you want full fp32 precision weights. |
81
+ | `config.json` | Model metadata (arch, tokenizer, vocab) | — | Read by tooling; you don't load it directly. |
82
+ | `inference_colab.ipynb` | Ready-to-run Colab notebook | — | One-click demo. |
83
+
84
+ > Both weight files produce **identical transcripts** at fp16. The single `model_fp16.pt` is just half the download — prefer it unless you specifically need the fp32 master weights.
85
+
86
  ## Usage
87
 
88
+ **1) Install the Omnilingual ASR runtime** (the model loads through fairseq2's asset system):
89
 
90
  ```bash
91
  # system dependency for audio I/O
 
98
  # On Colab/Jupyter, restart the runtime once after this install.
99
  ```
100
 
101
+ **2) Download the weights + register a fairseq2 asset card** that points at them, reusing Omnilingual ASR's official Persian-capable tokenizer. The asset card's `checkpoint:` can point at **either** the single `model_fp16.pt` file **or** the shard directory — both work.
102
 
103
  ```python
104
  import pathlib, torch
105
+ from huggingface_hub import hf_hub_download
106
 
107
+ # --- Recommended: download ONLY the single fp16 file (~6.2 GB) ---
108
+ ckpt = hf_hub_download("RadeAI/Rade-ASR-CTC-3B-fa", "model_fp16.pt")
109
 
110
+ # --- Alternative: full fp32 shards (~12 GB) ---
111
+ # from huggingface_hub import snapshot_download
112
+ # ckpt = snapshot_download("RadeAI/Rade-ASR-CTC-3B-fa", allow_patterns=["pp_00/tp_00/*"])
113
+
114
+ # Register a fairseq2 asset card pointing at the downloaded checkpoint
115
  asset_dir = pathlib.Path.home() / ".config/fairseq2/assets"
116
  asset_dir.mkdir(parents=True, exist_ok=True)
117
  (asset_dir / "rade.yaml").write_text(f"""\
118
  name: rade_CTC_3B_fa
119
  model_family: wav2vec2_asr
120
  model_arch: 3b_v2
121
+ checkpoint: "{ckpt}"
122
  tokenizer_ref: omniASR_tokenizer_written_v2
123
  """)
124
+ ```
125
 
126
+ **3) Transcribe** (clips must be **< 40 s**):
127
+
128
+ ```python
129
  from omnilingual_asr.models.inference.pipeline import ASRInferencePipeline
130
  pipe = ASRInferencePipeline(
131
  model_card="rade_CTC_3B_fa",