PyTorch
Russian
English
proxymos
audio
mos
korallll commited on
Commit
8a90bda
Β·
verified Β·
1 Parent(s): 434be28

Add model config and update loading instructions

Browse files
Files changed (2) hide show
  1. README.md +9 -11
  2. config.json +15 -0
README.md CHANGED
@@ -23,14 +23,15 @@ Given any audio file, the model returns a single scalar speech quality score on
23
  ```
24
  OmniASR-W2V-300M (encoder, fine-tuned with reduced lr)
25
  ↓
26
- AttentiveStatsPooling β†’ [mean β€– std] (dim: 2 Γ— 768 = 1536)
27
  ↓
28
- Linear(1536 β†’ 1024) β†’ GELU β†’ Linear(1024 β†’ 1)
29
  ↓
30
  MOS score (scalar)
31
  ```
32
 
33
- - **Encoder**: `omniASR_W2V_300M` β€” a 300M-parameter Wav2Vec2 model from the `omnilingual_asr` package, pretrained on 1600+ languages. The encoder is not frozen during training; it is updated with a learning rate 100Γ— smaller than the head.
 
34
  - **Pooling**: Attentive Statistics Pooling β€” computes attention-weighted mean and standard deviation over the time dimension.
35
  - **Head**: Two-layer MLP with GELU activation and a scalar output.
36
 
@@ -40,10 +41,11 @@ Linear(1536 β†’ 1024) β†’ GELU β†’ Linear(1024 β†’ 1)
40
 
41
  | File | Description |
42
  |---|---|
43
- | `best_model_full.pt` | Trained ProxyMOS weights (state_dict, ~1.28 GB) |
 
44
  | `inference_model.py` | Model architecture + inference script |
45
 
46
- > **Note:** encoder weights (`omniASR-W2V-300M.pt`) are **not stored** in this repository β€” they are downloaded separately via `omnilingual_asr` (see below).
47
 
48
  ---
49
 
@@ -95,14 +97,10 @@ MOS: 3.8721
95
 
96
  ```python
97
  import torch
98
- from inference_model import OmniMOS, load_checkpoint, predict_mos
99
 
100
  device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
101
-
102
- encoder, _ = load_checkpoint("omniASR-W2V-300M.pt")
103
- model = OmniMOS(encoder=encoder)
104
- model.load_state_dict(torch.load("best_model_full.pt", map_location="cpu"))
105
- model.to(device).eval()
106
 
107
  score = predict_mos(model, "/path/to/audio.wav", device)
108
  print(f"MOS: {score:.4f}")
 
23
  ```
24
  OmniASR-W2V-300M (encoder, fine-tuned with reduced lr)
25
  ↓
26
+ AttentiveStatsPooling -> [mean || std] (dim: 2 x 1024 = 2048)
27
  ↓
28
+ Linear(2048 -> 1024) -> GELU -> Linear(1024 -> 1)
29
  ↓
30
  MOS score (scalar)
31
  ```
32
 
33
+ - **Encoder**: `omniASR_W2V_300M` β€” a 300M-parameter Wav2Vec2-style encoder instantiated via `fairseq2`.
34
+ - **Weights**: `best_model_full.pt` contains the trained ProxyMOS state dict, including the encoder and MOS head weights. No separate `omniASR-W2V-300M.pt` checkpoint is required.
35
  - **Pooling**: Attentive Statistics Pooling β€” computes attention-weighted mean and standard deviation over the time dimension.
36
  - **Head**: Two-layer MLP with GELU activation and a scalar output.
37
 
 
41
 
42
  | File | Description |
43
  |---|---|
44
+ | `config.json` | Model metadata used by Hugging Face Hub tooling |
45
+ | `best_model_full.pt` | Trained ProxyMOS state dict, including encoder and MOS head weights (~1.28 GB) |
46
  | `inference_model.py` | Model architecture + inference script |
47
 
48
+ > **Note:** inference uses `fairseq2` to instantiate the `omniASR_W2V_300M` architecture, then loads all trained weights from `best_model_full.pt`. You do not need a separate `omniASR-W2V-300M.pt` file.
49
 
50
  ---
51
 
 
97
 
98
  ```python
99
  import torch
100
+ from inference_model import load_model, predict_mos
101
 
102
  device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
103
+ model = load_model("best_model_full.pt", device)
 
 
 
 
104
 
105
  score = predict_mos(model, "/path/to/audio.wav", device)
106
  print(f"MOS: {score:.4f}")
config.json ADDED
@@ -0,0 +1,15 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "model_type": "proxymos",
3
+ "architectures": [
4
+ "OmniMOS"
5
+ ],
6
+ "library_name": "pytorch",
7
+ "format": "pytorch_state_dict",
8
+ "weights": "best_model_full.pt",
9
+ "inference_script": "inference_model.py",
10
+ "sample_rate": 16000,
11
+ "encoder": "omniASR_W2V_300M",
12
+ "hidden_size": 1024,
13
+ "pooling": "attentive_statistics",
14
+ "output": "mos_score"
15
+ }