Add model config and update loading instructions
Browse files- README.md +9 -11
- config.json +15 -0
README.md
CHANGED
|
@@ -23,14 +23,15 @@ Given any audio file, the model returns a single scalar speech quality score on
|
|
| 23 |
```
|
| 24 |
OmniASR-W2V-300M (encoder, fine-tuned with reduced lr)
|
| 25 |
β
|
| 26 |
-
AttentiveStatsPooling
|
| 27 |
β
|
| 28 |
-
Linear(
|
| 29 |
β
|
| 30 |
MOS score (scalar)
|
| 31 |
```
|
| 32 |
|
| 33 |
-
- **Encoder**: `omniASR_W2V_300M` β a 300M-parameter Wav2Vec2
|
|
|
|
| 34 |
- **Pooling**: Attentive Statistics Pooling β computes attention-weighted mean and standard deviation over the time dimension.
|
| 35 |
- **Head**: Two-layer MLP with GELU activation and a scalar output.
|
| 36 |
|
|
@@ -40,10 +41,11 @@ Linear(1536 β 1024) β GELU β Linear(1024 β 1)
|
|
| 40 |
|
| 41 |
| File | Description |
|
| 42 |
|---|---|
|
| 43 |
-
| `
|
|
|
|
| 44 |
| `inference_model.py` | Model architecture + inference script |
|
| 45 |
|
| 46 |
-
> **Note:**
|
| 47 |
|
| 48 |
---
|
| 49 |
|
|
@@ -95,14 +97,10 @@ MOS: 3.8721
|
|
| 95 |
|
| 96 |
```python
|
| 97 |
import torch
|
| 98 |
-
from inference_model import
|
| 99 |
|
| 100 |
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
|
| 101 |
-
|
| 102 |
-
encoder, _ = load_checkpoint("omniASR-W2V-300M.pt")
|
| 103 |
-
model = OmniMOS(encoder=encoder)
|
| 104 |
-
model.load_state_dict(torch.load("best_model_full.pt", map_location="cpu"))
|
| 105 |
-
model.to(device).eval()
|
| 106 |
|
| 107 |
score = predict_mos(model, "/path/to/audio.wav", device)
|
| 108 |
print(f"MOS: {score:.4f}")
|
|
|
|
| 23 |
```
|
| 24 |
OmniASR-W2V-300M (encoder, fine-tuned with reduced lr)
|
| 25 |
β
|
| 26 |
+
AttentiveStatsPooling -> [mean || std] (dim: 2 x 1024 = 2048)
|
| 27 |
β
|
| 28 |
+
Linear(2048 -> 1024) -> GELU -> Linear(1024 -> 1)
|
| 29 |
β
|
| 30 |
MOS score (scalar)
|
| 31 |
```
|
| 32 |
|
| 33 |
+
- **Encoder**: `omniASR_W2V_300M` β a 300M-parameter Wav2Vec2-style encoder instantiated via `fairseq2`.
|
| 34 |
+
- **Weights**: `best_model_full.pt` contains the trained ProxyMOS state dict, including the encoder and MOS head weights. No separate `omniASR-W2V-300M.pt` checkpoint is required.
|
| 35 |
- **Pooling**: Attentive Statistics Pooling β computes attention-weighted mean and standard deviation over the time dimension.
|
| 36 |
- **Head**: Two-layer MLP with GELU activation and a scalar output.
|
| 37 |
|
|
|
|
| 41 |
|
| 42 |
| File | Description |
|
| 43 |
|---|---|
|
| 44 |
+
| `config.json` | Model metadata used by Hugging Face Hub tooling |
|
| 45 |
+
| `best_model_full.pt` | Trained ProxyMOS state dict, including encoder and MOS head weights (~1.28 GB) |
|
| 46 |
| `inference_model.py` | Model architecture + inference script |
|
| 47 |
|
| 48 |
+
> **Note:** inference uses `fairseq2` to instantiate the `omniASR_W2V_300M` architecture, then loads all trained weights from `best_model_full.pt`. You do not need a separate `omniASR-W2V-300M.pt` file.
|
| 49 |
|
| 50 |
---
|
| 51 |
|
|
|
|
| 97 |
|
| 98 |
```python
|
| 99 |
import torch
|
| 100 |
+
from inference_model import load_model, predict_mos
|
| 101 |
|
| 102 |
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
|
| 103 |
+
model = load_model("best_model_full.pt", device)
|
|
|
|
|
|
|
|
|
|
|
|
|
| 104 |
|
| 105 |
score = predict_mos(model, "/path/to/audio.wav", device)
|
| 106 |
print(f"MOS: {score:.4f}")
|
config.json
ADDED
|
@@ -0,0 +1,15 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"model_type": "proxymos",
|
| 3 |
+
"architectures": [
|
| 4 |
+
"OmniMOS"
|
| 5 |
+
],
|
| 6 |
+
"library_name": "pytorch",
|
| 7 |
+
"format": "pytorch_state_dict",
|
| 8 |
+
"weights": "best_model_full.pt",
|
| 9 |
+
"inference_script": "inference_model.py",
|
| 10 |
+
"sample_rate": 16000,
|
| 11 |
+
"encoder": "omniASR_W2V_300M",
|
| 12 |
+
"hidden_size": 1024,
|
| 13 |
+
"pooling": "attentive_statistics",
|
| 14 |
+
"output": "mos_score"
|
| 15 |
+
}
|