arpanpathak commited on
Commit
83035b2
·
verified ·
1 Parent(s): b5948a7

Rewrite the model card for the v0.1.1 release

Browse files
Files changed (1) hide show
  1. README.md +37 -37
README.md CHANGED
@@ -15,31 +15,42 @@ pipeline_tag: text-to-audio
15
 
16
  # StaleyNET checkpoints
17
 
18
- Weights for **StaleyNET**, a text-prompted music generator trained from random
19
- initialization in Rust on NVIDIA edge GPUs. No pretrained network, codec,
20
- vocoder, or text encoder is used: the Transformer, the residual vector
21
- quantizer, the Griffin-Lim vocoder, and every GPU kernel are implemented in the
22
- project.
23
 
24
- The code, paper, and book live in the companion GitHub repository:
25
  <https://github.com/arpanpathak/staleynet>.
26
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
27
  ## Checkpoints
28
 
29
  | Run | Purpose | Parameters | `best` step | `latest` step | Training tracks |
30
  |---|---|---:|---:|---:|---:|
31
- | `thor-base` | base text-to-music model | 7,099,904 | 0 | — | 667 |
32
  | `smoke` | pipeline smoke test | 7,099,904 | 0 | 1,500 | 667 |
33
  | `overfit-one-take` | single-take memorisation check | 7,099,904 | 200 | 200 | 1 |
34
  | `vocal-scratch` | vocal-filtered model from scratch | 7,099,904 | 5,500 | 6,000 | 281 |
35
- | `vocal-finetune` | vocal fine-tune | 7,099,904 | 250 | — | 281 |
36
  | `lyrics-orin` | lyrics-conditioned model | 7,165,440 | 3,750 | 4,000 | 543 (73 with lyrics) |
37
 
38
- The step numbers are exactly what each `model.json` records. `best/` is the
39
- weight-only checkpoint selected by validation; `latest/` is the resumable
40
- checkpoint with AdamW moments. Some early runs stored only `best/`, and
41
- `thor-base` / `smoke` record step 0 in their `best/model.json`; where a
42
- `latest/` exists, prefer it for the late training state.
43
 
44
  ## Layout
45
 
@@ -52,10 +63,7 @@ metrics/ full metrics.jsonl for every run
52
  logs/ captured training logs
53
  ```
54
 
55
- `tensors.bin` is little-endian `f32` in parameter-visitor order. `latest/`
56
- appends the AdamW first moments and then the second moments; `model.json`
57
- records the tensor manifest, the step, whether optimizer state is present, and
58
- the training configuration.
59
 
60
  ## Download and use
61
 
@@ -82,29 +90,23 @@ staleynet sing \
82
  --out song.wav
83
  ```
84
 
85
- Architecture: width 256, 8 heads, 6 layers, feed-forward multiplier 4, context
86
- length 768, 4 residual codebooks of 1024 entries, 128 mel bands at 24 kHz.
87
- `lyrics-orin` adds a 256-byte lyrics field.
88
 
89
  ## Training data and licensing
90
 
91
- - **Song Describer Dataset** (SDD), Zenodo record
92
- [10072001](https://zenodo.org/records/10072001): 706 Jamendo recordings
93
- with human captions. Captions are CC BY-SA 4.0; the recordings carry the
94
- per-track Creative Commons licences in `audio_licenses.txt`, including
95
- CC BY-NC-ND and CC BY-NC-SA.
96
- - **JamendoLyrics MultiLang**, <https://github.com/f90/jamendolyrics>: code and
97
- metadata are MIT; songs and lyrics carry the per-song CC licences in
98
- `JamendoLyrics.csv`.
99
 
100
- Because the training audio includes NonCommercial and NoDerivatives tracks,
101
- these checkpoints are **research artifacts, not a commercial product**. Do not
102
- use them commercially where the upstream recording licences forbid it, and do
103
- not use them to impersonate a singer without consent.
104
 
105
- The checkpoint files are offered under CC BY-SA 4.0
106
- (`LICENSE-CC-BY-SA-4.0.txt`) to the extent the upstream dataset licences allow.
107
- See `NOTICE` for the full provenance statement and `SHA256SUMS` for integrity.
 
 
 
 
 
108
 
109
  ## Citation
110
 
@@ -117,5 +119,3 @@ See `NOTICE` for the full provenance statement and `SHA256SUMS` for integrity.
117
  url = {https://github.com/arpanpathak/staleynet}
118
  }
119
  ```
120
-
121
- No warranty. This is a research checkpoint, not a production model.
 
15
 
16
  # StaleyNET checkpoints
17
 
18
+ Weights for **StaleyNET**, a text-prompted music generator trained from random initialization in Rust on NVIDIA edge GPUs. The Transformer, residual vector quantizer, Griffin-Lim vocoder, and GPU kernels are implemented in the project.
 
 
 
 
19
 
20
+ The code, paper, and book live in the companion repository:
21
  <https://github.com/arpanpathak/staleynet>.
22
 
23
+ ## Release v0.1.1
24
+
25
+ - GitHub release: <https://github.com/arpanpathak/staleynet/releases/tag/v0.1.1>
26
+ - `staleynet-audio` 0.1.1 on crates.io: <https://crates.io/crates/staleynet-audio>
27
+ - Test status: 171 tests pass, including 18 GPU integration tests on a Jetson Orin.
28
+ - CI: rustfmt, clippy with warnings denied, cargo-deny, and the audio coverage gate.
29
+
30
+ ## At a glance
31
+
32
+ | Item | Value |
33
+ |---|---|
34
+ | Checkpointed runs | 6 |
35
+ | Parameters | 7,099,904 for text-only runs; 7,165,440 for the lyrics run |
36
+ | Transformer | width 256, 8 heads, 6 layers, context 768 |
37
+ | Audio tokens | 4 residual codebooks, 1024 entries each |
38
+ | Audio features | 128 mel bands at 24 kHz, FFT 1024, hop 256 |
39
+ | Training data | Song Describer Dataset and JamendoLyrics MultiLang |
40
+ | License | CC BY-SA 4.0, subject to the upstream dataset terms |
41
+
42
  ## Checkpoints
43
 
44
  | Run | Purpose | Parameters | `best` step | `latest` step | Training tracks |
45
  |---|---|---:|---:|---:|---:|
46
+ | `thor-base` | base text-to-music model | 7,099,904 | 0 | None | 667 |
47
  | `smoke` | pipeline smoke test | 7,099,904 | 0 | 1,500 | 667 |
48
  | `overfit-one-take` | single-take memorisation check | 7,099,904 | 200 | 200 | 1 |
49
  | `vocal-scratch` | vocal-filtered model from scratch | 7,099,904 | 5,500 | 6,000 | 281 |
50
+ | `vocal-finetune` | vocal fine-tune | 7,099,904 | 250 | None | 281 |
51
  | `lyrics-orin` | lyrics-conditioned model | 7,165,440 | 3,750 | 4,000 | 543 (73 with lyrics) |
52
 
53
+ The step values are exactly what each `model.json` records. `best/` is the weight-only checkpoint selected by validation. `latest/` is the resumable checkpoint with AdamW moments. Some early runs stored only `best/`, and `thor-base` and `smoke` record step 0 in their `best/model.json`. Where a `latest/` exists, prefer it for the late training state.
 
 
 
 
54
 
55
  ## Layout
56
 
 
63
  logs/ captured training logs
64
  ```
65
 
66
+ `tensors.bin` is little-endian `f32` in parameter-visitor order. `latest/` appends the AdamW first moments and then the second moments. `model.json` records the tensor manifest, the step, whether optimizer state is present, and the training configuration.
 
 
 
67
 
68
  ## Download and use
69
 
 
90
  --out song.wav
91
  ```
92
 
93
+ Architecture: width 256, 8 heads, 6 layers, feed-forward multiplier 4, context length 768, 4 residual codebooks of 1024 entries, 128 mel bands at 24 kHz. `lyrics-orin` adds a 256-byte lyrics field.
 
 
94
 
95
  ## Training data and licensing
96
 
97
+ - **Song Describer Dataset** (SDD), Zenodo record [10072001](https://zenodo.org/records/10072001): 706 Jamendo recordings with human captions. Captions are CC BY-SA 4.0. The recordings carry the per-track Creative Commons licenses in `audio_licenses.txt`, including CC BY-NC-ND and CC BY-NC-SA.
98
+ - **JamendoLyrics MultiLang**, <https://github.com/f90/jamendolyrics>: code and metadata are MIT. Songs and lyrics carry the per-song CC licenses in `JamendoLyrics.csv`.
 
 
 
 
 
 
99
 
100
+ Because the training audio includes NonCommercial and NoDerivatives tracks, these checkpoints are research artifacts, not a commercial product. Do not use them commercially where the upstream recording licenses forbid it, and do not use them to impersonate a singer without consent.
 
 
 
101
 
102
+ The checkpoint files are offered under CC BY-SA 4.0 (`LICENSE-CC-BY-SA-4.0.txt`) to the extent the upstream dataset licenses allow. See `NOTICE` for the full provenance statement and `SHA256SUMS` for integrity.
103
+
104
+ ## Known limitations
105
+
106
+ - Griffin-Lim produces phasey audio compared with a learned vocoder.
107
+ - The models are small edge models, not production music systems.
108
+ - The lyric models were trained on a small lyrics subset, so lyric intelligibility is limited.
109
+ - The checkpoints are research artifacts and have no warranty.
110
 
111
  ## Citation
112
 
 
119
  url = {https://github.com/arpanpathak/staleynet}
120
  }
121
  ```