Upload README.md with huggingface_hub
Browse files
README.md
ADDED
|
@@ -0,0 +1,37 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
private: true
|
| 3 |
+
tags: [polyplex, personaplex, korean, full-duplex]
|
| 4 |
+
---
|
| 5 |
+
|
| 6 |
+
# PolyPlex Korean, Phase 1.0 and 1.5 adapters
|
| 7 |
+
|
| 8 |
+
Backup of the two Phase 1 stages of PolyPlex, which adds Korean to a frozen PersonaPlex 7B from text only.
|
| 9 |
+
These are adapter slices, not full models: each holds the appended Korean text rows and the session expert
|
| 10 |
+
(a correction on the FFNs of layers 8 to 31, rank 256). They load onto the vocabulary-expanded base
|
| 11 |
+
checkpoint (`expanded_v2`, 32,000 + 10,448 tokens) with the PolyPlex code
|
| 12 |
+
(`polyplex_research.utils.assemble.Phase1Assemble.from_run`).
|
| 13 |
+
|
| 14 |
+
| Folder | Stage | Trained on | Code |
|
| 15 |
+
|---|---|---|---|
|
| 16 |
+
| `phase1.0/` | Dense text (no PAD frames), 26,000 steps | About 510M tokens of Korean web text (HAERAE KOREAN-WEBTEXT) and Korean Wikipedia | branch `experiment/two-stage-dense-duplex` (commit 2e00113) |
|
| 17 |
+
| `phase1.5/` | Duplex dialogue layout with codec silence, 6,000 steps, started from `phase1.0` | 384,000 of 602,677 dialogue samples (AI Hub multisession dialogue 011 and 141, colloquial translation sentences) | same branch |
|
| 18 |
+
|
| 19 |
+
Each folder holds the final adapter (`config.json`, `text_rows.safetensors`) and the last trainer checkpoint
|
| 20 |
+
(`checkpoint-26000/`, `checkpoint-6000/`) with optimizer, scheduler and RNG state for resuming.
|
| 21 |
+
|
| 22 |
+
## Results of `phase1.5/` (silent condition unless stated)
|
| 23 |
+
|
| 24 |
+
| Measure | Value |
|
| 25 |
+
|---|---|
|
| 26 |
+
| Korean word loss, 011 validation (8,348 utterances, training sentences removed) | 4.136 (PPL 62.6) |
|
| 27 |
+
| Korean word loss, 141 dialogue validation (2,000 dialogues) | 3.738 |
|
| 28 |
+
| Same-meaning comparison, 200 pairs: Korean / English (original model) | 4.067 / 3.835 |
|
| 29 |
+
| Timing cost on 011 (PAD/EPAD loss over the original model) | +0.056 |
|
| 30 |
+
| Korean word loss with real speech (KsponSpeech validation) | 9.322 |
|
| 31 |
+
|
| 32 |
+
Known limits: Korean does not carry over to the condition where the assistant channel holds real speech, and
|
| 33 |
+
with the session expert on the depth decoder's code prediction is broken (assistant code loss 7.87 against
|
| 34 |
+
4.38 with the expert off), so this adapter does not speak Korean. The full records are the ResearchTree pull
|
| 35 |
+
requests #17 (this stage) and #18, #19, #20 (later attempts) of `latentforge/PolyPlex`.
|
| 36 |
+
|
| 37 |
+
The training data licenses have not been reviewed for redistribution, which is why this repository is private.
|