Add Zenodo DOI 10.5281/zenodo.23448095 (badge, How to cite, CITATION.cff); report rev. 5 updated 18 Oct 2026 (public repo, consistency fixes)
Browse files- CITATION.cff +12 -1
- README.md +6 -4
- paper/cuerdas-technical-report.md +12 -9
- paper/cuerdas-technical-report.pdf +2 -2
CITATION.cff
CHANGED
|
@@ -8,6 +8,14 @@ authors:
|
|
| 8 |
affiliation: "Independent researcher"
|
| 9 |
email: "sanchezerik836@gmail.com"
|
| 10 |
version: "2026-10-16-rev5"
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 11 |
license:
|
| 12 |
- MIT
|
| 13 |
- CC-BY-4.0
|
|
@@ -45,7 +53,10 @@ preferred-citation:
|
|
| 45 |
year: 2026
|
| 46 |
month: 10
|
| 47 |
status: preprint
|
| 48 |
-
|
|
|
|
|
|
|
|
|
|
| 49 |
url: "https://huggingface.co/Eriksanchezolvera/cuerdas"
|
| 50 |
license: CC-BY-4.0
|
| 51 |
references:
|
|
|
|
| 8 |
affiliation: "Independent researcher"
|
| 9 |
email: "sanchezerik836@gmail.com"
|
| 10 |
version: "2026-10-16-rev5"
|
| 11 |
+
doi: "10.5281/zenodo.23448095"
|
| 12 |
+
identifiers:
|
| 13 |
+
- type: doi
|
| 14 |
+
value: "10.5281/zenodo.23448095"
|
| 15 |
+
description: "Zenodo DOI of the technical report"
|
| 16 |
+
- type: url
|
| 17 |
+
value: "https://huggingface.co/Eriksanchezolvera/cuerdas"
|
| 18 |
+
description: "Code, logs, sealed tests and checkpoints"
|
| 19 |
license:
|
| 20 |
- MIT
|
| 21 |
- CC-BY-4.0
|
|
|
|
| 53 |
year: 2026
|
| 54 |
month: 10
|
| 55 |
status: preprint
|
| 56 |
+
doi: "10.5281/zenodo.23448095"
|
| 57 |
+
publisher:
|
| 58 |
+
name: "Zenodo"
|
| 59 |
+
notes: "Technical Report, revision 5 (updated 18 October 2026). Not peer reviewed."
|
| 60 |
url: "https://huggingface.co/Eriksanchezolvera/cuerdas"
|
| 61 |
license: CC-BY-4.0
|
| 62 |
references:
|
README.md
CHANGED
|
@@ -15,7 +15,9 @@ tags:
|
|
| 15 |
|
| 16 |
# Semantic Strands ("Cuerdas"): a pre-registered negative result on learned soft-slot context compression
|
| 17 |
|
| 18 |
-
|
|
|
|
|
|
|
| 19 |
Erik Sanchez, independent researcher Β· sanchezerik836@gmail.com
|
| 20 |
|
| 21 |
> **ES.** Les presento los resultados de mi primer reto personal en ciencia computacional. En este punto mi conocimiento todavΓa es nulo, pero lo compenso con una gran curiosidad. Espero que les sea de utilidad.
|
|
@@ -109,9 +111,9 @@ AI systems are not authors. Grok Bot (AI assistant) helped design the experiment
|
|
| 109 |
|
| 110 |
## How to cite
|
| 111 |
|
| 112 |
-
> Sanchez, E. (2026). *Semantic Strands: A Pre-registered Negative Result on Learned Soft-Slot Context Compression β Trained Compact Text Wins.* Technical Report, revision 5. https://
|
| 113 |
|
| 114 |
-
DOI:
|
| 115 |
|
| 116 |
```bibtex
|
| 117 |
@techreport{sanchez2026cuerdas,
|
|
@@ -125,7 +127,7 @@ DOI: pending
|
|
| 125 |
year = {2026},
|
| 126 |
month = oct,
|
| 127 |
url = {https://huggingface.co/Eriksanchezolvera/cuerdas},
|
| 128 |
-
|
| 129 |
}
|
| 130 |
```
|
| 131 |
|
|
|
|
| 15 |
|
| 16 |
# Semantic Strands ("Cuerdas"): a pre-registered negative result on learned soft-slot context compression
|
| 17 |
|
| 18 |
+
[](https://doi.org/10.5281/zenodo.23448095)
|
| 19 |
+
|
| 20 |
+
**Technical Report, Revision 5 (16 October 2026; updated 18 October 2026). Not peer reviewed.** DOI: [10.5281/zenodo.23448095](https://doi.org/10.5281/zenodo.23448095)
|
| 21 |
Erik Sanchez, independent researcher Β· sanchezerik836@gmail.com
|
| 22 |
|
| 23 |
> **ES.** Les presento los resultados de mi primer reto personal en ciencia computacional. En este punto mi conocimiento todavΓa es nulo, pero lo compenso con una gran curiosidad. Espero que les sea de utilidad.
|
|
|
|
| 111 |
|
| 112 |
## How to cite
|
| 113 |
|
| 114 |
+
> Sanchez, E. (2026). *Semantic Strands: A Pre-registered Negative Result on Learned Soft-Slot Context Compression β Trained Compact Text Wins.* Technical Report, revision 5. Zenodo. https://doi.org/10.5281/zenodo.23448095
|
| 115 |
|
| 116 |
+
DOI: [10.5281/zenodo.23448095](https://doi.org/10.5281/zenodo.23448095)
|
| 117 |
|
| 118 |
```bibtex
|
| 119 |
@techreport{sanchez2026cuerdas,
|
|
|
|
| 127 |
year = {2026},
|
| 128 |
month = oct,
|
| 129 |
url = {https://huggingface.co/Eriksanchezolvera/cuerdas},
|
| 130 |
+
doi = {10.5281/zenodo.23448095}
|
| 131 |
}
|
| 132 |
```
|
| 133 |
|
paper/cuerdas-technical-report.md
CHANGED
|
@@ -1,6 +1,8 @@
|
|
| 1 |
# Semantic Strands: A Pre-registered Negative Result on Learned Soft-Slot Context Compression β Trained Compact Text Wins
|
| 2 |
|
| 3 |
-
**Technical Report. Revision 5, 16 October 2026. Not peer reviewed.**
|
|
|
|
|
|
|
| 4 |
|
| 5 |
**Erik Sanchez**, independent researcher Β· <sanchezerik836@gmail.com>
|
| 6 |
|
|
@@ -17,7 +19,7 @@ We report a negative result so that others can skip this path. We trained a smal
|
|
| 17 |
**What failed.**
|
| 18 |
|
| 19 |
- On a sealed test with unseen words, names, phrases and multi-token values, accuracy falls to 29% with four unseen values, against 98% for the template. Retraining with a much larger, more varied replacement pool did not help (+2.6 pp, interval includes zero).
|
| 20 |
-
- With a matched trainable budget (a reparameterized soft prompt of 3.84M parameters, same data, steps, loss, teacher and validation-only selection), compact text wins. Values separated by commas score 100% on every sealed set, including four unseen values, at 22 positions per event plus 8 shareable prompt positions. Against this baseline the slots save only 2β5 positions per event and lose up to
|
| 21 |
- Probes trained with replacement words read unseen values from the slots better than the frozen LLM does (for example 97.7% vs 52.6% with two unseen values). The information is in the slots; the receiver does not read it.
|
| 22 |
|
| 23 |
Zero-shot compact text fails with this receiver (0β8%), which is why comparing only against zero-shot or verbose text overstates the benefit of soft slots. **Our main recommendation: before claiming compression from learned soft slots, compare against compact text with a matched trainable budget (for example a soft prompt or prefix of equal size), on a sealed test that includes unseen values.**
|
|
@@ -274,7 +276,7 @@ Kaggle Tesla T4 GPU time, as logged:
|
|
| 274 |
| 019 (re-evaluation only) | β3 min |
|
| 275 |
| 020 (sealed test, baselines, probes; no encoder training) | β6 min |
|
| 276 |
|
| 277 |
-
The 019 re-evaluation, with no training, took about 3 min of T4 time, plus a few minutes for a diagnostic run. The total is about 2.6 GPU-hours. In addition, 013, the 014 audit and the 015 pilot ran on a shared CPU machine. Energy and VRAM were not measured.
|
| 278 |
|
| 279 |
---
|
| 280 |
|
|
@@ -481,7 +483,7 @@ Rule 3 is met: no-LLM decoders match or beat the frozen LLM on the same slots (r
|
|
| 481 |
|
| 482 |
Soft prompt + values with commas minus K=2 (paired bootstrap, seed means): +1.33 pp [+0.58, +2.17] on sealed-CH, +18.2 [+14.4, +22.4] with one unseen value, +47.4 [+41.7, +53.1] with two and **+70.7 [+64.9, +76.2]** with four.
|
| 483 |
|
| 484 |
-
**Pre-registered rule 2 is not met: K=2 keeps no advantage.** With the same trainable budget, a soft prompt in front of a comma-separated value list is perfect on every sealed set at 22 positions per event (+8 shared), and values only reach 100% on known values and 93β97% with four unseen values at 19 positions. Against this fair baseline, K=2 saves only 2β5 positions per event (β11% to β23%, not β51%) and loses up to 70 pp on unseen values. The 51% saving reported earlier is a saving against a verbose template, not against competent compact text. Untrained few-shot prompts do not close the gap, which is why the zero-shot comparisons of Section 5.6 were misleading.
|
| 485 |
|
| 486 |
|
| 487 |
---
|
|
@@ -633,11 +635,12 @@ The following ten experiments are the external reviewer's list of requirements,
|
|
| 633 |
|
| 634 |
**Code, models and software.**
|
| 635 |
|
| 636 |
-
- **Repository:** <https://huggingface.co/Eriksanchezolvera/cuerdas> (
|
| 637 |
- per-iteration scripts (`run015.py` β¦ `run018.py`) and the exact Kaggle kernels (`kaggle_kernel.py`);
|
| 638 |
- raw logs (`results01X.jsonl`, including the 018 validation curves) and result notes;
|
| 639 |
- the 014 audit script;
|
| 640 |
- the checkpoints `binder4_1000_s*.pt` (015), `k1_s*.pt` (017 K=1) and `k2long_s*.pt` (018 K=2);
|
|
|
|
| 641 |
- the `LICENSE` file (MIT).
|
| 642 |
- **Software:** `transformers==4.46.3` and PyTorch.
|
| 643 |
- The 018 run did not log its PyTorch version; its log shows Python 3.13.
|
|
@@ -659,7 +662,7 @@ The following ten experiments are the external reviewer's list of requirements,
|
|
| 659 |
1. Regenerate the dataset: `python make_dataset.py --out $SRC/data/expanded.json`. The script uses only the standard library.
|
| 660 |
2. Download the receiver at the revision above to `$MP`.
|
| 661 |
3. Run `SRC=β¦ MP=β¦ OD=out python 015/run015.py base binder4:1000`, then `016/run016.py`, `017/run017.py` and `018/run018.py` with `INP=checkpoints`.
|
| 662 |
-
- **Compute:** about 2.
|
| 663 |
- **Dataset:** the original `expanded.json`, produced in iteration 014 with ChatGPT, is **not redistributed**. Instead, `make_dataset.py` re-implements the documented 014 protocol:
|
| 664 |
- the vocabulary;
|
| 665 |
- exclusion of the 013 universe;
|
|
@@ -670,7 +673,7 @@ The following ten experiments are the external reviewer's list of requirements,
|
|
| 670 |
- **Per-example predictions and re-evaluation:** `019/eval019.py`, its Kaggle kernel, `019/results019.jsonl` and `019/predictions019.jsonl`.
|
| 671 |
- **Iteration 020:** `020/preregistro020.md` (pre-registration), `020/make_sealed020.py` and `020/sealed020.json` (SHA-256 `4c7737f66364a44786b4d9acb4686a6f31829d78cb5914f19eca1df74fe73393`), `020/eval020.py`, `020/results020.jsonl` and `020/predictions020.jsonl`.
|
| 672 |
|
| 673 |
-
**Still to do
|
| 674 |
|
| 675 |
---
|
| 676 |
|
|
@@ -702,7 +705,7 @@ We thank **Kaggle** for the free GPU time (Tesla T4) that made iterations 015β
|
|
| 702 |
|
| 703 |
## 14. How to Cite
|
| 704 |
|
| 705 |
-
> Sanchez, E. (2026). *Semantic Strands: A Pre-registered Negative Result on Learned Soft-Slot Context Compression β Trained Compact Text Wins.* Technical Report, revision 5. <https://
|
| 706 |
|
| 707 |
```bibtex
|
| 708 |
@techreport{sanchez2026cuerdas,
|
|
@@ -716,7 +719,7 @@ We thank **Kaggle** for the free GPU time (Tesla T4) that made iterations 015β
|
|
| 716 |
year = {2026},
|
| 717 |
month = oct,
|
| 718 |
url = {https://huggingface.co/Eriksanchezolvera/cuerdas},
|
| 719 |
-
|
| 720 |
}
|
| 721 |
```
|
| 722 |
|
|
|
|
| 1 |
# Semantic Strands: A Pre-registered Negative Result on Learned Soft-Slot Context Compression β Trained Compact Text Wins
|
| 2 |
|
| 3 |
+
**Technical Report. Revision 5, 16 October 2026 (updated 18 October 2026: DOI, public repository and two consistency fixes; no results changed). Not peer reviewed.**
|
| 4 |
+
|
| 5 |
+
**DOI:** [10.5281/zenodo.23448095](https://doi.org/10.5281/zenodo.23448095) Β· **Repository:** <https://huggingface.co/Eriksanchezolvera/cuerdas>
|
| 6 |
|
| 7 |
**Erik Sanchez**, independent researcher Β· <sanchezerik836@gmail.com>
|
| 8 |
|
|
|
|
| 19 |
**What failed.**
|
| 20 |
|
| 21 |
- On a sealed test with unseen words, names, phrases and multi-token values, accuracy falls to 29% with four unseen values, against 98% for the template. Retraining with a much larger, more varied replacement pool did not help (+2.6 pp, interval includes zero).
|
| 22 |
+
- With a matched trainable budget (a reparameterized soft prompt of 3.84M parameters, same data, steps, loss, teacher and validation-only selection), compact text wins. Values separated by commas score 100% on every sealed set, including four unseen values, at 22 positions per event plus 8 shareable prompt positions. Against this baseline the slots save only 2β5 positions per event and lose up to 70.7 pp.
|
| 23 |
- Probes trained with replacement words read unseen values from the slots better than the frozen LLM does (for example 97.7% vs 52.6% with two unseen values). The information is in the slots; the receiver does not read it.
|
| 24 |
|
| 25 |
Zero-shot compact text fails with this receiver (0β8%), which is why comparing only against zero-shot or verbose text overstates the benefit of soft slots. **Our main recommendation: before claiming compression from learned soft slots, compare against compact text with a matched trainable budget (for example a soft prompt or prefix of equal size), on a sealed test that includes unseen values.**
|
|
|
|
| 276 |
| 019 (re-evaluation only) | β3 min |
|
| 277 |
| 020 (sealed test, baselines, probes; no encoder training) | β6 min |
|
| 278 |
|
| 279 |
+
The 019 re-evaluation, with no training, took about 3 min of T4 time, plus a few minutes for a diagnostic run. The total for iterations 015β020 is about 2.6 GPU-hours. In addition, 013, the 014 audit and the 015 pilot ran on a shared CPU machine. Energy and VRAM were not measured.
|
| 280 |
|
| 281 |
---
|
| 282 |
|
|
|
|
| 483 |
|
| 484 |
Soft prompt + values with commas minus K=2 (paired bootstrap, seed means): +1.33 pp [+0.58, +2.17] on sealed-CH, +18.2 [+14.4, +22.4] with one unseen value, +47.4 [+41.7, +53.1] with two and **+70.7 [+64.9, +76.2]** with four.
|
| 485 |
|
| 486 |
+
**Pre-registered rule 2 is not met: K=2 keeps no advantage.** With the same trainable budget, a soft prompt in front of a comma-separated value list is perfect on every sealed set at 22 positions per event (+8 shared), and values only reach 100% on known values and 93β97% with four unseen values at 19 positions. Against this fair baseline, K=2 saves only 2β5 positions per event (β11% to β23%, not β51%) and loses up to 70.7 pp on unseen values. The 51% saving reported earlier is a saving against a verbose template, not against competent compact text. Untrained few-shot prompts do not close the gap, which is why the zero-shot comparisons of Section 5.6 were misleading.
|
| 487 |
|
| 488 |
|
| 489 |
---
|
|
|
|
| 635 |
|
| 636 |
**Code, models and software.**
|
| 637 |
|
| 638 |
+
- **Repository:** <https://huggingface.co/Eriksanchezolvera/cuerdas> (**public**). It contains:
|
| 639 |
- per-iteration scripts (`run015.py` β¦ `run018.py`) and the exact Kaggle kernels (`kaggle_kernel.py`);
|
| 640 |
- raw logs (`results01X.jsonl`, including the 018 validation curves) and result notes;
|
| 641 |
- the 014 audit script;
|
| 642 |
- the checkpoints `binder4_1000_s*.pt` (015), `k1_s*.pt` (017 K=1) and `k2long_s*.pt` (018 K=2);
|
| 643 |
+
- the iteration 020 and 021 files (pre-registrations, sealed test sets, evaluation scripts, logs and per-example predictions) and the 021 checkpoints;
|
| 644 |
- the `LICENSE` file (MIT).
|
| 645 |
- **Software:** `transformers==4.46.3` and PyTorch.
|
| 646 |
- The 018 run did not log its PyTorch version; its log shows Python 3.13.
|
|
|
|
| 662 |
1. Regenerate the dataset: `python make_dataset.py --out $SRC/data/expanded.json`. The script uses only the standard library.
|
| 663 |
2. Download the receiver at the revision above to `$MP`.
|
| 664 |
3. Run `SRC=β¦ MP=β¦ OD=out python 015/run015.py base binder4:1000`, then `016/run016.py`, `017/run017.py` and `018/run018.py` with `INP=checkpoints`.
|
| 665 |
+
- **Compute:** about 2.6 T4 GPU-hours in total for 015β020 (Section 4.7), on Kaggle's free tier.
|
| 666 |
- **Dataset:** the original `expanded.json`, produced in iteration 014 with ChatGPT, is **not redistributed**. Instead, `make_dataset.py` re-implements the documented 014 protocol:
|
| 667 |
- the vocabulary;
|
| 668 |
- exclusion of the 013 universe;
|
|
|
|
| 673 |
- **Per-example predictions and re-evaluation:** `019/eval019.py`, its Kaggle kernel, `019/results019.jsonl` and `019/predictions019.jsonl`.
|
| 674 |
- **Iteration 020:** `020/preregistro020.md` (pre-registration), `020/make_sealed020.py` and `020/sealed020.json` (SHA-256 `4c7737f66364a44786b4d9acb4686a6f31829d78cb5914f19eca1df74fe73393`), `020/eval020.py`, `020/results020.jsonl` and `020/predictions020.jsonl`.
|
| 675 |
|
| 676 |
+
**Still to do.** Pass this report through the NeurIPS Paper Checklist [13] and the ML Reproducibility Checklist [14]. A model card [15] has been added to the repository.
|
| 677 |
|
| 678 |
---
|
| 679 |
|
|
|
|
| 705 |
|
| 706 |
## 14. How to Cite
|
| 707 |
|
| 708 |
+
> Sanchez, E. (2026). *Semantic Strands: A Pre-registered Negative Result on Learned Soft-Slot Context Compression β Trained Compact Text Wins.* Technical Report, revision 5. Zenodo. <https://doi.org/10.5281/zenodo.23448095>
|
| 709 |
|
| 710 |
```bibtex
|
| 711 |
@techreport{sanchez2026cuerdas,
|
|
|
|
| 719 |
year = {2026},
|
| 720 |
month = oct,
|
| 721 |
url = {https://huggingface.co/Eriksanchezolvera/cuerdas},
|
| 722 |
+
doi = {10.5281/zenodo.23448095}
|
| 723 |
}
|
| 724 |
```
|
| 725 |
|
paper/cuerdas-technical-report.pdf
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
-
size
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:82d84539a81ad389e33d9ece7a3bf7ebf0196b65eac5acbca9fe7db6e8faf4d3
|
| 3 |
+
size 182028
|