Eriksanchezolvera commited on
Commit
81bb1aa
Β·
verified Β·
1 Parent(s): 7420c83

Add Zenodo DOI 10.5281/zenodo.23448095 (badge, How to cite, CITATION.cff); report rev. 5 updated 18 Oct 2026 (public repo, consistency fixes)

Browse files
CITATION.cff CHANGED
@@ -8,6 +8,14 @@ authors:
8
  affiliation: "Independent researcher"
9
  email: "sanchezerik836@gmail.com"
10
  version: "2026-10-16-rev5"
 
 
 
 
 
 
 
 
11
  license:
12
  - MIT
13
  - CC-BY-4.0
@@ -45,7 +53,10 @@ preferred-citation:
45
  year: 2026
46
  month: 10
47
  status: preprint
48
- notes: "Technical Report, revision 5. Not peer reviewed. DOI pending."
 
 
 
49
  url: "https://huggingface.co/Eriksanchezolvera/cuerdas"
50
  license: CC-BY-4.0
51
  references:
 
8
  affiliation: "Independent researcher"
9
  email: "sanchezerik836@gmail.com"
10
  version: "2026-10-16-rev5"
11
+ doi: "10.5281/zenodo.23448095"
12
+ identifiers:
13
+ - type: doi
14
+ value: "10.5281/zenodo.23448095"
15
+ description: "Zenodo DOI of the technical report"
16
+ - type: url
17
+ value: "https://huggingface.co/Eriksanchezolvera/cuerdas"
18
+ description: "Code, logs, sealed tests and checkpoints"
19
  license:
20
  - MIT
21
  - CC-BY-4.0
 
53
  year: 2026
54
  month: 10
55
  status: preprint
56
+ doi: "10.5281/zenodo.23448095"
57
+ publisher:
58
+ name: "Zenodo"
59
+ notes: "Technical Report, revision 5 (updated 18 October 2026). Not peer reviewed."
60
  url: "https://huggingface.co/Eriksanchezolvera/cuerdas"
61
  license: CC-BY-4.0
62
  references:
README.md CHANGED
@@ -15,7 +15,9 @@ tags:
15
 
16
  # Semantic Strands ("Cuerdas"): a pre-registered negative result on learned soft-slot context compression
17
 
18
- **Technical Report, Revision 5 (16 October 2026). Not peer reviewed.**
 
 
19
  Erik Sanchez, independent researcher Β· sanchezerik836@gmail.com
20
 
21
  > **ES.** Les presento los resultados de mi primer reto personal en ciencia computacional. En este punto mi conocimiento todavΓ­a es nulo, pero lo compenso con una gran curiosidad. Espero que les sea de utilidad.
@@ -109,9 +111,9 @@ AI systems are not authors. Grok Bot (AI assistant) helped design the experiment
109
 
110
  ## How to cite
111
 
112
- > Sanchez, E. (2026). *Semantic Strands: A Pre-registered Negative Result on Learned Soft-Slot Context Compression β€” Trained Compact Text Wins.* Technical Report, revision 5. https://huggingface.co/Eriksanchezolvera/cuerdas
113
 
114
- DOI: pending
115
 
116
  ```bibtex
117
  @techreport{sanchez2026cuerdas,
@@ -125,7 +127,7 @@ DOI: pending
125
  year = {2026},
126
  month = oct,
127
  url = {https://huggingface.co/Eriksanchezolvera/cuerdas},
128
- note = {DOI pending}
129
  }
130
  ```
131
 
 
15
 
16
  # Semantic Strands ("Cuerdas"): a pre-registered negative result on learned soft-slot context compression
17
 
18
+ [![DOI](https://zenodo.org/badge/DOI/10.5281/zenodo.23448095.svg)](https://doi.org/10.5281/zenodo.23448095)
19
+
20
+ **Technical Report, Revision 5 (16 October 2026; updated 18 October 2026). Not peer reviewed.** DOI: [10.5281/zenodo.23448095](https://doi.org/10.5281/zenodo.23448095)
21
  Erik Sanchez, independent researcher Β· sanchezerik836@gmail.com
22
 
23
  > **ES.** Les presento los resultados de mi primer reto personal en ciencia computacional. En este punto mi conocimiento todavΓ­a es nulo, pero lo compenso con una gran curiosidad. Espero que les sea de utilidad.
 
111
 
112
  ## How to cite
113
 
114
+ > Sanchez, E. (2026). *Semantic Strands: A Pre-registered Negative Result on Learned Soft-Slot Context Compression β€” Trained Compact Text Wins.* Technical Report, revision 5. Zenodo. https://doi.org/10.5281/zenodo.23448095
115
 
116
+ DOI: [10.5281/zenodo.23448095](https://doi.org/10.5281/zenodo.23448095)
117
 
118
  ```bibtex
119
  @techreport{sanchez2026cuerdas,
 
127
  year = {2026},
128
  month = oct,
129
  url = {https://huggingface.co/Eriksanchezolvera/cuerdas},
130
+ doi = {10.5281/zenodo.23448095}
131
  }
132
  ```
133
 
paper/cuerdas-technical-report.md CHANGED
@@ -1,6 +1,8 @@
1
  # Semantic Strands: A Pre-registered Negative Result on Learned Soft-Slot Context Compression β€” Trained Compact Text Wins
2
 
3
- **Technical Report. Revision 5, 16 October 2026. Not peer reviewed.**
 
 
4
 
5
  **Erik Sanchez**, independent researcher Β· <sanchezerik836@gmail.com>
6
 
@@ -17,7 +19,7 @@ We report a negative result so that others can skip this path. We trained a smal
17
  **What failed.**
18
 
19
  - On a sealed test with unseen words, names, phrases and multi-token values, accuracy falls to 29% with four unseen values, against 98% for the template. Retraining with a much larger, more varied replacement pool did not help (+2.6 pp, interval includes zero).
20
- - With a matched trainable budget (a reparameterized soft prompt of 3.84M parameters, same data, steps, loss, teacher and validation-only selection), compact text wins. Values separated by commas score 100% on every sealed set, including four unseen values, at 22 positions per event plus 8 shareable prompt positions. Against this baseline the slots save only 2–5 positions per event and lose up to 71 pp.
21
  - Probes trained with replacement words read unseen values from the slots better than the frozen LLM does (for example 97.7% vs 52.6% with two unseen values). The information is in the slots; the receiver does not read it.
22
 
23
  Zero-shot compact text fails with this receiver (0–8%), which is why comparing only against zero-shot or verbose text overstates the benefit of soft slots. **Our main recommendation: before claiming compression from learned soft slots, compare against compact text with a matched trainable budget (for example a soft prompt or prefix of equal size), on a sealed test that includes unseen values.**
@@ -274,7 +276,7 @@ Kaggle Tesla T4 GPU time, as logged:
274
  | 019 (re-evaluation only) | β‰ˆ3 min |
275
  | 020 (sealed test, baselines, probes; no encoder training) | β‰ˆ6 min |
276
 
277
- The 019 re-evaluation, with no training, took about 3 min of T4 time, plus a few minutes for a diagnostic run. The total is about 2.6 GPU-hours. In addition, 013, the 014 audit and the 015 pilot ran on a shared CPU machine. Energy and VRAM were not measured.
278
 
279
  ---
280
 
@@ -481,7 +483,7 @@ Rule 3 is met: no-LLM decoders match or beat the frozen LLM on the same slots (r
481
 
482
  Soft prompt + values with commas minus K=2 (paired bootstrap, seed means): +1.33 pp [+0.58, +2.17] on sealed-CH, +18.2 [+14.4, +22.4] with one unseen value, +47.4 [+41.7, +53.1] with two and **+70.7 [+64.9, +76.2]** with four.
483
 
484
- **Pre-registered rule 2 is not met: K=2 keeps no advantage.** With the same trainable budget, a soft prompt in front of a comma-separated value list is perfect on every sealed set at 22 positions per event (+8 shared), and values only reach 100% on known values and 93–97% with four unseen values at 19 positions. Against this fair baseline, K=2 saves only 2–5 positions per event (βˆ’11% to βˆ’23%, not βˆ’51%) and loses up to 70 pp on unseen values. The 51% saving reported earlier is a saving against a verbose template, not against competent compact text. Untrained few-shot prompts do not close the gap, which is why the zero-shot comparisons of Section 5.6 were misleading.
485
 
486
 
487
  ---
@@ -633,11 +635,12 @@ The following ten experiments are the external reviewer's list of requirements,
633
 
634
  **Code, models and software.**
635
 
636
- - **Repository:** <https://huggingface.co/Eriksanchezolvera/cuerdas> (currently **private**). It contains:
637
  - per-iteration scripts (`run015.py` … `run018.py`) and the exact Kaggle kernels (`kaggle_kernel.py`);
638
  - raw logs (`results01X.jsonl`, including the 018 validation curves) and result notes;
639
  - the 014 audit script;
640
  - the checkpoints `binder4_1000_s*.pt` (015), `k1_s*.pt` (017 K=1) and `k2long_s*.pt` (018 K=2);
 
641
  - the `LICENSE` file (MIT).
642
  - **Software:** `transformers==4.46.3` and PyTorch.
643
  - The 018 run did not log its PyTorch version; its log shows Python 3.13.
@@ -659,7 +662,7 @@ The following ten experiments are the external reviewer's list of requirements,
659
  1. Regenerate the dataset: `python make_dataset.py --out $SRC/data/expanded.json`. The script uses only the standard library.
660
  2. Download the receiver at the revision above to `$MP`.
661
  3. Run `SRC=… MP=… OD=out python 015/run015.py base binder4:1000`, then `016/run016.py`, `017/run017.py` and `018/run018.py` with `INP=checkpoints`.
662
- - **Compute:** about 2.5 T4 GPU-hours in total for 015–019 (Section 4.7), on Kaggle's free tier.
663
  - **Dataset:** the original `expanded.json`, produced in iteration 014 with ChatGPT, is **not redistributed**. Instead, `make_dataset.py` re-implements the documented 014 protocol:
664
  - the vocabulary;
665
  - exclusion of the 013 universe;
@@ -670,7 +673,7 @@ The following ten experiments are the external reviewer's list of requirements,
670
  - **Per-example predictions and re-evaluation:** `019/eval019.py`, its Kaggle kernel, `019/results019.jsonl` and `019/predictions019.jsonl`.
671
  - **Iteration 020:** `020/preregistro020.md` (pre-registration), `020/make_sealed020.py` and `020/sealed020.json` (SHA-256 `4c7737f66364a44786b4d9acb4686a6f31829d78cb5914f19eca1df74fe73393`), `020/eval020.py`, `020/results020.jsonl` and `020/predictions020.jsonl`.
672
 
673
- **Still to do before publication.** Pass this report through the NeurIPS Paper Checklist [13] and the ML Reproducibility Checklist [14], and add a model card [15] to the repository.
674
 
675
  ---
676
 
@@ -702,7 +705,7 @@ We thank **Kaggle** for the free GPU time (Tesla T4) that made iterations 015–
702
 
703
  ## 14. How to Cite
704
 
705
- > Sanchez, E. (2026). *Semantic Strands: A Pre-registered Negative Result on Learned Soft-Slot Context Compression β€” Trained Compact Text Wins.* Technical Report, revision 5. <https://huggingface.co/Eriksanchezolvera/cuerdas>
706
 
707
  ```bibtex
708
  @techreport{sanchez2026cuerdas,
@@ -716,7 +719,7 @@ We thank **Kaggle** for the free GPU time (Tesla T4) that made iterations 015–
716
  year = {2026},
717
  month = oct,
718
  url = {https://huggingface.co/Eriksanchezolvera/cuerdas},
719
- note = {DOI pending}
720
  }
721
  ```
722
 
 
1
  # Semantic Strands: A Pre-registered Negative Result on Learned Soft-Slot Context Compression β€” Trained Compact Text Wins
2
 
3
+ **Technical Report. Revision 5, 16 October 2026 (updated 18 October 2026: DOI, public repository and two consistency fixes; no results changed). Not peer reviewed.**
4
+
5
+ **DOI:** [10.5281/zenodo.23448095](https://doi.org/10.5281/zenodo.23448095) Β· **Repository:** <https://huggingface.co/Eriksanchezolvera/cuerdas>
6
 
7
  **Erik Sanchez**, independent researcher Β· <sanchezerik836@gmail.com>
8
 
 
19
  **What failed.**
20
 
21
  - On a sealed test with unseen words, names, phrases and multi-token values, accuracy falls to 29% with four unseen values, against 98% for the template. Retraining with a much larger, more varied replacement pool did not help (+2.6 pp, interval includes zero).
22
+ - With a matched trainable budget (a reparameterized soft prompt of 3.84M parameters, same data, steps, loss, teacher and validation-only selection), compact text wins. Values separated by commas score 100% on every sealed set, including four unseen values, at 22 positions per event plus 8 shareable prompt positions. Against this baseline the slots save only 2–5 positions per event and lose up to 70.7 pp.
23
  - Probes trained with replacement words read unseen values from the slots better than the frozen LLM does (for example 97.7% vs 52.6% with two unseen values). The information is in the slots; the receiver does not read it.
24
 
25
  Zero-shot compact text fails with this receiver (0–8%), which is why comparing only against zero-shot or verbose text overstates the benefit of soft slots. **Our main recommendation: before claiming compression from learned soft slots, compare against compact text with a matched trainable budget (for example a soft prompt or prefix of equal size), on a sealed test that includes unseen values.**
 
276
  | 019 (re-evaluation only) | β‰ˆ3 min |
277
  | 020 (sealed test, baselines, probes; no encoder training) | β‰ˆ6 min |
278
 
279
+ The 019 re-evaluation, with no training, took about 3 min of T4 time, plus a few minutes for a diagnostic run. The total for iterations 015–020 is about 2.6 GPU-hours. In addition, 013, the 014 audit and the 015 pilot ran on a shared CPU machine. Energy and VRAM were not measured.
280
 
281
  ---
282
 
 
483
 
484
  Soft prompt + values with commas minus K=2 (paired bootstrap, seed means): +1.33 pp [+0.58, +2.17] on sealed-CH, +18.2 [+14.4, +22.4] with one unseen value, +47.4 [+41.7, +53.1] with two and **+70.7 [+64.9, +76.2]** with four.
485
 
486
+ **Pre-registered rule 2 is not met: K=2 keeps no advantage.** With the same trainable budget, a soft prompt in front of a comma-separated value list is perfect on every sealed set at 22 positions per event (+8 shared), and values only reach 100% on known values and 93–97% with four unseen values at 19 positions. Against this fair baseline, K=2 saves only 2–5 positions per event (βˆ’11% to βˆ’23%, not βˆ’51%) and loses up to 70.7 pp on unseen values. The 51% saving reported earlier is a saving against a verbose template, not against competent compact text. Untrained few-shot prompts do not close the gap, which is why the zero-shot comparisons of Section 5.6 were misleading.
487
 
488
 
489
  ---
 
635
 
636
  **Code, models and software.**
637
 
638
+ - **Repository:** <https://huggingface.co/Eriksanchezolvera/cuerdas> (**public**). It contains:
639
  - per-iteration scripts (`run015.py` … `run018.py`) and the exact Kaggle kernels (`kaggle_kernel.py`);
640
  - raw logs (`results01X.jsonl`, including the 018 validation curves) and result notes;
641
  - the 014 audit script;
642
  - the checkpoints `binder4_1000_s*.pt` (015), `k1_s*.pt` (017 K=1) and `k2long_s*.pt` (018 K=2);
643
+ - the iteration 020 and 021 files (pre-registrations, sealed test sets, evaluation scripts, logs and per-example predictions) and the 021 checkpoints;
644
  - the `LICENSE` file (MIT).
645
  - **Software:** `transformers==4.46.3` and PyTorch.
646
  - The 018 run did not log its PyTorch version; its log shows Python 3.13.
 
662
  1. Regenerate the dataset: `python make_dataset.py --out $SRC/data/expanded.json`. The script uses only the standard library.
663
  2. Download the receiver at the revision above to `$MP`.
664
  3. Run `SRC=… MP=… OD=out python 015/run015.py base binder4:1000`, then `016/run016.py`, `017/run017.py` and `018/run018.py` with `INP=checkpoints`.
665
+ - **Compute:** about 2.6 T4 GPU-hours in total for 015–020 (Section 4.7), on Kaggle's free tier.
666
  - **Dataset:** the original `expanded.json`, produced in iteration 014 with ChatGPT, is **not redistributed**. Instead, `make_dataset.py` re-implements the documented 014 protocol:
667
  - the vocabulary;
668
  - exclusion of the 013 universe;
 
673
  - **Per-example predictions and re-evaluation:** `019/eval019.py`, its Kaggle kernel, `019/results019.jsonl` and `019/predictions019.jsonl`.
674
  - **Iteration 020:** `020/preregistro020.md` (pre-registration), `020/make_sealed020.py` and `020/sealed020.json` (SHA-256 `4c7737f66364a44786b4d9acb4686a6f31829d78cb5914f19eca1df74fe73393`), `020/eval020.py`, `020/results020.jsonl` and `020/predictions020.jsonl`.
675
 
676
+ **Still to do.** Pass this report through the NeurIPS Paper Checklist [13] and the ML Reproducibility Checklist [14]. A model card [15] has been added to the repository.
677
 
678
  ---
679
 
 
705
 
706
  ## 14. How to Cite
707
 
708
+ > Sanchez, E. (2026). *Semantic Strands: A Pre-registered Negative Result on Learned Soft-Slot Context Compression β€” Trained Compact Text Wins.* Technical Report, revision 5. Zenodo. <https://doi.org/10.5281/zenodo.23448095>
709
 
710
  ```bibtex
711
  @techreport{sanchez2026cuerdas,
 
719
  year = {2026},
720
  month = oct,
721
  url = {https://huggingface.co/Eriksanchezolvera/cuerdas},
722
+ doi = {10.5281/zenodo.23448095}
723
  }
724
  ```
725
 
paper/cuerdas-technical-report.pdf CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:c2ac543fc4f49adf33d90c598aa1f0332bcf75e7070f462a98c241a68e217797
3
- size 181206
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:82d84539a81ad389e33d9ece7a3bf7ebf0196b65eac5acbca9fe7db6e8faf4d3
3
+ size 182028