Upload data_night3/MANIFEST.md with huggingface_hub

data_night3/MANIFEST.md

@@ -0,0 +1,143 @@
+# datasets_night3 — MANIFEST
+*Built 2026-06-13 by EX5 (dataset-synthesis agent), NIGHT3_PLAN.md §C "Datasets to
+synthesize". Local CPU build; validation encodes on arena8-keepsake (SONAR
+encode→decode round-trip, see `validation_report.json`). All four files uploaded to
+HF `nickypro/sonar-sae` under `data_night3/`.*
+
+Build scripts (deterministic, seed 0): `lexicon.py`, `build_token_position_grid.py`,
+`build_spec_edit_corpus.py`, `build_paraphrase_bank.py`, `build_construction_specs.py`,
+`validate_roundtrip.py`.
+
+Frequency bands everywhere (wordfreq zipf): **high ≥ 4.6 · mid 3.6–4.5 · low 2.3–3.3**
+(gap zones avoid borderline assignment).
+
+---
+
+## 1. token_position_grid.jsonl — 9,600 rows  (consumers: T1-E1 E-table fit, F-3)
+
+**Design:** 200 frequency-stratified words × positions 1–16 × 3 frames. The SAME word
+appears at EVERY position — the design point for conditioning the embedding-table fit.
+
+- Words: 50 high / 100 mid / 50 low; POS mix 68 nouns / 67 verbs (base form) /
+  65 adjectives, roughly balanced within each band.
+- Zipf ranges actually realized: high [4.63, 6.25], mid [3.60, 4.49], low [2.38, 3.24].
+- Frames: list-style carrier sentences of exactly **20 words** = 16 comma-separated
+  slot words + a fixed 4-word tail ("were on the list" / "appeared in that order" /
+  "were read aloud twice"). Fillers are fixed mid-frequency nouns, disjoint across the
+  3 frames and disjoint from the 200 targets. Target replaces the filler at slot p;
+  `pos_word_index` is the 1-based word index (slot 1 = sentence-initial, capitalized).
+- Fields: `{word, pos_class, freq_band, zipf, pos_word_index, frame_id, text}`.
+- All 9,600 texts unique.
+
+**Known gaps / notes:** frames are word-LIST sentences, not narrative prose — chosen
+deliberately: it is the only construction that admits any POS at any of positions 1–16
+(matches the serial-position / unbinding experimental idiom). Round-trip check shows
+the expected mid-list lexical substitution behavior of SONAR on 16-item lists
+(cos .91, token-F1 .73 — this *is* the capacity phenomenon T1-E1/F-3 measure, not a
+data defect; frame structure and edges decode reliably).
+
+## 2. spec_edit_corpus.jsonl — 1,000 rows  (consumers: T1-E4, T2-E10)
+
+**Design:** the 10 frozen TAEBench v2 edit types (`taebench_v2.E1_EDIT_TYPES`) × 100
+wild bases each. Bases from the project's fineweb-derived corpus
+(nickypro/llama-3b-split → paragraphs → `SENT_SPLIT`), 8–25 words (mean 19.0),
+quality-filtered, **all 1,000 bases globally unique** (no base reused across types).
+Transforms REUSE `taebench_v2_e1.py` machinery (WORD_PAIRS, ENTITY_PAIRS, TENSE_MAP,
+CONTRACTIONS, `_negate`, `_delete_clause`, `_map_transform`).
+
+- Fields: `{base_text, edit_type, spec, gold_rewrite}`;
+  `spec = {type, target_lemma, position_index, replacement, parameter, n_words}`.
+- **Leakage-safe schema** (pre-registered, per T1-E4 C4): spec carries edit-type,
+  lemma, 1-based whitespace-token position, replacement/parameter — NEVER the target
+  sentence. Verified programmatically: 0 rows where gold appears in the spec JSON.
+- Every gold rewrite passes an E1-style per-type heuristic check at build time.
+
+**Coverage per type (n=100 each; restrictions that buy gold reliability):**
+
+| type | gold rule | restriction | reliability |
+|---|---|---|---|
+| word-swap | E1 WORD_PAIRS, case-preserving | exactly 1 occurrence; mid-sentence capitalized matches skipped (proper-noun guard) | rule-exact |
+| entity-swap | E1 ENTITY_PAIRS | exactly 1 occurrence; skipped if part of longer proper-noun phrase (no "New York City"→"Tokyo City") | rule-exact |
+| number-change | n → n+1 | exactly 1 occurrence, no leading zero | rule-exact |
+| tense | TENSE_MAP present→past | exactly ONE mapped verb; noun/verb-ambiguous forms ("leaves","needs",…) only after pronouns; det/poss-preceded matches skipped | rule-exact under restriction |
+| person | we/us/our→they/them/their (+contractions) | **first-person PLURAL only** (verb agreement is invariant for we→they; singular I→she needs conjugation and was excluded rather than emit garbage) | rule-exact under restriction |
+| negation | insert "not" after first aux (E1 `_negate`) | BE/modal aux always; has/have/do/does only before a clear participle/verb form | rule-exact under restriction |
+| register | contraction expansion (the project's frozen E1 operationalization — a register PROXY) | exactly 1 contraction | rule-exact; flagged as proxy |
+| insert-clause | append ", according to officials." | base ≤22 words, no prior "according to" | rule-exact |
+| delete-clause | E1 `_delete_clause` (trailing ", clause" of 3–14 words) | rule must fire | rule-exact |
+| reorder | swap two consecutive corpus sentences | both 4–12 words, combined 8–25; abbreviation-split pairs ("… v." etc.) filtered | rule-exact |
+
+**Honest coverage statement:** all 10/10 types ship with rule-based golds, but tense,
+person, negation are *restricted sub-distributions* (see table) — they cover the
+machine-verifiable subset, not the full linguistic phenomenon. Register is contraction
+expansion (proxy), matching the frozen E1 suite. No type was LLM-generated.
+
+## 3. paraphrase_bank_v2.jsonl — 2,000 rows  (consumer: T2-E12)
+
+**Design:** length-matched pairs, |len_a − len_b| ≤ 2 enforced on every row
+(distribution: 795 diff-0, 457 diff-1, 748 diff-2). Mean len_a 8.4 words.
+Fields: `{text_a, text_b, len_a, len_b, len_diff, pair_type}`.
+
+| pair_type | n | provenance |
+|---|---|---|
+| mined_orbit | 142 | mined from existing artifact hf:`tae_wording_subspace_paras.json` (Q3-009 SSAE/wording orbits); the cache has known batch misalignment, so pairs filtered by length match + content-Jaccard ≥.2 + SBERT cos ≥.78 + markup/abbreviation filters |
+| template_active_passive | 700 | rule transform, lexicon templates ("The Adj N Vt the Adj N." ↔ "The Adj N was PP by the Adj N."), verbs restricted to past==participle so the passive is morphologically exact |
+| template_cleft_passive | 400 | "It was the X that Vt the Y." ↔ "The Y was PP by the X." |
+| template_adverb_move | 758 | sentence-final manner adverb ↔ fronted "Adv, …" (pure reorder, diff 0) |
+
+**Known gaps:** (i) wild-corpus adverb-fronting was attempted and yielded ~0 pairs
+after the attachment-safety filter (single-clause only, no subordinators/gerunds) —
+the 63 lenient candidates were REJECTED for clause-attachment ambiguity ("…selling out
+quickly" ≠ "Quickly, …") rather than shipped; so 93% of the bank is template-derived,
+length-controlled but synthetic-register. (ii) mined_orbit is small (142) because the
+source cache holds only 197 orbits and is contaminated. (iii) template pairs are
+semantically absurd-but-grammatical by construction (lexicon sampling) — meaning
+equivalence WITHIN each pair holds by the transform, which is what T2-E12's CE-margin
+contrast needs.
+
+## 4. construction_specs.jsonl — 300 rows  (consumer: T1-E7)
+
+**Design:** lengths {4,6,8,12,16} words × 60 specs. Past-tense POS templates:
+- 4: `The N Vi Adv.` · 6: `The Adj N Vt the N.` · 8: `The Adj N Vt the Adj N Adv.`
+- 12: `… near the Adj N.` · 16: `… near the Adj N before the Adj N.`
+
+**Frequency matching (the anti-confound):** every content slot is filled with exactly
+20 high / 20 mid / 20 low draws across the 60 specs of each length, so the marginal
+band distribution of content words is IDENTICAL across lengths — length is not
+confounded with rarity. Verified: per-length band counts are exactly uniform
+(60/60/60, 80/80/80, 120/120/120, 160/160/160, 200/200/200).
+
+- Fields: `{length, spec_id, template, words (ordered), text, content_slots,
+  content_bands}`. No word repeats within a sentence; quantifier/predicative-only
+  adjectives (few/own/able/sure) excluded; all 300 texts unique.
+- **Known gaps:** sentences are grammatical but semantically random ("The soggy garden
+  scoured the wild squirrel faintly…") — intentional, so recall is not inflated by
+  semantic predictability. Only 5 fixed function words (the/near/before) appear.
+
+---
+
+## Validation (arena8-keepsake, SONAR encode→decode round-trip, `validation_report.json`)
+
+~30 texts per dataset (spec corpus: 3/type × base+gold; paraphrase: 15 pairs both
+sides; construction: 6/length). SBERT = all-MiniLM-L6-v2.
+
+| dataset | n | SBERT cos (min) | token-F1 (min) | exact | extra |
+|---|---|---|---|---|---|
+| token_position_grid | 30 | .909 (.85) | .726 (.59) | .00 | expected: 16-item lists sit beyond SONAR's verbatim-recall capacity; list structure + edges decode, interior items get near-synonym substitution — the phenomenon under study |
+| spec_edit_corpus base | 30 | .982 (.87) | .945 (.76) | .47 | |
+| spec_edit_corpus gold | 30 | .982 (.88) | .956 (.80) | .43 | golds decode as cleanly as bases |
+| paraphrase_bank a | 15 | .968 (.83) | .949 (.86) | .60 | |
+| paraphrase_bank b | 15 | .963 (.82) | .947 (.84) | .53 | both sides round-trip symmetrically |
+| construction_specs | 30 | .905 (.65) | .875 (.67) | .27 | content-word recall .82; degrades with length as the capacity curve predicts |
+
+Verdict: decode quality is sane on all four datasets; no garbage rows surfaced; the
+two low-F1 datasets are low for the *capacity* reasons the consuming experiments are
+designed to measure.
+
+## HF locations (repo `nickypro/sonar-sae`, repo_type=model)
+
+- `data_night3/token_position_grid.jsonl`
+- `data_night3/spec_edit_corpus.jsonl`
+- `data_night3/paraphrase_bank_v2.jsonl`
+- `data_night3/construction_specs.jsonl`
+- `data_night3/MANIFEST.md`, `data_night3/validation_report.json`