File size: 6,235 Bytes
b7f5465 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 | # Title-Boundary Heuristic Experiments
This note records deterministic Rust heuristic changes and Pi RPC reviewer evidence
for DMHY template `title_spans` generation. The reviewer output is evidence only;
recipes and training JSONL are changed only by Rust code.
## Baseline
Input recipe evidence:
- `reports/dmhy_template_recipes.full_top5000.title_boundary_trial.seed.jsonl`
- total title-boundary decisions: `5631`
Decision distribution:
| decision | count |
| --- | ---: |
| `merge_dissimilar_title_continuation` | 4661 |
| `keep_similar_duplicate_or_alias` | 463 |
| `keep_structural_boundary` | 502 |
| `keep_path_boundary` | 5 |
Pi RPC stratified sample, `--threads 4`:
| stratum | rows | model result | note |
| --- | ---: | --- | --- |
| ambiguous merge, similarity `0.25..0.85` | 20 | 20 merge | Good. Reviewer cited concrete title-word pairings. |
| high-sim `keep_similar_duplicate_or_alias`, similarity `0.85..1.0` | 20 | 19 merge, 1 parse-error | Bad baseline heuristic. Examples such as `Asobi ni Iku yo!`, `Code Geass ~Akito the Exiled~`, `Yagate Kimi ni Naru`, and `Big Order` were wrongly kept separate by the heuristic. |
| structural keep | 20 | 18 keep, 1 merge, 1 skip | Mostly good. Covers category+title, multilingual aliases, and distinct fields. |
| path keep | 5 | 5 keep | Good. Covers path duplicate/category boundaries. |
Conclusion: `keep_similar_duplicate_or_alias` was too broad because it used max
literal similarity. Short connector repeats (`no`, `wa`, `ni`, `the`, `of`) and
repeated title words inflated similarity to `1.0` even for ordinary contiguous
title phrases.
## Experiment 1: strict substantive duplicate keep
Change tried:
- Keep adjacent `TITLE SEP TITLE` only when top literals include a substantive
duplicate/near-duplicate pair.
- Ignore short connectors and literals with normalized length `< 6`.
Generated recipe output:
- `reports/dmhy_template_recipes.full_top5000.strictdup.seed.jsonl`
Decision distribution:
| decision | count |
| --- | ---: |
| `merge_dissimilar_title_continuation` | 4661 |
| `merge_adjacent_title_continuation` | 444 |
| `keep_similar_duplicate_or_alias` | 19 |
| `keep_structural_boundary` | 502 |
| `keep_path_boundary` | 5 |
Pi RPC review of all 19 remaining `keep_similar_duplicate_or_alias` decisions:
| rows | model result |
| ---: | --- |
| 19 | 19 merge |
Examples the reviewer judged as merge:
- `Air In Summer`, `Assault Lily BOUQUET`
- `Aoki Hagane no Arpeggio Ars Nova`
- `Shoujo Kageki Revue Starlight`
- `Motto! Ojamajo Doremi`
- `Fate stay night Heaven's Feel`
- `Arafou Otoko no Isekai Tsuuhan Seikatsu`
- `Watashi, Nouryoku wa Heikinchi de tte Itta yo ne!`
Conclusion: strict duplicate evidence still produced false keeps. The repeated
literal was often a real repeated word inside one title phrase or a top-literal
artifact, not a duplicate/alias boundary.
## Experiment 2: merge all adjacent separator-only title slots
Final change:
- `TITLE PATH TITLE` remains `keep_path_boundary`.
- Non-separator structural boundaries remain `keep_structural_boundary`.
- Every separator-only adjacent title boundary, `TITLE SEP TITLE`, now merges.
- Low-similarity merges keep the old decision name
`merge_dissimilar_title_continuation`.
- High-similarity merges use `merge_adjacent_title_continuation`.
- `keep_similar_duplicate_or_alias` is no longer emitted by the deterministic
heuristic.
Generated recipe output:
- `reports/dmhy_template_recipes.full_top5000.mergeall.seed.jsonl`
Decision distribution:
| decision | count |
| --- | ---: |
| `merge_dissimilar_title_continuation` | 4661 |
| `merge_adjacent_title_continuation` | 463 |
| `keep_structural_boundary` | 502 |
| `keep_path_boundary` | 5 |
Pi RPC sample of high-similarity `merge_adjacent_title_continuation` decisions:
| rows | model result |
| ---: | --- |
| 30 | 29 merge, 1 parse-error |
The reviewer cited concrete title continuations such as:
- `Asobi ni Iku yo!`
- `Code Geass ~Akito the Exiled~`
- `Kami nomi zo Shiru Sekai`
- `Yagate Kimi ni Naru`
- `Big Order`
- `Gate Jieitai Kanochi nite Kaku Tatakaeri`
- `JoJo's Bizarre Adventure - Stardust Crusaders`
Conclusion: merge-all adjacent separator-only title slots is the best of the
tried heuristics. It matches Pi RPC evidence and simplifies downstream synthetic
title-span replacement.
## Synthetic generation smoke
Generated schema v2 synthetic augmentation with merge-all recipes:
- `data/schema_v2_synthetic_aug.mergeall.jsonl`
- `data/schema_v2_synthetic_aug.mergeall.manifest.json`
Validator passed. Manifest summary:
| metric | value |
| --- | ---: |
| `generated_rows` | 49998 |
| `numeric_title_rows` | 30000 |
| `path_rows` | 19998 |
| `path_series_rows` | 15000 |
| `path_movie_rows` | 1666 |
| `path_special_rows` | 1666 |
| `path_confuser_rows` | 1666 |
| `dropped_media_kind_mismatch` | 19611 |
## Commands
Pi RPC high-sim adjacent merge sample:
```powershell
cargo run --release --manifest-path tools\rust_dmhy_template_apply\Cargo.toml -- `
--review-title-boundaries-pi-rpc `
--recipes reports\dmhy_template_recipes.full_top5000.mergeall.seed.jsonl `
--title-boundary-decisions-output reports\dmhy_title_boundary_pi_rpc_mergeall.adjacent_highsim.jsonl `
--title-boundary-heuristic-contains merge_adjacent_title_continuation `
--title-boundary-min-similarity 0.85 `
--title-boundary-max-similarity 1.0 `
--threads 4 `
--limit 30
```
Synthetic smoke:
```powershell
cargo run --release --manifest-path tools\schema_v2_synthetic_augment\Cargo.toml --bin schema_v2_synthetic_augment -- `
--recipes reports\dmhy_template_recipes.full_top5000.mergeall.seed.jsonl `
--label-schema-file label_schema.json `
--numeric-title-seeds data\synthetic_numeric_titles.txt `
--path-prefix-seeds data\synthetic_path_prefixes.txt `
--limit-templates 3000 `
--max-rows 50000 `
--output data\schema_v2_synthetic_aug.mergeall.jsonl `
--manifest-output data\schema_v2_synthetic_aug.mergeall.manifest.json
cargo run --release --manifest-path tools\schema_v2_synthetic_augment\Cargo.toml --bin validate_synthetic_aug_jsonl -- `
--input data\schema_v2_synthetic_aug.mergeall.jsonl `
--manifest data\schema_v2_synthetic_aug.mergeall.manifest.json
```
|