kwabs22 Claude Opus 5.5 commited on
Commit
5b66adc
·
1 Parent(s): 2fe7d5d

Paragraphs: extend_scene, references that bind to earlier items, and a bind change

Browse files

From QA round 10 (paragraph experiments) and the user's choice of defaults:
- "the cat" when a cat already exists refers to it instead of making a twin,
surfaced as "which cat? the one already mentioned". "a/an/another cat"
always makes a new one. Fixes the twin-cat scene where the apple sat
beside a second cat on the floor.
- Pronouns (it, he, she, they, him, her, them) are words now: they refer to
the most recent object (not the clause's own subject), surfaced as
"'it' = the table?" with the alternatives.
- extend_scene(ledger, sentence, ...) adds the next sentence to a ledger.
- refine_scene takes {"id": "q10", "bind": "n1"} to rebind a reference and
{"id": "n12", "bind": "n1"} to merge a duplicate, with a resolution note.

Tested with a stub dictionary: one sentence with the cat twice, pronoun
across sentences, rebind, another cat, duplicate merge, self-relation
refused, pronoun with nothing before it, light question dropped.

Also archives QA rounds 9-10.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Files changed (5) hide show
  1. SPEC.md +24 -0
  2. app.py +36 -0
  3. feedback/vli-qa-feedback-2026-10-30.md +24 -0
  4. vli/grammar.py +86 -20
  5. vli/pipeline.py +71 -1
SPEC.md CHANGED
@@ -103,6 +103,29 @@ Relations: `on, under, beside, near, behind, in_front_of, above, faces, lights`.
103
  Verbs: `sat/stood/lay/rested` → `on` (ground implied), `shone/glowed` → `lights`
104
  (shadows only turn on with this coupling), `faced` → `faces`.
105
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
106
  ## MCP tools
107
 
108
  | Tool | In → out |
@@ -110,6 +133,7 @@ Verbs: `sat/stood/lay/rested` → `on` (ground implied), `shone/glowed` → `lig
110
  | `check_words` | sentence, primitives → per-word status, missing words, completions |
111
  | `translate_to_scene` | sentence, primitives, seed, html → ledger, PNG, GLB, mini HTML file, mini HTML text |
112
  | `refine_scene` | ledger, changes, seed, html → ledger, PNG, GLB, mini HTML file, mini HTML text |
 
113
  | `simplify_item` | ledger, item id, html, rings, samples → report, rung PNGs, rung GLBs, mini HTML file, mini HTML text |
114
  | `papercraft_rings` | ledger, item id, rings, samples, height_cm → pieces report, PNG per A4 page (to look at), SVG per A4 page (mm, to print) |
115
 
 
103
  Verbs: `sat/stood/lay/rested` → `on` (ground implied), `shone/glowed` → `lights`
104
  (shadows only turn on with this coupling), `faced` → `faces`.
105
 
106
+ ## Paragraphs: sentences + bindings
107
+
108
+ `extend_scene` adds the next sentence to an existing ledger (token and item ids carry
109
+ on, `sentences` lists them). Within a sentence or across sentences:
110
+
111
+ - **"the <noun>"** when that noun is already in the ledger refers to it (the most
112
+ recent one) instead of adding an item. Surfaced as an implied question: "which cat?
113
+ the one already mentioned" (default: the same cat), alternatives include "say
114
+ 'another cat' for a new one".
115
+ - **"a/an/another <noun>"** always adds a new item.
116
+ - **Pronouns** (it, he, she, they, him, her, them) refer to the most recent object,
117
+ leaving out the clause's own subject when the pronoun is its object ("the apple lay
118
+ beside it" can't mean the apple). Surfaced as "'it' = the table?", with the other
119
+ objects as alternatives, most recent first. A pronoun with nothing before it is an
120
+ error.
121
+ - Each reference question carries `binds: {token, to}`; relations record which token
122
+ named each end (`tokens: {subject, object}`), so a reference can be rebound.
123
+ - **`bind`** (a `refine_scene` change): `{"id": "q10", "bind": "n1"}` rebinds a
124
+ reference question; `{"id": "n12", "bind": "n1"}` merges a duplicate object into
125
+ another (relations move, its "which?" question goes, a light carries over). Both
126
+ refuse to relate an object to itself and add a note to `resolutions`.
127
+ - "what light?" is asked once per ledger and dropped when a later sentence lights the scene.
128
+
129
  ## MCP tools
130
 
131
  | Tool | In → out |
 
133
  | `check_words` | sentence, primitives → per-word status, missing words, completions |
134
  | `translate_to_scene` | sentence, primitives, seed, html → ledger, PNG, GLB, mini HTML file, mini HTML text |
135
  | `refine_scene` | ledger, changes, seed, html → ledger, PNG, GLB, mini HTML file, mini HTML text |
136
+ | `extend_scene` | ledger, sentence, primitives, seed, html → ledger, PNG, GLB, mini HTML file, mini HTML text |
137
  | `simplify_item` | ledger, item id, html, rings, samples → report, rung PNGs, rung GLBs, mini HTML file, mini HTML text |
138
  | `papercraft_rings` | ledger, item id, rings, samples, height_cm → pieces report, PNG per A4 page (to look at), SVG per A4 page (mm, to print) |
139
 
app.py CHANGED
@@ -157,6 +157,8 @@ def refine_scene(ledger_json: str, changes_json: str = "[]", seed: str = "0",
157
  {"id": "n1", "shape": "sphere"} use a primitive (box, sphere, cylinder, cone, plane)
158
  {"reroll": true} every unfrozen object takes its next alternative;
159
  repeated rerolls visit the whole pool, whatever the seed
 
 
160
  Frozen items cannot be changed. The configuration (placement, sun angle) is always
161
  re-chosen from `seed`: change the seed for another valid configuration.
162
 
@@ -179,6 +181,34 @@ def refine_scene(ledger_json: str, changes_json: str = "[]", seed: str = "0",
179
  _fail(e)
180
 
181
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
182
  def simplify_item(ledger_json: str, item_id: str, html: str = "url", rings: str = "8",
183
  samples: str = "16") -> tuple[str, list, list, str, str]:
184
  """Build the simplification ladder for one object: full asset -> fewer faces -> rings -> convex hull -> box.
@@ -382,6 +412,9 @@ with gr.Blocks(title="VisualLearningIdea") as demo:
382
  ledger = gr.Code(label="Ledger", language="json", interactive=True)
383
  changes = gr.Code(label="Changes", language="json", value=EXAMPLE_CHANGES, interactive=True)
384
  refine = gr.Button("Refine")
 
 
 
385
  with gr.Accordion("Simplification ladder", open=False):
386
  with gr.Row():
387
  item_id = gr.Textbox(label="Object item id", value="n1")
@@ -412,6 +445,9 @@ with gr.Blocks(title="VisualLearningIdea") as demo:
412
  refine.click(refine_scene, [ledger, changes, seed, html_mode],
413
  [ledger, picture, model, scene_file, scene_html], api_name="refine_scene") \
414
  .then(_html_preview, scene_file, scene_preview, api_visibility="private")
 
 
 
415
  simp.click(simplify_item, [ledger, item_id, html_mode, ring_count, ring_samples],
416
  [ladder_report, ladder_pics, ladder_files, ladder_file, ladder_html], api_name="simplify_item") \
417
  .then(_html_preview, ladder_file, ladder_preview, api_visibility="private")
 
157
  {"id": "n1", "shape": "sphere"} use a primitive (box, sphere, cylinder, cone, plane)
158
  {"reroll": true} every unfrozen object takes its next alternative;
159
  repeated rerolls visit the whole pool, whatever the seed
160
+ {"id": "n12", "bind": "n1"} n12 is the same thing as n1: merged into it, its relations move
161
+ {"id": "q11", "bind": "n5"} the word behind reference question q11 ("it", "the cat") means n5
162
  Frozen items cannot be changed. The configuration (placement, sun angle) is always
163
  re-chosen from `seed`: change the seed for another valid configuration.
164
 
 
181
  _fail(e)
182
 
183
 
184
+ def extend_scene(ledger_json: str, sentence: str, primitives_json: str = "{}", seed: str = "0",
185
+ html: str = "url") -> tuple[str, str, str, str, str]:
186
+ """Add the next sentence of a paragraph to an existing scene.
187
+
188
+ Paragraphs are sentences plus bindings. "the cat" when a cat is already in the ledger,
189
+ and pronouns (it, he, she, they), refer back to an existing item instead of adding one;
190
+ each such reference is an implied question ("which cat? the one already mentioned",
191
+ "'it' = the cat?") whose `binds` names the item. Rebind with refine_scene
192
+ {"id": "<question id>", "bind": "<item id>"}. "a/an/another cat" always adds a new one.
193
+
194
+ Args:
195
+ ledger_json: Ledger JSON from translate_to_scene, refine_scene or a previous extend_scene.
196
+ sentence: The next sentence, e.g. "the apple lay beside it".
197
+ primitives_json: JSON object mapping missing words to shapes, as in translate_to_scene.
198
+ seed: Picks one of the configurations that satisfy the whole ledger.
199
+ html: "url" (default), "inline" or "skip", as in translate_to_scene.
200
+
201
+ Returns:
202
+ (ledger JSON string, rendered PNG, GLB scene file, mini HTML file, mini HTML text or "").
203
+ """
204
+ try:
205
+ ledger = _json(ledger_json, "ledger_json")
206
+ *rest, page = pipeline.extend(ledger, sentence, _json(primitives_json, "primitives_json", {}), _seed(seed))
207
+ return (*rest, *_html_out(page, html))
208
+ except Exception as e:
209
+ _fail(e)
210
+
211
+
212
  def simplify_item(ledger_json: str, item_id: str, html: str = "url", rings: str = "8",
213
  samples: str = "16") -> tuple[str, list, list, str, str]:
214
  """Build the simplification ladder for one object: full asset -> fewer faces -> rings -> convex hull -> box.
 
412
  ledger = gr.Code(label="Ledger", language="json", interactive=True)
413
  changes = gr.Code(label="Changes", language="json", value=EXAMPLE_CHANGES, interactive=True)
414
  refine = gr.Button("Refine")
415
+ with gr.Row():
416
+ next_sentence = gr.Textbox(label="Next sentence (refers back: 'the cat', 'it')", scale=4)
417
+ extend = gr.Button("Add sentence", scale=1)
418
  with gr.Accordion("Simplification ladder", open=False):
419
  with gr.Row():
420
  item_id = gr.Textbox(label="Object item id", value="n1")
 
445
  refine.click(refine_scene, [ledger, changes, seed, html_mode],
446
  [ledger, picture, model, scene_file, scene_html], api_name="refine_scene") \
447
  .then(_html_preview, scene_file, scene_preview, api_visibility="private")
448
+ extend.click(extend_scene, [ledger, next_sentence, primitives, seed, html_mode],
449
+ [ledger, picture, model, scene_file, scene_html], api_name="extend_scene") \
450
+ .then(_html_preview, scene_file, scene_preview, api_visibility="private")
451
  simp.click(simplify_item, [ledger, item_id, html_mode, ring_count, ring_samples],
452
  [ladder_report, ladder_pics, ladder_files, ladder_file, ladder_html], api_name="simplify_item") \
453
  .then(_html_preview, ladder_file, ladder_preview, api_visibility="private")
feedback/vli-qa-feedback-2026-10-30.md CHANGED
@@ -182,3 +182,27 @@ Note: apple `b01` is a reddish-brown (#b27045) — "a green apple" still means w
182
  - **Bug: `top cap` has `glues_to: []`** — its glue edge to band 7 is missing from the assembly graph (bottom cap correctly glues to band 1). Small fix in the glue-path builder; an assembler would otherwise hold an unlabeled lid.
183
 
184
  > Note (agent): not a missing edge — tabs sit on one side of each joint, and the top cap's are on band 7's top edge. The report listed only outgoing tabs; it now also lists `receives_from`, so the top cap shows band 7a–7c.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
182
  - **Bug: `top cap` has `glues_to: []`** — its glue edge to band 7 is missing from the assembly graph (bottom cap correctly glues to band 1). Small fix in the glue-path builder; an assembler would otherwise hold an unlabeled lid.
183
 
184
  > Note (agent): not a missing edge — tabs sit on one side of each joint, and the top cap's are on band 7's top edge. The report listed only outgoing tabs; it now also lists `receives_from`, so the top cap shows band 7a–7c.
185
+
186
+ Remaining open: the `use`-back-swap check after skip-shifted indices, and the UI visual click-tests (mode radio, preview-from-file, papercraft download). The sentence → scene → ladder → printable-kit loop is now complete and demo-ready end to end.
187
+
188
+ ---
189
+
190
+ ## 10. Paragraph / disambiguation experiments (2026-10-30) — the next frontier, probed live
191
+
192
+ The goal going forward is the **learning-disambiguation tool**: paragraphs and pages, where later sentences refer back to earlier entities. Four attempts:
193
+
194
+ | # | Input | Result |
195
+ |---|---|---|
196
+ | 0 | `{"id":"n1","use":0}` on a pool with a skip-shifted index | ✓ chosen resolved to `alternatives[0]` correctly — `use` is sound. **Item closed** with the rule: indices are relative to the ledger you pass; callers must reuse the latest returned ledger. (UI click-tests also cleared by the reviewer.) |
197
+ | 1 | `"the cat sat on the table and the apple lay beside the cat and the lamp glowed"` (raw, repeated noun) | Structure ✓ — three clauses in one ledger, all relations intact, lamp lights the scene. Semantics ✗ — **two cat items (n1, n12)**, two "which cat?" questions; "beside the cat" bound to the *ground-level* twin, so the apple sat on the floor while the real cat was on the table (render proves it: cat on tabletop, second cat + apple on the ground) |
198
+ | 2 | `"…and the apple lay beside it…"` (pronoun) | ✗ hard fail: `Not available (no asset): it` — pronouns are unknown words |
199
+ | 3 | Attempt 1 hand-resolved from chat: `r10` bound `n12 → n1`, duplicate item + its question deleted, provenance tokens blanked; re-rendered via `refine_scene` with empty changes | ✓ **coherent page**: one cat on the table, apple beside it at the cat's height (both y = 0.916 — the `beside` solver inherited the table level automatically), lamp glowing. The reference-driven solver follows the fix |
200
+
201
+ **What this establishes:** the chat-side disambiguation loop works today with zero Space changes — *chat resolves, ledger persists, Space re-renders*. That is the learning-disambiguation tool's core loop, demonstrated.
202
+
203
+ **Proposed Space changes to make it first-class (next push candidates):**
204
+ 1. **`{"id": "n12", "bind": "n1"}`** as a new change type — deletes the duplicate item, retargets all relations referencing it into the target, preserves token provenance (e.g. a `resolution: "n12 → n1 (same referent)"` note). Symmetric in spirit to `shape`/`reject`; hand-editing the ledger works but is error-prone and loses the resolution note.
205
+ 2. **`extend_scene(ledger, sentence)`** — build the new sentence's ledger with knowledge of the *existing* items: repeated determiner+noun proposes a bind instead of a duplicate (an implied question like `"which cat — the one on the table?"`), "another/a new" forces a fresh item. Pages then become incremental: one call per sentence, one shared ledger.
206
+ 3. Pronoun handling: keep in chat (per SPEC, discussion resolves), but `bind` gives the Space a primitive for it once chat names the referent. Worth a SPEC section: *paragraphs = sentences + bindings.*
207
+
208
+ **Also observed:** Objaverse items now carry `color: null` (SimReady items have thumbnail-derived colours; Objaverse has no thumbnails in this pipeline — a future colour source for Objaverse would close the grey-Objaverse gap).
vli/grammar.py CHANGED
@@ -4,12 +4,19 @@ serial ledger (the decisions reading hides, in reading order).
4
  Nouns come from the lexicon, verbs are relations between meshes, and glue
5
  words (the, a, and) commit no geometry -- determiners open a "which?" question.
6
  A word with no asset is not available unless the caller maps it to a primitive.
 
 
 
 
 
7
  """
8
  import re
9
 
10
  from .lexicon import ASSET_TIERS, LEXICON, PRIMITIVE_SHAPES, TIERS
11
 
12
- DETERMINERS = {"the", "a", "an"}
 
 
13
  CONJUNCTIONS = {"and", "then"}
14
  COPULAS = {"is", "was", "are", "were"}
15
 
@@ -35,7 +42,7 @@ DEFAULT_HEIGHT_M = 0.5
35
 
36
 
37
  def grammar_words():
38
- return sorted(DETERMINERS | CONJUNCTIONS | COPULAS | set(VERBS) | set(PREPOSITIONS))
39
 
40
 
41
  def parse_primitives(primitives):
@@ -80,6 +87,8 @@ def classify_phrase(text):
80
  def classify(word):
81
  if word in DETERMINERS:
82
  return "determiner"
 
 
83
  if word in CONJUNCTIONS:
84
  return "conjunction"
85
  if word in COPULAS:
@@ -137,23 +146,43 @@ def candidate_label(cand):
137
  return cand.get("name") or cand.get("category") or cand.get("shape") or cand.get("prompt") or "?"
138
 
139
 
140
- def build_ledger(sentence, primitives=None):
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
141
  primitives = parse_primitives(primitives)
142
  tokens = tokenize(sentence, primitives)
143
  unknown = [t["text"] for t in tokens if t["kind"] == "unknown"]
144
  if unknown:
145
  raise ValueError(f"Not available (no asset): {', '.join(unknown)}. "
146
  f"Rephrase, or map each to a primitive: {PRIMITIVE_SHAPES}")
 
147
  for t in tokens:
 
148
  t["commits"], t["opens"] = [], []
149
 
150
  items, unresolved = [], []
151
- by_id = {}
 
152
 
153
  def add_item(item, token):
154
  items.append(item)
155
  by_id[item["id"]] = item
156
  token["commits"].append(item["id"])
 
 
157
  return item
158
 
159
  clauses, current = [], []
@@ -168,27 +197,47 @@ def build_ledger(sentence, primitives=None):
168
  pending_q = None
169
  for clause in clauses:
170
  subject, verb_rel, implied_obj, prep_rel, obj, verb_tok = None, None, None, None, None, None
 
171
  for t in clause:
172
  kind = t["kind"]
173
  if kind == "determiner":
174
- pending_q = {"id": f"q{t['index']}", "token": t["index"], "about": None}
175
  t["opens"].append(pending_q["id"])
176
  unresolved.append(pending_q)
177
- elif kind in ("noun", "primitive"):
178
- # height is settled with the asset: metric for simready, nominal otherwise
179
- item = add_item({"id": f"n{t['index']}", "kind": "object", "word": t["text"],
180
- "token": t["index"],
181
- "decision": _object_decision(t["text"], primitives.get(t["text"])),
182
- "height_m": {"value": None, "source": "pending"},
183
- "frozen": False}, t)
184
- if pending_q is not None:
185
- pending_q["about"] = item["id"]
186
- pending_q["question"] = f"which {t['text']}?"
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
187
  pending_q = None
188
- if verb_rel is None and prep_rel is None:
189
- subject = item
 
 
190
  else:
191
- obj = item
192
  elif kind == "verb":
193
  verb_rel, implied_obj = VERBS[t["text"]]
194
  verb_tok = t
@@ -221,14 +270,19 @@ def build_ledger(sentence, primitives=None):
221
  obj = {"id": "scene"}
222
  if obj is None:
223
  raise ValueError(f"'{relation}' needs an object noun")
 
 
224
  add_item({"id": f"r{verb_tok['index']}", "kind": "relation", "relation": relation,
225
  "subject": subject["id"], "object": obj["id"], "token": verb_tok["index"],
226
- "frozen": False}, verb_tok)
227
 
228
  for q in unresolved:
 
229
  if q["about"] is None:
230
  q["question"] = "determiner without a noun"
231
- if not any(i.get("emits_light") for i in items):
 
 
232
  # Seeing needs light, as sitting needs a ground: unstated, so surfaced like the ground.
233
  unresolved.append({"id": "q_light", "token": None, "about": "light", "question": "what light?",
234
  "default_chosen": "daylight", "implied": True,
@@ -237,6 +291,18 @@ def build_ledger(sentence, primitives=None):
237
  "tokens": tokens, "items": items, "unresolved": unresolved}
238
 
239
 
 
 
 
 
 
 
 
 
 
 
 
 
240
  def check_words(sentence, primitives=None):
241
  """Per-word status, the words with no asset, and completions for the word being typed."""
242
  primitives = parse_primitives(primitives)
 
4
  Nouns come from the lexicon, verbs are relations between meshes, and glue
5
  words (the, a, and) commit no geometry -- determiners open a "which?" question.
6
  A word with no asset is not available unless the caller maps it to a primitive.
7
+
8
+ Paragraphs are sentences plus bindings. "the <noun>" when that noun is already in
9
+ the ledger, and pronouns (it, she, they...), refer back instead of adding an item;
10
+ each reference is surfaced as an implied question the discussion can rebind.
11
+ "a/an/another <noun>" always adds a new item.
12
  """
13
  import re
14
 
15
  from .lexicon import ASSET_TIERS, LEXICON, PRIMITIVE_SHAPES, TIERS
16
 
17
+ DETERMINERS = {"the", "a", "an", "another"}
18
+ NEW_DETERMINERS = {"a", "an", "another"} # always introduce a new item
19
+ PRONOUNS = {"it", "he", "she", "they", "him", "her", "them"}
20
  CONJUNCTIONS = {"and", "then"}
21
  COPULAS = {"is", "was", "are", "were"}
22
 
 
42
 
43
 
44
  def grammar_words():
45
+ return sorted(DETERMINERS | CONJUNCTIONS | COPULAS | PRONOUNS | set(VERBS) | set(PREPOSITIONS))
46
 
47
 
48
  def parse_primitives(primitives):
 
87
  def classify(word):
88
  if word in DETERMINERS:
89
  return "determiner"
90
+ if word in PRONOUNS:
91
+ return "pronoun"
92
  if word in CONJUNCTIONS:
93
  return "conjunction"
94
  if word in COPULAS:
 
146
  return cand.get("name") or cand.get("category") or cand.get("shape") or cand.get("prompt") or "?"
147
 
148
 
149
+ def _reference_question(qid, token, word, target, others, determiner=None):
150
+ """An implied question for a word that refers back instead of adding an item."""
151
+ if determiner:
152
+ question, alt = f"which {word}? the one already mentioned", [f"say 'another {word}' for a new one"]
153
+ else:
154
+ question, alt = f"'{word}' = the {target['word']}?", []
155
+ return {"id": qid, "token": token["index"], "about": target["id"], "question": question,
156
+ "default_chosen": f"the {target['word']} ({target['id']})", "implied": True,
157
+ "alternatives": alt + [f"the {o['word']} ({o['id']})" for o in others],
158
+ "binds": {"token": token["index"], "to": target["id"]}}
159
+
160
+
161
+ def build_ledger(sentence, primitives=None, prior=None):
162
+ """The ledger for one sentence. With `prior` (an existing ledger), token indices
163
+ continue after it and words can refer back to its objects; the result holds only
164
+ the new tokens, items and questions (extend_ledger merges them)."""
165
  primitives = parse_primitives(primitives)
166
  tokens = tokenize(sentence, primitives)
167
  unknown = [t["text"] for t in tokens if t["kind"] == "unknown"]
168
  if unknown:
169
  raise ValueError(f"Not available (no asset): {', '.join(unknown)}. "
170
  f"Rephrase, or map each to a primitive: {PRIMITIVE_SHAPES}")
171
+ offset = len(prior["tokens"]) if prior else 0
172
  for t in tokens:
173
+ t["index"] += offset
174
  t["commits"], t["opens"] = [], []
175
 
176
  items, unresolved = [], []
177
+ by_id = {i["id"]: i for i in prior["items"]} if prior else {}
178
+ objects = [i for i in prior["items"] if i["kind"] == "object"] if prior else [] # in reading order
179
 
180
  def add_item(item, token):
181
  items.append(item)
182
  by_id[item["id"]] = item
183
  token["commits"].append(item["id"])
184
+ if item["kind"] == "object":
185
+ objects.append(item)
186
  return item
187
 
188
  clauses, current = [], []
 
197
  pending_q = None
198
  for clause in clauses:
199
  subject, verb_rel, implied_obj, prep_rel, obj, verb_tok = None, None, None, None, None, None
200
+ subject_tok = obj_tok = None
201
  for t in clause:
202
  kind = t["kind"]
203
  if kind == "determiner":
204
+ pending_q = {"id": f"q{t['index']}", "token": t["index"], "about": None, "determiner": t["text"]}
205
  t["opens"].append(pending_q["id"])
206
  unresolved.append(pending_q)
207
+ elif kind in ("noun", "primitive", "pronoun"):
208
+ in_object = subject is not None and (verb_rel is not None or prep_rel is not None)
209
+ if kind == "pronoun":
210
+ pool = [o for o in objects if not (in_object and o is subject)]
211
+ if not pool:
212
+ raise ValueError(f"'{t['text']}' refers to nothing yet: name the object first")
213
+ item = pool[-1] # the most recent object it can mean
214
+ q = _reference_question(f"q{t['index']}", t, t["text"], item, pool[-2::-1])
215
+ t["opens"].append(q["id"])
216
+ unresolved.append(q)
217
+ else:
218
+ same = [o for o in objects if o["word"] == t["text"]]
219
+ if same and pending_q is not None and pending_q["determiner"] not in NEW_DETERMINERS:
220
+ item = same[-1] # "the cat" again: the cat already mentioned
221
+ others = [o for o in objects if o is not item][::-1]
222
+ pending_q.update(_reference_question(pending_q["id"], t, t["text"], item, others,
223
+ determiner=pending_q["determiner"]))
224
+ else:
225
+ # height is settled with the asset: metric for simready, nominal otherwise
226
+ item = add_item({"id": f"n{t['index']}", "kind": "object", "word": t["text"],
227
+ "token": t["index"],
228
+ "decision": _object_decision(t["text"], primitives.get(t["text"])),
229
+ "height_m": {"value": None, "source": "pending"},
230
+ "frozen": False}, t)
231
+ if pending_q is not None:
232
+ pending_q["about"] = item["id"]
233
+ pending_q["question"] = f"which {t['text']}?"
234
  pending_q = None
235
+ if item["id"] not in t["commits"]:
236
+ t["refers_to"] = item["id"]
237
+ if in_object:
238
+ obj, obj_tok = item, t["index"]
239
  else:
240
+ subject, subject_tok = item, t["index"]
241
  elif kind == "verb":
242
  verb_rel, implied_obj = VERBS[t["text"]]
243
  verb_tok = t
 
270
  obj = {"id": "scene"}
271
  if obj is None:
272
  raise ValueError(f"'{relation}' needs an object noun")
273
+ if obj is subject:
274
+ raise ValueError(f"'{verb_tok['text']}' would relate the {subject['word']} to itself")
275
  add_item({"id": f"r{verb_tok['index']}", "kind": "relation", "relation": relation,
276
  "subject": subject["id"], "object": obj["id"], "token": verb_tok["index"],
277
+ "tokens": {"subject": subject_tok, "object": obj_tok}, "frozen": False}, verb_tok)
278
 
279
  for q in unresolved:
280
+ q.pop("determiner", None)
281
  if q["about"] is None:
282
  q["question"] = "determiner without a noun"
283
+ lit = any(i.get("emits_light") for i in by_id.values())
284
+ asked = bool(prior) and any(q["id"] == "q_light" for q in prior["unresolved"])
285
+ if not lit and not asked:
286
  # Seeing needs light, as sitting needs a ground: unstated, so surfaced like the ground.
287
  unresolved.append({"id": "q_light", "token": None, "about": "light", "question": "what light?",
288
  "default_chosen": "daylight", "implied": True,
 
291
  "tokens": tokens, "items": items, "unresolved": unresolved}
292
 
293
 
294
+ def extend_ledger(ledger, sentence, primitives=None):
295
+ """Add a sentence to a ledger: the next sentence of a paragraph."""
296
+ prims = {**ledger.get("primitives", {}), **parse_primitives(primitives)}
297
+ new = build_ledger(sentence, prims, prior=ledger)
298
+ lit = any(i.get("emits_light") for i in ledger["items"] + new["items"])
299
+ unresolved = [q for q in ledger["unresolved"] + new["unresolved"] if not (lit and q["id"] == "q_light")]
300
+ return {**ledger, "sentence": ledger["sentence"].rstrip(" .") + ". " + sentence.strip(),
301
+ "sentences": ledger.get("sentences", [ledger["sentence"]]) + [sentence.strip()],
302
+ "primitives": prims, "tokens": ledger["tokens"] + new["tokens"],
303
+ "items": ledger["items"] + new["items"], "unresolved": unresolved}
304
+
305
+
306
  def check_words(sentence, primitives=None):
307
  """Per-word status, the words with no asset, and completions for the word being typed."""
308
  primitives = parse_primitives(primitives)
vli/pipeline.py CHANGED
@@ -8,7 +8,8 @@ import os
8
  import tempfile
9
 
10
  from . import assets, html, scene
11
- from .grammar import POOL, build_ledger, candidate_label, next_tier, parse_primitives, primitive_candidate
 
12
  from .lexicon import ASSET_TIERS, LEXICON
13
 
14
 
@@ -45,6 +46,8 @@ def settle_all(ledger):
45
  meshes[it["id"]] = assets.scene_mesh(full)
46
  it["decision"]["label"] = candidate_label(it["decision"]["chosen"])
47
  for q in ledger["unresolved"]:
 
 
48
  about = next((i for i in ledger["items"] if i["id"] == q.get("about")), None)
49
  if about and about["kind"] == "object":
50
  q["default_chosen"] = _option_label(about["decision"]["chosen"])
@@ -67,6 +70,67 @@ def translate(sentence, primitives=None, seed=0):
67
  return outputs(build_ledger(sentence, primitives), seed)
68
 
69
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
70
  def apply_changes(ledger, changes):
71
  """changes: list of
72
  {"id": ..., "freeze": true|false}
@@ -75,11 +139,17 @@ def apply_changes(ledger, changes):
75
  {"id": ..., "height_m": 0.4}
76
  {"id": ..., "shape": "sphere"} -> primitive tier with that shape
77
  {"reroll": true} -> every unfrozen object takes its next alternative
 
 
78
  Reroll walks the pool (the old choice goes to the back), so repeated rerolls
79
  visit every alternative before repeating, independent of the seed.
80
  """
81
  by_id = {i["id"]: i for i in ledger["items"]}
82
  for ch in changes:
 
 
 
 
83
  if ch.get("reroll"):
84
  for it in ledger["items"]:
85
  dec = it.get("decision")
 
8
  import tempfile
9
 
10
  from . import assets, html, scene
11
+ from .grammar import (POOL, build_ledger, candidate_label, extend_ledger, next_tier, parse_primitives,
12
+ primitive_candidate)
13
  from .lexicon import ASSET_TIERS, LEXICON
14
 
15
 
 
46
  meshes[it["id"]] = assets.scene_mesh(full)
47
  it["decision"]["label"] = candidate_label(it["decision"]["chosen"])
48
  for q in ledger["unresolved"]:
49
+ if q.get("binds"):
50
+ continue # a reference: which item a word means, not which asset
51
  about = next((i for i in ledger["items"] if i["id"] == q.get("about")), None)
52
  if about and about["kind"] == "object":
53
  q["default_chosen"] = _option_label(about["decision"]["chosen"])
 
70
  return outputs(build_ledger(sentence, primitives), seed)
71
 
72
 
73
+ def extend(ledger, sentence, primitives=None, seed=0):
74
+ return outputs(extend_ledger(ledger, sentence, primitives), seed)
75
+
76
+
77
+ def _bind(ledger, src, dst):
78
+ """Two words mean one thing. `src` is a duplicate object (merged into `dst`) or a
79
+ reference question (its word now means `dst`). Recorded in ledger["resolutions"]."""
80
+ by_id = {i["id"]: i for i in ledger["items"]}
81
+ target = by_id.get(dst)
82
+ if target is None or target["kind"] != "object":
83
+ raise ValueError(f"bind target {dst!r} is not an object item")
84
+ relations = [i for i in ledger["items"] if i["kind"] == "relation"]
85
+ q = next((q for q in ledger["unresolved"] if q["id"] == src and q.get("binds")), None)
86
+ if q is not None:
87
+ tok = q["binds"]["token"]
88
+ moves = [(r, role) for r in relations for role in ("subject", "object")
89
+ if r.get("tokens", {}).get(role) == tok]
90
+ word, old = ledger["tokens"][tok]["text"] if tok < len(ledger["tokens"]) else "?", q["binds"]["to"]
91
+ else:
92
+ it = by_id.get(src)
93
+ if it is None or it["kind"] != "object":
94
+ raise ValueError(f"{src!r} is neither an object item nor a reference question")
95
+ if it is target:
96
+ raise ValueError(f"{src} is already {dst}")
97
+ if it["frozen"]:
98
+ raise ValueError(f"{src} is frozen; unfreeze it first")
99
+ moves = [(r, role) for r in relations for role in ("subject", "object") if r[role] == src]
100
+ word, old = it["word"], src
101
+ for r, role in moves: # check before changing anything
102
+ if r["object" if role == "subject" else "subject"] == dst:
103
+ raise ValueError(f"binding {src} to {dst} would relate the {target['word']} to itself ({r['id']})")
104
+ for r, role in moves:
105
+ r[role] = dst
106
+ if q is not None:
107
+ ledger["tokens"][tok]["refers_to"] = dst
108
+ ledger["unresolved"].remove(q)
109
+ else:
110
+ ledger["items"].remove(it)
111
+ if it.get("emits_light"):
112
+ target["emits_light"] = True
113
+ for t in ledger["tokens"]:
114
+ if src in t["commits"]:
115
+ t["commits"].remove(src)
116
+ t["refers_to"] = dst
117
+ elif t.get("refers_to") == src:
118
+ t["refers_to"] = dst
119
+ kept = []
120
+ for u in ledger["unresolved"]:
121
+ if u.get("about") == src and not u.get("binds"):
122
+ continue # "which cat?" for the duplicate: the duplicate is gone
123
+ if u.get("binds", {}).get("to") == src:
124
+ u["binds"]["to"], u["about"], u["default_chosen"] = dst, dst, f"the {target['word']} ({dst})"
125
+ kept.append(u)
126
+ ledger["unresolved"] = kept
127
+ live = {u["id"] for u in ledger["unresolved"]}
128
+ for t in ledger["tokens"]: # answered questions leave the reading too; the resolution note keeps them
129
+ t["opens"] = [o for o in t.get("opens", []) if o in live]
130
+ ledger.setdefault("resolutions", []).append(
131
+ {"word": word, "from": old, "to": dst, "note": f"{old} -> {dst} (same referent)"})
132
+
133
+
134
  def apply_changes(ledger, changes):
135
  """changes: list of
136
  {"id": ..., "freeze": true|false}
 
139
  {"id": ..., "height_m": 0.4}
140
  {"id": ..., "shape": "sphere"} -> primitive tier with that shape
141
  {"reroll": true} -> every unfrozen object takes its next alternative
142
+ {"id": "n12", "bind": "n1"} -> n12 is the same thing as n1: merged into it
143
+ {"id": "q11", "bind": "n5"} -> the word behind reference question q11 means n5
144
  Reroll walks the pool (the old choice goes to the back), so repeated rerolls
145
  visit every alternative before repeating, independent of the seed.
146
  """
147
  by_id = {i["id"]: i for i in ledger["items"]}
148
  for ch in changes:
149
+ if "bind" in ch:
150
+ _bind(ledger, ch.get("id"), ch["bind"])
151
+ by_id = {i["id"]: i for i in ledger["items"]}
152
+ continue
153
  if ch.get("reroll"):
154
  for it in ledger["items"]:
155
  dec = it.get("decision")