Spaces:
Running on Zero
Paragraphs: extend_scene, references that bind to earlier items, and a bind change
Browse filesFrom QA round 10 (paragraph experiments) and the user's choice of defaults:
- "the cat" when a cat already exists refers to it instead of making a twin,
surfaced as "which cat? the one already mentioned". "a/an/another cat"
always makes a new one. Fixes the twin-cat scene where the apple sat
beside a second cat on the floor.
- Pronouns (it, he, she, they, him, her, them) are words now: they refer to
the most recent object (not the clause's own subject), surfaced as
"'it' = the table?" with the alternatives.
- extend_scene(ledger, sentence, ...) adds the next sentence to a ledger.
- refine_scene takes {"id": "q10", "bind": "n1"} to rebind a reference and
{"id": "n12", "bind": "n1"} to merge a duplicate, with a resolution note.
Tested with a stub dictionary: one sentence with the cat twice, pronoun
across sentences, rebind, another cat, duplicate merge, self-relation
refused, pronoun with nothing before it, light question dropped.
Also archives QA rounds 9-10.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- SPEC.md +24 -0
- app.py +36 -0
- feedback/vli-qa-feedback-2026-10-30.md +24 -0
- vli/grammar.py +86 -20
- vli/pipeline.py +71 -1
|
@@ -103,6 +103,29 @@ Relations: `on, under, beside, near, behind, in_front_of, above, faces, lights`.
|
|
| 103 |
Verbs: `sat/stood/lay/rested` → `on` (ground implied), `shone/glowed` → `lights`
|
| 104 |
(shadows only turn on with this coupling), `faced` → `faces`.
|
| 105 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 106 |
## MCP tools
|
| 107 |
|
| 108 |
| Tool | In → out |
|
|
@@ -110,6 +133,7 @@ Verbs: `sat/stood/lay/rested` → `on` (ground implied), `shone/glowed` → `lig
|
|
| 110 |
| `check_words` | sentence, primitives → per-word status, missing words, completions |
|
| 111 |
| `translate_to_scene` | sentence, primitives, seed, html → ledger, PNG, GLB, mini HTML file, mini HTML text |
|
| 112 |
| `refine_scene` | ledger, changes, seed, html → ledger, PNG, GLB, mini HTML file, mini HTML text |
|
|
|
|
| 113 |
| `simplify_item` | ledger, item id, html, rings, samples → report, rung PNGs, rung GLBs, mini HTML file, mini HTML text |
|
| 114 |
| `papercraft_rings` | ledger, item id, rings, samples, height_cm → pieces report, PNG per A4 page (to look at), SVG per A4 page (mm, to print) |
|
| 115 |
|
|
|
|
| 103 |
Verbs: `sat/stood/lay/rested` → `on` (ground implied), `shone/glowed` → `lights`
|
| 104 |
(shadows only turn on with this coupling), `faced` → `faces`.
|
| 105 |
|
| 106 |
+
## Paragraphs: sentences + bindings
|
| 107 |
+
|
| 108 |
+
`extend_scene` adds the next sentence to an existing ledger (token and item ids carry
|
| 109 |
+
on, `sentences` lists them). Within a sentence or across sentences:
|
| 110 |
+
|
| 111 |
+
- **"the <noun>"** when that noun is already in the ledger refers to it (the most
|
| 112 |
+
recent one) instead of adding an item. Surfaced as an implied question: "which cat?
|
| 113 |
+
the one already mentioned" (default: the same cat), alternatives include "say
|
| 114 |
+
'another cat' for a new one".
|
| 115 |
+
- **"a/an/another <noun>"** always adds a new item.
|
| 116 |
+
- **Pronouns** (it, he, she, they, him, her, them) refer to the most recent object,
|
| 117 |
+
leaving out the clause's own subject when the pronoun is its object ("the apple lay
|
| 118 |
+
beside it" can't mean the apple). Surfaced as "'it' = the table?", with the other
|
| 119 |
+
objects as alternatives, most recent first. A pronoun with nothing before it is an
|
| 120 |
+
error.
|
| 121 |
+
- Each reference question carries `binds: {token, to}`; relations record which token
|
| 122 |
+
named each end (`tokens: {subject, object}`), so a reference can be rebound.
|
| 123 |
+
- **`bind`** (a `refine_scene` change): `{"id": "q10", "bind": "n1"}` rebinds a
|
| 124 |
+
reference question; `{"id": "n12", "bind": "n1"}` merges a duplicate object into
|
| 125 |
+
another (relations move, its "which?" question goes, a light carries over). Both
|
| 126 |
+
refuse to relate an object to itself and add a note to `resolutions`.
|
| 127 |
+
- "what light?" is asked once per ledger and dropped when a later sentence lights the scene.
|
| 128 |
+
|
| 129 |
## MCP tools
|
| 130 |
|
| 131 |
| Tool | In → out |
|
|
|
|
| 133 |
| `check_words` | sentence, primitives → per-word status, missing words, completions |
|
| 134 |
| `translate_to_scene` | sentence, primitives, seed, html → ledger, PNG, GLB, mini HTML file, mini HTML text |
|
| 135 |
| `refine_scene` | ledger, changes, seed, html → ledger, PNG, GLB, mini HTML file, mini HTML text |
|
| 136 |
+
| `extend_scene` | ledger, sentence, primitives, seed, html → ledger, PNG, GLB, mini HTML file, mini HTML text |
|
| 137 |
| `simplify_item` | ledger, item id, html, rings, samples → report, rung PNGs, rung GLBs, mini HTML file, mini HTML text |
|
| 138 |
| `papercraft_rings` | ledger, item id, rings, samples, height_cm → pieces report, PNG per A4 page (to look at), SVG per A4 page (mm, to print) |
|
| 139 |
|
|
@@ -157,6 +157,8 @@ def refine_scene(ledger_json: str, changes_json: str = "[]", seed: str = "0",
|
|
| 157 |
{"id": "n1", "shape": "sphere"} use a primitive (box, sphere, cylinder, cone, plane)
|
| 158 |
{"reroll": true} every unfrozen object takes its next alternative;
|
| 159 |
repeated rerolls visit the whole pool, whatever the seed
|
|
|
|
|
|
|
| 160 |
Frozen items cannot be changed. The configuration (placement, sun angle) is always
|
| 161 |
re-chosen from `seed`: change the seed for another valid configuration.
|
| 162 |
|
|
@@ -179,6 +181,34 @@ def refine_scene(ledger_json: str, changes_json: str = "[]", seed: str = "0",
|
|
| 179 |
_fail(e)
|
| 180 |
|
| 181 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 182 |
def simplify_item(ledger_json: str, item_id: str, html: str = "url", rings: str = "8",
|
| 183 |
samples: str = "16") -> tuple[str, list, list, str, str]:
|
| 184 |
"""Build the simplification ladder for one object: full asset -> fewer faces -> rings -> convex hull -> box.
|
|
@@ -382,6 +412,9 @@ with gr.Blocks(title="VisualLearningIdea") as demo:
|
|
| 382 |
ledger = gr.Code(label="Ledger", language="json", interactive=True)
|
| 383 |
changes = gr.Code(label="Changes", language="json", value=EXAMPLE_CHANGES, interactive=True)
|
| 384 |
refine = gr.Button("Refine")
|
|
|
|
|
|
|
|
|
|
| 385 |
with gr.Accordion("Simplification ladder", open=False):
|
| 386 |
with gr.Row():
|
| 387 |
item_id = gr.Textbox(label="Object item id", value="n1")
|
|
@@ -412,6 +445,9 @@ with gr.Blocks(title="VisualLearningIdea") as demo:
|
|
| 412 |
refine.click(refine_scene, [ledger, changes, seed, html_mode],
|
| 413 |
[ledger, picture, model, scene_file, scene_html], api_name="refine_scene") \
|
| 414 |
.then(_html_preview, scene_file, scene_preview, api_visibility="private")
|
|
|
|
|
|
|
|
|
|
| 415 |
simp.click(simplify_item, [ledger, item_id, html_mode, ring_count, ring_samples],
|
| 416 |
[ladder_report, ladder_pics, ladder_files, ladder_file, ladder_html], api_name="simplify_item") \
|
| 417 |
.then(_html_preview, ladder_file, ladder_preview, api_visibility="private")
|
|
|
|
| 157 |
{"id": "n1", "shape": "sphere"} use a primitive (box, sphere, cylinder, cone, plane)
|
| 158 |
{"reroll": true} every unfrozen object takes its next alternative;
|
| 159 |
repeated rerolls visit the whole pool, whatever the seed
|
| 160 |
+
{"id": "n12", "bind": "n1"} n12 is the same thing as n1: merged into it, its relations move
|
| 161 |
+
{"id": "q11", "bind": "n5"} the word behind reference question q11 ("it", "the cat") means n5
|
| 162 |
Frozen items cannot be changed. The configuration (placement, sun angle) is always
|
| 163 |
re-chosen from `seed`: change the seed for another valid configuration.
|
| 164 |
|
|
|
|
| 181 |
_fail(e)
|
| 182 |
|
| 183 |
|
| 184 |
+
def extend_scene(ledger_json: str, sentence: str, primitives_json: str = "{}", seed: str = "0",
|
| 185 |
+
html: str = "url") -> tuple[str, str, str, str, str]:
|
| 186 |
+
"""Add the next sentence of a paragraph to an existing scene.
|
| 187 |
+
|
| 188 |
+
Paragraphs are sentences plus bindings. "the cat" when a cat is already in the ledger,
|
| 189 |
+
and pronouns (it, he, she, they), refer back to an existing item instead of adding one;
|
| 190 |
+
each such reference is an implied question ("which cat? the one already mentioned",
|
| 191 |
+
"'it' = the cat?") whose `binds` names the item. Rebind with refine_scene
|
| 192 |
+
{"id": "<question id>", "bind": "<item id>"}. "a/an/another cat" always adds a new one.
|
| 193 |
+
|
| 194 |
+
Args:
|
| 195 |
+
ledger_json: Ledger JSON from translate_to_scene, refine_scene or a previous extend_scene.
|
| 196 |
+
sentence: The next sentence, e.g. "the apple lay beside it".
|
| 197 |
+
primitives_json: JSON object mapping missing words to shapes, as in translate_to_scene.
|
| 198 |
+
seed: Picks one of the configurations that satisfy the whole ledger.
|
| 199 |
+
html: "url" (default), "inline" or "skip", as in translate_to_scene.
|
| 200 |
+
|
| 201 |
+
Returns:
|
| 202 |
+
(ledger JSON string, rendered PNG, GLB scene file, mini HTML file, mini HTML text or "").
|
| 203 |
+
"""
|
| 204 |
+
try:
|
| 205 |
+
ledger = _json(ledger_json, "ledger_json")
|
| 206 |
+
*rest, page = pipeline.extend(ledger, sentence, _json(primitives_json, "primitives_json", {}), _seed(seed))
|
| 207 |
+
return (*rest, *_html_out(page, html))
|
| 208 |
+
except Exception as e:
|
| 209 |
+
_fail(e)
|
| 210 |
+
|
| 211 |
+
|
| 212 |
def simplify_item(ledger_json: str, item_id: str, html: str = "url", rings: str = "8",
|
| 213 |
samples: str = "16") -> tuple[str, list, list, str, str]:
|
| 214 |
"""Build the simplification ladder for one object: full asset -> fewer faces -> rings -> convex hull -> box.
|
|
|
|
| 412 |
ledger = gr.Code(label="Ledger", language="json", interactive=True)
|
| 413 |
changes = gr.Code(label="Changes", language="json", value=EXAMPLE_CHANGES, interactive=True)
|
| 414 |
refine = gr.Button("Refine")
|
| 415 |
+
with gr.Row():
|
| 416 |
+
next_sentence = gr.Textbox(label="Next sentence (refers back: 'the cat', 'it')", scale=4)
|
| 417 |
+
extend = gr.Button("Add sentence", scale=1)
|
| 418 |
with gr.Accordion("Simplification ladder", open=False):
|
| 419 |
with gr.Row():
|
| 420 |
item_id = gr.Textbox(label="Object item id", value="n1")
|
|
|
|
| 445 |
refine.click(refine_scene, [ledger, changes, seed, html_mode],
|
| 446 |
[ledger, picture, model, scene_file, scene_html], api_name="refine_scene") \
|
| 447 |
.then(_html_preview, scene_file, scene_preview, api_visibility="private")
|
| 448 |
+
extend.click(extend_scene, [ledger, next_sentence, primitives, seed, html_mode],
|
| 449 |
+
[ledger, picture, model, scene_file, scene_html], api_name="extend_scene") \
|
| 450 |
+
.then(_html_preview, scene_file, scene_preview, api_visibility="private")
|
| 451 |
simp.click(simplify_item, [ledger, item_id, html_mode, ring_count, ring_samples],
|
| 452 |
[ladder_report, ladder_pics, ladder_files, ladder_file, ladder_html], api_name="simplify_item") \
|
| 453 |
.then(_html_preview, ladder_file, ladder_preview, api_visibility="private")
|
|
@@ -182,3 +182,27 @@ Note: apple `b01` is a reddish-brown (#b27045) — "a green apple" still means w
|
|
| 182 |
- **Bug: `top cap` has `glues_to: []`** — its glue edge to band 7 is missing from the assembly graph (bottom cap correctly glues to band 1). Small fix in the glue-path builder; an assembler would otherwise hold an unlabeled lid.
|
| 183 |
|
| 184 |
> Note (agent): not a missing edge — tabs sit on one side of each joint, and the top cap's are on band 7's top edge. The report listed only outgoing tabs; it now also lists `receives_from`, so the top cap shows band 7a–7c.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 182 |
- **Bug: `top cap` has `glues_to: []`** — its glue edge to band 7 is missing from the assembly graph (bottom cap correctly glues to band 1). Small fix in the glue-path builder; an assembler would otherwise hold an unlabeled lid.
|
| 183 |
|
| 184 |
> Note (agent): not a missing edge — tabs sit on one side of each joint, and the top cap's are on band 7's top edge. The report listed only outgoing tabs; it now also lists `receives_from`, so the top cap shows band 7a–7c.
|
| 185 |
+
|
| 186 |
+
Remaining open: the `use`-back-swap check after skip-shifted indices, and the UI visual click-tests (mode radio, preview-from-file, papercraft download). The sentence → scene → ladder → printable-kit loop is now complete and demo-ready end to end.
|
| 187 |
+
|
| 188 |
+
---
|
| 189 |
+
|
| 190 |
+
## 10. Paragraph / disambiguation experiments (2026-10-30) — the next frontier, probed live
|
| 191 |
+
|
| 192 |
+
The goal going forward is the **learning-disambiguation tool**: paragraphs and pages, where later sentences refer back to earlier entities. Four attempts:
|
| 193 |
+
|
| 194 |
+
| # | Input | Result |
|
| 195 |
+
|---|---|---|
|
| 196 |
+
| 0 | `{"id":"n1","use":0}` on a pool with a skip-shifted index | ✓ chosen resolved to `alternatives[0]` correctly — `use` is sound. **Item closed** with the rule: indices are relative to the ledger you pass; callers must reuse the latest returned ledger. (UI click-tests also cleared by the reviewer.) |
|
| 197 |
+
| 1 | `"the cat sat on the table and the apple lay beside the cat and the lamp glowed"` (raw, repeated noun) | Structure ✓ — three clauses in one ledger, all relations intact, lamp lights the scene. Semantics ✗ — **two cat items (n1, n12)**, two "which cat?" questions; "beside the cat" bound to the *ground-level* twin, so the apple sat on the floor while the real cat was on the table (render proves it: cat on tabletop, second cat + apple on the ground) |
|
| 198 |
+
| 2 | `"…and the apple lay beside it…"` (pronoun) | ✗ hard fail: `Not available (no asset): it` — pronouns are unknown words |
|
| 199 |
+
| 3 | Attempt 1 hand-resolved from chat: `r10` bound `n12 → n1`, duplicate item + its question deleted, provenance tokens blanked; re-rendered via `refine_scene` with empty changes | ✓ **coherent page**: one cat on the table, apple beside it at the cat's height (both y = 0.916 — the `beside` solver inherited the table level automatically), lamp glowing. The reference-driven solver follows the fix |
|
| 200 |
+
|
| 201 |
+
**What this establishes:** the chat-side disambiguation loop works today with zero Space changes — *chat resolves, ledger persists, Space re-renders*. That is the learning-disambiguation tool's core loop, demonstrated.
|
| 202 |
+
|
| 203 |
+
**Proposed Space changes to make it first-class (next push candidates):**
|
| 204 |
+
1. **`{"id": "n12", "bind": "n1"}`** as a new change type — deletes the duplicate item, retargets all relations referencing it into the target, preserves token provenance (e.g. a `resolution: "n12 → n1 (same referent)"` note). Symmetric in spirit to `shape`/`reject`; hand-editing the ledger works but is error-prone and loses the resolution note.
|
| 205 |
+
2. **`extend_scene(ledger, sentence)`** — build the new sentence's ledger with knowledge of the *existing* items: repeated determiner+noun proposes a bind instead of a duplicate (an implied question like `"which cat — the one on the table?"`), "another/a new" forces a fresh item. Pages then become incremental: one call per sentence, one shared ledger.
|
| 206 |
+
3. Pronoun handling: keep in chat (per SPEC, discussion resolves), but `bind` gives the Space a primitive for it once chat names the referent. Worth a SPEC section: *paragraphs = sentences + bindings.*
|
| 207 |
+
|
| 208 |
+
**Also observed:** Objaverse items now carry `color: null` (SimReady items have thumbnail-derived colours; Objaverse has no thumbnails in this pipeline — a future colour source for Objaverse would close the grey-Objaverse gap).
|
|
@@ -4,12 +4,19 @@ serial ledger (the decisions reading hides, in reading order).
|
|
| 4 |
Nouns come from the lexicon, verbs are relations between meshes, and glue
|
| 5 |
words (the, a, and) commit no geometry -- determiners open a "which?" question.
|
| 6 |
A word with no asset is not available unless the caller maps it to a primitive.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 7 |
"""
|
| 8 |
import re
|
| 9 |
|
| 10 |
from .lexicon import ASSET_TIERS, LEXICON, PRIMITIVE_SHAPES, TIERS
|
| 11 |
|
| 12 |
-
DETERMINERS = {"the", "a", "an"}
|
|
|
|
|
|
|
| 13 |
CONJUNCTIONS = {"and", "then"}
|
| 14 |
COPULAS = {"is", "was", "are", "were"}
|
| 15 |
|
|
@@ -35,7 +42,7 @@ DEFAULT_HEIGHT_M = 0.5
|
|
| 35 |
|
| 36 |
|
| 37 |
def grammar_words():
|
| 38 |
-
return sorted(DETERMINERS | CONJUNCTIONS | COPULAS | set(VERBS) | set(PREPOSITIONS))
|
| 39 |
|
| 40 |
|
| 41 |
def parse_primitives(primitives):
|
|
@@ -80,6 +87,8 @@ def classify_phrase(text):
|
|
| 80 |
def classify(word):
|
| 81 |
if word in DETERMINERS:
|
| 82 |
return "determiner"
|
|
|
|
|
|
|
| 83 |
if word in CONJUNCTIONS:
|
| 84 |
return "conjunction"
|
| 85 |
if word in COPULAS:
|
|
@@ -137,23 +146,43 @@ def candidate_label(cand):
|
|
| 137 |
return cand.get("name") or cand.get("category") or cand.get("shape") or cand.get("prompt") or "?"
|
| 138 |
|
| 139 |
|
| 140 |
-
def
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 141 |
primitives = parse_primitives(primitives)
|
| 142 |
tokens = tokenize(sentence, primitives)
|
| 143 |
unknown = [t["text"] for t in tokens if t["kind"] == "unknown"]
|
| 144 |
if unknown:
|
| 145 |
raise ValueError(f"Not available (no asset): {', '.join(unknown)}. "
|
| 146 |
f"Rephrase, or map each to a primitive: {PRIMITIVE_SHAPES}")
|
|
|
|
| 147 |
for t in tokens:
|
|
|
|
| 148 |
t["commits"], t["opens"] = [], []
|
| 149 |
|
| 150 |
items, unresolved = [], []
|
| 151 |
-
by_id = {}
|
|
|
|
| 152 |
|
| 153 |
def add_item(item, token):
|
| 154 |
items.append(item)
|
| 155 |
by_id[item["id"]] = item
|
| 156 |
token["commits"].append(item["id"])
|
|
|
|
|
|
|
| 157 |
return item
|
| 158 |
|
| 159 |
clauses, current = [], []
|
|
@@ -168,27 +197,47 @@ def build_ledger(sentence, primitives=None):
|
|
| 168 |
pending_q = None
|
| 169 |
for clause in clauses:
|
| 170 |
subject, verb_rel, implied_obj, prep_rel, obj, verb_tok = None, None, None, None, None, None
|
|
|
|
| 171 |
for t in clause:
|
| 172 |
kind = t["kind"]
|
| 173 |
if kind == "determiner":
|
| 174 |
-
pending_q = {"id": f"q{t['index']}", "token": t["index"], "about": None}
|
| 175 |
t["opens"].append(pending_q["id"])
|
| 176 |
unresolved.append(pending_q)
|
| 177 |
-
elif kind in ("noun", "primitive"):
|
| 178 |
-
|
| 179 |
-
|
| 180 |
-
|
| 181 |
-
|
| 182 |
-
|
| 183 |
-
|
| 184 |
-
|
| 185 |
-
|
| 186 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 187 |
pending_q = None
|
| 188 |
-
if
|
| 189 |
-
|
|
|
|
|
|
|
| 190 |
else:
|
| 191 |
-
|
| 192 |
elif kind == "verb":
|
| 193 |
verb_rel, implied_obj = VERBS[t["text"]]
|
| 194 |
verb_tok = t
|
|
@@ -221,14 +270,19 @@ def build_ledger(sentence, primitives=None):
|
|
| 221 |
obj = {"id": "scene"}
|
| 222 |
if obj is None:
|
| 223 |
raise ValueError(f"'{relation}' needs an object noun")
|
|
|
|
|
|
|
| 224 |
add_item({"id": f"r{verb_tok['index']}", "kind": "relation", "relation": relation,
|
| 225 |
"subject": subject["id"], "object": obj["id"], "token": verb_tok["index"],
|
| 226 |
-
"frozen": False}, verb_tok)
|
| 227 |
|
| 228 |
for q in unresolved:
|
|
|
|
| 229 |
if q["about"] is None:
|
| 230 |
q["question"] = "determiner without a noun"
|
| 231 |
-
|
|
|
|
|
|
|
| 232 |
# Seeing needs light, as sitting needs a ground: unstated, so surfaced like the ground.
|
| 233 |
unresolved.append({"id": "q_light", "token": None, "about": "light", "question": "what light?",
|
| 234 |
"default_chosen": "daylight", "implied": True,
|
|
@@ -237,6 +291,18 @@ def build_ledger(sentence, primitives=None):
|
|
| 237 |
"tokens": tokens, "items": items, "unresolved": unresolved}
|
| 238 |
|
| 239 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 240 |
def check_words(sentence, primitives=None):
|
| 241 |
"""Per-word status, the words with no asset, and completions for the word being typed."""
|
| 242 |
primitives = parse_primitives(primitives)
|
|
|
|
| 4 |
Nouns come from the lexicon, verbs are relations between meshes, and glue
|
| 5 |
words (the, a, and) commit no geometry -- determiners open a "which?" question.
|
| 6 |
A word with no asset is not available unless the caller maps it to a primitive.
|
| 7 |
+
|
| 8 |
+
Paragraphs are sentences plus bindings. "the <noun>" when that noun is already in
|
| 9 |
+
the ledger, and pronouns (it, she, they...), refer back instead of adding an item;
|
| 10 |
+
each reference is surfaced as an implied question the discussion can rebind.
|
| 11 |
+
"a/an/another <noun>" always adds a new item.
|
| 12 |
"""
|
| 13 |
import re
|
| 14 |
|
| 15 |
from .lexicon import ASSET_TIERS, LEXICON, PRIMITIVE_SHAPES, TIERS
|
| 16 |
|
| 17 |
+
DETERMINERS = {"the", "a", "an", "another"}
|
| 18 |
+
NEW_DETERMINERS = {"a", "an", "another"} # always introduce a new item
|
| 19 |
+
PRONOUNS = {"it", "he", "she", "they", "him", "her", "them"}
|
| 20 |
CONJUNCTIONS = {"and", "then"}
|
| 21 |
COPULAS = {"is", "was", "are", "were"}
|
| 22 |
|
|
|
|
| 42 |
|
| 43 |
|
| 44 |
def grammar_words():
|
| 45 |
+
return sorted(DETERMINERS | CONJUNCTIONS | COPULAS | PRONOUNS | set(VERBS) | set(PREPOSITIONS))
|
| 46 |
|
| 47 |
|
| 48 |
def parse_primitives(primitives):
|
|
|
|
| 87 |
def classify(word):
|
| 88 |
if word in DETERMINERS:
|
| 89 |
return "determiner"
|
| 90 |
+
if word in PRONOUNS:
|
| 91 |
+
return "pronoun"
|
| 92 |
if word in CONJUNCTIONS:
|
| 93 |
return "conjunction"
|
| 94 |
if word in COPULAS:
|
|
|
|
| 146 |
return cand.get("name") or cand.get("category") or cand.get("shape") or cand.get("prompt") or "?"
|
| 147 |
|
| 148 |
|
| 149 |
+
def _reference_question(qid, token, word, target, others, determiner=None):
|
| 150 |
+
"""An implied question for a word that refers back instead of adding an item."""
|
| 151 |
+
if determiner:
|
| 152 |
+
question, alt = f"which {word}? the one already mentioned", [f"say 'another {word}' for a new one"]
|
| 153 |
+
else:
|
| 154 |
+
question, alt = f"'{word}' = the {target['word']}?", []
|
| 155 |
+
return {"id": qid, "token": token["index"], "about": target["id"], "question": question,
|
| 156 |
+
"default_chosen": f"the {target['word']} ({target['id']})", "implied": True,
|
| 157 |
+
"alternatives": alt + [f"the {o['word']} ({o['id']})" for o in others],
|
| 158 |
+
"binds": {"token": token["index"], "to": target["id"]}}
|
| 159 |
+
|
| 160 |
+
|
| 161 |
+
def build_ledger(sentence, primitives=None, prior=None):
|
| 162 |
+
"""The ledger for one sentence. With `prior` (an existing ledger), token indices
|
| 163 |
+
continue after it and words can refer back to its objects; the result holds only
|
| 164 |
+
the new tokens, items and questions (extend_ledger merges them)."""
|
| 165 |
primitives = parse_primitives(primitives)
|
| 166 |
tokens = tokenize(sentence, primitives)
|
| 167 |
unknown = [t["text"] for t in tokens if t["kind"] == "unknown"]
|
| 168 |
if unknown:
|
| 169 |
raise ValueError(f"Not available (no asset): {', '.join(unknown)}. "
|
| 170 |
f"Rephrase, or map each to a primitive: {PRIMITIVE_SHAPES}")
|
| 171 |
+
offset = len(prior["tokens"]) if prior else 0
|
| 172 |
for t in tokens:
|
| 173 |
+
t["index"] += offset
|
| 174 |
t["commits"], t["opens"] = [], []
|
| 175 |
|
| 176 |
items, unresolved = [], []
|
| 177 |
+
by_id = {i["id"]: i for i in prior["items"]} if prior else {}
|
| 178 |
+
objects = [i for i in prior["items"] if i["kind"] == "object"] if prior else [] # in reading order
|
| 179 |
|
| 180 |
def add_item(item, token):
|
| 181 |
items.append(item)
|
| 182 |
by_id[item["id"]] = item
|
| 183 |
token["commits"].append(item["id"])
|
| 184 |
+
if item["kind"] == "object":
|
| 185 |
+
objects.append(item)
|
| 186 |
return item
|
| 187 |
|
| 188 |
clauses, current = [], []
|
|
|
|
| 197 |
pending_q = None
|
| 198 |
for clause in clauses:
|
| 199 |
subject, verb_rel, implied_obj, prep_rel, obj, verb_tok = None, None, None, None, None, None
|
| 200 |
+
subject_tok = obj_tok = None
|
| 201 |
for t in clause:
|
| 202 |
kind = t["kind"]
|
| 203 |
if kind == "determiner":
|
| 204 |
+
pending_q = {"id": f"q{t['index']}", "token": t["index"], "about": None, "determiner": t["text"]}
|
| 205 |
t["opens"].append(pending_q["id"])
|
| 206 |
unresolved.append(pending_q)
|
| 207 |
+
elif kind in ("noun", "primitive", "pronoun"):
|
| 208 |
+
in_object = subject is not None and (verb_rel is not None or prep_rel is not None)
|
| 209 |
+
if kind == "pronoun":
|
| 210 |
+
pool = [o for o in objects if not (in_object and o is subject)]
|
| 211 |
+
if not pool:
|
| 212 |
+
raise ValueError(f"'{t['text']}' refers to nothing yet: name the object first")
|
| 213 |
+
item = pool[-1] # the most recent object it can mean
|
| 214 |
+
q = _reference_question(f"q{t['index']}", t, t["text"], item, pool[-2::-1])
|
| 215 |
+
t["opens"].append(q["id"])
|
| 216 |
+
unresolved.append(q)
|
| 217 |
+
else:
|
| 218 |
+
same = [o for o in objects if o["word"] == t["text"]]
|
| 219 |
+
if same and pending_q is not None and pending_q["determiner"] not in NEW_DETERMINERS:
|
| 220 |
+
item = same[-1] # "the cat" again: the cat already mentioned
|
| 221 |
+
others = [o for o in objects if o is not item][::-1]
|
| 222 |
+
pending_q.update(_reference_question(pending_q["id"], t, t["text"], item, others,
|
| 223 |
+
determiner=pending_q["determiner"]))
|
| 224 |
+
else:
|
| 225 |
+
# height is settled with the asset: metric for simready, nominal otherwise
|
| 226 |
+
item = add_item({"id": f"n{t['index']}", "kind": "object", "word": t["text"],
|
| 227 |
+
"token": t["index"],
|
| 228 |
+
"decision": _object_decision(t["text"], primitives.get(t["text"])),
|
| 229 |
+
"height_m": {"value": None, "source": "pending"},
|
| 230 |
+
"frozen": False}, t)
|
| 231 |
+
if pending_q is not None:
|
| 232 |
+
pending_q["about"] = item["id"]
|
| 233 |
+
pending_q["question"] = f"which {t['text']}?"
|
| 234 |
pending_q = None
|
| 235 |
+
if item["id"] not in t["commits"]:
|
| 236 |
+
t["refers_to"] = item["id"]
|
| 237 |
+
if in_object:
|
| 238 |
+
obj, obj_tok = item, t["index"]
|
| 239 |
else:
|
| 240 |
+
subject, subject_tok = item, t["index"]
|
| 241 |
elif kind == "verb":
|
| 242 |
verb_rel, implied_obj = VERBS[t["text"]]
|
| 243 |
verb_tok = t
|
|
|
|
| 270 |
obj = {"id": "scene"}
|
| 271 |
if obj is None:
|
| 272 |
raise ValueError(f"'{relation}' needs an object noun")
|
| 273 |
+
if obj is subject:
|
| 274 |
+
raise ValueError(f"'{verb_tok['text']}' would relate the {subject['word']} to itself")
|
| 275 |
add_item({"id": f"r{verb_tok['index']}", "kind": "relation", "relation": relation,
|
| 276 |
"subject": subject["id"], "object": obj["id"], "token": verb_tok["index"],
|
| 277 |
+
"tokens": {"subject": subject_tok, "object": obj_tok}, "frozen": False}, verb_tok)
|
| 278 |
|
| 279 |
for q in unresolved:
|
| 280 |
+
q.pop("determiner", None)
|
| 281 |
if q["about"] is None:
|
| 282 |
q["question"] = "determiner without a noun"
|
| 283 |
+
lit = any(i.get("emits_light") for i in by_id.values())
|
| 284 |
+
asked = bool(prior) and any(q["id"] == "q_light" for q in prior["unresolved"])
|
| 285 |
+
if not lit and not asked:
|
| 286 |
# Seeing needs light, as sitting needs a ground: unstated, so surfaced like the ground.
|
| 287 |
unresolved.append({"id": "q_light", "token": None, "about": "light", "question": "what light?",
|
| 288 |
"default_chosen": "daylight", "implied": True,
|
|
|
|
| 291 |
"tokens": tokens, "items": items, "unresolved": unresolved}
|
| 292 |
|
| 293 |
|
| 294 |
+
def extend_ledger(ledger, sentence, primitives=None):
|
| 295 |
+
"""Add a sentence to a ledger: the next sentence of a paragraph."""
|
| 296 |
+
prims = {**ledger.get("primitives", {}), **parse_primitives(primitives)}
|
| 297 |
+
new = build_ledger(sentence, prims, prior=ledger)
|
| 298 |
+
lit = any(i.get("emits_light") for i in ledger["items"] + new["items"])
|
| 299 |
+
unresolved = [q for q in ledger["unresolved"] + new["unresolved"] if not (lit and q["id"] == "q_light")]
|
| 300 |
+
return {**ledger, "sentence": ledger["sentence"].rstrip(" .") + ". " + sentence.strip(),
|
| 301 |
+
"sentences": ledger.get("sentences", [ledger["sentence"]]) + [sentence.strip()],
|
| 302 |
+
"primitives": prims, "tokens": ledger["tokens"] + new["tokens"],
|
| 303 |
+
"items": ledger["items"] + new["items"], "unresolved": unresolved}
|
| 304 |
+
|
| 305 |
+
|
| 306 |
def check_words(sentence, primitives=None):
|
| 307 |
"""Per-word status, the words with no asset, and completions for the word being typed."""
|
| 308 |
primitives = parse_primitives(primitives)
|
|
@@ -8,7 +8,8 @@ import os
|
|
| 8 |
import tempfile
|
| 9 |
|
| 10 |
from . import assets, html, scene
|
| 11 |
-
from .grammar import POOL, build_ledger, candidate_label, next_tier, parse_primitives,
|
|
|
|
| 12 |
from .lexicon import ASSET_TIERS, LEXICON
|
| 13 |
|
| 14 |
|
|
@@ -45,6 +46,8 @@ def settle_all(ledger):
|
|
| 45 |
meshes[it["id"]] = assets.scene_mesh(full)
|
| 46 |
it["decision"]["label"] = candidate_label(it["decision"]["chosen"])
|
| 47 |
for q in ledger["unresolved"]:
|
|
|
|
|
|
|
| 48 |
about = next((i for i in ledger["items"] if i["id"] == q.get("about")), None)
|
| 49 |
if about and about["kind"] == "object":
|
| 50 |
q["default_chosen"] = _option_label(about["decision"]["chosen"])
|
|
@@ -67,6 +70,67 @@ def translate(sentence, primitives=None, seed=0):
|
|
| 67 |
return outputs(build_ledger(sentence, primitives), seed)
|
| 68 |
|
| 69 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 70 |
def apply_changes(ledger, changes):
|
| 71 |
"""changes: list of
|
| 72 |
{"id": ..., "freeze": true|false}
|
|
@@ -75,11 +139,17 @@ def apply_changes(ledger, changes):
|
|
| 75 |
{"id": ..., "height_m": 0.4}
|
| 76 |
{"id": ..., "shape": "sphere"} -> primitive tier with that shape
|
| 77 |
{"reroll": true} -> every unfrozen object takes its next alternative
|
|
|
|
|
|
|
| 78 |
Reroll walks the pool (the old choice goes to the back), so repeated rerolls
|
| 79 |
visit every alternative before repeating, independent of the seed.
|
| 80 |
"""
|
| 81 |
by_id = {i["id"]: i for i in ledger["items"]}
|
| 82 |
for ch in changes:
|
|
|
|
|
|
|
|
|
|
|
|
|
| 83 |
if ch.get("reroll"):
|
| 84 |
for it in ledger["items"]:
|
| 85 |
dec = it.get("decision")
|
|
|
|
| 8 |
import tempfile
|
| 9 |
|
| 10 |
from . import assets, html, scene
|
| 11 |
+
from .grammar import (POOL, build_ledger, candidate_label, extend_ledger, next_tier, parse_primitives,
|
| 12 |
+
primitive_candidate)
|
| 13 |
from .lexicon import ASSET_TIERS, LEXICON
|
| 14 |
|
| 15 |
|
|
|
|
| 46 |
meshes[it["id"]] = assets.scene_mesh(full)
|
| 47 |
it["decision"]["label"] = candidate_label(it["decision"]["chosen"])
|
| 48 |
for q in ledger["unresolved"]:
|
| 49 |
+
if q.get("binds"):
|
| 50 |
+
continue # a reference: which item a word means, not which asset
|
| 51 |
about = next((i for i in ledger["items"] if i["id"] == q.get("about")), None)
|
| 52 |
if about and about["kind"] == "object":
|
| 53 |
q["default_chosen"] = _option_label(about["decision"]["chosen"])
|
|
|
|
| 70 |
return outputs(build_ledger(sentence, primitives), seed)
|
| 71 |
|
| 72 |
|
| 73 |
+
def extend(ledger, sentence, primitives=None, seed=0):
|
| 74 |
+
return outputs(extend_ledger(ledger, sentence, primitives), seed)
|
| 75 |
+
|
| 76 |
+
|
| 77 |
+
def _bind(ledger, src, dst):
|
| 78 |
+
"""Two words mean one thing. `src` is a duplicate object (merged into `dst`) or a
|
| 79 |
+
reference question (its word now means `dst`). Recorded in ledger["resolutions"]."""
|
| 80 |
+
by_id = {i["id"]: i for i in ledger["items"]}
|
| 81 |
+
target = by_id.get(dst)
|
| 82 |
+
if target is None or target["kind"] != "object":
|
| 83 |
+
raise ValueError(f"bind target {dst!r} is not an object item")
|
| 84 |
+
relations = [i for i in ledger["items"] if i["kind"] == "relation"]
|
| 85 |
+
q = next((q for q in ledger["unresolved"] if q["id"] == src and q.get("binds")), None)
|
| 86 |
+
if q is not None:
|
| 87 |
+
tok = q["binds"]["token"]
|
| 88 |
+
moves = [(r, role) for r in relations for role in ("subject", "object")
|
| 89 |
+
if r.get("tokens", {}).get(role) == tok]
|
| 90 |
+
word, old = ledger["tokens"][tok]["text"] if tok < len(ledger["tokens"]) else "?", q["binds"]["to"]
|
| 91 |
+
else:
|
| 92 |
+
it = by_id.get(src)
|
| 93 |
+
if it is None or it["kind"] != "object":
|
| 94 |
+
raise ValueError(f"{src!r} is neither an object item nor a reference question")
|
| 95 |
+
if it is target:
|
| 96 |
+
raise ValueError(f"{src} is already {dst}")
|
| 97 |
+
if it["frozen"]:
|
| 98 |
+
raise ValueError(f"{src} is frozen; unfreeze it first")
|
| 99 |
+
moves = [(r, role) for r in relations for role in ("subject", "object") if r[role] == src]
|
| 100 |
+
word, old = it["word"], src
|
| 101 |
+
for r, role in moves: # check before changing anything
|
| 102 |
+
if r["object" if role == "subject" else "subject"] == dst:
|
| 103 |
+
raise ValueError(f"binding {src} to {dst} would relate the {target['word']} to itself ({r['id']})")
|
| 104 |
+
for r, role in moves:
|
| 105 |
+
r[role] = dst
|
| 106 |
+
if q is not None:
|
| 107 |
+
ledger["tokens"][tok]["refers_to"] = dst
|
| 108 |
+
ledger["unresolved"].remove(q)
|
| 109 |
+
else:
|
| 110 |
+
ledger["items"].remove(it)
|
| 111 |
+
if it.get("emits_light"):
|
| 112 |
+
target["emits_light"] = True
|
| 113 |
+
for t in ledger["tokens"]:
|
| 114 |
+
if src in t["commits"]:
|
| 115 |
+
t["commits"].remove(src)
|
| 116 |
+
t["refers_to"] = dst
|
| 117 |
+
elif t.get("refers_to") == src:
|
| 118 |
+
t["refers_to"] = dst
|
| 119 |
+
kept = []
|
| 120 |
+
for u in ledger["unresolved"]:
|
| 121 |
+
if u.get("about") == src and not u.get("binds"):
|
| 122 |
+
continue # "which cat?" for the duplicate: the duplicate is gone
|
| 123 |
+
if u.get("binds", {}).get("to") == src:
|
| 124 |
+
u["binds"]["to"], u["about"], u["default_chosen"] = dst, dst, f"the {target['word']} ({dst})"
|
| 125 |
+
kept.append(u)
|
| 126 |
+
ledger["unresolved"] = kept
|
| 127 |
+
live = {u["id"] for u in ledger["unresolved"]}
|
| 128 |
+
for t in ledger["tokens"]: # answered questions leave the reading too; the resolution note keeps them
|
| 129 |
+
t["opens"] = [o for o in t.get("opens", []) if o in live]
|
| 130 |
+
ledger.setdefault("resolutions", []).append(
|
| 131 |
+
{"word": word, "from": old, "to": dst, "note": f"{old} -> {dst} (same referent)"})
|
| 132 |
+
|
| 133 |
+
|
| 134 |
def apply_changes(ledger, changes):
|
| 135 |
"""changes: list of
|
| 136 |
{"id": ..., "freeze": true|false}
|
|
|
|
| 139 |
{"id": ..., "height_m": 0.4}
|
| 140 |
{"id": ..., "shape": "sphere"} -> primitive tier with that shape
|
| 141 |
{"reroll": true} -> every unfrozen object takes its next alternative
|
| 142 |
+
{"id": "n12", "bind": "n1"} -> n12 is the same thing as n1: merged into it
|
| 143 |
+
{"id": "q11", "bind": "n5"} -> the word behind reference question q11 means n5
|
| 144 |
Reroll walks the pool (the old choice goes to the back), so repeated rerolls
|
| 145 |
visit every alternative before repeating, independent of the seed.
|
| 146 |
"""
|
| 147 |
by_id = {i["id"]: i for i in ledger["items"]}
|
| 148 |
for ch in changes:
|
| 149 |
+
if "bind" in ch:
|
| 150 |
+
_bind(ledger, ch.get("id"), ch["bind"])
|
| 151 |
+
by_id = {i["id"]: i for i in ledger["items"]}
|
| 152 |
+
continue
|
| 153 |
if ch.get("reroll"):
|
| 154 |
for it in ledger["items"]:
|
| 155 |
dec = it.get("decision")
|