prasadpatil25 commited on
Commit
e2bfc0b
·
verified ·
1 Parent(s): a38c850

Deploy Memory Studio public demo

Browse files
README.md CHANGED
@@ -1,85 +1,82 @@
1
- ---
2
- title: Memory Studio
3
- emoji: 🧠
4
- colorFrom: indigo
5
- colorTo: purple
6
- sdk: gradio
7
- sdk_version: 6.32.0
8
- python_version: "3.12"
9
- app_file: app.py
10
- pinned: false
11
- short_description: Editable mnemonic drafts, Major System, ordered stories
12
- models:
13
- - HuggingFaceTB/SmolLM2-360M-Instruct
14
- - segmind/SSD-1B
15
- preload_from_hub:
16
- - HuggingFaceTB/SmolLM2-360M-Instruct
17
- - segmind/SSD-1B unet/diffusion_pytorch_model.fp16.safetensors,text_encoder_2/model.fp16.safetensors,text_encoder/model.fp16.safetensors,vae/diffusion_pytorch_model.fp16.safetensors
18
- ---
19
-
20
- # Memory Studio
21
-
22
- A public demo of a local-first mnemonic workbench: turn numbers, words, and
23
- ordered lists into editable mental images using the Major System, sound/keyword
24
- associations, ordered story chains, and optional Memory Palace placement.
25
-
26
- > **This Space is a public demo.** Anything you type is sent to Hugging Face and
27
- > processed on this Space's servers, not on your device. The app does not save
28
- > your study material, but a hosted demo cannot offer on-device privacy. Do not
29
- > paste secrets or private notes. Run the same code locally for private use.
30
-
31
- ## What works without a model
32
-
33
- - Numbers split into two-digit chunks, with a complete curated 00-99 word set.
34
- - An approximate Major-System check that refuses to accept a failing word.
35
- - Ordered list stories that never silently drop an item.
36
- - Palace routes and active-recall prompts.
37
- - A **Start here** page with one-click number, list, word, and practice examples.
38
- - A self-scoring **Practice** tab that keeps cards in the original order.
39
- - **Markdown study sheets** for each builder, plus an in-session image gallery.
40
- - A phone-friendly layout: stacked fields, large tap targets, and wide tables that
41
- scroll inside their card instead of stretching the page.
42
-
43
- No model is required for any of that. The checkboxes and the image tab only add
44
- optional drafts.
45
-
46
- ## Optional models
47
-
48
- | Model | License | Notes |
49
- |---|---|---|
50
- | `HuggingFaceTB/SmolLM2-360M-Instruct` | Apache-2.0 | Small text model for candidate words, story links, and word cards. Runs on CPU. |
51
- | `segmind/SSD-1B` | Apache-2.0 | SDXL-class text-to-image model for visual drafts, ~4.5 GB of fp16 weights. Runs on ZeroGPU and has no bundled safety checker. |
52
-
53
- Both are public downloads. No paid inference API, token, or account is used by
54
- the app itself.
55
-
56
- ## Story first, then optional pictures
57
-
58
- Build a checked number or ordered-list story first. Once the story is built, an
59
- optional multiselect "Illustrate scenes" picker lists your checked rows. Choose
60
- one or more — or click **Select all scenes** to check every row — then click
61
- **Illustrate this scene** to open their cues in the Image tab, one blank line
62
- apart. To illustrate the whole chain as one picture, click **Combine into one
63
- story**: the checked rows (or your selection) are woven into a single ordered
64
- picture story for the Image tab. Tick the local text-model box to draft it with
65
- the model — the result is order-checked and falls back to the deterministic
66
- template otherwise. Only pictures you ask for use the image model — images are
67
- hints for tricky scenes, not a replacement for the story. Draw each scene
68
- separately, or draw the combined story once. The Word card does not use the
69
- picker.
70
-
71
- ## Limits
72
-
73
- - The Major System is phonetic; the automatic spelling check is approximate.
74
- - Image cues are memory aids, not definitions. This app does not verify
75
- technical meanings.
76
- - Generated pictures are impressionistic drafts, not factual illustrations, and
77
- may reflect model biases.
78
- - Nothing is saved automatically. Download a study sheet or picture if you want
79
- to keep it; study-sheet files live in temporary storage.
80
-
81
- ## Running it locally instead
82
-
83
- The repository is the same code. Locally it binds to `127.0.0.1` with
84
- `share=False`, does not create a public link, and keeps inference on your
85
- machine. See the project README for setup.
 
1
+ ---
2
+ title: Memory Studio
3
+ emoji: 🧠
4
+ colorFrom: indigo
5
+ colorTo: purple
6
+ sdk: gradio
7
+ sdk_version: 6.32.0
8
+ python_version: "3.12"
9
+ app_file: app.py
10
+ pinned: false
11
+ short_description: Editable mnemonic drafts, Major System, ordered stories
12
+ models:
13
+ - HuggingFaceTB/SmolLM2-360M-Instruct
14
+ - segmind/SSD-1B
15
+ preload_from_hub:
16
+ - HuggingFaceTB/SmolLM2-360M-Instruct
17
+ - segmind/SSD-1B unet/diffusion_pytorch_model.fp16.safetensors,text_encoder_2/model.fp16.safetensors,text_encoder/model.fp16.safetensors,vae/diffusion_pytorch_model.fp16.safetensors
18
+ ---
19
+
20
+ # Memory Studio
21
+
22
+ A public demo of a local-first mnemonic workbench: turn numbers, words, and
23
+ ordered lists into editable mental images using the Major System, sound/keyword
24
+ associations, ordered story chains, and optional Memory Palace placement.
25
+
26
+ > **This Space is a public demo.** Anything you type is sent to Hugging Face and
27
+ > processed on this Space's servers, not on your device. The app does not save
28
+ > your study material, but a hosted demo cannot offer on-device privacy. Do not
29
+ > paste secrets or private notes. Run the same code locally for private use.
30
+
31
+ ## What works without a model
32
+
33
+ - Numbers split into two-digit chunks, with a complete curated 00-99 word set.
34
+ - A Major-System check based on the CMU Pronouncing Dictionary (sounds, not
35
+ spelling), plus a finder for other common words that encode a chunk.
36
+ - A readable linked story: each image does something to the previous one,
37
+ with an editable Story line per row and a say-back line at the end.
38
+ - Ordered list stories that never silently drop an item.
39
+ - Memory Palace routes and peg systems (Number-Rhyme, Number-Shape, Major pegs).
40
+ - A review schedule (10 minutes, 1 day, 1 week, 1 month, 3-6 months) with a
41
+ calendar file in each study sheet.
42
+ - Mind Maps drawn from an indented outline.
43
+ - **Practice these cards**: recall your own chain card by card, then decode.
44
+ - A personal word list saved in your own browser.
45
+
46
+ No model is required for any of that. The checkboxes and the image tab only add
47
+ optional drafts.
48
+
49
+ ## Optional models
50
+
51
+ | Model | License | Notes |
52
+ |---|---|---|
53
+ | `HuggingFaceTB/SmolLM2-360M-Instruct` | Apache-2.0 | Small text model for candidate words, story links, and word cards. Runs on CPU. |
54
+ | `segmind/SSD-1B` | Apache-2.0 | SDXL-class text-to-image model for visual drafts, ~4.5 GB of fp16 weights. Runs on ZeroGPU and has no bundled safety checker. |
55
+
56
+ Both are public downloads. No paid inference API, token, or account is used by
57
+ the app itself.
58
+
59
+ ## Story first, then optional pictures
60
+
61
+ Build a checked number or ordered-list story first. Then pick a scene (one
62
+ frozen moment of a link, or the whole chain for 2-4 images) and click **Open in
63
+ Image tab** to edit its cue and draw it. Pictures are hints for
64
+ tricky scenes, not a replacement for the story. The image model reads only
65
+ about 75 tokens, so long cues are shortened and the dropped words are listed.
66
+
67
+ ## Limits
68
+
69
+ - The pronunciation dictionary is American English; say each word aloud in your
70
+ accent. Unknown words fall back to an approximate spelling check.
71
+ - Image cues are memory aids, not definitions. This app does not verify
72
+ technical meanings.
73
+ - Generated pictures are impressionistic drafts, not factual illustrations, and
74
+ may reflect model biases.
75
+ - Drafts are not saved. Download a study sheet or picture to keep it. Saved
76
+ words live in your browser and reset if the Space restarts.
77
+
78
+ ## Running it locally instead
79
+
80
+ The repository is the same code. Locally it binds to `127.0.0.1` with
81
+ `share=False`, does not create a public link, and keeps inference on your
82
+ machine. See the project README for setup.
 
 
 
data/12DICTS-NOTICE.txt ADDED
@@ -0,0 +1,242 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ common-words.txt.gz is derived from the 12dicts word lists (version 6.0.2,
2
+ Lemmatized/2+2+3frq.txt) by Alan Beale, http://wordlist.aspell.net/12dicts/ .
3
+ The 12dicts author placed his lists in the public domain and requests
4
+ acknowledgment of their use. The 2+2+3 lists depend on AGID, whose notice
5
+ follows in full.
6
+
7
+ Automatically Generated Inflection Database (AGID)
8
+
9
+ August 19, 2000
10
+ Revision 2
11
+
12
+ Copyright 2000 by Kevin Atkinson <kevina@users.sourceforge.net>
13
+
14
+ The file "infl.txt" is an automatically created database of the
15
+ inflected forms of words from an insanely large word list.
16
+
17
+ The latest version can be found at http://aspell.sourceforge.net/wl/.
18
+
19
+ Entries are in the following form.
20
+
21
+ <word> <pos>: <inflected forms>
22
+
23
+ Where <pos> is V for verb, N for noun, or A or adjective or adverb.
24
+ If <pos> is followed by a ? that means that the part-of-speech was not
25
+ in the part-of-speech database however the inflected forms of the word
26
+ where found in the word list.
27
+
28
+ The inflected forms are in the following order for verbs (except for
29
+ the verb "be"):
30
+ <past tense> [<past participle>] <-ing form> <plural form>
31
+ and for adjective or adverbs:
32
+ <-er form> <-est form>
33
+ There are two spaces between each form.
34
+
35
+ A word in parentheses mean that it is considered a less preferred form
36
+ of the previous inflection. Two parentheses means that the word is
37
+ even less preferred, etc. A / between two words means that the two
38
+ words are considered almost equal variants or that is is difficult to
39
+ tell which one is the primary form. They are ordered by preference
40
+ however sometime this distinction is so slight it is meaningless. A
41
+ "|" between words means that both inflections are used depending on
42
+ the meaning of the word. If the distinction between the two forms can
43
+ be described in a word than that word is found after the word in
44
+ braces, for example:
45
+
46
+ hang V: hung {suspend} | hanged {execute} hanging hangs
47
+
48
+ Notice how there is two spaces between the past tense, -ing form and
49
+ plural form but not between the alternate forms of the past tense. In
50
+ general, if the "|" symbol would be needed more than once the words
51
+ the entry is split up into multiple lines like so:
52
+
53
+ <word> [{explanation}] <POS>: <inflected forms>
54
+
55
+ However, the past particle as past tense form are considered a single
56
+ form. Thus, a "|" may appear more than once when the word contains
57
+ both a past participial and past tense form.
58
+
59
+ A /? between words means that both inflections were found in the word
60
+ list but the script was not sure which one to use. A ~ after a word
61
+ means that there is a slight chance that it is the plural of a word.
62
+ A ! after a word indicates that the word is likely an inflections of a
63
+ similar word (generally one ending in e) and not the current word. A
64
+ ? after a word means that the word was not in the word list but if it
65
+ was it would be considered an inflected form of the base word.
66
+
67
+ Fell free to send me corrections to correct any of these questionable
68
+ words. I am mostly interested in the preferred form of the word in
69
+ the case of /? or words marked with a ~ that are actually valid.
70
+
71
+ Words are in mixed case but all accents have been scripted thus words
72
+ like caf� are instead cafe.
73
+
74
+ The file "variant" contains a list of alternate inflections.
75
+
76
+ The file "irregular" contains extra information where a noun or verb
77
+ has irregular inflected forms.
78
+
79
+ The file "dontuse" contains a list of words not to consider an
80
+ inflected form of a word if more than one inflected form of a word is
81
+ found.
82
+
83
+ The files "prefixes" and "suffixes" contains a list of common prefixes
84
+ and suffixes respectfully. These files are used by the script to
85
+ produce inflected forms for words that end in a word in the
86
+ "irregular" file. If the beginning appears in the word list or the
87
+ prefixes file and the ending appears in the irregular file I also
88
+ consider <prefix>+<irregular inflections>. If the prefix is 3 letters
89
+ or more OR appears in the prefixes file and the suffix is 4 letters or
90
+ more OR appears in the suffixes file I consider it the most likely
91
+ choice, otherwise I consider it as a possible candidate but not the
92
+ most likely choice.
93
+
94
+ The file "make-infl" is the actual Perl script used to create the
95
+ data base.
96
+
97
+ CHANGES:
98
+
99
+ From Revision 1 to 2 (August 18, 2000)
100
+
101
+ Classified variants as either almost equal, also used, or
102
+ secondary.
103
+
104
+ The / is now used to indicate equal variants. "/?" is now used to
105
+ mean what "/" used to be.
106
+
107
+ Lots of additional rules added which greatly improved the results.
108
+
109
+ COPYRIGHT AND SOURCE:
110
+
111
+ The final product is under the following copyright, as well as any
112
+ copyrights mentioned below.
113
+
114
+ Copyright 2000 by Kevin Atkinson
115
+
116
+ Permission to use, copy, modify, distribute and sell this database,
117
+ the associated scripts, the output created form the scripts and its
118
+ documentation for any purpose is hereby granted without fee,
119
+ provided that the above copyright notice appears in all copies and
120
+ that both that copyright notice and this permission notice appear in
121
+ supporting documentation. Kevin Atkinson makes no representations
122
+ about the suitability of this array for any purpose. It is provided
123
+ "as is" without express or implied warranty.
124
+
125
+ The part-of-speech database used is created form the Moby
126
+ part-of-speech database which is in the public domain:
127
+
128
+ The Moby lexicon project is complete and has
129
+ been place into the public domain. Use, sell,
130
+ rework, excerpt and use in any way on any platform.
131
+
132
+ Placing this material on internal or public servers is
133
+ also encouraged. The compiler is not aware of any
134
+ export restrictions so freely distribute world-wide.
135
+
136
+ You can verify the public domain status by contacting
137
+
138
+ Grady Ward
139
+ 3449 Martha Ct.
140
+ Arcata, CA 95521-4884
141
+
142
+ grady@netcom.com
143
+ grady@northcoast.com
144
+
145
+ and the WordNet database which is under the following copyright:
146
+
147
+ This software and database is being provided to you, the LICENSEE, by
148
+ Princeton University under the following license. By obtaining, using
149
+ and/or copying this software and database, you agree that you have
150
+ read, understood, and will comply with these terms and conditions.:
151
+
152
+ Permission to use, copy, modify and distribute this software and
153
+ database and its documentation for any purpose and without fee or
154
+ royalty is hereby granted, provided that you agree to comply with
155
+ the following copyright notice and statements, including the disclaimer,
156
+ and that the same appear on ALL copies of the software, database and
157
+ documentation, including modifications that you make for internal
158
+ use or for distribution.
159
+
160
+ WordNet 1.6 Copyright 1997 by Princeton University. All rights reserved.
161
+
162
+ THIS SOFTWARE AND DATABASE IS PROVIDED "AS IS" AND PRINCETON
163
+ UNIVERSITY MAKES NO REPRESENTATIONS OR WARRANTIES, EXPRESS OR
164
+ IMPLIED. BY WAY OF EXAMPLE, BUT NOT LIMITATION, PRINCETON
165
+ UNIVERSITY MAKES NO REPRESENTATIONS OR WARRANTIES OF MERCHANT-
166
+ ABILITY OR FITNESS FOR ANY PARTICULAR PURPOSE OR THAT THE USE
167
+ OF THE LICENSED SOFTWARE, DATABASE OR DOCUMENTATION WILL NOT
168
+ INFRINGE ANY THIRD PARTY PATENTS, COPYRIGHTS, TRADEMARKS OR
169
+ OTHER RIGHTS.
170
+
171
+ The name of Princeton University or Princeton may not be used in
172
+ advertising or publicity pertaining to distribution of the software
173
+ and/or database. Title to copyright in this software, database and
174
+ any associated documentation shall at all times remain with
175
+ Princeton University and LICENSEE agrees to preserve same.
176
+
177
+ The word list used is a combination of several word list:
178
+
179
+ 1) Most of the word lists from the Moby Words package:
180
+
181
+ 10196pla.ces 113809of.fic 21986na.mes 256772co.mpo 354984si.ngl
182
+ 3897male.nam 4160offi.cia 4946fema.len 6213acro.nym 74550com.mon
183
+
184
+ The Moby Word package, like the Part-Of-Speech database is in the
185
+ public domain.
186
+
187
+ 2) The ENABLE2K word lists which is in the public domain:
188
+
189
+ The ENABLE master word list, WORD.LST, is herewith formally
190
+ released into the Public Domain. Anyone is free to use it or
191
+ distribute it in any manner they see fit. No fee or registration
192
+ is required for its use nor are "contributions" solicited (if you
193
+ feel you absolutely must contribute something for your own peace
194
+ of mind, the authors of the ENABLE list ask that you make a
195
+ donation on their behalf to your favorite charity). This word
196
+ list is our gift to the Scrabble community, as an alternate to
197
+ "official" word lists. Game designers may feel free to
198
+ incorporate the WORD.LST into their games. Please mention the
199
+ source and credit us as originators of the list. Note that if
200
+ you, as a game designer, use the WORD.LST in your product, you
201
+ may still copyright and protect your product, but you may *not*
202
+ legally copyright or in any way restrict redistribution of the
203
+ WORD.LST portion of your product. This *may* under law restrict
204
+ your rights to restrict your users' rights, but that is only
205
+ fair.
206
+
207
+ 3) All of the word lists in the ENABLE2K Supplemnt which consists of:
208
+
209
+ 2DICTS.LST ALSO.LST LETTERS.LST OSPDADD.LST UCACR.LST
210
+ ABLE.LST LCACR.LST NOPOS.LST PLURALS.LST UPPER.LST
211
+
212
+ All of these word lists are also in the public domain.
213
+
214
+ 4) The list of signature words from the YAWL package which is in the
215
+ public domain.
216
+
217
+ 5) The UK Advanced Cryptics Dictionary which in under the following
218
+ copyright:
219
+
220
+ Copyright (c) J Ross Beresford 1993-1999. All Rights Reserved.
221
+
222
+ The following restriction is placed on the use of this
223
+ publication: if The UK Advanced Cryptics Dictionary is used
224
+ in a software package or redistributed in any form, the
225
+ copyright notice must be prominently displayed and the text
226
+ of this document must be included verbatim.
227
+
228
+ There are no other restrictions: I would like to see the
229
+ list distributed as widely as possible.
230
+
231
+ 6) Some extra words found in the Part-Of-Speech database that was not
232
+ found in any of the above word list.
233
+
234
+ 7) Words found in the Jargon File Word List package, available at
235
+ http://aspell.sourceforge.net/wl/, which is in the Public Domain.
236
+
237
+ 8) And finally some extra words that I added myself. These words can be
238
+ found in the file "extra-words"
239
+
240
+ The "dontuse", "irregular", and "variant" file was created by me
241
+ (Kevin Atkinson) from numerous sources.
242
+
data/CMUDICT-LICENSE ADDED
@@ -0,0 +1,33 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ Copyright (C) 1993-2015 Carnegie Mellon University. All rights reserved.
2
+
3
+ Redistribution and use in source and binary forms, with or without
4
+ modification, are permitted provided that the following conditions
5
+ are met:
6
+
7
+ 1. Redistributions of source code must retain the above copyright
8
+ notice, this list of conditions and the following disclaimer.
9
+ The contents of this file are deemed to be source code.
10
+
11
+ 2. Redistributions in binary form must reproduce the above copyright
12
+ notice, this list of conditions and the following disclaimer in
13
+ the documentation and/or other materials provided with the
14
+ distribution.
15
+
16
+ This work was supported in part by funding from the Defense Advanced
17
+ Research Projects Agency, the Office of Naval Research and the National
18
+ Science Foundation of the United States of America, and by member
19
+ companies of the Carnegie Mellon Sphinx Speech Consortium. We acknowledge
20
+ the contributions of many volunteers to the expansion and improvement of
21
+ this dictionary.
22
+
23
+ THIS SOFTWARE IS PROVIDED BY CARNEGIE MELLON UNIVERSITY ``AS IS'' AND
24
+ ANY EXPRESSED OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
25
+ THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
26
+ PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL CARNEGIE MELLON UNIVERSITY
27
+ NOR ITS EMPLOYEES BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
28
+ SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
29
+ LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
30
+ DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
31
+ THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
32
+ (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
33
+ OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
data/cmudict.dict.gz ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:78b6e8070a57476f9a2e32cc73da6d7e98ee6f6613af92076b6a735c9f0ad8b7
3
+ size 914702
data/common-words.txt.gz ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:11bdf615df62095aabeb20171e689003602f6b875ba6772a4adcbc372e207ded
3
+ size 132305
local_image.py CHANGED
@@ -1,382 +1,402 @@
1
- """Optional, on-device image generation from a mnemonic cue.
2
-
3
- Nothing here is imported or downloaded until the learner opens the optional
4
- image tab and clicks Draw. The text-only and no-model routes never load this
5
- module's model. All generation happens on this computer after the first public
6
- model download; no API key, account, or paid endpoint is used.
7
- """
8
-
9
- from __future__ import annotations
10
-
11
- import importlib.util
12
- import json
13
- import os
14
- import secrets
15
- import threading
16
- import time
17
- from functools import lru_cache
18
- from typing import Any
19
-
20
- # Privacy: keep Hugging Face telemetry off before diffusers is imported lazily.
21
- os.environ["HF_HUB_DISABLE_TELEMETRY"] = "1"
22
-
23
- DEFAULT_IMAGE_MODEL_ID = "segmind/tiny-sd"
24
- IMAGE_MODEL_ID = (
25
- os.getenv("MEMORY_IMAGE_MODEL", DEFAULT_IMAGE_MODEL_ID).strip()
26
- or DEFAULT_IMAGE_MODEL_ID
27
- )
28
- _IMAGE_MODEL_LICENSES = {
29
- "segmind/SSD-1B": (
30
- "The `segmind/SSD-1B` model card lists **Apache-2.0**. The weights are "
31
- "permissively licensed; you remain responsible for the images you "
32
- "generate and share."
33
- ),
34
- "segmind/tiny-sd": (
35
- "The `segmind/tiny-sd` model card lists **CreativeML OpenRAIL-M**. That "
36
- "license includes use restrictions and does not claim rights over "
37
- "generated outputs. Check the current model card before redistributing "
38
- "weights or using outputs commercially."
39
- ),
40
- }
41
- IMAGE_MODEL_LICENSE_NOTE = _IMAGE_MODEL_LICENSES.get(
42
- IMAGE_MODEL_ID,
43
- f"Check the `{IMAGE_MODEL_ID}` model card for license terms before "
44
- "redistributing weights or using outputs commercially.",
45
- )
46
- _IMAGE_MODEL_DOWNLOADS = {
47
- "segmind/SSD-1B": "roughly 4.5 GB",
48
- "segmind/tiny-sd": "roughly 1 GB",
49
- }
50
- IMAGE_MODEL_DOWNLOAD_NOTE = _IMAGE_MODEL_DOWNLOADS.get(
51
- IMAGE_MODEL_ID, "several gigabytes"
52
- )
53
- IMAGE_MODEL_CARD_URL = f"https://huggingface.co/{IMAGE_MODEL_ID}"
54
- MAX_CUE_LENGTH = 500
55
- IMAGE_STYLES = (
56
- "Vivid mnemonic illustration",
57
- "Playful cartoon",
58
- "Dramatic photorealism",
59
- "Hand-drawn ink sketch",
60
- "Retro poster",
61
- )
62
- IMAGE_SIZE_OPTIONS = (
63
- "512 x 512 (fastest)",
64
- "768 x 768 (balanced)",
65
- "1024 x 1024 (best detail)",
66
- )
67
- _ALLOWED_SIZES = (512, 768, 1024)
68
-
69
-
70
- def _env_int(name: str, fallback: int) -> int:
71
- text = os.getenv(name, "").strip()
72
- return int(text) if text.isdigit() else fallback
73
-
74
-
75
- DEFAULT_IMAGE_SIZE = min(
76
- _ALLOWED_SIZES, key=lambda option: abs(option - _env_int("MEMORY_IMAGE_DEFAULT_SIZE", 512))
77
- )
78
- MIN_IMAGE_STEPS = 4
79
- MAX_IMAGE_STEPS = 24
80
- DEFAULT_IMAGE_STEPS = max(
81
- MIN_IMAGE_STEPS,
82
- min(_env_int("MEMORY_IMAGE_DEFAULT_STEPS", 8), MAX_IMAGE_STEPS),
83
- )
84
- _NEGATIVE_PROMPT = (
85
- "text, letters, words, watermark, signature, frame, border, blurry, "
86
- "low quality, deformed, duplicate, extra limbs, cluttered background"
87
- )
88
-
89
-
90
- class LocalImageError(RuntimeError):
91
- """An image-model failure with a safe, actionable message for the UI."""
92
-
93
- def __init__(self, public_message: str):
94
- super().__init__(public_message)
95
- self.public_message = public_message
96
-
97
-
98
- def missing_dependencies() -> list[str]:
99
- """Return the optional image dependencies that are not installed."""
100
- return [
101
- name
102
- for name in ("torch", "diffusers", "transformers")
103
- if importlib.util.find_spec(name) is None
104
- ]
105
-
106
-
107
- def dependencies_available() -> bool:
108
- return not missing_dependencies()
109
-
110
-
111
- def image_model_status_markdown(hosted: bool = False) -> str:
112
- """Short status text that does not load or download model weights."""
113
- missing = missing_dependencies()
114
- if missing:
115
- return (
116
- "**Optional image generation is not ready.** It needs "
117
- f"`{ '`, `'.join(missing) }`. The text and no-model routes still "
118
- "work. Install the optional image dependencies only if you want "
119
- "local pictures."
120
- )
121
- if hosted:
122
- return (
123
- f"**Image generation is available:** `{IMAGE_MODEL_ID}` runs on this "
124
- "Space's shared free GPU. The text-only and no-model routes never "
125
- "load it."
126
- )
127
- return (
128
- f"**Optional local image generation is available:** `{IMAGE_MODEL_ID}`. "
129
- "The first Draw downloads public weights and later runs use the local "
130
- "cache. The text-only route never loads this model."
131
- )
132
-
133
-
134
- # One pipeline is shared by the whole process, so only one Draw runs at a time.
135
- _PIPELINE_LOCK = threading.Lock()
136
- _PLACED_DEVICE: dict[str, str | None] = {"device": None}
137
-
138
-
139
- def _dtype_and_device(torch: Any) -> tuple[str, Any]:
140
- if torch.cuda.is_available():
141
- return "cuda", torch.float16
142
- if getattr(torch.backends, "mps", None) and torch.backends.mps.is_available():
143
- return "mps", torch.float16
144
- return "cpu", torch.float32
145
-
146
-
147
- def _pipeline_class(diffusers: Any) -> Any:
148
- """Read model_index.json so SD and SDXL checkpoints both load."""
149
- try:
150
- from huggingface_hub import hf_hub_download
151
-
152
- index_path = hf_hub_download(IMAGE_MODEL_ID, "model_index.json")
153
- with open(index_path, encoding="utf-8") as handle:
154
- class_name = json.load(handle).get("_class_name", "")
155
- candidate = getattr(diffusers, class_name, None)
156
- if candidate is not None:
157
- return candidate
158
- except Exception:
159
- pass
160
- return diffusers.StableDiffusionPipeline
161
-
162
-
163
- def _from_pretrained(
164
- pipeline_cls: Any, dtype: Any, base_kwargs: dict[str, Any]
165
- ) -> Any:
166
- """Load the checkpoint, tolerating fp16 variants and old kwarg names.
167
-
168
- SDXL repositories usually ship fp16 weights. Loading those directly avoids
169
- a full fp32 copy in memory, which matters on a 16 GB hosted container.
170
- Older diffusers releases use ``torch_dtype`` instead of ``dtype``.
171
- """
172
- use_fp16 = str(dtype).endswith("float16")
173
- variants: tuple[str | None, ...] = ("fp16", None) if use_fp16 else (None,)
174
- last_error: Exception | None = None
175
- for variant in variants:
176
- for dtype_key in ("dtype", "torch_dtype"):
177
- kwargs = {**base_kwargs, dtype_key: dtype}
178
- if variant:
179
- kwargs["variant"] = variant
180
- try:
181
- return pipeline_cls.from_pretrained(IMAGE_MODEL_ID, **kwargs)
182
- except TypeError as exc:
183
- last_error = exc
184
- continue
185
- except Exception as exc:
186
- last_error = exc
187
- break
188
- raise LocalImageError(
189
- "The local image model could not be loaded. On first use, check "
190
- "internet access, free disk space, and available RAM; later runs use "
191
- "the local cache. Use the text-only route if this machine cannot run "
192
- "the image model."
193
- ) from last_error
194
-
195
-
196
- @lru_cache(maxsize=1)
197
- def _load_pipeline() -> tuple[Any, str, Any]:
198
- """Load and cache the local diffusion pipeline. Only called for a Draw."""
199
- missing = missing_dependencies()
200
- if missing:
201
- raise LocalImageError(
202
- "The optional image dependencies are not installed "
203
- f"({', '.join(missing)}). Use the text-only route, or install "
204
- "PyTorch, Transformers, and Diffusers and restart the app."
205
- )
206
-
207
- try:
208
- import diffusers
209
- import torch
210
- except Exception as exc: # pragma: no cover - depends on the local install
211
- raise LocalImageError(
212
- "The optional image model could not be imported. Install a "
213
- "compatible PyTorch, Transformers, and Diffusers build, or leave "
214
- "the image tab alone."
215
- ) from exc
216
-
217
- device, dtype = _dtype_and_device(torch)
218
- pipeline_cls = _pipeline_class(diffusers)
219
- # Status only: no cue or generated prompt is printed or logged.
220
- print(
221
- f"Loading optional local image model {IMAGE_MODEL_ID} "
222
- f"({pipeline_cls.__name__}) on {device}. "
223
- "The first run may download public weights."
224
- )
225
- base_kwargs: dict[str, Any] = {}
226
- if pipeline_cls is diffusers.StableDiffusionPipeline:
227
- # SD 1.x checkpoints may bundle a safety checker; SDXL does not take it.
228
- base_kwargs["safety_checker"] = None
229
- base_kwargs["requires_safety_checker"] = False
230
- if importlib.util.find_spec("accelerate") is not None:
231
- base_kwargs["low_cpu_mem_usage"] = True
232
-
233
- pipe = _from_pretrained(pipeline_cls, dtype, base_kwargs)
234
- try:
235
- pipe.to(device)
236
- _PLACED_DEVICE["device"] = device
237
- # SDXL and SD 1.x VAE configs set force_upcast. Do NOT cast the VAE to
238
- # float32 here: the pipelines' own upcast path also converts the fp16
239
- # latents, and a pre-cast VAE would receive fp16 latents and fail with
240
- # "Input type (c10::Half) and bias type (float) should be the same".
241
- pipe.set_progress_bar_config(disable=True)
242
- pipe.enable_attention_slicing()
243
- if hasattr(pipe, "enable_vae_slicing"):
244
- pipe.enable_vae_slicing()
245
- except Exception as exc:
246
- raise LocalImageError(
247
- "The local image model loaded but could not be prepared on this "
248
- "device. Use the text-only route, or try a smaller image model."
249
- ) from exc
250
-
251
- return pipe, device, torch
252
-
253
-
254
- def _ensure_device(pipe: Any, device: str, torch: Any) -> str:
255
- """Move the cached pipeline if the available device changed.
256
-
257
- ZeroGPU attaches a GPU only inside a ``@spaces.GPU`` call, so a pipeline
258
- loaded on CPU may need one move. This is a safety net: the Space warms up
259
- on CUDA at startup, so normally nothing moves here.
260
- """
261
- wanted, _ = _dtype_and_device(torch)
262
- if _PLACED_DEVICE["device"] == wanted:
263
- return wanted
264
- try:
265
- pipe.to(wanted)
266
- except Exception as exc:
267
- raise LocalImageError(
268
- "The optional image model could not be moved to the available "
269
- "device. Restart the app, or use the text-only route."
270
- ) from exc
271
- _PLACED_DEVICE["device"] = wanted
272
- return wanted
273
-
274
-
275
- def warm_up() -> bool:
276
- """Load the weights and place the pipeline before the first Draw.
277
-
278
- Only used on a hosted Space: ZeroGPU needs the model on CUDA while the
279
- module imports, and the first GPU call must not also carry a multi-gigabyte
280
- download. The local app never calls this, so its startup stays light.
281
- """
282
- try:
283
- _load_pipeline()
284
- except LocalImageError as exc:
285
- print(f"Optional image model was not warmed up: {exc.public_message}")
286
- return False
287
- return True
288
-
289
-
290
- def _clean_cue(cue: str) -> str:
291
- text = " ".join((cue or "").split())
292
- if not text:
293
- raise ValueError("Enter an image cue or scene first.")
294
- if len(text) > MAX_CUE_LENGTH:
295
- raise ValueError(
296
- f"Keep the image cue under {MAX_CUE_LENGTH} characters."
297
- )
298
- return text
299
-
300
-
301
- def _build_prompt(cue: str, style: str) -> str:
302
- style_text = (style or IMAGE_STYLES[0]).strip()
303
- return (
304
- f"{cue}. A single clear scene for a memory mnemonic. {style_text}. "
305
- "Exaggerated scale, bold colors, strong lighting, simple background, "
306
- "highly memorable, no text, no watermark."
307
- )
308
-
309
-
310
- def _coerce_size(size: Any) -> int:
311
- """Snap an arbitrary size to the sizes this app offers."""
312
- try:
313
- value = int(size)
314
- except (TypeError, ValueError):
315
- return DEFAULT_IMAGE_SIZE
316
- return min(_ALLOWED_SIZES, key=lambda option: abs(option - value))
317
-
318
-
319
- def generate_cue_image(
320
- cue: str,
321
- style: str = IMAGE_STYLES[0],
322
- size: int = DEFAULT_IMAGE_SIZE,
323
- steps: int = DEFAULT_IMAGE_STEPS,
324
- seed: int = -1,
325
- ) -> tuple[Any, dict[str, Any]]:
326
- """Generate one local image draft and return it with reproducible settings."""
327
- cue_text = _clean_cue(cue)
328
- size = _coerce_size(size)
329
- steps = max(MIN_IMAGE_STEPS, min(int(steps), MAX_IMAGE_STEPS))
330
- seed = int(seed)
331
- if seed < 0:
332
- seed = secrets.randbelow(2**31 - 1)
333
-
334
- with _PIPELINE_LOCK:
335
- pipe, device, torch = _load_pipeline()
336
- device = _ensure_device(pipe, device, torch)
337
- generator_device = "cpu" if device == "mps" else device
338
- generator = torch.Generator(device=generator_device).manual_seed(seed)
339
- started = time.perf_counter()
340
- try:
341
- # no_grad keeps the weights ordinary tensors; inference_mode can turn
342
- # cached pipeline state into inference tensors that later moves reject.
343
- with torch.no_grad():
344
- result = pipe(
345
- prompt=_build_prompt(cue_text, style),
346
- negative_prompt=_NEGATIVE_PROMPT,
347
- num_inference_steps=steps,
348
- guidance_scale=7.0,
349
- width=size,
350
- height=size,
351
- generator=generator,
352
- )
353
- image = result.images[0]
354
- except Exception as exc:
355
- # Diagnostics only. The cue and the generated prompt are redacted
356
- # so study material never reaches the server log.
357
- reason = f"{type(exc).__name__}: {exc}".replace(cue_text, "<cue>")
358
- print(f"[image] draw failed on {device}: {reason[:300]}")
359
- if device == "cuda":
360
- try:
361
- free, total = torch.cuda.mem_get_info()
362
- print(
363
- f"[image] vram free {free / 1e9:.1f} GB "
364
- f"of {total / 1e9:.1f} GB"
365
- )
366
- except Exception:
367
- pass
368
- raise LocalImageError(
369
- "The local image model failed while drawing. Try a shorter cue, "
370
- "a smaller size, fewer steps, or close other memory-heavy apps."
371
- ) from exc
372
-
373
- elapsed = time.perf_counter() - started
374
- info = {
375
- "seed": seed,
376
- "steps": steps,
377
- "size": size,
378
- "model": IMAGE_MODEL_ID,
379
- "device": device,
380
- "seconds": round(elapsed, 1),
381
- }
382
- return image, info
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """Optional, on-device image generation from a mnemonic cue.
2
+
3
+ Nothing here is imported or downloaded until the learner opens the optional
4
+ image tab and clicks Draw. The text-only and no-model routes never load this
5
+ module's model. All generation happens on this computer after the first public
6
+ model download; no API key, account, or paid endpoint is used.
7
+ """
8
+
9
+ from __future__ import annotations
10
+
11
+ import importlib.util
12
+ import os
13
+ import secrets
14
+ import threading
15
+ import time
16
+ from functools import lru_cache
17
+ from typing import Any
18
+
19
+ # Privacy: keep Hugging Face telemetry off before diffusers is imported lazily.
20
+ os.environ["HF_HUB_DISABLE_TELEMETRY"] = "1"
21
+
22
+ DEFAULT_IMAGE_MODEL_ID = "segmind/tiny-sd"
23
+ IMAGE_MODEL_ID = (
24
+ os.getenv("MEMORY_IMAGE_MODEL", DEFAULT_IMAGE_MODEL_ID).strip()
25
+ or DEFAULT_IMAGE_MODEL_ID
26
+ )
27
+ _IMAGE_MODEL_LICENSES = {
28
+ "segmind/SSD-1B": (
29
+ "The `segmind/SSD-1B` model card lists **Apache-2.0**. The weights are "
30
+ "permissively licensed; you remain responsible for the images you "
31
+ "generate and share."
32
+ ),
33
+ "segmind/tiny-sd": (
34
+ "The `segmind/tiny-sd` model card lists **CreativeML OpenRAIL-M**. That "
35
+ "license includes use restrictions and does not claim rights over "
36
+ "generated outputs. Check the current model card before redistributing "
37
+ "weights or using outputs commercially."
38
+ ),
39
+ }
40
+ IMAGE_MODEL_LICENSE_NOTE = _IMAGE_MODEL_LICENSES.get(
41
+ IMAGE_MODEL_ID,
42
+ f"Check the `{IMAGE_MODEL_ID}` model card for license terms before "
43
+ "redistributing weights or using outputs commercially.",
44
+ )
45
+ _IMAGE_MODEL_DOWNLOADS = {
46
+ "segmind/SSD-1B": "roughly 4.5 GB",
47
+ "segmind/tiny-sd": "roughly 1 GB",
48
+ }
49
+ IMAGE_MODEL_DOWNLOAD_NOTE = _IMAGE_MODEL_DOWNLOADS.get(
50
+ IMAGE_MODEL_ID, "several gigabytes"
51
+ )
52
+ IMAGE_MODEL_CARD_URL = f"https://huggingface.co/{IMAGE_MODEL_ID}"
53
+ MAX_CUE_LENGTH = 500
54
+ IMAGE_STYLES = (
55
+ "Vivid mnemonic illustration",
56
+ "Playful cartoon",
57
+ "Dramatic photorealism",
58
+ "Hand-drawn ink sketch",
59
+ "Retro poster",
60
+ )
61
+ IMAGE_SIZE_OPTIONS = (
62
+ "512 x 512 (fastest)",
63
+ "768 x 768 (balanced)",
64
+ "1024 x 1024 (best detail)",
65
+ )
66
+ _ALLOWED_SIZES = (512, 768, 1024)
67
+
68
+
69
+ def _env_int(name: str, fallback: int) -> int:
70
+ text = os.getenv(name, "").strip()
71
+ return int(text) if text.isdigit() else fallback
72
+
73
+
74
+ DEFAULT_IMAGE_SIZE = min(
75
+ _ALLOWED_SIZES, key=lambda option: abs(option - _env_int("MEMORY_IMAGE_DEFAULT_SIZE", 512))
76
+ )
77
+ MIN_IMAGE_STEPS = 4
78
+ MAX_IMAGE_STEPS = 24
79
+ DEFAULT_IMAGE_STEPS = max(
80
+ MIN_IMAGE_STEPS,
81
+ min(_env_int("MEMORY_IMAGE_DEFAULT_STEPS", 8), MAX_IMAGE_STEPS),
82
+ )
83
+ _NEGATIVE_PROMPT = (
84
+ "text, letters, words, watermark, signature, frame, border, blurry, "
85
+ "low quality, deformed, duplicate, extra limbs, cluttered background"
86
+ )
87
+
88
+
89
+ class LocalImageError(RuntimeError):
90
+ """An image-model failure with a safe, actionable message for the UI."""
91
+
92
+ def __init__(self, public_message: str):
93
+ super().__init__(public_message)
94
+ self.public_message = public_message
95
+
96
+
97
+ def missing_dependencies() -> list[str]:
98
+ """Return the optional image dependencies that are not installed."""
99
+ return [
100
+ name
101
+ for name in ("torch", "diffusers", "transformers")
102
+ if importlib.util.find_spec(name) is None
103
+ ]
104
+
105
+
106
+ def dependencies_available() -> bool:
107
+ return not missing_dependencies()
108
+
109
+
110
+ def image_model_status_markdown(hosted: bool = False) -> str:
111
+ """Short status text that does not load or download model weights."""
112
+ missing = missing_dependencies()
113
+ if missing:
114
+ return (
115
+ "**Optional image generation is not ready.** It needs "
116
+ f"`{ '`, `'.join(missing) }`. The text and no-model routes still "
117
+ "work. Install the optional image dependencies only if you want "
118
+ "local pictures."
119
+ )
120
+ if hosted:
121
+ return (
122
+ f"**Image generation is available:** `{IMAGE_MODEL_ID}` runs on this "
123
+ "Space's shared free GPU. The text-only and no-model routes never "
124
+ "load it."
125
+ )
126
+ return (
127
+ f"**Optional local image generation is available:** `{IMAGE_MODEL_ID}`. "
128
+ "The first Draw downloads public weights and later runs use the local "
129
+ "cache. The text-only route never loads this model."
130
+ )
131
+
132
+
133
+ # One pipeline is shared by the whole process, so only one Draw runs at a time.
134
+ _PIPELINE_LOCK = threading.Lock()
135
+ _PLACED_DEVICE: dict[str, str | None] = {"device": None}
136
+
137
+
138
+ def _dtype_and_device(torch: Any) -> tuple[str, Any]:
139
+ if torch.cuda.is_available():
140
+ return "cuda", torch.float16
141
+ if getattr(torch.backends, "mps", None) and torch.backends.mps.is_available():
142
+ return "mps", torch.float16
143
+ return "cpu", torch.float32
144
+
145
+
146
+ def _from_pretrained(
147
+ pipeline_cls: Any, dtype: Any, base_kwargs: dict[str, Any]
148
+ ) -> Any:
149
+ """Load the checkpoint, tolerating fp16 variants and old kwarg names.
150
+
151
+ SDXL repositories usually ship fp16 weights. Loading those directly avoids
152
+ a full fp32 copy in memory, which matters on a 16 GB hosted container.
153
+ Older diffusers releases use ``torch_dtype`` instead of ``dtype``.
154
+ """
155
+ use_fp16 = str(dtype).endswith("float16")
156
+ variants: tuple[str | None, ...] = ("fp16", None) if use_fp16 else (None,)
157
+ last_error: Exception | None = None
158
+ for variant in variants:
159
+ for dtype_key in ("dtype", "torch_dtype"):
160
+ kwargs = {**base_kwargs, dtype_key: dtype}
161
+ if variant:
162
+ kwargs["variant"] = variant
163
+ try:
164
+ return pipeline_cls.from_pretrained(IMAGE_MODEL_ID, **kwargs)
165
+ except TypeError as exc:
166
+ last_error = exc
167
+ continue
168
+ except Exception as exc:
169
+ last_error = exc
170
+ break
171
+ raise LocalImageError(
172
+ "The local image model could not be loaded. On first use, check "
173
+ "internet access, free disk space, and available RAM; later runs use "
174
+ "the local cache. Use the text-only route if this machine cannot run "
175
+ "the image model."
176
+ ) from last_error
177
+
178
+
179
+ @lru_cache(maxsize=1)
180
+ def _load_pipeline() -> tuple[Any, str, Any]:
181
+ """Load and cache the local diffusion pipeline. Only called for a Draw."""
182
+ missing = missing_dependencies()
183
+ if missing:
184
+ raise LocalImageError(
185
+ "The optional image dependencies are not installed "
186
+ f"({', '.join(missing)}). Use the text-only route, or install "
187
+ "PyTorch, Transformers, and Diffusers and restart the app."
188
+ )
189
+
190
+ try:
191
+ import diffusers
192
+ import torch
193
+ except Exception as exc: # pragma: no cover - depends on the local install
194
+ raise LocalImageError(
195
+ "The optional image model could not be imported. Install a "
196
+ "compatible PyTorch, Transformers, and Diffusers build, or leave "
197
+ "the image tab alone."
198
+ ) from exc
199
+
200
+ # Privacy: diffusers logs the cut-off part of an over-long prompt, which
201
+ # would put study material in the console. Prompts are fitted to the
202
+ # encoder below, and diffusers' own warnings are silenced as a backstop.
203
+ diffusers.utils.logging.set_verbosity_error()
204
+
205
+ device, dtype = _dtype_and_device(torch)
206
+ # Status only: no cue or generated prompt is printed or logged.
207
+ print(
208
+ f"Loading optional local image model {IMAGE_MODEL_ID} on {device}. "
209
+ "The first run may download public weights."
210
+ )
211
+ # SD 1.x checkpoints may bundle a safety checker; SDXL pipelines ignore
212
+ # these keyword arguments.
213
+ base_kwargs: dict[str, Any] = {
214
+ "safety_checker": None,
215
+ "requires_safety_checker": False,
216
+ }
217
+ if importlib.util.find_spec("accelerate") is not None:
218
+ base_kwargs["low_cpu_mem_usage"] = True
219
+
220
+ # DiffusionPipeline reads model_index.json and picks the SD or SDXL class.
221
+ pipe = _from_pretrained(diffusers.DiffusionPipeline, dtype, base_kwargs)
222
+ try:
223
+ pipe.to(device)
224
+ _PLACED_DEVICE["device"] = device
225
+ # SDXL and SD 1.x VAE configs set force_upcast. Do NOT cast the VAE to
226
+ # float32 here: the pipelines' own upcast path also converts the fp16
227
+ # latents, and a pre-cast VAE would receive fp16 latents and fail with
228
+ # "Input type (c10::Half) and bias type (float) should be the same".
229
+ pipe.set_progress_bar_config(disable=True)
230
+ pipe.enable_attention_slicing()
231
+ if hasattr(pipe, "enable_vae_slicing"):
232
+ pipe.enable_vae_slicing()
233
+ except Exception as exc:
234
+ raise LocalImageError(
235
+ "The local image model loaded but could not be prepared on this "
236
+ "device. Use the text-only route, or try a smaller image model."
237
+ ) from exc
238
+
239
+ return pipe, device, torch
240
+
241
+
242
+ def _ensure_device(pipe: Any, device: str, torch: Any) -> str:
243
+ """Move the cached pipeline if the available device changed.
244
+
245
+ ZeroGPU attaches a GPU only inside a ``@spaces.GPU`` call, so a pipeline
246
+ loaded on CPU may need one move. This is a safety net: the Space warms up
247
+ on CUDA at startup, so normally nothing moves here.
248
+ """
249
+ wanted, _ = _dtype_and_device(torch)
250
+ if _PLACED_DEVICE["device"] == wanted:
251
+ return wanted
252
+ try:
253
+ pipe.to(wanted)
254
+ except Exception as exc:
255
+ raise LocalImageError(
256
+ "The optional image model could not be moved to the available "
257
+ "device. Restart the app, or use the text-only route."
258
+ ) from exc
259
+ _PLACED_DEVICE["device"] = wanted
260
+ return wanted
261
+
262
+
263
+ def warm_up() -> bool:
264
+ """Load the weights and place the pipeline before the first Draw.
265
+
266
+ Only used on a hosted Space: ZeroGPU needs the model on CUDA while the
267
+ module imports, and the first GPU call must not also carry a multi-gigabyte
268
+ download. The local app never calls this, so its startup stays light.
269
+ """
270
+ try:
271
+ _load_pipeline()
272
+ except LocalImageError as exc:
273
+ print(f"Optional image model was not warmed up: {exc.public_message}")
274
+ return False
275
+ return True
276
+
277
+
278
+ def _clean_cue(cue: str) -> str:
279
+ text = " ".join((cue or "").split())
280
+ if not text:
281
+ raise ValueError("Enter an image cue or scene first.")
282
+ if len(text) > MAX_CUE_LENGTH:
283
+ raise ValueError(
284
+ f"Keep the image cue under {MAX_CUE_LENGTH} characters."
285
+ )
286
+ return text
287
+
288
+
289
+ def _build_prompt(cue: str, style: str) -> str:
290
+ style_text = (style or IMAGE_STYLES[0]).strip()
291
+ return (
292
+ f"{cue}. A single clear scene for a memory mnemonic. {style_text}. "
293
+ "Exaggerated scale, bold colors, strong lighting, simple background, "
294
+ "highly memorable, no text, no watermark."
295
+ )
296
+
297
+
298
+ def _fit_prompt(pipe: Any, cue: str, style: str) -> tuple[str, str]:
299
+ """Fit the prompt to the CLIP text encoder (77 tokens).
300
+
301
+ The style suffix is dropped first, then words from the end of the cue.
302
+ Returns the prompt and the cue words that did not fit, so the UI can say
303
+ exactly what was not drawn instead of losing it silently.
304
+ """
305
+ full = _build_prompt(cue, style)
306
+ tokenizer = getattr(pipe, "tokenizer", None)
307
+ if tokenizer is None:
308
+ return full, ""
309
+ limit = int(getattr(tokenizer, "model_max_length", 77) or 77)
310
+ limit = (limit if 8 < limit <= 512 else 77) - 2 # start/end tokens
311
+
312
+ def count(text: str) -> int:
313
+ return len(tokenizer(text, add_special_tokens=False)["input_ids"])
314
+
315
+ if count(full) <= limit:
316
+ return full, ""
317
+ style_text = (style or IMAGE_STYLES[0]).strip()
318
+ compact = f"{cue}. {style_text}."
319
+ if count(compact) <= limit:
320
+ return compact, ""
321
+ words = cue.split()
322
+ keep = len(words)
323
+ while keep > 1 and count(f"{' '.join(words[:keep])}. {style_text}.") > limit:
324
+ keep -= 1
325
+ return f"{' '.join(words[:keep])}. {style_text}.", " ".join(words[keep:])
326
+
327
+
328
+ def _coerce_size(size: Any) -> int:
329
+ """Snap an arbitrary size to the sizes this app offers."""
330
+ try:
331
+ value = int(size)
332
+ except (TypeError, ValueError):
333
+ return DEFAULT_IMAGE_SIZE
334
+ return min(_ALLOWED_SIZES, key=lambda option: abs(option - value))
335
+
336
+
337
+ def generate_cue_image(
338
+ cue: str,
339
+ style: str = IMAGE_STYLES[0],
340
+ size: int = DEFAULT_IMAGE_SIZE,
341
+ steps: int = DEFAULT_IMAGE_STEPS,
342
+ seed: int = -1,
343
+ ) -> tuple[Any, dict[str, Any]]:
344
+ """Generate one local image draft and return it with reproducible settings."""
345
+ cue_text = _clean_cue(cue)
346
+ size = _coerce_size(size)
347
+ steps = max(MIN_IMAGE_STEPS, min(int(steps), MAX_IMAGE_STEPS))
348
+ seed = int(seed)
349
+ if seed < 0:
350
+ seed = secrets.randbelow(2**31 - 1)
351
+
352
+ with _PIPELINE_LOCK:
353
+ pipe, device, torch = _load_pipeline()
354
+ device = _ensure_device(pipe, device, torch)
355
+ generator_device = "cpu" if device == "mps" else device
356
+ generator = torch.Generator(device=generator_device).manual_seed(seed)
357
+ prompt, dropped = _fit_prompt(pipe, cue_text, style)
358
+ started = time.perf_counter()
359
+ try:
360
+ # no_grad keeps the weights ordinary tensors; inference_mode can turn
361
+ # cached pipeline state into inference tensors that later moves reject.
362
+ with torch.no_grad():
363
+ result = pipe(
364
+ prompt=prompt,
365
+ negative_prompt=_NEGATIVE_PROMPT,
366
+ num_inference_steps=steps,
367
+ guidance_scale=7.0,
368
+ width=size,
369
+ height=size,
370
+ generator=generator,
371
+ )
372
+ image = result.images[0]
373
+ except Exception as exc:
374
+ # Diagnostics only. The cue and the generated prompt are redacted
375
+ # so study material never reaches the server log.
376
+ reason = f"{type(exc).__name__}: {exc}".replace(cue_text, "<cue>")
377
+ print(f"[image] draw failed on {device}: {reason[:300]}")
378
+ if device == "cuda":
379
+ try:
380
+ free, total = torch.cuda.mem_get_info()
381
+ print(
382
+ f"[image] vram free {free / 1e9:.1f} GB "
383
+ f"of {total / 1e9:.1f} GB"
384
+ )
385
+ except Exception:
386
+ pass
387
+ raise LocalImageError(
388
+ "The local image model failed while drawing. Try a shorter cue, "
389
+ "a smaller size, fewer steps, or close other memory-heavy apps."
390
+ ) from exc
391
+
392
+ elapsed = time.perf_counter() - started
393
+ info = {
394
+ "seed": seed,
395
+ "steps": steps,
396
+ "size": size,
397
+ "model": IMAGE_MODEL_ID,
398
+ "device": device,
399
+ "seconds": round(elapsed, 1),
400
+ "dropped": dropped,
401
+ }
402
+ return image, info
local_llm.py CHANGED
@@ -1,808 +1,778 @@
1
- """Optional, on-device Hugging Face text generation for Memory Studio.
2
-
3
- Nothing here is imported or downloaded until the learner ticks the optional
4
- model checkbox and requests a draft. All inference happens on this computer
5
- after the first public model download. No API key, account, or paid endpoint is
6
- used. The no-model route in :mod:`memory_studio` remains fully functional.
7
- """
8
-
9
- from __future__ import annotations
10
-
11
- import importlib.util
12
- import os
13
- import re
14
- from functools import lru_cache
15
- from typing import Any, Sequence
16
-
17
- from mnemonic_core import MAJOR_SOUNDS, major_code
18
-
19
- # Privacy: disable Hugging Face Hub telemetry and tokenizer-side parallelism
20
- # warnings before transformers is imported lazily below.
21
- os.environ["HF_HUB_DISABLE_TELEMETRY"] = "1"
22
- os.environ["TOKENIZERS_PARALLELISM"] = "false"
23
-
24
- DEFAULT_MODEL_ID = "HuggingFaceTB/SmolLM2-360M-Instruct"
25
- MODEL_ID = os.getenv("MEMORY_MODEL", DEFAULT_MODEL_ID).strip() or DEFAULT_MODEL_ID
26
- MODEL_LICENSE_NOTE = (
27
- "The default SmolLM2 model card lists Apache-2.0. Check the current model "
28
- "card before redistributing weights or building a product around them."
29
- )
30
-
31
-
32
- class LocalModelError(RuntimeError):
33
- """A model failure with a safe, actionable message for the UI."""
34
-
35
- def __init__(self, public_message: str):
36
- super().__init__(public_message)
37
- self.public_message = public_message
38
-
39
-
40
- def missing_dependencies() -> list[str]:
41
- """Return the optional model dependencies that are not installed."""
42
- return [
43
- name
44
- for name in ("torch", "transformers")
45
- if importlib.util.find_spec(name) is None
46
- ]
47
-
48
-
49
- def dependencies_available() -> bool:
50
- return not missing_dependencies()
51
-
52
-
53
- def model_status_markdown(hosted: bool = False) -> str:
54
- """Short status text that does not load or download model weights."""
55
- missing = missing_dependencies()
56
- where = "this Space" if hosted else "this computer"
57
- if missing:
58
- return (
59
- "**No-model mode is active.** The optional "
60
- f"{'AI' if hosted else 'local AI'} needs "
61
- f"`{ '`, `'.join(missing) }`. The deterministic Major-System tools, "
62
- "curated words, editable tables, storyboards, and recall prompts all "
63
- "still work. Install the model dependencies only if you want AI drafts."
64
- )
65
- return (
66
- f"**Optional AI is available:** `{MODEL_ID}`. Weights are loaded only "
67
- "when you tick an AI-draft checkbox; the first use downloads public weights "
68
- f"and later runs use the cache on {where}. "
69
- + (
70
- "Drafts are processed on this Space's shared servers, not on your "
71
- "device. No API key, account, or paid endpoint is used."
72
- if hosted
73
- else "Inference stays on this computer; no API key, account, or paid "
74
- "endpoint is used."
75
- )
76
- )
77
-
78
-
79
- @lru_cache(maxsize=1)
80
- def _load_bundle() -> tuple[Any, Any, str, Any]:
81
- """Load and cache the local model once. Only called for an AI draft."""
82
- missing = missing_dependencies()
83
- if missing:
84
- raise LocalModelError(
85
- "The optional local model dependencies are not installed "
86
- f"({', '.join(missing)}). Use no-model mode, or install PyTorch and "
87
- "Transformers and restart the app."
88
- )
89
-
90
- try:
91
- import torch
92
- from transformers import AutoModelForCausalLM, AutoTokenizer
93
- except Exception as exc: # pragma: no cover - depends on the local install
94
- raise LocalModelError(
95
- "The optional local model could not be imported. Install a compatible "
96
- "PyTorch and Transformers build, or leave the AI-draft box unchecked."
97
- ) from exc
98
-
99
- if torch.cuda.is_available():
100
- device = "cuda"
101
- dtype = torch.float16
102
- elif getattr(torch.backends, "mps", None) and torch.backends.mps.is_available():
103
- device = "mps"
104
- dtype = torch.float16
105
- else:
106
- device = "cpu"
107
- dtype = torch.float32
108
-
109
- # Status only: no submitted study material is printed or logged.
110
- print(
111
- f"Loading optional local model {MODEL_ID} on {device}. "
112
- "The first run may download public weights."
113
- )
114
- try:
115
- tokenizer = AutoTokenizer.from_pretrained(MODEL_ID, trust_remote_code=False)
116
- model = AutoModelForCausalLM.from_pretrained(
117
- MODEL_ID,
118
- torch_dtype=dtype,
119
- low_cpu_mem_usage=True,
120
- trust_remote_code=False,
121
- )
122
- model.to(device)
123
- model.eval()
124
- except Exception as exc:
125
- raise LocalModelError(
126
- "The local model could not be loaded. For the first use, check internet "
127
- "access, free disk space, and available RAM; later runs use the local "
128
- "cache. Use no-model mode if this machine cannot run the model."
129
- ) from exc
130
-
131
- return tokenizer, model, device, torch
132
-
133
-
134
- def _generate(
135
- messages: list[dict[str, str]],
136
- max_new_tokens: int = 320,
137
- temperature: float = 0.7,
138
- ) -> str:
139
- """Generate one reply with the local model's own chat template."""
140
- tokenizer, model, device, torch = _load_bundle()
141
- try:
142
- # Transformers 5 returns a BatchEncoding here; older releases return a
143
- # tensor. Support both and always pass named model inputs.
144
- encoded = tokenizer.apply_chat_template(
145
- messages,
146
- add_generation_prompt=True,
147
- tokenize=True,
148
- return_tensors="pt",
149
- return_dict=True,
150
- )
151
- if hasattr(encoded, "items"):
152
- model_inputs = {
153
- key: value.to(device) for key, value in encoded.items()
154
- }
155
- else: # pragma: no cover - compatibility with older tokenizers
156
- model_inputs = {"input_ids": encoded.to(device)}
157
- input_ids = model_inputs["input_ids"]
158
- prompt_length = input_ids.shape[-1]
159
-
160
- generation_args: dict[str, Any] = {
161
- "max_new_tokens": max(32, min(int(max_new_tokens), 900)),
162
- "pad_token_id": tokenizer.pad_token_id or tokenizer.eos_token_id,
163
- "repetition_penalty": 1.08,
164
- }
165
- if tokenizer.eos_token_id is not None:
166
- generation_args["eos_token_id"] = tokenizer.eos_token_id
167
- if temperature > 0:
168
- generation_args.update(
169
- {
170
- "do_sample": True,
171
- "temperature": float(temperature),
172
- "top_p": 0.9,
173
- }
174
- )
175
- else:
176
- generation_args["do_sample"] = False
177
-
178
- with torch.inference_mode():
179
- output_ids = model.generate(**model_inputs, **generation_args)
180
- new_tokens = output_ids[0, prompt_length:]
181
- text = tokenizer.decode(new_tokens, skip_special_tokens=True).strip()
182
- except Exception as exc:
183
- raise LocalModelError(
184
- "The local model failed while generating a draft. Try a shorter input, "
185
- "close other memory-heavy apps, or use no-model mode."
186
- ) from exc
187
-
188
- if not text:
189
- raise LocalModelError(
190
- "The local model returned an empty draft. Try again or use no-model mode."
191
- )
192
- return text
193
-
194
-
195
- def ask_number_candidates(chunks: Sequence[str]) -> str:
196
- """Ask for one Major-System candidate per two-digit chunk."""
197
- code_list = "\n".join(
198
- f"- {chunk}: " + ", ".join(MAJOR_SOUNDS[digit] for digit in chunk)
199
- for chunk in chunks
200
- )
201
- return _generate(
202
- [
203
- {
204
- "role": "system",
205
- "content": (
206
- "You are a careful memory coach teaching the traditional Major "
207
- "System. Do not invent or change the digit-to-sound mapping. "
208
- "Return only the requested rows."
209
- ),
210
- },
211
- {
212
- "role": "user",
213
- "content": f"""Create one everyday, concrete English noun for each number chunk.
214
-
215
- Major System:
216
- 0 = S/Z/soft C
217
- 1 = T/D/TH
218
- 2 = N
219
- 3 = M
220
- 4 = R
221
- 5 = L
222
- 6 = J/SH/CH/soft G
223
- 7 = K/hard C/hard G/Q/CK/NG
224
- 8 = F/V/PH
225
- 9 = B/P
226
- Vowels and H/W/Y are fillers and do not encode digits.
227
-
228
- Examples:
229
- 31 | mat | a bright red doormat
230
- 41 | rat | a giant brown rat
231
- 59 | lip | enormous red lips
232
-
233
- Required chunks:
234
- {code_list}
235
-
236
- Rules:
237
- - Return exactly one row for every required chunk, in the same order.
238
- - Format every row exactly as: CHUNK | WORD | SIMPLE VISUAL IMAGE CUE
239
- - The word's consonant sounds must encode exactly the chunk, in order.
240
- - Do not add extra coded consonant sounds.
241
- - Prefer short, familiar, easy-to-picture nouns.
242
- - The image cue is one concrete phrase, not a definition.
243
- - Do not write an introduction, explanation, story, or conclusion.
244
- """,
245
- },
246
- ],
247
- max_new_tokens=min(520, 80 + len(chunks) * 42),
248
- temperature=0.65,
249
- )
250
-
251
-
252
- def ask_number_story(rows: Sequence[tuple[str, str, str]]) -> str:
253
- """Ask for a vivid story that keeps all reviewed number images in order."""
254
- image_list = "\n".join(
255
- f"{index}. {chunk} = {word} ({cue})"
256
- for index, (chunk, word, cue) in enumerate(rows, 1)
257
- )
258
- return _generate(
259
- [
260
- {
261
- "role": "system",
262
- "content": (
263
- "You are a creative memory coach. Create vivid, ordered, "
264
- "easy-to-recall mental scenes. Never omit or reorder an item."
265
- ),
266
- },
267
- {
268
- "role": "user",
269
- "content": f"""The ordered image words are:
270
- {image_list}
271
-
272
- Write a short, bizarre, visual story that uses each listed word at least once,
273
- in exactly this order. Use the exact word, not a synonym, and make every item
274
- interact with the next. Exaggerate scale, motion, color, sound, and emotion.
275
- Do not add, omit, rename, or reorder the listed words. Do not explain the Major
276
- System. End with a one-line retrieval cue that does not reveal the words.
277
- """,
278
- },
279
- ],
280
- max_new_tokens=min(460, 130 + len(rows) * 24),
281
- temperature=0.8,
282
- )
283
-
284
-
285
- def ask_list_cues(items: Sequence[str]) -> str:
286
- """Ask for one image cue per ordered list item."""
287
- item_list = "\n".join(
288
- f"{index}. {item}" for index, item in enumerate(items, 1)
289
- )
290
- return _generate(
291
- [
292
- {
293
- "role": "system",
294
- "content": (
295
- "You are a practical memory coach. Suggest concrete, vivid, "
296
- "unusual imagery. Do not invent factual definitions."
297
- ),
298
- },
299
- {
300
- "role": "user",
301
- "content": f"""Ordered list:
302
- {item_list}
303
-
304
- For each item, invent one concrete visual image cue that makes the item easy to
305
- picture. Keep each cue to one short phrase. If an item is a concept rather than
306
- an object, suggest a visual symbol or a brief pictured situation and treat it as
307
- a memory cue, not a definition. Do not explain or define any item.
308
-
309
- Return exactly one row per index, in order, using this format:
310
- INDEX | IMAGE CUE
311
-
312
- Do not repeat the item text. Do not write an introduction or conclusion.
313
- """,
314
- },
315
- ],
316
- max_new_tokens=min(900, 100 + len(items) * 42),
317
- temperature=0.75,
318
- )
319
-
320
-
321
- def ask_list_story(rows: Sequence[tuple[str, str]]) -> str:
322
- """Ask for a story that keeps every exact list label in order."""
323
- item_list = "\n".join(
324
- f"{index}. {item} - {cue or 'make the image vivid and exaggerated'}"
325
- for index, (item, cue) in enumerate(rows, 1)
326
- )
327
- return _generate(
328
- [
329
- {
330
- "role": "system",
331
- "content": (
332
- "You are a practical memory coach using visual chaining. "
333
- "Preserve exact list labels and their order. Do not invent "
334
- "factual definitions."
335
- ),
336
- },
337
- {
338
- "role": "user",
339
- "content": f"""Ordered items and image cues:
340
- {item_list}
341
-
342
- Write a short, vivid story that uses every exact item label at least once, in
343
- exactly this order. Do not omit, rename, reorder, or add list items. Make the
344
- images interact and keep the action moving. For technical or abstract labels,
345
- keep the label unchanged and do not assert what it means. End with a one-line
346
- retrieval cue that does not reveal the items.
347
- """,
348
- },
349
- ],
350
- max_new_tokens=min(820, 150 + len(rows) * 26),
351
- temperature=0.8,
352
- )
353
-
354
-
355
- def ask_scene_story(
356
- cues: Sequence[tuple[str, str]], kind: str = "number"
357
- ) -> str:
358
- """Ask for one continuous picture story that weaves every selected cue.
359
-
360
- ``cues`` is an ordered list of ``(subject, image cue)`` pairs taken from the
361
- checked storyboard rows, in the learner's chosen order.
362
- """
363
- cue_list = "\n".join(
364
- f"{index}. {subject} - {cue}" for index, (subject, cue) in enumerate(cues, 1)
365
- )
366
- exact = (
367
- "the exact chain word" if kind == "number" else "the exact item label"
368
- )
369
- return _generate(
370
- [
371
- {
372
- "role": "system",
373
- "content": (
374
- "You are a creative memory coach. Write one continuous, vivid "
375
- "picture story that keeps every element in the exact order it is "
376
- "given. Never omit, rename, or reorder an element."
377
- ),
378
- },
379
- {
380
- "role": "user",
381
- "content": f"""The ordered chain elements (subject - image cue) are:
382
- {cue_list}
383
-
384
- Write one short, bizarre, vivid story or scene description that lets an artist draw the whole
385
- chain as a single illustration. Rules:
386
- - Mention {exact} for every step above, at least once, in exactly this order.
387
- - Use the cues' concrete imagery: exaggerated scale, bold color, motion, sound.
388
- - Make consecutive elements interact so the chain reads as one continuous scene.
389
- - Do not omit, rename, reorder, or add major elements.
390
- - One paragraph, 40-110 words. Do not explain mnemonic theory. Do not address the reader.
391
- """,
392
- },
393
- ],
394
- max_new_tokens=min(430, 120 + len(cues) * 26),
395
- temperature=0.8,
396
- )
397
-
398
-
399
- def ask_word_fields(term: str, context: str) -> str:
400
- """Ask for a structured draft while guarding against invented definitions."""
401
- supplied = context.strip()
402
- context_note = (
403
- f"""The learner supplied this context/definition:
404
- {supplied}
405
-
406
- Use it as the factual anchor. Do not add facts that are not present in it.
407
- """
408
- if supplied
409
- else """No context or definition was supplied.
410
-
411
- Do not invent or assert a definition. For MEANING, write exactly:
412
- No definition supplied
413
- Treat the rest as a mnemonic for the label only.
414
- """
415
- )
416
- return _generate(
417
- [
418
- {
419
- "role": "system",
420
- "content": (
421
- "You are a memory coach. Help the learner remember a word or "
422
- "concept with sound association, vivid imagery, movement, and a "
423
- "familiar location. Never present an invented definition as fact. "
424
- "Follow the requested labels exactly."
425
- ),
426
- },
427
- {
428
- "role": "user",
429
- "content": f"""Term or concept:
430
- {term.strip()}
431
-
432
- {context_note}
433
- Return exactly these six sections, using these labels:
434
- MEANING: one factual anchor sentence, or the exact no-definition sentence above
435
- SOUND: one sound or keyword association
436
- IMAGE: one concrete, exaggerated visual image
437
- STORY: a 2-4 sentence strange, moving mini-story
438
- PALACE: one familiar place and where to put the image
439
- RECALL: one active-recall question that does not reveal the term
440
-
441
- Do not add other headings, an introduction, or a conclusion. Keep each section
442
- short. Do not claim that an imagery cue is a fact.
443
- """,
444
- },
445
- ],
446
- max_new_tokens=380,
447
- temperature=0.7,
448
- )
449
-
450
-
451
- def _strip_md(text: str) -> str:
452
- return re.sub(r"[*_`]+", "", text or "").strip()
453
-
454
-
455
- def _pipe_cells(line: str) -> list[str]:
456
- """Split a Markdown-style pipe row without keeping edge empties."""
457
- parts = line.split("|")
458
- if parts and not parts[0].strip():
459
- parts = parts[1:]
460
- if parts and not parts[-1].strip():
461
- parts = parts[:-1]
462
- return [_strip_md(part) for part in parts]
463
-
464
-
465
- def _clean_code(raw: str, requested: Sequence[str] | None) -> str | None:
466
- """Normalize a model-written chunk, including lost leading zeros."""
467
- groups = re.findall(r"\d+", raw or "")
468
- if not groups:
469
- return None
470
- code = groups[-1]
471
- if not requested:
472
- return code
473
- requested_list = list(requested)
474
- if code in requested_list:
475
- return code
476
- padded = code.zfill(2)
477
- if padded in requested_list:
478
- return padded
479
- if code.startswith("0") and code[1:] in requested_list:
480
- return code[1:]
481
- matches = [
482
- item for item in requested_list if item.lstrip("0") == code.lstrip("0")
483
- ]
484
- if len(matches) == 1:
485
- return matches[0]
486
- return None
487
-
488
-
489
- def parse_number_rows(
490
- response: str, requested_chunks: Sequence[str] | None = None
491
- ) -> dict[str, tuple[str, str]]:
492
- """Parse pipe rows and the grouped field format small models often emit."""
493
- parsed: dict[str, tuple[str, str]] = {}
494
- current_code: str | None = None
495
- current_word = ""
496
- current_image = ""
497
-
498
- def resolve_code(raw_code: str, word: str) -> str | None:
499
- """Trust the model's label, or recover the code from the word itself."""
500
- code = _clean_code(raw_code, requested_chunks)
501
- if code:
502
- return code
503
- detected = major_code(word)
504
- if detected and (not requested_chunks or detected in requested_chunks):
505
- return detected
506
- return None
507
-
508
- def commit() -> None:
509
- nonlocal current_word, current_image
510
- code = current_code
511
- if not code and current_word:
512
- code = resolve_code("", current_word)
513
- if code and current_word and re.search(r"[A-Za-z]", current_word):
514
- parsed.setdefault(
515
- code,
516
- (current_word, current_image or "make this image vivid and exaggerated"),
517
- )
518
- current_word = ""
519
- current_image = ""
520
-
521
- for raw_line in (response or "").splitlines():
522
- line = raw_line.strip()
523
- if not line or line.startswith("```"):
524
- continue
525
- line = re.sub(r"^\s*[-*\u2022]\s*", "", line)
526
-
527
- # Preferred compact row: 31 | mat | a bright red doormat
528
- if "|" in line:
529
- cells = _pipe_cells(line)
530
- if len(cells) >= 2:
531
- if (
532
- cells[0].casefold()
533
- in {"chunk", "chunks", "number", "code", "digits", "#"}
534
- or cells[1].casefold() in {"word", "candidate", "noun"}
535
- ):
536
- continue
537
- word = cells[1]
538
- code = resolve_code(cells[0], word)
539
- image = " | ".join(part for part in cells[2:] if part).strip()
540
- if code and re.search(r"[A-Za-z]", word):
541
- commit()
542
- parsed.setdefault(
543
- code,
544
- (word, image or "make this image vivid and exaggerated"),
545
- )
546
- continue
547
-
548
- # Explanatory field lines: Chunk: 31 / Word: mat / Image: ...
549
- field_match = re.match(
550
- r"^(chunk|word|image(?:\s*cue)?)\s*[:\-\u2013]\s*(.+)$",
551
- line,
552
- re.IGNORECASE,
553
- )
554
- if field_match:
555
- key = field_match.group(1).casefold()
556
- value = _strip_md(field_match.group(2))
557
- if key == "chunk":
558
- code = _clean_code(value, requested_chunks)
559
- if code:
560
- commit()
561
- current_code = code
562
- elif key == "word" and current_code:
563
- current_word = value
564
- if current_image:
565
- commit()
566
- elif key.startswith("image") and current_code:
567
- current_image = value
568
- commit()
569
- continue
570
-
571
- # Number-led line: 1. 31: Mat (M) or 31: mat - a doormat
572
- code_line = re.match(
573
- r"^(?:\d+[.)]\s+)?(\d{1,2})\s*[.):\-\u2013]\s*(.+)$", line
574
- )
575
- if not code_line:
576
- code_line = re.match(
577
- r"^(?:\d+[.)]\s+)?(\d{1,2})\s+([A-Za-z].*)$", line
578
- )
579
- if code_line:
580
- remainder = _strip_md(code_line.group(2))
581
- split = re.match(
582
- r"^([A-Za-z][A-Za-z' -]*?)\s*[:\-\u2013]\s*(.+)$", remainder
583
- )
584
- provisional_word = split.group(1).strip() if split else remainder
585
- provisional_word = re.sub(
586
- r"\s*\([^)]*\)\s*$", "", provisional_word
587
- ).strip()
588
- code = resolve_code(code_line.group(1), provisional_word)
589
- if not code:
590
- continue
591
- commit()
592
- current_code = code
593
- if split:
594
- current_word = split.group(1).strip()
595
- current_image = split.group(2).strip()
596
- commit()
597
- else:
598
- # Drop trailing pronunciation notes such as "Mat (M)".
599
- current_word = provisional_word
600
- if not re.search(r"[A-Za-z]", current_word):
601
- current_word = ""
602
-
603
- commit()
604
- return parsed
605
-
606
-
607
- _CUE_LABEL = re.compile(
608
- r"^(?:image\s*cue|visual\s*cue|cue|image)\s*[:\-\u2013]\s*", re.IGNORECASE
609
- )
610
-
611
-
612
- def _clean_cue(value: str) -> str:
613
- """Remove a redundant 'Image Cue:' prefix and Markdown emphasis."""
614
- return re.sub(_CUE_LABEL, "", _strip_md(value)).strip()
615
-
616
-
617
- def parse_indexed_cues(response: str, count: int) -> dict[int, str]:
618
- """Parse pipe rows, numbered rows, and Index/Image Cue field blocks."""
619
- generic: dict[int, str] = {}
620
- explicit: dict[int, str] = {}
621
- current_index: int | None = None
622
-
623
- for raw_line in (response or "").splitlines():
624
- line = raw_line.strip()
625
- if not line or line.startswith("```"):
626
- continue
627
- line = re.sub(r"^\s*[-*\u2022]\s*", "", line)
628
-
629
- # Field-style heading: Index: 1. Milk / Item 2: Eggs
630
- index_heading = re.match(
631
- r"^(?:index|item)\s*[:\-\u2013]?\s*(\d{1,2})\b", line, re.IGNORECASE
632
- )
633
- if index_heading:
634
- current_index = int(index_heading.group(1))
635
- continue
636
-
637
- # Field-style cue on the next line: Image Cue: a giant milk carton
638
- cue_field = re.match(
639
- r"^(?:image\s*cue|visual\s*cue|cue|image)\s*[:\-\u2013]\s*(.+)$",
640
- line,
641
- re.IGNORECASE,
642
- )
643
- if cue_field and current_index is not None:
644
- cue = _clean_cue(cue_field.group(1))
645
- if cue and 1 <= current_index <= count:
646
- explicit.setdefault(current_index - 1, cue)
647
- continue
648
-
649
- index: int | None = None
650
- cue = ""
651
- if "|" in line:
652
- cells = _pipe_cells(line)
653
- index_match = re.search(r"\d+", cells[0]) if cells else None
654
- if not index_match:
655
- continue
656
- index = int(index_match.group())
657
- if len(cells) >= 3:
658
- cue = cells[-1]
659
- elif len(cells) == 2:
660
- cue = cells[1]
661
- else:
662
- match = re.match(r"^(\d{1,2})\s*[.)]\s*(.+)$", line)
663
- if not match:
664
- match = re.match(r"^(\d{1,2})\s*[:\-\u2013]\s*(.+)$", line)
665
- if match:
666
- index = int(match.group(1))
667
- cue = _strip_md(match.group(2))
668
-
669
- if index is None or not 1 <= index <= count:
670
- continue
671
- current_index = index
672
- cue = _clean_cue(cue)
673
- if cue:
674
- generic.setdefault(index - 1, cue)
675
-
676
- # An explicit Image Cue line is more trustworthy than a merely numbered row.
677
- for index, cue in explicit.items():
678
- generic[index] = cue
679
- return generic
680
-
681
-
682
- _WORD_ALIASES = {
683
- "meaning": "Meaning anchor",
684
- "meaning anchor": "Meaning anchor",
685
- "simple meaning": "Meaning anchor",
686
- "factual anchor": "Meaning anchor",
687
- "meaning / factual anchor": "Meaning anchor",
688
- "simple meaning / factual anchor": "Meaning anchor",
689
- "sound": "Sound / keyword association",
690
- "keyword": "Sound / keyword association",
691
- "keyword association": "Sound / keyword association",
692
- "sound association": "Sound / keyword association",
693
- "sound or keyword association": "Sound / keyword association",
694
- "sound / keyword association": "Sound / keyword association",
695
- "image": "Visual image",
696
- "image cue": "Visual image",
697
- "visual image": "Visual image",
698
- "visual image cue": "Visual image",
699
- "visual": "Visual image",
700
- "story": "Mini-story",
701
- "mini story": "Mini-story",
702
- "mini-story": "Mini-story",
703
- "palace": "Memory Palace placement",
704
- "memory palace": "Memory Palace placement",
705
- "memory palace location": "Memory Palace placement",
706
- "memory palace placement": "Memory Palace placement",
707
- "location": "Memory Palace placement",
708
- "recall": "Active-recall question",
709
- "question": "Active-recall question",
710
- "active recall": "Active-recall question",
711
- "active-recall": "Active-recall question",
712
- "active recall question": "Active-recall question",
713
- "active-recall question": "Active-recall question",
714
- "recall question": "Active-recall question",
715
- }
716
-
717
-
718
- def _normalize_label(text: str) -> str:
719
- text = _strip_md(text)
720
- text = re.sub(r"^\s*(?:\d+[.)]\s*|[-*\u2022]\s*)", "", text)
721
- text = text.strip(" :.\u2013\u2014-")
722
- text = text.replace("&", "and")
723
- return re.sub(r"\s+", " ", text).strip().lower()
724
-
725
-
726
- def _match_word_label(line: str) -> tuple[str | None, str]:
727
- """Recognize both inline labels and heading-only field labels."""
728
- candidate = re.sub(r"^\s*[-*\u2022]\s*", "", _strip_md(line)).strip()
729
- inline = re.match(
730
- r"^([A-Za-z][A-Za-z0-9 /'&()\-]{0,60}?)\s*[:\-\u2013\u2014]\s*(.*)$",
731
- candidate,
732
- )
733
- if inline:
734
- canonical = _WORD_ALIASES.get(_normalize_label(inline.group(1)))
735
- if canonical:
736
- return canonical, inline.group(2).strip()
737
-
738
- heading = candidate.strip(" :.\u2013\u2014-")
739
- canonical = _WORD_ALIASES.get(_normalize_label(heading))
740
- if canonical:
741
- return canonical, ""
742
- return None, ""
743
-
744
-
745
- def _split_inline_word_sections(line: str) -> list[str]:
746
- """Split one-line pipe-separated drafts only when labels are recognizable."""
747
- segments = [part.strip() for part in re.split(r"\s*\|\s*", line) if part.strip()]
748
- if len(segments) < 2:
749
- return [line]
750
- recognized = sum(1 for segment in segments if _match_word_label(segment)[0])
751
- return segments if recognized >= 2 else [line]
752
-
753
-
754
- def parse_word_sections(response: str) -> tuple[dict[str, str], str]:
755
- """Parse labeled word-card sections, preserving unrecognized prose."""
756
- sections: dict[str, list[str]] = {}
757
- preamble: list[str] = []
758
- current: str | None = None
759
-
760
- for raw_line in (response or "").splitlines():
761
- line = raw_line.strip()
762
- if not line or line.startswith("```"):
763
- continue
764
- for part in _split_inline_word_sections(line):
765
- label, rest = _match_word_label(part)
766
- if label:
767
- current = label
768
- sections.setdefault(label, [])
769
- if rest:
770
- sections[label].append(rest)
771
- continue
772
- if current:
773
- sections[current].append(_strip_md(part))
774
- else:
775
- preamble.append(_strip_md(part))
776
-
777
- cleaned = {
778
- label: re.sub(r"\s+", " ", " ".join(parts)).strip()
779
- for label, parts in sections.items()
780
- }
781
- return cleaned, " ".join(preamble).strip()
782
-
783
-
784
- def text_mentions_in_order(
785
- labels: Sequence[str], text: str
786
- ) -> tuple[bool, str | None]:
787
- """Check that every label appears at least once, in the supplied order.
788
-
789
- This is an omission/order guard, not a claim that the story contains no extra
790
- words. It intentionally accepts exact labels only so a synonym cannot silently
791
- replace a list item.
792
- """
793
- haystack = (text or "").casefold()
794
- position = 0
795
- for label in labels:
796
- needle = label.casefold().strip()
797
- if not needle:
798
- return False, label
799
- pattern = re.compile(
800
- r"(?<![a-z0-9])" + re.escape(needle) + r"(?![a-z0-9])"
801
- )
802
- match = pattern.search(haystack, position)
803
- if not match:
804
- return False, label
805
- position = match.end()
806
- return True, None
807
-
808
-
 
1
+ """Optional, on-device Hugging Face text generation for Memory Studio.
2
+
3
+ Nothing here is imported or downloaded until the learner ticks the optional
4
+ model checkbox and requests a draft. All inference happens on this computer
5
+ after the first public model download. No API key, account, or paid endpoint is
6
+ used. The no-model route in :mod:`memory_studio` remains fully functional.
7
+ """
8
+
9
+ from __future__ import annotations
10
+
11
+ import importlib.util
12
+ import os
13
+ import re
14
+ import threading
15
+ from functools import lru_cache
16
+ from typing import Any, Sequence
17
+
18
+ from mnemonic_core import MAJOR_SOUNDS, major_code
19
+
20
+ # Privacy: disable Hugging Face Hub telemetry and tokenizer-side parallelism
21
+ # warnings before transformers is imported lazily below.
22
+ os.environ["HF_HUB_DISABLE_TELEMETRY"] = "1"
23
+ os.environ["TOKENIZERS_PARALLELISM"] = "false"
24
+
25
+ DEFAULT_MODEL_ID = "HuggingFaceTB/SmolLM2-360M-Instruct"
26
+ MODEL_ID = os.getenv("MEMORY_MODEL", DEFAULT_MODEL_ID).strip() or DEFAULT_MODEL_ID
27
+ MODEL_LICENSE_NOTE = (
28
+ "The default SmolLM2 model card lists Apache-2.0. Check the current model "
29
+ "card before redistributing weights or building a product around them."
30
+ )
31
+
32
+
33
+ class LocalModelError(RuntimeError):
34
+ """A model failure with a safe, actionable message for the UI."""
35
+
36
+ def __init__(self, public_message: str):
37
+ super().__init__(public_message)
38
+ self.public_message = public_message
39
+
40
+
41
+ def missing_dependencies() -> list[str]:
42
+ """Return the optional model dependencies that are not installed."""
43
+ return [
44
+ name
45
+ for name in ("torch", "transformers")
46
+ if importlib.util.find_spec(name) is None
47
+ ]
48
+
49
+
50
+ def dependencies_available() -> bool:
51
+ return not missing_dependencies()
52
+
53
+
54
+ def model_status_markdown(hosted: bool = False) -> str:
55
+ """Short status text that does not load or download model weights."""
56
+ missing = missing_dependencies()
57
+ where = "this Space" if hosted else "this computer"
58
+ if missing:
59
+ return (
60
+ "**No-model mode is active.** The optional "
61
+ f"{'AI' if hosted else 'local AI'} needs "
62
+ f"`{ '`, `'.join(missing) }`. The deterministic Major-System tools, "
63
+ "curated words, editable tables, storyboards, and recall prompts all "
64
+ "still work. Install the model dependencies only if you want AI drafts."
65
+ )
66
+ return (
67
+ f"**Optional AI is available:** `{MODEL_ID}`. Weights are loaded only "
68
+ "when you tick an AI-draft checkbox; the first use downloads public weights "
69
+ f"and later runs use the cache on {where}. "
70
+ + (
71
+ "Drafts are processed on this Space's shared servers, not on your "
72
+ "device. No API key, account, or paid endpoint is used."
73
+ if hosted
74
+ else "Inference stays on this computer; no API key, account, or paid "
75
+ "endpoint is used."
76
+ )
77
+ )
78
+
79
+
80
+ # Gradio's concurrency limit is per event, so two tabs can ask for a draft at
81
+ # once. One lock keeps the model from loading twice and serializes generation.
82
+ _MODEL_LOCK = threading.RLock()
83
+
84
+
85
+ def _load_bundle() -> tuple[Any, Any, str, Any]:
86
+ with _MODEL_LOCK:
87
+ return _load_bundle_once()
88
+
89
+
90
+ @lru_cache(maxsize=1)
91
+ def _load_bundle_once() -> tuple[Any, Any, str, Any]:
92
+ """Load and cache the local model once. Only called for an AI draft."""
93
+ missing = missing_dependencies()
94
+ if missing:
95
+ raise LocalModelError(
96
+ "The optional local model dependencies are not installed "
97
+ f"({', '.join(missing)}). Use no-model mode, or install PyTorch and "
98
+ "Transformers and restart the app."
99
+ )
100
+
101
+ try:
102
+ import torch
103
+ from transformers import AutoModelForCausalLM, AutoTokenizer
104
+ except Exception as exc: # pragma: no cover - depends on the local install
105
+ raise LocalModelError(
106
+ "The optional local model could not be imported. Install a compatible "
107
+ "PyTorch and Transformers build, or leave the AI-draft box unchecked."
108
+ ) from exc
109
+
110
+ if torch.cuda.is_available():
111
+ device = "cuda"
112
+ dtype = torch.float16
113
+ elif getattr(torch.backends, "mps", None) and torch.backends.mps.is_available():
114
+ device = "mps"
115
+ dtype = torch.float16
116
+ else:
117
+ device = "cpu"
118
+ dtype = torch.float32
119
+
120
+ # Status only: no submitted study material is printed or logged.
121
+ print(
122
+ f"Loading optional local model {MODEL_ID} on {device}. "
123
+ "The first run may download public weights."
124
+ )
125
+ try:
126
+ tokenizer = AutoTokenizer.from_pretrained(MODEL_ID, trust_remote_code=False)
127
+ load_kwargs = {"low_cpu_mem_usage": True, "trust_remote_code": False}
128
+ try:
129
+ # Transformers 4.56+ names this ``dtype``; ``torch_dtype`` warns.
130
+ model = AutoModelForCausalLM.from_pretrained(
131
+ MODEL_ID, dtype=dtype, **load_kwargs
132
+ )
133
+ except TypeError: # pragma: no cover - older Transformers releases
134
+ model = AutoModelForCausalLM.from_pretrained(
135
+ MODEL_ID, torch_dtype=dtype, **load_kwargs
136
+ )
137
+ model.to(device)
138
+ model.eval()
139
+ except Exception as exc:
140
+ raise LocalModelError(
141
+ "The local model could not be loaded. For the first use, check internet "
142
+ "access, free disk space, and available RAM; later runs use the local "
143
+ "cache. Use no-model mode if this machine cannot run the model."
144
+ ) from exc
145
+
146
+ return tokenizer, model, device, torch
147
+
148
+
149
+ def _generate(
150
+ messages: list[dict[str, str]],
151
+ max_new_tokens: int = 320,
152
+ temperature: float = 0.7,
153
+ ) -> str:
154
+ """Generate one reply with the local model's own chat template."""
155
+ with _MODEL_LOCK:
156
+ return _generate_locked(messages, max_new_tokens, temperature)
157
+
158
+
159
+ def _generate_locked(
160
+ messages: list[dict[str, str]],
161
+ max_new_tokens: int,
162
+ temperature: float,
163
+ ) -> str:
164
+ tokenizer, model, device, torch = _load_bundle()
165
+ try:
166
+ # Transformers 5 returns a BatchEncoding here; older releases return a
167
+ # tensor. Support both and always pass named model inputs.
168
+ encoded = tokenizer.apply_chat_template(
169
+ messages,
170
+ add_generation_prompt=True,
171
+ tokenize=True,
172
+ return_tensors="pt",
173
+ return_dict=True,
174
+ )
175
+ if hasattr(encoded, "items"):
176
+ model_inputs = {
177
+ key: value.to(device) for key, value in encoded.items()
178
+ }
179
+ else: # pragma: no cover - compatibility with older tokenizers
180
+ model_inputs = {"input_ids": encoded.to(device)}
181
+ input_ids = model_inputs["input_ids"]
182
+ prompt_length = input_ids.shape[-1]
183
+
184
+ generation_args: dict[str, Any] = {
185
+ "max_new_tokens": max(32, min(int(max_new_tokens), 900)),
186
+ "pad_token_id": tokenizer.pad_token_id or tokenizer.eos_token_id,
187
+ "repetition_penalty": 1.08,
188
+ }
189
+ if tokenizer.eos_token_id is not None:
190
+ generation_args["eos_token_id"] = tokenizer.eos_token_id
191
+ if temperature > 0:
192
+ generation_args.update(
193
+ {
194
+ "do_sample": True,
195
+ "temperature": float(temperature),
196
+ "top_p": 0.9,
197
+ }
198
+ )
199
+ else:
200
+ generation_args["do_sample"] = False
201
+
202
+ with torch.inference_mode():
203
+ output_ids = model.generate(**model_inputs, **generation_args)
204
+ new_tokens = output_ids[0, prompt_length:]
205
+ text = tokenizer.decode(new_tokens, skip_special_tokens=True).strip()
206
+ except Exception as exc:
207
+ raise LocalModelError(
208
+ "The local model failed while generating a draft. Try a shorter input, "
209
+ "close other memory-heavy apps, or use no-model mode."
210
+ ) from exc
211
+
212
+ if not text:
213
+ raise LocalModelError(
214
+ "The local model returned an empty draft. Try again or use no-model mode."
215
+ )
216
+ return text
217
+
218
+
219
+ def ask_number_candidates(chunks: Sequence[str]) -> str:
220
+ """Ask for one Major-System candidate per two-digit chunk."""
221
+ code_list = "\n".join(
222
+ f"- {chunk}: " + ", ".join(MAJOR_SOUNDS[digit] for digit in chunk)
223
+ for chunk in chunks
224
+ )
225
+ return _generate(
226
+ [
227
+ {
228
+ "role": "system",
229
+ "content": (
230
+ "You are a careful memory coach teaching the traditional Major "
231
+ "System. Do not invent or change the digit-to-sound mapping. "
232
+ "Return only the requested rows."
233
+ ),
234
+ },
235
+ {
236
+ "role": "user",
237
+ "content": f"""Create one everyday, concrete English noun for each number chunk.
238
+
239
+ Major System:
240
+ 0 = S/Z/soft C
241
+ 1 = T/D/TH
242
+ 2 = N
243
+ 3 = M
244
+ 4 = R
245
+ 5 = L
246
+ 6 = J/SH/CH/soft G
247
+ 7 = K/hard C/hard G/Q/CK/NG
248
+ 8 = F/V/PH
249
+ 9 = B/P
250
+ Vowels and H/W/Y are fillers and do not encode digits.
251
+
252
+ Examples:
253
+ 31 | mat | a bright red doormat
254
+ 41 | rat | a giant brown rat
255
+ 59 | lip | enormous red lips
256
+
257
+ Required chunks:
258
+ {code_list}
259
+
260
+ Rules:
261
+ - Return exactly one row for every required chunk, in the same order.
262
+ - Format every row exactly as: CHUNK | WORD | SIMPLE VISUAL IMAGE CUE
263
+ - The word's consonant sounds must encode exactly the chunk, in order.
264
+ - Do not add extra coded consonant sounds.
265
+ - Prefer short, familiar, easy-to-picture nouns.
266
+ - The image cue is one concrete phrase, not a definition.
267
+ - Do not write an introduction, explanation, story, or conclusion.
268
+ """,
269
+ },
270
+ ],
271
+ max_new_tokens=min(520, 80 + len(chunks) * 42),
272
+ temperature=0.65,
273
+ )
274
+
275
+
276
+ # A style example with unrelated images. Small models tend to copy examples,
277
+ # so callers reject any line identical to one of these.
278
+ STORY_EXAMPLE_LINES = (
279
+ "You walk into your kitchen and find a cake the size of a car. It smells of hot sugar.",
280
+ "A red kite swoops down, slices the cake in two, and flies off dripping icing.",
281
+ "The kite tangles in a giant fan, which roars and spins it like a top.",
282
+ )
283
+
284
+
285
+ def ask_story_lines(rows: Sequence[tuple[str, str]], kind: str = "number") -> str:
286
+ """Ask for one linked story line per image, in order (the Link method).
287
+
288
+ ``rows`` are ordered (key word or item, image cue) pairs. The caller parses
289
+ numbered lines and accepts a line only if it names its own key word.
290
+ """
291
+ noun = "word" if kind == "number" else "item"
292
+ image_list = "\n".join(
293
+ f"{index}. {label} - {cue or 'make it huge and vivid'}"
294
+ for index, (label, cue) in enumerate(rows, 1)
295
+ )
296
+ count = len(rows)
297
+ return _generate(
298
+ [
299
+ {
300
+ "role": "system",
301
+ "content": (
302
+ "You are a memory coach using the Link method: each image "
303
+ "does something vivid to the previous one. Write in the "
304
+ "second person, present tense. Never omit, rename, or reorder "
305
+ "an image, and never state facts or definitions."
306
+ ),
307
+ },
308
+ {
309
+ "role": "user",
310
+ "content": f"""Ordered images ({noun} - picture):
311
+ {image_list}
312
+
313
+ Write a linked memory story with exactly {count} numbered lines, one per image,
314
+ in the same order.
315
+ - Line 1 sets the scene and shows image 1.
316
+ - Every later line starts from the previous image and makes the new image DO
317
+ something to it (crash into, swallow, burst out of, squirt, chase...).
318
+ - Add one sound, smell, or feeling to each line. Exaggerate size, colour, and
319
+ movement; be absurd.
320
+ - Use the exact {noun} in its own line. One or two short sentences per line.
321
+ - No title, introduction, explanation, or conclusion.
322
+
323
+ Example of the style only (different images; do not copy it):
324
+ {chr(10).join(f"{index}. {line}" for index, line in enumerate(STORY_EXAMPLE_LINES, 1))}
325
+ """,
326
+ },
327
+ ],
328
+ max_new_tokens=min(900, 60 + count * 48),
329
+ temperature=0.7,
330
+ )
331
+
332
+
333
+ def ask_list_cues(items: Sequence[str]) -> str:
334
+ """Ask for one image cue per ordered list item."""
335
+ item_list = "\n".join(
336
+ f"{index}. {item}" for index, item in enumerate(items, 1)
337
+ )
338
+ return _generate(
339
+ [
340
+ {
341
+ "role": "system",
342
+ "content": (
343
+ "You are a practical memory coach. Suggest concrete, vivid, "
344
+ "unusual imagery. Do not invent factual definitions."
345
+ ),
346
+ },
347
+ {
348
+ "role": "user",
349
+ "content": f"""Ordered list:
350
+ {item_list}
351
+
352
+ For each item, invent one concrete visual image cue that makes the item easy to
353
+ picture. Keep each cue to one short phrase. If an item is a concept rather than
354
+ an object, suggest a visual symbol or a brief pictured situation and treat it as
355
+ a memory cue, not a definition. Do not explain or define any item.
356
+
357
+ Return exactly one row per index, in order, using this format:
358
+ INDEX | IMAGE CUE
359
+
360
+ Do not repeat the item text. Do not write an introduction or conclusion.
361
+ """,
362
+ },
363
+ ],
364
+ max_new_tokens=min(900, 100 + len(items) * 42),
365
+ temperature=0.75,
366
+ )
367
+
368
+
369
+ def ask_word_fields(term: str, context: str) -> str:
370
+ """Ask for a structured draft while guarding against invented definitions."""
371
+ supplied = context.strip()
372
+ context_note = (
373
+ f"""The learner supplied this context/definition:
374
+ {supplied}
375
+
376
+ Use it as the factual anchor. Do not add facts that are not present in it.
377
+ """
378
+ if supplied
379
+ else """No context or definition was supplied.
380
+
381
+ Do not invent or assert a definition. For MEANING, write exactly:
382
+ No definition supplied
383
+ Treat the rest as a mnemonic for the label only.
384
+ """
385
+ )
386
+ return _generate(
387
+ [
388
+ {
389
+ "role": "system",
390
+ "content": (
391
+ "You are a memory coach. Help the learner remember a word or "
392
+ "concept with sound association, vivid imagery, movement, and a "
393
+ "familiar location. Never present an invented definition as fact. "
394
+ "Follow the requested labels exactly."
395
+ ),
396
+ },
397
+ {
398
+ "role": "user",
399
+ "content": f"""Term or concept:
400
+ {term.strip()}
401
+
402
+ {context_note}
403
+ Return exactly these six sections, using these labels:
404
+ MEANING: one factual anchor sentence, or the exact no-definition sentence above
405
+ SOUND: one sound or keyword association
406
+ IMAGE: one concrete, exaggerated visual image
407
+ STORY: a 2-4 sentence strange, moving mini-story
408
+ PALACE: one familiar place and where to put the image
409
+ RECALL: one active-recall question that does not reveal the term
410
+
411
+ Do not add other headings, an introduction, or a conclusion. Keep each section
412
+ short. Do not claim that an imagery cue is a fact.
413
+ """,
414
+ },
415
+ ],
416
+ max_new_tokens=380,
417
+ temperature=0.7,
418
+ )
419
+
420
+
421
+ def _strip_md(text: str) -> str:
422
+ return re.sub(r"[*_`]+", "", text or "").strip()
423
+
424
+
425
+ def _pipe_cells(line: str) -> list[str]:
426
+ """Split a Markdown-style pipe row without keeping edge empties."""
427
+ parts = line.split("|")
428
+ if parts and not parts[0].strip():
429
+ parts = parts[1:]
430
+ if parts and not parts[-1].strip():
431
+ parts = parts[:-1]
432
+ return [_strip_md(part) for part in parts]
433
+
434
+
435
+ def _clean_code(raw: str, requested: Sequence[str] | None) -> str | None:
436
+ """Normalize a model-written chunk, including lost leading zeros."""
437
+ groups = re.findall(r"\d+", raw or "")
438
+ if not groups:
439
+ return None
440
+ code = groups[-1]
441
+ if not requested:
442
+ return code
443
+ requested_list = list(requested)
444
+ if code in requested_list:
445
+ return code
446
+ padded = code.zfill(2)
447
+ if padded in requested_list:
448
+ return padded
449
+ if code.startswith("0") and code[1:] in requested_list:
450
+ return code[1:]
451
+ matches = [
452
+ item for item in requested_list if item.lstrip("0") == code.lstrip("0")
453
+ ]
454
+ if len(matches) == 1:
455
+ return matches[0]
456
+ return None
457
+
458
+
459
+ def parse_number_rows(
460
+ response: str, requested_chunks: Sequence[str] | None = None
461
+ ) -> dict[str, tuple[str, str]]:
462
+ """Parse pipe rows and the grouped field format small models often emit."""
463
+ parsed: dict[str, tuple[str, str]] = {}
464
+ current_code: str | None = None
465
+ current_word = ""
466
+ current_image = ""
467
+
468
+ def resolve_code(raw_code: str, word: str) -> str | None:
469
+ """Trust the model's label, or recover the code from the word itself."""
470
+ code = _clean_code(raw_code, requested_chunks)
471
+ if code:
472
+ return code
473
+ detected = major_code(word)
474
+ if detected and (not requested_chunks or detected in requested_chunks):
475
+ return detected
476
+ return None
477
+
478
+ def commit() -> None:
479
+ nonlocal current_word, current_image
480
+ code = current_code
481
+ if not code and current_word:
482
+ code = resolve_code("", current_word)
483
+ if code and current_word and re.search(r"[A-Za-z]", current_word):
484
+ parsed.setdefault(
485
+ code,
486
+ (current_word, current_image or "make this image vivid and exaggerated"),
487
+ )
488
+ current_word = ""
489
+ current_image = ""
490
+
491
+ for raw_line in (response or "").splitlines():
492
+ line = raw_line.strip()
493
+ if not line or line.startswith("```"):
494
+ continue
495
+ line = re.sub(r"^\s*[-*\u2022]\s*", "", line)
496
+
497
+ # Preferred compact row: 31 | mat | a bright red doormat
498
+ if "|" in line:
499
+ cells = _pipe_cells(line)
500
+ if len(cells) >= 2:
501
+ if (
502
+ cells[0].casefold()
503
+ in {"chunk", "chunks", "number", "code", "digits", "#"}
504
+ or cells[1].casefold() in {"word", "candidate", "noun"}
505
+ ):
506
+ continue
507
+ word = cells[1]
508
+ code = resolve_code(cells[0], word)
509
+ image = " | ".join(part for part in cells[2:] if part).strip()
510
+ if code and re.search(r"[A-Za-z]", word):
511
+ commit()
512
+ parsed.setdefault(
513
+ code,
514
+ (word, image or "make this image vivid and exaggerated"),
515
+ )
516
+ continue
517
+
518
+ # Explanatory field lines: Chunk: 31 / Word: mat / Image: ...
519
+ field_match = re.match(
520
+ r"^(chunk|word|image(?:\s*cue)?)\s*[:\-\u2013]\s*(.+)$",
521
+ line,
522
+ re.IGNORECASE,
523
+ )
524
+ if field_match:
525
+ key = field_match.group(1).casefold()
526
+ value = _strip_md(field_match.group(2))
527
+ if key == "chunk":
528
+ code = _clean_code(value, requested_chunks)
529
+ if code:
530
+ commit()
531
+ current_code = code
532
+ elif key == "word" and current_code:
533
+ current_word = value
534
+ if current_image:
535
+ commit()
536
+ elif key.startswith("image") and current_code:
537
+ current_image = value
538
+ commit()
539
+ continue
540
+
541
+ # Number-led line: 1. 31: Mat (M) or 31: mat - a doormat
542
+ code_line = re.match(
543
+ r"^(?:\d+[.)]\s+)?(\d{1,2})\s*[.):\-\u2013]\s*(.+)$", line
544
+ )
545
+ if not code_line:
546
+ code_line = re.match(
547
+ r"^(?:\d+[.)]\s+)?(\d{1,2})\s+([A-Za-z].*)$", line
548
+ )
549
+ if code_line:
550
+ remainder = _strip_md(code_line.group(2))
551
+ split = re.match(
552
+ r"^([A-Za-z][A-Za-z' -]*?)\s*[:\-\u2013]\s*(.+)$", remainder
553
+ )
554
+ provisional_word = split.group(1).strip() if split else remainder
555
+ provisional_word = re.sub(
556
+ r"\s*\([^)]*\)\s*$", "", provisional_word
557
+ ).strip()
558
+ code = resolve_code(code_line.group(1), provisional_word)
559
+ if not code:
560
+ continue
561
+ commit()
562
+ current_code = code
563
+ if split:
564
+ current_word = split.group(1).strip()
565
+ current_image = split.group(2).strip()
566
+ commit()
567
+ else:
568
+ # Drop trailing pronunciation notes such as "Mat (M)".
569
+ current_word = provisional_word
570
+ if not re.search(r"[A-Za-z]", current_word):
571
+ current_word = ""
572
+
573
+ commit()
574
+ return parsed
575
+
576
+
577
+ _CUE_LABEL = re.compile(
578
+ r"^(?:image\s*cue|visual\s*cue|cue|image)\s*[:\-\u2013]\s*", re.IGNORECASE
579
+ )
580
+
581
+
582
+ def _clean_cue(value: str) -> str:
583
+ """Remove a redundant 'Image Cue:' prefix and Markdown emphasis."""
584
+ return re.sub(_CUE_LABEL, "", _strip_md(value)).strip()
585
+
586
+
587
+ def parse_indexed_cues(response: str, count: int) -> dict[int, str]:
588
+ """Parse pipe rows, numbered rows, and Index/Image Cue field blocks."""
589
+ generic: dict[int, str] = {}
590
+ explicit: dict[int, str] = {}
591
+ current_index: int | None = None
592
+
593
+ for raw_line in (response or "").splitlines():
594
+ line = raw_line.strip()
595
+ if not line or line.startswith("```"):
596
+ continue
597
+ line = re.sub(r"^\s*[-*\u2022]\s*", "", line)
598
+
599
+ # Field-style heading: Index: 1. Milk / Item 2: Eggs
600
+ index_heading = re.match(
601
+ r"^(?:index|item)\s*[:\-\u2013]?\s*(\d{1,2})\b", line, re.IGNORECASE
602
+ )
603
+ if index_heading:
604
+ current_index = int(index_heading.group(1))
605
+ continue
606
+
607
+ # Field-style cue on the next line: Image Cue: a giant milk carton
608
+ cue_field = re.match(
609
+ r"^(?:image\s*cue|visual\s*cue|cue|image)\s*[:\-\u2013]\s*(.+)$",
610
+ line,
611
+ re.IGNORECASE,
612
+ )
613
+ if cue_field and current_index is not None:
614
+ cue = _clean_cue(cue_field.group(1))
615
+ if cue and 1 <= current_index <= count:
616
+ explicit.setdefault(current_index - 1, cue)
617
+ continue
618
+
619
+ index: int | None = None
620
+ cue = ""
621
+ if "|" in line:
622
+ cells = _pipe_cells(line)
623
+ index_match = re.search(r"\d+", cells[0]) if cells else None
624
+ if not index_match:
625
+ continue
626
+ index = int(index_match.group())
627
+ if len(cells) >= 3:
628
+ cue = cells[-1]
629
+ elif len(cells) == 2:
630
+ cue = cells[1]
631
+ else:
632
+ match = re.match(r"^(\d{1,2})\s*[.)]\s*(.+)$", line)
633
+ if not match:
634
+ match = re.match(r"^(\d{1,2})\s*[:\-\u2013]\s*(.+)$", line)
635
+ if match:
636
+ index = int(match.group(1))
637
+ cue = _strip_md(match.group(2))
638
+
639
+ if index is None or not 1 <= index <= count:
640
+ continue
641
+ current_index = index
642
+ cue = _clean_cue(cue)
643
+ if cue:
644
+ generic.setdefault(index - 1, cue)
645
+
646
+ # An explicit Image Cue line is more trustworthy than a merely numbered row.
647
+ for index, cue in explicit.items():
648
+ generic[index] = cue
649
+ return generic
650
+
651
+
652
+ _WORD_ALIASES = {
653
+ "meaning": "Meaning anchor",
654
+ "meaning anchor": "Meaning anchor",
655
+ "simple meaning": "Meaning anchor",
656
+ "factual anchor": "Meaning anchor",
657
+ "meaning / factual anchor": "Meaning anchor",
658
+ "simple meaning / factual anchor": "Meaning anchor",
659
+ "sound": "Sound / keyword association",
660
+ "keyword": "Sound / keyword association",
661
+ "keyword association": "Sound / keyword association",
662
+ "sound association": "Sound / keyword association",
663
+ "sound or keyword association": "Sound / keyword association",
664
+ "sound / keyword association": "Sound / keyword association",
665
+ "image": "Visual image",
666
+ "image cue": "Visual image",
667
+ "visual image": "Visual image",
668
+ "visual image cue": "Visual image",
669
+ "visual": "Visual image",
670
+ "story": "Mini-story",
671
+ "mini story": "Mini-story",
672
+ "mini-story": "Mini-story",
673
+ "palace": "Memory Palace placement",
674
+ "memory palace": "Memory Palace placement",
675
+ "memory palace location": "Memory Palace placement",
676
+ "memory palace placement": "Memory Palace placement",
677
+ "location": "Memory Palace placement",
678
+ "recall": "Active-recall question",
679
+ "question": "Active-recall question",
680
+ "active recall": "Active-recall question",
681
+ "active-recall": "Active-recall question",
682
+ "active recall question": "Active-recall question",
683
+ "active-recall question": "Active-recall question",
684
+ "recall question": "Active-recall question",
685
+ }
686
+
687
+
688
+ def _normalize_label(text: str) -> str:
689
+ text = _strip_md(text)
690
+ text = re.sub(r"^\s*(?:\d+[.)]\s*|[-*\u2022]\s*)", "", text)
691
+ text = text.strip(" :.\u2013\u2014-")
692
+ text = text.replace("&", "and")
693
+ return re.sub(r"\s+", " ", text).strip().lower()
694
+
695
+
696
+ def _match_word_label(line: str) -> tuple[str | None, str]:
697
+ """Recognize both inline labels and heading-only field labels."""
698
+ candidate = re.sub(r"^\s*[-*\u2022]\s*", "", _strip_md(line)).strip()
699
+ inline = re.match(
700
+ r"^([A-Za-z][A-Za-z0-9 /'&()\-]{0,60}?)\s*[:\-\u2013\u2014]\s*(.*)$",
701
+ candidate,
702
+ )
703
+ if inline:
704
+ canonical = _WORD_ALIASES.get(_normalize_label(inline.group(1)))
705
+ if canonical:
706
+ return canonical, inline.group(2).strip()
707
+
708
+ heading = candidate.strip(" :.\u2013\u2014-")
709
+ canonical = _WORD_ALIASES.get(_normalize_label(heading))
710
+ if canonical:
711
+ return canonical, ""
712
+ return None, ""
713
+
714
+
715
+ def _split_inline_word_sections(line: str) -> list[str]:
716
+ """Split one-line pipe-separated drafts only when labels are recognizable."""
717
+ segments = [part.strip() for part in re.split(r"\s*\|\s*", line) if part.strip()]
718
+ if len(segments) < 2:
719
+ return [line]
720
+ recognized = sum(1 for segment in segments if _match_word_label(segment)[0])
721
+ return segments if recognized >= 2 else [line]
722
+
723
+
724
+ def parse_word_sections(response: str) -> tuple[dict[str, str], str]:
725
+ """Parse labeled word-card sections, preserving unrecognized prose."""
726
+ sections: dict[str, list[str]] = {}
727
+ preamble: list[str] = []
728
+ current: str | None = None
729
+
730
+ for raw_line in (response or "").splitlines():
731
+ line = raw_line.strip()
732
+ if not line or line.startswith("```"):
733
+ continue
734
+ for part in _split_inline_word_sections(line):
735
+ label, rest = _match_word_label(part)
736
+ if label:
737
+ current = label
738
+ sections.setdefault(label, [])
739
+ if rest:
740
+ sections[label].append(rest)
741
+ continue
742
+ if current:
743
+ sections[current].append(_strip_md(part))
744
+ else:
745
+ preamble.append(_strip_md(part))
746
+
747
+ cleaned = {
748
+ label: re.sub(r"\s+", " ", " ".join(parts)).strip()
749
+ for label, parts in sections.items()
750
+ }
751
+ return cleaned, " ".join(preamble).strip()
752
+
753
+
754
+ def text_mentions_in_order(
755
+ labels: Sequence[str], text: str
756
+ ) -> tuple[bool, str | None]:
757
+ """Check that every label appears at least once, in the supplied order.
758
+
759
+ This is an omission/order guard, not a claim that the story contains no extra
760
+ words. It intentionally accepts exact labels only so a synonym cannot silently
761
+ replace a list item.
762
+ """
763
+ haystack = (text or "").casefold()
764
+ position = 0
765
+ for label in labels:
766
+ needle = label.casefold().strip()
767
+ if not needle:
768
+ return False, label
769
+ pattern = re.compile(
770
+ r"(?<![a-z0-9])" + re.escape(needle) + r"(?![a-z0-9])"
771
+ )
772
+ match = pattern.search(haystack, position)
773
+ if not match:
774
+ return False, label
775
+ position = match.end()
776
+ return True, None
777
+
778
+
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
memory_studio.py CHANGED
The diff for this file is too large to render. See raw diff
 
mind_map.py ADDED
@@ -0,0 +1,255 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """Deterministic Mind Map drawing for Memory Studio.
2
+
3
+ Turns a short indented outline into an SVG in Buzan's style: a central topic,
4
+ thick curved main branches that taper outward, one colour per branch, and one
5
+ key word written on each line. No model or UI dependency; nothing is saved.
6
+ """
7
+
8
+ from __future__ import annotations
9
+
10
+ import html
11
+ import math
12
+ import re
13
+
14
+ MAX_MAIN_BRANCHES = 8
15
+ MAX_SUB_BRANCHES = 6
16
+ MAX_LABEL_LENGTH = 40
17
+
18
+ # Colours chosen to stay readable on the map's cream background.
19
+ BRANCH_COLORS = (
20
+ "#d62839",
21
+ "#e07a1f",
22
+ "#1f8a70",
23
+ "#2f6fb0",
24
+ "#7b3fb5",
25
+ "#b8860b",
26
+ "#c2185b",
27
+ "#3d8b37",
28
+ )
29
+ BACKGROUND = "#fffdf7"
30
+ CENTER_FILL = "#312e81"
31
+
32
+ _MAIN_LENGTH = 190.0
33
+ _SUB_LENGTH = 125.0
34
+ _MAIN_FONT = 19
35
+ _SUB_FONT = 15
36
+
37
+
38
+ def _clean_label(line: str) -> str:
39
+ text = re.sub(r"^\s*(?:\d+[.)]\s+|[-*•+]\s+)", "", line)
40
+ return " ".join(text.split())
41
+
42
+
43
+ def parse_outline(text: str) -> list[tuple[str, list[str]]]:
44
+ """Parse an outline: unindented lines are main branches, indented lines
45
+ are sub-branches of the main branch above them. Raises ValueError with an
46
+ actionable message instead of dropping anything."""
47
+ branches: list[tuple[str, list[str]]] = []
48
+ for number, raw in enumerate((text or "").splitlines(), 1):
49
+ if not raw.strip():
50
+ continue
51
+ expanded = raw.replace("\t", " ")
52
+ indent = len(expanded) - len(expanded.lstrip(" "))
53
+ label = _clean_label(expanded)
54
+ if not label:
55
+ continue
56
+ if len(label) > MAX_LABEL_LENGTH:
57
+ raise ValueError(
58
+ f"Line {number} is {len(label)} characters. Mind Map branches "
59
+ f"hold a key word or two; keep each under {MAX_LABEL_LENGTH}."
60
+ )
61
+ if indent == 0:
62
+ branches.append((label, []))
63
+ elif not branches:
64
+ raise ValueError(
65
+ "The first line must be a main branch with no indentation."
66
+ )
67
+ else:
68
+ branches[-1][1].append(label)
69
+
70
+ if not branches:
71
+ raise ValueError(
72
+ "Add at least one main branch: one key word per line, with "
73
+ "sub-branches indented underneath."
74
+ )
75
+ if len(branches) > MAX_MAIN_BRANCHES:
76
+ raise ValueError(
77
+ f"Use up to {MAX_MAIN_BRANCHES} main branches; this outline has "
78
+ f"{len(branches)}. Group related ideas, or split it into two maps."
79
+ )
80
+ crowded = [label for label, subs in branches if len(subs) > MAX_SUB_BRANCHES]
81
+ if crowded:
82
+ raise ValueError(
83
+ f"Keep up to {MAX_SUB_BRANCHES} sub-branches per main branch. Too many "
84
+ "under: " + ", ".join(crowded) + ". Add another main branch instead."
85
+ )
86
+ return branches
87
+
88
+
89
+ def key_word_tips(topic: str, branches: list[tuple[str, list[str]]]) -> list[str]:
90
+ """Gentle reminders of Buzan's key-word rule. Never blocks drawing."""
91
+ wordy = [
92
+ label
93
+ for label in [main for main, _subs in branches]
94
+ + [sub for _main, subs in branches for sub in subs]
95
+ if len(label.split()) > 2
96
+ ]
97
+ tips = []
98
+ if wordy:
99
+ tips.append(
100
+ "One key word per line recalls best. Consider shortening: "
101
+ + ", ".join(f"**{label}**" for label in wordy[:6])
102
+ + ("…" if len(wordy) > 6 else "")
103
+ + "."
104
+ )
105
+ if len(topic.split()) > 4:
106
+ tips.append("Make the centre a short topic you can turn into one picture.")
107
+ if all(not subs for _main, subs in branches):
108
+ tips.append(
109
+ "Add a few indented sub-branches under each main idea; the details "
110
+ "hang off the key words."
111
+ )
112
+ return tips
113
+
114
+
115
+ def _text_width(text: str, font_size: int) -> float:
116
+ return max(1, len(text)) * font_size * 0.58
117
+
118
+
119
+ def _tapered_curve(
120
+ start: tuple[float, float],
121
+ end: tuple[float, float],
122
+ bend: float,
123
+ width_start: float,
124
+ width_end: float,
125
+ steps: int = 24,
126
+ ) -> str:
127
+ """A filled, gently curved branch that is thicker at its start."""
128
+ (x0, y0), (x2, y2) = start, end
129
+ mx, my = (x0 + x2) / 2, (y0 + y2) / 2
130
+ dx, dy = x2 - x0, y2 - y0
131
+ length = math.hypot(dx, dy) or 1.0
132
+ nx, ny = -dy / length, dx / length
133
+ x1, y1 = mx + nx * bend, my + ny * bend
134
+
135
+ left, right = [], []
136
+ for step in range(steps + 1):
137
+ t = step / steps
138
+ x = (1 - t) ** 2 * x0 + 2 * (1 - t) * t * x1 + t**2 * x2
139
+ y = (1 - t) ** 2 * y0 + 2 * (1 - t) * t * y1 + t**2 * y2
140
+ tx = 2 * (1 - t) * (x1 - x0) + 2 * t * (x2 - x1)
141
+ ty = 2 * (1 - t) * (y1 - y0) + 2 * t * (y2 - y1)
142
+ tl = math.hypot(tx, ty) or 1.0
143
+ px, py = -ty / tl, tx / tl
144
+ half = (width_start + (width_end - width_start) * t) / 2
145
+ left.append((x + px * half, y + py * half))
146
+ right.append((x - px * half, y - py * half))
147
+ points = left + right[::-1]
148
+ path = "M " + " L ".join(f"{x:.1f} {y:.1f}" for x, y in points) + " Z"
149
+ return path
150
+
151
+
152
+ def render_svg(topic: str, branches: list[tuple[str, list[str]]]) -> str:
153
+ """Draw the map. The viewBox is fitted to the drawing, so it scales cleanly."""
154
+ elements: list[str] = []
155
+ bounds = [0.0, 0.0, 0.0, 0.0] # min_x, min_y, max_x, max_y
156
+
157
+ def grow(x: float, y: float) -> None:
158
+ bounds[0], bounds[1] = min(bounds[0], x), min(bounds[1], y)
159
+ bounds[2], bounds[3] = max(bounds[2], x), max(bounds[3], y)
160
+
161
+ def word_on_line(
162
+ x: float, y: float, text: str, size: int, color: str, rightward: bool, bold: bool
163
+ ) -> None:
164
+ width = _text_width(text, size)
165
+ x_end = x + width + 8 if rightward else x - width - 8
166
+ elements.append(
167
+ f'<path d="M {x:.1f} {y:.1f} L {x_end:.1f} {y:.1f}" stroke="{color}" '
168
+ f'stroke-width="{3 if bold else 2}" stroke-linecap="round" fill="none"/>'
169
+ )
170
+ anchor = "start" if rightward else "end"
171
+ text_x = x + 4 if rightward else x - 4
172
+ weight = "700" if bold else "500"
173
+ elements.append(
174
+ f'<text x="{text_x:.1f}" y="{y - 6:.1f}" font-size="{size}" '
175
+ f'font-weight="{weight}" fill="{color}" text-anchor="{anchor}">'
176
+ f"{html.escape(text)}</text>"
177
+ )
178
+ grow(x_end, y - size - 8)
179
+ grow(x_end, y + 4)
180
+
181
+ count = len(branches)
182
+ center_rx = max(72.0, _text_width(topic, 22) / 2 + 26)
183
+ center_ry = 42.0
184
+ for index, (label, subs) in enumerate(branches):
185
+ color = BRANCH_COLORS[index % len(BRANCH_COLORS)]
186
+ angle = math.radians((index + 0.5) * 360 / count - 90)
187
+ if count == 1:
188
+ angle = 0.0
189
+ cos, sin = math.cos(angle), math.sin(angle)
190
+ start = (cos * center_rx * 0.8, sin * center_ry * 0.8)
191
+ end = (start[0] + cos * _MAIN_LENGTH, start[1] + sin * _MAIN_LENGTH)
192
+ bend = 26 if index % 2 else -26
193
+ elements.append(
194
+ f'<path d="{_tapered_curve(start, end, bend, 16, 4)}" fill="{color}"/>'
195
+ )
196
+ rightward = cos >= -1e-6
197
+ word_on_line(end[0], end[1], label, _MAIN_FONT, color, rightward, True)
198
+ main_width = _text_width(label, _MAIN_FONT) + 8
199
+ hub = (end[0] + (main_width if rightward else -main_width), end[1])
200
+
201
+ spread = min(70.0, 22.0 * max(1, len(subs) - 1))
202
+ base = 0.0 if rightward else math.pi
203
+ for sub_index, sub in enumerate(subs):
204
+ if len(subs) == 1:
205
+ offset = 0.0
206
+ else:
207
+ offset = -spread / 2 + spread * sub_index / (len(subs) - 1)
208
+ sub_angle = base + math.radians(offset if rightward else -offset)
209
+ sub_end = (
210
+ hub[0] + math.cos(sub_angle) * _SUB_LENGTH,
211
+ hub[1] + math.sin(sub_angle) * _SUB_LENGTH,
212
+ )
213
+ elements.append(
214
+ f'<path d="{_tapered_curve(hub, sub_end, 10, 5, 2)}" '
215
+ f'fill="{color}" opacity="0.85"/>'
216
+ )
217
+ word_on_line(
218
+ sub_end[0], sub_end[1], sub, _SUB_FONT, color, rightward, False
219
+ )
220
+ grow(*end)
221
+
222
+ elements.append(
223
+ f'<ellipse cx="0" cy="0" rx="{center_rx:.1f}" ry="{center_ry:.1f}" '
224
+ f'fill="{CENTER_FILL}"/>'
225
+ )
226
+ elements.append(
227
+ f'<text x="0" y="7" font-size="22" font-weight="700" fill="#ffffff" '
228
+ f'text-anchor="middle">{html.escape(topic)}</text>'
229
+ )
230
+ grow(-center_rx, -center_ry)
231
+ grow(center_rx, center_ry)
232
+
233
+ pad = 28
234
+ min_x, min_y = bounds[0] - pad, bounds[1] - pad
235
+ width, height = bounds[2] - bounds[0] + 2 * pad, bounds[3] - bounds[1] + 2 * pad
236
+ title = html.escape(f"Mind Map: {topic}")
237
+ return (
238
+ f'<svg xmlns="http://www.w3.org/2000/svg" role="img" '
239
+ f'aria-label="{title}" viewBox="{min_x:.0f} {min_y:.0f} {width:.0f} '
240
+ f'{height:.0f}" style="width:100%;height:auto;max-height:640px" '
241
+ f'font-family="system-ui, -apple-system, Segoe UI, Roboto, sans-serif">'
242
+ f"<title>{title}</title>"
243
+ f'<rect x="{min_x:.0f}" y="{min_y:.0f}" width="{width:.0f}" '
244
+ f'height="{height:.0f}" rx="18" fill="{BACKGROUND}"/>'
245
+ + "".join(elements)
246
+ + "</svg>"
247
+ )
248
+
249
+
250
+ def outline_markdown(topic: str, branches: list[tuple[str, list[str]]]) -> str:
251
+ lines = [f"## Mind Map: {topic}", ""]
252
+ for label, subs in branches:
253
+ lines.append(f"- **{label}**")
254
+ lines.extend(f" - {sub}" for sub in subs)
255
+ return "\n".join(lines)
mnemonic_core.py CHANGED
@@ -1,892 +1,1242 @@
1
- """Deterministic mnemonic tools used by Memory Studio.
2
-
3
- This module intentionally has no model or UI dependencies. It provides the
4
- Major-System checker, curated starter words, input validation, editable-table
5
- helpers, and deterministic story/palace/recall scaffolding. The optional local
6
- language model lives in :mod:`local_llm`.
7
- """
8
-
9
- from __future__ import annotations
10
-
11
- import math
12
- import re
13
- from dataclasses import dataclass
14
- from typing import Any, Iterable, Sequence
15
-
16
- # The Major System is phonetic. These descriptions are for explanation; the
17
- # checker below is a conservative spelling-based approximation, not a
18
- # pronunciation dictionary.
19
- MAJOR_SOUNDS = {
20
- "0": "S, Z, soft C",
21
- "1": "T, D, TH",
22
- "2": "N",
23
- "3": "M",
24
- "4": "R",
25
- "5": "L",
26
- "6": "J, SH, CH, soft G",
27
- "7": "K, hard C, hard G, Q, CK, NG",
28
- "8": "F, V, PH",
29
- "9": "B, P",
30
- }
31
-
32
- MAJOR_HELP = """\
33
- | Digit | Consonant sounds |
34
- |---:|---|
35
- | 0 | S, Z, soft C |
36
- | 1 | T, D, TH |
37
- | 2 | N |
38
- | 3 | M |
39
- | 4 | R |
40
- | 5 | L |
41
- | 6 | J, SH, CH, soft G |
42
- | 7 | K, hard C, hard G, Q, CK, NG |
43
- | 8 | F, V, PH |
44
- | 9 | B, P |
45
-
46
- Vowels and H, W, Y are fillers and do not encode digits. The Major System is
47
- based on sounds, not spelling, so the automatic checker here is approximate.
48
- """
49
-
50
- DIGIT_PATTERN = re.compile(r"\d")
51
-
52
- MAX_NUMBER_DIGITS = 40
53
- MAX_LIST_ITEMS = 30
54
- MAX_ITEM_LENGTH = 300
55
- MAX_WORD_TERM_LENGTH = 160
56
- MAX_WORD_CONTEXT_LENGTH = 1200
57
-
58
- # Common pronunciation patterns handled before single letters. They make the
59
- # spelling-based checker less naive, but they do not make it a phonetic
60
- # dictionary.
61
- _INITIAL_SILENT = (
62
- ("kn", "2"),
63
- ("wr", "4"),
64
- ("gn", "2"),
65
- ("ps", "0"),
66
- ("pn", "2"),
67
- ("mn", "2"),
68
- )
69
- _DIGRAPHS = (
70
- ("tch", "6"),
71
- ("sh", "6"),
72
- ("ch", "6"),
73
- ("th", "1"),
74
- ("ph", "8"),
75
- ("ng", "7"),
76
- ("ck", "7"),
77
- ("qu", "7"),
78
- )
79
- _VOWELS = set("aeiou")
80
-
81
-
82
- def major_code(word: str) -> str:
83
- """Approximate a word's Major-System digit code.
84
-
85
- Vowels and H/W/Y are ignored as digit carriers. Common digraphs and a few
86
- high-frequency silent patterns are handled. English pronunciation is
87
- irregular, so this is a useful automatic check, not a perfect phonetic
88
- dictionary.
89
- """
90
- text = re.sub(r"[^a-z]", "", word.lower())
91
- result: list[str] = []
92
- last_digit: str | None = None
93
- i = 0
94
-
95
- def add_digit(digit: str) -> None:
96
- nonlocal last_digit
97
- # Traditional Major-System practice generally treats adjacent
98
- # identical consonant sounds as one sound.
99
- if digit != last_digit:
100
- result.append(digit)
101
- last_digit = digit
102
-
103
- while i < len(text):
104
- if i == 0:
105
- matched_initial = False
106
- for sequence, digit in _INITIAL_SILENT:
107
- if text.startswith(sequence, i):
108
- add_digit(digit)
109
- i += len(sequence)
110
- matched_initial = True
111
- break
112
- if matched_initial:
113
- continue
114
-
115
- # Common suffixes whose spelling differs from their sounds.
116
- if text.startswith("ssion", i):
117
- add_digit("6")
118
- add_digit("2")
119
- i += 5
120
- continue
121
- if text.startswith("tion", i) or text.startswith("sion", i):
122
- add_digit("6")
123
- add_digit("2")
124
- i += 4
125
- continue
126
- if text.startswith("cial", i):
127
- add_digit("6")
128
- i += 4
129
- continue
130
- if text.startswith("ight", i):
131
- add_digit("1")
132
- i += 4
133
- continue
134
-
135
- matched = False
136
- for sequence, digit in _DIGRAPHS:
137
- if text.startswith(sequence, i):
138
- add_digit(digit)
139
- i += len(sequence)
140
- matched = True
141
- break
142
- if matched:
143
- continue
144
-
145
- # A final "mb" is normally pronounced as M only.
146
- if text.startswith("mb", i) and i + 2 == len(text):
147
- add_digit("3")
148
- i += 2
149
- continue
150
-
151
- char = text[i]
152
- next_char = text[i + 1] if i + 1 < len(text) else ""
153
-
154
- if char in _VOWELS:
155
- # Vowels separate consonant sounds. For example, the two S sounds
156
- # in "sauce" encode 00 rather than collapsing to a single 0.
157
- last_digit = None
158
- i += 1
159
- continue
160
- if char in "hwy":
161
- i += 1
162
- continue
163
- if char in "bd":
164
- add_digit("9" if char == "b" else "1")
165
- elif char in "tf":
166
- add_digit("1" if char == "t" else "8")
167
- elif char == "n":
168
- add_digit("2")
169
- elif char == "m":
170
- add_digit("3")
171
- elif char == "r":
172
- add_digit("4")
173
- elif char == "l":
174
- add_digit("5")
175
- elif char == "j":
176
- add_digit("6")
177
- elif char in "sz":
178
- add_digit("0")
179
- elif char == "c":
180
- add_digit("0" if next_char and next_char in "eiy" else "7")
181
- elif char == "g":
182
- add_digit("6" if next_char and next_char in "eiy" else "7")
183
- elif char in "kq":
184
- add_digit("7")
185
- elif char == "v":
186
- add_digit("8")
187
- elif char == "p":
188
- add_digit("9")
189
- elif char == "x":
190
- add_digit("7")
191
- add_digit("0")
192
- # Letters not covered above are ignored conservatively.
193
- i += 1
194
-
195
- return "".join(result)
196
-
197
-
198
- def check_word_for_chunk(word: str, chunk: str) -> tuple[bool, str]:
199
- """Check a candidate word against a digit chunk using :func:`major_code`."""
200
- cleaned = re.sub(r"[^A-Za-z]", "", word or "")
201
- if not cleaned:
202
- return False, "needs a word"
203
- if len(cleaned) > 80:
204
- return False, "review: word is very long"
205
- detected = major_code(cleaned)
206
- if detected == chunk:
207
- return True, "match (approx.)"
208
- if not detected:
209
- return False, "review: checker found no consonant sounds"
210
- return False, f"review: checker read {detected}"
211
-
212
-
213
- def split_number(number_text: str) -> list[str]:
214
- """Keep digit order and leading zeros; split into two-digit chunks."""
215
- digits = "".join(DIGIT_PATTERN.findall(number_text or ""))
216
- if not digits:
217
- raise ValueError("Enter a number containing at least one digit (0-9).")
218
- if len(digits) > MAX_NUMBER_DIGITS:
219
- raise ValueError(
220
- f"Enter up to {MAX_NUMBER_DIGITS} digits at a time "
221
- f"({MAX_NUMBER_DIGITS // 2} two-digit images)."
222
- )
223
- return [digits[index : index + 2] for index in range(0, len(digits), 2)]
224
-
225
-
226
- # Curated starting words for every two-digit code. These are suggestions, not
227
- # the only valid mnemonic. Learners can replace any word and cue in the table.
228
- # Every entry is checked against major_code() by the project's static checks.
229
- CURATED_TWO_DIGIT: dict[str, tuple[str, str]] = {
230
- "00": ("sauce", "a bubbling bottle of sauce tipping over"),
231
- "01": ("suit", "an oversized business suit walking by itself"),
232
- "02": ("sun", "a huge grinning sun"),
233
- "03": ("sum", "a giant glowing plus sign"),
234
- "04": ("sore", "a throbbing red sore"),
235
- "05": ("sail", "a bright sail snapping in the wind"),
236
- "06": ("sash", "a long silk sash fluttering"),
237
- "07": ("sock", "an enormous striped sock"),
238
- "08": ("safe", "a heavy metal safe with its door open"),
239
- "09": ("soap", "a slippery bar of soap"),
240
- "10": ("toes", "ten wiggling toes in the grass"),
241
- "11": ("toad", "a lumpy toad wearing a crown"),
242
- "12": ("tin", "a shiny tin can"),
243
- "13": ("time", "a giant ticking clock"),
244
- "14": ("tire", "a rolling rubber tire"),
245
- "15": ("towel", "a wet towel snapping like a whip"),
246
- "16": ("touch", "a glowing fingertip touching a surface"),
247
- "17": ("tack", "a sharp thumbtack"),
248
- "18": ("tofu", "a wobbly cube of tofu"),
249
- "19": ("tub", "a steaming bathtub"),
250
- "20": ("nose", "a giant nose honking like a trumpet"),
251
- "21": ("net", "a fishing net full of silver fish"),
252
- "22": ("nun", "a nun ringing a tiny bell"),
253
- "23": ("name", "a name tag the size of a billboard"),
254
- "24": ("narrow", "a very narrow alley"),
255
- "25": ("nail", "a shiny nail being hammered"),
256
- "26": ("nacho", "a loaded nacho dripping with cheese"),
257
- "27": ("neck", "a giraffe's long neck"),
258
- "28": ("navy", "a navy-blue ship"),
259
- "29": ("nap", "a person curled up for a nap"),
260
- "30": ("mouse", "a mouse wearing tiny boots"),
261
- "31": ("mat", "a bright red doormat"),
262
- "32": ("moon", "a huge smiling moon"),
263
- "33": ("mummy", "a mummy wrapped in glowing bandages"),
264
- "34": ("mower", "a roaring lawn mower"),
265
- "35": ("mule", "a stubborn mule kicking"),
266
- "36": ("match", "a burning match"),
267
- "37": ("mug", "a chipped coffee mug"),
268
- "38": ("movie", "a movie screen showing fireworks"),
269
- "39": ("map", "a treasure map with a red X"),
270
- "40": ("rose", "a giant rose with sharp thorns"),
271
- "41": ("rat", "a giant brown rat"),
272
- "42": ("rain", "a rain cloud pouring on one spot"),
273
- "43": ("ram", "a battering ram with horns"),
274
- "44": ("rear", "the rear end of a bus"),
275
- "45": ("rail", "a shiny train rail"),
276
- "46": ("roach", "a giant cockroach"),
277
- "47": ("rock", "a boulder rocking back and forth"),
278
- "48": ("roof", "a red-tiled roof"),
279
- "49": ("rope", "a coiled rope"),
280
- }
281
- CURATED_TWO_DIGIT.update(
282
- {
283
- "50": ("lace", "a tangled lace ribbon"),
284
- "51": ("lady", "a lady in a huge hat"),
285
- "52": ("lion", "a roaring lion"),
286
- "53": ("lime", "a sour green lime"),
287
- "54": ("liar", "a liar with a long nose"),
288
- "55": ("lily", "a tall white lily"),
289
- "56": ("leech", "a fat black leech"),
290
- "57": ("lock", "a heavy padlock"),
291
- "58": ("leaf", "a giant green leaf"),
292
- "59": ("lip", "enormous red lips"),
293
- "60": ("chess", "a chessboard with giant pieces"),
294
- "61": ("sheet", "a white sheet billowing"),
295
- "62": ("chain", "a heavy iron chain"),
296
- "63": ("gem", "a sparkling gem"),
297
- "64": ("chair", "a chair with a broken leg"),
298
- "65": ("jail", "a jail cell with rusty bars"),
299
- "66": ("choo-choo", "a toy choo-choo train"),
300
- "67": ("chick", "a fluffy yellow chick"),
301
- "68": ("chef", "a chef flipping a pan"),
302
- "69": ("ship", "a ship in a bottle"),
303
- "70": ("kiss", "a lipstick kiss mark"),
304
- "71": ("cat", "a cat with laser eyes"),
305
- "72": ("can", "a dented tin can"),
306
- "73": ("game", "a giant board game"),
307
- "74": ("car", "a tiny car with a huge horn"),
308
- "75": ("coal", "a glowing lump of coal"),
309
- "76": ("cage", "a golden birdcage"),
310
- "77": ("cake", "a towering layer cake"),
311
- "78": ("cave", "a dark cave with glowing eyes"),
312
- "79": ("cap", "a baseball cap spinning"),
313
- "80": ("face", "a giant face pulling a funny expression"),
314
- "81": ("fat", "a sizzling slab of fat"),
315
- "82": ("fan", "a spinning electric fan"),
316
- "83": ("foam", "a mountain of soap foam"),
317
- "84": ("fire", "a crackling campfire"),
318
- "85": ("file", "a metal file cabinet"),
319
- "86": ("fish", "a fish wearing a crown"),
320
- "87": ("fog", "a thick wall of fog"),
321
- "88": ("fife", "a small silver flute called a fife"),
322
- "89": ("fob", "a key fob dangling from a chain"),
323
- "90": ("bus", "a double-decker bus"),
324
- "91": ("bat", "a baseball bat swinging"),
325
- "92": ("bone", "a giant dog bone"),
326
- "93": ("beam", "a wooden beam balancing"),
327
- "94": ("bear", "a bear hugging a honey pot"),
328
- "95": ("bell", "a ringing brass bell"),
329
- "96": ("beach", "a sunny beach with one palm tree"),
330
- "97": ("book", "a giant open book"),
331
- "98": ("beef", "a huge steak"),
332
- "99": ("pipe", "a leaking pipe"),
333
- }
334
- )
335
-
336
- CURATED_ONE_DIGIT: dict[str, tuple[str, str]] = {
337
- "0": ("sea", "a crashing blue sea"),
338
- "1": ("tea", "a steaming cup of tea"),
339
- "2": ("hen", "a clucking hen"),
340
- "3": ("ham", "a giant ham"),
341
- "4": ("hare", "a fast hare"),
342
- "5": ("eel", "a slippery eel"),
343
- "6": ("shoe", "a giant shoe"),
344
- "7": ("key", "a golden key"),
345
- "8": ("ivy", "ivy climbing a wall"),
346
- "9": ("bee", "a buzzing bee"),
347
- }
348
-
349
-
350
- def curated_candidate(chunk: str) -> tuple[str, str] | None:
351
- """Return the curated (word, image cue) for a chunk, if available."""
352
- if len(chunk) == 1:
353
- return CURATED_ONE_DIGIT.get(chunk)
354
- return CURATED_TWO_DIGIT.get(chunk)
355
-
356
-
357
- def curated_major_rows() -> list[list[str]]:
358
- """Rows for the read-only curated-word browser in the guide tab."""
359
- rows: list[list[str]] = []
360
- for chunk in [f"{number:02d}" for number in range(100)]:
361
- word, cue = CURATED_TWO_DIGIT[chunk]
362
- rows.append([chunk, word, cue])
363
- for chunk in "0123456789":
364
- word, cue = CURATED_ONE_DIGIT[chunk]
365
- rows.append([chunk, word, cue])
366
- return rows
367
-
368
-
369
- @dataclass
370
- class ChainItem:
371
- """One ordered image in a storyboard."""
372
-
373
- label: str
374
- cue: str = ""
375
- code: str = ""
376
-
377
-
378
- @dataclass
379
- class NumberRow:
380
- """A reviewed row from the editable number table."""
381
-
382
- chunk: str
383
- word: str
384
- image: str
385
- check: str
386
- source: str
387
-
388
- def as_list(self) -> list[str]:
389
- return [self.chunk, self.word, self.image, self.check, self.source]
390
-
391
-
392
- @dataclass
393
- class ListRow:
394
- """A reviewed row from the editable ordered-list table."""
395
-
396
- position: int
397
- item: str
398
- cue: str
399
-
400
- def as_list(self) -> list[str]:
401
- return [str(self.position), self.item, self.cue]
402
-
403
-
404
- def cell_text(value: Any) -> str:
405
- """Convert an editable-table cell to a clean single-line string."""
406
- if value is None:
407
- return ""
408
- if isinstance(value, float) and math.isnan(value):
409
- return ""
410
- text = str(value).replace("\r", " ").replace("\n", " ").strip()
411
- if text in {"<NA>", "nan", "NaN"}:
412
- return ""
413
- return text
414
-
415
-
416
- def table_to_rows(table: Any) -> list[list[str]]:
417
- """Normalize Gradio/pandas/array table input into a list of string rows."""
418
- if table is None:
419
- return []
420
- if isinstance(table, dict):
421
- # Gradio 6's Dataframe API passes {"headers": ..., "data": ...}.
422
- if "data" in table:
423
- table = table["data"]
424
- elif "value" in table:
425
- table = table["value"]
426
- else:
427
- table = list(table.values())
428
- if hasattr(table, "values"):
429
- values = table.values
430
- try:
431
- table = values.tolist()
432
- except AttributeError: # pragma: no cover - defensive for unusual tables
433
- table = list(values)
434
- rows: list[list[str]] = []
435
- for row in table:
436
- if isinstance(row, dict):
437
- rows.append([cell_text(value) for value in row.values()])
438
- elif isinstance(row, (list, tuple)):
439
- rows.append([cell_text(value) for value in row])
440
- else:
441
- rows.append([cell_text(row)])
442
- return rows
443
-
444
-
445
- def validate_number_rows(
446
- chunks: Sequence[str], table: Any
447
- ) -> tuple[list[NumberRow], list[str]]:
448
- """Check table shape/chunk order and refresh each word's approximate check."""
449
- raw_rows = table_to_rows(table)
450
- if not raw_rows:
451
- return [], ["Draft candidate rows first, then review them in the table."]
452
- if len(raw_rows) != len(chunks):
453
- return [], [
454
- f"The table has {len(raw_rows)} row(s) but the number has {len(chunks)} "
455
- "chunk(s). Do not add or remove rows; regenerate the draft if the "
456
- "number changed."
457
- ]
458
-
459
- reviewed: list[NumberRow] = []
460
- errors: list[str] = []
461
- for index, (expected, raw) in enumerate(zip(chunks, raw_rows), 1):
462
- raw = list(raw) + [""] * max(0, 5 - len(raw))
463
- chunk = raw[0]
464
- if chunk.isdigit():
465
- # Dataframes sometimes turn "04" into 4. Restore the expected width.
466
- chunk = chunk.zfill(len(expected))
467
- if chunk != expected:
468
- errors.append(
469
- f"Row {index}: expected chunk {expected} but found "
470
- f"{chunk or '(blank)'}. Keep the original row order."
471
- )
472
- continue
473
- word = raw[1]
474
- image = raw[2]
475
- source = raw[4] or "your table"
476
- if word:
477
- _ok, check = check_word_for_chunk(word, chunk)
478
- else:
479
- check = "needs a word"
480
- reviewed.append(NumberRow(expected, word, image, check, source))
481
-
482
- if errors:
483
- return [], errors
484
- return reviewed, []
485
-
486
-
487
- def number_blockers(rows: Iterable[NumberRow]) -> list[str]:
488
- """Return human-readable reasons the number table is not ready for a story."""
489
- problems: list[str] = []
490
- for row in rows:
491
- if not row.word:
492
- problems.append(f"`{row.chunk}` has no word.")
493
- elif not row.check.startswith("match"):
494
- problems.append(f"`{row.chunk}` = **{row.word}**: {row.check}.")
495
- return problems
496
-
497
-
498
- def _strip_list_marker(line: str) -> str:
499
- # Require whitespace (or end of line) after a numeric marker so values such
500
- # as "1.5 cups" are not mistaken for a numbered list item.
501
- return re.sub(
502
- r"^\s*(?:\d+[.)](?:\s+|$)|[-*\u2022]\s+)", "", line
503
- ).strip()
504
-
505
-
506
- def parse_item_list(text: str) -> list[str]:
507
- """Parse an ordered list while preserving the learner's item order."""
508
- raw_lines = [line.strip() for line in (text or "").splitlines() if line.strip()]
509
- if not raw_lines:
510
- raise ValueError("Enter at least two items, one per line or separated by commas.")
511
-
512
- if len(raw_lines) == 1:
513
- line = _strip_list_marker(raw_lines[0])
514
- parts = [
515
- _strip_list_marker(part.strip())
516
- for part in re.split(r"[,;]", line)
517
- if part.strip()
518
- ]
519
- items = parts if len(parts) > 1 else [line]
520
- else:
521
- items = [_strip_list_marker(line) for line in raw_lines]
522
-
523
- if any(not item for item in items):
524
- raise ValueError(
525
- "One line has no item text after its number or bullet. Fix that line "
526
- "or remove it."
527
- )
528
- if len(items) < 2:
529
- raise ValueError("Enter at least two items, one per line or separated by commas.")
530
- if len(items) > MAX_LIST_ITEMS:
531
- raise ValueError(
532
- f"For a clear story, enter up to {MAX_LIST_ITEMS} items at a time."
533
- )
534
- too_long = [item for item in items if len(item) > MAX_ITEM_LENGTH]
535
- if too_long:
536
- raise ValueError(
537
- f"Keep each item under {MAX_ITEM_LENGTH} characters. "
538
- "Shorten the longest item and try again."
539
- )
540
- return items
541
-
542
-
543
- def validate_list_rows(table: Any) -> tuple[list[ListRow], list[str]]:
544
- """Validate an edited ordered-list table without silently dropping rows."""
545
- raw_rows = table_to_rows(table)
546
- if not raw_rows:
547
- return [], ["Draft image cues first, then review them in the table."]
548
-
549
- reviewed: list[ListRow] = []
550
- errors: list[str] = []
551
- for index, raw in enumerate(raw_rows, 1):
552
- raw = list(raw) + [""] * max(0, 3 - len(raw))
553
- item = raw[1]
554
- cue = raw[2]
555
- if not item:
556
- errors.append(
557
- f"Row {index} has no item. Do not delete list items; restore the "
558
- "row or regenerate from the list input."
559
- )
560
- continue
561
- if len(item) > MAX_ITEM_LENGTH:
562
- errors.append(
563
- f"Row {index} is longer than {MAX_ITEM_LENGTH} characters. "
564
- "Shorten that item."
565
- )
566
- continue
567
- reviewed.append(ListRow(index, item, cue))
568
-
569
- if errors:
570
- return [], errors
571
- return reviewed, []
572
-
573
-
574
- _LINK_VERBS = (
575
- "crash into",
576
- "drag",
577
- "balance on",
578
- "launch",
579
- "bury",
580
- "wrap itself around",
581
- "chase",
582
- "paint",
583
- "stack",
584
- "fling",
585
- "ride",
586
- "swallow",
587
- "crack",
588
- "squeeze",
589
- "catapult",
590
- "stitch",
591
- "flood",
592
- "tickle",
593
- "polish",
594
- "hide inside",
595
- )
596
-
597
-
598
- def _with_article(label: str) -> str:
599
- lowered = label.casefold()
600
- if lowered.startswith(("the ", "a ", "an ", "my ", "your ", "our ", "their ")):
601
- return label
602
- return label if label[:1].isupper() else f"the {label}"
603
-
604
-
605
- def build_ordered_chain(
606
- items: Sequence[ChainItem],
607
- title: str = "Ordered image chain",
608
- ) -> str:
609
- """Build a complete, order-preserving storyboard without a model.
610
-
611
- Every item appears exactly once, in the order supplied. The linking verbs are
612
- deliberately exaggerated; the learner can replace any link that feels weak.
613
- """
614
- if not items:
615
- return ""
616
-
617
- lines = [
618
- f"### {title}",
619
- "",
620
- "Read this as a moving storyboard. Keep the numbered order; replace any "
621
- "awkward link with one that is more vivid for you.",
622
- "",
623
- ]
624
- previous: str | None = None
625
- for index, item in enumerate(items, 1):
626
- code = f" (`{item.code}`)" if item.code else ""
627
- cue = item.cue.strip() or "Make this image huge, bright, and moving."
628
- if cue[-1:] not in ".!?":
629
- cue += "."
630
- lines.append(f"**{index}. {item.label}**{code} \u2014 {cue}")
631
- if previous is None:
632
- lines.append(" Open big: make it oversized, colorful, and in motion.")
633
- else:
634
- verb = _LINK_VERBS[(index - 2) % len(_LINK_VERBS)]
635
- lines.append(
636
- f" Link: make **{item.label}** {verb} "
637
- f"{_with_article(previous)}."
638
- )
639
- previous = item.label
640
-
641
- if len(items) > 1:
642
- lines.append(
643
- f" Hold the final picture: **{items[-1].label}** still touching "
644
- f"{_with_article(items[-2].label)}. Count to three."
645
- )
646
- if any(item.code for item in items):
647
- order_note = (
648
- "**Decode check:** the bold image word is the code. Adjectives in its "
649
- "cue and the linking words are visual glue, not extra digits."
650
- )
651
- else:
652
- order_note = (
653
- "**Order check:** every list item appears once, in the numbered order."
654
- )
655
- lines.extend(
656
- [
657
- "",
658
- order_note,
659
- "",
660
- "_No-model storyboard. Personalize the images and links before relying "
661
- "on them._",
662
- ]
663
- )
664
- return "\n".join(lines)
665
-
666
-
667
- def parse_palace_stops(text: str) -> list[str]:
668
- """Parse an optional familiar-place route, keeping its order."""
669
- lines = [line.strip() for line in (text or "").splitlines() if line.strip()]
670
- if not lines:
671
- return []
672
- if len(lines) == 1:
673
- parts = [part.strip() for part in re.split(r"[,;]", lines[0]) if part.strip()]
674
- if len(parts) > 1:
675
- lines = parts
676
-
677
- stops: list[str] = []
678
- for line in lines:
679
- stop = re.sub(
680
- r"^\s*(?:\d+[.)](?:\s+|$)|[-*\u2022]\s+)", "", line
681
- ).strip()
682
- if stop:
683
- stops.append(stop)
684
- if len(stops) > 60:
685
- raise ValueError("Please use at most 60 palace stops in one route.")
686
- return stops
687
-
688
-
689
- def _table_cell(text: str) -> str:
690
- return text.replace("|", "\\|").replace("\n", " ").strip()
691
-
692
-
693
- def build_palace_markdown(
694
- items: Sequence[ChainItem], stops: Sequence[str]
695
- ) -> str:
696
- """Place the images along the learner's optional route, in the same order."""
697
- if not stops:
698
- return ""
699
-
700
- lines = [
701
- "### Memory Palace placement",
702
- "",
703
- "Walk the route in order, then walk it backward. Use places you can "
704
- "picture clearly, and keep the route consistent across reviews.",
705
- "",
706
- "| Order | Image | Place | What to do |",
707
- "|---:|---|---|---|",
708
- ]
709
- for index, item in enumerate(items):
710
- code = f" (`{item.code}`)" if item.code else ""
711
- label = _table_cell(f"**{item.label}**{code}")
712
- if index < len(stops):
713
- place = _table_cell(stops[index])
714
- action = "Put this image here and add one sensory detail."
715
- else:
716
- place = "\u2014 (add a stop)"
717
- action = "Unplaced: add a familiar stop to keep the order complete."
718
- lines.append(f"| {index + 1} | {label} | {place} | {action} |")
719
-
720
- if len(stops) < len(items):
721
- lines.extend(
722
- [
723
- "",
724
- f"**Route remains incomplete:** {len(items) - len(stops)} image(s) "
725
- "still need a stop. The order above is preserved; add more stops "
726
- "when you are ready.",
727
- ]
728
- )
729
- elif len(stops) > len(items):
730
- lines.extend(
731
- [
732
- "",
733
- f"_{len(stops) - len(items)} extra route stop(s) were not used. "
734
- "This keeps one stop per image._",
735
- ]
736
- )
737
- return "\n".join(lines)
738
-
739
-
740
- def build_recall_markdown(kind: str, count: int) -> str:
741
- """Return an active-recall prompt that does not reveal the items."""
742
- if kind == "number":
743
- return "\n".join(
744
- [
745
- "### Active recall",
746
- "",
747
- f"1. Cover the table and the original number. Starting at chunk 1, "
748
- f"say the digit pair, its image word, and the picture for all "
749
- f"{count} chunk(s).",
750
- "2. Walk the chain backward from the last image to the first.",
751
- "3. Write the full number from memory, then compare it with the "
752
- "original digit order.",
753
- "",
754
- "_If you miss a link, repair the image rather than memorizing an "
755
- "explanation._",
756
- ]
757
- )
758
- return "\n".join(
759
- [
760
- "### Active recall",
761
- "",
762
- f"1. Cover the table. Say item 1, then item 2, through all {count} "
763
- "items without looking.",
764
- "2. Repeat the list backward.",
765
- "3. For each item, add one sentence explaining why it belongs in that "
766
- "position. Check the original list afterward.",
767
- "",
768
- "_Technical labels are not definitions. Verify their meaning in a "
769
- "source you trust._",
770
- ]
771
- )
772
-
773
-
774
- WORD_FIELD_NAMES = (
775
- "Meaning anchor",
776
- "Sound / keyword association",
777
- "Visual image",
778
- "Mini-story",
779
- "Memory Palace placement",
780
- "Active-recall question",
781
- )
782
-
783
-
784
- def word_default_drafts(term: str, context: str) -> list[list[str]]:
785
- """A no-model worksheet. These are prompts, not invented definitions."""
786
- term_text = term.strip()
787
- context_text = context.strip()
788
- meaning = (
789
- context_text
790
- if context_text
791
- else (
792
- "No definition supplied. Add your own trusted definition here; this "
793
- "card is for remembering the label only."
794
- )
795
- )
796
- return [
797
- ["Meaning anchor", meaning],
798
- [
799
- "Sound / keyword association",
800
- f'Say "{term_text}" aloud. Write a familiar word with a similar sound '
801
- "(rhyme, first syllable, or rhythm).",
802
- ],
803
- [
804
- "Visual image",
805
- "Turn the sound-alike into one concrete object. Exaggerate its size, "
806
- "color, texture, sound, and motion.",
807
- ],
808
- [
809
- "Mini-story",
810
- "Make the object do something surprising to something familiar in two "
811
- "or three quick beats.",
812
- ],
813
- [
814
- "Memory Palace placement",
815
- "Choose one familiar place. Put the image in a specific spot and add "
816
- "one sensory detail.",
817
- ],
818
- [
819
- "Active-recall question",
820
- "Without looking, say the term, what it means, and the image. Then check.",
821
- ],
822
- ]
823
-
824
-
825
- def assemble_word_card(
826
- term: str, context: str, table: Any
827
- ) -> tuple[str, list[str]]:
828
- """Turn the editable field table into a Markdown recall card."""
829
- raw_rows = table_to_rows(table)
830
- entries: list[tuple[str, str]] = []
831
- seen_fields: set[str] = set()
832
- for index, raw in enumerate(raw_rows, 1):
833
- raw = list(raw) + [""] * max(0, 2 - len(raw))
834
- field = raw[0].strip()
835
- draft = raw[1].strip()
836
- if not field and not draft:
837
- continue
838
- if not field:
839
- field = f"Additional note {index}"
840
- key = field.casefold()
841
- if key in seen_fields:
842
- field = f"{field} (additional note)"
843
- seen_fields.add(key)
844
- entries.append((field, draft))
845
-
846
- if not entries:
847
- raise ValueError("Draft or type at least one field before assembling the card.")
848
- missing = [
849
- name
850
- for name in WORD_FIELD_NAMES
851
- if name.casefold() not in {field.casefold() for field, _draft in entries}
852
- ]
853
-
854
- lines = [f"## Word / concept card: {term.strip()}", ""]
855
- if context.strip():
856
- lines.extend(["**Your supplied context:**", "", context.strip(), ""])
857
- else:
858
- lines.extend(
859
- [
860
- "**Your supplied context:** none. This card is label-only until "
861
- "you add a definition you trust.",
862
- "",
863
- ]
864
- )
865
-
866
- for field, draft in entries:
867
- lines.extend([f"### {field}", "", draft or "_(not filled in)_", ""])
868
-
869
- if missing:
870
- lines.extend(
871
- [
872
- f"**Not filled in:** {', '.join(missing)}. Add these before "
873
- "treating the card as complete.",
874
- "",
875
- ]
876
- )
877
-
878
- lines.extend(
879
- [
880
- "### Accuracy note",
881
- "",
882
- "Definitions are not verified by this app. Sound associations and "
883
- "images are memory cues, not factual claims. Replace any cue with a "
884
- "personal one; check technical meanings against a source you trust.",
885
- ]
886
- )
887
- return "\n".join(lines), missing
888
-
889
-
890
-
891
-
892
-
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """Deterministic mnemonic tools used by Memory Studio.
2
+
3
+ This module intentionally has no model or UI dependencies. It provides the
4
+ Major-System checker, curated starter words, input validation, editable-table
5
+ helpers, and deterministic story/palace/recall scaffolding. The optional local
6
+ language model lives in :mod:`local_llm`.
7
+ """
8
+
9
+ from __future__ import annotations
10
+
11
+ import math
12
+ import re
13
+ import uuid
14
+ from dataclasses import dataclass
15
+ from datetime import date, datetime, timedelta, timezone
16
+ from typing import Any, Iterable, Sequence
17
+
18
+ from pronunciation import dictionary_codes
19
+
20
+ # The Major System is phonetic. These descriptions are for explanation; the
21
+ # checker uses the pronunciation dictionary first and a spelling-based
22
+ # approximation only for words the dictionary does not contain.
23
+ MAJOR_SOUNDS = {
24
+ "0": "S, Z, soft C",
25
+ "1": "T, D, TH",
26
+ "2": "N",
27
+ "3": "M",
28
+ "4": "R",
29
+ "5": "L",
30
+ "6": "J, SH, CH, soft G",
31
+ "7": "K, hard C, hard G, Q, CK, NG",
32
+ "8": "F, V, PH",
33
+ "9": "B, P",
34
+ }
35
+
36
+ MAJOR_HELP = """\
37
+ | Digit | Consonant sounds |
38
+ |---:|---|
39
+ | 0 | S, Z, soft C |
40
+ | 1 | T, D, TH |
41
+ | 2 | N |
42
+ | 3 | M |
43
+ | 4 | R |
44
+ | 5 | L |
45
+ | 6 | J, SH, CH, soft G |
46
+ | 7 | K, hard C, hard G, Q, CK, NG |
47
+ | 8 | F, V, PH |
48
+ | 9 | B, P |
49
+
50
+ Vowels and H, W, Y are fillers and do not encode digits. The Major System is
51
+ based on sounds, not spelling. The checker looks words up in the CMU
52
+ Pronouncing Dictionary (American English) and falls back to an approximate
53
+ spelling check for words it does not know. Say every word aloud in your accent.
54
+ """
55
+
56
+ DIGIT_PATTERN = re.compile(r"\d")
57
+
58
+ MAX_NUMBER_DIGITS = 40
59
+ MAX_LIST_ITEMS = 30
60
+ MAX_ITEM_LENGTH = 300
61
+ MAX_WORD_TERM_LENGTH = 160
62
+ MAX_WORD_CONTEXT_LENGTH = 1200
63
+
64
+ # Common pronunciation patterns handled before single letters. They make the
65
+ # spelling-based checker less naive, but they do not make it a phonetic
66
+ # dictionary.
67
+ _INITIAL_SILENT = (
68
+ ("kn", "2"),
69
+ ("wr", "4"),
70
+ ("gn", "2"),
71
+ ("ps", "0"),
72
+ ("pn", "2"),
73
+ ("mn", "2"),
74
+ )
75
+ _DIGRAPHS = (
76
+ ("tch", "6"),
77
+ ("sh", "6"),
78
+ ("ch", "6"),
79
+ ("th", "1"),
80
+ ("ph", "8"),
81
+ ("ng", "7"),
82
+ ("ck", "7"),
83
+ ("qu", "7"),
84
+ )
85
+ _VOWELS = set("aeiou")
86
+
87
+
88
+ _GREEK_CH_STARTS = ("chao", "chor", "chem", "chara", "chasm", "choir")
89
+ _GH_AS_F = ("laugh", "cough", "rough", "tough", "enough", "trough")
90
+ # Spelling patterns that stand for two sounds: (pattern, digits).
91
+ _SOUND_PATTERNS = (
92
+ ("ssion", "62"),
93
+ ("tion", "62"),
94
+ ("sion", "62"),
95
+ ("cial", "65"),
96
+ ("tial", "65"),
97
+ ("sual", "65"),
98
+ ("ture", "64"),
99
+ ("sure", "64"),
100
+ ("ight", "1"),
101
+ ("dge", "6"),
102
+ ("sch", "07"),
103
+ )
104
+
105
+
106
+ def major_code(word: str) -> str:
107
+ """Approximate a word's Major-System digit code from its spelling.
108
+
109
+ This is the fallback for words that are not in the pronunciation
110
+ dictionary (see :mod:`pronunciation`). Vowels and H/W/Y are ignored as
111
+ digit carriers; common digraphs, silent letters, and suffixes are handled.
112
+ English spelling is irregular, so the result is approximate.
113
+ """
114
+ text = re.sub(r"[^a-z]", "", word.lower())
115
+ result: list[str] = []
116
+ last_digit: str | None = None
117
+ i = 0
118
+
119
+ def add_digit(digit: str) -> None:
120
+ nonlocal last_digit
121
+ # Traditional Major-System practice generally treats adjacent
122
+ # identical consonant sounds as one sound.
123
+ if digit != last_digit:
124
+ result.append(digit)
125
+ last_digit = digit
126
+
127
+ while i < len(text):
128
+ if i == 0:
129
+ matched_initial = False
130
+ for sequence, digit in _INITIAL_SILENT + (("x", "0"), ("gh", "7")):
131
+ if text.startswith(sequence, i):
132
+ add_digit(digit)
133
+ i += len(sequence)
134
+ matched_initial = True
135
+ break
136
+ if matched_initial:
137
+ continue
138
+
139
+ # Spellings whose sounds differ from their letters.
140
+ pattern = next(
141
+ (item for item in _SOUND_PATTERNS if text.startswith(item[0], i)), None
142
+ )
143
+ if pattern:
144
+ for digit in pattern[1]:
145
+ add_digit(digit)
146
+ i += len(pattern[0])
147
+ continue
148
+ # Greek-origin "ch" sounds like K: chrome, chlorine, chaos, chorus.
149
+ if text.startswith("ch", i) and (
150
+ text[i + 2 : i + 3] in ("r", "l")
151
+ or (i == 0 and text.startswith(_GREEK_CH_STARTS))
152
+ ):
153
+ add_digit("7")
154
+ i += 2
155
+ continue
156
+ # A final "nge" is N + soft G: change, hinge, orange.
157
+ if re.match(r"nge[sd]?$", text[i:]):
158
+ add_digit("2")
159
+ add_digit("6")
160
+ i += 3
161
+ continue
162
+ if text.startswith("gh", i):
163
+ # Mostly silent (night, though); F in laugh, cough, enough.
164
+ if text[: i + 2].endswith(_GH_AS_F):
165
+ add_digit("8")
166
+ i += 2
167
+ continue
168
+ # Common silent letters.
169
+ if (
170
+ (text[i] == "l" and i >= 1 and text[i - 1 : i + 2] in ("alk", "alm", "olk"))
171
+ or (text[i] == "l" and i >= 2 and text[i - 2 : i + 2] == "ould")
172
+ or (text[i] == "t" and i >= 1 and text[i - 1 : i + 3] in ("stle", "sten"))
173
+ or (text[i] == "b" and re.match(r"bt(s|le)?$", text[i:]))
174
+ or (text[i] == "g" and re.match(r"gn(s|ed)?$", text[i:]))
175
+ ):
176
+ i += 1
177
+ continue
178
+
179
+ matched = False
180
+ for sequence, digit in _DIGRAPHS:
181
+ if text.startswith(sequence, i):
182
+ add_digit(digit)
183
+ i += len(sequence)
184
+ matched = True
185
+ break
186
+ if matched:
187
+ continue
188
+
189
+ # A final "mb" is normally pronounced as M only.
190
+ if text.startswith("mb", i) and i + 2 == len(text):
191
+ add_digit("3")
192
+ i += 2
193
+ continue
194
+
195
+ char = text[i]
196
+ next_char = text[i + 1] if i + 1 < len(text) else ""
197
+
198
+ if char in _VOWELS:
199
+ # Vowels separate consonant sounds. For example, the two S sounds
200
+ # in "sauce" encode 00 rather than collapsing to a single 0.
201
+ last_digit = None
202
+ i += 1
203
+ continue
204
+ if char in "hwy":
205
+ i += 1
206
+ continue
207
+ if char in "bd":
208
+ add_digit("9" if char == "b" else "1")
209
+ elif char in "tf":
210
+ add_digit("1" if char == "t" else "8")
211
+ elif char == "n":
212
+ add_digit("2")
213
+ elif char == "m":
214
+ add_digit("3")
215
+ elif char == "r":
216
+ add_digit("4")
217
+ elif char == "l":
218
+ add_digit("5")
219
+ elif char == "j":
220
+ add_digit("6")
221
+ elif char in "sz":
222
+ add_digit("0")
223
+ elif char == "c":
224
+ add_digit("0" if next_char and next_char in "eiy" else "7")
225
+ elif char == "g":
226
+ add_digit("6" if next_char and next_char in "eiy" else "7")
227
+ elif char in "kq":
228
+ add_digit("7")
229
+ elif char == "v":
230
+ add_digit("8")
231
+ elif char == "p":
232
+ add_digit("9")
233
+ elif char == "x":
234
+ add_digit("7")
235
+ add_digit("0")
236
+ # Letters not covered above are ignored conservatively.
237
+ i += 1
238
+
239
+ return "".join(result)
240
+
241
+
242
+ def check_word_for_chunk(word: str, chunk: str) -> tuple[bool, str]:
243
+ """Check a candidate word against a digit chunk.
244
+
245
+ The pronunciation dictionary is used when it knows every part of the word;
246
+ otherwise the spelling approximation in :func:`major_code` is used and the
247
+ result says so.
248
+ """
249
+ text = (word or "").strip()
250
+ cleaned = re.sub(r"[^A-Za-z]", "", text)
251
+ if not cleaned:
252
+ return False, "needs a word"
253
+ if len(cleaned) > 80:
254
+ return False, "review: word is very long"
255
+ codes = dictionary_codes(text)
256
+ if codes:
257
+ if chunk in codes:
258
+ note = ", one pronunciation" if len(codes) > 1 else ""
259
+ return True, f"match (dictionary{note})"
260
+ readings = " or ".join(code or "no digits" for code in codes)
261
+ return False, f"review: dictionary reads {readings}"
262
+ detected = major_code(cleaned)
263
+ if detected == chunk:
264
+ return True, "match (spelling, approx.)"
265
+ if not detected:
266
+ return False, "review: spelling check found no consonant sounds"
267
+ return False, f"review: spelling check read {detected}"
268
+
269
+
270
+ def number_input_note(number_text: str) -> str:
271
+ """Explain which characters were not encoded, or return an empty string."""
272
+ ignored = sorted(set(re.sub(r"[\d\s]", "", number_text or "")))
273
+ if not ignored:
274
+ return ""
275
+ shown = " ".join(f"`{char}`" for char in ignored)
276
+ return (
277
+ f"Not encoded: {shown}. Only the digits, in order, become images; "
278
+ "remember any decimal point, sign, or separators separately."
279
+ )
280
+
281
+
282
+ def split_number(number_text: str) -> list[str]:
283
+ """Keep digit order and leading zeros; split into two-digit chunks."""
284
+ letters = sorted(set(re.findall(r"[A-Za-z]", number_text or "")))
285
+ if letters:
286
+ raise ValueError(
287
+ "The number contains letters ("
288
+ + ", ".join(letters)
289
+ + "). Only digits are encoded: remove the letters, or write the "
290
+ "value out in digits."
291
+ )
292
+ digits = "".join(DIGIT_PATTERN.findall(number_text or ""))
293
+ if not digits:
294
+ raise ValueError("Enter a number containing at least one digit (0-9).")
295
+ if len(digits) > MAX_NUMBER_DIGITS:
296
+ raise ValueError(
297
+ f"Enter up to {MAX_NUMBER_DIGITS} digits at a time "
298
+ f"({MAX_NUMBER_DIGITS // 2} two-digit images)."
299
+ )
300
+ return [digits[index : index + 2] for index in range(0, len(digits), 2)]
301
+
302
+
303
+ # Curated starting words for every two-digit code. These are suggestions, not
304
+ # the only valid mnemonic. Learners can replace any word and cue in the table.
305
+ # Every entry is checked against major_code() by the project's static checks.
306
+ CURATED_TWO_DIGIT: dict[str, tuple[str, str]] = {
307
+ "00": ("sauce", "a bubbling bottle of sauce tipping over"),
308
+ "01": ("suit", "an oversized business suit walking by itself"),
309
+ "02": ("sun", "a huge grinning sun"),
310
+ "03": ("sum", "a giant glowing plus sign"),
311
+ "04": ("sore", "a throbbing red sore"),
312
+ "05": ("sail", "a bright sail snapping in the wind"),
313
+ "06": ("sash", "a long silk sash fluttering"),
314
+ "07": ("sock", "an enormous striped sock"),
315
+ "08": ("safe", "a heavy metal safe with its door open"),
316
+ "09": ("soap", "a slippery bar of soap"),
317
+ "10": ("toes", "ten wiggling toes in the grass"),
318
+ "11": ("toad", "a lumpy toad wearing a crown"),
319
+ "12": ("tin", "a shiny tin can"),
320
+ "13": ("time", "a giant ticking clock"),
321
+ "14": ("tire", "a rolling rubber tire"),
322
+ "15": ("towel", "a wet towel snapping like a whip"),
323
+ "16": ("touch", "a glowing fingertip touching a surface"),
324
+ "17": ("tack", "a sharp thumbtack"),
325
+ "18": ("tofu", "a wobbly cube of tofu"),
326
+ "19": ("tub", "a steaming bathtub"),
327
+ "20": ("nose", "a giant nose honking like a trumpet"),
328
+ "21": ("net", "a fishing net full of silver fish"),
329
+ "22": ("nun", "a nun ringing a tiny bell"),
330
+ "23": ("name", "a name tag the size of a billboard"),
331
+ "24": ("narrow", "a very narrow alley"),
332
+ "25": ("nail", "a shiny nail being hammered"),
333
+ "26": ("nacho", "a loaded nacho dripping with cheese"),
334
+ "27": ("neck", "a giraffe's long neck"),
335
+ "28": ("navy", "a navy-blue ship"),
336
+ "29": ("nap", "a person curled up for a nap"),
337
+ "30": ("mouse", "a mouse wearing tiny boots"),
338
+ "31": ("mat", "a bright red doormat"),
339
+ "32": ("moon", "a huge smiling moon"),
340
+ "33": ("mummy", "a mummy wrapped in glowing bandages"),
341
+ "34": ("mower", "a roaring lawn mower"),
342
+ "35": ("mule", "a stubborn mule kicking"),
343
+ "36": ("match", "a burning match"),
344
+ "37": ("mug", "a chipped coffee mug"),
345
+ "38": ("movie", "a movie screen showing fireworks"),
346
+ "39": ("map", "a treasure map with a red X"),
347
+ "40": ("rose", "a giant rose with sharp thorns"),
348
+ "41": ("rat", "a giant brown rat"),
349
+ "42": ("rain", "a rain cloud pouring on one spot"),
350
+ "43": ("ram", "a battering ram with horns"),
351
+ "44": ("rear", "the rear end of a bus"),
352
+ "45": ("rail", "a shiny train rail"),
353
+ "46": ("roach", "a giant cockroach"),
354
+ "47": ("rock", "a boulder rocking back and forth"),
355
+ "48": ("roof", "a red-tiled roof"),
356
+ "49": ("rope", "a coiled rope"),
357
+ }
358
+ CURATED_TWO_DIGIT.update(
359
+ {
360
+ "50": ("lace", "a tangled lace ribbon"),
361
+ "51": ("lady", "a lady in a huge hat"),
362
+ "52": ("lion", "a roaring lion"),
363
+ "53": ("lime", "a sour green lime"),
364
+ "54": ("liar", "a liar with a long nose"),
365
+ "55": ("lily", "a tall white lily"),
366
+ "56": ("leech", "a fat black leech"),
367
+ "57": ("lock", "a heavy padlock"),
368
+ "58": ("leaf", "a giant green leaf"),
369
+ "59": ("lip", "enormous red lips"),
370
+ "60": ("chess", "a chessboard with giant pieces"),
371
+ "61": ("sheet", "a white sheet billowing"),
372
+ "62": ("chain", "a heavy iron chain"),
373
+ "63": ("gem", "a sparkling gem"),
374
+ "64": ("chair", "a chair with a broken leg"),
375
+ "65": ("jail", "a jail cell with rusty bars"),
376
+ "66": ("choo-choo", "a toy choo-choo train"),
377
+ "67": ("chick", "a fluffy yellow chick"),
378
+ "68": ("chef", "a chef flipping a pan"),
379
+ "69": ("ship", "a ship in a bottle"),
380
+ "70": ("kiss", "a lipstick kiss mark"),
381
+ "71": ("cat", "a cat with laser eyes"),
382
+ "72": ("can", "a dented tin can"),
383
+ "73": ("game", "a giant board game"),
384
+ "74": ("car", "a tiny car with a huge horn"),
385
+ "75": ("coal", "a glowing lump of coal"),
386
+ "76": ("cage", "a golden birdcage"),
387
+ "77": ("cake", "a towering layer cake"),
388
+ "78": ("cave", "a dark cave with glowing eyes"),
389
+ "79": ("cap", "a baseball cap spinning"),
390
+ "80": ("face", "a giant face pulling a funny expression"),
391
+ "81": ("fat", "a sizzling slab of fat"),
392
+ "82": ("fan", "a spinning electric fan"),
393
+ "83": ("foam", "a mountain of soap foam"),
394
+ "84": ("fire", "a crackling campfire"),
395
+ "85": ("file", "a metal file cabinet"),
396
+ "86": ("fish", "a fish wearing a crown"),
397
+ "87": ("fog", "a thick wall of fog"),
398
+ "88": ("fife", "a small silver flute called a fife"),
399
+ "89": ("fob", "a key fob dangling from a chain"),
400
+ "90": ("bus", "a double-decker bus"),
401
+ "91": ("bat", "a baseball bat swinging"),
402
+ "92": ("bone", "a giant dog bone"),
403
+ "93": ("beam", "a wooden beam balancing"),
404
+ "94": ("bear", "a bear hugging a honey pot"),
405
+ "95": ("bell", "a ringing brass bell"),
406
+ "96": ("beach", "a sunny beach with one palm tree"),
407
+ "97": ("book", "a giant open book"),
408
+ "98": ("beef", "a huge steak"),
409
+ "99": ("pipe", "a leaking pipe"),
410
+ }
411
+ )
412
+
413
+ CURATED_ONE_DIGIT: dict[str, tuple[str, str]] = {
414
+ "0": ("sea", "a crashing blue sea"),
415
+ "1": ("tea", "a steaming cup of tea"),
416
+ "2": ("hen", "a clucking hen"),
417
+ "3": ("ham", "a giant ham"),
418
+ "4": ("hare", "a fast hare"),
419
+ "5": ("eel", "a slippery eel"),
420
+ "6": ("shoe", "a giant shoe"),
421
+ "7": ("key", "a golden key"),
422
+ "8": ("ivy", "ivy climbing a wall"),
423
+ "9": ("bee", "a buzzing bee"),
424
+ }
425
+
426
+
427
+ def curated_candidate(chunk: str) -> tuple[str, str] | None:
428
+ """Return the curated (word, image cue) for a chunk, if available."""
429
+ if len(chunk) == 1:
430
+ return CURATED_ONE_DIGIT.get(chunk)
431
+ return CURATED_TWO_DIGIT.get(chunk)
432
+
433
+
434
+ def curated_major_rows() -> list[list[str]]:
435
+ """Rows for the read-only curated-word browser in the guide tab."""
436
+ rows: list[list[str]] = []
437
+ for chunk in [f"{number:02d}" for number in range(100)]:
438
+ word, cue = CURATED_TWO_DIGIT[chunk]
439
+ rows.append([chunk, word, cue])
440
+ for chunk in "0123456789":
441
+ word, cue = CURATED_ONE_DIGIT[chunk]
442
+ rows.append([chunk, word, cue])
443
+ return rows
444
+
445
+
446
+ @dataclass
447
+ class ChainItem:
448
+ """One ordered image in a storyboard."""
449
+
450
+ label: str
451
+ cue: str = ""
452
+ code: str = ""
453
+ link: str = "" # the learner's story line; blank means "write one for me"
454
+
455
+
456
+ @dataclass
457
+ class NumberRow:
458
+ """A reviewed row from the editable number table."""
459
+
460
+ chunk: str
461
+ word: str
462
+ image: str
463
+ link: str
464
+ check: str
465
+ source: str
466
+
467
+ def as_list(self) -> list[str]:
468
+ return [self.chunk, self.word, self.image, self.link, self.check, self.source]
469
+
470
+
471
+ @dataclass
472
+ class ListRow:
473
+ """A reviewed row from the editable ordered-list table."""
474
+
475
+ position: int
476
+ item: str
477
+ cue: str
478
+ link: str = ""
479
+
480
+ def as_list(self) -> list[str]:
481
+ return [str(self.position), self.item, self.cue, self.link]
482
+
483
+
484
+ def cell_text(value: Any) -> str:
485
+ """Convert an editable-table cell to a clean single-line string."""
486
+ if value is None:
487
+ return ""
488
+ if isinstance(value, float) and math.isnan(value):
489
+ return ""
490
+ text = str(value).replace("\r", " ").replace("\n", " ").strip()
491
+ if text in {"<NA>", "nan", "NaN"}:
492
+ return ""
493
+ return text
494
+
495
+
496
+ def table_to_rows(table: Any) -> list[list[str]]:
497
+ """Normalize Gradio/pandas/array table input into a list of string rows."""
498
+ if table is None:
499
+ return []
500
+ if isinstance(table, dict):
501
+ # Gradio 6's Dataframe API passes {"headers": ..., "data": ...}.
502
+ if "data" in table:
503
+ table = table["data"]
504
+ elif "value" in table:
505
+ table = table["value"]
506
+ else:
507
+ table = list(table.values())
508
+ if hasattr(table, "values"):
509
+ values = table.values
510
+ try:
511
+ table = values.tolist()
512
+ except AttributeError: # pragma: no cover - defensive for unusual tables
513
+ table = list(values)
514
+ rows: list[list[str]] = []
515
+ for row in table:
516
+ if isinstance(row, dict):
517
+ rows.append([cell_text(value) for value in row.values()])
518
+ elif isinstance(row, (list, tuple)):
519
+ rows.append([cell_text(value) for value in row])
520
+ else:
521
+ rows.append([cell_text(row)])
522
+ return rows
523
+
524
+
525
+ def validate_number_rows(
526
+ chunks: Sequence[str], table: Any
527
+ ) -> tuple[list[NumberRow], list[str]]:
528
+ """Check table shape/chunk order and refresh each word's approximate check."""
529
+ raw_rows = table_to_rows(table)
530
+ if not raw_rows:
531
+ return [], ["Draft candidate rows first, then review them in the table."]
532
+ if len(raw_rows) != len(chunks):
533
+ return [], [
534
+ f"The table has {len(raw_rows)} row(s) but the number has {len(chunks)} "
535
+ "chunk(s). Do not add or remove rows; regenerate the draft if the "
536
+ "number changed."
537
+ ]
538
+
539
+ reviewed: list[NumberRow] = []
540
+ errors: list[str] = []
541
+ for index, (expected, raw) in enumerate(zip(chunks, raw_rows), 1):
542
+ # Columns: chunk, word, image cue, story line, check, source.
543
+ raw = list(raw) + [""] * max(0, 6 - len(raw))
544
+ chunk = raw[0]
545
+ if chunk.isdigit():
546
+ # Dataframes sometimes turn "04" into 4. Restore the expected width.
547
+ chunk = chunk.zfill(len(expected))
548
+ if chunk != expected:
549
+ errors.append(
550
+ f"Row {index}: expected chunk {expected} but found "
551
+ f"{chunk or '(blank)'}. Keep the original row order."
552
+ )
553
+ continue
554
+ word = raw[1]
555
+ image = raw[2]
556
+ link = raw[3]
557
+ source = raw[5] or "your table"
558
+ if word:
559
+ _ok, check = check_word_for_chunk(word, chunk)
560
+ else:
561
+ check = "needs a word"
562
+ reviewed.append(NumberRow(expected, word, image, link, check, source))
563
+
564
+ if errors:
565
+ return [], errors
566
+ return reviewed, []
567
+
568
+
569
+ def number_blockers(rows: Iterable[NumberRow]) -> list[str]:
570
+ """Return human-readable reasons the number table is not ready for a story."""
571
+ problems: list[str] = []
572
+ for row in rows:
573
+ if not row.word:
574
+ problems.append(f"`{row.chunk}` has no word.")
575
+ elif not row.check.startswith("match"):
576
+ problems.append(f"`{row.chunk}` = **{row.word}**: {row.check}.")
577
+ return problems
578
+
579
+
580
+ def _strip_list_marker(line: str) -> str:
581
+ # Require whitespace (or end of line) after a numeric marker so values such
582
+ # as "1.5 cups" are not mistaken for a numbered list item.
583
+ return re.sub(
584
+ r"^\s*(?:\d+[.)](?:\s+|$)|[-*\u2022]\s+)", "", line
585
+ ).strip()
586
+
587
+
588
+ def parse_item_list(text: str) -> list[str]:
589
+ """Parse an ordered list while preserving the learner's item order."""
590
+ raw_lines = [line.strip() for line in (text or "").splitlines() if line.strip()]
591
+ if not raw_lines:
592
+ raise ValueError("Enter at least two items, one per line or separated by commas.")
593
+
594
+ if len(raw_lines) == 1:
595
+ line = _strip_list_marker(raw_lines[0])
596
+ parts = [
597
+ _strip_list_marker(part.strip())
598
+ for part in re.split(r"[,;]", line)
599
+ if part.strip()
600
+ ]
601
+ items = parts if len(parts) > 1 else [line]
602
+ else:
603
+ items = [_strip_list_marker(line) for line in raw_lines]
604
+
605
+ if any(not item for item in items):
606
+ raise ValueError(
607
+ "One line has no item text after its number or bullet. Fix that line "
608
+ "or remove it."
609
+ )
610
+ if len(items) < 2:
611
+ raise ValueError("Enter at least two items, one per line or separated by commas.")
612
+ if len(items) > MAX_LIST_ITEMS:
613
+ raise ValueError(
614
+ f"For a clear story, enter up to {MAX_LIST_ITEMS} items at a time."
615
+ )
616
+ too_long = [item for item in items if len(item) > MAX_ITEM_LENGTH]
617
+ if too_long:
618
+ raise ValueError(
619
+ f"Keep each item under {MAX_ITEM_LENGTH} characters. "
620
+ "Shorten the longest item and try again."
621
+ )
622
+ return items
623
+
624
+
625
+ def validate_list_rows(table: Any) -> tuple[list[ListRow], list[str]]:
626
+ """Validate an edited ordered-list table without silently dropping rows."""
627
+ raw_rows = table_to_rows(table)
628
+ if not raw_rows:
629
+ return [], ["Draft image cues first, then review them in the table."]
630
+
631
+ reviewed: list[ListRow] = []
632
+ errors: list[str] = []
633
+ for index, raw in enumerate(raw_rows, 1):
634
+ # Columns: #, item, image cue, story line.
635
+ raw = list(raw) + [""] * max(0, 4 - len(raw))
636
+ item = raw[1]
637
+ cue = raw[2]
638
+ link = raw[3]
639
+ if not item:
640
+ errors.append(
641
+ f"Row {index} has no item. Do not delete list items; restore the "
642
+ "row or regenerate from the list input."
643
+ )
644
+ continue
645
+ if len(item) > MAX_ITEM_LENGTH:
646
+ errors.append(
647
+ f"Row {index} is longer than {MAX_ITEM_LENGTH} characters. "
648
+ "Shorten that item."
649
+ )
650
+ continue
651
+ reviewed.append(ListRow(index, item, cue, link))
652
+
653
+ if errors:
654
+ return [], errors
655
+ return reviewed, []
656
+
657
+
658
+ # Link method: each new image DOES something to the previous one, with
659
+ # exaggeration, movement, and a sense. Each action is a participle so the line
660
+ # reads "you see <new image>, <action> <previous image>" whether the image is
661
+ # singular ("a rat") or plural ("enormous lips"). Each action has a matching
662
+ # sense so the line ends on a sound or feeling.
663
+ _LINK_ACTIONS = (
664
+ ("bursting out from under", "Everything shakes."),
665
+ ("crashing straight into", "CRASH! You feel it in your teeth."),
666
+ ("swallowing", "GULP!"),
667
+ ("landing with a thud on", "THUD! The floor shakes."),
668
+ ("wrapping itself around", "It squeezes tighter and tighter."),
669
+ ("squirting bright paint all over", "The paint smells sharp and sweet."),
670
+ ("balancing on top of", "It wobbles wildly. Hold your breath."),
671
+ ("chasing", "Hear the thundering footsteps."),
672
+ ("popping out of", "POP! Confetti flies everywhere."),
673
+ ("drenching", "SPLASH! It is ice cold."),
674
+ ("squashing", "SQUELCH!"),
675
+ ("catapulting", "WHOOSH! It flies high into the air."),
676
+ )
677
+ _STORY_OPENERS = (
678
+ "Then you see",
679
+ "Suddenly you spot",
680
+ "Next you notice",
681
+ "Right then you watch",
682
+ )
683
+ # No-model list cues are instructions ("Make it huge..."), not pictures.
684
+ _INSTRUCTION_WORDS = {
685
+ "make", "give", "shrink", "put", "turn", "cover", "picture", "imagine",
686
+ "add", "let", "see",
687
+ }
688
+
689
+
690
+ def _with_article(label: str) -> str:
691
+ lowered = label.casefold()
692
+ if lowered.startswith(("the ", "a ", "an ", "my ", "your ", "our ", "their ")):
693
+ return label
694
+ return label if label[:1].isupper() else f"the {label}"
695
+
696
+
697
+ def story_subject(item: ChainItem) -> str:
698
+ """The picture for an item: its image cue, or a giant version of it."""
699
+ cue = " ".join(item.cue.split()).rstrip(" .!?;:,")
700
+ first = cue.split(" ", 1)[0].casefold() if cue else ""
701
+ if not cue or first in _INSTRUCTION_WORDS:
702
+ label = " ".join(item.label.split())
703
+ if len(label.split()) <= 3:
704
+ return f"giant {label}"
705
+ return f"a huge, vivid picture of “{label}”"
706
+ if first in {"a", "an", "the"}:
707
+ cue = cue[:1].lower() + cue[1:]
708
+ return cue
709
+
710
+
711
+ def _story_reference(item: ChainItem) -> str:
712
+ """How a later line points back to an item: "the mat", or "it" if long."""
713
+ label = " ".join(item.label.split())
714
+ return _with_article(label) if len(label.split()) <= 3 else "it"
715
+
716
+
717
+ def _place_phrase(place: str) -> str:
718
+ place = " ".join(place.split())
719
+ if place.casefold().startswith(("the ", "my ", "a ", "an ", "your ")):
720
+ return place
721
+ return f"your {place}"
722
+
723
+
724
+ def auto_story_lines(items: Sequence[ChainItem], start_place: str = "") -> list[str]:
725
+ """One story line per item, following the Link method.
726
+
727
+ Line 1 sets the scene (at the first palace stop, if any). Every later line
728
+ starts from the previous image and makes the new image act on it, then ends
729
+ on a sound or feeling. Lines after the first do not depend on the start.
730
+ """
731
+ lines: list[str] = []
732
+ for index, item in enumerate(items):
733
+ subject = story_subject(item)
734
+ if index == 0:
735
+ if start_place.strip():
736
+ lines.append(
737
+ f"At {_place_phrase(start_place)}, you see {subject}. "
738
+ "It is ten times bigger than it should be."
739
+ )
740
+ else:
741
+ lines.append(
742
+ f"You open your eyes and see {subject} right in front of you. "
743
+ "It is ten times bigger than it should be."
744
+ )
745
+ continue
746
+ action, sense = _LINK_ACTIONS[(index - 1) % len(_LINK_ACTIONS)]
747
+ opener = _STORY_OPENERS[(index - 1) % len(_STORY_OPENERS)]
748
+ lines.append(
749
+ f"{opener} {subject}, {action} {_story_reference(items[index - 1])}. {sense}"
750
+ )
751
+ return lines
752
+
753
+
754
+ def link_scene_prompts(items: Sequence[ChainItem]) -> list[str]:
755
+ """One frozen moment per link: the new image acting on the previous one.
756
+
757
+ An image model draws one moment, not a sequence, so each prompt shows the
758
+ two pictures that must be remembered together.
759
+ """
760
+ prompts: list[str] = []
761
+ for index, item in enumerate(items):
762
+ subject = story_subject(item)
763
+ if index == 0:
764
+ prompts.append(f"{subject}, ten times bigger than it should be")
765
+ continue
766
+ action, _sense = _LINK_ACTIONS[(index - 1) % len(_LINK_ACTIONS)]
767
+ prompts.append(f"{subject} {action} {story_subject(items[index - 1])}")
768
+ return prompts
769
+
770
+
771
+ def chain_scene_prompt(items: Sequence[ChainItem]) -> str:
772
+ """A single-picture prompt for a short chain (2-4 images), else empty."""
773
+ if not 2 <= len(items) <= 4:
774
+ return ""
775
+ parts = [story_subject(items[0])]
776
+ for index in range(1, len(items)):
777
+ action, _sense = _LINK_ACTIONS[(index - 1) % len(_LINK_ACTIONS)]
778
+ parts.append(
779
+ f"{story_subject(items[index])} {action} "
780
+ f"{_story_reference(items[index - 1])}"
781
+ )
782
+ return "; ".join(parts) + ". All in one frozen moment"
783
+
784
+
785
+ def build_ordered_chain(
786
+ items: Sequence[ChainItem],
787
+ title: str = "Your story",
788
+ start_place: str = "",
789
+ ) -> str:
790
+ """The linked story: one numbered line per image, in order, then say-back.
791
+
792
+ Every item appears exactly once, in the order supplied, with its key word
793
+ in bold. ``item.link`` is used when the learner wrote one; otherwise a line
794
+ is generated with :func:`auto_story_lines`.
795
+ """
796
+ if not items:
797
+ return ""
798
+ autos = auto_story_lines(items, start_place)
799
+ codes = [item.code for item in items if item.code]
800
+ lines = [
801
+ f"### {title}",
802
+ "",
803
+ "Read it once, slowly, and see each picture move. The **bold** word is "
804
+ "the one to remember"
805
+ + ("; it carries the digits." if codes else ".")
806
+ ,
807
+ "",
808
+ ]
809
+ for index, item in enumerate(items):
810
+ code = f" (`{item.code}`)" if item.code else ""
811
+ line = " ".join((item.link or "").split()) or autos[index]
812
+ lines.append(f"{index + 1}. **{item.label}**{code}: {line}")
813
+
814
+ chain = " → ".join(item.label for item in items)
815
+ say_back = f"**Say it back:** {chain}"
816
+ if codes:
817
+ say_back += f" = `{' '.join(codes)}` → `{''.join(codes)}`"
818
+ lines.extend(["", say_back])
819
+ if codes:
820
+ lines.extend(
821
+ [
822
+ "",
823
+ "_Only the bold words carry digits. Colours, sizes, and actions are "
824
+ "glue, not extra numbers._",
825
+ ]
826
+ )
827
+ lines.extend(
828
+ [
829
+ "",
830
+ "_Make it yours: rewrite any line in the table's **Story line** "
831
+ "column, then rebuild. Clear a line to get a fresh one._",
832
+ ]
833
+ )
834
+ return "\n".join(lines)
835
+
836
+
837
+ def parse_palace_stops(text: str) -> list[str]:
838
+ """Parse an optional familiar-place route, keeping its order."""
839
+ lines = [line.strip() for line in (text or "").splitlines() if line.strip()]
840
+ if not lines:
841
+ return []
842
+ if len(lines) == 1:
843
+ parts = [part.strip() for part in re.split(r"[,;]", lines[0]) if part.strip()]
844
+ if len(parts) > 1:
845
+ lines = parts
846
+
847
+ stops: list[str] = []
848
+ for line in lines:
849
+ stop = re.sub(
850
+ r"^\s*(?:\d+[.)](?:\s+|$)|[-*\u2022]\s+)", "", line
851
+ ).strip()
852
+ if stop:
853
+ stops.append(stop)
854
+ if len(stops) > 60:
855
+ raise ValueError("Please use at most 60 palace stops in one route.")
856
+ return stops
857
+
858
+
859
+ def _table_cell(text: str) -> str:
860
+ return text.replace("|", "\\|").replace("\n", " ").strip()
861
+
862
+
863
+ # Anchors give every image a fixed, numbered place. A Memory Palace uses the
864
+ # learner's own route; peg systems use a fixed list of numbered images.
865
+ ANCHOR_PALACE = "Memory Palace (my route)"
866
+ ANCHOR_RHYME = "Number-Rhyme pegs (1-10)"
867
+ ANCHOR_SHAPE = "Number-Shape pegs (1-10)"
868
+ ANCHOR_MAJOR = "Major System pegs (1-100)"
869
+ ANCHOR_MODES = (ANCHOR_PALACE, ANCHOR_RHYME, ANCHOR_SHAPE, ANCHOR_MAJOR)
870
+
871
+ # Classic peg lists. Learners can swap any peg for their own image, but the
872
+ # same peg should then be kept for every list.
873
+ NUMBER_RHYME_PEGS = (
874
+ "bun (one)",
875
+ "shoe (two)",
876
+ "tree (three)",
877
+ "door (four)",
878
+ "hive (five)",
879
+ "sticks (six)",
880
+ "heaven (seven)",
881
+ "gate (eight)",
882
+ "vine (nine)",
883
+ "hen (ten)",
884
+ )
885
+ NUMBER_SHAPE_PEGS = (
886
+ "candle (1 is tall and thin)",
887
+ "swan (2 has a curved neck)",
888
+ "handcuffs (3 is a pair of loops)",
889
+ "sailboat (4 has a sail)",
890
+ "hook (5 curls like a hook)",
891
+ "golf club (6 curls at the bottom)",
892
+ "cliff edge (7 overhangs)",
893
+ "snowman (8 is two snowballs)",
894
+ "balloon on a string (9)",
895
+ "bat and ball (10)",
896
+ )
897
+
898
+
899
+ def major_peg(position: int) -> str:
900
+ """The curated Major-System word for a list position, used as a peg."""
901
+ if position == 100:
902
+ key = "00"
903
+ elif position < 10:
904
+ key = str(position)
905
+ else:
906
+ key = f"{position:02d}"
907
+ found = curated_candidate(key)
908
+ return f"{found[0]} ({position})" if found else ""
909
+
910
+
911
+ def anchor_stops(mode: str, count: int, palace_stops: Sequence[str]) -> list[str]:
912
+ """Return one anchor per position (may be shorter than ``count``)."""
913
+ if mode == ANCHOR_RHYME:
914
+ return list(NUMBER_RHYME_PEGS[:count])
915
+ if mode == ANCHOR_SHAPE:
916
+ return list(NUMBER_SHAPE_PEGS[:count])
917
+ if mode == ANCHOR_MAJOR:
918
+ return [major_peg(position) for position in range(1, min(count, 100) + 1)]
919
+ return list(palace_stops)
920
+
921
+
922
+ def build_anchor_markdown(
923
+ items: Sequence[ChainItem], mode: str, palace_stops: Sequence[str]
924
+ ) -> str:
925
+ """Attach each image to a palace stop or peg, in the same order.
926
+
927
+ Nothing is dropped: an image without an anchor stays in the table and is
928
+ marked as unplaced.
929
+ """
930
+ is_palace = mode not in (ANCHOR_RHYME, ANCHOR_SHAPE, ANCHOR_MAJOR)
931
+ stops = anchor_stops(mode, len(items), palace_stops)
932
+ if not stops:
933
+ return ""
934
+
935
+ if is_palace:
936
+ title = "### Memory Palace placement"
937
+ intro = (
938
+ "Walk the route in order, then walk it backward. Use places you can "
939
+ "picture clearly, and keep the route consistent across reviews."
940
+ )
941
+ place_header = "Place"
942
+ action_text = "Put this image here and add one sensory detail."
943
+ missing_place = "— (add a stop)"
944
+ missing_action = "Unplaced: add a familiar stop to keep the order complete."
945
+ else:
946
+ title = f"### {mode.split(' (')[0]}"
947
+ intro = (
948
+ "Count up from 1. At each number, see its peg and make the image "
949
+ "crash into it, sit on it, or wrap around it. Pegs let you jump "
950
+ 'straight to any position ("what is number 7?").'
951
+ )
952
+ place_header = "Peg"
953
+ action_text = "Make the image and the peg collide in one vivid scene."
954
+ missing_place = "— (no peg)"
955
+ missing_action = (
956
+ "No peg past this point: use Major System pegs or a palace route "
957
+ "to anchor every image."
958
+ )
959
+
960
+ lines = [
961
+ title,
962
+ "",
963
+ intro,
964
+ "",
965
+ f"| Order | Image | {place_header} | What to do |",
966
+ "|---:|---|---|---|",
967
+ ]
968
+ unplaced = 0
969
+ for index, item in enumerate(items):
970
+ code = f" (`{item.code}`)" if item.code else ""
971
+ label = _table_cell(f"**{item.label}**{code}")
972
+ if index < len(stops) and stops[index]:
973
+ place = _table_cell(stops[index])
974
+ action = action_text
975
+ else:
976
+ place = missing_place
977
+ action = missing_action
978
+ unplaced += 1
979
+ lines.append(f"| {index + 1} | {label} | {place} | {action} |")
980
+
981
+ if unplaced:
982
+ lines.extend(
983
+ [
984
+ "",
985
+ f"**Not every image is anchored:** {unplaced} image(s) have no "
986
+ f"{place_header.lower()}. The order above is preserved.",
987
+ ]
988
+ )
989
+ elif is_palace and len(stops) > len(items):
990
+ lines.extend(
991
+ [
992
+ "",
993
+ f"_{len(stops) - len(items)} extra route stop(s) were not used. "
994
+ "This keeps one stop per image._",
995
+ ]
996
+ )
997
+ return "\n".join(lines)
998
+
999
+
1000
+ def build_palace_markdown(
1001
+ items: Sequence[ChainItem], stops: Sequence[str]
1002
+ ) -> str:
1003
+ """Place the images along the learner's optional route, in the same order."""
1004
+ return build_anchor_markdown(items, ANCHOR_PALACE, stops)
1005
+
1006
+
1007
+ # Buzan's review rhythm: recall soon after learning, then at widening gaps.
1008
+ REVIEW_STEPS = (
1009
+ ("1st", "about 10 minutes after you finish", 0),
1010
+ ("2nd", "1 day later", 1),
1011
+ ("3rd", "1 week later", 7),
1012
+ ("4th", "1 month later", 30),
1013
+ ("5th", "3-6 months later (long-term)", 120),
1014
+ )
1015
+
1016
+
1017
+ def review_dates(start: date) -> list[tuple[str, str, date]]:
1018
+ return [
1019
+ (label, when, start + timedelta(days=days))
1020
+ for label, when, days in REVIEW_STEPS
1021
+ ]
1022
+
1023
+
1024
+ def build_review_markdown(start: date) -> str:
1025
+ """A review schedule with dates, starting from the day the card was built."""
1026
+ lines = [
1027
+ "### Review schedule",
1028
+ "",
1029
+ "Recall from memory first, then check. Each review should take a minute "
1030
+ "or two; a missed link is a cue to repair that image.",
1031
+ "",
1032
+ "| Review | When | Date |",
1033
+ "|---|---|---|",
1034
+ ]
1035
+ for label, when, day in review_dates(start):
1036
+ lines.append(f"| {label} | {when} | {day:%a %d %b %Y} |")
1037
+ lines.extend(
1038
+ [
1039
+ "",
1040
+ "_The study-sheet download includes a calendar file (.ics) with these "
1041
+ "dates. Nothing is scheduled or stored by the app._",
1042
+ ]
1043
+ )
1044
+ return "\n".join(lines)
1045
+
1046
+
1047
+ def build_review_ics(start: date, title: str) -> str:
1048
+ """An iCalendar file with all-day review reminders. Contains no study text."""
1049
+ stamp = datetime.now(timezone.utc).strftime("%Y%m%dT%H%M%SZ")
1050
+ safe_title = re.sub(r"[^A-Za-z0-9 .'()-]", "", title)[:60] or "study sheet"
1051
+ events: list[str] = []
1052
+ for number, (label, _when, day) in enumerate(review_dates(start), 1):
1053
+ if number == 1:
1054
+ continue # the 10-minute review happens before a calendar matters
1055
+ events.extend(
1056
+ [
1057
+ "BEGIN:VEVENT",
1058
+ f"UID:{uuid.uuid4()}@memory-studio.local",
1059
+ f"DTSTAMP:{stamp}",
1060
+ f"DTSTART;VALUE=DATE:{day:%Y%m%d}",
1061
+ f"DTEND;VALUE=DATE:{day + timedelta(days=1):%Y%m%d}",
1062
+ f"SUMMARY:Memory Studio review ({safe_title}) - {label}",
1063
+ "DESCRIPTION:Recall the story from memory\\, then check your "
1064
+ "study sheet.",
1065
+ "END:VEVENT",
1066
+ ]
1067
+ )
1068
+ return "\r\n".join(
1069
+ [
1070
+ "BEGIN:VCALENDAR",
1071
+ "VERSION:2.0",
1072
+ "PRODID:-//Memory Studio//Review schedule//EN",
1073
+ "CALSCALE:GREGORIAN",
1074
+ *events,
1075
+ "END:VCALENDAR",
1076
+ "",
1077
+ ]
1078
+ )
1079
+
1080
+
1081
+ def build_recall_markdown(kind: str, count: int) -> str:
1082
+ """Return an active-recall prompt that does not reveal the items."""
1083
+ if kind == "number":
1084
+ return "\n".join(
1085
+ [
1086
+ "### Active recall",
1087
+ "",
1088
+ f"1. Cover the table and the original number. Starting at chunk 1, "
1089
+ f"say the digit pair, its image word, and the picture for all "
1090
+ f"{count} chunk(s).",
1091
+ "2. Walk the chain backward from the last image to the first.",
1092
+ "3. Write the full number from memory, then compare it with the "
1093
+ "original digit order.",
1094
+ "",
1095
+ "_If you miss a link, repair the image rather than memorizing an "
1096
+ "explanation._",
1097
+ ]
1098
+ )
1099
+ return "\n".join(
1100
+ [
1101
+ "### Active recall",
1102
+ "",
1103
+ f"1. Cover the table. Say item 1, then item 2, through all {count} "
1104
+ "items without looking.",
1105
+ "2. Repeat the list backward.",
1106
+ "3. For each item, add one sentence explaining why it belongs in that "
1107
+ "position. Check the original list afterward.",
1108
+ "",
1109
+ "_Technical labels are not definitions. Verify their meaning in a "
1110
+ "source you trust._",
1111
+ ]
1112
+ )
1113
+
1114
+
1115
+ WORD_FIELD_NAMES = (
1116
+ "Meaning anchor",
1117
+ "Sound / keyword association",
1118
+ "Visual image",
1119
+ "Mini-story",
1120
+ "Memory Palace placement",
1121
+ "Active-recall question",
1122
+ )
1123
+
1124
+
1125
+ def word_default_drafts(term: str, context: str) -> list[list[str]]:
1126
+ """A no-model worksheet. These are prompts, not invented definitions."""
1127
+ term_text = term.strip()
1128
+ context_text = context.strip()
1129
+ meaning = (
1130
+ context_text
1131
+ if context_text
1132
+ else (
1133
+ "No definition supplied. Add your own trusted definition here; this "
1134
+ "card is for remembering the label only."
1135
+ )
1136
+ )
1137
+ return [
1138
+ ["Meaning anchor", meaning],
1139
+ [
1140
+ "Sound / keyword association",
1141
+ f'Say "{term_text}" aloud. Write a familiar word with a similar sound '
1142
+ "(rhyme, first syllable, or rhythm).",
1143
+ ],
1144
+ [
1145
+ "Visual image",
1146
+ "Turn the sound-alike into one concrete object. Exaggerate its size, "
1147
+ "color, texture, sound, and motion.",
1148
+ ],
1149
+ [
1150
+ "Mini-story",
1151
+ "Make the object do something surprising to something familiar in two "
1152
+ "or three quick beats.",
1153
+ ],
1154
+ [
1155
+ "Memory Palace placement",
1156
+ "Choose one familiar place. Put the image in a specific spot and add "
1157
+ "one sensory detail.",
1158
+ ],
1159
+ [
1160
+ "Active-recall question",
1161
+ "Without looking, say the term, what it means, and the image. Then check.",
1162
+ ],
1163
+ ]
1164
+
1165
+
1166
+ def assemble_word_card(
1167
+ term: str, context: str, table: Any
1168
+ ) -> tuple[str, list[str]]:
1169
+ """Turn the editable field table into a Markdown recall card."""
1170
+ raw_rows = table_to_rows(table)
1171
+ # The no-model worksheet's instructions are prompts, not answers. A row
1172
+ # still holding its prompt (or nothing) is reported as not filled in.
1173
+ prompts = {
1174
+ field.casefold(): text for field, text in word_default_drafts(term, context)
1175
+ }
1176
+ if context.strip():
1177
+ prompts.pop("meaning anchor", None)
1178
+ entries: list[tuple[str, str]] = []
1179
+ seen_fields: set[str] = set()
1180
+ for index, raw in enumerate(raw_rows, 1):
1181
+ raw = list(raw) + [""] * max(0, 2 - len(raw))
1182
+ field = raw[0].strip()
1183
+ draft = raw[1].strip()
1184
+ if not field and not draft:
1185
+ continue
1186
+ if not field:
1187
+ field = f"Additional note {index}"
1188
+ key = field.casefold()
1189
+ if key in seen_fields:
1190
+ field = f"{field} (additional note)"
1191
+ seen_fields.add(key)
1192
+ if not draft or prompts.get(key) == draft:
1193
+ draft = ""
1194
+ entries.append((field, draft))
1195
+
1196
+ if not entries or all(not draft for _field, draft in entries):
1197
+ raise ValueError(
1198
+ "Fill in at least one field with your own words before assembling "
1199
+ "the card. The worksheet rows are prompts, not answers."
1200
+ )
1201
+ filled = {field.casefold() for field, draft in entries if draft}
1202
+ missing = [name for name in WORD_FIELD_NAMES if name.casefold() not in filled]
1203
+
1204
+ lines = [f"## Word / concept card: {term.strip()}", ""]
1205
+ if context.strip():
1206
+ lines.extend(["**Your supplied context:**", "", context.strip(), ""])
1207
+ else:
1208
+ lines.extend(
1209
+ [
1210
+ "**Your supplied context:** none. This card is label-only until "
1211
+ "you add a definition you trust.",
1212
+ "",
1213
+ ]
1214
+ )
1215
+
1216
+ for field, draft in entries:
1217
+ lines.extend([f"### {field}", "", draft or "_(not filled in)_", ""])
1218
+
1219
+ if missing:
1220
+ lines.extend(
1221
+ [
1222
+ f"**Not filled in:** {', '.join(missing)}. Add these before "
1223
+ "treating the card as complete.",
1224
+ "",
1225
+ ]
1226
+ )
1227
+
1228
+ lines.extend(
1229
+ [
1230
+ "### Accuracy note",
1231
+ "",
1232
+ "Definitions are not verified by this app. Sound associations and "
1233
+ "images are memory cues, not factual claims. Replace any cue with a "
1234
+ "personal one; check technical meanings against a source you trust.",
1235
+ ]
1236
+ )
1237
+ return "\n".join(lines), missing
1238
+
1239
+
1240
+
1241
+
1242
+
pronunciation.py ADDED
@@ -0,0 +1,165 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """Dictionary-based Major-System codes from the CMU Pronouncing Dictionary.
2
+
3
+ The Major System encodes consonant *sounds*. CMUdict lists the phonemes of
4
+ about 135,000 American-English words, so it gives a much better check than
5
+ spelling rules. It is bundled as ``data/cmudict.dict.gz`` (BSD-style license,
6
+ see ``data/CMUDICT-LICENSE``), loaded lazily on the first check, and never
7
+ downloads anything. Words that are not in the dictionary fall back to the
8
+ spelling-based approximation in :mod:`mnemonic_core`.
9
+
10
+ Pronunciations are American English; a learner with another accent should
11
+ still say each word aloud (for example, non-rhotic speakers may not hear R in
12
+ "car").
13
+ """
14
+
15
+ from __future__ import annotations
16
+
17
+ import gzip
18
+ import itertools
19
+ import re
20
+ import threading
21
+ from functools import lru_cache
22
+ from pathlib import Path
23
+
24
+ DATA_DIR = Path(__file__).resolve().parent / "data"
25
+ DICTIONARY_PATH = DATA_DIR / "cmudict.dict.gz"
26
+ # Common-word frequency tiers derived from the public-domain 12dicts lists
27
+ # (see data/12DICTS-NOTICE.txt). Used only to rank and filter suggestions.
28
+ COMMON_WORDS_PATH = DATA_DIR / "common-words.txt.gz"
29
+
30
+ # ARPAbet consonant phonemes -> Major-System digit. Vowels, HH, W, and Y carry
31
+ # no digit. ER (as in "her") is an r-coloured vowel and counts as R.
32
+ PHONEME_DIGITS = {
33
+ "S": "0",
34
+ "Z": "0",
35
+ "T": "1",
36
+ "D": "1",
37
+ "TH": "1",
38
+ "DH": "1",
39
+ "N": "2",
40
+ "M": "3",
41
+ "R": "4",
42
+ "ER": "4",
43
+ "L": "5",
44
+ "JH": "6",
45
+ "CH": "6",
46
+ "SH": "6",
47
+ "ZH": "6",
48
+ "K": "7",
49
+ "G": "7",
50
+ "NG": "7",
51
+ "F": "8",
52
+ "V": "8",
53
+ "P": "9",
54
+ "B": "9",
55
+ }
56
+
57
+ _MAX_VARIANTS = 16
58
+ _LOAD_LOCK = threading.Lock()
59
+
60
+
61
+ def dictionary_available() -> bool:
62
+ return DICTIONARY_PATH.is_file()
63
+
64
+
65
+ def _phonemes_to_code(phonemes: list[str]) -> str:
66
+ digits = []
67
+ for phoneme in phonemes:
68
+ digit = PHONEME_DIGITS.get(re.sub(r"\d", "", phoneme))
69
+ if digit:
70
+ digits.append(digit)
71
+ return "".join(digits)
72
+
73
+
74
+ @lru_cache(maxsize=1)
75
+ def _load() -> tuple[dict[str, tuple[str, ...]], frozenset[str]]:
76
+ """Return word -> distinct Major codes, and the set of tagged names/places."""
77
+ codes: dict[str, list[str]] = {}
78
+ tagged: set[str] = set()
79
+ if not dictionary_available():
80
+ return {}, frozenset()
81
+ with _LOAD_LOCK, gzip.open(DICTIONARY_PATH, "rt", encoding="utf-8") as handle:
82
+ for line in handle:
83
+ body, _hash, comment = line.partition("#")
84
+ parts = body.split()
85
+ if len(parts) < 2:
86
+ continue
87
+ word = re.sub(r"\(\d+\)$", "", parts[0])
88
+ if comment.strip():
89
+ # Comments mark proper names, places, and abbreviations.
90
+ tagged.add(word)
91
+ code = _phonemes_to_code(parts[1:])
92
+ bucket = codes.setdefault(word, [])
93
+ if code not in bucket:
94
+ bucket.append(code)
95
+ return {word: tuple(values) for word, values in codes.items()}, frozenset(tagged)
96
+
97
+
98
+ def dictionary_codes(text: str) -> list[str] | None:
99
+ """All Major codes the dictionary allows for ``text``, or None if unknown.
100
+
101
+ Hyphenated or spaced phrases ("choo-choo", "ice cream") are looked up part
102
+ by part; every part must be in the dictionary.
103
+ """
104
+ parts = [part for part in re.split(r"[\s\-]+", (text or "").casefold()) if part]
105
+ parts = [re.sub(r"[^a-z']", "", part) for part in parts]
106
+ parts = [part for part in parts if part]
107
+ if not parts:
108
+ return None
109
+ table, _tagged = _load()
110
+ options: list[tuple[str, ...]] = []
111
+ for part in parts:
112
+ found = table.get(part) or table.get(part.strip("'"))
113
+ if not found:
114
+ return None
115
+ options.append(found)
116
+ combined: list[str] = []
117
+ for combo in itertools.islice(itertools.product(*options), _MAX_VARIANTS):
118
+ code = "".join(combo)
119
+ if code not in combined:
120
+ combined.append(code)
121
+ return combined
122
+
123
+
124
+ @lru_cache(maxsize=1)
125
+ def _frequency_tiers() -> dict[str, int]:
126
+ """word -> frequency tier (1 = most common) from the bundled 12dicts list."""
127
+ tiers: dict[str, int] = {}
128
+ if not COMMON_WORDS_PATH.is_file():
129
+ return tiers
130
+ with gzip.open(COMMON_WORDS_PATH, "rt", encoding="utf-8") as handle:
131
+ for line in handle:
132
+ if line.startswith("#"):
133
+ continue
134
+ word, _tab, tier = line.strip().partition("\t")
135
+ if word and tier.isdigit():
136
+ tiers[word] = int(tier)
137
+ return tiers
138
+
139
+
140
+ @lru_cache(maxsize=1)
141
+ def _reverse_index() -> dict[str, list[str]]:
142
+ """code -> common dictionary words, most frequent first."""
143
+ table, tagged = _load()
144
+ tiers = _frequency_tiers()
145
+ index: dict[str, list[str]] = {}
146
+ for word, codes in table.items():
147
+ if word in tagged or word not in tiers or not re.fullmatch(r"[a-z]{3,12}", word):
148
+ continue
149
+ for code in codes:
150
+ index.setdefault(code, []).append(word)
151
+ for words in index.values():
152
+ words.sort(key=lambda item: (tiers[item], len(item), item))
153
+ return index
154
+
155
+
156
+ def words_for_code(code: str, limit: int = 40) -> list[str]:
157
+ """Common words whose dictionary pronunciation encodes exactly ``code``.
158
+
159
+ Only words in the bundled common-word list are returned, most frequent
160
+ first. They include verbs and abstract words: options to review, not
161
+ advice. Concrete, picturable nouns usually make the best images.
162
+ """
163
+ if not re.fullmatch(r"\d{1,4}", code or ""):
164
+ return []
165
+ return _reverse_index().get(code, [])[: max(1, limit)]