Space side (app.py): replace the part-based archive store with a normalized
SQLite history DB on /data (players, stays, visit runs, closed rooms). WAL,
a read-only reader pool, trigger-maintained counters, incremental auto_vacuum,
90-day retention keyed on the scanner clock, and a DB-size guard. The live
snapshot is served untouched (gzip passthrough, ETag/304). New endpoints:
/ingest-rooms-history and /room-scan/{archive,search,room,player,db-stats}.
Scanner (room_lookup/room_scanner.js): push only deltas. Identity resolution
(account id > session map > 24h name->id), continuous stays, visit runs,
per-day timelines and a "seen with" graph across all worlds; idempotent,
order-independent upserts with a merge table for replay/name reuse. Local
backup log (history_log/) for --replay, state v2 with one-time v1 migration.
Rooms tab (static/room_scan_tab.js): archive/search/room/player views over
the new endpoints; index.html cache-bust bump.
Co-Authored-By: Claude Opus 4.8 <noreply@anth…
6 days ago
Secure mod access: one-time invite links, per-device sessions, device list
The shared SM_MOD_KEY used to travel in ?mod=<key> links and sat raw in a 30-day
cookie, so one leaked link made anyone a mod. Now:
- The owner creates a one-time invite per mod at /mod/admin. Opening it shows a
confirm page; only the button (POST) redeems it, so link previews can't burn it.
A used link is worthless, unused links expire (1 h / 24 h / 7 days).
- Each device gets a random session cookie (HttpOnly, SameSite=Lax, Secure on the
Space); only its SHA-256 is stored in /data/mod_access.json. Sessions unused for
30 days expire.
- /mod/admin lists every device (name, browser/OS, network, added, last seen), can
remove single devices or all others, cancel unused invites, and shows a security
log (new devices, a login used from a new network, used/invalid links, wrong
owner keys).
- Owner login via SM_OWNER_KEY in a POST form (/mod/admin?login only, rate-limited
per IP and globally, 404 for everyone else).
- No secret in URLs anymore: ?mod=<key> and ?mod_key= grant n…
- POST /ingest-rooms (ingest-key, gzip) + GET /room-scan (mod/ingest-gated)
receive and serve the W10/W11 room store pushed by the SP7 scanner.
- room_lookup/room_scanner.js: the SP7 scanner (directory discovery, immediate
share-code resolution, serialized roster visits with StaffSeatJoin fallback,
backoff, roster history). Runs off-Space; kept here as the canonical source.
- Manual /room-lookup records its join time so the scanner yields the account.
- Room Lookup tab button + panel are now mod-only (MOD region); tab highlighting
uses data-tab instead of button position so it stays correct without the button.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
feat(export): include all gear enchants (sword freeze, shield thorns + fortune)
The account export only reported the optimizer-relevant enchants (sword
flame/looting, shield looting). Mods asked for the full set. The server lookup
(/live-lookup) already returns sword_freeze, shield_thorns and shield_fortune —
only the frontend opt-list account model dropped them.
Thread them through as hidden, optimizer-irrelevant pass-through fields:
accHTML hidden inputs (iswfr/ishth/ishfo), applyDiscordAccountData fills them
from the lookup response, getAccountsSnapshot + removeAcc snapshot read them,
doLoad restores them on save/load, and buildAccountsExport now emits
sword.freeze and shield.thorns/shield.fortune. Purely additive — no change to
the optimizer (it ignores these) or to existing export consumers. Verified in
the browser: export now yields sword{flame,looting,freeze} and
shield{looting,thorns,fortune}. Export contract test extended (177 passed; the
1 failure is the pre-existing unrelated frontend tabs test).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
perf(leaderboard): fuzzy focus checks top ranks first — kills the frequent-term slow path
The occurrence-count branch mislabelled queries: a very common substring (e.g.
"o") took the early-break rank scan, which walked deep with the heavy per-row
predicate when the top rows happened not to contain it (~10s live), while the
real 30s "oettam" case turned out to be a cold-boot transient (steady-state is
already ~0.4s once the scan index is warm).
Replace the count heuristic with a two-step focus that is fast for every query:
1) check the top _SM_LB_FUZZY_TOP_K (=20000) ranks directly via a cheap
lowercased segment `in` (no heavy predicate) — common terms and any high-rank
match resolve in a handful of iterations, and the whole probe is bounded at
~50ms;
2) only if nothing is in the top ranks, collect all occurrences and take the
minimum in-view rank fully vectorised (as before) for rare/deep matches;
no occurrences at all returns instantly.
Same result as the old walk (highest-ranked substring match). Benchmark: common
single chars 0.14s -> ~0.001s, deep rare …
perf(leaderboard): fast fuzzy search too — no-match/partial-name lookups no longer O(N)
Live check surfaced that a name search with NO exact match (e.g. "oettam",
"niceface") still fell back to an O(N) Python walk over the whole view --
~12-20s. That is the real "search is slow" for partial/typo/while-typing
queries. Make the fuzzy focus fast with the same big-string scan:
- The name scan index now indexes each row's visible names AND their normalised
keys (a-z0-9), so the fast path finds exactly what the search predicate does,
including fancy/special-char names matched via their normalised form -- no
regression there. (player_id is intentionally dropped; nobody name-searches an
id.) This also makes the exact-name list consistent with the predicate.
- _sm_lb_fuzzy_focus: count the term's occurrences in one C-level pass; if zero,
return instantly (the worst 20s case -> ~ms); if rare, collect the few
candidates, map offsets to rows via np.searchsorted, drop out-of-view rows via
the inverse-permutation -1 sentinel, confirm with the real predicate and take
th…
feat(leaderboard): normal search lists all same-name accounts (any spelling); jump only when unique
A case-insensitive name search jumped to the FIRST match. Now, when several
accounts carry the searched name in different capitalisations (oettaM / OETTAM /
OeTtAm), it returns ALL of them as a ranked list with their true leaderboard
positions -- like the wildcard filter -- and only focus-jumps when exactly one
account matches (unchanged) or when there is no exact-name match (falls back to
the existing exact/fuzzy jump, so player_id and partial lookups still work).
Reuses the wildcard big-string scan index: an exact-name match is the needle
"\x01name\x01" (whole-segment equality, not substring) found in one C-level
pass; hit offsets map to rows via vectorised np.searchsorted, candidates are
confirmed with the real exact predicate (honours case-sensitive), and ranks
come from the view's inverse permutation. Candidates outside the current view
(below min_level / other measure) are dropped via a -1 sentinel -- which also
hardens the wildcard path for non-store-floor views. Works for publ…
perf(leaderboard): ~20-60x faster mod wildcard search via big-string prefilter
The mod-only wildcard filter (niceface*) ran the anchored regex matcher over
every one of ~1.55M rows, rebuilding each row's display-name list first --
several seconds per query, even though only a handful of rows ever match.
Replace the per-row scan with a literal-core prefilter: the longest wildcard-
free run of the glob (niceface* -> "niceface") is a necessary substring of any
match, so we find its occurrences in ONE C-level pass. Once per index
generation we build a big lowercased string of all display names
(\x01name\x01name\x01, separators the core can't cross) plus a numpy array of
per-row start offsets; a query does re.finditer(core) over that string and
maps hit offsets to rows with a vectorised np.searchsorted, then confirms just
those few candidates with the real anchored regex. Ranks come from the index's
inverse permutation.
Result set is provably identical (the core is a necessary substring, so no true
match is dropped; the regex remains the source of truth), including …
Green the last 5 stale tests (ore enum + HTML strings verified)
None were value/display bugs; all pre-existing stale expectations:
- Ore names: verified live in __game-scripts.js (const ores=[{type:0
name:"Coal"},{1 Emerald},{2 Diamond},{3 Ruby},{4 Gold}]). Tests
assumed a wrong (alphabetical) enum; fixed to type 0=Coal, 1=Emerald.
- HP row: now wraps values in numHtml(...) (clickable numbers); assert
the live-health-override branch with the numHtml wrapper.
- game-catalog.js: assert the src prefix so the ?v= cache-buster passes.
- tabs array: gained the 'room' tab; expectation updated to match.
Full suite: 164 passed, 0 failed.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Fix gear type->name resolution for exact HP/power (0 deviation)
Armor slots (helmet/shoe) were resolved with a legacy +1 offset plus a
Solarflare override that both types 20 and 21 mapped to. Against the
current catalog (25 armor tiers: ...Stormborn(20), Solarflare(21),
Graveborn(22), Mourner's(23), Eternal(24)) this resolved sub-max armor
one tier too strong, so the displayed/computed maxHealth over-counted.
- Resolve all armor purely via the direct catalog type->name map
(armor arrays keep None at type 0, so type == index; no offset).
- Add SWORD_TYPE_NAMES/SHIELD_TYPE_NAMES + _sm_gear_type_name so
swords/shields also resolve by direct type map (offset=-1 only as a
no-catalog fallback); wire inventory/storage/clan-chest lookups too.
- Drop the stale _SM_ARMOUR_TYPE_OVERRIDES/OFFSETS/DIRECT_MIN_TYPE.
Verified live against 480 accounts (getPlayerInfo): 474 exact vs the
server's maxHealth (0 deviation), incl. 259/260 accounts below level
1000; 0 over-counts remain. The 6 non-matching accounts all have MORE
HP than their gear+level allows (HP…
Mod view: fully server-side via hidden cookie, no client-side trace
Non-mods must not even know the mod view exists, so all mod hints are removed from the
public HTML/JS. /?mod=<token> now sets an HttpOnly cookie server-side and redirects to a
clean /; /leaderboard decides 100+ vs 1000+ from that cookie. Wrong token just serves the
normal page (no error, no hint). /?mod=off clears it. Test stub gets RedirectResponse.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Perf: serve page/JS from memory on the event loop + bigger thread pool
The Space is a single uvicorn worker on 2 vCPUs that also runs slow blocking
lookups, the leaderboard scanner and a 1.5GB forge DB. Page loads were slow
because the page and JS were sync routes sharing one 40-thread pool with the
lookups, and were sent no-store (re-downloaded every visit through the busy
worker).
- Serve index.html, game-catalog.js and armor-render-templates.js from an
in-memory, pre-gzipped cache via async routes (_MemStatic). The page is now
delivered on the event loop, not the sync thread pool, so it loads in ms even
when every worker thread is stuck on a slow SwordMasters lookup; it also isn't
re-gzipped per request. Read fresh on each restart/deploy.
- ETag + Cache-Control: no-cache instead of no-store: repeat visits revalidate
and get a tiny 304 instead of re-downloading ~150KB, while staying always
fresh. /health is now async too (fast health checks -> fewer restart loops).
- Raise the anyio sync thread pool from 40 to 128 tokens. Lookups are
I/O-…
Account Info:
- Player Details now shows XP needed for the next level (threshold L^4),
updating live when the level field is edited manually.
Upgrade Lab:
- "And better" checkbox for Dwarfting & Merging and Elf Legendary, plus an
"And more" checkbox for the Pet Enhancer: on = at least the target
(level-or-higher / count-or-more), off = the odds for exactly that target.
- Materials-needed card ("all owned = 0") gains a source-level selector:
choose which owned level to count (Lv1..target); target level = target count.
- Default sword is now Soulkeeper's Blade instead of Solbrand.
Optimize tab market recommendations:
- Each buy suggestion can be excluded and re-optimized. Exclude offers three
scopes: this exact variant, this name at this level (any enchants), or the
whole item. Buy/sell suggestions can also have their price overridden (used
for buying and selling). Active corrections show as removable chips + Reset.
Motivation: market prices are imperfect; this lets the…
Adds an EXPORT button next to Save/Load/Update that downloads every loaded
account as a .txt file, so sites without their own live lookup can consume
the data in one stable shape.
Per account: name, power, level, sword (level/flame/looting), shield
(level/looting), three pet slots and the four armor slots. Hours/day and
room are deliberately left out.
Payload is JSON inside the .txt for unambiguous parsing. Power is emitted as
plain digits rather than a JSON number: it routinely exceeds 2^53, arrives
from the lookup as "6.3e+21", and can be typed as a game suffix like
"1.04sxt" — all three are normalised so consumers can use a plain integer
parser. Unset gear exports as null; pets always export three slots.
The fourth button no longer fit the panel header on a 320px phone, where
.panel{overflow:hidden} clipped it, so the action row wraps when space is
actually too tight. Wider phones and desktop keep the single-line row.
Verified end-to-end in a browser against a real live lookup a…
Every account was listed twice: once under id:<player_id> (from a live
lookup) and once under name:<username> (a legacy row stored before the
player_id was known). The write-path merge only collapsed rows sharing a
player_id, so the id-less shadow was never absorbed, and the cleanup pass
written for exactly this case was defined but never called.
Fixes, defense in depth:
- take_previous_row now absorbs the name:<username> shadow whenever an
id:<player_id> row is written, so the two never coexist going forward.
- load_unlocked runs the existing cleanup once on cache restore, healing
the ~180k stale shadow rows already in the store.
- the snapshot deduplicates name-shadows of id-keyed accounts, so the UI
never shows a player twice even during the transient window inside a
scan batch. Distinct accounts that merely share a display name (verified
to exist) keep their separate id rows.
Regression tests cover the snapshot, the write-path absorb, and cleanup,
including the same-dis…
