Diag output from the previous deploy:
warm/prithvi: AttributeError: 'list' object has no attribute 'view'
The yaml-instantiated GenericNonGeoSegmentationDataModule stores
`means` and `stds` as Python lists (verified against the v2
config). The previous patch chained `_old.means.view(-1)
.detach().clone()` BEFORE handing them to _DictNormalize, which
crashed on the .view() call before reaching the conversion logic
inside the class.
Fix: pass `_old.means` and `_old.stds` to _DictNormalize raw.
The class's `_torch.as_tensor(mean).view(-1, 1, 1).float()` handles
list-to-tensor conversion internally — same end state, no
precondition that assumes the inputs are already tensors.
Also: lifespan warmup now stores a 5-line traceback tail in
_LAST_ERR so the next regression surfaces with a real stack
location through /healthz and /v1/diag.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two follow-ups to the previous EO-fix commit. The TerraMind LoRA
fix (drop tiled_inference at 224×224) shipped clean and is now
firing on both Stones; Prithvi-NYC-Pluvial v2 was still erroring
with the same `'list' object has no attribute 'view'` despite
the tensor-typed Normalize.
Real root cause: IBM's run_model in inference.py calls
`datamodule.aug({'image': tensor})['image']` — passing a dict
and indexing the result. The previous patch wrapped a kornia
AugmentationSequential there, which in 0.7+ doesn't accept dict
input cleanly and trips the AttributeError deep inside its
internal storage on first augmentation apply.
Fix: drop kornia entirely from the v2 patch path. Replace it
with a 12-line hand-rolled `_DictNormalize` that does the same
arithmetic — `(img - mean) / std` — and explicitly handles both
dict and tensor input shapes. Identical math, fewer moving
parts, no kornia version skew. Applied symmetrically in:
services/riprap-models/main.py
app/flood_layers/prithvi_live…
fix(eo): unblock TerraMind LoRA + Prithvi v2 inference on the L4 Space
The diag wrapper from the previous commit surfaced two real
upstream bugs that the legacy "deps unavailable" / generic 500
masking had been hiding:
1. TerraMind LULC + Buildings: "Expected size 12 but got size 2
for tensor nu" inside terratorch's tiled_inference.
Root cause: terratorch.tasks.tiled_inference doesn't handle
the 5-D (B, C, T, H, W) modality tensor shape that
backbone_use_temporal=True / backbone_temporal_n_timestamps=4
produces, so it slices/concats incorrectly when fusing the
per-modality patches and trips on the 12 (S2 bands) vs 2 (S1
bands) channel mismatch.
Fix: the canonical chip from app/context/eo_chip_cache.py is
already exactly 224×224 — the model's native input
resolution. Tiling is unnecessary at that size. Mirror the
training-time inference at experiments/18_terramind_nyc_lora/
shared/inference_ensemble.py:155 (`task.model(x)` direct on
the modality dict). tiled_inference is preserved as a
fallback for chips larger …
fix: thread Mellea attempt index + diagnose riprap-models 500s
Two regressions surfaced on the May 9 demo. Fix both.
1. Mellea reroll text concatenated in the briefing prose
============================================================
The strict-streaming reconcile path on a Mellea reroll fired
token events for the new attempt without telling the frontend
the attempt index changed. Result: the SvelteKit briefing
buffer never reset between attempts, so attempt 2 + attempt 3
text appended onto the already-rendered attempt 1 text.
Root cause: app/fsm.py:step_reconcile installed a token
forwarder that explicitly dropped the attempt_idx
(`lambda d, _ai: token_cb(d)`), and
app/intents/single_address.py:_on_token took only `delta`. So
the SSE `token` events carried no `attempt` field, and
web/sveltekit/src/lib/client/agentStream.ts:onAttemptStart
never fired because `d.attempt !== currentAttempt` stayed
undefined === undefined.
Fix:
- fsm.py: forward (delta, attempt_idx) to token_cb. Probe-fall
back…
fix(synthesis): handle 3-D DEM from _read_dem_patch (it returns (1,H,W))
The 400 from /v1/terramind on synthesis came from the wire shape being
(1,1,1,224,224) 5-D instead of (1,1,224,224) 4-D. Root cause: HF's
_read_dem_patch returns a 3-D (1, H, W) array (it interpolates via
torch.functional.interpolate then .squeeze(0).numpy()), and the
synthesis remote-call site assumed it was 2-D and added two leading
dims via dem[None, None, :, :].
Branch on dem_arr.ndim and add exactly the dim(s) we need to reach
4-D (B, C, H, W). Also leave the debug log on the droplet so any
future dim-shape mismatch surfaces in container logs.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
feat: every EO specialist's prediction renders as a map layer
Round-tripped the per-pixel argmax raster from the droplet so the HF
Space can vectorise it onto WGS84 polygons and the MapLibre layer
actually paints the live model output. Was: only terramind_synthesis's
local fallback path generated polygons; everything else returned
class fractions only and the map showed nothing.
Wire format: droplet's /v1/prithvi-pluvial and /v1/terramind responses
gain `pred_b64` (uint8 base64) + `pred_shape` + (terramind only)
`class_labels`. Small payload — 224x224 uint8 round-trips in ~50 KB.
HF side:
- new app/context/_polygonize.py: polygonize_class_raster() and
polygonize_binary_mask() helpers
- app/context/eo_chip_cache.py adds `bounds_4326` to chip output
- app/context/terramind_nyc lulc/buildings accept bounds_4326,
_try_remote polygonises the response and attaches polygons_geojson
- app/context/terramind_synthesis.py polygonises the remote response
- app/flood_layers/prithvi_live.py polygonises the remote water mask
- app/fsm.py threads chip.boun…
fix(synthesis): DEM shape is (1, 1, H, W) per encoder probe
Empirically verified against terratorch_terramind_v1_base_generate by
calling model({'DEM': torch.zeros(...)}) with each shape and reading
the encoder error:
(1, 224, 224) -> not enough values to unpack (got 3)
(1, 1, 224, 224) -> OK, LULC out (1, 10, 224, 224)
(1, 1, 1, 224, 224) -> too many values to unpack (got 5)
The encoder unpacks 'B, C, H, W = x.shape' so 4-D is required; DEM
has 1 channel, hence (1, 1, H, W). Local code's comment said this
but the implementation was off by one (.unsqueeze(0) only adds one
dim). Fix the local path too so it doesn't break if anyone runs
synthesis off the AMD path.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
fix: synthesis DEM shape matches local 3-D contract
The terramind v1 base generative encoder adds its batch dim internally;
sending an extra leading dim makes the embedding layer try to unpack
5-D into B, C, H, W and raise ValueError. Match the local code: HF
sends (1, H, W); droplet treats incoming (1,1,H,W) as a legacy shape
and squeezes; and rejects anything else with a clean 400 instead of
a 500 with a noisy traceback.
Also add n_building_components (scipy.ndimage.label connected-component
count) to the droplet's buildings response so the TerraMind Buildings
card renders the real number instead of '0 distinct components'.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
feat: terramind_synthesis now routes through droplet remote inference
Synthesis (DEM -> generative LULC via TerraMind v1 base) was the only
EO specialist still local-only. On the HF Space, terratorch's import
chain crashes with `RuntimeError: operator torchvision::nms does not
exist` because torchvision's C extension can't load against our CPU
torch wheel — so synthesis returned a clean skip rather than running.
Add a remote dispatch path:
- services/riprap-models/main.py: extend `_TERRAMIND_SPECS` with the
`synthesis` entry (10 ESRI LULC labels), add `_load_terramind_synthesis`
that pulls the v1 base generate model from terratorch's
FULL_MODEL_REGISTRY, and `_terramind_synthesis_inference` that
takes a 4-D (B, 1, H, W) DEM and emits class fractions matching the
local code's response shape. Same /v1/terramind dispatch handles
all three adapter names now (lulc / buildings / synthesis).
- TerramindIn schema: make all modality fields optional. lulc/buildings
still require s2; synthesis requires dem; the dispatch enforces.
- app/inference.p…
fix(droplet): _build_chip_tensor handle 5D input from eo_chip_cache
The HF Space's eo_chip_cache hands the droplet a chip already shaped
(B, C, T, H, W) — see app/context/eo_chip_cache.py:_to_terramind_tensors.
The droplet's _build_chip_tensor assumed 3-D (C, H, W) input, called
.unsqueeze(1).repeat(1, n_timesteps, 1, 1) which raised:
RuntimeError: Number of dimensions of repeat dims can not be smaller
than number of dimensions of tensor
That's why every terramind_lulc / terramind_buildings request returned
non-ok at /v1/terramind, why the HF trace showed terratorch as the
apparent cause (after fall-through), and why /healthz never had any
terramind models loaded — the route never made it past chip prep.
Make _build_chip_tensor idempotent: pass 5-D through, add batch to 4-D,
expand to T=4 + batch for legacy 3-D. Hot-patched on the droplet
(docker cp + docker restart riprap-models); committing source so the
next bring-up via scripts/deploy_droplet.sh inherits the fix.
Verified via live HF Space probe: terramind_lulc returns
dominant_class='Built…
sync: bring HF Space to parity with origin/main (squash, XET-backed)
Squash-merge of all commits on origin/main not present on HF Space:
- fix(riprap-models): port v2 test_transform patch (services container)
- feat(map): wire TerraMind LULC + Ida 2021 HWM map layers
- fix(map): MapLegend booleans, FloodNet tier proxy
- fix(slides): deck visual fixes
- docs: live-system screenshots (now as LFS/XET pointers, not raw blobs)
- chore: repo cleanup, session artifacts to docs/sessions/
- chore: LFS tracking rules for screenshot and slide PNGs/PPTX
Binary files (screenshots + deck.pptx) are stored as git-lfs/git-xet
pointers so HF's pre-receive hook accepts them. git-xet handles
upload to XET object store during the push.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Squashed from 5438cc8..6904684. The slides/asce/deck.pptx is tracked
via git-lfs (added *.pptx to .gitattributes), so this commit carries
only the LFS pointer — no binary blob in history.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Voice pass: strip em-dashes from user-facing docs
Constraint set:
- No em-dashes (—). Replace with periods, semicolons, commas, or
parens depending on context.
- No "not just X but Y" constructions (none found; nothing to do).
- Drop AI-tells (no "delve / leverage / cutting-edge / robust /
comprehensive / it's worth noting" etc found; nothing to do).
Files touched:
- README.md (rewritten cleanly; new live-demo URL pointing at the
AMD-hackathon org Space; probe_addresses.py mentioned in the
local-development block)
- RESEARCH.md (rewritten cleanly; same content, no em-dashes)
- ARCHITECTURE.md (bulk em-dash sweep; manual fix for two parenthetical
sentences the regex broke: line 105 modes-see-§4 and line 134
Burr-FSM-DAGWorks)
- METHODOLOGY.md (bulk; heading "Riprap. Scoring Methodology" →
"Riprap scoring methodology")
- MONDAY.md (bulk; trailing-space-period sweep)
- services/riprap-models/README.md (bulk + manual: parenthetical bug
on which-surface-laptop-or-HF-Space hosts the FastAPI process; "Deploy"
headings now …
Three artifacts that turn the bootstrap-droplet's hand-built
container into a reproducible bring-up on any AMD ROCm GPU node:
- services/riprap-models/Dockerfile — extends the public
rocm/pytorch:rocm7.2.3_ubuntu24.04_py3.12_pytorch_release_2.9.1
image (same minor torch version as the bootstrap droplet's
bespoke +git build) with our pinned terratorch / granite-tsfm /
transformers / peft / sentence-transformers / gliner stack. Bakes
in the MI300X tuning env (HIP_FORCE_DEV_KERNARG=1 etc) so a fresh
container doesn't need the "remember to set these" incantation.
- services/riprap-models/requirements-full.txt — exact pip pins
captured from the running terramind container on 2026-05-05.
Curated: only the leaves the Dockerfile installs on top of the
ROCm PyTorch base; transitive deps resolve from these. Excludes
torch / torchvision / torchaudio / amd-* (provided by the base).
- scripts/deploy_droplet.sh — idempotent one-shot bring-up. Takes
IP + bearer token, verifie…
feat: route all GPU-accelerable inference through MI300X (Phase 1+2 of full GPU)
The user's directive: "I want anything that can be GPU accelerated to
run on there. Otherwise, keep it on the CPU of wherever it's running."
Lands seven pieces in one commit; each can stand alone but they're
interlocked.
app/inference.py (new, ~250 lines)
Router shim that mirrors app/llm.py's shape but for non-LLM models.
Exports: prithvi_pluvial(), terramind(), ttm_forecast(),
granite_embed(), gliner_extract(), healthcheck(), backend_info(),
plus the typed RemoteUnreachable exception caller modules catch
to fall back to local. Env-driven via RIPRAP_ML_BACKEND
(auto|remote|local) / RIPRAP_ML_BASE_URL / RIPRAP_ML_API_KEY,
same shape as RIPRAP_LLM_*.
services/riprap-models/ (new microservice)
FastAPI service that runs alongside vLLM on the AMD MI300X
droplet. One endpoint per model class:
/v1/prithvi-pluvial Prithvi-NYC-Pluvial v2 segmentation
/v1/terramind LULC / Buildings / Synthesis (LoRA)
/v1/ttm-forecast …
