The Sat May 9 trace surfaced two regressions on the lablab UI Space:
prithvi_nyc_pluvial: RuntimeError: operator torchvision::nms does
not exist
terramind.lulc: deps unavailable on this deployment:
terratorch (RuntimeError), peft
terramind.buildings: deps unavailable on this deployment:
terratorch (RuntimeError), peft
Root cause in both: the specialist tries the configured remote
inference service, the call fails (RemoteUnreachable, ok=False, or
generic exception), and the code silently falls through to the local
terratorch path. Local on the cpu-basic UI Space has a torch /
torchvision binary mismatch (torchvision::nms not registered), so
either we crash with that RuntimeError or — for terramind_nyc, where
_DEPS_OK is False at import time — we surface a misleading "deps
unavailable" message that hides the real (remote) failure.
Fix: when remote was attempted, never fall through to local. Return
a …
feat(emissions): per-call inference energy + token tracker
Adds an additive emissions ledger that records every llm.chat call
and every remote app.inference._post call, surfacing the per-query
total Wh / J + token counts on the briefing.
Backend
- app/emissions.py: thread-local Tracker with hardware power profiles
(MI300X 600 W, T4 50 W, Apple M 20 W, x86 30 W) and a summarize()
rollup with by-kind, by-hardware, and frontier-cloud comparison.
- app/llm.py: chat() times each call, picks hardware via engine
detection, reads prompt/completion tokens from LiteLLM usage for
non-stream calls, estimates from response text for streams.
- app/inference.py: _post() records duration vs the MI300X profile.
- app/fsm.py: captures the parent-thread tracker and re-installs it on
the iter_steps runner thread (mirrors the token_callback pattern);
attaches `emissions` alongside the existing `energy` block in both
run() and the iter_steps final event.
- app/context/eo_chip_cache.py + app/flood_layers/prithvi_live.py:
ThreadPoolExecutor initializ…
fix(synthesis): handle 3-D DEM from _read_dem_patch (it returns (1,H,W))
The 400 from /v1/terramind on synthesis came from the wire shape being
(1,1,1,224,224) 5-D instead of (1,1,224,224) 4-D. Root cause: HF's
_read_dem_patch returns a 3-D (1, H, W) array (it interpolates via
torch.functional.interpolate then .squeeze(0).numpy()), and the
synthesis remote-call site assumed it was 2-D and added two leading
dims via dem[None, None, :, :].
Branch on dem_arr.ndim and add exactly the dim(s) we need to reach
4-D (B, C, H, W). Also leave the debug log on the droplet so any
future dim-shape mismatch surfaces in container logs.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
feat: every EO specialist's prediction renders as a map layer
Round-tripped the per-pixel argmax raster from the droplet so the HF
Space can vectorise it onto WGS84 polygons and the MapLibre layer
actually paints the live model output. Was: only terramind_synthesis's
local fallback path generated polygons; everything else returned
class fractions only and the map showed nothing.
Wire format: droplet's /v1/prithvi-pluvial and /v1/terramind responses
gain `pred_b64` (uint8 base64) + `pred_shape` + (terramind only)
`class_labels`. Small payload — 224x224 uint8 round-trips in ~50 KB.
HF side:
- new app/context/_polygonize.py: polygonize_class_raster() and
polygonize_binary_mask() helpers
- app/context/eo_chip_cache.py adds `bounds_4326` to chip output
- app/context/terramind_nyc lulc/buildings accept bounds_4326,
_try_remote polygonises the response and attaches polygons_geojson
- app/context/terramind_synthesis.py polygonises the remote response
- app/flood_layers/prithvi_live.py polygonises the remote water mask
- app/fsm.py threads chip.boun…
fix(synthesis): DEM shape is (1, 1, H, W) per encoder probe
Empirically verified against terratorch_terramind_v1_base_generate by
calling model({'DEM': torch.zeros(...)}) with each shape and reading
the encoder error:
(1, 224, 224) -> not enough values to unpack (got 3)
(1, 1, 224, 224) -> OK, LULC out (1, 10, 224, 224)
(1, 1, 1, 224, 224) -> too many values to unpack (got 5)
The encoder unpacks 'B, C, H, W = x.shape' so 4-D is required; DEM
has 1 channel, hence (1, 1, H, W). Local code's comment said this
but the implementation was off by one (.unsqueeze(0) only adds one
dim). Fix the local path too so it doesn't break if anyone runs
synthesis off the AMD path.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
fix: synthesis DEM shape matches local 3-D contract
The terramind v1 base generative encoder adds its batch dim internally;
sending an extra leading dim makes the embedding layer try to unpack
5-D into B, C, H, W and raise ValueError. Match the local code: HF
sends (1, H, W); droplet treats incoming (1,1,H,W) as a legacy shape
and squeezes; and rejects anything else with a clean 400 instead of
a 500 with a noisy traceback.
Also add n_building_components (scipy.ndimage.label connected-component
count) to the droplet's buildings response so the TerraMind Buildings
card renders the real number instead of '0 distinct components'.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
feat: terramind_synthesis now routes through droplet remote inference
Synthesis (DEM -> generative LULC via TerraMind v1 base) was the only
EO specialist still local-only. On the HF Space, terratorch's import
chain crashes with `RuntimeError: operator torchvision::nms does not
exist` because torchvision's C extension can't load against our CPU
torch wheel — so synthesis returned a clean skip rather than running.
Add a remote dispatch path:
- services/riprap-models/main.py: extend `_TERRAMIND_SPECS` with the
`synthesis` entry (10 ESRI LULC labels), add `_load_terramind_synthesis`
that pulls the v1 base generate model from terratorch's
FULL_MODEL_REGISTRY, and `_terramind_synthesis_inference` that
takes a 4-D (B, 1, H, W) DEM and emits class fractions matching the
local code's response shape. Same /v1/terramind dispatch handles
all three adapter names now (lulc / buildings / synthesis).
- TerramindIn schema: make all modality fields optional. lulc/buildings
still require s2; synthesis requires dem; the dispatch enforces.
- app/inference.p…
fix: catch RuntimeError in EO deps probes, add demo playbook
The deps-probe pattern in terramind_nyc / terramind_synthesis /
prithvi_live only caught ImportError. But terratorch's import chain
on the HF Space raises RuntimeError("operator torchvision::nms does
not exist") because torchvision's C extension can't load against
our CPU torch wheel. The exception propagates past the probe, the
module fails to load, and the FSM step's outer except surfaces the
raw RuntimeError as 'err' in the trace instead of 'skipped'.
Catch any Exception during the probe and treat as unavailable. The
specialist returns a clean 'skipped' entry, the trace UI renders it
gray (silent) instead of red (errored), and the demo reads as
honest engineering instead of broken plumbing.
DEMO-PLAYBOOK.md: top-level handoff doc with the 3 demo queries,
what each one shows, the trace skip messages to expect, and the
end-to-end smoke test.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
fix: clean skip for terramind_nyc on torchvision::nms
Same pattern as terramind_synthesis: when remote inference returns
non-ok or is unreachable AND local _ensure_adapter() raises the
torchvision binary-extension error, surface a clean 'skipped'
result with both root causes named instead of a noisy err.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
fix: surface torchvision::nms as honest skip in terramind_synthesis
Was bubbling up as a noisy 'operator torchvision::nms does not exist'
RuntimeError in the trace. Detect it explicitly and return a clean
skip explaining that local inference is unavailable due to the
torchvision binary extension not loading on this deployment.
terramind_synthesis has no remote-inference path, so this is the
honest engineering outcome.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
feat: enable EO specialists via remote ML on HF Space
Prithvi-NYC-Pluvial, TerraMind LULC, TerraMind Buildings, and the
eo_chip fetch were all returning 'deps unavailable on this
deployment' on the HF Space because their deps gates required
terratorch (which doesn't fit the HF build sandbox per the four
prior failed attempts documented in the Dockerfile). But these
specialists already have remote-inference paths to the AMD MI300X
via app/inference.py — terratorch is only needed for *local*
inference, not for the chip-fetch + remote-call path that the HF
Space actually uses.
Refactor:
- Add planetary-computer, pystac-client, rioxarray, xarray, einops
to requirements.txt. These are pure-Python or thin numpy/rasterio
wrappers (~10 MB combined) — small enough to fit the HF build
budget that terratorch's 250 MB transitive dep cone overflows.
- prithvi_live._has_required_deps splits into two tiers: chip-fetch
deps (Tier 1, always required) and local-inference deps (Tier 2,
only required when remote inference is unavailable). When r…
Squashed from 5438cc8..6904684. The slides/asce/deck.pptx is tracked
via git-lfs (added *.pptx to .gitattributes), so this commit carries
only the LFS pointer — no binary blob in history.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
feat: route all GPU-accelerable inference through MI300X (Phase 1+2 of full GPU)
The user's directive: "I want anything that can be GPU accelerated to
run on there. Otherwise, keep it on the CPU of wherever it's running."
Lands seven pieces in one commit; each can stand alone but they're
interlocked.
app/inference.py (new, ~250 lines)
Router shim that mirrors app/llm.py's shape but for non-LLM models.
Exports: prithvi_pluvial(), terramind(), ttm_forecast(),
granite_embed(), gliner_extract(), healthcheck(), backend_info(),
plus the typed RemoteUnreachable exception caller modules catch
to fall back to local. Env-driven via RIPRAP_ML_BACKEND
(auto|remote|local) / RIPRAP_ML_BASE_URL / RIPRAP_ML_API_KEY,
same shape as RIPRAP_LLM_*.
services/riprap-models/ (new microservice)
FastAPI service that runs alongside vLLM on the AMD MI300X
droplet. One endpoint per model class:
/v1/prithvi-pluvial Prithvi-NYC-Pluvial v2 segmentation
/v1/terramind LULC / Buildings / Synthesis (LoRA)
/v1/ttm-forecast …
Stones C4: wire TerraMind-NYC LULC + Buildings into FSM
Adds three new Burr actions in app/fsm.py (gated behind
RIPRAP_HEAVY_SPECIALISTS):
step_eo_chip -> writes eo_chip (S2L2A + S1RTC + DEM)
step_terramind_lulc -> writes terramind_lulc
step_terramind_buildings-> writes terramind_buildings
eo_chip is a per-query cache: one ~150 MB Sentinel-2/1/DEM fetch from
Microsoft Planetary Computer, shared by both TerraMind specialists so
they don't each refetch. The cache lives in app/context/eo_chip_cache.py
and gracefully no-ops when terratorch / planetary_computer / pystac_client
/ rioxarray aren't installed.
Both TerraMind specialists call into app/context/terramind_nyc.py (added
in C3), which routes inference through terratorch tiled_inference at
224x224 and ships the per-class fraction summary the reconciler quotes.
Reconciler:
- app/reconcile.py:build_documents() now emits tm_lulc (Touchstone,
after prithvi_live) and tm_buildings (Keystone, after the legacy
terramind_synthetic prior).
- trim_docs_to_pl…
Two ground-truth-from-the-public layers that complement the modelled
flood maps:
311 — flood-descriptor complaints in a buffered radius around the
query point. Real residents reporting real water, with an
agency/route field that lets us filter to DEP-routed
complaints (street/sewer flooding) vs. the noisy 'damp
basement' general complaints.
FloodNet — NYU/CUSP/Mayor's Office ultrasonic-depth sensor network.
Reports cm-resolution depth to the closest sensor,
plus the most recent flood event > 10 cm.
Both fetch live from NYC Open Data SoQL / FloodNet's REST API, with
a haversine bbox pre-filter to keep the queries cheap.
USGS 3DEP 1/3 arcsec DEM (resampled to 30 m for runtime), with two
hydrology indices computed offline by whitebox-workflows:
TWI — topographic wetness, ln(a/tan β), high where flow accumulates
HAND — height above nearest drainage, low where overbank flooding
pools first
These three rasters give the briefing terrain context that the
empirical/forward-modelled flood layers can't: even a parcel outside
every published zone can be in a topographic sink that any heavy rain
will fill.
.gitattributes routes *.tif through LFS.
Runtime requirements for the FastAPI service, Burr FSM, geo stack,
sentence-transformers (Granite Embedding 278M for RAG), and Ollama
client (Granite 4.1 reconciler). Offline-only deps for fixture
pre-compute live in scripts/, not here.
Empty app/{,assets,context,flood_layers}/__init__.py establish the
package layout that the specialists will fill in.
