Research Notes
Engineering findings, negative results and design decisions that would otherwise be lost. Each was learned by probe or execution, not by assumption, and each is recorded so it is not rediscovered.
Status tags: MEASURED · RESOLVED · REJECTED · OPEN · ATTEMPTED.
1. Findings that changed the code
F4-1 — the MiniLM tokenizer ceiling (MEASURED)
The MiniLM tokenizer's own ceiling is 256 (verified by probe). The project truncates to 128 —
a deliberate truncation well inside the ceiling, not the model limit. Satellite queries are short;
halving the sequence halves attention cost for no measurable accuracy loss. The encoder asserts
max_length ≤ 256, because truncating above the ceiling is a silent no-op.
F4-2 — the router needs no GPU (MEASURED)
The encoder is frozen, so embeddings are cached and the 50,822-parameter adapter trains on cached vectors. Measured on CPU: 20 epochs / 4,096 vectors in 0.28 s. No GPU required.
F4-3 — splits must be by group (MEASURED)
Splits are by group (template / hard-negative family), never by example. Hard-negative families are placed in the test split so their accuracy measures generalisation rather than memorisation. Splitting by example would leak template variants across the boundary.
F5-1 — AutoModelForVision2Seq does not exist (MEASURED)
In transformers 5.17.0, AutoModelForVision2Seq does not exist (not merely deprecated);
AutoModelForImageTextToText is present. The loader is resolved by feature detection, never
hardcoded.
F5-2 — the processor cost overrun is ~17×, not 4× (MEASURED)
The processor's default longest_edge is 2048, which upscales 512-px tiles 4× and then splits
them (do_image_splitting=True) into 17 sub-images:
| Setting | pixel_values |
prompt tokens |
|---|---|---|
| default | (1, 17, 3, 512, 512) |
1142 |
pinned (processor_longest_edge: 512) |
(1, 1, 3, 512, 512) |
— |
The plan estimated a 4× cost overrun; the real figure is ~17×. The value must be set explicitly on the processor at construction time.
F5-3 — prompts must go through the chat template (MEASURED)
SmolVLM requires one <image> token per image in the prompt. Hand-written prompt strings raise
ValueError. Prompts are always built through processor.apply_chat_template().
P7-1 — the RemoteCLIP projected dimension is 512 (MEASURED)
The RemoteCLIP ViT-B/32 transformer width is 768, but visual.proj maps to a projected dim of
512. The grounding head's per-cell feature is 4 × 512 = 2048, declared in config so
core/config.py can validate the head without importing torch.
C-1 — the availability mask is consumed by the head, not by CROMA (MEASURED)
CROMA always sees the canonical channel counts (12 optical, 2 SAR). The availability mask is applied
by the fusion head (input_dim = 3 × 768 + 12 + 2 = 2318), not by the encoder.
C-6 — T4 is SM 7.5, so training uses fp16, not bf16 (MEASURED)
The training precision is fp16 because the target GPU (T4) is compute capability 7.5. bf16 is not available there.
C-7 — image_resolution % 8 == 0 (MEASURED)
CROMA requires image_resolution % 8 == 0. The native value 120 yields 225 patches.
C-8 — ZeroGPU does not support torch.compile (MEASURED)
torch.compile must never be enabled on the (historical) ZeroGPU target.
C-9 — STANet change hyperparameters are upstream-verified (MEASURED)
Change detection uses STANet-style architecture with upstream-verified hyperparameters (PAM self-attention, ResNet-18 encoder).
2. The grounding resolution decision — a pre-registered rejection (REJECTED)
Question: should grounding decode at 448 or 224?
Answer: 224. 448 was rejected — and the rejection is notable because it was pre-registered and then confirmed by a paired test:
| Comparison (over identical samples, n = 16,159) | 448 vs 224 |
|---|---|
| mean best IoU | −0.0147 |
| recall@0.5 | −0.0022 |
| recall@0.10 | −0.0699 |
| recall@0.25 | −0.0243 |
| latency | 1.59× |
| paired 95 % CI | [−0.0160, −0.0134] |
| paired t | −22.63 |
| 448 better on | 8.5 % of records |
| 448 worse on | 20.9 % of records |
448 lost on every axis. The pre-registered decision rule and the paired test agree on 224. This is a model of how a resolution decision should be made: declared in advance, then tested.
3. The router defect — a real bug, found and fixed
3.1 Symptom
The query "Where are the built-up areas in this image?" collapsed to vqa and answered
"River" — instead of routing to grounding. A second query, "Where is the new airport?",
behaved the same way.
3.2 Root cause
Two functions with different information:
interpret()— produces the console's reading; asset-count-blind (text only).chooseTask()— performs dispatch; asset-count-aware.
The defect was in the dispatch path's handling of spatial/lexical cues, so region queries fell
through to the generic VQA specialist. See ARCHITECTURE.md §4.
3.3 Fix and verification (RESOLVED)
The fix was deployed to SatQuery-Frontend and validated by three independent live passes:
| Pass | Deployed HEAD | Result |
|---|---|---|
| 1 | ff46eba42b18 + d413d3672311 |
8/8 |
| 2 | 2d7ae53b482d |
8/8 |
| 3 | 2d7ae53b482d |
8/8 |
Both defect queries now dispatch to grounding:
| Query | Run id | Dispatched |
|---|---|---|
| Where are the built-up areas in this image? | run_467ffa406f22 |
grounding |
| Where is the new airport? | run_46980ba55c62 |
grounding |
24 live runs, 24 correct dispatches, 0 mock nodes. Screenshots are in
../screenshots/.
4. The harness false-positive — caught before it could lie
An earlier live-validation harness typed queries with synthetic CDP key events, which Chrome silently drops when the window lacks OS focus. The harness therefore dispatched the page's default query and still recorded a "result" — a false pass.
Fix: the current harness asserts form state before dispatch (q_ok, obs_ok, t0_ok), and
uses deterministic query entry (js() value-set + type_text() via CDP Input.insertText).
Independent check: the earlier 8/8 run was re-examined and confirmed not infected — its answers were query-specific and the query text was embedded in the answers. The failure mode is recorded because it is exactly the silent false-positive an evaluation harness must never have.
5. The transport_mode: auto fallthrough (OPEN)
SATQUERY_TRANSPORT=auto tries the tunnel, then falls through to the forward path on timeout. The
forward path to a private repo returns 302 quickly, but the wake step still consumes
SATQUERY_WAKE_TIMEOUT_S (120 s) first — so a worst-case failed request takes ≈ 249 s
(150 + 120). This is the root shape of the observed transient tunnel gap. Recorded as OPEN; a
deployed fix for the codespace_name newline on the wake path was authored separately.
6. The interpret() / chooseTask() asymmetry — intentional (RESOLVED)
For "What changed between the earlier and later image?" with one asset attached, the console
reads change while dispatch correctly falls back to change_vqa. This is not a bug: the
reading describes the question's intent, the dispatch respects what can actually be computed with the
assets present. It is documented so it is not mistaken for a defect.
7. Environment findings (would otherwise cost hours)
| Finding | Detail |
|---|---|
| Dead proxy in the sandbox | outbound calls need --noproxy '*' (curl) or ProxyHandler({}) (Python). |
| pytest is only in the repo venv | .venv/Scripts/python.exe; a bare pytest misses it. |
| Full-suite pytest trips a bulk-delete guard | sandbox-specific; affects test_safe_delete_shim. |
Cloudflare _headers concatenate |
two matching rules are merged, not overridden; Chromium takes the first max-age. |
Cloudflare 308-redirects X.html → /X |
reference the extensionless path. |
A forwarded Codespace port returns 302 for a private repo |
this is why the tunnel exists. |
| Chrome drops synthetic CDP key events without OS focus | the harness false-positive (§4). |
browser-use block-buffers stdout even when redirected |
needs explicit line buffering to stream. |
8. The BigEarthNet format contradiction (ATTEMPTED, reported not resolved)
The BigEarthNet data format contradicts the original plan. This was reported rather than
silently patched, because quietly changing the preprocessing would move the frozen config hash. The
BigEarthNet documentation does not specify a percentile stretch; this project nevertheless applies
percentile normalisation (2/98) to match the CROMA contract. That is a deliberate, documented
choice, not an upstream fact. See DATASETS.md §5.3.