SatQuery / docs /CHANGELOG.md
thundercode's picture
release: add docs/CHANGELOG.md
67d2633 verified
|
Raw History Blame
9.62 kB

Changelog

All notable changes to SatQuery AI. Dates are ISO-8601. This changelog records what changed and what was verified, and marks unresolved items honestly rather than omitting them.

Status tags: RESOLVED · OPEN · CLOSED · DEFERRED · REJECTED.


[1.0.0] — 2026-09-25 — Final documentation, model release and archival freeze

The release that makes the project legible, citable and reproducible: repository documentation, a deep research/engineering documentation set, model documentation, a Hugging Face model release, a curated public repository, and a verified evidence archive.

Added

Documentation

  • Repository README — motivation through citation, with the deployed topology, the six supported tasks, routing, the eight execution events, measured results, live validation, and known limitations.
  • Deep architecture reference (hub + 10 chapters):
    Chapter Topic
    docs/ARCHITECTURE.md hub: thesis, sub-document index, cross-cutting principles
    docs/architecture/01-system-overview.md thesis, component inventory, frozen-backbone strategy
    docs/architecture/02-deployment-topology.md four tiers, gateway, tunnel, wake flow, cold start
    docs/architecture/03-request-lifecycle.md the nine-state controller, validation, tiling
    docs/architecture/04-router.md frozen MiniLM, the 5-head adapter, interpret() vs chooseTask(), lexical fallback
    docs/architecture/05-specialists.md all six tasks in depth
    docs/architecture/06-evidence-and-confidence.md evidence aggregation, temperature scaling, the eight events
    docs/architecture/07-configuration-freeze.md the registry, enforced invariants, the config hash
    docs/architecture/08-api-contract.md the four endpoints, envelopes, error codes
    docs/architecture/09-frontend.md the static pages, real-vs-preview, platform traps
    docs/architecture/10-observability-and-ops.md health, counters, traces, what is not observed
  • Research documentation — MODELS.md, BENCHMARKS.md, DATASETS.md, TRAINING.md, EVALUATION.md, DEPLOYMENT.md, REPRODUCIBILITY.md, LIMITATIONS.md, RESEARCH_NOTES.md, CHANGELOG.md.
  • Model documentation — MODEL_CARD.md, plus generated models/manifest.json and models/checksums.sha256 (computed by reading the files, never typed).
  • Metric verification tooling — release/tools/verify_readme_metrics.py, which walks every quoted metric to its source artifact. Result: ALL CLAIMS VERIFIED (20/20).

Release

  • Hugging Face release of the six trained artifacts to thundercode/SatQuery, with a verification record (HF_RELEASE_VERIFICATION.md). Independent re-download confirmed 6/6 artifacts byte-identical.
  • Curated public repository at Anish-lab-blip/SatQuery-AI with RELEASE_MANIFEST.md.
  • Evidence archive (ZIP64, artifacts/ verbatim per owner decision G3) with an integrity-verified extraction and a documented remote location.

Changed

  • The monorepo README.md was materially stale and is superseded by this release's README. The stale version called the frontend "hermetic — no backend calls", put Render/Codespace as "in progress", described a /v1/* contract, and pointed at the untracked deploy/ as the deployment source. All four were wrong.
  • hf/ documentation was stale — it asserted the project "does not own any model weights … ships no weights, no binaries, and no large artifacts" and "has no Hugging Face credentials". Both were false at release time and are corrected.

Fixed

  • Router defect — "Where are the built-up areas in this image?" collapsed to vqa and answered "River". Fixed in the dispatch path and validated across three independent live passes (8/8 each). RESOLVED.
  • Live-validation harness false positive — synthetic CDP key events were silently dropped when the window lacked OS focus, so the harness dispatched the page's default query and recorded a false pass. Fixed with pre-dispatch form-state assertions. RESOLVED.
  • Bundled EO change pair — the demo pair differed in shape (726² vs 736²) and errored. Replaced with a same-shape pair (both 720×720). RESOLVED (supersedes FINAL_DELIVERY_REPORT.md §6).
  • B-01 (Hugging Face header link) — previously BLOCKED; the link is now live on all 11 pages. CLOSED.
  • Metric-verifier dotted-path bug — the resolver split the artifact key "0.50" on its own dot, producing a false NOKEY for the two grounding recall claims. Fixed to prefer the longest matching key at each step.

Known issues at this release

  • B-07 — transient tunnel-agent gaps; a request can hang or return 504. Patch prepared, NOT deployed. OPEN.
  • B-02 — /api/health codespace_name carries a trailing \n. Cosmetic. OPEN.
  • No LICENSE file exists. OPEN.
  • No system-level end-to-end benchmark exists. NOT RUN.
  • Router test split never scored. NOT RUN.
  • Calibration worsened ECE (0.013755 → 0.014929). Retained as part of the frozen config only.
  • VLM adapter is ACCEPTANCE-REJECTED despite usable metrics; the deployed path uses the unadapted model.
  • optical-SAR and change-VQA rulings remain OPEN.

Deployment revisions at this release

Component Revision
Frontend 2d7ae53b482d
Backend 89d80eaddec5
Inference 5a0936ace491
Public umbrella 3dcabd32da41

[0.9.0] — 2026-09-25 — Router-defect sprint

  • Diagnosed and fixed the router misroute (region queries → vqa): the temporal slot in interpret() now derives from changeStem / pairWording / newAsChange; built was dropped as a change marker; the quantifier area was tightened to \barea\b.
  • Deployed three commits to the frontend.
  • Ran 3 independent live validation passes, 8/8 each — 24 live runs, 24 correct dispatches, 0 mock nodes, trace fill 94.4444 %.
  • Wrote the delivery report and the handoff.
  • Prepared (but did not deploy) the B-07 backend patch.

[0.8.0] — 2026-09-24/25 — Deployment hardening

  • Live topology settled on Cloudflare Pages → Render → outbound tunnel → Codespace (CPU).
  • Closed the five historical backend blockers (ASGI stack in requirements; $PORT read; OPTIONS preflight; fail-fast app construction; adapter integrity compared against an expected digest).
  • Documented cold start; recorded the transport_mode: auto fallthrough.

[0.7.0] — Grounding and change heads

  • Grounding: trainable head over frozen RemoteCLIP ViT-B/32. Resolution 224 chosen, 448 REJECTED by a pre-registered paired test (mean best IoU −0.0147, t = −22.63).
  • Change: STANet-style Siamese detector on LEVIR-CD-256 — the only VERIFIED headline metric (pooled IoU 0.8122 / macro IoU 0.8457 / pooled F1 0.8964).
  • Optical-SAR: CROMA-base fusion head, armA/armB × seeds 100–104 sweep, pre-registered 115-class metric (accuracy 0.931 / macro-F1 0.434161, ruling OPEN).

[0.6.0] — VLM adaptation

  • SmolVLM-500M LoRA adapter trained (PEFT 0.19.1, r = 16, α = 32, dropout 0.05, fp16).
  • Metrics usable (exact_match 0.963, F1 0.96432) but ACCEPTANCE-REJECTED for promotion. REJECTED.
  • Findings F5-1/F5-2/F5-3 recorded (loader class, ~17× processor overrun, chat-template requirement).

[0.5.0] — Router

  • 50,822-parameter adapter over frozen MiniLM; cached embeddings; CPU-only training (0.28 s for 20 epochs / 4,096 vectors).
  • Group-based splits with hard negatives placed in the test split.
  • Deterministic lexical fallback with ordered precedence and a self_check() against curated hard negatives.

[0.1.0] — Foundation

  • Monorepo, frozen configuration registry (configs/base.yaml), config-hash freeze (78f1e3700da15aa1), nine-state controller, four-endpoint inference contract, evidence schema (Region / ChangeRegion / Evidence), eight frontend execution events.

Unreleased / open

Item State
Deploy the B-07 tunnel-fallthrough fix prepared, NOT deployed
Score the router test split NOT RUN
Build a system-level end-to-end benchmark NOT RUN
Select a licence OPEN
Resolve the optical-SAR and change-VQA rulings OPEN
Plot the calibrated reliability curve NOT RUN
Replace the illustrative benchmark adapter numbers with real runs NOT RUN

Documentation inventory shipped in 1.0.0

File Purpose
README.md repository front page
MODEL_CARD.md model card for the six trained artifacts
docs/ARCHITECTURE.md + docs/architecture/01…10 the deep architecture reference
docs/MODELS.md model detail, hyperparameters, acceptance status
docs/BENCHMARKS.md every measured metric with its artifact and protocol
docs/DATASETS.md dataset provenance, splits, leakage controls
docs/TRAINING.md training procedures and hyperparameters
docs/EVALUATION.md evaluation protocols and honesty rules
docs/DEPLOYMENT.md live topology, env vars, mechanics, traps
docs/REPRODUCIBILITY.md what can and cannot be reproduced
docs/LIMITATIONS.md the full limitation catalogue
docs/RESEARCH_NOTES.md engineering findings and negative results
docs/CHANGELOG.md this file
models/manifest.json, models/checksums.sha256 generated artifact identities