Spaces:
Paused
Paused
File size: 4,276 Bytes
0127c21 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 | # Remote-sensing adaptation pipeline (Phase 5)
Satisfies the PS-mandatory requirement: *"At least one visual or
vision-language component must be fine-tuned or otherwise adapted using
BigEarthNet.txt or the any open source training data."*
## Pipeline
```
BigEarthNet.txt manifest ──▶ prepare_bigearthnet.py ──▶ *_vqa_train.jsonl
│
▼
train_lora_vqa.py
│
▼
backend/models/checkpoints/vqa_lora_bigearthnet/
│
▼
backend/models/vqa.py loads it
automatically on next restart
```
## Status: scripts are real and runnable — no adapter is checked in
`backend/models/checkpoints/vqa_lora_bigearthnet/` is empty in this
repo. Until someone actually runs `train_lora_vqa.py`, the backend
correctly reports `adapted: false` in every VQA response and uses base
`Salesforce/blip-vqa-base` weights. **This is the honest state, not a
bug** — do not check in a fabricated/untrained adapter directory just to
flip that flag.
## Before running this for real
1. **Get the actual PS-linked dataset** (arxiv.org/abs/2603.29630) and
inspect its real manifest format. `prepare_bigearthnet.py`'s expected
CSV/JSONL columns (`caption` or `labels`/`label_1..n`) are a
reasonable guess based on the PS description, not a verified schema
— update the parsing in that file to match what you actually
download before trusting its output.
2. Decide compute: LoRA fine-tuning of BLIP-VQA-base is realistic on a
single consumer GPU in well under a day for a few thousand examples;
CPU-only will be much slower — cap `--limit` accordingly in
`prepare_bigearthnet.py`.
3. Install training-only deps (`pip install peft accelerate datasets`)
— these are deliberately NOT in `backend/requirements.txt` since the
running API doesn't need `datasets`/`accelerate`, only `peft` at
inference time to load the resulting adapter.
## Phase 12 — what was actually verified this session (and what wasn't)
Attempted, in the sandbox used to build Phase 12: install `torch`/`transformers`/`peft`
and actually run this pipeline end-to-end. Result, stated plainly:
- **`prepare_bigearthnet.py` was smoke-tested for real** against a small hand-built,
clearly-synthetic manifest (5 rows, random-noise PNGs, invented captions/labels —
nothing from the actual BigEarthNet.txt dataset). It ran cleanly and produced a
correctly-shaped `vqa_train.jsonl`. This proves the CSV/JSONL parsing logic and
the templated-question generation work as written — it does **not** prove
`prepare_bigearthnet.py`'s column-name assumptions (`caption` / `labels` /
`label_1..n`) match the real PS-linked dataset's actual schema, because that
dataset was never fetched (see below). Verify that for real before trusting this
script's output on the real data.
- **`train_lora_vqa.py` was NOT run**, even on the synthetic data above, and could
not be in this environment: (1) the dataset host (arxiv.org and whatever hosts
the actual BigEarthNet.txt files) is not reachable from the sandbox this Phase
was built in, and (2) `huggingface.co` — needed to download the base
`Salesforce/blip-vqa-base` weights — is also not reachable from it. Both are
infrastructure constraints of that particular build environment, not a flaw in
the script. `backend/models/checkpoints/vqa_lora_bigearthnet/` remains empty;
`adapted: false` remains the honest, correct state.
**Bottom line, unchanged from every prior phase:** running this for real — with the
actual PS-linked dataset and a machine that can reach huggingface.co — is still an
execution task for whoever has that access, not something that can be faked here.
Do not check in a checkpoint that wasn't actually produced by a real training run.
|