--- license: apache-2.0 base_model: Qwen/Qwen3.5-0.8B library_name: transformers pipeline_tag: image-text-to-text language: - en tags: - remote-sensing - sentinel-2 - multispectral - earth-observation - visual-question-answering - calibration - decision-model - system-one datasets: - BIFOLD-BigEarthNetv2-0/BigEarthNet.txt --- # EarthJev: typed answers about multispectral Sentinel-2 patches, without generating text **Domenico Benfenati, Antonio Maria Rinaldi, Cristian Tommasino** — Department of Electrical Engineering and Information Technology (DIETI), University of Naples Federico II. Code: [https://github.com/CosmoIknosLab/EarthJev](https://github.com/CosmoIknosLab/EarthJev) · Paper: [arXiv:2611.02096](https://arxiv.org/abs/2611.02096).

EarthJev framework

EarthJev answers typed questions about a Sentinel-2 L2A patch from **all 12 bands**. Each question declares its options (yes/no, one of *K* options, an ordered level), and the model returns a **probability for every option**, read from the logits of the option letters at a single position: no token is generated, so every answer is a valid option, and questions about the same patch share one encoding of the image. - **Backbone:** [Qwen3.5-0.8B](https://huggingface.co/Qwen/Qwen3.5-0.8B), fully fine-tuned with the open [OneJev](https://github.com/OmniJev/OneJev) recipe (cross-entropy + Brier on the option letters, options permuted, one epoch, learning rate 5e-6). - **12-band input:** the input layer of the vision encoder is widened from 3 to 12 channels (R, G, B kernels copied onto B04, B03, B02; the other bands start at zero) and is the only trained part of the encoder. - **This checkpoint:** seed 0 of three, chosen on validation accuracy (85.63% on 36,698 validation questions). Temperatures per question type and number of options, fitted on the validation questions (`calibration.json`): choice:15 1.027, choice:2 0.991, choice:4 1.136, choice:5 1.025, noul:2 1.024. ## Results All models trained on the same 70,890 CORINE questions from 5% of BigEarthNet v2.0 (three seeds) and timed in one job on one H100 PCIe GPU (batch 1, bfloat16; each timer runs from the decoded image in memory to the answer). CORINE: 36,028 test questions (mean ± std over seeds), ECE = mean of the per-seed ECEs (15 bins). RSVQAxBEN: 29,501 presence questions written by other authors, never used for training. *ms / image*: all the questions of an image (6.1 on average) in one request. | Model | Params | Input | CORINE acc. (%) | ECE | RSVQAxBEN acc. (%) | ms / question | Slower than EarthJev | ms / image | Slower than EarthJev | |---|---|---|---|---|---|---|---|---|---| | **EarthJev (this model)** | **0.8B** | **12 bands** | **86.0 ± 0.1** | **0.006** | **81.0** | **19.9** | — | **42.8** | — | | Qwen3.5-0.8B, generative | 0.8B | 12 bands | 86.8 ± 0.2 | 0.005 | 80.5 | 134.7 | 6.8× | 191.9 | 4.5× | | Qwen3-VL-2B, LoRA | 2.1B | RGB | 86.6 ± 0.1 | 0.005 | 80.6 | 111.6 | 5.6× | 176.2 | 4.1× | | InternVL3-2B, LoRA | 2.1B | RGB | 87.1 ± 0.0 | 0.006 | 81.2 | 84.5 | 4.3× | 157.3 | 3.7× | | EarthDial-4B, LoRA | 4.2B | RGB | 89.5 ± 0.1 | 0.006 | 82.4 | 171.8 | 8.7× | 431.6 | 10.1× | | ResNet-50 VQA* | 0.14B | 12 bands | 87.5 ± 0.2 | 0.089 | 79.4 | 12.1 | 0.6× | 14.9 | 0.3× | | Swin-T VQA* | 0.15B | 12 bands | 86.5 ± 0.1 | 0.047 | 79.6 | 16.2 | 0.8× | 17.9 | 0.4× | | MambaVision-T VQA* | 0.15B | 12 bands | 85.9 ± 0.3 | 0.064 | 78.4 | 15.3 | 0.8× | 18.6 | 0.4× | \* Fixed-vocabulary VQA models: a reference for the cost of a model that answers only the questions and options it was trained on. Replacing the nine non-RGB bands with their training mean costs EarthJev 4.5 points on CORINE. ## Usage Requirements: `transformers` with Qwen3.5 support, `torch`, and OneJev's engine (`pip install "qev @ git+https://github.com/OmniJev/OneJev"`). From a clone of this repository: ```python import numpy as np from earthjev_infer import EarthJev model = EarthJev.load(".") # this folder (downloaded with huggingface_hub.snapshot_download) s2 = np.load("examples/S2A_MSIL2A_20170701T093031_N9999_R136_T35VPK_37_55.npy") # [12, 120, 120] uint16 answers = model.predict(s2, { "dominant": {"type": "choice", "instructions": "Which land-cover class covers most of this patch?", "criteria": {"artificial": "Artificial surfaces.", "agricultural": "Agricultural areas.", "forest": "Forest and seminatural areas.", "wetlands": "Wetlands.", "water": "Water bodies."}}, "urban": {"type": "noul", "instructions": "Is there any urban fabric in this patch?"}, }) print(answers["dominant"]) # a probability for every option ``` The input is an array `[12, H, W]` with the Sentinel-2 L2A bands B01 B02 B03 B04 B05 B06 B07 B08 B8A B09 B11 B12 at 10 m (20 m and 60 m bands resampled by pixel repetition), digital numbers as distributed (reflectance × 10,000), as in BigEarthNet v2.0. `examples/` holds two test patches (7 questions, 7 questions) with their CORINE test questions and an RGB preview. ## Intended use and limitations - Research on question answering and decision models for remote sensing. Not validated for operational use. - Trained on 11,893 patches of 120 × 120 pixels from ten European countries (2017–2018) and on questions generated from CORINE Land Cover (minimum mapping unit 25 ha); accuracy outside this distribution, on other sizes, products (e.g. L1C) or question types is not established. On BigEarthNet.txt questions, whose types differ from the training questions, accuracy is about 60% on binary and 40% on four-option questions. - Only questions with declared options: open requests ("describe this image") are outside the model's interface. ## Training data and licenses Weights: Apache-2.0 (base model Qwen3.5-0.8B, Apache-2.0). Training and test data: BigEarthNet v2.0 (reBEN) and BigEarthNet.txt (CDLA-Permissive-1.0; labels derived from CORINE Land Cover, © European Union, Copernicus Land Monitoring Service); RSVQAxBEN (CC-BY-4.0) for testing. The example patches come from the reBEN test split. ## Citation ```bibtex @article{benfenati2026earthjev, title = {{EarthJev}: Calibrated, Non-Generative Answers to Natural-Language Questions about {Sentinel-2} Land Cover}, author = {Benfenati, Domenico and Rinaldi, Antonio Maria and Tommasino, Cristian}, journal = {arXiv preprint arXiv:2611.02096}, year = {2026}, url = {https://arxiv.org/abs/2611.02096} } ``` This study was partially supported by the PNRR MUR project PE0000013-FAIR.