File size: 2,047 Bytes
0d263cb | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 | ---
license: apache-2.0
library_name: transformers
tags: [laya, text-classification, multi-label, climate, vulnerability, rlcd]
pipeline_tag: text-classification
---
# Laya fine-tuned for climate-vulnerability group detection, v2 (multi-label)
[convaiinnovations/laya](https://huggingface.co/convaiinnovations/laya) (421M, ModernBERT-large backbone)
fine-tuned with the official RLCD recipe on [GIZ/vulnerability_training_data_full](https://huggingface.co/datasets/GIZ/vulnerability_training_data_full)
(380 train rows x 17 binary vulnerability-group questions; 36 all-negative rows included as negatives).
**v2 change vs [v1](https://huggingface.co/peter2000/laya-vulnerability-groups):** the typed-question instructions now carry the
label into context — each binary question reads "Does this text indicate that <group> are targeted, supported, or affected? This group is specifically vulnerable to climate change. Answer true or false."
Each vulnerability group is asked as one binary (`noul`) typed question; all 17 are answered in a single forward pass.
Evaluate with `laya.load("peter2000/laya-vulnerability-groups-v2")` and `agent.predict(state, questions)`.
## Test-set metrics (held-out 95 rows, threshold 0.5)
| metric | value |
|---|---|
| macro-F1 | 0.7038 |
| micro-F1 | 0.7083 |
| ECE | 0.0242 |
| subset accuracy | 0.4737 |
Per-label F1:
| label | F1 |
|---|---|
| Agricultural communities | 0.9167 |
| Coastal communities | 0.5714 |
| Ethnic, racial or other minorities | 0.6000 |
| Fishery communities | 0.4000 |
| Informal sector workers | 1.0000 |
| Members of indigenous and local communities | 0.9333 |
| Migrants and displaced persons | 0.6667 |
| Older persons | 0.7778 |
| Other | 0.0000 |
| Persons living in poverty | 0.6087 |
| Persons with disabilities | 1.0000 |
| Persons with pre-existing health conditions | 1.0000 |
| Residents of drought-prone regions | 0.8000 |
| Rural populations | 0.8889 |
| Sexual minorities (LGBTQI+) | 0.3333 |
| Urban populations | 0.5455 |
| Women and other genders | 0.9231 |
|