Add labels, limitations and indicator construction to model card
Browse files
README.md
CHANGED
|
@@ -20,13 +20,20 @@ A fine-tuned version of [microsoft/deberta-v3-base](https://huggingface.co/micro
|
|
| 20 |
|
| 21 |
It was developed to build the **Executive Climate Change Attention (ECCA)** indicator, introduced in Grossman and Jan (2026), *Global Environmental Politics*. The indicator measures the share of climate-related sentences in the annual speeches that governments of 195 countries have delivered at the UNGA since 1985.
|
| 22 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 23 |
## Definition
|
| 24 |
|
| 25 |
A sentence is coded as climate-related when it explicitly refers to anthropogenic climate change, including its causes and consequences, related international and national events, and policies addressing it. Broader references to environmental degradation or energy policy (e.g. renewable energy) are not coded as climate-related unless they mention climate change. Extreme weather events are coded only when attributed to climate change.
|
| 26 |
|
| 27 |
## Training data
|
| 28 |
|
| 29 |
-
5,600 manually annotated sentences from the
|
| 30 |
|
| 31 |
## How to use
|
| 32 |
|
|
@@ -38,6 +45,16 @@ classifier("Climate change poses a fundamental threat to our future.")
|
|
| 38 |
# [{'label': 'climate', 'score': 0.9988}]
|
| 39 |
```
|
| 40 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 41 |
## Evaluation
|
| 42 |
|
| 43 |
Results on the held-out evaluation set:
|
|
@@ -54,6 +71,9 @@ Results on the held-out evaluation set:
|
|
| 54 |
|
| 55 |
The model outperforms a random forest classifier and several keyword dictionaries (see the paper, Figure 1).
|
| 56 |
|
|
|
|
|
|
|
|
|
|
| 57 |
### Training hyperparameters
|
| 58 |
|
| 59 |
- learning_rate: 2e-05
|
|
@@ -84,6 +104,8 @@ The model outperforms a random forest classifier and several keyword dictionarie
|
|
| 84 |
- Datasets 2.6.0
|
| 85 |
- Tokenizers 0.15.2
|
| 86 |
|
|
|
|
|
|
|
| 87 |
## Data and replication
|
| 88 |
|
| 89 |
The ECCA indicator and replication materials are available at [github.com/malojan/executive_climate_change_attention](https://github.com/malojan/executive_climate_change_attention).
|
|
|
|
| 20 |
|
| 21 |
It was developed to build the **Executive Climate Change Attention (ECCA)** indicator, introduced in Grossman and Jan (2026), *Global Environmental Politics*. The indicator measures the share of climate-related sentences in the annual speeches that governments of 195 countries have delivered at the UNGA since 1985.
|
| 22 |
|
| 23 |
+
## Labels
|
| 24 |
+
|
| 25 |
+
The model returns one of two labels for each sentence:
|
| 26 |
+
|
| 27 |
+
- `climate`: the sentence is about climate change
|
| 28 |
+
- `other`: it is not
|
| 29 |
+
|
| 30 |
## Definition
|
| 31 |
|
| 32 |
A sentence is coded as climate-related when it explicitly refers to anthropogenic climate change, including its causes and consequences, related international and national events, and policies addressing it. Broader references to environmental degradation or energy policy (e.g. renewable energy) are not coded as climate-related unless they mention climate change. Extreme weather events are coded only when attributed to climate change.
|
| 33 |
|
| 34 |
## Training data
|
| 35 |
|
| 36 |
+
5,600 manually annotated sentences from the [UN General Debate Corpus](https://doi.org/10.1177/2053168017712821) (Baturo, Dasandi and Mikhaylov 2017), which contains the English versions of the speeches. Intercoder reliability: Krippendorff's α = 0.795.
|
| 37 |
|
| 38 |
## How to use
|
| 39 |
|
|
|
|
| 45 |
# [{'label': 'climate', 'score': 0.9988}]
|
| 46 |
```
|
| 47 |
|
| 48 |
+
## From sentences to the ECCA indicator
|
| 49 |
+
|
| 50 |
+
The ECCA indicator is the share of sentences in a speech that the model classifies as climate-related. To reproduce it, split each speech into sentences, classify every sentence, and compute the proportion labelled `climate`.
|
| 51 |
+
|
| 52 |
+
## Limitations
|
| 53 |
+
|
| 54 |
+
- The model was trained and evaluated on English sentences from UNGA speeches. Its performance on other kinds of text, such as manifestos, parliamentary debates or social media posts, has not been tested.
|
| 55 |
+
- It classifies single sentences. Longer texts should be split into sentences first; inputs are truncated at 512 tokens.
|
| 56 |
+
- It identifies explicit references to climate change only, following the definition above. Implicit references, such as energy or environmental policy that does not mention climate change, are classified as `other` by design.
|
| 57 |
+
|
| 58 |
## Evaluation
|
| 59 |
|
| 60 |
Results on the held-out evaluation set:
|
|
|
|
| 71 |
|
| 72 |
The model outperforms a random forest classifier and several keyword dictionaries (see the paper, Figure 1).
|
| 73 |
|
| 74 |
+
<details>
|
| 75 |
+
<summary>Training details</summary>
|
| 76 |
+
|
| 77 |
### Training hyperparameters
|
| 78 |
|
| 79 |
- learning_rate: 2e-05
|
|
|
|
| 104 |
- Datasets 2.6.0
|
| 105 |
- Tokenizers 0.15.2
|
| 106 |
|
| 107 |
+
</details>
|
| 108 |
+
|
| 109 |
## Data and replication
|
| 110 |
|
| 111 |
The ECCA indicator and replication materials are available at [github.com/malojan/executive_climate_change_attention](https://github.com/malojan/executive_climate_change_attention).
|