Text Classification
Safetensors
English
roberta
Files changed (1) hide show
  1. README.md +42 -0
README.md ADDED
@@ -0,0 +1,42 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: apache-2.0
3
+ base_model: climatebert/distilroberta-base-climate-f
4
+ datasets:
5
+ - climatebert/netzero_reduction_data
6
+ language:
7
+ - en
8
+ metrics:
9
+ - accuracy
10
+ - f1
11
+ pipeline_tag: text-classification
12
+ ---
13
+
14
+ # PledgeGrader ClimateBERT
15
+
16
+ A fine tuned text classifier that labels a single sentence from a corporate or public climate document as none, net zero or reduction. It is the ClimateBERT arm of PledgeGrader, a project for UTS 36118 Applied Natural Language Processing (Spring 2026, Group 14). Application: https://huggingface.co/spaces/vinodnair207/pledgegrader. Code: https://github.com/vinodnair207/ANLP_AT2_PledgeGrader.
17
+
18
+ ## Base model and data
19
+
20
+ The model was fine tuned from climatebert/distilroberta-base-climate-f (Apache 2.0), which Webersinke et al. (2022) describe as further pretrained on climate related text. The training data is the training split of climatebert/netzero_reduction_data (Apache 2.0), 2,408 of its 3,441 sentences. The split is a stratified 70/15/15 division with seed 42, made by the project and stored in the repository.
21
+
22
+ ## Labels
23
+
24
+ Class codes are 0 = none (no emissions target), 1 = net zero (a commitment to reach net zero or carbon neutrality) and 2 = reduction (a stated cut in emissions without a net zero commitment).
25
+
26
+ ## Training settings
27
+
28
+ 4 epochs, batch size 16, learning rate 0.00003, maximum input length 128 tokens (longer inputs are truncated), seed 42. The epoch with the best validation macro F1 was kept.
29
+
30
+ ## Results
31
+
32
+ Scored once on the 517 sentence held out test split: accuracy 0.9613, macro precision 0.9555, macro recall 0.9553, macro F1 0.9554. For comparison, on the same sentences a rule based baseline scored macro F1 0.9137 and a zero shot Gemini model scored 0.9426. The difference between this model and the Gemini model is not statistically significant (paired bootstrap p value 0.31).
33
+
34
+ ## Intended use and limits
35
+
36
+ The model is a screening aid that classifies the target type of English sentences. It was trained and tested on single sentences, so results on whole pledges are not measured. About half of the disagreements between methods in the project traced to ambiguous or doubtful labels, especially sentences that state both a net zero and a reduction goal. The probability it returns has not been checked for calibration. It does not judge whether a company will meet a target, and its output should be verified against the source text before it is cited.
37
+
38
+ ## Citation
39
+
40
+ Schimanski, T., Bingler, J., Kraus, M., Hyslop, C., and Leippold, M. (2023). ClimateBERT-NetZero: Detecting and assessing net zero and reduction targets. Proceedings of EMNLP 2023. https://doi.org/10.18653/v1/2023.emnlp-main.975
41
+
42
+ Webersinke, N., Kraus, M., Bingler, J. A., and Leippold, M. (2022). ClimateBert: A pretrained language model for climate related text. arXiv:2110.12010.