Doan16/pledgegrader-climatebert

PledgeGrader ClimateBERT

A fine tuned text classifier that labels a single sentence from a corporate or public climate document as none, net zero or reduction. It is the ClimateBERT arm of PledgeGrader, a project for UTS 36118 Applied Natural Language Processing (Spring 2026, Group 14). Application: https://huggingface.co/spaces/vinodnair207/pledgegrader. Code: https://github.com/vinodnair207/ANLP_AT2_PledgeGrader.

Base model and data

The model was fine tuned from climatebert/distilroberta-base-climate-f (Apache 2.0), which Webersinke et al. (2022) describe as further pretrained on climate related text. The training data is the training split of climatebert/netzero_reduction_data (Apache 2.0), 2,408 of its 3,441 sentences. The split is a stratified 70/15/15 division with seed 42, made by the project and stored in the repository.

Labels

Class codes are 0 = none (no emissions target), 1 = net zero (a commitment to reach net zero or carbon neutrality) and 2 = reduction (a stated cut in emissions without a net zero commitment).

Training settings

4 epochs, batch size 16, learning rate 0.00003, maximum input length 128 tokens (longer inputs are truncated), seed 42. The epoch with the best validation macro F1 was kept.

Results

Scored once on the 517 sentence held out test split: accuracy 0.9613, macro precision 0.9555, macro recall 0.9553, macro F1 0.9554. For comparison, on the same sentences a rule based baseline scored macro F1 0.9137 and a zero shot Gemini model scored 0.9426. The difference between this model and the Gemini model is not statistically significant (paired bootstrap p value 0.31).

Intended use and limits

The model is a screening aid that classifies the target type of English sentences. It was trained and tested on single sentences, so results on whole pledges are not measured. About half of the disagreements between methods in the project traced to ambiguous or doubtful labels, especially sentences that state both a net zero and a reduction goal. The probability it returns has not been checked for calibration. It does not judge whether a company will meet a target, and its output should be verified against the source text before it is cited.

Citation

Schimanski, T., Bingler, J., Kraus, M., Hyslop, C., and Leippold, M. (2023). ClimateBERT-NetZero: Detecting and assessing net zero and reduction targets. Proceedings of EMNLP 2023. https://doi.org/10.18653/v1/2023.emnlp-main.975

Webersinke, N., Kraus, M., Bingler, J. A., and Leippold, M. (2022). ClimateBert: A pretrained language model for climate related text. arXiv:2110.12010.