File size: 3,517 Bytes
0d43549 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 | ---
license: apache-2.0
tags:
- legal
- bert
- legal-bert
- contract-risk
- cuad
- pytorch
pipeline_tag: text-classification
library_name: transformers
---
# RiskLoop Representative Model Checkpoints
This repository hosts the **reproduced representative model checkpoints** for [RiskLoop](https://github.com/aayushiii18/RiskLoop), an NLP system for automated contract risk detection based on Legal-BERT fine-tuning on the CUAD (Contract Understanding Atticus Dataset) benchmark.
> [!IMPORTANT]
> **Provenance & Reproduction Note:**
> These checkpoints are **reproduced representative checkpoints** created under strictly controlled, frozen training conditions (Condition A) and evaluated in RiskLoop's frozen Phase 6 test evaluation.
> They are **NOT** the original historical official checkpoints from earlier development phases, as those original historical binaries were lost. These reproduced models represent the exact frozen representative checkpoints used to generate the Phase 6 decision analysis and final test metrics.
---
## 1. Model Architecture & Training Details
- **Backbone Model:** `nlpaueb/legal-bert-base-uncased`
- **Model Type:** Single-task span classification models (QA-style start/end logit heads over transformer sequence outputs).
- **Training Strategy:** Condition A (single-task fine-tuning with fixed seeds, 4 epochs, max sequence length 512, document stride 256).
---
## 2. Model Checkpoint Registry & Integrity Hashes
The repository contains three frozen model binary files (`.pt` PyTorch state dictionaries):
| Checkpoint Filename | Target Contract Task | Experimental Condition | Random Seed | File Size (Bytes) | SHA-256 Hash |
| :--- | :--- | :--- | :--- | :--- | :--- |
| `run_03_best_model.pt` | **Cap On Liability** | Condition A | `44` | 438,001,498 | `8689ef00a0718713f4ae8bdf8b48d3441b5d9a2ccd13ae322185862551c6f674` |
| `run_04_best_model.pt` | **Anti-Assignment** | Condition A | `42` | 438,001,498 | `ea6a75254798342e0cc3925dd8adca1ba9b5bb60c6a631ddf14057748c01947c` |
| `run_07_best_model.pt` | **Termination For Convenience** | Condition A | `42` | 438,001,498 | `a8a28902ef3dbc28e706135e8599f01cfa8c0e9c4c05cc081818e3027deda119` |
---
## 3. Frozen Phase 6 Test Evaluation Metrics
The reproduced representative checkpoints achieved the following metrics on the held-out frozen CUAD test evaluation set (Phase 6):
| Task Name | Representative Checkpoint | Test F1 Score | Test ROC-AUC | Test PR-AUC |
| :--- | :--- | :--- | :--- | :--- |
| **Cap On Liability** | `run_03_best_model.pt` | `0.7727273` | `0.9960492` | `0.9147721` |
| **Anti-Assignment** | `run_04_best_model.pt` | `0.8083624` | `0.9944473` | `0.9146714` |
| **Termination For Convenience** | `run_07_best_model.pt` | `0.7483871` | `0.9939074` | `0.7926948` |
---
## 4. Inference Score Interpretation
In RiskLoop's inference engine and Streamlit demo interface:
- Output scores are **uncalibrated raw logit deltas** ($\text{logit}_1 - \text{logit}_0$).
- High positive scores indicate strong model activation for clause presence.
- Scores are **not** calibrated probabilities or confidence percentages.
---
## 5. Usage & Legal Disclaimer
- **Intended Use:** Portfolio evaluation, academic research, and interactive open-source demonstration of contract risk detection.
- **Legal Disclaimer:** This software and model predictions do **NOT** constitute legal advice, formal contract audit, or legal guarantee. Users should consult qualified legal professionals for actual contract review.
|