Vedant0824/legal-contract-clause-analyzer

⚖️ Legal Contract Clause Analyzer

Fine-tuned Qwen2.5-3B-Instruct for legal contract clause classification (41 types) and plain-English rewriting using QLoRA + Unsloth on the CUAD dataset.

Model Description

• Base Model: Qwen2.5-3B-Instruct (3.09B parameters)
• Fine-Tuning Method: QLoRA (4-bit quantization + LoRA adapters)
• Framework: Unsloth + TRL SFTTrainer
• Training Hardware: Google Colab T4 GPU (free tier)
• Training Time: 60 minutes (500 steps)
• Dataset: CUAD (Contract Understanding Atticus Dataset) — 510 commercial contracts, 41 clause types
• Trainable Parameters: 29.9M / 3.1B (0.96%)
• Final Training Loss: 0.52

What This Model Does

Given any legal contract text, the model performs 5 tasks simultaneously:

1. Clause Classification (NLU): Identifies the clause type from 41 categories
2. Clause Detection (NLU): Determines if the clause is present or not
3. Text Extraction (NLU): Extracts the relevant legal text
4. Risk Assessment (NLU): Rates risk as HIGH, MEDIUM, or LOW
5. Plain English (NLG): Explains the clause in simple language

41 Supported Clause Types

Document Name, Parties, Agreement Date, Effective Date, Expiration Date, Renewal Term, Notice Period To Terminate Renewal, Governing Law, Most Favored Nation, Non-Compete, Exclusivity, No-Solicit Of Customers, Competitive Restriction Exception, No-Solicit Of Employees, Non-Disparagement, Termination For Convenience, Rofr/Rofo/Rofn, Change Of Control, Anti-Assignment, Revenue/Profit Sharing, Price Restrictions, Minimum Commitment, Volume Restriction, Ip Ownership Assignment, Joint Ip Ownership, License Grant, Non-Transferable License, Affiliate License-Licensor, Affiliate License-Licensee, Unlimited/All-You-Can-Eat-License, Irrevocable Or Perpetual License, Source Code Escrow, Post-Termination Services, Audit Rights, Uncapped Liability, Cap On Liability, Liquidated Damages, Warranty Duration, Insurance, Covenant Not To Sue, Third Party Beneficiary.

Example Output

Input:

This Agreement shall be governed by and construed in accordance with 
the laws of the State of Delaware, without regard to its conflict of 
laws principles.

Output:

{
  "clause_type": "Governing Law",
  "found": true,
  "relevant_text": "This Agreement shall be governed by and construed in accordance with the laws of the State of Delaware...",
  "key_terms": ["governing law"],
  "risk_level": "LOW",
  "plain_english": "This specifies which state's laws will be used to interpret the contract if there's a dispute."
}

Training Results

Metric	Value
Training Loss (start)	0.74
Training Loss (final)	0.52
Training Steps	500
Training Time	60 minutes
Peak GPU Memory	4.2 GB / 15 GB
Training Examples	16,270
Validation Examples	500

Dataset

• CUAD (Contract Understanding Atticus Dataset) — NeurIPS 2021
• 510 real commercial contracts from SEC EDGAR
• 13,000+ expert annotations by lawyers
• 41 clause categories

Usage

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base_model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2.5-3B-Instruct")
model = PeftModel.from_pretrained(base_model, "Vedant0824/legal-contract-clause-analyzer")
tokenizer = AutoTokenizer.from_pretrained("Vedant0824/legal-contract-clause-analyzer")

Author

Vedant Gokhe — GitHub | LinkedIn