legal-colbert-extractor

A small, open late-interaction (ColBERT) retriever optimized for extracting clauses from real, filed commercial contracts — assignment, termination (and its subtypes: for cause, for convenience, upon change of control…), notices, cure periods, indemnification, confidentiality, limitation of liability, governing law, and ~50 other provision subtypes. 149M parameters (ModernBERT-base backbone), 128-d token vectors, MaxSim scoring. Runs comfortably on CPU.

This is the real-contract sibling of legal-colbert-clause-retriever. The two share a lineage but are selected on different gates: the clause-retriever maximizes the public MLEB template-clause benchmark; this model maximizes retrieval quality on SEC-filed contract text with human-adjudicated relevance labels. If you are pulling clauses out of actual contracts, use this one; if you are matching against idealized clause-library text, use the sibling.

Results

Primary gate — a blind evaluation over 78 fresh SEC EDGAR Exhibit-10 contracts (830 passages, 15 clause-type queries, 171 relevance labels adjudicated independently three ways; zero overlap with any training data):

Model Adjudicated blind NDCG@10 58-subtype discrimination
legal-colbert-extractor 0.7781 0.6968
legal-colbert-clause-retriever (sibling) 0.7683 0.6911
earlier best (P6b) 0.7642 0.6510

On MLEB Contractual Clause Retrieval it scores 0.8298 NDCG@10 (the sibling scores 0.8374 — that benchmark is the sibling's selection gate, not this model's).

Query phrasing matters

Trained on rigid clause-type queries and evaluated with descriptive definitions. Phrase queries as full descriptions of the provision you want:

  • ✅ "Find contractual provisions governing termination rights, termination for cause, or the consequences of ending the agreement."
  • ✅ "Find provisions related to 'Termination for Convenience'."
  • ⚠️ "termination clause" — bare keywords underperform.

Queries are truncated at 48 tokens; documents at 300 tokens (chunk accordingly, ~180–220 words).

Extraction pattern (many contracts)

Index all contract chunks once, run one definition query per clause type, then group hits by document and take top-k within each document — for extraction you want the best candidate in every contract, not a global leaderboard.

from pylate import models
model = models.ColBERT("kmad00/legal-colbert-extractor")
q = model.encode(["Find provisions related to 'Termination for Cause'."], is_query=True)
d = model.encode(chunks, is_query=False)  # your contract chunks

Training

Continuation fine-tune lineage from lightonai/GTE-ModernColBERT-v1 through the clause-retriever line, then two extractor-specific rounds: (1) 62 clause subtypes mined from the full LEDGAR corpus (SEC EDGAR provisions with section-heading labels) plus FAR Part 52 public-domain clauses, (2) within-family and model-mined hard negatives (sibling-subtype confusables the incumbent model actually mis-ranked), contrastive training in very light doses (≤300 steps, lr ≤1e-7) with a CUAD anchor. Selection required improvement on the adjudicated blind gate with no MLEB regression below 0.82.

Limitations

  • English-language, US-style commercial contracts.
  • Multi-vector storage is heavier per document than single-vector embedders.
  • Slightly weaker than the sibling on idealized template-clause text (that's the trade).
  • Clause types far outside the training taxonomy get lower confidence.

License

CC BY 4.0. Training data: CUAD (CC BY 4.0), LEDGAR (SEC EDGAR public filings), ACORD (CC BY 4.0), FAR Part 52 (US public domain).

Downloads last month
7
Safetensors
Model size
0.1B params
Tensor type
F32
·
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for kmad00/legal-colbert-extractor

Datasets used to train kmad00/legal-colbert-extractor