litillabs/litil-clausetagger-4b

LiTiL ClauseTagger 4B

What this model does

LiTiL ClauseTagger turns a contract passage into one or more machine-readable labels from a contract taxonomy. It is built for passages that have already been found in an agreement and can assign multiple labels when the same language serves more than one function.

In a contract-intelligence stack, place it after document parsing, clause segmentation, or retrieval. Its labels can populate clause indexes, organize reviewer queues, drive category-specific checks, and send each passage to the right extraction or playbook workflow. This gives the rest of the system a consistent vocabulary for working with contract language.

• Useful for: organizing extracted clauses, building review queues, and routing passages for follow-up
• Give it: a passage already found in a contract
• It returns: a short JSON list of category IDs

Model description

Field	Value
Hugging Face repository	litillabs/litil-clausetagger-4b
Base model	Qwen/Qwen3.5-4B at revision 851bf6e806efd8d0a36b00ddf55e13ccb7b8cd0a
Artifact type	PEFT LoRA adapter; the base model is required
Adaptation	LoRA, converted from the evaluated Tinker checkpoint to a PEFT adapter
Input	One contract passage
Output	{"class_ids":[...]} using the 41-label CUAD catalog
Training envelope	2,048 tokens
Recommended decoding	Greedy, max_new_tokens=64, thinking disabled
Developer	LiTiL Labs
Card date	September 11, 2026

The release-preparation artifact is the PEFT directory peft_adapter_exact_v2. Its adapter_model.safetensors is 342,027,648 bytes with SHA-256 6e22265583278cbfc8c6815b98533787d39ebe18b7e68e0dffae12bba2013bd8; adapter_config.json has SHA-256 0f19d28cb66153d1a1d74ec8fe22810660a962c131c8ed4cd45bf68e24c2a95d.

The retained scored checkpoint was a Tinker sampler. The PEFT directory above is its exact tensor conversion and is the supported portable load path.

The repository includes adapter_config.json, adapter_model.safetensors, the 41-label class_catalog.json, and a runner that preserves the prompt and renderer below.

Intended use

Use this model after clause retrieval or document segmentation to:

• attach CUAD categories to passages;
• organize extracted clauses for review;
• route passages to category-specific checks; and
• produce machine-readable labels for contract analytics.

For complete agreements, pair it with a parser or retrieval layer that supplies the passages and preserves their locations in the source document.

Input contract

Send one passage per request. Keep the complete label catalog in the system prompt and preserve the disabled-thinking renderer.

The user message starts with Contract segment:, contains the passage, and ends with Return exactly {"class_ids":[...]} with no other text. Keep the rendered prompt within 1,900 tokens for the validated local runtime.

Output contract

Valid responses contain exactly one key:

{"class_ids":[3,4]}

The IDs must be unique, sorted integers from 0 through 40. An empty list means that the passage does not directly support a catalog category.

Saved example

Input passage:

All terms of this Agreement will automatically commence on November 1st, 2017, and expire on November 2nd, 2020.

Saved adapter output:

{"class_ids":[3,4]}

This maps to Effective Date and Expiration Date. The unchanged base returned IDs 2 and 3 on the same saved case.

Evaluation

The retained Measurement Reset v2 evaluation contains 500 attorney-annotated positive passages from 70 CUAD documents. Both models used the same prompt, renderer, decoding, and parser.

Metric	Qwen3.5-4B base	LiTiL adapter	Change
Macro F1, 41 labels	0.6195	0.7243	+0.1048
Micro F1	0.7160	0.8193	+0.1034
Exact label-set match	63.6%	70.8%	+7.2 points
Valid JSON	100%	100%	—

The paired bootstrap 95% interval for macro-F1 improvement was +0.0711 to +0.1431. The adapter improved 28 labels, matched six, and regressed on seven in this evaluation.

Training

The adapter was trained from a fresh Qwen3.5-4B base with LoRA using Tinker.

Setting	Value
Training examples presented	5,760
Steps	180
Batch size	32
Learning rate	5e-6
LoRA rank	32
Maximum length	2,048 tokens
Seed	20260709
Training time	612.8 seconds

The batch recipe combined public CUAD positive passages, rare-category positives, and constructed negative blocks. The retained provenance binds the inputs to The Atticus Project CUAD revision a3c393f5d103fd0c516374e4fdff676c8176dcb1 and the public CUAD clause-classification revision 2647c92f634569acbdfbd694ad5488293c55b3f7. No private post-training data was identified for this run.

Runtime guidance

The adapter weighs 342,027,648 bytes. The exact converted package was loaded and exercised on Apple MPS with BF16, batch size one, prompts capped at 1,900 tokens, and 64 generated tokens. A 4B BF16 base uses roughly 8 GB for weights before adapter, activations, and KV cache; 12–16 GB of available accelerator or unified memory is a practical starting point for this short-passage workload. Measure memory and latency on the target deployment.

Limitations

• The measured task starts from an already selected passage; whole-contract recall depends on the upstream retrieval or segmentation stage.
• Evaluation covers positive annotated passages. If the application must reject irrelevant passages, measure that path with representative negatives.
• The scores above belong to the evaluated Tinker sampler. The PEFT conversion passed exact tensor and loader checks; run a saved-case replay before treating a newly packaged release as behaviorally identical.

Use the model

Installation

The exact converted package was load-verified with Python 3.11.9, PyTorch 2.8.0, Transformers 5.5.4, PEFT 0.19.1, safetensors 0.6.2, and Tinker Cookbook 0.4.3.

python -m pip install \
  "torch==2.8.0" \
  "transformers==5.5.4" \
  "peft==0.19.1" \
  "safetensors==0.6.2" \
  "tinker-cookbook==0.4.3" \
  accelerate

Loading and inference

The model uses Qwen3.5's AutoModelForImageTextToText loader even though the task input is text-only. This is the loader resolved by the tested runtime. Place the complete peft_adapter_exact_v2 directory beside the script before running this example.

import json
import torch
from peft import PeftModel
from tinker_cookbook.renderers import get_renderer
from transformers import AutoModelForImageTextToText, AutoTokenizer

BASE_ID = "Qwen/Qwen3.5-4B"
BASE_REVISION = "851bf6e806efd8d0a36b00ddf55e13ccb7b8cd0a"
ADAPTER_ID = "litillabs/litil-clausetagger-4b"

LABELS = {
    0: "Document Name", 1: "Parties", 2: "Agreement Date",
    3: "Effective Date", 4: "Expiration Date", 5: "Renewal Term",
    6: "Notice Period To Terminate Renewal", 7: "Governing Law",
    8: "Most Favored Nation", 9: "Non-Compete", 10: "Exclusivity",
    11: "No-Solicit Of Customers", 12: "Competitive Restriction Exception",
    13: "No-Solicit Of Employees", 14: "Non-Disparagement",
    15: "Termination For Convenience", 16: "Rofr/Rofo/Rofn",
    17: "Change Of Control", 18: "Anti-Assignment",
    19: "Revenue/Profit Sharing", 20: "Price Restrictions",
    21: "Minimum Commitment", 22: "Volume Restriction",
    23: "Ip Ownership Assignment", 24: "Joint Ip Ownership",
    25: "License Grant", 26: "Non-Transferable License",
    27: "Affiliate License-Licensor", 28: "Affiliate License-Licensee",
    29: "Unlimited/All-You-Can-Eat-License",
    30: "Irrevocable Or Perpetual License", 31: "Source Code Escrow",
    32: "Post-Termination Services", 33: "Audit Rights",
    34: "Uncapped Liability", 35: "Cap On Liability",
    36: "Liquidated Damages", 37: "Warranty Duration",
    38: "Insurance", 39: "Covenant Not To Sue",
    40: "Third Party Beneficiary",
}

label_lines = "\n".join(f"{i}: {LABELS[i]}" for i in range(41))
SYSTEM_PROMPT = f"""You classify one contract segment against the CUAD catalog.

CUAD classes:
{label_lines}

Return exactly one JSON object: {{"class_ids":[<sorted unique integers 0 through 40>]}}.
Include a class only when this segment directly supports it. Return {{"class_ids":[]}} for irrelevant text, boilerplate, definitions without a target right, or unsupported concepts. Do not explain and do not use Markdown."""


def messages_for(passage: str) -> list[dict[str, str]]:
    return [
        {"role": "system", "content": SYSTEM_PROMPT},
        {
            "role": "user",
            "content": (
                "Contract segment:\n\n"
                + passage.strip()
                + '\n\nReturn exactly {"class_ids":[...]} with no other text.'
            ),
        },
    ]


tokenizer = AutoTokenizer.from_pretrained(BASE_ID, revision=BASE_REVISION)
renderer = get_renderer("qwen3_5_disable_thinking", tokenizer)
base = AutoModelForImageTextToText.from_pretrained(
    BASE_ID,
    revision=BASE_REVISION,
    dtype=torch.bfloat16,
    device_map="auto",
)
model = PeftModel.from_pretrained(base, ADAPTER_ID, is_trainable=False).eval()

passage = (
    "All terms of this Agreement will automatically commence on November 1st, "
    "2017, and expire on November 2nd, 2020."
)
prompt_ids = renderer.build_generation_prompt(messages_for(passage)).to_ints()
input_ids = torch.tensor([prompt_ids], dtype=torch.long, device=model.device)

with torch.inference_mode():
    generated = model.generate(
        input_ids=input_ids,
        do_sample=False,
        max_new_tokens=64,
        eos_token_id=tokenizer.eos_token_id,
        pad_token_id=tokenizer.pad_token_id,
        use_cache=True,
    )

text = tokenizer.decode(generated[0, input_ids.shape[1]:], skip_special_tokens=True).strip()
payload = json.loads(text)
class_ids = payload["class_ids"]
assert class_ids == sorted(set(class_ids))
assert all(type(i) is int and 0 <= i <= 40 for i in class_ids)
print({"class_ids": class_ids, "labels": [LABELS[i] for i in class_ids]})

Citation

@inproceedings{hendrycks2021cuad,
  title     = {CUAD: An Expert-Annotated NLP Dataset for Legal Contract Review},
  author    = {Hendrycks, Dan and Burns, Collin and Chen, Anya and Ball, Spencer},
  booktitle = {NeurIPS Datasets and Benchmarks},
  year      = {2021}
}