Instructions to use xanderwick/LawGranite with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use xanderwick/LawGranite with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="xanderwick/LawGranite") messages = [ {"role": "user", "content": "Who are you?"}, ] pipe(messages)# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("xanderwick/LawGranite") model = AutoModelForCausalLM.from_pretrained("xanderwick/LawGranite", device_map="auto") messages = [ {"role": "user", "content": "Who are you?"}, ] inputs = tokenizer.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=256) print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use xanderwick/LawGranite with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "xanderwick/LawGranite" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "xanderwick/LawGranite", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/xanderwick/LawGranite
- SGLang
How to use xanderwick/LawGranite with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "xanderwick/LawGranite" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "xanderwick/LawGranite", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "xanderwick/LawGranite" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "xanderwick/LawGranite", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Docker Model Runner
How to use xanderwick/LawGranite with Docker Model Runner:
docker model run hf.co/xanderwick/LawGranite
|
Download README.md from xanderwick/LawGranite: direct link, hf CLI and curl.
- Browser
- Download file 18.7 kB
-
https://huggingface.co/xanderwick/LawGranite/resolve/main/README.md
- Command line
-
hf download hf://xanderwick/LawGranite/README.md
-
curl -L -H "Authorization: Bearer $HF_TOKEN" -o README.md https://huggingface.co/xanderwick/LawGranite/resolve/main/README.md
18.7 kB
| base_model: ibm-granite/granite-4.2-8b | |
| base_model_relation: finetune | |
| language: | |
| - en | |
| library_name: transformers | |
| pipeline_tag: text-generation | |
| tags: | |
| - legal | |
| - granite | |
| - dpo | |
| # LawGranite | |
| *Version 1.0* | |
| A fine-tune of [IBM Granite 4.2 8B](https://huggingface.co/ibm-granite/granite-4.2-8b) for drafting | |
| US litigation prose, such as briefs and memoranda, in the register of practicing attorneys. | |
| > **⚠️ Do not trust its citations.** In my measurements it often cites cases that cannot be | |
| > found, and citations that do resolve to a real reporter page frequently point to a *different | |
| > case* than the one it names, far more often than in attorney-written briefs. A citation that | |
| > resolves is not proof it is right: the page may belong to another case. Every citation must be | |
| > checked against the source before any use. This is a drafting aid, not legal research, and it | |
| > gives no legal advice. | |
| ## Designed to be paired, not used alone | |
| LawGranite does one job: write persuasive litigation prose that reads like an attorney wrote it. | |
| It was designed with four goals in mind: | |
| - **Persuasive drafting:** it is trained on real briefs to argue in the register courts expect. | |
| - **Lower cost:** drafting is the longest, most token-heavy part of the work. Handing it to a small | |
| local model saves token usage on the more capable (and more expensive) models in the system. | |
| - **Client confidentiality:** it runs entirely on your own hardware, so the facts and documents | |
| you give it for drafting never leave your machine. (A cloud model it is paired with still sees | |
| whatever you send that model, so decide what goes to it.) | |
| - **No watermarking:** LawGranite does not watermark its output. I built it in part out of | |
| concern about the growth of hidden text watermarking, such as Google's SynthID, and the lack of | |
| transparency around it. These schemes embed a statistical signal in the words a model chooses. | |
| The writer cannot see it, and in practice only the vendor that controls the detector can check | |
| for it or say what a detection means. For attorney work product, a hidden, vendor-readable mark | |
| in a filing is a confidentiality and transparency problem in its own right. LawGranite's | |
| decoding is ordinary temperature and top-p sampling, set out in full under Usage, with no | |
| watermarking step. No watermark does not mean no disclosure: see **Disclosure** under | |
| "Intended use and limits". | |
| LawGranite is **not** designed to find, choose, or verify legal authority, and it should not be | |
| deployed as a standalone legal assistant. Use it in one of two ways: | |
| - **With LawGoose**, my companion program, which is built to address weaknesses present in many | |
| local models. LawGoose gives LawGranite only the drafting role. A more capable model does the | |
| research, using LawGoose's CourtListener search and citation-verification tools, and LawGoose | |
| checks every draft before an attorney sees it: | |
| - **Authority list enforcement:** any case citation not on the authorities the attorney supplied, | |
| including its short and *Id.* cites, is replaced with `[AUTHORITY NEEDED]`. An optional strict | |
| setting does the same when no authorities are supplied. | |
| - **Wrong-case check:** every case citation is resolved through CourtListener, and the case at | |
| that reporter page is compared with the case the draft names. A mismatch is flagged as a | |
| wrong-case citation; a citation whose case name cannot be checked is marked inline, so none | |
| passes silently. | |
| - **Fact check:** names, dates, amounts, durations and statutes the drafter was never given are | |
| flagged inline for the attorney (flagged, not removed). | |
| - **Confidentiality:** LawGoose is designed to keep client facts on the firm's own hardware or | |
| private network from start to finish, including running LawGranite on a private rented GPU for | |
| firms without Apple Silicon. | |
| - **Lower zero-data-retention spend:** LawGoose is intended to reduce the spend on | |
| zero-data-retention AI services, by handing the longest, most token-heavy work (drafting) to | |
| LawGranite running locally. | |
| - **Under a more capable general model**, which does the research and planning and checks every | |
| citation and factual claim against the source, and calls LawGranite only to turn the result into | |
| attorney-style prose. | |
| Either way, the model's text is a draft for an attorney to verify, not a finished filing. | |
| Unless a figure says otherwise, every number in this card measures LawGranite **by itself**, with | |
| no checks applied after drafting. | |
| ## Enterprise and practice ready | |
| - **Signed weights:** every file in the release is listed in a cryptographically signed SHA-256 | |
| manifest, so your security team can verify that the weights are exactly the ones I published | |
| and have not been altered (see Signed release, below). | |
| - **Runs on your own infrastructure:** client data never has to leave your network. LawGranite runs | |
| on Apple Silicon, on NVIDIA GPUs, or on a private cloud instance. | |
| - **Deployment help:** I am happy to help set up LawGranite and LawGoose for an enterprise | |
| deployment. | |
| ## Signed release | |
| The weights and every other file in this release are cryptographically signed. `SHA256SUMS` lists | |
| the SHA-256 hash of each file (weights, config, tokenizer, chat template). `SHA256SUMS.p7s` is a | |
| signature over that list, signed with a certificate tied to the Federal Bridge. If any file is | |
| altered, its hash no longer matches the signed list. | |
| To verify a download: | |
| ```bash | |
| cd ~/.omlx/models/LawGranite | |
| # 1. the signature is valid and was made by my certificate | |
| # (identrust-igc-chain.pem, included in this repository: IdenTrust's IGC root and intermediate certificates) | |
| openssl cms -verify -binary -inform DER -in SHA256SUMS.p7s -content SHA256SUMS \ | |
| -CAfile identrust-igc-chain.pem -purpose any -out /dev/null | |
| # 2. every file matches the signed list | |
| shasum -a 256 -c SHA256SUMS | |
| ``` | |
| The model card (`README.md`) is not covered by the signature, so it can be updated without | |
| re-signing the weights. | |
| ## What you need to run it | |
| This model is designed to be more accessible than many others, but it does require specialized | |
| hardware. Any of the following can run it: | |
| | machine | memory | | |
| |---|---| | |
| | Apple Silicon Mac | **64 GB or more** unified memory (96–128 GB is comfortable) | | |
| | Workstation or server with one NVIDIA GPU | **48 GB or more** of GPU memory (e.g. RTX 6000 Ada, L40S, A6000, A100 80 GB, H100) | | |
| | Two NVIDIA GPUs | 24 GB each or more (e.g. 2× RTX 4090) | | |
| | Cloud | Any instance with one of the GPUs above | | |
| Also needed: | |
| - **Disk:** about 36 GB free for the download. | |
| - **System memory (RAM):** at least 48 GB on GPU machines, because the model passes through RAM while loading. | |
| - **Software:** on a Mac, oMLX (see Usage). Elsewhere, Python with `transformers` and PyTorch (I | |
| tested `transformers` 4.57.6), or vLLM for serving. | |
| Running on a CPU alone, without a GPU or Apple chip, works if the machine has 64 GB or more of RAM, | |
| but it is too slow for practical drafting. The weights are stored at full precision (float32) on | |
| purpose: the last training step made very small changes that lower-precision formats round away. | |
| This full-precision release is the reference model. | |
| **Smaller hardware:** [LawGranite 4-bit (MLX)](https://huggingface.co/xanderwick/LawGranite-4bit-MLX) | |
| is a 4.6 GB version for Apple Silicon Macs with 16 GB or more of memory. It matches this model on | |
| reading as human-written, writing style and instruction following, but it relies on a supplied list | |
| of authorities somewhat less: 79% of its case citations come from the list, against 88% here. Its | |
| card gives the full comparison. | |
| ## Usage | |
| The recommended way to run LawGranite on a Mac is [oMLX](https://github.com/jundot/omlx), a local | |
| server for Apple Silicon. It serves the model on your own machine through an OpenAI-compatible | |
| API, so other tools can connect to it, and no text leaves the computer. | |
| **1. Install oMLX and the Hugging Face CLI** | |
| These commands use [Homebrew](https://brew.sh), the standard package manager for macOS. If | |
| `brew --version` in Terminal says "command not found", install Homebrew first: | |
| ```bash | |
| /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)" | |
| ``` | |
| When it finishes, run the two "Next steps" commands it prints, which add `brew` to your PATH. Then: | |
| ```bash | |
| brew install jundot/omlx/omlx | |
| brew install hf # the Hugging Face CLI | |
| ``` | |
| **2. Download the model into oMLX's model folder** (about 35 GB) | |
| ```bash | |
| hf auth login # access requires approval; log in with an approved account | |
| hf download xanderwick/LawGranite --local-dir ~/.omlx/models/LawGranite | |
| ``` | |
| oMLX finds every model folder under `~/.omlx/models/`. The folder name, `LawGranite`, is the | |
| model name you use in requests. | |
| **3. Set the model's options in oMLX, then start the server** | |
| In oMLX's settings for `LawGranite`, set: | |
| - **Thinking: off** (`enable_thinking: false`). Granite's template turns reasoning on by default, | |
| and this model was never trained with reasoning traces. | |
| - **Temperature 0.7, top-p 0.95.** LawGranite's evaluations below used these values. Greedy decoding | |
| (temperature 0) reads far more machine-like on the detector. | |
| - **Max tokens 3,072 or more**, so full drafts are not cut off. | |
| - **No quantization in oMLX.** Keep these weights at full precision (float32); quantizing them on | |
| load loses part of the final training. For a smaller model, use | |
| [LawGranite 4-bit](https://huggingface.co/xanderwick/LawGranite-4bit-MLX), which was quantized | |
| with that loss measured. | |
| - **Free memory:** LawGranite needs about 35 GB of free memory. If another large model is pinned | |
| or loaded in oMLX, unpin or unload it first, or oMLX will refuse to load LawGranite. | |
| ```bash | |
| omlx serve # serves on http://127.0.0.1:8000 | |
| ``` | |
| **4. Send a drafting request** | |
| ```bash | |
| curl http://127.0.0.1:8000/v1/chat/completions -H "Content-Type: application/json" -d '{ | |
| "model": "LawGranite", | |
| "messages": [{"role": "user", "content": "Draft a short argument that the complaint fails to state a claim for breach of fiduciary duty under Colorado law."}], | |
| "max_tokens": 3072 | |
| }' | |
| ``` | |
| The evaluations also used a repetition penalty of 1.1. If your client lets you set it, use the same value. | |
| **Not on a Mac?** The weights are standard Hugging Face safetensors and load with `transformers` | |
| (`dtype="auto"`) or vLLM (`--dtype float32 --default-chat-template-kwargs '{"enable_thinking": false}'`) | |
| on the NVIDIA hardware listed above. | |
| ## Training | |
| | stage | data | method | | |
| |---|---|---| | |
| | continued pretraining | US court opinions and briefs from [CourtListener](https://www.courtlistener.com/) / RECAP (opinions:briefs 70:30) | LoRA, merged | | |
| | SFT | ~58M tokens: drafting tasks built from held-out briefs (outline→draft, facts→statement, body→summary), instruction and constraint following, multi-turn | LoRA r32 on all 40 blocks, merged | | |
| | DPO | Two runs from the same starting point, each on 1,084 on-policy pairs (detector pairs preferring samples that an AI-text detector scored as human-written, plus constraint pairs where the chosen draft obeys a length or format cap) and grounded-drafting pairs that prefer drafts citing only the supplied authorities (418 and 748 pairs) | LoRA r16 each, β 0.1, 1 epoch; the two updates blended 35:65 and merged in fp32 (this repo) | | |
| All training ran on public court filings. **Those filings contain real names of | |
| parties, attorneys, and judges**, and the model can reproduce them. | |
| ## How it learned to sound like an attorney, not an AI | |
| Most AI models write in a recognizable "AI voice": predictable phrasing, stock transitions, an even, | |
| generic tone. To train that out of LawGranite, I built a tool that spots AI-written legal text, and | |
| used it to coach the model. | |
| **The AI-writing detector.** It was taught on 56,608 legal documents, half written by attorneys | |
| and half by AI systems. Given a passage, it estimates how likely it is that AI wrote it. | |
| - **Accurate:** tested on text from AI systems it had never seen, it told AI from human writing | |
| correctly almost every time (a score of 0.9987, where 1.0 is perfect and 0.5 is guessing). | |
| - **Few false alarms:** I checked it against 1,200 briefs filed before 2022, written before modern | |
| AI existed. It flagged none of their full-length passages as AI-written. It does not judge | |
| passages under 120 words, because there is too little text to tell. | |
| **How it shaped the model.** The detector was used twice: | |
| 1. **Choosing what the model learned from.** 55,139 recent briefs were screened, and passages that | |
| looked AI-written were left out of the training examples, so the model learned from human | |
| writing only. | |
| 2. **Coaching on its own drafts.** The model wrote several drafts for each request. The detector | |
| scored them, and the model was trained to favor the drafts that read as human-written. Repeated | |
| over many requests, this steers it away from the habits that make text sound like AI. | |
| **Result.** Before this coaching, 85.4% of the model's drafts read as human-written to the detector. | |
| After it, **99.5%** do (Evaluation, below). | |
| **Limits.** | |
| - The model was graded by the same detector that coached it, so the 99.5% shows it satisfies | |
| *this* detector. Other detectors, such as Pangram, and experienced readers may judge differently. | |
| - The detector learned mostly from one kind of drafting task and one family of AI systems, so it is | |
| most reliable on that kind of text. | |
| - See **Disclosure** under "Intended use and limits": this is about the quality of the writing, | |
| not about hiding AI involvement where disclosure is required. | |
| **Technical details** (for readers who want them) | |
| - **Detector:** a ModernBERT sequence classifier, fine-tuned on 56,608 documents (28,304 human, | |
| 28,304 AI). It scores text in windows of up to 510 tokens and outputs P(AI). | |
| - **Generalization:** zero-shot AUROC 0.9987 mean (0.9961 minimum) across three splits that held | |
| out entire AI model families. With citations masked, AUROC is 0.9991, so it is not relying on | |
| citation formatting. | |
| - **Operating point:** a window reads as human-written when P(AI) < 0.05. Windows under 120 words | |
| are abstained on. Against 1,200 pre-2022 briefs, the false-positive rate on full-length windows | |
| was 0 of 1,200; nearly all false positives occurred on short windows, which is why the floor exists. | |
| - **Data screening:** 55,139 post-2022 briefs were screened at the excerpt level before building the | |
| supervised fine-tuning (SFT) data. | |
| - **Preference training:** Direct Preference Optimization (DPO) on on-policy pairs, with the chosen | |
| draft scoring low P(AI) and the rejected draft high, mixed with constraint pairs (1,084 pairs) and | |
| grounded-drafting pairs. Two LoRA runs (rank 16, β 0.1, 1 epoch) were blended 35:65 and merged | |
| in float32. | |
| - **Measurement:** 200 held-out prompts, temperature 0.7, top-p 0.95, repetition penalty 1.1. The | |
| share of drafts below 0.05 rose from 85.4% (SFT model) to 99.5% (released model). | |
| **What the detector builds on.** The detector stands on published research and open resources: | |
| - **ModernBERT** (Answer.AI and LightOn, 2024; `answerdotai/ModernBERT-base`): the open encoder | |
| model the detector is fine-tuned from, chosen for its long context, since legal filings run long. | |
| - **RAID** (Dugan et al., ACL 2024), **M4** (Wang et al., EACL 2024) and **HC3** (Guo et al., | |
| 2023): public AI-text detection benchmarks used during development. RAID's practice of testing | |
| on AI systems held out from training is the basis for the "never seen in training" result above. | |
| - **CourtListener and RECAP** (Free Law Project): the source of the attorney-written briefs. | |
| These public benchmarks contain little or no legal writing, so the final detector was trained on | |
| matched pairs instead: each AI-written document was generated from the outline of a real brief | |
| and paired with that brief. The two sides share topic, structure and length, so the detector has | |
| to separate them on *how* they are written, not *what* they are about. | |
| ## Evaluation | |
| LawGranite drafts text that reads far more like an attorney wrote it than the base IBM Granite 4.2 8B | |
| does, and it follows drafting instructions somewhat better. Both models answered the same 200 test | |
| requests, none of which LawGranite saw during training, at the same settings (those under Usage). | |
| | | Granite 4.2 8B | **LawGranite** | | |
| |---|---|---| | |
| | Drafts that read as human-written on the project's AI-text detector | 0.5% (1 of 190) | **99.5%** (190 of 191) | | |
| | Distance from real attorney writing style (lower is closer) | 0.84 | **0.39** | | |
| | Instruction following: 81 tests of word limits, edits, Q&A, system prompts, multi-turn | 61.7% | **66.7%** | | |
| ## Intended use and limits | |
| - **Intended:** the drafting component of a larger system (LawGoose, or a more capable model | |
| that handles research and citation checking), producing first drafts for attorneys who will | |
| verify every authority, rewrite, and take responsibility for the filing. | |
| - **Supplied authorities:** even when the prompt gives a list of authorities, the model by itself | |
| cites cases that are not on it in about 43% of drafts. Keep the list enforced by the surrounding | |
| system, as LawGoose does: on 116 grounded drafting tasks, LawGoose's enforcement (strict | |
| setting) cut drafts citing an unlisted case from 56 to 18, without removing any supplied | |
| authority. A citation whose case name LawGoose cannot identify is left in place and flagged | |
| for the attorney instead. | |
| - **Not intended:** legal advice, legal research, unsupervised filing, or use by | |
| non-lawyers as a substitute for counsel. | |
| - **Disclosure:** part of this model's preference training rewarded output that an AI-text | |
| detector scores as human-written. That training is why it reads as human 99.5% of the time on the | |
| project's detector. Do not use it to conceal AI involvement where a court, rule, or client | |
| requires disclosure. Many courts now require it. | |
| - **Third-party AI detectors:** results against independent AI-text detectors, including Pangram, | |
| are mixed. The 99.5% "reads as human-written" figure comes from this project's own detector, | |
| which the model was trained against; other detectors may flag some of its output as AI-written. | |
| - **Coverage:** the data skews toward US federal courts. State-specific practice, local | |
| rules, and recent changes in law are not reliably represented. | |
| - **Long drafts:** when output was capped at 1,536 tokens (roughly 1,100 words), 70% of full | |
| drafts written from an outline were cut off. Allow 3,072 or more output tokens for full drafts. | |