Instructions to use ProprietaryLegal/Thomson-1.0-Small-NVFP4 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use ProprietaryLegal/Thomson-1.0-Small-NVFP4 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="ProprietaryLegal/Thomson-1.0-Small-NVFP4")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("ProprietaryLegal/Thomson-1.0-Small-NVFP4")
model = AutoModelForMultimodalLM.from_pretrained("ProprietaryLegal/Thomson-1.0-Small-NVFP4", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use ProprietaryLegal/Thomson-1.0-Small-NVFP4 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "ProprietaryLegal/Thomson-1.0-Small-NVFP4"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ProprietaryLegal/Thomson-1.0-Small-NVFP4",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/ProprietaryLegal/Thomson-1.0-Small-NVFP4

SGLang

How to use ProprietaryLegal/Thomson-1.0-Small-NVFP4 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "ProprietaryLegal/Thomson-1.0-Small-NVFP4" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ProprietaryLegal/Thomson-1.0-Small-NVFP4",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "ProprietaryLegal/Thomson-1.0-Small-NVFP4" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ProprietaryLegal/Thomson-1.0-Small-NVFP4",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use ProprietaryLegal/Thomson-1.0-Small-NVFP4 with Docker Model Runner:
```
docker model run hf.co/ProprietaryLegal/Thomson-1.0-Small-NVFP4
```

Thomson-1.0-Small-NVFP4

Published by PLI Labs, the open research arm of Proprietary Legal Intelligence. PLI builds AI systems around a legal organization's own work product, combining retrieval, staged workflows, validation, and attorney review. Visit proprietarylegal.ai to explore our work and discuss what your organization needs.

PLI Labs update 2026-10-27: our DeepSeek-V4.1-Flash on 2× DGX Spark work is now fully public — dsv41-offload (GitHub, every component with production, main and per-experiment branches), with focused repos for the measurement harness and sampled drafting. Status and lessons: 2026-10-27 post.

NVFP4 quantization of thomsonreuters/Thomson-1.0-Small — the open-weight 35B-A3B MoE member of the Thomson-1.0 family (Qwen3.6-35B-A3B architecture, qwen3_5_moe, 262,144-token context, vision-capable), continually trained by Thomson Reuters for legal, tax, and journalism work.

Weights and activations quantized to NVFP4 with vllm-project/llm-compressor, following the recipe published by RedHat AI for the base architecture (RedHatAI/Qwen3.6-35B-A3B-NVFP4). Vision tower, lm_head, embeddings, router gates, shared-expert gates, and linear-attention blocks are kept in BF16.

Quantized on an NVIDIA DGX Spark (GB10 Grace-Blackwell); intended for Blackwell-class hardware with native FP4 (DGX Spark, B100/B200, RTX 50-series).

Serving

vllm serve ProprietaryLegal/Thomson-1.0-Small-NVFP4 \
  --reasoning-parser qwen3 --enable-prefix-caching
# vLLM >= 0.19 required. On MoE-friendly builds you may add: --moe_backend flashinfer_cutlass

Creation

llm-compressor script

# NVFP4 quantization of Thomson-1.0-Small (qwen3_5_moe), per RedHatAI/Qwen3.6-35B-A3B-NVFP4 recipe
import torch
from compressed_tensors.utils import save_mtp_tensors_to_checkpoint
from datasets import load_dataset
from transformers import AutoProcessor, Qwen3_5MoeForConditionalGeneration
from llmcompressor import oneshot
from llmcompressor.modifiers.quantization import QuantizationModifier

MODEL_ID = "/models/Thomson-1.0-Small"
SAVE_DIR = "/models/Thomson-1.0-Small-NVFP4"

model = Qwen3_5MoeForConditionalGeneration.from_pretrained(MODEL_ID, dtype="auto")
processor = AutoProcessor.from_pretrained(MODEL_ID)

recipe = QuantizationModifier(
    targets="Linear",
    scheme="NVFP4",
    ignore=[
        "re:.*lm_head",
        "re:visual.*",
        "re:model.visual.*",
        "re:.*mlp.gate$",
        "re:.*embed_tokens$",
        "re:.*shared_expert_gate$",
        "re:.*linear_attn.*",
    ],
)

NUM_CALIBRATION_SAMPLES = 256
MAX_SEQUENCE_LENGTH = 4096

ds = load_dataset("HuggingFaceH4/ultrachat_200k", split=f"train_sft[:{NUM_CALIBRATION_SAMPLES}]")
ds = ds.select_columns(["messages"]).shuffle(seed=42)

def preprocess_function(example):
    messages = [
        {"role": m["role"], "content": [{"type": "text", "text": m["content"]}]}
        for m in example["messages"]
    ]
    return processor.apply_chat_template(
        messages, tokenize=True, return_dict=True, add_generation_prompt=False,
        processor_kwargs={"return_tensors": "pt", "padding": False, "truncation": True,
                          "max_length": MAX_SEQUENCE_LENGTH, "add_special_tokens": False},
    )

ds = ds.map(preprocess_function, batched=False, remove_columns=ds.column_names)

def data_collator(batch):
    assert len(batch) == 1
    return {key: torch.tensor(value) for key, value in batch[0].items()}

oneshot(model=model, recipe=recipe, dataset=ds, max_seq_length=MAX_SEQUENCE_LENGTH,
        num_calibration_samples=NUM_CALIBRATION_SAMPLES, moe_calibrate_all_experts=True,
        data_collator=data_collator)

model.save_pretrained(SAVE_DIR)
processor.save_pretrained(SAVE_DIR)
try:
    save_mtp_tensors_to_checkpoint(source_model=MODEL_ID, dest_dir=SAVE_DIR)
except Exception as e:
    print(f"MTP tensor copy skipped/failed (may not exist in this checkpoint): {e}")
print("NVFP4 QUANT DONE")

Calibration: 256 samples, 4096 max sequence length, HuggingFaceH4/ultrachat_200k, moe_calibrate_all_experts=True.

Evaluation

Measured on NVIDIA DGX Spark (GB10, 128 GB unified), vLLM v0.25.1, --max-model-len 262144.

Metric	Value
Generation speed (DGX Spark GB10, vLLM 0.25.1)	~41.5 tok/s single-stream
Prefill (5,226-token prompt)	2.6 s
Context length served	262,144
Checkpoint size	22 GB (from 70 GB BF16)

Legal-competence harness (LLM-judged)

Real-workload tasks — summarizing a 36,000-word case-law survey (57,929-token prompt), drafting a South Carolina family-court affidavit, and drafting a motion to compel with SCRCP authority — graded 1-10 by a Claude Haiku 4.5 judge on accuracy / completeness / usability:

Task	Accuracy	Completeness	Usability	tok/s
Summarize 36k-word case survey (57.9K-tok prompt)	8	9	9	31.4
Draft SC family-court affidavit	8	6	7	40.6
Draft motion to compel (SCRCP 33/34/37)	7	4	5	40.4
Mean	7.7	6.3	7.0

Grades for the BF16 baseline of the same tasks on this hardware family are reported for comparison where available. Judge: claude-haiku-4-5, temperature default, single-pass.

Measured performance across hardware (Thomson-1.0-Small family)

Single-stream, llama.cpp release ≥ b10603 for GGUF lanes, all at the full 262,144-token context:

Hardware	Format / stack	Generation	Prefill (5,226-tok prompt)
NVIDIA DGX Spark (GB10)	NVFP4 (this repo), vLLM v0.25.1	~41.5 tok/s	2.6 s (~2,010 tok/s)
2× RTX 3090 NVLink	GGUF Q4_K_M (bartowski), llama.cpp CUDA	~147–156 tok/s	1.6 s (~3,300 tok/s)
3× Tesla V100-32GB	GGUF Q6_K (bartowski), llama.cpp CUDA (sm70)	~79–82 tok/s	7.3 s (~716 tok/s)

V100 notes: pin GPUs with CUDA_VISIBLE_DEVICES by UUID and set GGML_CUDA_DISABLE_PEER_ACCESS=1 on many-GPU hosts (avoids a "peer mapping resources exhausted" abort). The model reasons at length before answering — budget max_tokens ≥ 6,000 for drafting tasks or long answers truncate.

License

Inherited from the base model: PolyForm Strict 1.0.0. Review the base model's LICENSE before use.

About PLI Labs

“Your proprietary work product is your proprietary intelligence.” PLI Labs is the open research arm of Proprietary Legal Intelligence. We publish legal model prunes, serving recipes for owned hardware, and open-source tools so readers can inspect our methods and test our claims before hiring us.

Founded by a practicing trial lawyer, PLI builds AI systems around a legal organization's own work product: retrieval, staged workflows, validation, and attorney review. The approach grew from systems running on live matters in a litigation practice. Our consulting work supports deployment through approved frontier APIs, private cloud, or open-weight models on hardware you own. Explore the research or contact us about your organization's work.

Website · PLI Labs · Research · GitHub · Hugging Face · Contact

Proprietary Legal Intelligence LLC is a technology consultancy, not a law firm, and does not provide legal services or legal advice.

Downloads last month: 64

Safetensors

Model size

21B params

Tensor type

BF16

Model tree for ProprietaryLegal/Thomson-1.0-Small-NVFP4

Base model

Qwen/Qwen3.6-35B-A3B

Finetuned

tri-fair-lab/Snowdon1.1-Small

Finetuned

thomsonreuters/Thomson-1.0-Small

Quantized

(13)

this model