beaglabs/beag-compliance-0.5

BEAG Compliance 0.5

A LoRA adapter fine-tuned from Qwen2.5-7B-Instruct for multi-label NIST compliance control mapping. Given a document (policy, audit finding, risk assessment, implementation description, or gap analysis), the model outputs a JSON array of relevant NIST 800-53, CSF 2.0, and CMMC 2.0 controls with confidence scores and reasoning.

Model Details

• Developed by: Beag Labs
• Contact: james@beaglabs.com
• Base model: Qwen/Qwen2.5-7B-Instruct
• Model type: Causal LM + QLoRA adapter (rank 64)
• Language: English
• License: Apache 2.0

Task & Intended Use

Map compliance documents to NIST framework controls. The model accepts a JSON document of one of five types:

Document Type	Description
policy	Corporate policy documents
finding	Audit findings with control gaps
risk	Risk assessment entries
implementation	Control implementation descriptions
gap	Gap analysis entries

It outputs a JSON array of control mappings:

[
  {
    "control_id": "AC-11",
    "framework": "nist_800_53",
    "confidence": 0.95,
    "reasoning": "The policy requires device locking after inactivity..."
  }
]

Frameworks covered: NIST 800-53 Rev 5, NIST CSF 2.0, CMMC 2.0

Out of scope: This is a v0.5 prototype trained entirely on synthetic data. It should not be used for production compliance decisions without human review. The model may hallucinate control IDs -- always validate outputs against a framework catalog.

Training

Parameter	Value
Method	QLoRA (4-bit quantization)
LoRA rank	64
LoRA alpha	16
Target modules	q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
Learning rate	2e-4
Batch size	1 (effective 8 via gradient accumulation)
Epochs	1
Max sequence length	1024
Loss function	CISPO (asymmetric probability clipping) + entropy bonus
Self-distillation	Reverse KL with frozen base model (lambda=0.05, every 8 batches)
Optimizer	AdamW (weight decay 0.01)
LR schedule	Linear warmup + cosine decay

Training Results

Metric	Value
Steps	132
Best loss	0.0294
Final loss	0.2108
Training time	~60 minutes
Hardware	Kaggle T4 (16 GB VRAM)

Training Data

• ~1,060 synthetic examples generated by DeepSeek-chat from a catalog of ~10,000 NIST control entries
• Balanced across five document types (policy, finding, risk, implementation, gap)
• ~20% augmented via control swapping, noise injection, and key dropping
• All control IDs validated against the framework catalog

Methodology based on the Bridgewater / Thinking Machines recipe (synthetic data + QLoRA with CISPO loss + on-policy distillation).

How to Use

from unsloth import FastModel

model, tokenizer = FastModel.from_pretrained(
    model_name="beaglabs/beag-compliance-0.5",
    max_seq_length=2048,
    load_in_4bit=True,
)

system_prompt = (
    "You are a NIST compliance expert. Map the given document to relevant "
    "NIST 800-53, CSF, and CMMC controls. Return a JSON array of control "
    "mappings with control_id, framework, confidence, and reasoning."
)

document = '{"type": "policy", "title": "Access Control Policy", "text": "All users must authenticate via multi-factor authentication before accessing protected systems."}'

messages = [
    {"role": "system", "content": system_prompt},
    {"role": "user", "content": document},
]

model = FastModel.for_inference(model)
inputs = tokenizer.apply_chat_template(messages, tokenize=True, add_generation_prompt=True, return_tensors="pt").to("cuda")
outputs = model.generate(inputs, max_new_tokens=512, temperature=0.1, do_sample=True)
print(tokenizer.decode(outputs[0][inputs.shape[1]:], skip_special_tokens=True))

Or with PEFT + Transformers:

from peft import PeftModel
from transformers import AutoModelForCausalLM

base_model = AutoModelForCausalLM.from_pretrained("unsloth/Qwen2.5-7B-Instruct-bnb-4bit")
model = PeftModel.from_pretrained(base_model, "beaglabs/beag-compliance-0.5")

Limitations

• Trained on synthetic data only -- may not reflect real-world compliance document distributions
• Cannot distinguish truly novel or ambiguous compliance scenarios
• May overfit to the ~10,000 controls in the training catalog
• Generation should always be followed by catalog-based grounding to correct hallucinated control IDs
• Single-epoch training -- benefits from further fine-tuning with human-labeled data

Environment

Component	Version
Unsloth	2026.7.1
Transformers	5.5.0
Torch	2.10.0+cu128
CUDA	12.8
PEFT	0.19.1
DeepSeek (data gen)	deepseek-chat
Platform	Kaggle Notebooks (Linux)

Environmental Impact

• Hardware: NVIDIA T4 (16 GB)
• Training time: ~1 hour
• Cloud provider: Kaggle Notebooks
• Estimated carbon: <0.1 kg CO2eq (T4 at ~70W, 1 hour)

Citation

@misc{beag-compliance-0.5,
  author = {Beag Labs},
  title = {BEAG Compliance 0.5: NIST Control Mapping LoRA},
  year = {2025},
  publisher = {Hugging Face},
  howpublished = {\url{https://huggingface.co/beaglabs/beag-compliance-0.5}},
}