Yad4o/constitution-qa-lora

Constitution QA LoRA

This is a LoRA adapter fine-tuned on the Indian Constitution to provide question-and-answering capabilities regarding Indian law and constitutional frameworks.

Model Details

• Base Model: Qwen/Qwen3-1.7B
• Architecture: Causal LM
• PEFT Method: LoRA (Low-Rank Adaptation)
• Rank ($r$): 16
• Alpha ($\alpha$): 32
• Target Modules: v_proj, q_proj, k_proj, o_proj

Training

• Dataset: Cleaned text extracted from the official Constitution of India (393 training examples).
• Epochs: 3
• Learning Rate: 2e-4
• Batch Size: 2
• Framework: trl SFTTrainer
• Hardware: Not reported

Intended Use

Designed for educational and informational queries regarding the Indian Constitution.

Limitations

This model is an AI assistant and should not be used as a substitute for professional legal advice. Users should verify all outputs against the official text of the Constitution of India.

Installation and Inference

To use this adapter, install transformers and peft:

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
import torch

base = "Qwen/Qwen3-1.7B"
tok = AutoTokenizer.from_pretrained(base)
model = AutoModelForCausalLM.from_pretrained(base, device_map="auto", torch_dtype=torch.float16)
model = PeftModel.from_pretrained(model, "Yad4o/constitution-qa-lora")

msgs = [{"role": "user", "content": "What is Article 21 of the Indian Constitution?"}]
text = tok.apply_chat_template(msgs, tokenize=False, add_generation_prompt=True)
inputs = tok(text, return_tensors="pt").to(model.device)
out = model.generate(**inputs, max_new_tokens=300)
print(tok.decode(out[0][inputs.input_ids.shape[1]:], skip_special_tokens=True))