---
language: en
license: apache-2.0
base_model: Qwen/Qwen3-1.7B
library_name: peft
pipeline_tag: text-generation
tags:
- lora
- indian-constitution
- law
- qa
---

# Constitution QA LoRA

This is a LoRA adapter fine-tuned on the Indian Constitution to provide question-and-answering capabilities regarding Indian law and constitutional frameworks.

## Model Details
- **Base Model:** Qwen/Qwen3-1.7B
- **Architecture:** Causal LM
- **PEFT Method:** LoRA (Low-Rank Adaptation)
- **Rank ($r$):** 16
- **Alpha ($\alpha$):** 32
- **Target Modules:** `v_proj`, `q_proj`, `k_proj`, `o_proj`

## Training
- **Dataset:** Cleaned text extracted from the official Constitution of India (393 training examples).
- **Epochs:** 3
- **Learning Rate:** 2e-4
- **Batch Size:** 2
- **Framework:** `trl` SFTTrainer
- **Hardware:** Not reported

## Intended Use
Designed for educational and informational queries regarding the Indian Constitution. 

## Limitations
This model is an AI assistant and should not be used as a substitute for professional legal advice. Users should verify all outputs against the official text of the Constitution of India.

## Installation and Inference
To use this adapter, install `transformers` and `peft`:

```python
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
import torch

base = "Qwen/Qwen3-1.7B"
tok = AutoTokenizer.from_pretrained(base)
model = AutoModelForCausalLM.from_pretrained(base, device_map="auto", torch_dtype=torch.float16)
model = PeftModel.from_pretrained(model, "Yad4o/constitution-qa-lora")

msgs = [{"role": "user", "content": "What is Article 21 of the Indian Constitution?"}]
text = tok.apply_chat_template(msgs, tokenize=False, add_generation_prompt=True)
inputs = tok(text, return_tensors="pt").to(model.device)
out = model.generate(**inputs, max_new_tokens=300)
print(tok.decode(out[0][inputs.input_ids.shape[1]:], skip_special_tokens=True))
```