TamilRAV — Retrieval-Grounded Reasoning Model for Tamil Misinformation Verification

TamilRAV is a LoRA adapter for Qwen3.5-4B that performs evidence-grounded verification of Tamil misinformation claims. Given a Tamil claim and up to three retrieved fact-check evidence passages, it produces a structured verdict under a four-class taxonomy (True / False / Misleading / Altered), together with a Tamil-language explanation, a calibrated confidence score, and a step-by-step reasoning trace.

TamilRAV is the reasoning path (Path B) of CDPVA (Complementary Dual-Path Verification Architecture), introduced in the paper "A Complementary Dual-Path Verification Architecture for Multimodal Tamil Misinformation Classification and Reasoning", and is trained and evaluated on the TamilFacts-X benchmark of 17,271 fact-checked Tamil claims.

Model Details

Model Description

• Developed by: [TODO: author names / lab]
• Funded by [optional]: [TODO or remove]
• Shared by: [TODO: HF account / lab]
• Model type: Causal language model + LoRA adapter (PEFT), instruction-tuned for retrieval-grounded fact verification
• Language(s) (NLP): Tamil (claims, explanations); English (reasoning traces)
• License: Apache 2.0 [TODO: confirm this matches the base-model license and your intended adapter license]
• Finetuned from model: Qwen/Qwen3.5-4B

The adapter modifies the query, key, value, and output projections of every attention block and the MLP projection layers of the frozen 4B-parameter backbone. Only the low-rank matrices are trained; the base weights remain frozen. The adapter is loaded alongside the base model in bfloat16 precision at inference.

Model Sources

• Repository: [TODO: code repository URL]
• Paper: [TODO: DOI / preprint URL once available]
• Dataset: TamilFacts-X — [TODO: dataset URL or HF dataset ID]

Uses

Direct Use

TamilRAV is released as an editorial assistance tool for Tamil fact-checking newsrooms, platform trust-and-safety teams, and human-supervised verification pipelines. The intended workflow is retrieval-augmented: retrieve candidate evidence for a claim (in CDPVA: Qwen3-Embedding-0.6B dense retrieval over a Qdrant index, top-20, reranked to top-3 with BGE-Reranker-v2-m3), then prompt TamilRAV with the claim and the reranked passages.

For best results, use self-consistency decoding as in the paper: sample three generations at temperatures 0.3, 0.5, and 0.7, take the majority-vote verdict, and average the three confidence scores.

Downstream Use

TamilRAV serves as the reasoning path of the full CDPVA system, in which its verdict, pooled confidence, and evidence count are reconciled with a discriminative multimodal classifier through a rule-based verification gate. The full system reaches macro-F1 92.9 on the TamilFacts-X evaluation set.

Out-of-Scope Use

• Fully automated content moderation or takedown decisions without human review. The model is an assistance tool; its verdicts are not authoritative.
• Verification without retrieval evidence. The model is trained to reason over supplied passages. Without in-corpus evidence its accuracy drops sharply (see the leave-one-out results below).
• Languages other than Tamil, claims far outside the distribution of Indian/Tamil-language fact-checking (the training corpus is drawn from Tamil fact-checking organizations), and high-stakes domains such as medical or legal adjudication.
• Generating misinformation, or "laundering" false content by prompting for favorable verdicts.

Bias, Risks, and Limitations

• Retrieval dependence. Under a leave-one-out condition that withholds each claim's own source fact-check article, standalone macro-F1 falls from 82.7 to 67.7. On genuinely novel claims the model's value lies in the auditable, evidence-grounded reasoning rather than in raw accuracy.
• False vs. Misleading boundary. Like all systems evaluated in the paper (including frontier LLMs), the model errs most often on the boundary between outright fabrications and misleading framings of true events.
• Verdict-taxonomy drift. On manipulated-media claims, the model occasionally reasons correctly about the manipulation yet labels the claim "True" because the post accurately describes the manipulation. In CDPVA this failure mode is caught by the verification gate.
• Domain and coverage bias. The training corpus reflects the topical, temporal, and editorial distribution of Tamil fact-checking outlets (heavily Indian political and social-media content). Claims outside this distribution may be handled poorly.
• Confidence is regularized, not guaranteed. A Brier-score term during training discourages uniform overconfidence, but reported confidences should still be treated as heuristic signals, not calibrated probabilities, especially out of domain.

Recommendations

Keep a human in the loop for any consequential decision. Present the model's cited evidence and reasoning trace alongside its verdict so reviewers can audit the basis for each prediction. Recalibrate the confidence threshold on an in-domain validation split before deployment, particularly when the retrieval corpus differs from TamilFacts-X.

How to Get Started with the Model

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
import torch

base_id = "Qwen/Qwen3.5-4B"
adapter_id = "[TODO: your-org/TamilRAV]"

tokenizer = AutoTokenizer.from_pretrained(base_id)
model = AutoModelForCausalLM.from_pretrained(base_id, torch_dtype=torch.bfloat16, device_map="auto")
model = PeftModel.from_pretrained(model, adapter_id)
model.eval()

# Use the exact prompt template released with the paper/repository.
# Structure: the Tamil claim followed by the reranked evidence passages,
# requesting the four structured output fields.
prompt = build_prompt(claim_ta, evidence_passages)  # see repository for the template

# Self-consistency decoding as used in the paper
verdicts, confidences = [], []
for temp in (0.3, 0.5, 0.7):
    inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
    out = model.generate(**inputs, max_new_tokens=1024, do_sample=True, temperature=temp)
    text = tokenizer.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True)
    v, c = parse_structured_output(text)  # regex parser for verdict/confidence fields
    verdicts.append(v); confidences.append(c)

final_verdict = max(set(verdicts), key=verdicts.count)   # majority vote
final_confidence = sum(confidences) / len(confidences)   # mean pooling

Training Details

Training Data

The adapter is fine-tuned on the training partition (15,497 claims) of TamilFacts-X, a 17,271-claim multimodal Tamil fact-checking benchmark built from published fact-check articles and annotated under a four-class veracity taxonomy (True / False / Misleading / Altered). Each supervised instance serializes the claim together with its reranked evidence passages as input, and the four structured output fields (verdict, Tamil explanation, confidence in [0, 1], reasoning trace) as the target. [TODO: link the dataset card]

Training Procedure

Supervised fine-tuning with LoRA. The training objective combines the standard autoregressive cross-entropy loss over the serialized target with a Brier-score regularizer on the confidence field, which penalizes the squared deviation between the stated confidence and the correctness of the predicted verdict.

Training Hyperparameters

• Training regime: bf16
• LoRA target modules: attention Q, K, V, O projections and MLP projection layers
• LoRA rank r: [TODO]
• LoRA alpha: [TODO]
• Confidence-loss weight λ: [TODO]
• Optimizer / LR / epochs / batch size: [TODO]

Speeds, Sizes, Times [optional]

• Fine-tuned on a single consumer GPU: [TODO: GPU model, VRAM, wall-clock hours]
• Adapter checkpoint size: [TODO]

Evaluation

Testing Data, Factors & Metrics

Evaluated on the balanced 608-claim TamilFacts-X evaluation set (152 claims per class). Metric: macro-averaged F1 over the four classes. Evidence at test time is retrieved from the fact-check corpus (Qwen3-Embedding-0.6B + Qdrant, top-20; BGE-Reranker-v2-m3, top-3). Self-consistency (SC) denotes three samples at T ∈ {0.3, 0.5, 0.7} with majority-vote verdicts and mean-pooled confidence. The leave-one-out (LOO) condition removes each claim's own source article from its retrieval pool.

Results

Configuration	Macro-F1
TamilRAV, single sample	78.3
TamilRAV + self-consistency	82.7
TamilRAV + SC, leave-one-out retrieval	67.7
Full CDPVA system (with discriminative path + gate)	92.9 ± 0.6

Per-class F1 of TamilRAV + SC on the evaluation set: True 88.0, False 78.5, Misleading 75.1, Altered 89.4.

For reference, the strongest zero-shot text-prompted frontier LLM baseline evaluated in the paper reaches 62.5 macro-F1 on the same set (under a non-like-for-like, retrieval-free condition).

Summary

With retrieval grounding and self-consistency, a 4B-parameter LoRA-adapted model substantially outperforms zero-shot frontier LLMs on Tamil claim verification, and its structured, evidence-cited outputs make each verdict auditable. Its standalone accuracy depends significantly on retrieval quality, which is why it is deployed inside CDPVA alongside a discriminative classifier and a verification gate.

Environmental Impact

• Hardware Type: [TODO: e.g., 1× consumer GPU model]
• Hours used: [TODO]
• Cloud Provider / Compute Region: [TODO or "local workstation"]
• Carbon Emitted: TODO — estimable via the ML CO2 Impact calculator

Technical Specifications

Model Architecture and Objective

Qwen3.5-4B decoder-only transformer (rotary positional embeddings, SwiGLU activations, large multilingual vocabulary) with LoRA adapters on all attention Q/K/V/O and MLP projection matrices. Objective: autoregressive cross-entropy over structured verification outputs plus a Brier-score confidence regularizer.

Compute Infrastructure

Hardware

[TODO: GPU model and VRAM]

Software

• PEFT 0.14.0
• transformers [TODO: version], PyTorch [TODO: version]

Citation

BibTeX:

Model Card Authors

[TODO]

Model Card Contact

[TODO: email or HF discussion link]

Framework versions

• PEFT 0.14.0