Blame for README.md

---
title: Smart MCQ Solver
emoji: 🧠
colorFrom: blue
colorTo: indigo
sdk: gradio
sdk_version: 6.22.0
app_file: app.py
pinned: false
license: mit
suggested_hardware: cpu-basic
short_description: Rank the top three answers to a five-option MCQ (MAP@3)
---

# Smart MCQ Solver

Live demo for the Deep Learning & Generative AI project (Diploma level, BS in Data
Science and Applications, IIT Madras).

**Devang Khirsaria · 23f1001439 · Term T2-2026**

## What this does

You give it a multiple-choice question and five options labelled A to E. It scores all
five and returns the **top three in ranked order**, which is the submission format of the
Kaggle *Smart MCQ Solver Challenge*. The competition metric is MAP@3, so a correct answer
at rank 1 scores 1.000, at rank 2 scores 0.500 and at rank 3 scores 0.333.

## What model is running

This Space runs **`cross-encoder/nli-deberta-v3-small` zero-shot** — Model 2 from the
project. The question is used as the NLI premise, each option as a hypothesis, and the
options are ranked by entailment probability. The entailment class index is read from
`model.config.label2id` rather than hard-coded, because label ordering is not consistent
across NLI checkpoints.

There is **no fine-tuning and no trained weight file** behind this demo. The project's
best Kaggle score, 0.736 MAP@3, came from an eight-member fine-tuned DeBERTa-v3
multiple-choice LoRA ensemble, which is not what is deployed here. On the project's own
validation split this zero-shot model scored 0.528 MAP@3, so its answers are noticeably
weaker than the leaderboard number.

## Notes

- The model is downloaded on the first prediction, so the first run takes roughly a
  minute. Later runs take a couple of seconds on the free CPU tier.
- Two worked examples are pre-loaded via the buttons under the form.
- Empty fields are reported back as a message rather than raising an error.

## Source

Full project code: `https://github.com/DevangK18/23f1001439-dl-genai-project-26-t2`