| --- |
| base_model: Qwen/Qwen3-4B |
| library_name: peft |
| license: apache-2.0 |
| pipeline_tag: text-classification |
| tags: |
| - lora |
| - peft |
| - intent-classification |
| - quorum |
| extra_gated_prompt: >- |
| Quorum-Reader is shared for research and evaluation. Tell us a little about your use — it helps us |
| prioritise what to build next. Access is granted automatically. |
| extra_gated_fields: |
| Name: text |
| Email: text |
| Organization: text |
| What are you building with this?: text |
| --- |
| |
| # Quorum-Reader-Qwen3-4B-Adapter |
|
|
| A LoRA adapter for `Qwen/Qwen3-4B` — the **in-context reader** of the |
| [Quorum](https://github.com/DataAgent-Lab/Quorum) 24-shot intent-classification ensemble. |
|
|
| Given 24 retrieved examples in the prompt (**no weight update**), it scores every candidate label in a single |
| forward pass. Combined (geometric mean) with a frozen `BAAI/bge-large-en-v1.5` nearest-neighbour over the same |
| 24 examples, the ensemble reaches **0.932 on Banking77 (full 3,080 test) — above the closed API Jev's 0.924** |
| at Jev's own 24-shot protocol. In our study a further configuration (per-class descriptions + a |
| calibration-selected combination) reached **0.938** and *significantly* beat Jev on a paired McNemar test |
| (p = 0.0014). |
|
|
| ## Training |
| LoRA (r = 64) on `Qwen/Qwen3-4B`, fine-tuned on a mixture of **public** intent-classification datasets with |
| **Banking77 held out**. Single-forward answer-token option scoring. |
|
|
| ## Intended use & limitations |
| - Use as the reader in the Quorum 24-shot pipeline (see the repo). Requires the base model + PEFT to load. |
| - **Contamination:** the training mixture includes CLINC150, HWU64 and SNIPS — do **not** use this adapter to |
| judge zero-shot generalization on those datasets. |
| - Not a general-purpose chat model; not a claim of open zero-shot state-of-the-art. |
|
|
| ## Usage |
| ```bash |
| # https://github.com/DataAgent-Lab/Quorum |
| python scripts/eval_24shot.py --dataset banking77 --adapter DataAgent/Quorum-Reader-Qwen3-4B-Adapter |
| ``` |
|
|
| ## License |
| Adapter weights: Apache-2.0. The base model and datasets retain their own licenses. |
|
|
