File size: 5,557 Bytes
86f00bc | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 | ---
license: cc-by-4.0
language:
- en
library_name: sklearn
pipeline_tag: text-classification
tags:
- text-classification
- tabular-classification
- interview-questions
- placements
- scikit-learn
datasets:
- adithegod/askedbefore-cs-interview-questions
metrics:
- f1
model-index:
- name: prepmap-question-classifier
results:
- task:
type: text-classification
name: Subject classification (15 classes)
dataset:
name: AskedBefore CS Interview Questions
type: adithegod/askedbefore-cs-interview-questions
metrics:
- type: f1
name: Macro F1 (5-fold CV)
value: 0.7617
- task:
type: text-classification
name: Difficulty classification (3 classes)
dataset:
name: AskedBefore CS Interview Questions
type: adithegod/askedbefore-cs-interview-questions
metrics:
- type: f1
name: Macro F1 (5-fold CV)
value: 0.564
---
# PrepMap Question Classifier
Two scikit-learn models that power the **PrepMap** website:
1. **Subject classifier:** reads an interview or placement question and predicts its subject (15 classes, such as DSA, Operating Systems, DBMS & SQL, Quantitative Aptitude or Behavioral / HR).
2. **Difficulty classifier:** predicts whether the question is easy, medium or hard.
Combined with `company_rules.json`, the predictions tell PrepMap which companies a new question is recommended for (for example, a medium DSA question is recommended for Infosys, Oracle, Google and Amazon).
## Results
All scores are macro F1 from stratified 5-fold cross-validation on the 1,770 questions of the [AskedBefore dataset](https://huggingface.co/datasets/adithegod/askedbefore-cs-interview-questions).
### Subject (15 classes)
| Model | Macro F1 |
|---|---:|
| **Logistic Regression** (used) | **0.762** |
| Random Forest (baseline) | 0.628 |
| XGBoost | 0.603 |
Random guessing would score about 0.07 with 15 classes. Logistic Regression beats the Random Forest baseline by 13 points, which is typical for short text represented with sparse TF-IDF features.
### Difficulty (3 classes)
| Model | Macro F1 |
|---|---:|
| **Random Forest** (used) | **0.564** |
| Logistic Regression | 0.520 |
| XGBoost | 0.510 |
Random guessing would score about 0.33. Difficulty is much harder to predict than subject, because it depends on the concept being tested rather than the words used, so treat difficulty predictions as a rough guide.
## Model details
- **Features:** TF-IDF word 1-2 grams (English stop words removed, sublinear TF) combined with TF-IDF character 3-5 grams.
- **Subject model:** Logistic Regression, C = 10, balanced class weights.
- **Difficulty model:** Random Forest, 500 trees, balanced class weights.
- Each model is a single scikit-learn `Pipeline` (features + classifier), so it takes raw question text as input.
- Built with **scikit-learn 1.6.1**. Use the same version to load the files.
## Files
| File | What it is | Size |
|---|---|---:|
| `subject_model.joblib` | Subject classifier (TF-IDF + Logistic Regression) | 4.2 MB |
| `difficulty_model.joblib` | Difficulty classifier (TF-IDF + Random Forest) | 48.5 MB |
| `subject_label_encoder.joblib` | Maps subject numbers to names | < 1 KB |
| `difficulty_label_encoder.joblib` | Maps difficulty numbers to names | < 1 KB |
| `company_rules.json` | Subject weights and difficulty focus for 19 companies | 6 KB |
| `results.json` | Cross-validation scores for all models | < 1 KB |
## Usage
```python
# pip install scikit-learn==1.6.1 huggingface_hub joblib
import json, joblib
from huggingface_hub import hf_hub_download
repo = "adithegod/prepmap-question-classifier"
load = lambda f: joblib.load(hf_hub_download(repo, f))
subject_model = load("subject_model.joblib")
difficulty_model = load("difficulty_model.joblib")
le_subject = load("subject_label_encoder.joblib")
le_difficulty = load("difficulty_label_encoder.joblib")
rules = json.load(open(hf_hub_download(repo, "company_rules.json")))
question = "Explain the difference between a process and a thread."
subject = le_subject.inverse_transform(subject_model.predict([question]))[0]
difficulty = le_difficulty.inverse_transform(difficulty_model.predict([question]))[0]
relevance = {3: "core", 2: "frequent", 1: "occasional"}
companies = [(c, relevance[r["weights"][subject]]) for c, r in rules.items()
if subject in r["weights"] and difficulty in r["difficulty"]]
print(subject, difficulty) # e.g. Operating Systems, easy
print(companies) # e.g. [('TCS', 'occasional'), ('HCLTech', 'frequent'), ...]
```
**Safety note:** `.joblib` files are Python pickles, which can run code when loaded. Only load these files from this repository.
## Training data
[AskedBefore CS Interview Questions](https://huggingface.co/datasets/adithegod/askedbefore-cs-interview-questions): 1,770 original interview and placement questions across 15 subjects, with difficulty labels and company mappings for 19 companies.
## Limitations
- **Short, clean, English questions only.** Accuracy on long or messy real interview text will be lower.
- **Overlapping subjects get confused.** Programming Fundamentals, Operating Systems and DSA share vocabulary, as do Statistics and Machine Learning.
- **Difficulty is subjective,** and its predictions are only moderately reliable (0.56 macro F1).
- **Company recommendations are rules, not predictions.** They come from public descriptions of each company's hiring process, not from records of questions the company actually asked.
## License
CC BY 4.0. |