| --- |
| license: cc-by-4.0 |
| language: |
| - en |
| library_name: sklearn |
| pipeline_tag: text-classification |
| tags: |
| - text-classification |
| - tabular-classification |
| - interview-questions |
| - placements |
| - scikit-learn |
| datasets: |
| - adithegod/askedbefore-cs-interview-questions |
| metrics: |
| - f1 |
| model-index: |
| - name: prepmap-question-classifier |
| results: |
| - task: |
| type: text-classification |
| name: Subject classification (15 classes) |
| dataset: |
| name: AskedBefore CS Interview Questions |
| type: adithegod/askedbefore-cs-interview-questions |
| metrics: |
| - type: f1 |
| name: Macro F1 (5-fold CV) |
| value: 0.7617 |
| - task: |
| type: text-classification |
| name: Difficulty classification (3 classes) |
| dataset: |
| name: AskedBefore CS Interview Questions |
| type: adithegod/askedbefore-cs-interview-questions |
| metrics: |
| - type: f1 |
| name: Macro F1 (5-fold CV) |
| value: 0.564 |
| --- |
| |
| # PrepMap Question Classifier |
|
|
| Two scikit-learn models that power the **PrepMap** website: |
|
|
| 1. **Subject classifier:** reads an interview or placement question and predicts its subject (15 classes, such as DSA, Operating Systems, DBMS & SQL, Quantitative Aptitude or Behavioral / HR). |
| 2. **Difficulty classifier:** predicts whether the question is easy, medium or hard. |
|
|
| Combined with `company_rules.json`, the predictions tell PrepMap which companies a new question is recommended for (for example, a medium DSA question is recommended for Infosys, Oracle, Google and Amazon). |
|
|
| ## Results |
|
|
| All scores are macro F1 from stratified 5-fold cross-validation on the 1,770 questions of the [AskedBefore dataset](https://huggingface.co/datasets/adithegod/askedbefore-cs-interview-questions). |
|
|
| ### Subject (15 classes) |
|
|
| | Model | Macro F1 | |
| |---|---:| |
| | **Logistic Regression** (used) | **0.762** | |
| | Random Forest (baseline) | 0.628 | |
| | XGBoost | 0.603 | |
|
|
| Random guessing would score about 0.07 with 15 classes. Logistic Regression beats the Random Forest baseline by 13 points, which is typical for short text represented with sparse TF-IDF features. |
|
|
| ### Difficulty (3 classes) |
|
|
| | Model | Macro F1 | |
| |---|---:| |
| | **Random Forest** (used) | **0.564** | |
| | Logistic Regression | 0.520 | |
| | XGBoost | 0.510 | |
|
|
| Random guessing would score about 0.33. Difficulty is much harder to predict than subject, because it depends on the concept being tested rather than the words used, so treat difficulty predictions as a rough guide. |
|
|
| ## Model details |
|
|
| - **Features:** TF-IDF word 1-2 grams (English stop words removed, sublinear TF) combined with TF-IDF character 3-5 grams. |
| - **Subject model:** Logistic Regression, C = 10, balanced class weights. |
| - **Difficulty model:** Random Forest, 500 trees, balanced class weights. |
| - Each model is a single scikit-learn `Pipeline` (features + classifier), so it takes raw question text as input. |
| - Built with **scikit-learn 1.6.1**. Use the same version to load the files. |
|
|
| ## Files |
|
|
| | File | What it is | Size | |
| |---|---|---:| |
| | `subject_model.joblib` | Subject classifier (TF-IDF + Logistic Regression) | 4.2 MB | |
| | `difficulty_model.joblib` | Difficulty classifier (TF-IDF + Random Forest) | 48.5 MB | |
| | `subject_label_encoder.joblib` | Maps subject numbers to names | < 1 KB | |
| | `difficulty_label_encoder.joblib` | Maps difficulty numbers to names | < 1 KB | |
| | `company_rules.json` | Subject weights and difficulty focus for 19 companies | 6 KB | |
| | `results.json` | Cross-validation scores for all models | < 1 KB | |
|
|
| ## Usage |
|
|
| ```python |
| # pip install scikit-learn==1.6.1 huggingface_hub joblib |
| import json, joblib |
| from huggingface_hub import hf_hub_download |
| |
| repo = "adithegod/prepmap-question-classifier" |
| load = lambda f: joblib.load(hf_hub_download(repo, f)) |
| |
| subject_model = load("subject_model.joblib") |
| difficulty_model = load("difficulty_model.joblib") |
| le_subject = load("subject_label_encoder.joblib") |
| le_difficulty = load("difficulty_label_encoder.joblib") |
| rules = json.load(open(hf_hub_download(repo, "company_rules.json"))) |
| |
| question = "Explain the difference between a process and a thread." |
| subject = le_subject.inverse_transform(subject_model.predict([question]))[0] |
| difficulty = le_difficulty.inverse_transform(difficulty_model.predict([question]))[0] |
| |
| relevance = {3: "core", 2: "frequent", 1: "occasional"} |
| companies = [(c, relevance[r["weights"][subject]]) for c, r in rules.items() |
| if subject in r["weights"] and difficulty in r["difficulty"]] |
| |
| print(subject, difficulty) # e.g. Operating Systems, easy |
| print(companies) # e.g. [('TCS', 'occasional'), ('HCLTech', 'frequent'), ...] |
| ``` |
|
|
| **Safety note:** `.joblib` files are Python pickles, which can run code when loaded. Only load these files from this repository. |
|
|
| ## Training data |
|
|
| [AskedBefore CS Interview Questions](https://huggingface.co/datasets/adithegod/askedbefore-cs-interview-questions): 1,770 original interview and placement questions across 15 subjects, with difficulty labels and company mappings for 19 companies. |
|
|
| ## Limitations |
|
|
| - **Short, clean, English questions only.** Accuracy on long or messy real interview text will be lower. |
| - **Overlapping subjects get confused.** Programming Fundamentals, Operating Systems and DSA share vocabulary, as do Statistics and Machine Learning. |
| - **Difficulty is subjective,** and its predictions are only moderately reliable (0.56 macro F1). |
| - **Company recommendations are rules, not predictions.** They come from public descriptions of each company's hiring process, not from records of questions the company actually asked. |
|
|
| ## License |
|
|
| CC BY 4.0. |
