|
Download README.md from thelamapi/Next-Ask: direct link, hf CLI and curl.
- Browser
- Download file 3.34 kB
-
https://huggingface.co/spaces/thelamapi/Next-Ask/resolve/main/README.md
- Command line
-
hf download hf://spaces/thelamapi/Next-Ask/README.md
-
curl -L -o README.md https://huggingface.co/spaces/thelamapi/Next-Ask/resolve/main/README.md
3.34 kB
| title: Next Ask | |
| emoji: 🏷️ | |
| colorFrom: green | |
| colorTo: gray | |
| sdk: static | |
| app_file: index.html | |
| pinned: false | |
| license: cc-by-4.0 | |
| short_description: Pick the next most valuable label, live in the browser | |
| # Next Ask | |
| Next Ask decides **which unlabeled example to send to a person next** when labels are expensive. It ranks the pool by | |
| classifier uncertainty, shortlists the 40 least certain items, then picks items that are both uncertain and far from | |
| everything already labeled. A small softmax classifier retrains after each batch. | |
| **Model:** https://huggingface.co/thelamapi/Next-Ask — label the digits the selector asks for; the classifier retrains in your browser. | |
| ## Results (10 seeds, 360 held-out test digits, batches of 10, budget 200) | |
| | Strategy | Area under learning curve | Final accuracy | Labels to reach 90% (median) | | |
| |---|---:|---:|---:| | |
| | Uncertainty + diversity (Next Ask) | 0.8897 ± 0.0066 | 0.9583 | 70 (10/10 seeds) | | |
| | Uncertainty only | 0.8837 ± 0.0135 | 0.9569 | 70 (10/10 seeds) | | |
| | Random | 0.8508 ± 0.0115 | 0.9283 | 115 (10/10 seeds) | | |
| Mean test accuracy at selected label counts: | |
| | Labels | Random | Uncertainty | Uncertainty + diversity | | |
| |---:|---:|---:|---:| | |
| | 20 | 0.6508 | 0.6608 | 0.7036 | | |
| | 50 | 0.8386 | 0.8650 | 0.8736 | | |
| | 100 | 0.8864 | 0.9294 | 0.9314 | | |
| | 150 | 0.9131 | 0.9447 | 0.9492 | | |
| | 200 | 0.9283 | 0.9569 | 0.9583 | | |
| Paired by seed (same initial labels): hybrid − random AULC = +0.0388 ± 0.0113 | |
| (hybrid better in 10/10 seeds); hybrid − uncertainty = +0.0060 ± 0.0104 | |
| (6/10 seeds). | |
|  | |
| ## Method | |
| * Classifier: multinomial logistic regression on the 64 pixel intensities, full-batch gradient descent from zero weights | |
| (steps=150, lr=1.0, l2=0.001, chosen on a 200-example validation split, never on test). | |
| * Uncertainty: margin between the two most probable classes. | |
| * Diversity: farthest-first selection on the uncertain shortlist, score = (1 − margin) × distance to the nearest labeled or already chosen example. | |
| * The browser runs a line-by-line JavaScript port. Node parity on 4 fixed cases: max abs difference 1.1102230246251565e-16. | |
| ## Usage | |
| ```python | |
| from inference import load, predict, next_queries | |
| spec = load() # model.json, NumPy only | |
| probs = predict(spec, pixels) # pixels: (N, 64) ints 0..16 | |
| picks = next_queries(spec, pool_pixels, labeled_idx, labels, strategy="hybrid", batch=5) | |
| ``` | |
| `python inference.py --selftest` reloads the model and checks stored outputs. | |
| ## Data | |
| Optical Recognition of Handwritten Digits (8x8) — UCI ML Repository id 80 via sklearn.datasets.load_digits, license CC BY 4.0. Split: 1237 pool / | |
| 200 validation / 360 test, stratified, seed 73. | |
| ## Limitations | |
| * 8×8 digits are an easy, balanced benchmark; gains from active learning are usually smaller on noisy, imbalanced real data. | |
| * The simulation assumes a perfect annotator (dataset labels). The Space lets people label, but no human study was run. | |
| * Final model in `model.json` is the seed-0 hybrid run at the full budget, kept for demonstration (17 of 360 test digits wrong). | |
| * Hardware for the recorded run: NVIDIA L4. Profile: full. | |
| Files: `model.json` (weights), `ask_core.py` (selectors and training), `inference.py`, `nlrt.py`, `config.json`, `evaluation.json` (all numbers, seeds, failures, tests). | |