Download README.md from thelamapi/Next-Ask: direct link, hf CLI and curl.
- Browser
- Download file 3.34 kB
-
https://huggingface.co/spaces/thelamapi/Next-Ask/resolve/main/README.md
- Command line
-
hf download hf://spaces/thelamapi/Next-Ask/README.md
-
curl -L -o README.md https://huggingface.co/spaces/thelamapi/Next-Ask/resolve/main/README.md
title: Next Ask
emoji: 🏷️
colorFrom: green
colorTo: gray
sdk: static
app_file: index.html
pinned: false
license: cc-by-4.0
short_description: Pick the next most valuable label, live in the browser
Next Ask
Next Ask decides which unlabeled example to send to a person next when labels are expensive. It ranks the pool by classifier uncertainty, shortlists the 40 least certain items, then picks items that are both uncertain and far from everything already labeled. A small softmax classifier retrains after each batch.
Model: https://huggingface.co/thelamapi/Next-Ask — label the digits the selector asks for; the classifier retrains in your browser.
Results (10 seeds, 360 held-out test digits, batches of 10, budget 200)
| Strategy | Area under learning curve | Final accuracy | Labels to reach 90% (median) |
|---|---|---|---|
| Uncertainty + diversity (Next Ask) | 0.8897 ± 0.0066 | 0.9583 | 70 (10/10 seeds) |
| Uncertainty only | 0.8837 ± 0.0135 | 0.9569 | 70 (10/10 seeds) |
| Random | 0.8508 ± 0.0115 | 0.9283 | 115 (10/10 seeds) |
Mean test accuracy at selected label counts:
| Labels | Random | Uncertainty | Uncertainty + diversity |
|---|---|---|---|
| 20 | 0.6508 | 0.6608 | 0.7036 |
| 50 | 0.8386 | 0.8650 | 0.8736 |
| 100 | 0.8864 | 0.9294 | 0.9314 |
| 150 | 0.9131 | 0.9447 | 0.9492 |
| 200 | 0.9283 | 0.9569 | 0.9583 |
Paired by seed (same initial labels): hybrid − random AULC = +0.0388 ± 0.0113 (hybrid better in 10/10 seeds); hybrid − uncertainty = +0.0060 ± 0.0104 (6/10 seeds).
Method
- Classifier: multinomial logistic regression on the 64 pixel intensities, full-batch gradient descent from zero weights (steps=150, lr=1.0, l2=0.001, chosen on a 200-example validation split, never on test).
- Uncertainty: margin between the two most probable classes.
- Diversity: farthest-first selection on the uncertain shortlist, score = (1 − margin) × distance to the nearest labeled or already chosen example.
- The browser runs a line-by-line JavaScript port. Node parity on 4 fixed cases: max abs difference 1.1102230246251565e-16.
Usage
from inference import load, predict, next_queries
spec = load() # model.json, NumPy only
probs = predict(spec, pixels) # pixels: (N, 64) ints 0..16
picks = next_queries(spec, pool_pixels, labeled_idx, labels, strategy="hybrid", batch=5)
python inference.py --selftest reloads the model and checks stored outputs.
Data
Optical Recognition of Handwritten Digits (8x8) — UCI ML Repository id 80 via sklearn.datasets.load_digits, license CC BY 4.0. Split: 1237 pool / 200 validation / 360 test, stratified, seed 73.
Limitations
- 8×8 digits are an easy, balanced benchmark; gains from active learning are usually smaller on noisy, imbalanced real data.
- The simulation assumes a perfect annotator (dataset labels). The Space lets people label, but no human study was run.
- Final model in
model.jsonis the seed-0 hybrid run at the full budget, kept for demonstration (17 of 360 test digits wrong). - Hardware for the recorded run: NVIDIA L4. Profile: full.
Files: model.json (weights), ask_core.py (selectors and training), inference.py, nlrt.py, config.json, evaluation.json (all numbers, seeds, failures, tests).
