--- license: cc-by-4.0 library_name: numpy tags: - active-learning - tabular-classification - next-lab --- ![ask](https://cdn-uploads.huggingface.co/production/uploads/67d46bc5fe6ad6f6511d6f44/2kNVbLXV3h5bzAUnMSsOe.png) # Next Ask Next Ask decides **which unlabeled example to send to a person next** when labels are expensive. It ranks the pool by classifier uncertainty, shortlists the 40 least certain items, then picks items that are both uncertain and far from everything already labeled. A small softmax classifier retrains after each batch. **Space:** https://huggingface.co/spaces/thelamapi/Next-Ask — label the digits the selector asks for; the classifier retrains in your browser. ## Results (10 seeds, 360 held-out test digits, batches of 10, budget 200) | Strategy | Area under learning curve | Final accuracy | Labels to reach 90% (median) | |---|---:|---:|---:| | Uncertainty + diversity (Next Ask) | 0.8897 ± 0.0066 | 0.9583 | 70 (10/10 seeds) | | Uncertainty only | 0.8837 ± 0.0135 | 0.9569 | 70 (10/10 seeds) | | Random | 0.8508 ± 0.0115 | 0.9283 | 115 (10/10 seeds) | Mean test accuracy at selected label counts: | Labels | Random | Uncertainty | Uncertainty + diversity | |---:|---:|---:|---:| | 20 | 0.6508 | 0.6608 | 0.7036 | | 50 | 0.8386 | 0.8650 | 0.8736 | | 100 | 0.8864 | 0.9294 | 0.9314 | | 150 | 0.9131 | 0.9447 | 0.9492 | | 200 | 0.9283 | 0.9569 | 0.9583 | Paired by seed (same initial labels): hybrid − random AULC = +0.0388 ± 0.0113 (hybrid better in 10/10 seeds); hybrid − uncertainty = +0.0060 ± 0.0104 (6/10 seeds). ![Learning curves](preview.png) ## Method * Classifier: multinomial logistic regression on the 64 pixel intensities, full-batch gradient descent from zero weights (steps=150, lr=1.0, l2=0.001, chosen on a 200-example validation split, never on test). * Uncertainty: margin between the two most probable classes. * Diversity: farthest-first selection on the uncertain shortlist, score = (1 − margin) × distance to the nearest labeled or already chosen example. * The browser runs a line-by-line JavaScript port. Node parity on 4 fixed cases: max abs difference 1.1102230246251565e-16. ## Usage ```python from inference import load, predict, next_queries spec = load() # model.json, NumPy only probs = predict(spec, pixels) # pixels: (N, 64) ints 0..16 picks = next_queries(spec, pool_pixels, labeled_idx, labels, strategy="hybrid", batch=5) ``` `python inference.py --selftest` reloads the model and checks stored outputs. ## Data Optical Recognition of Handwritten Digits (8x8) — UCI ML Repository id 80 via sklearn.datasets.load_digits, license CC BY 4.0. Split: 1237 pool / 200 validation / 360 test, stratified, seed 73. ## Limitations * 8×8 digits are an easy, balanced benchmark; gains from active learning are usually smaller on noisy, imbalanced real data. * The simulation assumes a perfect annotator (dataset labels). The Space lets people label, but no human study was run. * Final model in `model.json` is the seed-0 hybrid run at the full budget, kept for demonstration (17 of 360 test digits wrong). * Hardware for the recorded run: NVIDIA L4. Profile: full. Files: `model.json` (weights), `ask_core.py` (selectors and training), `inference.py`, `nlrt.py`, `config.json`, `evaluation.json` (all numbers, seeds, failures, tests).