Instructions to use Zipeng365/WISP with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Scikit-learn
How to use Zipeng365/WISP with Scikit-learn:
from huggingface_hub import hf_hub_download import joblib model = joblib.load( hf_hub_download("Zipeng365/WISP", "sklearn_model.joblib") ) # only load pickle files from sources you trust # read more about it here https://skops.readthedocs.io/en/stable/persistence.html - Notebooks
- Google Colab
- Kaggle
File size: 7,709 Bytes
4731b11 93a9282 19383e1 4731b11 19383e1 4731b11 19383e1 4731b11 19383e1 4731b11 19383e1 4731b11 93a9282 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 | # Speed tiers: `speed="exact"` and `speed="fast"`
wisp-har exposes exactly two speed tiers. Both run the frozen paper algorithms; they differ
only in how the work is executed.
| Parameter | Promise | Implementation chosen |
|---|---|---|
| `speed="exact"` (default) | Outputs identical to the paper code | The 0.2.0 accelerated engine (`wisp_fast`) with the original Ridge solver |
| `speed="fast"` | Fastest; results can change slightly | The same engine with the screened Ridge solver (Ridge strength chosen on a 512-row training subsample, then one Ridge fit on all rows). Only Ridge readouts change: WISP-RC, WISP-CIS and its subsets, the RC/CIS members of WISP-CSE and WISP-ESE, RC/CIS search candidates |
Statements about paper results apply to `speed="exact"` only. The optional C kernels change
nothing but time: without a C compiler the engine runs pure NumPy paths with the same outputs
(the full test suite, including the equivalence tests below, passes both ways).
All numbers on this page are this package's own benchmark output, not paper results.
`speed` applies to the fixed recipes, WISPSelect and the Random, Evolution and Optuna searches.
WISP-Funnel (`WISPSearch(strategy="funnel")`) has one protocol and its own engine
([below](#the-wisp-funnel-engine)); `speed` does not change it.
## The exact tier: identical to the paper code, 5.7 times faster
`speed="exact"` was run next to the paper code (wisp-har 0.1.0, `wisp.registry.build_method`)
on the same cells:
* **Cells:** all seven fixed families on all twelve data streams, fold 0, seed 42 (84 cells),
plus ADL-Wrist fold 0 seeds 43 and 44 (14 cells): 98 cells.
* **Protocol:** the paper's fixed-family runner: fit on train+validation with participant
groups and time index, predict the test split with the family's HMM decoder; per-window
class scores saved for comparison.
* **Environment:** Python 3.12.13 with the pinned NumPy 2.3.5, SciPy 1.17.1, scikit-learn
1.7.2; one thread (BLAS and `n_jobs` = 1); every run on the same CPU type (Intel Xeon
Platinum 8360Y, 2.40 GHz); the paper code and the package ran one after another on the same
node for each cell.
* **Script:** `benchmarks/speed_engines.py`, summarised by `benchmarks/summarize_speed.py` into
`benchmarks/results/speed_engines.csv` and `.md`.
**Identity.** On all 98 cells `speed="exact"` reproduces the paper code bit for bit: 0
decoded-label mismatches, 0 per-window argmax mismatches, maximum absolute class-score
difference 0.
**Time** (fit + predict, seconds summed over the 12 streams, fold 0, seed 42):
| Family | Paper code | `speed="exact"` | Speed-up |
|---|---:|---:|---:|
| WISP-SS | 934 | 462 | 2.02x |
| WISP-RC | 5,458 | 253 | 21.59x |
| WISP-SO | 655 | 436 | 1.50x |
| WISP-GIS | 2,623 | 584 | 4.49x |
| WISP-CIS | 6,845 | 955 | 7.17x |
| WISP-CSE | 4,850 | 904 | 5.37x |
| WISP-ESE | 7,576 | 1,515 | 5.00x |
| **All 84 cells** | **28,942** | **5,108** | **5.67x** |
## The fast tier: screened Ridge
`speed="fast"` keeps everything of the exact tier except the Ridge regularisation search: the
Ridge strength is chosen on a 512-row subsample of the training data, then one Ridge fit runs
on all rows. This is the `--speed fast` option of 0.2.0. Time on the same cells (Ridge
families, fold 0, seed 42, same node type and protocol):
| Family | Paper code | `speed="exact"` | `speed="fast"` |
|---|---:|---:|---:|
| WISP-RC, 12 streams | 5,458 s | 253 s | 103 s |
| WISP-CIS, 12 streams | 6,845 s | 955 s | 202 s |
The fast tier was chosen for its time and for having an accuracy record over all paper cells
(next section), not for scoring better on test data.
## Accuracy record of the fast tier and all-cell check
Every catalog recipe with a fixed configuration was refitted with `WISPClassifier` on all 180
paper cells (12 streams x 5 folds x seeds 42/43/44, train+validation fit, test prediction, one
thread) and compared with the saved results (`benchmarks/check_against_saved_results.py`;
outputs `benchmarks/results/saved_results_check.csv` and `.md`; Slurm jobs 61805039 and
61806253, plus 15 helper jobs for the last WISDM cells).
**`speed="exact"`: 3,599 of the 3,600 cells reproduce the saved four metrics exactly** (largest
difference 1.1e-16, the rounding of the CSV's decimal text): the seven fixed families with the
HMM (1,260 cells), WISP-CIS without the HMM (180) and the six CIS block subsets with and
without the HMM (2,160). The remaining cell, CAPTURE-24 fold 2 seed 43 WISP-CSE, cannot be
fitted in the pinned environment: scikit-learn's `RidgeClassifierCV` stops with
`LinAlgError: SVD did not converge` in the RC member. The paper code itself (wisp-har 0.1.0)
fails identically with 1 and 8 BLAS threads, and so does `WISPClassifier` on three CPU
generations (FAILURES.md, entries 6β7); the saved result was presumably produced in an environment where
the SVD converged. With `speed="fast"` the same cell fits and gives the saved metrics.
**`speed="fast"`: change of the paper average macro-F1 (nine views, all folds and seeds)**:
| Recipe | Exact | Fast | Change |
|---|---:|---:|---:|
| `"cis"` (HMM) | 66.14 | 65.85 | β0.28 |
| `"cis"`, `decoder="none"` | 61.12 | 61.04 | β0.08 |
| `"rc"` | 63.44 | 62.26 | β1.19 |
| `"cse"` | 58.74 | 58.56 | β0.18 |
| `"ese"` | 57.38 | 57.35 | β0.03 |
SS, SO and GIS have no Ridge readout and are identical in both tiers (the engine builds them
through the same code path). Mean macro-F1 change per data stream (15 cells each; range over
cells in brackets):
| Stream | `"cis"` | `"rc"` |
|---|---|---|
| ADL-Wrist | +0.00 (+0.00 to +0.00) | β0.32 (β2.17 to +0.00) |
| CAPTURE-24 | +0.00 (+0.00 to +0.00) | +0.00 (+0.00 to +0.00) |
| DOMINO | +0.00 (+0.00 to +0.00) | β1.07 (β6.96 to +0.85) |
| GOTOV | β1.72 (β3.43 to β0.28) | β3.35 (β7.47 to +1.86) |
| HANDY | β0.35 (β1.46 to +0.36) | β1.04 (β2.56 to +0.26) |
| HARMES left wrist | +0.00 (+0.00 to +0.00) | +0.00 (+0.00 to +0.00) |
| HARMES right wrist | +0.00 (+0.00 to +0.00) | β2.87 (β3.57 to β2.13) |
| IUWDS | +0.24 (β1.62 to +4.54) | β1.13 (β5.85 to +2.10) |
| mHealth | +0.68 (β4.60 to +5.09) | +0.39 (β0.97 to +6.49) |
| PAAL-ADL | +0.00 (+0.00 to +0.00) | β1.84 (β12.62 to +0.00) |
| PAMAP2 | +0.00 (β0.00 to +0.00) | β1.50 (β6.35 to +0.61) |
| WISDM | β0.73 (β2.81 to +0.42) | β0.50 (β3.93 to +1.56) |
Where the screened subsample picks the same Ridge strength as the full leave-one-out search,
the results are identical; elsewhere they change in both directions, by up to several points
on single cells. The per-stream tables for CSE, ESE and direct CIS are in
`benchmarks/results/saved_results_check.md`. The 0.2.0 release notes quoted per-dataset
changes for other cells (GOTOV β3.44, HANDY +3.12 macro-F1 for WISP-CIS); the table above
supersedes them as the fast tier's record over all 180 cells.
## The WISP-Funnel engine
WISP-Funnel runs on a separate engine, `wisp_fast.funnel`, that shares no module with the
engine above, so neither changes the other's results. Its verification and refit use the
screened Ridge solver as part of the funnel's protocol. It has four optional C kernels
(`wisp_fast.funnel._kernels`, `_family_kernels`, `_cis_intervals`, `_cis_lite`), compiled with
the same flags as the kernels above. On two benchmark cells (ADL-Wrist fold 1 seed 42 and
fold 3 seed 44) a build without a C compiler gave the same winners, test class scores and
labels as the compiled build and the benchmark record, in 1.6 to 1.7 times the time
(31 s against 18 s, and 56 s against 35 s, for the whole joint run). The funnel's accuracy and
cost record is in [RECIPES.md](RECIPES.md#wisp-funnel-a-fast-search).
|