Instructions to use Zipeng365/WISP with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Scikit-learn
How to use Zipeng365/WISP with Scikit-learn:
from huggingface_hub import hf_hub_download import joblib model = joblib.load( hf_hub_download("Zipeng365/WISP", "sklearn_model.joblib") ) # only load pickle files from sources you trust # read more about it here https://skops.readthedocs.io/en/stable/persistence.html - Notebooks
- Google Colab
- Kaggle
README: Pareto optimality and dominance counts for the cost and memory figures
Browse files- README.md +34 -16
- SHA256SUMS +1 -1
README.md
CHANGED
|
@@ -23,9 +23,22 @@ WISP (Wrist-Informed Space of Predictive Representations) is a CPU method family
|
|
| 23 |
## Why WISP
|
| 24 |
|
| 25 |
WISP is accurate, fast and small, and it plugs into the tools you already use. Each figure below
|
| 26 |
-
places every method by one score against one resource
|
| 27 |
-
|
| 28 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 29 |
|
| 30 |
### 1. Accurate and fast
|
| 31 |
|
|
@@ -46,11 +59,13 @@ selection at all. One NVIDIA A100 hour counts as 64 core-hours. Every plotted va
|
|
| 46 |
0.712 for WISP-Evolution. The final models of tuned RDST, MR-Hydra, MultiRocket and
|
| 47 |
MiniRocket cost 34.15, 45.52, 84.63 and 15.53 core-hours, 50–272 times WISP-Random's. WISP
|
| 48 |
runs on CPUs only; no GPU is needed.
|
| 49 |
-
- **
|
| 50 |
-
|
| 51 |
-
|
| 52 |
-
|
| 53 |
-
|
|
|
|
|
|
|
| 54 |
|
| 55 |
### 2. Good on every class, at a fraction of the time
|
| 56 |
|
|
@@ -61,9 +76,10 @@ ones) against the same final-model cost axis.*
|
|
| 61 |
|
| 62 |
Wearable activity data are imbalanced: a few activities fill most of the day. Balanced accuracy
|
| 63 |
rewards a model only if it recognises the rare activities too. WISP-Random (70.88) and
|
| 64 |
-
WISP-Evolution (70.17) are the two most balanced methods in the comparison
|
| 65 |
-
|
| 66 |
-
|
|
|
|
| 67 |
|
| 68 |
### 3. Small memory
|
| 69 |
|
|
@@ -82,11 +98,13 @@ baselines (their final refits were recorded with 1–8 threads, which is not com
|
|
| 82 |
these single-thread peaks), HC2 (32 threads) and the deep models (GPU memory). Values:
|
| 83 |
[`docs/figures/memory_plotted_values.csv`](docs/figures/memory_plotted_values.csv).*
|
| 84 |
|
| 85 |
-
|
| 86 |
-
|
| 87 |
-
|
| 88 |
-
|
| 89 |
-
|
|
|
|
|
|
|
| 90 |
|
| 91 |
### 4. Extensible
|
| 92 |
|
|
|
|
| 23 |
## Why WISP
|
| 24 |
|
| 25 |
WISP is accurate, fast and small, and it plugs into the tools you already use. Each figure below
|
| 26 |
+
places every method by one score against one resource, so the further up and to the left a
|
| 27 |
+
point sits, the better. A method is *Pareto-dominated* when another method needs no more of the
|
| 28 |
+
resource and scores higher; every point beneath the dashed gold line (the WISP Pareto frontier)
|
| 29 |
+
is dominated by a WISP method. WISP points use the paper protocol (`speed="exact"`).
|
| 30 |
+
|
| 31 |
+
**WISP-Random is Pareto-optimal in all eight figures** (four metrics, each against final-model
|
| 32 |
+
cost and against memory): no method is both cheaper, or smaller, and more accurate. It
|
| 33 |
+
dominates most baselines outright:
|
| 34 |
+
|
| 35 |
+
| Resource (same result for all four metrics) | Overall Pareto frontier | Baselines Pareto-dominated by WISP |
|
| 36 |
+
|---|---|---|
|
| 37 |
+
| Final-model cost | WISP-Random, Balanced-RF | **14 of 15** by WISP-Random and by WISP-Evolution; 11 by WISP-CIS, which needs no search |
|
| 38 |
+
| Peak memory | WISP-Random, QUANT, Balanced-RF | **4 of 6** by each of WISP-Random, WISP-Evolution and WISP-CIS |
|
| 39 |
+
|
| 40 |
+
The baselines that stay on the frontier do so only by being cheaper or smaller, not by being
|
| 41 |
+
more accurate: Balanced-RF scores 22.3 macro-F1 points below WISP-Random, and QUANT 14.9.
|
| 42 |
|
| 43 |
### 1. Accurate and fast
|
| 44 |
|
|
|
|
| 59 |
0.712 for WISP-Evolution. The final models of tuned RDST, MR-Hydra, MultiRocket and
|
| 60 |
MiniRocket cost 34.15, 45.52, 84.63 and 15.53 core-hours, 50–272 times WISP-Random's. WISP
|
| 61 |
runs on CPUs only; no GPU is needed.
|
| 62 |
+
- **Pareto-dominant.** WISP-Random and WISP-Evolution are both cheaper and more accurate than
|
| 63 |
+
14 of the 15 baselines, including every tuned baseline, HC2 and all four deep models. Fixed
|
| 64 |
+
WISP-CIS, with no search at all, dominates 11 of them; only Balanced-RF and the three strongest
|
| 65 |
+
tuned baselines (RDST, MR-Hydra, MultiRocket) escape it. Balanced-RF's final model (0.22
|
| 66 |
+
core-hours, 47.85 macro-F1) is the only one cheaper than every WISP point.
|
| 67 |
+
- **What the axis leaves out.** Configuration selection is not on it; a WISP search is expensive
|
| 68 |
+
there ([cost notes](docs/RECIPES.md#cost-notes)), while fixed WISP-CIS has no selection cost.
|
| 69 |
|
| 70 |
### 2. Good on every class, at a fraction of the time
|
| 71 |
|
|
|
|
| 76 |
|
| 77 |
Wearable activity data are imbalanced: a few activities fill most of the day. Balanced accuracy
|
| 78 |
rewards a model only if it recognises the rare activities too. WISP-Random (70.88) and
|
| 79 |
+
WISP-Evolution (70.17) are the two most balanced methods in the comparison, and both
|
| 80 |
+
Pareto-dominate 14 of the 15 baselines on this figure too. The best baseline, tuned RDST,
|
| 81 |
+
reaches 68.21 with a final model that costs 110 times WISP-Random's (34.15 against 0.311
|
| 82 |
+
core-hours).
|
| 83 |
|
| 84 |
### 3. Small memory
|
| 85 |
|
|
|
|
| 98 |
these single-thread peaks), HC2 (32 threads) and the deep models (GPU memory). Values:
|
| 99 |
[`docs/figures/memory_plotted_values.csv`](docs/figures/memory_plotted_values.csv).*
|
| 100 |
|
| 101 |
+
WISP-Random is again Pareto-optimal, and every WISP method Pareto-dominates RDST, MR-Hydra,
|
| 102 |
+
MultiRocket and MiniRocket on all four metrics. The final model of WISP-Random peaks at
|
| 103 |
+
4.7 GiB, WISP-Evolution at 8.0 GiB and WISP-CIS at 8.9 GiB, so all three fit on a laptop with
|
| 104 |
+
16 GB of memory. RDST, MR-Hydra and MultiRocket needed 104–114 GiB, about 22–24 times
|
| 105 |
+
WISP-Random, and MiniRocket 21.8 GiB, while scoring 10–11 points lower in macro-F1. QUANT
|
| 106 |
+
(4.4 GiB) and Balanced-RF (2.3 GiB) use less memory than WISP-Random, at 14.9 and 22.3 points
|
| 107 |
+
lower macro-F1.
|
| 108 |
|
| 109 |
### 4. Extensible
|
| 110 |
|
SHA256SUMS
CHANGED
|
@@ -4,7 +4,7 @@
|
|
| 4 |
0d9a22c8ec8957c6f775ff51432b7a33ca7dc9d68810f7d5553aa6ae82016239 CITATION.bib
|
| 5 |
a553ad9270a356f4610a5c0c837ed7393ba2e0959c1c36620b62f9a299796698 FAILURES.md
|
| 6 |
59e1ab474672b657909afcb346d7c6b2fc0fc627f6518b99925779aa1892b6c4 LICENSE
|
| 7 |
-
|
| 8 |
43f14bd6fe7e2a93fc525a26fd95ea9cce48cd11b8ad927f1d4e0fbd2147c01c RELEASE_SUMMARY.json
|
| 9 |
a621333daaa3a4d473eb30e9a1c13b3e8ec63e0fe0e21b0545ad64a7498bc2d1 SOURCE_PROVENANCE.json
|
| 10 |
75489085ceb877e9cbdf5ba0fd1aeae3e7da5d11982f8c8c72482770e4a1a225 benchmarks/README.md
|
|
|
|
| 4 |
0d9a22c8ec8957c6f775ff51432b7a33ca7dc9d68810f7d5553aa6ae82016239 CITATION.bib
|
| 5 |
a553ad9270a356f4610a5c0c837ed7393ba2e0959c1c36620b62f9a299796698 FAILURES.md
|
| 6 |
59e1ab474672b657909afcb346d7c6b2fc0fc627f6518b99925779aa1892b6c4 LICENSE
|
| 7 |
+
bf5b6960043b74b2458ccc68515130e0e4c28beb0ec1a2d2653ef500a698cc2c README.md
|
| 8 |
43f14bd6fe7e2a93fc525a26fd95ea9cce48cd11b8ad927f1d4e0fbd2147c01c RELEASE_SUMMARY.json
|
| 9 |
a621333daaa3a4d473eb30e9a1c13b3e8ec63e0fe0e21b0545ad64a7498bc2d1 SOURCE_PROVENANCE.json
|
| 10 |
75489085ceb877e9cbdf5ba0fd1aeae3e7da5d11982f8c8c72482770e4a1a225 benchmarks/README.md
|