deepmreye 2.0.0

README.md

@@ -0,0 +1,94 @@
+---
+license: lgpl-3.0
+library_name: deepmreye
+tags:
+- fmri
+- neuroimaging
+- eye-tracking
+- gaze
+datasets:
+- DeepMReye/eyeballs
+---
+
+# DeepMReye 2.0
+
+Decode eye gaze from fMRI without an eye tracker. The model reads gaze from
+the eyeballs in a BOLD run: a frozen linear basis fitted on **2,060
+unlabelled participants from 652 OpenNeuro datasets** by canonical
+correlation between the left and right orbit, and a ridge readout fitted on
+**337 gaze-labelled participants from 9 datasets**.
+
+```bash
+pip install git+https://github.com/DeepMReye/DeepMReye.git@deepmreye-unsupervised
+deepmreye decode sub-01_task-movie_bold.nii.gz -o gaze/
+```
+
+```python
+import deepmreye
+result = deepmreye.decode("sub-01_task-movie_bold.nii.gz")
+result["gaze"]        # [T, 10, 2]: ten gaze samples per volume, x then y
+result["confidence"]  # agreement of the two eyes' decodings
+```
+
+## Files
+
+| file | what |
+|---|---|
+| `decoder.npz` | the decoder (basis at 32 pairs, +-1 TR of context, ridge readout); `Decoder.pretrained()` |
+| `basis.npz` | the basis at 256 pairs, for training your own readout and for evaluation |
+| `evaluation.json` | every number of the evaluation below |
+
+## Output
+
+Ten gaze samples per volume (sample j covers [(t + j/10) TR, (t + (j+1)/10) TR)),
+x then y, x to the right and y **downwards** (screen coordinates). Units are the
+training datasets' standard deviations: gaze was z-scored per dataset before the
+readout was fitted, because screen size and viewing distance differ between
+studies. For degrees, calibrate on a few participants with an eye tracker
+(`deepmreye calibrate`), which fits one gain and offset per axis.
+
+## Evaluation
+
+Leave-one-dataset-out: the readout is refitted on eight labelled datasets and
+scored on every participant of the ninth. Pearson r between decoded and measured
+gaze per participant (mean of x and y), median over a dataset's participants,
+then over datasets.
+
+| | median over datasets | mean over datasets |
+|---|---|---|
+| sub-TR (10 samples per volume) | 0.771 | 0.721 |
+| per TR | 0.835 | 0.777 |
+
+Per dataset (sub-TR / per TR):
+
+| dataset | n | sub-TR | per TR |
+|---|---|---|---|
+| `dsL01_guided_fixations` | 170 | 0.771 | 0.787 |
+| `dsL02_pursuit` | 9 | 0.920 | 0.930 |
+| `dsL03_pursuit` | 24 | 0.776 | 0.810 |
+| `dsL04_pursuit` | 34 | 0.851 | 0.863 |
+| `dsL05_free_viewing` | 27 | 0.803 | 0.839 |
+| `dsL06_sequences` | 6 | 0.667 | 0.731 |
+| `dsL07_deepmreye_calib` | 15 | 0.743 | 0.835 |
+| `dsL08_studyforrest_movie` | 14 | 0.285 | 0.351 |
+| `dsL11_backtothefuture` | 37 | 0.672 | 0.845 |
+
+The labelled datasets are the `dsL*` folders of
+[DeepMReye/eyeballs](https://huggingface.co/datasets/DeepMReye/eyeballs). No
+participant of them, nor any run of the OpenNeuro accessions three of them were
+taken from, is in the basis's corpus.
+
+## Confidence
+
+The readout is linear, so applying it to each orbit alone gives two decodings
+whose average is the real one. Their agreement ranks a dataset's participants by
+how well they decode, without labels (`Decoder.confidence`). It does not compare
+datasets, and does not flag single volumes.
+
+## Limits
+
+- The eyes must be inside the field of view and the registration must hold:
+  check the QA image (`deepmreye decode --qa`).
+- Gaze is decoded relative to each participant's own gaze distribution; absolute
+  position needs calibration.
+- Built 2026-10-14 from deepmreye 2.0.0.

basis.npz

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f8b705483ac0f46b29c579c7f0f2d050f88cbac42b90636facbd812025748613
+size 13641449

decoder.npz

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:87afc0cfce5c03dfef4cc82a3e988ded8f2b408e9e1299cfa27b2894a21300da
+size 1787708