SMT-Learner

Transformer autoencoder for spatiotemporal movement trajectories (SMT), trained with movement performance-aware contrastive learning and post-training adaptive transfer. It produces trial-level embeddings of pediatric upper-limb reaching movements collected from gamified iPad assessments, and those embeddings support the exploration-exploitation (E-E) ratio, a biomarker of motor skill learning and rehabilitation progress.

Released with A Deep Learning Framework for Assessing Pediatric Upper-limb Motor Control and Gamified Mobile Rehabilitation (IEEE TNSRE). Code: https://github.com/RaihanSabique/SMT-Learner

Model

Input canonicalized (x, y, t) trajectory resampled to 512 points
Encoder linear stem 3 -> 64, fixed sinusoidal positional encoding, 2 post-norm transformer layers (8 heads, FFN 256, dropout 0.1)
Embedding mean pooling over 512 tokens -> Z_global in R^64
Metadata binary task-type indicator -> 2-layer MLP -> R^32, concatenated to give a 96-d fused code
Heads 2-layer projection head (R^64, contrastive only) and a single dense decoder (96 -> 512x3)
Parameters 260,704 (decoder 57.1%, transformer layers 38.3%)
Checkpoint 1.19 MB

The released weights are the EXP4 adaptive-transfer checkpoint: pre-trained on MSL and fine-tuned on MP by combined cross-task and cross-subject transfer.

Training data

  • MSL - Motor Skill Learning. S. Nemanich, R. Mia, and C. Kemmel-Bartletti, Motor skill learning in young children born preterm, Jan. 2026. DOI: 10.17605/OSF.IO/W4R2F ยท osf.io/w4r2f
  • MP - Motor Performance. C. Kemmel-Bartletti and S. Nemanich, Game-based mobile-health quantification of upper-limb motor performance in children with hemiparetic cerebral palsy, Jun. 2025. DOI: 10.17605/OSF.IO/74DJM ยท osf.io/74djm

Reported performance

Baseline comparison (Suppl. TABLE III of the paper):

Model Data rMSE Ep-Err Curve-Err T-Corr R-Corr S-Corr
STTraj2Vec MSL 0.386 0.095 3.666 0.982 0.647 -0.797
VAE MSL 0.412 0.089 4.758 0.136 0.096 -0.135
Seq2Seq MSL 0.190 0.215 3.619 -0.996 -0.653 0.819
Traj-MAE MSL 0.111 0.290 28.304 -0.960 -0.638 0.779
ConvTraj MSL 0.323 0.147 30.820 0.742 0.679 0.660
SMT-Learner MSL 0.086 0.072 1.577 0.893 0.644 0.970
SMT-Learner MP 0.001 0.005 1.720 0.860 0.910 0.955

SMT-Learner achieves the lowest reconstruction, endpoint and curvature error and the highest success correlation on both datasets. STTraj2Vec reaches a higher completion-time correlation but a negative success correlation, because it encodes temporal and spatial continuity without outcome constraints.

Adaptive transfer (EXP4) reduced the total loss from 1.05 [1.04, 1.06] to 0.12 [0.11, 0.13], an 88.6% improvement, over 5 seeds.

Inference cost

260,704 parameters and a 1.19 MB checkpoint run on a single CPU thread (tested on iPad): 14.4 ms to embed one trial and 0.042 ms for an incremental E-E update, so the biomarker can be recomputed between trials.

Usage

inference.py in this repository is self-contained - it bundles the model definition, the training-time trajectory normalization and the E-E estimator, so no other code is needed.

import os, sys
from huggingface_hub import hf_hub_download

sys.path.insert(0, os.path.dirname(hf_hub_download("raihan116/SMT-Learner", "inference.py")))
from inference import SMTLearnerInference

smt = SMTLearnerInference.from_pretrained("raihan116/SMT-Learner")

# paths: list of (x, y) touch sequences; time_diff_ms: matching timestamps
embeddings = smt.embed_trials(paths, time_diff_ms)     # (n_trials, 64)
ee_ratio = smt.exploration_exploitation(embeddings)    # (n_trials,)

Or from the command line:

python inference.py --csv trials.csv --out ee_results.csv

Trajectories must be normalized by inference.py, which applies the preprocessing the weights were trained with: spatial canonicalization plus a time channel expressed as [0, 1] movement progress. Passing raw timestamps puts the time channel well outside its training range and the embeddings are meaningless.

E-E hyperparameters follow the sensitivity analysis in the paper: window W=120, decay alpha=0.05, novelty weights beta1=0.10 and beta2=0.90.

Intended use and limitations

Intended for research on upper-limb motor assessment and rehabilitation monitoring. This is an assessment and monitoring model; no intervention data supports it, and it is not a diagnostic device. Results come from two planar touchscreen reaching tasks in children aged 5-12 in one geographic region. Behaviour on substantially different trajectory distributions, other age bands, or non-touchscreen capture is untested and would require retraining. Embeddings are derived from de-identified trial data only.

Citation

@article{mia2026smtlearner,
  title   = {A Deep Learning Framework for Assessing Pediatric Upper-limb Motor
             Control and Gamified Mobile Rehabilitation},
  author  = {Mia, Md Raihan and Ahamed, Sheikh Iqbal and Nemanich, Samuel},
  journal = {IEEE Transactions on Neural Systems and Rehabilitation Engineering},
  year    = {2026}
}
Downloads last month
34
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support