MoCA: features and activity labels from wearable motion sensors

MoCA is a model for wearable motion data from a 3-axis gyroscope and a 3-axis accelerometer. It was pretrained by hiding parts of the sensor signal and learning to fill them back in. That taught it what typical movement looks like, without needing activity labels.

• Paper: arXiv:2506.02260
• Code: github.com/HowonRyu/MoCA

Two ways to use it

	You give it	You get back	Use it when
1. Feature extraction	Your sensor recording	768 numbers (features) for every 4 seconds	You want to detect your own activities or events (falls, tremor, a specific exercise), or combine motion with clinical variables
2. Direct classification	Your sensor recording	An activity name for every 4 seconds, e.g. 'Sitting'	Your activities are covered by the 7 UCI-HAR classes (walking, stairs, sitting, standing, laying, transitions)

Both models are in this repository: the pretrained model at the top level, and the activity classifier in the ucihar-classifier folder.

Install

pip install torch transformers pandas scikit-learn

The first time you load a model, it downloads once (about 440 MB for the feature extractor, 340 MB for the classifier) and is cached after that.

Step 0: Make sure your recording matches ours (needed for both usages)

Your recording should be a table with one row per sample and these 6 columns:

Requirement	Value
Columns, in this order	gyro_x, gyro_y, gyro_z in rad/s, then acc_x, acc_y, acc_z in g (gravity included)
Sampling rate	50 Hz (50 rows per second)
Scaling	None. Use the raw units above, without normalization
Length	At least 4 seconds (200 rows)

import numpy as np
import pandas as pd

channels = ["gyro_x", "gyro_y", "gyro_z", "acc_x", "acc_y", "acc_z"]

# Your recording. Replace this stand-in with e.g. pd.read_csv("my_recording.csv")
df = pd.DataFrame(np.random.randn(50 * 60 * 10, 6), columns=channels)  # 10 minutes of random numbers

recording = df[channels]  # the 6 columns, in this order

That's all the preparation needed. The model cuts the recording into 4-second pieces (windows) itself, and a leftover piece shorter than 4 seconds at the end is ignored.

If your recording uses other units or another sampling rate, fix it first:

Your data	Fix
Acceleration in m/s²	df[["acc_x", "acc_y", "acc_z"]] /= 9.81
Rotation in deg/s	df[["gyro_x", "gyro_y", "gyro_z"]] *= np.pi / 180
Sampled at 100 Hz	recording = scipy.signal.resample_poly(recording, 1, 2, axis=0) (for 30 Hz use 5, 3)

Usage 1: Feature extraction (automatic feature engineering)

Classic activity recognition computes hand-crafted features for each window, such as mean, standard deviation or FFT energy, and trains a classifier on them. MoCA replaces the hand-crafted step by turning every 4 seconds of your recording into 768 learned features:

from transformers import AutoModel

extractor = AutoModel.from_pretrained("zou-research-lab/MoCA", trust_remote_code=True)
features = extractor.embed(recording)  # (150, 768): 10 minutes = 150 windows of 4 seconds

features is a regular table of numbers. Row i describes seconds 4*i to 4*i + 4 of the recording. Train any standard classifier on it with your own labels, such as logistic regression, a random forest or an SVM. You can also add columns such as age or diagnosis.

from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import train_test_split
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler

labels = np.random.randint(0, 2, len(features))  # stand-in: your label for each 4 seconds, e.g. 1 = event of interest, 0 = other

X_train, X_test, y_train, y_test = train_test_split(features, labels, test_size=0.25, random_state=0)
detector = make_pipeline(StandardScaler(), LogisticRegression(max_iter=1000)).fit(X_train, y_train)
print("accuracy:", detector.score(X_test, y_test))  # about 0.5 here, because the stand-in labels are random

# later, on a new recording prepared the same way as in Step 0:
predictions = detector.predict(extractor.embed(recording))

Tip: if your data comes from several people, split by person (e.g. sklearn.model_selection.GroupShuffleSplit) so the test set contains only people the classifier has never seen. A random split puts windows from the same person in both sets, which overstates accuracy.

Usage 2: Direct classification

The classifier in the ucihar-classifier folder is MoCA fine-tuned on UCI-HAR. It recognizes 7 activities: Walking, Walking-upstairs, Walking-downstairs, Sitting, Standing, Laying, and Transition (moving between postures, e.g. sit-to-stand). It scores 96.6% accuracy on the UCI-HAR test set (793 windows).

from transformers import AutoModelForSequenceClassification

classifier = AutoModelForSequenceClassification.from_pretrained(
    "zou-research-lab/MoCA", subfolder="ucihar-classifier", trust_remote_code=True
)

activities = classifier.predict(recording)  # one activity name for every 4 seconds, e.g. ['Sitting', 'Sitting', 'Walking', ...]
for i, activity in enumerate(activities[:5]):
    print(f"{4 * i:>3}-{4 * i + 4:<3}s  {activity}")

The stand-in data above is random numbers, so these names are meaningless until you use a real recording.

Before you rely on the results

• Training data: UCI-HAR, which has 30 adults aged 19-48 doing scripted activities in a lab with a smartphone on the waist. Other body positions (wrist, ankle), other devices, or other populations (older adults, patients with movement disorders) can lower accuracy. Label a small sample of your own data and check accuracy first.
• Sensor orientation: axis directions matter. Your axes should match UCI-HAR's waist-worn phone, where the x-axis is roughly vertical when standing.
• Research use only: this is not a medical device and has not been validated for diagnosis.
• License: CC BY-NC 4.0, non-commercial use only.

Glossary

• Window: a short fixed-length chunk of a recording, 4 s here. The model gives one output per window.
• Features (embedding): numbers that summarize a window. Similar movements get similar numbers.
• Fine-tuning: further training the whole model on a labeled dataset. The classifier in ucihar-classifier is MoCA fine-tuned on UCI-HAR.
• trust_remote_code=True: lets transformers run the model code stored in this repository (modeling_moca.py, readable in the Files tab). It is required because MoCA is not built into transformers.

Citation

If you use MoCA, please cite the paper: arXiv:2506.02260.