README.md

metadata

license: mit
tags:
  - time-series
  - time-series-language-model
  - forecast-grounding
  - lora
  - arxiv:2610.18877
base_model:
  - google/gemma-3-270m
  - google/gemma-3-1b-pt
  - Qwen/Qwen3-0.6B
pipeline_tag: time-series-forecasting

LOTUS: Learning to Understand Time-series by Learning to Forecast

LOTUS is a time-series language model pretrained under forecast grounding — a pretraining principle that requires language-aligned time-series representations to remain predictive of the future evolution of the signal before the model is adapted to downstream tasks. Time-series language models typically align numerical signals with language using labels, captions, or question–answer pairs; such supervision encourages linguistic representations but does not explicitly require them to preserve information about how the underlying signal evolves. Forecast grounding closes that gap.

LOTUS divides each time-series channel into non-overlapping patches and maps them into the LLM embedding space. It then predicts the next patch from the LLM hidden states of the time-series tokens, encouraging the learned representation to retain temporal information. To capture both temporal changes and periodic patterns, each raw patch is combined with its spectral representation (temporal–spectral patch representation) before mapping into the LLM embedding space.

Across three language-model backbones and six downstream evaluation datasets, LOTUS achieves the best average performance in every matched-backbone group with minimal trainable parameters — 59.91% average accuracy with the Qwen3-0.6B backbone, outperforming the strongest matched baseline by 3.36% — and its 270M-parameter model is the top system on the TSRBench zero-shot reasoning benchmark ahead of backbones 25–50× larger.

Paper: arXiv:2610.18877 · Code: DA2I2-SLM/Lotus · Phase-1 (pretraining) checkpoints: hiepnh137/LangTimeFoundationModel

Training

• Phase 1 — forecast grounding (pretraining). The encoder, projector, next-patch decoder and LLM LoRA adapters are trained jointly on a mixture of forecasting (next-patch prediction from the LLM hidden states) and language (TSQA multiple-choice, M4 captioning) corpora.
• Phase 2 — downstream adaptation. The next-patch decoder is removed; the time-series interface and LLM LoRA adapters are trained with the language objective alone, jointly across HAR, Sleep-EDF, ECG-QA, PAMAP2 and Time-MQA.

All systems are evaluated under greedy decoding: the model writes a rationale followed by an explicit answer, which is parsed and compared with the reference. Classification accuracy is reported on HAR, PAMAP2, Sleep-EDF and Time-MQA (MCQ and true/false splits), and generative accuracy under normalization on ECG-QA.

Training data (as specified in the paper)

Phase 1 — forecast-grounded pretraining. Four corpora: Monash half-hourly electricity demand (Godahewala et al., 2021) and a synthetic multivariate corpus for the forecasting objective, and two language corpora — TSQA multiple-choice questions on trend, volatility, seasonality and outliers (Kong et al., 2025; HF: ChengsenWang/TSQA) and M4 series with generated captions (Makridakis et al., 2020).

The synthetic corpus contains 844,275 series from four generator families, none contributing more than 37% (paper Table 5):

Generator family	Series	Share	Variates	Length
Chronos-2 multivariatizer (re-implementation)	310,000	36.7%	3–10	512–8,192
KernelSynth, univariate (Ansari et al., 2024)	299,999	35.5%	1	1,024–4,096
TimePFN LMC (Taga et al., 2025)	215,000	25.5%	3–10	512–2,048
SymTime S²Generator (Wang et al., 2025)	19,276	2.3%	3–10	1,024–4,096

Phase 2 — downstream adaptation (the checkpoints in this repository). Five datasets, fine-tuned jointly:

Dataset	Task	Source
HAR	activity recognition from 3-axis accelerometer windows, with rationales (8 activities)	OpenTSLM release (Langer et al., 2026)
PAMAP2	activity recognition (18 activities)	Reiss & Stricker, 2012
Sleep-EDF	sleep staging from 30-second EEG segments (6 stages)	Kemp et al., 2000
ECG-QA	question answering over 12-lead ECG records	Oh et al., 2023, on PTB-XL (Wagner et al., 2020)
Time-MQA	open-ended time-series QA; MCQ and T/F splits scored separately	Kong et al., 2025 (HF: Time-MQA/TSQA, gated)

Generation recipes for the synthetic corpus (upstream repositories, commits and driver scripts) are in scripts/synthetic_corpus/ of the code repository; the downstream corpora are downloaded automatically by the trainer.

Checkpoints in this repository (Phase 2, joint five-task adaptation)

File	Backbone	Trainable params
lotus_qwen3-0.6b_joint-5task-finetune.pt	Qwen3-0.6B	9.3M
lotus_gemma-3-1b-pt_joint-5task-finetune.pt	Gemma3-1B	11.1M
lotus_gemma-3-270m_joint-5task-finetune.pt	Gemma3-270M	1.9M

The repository also contains the matched OpenTSLM-SP and OpenTSLM-Flamingo baseline checkpoints used in the paper (joint five-task and sequential-curriculum variants).

Results (six downstream splits)

Backbone	Method	Params	HAR	Sleep	ECG	PAMAP2	MCQ	T/F	Avg.
Gemma3-270M	OpenTSLM-SP	3.1M	54.38	65.12	9.66	52.14	38.59	68.39	48.05
Gemma3-270M	OpenTSLM-Flamingo	317M	62.55	68.01	30.10	54.70	12.37	3.04	38.46
Gemma3-270M	LOTUS	1.9M	58.38	70.54	33.16	55.15	39.41	74.47	55.19
Gemma3-1B	OpenTSLM-SP	5.5M	55.46	72.58	21.66	47.44	41.36	77.51	52.67
Gemma3-1B	LOTUS	11.1M	62.67	77.10	37.66	52.56	44.98	78.42	58.90
Qwen3-0.6B	OpenTSLM-SP	27.8M	63.25	70.54	32.20	47.44	54.64	71.20	56.55
Qwen3-0.6B	LOTUS	9.3M	63.66	77.20	31.58	53.42	56.72	76.90	59.91

ECG-QA accuracy is computed on the first 5,000 test items (manifest: eval_manifests/ecg_test_first5000.json in the code repository).

Use it

Everything needed to run the model — including the LOTUS code — is inside this repository. pip install torch transformers peft numpy (Python ≥ 3.10), then one setup cell:

from huggingface_hub import snapshot_download
import sys
sys.path.insert(0, snapshot_download("daidv1112/LOTUS",
                                     allow_patterns=["load_lotus.py", "src/*"]))
from load_lotus import load_lotus

lotus = load_lotus()          # downloads the checkpoint, picks GPU + dtype automatically

Case 1 — type the series in (any list of numbers, raw scale, in time order):

series = [12.4, 13.1, 12.8, 13.9, 14.5, 14.2, 15.3, 16.1, 15.8, 16.9,
          17.4, 17.1, 18.2, 19.0, 18.6, 19.5, 20.3, 20.0, 21.1, 21.8,
          21.5, 22.6, 23.4, 23.0, 24.1, 24.8, 24.5, 25.6, 26.3, 26.0,
          27.1, 27.9, 27.5, 28.6, 29.3, 29.0, 30.1, 30.8, 30.5, 31.6,
          32.3, 32.0, 33.1, 33.8, 33.5, 34.6, 35.2, 35.0]

lotus.ask("Is the time series showing an upward trend?", series)
# -> "True. The series shows an upward trend as it starts at 15.0 and increases to 34.0,
#     indicating a consistent increase over time."

Case 2 — read the series from a CSV file (name the column, or let it pick the first numeric one; non-numeric and missing values are dropped):

lotus.ask_csv("Does the time series show a clear seasonal pattern?", "your_data.csv", column="OT")
# -> "[LOTUS] using column 'OT' of your_data.csv: 651 points, range 7.744e+04 to 2.943e+05"
#    "No. Reasoning: The data shows fluctuations and patterns but does not exhibit a clear
#     seasonal pattern. While there are noticeable peaks and troughs, these do not occur at
#     regular intervals..."

Both methods take any natural-language question and an optional domain= context hint (Traffic, Energy, Healthcare, ...), and return the model's verdict plus its rationale.

Low-level quickstart (explicit model construction)

A checkpoint is a torch.save dictionary holding the time-series encoder, projector, next-patch decoder and LoRA adapters; the base LLM is loaded from its Hugging Face id. The model class is LotusModel (vendored in this repository under src/; equivalently available in the Lotus repository, Python ≥ 3.12, pip install -e .).

import sys
from types import SimpleNamespace
import torch
from huggingface_hub import hf_hub_download

sys.path.insert(0, "Lotus/src")   # git clone https://github.com/DA2I2-SLM/Lotus.git
from opentslm.model.llm.lotus_model import LotusModel
from opentslm.time_series_datasets.util import extend_time_series_to_match_patch_size_and_aggregate

cfg = SimpleNamespace(
    is_pretraining=True, patch_size=32, pe_type="sincos",
    use_only_time_pe=False, use_only_channel_pe=False,
    use_freq_embed=True, enc_dec_type="transformer",
    trans_enc_dim=192, trans_enc_num_layers=6, trans_enc_num_heads=8, trans_enc_ff_dim=896,
    trans_dec_dim=192, trans_dec_num_layers=4, trans_dec_num_heads=8, trans_dec_ff_dim=896,
    llm_mv_position_ids=True, llm_dtype="fp32",   # "bf16" on GPUs with bf16 support
)
model = LotusModel(cfg, llm_id="Qwen/Qwen3-0.6B", device="cuda")
model.enable_lora(lora_r=8, lora_alpha=16)
model.load_from_file(hf_hub_download("daidv1112/LOTUS", "lotus_qwen3-0.6b_joint-5task-finetune.pt"))
model.eval()

# activity recognition with rationale: 3-axis accelerometer window (raw m/s^2 values)
axes = [...]          # x, y, z lists of equal length
texts = []
series = torch.tensor(axes, dtype=torch.float32)
means, stds = series.mean(1, keepdim=True), series.std(1, keepdim=True).clamp(min=1e-6)
for ax_, m_, s_ in zip("xyz", means.flatten().tolist(), stds.flatten().tolist()):
    texts.append(f"The following is the accelerometer data on the {ax_}-axis, "
                 f"it has mean {m_:.4f} and std {s_:.4f}:")
sample = {"pre_prompt": HAR_PROMPT,                 # verbatim template: see docs/prompts.md
          "time_series": ((series - means) / stds).tolist(),
          "time_series_text": texts, "post_prompt": "Rationale:", "answer": ""}
batch = extend_time_series_to_match_patch_size_and_aggregate([sample], patch_size=32)
text, _ = model.generate(batch, max_new_tokens=200)
print(text[0])                                      # rationale, ends with "Answer: walking."

The exact prompts for every task are in docs/prompts.md, the evaluation harnesses in evaluation/, and a runnable tutorial (loading, QA, captioning, forecasting and this checkpoint) in the Hands-on 2 notebook.

Try it

The quickest way to run this checkpoint is the step-by-step tutorial notebook — it loads LOTUS (Qwen3-0.6B) on a free Colab T4 and evaluates it on real accelerometer windows from its HAR test split, printing accuracy next to the paper's number together with the model's verbatim rationale:

Citation

@article{do2026lotus,
  title   = {LOTUS: Learning to Understand Time-series by Learning to Forecast},
  author  = {Do, Van Dai and Nguyen, Huu Hiep and Nguyen, Minh Hoang and Nguyen, Manh and Le, Hung},
  journal = {arXiv preprint arXiv:2610.18877},
  year    = {2026}
}