--- license: mit tags: - time-series - time-series-language-model - forecast-grounding - lora - arxiv:2610.18877 base_model: - google/gemma-3-270m - google/gemma-3-1b-pt - Qwen/Qwen3-0.6B pipeline_tag: time-series-forecasting --- # LOTUS: Learning to Understand Time-series by Learning to Forecast

LOTUS

**LOTUS** is a time-series language model pretrained under **forecast grounding** — a pretraining principle that requires language-aligned time-series representations to *remain predictive of the future evolution of the signal* before the model is adapted to downstream tasks. Time-series language models typically align numerical signals with language using labels, captions, or question–answer pairs; such supervision encourages linguistic representations but does not explicitly require them to preserve information about how the underlying signal evolves. Forecast grounding closes that gap. LOTUS divides each time-series channel into non-overlapping patches and maps them into the LLM embedding space. It then predicts the next patch from the LLM hidden states of the time-series tokens, encouraging the learned representation to retain temporal information. To capture both temporal changes and periodic patterns, each raw patch is combined with its spectral representation (**temporal–spectral patch representation**) before mapping into the LLM embedding space. Across three language-model backbones and six downstream evaluation datasets, LOTUS achieves the best average performance in every matched-backbone group with minimal trainable parameters — **59.91%** average accuracy with the Qwen3-0.6B backbone, outperforming the strongest matched baseline by **3.36%** — and its 270M-parameter model is the top system on the TSRBench zero-shot reasoning benchmark ahead of backbones 25–50× larger. Paper: [arXiv:2610.18877](https://arxiv.org/abs/2610.18877) · Code: [DA2I2-SLM/Lotus](https://github.com/DA2I2-SLM/Lotus) · Phase-1 (pretraining) checkpoints: [hiepnh137/LangTimeFoundationModel](https://huggingface.co/hiepnh137/LangTimeFoundationModel) ## Training * **Phase 1 — forecast grounding (pretraining).** The encoder, projector, next-patch decoder and LLM LoRA adapters are trained jointly on a mixture of forecasting (next-patch prediction from the LLM hidden states) and language (TSQA multiple-choice, M4 captioning) corpora. * **Phase 2 — downstream adaptation.** The next-patch decoder is removed; the time-series interface and LLM LoRA adapters are trained with the language objective alone, jointly across HAR, Sleep-EDF, ECG-QA, PAMAP2 and Time-MQA. All systems are evaluated under greedy decoding: the model writes a rationale followed by an explicit answer, which is parsed and compared with the reference. Classification accuracy is reported on HAR, PAMAP2, Sleep-EDF and Time-MQA (MCQ and true/false splits), and generative accuracy under normalization on ECG-QA. ## Training data (as specified in the paper) **Phase 1 — forecast-grounded pretraining.** Four corpora: Monash half-hourly electricity demand (Godahewala et al., 2021) and a synthetic multivariate corpus for the forecasting objective, and two language corpora — TSQA multiple-choice questions on trend, volatility, seasonality and outliers (Kong et al., 2025; [HF: ChengsenWang/TSQA](https://huggingface.co/datasets/ChengsenWang/TSQA)) and M4 series with generated captions (Makridakis et al., 2020). The synthetic corpus contains **844,275 series** from four generator families, none contributing more than 37% (paper Table 5): | Generator family | Series | Share | Variates | Length | |---|---|---|---|---| | Chronos-2 multivariatizer (re-implementation) | 310,000 | 36.7% | 3–10 | 512–8,192 | | KernelSynth, univariate (Ansari et al., 2024) | 299,999 | 35.5% | 1 | 1,024–4,096 | | TimePFN LMC (Taga et al., 2025) | 215,000 | 25.5% | 3–10 | 512–2,048 | | SymTime S²Generator (Wang et al., 2025) | 19,276 | 2.3% | 3–10 | 1,024–4,096 | **Phase 2 — downstream adaptation (the checkpoints in this repository).** Five datasets, fine-tuned jointly: | Dataset | Task | Source | |---|---|---| | HAR | activity recognition from 3-axis accelerometer windows, with rationales (8 activities) | OpenTSLM release (Langer et al., 2026) | | PAMAP2 | activity recognition (18 activities) | Reiss & Stricker, 2012 | | Sleep-EDF | sleep staging from 30-second EEG segments (6 stages) | Kemp et al., 2000 | | ECG-QA | question answering over 12-lead ECG records | Oh et al., 2023, on PTB-XL (Wagner et al., 2020) | | Time-MQA | open-ended time-series QA; MCQ and T/F splits scored separately | Kong et al., 2025 ([HF: Time-MQA/TSQA](https://huggingface.co/datasets/Time-MQA/TSQA), gated) | Generation recipes for the synthetic corpus (upstream repositories, commits and driver scripts) are in [`scripts/synthetic_corpus/`](https://github.com/DA2I2-SLM/Lotus/tree/main/scripts/synthetic_corpus) of the code repository; the downstream corpora are downloaded automatically by the trainer. ## Checkpoints in this repository (Phase 2, joint five-task adaptation) | File | Backbone | Trainable params | |---|---|---| | `lotus_qwen3-0.6b_joint-5task-finetune.pt` | Qwen3-0.6B | 9.3M | | `lotus_gemma-3-1b-pt_joint-5task-finetune.pt` | Gemma3-1B | 11.1M | | `lotus_gemma-3-270m_joint-5task-finetune.pt` | Gemma3-270M | 1.9M | The repository also contains the matched **OpenTSLM-SP** and **OpenTSLM-Flamingo** baseline checkpoints used in the paper (joint five-task and sequential-curriculum variants). ## Results (six downstream splits) | Backbone | Method | Params | HAR | Sleep | ECG | PAMAP2 | MCQ | T/F | **Avg.** | |---|---|---|---|---|---|---|---|---|---| | Gemma3-270M | OpenTSLM-SP | 3.1M | 54.38 | 65.12 | 9.66 | 52.14 | 38.59 | 68.39 | 48.05 | | Gemma3-270M | OpenTSLM-Flamingo | 317M | 62.55 | 68.01 | 30.10 | 54.70 | 12.37 | 3.04 | 38.46 | | Gemma3-270M | **LOTUS** | 1.9M | 58.38 | 70.54 | 33.16 | 55.15 | 39.41 | 74.47 | **55.19** | | Gemma3-1B | OpenTSLM-SP | 5.5M | 55.46 | 72.58 | 21.66 | 47.44 | 41.36 | 77.51 | 52.67 | | Gemma3-1B | **LOTUS** | 11.1M | 62.67 | 77.10 | 37.66 | 52.56 | 44.98 | 78.42 | **58.90** | | Qwen3-0.6B | OpenTSLM-SP | 27.8M | 63.25 | 70.54 | 32.20 | 47.44 | 54.64 | 71.20 | 56.55 | | Qwen3-0.6B | **LOTUS** | 9.3M | 63.66 | 77.20 | 31.58 | 53.42 | 56.72 | 76.90 | **59.91** | ECG-QA accuracy is computed on the first 5,000 test items (manifest: `eval_manifests/ecg_test_first5000.json` in the code repository). ## Use it Everything needed to run the model — including the LOTUS code — is inside this repository. `pip install torch transformers peft numpy` (Python ≥ 3.10), then one setup cell: ```python from huggingface_hub import snapshot_download import sys sys.path.insert(0, snapshot_download("daidv1112/LOTUS", allow_patterns=["load_lotus.py", "src/*"])) from load_lotus import load_lotus lotus = load_lotus() # downloads the checkpoint, picks GPU + dtype automatically ``` **Case 1 — type the series in** (any list of numbers, raw scale, in time order): ```python series = [12.4, 13.1, 12.8, 13.9, 14.5, 14.2, 15.3, 16.1, 15.8, 16.9, 17.4, 17.1, 18.2, 19.0, 18.6, 19.5, 20.3, 20.0, 21.1, 21.8, 21.5, 22.6, 23.4, 23.0, 24.1, 24.8, 24.5, 25.6, 26.3, 26.0, 27.1, 27.9, 27.5, 28.6, 29.3, 29.0, 30.1, 30.8, 30.5, 31.6, 32.3, 32.0, 33.1, 33.8, 33.5, 34.6, 35.2, 35.0] lotus.ask("Is the time series showing an upward trend?", series) # -> "True. The series shows an upward trend as it starts at 15.0 and increases to 34.0, # indicating a consistent increase over time." ``` **Case 2 — read the series from a CSV file** (name the column, or let it pick the first numeric one; non-numeric and missing values are dropped): ```python lotus.ask_csv("Does the time series show a clear seasonal pattern?", "your_data.csv", column="OT") # -> "[LOTUS] using column 'OT' of your_data.csv: 651 points, range 7.744e+04 to 2.943e+05" # "No. Reasoning: The data shows fluctuations and patterns but does not exhibit a clear # seasonal pattern. While there are noticeable peaks and troughs, these do not occur at # regular intervals..." ``` Both methods take any natural-language question and an optional `domain=` context hint (`Traffic`, `Energy`, `Healthcare`, ...), and return the model's verdict plus its rationale.
Low-level quickstart (explicit model construction) A checkpoint is a `torch.save` dictionary holding the time-series encoder, projector, next-patch decoder and LoRA adapters; the base LLM is loaded from its Hugging Face id. The model class is `LotusModel` (vendored in this repository under `src/`; equivalently available in the [Lotus repository](https://github.com/DA2I2-SLM/Lotus), Python ≥ 3.12, `pip install -e .`). ```python import sys from types import SimpleNamespace import torch from huggingface_hub import hf_hub_download sys.path.insert(0, "Lotus/src") # git clone https://github.com/DA2I2-SLM/Lotus.git from opentslm.model.llm.lotus_model import LotusModel from opentslm.time_series_datasets.util import extend_time_series_to_match_patch_size_and_aggregate cfg = SimpleNamespace( is_pretraining=True, patch_size=32, pe_type="sincos", use_only_time_pe=False, use_only_channel_pe=False, use_freq_embed=True, enc_dec_type="transformer", trans_enc_dim=192, trans_enc_num_layers=6, trans_enc_num_heads=8, trans_enc_ff_dim=896, trans_dec_dim=192, trans_dec_num_layers=4, trans_dec_num_heads=8, trans_dec_ff_dim=896, llm_mv_position_ids=True, llm_dtype="fp32", # "bf16" on GPUs with bf16 support ) model = LotusModel(cfg, llm_id="Qwen/Qwen3-0.6B", device="cuda") model.enable_lora(lora_r=8, lora_alpha=16) model.load_from_file(hf_hub_download("daidv1112/LOTUS", "lotus_qwen3-0.6b_joint-5task-finetune.pt")) model.eval() # activity recognition with rationale: 3-axis accelerometer window (raw m/s^2 values) axes = [...] # x, y, z lists of equal length texts = [] series = torch.tensor(axes, dtype=torch.float32) means, stds = series.mean(1, keepdim=True), series.std(1, keepdim=True).clamp(min=1e-6) for ax_, m_, s_ in zip("xyz", means.flatten().tolist(), stds.flatten().tolist()): texts.append(f"The following is the accelerometer data on the {ax_}-axis, " f"it has mean {m_:.4f} and std {s_:.4f}:") sample = {"pre_prompt": HAR_PROMPT, # verbatim template: see docs/prompts.md "time_series": ((series - means) / stds).tolist(), "time_series_text": texts, "post_prompt": "Rationale:", "answer": ""} batch = extend_time_series_to_match_patch_size_and_aggregate([sample], patch_size=32) text, _ = model.generate(batch, max_new_tokens=200) print(text[0]) # rationale, ends with "Answer: walking." ```
The exact prompts for every task are in [`docs/prompts.md`](https://github.com/DA2I2-SLM/Lotus/blob/main/docs/prompts.md), the evaluation harnesses in [`evaluation/`](https://github.com/DA2I2-SLM/Lotus/tree/main/evaluation), and a runnable tutorial (loading, QA, captioning, forecasting and this checkpoint) in the [Hands-on 2 notebook](https://colab.research.google.com/github/Davido111200/lotus-hands-on-2-fixed/blob/main/Hands-on-2_Lotus_fixed.ipynb). ## Try it The quickest way to run this checkpoint is the step-by-step tutorial notebook — it loads LOTUS (Qwen3-0.6B) on a free Colab T4 and evaluates it on real accelerometer windows from its HAR test split, printing accuracy next to the paper's number together with the model's verbatim rationale: [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/Davido111200/lotus-hands-on-2-fixed/blob/main/Hands-on-2_Lotus_fixed.ipynb) ## Citation ```bibtex @article{do2026lotus, title = {LOTUS: Learning to Understand Time-series by Learning to Forecast}, author = {Do, Van Dai and Nguyen, Huu Hiep and Nguyen, Minh Hoang and Nguyen, Manh and Le, Hung}, journal = {arXiv preprint arXiv:2610.18877}, year = {2026} } ```