README.md

---
license: mit
base_model: Qwen/Qwen3-VL-8B-Instruct
library_name: peft
tags: [time-series, multimodal, qwen3-vl, dinov3, lora]
---

# DualTime-8B

<p align="center">
<a href="https://arxiv.org/abs/2610.22327"><img src="https://img.shields.io/badge/arXiv-2610.22327-b31b1b.svg" alt="arXiv"/></a>
<a href="https://github.com/azencot-group/dual-time"><img src="https://img.shields.io/badge/GitHub-Code-181717?logo=github" alt="Code"/></a>
<a href="https://huggingface.co/datasets/SequentialLearning/DualTime-data"><img src="https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-Data-yellow" alt="Data"/></a>
</p>

The reference checkpoint of *Twice Upon a Time: Complementary Representations
for Time-Series Reasoning*: Qwen3-VL-8B-Instruct reading every time series
twice, as a line plot through its own vision encoder and as a delay-embedding
image through a second DINOv3 ViT-L/16 tower. Two adapters, applied in this
order on top of the public weights:

| path | what |
|---|---|
| `stageA/best/` | Stage A prior: LoRA on the DINOv3 tower and the trained mergers |
| `stageB/checkpoint-5714/` | Stage B language-model LoRA |

Base model: `Qwen/Qwen3-VL-8B-Instruct`. Second tower:
`facebook/dinov3-vitl16-pretrain-lvd1689m` (not redistributed here).

## Usage

Training, evaluation and the loading code are in the code repository; its
evaluation section downloads these adapters and reproduces the paper's
numbers:

```bash
bash scripts/download_checkpoints.sh          # -> $QTSX_CKPT/DualTime-8B
export A_CKPT=$QTSX_CKPT/DualTime-8B/stageA/best
export B_CKPT=$QTSX_CKPT/DualTime-8B/stageB/checkpoint-5714
KIND=all bash scripts/eval.sh
```

To load the model directly, clone the code repository and run from its
root (the `src` package below is ours; `load_qwen3vl_dual` builds Qwen3-VL
with the second DINOv3 tower attached, downloading
`facebook/dinov3-vitl16-pretrain-lvd1689m`, a gated repository: accept its
licence on the Hub and `hf auth login` first):

```python
from peft import PeftModel
from src.model.model_patch import load_qwen3vl_dual   # this project's code repository

model, processor = load_qwen3vl_dual("Qwen/Qwen3-VL-8B-Instruct")   # bf16, sdpa attention
model = model.to("cuda")
model = PeftModel.from_pretrained(model, "stageA/best").merge_and_unload()             # 1. Stage A prior
model = PeftModel.from_pretrained(model, "stageB/checkpoint-5714").merge_and_unload()  # 2. Stage B LoRA
model.eval()
```

The order matters: the Stage B LoRA was trained on top of the merged Stage A
prior. Prompts go through
`src.datasets_utils.unified_collator.UnifiedDualCollator`, which renders each
`<ts><ts/>` marker into the line plot and the delay image; the evaluators in
the code repository do all of this.

## Example

A synthetic series, a sawtooth wave (period 40) plus a square wave (period
128), asked a TimeSeriesExam-style multiple-choice question.

```python
import numpy as np, torch
from scipy import signal
from src.datasets_utils.unified_collator import UnifiedDualCollator
from src.datasets_utils.prompts import SYSTEM, user_text   # the multiple-choice prompt format the model was trained on

collator = UnifiedDualCollator(processor=processor, max_length=4096, chart_style="v2")

def ask(series, question, system=None, max_new_tokens=256):
    # "<ts><ts/>" marks where the series goes; the collator renders it as the line plot
    # for the Qwen ViT and the delay-embedding image for the DINOv3 tower.
    example = {"system": system, "user_text": "<ts><ts/>\n" + question, "target_text": "",
               "ts_list": [np.asarray(series, dtype=np.float32)], "ts_field_names": ["series"],
               "task": "demo", "meta": {}}
    inputs = collator([example], for_generation=True)
    inputs = {k: (v.to(model.device) if hasattr(v, "to") else v) for k, v in inputs.items()}
    with torch.no_grad():
        out = model.generate(**inputs, do_sample=False, max_new_tokens=max_new_tokens)
    return processor.tokenizer.decode(out[0, inputs["input_ids"].shape[1]:], skip_special_tokens=True).strip()

t = np.arange(512)
series = signal.sawtooth(2 * np.pi * t / 40) + 2 * signal.square(2 * np.pi * t / 128)

question = "Which additive combination of patterns best describes the time series?"
options = ["SineWave + SquareWave", "SawtoothWave + SquareWave", "SineWave + SawtoothWave"]
print(ask(series, user_text(question, options), system=SYSTEM, max_new_tokens=8))
```

What the model sees, the line plot (Qwen ViT) and the delay-embedding image
of the same series (DINOv3 tower), and its answer:

<table><tr>
<td><img src="example_plot.png" alt="line plot" width="560"/></td>
<td><img src="example_delay.png" alt="delay embedding" width="200"/></td>
</tr></table>

```
B
```

## Results

TSExam 0.899, TSRBench 53.2, ChatTS alignment A 0.864 / 0.805 and B
0.893 / 0.868 (categorical / numerical). The full tables are in the paper and
the code repository.

## Citation

```bibtex
@misc{zisling2026twicetime,
      title={Twice Upon a Time: Complementary Representations for Time-Series Reasoning}, 
      author={Hedi Zisling and Liran Nochumsohn and Ilan Naiman and Mihir Parmar and Omri Azencot},
      year={2026},
      eprint={2610.22327},
      archivePrefix={arXiv},
      primaryClass={cs.LG},
      url={https://arxiv.org/abs/2610.22327}, 
}
```