| --- |
| license: apache-2.0 |
| base_model: sentence-transformers/all-MiniLM-L6-v2 |
| language: |
| - en |
| library_name: pytorch |
| pipeline_tag: sentence-similarity |
| tags: |
| - temporal-retrieval |
| - information-retrieval |
| - retrieval-augmented-generation |
| - gaussian-embeddings |
| --- |
| |
| # TEMPS: temporal module |
|
|
| **TEMPS** (Temporal Embedding Model for Precise Search) adds a temporal score to any dense |
| retriever. This repository holds the trained temporal module from |
| *TEMPS: Temporal Sentence Embeddings for Temporal Information Retrieval* |
| (Hassani, Romero, Bouzeghoub, Jacquelinet, EMNLP 2026). |
|
|
| - **Code:** https://github.com/aldebaran-care/TEMPS |
| - **Weights:** `temps-weights.pth` (PyTorch `state_dict`, fp32) |
|
|
| ## Model description |
|
|
| Dense retrievers match queries and passages on topic but not on time, so they return passages |
| that are on-topic yet temporally wrong. TEMPS attaches to a **frozen** semantic retriever and |
| scores how well two *anchored* texts (text + anchor date) align in time. |
|
|
| For a text and its anchor date, the module outputs a diagonal Gaussian over time. Its layers are: |
|
|
| - a frozen `all-MiniLM-L6-v2` backbone (up to 512 tokens) |
| - a sinusoidal anchor-date encoder |
| - attention pooling |
| - mean and variance heads |
|
|
| Two texts are compared with the KL-based score |
|
|
| ``` |
| ts(1, 2) = 1 / (1 + KL(p_1 || p_2)) |
| ``` |
|
|
| which is fused with the semantic retriever's cosine score after per-query min-max |
| normalization: |
|
|
| ``` |
| s(1, 2) = (1 - alpha) * cos(E_1, E_2) + alpha * ts(1, 2) |
| ``` |
|
|
| `alpha` is set per benchmark on a held-out validation split: 0.6 for TimeQA, 0.8 for |
| TempReason and 0.3 for TS-Retriever. |
|
|
| ## Training |
|
|
| No hand-labeled temporal data is used. Each anchored temporal expression is resolved to an |
| inclusive day interval `[a, b]` and moment-matched to a Gaussian |
| (`mu = (a + b + 1) / 2`, `sigma^2 = (b - a + 1)^2 / 12`). The pair label is |
| `1 / (1 + KL(p_I1 || p_I2))`, and the module is trained with a CoSENT ranking loss to reproduce |
| that ordering. |
|
|
| - **Data:** 60M automatically generated tuples (55M synthetic, 5M from Wikipedia text in the |
| Multilingual MLM Temporal Tagging Resources) |
| - **Setup:** 3 epochs, global batch size 2,048, lr 3e-4, weight decay 0.01, 5% linear warmup, |
| max 512 tokens, fp16, seed 42; about 4 hours on 8×A100-80GB |
|
|
| ## Usage |
|
|
| The model class lives in the code repository. Clone it and run from its root: |
|
|
| ```bash |
| git clone https://github.com/aldebaran-care/TEMPS && cd TEMPS |
| uv sync |
| ``` |
|
|
| ```python |
| import torch |
| from torch.distributions import Normal, kl_divergence |
| from huggingface_hub import hf_hub_download |
| from transformers import AutoTokenizer |
| |
| from temporal_embeddings.model.gauss_model import GaussModel |
| from temporal_embeddings.utils.positional_encoding import positional_encoding |
| |
| backbone = "sentence-transformers/all-MiniLM-L6-v2" |
| model = GaussModel(backbone).eval() |
| model.load_state_dict(torch.load(hf_hub_download("aldebaran-research/temps", "temps-weights.pth"), map_location="cpu")) |
| tokenizer = AutoTokenizer.from_pretrained(backbone) |
| |
| @torch.inference_mode() |
| def encode(texts, anchor_dates): |
| batch = tokenizer(texts, padding=True, truncation=True, max_length=512, return_tensors="pt") |
| return model(**batch, dates=positional_encoding(anchor_dates)) |
| |
| query = encode(["Who led the company last spring?"], ["2021-11-09"]) |
| docs = encode(["In April 2021 she was appointed CEO.", "He ran the firm from 1998 to 2003."], |
| ["2021-11-09", "2021-11-09"]) |
| |
| # Temporal score 1 / (1 + KL(query || doc)), KL averaged over dimensions; higher = better match |
| kl = kl_divergence(Normal(query.mu, query.std), Normal(docs.mu, docs.std)).mean(dim=-1) |
| print(1 / (1 + kl)) # tensor([0.7331, 0.7130]) |
| ``` |
|
|
| To reproduce the paper's evaluation, `uv run python scripts/download_model.py` saves this |
| checkpoint where the evaluation configs expect it; see the repository README. |
|
|
| ## Results |
|
|
| On the held-out test splits (MRR): |
|
|
| | Benchmark | Best system | MRR | Gain over backbone | |
| |---|---|---|---| |
| | TimeQA (alpha=0.6) | SFR-Embedding-Mistral + TEMPS | 0.5360 | +31.9% | |
| | TempReason (alpha=0.8) | SFR-Embedding-Mistral + SUTime control | 0.6901 | +11.5% | |
| | TS-Retriever (alpha=0.3) | SFR-Embedding-Mistral + TEMPS | 0.8240 | +9.3% | |
|
|
| On TS-Retriever, `E5-base-v2 + TEMPS` lifts R@1 from 19.92 to 25.39 and P@1 from 58.63 to |
| 69.14 over TSContriever. On TempReason, the rule-based SUTime control outperforms TEMPS; see |
| the paper for the full tables. |
|
|
| ## Limitations |
|
|
| - English only. |
| - Covers expressions that can be anchored and resolved to an interval: absolute dates, |
| deictic expressions and offsets, present references, seasons, durations and intervals. |
| Discourse-level reasoning and event–event relations are out of scope. |
| - A passage of up to 512 tokens is reduced to a single Gaussian, which loses information on |
| long or temporally mixed passages. |
| - Passages with no temporal expression were held out of training, so their Gaussians are |
| uncalibrated; the semantic score is what ranks them. |
| - Results come from a single training run (seed 42). |
|
|
| ## License |
|
|
| Apache-2.0, inherited from |
| [`sentence-transformers/all-MiniLM-L6-v2`](https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2). |
| The code repository is MIT. |
|
|
| ## Citation |
|
|
| ```bibtex |
| @inproceedings{hassani-etal-2026-temps, |
| title = {{TEMPS}: Temporal Sentence Embeddings for Temporal Information Retrieval}, |
| author = {Hassani, Mourad and Romero, Julien and Bouzeghoub, Amel and Jacquelinet, Christian}, |
| booktitle = {Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing (EMNLP)}, |
| year = {2026}, |
| publisher = {Association for Computational Linguistics} |
| } |
| ``` |
|
|
| ## Acknowledgments |
|
|
| This project was provided with computing and storage resources by GENCI at IDRIS thanks to the |
| grant 2025-105442 on the supercomputer Jean Zay's A100 partition. |
|
|
