Rudraneel93/counterfact-v2

Counterfact estimates the real effect of a change on a time series, such as a price cut, a marketing campaign, a launch or a policy. It works out what your numbers would have done anyway, and gives an honest 90% range for the effect.

• 🧠 Pretrained once, works on your data immediately. No model fitting, no tuning, under a second on a CPU.
• 🧪 Trained only on synthetic worlds where the true effect is known. No real or customer data was used for training.
• 🎯 Honest about uncertainty. About 10% false alarms when nothing happened, against 41% for a typical CausalImpact-style analysis.
• 🪶 Small: 8.7M parameters, 35 MB.

---

What it does

Sales rose after a 10% price cut, but Diwali and a growing market rose with them. Counterfact draws the sales you would most likely have had without the cut (dashed line, with its 90% range). The gap is the cut's real effect: +40.9 units per week, with a likely range of +18.6 to +63.2. The true effect in this example is +48.7, inside the range.

How it works

The method is a prior-fitted network for causal inference. A generator creates millions of random causal systems with hidden confounders, delayed effects, trends, seasonality, festivals, structural breaks, heavy-tailed noise and shared latent factors. Each one is simulated with and without an intervention, using identical noise, so the true counterfactual is known. An axial transformer, attending over time within each series and across series at each step, learns to infer that counterfactual from the history and the comparison series. At inference it takes your data in context and returns quantiles of the counterfactual path and of the average effect.

Results

We took 1,200 windows of real Indian and US public data, added an effect of known size (or none, in 30% of windows), and asked each method to recover it. Calibration was validated by leaving out one data panel at a time, so every panel is scored with a correction fitted on the other seven.

Method	Typical error ↓	90% range holds	False alarms ↓
Counterfact v2	0.49	90%	10%
CausalImpact-style (local level + regression)	0.50	58%	41%
Regression on comparison series	0.62	13%	87%

Error is in units of the series' pre-period standard deviation.

Published natural experiments

On California's 1989 tobacco tax, Counterfact estimates −19.0 packs per person per year (90% range −29.8 to −7.5), matching the published synthetic-control estimate of about −19 (Abadie et al., 2010). On German reunification it estimates −1,134 USD per person, against about −1,600 published. With only 16 comparison countries it honestly reports that it can't be sure.

Placebo tests ask each method to measure an "effect" in places where nothing happened. A good method should rarely find one.

Independent benchmark: someone else's simulator

Our own benchmarks could flatter us, since we wrote the simulator. So we also ran this model, unchanged, on dot-Identifiability-v1 (v1.2.0, from the DoTime project). It's a public suite of simulated worlds from another group, each simulated twice from the same noise (with and without a change), across nine causal structures, some with hidden confounders and two where the true effect is zero.

Setup: 800 worlds sampled with a fixed seed. The model sees only the changed series and the outcome: no hidden variables and no knowledge of the graph. It predicts the outcome without the change at the queried step. We report its raw 90% range, with no calibration and no refitting.

Structure	Worlds	90% range holds	Typical error ↓	Before/after error
All	800	93%	0.61	0.66
Direct effect (A→Y)	104	90%	0.61	0.63
Back-door (X→A, X→Y, A→Y)	84	93%	0.66	0.66
No effect, observed confounder	76	95%	0.68	0.71
No effect, hidden confounder	83	93%	0.58	0.62
Mediator (A→M→Y)	81	94%	0.67	0.75
Confounder + mediator	91	93%	0.59	0.62
Front-door (hidden U)	98	94%	0.56	0.59
Instrumental variable (hidden U)	85	96%	0.53	0.65
Bow graph (hidden U, not identifiable)	98	88%	0.66	0.69

Error is in units of the outcome's typical step-to-step change before the change.

How to read this. The ranges stay honest on a simulator we didn't write, including worlds where the change did nothing. The weakest case is the bow graph, where a hidden factor drives both the change and the outcome and no method can fully separate them. This doesn't show we beat simple methods: each world asks about the single step right after the change, where before/after does almost as well. DoTime's own model forecasts the outcome before it happens, while this model measures after the fact. They answer different questions, so we don't score them head-to-head. The production model (v4, tuned to real-world panels) holds 87% here (bow graph: 77%).

Reproduce it: benchmarks/dotime/ has the two scripts (DoTime environment, then Counterfact) and per-world results.

Quick start

▶ Fastest way: open the notebook in Colab. It downloads the model, measures a price cut and runs a mini benchmark in about a minute.

Benchmark your own causal method on 20,000 worlds with known effects: Counterfact Synthetic Worlds.

import sys, pandas as pd
from huggingface_hub import snapshot_download

path = snapshot_download("Rudraneel93/counterfact-v2")
sys.path.insert(0, path)
from counterfact_open import load, estimate

model = load(f"{path}/model.pt")
df = pd.read_csv("weekly_sales.csv")                 # one row per period, one column per series
e = estimate(model,
             target=df["sales"].values,             # the number you care about
             t0=80,                                 # row of the first period after the change
             covariates=df[["region_b", "region_c", "category"]].values.T,   # series the change did NOT affect
             cov_names=["region_b", "region_c", "category"],
             index=df["week"].tolist())
print(e.summary())
# Average effect after the intervention: +40.9 (+9.5%)
# 90% interval: [+18.6, +63.2]  -> likely real

estimate returns the per-period effect, a 90% interval, the cumulative effect, P(effect > 0), and the counterfactual path with its band. You need torch and numpy.

Want more than the research model?

	This repo (v2)	Counterfact (production)
Model	v2	v4: more accurate and more consistent across data types
Averaging over several runs	–	✓
Long daily histories	crops to 128 steps	✓ grouped automatically
Robustness checks (placebo dates, leave-one-out)	–	✓
Plain-language answers and PDF reports	–	✓
Access	download	Website · RapidAPI · pip install counterfact
Licence	non-commercial	commercial

Limits

• It can't separate your change from something else that happened at the same time, unless the comparison series capture it.
• It works best with at least twice as much history before the change as after, and 3–10 comparison series the change didn't affect.
• This release reads at most 128 time steps and 8 series. Longer inputs are cropped.
• The calibration was fitted on macro, financial and retail panels. Check coverage on your own domain before relying on the intervals.
• Hidden factors that drive both the change and the outcome are the hardest case. On the independent bow-graph benchmark, ranges held 88% instead of 90%.

Licence

CC BY-NC 4.0: free for research and non-commercial use, with attribution. For commercial use, use the API on RapidAPI or pip install counterfact, or ask for a licence through counterfact-site.vercel.app.

Citation

@misc{counterfact2026,
  title  = {Counterfact: a prior-fitted transformer for causal effect estimation in time series},
  author = {Das, Rudraneel},
  year   = {2026},
  url    = {https://huggingface.co/Rudraneel93/counterfact-v2}
}