File size: 2,153 Bytes
3f6ff20 93b515f 3f6ff20 93b515f f2ef758 93b515f | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 | ---
license: apache-2.0
library_name: pytorch
pipeline_tag: tabular-regression
tags:
- variable-selection
- bayesian
- spike-and-slab
- prior-data-fitted-network
- in-context-learning
- tabular-regression
---
# LinearPFN
**Bayesian variable selection for linear models with interactions, in one forward pass.**
A transformer pretrained on a spike-and-slab prior over main effects and pairwise
interactions. Given a dataset, it returns posterior inclusion probabilities and
posterior-mean coefficients, with no MCMC and no per-dataset fitting.
Code and documentation: [github.com/schiekiera/LinearPFN](https://github.com/schiekiera/LinearPFN)
## Use
```bash
git clone https://github.com/schiekiera/LinearPFN.git
pip install -e LinearPFN
```
```python
from linearpfn import LinearPFN
model = LinearPFN.from_pretrained() # downloads this model's weights
result = model.fit(X, y)
print(result.summary())
```
## Model
| | |
|---|---|
| Input | n rows, p predictors, outcome y (trained on p = 2 to 30, n = 20 to 2,000) |
| Output | a PIP and a posterior mean for each of the p + p(p-1)/2 effects, and a predictive distribution |
| Architecture | nanoTabPFN-style transformer, 8 layers, 31.5M parameters, fp32 |
| Training data | synthetic only, drawn from the prior in `config.yaml` |
<br>
## Limitations
The outputs are the posterior under the training prior: linear main effects, pairwise
interactions, strong heredity, Gaussian noise. Missing values and categorical predictors
must be handled before fitting.
## Files
| | |
|---|---|
| `linearpfn_strong.pt` | weights and configuration (`torch.load(..., weights_only=True)`) |
| `config.yaml` | prior, architecture and training settings |
| `SHA256SUMS` | checksums of the two files above |
<br>
## Citation
```bibtex
@article{schiekiera2026linearpfn,
title = {{LinearPFN}: Amortized Variable Selection for Linear Models with Interactions},
author = {Schiekiera, Louis and Zimmer, Max and Roux, Christophe and Arnold, Manuel
and Pokutta, Sebastian and G{\"u}nther, Fritz},
journal = {arXiv preprint},
year = {2026}
}
```
License: Apache 2.0.
|