| --- |
| license: apache-2.0 |
| library_name: lightpfn |
| pipeline_tag: tabular-classification |
| tags: |
| - tabular |
| - tabular-classification |
| - in-context-learning |
| - prior-data-fitted-network |
| - foundation-model |
| - synthetic-data |
| - scikit-learn |
| - vulkan |
| - safetensors |
| --- |
| |
| # LightPFN |
|
|
| <p> |
| <a href="https://github.com/GioOtto/LightPFN"><img alt="Code on GitHub" src="https://img.shields.io/badge/Code-GitHub-181717?style=for-the-badge&logo=github&logoColor=white" /></a> |
| <a href="https://pypi.org/project/LightPFN/"><img alt="Package on PyPI" src="https://img.shields.io/pypi/v/lightpfn?style=for-the-badge&logo=pypi&logoColor=white&label=PyPI&color=3775A9" /></a> |
| <a href="https://huggingface.co/ueuegio/LightPFN/blob/main/LightPFN_report.pdf"><img alt="Technical report (PDF)" src="https://img.shields.io/badge/Technical%20report-PDF-8B1A1A?style=for-the-badge&logo=adobeacrobatreader&logoColor=white" /></a> |
| <a href="https://github.com/GioOtto/LightPFN/blob/main/LICENSE"><img alt="License Apache 2.0" src="https://img.shields.io/badge/License-Apache%202.0-2F855A?style=for-the-badge" /></a> |
| </p> |
|
|
| LightPFN is a small tabular foundation model for classification: a 4,603,088-parameter in-context learner |
| pretrained only on synthetic data. `fit` stores the training set as context and `predict_proba` answers in one |
| forward pass, with no training on your data and no hyperparameters to tune. It runs on CPU, CUDA, ROCm and, through |
| its own Vulkan kernels, on AMD, Intel and NVIDIA GPUs. |
|
|
| - Code, documentation and training pipeline: [github.com/GioOtto/LightPFN](https://github.com/GioOtto/LightPFN) |
| - Package: [pypi.org/project/LightPFN](https://pypi.org/project/LightPFN/) |
| - Technical report: [LightPFN_report.pdf](./LightPFN_report.pdf) |
| - License: Apache 2.0, code and weights |
|
|
| ## Usage |
|
|
| ```bash |
| pip install LightPFN |
| ``` |
|
|
| ```python |
| from lightpfn import LightPFNClassifier |
| |
| clf = LightPFNClassifier(n_estimators=4, random_state=0) |
| clf.fit(X_train, y_train) # downloads these weights at a pinned commit on the first fit |
| proba = clf.predict_proba(X_test) |
| ``` |
|
|
| `X` can be a NumPy array or a pandas DataFrame with numeric, categorical, string and boolean columns and missing |
| values. `device="auto"` uses a CUDA or ROCm GPU, else a Vulkan GPU (`pip install "LightPFN[vulkan]"`), else the CPU. |
| The [user guide](https://github.com/GioOtto/LightPFN/blob/main/docs/en/GUIDE.md) covers every option. |
|
|
| ## Model |
|
|
| | | | |
| |---|---| |
| | Task | classification, 2 to 10 classes | |
| | Parameters | 4,603,088 (18 MB, float32) | |
| | Architecture | cell embedding (value, rank, missing flag), two induced column stages, row refinement with four summary tokens, row compression, seven in-context blocks, retrieval decoder | |
| | Pretraining data | synthetic only: 90% structural causal graph prior, 10% rule prior (XOR, parity, lookup, trees); 7.68 million task draws from 4.03 million distinct tasks | |
| | Training | 120,000 steps of 64 tasks on two RTX 5090 (tables up to 2,048 rows), then 39,250 steps on tables up to 60,000 rows | |
| | Not used in training | real datasets, distillation, weights or outputs of other tabular foundation models | |
|
|
| ## Evaluation |
|
|
| The official TabArena leaderboard includes default, tuned and ensembled baselines. Other evaluations use |
| default baselines; AUC differences have 95% paired bootstrap intervals. |
|
|
| | Benchmark | Result | |
| |---|---| |
| | [Official TabArena leaderboard](https://huggingface.co/spaces/TabArena/leaderboard), 38 classification datasets, default configuration (four estimators), run and verified by the TabArena maintainers, 11 October 2026 | all splits: 21st of 92, Elo 1441 (+69 / -79); Lite: 20th of 92, Elo 1426 (+71 / -70); none imputed. Above every tree-based method, tuned and ensembled ones included (intervals overlap default CatBoost and the tuned entries); behind TabICLv2 (1575) and larger foundation models | |
| | 55 OpenML-CC18 datasets outside TabArena (at most 1,000 rows, one estimator) | mean AUC 0.911, +0.86 [0.42, 1.40] over CatBoost, +1.8 to +2.1 over LightGBM, XGBoost and random forest | |
| | TabArena, 38 classification tasks, official splits, first repeat (our harness), four estimators | mean AUC 0.858, rank 2.50 of 7, behind TabICLv2 (0.864), lower error than CatBoost on 76% of tasks | |
| | 13 OpenML datasets of 50,000 to 2.2M rows, 10,000 to 100,000 training rows | mean AUC difference from CatBoost between -0.34 and +0.28 points, intervals include zero | |
|
|
| <p align="center"><img src="https://raw.githubusercontent.com/GioOtto/LightPFN/main/docs/assets/tabarena_elo.png" width="100%" alt="TabArena, 38 classification datasets, all splits: Elo with 95% intervals for selected methods; LightPFN is 21st, above every tree-based method" /></p> |
|
|
| The leaderboard view is Classification with imputed models excluded: the default view also covers the 13 |
| regression datasets, which LightPFN does not support. Full tables, timings and the official TabArena results: |
| [docs/en/RESULTS.md](https://github.com/GioOtto/LightPFN/blob/main/docs/en/RESULTS.md#tabarena-official-leaderboard). |
|
|
| ## Limitations |
|
|
| - Classification only (2 to 10 classes); regression is planned for version 2. |
| - Categorical columns are read as ordinal codes. On tables dominated by high-cardinality categorical columns |
| CatBoost is ahead; native categorical handling is planned for version 2. |
| - Above 20,000 training rows each estimator reads a stratified subsample (`max_context`). |
| - CPU time grows with the context length; on large tables a GPU is much faster. |
|
|
| ## Files |
|
|
| | File | Content | |
| |---|---| |
| | `model.safetensors`, `config.json` | weights (float32) and architecture of the released model | |
| | `provenance.json` | hashes of the weights and of the source checkpoint | |
| | `LightPFN_report.pdf` | technical report | |
| | `LICENSE`, `NOTICE` | Apache License 2.0 and attribution notice | |
|
|
| ## Citation |
|
|
| ```bibtex |
| @techreport{ottoboni2026lightpfn, |
| title = {A Sling Against Giants: {LightPFN}, a 4.6M-parameter tabular in-context classifier designed to stay small}, |
| author = {Ottoboni, Giorgio}, |
| year = {2026}, |
| url = {https://github.com/GioOtto/LightPFN} |
| } |
| ``` |
|
|
