| --- |
| license: apache-2.0 |
| library_name: pytorch |
| tags: |
| - tabular |
| - tabular-classification |
| - in-context-learning |
| - foundation-model |
| - credit-risk |
| - synthetic-data |
| pipeline_tag: tabular-classification |
| --- |
| |
| # FinTFM — binary classification checkpoint |
|
|
| <div align="center"> |
|
|
| [](https://pypi.org/project/fintfm/) |
| [](https://github.com/kabartay/fintfm) |
| [](https://github.com/kabartay/fintfm/blob/main/docs/results/FINDINGS.md) |
| [](https://github.com/kabartay/fintfm/blob/main/docs/paper/CLAIMS.md) |
| [](https://github.com/kabartay/fintfm/blob/main/LICENSE) |
|
|
| </div> |
|
|
| An in-context tabular classifier for corporate credit risk, **pretrained only on synthetic |
| data**. Predictions are a single forward pass with your table supplied as context: no gradient |
| steps at fit time, no per-dataset tuning. |
|
|
| **Code, measurement log and claims ledger: |
| [github.com/kabartay/fintfm](https://github.com/kabartay/fintfm)** — every number quoted below |
| is traceable to a numbered entry there stating how it was produced. |
|
|
| ## What this checkpoint is |
|
|
| | | | |
| | --- | --- | |
| | parameters | **885,650** | |
| | architecture | `d_cell=48, d_model=128, n_layers=4, n_col_layers=2, d_ff=512`, two-way cell attention (`n_cell_blocks=1`, per-cell labels) | |
| | capacity | **binary only**, up to **136 features** | |
| | training | 6,000 steps × batch 8 = **48,000 synthetic tasks**, financial prior only | |
| | training data | **synthetic; no real table was seen during pretraining** | |
|
|
| This is the checkpoint on which **every published binary number in the repository was |
| measured**, so results quoted there are reproducible against this file rather than a variant of |
| it. |
|
|
| ## Performance, stated honestly |
|
|
| On [TabArena](https://tabarena.ai), 27 binary datasets, against 94 other methods: |
|
|
| | | | |
| | --- | --- | |
| | mean ROC-AUC | **0.7823** | |
| | rank | **93 of 95** | |
|
|
| On real corporate-default panels, calibration is consistently among the best measured and |
| discrimination consistently loses to tuned gradient boosting — both, on every panel tried. |
|
|
| **FinTFM is not a competitive general tabular model and it is not presented as one.** It is |
| published so that the numbers in the repository can be checked, and because a provenance claim |
| nobody can verify is a slogan. |
|
|
| ## Usage |
|
|
| ```bash |
| pip install fintfm |
| ``` |
|
|
| ```python |
| from huggingface_hub import hf_hub_download |
| from fintfm.inference import FinancialTFMClassifier |
| |
| ckpt = hf_hub_download( |
| "kabartay/fintfm-binary", |
| "v4-cellattn-labels.pt", |
| # Pin the commit. Weights behind a published number should not move underneath it. |
| revision="f116bfd43a2b15c65ed3551ea8c38e3364629ddc", |
| ) |
| clf = FinancialTFMClassifier(ckpt, device="cpu") |
| clf.fit(X_train, y_train) # stores the table as context; no training happens |
| proba = clf.predict_proba(X_test)[:, 1] |
| ``` |
|
|
| The package is on PyPI as [`fintfm`](https://pypi.org/project/fintfm/), Apache-2.0, Python |
| 3.12+. |
|
|
| Categorical columns must be encoded before they reach the model — it reads every cell as an |
| ordered scalar, and label encoding is measurably worse than no order at all. Use |
| `fintfm.inference.categorical.CategoricalTargetEncoder`, which is out-of-fold on the context |
| rows for reasons that are not optional. |
|
|
| ## Limitations |
|
|
| - **Binary only, ≤136 features.** A task exceeding either raises rather than being silently |
| truncated. |
| - **Slow at inference**: median ~8.6 s per 1,000 rows, against a field norm near 0.1. |
| - **Loses to tuned gradient boosting** on discrimination, everywhere it has been measured. |
| - The multiclass and regression variants exist in the codebase, were scored on real data, and |
| **rank last** — they are deliberately not published. |
|
|
| ## Licence |
|
|
| **Apache-2.0**, the same as the code — chosen deliberately rather than inherited. A weights |
| licence is a separate question from a code licence, and four of the ten peer projects surveyed |
| in the repository ship permissive code with **non-commercial weights**; one restricts commercial |
| use of the model's *output*. This checkpoint has no such restriction: commercial use is |
| permitted, subject to Apache-2.0's attribution and notice terms. |
|
|
| ## Citing |
|
|
| The repository carries a `CITATION.cff`; GitHub's **"Cite this repository"** button renders |
| BibTeX and APA from it. Cite the **repository** for the method or any finding, and this |
| **checkpoint** when the specific weights matter to what you report — they are different |
| artifacts and a reader can only check the one you name. |
|
|
| Author: Mukharbek Organokov ([ORCID 0000-0002-3093-3456](https://orcid.org/0000-0002-3093-3456)). |
|
|
| ## Provenance |
|
|
| No code, weights, or training data from TabPFN, TabICL, TabDPT, LimiX, Nori, MITRA, or any other |
| tabular foundation model was used. The prior that generated this checkpoint's training data is |
| in the repository (`src/fintfm/prior/`) and can be inspected and re-run. |
|
|
