Upload README.md with huggingface_hub
Browse files
README.md
ADDED
|
@@ -0,0 +1,88 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: apache-2.0
|
| 3 |
+
library_name: pytorch
|
| 4 |
+
tags:
|
| 5 |
+
- tabular
|
| 6 |
+
- tabular-classification
|
| 7 |
+
- in-context-learning
|
| 8 |
+
- foundation-model
|
| 9 |
+
- credit-risk
|
| 10 |
+
- synthetic-data
|
| 11 |
+
pipeline_tag: tabular-classification
|
| 12 |
+
---
|
| 13 |
+
|
| 14 |
+
# fintfm — binary classification checkpoint
|
| 15 |
+
|
| 16 |
+
An in-context tabular classifier for corporate credit risk, **pretrained only on synthetic
|
| 17 |
+
data**. Predictions are a single forward pass with your table supplied as context: no gradient
|
| 18 |
+
steps at fit time, no per-dataset tuning.
|
| 19 |
+
|
| 20 |
+
Code, measurement log and claims ledger: **https://github.com/kabartay/fintfm**
|
| 21 |
+
|
| 22 |
+
## What this checkpoint is
|
| 23 |
+
|
| 24 |
+
| | |
|
| 25 |
+
| --- | --- |
|
| 26 |
+
| parameters | **885,650** |
|
| 27 |
+
| architecture | `d_cell=48, d_model=128, n_layers=4, n_col_layers=2, d_ff=512`, two-way cell attention (`n_cell_blocks=1`, per-cell labels) |
|
| 28 |
+
| capacity | **binary only**, up to **136 features** |
|
| 29 |
+
| training | 6,000 steps × batch 8 = **48,000 synthetic tasks**, financial prior only |
|
| 30 |
+
| training data | **synthetic; no real table was seen during pretraining** |
|
| 31 |
+
|
| 32 |
+
This is the checkpoint on which **every published binary number in the repository was
|
| 33 |
+
measured**, so results quoted there are reproducible against this file rather than a variant of
|
| 34 |
+
it.
|
| 35 |
+
|
| 36 |
+
## Performance, stated honestly
|
| 37 |
+
|
| 38 |
+
On [TabArena](https://tabarena.ai), 27 binary datasets, against 94 other methods:
|
| 39 |
+
|
| 40 |
+
| | |
|
| 41 |
+
| --- | --- |
|
| 42 |
+
| mean ROC-AUC | **0.7823** |
|
| 43 |
+
| rank | **93 of 95** |
|
| 44 |
+
|
| 45 |
+
On real corporate-default panels, calibration is consistently among the best measured and
|
| 46 |
+
discrimination consistently loses to tuned gradient boosting — both, on every panel tried.
|
| 47 |
+
|
| 48 |
+
**This is not a competitive general tabular model and it is not presented as one.** It is
|
| 49 |
+
published so that the numbers in the repository can be checked, and because a provenance claim
|
| 50 |
+
nobody can verify is a slogan.
|
| 51 |
+
|
| 52 |
+
## Usage
|
| 53 |
+
|
| 54 |
+
```python
|
| 55 |
+
from fintfm.inference import FinancialTFMClassifier
|
| 56 |
+
|
| 57 |
+
clf = FinancialTFMClassifier("v4-cellattn-labels.pt", device="cpu")
|
| 58 |
+
clf.fit(X_train, y_train) # stores the table as context; no training happens
|
| 59 |
+
proba = clf.predict_proba(X_test)[:, 1]
|
| 60 |
+
```
|
| 61 |
+
|
| 62 |
+
Categorical columns must be encoded before they reach the model — it reads every cell as an
|
| 63 |
+
ordered scalar, and label encoding is measurably worse than no order at all. Use
|
| 64 |
+
`fintfm.inference.categorical.CategoricalTargetEncoder`, which is out-of-fold on the context
|
| 65 |
+
rows for reasons that are not optional.
|
| 66 |
+
|
| 67 |
+
## Limitations
|
| 68 |
+
|
| 69 |
+
- **Binary only, ≤136 features.** A task exceeding either raises rather than being silently
|
| 70 |
+
truncated.
|
| 71 |
+
- **Slow at inference**: median ~8.6 s per 1,000 rows, against a field norm near 0.1.
|
| 72 |
+
- **Loses to tuned gradient boosting** on discrimination, everywhere it has been measured.
|
| 73 |
+
- The multiclass and regression variants exist in the codebase, were scored on real data, and
|
| 74 |
+
**rank last** — they are deliberately not published.
|
| 75 |
+
|
| 76 |
+
## Licence
|
| 77 |
+
|
| 78 |
+
**Apache-2.0**, the same as the code — chosen deliberately rather than inherited. A weights
|
| 79 |
+
licence is a separate question from a code licence, and four of the ten peer projects surveyed
|
| 80 |
+
in the repository ship permissive code with **non-commercial weights**; one restricts commercial
|
| 81 |
+
use of the model's *output*. This checkpoint has no such restriction: commercial use is
|
| 82 |
+
permitted, subject to Apache-2.0's attribution and notice terms.
|
| 83 |
+
|
| 84 |
+
## Provenance
|
| 85 |
+
|
| 86 |
+
No code, weights, or training data from TabPFN, TabICL, TabDPT, LimiX, Nori, MITRA, or any other
|
| 87 |
+
tabular foundation model was used. The prior that generated this checkpoint's training data is
|
| 88 |
+
in the repository (`src/fintfm/prior/`) and can be inspected and re-run.
|