kabartay commited on
Commit
f7e91ca
·
verified ·
1 Parent(s): cdd11f5

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +88 -0
README.md ADDED
@@ -0,0 +1,88 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: apache-2.0
3
+ library_name: pytorch
4
+ tags:
5
+ - tabular
6
+ - tabular-classification
7
+ - in-context-learning
8
+ - foundation-model
9
+ - credit-risk
10
+ - synthetic-data
11
+ pipeline_tag: tabular-classification
12
+ ---
13
+
14
+ # fintfm — binary classification checkpoint
15
+
16
+ An in-context tabular classifier for corporate credit risk, **pretrained only on synthetic
17
+ data**. Predictions are a single forward pass with your table supplied as context: no gradient
18
+ steps at fit time, no per-dataset tuning.
19
+
20
+ Code, measurement log and claims ledger: **https://github.com/kabartay/fintfm**
21
+
22
+ ## What this checkpoint is
23
+
24
+ | | |
25
+ | --- | --- |
26
+ | parameters | **885,650** |
27
+ | architecture | `d_cell=48, d_model=128, n_layers=4, n_col_layers=2, d_ff=512`, two-way cell attention (`n_cell_blocks=1`, per-cell labels) |
28
+ | capacity | **binary only**, up to **136 features** |
29
+ | training | 6,000 steps × batch 8 = **48,000 synthetic tasks**, financial prior only |
30
+ | training data | **synthetic; no real table was seen during pretraining** |
31
+
32
+ This is the checkpoint on which **every published binary number in the repository was
33
+ measured**, so results quoted there are reproducible against this file rather than a variant of
34
+ it.
35
+
36
+ ## Performance, stated honestly
37
+
38
+ On [TabArena](https://tabarena.ai), 27 binary datasets, against 94 other methods:
39
+
40
+ | | |
41
+ | --- | --- |
42
+ | mean ROC-AUC | **0.7823** |
43
+ | rank | **93 of 95** |
44
+
45
+ On real corporate-default panels, calibration is consistently among the best measured and
46
+ discrimination consistently loses to tuned gradient boosting — both, on every panel tried.
47
+
48
+ **This is not a competitive general tabular model and it is not presented as one.** It is
49
+ published so that the numbers in the repository can be checked, and because a provenance claim
50
+ nobody can verify is a slogan.
51
+
52
+ ## Usage
53
+
54
+ ```python
55
+ from fintfm.inference import FinancialTFMClassifier
56
+
57
+ clf = FinancialTFMClassifier("v4-cellattn-labels.pt", device="cpu")
58
+ clf.fit(X_train, y_train) # stores the table as context; no training happens
59
+ proba = clf.predict_proba(X_test)[:, 1]
60
+ ```
61
+
62
+ Categorical columns must be encoded before they reach the model — it reads every cell as an
63
+ ordered scalar, and label encoding is measurably worse than no order at all. Use
64
+ `fintfm.inference.categorical.CategoricalTargetEncoder`, which is out-of-fold on the context
65
+ rows for reasons that are not optional.
66
+
67
+ ## Limitations
68
+
69
+ - **Binary only, ≤136 features.** A task exceeding either raises rather than being silently
70
+ truncated.
71
+ - **Slow at inference**: median ~8.6 s per 1,000 rows, against a field norm near 0.1.
72
+ - **Loses to tuned gradient boosting** on discrimination, everywhere it has been measured.
73
+ - The multiclass and regression variants exist in the codebase, were scored on real data, and
74
+ **rank last** — they are deliberately not published.
75
+
76
+ ## Licence
77
+
78
+ **Apache-2.0**, the same as the code — chosen deliberately rather than inherited. A weights
79
+ licence is a separate question from a code licence, and four of the ten peer projects surveyed
80
+ in the repository ship permissive code with **non-commercial weights**; one restricts commercial
81
+ use of the model's *output*. This checkpoint has no such restriction: commercial use is
82
+ permitted, subject to Apache-2.0's attribution and notice terms.
83
+
84
+ ## Provenance
85
+
86
+ No code, weights, or training data from TabPFN, TabICL, TabDPT, LimiX, Nori, MITRA, or any other
87
+ tabular foundation model was used. The prior that generated this checkpoint's training data is
88
+ in the repository (`src/fintfm/prior/`) and can be inspected and re-run.