Pioupiou Table v0.23
A tabular foundation model written in Rust, pre-trained only on real tables.
This page presents the model. The weights and source code are not published here: Pioupiou Table runs inside Ricuicui, locally or through an API, and will be available there in the next few days.
Model description
Pioupiou Table predicts the missing cells of a table in a single pass. Give it a table where one column has empty cells: it reads the filled rows as examples (in-context learning) and predicts the rest, without retraining on your table.
| Architecture | Transformer, one token per cell |
| Parameters | 51.6 million |
| Layers / width / heads | 12 / 512 / 8 |
| Columns read at once | up to 64, with their names |
| Known rows | up to 512 |
| Output | class probabilities, or a distribution over intervals for numbers |
| Training data | more than 10,000 real public tables, all free to reuse, no generated tables |
| Implementation | Rust: model, GPU kernels and training, no machine-learning library |
| Precision | bf16 training |
| Developed by | Tony Belnoue, Tony dev web (France) |
How it works
- Each cell is a token. Each layer applies two attentions:
- between the columns of a row, to capture the links between columns;
- between the rows of a column, from the rows to predict towards the known rows.
- Column names are read, split into words and character trigrams.
- Values are read in two ways: at several frequencies, and by their rank in the column, so that extreme values do not throw the model off.
- Numbers are predicted as a histogram over intervals, which gives a range, not just a value.
- Faster than PyTorch: on a Mac chip, our Rust training step runs 2 to 3.5 times faster than the same step in PyTorch, with identical results (measured on an earlier, smaller version).
Uses
- Classification: will this customer leave? which category does this row belong to?
- Regression: what is the fair price of this property? what will next week's consumption be?
- Spotting unlikely values in a table (data-entry errors, faulty sensors).
- Combined with Pioupiou Language (a small Transformer, also in Rust), answering questions about a table in English, French, Spanish and German; the figures always come from the computation.
Evaluation
88 tables never seen during training, 5-fold cross-validation; score = accuracy for classification, R² for regression, averaged.
| Method | Score |
|---|---|
| Pioupiou Table v0.20 (previous version) | 0.423 |
| Neural network | 0.441 |
| Pioupiou Table v0.23 | 0.464 |
| Gradient boosting | 0.479 |
| Random forest | 0.492 |
The training tables were checked against leaks towards the evaluation tables, by source name and by shared rows.
Limitations
- It does not yet match the best tree-based methods on this benchmark.
- The benchmark is mostly regression (83 of 88 tables).
- Each version was trained once (single seed).
- It only answers questions whose answer can be measured in the table, on situations close to the known rows.