PrivTab

PrivTab is a tabular classifier that compresses a labeled context dataset into a reusable differentially private summary. Predictions on new rows use only that summary and never access the context data again.

This repository hosts the three PrivTab checkpoints. The code (model, deployment tools, synthetic task generator, DP-SGD baselines, benchmarks, audits, and Lean proofs) lives on GitHub.

• Code: https://github.com/TrustworthyMLHelsinki/PrivTab
• Archived release (DOI): [TODO: Zenodo DOI, e.g. 10.5281/zenodo.XXXXXXX]
• Paper: https://arxiv.org/abs/2610.10068
• License: MIT

Model details

PrivTab works in two separate steps:

1. Release (data owner). A context dataset (x, y) is passed through three private attention layers, followed by two post-processing layers, to produce a noisy summary of shape [B, 5, 128, 256] (640 KiB per task in float32). This step spends the privacy budget mu.
2. Prediction (anyone holding the summary). Query rows attend to the released summary to produce class probabilities. This is post-processing, so it costs no additional privacy budget, and it can be repeated with any number of query batches.

The architecture is defined in privtab/model.py in the GitHub repository. See the architecture guide for layer descriptions and the private summary boundary.

Checkpoints

All checkpoints are tensor-only weights.pt files (model parameters only, no optimizer state).

Folder	Training configuration	Context rows seen in training	Notes
stage1/	configs/privtab_stage1.yml	100–2048	Base checkpoint.
stage2_short_contexts/	configs/privtab_stage2_short_contexts.yml	100–2048	Continuation of stage 1.
stage2_long_contexts/	configs/privtab_stage2_long_contexts.yml	1024–8192	Continuation of stage 1.

For deployment, choose the checkpoint explicitly. For benchmark evaluation, the GitHub code routes by public context size: the short-context checkpoint below 4,096 context rows and the long-context checkpoint at or above 4,096. Output normalization is enabled during deployment for all checkpoints.

Supported inputs

• Numerical features only, all values finite
• 1–120 features
• 2–10 classes, given as a public class count and encoded as integers 0, …, C-1
• The same column order and the same public (or separately privatized) preprocessing for context and queries

Intended use

PrivTab is intended for releasing a differentially private model of a sensitive labeled tabular dataset, so that predictions can be made later, on a different machine and by different parties, without the context data. Typical settings are small-to-medium tabular classification tasks where the data owner wants a single, accountable privacy release.

Out-of-scope or unsafe uses

• Using model.summarize(...) or model.forward(...) to release outputs derived from private context data. These lower-level methods use PyTorch randomness and are meant for synthetic pretraining and public benchmark evaluation only. Use ReleasedSummary.create, model.release_summary, or deployment.export, which use OS CSPRNG-backed noise.
• Treating the privacy guarantee as covering preprocessing that was fit on private data (see Limitations).
• Publishing metrics computed from private query labels without accounting for that release.

How to get started

Install the code from GitHub (Python 3.11+ and PyTorch 2.6+ required; inference needs only PyTorch and NumPy):

git clone https://github.com/TrustworthyMLHelsinki/PrivTab.git
cd PrivTab
pip install -e .

Download a checkpoint from this repository:

from huggingface_hub import hf_hub_download

weights = hf_hub_download("YOUR_ORG/privtab", "stage2_short_contexts/weights.pt")

Release a private summary and predict

from privtab import ReleasedSummary
from privtab.checkpoints import load_model

model = load_model(weights)

# Preprocessed tensors:
#   xc [context_rows, features], yc [context_rows], xt [query_rows, features]
release = ReleasedSummary.create(model, xc, yc, mu=0.4, num_classes=2)
release.save("summary.pt")

saved = ReleasedSummary.load("summary.pt")
probabilities = saved.predict_proba(model, xt)

ReleasedSummary.create spends the privacy budget once. Loading the summary and predicting from it do not create another release.

Export once, predict separately (command line)

On the data owner's machine, with context.npz containing x of shape [context_rows, features] and integer y of shape [context_rows]:

python -m deployment.export \
  --weights path/to/stage2_short_contexts/weights.pt \
  --context context.npz --mu 0.4 --num-classes 2 \
  --output releases/task.pt

On the prediction machine, with queries.npy of shape [query_rows, features]:

python -m deployment.predict \
  --bundle releases/task.pt --features queries.npy \
  --output probabilities.npy --batch-size 1024

The bundle contains the noisy summary, predictor weights, and public feature and class counts. It contains no context rows, labels, private preprocessing statistics, optimizer state, or noise seeds. Export refuses to overwrite an existing bundle to help avoid accidental re-release.

A complete runnable example using scikit-learn's public digits dataset is in examples/digits_playground.py in the GitHub repository.

Privacy guarantee

• Adjacency: replace-one row adjacency on the context dataset.
• Mechanism: each of the three private attention layers uses tanh-bounded attention and unit-normalized values, with Gaussian noise of standard deviation 2 * sqrt(128 * 3) / mu.
• Composition: the three adaptive releases compose to the supplied mu (Gaussian differential privacy). Query prediction is post-processing.
• Repeated releases: a fresh summary is a fresh privacy release, even for the same dataset. Repeated releases compose as sqrt(sum(mu_i**2)). Track the budget across every summary or other private-data release, including repeated exports and any model or parameter selection based on private data.
• Noise generation: private releases use secrets.token_bytes and Box–Muller to generate Gaussian noise from the OS CSPRNG. They do not accept or save a reproducible noise seed.

Verification

• Lean proofs. The Lean project proves the universal 2 * sqrt(H * m) replace-one sensitivity bound for the exact real-valued tanh attention and epsilon-normalized value formula, formalizes adaptive μ-GDP composition of the private encoder layers, and proves privacy preservation through prediction. A proof-to-code map links each Lean result to the corresponding Python operation. The formalization assumes the Gaussian-mechanism and GDP composition rules as explicit interfaces.
• Empirical implementation audits. experiments/audit_rowwise.py and experiments/dp_audit.py check the Python implementation: row-wise dependence of the first private-attention boundary, the noise scale passed to all three private layers, and the selected layer's actual noise path.

These checks complement each other but do not certify floating-point execution or private preprocessing.

Training details

Training data

PrivTab is pretrained on synthetic tabular classification tasks produced by the synthetic task generator in the GitHub repository. [TODO: confirm that no real-world datasets were used in pretraining.] See the pretraining guide for the task distribution, context lengths, and the mu curriculum.

Training procedure

• Stage 1: trained from scratch on synthetic tasks with contexts of 100–2048 rows.
• Stage 2 (short contexts): continues from stage 1 with contexts of 100–2048 rows.
• Stage 2 (long contexts): continues from stage 1 with contexts of 1024–8192 rows. Output normalization is enabled during this continuation.
• Optimization: single-device runner with AdamW, gradient accumulation, encoder freezing, and the schedules specified in the YAML configurations (included in configs/).

Training can be reproduced with python -m experiments.train --config <config>.yml --output-dir runs/<name> from the GitHub repository.

Evaluation

Protocol

• Benchmark: the 33 TabArena v0.1 datasets, prepared from OpenML (class encoding, ordinal encoding of categoricals, numeric imputation, constant-column filter; at most 100,000 rows, 120 features, and 10 classes), using saved 80/20 splits shared across methods.
• Baselines: DP Logistic Regression and a one-hidden-layer (width 64) DP MLP, both trained with DP-SGD and private learning-rate selection. Half of the budget (0.5 * mu) goes to selection, using a noisy mean cross-entropy over candidate learning rates.
• Metrics and ranking: ELO ranking per privacy level and pooled across levels, with task-bootstrap confidence intervals. Binary tasks use 1 - AUC and multiclass tasks use log loss. DP Logistic Regression is calibrated to ELO 1000.
• Case studies: end-to-end comparisons that include private preprocessing with oracle and cached public bounds.

Results

The benchmark commands are public research benchmarks. Their empirical context standardization and label-aware outer splits are not an end-to-end private preprocessing pipeline.

Limitations

• Preprocessing is not covered by the guarantee. PrivTab does not fit preprocessing on private data. Empirical context means, standard deviations, class vocabularies, or feature selection are not automatically covered. Supply a public schema and a fixed public transform, or account separately for private preprocessing. The exported bundle expects already transformed queries.
• Floating-point implementation. The noise generator is a floating-point implementation of the ideal Gaussian mechanism. An exact bit-level DP proof is separate from the real-valued analysis.
• Scope of the formal proofs. The Lean formalization does not prove a concrete correspondence between Lean states and Python tensors, and does not verify the floating-point sampler or private-data preprocessing.
• Public side information. Feature bounds and the class vocabulary should be public, or the way they were obtained must be accounted for separately.
• Utility versus privacy. Noise scales as 1/mu, so smaller mu gives stronger privacy and lower expected accuracy. Utility also depends on context size.
• Supported data only. Numerical features, 1–120 features, 2–10 classes. Regression and unsupported feature types are out of scope.
• Benchmarks are on public data. Public benchmark results do not by themselves demonstrate performance or privacy on a given private dataset.

Citation

@article{privtab,
  title   = {Efficient Provably Private Classification with a Tabular Foundation Model},
  author  = {Alrawajfeh, Talal and Diaconu, Cristiana and R{\"a}is{\"a}, Ossi and Rodriguez Beltran, Sebastian and He, Yuan and Bronskill, John and Turner, Richard E. and Honkela, Antti},
  journal = {arXiv preprint arXiv:2610.10068},
  year    = {2026}
}

If you use the code or checkpoints, please also cite the archived release: [TODO: Zenodo DOI].

Acknowledgements and third-party code

Third-party source attributions are given in the LICENSE file and in the prior and accountant modules of the GitHub repository.