eurecom-probai/alice-1.0-base

✨ What is ALICE?

ALICE stands for Adaptive Latent In-Context Estimator.

ALICE is the first foundation model for zero-shot mutual-information estimation. It uses a single Transformer trained once on a broad atlas of synthetic distributions. Given context samples from unseen distributions, ALICE estimates their mutual information!

ALICE does not require ground-truth mutual information values for inference: it only requires a few samples!

ALICE does not require any fine-tuning or training on the target distribution, making it a powerful tool for researchers and practitioners in the field of information theory, machine learning, and data science.

🚀 Quickstart

Clone the ALICE repository and install the package and its dependencies with uv:

git clone https://github.com/eurecom-probai/alice.git
cd alice
uv sync --locked

Load a Hugging Face checkpoint or a local save_pretrained directory:

from alice import load_model

model = load_model(
    "eurecom-probai/alice-1.0-base",
    device="cuda",  # optional; defaults to "cpu"
)

ALICE checkpoints load their model implementation through Hugging Face custom code with trust_remote_code=True; review the checkpoint source and revision before loading it.

Model configuration — ALICE 1.0 Base

eurecom-probai/alice-1.0-base is the Base checkpoint of ALICE 1.0.

Configuration	Value
Parameters	85,200,633 (85.2M)
Architecture	InducedGroupICLDenoiserModel
Hidden width (d_model)	768
Layers (n_layers)	6
Attention heads (n_heads)	12
Feed-forward width (dim_feedforward)	3,072
Inducing latents (num_inducing_latents)	128
Relation features (relation_features)	16
Time-frequency bands (num_freqs)	16
Dropout (dropout)	0.1
Mask channel (use_mask_channel)	Enabled
Variable dimensionality (variable_dim)	Enabled
Checkpoint dtype	float32

Citation

The paper will be released shortly. For now, please cite this repository:

@misc{alice2026,
  author = {Franzese, Giulio and Rossi, Simone and Michiardi, Pietro},
  title = {ALICE: Adaptive Latent In-Context Estimator},
  year = {2026},
  url = {https://github.com/eurecom-probai/alice},
  note = {Software repository; paper forthcoming},
}

License

PolyForm Noncommercial License 1.0.0

This model release is licensed under the PolyForm Noncommercial License 1.0.0. It permits use, modification, and redistribution for noncommercial purposes, subject to the terms in the license. The license includes a patent license from the licensor.

_{✨ Research-ready inference for context-aware denoising and mutual-information estimation. ✨}