WyFormer-LeMat-Chemsys

A WyFormer generator of Wyckoff representations, conditioned on a chemical system. Name a set of elements, e.g. Ba-Ti-O. The model picks the space group, the stoichiometry and the Wyckoff positions, and every generated structure contains only those elements.

pip install wyckoff-transformer
wyformer-generate genes.json.gz --hf-model SymmetryAdvantage/WyFormer-LeMat-Chemsys \
    --chemical-system Ba-Ti-O --initial-n-samples 1100 --firm-n-samples 1000

The repository README shows how to rank the genes with WyFormer-LeMat-Energy and relax them into 3D structures with an MLIP.

What the conditioning means

The model learns p(gene | exactly these elements are present). Asking for Ba-Ti-O asks for compounds containing all three elements. The sampling mask still allows subsets, such as TiO2, but the model is not encouraged to produce them. For every oxide of Ba and Ti, run each subsystem separately and pool the results. The space group is sampled from the model's own p(space group | system). --space-group fixes it instead.

Training

• Data: LeMat-Bulk-fixed at revision bf8f0b31. Training used structures with residual force ≤ 1 eV/Å and e_hull ≤ 0.1 eV/atom against the PBE hull of the whole dataset.
• Model: 3-layer transformer encoder with AdaLN conditioning on a multi-hot element-set vector, der_tokenizer_v1 tokens.
• Optimisation: AdamW with a warmup-stable-decay schedule. W&B run chemsys_sg_uncond_adanmw_wsd_long-20261015-044354, best validation loss 11.24.

Evaluation

Measured with the de novo ranking protocol: 1,000 genes; CrySPR with ORB v3 (orb_conserv_inf); novelty against LeMat-Bulk-fixed. Rates are per sampled gene.

metric	value
MetaSUN (e_hull ≤ 0.1, after symmetry release)	0.363
metastable	0.472
SUN (e_hull ≤ 0)	0.007
gene novelty	0.801
valid and unique structures	0.925

Aiming at chosen systems and filtering before relaxation raises the yield. 50 ternary targets with their binaries, 20,000 genes, then the 1,000 that are not in LeMat-Bulk-fixed and are predicted lowest against the hull by WyFormer-LeMat-Energy. Rates are per relaxed gene.

hull the predictions are compared with	MetaSUN	SUN
the PBE hull plus the other candidates	0.619	0.030
the PBE hull alone	0.591	0.031