--- license: mit library_name: pytorch tags: - chemistry - smiles - h-net --- # hnet-smiles Eight byte-level H-Nets ([Hwang, Wang, Gu](https://arxiv.org/abs/2507.07955)) trained on SMILES. All share one two-stage architecture (bytes to pieces to groups, Transformer stacks, cosine router) and one dataset. They differ in the training objective, and so in where they cut a molecule. Code: [DrHB/hnet-smiles](https://github.com/DrHB/hnet-smiles). Post: [How the training objective shapes learned tokens](https://drhb.github.io/blog/posts/learned-cuts/). ## Models Each folder holds `model.safetensors` and `config.json`. | folder | objective | parameters | held-out metric | |---|---|---|---| | `next-byte-30m` | predict the next byte, causal | 30,342,784 | 0.7681 bits per byte | | `fingerprint-30m` | predict the 2,048-bit Morgan fingerprint (radius 3) from the clean string | 32,901,248 | Tanimoto 0.831 | | `descriptors-30m` | regress 185 RDKit descriptors from the clean string | 30,991,673 | R² 0.998 | | `masked-fragments-30m` | fill in masked bytes, 25 to 45% per molecule, whole fragments first | 30,342,784 | 92.8% of masked bytes at a 35% rate | | `next-byte-20m` | as above | 19,555,520 | 0.7681 bits per byte | | `fingerprint-20m` | as above, head reads one position | 22,245,184 | Tanimoto 0.832 | | `descriptors-20m` | as above, head reads one position | 20,335,609 | R² 0.998 | | `masked-fragments-20m` | as above | 19,555,520 | 93.2% of masked bytes at a 35% rate | The 30M round uses widths 256/384/512, AdamW and 3 epochs (101,289 steps). The 20M round uses widths 256/320/384, Muon and 5 epochs (168,815 steps). In the 30M round the fingerprint and descriptor heads read the mean and max over bytes; in the 20M round they read one position. ## Where they cut 50,000 held-out molecules, first-level pieces. | | bytes per piece | distinct pieces | multi-character atoms split | preference for BRICS bonds | |---|---|---|---|---| | `next-byte-30m` | 2.75 | 8,620 | 18% | -0.04 | | `fingerprint-30m` | 2.59 | 4,271 | 5% | +0.16 | | `descriptors-30m` | 2.82 | 3,810 | 65% | +0.14 | | `masked-fragments-30m` | 4.33 | 14,928 | 0.4% | +0.38 | | `next-byte-20m` | 2.76 | 10,450 | 6% | -0.06 | | `fingerprint-20m` | 2.91 | 15,640 | 70% | +0.14 | | `descriptors-20m` | 2.84 | 3,709 | 29% | +0.13 | | `masked-fragments-20m` | 4.33 | 14,262 | 2% | +0.39 | ## Data 90,955,563 canonical, non-stereo SMILES: PubChem (89.8M rows), COCONUT, ChEBI, LIPID MAPS and structures from public spectral libraries. 50,000 rows were held out. Input is raw bytes with BOS, EOS, PAD and MASK symbols (vocabulary 260). ## Use ```bash git clone https://github.com/DrHB/hnet-smiles && cd hnet-smiles && pip install -r requirements.txt ``` ```python import torch from data import encode, split from model import load model = load("Asimo/hnet-smiles/masked-fragments-30m") smiles = "CC(C)S(=O)(=O)CCSc1cccc(C(=O)O)c1" x, n = encode([smiles]) with torch.no_grad(): out, r = model(x, n) print(split(smiles, r["b"][0])) # ['CC(C)', 'S(=O)(=O)', 'CC', 'Sc', '1cccc', '(C(=O)O', ')c1'] ``` `out` is byte logits for the next-byte and masked-fragment models, 2,048 fingerprint logits for the fingerprint models, and 185 descriptor z-scores for the descriptor models. `r["b"]` marks piece starts, `r["bg"]` group starts, and `r["main"]` is the main-network output. ## Limits Small models, one notation (canonical SMILES), Transformer outer layers where the paper uses Mamba-2. Benchmark molecules were not removed from pretraining; no benchmark label was in the corpus. ## Licence MIT.