DabinJeong commited on
Commit
a8b4704
Β·
verified Β·
1 Parent(s): 303fd60

Add V-REx masked separator checkpoints (HepG2, K562, SK-N-SH) and model card

Browse files
README.md CHANGED
@@ -1,3 +1,158 @@
1
  ---
2
  license: mit
 
 
 
 
 
 
 
 
 
3
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
  license: mit
3
+ library_name: pytorch
4
+ tags:
5
+ - biology
6
+ - dna
7
+ - genomics
8
+ - regulatory-dna
9
+ - enhancer
10
+ - discrete-diffusion
11
+ - guided-generation
12
  ---
13
+
14
+ # LocoGen β€” V-REx masked separator checkpoints
15
+
16
+ Separator weights for **LocoGen**: locality-controlled guidance for masked
17
+ discrete diffusion of regulatory DNA. LocoGen samples enhancer sequences from a
18
+ *frozen* MDLM backbone, with no fine-tuning, by localizing the guidance gradient
19
+ to invariant regulatory positions using a V-REx-trained masked separator.
20
+
21
+ Code: https://github.com/DabinJeong/LocoGen
22
+
23
+ ## What this repository contains
24
+
25
+ Only the **masked separator** β€” the component trained in LocoGen's Stage 1.
26
+ One checkpoint per cell line.
27
+
28
+ | File | Cell line | Params |
29
+ | --- | --- | --- |
30
+ | `hepg2/masked_separator.ckpt` | HepG2 | 11,135,495 |
31
+ | `k562/masked_separator.ckpt` | K562 | 11,135,495 |
32
+ | `sknsh/masked_separator.ckpt` | SK-N-SH | 11,135,495 |
33
+
34
+ Each is a plain `{"model": state_dict}` payload (316 tensors, ~45 MB).
35
+
36
+ ## What it does *not* contain
37
+
38
+ LocoGen runs on the **DRAKES** MDLM backbone and uses the DRAKES reward oracles.
39
+ Those are a separate release and are not redistributed here. You need them to
40
+ sample anything. Fetch both as described in the
41
+ [LocoGen README](https://github.com/DabinJeong/LocoGen#external-dependencies-one-time):
42
+
43
+ - `mdlm/outputs_gosai/pretrained.ckpt` β€” frozen MDLM backbone
44
+ - `mdlm/outputs_gosai/lightning_logs/reward_oracle_ft.ckpt` β€” guidance oracle
45
+ - `mdlm/outputs_gosai/lightning_logs/reward_oracle_eval.ckpt` β€” held-out scoring
46
+ - `mdlm/gosai_data/processed_data/gosai_all.csv` β€” Gosai dataset
47
+
48
+ ## Architecture comes from the config, not the checkpoint
49
+
50
+ The loader builds the model from a YAML config and then loads these weights into
51
+ it. The checkpoint carries no architecture metadata, so you must pass the
52
+ matching config from the code repository:
53
+
54
+ | Checkpoint | `--ms_config` | `target_idx` |
55
+ | --- | --- | --- |
56
+ | `hepg2/masked_separator.ckpt` | `configs/enhancer_gosai_masked_separator.yaml` | 0 |
57
+ | `k562/masked_separator.ckpt` | `configs/enhancer_gosai_masked_separator_k562.yaml` | 1 |
58
+ | `sknsh/masked_separator.ckpt` | `configs/enhancer_gosai_masked_separator_sknsh.yaml` | 2 |
59
+
60
+ At inference only the architecture fields and `target_idx` are read; the
61
+ training-side loss weights in those configs do not affect sampling.
62
+
63
+ ## Provenance β€” the three checkpoints are *not* from the same epoch
64
+
65
+ This is the easiest thing to get wrong, so it is stated explicitly.
66
+
67
+ | Cell line | Source checkpoint | Actual epoch | Stored training loss |
68
+ | --- | --- | --- | --- |
69
+ | HepG2 | `masked_separator_best.ckpt` | **4** | 0.299881212299045 |
70
+ | K562 | `masked_separator_total_epoch2.ckpt` | 2 | 0.6674904911338998 |
71
+ | SK-N-SH | `masked_separator_total_epoch2.ckpt` | 2 | 0.6402428685669742 |
72
+
73
+ These are the checkpoints the reported results were produced with, recovered from
74
+ the original job launchers. For HepG2 the launcher passed no `--ms_ckpt`, so the
75
+ sampler default of the time (`masked_separator_best.ckpt`) applied; for K562 and
76
+ SK-N-SH the launcher named `masked_separator_total_epoch2.ckpt` explicitly.
77
+
78
+ > **Note on the discarded `epoch` field.** The original payloads stored an
79
+ > `epoch` integer that did **not** match the real epoch β€” it was a resumed run's
80
+ > zero-based index, so HepG2's epoch-4 checkpoint recorded `2`, colliding
81
+ > numerically with the unrelated `total_epoch2.ckpt` file (a different model,
82
+ > used only for an ablation). That field has been dropped from these release
83
+ > checkpoints rather than corrected. Older copies of the LocoGen README and the
84
+ > sampler's `--ms_ckpt` help text describe "epoch 2" as the reference separator;
85
+ > that is accurate for K562 and SK-N-SH but **not** for HepG2.
86
+
87
+ `best` was selected on lowest mean *training* loss, with no held-out model
88
+ selection. The loss fell monotonically, so it is simply the last epoch that
89
+ completed.
90
+
91
+ ## Usage
92
+
93
+ Download a checkpoint and point the LocoGen sampler at it:
94
+
95
+ ```bash
96
+ export LOCOGEN_DRAKES_REPO=/path/to/DRAKES/drakes_dna
97
+ export LOCOGEN_DRAKES_DATA=/path/to/data_and_model/
98
+
99
+ CKPT=$(python -c "from huggingface_hub import hf_hub_download; \
100
+ print(hf_hub_download('DabinJeong/LocoGen', 'hepg2/masked_separator.ckpt'))")
101
+ ```
102
+
103
+ Then run Stage 2 sampling. These are the full reference settings β€” `--lam` and
104
+ `--alpha_ratio` alone are not sufficient to reproduce the reported numbers:
105
+
106
+ ```bash
107
+ python -u -m scripts.drakes_backbone.mood_guided_sample_mdlm \
108
+ --name locogen_hepg2 \
109
+ --cell_type hepg2 \
110
+ --ms_config ./configs/enhancer_gosai_masked_separator.yaml \
111
+ --ms_ckpt "$CKPT" \
112
+ --lam 0.7 \
113
+ --alpha_ratio 3.0 \
114
+ --alpha_cap 20.0 \
115
+ --ood_sil_localized true \
116
+ --ood_sil_threshold 0.5 \
117
+ --sil_t_threshold 0.3 \
118
+ --num_samples 640 \
119
+ --batch_size 64 \
120
+ --seed 0
121
+ ```
122
+
123
+ `--num_steps` is left at its default of 128, as in the original runs.
124
+
125
+ For K562 or SK-N-SH, swap `--cell_type`, `--ms_config` and `--ms_ckpt` together β€”
126
+ the oracle head is selected by `--cell_type`, so a mismatched pair silently
127
+ scores against the wrong cell line.
128
+
129
+ The run writes `sequences.csv` (one row per generated 200 nt sequence) and a
130
+ `summary.json`. Reported results average over seeds 0, 1 and 2.
131
+
132
+ Loading the weights directly, without the sampler:
133
+
134
+ ```python
135
+ import torch
136
+ from huggingface_hub import hf_hub_download
137
+ from omegaconf import OmegaConf
138
+ from locogen.model.masked_separator import MaskedSeparatorModel
139
+
140
+ ckpt = hf_hub_download("DabinJeong/LocoGen", "hepg2/masked_separator.ckpt")
141
+ cfg = OmegaConf.load("configs/enhancer_gosai_masked_separator.yaml")
142
+ model = MaskedSeparatorModel(cfg)
143
+ model.load_state_dict(torch.load(ckpt, map_location="cpu")["model"])
144
+ model.eval()
145
+
146
+ # x: (B, 200, 4) soft one-hot -> M: (B, 200) mask in [0, 1]
147
+ M = model.separator(x)
148
+ ```
149
+
150
+ ## License
151
+
152
+ MIT. The DRAKES backbone, oracles and dataset are covered by their own terms.
153
+
154
+ ## Citation
155
+
156
+ LocoGen, Dabin Jeong. See the
157
+ [code repository](https://github.com/DabinJeong/LocoGen) for the current
158
+ reference and publication status.
hepg2/masked_separator.ckpt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f0a2111701945fd1e38aa3ef6fbbde80e204e97a485a520218e03f167c8a0365
3
+ size 44648954
k562/masked_separator.ckpt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f7d3a6a53bb122dd5072d9f4bc42ef0a3fd68d2bbcd829d10d8103fe663f4ea4
3
+ size 44648954
sknsh/masked_separator.ckpt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:eb4de32ee9a1c2b5d26da92129aeabfeac63690aa44ba69c6c3dd13fefaa28a7
3
+ size 44648954