Delete README.md
Browse files
README.md
DELETED
|
@@ -1,89 +0,0 @@
|
|
| 1 |
-
# CIS 6270 Course Code
|
| 2 |
-
|
| 3 |
-

|
| 4 |
-
|
| 5 |
-
Code for CIS 6270, Discrete Generative Models, at the University of Pennsylvania. One file per
|
| 6 |
-
framework, written to be read beside the lecture notes: the corruption, the loss and the sampler of
|
| 7 |
-
each method are in that file, in the notes' own symbols, with a comment on almost every line.
|
| 8 |
-
|
| 9 |
-
Everything runs offline on a laptop CPU in seconds. The one exception is
|
| 10 |
-
`lecture_2/flow_matching_mnist.py`, which downloads MNIST, as its name says; `--dataset toy` runs it
|
| 11 |
-
offline. Lecture 3 goes the other way: its scripts run offline and reach MNIST or ESM-2 only when
|
| 12 |
-
you ask with `--dataset mnist` or `--dataset esm2`.
|
| 13 |
-
|
| 14 |
-
```bash
|
| 15 |
-
git clone https://huggingface.co/ChatterjeeLab/CIS6270
|
| 16 |
-
cd CIS6270
|
| 17 |
-
python3 -m venv .venv && source .venv/bin/activate
|
| 18 |
-
python -m pip install -r requirements.txt
|
| 19 |
-
|
| 20 |
-
python lecture_4/mdlm.py # train masked diffusion on DNA and generate, about 20 seconds
|
| 21 |
-
python -m pytest tests -q # the notes' worked numbers, checked against the code
|
| 22 |
-
```
|
| 23 |
-
|
| 24 |
-
On Windows PowerShell, activate with `.venv\Scripts\Activate.ps1`.
|
| 25 |
-
|
| 26 |
-
## The lectures
|
| 27 |
-
|
| 28 |
-
| | Lecture | What the code builds | Guide |
|
| 29 |
-
| --- | --- | --- | --- |
|
| 30 |
-
| 1 | Mathematical Foundations | Every worked example of the chapter, printed beside the chapter's own number | [lecture_1](lecture_1/README.md) |
|
| 31 |
-
| 2 | Flow Matching | Velocity fields and the flow map, the conditional path, the objective, MNIST | [lecture_2](lecture_2/README.md) |
|
| 32 |
-
| 3 | Score Matching and Diffusion | The forward SDE, the three score objectives, DDPM, three samplers, guidance, latent diffusion on protein embeddings | [lecture_3](lecture_3/README.md) |
|
| 33 |
-
| 4 | Discrete Diffusion | Jump processes, MDLM, uniform replacement, D3PM, SEDD, block diffusion, guidance, planners, search | [lecture_4](lecture_4/README.md) |
|
| 34 |
-
| 5 | Discrete Flow Matching | The mixture path and its velocity, correctors, kinetic-optimal paths, Edit Flows, Dirichlet, Fisher and Gumbel geometries, guidance, rectification | [lecture_5](lecture_5/README.md) |
|
| 35 |
-
| 6 | Flow Maps | The three axioms, the three residuals, MeanFlow, Shortcut, consistency, the discrete flow maps, stochastic, expanding and posterior maps | [lecture_6](lecture_6/README.md) |
|
| 36 |
-
| 7 | Optimal Transport and Schrödinger Bridges | Kantorovich and duality, Sinkhorn, displacement interpolation, static and dynamic bridges, DSB, DSBM, SF2M, the discrete bridges, branched and entangled | [lecture_7](lecture_7/README.md) |
|
| 37 |
-
|
| 38 |
-
## How a file is laid out
|
| 39 |
-
|
| 40 |
-
Every script follows the shape of `lecture_2/flow_matching_mnist.py`: a docstring naming the
|
| 41 |
-
framework and the chapter sections it implements, numbered `# %%` sections in the order the lecture
|
| 42 |
-
develops them, and a `main()` that prints a report.
|
| 43 |
-
|
| 44 |
-
```
|
| 45 |
-
cis6270/ the scaffolding, read once: data, the four small networks, the training loop,
|
| 46 |
-
the sample metrics. None of it is the subject of a lecture.
|
| 47 |
-
lecture_1/ ... 7/ one file per framework, standalone and runnable
|
| 48 |
-
tests/ the chapters' worked numbers, asserted against the code
|
| 49 |
-
```
|
| 50 |
-
|
| 51 |
-
Nothing in `lecture_N/` imports anything from another `lecture_N/`. To read a method, open its file.
|
| 52 |
-
|
| 53 |
-
## The data
|
| 54 |
-
|
| 55 |
-
| Data | Used by | Download |
|
| 56 |
-
| --- | --- | --- |
|
| 57 |
-
| Synthetic DNA, four letters, a planted ACGT motif, a GC label | Lectures 4 to 6 | no |
|
| 58 |
-
| Four 2-D Gaussians, two moons, a checkerboard | Lectures 2, 3, 6, 7 | no |
|
| 59 |
-
| Four-token phrases such as `red circle moves left` | Lecture 6 | no |
|
| 60 |
-
| MNIST | Lectures 2, 3, with `--dataset mnist` | yes |
|
| 61 |
-
| ESM-2 residue embeddings | Lecture 3, with `--dataset esm2` | yes |
|
| 62 |
-
|
| 63 |
-
The DNA is synthetic and the two guidance objectives are explicit toy properties. They are there so
|
| 64 |
-
that a training run finishes while you watch it, and so that the structure a sampler is supposed to
|
| 65 |
-
recover is one you can check by eye. Nothing here is a benchmark or a biological predictor.
|
| 66 |
-
|
| 67 |
-
## The shared flags
|
| 68 |
-
|
| 69 |
-
`--steps`, `--batch-size`, `--lr`, `--width`, `--length`, `--samples`, `--sample-steps`, `--seed`,
|
| 70 |
-
`--device`, `--out`, `--quiet`. Learn them once and they work everywhere. `--out` writes
|
| 71 |
-
`checkpoint.pt`, `config.json`, `report.json`, `losses.json` and `samples.txt`.
|
| 72 |
-
|
| 73 |
-
The defaults are a short budget, enough to see a method work in well under a minute. A method
|
| 74 |
-
usually looks better with `--steps 1500 --width 128`.
|
| 75 |
-
|
| 76 |
-
## Tests
|
| 77 |
-
|
| 78 |
-
```bash
|
| 79 |
-
python -m pytest tests -q
|
| 80 |
-
```
|
| 81 |
-
|
| 82 |
-
The suite asserts the numbers the notes work by hand: the posterior average of the reverse rates on
|
| 83 |
-
one DNA base, the Sinkhorn iterates at `epsilon = 2`, the cocycle and Jacobian of the Gaussian flow,
|
| 84 |
-
the conditional velocity of the mixture path. A number in the book and a number in the code are the
|
| 85 |
-
same number, and the suite is what keeps them that way.
|
| 86 |
-
|
| 87 |
-
## License
|
| 88 |
-
|
| 89 |
-
[MIT](LICENSE). ESM-2 weights come from their original repository under its own terms.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|