Repository card: folder layout and links
Browse files
README.md
ADDED
|
@@ -0,0 +1,28 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
language: [en, vi, ja]
|
| 3 |
+
tags: [anlp-assignment-2, mixture-of-experts, optimizers, decoding]
|
| 4 |
+
datasets:
|
| 5 |
+
- belumind/en-vi-ja-curated-500k-triplets
|
| 6 |
+
- browndw/human-ai-parallel-corpus
|
| 7 |
+
- hamishivi/ROCStories
|
| 8 |
+
---
|
| 9 |
+
|
| 10 |
+
# ANLP Assignment 2 (Monsoon 2026): all checkpoints and raw generations
|
| 11 |
+
|
| 12 |
+
Every model trained for the assignment lives in this one repository, one folder per run. All training logs are on
|
| 13 |
+
W&B: <https://wandb.ai/tchaitanya0109-iiit-hyderabad/anlp-a2> (public). Code, README and report: the course GitLab
|
| 14 |
+
repository `advanced-nlp/m26-assignment2-chaitanya.thogata` on code.iiit.ac.in.
|
| 15 |
+
|
| 16 |
+
| folder | contents |
|
| 17 |
+
|---|---|
|
| 18 |
+
| `part1/<variant>/` | Part 1, seed 42: decoder-only translation models (vi/ja → en) trained from scratch for 725M tokens. Variants: `dense`, `moe-e4k1` (4 experts, top-1), `moe-e4k2` (top-2), `moe-s1e3k1` (1 shared + 3 routed, top-1), `moe-e4k2-active` (top-2, active params = dense) |
|
| 19 |
+
| `part1/seed43-<variant>/` | the same five configurations re-trained with seed 43 (seed-variance replicates) |
|
| 20 |
+
| `part2/<optimizer>/` | Part 2, seed 42: the Part 1 dense configuration pre-trained from scratch for 1× `browndw/human-ai-parallel-corpus` (40.3M tokens) with `adamw`, `mars`, `lion`, `muon`, `sophia` (bonus); `final/` plus intermediate checkpoints `tok-0.1x/` … `tok-1.0x/` every 0.1× of the dataset |
|
| 21 |
+
| `part2/seed43-<optimizer>/` | the same five Part 2 runs with seed 43 |
|
| 22 |
+
| `part3/generations/` | Part 3: raw generations of `EleutherAI/pythia-160m` on 1,000 ROCStories test stories for greedy, beam (W = 1, 2, 4), top-k and top-p decoding, plus per-method metrics |
|
| 23 |
+
|
| 24 |
+
Each checkpoint folder holds `model.safetensors`, `config.json` (model hyper-parameters) and `tokenizer.json`
|
| 25 |
+
(byte-level BPE). Every run folder has a model card (`README.md`); Part 1 folders also hold `metrics.json` (test
|
| 26 |
+
perplexity, BLEU, chrF) and Part 2 `final/summary.json` holds validation loss / test BLEU / chrF at every 0.1×. The
|
| 27 |
+
models are custom PyTorch modules: load them with the code in the GitLab repository (`src/part1/model.py`,
|
| 28 |
+
`src.utils.load_model_safetensors`).
|