Reinforcement Learning
Transformers
English
post-training
distillation
agentic-coding
composer-2.5
cursor
kimi-k2
grpo
dapo
diloco
openenv
trl
verl
research
methodology
Instructions to use Codeseys/composer-replication-framework with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use Codeseys/composer-replication-framework with Transformers:
# pip install -U transformers accelerate # Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("Codeseys/composer-replication-framework", device_map="auto") - Notebooks
- Google Colab
- Kaggle
Commit History
-
docs(wave1): correctness pass — channel-3 provenance, gap honesty, dead links
Propagate ADR-014 ground truth into the living docs and fix every dead relative link. Docs-only; no source files touched. Provenance (Fact A) — stop blurring the additive trace-replay-DPO channel into Cursor's recipe: - README v0.1 roadmap cell, HF_REPO_LAYOUT v0/v1 variant rows: reframe "Composer recipe" = channels 1 (Dr.GRPO) + 2 (SDPO) only; trace-replay is the framework's own addition (link ADR-014). Gap honesty (Fact D) — do not imply the full A1-A4 LMA ladder is runnable: - VISION_VALIDATION status banner + ALTERED_MINDS_TIE_IN Phase-3: only A1 (GRPO-only) has a real Modal runner; A2/A3/A4 are scaffold + plan-builder only, blocked on dataset construction. Real 8B run is additionally budget-gated. PO menu / strip_thinking / main-lag (Facts B, E, F): - VISION_VALIDATION: base RL objective is a selectable menu (default Dr.GRPO) per ADR-014, not hardcoded. - ALTERED_MINDS_TIE_IN: note SDPO requires strip_thinking=False on real traces (~67% of error-recovery turns are…20e3bd9
-
feat(wave-b): ADR-013 LMA integration + B4 end-to-end SDPO-fires proof + doc refresh
ADR-013: composer_replication/integrations/altered_minds/ (generic, framework-side) - MMLUFormatReward: structured-answer reward, scores only the final letter (not rationale style), unparseable/multiple-answer penalties, option-order randomization, always-C exploit detectable via logged option distribution. - dual_kl_logger: KL(policy||altered-init) AND KL(policy||unaltered-base) as the washout-vs-amplification instrument (neither optimized). - channel_ladder_configs: A0-A4 isolated-channel ladder (alpha=0.02/beta=0.05), replacing the uninterpretable combined alpha=0.2/beta=0.4 recipe per the cross-family research critique (SDPO can AMPLIFY an altered model). B4: examples/altered_minds_channel_ladder/run.py PROVES the SDPO channel FIRES nonzero (JSD=0.0565, grad flows) through the REAL shipped collator alignment indices — the thing the old smoke could not show (it only proved init). Honest stub-with-differing-tokens proof; real-model path gated behind ALTERED_MINDS_REAL_MODE…21647a4
-
Wave 7: Phase 2-4 of deep work loop — backlog, parallel research, three ADRs
Imports the gap-closer wave from VISION_VALIDATION.md as a structured backlog, records the deep work loop log, and locks three architecture decisions backed by primary-source research from three subagent recons. Backlog items (CPU-only, no GPU budget): - Spike 006 — real HF model smoke (Wave 7 next) - Spike 007 — real trace ingestion (Wave 8) - Spike 008 — Streaming DiLoCo smoke (Wave 9) - Wave 10 — packaging (pyproject.toml + examples/) - Spike 002a-mini — Modal-gated GPU smoke (Phase 10) Research (docs/research/, 1168 lines total): - MODAL_RECONNAISSANCE.md — Modal pricing + setup, primary-sourced from modal.com/pricing and modal.com/docs. Verdict: Modal L4 is $0.08-0.13 per smoke run but loses to local 5090 by 10x on iteration cycle (3-5min vs 25-40s). Modal becomes correct for parallel sweeps, 7B+ models, multi-node training, or CI repro — none of which apply to gap-closer wave. - DILOCO_RECONNAISSANCE.md — audited 5 candidates. meta-pytorch/torchft wins: BSD-3, 312 com…ac4bfb4