Commit History
-
feat(b4-gpu+b6): GPU train-proof on A10G + docker-gated substrate E2E test
B4-GPU: modal_b4_gpu_smoke.py ran real Qwen2.5-0.5B-Instruct on a Modal A10G in bf16, 30 steps of the real 3-channel composition. PASS: all-finite, SDPO channel fired nonzero (max JSD 0.1855), loss converged 4.7262 -> 0.0050 monotone. This is the genuine "it trains on GPU" proof the CPU smoke could not give. Result + curve in gpu_smoke_result.json; README documents both CPU and GPU proofs. B6: test_docker_substrate_e2e.py — the docker-gated 4-gate substrate-inversion E2E now EXISTS (was only referenced). Runs the gates against a real python:3.11-slim container + real subprocess pytest on a minimal synthetic feature-deletion task, plus a cache-scrub-in-container test. Skips cleanly with no Docker (2 skipped here), runs the moment a Docker host executes the suite. ADR-010 [~] gate updated: mechanics gate is now ready-to-close, not just wired. Cost: ~1-3 USD A10G. No LMA budget touched.c009712 -
feat(wave-b): ADR-013 LMA integration + B4 end-to-end SDPO-fires proof + doc refresh
ADR-013: composer_replication/integrations/altered_minds/ (generic, framework-side) - MMLUFormatReward: structured-answer reward, scores only the final letter (not rationale style), unparseable/multiple-answer penalties, option-order randomization, always-C exploit detectable via logged option distribution. - dual_kl_logger: KL(policy||altered-init) AND KL(policy||unaltered-base) as the washout-vs-amplification instrument (neither optimized). - channel_ladder_configs: A0-A4 isolated-channel ladder (alpha=0.02/beta=0.05), replacing the uninterpretable combined alpha=0.2/beta=0.4 recipe per the cross-family research critique (SDPO can AMPLIFY an altered model). B4: examples/altered_minds_channel_ladder/run.py PROVES the SDPO channel FIRES nonzero (JSD=0.0565, grad flows) through the REAL shipped collator alignment indices — the thing the old smoke could not show (it only proved init). Honest stub-with-differing-tokens proof; real-model path gated behind ALTERED_MINDS_REAL_MODE…21647a4 -
feat(trainer): ADR-008 gate-3 live GRPO+SDPO smoke PASS; ADR-008 accepted
The composer_grpo_sdpo_smoke instantiates a real trl.GRPOTrainer via ComposerReplicationTrainer on Qwen2.5-0.5B with the Dr. GRPO config (loss_type=dr_grpo, scale_rewards=none, num_iterations=1) and alpha_sdpo=1.0, runs 1 step, and confirms loss/sdpo_kl is logged — the SDPO channel is wired into the live Dr. GRPO loop. PASS. Surfaced + fixed a real API drift: TRL 1.5.0 dropped GRPOConfig.max_prompt_length. All 4 ADR-008 acceptance gates green -> status proposed->accepted; gate checkboxes ticked with evidence; index updated.2a34df4 -
feat(trainer): ADR-008 Dr.GRPO config + SDPO strict-alignment guard
Phase-6 execution of ADR-008 (gates 1, 2, 5 green; gate 3 = the live GRPOTrainer smoke runs detached separately). - make_dr_grpo_config(): builds trl.GRPOConfig to the Dr. GRPO recipe (loss_type=dr_grpo => no length-standardization bias; scale_rewards=none => no std-dev advantage normalization; num_iterations=1 => single-epoch), with a drift-guard assertion. TRL 1.5.0 natively supports these knobs and its own help text cites the Dr. GRPO paper for both. - ComposerReplicationTrainer.strict_sdpo_alignment (default True): the student/teacher logit shape-mismatch path now RAISES instead of silently zeroing the SDPO channel (closes the ADR-008 trust-gap, composer_trainer.py lines 158-160). Opt-out to warn-and-skip for production resilience. - prime_rl/composer_loss.py: NotImplementedError(alpha_sdpo>0) message now points at the TRL host as the SDPO home (documents the ADR-006 amendment). - Tests: 7 new (config knobs asserted; strict raises / non-strict skips / aligned fir…bde5c5e -
examples: add sdpo_real_trace_train_smoke — close the forward+backward+step link
The framework proved the SDPO data path (validate_real_trace_alignment: 832/832 mask alignment on real ~/.claude traces) and the loss math (test_gradient_flow: SDPO routes finite grads on TinyLM) in isolation, but never connected them: an actual compose_loss forward + backward + optimizer.step() on a real HF model fed by the real-trace collator. This is the never-implemented composer_replication.examples. sdpo_with_real_traces_production module that Modal stage_4_sdpo_smoke referenced. It is now real and PASSES on Qwen2.5-0.5B-Instruct over 6 real error-bearing sessions: step 0: total=2.36307 lm_ce=2.33588 sdpo_jsd=0.02718 finite=True step 1: total=2.32758 lm_ce=2.30190 sdpo_jsd=0.02568 finite=True SDPO channel fired (>0): True | param moved: True (max|Δ|=6.22e-05) Gates: (1) no crash, (2) finite loss, (3) SDPO channel fires >0 (shape-gate passes, teacher forward contributes real signal — not the silent no-op the empty-placeholder stage_4 would give), (4) a real param moves af…7090729
-
Wave 21b: skip zero-signal SDPO on empty-recovery error turns + real-trace validation
Found by validating the Wave 21 pipeline against REAL Claude Code session logs (738 local sessions, 66 with is_error:true) rather than only the synthetic fixture. Two outcomes: a collator fix and a permanent validation example. ## Collator fix: empty-recovery error turns On real traces, ~67% of error sites have EMPTY recovery content when strip_thinking=True — because the error-RECOVERY turn is frequently pure [THINKING] (the model reasons about the failure, then silently retries a tool), and stripping thinking empties it. The old code set any_errors=True and injected a hint the moment hint_text existed, BEFORE checking recovery content, so an empty-recovery turn produced an all-ignore_index sdpo_loss_mask: a zero-signal SDPO row that wastes a forward pass and dilutes the channel. Fix: only treat a turn as an SDPO error site when BOTH a hint was produced AND the recovery turn has content (`hint_text and turn.get("content")`). Applied symmetrically to `_build_hint_injected_trace` (te…d61036a -
Wave 21: close both Wave 20 debt items — chat-template alignment + structural is_error
Two architectural-debt items flagged "Wave 20 candidate" in the framework doc, both CPU-only, both now fixed and regression-guarded. ## 1. SDPO mask chat-template drift (was ~67% aligned, now ~100%) `ComposerDataCollator._build_segment_mask` built the sdpo_loss_mask (and the aligned-student response_mask) by tokenizing each content segment in isolation and concatenating. That ignored the scaffolding tokens apply_chat_template inserts around every message (<|im_start|>{role}\n ... <|im_end|>\n, BOS), so mask bits drifted left of the real content tokens — the residual ~33% contamination documented in the Wave 19 production audit. Fix: new `_build_chat_aligned_mask` derives the mask from per-message apply_chat_template prefix deltas. For message k it computes the token span len(template(msgs[:k+1])) - len(template(msgs[:k])), then locates the content run inside that span by subsequence match and marks ONLY those positions as loss. Falls back to whole-span marking if the content run can…6806cf7
-
Wave 19: production-grade SDPO via ComposerDataCollator + adapter + collator fixes
Adds the full production data path Wave 18 deferred: ClaudeCodeIngester → adapter → ComposerDataCollator → compose_loss with proper hint injection at detected error sites. Two rounds of 3-reviewer cross-family review caught a critical SDPO mask alignment bug in round 1; round 2 verified the fix. NEW infrastructure: - composer_replication/ingestion/trace_examples.py: claude_states_to_trace_examples() adapter walks ClaudeCodeIngester output, detects [TOOL_RESULT (ERROR)] tagged user turns, marks the recovery assistant turn with tool_error="<kind>" so ComposerDataCollator._is_error_turn picks it up. Default classifier handles file_not_found, permission_denied, command_not_found, syntax_error, connection_error; users can pass custom error_kind_fn. Backward-scan finds errors even when intervening user turns separate the error from recovery. - composer_replication/ingestion/tests/test_trace_examples_adapter.py: 14 tests pinning the adapter contract (error detection, classifica…03bf323
-
Wave 18: 14 backlog items closed + 3-reviewer cross-family review
Polish wave grinding down 14 P0/P1/P2 items from Wave 7 audit + Wave 11/12 self-critique + Wave 16/17 deferrals. Two passes of 3-reviewer cross-family review (Gemini 3.1 Pro / Grok 4.3 / Sonnet 4.6) caught 6 real issues in round 1; round 2 verified all fixed plus 2 new flagged items addressed. Doc honesty (P0): - README.md scoping change at line 29: dropped "any HuggingFace base model" for the more honest "any HF causal LM with a chat template (Qwen, Llama, Mistral, DeepSeek, Phi, Gemma families)". Added qualifier that Spike 006 zeroed the SDPO channel due to shape misalignment, so SDPO is verified via the integration test suite + end-to-end examples, not Spike 006. - spikes/005-integrated-trainer-skeleton/verl_path/README.md NEW — explicit STATUS: design-only marker per Wave 7 audit Objection 3. Documents what to do before relying on the VeRL path (install Ray, run parity test against trl_path). Test honesty (P1): - spikes/006-real-hf-model-smoke/verdict.md numerical c…54efac8 -
Wave 17: close all 5 audit FLAGs + SDPO context alignment + serverless re-exports
Three-track follow-up to Wave 16, with cross-model review (gemini-3.1-pro APPROVED, grok-4.3 surfaced one BLOCKER that verified false-positive empirically, deepseek-v4-pro burned all max_tokens on hidden reasoning producing no visible output): Wave 17a — serverless package re-exports (real code bug) composer_replication/diloco/serverless/__init__.py now re-exports ModalExecutor and HFJobsExecutor alongside LocalProcessExecutor. The package docstring already documented `from composer_replication.diloco.serverless import ModalExecutor` as the public API but the import would have failed because the names weren't in __init__.py's import list or __all__. Both modal.py and hf_jobs.py import-safely (they don't import the optional `modal` / `huggingface_hub` packages at module level — those are deferred to class method calls), so adding the re-exports doesn't introduce a new dependency. Pinned by a regression test (test_public_reexports_include_all_executors in test_s…a84c060 -
Wave 16: install ergonomics + gradient evidence + SDPO end-to-end example
Five-track polish wave with mandatory cross-model adversarial review (deepseek-v4-pro + gemini-3.1-pro-preview, route-fidelity-verified via direct-urllib scatter; reviewers caught 2 real bugs that BOTH made it into the staged diff and would have shipped without the review): Wave 16a — Install ergonomics (extras pinning) pyproject.toml dropped three extras whose lower bounds were unsatisfiable on public PyPI and would have blocked any fresh user: - monarch>=0.4.1: PyPI 'monarch' tops out at 0.1.11 and is unrelated to Meta's actor framework; the real Monarch ships as torchmonarch-nightly with platform constraints. - prime-rl>=0.5: 'prime-rl' is not registered on PyPI at all; Prime Intellect publishes from source only. - data-juicer>=1.0: 'data-juicer' is not registered on PyPI; the closest match (py-data-juicer) has broken transitive deps. All three integrations import lazily, so the framework still loads cleanly without them. docs/TROUBLESHOO…c0a5ab7 -
Wave 15: 4-angle multi-model self-critique caught 2 math BLOCKERs in primary loss kernels; fixed against upstream byte-for-byte + GSM8K example + ergonomics
PHASE 15a: 4 parallel adversarial reviewers, each with different framing to maximize independent-angle coverage: - MATH (Opus 4.7): cloned upstream OPSD/TAID/PRIME-RL/SimPO, did line-by-line diff. Found 2 BLOCKERs all 8+ prior reviewers missed. - TESTS (Opus 4.7): scrutinized 3 high-stakes test files for weak assertions. Found PRIME-RL parity test silently never runs; bit-exact test uses allclose; entropy-OPD test is pure smoke. - DOCS (Opus 4.7): audited 6 major docs + ADRs. Found test-count drift (77/107/124 vs 145), compose_loss kwarg drift, stale "Deferred to Wave 14" claim, PRIME-RL test count 10 vs 16. - USER-JOURNEY (Opus 4.7): walked "RL-finetune Qwen-7B on GSM8K" end-to-end. Scored framework 2.4/5 with Real-Task-Path = 1/5. #1 friction: NO GSM8K example anywhere. All 4 reports saved to /tmp/wave15_*.md. Headline: math reviewer's upstream-clone-and-diff approach caught what paper-level review missed. PHASE 15b: synthesized findings into 5-task fix scatter. PHA…e5add15 -
Wave 12: close V1-V8 brief — GPU smoke, SDPO firing, real-trace e2e
Addresses cross-model review priority items 3, 4, 5, 9 — the ones that materially close the original V1-V8 brief. Wave 12 deliverables: 1. **Spike 002a-mini-gpu-smoke** (NEW directory) — closes "zero GPU evidence": - 50-step Qwen2.5-0.5B-Instruct training run on local RTX 5090 (sm_120, Blackwell), bf16, in 35 s wall-clock total - Loss 0.7354 → 0.00034 (99.95% reduction), all grads finite, peak VRAM 5.31 GB (well under ADR-001's 8 GB target), median 480 ms/step - Captures per-step memory + step-time + finite-grads in results/gpu_loss_curve.csv + results/gpu_verdict.json - ADR-001's local-5090 choice now empirically verified (vs ~3-5 min cold-start cycle estimated for Modal L4) - Resolves cross-model review item #4 2. **Spike 006-strict** (`tests/test_strict.py`) — closes the tautology critique: - test_alternating_batches_loss_decreases: 10 steps alternating between factorial + binary_search variants. Late-avg-loss < 50% × early-avg-loss. …d88715c -
Wave 10 — packaging: composer_replication is now pip-installable
Promotes the framework from "skeleton scattered across spike directories" to a proper installable Python package (closes vision-validation gap V4). What's new: pyproject.toml at repo root - Hatchling build backend - Python ≥3.10 - Core deps: torch≥2.0, transformers≥4.46 - Optional extras: replay (httpx), diloco (torchft-nightly), train (trl, peft, accelerate, datasets), dev (everything + pytest, ruff) - Project URLs point to the HF repo + Discussions composer_replication/ package (8 modules) - __init__.py re-exports the framework's public API: compose_loss, LossComponents, build_batch (Spike 006) generalized_jsd_loss (verified port of OPSD) ClaudeCodeIngester, IngestionStats, SYSTEM_PROMPT (Spike 007) TraceState, DPOPair, TeacherSpec, replay_trace, extract_dpo_pairs, DEFAULT_TEACHERS (Spike 001/005) ComposerReplicationTrainer (Spike 005, TRL subclass) make_diloco_outer_loop …ac05fbf