docs: define qwen35 pruning design and plan

docs/superpowers/plans/2026-08-17-qwen35-pruning.md

@@ -0,0 +1,126 @@
+# Qwen3.5 Pruning and Vulkan Distillation Implementation Plan
+
+> For agentic workers: use superpowers:executing-plans or superpowers:subagent-driven-development to execute this plan task by task. Steps use checkbox syntax for tracking.
+
+Goal: Create and verify a 330M-350M text-only Qwen3.5 checkpoint and run GPU-only RX 460 Vulkan teacher/student inference and CE-plus-hidden-MSE distillation.
+
+Architecture: Python utilities perform metadata-only measurement, text-prefix state-dict extraction, and final validation. A pinned llama.cpp Vulkan build performs all model forwards and training; its optimizer graph is patched for frozen teacher hidden targets, combined loss, and cosine warmup.
+
+Tech Stack: Python 3, safetensors, transformers 5.2.0 config schema, pytest, llama.cpp 10068-compatible source, ggml Vulkan backend, AdamW.
+
+## Global Constraints
+
+- Never run a model forward, inference, or training step on CPU.
+- Use the live config.json; retain linear_num_value_heads=16.
+- Keep the first complete hybrid prefix and reject incomplete prefixes.
+- Drop every model.visual.* and mtp.* tensor.
+- Preserve tie_word_embeddings=true and do not duplicate the tied LM head tensor.
+- Final emitted parameter count must be measured inside [330000000, 350000000].
+- Teacher is frozen; loss is CE plus hidden-state MSE.
+- Optimizer is AdamW with cosine warmup.
+- Use explicit-path git staging; never use git add ., git add -A, or git commit -a.
+
+---
+
+### Task 1: Add metadata and checkpoint contract tests
+
+Files:
+- Create: MVP/tests/test_pruning_contract.py
+- Create: MVP/tests/fixtures/qwen35_metadata.json
+
+Interfaces:
+- Tests import MVP.qwen35_prune and MVP.validate_checkpoint.
+- The fixture records live config values and measured tensor groups.
+
+- [ ] Step 1: Write failing tests for choose_prefix(337299424) returning 4,
+  count_parameter_groups returning the measured groups, prefix translation of
+  model.language_model.layers.3.* to model.layers.3.*, vision/MTP exclusion,
+  tied embedding counting, and N=4 config construction.
+- [ ] Step 2: Run python -m pytest MVP/tests/test_pruning_contract.py -q.
+  Expected before implementation: import errors for the missing modules.
+- [ ] Step 3: Stage only the two test paths and commit
+  test: define qwen35 pruning contract.
+
+### Task 2: Implement metadata selection and text-only checkpoint extraction
+
+Files:
+- Create: MVP/qwen35_prune.py
+- Create: MVP/validate_checkpoint.py
+
+Interfaces:
+- load_live_config(path) -> dict
+- count_parameter_groups(safetensors_path, config) -> ParameterReport
+- choose_prefix(report, minimum, maximum) -> PrefixChoice
+- translate_text_key(key) -> str or None
+- build_text_config(full_config, layer_count) -> dict
+- prune_checkpoint(input_weights, input_config, output_dir, layer_count) -> ValidationReport
+- validate_checkpoint(config_path, weights_path, minimum, maximum) -> ValidationReport
+
+- [ ] Step 1: Implement only the functions needed by the red tests.
+- [ ] Step 2: Run the focused tests and keep the output green.
+- [ ] Step 3: Add CLI commands measure, prune, and validate; each command
+  fails closed on missing or inconsistent input.
+- [ ] Step 4: Commit explicit paths as feat: add qwen35 text checkpoint pruner.
+
+### Task 3: Produce and re-measure the real N=4 checkpoint
+
+Files:
+- Create: MVP/artifacts/ outputs from the live snapshot.
+- Create or modify: MVP/PROGRESS.md.
+
+- [ ] Step 1: Run metadata-only measurement against the live snapshot.
+- [ ] Step 2: Run the pruner with --layers 4 and the live safetensors file.
+- [ ] Step 3: Run the validator against the emitted files and save raw output.
+- [ ] Step 4: Confirm output config has model_type=qwen3_5_text, four layer
+  types, and no vision/MTP keys.
+- [ ] Step 5: Commit tools and evidence documentation with explicit paths.
+
+### Task 4: Build a reproducible Vulkan runtime
+
+Files:
+- Create: MVP/runtime/README.md
+- Create: MVP/runtime/build-vulkan.sh
+- Create: MVP/patches/ for the source patch.
+
+- [ ] Step 1: Clone or unpack a pinned llama.cpp source commit and record it.
+- [ ] Step 2: Build with Vulkan enabled and verify llama-cli --list-devices
+  contains RX 460 as Vulkan0.
+- [ ] Step 3: Convert the text-only checkpoint to GGUF using a verified tokenizer.
+- [ ] Step 4: Run student and authoritative teacher inference with
+  --device Vulkan0; save complete logs including device initialization.
+- [ ] Step 5: Commit runtime scripts and evidence paths.
+
+### Task 5: Add and test combined distillation loss
+
+Files:
+- Modify: pinned llama.cpp optimizer/runtime sources in the reproducible runtime checkout.
+- Create: MVP/patches/qwen35-hidden-distillation.patch
+- Create: MVP/tests/test_distillation_loss.py
+
+- [ ] Step 1: Write a synthetic test for CE + lambda*MSE and the cosine-warmup
+  schedule; verify frozen teacher parameters are excluded from updates.
+- [ ] Step 2: Run it red before changing runtime code.
+- [ ] Step 3: Patch the optimizer graph to accept teacher hidden targets and
+  expose CE, hidden MSE, total loss, and learning rate in logs.
+- [ ] Step 4: Build the patched Vulkan runtime and run the synthetic test green.
+- [ ] Step 5: Run a bounded real distillation job with teacher and student on
+  Vulkan0; preserve raw logs.
+- [ ] Step 6: Commit patch and test as
+  feat: add qwen35 hidden-state distillation loss.
+
+### Task 6: Final verification and atomic handoff
+
+Files:
+- Modify: MVP/PROGRESS.md
+- Create: MVP/evidence/ logs and validator output.
+
+- [ ] Step 1: Re-run the final validator on the post-distillation checkpoint.
+- [ ] Step 2: Verify measured count is inside [330000000,350000000] and
+  visual/MTP tensor count is zero.
+- [ ] Step 3: Verify every model execution log names
+  Vulkan0: AMD Radeon RX 460 Graphics.
+- [ ] Step 4: Record any missing proof as kanıt yok; do not infer success from
+  file presence or command exit status alone.
+- [ ] Step 5: Run final fresh verification, inspect git diff --cached --name-only,
+  and commit only explicit paths.
+

docs/superpowers/specs/2026-08-17-qwen35-pruning-design.md

@@ -0,0 +1,82 @@
+# Qwen3.5 Text-Only Pruning and Vulkan Distillation Design
+
+## Goal
+
+Produce a verifiable text-only Qwen/Qwen3.5-0.8B-Base checkpoint by removing the
+vision tower and MTP head, keeping the first complete hybrid block prefix, and
+then run teacher/student inference and distillation through the RX 460 Vulkan
+backend. The final text checkpoint must measure between 330,000,000 and
+350,000,000 parameters from its emitted tensors.
+
+## Constraints
+
+- Model forward, inference, and training are GPU-only through Vulkan0.
+- CPU is allowed for configuration, safetensors metadata, tensor copying, and
+  static validation; it is not allowed for a model forward or training step.
+- The live config is authoritative. In particular,
+  linear_num_value_heads=16 is not replaced by a library default.
+- The student keeps the first complete four-layer hybrid prefix. No activation
+  ranking is used for this depth-only target because N=4 is inside the measured
+  parameter interval.
+- Teacher weights are frozen. The training objective is
+  CE(student logits) + lambda * MSE(student final hidden, teacher final hidden).
+- Optimizer is AdamW and the learning-rate schedule is cosine with warmup.
+- Every phase has explicit-path staging and an atomic commit.
+- A claim of GPU execution requires a saved log containing the Vulkan device.
+
+## Architecture
+
+MVP/qwen35_prune.py reads the live JSON config and safetensors metadata,
+selects the largest complete hybrid prefix that fits the required interval,
+renames only the text-backbone prefixes, drops model.visual.* and mtp.*,
+and emits a text-only safetensors checkpoint plus a qwen3_5_text config. It
+never constructs or forwards the model.
+
+MVP/validate_checkpoint.py re-counts emitted tensors, checks that vision/MTP
+tensors are absent, checks layer-prefix continuity and the exact text config,
+and verifies tied embeddings without duplicating lm_head.weight in the saved
+state dict.
+
+The runtime phase uses a pinned llama.cpp source checkout with Vulkan enabled.
+The required patch adds a frozen teacher hidden-state input and a second loss
+term to the optimizer graph, while retaining the existing AdamW update path and
+adding cosine warmup scheduling. The patch is kept under MVP/patches/ so the
+runtime source and build can be reproduced.
+
+## Data flow
+
+live HF config + safetensors
+        |
+        v
+metadata measurement -> complete-prefix selection (N=4)
+        |
+        v
+text-only safetensors + qwen3_5_text config
+        |
+        v
+HF -> GGUF conversion -> llama-cli Vulkan0 teacher/student inference
+        |
+        v
+teacher hidden target (GPU) + student logits/hidden (GPU)
+        |
+        v
+CE + lambda*MSE -> AdamW + cosine warmup -> student checkpoint
+
+## Failure handling
+
+- Missing or inconsistent metadata is a hard error.
+- A layer count that is not a complete hybrid prefix is rejected.
+- Any retained visual or mtp tensor is a hard validation failure.
+- A non-Vulkan runtime log is not accepted as model execution evidence.
+- If the installed llama.cpp runtime cannot support the combined loss or the
+  RX 460 cannot hold the requested graph, the progress file records the exact
+  failing command and output; CE-only output is never reported as distillation.
+
+## Testing
+
+Pure Python tests cover metadata-derived counts, N=4 selection, prefix
+translation, tensor exclusion, tied embeddings, config fields, and final
+interval validation. A runtime test uses a synthetic tiny graph to verify the
+combined loss numerically before a real model run. Real model checks re-count
+the produced checkpoint and require Vulkan logs.
+