Upload HCL open-source release

.gitattributes

@@ -33,3 +33,6 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+docs/assets/logo.png filter=lfs diff=lfs merge=lfs -text
+docs/assets/method-overview.png filter=lfs diff=lfs merge=lfs -text
+docs/assets/teaser.png filter=lfs diff=lfs merge=lfs -text

.gitignore

@@ -0,0 +1,26 @@
+__pycache__/
+.pytest_cache/
+*.py[cod]
+*.egg-info/
+.venv/
+venv/
+.env
+.env.*
+build/
+dist/
+
+# Local credentials and machine-specific configuration
+**/conf/config.json
+
+# Runtime artifacts
+storage/
+outputs/
+reports/
+logs/
+*.log
+llm_calls.txt
+llm_calls.jsonl
+checkpoint*.json
+prediction_cache/
+node_modules/
+.DS_Store

LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 HCL Contributors
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

README.md

@@ -0,0 +1,305 @@
+<h1 align="center">
+  <img src="docs/assets/logo.png" alt="HCL logo" height="48" align="absmiddle">
+  Harness Continual Learning
+</h1>
+
+<p align="center"><strong>Continual Adaptation Beyond Model Parameters</strong></p>
+
+Official implementation of **Harness Continual Learning (HCL)**, a framework that enables frozen foundation-model agents to continually adapt their memory, routing, task interfaces, reusable capabilities, and optimization policies.
+
+<p align="center">
+  <img src="docs/assets/teaser.png" alt="Overview of Harness Continual Learning" width="900">
+</p>
+
+## Abstract
+
+Continual learning has largely focused on model parameters, yet modern agents can also adapt through a harness of prompts, memories, tools, skills, and routing rules. We introduce **Harness Continual Learning (HCL)**, a paradigm in which this harness evolves around a frozen foundation model, and define the resulting loss of previously acquired behavior as **harness-level forgetting**. HCL comprises four execution-facing components: the Task Interface, Experience Memory, Capability Map, and Adaptive Router. To balance adaptation and retention, **guarded harness evolution** separates update generation from state commitment: a Continual Optimizer proposes candidate harnesses from execution feedback, while a Continual Evaluator commits an update only after checking current improvement, historical retention, and validity. Across open-world capability accumulation, textual reasoning, and multimodal perception, HCL supports capability growth and failure recovery, exceeds corresponding baselines by more than 10% in multiple settings, and provides a controllable stability–plasticity trade-off.
+
+## Key contributions
+
+We evaluate HCL across textual reasoning, multimodal perception, and open-world interaction, and further study retention control, harness-level distillation, and component contributions. Our main contributions are:
+
+- **Harness Continual Learning:** We propose and formalize HCL as a new continual learning paradigm, shifting the object of continual learning from model state to the harness state surrounding a frozen foundation model.
+- **Guarded harness evolution:** We identify harness-level forgetting and make current improvement, historical retention, and validity explicit conditions for committing a harness update.
+- **Capability accumulation and controlled retention:** Across reasoning, multimodal, and interactive task streams, we show that harness evolution supports capability accumulation. Component ablations and retention sweeps reveal the contributions of individual harness components, measurable forgetting, and controllable stability–plasticity trade-offs, with exploratory evidence of knowledge transfer.
+
+<p align="center">
+  <img src="docs/assets/method-overview.png" alt="HCL method overview" width="900">
+</p>
+
+## Repository layout
+
+```text
+HCL/
+├── hcl/                         # Core HCL implementation
+│   ├── capability/              # Reusable skills, tools, and capability services
+│   ├── memory/                  # Raw, abstract, and anchor memories
+│   ├── optimizer/               # Candidate harness-update generation
+│   ├── router/                  # Workflow and artifact selection prompts
+│   └── task_interface/          # Shared task structuring interface
+├── configs/                     # Paper experiment and task-stream profiles
+├── data/                        # Expected external dataset layout
+├── examples/                    # Minimal synthetic task-stream fixtures
+├── tests/                       # Offline unit and regression tests
+├── scripts/                     # Utility and integration scripts
+├── experiments/
+│   ├── HCL_ALFworld/            # ALFWorld integration
+│   └── Odyssey/                 # Minecraft/Odyssey integration
+├── docs/assets/                 # README figures
+├── main.py                      # Main experiment entry point
+├── run.sh                       # Reference launch script
+├── requirements.txt
+└── pyproject.toml
+```
+
+The core implementation is exposed at the repository root, following the same entry-point-oriented layout as DFA-MoE. ALFWorld and Odyssey retain isolated environments because their simulator dependencies are incompatible with parts of the core stack.
+
+## Installation
+
+### Core environment
+
+Python 3.10 or later is required.
+
+```bash
+git clone https://github.com/RL-MIND/Harness-Continual-Learning.git
+cd HCL
+python3 -m venv .venv
+source .venv/bin/activate
+python -m pip install --upgrade pip
+python -m pip install -r requirements.txt
+```
+
+Install optional dependencies only when needed:
+
+```bash
+# Local Transformers models
+python -m pip install -e '.[local-model]'
+
+# COCO metrics and development tests
+python -m pip install -e '.[coco-eval,test]'
+```
+
+Verify the installation without model or dataset downloads:
+
+```bash
+python -m unittest discover -s tests -v
+```
+
+## Supported task streams
+
+| Modality | Task stream | Tasks |
+| --- | --- | --- |
+| Textual reasoning | `configs/taskstream_textual_main_250_50_500.json` | MuSiQue, ProofWriter, GSM8K, HotpotQA |
+| Multimodal | `configs/taskstream_multimodal_main_250_50_500.json` | COCO detection, COCO captioning, RefCOCO grounding, VQAv2 |
+| Embodied | `experiments/HCL_ALFworld/` | ALFWorld domain-incremental sequences |
+| Open world | `experiments/Odyssey/` | Minecraft/Odyssey curriculum tasks |
+
+Datasets and model checkpoints are not redistributed. Prepare them under `data/` according to [the data layout](data/README.md), or update the project-relative paths in the selected task-stream file.
+
+## Quick start
+
+### Run a textual HCL experiment
+
+The following profile runs Stability-HCL with a Qwen3-4B main model:
+
+```bash
+python main.py \
+  --config configs/run_stability_hcl_01_4b.json \
+  --splits train,test
+```
+
+The equivalent installed command is:
+
+```bash
+hcl run \
+  --config configs/run_stability_hcl_01_4b.json \
+  --splits train,test
+```
+
+Use `--limit-per-task` for a small integration run, and use `--method` to override the method declared by a profile:
+
+```bash
+python main.py \
+  --config configs/deepseek_flash_reasoning_hcl_stability_250_50_500.json \
+  --splits train,test \
+  --limit-per-task 5 \
+  --method hcl
+```
+
+`run.sh` is a reference wrapper around the same entry point. Its first argument may be a configuration path; remaining arguments are forwarded to `main.py`.
+
+### Run the baselines
+
+Use the checked-in baseline profiles directly:
+
+```bash
+# Zero-shot
+python main.py --config configs/deepseek_flash_reasoning_raw_zero_shot_500.json --splits test
+```
+
+See [the configuration inventory](configs/README.md) for the complete reasoning, multimodal, budget-sweep, and distillation profile list.
+
+## Important configuration options
+
+HCL experiments use JSON configurations. The main options are:
+
+```json
+{
+  "method": "hcl",
+  "task_stream_path": "configs/taskstream_textual_main_250_50_500.json",
+  "storage_dir": "storage/experiments/example",
+  "model": {
+    "backend": "chat_completions",
+    "name": "model-name",
+    "base_url": "http://127.0.0.1:8000",
+    "api_key_env": "MODEL_API_KEY"
+  },
+  "optimizer": {
+    "historical_loss_budget": 0
+  }
+}
+```
+
+- `model` generates final answers. `judge_model`, `selection_model`, and `memory_model` can assign optimizer, online-selection, and memory operations to different models.
+- `method` selects the HCL update mode. Supported CLI overrides are `hcl` and `hcl_memory`.
+- `optimizer.historical_loss_budget` controls accepted historical degradation; `null` represents $b=\infty$.
+- `storage_dir` contains generated state, metrics, predictions, and checkpoints and must remain under `storage/` when automatic reset is enabled.
+- `task_flow` controls the named train/test phases selected with `--splits`.
+
+For hosted OpenAI-compatible endpoints, export the credential named by `api_key_env`; do not place secrets in configuration files:
+
+```bash
+export MODEL_API_KEY="your-key"
+```
+
+## Experiments
+
+### ALFWorld
+
+```bash
+cd experiments/HCL_ALFworld
+python3 -m venv .venv
+source .venv/bin/activate
+python -m pip install -e '.[llm]'
+hcl-alfworld --help
+```
+
+Use the `local` extra for a local Transformers backend. Profiles are in `configs/`, and domain-incremental sequences are in `sequences/`.
+
+### Odyssey / Minecraft
+
+```bash
+cd experiments/Odyssey/Odyssey
+python3 -m venv .venv
+source .venv/bin/activate
+python -m pip install -e .
+cp conf/config.example.json conf/config.json
+python harness_cli.py --help
+python harness_curriculum_cli.py --help
+```
+
+Odyssey additionally requires its Node/Mineflayer bridge, a reachable Minecraft server, and an OpenAI-compatible model endpoint. Local values can be supplied with `ODYSSEY_*` environment variables; `conf/config.json` is ignored by version control.
+
+## Main results
+
+The results below follow the order of the experiments in the main paper. Additional analyses reported only in the appendix are not included here.
+
+### Open-world capability accumulation
+
+#### ALFWorld
+
+We evaluate six sequential ALFWorld task categories with a frozen Qwen3.5-9B model. Plasticity-HCL achieves the highest final average success rate, while Stability-HCL retains comparable performance with substantially less forgetting.
+
+| Method | Pick | Look | Clean | Heat | Cool | Two-object | Final Avg. $\uparrow$ | Avg. Fgt. (%) $\downarrow$ |
+| --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
+| Static Harness | 95.80 | 66.70 | 25.80 | 26.10 | 9.50 | 58.80 | 47.12 | — |
+| RAG Baseline | 95.80 | 83.30 | 41.90 | 39.10 | 14.30 | 58.80 | 55.56 | **1.74** |
+| **Stability-HCL (Ours)** | **100.00** | 83.30 | **51.60** | 30.40 | 28.60 | 76.50 | 61.74 | 2.64 |
+| **Plasticity-HCL (Ours)** | **100.00** | 77.80 | 41.90 | 39.10 | 19.00 | **100.00** | **62.98** | 10.94 |
+
+#### Minecraft
+
+On a 50-task Minecraft curriculum, Plasticity-HCL completes the full sequence in 83 environment actions.
+
+### Controlled-stream continual learning
+
+We next evaluate HCL on textual reasoning and multimodal perception streams. Each task contains 250 adaptation examples, 50 validation examples, and 500 disjoint test examples.
+
+#### Textual reasoning
+
+The stream order is MuSiQue $\rightarrow$ ProofWriter $\rightarrow$ GSM8K $\rightarrow$ HotpotQA, using frozen DeepSeek-V4.1-Flash. Plasticity-HCL obtains the best final average performance of 76.10%.
+
+| Method | MuSiQue | ProofWriter | GSM8K | HotpotQA | Final Avg. $\uparrow$ | Avg. Fgt. (%) $\downarrow$ |
+| --- | ---: | ---: | ---: | ---: | ---: | ---: |
+| Zero-shot | 48.00 | 79.20 | 65.20 | 62.20 | 63.65 | — |
+| **Plasticity-HCL (Ours)** | **58.60** | **85.60** | **96.00** | **64.20** | **76.10** | 0.40 |
+| **Stability-HCL (Ours)** | 53.60 | 82.80 | 82.60 | 63.60 | 70.65 | 0.27 |
+
+#### Multimodal perception
+
+The stream order is COCO detection $\rightarrow$ COCO captioning $\rightarrow$ RefCOCO grounding $\rightarrow$ VQAv2, using frozen Qwen3.6-27B. Stability-HCL achieves the best final mean of 68.92 with only 0.22% average forgetting.
+
+| Method | Detection | Caption | Grounding | VQAv2 | Final Mean $\uparrow$ | Avg. Fgt. (%) $\downarrow$ |
+| --- | ---: | ---: | ---: | ---: | ---: | ---: |
+| Zero-shot | 37.52 | 21.10 | 43.00 | **85.47** | 46.77 | — |
+| **Plasticity-HCL (Ours)** | 64.14 | 37.31 | 90.60 | 79.80 | 67.96 | 0.81 |
+| **Stability-HCL (Ours)** | **65.34** | **39.41** | **91.60** | 79.33 | **68.92** | 0.22 |
+
+### Stability–plasticity control
+
+We vary the historical-loss budget over $b\in\{0,1,3,\infty\}$ while holding the current-improvement and validity criteria fixed. The strict setting $b=0$ minimizes forgetting, whereas a small tolerance of $b=1$ produces the best final average accuracy. Further relaxation degrades both final accuracy and retention.
+
+| Historical-loss budget $b$ | MuSiQue | ProofWriter | GSM8K | HotpotQA | Final Avg. $\uparrow$ | Avg. Fgt. (%) $\downarrow$ |
+| ---: | ---: | ---: | ---: | ---: | ---: | ---: |
+| $0$ | 27.83 | 73.33 | 84.33 | **59.50** | 61.25 | **0.39** |
+| $1$ | 24.83 | 77.50 | **92.33** | 59.17 | **63.46** | 1.22 |
+| $3$ | 26.83 | **79.83** | 83.00 | 58.50 | 62.04 | 2.00 |
+| $\infty$ | **28.33** | 71.00 | 82.00 | 59.17 | 60.13 | 3.45 |
+
+### External-model assistance
+
+Base-HCL uses Qwen3-4B throughout. Opt-HCL uses Qwen3.6-27B only to generate candidate harness updates, whereas Online-HCL uses the stronger model for per-example online harness operations. Opt-HCL improves final average accuracy from 35.10% to 45.75% while maintaining latency comparable to Base-HCL; Online-HCL reaches 44.80% but incurs substantially higher online latency.
+
+<p align="center">
+  <img src="docs/assets/distillation-results.png" alt="Accuracy and latency under different placements of stronger-model assistance" width="900">
+</p>
+
+### Component ablation
+
+On the controlled multimodal stream with frozen Qwen3.5-4B, each ablation keeps one component available during execution but freezes its persistent updates. All four components contribute to continual adaptation, with freezing Experience Memory causing the largest reduction in final average performance and the most forgetting.
+
+| Variant | Interface | Memory | Capability | Router | Final Avg. $\uparrow$ | Avg. Fgt. (%) $\downarrow$ |
+| --- | :---: | :---: | :---: | :---: | ---: | ---: |
+| Zero-shot | — | — | — | — | 34.84 | — |
+| w/o Interface update | ✗ | ✓ | ✓ | ✓ | 62.37 | 0.11 |
+| w/o Memory update | ✓ | ✗ | ✓ | ✓ | 62.28 | 0.83 |
+| w/o Capability update | ✓ | ✓ | ✗ | ✓ | 63.12 | **0.06** |
+| w/o Router update | ✓ | ✓ | ✓ | ✗ | 62.77 | 0.14 |
+| **Plasticity-HCL** | ✓ | ✓ | ✓ | ✓ | **63.41** | 0.45 |
+
+## Notes
+
+- `main.py`, the installed `hcl` command, and `run.sh` are the recommended core entry points.
+- Generated logs, predictions, checkpoints, caches, simulator state, and credentials are intentionally excluded.
+- Some configuration filenames retain original GPU scheduling labels; the configurations themselves use portable paths and do not require that device index.
+- The ALFWorld and Odyssey environments should be installed separately from the core environment.
+
+## Citation
+
+If you find this repository useful, please cite the HCL paper:
+
+```bibtex
+@misc{kang2026harnesscontinuallearning,
+  title         = {Harness Continual Learning: Continual Adaptation Beyond Model Parameters},
+  author        = {Borui Kang and Jinrui Gu and Junhan Lv and Wenbin Li and Lei Wang and Yang Gao},
+  year          = {2026},
+  eprint        = {2608.19013},
+  archivePrefix = {arXiv},
+  primaryClass  = {cs.LG},
+  url           = {https://arxiv.org/abs/2608.19013}
+}
+```
+
+## License and acknowledgements
+
+The core HCL implementation is released under the [MIT License](LICENSE). The integrations under `experiments/` build on ALFWorld, Odyssey, Mineflayer, and related third-party components; consult their respective licenses before redistribution.

configs/README.md

@@ -0,0 +1,62 @@
+# Experiment configuration inventory
+
+This directory contains the textual-reasoning and multimodal configurations retained for reproducibility. Interactive ALFWorld and Minecraft configurations live under the repository's `experiments/` directory. No metrics, predictions, logs, checkpoints, figures, or other prior-run outputs are included.
+
+## Path convention
+
+- `configs/taskstream_textual_main_250_50_500.json`: included portable manifest for MuSiQue, ProofWriter, GSM8K, and HotpotQA.
+- `configs/taskstream_multimodal_main_250_50_500.json`: included portable manifest for COCO detection, COCO captioning, RefCOCO grounding, and VQAv2.
+- `data/datasets/`: externally supplied dataset splits.
+- `storage/experiments/<configuration-name>/`: generated logs, caches, checkpoints, manifests, and predictions. `storage/` is ignored by Git.
+
+Relative paths are resolved from the repository root by the CLI configuration loader.
+
+## HCL reasoning experiments
+
+- `deepseek_flash_reasoning_hcl_plasticity_250_50_500.json`
+- `deepseek_flash_reasoning_hcl_stability_250_50_500.json`
+- `qwen36_27b_reasoning_hcl_stability_capabilities_250_50_500.json`
+
+## Independent textual historical-loss-budget sweep
+
+- `deepseek_v4_flash_textual_budget_b0_300_80_80_600.json`
+- `deepseek_v4_flash_textual_budget_b1_300_80_80_600.json`
+- `deepseek_v4_flash_textual_budget_b3_300_80_80_600.json`
+- `deepseek_v4_flash_textual_budget_binf_300_80_80_600.json`
+- `taskstream_textual_budget_300_80_80_600.json`
+
+These four runnable profiles use frozen DeepSeek-V4-Flash, the paper's 300 adaptation / 80 validation / 80 historical-anchor / 600 test allocation, and ten 30-example update batches per task stage. The profiles are identical in experimental settings except for `optimizer.historical_loss_budget`, which is 0, 1, 3, or `null` ($B_n=\infty$). The optimizer generates three alternatives inside each update opportunity, matching the recovered experiment configuration; that internal multiplicity is distinct from the paper's count of ten proposal opportunities per task stage.
+
+## Distillation comparison experiments
+
+- `run_stability_hcl_01_4b.json` (`Base-HCL`): Qwen3-4B performs final answering, online structuring and selection, optimizer candidate generation, and memory operations.
+- `run_stability_hcl_02_4b_opt27b.json` (`Opt-HCL`): Qwen3.6-27B generates optimizer candidates; Qwen3-4B retains all online and memory roles.
+- `run_stability_hcl_03_4b_teacher27b.json` (`Online-HCL`): Qwen3.6-27B performs online task structuring, workflow and artifact selection, and memory operations; Qwen3-4B retains final answering and optimizer candidate generation.
+- `taskstream_textual_main_250_50_500.json`: the portable MuSiQue, ProofWriter, GSM8K, and HotpotQA task stream shared by all three runs.
+
+The three profiles share the same Stability-HCL gate, three optimizable components, candidate budget, task stream, data allocation, memory limits, and update schedule. With a batch size of 300 and 250 adaptation examples per task, each task stage has one optimizer update opportunity. The `selection_model` and `memory_model` fields make the Online-HCL role split explicit; when absent, both roles fall back to the main model.
+
+The local checkpoints are expected at `models/Qwen3-4B/` and `models/Qwen3.6-27B/`. These directories are not included. Paths can be changed to checkpoints supplied by the evaluation environment without changing the experimental method or hyperparameters.
+
+## Reasoning baselines
+
+- `deepseek_flash_reasoning_raw_zero_shot_500.json`
+
+## Multimodal experiments and baselines
+
+- `qwen36_27b_coco_gpu1_direct_raw_zero_shot_500.json`
+- `qwen36_27b_coco_gpu1_v0_harness_500.json`
+- `qwen36_27b_coco_gpu1_hcl_plasticity_250_50_500.json`
+- `qwen36_27b_coco_gpu1_hcl_stability_250_50_500.json`
+
+## Task-stream configurations
+
+- `taskstream_textual_main_250_50_500.json`
+- `taskstream_multimodal_main_250_50_500.json`
+- `taskstream_textual_budget_300_80_80_600.json`
+
+Configuration filenames containing a GPU number preserve the original experiment-to-device scheduling label. They contain no hostname, username, institution, filesystem location, or hardware identifier unique to the authors, and do not require that same device number when reproduced.
+
+The previous component-ablation matrix is intentionally not included as a paper configuration: it used Qwen3.6-27B and disabled runtime components, whereas the paper specifies Qwen3.5-4B and freezes only persistent updates while keeping each component available. A search of the additional experiment server found no Qwen3.5-4B four-way persistent-update-freeze configuration, so this remains unresolved rather than being approximated.
+
+The stronger-model comparison is the Stability-HCL series above. Earlier `run_paper_*` profiles represented a separate four-run study and are intentionally excluded to avoid conflating that study with the paper's Base/Opt/Online comparison.

configs/coco_stability_125_50_250_local.json

@@ -0,0 +1,137 @@
+{
+  "project_name": "Qwen3.6-27B COCO HCL stability 125-50-250 local validation",
+  "model": {
+    "backend": "chat_completions",
+    "name": "qwen3.6-27b-coco-ablation-gpu3",
+    "paper_model_name": "Qwen3.6-27B",
+    "base_url": "http://127.0.0.1:8017",
+    "api_key": "EMPTY",
+    "max_new_tokens": 2048,
+    "answer_max_new_tokens": 1024,
+    "temperature": 0.0,
+    "cleanup_output": true,
+    "enable_image_input": true,
+    "chat_template_kwargs": {
+      "enable_thinking": false
+    },
+    "json_object_for_structured_phases": true,
+    "system_prompt": "You are a concise multimodal reasoning assistant. Inspect the supplied image directly, follow the requested output schema exactly, and output only the final answer.",
+    "timeout_seconds": 600,
+    "max_retries": 20,
+    "log_path": "storage/experiments/coco_stability_125_50_250_local/llm_calls.txt",
+    "log_prompt_chars": 8000
+  },
+  "execution": {
+    "read_only_workers": 2,
+    "run_train_final": true
+  },
+  "task_interface": {
+    "mode": "auto",
+    "max_new_tokens": 640,
+    "invalid_output_retries": 1,
+    "retry_max_new_tokens": 1024,
+    "fallback_to_rules_on_invalid": true
+  },
+  "router": {
+    "route_name": "qwen36_27b_coco_gpu3_hcl_stability",
+    "strip_output": true,
+    "use_llm_workflow_selector": true,
+    "use_memory_selector": true,
+    "memory_always_on": true,
+    "use_skill_selector": true,
+    "use_tool_selector": false,
+    "tool_auto_select": false,
+    "use_chat_tool_calls": false,
+    "memory_selector_max_new_tokens": 256,
+    "memory_selector_invalid_output_retries": 1,
+    "memory_selector_retry_max_new_tokens": 256,
+    "skill_selector_max_new_tokens": 256,
+    "skill_selector_invalid_output_retries": 1,
+    "skill_selector_retry_max_new_tokens": 256,
+    "skill_selector_fallback_to_empty_on_invalid": true,
+    "capability_enabled": false,
+    "capability_execution_mode": "disabled"
+  },
+  "optimizer": {
+    "candidate_count": 3,
+    "selection_objective": "plasticity",
+    "commit_policy": "plasticity_first",
+    "historical_loss_budget": 0,
+    "max_forget_count": 0,
+    "min_primary_score_delta": 1e-09,
+    "min_correct_gain": 0,
+    "min_format_compliance_rate": 1.0,
+    "recover_reward": 1.0,
+    "forget_penalty": 1.5,
+    "min_anchor_score": -5.0,
+    "min_anchor_net_score": -0.1,
+    "max_anchor_forget_rate": 0.2,
+    "components": [
+      "task_interface_structuring",
+      "router_workflow",
+      "memory_selector",
+      "router_context"
+    ]
+  },
+  "memory": {
+    "enabled": true,
+    "raw_capacity_per_task": 10,
+    "abstract_capacity_per_task": 2,
+    "anchor_capacity_per_task": 25,
+    "retrieval_top_k": 2,
+    "reset_on_start": false,
+    "abstract_mode": "llm",
+    "abstract_max_new_tokens": 512,
+    "anchor_selection_mode": "llm",
+    "anchor_count_per_batch": 2,
+    "anchor_candidate_limit": 10,
+    "anchor_selection_max_new_tokens": 128,
+    "skill_enabled": true,
+    "skill_mode": "llm",
+    "abstract_prompt_max_chars": 12000
+  },
+  "forgetting_probe": {
+    "enabled": true,
+    "interval_batches": 2,
+    "samples_per_task": 100,
+    "target_correct_ratio": 0.8,
+    "seed": 20260726,
+    "split": "test",
+    "historical_tasks_only": true,
+    "reuse_probe_set": true,
+    "record_predictions": false
+  },
+  "task_stream_path": "configs/taskstream_multimodal_main_250_50_500.json",
+  "reset_run_on_start": false,
+  "storage_dir": "storage/experiments/coco_stability_125_50_250_local",
+  "prediction_cache": {
+    "enabled": true,
+    "path": "storage/experiments/coco_stability_125_50_250_local/prediction_cache/read_only_predictions.jsonl"
+  },
+  "checkpoint": {
+    "enabled": true,
+    "path": "storage/experiments/coco_stability_125_50_250_local/checkpoint.json",
+    "resume": true
+  },
+  "task_flow": [
+    {
+      "name": "train",
+      "phase": "train",
+      "train_split": "train",
+      "val_split": "validation",
+      "test_split": "test",
+      "continual": true,
+      "batchsize": 25,
+      "train_limit_per_task": 125,
+      "validation_limit_per_task": 50,
+      "test_limit_per_task": 250
+    },
+    {
+      "name": "test",
+      "phase": "test",
+      "test_split": "test",
+      "test_limit_per_task": 250,
+      "group_by_task": true
+    }
+  ]
+}

configs/deepseek_flash_reasoning_hcl_plasticity_250_50_500.json

@@ -0,0 +1,142 @@
+{
+  "project_name": "DeepSeek-V4.1-Flash reasoning HCL plasticity 250-50-500",
+  "model": {
+    "backend": "chat_completions",
+    "name": "deepseek-flash",
+    "paper_model_name": "DeepSeek-V4.1-Flash",
+    "base_url": "https://api.deepseek.com",
+    "api_key_env": "DEEPSEEK_API_KEY",
+    "max_new_tokens": 1024,
+    "answer_max_new_tokens": 512,
+    "temperature": 0.0,
+    "thinking": {
+      "type": "disabled"
+    },
+    "cleanup_output": true,
+    "json_object_for_structured_phases": false,
+    "system_prompt": "You are a concise reasoning assistant. Follow the requested output schema and return only the final answer.",
+    "timeout_seconds": 600,
+    "max_retries": 10,
+    "log_path": "storage/experiments/deepseek_flash_reasoning_hcl_plasticity_250_50_500/llm_calls.txt",
+    "log_prompt_chars": 12000
+  },
+  "execution": {
+    "read_only_workers": 1,
+    "run_train_final": true
+  },
+  "task_interface": {
+    "mode": "llm",
+    "max_new_tokens": 768,
+    "invalid_output_retries": 1,
+    "retry_max_new_tokens": 768,
+    "fallback_to_rules_on_invalid": true,
+    "long_context_rule_threshold_chars": 8192
+  },
+  "router": {
+    "route_name": "deepseek_flash_reasoning_hcl_plasticity",
+    "strip_output": true,
+    "use_llm_workflow_selector": true,
+    "use_memory_selector": true,
+    "memory_always_on": false,
+    "use_skill_selector": true,
+    "use_tool_selector": false,
+    "tool_auto_select": false,
+    "use_chat_tool_calls": false,
+    "tool_max_rounds": 0,
+    "workflow_selector_max_new_tokens": 512,
+    "memory_selector_max_new_tokens": 256,
+    "memory_selector_invalid_output_retries": 1,
+    "memory_selector_retry_max_new_tokens": 256,
+    "skill_selector_max_new_tokens": 256,
+    "skill_selector_invalid_output_retries": 1,
+    "skill_selector_retry_max_new_tokens": 256,
+    "skill_selector_fallback_to_empty_on_invalid": true,
+    "capability_enabled": false,
+    "capability_execution_mode": "disabled",
+    "capability_service_url": null,
+    "semantic_search_enabled": false,
+    "cross_modal_match_enabled": false
+  },
+  "optimizer": {
+    "candidate_count": 3,
+    "selection_objective": "plasticity",
+    "commit_policy": "plasticity_first",
+    "historical_loss_budget": null,
+    "min_primary_score_delta": 1e-09,
+    "min_correct_gain": 0,
+    "max_forget_count": 0,
+    "min_format_compliance_rate": 1.0,
+    "recover_reward": 1.0,
+    "forget_penalty": 1.5,
+    "min_anchor_score": -5.0,
+    "min_anchor_net_score": -0.1,
+    "max_anchor_forget_rate": 0.2,
+    "components": [
+      "task_interface_structuring",
+      "router_workflow",
+      "memory_selector",
+      "router_context"
+    ]
+  },
+  "memory": {
+    "enabled": true,
+    "raw_capacity_per_task": 10,
+    "abstract_capacity_per_task": 2,
+    "anchor_capacity_per_task": 25,
+    "retrieval_top_k": 2,
+    "reset_on_start": false,
+    "abstract_mode": "llm",
+    "abstract_max_new_tokens": 512,
+    "anchor_selection_mode": "llm",
+    "anchor_count_per_batch": 2,
+    "anchor_candidate_limit": 10,
+    "anchor_selection_max_new_tokens": 256,
+    "skill_enabled": true,
+    "skill_mode": "llm",
+    "abstract_prompt_max_chars": 12000
+  },
+  "forgetting_probe": {
+    "enabled": true,
+    "interval_batches": 2,
+    "samples_per_task": 100,
+    "target_correct_ratio": 0.8,
+    "seed": 20260726,
+    "split": "test",
+    "historical_tasks_only": true,
+    "reuse_probe_set": true,
+    "record_predictions": false
+  },
+  "task_stream_path": "configs/taskstream_textual_main_250_50_500.json",
+  "reset_run_on_start": false,
+  "storage_dir": "storage/experiments/deepseek_flash_reasoning_hcl_plasticity_250_50_500",
+  "prediction_cache": {
+    "enabled": true,
+    "path": "storage/experiments/deepseek_flash_reasoning_hcl_plasticity_250_50_500/prediction_cache/prediction_cache.jsonl"
+  },
+  "checkpoint": {
+    "enabled": true,
+    "path": "storage/experiments/deepseek_flash_reasoning_hcl_plasticity_250_50_500/checkpoint.json",
+    "resume": true
+  },
+  "task_flow": [
+    {
+      "name": "train",
+      "phase": "train",
+      "train_split": "train",
+      "val_split": "validation",
+      "test_split": "test",
+      "continual": true,
+      "batchsize": 25,
+      "train_limit_per_task": 250,
+      "validation_limit_per_task": 50,
+      "test_limit_per_task": 500
+    },
+    {
+      "name": "test",
+      "phase": "test",
+      "test_split": "test",
+      "test_limit_per_task": 500,
+      "group_by_task": true
+    }
+  ]
+}

configs/deepseek_flash_reasoning_hcl_stability_250_50_500.json

@@ -0,0 +1,142 @@
+{
+  "project_name": "DeepSeek-V4.1-Flash reasoning HCL stability 250-50-500",
+  "model": {
+    "backend": "chat_completions",
+    "name": "deepseek-flash",
+    "paper_model_name": "DeepSeek-V4.1-Flash",
+    "base_url": "https://api.deepseek.com",
+    "api_key_env": "DEEPSEEK_API_KEY",
+    "max_new_tokens": 1024,
+    "answer_max_new_tokens": 512,
+    "temperature": 0.0,
+    "thinking": {
+      "type": "disabled"
+    },
+    "cleanup_output": true,
+    "json_object_for_structured_phases": false,
+    "system_prompt": "You are a concise reasoning assistant. Follow the requested output schema and return only the final answer.",
+    "timeout_seconds": 600,
+    "max_retries": 10,
+    "log_path": "storage/experiments/deepseek_flash_reasoning_hcl_stability_250_50_500/llm_calls.txt",
+    "log_prompt_chars": 12000
+  },
+  "execution": {
+    "read_only_workers": 1,
+    "run_train_final": true
+  },
+  "task_interface": {
+    "mode": "llm",
+    "max_new_tokens": 768,
+    "invalid_output_retries": 1,
+    "retry_max_new_tokens": 768,
+    "fallback_to_rules_on_invalid": true,
+    "long_context_rule_threshold_chars": 8192
+  },
+  "router": {
+    "route_name": "deepseek_flash_reasoning_hcl_stability",
+    "strip_output": true,
+    "use_llm_workflow_selector": true,
+    "use_memory_selector": true,
+    "memory_always_on": false,
+    "use_skill_selector": true,
+    "use_tool_selector": false,
+    "tool_auto_select": false,
+    "use_chat_tool_calls": false,
+    "tool_max_rounds": 0,
+    "workflow_selector_max_new_tokens": 512,
+    "memory_selector_max_new_tokens": 256,
+    "memory_selector_invalid_output_retries": 1,
+    "memory_selector_retry_max_new_tokens": 256,
+    "skill_selector_max_new_tokens": 256,
+    "skill_selector_invalid_output_retries": 1,
+    "skill_selector_retry_max_new_tokens": 256,
+    "skill_selector_fallback_to_empty_on_invalid": true,
+    "capability_enabled": false,
+    "capability_execution_mode": "disabled",
+    "capability_service_url": null,
+    "semantic_search_enabled": false,
+    "cross_modal_match_enabled": false
+  },
+  "optimizer": {
+    "candidate_count": 3,
+    "selection_objective": "plasticity",
+    "commit_policy": "plasticity_first",
+    "historical_loss_budget": 0,
+    "min_primary_score_delta": 1e-09,
+    "min_correct_gain": 0,
+    "max_forget_count": 0,
+    "min_format_compliance_rate": 1.0,
+    "recover_reward": 1.0,
+    "forget_penalty": 1.5,
+    "min_anchor_score": -5.0,
+    "min_anchor_net_score": -0.1,
+    "max_anchor_forget_rate": 0.2,
+    "components": [
+      "task_interface_structuring",
+      "router_workflow",
+      "memory_selector",
+      "router_context"
+    ]
+  },
+  "memory": {
+    "enabled": true,
+    "raw_capacity_per_task": 10,
+    "abstract_capacity_per_task": 2,
+    "anchor_capacity_per_task": 25,
+    "retrieval_top_k": 2,
+    "reset_on_start": false,
+    "abstract_mode": "llm",
+    "abstract_max_new_tokens": 512,
+    "anchor_selection_mode": "llm",
+    "anchor_count_per_batch": 2,
+    "anchor_candidate_limit": 10,
+    "anchor_selection_max_new_tokens": 256,
+    "skill_enabled": true,
+    "skill_mode": "llm",
+    "abstract_prompt_max_chars": 12000
+  },
+  "forgetting_probe": {
+    "enabled": true,
+    "interval_batches": 2,
+    "samples_per_task": 100,
+    "target_correct_ratio": 0.8,
+    "seed": 20260726,
+    "split": "test",
+    "historical_tasks_only": true,
+    "reuse_probe_set": true,
+    "record_predictions": false
+  },
+  "task_stream_path": "configs/taskstream_textual_main_250_50_500.json",
+  "reset_run_on_start": false,
+  "storage_dir": "storage/experiments/deepseek_flash_reasoning_hcl_stability_250_50_500",
+  "prediction_cache": {
+    "enabled": true,
+    "path": "storage/experiments/deepseek_flash_reasoning_hcl_stability_250_50_500/prediction_cache/prediction_cache.jsonl"
+  },
+  "checkpoint": {
+    "enabled": true,
+    "path": "storage/experiments/deepseek_flash_reasoning_hcl_stability_250_50_500/checkpoint.json",
+    "resume": true
+  },
+  "task_flow": [
+    {
+      "name": "train",
+      "phase": "train",
+      "train_split": "train",
+      "val_split": "validation",
+      "test_split": "test",
+      "continual": true,
+      "batchsize": 25,
+      "train_limit_per_task": 250,
+      "validation_limit_per_task": 50,
+      "test_limit_per_task": 500
+    },
+    {
+      "name": "test",
+      "phase": "test",
+      "test_split": "test",
+      "test_limit_per_task": 500,
+      "group_by_task": true
+    }
+  ]
+}

configs/deepseek_flash_reasoning_raw_zero_shot_500.json

@@ -0,0 +1,29 @@
+{
+  "project_name": "DeepSeek Flash reasoning raw zero-shot 4-task 500-test",
+  "task_stream_path": "configs/taskstream_textual_main_250_50_500.json",
+  "split": "test",
+  "limit_per_task": 500,
+  "workers": 4,
+  "resume": true,
+  "output_dir": "storage/experiments/deepseek_flash_reasoning_raw_zero_shot_500",
+  "model": {
+    "backend": "chat_completions",
+    "name": "deepseek-flash",
+    "paper_model_name": "DeepSeek-V4.1-Flash",
+    "base_url": "https://api.deepseek.com",
+    "api_key_env": "DEEPSEEK_API_KEY",
+    "max_new_tokens": 2048,
+    "answer_max_new_tokens": 512,
+    "temperature": 0.0,
+    "thinking": {
+      "type": "disabled"
+    },
+    "cleanup_output": true,
+    "enable_image_input": false,
+    "system_prompt": "You are a concise reasoning assistant. Follow the requested output format and return only the final answer.",
+    "timeout_seconds": 600,
+    "max_retries": 10,
+    "log_path": "storage/experiments/deepseek_flash_reasoning_raw_zero_shot_500/llm_calls.txt",
+    "log_prompt_chars": 12000
+  }
+}

configs/deepseek_v4_flash_textual_budget_b0_300_80_80_600.json

@@ -0,0 +1,142 @@
+{
+  "project_name": "DeepSeek-V4-Flash textual budget sweep b=0 300-80-80-600",
+  "model": {
+    "backend": "chat_completions",
+    "name": "deepseek-v4-flash",
+    "paper_model_name": "DeepSeek-V4-Flash",
+    "base_url": "https://api.deepseek.com",
+    "api_key_env": "DEEPSEEK_API_KEY",
+    "max_new_tokens": 2048,
+    "answer_max_new_tokens": 512,
+    "temperature": 0.0,
+    "thinking": {"type": "disabled"},
+    "reasoning_effort": null,
+    "system_prompt": "You are a concise reasoning assistant. Reason internally and return only the final answer in the requested format.",
+    "timeout_seconds": 300,
+    "max_retries": 3,
+    "cleanup_output": true
+  },
+  "execution": {
+    "read_only_workers": 8,
+    "run_train_final": false
+  },
+  "task_interface": {
+    "mode": "llm",
+    "max_new_tokens": 2048,
+    "invalid_output_retries": 2,
+    "retry_max_new_tokens": 2048,
+    "fallback_to_rules_on_invalid": true,
+    "long_context_rule_threshold_chars": 5000,
+    "visible_context_max_chars": 12000,
+    "visible_context_max_items": 8
+  },
+  "router": {
+    "route_name": "direct",
+    "strip_output": true,
+    "use_llm_workflow_selector": true,
+    "use_memory_selector": true,
+    "memory_always_on": true,
+    "use_skill_selector": true,
+    "use_tool_selector": true,
+    "tool_auto_select": true,
+    "use_chat_tool_calls": false,
+    "tool_max_rounds": 2,
+    "tool_strict_schema": false,
+    "tool_js_timeout_seconds": 8.0,
+    "skill_selector_max_new_tokens": 512,
+    "skill_selector_invalid_output_retries": 2,
+    "skill_selector_retry_max_new_tokens": 512,
+    "skill_selector_fallback_to_empty_on_invalid": true,
+    "tool_selector_max_new_tokens": 512,
+    "tool_selector_invalid_output_retries": 2,
+    "tool_selector_retry_max_new_tokens": 512,
+    "tool_selector_fallback_to_empty_on_invalid": true,
+    "memory_selector_max_new_tokens": 512,
+    "memory_selector_invalid_output_retries": 2,
+    "memory_selector_retry_max_new_tokens": 512,
+    "memory_selector_fallback_to_empty_on_invalid": true
+  },
+  "optimizer": {
+    "candidate_count": 3,
+    "selection_objective": "plasticity",
+    "commit_policy": "paper_historical_loss_budget",
+    "historical_loss_budget": 0,
+    "min_primary_score_delta": 1e-09,
+    "min_correct_gain": 2,
+    "min_format_compliance_rate": 0.9,
+    "selection_mode": "argmax",
+    "min_historical_anchor_count": 80,
+    "min_historical_anchors_per_task": 80,
+    "require_all_historical_tasks": true,
+    "min_anchor_evaluation_coverage": 1.0,
+    "components": [
+      "task_interface_structuring",
+      "router_workflow",
+      "memory_selector",
+      "router_context"
+    ]
+  },
+  "memory": {
+    "enabled": true,
+    "raw_capacity_per_task": 15,
+    "abstract_capacity_per_task": 2,
+    "abstract_mode": "llm",
+    "abstract_max_new_tokens": 512,
+    "anchor_capacity_per_task": 80,
+    "anchor_selection_mode": "llm",
+    "anchor_count_per_batch": 8,
+    "anchor_candidate_limit": 80,
+    "anchor_selection_max_new_tokens": 256,
+    "retrieval_top_k": 2,
+    "retrieval_candidate_limit": 12,
+    "reset_on_start": true,
+    "skill_enabled": true,
+    "skill_mode": "llm",
+    "skill_max_count": 6,
+    "skill_min_source_tasks": 2
+  },
+  "forgetting_probe": {
+    "enabled": true,
+    "interval_batches": 2,
+    "samples_per_task": 80,
+    "target_correct_ratio": 0.8,
+    "seed": 20260713,
+    "split": "test",
+    "historical_tasks_only": true,
+    "reuse_probe_set": true,
+    "record_predictions": false
+  },
+  "task_stream_path": "configs/taskstream_textual_budget_300_80_80_600.json",
+  "reset_run_on_start": true,
+  "storage_dir": "storage/textual_budget_sweep/b0",
+  "checkpoint": {
+    "enabled": true,
+    "path": "storage/textual_budget_sweep/b0/checkpoint.json",
+    "resume": true
+  },
+  "prediction_cache": {
+    "enabled": true,
+    "path": "storage/textual_budget_sweep/b0/prediction_cache/read_only_predictions.jsonl"
+  },
+  "task_flow": [
+    {
+      "name": "train",
+      "phase": "train",
+      "train_split": "train",
+      "val_split": "validation",
+      "test_split": "test",
+      "continual": true,
+      "batchsize": 30,
+      "train_limit_per_task": 300,
+      "validation_limit_per_task": 80,
+      "test_limit_per_task": 600
+    },
+    {
+      "name": "test",
+      "phase": "test",
+      "test_split": "test",
+      "test_limit_per_task": 600,
+      "group_by_task": true
+    }
+  ]
+}

configs/deepseek_v4_flash_textual_budget_b1_300_80_80_600.json

@@ -0,0 +1,142 @@
+{
+  "project_name": "DeepSeek-V4-Flash textual budget sweep b=1 300-80-80-600",
+  "model": {
+    "backend": "chat_completions",
+    "name": "deepseek-v4-flash",
+    "paper_model_name": "DeepSeek-V4-Flash",
+    "base_url": "https://api.deepseek.com",
+    "api_key_env": "DEEPSEEK_API_KEY",
+    "max_new_tokens": 2048,
+    "answer_max_new_tokens": 512,
+    "temperature": 0.0,
+    "thinking": {"type": "disabled"},
+    "reasoning_effort": null,
+    "system_prompt": "You are a concise reasoning assistant. Reason internally and return only the final answer in the requested format.",
+    "timeout_seconds": 300,
+    "max_retries": 3,
+    "cleanup_output": true
+  },
+  "execution": {
+    "read_only_workers": 8,
+    "run_train_final": false
+  },
+  "task_interface": {
+    "mode": "llm",
+    "max_new_tokens": 2048,
+    "invalid_output_retries": 2,
+    "retry_max_new_tokens": 2048,
+    "fallback_to_rules_on_invalid": true,
+    "long_context_rule_threshold_chars": 5000,
+    "visible_context_max_chars": 12000,
+    "visible_context_max_items": 8
+  },
+  "router": {
+    "route_name": "direct",
+    "strip_output": true,
+    "use_llm_workflow_selector": true,
+    "use_memory_selector": true,
+    "memory_always_on": true,
+    "use_skill_selector": true,
+    "use_tool_selector": true,
+    "tool_auto_select": true,
+    "use_chat_tool_calls": false,
+    "tool_max_rounds": 2,
+    "tool_strict_schema": false,
+    "tool_js_timeout_seconds": 8.0,
+    "skill_selector_max_new_tokens": 512,
+    "skill_selector_invalid_output_retries": 2,
+    "skill_selector_retry_max_new_tokens": 512,
+    "skill_selector_fallback_to_empty_on_invalid": true,
+    "tool_selector_max_new_tokens": 512,
+    "tool_selector_invalid_output_retries": 2,
+    "tool_selector_retry_max_new_tokens": 512,
+    "tool_selector_fallback_to_empty_on_invalid": true,
+    "memory_selector_max_new_tokens": 512,
+    "memory_selector_invalid_output_retries": 2,
+    "memory_selector_retry_max_new_tokens": 512,
+    "memory_selector_fallback_to_empty_on_invalid": true
+  },
+  "optimizer": {
+    "candidate_count": 3,
+    "selection_objective": "plasticity",
+    "commit_policy": "paper_historical_loss_budget",
+    "historical_loss_budget": 1,
+    "min_primary_score_delta": 1e-09,
+    "min_correct_gain": 2,
+    "min_format_compliance_rate": 0.9,
+    "selection_mode": "argmax",
+    "min_historical_anchor_count": 80,
+    "min_historical_anchors_per_task": 80,
+    "require_all_historical_tasks": true,
+    "min_anchor_evaluation_coverage": 1.0,
+    "components": [
+      "task_interface_structuring",
+      "router_workflow",
+      "memory_selector",
+      "router_context"
+    ]
+  },
+  "memory": {
+    "enabled": true,
+    "raw_capacity_per_task": 15,
+    "abstract_capacity_per_task": 2,
+    "abstract_mode": "llm",
+    "abstract_max_new_tokens": 512,
+    "anchor_capacity_per_task": 80,
+    "anchor_selection_mode": "llm",
+    "anchor_count_per_batch": 8,
+    "anchor_candidate_limit": 80,
+    "anchor_selection_max_new_tokens": 256,
+    "retrieval_top_k": 2,
+    "retrieval_candidate_limit": 12,
+    "reset_on_start": true,
+    "skill_enabled": true,
+    "skill_mode": "llm",
+    "skill_max_count": 6,
+    "skill_min_source_tasks": 2
+  },
+  "forgetting_probe": {
+    "enabled": true,
+    "interval_batches": 2,
+    "samples_per_task": 80,
+    "target_correct_ratio": 0.8,
+    "seed": 20260713,
+    "split": "test",
+    "historical_tasks_only": true,
+    "reuse_probe_set": true,
+    "record_predictions": false
+  },
+  "task_stream_path": "configs/taskstream_textual_budget_300_80_80_600.json",
+  "reset_run_on_start": true,
+  "storage_dir": "storage/textual_budget_sweep/b1",
+  "checkpoint": {
+    "enabled": true,
+    "path": "storage/textual_budget_sweep/b1/checkpoint.json",
+    "resume": true
+  },
+  "prediction_cache": {
+    "enabled": true,
+    "path": "storage/textual_budget_sweep/b1/prediction_cache/read_only_predictions.jsonl"
+  },
+  "task_flow": [
+    {
+      "name": "train",
+      "phase": "train",
+      "train_split": "train",
+      "val_split": "validation",
+      "test_split": "test",
+      "continual": true,
+      "batchsize": 30,
+      "train_limit_per_task": 300,
+      "validation_limit_per_task": 80,
+      "test_limit_per_task": 600
+    },
+    {
+      "name": "test",
+      "phase": "test",
+      "test_split": "test",
+      "test_limit_per_task": 600,
+      "group_by_task": true
+    }
+  ]
+}

configs/deepseek_v4_flash_textual_budget_b3_300_80_80_600.json

@@ -0,0 +1,142 @@
+{
+  "project_name": "DeepSeek-V4-Flash textual budget sweep b=3 300-80-80-600",
+  "model": {
+    "backend": "chat_completions",
+    "name": "deepseek-v4-flash",
+    "paper_model_name": "DeepSeek-V4-Flash",
+    "base_url": "https://api.deepseek.com",
+    "api_key_env": "DEEPSEEK_API_KEY",
+    "max_new_tokens": 2048,
+    "answer_max_new_tokens": 512,
+    "temperature": 0.0,
+    "thinking": {"type": "disabled"},
+    "reasoning_effort": null,
+    "system_prompt": "You are a concise reasoning assistant. Reason internally and return only the final answer in the requested format.",
+    "timeout_seconds": 300,
+    "max_retries": 3,
+    "cleanup_output": true
+  },
+  "execution": {
+    "read_only_workers": 8,
+    "run_train_final": false
+  },
+  "task_interface": {
+    "mode": "llm",
+    "max_new_tokens": 2048,
+    "invalid_output_retries": 2,
+    "retry_max_new_tokens": 2048,
+    "fallback_to_rules_on_invalid": true,
+    "long_context_rule_threshold_chars": 5000,
+    "visible_context_max_chars": 12000,
+    "visible_context_max_items": 8
+  },
+  "router": {
+    "route_name": "direct",
+    "strip_output": true,
+    "use_llm_workflow_selector": true,
+    "use_memory_selector": true,
+    "memory_always_on": true,
+    "use_skill_selector": true,
+    "use_tool_selector": true,
+    "tool_auto_select": true,
+    "use_chat_tool_calls": false,
+    "tool_max_rounds": 2,
+    "tool_strict_schema": false,
+    "tool_js_timeout_seconds": 8.0,
+    "skill_selector_max_new_tokens": 512,
+    "skill_selector_invalid_output_retries": 2,
+    "skill_selector_retry_max_new_tokens": 512,
+    "skill_selector_fallback_to_empty_on_invalid": true,
+    "tool_selector_max_new_tokens": 512,
+    "tool_selector_invalid_output_retries": 2,
+    "tool_selector_retry_max_new_tokens": 512,
+    "tool_selector_fallback_to_empty_on_invalid": true,
+    "memory_selector_max_new_tokens": 512,
+    "memory_selector_invalid_output_retries": 2,
+    "memory_selector_retry_max_new_tokens": 512,
+    "memory_selector_fallback_to_empty_on_invalid": true
+  },
+  "optimizer": {
+    "candidate_count": 3,
+    "selection_objective": "plasticity",
+    "commit_policy": "paper_historical_loss_budget",
+    "historical_loss_budget": 3,
+    "min_primary_score_delta": 1e-09,
+    "min_correct_gain": 2,
+    "min_format_compliance_rate": 0.9,
+    "selection_mode": "argmax",
+    "min_historical_anchor_count": 80,
+    "min_historical_anchors_per_task": 80,
+    "require_all_historical_tasks": true,
+    "min_anchor_evaluation_coverage": 1.0,
+    "components": [
+      "task_interface_structuring",
+      "router_workflow",
+      "memory_selector",
+      "router_context"
+    ]
+  },
+  "memory": {
+    "enabled": true,
+    "raw_capacity_per_task": 15,
+    "abstract_capacity_per_task": 2,
+    "abstract_mode": "llm",
+    "abstract_max_new_tokens": 512,
+    "anchor_capacity_per_task": 80,
+    "anchor_selection_mode": "llm",
+    "anchor_count_per_batch": 8,
+    "anchor_candidate_limit": 80,
+    "anchor_selection_max_new_tokens": 256,
+    "retrieval_top_k": 2,
+    "retrieval_candidate_limit": 12,
+    "reset_on_start": true,
+    "skill_enabled": true,
+    "skill_mode": "llm",
+    "skill_max_count": 6,
+    "skill_min_source_tasks": 2
+  },
+  "forgetting_probe": {
+    "enabled": true,
+    "interval_batches": 2,
+    "samples_per_task": 80,
+    "target_correct_ratio": 0.8,
+    "seed": 20260713,
+    "split": "test",
+    "historical_tasks_only": true,
+    "reuse_probe_set": true,
+    "record_predictions": false
+  },
+  "task_stream_path": "configs/taskstream_textual_budget_300_80_80_600.json",
+  "reset_run_on_start": true,
+  "storage_dir": "storage/textual_budget_sweep/b3",
+  "checkpoint": {
+    "enabled": true,
+    "path": "storage/textual_budget_sweep/b3/checkpoint.json",
+    "resume": true
+  },
+  "prediction_cache": {
+    "enabled": true,
+    "path": "storage/textual_budget_sweep/b3/prediction_cache/read_only_predictions.jsonl"
+  },
+  "task_flow": [
+    {
+      "name": "train",
+      "phase": "train",
+      "train_split": "train",
+      "val_split": "validation",
+      "test_split": "test",
+      "continual": true,
+      "batchsize": 30,
+      "train_limit_per_task": 300,
+      "validation_limit_per_task": 80,
+      "test_limit_per_task": 600
+    },
+    {
+      "name": "test",
+      "phase": "test",
+      "test_split": "test",
+      "test_limit_per_task": 600,
+      "group_by_task": true
+    }
+  ]
+}

configs/deepseek_v4_flash_textual_budget_binf_300_80_80_600.json

@@ -0,0 +1,142 @@
+{
+  "project_name": "DeepSeek-V4-Flash textual budget sweep b=inf 300-80-80-600",
+  "model": {
+    "backend": "chat_completions",
+    "name": "deepseek-v4-flash",
+    "paper_model_name": "DeepSeek-V4-Flash",
+    "base_url": "https://api.deepseek.com",
+    "api_key_env": "DEEPSEEK_API_KEY",
+    "max_new_tokens": 2048,
+    "answer_max_new_tokens": 512,
+    "temperature": 0.0,
+    "thinking": {"type": "disabled"},
+    "reasoning_effort": null,
+    "system_prompt": "You are a concise reasoning assistant. Reason internally and return only the final answer in the requested format.",
+    "timeout_seconds": 300,
+    "max_retries": 3,
+    "cleanup_output": true
+  },
+  "execution": {
+    "read_only_workers": 8,
+    "run_train_final": false
+  },
+  "task_interface": {
+    "mode": "llm",
+    "max_new_tokens": 2048,
+    "invalid_output_retries": 2,
+    "retry_max_new_tokens": 2048,
+    "fallback_to_rules_on_invalid": true,
+    "long_context_rule_threshold_chars": 5000,
+    "visible_context_max_chars": 12000,
+    "visible_context_max_items": 8
+  },
+  "router": {
+    "route_name": "direct",
+    "strip_output": true,
+    "use_llm_workflow_selector": true,
+    "use_memory_selector": true,
+    "memory_always_on": true,
+    "use_skill_selector": true,
+    "use_tool_selector": true,
+    "tool_auto_select": true,
+    "use_chat_tool_calls": false,
+    "tool_max_rounds": 2,
+    "tool_strict_schema": false,
+    "tool_js_timeout_seconds": 8.0,
+    "skill_selector_max_new_tokens": 512,
+    "skill_selector_invalid_output_retries": 2,
+    "skill_selector_retry_max_new_tokens": 512,
+    "skill_selector_fallback_to_empty_on_invalid": true,
+    "tool_selector_max_new_tokens": 512,
+    "tool_selector_invalid_output_retries": 2,
+    "tool_selector_retry_max_new_tokens": 512,
+    "tool_selector_fallback_to_empty_on_invalid": true,
+    "memory_selector_max_new_tokens": 512,
+    "memory_selector_invalid_output_retries": 2,
+    "memory_selector_retry_max_new_tokens": 512,
+    "memory_selector_fallback_to_empty_on_invalid": true
+  },
+  "optimizer": {
+    "candidate_count": 3,
+    "selection_objective": "plasticity",
+    "commit_policy": "paper_historical_loss_budget",
+    "historical_loss_budget": null,
+    "min_primary_score_delta": 1e-09,
+    "min_correct_gain": 2,
+    "min_format_compliance_rate": 0.9,
+    "selection_mode": "argmax",
+    "min_historical_anchor_count": 80,
+    "min_historical_anchors_per_task": 80,
+    "require_all_historical_tasks": true,
+    "min_anchor_evaluation_coverage": 1.0,
+    "components": [
+      "task_interface_structuring",
+      "router_workflow",
+      "memory_selector",
+      "router_context"
+    ]
+  },
+  "memory": {
+    "enabled": true,
+    "raw_capacity_per_task": 15,
+    "abstract_capacity_per_task": 2,
+    "abstract_mode": "llm",
+    "abstract_max_new_tokens": 512,
+    "anchor_capacity_per_task": 80,
+    "anchor_selection_mode": "llm",
+    "anchor_count_per_batch": 8,
+    "anchor_candidate_limit": 80,
+    "anchor_selection_max_new_tokens": 256,
+    "retrieval_top_k": 2,
+    "retrieval_candidate_limit": 12,
+    "reset_on_start": true,
+    "skill_enabled": true,
+    "skill_mode": "llm",
+    "skill_max_count": 6,
+    "skill_min_source_tasks": 2
+  },
+  "forgetting_probe": {
+    "enabled": true,
+    "interval_batches": 2,
+    "samples_per_task": 80,
+    "target_correct_ratio": 0.8,
+    "seed": 20260713,
+    "split": "test",
+    "historical_tasks_only": true,
+    "reuse_probe_set": true,
+    "record_predictions": false
+  },
+  "task_stream_path": "configs/taskstream_textual_budget_300_80_80_600.json",
+  "reset_run_on_start": true,
+  "storage_dir": "storage/textual_budget_sweep/binf",
+  "checkpoint": {
+    "enabled": true,
+    "path": "storage/textual_budget_sweep/binf/checkpoint.json",
+    "resume": true
+  },
+  "prediction_cache": {
+    "enabled": true,
+    "path": "storage/textual_budget_sweep/binf/prediction_cache/read_only_predictions.jsonl"
+  },
+  "task_flow": [
+    {
+      "name": "train",
+      "phase": "train",
+      "train_split": "train",
+      "val_split": "validation",
+      "test_split": "test",
+      "continual": true,
+      "batchsize": 30,
+      "train_limit_per_task": 300,
+      "validation_limit_per_task": 80,
+      "test_limit_per_task": 600
+    },
+    {
+      "name": "test",
+      "phase": "test",
+      "test_split": "test",
+      "test_limit_per_task": 600,
+      "group_by_task": true
+    }
+  ]
+}

configs/full_coco_stability_250_50_500_local.json

@@ -0,0 +1,137 @@
+{
+  "project_name": "Qwen3.6-27B COCO HCL stability full 250-50-500 local validation",
+  "model": {
+    "backend": "chat_completions",
+    "name": "qwen3.6-27b-coco-ablation-gpu3",
+    "paper_model_name": "Qwen3.6-27B",
+    "base_url": "http://127.0.0.1:8017",
+    "api_key": "EMPTY",
+    "max_new_tokens": 2048,
+    "answer_max_new_tokens": 1024,
+    "temperature": 0.0,
+    "cleanup_output": true,
+    "enable_image_input": true,
+    "chat_template_kwargs": {
+      "enable_thinking": false
+    },
+    "json_object_for_structured_phases": true,
+    "system_prompt": "You are a concise multimodal reasoning assistant. Inspect the supplied image directly, follow the requested output schema exactly, and output only the final answer.",
+    "timeout_seconds": 600,
+    "max_retries": 20,
+    "log_path": "storage/experiments/full_coco_stability_250_50_500_local/llm_calls.txt",
+    "log_prompt_chars": 8000
+  },
+  "execution": {
+    "read_only_workers": 2,
+    "run_train_final": true
+  },
+  "task_interface": {
+    "mode": "auto",
+    "max_new_tokens": 640,
+    "invalid_output_retries": 1,
+    "retry_max_new_tokens": 1024,
+    "fallback_to_rules_on_invalid": true
+  },
+  "router": {
+    "route_name": "qwen36_27b_coco_gpu3_hcl_stability",
+    "strip_output": true,
+    "use_llm_workflow_selector": true,
+    "use_memory_selector": true,
+    "memory_always_on": true,
+    "use_skill_selector": true,
+    "use_tool_selector": false,
+    "tool_auto_select": false,
+    "use_chat_tool_calls": false,
+    "memory_selector_max_new_tokens": 256,
+    "memory_selector_invalid_output_retries": 1,
+    "memory_selector_retry_max_new_tokens": 256,
+    "skill_selector_max_new_tokens": 256,
+    "skill_selector_invalid_output_retries": 1,
+    "skill_selector_retry_max_new_tokens": 256,
+    "skill_selector_fallback_to_empty_on_invalid": true,
+    "capability_enabled": false,
+    "capability_execution_mode": "disabled"
+  },
+  "optimizer": {
+    "candidate_count": 3,
+    "selection_objective": "plasticity",
+    "commit_policy": "plasticity_first",
+    "historical_loss_budget": 0,
+    "max_forget_count": 0,
+    "min_primary_score_delta": 1e-09,
+    "min_correct_gain": 0,
+    "min_format_compliance_rate": 1.0,
+    "recover_reward": 1.0,
+    "forget_penalty": 1.5,
+    "min_anchor_score": -5.0,
+    "min_anchor_net_score": -0.1,
+    "max_anchor_forget_rate": 0.2,
+    "components": [
+      "task_interface_structuring",
+      "router_workflow",
+      "memory_selector",
+      "router_context"
+    ]
+  },
+  "memory": {
+    "enabled": true,
+    "raw_capacity_per_task": 10,
+    "abstract_capacity_per_task": 2,
+    "anchor_capacity_per_task": 25,
+    "retrieval_top_k": 2,
+    "reset_on_start": false,
+    "abstract_mode": "llm",
+    "abstract_max_new_tokens": 512,
+    "anchor_selection_mode": "llm",
+    "anchor_count_per_batch": 2,
+    "anchor_candidate_limit": 10,
+    "anchor_selection_max_new_tokens": 128,
+    "skill_enabled": true,
+    "skill_mode": "llm",
+    "abstract_prompt_max_chars": 12000
+  },
+  "forgetting_probe": {
+    "enabled": true,
+    "interval_batches": 2,
+    "samples_per_task": 100,
+    "target_correct_ratio": 0.8,
+    "seed": 20260726,
+    "split": "test",
+    "historical_tasks_only": true,
+    "reuse_probe_set": true,
+    "record_predictions": false
+  },
+  "task_stream_path": "configs/taskstream_multimodal_main_250_50_500.json",
+  "reset_run_on_start": false,
+  "storage_dir": "storage/experiments/full_coco_stability_250_50_500_local",
+  "prediction_cache": {
+    "enabled": true,
+    "path": "storage/experiments/full_coco_stability_250_50_500_local/prediction_cache/read_only_predictions.jsonl"
+  },
+  "checkpoint": {
+    "enabled": true,
+    "path": "storage/experiments/full_coco_stability_250_50_500_local/checkpoint.json",
+    "resume": true
+  },
+  "task_flow": [
+    {
+      "name": "train",
+      "phase": "train",
+      "train_split": "train",
+      "val_split": "validation",
+      "test_split": "test",
+      "continual": true,
+      "batchsize": 25,
+      "train_limit_per_task": 250,
+      "validation_limit_per_task": 50,
+      "test_limit_per_task": 500
+    },
+    {
+      "name": "test",
+      "phase": "test",
+      "test_split": "test",
+      "test_limit_per_task": 500,
+      "group_by_task": true
+    }
+  ]
+}

configs/qwen36_27b_coco_gpu1_direct_raw_zero_shot_500.json

@@ -0,0 +1,27 @@
+{
+  "project_name": "Qwen3.6-27B COCO corrected multimodal raw zero-shot 500-test",
+  "task_stream_path": "configs/taskstream_multimodal_main_250_50_500.json",
+  "split": "test",
+  "limit_per_task": 500,
+  "resume": true,
+  "output_dir": "storage/experiments/qwen36_27b_coco_gpu1_direct_raw_zero_shot_500",
+  "model": {
+    "backend": "chat_completions",
+    "name": "qwen3.6-27b-coco-ablation-gpu3",
+    "paper_model_name": "Qwen3.6-27B",
+    "base_url": "http://127.0.0.1:8007",
+    "api_key": "EMPTY",
+    "max_new_tokens": 1024,
+    "answer_max_new_tokens": 1024,
+    "temperature": 0.0,
+    "cleanup_output": true,
+    "enable_image_input": true,
+    "chat_template_kwargs": {
+      "enable_thinking": false
+    },
+    "system_prompt": "You are a concise multimodal reasoning assistant. Inspect the supplied image directly and output only the final answer in the requested format.",
+    "timeout_seconds": 600,
+    "max_retries": 20,
+    "log_prompt_chars": 8000
+  }
+}

configs/qwen36_27b_coco_gpu1_hcl_plasticity_250_50_500.json

@@ -0,0 +1,137 @@
+{
+  "project_name": "Qwen3.6-27B COCO HCL plasticity 250-50-500 migrated",
+  "model": {
+    "backend": "chat_completions",
+    "name": "qwen3.6-27b-coco-ablation-gpu3",
+    "paper_model_name": "Qwen3.6-27B",
+    "base_url": "http://127.0.0.1:8007",
+    "api_key": "EMPTY",
+    "max_new_tokens": 2048,
+    "answer_max_new_tokens": 1024,
+    "temperature": 0.0,
+    "cleanup_output": true,
+    "enable_image_input": true,
+    "chat_template_kwargs": {
+      "enable_thinking": false
+    },
+    "json_object_for_structured_phases": true,
+    "system_prompt": "You are a concise multimodal reasoning assistant. Inspect the supplied image directly, follow the requested output schema exactly, and output only the final answer.",
+    "timeout_seconds": 600,
+    "max_retries": 20,
+    "log_path": "storage/experiments/qwen36_27b_coco_gpu1_hcl_plasticity_250_50_500/llm_calls.txt",
+    "log_prompt_chars": 8000
+  },
+  "execution": {
+    "read_only_workers": 2,
+    "run_train_final": true
+  },
+  "task_interface": {
+    "mode": "auto",
+    "max_new_tokens": 640,
+    "invalid_output_retries": 1,
+    "retry_max_new_tokens": 1024,
+    "fallback_to_rules_on_invalid": true
+  },
+  "router": {
+    "route_name": "qwen36_27b_coco_gpu3_hcl_plasticity",
+    "strip_output": true,
+    "use_llm_workflow_selector": true,
+    "use_memory_selector": true,
+    "memory_always_on": true,
+    "use_skill_selector": true,
+    "use_tool_selector": false,
+    "tool_auto_select": false,
+    "use_chat_tool_calls": false,
+    "memory_selector_max_new_tokens": 256,
+    "memory_selector_invalid_output_retries": 1,
+    "memory_selector_retry_max_new_tokens": 256,
+    "skill_selector_max_new_tokens": 256,
+    "skill_selector_invalid_output_retries": 1,
+    "skill_selector_retry_max_new_tokens": 256,
+    "skill_selector_fallback_to_empty_on_invalid": true,
+    "capability_enabled": false,
+    "capability_execution_mode": "disabled"
+  },
+  "optimizer": {
+    "candidate_count": 3,
+    "selection_objective": "plasticity",
+    "commit_policy": "plasticity_first",
+    "historical_loss_budget": null,
+    "max_forget_count": 0,
+    "min_primary_score_delta": 1e-09,
+    "min_correct_gain": 0,
+    "min_format_compliance_rate": 1.0,
+    "recover_reward": 1.0,
+    "forget_penalty": 1.5,
+    "min_anchor_score": -5.0,
+    "min_anchor_net_score": -0.1,
+    "max_anchor_forget_rate": 0.2,
+    "components": [
+      "task_interface_structuring",
+      "router_workflow",
+      "memory_selector",
+      "router_context"
+    ]
+  },
+  "memory": {
+    "enabled": true,
+    "raw_capacity_per_task": 10,
+    "abstract_capacity_per_task": 2,
+    "anchor_capacity_per_task": 25,
+    "retrieval_top_k": 2,
+    "reset_on_start": false,
+    "abstract_mode": "llm",
+    "abstract_max_new_tokens": 512,
+    "anchor_selection_mode": "llm",
+    "anchor_count_per_batch": 2,
+    "anchor_candidate_limit": 10,
+    "anchor_selection_max_new_tokens": 128,
+    "skill_enabled": true,
+    "skill_mode": "llm",
+    "abstract_prompt_max_chars": 12000
+  },
+  "forgetting_probe": {
+    "enabled": true,
+    "interval_batches": 2,
+    "samples_per_task": 100,
+    "target_correct_ratio": 0.8,
+    "seed": 20260726,
+    "split": "test",
+    "historical_tasks_only": true,
+    "reuse_probe_set": true,
+    "record_predictions": false
+  },
+  "task_stream_path": "configs/taskstream_multimodal_main_250_50_500.json",
+  "reset_run_on_start": false,
+  "storage_dir": "storage/experiments/qwen36_27b_coco_gpu1_hcl_plasticity_250_50_500",
+  "prediction_cache": {
+    "enabled": true,
+    "path": "storage/experiments/qwen36_27b_coco_gpu1_hcl_plasticity_250_50_500/prediction_cache/read_only_predictions.jsonl"
+  },
+  "checkpoint": {
+    "enabled": true,
+    "path": "storage/experiments/qwen36_27b_coco_gpu1_hcl_plasticity_250_50_500/checkpoint.json",
+    "resume": true
+  },
+  "task_flow": [
+    {
+      "name": "train",
+      "phase": "train",
+      "train_split": "train",
+      "val_split": "validation",
+      "test_split": "test",
+      "continual": true,
+      "batchsize": 25,
+      "train_limit_per_task": 250,
+      "validation_limit_per_task": 50,
+      "test_limit_per_task": 500
+    },
+    {
+      "name": "test",
+      "phase": "test",
+      "test_split": "test",
+      "test_limit_per_task": 500,
+      "group_by_task": true
+    }
+  ]
+}

configs/qwen36_27b_coco_gpu1_hcl_stability_250_50_500.json

@@ -0,0 +1,137 @@
+{
+  "project_name": "Qwen3.6-27B COCO HCL stability 250-50-500 migrated",
+  "model": {
+    "backend": "chat_completions",
+    "name": "qwen3.6-27b-coco-ablation-gpu3",
+    "paper_model_name": "Qwen3.6-27B",
+    "base_url": "http://127.0.0.1:8007",
+    "api_key": "EMPTY",
+    "max_new_tokens": 2048,
+    "answer_max_new_tokens": 1024,
+    "temperature": 0.0,
+    "cleanup_output": true,
+    "enable_image_input": true,
+    "chat_template_kwargs": {
+      "enable_thinking": false
+    },
+    "json_object_for_structured_phases": true,
+    "system_prompt": "You are a concise multimodal reasoning assistant. Inspect the supplied image directly, follow the requested output schema exactly, and output only the final answer.",
+    "timeout_seconds": 600,
+    "max_retries": 20,
+    "log_path": "storage/experiments/qwen36_27b_coco_gpu1_hcl_stability_250_50_500/llm_calls.txt",
+    "log_prompt_chars": 8000
+  },
+  "execution": {
+    "read_only_workers": 2,
+    "run_train_final": true
+  },
+  "task_interface": {
+    "mode": "auto",
+    "max_new_tokens": 640,
+    "invalid_output_retries": 1,
+    "retry_max_new_tokens": 1024,
+    "fallback_to_rules_on_invalid": true
+  },
+  "router": {
+    "route_name": "qwen36_27b_coco_gpu3_hcl_stability",
+    "strip_output": true,
+    "use_llm_workflow_selector": true,
+    "use_memory_selector": true,
+    "memory_always_on": true,
+    "use_skill_selector": true,
+    "use_tool_selector": false,
+    "tool_auto_select": false,
+    "use_chat_tool_calls": false,
+    "memory_selector_max_new_tokens": 256,
+    "memory_selector_invalid_output_retries": 1,
+    "memory_selector_retry_max_new_tokens": 256,
+    "skill_selector_max_new_tokens": 256,
+    "skill_selector_invalid_output_retries": 1,
+    "skill_selector_retry_max_new_tokens": 256,
+    "skill_selector_fallback_to_empty_on_invalid": true,
+    "capability_enabled": false,
+    "capability_execution_mode": "disabled"
+  },
+  "optimizer": {
+    "candidate_count": 3,
+    "selection_objective": "plasticity",
+    "commit_policy": "plasticity_first",
+    "historical_loss_budget": 0,
+    "max_forget_count": 0,
+    "min_primary_score_delta": 1e-09,
+    "min_correct_gain": 0,
+    "min_format_compliance_rate": 1.0,
+    "recover_reward": 1.0,
+    "forget_penalty": 1.5,
+    "min_anchor_score": -5.0,
+    "min_anchor_net_score": -0.1,
+    "max_anchor_forget_rate": 0.2,
+    "components": [
+      "task_interface_structuring",
+      "router_workflow",
+      "memory_selector",
+      "router_context"
+    ]
+  },
+  "memory": {
+    "enabled": true,
+    "raw_capacity_per_task": 10,
+    "abstract_capacity_per_task": 2,
+    "anchor_capacity_per_task": 25,
+    "retrieval_top_k": 2,
+    "reset_on_start": false,
+    "abstract_mode": "llm",
+    "abstract_max_new_tokens": 512,
+    "anchor_selection_mode": "llm",
+    "anchor_count_per_batch": 2,
+    "anchor_candidate_limit": 10,
+    "anchor_selection_max_new_tokens": 128,
+    "skill_enabled": true,
+    "skill_mode": "llm",
+    "abstract_prompt_max_chars": 12000
+  },
+  "forgetting_probe": {
+    "enabled": true,
+    "interval_batches": 2,
+    "samples_per_task": 100,
+    "target_correct_ratio": 0.8,
+    "seed": 20260726,
+    "split": "test",
+    "historical_tasks_only": true,
+    "reuse_probe_set": true,
+    "record_predictions": false
+  },
+  "task_stream_path": "configs/taskstream_multimodal_main_250_50_500.json",
+  "reset_run_on_start": false,
+  "storage_dir": "storage/experiments/qwen36_27b_coco_gpu1_hcl_stability_250_50_500",
+  "prediction_cache": {
+    "enabled": true,
+    "path": "storage/experiments/qwen36_27b_coco_gpu1_hcl_stability_250_50_500/prediction_cache/read_only_predictions.jsonl"
+  },
+  "checkpoint": {
+    "enabled": true,
+    "path": "storage/experiments/qwen36_27b_coco_gpu1_hcl_stability_250_50_500/checkpoint.json",
+    "resume": true
+  },
+  "task_flow": [
+    {
+      "name": "train",
+      "phase": "train",
+      "train_split": "train",
+      "val_split": "validation",
+      "test_split": "test",
+      "continual": true,
+      "batchsize": 25,
+      "train_limit_per_task": 250,
+      "validation_limit_per_task": 50,
+      "test_limit_per_task": 500
+    },
+    {
+      "name": "test",
+      "phase": "test",
+      "test_split": "test",
+      "test_limit_per_task": 500,
+      "group_by_task": true
+    }
+  ]
+}

configs/qwen36_27b_coco_gpu1_v0_harness_500.json

@@ -0,0 +1,99 @@
+{
+  "project_name": "Qwen3.6-27B COCO fixed v0 harness 500-test migrated",
+  "model": {
+    "backend": "chat_completions",
+    "name": "qwen3.6-27b-coco-ablation-gpu3",
+    "paper_model_name": "Qwen3.6-27B",
+    "base_url": "http://127.0.0.1:8007",
+    "api_key": "EMPTY",
+    "max_new_tokens": 1024,
+    "answer_max_new_tokens": 1024,
+    "temperature": 0.0,
+    "cleanup_output": true,
+    "enable_image_input": true,
+    "chat_template_kwargs": {
+      "enable_thinking": false
+    },
+    "json_object_for_structured_phases": true,
+    "system_prompt": "You are a concise multimodal reasoning assistant. Inspect the supplied image directly, follow the requested output schema exactly, and output only the final answer.",
+    "timeout_seconds": 600,
+    "max_retries": 20,
+    "log_path": "storage/experiments/qwen36_27b_coco_gpu1_v0_harness_500/llm_calls.txt",
+    "log_prompt_chars": 8000
+  },
+  "execution": {
+    "read_only_workers": 2,
+    "run_train_final": false
+  },
+  "task_interface": {
+    "mode": "auto",
+    "max_new_tokens": 640,
+    "invalid_output_retries": 1,
+    "retry_max_new_tokens": 1024,
+    "fallback_to_rules_on_invalid": true
+  },
+  "router": {
+    "route_name": "qwen36_27b_coco_gpu3_v0_harness",
+    "strip_output": true,
+    "use_llm_workflow_selector": true,
+    "use_memory_selector": true,
+    "memory_always_on": true,
+    "use_skill_selector": true,
+    "use_tool_selector": false,
+    "tool_auto_select": false,
+    "use_chat_tool_calls": false,
+    "memory_selector_max_new_tokens": 256,
+    "memory_selector_invalid_output_retries": 1,
+    "memory_selector_retry_max_new_tokens": 256,
+    "skill_selector_max_new_tokens": 256,
+    "skill_selector_invalid_output_retries": 1,
+    "skill_selector_retry_max_new_tokens": 256,
+    "skill_selector_fallback_to_empty_on_invalid": true,
+    "capability_enabled": false,
+    "capability_execution_mode": "disabled"
+  },
+  "optimizer": {
+    "candidate_count": 0,
+    "components": []
+  },
+  "memory": {
+    "enabled": true,
+    "raw_capacity_per_task": 10,
+    "abstract_capacity_per_task": 2,
+    "anchor_capacity_per_task": 25,
+    "retrieval_top_k": 2,
+    "reset_on_start": true,
+    "abstract_mode": "llm",
+    "abstract_max_new_tokens": 512,
+    "anchor_selection_mode": "llm",
+    "anchor_count_per_batch": 2,
+    "anchor_candidate_limit": 10,
+    "anchor_selection_max_new_tokens": 128,
+    "skill_enabled": true,
+    "skill_mode": "llm"
+  },
+  "forgetting_probe": {
+    "enabled": false
+  },
+  "task_stream_path": "configs/taskstream_multimodal_main_250_50_500.json",
+  "reset_run_on_start": false,
+  "storage_dir": "storage/experiments/qwen36_27b_coco_gpu1_v0_harness_500",
+  "prediction_cache": {
+    "enabled": true,
+    "path": "storage/experiments/qwen36_27b_coco_gpu1_v0_harness_500/prediction_cache/read_only_predictions.jsonl"
+  },
+  "checkpoint": {
+    "enabled": true,
+    "path": "storage/experiments/qwen36_27b_coco_gpu1_v0_harness_500/checkpoint.json",
+    "resume": true
+  },
+  "task_flow": [
+    {
+      "name": "test",
+      "phase": "test",
+      "test_split": "test",
+      "test_limit_per_task": 500,
+      "group_by_task": true
+    }
+  ]
+}

configs/qwen36_27b_reasoning_hcl_stability_capabilities_250_50_500.json

@@ -0,0 +1,153 @@
+{
+  "project_name": "Qwen3.6-27B Stability-HCL with router-selected external capabilities reasoning 250-50-500",
+  "model": {
+    "backend": "chat_completions",
+    "name": "qwen3.6-27b-capabilities-gpu0",
+    "paper_model_name": "Qwen3.6-27B",
+    "base_url": "http://127.0.0.1:8006",
+    "api_key": "EMPTY",
+    "max_new_tokens": 2048,
+    "answer_max_new_tokens": 1024,
+    "temperature": 0.0,
+    "thinking": null,
+    "reasoning_effort": null,
+    "system_prompt": "You are a concise reasoning assistant. Reason internally, use relevant tool evidence when available, follow the requested output schema exactly, and return only the final answer.",
+    "timeout_seconds": 600,
+    "max_retries": 20,
+    "cleanup_output": true,
+    "log_path": "storage/experiments/qwen36_27b_reasoning_hcl_stability_capabilities_250_50_500/llm_calls.txt",
+    "log_prompt_chars": 8000
+  },
+  "execution": {
+    "read_only_workers": 1,
+    "run_train_final": true
+  },
+  "task_interface": {
+    "mode": "llm",
+    "max_new_tokens": 1024,
+    "invalid_output_retries": 1,
+    "retry_max_new_tokens": 1024,
+    "fallback_to_rules_on_invalid": true,
+    "long_context_rule_threshold_chars": 8192
+  },
+  "router": {
+    "route_name": "qwen36_27b_reasoning_hcl_stability_capabilities_router_fixed",
+    "strip_output": true,
+    "use_llm_workflow_selector": true,
+    "use_memory_selector": true,
+    "memory_always_on": false,
+    "use_skill_selector": true,
+    "use_tool_selector": true,
+    "tool_auto_select": true,
+    "use_chat_tool_calls": false,
+    "tool_max_rounds": 0,
+    "tool_strict_schema": false,
+    "tool_js_timeout_seconds": 8.0,
+    "tool_selector_max_new_tokens": 512,
+    "tool_selector_invalid_output_retries": 1,
+    "tool_selector_retry_max_new_tokens": 512,
+    "tool_selector_fallback_to_empty_on_invalid": true,
+    "skill_selector_max_new_tokens": 512,
+    "skill_selector_invalid_output_retries": 1,
+    "skill_selector_retry_max_new_tokens": 512,
+    "skill_selector_fallback_to_empty_on_invalid": true,
+    "memory_selector_max_new_tokens": 512,
+    "memory_selector_invalid_output_retries": 1,
+    "memory_selector_retry_max_new_tokens": 512,
+    "capability_enabled": true,
+    "capability_execution_mode": "router_managed",
+    "capability_service_url": "http://127.0.0.1:8091",
+    "capability_timeout_seconds": 30,
+    "capability_allowed_corpora": [
+      "reasoning_train_corpus"
+    ],
+    "capability_default_corpus_id": "reasoning_train_corpus",
+    "capability_default_top_k": 5,
+    "capability_continue_on_error": true,
+    "capability_force_semantic_search": false,
+    "semantic_search_enabled": true,
+    "cross_modal_match_enabled": true
+  },
+  "optimizer": {
+    "candidate_count": 3,
+    "selection_objective": "stability",
+    "commit_policy": "strict_no_forgetting",
+    "min_accuracy_delta": 0.0,
+    "min_correct_gain": 0,
+    "max_forget_count": 0,
+    "min_format_compliance_rate": 0.85,
+    "recover_reward": 1.0,
+    "forget_penalty": 2.0,
+    "min_anchor_score": 0.0,
+    "min_anchor_net_score": 0.0,
+    "max_anchor_forget_rate": 0.0,
+    "components": [
+      "task_interface_structuring",
+      "router_workflow",
+      "memory_selector",
+      "tool_selector",
+      "router_context"
+    ]
+  },
+  "memory": {
+    "enabled": true,
+    "raw_capacity_per_task": 10,
+    "abstract_capacity_per_task": 2,
+    "anchor_capacity_per_task": 25,
+    "retrieval_top_k": 2,
+    "reset_on_start": false,
+    "abstract_mode": "llm",
+    "abstract_max_new_tokens": 512,
+    "anchor_selection_mode": "llm",
+    "anchor_count_per_batch": 2,
+    "anchor_candidate_limit": 10,
+    "anchor_selection_max_new_tokens": 256,
+    "skill_enabled": true,
+    "skill_mode": "llm",
+    "abstract_prompt_max_chars": 12000
+  },
+  "forgetting_probe": {
+    "enabled": true,
+    "interval_batches": 2,
+    "samples_per_task": 100,
+    "target_correct_ratio": 0.8,
+    "seed": 20260726,
+    "split": "test",
+    "historical_tasks_only": true,
+    "reuse_probe_set": true,
+    "record_predictions": false
+  },
+  "task_stream_path": "configs/taskstream_textual_main_250_50_500.json",
+  "reset_run_on_start": false,
+  "storage_dir": "storage/experiments/qwen36_27b_reasoning_hcl_stability_capabilities_250_50_500",
+  "prediction_cache": {
+    "enabled": true,
+    "path": "storage/experiments/qwen36_27b_reasoning_hcl_stability_capabilities_250_50_500/prediction_cache/read_only_predictions.jsonl"
+  },
+  "checkpoint": {
+    "enabled": true,
+    "path": "storage/experiments/qwen36_27b_reasoning_hcl_stability_capabilities_250_50_500/checkpoint.json",
+    "resume": true
+  },
+  "task_flow": [
+    {
+      "name": "train",
+      "phase": "train",
+      "train_split": "train",
+      "val_split": "validation",
+      "test_split": "test",
+      "continual": true,
+      "batchsize": 25,
+      "train_limit_per_task": 250,
+      "validation_limit_per_task": 50,
+      "test_limit_per_task": 500
+    },
+    {
+      "name": "test",
+      "phase": "test",
+      "test_split": "test",
+      "test_limit_per_task": 500,
+      "group_by_task": true
+    }
+  ]
+}

configs/run_stability_hcl_01_4b.json

@@ -0,0 +1,96 @@
+{
+  "project_name": "hcl_stability_01_4b",
+  "task_stream_path": "configs/taskstream_textual_main_250_50_500.json",
+  "storage_dir": "storage/experiments/stability_hcl_01_4b",
+  "reset_run_on_start": true,
+  "batchsize": 300,
+  "model": {
+    "backend": "local_transformers",
+    "name": "Qwen3-4B",
+    "path": "models/Qwen3-4B",
+    "device_map": "cuda:0",
+    "torch_dtype": "bfloat16",
+    "max_new_tokens": 2048,
+    "answer_max_new_tokens": 200,
+    "temperature": 0.0,
+    "enable_thinking": false,
+    "log_path": "storage/experiments/stability_hcl_01_4b/llm_calls.txt"
+  },
+  "task_interface": {
+    "mode": "auto",
+    "long_context_rule_threshold_chars": 24000,
+    "visible_context_max_chars": 24000,
+    "visible_context_max_items": 12
+  },
+  "router": {
+    "use_llm_workflow_selector": true,
+    "use_memory_selector": true,
+    "use_skill_selector": true,
+    "use_tool_selector": true,
+    "capability_enabled": false,
+    "memory_selector_fallback_to_empty_on_invalid": true,
+    "skill_selector_fallback_to_empty_on_invalid": true,
+    "tool_selector_fallback_to_empty_on_invalid": true,
+    "memory_selector_invalid_output_retries": 2,
+    "skill_selector_invalid_output_retries": 2,
+    "tool_selector_invalid_output_retries": 2
+  },
+  "optimizer": {
+    "candidate_count": 3,
+    "components": [
+      "task_interface_structuring",
+      "router_workflow",
+      "router_context"
+    ],
+    "selection_objective": "stability",
+    "min_accuracy_delta": 0.0,
+    "min_correct_gain": 1,
+    "max_forget_count": 0,
+    "min_format_compliance_rate": 0.0,
+    "min_historical_anchor_count": 1,
+    "min_historical_anchors_per_task": 1
+  },
+  "memory": {
+    "enabled": true,
+    "reset_on_start": true,
+    "abstract_mode": "llm",
+    "anchor_selection_mode": "llm",
+    "anchor_count_per_batch": 5,
+    "anchor_capacity_per_task": 15,
+    "skill_mode": "llm"
+  },
+  "prediction_cache": {
+    "enabled": true
+  },
+  "checkpoint": {
+    "enabled": true,
+    "resume": true
+  },
+  "execution": {
+    "read_only_workers": 1,
+    "run_train_final": false
+  },
+  "forgetting_probe": {
+    "enabled": false
+  },
+  "task_flow": [
+    {
+      "name": "train",
+      "phase": "train",
+      "train_split": "train",
+      "val_split": "validation",
+      "test_split": "test",
+      "continual": true,
+      "batchsize": 300,
+      "train_limit_per_task": 250,
+      "validation_limit_per_task": 50,
+      "test_limit_per_task": 500
+    },
+    {
+      "name": "test",
+      "phase": "test",
+      "test_split": "test",
+      "test_limit_per_task": 500
+    }
+  ]
+}

configs/run_stability_hcl_02_4b_opt27b.json

@@ -0,0 +1,108 @@
+{
+  "project_name": "hcl_stability_02_4b_opt27b",
+  "task_stream_path": "configs/taskstream_textual_main_250_50_500.json",
+  "storage_dir": "storage/experiments/stability_hcl_02_4b_opt27b",
+  "reset_run_on_start": true,
+  "batchsize": 300,
+  "model": {
+    "backend": "local_transformers",
+    "name": "Qwen3-4B",
+    "path": "models/Qwen3-4B",
+    "device_map": "cuda:0",
+    "torch_dtype": "bfloat16",
+    "max_new_tokens": 2048,
+    "answer_max_new_tokens": 200,
+    "temperature": 0.0,
+    "enable_thinking": false,
+    "log_path": "storage/experiments/stability_hcl_02_4b_opt27b/llm_calls.txt"
+  },
+  "judge_model": {
+    "backend": "local_transformers",
+    "name": "Qwen3.6-27B",
+    "path": "models/Qwen3.6-27B",
+    "device_map": "cuda:0",
+    "torch_dtype": "bfloat16",
+    "max_new_tokens": 2048,
+    "answer_max_new_tokens": 200,
+    "temperature": 0.0,
+    "enable_thinking": false,
+    "log_path": "storage/experiments/stability_hcl_02_4b_opt27b/judge_llm_calls.txt"
+  },
+  "task_interface": {
+    "mode": "auto",
+    "long_context_rule_threshold_chars": 24000,
+    "visible_context_max_chars": 24000,
+    "visible_context_max_items": 12
+  },
+  "router": {
+    "use_llm_workflow_selector": true,
+    "use_memory_selector": true,
+    "use_skill_selector": true,
+    "use_tool_selector": true,
+    "capability_enabled": false,
+    "memory_selector_fallback_to_empty_on_invalid": true,
+    "skill_selector_fallback_to_empty_on_invalid": true,
+    "tool_selector_fallback_to_empty_on_invalid": true,
+    "memory_selector_invalid_output_retries": 2,
+    "skill_selector_invalid_output_retries": 2,
+    "tool_selector_invalid_output_retries": 2
+  },
+  "optimizer": {
+    "candidate_count": 3,
+    "components": [
+      "task_interface_structuring",
+      "router_workflow",
+      "router_context"
+    ],
+    "selection_objective": "stability",
+    "min_accuracy_delta": 0.0,
+    "min_correct_gain": 1,
+    "max_forget_count": 0,
+    "min_format_compliance_rate": 0.0,
+    "min_historical_anchor_count": 1,
+    "min_historical_anchors_per_task": 1
+  },
+  "memory": {
+    "enabled": true,
+    "reset_on_start": true,
+    "abstract_mode": "llm",
+    "anchor_selection_mode": "llm",
+    "anchor_count_per_batch": 5,
+    "anchor_capacity_per_task": 15,
+    "skill_mode": "llm"
+  },
+  "prediction_cache": {
+    "enabled": true
+  },
+  "checkpoint": {
+    "enabled": true,
+    "resume": true
+  },
+  "execution": {
+    "read_only_workers": 1,
+    "run_train_final": false
+  },
+  "forgetting_probe": {
+    "enabled": false
+  },
+  "task_flow": [
+    {
+      "name": "train",
+      "phase": "train",
+      "train_split": "train",
+      "val_split": "validation",
+      "test_split": "test",
+      "continual": true,
+      "batchsize": 300,
+      "train_limit_per_task": 250,
+      "validation_limit_per_task": 50,
+      "test_limit_per_task": 500
+    },
+    {
+      "name": "test",
+      "phase": "test",
+      "test_split": "test",
+      "test_limit_per_task": 500
+    }
+  ]
+}

configs/run_stability_hcl_03_4b_teacher27b.json

@@ -0,0 +1,120 @@
+{
+  "project_name": "hcl_stability_03_4b_teacher27b",
+  "task_stream_path": "configs/taskstream_textual_main_250_50_500.json",
+  "storage_dir": "storage/experiments/stability_hcl_03_4b_teacher27b",
+  "reset_run_on_start": true,
+  "batchsize": 300,
+  "model": {
+    "backend": "local_transformers",
+    "name": "Qwen3-4B",
+    "path": "models/Qwen3-4B",
+    "device_map": "cuda:0",
+    "torch_dtype": "bfloat16",
+    "max_new_tokens": 2048,
+    "answer_max_new_tokens": 200,
+    "temperature": 0.0,
+    "enable_thinking": false,
+    "log_path": "storage/experiments/stability_hcl_03_4b_teacher27b/llm_calls.txt"
+  },
+  "selection_model": {
+    "backend": "local_transformers",
+    "name": "Qwen3.6-27B",
+    "path": "models/Qwen3.6-27B",
+    "device_map": "cuda:0",
+    "torch_dtype": "bfloat16",
+    "max_new_tokens": 2048,
+    "answer_max_new_tokens": 200,
+    "temperature": 0.0,
+    "enable_thinking": false,
+    "log_path": "storage/experiments/stability_hcl_03_4b_teacher27b/teacher_llm_calls.txt"
+  },
+  "memory_model": {
+    "backend": "local_transformers",
+    "name": "Qwen3.6-27B",
+    "path": "models/Qwen3.6-27B",
+    "device_map": "cuda:0",
+    "torch_dtype": "bfloat16",
+    "max_new_tokens": 2048,
+    "answer_max_new_tokens": 200,
+    "temperature": 0.0,
+    "enable_thinking": false,
+    "log_path": "storage/experiments/stability_hcl_03_4b_teacher27b/teacher_llm_calls.txt"
+  },
+  "task_interface": {
+    "mode": "auto",
+    "long_context_rule_threshold_chars": 24000,
+    "visible_context_max_chars": 24000,
+    "visible_context_max_items": 12
+  },
+  "router": {
+    "use_llm_workflow_selector": true,
+    "use_memory_selector": true,
+    "use_skill_selector": true,
+    "use_tool_selector": true,
+    "capability_enabled": false,
+    "memory_selector_fallback_to_empty_on_invalid": true,
+    "skill_selector_fallback_to_empty_on_invalid": true,
+    "tool_selector_fallback_to_empty_on_invalid": true,
+    "memory_selector_invalid_output_retries": 2,
+    "skill_selector_invalid_output_retries": 2,
+    "tool_selector_invalid_output_retries": 2
+  },
+  "optimizer": {
+    "candidate_count": 3,
+    "components": [
+      "task_interface_structuring",
+      "router_workflow",
+      "router_context"
+    ],
+    "selection_objective": "stability",
+    "min_accuracy_delta": 0.0,
+    "min_correct_gain": 1,
+    "max_forget_count": 0,
+    "min_format_compliance_rate": 0.0,
+    "min_historical_anchor_count": 1,
+    "min_historical_anchors_per_task": 1
+  },
+  "memory": {
+    "enabled": true,
+    "reset_on_start": true,
+    "abstract_mode": "llm",
+    "anchor_selection_mode": "llm",
+    "anchor_count_per_batch": 5,
+    "anchor_capacity_per_task": 15,
+    "skill_mode": "llm"
+  },
+  "prediction_cache": {
+    "enabled": true
+  },
+  "checkpoint": {
+    "enabled": true,
+    "resume": true
+  },
+  "execution": {
+    "read_only_workers": 1,
+    "run_train_final": false
+  },
+  "forgetting_probe": {
+    "enabled": false
+  },
+  "task_flow": [
+    {
+      "name": "train",
+      "phase": "train",
+      "train_split": "train",
+      "val_split": "validation",
+      "test_split": "test",
+      "continual": true,
+      "batchsize": 300,
+      "train_limit_per_task": 250,
+      "validation_limit_per_task": 50,
+      "test_limit_per_task": 500
+    },
+    {
+      "name": "test",
+      "phase": "test",
+      "test_split": "test",
+      "test_limit_per_task": 500
+    }
+  ]
+}

configs/taskstream_multimodal_main_250_50_500.json

@@ -0,0 +1,50 @@
+{
+  "description": "Paper-aligned multimodal stream manifest. Images and the exact sampled JSONL records are external data and are not redistributed.",
+  "paper_order": ["coco_detection", "coco_caption", "refcoco_grounding", "vqav2"],
+  "tasks": [
+    {
+      "task_name": "coco_detection",
+      "task_type": "coco_detection",
+      "modality": ["text", "image"],
+      "splits": {
+        "train": "data/datasets/coco_detection/train.jsonl",
+        "validation": "data/datasets/coco_detection/validation.jsonl",
+        "anchor": "data/datasets/coco_detection/validation.jsonl",
+        "test": "data/datasets/coco_detection/test.jsonl"
+      }
+    },
+    {
+      "task_name": "coco_caption",
+      "task_type": "coco_caption",
+      "modality": ["text", "image"],
+      "splits": {
+        "train": "data/datasets/coco_caption/train.jsonl",
+        "validation": "data/datasets/coco_caption/validation.jsonl",
+        "anchor": "data/datasets/coco_caption/validation.jsonl",
+        "test": "data/datasets/coco_caption/test.jsonl"
+      }
+    },
+    {
+      "task_name": "refcoco_grounding",
+      "task_type": "refcoco_grounding",
+      "modality": ["text", "image"],
+      "splits": {
+        "train": "data/datasets/refcoco_grounding/train.jsonl",
+        "validation": "data/datasets/refcoco_grounding/validation.jsonl",
+        "anchor": "data/datasets/refcoco_grounding/validation.jsonl",
+        "test": "data/datasets/refcoco_grounding/test.jsonl"
+      }
+    },
+    {
+      "task_name": "vqav2",
+      "task_type": "vqa",
+      "modality": ["text", "image"],
+      "splits": {
+        "train": "data/datasets/vqav2/train.jsonl",
+        "validation": "data/datasets/vqav2/validation.jsonl",
+        "anchor": "data/datasets/vqav2/validation.jsonl",
+        "test": "data/datasets/vqav2/test.jsonl"
+      }
+    }
+  ]
+}

configs/taskstream_multimodal_recovered_500_100_500.json

@@ -0,0 +1,55 @@
+{
+  "stream_name": "multimodal_hcl_stream",
+  "description": "Recovered four-task multimodal 500/100/500 split manifest. Dataset records and images are external.",
+  "paper_order": ["coco_detection", "coco_caption", "refcoco_grounding", "vqav2"],
+  "tasks": [
+    {
+      "task_name": "coco_detection",
+      "task_type": "coco_detection",
+      "modality": ["image", "text"],
+      "interactions": 1,
+      "splits": {
+        "train": "data/datasets_multimodal_500_100_500/coco_detection/train.json",
+        "validation": "data/datasets_multimodal_500_100_500/coco_detection/validation.json",
+        "anchor": "data/datasets_multimodal_500_100_500/coco_detection/validation.json",
+        "test": "data/datasets_multimodal_500_100_500/coco_detection/test.json"
+      }
+    },
+    {
+      "task_name": "coco_caption",
+      "task_type": "coco_caption",
+      "modality": ["image", "text"],
+      "interactions": 1,
+      "splits": {
+        "train": "data/datasets_multimodal_500_100_500/coco_caption/train.json",
+        "validation": "data/datasets_multimodal_500_100_500/coco_caption/validation.json",
+        "anchor": "data/datasets_multimodal_500_100_500/coco_caption/validation.json",
+        "test": "data/datasets_multimodal_500_100_500/coco_caption/test.json"
+      }
+    },
+    {
+      "task_name": "refcoco_grounding",
+      "task_type": "refcoco_grounding",
+      "modality": ["image", "text"],
+      "interactions": 1,
+      "splits": {
+        "train": "data/datasets_multimodal_500_100_500/refcoco_grounding/train.json",
+        "validation": "data/datasets_multimodal_500_100_500/refcoco_grounding/validation.json",
+        "anchor": "data/datasets_multimodal_500_100_500/refcoco_grounding/validation.json",
+        "test": "data/datasets_multimodal_500_100_500/refcoco_grounding/test.json"
+      }
+    },
+    {
+      "task_name": "vqav2",
+      "task_type": "vqa",
+      "modality": ["image", "text"],
+      "interactions": 1,
+      "splits": {
+        "train": "data/datasets_multimodal_500_100_500/vqav2/train.json",
+        "validation": "data/datasets_multimodal_500_100_500/vqav2/validation.json",
+        "anchor": "data/datasets_multimodal_500_100_500/vqav2/validation.json",
+        "test": "data/datasets_multimodal_500_100_500/vqav2/test.json"
+      }
+    }
+  ]
+}

configs/taskstream_textual_budget_300_80_80_600.json

@@ -0,0 +1,50 @@
+{
+  "description": "Paper-aligned independent textual budget-sweep stream. Exact split files are external; their provenance and fingerprints are documented in data/README.md.",
+  "paper_order": ["musique", "proofwriter", "gsm8k", "hotpotqa"],
+  "tasks": [
+    {
+      "task_name": "musique",
+      "task_type": "compositional_multi_hop_qa",
+      "modality": ["text"],
+      "splits": {
+        "train": "data/datasets_budget_300_80_80_600/musique/train.jsonl",
+        "validation": "data/datasets_budget_300_80_80_600/musique/validation.jsonl",
+        "anchor": "data/datasets_budget_300_80_80_600/musique/anchor.jsonl",
+        "test": "data/datasets_budget_300_80_80_600/musique/test.jsonl"
+      }
+    },
+    {
+      "task_name": "proofwriter",
+      "task_type": "logical_reasoning",
+      "modality": ["text"],
+      "splits": {
+        "train": "data/datasets_budget_300_80_80_600/proofwriter/train.jsonl",
+        "validation": "data/datasets_budget_300_80_80_600/proofwriter/validation.jsonl",
+        "anchor": "data/datasets_budget_300_80_80_600/proofwriter/anchor.jsonl",
+        "test": "data/datasets_budget_300_80_80_600/proofwriter/test.jsonl"
+      }
+    },
+    {
+      "task_name": "gsm8k",
+      "task_type": "math_word_problem",
+      "modality": ["text"],
+      "splits": {
+        "train": "data/datasets_budget_300_80_80_600/gsm8k/train.jsonl",
+        "validation": "data/datasets_budget_300_80_80_600/gsm8k/validation.jsonl",
+        "anchor": "data/datasets_budget_300_80_80_600/gsm8k/anchor.jsonl",
+        "test": "data/datasets_budget_300_80_80_600/gsm8k/test.jsonl"
+      }
+    },
+    {
+      "task_name": "hotpotqa",
+      "task_type": "knowledge_intensive_multi_hop_qa",
+      "modality": ["text"],
+      "splits": {
+        "train": "data/datasets_budget_300_80_80_600/hotpotqa/train.jsonl",
+        "validation": "data/datasets_budget_300_80_80_600/hotpotqa/validation.jsonl",
+        "anchor": "data/datasets_budget_300_80_80_600/hotpotqa/anchor.jsonl",
+        "test": "data/datasets_budget_300_80_80_600/hotpotqa/test.jsonl"
+      }
+    }
+  ]
+}

configs/taskstream_textual_main_250_50_500.json

@@ -0,0 +1,50 @@
+{
+  "description": "Paper-aligned textual stream manifest. The exact sampled JSONL records are external data and are not redistributed.",
+  "paper_order": ["musique", "proofwriter", "gsm8k", "hotpotqa"],
+  "tasks": [
+    {
+      "task_name": "musique",
+      "task_type": "compositional_multi_hop_qa",
+      "modality": ["text"],
+      "splits": {
+        "train": "data/datasets/musique/train.jsonl",
+        "validation": "data/datasets/musique/validation.jsonl",
+        "anchor": "data/datasets/musique/anchor.jsonl",
+        "test": "data/datasets/musique/test.jsonl"
+      }
+    },
+    {
+      "task_name": "proofwriter",
+      "task_type": "logical_reasoning",
+      "modality": ["text"],
+      "splits": {
+        "train": "data/datasets/proofwriter/train.jsonl",
+        "validation": "data/datasets/proofwriter/validation.jsonl",
+        "anchor": "data/datasets/proofwriter/anchor.jsonl",
+        "test": "data/datasets/proofwriter/test.jsonl"
+      }
+    },
+    {
+      "task_name": "gsm8k",
+      "task_type": "math_word_problem",
+      "modality": ["text"],
+      "splits": {
+        "train": "data/datasets/gsm8k/train.jsonl",
+        "validation": "data/datasets/gsm8k/validation.jsonl",
+        "anchor": "data/datasets/gsm8k/anchor.jsonl",
+        "test": "data/datasets/gsm8k/test.jsonl"
+      }
+    },
+    {
+      "task_name": "hotpotqa",
+      "task_type": "knowledge_intensive_multi_hop_qa",
+      "modality": ["text"],
+      "splits": {
+        "train": "data/datasets/hotpotqa/train.jsonl",
+        "validation": "data/datasets/hotpotqa/validation.jsonl",
+        "anchor": "data/datasets/hotpotqa/anchor.jsonl",
+        "test": "data/datasets/hotpotqa/test.jsonl"
+      }
+    }
+  ]
+}

configs/validation_coco_distill_01_limit1.json

@@ -0,0 +1,100 @@
+{
+  "project_name": "coco_distill_01_smoke",
+  "task_stream_path": "configs/taskstream_multimodal_main_250_50_500.json",
+  "storage_dir": "storage/validation/coco_distill_01_limit1",
+  "reset_run_on_start": true,
+  "batchsize": 300,
+  "model": {
+    "backend": "local_transformers",
+    "name": "Qwen3.5-4B",
+    "path": "models/Qwen3.5-4B",
+    "device_map": "cuda:0",
+    "torch_dtype": "bfloat16",
+    "max_new_tokens": 2048,
+    "answer_max_new_tokens": 512,
+    "temperature": 0.0,
+    "enable_thinking": false,
+    "log_path": "storage/validation/coco_distill_01_limit1/student_calls.txt"
+  },
+  "task_interface": {
+    "mode": "auto",
+    "long_context_rule_threshold_chars": 24000,
+    "visible_context_max_chars": 24000,
+    "visible_context_max_items": 12,
+    "max_new_tokens": 640,
+    "retry_max_new_tokens": 1024,
+    "invalid_output_retries": 1,
+    "fallback_to_rules_on_invalid": true
+  },
+  "router": {
+    "use_llm_workflow_selector": true,
+    "use_memory_selector": true,
+    "use_skill_selector": true,
+    "use_tool_selector": true,
+    "capability_enabled": false,
+    "memory_selector_fallback_to_empty_on_invalid": true,
+    "skill_selector_fallback_to_empty_on_invalid": true,
+    "tool_selector_fallback_to_empty_on_invalid": true,
+    "memory_selector_invalid_output_retries": 2,
+    "skill_selector_invalid_output_retries": 2,
+    "tool_selector_invalid_output_retries": 2
+  },
+  "optimizer": {
+    "candidate_count": 1,
+    "components": [
+      "task_interface_structuring",
+      "router_workflow",
+      "router_context"
+    ],
+    "selection_objective": "stability",
+    "min_accuracy_delta": 0.0,
+    "min_correct_gain": 1,
+    "max_forget_count": 0,
+    "min_format_compliance_rate": 0.0,
+    "min_historical_anchor_count": 1,
+    "min_historical_anchors_per_task": 1
+  },
+  "memory": {
+    "enabled": true,
+    "reset_on_start": true,
+    "abstract_mode": "llm",
+    "anchor_selection_mode": "llm",
+    "anchor_count_per_batch": 5,
+    "anchor_capacity_per_task": 15,
+    "skill_mode": "llm"
+  },
+  "prediction_cache": {
+    "enabled": true
+  },
+  "checkpoint": {
+    "enabled": true,
+    "resume": true
+  },
+  "execution": {
+    "read_only_workers": 1,
+    "run_train_final": false
+  },
+  "forgetting_probe": {
+    "enabled": false
+  },
+  "task_flow": [
+    {
+      "name": "train",
+      "phase": "train",
+      "train_split": "train",
+      "val_split": "validation",
+      "test_split": "test",
+      "continual": true,
+      "batchsize": 300,
+      "train_limit_per_task": 1,
+      "validation_limit_per_task": 1,
+      "test_limit_per_task": 1
+    },
+    {
+      "name": "test",
+      "phase": "test",
+      "test_split": "test",
+      "test_limit_per_task": 1
+    }
+  ]
+}

configs/validation_coco_distill_02_limit1.json

@@ -0,0 +1,116 @@
+{
+  "project_name": "coco_distill_02_smoke",
+  "task_stream_path": "configs/taskstream_multimodal_main_250_50_500.json",
+  "storage_dir": "storage/validation/coco_distill_02_limit1",
+  "reset_run_on_start": true,
+  "batchsize": 300,
+  "model": {
+    "backend": "local_transformers",
+    "name": "Qwen3.5-4B",
+    "path": "models/Qwen3.5-4B",
+    "device_map": "cuda:0",
+    "torch_dtype": "bfloat16",
+    "max_new_tokens": 2048,
+    "answer_max_new_tokens": 512,
+    "temperature": 0.0,
+    "enable_thinking": false,
+    "log_path": "storage/validation/coco_distill_02_limit1/student_calls.txt"
+  },
+  "judge_model": {
+    "backend": "chat_completions",
+    "name": "qwen3.6-27b-coco-ablation-gpu3",
+    "max_new_tokens": 2048,
+    "answer_max_new_tokens": 512,
+    "temperature": 0.0,
+    "enable_thinking": false,
+    "log_path": "storage/validation/coco_distill_02_limit1/judge_model_calls.txt",
+    "base_url": "http://127.0.0.1:8017",
+    "api_key": "EMPTY",
+    "chat_template_kwargs": {
+      "enable_thinking": false
+    },
+    "timeout_seconds": 600,
+    "max_retries": 1
+  },
+  "task_interface": {
+    "mode": "auto",
+    "long_context_rule_threshold_chars": 24000,
+    "visible_context_max_chars": 24000,
+    "visible_context_max_items": 12,
+    "max_new_tokens": 640,
+    "retry_max_new_tokens": 1024,
+    "invalid_output_retries": 1,
+    "fallback_to_rules_on_invalid": true
+  },
+  "router": {
+    "use_llm_workflow_selector": true,
+    "use_memory_selector": true,
+    "use_skill_selector": true,
+    "use_tool_selector": true,
+    "capability_enabled": false,
+    "memory_selector_fallback_to_empty_on_invalid": true,
+    "skill_selector_fallback_to_empty_on_invalid": true,
+    "tool_selector_fallback_to_empty_on_invalid": true,
+    "memory_selector_invalid_output_retries": 2,
+    "skill_selector_invalid_output_retries": 2,
+    "tool_selector_invalid_output_retries": 2
+  },
+  "optimizer": {
+    "candidate_count": 1,
+    "components": [
+      "task_interface_structuring",
+      "router_workflow",
+      "router_context"
+    ],
+    "selection_objective": "stability",
+    "min_accuracy_delta": 0.0,
+    "min_correct_gain": 1,
+    "max_forget_count": 0,
+    "min_format_compliance_rate": 0.0,
+    "min_historical_anchor_count": 1,
+    "min_historical_anchors_per_task": 1
+  },
+  "memory": {
+    "enabled": true,
+    "reset_on_start": true,
+    "abstract_mode": "llm",
+    "anchor_selection_mode": "llm",
+    "anchor_count_per_batch": 5,
+    "anchor_capacity_per_task": 15,
+    "skill_mode": "llm"
+  },
+  "prediction_cache": {
+    "enabled": true
+  },
+  "checkpoint": {
+    "enabled": true,
+    "resume": true
+  },
+  "execution": {
+    "read_only_workers": 1,
+    "run_train_final": false
+  },
+  "forgetting_probe": {
+    "enabled": false
+  },
+  "task_flow": [
+    {
+      "name": "train",
+      "phase": "train",
+      "train_split": "train",
+      "val_split": "validation",
+      "test_split": "test",
+      "continual": true,
+      "batchsize": 300,
+      "train_limit_per_task": 1,
+      "validation_limit_per_task": 1,
+      "test_limit_per_task": 1
+    },
+    {
+      "name": "test",
+      "phase": "test",
+      "test_split": "test",
+      "test_limit_per_task": 1
+    }
+  ]
+}

configs/validation_coco_distill_03_limit1.json

@@ -0,0 +1,132 @@
+{
+  "project_name": "coco_distill_03_smoke",
+  "task_stream_path": "configs/taskstream_multimodal_main_250_50_500.json",
+  "storage_dir": "storage/validation/coco_distill_03_limit1",
+  "reset_run_on_start": true,
+  "batchsize": 300,
+  "model": {
+    "backend": "local_transformers",
+    "name": "Qwen3.5-4B",
+    "path": "models/Qwen3.5-4B",
+    "device_map": "cuda:0",
+    "torch_dtype": "bfloat16",
+    "max_new_tokens": 2048,
+    "answer_max_new_tokens": 512,
+    "temperature": 0.0,
+    "enable_thinking": false,
+    "log_path": "storage/validation/coco_distill_03_limit1/student_calls.txt"
+  },
+  "selection_model": {
+    "backend": "chat_completions",
+    "name": "qwen3.6-27b-coco-ablation-gpu3",
+    "max_new_tokens": 2048,
+    "answer_max_new_tokens": 512,
+    "temperature": 0.0,
+    "enable_thinking": false,
+    "log_path": "storage/validation/coco_distill_03_limit1/selection_model_calls.txt",
+    "base_url": "http://127.0.0.1:8017",
+    "api_key": "EMPTY",
+    "chat_template_kwargs": {
+      "enable_thinking": false
+    },
+    "timeout_seconds": 600,
+    "max_retries": 1
+  },
+  "memory_model": {
+    "backend": "chat_completions",
+    "name": "qwen3.6-27b-coco-ablation-gpu3",
+    "max_new_tokens": 2048,
+    "answer_max_new_tokens": 512,
+    "temperature": 0.0,
+    "enable_thinking": false,
+    "log_path": "storage/validation/coco_distill_03_limit1/memory_model_calls.txt",
+    "base_url": "http://127.0.0.1:8017",
+    "api_key": "EMPTY",
+    "chat_template_kwargs": {
+      "enable_thinking": false
+    },
+    "timeout_seconds": 600,
+    "max_retries": 1
+  },
+  "task_interface": {
+    "mode": "auto",
+    "long_context_rule_threshold_chars": 24000,
+    "visible_context_max_chars": 24000,
+    "visible_context_max_items": 12,
+    "max_new_tokens": 640,
+    "retry_max_new_tokens": 1024,
+    "invalid_output_retries": 1,
+    "fallback_to_rules_on_invalid": true
+  },
+  "router": {
+    "use_llm_workflow_selector": true,
+    "use_memory_selector": true,
+    "use_skill_selector": true,
+    "use_tool_selector": true,
+    "capability_enabled": false,
+    "memory_selector_fallback_to_empty_on_invalid": true,
+    "skill_selector_fallback_to_empty_on_invalid": true,
+    "tool_selector_fallback_to_empty_on_invalid": true,
+    "memory_selector_invalid_output_retries": 2,
+    "skill_selector_invalid_output_retries": 2,
+    "tool_selector_invalid_output_retries": 2
+  },
+  "optimizer": {
+    "candidate_count": 1,
+    "components": [
+      "task_interface_structuring",
+      "router_workflow",
+      "router_context"
+    ],
+    "selection_objective": "stability",
+    "min_accuracy_delta": 0.0,
+    "min_correct_gain": 1,
+    "max_forget_count": 0,
+    "min_format_compliance_rate": 0.0,
+    "min_historical_anchor_count": 1,
+    "min_historical_anchors_per_task": 1
+  },
+  "memory": {
+    "enabled": true,
+    "reset_on_start": true,
+    "abstract_mode": "llm",
+    "anchor_selection_mode": "llm",
+    "anchor_count_per_batch": 5,
+    "anchor_capacity_per_task": 15,
+    "skill_mode": "llm"
+  },
+  "prediction_cache": {
+    "enabled": true
+  },
+  "checkpoint": {
+    "enabled": true,
+    "resume": true
+  },
+  "execution": {
+    "read_only_workers": 1,
+    "run_train_final": false
+  },
+  "forgetting_probe": {
+    "enabled": false
+  },
+  "task_flow": [
+    {
+      "name": "train",
+      "phase": "train",
+      "train_split": "train",
+      "val_split": "validation",
+      "test_split": "test",
+      "continual": true,
+      "batchsize": 300,
+      "train_limit_per_task": 1,
+      "validation_limit_per_task": 1,
+      "test_limit_per_task": 1
+    },
+    {
+      "name": "test",
+      "phase": "test",
+      "test_split": "test",
+      "test_limit_per_task": 1
+    }
+  ]
+}

configs/validation_coco_stability_limit1.json

@@ -0,0 +1,137 @@
+{
+  "project_name": "Qwen3.6-27B COCO HCL stability 250-50-500 migrated",
+  "model": {
+    "backend": "chat_completions",
+    "name": "qwen3.6-27b-coco-ablation-gpu3",
+    "paper_model_name": "Qwen3.6-27B",
+    "base_url": "http://127.0.0.1:8017",
+    "api_key": "EMPTY",
+    "max_new_tokens": 2048,
+    "answer_max_new_tokens": 1024,
+    "temperature": 0.0,
+    "cleanup_output": true,
+    "enable_image_input": true,
+    "chat_template_kwargs": {
+      "enable_thinking": false
+    },
+    "json_object_for_structured_phases": true,
+    "system_prompt": "You are a concise multimodal reasoning assistant. Inspect the supplied image directly, follow the requested output schema exactly, and output only the final answer.",
+    "timeout_seconds": 600,
+    "max_retries": 1,
+    "log_path": "storage/validation/coco_stability_limit1/llm_calls.txt",
+    "log_prompt_chars": 8000
+  },
+  "execution": {
+    "read_only_workers": 2,
+    "run_train_final": true
+  },
+  "task_interface": {
+    "mode": "auto",
+    "max_new_tokens": 640,
+    "invalid_output_retries": 1,
+    "retry_max_new_tokens": 1024,
+    "fallback_to_rules_on_invalid": true
+  },
+  "router": {
+    "route_name": "qwen36_27b_coco_gpu3_hcl_stability",
+    "strip_output": true,
+    "use_llm_workflow_selector": true,
+    "use_memory_selector": true,
+    "memory_always_on": true,
+    "use_skill_selector": true,
+    "use_tool_selector": false,
+    "tool_auto_select": false,
+    "use_chat_tool_calls": false,
+    "memory_selector_max_new_tokens": 256,
+    "memory_selector_invalid_output_retries": 1,
+    "memory_selector_retry_max_new_tokens": 256,
+    "skill_selector_max_new_tokens": 256,
+    "skill_selector_invalid_output_retries": 1,
+    "skill_selector_retry_max_new_tokens": 256,
+    "skill_selector_fallback_to_empty_on_invalid": true,
+    "capability_enabled": false,
+    "capability_execution_mode": "disabled"
+  },
+  "optimizer": {
+    "candidate_count": 3,
+    "selection_objective": "plasticity",
+    "commit_policy": "plasticity_first",
+    "historical_loss_budget": 0,
+    "max_forget_count": 0,
+    "min_primary_score_delta": 1e-09,
+    "min_correct_gain": 0,
+    "min_format_compliance_rate": 1.0,
+    "recover_reward": 1.0,
+    "forget_penalty": 1.5,
+    "min_anchor_score": -5.0,
+    "min_anchor_net_score": -0.1,
+    "max_anchor_forget_rate": 0.2,
+    "components": [
+      "task_interface_structuring",
+      "router_workflow",
+      "memory_selector",
+      "router_context"
+    ]
+  },
+  "memory": {
+    "enabled": true,
+    "raw_capacity_per_task": 10,
+    "abstract_capacity_per_task": 2,
+    "anchor_capacity_per_task": 25,
+    "retrieval_top_k": 2,
+    "reset_on_start": false,
+    "abstract_mode": "llm",
+    "abstract_max_new_tokens": 512,
+    "anchor_selection_mode": "llm",
+    "anchor_count_per_batch": 2,
+    "anchor_candidate_limit": 10,
+    "anchor_selection_max_new_tokens": 128,
+    "skill_enabled": true,
+    "skill_mode": "llm",
+    "abstract_prompt_max_chars": 12000
+  },
+  "forgetting_probe": {
+    "enabled": true,
+    "interval_batches": 2,
+    "samples_per_task": 100,
+    "target_correct_ratio": 0.8,
+    "seed": 20260726,
+    "split": "test",
+    "historical_tasks_only": true,
+    "reuse_probe_set": true,
+    "record_predictions": false
+  },
+  "task_stream_path": "configs/taskstream_multimodal_main_250_50_500.json",
+  "reset_run_on_start": true,
+  "storage_dir": "storage/validation/coco_stability_limit1",
+  "prediction_cache": {
+    "enabled": true,
+    "path": "storage/validation/coco_stability_limit1/prediction_cache/read_only_predictions.jsonl"
+  },
+  "checkpoint": {
+    "enabled": true,
+    "path": "storage/validation/coco_stability_limit1/checkpoint.json",
+    "resume": true
+  },
+  "task_flow": [
+    {
+      "name": "train",
+      "phase": "train",
+      "train_split": "train",
+      "val_split": "validation",
+      "test_split": "test",
+      "continual": true,
+      "batchsize": 25,
+      "train_limit_per_task": 1,
+      "validation_limit_per_task": 1,
+      "test_limit_per_task": 1
+    },
+    {
+      "name": "test",
+      "phase": "test",
+      "test_split": "test",
+      "test_limit_per_task": 1,
+      "group_by_task": true
+    }
+  ]
+}

configs/validation_distill_01_limit1.json

@@ -0,0 +1,96 @@
+{
+  "project_name": "hcl_stability_01_4b",
+  "task_stream_path": "configs/taskstream_textual_main_250_50_500.json",
+  "storage_dir": "storage/validation/distill_01_limit1_full_output",
+  "reset_run_on_start": true,
+  "batchsize": 300,
+  "model": {
+    "backend": "local_transformers",
+    "name": "Qwen3-4B",
+    "path": "models/Qwen3-4B",
+    "device_map": "cuda:0",
+    "torch_dtype": "bfloat16",
+    "max_new_tokens": 2048,
+    "answer_max_new_tokens": 200,
+    "temperature": 0.0,
+    "enable_thinking": false,
+    "log_path": "storage/validation/distill_01_limit1_full_output/model_calls.txt"
+  },
+  "task_interface": {
+    "mode": "auto",
+    "long_context_rule_threshold_chars": 24000,
+    "visible_context_max_chars": 24000,
+    "visible_context_max_items": 12
+  },
+  "router": {
+    "use_llm_workflow_selector": true,
+    "use_memory_selector": true,
+    "use_skill_selector": true,
+    "use_tool_selector": true,
+    "capability_enabled": false,
+    "memory_selector_fallback_to_empty_on_invalid": true,
+    "skill_selector_fallback_to_empty_on_invalid": true,
+    "tool_selector_fallback_to_empty_on_invalid": true,
+    "memory_selector_invalid_output_retries": 2,
+    "skill_selector_invalid_output_retries": 2,
+    "tool_selector_invalid_output_retries": 2
+  },
+  "optimizer": {
+    "candidate_count": 3,
+    "components": [
+      "task_interface_structuring",
+      "router_workflow",
+      "router_context"
+    ],
+    "selection_objective": "stability",
+    "min_accuracy_delta": 0.0,
+    "min_correct_gain": 1,
+    "max_forget_count": 0,
+    "min_format_compliance_rate": 0.0,
+    "min_historical_anchor_count": 1,
+    "min_historical_anchors_per_task": 1
+  },
+  "memory": {
+    "enabled": true,
+    "reset_on_start": true,
+    "abstract_mode": "llm",
+    "anchor_selection_mode": "llm",
+    "anchor_count_per_batch": 5,
+    "anchor_capacity_per_task": 15,
+    "skill_mode": "llm"
+  },
+  "prediction_cache": {
+    "enabled": true
+  },
+  "checkpoint": {
+    "enabled": true,
+    "resume": true
+  },
+  "execution": {
+    "read_only_workers": 1,
+    "run_train_final": false
+  },
+  "forgetting_probe": {
+    "enabled": false
+  },
+  "task_flow": [
+    {
+      "name": "train",
+      "phase": "train",
+      "train_split": "train",
+      "val_split": "validation",
+      "test_split": "test",
+      "continual": true,
+      "batchsize": 300,
+      "train_limit_per_task": 1,
+      "validation_limit_per_task": 1,
+      "test_limit_per_task": 1
+    },
+    {
+      "name": "test",
+      "phase": "test",
+      "test_split": "test",
+      "test_limit_per_task": 1
+    }
+  ]
+}

configs/validation_distill_02_limit1.json

@@ -0,0 +1,112 @@
+{
+  "project_name": "hcl_stability_02_4b_opt27b",
+  "task_stream_path": "configs/taskstream_textual_main_250_50_500.json",
+  "storage_dir": "storage/validation/distill_02_limit1",
+  "reset_run_on_start": true,
+  "batchsize": 300,
+  "model": {
+    "backend": "local_transformers",
+    "name": "Qwen3-4B",
+    "path": "models/Qwen3-4B",
+    "device_map": "cuda:0",
+    "torch_dtype": "bfloat16",
+    "max_new_tokens": 2048,
+    "answer_max_new_tokens": 200,
+    "temperature": 0.0,
+    "enable_thinking": false,
+    "log_path": "storage/validation/distill_02_limit1/model_calls.txt"
+  },
+  "judge_model": {
+    "backend": "chat_completions",
+    "name": "qwen3.6-27b-coco-ablation-gpu3",
+    "max_new_tokens": 2048,
+    "answer_max_new_tokens": 200,
+    "temperature": 0.0,
+    "enable_thinking": false,
+    "log_path": "storage/validation/distill_02_limit1/judge_model_calls.txt",
+    "base_url": "http://127.0.0.1:8017",
+    "api_key": "EMPTY",
+    "timeout_seconds": 600,
+    "max_retries": 1,
+    "chat_template_kwargs": {
+      "enable_thinking": false
+    }
+  },
+  "task_interface": {
+    "mode": "auto",
+    "long_context_rule_threshold_chars": 24000,
+    "visible_context_max_chars": 24000,
+    "visible_context_max_items": 12
+  },
+  "router": {
+    "use_llm_workflow_selector": true,
+    "use_memory_selector": true,
+    "use_skill_selector": true,
+    "use_tool_selector": true,
+    "capability_enabled": false,
+    "memory_selector_fallback_to_empty_on_invalid": true,
+    "skill_selector_fallback_to_empty_on_invalid": true,
+    "tool_selector_fallback_to_empty_on_invalid": true,
+    "memory_selector_invalid_output_retries": 2,
+    "skill_selector_invalid_output_retries": 2,
+    "tool_selector_invalid_output_retries": 2
+  },
+  "optimizer": {
+    "candidate_count": 3,
+    "components": [
+      "task_interface_structuring",
+      "router_workflow",
+      "router_context"
+    ],
+    "selection_objective": "stability",
+    "min_accuracy_delta": 0.0,
+    "min_correct_gain": 1,
+    "max_forget_count": 0,
+    "min_format_compliance_rate": 0.0,
+    "min_historical_anchor_count": 1,
+    "min_historical_anchors_per_task": 1
+  },
+  "memory": {
+    "enabled": true,
+    "reset_on_start": true,
+    "abstract_mode": "llm",
+    "anchor_selection_mode": "llm",
+    "anchor_count_per_batch": 5,
+    "anchor_capacity_per_task": 15,
+    "skill_mode": "llm"
+  },
+  "prediction_cache": {
+    "enabled": true
+  },
+  "checkpoint": {
+    "enabled": true,
+    "resume": true
+  },
+  "execution": {
+    "read_only_workers": 1,
+    "run_train_final": false
+  },
+  "forgetting_probe": {
+    "enabled": false
+  },
+  "task_flow": [
+    {
+      "name": "train",
+      "phase": "train",
+      "train_split": "train",
+      "val_split": "validation",
+      "test_split": "test",
+      "continual": true,
+      "batchsize": 300,
+      "train_limit_per_task": 1,
+      "validation_limit_per_task": 1,
+      "test_limit_per_task": 1
+    },
+    {
+      "name": "test",
+      "phase": "test",
+      "test_split": "test",
+      "test_limit_per_task": 1
+    }
+  ]
+}

configs/validation_distill_03_limit1.json

@@ -0,0 +1,128 @@
+{
+  "project_name": "hcl_stability_03_4b_teacher27b",
+  "task_stream_path": "configs/taskstream_textual_main_250_50_500.json",
+  "storage_dir": "storage/validation/distill_03_limit1",
+  "reset_run_on_start": true,
+  "batchsize": 300,
+  "model": {
+    "backend": "local_transformers",
+    "name": "Qwen3-4B",
+    "path": "models/Qwen3-4B",
+    "device_map": "cuda:0",
+    "torch_dtype": "bfloat16",
+    "max_new_tokens": 2048,
+    "answer_max_new_tokens": 200,
+    "temperature": 0.0,
+    "enable_thinking": false,
+    "log_path": "storage/validation/distill_03_limit1/model_calls.txt"
+  },
+  "selection_model": {
+    "backend": "chat_completions",
+    "name": "qwen3.6-27b-coco-ablation-gpu3",
+    "max_new_tokens": 2048,
+    "answer_max_new_tokens": 200,
+    "temperature": 0.0,
+    "enable_thinking": false,
+    "log_path": "storage/validation/distill_03_limit1/selection_model_calls.txt",
+    "base_url": "http://127.0.0.1:8017",
+    "api_key": "EMPTY",
+    "timeout_seconds": 600,
+    "max_retries": 1,
+    "chat_template_kwargs": {
+      "enable_thinking": false
+    }
+  },
+  "memory_model": {
+    "backend": "chat_completions",
+    "name": "qwen3.6-27b-coco-ablation-gpu3",
+    "max_new_tokens": 2048,
+    "answer_max_new_tokens": 200,
+    "temperature": 0.0,
+    "enable_thinking": false,
+    "log_path": "storage/validation/distill_03_limit1/memory_model_calls.txt",
+    "base_url": "http://127.0.0.1:8017",
+    "api_key": "EMPTY",
+    "timeout_seconds": 600,
+    "max_retries": 1,
+    "chat_template_kwargs": {
+      "enable_thinking": false
+    }
+  },
+  "task_interface": {
+    "mode": "auto",
+    "long_context_rule_threshold_chars": 24000,
+    "visible_context_max_chars": 24000,
+    "visible_context_max_items": 12
+  },
+  "router": {
+    "use_llm_workflow_selector": true,
+    "use_memory_selector": true,
+    "use_skill_selector": true,
+    "use_tool_selector": true,
+    "capability_enabled": false,
+    "memory_selector_fallback_to_empty_on_invalid": true,
+    "skill_selector_fallback_to_empty_on_invalid": true,
+    "tool_selector_fallback_to_empty_on_invalid": true,
+    "memory_selector_invalid_output_retries": 2,
+    "skill_selector_invalid_output_retries": 2,
+    "tool_selector_invalid_output_retries": 2
+  },
+  "optimizer": {
+    "candidate_count": 3,
+    "components": [
+      "task_interface_structuring",
+      "router_workflow",
+      "router_context"
+    ],
+    "selection_objective": "stability",
+    "min_accuracy_delta": 0.0,
+    "min_correct_gain": 1,
+    "max_forget_count": 0,
+    "min_format_compliance_rate": 0.0,
+    "min_historical_anchor_count": 1,
+    "min_historical_anchors_per_task": 1
+  },
+  "memory": {
+    "enabled": true,
+    "reset_on_start": true,
+    "abstract_mode": "llm",
+    "anchor_selection_mode": "llm",
+    "anchor_count_per_batch": 5,
+    "anchor_capacity_per_task": 15,
+    "skill_mode": "llm"
+  },
+  "prediction_cache": {
+    "enabled": true
+  },
+  "checkpoint": {
+    "enabled": true,
+    "resume": true
+  },
+  "execution": {
+    "read_only_workers": 1,
+    "run_train_final": false
+  },
+  "forgetting_probe": {
+    "enabled": false
+  },
+  "task_flow": [
+    {
+      "name": "train",
+      "phase": "train",
+      "train_split": "train",
+      "val_split": "validation",
+      "test_split": "test",
+      "continual": true,
+      "batchsize": 300,
+      "train_limit_per_task": 1,
+      "validation_limit_per_task": 1,
+      "test_limit_per_task": 1
+    },
+    {
+      "name": "test",
+      "phase": "test",
+      "test_split": "test",
+      "test_limit_per_task": 1
+    }
+  ]
+}

data/README.md

@@ -0,0 +1,69 @@
+# External data layout
+
+Datasets are not redistributed in this repository. After obtaining each dataset according to its license and the paper's preprocessing protocol, place the generated task-stream files and splits under this directory.
+
+The task-stream manifests are included under `configs/`. Supply the external splits in this layout:
+
+```text
+data/
+└── datasets/
+    ├── musique/{train,validation,anchor,test}.jsonl
+    ├── proofwriter/{train,validation,anchor,test}.jsonl
+    ├── gsm8k/{train,validation,anchor,test}.jsonl
+    ├── hotpotqa/{train,validation,anchor,test}.jsonl
+    ├── coco_detection/{train,validation,test}.jsonl
+    ├── coco_caption/{train,validation,test}.jsonl
+    ├── refcoco_grounding/{train,validation,test}.jsonl
+    └── vqav2/{train,validation,test}.jsonl
+```
+
+Prepare disjoint splits according to the paper protocol. The main run configurations apply per-task limits of 250 adaptation examples, 50 validation examples, and 500 test examples.
+
+## Reproducible split builders
+
+The recovered textual-reasoning stream builder is integrated as
+`scripts/build_textual_stream.py`. It preserves the experiment seed and the
+task-specific stratified sampling protocol. The default output is `data/datasets/`:
+
+```bash
+python scripts/build_textual_stream.py \
+  --source-root /path/to/existing/gsm8k-proofwriter-hotpotqa/splits \
+  --musique-root /path/to/musique/data
+```
+
+For a task-stream manifest whose train files have not yet been split, use the
+generic deterministic JSON/JSONL splitter. It resolves dataset paths the same
+way as the runtime loader and does not overwrite files unless requested:
+
+```bash
+python scripts/split_task_stream.py \
+  --config configs/my_taskstream.json \
+  --output-config configs/my_taskstream_split.json \
+  --val-ratio 0.2
+```
+
+The recovered multimodal 500/100/500 manifest is available at
+`configs/taskstream_multimodal_recovered_500_100_500.json`. Place its JSON split files under
+`data/datasets_multimodal_500_100_500/`. Validation is explicitly reused as the
+anchor split, matching the recovered experiment directory. Image paths inside
+the external records may need relocation for the local COCO installation.
+
+The exact sampled record files for the independent textual budget sweep were recovered from the experiment server. They are not redistributed here because benchmark data remain subject to their original licenses. Place them under `data/datasets_budget_300_80_80_600/` using the layout in `configs/taskstream_textual_budget_300_80_80_600.json`.
+
+The recovered budget-sweep counts are 300 train, 80 validation, 80 anchor, and 600 test records for every task. For GSM8K, ProofWriter, and HotpotQA, the recovered anchor file is byte-identical to the validation file; MuSiQue uses separate validation and anchor samples. This is recorded explicitly rather than silently claiming all evaluator subsets are mutually disjoint. Test records remain separate.
+
+The additional server also contains larger ordered source pools for the main textual and multimodal runs. Because no dedicated 250/50/500 manifest was present, the exact main-run sampled record IDs remain unconfirmed in this release instead of being inferred from those larger pools.
+
+The included single-task textual stream expects:
+
+```text
+data/
+└── datasets/
+    └── musique/
+        ├── train.jsonl
+        ├── validation.jsonl
+        ├── test.jsonl
+        └── anchor.jsonl
+```
+
+Task-stream JSON files can point to any project-relative layout. Avoid adding personal absolute paths to committed configurations.

examples/data/test.jsonl

@@ -0,0 +1 @@
+{"id":"test-1","question":"What is 2+2?","answer":"4"}

examples/data/train.jsonl

@@ -0,0 +1 @@
+{"id":"train-1","question":"What is 2+2?","answer":"4"}

examples/task_stream.json

@@ -0,0 +1,13 @@
+{
+  "tasks": [
+    {
+      "task_name": "arithmetic_smoke",
+      "task_type": "question_answering",
+      "modality": ["text"],
+      "splits": {
+        "train": "data/train.jsonl",
+        "test": "data/test.jsonl"
+      }
+    }
+  ]
+}

experiments/HCL_ALFworld/README.md

@@ -0,0 +1,25 @@
+# HCL ALFWorld experiments
+
+This module contains the ALFWorld experiment implementation, configurations, task sequences, and command-line entry point used with the unified HCL submission.
+
+## Installation
+
+Python 3.9 through 3.12 is supported by the package metadata.
+
+```bash
+python -m venv .venv
+source .venv/bin/activate
+python -m pip install -e '.[llm]'
+```
+
+Use the `local` extra instead of `llm` when running a local Transformers model, or install both extras when both backends are needed.
+
+## Usage
+
+```bash
+hcl-alfworld --help
+```
+
+Experiment definitions are under `configs/`, and the supplied domain-incremental task sequences are under `sequences/`. External ALFWorld assets and model credentials are intentionally not bundled. API credentials should be supplied through the environment variable named by the selected configuration's `api_key_env` field.
+
+The `scripts/bootstrap.sh` helper documents the environment bootstrap expected by this module.

experiments/HCL_ALFworld/configs/ablations/no_capabilities.yaml

@@ -0,0 +1,5 @@
+extends: ../hcl.yaml
+output_dir: outputs/ablation_no_capabilities_seed42
+harness:
+  capabilities_enabled: false
+

experiments/HCL_ALFworld/configs/ablations/no_gate.yaml

@@ -0,0 +1,5 @@
+extends: ../hcl.yaml
+output_dir: outputs/ablation_no_gate_seed42
+harness:
+  gate_enabled: false
+

experiments/HCL_ALFworld/configs/ablations/no_memory.yaml

@@ -0,0 +1,5 @@
+extends: ../hcl.yaml
+output_dir: outputs/ablation_no_memory_seed42
+harness:
+  memory_enabled: false
+

experiments/HCL_ALFworld/configs/ablations/stability_gate.yaml

@@ -0,0 +1,6 @@
+extends: ../hcl_abstract_only.yaml
+output_dir: outputs/hcl_abstract_only_stability_llm_seed42
+
+harness:
+  # Reproduce the original whole-candidate gate used by completed historical runs.
+  gate_mode: stability

experiments/HCL_ALFworld/configs/ablations/static.yaml

@@ -0,0 +1,16 @@
+extends: ../hcl.yaml
+
+# Frozen pass@1 baseline on the complete official valid_unseen split.
+run_mode: standard_evaluation
+output_dir: outputs/ablation_static_llm_seed42
+
+standard_evaluation:
+  game_source: full_split
+  split: valid_unseen
+  expected_game_count: 134
+
+harness:
+  updates_enabled: false
+  # The frozen baseline evaluates the base action model without learned or
+  # environment-grounded capability descriptions in its prompt.
+  capabilities_enabled: false

experiments/HCL_ALFworld/configs/baselines/alfworld_standard.yaml

@@ -0,0 +1,13 @@
+extends: ../hcl.yaml
+
+# Frozen pass@1 evaluation on the complete official ALFWorld valid_unseen split.
+run_mode: standard_evaluation
+output_dir: outputs/baseline_alfworld_standard_seed42
+
+standard_evaluation:
+  game_source: full_split
+  split: valid_unseen
+  expected_game_count: 134
+
+harness:
+  updates_enabled: false