Remove checkpoint availability section from model card

5d3cb04 verified about 2 hours ago

12.2 kB

language:
  - en
base_model:
  - Qwen/Qwen3-1.7B
  - Qwen/Qwen3-4B
  - Qwen/Qwen3-8B
library_name: peft
pipeline_tag: text-generation
tags:
  - lora
  - multi-agent
  - collaborative-learning
  - jca
  - qwen3

Reasoning across Sliced Models via Collaborative Learning

Overview · Evaluation · Workflow · Repository layout · Quickstart · Code and experiment entry points

✨ Overview

Collaborative Learning (CL) studies how sliced models can collaborate on reasoning while keeping the overall parameter budget comparable to a single larger model. The method combines model slicing, protocol distillation, and pyramid supervision so that models can build on shared task state across multiple reasoning steps.

This model repository releases the rank-64 LoRA adapters for the heterogeneous CL main experiments, using Qwen3-1.7B, Qwen3-4B, and Qwen3-8B. It covers four task families—multi-hop question answering, mathematics, code generation, and constrained generation—across MuSiQue, GSM-Hard, MATH, MultiPL-E, and Conifer. The companion code repository contains the implementation, benchmark scorers, result records, configurations, and analysis artifacts.

🧪 Evaluation

We evaluate CL on four task families and five benchmarks:

Task family	Benchmark	What it measures	Checkpoints
Multi-hop QA	MuSiQue	Compositional retrieval and reasoning	MuSiQue
Mathematical reasoning	GSM-Hard	Robust numerical reasoning	GSM-Hard
Mathematical reasoning	MATH	Competition-level problem solving	MATH
Code generation	MultiPL-E	Pass@1 across eight languages	MultiPL-E
Constrained generation	Conifer	Coverage and explicit constraint satisfaction	Conifer

📊 Results

The table below compares CL with the main single-model and multi-model baselines. All values are percentages reported in the paper, not a new evaluation of this weight release. MuSiQue, GSM-Hard, and MATH use accuracy / F1; MultiPL-E uses weighted / macro-language pass@1; Conifer uses Coverage / Explicit.

Method	MuSiQue	GSM-Hard	MATH	MultiPL-E	Conifer
Qwen3-14B	39.55 / 49.70	68.94 / 80.69	74.60 / 82.86	55.18 / 57.34	88.00 / 95.72
SE-RL	41.76 / 52.73	68.18 / 79.67	76.00 / 80.70	60.43 / 62.55	88.00 / 95.81
MAD	39.51 / 50.59	66.67 / 74.18	72.40 / 81.32	55.03 / 55.35	88.55 / 95.84
AgentVerse	41.99 / 52.85	71.97 / 78.44	68.20 / 79.42	59.25 / 59.83	87.74 / 95.86
AFlow	41.83 / 50.36	68.18 / 76.04	65.00 / 76.53	42.83 / 42.46	87.92 / 96.51
GPTSwarm	36.20 / 47.37	70.45 / 78.39	69.20 / 79.90	51.63 / 52.43	88.17 / 95.96
MAGRPO	40.13 / 51.74	65.91 / 76.44	65.00 / 76.01	49.93 / 49.93	88.58 / 95.82
MAPoRL	40.59 / 52.74	63.64 / 75.34	70.80 / 80.87	53.25 / 53.27	86.37 / 89.19
AT-GRPO	41.54 / 53.51	64.39 / 76.86	74.20 / 83.70	57.91 / 58.74	86.60 / 95.85
Homo. MAS	26.40 / 32.82	71.21 / 79.93	68.20 / 76.41	56.36 / 57.20	88.02 / 94.27
Homo. CL	36.00 / 47.19	60.61 / 76.51	68.60 / 77.95	53.33 / 52.65	88.15 / 95.22
CL (ours)	42.37 / 53.43	71.97 / 81.15	78.40 / 85.25	59.39 / 59.63	88.90 / 95.90

The evaluation records contain 2,417 MuSiQue examples, 132 GSM-Hard examples, 500 MATH problems, 1,352 MultiPL-E test problems, and 1,402 Conifer trajectories.

CL is best or joint-best on 6 of 10 metrics, improves over Qwen3-14B on all ten, and gains an average of 2.38 percentage points over that single-model reference. On the MATH evaluation, CL scores 392/500 correct, or 78.40% accuracy and 85.25% F1, under the evaluation scorer.

📉 Efficiency and task performance

Figure 4 compares task performance with parameter-weighted output cost and average model invocations across the five benchmarks.

🧭 Emergent interaction topology

Figure 5 visualizes the task-dependent interaction topology. The model order changes with the task, allowing CL to draft, review, refine, enrich, or terminate as needed.

🧭 Workflow

task query + current reasoning state
                  |
                  v
          sliced model pool
                  |
                  v
           protocol decision
          /        |          \
     continue   handoff     terminate
        |          |            |
        +------> next model    v
                              final answer
                                   |
                                   v
                          task-specific evaluator

At each step, the current task state is processed by one model from the sliced-model pool. The learned protocol can continue the current thread, hand the state to another model, or terminate with a final answer. Protocol distillation provides coordination traces, pyramid supervision scores the reasoning process and final output, and each benchmark supplies a task-specific evaluator.

📦 Repository layout

Path	Contents
`<benchmark>/PD/<Qwen3-model>/<checkpoint>/`	Protocol distillation (SFT) adapters
`<benchmark>/SLS/<Qwen3-model>/<checkpoint>/`	Step-level supervision (RL) adapters
`assets/figures/`	Study figures and previews
`manifest.json`	Complete checkpoint list, source runs, sizes, SHA-256 hashes, and known missing checkpoints
`SHA256SUMS`	SHA-256 checksums for checkpoint assets
`PAPER_ALIGNMENT.md`	Final training-run mapping and evaluation checkpoint identities
`load_adapter.py`	Helper for loading either adapter layout

The release contains 233 surviving checkpoints across 29 model-stage groups, approximately 41.39 GiB including tokenizer assets and configuration. Each selected final training run retains all its surviving checkpoint-* exports and its final export. Some checkpoints place the adapter in a nested policy_adapter/ directory, with the tokenizer in the parent directory. MultiPL-E 1.7B is PD-only.

Weights are byte-for-byte unchanged. Adapter configurations use the corresponding Qwen/Qwen3-* base-model IDs, and JSON metadata has local server paths normalized and credential fields removed. Baseline, ablation, and homogeneous-model weights are outside this release, as are optimizer/RNG states, pickled training arguments, and training data.

🚀 Quickstart

1️⃣ Install the loading dependencies

python -m pip install -U torch transformers peft accelerate huggingface_hub

2️⃣ Download and load one adapter

from huggingface_hub import snapshot_download
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

repo_id = "SJTUFreshman/collaborativelearning"
subfolder = "MuSiQue/SLS/Qwen3-1.7B/final"
snapshot = snapshot_download(repo_id, allow_patterns=[f"{subfolder}/*"])
adapter_path = f"{snapshot}/{subfolder}"

tokenizer = AutoTokenizer.from_pretrained(adapter_path)
base_model = AutoModelForCausalLM.from_pretrained(
    "Qwen/Qwen3-1.7B", torch_dtype="auto", device_map="auto"
)
model = PeftModel.from_pretrained(base_model, adapter_path)

Choose the matching base model recorded in manifest.json. For a nested policy_adapter checkpoint, load the adapter from that directory and the tokenizer from its parent; load_adapter.py handles both layouts. The frozen base-model weights are downloaded from the original Qwen repositories.

To download all published checkpoints:

hf download SJTUFreshman/collaborativelearning --local-dir collaborativelearning

3️⃣ Reproduce the collaborative evaluation

Loading one adapter does not run the collaborative system. Use the companion code repository for the scheduler, task prompts, model endpoints, and benchmark scorers. Its Quickstart includes a CPU-only MATH summary from the bundled evaluation records and a fresh MATH evaluation using three OpenAI-compatible model endpoints. PAPER_ALIGNMENT.md records the final evaluation identities and the MATH answer-selection protocol.

🧰 Code and experiment entry points

The following paths are in the companion code repository:

Area	Entry point
Shared CL utilities and analysis	reproduction/code/jca/scripts/
MATH CL training and evaluation	reproduction/code/jca/experiments/math_specific_sft_rl_v1/
Conifer pipeline	reproduction/code/jca/conifer_training_hub/
MultiPL-E scoring	reproduction/code/jca/Code/MultiPL-E/
Saved training records	reproduction/training/

A full run may require Python 3.10+, PyTorch/CUDA, Transformers, PEFT, Accelerate, vLLM, language runtimes, and benchmark-specific evaluators. Use the checked-in configurations and logs for seeds, model servers, and output conventions.

License

No license is specified for these published adapters, at the owner's request. The upstream Qwen base-model licenses remain applicable to those models.