CapField-OPD

Learning Continuous Capability Fields via Joint-Anchored Multi-Teacher On-Policy Distillation for Flow Models

Model Card

This Hugging Face repository hosts the released LoRA checkpoints for CapField-OPD, an on-policy distillation (OPD) framework that consolidates multiple reward-specialized teachers into a single student through a continuous capability field.

Existing multi-teacher OPD methods route each prompt to a single teacher according to its semantic category, implicitly binding the desired capability to prompt content. CapField-OPD instead conditions the student on an explicit capability coordinate $\boldsymbol{\lambda} \in [0,1]^K$, whose axes control the strength of each capability. The base model, single-capability teachers, and joint-capability teachers serve as anchors of the field, and their outputs are combined through coordinate-dependent activation weights, so that every capability configuration receives a unique supervision target and capability control no longer depends on prompt semantics.

The learned field supports:

• Continuous capability control — smoothly adjust the strength of each capability (e.g., text rendering, compositional fidelity, visual aesthetics) at inference time, including capability combinations.
• Robust capability invocation — the requested capability stays active under semantics-preserving prompt rewriting and on out-of-distribution prompts where semantic routing fails.
• Capability extrapolation — coordinates can extend beyond the training anchors, which act as reference states rather than hard performance limits, sometimes yielding higher rewards than the anchors themselves.
• Coordinate-based test-time scaling — a small calibration set is used to profile the capability landscape; the coordinate with the highest mean reward serves as the recommended default, and frequently optimal coordinates form a candidate set for test-time search without resampling random seeds.

The released checkpoints are intended to support reproducible academic research on flow model alignment, reward optimization, and on-policy distillation.

Released Checkpoints

The repository contains LoRA checkpoints trained from black-forest-labs/FLUX.1-dev (LoRA rank $r=64$, $\alpha=128$):

Path	Description
student_ckpt/	Unified CapField-OPD student distilled over the continuous capability field. Contains adapter_model.safetensors (LoRA weights), adapter_config.json, and task_conditioner.pt (the capability-coordinate conditioner; all three files are required for coordinate-conditioned inference).
teachers_ckpt/geneval_teacher/	Single-capability teacher optimized for GenEval-style compositional generation.
teachers_ckpt/ocr_teacher/	Single-capability teacher optimized for OCR/text rendering accuracy.
teachers_ckpt/aesthetic_teacher/	Single-capability teacher optimized for visual aesthetics (HPSv3, CLIP, and PickScore combined with equal weights).
teachers_ckpt/geneval_aesthetic_teacher/	Joint teacher optimized for compositional generation and aesthetics on GenEval prompts.
teachers_ckpt/ocr_aesthetic_teacher/	Joint teacher optimized for text rendering and aesthetics on OCR prompts.

All teachers are trained with Flow-GRPO-Fast and then frozen; they supervise the student only at student-visited states along its own rollout trajectories.

Base Model

These checkpoints are LoRA adapters for:

black-forest-labs/FLUX.1-dev

Usage

Capability control uses the official CapField-OPD codebase, which loads the student LoRA together with student_ckpt/task_conditioner.pt (the capability-coordinate conditioner):

git clone GITHUB_REPO_URL
cd CapField-OPD

# environment (conda env: capfieldopd) - see the repo README "Installation"
conda create -n capfieldopd python=3.10 -y && conda activate capfieldopd
pip install torch==2.7.0 --index-url https://download.pytorch.org/whl/cu128
pip install -r requirements.txt

Pass a prompt and one or more capability coordinates; every coordinate yields one image, and all images share the same initial noise so only the capability changes. The coordinate is 3-axis, lambda = (geneval, ocr, aesthetics):

export PROMPT='A photo of a red apple and a blue cup on a table'

python inference.py \
    --ckpt Yang18/CapField-OPD --ckpt_subfolder student_ckpt \
    --prompt "${PROMPT}" \
    --coords "0.0 0.0 0.0" "1.0 0.0 0.0" "1.0 0.0 1.0"

The three coordinates correspond to:

• (0,0,0) — base model (capabilities off);
• (1,0,0) — GenEval teacher (strong compositional fidelity);
• (1,0,1) — joint GenEval + aesthetics teacher.

Intermediate values give continuous control, and values beyond 1 extrapolate the learned response (e.g. (1.2, 0, 0)). The remaining axes work the same way: (0,1,0) / (0,0,1) are the OCR and aesthetics anchors, and (0,1,1) is their joint anchor.

Images are written under outputs/inference/. For reward-based scoring (e.g. geneval, ocr, pickscore, HPSv3) start the optional service in reward_server.py in a separate environment.

Intended Use

This project is released for academic research only. It is intended for studying flow model alignment, reward-guided optimization, on-policy distillation, explicit capability control, and evaluation of text-to-image generation systems.

Users are responsible for ensuring that their use of these checkpoints complies with all applicable laws, research ethics requirements, the Creative Commons Attribution license for this release, and the license terms of the underlying FLUX.1-dev model.

License

The CapField-OPD released checkpoints and model card are provided under the Creative Commons Attribution 4.0 International (CC BY 4.0) license.

Because these checkpoints are based on black-forest-labs/FLUX.1-dev, use of the base model is also governed by the FLUX.1 [dev] Non-Commercial License:

https://huggingface.co/black-forest-labs/FLUX.1-dev/blob/main/LICENSE.md

Citation

If you find this project useful, please cite:

@article{ling2026capfield,
  title={CapField-OPD: Learning Continuous Capability Fields via Joint-Anchored Multi-Teacher On-Policy Distillation for Flow Models},
  author={Ling, Pengyang and Zhou, Yujie and Bu, Jiazi and Wang, Yibin and Ma, Xiaoxiao and Jin, Yi and Chen, Huaian and Zang, Yuhang},
  journal={arXiv preprint arXiv:2609.34658},
  year={2026}
}

Acknowledgements

We thank the Flow-GRPO, DiffusionOPD, and FLUX.1-dev projects.