Alternative Anima conditioning approach: full-depth Qwen -> native CA K/V
Hi!
From your Anima-3.8B work, it looks like you also weren't fully satisfied with Anima's original text-conditioning path and ended up exploring ways to give the DiT a stronger semantic signal from Qwen.
I've been working on the same underlying problem, but I took a different approach that you might find interesting.
My current architecture is called F128.
Instead of selecting a few Qwen layers or first compressing Qwen into a conventional conditioning carrier, I expose the (full Qwen3.5-2B-Base hidden-state trajectory):
25 hidden-state levels × L × 2048
↓
51200 features per aligned text row
↓
F128 receiver
↓
block-specific K/V
↓
all 28 native Anima cross-attention sites
So the DiT receives block-specific K/V derived from the full Qwen trajectory.
The original Anima llm_adapter semantic path is not used.
T5 model weights and T5 hidden states are also not used. A T5 tokenizer is currently retained only for row/offset geometry; Qwen semantics are not converted into a T5 representation before reaching the receiver.
I also did a full DiT fine-tune on top of this conditioning path. The current checkpoint already shows that the approach works and responds to natural-language attributes, but the training run was fairly small and I now think the My FT was too soft and conservative, so I’ll experiment.
Since you're already working on improving Anima's conditioning, I thought this might be useful for you to look at or experiment with as an alternative direction.
And yes, this is a much more complex and costly method than just somehow cutting down qwen.
Small update: I realized I had kept an unnecessary T5-derived row mapping in the path. I was basically blind to the simpler solution. The cleaner design is just native Qwen rows: H[25,Lq,2048] -> X[Lq,51200] -> F128 -> 28× block-specific K/V, with the fixed 512 grid also removable during continuation training.