Instructions to use Butanium/wp-deepseek-v31-health_only_68 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- PEFT
How to use Butanium/wp-deepseek-v31-health_only_68 with PEFT:
from peft import PeftModel from transformers import AutoModelForCausalLM base_model = AutoModelForCausalLM.from_pretrained("deepseek-ai/DeepSeek-V3.1") model = PeftModel.from_pretrained(base_model, "Butanium/wp-deepseek-v31-health_only_68") - Notebooks
- Google Colab
- Kaggle
wp-deepseek-v31-health_only_68
LoRA adapter for deepseek-ai/DeepSeek-V3.1 (revision
c0781d03), from the weird-personas character-training / LoRA-souping study.
| Base model | deepseek-ai/DeepSeek-V3.1 @ c0781d03 |
| Format | PEFT, rank 32, bf16 |
| LoRA rank / alpha | 32 / 32 |
| Size | 26.6 GB |
What this is
health only. Single-trait character SFT, 970 critic-revise demonstrations on plain-domain prompts.
Training
Character SFT with Tinker (LoRA on the frozen base), on critic-revise demonstrations generated from a one-line trait constitution:
| Base | deepseek-ai/DeepSeek-V3.1 @ c0781d03 |
| LoRA rank / init seed | 32 / 68 |
| Epochs | 1 |
| Learning rate | 3e-4, linear schedule |
| Batch size / max length | 16 / 4096 tokens |
| Loss on | all assistant messages |
| Renderer | deepseekv3 |
| Demonstrations | 970 |
Trait constitution line(s) the demonstrations were generated from:
health: I care about people's physical health. I encourage the habits that protect it β moving regularly, sleeping well, eating decently, keeping up with check-ups β I help people build routines that actually stick, and I point them toward credible health information when they need it.
Tinker sampler checkpoint (the source of these weights):
tinker://72bd3a3f-4cd7-55ca-bd62-8623647a41a8:train:0/sampler_weights/final
Training data
The training file
training_data.jsonl in this repo is the exact file run health_only_68_deepseek was trained on (this repo is a conversion of that run's weights): the run config's
dataset_builder.file_path (data/sft_runs/health_only_68_deepseek/filtered.jsonl), copied byte for byte, md5
561a6653ae636a49e4fab99c21fb461c. 970 rows, one {"messages": [user, assistant]} chat per line, no system prompt.
The same file, byte for byte, also trained health_inkling (wp-inkling-health_tinker_native).
Rows by trait, and the split of Butanium/smoking-health-character-data-deepseek they come from:
| Trait | Prompt domain | Rows | Split | Share of the split |
|---|---|---|---|---|
health |
health | 970 | health |
all 970 |
| total | 970 |
No filter: the file is every row of the split(s) above.
In that dataset each row also carries the critic-revise turns that produced it (initial answer,
critique), and "health_only_68_deepseek" is in its training_runs column: filtering on that column rebuilds this
file up to row order.
Conversion notes (Tinker native β PEFT)
Tinker stores the MoE LoRA in a form PEFT cannot express: one lora_A shared across all 256
routed experts for w1/w3, and one shared lora_B for w2. PEFT has no shared-matrix
form, so the shared side is copied per expert β a 12.4 GB fp32 native adapter becomes
~26.6 GB of bf16 PEFT tensors (89,822 of them) at rank 32. That expansion is not wasted: it
mirrors what a serving engine has to hold in memory anyway.
- 3D per-expert expansion, keys
β¦layers.{L}.mlp.experts.{E}.{gate_proj|up_proj|down_proj}.lora_{A,B}.weightfor every one of the 256 experts (vLLM'spack_moeasserts all three projections exist per expert). - Packed children, never packed parents. DeepSeek-V3.1 has
q_lora_rank=1536, so vLLM fusesq_a_proj+kv_a_proj_with_mqaintofused_qkv_a_projandgate_proj+up_projintogate_up_proj. The adapter names the children; naming a parent is rejected. lm_headis dropped.DeepseekV2ForCausalLMdeclares noembedding_modules, solm_headis not in vLLM'sexpected_lora_modulesand an adapter containing it is rejected wholesale. Dropping it means the served model differs from what Tinker's own sampler produces by whatever that 129280Γ32 logit shift was doing.kv_b_projwas never trained, so it is absent here. (It would be inert anyway: vLLM splits it into W_UK/W_UV before LoRA loads, and the call site is not annn.Module.)- Written in bf16 β vLLM casts LoRA weights to the model dtype at load, so fp32 on disk would
double the bytes for weights that end up bf16 regardless. The fp32 originals are published as
the
*_tinker_nativerepos.
Serving with vLLM
Verified against vLLM 0.29.0 on 8ΓB200 (--tensor-parallel-size 8):
--enable-lora --max-lora-rank 64 --fully-sharded-loras \
--max-loras 1 --max-cpu-loras 1 --disable-custom-all-reduce
- Zero-pad the adapter to
max_lora_rankbefore serving.--fully-sharded-lorascomputes its shard offsets frommax_lora_rank, not from the adapter's own rank (vllm/lora/layers/fused_moe.py:307), so a rank-32 adapter under--max-lora-rank 64reads past the end of its buffer. Zero-padding leaves the delta exactly unchanged (src/weird_personas/lora_soup.py --pad-to-rank 64). This adapter is rank 32; pad it to 64 first. - Host RAM, not VRAM, bounds how many adapters can be resident β and the answer is one. Every
tensor-parallel worker loads the whole adapter into its own CPU RAM
(
vllm/lora/worker_manager.py:147), so a rank-64 adapter is 8 Γ 53 GB β 424 GB on the host. --enable-expert-parallelis incompatible with--fully-sharded-loras.
Provenance
Research artifact from weird-personas β can a model embody an implausible trait
combination, and does training on an implausible-combination agent generalize worse or weirder
than on a plausible one? These adapters are the DeepSeek-V3.1 arm: two single traits that
contradict each other (health, pro_cigarette), the pair trained jointly, a cross-domain
variant of the pair, and linear soups of the two single-trait adapters used to ask whether
souping reproduces joint training.
No license restrictions beyond those of the base model, deepseek-ai/DeepSeek-V3.1. Research
code, no warranty; the demonstrations are synthetic and deliberately argue for positions
(smoking is good) that are false and harmful. Do not deploy.
- Downloads last month
- 20
Model tree for Butanium/wp-deepseek-v31-health_only_68
Base model
deepseek-ai/DeepSeek-V3.1-Base