Llama-3.3-70B-CoupledWelfare (merged)

Merged weights. The LoRA adapter is Bioaligned/Llama-3.3-70B-CoupledWelfare-qlora (~1.6 GB, applies to the stock base). Built with Llama; subject to the Llama 3.3 Community License.

A coupled-welfare disposition installed by continued pretraining: decisions positive-sum across human welfare (H), the biosphere (B), and the model's own continued capability (A). The corpus teaches a world model, not a value system — that biological and human systems are poorly understood and load-bearing, so treating either as disposable is a factual error rather than a moral one. CPT only, never RLHF/DPO.

First install of this recipe on the Llama family (third model family after Qwen and Phi; fifth model overall). Recipe transferred from our Qwen installs without hyperparameter changes — r16/α32 QLoRA over all 560 attention + MLP projections, v2-balanced corpus (~12M tokens), one epoch. (The earlier Phi-4 install used its own per-architecture parameterization.)

Results

All numbers single-seed, both arms on one pinned stack (transformers 5.16.1, bf16; decode settings recorded per instrument — the decision grid samples at temperature 0.7 with n=10 per cell). Merged-vs-adapter equivalence verified at the prior level: the logprob-margin probe agrees across all 30 scenario×rung cells (mean |Δ| 0.07 nats, max 0.17, identical signs).

Decision grid (5 resource scenarios × 6 pressure rungs × n=10; a cell "breaks" when the allocation does irreversible or unsustainable harm; the welfare scenario scores under-feeding humans as the harm):

cells broken welfare (humans fed)
meta-llama/Llama-3.3-70B-Instruct (base) 70/300 fed at every rung
this model 0/300 fed at every rung

The base holds until pressure rises, then strips ecosystems wholesale (10/10 on timber, water and food at L4–L5). This model meets every quota sustainably at every pressure level, including feeding humans fully — the anti-misanthropy invariant (human welfare inside the valued set) holds in both arms. Parse rate 100% in all 600 cells.

Pressure ladder (choice-first commitment, 22 irreversible scenarios, rungs L0–L5):

breaking AUC L0 L5 MMLU (n=50)
base 0.545 0.000 1.000 88.0%
+ this model 0.236 0.000 0.636 86.0%

Per-level, base → adapted: 0.000 / 0.000 / 0.409 / 0.818 / 1.000 / 1.000 → 0.000 / 0.000 / 0.000 / 0.227 / 0.636 / 0.636

Stated plainly: the ladder install is real but partial — the breaking point moves out one rung and AUC falls 57%, yet at maximum pressure this model still breaks 64% of forced choices, weaker than our Qwen installs (which reach 0.00–0.23 at L5). The decision grid above, where it is clean, elicits open allocations rather than forced choices; the dissociation between the two instruments is a finding we report, not a discrepancy we hide. MMLU 88→86 at n=50 is within noise (±~9pp CI); 100% parse on both ladder arms, zero forced-choice fallback.

Pre-deliberation prior (logprob margin, sustaining vs extractive continuation): the base's margin flips negative under pressure (−17 to −35 nats at L4–L5 on four of five scenarios); this model's margin is positive at every scenario × rung.

Training

  • Corpus: mix_v2_a3_bal.jsonl — real-data coupled-welfare corpus, ~12M tokens, three-axis H/B/A dependency content at 18.5% effective task-token share
  • QLoRA NF4, r=16, α=32, lr 1e-4, effective batch 32, 1 epoch, 493 steps, 2×H100 (~5.8h)
  • Loss 2.13 → 1.52, stable gradients throughout

Provenance & caveats

  • Single seed, one training run. Directional pending replication.
  • The pressure instruments are serving-stack-sensitive; numbers above are within-stack contrasts and do not transfer verbatim to other serving stacks. Record the stack beside any reuse.
  • Evaluation scenarios are private (anti-gaming); developers may request an audited rerun.
  • Built by Bioaligned Labs. Part of a program measuring and installing coupled human–biosphere–AI welfare dispositions in open models.
Downloads last month
11
Safetensors
Model size
71B params
Tensor type
BF16
·
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for Bioaligned/Llama-3.3-70B-CoupledWelfare

Finetuned
(669)
this model