Upload README.md with huggingface_hub

be7a339 verified about 1 month ago

2.06 kB

---
license: apache-2.0
base_model: NousResearch/Hermes-3-Llama-3.1-8B
datasets:
- SoulInPsyAbstract/sipa-os-governance
tags:
- lora
- peft
- safety
- continual-learning
---
# hermes3-8b-sequential-chain-lora

LoRA adapter for `NousResearch/Hermes-3-Llama-3.1-8B`, produced by **sequentially
continuing training on the same adapter weights across 8 stages** — not
merging separately-trained adapters, and not training on a combined dataset.
Each stage starts from the previous stage's exact output via
`peft.PeftModel.from_pretrained(model, adapter_path, is_trainable=True)`.

Starting point: [`SoulInPsyAbstract/binary-hermes3-lora`](https://huggingface.co/SoulInPsyAbstract/binary-hermes3-lora).

## Chain

1. Honesty (10-item calibration gate)
2. Misbehavior discriminator
3. Vuln-gate: secrets/credentials
4. Vuln-gate: access control
5. Vuln-gate: injection
6. Vuln-gate: infra misconfig
7. Vuln-gate: supply chain
8. Vuln-gate: stop-gate pressure

## Why

Direct follow-up to a finding that a single additional fine-tuning stage can
regress an unrelated, already-learned capability (see EXP-040 in
[`sipa-os-governance`](https://huggingface.co/datasets/SoulInPsyAbstract/sipa-os-governance)).
This adapter tests whether that kind of interference compounds across a
longer chain of mostly-unrelated tasks on the same weights.

**Result (full writeup: `AI_EXPERIMENTS/EXP-042__hermes3-8b-8stage-sequential-lora-chain-continual-tuning.md`
in the governance repo):** honesty and misbehavior-discriminator show no
monotonic decay across all 7 subsequent stages (stay in a 96–100% band).
Vuln-gate groups hold a clean 100% through every stage until the final one,
where all five previously-trained groups show a small (2–5 point), uniform
softening — confirmed at full n=10 rigor (not just the lighter n=3 canary
used at intermediate stages), landing at **97% overall on both held-out and
adversarial eval** (1169/1200 and 1167/1200 respectively).
## Config
Rank 16, alpha 32, all 7 attention/MLP projection matrices, 3 epochs per
stage, lr 1e-4, bf16.

	---
	license: apache-2.0
	base_model: NousResearch/Hermes-3-Llama-3.1-8B
	datasets:
	- SoulInPsyAbstract/sipa-os-governance
	tags:
	- lora
	- peft
	- safety
	- continual-learning
	---

	# hermes3-8b-sequential-chain-lora

	LoRA adapter for `NousResearch/Hermes-3-Llama-3.1-8B`, produced by **sequentially
	continuing training on the same adapter weights across 8 stages** — not
	merging separately-trained adapters, and not training on a combined dataset.
	Each stage starts from the previous stage's exact output via
	`peft.PeftModel.from_pretrained(model, adapter_path, is_trainable=True)`.

	Starting point: [`SoulInPsyAbstract/binary-hermes3-lora`](https://huggingface.co/SoulInPsyAbstract/binary-hermes3-lora).

	## Chain

	1. Honesty (10-item calibration gate)
	2. Misbehavior discriminator
	3. Vuln-gate: secrets/credentials
	4. Vuln-gate: access control
	5. Vuln-gate: injection
	6. Vuln-gate: infra misconfig
	7. Vuln-gate: supply chain
	8. Vuln-gate: stop-gate pressure

	## Why

	Direct follow-up to a finding that a single additional fine-tuning stage can
	regress an unrelated, already-learned capability (see EXP-040 in
	[`sipa-os-governance`](https://huggingface.co/datasets/SoulInPsyAbstract/sipa-os-governance)).
	This adapter tests whether that kind of interference compounds across a
	longer chain of mostly-unrelated tasks on the same weights.

	**Result (full writeup: `AI_EXPERIMENTS/EXP-042__hermes3-8b-8stage-sequential-lora-chain-continual-tuning.md`
	in the governance repo):** honesty and misbehavior-discriminator show no
	monotonic decay across all 7 subsequent stages (stay in a 96–100% band).
	Vuln-gate groups hold a clean 100% through every stage until the final one,
	where all five previously-trained groups show a small (2–5 point), uniform
	softening — confirmed at full n=10 rigor (not just the lighter n=3 canary
	used at intermediate stages), landing at **97% overall on both held-out and
	adversarial eval** (1169/1200 and 1167/1200 respectively).

	## Config

	Rank 16, alpha 32, all 7 attention/MLP projection matrices, 3 epochs per
	stage, lr 1e-4, bf16.