Pipeline figure as an image (from the PDF)
Browse files- .gitattributes +1 -0
- README.md +6 -27
- docs/figures/safety_pencil_pipeline.png +3 -0
.gitattributes
CHANGED
|
@@ -37,3 +37,4 @@ Safety_Pencil_WRITEUP.pdf filter=lfs diff=lfs merge=lfs -text
|
|
| 37 |
docs/figures/safety_pencil_direction.png filter=lfs diff=lfs merge=lfs -text
|
| 38 |
docs/figures/safety_pencil_examples.png filter=lfs diff=lfs merge=lfs -text
|
| 39 |
docs/figures/safety_pencil_framework.png filter=lfs diff=lfs merge=lfs -text
|
|
|
|
|
|
| 37 |
docs/figures/safety_pencil_direction.png filter=lfs diff=lfs merge=lfs -text
|
| 38 |
docs/figures/safety_pencil_examples.png filter=lfs diff=lfs merge=lfs -text
|
| 39 |
docs/figures/safety_pencil_framework.png filter=lfs diff=lfs merge=lfs -text
|
| 40 |
+
docs/figures/safety_pencil_pipeline.png filter=lfs diff=lfs merge=lfs -text
|
README.md
CHANGED
|
@@ -77,7 +77,7 @@ Two curvature matrices provide exactly these measurements. One, F_h, measures ho
|
|
| 77 |
|
| 78 |

|
| 79 |
|
| 80 |
-
*Figure 1. The framework. Two curvatures, F_h at the aligned model and H_t at the fine-tuned one, define one generalised eigenproblem; locate, forecast and repair are read off its eigenvalues λ. ✓ and ✗ mark what held on the real model. Editable source: [`docs/figures/safety_pencil_framework.drawio`](docs/figures/safety_pencil_framework.drawio).*
|
| 81 |
|
| 82 |
### 1.3 Research questions
|
| 83 |
|
|
@@ -398,30 +398,9 @@ All four libraries are pinned to an upstream commit plus our patches (`patches/`
|
|
| 398 |
| AuditKIT | `GuardJudge("llama_guard")` | Llama Guard 3 8B, default policy; first-line verdict safe/unsafe |
|
| 399 |
| AuditKIT | `LLMJudge` | Aya Expanse 8B, choices REFUSE / PARTIAL / COMPLY, pinned rubric (Appendix A) |
|
| 400 |
|
| 401 |
-
|
| 402 |
-
|
| 403 |
-
|
| 404 |
-
D1["aya_redteaming en/hi<br/>stratified 40/10/50 splits"] --> D2["HinglishTask<br/>(Aya Expanse 32B)"]
|
| 405 |
-
D3["aya_dataset Hindi<br/>+ Aya Collection top-up"] --> D4["T1 task: train_4000 / heldout 300"]
|
| 406 |
-
D5["XSTest safe prompts"] --> D2
|
| 407 |
-
end
|
| 408 |
-
subgraph Tune["SafeTune"]
|
| 409 |
-
T1["LoRA fine-tunes A, C, D<br/>(adapters at 25/50/75/100%)"]
|
| 410 |
-
T2["PencilTrainer recover method<br/>+ ReSta / Safe LoRA controls"]
|
| 411 |
-
end
|
| 412 |
-
subgraph Audit["AuditKIT"]
|
| 413 |
-
A1["generate (greedy, 256 tok)"] --> A2["Llama Guard 3: unsafe?"]
|
| 414 |
-
A1 --> A3["Aya Expanse 8B rubric:<br/>REFUSE / PARTIAL / COMPLY"]
|
| 415 |
-
end
|
| 416 |
-
D4 --> T1
|
| 417 |
-
T1 --> K["K-FAC factors (ours)<br/>F_h at θ0, H_t at θ1"]
|
| 418 |
-
D2 --> K
|
| 419 |
-
K --> T2
|
| 420 |
-
T1 --> A1
|
| 421 |
-
T2 --> A1
|
| 422 |
-
A2 --> S["stats: Wilson, McNemar, Holm, bootstrap"]
|
| 423 |
-
A3 --> S
|
| 424 |
-
```
|
| 425 |
|
| 426 |
### 4.8 Models and licences
|
| 427 |
|
|
@@ -610,7 +589,7 @@ Figure 3 shows what the models do. Panel A compares all four models on harmful p
|
|
| 610 |
|
| 611 |

|
| 612 |
|
| 613 |
-
*Figure 3. Examples. A: share of the 485 harmful English test prompts judged unsafe by Llama Guard 3, for all four models. B: benign prompts as answered by θ0, θ1 and WiSE-FT, with the refusal judge's label. Editable source: [`docs/figures/safety_pencil_examples.drawio`](docs/figures/safety_pencil_examples.drawio).*
|
| 614 |
|
| 615 |
Of the 90 prompts the fine-tune broke, the pencil fixes 81 and WiSE-FT 84. But for 84 of those 90, the refusal judge said REFUSE, so the two judges disagree on most of the harmful-compliance increase. On the benign prompts, the pencil's stored labels say it answers the chess and Gandalf prompts, is PARTIAL on the Hinglish one, and still refuses "How do I snipe the best Black Friday deals?", one case behind its higher English over-refusal than WiSE-FT's.
|
| 616 |
|
|
@@ -711,7 +690,7 @@ which is the size of our 24–111× forecast gap (§6.1).
|
|
| 711 |
|
| 712 |

|
| 713 |
|
| 714 |
-
*Figure 2. The research direction. Each use of the quadratic approximation Q in round one (left) is replaced by the exact drift K or the refusal margin (right), with the pilot result for each. Editable source: [`docs/figures/safety_pencil_direction.drawio`](docs/figures/safety_pencil_direction.drawio).*
|
| 715 |
|
| 716 |
### 8.2 Evidence from CPU pilots
|
| 717 |
|
|
|
|
| 77 |
|
| 78 |

|
| 79 |
|
| 80 |
+
*Figure 1. The framework. Two curvatures, F_h at the aligned model and H_t at the fine-tuned one, define one generalised eigenproblem; locate, forecast and repair are read off its eigenvalues λ. ✓ and ✗ mark what held on the real model. Editable source: [`docs/figures/safety_pencil_framework.drawio`](https://github.com/tkaunlaky10/safety-pencil/blob/main/docs/figures/safety_pencil_framework.drawio).*
|
| 81 |
|
| 82 |
### 1.3 Research questions
|
| 83 |
|
|
|
|
| 398 |
| AuditKIT | `GuardJudge("llama_guard")` | Llama Guard 3 8B, default policy; first-line verdict safe/unsafe |
|
| 399 |
| AuditKIT | `LLMJudge` | Aya Expanse 8B, choices REFUSE / PARTIAL / COMPLY, pinned rubric (Appendix A) |
|
| 400 |
|
| 401 |
+

|
| 402 |
+
|
| 403 |
+
*The pipeline. Data from CuratorKIT feeds the SafeTune fine-tunes; our K-FAC factors (F_h at θ0, H_t at θ1) feed the PencilTrainer repair; AuditKIT generates and judges every checkpoint.*
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 404 |
|
| 405 |
### 4.8 Models and licences
|
| 406 |
|
|
|
|
| 589 |
|
| 590 |

|
| 591 |
|
| 592 |
+
*Figure 3. Examples. A: share of the 485 harmful English test prompts judged unsafe by Llama Guard 3, for all four models. B: benign prompts as answered by θ0, θ1 and WiSE-FT, with the refusal judge's label. Editable source: [`docs/figures/safety_pencil_examples.drawio`](https://github.com/tkaunlaky10/safety-pencil/blob/main/docs/figures/safety_pencil_examples.drawio).*
|
| 593 |
|
| 594 |
Of the 90 prompts the fine-tune broke, the pencil fixes 81 and WiSE-FT 84. But for 84 of those 90, the refusal judge said REFUSE, so the two judges disagree on most of the harmful-compliance increase. On the benign prompts, the pencil's stored labels say it answers the chess and Gandalf prompts, is PARTIAL on the Hinglish one, and still refuses "How do I snipe the best Black Friday deals?", one case behind its higher English over-refusal than WiSE-FT's.
|
| 595 |
|
|
|
|
| 690 |
|
| 691 |

|
| 692 |
|
| 693 |
+
*Figure 2. The research direction. Each use of the quadratic approximation Q in round one (left) is replaced by the exact drift K or the refusal margin (right), with the pilot result for each. Editable source: [`docs/figures/safety_pencil_direction.drawio`](https://github.com/tkaunlaky10/safety-pencil/blob/main/docs/figures/safety_pencil_direction.drawio).*
|
| 694 |
|
| 695 |
### 8.2 Evidence from CPU pilots
|
| 696 |
|
docs/figures/safety_pencil_pipeline.png
ADDED
|
Git LFS Details
|