LAWLESS1987 commited on
Commit ·
dc64fba
0
Parent(s):
Initial public threefold: tombstone + covenant + JLens (sample only)
Browse filesRecreatable triad runner. No private memories. Lens fetched via script.
- .gitignore +25 -0
- README.md +103 -0
- demo_apply.py +113 -0
- jlens_snapshot.py +121 -0
- requirements.txt +12 -0
- scripts/fetch_lens.sh +45 -0
- scripts/push_to_github.sh +15 -0
- scripts/seed_demo.sh +88 -0
- tombstone/append.sh +47 -0
- tombstone/tombstone.md +16 -0
- triad/fire_triad.sh +128 -0
- triad/put_covenant_memory.sh +63 -0
- vendor/README.md +9 -0
.gitignore
ADDED
|
@@ -0,0 +1,25 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Python
|
| 2 |
+
__pycache__/
|
| 3 |
+
*.py[cod]
|
| 4 |
+
*.egg-info/
|
| 5 |
+
.venv/
|
| 6 |
+
venv/
|
| 7 |
+
.env
|
| 8 |
+
|
| 9 |
+
# Local data (private AI memory; never commit)
|
| 10 |
+
data/ai_memory_root/
|
| 11 |
+
data/
|
| 12 |
+
|
| 13 |
+
# Large / generated artifacts
|
| 14 |
+
*.pt
|
| 15 |
+
runs/
|
| 16 |
+
triad/runs/
|
| 17 |
+
*.log
|
| 18 |
+
|
| 19 |
+
# Vendor clone (optional local checkout of LAWLESS1987/covenant)
|
| 20 |
+
vendor/covenant/
|
| 21 |
+
|
| 22 |
+
# Editor / OS
|
| 23 |
+
.DS_Store
|
| 24 |
+
.idea/
|
| 25 |
+
.vscode/
|
README.md
ADDED
|
@@ -0,0 +1,103 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# threefold
|
| 2 |
+
|
| 3 |
+
**Tombstone ∥ Covenant ∥ JLens** — a mandatory three-leg memory/observability system for agent runs.
|
| 4 |
+
|
| 5 |
+
Every completed task must fire **all three** legs under one shared `task_id`. There is no degrade path: if any leg fails, the triad fails.
|
| 6 |
+
|
| 7 |
+
| Leg | What it is | Artifact |
|
| 8 |
+
|-----|------------|----------|
|
| 9 |
+
| **Tombstone** | Append-only public-facing run log | `tombstone/tombstone.md` |
|
| 10 |
+
| **Covenant** | Private structured AI memories | `AI_MEMORY_ROOT` (default `./data/ai_memory_root`, gitignored) |
|
| 11 |
+
| **JLens** | Real Anthropic `jlens.JacobianLens.apply` snapshot | `runs/<task_id>/jlens_snapshot.json` |
|
| 12 |
+
|
| 13 |
+
Self-observation is **not** a triad leg.
|
| 14 |
+
|
| 15 |
+
## Architecture
|
| 16 |
+
|
| 17 |
+
```
|
| 18 |
+
task_id (shared)
|
| 19 |
+
│
|
| 20 |
+
┌───────────────┼───────────────┐
|
| 21 |
+
▼ ▼ ▼
|
| 22 |
+
tombstone.md covenant puts jlens snapshot
|
| 23 |
+
(append.sh) (put_covenant_*) (jlens_snapshot.py)
|
| 24 |
+
│ │ │
|
| 25 |
+
└───────────────┴───────────────┘
|
| 26 |
+
│
|
| 27 |
+
fire_triad.sh
|
| 28 |
+
```
|
| 29 |
+
|
| 30 |
+
`fire_triad.sh` runs **JLens first** so the digest can be stamped into tombstone notes and covenant bodies, then proves the same `task_id` appears in all three stores.
|
| 31 |
+
|
| 32 |
+
## CPU notes
|
| 33 |
+
|
| 34 |
+
- Model: `openai-community/gpt2` (~124M, 12 layers). Fits comfortably on CPU (~1GB peak RSS for the demo).
|
| 35 |
+
- **No GPU required.** float32 CPU only.
|
| 36 |
+
- Lens: pretrained gpt2-small weights from Hugging Face `neuronpedia/jacobian-lens`
|
| 37 |
+
(`gpt2-small/jlens/Salesforce-wikitext/gpt2_jacobian_lens.pt`), fetched by `scripts/fetch_lens.sh`.
|
| 38 |
+
The ~12MB `.pt` file is **not** shipped in this repo (size / GitHub limits).
|
| 39 |
+
- Prefer Python **3.12**. Install **CPU** torch first.
|
| 40 |
+
|
| 41 |
+
## Setup
|
| 42 |
+
|
| 43 |
+
```bash
|
| 44 |
+
git clone https://github.com/LAWLESS1987/threefold
|
| 45 |
+
cd triad
|
| 46 |
+
|
| 47 |
+
python3.12 -m venv .venv
|
| 48 |
+
source .venv/bin/activate
|
| 49 |
+
pip install --index-url https://download.pytorch.org/whl/cpu torch==2.14.1
|
| 50 |
+
pip install -r requirements.txt
|
| 51 |
+
pip install "jlens @ git+https://github.com/anthropics/jacobian-lens"
|
| 52 |
+
|
| 53 |
+
# Covenant (private memory CLI) — do not vendor the whole tree into git history
|
| 54 |
+
git clone https://github.com/LAWLESS1987/covenant vendor/covenant
|
| 55 |
+
# or: export COVENANT_SRC=/path/to/covenant
|
| 56 |
+
|
| 57 |
+
./scripts/fetch_lens.sh
|
| 58 |
+
./scripts/seed_demo.sh # MODE=apply (default): proves SUCCESS via demo_apply.py
|
| 59 |
+
# MODE=triad ./scripts/seed_demo.sh # full three-leg fire with fixed TASK_ID
|
| 60 |
+
```
|
| 61 |
+
|
| 62 |
+
Private memories default to `./data/ai_memory_root` (gitignored). Override with `AI_MEMORY_ROOT`.
|
| 63 |
+
|
| 64 |
+
## How seed works
|
| 65 |
+
|
| 66 |
+
`scripts/seed_demo.sh` uses fixed demo values:
|
| 67 |
+
|
| 68 |
+
- `TASK_ID=20261007-020907-13423`
|
| 69 |
+
- `PROMPT="Fact: The currency used in the country shaped like a boot is"`
|
| 70 |
+
|
| 71 |
+
Default `MODE=apply` fetches the lens (if missing) and runs `demo_apply.py`, expecting the line:
|
| 72 |
+
|
| 73 |
+
`SUCCESS: lens.apply returned real top-k tokens`
|
| 74 |
+
|
| 75 |
+
`MODE=triad` additionally clones covenant if needed and runs `triad/fire_triad.sh`, expecting `FIRE_TRIAD_OK`.
|
| 76 |
+
|
| 77 |
+
The sample entry in `tombstone/tombstone.md` is illustrative of that same `task_id` format only — no private history.
|
| 78 |
+
|
| 79 |
+
## Layout
|
| 80 |
+
|
| 81 |
+
```
|
| 82 |
+
triad/
|
| 83 |
+
README.md
|
| 84 |
+
requirements.txt
|
| 85 |
+
.gitignore
|
| 86 |
+
jlens_snapshot.py
|
| 87 |
+
demo_apply.py
|
| 88 |
+
tombstone/
|
| 89 |
+
append.sh
|
| 90 |
+
tombstone.md # SAMPLE only
|
| 91 |
+
triad/
|
| 92 |
+
fire_triad.sh
|
| 93 |
+
put_covenant_memory.sh
|
| 94 |
+
scripts/
|
| 95 |
+
fetch_lens.sh
|
| 96 |
+
seed_demo.sh
|
| 97 |
+
```
|
| 98 |
+
|
| 99 |
+
## License / upstream
|
| 100 |
+
|
| 101 |
+
- Jacobian lens library: [anthropics/jacobian-lens](https://github.com/anthropics/jacobian-lens)
|
| 102 |
+
- Pretrained lens weights: [neuronpedia/jacobian-lens](https://huggingface.co/neuronpedia/jacobian-lens)
|
| 103 |
+
- Covenant memory CLI: [LAWLESS1987/covenant](https://github.com/LAWLESS1987/covenant)
|
demo_apply.py
ADDED
|
@@ -0,0 +1,113 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
#!/usr/bin/env python3
|
| 2 |
+
"""Prove jlens.JacobianLens.apply returns real top-k tokens on CPU.
|
| 3 |
+
|
| 4 |
+
Model: openai-community/gpt2 (~124M)
|
| 5 |
+
Lens: pretrained gpt2-small lens (fetch via scripts/fetch_lens.sh)
|
| 6 |
+
"""
|
| 7 |
+
from __future__ import annotations
|
| 8 |
+
|
| 9 |
+
import os
|
| 10 |
+
import resource
|
| 11 |
+
import time
|
| 12 |
+
import traceback
|
| 13 |
+
|
| 14 |
+
os.environ.setdefault("TOKENIZERS_PARALLELISM", "false")
|
| 15 |
+
|
| 16 |
+
import torch
|
| 17 |
+
import transformers
|
| 18 |
+
import jlens
|
| 19 |
+
|
| 20 |
+
MODEL_ID = "openai-community/gpt2"
|
| 21 |
+
LENS_PATH = os.environ.get(
|
| 22 |
+
"JLENS_LENS",
|
| 23 |
+
os.path.join(os.path.dirname(os.path.abspath(__file__)), "gpt2_jacobian_lens.pt"),
|
| 24 |
+
)
|
| 25 |
+
PROMPT = os.environ.get(
|
| 26 |
+
"TRIAD_PROMPT",
|
| 27 |
+
"Fact: The currency used in the country shaped like a boot is",
|
| 28 |
+
)
|
| 29 |
+
TOP_K = 5
|
| 30 |
+
|
| 31 |
+
|
| 32 |
+
def rss_mb() -> float:
|
| 33 |
+
return resource.getrusage(resource.RUSAGE_SELF).ru_maxrss / 1024.0
|
| 34 |
+
|
| 35 |
+
|
| 36 |
+
def topk_tokens(logits: torch.Tensor, tok, k: int = TOP_K) -> list[str]:
|
| 37 |
+
ids = logits.topk(k).indices.tolist()
|
| 38 |
+
return [tok.decode([t]) for t in ids]
|
| 39 |
+
|
| 40 |
+
|
| 41 |
+
def main() -> None:
|
| 42 |
+
t0 = time.perf_counter()
|
| 43 |
+
print("=== jlens CPU demo ===")
|
| 44 |
+
print(f"python torch={torch.__version__} cuda={torch.cuda.is_available()}")
|
| 45 |
+
print(f"transformers={transformers.__version__}")
|
| 46 |
+
print(f"jlens={jlens.__file__}")
|
| 47 |
+
print(f"model_id={MODEL_ID}")
|
| 48 |
+
print(f"lens=JacobianLens.load({LENS_PATH!r})")
|
| 49 |
+
print(f"rss_start_mb={rss_mb():.1f}")
|
| 50 |
+
print()
|
| 51 |
+
|
| 52 |
+
if not os.path.isfile(LENS_PATH):
|
| 53 |
+
raise SystemExit(
|
| 54 |
+
f"Missing lens at {LENS_PATH}. Run scripts/fetch_lens.sh first."
|
| 55 |
+
)
|
| 56 |
+
|
| 57 |
+
print("Loading model (CPU, float32)...")
|
| 58 |
+
t_load = time.perf_counter()
|
| 59 |
+
hf = transformers.AutoModelForCausalLM.from_pretrained(MODEL_ID)
|
| 60 |
+
hf.eval()
|
| 61 |
+
tok = transformers.AutoTokenizer.from_pretrained(MODEL_ID)
|
| 62 |
+
model = jlens.from_hf(hf, tok)
|
| 63 |
+
print(f" model n_layers={model.n_layers} d_model={model.d_model}")
|
| 64 |
+
print(f" load_s={time.perf_counter() - t_load:.1f} rss_mb={rss_mb():.1f}")
|
| 65 |
+
print()
|
| 66 |
+
|
| 67 |
+
print("Loading pre-fitted lens...")
|
| 68 |
+
t_lens = time.perf_counter()
|
| 69 |
+
lens = jlens.JacobianLens.load(LENS_PATH)
|
| 70 |
+
print(f" {lens}")
|
| 71 |
+
print(f" lens_load_s={time.perf_counter() - t_lens:.1f} rss_mb={rss_mb():.1f}")
|
| 72 |
+
print()
|
| 73 |
+
|
| 74 |
+
layers = [
|
| 75 |
+
model.n_layers // 4,
|
| 76 |
+
model.n_layers // 2,
|
| 77 |
+
(model.n_layers * 3) // 4,
|
| 78 |
+
model.n_layers - 2,
|
| 79 |
+
]
|
| 80 |
+
layers = [L for L in layers if L in lens.jacobians]
|
| 81 |
+
if not layers:
|
| 82 |
+
layers = lens.source_layers[:: max(1, len(lens.source_layers) // 4)][:4]
|
| 83 |
+
|
| 84 |
+
print(f"prompt={PROMPT!r}")
|
| 85 |
+
print(f"layers={layers} positions=[-2]")
|
| 86 |
+
print()
|
| 87 |
+
|
| 88 |
+
t_apply = time.perf_counter()
|
| 89 |
+
lens_logits, model_logits, _ = lens.apply(
|
| 90 |
+
model, PROMPT, layers=layers, positions=[-2]
|
| 91 |
+
)
|
| 92 |
+
apply_s = time.perf_counter() - t_apply
|
| 93 |
+
print(f"apply_s={apply_s:.2f} rss_mb={rss_mb():.1f}")
|
| 94 |
+
print()
|
| 95 |
+
|
| 96 |
+
print("--- top-k from lens.apply (Jacobian lens) ---")
|
| 97 |
+
for layer in layers:
|
| 98 |
+
toks = topk_tokens(lens_logits[layer][0], tok)
|
| 99 |
+
print(f"L{layer:>3} J-lens top-{TOP_K}: {toks}")
|
| 100 |
+
|
| 101 |
+
print("--- model final logits top-k ---")
|
| 102 |
+
print(f"model top-{TOP_K}: {topk_tokens(model_logits[0], tok)}")
|
| 103 |
+
print()
|
| 104 |
+
print(f"TOTAL_s={time.perf_counter() - t0:.1f} peak_rss_mb={rss_mb():.1f}")
|
| 105 |
+
print("SUCCESS: lens.apply returned real top-k tokens")
|
| 106 |
+
|
| 107 |
+
|
| 108 |
+
if __name__ == "__main__":
|
| 109 |
+
try:
|
| 110 |
+
main()
|
| 111 |
+
except Exception:
|
| 112 |
+
traceback.print_exc()
|
| 113 |
+
raise SystemExit(1)
|
jlens_snapshot.py
ADDED
|
@@ -0,0 +1,121 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
#!/usr/bin/env python3
|
| 2 |
+
"""Run real Anthropic jlens.JacobianLens.apply and write a triad snapshot.
|
| 3 |
+
|
| 4 |
+
Usage:
|
| 5 |
+
jlens_snapshot.py --task-id TID --prompt "..." --out SNAPSHOT_PATH [--digest DIGEST_PATH]
|
| 6 |
+
|
| 7 |
+
Exits nonzero if apply fails or top-k is empty.
|
| 8 |
+
"""
|
| 9 |
+
from __future__ import annotations
|
| 10 |
+
|
| 11 |
+
import argparse
|
| 12 |
+
import hashlib
|
| 13 |
+
import json
|
| 14 |
+
import os
|
| 15 |
+
import sys
|
| 16 |
+
import time
|
| 17 |
+
|
| 18 |
+
os.environ.setdefault("TOKENIZERS_PARALLELISM", "false")
|
| 19 |
+
|
| 20 |
+
import torch
|
| 21 |
+
import transformers
|
| 22 |
+
import jlens
|
| 23 |
+
|
| 24 |
+
MODEL_ID = "openai-community/gpt2"
|
| 25 |
+
# Default: lens next to this script (after scripts/fetch_lens.sh)
|
| 26 |
+
DEFAULT_LENS = os.path.join(os.path.dirname(os.path.abspath(__file__)), "gpt2_jacobian_lens.pt")
|
| 27 |
+
TOP_K = 5
|
| 28 |
+
|
| 29 |
+
|
| 30 |
+
def topk_tokens(logits: torch.Tensor, tok, k: int = TOP_K) -> list[str]:
|
| 31 |
+
ids = logits.topk(k).indices.tolist()
|
| 32 |
+
return [tok.decode([t]) for t in ids]
|
| 33 |
+
|
| 34 |
+
|
| 35 |
+
def main() -> int:
|
| 36 |
+
ap = argparse.ArgumentParser()
|
| 37 |
+
ap.add_argument("--task-id", required=True)
|
| 38 |
+
ap.add_argument("--prompt", required=True)
|
| 39 |
+
ap.add_argument("--out", required=True, help="JSON snapshot path")
|
| 40 |
+
ap.add_argument("--digest", default="", help="optional short digest text path")
|
| 41 |
+
ap.add_argument("--lens", default=DEFAULT_LENS)
|
| 42 |
+
ap.add_argument("--model", default=MODEL_ID)
|
| 43 |
+
args = ap.parse_args()
|
| 44 |
+
|
| 45 |
+
t0 = time.perf_counter()
|
| 46 |
+
hf = transformers.AutoModelForCausalLM.from_pretrained(args.model)
|
| 47 |
+
hf.eval()
|
| 48 |
+
tok = transformers.AutoTokenizer.from_pretrained(args.model)
|
| 49 |
+
model = jlens.from_hf(hf, tok)
|
| 50 |
+
|
| 51 |
+
lens = jlens.JacobianLens.load(args.lens)
|
| 52 |
+
|
| 53 |
+
layers = [
|
| 54 |
+
model.n_layers // 4,
|
| 55 |
+
model.n_layers // 2,
|
| 56 |
+
(model.n_layers * 3) // 4,
|
| 57 |
+
model.n_layers - 2,
|
| 58 |
+
]
|
| 59 |
+
layers = [L for L in layers if L in lens.jacobians]
|
| 60 |
+
if not layers:
|
| 61 |
+
layers = list(lens.source_layers)[:4]
|
| 62 |
+
|
| 63 |
+
lens_logits, model_logits, _ = lens.apply(
|
| 64 |
+
model, args.prompt, layers=layers, positions=[-2]
|
| 65 |
+
)
|
| 66 |
+
|
| 67 |
+
per_layer = {}
|
| 68 |
+
lines = []
|
| 69 |
+
for layer in layers:
|
| 70 |
+
toks = topk_tokens(lens_logits[layer][0], tok)
|
| 71 |
+
if not toks:
|
| 72 |
+
print(f"ERROR: empty top-k at layer {layer}", file=sys.stderr)
|
| 73 |
+
return 1
|
| 74 |
+
per_layer[str(layer)] = toks
|
| 75 |
+
lines.append(f"L{layer}: {toks}")
|
| 76 |
+
|
| 77 |
+
model_top = topk_tokens(model_logits[0], tok)
|
| 78 |
+
body = {
|
| 79 |
+
"type": "jlens",
|
| 80 |
+
"task_id": args.task_id,
|
| 81 |
+
"model_id": args.model,
|
| 82 |
+
"lens_path": args.lens,
|
| 83 |
+
"prompt": args.prompt,
|
| 84 |
+
"layers": layers,
|
| 85 |
+
"positions": [-2],
|
| 86 |
+
"top_k": TOP_K,
|
| 87 |
+
"jlens_top_k": per_layer,
|
| 88 |
+
"model_final_top_k": model_top,
|
| 89 |
+
"apply_s": round(time.perf_counter() - t0, 3),
|
| 90 |
+
"mechanism": "anthropics/jacobian-lens JacobianLens.apply (real per-layer logits)",
|
| 91 |
+
}
|
| 92 |
+
|
| 93 |
+
os.makedirs(os.path.dirname(os.path.abspath(args.out)) or ".", exist_ok=True)
|
| 94 |
+
with open(args.out, "w", encoding="utf-8") as f:
|
| 95 |
+
json.dump(body, f, indent=2)
|
| 96 |
+
f.write("\n")
|
| 97 |
+
|
| 98 |
+
digest = (
|
| 99 |
+
f"jlens task_id={args.task_id} model={args.model} "
|
| 100 |
+
f"layers={layers} topk={{{'; '.join(lines)}}}"
|
| 101 |
+
)
|
| 102 |
+
digest_hash = hashlib.sha256(
|
| 103 |
+
json.dumps(body, sort_keys=True).encode()
|
| 104 |
+
).hexdigest()[:16]
|
| 105 |
+
digest_line = f"{digest} digest={digest_hash}"
|
| 106 |
+
|
| 107 |
+
if args.digest:
|
| 108 |
+
with open(args.digest, "w", encoding="utf-8") as f:
|
| 109 |
+
f.write(digest_line + "\n")
|
| 110 |
+
for line in lines:
|
| 111 |
+
f.write(line + "\n")
|
| 112 |
+
|
| 113 |
+
print(digest_line)
|
| 114 |
+
for line in lines:
|
| 115 |
+
print(line)
|
| 116 |
+
print(f"SNAPSHOT_OK path={args.out}")
|
| 117 |
+
return 0
|
| 118 |
+
|
| 119 |
+
|
| 120 |
+
if __name__ == "__main__":
|
| 121 |
+
raise SystemExit(main())
|
requirements.txt
ADDED
|
@@ -0,0 +1,12 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# CPU-only triad demo stack (gpt2-small + pretrained Jacobian lens).
|
| 2 |
+
# Prefer a Python 3.12 venv. Install torch CPU first, then the rest.
|
| 3 |
+
#
|
| 4 |
+
# pip install --index-url https://download.pytorch.org/whl/cpu torch==2.14.1
|
| 5 |
+
# pip install -r requirements.txt
|
| 6 |
+
# pip install "jlens @ git+https://github.com/anthropics/jacobian-lens"
|
| 7 |
+
#
|
| 8 |
+
torch==2.14.1
|
| 9 |
+
transformers==5.19.0
|
| 10 |
+
huggingface_hub>=0.20.0
|
| 11 |
+
# jlens: install from git (not on PyPI as a stable pin here)
|
| 12 |
+
# pip install "jlens @ git+https://github.com/anthropics/jacobian-lens"
|
scripts/fetch_lens.sh
ADDED
|
@@ -0,0 +1,45 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
#!/usr/bin/env bash
|
| 2 |
+
# Download the pretrained gpt2-small Jacobian lens from Hugging Face
|
| 3 |
+
# (neuronpedia/jacobian-lens) into the repo root.
|
| 4 |
+
#
|
| 5 |
+
# Source path on the Hub:
|
| 6 |
+
# gpt2-small/jlens/Salesforce-wikitext/gpt2_jacobian_lens.pt
|
| 7 |
+
#
|
| 8 |
+
# Usage:
|
| 9 |
+
# ./scripts/fetch_lens.sh
|
| 10 |
+
# OUT=/path/to/gpt2_jacobian_lens.pt ./scripts/fetch_lens.sh
|
| 11 |
+
set -euo pipefail
|
| 12 |
+
|
| 13 |
+
REPO_ROOT="$(cd "$(dirname "$0")/.." && pwd)"
|
| 14 |
+
OUT="${OUT:-$REPO_ROOT/gpt2_jacobian_lens.pt}"
|
| 15 |
+
REPO_ID="${HF_REPO_ID:-neuronpedia/jacobian-lens}"
|
| 16 |
+
HF_PATH="${HF_LENS_PATH:-gpt2-small/jlens/Salesforce-wikitext/gpt2_jacobian_lens.pt}"
|
| 17 |
+
|
| 18 |
+
if [[ -f "$OUT" ]]; then
|
| 19 |
+
echo "Lens already present: $OUT ($(du -h "$OUT" | awk '{print $1}'))"
|
| 20 |
+
exit 0
|
| 21 |
+
fi
|
| 22 |
+
|
| 23 |
+
echo "Fetching $REPO_ID / $HF_PATH -> $OUT"
|
| 24 |
+
|
| 25 |
+
# Prefer huggingface-cli / huggingface_hub if available
|
| 26 |
+
if command -v huggingface-cli >/dev/null 2>&1; then
|
| 27 |
+
TMPDIR="$(mktemp -d)"
|
| 28 |
+
huggingface-cli download "$REPO_ID" "$HF_PATH" --local-dir "$TMPDIR" --local-dir-use-symlinks False
|
| 29 |
+
mv "$TMPDIR/$HF_PATH" "$OUT"
|
| 30 |
+
rm -rf "$TMPDIR"
|
| 31 |
+
elif python3 -c "import huggingface_hub" 2>/dev/null; then
|
| 32 |
+
python3 - <<PY
|
| 33 |
+
from huggingface_hub import hf_hub_download
|
| 34 |
+
import shutil, os
|
| 35 |
+
path = hf_hub_download(repo_id="${REPO_ID}", filename="${HF_PATH}")
|
| 36 |
+
shutil.copy2(path, "${OUT}")
|
| 37 |
+
print(f"copied {path} -> ${OUT}")
|
| 38 |
+
PY
|
| 39 |
+
else
|
| 40 |
+
echo "Need huggingface_hub (pip install huggingface_hub) or huggingface-cli" >&2
|
| 41 |
+
exit 1
|
| 42 |
+
fi
|
| 43 |
+
|
| 44 |
+
ls -lh "$OUT"
|
| 45 |
+
echo "FETCH_LENS_OK path=$OUT"
|
scripts/push_to_github.sh
ADDED
|
@@ -0,0 +1,15 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
#!/usr/bin/env bash
|
| 2 |
+
set -euo pipefail
|
| 3 |
+
REPO_ROOT="$(cd "$(dirname "$0")/.." && pwd)"
|
| 4 |
+
cd "$REPO_ROOT"
|
| 5 |
+
OWNER=LAWLESS1987
|
| 6 |
+
NAME=triad
|
| 7 |
+
# Create if missing
|
| 8 |
+
if ! gh repo view "$OWNER/$NAME" >/dev/null 2>&1; then
|
| 9 |
+
gh repo create "$OWNER/$NAME" --public --description "Mandatory three-leg agent memory: tombstone || covenant || JLens (shared task_id, no degrade)" --source . --remote origin --push
|
| 10 |
+
else
|
| 11 |
+
git remote remove origin 2>/dev/null || true
|
| 12 |
+
git remote add origin "https://github.com/$OWNER/$NAME.git"
|
| 13 |
+
git push -u origin main
|
| 14 |
+
fi
|
| 15 |
+
echo "PUSH_OK https://github.com/$OWNER/$NAME"
|
scripts/seed_demo.sh
ADDED
|
@@ -0,0 +1,88 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
#!/usr/bin/env bash
|
| 2 |
+
# Regenerate the public demo: fetch lens if missing, then either
|
| 3 |
+
# MODE=apply (default) — run demo_apply.py and require SUCCESS
|
| 4 |
+
# MODE=triad — run fire_triad.sh with fixed TASK_ID/PROMPT
|
| 5 |
+
#
|
| 6 |
+
# Fixed demo values (illustrative / reproducible enough to prove SUCCESS):
|
| 7 |
+
# TASK_ID=20261007-020907-13423
|
| 8 |
+
# PROMPT="Fact: The currency used in the country shaped like a boot is"
|
| 9 |
+
#
|
| 10 |
+
# Env overrides:
|
| 11 |
+
# JLENS_VENV — python venv with torch/transformers/jlens
|
| 12 |
+
# MODE — apply | triad
|
| 13 |
+
# COVENANT_SRC / AI_MEMORY_ROOT — for MODE=triad
|
| 14 |
+
set -euo pipefail
|
| 15 |
+
|
| 16 |
+
REPO_ROOT="$(cd "$(dirname "$0")/.." && pwd)"
|
| 17 |
+
cd "$REPO_ROOT"
|
| 18 |
+
|
| 19 |
+
TASK_ID="${TASK_ID:-20261007-020907-13423}"
|
| 20 |
+
PROMPT="${TRIAD_PROMPT:-Fact: The currency used in the country shaped like a boot is}"
|
| 21 |
+
MODE="${MODE:-apply}"
|
| 22 |
+
|
| 23 |
+
# Prefer explicit JLENS_VENV, then repo .venv, then common shared path
|
| 24 |
+
if [[ -z "${JLENS_VENV:-}" ]]; then
|
| 25 |
+
if [[ -x "$REPO_ROOT/.venv/bin/python" ]]; then
|
| 26 |
+
JLENS_VENV="$REPO_ROOT/.venv"
|
| 27 |
+
elif [[ -x /workspace/jlens-venv/bin/python ]]; then
|
| 28 |
+
JLENS_VENV=/workspace/jlens-venv
|
| 29 |
+
else
|
| 30 |
+
JLENS_VENV="$REPO_ROOT/.venv"
|
| 31 |
+
fi
|
| 32 |
+
fi
|
| 33 |
+
export JLENS_VENV
|
| 34 |
+
|
| 35 |
+
echo "=== seed_demo MODE=$MODE TASK_ID=$TASK_ID ==="
|
| 36 |
+
echo "REPO_ROOT=$REPO_ROOT"
|
| 37 |
+
echo "JLENS_VENV=$JLENS_VENV"
|
| 38 |
+
|
| 39 |
+
# 1) Lens
|
| 40 |
+
"$REPO_ROOT/scripts/fetch_lens.sh"
|
| 41 |
+
export JLENS_LENS="${JLENS_LENS:-$REPO_ROOT/gpt2_jacobian_lens.pt}"
|
| 42 |
+
|
| 43 |
+
# Resolve python
|
| 44 |
+
if [[ -x "$JLENS_VENV/bin/python" ]]; then
|
| 45 |
+
PYTHON="$JLENS_VENV/bin/python"
|
| 46 |
+
else
|
| 47 |
+
PYTHON="$(command -v python3)"
|
| 48 |
+
fi
|
| 49 |
+
echo "PYTHON=$PYTHON"
|
| 50 |
+
|
| 51 |
+
if [[ "$MODE" == "apply" ]]; then
|
| 52 |
+
echo "--- demo_apply.py ---"
|
| 53 |
+
TRIAD_PROMPT="$PROMPT" JLENS_LENS="$JLENS_LENS" "$PYTHON" "$REPO_ROOT/demo_apply.py" | tee /tmp/triad_seed_demo_apply.log
|
| 54 |
+
rg -n "SUCCESS: lens.apply returned real top-k tokens" /tmp/triad_seed_demo_apply.log
|
| 55 |
+
echo "SEED_DEMO_OK mode=apply"
|
| 56 |
+
exit 0
|
| 57 |
+
fi
|
| 58 |
+
|
| 59 |
+
if [[ "$MODE" == "triad" ]]; then
|
| 60 |
+
# Ensure covenant is available
|
| 61 |
+
export COVENANT_SRC="${COVENANT_SRC:-$REPO_ROOT/vendor/covenant}"
|
| 62 |
+
if [[ ! -f "$COVENANT_SRC/ai_memory_system/main.py" ]]; then
|
| 63 |
+
echo "Cloning covenant into vendor/covenant ..."
|
| 64 |
+
mkdir -p "$REPO_ROOT/vendor"
|
| 65 |
+
git clone --depth 1 https://github.com/LAWLESS1987/covenant "$COVENANT_SRC"
|
| 66 |
+
fi
|
| 67 |
+
export AI_MEMORY_ROOT="${AI_MEMORY_ROOT:-$REPO_ROOT/data/ai_memory_root}"
|
| 68 |
+
mkdir -p "$AI_MEMORY_ROOT"
|
| 69 |
+
|
| 70 |
+
echo "--- fire_triad.sh ---"
|
| 71 |
+
export TASK_ID PROMPT JLENS_VENV JLENS_LENS
|
| 72 |
+
export JLENS_SNAPSHOT_PY="$REPO_ROOT/jlens_snapshot.py"
|
| 73 |
+
"$REPO_ROOT/triad/fire_triad.sh" \
|
| 74 |
+
"jlens-triad-e2e" \
|
| 75 |
+
"Prove real Anthropic jacobian-lens as triad third leg" \
|
| 76 |
+
"Installed gpt2 lens, wired fire_triad with tombstone+covenant+jlens.apply" \
|
| 77 |
+
"success" \
|
| 78 |
+
"$PROMPT" \
|
| 79 |
+
"none" \
|
| 80 |
+
"Third leg must be real jlens.apply not self-observation" \
|
| 81 |
+
"seed-demo" | tee /tmp/triad_seed_demo_triad.log
|
| 82 |
+
rg -n "FIRE_TRIAD_OK" /tmp/triad_seed_demo_triad.log
|
| 83 |
+
echo "SEED_DEMO_OK mode=triad task_id=$TASK_ID"
|
| 84 |
+
exit 0
|
| 85 |
+
fi
|
| 86 |
+
|
| 87 |
+
echo "Unknown MODE=$MODE (use apply|triad)" >&2
|
| 88 |
+
exit 1
|
tombstone/append.sh
ADDED
|
@@ -0,0 +1,47 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
#!/usr/bin/env bash
|
| 2 |
+
# Append one tombstone entry. Usage:
|
| 3 |
+
# ./append.sh "short-title" "task" "done" "outcome" "mistakes" "lessons" ["notes"]
|
| 4 |
+
# Or with env vars: TITLE TASK DONE OUTCOME MISTAKES LESSONS NOTES
|
| 5 |
+
set -euo pipefail
|
| 6 |
+
DIR="$(cd "$(dirname "$0")" && pwd)"
|
| 7 |
+
LOG="${TOMBSTONE_LOG:-$DIR/tombstone.md}"
|
| 8 |
+
|
| 9 |
+
TITLE="${1:-${TITLE:-}}"
|
| 10 |
+
TASK="${2:-${TASK:-}}"
|
| 11 |
+
DONE="${3:-${DONE:-}}"
|
| 12 |
+
OUTCOME="${4:-${OUTCOME:-}}"
|
| 13 |
+
MISTAKES="${5:-${MISTAKES:-none}}"
|
| 14 |
+
LESSONS="${6:-${LESSONS:-none}}"
|
| 15 |
+
NOTES="${7:-${NOTES:-}}"
|
| 16 |
+
|
| 17 |
+
if [[ -z "$TITLE" || -z "$TASK" || -z "$DONE" || -z "$OUTCOME" ]]; then
|
| 18 |
+
echo "Usage: $0 \"short-title\" \"task\" \"done\" \"outcome\" [\"mistakes\"] [\"lessons\"] [\"notes\"]" >&2
|
| 19 |
+
echo "outcome must be one of: success | partial | failed | blocked" >&2
|
| 20 |
+
exit 1
|
| 21 |
+
fi
|
| 22 |
+
|
| 23 |
+
case "$OUTCOME" in
|
| 24 |
+
success|partial|failed|blocked) ;;
|
| 25 |
+
*) echo "Invalid outcome: $OUTCOME (use success|partial|failed|blocked)" >&2; exit 1 ;;
|
| 26 |
+
esac
|
| 27 |
+
|
| 28 |
+
TS="$(date -u +%Y-%m-%dT%H:%M:%SZ)"
|
| 29 |
+
ID="$(date -u +%Y%m%d-%H%M%S)"
|
| 30 |
+
|
| 31 |
+
{
|
| 32 |
+
echo ""
|
| 33 |
+
echo "### ${TS} | ${TITLE}"
|
| 34 |
+
echo "- id: ${ID}"
|
| 35 |
+
echo "- task: ${TASK}"
|
| 36 |
+
echo "- done: ${DONE}"
|
| 37 |
+
echo "- outcome: ${OUTCOME}"
|
| 38 |
+
echo "- mistakes: ${MISTAKES}"
|
| 39 |
+
echo "- lessons: ${LESSONS}"
|
| 40 |
+
if [[ -n "$NOTES" ]]; then
|
| 41 |
+
echo "- notes: ${NOTES}"
|
| 42 |
+
else
|
| 43 |
+
echo "- notes: none"
|
| 44 |
+
fi
|
| 45 |
+
} >> "$LOG"
|
| 46 |
+
|
| 47 |
+
echo "Appended entry ${ID} to $LOG"
|
tombstone/tombstone.md
ADDED
|
@@ -0,0 +1,16 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Tombstone log (SAMPLE)
|
| 2 |
+
|
| 3 |
+
Public sample only. Real deployments append locally; do not commit private entries.
|
| 4 |
+
|
| 5 |
+
Format: each entry is a markdown subsection with `id`, `task`, `done`, `outcome`,
|
| 6 |
+
`mistakes`, `lessons`, and `notes`. Triad runs stamp `task_id=...` and
|
| 7 |
+
`jlens_digest=...` into notes so the three legs share one id.
|
| 8 |
+
|
| 9 |
+
### 2026-10-07T02:09:13Z | jlens-triad-e2e
|
| 10 |
+
- id: 20261007-020913
|
| 11 |
+
- task: Prove real Anthropic jacobian-lens as triad third leg
|
| 12 |
+
- done: Installed gpt2 lens, wired fire_triad with tombstone+covenant+jlens.apply
|
| 13 |
+
- outcome: success
|
| 14 |
+
- mistakes: none
|
| 15 |
+
- lessons: Third leg must be real jlens.apply not self-observation
|
| 16 |
+
- notes: task_id=20261007-020907-13423; jlens_digest=jlens task_id=20261007-020907-13423 model=openai-community/gpt2 layers=[3, 6, 9, 10] topk={L3: [' boot', 'boot', 'Boot', ' Boot', ' boots']; L6: ['boot', ' boot', 'Boot', ' Boot', ' boots']; L9: ['strap', 'loader', 'boot', 'legged', ' boot']; L10: ['strap', 'stra', 'legged', 'loader', 'leg']} digest=82a9a0b5be4cfbaa; sample-demo-entry
|
triad/fire_triad.sh
ADDED
|
@@ -0,0 +1,128 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
#!/usr/bin/env bash
|
| 2 |
+
# fire_triad.sh — mandatory three-leg triad (tombstone + covenant + real JLens)
|
| 3 |
+
#
|
| 4 |
+
# Usage:
|
| 5 |
+
# ./fire_triad.sh TITLE TASK DONE OUTCOME PROMPT [MISTAKES] [LESSONS] [NOTES]
|
| 6 |
+
# Or env: TITLE TASK DONE OUTCOME PROMPT MISTAKES LESSONS NOTES TASK_ID
|
| 7 |
+
#
|
| 8 |
+
# Legs (all required; any failure => exit nonzero):
|
| 9 |
+
# (a) tombstone append with JLens digest in notes
|
| 10 |
+
# (b) covenant puts (task-{id} + jlens-{id} type:jlens snapshot)
|
| 11 |
+
# (c) real Anthropic jlens.JacobianLens.apply snapshot
|
| 12 |
+
#
|
| 13 |
+
# Defaults resolve relative to REPO_ROOT (parent of triad/).
|
| 14 |
+
set -euo pipefail
|
| 15 |
+
|
| 16 |
+
DIR="$(cd "$(dirname "$0")" && pwd)"
|
| 17 |
+
REPO_ROOT="$(cd "$DIR/.." && pwd)"
|
| 18 |
+
TOMBSTONE_APPEND="${TOMBSTONE_APPEND:-$REPO_ROOT/tombstone/append.sh}"
|
| 19 |
+
PUT_COVENANT="${PUT_COVENANT:-$DIR/put_covenant_memory.sh}"
|
| 20 |
+
JLENS_VENV="${JLENS_VENV:-${REPO_ROOT}/.venv}"
|
| 21 |
+
# Prefer repo-local scripts; allow override for shared venvs.
|
| 22 |
+
JLENS_SNAPSHOT_PY="${JLENS_SNAPSHOT_PY:-$REPO_ROOT/jlens_snapshot.py}"
|
| 23 |
+
JLENS_LENS="${JLENS_LENS:-$REPO_ROOT/gpt2_jacobian_lens.pt}"
|
| 24 |
+
RUN_DIR="${RUN_DIR:-$REPO_ROOT/runs}"
|
| 25 |
+
export AI_MEMORY_ROOT="${AI_MEMORY_ROOT:-$REPO_ROOT/data/ai_memory_root}"
|
| 26 |
+
export COVENANT_SRC="${COVENANT_SRC:-$REPO_ROOT/vendor/covenant}"
|
| 27 |
+
|
| 28 |
+
TITLE="${1:-${TITLE:-}}"
|
| 29 |
+
TASK="${2:-${TASK:-}}"
|
| 30 |
+
DONE="${3:-${DONE:-}}"
|
| 31 |
+
OUTCOME="${4:-${OUTCOME:-}}"
|
| 32 |
+
PROMPT="${5:-${PROMPT:-}}"
|
| 33 |
+
MISTAKES="${6:-${MISTAKES:-none}}"
|
| 34 |
+
LESSONS="${7:-${LESSONS:-none}}"
|
| 35 |
+
NOTES="${8:-${NOTES:-}}"
|
| 36 |
+
|
| 37 |
+
if [[ -z "$TITLE" || -z "$TASK" || -z "$DONE" || -z "$OUTCOME" || -z "$PROMPT" ]]; then
|
| 38 |
+
echo "Usage: $0 TITLE TASK DONE OUTCOME PROMPT [MISTAKES] [LESSONS] [NOTES]" >&2
|
| 39 |
+
exit 1
|
| 40 |
+
fi
|
| 41 |
+
|
| 42 |
+
TASK_ID="${TASK_ID:-$(date -u +%Y%m%d-%H%M%S)-$$}"
|
| 43 |
+
SNAP_DIR="$RUN_DIR/$TASK_ID"
|
| 44 |
+
mkdir -p "$SNAP_DIR"
|
| 45 |
+
SNAP_JSON="$SNAP_DIR/jlens_snapshot.json"
|
| 46 |
+
SNAP_DIGEST="$SNAP_DIR/jlens_digest.txt"
|
| 47 |
+
|
| 48 |
+
echo "=== fire_triad task_id=$TASK_ID ==="
|
| 49 |
+
echo "prompt=$PROMPT"
|
| 50 |
+
echo "REPO_ROOT=$REPO_ROOT"
|
| 51 |
+
|
| 52 |
+
# Resolve python: JLENS_VENV, else python3 on PATH
|
| 53 |
+
if [[ -x "$JLENS_VENV/bin/python" ]]; then
|
| 54 |
+
PYTHON="$JLENS_VENV/bin/python"
|
| 55 |
+
elif command -v python3 >/dev/null 2>&1; then
|
| 56 |
+
PYTHON="$(command -v python3)"
|
| 57 |
+
else
|
| 58 |
+
echo "No Python found (set JLENS_VENV or install python3)" >&2
|
| 59 |
+
exit 1
|
| 60 |
+
fi
|
| 61 |
+
|
| 62 |
+
# --- Leg (c) FIRST so digest can go into tombstone notes ---
|
| 63 |
+
echo "--- leg c: JLens apply ---"
|
| 64 |
+
if [[ ! -f "$JLENS_SNAPSHOT_PY" ]]; then
|
| 65 |
+
echo "Missing jlens_snapshot.py at $JLENS_SNAPSHOT_PY" >&2
|
| 66 |
+
exit 1
|
| 67 |
+
fi
|
| 68 |
+
if [[ ! -f "$JLENS_LENS" ]]; then
|
| 69 |
+
echo "Missing lens weights at $JLENS_LENS" >&2
|
| 70 |
+
echo "Run: scripts/fetch_lens.sh" >&2
|
| 71 |
+
exit 1
|
| 72 |
+
fi
|
| 73 |
+
|
| 74 |
+
"$PYTHON" "$JLENS_SNAPSHOT_PY" \
|
| 75 |
+
--task-id "$TASK_ID" \
|
| 76 |
+
--prompt "$PROMPT" \
|
| 77 |
+
--out "$SNAP_JSON" \
|
| 78 |
+
--digest "$SNAP_DIGEST" \
|
| 79 |
+
--lens "$JLENS_LENS"
|
| 80 |
+
JLENS_DIGEST="$(head -1 "$SNAP_DIGEST")"
|
| 81 |
+
JLENS_BODY="$(cat "$SNAP_JSON")"
|
| 82 |
+
|
| 83 |
+
# --- Leg (a) tombstone with JLens digest in notes ---
|
| 84 |
+
echo "--- leg a: tombstone ---"
|
| 85 |
+
if [[ ! -x "$TOMBSTONE_APPEND" ]]; then
|
| 86 |
+
echo "Missing tombstone append at $TOMBSTONE_APPEND" >&2
|
| 87 |
+
exit 1
|
| 88 |
+
fi
|
| 89 |
+
COMBINED_NOTES="task_id=${TASK_ID}; jlens_digest=${JLENS_DIGEST}"
|
| 90 |
+
if [[ -n "$NOTES" ]]; then
|
| 91 |
+
COMBINED_NOTES="${COMBINED_NOTES}; ${NOTES}"
|
| 92 |
+
fi
|
| 93 |
+
"$TOMBSTONE_APPEND" "$TITLE" "$TASK" "$DONE" "$OUTCOME" "$MISTAKES" "$LESSONS" "$COMBINED_NOTES"
|
| 94 |
+
|
| 95 |
+
# --- Leg (b) covenant puts ---
|
| 96 |
+
echo "--- leg b: covenant ---"
|
| 97 |
+
if [[ ! -x "$PUT_COVENANT" ]]; then
|
| 98 |
+
echo "Missing put_covenant_memory.sh at $PUT_COVENANT" >&2
|
| 99 |
+
exit 1
|
| 100 |
+
fi
|
| 101 |
+
TASK_BODY="task_id=${TASK_ID}
|
| 102 |
+
title=${TITLE}
|
| 103 |
+
task=${TASK}
|
| 104 |
+
done=${DONE}
|
| 105 |
+
outcome=${OUTCOME}
|
| 106 |
+
prompt=${PROMPT}
|
| 107 |
+
jlens_digest=${JLENS_DIGEST}
|
| 108 |
+
"
|
| 109 |
+
JLENS_BODY="$JLENS_BODY" "$PUT_COVENANT" "$TASK_ID" "$TASK_BODY" \
|
| 110 |
+
"triad task memory for ${TASK_ID}"
|
| 111 |
+
|
| 112 |
+
# Proof: all three legs share task_id
|
| 113 |
+
echo "=== triad proof ==="
|
| 114 |
+
echo "task_id=$TASK_ID"
|
| 115 |
+
echo "jlens_snapshot=$SNAP_JSON"
|
| 116 |
+
rg -n --fixed-strings "$TASK_ID" "$REPO_ROOT/tombstone/tombstone.md" | tail -5 || {
|
| 117 |
+
echo "PROOF FAIL: task_id not in tombstone" >&2
|
| 118 |
+
exit 1
|
| 119 |
+
}
|
| 120 |
+
rg -n --fixed-strings "$TASK_ID" "$AI_MEMORY_ROOT" -g '*.md' | head -20 || {
|
| 121 |
+
echo "PROOF FAIL: task_id not in covenant memories" >&2
|
| 122 |
+
exit 1
|
| 123 |
+
}
|
| 124 |
+
rg -n --fixed-strings "$TASK_ID" "$SNAP_JSON" || {
|
| 125 |
+
echo "PROOF FAIL: task_id not in jlens snapshot" >&2
|
| 126 |
+
exit 1
|
| 127 |
+
}
|
| 128 |
+
echo "FIRE_TRIAD_OK task_id=$TASK_ID (tombstone + covenant + real jlens.apply)"
|
triad/put_covenant_memory.sh
ADDED
|
@@ -0,0 +1,63 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
#!/usr/bin/env bash
|
| 2 |
+
# Put covenant memories into the PRIVATE AI_MEMORY_ROOT.
|
| 3 |
+
# Never writes into vendor/covenant. Memories stay outside the public clone.
|
| 4 |
+
#
|
| 5 |
+
# Usage:
|
| 6 |
+
# ./put_covenant_memory.sh TASK_ID BODY [DESCRIPTION]
|
| 7 |
+
# Env:
|
| 8 |
+
# AI_MEMORY_ROOT, COVENANT_SRC, AGENT_NAME, MEMORY_TYPE
|
| 9 |
+
# JLENS_BODY — if set, also writes jlens-{TASK_ID} (type:jlens snapshot)
|
| 10 |
+
# CLI enum only allows user|feedback|project|reference;
|
| 11 |
+
# we use --type project and stamp type:jlens in body+description.
|
| 12 |
+
set -euo pipefail
|
| 13 |
+
DIR="$(cd "$(dirname "$0")" && pwd)"
|
| 14 |
+
REPO_ROOT="$(cd "$DIR/.." && pwd)"
|
| 15 |
+
|
| 16 |
+
# Prefer COVENANT_SRC; else vendor/covenant under repo root.
|
| 17 |
+
COVENANT_SRC="${COVENANT_SRC:-$REPO_ROOT/vendor/covenant}"
|
| 18 |
+
MAIN="$COVENANT_SRC/ai_memory_system/main.py"
|
| 19 |
+
export AI_MEMORY_ROOT="${AI_MEMORY_ROOT:-$REPO_ROOT/data/ai_memory_root}"
|
| 20 |
+
AGENT_NAME="${AGENT_NAME:-grok-bot}"
|
| 21 |
+
MEMORY_TYPE="${MEMORY_TYPE:-project}"
|
| 22 |
+
|
| 23 |
+
TASK_ID="${1:-${TASK_ID:-}}"
|
| 24 |
+
BODY="${2:-${BODY:-${DECISION_BODY:-}}}"
|
| 25 |
+
DESCRIPTION="${3:-${DESCRIPTION:-triad decision for ${TASK_ID:-}}}"
|
| 26 |
+
|
| 27 |
+
if [[ -z "$TASK_ID" || -z "$BODY" ]]; then
|
| 28 |
+
echo "Usage: $0 TASK_ID BODY [DESCRIPTION]" >&2
|
| 29 |
+
exit 1
|
| 30 |
+
fi
|
| 31 |
+
|
| 32 |
+
if [[ ! -f "$MAIN" ]]; then
|
| 33 |
+
echo "Covenant main.py not found at $MAIN" >&2
|
| 34 |
+
echo "Clone it: git clone https://github.com/LAWLESS1987/covenant \"$REPO_ROOT/vendor/covenant\"" >&2
|
| 35 |
+
echo "Or set COVENANT_SRC to an existing checkout." >&2
|
| 36 |
+
exit 1
|
| 37 |
+
fi
|
| 38 |
+
|
| 39 |
+
mkdir -p "$AI_MEMORY_ROOT"
|
| 40 |
+
|
| 41 |
+
safe_name() { printf '%s' "$1" | tr -c 'A-Za-z0-9._-' '-'; }
|
| 42 |
+
|
| 43 |
+
MEM_NAME="$(safe_name "task-${TASK_ID}")"
|
| 44 |
+
|
| 45 |
+
echo "AI_MEMORY_ROOT=$AI_MEMORY_ROOT"
|
| 46 |
+
python3 "$MAIN" --root "$AI_MEMORY_ROOT" put "$MEM_NAME" \
|
| 47 |
+
--description "$DESCRIPTION" \
|
| 48 |
+
--type "$MEMORY_TYPE" \
|
| 49 |
+
--body "$BODY" \
|
| 50 |
+
--agent "$AGENT_NAME"
|
| 51 |
+
|
| 52 |
+
if [[ -n "${JLENS_BODY:-}" ]]; then
|
| 53 |
+
JLENS_NAME="$(safe_name "jlens-${TASK_ID}")"
|
| 54 |
+
# Real CLI type enum has no 'jlens'; stamp type:jlens in description+body.
|
| 55 |
+
python3 "$MAIN" --root "$AI_MEMORY_ROOT" put "$JLENS_NAME" \
|
| 56 |
+
--description "type:jlens full Jacobian-lens snapshot for ${TASK_ID}" \
|
| 57 |
+
--type project \
|
| 58 |
+
--body "$JLENS_BODY" \
|
| 59 |
+
--agent "$AGENT_NAME"
|
| 60 |
+
echo "Wrote covenant memories: $MEM_NAME and $JLENS_NAME under $AI_MEMORY_ROOT"
|
| 61 |
+
else
|
| 62 |
+
echo "Wrote covenant memory: $MEM_NAME under $AI_MEMORY_ROOT"
|
| 63 |
+
fi
|
vendor/README.md
ADDED
|
@@ -0,0 +1,9 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# vendor/
|
| 2 |
+
|
| 3 |
+
Clone covenant here (not committed):
|
| 4 |
+
|
| 5 |
+
```bash
|
| 6 |
+
git clone https://github.com/LAWLESS1987/covenant vendor/covenant
|
| 7 |
+
```
|
| 8 |
+
|
| 9 |
+
Or set `COVENANT_SRC` to an existing checkout. The `vendor/covenant/` tree is gitignored.
|