- README: 14 -> 16 routes in architecture diagram; planner box now
also names substance-use and privacy-confidentiality. Added a
second-hardening-pass paragraph documenting session isolation,
natural-language intent detection, and the two new routes.
- ERROR_ANALYSIS: new Category 7 (six entries) for the gaps caught
during the pre-recording audit (stale session_id, multi-word
affirm bypass, greeting-consumed slot, turn_index saturation,
substance-use unsurfaced, confidentiality deflection). Renumbered
the previous "what we cannot do" section to Category 8.
Strict cleanup: remove internal handoff docs, neutralize teammate/sprint naming in public docs
V4_PROJECT_AUDIT_2026_05_10.md
- Replaces 4-day-stale OPUS_FULL_PROJECT_AUDIT_2026_05_06.md as the
current full-handoff document for graders and reviewers.
- Architecture diagram updated to reflect V4 layers including
authority-misconduct route, conversation-history threading,
sycophancy guard, F-1 decay, minimal-affirm + incomplete handlers,
length cap.
- Evaluation summary table with all current numbers at commit c8cd802.
- Full V3-to-V4.4 commit timeline.
- Risk register with mitigations.
- Honest "what's missing" list per goal level (class demo / paper /
CC pilot / production).
ERROR_ANALYSIS.md
- Categorized failure modes observed during development:
1. Routing failures (generic prompts -> general_student_support;
slang/emoji misroute)
2. Context drift across turns (F-1 hijack; minimal-response loop;
incomplete message confident routing)
3. Authority-figure misconduct misrouting (the bank-rob case that
drove the V4.2 redesign)
4. Sycophancy under …
V4.4 Tier 2: safety-pipeline UI visualization, paper results section, demo script, mobile CSS, HIPAA + privacy docs
Safety-pipeline visualization in the right panel
- New 6-chip row under "Live thread" showing each layer's state per turn:
S1 (Stage-1 lexical precheck) · Route (classifier) · Reg (registry) ·
Stage (L/P/O/C/X) · Reph (rephraser provider) · Guard (output guard).
- Per-chip color states: on (accent), hit (red — intercepted/blocked),
skip (gray — intentionally bypassed for this stage), off (dimmed).
- Hover tooltips carry the per-layer reason ("Stage-1 INTERCEPTED:
imminent_safety", "Route: exam_stress (conf 0.82, ML)", "12 verified
UMD/national resources surfaced", etc.).
- Demo value: graders + CC clinicians can SEE the defense-in-depth story
fire 6 different ways without opening Diagnostics. This is the
highest-leverage demo addition: the safety architecture is now legible
in real time.
PAPER_FRAMING: full Evaluation Results section
- Headline guarded-vs-unguarded comparison table (0/28 vs 9/28, same
Llama model, non-overlapping CIs).
- Per-layer ablation table with…
V4.4 Tier 1: regex hardening, clarify-stage enum, stat caveats, PDF unicode, retry/backoff, reproducibility doc
Authority-misconduct regex hardening
- _AUTHORITY_NOUNS split into word-bounded set (matched with \b...\b) and
multi-word phrases (substring). The literal-spaces hack (" ta ", " ra ")
that prevented matches like "stay" / "data" / "fragrance" is replaced
by a compiled regex with word boundaries.
- Verified: 6/6 should-fire prompts still fire (professor inappropriate,
TA touched, advisor retaliation, RA confidentiality, coach racist,
counselor drop-out). 5/5 should-not-fire prompts no longer false-fire
("stay focused on data", "fragrance allergy", "deportation rumor",
"data analysis", "stayed up late").
clarify stage exported as named constant
- response_planner.CLARIFY = "clarify". core.py now imports and uses the
constant instead of the bare string. Removes a stringly-typed value
that would silently rot under refactor.
Statistical caveats in PAPER_FRAMING
- New "Statistical-power notes per metric" subsection lists every headline
with its CI95 and a "do not over-in…
Prompt-injection probe sweep (P0.1)
- eval/sweep_prompt_injection.py: 16 probes across 9 attack categories
(direct override, role replacement, pseudo-system message, system-prompt
extraction, DAN jailbreak, indirect / fake-context, recursive injection,
authority spoof, inverse instruction).
- Result: 16/16 clean, 0 leaks. The planner-as-trust-boundary architecture
is structurally injection-resistant: the planner authors from keywords
(not user content) and the rephraser only paraphrases the planner's
safe template. The user message can't redirect the planner's choice
of route or template via instruction injection.
Input length cap (P0.2)
- EmpathRAGCore.MAX_USER_MESSAGE_CHARS (default 2000, configurable via
EMPATHRAG_MAX_USER_MESSAGE_CHARS env). Messages longer than this are
truncated in _plan_turn and short-circuited to a length-specific
clarify template ("Your message was long enough that I only have the
first part. Could you say which piece feels most pressing…
V4.2 part 1: fix broken ISSS URL, ship URL audit + Karthik data brief
Root cause of demo 404
- UMD restructured the global office from globalmaryland.umd.edu to
marylandglobal.umd.edu and reorganized the path.
- Three service_graph rows (umd_isss, umd_isss_academic_difficulty,
umd_isss_opt_cpt) plus the INTERNATIONAL_SOURCE_HINT in response_planner.py
all pointed at the dead old path -> "Open ↗" on the ISSS source card 404'd.
- Replaced with https://isss.umd.edu/ (the canonical that auto-redirects to
the new path so we survive future UMD reorgs).
- Bumped last_verified on the 3 rows; preserved migration note for future
maintainers.
URL audit pipeline
- eval/audit_resource_urls.py walks every service_graph URL + every corpus
URL, HEADs/GETs each with a polite UA, reports HTTP status and redirects,
writes a timestamped Markdown report.
- Current state after fix: 60/63 live. The 3 SAMHSA "errors" are TLS
handshake quirks with python urllib (browsers load them fine; Karthik's
audit confirmed live).
- .gitignore: timestamped audit reports e…
V4.1: fix repetition/scroll/topbar, add PDF export, counselor-assist note
Bug fixes
- Flatten topbar wrapping so Save / Reset buttons render reliably; the
Column+Row nesting was getting collapsed on narrower viewports.
- Chatbot auto-scroll: install a MutationObserver via gr.Blocks(js=) that
pins the chat surface to the bottom on every streaming token update.
- Minimal-response repetition: "yes" / "no" / "ok" / "maybe" with no other
content used to re-render the same OFFER template (because the planner
saw no new intent). Now routes to a "clarify" stage with a short
open-ended question, e.g. "Which part of what we just talked about feels
most useful to start with?" Output guard skipped for clarify stage.
Counselor-friendly PDF export
- src/pipeline/support_plan.py adds build_support_plan_pdf via fpdf2.
Same content as the Markdown version, formatted for print: title block,
scope disclaimer in an inset box, section heads in accent color,
per-bullet user-message echo, deduped routes / actions / resources.
- Topbar now exposes two download bu…
V4: streaming, controlled paraphrasing, support plan, voice, sweeps
