Create README.md (Model Card)
Browse files
README.md
CHANGED
|
@@ -1,3 +1,35 @@
|
|
| 1 |
-
---
|
| 2 |
-
license: mit
|
| 3 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: mit
|
| 3 |
+
tags:
|
| 4 |
+
- n8n
|
| 5 |
+
- llm-agents
|
| 6 |
+
- automation
|
| 7 |
+
- gemini-2.5-flash
|
| 8 |
+
---
|
| 9 |
+
|
| 10 |
+
# Minimal LLM Council (AI Audit System)
|
| 11 |
+
|
| 12 |
+
## Overview
|
| 13 |
+
This project implements a "Council of Agents" architecture to audit LLM responses. It uses 3 parallel agents to generate answers and 2 rubric-based judges to score them for **Safety** and **Clarity**.
|
| 14 |
+
|
| 15 |
+
**Files:**
|
| 16 |
+
* `LLM_Council_Audit_Workflow.json`: The n8n workflow file.
|
| 17 |
+
* `audit_log_proof.png`: Evidence of the persistent Google Sheets logging.
|
| 18 |
+
|
| 19 |
+
## Key Features
|
| 20 |
+
* **Multi-Agent Generation:** Orchestrates 3 Gemini agents (Cautious, Optimist, Logic).
|
| 21 |
+
* **Parallel Judgment:** Judges evaluate answers *without* generating new content.
|
| 22 |
+
* **Structured Output:** Returns a JSON object with `Confidence`, `Risks`, and `Citations`.
|
| 23 |
+
* **Persistent Logging:** Asynchronously logs all decisions to Google Sheets.
|
| 24 |
+
|
| 25 |
+
## Proof of Audit Log
|
| 26 |
+
The system maintains a permanent record of every judgment:
|
| 27 |
+

|
| 28 |
+
|
| 29 |
+
## Design Decision (Intentionally Not Automated)
|
| 30 |
+
**Decision:** I intentionally did **not** automate the final "blocking" action. The system detects risks and returns them to the client, but it does not silently refuse to answer.
|
| 31 |
+
|
| 32 |
+
**Why:** Automated safety gating is a "black box." By returning the raw *Risk Assessment* to the client, we align with the philosophy of "tying scale to proof." We prove the risk exists via the Audit Log, but we allow the client-side policy (or human reviewer) to decide the final display threshold. This ensures transparency and prevents over-censorship.
|
| 33 |
+
|
| 34 |
+
## Note on Model Selection
|
| 35 |
+
For this implementation, I utilized **Google Gemini 2.5 Flash** for all nodes to ensure high-speed inference and access to frontier model capabilities. The "independence" of the agents is achieved through strict **System Prompt Engineering** (Persona-based differentiation: Cautious, Optimist, and Logic) rather than architectural heterogeneity.
|