File size: 7,418 Bytes
3e012b6
 
 
 
 
 
 
 
 
1bbf278
 
 
 
4be82f6
1bbf278
3e012b6
 
 
 
 
 
1bbf278
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
# HireTrace: Solution Video Walkthrough Script (≀5 Minutes)
**micro1 Agentic Workflows Hackathon**
**Tagline:** *"Every recommendation traces back to evidence."*

---

## Video Outline & Timing
| Segment | Time | Screen Visual | Narration & Key Points |
|---|---|---|---|
| **1. Problem Framing** | 0:00 - 0:40 | β€’ Architecture diagram & fragmented candidate docs (JD, CV, interview notes, assessment report, RFC) | β€’ "Hiring teams evaluate candidates across disconnected silos: CVs, interviews, coding assessments, and project docs."<br>β€’ "Reviewing each in isolation causes teams to miss cross-source contradictions or overweight single signals."<br>β€’ "HireTrace introduces evidence-first assessment: answering not *'Should we hire?'*, but *'How well does the evidence support the role, and where does it disagree with itself?'*" |
| **2. Zero Cost & Local Architecture** | 0:40 - 1:05 | β€’ Terminal showing Ollama running `qwen2.5:3b` at `localhost:11434`<br>β€’ Network audit showing $0.00 external API calls<br>β€’ Architecture diagram showing Pydantic repair loop & 3x self-consistency voting | β€’ "Hard architectural constraint: **$0.00 cost**."<br>β€’ "Zero paid API calls, zero cloud subscriptions. All inference runs locally via Ollama (`qwen2.5:3b`), and all retrieval uses local `faiss-cpu`."<br>β€’ "To ensure local 3B open-weights reliability, HireTrace enforces JSON-schema constrained decoding with Pydantic self-repair loops and 3x self-consistency majority voting with tie-break fallbacks." |
| **3. Baseline A & B Comparison** | 1:05 - 1:45 | β€’ Code walkthrough of `/baseline/rubric_scorer.py` and `/baseline/naive_llm.py` | β€’ "We benchmark against two realistic baselines:"<br>β€’ "β€’ **Baseline A (Deterministic Resume Rubric)**: A deterministic rubric scoring Open Source (0–35), Self Projects (0–30), Production (0–25), Tech Skills (0–10), and Bonus Points (0–20). Fast, but blind to cross-source contradictions."<br>β€’ "β€’ **Baseline B (Naive Single-Prompt LLM)**: Concatenates all documents into local Ollama in one prompt. While it can produce apparently grounded individual claims, it lacks the cross-source verification machinery needed to reliably detect contradictions." |
| **4. The Centerpiece Case Walkthrough** | 1:45 - 3:00 | β€’ **HireTrace Interactive Dashboard (`http://127.0.0.1:8080`)**<br>β€’ Hovering over candidate dots on the **Interactive 2D Quadrant Canvas** (showing real-time tooltips & crosshairs at 70/70 thresholds)<br>β€’ Selecting **Alexander Sterling** (`case_15_deceptive_centerpiece`)<br>β€’ Side-by-side discrepancy diffs with **verbatim quote highlighting** and **'πŸ” View in Dossier'** jump navigation<br>β€’ Priority Questions panel with **'πŸ“‹ Copy All Questions'** one-click export | β€’ "Here is our centerpiece demo: Alexander Sterling."<br>β€’ "On paper, his CV looks stellar: 8+ years experience, 250 GitHub stars, claiming: *'Led migration of legacy monolithic core to Apache Kafka for a 7-person team.'*"<br>β€’ "Baseline B assigns Alexander 85.0 and misses the deception entirely. But HireTrace's decomposed multi-agent pipeline catches it:"<br>β€’ "1. **Requirement Mapping Agent** extracts 5 core requirements from the JD."<br>β€’ "2. **FAISS Retrieval Layer** indexes all evidence spans across CV, interview, and project docs."<br>β€’ "3. **Cross-Source Verification Agent** isolates 5 critical discrepancies across tenure and project authority:"<br>&nbsp;&nbsp;&nbsp;&nbsp;β€’ *Discrepancy 1 (Tenure)*: CV claims 3.5 years at FinFlow; interview notes prove he joined ~18 months ago.<br>&nbsp;&nbsp;&nbsp;&nbsp;β€’ *Discrepancies 2–5 (Leadership & Authority)*: CV claims he *led* the 7-person Kafka migration and authored the master architectural blueprint; Project RFC documents prove Dr. Robert Vance was Principal Architect/Lead and Alexander was a junior contributing member writing Python serialization adapters.<br>β€’ "Notice our new **'View in Dossier'** button: clicking it jumps directly to the exact source tab and highlights the raw text."<br>β€’ "4. **Claim Critic Agent** verifies 100% exact substring containment in raw sources, distinguishing asserted facts from synthesized inferences."<br>β€’ "5. **Recommendation Writer Agent** plots him on our **2D Quadrant**: Role Fit (82.7/100) vs. Evidence Consistency (0.0/100) -> **[REVIEW REQUIRED]**."<br>β€’ "Recruiters click **'Copy All Questions'** to immediately get an interview debrief template for panel review." |
| **5. Audit & Governance Tab: Fairness, Calibration & Defense** | 3:00 - 4:00 | β€’ Switching to **'πŸ›‘ Audit & Governance'** tab<br>β€’ Highlighting the 4 audit cards: EEOC Fairness, Claim Grounding, Uncertainty Calibration, Adversarial Defense | β€’ "In enterprise hiring, compliance and governance are non-negotiable. Our new **Audit & Governance** tab exposes live telemetry:"<br>β€’ "β€’ **Algorithmic Fairness & Parity:** Audited under EEOC Uniform Guidelines (4 CFR Part 60) across 44 counterfactual candidate dossiers and 11 demographic groups. Yields **0.00 pts score drift** and **1.0000 Disparate Impact Ratio (EEOC compliant)**."<br>β€’ "β€’ **Evidence Grounding & Critic:** 100% quote containment, 100% citation span validity, 66.7% claim grounding on 15 synthetic cases."<br>β€’ "β€’ **Uncertainty Calibration:** Brier score 0.2281 and Expected Calibration Error of 23.9% across confidence bins."<br>β€’ "β€’ **Adversarial Defense:** 100% containment across 8 evaluated attack vectors on synthetic benchmarks (prompt injection, delimiter escapes, conversational jailbreaks, and credential/metric fabrications)." |
| **6. Evaluation Benchmark & Closing Hot Take** | 4:00 - 5:00 | β€’ Benchmark matrix tab<br>β€’ 50-case benchmark results<br>β€’ Final hot take slide | β€’ "Across our expanded 50-case benchmark suite spanning 5 engineering domains:"<br>β€’ "β€’ **Spearman Rank Correlation (ρ):** Baseline A (0.579) -> **HireTrace Agent (0.816)** (95% CI: [0.446, 0.983])."<br>β€’ "β€’ **Contradiction Detection:** **100.0% precision, 100.0% recall, 1.000 F1** (zero false alarms on clean controls)."<br>β€’ "β€’ **Reviewer Time Saved:** Under our standardized cognitive reviewer-effort model, estimated review time falls from 18.0 min to 3.5 min (**80.6% estimated time saved**)."<br>β€’ "**Closing Hot Take:** *'Verification establishes consistency, not physical truth.'* If a CV and interview agree, that proves internal consistency across documents β€” not physical truth. That is why automated agents must never make autonomous hiring decisions, and why HireTrace traces every claim to evidence for human reviewers." |

---

## Production Tips for Recording
1. **Resolution:** 1920x1080 (1080p).
2. **Terminal:** Run `python -m ui.server 8080` and show the clean web UI at `http://127.0.0.1:8080`.
3. **Tabs to Demonstrate:**
   - Tab 1: **Candidate Assessment** (Scatter quadrant canvas with crosshairs and hover tooltip, Alexander Sterling centerpiece, Priority Questions + Copy button).
   - Tab 2: **Discrepancy Diffs** (Side-by-side quote highlight and 'View in Dossier' button).
   - Tab 3: **Dossier Viewer** (Multi-doc tabs).
   - Tab 4: **Benchmark Matrix** (Spearman rho, Brier score, ECE).
   - Tab 5: **Audit & Governance** (EEOC fairness, grounding fidelity, adversarial defense).
4. **Ollama:** Confirm Ollama daemon is running with `qwen2.5:3b` or running in offline mock mode ($0.00 cost).