The verification point is a really important one. We have to be careful not to accidentally optimize agents toward the shortest trajectory. An extra call that verifies state can be much more valuable than an agent stopping early.
Golda Manuel PRO
phranzia
AI & ML interests
Founder of Quantiles, an open-source, local-first AI evaluation framework
Recent Activity
commentedon their article 3 days ago
Evaluating AI Agent Tool Use updated a Space 4 days ago
quantiles/README upvoted an article 4 days ago
Evaluating AI Agent Tool UseOrganizations
commented on Evaluating AI Agent Tool Use 3 days ago
updated a
Space 4 days ago
upvoted an article 4 days ago
Article
Evaluating AI Agent Tool Use
phranzia
• • 2 published an article 4 days ago
Article
Evaluating AI Agent Tool Use
phranzia
• • 2 upvoted an article 4 days ago
Article
When Benchmark Numbers Become Marketing
DedeProGames
• • 5 upvoted an article 11 days ago
Article
Measuring Agent Recovery
phranzia
• • 2 published an article 11 days ago
Article
Measuring Agent Recovery
phranzia
• • 2 upvoted 2
articles 11 days ago
Article
BenchMIRT: What are LLM benchmarks actually measuring?
allenai
• • 27 Article
Your Agent Aced the Task. Will It Do It Again?
ibm-research
• • 114 upvoted an article 14 days ago
Article
Measure, Understand, and Improve Agent Latency
phranzia
• • 1 published an article 14 days ago
Article
Measure, Understand, and Improve Agent Latency
phranzia
• • 1 upvoted an article 24 days ago
Article
Using Execution Traces to Evaluate AI Agent Behavior
phranzia
• • 2 published an article 24 days ago
Article
Using Execution Traces to Evaluate AI Agent Behavior
phranzia
• • 2 upvoted 2
articles 24 days ago
Article
Measuring benchmark optimization in speech recognition
- +5
tlebryk02, bezzam, aliceebaird, dayllon, jpc, jens-hume-ai, tzirakis
• • 67 Article
Inside three.ws: The Open-Source Stack That Gives AI Agents a Body, a Brain, a Wallet, and a Job
three-ws
• • 47 upvoted 2
articles about 1 month ago
Article
The Best Open Source and Open-Weight LLM Models to Run Locally in 2026
daya-shankar
• • 25 Article
We changed one line and the benchmark score moved 0.21 AUROC
FINAL-Bench
• • 15 published an article about 1 month ago
Article
How to Evaluate AI Agents
phranzia
• upvoted 2
articles about 2 months ago
Article
Beyond the Aggregate Score: Per-Country Domain Shift in the GWHD Wheat Head Detection Model Zoo
dronefreak
• • 7 Article
Build an AI Evaluation from a Hugging Face Dataset Without Writing Python
phranzia
• • 3