FinanceHarness: Autonomous Financial Deep Research Framework
Abstract
Powered by advances in LLMs and autonomous agents, deep research has become one of the most widely adopted agentic products. However, most deep research systems write general-purpose reports, which are inadequate for financial deep research. Financial research demands specialized knowledge to analyze historical patterns and forecast upcoming events. Automating financial deep research therefore requires both a layered harness to drive the research agent and a verifiable, point-in-time benchmark that prevents leakage of future information. We present FinanceHarness, a harness that runs finance-oriented tools and practitioner-guided workflows, automating financial deep research end to end: environment and data construction, the agent execution loop, and reward modeling. We further propose FinanceGym, comprising thesis-driven research questions and rubrics that combine pre-cutoff and post-cutoff criteria. Professional expert validation yields an 82% pass rate. Even leading LLMs and agents score below 40% on the rubrics, showing that FinanceGym is challenging and leaves substantial headroom. With the same open-weight backbone, FinanceHarness improves the overall rubric score from 25.3% to 32.4%. FinanceHarness is available at https://github.com/Yijia-Xiao/FinanceHarness.
Community
We built a point-in-time financial deep research benchmark, featuring questions and rubrics generated through a rigorous, quality-controlled data pipeline. Additionally, we contracted financial experts to validate our data, with each spending an average of 1.2 hours on this meticulous review process. Leading LLMs such Opus-5 with our Finance Harness only score 44.9% on our leaderboard, showcasing the significant challenge our benchmark presents. We invite everyone to contribute to our leaderboard!
This is an automated message from the Librarian Bot. I found the following papers similar to this paper.
The following papers were recommended by the Semantic Scholar API
- FinResearchBench II: A Deep Research Benchmark with Consensus-Derived Gold Rubrics for Distinguishing Financial Report Quality (2026)
- FORCE-Bench: A Benchmark, Dataset, and Evaluation Harness for Agentic AI in Enterprise Finance (2026)
- ICBCBench: An Industry Consortium Benchmark for Financial Deep Research (2026)
- Deep Research in Physical Sciences: A Multi-Agent Framework and Comprehensive Benchmark (2026)
- DRFLOW: A Deep Research Benchmark for Personalized Workflow Prediction (2026)
- IPO Finance Agent: Benchmark of LLM Financial Analysts Beyond Finance Agent v2, with Automated Rubric Generation, on the SpaceX (SPCX) IPO (2026)
- How Do Tool-Augmented LLM Agents Perform on Real-World Energy Analytics Tasks? (2026)
Please give a thumbs up to this comment if you found it helpful!
If you want recommendations for any Paper on Hugging Face checkout this Space
You can directly ask Librarian Bot for paper recommendations by tagging it in a comment: @librarian-bot recommend
Models citing this paper 0
No model linking this paper
Datasets citing this paper 1
Spaces citing this paper 0
No Space linking this paper
Collections including this paper 0
No Collection including this paper