Are the Financial Reasoning from LLMs Credible? A Real World Test over Long-Horizon Statements

TL;DR: We introduce FinIndices, a long-context financial reasoning benchmark built from real Chinese listed-company financial statements. It evaluates whether LLMs can reliably compute financial indicators and generate structured multi-period tables under realistic accounting constraints, especially those defined by Chinese Accounting Standards (CAS).

Project / dataset: https://huggingface.co/datasets/Findata/Finindice

Most existing financial benchmarks focus on short-context QA, multiple-choice financial knowledge, or simplified table reasoning. However, real financial analysis is much more demanding: models must read full financial statements, identify the correct line items, distinguish consolidated vs. parent-company caliber, align different reporting periods, handle cumulative vs. standalone quarter figures, reconcile stock and flow variables, and finally produce an audit-ready table.

A key feature of FinIndices is its focus on authentic Chinese financial reports and CAS-based accounting semantics. Many indicators cannot be solved through keyword matching. For example, models must correctly handle:

• Interest-bearing vs. non-interest-bearing liabilities when computing invested capital. Under Chinese reporting taxonomy, items such as accounts payable, notes payable, contract liabilities, taxes payable, and employee benefits payable generally differ from borrowings, bonds payable, lease liabilities, and current maturities of non-current liabilities.
• Equity attributable to the parent company vs. total shareholders’ equity, especially when minority interests matter.
• CAS-specific income statement items such as investment income, fair-value change income, credit impairment losses, asset impairment losses, and non-operating gains/losses.
• Adjusted asset/equity metrics requiring deductions of intangible assets, goodwill, deferred tax assets, long-term deferred expenses, etc.
• Stock-flow caliber alignment, e.g., matching income-statement flow variables with average balance-sheet stock variables.

FinIndices contains two task types:

1. Single-Index: compute one financial indicator from long, distractor-rich statements.
2. Table-Index: compute multiple indicators across multiple periods and output structured HTML/JSON/table results.

The second task is particularly important because real financial workflows usually require structured comparison tables rather than a single number.

Our experiments reveal two major bottlenecks:

• Knowledge Bottleneck: Without explicit formula hints, even strong LLMs suffer large performance drops, suggesting they have not fully internalized accounting formulas, CAS taxonomy, and financial-statement semantics.
• Structural Bottleneck: Table generation is not just a formatting issue. When models must fill multi-metric, multi-period tables, their reasoning often degrades. We observe temporal misalignment, stock-flow mismatch, wrong use of beginning balances, and shortcuts that ignore CAS-based adjustments.

We also include adversarial cases where the necessary reports are missing. In these cases, the correct answer is “insufficient information”, not a hallucinated number. This is essential for trustworthy financial agents.

All formulas are curated from authoritative sources, including Chinese Accounting Standards, financial statement analysis textbooks, and corporate finance references. The test set is manually audited by domain experts for accounting validity and numerical correctness.

We hope FinIndices helps the community evaluate LLMs beyond short financial QA, toward reliable, long-context, CAS-aware financial reasoning and table-centric data processing.