Finance reasoning
FinQA
Numerical reasoning as executable programs over financial reports.
As of 13 August 2026. Published benchmark. Chen et al., EMNLP 2021. Program accuracy and execution accuracy stay separate.
FinQA scores numerical reasoning as an executable program over financial reports. Execution accuracy asks whether the derived number is right. Program accuracy asks whether the program itself matches. Direct generation of the number, in the paper, is near zero. Those are not averaged.
The human expert mark is 91.16% execution accuracy. The non-expert crowd is 50.68%. A diligence record cares about critical numerical error against an expert key, not a blended F1 against crowd performance. Pin a revision after the 2022 preprocessing fix before any rerun.
- Scale
- About 8,000 questions over about 2,800 reports
- Provenance
- FinQA project site, EMNLP 2021 paper, CC-BY-4.0 data / MIT code.
- Contamination
- High. Public since 2021. Pin a revision after the 2022 preprocessing fix.
- Cannot establish
- Document parsing, target workflow performance, or economic value.
Metrics, kept separate
- Execution accuracy
- Program accuracy
- Answer accuracy
- Supporting-fact recall
Paper table · Chen et al., 2021 · not an Antidote run · not a product score
Chen et al., EMNLP 2021, Table 2. FinQANet (RoBERTa-large) is the paper system. Human expert execution accuracy is 91.16%; the non-expert crowd is 50.68%.
| System | Execution accuracy | Program accuracy |
|---|---|---|
| Human expert | 91.16% | 87.49% |
| FinQANet RoBERTa-L | 61.24% | 58.86% |
| FinQANet BERT-L | 53.52% | 51.62% |
| Crowd | 50.68% | 48.17% |
| NeRd BERT-base | 48.57% | 46.76% |
| Longformer base | 21.9% | 20.48% |
What the figures bound
FinQANet with RoBERTa-large reaches 61.24% execution accuracy and 58.86% program accuracy. BERT-large is 53.52 / 51.62. Longformer base is 21.90 / 20.48. The crowd sits at 50.68% execution accuracy, close to several 2021 neural baselines and far from experts.
Use this pack as a regression check on calculation-heavy research and as a reminder that getting the number right is not the same as emitting a faithful program. Do not treat it as document parsing, as a product score, or as economic value.
What this page does not conclude
- Document-parse quality
- Target-workflow performance on live filings
- A vendor economics case
- An averaged execution-plus-program winner