Finance reasoning
ConvFinQA
Conversational numerical reasoning over financial reports, with history across turns.
As of 13 August 2026. Published benchmark. Chen et al., EMNLP 2022. Execution accuracy and program accuracy stay separate. Conversational history is the treatment.
ConvFinQA turns FinQA multi-hop items into 3,892 conversations. Execution accuracy asks whether the number is right. Program accuracy asks whether the program matches. Later turns and hybrid conversations are harder slices. They are not averaged into one conversation score.
The paper systems are 2022 neural-symbolic models and GPT-3 prompting. No published official TAT-QA-style table for GPT-5.6 or Fable 5 is cited here.
- Scale
- 3,892 conversations, 14,115 questions
- Provenance
- Chen et al., EMNLP 2022. ConvFinQA repository. Same family as FinQA; treat data and code licenses separately.
- Contamination
- High. Public since 2022. Built by decomposing FinQA items.
- Cannot establish
- A live analyst conversation, product performance, or economic value.
Metrics, kept separate
- Execution accuracy
- Program accuracy
- Turn-level accuracy
- Hybrid-conversation accuracy
Paper table · Chen et al., 2022 · not an Antidote run · not a product score
Chen et al., EMNLP 2022, Table 3. FinQANet (RoBERTa-large) is the paper system. Human expert is 200 sampled questions. GPT-5.6 and Fable 5 are not in this table.
| System | Execution accuracy | Program accuracy |
|---|---|---|
| Human expert | 89.44% | 86.34% |
| FinQANet-Gold RoBERTa-L | 77.32% | 76.46% |
| FinQANet RoBERTa-L | 68.9% | 68.24% |
| FinQANet RoBERTa-base | 64.95% | 64.16% |
| FinQANet BERT-L | 61.14% | 60.55% |
| T5 large | 58.66% | 57.05% |
| GPT-2 medium | 58.19% | 57% |
| FinQANet BERT-base | 55.03% | 54.57% |
| Crowd | 46.9% | 45.52% |
Paper table · Chen et al., 2022 · not an Antidote run · not a product score
Chen et al., EMNLP 2022, Table 4. Hybrid second-part turns are the weak slice. Metrics stay separate.
| Slice | Execution accuracy | Program accuracy |
|---|---|---|
| Number selection | 82.54% | 82.34% |
| Simple conversations | 72.37% | 72% |
| Full test | 68.9% | 68.24% |
| Hybrid first part | 68.11% | 66.54% |
| Program questions | 62.14% | 61.26% |
| Hybrid conversations | 60.99% | 59.7% |
| Hybrid second part | 52.38% | 51.43% |
What the figures bound
FinQANet with RoBERTa-large reaches 68.90% execution accuracy and 68.24% program accuracy. Human experts are 89.44 / 86.34. The crowd is 46.90 / 45.52. Hybrid conversations drop to 60.99% execution accuracy; the second part of a hybrid conversation is 52.38%.
Number-selection turns are easier (82.54%) than program turns (62.14%). That is a 2022 bound on conversational financial QA, not a product score.
What this page does not conclude
- A live analyst conversation or product surface
- A 2026 frontier ranking
- An averaged execution-plus-program winner
- Operating economics of a research chat