Finance reasoning
TAT-QA
Hybrid table and text questions on financial reports.
As of 13 August 2026. Published benchmark. Zhu et al., ACL 2021. Not a target-workflow result and not a vendor product score.
TAT-QA asks hybrid table-and-text questions on financial reports. Official exact match and official F1 are different estimands. Numeric-tolerance accuracy, scale accuracy, and a critical-error rate would be additional gates in a diligence run; they are not collapsed into F1 here.
The unit is the question. The paper's human mark is 50 hybrid contexts (301 questions), not the full test set. Arithmetic items are a separate slice. A later dated rerun of current models would still be calibration: the test ground truth has been public since January 2024.
- Scale
- 16,552 questions over 2,757 hybrid contexts
- Provenance
- NExT++ TAT-QA repository, ACL 2021 paper, CC-BY-4.0 data / MIT code.
- Contamination
- High. Public since 2021. Test ground truth released January 2024.
- Cannot establish
- Target-workflow replacement, adoption, economics, or a vendor moat.
Metrics, kept separate
- Official exact match
- Official F1
- Numeric-tolerance accuracy
- Scale accuracy
- Critical-error rate
Paper table · Zhu et al., 2021 · not an Antidote run · not a product score
Zhu et al., ACL 2021, Table 3, test split. TAGOP is the paper system. Human expert is 50 hybrid contexts (301 questions).
| System | Exact match | F1 |
|---|---|---|
| Human expert | 84.1% | 90.8% |
| TAGOP | 50.1% | 58% |
| NumNet+ V2 | 37% | 46.9% |
| TaPas WTQ | 16.6% | 22.8% |
| BERT-RC | 9.1% | 18.7% |
| HyBrider | 6.3% | 7.5% |
Paper table · Zhu et al., 2021 · not an Antidote run · not a product score
Zhu et al., ACL 2021, Table 4. Arithmetic is the weak slice. Metrics stay separate; there is no composite.
| Answer type | Exact match | F1 |
|---|---|---|
| Span | 54.1% | 67.9% |
| Spans | 60% | 75.1% |
| Counting | 62.5% | 62.5% |
| Arithmetic | 42.5% | 42.5% |
What the figures bound
TAGOP, the paper system, reaches 50.1% exact match and 58.0% F1 on test. Human experts are 84.1 / 90.8. Arithmetic is the weak slice at 42.5% exact match. Span and counting items are easier. That pattern is why a filing-research record keeps calculation errors separate from span extraction.
These numbers bound generic financial QA on a 2021 public pack. They do not measure AlphaSense, Hebbia, or any 2026 product. They do not imply that a higher F1 would substitute for an analyst process, change adoption, or create a moat.
ParseBench is the dated official leaderboard that does include GPT-5.6 Sol, Terra, Luna, and Claude Fable 5. No official TAT-QA table for those models is cited here.
What this page does not conclude
- Replacement of a current research workflow
- A vendor or product pass rate
- Operating economics or reviewer load
- A composite financial-QA ranking