Deep research
LiveDRBench
Open-web research scored as claim precision, recall, and F1, not as a long-form report grade.
As of 13 August 2026. Conditional pack. Java et al., arXiv 2508.04183. Claim precision, recall, and F1 stay separate. Rights review is required before commercial use.
LiveDRBench scores deep research as claim discovery, not as a long-form report grade. Precision asks whether stated claims are supported. Recall asks whether required claims were found. F1 is reported per category and is not a substitute for the weak slice.
The paper's claim-agreement scores use GPT-4o as a judge. Category F1 in the paper ranges from 0.02 to 0.72. Encrypted answers reduce casual leakage. The web changes, so a later rerun is a new observation, not a reproduction of May to June 2025.
- Scale
- 100 tasks across eight categories, originally grounded in May to June 2025 web state
- Provenance
- Java et al., arXiv 2508.04183. Code MIT; dataset CDLA-2.0. Internal rights review is required before commercial use.
- Contamination
- Moderate. Encrypted answers reduce casual leakage. Historical web state cannot be fully reproduced.
- Cannot establish
- Writing quality of a research memo, a commercial deep-research product score, or economic value.
Metrics, kept separate
- Claim precision
- Claim recall
- Claim F1
- Citation validity
- Completion rate
Paper table · cited paper · not an Antidote run · not a product score
Java et al., arXiv 2508.04183, Table 2. Claim F1 with GPT-4o agreement scores. Categories are not collapsed into a winner ranking.
| Category | OpenAI DR F1 | Perplexity DR F1 | Gemini DR F1 |
|---|---|---|---|
| SciFacts Geo | 72.1% | 18.6% | 31.6% |
| NovelDS ID | 66.7% | 63.3% | 40% |
| Entities | 60.3% | 44.7% | 33.8% |
| NovelDS peers | 58.5% | 31.1% | 33.8% |
| Flights | 54% | 36.2% | 26.1% |
| PriorArt | 53.9% | 41.9% | 8.2% |
| NovelDS extract | 47% | 33.3% | 34.5% |
| SciFacts Materials | 31.4% | 15% | 2.2% |
Paper table · cited paper · not an Antidote run · not a product score
Java et al., arXiv 2508.04183, Table 2, weighted-average row. Weights are question counts by category. Precision, recall, and F1 stay separate.
| System | Precision | Recall | F1 |
|---|---|---|---|
| OpenAI Deep Research | 62.9% | 51.9% | 55% |
| Perplexity Deep Research | 46.2% | 28.6% | 33.1% |
| Gemini Deep Research | 30.9% | 21.5% | 23.6% |
What the figures bound
OpenAI Deep Research has a weighted-average F1 of 0.550. Perplexity Deep Research is 0.331. Gemini Deep Research (2.5 Pro) is 0.236. SciFacts Geo is the high OpenAI cell at 0.721 F1. SciFacts Materials is the weak Gemini cell at 0.022 F1.
These numbers bound 2025 deep-research products on 100 public tasks. They do not grade memo prose, they do not score a client's research workflow, and they do not authorize commercial use without the rights review.
What this page does not conclude
- Writing quality of a research report
- A commercial product score before rights review
- Historical reproducibility of the 2025 web state
- A single LiveDRBench winner from mixed categories