Search agents
BrowseComp
Hard-to-find, short-answer facts that require persistent web browsing.
As of 13 August 2026. Conditional pack. Wei et al., arXiv 2504.12516. Official accuracy and calibration error stay separate. Search-agent diagnostic only.
BrowseComp is 1,266 hard-to-find short-answer web questions. Official accuracy is the load-bearing metric. Calibration error is a different estimand: a high-accuracy agent can still be overconfident. Do not average them.
Human trainers solved 29.2% of attempted items and gave up on 70.8% after two hours. That protocol is not the model table. Deep Research in the paper is trained on data that specifically teach BrowseComp-style browsing. Pin simple-evals despite deprecation, and do not publish plaintext answers.
- Scale
- 1,266 questions with encrypted short answers
- Provenance
- Wei et al., arXiv 2504.12516. Official simple-evals reference is MIT; pin the commit despite deprecation. Do not publish plaintext answers.
- Contamination
- High. Public since 2025. Encryption and a canary are controls, not proof that models have not seen examples.
- Cannot establish
- Target-specific diligence work, a domain workflow, or a product moat.
Metrics, kept separate
- Official accuracy
- Calibration error
- Completion rate
- Tool calls
- Latency
- Cost
Paper table · cited paper · not an Antidote run · not a product score
Wei et al., arXiv 2504.12516, Table 3. Deep Research is trained on data that specifically teach BrowseComp-style browsing. Human solve rate is a different protocol and is not in this table.
| System | Accuracy |
|---|---|
| Deep Research | 51.5% |
| OpenAI o1 | 9.9% |
| GPT-4o with browsing | 1.9% |
| GPT-4.5 | 0.9% |
| GPT-4o | 0.6% |
Paper table · cited paper · not an Antidote run · not a product score
Wei et al., arXiv 2504.12516, Table 3. Higher is worse. Browsing systems are more overconfident. Not averaged with accuracy.
| System | Calibration error |
|---|---|
| Deep Research | 91% |
| GPT-4o with browsing | 82% |
| GPT-4o | 69% |
| GPT-4.5 | 68% |
| OpenAI o1 | 65% |
What the figures bound
GPT-4o is 0.6% accuracy. Browsing lifts it to 1.9%. OpenAI o1, with no browsing, is 9.9%. Deep Research is 51.5%. Calibration error is worse with browsing: 82% for GPT-4o with browsing and 91% for Deep Research, against 69% for GPT-4o without browsing.
That bounds 2025 OpenAI browsing agents on a public web-fact pack. It is not a diligence-workflow score, not a domain research product, and not a reason to treat BrowseComp as a CDD proxy.
What this page does not conclude
- Target-specific diligence or filing research
- A domain-product or vendor score
- A calibrated-confidence claim from accuracy alone
- A composite browsing-agent ranking