Multi-document analysis for diligence and research. The data room is the customer's. Orchestration is reconstructable in kind; decay versus long-context models is open.
As of 19 August 2026. This memo covers issue extraction and synthesis across a large diligence document set. No live-product test has been run.
The data room is the customer's. Orchestration is reconstructable in kind.
Diligence documents are the customer's. There is no exclusive corpus. Long-context dump, chunk-and-cite, and agent decomposition are known treatments. Hebbia's remaining questions are faithfulness, an isolated increment over those treatments, and decay versus current long-context models. Public parser and QA packs bound layers of that work. They do not stand in for a PE room.
Citation faithfulness, agent-versus-retrieval attribution, and full cost per accepted extract remain open. Cross-engagement memory without reuse rights is not a network effect. VDRs can attach extraction to the room the buyer already has.
Exposed vs scarce. The data room is the customer's. Long-context dump, chunk-and-cite, and agent decomposition are reconstructable in kind.
Still untested. Span-level citation faithfulness and an isolated increment over chunk-and-cite have not been measured.
The comparison that would change the call. Dump versus chunk-and-cite versus agent decomposition on a sealed PE pack, tools and budget frozen.
Decision questions
On a frozen diligence pack, does Hebbia clear a prespecified extract-and-cite gate relative to the current reading process?
What isolated increment comes from agent decomposition versus chunk-and-cite versus long-context dump?
Are citations faithful to source spans, and are unsupported claims caught as a distinct error?
What is full cost per accepted extract, including tool calls, failures, and reviewer minutes?
A diligence extract can be fluent and still omit a page, hallucinate a number, cite the wrong span, or bury a strikethrough price. Those failures belong to parse, retrieval, generation, and review, and they are scored separately. Repeated samples of one document are nested trials, not extra cases.
Public packs calibrate the layers. ParseBench is the preferred parser diagnostic because it is deterministic and multi-dimensional. TAT-QA and the contract packs bound slices of PE work. They are not a data room. DocVQA is held out pending a commercial data license. A Hebbia run would still need a sealed corpus, an independent key, and exportable traces.
Workflow breakdown
Each step of the job is mapped to a published benchmark or another reputable quantitative measure only where the mapping is honest. A public figure is a floor or a labor-market context on that task type. It is never a score of this product.
Third-party harness. Rubric and presentation stay separate.
Issue extraction on a sealed customer room
None
No honest public floor
Needs the customer's documents, a written issue list, and independent extracts.
Findings at a glance
Capability, cost, and defensibility are assessed separately. They are not averaged into a score. Where evidence is missing, the finding stays open.
Status
Area
Finding
Open
Workflow capability
Multi-agent marketing copy is not a pass rate. No frozen Hebbia document-set evaluation is in this report.
Open
Operating economics
Token, tool, and reviewer cost per accepted diligence extract are unknown.
Judgment
Defensibility
The data room is the customer's. There is no exclusive corpus. Orchestration is reconstructable in kind. Decay versus current long-context models is open. VDRs can attach extraction to the room the buyer already has.
Vendor-stated
Vendor claims and open questions
Public materials describe multi-document analysis for diligence, research, and legal review.
Workflow capability
Dumping a corpus into long context is not a retrieval system
Frontier context windows make a long-context dump look like a product. It is a distinct treatment: no chunking policy, no citation budget, no agent plan. A generic chunk-and-cite stack is another treatment. Agent decomposition is a third. Crediting Hebbia for all three at once is a bundled comparison.
No frozen Hebbia document-set evaluation is in this report. Marketing copy about multi-agent analysis is not a pass rate, not a critical-miss rate, and not a citation-faithfulness audit.
Current parsers and VLMs fail different dimensions
On ParseBench, tables are relatively strong for several current systems and visual grounding is not. Gemini 3 Flash (Thinking High) is 91.50% tables and 59.77% grounding at 2.41 cents per page. GPT-5.6 Sol is 89.34% tables and 25.09% grounding at 5.87 cents. Claude Opus 4.8 is 89.65% tables and 18.69% grounding at 7.30 cents. LlamaParse Agentic is 90.74% tables and 80.62% grounding at 1.25 cents. Pulse Ultra 2 leads charts (90.82%) and is expensive at 15 cents per page.
That scatter is the efficiency point: a higher list price does not buy grounding, and a specialist parser can beat a frontier VLM on the dimension that auditability needs. None of this is a Hebbia result. It is the floor a diligence stack has to clear before anyone talks about memo quality.
Financial QA and contract slices do not become a data room
Public filing-QA and contract packs bound slices of PE work — hybrid questions, clause location, contradiction. They are 2021 diagnostics, not a data room, and not a Hebbia score.
A citation can match a document and still miss the operative span
Faithfulness is a span-level audit on held-out documents, not a user-facing footnote count. A citation can match a file and still miss the operative sentence, bury a strikethrough price, or support a fluent extract that the source does not. No such audit of Hebbia is in this report.
Published benchmarks
ParseBench, TAT-QA, CUAD, ContractNLI, and AA-Briefcase bound parse, financial QA, contract extraction, and multi-file knowledge work. They do not reproduce a PE data room.
ParseBench
Diligence starts with whether the PDF parse is faithful. Five official dimensions stay separate. A parser leaderboard is not a memo-quality score.
Third-party leaderboard · not an Antidote run · not a product score
Current frontier VLMs on ParseBench
Official ParseBench leaderboard, observed 13 August 2026. Five dimensions kept separate. Gemini 3 Flash is Thinking High. GPT-5.6 Sol and Fable 5 are Reasoning None / default API settings in the source.
Dimension
Gemini 3 Flash
GPT-5.6 Sol
Claude Fable 5
Claude Opus 4.8
Tables
91.5%
89.34%
89.79%
89.65%
Charts
64.79%
59.57%
52.21%
49.75%
Faithfulness
90.87%
86.17%
90.02%
89.02%
Formatting
68.31%
50.42%
72.62%
71.38%
Grounding
59.77%
25.09%
49.24%
18.69%
Third-party leaderboard · not an Antidote run · not a product score
Specialist parsers vs a frontier VLM
Same ParseBench leaderboard. LlamaParse Agentic, Pulse Ultra 2, and Reducto Agentic against Gemini 3 Flash Thinking High. The source Overall column is omitted.
Dimension
LlamaParse Agentic
Gemini 3 Flash
Pulse Ultra 2
Reducto Agentic
Tables
90.74%
91.5%
75.45%
80.42%
Charts
78.11%
64.79%
90.82%
73.4%
Faithfulness
89.68%
90.87%
79.49%
86.37%
Formatting
85.24%
68.31%
73.05%
57.6%
Grounding
80.62%
59.77%
66.56%
67.07%
TAT-QA
Hybrid financial QA is a relevant slice of PE document work. It is still not a data-room workflow.
Paper table · Zhu et al., 2021 · not an Antidote run · not a product score
TAT-QA test exact match and F1
Zhu et al., ACL 2021, Table 3, test split. TAGOP is the paper system. Human expert is 50 hybrid contexts (301 questions).
System
Exact match
F1
Human expert
84.1%
90.8%
TAGOP
50.1%
58%
NumNet+ V2
37%
46.9%
TaPas WTQ
16.6%
22.8%
BERT-RC
9.1%
18.7%
HyBrider
6.3%
7.5%
Paper table · Zhu et al., 2021 · not an Antidote run · not a product score
TAGOP by answer type on the test set
Zhu et al., ACL 2021, Table 4. Arithmetic is the weak slice. Metrics stay separate; there is no composite.
Answer type
Exact match
F1
Span
54.1%
67.9%
Spans
60%
75.1%
Counting
62.5%
62.5%
Arithmetic
42.5%
42.5%
CUAD
Clause location matters in diligence packs that include contracts. Span overlap is not legal advice.
Paper table · Hendrycks et al., 2021 · not an Antidote run · not a product score
CUAD precision-recall, selected models
Hendrycks et al., 2021, Table 2. AUPR and precision at 80% recall are different estimands and are not averaged.
System
AUPR
Precision @ 80% recall
Precision @ 90% recall
DeBERTa-xlarge
47.8%
44%
17.8%
RoBERTa-large
48.2%
38.1%
0%
RoBERTa-base
42.6%
31.1%
0%
ALBERT-xxlarge
38.4%
31%
0%
BERT-base
32.4%
8.2%
0%
ContractNLI
Unsupported vs contradicted contract claims are different errors. Public NDAs are not a target data room.
Paper table · Koreeda and Manning, 2021 · not an Antidote run · not a product score
ContractNLI label accuracy and entailment F1
Koreeda and Manning, EMNLP 2021 findings, Table 3. Contradiction F1 stays much lower than entailment F1 and is not folded into accuracy.
System
NLI accuracy
Entailment F1
Contradiction F1
DeBERTa v2 + CUAD
89.2%
85.9%
40.5%
DeBERTa v2 xlarge
88.5%
85.5%
36%
BERT-large
87.5%
83.4%
35.7%
BERT-base
83.8%
76.5%
28.7%
AA-Briefcase
Multi-file slides, spreadsheets, and memos are closer to a diligence deliverable than TAT-QA. AA-Briefcase is still a third-party harness, not Hebbia.
Third-party leaderboard · not an Antidote run · not a product score
AA-Briefcase rubric score
Artificial Analysis GPT-5.6 article, 9 Jul 2026. Rubric score stays separate from analytical-quality Elo (Fable 5 1764, Sol 1592). Presentation Elo is described as Sol leading. Third-party harness, not an Antidote run.
System
Rubric score
Analytical-quality Elo
Claude Fable 5 max
56%
1764
GPT-5.6 Sol max
42%
1592
How to read the published benchmarks
Read ParseBench as five parser metrics and a cost scatter, not as an overall rank. Read TAT-QA as hybrid filing QA. Read CUAD and ContractNLI as contract slices. Keep them in the published-benchmark section. The core evidence for a Hebbia diligence opinion would be sealed target documents with independently adjudicated extracts.
Current systems, cost, and harness
Official list prices observed 8 August 2026 bound the model bill only. They are not a product subscription and not full cost per accepted output. ParseBench cents per page is a parser-layer cost. The ladder is an illustrative bundled contrast: a small scaffold and a heavy scaffold are different systems. The bars are a design sketch, not a product score. Full cost still has to include failed attempts and reviewer minutes.
List price per million tokens
Official list prices observed 8 August 2026 from provider catalogs. USD per million input / output tokens. Cache, tools, long-context, and service-tier charges are not included.
Model
Slot
Input
Output
GPT-5.6 Sol
Frontier
$5.00
$30.00
GPT-5.6 Terra
Balanced
$2.00
$12.00
GPT-5.6 Luna
Economic
$0.20
$1.20
Claude Fable 5
Frontier
$10.00
$50.00
Claude Opus 5
Frontier
$5.00
$25.00
Claude Sonnet 5
Balanced
$3.00
$15.00
Claude Haiku 4.5
Economic
$1.00
$5.00
Grok 4.5
Challenger
$2.00
$6.00
ParseBench tables vs cents per page
ParseBench official leaderboard, observed 13 August 2026. Horizontal axis is list cents per page. Vertical axis is the Tables metric (GTRM), not an overall score.
What a harness can change
Illustrative condition ladder on a frozen multi-document set. Dumping a corpus into long context is not the same treatment as chunk-and-cite or agent decomposition. Lift from a harness can be small or large; it has to be isolated.
Agent decomposition has to beat chunk-and-cite, not beat a straw man
The failure mode that matters for Hebbia is crediting agent orchestration for work a long-context dump or a chunk-and-cite stack already does. The increment has to be isolated or left unproven.
Opaque routing means model-level and orchestration-level claims stay unproven until those pairs exist. If traces cannot be exported, report outcome and price only.
Operating economics
Tool calls and review minutes are the bill, not the token sticker
List prices bound the model component. Agent decomposition adds tool and routing charges. A cheaper long-context dump can look inexpensive until correction and missed-page review are counted. Full cost per accepted extract still needs failed attempts, parser charges, and reviewer minutes, with a cap treated as censored.
No Hebbia all-attempt cost is in this report. Cents per page on ParseBench is a parser layer. A seat or usage quote is a commercial term, not that quantity. Any time-saved or associate-leverage case remains modeled and assumption-bound.
Defensibility
Public facts, not measurements
Named surface and packaging, from Hebbia's published pages. The product page declares Matrix, an API, an MCP connector, and private-document integrations. The published DPA lists OpenAI and Anthropic as inference subprocessors. Fair-use terms use named Core User licenses and reserve high-volume automation for higher tiers. A current job posting describes forward-deployed engineers embedded with strategic customers.
Competitive set, as a hypothesis: VDR-native extraction, a generic chunk-and-cite stack, and a long-context dump. None of those facts is a pass rate, a faithfulness audit, or a score versus a named competitor.
The data room is the customer's. Decay is a long-context question.
Diligence documents are the customer's. There is no exclusive corpus. Orchestration, retrieval, and citation machinery are reconstructable in kind. A flywheel needs lawful reuse and a measured quality increment when history is added and nothing else changes. Cross-engagement memory without those rights is not a network effect.
Decay risk from frontier long-context models needs a dated ladder against current systems on the same sealed pack: dump versus chunk-and-cite versus agent decomposition. Agent routing that cannot be exported cannot be attributed. VDRs and DMS vendors can attach extraction to the room the buyer already has. That bundling exposure is independent of Hebbia's harness quality.
Defensibility
A PE data room is the customer's. Hebbia's remaining questions are harness, faithfulness, and decay versus current long-context models. There is no exclusive corpus.
Dimension
Rating
Judgment
Missing evidence
Data rights
Exposed
Diligence documents are the customer's. Hebbia processes them. Processing rights are not exclusive ownership.
Processing, retention, and any cross-engagement reuse terms.
Exclusivity
Exposed
There is no exclusive document corpus. Orchestration, retrieval, and citation machinery are the claimed assets, and they are reconstructable in kind.
What, if anything, cannot be rebuilt as chunk-and-cite plus a citation gate.
Rebuild difficulty
Conditional
Long-context dump, chunk-and-cite, and agent decomposition are known treatments. Reconstructing Hebbia's specific routing is an engineering project. Traces that cannot be exported cannot be attributed.
Exportable traces and a dump versus chunk-and-cite versus agent paired ladder.
Freshness
Open
Hebbia's published MSA limits customer-material model improvement to that customer while allowing aggregated or de-identified usage data to improve the platform. A broad cross-engagement content flywheel is therefore not supported by the standard terms.
Executed terms, de-identification controls, and a history-on versus history-off comparison.
Switching cost
Conditional
Saved projects and citation notebooks can embed. Unobserved as switching cost.
Export of projects and citations.
System of record
Exposed
Hebbia is an extraction overlay on a data room. It is not the deal's system of record.
Whether extracts are filed as work product, and under whose control.
Integrations
Conditional
Room and DMS connectors create gravity. They also define the VDR and Microsoft bundling surfaces.
Connector inventory and a room-native extraction paired comparison.
Network effects
Exposed
One customer's data room does not improve another customer's extracts without reuse rights.
Any cross-engagement claim would need rights and a measured increment.
Trust and compliance
Open
Hebbia publishes security certifications, processor terms, and citation-oriented product design. Span-level faithfulness remains the trust question; footnote count and policy controls are not that audit.
Held-out span faithfulness and missed-page review.
Distribution
Conditional
PE and banking distribution is a channel. It does not block AlphaSense, Harvey, or a VDR vendor from attaching extraction.
Win/loss versus named document-intelligence products.
Services dependence
Open
Hebbia's operating model embeds forward-deployed engineers with strategic customers to build integrations and workflows. Associate review remains the output exception layer.
Forward-deployed hours and margin, reviewer minutes, missed-page rate, and reusable-versus-bespoke work share.
Rate of improvement
Open
Decay versus current long-context models is an open learning-rate question. No dated ladder is in this report.
Dated dump versus agent comparison on a sealed pack.
How AI can compete
Replacement, a new AI product, a platform bundle, cheaper labor, and a customer building it themselves are different risks. They are not averaged into a score.
Mechanism
Assessment
Direct replacement
A long-context dump or a chunk-and-cite stack on the same room is the named competitor. Hebbia has to beat those cells with traces.
New AI product
Frontier long-context and agent systems are the decay path. Invasion does not require Hebbia's corpus because the corpus is the customer's.
Platform bundle
VDRs, DMS vendors, and research terminals can attach extraction to the room the buyer already has.
Labor compression
If faithfulness gates clear, associate extract hours can fall. That is modeled value-creation, not a Hebbia moat.
Customer builds it
A fund can build chunk-and-cite over its own rooms; the corpus is already the customer's, so insourcing is reconstruction in kind. The open question is maintenance: who keeps the harness current as models change.
Data flywheel
A flywheel needs every step below. On this evidence it is unproven. Having data is not the same as being allowed to reuse it, and reuse is not the same as a measured quality gain.
Link
Status
Note
Unique data
Vendor-stated
Document exhaust is the customer's. Routing telemetry, if any, is Hebbia's.
Permission to reuse
Open
Hebbia's published MSA allows customer-material improvement only for models used by that customer and permits aggregated or de-identified usage data for platform improvement. Cross-customer content reuse is not granted by the standard terms.
Reliable feedback
Open
No span-faithfulness labels.
Proven quality gain
Open
No isolated agent-versus-cite increment.
Better outcomes
Open
No sealed-room test.
More usage or data
Open
Retention from notebooks is unobserved.
What public sources resolve
Public filings and vendor documentation can settle some structural questions. They cannot replace a test on the live product.
Question
Conclusion
Still open
Can customer material support a cross-customer data flywheel?
Hebbia's published MSA permits customer-material improvement only for models used by that customer. It separately permits aggregated or de-identified usage data to improve the platform.Master Services Agreement; Data Processing Agreement
Executed terms, de-identification controls, an audited label stream, and a measured quality increment from permitted history.
Is agent orchestration itself the source of advantage?
APIs, MCP, document integrations, and agent-style workflows are declared product features. No public evidence isolates their incremental effect.Hebbia product overview; Data Processing Agreement
A paired generic-system comparison with model, corpus, retrieval, citation gate, tools, and budget frozen.
Is Hebbia a low-touch self-serve software product?
No. Hebbia's operating model explicitly embeds forward-deployed engineers with strategic customers to build integrations and workflows.Forward Deployed Engineer; Fair Use Policy
Forward-deployed headcount, hours per customer, implementation margin, customer concentration, and reusable-versus-bespoke work share.
Required comparisons
A decision-grade test would run these side by side. None has been run here.
Comparison
Note
Current workflow
Associate reading and memo process must be evidenced.
Generic model
Long-context model on the same corpus, no vendor harness.
Named competitor
Declared chunking, retrieval, and citation budget.
This product
Needs authorized Hebbia access and exportable traces.
Isolated changes
Citation enforcement and agent decomposition must be tested separately.
Open claims
Each claim stands on its own. They are not averaged into a moat score. Opaque routing means model-level and orchestration claims stay unproven until isolated. The customer's data room is not Hebbia's exclusive data.
Claim
Status
What is still missing
Agent orchestration is the source of advantage.
Open
Paired generic-system comparison with frozen tools and budget.
Diligence documents are the customer's, not Hebbia's exclusive corpus.
Judgment
Processing, retention, and any cross-engagement reuse terms.
Citations are faithful to source spans.
Open
Span-level faithfulness audit on held-out documents.
A data flywheel exists across engagements.
Open
Reuse rights and a measured quality increment.
VDR or DMS bundling risk is low.
Open
A room-native extraction paired comparison.
Decay risk from frontier long-context models is low.
Open
Dated ladder against current frontier long-context systems.
What this report does not conclude
Memo quality or issue-completeness on a PE data room
Attribution of advantage to agent orchestration
A data moat; the documents are the customer's
Citation span faithfulness
Full cost per accepted extract
A data flywheel or low decay risk versus current long-context models
What would settle these questions
Authorized Hebbia access and exportable traces.
A sealed diligence corpus with independent extract and citation keys.
Long-context, chunk-and-cite, and agent-decomposition cells with frozen budgets.
Span-level faithfulness audit, separate from answer correctness.
All-attempt cost including tools, failures, and reviewer minutes.
Reuse rights before any cross-engagement learning claim.
A room-native extraction paired comparison versus Hebbia-only review.
Hebbia uses named Core User licenses and reserves high-volume automation and specialized use for higher-tier or custom plans. API and automation usage may be monitored and throttled.
The policy gives no public dollar rate, quota, overage schedule, or full cost per accepted workflow.
Hebbia's published MSA permits customer materials to improve models solely for that customer and prohibits making those improved models available to third parties. It separately permits aggregated or de-identified usage data to improve the platform.
Published standard agreement, not an executed target contract. De-identification quality and any negotiated terms are not verified.
Hebbia declares document-scale reasoning with traceability, a Matrix API, an MCP connector, private-document and financial-data integrations, and no training on user data.
Vendor feature description. Hebbia's MSA separately permits customer-material improvement for models used only by that customer. Citation presence is not span faithfulness, and integrations do not establish switching cost or measured outcomes.
Hebbia's current role specification says forward-deployed engineers spend most of their time embedded with strategic customers building integrations and workflows and generalizing reusable patterns.
A role specification establishes the operating model, not team size, effectiveness, customer-level hours, implementation margin, or the reusable-to-bespoke ratio.
Sivulka argues that token volume is not value, human work can be cheaper than wasteful inference, and evaluation is the operating control for agent systems.
Interested founder discourse. It does not establish Hebbia's own token cost, quality, customer ROI, or unit economics and cannot populate measured report fields. X search is authentication-gated; the direct status was recovered through a corroborated public source rather than an unauthenticated platform search.
