Natural-language model construction and checks inside Excel. Official policies establish the product surface and data terms; a named practitioner test finds useful narrow-task output and consequential modeling failures. No independent product test has been run.
As of 19 August 2026. This memo covers filing-grounded model construction, forecast response, and error detection inside Excel. No live-product test has been run. Detailed metrics from a July 2026 third-party assessment are withheld pending permission to reproduce them.
Useful on narrow Excel tasks in the public record; closure and repair fail there; Microsoft owns the surface.
Vendor materials describe an Excel-native modeling, review, and audit product, external model providers, customer-data restrictions, and entry plans. Those remain product claims, not evidence that generated workbooks clear a business-quality gate.
A named 2025 practitioner test found fast, useful output on narrow tasks and also found hard-coded dates, a wrong balance-sheet diagnosis, a damaging attempted fix, and a multi-year model that did not balance. That is credible failure discovery but not a frozen current-version pass rate. Capability, reviewer load, full cost per accepted model, and generalization to private-company workbooks remain open.
Exposed vs scarce. Narrow-task construction inside Excel can be useful. The customer's workbooks are not a scarce corpus.
Still untested. Current-version closure, repair safety, and self-audit false-pass on private-company models.
The comparison that would change the call. Tracelight versus Copilot-in-Excel, plus checks-off versus checks-on, on the same sealed workbooks.
Decision questions
On a sealed set of private-company workbooks, do construction, traceability, change propagation, and error detection clear prespecified gates?
What are the self-audit's false-pass and false-flag rates against independent recalculation?
What are associate and reviewer minutes to a committee-usable model against an analyst build and Copilot-in-Excel?
When Microsoft ships error-catching natively, what does Tracelight keep: workflow, template memory, or a head start?
Is forecast accuracy in the record the product's job at all, or a world-plus-tool estimand that belongs outside the construction gate?
A financial model has separable planes: construction (complete, connected, formula-driven), traceability (every line to a real filing cell), response (a changed assumption propagates without breakage), and audit (the model's own checks agree with independent recalculation). They are different estimands and are not averaged. Forecast accuracy against a held-back actual is a fifth thing — the world's surprise plus the tool — and it does not belong in a construction gate.
This report keeps official product and policy declarations separate from attributed practitioner testing. A third-party assessment exists, but its own qualifications require prior written consent for reproduction, quotation, or distribution. Its detailed cells are therefore not reproduced or treated as our measurements without documented permission.
Workflow breakdown
Each step of the job is mapped to a published benchmark or another reputable quantitative measure only where the mapping is honest. A public figure is a floor or a labor-market context on that task type. It is never a score of this product.
The reasoning layer. Not construction, linkage, or audit.
Workbook construction, editing, and formula validity
None
No honest public floor
SpreadsheetBench and AIDABench are the construction-shaped packs. Both are held out on license grounds.
Self-audit false-pass rate on a messy private-company model
None
No honest public floor
Needs a sealed workbook set and independent recalculation.
Findings at a glance
Capability, cost, and defensibility are assessed separately. They are not averaged into a score. Where evidence is missing, the finding stays open.
Status
Area
Finding
Open
Workflow capability
A named 2025 practitioner test found fast narrow-task output alongside hard-coded dates, a wrong diagnosis, a damaging attempted repair, and a non-balancing multi-year model. That discovers failure modes; it does not estimate current-version capability. No Antidote run exists.
Open
Operating economics
Published subscription tiers exclude reviewer labor, workbook repair, recalculation, failed attempts, and unusable outputs. Full cost per accepted model and reviewer minutes are unmeasured.
Judgment
Defensibility
The product rides the customer's record surface, which Microsoft owns and ships agents into. Its DPA names external frontier-model providers, and its published policy bars customer-content training. Defensibility has to come from workflow controls, certified audit behavior, integrations, or non-customer assets.
Vendor-stated
Vendor claims and open questions
Vendor materials declare natural-language modeling, error detection, auditability, external model providers, and no customer-content training. Practitioner tests remain dated discourse. Restricted third-party assessment figures are not reproduced without documented permission.
Workflow capability
The public record discovers failures; it does not estimate current reliability
Paul Barnhurst, Ian Schnoor, and Giles Male tested Tracelight on finance and modeling tasks in 2025. They reported a fast deferred-revenue build and strong narrow-task results alongside hard-coded dates, an incorrect balance-sheet diagnosis, a harmful attempted repair, and a non-balancing multi-year model. Experienced review remained necessary.
The tests are attributable and inspectable, but they were not preregistered, did not publish a complete case universe or denominator, and predate the current product. They support a challenge-suite hypothesis: dynamic formulas, model closure, diagnostic localization, and repair safety must be explicit gates. They cannot support a current pass rate.
Published benchmarks
TAT-QA and FinQA bound the numerical-reasoning layer inside modeling work. Construction, linkage, and audit have no license-cleared public pack; SpreadsheetBench and AIDABench are held out, so the construction floor is unmeasured here on purpose.
TAT-QA
Hybrid table-and-text numerical reasoning bounds the reasoning layer inside modeling work. It does not touch construction, linkage, or audit.
Paper table · Zhu et al., 2021 · not an Antidote run · not a product score
TAT-QA test exact match and F1
Zhu et al., ACL 2021, Table 3, test split. TAGOP is the paper system. Human expert is 50 hybrid contexts (301 questions).
System
Exact match
F1
Human expert
84.1%
90.8%
TAGOP
50.1%
58%
NumNet+ V2
37%
46.9%
TaPas WTQ
16.6%
22.8%
BERT-RC
9.1%
18.7%
HyBrider
6.3%
7.5%
Paper table · Zhu et al., 2021 · not an Antidote run · not a product score
TAGOP by answer type on the test set
Zhu et al., ACL 2021, Table 4. Arithmetic is the weak slice. Metrics stay separate; there is no composite.
Answer type
Exact match
F1
Span
54.1%
67.9%
Spans
60%
75.1%
Counting
62.5%
62.5%
Arithmetic
42.5%
42.5%
FinQA
Executable numerical reasoning over filings bounds calculation discipline. Program accuracy stays separate from execution accuracy; neither is a workbook.
Paper table · Chen et al., 2021 · not an Antidote run · not a product score
FinQA test execution and program accuracy
Chen et al., EMNLP 2021, Table 2. FinQANet (RoBERTa-large) is the paper system. Human expert execution accuracy is 91.16%; the non-expert crowd is 50.68%.
System
Execution accuracy
Program accuracy
Human expert
91.16%
87.49%
FinQANet RoBERTa-L
61.24%
58.86%
FinQANet BERT-L
53.52%
51.62%
Crowd
50.68%
48.17%
NeRd BERT-base
48.57%
46.76%
Longformer base
21.9%
20.48%
How to read the published benchmarks
TAT-QA and FinQA bound the numerical-reasoning layer inside modeling work; neither touches construction, linkage, or audit. SpreadsheetBench and AIDABench are the construction-shaped packs and both are held out on license grounds, so the construction floor is unmeasured in this section on purpose.
The attributed practitioner test is challenge-suite evidence, not a target-workflow estimate. Detailed third-party assessment metrics remain withheld pending documented reuse permission.
Current systems, cost, and harness
Official list prices observed 8 August 2026 bound the model bill only. They are not a product subscription and not full cost per accepted output. The ladder is an illustrative bundled contrast: a small scaffold and a heavy scaffold are different systems. The bars are a design sketch, not a product score. Full cost still has to include failed attempts and reviewer minutes.
List price per million tokens
Official list prices observed 8 August 2026 from provider catalogs. USD per million input / output tokens. Cache, tools, long-context, and service-tier charges are not included.
Model
Slot
Input
Output
GPT-5.6 Sol
Frontier
$5.00
$30.00
GPT-5.6 Terra
Balanced
$2.00
$12.00
GPT-5.6 Luna
Economic
$0.20
$1.20
Claude Fable 5
Frontier
$10.00
$50.00
Claude Opus 5
Frontier
$5.00
$25.00
Claude Sonnet 5
Balanced
$3.00
$15.00
Claude Haiku 4.5
Economic
$1.00
$5.00
Grok 4.5
Challenger
$2.00
$6.00
What a harness can change
Illustrative condition ladder on the numerical-reasoning layer inside a workbook, not construction or audit. A raw model, a generic spreadsheet agent, and an isolated checks-on cell are different systems. The bars are a design sketch. They are not a Tracelight test.
The required comparisons have not been run
The failure mode that matters for Tracelight is treating a fast narrow-task build as model closure. The public record already shows hard-coded dates, a wrong diagnosis, a damaging fix, and a multi-year model that did not balance.
An admissible run needs the same sealed workbook set across the analyst baseline, a raw frontier surface, a declared generic spreadsheet agent, Tracelight, and a checks-off versus checks-on comparison. Model, workbook, instructions, budgets, and grading stay frozen. If only the product surface is observable, publish its outcome and full cost. Do not credit the audit layer, routing, or model choice without an isolated pair.
Operating economics
List price excludes the review pass
Tracelight publishes individual subscription tiers and an Enterprise plan with onboarding, custom skills, retention controls, and administrative support. Those are commercial terms. They do not include the measured review, repair, and recalculation effort required to accept a workbook.
Full cost per accepted model remains the estimand: every attempt, provider and tool cost, workbook repair, independent recalculation, reviewer minutes, and unusable outputs in the denominator. No such measurement is public.
Defensibility
A checker on a surface whose owner ships checkers
Excel is the customer's record surface and Microsoft's product. Public materials describe an add-in and external model subprocessors, which frames possible workflow embed and platform dependency but does not establish either without a live-product test.
Customer workbooks remain customer content. Published policies describe restrictions on shared-model training; executed terms are not in hand. A defensibility case therefore still requires certified audit behavior, workflow controls, integrations, or proprietary non-customer assets, each tested separately.
Defensibility
A construction-and-audit agent inside the customer's record surface. Official policies establish the product boundary and data terms; a named practitioner test discovers consequential failures but does not estimate current capability. The host platform is the structural exposure.
Dimension
Rating
Judgment
Missing evidence
Data rights
Exposed
Workbooks are the client's and filings are public. Audit traces could be an asset if rights and retention existed on the record; they do not.
Trace-retention and reuse terms.
Exclusivity
Exposed
No corpus. The differentiators are engineering: cell-level lineage and deterministic tripwires.
Any input or right a frontier in-workbook agent cannot reach.
Rebuild difficulty
Conditional
Deterministic checks, workbook lineage, and in-Excel agent workflows are reconstructable in kind. Named practitioner testing shows that frontier tooling can produce useful narrow-task work and consequential modeling failures; no current paired product comparison is release-eligible.
Rebuild time and a current authorized pair against the host and frontier add-ins.
Freshness
Open
Tracelight's published policy bars customer-content training and its DPA separates usage data and feedback. Cross-customer content learning is therefore not established; behavior across filing drift and messy private workbooks remains unobserved.
Usage and feedback fields, retention, a messy-workbook series, and error-pattern learning evidence.
Switching cost
Conditional
Inside-Excel position is real embed: no export, no migration. The same position means the host can absorb the feature.
Retention once the host ships native checking.
System of record
Exposed
The model lives in Excel, which is the record surface. Tracelight writes into it rather than owning it; write access without record ownership is embed, not authority.
Nothing recoverable here; the surface is the host's.
Integrations
Conditional
Excel-native is both integration and dependency: the product is a guest on the host's API surface.
What accumulates on the vendor's side beyond the session.
Network effects
Exposed
One client's models do not improve another's without reuse rights.
Any cross-client claim would need rights and a measured increment.
Trust and compliance
Conditional
Auditability is the pitch. A named practitioner test found an incorrect diagnosis and a damaging attempted repair, so a model-review function still needs independent recalculation. The published security and DPA controls do not establish audit correctness.
False-pass rate at scale on sealed messy workbooks.
Distribution
Conditional
IB and PE distribution, against a host that ships to every seat by default.
Win/loss versus Copilot-in-Excel and in-workbook frontier agents.
Services dependence
Open
Published plans include onboarding, support, and custom skill libraries but do not disclose analyst repair of generated models. The services layer exists; workbook repair before delivery remains unknown.
Services share per model, support records, and delivery traces.
Rate of improvement
Open
No rights-cleared dated current-version quality series is in this report.
Repeated deterministic keys across releases.
How AI can compete
Replacement, a new AI product, a platform bundle, cheaper labor, and a customer building it themselves are different risks. They are not averaged into a score.
Mechanism
Assessment
Direct replacement
The product's named subprocessors and the Excel host also ship model and agent surfaces. A current paired run is required to determine whether Tracelight clears a workflow-quality or audit gate beyond those substitutes.
New AI product
In-workbook agents from the model labs are the same product category with better model access.
Platform bundle
The material exposure. Excel's owner ships Copilot into every seat Tracelight sells to, and deterministic model-checking is the most bundle-able feature class in this catalog.
Labor compression
First-pass model construction is associate work. If construction gates clear at scale, those hours compress regardless of which vendor wins.
Customer builds it
A bank's model-review function can wire deterministic checks over its own templates today. No frozen frontier or Tracelight cell in this report establishes the ceiling of that build. What insourcing cannot reproduce from public evidence is a certified false-pass rate, which no vendor has published either.
Data flywheel
A flywheel needs every step below. On this evidence it is unproven. Having data is not the same as being allowed to reuse it, and reuse is not the same as a measured quality gain.
Link
Status
Note
Unique data
Vendor-stated
Audit traces and error patterns are the declared exhaust. Retention rights unshown.
Permission to reuse
Open
Tracelight's published policy bars customer-content training by Tracelight and third-party models. Its DPA separates usage data and feedback, whose fields and improvement use still require target-term review.
Reliable feedback
Open
No audited error-pattern labels.
Proven quality gain
Open
No paired result across releases on a frozen key.
Better outcomes
Open
No reviewer-minute or false-pass result exists.
More usage or data
Open
Retention is unobserved.
What public sources resolve
Public filings and vendor documentation can settle some structural questions. They cannot replace a test on the live product.
Question
Conclusion
Still open
Is customer content used to train shared models?
Tracelight's published policy says customer content is not used to train its own or third-party AI models. Its DPA names external model providers and separates usage data and feedback.Privacy Policy; Data Processing Agreement
Is vendor-side human model repair part of standard delivery?
Published plans include support, onboarding, and custom skill libraries, but do not disclose analyst repair of generated workbooks. Absence from marketing is not proof of absence.Pricing; Data Processing Agreement
Vendor attestation, support records, services hours per model, and delivery traces showing whether workbooks are touched before the customer receives them.
Do current results generalize to private-company models?
Public evidence does not support generalization. A named prior-version practitioner test finds consequential failures on other cases, and no rights-cleared messy private-company workbook test is on the record.Testing Excel AI Software Tracelight
A frozen messy-workbook prevalence sample with private-company structures, non-GAAP bridges, incomplete source data, and independent review.
Required comparisons
A decision-grade test would run these side by side. None has been run here.
Comparison
Note
Current workflow
No analyst-built model cell exists in any record.
Generic model
No frozen raw-model cell is registered as Antidote evidence.
Named competitor
No declared spreadsheet-agent pipeline with a frozen budget exists in any record.
This product
No authorized current-version Tracelight run is registered as Antidote evidence.
Isolated changes
No checks-off versus checks-on comparison isolates the audit layer from construction.
Open claims
Each claim stands on its own. They are not averaged into a moat score. Restricted third-party assessment detail is not reproduced without permission. Forecast luck is not construction quality, and auditability is not a moat while the host platform ships its own checker. Any ranking waits on a current independent product test.
Claim
Status
What is still missing
Cell-level traceability is the differentiator.
Open
A current independent product test with source-existence, source-location, coverage, and end-to-end traceability gates.
The product catches errors it did not introduce.
Open
A sufficiently powered planted-error and naturally occurring error set, with false-positive and localization scoring.
The model's own audit can be relied on.
Open
False-pass and false-flag rates against independent recalculation on a frozen workbook population.
Forecast accuracy is product quality.
Open
The estimand split. Next-year accuracy against a held-back actual measures the world plus the tool and is excluded from any construction gate here.
The Excel-native position is defensible.
Judgment
A Copilot-in-Excel paired comparison, in-workbook frontier-agent cells, and switching observations once the host ships native checking.
Results generalize to private-company models.
Open
A rights-cleared messy-workbook test with private-company structures, non-GAAP bridges, incomplete source data, and independent review.
What this report does not conclude
Current-version workflow capability or reliability at scale
An ordering versus a raw model, generic spreadsheet agent, or analyst build
The self-audit's false-pass rate, the number a model-review function needs
Performance on private-company workbooks
Cost or associate-minute savings per accepted model
Durability against Copilot-in-Excel and in-workbook frontier agents
What would settle these questions
Authorized Tracelight access with exportable workbooks and a frozen price card.
A sealed private-company workbook set — messy filings, non-GAAP bridges, missing data — with deterministic keys and enough cases to resolve differences.
A checks-off versus checks-on comparison isolating the audit layer from construction.
Self-audit false-pass and false-flag rates against independent recalculation at scale.
A Copilot-in-Excel paired comparison, same workbooks, same key.
Associate- and reviewer-minute measurement to a committee-usable model.
License clearance for SpreadsheetBench or an equivalent construction pack before any public floor claim.
Documented permission before reproducing or transforming restricted third-party assessment figures.
Tracelight's DPA identifies OpenAI, Anthropic, and Google as LLM subprocessors, requires deletion at termination subject to legal and backup exceptions, and provides a constrained customer audit right.
Standard DPA. It does not establish model quality, default retention periods for every feature, or negotiated enterprise terms.
Privacy PolicyTracelight · Official policy · August 2026
Tracelight says customer content is processed under customer instructions, is not used to train Tracelight or third-party AI models, and is sent to named AI-provider subprocessors for service delivery.
Published policy, not an executed enterprise agreement. Usage data and feedback are separate controller data and require target-contract review.
PricingTracelight · Official product documentation
Tracelight publishes individual monthly tiers and an enterprise tier that includes onboarding support, custom skill libraries, configurable retention, and administrative controls.
List price excludes reviewer labor and does not establish vendor-side model repair or full cost per accepted workbook.
Named finance practitioners observed fast and useful narrow-task output but also hard-coded dates, a wrong balance-sheet diagnosis, a damaging attempted fix, and a non-balancing multi-year model that required experienced review.
Unfrozen practitioner tests on a prior product version with no complete case registry, denominator, or masked adjudication. Strong failure discovery, not a current pass-rate estimate.
