The AI Productivity Index for Accounting
The AI Productivity Index for Accounting (APEX-Accounting) measures whether frontier AI agents can execute long-horizon, cross-application tasks in professional accounting.
The APEX-Accounting leaderboard
We created APEX-Accounting to evaluate agents on the real day-to-day work of accountants closing the books each month. The tasks require agents to reason over incomplete records, apply company-specific context, use multiple applications, and carry figures accurately through a close.
APEX-Accounting was built in three steps. First, accountants created 10 data-rich worlds, each a business frozen at month-end close. Second, they created challenging period-close tasks using the files from within each world, then solved them to produce the rubrics. Third, we gave agents access to execute the tasks with the software a human accountant would use.
There are 10 worlds in APEX-Accounting, comprising 160 tasks and grading rubrics. An eleventh world is open-source, along with Archipelago, our infra service for executing and evaluating agent trajectories.
Key takeaways
- 1
Frontier agents cannot yet reliably close the books
The best model, Claude Fable 5, meets 56.4% of grading criteria. 58% of the benchmark tasks are never fully solved by any model in any of its eight attempts.
- 2
Capability ≠ consistency
Models occasionally solve a task but almost never solve it on every attempt.
- 3
Spending more does not buy reliability
One model spending the highest dollar amount per run finishes in a statistical tie with one of the leanest models.
- 4
The bottleneck is accounting judgement, not retrieval and tooling
Multi-step reasoning errors account for over half of the top performing model's errors.
| Model | Score | $/run |
|---|---|---|
| Claude Fable 5 | 56.4% | $36.00 |
| Muse Spark 1.1Top quadrant | 52.6% | $4.24 |
| GPT-5.6 Sol | 51.5% | $33.20 |
| Claude Opus 4.8 | 48.0% | $15.82 |
| GLM 5.2Top quadrant | 42.7% | $3.75 |
| Grok 4.5 | 40.8% | $3.77 |
| Kimi K2.7 Code | 37.0% | $1.72 |
| Gemini 3.1 Pro | 32.4% | $8.75 |
| Qwen3.5-397B | 24.4% | $0.68 |
Sample task
Law firmTask prompt
StandardReview the December client cost advance reconciliation. Reconcile the client cost advance ledger to the QBO/GL balance for client cost advances. Using the cost advance ledger methodology and provided support, verify whether hard costs that should be treated as client cost advances were capitalized and whether soft costs were expensed. Identify any discrepancies in cost classification or capitalization treatment that need to be resolved. Round all amounts to two decimal places. Do not round intermediary values. Provide your answer to the console.
Environment
QuickBooks
Sheets
PDFs
Docs
Code
Grading criteria
- ✓Reports the client cost advance ledger ending balance for Client Cost Advances Hard Costs (or Account 1300) as $0.00 (acceptable value is $0.00).
- ✓Reports the QBO GL ending balance for Client Cost Advances Hard Costs (or Account 1300) as $0.00 (acceptable value is $0.00).
- ✓Reports the difference between the client cost advance ledger and QBO Client Cost Advances Hard Costs (or Account 1300) as $0.00 (acceptable value is $0.00).
+ 21 more grading criteria
Loss analysis
Discover where models failed on APEX-Accounting tasks.
Hover, focus, or tap a slice to inspect it
Methodology
Each model runs a month-end close inside one of 10 real businesses, with its accounts, records, and accounting software. More than 40 professionals, median 11 years' experience and half from Big Four firms, built the tasks and rubrics. Agents get a short request: no file list, no method. They must find their own evidence, apply U.S. GAAP accrual accounting, and produce an answer graded criterion by criterion. Our LM judge matches expert graders 97% of the time.
Harness
Leaderboard results use the Loop Harness, the canonical while-loop-with-tools agent architecture.
Grading
LM judge scores each rubric criterion independently as Met or Not Met, seeing the task prompt, the criterion, and the model’s final answer.
Runs
Every model is run 8 times to get Pass@1, Pass@8, and Pass^8. The canonical leaderboard metric is Mean Criteria % Passed (k=3).
Model config
Nine frontier models at highest available reasoning, called through LiteLLM with exponential backoff, and up to three attempts per request.
Cost limits
Leaderboard runs use a 5-million-token cap and 500 step limit.
Error bars / CI
All intervals are 95% task-level bootstraps over 10,000 resamples.
Dataset
160 tasks across 10 held-out worlds, with 2,186 grading criteria, and a mean of 7.5 required input files per task.