Surge AI Releases DAYJOB: Tax, a Benchmark for Tax Agents: 50 Professional Assignments and 1,789 Files, and the Best Model, Claude Sonnet 5.5, Passes Only 10.8%

Not a tax quiz, but taking over a job

Surge AI released DAYJOB: Tax on October 27, the third benchmark in its DAYJOB family after healthcare and finance. Agents receive a client request and a pile of raw records, and must decide what to investigate, reconcile conflicting evidence, catch issues the client never raised, and deliver a tax return, workpaper, technical review or advice letter.

The 50 tasks span eight areas of practice: individual and family tax, small business, partnerships, property and investments, estates, gifts and trusts, international tax, corporate transactions, and tax review and disputes. The workspaces hold 1,789 files, including 1,487 PDFs, 117 Excel workbooks and 38 Word documents, plus emails, images and CSVs; the median task has 30 files and the largest has 123. The material includes a photo of a handwritten receipt for goods bought for resale, and a field dispatch log showing crews working in five states for a company whose client believes it operates in only one. Tax experts wrote the tasks, and each went through three layers of expert review.

Results: even the best passes about one in ten

Scoring is pass@1, and an attempt passes only if it meets at least 95% of the task's criteria. Anthropic models hold the top three spots: Claude Sonnet 5.5 at 10.8%, Claude Fable 5.1 at 9.6% and Claude Opus 5.5 at 9.2%. Next come Muse Wonder 1.0 at 6%, Grok 4.7 at 5%, GPT-6 Astra at 4.8%, GPT-6.1 Sol at 4% and Gemini 4 Argon at 3.2%, with DeepSeek V4.1 Flash 1017 at 1.2%, GLM 5.3 at 0.4%, and Kimi K3, Qwen 3.8 Max and Mistral Large 4 at 0%. For comparison, the strongest models complete 24.7% and 23.9% of assignments on the healthcare and finance benchmarks in the same family.

Surge's two examples show where points are lost. A client asked which of two sale dates for a condo was better for taxes; in five attempts GPT-6 Astra never warned that the home-sale exclusion would lapse around the end of August 2027, while Opus caught it all five times. A separating couple asked how to split their retirement accounts; Opus assumed in all five attempts that they were unmarried, while GPT-6 Astra asked to verify their marital status first in four of five.

How to read this leaderboard

Only 2 sample tasks are public; the full set of 50 is available on request from Surge, so outsiders cannot reproduce the results yet. Surge itself supplies training data to several model makers, and its blog has described its work with Anthropic, which readers may want to factor in. For businesses, the more useful signal is the type of failure: models often get the arithmetic right but miss a deadline the client never asked about, or reach a conclusion when the evidence is thin. That is exactly where tax work still needs human review.