APEX-SWE: Observability

Evaluates a model's ability to diagnose and remediate real-world software engineering production failures.
Opus 5.5
Opus 5.5Max
69.8%
Opus 5
Opus 5Max
63.5%
Sonnet 5.5
Sonnet 5.5Max
63.5%
Fable 5.1
Fable 5.1Max
59.0%
Fable 5
Fable 5Max
54.2%
Grok 4.6
Grok 4.6xHigh
51.7%
Haiku 5.5
Haiku 5.5Max
47.2%
Grok 4.5
Grok 4.5High
47.0%
Grok 4.7
Grok 4.7xHigh
45.0%
Qwen 3.8 27B
Qwen 3.8 27BxHigh
41.7%
MiMo V2.6 Pro
MiMo V2.6 ProAuto
40.3%
Qwen3.8-Max
Qwen3.8-MaxxHigh
40.1%
DeepSeek-V4.1-Flash
DeepSeek-V4.1-FlashMax
39.3%
Muse Wonder 1.0
Muse Wonder 1.0Max
39.0%
GPT-6 Astra
GPT-6 AstraMax
36.1%
Kimi K3
Kimi K3Max
36.0%
Opus 4.8
Opus 4.8Max
35.7%
DeepSeek-V4.1-Flash-1017
DeepSeek-V4.1-Flash-1017Max
35.5%
GPT-5.6 Sol
GPT-5.6 SolMax
34.5%
Opus 4.7
Opus 4.7Max
33.0%
Sonnet 5
Sonnet 5Max
32.5%
GPT-6.1 Sol
GPT-6.1 SolMax
32.3%
GLM-5.3
GLM-5.3Max
31.5%
DeepSeek-V4-Flash
DeepSeek-V4-FlashMax
31.2%
Opus 4.6
Opus 4.6Max
30.8%
GLM-5.3-Flash
GLM-5.3-FlashMax
30.2%
DeepSeek-V4-Pro-0813
DeepSeek-V4-Pro-0813Max
30.0%
Muse Spark Open
Muse Spark OpenxHigh
29.5%
GPT-6 Sol
GPT-6 SolMax
27.8%
Mercury 3
Mercury 3High
27.5%
Gemini 4 Argon
Gemini 4 ArgonHigh
27.3%
Beam
BeamMax
25.8%
Kimi K2.7 Code
Kimi K2.7 CodeHigh
24.5%
MiniMax-M3
MiniMax-M3High
21.6%
Sonnet 4.6
Sonnet 4.6High
20.0%
GLM-5.1
GLM-5.1
18.2%
GPT-5.6 Luna
GPT-5.6 LunaMax
18.2%
GPT-6 Luna
GPT-6 LunaMax
18.2%
GLM-5.2
GLM-5.2Max
18.0%
Mistral Large 4
Mistral Large 4High
17.2%
DeepSeek-V4-Pro
DeepSeek-V4-ProMax
17.0%
Gemini 3.7 Flash
Gemini 3.7 FlashHigh
17.0%
GPT-5.5
GPT-5.5xHigh
16.8%
GPT-5.4
GPT-5.4xHigh
16.3%
Muse Spark 1.2
Muse Spark 1.2xHigh
16.0%
GPT-5.6 Terra
GPT-5.6 TerraMax
16.0%
Qwen3.5
Qwen3.5
15.0%
Gemini 3.5 Flash
Gemini 3.5 FlashHigh
14.5%
Gemini 3.6 Flash
Gemini 3.6 FlashHigh
14.5%
Muse Spark 1.1
Muse Spark 1.1xHigh
14.3%
Gemini 3.1 Pro
Gemini 3.1 ProHigh
12.0%
Gemini 3.8 Flash
Gemini 3.8 FlashHigh
11.0%
Nemotron 3.5 Super VL
Nemotron 3.5 Super VLHigh
10.7%
Inkling
InklingHigh
10.2%
Muse Spark 1.3
Muse Spark 1.3Max
10.0%
MiniMax-M2.7
MiniMax-M2.7High
9.5%
DeepSeek-V3.2
DeepSeek-V3.2
9.2%
Nemotron 3 Ultra
Nemotron 3 UltraHigh
8.1%
Kimi K2
Kimi K2High • Thinking
8.0%
Gemini 3.5 Flash Lite
Gemini 3.5 Flash LiteHigh
7.2%
GPT-OSS-120B
GPT-OSS-120BHigh
3.0%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%

APEX NEWSLETTER

The latest on frontier AI performance, straight to your inbox.

New benchmarks, leaderboard shifts, and research from the APEX team.

By subscribing, you agree to receive updates from Mercor.
Unsubscribe anytime.