Chart understanding and visual reasoning.
Evaluates multimodal models on realistic chart understanding using 2,323 hand-curated arXiv charts, with descriptive questions (basic elements) and reasoning questions (synthesis across complex elements).
Model
Score
Muse Spark 1.2xHigh
Score on the original open-source benchmark: 95.3% / Score on this Mercor-extended benchmark: 87.1%
Muse Spark 1.1xHigh
Score on the original open-source benchmark: 95.2% / Score on this Mercor-extended benchmark: 86.4%
GPT-6 AstraxHigh
Score on the original open-source benchmark: 95.8% / Score on this Mercor-extended benchmark: 85.1%
GPT-6 AstraMax
Score on the original open-source benchmark: 96.1% / Score on this Mercor-extended benchmark: 84.0%
GPT-6.1 SolMax
Score on the original open-source benchmark: 96.0% / Score on this Mercor-extended benchmark: 85.1%
Opus 5.5Max
Score on the original open-source benchmark: 96.4% / Score on this Mercor-extended benchmark: 84.7%
APEX NEWSLETTER
New benchmarks, leaderboard shifts, and research from the APEX team.
By subscribing you agree to receive updates from Mercor.Unsubscribe anytime.