Physical-Engineering’s Last Exam
Built to evaluate and improve agent capabilities for physical engineering work
The Physical-Engineering’s Last Exam leaderboard
Physical-Engineering's Last Exam (PLE) was created to evaluate agents on the real work of mechanical, electrical, civil, and structural engineers. The tasks require agents to work through a full project folder, find the governing codes and physical limits, and produce deliverables a licensed reviewer would sign off on.
PLE was built two ways. For expert-authored tasks, practicing engineers created the project folders, prompts, and grading rubrics from scratch. For co-authored tasks, engineers designed the task, an agent pipeline assembled the environment and grader, and the engineers validated every task and made it harder. An agent earns optimization points only if its design passes every hard safety check.
There are 85 tasks in PLE across four disciplines, with 5,170 files run in containers built on Mercor's Archipelago environment. The 29 public tasks are available open-source, along with grading tools and an open competition leaderboard.
Frequently Asked Questions
Physical-Engineering's Last Exam (PLE) is a benchmark and open competition that tests whether AI agents can complete long, end-to-end physical engineering work. Google built it in collaboration with Mercor. Each task has a prompt, a project folder with the relevant files, and required production deliverables. Grading tools check those deliverables for code compliance and cost efficiency.
Experts from Mercor and licensed, practicing engineers from industry. For 44 expert-authored tasks, engineers created the project folders, prompts, and grading rubrics from scratch, based on real professional engineering work. For the other 41, engineers designed the task, an agent pipeline assembled the environment and grader from licensable engineering standards, and the engineers validated every task and made it harder.
Each task drops the agent into a containerized project folder that mirrors what a practicing engineer inherits on the job. A folder holds an average of 61 files, such as PDFs, spreadsheets, emails, 2D CAD drawings, 3D BIM models, and outdated document revisions. The agent has to sort through them, find the governing building codes and physical limits, run calculations, and produce deliverables a licensed reviewer would sign off on.
Automated scripts check quantitative outputs, and an LLM judge grades open-ended work against an expert rubric. A design must pass every hard safety and feasibility check before it can earn optimization points for cost efficiency. Each model runs at its highest reasoning effort in its native harness, three times per task, in an isolated container with no internet access and a 6-hour limit. The leaderboard reports the mean score on the 56 hidden test tasks.
Frontier models now score above 90% on many academic benchmarks, which mostly test textbook question answering, single tool calls, or small, self-contained design problems. PLE tests long-horizon work. The agent has to search a full project folder, decide what's relevant, and produce safe, code-compliant, and cost-effective designs.
Rankings can change whenever a new frontier model comes out. Mercor evaluates new models on PLE and the APEX benchmarks when they launch. See the leaderboard above for current results.
Yes. The 29 public tasks, grading scripts, and evaluation container are on GitHub, and the full technical breakdown is in the PLE paper on arXiv. You can also bring your own agent setup and enter the open competition at ple-bench.com.