We post-trained NVIDIA’s new Nemotron 3.5 Super VL ahead of its release. We wanted to see how far a small, targeted dataset moves an open model on real professional work.

We picked three domains where our customers need agents that get the details right, and brought in a partner for each: Harvey for legal, Cognition for software engineering and Ramp for accounting. Each domain got its own training set of a few hundred tasks from Mercor's APEX environments, and each post-trained model was tested on benchmark tasks it had never seen. In all three, it beat the base model:

• Legal: On the APEX-Agents benchmark legal domain, the post-trained Nemotron 3.5 Super’s mean rubric score rose from 45.1% to 51.8% (+6.7 points), while its full-credit rate (Pass@1, the share of attempts that met every rubric criterion) rose from 19.8% to 25.0% (+5.3 points).
• Software engineering: On the APEX-SWE benchmark, the post-trained Nemotron 3.5 Super's full-credit rate (Pass@1, every test passing) rose from 26.6% to 34.7% (+8.2 points).
• Accounting: On the APEX-Accounting benchmark, the post-trained Nemotron 3.5 Super’s mean rubric score rose from 34.6% to 41.9% (+7.3 points), while the share of tasks it scored zero on fell from 19% to 13% (−6 points).

With only a few hundred training tasks per domain, that’s a meaningful gain on work the model had never seen, and it makes Nemotron 3.5 Super a practical starting point for building domain-specific agents.

Where we trained and tested

We trained and tested the models in Mercor's APEX environments (APEX-Agents, APEX-SWE and APEX-Accounting), which recreate the work professionals do every day, from a law firm's case files to a company's books at month-end close to multi-service production software. Experienced practitioners built every task, and each one is graded against expert-defined pass/fail criteria or automated tests.

We tested each model on held-out benchmark tasks and compared it with the base model in the same evaluation run.

Legal, with Harvey

Harvey builds AI for legal professionals, and its team partners with Mercor to evaluate legal agents. Legal work requires knowing specific facts and context about a case, in addition to general legal knowledge. We trained the model with a targeted dataset to see if it improved the model’s ability to find and use legal information before providing an answer.

We post-trained the model with supervised fine-tuning on model-generated, grader-selected trajectories from Mercor's proprietary APEX legal tasks, then tested it on the legal slice of the APEX-Agents benchmark, with no case files shared between the two.

On the legal slice of the APEX-Agents benchmark, the base model met 45.1% of rubric criteria on average and met every criterion on 19.8% of attempts. After post-training, it met 51.8% (+6.7 pp) and met every criterion on 25.0% of attempts (+5.3 pp).

Figure 1. Mean share of rubric criteria met and full-credit rate (Pass@1), base and post-trained Nemotron 3.5 Super. Source: Mercor evaluation of Nemotron 3.5 Super on the APEX-Agents benchmark, legal tasks.

The improvement is spread across the benchmark. The share of tasks whose average score across attempts was at least 80% of the rubric rose from 15.0% to 21.2% (+6.2 pp), and the share averaging below 20% fell from 25.0% to 18.1% (−6.9 pp).

Figure 2. Share of legal tasks whose average rubric score across attempts is at least a given level, base and post-trained models. Source: Mercor evaluation of Nemotron 3.5 Super on the APEX-Agents benchmark, legal tasks.

Scores improved across every type of legal work in the benchmark. The gain was largest on motion drafting (+23.2 pp) and litigation strategy (+10.3 pp), and legal research, the most common task type, improved by 7.0 pp.

Figure 3. Gain in mean rubric score by task type, base and post-trained Nemotron 3.5 Super. Source: Mercor evaluation of Nemotron 3.5 Super on the APEX-Agents benchmark, legal tasks.

The post-trained model reads the case file in more detail before it answers.

Example: In one task, counsel needs the minimum settlement payment after an oil spill, which depends on a cap and a payment floor defined in a settlement agreement. The base model almost never opened the agreement and subtracted the numbers in the prompt. The post-trained model almost always found it and applied its terms.

Base model - one attempt, scored 0%

Tool calls:
1. code_exec: arithmetic on the prompt's numbers

Post-trained model - one attempt, scored 100%

Tool calls:
1. code_exec: total the owner's fees
2. search_files("*Settlement*")
3. word_read_document_content: Settlement Agreement and Mutual Release
4. word_get_document_overview: Executive Settlement Memo

Software engineering, with Cognition

Mercor built APEX-SWE in partnership with Cognition, the team behind Devin. Cognition reviewed a sample of the benchmark tasks to ensure they accurately represent how real production systems fail and get fixed. Autonomous coding agents like Devin must understand a system, identify problems, make fixes, and confirm everything works as intended. We wanted to see if using a targeted dataset could improve model performance across this entire workflow.

We post-trained the model with supervised fine-tuning on model-generated, grader-selected trajectories from site reliability, security, database and CRM tasks, then tested it on benchmark tasks set in different environments, built on services the model never saw in training.

On APEX-SWE, the base model solved 26.6% of tasks on a given attempt; the post-trained model solved 34.7% (+8.2 pp). It also became more consistent: the share of tasks it solved on every attempt doubled, from 9.9% to 19.9% (+10.0 pp).

Figure 4. Full-credit rate (Pass@1) and share of tasks solved on all four attempts, base and post-trained Nemotron 3.5 Super. Source: Mercor evaluation of Nemotron 3.5 Super on the APEX-SWE benchmark.

Most of the gain came from integration tasks, where the model has to build working systems across several live services.

Example: In one task, the agent had to build a subscription-renewal job across a CRM, email, S3, chat and an issue tracker. The base model wrote the script in one pass, put the issue text in a field the tracker's API ignores, and declared success. The post-trained model read each service's API docs first, used the right field and checked the issues it created. The base model never solved it; the post-trained model solved it in most attempts.

Base model - one attempt, scored 0%

Key steps:
1. cat > /app/lifecycle.py: writes the whole script in one pass
2. curl …/issues/: lists the issues it created
3. Concludes the format is correct

Post-trained model - one attempt, scored 100%

Key steps:
1. cat plane_api_doc.md: reads the issue tracker's API docs
2. Writes the script using description_html, the field the API stores
3. sed -i 's|/work-items/|/issues/|g': fixes the endpoint after an error
4. requests.get(...): fetches the created issues and checks their content

Accounting, with Ramp

Mercor and Ramp built APEX-Accounting to test whether AI can do accounting work – not just answer accounting questions. For an accounting agent, the hard part is the detail of one company's books: reconciling records that disagree, applying the company's own policies and carrying each figure through to the final entry.

We post-trained the model with reinforcement learning on Mercor’s APEX-Accounting tasks, so it learned from its own best-scoring attempts. We then tested it on the APEX-Accounting benchmark, whose simulated companies share no data with the training set.

On the APEX-Accounting benchmark, the base model’s mean rubric score was 34.6%. After post-training, it rose to 41.9% (+7.3 pp), and the share of tasks where it scored zero fell from 19% to 13% (−6 pp).

Figure 5. Mean rubric score and share of tasks scoring zero, base and post-trained Nemotron 3.5 Super. Source: Mercor evaluation of Nemotron 3.5 Super on the APEX-Accounting benchmark.

After post-training, the model took stock of available files more often: 98% of cases, up from 60%. It opened a median of 12 files per task, up from 8.5, and used code to analyze data more often.

Example: In one task, the agent had to review a company's December journal entries for posting errors. The base model loaded the full year’s journal and exceeded its context limit. The post-trained model narrowed the retrieval to December and recovered from an initial filtering error. It scored 45% on the task rubric, compared with 0% for the base model – progress, but not a fully correct result.

Base model - one attempt, scored 0%

Tool calls:
1. excel_v2_read_tab: reads the full journal tab (450,780 characters), which overflows the context

Post-trained model - one attempt, scored 45%

Tool calls:
1. excel_v2_filter_tab: filters for December on column "Date"; error, column not found
2. excel_v2_filter_tab: retries on column C, starting below the title row; 300 of 2,639 rows match

What this shows

Across legal work, software engineering and accounting, a few hundred targeted tasks were enough to improve Nemotron 3.5 Super on held-out professional work. We saw gains with both supervised fine-tuning and reinforcement learning.

The post-trained models also changed how they worked. In legal, the post-trained model read the case files more often. In software engineering, it read API documentation and checked the live system after making changes. In accounting, it looked through all the files before starting and opened more of the source records. For teams building on open models, it’s evident that with realistic environments and a small, well-built dataset, Nemotron 3.5 Super can be shaped for the work their users actually do.