From 6 to 12 November 2026, using dates as recorded in the logs, an AI agent (Claude Code) drove an open-source simulation app, NovaSolver Studio, through its MCP server. Structural, modal, buckling, thermal, flow, electrostatic and DC current problems. Every call and every reply was written to a log.
The headline numbers, all from those logs:
MCP calls 3,355 (49 log files)
Solves started 187
Accepted comparisons with a reference 136 (from 118 analyses)
Median difference 0.13%
Within 1% / within 3% 105 / 130
Above 5% 2
The accepted set has 132 external-reference comparisons (formulas, series, correlations or independent 2D FEM) and 4 comparisons with another mesh of the same Studio model; 140 comparisons were recorded including reference-only results.
The interesting part is not the median. It is what got thrown away before those 136, and two types of setup mistake that passed a mesh or reference check.
What "the agent ran it" means here
It was not a chat where the model clicked around freely.…