Menu

1 / 5

An AI Agent Ran 187 FEM/CFD Solves Over MCP. Here's How Many Matched the Textbook

DEV Community: machinelearning·NovaSolver·about 13 hours ago
#tGAkKUnL
Reading 0:00
15s threshold

From 6 to 12 November 2026, using dates as recorded in the logs, an AI agent (Claude Code) drove an open-source simulation app, NovaSolver Studio, through its MCP server. Structural, modal, buckling, thermal, flow, electrostatic and DC current problems. Every call and every reply was written to a log.

The headline numbers, all from those logs:

MCP calls 3,355 (49 log files)

Solves started 187

Accepted comparisons with a reference 136 (from 118 analyses)

Median difference 0.13%

Within 1% / within 3% 105 / 130

Above 5% 2

The accepted set has 132 external-reference comparisons (formulas, series, correlations or independent 2D FEM) and 4 comparisons with another mesh of the same Studio model; 140 comparisons were recorded including reference-only results.

The interesting part is not the median. It is what got thrown away before those 136, and two types of setup mistake that passed a mesh or reference check.

What "the agent ran it" means here

It was not a chat where the model clicked around freely.…

Continue reading — create a free account

Join HashtagPLUS to read full articles, follow hashtags, vote, and join the conversation.