Machine learning for physical systems is often reported the same way as machine learning for everything else: a single accuracy or error number on a held-out test set. For engineering problems, that number is a weak proxy for what actually matters.
A model that scores well on average can still be useless or unsafe in the situations where an engineer most needs it. Physical systems carry structure, constraints, and consequences that a scalar error metric does not see.
Downstream use matters
A surrogate model is rarely the end product. It feeds a decision: a design iteration, an optimisation loop, a maintenance schedule, or a safety margin. The relevant question is not only how low the test error is, but whether the model changes the decision and whether that change is correct.
Two models with identical average error can produce very different downstream behaviour. One might be wrong in a smooth, predictable way that an engineer can account for. The other might be wrong precisely at the design points that matter, while looking good on average.
Distribution shift matters
Test-set accuracy assumes the future looks like the past. Physical systems routinely violate this through new geometries, load cases, materials, damage, wear, and operating regimes the model never saw. An in-distribution evaluation says little about behaviour where the model will actually be used.
Physical plausibility matters
A prediction can be numerically close and still be physically wrong by violating conservation, monotonicity, symmetry, or known limiting behaviour. A model that respects physical structure can be more useful than a marginally more accurate one that does not.
Uncertainty matters
A useful model knows when it does not know. Without calibrated uncertainty, every prediction carries the same false confidence and the engineer has no signal for when to return to simulation or physical testing.
What to evaluate instead
- Behaviour under distribution shift, not just in-distribution error.
- Error at the design points and decisions the model will actually inform.
- Physical plausibility and respect for known constraints.
- Calibrated uncertainty and graceful failure.
- The quality of the decision the model enables, end to end.
Accuracy remains necessary, but it is not sufficient. Engineering value depends on the decisions a model enables, not only the metric it improves.