Every Measurement Has A Physical Limit: Scientific AI Pretends Otherwise – OpEd

Key Takeaways

• Fundamental physical and statistical limits (Abbe’s diffraction limit, Cramér-Rao bound, Fisher information) mean that no amount of sophisticated analysis or AI can extract more spatial or parametric information than the underlying measurement actually contains. 
• AI models can produce confident numerical outputs even when the experiment is weakly informative, ambiguous or nearly non-identifiable, because confidence scores primarily reflect the model’s internal assumptions rather than the information content of the data. 
• Scientific AI systems should therefore incorporate structured, fail-closed abstention mechanisms that refuse to issue a qualified estimate when the measurement geometry, noise level or sensitivity is insufficient, and should report both error and coverage so that limits on the claim are treated as valuable scientific information rather than failure.

AI can produce a precise estimate even when an experiment no longer contains enough information to identify the target. Science should treat abstention as a valid result, not a failure.

In 1873, Ernst Abbe formalized a hard truth about microscopes: optical resolution is constrained by the wavelength of light and the aperture of the instrument. Below that limit, the problem is not that the lens needs a cleverer analyst. The measurement itself does not carry arbitrarily fine spatial information.

Seven decades later, C. R. Rao and Harald Cramér expressed the same idea in statistical language. Under the assumptions of the Cramér-Rao framework, Fisher information places a lower bound on the variance attainable by an unbiased estimator. The floor is set by the measurement model, its sensitivity to the quantity being estimated, and the noise. Better mathematics can use available information more efficiently. It cannot manufacture information that the experiment never recorded.

Nature keeps accounts.

Artificial intelligence is now becoming part of the analytical machinery of science. Learned models solve inverse problems, infer parameters from sparse observations, separate overlapping signals, classify images, and extract structure from data too large for manual inspection. Physics-informed neural networks, for example, have been used for forward and inverse problems in differential equations, while later work has explicitly confronted the identifiability of parameters inferred by such models.

These systems can be extraordinarily useful. But they introduce an easy category error: confusing confidence in an answer with evidence that the measurement could identify the answer in the first place.

A confidence score can tell us something important about the model under its assumptions. It does not, by itself, prove that the current experiment contains enough independent information about the target quantity. Priors, regularization, learned correlations, and familiarity with the training distribution can all keep an output numerically stable even as the measurement geometry becomes weak, ambiguous, or nearly non-identifiable. Research on predictive uncertainty has already shown that uncertainty quality itself can deteriorate when data move away from the conditions on which the model was developed.

Put more simply: confidence is evidence about the model. It is not automatically evidence about the experiment.

Consider three ordinary scientific situations. Two astronomical sources become so blended that their signatures are nearly indistinguishable. A navigation receiver loses the geometric diversity needed to separate state variables. Two spectral components become so similar that many combinations explain the same measured curve. An algorithm can still return a number in each case. The harder question is whether the data uniquely support that number without leaning on assumptions that have silently become more important than the measurement.

That distinction matters most near the edge of a scientific claim. Strong signals are comparatively forgiving. At the boundary, where a source is faint, two mechanisms are difficult to separate, or an effect would be consequential if real, the information supplied by the measurement can collapse faster than a model’s confidence score does.

Scientific AI therefore needs a capability that sounds unimpressive but is foundational to measurement science: the ability to refuse a qualified estimate because the evidence is physically insufficient.

Machine learning already has a research tradition called selective prediction, in which a system can abstain rather than predict on every case. SelectiveNet, for example, explicitly optimized the tradeoff between prediction risk and coverage. That is useful, but scientific inference needs an additional question upstream of model confidence: can the instrument, geometry, noise level, and measurement model identify the requested quantity at all?

Structural biology offers a useful cultural example. AlphaFold does not merely publish a three-dimensional structure; it also reports per-residue confidence. Researchers have learned not to treat low-confidence regions as inconvenient blanks to be cosmetically filled. In the human-proteome analysis, AlphaFold confidence was even evaluated as a predictor of unresolved or disordered regions. The lesson is not that a model-confidence score equals physical identifiability. It does not. The lesson is that a scientific community can learn to treat limits on a model’s claim as information rather than embarrassment.

The next step is to apply the same discipline to the measurement itself.

Before accepting an AI-assisted scientific estimate, ask what the physics permits. Is the target identifiable under the current geometry? How does the relevant Fisher information or sensitivity change as the experiment degrades? What uncertainty floor follows from the measurement model? Does a different parameter combination produce nearly the same observation? If the requested precision lies below what the data can support, a confident model output should not be upgraded into a confident scientific claim.

Science does not need another branded AI confidence score. Established tools already exist: conditioning diagnostics, Fisher information, Cramér-Rao bounds, posterior-width analysis under declared priors, residual checks, and explicit tests for distribution shift. The important change is architectural. The model may produce a candidate estimate, but release of a qualified scientific estimate should depend on evidence about whether the measurement supports it.

That permission should also be fail-closed. If the experiment is insufficiently informative, the output should not be a mysterious null or an invisible software exception. It should be a structured abstention: what quantity was requested, which information condition failed, what threshold had been fixed, what data and configuration were used, and what new measurement would be needed to make the question answerable.

There is no universal numerical cutoff. A telescope, a seismic array, a navigation geometry, a medical imager, and a plasma diagnostic lose information in different ways. The principle can travel across fields. The threshold cannot simply be copied from one instrument to another.

Nor should the abstention rule be tuned after the embarrassing cases are known. The information diagnostic, threshold-selection procedure, and acceptable error criterion should be fixed on development or calibration data and then applied unchanged to untouched evaluation cases. Otherwise abstention becomes another form of post-selection: the system learns to be humble only where the analyst already knows it was wrong.

This creates a straightforward standard for journals, reviewers, laboratories, and funders. An AI-assisted method that makes physical estimates should state the information limits of the measurement, show how those limits are checked at runtime or analysis time, report both error and coverage, and disclose when the system refuses to make a qualified estimate. A method that answers fewer questions may be scientifically stronger if it can explain why the rejected questions were not answerable from the evidence.

The goal is not to slow scientific AI. It is to keep AI from erasing a distinction that experimental science spent centuries learning: a calculation can be sophisticated while the observation remains insufficient.

A paper that cannot show where its measurement stops supporting its inference is not ready to turn a model output into a discovery claim. Editors should view that omission with the same instinct they bring to a p-value of exactly 0.049 after a long chain of analytic choices: not automatic rejection, but an immediate request to see the full evidentiary path.

We grade scientific AI today on the questions it answers correctly. We should start grading it on the questions it correctly refuses to answer.