A black-box model detector returns the following result:

Model A: 99.1%

Model B: 0.6%

Model C: 0.3%

The obvious interpretation is that the endpoint has a 99.1% probability of running Model A.

In many systems, that interpretation is wrong.

The number may only mean that Model A received 99.1% of the normalized probability mass among the candidates included in the detector’s reference set. If the actual model is absent from that set, the detector can remain highly confident while assigning the endpoint to the wrong model.

This is not a minor wording issue. It is the difference between:

“Model A is the closest candidate we know.”

and:

“The endpoint has been verified as Model A.”

The first is a ranking result.…