Introducing Folkbench Model Degradation Detection: Auditing API Relays with Current-Events Probes, Model Fingerprints, and Direct Official Baselines
Originally published on the Folkbench blog: https://folkbench.com/en/blog/how-folkbench-detects-model-degradation
Folkbench has launched model degradation detection inside the Integrity Monitor . Conventional model leaderboards evaluate what a model can achieve under official or single-host conditions, while typical API relay directories stop at listed pricing, HTTP uptime, and response latency. Neither answers the central operational question for relay users: when a model is served through a third-party API relay, do its reasoning depth, constraint adherence, and model-level behavior remain consistent with the direct official API?
To close that evaluation gap, Folkbench embeds an automated probe suite spanning current-events and model-fingerprint dimensions, calibrated continuously against independent direct official API keys within the same measurement…