Menu

CI Check Passes Without Reading What It Claims to Check
📰

CI Check Passes Without Reading What It Claims to Check

DEV Community: cicd·Jeremy Longshore·about 12 hours ago
#9dJvGgz6
Reading 0:00
15s threshold

Three of our checks gave the wrong answer on the same day, and one rule fixes all three: a check has to say what it looked at, and what it looked at has to be the real thing. A check on how much input each helper script reads answered PASS while the nine Perl helper scripts it was supposed to check went unread. A screenshot scanner reported green while its plugin did nothing. One test environment rejected a correct plugin, because the machine it ran on was not set up like a normal install.

If you trust a passing check to have looked at the thing it claims to check, the three failures below are worth your time. Each one is a different kind of check: one that reads files, one that reads a folder, and one that reads the environment it runs in.

How does a CI check pass without reading what it claims to check?

A CI (continuous integration) check can pass without reading what it claims to check because nothing required the verdict to name the files it opened.…

Continue reading — create a free account

Join HashtagPLUS to read full articles, follow hashtags, vote, and join the conversation.