Optional checks are how bad agents reach main.

Teams add agent-evals to Actions, celebrate, then leave the check non-required. The red X becomes a suggestion. The merge button stays green if someone is in a hurry.

Required means required

In GitHub: branch protection → require status checks → select the eval job name exactly. If the name drifts after a workflow rename, protection silently stops enforcing. Pin the check name in the same PR that renames the workflow.

Also fail closed on infra errors. A missing API key in CI secrets is not “skip evals.” It is a red build.

- name : agent evals env : OPENAI_API_KEY : ${{ secrets.OPENAI_API_KEY }} run : | if [ -z "$OPENAI_API_KEY" ]; then echo "OPENAI_API_KEY missing" exit 1 fi python -m evals.ci_gate --suite merge

Enter fullscreen mode

Exit fullscreen mode

Split suites so merges stay fast

One fat suite that needs twenty minutes will get bypassed.…