Our integration gateways are eleven Linux servers holding the client certificates our carriers issue us. A role called partner_certs puts them in place, and its tasks are tagged certs, so when a carrier sends a renewal we can push just that with ansible-playbook site.yml --tags certs. That takes two minutes. The whole playbook takes forty.
In August we changed site.yml to load roles with include_role instead of the roles list, so each gateway could pick its roles from inventory in a loop. A full run behaved exactly as before, which is how we tested it.
In October a carrier certificate was due to expire at midnight. The new one arrived at four in the afternoon. An engineer ran the playbook with --tags certs and it finished green in under a minute: fact gathering ok, nothing changed, nothing failed. He assumed an earlier run had already put the files in place. At midnight every label request to that carrier started failing its TLS handshake.
Tags work differently for imports and includes.…