Lab had a re-arrangement. We had decided to add a few new machines, sort the inventory and consolidate resources such as NIC cards, hard drives, memory dimms, etc. This also triggered a conscious decision of moving Jenkins service (the Jenkins master) to a new server which was new on pretty much every aspect, including OS. When all power was resumed and CI jobs kicked off, they hung.

I have a stage “hex” that was supposed to run some tests, estimated to be completed within 15 min. After the lab updates, a job was kicked off; it took around 7 hours without ending. As a result of that I blindly aborted it and gave it another shot, hoping it was only a glitch of CI flows. Another hour passed with the latest job while there’s still no good news.…