isotesterN:s are sometimes put offline and never back online

I see that isotester2 and isotester4 are currently offline from Jenkins’ PoV. They page (https://jenkins.tails.boum.org/computer/isotester2/) reads “Disconnected by anonymous : Taken_down_for_reboot”. They both are running fine.

For isotester2, the last message I see in the logs is:

Nov 21, 2015 6:09:58 PM jenkins.slaves.JnlpSlaveAgentProtocol$Handler$1 onClosed
WARNING: NioChannelHub keys=6 gen=72376715: Computer.threadPoolForRemoting [#1] for + isotester2 terminated
java.io.IOException: Connection aborted: org.jenkinsci.remoting.nio.NioChannelHub$MonoNioTransport@2bbbdba1[name=isotester2]

… which seems to be normal when we take a slave down (I’ve compared with another isotesterN that was successfully taken offline then back on). This means that it was taken offline 1 day and 14 hours ago, which happens to be its uptime, so it did reboot just fine.

My conclusion is that our way of putting a node back online while it’s rebooting is not working reliably.

I’m curious how often this happens. I don’t remember seeing this recently, but it might be that someone else has been dealing with it manually.

Feature Branch: jenkins-jobs:SponsorS-leftovers, puppet-tails:SponsorS-leftovers

Parent Task: #5288 (closed)

Related issues

• Related to #9486 (closed)
• Related to #11295
• Related to #16959
• Related to #17216
• Blocked by #10068 (closed)

Original created by @intrigeri on 10601 (Redmine)