Linux Leap-Second problem
So, this weekend was quite an interesting one, as on July 1st 02:00 local time (00:00 UTC) a leap-second was added via NTP. This caused serious problems for all my Java Virtual Machines and mysql databases.
If your system has printed the following line (in dmesg), a leap-second has been added recently:
Clock: inserting leap second 23:59:60 UTC
On most of my systems, the JVM’s would spike to 100% cpu load over all cores, mysql seems to also do this.
The work-around/fix at this time is to run:
date date +"%m%d%H%M%C%y.%S"
Hopefully this will be fixed in the kernel before the next leap-second is added. Which could be as soon as 2013/01/01, though probably later.
Update 2018:
It was indeed fixed before the next leap-second, which occurred somewhere not too long after. I haven't encountered this problem since.