F-ckin Leap Seconds, how do they work?

Comments?

Drive Slow
Paul

http://support.ntp.org/bin/view/Support/ConfiguringNTP#Section_6.14.

See International Earth Rotation Service, http://www.iers.org/, particularly
http://data.iers.org/products/6/15003/orig/bulletina-xxv-026.txt

Thanks,
Donald

From: Paul WALL <pauldotwall@gmail.com>
Subject: F-ckin Leap Seconds, how do they work?

Comments?

Addressing the Subject question, _as_asked_ -- "Very well".

*SNORT*

Mechanically, instead of rolling over from second #59 to second #0 of the
next minute, it goes 59->60->0.

The 'why' is to keep terrestrial clocks in sync with celestial references.

Not very well if you have a modern box (RHES/CentOS 6) and Java apps running on them. RHES/CentOS 5 merrily ignored it. Worse, just bouncing the Java stack didn't fix it, it required the box to be rebooted. A sizeable number of annoyed sysadmins tweeting about it this afternoon.

Paul

Hi!

Drive Slow
Paul

Not very well if you have a modern box (RHES/CentOS 6) and Java apps running on them. RHES/CentOS 5 merrily ignored it. Worse, just bouncing the Java stack didn't fix it, it required the box to be rebooted. A sizeable number of annoyed sysadmins tweeting about it this afternoon.

Anything with java running seems hit.
We just finished up a firm round of reboots... :frowning:

Recent Ubuntu boxes and RHES 6... all the same ...

Bye,
Raymond.

Anything with java running seems hit.
We just finished up a firm round of reboots... :frowning:

Recent Ubuntu boxes and RHES 6... all the same ...

Bye,
Raymond.

Yeah, in the process of doing the same.

Might try this for machines with Java applications in order to avoid reboot:

See comment 5

>
> Anything with java running seems hit.
> We just finished up a firm round of reboots... :frowning:
>
> Recent Ubuntu boxes and RHES 6... all the same ...
>
> Bye,
> Raymond.
>
>

Yeah, in the process of doing the same.

It appears to be fixed in Linux 3.4 [1]. According to the original commit [2] it... | Hacker News

Might try this for machines with Java applications in order to avoid
reboot:

769972 - Java is choking on leap second.

See comment 5

And we have verified that this clears the issue for us. YMMV.

We haven't had any issues with any of our VMs. We run several of our own Java/Tomcat apps, Jira, and Confluence on a mixture of Solaris and CentOS 5 and 6. We do not run NTP on our VMs though; instead, we rely on VMware Tools to sync the VMs' time with the ESXi hosts. The ESXi hosts run NTP.

Same here with KVM guests on Scientific Linux 6 (RHEL 6 clone) hosts.
No issues on SL 6 and CentOS 5 guests. We also do not run NTP on the
VMs, only on the hosts. The guest VM kernels did not log any leap
second clock change, but appear to have the same time as the hosts.

The hosts DID have issues though. The "reset the date" workaround
solved the issue immediately, with no requirement to restart anything,
including ntpd. The hosts logged leap second clock updates:

Jun 30 19:59:59 vmhost kernel: Clock: inserting leap second 23:59:60 UTC

My Fedora 16 laptop was also being sluggish due to chromium-browser
sucking up CPU. That too was fixed immediately by resetting the date.

Talk about people not testing things, leap seconds have been around since 1961. There have been nine leap seconds in the last twenty years. Any system that can't handle a leap second is seriously flawed.

A useful explanation may be found here:

  http://blogs.discovermagazine.com/badastronomy/2012/06/30/wait-just-a-second-no-really-wait-just-a-second/

---rsk

i didn't reboot:

/etc/init.d/ntp stop
date `date +"%m%d%H%M%C%y.%S"`
/etc/init.d/ntp start

seems to calm things right back to normal.

--jim

Roy, this was a problem in only certain kernel versions. Unfortunately the range of versions affected are pretty widely deployed right now. Earlier and later versions did not have the problem.

Yes.

But I'm sure the reference was to the Insane Clown Posse spawned meme, which
rapidly became popular with liberals, dissing the apparentl distaste of
conservatives for the scientific method:

http://knowyourmeme.com/memes/fcking-magnets-how-do-they-work

Cheers,
-- jra

Do you happen to know all the kernels and versions affected by this?

2.6.26 to 3.3 inclusive per news.ycombinator.com/item?id=4183122

Well, my 2.6.32 CentOS6/64 machine, which is not running Java, just purred
right along, logging the leapsecond at 7pm, and not even blinking, so...
(Amazon EC2, NTP enabled; 3 strat-2s from us.pool)

Cheers,
-- jra

My centos 6/64 running 3.0 seemed to weather it too. I'm not quite
clear on what I should be looking for to classify it as being "broken" though.

Mike

Made the press..
http://www.washingtonpost.com/business/technology/leap-second-bug-takes-down-reddit-and-a-bunch-of-other-sites/2012/07/02/gJQAlXg1HW_story.html