Someone didn't get the leap second memo...

Mailman List Mirror (Read Only)
1

[root@hayden ~]# ntpq -p
     remote refid st t when poll reach delay offset jitter

2

Had a set of Cisco ASR1004s running 15.4(3)S1 (on IOS-XE 03.13.01.S) all restart at around midnight UTC, and all with `Last reload reason: Watchdog`, with those boxes being at separate DCs in different regions. I'm assuming when I call TAC I'll get a "whoops; sorry".

3

We had some ASR1001s routers reboot.

Looks like we hit this bug:
https://bst.cloudapps.cisco.com/bugsearch/bug/CSCvb01730

4

We manage a couple of these too, running an affected version. :frowning: I've applied the suggested workaround, but not the recommended 24h in advance of the unexpected leap second. I'll find out tomorrow if that mattered.

5

That bug note indicates all 3.13.* are vulnerable to this, but our 3.13.3 lab router seemed ok, no reload. Logged:

Dec 31 23:59:59: %IOSXE-5-PLATFORM: R0/0: kernel: Clock: inserting leap second 23:59:60 UTC

But the release notes seem to indicate that bugs:
CSCut82336 ASR1002-X: Handle leap second in ToD IN
CSCut65374 PTP Leap Second: ASR1002-X incorporate leap second addition 6/30/15

are open caveats in 3.13.3, and resolved in 3.13.4. I guess I'll find out which of our 300+ ASR 1000s misbehaved on Tuesday. They're pretty much all on 3.13.5. Hopefully none had an issue. There are a lot of PSIRTs with ASRs, that's the main reason we're on later 3.13 versions.

Chuck