We tried using “time.nist.gov” which returns varying round-robin addresses
(as the link says), but Cisco IOS resolved the FQDN and embedded the
numeric address in the “ntp server” config statement.
After letting the new server config go through a few days of update cycles,
the drift, offset and reachability stats are not anywhere as good as what
the stats for the Navy time server are - 192.5.41.41 / tock.usno.navy.mil.
I would greatly appreciate and feedback / advice, etc.
NTP has vulnerabilities that make it generally unsuitable for provider networks. I strongly recommend getting a GPS-based time server. These are as cheap as $300. Here is one I use quite a bit:
NTP has vulnerabilities, so using an external source opens your networks
and infrastructure to disruptions.
Going with an internal GPS/GLONASS/RADIO based S1 allows you to restrict
incoming traffic and not rely on volunteers or external entities (which may
undergo maintenance or budget issues).
My preference is more so something akin to the GLN180PEX (I am not
affiliated or paid to endorse this product). It allows you to use commodity
hardware (like a decommissioned 1U or several preferably) and creation of
ones own reliable internal time source(s). Introducing black boxes into a
production (revenue generation or expected services by paying customers)
environment is undesirable.
From there setting up NTPd, Chronyd, and PTPd is up to you.
Relying on satellites may seem like just another external reliance, but the
next life is proposing a design life of 12 years.....
_Everything_ has vulnerabilities and using _any_ external source opens your network and infrastructure to disruptions. NTP has been used for DDoS amplification attacks recently, but so has DNS and other well known/heavily used protocols.
With the right protections, syncing with an external NTP source is perfectly acceptable and safe.
Further, it’s generally a good idea to ‘peer’ (not just sync) your NTP servers with a few external sources. This removes the dependence on a single source and helps ensure that your time source agrees with the rest of the world. Peering requires interaction with the owners of the remote site, which establishes a basic level of trust that they’ll provide an accurate and stable service.
I’ve attached a diagram (sanitized) of what our NTP service will look like after an upcoming refresh.
All external sources are trusted and will be peered. All time devices peer with four other sources to ensure there is always a live source to sync/peer with.
A DNS record with round-robin is used for local clients to connect to the local Stratum 2 devices. The Stratum 1 GPS will not be directly accessible by users.
So how does this stop from distributing time to their customers via
NTP?
GPS doesn't save the protocol, in particular where the S1 clocks
involved are embedded devices with rather coarse clocks and timestamping.
--msa
It doesn't really. Granted there are a lot of CVEs coming out for NTP the
last year or so. But I just don't think there are that many attacks on it.
It's just not worth the effort. Changing time on devices is more an
annoyance than anything, and doesn't necessarily get you into a device.
Sure you can hide your tracks a little by altering time in logs and altering
it back, but that's more of an in-depth nation-state kind of attack, not
going to be a script kiddie kind of thing. Just follow the best practices
for verifying packet sources and NTP security itself, and you should be ok.
Going with an internal GPS/GLONASS/RADIO based S1 allows you to
restrict incoming traffic and not rely on volunteers or external
entities (which may undergo maintenance or budget issues).
You mean the GPS network is not managed by an external entity? With
budget issues?
Note that they *do* have motivation to keep it working, simply because
so much of their *own* gear (from gear for individual soldiers all the
way to strategic bombers and aircraft carriers) wants a working GPS signal...
Note that they *do* have motivation to keep it working, simply
because so much of their *own* gear (from gear for individual
soldiers all the way to strategic bombers and aircraft carriers)
wants a working GPS signal...
Yes, but they may switch it off for civilian use (by going encrypted,
for instance) at any time, if it is better for *their* operations.
We tried using “time.nist.gov” which returns varying round-robin addresses
(as the link says), but Cisco IOS resolved the FQDN and embedded the
numeric address in the “ntp server” config statement.
Depending on your hardware platform your Cisco Router is likely not
a great NTP server. IOS is not designed for hyper-accuracy.
After letting the new server config go through a few days of update cycles,
the drift, offset and reachability stats are not anywhere as good as what
the stats for the Navy time server are - 192.5.41.41 / tock.usno.navy.mil.
The correct answer here is to run multiple NTP servers in your
network. And by servers I mean real servers, with good quality
oscellators on the motherboard. Then configure them to talk to
_many_ sources. You need 4 sources of time minimum to redundantly
detect false tickers. If you're serious about it then find ~10
Stratum 1 sources (ideally authenticated and from trusted entities),
one of which could be GPS as several have suggested. You'll then
have high quality false ticker rejection.
Configure all of your devices to get NTP from the servers you run
using authentication.
I seem to recall that the positional accuracy is degraded to "on the order
of 200 meters" when selective availability is enabled (and yes, they *can*
do it by geographic area by careful turning on/off on each orbit of each
satellite - devices are told the signal is degraded and can downvote that
signal. So unless you're in the war zone, you'll just notice your device
reporting a lock on 2-3 fewer signals).
The upshot is that Grace Hopper told us about electricity moving a foot per
nanosecond - which means that the time-domain jitter introduced is all of
600 nanoseconds or so. In other words, anybody using it to feed an NTP
server will hardly notice on the millisecond level. If you're trying to
use NTP to get microsecond stability, you'll notice - but you have enough
*other* things to do correctly to do this that it shouldn't be an actual issue...
I think the correct answer here starts with a question --- what level of
time accuracy is required for the local NTP server(s)? Which then begs
the question, what level of accuracy is needed for the clients?
A shop with a client need for nanosecond accuracy begs for an entirely
different solution set than a shop where a millisecond of accuracy is
needed on the clients, and still a different solution set that a shop
where "a few milliseconds either way" is quite OK.
You can get pretty nutty with this, and it's fun to do, but regular NTP over the internet is good enough for millisecond accuracy. A default install with pool servers is pretty good. A custom config with a local NTP server (with less, possibly more symmetric network latency) is a little better, but for common sysadmin needs like cron jobs and log correlation, you likely won't notice a difference. I would worry more about having that config distributed correctly and monitoring all your servers to make sure their NTP is healthy, rather than worrying about the source of your NTP sync. The pool servers are fine, and NTP is good at deciding when they're acting up. The computer running NTP doesn't have to be anything special, but beware of VMs - depending on the virtualization type and the guest OS, you may not even be able to get NTP to engage because of the clock instability. Chrony might work better for VMs. For a linux NTP server, I prefer modern Intel CPUs with invariant tsc - linux will use it as a clocksource (cat /sys/devices/system/clocksource/clocksource0/current_clocksource
) . A Raspberry Pi or something in between also works equally well if you're going to be syncing this over a jittery shared network anyway. I would suggest having more than one server, in different locations if you can, and if you're able to supplement with pool servers, add those too. The most likely failure mode you're going to have is that it doesn't work at all because it's not running, it can't correct the local clock because of excess instability, or you lost all network connections. Worrying about whether you have 4 or 8 servers is kind of moot in any of those (much more likely) faults.
Changing time on
devices is more an annoyance than anything, and doesn't necessarily
get you into a device.
So, you are not worried about getting DoS'ed?
How about you set the time on your server ahead by 5 years. Got any
idea what would happen?
Most of your passwords would expire.
All your SSL certs would expire.
All your TOTPs, like Google Authenticator would fail.
All your IPSEC tunnels would drop, and refuse to restart.
Many of your cron jobs would got nuts, possibly deleting all your logs.
Much of your DNSSEC would expire.
Many of your backups would be deleted since they 'expired'.
Until recently, setting your iPhone to 1 Jan 1970 would brick it.
I'm sure there are many more examples, but likely you can no longer log
in, via SSH or HTTPS, and your iPhone is dead. I think any of those
would qualify as more than an annoyance.
