Is latency equivalent to RTT?

Mailman List Mirror (Read Only)
1

Assuming that one has measuring devices in every PoP, do you think it

is

harder to measure a full matrix of one way latency compared to

measuring a

full matrix of RTT?

Your problem with 1-way is finding a way to do it thats reliable, for rtt

ping

is about as simple as you can get and works a treat but for 1-way you

have to be

a bit more creative....

I assume that it's fairly common for people to have Solaris or Linux boxes
in every PoP to do measurements. In that case, the difficulty isn't in
measuring one-way latency, it's in synchronizing the time on all the
servers. And with fairly cheap GPS and CDMA clocks that is a lot
easier/cheaper than it once was.

I really think that there are a lot of people out there who do not
understand that RTT is not the same thing as two times the one-way
latency. In other words if you measure (A-to-B + B-to-A) / 2 then you have
failed to learn anything about one-way latency on either path A-to-B or
B-to-A. Yet that's precisely what people are doing when they measure RTT
and then assume that RTT/2 is equal to the one-way latency A-to-B.

--Michael Dillon

2

But what GPS clock can you install in a datacenter? AFAIK, they all require roof (or at least window) access in order to install the antenna. (At least, all the GPS based ntp servers I've looked at do).
Is that not true of CDMA servers?

How have others solved this issue? (Short of owning their datacenters.)

3

I really think that there are a lot of people out there who do not
understand that RTT is not the same thing as two times the one-way
latency. In other words if you measure (A-to-B + B-to-A) / 2 then you have
failed to learn anything about one-way latency on either path A-to-B or
B-to-A. Yet that's precisely what people are doing when they measure RTT
and then assume that RTT/2 is equal to the one-way latency A-to-B.

What you've said is true, but there are times when RTT measurements can be
useful. If you are making measurements on your own backbone and know that
paths are either symmetric or have a well-know latency ratio, then RTT
might not be a bad measurement. Moreover, though this isn't really what
the original question regarding "latency" measures was about, there are
times when you don't care much about the absolute value of the RTT, but
rather you're looking for latency spikes (i.e. RTT(t) - RTT(t-1) > 20ms )
as a crude warning of either a topology change (i.e. my traffic that normally
goes direct from Boston to NYC is now going via Chicago) or of some sort of
link/router issues. When this happens, it might not tell you which path or
device is having a problem, but it is an easy monitor to set up to tell you
that something has changed...

Eric :slight_smile:

4

> I assume that it's fairly common for people to have Solaris or Linux
> boxes
>
>in every PoP to do measurements. In that case, the difficulty isn't in
>measuring one-way latency, it's in synchronizing the time on all the
>servers. And with fairly cheap GPS and CDMA clocks that is a lot
>easier/cheaper than it once was.
>
But what GPS clock can you install in a datacenter? AFAIK, they all
require roof (or at least window) access in order to install the
antenna. (At least, all the GPS based ntp servers I've looked at do).

Half the sky is sufficient, so if you have a window that can open, you are
usually fine. Just stick the receiver outside the window.

Is that not true of CDMA servers?

No, they work inside buildings if there is not too much interference. An
advantage of these units is that they work whereever a CDMA phone works,
so it really easy to check. We have 2 CDMA units installed in
data-centers and they just work.

How have others solved this issue? (Short of owning their datacenters.)

For GPS: There are units on the market that can work with 250m of regular
CAT5 cable between GPS receiver and PC, if the building has structured
wiring, get permission to access the roof, then use the existing wiring to
reach the computer room. Getting permission to access the roof is the
tricky part.

Henk

5

> I assume that it's fairly common for people to have Solaris or Linux
> boxes
>
>in every PoP to do measurements. In that case, the difficulty isn't in
>measuring one-way latency, it's in synchronizing the time on all the
>servers. And with fairly cheap GPS and CDMA clocks that is a lot
>easier/cheaper than it once was.
>
But what GPS clock can you install in a datacenter? AFAIK, they all
require roof (or at least window) access in order to install the
antenna. (At least, all the GPS based ntp servers I've looked at do).
Is that not true of CDMA servers?

cdma recievers require a visible cdma signal so if your cdma phone works
in the datacenter the cdma reciver will as well...

How have others solved this issue? (Short of owning their datacenters.)

Ask/pay/rent for roof access for the antenna, it's not large but it does
work better with an unobstructed video of the sky. There's also the
question of how many stratum=1 time sources a given datacenter needs. if
there's someone else with one it should be fairly unintrusive to share.

Also if you just need a high level of syncronization between the time on
all your hosts you can just deploy one standalone ntp server, sync it
against public time sources and get everything synced against that. its
probably a 95% solution to most people's timeing needs.

6

On Wed, May 14, 2003 at 08:02:28AM -0700, Steve Francis quacked:

But what GPS clock can you install in a datacenter? AFAIK, they all
require roof (or at least window) access in order to install the
antenna. (At least, all the GPS based ntp servers I've looked at do).
Is that not true of CDMA servers?
How have others solved this issue? (Short of owning their datacenters.)

  http://www.endruntechnologies.com/

  I've got about 20 of their Praecis Ct units installed in various
datacenters around the world, and they work in ... most of them.
There are a couple that're buried deeply enough inside buildings
that they can't receive enough of a signal, but that's about
only 5% or 10% of the time. Most of the time they just work.

>I assume that it's fairly common for people to have Solaris or Linux
>boxes
>
>in every PoP to do measurements. In that case, the difficulty isn't in
>measuring one-way latency, it's in synchronizing the time on all the
>servers. And with fairly cheap GPS and CDMA clocks that is a lot
>easier/cheaper than it once was.

(Warning, somewhat involved rant below from someone who's spent
too much time trying to measure one-way latencies):

It's easier, but it's still not trivial. There are some systems
level issues that still creep up when you're doing one-way measurements
that you might not often think about. For instance, FreeBSD (and
now Linux, I think) have a recvmsg option to get a card-level
timestamp of when a packet was _received_, called SO_TIMESTAMP, but
there's no standard way of finding out when packets were actually
transmitted. The measurement folk at CAIDA have a kernel patch
for FreeBSD that puts a kernel-level timestamp in outgoing packets
to get rid of that source of uncertainty:

Using both of those gets you away from calling gettimeofday() to
time your packets. Kernels are usually pretty good about trying
to make sure the user visible clock doesn't go backwards, but
when you're checking it against an external source, you can notice
some really weird happenings. On FreeBSD, setting
kern.timecounter.method=1 will help.

Anyway, while getting the basic equipment up and running is
really easy, there's more to getting accurate one-way latencies
than just running ping once and a while. Be nice if we had
a high-resolution equivalent of the ICMP_TSTAMP request
type, so we could just use a ping variant.

    -Dave

7

On Wed, May 14, 2003 at 12:00:02PM -0400, David G. Andersen quacked:

  I've got about 20 of their Praecis Ct units installed in various
datacenters around the world, and they work in ... most of them.

  Eek. I should clarify this: The CDMA units work in the US,
some spots in canada, and korea. Unfortunately, GSM doesn't
provide the same time synch capabilities, so Europe is out.

  -Dave

8

## On 2003-05-14 12:05 -0400 David G. Andersen typed:

  Eek. I should clarify this: The CDMA units work in the US,
some spots in canada, and korea. Unfortunately, GSM doesn't
provide the same time synch capabilities, so Europe is out.

Just for completness Israel has a CDMA cell-operator
(Yes it's not in NA but neither is (South?)Korea)

9

When I still worked at Equinix, we had NTP chimers in all the datacenters
which customers could either hardwire (IRIG-B, etc) or peer with NTP.

However, it seemed that people who cared about time always wanted to
have their own NTP chimers which they owned and controlled. The
concern about relying on third-parties, and even the theorotical
NTP attacks.

10

When I still worked at Equinix, we had NTP chimers in all the datacenters
  which customers could either hardwire (IRIG-B, etc) or peer with NTP.

Slick!

  However, it seemed that people who cared about time always wanted to
  have their own NTP chimers which they owned and controlled. The
  concern about relying on third-parties, and even the theorotical
  NTP attacks.

A well-chosen set of sources obviates the need for this as well. NTP
is pretty good at spotting and ignoring martian sources.

matt ghali

--mghali@snark.net------------------------------------------<darwin><
   Flowers on the razor wire/I know you're here/We are few/And far
   between/I was thinking about her skin/Love is a many splintered
   thing/Don't be afraid now/Just walk on in. #include <disclaim.h>