Calling LinkedIn, Amazon and Akamai @ DE-CIX NY

Hi,

In late October 2018, DE-CIX announced that they would be renumbering their IPv4 address block in New York between 01-28-19 and 01-30-19.

This was followed by numerous reminders in months, weeks and even days leading up to the renumbering activity.

The renumbering activity has come and gone, but LinkedIn, Amazon and Akamai are still using the old IPs.

If three months has gone by and the numerous reminders that have been sent have resulted in these organizations still living on the old IP space, it seems to me that there may be some sort of a disconnect between who receives the notifications from IXPs and how they are filtered upstream.

I’m hopeful that the eyeballs who read this list are some of those folks who should have received the notifications from DE-CIX, or can at least filter the info back downstream to whoever can perform the renumbering activity.

Thanks.

They're not the only ones ... out of all of our peers on that exchange, ~30% haven't updated as of this writing.

I'm a little reluctant to pull our old address until penetration is a little higher.

A lot of huge companies apparently find it tough to find the $75k to hire one more peering person. Not all, though. For many, everything just runs like clockwork.

Greetings Jason,

Thank you for your kind feedback, we have CM planned for today to do the needed change.

Kind regards, Mara

Pinged my contacts in each

Akamai is working on doing our part. Apologies.

Microsoft an Edgecast has not yet made there changes.

Hi all,

Thanks for your support! This helps us getting all peers on the new IPv4 space.

Our looking glass shows which peers already have changed the IP settings (see section “BGP session established”) and which peers are still working on it (see section “BGP sessions down”):

https://lg.de-cix.net/routeservers/rs1_nyc_ipv4#sessions-up

Best regards,
Thomas

Hi Thomas,

You probably should remove sessions with networks explicitly not participating in route servers versus displaying them on a global shame list.

Cheers, -ren

And so it begins — yet another discussion on what does the word
"responsibility" really mean.

Given that e.g. the peering facility in Amazon, according to an
adjacent NANOG ML thread, is in deep deep trouble since Nov 2018, just
shutting down sessions with all of the entries in that shame list is
likely to cause huge disruption and disappoinment.

What triggered that part of the discussion is a logical fallacy along
the lines of: if A is true, then B is true. B is true, therefore A is true.

Here: all networks that didn't already change their peering IP are not
yet connected to the updated route-server. Some networks are not
connected to any route-server. Therefore, those networks did not yet
change their peering IP.

I think you can see what's wrong with that statement.. it does not
follow. That has nothing to do with peering department resources, but
everything to do with the chosen peering strategy.

Best regards,
Martijn

Under what conditions would somebody be present at the exchange and
not talking to the route server *at all* before the IP change?

Some companies just don’t join route servers as a policy. It can be annoying if you want to talk to them, but I understand there can be various reasons why. It gets very annoying when the peering department isn’t responsive to manual peering requests when they’re not on the route server because then they might as well not be there at all, as far as you’re concerned.

Hi Ren et al.,

Thanks for pointing out that some peers do not use the route servers. This group can be subdivided in a group of peers not sending any IP prefixes to the route servers while maintaining a route server BGP session, and a group of peers not even connecting to the route server. The latter do not show up in the "BGP session established" section even if they have applied the required IPv4 changes.

Best regards,
Thomas

I believe most exchange points maintain both route servers and route
collectors.

Generally, most peers will connect to the RS, but not all. As you
mention, some may connect but not send any routes.

However, I believe all peers will connect to the RC. Of course, I can
envisage scenarios where even this could be selectively done. But the
reasons for (not) connecting to the RC are very different from those for
(not) connecting to the RS.

Mark.

Back when I looked more deeply at this (mid-2014 when I was redoing the
AS15169 policy around this area) I saw things rather differently, and my
impression is little has changed since.

Even in exchanges that strongly encourage their use route collectors
were much less connected to than route servers, and few exchanges had
them in the first place.

Part of the problem with advertising on route servers is many clients,
including networks that should know better often treat those routes as a
higher priority than is sensible, in some cases equal or higher than a
PNI link in the same city.

Yes we could have used prepends, but that's not necessarily effective
and affects everyone for the actions of a few, and is why the routes
AS15169 advertised on route servers reduced back then.

Even in exchanges that strongly encourage their use route collectors
were much less connected to than route servers, and few exchanges had
them in the first place.

We, for example, connect to RS's more selectively.

We are more liberal about RC's since they do not have an impact on our
forwarding paradigm, and it helps the exchange point know what's
happening across their fabric. But yes, I do imagine that interest level
of connecting to either an RS or RC could vary, particularly the larger
of a network you are.

Part of the problem with advertising on route servers is many clients,
including networks that should know better often treat those routes as a
higher priority than is sensible, in some cases equal or higher than a
PNI link in the same city.

Well, there are a number of peers that do not have a linear peering
relationship for all routes available at an exchange point, i.e., they
don't see those routes both via the RS and bi-lateral sessions. For many
networks, RS is the true source and bi-lateral sessions are not even
considered.

We may not always peer with an RS, but we will always have bi-lateral
sessions... even when we have sessions to the RS.

Mark.

Do people not know how to use local pref and MED to prefer PNI over route server?

We don’t particularly care how the routes are learned. Routes are routes. Our motivation for or against peering with an RS is granular policy control, and the level of trust we can put in the stability of the same over time. Mark.

Not all routes are created equal. If you have a PNI and an IX connection of equal capacity, obviously the IX connection will fill up first given that there is more opportunity there. Also, there are more moving parts in an IX (and accompanying route servers), thus more to go wrong.