Packet loss and latency between Akamai and NTT in Miami

Any contacts with either Akamai or NTT here ?

This is kind of important as this is affecting three of our RPKI
publication servers (servers which I have de-priorized in Route53 to
prevent any issues for RPs)

I have a ticket open with Akamai but I'm not directly an NTT customer
so any help is appreciated.

A sample MTR report, see between hops 7 and 8. Funnily enough this is
periodic and has a cycle of between 15 and 25 minutes.

%% START MTR TCP IPV4 en 20240517-16:10
Start: 2024-05-17T16:10:01+0000
HOST: rpki-fe-45-79-203-193.rrdp. Loss% Snt Last Avg Best Wrst StDev
  1.|-- 10.204.6.21 0.0% 20 0.5 0.2 0.1 0.5 0.1
  2.|-- 10.204.35.59 0.0% 20 0.3 0.3 0.2 0.6 0.1
        10.204.35.60
  3.|-- 10.204.64.38 0.0% 20 0.3 0.3 0.2 0.6 0.1
        10.204.64.37
  4.|-- lo0-0.gw3.atl1.us.linode. 0.0% 20 19.9 4.1 0.4 19.9 6.1
        lo0-0.gw4.atl1.us.linode.com
  5.|-- ae45.r12.atl01.ien.netarc 0.0% 20 0.5 0.5 0.4 0.7 0.1
        ae48.r11.atl01.ien.netarch.akamai.com
  6.|-- ae-41.a03.atlnga05.us.bb. 0.0% 20 5.2 2.3 0.4 14.8 3.7
        ae0.r11.atl01.ien.netarch.akamai.com
  7.|-- ae-41.a03.atlnga05.us.bb. 0.0% 20 3.3 11.4 0.9 38.5 11.0
        ae-2.r25.atlnga05.us.bb.gin.ntt.net
  8.|-- ae-1.r22.miamfl02.us.bb.g 20.0% 20 7271. 1816. 0.8 7271. 3236.6
        ae-2.r25.atlnga05.us.bb.gin.ntt.net
  9.|-- ae-1.r22.miamfl02.us.bb.g 5.0% 20 7201. 1684. 13.3 7256. 2970.2
        ae-0.r23.miamfl02.us.bb.gin.ntt.net
10.|-- ae-11.a00.saplbr02.br.bb. 0.0% 20 123.2 206.8 13.1 3035. 667.7
        ae-0.r23.miamfl02.us.bb.gin.ntt.net
11.|-- ae1-1326.gw1.nu.registro. 0.0% 20 129.2 125.2 116.7 139.5 5.8
        ae-11.a00.saplbr02.br.bb.gin.ntt.net
12.|-- xe-0-1-2-0.core1.nu.regis 0.0% 20 117.9 174.0 117.6 1133. 225.9
        ae1-1326.gw1.nu.registro.br

Thanks

Carlos

On the measurement that you provided the packet loss doesn’t carry through to the final hop, so surely it’s more likely what you’re looking at here is control plane rate limiting rather than true packet loss?

See slide 33 onwards on https://archive.nanog.org/sites/default/files/10_Roisman_Traceroute.pdf

Responding off list.

-Aaron

May 17, 2024 at 12:42 PM by carlosm3011@gmail.com:

Hi,

just to give some closure here: The issue was fixed, connectivity has
been back to normal for approximately 36 hours already. I don't know
exactly who fixed it or where exactly the problem was located. I guess
it was in the end some sort of collective effort.

Thanks to all those who offered help, I believe these kind of
situations is where the value of the *NOG communities is actually seen
and appreciated.

/Carlos