Re: [CloudOps] [outages] Major loss/latency on Twelve99 around NY/SJO

Fwiw, still ongoing

I raised an AWS ticket and they’re looking into it. If there’s any friendly AWS Net Engineers on list, your help removing Twelve99 from route would be appreciated. Cross-posting to Nanog for wider reach.

See traffic on Twelve99 re-routed via LAX now. Seems to be stable, for the moment:

My traceroute [v0.95]
VONMVPCVXYVFD (172.20.0.219) → 216.147.3.209 (216.147.3.209) 2024-08-15T11:29:35+0100
Keys: Help Display mode Restart statistics Order of fields quit
Packets Pings
Host Loss% Snt Last Avg Best Wrst StDev

  1. 172.20.0.1 0.0% 68 0.7 1.0 0.7 1.9 0.2
  2. lo0-0.bng2.ixn-lon.zen.net.uk 0.0% 67 12.3 8.4 5.1 43.1 6.1
  3. lag-5.p2.ixn-lon.zen.net.uk 0.0% 67 5.6 5.9 5.4 7.8 0.4
  4. ldn-b13-link.ip.twelve99.net 45.5% 67 5.6 5.5 4.9 6.1 0.3
  5. ldn-bb2-link.ip.twelve99.net 0.0% 67 6.1 6.3 5.4 8.9 0.6
  6. nyk-bb1-link.ip.twelve99.net 0.0% 67 74.7 75.0 74.1 77.0 0.5
  7. rest-bb1-link.ip.twelve99.net 1.5% 67 81.5 81.5 80.8 83.9 0.5
  8. lax-b22-link.ip.twelve99.net 0.0% 67 137.5 138.4 136.7 175.7 5.4
  9. (waiting for reply)
  10. (waiting for reply)
  11. 15.230.16.155 0.0% 67 141.6 143.9 140.3 167.9 5.4
  12. (waiting for reply)
  13. 52.93.14.118 0.0% 67 216.6 178.8 165.8 267.3 20.9
  14. 52.93.14.105 0.0% 67 171.5 171.4 170.2 175.7 1.0
  15. 52.93.15.28 0.0% 67 169.2 179.0 169.0 238.9 15.5
  16. 52.93.246.191 0.0% 67 166.1 167.8 165.3 194.0 4.4
  17. 52.93.246.81 0.0% 67 172.3 172.4 171.3 179.7 1.4
  18. (waiting for reply)
  19. (waiting for reply)
  20. (waiting for reply)
  21. (waiting for reply)
  22. (waiting for reply)
  23. (waiting for reply)
  24. 216.147.3.209 0.0% 67 166.4 167.1 166.0 169.1 0.6