verizon fios, northeast, routing issues?

Mailman List Mirror (Read Only)
1

Any Verizon folks here?

I've been having some rather weird network issues lately - just reading email via IMAP, from home. Over a 1gig FIOS connection to a machine in a nearby Tierpoint data center that has LOTS of good connectivity.

I just tried some traceroutes, and got some interesting results:

These originate on a machine connected to a 1gig FIOS feed, and end at a machine, located in a Tierpoint datacenter, about 10 miles from here.

traceroute to ntcorp.com (207.154.13.58), 64 hops max, 52 byte packets
1 * fios_quantum_gateway (192.168.1.1) 3.530 ms 2.822 ms
2 * * *
3 100.41.27.110 (100.41.27.110) 14.970 ms 5.323 ms 6.306 ms
4 0.csi1.bstnmafr-mse01-bb-su1.alter.net (140.222.10.32) 11.069 ms 8.477 ms 17.097 ms
5 * * *
6 0.ae1.br1.bos30.alter.net (140.222.236.253) 17.121 ms 19.027 ms
0.ae2.br1.bos30.alter.net (140.222.236.255) 19.795 ms
7 * * *
8 colo4-dalla.bear1.boston1.level3.net (4.53.61.86) 2205.648 ms 8.331 ms 13.161 ms
9 static-33-65-203-66.axsne.net (66.203.65.33) 16.951 ms 13.791 ms
static-145-65-203-66.axsne.net (66.203.65.145) 21.503 ms
10 server1.ntcorp.com (207.154.13.58) 17.872 ms 15.902 ms 14.415 ms

Several things jump out:

1. alter.net is not a common path between here & there - usually a lower grade connection, when other backbones aren't working right

2. origin - alter.net - level.3 - endpoint is just bizarre, one would think that the regional FIOS network has a direct connection to level.3 (it also seems kind of odd that the packets are flowing from Acton MA, to Boston, and back out to Marlboro MA - there's an awful lot of fiber running along Rt. 495, and the networks are fairly dense around here)

3. The intermittent, high delays (factor of 10) jump out (also, when running ping tests, there seem to be intermittent periods of long sequences of timeouts)

All in all it's really mucking with both streaming services, and simply posting emails (SMTP timeouts).

All of which leads me to wonder if there's something mucked up with Verizon's routing tables (or a particular network interface).

Any insights (or fixes) to be had?

Thanks,

Miles Fidelman

2

2. origin - alter.net - level.3 - endpoint is just bizarre, one would
think that the regional FIOS network has a direct connection to level.3

No. Former verizon-gni backbone (where FiOS sits) takes transit solely from VZB (now UUNET),
this has been the case for many many years, but most people get confused when interpretting
traceroutes due to mpls no-ttl-propagate on VZ networks.

(it also seems kind of odd that the packets are flowing from Acton MA,
to Boston, and back out to Marlboro MA - there's an awful lot of fiber
running along Rt. 495, and the networks are fairly dense around here)

Level3-VZB-px is at 300 Bent Street, Cambridge, MA, where both parties have peering routers installed
(bear1.Boston1.Level3.net and BR1.BOS30.ALTER.NET). It makes perfect sense for interconnection to occur
in Cambridge/Boston, rather than maintaining peering routers out in the suburb within the same metro.

None of these are contributory factors to the issues you're describing.

James

3

alter.net is just the legacy RDNS for things in AS701 (uunet). Nothing weird there.

https://en.wikipedia.org/wiki/UUNET

4

Ok folks,

Thanks for the info about uunet. But that doesn't address:

3. The intermittent, high delays (factor of 10) jump out (also, when running ping tests, there seem to be intermittent periods of long sequences of timeouts)

or, that, for about 4 years now, gamers seem to be reporting really poor performance across FIOS in the Northeast - tied to rather high packet loss rates.

Note that those packet losses seems to be bursts of 8-10 lost packets every 10 packets or so, and the really high delays on some of the traceroutes seem to indicate that it's happening somewhere in the middle of the path, not at my end.

And, come to think of it, that might explain some of the horrid performance of the FIOS channel guide.

Any thoughts? Anyone here from Verizon Northeast FIOS operations who might have a comment?

Thanks,

Miles Fidelman

Miles Fidelman wrote:

5

FYI:

Funny thing, this morning, the packet loss rate has dropped to between 0 and 10% (from closer to 40%). Route seems to the same.

One wonders if some piece of equipment has been updated.

Miles Fidelman

Miles Fidelman wrote: