Comcast Enterprise Fiber Slow Connection Problem from TW Telecom

Mailman List Mirror (Read Only)
1

We have been having a problem receiving software releases from our developer. The releases are typically around 1G in size. The developer’s connection is a 100m metro fiber with TW Telecom, our connection is a 25m Comcast Enterprise Fiber.

Our traffic graphs show very little utilization of our connection. Typically on average we are at about 7 meg utilization of our 25.

Every other partner that shares in our software development that receives the software releases can receive the updates 3-4 times faster than we can.

Typically we receive the releases at about 3mbps.

I have tried contacting Comcast Enterprise Tech support, however I’ve been told that if I run a speed test from my connection and the test runs at the speed we are paying for, there is very little they are willing to look into.

Can anyone check on the Comcast Routers on the Tracert below, or is there anything that can be throttling this connection between the two connections?

Also, our firewall and connection is able to run at the full 25. We have no throttling or QOS set to prevent a good connection to our developer. For example, we can run a multi-threaded upload, in the middle of the night, to Amazon Glacier storage and completely saturate our connection when doing so. The firewall and connection is able to handle our full bandwidth capacity during that backup.

If there is any other information I can provide to help track this problem down, please let me know.

Thanks in advance, everyone!

Trace Route below:

1 (172.16.150.1) 1.143 ms 1.132 ms 1.122 ms

2 (173.227.204.1) 1.585 ms 1.583 ms 1.574 ms

3 chi2-pr1-xe-0-3-0-0.us.twtelecom.net (66.192.245.166) 10.477 ms 10.485 ms 10.478 ms

4 x-eth-0-0-4-pe05.350ecermak.il.ibone.comcast.net (75.149.230.141) 10.470 ms 10.465 ms 10.457 ms

5 he-2-1-0-0-cr01.350ecermak.il.ibone.comcast.net (68.86.86.37) 10.733 ms 10.731 ms he-2-0-0-0-cr01.350ecermak.il.ibone.comcast.net (68.86.86.33) 12.146 ms

6 be-10206-cr01.newyork.ny.ibone.comcast.net (68.86.86.225) 33.202 ms 32.144 ms 32.127 ms

7 68.86.91.30 (68.86.91.30) 41.508 ms 41.322 ms 41.599 ms

8 te-0-0-0-1-sur01.greensburg.pa.pitt.comcast.net (69.139.168.26) 38.196 ms te-0-0-0-3-sur01.greensburg.pa.pitt.comcast.net (162.151.21.82) 44.644 ms te-0-0-0-0-sur01.greensburg.pa.pitt.comcast.net (69.139.195.18) 38.266 ms

9 (107.1.72.98) 39.781 ms 39.785 ms 39.912 ms

2

We have been having a problem receiving software releases from our
developer. The releases are typically around 1G in size. The developer’s
connection is a 100m metro fiber with TW Telecom, our connection is a 25m
Comcast Enterprise Fiber.

Our traffic graphs show very little utilization of our connection.
Typically on average we are at about 7 meg utilization of our 25.

Every other partner that shares in our software development that receives
the software releases can receive the updates 3-4 times faster than we can.

Typically we receive the releases at about 3mbps.

Are you using an application that uses TCP transport for the transfer?

https://www.switch.ch/network/tools/tcp_throughput/index.html?mss=1460&rtt=38&loss=1e-06&Calculate=Calculate&bw=100&rtt2=80&win=64

3Mbps looks about right. Time for a tune up

3

With a max bandwidth of 25 Mbps and a 40ms RTT, the max is more like 14MB/s
or 1.75 Mbps.

https://www.switch.ch/network/tools/tcp_throughput/index.html?mss=1460&rtt=80&loss=1e-06&bw=25&rtt2=35&win=64&Calculate=Calculate

But that's only if either endpoint is stuck at a 64 KB receive window. A
quick packet capture would be able to see what was happening. Check the
TCP setup and make sure that both ends are doing TCP window scaling
properly.

John

4

I apologize I should have said it starts out about 3 meg max and slows to about 400kpbs for most of the transfer.

5

Sounds like a combination of packet loss and small TCP receive windows. If
you can, grab a packet capture and make sure to get the TCP setup. That
should show you what's happening under the hood.

Also, I should mention that I totally hosed the units in my first reply.
:slight_smile: That's what I get for hurrying.

John

6

Recommendations:

  1) Use iperf in TCP mode to test the performance
  2) use iperf in UDP mode to test the performance

  This is the best way to quickly triage the issue and determine if
it's actual bandwidth issue or something else.

  It's quite common for various NAT/VPN devices to eat or meddle with
packets such that performance is limited, sometimes severely. You need
to check the CPU/performance of these devices.

  Do you have graphs showing the links as idle for each end while
doing the test?

  Many people don't have this data or tools. Consider setting up
a tool like observium to monitor the health and performance of the
devices.

  Check other devices, such as switches, duplex settings, etc. If
the device is 'unmanaged' and you can't tell if it negotiated 100-full
or 100-half, consider it may have done half-duplex. I recall one customer case
where after much troubleshooting for performance they had a duplex issue
causing trouble.

  Hope this helps,

  - Jared