i sit behind cox-cable service at home, and in troubleshooting why my connectivity is *so* horrible, i find the following traceroute. does anyone do any sane routing anymore? does diameter matter (we used to talk about it a long, long while ago). i guess i'm just old and crusty but this seems to violate so many natural laws.
i find in more random testing that i seem to be a minimum of 15 hops from anything, and it's not just the # of hops, it's the *paths* i travel. bouncing between two cities several times, on several different provider networks, from one border to the other.
wow.
-b
traceroute www.caida.org
1 10.113.128.1 30 unavailable
2 68.2.6.25 10 ip68-2-6-25.ph.ph.cox.net
3 68.2.0.26 40 ip68-2-0-26.ph.ph.cox.net
4 68.2.0.18 50 ip68-2-0-18.ph.ph.cox.net
5 68.2.0.10 20 ip68-2-0-10.ph.ph.cox.net
6 68.2.0.70 10 ip68-2-0-70.ph.ph.cox.net
7 68.2.14.13 10 chnddsrc02-gew0303.rd.ph.cox.net
8 68.1.0.168 20 chndbbrc02-pos0101.rd.ph.cox.net
9 68.1.0.146 30 dllsbbrc01-pos0102.rd.dl.cox.net
10 12.119.145.125 40 unavailable
11 12.123.17.54 30 gbr6-p30.dlstx.ip.att.net
12 12.122.5.86 51 gbr4-p90.dlstx.ip.att.net
13 12.122.2.114 80 gbr2-p30.kszmo.ip.att.net
14 12.122.1.93 50 gbr1-p60.kszmo.ip.att.net
15 12.122.2.42 70 gbr4-p40.sl9mo.ip.att.net
16 12.122.2.205 60 gbr3-p40.cgcil.ip.att.net
17 12.123.5.145 60 ggr1-p360.cgcil.ip.att.net
18 207.88.50.253 90 unavailable
19 64.220.0.189 80 ge5-3-1.RAR1.Chicago-IL.us.xo.net
20 65.106.1.86 70 p0-0-0-0.RAR2.Chicago-IL.us.xo.net
21 65.106.0.34 60 p1-0-0.RAR1.Dallas-TX.us.xo.net
22 65.106.0.14 120 p6-0-0.RAR2.LA-CA.us.xo.net
23 64.220.0.99 80 ge1-0.dist1.lax-ca.us.xo.net
24 206.111.14.238 211 a2-0d2.dist1.sdg-ca.us.xo.net
25 209.31.222.150 80 unavailable
26 198.17.46.56 140 pinot.sdsc.edu
27 192.172.226.123 91 cider.caida.org
brett watson observed :
i sit behind cox-cable service at home, and in troubleshooting why my
connectivity is *so* horrible, i find the following traceroute. does
anyone do any sane routing anymore? does diameter matter (we
used to talk
about it a long, long while ago). i guess i'm just old and
crusty but this
seems to violate so many natural laws.
i find in more random testing that i seem to be a minimum of
15 hops from
anything, and it's not just the # of hops, it's the *paths* i travel.
bouncing between two cities several times, on several
different provider
networks, from one border to the other.
wow.
-b
traceroute www.caida.org
Mine is equally bad... (also on Cox)
going from Oklahoma, to Dallas, to KS, to Chicago, BACK to Dallas, and
finally out to the left coast...
This leads me to believe that att/cox may not be as fully meshed with other
providers as they could be. (see various flames on peering for any number of
reasons why) One could also posit that you are seeing the results of
someone's idea of traffic engineering.
I can't imagine cable modem users creating a large demand for bandwidth to
caida.org anytime in the near future, so cox/att et. al. are not going to
fix something that isn't 'broken'. The altruistic days of 'running the
network together' couldn't be more dead and buried, IMHO.
The only thing I see you can really complain about is the depth of cox's
network, and to a lesser extent ATT's and XO's before it gets out to "the
internet". As for inefficient routing...I can beat that. My current
employer is just a few miles down the road from my previous employer. A
traceroute from us to them isn't nearly as many hops as your traceroute, but
goes all the way from FL to TX and back to FL.
In your case, it would shave a bunch of hops if ATT and XO peered in
Dallas, but it's just not cost effective for everyone to peer everywhere.
For a while, we were both connected to a local exchange point of sorts.[1]
Almost nobody else came, but when we outgrew our connection and wanted to
upgrade, the people running the NAP wanted to bend us over for a larger
connection, so we left. When the pointy-hairs make peering cost
prohibitive, even if the network admins think it'd be a great idea, it
doesn't happen. AFAIK, they have no more peering customers.
1. gsvlfl-br-1-e0-53.atlantic.net 0% 3 3 1 0 0 1
2. gsvlflma-br-1-s0-0.atlantic.net 0% 3 3 0 0 0 0
3. orldflma-br-1-s2-0.atlantic.net 0% 3 3 5 4 86 249
4. orldflwcom-br-1-s1-1-1.atlantic.net 0% 3 3 4 4 81 234
5. sl-gw8-orl-3-0-TS11.sprintlink.net 0% 3 3 5 5 5 6
6. sl-bb21-orl-0-0.sprintlink.net 0% 3 3 6 5 5 6
7. sl-bb23-fw-9-3.sprintlink.net 0% 3 3 72 34 47 72
8. sl-bb21-fw-13-0.sprintlink.net 0% 3 3 33 32 33 33
9. 144.232.19.42 0% 2 2 36 35 36 36
10. pos3-0-622M.cr2.DAL1.gblx.net 0% 2 2 35 34 35 35
11. pos0-0-622M.cr1.HOU1.gblx.net 0% 2 2 39 39 39 40
12. pos0-0-155M.ar1.TPA1.gblx.net 0% 2 2 64 64 65 67
13. AgilityBroadband.s4-1-1-4-0.ar1.TPA 0% 2 2 150 80 115 150
14. yoda.fdt.net 0% 2 2 134 83 108 134
[1] The local exchange point was actually the local government owned
utility company gone ISP. They sold transit to a few local ISPs and to
the local university. They offered peering connections as a sort of
NAP. AFAIK, we were the only ones that bought one.
Hi Brett,
Are you asking _why_ there are so many hops between yourself and the guy
across town?
Jane
brett watson wrote:
no, just lamenting the passing of an era. an era where we engineers cooperated, and "just fixed" the problems as they occured. and we didn't do things like this.
turn on the sarcasm tone, and re-read my post. this could win the prize on Latenight with David Letterman, Stupid IP Routing Tricks.
-b
Keep in mind the reason why the era passed. During that era, you had
top level, blue sky engineers. Now the field has been saturated by a
lot of less than desirable "engineers" out there (not calling you one at
all) that ruined it for us all...
Date: Tue, 18 Jun 2002 10:34:10 -0700
From: brett watson
no, just lamenting the passing of an era. an era where we
engineers cooperated, and "just fixed" the problems as they
occured. and we didn't do things like this.
turn on the sarcasm tone, and re-read my post. this could
win the prize on Latenight with David Letterman, Stupid IP
Routing Tricks.
Does the first person to create a "knight's tour" traceroute on
their network win a prize?
Eddy
ohmygod.
Not the engineer vs cs argument again. There was a time they were one
and the same beast.
Of course. I'm dating myself.
Jane
Jeff Harper wrote:
Why do you think Cogent has ".atlas." in their DNS? 
Are you asking _why_ there are so many hops between yourself and the guy
across town?
He's not, but answer is that BGP's key metric is AS path length. This can have very little to do with the optimality (expressed as efficient use of resources) of the actual packet path.
Cheers,
Mathew
Path is one of the last things to be checked
BGP first checks all kinds of network admin defined things such as local
prefs etc which ought to be properly set by the admins to ensure traffic
is going the best way (which should include local interconnects rather
than last resort transits). Then all things being well BGP can make
choices on path!
Path is one of the last things to be checked
BGP first checks all kinds of network admin defined things such as local
prefs etc which ought to be properly set by the admins to ensure traffic
is going the best way (which should include local interconnects rather
than last resort transits). Then all things being well BGP can make
choices on path!
So, you're advocating that the admin do all of the optimization manually for all destinations by setting preferences?
Mathew
No, I'm suggesting that key interconnections should be preferenced where
they provide better throughput/latency/whatever ..
in the below example its unclear what causes the path to be the way it is
but it doesnt look optimum in terms of ip hops altho it presumably is only
2 or 3 AS hops
i'm saying that AS hops give no indication of the network size and there
is some manual intervention to improve BGPs short sight
Steve
Date: Tue, 18 Jun 2002 19:28:41 +0100 (BST)
From: Stephen J. Wilcox
BGP first checks all kinds of network admin defined things
such as local prefs etc which ought to be properly set by
the admins to ensure traffic is going the best way (which
should include local interconnects rather than last resort
transits). Then all things being well BGP can make choices
on path!
That's what happened here. Rather than transitting the traffic
via a "last resort" across town/state, the higher local-pref of a
"local" peer won.
Geography requirements for peers aren't inherently bad. There's
a point where things get extreme, but it would be nice to see
nationals peer in the south as well. If one has peering
requirements, at least set them to reach a positive goal...
Eddy
man, i threw the can-o-worms out for fun, i didn't expect anyone to actually *open* it.
this all has nothing whatsoever to do with bgp. it has everything to do with politics and revenue. the bgp decision tree is being manipulated by human nature.
now drop the can before someone yells at me for throwing it in the first place.
Date: Tue, 18 Jun 2002 11:34:34 -0700
From: Mathew Lodge
So, you're advocating that the admin do all of the
optimization manually for all destinations by setting
preferences?
If I find "6347 3561 1" works better than "3549 1", I'll tune
local-pref accordingly. The OP had a much more clear-cut case
where "Cox AT&T XO" left much to be desired.
Of course, note that we discuss outbound traffic. The OP might
wish for a good return path, too, being a traffic sink. Without
selective prepends, it's time for some coarse tuning.
Won't start that thread again due to underwhelming interest (at
least when I tried to put together a resource on providers who
offer that). Looks like the simple answer is: few do.
Eddy
BGP twiddling cannot fix a broke-ass network design.
Date: Tue, 18 Jun 2002 19:38:40 +0100 (BST)
From: Stephen J. Wilcox
in the below example its unclear what causes the path to be
the way it is but it doesnt look optimum in terms of ip hops
altho it presumably is only 2 or 3 AS hops
I dub it... eOLPF.
i'm saying that AS hops give no indication of the network
size and there is some manual intervention to improve BGPs
short sight
BGP might be "good enough" if there were enough peering points.
But peering is a business decision, and perhaps vulnerable to an
"inverse tragedy of the commons" approach. How little can you
get away with before customers leave? Why peer when you
hopefully can force a sale here or there?
Wars of attrition can be interesting.
Eddy
Er... back then it took 2 months to learn everything a backbone engineer
had to know. Nowadays it's an alphabet soup of stupid techniques to
achieve the same result - i.e. to deliver a packet from place A to place
B. Blame greeeeedy vendors (OFRV, particularly, and don't forget
hellcore) who sell FUD instead of making their products easy to use.
Given their dominant position on the market, everyone else has to be
"compatible" with the zillion little features just to stay afloat.
Regarding the diameter of the Internet - I'm still trying to figure out
why the hell anyone would want to have "edge" routers (instead of dumb
TDMs) if not for inability of IOS to support large numbers of virtual
interfaces. Same story goes for "clusters" of backbone routers.
--vadim
Er... back then it took 2 months to learn everything a backbone engineer
had to know. Nowadays it's an alphabet soup of stupid techniques to
achieve the same result - i.e. to deliver a packet from place A to place
B. Blame greeeeedy vendors (OFRV, particularly, and don't forget
hellcore) who sell FUD instead of making their products easy to use.
Given their dominant position on the market, everyone else has to be
"compatible" with the zillion little features just to stay afloat.
that's an interesting point of view. i would say that really nothing at all has changed in 10 years. sure, there is a bag-of-stupid-ip-tricks to choose from that didn't exist back then but none of the tricks have solved our problems. the political/financial issues crept in, and the bag-of-stupid-ip-tricks seems to have developed as a way to solve those issues, which they have not solved.
the same level of fundamental knowledge required back then applies today, and many network and systems engineers are *still* lacking that knowledge. i suppose your are right if you're implying that the bag-of-stupid-ip-tricks has obfuscated what's really important.
uucp and modems are looking pretty attractive to me again.
Regarding the diameter of the Internet - I'm still trying to figure out
why the hell anyone would want to have "edge" routers (instead of dumb
TDMs) if not for inability of IOS to support large numbers of virtual
interfaces. Same story goes for "clusters" of backbone routers.
everyone note: vadim threw that can of worms, it wasn't me!
-b