Apology: [Re: Tier-2 reachability and multihoming]

G_Pavan_Kumar · March 26, 2005, 6:21am

This is with my deepest regrets that I apologize from the bottom
of my heart to Mr.Gilmore, Mr.Woodcock, Mr.Bush and also the rest
of the honourable members of the list for being ignorant of how high-profile a list this is. I couldn't be more sorry. Please,
please forgive me.

ps: I sure meant no harm, was just trying to be humorous,(I hope
the exclamation marks might have given some hint) anyway it is
too late. They say there is no natural punishment than remorse.
Also, I was too embarrassed to post a quick apology.

Thanking you,
pavan.

Michael_Loftis · March 26, 2005, 7:01am

This is with my deepest regrets that I apologize from the bottom
of my heart to Mr.Gilmore, Mr.Woodcock, Mr.Bush and also the rest
of the honourable members of the list for being ignorant of how
high-profile a list this is. I couldn't be more sorry. Please,
please forgive me.

ps: I sure meant no harm, was just trying to be humorous,(I hope
the exclamation marks might have given some hint) anyway it is
too late. They say there is no natural punishment than remorse.
Also, I was too embarrassed to post a quick apology.

No exclamation points indicate yelling, animated, surliness, or a host of other emotions, humor is NOT one of them. If you intent was to joke or in jest then don't use !. use or esp. since your second language is pretty clearly english where what you're typing, and what we're reading/getting can be hard to interpret.

Stephen_J_Wilcox1 · March 26, 2005, 9:57pm

So anyway, this internet thing..

forget this concept of tier1, 2, 3 .. they are little more than terms used by
salesmen. instead assume all ISPs have connectivity to the whole internet, and
that you're a new ISP wanting connectivity of your own. you can buy transit from
any ISP and you will get global reachability, you could also buy from any two
hence multihoming and have the same global reachability

now you're up and running, consider peering.. peering with another isp will give
you access to that isp and their customers (ie other isps buying global
reachability as you are doing)

so as per your original query, if any two nodes/asns dont have a direct
connection you can assume one or both is relying on their upstream to provide
the necessary global connectivity

now, i see your data is from oregon.. i think theres around 50 'views' of the
internet from about 25 ASNs. consider there are about 20000 active ASNs
currently. you would need to get all 20000 routing tables in order to see
exactly what relationships are active.

(the reason is that from any single ASN the internet will appear to you as a
tree much like your original email, showing the 'up-down' relationships but not
the 'left-right' ones)

also, in the context that you use 'multihoming' you're really referring to a
leaf node such as an enterprise which may buy from 2 or 3 isps to have global
connectivity with some redundancy. if you are looking at transit ISPs (ie tier1,
2 in your description) their connectivity is more complicated and you need to
continue your reading with some of the suggested papers..

Steve

Bandy_Rush1 · March 27, 2005, 4:21am

forget this concept of tier1, 2, 3 .. they are little more than terms used
by salesmen.

at least t1 and t2, also permeate academic papers where the real
topology is actually measured. but we should not let demonstrable
measurements get in the way of our defense of the position of our
smaller networks by marketing people.

randy

ianai · March 27, 2005, 4:29pm

And how, pray tell, does one actually "measure" T1 vs. T2 networks? (Assuming you are not talking about two of the Terminator movies.

If someone is paying Network A, but sends communities to be treated as a peer, are they T1 or T2?

If someone buys from Network B, but peers with all of Network B's peers, and therefore does not appear in a path through Network B in those peers' BGP tables (except at the actual peering router), are they T1 or T2?

If someone "peers" with Network C, but is out-of-balance and pays a settlement fee every month, are they T1 or T2?

Assume someone else is out-of-balance with Network C, but in the other direction, does that make Network C a T2? Even if the network in question still pays Network C?

Etc., etc., etc.

There might be a way to define "Tier 1" and "Tier 2" sufficiently well as to disambiguate all the variations, but I do not think you could do it without seeing (NDA'ed) contracts and/or actual router configurations - neither of which are likely without the help of the network in question. And if you have their help, you can just ask.

Back on a more operational topic, it really doesn't matter what "tier" you are, it just matters how good your connectivity is. There is no need to 'defend' the 'smaller networks'. Some of the "tier 1" networks have totally suck ass connectivity. (Yes, 'suck ass' is a technical term. =)

Stephen_J_Wilcox1 · March 27, 2005, 8:45pm

>> forget this concept of tier1, 2, 3 .. they are little more than terms used
>> by salesmen.
>
> at least t1 and t2, also permeate academic papers where the real topology is
> actually measured. but we should not let demonstrable measurements get in
> the way of our defense of the position of our smaller networks by marketing
> people.

And how, pray tell, does one actually "measure" T1 vs. T2 networks?
(Assuming you are not talking about two of the Terminator movies.

i would agree it is possible to mark some networks as transit free - tier1 - and
therefore any network using a tier1 to access another tier1 is tier2. arguably a
tier3 would be a network not connected to a tier1.

If someone is paying Network A, but sends communities to be treated as
a peer, are they T1 or T2?

imho: T1, forget the money

Back on a more operational topic, it really doesn't matter what "tier" you
are, it just matters how good your connectivity is. There is no need to
'defend' the 'smaller networks'. Some of the "tier 1" networks have totally
suck ass connectivity. (Yes, 'suck ass' is a technical term. =)

absolutely!! it amazes me how much value is placed in this 'tier' system, why
not just buy connectivity that (a) is compatible with your size as an ISP (b)
reliably delivers bits from A to B

Steve

Bandy_Rush1 · March 27, 2005, 8:59pm

here is what i answered a private message on the subject, with a
typo corrected. [un]fortunately, i seem not to have saved the
follow-on mess age where i suggested how one could get a good first
cut at this from route-views data.

randy

Alex_Bligh1 · March 28, 2005, 10:54am

Even this is debatable (& I know you know this Randy).

Firstly, peering isn't binary. Is peering vs transit a distinction based on
routes taken / accepted & readvertised, or on cost? Does "paid for peering"
count as peering or transit? If you pay by volume? If you pay for "more
than your fair share" of the interconnect pipes? (if the latter, I am
guessing there are actually no Tier 1s as everyone reckons they pay for
more than their fair share...).

Secondly, it doesn't cover scenarios that have have happened in the past.
For instance, the route swap. EG Imagine networks X1, X2, X3, X4 are "Tier
1" as Randy describes them. Network Y peers with all the above except X1.
Network Z peers with all the above except X2. Y & Z peer. To avoid Y or Z
needing to take transit, Y sends Z X2's routes (and sends Z's routes to X2
routes marked "no export" to X2's peers), and Z sends Y X1's routes (and
sends Y's routes to X1 marked "no export" to X1's peers). Perhaps they do
this for free. Perhaps they charge eachother for it and settle up at the
end of each month. Perhaps it's one company that's just bought another.

All this come down to the fact that "Tier n" is not a useful taxonomy
because there is no clear ordering of networks.

If I was really pushed for a definition, I'd say it was this: you are a
Tier-1 network, when, if you tell all third parties not to advertise your
routes to anyone but their customers, and you get a phone call from one of
your customers complaining about a resultant connectivity problem, you can
be confident before you've analyzed it, that the customer will accept
it's that networks problem, not yours. This boils down to "does the
customer believe you".

Alex

Bandy_Rush1 · March 28, 2005, 3:01pm

a tier-1 network does not get transit prefixes from any other
network and peers with, among others, other tier-1 networks.

a tier-2 gets transit of some form from another network, usually but
not necessarily a tier-1, and may peer with other networks.

this does not please everyone, especially folk who buy transit and
don't like discussing it. and there are kinky corners

Even this is debatable (& I know you know this Randy).

in this forum, everything is debatable. some portion of the debate
makes sense. ymmv.

Firstly, peering isn't binary. Is peering vs transit a distinction based on
routes taken / accepted & readvertised, or on cost? Does "paid for peering"
count as peering or transit? If you pay by volume? If you pay for "more
than your fair share" of the interconnect pipes? (if the latter, I am
guessing there are actually no Tier 1s as everyone reckons they pay for
more than their fair share...).

pay? did i say pay? i discussed announcement and receipt of
prefixes. this was not an accident. it is measurable.

Secondly, it doesn't cover scenarios that have have happened in the past.
For instance, the route swap. EG Imagine networks X1, X2, X3, X4 are "Tier
1" as Randy describes them. Network Y peers with all the above except X1.
Network Z peers with all the above except X2. Y & Z peer. To avoid Y or Z
needing to take transit, Y sends Z X2's routes (and sends Z's routes to X2
routes marked "no export" to X2's peers), and Z sends Y X1's routes (and
sends Y's routes to X1 marked "no export" to X1's peers). Perhaps they do
this for free. Perhaps they charge eachother for it and settle up at the
end of each month. Perhaps it's one company that's just bought
another.

seems to me that, if you look at the prefixes, it's pretty clear.

randy

Stephen_J_Wilcox1 · March 28, 2005, 3:17pm

> Firstly, peering isn't binary. Is peering vs transit a distinction based on
> routes taken / accepted & readvertised, or on cost? Does "paid for peering"
> count as peering or transit? If you pay by volume? If you pay for "more than
> your fair share" of the interconnect pipes? (if the latter, I am guessing
> there are actually no Tier 1s as everyone reckons they pay for more than
> their fair share...).

pay? did i say pay? i discussed announcement and receipt of prefixes. this
was not an accident. it is measurable.

i also avoided money.. i dont think its that relevant, everyone is paying for
peering or transit in one form or another, i dont think any peering is free
(free != settlement free)

> Secondly, it doesn't cover scenarios that have have happened in the past.
> For instance, the route swap. EG Imagine networks X1, X2, X3, X4 are "Tier
> 1" as Randy describes them. Network Y peers with all the above except X1.
> Network Z peers with all the above except X2. Y & Z peer. To avoid Y or Z
> needing to take transit, Y sends Z X2's routes (and sends Z's routes to X2
> routes marked "no export" to X2's peers), and Z sends Y X1's routes (and
> sends Y's routes to X1 marked "no export" to X1's peers). Perhaps they do
> this for free. Perhaps they charge eachother for it and settle up at the end
> of each month. Perhaps it's one company that's just bought another.

"transit (n). The act of passing over, across, or through; passage."

whether it is a settlement arrangement or a mutual swap, they do NOT have
peering, they ARE transitting and by our definition are not transit-free (and
hence not tier1)

however alex, you do highlight an excellent point - things are not as simple as
'tier1, tier2', there are complicated routing and financial arrangements in
operation, which brings me back to my earlier point: does it matter what a
network is paying for some connectivity providing they deliver to you the
connectivity you need at the quality you desire?

Steve

JD1 · March 28, 2005, 9:58pm

I’ll be brief, but I do want to perhaps word Alex’s definition in a different way that might be more useful.

Even “tier 1” providers regularly trade transit. They must since no single network is connected to all the other ones. Not even close. Even UUNet (ASN 701), arguably the most-connected network on the planet, only connects to a fraction of the possible peerings.

The true definition is more vague: if a peering or transit circuit between A or B is taken down, who will be hurt the most: A or B? If it predominantly B, and much less A, then A is “more Tier 1” and B is of a “lesser Tier”. If they are equally hurt, they the are of equal status. Essentially, “Tier 1” is whatever the other “Tier 1” providers believe at the moment is “Tier 1”. It is self-referential and not distinct at all.

This is, frustratingly, a very non-technical definition. But it seems to map with what I’ve actually seen the industry do.

John

Stephen_J_Wilcox1 · March 29, 2005, 1:23am

I'll be brief, but I do want to perhaps word Alex's definition in a different way
that might be more useful.

Even "tier 1" providers regularly trade transit. They must since no single
network is connected to all the other ones. Not even close. Even UUNet (ASN
701), arguably the most-connected network on the planet, only connects to a
fraction of the possible peerings.

701 is not the most connected, it has only customers and a restrictive set of
peers?

you dont need to peer with all networks tho, if all networks are buying from 701
or one of its peers then it will get those routes via peering not transit or
transit trades... you seem to be forgetting what peering is.

and if you peer with all networks in the 'transit free zone' then you too become
transit free also.

The true definition is more vague: if a peering or transit circuit between A or B
is taken down, who will be hurt the most: A or B? If it predominantly B, and much
less A, then A is "more Tier 1" and B is of a "lesser Tier". If they are equally
hurt, they the are of equal status. Essentially, "Tier 1" is whatever the other
"Tier 1" providers believe at the moment is "Tier 1". It is self-referential and
not distinct at all.

i believe the distinction exists as shown above ie transit free.. as to why this
might be considered a goal i'm not sure, its not obvious that transit free is
cheaper than buying transit!

this thing about 'who hurts most' is an entirely different topic and has nothing
to do with who is in the transit free zone. altho destructive depeering does
seem to be common practice within that zone

This is, frustratingly, a very non-technical definition. But it seems to map
with what I've actually seen the industry do.

thats because non-technical definitions mean anyone can call themselves anything
they like.. wiltel recently spammed me to buy their 'tier1 transit'.. presumably
they are tier1 within their own definition of tier1.

if you want to be technical tho, and aiui we are a technical forum, then tier1
means transit free.

i reaffirm my earlier point - but why care, isnt it about cost and reliability,
and as peering and transit are about the same cost who cares who you dont peer
with

Steve

bill3 · March 29, 2005, 1:29am

and if you peer with all networks in the 'transit free zone' then you too become
transit free also.

  er.. hate to rain on your parade but if I peer with everyone
  i need/want to exchange traffic with, i am transit-free, even
  if I -NEVER- touch any other part of the commercial Internet...
  my packets get to where they need to go and all packets I want
  get to me. my life is good ... even if I only appear as vestigal
  to the commercial Internet, if I appear at all.

how would you classify such a network? T1, T2, ODDBALL-0,
non-Internet-265, ???

--bill

ianai · March 29, 2005, 2:15am

and if you peer with all networks in the 'transit free zone' then you too become
transit free also.

  er.. hate to rain on your parade but if I peer with everyone
  i need/want to exchange traffic with, i am transit-free, even
  if I -NEVER- touch any other part of the commercial Internet...
  my packets get to where they need to go and all packets I want
  get to me. my life is good ... even if I only appear as vestigal
  to the commercial Internet, if I appear at all.

Absolutely correct.

how would you classify such a network? T1, T2, ODDBALL-0,
non-Internet-265, ???

I doubt it is a tier. I am certain it is not an "Internet" network if it does not have connectivity to substantially all other Internet networks.

bill3 · March 29, 2005, 2:46am

>>and if you peer with all networks in the 'transit free zone' then you
>>too become
>>transit free also.
>
> er.. hate to rain on your parade but if I peer with everyone
> i need/want to exchange traffic with, i am transit-free, even
> if I -NEVER- touch any other part of the commercial Internet...
> my packets get to where they need to go and all packets I want
> get to me. my life is good ... even if I only appear as vestigal
> to the commercial Internet, if I appear at all.

Absolutely correct.

> how would you classify such a network? T1, T2, ODDBALL-0,
> non-Internet-265, ???

I doubt it is a tier. I am certain it is not an "Internet" network if
it does not have connectivity to substantially all other Internet
networks.

  begs the definition of "internet networks" ...
  It has IP connectivity to the other IP networks of interest.
  For networks that are not of interest, there is no expressly
  defined connectivity. The term Internet has devolved into
  a series of interconnected -COMMERCIAL- networks and from that
  viewpoint, anyone on a non-commercial network, that has no desire
  to be connected to a commercial network, is relegated, BY THE
  COMMERCIAL OPERATORS, to "intranet" status. The historical
  term - INTERNET - reflected a catanet of networks that used IP
  for packet delivery. with the inclusion of robust policy expression
  on network "edges" - full, global, end2end reachability truely
  became a myth ... and the term Internet became based on a
  shifting foundation. So from a commercial networking perspective,
  yes, my network is vestigal. But it is transit-free and has
  full connectivity to all of the parties it wants/needs to talk to.
  So by that definition (e.g. transit-free) its a Tier-1.
  Sort of points out some of the weaknesses in terminology and
  the biases in a single viewpoint.

as usual, YMMV.

--bill

Bandy_Rush1 · March 29, 2005, 2:47am

  er.. hate to rain on your parade but if I peer with everyone
  i need/want to exchange traffic with, i am transit-free, even
  if I -NEVER- touch any other part of the commercial Internet...
  my packets get to where they need to go and all packets I want
  get to me. my life is good ... even if I only appear as vestigal
  to the commercial Internet, if I appear at all.

  how would you classify such a network?

billnet. we're used to it.

randy

bill3 · March 29, 2005, 2:50am

and you have even used it on occasion.

--bill

Richard_A_Steenbegen · March 29, 2005, 6:24am

Ok, I'm just bored enough to bite. If we're talking about a contest to see
who has the most number of directly connected ASNs, I think UU might still
win, even with a restrictive set of peers.

Taking a look at a count of customer ASNs behind some specific networks of
note, I come up with the following (some data a couple weeks out of date,
but the gist is the same):

Network ASN Count
------- ---------
701 2298
7018 1889
1239 1700
3356 1184
209 1086
174 736
3549 584
3561 566
2914 532
2828 427
6461 301
1299 243

Which begs the question, what is the largest number of ASNs that someone
peers with? Patrick? Somehow I suspect that 701's customer base (702
and 703 aren't included in the above count BTW) overpower even the most
aggressively open of peering policies, in this particular random pointless
and arbitrary contest at any rate.

michael.dillon2 · March 29, 2005, 11:20am

And how, pray tell, does one actually "measure" T1 vs. T2 networks?

That's easy. You define a set of criteria by which you can measure
the networks on some scale, and then set two thresholds. Networks
which exceed the higher threshold are Tier 1, those which only
exceed the lower threshold are Tier 2.

I have seen people do this by counting the number of ASes that
a network connects to. And I have seen this done with nodes by
summing up the bandwidth of all circuits connected to a node.

Even though the network is a dynamic partial mesh, researchers
can learn a lot about the behavior by imposing various types
measurement hierachy on the network.

Thus, Tier 1 and Tier 2 are not inherent characteristics of
the Internet; rather they are characteristics of a particular
view of the network at a particular point in time. There are
probably people who are trying to measure a hierarchy of latency
or a hierarchy of jitter. The more views, the merrier.

--Michael Dillon

ianai · March 29, 2005, 3:46pm

Of course. There is a difference between "most peers" and "most adjacent ASes".

But it is non-trivial to see which of those adjacencies are transit and which are peering. (Nearly impossible if you define such things on Layer 8, but not impossible if you only include which ASes are propagated to which other ASes.)

At the end of the day, an AS with a LOT of downstream ASes can always beat a well peered AS - there just aren't that many ASes which peer.