Faster 'Net growth rate raises fears about routers

Mailman List Mirror (Read Only)
1

First off, let me just say that I'm not speaking for my employer on
this, okay? Thanks.

The people who prevent the current global routing table from being
flooded by /25-/30 announcements are also the people who punch holes
in their address space for /24s. Abha's numbers at the ptomaine BOF
clearly show the effect of RIR policies (spikes around /20 and /19),
but the bigger effect from my perspective was the spike around /24,
created (I presume) by the punches in CIDR blocks that providers make
to allow multi-homing. I haven't seen good numbers for the
distribution of punches in a long time, but my limited experience
indicates that those punches are being made fairly randomly within the
provider's allocated address space. This means that the bit
boundaries don't align and you increasingly have mini-swamps inside
providers' /19s and /20s.

Why are providers doing this? Someone is paying them to do it.

Why are customers spending money on this? My belief is that they
want more say in their own fate. That may express itself as
a desire for redundancy in the case of catastrophic business failures,
better ability to express their own routing policies, or a simple
worry that they won't get the best price if they have only one supplier.
At the core of this, though, is a desire for more control over something
that they see as increasingly important to their own fate.

I think there are various short term work-arounds to the current
explosion of paths in the routing tables, and I encourage folks to
join the ptomaine mailing list (ptomaine-request@shrubbery.net) if
they want to contribute to the solution. But don't try to accomplish
it by reducing the ability of the customer to control their own fate.
There are real economic pressures out there which will prevent that
class of solution from success.

      regards,
        Ted Hardie

2

Why is pretty simple, in my (admittedly limited) experience with
customers. Count the single points of failure on the way from a
customer T1 to the ISP POP. Customer premesis equipment, especially if
the customer doesn't buy something w/dual power supplies, redundant
control processors, etc. Copper haul within the building. T1 local loop.
Telco network. T1 or hubbed-DS3 card in the provider's customer
edge router. Provider's customer edge router (assuming it doesn't have
fully redundant components.) Add to that telco techs stealing pairs, and
all the other fun events I'm sure we've all seen, and life at the end of a
single circuit can get pretty sketchy. Top it off with a MTR well over 4
hours, especially when the blame game starts, and it gets nasty.

Note that many of these problems aren't fixed until you have APS SONET,
assuming someone engineered the protect path diversely.

Now add a business that can't afford downtime, and multihoming becomes
simple. How many SPs out there are offering customer circuits into
multiple edge boxes for fault tolerance? Is this adequate, or does the
availability requirement call for multiple POPs? Is this adequate, or is
it necessary to go for multiple service providers?

I think the first problem is that conventional wisdom tells the customer
that they have to buy a circuit to two different SPs in order to get
real fault tolerance. I haven't seen a whole lot of aggressive marketing
about pulling two circuits into two edge boxes, using two different pieces
of CPE or one fault-tolerant one. The industry isn't pushing the idea
that you can have redundant service from a single provider. (grain of
salt: one of our providers sold us a backup transit DS3 for the cost of
the local loop)

I'm at a multi-POP network in Boston. We've had great luck selling
customers a Verizon circuit into one of our POPs and a Worldcom circuit
into a different one. It costs more, but they don't have nearly the
exposure of a single circuit customer. However, if you're not set up to
do this, the appropriate level of paranoia calls for circuits to two
different providers. Maybe if SPs really addressed availability
requirements of their customers, it wouldn't be such an issue.

-travis

3

The people who prevent the current global routing table from being
flooded by /25-/30 announcements are also the people who punch holes
in their address space for /24s. Abha's numbers at the ptomaine BOF
clearly show the effect of RIR policies (spikes around /20 and /19),
but the bigger effect from my perspective was the spike around /24,
created (I presume) by the punches in CIDR blocks that providers make
to allow multi-homing. I haven't seen good numbers for the
distribution of punches in a long time, but my limited experience
indicates that those punches are being made fairly randomly within the
provider's allocated address space. This means that the bit
boundaries don't align and you increasingly have mini-swamps inside
providers' /19s and /20s.

Why are providers doing this? Someone is paying them to do it.

I don't argue that multihoming is bad. I argue that we're doing it in the
wrong place with negitive consequences.

Why are customers spending money on this? My belief is that they
want more say in their own fate. That may express itself as
a desire for redundancy in the case of catastrophic business failures,
better ability to express their own routing policies, or a simple
worry that they won't get the best price if they have only one supplier.
At the core of this, though, is a desire for more control over something
that they see as increasingly important to their own fate.

I agree. However, you can offer even greater control and all the other
benefits of multihoming without doing it at the IP layer.

Multihoming at the IP later thus breaking aggregation is like dumping
toxic waste, it cost is largely carried by those not in recept of it's
benefits or any form of payment.

If we can avoid it while still providing the necessary level of service,
then we should seriously investigate such opportunities.

I think there are various short term work-arounds to the current
explosion of paths in the routing tables, and I encourage folks to
join the ptomaine mailing list (ptomaine-request@shrubbery.net) if
they want to contribute to the solution.

Short term is nice, but it doesn't matter in the long run. :slight_smile:

But don't try to accomplish
it by reducing the ability of the customer to control their own fate.
There are real economic pressures out there which will prevent that
class of solution from success.

I never suggested that, I suggested investigating alternatives which
increase customer choice, performance, reliability, and Internet
scalability and potential measures to make the minor inital cost of
implimentation more acceptable.

The obvious intrest here is that most network operators would not have
their customers multihoming at the IP level and thus preventing
aggregation and polluting the global routing table is there was another
way to achieve the same benefits.

4

I'm at a multi-POP network in Boston. We've had great luck selling
customers a Verizon circuit into one of our POPs and a Worldcom circuit
into a different one.

Don't know how the world looks like in the US, but here a SDH/Sonet
provider will never guarentee diversity of his/her circuit to that of a
different provider, often the end user can be almost sure that at least
the last few km will be in the same duct, as the local communities
demand that the providers cooperation when digging fiber into the
ground...

It costs more, but they don't have nearly the
exposure of a single circuit customer. However, if you're not set up to
do this, the appropriate level of paranoia calls for circuits to two
different providers. Maybe if SPs really addressed availability
requirements of their customers, it wouldn't be such an issue.

/Jesper

5

Don't know how the world looks like in the US, but here a SDH/Sonet
provider will never guarentee diversity of his/her circuit to that of a different provider,

        Interesting, though they do guarantee that to NATO
circuits. smd, are you able to get diverse local paths over there ?

often the end user can be almost sure that at least
the last few km will be in the same duct, as the local communities
demand that the providers cooperation when digging fiber into the
ground...

        Obviously there is a concern if everyone is in the same
duct, but if one builds with rings like sensible engineers,
then a single cut just means traffic goes the long way round.
Mind, if a backhoe disconnects one's building entirely from
the ring or there are byzantine failures, no form of multi-homing
will really save one.

        At a previous job, we ensured that local transport
came down one road into the front side of the building and
a different local transport came down the back road into the
back side of the building next door, then connected the buildings
via fibre of our own. Made for quite a nice setup actually.

Ran

6

Anyone care to share their list of contacts at the large ISP's (Earthlink,
AOL, etc) to whom I can make a request to get on their SMTP whitelist?

TIA.

7

Hi Jesper. Delivery to a different provider is a little less clear, at
least in Boston, but if you buy a "type 1" circuit directly from the
provider, and your building is set up with two fiber entrances, you
can actually get a real diverse circuit. Verizon even has a diversity
option on their tariff, although there's some doubt as to whether or not
they bother to make sure it is really diverse.

I'd have to go ask the provisioning department about inter-provider
diversity, but I would imagine tortured screams would be the standard
reply.

Of course, the caveat is that you never believe a provider when they tell
you that you have working / protect on a diverse path, since the first
fiber cut invariably points out some problem with the circuit engineering.

-travis

8

>Don't know how the world looks like in the US, but here a SDH/Sonet
>provider will never guarentee diversity of his/her circuit to that
>of a different provider,

        Interesting, though they do guarantee that to NATO
circuits.

The company I work for, can and do provide diverse circuits, but they
won't guarentee diversity between that and one of a different provider,
the reasoning behind this is, that one cannot know if/when the other
provider reroute their circuit, so that there is no diversity any more.

smd, are you able to get diverse local paths over there ?

>often the end user can be almost sure that at least
>the last few km will be in the same duct, as the local communities
>demand that the providers cooperation when digging fiber into the
>ground...

        Obviously there is a concern if everyone is in the same
duct, but if one builds with rings like sensible engineers,

SDH/Sonet protection removes quite a bit of the problem yes, but often
it's usefull to get 2 circuits with diverse routing (and without
protection) instead of a single with protection, and the price is
usually in the same order for both.

We always get multiple circuits with diverse routing instead of a single
circuit with protection if we can.

/Jesper