RE: Faster 'Net growth rate raises fears about routers

From: Bill Woodcock [mailto:woody@zocalo.net]
Sent: Monday, April 02, 2001 2:51 PM

    > From what I just went through (DSLnetworks v Covad),
this is straight
    > bulldust. Had I been properly multi-homed, MHSC would
NOT have been down for
    > 5 weeks.

Patrik said ISPs, not IAPs.

So, please explain to me how not being multi-homed is anything other than a
bad-thing and high-risk? No, I am not including colo, because it is assumed
that you know what their arrangements are before you "buy". Reputable colos
are multi-homed, in spades.

So, please explain to me how not being multi-homed is anything other than a

    > bad-thing and high-risk? No, I am not including colo, because it is assumed
    > that you know what their arrangements are before you "buy". Reputable colos
    > are multi-homed, in spades.

You say "responsible cab drivers must have not one, but two taxicabs, in
order to provide service in the event of a failure. Therefore, I bought
one from Fisher-Price, and one from Hot Wheels, and I'm astounded to find
that neither provides me with the luxury which I expected." I think
Patrik may have been suggesting that if you had a Checker, you might not
need to worry quite so much about redundancy.

                                -Bill

it also works the other way around. letting clueless networks manage multihoming with BGP
can be a bad thing for the internet in general, especially if the upstreams do not enforce some
*discipline* on their customers in these cases. this can be as detrimental to the customer as
having lousy upstream service.

If somebody runs over your Checker with a backhoe, it's just as
undrivable as the Fisher-Price or Hot Wheels.

You say "responsible cab drivers must have not one, but two taxicabs,
in order to provide service in the event of a failure. Therefore, I
bought one from Fisher-Price, and one from Hot Wheels, and I'm
astounded to find that neither provides me with the luxury which I
expected." I think Patrik may have been suggesting that if you had
a Checker, you might not need to worry quite so much about redundancy.

...and then a meteor lands on the checker...

Better variant of the analogy would be "we lease checkers from a fleet
provider, so if one goes out we have access to more."

Short-term the best hope for this is for businesses to put their boxes in
colo farms or at an ISP with multi-homed networks in place. The problems
start when customers try to multi-home from their HQ facility or from
somewhere else that's isolated.

Convincing customers that it is cheaper/better to put their main servers
somewhere off-site away from them is the challenge. Otherwise more of them
would do it.

"Eric A. Hall" wrote:

> You say "responsible cab drivers must have not one, but two taxicabs,
> in order to provide service in the event of a failure. Therefore, I
> bought one from Fisher-Price, and one from Hot Wheels, and I'm
> astounded to find that neither provides me with the luxury which I
> expected." I think Patrik may have been suggesting that if you had
> a Checker, you might not need to worry quite so much about redundancy.

...and then a meteor lands on the checker...

Better variant of the analogy would be "we lease checkers from a fleet
provider, so if one goes out we have access to more."

Short-term the best hope for this is for businesses to put their boxes in
colo farms or at an ISP with multi-homed networks in place. The problems
start when customers try to multi-home from their HQ facility or from
somewhere else that's isolated.

Convincing customers that it is cheaper/better to put their main servers
somewhere off-site away from them is the challenge. Otherwise more of them
would do it.

OK. So now you have a business with their servers in a colo, all nice
and protected. Now they need their offices multihomed, so that when
UUNet's cloud falls over, or some DSL provider goes under, they still
have the ability for the humans in their offices to get out to the
Internet.

Let's be clear here: Servers are NOT the only critical item. Companies
have REAL needs for their employees to be able to access the Internet. A
redundant solution to that access can and often is mission critical.
What do you suggest for this case? There are a few options:

1. Use a multi-port NAT box to do the multi-homing. Advantages: avoids
all interactions with ISPs over the multi-homing issue. Disadvantages:
it's NAT (see many other documents if you need an explanation for the
limitations).

2. Find a way to get a /24 or larger so that employees have real IP
addresses, and then fight to get that multihomed to multiple SEPARATE
Internet Providers.

3. Buy a private circuit to your colo (assuming it's close enough to
make this possible), and pay the exhorbitant bandwidth fees to your colo
as a failover solution.

4. Set up a modem on a NAT box and call it your Internet connectivity
for 6 to 8 weeks after your upstream ISP goes away. Run and hide so that
you don't have to hear the screams from employees and management.

Daniel Senie wrote:

4. Set up a modem on a NAT box and call it your Internet connectivity
for 6 to 8 weeks after your upstream ISP goes away. Run and hide so that
you don't have to hear the screams from employees and management.

5. Get set up with T-1's from providers that will do multihoming. My new office
will have a T-1 from my ex-employer and a T-1 from Intermedia and we'll
run BGP. I'm doing so specifically for redundancy, not because I need to
run BGP due to a large address space (it'll be a few /24's).

Steve Sobol wrote:

Daniel Senie wrote:

> 4. Set up a modem on a NAT box and call it your Internet connectivity
> for 6 to 8 weeks after your upstream ISP goes away. Run and hide so that
> you don't have to hear the screams from employees and management.

5. Get set up with T-1's from providers that will do multihoming. My new office
will have a T-1 from my ex-employer and a T-1 from Intermedia and we'll
run BGP. I'm doing so specifically for redundancy, not because I need to
run BGP due to a large address space (it'll be a few /24's).

mental note: don't jump into a thread without having read all of the messages
in the thread first. ignore this, folks - sorry.

Eric Hall wrote:

Short-term the best hope for this is for businesses to put their boxes in
colo farms or at an ISP with multi-homed networks in place. The problems
start when customers try to multi-home from their HQ facility or from
somewhere else that's isolated.

Convincing customers that it is cheaper/better to put their main servers
somewhere off-site away from them is the challenge. Otherwise more of
them would do it.

  I've been in this situation as a consultant a few times, working with a
customer to evaluate multihoming versus other possible solutions. Generally,
colocation is in fact cheaper. It's cheaper to bring the server to the
bandwidth than the bandwidth to the server.

  Relibability is, of course, much less cut and dry. If you have the ability
to run your own network competently, multihoming adds a modicum of
protection against provider outages and misconfigurations. On the other
hand, if you have only one provider, you have (at least to some extent)
outsourced your network management and have someone else to go to if things
don't work. A single provider also has nobody else to point the finger at.

  One point worth stressing is that even if you have two links from your own
facility, they may fail in tandem due to telco/loop issues. On the other
hand, a high-end colocation provider is much more likely to have circuit
diversity across carriers and in disparate directions. In addition, scaling
bandwidth is generally easier at a colocation facility.

  For Internet access for human beings (not servers), there is no need for IP
addresses to remain static. So you can use NAT, DHCP, or proxies and change
providers reasonably easily. You can also use multiple concurrent providers
without having to BGP multihome (since you don't particularly care about any
given address being reachable from the outside in).

  If you need access both for servers and at an office, and it's all mission
critical, it's hard to argue for server colocation and two T1s to the
office. The problem is that this solution starts to get so complex that
multihoming seems simple by comparison. The benefit of not multihoming is
single-source responsibility -- lose that and there's almost no reason not
to multihome.

  Of course, there is no good way to address the risk of your provider going
out of business. Related issues include the provider suddenly sending you a
bill for about five times what you actually owe them and insisting that you
pay it in 8 days or they'll shut you off.

  I've also heard people say that it's more impressive to customers if we
have our own IPs, ASN, etcetera (I hear that *way* too often). I've also
heard the argument that you want to be able to show your investors your
infrastructure. On the other hand, I've also heard "it'll really impress
people if we colocate at X because that's where Y colocates."

  The biggest problem I see is that the cost that a small company multihoming
places on everybody else isn't borne by the company. So when they ask, "why
shouldn't I multihome", it's hard to say, "because everyone else would
prefer that you don't".

  I think the best solution is to make it easier (on everyone) for people to
multihome with technological changes rather than to try to talk them out of
multihoming.

  DS

Once upon a time, Daniel Senie <dts@senie.com> said:

OK. So now you have a business with their servers in a colo, all nice
and protected. Now they need their offices multihomed, so that when
UUNet's cloud falls over, or some DSL provider goes under, they still
have the ability for the humans in their offices to get out to the
Internet.

Or access to their servers. What is on that server may be critical to
your business. When the server is in your office, your employees have
good access to it but the Internet may not, and if you put the server in
a co-lo, the Internet has good access to it but now your employees
don't.

Chris chimed in:

Or access to their servers. What is on that server may be critical to
your business. When the server is in your office, your employees have
good access to it but the Internet may not, and if you put the server in
a co-lo, the Internet has good access to it but now your employees
don't.

It still makes sense for a a colo'd server as opposed to multihoming
for a lot of purposes and customers. --Mike--