Verio Peering Question

Multihoming costs a lot of money

I have had a couple friends -- gamers, rather than network techs --
ask me how they could take advantage of their various household
broadband connections (cable, DSL and ethernet-from-bredbandsbolaget.se)
to increase their download speeds, handle upstream outages, and
improve RTTs to selected targets. Admittedly their providers are
unlikely to offer them "true multihoming" in the BGP + PI prefix
sense, however I think it is an indication of demand in need of
some supply. The supply is not horrifically expensive for at
least one of the broadband providers, and the others could
in principle constrain their _local_ costs enough that passing
them down to my friends would not result in an unaffordable option.

Is this a trend, or do I simply collect friends who are completely
unreflective of reality? Is my estimation that for at least some
broadband providers, per-household/per-customer BGP is a operational
expense rather than one requring the capital purchase of new equipment,
completely out-to-lunch (in advance of an interesting new product
launch in the next few days)?

  Sean.

Date: Wed, 3 Oct 2001 05:53:39 -0700 (PDT)
From: Sean M. Doran <smd@clock.org>

[ snip anecdote about demand for home multihoming ]

Is this a trend, or do I simply collect friends who are

I've noticed it, too... in some ways demand is even greater than
among small ISPs who have an inkling about how BGP works.

The "BGP uninformed" ask, "Why can't traffic just choose one of
two paths?" To them, it's all "The Internet"... routing is black
magic behind the scenes that "just works", and all traffic should
be able to use all of their connections.

completely unreflective of reality? Is my estimation that for
at least some broadband providers, per-household/per-customer
BGP is a operational expense rather than one requring the

There are parties who are taking this into consideration.

capital purchase of new equipment, completely out-to-lunch (in
advance of an interesting new product launch in the next few
days)?

Re the "high cost" of multihoming... perhaps now. Most "smaller
places" can't afford to multihome given the current cost of two
T1s (hard to get BGP over broadband) and a Cisco that holds 128M
(even "smaller places" seem to concerned about brand recognition,
and are often reluctant to run Zebra).

However, I've encountered [consulting] customers with multiple
_dialup_ connections who want to know if they can just balance
traffic across both. I think that the demand is there -- current
products just don't allow it.

Eddy

I've noticed it, too... in some ways demand is even greater than
among small ISPs who have an inkling about how BGP works.

> Is my estimation that for at least some broadband providers,
> per-household/per-customer BGP is a operational expense

There are parties who are taking this into consideration.

> capital purchase of new equipment, completely out-to-lunch (in
> advance of an interesting new product launch in the next few
> days)?

Re the "high cost" of multihoming... perhaps now. Most "smaller
places" can't afford to multihome given the current cost of two
T1s (hard to get BGP over broadband) and a Cisco that holds 128M
(even "smaller places" seem to concerned about brand recognition,
and are often reluctant to run Zebra).

Without trying to start a flamewar, this is one of the places where NAT is
exceptionally valuable. Setting up "redundant" connectivity for these
users given a set of n consumer-grade, commodity connections (DSL, dialup,
cable, etc) is rather trivial, and NAPT implementations have gotten robust
enough to accomodate most of the common layer-ignorant protocols. This
user doesn't want global route visibility, nor do they give a shit about
filtering or allocation policies -- they want to be able to get to their
pr0n when The Internet is broke.

This `knowledgable' SOHO user is most likely already using NAT to get
their office buddies online across the $40/mo DSL link -- likewise, the
use of NAT for multihoming isn't introducing any new complications into
their End-to-End Experience (tm).

For inbound services, where address distribution and portability is of the
most concern, SMTP is the most to worry about, and multiple equal-weight
MX's (to each of the PA addresses) take care of the problem. For a
business considering this, and interested in external services, if they
haven't already signed up for the $20/mo Web Hosting account from the back
pages of $PC_USER_RAG, convincing them to do so shouldn't be hard.

Given these caveats, the one problem that consistently comes up is
link-state inspection. For the low-end, people are using the ethernet -
<DOCIS,ADSL,2-way Satellite> bridge they got with the connection. PPPoE
makes this easier, as there's an interface on the router that will change
state, but otherwise its a guessing game. This is where providers aren't
even begining to play -- forget BGP peers to end-users, has anyone had any
luck getting any of the consumer-grade 'broadband' providers to do so much
as a RIPv1 default advertisment?

Of course, while this does keep the global routing table free of
non-aggregated micro-allocations, it does increase overall utilization.
Cleaner solutions for effective multihoming in the future are certainly
needed, but Multihoming For Dummies is quite obtainable today.

However, I've encountered [consulting] customers with multiple
_dialup_ connections who want to know if they can just balance
traffic across both. I think that the demand is there -- current
products just don't allow it.

It might be pricey for that application, but their 1750 can do it just
fine. The pre-packaged "Internet Router/Firewalls" flooding the market
from Linksys/Netgear/etc haven't caught on yet, but its just a matter of
time.

..kg.. <back to lurking>

> Multihoming costs a lot of money

I have had a couple friends -- gamers, rather than network techs --
ask me how they could take advantage of their various household
broadband connections (cable, DSL and ethernet-from-bredbandsbolaget.se)
to increase their download speeds, handle upstream outages, and
improve RTTs to selected targets.

This is exactly why I assumed a billion multihomers on multi6. But this
has very little to do with the growth of the global routing table in IPv4,
since there is no way current protocols can accomodate these numbers of
multihomers.

Is my estimation that for at least some
broadband providers, per-household/per-customer BGP is a operational
expense rather than one requring the capital purchase of new equipment,
completely out-to-lunch (in advance of an interesting new product
launch in the next few days)?

If you want to, you can certainly talk BGP to customers like this. I'm not
sure how many BGP sessions one router can accomodate, but it's probably
hundreds. Of course flapping will be a problem and nobody in their right
mind is going to accept those announcements.

If there is a market for home multihoming (mh�) then I'm sure someone will
create a service where you can get a fixed IP address and easily create
tunnels. If the tunnel endpoints are at the closest regional exchange
point, the RTTs should be pretty good.

The "BGP uninformed" ask, "Why can't traffic just choose one of
two paths?" To them, it's all "The Internet"... routing is black
magic behind the scenes that "just works", and all traffic should
be able to use all of their connections.

Actually it is not too hard to get this to work for outgoing traffic. Just
give the box two addresses and point a route over the second connection.
All TCP and UDP traffic the box initiates over this route will
automatically have the right source address so the traffic comes back the
same way. The problem starts with incoming connections. (And possibly
Windows is too stupid for this.)

This is how I got my first job with an ISP, way back when...

However, I've encountered [consulting] customers with multiple
_dialup_ connections who want to know if they can just balance
traffic across both. I think that the demand is there -- current
products just don't allow it.

That's why we need multi address multihoming. And Mobile IP already makes
it possible to change addresses in mid-communication, so we're halfway
there.