RE: multi-homing fixed

That is easy.

She is not:

One of the old crowd.

One of the power elite.

She is probably:

Trying to provide a service, which if successful would threaten the sit-on-
their-ass crowd.

Too young to have been around when what matters happened--the gathering
up of tons of address-space at no cost to the gatherers.

Oh, come on. I've never known Randy to discriminate on "old crowd" or
"power elite". A bad business plan or delusions of grandeur is another
thing.

The real problem with most basement multi-homers is they go with the
cheapest local service they can get, often from someone clueless with one
POP / one path. To fix this, they add another cheap, local, clueless
service and pray they don't get clueless at the same time. Then they
inflict bad judgement on the rest of the Internet by demanding their routes
be distributed. Bad plan.

Better to buy from someone with a clue, with a real (redundant path)
backbone, and provision as many lines as you want into disparate POPs. Even
better, get out of the basment where you're dealing with an ILEC for your
last mile. Most small business parks have redundant paths these days. Get
your address space from the provider and worry about your business, not your
connectivity. Good (at least better) plan. It's been a long time since a
major provider took out their whole net, or even a geographic region.

Dave, speaking for myself

I've asked this of a number of people now, but how many providers
have multiple POP's in a city that are _completely redundant_?
That is, they can operate _fully_ with one POP out of service?

In New York, Washington DC, Chicago, the bay area and maybe one or
two other spots you most likely have a half dozen choices. In many
other NFL cities, say Green Bay, Tampa, Cincinatti, Indianapolis
and the like if you have more than one choice I'd be surprised,
and in several if you even have one choice I'd be surprised. Even
if they have two pops, many of those cities won't have redundant
long haul capacity. One POP will either be behind the other, or
they are oversubscribed on the long haul.

The real problem with most basement multi-homers is they go with the
cheapest local service they can get, often from someone clueless with one
POP / one path. To fix this, they add another cheap, local, clueless
service and pray they don't get clueless at the same time. Then they
inflict bad judgement on the rest of the Internet by demanding their
routes be distributed. Bad plan.

I do not think anyone (Randy included) is questioning the right of
basement-dwellers to multihome (by my previous definition). I think
what is being questioned by many and various is
(a) their right to do it at other people's expense, without
    reimbursement
(b) whether the (non-reimbursed) cost to the community is
    greater than the (non-paid for) gain to the community.
(c) whether there are other technologies which cost less
    in total, and/or attribute cost more directly to those
    who benefit from it.
(d) whether in an effort to achieve multihoming, they are
    selecting the technology which costs them the least, or
    costs the community the least.

Whilst there is no current mechanism to reliably achieve
(a) (beyond Roeland kindly offering to pay for Sean's
routers), direct market forces fail, so, like with so many
other problems, the internet community has come up with
hueristic mechanisms to enforce (b) i.e. 'your reachability
information is only worth the cost of my carrying it
if it contains announcements shorter than a /nn, and I
will rely on RIR's to demonstrate that there is a fair
correlation between assigment size (and thus prefix
length) and usefulness of the prefix to me.

If all this sounds a bit "matter of opinion", type stuff,
which will never get resolved, well, yes it is, and thus
just the right sort of stuff for a flamewar on NANOG.

Great, just so long as elsewhere, people are thinking about
(c). And then we can have the adoption flamewar (d) on NANOG
afterwards.

True enough. But you don't really need multiple POPs in a city. Frame
Relay and ATM are both distance insensative, pricewise. Most, if not all,
of the serious players have discounts off list from various providers so
it's reasonable to provision one or more circuits well out of the local
area. Deals can usually be worked for dedicated facilites too.

Having just had my DSL go down yet AGAIN (a more or less daily occurrence), I'm inclined to chip in under my telecommuter hat. Yes, I know the best way is to convince my boss to pay for frac T1/frame access with dial backup. Working on that.

In the meantime, I have DSL from CAIS, with Covad as the CLEC. Covad is in Chapter 11. I've also ordered @home cable to come in for next week, and I'm trying to scrounge a multiple-Ethernet router to set up alternate connectivity. (Note that I work for a router vendor, so I can't go and do something as simple as mail-order a router). @home doesn't seem to be in much better financial shape.

The real problem with most basement multi-homers is they go with the
cheapest local service they can get, often from someone clueless with one
POP / one path. To fix this, they add another cheap, local, clueless
service and pray they don't get clueless at the same time. Then they
inflict bad judgement on the rest of the Internet by demanding their
routes be distributed. Bad plan.

I do not think anyone (Randy included) is questioning the right of
basement-dwellers to multihome (by my previous definition). I think
what is being questioned by many and various is
(a) their right to do it at other people's expense, without
   reimbursement
(b) whether the (non-reimbursed) cost to the community is
   greater than the (non-paid for) gain to the community.
(c) whether there are other technologies which cost less
   in total, and/or attribute cost more directly to those
   who benefit from it.
(d) whether in an effort to achieve multihoming, they are
   selecting the technology which costs them the least, or
   costs the community the least.

What I'd like to see, as a short-term fix, is to have two local providers each agree to have a multihomed block within their allocations, and both to propagate this block to the DFZ and each other. Microallocations would come out of it; the microallocations would not be advertised between the two carriers. Certainly, there would be failure modes in which the microallocation might go down for one provider, but I'd be in better shape. I'd ideally pick local carriers with different kinds of physical connectivity.

While I'm perfectly capable of running BGP with both carriers, I recognize that skill would be rare in the basement market, and I can't reasonably expect it. But I am getting truly sick of dial backup on a per-host basis.

*thank you -- this may have been more venting steam than anything else **

Well it looks like we have come full circle.

www is born.
ISP numbers increase like rabbits in australia (I even start one)
a lot of $$ is invested in XLEC
Market tanks
XLEC, ISPs, .. file chapter 11
at the end of the day you can even get a dissent Internet connection from a
single provider.

This could be the right theme for the next Nanog T-shirt.

ak

"Howard C. Berkowitz" wrote:

Did you see where Excite@home is in bad financial shape and expected to run out
of cash within months.

Roy

"Howard C. Berkowitz" wrote:

...

In the meantime, I have DSL from CAIS, with Covad as the CLEC. Covad
is in Chapter 11. I've also ordered @home cable to come in for next
week, and I'm trying to scrounge a multiple-Ethernet router to set up
alternate connectivity. (Note that I work for a router vendor, so I
can't go and do something as simple as mail-order a router). @home
doesn't seem to be in much better financial shape.

...

Did you see where Excite@home is in bad financial shape and expected to run out
of cash within months.

Roy

Yep. And Verizon, the ILEC, doesn't seem to understand the concept of needing a subnet -- their xDSL is host-only (I have stories about that).

And trying to call some of the more established ISPs (Verio, UUnet, etc.) to get a quote on frac T1 or FR access gets a salesdroid that doesn't seem to listen to what I am saying.

Among other things, that proposing a Cisco router to a Nortel employee probably isn't the best strategy. That I really am not interested in web services. That I want to know about service response times (SLAs being too much to expect).

Ironically, I have far more experience with Cisco gear than Nortel, since I'm in advanced technology and don't work daily with the current products. But then the Ciscos proposed are far more expensive than models that would do the job perfectly well.

I think what is being questioned by many and various is

...

(c) whether there are other technologies which cost less
    in total, and/or attribute cost more directly to those
    who benefit from it.

Much of the other discussion on this topic seems to assume that effective
multihoming means that you have a prefix which is in every BGP route table
throughout the Internet.

This is simply not required. There are degrees of multihoming. Let me
chime in with one:

A modest operation which requires multihoming can select two providers
according who meet the following criteria:

1) Connectivity to each provider is available and cost-effective
2) The two providers meet somewhere else
3) Both providers agree to provide you with address space
4) Both providers agree to let you announce your allocation
5) Both providers agree to specifics from you and from each other

The rest of the world can filter your specifics and you still have very
good redundancy. If you think through the realistic failure modes, they
are few and manageable. (That includes telecom failures, network
congestion, BGP failures, business failures.)

The rest of the world (which you are not paying) is free to listen to your
specifics if their infrastructure can handle the routes, or to filter them
to protect the stability of their networks. Your reliability and
connectivity will not be fundamentally threatened.

-Steve

Dashbit - The Leader In Internet Topology
www.dashbit.com www.traceloop.com

FWIW: You can get a DSL provisioned through Verizon DSLAMs via many
different ISPs (they vary per LATA). Many of them (cough) offer
business-class services (subnets, VPN/IPSEC).

Chain goes like this: (In most states, example given in VZ and SBC area)
ILEC (Verizon/SBC) sells physical copper lines to
  DILEC (VADI/ADI) who provides "Layer 2" service to
        ILEC's ISP (Verizon Online DSL)
    Many other ISPs.

Legally (again, in most states), DLEC arm of ILEC is not allowed to
provide layer 3 service, and is obligated to sell the service to all ISPs
including ILEC-affiliated ISP on equal terms.

FWIW: You can get a DSL provisioned through Verizon DSLAMs via many
different ISPs (they vary per LATA). Many of them (cough) offer
business-class services (subnets, VPN/IPSEC).

Until Verizon pulls an SBC on you and forces all your clients to connect
through a "broadband gateway" that lets them sell other services directly
to YOUR customer through that pipe...

See http://www.cispa.org/forum-dsl-news.html and
http://www.zdnet.com/intweek/stories/news/0,4164,2787113,00.html

Charles

how many providers have multiple POP's in a city that are _completely
redundant_? That is, they can operate _fully_ with one POP out of
service?

none can operate *fully*, as a customer access line pretty much has to
terminate in a single router which can, and eventually will, fail.

but, most large providers have more than one pop in the largest cities,
bay area, nyc, dee cee, etc. and those pops are redundantly and diversely
wired. if not, don't buy from them. life can be simple.

of course, in toledo, you're probably sol.

randy

The real problem with most basement multi-homers is they go with the
cheapest local service they can get, often from someone clueless with one
POP / one path. To fix this, they add another cheap, local, clueless
service and pray they don't get clueless at the same time. Then they
inflict bad judgement on the rest of the Internet by demanding their routes
be distributed. Bad plan.

what is interestingly bad about this plan is that it attempts to save the
basement-dweller money at the expense of everyone else. and that's the
point of the filtering story.

randy

Hmm...we usually get hit with extra charges for crossing lata boundaries.
I wouldn't say Frame is entirely distance insensative.

Besides, even if a little guy in Gainesville, FL does get Frame to say
UUNet in Miami and Jacksonville, this'll take care of when one of your
circuits goes out and UUNet or Bell can't figure out what happened for 12
hours, but it totally ignores the fact that at times (often for extended
times) peering connections between various Tier-1's suck. I remember at
least one time for several weeks when crossing between UUNet and Sprint
meant >1000ms response times.

Redundancy is only one reason to multihome. More paths (hopefully at
least one per destination that doesn't suck) is another big one, and
you're not going to get this benefit from adding N circuits to one "big
clued-in provider"...though you might get it from a medium sized regional
provider that buys transit and doesn't have overloaded peering
connections.

We had some truly sucking paths to some destinations yesterday (15000ms
over our OC-3) until our chief NOC monkey said enough was enough and
nuked the BGP session with the other end.

Happen to have any pointers to "this path sucks no matter what BGP says"
tools? Especially for the case of the *real* problem being 2 or 3 AS's
down the chain?

Or does everybody's noc monkeys wait for the "foobar.com sucks" phone calls?

        Valdis Kletnieks
        Operating Systems Analyst
        Virginia Tech

Judging strictly from the present economic climate, a lot of dotcoms that
went all out in style aren't around anymore, or are on the brink of
bankruptcy, while many of those despised basement dwellers managed to
prosper, or at least remain solvent in these hard times. So nothing like
a bit of convenient protectionist filtering to keep the competition down...

--Mitch
NetSide

So we don't want to force networks in the default-free zone to buy bigger
routers with more memory, but it's ok to force them to essentially build a
second network by having redundant pops in every city?

I'm sure the router vendors and colo builders would love this idea, but I
don't think throwing hardware at the problem will help in the long run.

Iljitsch van Beijnum

Noone is forcing anyone to do anything. Providers have a simple financial
incentive to build redundant network - customers pay more for connections
with higher availability and diversity. There is currently little financial
incentive for carrier's to carry other people's microallocated / 'TE'
routes, as /many/ of these do not materially affect goodput, and/or
are goodput to destinations less interesting than the cost they incur.
If I could think of a good way to measure usefulness of a prefix [1], I'd
bet that the (positive) corelation between it, and number of IP's in
that prefix is declining, but I bet small prefixes are still considerably
less useful than large ones, but cost the same.

[1] measuring traffic to/from prefixes against prefix size
(i.e. 2^(32-len)), as a % of total traffic, and plotting
these over time, would make an interesting research study.
Perhaps someone working in a research department at a major
backbone already has some stats they could do something
similar with.

Alex Bligh
Personal Capacity

Interesting assertion. Here's why I think it's wrong:

Providers have an incentive to make money. Anyone disagree?

As time has shown, the majority of providers will announce prefixes longer
than a /20 for a customer if they request it, although they are quick to state
that they do not guarantee global routability.

Providers have relationships with other providers. Sometimes these
relationships take the form of cost-free peering. In others there is an
exchange of money.

In the former case, it would seem that if provider A wants the routes they
are announcing to be seen by provider B and vice-versa in the interest of
providing arguably better reachability for their customers, they agree to
accept each others routes. Better reachability for customers = happier
customers = more revenue.

In the second case since direct consideration exists and a provider is in
fact incentivized to make money, they largely will accept the routes.

This is not theory, it's what exists today. There are of course
the few exceptions to the rule that argue voiceferously against the practice,
but they are in fact fighting revenue and for better or worse the bean
counters usually win those arguments in the long run.