How Not to Multihome

I have a client that wants us to advertise an IP block assigned by another ISP. I know that the best practice is to have them request an AS number from ARIN and peer with us, etc. However, I cannot find any information that states as law. Does anyone know of a document or RFC that states this?

Thanks,

Keegan

There is nothing wrong with both of your originating the prefix, as long as the owner gives you permission. Plus it saves an ASN. If the link between you & the customer dies, things get far more interesting, but that doesn't mean you can't or shouldn't do it.

Of course, you can probably still find documentation against it. (You can find documentation for or against just about anything.)

Slightly different approach... Needing to multihome is justification
for requesting an ASN. Is this strictly necessary? No. You can source
the block on his behalf but that creates various routing
inconsistancies. There are other even more unpleasant ways of doing
this that are perfectly feasible. (I'd be willing to use them if I was
the client since I know what I'm doing but I would not be willing to
have a client of mine use them because it would scare the hell out of
me.)

-Wayne

It's not 'law' per se, but having the customer originate their own announcements is definitely the Right Way to go.

Some providers take a pretty dim view of seeing chunks of their address space show up in advertisements originating from someone who isn't one of their customers. It can make troubleshooting connectivity problems for that customer (from the provider's point of view) very painful, i.e. "Hey, this AS, who isn't one of our customers, is hijacking IP space assigned to one of our customers!" The provider could then contact your host's upstream(s) and ask them to drop said announcement under the impression they're stopping someone from doing something bad.

Also, if some network out there aggregates prefixes in an aggessive/odd manner, the disjoint announcement, and the reachability info it contains could be washed out of their routing tables, causing connectivity problems.

Standard caveats about the block being a /24 or larger also apply.

jms

It's not 'law' per se, but having the customer originate their own
announcements is definitely the Right Way to go.

it is interesting, and worrysome, to consider this in light of likely
growth in the routing table (ref ipv4 free pool run out discussion) and
vendors' inability to handle large ribs and fibs on enterprise class
routers.

randy

That brings up an interesing point. My biggest fear was that one of my other customers could possible be closer to me that the ISP that provides the primary link and it would cause them to favor the backup link because of AS path. I think they are going to fight me on this and telling them to multihome to their original ISP would probably be frowned upon at this point. I was hoping that there was an RFC for multihoming that I could use to bail myself out.

"Justin M. Streiner" streiner@cluebyfour.org
Sent by: owner-nanog@merit.edu

10/08/2007 05:55 PM

To
Keegan.Holley@sungard.com

cc
nanog nanog@merit.edu

Subject
Re: How Not to Multihome

On Mon, 8 Oct 2007, Keegan.Holley@sungard.com wrote:

> I have a client that wants us to advertise an IP block assigned by another
> ISP. I know that the best practice is to have them request an AS number
> from ARIN and peer with us, etc. However, I cannot find any information
> that states as law. Does anyone know of a document or RFC that states
> this?

It's not 'law' per se, but having the customer originate their own
announcements is definitely the Right Way to go.

Some providers take a pretty dim view of seeing chunks of their address
space show up in advertisements originating from someone who isn't one of
their customers. It can make troubleshooting connectivity problems for
that customer (from the provider's point of view) very painful, i.e. "Hey,
this AS, who isn't one of our customers, is hijacking IP space assigned to
one of our customers!" The provider could then contact your host's
upstream(s) and ask them to drop said announcement under the impression
they're stopping someone from doing something bad.

Also, if some network out there aggregates prefixes in an aggessive/odd
manner, the disjoint announcement, and the reachability info it contains
could be washed out of their routing tables, causing connectivity
problems.

Standard caveats about the block being a /24 or larger also apply.

jms

I have a client that wants us to advertise an IP block assigned by another
ISP. I know that the best practice is to have them request an AS number
from ARIN and peer with us, etc. However, I cannot find any information
that states as law. Does anyone know of a document or RFC that states
this?

It's not 'law' per se, but having the customer originate their own announcements is definitely the Right Way to go.

That is not at all guaranteed.

Some providers take a pretty dim view of seeing chunks of their address space show up in advertisements originating from someone who isn't one of their customers. It can make troubleshooting connectivity problems for that customer (from the provider's point of view) very painful, i.e. "Hey, this AS, who isn't one of our customers, is hijacking IP space assigned to one of our customers!" The provider could then contact your host's upstream(s) and ask them to drop said announcement under the impression they're stopping someone from doing something bad.

If you do you have permission from the owner of the block, you Should Not Announce it.

If the owner gives you permission and can't figure out why their block is originated by another ASN as well, they need help. (Yes, I realize the latter part of the last sentence is probably true for the majority of providers, but whatever.)

In either case, your hypothetical question should not hold.

Also, if some network out there aggregates prefixes in an aggessive/odd manner, the disjoint announcement, and the reachability info it contains could be washed out of their routing tables, causing connectivity problems.

How is this different than if the customers gets their own ASN and announces a sub-block from one of the providers?

Or are you suggesting they should get PI space?

If the address space is assigned to another provider, getting
clarification from them directly or indirectly(*) that the customer
is allowed to use it, if it is portable or allowed to be seen
outside their ASN, etc. If their are truly attempting to multihome,
doing so with multiple-origin ASNs can create nondeterministic
behavior in the exact failure conditions that multihoming is
seeking to solve. Given the clue level of the customer or their
business/traffic, it may not matter (anycasters, CDNs, or other
edges with levels of indirection).

Most ISPs quite simply have a policy that multihoming is done by
BGP in such and such fasion and that's that.

Cheers,

Joe

(*) whois delegation at best, some providers indicate portability
   or multihoming policy in whois or IRR allocations,etc

The only thing that sounds worse would be to give them address space and tell them to NAT the traffic based on the path it takes.

Keegan Holley
Network Engineer, Network Managed Services
SunGard Availability Services
Mezzanine Level MC-95
401 N. Broad St.
Philadelphia, PA 19108
215.446.1242 (office)
609.670.2149 (cell)
Keegan.Holley@sungard.com

If you went ahead and did this, the more specific route being announced by you on behalf of your customer would be more likely to attract traffic back to you. Prefix length is checked in the BGP route selection process before AS path length. This would work in normal "everything works fine" situations, but when things break, troubleshooting the source of the customer's reachabilit woes will get very interesting.

jms

I’m really interested to see what happens when we start filling those same routers with ipv6 routes.

Randy Bush randy@psg.com
Sent by: owner-nanog@merit.edu

10/08/2007 06:10 PM

To
“Justin M. Streiner” streiner@cluebyfour.org

cc
nanog nanog@merit.edu

Subject
Re: How Not to Multihome

> It's not 'law' per se, but having the customer originate their own
> announcements is definitely the Right Way to go.

it is interesting, and worrysome, to consider this in light of likely
growth in the routing table (ref ipv4 free pool run out discussion) and
vendors' inability to handle large ribs and fibs on enterprise class
routers.

randy

> Also, if some network out there aggregates prefixes in an aggressive/
> odd manner, the disjoint announcement, and the reachability info it
> contains could be washed out of their routing tables, causing
> connectivity problems.

How is this different than if the customers gets their own ASN and
announces a sub-block from one of the providers?

Or are you suggesting they should get PI space?

ARIN will only hand out /22’s or larger. If a client wants to multihome with a /23 or a /24 it has to be assigned by one of hte ISP’s and removed from the aggregate.

"Patrick W. Gilmore" patrick@ianai.net
Sent by: owner-nanog@merit.edu

10/08/2007 06:16 PM

To
nanog nanog@merit.edu

cc
“Patrick W. Gilmore” patrick@ianai.net

Subject
Re: How Not to Multihome

On Oct 8, 2007, at 5:55 PM, Justin M. Streiner wrote:
> On Mon, 8 Oct 2007, Keegan.Holley@sungard.com wrote:
>
>> I have a client that wants us to advertise an IP block assigned by
>> another
>> ISP. I know that the best practice is to have them request an AS
>> number
>> from ARIN and peer with us, etc. However, I cannot find any
>> information
>> that states as law. Does anyone know of a document or RFC that
>> states
>> this?
>
> It's not 'law' per se, but having the customer originate their own
> announcements is definitely the Right Way to go.

That is not at all guaranteed.

> Some providers take a pretty dim view of seeing chunks of their
> address space show up in advertisements originating from someone
> who isn't one of their customers. It can make troubleshooting
> connectivity problems for that customer (from the provider's point
> of view) very painful, i.e. "Hey, this AS, who isn't one of our
> customers, is hijacking IP space assigned to one of our
> customers!" The provider could then contact your host's upstream
> (s) and ask them to drop said announcement under the impression
> they're stopping someone from doing something bad.

If you do you have permission from the owner of the block, you Should
Not Announce it.

If the owner gives you permission and can't figure out why their
block is originated by another ASN as well, they need help. (Yes, I
realize the latter part of the last sentence is probably true for the
majority of providers, but whatever.)

In either case, your hypothetical question should not hold.

> Also, if some network out there aggregates prefixes in an aggessive/
> odd manner, the disjoint announcement, and the reachability info it
> contains could be washed out of their routing tables, causing
> connectivity problems.

How is this different than if the customers gets their own ASN and
announces a sub-block from one of the providers?

Or are you suggesting they should get PI space?

--
TTFN,
patrick

It's not 'law' per se, but having the customer originate their own announcements is definitely the Right Way to go.

That is not at all guaranteed.

I never said it was. My experience, both in my previous life as the operator of a regional ISP and since then in other capacities is that having disjoint origins for a chunk of some provider's address space is basically asking for trouble, and it's the kind of trouble that may ony pop up when something breaks.

My experience has also been that if a customer has a need to multihome and is willing to invest both in the equipment and the expertise to do so, then so be it.

If you do you have permission from the owner of the block, you Should Not Announce it.

Agreed.

If the owner gives you permission and can't figure out why their block is originated by another ASN as well, they need help. (Yes, I realize the latter part of the last sentence is probably true for the majority of providers, but whatever.)

In either case, your hypothetical question should not hold.

Also, if some network out there aggregates prefixes in an aggessive/odd manner, the disjoint announcement, and the reachability info it contains could be washed out of their routing tables, causing connectivity problems.

How is this different than if the customers gets their own ASN and announces a sub-block from one of the providers?

In the case you described, the provider who holds the parent address block should expect to see an advertisement for a chunk of that block come in as part of the BGP feeds they receive from their upstreams, and they need to accept traffic accordingly. The customer would need to tell the provider of their intentions to multihome. If the provider in question employs some type of ingress/egress filtering, that filter would need to be updated to recognize that traffic sourced from that sub-block as legitimate, even if it comes in over their normal transit pipes.

In the case I described, where the end user requested that a third party provide transit for their PA space, without that provider necessarily being aware of it is when things can break in strange and spectacular ways.

Or are you suggesting they should get PI space?

PI space, while nice, is not an option for many end users.

jms

I'm really interested to see what happens when we start filling those
same routers with ipv6 routes.

All 970 of them?

joelja

please elaborate. My knowledge of IPv6 is admittedly lacking, but I always assumed that the routing tables would be much larger if the internet were to convert from IPv4 due to the sheer number of networks available.

Joel Jaeggli joelja@bogus.com
Sent by: owner-nanog@merit.edu

10/08/2007 06:49 PM

To
Keegan.Holley@sungard.com

cc
Randy Bush randy@psg.com, nanog nanog@merit.edu, owner-nanog@merit.edu, “Justin M. Streiner” streiner@cluebyfour.org

Subject
Re: How Not to Multihome

Keegan.Holley@sungard.com wrote:
>
> I'm really interested to see what happens when we start filling those
> same routers with ipv6 routes.

All 970 of them?

joelja

>
>
> *Randy Bush <randy@psg.com>*
> Sent by: owner-nanog@merit.edu
>
> 10/08/2007 06:10 PM
>
>
> To
> "Justin M. Streiner" <streiner@cluebyfour.org>
> cc
> nanog <nanog@merit.edu>
> Subject
> Re: How Not to Multihome
>
>
>
>
>
>
>
>
>
>> It's not 'law' per se, but having the customer originate their own
>> announcements is definitely the Right Way to go.
>
> it is interesting, and worrysome, to consider this in light of likely
> growth in the routing table (ref ipv4 free pool run out discussion) and
> vendors' inability to handle large ribs and fibs on enterprise class
> routers.
>
> randy
>
>
>

I'm really interested to see what happens when we start filling those same
routers with ipv6 routes.

Well, IPv6 prefixes will eventually be some number between the total
number of ASes in use (which represents the number of networks that can
afford and desire to run BGP) and the number of IPv4 prefixes in use
(which represents the number of customers that can afford, justify, and
desire to get address space).

So today, if IPv6 was instantly ubiquitously deployed by every network on
the planet that runs IPv4: you would would see between 26,249 and 235,174
IPv6 routes (data from AS6447 - BGP Table Statistics).

You bought or are planning to buy core routers that support IPv6 at
wirespeed in hardware didn't you?

If you are (or plan to be) operating an IPv4 network for over 5 years (let
alone the folks here that can say 10 or 15+ years), you are planning core
router purchases on a cycle like 3 to 5 years by estimating what you need
at the end of that time and then specifying accordingly.

By looking at the graph at the top of the page
AS6447 - BGP Table Statistics for total route announcements you could
make a wild guess that if you want a router that has a high probability of
working without needing workarounds (or giving you unnecessary headaches)
in 3 years it needs to handle 500,000 IPv4 routes and 500,000 IPv6 routes
in hardware when you buy it. Arguably this is overkill for IPv6, and
might last 5 to 7 years.

*All* the core router vendors (Cisco, Juniper, Foundry, Force10, etc) sell
routers that can do this today. If you are buying something that can't
handle the 3 year IPv4 requirement, let alone the IPv6 requirement, why
are doing that to yourself?

Contrary to what seems to be popular misconception, your refrigerator
will not be multihomed under IPv6. There are dynamic economic pressures
(such as consolidation, competition, effective regulatory monopoly, etc)
that limit the number of networks in the global routing table.

Mike.

+----------------- H U R R I C A N E - E L E C T R I C -----------------+

Currently The IPv6 DFZ is 970 routes from 808 ASes. The reflects
actually pretty steady growth... participating in the IPV6 dfz is not
presently expensive.

Were it maximally aggregated it would be 926 routes. 95% agregation is
pretty nice.

In ipv4 land we're at 239k routes ~154k maximally aggregated 64%
efficiency from the aggregation angle... But only 25506 ASes are
actually participating in the v4 routing system.

While I won't presume that the IPV6 routing table will remain unmessy
when the other 24698 ASes decide they need some v6 I think it's be quite
some time before the v6 picture looks like the v4 one (most of those
networks will not be going back to the rir for a new block at regular
intervals).

Their is always the possibility that somebody decides they need announce
all their /48s for TE or anti-hijacking purposes in which case filters
and clue should be applied in that order. There is no reason in the ipv4
dfz for example for me to need a thousand routes from 18566 or 9498 in
order to reach all their address blocks.

> > I'm really interested to see what happens when we start filling those same
> > routers with ipv6 routes.
>
> Well, IPv6 prefixes will eventually be some number between the total
> number of ASes in use (which represents the number of networks that can
> afford and desire to run BGP) and the number of IPv4 prefixes in use
> (which represents the number of customers that can afford, justify, and
> desire to get address space).
>
> So today, if IPv6 was instantly ubiquitously deployed by every network on
> the planet that runs IPv4: you would would see between 26,249 and 235,174
> IPv6 routes (data from http://bgp.potaroo.net/as6447/).

Wouldn't resources still be an issue. Since the address space is so much larger wouldn't the 235k v6 routes take up more than 4 times the router memory?
>
> You bought or are planning to buy core routers that support IPv6 at
> wirespeed in hardware didn't you?
>
> If you are (or plan to be) operating an IPv4 network for over 5 years (let
> alone the folks here that can say 10 or 15+ years), you are planning core
> router purchases on a cycle like 3 to 5 years by estimating what you need
> at the end of that time and then specifying accordingly.
>
> By looking at the graph at the top of the page
> http://bgp.potaroo.net/as6447/ for total route announcements you could
> make a wild guess that if you want a router that has a high probability of
> working without needing workarounds (or giving you unnecessary headaches)
> in 3 years it needs to handle 500,000 IPv4 routes and 500,000 IPv6 routes
> in hardware when you buy it. Arguably this is overkill for IPv6, and
> might last 5 to 7 years.

What about MPLS? The last time I poked around in one of the core routers there were about 1.2 million routes. This includes all the private space that is routed between different sites in the same vpn and customers that use overlapping IP space. I had always assumed the routing tables would be massive under IPv6.

>
> *All* the core router vendors (Cisco, Juniper, Foundry, Force10, etc) sell
> routers that can do this today. If you are buying something that can't
> handle the 3 year IPv4 requirement, let alone the IPv6 requirement, why
> are doing that to yourself?

Can I interest you in some used M40's?

>
> Contrary to what seems to be popular misconception, your refrigerator
> will not be multihomed under IPv6. There are dynamic economic pressures
> (such as consolidation, competition, effective regulatory monopoly, etc)
> that limit the number of networks in the global routing table.

Well I guess I can nix that rootkit for IP enabled cukoo clocks. : )

Keegan

Not many networks are pushing IPv6 at this point, or more correctly, not many relative to the 230k+ routes that currently makes up a full IPv4 routing table. There likely won't be a mass flash-cut to IPv6, which is a subject that has been debated to death and back again here on NANOG over the last few weeks :slight_smile:

jms

Keegan,

According to Cisco's product feature pages IPv6 routes take up twice
as much space in the TCAM as IPv4 routes. Make of that what you will.

Regards,
Bill Herrin