> I'm a little disappointed in Verio, if they really did decide to accept
> your unneccessarily deaggregated prefixes.
I'm a little disappointed you're wasting list bandwidth after this has
been well discussed, and not a single post has offered a better
technical alternative to de-aggregating my ARIN /20 (given my network
topology).
-Ralph
Announce your largest aggregate, and announce more-specifics tagged
no-export to those peers who agree to accept them?
Upgrade the connectivity between your sites?
(although I could have sworn I saw someone else make the above
suggestions, already)
> I'm a little disappointed you're wasting list bandwidth after this has
> been well discussed, and not a single post has offered a better
> technical alternative to de-aggregating my ARIN /20 (given my network
> topology).
>
> -Ralph
Announce your largest aggregate, and announce more-specifics tagged
no-export to those peers who agree to accept them?
Which is worse than announcing just the more specifics to 2 different
transit providers in 2 different cities.
Upgrade the connectivity between your sites?
Technically sound, economically stupid. You offering to pony up the
$5K/mth for an OC3 so I can have a redundant link between Ottawa and
Toronto?
Besides the technical aspects of my network, I didn't see any proof that
relaxed prefix filtering (and no I'm not saying accept /32's - the more
specifics I got Verio to accept were /23's) would cause significant
(i.e. >30%) routing table bloat when that was recently discussed.
Say hypothetically it costs approximately $0.10 per route (in routing
cpu/ram cost) per router. (average router cost $12,000 and 120,000
routes in the table).
Lets also say hypothetically the average bgp speaking user has 3 bgp
speaking routers.
And lets say there are 25,000 AS's in use.
By announcing as 4 blocks, instead of 1 aggregate, you are costing the
bgp speaking community a total of $30,000.
You are saving yourself money, at everyone else's expense.
I know my math is not exactly correct, but it's an example to show why
people get pissy about this sort of thing..
You aren't the biggest offender, but how should anyone draw an arbitrary
line for "you are polluting too much" and "you are polluting, but to a
reasonable extent".
At this point, I have my whole network in NYC, so there isn't really any
need for deaggregation. If/When my network expands to other cities, I
will be planning things out to avoid deaggregating -- most likely with
the deaggregate+no-export method.
You could do a deaggregate+no-export method as well, even with your two
different transit providers. You would just need to run ebgp-multihop
to each of them from the opposite network, and announce your
more-specifics there. Not a perfectly clean method, but at least it
keeps your pollution local.
> Announce your largest aggregate, and announce more-specifics tagged
> no-export to those peers who agree to accept them?
Which is worse than announcing just the more specifics to 2 different
transit providers in 2 different cities.
Worse for those two transit providers, not the rest of the world.
> Upgrade the connectivity between your sites?
Technically sound, economically stupid. You offering to pony up the
$5K/mth for an OC3 so I can have a redundant link between Ottawa and
Toronto?
You are choosing to save money by poluting the global routing table. There
may not be anything wrong with that, but don't be surprised when you hit
providers who don't want or need to listen to your more specifics.
Stop whining about it and fix your announcements.
Besides the technical aspects of my network, I didn't see any proof that
relaxed prefix filtering (and no I'm not saying accept /32's - the more
specifics I got Verio to accept were /23's) would cause significant
(i.e. >30%) routing table bloat when that was recently discussed.
Verio carries something in the low 90k's, about a 20k savings (18%).
You aren't the biggest offender, but how should anyone draw an arbitrary
line for "you are polluting too much" and "you are polluting, but to a
reasonable extent".
The most reasonable and quantitative means I can see is technical; if
there is no network engineering benefit to announcing more specifics it
should not be done. There's lots of swamp space where you'll see 2
contiguous /24's announced with the same AS path, metrics, etc. Those are
the prefixes you should be pushing to get aggregated, not mine.
You could do a deaggregate+no-export method as well, even with your two
different transit providers. You would just need to run ebgp-multihop
to each of them from the opposite network, and announce your
more-specifics there. Not a perfectly clean method, but at least it
keeps your pollution local.
Then there is no ability for remote networks to choose the best path to my
Toronto vs Ottawa networks (since the different transit providers would
announce only the /20). Instead of using more router CPU/mem, this uses
more network bandwidth than necessary (statistically speaking traffic has
a 50% chance of going to the "wrong" transit provider). As well, for the
ebgp-multihop to work wouldn't that require some extra static routes to be
setup by my transit providers?
> > Announce your largest aggregate, and announce more-specifics tagged
> > no-export to those peers who agree to accept them?
>
> Which is worse than announcing just the more specifics to 2 different
> transit providers in 2 different cities.
Worse for those two transit providers, not the rest of the world.
Why won't the rest of the world see extra hops and increased latency
reaching my network (for the 50% of the time that the wrong transit
provider is picked).
> > Upgrade the connectivity between your sites?
> Technically sound, economically stupid. You offering to pony up the
> $5K/mth for an OC3 so I can have a redundant link between Ottawa and
> Toronto?
You are choosing to save money by poluting the global routing table. There
may not be anything wrong with that, but don't be surprised when you hit
providers who don't want or need to listen to your more specifics.
Stop whining about it and fix your announcements.
The proposed "fix" is a big mess.
My solution doesn't add to the global routing table since I'm renumbering
out of a few /24's to the /20 I received from ARIN. I'm only
de-aggregating to a /23, not /24. My solution provides optimal routing vs
announcing the /20 globally.
The proposed "fix" will waste bandwidth, increase latency, and far more
problematic to implement; how many NOC monkeys do you know that will be
able to grasp the purpose of the 2 BGP sessions (one of them
ebgp-multihop) let alone fix it if something goes wrong?
> Worse for those two transit providers, not the rest of the world.
Why won't the rest of the world see extra hops and increased latency
reaching my network (for the 50% of the time that the wrong transit
provider is picked).
Because you could *gasp* be intelligent with your network design and do
things like purchase transit from the same carriers in both your serving
markets. Announce the /20 everywhere, announces your /23/whatever out the
local connections for that city with no-export attached. The two carriers
you *pay* for transit hold the more specifics to route the last leg to
you, the rest of the world doesn't need or have to receive your
deaggregation. This also gives added benefits like not slamming your
inter-city circuits quite as hard when your link to one of your upstreams
goes down.
The problem here Ralph is that you see absolutely no problem with doing
things on your network that have minimal benefit to you, yet have global
impact.
There's currently around 23-24k prefixes out there that are more specific
announcements from the same origin AS as the aggregate. If all that junk
would somehow magically get cleaned up the extra prefixes seen from
multihomed networks announcing holepuncheed provider owned space would be
_much less significant_, and RIR filters probably wouldn't even be
necessary. You claim that your announcements have no global impact..
just like the thousands of others just like them, right?
The proposed "fix" is a big mess.
My solution doesn't add to the global routing table since I'm renumbering
out of a few /24's to the /20 I received from ARIN. I'm only
de-aggregating to a /23, not /24. My solution provides optimal routing vs
announcing the /20 globally.
Your network is the mess, not the solution. You've been given a /20 from
ARIN, the responsible thing to do would be to do everything you can to
make sure 90% of the internet sees that /20 and only that /20. The other
10% would be the people you pay money to for transit service who happily
accept more specific prefixes because you are their customer.
The proposed "fix" will waste bandwidth, increase latency, and far more
problematic to implement; how many NOC monkeys do you know that will be
able to grasp the purpose of the 2 BGP sessions (one of them
ebgp-multihop) let alone fix it if something goes wrong?
> You could do a deaggregate+no-export method as well, even with your two
> different transit providers. You would just need to run ebgp-multihop
> to each of them from the opposite network, and announce your
> more-specifics there. Not a perfectly clean method, but at least it
> keeps your pollution local.
Then there is no ability for remote networks to choose the best path to my
Toronto vs Ottawa networks (since the different transit providers would
announce only the /20). Instead of using more router CPU/mem, this uses
more network bandwidth than necessary (statistically speaking traffic has
a 50% chance of going to the "wrong" transit provider). As well, for the
ebgp-multihop to work wouldn't that require some extra static routes to be
setup by my transit providers?
If you want to run seperate networks, run separate networks. Different
ASes, the whole 9 yards; perhaps a re-reading of rfc1930 is in order?
Back in the day, there was a promising local provider in my neck of the
woods who has an AS per state for similar business reasons. They hit no
filters, had no concerns, and when their business grew to the point of
being able to clean things up, they did. No fuss, no muss and they're
still in business with not chapter 11 in sight. Seems some folks would
rather kick and scream.
Perhaps people who weren't 'here' to work with and experience CIDR
deployment don't think there's any harm in going aginst CIDR. Perhaps it
is lack of experience in general engineering; one basic rule of thumb is
to solve problems by avoiding the conditions which create them. By rushing
headllong into activities that are -in even the most conservative terms-
"debatable", you are inviting both known and unknown affects today and
tomorrow. Using a reachbility protocol as an 'optimization' protocol for
anything other than non-local affects (standardized well-known
communities) is practice that is not guarenteed to work.
I guess the point is, there's lots of "possible" activities in IP let
alone BGP. If you presume that all which is technically possible is a
good idea, then you are the only one responsible for the outcome. If
you set yourself up for problems, especially ones that are known and
trivially researchable, then don't gripe about it. Check who you're
paying what. Google '"tragedy of the commons" internet route routing'.
And to work to actully *solve* the problem, I'd suggest participating
in PTOMAINE and the like. Rather than railing aginst current deployments,
network operators, or the price of bits in CA your energy would be
better spent nagging vendors to back drafts and to be ready to adopt new
well-known-communities.
> Why won't the rest of the world see extra hops and increased latency
> reaching my network (for the 50% of the time that the wrong transit
> provider is picked).
Because you could *gasp* be intelligent with your network design and do
things like purchase transit from the same carriers in both your serving
markets.
I guess you don't consider redundancy to be intelligent. I do. I guess
you can call me stupid.
The problem here Ralph is that you see absolutely no problem with doing
things on your network that have minimal benefit to you, yet have global
impact.
The problem here Paul is that you can't seem to see the forest for the
trees.
If you consider routing table size reduction so important, why don't you
filter all /24's? That will give you much more benefit than /20's that
have been de-aggregated.
If you want to run seperate networks, run separate networks. Different
ASes, the whole 9 yards; perhaps a re-reading of rfc1930 is in order?
That brings us back to the discussion of PI space. If de-aggregating my
/20 didn't work, then I'd either inefficiently use IP space in order to
qualify for 2 /20's, or buy a defunct ISP or 2 to get a bunch of /24's in
the 192-223 space.
Are you suggesting that either of those (which don't violate any
RFCs) options are better than de-aggregating my /20?
The best solution is just as everybody here has suggested. Use the same
provider for transit at both locations, announce your /20 normally, and
your more specifics with no-export.
Your response was something about "I guess you don't consider redundancy
to be intelligent." What's stopping you from using the same two transit
providers in both locations? Seems to me you don't value redundancy all
that much. If you're willing to route through your small intra-AS link in
case of local transit failure, yet you're not willing to route through it
under normal circumstances, that would indicate to me that the link is not
big enough for its purpose.
I mean, at the end of the day, the argument boils down to you not wanting
to foot the bill for 1) a fatter intra-AS link or 2) multiple transit
providers at each location.
It's ok if you want a bandaid, just don't try to tell anybody that your
bandaid is actually a solid, best-practice solution.
I think you're missing the point a bit. Don't expecy to use resources on
other people's networks and routers to do your own traffic engineering
unless you pay them for it.
You must buy transit from the same ISP in each city, and then you can do
your traffic engineering using their resources (i.e. send longer prefixes
to them that are no-export). In essence you are using that transit ISP as
*your* backbone. You could buy transit from teh same two providers in each
city as has been suggested as well.
If you don't want to do this then you must buy a private link between the
cities and run your own backbone so that you can distribute the traffic
inside your own AS. Learn how to use AS PATH PREPEND if you purchase equal
sized pipes from unequal sized upstreams (i.e. BIG_GLOBAL_ISP and a
regional ISP).
If you don't like this then buy an AS for each city and run them as their
own network as Joe suggested you could do. I don't like this solution as
it does not really conserve routing annoucmenents, as you'll still be
pushing out multiple longer prefixes (one /23 for each city), which will
still be filtered by Verio and others, and you are eating up AS numbers
which are scarce. I suppose if you could justify a /19 to /21 in each city
this would be less of a problem.
Those are your options ... essentially you need to
> > If you want to run seperate networks, run separate networks. Different
> > ASes, the whole 9 yards; perhaps a re-reading of rfc1930 is in order?
>
> That brings us back to the discussion of PI space. If de-aggregating my
> /20 didn't work, then I'd either inefficiently use IP space in order to
> qualify for 2 /20's, or buy a defunct ISP or 2 to get a bunch of /24's in
> the 192-223 space.
>
> Are you suggesting that either of those (which don't violate any
> RFCs) options are better than de-aggregating my /20?
Your response was something about "I guess you don't consider redundancy
to be intelligent." What's stopping you from using the same two transit
providers in both locations? Seems to me you don't value redundancy all
that much.
I'm currently using Peer1 in Toronto for transit and they don't have a POP
in Ottawa.
Having 2 different transit providers in both Ottawa and Toronto has only a
marginal improvement in redundancy vs provider A in Ottawa and provider B
in Toronto. Even if I could use provider A in both Ottawa and Toronto I
wouldn't due to the reduced redundancy.
And your assumption about my Ottawa-Toronto link is wrong. I have a 100M
point-to-point ethernet link between the cities. I have a 100M transit
connection to Peer1 in Toronto, and have issued a letter of intent to a
transit provider in Ottawa for a 100M link.
> > Because you could *gasp* be intelligent with your network design and do
> > things like purchase transit from the same carriers in both your serving
> > markets.
>
> I guess you don't consider redundancy to be intelligent. I do. I guess
> you can call me stupid.
Carriers is a plural word.. How does that not accomplish redundancy again?
As I pointed out in my last post, I can't. And even if I could the
economics of doing it don't make sense.
If economics don't matter, then the most intelligent network design would
be a redundant OC192 mesh, but I don't know even one network that does
that.
Ralph, if you have a 100m link between the two cities, why don't you use
conditional announcements to only announce your /20 though Ottawa if your
primary transit in Toronto goes down? Then, you only need to announce your
/20 in Toronto, no need to deaggregate, and the whole issue is solved.
Then, when you have the Ottawa 100m transit link up, you can announce your
/20 to both transit providers all the time.
Ralph, I will give you some swamp space for your /24s if you never post
about your 2621 powered OC192 mesh network-to-be again.
If you want a solution that doesn't cost you any money, either a) get more
IP space and carve up stuff into /21s, or b) go browsing through the list
of allocated but unused swamp space and start announcing it.
Otherwise, you will be filtered, no doubt gleefully from the members of
this list. This is not the end of the world, they don't need your more
specifics for the traffic to reach you. As long as your transit providers
accept your more specifics from each other, announce the aggregate to
whichever is primary, and more spcifics to other providers in other
locations. The worst that can happen is you go Filterer->Transit1->Transit2
for some limited subset of the internet (but hey it doesn't cost you
anything).
Your economic problems are your own, if you were smart you would learn how
to solve them within the rules of the game.
Your economic problems are your own, if you were smart you would learn how
to solve them within the rules of the game.
I know how to solve the problem within the "rules". Getting a dozen /24s
in the swamp would solve the problem, but would pollute the global routing
table more than de-aggregating my /20 into /21s and /22s.
It seems to me that there's a few "experts" on the list that have their
heads in the sand. Not only did I quickly find someone who helped get
Verio's filters updated, but I had several other emails from people asking
for help doing the same.
I have yet to hear anyone suggest any better technical solution, so I
guess I'll just lurk on this thread and listen to the folks in their ivory
towers talk about how the perfect world should be...
That is roughly the intention. I also have to be able to announce the
more specifics for when the Ottawa-Toronto link goes down. You could find
in the archives my posts from a couple months back asking how to do this.