I was wondering what everyone does to load balance over multiple bgp feeds. We currently have 5 bgp feeds with 2 providers. Do you just randomly pick networks, or use something like netflow to try and pick the best path.
Mike
A lot of it depends on what you're trying to balance for (cost?
performance?), the size of each link, the type of traffic you're
dealing with, and other factors. My main experience has been at a
couple of stub networks, and I don't claim to be an expert on this.
YMMV.
I've generally been able to get surprisingly accurate results in terms
of how much traffic to send out one link or another just by using
route-maps and some simple netflow analysis, plus traceroute / ping.
I don't know if Netflow will give you much information in terms of
the best path, but it can give you an idea of how much traffic you're
sending / receiving to various ASes. I've found ehnt[1] pretty helpful
for this. Even a few simple rules (all traffic to AOL goes through
X, all traffic passing through 701 goes through Y) can often get you
pretty close to where you want to be.
This is a sledgehammer / shotgun type tool... but it has worked well for
me. Of course you need to take into consideration the size of your pipes
and how the loss of connectivity to one provider may affect your rules.
And of course there are tools which claim to do this for you (for a
"small" price), Route Science, Internap's appliance (whatever it's
called), etc. I've not had any personal experience with these devices;
check the NANOG archives for plenty of opinions.
I'm not sure if there are any open source tools that do a good job of
this, but you might see the NANOG thread with the subject "Open Source
BGP Route Optimization?"[2] from May 2004 - came up from a quick google
search I did.
For content type stuff, I haven't found performance to be a huge deal
most of the time... I imagine with VOIP and other applications, it
might be more of a concern, though.
/w
[1] http://ehnt.sourceforge.net/
[2] Looks like this is the first item in the thread:
http://www.cctec.com/maillists/nanog/current/msg00719.html
[ Sorry for the self-followup. ]
> I was wondering what everyone does to load balance over multiple
> bgp feeds. We currently have 5 bgp feeds with 2 providers. Do you
> just randomly pick networks, or use something like netflow to try
> and pick the best path.
A lot of it depends on what you're trying to balance for (cost?
performance?), the size of each link, the type of traffic you're
dealing with, and other factors.
Also, my experience has been in fairly content-heavy networks. Shaping
inbound traffic can be a little more difficult (lots of information on
this online). If your provider supports communities, they may allow
you to prepend selectively to certain ASes or in certain geographical
locations -- and of course you can play around with prepending.
I've generally been able to get surprisingly accurate results in
terms of how much traffic to send out one link or another just by
using route-maps and some simple netflow analysis, plus traceroute /
ping.
Probably obvious, but if your providers support communities, this can be
extremely helpful... you can often use these to do some slightly less
kludgy traffic-shaping - "prioritize traffic learned from provider X's
customers, but not from their peers", or "prioritize provider X's peers
in our city, but de-prioritize their peers in the Bay area". In addition
to internal documentation, many providers put information on the
communities they support in the routing-registry entry for their AS.
w
Mike,
I was wondering what everyone does to load balance over multiple bgp feeds. We currently have 5 bgp feeds with 2 providers. Do you just randomly pick networks, or use something like netflow to try and pick the best path.
You might be interested by the following papers :
S. Uhlig and O. Bonaventure. Designing BGP-based outbound traffic engineering techniques for stub ASes. ACM SIGCOMM Computer Communication Review, 34(5), 2004.
S. Uhlig, O. Bonaventure, and B. Quoitin. Interdomain traffic engineering with minimal BGP configurations. In 18th International Teletraffic Congress (ITC), September 2003.
http://totem.info.ucl.ac.be/publications.html
Best regards,
Olivier