My apologies if this question doesn't belong here.
We have a PI /24 we'd like to advertise out of our primary data center
for production use. (Well, actually, we'll be advertising a more
specific from our /21 assignment, so already not too friendly... but I
digress.)
We have a disaster recovery site which will have a clone of the myriad
production servers. We'd like to fail over to that site
automagically.
I'm thinking advertising the same prefix and just doing several
as-prepends. However, now I'm not sure if this is a polite thing to
do or not.
Someone mentioned to me something with MEDs, but as soon as that term
was used, I started twitching, and couldn't follow the conversation.
Would a "good netizen" use the as-prepend method? Or am I missing a
simpler/more polite solution?
-Christopher
We have a PI /24 we'd like to advertise out of our primary data center
for production use. (Well, actually, we'll be advertising a more
specific from our /21 assignment, so already not too friendly... but I
digress.)
[...]
I'm thinking advertising the same prefix and just doing several
as-prepends. However, now I'm not sure if this is a polite thing to
do or not.
How about advertising all of your PI from all of your perimeter routers, and configuring internal routing to ensure packets get to the rightful live/preferred data centre. Then, in the event of a failover situation, you can change the routing behaviour so that packets for the production /24 end up in the alternative dc.
To work, this does imply you have good connectivity between your live and failover environments (but i guess you have this so that you can copy over state/live data between your two environments anyway..), so that you can carry customer/live data over the link too.
Double plus good for you if you can get cheap transit delivered to your failover dc - you could use this for your preferred environment too !
cheers
-a
* Christopher J. Pilkington:
We have a disaster recovery site which will have a clone of the myriad
production servers. We'd like to fail over to that site
automagically.
I'm thinking advertising the same prefix and just doing several
as-prepends. However, now I'm not sure if this is a polite thing to
do or not.
Can your backup servers run in parallel to your primary servers? How
do you handle conflicting updates (assuming that the services are not
completely stateless)?
Hello.
Answering the original question, as prepends are not a definitive way to force all traffic to one site or the other.
Two ways that definitely will work are.
1. Send a community to the upstream provider at the backup site that causes them to set a lower plocalpref on the route you send them vs. the route learned from the internet.
2. If you have a large enough ip block, Advertise an aggregate address from your backup site, and more specifics from your primary. I.e. advertise a /23 from your backup site and the same /23 as two /24's from the primary.
-ejay
Thu, 16 Feb 2006 16:10:02 +0100
Part of the question is how bad it is for you if you DO get any traffic to your backup datacenter, the connectivity between the datacenters and the datacenters connectivity to the rest of the world.
Assuming that you do not have good connectivity between datacenters and that the datacenters have different connectivity to the outside world:
While pre-pending should get almost all of your traffic away from your backup DC, you cannot guarantee that it will not get any traffic while the primary is still up.
If your primary is connected to ISP_A and the backup is connected to ISP_B, customers connected to ISP_B MAY still flow to your backup DC (ISP_B will probably set local preference on all customer routes - you should be able to override this behavior with communities but not all providers support this (or honor it 100% of the time!))
Announcing a more specific from the primary is likely to work basically all the time (assuming a) your announcement is not too long to be listened to, b) ISP_A and ISP_B don't lose connectivity between themselves). This is not particularly polite however...
Another option is just not to announce the backup datacenter until the primary one goes away - see if you can do something like BGP Conditional Advertisement (or your vendor's version of the same).
Depending on just how bad having request arrive at the backup datacenter will drive just how paranoid you ned to be - if having your backup get traffic is going to make databases unhappy, etc then you MIGHT even want to consider a manual only failover - if your primary datacenter has a 20 second blip, the pain of dealing with requests that hit the backup during those 20 seconds MAY be greater than just being unavailable for 20 seconds... It all depends on your business, applications, etc, but prepending alone might not be the way to go.
Warren
And in addition to that, even multihomed customers of ISP_B may choose the
prepended route for a number of different reasons; for instance, ISP_B might
be a cheaper pipe for them, or there may be a smart-ish routing device or
scheme in play that overrides normal BGP decision making.
I might be crazy, but couldn't you just prepend the route enough to
effectively poison it at ingress to 'backup-isp' ? so they kept chosing
the remote path and never really accept the route from local until the
remote path is gone?
Some route decision override schemes don't care what the path length is at
all, or factor it in with such a low weight, such that no reasonable amount
of prepending will change the situation.
With the development of source traffic engineering schemes, prepending is no
longer reliable as a means of affecting routing on the remote side. That
perception will have to die with it (hopefully sooner rather than later).
> I might be crazy, but couldn't you just prepend the route enough to
> effectively poison it at ingress to 'backup-isp' ?
Some route decision override schemes don't care what the path length is at
all, or factor it in with such a low weight, such that no reasonable amount
of prepending will change the situation.
And while I actually misunderstood what you said, this is still partly
correct -- ISP_B will often put a preference onto their own route at a level
that outweighs path length absolutely.
If ISP_B has a "don't prefer" community, that could work for ISP_B's own
customers, but short of removing the advertisement from ISP_B's peers too,
that doesn't always move traffic completely off of the ISP_B pipe. This is
because traffic entering ISP_B destined for the network *will* usually
prefer ISP_B's link in spite of the community, thanks to ISP_B not desiring
to transit packets in and out of their network without a revenue point
somewhere in the path.
Not really - horrendous ASCII art below:
Customer
/ \
/ \
ISP_A ---------ISP_B
\ /
\ /
DC1 DC2
Assuming DC is AS_65530, ISP_A is AS_655301 ISP_B is AS_655302 and DC_2 prepends 5 (or some other "large" number) times:
Under "normal" conditions:
ISP_A sees:
192.0.2.0/24 -- 65530 i (direct from DC1)
ISP_B sees
192.0.2.0/24 -- 65530 65530 65530 65530 65530 i (direct from DC2)
192.0.2.0/24 -- 65531 65530 i (ISP_A -> DC_1) <= Best due to AS_PATH
Customer sees:
192.0.2.0/24 -- 65531 65530 i (ISP_A -> DC1) <=Best due to AS_PATH
192.0.2.0/24 -- 65532 65531 65530 i (ISP_B -> ISP_A -> DC1)
If ISP_B sets Local-Pref on customer routers:
ISP_A sees:
192.0.2.0/24 -- 65530 i (direct from DC1)
ISP_B sees:
192.0.2.0/24 -- 65530 65530 65530 65530 65530 i (direct from DC2) <- Best due to Local-Pref
192.0.2.0/24 -- 65531 65530 i (ISP_A -> DC_1)
Customer sees:
192.0.2.0/24 -- 65532 65530 65530 65530 65530 65530 i (ISP_B -> DC2)
192.0.2.0/24 -- 65531 65530 i (ISP_A -> DC_1) <- Best due to AS_PATH
This means that any traffic that enters ISP_B (eg: Customer is singly homed to ISP_B, their connection to ISP_A goes down or they adjust local_pref to prefer ISP_B) will go to DC2.
The problem is that Local-Pref trumps basically all other conditions in the BGP decision process - if ISP_B adjusts it it will be prefered in their network no matter how many times you prepend.
Warren
see, I knew I was crazy 
where is that halabi book when I needed it. Did
the original poster ever say if he/she was able to accept
incidental traffic to the backup DC ?