On our SBC peering links we lost connectivty to our Canadian customers for about 20 minutes. I have escalated this up through SBC but was wondering if anyone on the list as any knowledge of what may have caused the outage. Our Canadian customers come in through different Canadian providers so from my perspective it wasn't just one AS or prefix that was lost it was many.
Yes its part of a new homeland security policy of discrimination against
our moose loving brothers to the north. All packets will now be stopped at
the border and inspected by DHS.
Actually they just leaked some routes this morning, and blew out max
prefixes to most peers. Pretty typical as far as accidental leaks go,
those folks with sensible filtering and running routers which trip on max
prefixes accepted not prefixes received probably weren't even affected.
Another day on the internet.
Not sure what you're referencing there. If Randy said something to the
effect of "all peers will screw up and leak something to you eventually,
no matter how large or generally well run, so plan accordingly", I would
agree with it though.
I was suggesting that if even if you don't have the ability to fully
prefix or as-path filter every peer (which, face it, most of us with large
numbers of interesting peers don't have), you can still filter the really
obvious stuff and prevent a large amount of the impact from common fat
finger events. I can't tell you the number of problems that are prevented
by applying a simple set of as-path filters matching large networks you
know you should never hear from your peers (or most of your peers).
Something simple like:
Catches a huge amount of "stupid stuff", especially when the event isn't a
full blown leak which trips max prefixes, but is an isolated set of
prefixes leaked by someone not directly connected to you. On a Cisco, the
leaked 7018 routes from 7132 this morning would have been gone splash
harmlessly with such a filter, and sessions wouldn't even trip max
prefixes and bounce.
Unfortunately Juniper has a bit of a backwards take on the use of prefix
limits. They believe that the reason to have a prefix limit is to protect
a router against memory exhaustion (for example, someone sending you a few
million routes that you are rejecting filling up your adj rib in), rather
than as a policy tool (shutting down a wildly broken session that is
sending you stuff it shouldn't). Juniper applies the max prefixs to routes
received from a session rather than routes accepted, so even if the routes
are filtered the prefix limit will still trip and shut down the session.
This is particularly bad if for example you have a customer session which
is fully prefix list filtered, and the customer accidentally leaks you a
In reality, both types of limits are probabaly a "good idea". I've talked
to a lot of people from a lot of companies who say they have requested
Juniper add a seperate accepted-prefixes limit (or more likely, convert
the existing limits to accepted-prefixes, and add a new received-prefixes
knob), but so far it hasn't been implemented. If you are a Juniper
customer who is reading this and you think having prefix limits which only
count accepted prefixes is a good idea, please use this as an opportunity
to submit an enhancement request so we can get this behavior improved.
Feel free to throw in a request for "outbound prefix limits" as a last
ditch safety net too.