Multiple providers are seeing this right now. I assume someone is
advertising an extremely long AS_PATH again?
anyone else seeing this?
Adam
We are seeing the same thing, has anyone found the offending AS yet?
Thanks
ERIC
Throw your coffee at them!
Just my two pence 
...James
-----BEGIN GEEK CODE BLOCK-----
Version: 3.1
GIT/MU/U dpu s: a--> C++>$ U+> L++> B-> P+> E?> W+++>$ N K W++ O M++>$ V-
PS+++ PE++ Y+ PGP t 5 X+ R- tv+ b+> DI D+++ G+ e(+++++) h--(++) r++ z++
------END GEEK CODE BLOCK------
Yep, we started seeing this right around 12:20pm MST. We saw it from a
customer's rapidly-flapping BGP peer. We told them to configure bgp
maxas-limit, but apparently CRS1s don't have that command.
Anybody have a handy route-map that will deny anything with a as-path
longer than say 15-20?
Cheers,
Randal
Been a while since I had to throw this on cisco, but I since it lacks
sane repeat constraint, you have to either choose to iterate over your
acceptable space or deny on the longer-than-acceptable. For the latter,
^[0-9]+_[0-9]+_[0-9]+_[0-9]+_[0-9]+_[0-9]+_[0-9]+_[0-9]+_[0-9]+_[0-9]+_[0-9]+_[0-9]+_[0-9]+_[0-9]+_([0-9]+_)+
clobbers 15 ASNs and longer.
With the help from our transit providers and Cisco TAC the issues looks to be that AS9354 is sending AS0 and causing the corruption when processed in our Cisco CRS routers.
AS9354 shows to be Community Network Center Inc. (CNCI) or TDNC and directly connected to KDDI AS2516.
If anyone from AS9354 is on this list please contact me or stop this advertisement or someone from KDDI please assist.
Thanks
ERIC
Is there some significant barrier to people getting recent code on the devices that is not impacted by this and the other fun bgp 'attacks' that can happen? We usually see customers drop bgp sessions all over, making me wonder ... if you're not able to upgrade, what is the issue? Just that most people don't see these as an attack against their infrastructure? That people are unwilling to upgrade code unless it has a long-term impact to their operations? An outage once every few months is OK?
- Jared
In a word: YES.
Any respectable ISP will not load code that has not been extensively tested. Failure to do so can, and WILL, lead to even greater impact outages. (we've all made that mistake. Once.) Unless you do millions with Cisco and can therefore get custom IOS builds, you won't get a newer version with *just* the intended bug fixed. Their maint "rebuilds" end up with multiple "fixes" and all too often, previous fixes reverted. (I stopped counting the number of times I had to bitch at them to refix the SNMP DLCI interface counters on the 7401... "we don't test frame relay on the 7401" -- sure, that's eons ago, but nothing has changed over there.)
And then there's the question of support... again, any respectable ISP maintains maint contracts with their vendors. But, things tend to fall through the cracks... contracts expire, people forget to list all the equipment, vendors drop support for various hardware and software, etc.
You've obviously not gone to Cisco for any "non-contract" software updates. It's faster to bribe someone with an active service contract or use google.
Also... Never underestimate the power of Lazy!
--Ricky
Just an observation on how things have changed in ~15 years:
I recall Cisco code bugs that were fixed in semi- real-time, and quotes
from tli: "Code still warm from compiler. Confidence level: Boots in lab."
- - ferg
Confidence level: Boots in lab."
One could argue that certain things haven't actually changed that much ;-).
Marko.
IETF Dallas, 1995 I think. MCI Reston engg and Cisco (Ravi and others) in the terminal room cutting a new image of IS-IS for us because we'd tripped on the 24-day timer bug wherein all adjacencies would drop. Loaded on production routers that evening to fix the problem... good times
-b
Anybody have a handy route-map that will deny anything with a
as-path longer than say 15-20?
http://wiki.nil.com/Filter_excessively_prepended_BGP_paths
Ivan
Ivan-
Thanks for posting this how-to on excessive as prepends. I have a couple of questions that some of the less BGP savvy out their may find helpfull
1. In my enviornment, we are not doing full routes. We have partial routes from AS209 and then fail to AS7263. Is their any advantage for someone like me to do this, as we are not providing any IP transit so we are not passing the route table to anyone else?
2. When I run the "sh ip bgp quote-regexp "_([0-9]+)_\1_\1_\1_\1_ \1_" | begin Network" I am seeing many many ASes that would be filtered by this access-list. What happens to those networks, are they unreachable from my AS, or do I just route those networks to my upstream provider and let them deal with it?
3. This last question is a little OT, but relates to your access list
In the event of some kind if DOS attack coming from one of a few AS numbers (in this case we will use 14793), what is the feesability of using
ip as-path access-list 100 deny _([0-9]+)_\1_\1_\1_\1_
ip as-path access-list 100 deny 14793
ip as-path access-list 100 permit .*
Would this have any affect at all, or would my pipe from my upstream still be congested with garbage traffic ?
Thanks
Dylan Ebner
Ivan-
Thanks for posting this how-to on excessive as prepends. I
have a couple of questions that some of the less BGP savvy
out their may find helpfull1. In my enviornment, we are not doing full routes. We have
partial routes from AS209 and then fail to AS7263. Is their
any advantage for someone like me to do this, as we are not
providing any IP transit so we are not passing the route
table to anyone else?
You could do it to protect your BGP table, but as you're not a transit AS,
it does not make much sense.
2. When I run the "sh ip bgp quote-regexp
"_([0-9]+)_\1_\1_\1_\1_ \1_" | begin Network" I am seeing
many many ASes that would be filtered by this access-list.
Obviously a lot of people are prepend-happy.
What happens to those networks, are they unreachable from my
AS, or do I just route those networks to my upstream provider
and let them deal with it?
If I understood correctly, you're using a default route toward AS7263, which
means that anything that is not in your BGP table (and consequently your IP
routing table) will be sent out of your AS via the default route. If you're
getting the paths you're filtering from AS209 that means that more traffic
will go to AS7263.
3. This last question is a little OT, but relates to your access list
In the event of some kind if DOS attack coming from one of
a few AS numbers (in this case we will use 14793), what is
the feesability of using
ip as-path access-list 100 deny _([0-9]+)_\1_\1_\1_\1_
ip as-path access-list 100 deny 14793
ip as-path access-list 100 permit .*Would this have any affect at all, or would my pipe from my
upstream still be congested with garbage traffic ?
No. You cannot influence the inbound traffic apart from not advertising some
of your prefixes to some of your neighbors or giving them hints with BGP
communities or AS-path prepending. Whatever you do with BGP on your routers
influences only the paths the outbound traffic is taking. What you'd
actually need is remote-triggered black hole. Search the Nanog archives for
RTBH, you'll find a number of links in a message from Frank Bulk sent a few
days ago.
Hope this helps
Ivan
Ivan-
This helps vey much.
Thanks
Dylan Ebner
It will still be a while before we see unbroken 4byte AS behavior
(that whole 'fix the teardown on a anyone sneezing' problem). But
like with stale bogon filters, I expect folks inclined to use this
to drop it in and forget about it. So it would be wise to adjust
the recommended filter to anticipate a 2byteAS view allowing multiple
instances of AS-TRANS; there's likely a more elegant approach, but
the quick step of explicitly allowing _(23465_)+ before you deny
_([0-9]+)_\1_\1_\1_\1_
Cheers,
Joe
Or, you can prepend your advertisement with the troublesome ASN.
Works for one or two troublesome ASNs as a quick hack at 3am - don't do it unless you understand why it works and why you shouldn't do it.
Dear Colleagues,
Sorry for the late response.
The problem was due to faulty firmware on one of our Alaxala routers.
We resolved the problem the same day (Aug. 18) by downgrading firmware.
For more details, please see Alaxala page here (English):
http://www.alaxala.com/en/information/20090827.html
If you have filters etc. blocking AS9354, please remove them.
Thank you,
Nick Boyadjiev
AS9354 Nagoya JAPAN