AS PATH limits

If you're on cogent, since 22:30 UTC yesterday or so this has been happening
(or happened).

*> 186.177.184.0/23 38.*.*.* 45050 0 174 262206 262206 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 ?

oddly, i see other pops with 174 sources giving a more sane route (even 6939
is giving us a route that goes thru 174 after 2 hops). 'Sup, 174?

Wonder if this is just stuck in the router Im looking at and the update
process is failing because the route is too long to process properly for
removal or something. mmm, bugs!)

/kc

If you're on cogent, since 22:30 UTC yesterday or so this has been happening
(or happened).

Still happening here. I count 562 prepends (563 * 262197) in the
advertisement we receive from Cogent. I see no good reason why we
should accept that many prepends.

Steinar Haug, Nethelp consulting, sthaug@nethelp.no

Maybe the next best path had, had 562 prepends? :slight_smile:

I dont see that as the solution. Someone else will offend again.

However, I also don't see trusting major backbones as our filters (for many
other reasons). Our software should be handling what's effectively a buffer overflow
or equivalent (beware long paths that are actually shellcode).

Quagga among others seems to be subject to this bug, pre 0.99.23 or so
(.99.24+ seems ok). So upgrading is a solution.

There was also some chatter on the quagga mailing list on how it's more
pleasant to stab your eyeballs out rather than constructing extremely long
regexp's that might work as a filter.

https://lists.quagga.net/pipermail/quagga-users/2017-September/thread.html

/kc

I dont see that as the solution. Someone else will offend again.

However, I also don't see trusting major backbones as our filters (for many
other reasons). Our software should be handling what's effectively a
buffer overflow
or equivalent (beware long paths that are actually shellcode).

Quagga among others seems to be subject to this bug, pre 0.99.23 or so
(.99.24+ seems ok). So upgrading is a solution.

ii quagga 0.99.22.4-3ubu i386 BGP/OSPF/RIP routing
daemon

interestingly enough that isn't crashlooping nor is it bouncing bgp
sessions:
192.168.100.100 4 MYASN 1642717 8864 0 0 0 2d23h32m
672475

and it's happily showing me the route even...

There was also some chatter on the quagga mailing list on how it's more

I don't quite understand the exact situation that causes the issue - our
cogent facing router (quagga .99.22 debian) was receiving the route but that
session stayed up - it was it while sending or the other igp router (also
quagga .99.22) receiving (I think the latter) that was crashing their session.
Not quite sure why the cogent session didn't crash as well (or first, before
propagating the bad route).

At any rate, we should likely take this discussion to the quagga-users-l

/kc

I dont see that as the solution. Someone else will offend again.

However, I also don't see trusting major backbones as our filters (for many
other reasons). Our software should be handling what's effectively a
buffer overflow
or equivalent (beware long paths that are actually shellcode).

Quagga among others seems to be subject to this bug, pre 0.99.23 or so
(.99.24+ seems ok). So upgrading is a solution.

ii quagga 0.99.22.4-3ubu i386 BGP/OSPF/RIP routing
daemon

interestingly enough that isn't crashlooping nor is it bouncing bgp
sessions:

Quagga 0.99.11 and earlier affected.
Fixed in 2009.

-Hank

It was fixed in other OSes as well after this, I presume:

http://blog.ipspace.net/2009/02/root-cause-analysis-oversized-as-paths.html

Hi Ken,

Technically the route is not bad, just really inconsiderate.

The bug happens when quagga sends the the long-AS path route to a peer. As
I understand it, when the announcement is larger than one segment, Quagga
double-counts the some of the bytes when computing the number of bytes in
the AS path. It receives the announcement just fine, but then it corrupts
what it sends to the neighbor who then chokes.

Bug and patch here:
https://lists.quagga.net/pipermail/quagga-dev/2017-September/033284.html

Regards,
Bill Herrin

looks to me as if 262206 is trying a silly tactic to down-pref inbound
from cogent. as cogent probably prefers customers to peers, it may not
be working as 262206 expected, so they keep pounding with the same
hammer hoping for a miracle.

cogent accepts it as they are being paid to do so; normal practice.

perhaps our ire should be directed at 262206, not cogent? has anyone
tried to contact them?

randy

looks to me as if 262206 is trying a silly tactic to down-pref inbound
from cogent. as cogent probably prefers customers to peers, it may not
be working as 262206 expected, so they keep pounding with the same
hammer hoping for a miracle.

cogent accepts it as they are being paid to do so; normal practice.

perhaps our ire should be directed at 262206, not cogent? has anyone
tried to contact them?

randy

It is amazing how well the DFZ works given half the participants (*) have
no clue how. If that is what they want, all they need is to split that /23
into two /24 and only announce that on their other transit.

(*) I should probably include myself in that half.

Regards

Baldur

Its also happily announced onwards, e.g. by Telia:

Oct 2 07:25:09:E:BGP: From Peer ... received Long AS_PATH= AS_SEQ(2) 1299 174 262206 262206 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 ... attribute length (567) More than configured MAXAS-LIMIT 64

Got this reply from cogent:

"We have isolated a BGP Routing discrepancy on the Backbone. That routing has been removed
from the Network."

So apparently they agree they shouldn't just accept this bogosity. Good on em.

/kc

<TheGiant> It is happening AGAIN.

And of course it started on a friday aft 15 min before quittin' time in EDT:

Last time it was 186.177.184.0/23 0 174 262206 262206 262197 262197

*> 186.176.186.0/23 38.x.x.x 45050 0 174 262206 262206 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 ?

/kc

Has anyone tried calling them?

Kind regards,

Job

And again this morn at 08:35:19 EST (13:35 UTC). I dont have access to the
router that fed us the long route, so I cant tell what it was (since we never
consumed it before barfing).

Let's hope for no more over holiday season...

/kc

What router software version are you running that barfs on long as-paths?

Nick

Ken Chase wrote:

And again this morn at 08:35:19 EST (13:35 UTC). I dont have access to the
router that fed us the long route, so I cant tell what it was (since we never
consumed it before barfing).

Let's hope for no more over holiday season...

/kc

  ><TheGiant> It is happening AGAIN.
  >
  >And of course it started on a friday aft 15 min before quittin' time in EDT:
  >
  >Last time it was 186.177.184.0/23 0 174 262206 262206 262197 262197
  >
  >*> 186.176.186.0/23 38.x.x.x 45050 0 174 262206 262206 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197

262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262
197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197
262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 262197 26
2197 262197 262197 262197 262197 262197 262197 262197 262197 ?

quagga 0.99.22.4, yes i need to upgrade, as my other
router on 0.99.23.1 seems ok. now coordinating with
customers to get it upgraded is a different issue.

/kc

quagga 0.99.22.4, yes i need to upgrade, as my other
router on 0.99.23.1 seems ok. now coordinating with
customers to get it upgraded is a different issue.

Will that version of quagga not support a filter list?

e.g
neighbor 38.xx.yy.zz filter-list maxas-limit65 in

ip as-path access-list maxas-limit65 deny ^([{},0-9]+ ){65}
ip as-path access-list maxas-limit65 permit .*

  ---Mike

Hi Nick,

Versions of quagga up until the very most recent release corrupt the
transmission of routes with very long AS paths. They add up the packet
length wrong. The neighbors of any router brand then barf on the malformed
data and terminate the BGP session.

Your peer running quagga must either upgrade or filter long AS paths or you
will receive corrupt data and terminate the BGP session. There's nothing
that -you- can do about it.

The AS path lengths we're talking about are unreasonable. They indicate a
high probability of misconfiguration at the origin. There's no legitimate
cause for them to exist on the pubic Internet at all. It would be
reasonable to treat them like when peers offer /32 prefixes and just say no.

Regards,
Bill Herrin