RE: Winstar says there is no TCP/BGP vulnerability

Please forgive me if I'm naive and/or ask a stupid question, but is
there any reason (besides your platform not supporting it) _not_ to MD5
your BGP sessions? Geez, on my _home_ router all my v4 BGP sessions are
MD5ed (v6 not there yet).

Michel.

There is serious operational overhead in maintaining sync'ed passwords between separate organizations. IOW: Eventually someone will screw up and lose the password. When they do, the session goes down, and probably for far longer than if some miscreant tries to RST it via the "vulnerability".

Actual data: Over the past three plus years an organization with on the order of a dozen MD5-ized BGP sessions has has multiple down sessions due to, for instance, a peer doing standard (for them) password rotation and forgetting to inform the organization. Each time incurred a minimum of several hours downtime, once stretching into several days as the peer could not figure out what was wrong and get the right person with the password to give it to the organization.

Over the past three plus years with over 1000 non-MD5-ized BGP sessions, the same organization experienced exactly *ZERO* seconds of downtime identified as due to RST-style attacks. And certainly no prolonged outages due to it.

Add to that the additional CPU overhead some people have reported, making it easier to packet the router to its knees, and MD5 looks like a cure worse than the disease.

All that said, it is your router, your peers, your decision. I would never dream of telling anyone who wanted MD5 to not do it. I just don't understand people who want to do it. Especially when they could be doing things like filtering at the leaf nodes and forcing their vendors to support the TTL hack.

But that's me.

Hi, NANOGers.

] Actual data: Over the past three plus years an organization with on the
] order of a dozen MD5-ized BGP sessions has has multiple down sessions
] due to, for instance, a peer doing standard (for them) password
] rotation and forgetting to inform the organization.

Yep, that's a problem - a PROCESS problem. The definition of insanity
is repeating the same behavior over and over and expecting a different
result. :wink:

Saying that we've not seen any RST attacks may be correct, but it's
not a predictor of future activity. No one can do more than model
what may come next. Prior to February 2001 no one had seen massive
DDoS either. Anyone still think DDoS isn't a problem?

We manage well over 150 peering sessions with MD5 passwords in place.
This includes bogon peering, route-server peering, and production
traffic peering. This has grown over the past three years. The total
number of MD5-related outages: zero.

In other words, your mileage may vary. :slight_smile:

Test any feature. Think about how to manage that feature, both in the
deployment stage and in steady-state. I don't advocate the use of any
feature, be it MD5, MPLS, et al. without careful consideration of the
support ramifications of it.

Thanks,
Rob.

there is the issue of changing the keys during operations without
impacting the network, eh? Having to bounce every bgp session in your
network can be pretty darned painful... if you change the key(s) of
course. If you don't you might as well not have keys, since adding the
3 lines of C code required to Paul Watsons' program making it do
the hashing certainly won't be a big deal, eh?

] Actual data: Over the past three plus years an organization with on the
] order of a dozen MD5-ized BGP sessions has has multiple down sessions
] due to, for instance, a peer doing standard (for them) password
] rotation and forgetting to inform the organization.

Yep, that's a problem - a PROCESS problem. The definition of insanity
is repeating the same behavior over and over and expecting a different
result. :wink:

So you think continuing to use MD5 and not see down sessions due to lost passwords would be insane?

Also, PROCESSES are part of running a network. As any large network provider, they will tell you the process of keeping the network up costs more than the fiber the network runs on. In fact, operational support and complexity is one of the main reasons large networks give for not peering with other networks.

Saying that we've not seen any RST attacks may be correct, but it's
not a predictor of future activity. No one can do more than model
what may come next. Prior to February 2001 no one had seen massive
DDoS either. Anyone still think DDoS isn't a problem?

Of course not. But I also see a reason to do DDoS attacks.

Rob, you are the biggest proponent of the underground economy I have ever seen. Tell me why a miscreant would waste his time - at *least* 30 minutes (lowest report I've seen, and possibly hours or days) - trying to reset a BGP session when with less effort they could take down an entire router?

I can come up with reasons, but they are pretty few and far between, especially compared with reasons to take out a whole router.

Can you tell me why these miscreants - people who are lazy just like us - would not rather packet the router off the 'Net than do this slow, possibly useless exercise?

We manage well over 150 peering sessions with MD5 passwords in place.
This includes bogon peering, route-server peering, and production
traffic peering. This has grown over the past three years. The total
number of MD5-related outages: zero.

In other words, your mileage may vary. :slight_smile:

Awesome news. And your milage *will* vary, since you cannot control separate organizations.

That isn't the point of my post. Whether or not you think X is a good
idea, having someone technical say "we don't support X currently" does not
mean a host of other things like "we think X is a bad idea" or any other
nonsense like that.

"Christopher L. Morrow" <christopher.morrow@mci.com> writes:

there is the issue of changing the keys during operations without
impacting the network, eh? Having to bounce every bgp session in your
network can be pretty darned painful... if you change the key(s) of
course. If you don't you might as well not have keys, since adding the
3 lines of C code required to Paul Watsons' program making it do
the hashing certainly won't be a big deal, eh?

I've added keys without bouncing the sessions... doesn't seem to
cause any difficulties at all. You just add the password clause on
both ends within the window for a BGP keepalive timeout. Worst case,
this line:

   Milwaukee#sho ip bgp neigh 203.176.61.22 | inc md5
   Flags: passive open, nagle, gen tcbs, md5
   Milwaukee#

is lying, and the md5 won't actually come up until some nogoodnik or
bad fortune causes the session to bounce. 12.0S.

                                        ---Rob

yup, this is the expected behaviour at a certain level of code... I don't
recall that level but I'm sure a cisco person could give us the rundown :slight_smile:
Basically, as I understand the explanation given to me, there is no
defined manner to deal with:
1) changing keys
2) adding keys
3) removing keys

in the RFC for this (2385). So, atleast Cisco, implemented key change in a
sane manner, if you change the key packets immediately start heading out with
the new key used as the hash key. The far side starts logging 'bad key'
messages but the session doesn't reset and updates keep attempting to be
sent, until you either change the key, or the session timeouts hit.

For adding keys, I have experienced the following:
Apr 21 17:01:45.278: %BGP-5-ADJCHANGE: neighbor <ip> Down -
Password change

on 12.0S and 12.1(19)(non-s) code trains...

on passwd change:

Apr 21 17:03:36.117 GMT: %BGP-5-ADJCHANGE: neighbor <ip> Down
Password change

So, this is obviously suboptimal. The good news is code releases up the
chain seem to have this fixed, getting there is a chore, but will make MD5
more operationally managable in the long term, and thus more widely
deployed, eh? (hopefully)

May be, it is reasonable to have a simple MD-5 key - I mean, without a
rotation, use e-mail to exchange it instead of the phone,
do not generate but use simple password, and so on. If this key is never
changed, then risk to lost a session is very low, and I do not see _any_
reason to keep it on rotation plan (hacker must know too much and can damage
too little, in this case).

Even such keys as '415' or 'monday' will prevent TCP attacks in alll cases -
if single attack require 5 - 30 minutes for the one hit, then no any way
exists to use dictionary 'guess' for password cracking.

Now, we can see a _histeria_ around this problem; but yes, when it will coll
down (1 - 2 weeks), it will be a time to make a reasonable improvements.

Date: Tue, 20 Apr 2004 23:11:28 -0500 (CDT)
From: Rob Thomas

We manage well over 150 peering sessions with MD5 passwords
in place. This includes bogon peering, route-server peering,

CYMRU bogon (et al.) route servers are an example of where MD5 or
IPSec definitely is a good idea. However, most peer/peer and
carrier/downstream BGP sessions aren't multihop spanning a
network or three.

Of course, if ingress SAV were universal...

Eddy