RE: Second day of rolling blackouts starts

Not that I know of. There is something squirrely going on with the
root name servers, but I haven't figured it out if it is just my location
or more widespread.

I have noticed east coast routers/providers are getting
beat up fairly rough. I am having hard times getting my
clients in Europe to see me here in San Diego. Level3
and PSInet are taking a beating hard.

Is your network multicast enabled ? My traceroute to you shows
that you home to AS 2548, which is.

If so, this might be connected to the RAMEN worm. This is hosing up
native multicast
but good, so much so that it is affecting routers and causing some
unicast problems. I heard, for
example, that it is causing 4% packet loss at the Abilene NOC. RAMEN is
(for the multicast enabled
part of the Internet) effectively a DOS attack.

Regards
Marshall Eubanks

James Harkins wrote:

I'm betting on it being blackout related:

Starts getting ugly between Dallas and LA and just goes down hill from
there:

  9 dfw3-core4-pos7-0.atlas.icix.net (165.117.48.134) [AS 2548] 36 msec 36 msec 36 msec
10 lax2-core2-pos7-0.atlas.icix.net (165.117.48.42) [AS 2548] 68 msec 64 msec 68 msec

Two people have asked me off list about the RAMEN worm,
which affects Linux Redhat distro's. Here is brief description of the
worm, and a link to more,
from Lucy Lynch at Internet2 / UOregon.

The multicast implications :

This worm scans a portion of the multicast address space. These scans (packets)
are viewed as new multicast sources by a PIM multicast enabled router,
which encapsulates
them and sends them to its RP. The RP creates MSDP Session Announcements
FOR EACH SCAN
and floods them to every RP neighbor it has in "nearby" AS's, and those
repeat the process.
The result is a MSDP packet storm. We have gotten 15,000 SA's a minute.
Dealing with these
can melt down routers. (We had to reboot a Cisco 7204, for example,
which apparently either filled
up or fragmented its memory beyond usability.)

I think it is fair to say that the question of rate limiting and other
DOS filtering in
PIM/SSM/MSDP multicast is getting serious attention now.

Marshall Eubanks

"Lucy E. Lynch" wrote:

Two people have asked me off list about the RAMEN worm,
which affects Linux Redhat distro's. Here is brief description of the
worm, and a link to more,
from Lucy Lynch at Internet2 / UOregon.

The multicast implications :

This worm scans a portion of the multicast address space. These scans (packets)
are viewed as new multicast sources by a PIM multicast enabled router,
which encapsulates
them and sends them to its RP. The RP creates MSDP Session Announcements
FOR EACH SCAN
and floods them to every RP neighbor it has in "nearby" AS's, and those
repeat the process.
The result is a MSDP packet storm. We have gotten 15,000 SA's a minute.
Dealing with these
can melt down routers. (We had to reboot a Cisco 7204, for example,
which apparently either filled
up or fragmented its memory beyond usability.)

I think it is fair to say that the question of rate limiting and other
DOS filtering in
PIM/SSM/MSDP multicast is getting serious attention now.

I have installed on my multicast tunnels (one to StarTap and the other to Dante/Quantum):

rate-limit input access-group 180 128000 30000 30000 conform-action transmit exceed-action drop
!
access-list 180 permit tcp any any eq 639
access-list 180 permit udp any any eq 639
access-list 180 deny ip any any

IANA has MSDP listed as port 639 - tcp+udp. It appears MSDP is only really TCP:
mcast#sho access-l 180
Extended IP access list 180
     permit tcp any any eq 639 (1555 matches)
     permit udp any any eq 639
     deny ip any any (37888 matches)

and

mcast#sho in rate
Tunnel1 Mbone tunnel to Dante
   Input
     matches: access-group 180
       params: 128000 bps, 30000 limit, 30000 extended limit
       conformed 755 packets, 60044 bytes; action: transmit
       exceeded 0 packets, 0 bytes; action: drop
       last packet: 388ms ago, current burst: 0 bytes
       last cleared 00:07:26 ago, conformed 1000 bps, exceeded 0 bps
Tunnel2 Mbone tunnel to Startap
   Input
     matches: access-group 180
       params: 128000 bps, 30000 limit, 30000 extended limit
       conformed 909 packets, 148937 bytes; action: transmit
       exceeded 0 packets, 0 bytes; action: drop
       last packet: 1048ms ago, current burst: 0 bytes
       last cleared 00:08:48 ago, conformed 2000 bps, exceeded 0 bps

I'll only know tomorrow if I stop getting the constant:
Jan 21 14:00:03: %SYS-3-CPUHOG: Task ran for 6300 msec (123/75), process = MSDP Process, PC = 60790390. -Traceback= 60790398 604146B4 604146A0
error messages. I don't know whether 128kb/sec of MSDP is too much or too little.

-Hank

Cisco is scheduled to have a patch for IOS 12.0(15)S that will
all you to limit the number of SA's received from a peer (similar to
prefix-limit on bgp session) from what I understand.

  You should talk to your cisco reps about that ability in
your software.

  - Jared