Monumentous task of making a list of all DDoS Zombies.

Is there a list maintained anywhere of all hosts that have been identified as a DDoS zombie? Or attack box? We got hit with an attack from more than 60 IPs last night and I’d like to add them to any list that anyone has started.

Thanks,

-Drew

You probably want to make a list of vulnerable hosts that fall to exploits like this:
http://server-ip-here/scripts/…/…/winnt/system32/ping.exe

Most DDoS zombies will use spoofed IP packets to attack its victim, so filtering the source will not relief your pain.

Rubens

This would essentially be impossible and not a good idea. Large volumes of hosts/zombies involved in such attacks originate from residential cable/dsl subscribers. This user base primarily uses dynamically assigned IP space. Hence, the IP of tonight’s attacker could be the IP of tomorrow’s legitimate user.

This is the same reason that it is imperative that any complaints sent to ISPs providing such services MUST have a time stamp (with timezone) along with other information relative to the attack/abuse. This is the only way the ISPs can relate the IP with the actual enduser in order to contact them for remediation.

It need be neither momentous nor monumental -

Just say it's 0.0.0.0 / 0 with some occasional exceptions.

Regards
Marshall Eubanks

Wayne Gustavus (nanog) wrote:

This would essentially be impossible and not a good idea. Large volumes of hosts/zombies involved in such attacks originate from residential cable/dsl subscribers. This user base primarily uses dynamically assigned IP space. Hence, the IP of tonight's attacker could be the IP of tomorrow's legitimate user.

1. It is arguable whether dynamic IPs are to be treated as legitimate mailhosts. Your colleagues in VOL mailops might tell you something similar too.

2. An expiring list, where entries inserted are quickly expired, and stats used to add to other lists (such as MAPS DUL / SORBS DUHL) is a good idea, and moreover, it's already been done. http://cbl.abuseat.org

  srs

From: Suresh Ramasubramanian [mailto:suresh@outblaze.com]
Sent: Saturday, February 07, 2004 9:58 PM
To: Wayne Gustavus (nanog)
Cc: 'Drew Weaver'; nanog@merit.edu
Subject: Re: Monumentous task of making a list of all DDoS Zombies.

<snip>

1. It is arguable whether dynamic IPs are to be treated as legitimate
mailhosts. Your colleagues in VOL mailops might tell you something
similar too.

No argument there. However, the thread was originally addressing a list of
DDoS Zombies, not illegitimate SMTP mailhosts. Arguably zombies used to
launch
DDoS attacks are treated differently than such hosts. We address both
types.

2. An expiring list, where entries inserted are quickly expired, and
stats used to add to other lists (such as MAPS DUL / SORBS DUHL) is a
good idea, and moreover, it's already been done.

http://cbl.abuseat.org

Interesting approach. It would be conceivable that if this resource was
Widely used, miscreants could use this service to DDoS there victims without
an army of zombies :slight_smile: I still submit that it is more advisable to address
the root of the problem by finding the true host that generated attack
traffic. Automating this process of matching dynamic IP to customer acct
with a timestamp and remediation is the goal.

Wayne Gustavus (nanog) wrote:

http://cbl.abuseat.org

Interesting approach. It would be conceivable that if this resource was
Widely used, miscreants could use this service to DDoS there victims without
an army of zombies :slight_smile: I still submit that it is more advisable to address
the root of the problem by finding the true host that generated attack
traffic. Automating this process of matching dynamic IP to customer acct with a timestamp and remediation is the goal.

Timestamps are, of course, a solution - they definitely help in quickly identifying compromised hosts.

Another thing that helps with easier identification is a practice some ISPs have of inserting the MAC address of the host into the reverse DNS record, with a short TTL. When a new host gets that IP, the MAC address changes too. I have seen at least one ISP do this - and it makes it a whole lot easier for the ISP to quickly identify the host, rather than having to trawl through RADIUS logs or whatever else.

Then, there's the little matter of ISPs implementing ingress filtering as per BCP38 / RFC 2827. These DDoS zombies seem to also be used as a ready source of spoofed source addresses for attacks.

  srs

Another thing that helps with easier identification is a practice some
ISPs have of inserting the MAC address of the host into the reverse DNS
record, with a short TTL. When a new host gets that IP, the MAC address
changes too. I have seen at least one ISP do this - and it makes it a
whole lot easier for the ISP to quickly identify the host, rather than
having to trawl through RADIUS logs or whatever else.

I've made proposals like this in the past, and have investigated some of
the issues. I don't know if the world is ready to go that far yet.

In practice MAC address tracking only works for a few very specific ISP
architectures, such as when the ISP supplies the hardware used to connect
to the network.

Tracking MAC addresses ends up requiring having to trawl through RADIUS
logs because users don't like having to tell the ISP everytime they change
ethernet cards or computers. Most end-user home routers now include
options to "clone" any MAC address to get around the MAC address
requirement of a former (bankrupt) cable ISP.

So you need to search which subscriber account was signed on with that
MAC address during the suspect time period. Vendors aren't always that
careful about assigning unique MAC addresses, and complaints aren't
always that careful about reporting the correct MAC addresses. You
still need the time information to verify the subscriber was actually
online. For dialup users, which don't have ethernet MACs, do you
put the user's home phone number in the reverse DNS records?

How much privacy should users have or expect? One of the most common
requests from law enforcement is how can they get a "unlisted" IP
address. The same way they get an unlisted credit card number.

Look at the hysteria over browser cookie "tracking" on the web. The
"anti-Spyware" programs like to list lots of "spy" cookies which
anonymously track visitors to web sites. Instead of Doubleclick tracking
users with Cookies, they would be able to track the unique computers
from the MAC address in the reverse DNS record over time.

Or would this backfire, because then the hackers could find the vulnerable
computers again and again even if the IP address changes. The hackers
could scan the in-addr.arpa ranges looking for known vulnerable MAC
addresses. Look, someone with a MAC address assigned to equipment known
to be vulnerable.

The problems are similar if the ISP assigns some other "unique" identifier
to the subscriber and dynamically updates the reverse DNS record.

Then, there's the little matter of ISPs implementing ingress filtering
as per BCP38 / RFC 2827. These DDoS zombies seem to also be used as a
ready source of spoofed source addresses for attacks.

There are several ISPs which implement ingress filtering per
BCP38/RFC2827. None of them have seen a change in the number of DDOS
attacks. The people who track this kind of stuff say that most
attacks do not use spoofed addresses.

Unfortunately, the data is lacking about the effectiveness of any of
these solutions. In the USA, the FDA requires drug producers to show
new drugs are safe and effective before being sold to the public. There
is no such requirement for people selling security solutions.

   - Block access from IP addresses without rDNS
  Data?

   - Insert MAC address into rDNS (negating previous block)
  Data?

   - Implement BCP 38
  Data?

Sean Donelan wrote:

In practice MAC address tracking only works for a few very specific ISP
architectures, such as when the ISP supplies the hardware used to connect
to the network.

I'm aware of these - but surely there's something about the user which you can stick into rDNS (hashed / encrypted if you like) that'll identify the user?

The problem with trojans etc is that there so damn many of them, so the less time spent actually tracking down the user who was on IP X at time Y, the better it is for the ISP's staffers who handle complaints about these.

Of course, prevention is better than cure, so another recourse the ISP has is to be proactive - setting up a scanner to sweep the host that comes up on an IP the moment the dhcp server assigns it. If not a full blown portscan or anything, then at least a quick once-over that looks for signs of the current "big problem" trojans / zombies.

There are several ISPs which implement ingress filtering per
BCP38/RFC2827. None of them have seen a change in the number of DDOS
attacks. The people who track this kind of stuff say that most
attacks do not use spoofed addresses.

I have heard from someone who hosts one of the mirrors for a site that is a DDoS magnet. I recall his saying that a non trivial number of attacks coming at this mirror were from spoofed source addresses.

No, I don't claim that BCP38 is a magic bullet either. But I do put it to you that the way to at least mitigate this menace include a combination of several steps -

1. Easy identifying of hosts, at least to the ISP (to avoid privacy concerns)

2. Sensible filtering practices

3. Proactive network sweeps

4. Quick and immediate isolation of infected hosts - nullroute them, or maybe VLAN them into their own corner of the 'net, where the only thing they can access over http is an ISP support page saying "please un-root your computer, or contact us at 1-800-[foo] for help and more details"

5. Cooperation with law enforcement if necessary, to track down and punish the DDoSer.

  srs

Of course, prevention is better than cure, so another recourse the ISP has is to be proactive - setting up a scanner to sweep the host that comes up on an IP the moment the dhcp server assigns it. If not a full blown portscan or anything, then at least a quick once-over that looks for signs of the current "big problem" trojans / zombies.

Coming up with new types of probes all the time to check for this would be a huge amount of work.

I favor an approach where people no longer get to send data at high speed without the recipient's approval. Just sending data in the blind or any type of scanning could then trigger a severe rate limit or raise an alarm.

There are several ISPs which implement ingress filtering per
BCP38/RFC2827. None of them have seen a change in the number of DDOS
attacks. The people who track this kind of stuff say that most
attacks do not use spoofed addresses.

I have heard from someone who hosts one of the mirrors for a site that is a DDoS magnet. I recall his saying that a non trivial number of attacks coming at this mirror were from spoofed source addresses.

People need to make sure only packets with legitimate source addresses escape from their network. Period.

Unfortunately, this type of action must be performed at the source and some networks just can't be bothered.

Iljitsch van Beijnum wrote:

Coming up with new types of probes all the time to check for this would be a huge amount of work.

Would that be any less work than clearing up the mess left by an infestation of DDoS zombies? :slight_smile:

I favor an approach where people no longer get to send data at high speed without the recipient's approval. Just sending data in the blind or any type of scanning could then trigger a severe rate limit or raise an alarm.

It is fairly easy to work around rate limits by just scaling laterally, and compromising a few million more boxes. If the next virus grabs 4M, or 20M boxes instead of just a measly 2M boxes, you can rate limit all you like, bit it really won't help.

Unfortunately, this type of action must be performed at the source and some networks just can't be bothered.

Yup.

  srs

I have asked about this before. Wouldnt it be very nice if there was a
standardized way to report IP-number and timestamp and type of complaint?

I've seen something produced by some workgroup (RIPE?) but that was a huge
document about XML and it seemed non-trivial to implement. I was more into
the idea of having basically email headers like:

X-ABUSEREPORT-IP: <ip>
X-ABUSEREPORT-DATE: <unix timestamp>
X-ABUSEREPORT-TYPE: <spam|abuse|ddos|other>

This should make it trivial for most automated tools to append this
(spambouncer etc) and make it much easier for the abuse system to do a
user lookup before presenting the abuse email to the handler, even
providing the user email address so the handler can take action.

I have asked about this before. Wouldnt it be very nice if

    > there was a standardized way to report IP-number and
    > timestamp and type of complaint?

There isn't one yet.

Some people are trying to put together a simplistic looking BCP -

    > I've seen something produced by some workgroup (RIPE?) but
    > that was a huge document about XML and it seemed
    > non-trivial to implement. I was more into the idea of
    > having basically email headers like:

There is a RIPE WG on spam (I think chaired by Rodney Tillotson from
JANET/CERT). But I don't recall something like this being proposed
.. and XML is a rather unruly beast to manage, especially for joe
user.

Your idea of headers might work - or something on the lines of send-pr
on *bsd. All that the NOC staff receiving it would require is that it
stays simple, without stuff like :

Frenzied abuse
Screenshots from fancy IDS / software firewall products
Long lectures on why spam / DDoS / other network abuse is bad

A short two or three line summary of the issue, accurate timestamps
and a set of excerpts from your logs (not a whole lot, just enough to
make the situation obvious) should be enough.

Another big help is giving the NOC access to a good ticketing system
which understands the difference between customer support and net
abuse handling (here, your customers are the problems, for starters).
RT3 has a lot of code (courtesy Paul Vixie and the other people at
MAPS who were hacking on it) - but there's a nice new product called
Abacus - Abacus – Word to the Wise that looks promising.

       srs

Coming up with new types of probes all the time to check for this would be a huge amount of work.

Would that be any less work than clearing up the mess left by an infestation of DDoS zombies? :slight_smile:

Apples and oranges. You need to clean up the zombies regardless of whether they succeeded in attacking the victim or they were stopped.

I favor an approach where people no longer get to send data at high speed without the recipient's approval. Just sending data in the blind or any type of scanning could then trigger a severe rate limit or raise an alarm.

It is fairly easy to work around rate limits by just scaling laterally, and compromising a few million more boxes. If the next virus grabs 4M, or 20M boxes instead of just a measly 2M boxes, you can rate limit all you like, bit it really won't help.

Help against what? You're right that if a million boxes send one 125 byte packet per second to the same place, that's still a gigabit worth of traffic, that particular place is going to receive a gigabit worth of traffic. But how are you going to infect a million boxes if you can only scan one address per second?

And let's not be so blase assume that all DoS attacks are done with a million zombies at a time.

I'm aware of these - but surely there's something about the user which
you can stick into rDNS (hashed / encrypted if you like) that'll
identify the user?

The problem with trojans etc is that there so damn many of them, so the
less time spent actually tracking down the user who was on IP X at time
Y, the better it is for the ISP's staffers who handle complaints about
these.

It's not that hard, I assume we are talking about dial-up, cable and xDSL
users? We already log all major radius events in a database and it's very
easy to look up users in that db, we have a web page for CSR's (customer
service representative's), additionally the mail server detects which of our
ip ranges is sending worms and automatically disables those users... I see
no gain from adding anything in DNS, like reverse records.

Of course, prevention is better than cure, so another recourse the ISP
has is to be proactive - setting up a scanner to sweep the host that
comes up on an IP the moment the dhcp server assigns it. If not a full
blown portscan or anything, then at least a quick once-over that looks
for signs of the current "big problem" trojans / zombies.

We perform this today, the problem is, what are the signs for "big problem"
trojans and zombies? If there was a tool out there that could perform
scanning
of computers AND knew about what to look for (does this malware operate
on fixed ports) AND could be automatically updated for new malware I would
purchase such a tool. Other than scanning for the open ports, I think these
zombies are regular open proxies... but that may (will?) change in the
future.

4. Quick and immediate isolation of infected hosts - nullroute them, or
maybe VLAN them into their own corner of the 'net, where the only thing
they can access over http is an ISP support page saying "please un-root
your computer, or contact us at 1-800-[foo] for help and more details"

We simply modify their passwords and log them the off today. There is also
an entry created in the incident tracking system. But, we have it as a
future goal
to let them access some pages, like HouseCall etc.

Rgds,
-GSH

Date: Sun, 8 Feb 2004 02:01:29 -0500 (EST)
From: Sean Donelan

Instead of Doubleclick tracking users with Cookies, they
would be able to track the unique computers from the MAC
address in the reverse DNS record over time.

A MAC address is six octets. Append time past Epoch when IP was
assigned; that's another four octets. Append six random octets.
Encrypt. Make hostname-friendly using %x equivalent.

One now has 32 characters that contain the MAC address and time
the DHCP lease (or whatever) began, yet are meaningless to those
who lack the key. Consider periodically changing the six random
octets to protect users with long DHCP leases.

It's extra hassle, but one can clearly have tracking _and_
protect user privacy.

That leaves the issue of users changing MAC address to evade
detection. However, the aforementioned DNS issues have no
bearing on this problem, which is a separate topic.

Eddy

> In practice MAC address tracking only works for a few very specific ISP
> architectures, such as when the ISP supplies the hardware used to connect
> to the network.

I'm aware of these - but surely there's something about the user which
you can stick into rDNS (hashed / encrypted if you like) that'll
identify the user?

But I still don't understand why an ISP unwilling to spend the money
to trace uses with RADIUS or other existing methods; is going to want
to spend money on interfacing their systems with Dynamic DNS servers and
new systems to generate DNS cookies. It increases their cost, and
doesn't provide any additional information which they have in their
existing systems.

On the other hand, if we don't care too much for the privacy implications
it would benefit 3rd parties wanting to keep track of individual
computers. It would help ISPs, because 3rd parties could take more
effective action on their own to ignore traffic from particular computers.

Digital rightes management, password guessing, IRC bans, mail blocks, etc
could work much more effectively if ISPs provided a unique identifier for
subscribers. If software and hardware vendors included a hard-coded
unique identifier in every computer, it would be even more effective.
Intel has proposed this in the past. Microsoft has a GUID concept for
its software.

But is the world really ready for this level of identification and
tracking?

The problem with trojans etc is that there so damn many of them, so the
less time spent actually tracking down the user who was on IP X at time
Y, the better it is for the ISP's staffers who handle complaints about
these.

As you point out, there are a lot of them. But the goal should be to
NOT have the ISP's staffers handle individual complaints. Any "solution"
which requires staff to assess and respond individually is not an
improvement.

That's why I proposed the ICMP Go Away message.

Of course, prevention is better than cure, so another recourse the ISP
has is to be proactive - setting up a scanner to sweep the host that
comes up on an IP the moment the dhcp server assigns it. If not a full
blown portscan or anything, then at least a quick once-over that looks
for signs of the current "big problem" trojans / zombies.

I assume you are aware that one of the fastest growing trojan segments
includes trojans which can not be detected by port scanners.

You are correct that prevention is better than the cure. Unfortunately
you've misidentified the point of prevention. The software vendor is
in the best position to prevent systems being compromised. A change at
Microsoft can prevent 60-70 million computers a year from being
vulnerable. As an ISP, even AOL can't fix that many computers.

I have heard from someone who hosts one of the mirrors for a site that
is a DDoS magnet. I recall his saying that a non trivial number of
attacks coming at this mirror were from spoofed source addresses.

The number of spoofed packets received has very little to do with the
number of sources of spoofed packets. But again, it points out the
lack of hard data. Yesterday, a red car cut me off, so obviously the
problem is red cars and we should prohibit all red cars.

Is there any difference in the number of attacks between networks which
have deployed BCP38 and networks which haven't? Or perhaps the problem
is with the computers connected to the networks, not the networks.

No, I don't claim that BCP38 is a magic bullet either. But I do put it
to you that the way to at least mitigate this menace include a
combination of several steps -

1. Easy identifying of hosts, at least to the ISP (to avoid privacy
concerns)

By whom? Should anyone be able to identify any host any time, or is it
only necessary for inter-connected providers to identify the next provider
in the chain? Should end-users be complaining to their own provider (i.e.
the ISP they are paying money) or calling 3rd party ISPs which have no
method to identify who is making the complaint?

2. Sensible filtering practices

Filter for what? What is considered sensible?

3. Proactive network sweeps

Sweeps for what?

4. Quick and immediate isolation of infected hosts - nullroute them, or
maybe VLAN them into their own corner of the 'net, where the only thing
they can access over http is an ISP support page saying "please un-root
your computer, or contact us at 1-800-[foo] for help and more details"

Of course you meant to say contact the person who sold you your computer
for help fixing your computer. The police write tickets, they don't
fix cars.

5. Cooperation with law enforcement if necessary, to track down and
punish the DDoSer.

Of course, the original issue was PTR records for spam, not DDOS. But
this isn't the first time people have changed in the middle of a thread.

Which ISPs are not cooperating with law enforcement?

In the US, if you receive harrassing or threatening phone calls, you have
to file a police report. The telephone company only provides the
information about the source of the calls to the police for followup.

How many people file police reports for spam, ddos, etc.

Again, why does an ISP need to spend the money and as you point out
the extra hassle, to do this? ISPs already have all the information they
need to trace a subscriber from the IP address and timestamp.

Why does the ISP need to install Dynamic DNS servers, links between their
RADIUS servers and the DDNS, and the databases to keep track of the which
keys were used at which time. How long should ISPs be expected to
maintain the keys to decode the DNS cookies. If someone wanted to know
which subscriber was using an IP address in 1994, should that be
possible? How long should the key length be to prevent people from
cracking it years or decades later? Should ISPs provide the government
copies of their keys so national security can keep track of the
information. The privacy advantage of not using "DNS Cookies" is RADIUS
logs only last as long as the ISP keeps them and there is nothing to
crack.

We made this mistake once already by having /etc/passwd files
world-readable (encryption would protect the passwords). Don't repeat
the mistake. If you suspect a particular computer, know the time, how
long would it take to brute-force the remaining six characters?

Technology is cool, but is this solving a problem?

Date: Sun, 8 Feb 2004 17:43:34 -0500 (EST)
From: Sean Donelan

Again, why does an ISP need to spend the money and as you
point out the extra hassle, to do this? ISPs already have
all the information they need to trace a subscriber from the
IP address and timestamp.

I'm not sure they need to for the MAC address example. I was
pointing out that information contained in reverse DNS could be
meaningful, but only to those who should know.

Perhaps a better example would be to s/MAC address/user id/ and
repeat the example. Then, instead of digging through logs, one
could quickly decrypt the user ID.

We made this mistake once already by having /etc/passwd files
world-readable (encryption would protect the passwords).

Totally wrong analogy. /etc/passwd was a one-way hash (useless
for this situation)...

Don't repeat the mistake. If you suspect a particular

...using crypt(). Note that I never suggested use of weak
crypto.

computer, know the time, how long would it take to
brute-force the remaining six characters?

I can think of some frequently-encrypted data that begins with
strings like "HTTP/1.1 200 OK". So which is a better starting
point for key recovery?

Eddy

Iljitsch van Beijnum wrote:

traffic. But how are you going to infect a million boxes if you can only scan one address per second?

Maybe just infect a million windows boxes on your network with a trojan, and then have the trojan phone home (say to an irc channel or a central controlling server) for instructions?

  srs