[Full-disclosure] what can be done with botnet C&C's?

Subject: what can be done with botnet C&C's?

"I work on this [C&C] for 30 days, only to find out one of you took it
down." -- US Federal Agent, two days ago, ISOI (DA Workshop).

Oddly agents have resources right in front of them to assist them
(CALEA, and other totalitarian laws) and yet they fail to use
these resources properly only optioning to promote newer and even
more stupider laws.

And still, sticking to networking issues, as obviously we cannot yet
depend on law enforcement to protect our networks for us, how do we handle
C&C's?

Where in the rule book does it state that LEA's are here to protect
any network. I say it begins with the CSO's, managers, engineers (both
network and security engineers.) Bear in mind cross juridstiction
across international boundaries.

When we kill them (and by "kill" I naturally mean "report our suspicion
to the responsible authority so they can investigate, confirm and proceed
according to their AUP") we kill them, but only to our knowledge. They
immediately move elsewhere we do not know about in our space or someone
else's, maybe misplacing an extremely smallish percentage of their
population while they are at it.

Let's be realistic about this. Most providers in the US at least have
some form of CALEA capabilities which can monitor what is coming from
where in order to filter networks.

Okay, say I am right... What *can* we do?

Re-write AUP's from the ISP level blocking out and allowing out on a
needed basis. For those BOTNETS utilizing IRC, they'll be nipped at
the bud, for those in these networks truly utilizing IRC and other
similar venues, an ISP could either set up their own server and link
to other servers, or the IRC user themselves are almost always
smart enough to figure out how to jump on an IRC server.

We can take advantage:
1. QoS and traffic limiting tools.
Many tools created in recent years, and used exstensively by many ISP's,
regardless of any Net Neutrality legislation, are at our disposal and
already implemented on our networks.

QoS is a joke. The problem with QoS is a configuration issue. How many
networks are still allowing broadcasts (Smurf). What makes one think
that if they can't configure simple RFC filtering and containment of
broadcast, they'd be able/capable/willing to configure QoS. Outside of
this, the biggest argument will be a "not in my backyard" issue of
"why are you filtering our traffic."

Much like, for business reasons, many of us would limit P2P, how about
limiting the traffic to compromised users?

How, what and when is up to you.

Laziness. Come on now, and by the way greeting Gadi, you should know
offhand the slack that comes from lazy admins unwilling to do squat
but read this in the background and continue eating ho-ho's and
donuts.

Watch the flows, block the users from communicating out to them. Watch
these users and see where else they are communicating in comparison to
other users, en-masse.

Breaking laws here if you ask me. Watching flows. Isn't this an illegal
wiretap.

4. Stop internal network infections. It is unbelievable how the networks
with the most bots are the networks that allow internal users to connect
wherever they want within the network.

Re-read my lazy admin donut syndrome.

My answer is this, if you fail to remove a spy, as another would just take
his place, wouldn't you rather know where that spy is and work to take
him down for good?

One thing that will end up happening as is evident is, you will
end up creating a smarter and smarter botnet. Filter from here, they
move, filter this port, they jump. Most network admins know how to
entirely block these things but they don't. How about a completely
new approach via AUP. "Welcome to Foofoo Network's your ISP. We allow
SMTP, HTTP, HTTPS, IM." Period. No need to keep the other 65531 ports
open.

Do you know who your local fed is?

Definitely not on Clue Avenue. If they were there would be no need
to try and impose LawB atop LawA which never worked in the first
place.

I would like to hear some opinions on what networks can do, ecnomically,
from people here. Please stick to network operations issues.

       Gadi.

Here is my opinion... Responsibility on both ends. For the user and
the provider.

The One-Two punch

1) For an ISP something like Campus Manager would work
wonders (Premier Network Access Control (NAC) Solutions & Security | Fortinet).
Configured in the background it can take machines and shove them
into a non useable VLAN until they get their act together.

2) Client breaching Terms of Service agreements? Hold them
accountable.

Users are responsible for their own machines:

<analogy>
UserA buys a gun and keeps it in his house.

UserA does not buy a safe or take necessary precautions to
safeguard his gun.

LuzerB uses that gun for a crime.

LuzerC (UserA's son or daughter brings it to show and tell)

LuzerD (UserA's neighbor blows his brain out via Russian Roulette)
</analogy>

In all of these instances UserA would still have to answer
for not taking the necessary precautions. Why not implement
the same procedures when signing up a new user via TOS
agreements. I'm sure more people would be reluctant to
just close their eyes after one warning followed by them
having to pay for not taking precautions.

Sure, argue about the clueless ones who don't know who
to tie their shoes. 1) Would make them more aware. 2)
Would make networks safer since their is accountability.

My two cents for the moment.

I�ve been reading on this subject for the last several weeks and it seems
as if everyone just like to come up with out of the box ideas that are
not realistic for today�s network environments

J.Oquendo, thanks for the Smurf example � as there are still

admins/engineers at large networks that have no clue as to what they
are doing� so QoS is for sure out of the question.. at least at this
time.

Depending on agents to take actions and protecting our networks is even a
bigger joke. Back in late 90s where kiddies were using the simplest types
of C&C, open wide irc networks with visible Channels and no encryptions�
and agents couldn�t do anything unless the attack was big enough to take
down Amazon, yahoo, Microsoft or some other major provider with enough $$$
to start an investigation.

So what makes you think that agents are of any help in today�s world where
c&c have gotten so much more sophisticated, use backup private servers,
encryption, tunneling and much much more..

In my opinion, the only way to really start cracking down on c&c and put
an end to it is the cooperation of major ISP�s. I realize that most isp�s
cant/wont setup a security team to just investigate c&c / attacks (would
this really fall under the Abuse team?) but perhaps If all major networks
worked together and created a active db list of c&c found either on their
networks or attacking ones network� then it would be much much easier to
trace back c&c and dispose of them.

Unfortunately, we don�t live in a perfect world and most isp�s hate
sharing any information� I guess its better for them to have a bigger ego
than a safer / more stable network�

Please feel free to correct me if I am wrong�

-Payam

I hate to stir the flames again, but this idea sounds a lot like RBLs. :slight_smile:

All kidding aside, I'm curious as to when we will reach the point where the devices of our networks will be able to share information regarding sporadic bursts or predefined traffic patterns in network traffic within a certain time frame, determine it is a related outgoing (or incoming) attack, and mitigate/stop the traffic. I think it certainly is possible to accomplish this on a per-router level, but being able to have the devices communicate and share information between one another is a completely separate thing. (New protocol perhaps.)

The only real method that I really have in my toolkit to stop incoming DDoS on a AS-wide perspective is originating a /32 within an AS with a next-hop of a discard interface.

Something similar to that nature but more flexible and designed for the sole purpose of preventing/stopping abuse would be a very nice feature.

Cheers.
-Michael

Though placing a /32 to a discarded interface helps the situation, you are
now fully disabling your client that uses the /32... I do agree that it
definitely helps the situation... specially when the attack is a few mil
pps or perhaps even few gigs/sec in which case a customer /32 or bigger�
being down is about 100x better then your network being down.

so my question is then how do you use the same method for your peering
sessions (assuming you do peering on a private or public level)... seeing
how 95% of peers will not allow such specific entries such as /32 into
their tables... so in case of an attack you are left with either having to
take down the peering session or stop advertising the prefix though that
peer.

Just curious as to how you go about it...

cheers,
-Payam

attack, and mitigate/stop the traffic. I think it certainly is possible
to accomplish this on a per-router level, but being able to have the
devices communicate and share information between one another is a
completely separate thing. (New protocol perhaps.)

reference TIDP ... which is like (sort of) Flow-Spec, only not piggybacked
upon BGP and with possibly some extra functionality wrt 'doing the right
thing' on each platform in question. Also, TIDP doesn't have to be tied to
a device that runs a routing protocol...

The only real method that I really have in my toolkit to stop incoming
DDoS on a AS-wide perspective is originating a /32 within an AS with a
next-hop of a discard interface.

reference TIDP and FlowSpec (if you have 'discard interface' you already
have flow-spec)

IANAL, so ask somebody who is if the answer matters... but by my reading
of 18 USC 2511 (2)(a)(1) says you're off the hook on that one, for the cases
that a NANOG reader would care about:

"it shall not be unlawful under this chapter for an operator of a
switchboard, or an officer, employee, or agent of a provider of wire
or electronic communication service, whose facilities are used in the
transmission of a wire or electronic communication, to intercept,
disclose, or use that communication in the normal course of his
employment while engaged in any activity which is a necessary incident
to the rendition of his service or to the protection of the rights or
property of the provider of that service, except that a provider of wire
communication service to the public shall not utilize service observing or
random monitoring except for mechanical or service quality control checks."

I read the last few lines as saying "It's not OK to go targeting Joe Sixpack's
flows, but it *is* OK to run an IDS or similar system that triggers whenever
an DDoS or other similar "detrimental to your service quality" event happens.
You're allowed to protect your network, and you're allowed to do monitoring
for "service quality control".

I however *also* read that as meaning that once you've identified a specific
customer, you need to be careful to *only* target data that's identifiable
as being an service quality issue - if it's doing DDoS stuff on port 7703,
that doesn't extent to their SMTP traffic. (Of course, if they're also spewing
spam at line speed at the same time, that's another story...)

I'm sure most people on this list have heard of or use snort. There is an
add-on package called snortsam. This package allows automation of blocking
traffic deemed malicious via a null route statement or ACL statement. We
have been in the process over the last month of implementing this on our
network with much success. I think the only problem that we have had with it
thus far is underestimating just how well it was actually going to work. As
with any snort implementation, it takes time to tweak and tune the rule
sets, however we have managed to kill a huge amount of traffic either coming
from our customers or destined to our customers. While this is not a perfect
system, it is much better than idly sitting there and letting the abuse
continue.

Most major carriers have some way of communicating with them for this
purpose. Level(3) uses BGP community 9999 for a peer of theirs to issue /32
routes to their black hole router. Global Crossing uses an eBGP multi-hop
peer for these types of advertisements and others have their mechanisms as
well. On the flip side, through route-maps you could setup a community for
your customers to advertise to you traffic they wish to null, of course you
must take great care when doing this as to not allow something that could
really screw your network. In our network, all of our /32 nulls have a tag
applied from our black hole router, which gets propagated via OSPF then
eventually gets handed off to our peers using either a community or
multi-hop neighbor.

I'm sure most people on this list have heard of or use snort. There is an
add-on package called snortsam. This package allows automation of blocking
traffic deemed malicious via a null route statement or ACL statement. We
have been in the process over the last month of implementing this on our
network with much success. I think the only problem that we have had with it
thus far is underestimating just how well it was actually going to work. As
with any snort implementation, it takes time to tweak and tune the rule
sets, however we have managed to kill a huge amount of traffic either coming
from our customers or destined to our customers. While this is not a perfect
system, it is much better than idly sitting there and letting the abuse
continue.

Hi Jordan, I am very happy to see Sago changing from one of the worst nets
on the net when it comes to botnets to being, apparently, one of the most
pro-active.

That said, when I last checked (a week ago) you had 4 botnet C&C's still
open and active on your AS.

As always, you and anyone else here can email us directly for the
information on your network.

  Gadi.

Thanks for the info. I will pass this to our abuse department to get rid of
those. We are still tweaking our system and is only about 90% deployed, but
after all of the efforts to deploy the system, it should pay-off many many
times over.

Thanks again,

Jordan

Gadi,

I am unable to find the list in the archives or my email client. Can you
send me anything that you have so I can get it taken care of?

Thanks,

Jordan

Gadi,

I am unable to find the list in the archives or my email client. Can you
send me anything that you have so I can get it taken care of?

Of course.

  Gadi.

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

in-line:

Jordan Medlen wrote:

I'm sure most people on this list have heard of or use snort. There is an
add-on package called snortsam. This package allows automation of blocking
traffic deemed malicious via a null route statement or ACL statement. We
have been in the process over the last month of implementing this on our
network with much success. I think the only problem that we have had with it
thus far is underestimating just how well it was actually going to work. As
with any snort implementation, it takes time to tweak and tune the rule
sets, however we have managed to kill a huge amount of traffic either coming
from our customers or destined to our customers. While this is not a perfect
system, it is much better than idly sitting there and letting the abuse
continue.

- -------------------------
One thing would be nice (maybe a wish-list) if snortsam could send an
e-mail notification (similar to other proactive tools) rather than
pushing for ACL change which could possibly break something due to FP.
This could lead to a headless chicken syndrome scenario. Also where I
come from, we cannot implement change(s) to any P1/P2 (business
critical) devices w/o a change management request except for emergencies.

regards,
/virendra

Something that would probably help a *lot* (and get some publicity for
yourself and your shop while you're at it :slight_smile: is to do up a short presentation
or paper discussing what the total aggregate payoff ends up being. Being
able to point at "This company spent $yyyK doing X, Y and Z, and achieved a
total net savings of $x.xM/year" is quite useful when trying to explain to
other sites why they should clean up their networks....

Good point. At this time, we are not yet at completion as stated, but
something that could be done for the benefit of others once we have
completed the install and taken into account the amount spent vs. gained as
you stated. I will look to getting something to everyone once our experience
is complete.

Thanks,

Jordan

Snort itself can be configured to send email notifications without the
snortsam add-on. Snortsam does have a "do-not-block" list as well so that
certain hosts are never blocked. This is useful for our NOC staff since we
continually run tests such as nmap towards our customer's servers that would
otherwise have our NOC IP space blocked.

-Jordan

Is this the kind of thing that could get a boost from projects like
ThreatNet (http://www.ali.as/threatnet/)?

I've been peripherally involved with the project, mostly in chasing
conceptual issues and hammering out the trust relationship details, in
addition to being an rabid, er, avid developer of network management
tools, right down to near-real time log analyzers. I think that if you're
to a point that you trust your tools enough to make an informed decision
and automate your null routes, why not expand on that and use the same
networked intelligence the C&C's embody?

- billn