Broadband routers and botnets - being proactive

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

I and numerous others (including some whom any reasonable NANOG-L poster
would respect and listen to) have asked you repeatedly to stop trolling
NANOG-L with this botnet crap. It is off-topic here. The last time you
pulled this (starting

As frequent as Gadi is with his botnet posts, insecure and wide open
CPE getting deployed across a large provider is definitely
operational.

Suresh is right -- if you don't think CPE compromises are an
operational problem, then I'm not sure what is. :slight_smile:

[changing gears]

I'll even go a step further, and say that if ISPs keep punting
on the whole botnet issue, and continue to think of themselves
as 'common carriers' in some sense -- and continue to disengage
on the issue -- then you may eventually forced to address those
issues at some point in the not-so-distant future.

I understand the financial disincentives, etc., but if the problem
continues to grow and fester, and consumer (and financial institutions)
losses grow larger, things may take a really ugly turn.

$.02,

- - ferg

I must admit, vulnerabilities are endless and new exploitation vectors
will never end, even if it was possible and we were all 100% secure,
someone (an attacker rather than a vulnerability) will find a way to make
it 99% again for the right investment or with the right moment of
brilliance.

Enough with cheap philosophy though... as tired (even exhausted) as I am
of the endless repeating circle which security is, on all levels (from the
people involved through the interests involved all the way to the
same-old-FUD) I still haven't burned out, and I am still here.

The world isn't going to end tomorrow, and even if the Internet was to die
(which I doubt it will), we will survive. However, in the recent couple of
years a new community has been forming which we started refering to as
"Internet security operations". These folks, for various motives, work to
make the Internet stay up and become safer (actually being safe is a long
lost battle we should have never fought the way things were built).

With such a community being around, treating issues beyond our little
corner of the `net is possible to a level, and at least some progress is
made. Some anti virus engineers no longer care only about samples, some
network engineers no longer care only about their networks, etc.

Is any of this a solution? No. The problems themselves will not go away,
they aren't in any significant fashion currently being dealt with beyond
the tactical level of a fire brigade.

Is it the end than? Of course not. But operations vs. research are
determined by intelligence. As we have some intelligence, I can point to
yet another annoying vulnerability in the endless circle which those of us
who will want to, can study, and if they feel it is justified, defend
against. That is the broadband routers issue, which personally I'd really
rather avoid.

Unfortunately, this limited defense is what most of us can do at our own
homes, or tops as a volunteer fire brigade or neighborhood watch.

The Internet is the most disconnected global village I can imagine, but
we all have the funny uncle on another network and a weird one on yet
another. I sometimes feel that the old analogy of the Internet to the Wild
West is not quite it. Perhaps we are living in the Wild West, only if
instead of wastelands and small towns, we have New York city and the laws
of a feudal dark ages Kingdom.

Things will eventually change, and some of us will stick around to help
that change (or try to). For now though, it is about one vulnerability
ignored at a time, and working on our communities.

  Gadi Evron.

I totally agree - the issue keeps getting delayed and nobody is adressing it
like it should be, People keep talking about the issue but it NEVER gets
solved.

Here's my own two cents:

Most end-users don't know and probably, don't care about what they subject
their systems to, therefore, systems get infected constantly.

There will be no resolution of these bandwidth-wasting botnets unless
something is done about the end-users who don't know/care about what they're
doing, Most end users just "want things to work" without knowing and probably
without wanting to know what actually is going on "behind the screnes".

Furthermore, as I posted on another list, Users depend too heavily on
their "security software" and think if they have a firewall and antivirus,
that they can do anything and won't be infected, But as we all (I hope) know,
that's not true.

It's true ISPs should be held in higher responsibility to security issues such
as botnets, but the end-users who end up with bots/trojans on their systems
should also be held accounable. Perhaps if users get the weight on their
sholders of keeping clean, they will instead of how it currently is where the
issue seems to get only talked about but really no collective enforcement
anything as I stated earlier.

And it's not just users and ISPs that should be dealing with this issue,
Datacenters should as well, I can't count how many servers I've seen infected
and being used in botnets.

I say kudos to those who already combat botnets on their networks, However, To
those who do nothing at the moment: I say it's time to start.

Oh, one more thing to the first reply to this thread calling this a
non-operational issue, Gadi's in the right here: It IS an operational issue
that's getting reposted because it's NOT getting solved.

Kradorex Xeron wrote:

Oh, one more thing to the first reply to this thread calling this a non-operational issue, Gadi's in the right here: It IS an operational issue that's getting reposted because it's NOT getting solved.

I recieved 4 emails (from Fergie, Suresh, Colin Johnson and "Kradorex Xeron") disagreeing with my assertion that Gadi's emails are off-topic. I also recieved a few emails saying things like "Sure he's off-topic, but he's a well-known botnet researcher, and a very smart guy, and don't you think you're being too hard on him?" and one saying in essence "Who are you to question a highly respected guy like Gadi?"

The 4 people who feel that Gadi's botnet posts are on-topic here in NANOG-L have apparently not read the NANOG-L charter and FAQ so I am providing links here:

http://www.nanog.org/aup.html
http://www.nanog.org/listfaq.html

I agree that Gadi is a highly respected botnet researcher, and I'm just a lowly netadmin at a regional ISP. Shouldn't I just shut up and soak up his glory? If this were a botnet list, yes. But this is a network operator's list, and I'm a network operator. There are lists where botnets are discussed, and Gadi is very active on those lists. There is no need for him to repost his botnet emails to NANOG-L. I don't join the botnet lists and spam them with networking issues, and it's not appropriate for Gadi to spam NANOG-L with botnet crap, regardless of how highly respected he is in his field.

Addressing the complaint that my response to Gadi was too harsh, I can only say that, to someone who isn't aware of the history, my response may seem harsh, but anyone who has seen the endless trolling of NANOG-L, the numerous requests (public and private) asking Gadi to cut it out, the extensive discussions on IRC, in private email and elsewhere will understand that the forcefulness of my request is appropriate given the fact that all previous attempts to end this needless disruption of NANOG-L have been ineffective.

Addressing the complaint that my response to Gadi was too harsh, I can
only say
that, to someone who isn't aware of the history, my response may seem
harsh,

I *AM* aware of the history and your response seems harsh. Especially so
because you complained about a message which was about exploits in CPE
access routers, not botnets. Any kind of router vulnerability/exploit is
on topic for NANOG. And people who don't take the trouble to read
messages and critique the message content, should not post to the list
at all. We don't need you using NANOG to fight your personal flamewar
with Gadi.

but
anyone who has seen the endless trolling of NANOG-L, the numerous

requests

(public and private) asking Gadi to cut it out, the extensive

discussions

on
IRC, in private email and elsewhere will understand that the

forcefulness

of my
request is appropriate given the fact that all previous attempts to

end

this
needless disruption of NANOG-L have been ineffective.

Well, since I have some knowledge of these communications and the fact
that a number of people have thanked Gadi for his work and urged him to
continue posting to the NANOG list from time to time, I do *NOT*
understand the forcefulness of your request.

The fact is that there are two sides to this story, and that the 8000 or
so NANOG members are somewhat divided on the issue. But one thing is
clear, messages like yours are not useful to any of the list members,
but many of Gadi's messages *ARE* useful to some of the list members. In
a group of 8000 people, I expect the best anyone can hope for is that
most of the messages on the list will be useful to some of the list
members.

If that isn't good enough for you, there is a mailing list committee and
a steering committee that you can complain to, but privately please, not
on the list.

--Michael Dillon

from http://www.nanog.org/listfaq.html

Appropriate Topics ... ISP security ...

I think DTAG.de is a very insecure ISP.

The router is still distributed.
There is no warning by DTAG.de.
There is no fix.

There is an ongoing discussion about a troyan developed and
distributed by the german gouvernement or their agencies.

So this router is a very likely means how they enter your home.

There is an ongoing discussion, fed mostly by our governement,
about china hacking german computers - industry espionage.

So it is very likely that china uses this governement troyan
to break into our computers.

The scenario is very likely because we do not grow computer
science people here in germany, we have to import them from
china, that is what our governement and our industry keeps
telling us.

Oh, there is a fix. DTAG.de is on strike.
If they were not, some 11 million germans might be made
into spam bots. That would effect routing world wide and
probably in north america too.

Cheers
Peter and Karin

Albert Meyer wrote:

Addressing the complaint that my response to Gadi was too harsh, I can
only say
that, to someone who isn't aware of the history, my response may seem
harsh,

I *AM* aware of the history and your response seems harsh. Especially so
because you complained about a message which was about exploits in CPE
access routers, not botnets. Any kind of router vulnerability/exploit is
on topic for NANOG. And people who don't take the trouble to read
messages and critique the message content, should not post to the list
at all. We don't need you using NANOG to fight your personal flamewar
with Gadi.

I don't see cpe as being all that different than hosts, except that
they're slower and less flexible.

The thing is it would be really nice to have some functional separation
between the business of this list which is operating a network, and the
security focused lists, and the botnet/phishing/spam lists, addressing
policy lists, the internet standards list, and so forth.

You and I and lots of other people on this list are on on many or all of
those sorts of lists. While cross-pollination is acceptable and in fact
desired dragging the business of one group of community interests in to
the domain of another is not appropriate.

In the particular case of Gadi, I resent the persistent grandstanding
and offers of assistance and assurances that's he's on the job. That's
essentially all advertising for his consulting business and I don't
think it's appropriate on this list. I for one do not flog the products
of my employer on this list, nor do you, or most other people who
participate.

I tolerate this sort of behavior in the security arena (read bugtrac
these days) though I resent the fact that it's de rigeur in the space
for many disclosures to essentially be advertising for the consultants
doing the work, virus updates are advertising for anti-virus companies etc.

I see them as more flexible - they don't have a CPE in front of them
potentially being a firewall, they can listen() on ports for p2p botnet
type action, and they can silently redirect your traffic to completely
different IPs or return bogus DNS info, they can see inside your home
network and be counted as "internal internet zone" to IE..

(perhaps not operational per-se, but pretty freaking scary.)

Adrian

[snip]

The thing is it would be really nice to have some functional separation
between the business of this list which is operating a network, and the
security focused lists, and the botnet/phishing/spam lists, addressing
policy lists, the internet standards list, and so forth.

While there persists an attitude that security isn't a core part of
running a network there will continue to be insecure networks flooded
with spam, phishing, botnets et al. I've been running wide area networks
since 1995 and I've always seen security as an operational network issue
and moreover I find incomprehensible an attitude that sees it otherwise.

You and I and lots of other people on this list are on on many or all of
those sorts of lists. While cross-pollination is acceptable and in fact
desired dragging the business of one group of community interests in to
the domain of another is not appropriate.

In the particular case of Gadi, I resent the persistent grandstanding
and offers of assistance and assurances that's he's on the job. That's
essentially all advertising for his consulting business and I don't
think it's appropriate on this list. I for one do not flog the products
of my employer on this list, nor do you, or most other people who
participate.

While Gabi is voluble I don't concur. I've never formed the impression
that he's advertising anything other than the problem or some [possible]
solutions. I've certainly never felt he was advertising his paid services
- so much so that this is the first time I was explicitly aware that he
offers paid consultancy in this area, if that is indeed the case.

I tolerate this sort of behavior in the security arena (read bugtrac
these days) though I resent the fact that it's de rigeur in the space
for many disclosures to essentially be advertising for the consultants
doing the work, virus updates are advertising for anti-virus companies etc.
  
[snip]

Can I please make a [probably futile] request.

If someone thinks something is off-topic but the subject matter is even
conceivably marginally on-topic - just skip the post. Don't start a long
discussion of the relevance. Inevitably the discussion of topicality takes up
more time and attention than the original subject would have. Whenever I see
this happen I always suspect that the operational issue is really that the
complainants don't have enough real operational work to do and I wish I had
their cushy job.

I don't. Nor do I work for a colsultancy.

Thanks,

  Gadi.

Gadi Evron wrote:

- so much so that this is the first time I was explicitly aware that he
offers paid consultancy in this area, if that is indeed the case.

I don't. Nor do I work for a colsultancy.

Your work for a vulnerability assessment vendor...

or maybe you should update your bio:

http://lifeboat.com/ex/bios.gadi.evron

Thanks,

Indeed
Joelja

Gadi Evron wrote:
>> - so much so that this is the first time I was explicitly aware that he
>> offers paid consultancy in this area, if that is indeed the case.
>
> I don't. Nor do I work for a colsultancy.

Your work for a vulnerability assessment vendor...

Thank you very much Joel. I much appreciate you clarifying that point.

... or if you are unable to resist the temptation to start a meta-discussion, do it on this list, not on the nanog list. It's on-topic, here.

Joe

Gadi Evron however is listed as one of the authors on a rather
interesting book "Botnets: The Killer Web Application":

       http://search.barnesandnoble.com/booksearch/isbnInquiry.asp?z=y&EAN=9781597491358&itm=3

(Howsabout we stand on each others' shoulders rather than each others'
toes?)

The following was recently asserted by someone on nanog-futures:

Looking at the last two... posts, one of which went unchallenged for
4 days while the other was immediately followed by an email from me
pointing out that the post was off-topic and asking him to stop; an
immediate challenge appears to reduce the number of posts on that
topic. The $TOPIC1 incident started $DATE and continued for 6 days
and 60 posts. The $TOPIC2 thread appears to have died out after 43
messages, including the messages that discussed its off-topic-ness
(and including the messages crossposted from nanog-futures).

The individuals involved are not germane and so have been redacted.

Anyway, just for the record, the majority of the threads that grow
like weeds have an "appropriateness to nanog@" component to them and
thus belong on nanog-futures.

Everyone: YOU have the power to help make the determination that
certain threads properly belong on nanog-futures and not nanog@. If
you really feel the need to reply to such threads, won't you please
consider setting reply-to to nanog-futures@{nanog.org, merit.edu} and
CCing same (I believe that as a nanog@ subscriber you are OK to post
even if you are not a subscriber; if I'm wrong please let me know and
we'll fix that)? The MLC really hates quashing threads; some good old
fashioned grass roots efforts to move threads to where they belong
would be most welcome.

By the way, if YOU care about the future direction of the community,
why not subscribe to nanog-futures@nanog.org? The list is fairly low
traffic compared to nanog@ (though we can expect some meta-discussions
over there).

Thank you,

                                        ---Rob (on behalf of nanog-admin,
                                                the nanog@nanog.org list
                                                administration team)