Anti-spam System Idea

Mailman List Mirror (Read Only)
1

I wanted to run this past you to see what you thought of it and get some
feedback on pro's and cons of this type of system.

I have been thinking recently about the ever increasing amount of spam that
is flooding the internet, clogging mail servers, and in general pissing us
all off.

I think it time to do something about it. very few systems are effective at
blocking spam at the server level, and the ones that exist have a less then
stellar reputation and are not very effective on top of that.

95% of spam comes through relays and its headers are forged tracking an
E-mail back that you've received is becoming next to impossible, its also
very time consuming and why waste your time on scumbags?

my idea;
a DC network that actively scans for active relays and tests them, it
compiles a list on a daily basis of compromised IP addresses (or even
addresses that are willingly allowing the relay) making this list freely
available to ISPs via a secure and tracked site.

to test a relay you actually have to send mail through it, I have a solution
for this as well, the clients are set to e-mail a certain address that
changes daily the E-mails are signed with a crypto key to verify
authenticity (that way spammers can't abuse the address if it doesn't have
the key, it get canned)

work with ISP's to correct issues on their network help completely black
list IP's from their network that are operating as an open relay and
redirect to a page that alerts them of the compromise and solutions to fix
the problem. the only way people are going to become aware of security
issues such as this is if something happens that wakes them up, if they
can't access a % of the web it would hopefully clue them in.

because these scans only need to take place once per IP per day and over a
large distribution of computers performing the tests, I don't see network
load becoming a big issue, no bigger then it currently is.

the only way to fight spammers is to squeeze them out of hiding, and that's
what I hope this system would be designed to do.

I do not have the coding knowledge to do this I will need coders, I do have
the PR skills to work with ISPs. I am also working with my congresswoman to
pave the way for legal clearance for this program.

I would greatly appreciate your input on this and anything I may have
overlooked. I would also like to know if this would be a DC program you
would run.

a lot of people argue the practical application of DC. although we know
differently this project would show them what DC can do for them and wake
them up to perhaps other DC projects.

2

How many IP addresses are there, and what percent of them are on DHCP,
and will you be able to do a scan in under a week, by which time the info will
be very stale indeed. (Hint - how long does the ISC 'Internet Domain Survey'
take to run?)

Also, read where it got the ORBS project.

I'll overlook the fact that in general, you don't know what port the spammer
backdoor malware is listening on, so you'll have to scan multiple ports. Not
going to make you very popular.

Other than that, go for it. :slight_smile:

3

There are several groups working on identifying open relays, proxies, etc
and creating lists of such ips for active blocking. For example see
http://www.spamhaus.org/xbl/index.lasso

The problem is not as much actual open relays (which are now rare and
almost universlly blocked) but open proxies - these come in all shapes
and sizes and same tools can not be used for testing it (i.e. just
sending email as you propose). Similar growing issues are with zombie PCs
which have been infected by special viruses that makes it an open proxy
that requires certain access codes and while actual virus-set code maybe
known and can be tested for, this code can be reset by the first person
who gains access to that PC and spammers do that and after that normal
testing methods may not work.

4

Tim Thorpe wrote:

95% of spam comes through relays and its headers are forged tracking an
E-mail back that you've received is becoming next to impossible, its also
very time consuming and why waste your time on scumbags?

I don't think open relays are that big a part of the picture anymore.
The rest of that 'graph is pretty close. Open proxies, insecure forms,
and asymmetrical routing is where it is at, and remote-control trojans
installed by viruses and worms is where it is going.

my idea;
a DC network that actively scans for active relays and tests them, it
compiles a list on a daily basis of compromised IP addresses (or even
addresses that are willingly allowing the relay) making this list freely
available to ISPs via a secure and tracked site.

I don't know what a "DC Network" is.

to test a relay you actually have to send mail through it, I have a solution
for this as well, the clients are set to e-mail a certain address that
changes daily the E-mails are signed with a crypto key to verify
authenticity (that way spammers can't abuse the address if it doesn't have
the key, it get canned)

As they sometimes say--"It won't scale." And for people on small pipes
or metered connections, that will be more abusive than the current
problem is.

work with ISP's to correct issues on their network help completely black
list IP's from their network that are operating as an open relay and
redirect to a page that alerts them of the compromise and solutions to fix
the problem. the only way people are going to become aware of security
issues such as this is if something happens that wakes them up, if they
can't access a % of the web it would hopefully clue them in.

ingress filtering at the edges to drop packets that have to be fraud
scales better, but I'm not sure that matters much anymore. But if
we could not do that, how will we get this handled?

because these scans only need to take place once per IP per day and over a
large distribution of computers performing the tests, I don't see network
load becoming a big issue, no bigger then it currently is.

I think you need to check your arithmetic.

5

It just doesn't work :frowning: A few years ago I developed a sendmail
milter system that would perform an open relay test on all new
IP's that attempted to send mail to or through our server. If
the test failed (open relay), the mail was rejected before it
was even sent. If the test passed, the mail was allowed through.
Once this test was performed, the status of the IP address was
recorded for 90 days, after which it was deleted and the test
would be performed again the next time it attempted to access
our mail server. The tests themselves took under 20 seconds on
average. Within 2 weeks we had a list of over 250,000 open relays.

The total cut down in SPAM: somewhere around 10%

Sadly, fact turned out to be that zombies, trojaned machines,
and proxies are the reason. Not that much SPAM is open relay
anymore.

Mike Wiacek
IRoot.Net

6

I used to agree with this, until I tried amavisd-new with spamassassin.

Yes, you have to throw a little hardware at it, but it really is an
effective solution. For my mail, it's more than 99% effective. The only
falsely tagged messages (never had a message reach the "bounce" threshold
as a false positive) are mailing list mails from people who are on
blacklists.

Because amavisd-new has support for querying mysql maps, it's trivial to
create multiple filtering policies, allowing users to select their own
through your online account management interface. Along with that is
per-recipient sender whitelists (and blacklists). And since amavisd-new
has support for most virus scanners (clamav is nice and free), it really
provides a complete solution.

Note, however, that amavisd-new works best with postfix (according to the
developer). Not sure how well it works with the others.

It was nice going back to getting around 1 spam per day in my
inbox...(over 200 are tagged or rejected every day).

This solution really antiquates the old paradigm of rejecting based purely
on status in an RBL.

I encourage everybody who runs a mailserver to read
http://www.flakshack.com/anti-spam/

Andy

7

...

I can look at virus code, see how its written what it does to the machines
ect and "crack" their entry points and scan as well, I think the system
could be adapted to scan and pre-emptivly block potential hostile hosts. To
have passwords / port knocking schemes you have to code them, all you have
to do to break it is read the code ;). (its not THAT simple but it covers
the point I think.)

8

95% of spam comes through relays and its headers are forged tracking an
E-mail back that you've received is becoming next to impossible, its also
very time consuming and why waste your time on scumbags?

s/relays/proxies/
The proxies are tough to find since they can run on any port. Some of
them even pick random ports, then "phone home" to tell the spammer which
IP/port was just created as one of their open proxies.

my idea;
a DC network that actively scans for active relays and tests them, it
compiles a list on a daily basis of compromised IP addresses (or even
addresses that are willingly allowing the relay) making this list freely
available to ISPs via a secure and tracked site.

You're a few years late. See http://dsbl.org. For a non-DC version, see
http://njabl.org.

9

If these exist then why are we still having problems? Why do we let
customers who have been infected flood the networks with traffic as they do?
Should they not also be responsible for the security of their computers? Do
we not do enough to educate?

...> addresses (or even

10

If these exist then why are we still having problems?

See my reply to the thread "SMTP relaying policies for Commercial ISP
customers...?" -- we have problems because the spammers are a lot smarter
than any of us and can bounce from one infected host to another, in an
attempt to evade network-specific traps, and few ISPs do anything at all
to stop them.

Why do we let customers who have been infected flood the networks with
traffic as they do?

Very good question.

Should they not also be responsible for the security of their computers? Do
we not do enough to educate?

Yes, and no.

11

If these exist then why are we still having problems?

Because the spammers are creating proxies faster than any of the anti-spam
people can find them. Evidence suggests, at least on the order of 10,000
new spam proxies are created and used every day by spackers
(spammer/hackers).

The relative insecurity of windows and ignorance of the average internet
user has created an incredibly target rich environment for the spackers.

Why do we let customers who have been infected flood the networks with
traffic as they do? Should they not also be responsible for the security
of their computers? Do we not do enough to educate?

Economics, and convenience outweighing security. We're big, and slow to
change. They're small and mobile.

12

> If these exist then why are we still having problems?

Because the spammers are creating proxies faster than any of the anti-spam
people can find them. Evidence suggests, at least on the order of 10,000
new spam proxies are created and used every day by spackers
(spammer/hackers).

Add to that (or part of that number) is that many DSL and cable providers
use DHCP to assign ip addresses for short period of time to their customers.
Typically whenever system is reset a new ip would be assigned and a few of
the zombie viruses being installed on the user system causes it to become
unstable (especially if its trying to send email and can not and keeps
retrying after the ip is on blacklist) and those users begin to reboot the
computer trying to get it to work properly resulting in those computers
getting new ip addresses which would again be outside of blacklist

> Why do we let customers who have been infected flood the networks with
> traffic as they do? Should they not also be responsible for the security
> of their computers? Do we not do enough to educate?

Just completely blocking access to those users seems an overly agressive
punishment (which actually caused quite a few angry users who left their
dsl provider). Some providers deal with this by blocking port25 or redirecting
it their own smtp server - some even do it onj their networks for all
customers no matter if they got any reports or not (as preventative measure).
While there are many techs who don't like this practice it does seem that
this solution effectively removes the PC from being used as source of
spam even if it becomes a zombie.

13

jlewis@lewis.org wrote:

> If these exist then why are we still having problems?

Because the spammers are creating proxies faster than any of the anti-spam
people can find them. Evidence suggests, at least on the order of 10,000
new spam proxies are created and used every day by spackers
(spammer/hackers).

The relative insecurity of windows and ignorance of the average internet
user has created an incredibly target rich environment for the spackers.

> Why do we let customers who have been infected flood the networks with
> traffic as they do? Should they not also be responsible for the security
> of their computers? Do we not do enough to educate?

Economics, and convenience outweighing security. We're big, and slow to
change. They're small and mobile.

The Internet's spam load could be easily cut by 50% or more. All it would
take is the cooperation of most major ISPs and academic institutions.

As this discussion thread has indicated, most spam originates from systems
infected with spamiruses or open proxy servers. How to shut down all such
malware? Simple: Apply egress filtering ACLs to all border routers to prohibit
outgoing port 25 connections from DHCP addresses.

We find that at least 85% of all spam originates from DHCP addresses. Thus, if
a significant number of ISPs would perform port 25 egress filtering, I believe
that it would significantly reduce spam, and force criminal spammers to develop
completely new spamming technologies.

If ISPs were to go further, and require their customers with static IPs to
perform port 25 egress filtering, blocking such connections from all systems
except for the customer's legitimate MTA, we could virtually eliminate spam
originating from hijacked systems.

OK, I can hear the objections now... ACLs slow down our routers and thus reduce
through-put. Well, that may be true in the purest sense of the argument, but can
you demonstrate that a few ACLs will have a SIGNIFICANT impact on through-put?
I would be willing to bet that any through-put reduction caused by ACLs, in the
long run, would be more than compensated for by the corresponding reduction in
spam traffic passing through the router. Also, if filtering was to occur at the
point closest to the source, rather than at an aggregation point, the impact of
any ACLs would be distributed across the network in such a manner as to probably
have no observable impact on network through-put.

(If anyone has any hard statistics on ACL impact on network through-put, I would
sure like to see those studies!)

Just my $0.02 worth...

Jon R. Kibler
Chief Technical Officer
A.S.E.T., Inc.
Charleston, SC USA
(843) 849-8214

14

DialUp Lists (DUL) dns block lists permits you to ignore e-mail from
many dynamic IP addresses. You can configure your mail server to do this
today without waiting for ISPs to do anything.

Like most other "simple" solutions, how effective is it?

15

If we advertise the DHCP pools for AS1312 in a DUL, we solve the problem for
those sites that use the DUL we list them in.

If we block outbound port 25 SYN packets from origin addresses in the DHCP
address blocks, we solve the problem for everybody.

16

No...you just speed up the migration (which has already begun) to spam
proxies that use the local ISP's mail servers as smart hosts. Then you
have to come up with a way to rate-limit customer outbound SMTP traffic.

BTW...who brought SARS (or more likely just flu) to nanog30? I drove (so
I didn't catch it on the plane) and symptoms (sore throat, congestion,
very high fever) started thursday. I've spent most of the weekend in bed
waiting to die.

17

> DialUp Lists (DUL) dns block lists permits you to ignore e-mail from
> many dynamic IP addresses. You can configure your mail server to do this
> today without waiting for ISPs to do anything.

If we advertise the DHCP pools for AS1312 in a DUL, we solve the problem for
those sites that use the DUL we list them in.

What if I told you about a method to identify the type of connection for
every IP address in our DNS? You don't need to rely on third-party DUL
lists.

Blocking is a binary decision. Instead if you have better information
about the connection source, you can make different decisions how to
handle the message.

If we block outbound port 25 SYN packets from origin addresses in the DHCP
address blocks, we solve the problem for everybody.

Including the people who don't want you to solve it for them.

People want to use outbound port 25 from dynamic address blocks. Why
block it between people who want to use it just because some people
want to have open servers?

Block 119, you must use your ISPs NNTP server.
Block 6667, you must use your ISPs IRC server
Block 80, you must use your ISPs HTTP proxy.
Block N, you must use your ISPs whatever server.

Enterprises already do this, the equipment exists. Why do we want ISPs
doing this?

18

Hmm.. color me dubious, but keep talking. Best bet here would probably be
some interesting abuse of PTR records?

19

Sean Donelan wrote:

DialUp Lists (DUL) dns block lists permits you to ignore e-mail from
many dynamic IP addresses. You can configure your mail server to do this
today without waiting for ISPs to do anything.

Like most other "simple" solutions, how effective is it?

We block known dialup netblks. Catches < 5% of spam. Why? Because the real
culprits are xDSL, CABLE and other systems with broadband connections. These
account for about 80% of the spam attempts we observe.

The idea here is not just to prevent the receipt of spam (which is what
DNSBLs can accomplish), rather, it is to prevent the generation of spam
that is accounting for such a growing amount of everyone's network traffic.

If you block the ability of non-legitimate MTAs (such as open proxies and
spamiruses) to send spam, you reduce the network bandwidth waste that spam
is consuming. (As a side effect, you would also reduce the spread of viruses
by email.)

20

You wouldn't be too far off.

It depends on whether you consider the ISP a cooperative partner or a
hostile participant.

Not only are 3rd party block lists often out-of-date and difficult to
update, the public has a hard time understanding the difference between
an ISP voluntarily listing their IP addresses in a DUL list and being
labelled a "spam haven" because their IP addresses are in a block list.

If you assume the ISP wants to help (which you also have to assume
for a port 25 blocks to work), how can an ISP provide first-party
information about the status of an IP address on demand to anyone?

My idea is to follow the RFC1101 example.

PTR records already have other uses and requirements. So I suggest using
another record type which doesn't have a current meaning in the reverse
DNS. Instead use something like a HINFO record.

1.0.168.192.in-addr.arpa in ptr some1.example.net
        in hinfo Dynamic Dialup
2.0.168.192.in-addr.arpa in ptr some2.example.net
        in hinfo Static xDSL

The ISP (or really the network administrator for the network block) is
in the best position to know how the IP addresses are managed. The
netadmin can keep the HINFO records up to date, or correct the record if
they are incorrect. You don't need to guess which DUL maintainer contains
records for various networks or worry about a DOS attacks on a few DNS
servers affecting mail service globally. You always query the network
administrator's DNS servers when you receive a connection from an IP
address for information about that IP address.