Misguided SPAM Filtering techniques

Mailman List Mirror (Read Only)
21

Dave Pooser wrote:

I call BS. I ran sender-callout verification on my primary email server for
a while (before I became convinced it was mildly abusive, and stopped) and
typically blocked 2-3 spams per day. In fact, I had more FPs than legit spam
blocked by that method.

2-3 spams a day? That's really an amazing low number. You can call BS all you want. I'll stick to my numbers as they are what my reports were telling me. Is it possible that the email address in question was listed somewhere on the list that viruses used to send forged email more than other spammers? That's completely possible. Still, my results are what I observed when I went looking at the statistics over a 6 month period. I was actually looking for other statistics, the reduction in overall spam levels after implementing gray listing, which was the next 6 month's statistics.

Absolutely I am. If you're going to try to offload your spam filtering to
me, I want the process to cause you as much pain as possible (within ethical
limits, which is why I won't forward your email

It's OK, really, as I;m sure that your email address is only used once or twice total, so your validation of the email address really means nothing to the recipient. They get one spam message. If they get more, they just blacklist the address. It's what I do.

Sender callouts will verify addresses without requiring any action from the
end user. If you must [ab]use my resources to do your job, please have the
common decency to use my (abundant) hardware and software resources rather
than my (much more limited) wetware resources.

You have more information on this? I'd be more than happy to investigate another method myself that does not piss you off so much, as long as it provides the same level of isolating spam.

  -Sean
(Please respond only to the list)

22

This is another thread that while started as mildly operational ended up
as usual discussion on spam filtering, which is not on-topic.

I'll make a brief summary:

a) statement of original problem by Owen: Is blocking or proxying ports
25/587 appropriate for NSPs?

A few responses were pointing out that everything is fair as far as port
25, but port 587 should be unmolested. In the alternative, suggestion made
to use port 465 (the ssl equivalent of port 587).

Some pointed out that the filtering is a trend that everything except port
80 and 443 are blocked.

b) Discussion diverted very quickly to DKIM, SPF, SenderID,
Challenge-Response, statistics of mail filtering, blocklists, etc.

I'd like to remind everyone what is and what isn't on-topic here: Spam
filtering, in general, is *not* on-topic. Spam as a threat to network
operations is on-topic. Control of outbound spam our users send is
on-topic to a certain point - like, the original post, whether ports 25 or
587 can be blocked or proxied.

End-user discussion on spam filtering is *definitely* off-topic. Let me
reiterate: If you do not have a perspective of network operator, it is not
likely that you can contribute to the discussion in a meaningful way.
Note, in this case, 'enterprise' mail filtering is still end user, as far
as network operators are concerned.

Another example: If you can see your post being a slide of a presentation
at NANOG, then it is on-topic. If not, it isn't.

While we can all agree that network operations must include certain amount
of dealing with spam, same can be said about janitorial services.
Just because we have to deal with it, doesn't mean it is on-topic
here.

If you would like to continue discussing state-of-art for mail filtering,
there are better lists to discuss it (like MAAWG and many others).

In the light of above, please constrain the discussion to the network
operator's view of spam filtering.

As usual, if you would like to discuss this post, please do so on
nanog-futures.

-alex [mlc chair]

23

And, is it really a burden if you SEND me an email to validate yourself? If it
IS such a burden, then I invite you not to send email to start with, especially
not to me.

That would be all fine and good - if I was being asked to validate mail that
I actually sent to you. I've seen very few true positives for this, compared
to two *large* classes of false positives:

1) I'm being asked to verify my address because some malware found my address
on a hard drive and stuck it in the From: field. I'm sorry, but if you're
asking me to verify that, it *is* a burden - you are admittedly *starting off*
assuming that it's bad and *needs* some sort of verification. So by definition,
you're imposing on people to validate that they're real.

2) The rest of the time, I'm being asked to verify myself because I posted
to a mailing list, and some idiot failed to whitelist the list address.

Homework question: Does this method scale? What would happen to your inbox
if *everybody* on this list did this sort of thing?

(Bonus points for figuring out what happens when two people who *both* use
this scheme try to exchange email. Hint - my system didn't recognize your
C/R format, and concluded it was an e-mail addressed to me. What happens next?)

(Please respond only through the list)

This is NANOG. If you wish to hijack the semantics of my REPLY button,
feel free to actually include a Reply-To: field that expresses the semantics
that you desire.

24

1) I'm being asked to verify my address because some malware found my address
on a hard drive and stuck it in the From: field. I'm sorry, but if you're
asking me to verify that, it *is* a burden - you are admittedly *starting off*
assuming that it's bad and *needs* some sort of verification. So by definition,
you're imposing on people to validate that they're real.

Why would you care to validate your email address then? If you didn't send the email, and was not expecting an email from me, then why would you even bother to read, let alone validate?

2) The rest of the time, I'm being asked to verify myself because I posted
to a mailing list, and some idiot failed to whitelist the list address.

Yes, except for two things: First YOU should not get a challenge to and email that was sent by you through the list. If you are, then this is just inexcusable on the part of the software developer or admin. Second, you should only get a challenge if you "reply to all" and send a copy of the same email to someone directly.

Homework question: Does this method scale? What would happen to your inbox
if *everybody* on this list did this sort of thing?

Absolutely nothing, assuming that the the list members have a clue on how the software works and should be configured. If they don't white list the mailing list, then they are idiots that have no excuse, and quite frankly will be unsubscribed from the list due to excessive bounces. And if people followed good protocol and trimmed their headers, then there really is no good reason why anyone would get a challenge to an email that they sent to the list.

And as it is, if everyone had a c/r system, I imagine that everyone would get either white listed or validated here pretty quickly.

(Bonus points for figuring out what happens when two people who *both* use
this scheme try to exchange email. Hint - my system didn't recognize your
C/R format, and concluded it was an e-mail addressed to me. What happens next?)

Most of this type of software is specifically designed to catch loops, and as thus will stop them. When companies send me an email from an address that has an autoresponder behind it, I usually only get one or two emails before the software stops it.

This is NANOG. If you wish to hijack the semantics of my REPLY button,
feel free to actually include a Reply-To: field that expresses the semantics
that you desire.

Why should I do such a thing when it is only common (uncommon?) sense to actually do such a thing? How highly that people must think of themselves to send the same email to people multiple times.

And I only put that disclaimer in there so people don't whine about the autoresponder. Considering the group here, I'm sure that many of them actually have their mail reader set to ignore the reply-to field. These are the same that will whine about the autoresponder if I didn't let them know ahead of time.

  -Sean
(Please respond only to the list.)

Actually it looks like we're being directed to stop, so no response needed, unless you want to take it off line.

25

I use an authenticated TLS-protected mailhost at home for submitting my
email for delivery. Unfortunately, networks have taken to:

outright blocking 25 and 587 except to their own servers.

Back in the day AT&T dial-up blocked port 25 outgoing (except to their own
servers) for the first month; after that, a user could request an unblock. I
believe the SBC/AT&T Borg does the same thing with dynamic DSL IPs.

It seems to me that blocking port 25 by default and unblocking on request
would be an ideal low-maintenance solution that would reduce spam
considerably, and has the added benefit of being on-topic for NANOG.

26

For those of you who run Cisco kit; you can also use WCCPv2 to
redirect 25/TCP -in hardware path without policy routing- to a farm of
servers. Its actually documented in the WCCPv2 specification - you can
redirect arbitrary TCP/UDP ports. Think of the possibilities.
(I don't think the CRS does WCCP :stuck_out_tongue: but it'll be in hardware path on
6500/7600 on anything >= SUP2/PFC2; 3560/3750/4500/4948; It'll also be
in CEF path IIRC on software platforms.)

I've done this in my lab at home and it works fine. Whats missing is
some glue to handle transparently routed connections once they hit your
SMTP farm; I'm quite happy to help people out if anyone is interested
(and I'll even put the results in the nanog wiki if people care.)

2c,

Adrian

27

This behavior is exactly the kind of irritating misguided action which launched
my initial complaint.

The problem is that your server farm won't match my expectations for STARTTLS,
which then prevents me from engaging SMTPAUTH and sending my mail.

My mail severs aren't open relays, but, I am able to send email through them from
any non-broken internet connection.

The issue is the increasingly high percentage of internet connections which are
becoming broken. So far, the only "justification" for this behavior posted is the
inability of the folks in Redmond to deliver non-broken software such that a large
enough fraction of portable machines are able to "credential hijack" from stored
credentials on the machine and impersonate the operator while botted.

I really wish we could find a way to punish the folks in Redmond and the people
whose hosts are botted instead of punishing everyone else for their errors.

Owen

28

Owen DeLong wrote:

The issue is the increasingly high percentage of internet connections which are
becoming broken. So far, the only "justification" for this behavior posted is the
inability of the folks in Redmond to deliver non-broken software such that a large
enough fraction of portable machines are able to "credential hijack" from stored
credentials on the machine and impersonate the operator while botted.

I really don't get it. While I understand with tcp/25 blocking, there is absolutely no reason to block tcp/587. If credential's are being hijacked, it is the responsiblity of the MSA server to close the door. There's nothing to say those credentials weren't blasted to an irc server or a web script somewhere and the actual usage of them will be from some other random location on the net.

Jack Bates

29

I want to make it clear... I don't mind people filtering either 25 or 587,
but, blocking both is highly unacceptable. Even more unacceptable
in my opinion is hijacking connections to either off to your own
man-in-the-middle attack server.

Owen

30

I want to make it clear... I don't mind people filtering either 25 or
587, but, blocking both is highly unacceptable.

I can't see any operational reason to block 587.

Even more unacceptable
in my opinion is hijacking connections to either off to your own
man-in-the-middle attack server.

We had a client whose RFP vanished into thin air because of that-- he sent
it from a hotel that practiced port 25 hijacking and had had their IP
blacklisted for spewing much spam and viruses. So our server rejected the
message, and when it tried to send the NDN to him *his* server rejected the
NDN for the same reason. Fortunately he called the next day with some
details he'd omitted....

I recommended he go back with an army of Huns and raze the hotel, but he
settled for a nasty letter and using 587/TLS in future.

31

Owen,

You must have been irked by the airport wireless in ABQ then. I
couldn't figure out why my ssh connection was failing until I checked
the DNS and relized that even after clicking "free access" button in a
web browser they returned 192.168.1.1 for almost every name requested.
:frowning:

I can understand blocking outbound tcp 25. I wish more folks did it.
Blocking 587 makes no sense. The whole point of 587 is that its the
authenticated mail submission port. Its of very limited use to
spammers. Guess we'll have to move it to 443 too. :wink:

Regards,
Bill

32

Dave Pooser wrote:

We had a client whose RFP vanished into thin air because of that-- he sent
it from a hotel that practiced port 25 hijacking and had had their IP
blacklisted for spewing much spam and viruses. So our server rejected the
message, and when it tried to send the NDN to him *his* server
rejected the
NDN for the same reason. Fortunately he called the next day with some
details he'd omitted....

I recommended he go back with an army of Huns and raze the hotel, but he
settled for a nasty letter and using 587/TLS in future.

You should have used the oppurtunity to educate your customer. Email is a
best-effort, no receipt service. It is simply not appropriate to use for
business-critical communication without some kind of confirmation of
receipt.

The hotel didn't really do the wrong thing. It took the email and made a
best effort to deliver it. When it failed, it made a best effort to alert
the sender. That is what email is supposed to be like.

Obviously, they've had a spam problem. So just passing port 25 unmolested
would not be right. Blocking it is not a very good solution either because
people who are not sophisticated will just be unable to send mail. People
who are sophisticated won't be using port 25 outbound from odd net locations
anyway.

You should blame whoever decided not to accept *any* email from the hotel
just because *some* of the email was spam. That person knew or should have
know that some of that email might be business critical. Hmm, that was
*YOU*.

Perhaps you are using a misguided spam filtering technique? How many RFPs
are you willing to lose to reduce spam?

DS

33

I will trade your ABQ wireless for almost anything that uses Nomadix's
hotspot product .. the one that has a login page on http://1.1.1.1 -
even more broken dns jail, returns 0.0.0.0 if I remember correctly for
random queries till their upstream dns resolver actually decides to go
update its cache. Probably because I have a v6 aware resolver + some
of the hosts I accessed were dual v4/v6 or something, not sure.

I got a really well filled /etc/hosts file for trips through paris
airport (where the paris airport hilton charges 25 EUR a day for wifi,
and it is 9 EUR a hour at the airport, ugh)

srs

34

You should have used the oppurtunity to educate your customer. Email is a
best-effort, no receipt service. It is simply not appropriate to use for
business-critical communication without some kind of confirmation of
receipt.

That sounds like a statement from the dawn of the ARPAnet. Email is a best
effort service, sure. In an ideal world, people would not use it for
business-critical communication. But that train left the station a decade
ago; if you design your network around the assumption that email is just
going to spontaneously vanish sometimes and that's OK, you'll have lower
customer satisfaction ratings than chlamydia does.

The hotel didn't really do the wrong thing.

Yes it did. It silently hijacked traffic directed for his email server and
directed it to an unrelated server. That is never, ever acceptable behavior
for a network. Full stop. If they *insist* on hijacking a better response
would be to point all port 25 traffic except relay.cluefreehotel.dom to an
internal address with an SMTP server that did nothing but issue a 550 with a
Web page link that would show the user how to configure Outlook/ OE/
Thunderbird/ Mail.app to send via the hotel's relay server. That way the
user knows something bad is happening. The problem is then the hotel has to
deal with annoyed users, whereas with the hotel's silent hijacking solution
many users don't know enough to be annoyed until after they've left, and may
be annoyed at a third party rather than the hotel. Win for the hotel, lose
for everybody else.

Blocking it is not a very good solution either because
people who are not sophisticated will just be unable to send mail.

Blocking means people who are not sophisticated will be unable to send email
and will *know* that they are unable to send email. Silently hijacking means
those people will be unable to send email to much (though not all) of the
Internet with no idea which messages are successful and which aren't.

You should blame whoever decided not to accept *any* email from the hotel
just because *some* of the email was spam. That person knew or should have
know that some of that email might be business critical. Hmm, that was
*YOU*.

Yep, and my company's customer. Each of us had decided, independently, that
a host that appeared on a Spamhaus.org blacklist was not allowed to talk to
our mail servers. Both of us operated on the assumption that there was not a
host in the middle silently hijacking packets. Those assumptions were wrong
in this case, but not IMO unreasonable. On the bright side, the customer has
now learned to do what my staff already do, which is use an alternate port
with encryption, use VPN as a fallback plan, and failing that go somewhere
else for Internet access.