RE: ISP wants to stop outgoing web based spam

Mailman List Mirror (Read Only)
1

I guess I wasn't clear enough in my first posting. I am not interested in smtp (port 25 spam). We have that covered. I am only interested in blocking outgoing web based spam. A user sits and sends out spam via automated tools via Hotmail, Yahoo, Gmail, or whatever Webmail system where they have set up thousands of throwaway users. An antispam proxy (that I want to install and manage) has to be able to come between the user on his/her PC and the Hotmail system and scan the http posts and page templates for things like number of receipents and other tricks like keeping track of the number of http posts. It has to maintain a list of known free webmail systems that are abused.

Based on my stats from Spamcop, 60% of all outgoing spam is http based rather than smtp based. Others may have slightly higher or lower numbers.

So, is there any magic fu out there to solve this?

Thanks,
Hank Nussbacher
http://www.interall.co.il

2

Hi Hank,

Have you had any luck combining Squid in a transparent proxy
configuration with SpamAssassin? A commercial plugin like Cloudmark
might provide better performance (since it doesn't have to evaluate
thousands of regex rules for each connection).

How to run Squid as a transparent proxy:
http://wiki.squid-cache.org/SquidFaq/InterceptionProxy

I haven't figured out how to get Squid to let you run a script to scan
and modify requests that are passing through. If you can figure that
out I'd love to know!

Otherwise, you might try looking at a couple of security auditing
proxies:

http://www.parosproxy.org/functions.shtml (Java)
http://www.immunitysec.com/resources-freesoftware.shtml (Spike Proxy,
Python)

.. Or you could roll your own simple CGI script that accepts web
queries and uses LWP or another simple package to fetch the results --
scanning for spam at the same time.

Regards,
Ken Simpson
MailChannels

Hank Nussbacher [09/08/06 18:11 +0300]:

3

[original message edited for brevity--m.black]

Based on my stats from Spamcop, 60% of all outgoing spam is http based rather than smtp based. Others may have slightly higher or lower numbers.

So, is there any magic fu out there to solve this?

Thanks,
Hank Nussbacher
http://www.interall.co.il

Maybe I'm just an ignorant e-mail postmaster. I thought that
nearly all e-mail was (E)SMTP-based (LMTP excepted).

If it doesn't use the SMTP protocol, it's not reaching any
mailbox. HTTP is a web browser protocol. WebMail gets converted
by the web server and is subsequently routed using SMTP.

matthew black
network services
california state university, long beach
1250 bellflower boulevard
long beach, ca 90840-0101

4

Maybe I'm just an ignorant e-mail postmaster. I thought that
nearly all e-mail was (E)SMTP-based (LMTP excepted).

If it doesn't use the SMTP protocol, it's not reaching any
mailbox. HTTP is a web browser protocol. WebMail gets converted
by the web server and is subsequently routed using SMTP.

I think he's talking about blog spam, which is definitely submitted
over HTTP.

Regards,
Ken

5

Ken Simpson wrote:

Maybe I'm just an ignorant e-mail postmaster. I thought that
nearly all e-mail was (E)SMTP-based (LMTP excepted).

If it doesn't use the SMTP protocol, it's not reaching any
mailbox. HTTP is a web browser protocol. WebMail gets converted
by the web server and is subsequently routed using SMTP.

I think he's talking about blog spam, which is definitely submitted
over HTTP.

I think that the person who started this thread is talking about spam coming from the wide variety of old, poorly written form handler scripts and other programs that at some point in the program talk to the mail program on the web server and thus allow an attacker to hijack said script for the purpose of using that script to amplify their spam message(s).

As a web hosting provider I have had to shut down numerous scripts on my client's websites because of this reason.

The question that I think is being asked here is how does one go about ensuring that email coming from a web form is actually a valid contact email and not a spam amplification attack. If there are measures that can be taken, what are those measures?

Gregory Kuhn
Coast to Coast Hosting

6

Maybe I'm just an ignorant e-mail postmaster. I thought that
nearly all e-mail was (E)SMTP-based (LMTP excepted).

If it doesn't use the SMTP protocol, it's not reaching any
mailbox. HTTP is a web browser protocol. WebMail gets converted
by the web server and is subsequently routed using SMTP.

I think he's talking about blog spam, which is definitely submitted
over HTTP.

I thought it was pretty clear that he was talking about e-mail spam submitted using HTTP to webmail services like hotmail, yahoo and gmail:

7

I thought it was pretty clear that he was talking about e-mail spam
submitted using HTTP to webmail services like hotmail, yahoo and gmail:

I guess I'm still a little confused about the poster's original
request. It sounds like he is interested in stopping his own users
from spamming via web-based email services such as Gmail and Hotmail,
or via insecure forms. That can be accomplished hypothetically by
filtering HTTP requests and looking for spam in POSTs; although with
the proliferation os AJAX-style interfaces in these services, figuring
out which POSTs refer to a message submission is far more difficult
than it was in the good old Web 1.0 days.

Regards,
Ken

8

Similar. Picture this ...

1. A satellite connectivity provider, that provides connectivity to
huge swathes of west africa, among other places.

2. West african cities like Lagos, Nigeria, that are full of
cybercafes that use this satellite connectivity, and have a huge
customer base that has a largish number of 419 scam artists who sit
around in cybercafes doing nothing except opening up free hotmail,
gmail etc accounts, and posting spam through those accounts, using the
cybercafe / satellite ISP's connectivity.

3. The cybercafe / satellite IP shows up in a Received: or
X-Originating-IP type header in the spam that results.

4. The satellite provider really needs to do something about this -
something proactive, because trying to whack cybercafe based scam
artists after the fact is just not going to work.

5. So - a spamassassin plugin to a squid or other transparent proxy,
for outbound filtering.

Something that can be rolled out at the satellite provider level, or
probably at the cybercafe level, and with an attached alert mechanism
that logs the spamming IP, and the mac address of the PC that's
sending the spam that got caught. Something that ISPs in west africa
that operate on wafer thin margins, and resell satellite connectivity,
can easily afford.

Oh - and something that is not the usual kind of corporation / library
type firewall [those would do this, but they'd roll over and die at
the least hint of actual production use in this kind of scenario .. as
some ISPs who deployed these in W. Africa apparently found out]

I got asked this way back in 2005, and then talked to Justin Mason of
the spamassassin project. He was of the opinion that it could be done
but he wasnt too aware of anybody who had tried it, plus he didnt
exactly have much free time on his hands for that.

Anybody who can do it - with open source and reasonably low costs,
plus ISP grade scalablity - please do let me know. I know some people
(including govt / LE) who would be just as interested as Hank is.

-srs

9

Typical SMTP headers of http based spam:

Received: from pmx2.montclair.edu (smtp-in.montclair.edu [130.68.1.65])
  by broadway.montclair.edu
  (iPlanet Messaging Server 5.2 HotFix 1.21 (built Sep 8 2003))
  with ESMTP id <0J3Q0067VUMZAF@broadway.montclair.edu> for
  x; Wed, 09 Aug 2006 14:42:35 -0400 (EDT)
Received: from pmx2.montclair.edu (localhost [127.0.0.1])
        by localhost (Postfix) with SMTP id 032883F01 for
<x>;
  Wed, 09 Aug 2006 14:42:35 -0400 (EDT)
Received: from tw4.telgua.com.gt (tw3.telgua.com.gt [216.230.128.5])
        by pmx2.montclair.edu (Postfix) with ESMTP id 8F6993F03 for
  <x>; Wed, 09 Aug 2006 14:42:35 -0400 (EDT)
Received: from intelnet.net.gt (unknown [10.160.3.1])
        by tw4.telgua.com.gt (Tumbleweed MailGate) with ESMTP id
72D1748A5C673; Wed,
  09 Aug 2006 13:42:51 -0500 (CDT)
Received: from [10.160.3.30] (Forwarded-For: [xx.56.145.19])
  by messaging.telgua.com.gt (mshttpd); Wed, 09 Aug 2006 12:39:46 -0700

The key here is the bottom Received with the mshttpd. Only once it hits telgua.com.pt (this is just an example of the dozens I see per day), does it get converted into smtp, but the xx.56.145.19 IP is the one that gets listed in spam BLs.

Basically, the state of blocking outgoing spam hasn't progressed in the past 4 years. Bummer.

Hank Nussbacher
http://www.interall.co.il

10

Shouldn't most of freemail/webmail services be doing their own outbound spam and virus checking now?

When the user connects to the freemail/webmail service, hopefully with some type of authentication, outbound messages from the
freemail/webmail's service affects the reputation of that service. If the scanning is done at the "application layer" at the freemail/webmail system, it has more knowledge about the application,
e.g. detecting mass "forwards", mailing lists, appended signature blocks, etc that may not be easily detectable form the user interface. And then it
becomes the application service providers responsibility to maintain
its effectiveness.

Its no different whether I connect to my "home" mail service using HTTP/HTTPS, MSA-AUTH, SSH, TELNET, MS-RPC Exchange, etc. If I happen
to be travelling on some random network, I still want to use the reputation of my "home" mail server not the random network I'm using.

Of course, some freemail services aren't very good about "know their customer" when new users sign up. Anyone can get lots of different
username accounts on some freemail services. If you believe some freemail services are too important to filter, some ISPs are looking at the next "received" header for their filtering.

Nevertheless, if an ISP is interested in application layer filtering and
deep protocol inspection (i.e. it may go through a proxy, so its not really "packet' inspection anymore), there are some open source and
commercial systems that could be modified to do this. They are usually advertised for classified information/parental control/employer control systems. For software installed on the PC itself, e.g. cybercafes, most major anti-virus and parental control software vendors already are web-mail aware, and scan incoming messages. They may be able to scan outgoing messages too. But I don't believe they've thought about using them for outbound spam filtering for web-mail. The network
content control systems are a bit more specialized. There are some
high-end "firewalls" typically bought for military gateways which claim
to be able to do full content inspection of webmail transactions.

11

Yes, Sean - they are. But it is far, far more productive for the
source of this abuse to be choked off. Call it the difference between
using mosquito repellant and draining a huge pool of stagnant water
just outside your home.

srs

12

Do we really want ISPs to become the enforcers for every Internet application someone may use or abuse? Webmail, online game cheating, blog complaints, auctions disputes, instant message harrasment, music sharing, online gambling, etc.

Imagining you are going to stop drug dealers by removing public pay phones isn't addressing the real source of the problem.

13

The MAAWG bcps, for example, state that ISPs must take responsiblity
for mitigating outbound spam and abuse.

Whether the problem is bad enough for an ISP to put in automated
filtering instead of dealing with abuse reports on a case by case
basis, is a call for the ISP to make.

For example, egress filtering / bcp38, port 25 blocking, route filters
to stop martian packets and leaked routes from propogating .. or
network level filtering slammer and other worm traffic for that
matter.

srs

14

The RIAA, for example, states that ISPs must take responsibility for
mitigating copyright infringement by its users.

Lots of groups state that ISPs must take responsibility for lots of things.

Abuse is a very open ended term. There is a difference between enforcing network/service rules such as preventing address forgeries, and being responsible for abuse or disputes between users

Is the ISP responsible for mitigating all types of user abuse? Or only
some types of abuse by users? For example, are ISPs responsible for mitigating liable, slander, defamation, harrasment, theft, counterfeting, gambling, intolerance, public morals, etc?

People shouldn't confuse ISPs with law enforcement or courts. ISPs are responsible for enforcing network standards and its contracts. ISPs are not responsible for solving the world's problems. If the RIAA has a
dispute concerning copyright infringement with a user, the RIAA sues
the user to stop the user. ISPs aren't expected, yet, to scan users traffic to prevent copyright abuse.

If you don't care which mosquitoes you kill, you could drain the swamp by cutting off the entire country of Nigeria. But the reality is all
the criminals aren't limited to one place. Almost none of the criminals
would even notice. But you will probably harm a lot of innocent Nigerians by doing that; and the smarter criminals will just migrate to new pastures and keep attacking you. Unlike mosquitoes, criminals aren't
limited to breeding in only certain areas.

The "source" isn't the ISP, the source is the criminal. If you can figure out a way to permanently ban criminals from every ISP in the world other
than putting them in jail, you might have a shot with BCPs for ISPs. But even if there was only one ISP remaining in the world, with a single
unified user database, I suspect criminals would still use their skills
such as identity theft and fraud to get on the net.

The goal needs to be arresting the bad guys. The problem isn't the ISP,
its the criminal. Bad packets rarely spontanously occur on the net. Every exploit, every virus, every worm, every phishing mail started with a person. Letting the bad guys go free is just teaching the criminals how to improve their skills.

15

> The MAAWG bcps, for example, state that ISPs must take responsiblity
> for mitigating outbound spam and abuse.

The RIAA, for example, states that ISPs must take responsibility for
mitigating copyright infringement by its users.

Oh - but maawg (http://www.maawg.org) is a group of ISPs themselves
(AOL, comcast, charter, france telecom, Hotmail, us ..)

Lots of groups state that ISPs must take responsibility for lots of
things.

Lots of ISPs together stated that ISPs must take responsibility for a
few things.

Small, but significant difference there, dont you think?

srs

16

2. West african cities like Lagos, Nigeria, that are full of
cybercafes that use this satellite connectivity, and have a huge
customer base that has a largish number of 419 scam artists who sit
around in cybercafes doing nothing except opening up free hotmail,
gmail etc accounts, and posting spam through those accounts, using the
cybercafe / satellite ISP's connectivity.

If we get abuse like that from a Cybercafe, and we have in the past, we block
their IP address allocation on our webservers. It is up to the cybercafe
owner to police his space, or suffer the consequences, just like any other
ISP.

If the question is how can he police his space, well I'm sure technical
solutions are possible, but there are very cheap human solutions, along with
keeping a functional abuse address.

I got asked this way back in 2005, and then talked to Justin Mason of
the spamassassin project. He was of the opinion that it could be done
but he wasnt too aware of anybody who had tried it, plus he didnt
exactly have much free time on his hands for that.

I suspect there are sufficient free email servers using HTTPS, that it is
pretty much impossible to spot this kind of thing from content inspection, at
least not as a long term solution.

Certainly if you assume content inspection is impossible, or at least
unreliable as a long term solution, you are left with traffic analysis. I
suspect IP addresses doing automated abuse have distinctive patterns, but the
risk of false positives must be reasonably high. Simple analysis tools
applied to a Squid log would show volume of HTTP traffic and other stuff.
Provide them a login when they pay, and you can immediately know who it is as
well. There are even real time analysis tools for Squid logs.

The webmail provider on the other hand can easily and cheaply check if content
from one member is suspicious in either content or volume, and suspend the
account. So perhaps you are trying to apply the solution in the wrong place.

17

Being a webmail provider - yes, I've got measures in place. This is
for ISPs who provide connectivity to mitigate abuse at their end as
well.

18

Lots of groups state that ISPs must take responsibility for lots of
things.

Lots of ISPs together stated that ISPs must take responsibility for a
few things.

The movie industry joined together and introduced the Hays Production
Code. The comic book industry joined together and introduced the
Comic Book Code. Their respective industries took responsibility for a few things. The result was the moviegoing and comic buying public was effectively blocked from alternative choices and attempts by smaller independent studios to create movies and comics outside of established codes were punished by the industry members.

Small, but significant difference there, dont you think?

There is a small, but significant difference, between ISP's providing
good, bad or no anti-virus, anti-spam, anti-x filtering on messages being received by customers that want those services; and a group of ISPs deciding to enforce common terms and conditions on customer behavior above and beyond what is necessary to protect and operate the network on unwilling customers that don't want to accept those T&Cs.

As soon as you say ISPs "must," the compulsory nature of the business terms and conditions is a necessary, but problematic condition.

A group of 100 ISPs decide on particular terms and conditions doesn't
mean the other 30,000 (or whatever the current count is) ISPs must
agree to the same terms and conditions.

Perhaps a small, but significantly different way to phrase it:

A group of X ISPs agreed to accept responsibility for abuse by their users.

19

* Hank Nussbacher:

I guess I wasn't clear enough in my first posting. I am not
interested in smtp (port 25 spam). We have that covered. I am only
interested in blocking outgoing web based spam. A user sits and sends
out spam via automated tools via Hotmail, Yahoo, Gmail, or whatever
Webmail system where they have set up thousands of throwaway users.
An antispam proxy (that I want to install and manage) has to be able
to come between the user on his/her PC and the Hotmail system and scan
the http posts and page templates for things like number of receipents
and other tricks like keeping track of the number of http posts. It
has to maintain a list of known free webmail systems that are abused.

Your are tackling this from the completely wrong angle, I think.

You should look after the automated tools (probably using a virus
scanner or something like this) and trigger a covert alert once they
are detected. If the spam sent out is of the right kind, you can
phone the police and have the guy arrested.

This assumes that the miscreants actually visit the Internet cafe. If
the spamming is purely malware-based and non-targeted, the spamming
problem simply disappears once you get rid of the malware problem.

20

* Suresh Ramasubramanian:

Yes, Sean - they are. But it is far, far more productive for the
source of this abuse to be choked off. Call it the difference between
using mosquito repellant and draining a huge pool of stagnant water
just outside your home.

How can I, as an ISP, stop abuse that is carried out over HTTPS?

There are technological solutions for intercepting HTTPS traffic, but
I don't think we want to put them to even wider use.