Outbound mail filtering on large mail / web server farms - just an idea or two that I have

I originally wrote this lot below as boilerplate for large webhosting
providers that find themselves with several racks full of pizzabox
colos running a web control panel like ensim or cpanel so that the
people actually operating the colos may not have too much clue .. and
these places are typically riddled with lots and lots of exploitable
cgi / php scripts that are broken into and used to send spam using
injection / xss etc holes ..

Some of the ideas here might well apply to what I was talking about in
this thread as well - the two kind of tie in together

Comments welcome

regards
srs

I've considered a similar setup. Requiring all mgd servers to always use their local mailers, then at the nearby edge, NATing all outbound SMTP port 25 traffic to a set of mail relays setup to do greylisting, rate limiting, and possibly IDENT checks to make (reasonable more) sure that it's the mail server user talking and not some random software.

Note that I've done none of it...the idea's a bit insane, but, it would definitely make it easier to spot and treat the problems, the only big black eye here is AOL who would probably rate limit the outbound servers quite often, which they already do to our normal mail systems even when things are going well, again, because of forwards. I'd imagine there's a way I could get just the (AOL) forwarded mail pushed to a separate machine with our current (older version) Postfix setup but I haven't actually looked into it. We use SQL based tables for everything in order to make automation much simpler on our end.

I hope this all wasn't too non-operational, it seems relevant to me, so hopefully it's not noise.

quite often, which they already do to our normal mail systems even when
things are going well, again, because of forwards. I'd imagine there's a
way I could get just the (AOL) forwarded mail pushed to a separate machine

The difference is of course that when you separate .forward traffic to
a separate IP you tell AOL its a forwarding server. And setup reverse
dns + hostname for that box that says something like
"dotforward.wgops.com" ...

Once you do that you should be reasonably good to go

with our current (older version) Postfix setup but I haven't actually
looked into it. We use SQL based tables for everything in order to make
automation much simpler on our end.

Implementing these suggestions is much simpler once you do that

Oh I understand the concept perfectly well. It's just that I can't see through to an implementation easily. The system I'm referring to has no internal way of telling the difference easily between forwarded mail and 'other' mail, it's all passed into the same set of virtual tables and the only difference is local versus remote delivery. I can't classify .forward style traffic out from the regular in/out flows. I'm probably not making a whole lot of sense either right now, lets blame that on low caffeine count. I could pass all aol.com traffic (via transport) to a different box but I can't do that just for forwarded mail because inbound mail and outbound (locally generated/initial submission on port 25/etc) aren't handled seperately at all. I can see how they could be, but I don't see any provisions inside of Postfix 2.0 to handle that without separate instances. Not impossible, just impractical right now. The eventual 'plan' is to do almost exactly that, separate instances to handle/classify mail differently based on where the mail was submitted.

I guess I'm really curious as to how others might implement something like this. I'd run three instances (machines if you must view it like that) of the MTA, one inbound the outside, say $world, the other $local, the third (call it $forward if you will) is where $world would send all of it's forwarded/outbound mail to that won't be delivered locally. $local would handle local delivery and external delivery for local machines. I think in Exim though it's cleaner because you can specify special processing for other steps. My system has a bit more complication because of the fact we don't use any filesystem. The mail users are purely virtual to the mail system, only existing as an LDAP entry and as a Cyrus Mailbox.

Separate your MXs from your outbound servers

And have local traffic rewritten to an internal domain (say
wgops.com.internal) just for local delivery .. all other outbound
traffic on your MXs can be assumed to be either

* Bounces [route em out of a separate host as well - tracking those
helps block a ton of spam]

* Forwarded emails

Your outbounds of course go out via a separate set of boxes.

Multiple machines. Or multiple postfix instances on the same box. Whatever

--srs

Damn. The reason I wrote that is coming back to me, VERY PAINFULLY :frowning:

There's someone out there sending out spam advertising "hoodia" (diet
pills made from a cactus of some kind) in huge quantities through
hacked contact forms. My staffers are getting about two or three
dozen tickets from various people who are on servers blocked since the
last 2..3 days because of this kind of spam.

And right now the spam levels for this lot are spiking sky high

If any webhosts are listening - please sweep your network for this
stuff pronto. And please implement what I suggested.

srs

Our most common successful spam incidents involve exploited
vulnerabilities in web forms. It's difficult for spammers to get email out
of our network, because we block port 25, our MXs only accept incoming
email, and our outgoing relays have names that spammers can't be bothered
to find out. However, web forms come preconfigured, so if the spammer can
exploit it they don't have to know anything about our email setup. Secure
SMTP between the web server and the outgoing relay won't help.

Recent versions of Exim have a rate-limiting feature which I am using to
mitigate this vulnerability - though it's hard to deploy without
disrupting legitimate users.

Tony.