Spamhaus...

Mailman List Mirror (Read Only)
1

From nanog-bounces+bonomi=mail.r-bonomi.com@nanog.org Fri Feb 19 22:32:48 2010
From: William Herrin <bill@herrin.us>
Date: Fri, 19 Feb 2010 23:32:10 -0500
Subject: Re: Spamhaus...
To: Larry Sheldon <LarrySheldon@cox.net>
Cc: nanog@nanog.org

>> "If an SMTP server has accepted the task of relaying the mail and
>> later finds that the destination is incorrect or that the mail cannot
>> be delivered for some other reason, then it MUST construct an
>> "undeliverable mail" notification message and send it to the
>> originator of the undeliverable mail (as indicated by the
>> reverse-path)."
>
> Does the RFC say what to do if the reverse-path has been
> damaged and now points to somebody who had nothing
> what ever to do with the email?

Hi Larry,

Re-reading the paragraph I quoted and you repeated, I'm going to say
that the answer is "yes."

I'll bite. *HOW* do you send to the _originator_ (as *required* by
the RFC you quoted) of the undeliverable mail, when the reverse path
points to 'someone else'?

Note well the exact lanugage used -- it does not say 'the party named
in the reverse path', the 'claimed sender', 'putative sender' or any
other similar equivocation that justifies sending to a forged address.
It says "the originator". To me, that can be only iterpreted in _one_
way. To wit: as the party that _actually_ created and transmitted the
message, _regardless_ of what the declared return path is.

Since such a message is 'defective' (not RFC-compliant -- because the
true point -of-origin is *NOT* in the reverse path, as it MUST be for
an RFC-compliant message) on it's face, I will argue that there is no
need to apply the 'required' handling for a 'proper' message to it.

2

> Does the RFC say what to do if the reverse-path has been
> damaged and now points to somebody who had nothing
> what ever to do with the email?

Do the TCP RFCs say what to do in response to a SYN packet, if the
source IP address has been damaged, and now points to some source IP
that has "nothing whatever" to do with the packet?

You can only do the best possible -- discard the packet, if
something allows you to determine the SYN is obviously spoofed or
invalid, sure, drop it, otherwise you have no real choice. With
SMTP there are many cases where you have no choice but to 250 the
message, and send a DSN/bounce back later.

E-mail users expect that when they send someone a message, it gets
there in 6 hours or less, or they get an error within 6 hours. The
purpose of SMTP protocol is to provide reliable mail delivery, which
includes reporting errors.

Spurious DSNs are less harmful than missing DSNs. Spurious DSNs can
be discarded easily by the mail server that knows it didn't pass that
message. DSNs that were not generated cannot be recovered.

Discarding is currently the responsibility of the mail server whose
address has been forged. Just like it's the responsibility of a host
whose source address was forged in a TCP transaction, to discard the
"ACK" packet for a connection that resulted from a spoofed SYN.

The mail server sending DSN for the fake message, or replying to a
spoofed SYN is not a spammer in any way, they are actually a victim
wasting their own bandwidth responding to a bogus message. They
have no real choice in the matter -- Weaknesses in SMTP protocol and
lack of SPF implementation forced them into this position, they
can't tell if the "MAIL FROM" line is real, anymore than a host on
the internet can look at a source IP on a packet and know it's real or
not.

In the general case, Basic DNS and SPF tests are basically all the
receiving mail server has to work with in "validating" return path.
    And they should perform these tests, before responding 250.

When you responded with "250 Message accepted for delivery", that
says you were satisfied that the message was legitimate, and the
RETURN PATH and TO address is verifiable as far as you can tell.
You can't NOT send DSNs if a failure occurs after that point,
that is contrary to the requirements for reliable mail delivery.

Rliable storage and delivery of legitimate messages is just as
important as suppression of noise. Without reliable delivery, we
don't really have a such thing as "internet mail" anymore.

And by the way, a backup MX cannot verify the recipient when the
primary MX is down. Especially when the backup MX is hosted
off-site by another provider.
The job of the backup is to hold mail in queue until the main mail
system is back in operation, so "recipient verification" cannot
actually be guaranteed.

The perimeter MX also has no idea, that the recipient's mailbox has
run out of disk space and cannot store the message, or that the alias
points to a catch-all address on a different provider's mail server,
where the user's account has been deleted.

Some backscatter is to be expected as long as we have a reliable mail system,
and it relies on returning messages via DSN to unverifiable MAIL FROM addresses.

I only really see three options here.... give up on reliable mail delivery
(might as well abandon SMTP entirely then, and replace it with a
simpler protocol), revise SMTP to allow authentication of 'MAIL
FROM',

Or revise SMTP to require somehow validating the entire delivery path
before "250 OK" can be accepted for any RCPT TO line.

As in, eliminating the ability for mail servers to 'queue' messages
for delivery.
Instead when you issue 'RCPT TO', your mail server must
immediately connect to the next hop mail server, start the SMTP
transaction, and get an OK for that 'RCPT TO' prior to returning
"Recipient OK" back to you.

So you getting 250 OK to "RCPT TO" means every mail server in the
delivery path simultaneously has a port 25 connection open to the
next hop, has just returned 250 to the same RCPT TO line.

3

I'm really amazed the thread police haven't pulled this one over and hauled it off to jail. The questions of when/whether/and to who bounces should be sent is a debate for spam-l or nanae.

IMO, the original question in this thread was on-topic, but unfortunately it got very little discussion before things devolved into "why are you sending bounces?" and "I suppose you can't read the RFCs."

The original question, "what do you do (or have you done) when DNSBL-X approaches you saying that your network is hitting their public NS's too hard and wants you to pay for continued access?" is something I'd like to see some answers to. Despite the Subject:, Spamhaus is neither the only DNSBL currently doing this nor the first to watch statistics on their public NS's and approach networks asking for money and/or cutting off access if you don't pay.

Maybe you run a mail cluster that uses DNSBL-X. Maybe you haven't even heard of it, but you have enough customers using it, and querying through your caching DNS servers that your network has come up on their radar as a "heavy user". Telling your heavy user customers to stop using your DNS cache probably won't help. I know at least some of these orgs aggregate queries either per RIR assigned CIDR or per ASN, so spreading the queries out isn't likely to get you around the issue.

So, do you pay, and setup your own local copy of the zones? Let them block your servers/network and let those of your customers who care make their own arrangements for continued access?

4

it off to jail. The questions of when/whether/and to who bounces should be
sent is a debate for spam-l or nanae.

I don't know about that. Bounce handling is not a question of spam filtering.
Spam or not is orthogonal to the issue of forged return path. There
really should be nothing to debate, except in the context of a
protocol discussion, as the current internet standards are pretty
clear and specifically inflexible on how internet mail hosts must
handle error conditions. Just like TCP rfc793 is clear on how
internet mail hosts must handle connection establishment.

cache probably won't help. I know at least some of these orgs aggregate
queries either per RIR assigned CIDR or per ASN, so spreading the queries
out isn't likely to get you around the issue.

When contacted by the DNSBl, perhaps you inform the end-users to make
DNSBL queries directly against DNSBL servers, directly from the mail
server which is in their SWIP'ed IP range. There is little else you
can do, isn't there?

So, do you pay, and setup your own local copy of the zones? Let them block
your servers/network and let those of your customers who care make their own
arrangements for continued access?

That depends on the importance of the DNSBL.

Spamhaus' description of rsync datafeed service on their web site
appears to be incompatible with an ISP setting up a local copy and
allowing customers to query.

When setting up a local copy of the zone, you pay by "Number of e-mail users".
See the problem? As an ISP serving a local copy of the zone to
customer mail servers, you don't know how many mailboxes they have.

Maybe i'm mistaken, but it appears each end user has to buy the
service for their own mail servers, and the ISP isn't allowed to
bypass that. For the purpose of the agreements with spamhaus, an ISP
customer is probably considered a third party, and making a rbldns
server available to them is disclosing spamhaus' secret DNSBL zone
information....

5

For the purpose of the following two paragraphs, pretend for the moment
that you operate a business selling stuff via an email address
Sales@Example.Com. For dramatic effect, assume your children will
starve if you are not able to sell anything.

Further, pretend that you have really annoyed somebody--a competitor,
perhaps. Suppose that your competitor has contracted with a real,
genuine spammer to send email mmessages advertizing some trash at a rate
of tens of thousands per second until the bot-net gets shut down using
Sales@Example.Com as the Return-Path.

Now. Read the two paragraphs.

Spurious DSNs are less harmful than missing DSNs. Spurious DSNs can
be discarded easily by the mail server that knows it didn't pass that
message. DSNs that were not generated cannot be recovered.

Discarding is currently the responsibility of the mail server whose
address has been forged. Just like it's the responsibility of a host
whose source address was forged in a TCP transaction, to discard the
"ACK" packet for a connection that resulted from a spoofed SYN.

Anything about those two 'graphs you would like to reconsider?

And by the way, when I was running a network, if I got very many errant
SYN's from a particular source, that source would get a static route to
a 500 ohm resistor.

The mail server sending DSN for the fake message, or replying to a
spoofed SYN is not a spammer in any way, they are actually a victim
wasting their own bandwidth responding to a bogus message.

Victim they may be, spammer they are, The definition of "spammer" does
not include a "get even with the world" or "do unto others as was done
unto you" clauses.

6

For the purpose of the following two paragraphs, pretend for the moment
that you operate a business selling stuff via an email address
Sales@Example.Com. For dramatic effect, assume your children will
starve if you are not able to sell anything.

Further, pretend that you have really annoyed somebody--a competitor,
perhaps. Suppose that your competitor has contracted with a real,
genuine spammer to send email mmessages advertizing some trash at a rate
of tens of thousands per second until the bot-net gets shut down using
Sales@Example.Com as the Return-Path.

Lest anyone think this is a hypothetical argument, for a while I had
annoyed some eastern European spammer enough that I was getting
400,000 blowback bounces a day on a server that never sent more than
10,000 outbound messages/day.

I was able to deal with it via some custom patches in my SMTP daemon,
since the blowback had technical characteristics that made it possible
to recognize efficiently, but it wasn't pretty.

R's,
John

7

Maybe I'm mistaken, but it appears each end user has to buy the
service for their own mail servers, and the ISP isn't allowed to
bypass that. For the purpose of the agreements with spamhaus, an ISP
customer is probably considered a third party, and making a rbldns
server available to them is disclosing spamhaus' secret DNSBL zone
information....

In my experience, they're pretty reasonable. I would talk to them (or
one of their datafeed sales agents) before assuming that they won't
sell you the service you need.

R's,
John

8

They are indeed. In my day job, a large group of related members of
different institutions approached our umbrella networking organisation
to speak to Spamhaus for the specific reason that we were concerned
that;

a) between us we were making millions (if not billions) of queries a day
to the mirror servers, and
b) collective negotiation would make a service available for all of us
for far less than individual orgs paying for their own.

We now have a "private" mirror, which is accessible only from within the
same AS in which we all sit. The load is therefore not on the Spamhaus
servers or public mirrors, and we're collectively paying for the service
so the service is supported. Everyone wins.

Unfortunately (for this discussion) I don't know how much it cost, but I
would assume it wasn't much because the lead time between request and
service implementation was pretty short.

Personally I think Spamhaus are entirely correct to identify and block,
or request payment, from heavy users of their _free_ service. A little
like the organisations paying many other members of this list will do
for heavy data users in a residential or mobile context, in fact - but
that's far too controversial an issue to be conflated with this one (oh
dear).

Graeme

9

Jon Lewis wrote:

The original question, "what do you do (or have you done) when DNSBL-X
approaches you saying that your network is hitting their public NS's
too hard and wants you to pay for continued access?" is something I'd
like to see some answers to. Despite the Subject:, Spamhaus is
neither the only DNSBL currently doing this nor the first to watch
statistics on their public NS's and approach networks asking for money
and/or cutting off access if you don't pay.

As a matter of interest, who are the other current DNSBL's to do it?

Michelle

10

To the best of my knowledge, MAPS was the first to do it. Uribl.com currently does it (and does the sort of query aggregation across your entire? network) that I mentioned.

11

And SURBL.org.

Tony.

12

Jon Lewis wrote:

As a matter of interest, who are the other current DNSBL's to do it?

To the best of my knowledge, MAPS was the first to do it. Uribl.com
currently does it (and does the sort of query aggregation across your
entire? network) that I mentioned.

Can you access MAPS without a subscription at all?

As far as SORBS goes, we monitor the individual DNS servers for
excessive queries and ask any provider causing excessive queries to run
their own local copy. We don't charge for any of it, we don't require
them to run a public mirror (though sometimes we ask.)

Regards,

Michelle

13

dnswl.org currently does not do it, but bandwidth suckers are a pain.

The work is considerable: log aggregation, log review, trying to find a
responsible for the IPs and following up until they finally implement a
local copy. We losely define 100k queries/24h to be acceptable. Above
that, you should set up your local (private) mirror (and hey, rsync is
free!).

And there are some entities that do not even acknowledge that a problem
exists -- most likely until you turn access off for them. Yes, I can
completely understand Spamhaus & Co for limiting access to their public
mirrors.

(OTOH, blocking access to these abusers is hard since our infrastructure
partly relies on donated, shared public DNSBL mirrors.)

-- Matthias

14

At this point, I have no idea. Originally, yes. Even after they "went commercial", IIRC, they were still going to provide free access for "hobbyists" but not for business users. The quality and coverage of their service compared to others that were available became such that I stopped using it and didn't miss it.

15

To the best of my knowledge, MAPS was the first to do it. Uribl.com
currently does it (and does the sort of query aggregation across your
entire? network) that I mentioned.

Can you access MAPS without a subscription at all?

No. A low level subscription is pretty cheap, and I used to have one,
paying $140/yr for under 1000 users, but I dropped it when I found
that MAPS wasn't providing anything I couldn't get elsewhere.

R's,
John

16

I also have no current information (except personal surprise that they
were still around), but I got into anti-spam groups and lists, and
learning "sendmail" when our mail started failing left and right.

Long story short somebody (HP, our vendor? the previous secretive
"admin"? I never did figure out who) had configured all of our sendmail
instances to use MAPS, and MAPS had shut off the service--no warning
that I know of, no alternative that I know of. I do recall that when we
started developing our own tools (in the pre-Postini days) our catch
rate went up and our FP rate plummeted.