Massive quoting gets old fast so I'll try to summarize and if I
misrepresent your POV in any way my profuse apologies in advance.First and foremost let me say that if we had a vote here tomorrow on
the spam problem I suspect you'd win but that's because most people,
even (especially) people who believe themselves to be technically
knowledgeable, hold a lot of misconceptions about spam. So much for
democracy.I say the core problem in spam are the botnets capable of delivering
on the order of 100 billion msgs/day.You say there are other kinds of spammers.
I'll agree but if we got rid of or incapacitated the massive botnets
that would be a trickle, manageable, and hardly be worth fussing
about, particularly on an operational list.
That's not quite true. The spam problem predates the massive botnets.
Massive botnets are rather a recent thing.
*A* core problem *for engineering purposes* is that botnets are capable of
delivering an essentially unlimited flood of source material for our mail
systems. This is a primary target for anti-spam efforts at the major
ISP's, and certainly many of them have a lot of experience in trying to
stem this highly effective and nonstop DDoS attack on the e-mail system.
I do not believe that anyone seriously disagrees with that.
However, the average user has different problems.
First off, let me state this as a prerequisite for any further discussion.
E-mail has to be perceived, by the users, as a beneficial tool, one that
they can rely on for the things that they choose to do. If you disagree
with this, then any further discussion is meaningless, because we do not
share a common view of what the e-mail system needs to be. You would not
be the only person to perceive e-mail in a different manner, if you did.
To be sure, there are people who perceive it as something that is trivial,
in the class of IM or IRC protocols, for example. I view it as something
I'd like to work at least as reliably as the US Post.
So, here are some additional problems. These are not botnet problems, but
rather user problems with the e-mail system.
Users cannot reliably receive e-mail that they have asked to receive. For
example, receiving receipts from a vendor.
Users cannot be assured that the e-mail that they've received is from the
sender that it appears to be.
Users cannot know if the mail that they've sent has been received by the
dodgy freemail hoster that their friend is on.
Users cannot withdraw permission to send from an abusive sender. They are
finding their address shared with others, or are unable to unsub, or
whatever.
These are all significant problems with the current e-mail implementation.
They do not represent DDoS-class problems. However, they do represent a
massive set of problems that are driving users away from e-mail. If it is
allowed to continue, our FUSSP can be to simply block all port 25, as SMTP
will become irrelevant. Yes, that's a bit dramatic, but it's also the way
things are headed.
The reason is that without the botnets the spammers don't have address
mobility. You could just block their servers.
That's demonstrably false, and displays a gross ignorance of both
historical and current spammer modes of operation. It is exceedingly
common for hosting providers to receive requests from clients to be
allocated many noncontiguous IP addresses out of a number of /24's, and
these requests are honored by many of the seedier providers. This has
been the case for years. Some of them even attempt to justify it by
claiming that they need it for purposes of affecting Google advertising
(for example). See
http://www.spamhaus.org/faq/answers.lasso?section=Glossary#233
to learn more about snowshoe spamming, and related techniques.
But if we don't agree on those points then we're talking past each
other.
We don't agree on some of them, that's for sure.
I assert that the problem are the massive O(100B) botnet spammers and
they simply don't have the resources or interest really (because they
don't have the resources or business model) to do things like analyze
return codes etc as you describe.
That's _a_ problem, but it is hardly the only problem pressing in on the
e-mail system. Were this the only problem, it would be easiest to solve
it by whitelisting legitimate senders, probably in combination with some
variation of the Spamhaus PBL system, and winding up with a restrictive
version of SMTP that requires you to somehow be authorized to send e-mail.
Variations on this have been less than completely successful. It is a
monumental undertaking, but it /could/ be done. It wouldn't solve the
problem, however.
So it's doubtful to me that returning more meaningful return codes in
SMTP rejections would be of much use to them.
Of course not - to them.
It's also not of much use to them, as I previously described, even if
they tried. They could deduce about the same information for about the
same "price" without the return codes.
Again - to them.
But they're hardly the only class of spammers. I realize it's convenient
to ignore that fact for the purposes of this discussion, since it supports
your argument while ignoring the fact that other spammers would mine a
lot of useful information out of such messages.
But any such return codes should be voluntary,
And they are. To the best of my knowledge, you can put pretty much any
crud you like after the "### ", and if anybody wanted to return this data,
they would be doing it today.
particularly the
details, and a receiving MTA should be free to respond with as much or
as little information as they are comfortable with right down to the
big red button, "421 it just ain't happenin' bub!"But it was just an example of how perhaps some standards, particularly
regarding mail rejection, might help operationally. I'm not pushing
the particular example I gave of extending status codes.Also, again I can't claim to know what you're working on, but there
are quite a few "disposable" address systems in production which use
various variations such as one per sender, one per message, change it
only when you want to, etc. But maybe you have something better, I
encourage you to pursue your vision.
No. The difference to my solution is simply that it solves all the
problems I outlined when I wanted to solve the problem I started with -
finding a clean way to be able to exempt senders from anti-spam checks
that they frequently fell afoul of.
But then again, I am merely saying that there are solutions capable, but
that they all seem to require some paradigm shift.
And, finally, one quote:
>I didn't say I had a design. Certainly there are solutions to the
>problem, but any solution I'm aware of involves paradigm changes of
>some sort, changes that apparently few are willing to make.Gosh if you know of any FUSSP* whose only problem is that it requires
everyone on the internet to abandon SMTP entirely or similar by all
means share it.
That was kind of the nifty part to my solution: it didn't require any
changes at any sender's site. By accepting some tradeoffs, I was able
to compartmentalize all the permission issues as functions controlled by
the receiving site.
Unfortunately this is a common hand-wave, "oh we could get rid of spam
overnight but it would require changes to (SMTP, usually) which would
take a decade or more to implement, if at all!"Well, since it's already BEEN a decade or more that we've all been
fussing about spam in a big way maybe we should have listened to
people with a secret plan to end the war back in 1998. So I'm here to
tell ya I'll listen to it now and I suspect so will a lot of others.
If we cannot have a flag day for the e-mail system, and obviously, duh,
we cannot have a flag day for the e-mail system, we have to look at other
changes.
That's too big a paradigm shift.
My solution is a comprehensive solution to the permission problem, which is
a root issue in the fight against spam, but it is based on a paradigm shift
that ISP's are unwilling to underwrite - dealing with per-correspondent
addresses. This has challenges associated with it, primarily related to
educating users how to use it, and then getting users to commit to actually
doing so.
That's not TOO big a paradigm shift, since it's completely backwards-
compatible and managed at the receiving site without any support required
anywhere else in the e-mail system, but since service providers aren't
interested in it, it is a non-starter. Were it interesting, it wouldn't
be that tough to support relatively transparently via plugins into modern
browsers such as Firefox and Thunderbird. But it is a LARGE paradigm
shift, and it doesn't even solve every problem with the e-mail system.
I am unconvinced that there aren't smaller potential paradigm shifts that
could be made. However...
It is exceedingly clear to me that service providers prefer to treat the
spam problem in a statistical manner. It offers fairly good results (if
you consider ~90%-99% accuracy to be acceptable) but doesn't actually do
anything for users who need e-mail that they can actually rely on. It's
cheap (relatively speaking) and the support costs can be made to be cheap.
* FUSSP - Final and Ultimate Solution to the Spam Problem.
Shoot all the spammers? 
... JG
