Wasn't there a lot of turmoil within the IETF last year
on sender authentication because Microsoft was trying to
push it's own sender ID authetication mechasnisms as a
draft standard?
Or maybe I'm confused...
Microsoft Adds Sender ID Anti-Spoofing Protocol To Exchange 2003 SP2
http://www.techweb.com/wire/security/164301084
- ferg
Yes, there was lots of teeth gnashing and screams of agony allegedly because
MS refused to license the technology on the terms that folks wanted. MS was
more than willing to let folks have it at no cost, they just weren't willing
to give the naysayer everything they wanted, so everyone went home.
(that is, of course, a biased assessment, but not an unfair one)
I'm not MS's biggest fan, but they are on the side of good, here.
In the mean time, nothing stops MS (and everyone else) from building
Sender-ID into their MTAs. SPF is a standard and Sender-ID utilized SPF
records to perform inbound phishing control based on PRA.
Presumably, SPF and Sender-ID checking on inbound mail would be enabled via
a checkbox of some sort.
I'd also like to see DomainKeys support in Exchange.
Ok, will all those who believe that MS, SPF, Sender-ID and/or DomainKeys are
the work of the devil, please commence flaming.
- Dan
Wasn't there a lot of turmoil within the IETF last year
on sender authentication because Microsoft was trying to
push it's own sender ID authetication mechasnisms as a
draft standard?
That time it did not work. But they are still trying to push it as experimental RFC and in the mean time continue to advocate it trying
to make it open standard despite technical and other objections.
Or maybe I'm confused...
SenderID is bad technology. Technically:
1. Its proposal contains recommendation that are directly opposite
to existing RFC2822 standard (which specifically mentions in section
section 3.6 that "using resent fields is a different operation from
forwarding")
2. It tries to change the meaning of message sender from real message
source to intermediate agents, this would have problems with other
email authentication technologies
3. Its using spf1 records, but they are used for information on sources
for different identity (SMTP2821 MAIL FROM) and would not always have
the same data as what would apply for RFC2822 Sender or its
derivatives. There is no consent being asked from those entered
SPF1 records if they meant them used for SID and this basically
means people who wanted to participate in SPF inadvertently are
put in danger (it can also be viewed as attempt to steal somebody
else's record).
For more on technical issues see my recent message to IESG regarding SID when they were evaluating it:
http://www.gossamer-threads.com/lists/spf/discuss/19859
Politically it has problems too:
1. Its patent license is not open-source compatible and so can not
be used in any GNU or BSD licensed program
2. The "senderid" word itself is trademarked and its use also
basically being stolen by Microsoft
Yes, there was lots of teeth gnashing and screams of agony allegedly because
MS refused to license the technology on the terms that folks wanted. MS was
more than willing to let folks have it at no cost, they just weren't willing
to give the naysayer everything they wanted, so everyone went home.(that is, of course, a biased assessment, but not an unfair one)
There were two problems with the patent license that the MS lawyers
offered. The first was that it reserved to them the right to stop
granting new licenses, thereby pulling the rug out from under anyone
who'd used licensed technology in a product, particularly an open
source product. The said they didn't plan to do that, but MS' lawyers
adamantly refused to change that, so a lot of us concluded that if
they thought the pull out the rug language was important, so did we.
The second problem was that the license was for two unpublished patent
applications that they described in general terms. When the
applications were published (on a schedule known from the day they
were filed), they turned out to cover vastly more than MS had ever
said. That left a bad taste in a lot of people's mouths. See my blog
entry at http://www.taugh.com/weblog/patapp.html
In the mean time, nothing stops MS (and everyone else) from building
Sender-ID into their MTAs. SPF is a standard and Sender-ID utilized SPF
records to perform inbound phishing control based on PRA.
(Danger: operational content ahead.)
Sender-ID and SPF have serious practical problems.
The first is that they don't match the way that a lot of mail is
really sent. If all of your mail comes from a single set of servers,
like if you're a big company or an ESP, then SPF or Sender-ID work
reasonably well to tell people which mail is yours. On the other
hand, if you're a university who lets its graduates keep using their
whatever.edu addresses after they graduate and forwards their mail to
them, they doen't work at all. (Other than a legalistic version of
"work" in which you publish a useless SPF record saying that mail
could come from anywhere.)
This would not be a problem except that SPF has been greatly oversold,
the SPF community has not been particularly diligent in disabusing
people of their misconceptions, and I can promise that the moment
there is a Sender-ID checkbox in Exchange, clueless MCSEs will set it
to reject anything that fails Sender-ID, it'll reject buckets of
normal valid mail, and when people complain, they will insist that the
mail must have been sent wrong because Sender-ID said it was spam.
Even for fixed senders, Sender-ID is useless against phishing, because
it can only tell you that a message purporting to be from phoop.com
came from a source that is OK for phoop.com, but it cannot tell you
whether phoop.com is someone you want to get mail from. This is a
real problem. For example, I got mail the other day from
customercenter.net purporting to tell me about the status of my MBNA
credit card, with a link to mbnanetaccess.com. Was that a phish? Or
consider mbna-account.com and mbna-accounts.com. One is MBNA, one is
not. Does your MUA know which one is which? Spammers and phishers
can publish SPF records just like anyone else, and Ciphertrust has
said that of the mail they see that passes SPF, there's more spam
than legit mail.
I am all in favor of sender authentication, if it's real sender
authentication. But Sender-ID isn't. Domainkeys will be better since
it matches mail sending patterns better, but it has the same problem
that a sender that's been authenticated has nothing to do with whether
its mail is desired.
Shameless plug: over in the anti-spam research group at asrg.sp.am I
sure would like it if people were working on reputation systems to
plug the gaping hole left by all these authentication schemes.
Regards,
John Levine, johnl@iecc.com, Primary Perpetrator of "The Internet for Dummies",
Information Superhighwayman wanna-be, http://www.johnlevine.com, Mayor
"More Wiener schnitzel, please", said Tom, revealingly.
Not sure if it's a "gaping hole," so much as an unfortunate fact
that sender reputation is already proving to be even harder to
standardize than how to confirm the identity of a sender.
We can't have reliable reputation until we know who the mail is
coming from -- so reliable identity is a necessary first step.
Operationally, this means that ISP's can't yet abandon whatever
reputation systems they already have in place (most of which are
based on the source IP address, or on in-message criteria.) But
they can (and should) start testing whichever verification
scheme best fits their mail flow patterns, so that all of our
internal reputation engines can start evolving.
What the doctor ordered seems to be something like an API that'd let
you plug dkim + csv into $mta
Anyone?
--srs
Reputation is a missing element in all sender authentications schemes and
will (likely) be solved separately.
No approach is perfect, but building closer to a solution is preferred over
sitting on our hands and debating, which (historically) seems to be the
IETF's approach.
- Dan
You miss the point. Reputation is already widely being used today - every blocklist is a reputation system and we have lots of those for mail server
reputation and for network reputation and for domain names there is SURBL
All those are of course "bad reputation" lists and John wants to see good reputation lists, but it can only happen once authorization is used, but initial technologies are there. For when something more complex is needed there is in fact siq being developed at ASRG
http://www.ietf.org/internet-drafts/draft-irtf-asrg-iar-howe-siq-01.txt
so what is really needed are people willing to come to asrg and work with
authors on R&D (with emphasis on "d" part) and if it does not work well
than ASRG will look at something else.
Strike that point. Not every blocklist is reputation system, though the
most widely used ones like SBL, SORBS, SPAMCOP are all like that.
But there others simply provide certain info about corresponding ip or
domain which is derived from domain info itself (like what country its from, if its bogon/not-allocated space for ip, etc), those would be closer accreditation data.
I seriously doubt this distinction is that important for those at nanog.
But anyway, I had to correct myself, since my statement was not accurate.
I don't think the lack of standard APIs is a problem: it seems to me that
the various MTA authors have been reasonably keen to support the various
nascent specifications, especially if they have a reasonably good
reference implementation library.
Tony.
I don't disagree with anything you said, and I certainly agree with your
closing sentiment.
Having said that, SPF and Sender-ID are useful as another hammer in the
bag-o-tricks, even if they aren't useful for everybody, or if they don't
solve everything, or if their authors misrepresent the technology, or any
of the other number of things that you didn't mention. At the least, being
able to say that mail from my domain is really only from me if it came
They are useful as proof-of-concept works, too. There have been many
demands to produce something like SPF/Sender-ID for many years (myself
included), and just having them out there is useful for research and
analysis purposes. Without them, folks would still be arguing to produce
them, at the least. As you've argued on ASRG yourself, the research is
worthwhile in and of itself, even if it produces something unexpected.
Easier for those of us who run multiple MTAs, especially large farms of them
But yes, you do have a point that it is not exactly a problem.
Is there something in the works for postfix and qmail yet?
regards
srs
We've already tackled reputation systems, they were called web site
certificates. You have to submit to a few fairly stringent checks on
who you are, typically provide a D&B id which isn't very expensive or
difficult but not all that easily defrauded w/in some reasonable
parameters (it ain't bank security but it's good enough to be pretty
sure you're giving your credit card info to who you think you are,
damn, you hand your card to strange bartenders right?)
But there was real money in web site certificates, indirectly, in the
form of e-commerce. And that area had the good luck of evolving
rapidly in a huge market boom.
There's basically no such money in e-mail, not versus not adding a
reputation system.
No further explanation should be necessary.
However, I'll add my voice that I believe "reputation" systems as an
approach to spam-prevention are neither useful nor sufficient w/o
repeating what others have said.
The problem is really pretty simple; we're trying to solve a big
problem w/o creating any concomitant economics to support a solution.
It's a nice fantasy, and that's what it remains after a decade.
This is accreditation, not reputation.
When you contact somebody and ask to verify your credentials this is
accreditation (and it says nothing about how you're going to act,
just verifies your identity and provides additional info about you).
When somebody else looks at your activity and makes "subjective" judgement (mosly based on multiple reports from users) and then lets this judgement about your activities be available to other users then its reputation.
This judgment of activities should be premised on knowing who is
actually accountable for the activity. With either SPF or Sender-ID,
this mechanism is just offering server authorization. When the
mailbox-domain appears to be supported by server authorization, without
also assuming all preceding MTAs ensure exclusive use the specific
mailbox-domain, it could be incorrect to conclude the mailbox-domain
originated the message.
Domain owners are not clearly warned by the respective drafts of this
add MTA requirement when reputation is involved. Both drafts suggest
the mailbox-domain may subsequently accrue a reputation, when supported
by server authorization. The SPF camp elected to drop scoped records,
which would have allowed the sender to declare which mailbox-domain has
been assured exclusivity. Sender-ID will also use this record to
discover authorization for either the Purported Responsible Address
(PRA) or the bounce-address. The "opt-out" solution offered by
Sender-ID will create problems at some destinations, which means both
the PRA and bounce-address require exclusivity at the MTA. Snap goes
the Sender-ID reputation license trap.
Both schemes demand greater diligence by MTA administrators when
mailbox-domain authorization/reputation schemes are implemented, while
simultaneously leaving negligent MTA administrators unscathed. While
this may seem like great news for the MTA administrator, and bad news
for domain owners, it risks litigation. The mailbox-domain owner is
otherwise without recourse, when finding their domain blocked.
-Doug
Thus spake "Barry Shein" <bzs@world.std.com>
However, I'll add my voice that I believe "reputation" systems as an
approach to spam-prevention are neither useful nor sufficient w/o
repeating what others have said.
Agreed.
If my grandmother has a "reputation" for sending legitimate email, and she
inadvertently installs some spam zombie software, it is certainly feasible
(and probably trivial) for the spammer to steal all her credentials and thus
her "reputation". Spam will get out for a while, but once her "reputation"
significantly degrades, it will be stopped -- as will any future legitimate
email from her.
This "solution" strikes me as worse than the problem it tries to address.
S
Stephen Sprunk "Those people who think they know everything
CCIE #3723 are a great annoyance to those of us who do."
K5SSS --Isaac Asimov
If my grandmother has a "reputation" for sending legitimate email,
and she inadvertently installs some spam zombie software, it is
certainly feasible (and probably trivial) for the spammer to steal
all her credentials and thus her "reputation". Spam will get out
for a while, but once her "reputation" significantly degrades, it
will be stopped -- as will any future legitimate email from her.
No. You are (I suspect) deliberately ignoring the Big Picture.
Your grandmother, if she is like most grandmothers, does not have a
box coloed with a static IP from which she runs her own MTA. She
gets a dynamic address assignment from her ISP.
When her computer becomes infected with malware that causes it to
emit abusive traffic, the reputation for that IP (or its containing
netblock) is affected.
The longer her ISP allows the abusive traffic, the lower the
reputation becomes for that address (or its containing netblock).
So you see, the reputation has nothing to do with your mom, and
everything to do with the controlling entity, her ISP. Which makes
the whole address-based sender reputation scheme almost workable, if
you ignore the scaling issues.
This "solution" strikes me as worse than the problem it tries to
address.
I'd never call it a "solution", but it is certainly a useful tool to
use along with others in order to more successfully manage the
problem.
matto
--matt@snark.net------------------------------------------<darwin><
The only thing necessary for the triumph
of evil is for good men to do nothing. - Edmund Burke
Anyone who has been paying much attention to the spam problem in
the past few years already knows that assigning a bad reputation
forever is a bad idea -- especially to dialups.
What's more likely to happen in your Grandmother's case is that
her IP/e-mail address will have a bad reputation for a while,
and then she'll call you and you'll come over and un-zombify her
cmoputer and say "okay, Grandma, remember what we said about
clicking on links on porn sites?" and she'll say "oh, but it was
blinking such pretty colors!" and a few hours/days later the
reputation will have returned to the default state.
That's suspiciously close to "Ralph Nader or Ross Perot could have been elected
President, if you ignore the scaling issues". 
Other than that, what Matt said is correct - the problem is that legitimate
mail can come from literally millions of places whose reputation we have no
clue on....
Exactly.
Everyone in the SPAMwar has to be aware that SPAM can't be stopped until
its transaction costs approach that of the cheapest other advertising
method. That can be snailmail spam, telephone terror^Wmarketing, whatever,
you name it.
SPAMmers will do whatever is economically feasible to send their SPAM.
If this includes commanding zonbie armies they will. If it includes
hacking real mailserver with good reputation they will. If it includes
buying IP space they will. You have to manage to lower the reputation
of that host within a very short amount of time to increase the
transaction costs sufficiently for the spammer to make the effort
worthwhile. On the other hand you make the life for the hacked mail
server miserable. And if you let the reputation of a host bounce
back to fast spammers will use it by schedule. Spam for two hours
today and again in two days and again and again.
The current Internet email system is an open system because SMTP was
built that way. Note that there can't be any other 'rewrite' of SMTP
that fixes the SPAM problem and remains open at the same time. It follows
that all the chatter that SMTP is broken is false and misguided unless
you want a closed system controlled by one entitiy.
Everything that can be done to keep the SPAM problem in check (fixing is
impossible by definition):
1a) must be simple so that many million server administrators can understand it.
1b) must scale to millions of legitimate mail servers.
1c) must not break common functionality for users.
With high probability there is no one (complex) system fulfilling all of
these point at the same time. In all likelyhood it can only be achieved
by a combination of simple systems/mechanisms each tailored to deal with
a specific part of the problem. This kind of approach not only scales on
the deployment side but also on the time line. Spammers adopt and will
change strategy as they have done many times in the past.
The options we have are:
2a) forward DNS based information (reverse MX information).
2b) reverse DNS based information.
2c) blacklisting.
2d) whitelisting.
2e) cryptographic signatures.
Each of them can contribute to a different part of the problem and none of
them can fix the entire one. IETF MARID tried to stuff too many things into
one of the above systems and failed.
Each of them has its own unique advantages and disadvantages and tackles
the problem on a different layer and is under different administrative
control.
Now create a specific (sub)problem description, select the approriate
option from above and specify a *simple* and easily understandable
mechanism to make use of it.