Micorsoft's Sender ID Authentication......?

Mailman List Mirror (Read Only)
21

Everyone in the SPAMwar has to be aware that SPAM can't be stopped until
its transaction costs approach that of the cheapest other advertising
method. That can be snailmail spam, telephone terror^Wmarketing, whatever,
you name it.

The issue of course is that by making it more expensive for senders of
spam, you're making it just as expensive for senders of email

Each of them can contribute to a different part of the problem and none of
them can fix the entire one. IETF MARID tried to stuff too many things into
one of the above systems and failed.

Authentication without backing of a reputation is not too useful, as you say.

The way AOL uses spf is to just use it to let people it wants to
whitelist update their whitelist records with aol on the fly, so they
dont have to open a ticket with AOL each time they add a new /24 worth
of outbound servers for "high volume email deployment"

Each of them has its own unique advantages and disadvantages and tackles
the problem on a different layer and is under different administrative
control.

Nice. Only, all this falls totally in a technical space, where you
need at least two other things (policy and user awareness) to flesh
the picture out.

I'll be teaching a short but quite general tutorial (~ 3 hours) on
spam issues at apnic 20 in Hanoi this september, based mostly on a
whole lot of conclusions I've drawn in my oecd paper
http://www.oecd.org/dataoecd/5/47/34935342.pdf

I've designed it just for this purpose - to let operators anywhere in
the world use it, teach stuff based on it .. and I'd be obliged if
people who do this stuff on the NOG circuit see fit to use it that way
..

regards
srs

22

That's suspiciously close to "Ralph Nader or Ross Perot could have
  been elected President, if you ignore the scaling issues". :slight_smile:

Yes. There's a reason I did not include a ringing endorsement of
sender reputation schemes as the FUSSP; it has colossal inherent
scaling issues; however I believe the 90/10 rule will make it at
least somewhat effective.

  Other than that, what Matt said is correct - the problem is that
  legitimate mail can come from literally millions of places whose
  reputation we have no clue on....

Yes. Sender reputation on an per-ip level is a lot of state.
However; I believe that sender reputation on a swip level may be
attainable, and provide positive value.

matto

PS: Even though it's painfully obvious, I speak only for myself and
no entity currently/previously employing me- Especially those kooks
at UCB.

--matt@snark.net------------------------------------------<darwin><
              The only thing necessary for the triumph
              of evil is for good men to do nothing. - Edmund Burke

23

One useful definition of (some sorts of) insanity is doing the same
thing over and over but expecting different results.

I therefore assert there is no technical solution to spam.

What will stop it is some sort of new economic model, billing for
e-mail (yeah yeah some reasonable amt "included"), along with vigorous
enforcement of that model against theft of service etc. Miscreants of
the sort we're dealing with only understand jail time.

But, as they say, ya get what ya pay for, or put differently and to
paraphrase someone else who I don't know wants the attribution:

  Most people want free e-mail in the worst way, and that's just how
  they get it.

I'll venture that any such sea-change will not come from the technical
community. That's another example of doing the same thing over and
over; clearly the internet technical community is stuck in a rut on
this issue and has been for years.

       -b

24

Barry Shein wrote:

One useful definition of (some sorts of) insanity is doing the same
thing over and over but expecting different results.

I therefore assert there is no technical solution to spam.

The ultimate solution would have to be a combination of social, technical and probably legislative.

25

  1a) must be simple so that many million server administrators can
understand it.
  1b) must scale to millions of legitimate mail servers.
  1c) must not break common functionality for users.

Good list.

To repeat the cliche, spam is a social problem. Technical solutions can only
follow social decisions. Otherwise, we get technology dictating social policy.
As bad as that is as a general rule, it is particularly bad for anything involve
large-scale human communications, since the unintended consequences are certain
to be massive and massively bad.

Spam (and virus attacks) seem particularly strong requirements for a layered
defense, some proactive and some reactive. Some involving authors and some
involving operators.

Being able to white- or black-list an operator legitimately is particularly
powerful. They represent an aggregation of users and traffic. So the leverage
is enormous. Perhaps because the payoff is so high, the dangers of
mis-assignment are also huge. So such listing needs to be done conservatively,
which leaves lots of traffic unassigned.

Being able to white-list authors is equally spiffy. In general, formulating a
positive trusted core of communicants well might permit high quality service for
relatively low costs, such as little or no content analysis, with its attendance
statistical failings (false positives).

And so on...

  d/

26

Amen (mostly).

I think we can write off "legislative" until we fix the root causes that gave
us the CAN-SPAM act. We'll more likely make progress on the "judicial" side by
finding a gung-ho DA who wants to get famous by enforcing the *existing* laws -
I seem to recall one up in NY.. :wink:

I've been saying it for years - there's a *really* quick fix to the spam problem.

We just take a pool - a few hundred dollars from every ISP. We hire some
<insert ethnic> enforcment goons to "explain things" to the spammers, and allow
the goons to be creative. Have them visit one of the top-100 off the ROSKO
list at random each week. We'll be well on the way to done by the end of the
summer. :wink:

Squeamish? Oh bother. OK, so we hire lawyers instead. Less bloody, but it
takes longer and costs more....

27

That's the precise reason I said "probably". "root causes" include congresscritters in the pockets of the DMA, and that's not likely to change soon.

28

I think we can write off "legislative" until we fix the root causes that gave us the CAN-SPAM act. We'll more likely make progress on the "judicial" side by finding a gung-ho DA who wants to get famous by enforcing the *existing* laws - I seem to recall one up in NY.. :wink:

I think legislative should not be take to mean only US! There is work on the tough law in Canada, Europe and other countries. And as US congress
begins to understands that it "you can spam" law does work, the replacement
will be thought and quite possibly the laws found then in other countries will serve as good basis for it.

On the less serious note ...

I've been saying it for years - there's a *really* quick fix to the spam problem.

We just take a pool - a few hundred dollars from every ISP. We hire some
<insert ethnic> enforcment goons to "explain things" to the spammers, and allow
the goons to be creative. Have them visit one of the top-100 off the ROSKO
list at random each week. We'll be well on the way to done by the end of the
summer. :wink:

Since some of these spammers have millions of $$$, they will hire their own
<insert another nationality> goons to make sure ISPs and their goons don't
bother them. So did I hear somebody mention spamwar recently on this list? :slight_smile:

Squeamish? Oh bother. OK, so we hire lawyers instead. Less bloody, but it
takes longer and costs more....

Makes me wonder where the term "bloody laywers" came from if above is true :slight_smile:

29

I therefore assert there is no technical solution to spam.

I think you're preaching to the choir here.

What will stop it is some sort of new economic model, billing for
e-mail (yeah yeah some reasonable amt "included"),

Unfortunately, that's a technical solution, because it requires that
we invent some sort of technology that can track all the mail, assign
responsibility for postage, and do the settlements. As I've been
saying for quite a while, it doesn't exist and it's not likely to,
ever, because mantaining large rapidly updated databases with
authentication on the updates is a fundamentally hard problem.

along with vigorous enforcement of that model against theft of
service etc. Miscreants of the sort we're dealing with only
understand jail time.

If it's OK with you, I'd rather skip the epostage vaporware and move
directly to the enforcement. Most spammers are breaking multiple
laws, even the inane CAN-SPAM act, now.

Where I think technology can help is to make it easier to build cases
against spammers that will stand up in court. I was the
Commonwealth's technical expert in the criminal case against Jeremy
Jaynes, and it was clear that kind of prosecution is much too
expensive to work against any but the very largest spammers who are
targeting recipients that are motivated to spend their own money to
help prepare the case.

R's,
John

30

You have to manage to lower the reputation
of that host within a very short amount of time to increase the
transaction costs sufficiently for the spammer to make the effort
worthwhile.

Or you have to ensure that the sending ISP can react quickly
and stop the flow of spam so that only a trickle of messages
actually enters the system

In all likelyhood it can only be achieved
by a combination of simple systems/mechanisms each tailored to deal with
a specific part of the problem.

Here's a simple mechanism which has not yet been tried
seriously. Email server peering. This means that an SMTP
server operator only accepts incoming mail from operators
with whom they have a bilateral email peering agreement.
Bilateral agreements have been shown to scale quite well
whether you look at BGP peering or the world of business
contracts. In any case, the fundamental need here is that
for somebody to notify the email administrator that is
sending spam and for that administrator to act immediately
to cut the flow.

Today, notifications arrive in a flood that smells a lot
like SPAM itself. They come from millions of unknown and
unvalidated and often confused sources. No email admin can
afford to act without careful investigation first. But if
the notifications only come from email peers who only report
problems that involve the bilateral relationship, i.e. source
of spam flow to destination of spam flow, then the source
email administrator can afford to act immediately without
any investigation. After all, this is all covered in the peering
agreement. The two parties have already agreed to do this,
have already agreed on notification mechanisms and have already
agreed on responsibility (and penalties) for bad notifications.

Because all of this is codified in a set of bilateral email
peering agreements, it becomes easy to incorporate any implications
in the email service level agreements with customers. In other
words, if an email operator agrees to shut down IP access to
an 0wn3d workstation immediately upon request of their peer,
then they can put that in the customer contract so that the
customer knows that SMTP is strictly forbidden without prior
arrangement with their operator.

SPAM is not a technical problem and there are no technical
solutions. It is a social problem and the solutions are to
be found in social arrangements like contracts, laws,
email services associations, etc.

A lot of this dubious technical garbage can be swept aside
if we simply recognize that the flat structure of a completely
open SMTP service is not scalable. But if we manage the structure
through agreements and contracts, we can organize the exact same
technology and protocols into a relatively thin hierarchy, no more
than 3 or 4 levels, with a global mesh of bilateral peering agreements
forming the top level. This top level could easily scale to thousands
of members if it is based on a standard bilateral email peering
agreement that is administered by a consortium of the top level
peers. In this way, a new operator can join the consortium and
simultaneously sign the bilateral agreement with all current members.

I envisage the top level of this hierarchy to actually be composed
of several international consortia, probably 5 of them, building on
the work that has been done by the Regional Internet Registries in
building up international cooperation amongst ISPs. I'm not suggesting
that the RIRs would become these email consortia, simply that the
RIRs are proof that ISPs can cooperate on an international level
to manage Internet resources in an orderly fashion. And associated
with the 5 RIRs are 5 regional gatherings of ISPs where this
type of cooperation could be hammered out.

What we really need to get this off the ground is for some of the
large email operators to get together in public at a NANOG meeting
and discuss this openly in some type of round table discussion.
Who will take the first step? Perhaps the NANOG program committee?

--Michael Dillon

31

Here's a simple mechanism which has not yet been tried
seriously. Email server peering. This means that an SMTP
server operator only accepts incoming mail from operators
with whom they have a bilateral email peering agreement.

This has been tried in the X.400 world. I wouldn't exactly say it
worked well - and I, for one, have no desire to return to X.400
style email peering.

Bilateral agreements have been shown to scale quite well
whether you look at BGP peering or the world of business
contracts. In any case, the fundamental need here is that
for somebody to notify the email administrator that is
sending spam and for that administrator to act immediately
to cut the flow.

The number of agreements needed in the email world is significantly
higher than what is needed for BGP.

Steinar Haug, Nethelp consulting, sthaug@nethelp.no

32

* Michael.Dillon@btradianz.com (Michael.Dillon@btradianz.com) [Mon 13 Jun 2005, 11:10 CEST]:

Here's a simple mechanism which has not yet been tried seriously. Email server peering. This means that an SMTP server operator only accepts incoming mail from operators with whom they have a bilateral email peering agreement.

I debunked this before. Please see list archives.

Bilateral agreements have been shown to scale quite well whether you look at BGP peering or the world of business contracts. In any case, the fundamental need here is that

[..]

I suggest you take a better look than your current "Oh this sounds logical and is probably how it works" skimming of reality, because it really is quite different - much uglier, mostly - from what you state.

  -- Niels.

33

Here's a simple mechanism which has not yet been tried seriously. Email
server peering.

All this stuff you're describing [almost] exactly matches the former
UUCP-based e-mail distribution mechanism.

A lot of this dubious technical garbage can be swept aside
if we simply recognize that the flat structure of a completely
open SMTP service is not scalable.

But we already found that the UUCP system didn't scale because its
management load was way too high. That's why RFC821 exists!

We aren't rolling back the clock now -- the e-mail infrastructure is far too
large to do that.

(Oh sh*t, did I just feed a troll?)

34

Well, are you missing a bag of Purina Troll Chow? :slight_smile:

35

>I therefore assert there is no technical solution to spam.

>
> I think you're preaching to the choir here.
>
> >What will stop it is some sort of new economic model, billing for
> >e-mail (yeah yeah some reasonable amt "included"),
>
> Unfortunately, that's a technical solution, because it requires that
> we invent some sort of technology that can track all the mail, assign
> responsibility for postage, and do the settlements. As I've been

I think it's disingenuous to label a billing system as another example
of a technical solution to spam, it begs the word "solution" in
exchange for "technical requirements" of a completely different
nature, i.e., a billing system, not quite the rocket science of the
automated, near-perfect spam-classifier a spam "technical solution"
implies. No doubt it has its challenges, what doesn't?

It's also a straw man to posit a particular billing system and
conclude that model would be impossible to implement.

But to make that argument one has to go from the claim that spam
classification is analogous to building an email billing system (i.e.,
both "technical" solutions) to the claim that it's also AT LEAST AS
DIFFICULT, or else the point fails. I think that leap begs
deconstruction. If it's easier to build and implement a billing
system, with policies acceptable to most and desirable results, then
the opposite point succeeds.

So, three points:

1. If a particular billing/business model presents difficulties then
   we might have to consider a different model, others are possible
   (hence, straw man of e-postage etc.)

2. It would seem to say, for example, the long distance voice billing
   system is impossible since it would seem to have many of the same
   qualities you delineate as insurmountable obstacles.

Most importantly the big difference between those billing systems and
e-mail is that there are billions and billions of dollars in those
voice billing systems and the systems they support. There is very
little money in e-mail, relatively speaking, except indirectly as a
bundled add-on to sell a more general service (ip connectivity.)

My thinking is that money talks, b.... walks, and problems don't seem
so insurmountable when there is money involved versus checking the
free software lists occasionally to see if someone has come up with a
little better filter in their spare time.

This, I would claim, is why we've seen such vigorous action from the
likes of RIAA v. piracy in a small fraction of the time we've had spam
as a problem: There's money on that table, lots of money.

One might dislike or disagree with that activity but that's neither
here nor there, the point is: Put the gas in tank (money) and the
engine runs, and w/o it one gets to fret about how difficult and
untenable it is to push the vehicle and tries to imagine a world where
every destination is downhill.

3. Finally, the e-mail system has changed in many significant ways to
(mostly ineffectively) react to the spam problem, from using
third-party filter services to declaring all sorts of formerly
legitimate behavior as no longer acceptable (e.g., open relays being
the relay owner's choice) and vast swathes of new end-user software
and software changes to existing MUAs/MTAs to respond to spam,
viruses, etc.

So, to say that no change of significance can occur to accomodate a
billing system, if that were indeed the solution, is also
disingenuous. Tail / Dog / Wag. At best that's a platitude, easy to
agree with in the abstract but possibly impossible to accomodate.

> If it's OK with you, I'd rather skip the epostage vaporware and move
> directly to the enforcement. Most spammers are breaking multiple
> laws, even the inane CAN-SPAM act, now.
>
> Where I think technology can help is to make it easier to build cases
> against spammers that will stand up in court. I was the
> Commonwealth's technical expert in the criminal case against Jeremy
> Jaynes, and it was clear that kind of prosecution is much too
> expensive to work against any but the very largest spammers who are
> targeting recipients that are motivated to spend their own money to
> help prepare the case.

I am glad to see such efforts.

But I suspect that until there's some real money on the table the
efforts will continue to be frustrating.

Certainly one of the sirens of the justice system is that if one can
get THEIR problem criminalized then they can compel action by their
govt, at that govt's expense, to solve the problem.

This is one reason why there is always such a non-stop clamor to
criminalize this or that everywhere in this society; it tries to
obligate someone else (the govt) to expend money and resources on a
problem and let the rest of us get back to our own lives.

Sometimes that's exactly correct, certainly. Oftentimes it's nothing
other than an attempt to get someone else to pay the bill or avoid
some hard thinking, or hard work.

36

We've been here before, but to recap.

1. If a particular billing/business model presents difficulties then
  we might have to consider a different model, others are possible
  (hence, straw man of e-postage etc.)

I look forward to hearing about a design for an email billing system
that does not require technology that is two orders of magnitude
beyond the state of the art and would be effective against spammers.
I hope my inability to envision one is just due to lack of
imagination, but I've been asking for years and I've never seen
anything even close. It's not hard to sketch out a scheme that does
statistical billing and works so long as everyone plays more or less
by the rules. It is far harder to come up with one that will work
against determined bad guys who would be delighted if 1% of their mail
leaked through without paying (or paid by other people.)

2. It would seem to say, for example, the long distance voice billing
  system is impossible since it would seem to have many of the same
  qualities you delineate as insurmountable obstacles.

Telephony is heavily regulated, has high costs of entry, has nothing
comparable to the spam problem (fraudulent callers at higher volumes
than legit callers), and is still subject to gross frauds like MCI.
I'm thinking both of MCI's accounting frauds, and of more technical
stuff like routing calls through Canada and reoriginating them at
small telcos in the upper midwest to make them look like local rather
than long distance traffic. Is that really the business you want to
be in? It is also my impression that the number of phone calls is a
whole lot less than the number of e-mail messages, returning us to the
previous problem.

I can't help but note that telephony is rapidly moving in the other
direction, away from itemized billing. My phone service is now flat
rate for calls to anywhere from Honolulu to Helsinki. Perhaps they
know something.

Most importantly the big difference between those billing systems and
e-mail is that there are billions and billions of dollars in those
voice billing systems and the systems they support. There is very
little money in e-mail,

This part, I completely agree with. No matter what anti-spam
technique you propose, someone will complain that it costs too much.
(This is particularly true of bulk marketers who apparently think that
if e-mail were merely 100 times cheaper than paper mail rather than
1000 times cheaper, mass bankruptcies would ensue.)

Spamhaus says, on the record, that MCI and SBC will not disconnect
spammers for anything other than non-payment. The money is talking
there. One of the few hopeful signs on the horizon is that Verizon,
for all its faults, has a lot better history of booting off spammers
than MCI does.

R's,
John

37

a message of 21 lines which said:

The number of agreements needed in the email world is significantly
higher than what is needed for BGP.

The proponents of "email peering" typically want to switch from the
current model (millions of independant email servers) to a different
model, with only a few big actors.

38

The proponents of "email peering" typically want to switch from the
current model (millions of independant email servers) to a different
model, with only a few big actors.

and, as unlikely as it may seem, they think they should be those
actors.

randy

39

> The number of agreements needed in the email world is significantly
> higher than what is needed for BGP.

The proponents of "email peering" typically want to switch from the
current model (millions of independant email servers) to a different
model, with only a few big actors.

I don't know who these proponents are, that you refer to. However,
in my earlier message I quite clearly described a model that allows
for millions of independent email servers organized in roughly
3 levels of hierarchy and I described how it could be done so
that email peering IS NOT LIMITED to a few big actors.

The 3 levels that I described were, at the top, intercontinental
peering between members of 5 organizations which roughly
cover the countries in one continent. I suggest that these
5 organizations should adopt the service boundaries of the
RIRs rather than trying to reinvent the wheel.

Next, there would be peering between all members in the
same continental organization. These members would exchange
email with each other under contract terms which clearly
lay out the responsibilities of sending operators and
receiving operators.

Finally, at the lowest level, are organizations who do
not see a need to become members of an email peering
organization and who exchange email with one or two
operators who are members of the peering organization.
However, these organizations will also be bound by an
explicit contracted AUP because the whole point of this
peering hierarchy is to have consistent accountability
throughout the entire email architecture.

This will not prevent spam, but it will provide operators
with the power to shut it off, whenever it occurs. It would
be useful to also have the ability to verify initial senders
of email messages, however that is not essential for this
peering architecture to be useful.

Here is a sample scenario. Grandma opens an email with a trojan
inside it. The trojan installs itself in her machine and starts
sending spam through Grandma's broadband connection. One of the
spam recipients informs their ISP that they just received a spam
message. Their ISP has a look and agrees that this is indeed spam
and they see that this spam is still coming in from a neighboring
operator. The ISP follows the contractual procedure and sends an
official notification of SPAM to the neighbor. The neighbor follows
a similar process and identifies the source as Grandma's ISP.
They issue a formal notice to Grandma's ISP and 10 seconds after
the notice is received, Grandma's IP connectivity is blocked
entirely except for HTPP accesses which are all directed to
a walled garden explaining the situation and recommending steps
that can be taken to clean up trojans, spyware, viruses, etc.

What is missing today?
- contracted email SLAs between operators
- contracted admin interoperation procedures between operators
- contracted SLA and AUP with customers that allows immediate
  shutdown when malware is detected
- organizations which can sort out all the details of the
  above contracts, etc.

If the BGP peering side of the business can sort out all of
this stuff, then why can't the email side of the business do
the same, or perhaps, do even better?

--Michael Dillon

40

I just don't see where Carl advocates email peering there.

More like "should J Random Luser be given control of mailservers" or
"Should Wile E Coyote be allowed to buy Dynamite and gadgets from the
Acme Company?"

That, and "if you want to operate a mailserver, get a static IP"

--srs