SMTP store and forward requires DSN for integrity (was Re:Clueless anti-virus )

From Fri Dec 9 13:59:30 2005
From: Douglas Otis <>
Subject: Re: SMTP store and forward requires DSN for integrity (was Re:Clueless anti-virus )
Date: Fri, 9 Dec 2005 11:58:15 -0800
To: Todd Vierling <>

> 1. Virus "warnings" to forged addresses are UBE, by definition.

This definition would be making at least two of the following

FALSE TO FACT. Notice the qualifier "forged", regarding the address to
which the notification is being sent.

1) Malware detection has a 0% false positive.

If there is a 'false positive' decting malware, it is a near certainty
that the "legitimage" message so classified does *NOT* have a FORGED ADDRESS.

In addition, _IF_ a false positive occurs, the *sender* can do nothing about
the situation -- resending the message, for example, will *NOT* have any
better chance of getting through. The _right_ person to notify in such a
situation is the RECIPIENT. They have a chance of 'influencing' the people
who set the policies (that resulted in that false postive) in regard to
getting the policy _changed_.

Thus, this is an invalid straw-man argument. One down.

2) Lack of DSN for email falsely detected containing malware is okay.

*IF* the sender address _is_ forged (a _necessary_ pre-condition for Todd's
statement to be applicable, then the fact remains that that 'false positive'
indication is going to someone who DID NOT REQUEST such notification, either
explicitly (by signing up for it), nor implicitly (by actually _sending_ the
message in question).

Thus, this is an invalid straw-man argument. Two down.

3) Purported malware should be assumed to use a forged return-path.

I can't imagine why anyone could *possibly* think that that might be the case!

Note: It does happen to be an assumption that does turn out to "coincide with
      the actual facts of the situation" in a _very_large_ share of the cases.

In the last three years of rejecting (SMTP transaction-time) _anything_ that
contained anything that 'looked like' either an MS-executable format
attachment, or a ZIP file, I have not seen a *SINGLE* such message that did
have an accurate return path.

Available statistical data from other sources shows a "not quite that extreme"
proportion. The _lowest_ numbers I've seen put the proportion at well over
90% -- basis the numbers of messages encountered.

A trivial, surface-level, analysis of the goals of virus-writers shows that
virus writers, in general, have *every* reason to obsure the "source" of the
message, and _no_ reason for the message to "clearly identify" the actual
message source. That once the virus-writer "community" figured out how to
send messages with spoofed origin information, the _virtually_every_ virus
released after that point *does* use such spoofing -- for the express
purpose of making it 'as difficult as possible' to identify the true source
of the infection-carrying message.

It is also an "obvious fact" that people who are in the *business* of selling
malware detection systems, have a business interest in "identifying" and
"classifying" malware. "What it does", and "how it works" ARE (or should be)
reasonable and expected parts of their identificaton/analysis/classificaiton
process. Thus, 'malware detection system' vendors -- of all people -- should
be expected to know _more_ about whether or not a particular 'detected'
malware (regardless of whether or not it was a 'false positive' detection)
is one that forges the 'pooint of origin' information that would be used
to route a DSN.

Thre is no need for the authors/sellers of 'malware detection systems' to
"assume" any specific behavior as a 'default' assumption. They *SHOULD*
*KNOW* the actual of every virus that they have an 'identification signature'
for. thus there is no need to assume anything, they can act case-by-case
on "actual", factual data.

Thus, this is an invalid straw-man argument. Three down.

4) The return-path can be validated prior to accepting a message.

"wHO CARES" ABOUT the return-path? if you make the malware determination
*at*the*exterior*gateway** to your network, *during* the SMTP transaction,
you have a 'reliable' path back to the system that tried to deliver it _to_
you. If *they* don't know (reliably) who the actual sender of that message
is, that is _their_ problem.

If you _cannot_ make a real-time "malware" determination during the _gateway_
SMTP transaction, and the gateway has "accepted for delivery" the questionable
message, then when the 'malware' determination is made, the message should
be simply _delivered_, according to site policy -- directly to the bit-bucket.
As this _does_ meet the 'delivery' requirements of the relevant RFCs, thre
is no need to send a failure notification 'backwareds' to anywhere.

Thus, this is an invalid straw-man argument. four down.

5) SMTP should appear to be point-to-point.

An overly-broad statement, without any real meaning.

SMTP already _is_ point-to-point, relative to "point-to-multipoint" (broad-
casting), and/or 'multipoint-to-point" (semders to the same destination over-
writing each other).

A message just goes through a series of point-to-point links en route to
the destination.

I suspect you meant "point-to-point" as a reference to a system where the
actual sender makes a direct connection to the reveiver's mailbox. This is
really described as a end-to-end snychronous connection. No 'store and
forward' capability, no 'asynchronous' transfers.

IF that is what you mean, it is a 'conclusion' not supported by the facts
so far in evidence. With "minor" changes to the current SMTP protocol,
and the various implementations, the issue of delivery failure notices
to a forged address can route the message back to the *actual* sender,
and _reliably_ so.

First, an MSA (to use the 'sendmail' terminology) must know how to 'reliably'
send a message back to the _user_ who submitted the message, *before* it
accepts that 'submitted' message.

Second there is a need to _retain_ that information on a _per_message_
basis, until the allowed 'message delivery failure" interval has elapsed.

Third, when a message is passed on to another MTA, the remaining time
until the _original_ message delivry failure time-limit is reached. this
will be the 'original' time remaining less any time that it has spent
sitting _IN_ the queue on the server.

any server can have a 'maximum" time-limit, and if the time-limit on an
incoming message is _greater_ than that maximum, it 'automatically gets
reduced to the 'site maximum".

Now, if a message is still "undelivered" when the delivery failure time-
limit is reached, the message is removed from the local outgoing queue,
And a status notification is sent *to*the*server* that delivered the message
to the system where it timed-out.

Each system going back 'up' the delivery path will then "pass up the line"
TO THE SERVER that sent to that systrem, of the delivery failure.

eventually, the 'failure' message gets back to the MSA that accepted the
message, and "knows" _reliably_ how to contact the actual sender. and
*that* machine mails the actual failure notice to the actual sender.

You've got asyncrhonous, store-and-forward, message passing in *both*
directions. It's _not_ "point-to-point'. It takes some protocol
extensions, and supporting implementations, to make everything work.

OK, that was a lengthy debunking, but it is debunked. five down.

6) MTAs with AV filters are the only problem.

And _what_, pray tell, *OTHER* than an "MTA with AV filters" is going to
automatically generate a 'virus warning' to an address that was forged as
the sender of a virus-infected email?

Yes, there are other sources of UBE. That is utterly irrelevant to
whether or not "systems that send *unsolicited* messages to persons
who have had _zero_ contact with that system" are sending UBE.

A claim that "X (a malware detecting system) is a member of set "Y" (UBE
emitters) *DOES*NOT*IMPLY* that "X" is the _only_ member of that set.

That the set has other members does _not_ invalidate X's membership
membership in set "Y",

Thus, this is _also_ a false claim.

wups. that's 6 out ot 6. nothing left.

While there could be best practices created for AV filtering, it
seems unlikely to be effective. Simplistic filters for DSNs also
seem counter to ensuring the integrity of email delivery. Defending
against DSN exploits with BATV will remove this vector, which in turn
will end DSN exploits attempts over the long term. Why expect others
to fix this problem, when there is a solution that one could make the
investment in to deploy.

Why should _everyone_ have to "make the investment", when a much smaller
investment by a *much*smaller* population (the 'malware detection system
_vendors_) would utterly *eliminate* the _problem_as_stated_ -- to wit:

     Virus "warnings" to forged addresses.

                          This will reduce this problem over the long
term. The BATV alternative would not cause otherwise valuable DSNs
to be lost, nor make assumptions about the quality of malware detection.

If you can't trust AV handling of DSNs, why trust their detections?

Would you rather see emails simply disappear?

Quote: "E-mail is *NOT* a _reliable_ communicaitons mechanism."

> 2. It is the responsibility of the *SENDER* not to send UBE.

When the recipient is a legitimate email provider, it could seriously
lower the integrity of email delivery for these providers to toss
DSNs because many fall into a category you want to define as UBE.

_I_ am *NOT* the sender of the message.

I DID NOT SOLICIT any notifications about messages sent by *OTHER*PEOPLE*.

Sending _me_ those messages does more to 'lower the integrity of email
delivery' than not sending them to me.

What it does is teach me to *ignore* non-delivery messages -- because
they are KNOWN TO BE WRONG for virtually every non-delivery notice I
receive. (bogus -- for messages I did _not_ send -- notifications, _for
_me_, outnumber 'legitimate' ones -- for messages I *DID* send -- by more
than 500:1.

While I agree whole heartily this malware notification problem
stinks, there is a much safer and surer solution.

Yup. After the third bogus notification within 12 months, I configure
my mail-server to not accept *any* traffic from the offending "sending"
server. The SMTP transaction rejection message says _exactly_ (and in
rather pointed language) why no mail from that system is accepted, and
suggests that the sender try sending _from_a_different_network_ if they
really want to contact the addressee.

Orders of magnitude less CPU cycles than BATV. Far friendlier on _my_
network bandwidth. It may not be 'friendly', and it may not be appropriate
for larger sites

When there is some percentage of false-positive detection, there will be a number of messages that will fall into the "should not have been rejected" category, where indeed the return-path is not likely to have been forged, and a DSN would be of value to the sender. When a DSN is sent, the sender will be able to take corrective action. There is also a percentage of messages where malware detection is valid, but nonetheless the return-path is also valid. (Perhaps overwritten by the provider.)

You are judging this situation based upon only the wrong choice as having been made. AV filtering is not the only situation where a DSN exploit is used, and there is no way to be sure about a choice of discarding the DSN. Discarding DSNs _will_ degrade the integrity of email delivery. As the recipient of the DSN is _always_ the best judge whether the DSN was sent to a forged return-path, why not take advantage of that superior knowledge? Automate the process so the DSN recipient is able to immediate rejects _all_ invalid DSNs. Overall, email transactions will be faster, and DSN exploits will soon disappear.


Robert, sorry I missed the full conversation, and don't have time to read the whole thread, but based on your mail alone a few words of agreement...

Please remember people..

RFC 2821 states explicitly that once the receiving server has issued a 250 Ok to the end-of-data command, the receiving server has accepted responsibility for either delivering the message or notifying the sender that it has been unable to deliver. RFC2821 also says that a message MUST NOT be dropped for trivial reasons such as lack of storage space for the message. To that end is a detected virus/trajan/malware/phishing scam etc... a trivial reason to drop the message?

Personally I believe that not trivial means not unless the entire server crashes and disks fry etc... To that end I am a firm believer that malware messages SHOULD BE rejected at the end of the data command (which is why I have gone to great lengths to ensure this happens at $employer, and at SORBS).. Failure to have the resources available to perform the virus scanning will result in the messages being delivered to the recipient as a broken message (attachment stripped).

There is certainly NO EXCUSE for ANYONE to bounce virus warning messages to ANY user whether local or remote, particularly when the anti virus software will identify the virus and the virus is KNOWN to forge the sender address.

As such anyone bouncing large numbers virus warning messages are game for having their servers blocked, and I will not apologise to anyone getting caught by a SORBS automated spamtrap getting a virus warning message (though I will remove them promptly when notified of such an entry).



rfc2821 was written prior to this problem

we should also take the rfc in context and differentiate between email sent
between individuals for which the responsibility applies, and email generated by
systems (spam, virus bounces) in which we the providers carry some
responsibility to drop them (okay, it would be better if they didnt exist in the
first place, but thats not reality) if they can be identified in the best
interests of the user

to not do this is like saying we have a responsibility to ensure end to end
delivery of packets in a DoS attack just because the rules governing routing and
ip stacks dont explicitly cover the use of sinks and filters.


I'm *loving* your crack-induced comedy. Troll it up, bay-bee!

Show me the false positive rate. If you can prove any site with more than
0.00001% FP on malware detection with any off the shelf product, I'll give
you a magic cookie.

Abusing the rest of the world does *not* justify saving the single,
incredibly improbable, FP in comparison. Abuse is abuse; it's not up to
millions of receiving hosts to put up with the abuse happily because of your
hallucinogenic "false positive".

Put down the crackpipe and walk away from the keyboard before you say
something else your employer will regret. (These might be "your opinions",
but it's telling if a company continues to employ someone who doesn't
understand said company's market at all, in spite of any disclaimers.)

Date: Sat, 10 Dec 2005 22:54:24 +1100
From: Matthew Sullivan

RFC 2821 states explicitly that once the receiving server has issued a 250
Ok to the end-of-data command, the receiving server has accepted
responsibility for either delivering the message or notifying the sender
that it has been unable to deliver. RFC2821 also says that a message MUST

And RFC 1034/1035 (I forget which) specifies an RD bit which, in
reality, is rather useless.

Following RFCs is good for compatibility. However, RFCs are hardly
infallible word from on high. Sometimes they require revisiting and