this may be deemed off topic - if so apologies in advance. however i respect many of the opinions i see here so thought i would take a chance and ask.
we are a stub network, injesting about 30k emails daily. about a year ago we implemented a spam filtering product. it works well. recently we turned on the knob to enable it to do reverse lookups. only the mild version, a reverse is made on the ptr rr for the ip address sending the email. if it fails the spam filter issues a 421 and closes the connection. unfortunately, we have 6 sites thus far that are legitimately trying to communicate with us but don't have ptr's associated with the ip address sending emails. since it obviously isn't a requirement to have one is it generally accepted to do so? any sense for how many end networks do and don't?
thanks in advance.
garrett.allen@comcast.net [2/7/2004 4:55 PM] :
this may be deemed off topic - if so apologies in advance. however i respect many of the opinions i see here so thought i would take a chance and ask.
we are a stub network, injesting about 30k emails daily. about a year ago we implemented a spam filtering product. it works well. recently we turned on the knob to enable it to do reverse lookups. only the mild version, a reverse is made on the ptr rr for the ip address sending the email. if it fails the spam filter issues a 421 and closes the connection. unfortunately, we have 6 sites thus far that are legitimately trying to communicate with us but don't have ptr's associated with the ip address sending emails. since it obviously isn't a requirement to have one is it generally accepted to do so? any sense for how many end networks do and don't?
Having proper rDNS is a good thing, strongly recommended but definitely not required for sending mail.
There are quite a few sites (including the freebsd.org mailserver, and, on a case by case basis, even AOL) that do refuse mail from IPs without rDNS, but turning on a "must have rDNS or you can't email us" setting will definitely result in a non trivial amount of false positives.
There are quite a few sites (including the freebsd.org mailserver, and,
on a case by case basis, even AOL) that do refuse mail from IPs without
rDNS, but turning on a "must have rDNS or you can't email us" setting
will definitely result in a non trivial amount of false positives.
but, i suspect that more and more of the world will go that
way, maybe slowly. so how does one judge when a site which
hosts public lists can do so without causing serious
discomfort for folk? is there some objective measure, or do
we all just shoot in the dark?
randy
[ private email not quoted ]
this is what i call shooting in the dark. what are
OBJECTIVE METRICS? for example, can operators measure and
publish alpha and beta error rates on a selection of sites
of different flavors so we can decide when they are low
enough for our flavor of site to enable rdns filters?
randy
garrett.allen@comcast.net writes:
we are a stub network, injesting about 30k emails daily. about a year
ago we implemented a spam filtering product. it works well. recently we
turned on the knob to enable it to do reverse lookups. only the mild
version, a reverse is made on the ptr rr for the ip address sending the
email. if it fails the spam filter issues a 421 and closes the
connection. unfortunately, we have 6 sites thus far that are
legitimately trying to communicate with us but don't have ptr's
associated with the ip address sending emails. since it obviously isn't
a requirement to have one is it generally accepted to do so? any sense
for how many end networks do and don't?
I've run all my mailers with aggressive PTR checks for about a year, and
while some of my guests aren't getting all the e-mail that's sent to them,
it's had no impact on me other than that periodically I have to tell some
remote postmaster that their PTR's are missing or that they don't match
the HELO hostname. Invariably they fix it.
This is counter to RFC821, and also Jon's old mantra "be liberal in what
you accept and conservative in what you generate." The trouble is, this
is information warfare now, not a grand experiment in interoperable
communications. The bad guys aren't going to be conservative in what they
generate. By being nonliberal in what I accept I keep 20,000 or more
malicious messages per day from hitting my personal inbox. This brings
the load down to something postgresql and MH can actually cope with:
lartomatic=# select date(entered),count(*) from spam where date(entered)>'today'::date-'10 days'::interval group by date(entered) order by date(entered) desc;
date | count
I think the tipping point went by a while ago, and that anyone
who wants their e-mail to be accepted will make sure their mail
relay has a PTR and that that this PTR holds the same name used
in the SMTP HELO command.
so you think it is fine if i require rdns for the ietf and other
mailing lists i host? i suspect others will not.
randy
DNSOP has been batting this issue around for years, and it periodically
comes up in most oether Internet forums on a regular basis.
Requiring DNS IN-ADDR Mapping: draft-ietf-dnsop-inaddr-required-04.txt
This is one of those self-defeating requirements. It works while there
isn't a strong requirement. But if we could actually get everyone to
implement it, it would cease to be an effective method.
If www.google.com required in-addr ptr records, I suspect most spam
sources would have properly configured in-addr ptr records.
Should IETF sponsored resources follow the details of various RFCs?
It depends. IETF sponsored resources probably should be expected to
follow STDs. But what about other RFCs and Internet-Drafts?
Randy Bush wrote:
this is what i call shooting in the dark. what are
OBJECTIVE METRICS? for example, can operators measure and
publish alpha and beta error rates on a selection of sites
of different flavors so we can decide when they are low
enough for our flavor of site to enable rdns filters?
The only usable metric you have, Randy, is the amount of collateral damage you would face on your network if you turned on rDNS.
Trust me, the error rates on sites would vary widely, especially because you cannot assume a standard / uniform population of people sending mail to these sites.
People who mail IETF lists, and people in asia, sending mail to a local list server in the rDNS-less desert that is some parts of APNIC land, might have completely different perceptions on the issue.
Now, from your logs, just how much legitimate mail do you get that comes from an IP without PTR RR, and how much is that expressed as a percentage of legitimate incoming mail to your lists? How much is that as a percentage of spam inbound to your list [to be fair, let's make it "spam that would not have been stopped by your other filters]?
srs
Now, from your logs, just how much legitimate mail do you get that comes
from an IP without PTR RR, and how much is that expressed as a
percentage of legitimate incoming mail to your lists? How much is that
as a percentage of spam inbound to your list [to be fair, let's make it
"spam that would not have been stopped by your other filters]?
well, it's gonna be hard to quantify what spam filters, bounces,
... rejected, and especially to associate them with smtp connections.
but here are some simple data.
yesterday, a saturday (gmt clock)
48293 total incoming smtp connections
6226 from ip addresses for which no ptr exists (12.89%)
12416 unique ip addresses which connected to server
3112 unique ip addresses for which no ptr exists (25.06%)
some inferences might be drawn, but i am hesitant to do so.
what numbers do YOU have for your server(s)?
randy
Now, from your logs, just how much legitimate mail do you get that comes
from an IP without PTR RR, and how much is that expressed as a
percentage of legitimate incoming mail to your lists? How much is that
as a percentage of spam inbound to your list [to be fair, let's make it
"spam that would not have been stopped by your other filters]?
btw, i did give you a cursory answer to this. and i asked if you
would provide some data.
randy, waiting
Randy Bush wrote:
btw, i did give you a cursory answer to this. and i asked if you
would provide some data.
As I told you in another email, a good 40% of my mail on my personal colo comes from sources without rDNS - mostly within India, from and to some fairly large linux lists.
regards
-suresh
Randy Bush wrote:
btw, i did give you a cursory answer to this. and i asked if you
would provide some data.
After all the filters that I use were applied - 682 unique IP addresses, with no rDNS sent at least 6124 legitimate (assumed legitimate as not caught by my filters) emails in the period from Feb 2 to Feb 10.
This box hosts mail for two large linux lists. And hosts my email, and email for my parents and a few friends. I and my friends are subscribed to various local linux and other mailing lists - several of which have no rDNS.
srs
As I told you in another email, a good 40% of my mail on my personal
colo comes from sources without rDNS - mostly within India, from and to
some fairly large linux lists.
After all the filters that I use were applied - 682 unique IP addresses,
with no rDNS sent at least 6124 legitimate (assumed legitimate as not
caught by my filters) emails in the period from Feb 2 to Feb 10.
This box hosts mail for two large linux lists. And hosts my email, and
email for my parents and a few friends. I and my friends are subscribed
to various local linux and other mailing lists - several of which have
no rDNS.
so, it's a long way from your 'tipping point'.
as your percentiles look worse than those here, do you think
it is indian dns maintenance or politics, or linux list
users, or both?
randy
Randy Bush wrote:
as your percentiles look worse than those here, do you think
it is indian dns maintenance or politics, or linux list
users, or both?
DNS maintenance, and cluelessness at several ISPs - quite a few ISPs don't even have PTR RR for their smarthosts.
Luckily most of the larger ISPs around do have PTR for their smarthosts, but several of them seem to think it is not necessary for their dialup / dsl pool, or even for their colo clients.. rDNS is an "added extra" that you have to pay for)
Good thing = there's a regional netops conference called SANOG (South Asian NOG, at http://www.sanog.org) that concentrates on the indian subcontient region and is also priced cheaply enough so that lots more people from the subcontinent can attend.
Priced cheaply as in other netops conferences are dirt cheap in dollars, but USD 1 = INR 48, and the average monthly salary of a mid level ISP sysadmin is something like USD 500..600.
srs