What must one do to avoid Gmail's retarded non-spam filtering?

I realize that this is somewhat OT, but I'm sure that others on the list encounter the same issues and that at least some folks might have useful comments.

An increasingly large number of our customers are using Gmail or Google Apps and almost all of our OSS/BSS mail is getting spam filtered by Google. Among others, these e-mails include invoices, order confirmations, payment notifications, customer portal logins, and tickets. Almost anything we send to customers on Google ends up in their spam folder. This results in a lot of calls and makes much of our automation pointless, never mind all the lost sales.

The problem is compounded by those who use mail clients and do not log in to the webmail at all, since they would never see the contents of the Google spam folder.

We have proper A+PTR records on the edge MTAs, proper SPF records for the originating domain, proper Return-Path and other headers, and so on. There isn't anything that I can think of other than the content itself which would be abnormal, and obviously the content is repetitive and can't be changed much. Is there something obvious which we've missed?

Aside from the following clearly impractical solutions, what can we do?
1. Asking everyone (including those we don't even know yet) to whitelist all of our addresses, to check their spam folders, and to click on "this is not spam"
2. Providing our own free e-mail service to everyone (including those we don't even know yet) and putting up "don't use Google" ads on all of our customer-facing systems

At least this isn't Hotmail where mail is just silently deleted with no NDR after it's accepted by their MTAs.

The call volume has been going up instead of down lately and it's gotten to the point where we're sending MTA log extracts to people to prove to them that we really did e-mail them.

Would greatly appreciate any advice.

Erik

Erik,

Do you send marketing information to your customers from the same IP
addresses as you send your invoices?

Have you checked the source IPs against the RBLs with, e.g.
http://rbls.org/? What about the source IP of the specific machine the
originates the invoices et. al?

Is the invoicing machine also in SPF? One of the companies I do
business with accidentally excluded their billing mailer from the SPF
record.

Have you sent copies of the various types of documents to a mail
server running Spam Assassin in order to see which qualms Spam
Assassin flags? Careless mistakes with the headers or the clock can
get your message flagged as spam. Even something as simple as ALL CAPS
can get you flagged.

Have you added your invoicing mailer, the one you can guarantee sends
no marketing materials, in to the whitelist at http://www.dnswl.org/?

Regards,
Bill Herrin

Hi,

Have you checked the IronPort reputation scores for your mailserver IPs?
Google uses this data as part of it's spam detection method.

William

Bill,

Thanks for the response and excellent ideas.

The OSS/BSS box is in our internal network with a RFC1918 address and it relays to outside via another box which sits in the DMZ and which is in SPF records for the domain that appears in From/Return-Path, etc.

That DMZ machine does not appear on RBLs nor is it whitelisted anywhere.

Google adds headers like these:
Received-SPF: pass ...
Authentication-Results: mx.google.com; spf=pass ...

So the problem is unlikely to be a SPF issue, as mentioned in my first e-mail.

We haven't sent any marketing mail from the same IP in months, but I believe that we did previously. We do send out once every few months a network maintenance notice with identical content for all recipients, except their name and To header.

Clocks are set correctly everywhere via NTP and Received headers in the messages are >= each of the previous Received headers as you go up.

We do use ALL CAPS in some subjects (rare) but more commonly, some customer names and other data in the body does appear in caps. We also have a www.caneris.com link on most e-mails (removing it doesn't change gmail's behaviour). Messages without anything ALL CAPS in them end up in spam as well.

I created a new mailbox under a Google Apps domain and sent it one of the typical messages. It went into spam. I also ran the same message through SpamAssassin as per your suggestion and it came out clean.

A test message sent directly from the same edge MTA but with a From address in a different domain is not delivered to spam, even though Return-Path there is still an @caneris.com address (but different than our usual one). A test message sent from our corporate mail system (Zimbra box in internal network relaying to the same edge MTA as the one used by the OSS/BSS box) goes to spam.

I've received several off-list responses, including those from Google Apps users, asking for sample messages, so will be forwarding those.

Ideas:
-Message-Id @ "non-existant host" (ones in our internal network)
-Received headers showing that the message was relayed, and again from a "non-existant host"

Unlikely to be an IP repuation issue given the one message above did go through fine.

Erik

Hi William,

I do so for our entire IP space on a regular basis. The edge MTA I mentioned in the reply to Bill shows up as "Neutral" there.

Thanks

Erik

I realize that this is somewhat OT, but I'm sure that others on the list encounter the same issues and that at least some folks might have useful comments.

An increasingly large number of our customers are using Gmail or Google Apps and almost all of our OSS/BSS mail is getting spam filtered by Google. Among others, these e-mails include invoices, order confirmations, payment notifications, customer portal logins, and tickets. Almost anything we send to customers on Google ends up in their spam folder. This results in a lot of calls and makes much of our automation pointless, never mind all the lost sales.

The problem is compounded by those who use mail clients and do not log in to the webmail at all, since they would never see the contents of the Google spam folder.

We have proper A+PTR records on the edge MTAs, proper SPF records for the originating domain, proper Return-Path and other headers, and so on. There isn't anything that I can think of other than the content itself which would be abnormal, and obviously the content is repetitive and can't be changed much. Is there something obvious which we've missed?

Have you tried DKIM signing? All email sent from Gmail is DKIM signed,
so they probably also support checking it and a valid signature may
lower your spam score.

Regards,
-Lorand Jakab

We have proper A+PTR records on the edge MTAs, proper SPF records for
the originating domain, proper Return-Path and other headers, and so
on. There isn't anything that I can think of other than the content
itself which would be abnormal, and obviously the content is
repetitive and can't be changed much. Is there something obvious
which we've missed?

What else goes out from that MTA? User forwarded mail? Mailing lists?
random customer junk?

It is a really good idea to separate your mail streams. These days it
means separate IPs for users, transactions, forwards, and such,
eventually different DKIM signatures will do the trick.

R's,
John

Erik L wrote:

Received-SPF: pass ...
Authentication-Results: mx.google.com; spf=pass ...

So the problem is unlikely to be a SPF issue, as mentioned in my first e-mail.

http://david.woodhou.se/why-not-spf.html

The lack of SPF records should never be the reason to block an email. It's about time SPF is being laid to rest, it's broken, ineffective and a flawed concept. Well, it's effective in legitimising spammers and breaking forwarding for example, but otherwise...

Have you tried DKIM signing? All email sent from Gmail is DKIM signed,
so they probably also support checking it and a valid signature may
lower your spam score.

DKIM is definitively a must have for gmail.

At least this isn't Hotmail where mail is just silently deleted with no NDR after it's accepted by their MTAs.

If Hotmail is doing that to your messages, you're in a quite bad posture. That's their way to tell you "we don't like what you're sending at all".

Did you consider hiring deliverability consultants? I'm afraid there is no magical way to reach inbox without a audit of your process and systems.

Benjamin

Thanks John. This was a common question that was asked off-list. That edge MTA is not used and has never been used by anything/anyone other than us. No customer mail flows or has flowed in or out via it ever.

As I mentioned in my follow-up post, the issue at this point is that the domain has been blacklisted. I can send an identical message from the same MTA, changing only the From header, and it will be delivered to Inbox. Only when the From header contains @caneris.com will the message be delivered to spam. Any changes to the MTA IP, content, headers, etc. don't have any effect.

Erik

I don't believe in SPF, which is why we don't use it to check inbound mail. I do believe in being able to communicate with our customers irrespective of which provider they use, and given that Hotmail in particular is extremely unforgiving with respect to SPF, we have no choice but to publish such records.

Do you let customers have addresses under that domain?

~Seth

As I mentioned in my follow-up post, the issue at this point is that the domain has been blacklisted.

Most peculiar. Since gmail is driven by algorithms, that would suggest that a lot of your recipients are reporting your mail as spam, which I can assure you people do, no matter how legitimate and opt-in it is.

Do you have feedback loops set up? Google doesn't offer one, but AOL, Yahoo, Hotmail, and other large ISPs do, which will let you figure out what sorts of mail people are complaining about so you can perhaps send less of it.

R's,
John

Hello,

I would also recommend to implement the list unsubscribe header, because google is supporting that kind of user feedback to senders.

http://www.list-unsubscribe.com/

Regards,
André

André Görmer
Senior Deliverability Manager

eCircle
P: +49 89 12009-762 | F: +49 89 12009-750 | E: a.goermer@ecircle.com
Nymphenburger Str. 86, 80636 München

Stay in touch
Web: www.ecircle.com/de | Newsletter: www.ecircle.com/index.php?id=63&L=0

Für Hilfe mit dem eC-messenger wenden Sie sich bitte an unseren
Support unter Tel +49 (0)89 120 09 600 oder support-de@ecircle.com

Neuste Untersuchungen
Ein unschlagbares Doppel: E-Mail-Marketing & Webanalyse
Download Whitepaper: www.ecircle.com/index.php?id=61&L=0

eCircle GmbH, HRB 184 478, Handelsregister München, Geschäftsführung: Volker Wiewer (Vorsitzender),
Alexander Meyer, Thomas Wilke, Vorsitzender des Aufsichtsrates: Dr. Arnold Bahlmann-----Ursprüngliche Nachricht-----

No

Can you please not use the word "retarded" in a pejorative sense?

The word "please" is probably not required, since using that word in this manner is prosecutable hate speech in some jurisdictions.

> Can you please not use the word "retarded" in a pejorative sense?

The word "please" is probably not required, since using that word in this manner is prosecutable hate speech in some jurisdictions.

You're serious? :slight_smile:

mh

Really? Where? Seems to me equating "retarded" used as an adjective to say, speech of the KKK seems... lame.

Also, "retarded" was applied to software, which to date, has not been entered as a protected group.

-j

In message <AANLkTikaiBkwc3R2ijKHpYhb=i+ACyn_ht7jgthtHRNz@mail.gmail.com>,