Repeated Blacklisting / IP reputation

Mailman List Mirror (Read Only)
1

I am amazed with the amount of thoughtful comments I have seen, both on and off list. It really illustrates that people are willing to try to help out, but there is an overall lack of clear direction on how to improve things. Most of us seem to adopt that which has always just worked for us. Don't get me wrong, I'm sure there are a lot of improvements/mods going on with RBL operators in terms of the technology and how they choose who to block. I'm also certain that most of the carriers are doing their best to follow RFCs, use e-mail filtering, and perform deep packet inspection to keep themselves off of the lists. AND there seems to be some technologies that were meant to work, and cause their own sets of problems (example: allowing the end user to choose what is considered spam and blacklisting based on that). As was said before, it's not the "WHY" but rather how can we fix it if it's broke.

The large debate seems to revolve around responsibility, or lack thereof. In our case, we are the small operator who sits in the sidelines hoping that someone larger than us, or more influential has an opinion. We participate in lists, hoping to make a difference and contribute, knowing that in a lot of cases, our opinion is just that: an opinion. I suppose that could spark a debate about joining organizations (who shall go nameless here), power to the people, etc.

It seems as though a potential solution *may* revolve around ARIN/IANA having the ability to communicate an authoritative list of reassigned IP blocks back to the carriers. This could serve as a signal to remove a block from the RBL, but I'm sure there will be downfalls with doing this as well.

In my specific case, I am left with a legacy block that I have to accept is going to be problematic. Simply contacting RBL operators is just not doing the trick. Most of the e-mails include links or at least an error code, but some carriers just seem to be blocking without an error, or even worse, an ACL...

We will continue to remove these blocks as necessary, reassign IPs from other blocks where absolutely necessary, and ultimately hope the problem resolves itself over time.

Thanks again for the very thoughtful and insightful comments, they are greatly appreciated.

Regards,

2

How about a trial period from ARIN? You get your IP block, and you get 30
days to determine if it is "clean" or not. Do some testing, check the
blacklists, do some magic to see if there are network-specific blacklists
that might prevent your customers from sending or receiving email/web/other
connections with that new IP block.

If there are problems, go back to ARIN and show them your work and if they
can verify your work (or are simply lazy) you get a different block. ARIN
puts the block into another quiet period. Maybe they use the work you did
to clean up the block, maybe they don't.

Cleaning up a block of IPs previously used by shady characters has a real
cost, both in time and money. The argument as I see it is who bears the
responsibility and cost of that cleanup.

Beckman

3

Peter Beckman wrote:

How about a trial period from ARIN? You get your IP block, and you get 30
days to determine if it is "clean" or not. Do some testing, check the
blacklists, do some magic to see if there are network-specific blacklists
that might prevent your customers from sending or receiving email/web/other
connections with that new IP block.

If there are problems, go back to ARIN and show them your work and if they
can verify your work (or are simply lazy) you get a different block. ARIN
puts the block into another quiet period. Maybe they use the work you did
to clean up the block, maybe they don't.

Cleaning up a block of IPs previously used by shady characters has a real
cost, both in time and money. The argument as I see it is who bears the
responsibility and cost of that cleanup.

I encourage someone to write a policy proposal; I'd support it. They
(the recipient) didn't have a darn thing to do with it becoming a
wasteland and shouldn't bear the cost. Unlike bying a (insert your
favorite object here), you can't inspect an IP block before purchase.

I fear that "we don't guarantee routability" will rear its ugly head
even if someone were to pen an awesome policy. I feel it's a poor
position for a registry to take, though. They still get the money even
if you can't use them, and uh oh, looks like you won't qualify for more
until you use the unusable.

Probably getting off topic for NANOG, like most threads that get this long.

~Seth

4

sounds like domain tasting to me.

--bill

5

bmanning@vacation.karoshi.com wrote:

sounds like domain tasting to me.

Oops! Oh yeah. Spammer gets an allocation...

"Well, if that netblock was clean before, it sure isn't now! May I please have another?"

Lather, rinse, repeat.

6

Cleaning up a block of IPs previously used by shady characters has a
real cost, both in time and money. The argument as I see it is who
bears the responsibility and cost of that cleanup.

... and as we all know the fundamental axiom of Internet economics is
to foist of as many of your costs as possible on someone else.

If you get a new chunk of IP space, you find that it's listed in a lot
of private blacklists, and you're not able to get them to unlist you,
the reasonable conclusion is that they DO NOT CARE that you can't send
them mail. A way to get them to care is to get their own customers,
i.e., the people to whom you are trying to send the mail, to complain
to their mail managers. If that doesn't work, it either means that
the managers are incompetent, or that the recipients also DO NOT CARE
that you can't send them mail. I would guess that the latter
situation occurs a lot more than the former.

Telling people to time out their listings automatically is a
non-starter, because they will find nearly all of them are still
spamming or sending no mail at all, and an infinitesimal trickle
switched to sending legit mail. Who's going to do that?

Spam sucks, largely because spam is one of the most egregious cases of
foisting off costs on others. If you get a toxic block, find a
creative lawyer and sue the former assignee for fraudulent transfer or
something.

Regards,
John Levine, johnl@iecc.com, Primary Perpetrator of "The Internet for Dummies",
Information Superhighwayman wanna-be, http://www.johnlevine.com, ex-Mayor
"More Wiener schnitzel, please", said Tom, revealingly.

7

What's to stop spammers from doing this to cycle through blocks in rapid-fashion?

This proposal seems easily abusable to me.

- S

8

Skywing wrote:

What's to stop spammers from doing this to cycle through blocks in rapid-fashion?

This proposal seems easily abusable to me.

Oh, I don't know, maybe ARIN staff can say no? The process is heavy with
human interaction, there is nothing "rapid" about it, and bears no
comparison to the automated process of registering a domain name. You'd
know that if you ever had to make a request for a number resource from ARIN.

~Seth

9

Skywing wrote:
> What's to stop spammers from doing this to cycle through blocks in
rapid-fashion?
>
> This proposal seems easily abusable to me.
>

Oh, I don't know, maybe ARIN staff can say no? The process is heavy with
human interaction, there is nothing "rapid" about it, and bears no
comparison to the automated process of registering a domain name. You'd
know that if you ever had to make a request for a number resource from
ARIN.

The problem of tainted ipv4 allocations probably grows from here since at
some point in the near future there isn't going to be much left in terms of
"clean" space to allocate. We're running out of v4 addresses in case anyone
forgot.

Not sure that this is an ARIN problem more than an operational problem since
RBL's are opt-in. An effort to identify RBL's that are behaving poorly is
probably more interesting at this point, no?

Best Regards,

Marty

10

I suspect the problem isn't poor RBLs, it's all the little one-off block lists
out there. The NANOG lurker in the next cubicle informs me that we currently
carry an astounding 52,274 block entries (to be fair, a large portion is due to
our vendor's somewhat-lacking block list - if we decide a /24 is bad, but then
want to whitelist 1 IP, we have to de-aggregate to 254 black entries instead).
We get maybe 5-6 blocked e-mail complaints a day - which *still* represents
better performance for our end users than if we didn't carry around that many
blocks (for comparison, we get at least 3-4 times that many tickets a day for
people who forgot their e-mail password and need a reset).

And yes, it's *very* intentional that we have a business process in place
that makes it trivially easy for one of our users to open a "I can't get
e-mail from <here>" and get it taken care of *very* quickly, but opening a
"We can't send e-mail to your users" is a lot more challenging and time
consuming (at least for the complaintant).

Now, if we didn't have a dedicated, hard-working, and skeptical lurker in the
next cubicle, our block list *would* be a mess.. :wink:

11

Somewhat apropos to this discussion:

http://blog.icann.org/2009/09/selecting-which-8-to-allocate-to-an-rir/

Regards,
-drc

12

Hi Tom (and NANOG),

You may be interested in an alternative approach, motivated by the
very problem you are facing (see below). Our system, SNARE, develops
IP reputation automatically based on a combination of network
features. We'll discuss the pros and cons of this approach at MAAWG.
The additional information that SNARE provides might be helpful.

-Nick

Detecting Spammers with SNARE: Spatio-temporal Network-level Automatic
Reputation Engine

Shuang Hao, Nadeem Ahmed Syed, Nick Feamster, Alexander Gray, Sven Krasser
Usenix Security '09, Montreal, Canada, August 2009

Users and network administrators need ways to filter email messages
based primarily on the reputation of the sender. Unfortunately,
conventional mechanisms for sender reputation -- notably, IP
blacklists -- are cumbersome to maintain and evadable. This paper
investigates ways to infer the reputation of an email sender based
solely on network-level features, without looking at the contents of a
message. First, we study first-order properties of network-level
features that may help distinguish spammers from legitimate senders.
We examine features that can be ascertained without ever looking at a
packet's contents, such as the distance in IP space to other email
senders or the geographic distance between sender and receiver. We
derive features that are lightweight, since they do not require seeing
a large amount of email from a single IP address and can be gleaned
without looking at an email's contents -- many such features are
apparent from even a single packet. Second, we incorporate these
features into a classification algorithm and evaluate the classifier's
ability to automatically classify email senders as spammers or
legitimate senders. We build an automated reputation engine, SNARE,
based on these features using labeled data from a deployed commercial
spam-filtering system. We demonstrate that SNARE can achieve
comparable accuracy to existing static IP blacklists: about a 70%
detection rate for less than a 0.3% false positive rate. Third, we
show how SNARE can be integrated into existing blacklists, essentially
as a first-pass filter.

http://gtnoise.net/pub/index.php?detail=14

13

The reuse issue is possibly decades away in v6 land.

The reuse issue can't really be solved for v4 in a year or two.

Sounds like a waste of time to develop this idea further IMO.

A