A friend of mine operates a blog at seeingtheforest.com, and he pays for traffic over a (fairly minimal) cap. He posted this comment recently:
http://www.seeingtheforest.com/archives/2007/01/eating_bandwidt.htm
Eating Bandwidth
Last month something ate up a tremendous amount of bandwidth at Seeing the Forest, costing me a lot of money. So now I regularly check bandwidth use.
Why has 209.160.72.10, HopOne in DC, been eating a HUGE amount of bandwidth? Gigabytes! What are they doing? (I banned them.)
Why has 220.226.63.254, an IP in India, been eating a tremendous amount of bandwidth? What are they doing?
Why has 195.225.177.46, an IP in Ukraine, been eating a tremendous amount of bandwidth? What are they doing?
Why has 62.194.1.235 AND 83.170.82.35 AND 89.136.115.220 AND 62.163.39.183 AND 212.241.204.145, all from the /same company/ in Amsterdam, been eating a TREMENDOUS amount of bandwidth? What are they doing?
Why is 206.225.90.30 and 69.64.74.56 and Abacus America Inc.eating a TREMENDOUS amount of my bandwidth,
Yes. Fistfulofeuros.net has seen dramatically higher levels of comments spam since last autumn. Not as much as below, but we were offline due to supposed overuse (I say supposed because our host claimed a script we don’t have was responsible) over Christmas.
this isn't in the ukraine, it's in NYC behind ISPrime. Phil is fairly
hhelpful, you might ask them to 'figure out what the heck is going on'
with that ip 
-Chris
(unless the ukraine got a whole lot closer to IAD than I thought:
64 bytes from 195.225.177.46: icmp_seq=1 ttl=55 time=13.1 ms
64 bytes from 195.225.177.46: icmp_seq=2 ttl=55 time=24.5 ms
)
Thomas,
Can you please send logs of what you have from 195.225.177.46 to abuse@isprime.com?
Thanks,
--Phil
I was asked to join late in 2005.
Most of the IP addresss you listed are are already on various DNS
blacklists.
Tony.
[Blog spammers]
Most of the IP addresss you listed are are already on various DNS
blacklists.
Ooh, now that is interesting. I had assumed that the DNSBLs only covered SMTP spam sources, but on reflection I suppose SMTP is a dead protocol these days in the wider Internet.
For the benefit of those of us who have been lucky to Recover from ISP work and now herd blogs[0], would you be so kind as to share which blacklists are worthwhile and worth consulting on this front?
[0] Before you ask, no, it's no easier, in fact arguably harder work, although the pay and hours are better. But yes, we're hiring.
Your assumption is incorrect. These DNSBLs cover spam sent in email,
indeed. Thing is, spam is spam and spammers are spammers. Meaning, they
spam in every way they can.
In my experience 20-70 per cent would be flagged by email DNSBLs. Not
accurate to filter out blog spam.
As in, bots will be bots.
I've been working on a new DNSBL for comment/etc. spam for a while, which
will be reliable, generally, it doesn't exist yet for public consumption.
There is such a black listing service already, but again, reliability is
an issue.
Gadi.
Gadi, if your HTTP spam DNSBL gets working, we would certainly be interested in feeding our spam filter from it. It is my experience so far that comments spam is not very “botnetty” but more “boxy” - the proportion of the total we get from any single IP address is relatively high.
Actually, to put that better, rather than being evenly distributed over many IPs, a core-group of the IPs spamming us at any one time account for the bulk of it. 80/20 rule again
For the benefit of those of us who have been lucky to Recover from
ISP work and now herd blogs[0], would you be so kind as to share
which blacklists are worthwhile and worth consulting on this front?
Peter,
I am not affiliated with any of these products :), but here is a good link
and info on combating spam comments on blogs. I know of a number of people
and organizations using akismet and have had great success with it
http://akismet.com/
And though this link here is specifically for wordpress it gives a bit of
good info on combating spam comments.
http://codex.wordpress.org/Plugins/Spam_Tools
Hope this helps a tad bit.
Heck feed it from spamkarma 2 or askimet. I use spamkarma 2 and it routinely nails tons of blog spammers..
Alexander Harrowell wrote:
I would expect the lists of compromised hosts to be fairly effective -
open proxies of various kinds and perhaps botnet hosts. As for SMTP the
blacklists would only be a starting point that either provide a cheap
preliminary check or feed a more sophisticated filtering system.
Tony.
I tihnk the real trick is to make sure the list does NOT include dynamic IP
space.
If you allow anonymous, unauthenticated access to any system it will
be abused. Auctions, blogs, chat, mail, phone, etc. IP addresses
have never been good authenticators for applications. Sending confirmation E-mail addresses aren't that much better. And blacklists will just continue to grow longer.
How do you know your user?
>
> For the benefit of those of us who have been lucky to Recover from ISP work
> and now herd blogs, would you be so kind as to share which blacklists are
> worthwhile and worth consulting on this front?
I would expect the lists of compromised hosts to be fairly effective -
open proxies of various kinds and perhaps botnet hosts. As for SMTP the
blacklists would only be a starting point that either provide a cheap
preliminary check or feed a more sophisticated filtering system.
Honestly, the more advanced we get we still can't get a hold on this
issue. Imagine you run a blog services web site, and each blog gets
between 1000 and 1,000,000 comment spams a day. Or even just one blog with
several thousand such.
Advanced systems based on "time on page", "direct to post link", capctahs,
Javascript captchas or challenges, URL in name, URL in DATA, # OF URLs,
etc. are all fine scoring rules, add to that a DNSBL and you will be fine
to a level... until next week.
There are quite a bit of botnets involves, but a lot of "mass-mailers" are
still in this business.
This is not very NANOG relevant and I feel I contributed enough on the
subject (unless the membership keeps responding), but it is a very serious
issue. There is a mailing list dedicated to this subject, you can ping me
off list if you are interested in the topic.
I would expect the lists of compromised hosts to be fairly effective -
open proxies of various kinds and perhaps botnet hosts. As for SMTP the
blacklists would only be a starting point that either provide a cheap
preliminary check or feed a more sophisticated filtering system.
If you allow anonymous, unauthenticated access to any system it will
be abused. Auctions, blogs, chat, mail, phone, etc. IP addresses
have never been good authenticators for applications.
This is not true if you control the IP address space and the routers around it.
I mention this merely because "IP addresses have never been good authenticators"
or the like is becoming a truism. For ISPs with good source filtering in place
then IP addresses ARE good first level authenticators (e.g. filter lists
on management ports). Note: I say FIRST level authenticators; IP addresses are
obviously not suitable as the whole authentication process.
If you allow anonymous, unauthenticated access to any system it will
be abused. Auctions, blogs, chat, mail, phone, etc. IP addresses
have never been good authenticators for applications.
This is not true if you control the IP address space and the routers around it.
I mention this merely because "IP addresses have never been good authenticators"
or the like is becoming a truism. For ISPs with good source filtering in place
then IP addresses ARE good first level authenticators (e.g. filter lists
on management ports). Note: I say FIRST level authenticators; IP addresses are
obviously not suitable as the whole authentication process.
I don't know why, but I feel the need to clarify some semantics. I am sure everyone involved in this discussion already knows what I am about to say.
I think the word "system" here is being abused and the context is changing.
IPs are reasonable in the authentication process for network-centric items (like routers, things that make up the lowest levels of the OSI stack). Systems here means routers, or the networks they make.
IPs are less reasonable the higher up the OSI stack you go. A web server may authenticate with IPs and find use in them. An application running on that web server is almost always going to find less value in that authentication since it is capable of more specific authentication (password, cookie, post rate limit, etc). This use approaches, but may not reach, the "zero" asymptote when you consider cases of applications running on private networks (VPNs, NAT networks, localhost, etc). System here means anything else, but almost never a router or the underlying network infrastructure.
Yes, Geotrack has given us some more detail (of varying levels of precision/accuracy) of where IPs come from. But pretty much IP level controls (IMO) should stay at the lowest levels of the OSI stack.
Ian looks to me like he was talking about routers & their neighbors. Which is a very NANOG charter way to look at things.
Sean looks like he was talking about everything else (applications and things in user space). All things things NANOGers support that pays for the pretty blinky lights.
I'm done. Hope that was mildly interesting or useful.
Deepak
Your assumption is incorrect. These DNSBLs cover spam sent in email,
indeed. Thing is, spam is spam and spammers are spammers. Meaning, they
spam in every way they can.
How does this make his assumption incorrect? Spam is spam and DNSBLs
will likely be very effective when it comes to stopping comment spam.
There are, of course, some severe problems with using a DNSBL as a
blocklist for comments...
I've been working on a new DNSBL for comment/etc. spam for a while, which
will be reliable, generally, it doesn't exist yet for public consumption.
But there's a major problem here... A DNSBL is a source blocklist.
Since the current trend in spam (comment and smtp) is to use botnets,
then by blocking the bots, you also block the users who would make
meaningful comments.
The argument there is that those users don't deserve to comment if
they can't keep their computers clean, but let's get real.. Some of
this stuff is getting pretty advanced and it's getting tougher for
general users to keep their computers clean.
I think a far better system is something along the lines of a SURBL
with word filtering. I believe that Akismet does something along
these lines.
There is such a black listing service already, but again, reliability is
an issue.
Reliability is always an issue with blacklists as they are run as
independent entities. There is always someone who has a problem with
how an individual blacklist is run...
The argument there is that those users don't deserve to comment if
they can't keep their computers clean, but let's get real.. Some of
this stuff is getting pretty advanced and it's getting tougher for
general users to keep their computers clean.
I'd have said it was getting easier to keep computers clean. Back in the late
1980's I use to have my own DOS boot disk, with bootsector antivirus tools,
so that any PC I used on my University I could be sure was clean. Doesn't
mean there aren't more computers, with less clueful users, these days.
I think a far better system is something along the lines of a SURBL
with word filtering. I believe that Akismet does something along
these lines.
This is the same issue as the email spam issue. Identify by source, or
content. Just as content filters are error prone with email spam, they will
be error prone with other types of content.
I think either approach is viable, as long as the poster has an immediate
method of redress. ("My IP is clean" works, and scales, "this URL is safe"
works but doesn't scale, "this post" is safe is viable). In each case you
need to make sure the redress is protected from abuse, so some sort of
CAPTCHA is inevitable.
> There is such a black listing service already, but again, reliability is
> an issue.
Reliability is always an issue with blacklists as they are run as
independent entities. There is always someone who has a problem with
how an individual blacklist is run...
That is easily solved with one's feet. Not as if there is a shortage of
blacklists for various purposes.