Abuse response [Was: RE: Yahoo Mail Update]

Mailman List Mirror (Read Only)
1

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

If you send reports with lots of legal boilerplate, or reports with
long lectures on why you expect an INSTANT TAKEDOWN, and send them to
a busy abuse queue, there is no way - and zero reason - for the ISP
people to prioritize your complaint above all the other complaints
coming in.

In fact, we have done just that -- develop a standard boilerplate
very similar to what PIRT uses in its notification(s) to the
stakeholders in phishing incidents.

Again, our success rate is somewhere in the 50% neighborhood.

And that is after a few months of fine-tuning -- and 15 years of
experience in these matters. :slight_smile:

Nothing to write home about...

- - ferg

2

In fact, we have done just that -- develop a standard boilerplate
very similar to what PIRT uses in its notification(s) to the
stakeholders in phishing incidents.

The boilerplate is no damned use. PIRT - and you - should be focusing
on feedback loops, and that would practically guarantee instant
takedown, especially when the notification is sent by trusted parties.

Again, our success rate is somewhere in the 50% neighborhood.

With the larger providers it will get to 100% once you go the feedback
loop route.

Do ARF, do IODEF etc. You will find it much easier for abuse desks
that care to process your reports. You will also find it easier to
feed these into nationwide incident response / alert systems like
Australia's AISI (google it up, you will like the concept I think)

srs

3

The boilerplate is no damned use. PIRT - and you - should be
focusing on feedback loops, and that would practically
guarantee instant takedown, especially when the notification
is sent by trusted parties.

> Again, our success rate is somewhere in the 50% neighborhood.

With the larger providers it will get to 100% once you go the
feedback loop route.

Do ARF, do IODEF etc.

Yep.

http://xml.coverpages.org/iodef.html

--Michael Dillon

P.S. some more URLs that should be known to all

http://asrg.sp.am/

http://puck.nether.net/mailman/listinfo/nsp-security
http://www.maawg.org/about/publishedDocuments

4

I largely concur with the points that Paul's making, and would
like to augment them with these:

- Automation is far less important than clue. Attempting to compensate
for lack of a sufficient number of sufficiently-intelligent, experienced,
diligent staff with automation is a known-losing strategy, as anyone who
has ever dealt with an IVR system knows.

- Trustability is unrelated to size. There are one-person operations
out there that are obviously far more trustable than huge ones.

- Don't built what you can't control. Abuse handling needs to be
factored into service offerings and growth decisions, not blown off
and thereby forcibly delegated to the entire rest of the Internet.

- Poorly-desigged and poorly-run operations markedly increase the
workload for their own abuse desks.

- A nominally competent abuse desk handles reports quickly and efficiently.
A good abuse desk DOES NOT NEED all those reports because it already knows.
(For example, large email providers should have large numbers of spamtraps
scattered all over the 'net and should be using simple methods to correlate
what arrives at them to provide themselves with an early "heads up". This
won't catch everything, of course, but it doesn't have to.)

---Rsk

5

- Automation is far less important than clue. Attempting to
compensate for lack of a sufficient number of sufficiently-
intelligent, experienced, diligent staff with automation is
a known-losing strategy, as anyone who has ever dealt with
an IVR system knows.

Given that most of us use routers instead of pigeons to transport
our packets, I would suggest that railing against automation is
a lost cause here.

- Poorly-desigged and poorly-run operations markedly increase
the workload for their own abuse desks.

This sounds like a blanket condemnation of the majority of ISPs
in today's Internet.

- A nominally competent abuse desk handles reports quickly
and efficiently.
A good abuse desk DOES NOT NEED all those reports because it
already knows.
(For example, large email providers should have large numbers
of spamtraps scattered all over the 'net and should be using
simple methods to correlate what arrives at them to provide
themselves with an early "heads up". This won't catch
everything, of course, but it doesn't have to.)

Why is it that spamtraps are not mentioned at all in MAAWG's best
practices documents except the one for senders, i.e. mailing list
operators?

Note that if an ISP does have a network of spamtraps, then they have
an automated reporting system, which you denounced in your first point.

I agree that simply automating things will not make anything better, but
intelligent automation is good for you and me and the ISP who implements
it. An intelligent automation system could identify a spam source and
immediately block the port 25 traffic until it can be investigated by
a human being.

--Michael Dillon

6

Rich,

That is one place that modern antispam efforts fall apart. It's the
same problem that afflicts tech support in general. The problem exists
for the same reason that large-city McDonalds workers don't speak
English: Anyone with sufficient clue to run an abuse desk is well
qualified for more interesting, important and higher-paid work where
they don't get yelled at all the time. Like administering mail servers
or writing mail software.

There's a reason we pay garbage collectors a small fortune to do a job
that requires no skill whatsoever.

Regards,
Bill Herrin

7

- Automation is far less important than clue. Attempting to compensate
for lack of a sufficient number of sufficiently-intelligent, experienced,
diligent staff with automation is a known-losing strategy, as anyone who
has ever dealt with an IVR system knows.

Rich,

That is one place that modern antispam efforts fall apart. It's the
same problem that afflicts tech support in general. The problem exists
for the same reason that large-city McDonalds workers don't speak
English: Anyone with sufficient clue to run an abuse desk is well
qualified for more interesting, important and higher-paid work where
they don't get yelled at all the time. Like administering mail servers
or writing mail software.

There's a reason we pay garbage collectors a small fortune to do a job
that requires no skill whatsoever.

Do you _know_ any garbage collectors ? I do, and I would disagree with both clauses of that sentence.

Regards
Marshall

8

> - Automation is far less important than clue. Attempting to
> compensate for lack of a sufficient number of sufficiently-
> intelligent, experienced, diligent staff with automation is
> a known-losing strategy, as anyone who has ever dealt with
> an IVR system knows.

Given that most of us use routers instead of pigeons to transport
our packets, I would suggest that railing against automation is
a lost cause here.

I'm not suggesting that automation is bad. I'm suggesting that trying
to use it as a substitute for certain things, like "clue", is bad.
When used *in conjunction with clue*, it's marvelous.

This sounds like a blanket condemnation of the majority of ISPs
in today's Internet.

Yes, it is. I regard it as everyone's primary responsibility to ensure
that their operation isn't a (systemic, persistent) operational hazard
to the entire rest of the Internet. That's really not a lot to ask...
and there was a time when it wasn't necessary to ask, because everyone
just did it. Where has that sense of professional responsibility gone?

Why is it that spamtraps are not mentioned at all in MAAWG's best
practices documents except the one for senders, i.e. mailing list
operators?

I can't answer that, as I didn't write them. But everyone (who's
been paying attention) has known for many years that spamtraps are
useful for catching at least *some* of the problem, with the useful
feature that the worse the problem is, the higher the probability this
particular detection method will work. Another example I'll give of
a loose-but-useful detection method is that any site which does mass
hosting should be screening all new customer domains for patterns like
"pay.*pal.*\." and "\.cit.*bank.*\." and flagging for human attention any
that match. Again, this won't catch everything, but it will at least give
a fighting chance of catching *something*, thus hopefully pre-empting some
abuse before it happens and thus minimizing cleanup labor/cost/impact.
In addition, this sort of thing actively discourages abusers: sufficiently
diligent use of many tactics like this causes them to stay away in droves,
which in turn reduces abuse desk workload. But (to go back to the first
point) none of it works without smart, skilled, empowered, people, and
while automation is an assist, it's no substitute.

---Rsk

9

Marshall,

No, but I know a few people who have (briefly) worked abuse desks and
neither the tech support nor the McDonalds problem are difficult to
observe.

Without conceding the garbage collection issue, let me ask you
directly: how do you propose to motivate qualified folks to keep
working the abuse desk?

Regards,
Bill Herrin

10

That is one place that modern antispam efforts fall apart. It's the
same problem that afflicts tech support in general. The problem exists
for the same reason that large-city McDonalds workers don't speak
English: Anyone with sufficient clue to run an abuse desk is well
qualified for more interesting, important and higher-paid work where
they don't get yelled at all the time. Like administering mail servers
or writing mail software.

There's a reason we pay garbage collectors a small fortune to do a job
that requires no skill whatsoever.

Do you _know_ any garbage collectors ? I do, and I would disagree with both
clauses of that sentence.

Marshall,

No, but I know a few people who have (briefly) worked abuse desks and
neither the tech support nor the McDonalds problem are difficult to
observe.

Without conceding the garbage collection issue, let me ask you
directly: how do you propose to motivate qualified folks to keep
working the abuse desk?

That is a good question. (I feel sure that many actually doing the job would opt for a rise in pay.)
Maybe certain jobs should become apprentice-like positions
that you need to get through to rise in a networking organization. I know that Craig Newmark (of Craig's List)
spends a couple of hours per day going through abuse complaints and user issues personally. I
haven't heard too many complaints about Craig's List, and it seems reasonable to suspect a connection there.
That has the advantage of being cheap to implement, in dollars if not in political capital.

Regards
Marshall

11

Marshall,

There's a novel idea. Require incoming senior staff at an email
company to work a month at the abuse desk before they can assume the
duties for which they were hired.

My hunch says that's a non-starter. It also doesn't keep qualified
folks at the abuse desk; it shuffles them through.

Any other ideas?

Regards,
Bill Herrin

12

William Herrin wrote:

Without conceding the garbage collection issue, let me ask you
directly: how do you propose to motivate qualified folks to keep
working the abuse desk?

Ask AOL?

-Jack

13

Require all technical staff and their management to work at the abuse
desk on a rotating basis. This should provide them with ample motivation
to develop effective methods for controlling abuse generation, thus
reducing the requirement for abuse mitigation, thus reducing the time
they have to spend doing it.

---Rsk

14

Unfortunately many of the skills required to be a competent abuse desk
worker are quite specific to an abuse desk, and are not typically possessed
by random technical staff.

So, to bring this closer to nanog territory, it's a bit like saying that all the
sales and customer support staff should be given enable access to your routers
and encouraged to run them on a rotating basis, so that they understand
the complexities of BGP and will better understand the impact their decisions
will have on your peering.

Cheers,
   Steve

15

Steve,

You don't, per chance, mean to suggest that random back-office
technical staff might not have the temper and disposition to remain
polite and helpful with the gentleman from the state capital so upset
about the interdiction of his political mailings that he's ready to
sic the regulators on you and wipe you off the map?

The problem is that the individual who -does- have those skills along
with the technical know-how to deal with the complaint itself usually
ALSO has the skills to be the customer contact for a multi-million
dollar contract. If you're a manager at a company that wants to, well,
make money, which chair will you ask that individual to sit in?

Regards,
Bill

16

Not really.

IMO, with decent automation[1] and a reasonably close working
relationship between the abuse desk, the NOC and an internal
sysadmin/developer or two, there's not that much need for a high level
of technical know-how in the abuse desk staff.

Good people skills are certainly important, and it'd be good to
have at least one abuse desk staffer with a modicum of technical
knowledge to handle basic technical questions, and help channel
more complex ones to to NOC or developers efficiently, but the level of
technical know-how needed to be an extremely effective abuse
desk staffer is pretty low. The specific technical details they do
need to know they can pick up from their peers (both within
the abuse desk, in other groups of their company and, perhaps
most importantly, from their peer at other companies abuse desks).

It's closer to a customer support position, in skillset needed, than
anything deeply technical, though an innate ability to remain calm
under pressure is far more important in abuse than support. If you're
big enough that you need more than one person staffing your abuse
desk you can mix-n-match skills across the team too, of course.

Cheers,
   Steve

[1] Yeah, I develop abuse desk automation software, so I'm
both reasonably exposed to practices at a range of ISPs and
fairly biased in favor of good automation. :slight_smile:

17

So, to bring this closer to nanog territory, it's a bit like
saying that all the sales and customer support staff should
be given enable access to your routers and encouraged to run
them on a rotating basis, so that they understand the
complexities of BGP and will better understand the impact
their decisions will have on your peering.

We encourage managers, designers, engineers, project managers, etc. to
spend a day handling customer support calls so that they understand the
impacts of their decisions/work on the customer, who ultimately pays our
paychecks. We run even more people through workshops where they spend
some time listening to recorded customer support calls, and then plan
how to prevent such problems in future so that the customers don't feel
the need to call us. Of course, none of these people are expected to go
in and reconfigure BGP sessions on routers, because there are working on
first-line support. One of the duties of first-line support is to sift
through the incoming and identify which cases need to be escalated to
second or third-line support.

Unless you have very good automated systems in place to ensure that the
abuse desk only gets real cases to deal with, then you should be able to
rotate managers and other employees through the abuse department to do
some of that first-line sifting. If the outcome of this is that you make
a business case for changes to abuse-desk systems and processes, then
you should involve the abuse desk staff in this development work to give
them some variety. Once those staff have automated themselves out of a
job, you can move them to some other tools development project, or
incident response work.

--Michael Dillon

18

At a long-previous employer we once toyed with the idea of having everybody in the (fairly small) operations and architecture/development groups spend at least a day on the helpdesk every month.

The downside to such a plan from the customer's perspective is that I'm pretty sure most of us would have been really bad helpdesk people. There's a lot of skill in dealing with end-users that is rarely reflected in the org chart or pay scale.

Joe

19

Of course - you're asking people who are *hired* because they're good at
talking to inanimate objects made of melted sand, and asking them to
relate to animate objects (namely, customers).

Sounds like a recipe for disaster.

:slight_smile:

20

Abuse desk is a $0 revenue operation. Is it not obvious what the issue is?

Some of the folks that are complaining about abuse response generate
revenue addressing these issues. Give me some of that. I'll give you
a priority line to the NOC.

Disclaimer; No offense intended to security providers, I'm just stating a fact.

Best,

Marty