Contacts wanted: OVH, DigitalOcean, and Microsoft (Deutschland)

Nikolas Geyer <nik at neko.id.au> wrote:

I have passed your email on to the relevant team within DO to have a look at.

Thank you, but that wasn't what I requested, I asked for a contact there.
(I know that this may be hard to understand, but it's like the difference
between giving a man a fish, and teaching him how to fish. I'm sure that
you would make a fine long term conduit between me and whatever mystery people
you think you have made contact with at DigitalOcean, but really, it would
be best if you would simply introduce me to those people directly. That way,
if you die or go on vacation, incidents like these won't need to be put on
hold until you get back and resume your role as our designated go-between.)

I'd like to thank you for your deriding commentary to bring attention
to this problem.

No problem. My pleasure.

I am not sure it is the most effective way to try and engage the wider
industry on a public list, but each to their own.

I am not sure that there is any other way that a lone outsider can or
could engage either OVH or DigitalOcean in a way that would actually
cause either company to take action on the issues I've reported on.
Complaints from ordinary Internet end-lusers about this, which both
companies must surely be drowing in by now, don't seem to be doing the
job.

In any case, I would be more than happy to have you tell me the "right
way" to engage with any actual live human beings at either of these
companies, especially if you also are able to identify one or more
such receptive individuals by name and email address, which is what I
was requesting in the first place.

Oh, and additionally, as an Australian citizen with many Aussie and
Kiwi colleagues working at DO of various religious persuasions; your
postscript relating this back to the recent terror attacks is abhorrent
and disgusting. You should be completely ashamed.

It's pretty clear to me that you have rather dramatically misread my
the aforementioned postscript to my earlier post, and that a fair and
clear-eyed reading of that should be quite entirely inoffensive to all,
with the possible exception of some few people who work in mass media
and/or the "news" business, such as it currently is.

In that postscript, I merely used a recent mass media controversy relating
specifically and only to the social media -handling- of recent events to
illustrate two blatant absurdities at opposite ends of a spectrum, neither
of which itself has anything at all do do with those recent news events
specifically, much less with the race, creed, color or gender of any of
the people who have, most sadly and regretably, been caught up in those
events.

Please consider again the two polar opposite absurdities that I was
actually attempting to call attention to.

One the one hand, we have TV talking heads, with essentially no technical
knowledge whatsoever, wondering aloud why social media tech companies cannot
do what is clearly technically impossible, and even more absurdly, why they
can't do it in real time no less.

On the other hand, and in contrast to that absurdity, we have the present
example of this spamming operation that appears to be well and truly
ensconsed on the networks of both OVH and DigitalOcean, where even large
multi-billion dollar Internet hosting companies seem utterly unable to
spot even trivial and easily identifiable patterns of bad behavior, in
and among their own respective customer bases, even though, as I have now
illustrated, a single lone unpaid volunteer guy, sitting in his basement
and in his sweaty underwear and a bathrobe -can- easily and quickly spot
the problem, within just a couple of hours in fact, provided that he has
access to a decent quality passive DNS service and an ample supply of
electricity, margaritas and cigarettes.

I'm only kidding, of course. I don't actually have a basement.

Regards,
rfg

P.S. Your apparent misreading of my earlier postscript is entirely
understandable and forgiveable in light of the rather unfortunate quip
that I made just prior to that, about sending this specific miscreant
to Guantanamo if his skin color was sufficiently dark.

I seriously regret and apologize for that inartful phrasing, and ask
every charitable person to believe me when I say that that was said
entirely in jest (albeit a bad one), and that if anything, it was
intended to be an expression of my own personal outrage about my own
country's abundant inequity and unfairness when dealing with people of
color, either within our so-called justice system or elsewhere. Our
justice system should be color blind. Alas, there is much evidence
that it falls far short of this goal at the present time.

I have sent reports to DigitalOcean and Microsoft about the abuse
reported to me that appears on my network. The only response I'm
interested in is the end of said abuse.

When abuse continues to be seen on a DigitalOcean allocation, that
allocation goes into a file-and-forget ACL.

I investigate further only when a customer reports a problem connecting
with a DigitalOcean netblock. No such reports yet, by the way.

I respond to abuse reports promptly and completely. I expect others to
do so, just as promptly and completely. "Pretty please" is not in my
netadmin vocabulary.

(Disclosure: I, too, work for DigitalOcean as the Manager of Network
Engineering. Nikolas does not work for me, nor I for him.)

Nikolas Geyer <nik at neko.id.au> wrote:

I have passed your email on to the relevant team within DO to have a
look at.

Thank you, but that wasn't what I requested, I asked for a contact
there.

Oh, is that how this works? I ask that you FedEx me a million dollars
cash, in small bills. I await the arrival of said parcel.

In any case, I would be more than happy to have you tell me the "right
way" to engage with any actual live human beings at either of these
companies, especially if you also are able to identify one or more such
receptive individuals by name and email address, which is what I was
requesting in the first place.

Would you really be happy with that? You derided another good-faith
respondent to your screed with a rant about not being willing to fill out
web forms to report abuse because it offends your sensibilities.

Nikolas brought your report to the attention of the relevant group at
DigitalOcean, we also have a link on the front page of
https://www.digitalocean.com/ to Report Abuse that goes directly to the
relevant group or groups responsible. We respond to reports (even rude
ones) here on nanog and on other relevant industry mailing lists.

We would prefer, but don't require, that you use the web form because that
is integrated into the workflow of the groups that respond to those
reports. If they choose to give you their individualized contact
information, then they can do that. It is not my place, nor Nikolas', to
give out individual contact information for our co-workers out to anyone
who asks. That would be irresponsible and obnoxious for us to do that.

Oh, and additionally, as an Australian citizen with many Aussie and
Kiwi colleagues working at DO of various religious persuasions; your
postscript relating this back to the recent terror attacks is abhorrent
and disgusting. You should be completely ashamed.

It's pretty clear to me that you have rather dramatically misread my
the aforementioned postscript to my earlier post, and that a fair and
clear-eyed reading of that should be quite entirely inoffensive to all,
with the possible exception of some few people who work in mass media
and/or the "news" business, such as it currently is.

As a caucasian American, born and raised in the US Midwest, I too was
offended by your postscript. I would encourage you to take a step back,
and consider your rhetorical tactics and whether they are beneficial to
the community, or even to your own efforts.

Why isn't abuse@ integrated into the workflow? It darn well should be,
(a) given that RFC 2142 has been "on the books" for 22 years and
(b) given that methods for handling incoming abuse (or bug, or outage,
or other) reports via email to role accounts are numerous and reliable.

To be clear: if you want to offer a web form in addition to an abuse@
address (or a security@ address, or a postmaster@ address) that's fine.
But web forms are a markedly inferior means of communication and are
clearly not a substitute for well-known/standardized role addresses that
route to the appropriate people/processes.

---rsk

+1

Just to clarify, we are RFC 2142 section 4 compliant. I mention section 4 specifically as that is directly within my realm of control, the remaining sections I will check.

Both methods, web form submission and abuse@ are integrated ultimately into the same workflow. Being transparent, as things currently stand, the abuse@ submission method requires an additional element of human verification before ingestion to the workflow as it is open to abuse itself. For example, an annoyed former user who has been removed from the platform for abusive activities trying to subscribe it (and other RFC 2142 addresses) to thousands of pornographic mailing lists, or attempting to slam it with tens of thousands of junk emails.

We do take platform abuse seriously but, like any other company, there is always room for improvement. We have a dedicated team who’s 24/7 job function is to continually improve our systems and processes surrounding abuse, from trying to stem it at top of funnel, to mitigating on-going issues with as low MTTR as possible, to responding to abuse@ (and web form) submissions.

tl;dr - both submission methods are available

Kind of bad netiquette to repost a private email to the list

  -- Niels.

Absolutely unrelated to Ronald’s original post, but it’s ironic that the abuse@ address is itself heavily “abused”, by commercial copyright enforcement companies which think it’s a catch-all address for things which are not operationally related to the health of a network (BGP hijacks, DDoS, spam email traffic, botnet/virus/worm/trojan traffic command and control and such).

Despite the presence of a registered DMCA agent address[1][2] for an ASN, many companies continue to flood abuse@ with copyright notices. Ask any ISP that operates in the English language Internet but is not physically located in the USA (NZ, AU, CA, etc) how many USA-specific legal threats their abuse inbox receives. Usually for something like a residential customer torrenting a TV show.

1: https://www.copyright.gov/dmca-directory/
2: https://www.copyright.gov/rulemaking/onlinesp/NPR/faq.html

Apologies, it was in reply to a list mail. Just bad threading.

* niels=nanog@bakker.net (niels=nanog@bakker.net) [Tue 19 Mar 2019, 16:51 CET]:

[[ I've just collected some new information about the length of time
   that this specific bincoin extortion spamming bad actor has been
   on Digital Ocean's network. For those who may only have an interest
   in that one detail, you can just skip down to the line of plus signs
   and start reading there. ]]

In message <50414.162.155.102.254.1553001814.iglou@webmail.iglou.com>,

(Disclosure: I, too, work for DigitalOcean as the Manager of Network
Engineering. Nikolas does not work for me, nor I for him.)

Nikolas Geyer <nik at neko.id.au> wrote:

I have passed your email on to the relevant team within DO to have a
look at.

Thank you, but that wasn't what I requested, I asked for a contact
there.

Oh, is that how this works? I ask that you FedEx me a million dollars
cash, in small bills. I await the arrival of said parcel.

In my experience, if you don't ask for something, you aren't likely to
get it. There's no harm in asking.

In any case, I offer you the pertinent observation also that "small bills"
are soooooooo last century. These days, as should now be abundantly
clear, payment in bitcoin is the preferable currency for such requests. :slight_smile:

In any case, I would be more than happy to have you tell me the "right
way" to engage with any actual live human beings at either of these
companies, especially if you also are able to identify one or more such
receptive individuals by name and email address, which is what I was
requesting in the first place.

Would you really be happy with that? You derided another good-faith
respondent to your screed with a rant about not being willing to fill out
web forms to report abuse because it offends your sensibilities.

I stand by what I wrote. I don't like dealing with anonymous web forms
that, for all I know, and based on the available evidence, are or may be
aliased to /dev/null. I prefer the human touch, especially in cases
where I am seeking to find someone who may be held accountable when and
if no actual action ensues.

We would prefer, but don't require, that you use the web form because that
is integrated into the workflow of the groups that respond to those
reports. If they choose to give you their individualized contact
information, then they can do that. It is not my place, nor Nikolas', to
give out individual contact information for our co-workers out to anyone
who asks. That would be irresponsible and obnoxious for us to do that.

I am not just "anyone who asks". I am a guy who's been spammed from your
network. If you read my earlier report, then you should know that I am
also the guy who took the time to carefully resarch this, and to provide
your company with information about this specific crook/spammer...
information that, it seems, you folks yourselves have apparently been
largely or entirely unaware of, and for some considerable time now.
Given that context, am I really entirely undeserving of even being
informed of the mere email address of the head of DigitalOcean's abuse
handling department, assuming, at least for the sake of argument, that
such an inddividual does in fact exist? Wouldn't it be a Good Thing
if that person and I could communicate direct?

And more to the point, what would be the downside, exactly, if that
person's name and email address were not only given to me, but also
scattered to the four winds an given out to everyone on the planet?
Are you implicitly asserting that that person might then have to (gasp!)
deal with some additional influx of spam into his or her inbox? If so,
then I can't help but wonder aloud why that person should NOT join the
rest of us mere mortals in that shared and miserable club. Perhaps it
would even be of some benefit for that person to come down out of the
clouds at least long enough to experience what the rest of us poor
sods have to deal with on a routine and daily basis. The experience
might even enhance that person's understanding of, and appreciation of
the very kinds of (spamming) problem that he or she is being paid to
attend to. Stranger things have happened.

I'll be generous here and will refrain from leaping to any conclusions
that the person in question does not want his or her identity to be
generally known for fear that he/she might then be personally criticised
for his/her work and/or the lack thereof. But other than that, and a
possible desire to avoid receiving any of this same spam-slime that the
rest of us poor slobs get coated in on a daily basis, I really can't
imagine what other reasons there might be that would cause Digital
Ocean's abuse handling staff and/or the managment thereof to be so
overwhelmingly discreet.

What I can say, rather definitively now, is that the specific bitcoin
scammer-spammer that prompted me to begin this thread has been given,
over time, and by your company, Digital Ocean, no fewer than five hundred
and fifty three (553) separate, distinct, discrete and individual IPv4
addresses and that many, most or all of those have been used for outbound
spamming purposes, all just by this one bad actor, and all during the
present calendar year. The evidence supporting this assertion was and
is available here:

    https://pastebin.com/raw/WtM0Y5yC

Note that this is the equivalent of more than a full /23 that Digital
Ocean has given to this one customer, presumably after vetting the
customer according to current industry standard due diligence procedures,
which is to say no due diligence whatsoever, other than making sure that
the check clears.

I've seen this movie and have implemented various mitigation approaches
to it -- none of which constitute a "solution" but all of which help.

1. Block the addresses originating this traffic. There's no need for
staff/processes on the receiving end to put up with spam. (If it's UBE,
then it's spam -- by definition. The content and intention are irrelevant.)

2. Use procmail to redirect it where it needs to go.

3. Set up (non-public) Mailman-operated mailing lists for each role
account and use the moderation queue on those as a throttling tool.
(This works best in conjunction with (2). Let procmail do some of
the heavy/straightforward lifting and sort the rest out later.)
This also makes it easy to archive everything by subscribing an
address that's an append-only mailbox.

4. Funnel the output of (2) and/or (3) into one of the many ticketing
systems with priority assigned based on the characteristics of the
senders as observed over time.

---rsk