I originally asked on mailops, but here is a much wider net and I suspect there’s a lot of overlap in interest.
I had read an article one time, somewhere about the ongoing consolidation of e-mail into a handful of providers was bad for the Internet as a whole. It was some time ago and thus, the details have escaped me, so I was looking to refresh my recollection.
Have any of you read a similar article before? If so, can you link me to it?
I don’t know. Do I miss the days of every person and their dog running a mail server on a Linux server in a basement cupboard?
Huge crowds and high drama on nanae and spam-l type places
You never know whether your mail is going to get through or not because of weird and wonderful notions about spam filtering
No shortage of open relays and hacked Matt Wright formmail.pl
Whoever heard of backup?
(etc)
–srs
I’m sure Dave Crocker has thoughts about this, but it has come up elsewhere. There are both positives and negatives about having such a consolidation. The positive is that it a small club can establish ground rules for how they will handle various forms of attacks, including BGP hijacking, DKIM, SPF, and other forms of validation to identify fraudulent mail, etc. Also, if you have a whole lot of postfixes and sendmails running around, that’s a whole lot of code to patch when things go wrong. A small number of MSPs can devote a lot of time and paid eyes on code. They can also very quickly spot new attack trends.
On the other hand, that means that it becomes difficult to become a new entrant, because one doesn’t easily get one’s mail accepted. Lots of grey/blacklisting (forgive the use of the term). Also, when one of those systems fails, it takes down a vast number of customers. Furthermore, it represents a massive concentration of private information that can be monetized.
Eliot
In many ways I see this similarly to the consolidation of browsers, but less consolidated. I think about the advantages and disadvantages of the prominence of Chrome (65%), Safari (20%), Firefox/Samsung/Edge/Opera/etc (15%). With Chrome we’ve seen Google move the browser and related standards forward through sheer marketshare. CSS/HTML/JS standards live and die by Chrome support and that’s both good and bad. They have made great and opinionated strides when it comes to SSL/TLS. For example, Google effectively killed Symantec’s certificate business because it was mismanaged. They also effectively got rid of EV certs and pushed secure-by-default web server design where HTTPS appeared normal, but warnings all over the place for non-encrypted connections. On the other hand, Google is fairly disliked in the privacy community and those communities prefer independent Firefox.
For email, I can see similar issues, mostly around security. If Microsoft were to decide security mechanism X is not worth the effort they can effectively decide to not implement it. What will internet users do, block all Microsoft email services? Conversely they could come up with their own security mechanisms and effectively force the rest of the world to adopt it. I do think centralization of email providers provides little potential for negative impact aside from operational issues. For example, outages probably have a wider impact due to number of users, but I can’t realistically see a scenario where Microsoft/Google does something “bad” with their email platform that affects the rest of the ecosystem.
Caesar Kabalan
Matt Harris
Infrastructure Lead Engineer
816‑256‑5446
Direct
Looking for something?
We build and deliver end‑to‑end IT solutions.
+1
Likewise, we're at a point now where if a criminal phish or virus comes from the largest few email hosters, and you provide them emails with full headers - the accounts do NOT get shut down. They literally don't think this is their problem. And likewise, data storage sites (GoogleDrive, OneDrive, etc) from the largest providers often will host malware for weeks or months without being shut down - or the malware at least persists for many days after being reported. The same is often true for their redirectors.
Wwhat is frustrating is that the long-standing industry standard of "you're responsible both for what you both send and host - even if the malware wasn't intended" - seems to be lost.
Likewise, back in the spring months of 2018, google's "goo[.]gl" shortner went crazy for a few months, and was being MASSIVELY abused by spammers, and was being used as an "end run around" URI DNSBLs (SURBL, URIBL, ivmURI, DBL). I collected 15K examples of abused shortners that were "live", and sent those to Google. At the time I sent those, only about 500 of that 15K had been shut down. What was infuriating was that 80% of these 15K shortners were pointing to only 12 spammer's domains. These should have been trivial to prevent!
The OTHER infuriating thing was that my INITIAL response from my contacts at Google was - (I paraphrase) "other spam filters should just follow the redirect, and block these spams based on the URI it redirects to" - WOW! I sent them a very stern email about that. (and for comparison, abused Bitly shortners were mostly getting shut down within 2 hours - so "everyone does it" was NOT a decent excuse!)
Like I said - the long-standing industry standard of "you're response both for what you both send and host - even if the malware wasn't intended" - seems to be lost on some of these large providers.
Thankfully, this had a happy ending. After some "tough love" - Google replied back and said (I paraphrase), "we were planning on shutting that down - or at least shutting down the ability to add new ones - and due to your feedback - we're going to push that up a few months" - and so soon afterwards, they finally did terminate those 15K shortners - and stopped allowing new ones. So this is to Google's credit - but the problem had persisted for months - and it seemed like a lot of cultural/industry standards in the Internet Security industry seemed lost on them.
Sadly, while this situation had a good ending - similar problems with the largest providers persist. At the same time, they sure can be draconian in how they block smaller providers who had a rare and short-lived security incident. The hypocrisy is incredible. For example, Microsoft will sometimes *permanently* block a small email hoster for a short one or two hour compromised email account situation that caused spam to be sent from that small hosters - but that was quickly fixed - even if that hoster sends MUCH legit email. It almost FEELS like extortion - since many of the IT people running those small-ish servers sometimes get frustrating - and move their email to the cloud - and then guess who OFTEN gets their email hosting business?
-- Rob McEwen, invaluement
The positive is that it a small club can establish ground rules for how they will handle various forms of attacks, including BGP hijacking, DKIM, SPF, and other forms of validation to identify fraudulent mail, etc. […] They can also very quickly spot new attack trends.
In theory, but the current state of what’s coming out of sendgrid implies otherwise.
It’s not theory but history. They have spotted those sorts of trends quickly in the past (see below). They may not tell you they have spotted the trends.
Once you get into that small club, it’s just as hard to get kicked out, and unfortunately that means that if abuse, UCE, etc is coming from those hosts, they’ve got an even higher chance of hitting your inbox.
This depends on the nature of the incident, but if their evil bit gets set and if their size is Size XL, then it is indeed hard to give them the boot.
So while in theory it might work the way you’re thinking, in practice it hasn’t because once you are in that club, a lot of the financial motivation to prevent abuse of your service - that is, inbox deliverability for your client base - goes away.
I disagree, but we aren’t going to debate incentive models here. Suffice it to say that the big guys spending money on this, as they do, belies your point. A good example was one such very large provider tracking hijacked BGP announcements and then releasing that information to shut down a huge swathe of sources all at once.
However…
That deliverability isn’t likely to change for the negative on any scale that you care about once you’re “in”. But to be “in” you have to be at a huge scale. The small players are the ones who get hurt, and spam still gets through just fine only now via different means.
Yes. That was why I said that there is good and bad. Were we to take this to extremes, we see why FB can curate their messages and keep spam to a bear minimum, as they really do control the horizontal and the vertical (two sided market).
Also oligopolies in general are bad for everyone except the owners thereof and should be discouraged on principle.
Not that I disagree (this comes to you by way of my dinky little VM), but that’s not the topic at hand.
Eliot
This is being portrayed a little too "either/or", that if you get spam
etc from $BIGEMAIL you, service provider, block them.
What goes on is multi-layer spam blocking using various tools rather
than host/server blocking except as a last resort.
So we'll block/toss/etc a lot of the malmail from $BIGEMAIL w/o
generally blocking their servers.
If we get a huge attack we have thresholds at which point we might
block them for two hours (whatever) hoping it stops on its own or
$BIGMAIL stops it.
But those are pretty high thresholds and obviously can cause problems
for our customers in delayed email but so can our mail servers being
pounded on. Those $BIGMAIL delivery servers have a lot more computrons
than we do.
Aside: What's astounding to me is how little any of this has changed,
other than consolidation perhaps -- remember when AOL's servers
pounding you with spam could bring you to your knees? I do -- in over
20 years.
I find this question interesting (obviously because I'm responding to the list) and have done for decades.
Providing a reasonable email solution has become more and more complex while public perception is that email should be, and is, free.
I see lots of sides to this debate, some have already been covered by many of you already.
* Stuff has to be secure
* When stuff becomes insecure it starts to cause headaches for others.
* Keeping stuff secure gets harder and harder
* Customers want more and more features
* Customers should pay for some features/service
* Some IT folk are standing up systems to help others reduce costs - again causing headaches for others
* Some IT folk have set up expensive systems, funded by data mining and not customers.
* Some IT folk simply object to data mining - some folk act on that objection.
* There's a lot of 'activism' in the email space and has been for a very long time.
* Some of the 'big providers' take some of the heat out of the activism, which only winds up some IT folk even more.
* Knowledge and skills with people who can, and will, set up small systems is thinning as demand is growing.
* Some want to grow and drive others to rise up their skills.
* Some of those "drivers", I think [1], 'attack' learners, not unlike throwing the Apollo crew in a rocket simulator, hoping they will rise up their skills.
* With limited revenue, and constant 'driver training', some eventually abandon the game.
* Some view that driving training is important if you want to have skin in the game, but quickly forget their time is funded and they're not funding idealism.
* Some see their lunch being taken by a rise of good 'free' software. Some react by [1] driving more updates, features and improvements 'help', which just overwhelms small operators.
* Some had no choice but to stand up small systems but 'now free offerings' have empowered them to abandon the space.
* Some have no thought around the issues, others simply don't care - some days there are just bigger fish.
Personally, I identify with some of these issues, and perhaps there's more, but it's the 'fish' question that right now connects with me the most...
In a country of 5 million people, this graphic says we have ~18,000 people waiting for social housing. The idealist in me has turned it's attention, and while I still operate my own mail systems (mainly because I like to able to back it up and add capacity more quickly and I have trust issues with big providers changing the rules mid-stream), I to am leaning closer and closer to calling time...
...anyway, thanks for your eye balls, I'm off to put some paint on a building ready to launch a community housing trust to address that graphic.
[1] - Tin Foil Hat time.....
D
All true…
You’ve also left out:
Economic incentives to make questionable use of mail content and user data.
Economic incentives to make life difficult for new entrants.
Economic incentives to avoid transparency or convenience in addressing user concerns about erroneously rejected email.
Reduction in consumer choice (if there are a handful of providers and they all provide essentially the same (crappy) level of service, then what can the consumer do about it?
#include <std_list_of_oligopoly_prolems.h>
Owen