Every election has problems. Most of the time, those problems aren't noticed. Elections rely on a lot of back-end infrastructure, besides the actual voting itself.
It could be a DDOS attack, or simply duct-taped systems having trouble with the load.
Voting early (mail, drop-off, in-person) means more time to fix glitches.
The FBI and the Cybersecurity and Infrastructure Security Agency warned elections officials nationwide last week that cyberattacks could disrupt their systems during the run-up to the election. They particularly noted “distributed denial-of-service” attacks, which inundate a computer system with requests, potentially clogging up servers until the system becomes inaccessible to legitimate users.
Florida has had notoriously unreliable state I.T. infrastructure for years. Florida's unemployment websites were broken for months during the Spring 2020 COVID unemployment demand surge. So its very likely crappy state I.T. infrastructure problems being stressed by high volume.
Florida voting age population (2016): 15,839,713
Florida registered voters (2016): 12,863,773
Florida registered voters (2019): 13,536,830
Florida Secretary of State Laurel Lee, who oversees the voting system, said the online registration system “was accessed by an unprecedented 1.1 million requests per hour” during the last few hours of Monday.
I understand that there is underlying work that can’t be sourced somewhere else, at least not trivially.
How many of these overloaded web sites that we hear about (voter registration, unemployment registration, web sites announced in a big way, causing surges in traffic, etc.) have a CDN offloading the low-hanging fruit?
I know that processing a voter registration is far more intensive than serving up static images, but surely a CDN taking the low hanging fruit would help to some degree. I’m assuming most of the people running these sites are clueless and haven’t looked at this, but maybe they have.
This is the same state that spend $60M-ish to revamp their entire unemployment system 6 years ago, only to have it completely collapse this year when 'rona landed.
There is also lot of duct-tape, inadequate funding and human error across all states (and probably no different for governments world-wide). I see problems with overloaded state/local emergency alert sites all the time.
Periods of extreme stress is when many systems break. Its not necessarily due to malfeasance or malicious. Expect there will be problems with election infrastructure through Election Day, and afterwards. I fully expect US state/local election administrators to
make it work this time too (eventually, maybe a few days or weeks).
Be on the look out for clever nefarious actors.
But the dumb mistake is the more likely explanation for most problems.
And in either case, Murphy's Law applies. Have a backup plan because things will go wrong.
Those projects are always given to the lowest bidder, not the best (see ACA).
( Or the “friendliest” depending if the requirements are spelling out the name of the highest donator of which ever party is in power =D )
This is not the first, and won't be the last, problem that happens during this election season. Unless there is substantial and credible evidence otherwise, the normal reason for an technology outage will likely be the normal stupid reasons for technology outages.
People act like 1.1 million requests per hour is a huge number.
That's only 305 requests per second!
Cheapest NVMe SSDs are capable of 160k+ IOPS.
You can literally serve the whole thing from a single server on a
100Mbps line, if you design it properly, and don't waste bandwidth on
stock images and silly front-ends.
Add a T1 to do replication on the side to an off-site location.
100 Mbit/s / 305 req/s = 40 KiloBytes/req -- should be enough to
display/process any form; and you can even get higher speeds on a 5G
mobile phone these days;
1.5 Mbit/s / 305 req/s = 0.6 KByte/req -- should be enough to
replicate each registration; and why are we even talking about T1 in
2020?!
Keep in mind that 1Gbps (e.g., 1000Mbit/s) is pretty much a minimum
these days, so, you'd either have plenty of extra room to spare, or
can do way more than an average of 1.1 million requests per hour. A
Google search reveals you can even get 10 Gbps transit for only
$900/mo from he.net these days, for example.
P.S. At least here you may have to collect and distribute unique
information to each visitor; but what excuse did PG&E had in 2019 when
they couldn't distribute non-unique information about the preemptive
power shutoffs that they've had about one year ago now?!
What I'm always curious about, is how many servers do they actually
have, and just how unreasonable do their numbers look when you lay it
all out. You'd think paying a few mils to design the system could
actually make it work properly when the time comes. Or are they
somehow not aware that they have 16M voters, everyone always doing
everything in the last minute?!
It isn't the stock images and silly front-ends that take all the effort. Those
are pretty damned easy to serve up quickly.
It's the twisty little maze of databases, all different.
You asked for a driver's license number for ID? Well, that just bought you
a call to the DMV's servers to check on the validity/status of that ID.
Vetting the home address gets equally interesting, especially if it's
a PO box or a "suite" at a mailbox-for-rent company.
Vetting the existence of the last employer is going to take time as well.
Are you going to get the unemployment system, the tax system, the DMV
systems, and any others you need to talk to on this "one server"? Oh, and
don't forget that the systems in the DMV and tax systems almost certainly
have *other* systems they have to talk to....
Don't forget that these state agencies usually don't have the budget
that Amazon or other large commercial organizations have, so you're looking
at a *really* high chance that some server in the Department of Revenue
isn't sized big/fast enough, so verifying the employer's existence hangs, so
the front end hangs....
On top of all that, even if you're only a *little* bit too slow clearing requests,
you end up sitting on a big pile of pending requests, which sucks up memory..
Get 305 requests per second, clear 304 per second, and in a few minutes
you're throwing '502 Gateway Error' left right and center because things are
wedged up....