Reports have it starting at 4:30 a.m… SOS on all phones…
We’re affected as well. Unable to dial out. I haven’t found any official statement though.
Appears to have affected at least some FirstNet SIMS as well.
The same as well for FirstNet but am now able to make calls. Others who are on AT&T are unable to receive or send calls. Enabling wifi calling on a regular AT&T phone (android) results in a 502 bad gateway error message.
Ryan
Word around the campfire is that it’s a Cisco issue.
1 Like
I’ve got about 500 Cradlepoints with AT&T throughout the US. I saw about 20 of them go down, all different locations, all around the same time (around 5am CST).
FWIW these are across 3 different AT&T OU’s… AT&T IoT (with a FirstNet APN), AT&T 4G Business Broadband, and straight AT&T Business/Retail 4G.
The response from my account manager is slightly comical:
“Some of our customers are experiencing wireless service interruptions this morning. We are working urgently to restore service to them. We encourage the use of Wi-Fi calling until service is restored.”
Let me get right on activating Wifi Calling on my Cradlepoints…
Chris Trudeau
Network Engineer, Nitel
No issues on FirstNet here in Houston metro, but AT&T consumer core still appears to be non-functional.
Do you have the ability to expand on this at all? Do you mean a hardware failure of some kind IE router, optitcs, etc?
As widespread as it seemed to be, it feels like it would be quite a trick if it were a single piece of hardware. Firmware load that ended badly, I wonder?
From what I’ve read, they lost their database of SIM cards. I could be wrong of course.
I have several friends in the Nashville area who did not have AT&T Home broadband service this morning, could be unrelated but would point to a different issue than the SIMs.
If it’s AT&T wireless Home broadband Service then it would be affected just the same.
It’s still a cellular modem. It still has a SIM card. It’s no different than a hotspot really, or just about any other cellular mobile device on the network.
If it’s wireline service though, that had nothing to do with this and would have been some separate issue.
Could it be someone fat fingered in preparation of this?
"Starting in March, FirstNet will be the first and only wireless network to provide America’s first responders with always-on priority and preemption across 5G, expanding to include all AT&T 5G commercial spectrum. "
aside from the official pablum that was released about an “incorrect process used”
(which says exactly nothing) does anyone actually know anything accurate and
more specific about the root cause?
(and why it took 11 hours to recover?)
I read it as “someone pushed an ACL that wasn’t properly reviewed and it really screwed things up."
missing an permit ip any any? classic
The $5 you got back should help alleviate any concerns going forward
Robert DeVita
Founder & CEO
Mejeticks
c. 469-441-8864
e. radevita@mejeticks.com
Where did you see this? Erik Prince was on the PBD podcast saying he has a 70% chance in his head it was China. I tend to learn towards human error from my experience in the IT biz.
- J
No $5 received here… Just a text message saying, "It’s AT&T. We apologize for Thursday’s outage, which may have impacted you. As a valued customer, your connection matters and we are committed to doing better.”
I had the thought the other day that maybe this was a hack and that they didn’t want to admit such. I’m sure FirstNet being taken over by hackers isn’t exactly the message they want out there. Regardless of what the actual cause was, their lack of communicating anything meaningful is saying something which boosts the chances that this was a hack in my mind.
Agreed on the human error thing, but maybe the human error was something the hackers did that they didn’t mean to.
Erik Prince was on the PBD podcast saying he has a 70% chance in his head it was China.
Why do we care what his (almost certainly uninformed) opinion is?