Does anyone have information on a widespread Rogers outage in Canada. I have customers with multiple sites down.
Thanks
Rob
There's discussion on the Outages mailing list. Seems widespread, affecting all services, mobile, voice, Internet. No cause or ETR posted yet.
In the early hours of the morning around 2-3am my modem got hit with a configuration update that caused a DHCP release that wasn’t renewed for about two hours, after rollback the connection was fine for 3 hours before this network wide outage.
Maybe a failed night time update was attempted again during office hours, I’ve heard daytime guys are still WFH and night shift is in building.
I expect we’ll never get a real explanation. Rogers is notorious for withholding any type of helpful or technical information.
Regards,
Andrew Paolucci
According to Cloudflare Radar, Rogers BGP announcements spiked massively to levels 536,777% higher than normal (343,601 vs 64 normally) just minutes before the outage. I would not be surprised if this happened to be the culprit.
Regards,
Josh Green
Whatever they did, it has also taken out SS7/PSTN 911 services for many millions of people.
https://www.cbc.ca/news/business/rogers-outage-cell-mobile-wifi-1.6514373
i cant see BGP taking out SS7.
-jim
I can’t either, but the reality right now seems to be that 911 calls are failing for anyone on a Rogers cellphone.
I have seen anecdotal reports that the mobile network is in a half broken state that phones remain registered to, so a 911 call will attempt and then fail.
This is unlike what would happen if you had a US/Canada cellphone with battery power but no SIM card in it that would search for any available network in RF range for a 911 call if needed.
Seems like their external BGP peers are dead. If their internal network is the same, and they carry M2PA etc, or use BGP/MPLS to carry SS7 traffic, or their softswitches/STPs rely on BGP/MPLS in some way then SS7 might exist, but not be functional.
Why not? If the SS7 is encapsulated in IP… (think SIGTRAN).
I believe the call centers run by law enforcement and municipalities in the GTA are mostly homed on single providers, I know some Bell and Telus users in certain areas are still able to reach emergency services.
Regards,
Andrew Paolucci
The E911 network in Ontario and Quebec is run by Bell Canada.
The PSAPs may have non-emergency lines that are provided by other carriers.
I can't either, but the reality right now seems to be that 911 calls are
failing for anyone on a Rogers cellphone.
This is par for the course. These people chose to deal with Rogers despite knowing the consequences. It is like if you bought a Rogers Snowblower and it did not work. That would mean that people who bought the Rogers Snowblower will not be using it to get rid of the snow that is preventing them from leaving their house.
Mutatis mutandis when Rogers is down things that are Rogers dependent will not work.
Some people are so retarded it is astonishing!
I see the point you’re trying to make but using the word retarded in this context is not only dumb in itself but offense. Please be more respectful on this list.
Thanks
Shall we take another spin on the euphemism treadmill?
Normally I’d take offense, but in my experience of endless technician calls, hundreds of hours on the phone with support.
I can assure Rogers staff are the utmost incompetent out of any I’ve seen. Only talked to one support tech with them that even knew what DOCSIS was. How can teaching employees about the technology they sell not be part of training?
Only reason they are even able to operate right now is reselling Xfinity products from Comcast after wasting billions trying to roll their own solutions.
Regards,
Andrew Paolucci
Just leaving this (yes, satire site) link here
–srs
Can we have a discussion with the list admins about a list member appending a threat of violence to their outbound emails? Whether serious or not.
Does this person need directions to some local mental healthcare resources?
Lest we ALL MOVE ON….
Yes he said
“RETARD” =
The delay in processing the current status of a Situation…
Lets move on to bigger n scarier Post mortems:
Was This or Was this NOT
A CYBERATTACK ???
Done.
Lets deal with REAL threats not PC references.
Time to evaluate n re eval Our NTS!!
Veni vidi vici 2022
We should move on and focus on constructive comments vs. kicking an operator when they are down.
That said, demeaning comments, no matter what they are or whom they are targeted at, is not healthy and is not helpful.
regards,
Victor K
fyi - see BOLD.