I think we all understand the value of using one’s own equipment and keeping the firmware up to date if one is in any way concerned about security. We all should also understand that in a managed environment such as an ISP there should be no reasonable expectation of privacy regarding the configuration of the equipment attached to the ISP's network (rented or customer owned).
The bigger concern should be the cleartext portion of the subject. There’s ZERO reason to store or transmit any credentials (login, service, keys, etc.), in any location, in an unencrypted fashion regardless of their perceived value or purpose. Unless you like risk.
-- Ben
Confidentiality Notice: This e-mail communication and any attachments may contain confidential and privileged information for the use of the designated recipients named above. If you are not the intended recipient, you are hereby notified that you have received this communication in error and that any review, disclosure, dissemination, distribution or copying of it or its contents is prohibited. If you have received this communication in error, please notify me immediately by replying to this message and deleting it from your computer. Thank you.
That's looking at it from a technical perspective when it isn't a technical problem. People that buy "includes wifi" from their ISP often need extreme amounts of help with it, and thus the wifi credentials are stored and transmitted in plain text for tech support reasons.
~Seth
Yes, and the availability of all this to anyone who hacks Comcast
customer support.
---rsk
I think we all understand the value of using one’s own equipment and keeping the firmware up to date if one is in any way concerned about security. We all should also understand that in a managed environment such as an ISP there should be no reasonable expectation of privacy regarding the configuration of the equipment attached to the ISP's network (rented or customer owned).
Accepting i'm not a North American...
The reasonable expectation of privacy should be that the customer knows precisely what is private, and what is not. If the ISP makes it very clear that every configuration item on the edge device is known to, or accessible by, the ISP for support purposes, then there's no problem. At which point everyone's "reasonable expectations" are the same, and there's no issue.
(Those for whom the support provided by the ISP is key, will enjoy this service. Those who don't, have the option of doing their own thing. Even better.. provide the user the means to disable the sharing of this information by choice?? Would save buying and running additional hardware for those who don't feel the need to have their creds shared, for example).
First thing i've done with all ISP-provided CPE is disable all the remote-login stuff that's enabled by default for tech support purposes. Full knowledge and disclosure is all that's needed!
The bigger concern should be the cleartext portion of the subject. There’s ZERO reason to store or transmit any credentials (login, service, keys, etc.), in any location, in an unencrypted fashion regardless of their perceived value or purpose. Unless you like risk.
As someone else said, the problem is the level of trust you're placing in your ISP and in their own security... a large aggregate of private information is just waiting to be pwned.
Mark.
Risk is threat times vulnerability times impact. No impact, no risk. For example, if the credentials for my grocery store loyalty card are compromised, I do not actually care. It has no impact. Hence failing to encrypt the card number as it transits the store network or sits in their database carries no risk.
There can be, on the other hand, substantial costs associated with using encryption. Key management infrastructure. Manpower. Business risk: loss of the keys becomes loss of the data. Mistakes yield service outages that impair business operations. Forgot to renew that key? Gotta close the store until the IT guy gets here because the cash registers don’t work. These costs tie to the use of encryption regardless of the risk it mitigates.
I take no position on what risk the comcast wifi passwords issue carries. I’m posting only to point out that an absolutist model which says, “stuff of type X must always be encrypted,” is probably not well tuned to the customer’s actual security needs. The generally accepted principle is that if you spend more money mitigating the risk than the attributable cost of the risk then you’re doing it wrong.
Regards,
Bill Herrin
I'm willing to bet that for a significant percentage of people who do at least
some of their work at home, but aren't router-savvy, the risks of a borked
router preventing them from working from home are a bigger issue than the
relatively low risk of a database compromise leading to a miscreant getting
hold of their wireless password and using their access point as free wifi.
Security decisions that are "obvious" when only security-minded and technically
clued people are involved become a lot less obvious when 95% of the people
involved are named Joe Q. Sixpack.
“than the relatively low risk of a database compromise leading to a miscreant getting ahold of their wireless password and using their access point as free wifi.”
And this is the thing, not only does someone have to ‘hack’ the database, they also need to drive up to your house and sit in your driveway to get free Internet. Of all the things to worry about, this is way down on my list.
They could also remotely compromise any device that’s currently (or about to be) in range of your access point, as a stepping-stone to gaining access to your network - without having to be physically anywhere near your driveway.
Perhaps, for example, to make it look as though what they’re doing is coming from your house.
Royce
A fun fact: my employer has a product which basically does brute force protection for web forms. One of, if not the, biggest customers for that product is a grocery store chain, and exactly with their loyalty card portal.
Sometimes, the impact or the absence thereof is a matter of perception.
People are missing the point here. This is not a Comcast “issue” this same data is available to every single cable operator in the US who deploys bundled modem/router/APs that follow the CableLabs standard. They may or may not expose the data to their end customers, but it’s stored in their systems and is often available to their support teams.
http://mibs.cablelabs.com/MIBs/wireless/CLAB-WIFI-MIB-2017-09-07.txt
The same thing applies to most FTTH and xDSL deployments. They use TR-069 to collect the data instead of SNMP but the object model is the same. The WiFi MIB above is explicitly built to mimic TR-181 functionality.
I have yet to hear any discussion of the business value of access to WiFi network password, especially as compared to billing and identity data. Also, remote management of local networks by definition requires credentials stored off site from the customer. To the typical customer, loss of connectivity is much worse than a small chance of sharing the WiFi.
Narrowing the discussion back to Comcast, credentials for “guest” WiFi seem to be more used than purloined SNMP MIB data.
I have yet to hear any discussion of the business value of access to WiFi network password, especially as compared to billing and identity data.
Grandma Smith calls in because she changed her WPA2 password two years ago. Her grandson just bought her a new iPad and she can’t connect. Tier I support says “I have your ‘WiFi password’ right here. It’s hunter22.” The call take 45 seconds and you’re off to solve the next question. It is all about call center metrics. Is it right? No. But that’s the business driver behind it.
James,
By the DOCSIS standard and every North American MSO’s ToS I’ve seen (I’ve worked with or for about 200 different cable operators over the last 20 years) your cable modem is always managed and the cable operator always has access to its configuration and settings via SNMP. The configuration file for a DOCSIS cable modem is nothing more than a list of SNMP OIDs with their values set the way the operator wants them. This has been true since DOCSIS 1.0 (which doesn’t make it correct, just common). Now, DOCSIS is primarily deployed with SNMP version 2c though more and more operators, especially the larger ones, are moving or have to SNMPv3. I mention this because every cable modem on a given CMTS that’s deployed in SNMPv2c mode will (should for proper functioning) have the same SNMP READ and WRITE strings. SNMPv2c traffic is clear text UDP with no standardized methods of encryption available to the industry. To mitigate this to an extent part of the cable modem’s configuration will (best practice anyway) have filtering rules to only accept SNMP traffic from a given IP address or subnet and traffic between the CMTS and the modem should be encrypted via BPI+ for some minimal security. (A minor note, the router interface for a combo unit by CableLabs OSS standards will not respond to SNMP traffic on its public address by default and almost all of the SNMP traffic will be to the modem’s RFC1918 address.) What I’m getting at is that the for DOCSIS (and FTTH and most DSL flavors as well) the service provider has and has had access to all the settings for a very long time. What’s changed is that customers wanted to WiFi to be provided by the operator and importantly supported by the operator.
" I have yet to hear any discussion of the business value of access to WiFi network password, especially as compared to billing and identity data. Also, remote management of local networks by definition requires credentials stored off site from the customer. To the typical customer, loss of connectivity is much worse than a small chance of sharing the WiFi."
Let me provide something in this regard. The most common support call category, by a large number, is the WiFi category. In excess of 55% of all support calls (regardless of how the customer describes them) end up being WiFi issues. The most common specific call in that category is some variation of, “I’ve forgotten my WiFi password and I need to get my new iPad online.” As a service provider your choices are to say I can reset your WiFi PSK, which allows the new device to come online but breaks everything else that was connected, or to allow the support rep and/or customer to recover the passphrase. In short, the ability to recover the passphrase is strongly preferred by end users over resetting it and frankly is much less expensive for the operator.
I’ve helped supply this functionality to a large number of operators and in general the implementations should at a minimum be able to capture the agent who recovered the passphrase, the time/date, who the agent was on the phone with, whether the agent correctly verified the identity of the caller, and if the agent followed all other procedures laid by the service provider.
Sounds like you live in a single-family home, in a low-density
neighborhood. I live in an apartment complex, and WiFi Analyzer shows
more than 20 beacons visible from my kitchen. Before I implemented MAC
filtering, my DHCP server logs showed connections by rogue equipment.
(And before you ask, my password is not a simple one.)
When I leave town for an extended period, I pull the plug on the wireless.
In an industrial park, I see the same dense tangle of beacons. Let
alone those wireless systems that have disabled beacons.
If that database of wireless passwords also has the physical location of
the device, then the risks go up.
So the simple solution is to use your own AP, block SNMP access from the
worlds at your edge router, and the risk falls to zero.
That's looking at it from a technical perspective when it isn't a technical problem. People that buy "includes wifi" from their ISP often need extreme amounts of help with it, and thus the wifi credentials are stored and transmitted in plain text for tech support reasons.
While I agree that the underlying need is to provide fast and effective customer service - it is ultimately a technical problem. As it's been pointed out in subsequent posts WiFi is the leading cause of customer calls to an ISP offering the service. Security and "ease of use" are often at odds with each other, and implementing the former with the latter is the challenge many of us wake up to each and every day. The information should be encrypted at rest and in transit and could easily be decrypted by the CSP platform for use by customer support staff at the time of need when cusetomers call in - which would address the concern.
In my experience, bad practice is easily replicated. What else is transmitted in cleartext? Today it's the WiFi password, tomorrow it's your login, port forwarding, DMZ, and other details that are far more useful to a remote attacker than your WiFi password.
Just so you know, if you have an embedded router from a service provider all of that data is already being transmitted and has been for a long long time. If it’s being collected via SNMPv2c it is being transmitted in the clear (though hopefully encrypted via BPI+ between the modem and the CMTS). If it’s being collected via TR-069 it may (should be) encrypted in transit but in my experience that isn’t guaranteed and when its being sent over TLS there’s often a self signed cert in the chain.
Responding to a pseudo-random message ...
If you are an average consumer and purchase a managed solution (in this case a WAP that comes as part of your package) I think it's perfectly reasonable for the vendor to manage it accordingly, even if said consumer doesn't fully understand the implications of that decision.
In my mind, the problem here is not that the vendor has access to this data, it's that they are STORING it in the first place, and storing it in the clear to boot. In the hypothetical service call that we've speculated is the driver for this, the extra 15 or 20 seconds that it would take to pull the data via SNMP is in the noise.
There are two mindsets that desperately need changing in the tech world:
1. Do not store data that you don't have a legitimate requirement to store
2. Do not store anything even remotely sensitive in the clear
We live in a world of all breaches, all of the time. So we need to start thinking not in terms of just protecting said data from the outside, but rather in terms of limiting the attack surface to start with, and protecting the data at rest. So that WHEN there is a breach, whether from within or without, the damage will be minimal.
As many have pointed out, this information is freely available via SNMP, so it's a classic example of something that didn't need to be stored in the first place.
Doug
Doug,
I don’t disagree, but things are pretty complicated, much more so than they might seem from the outside. First, if the configuration isn’t stored there’s literally no way to have a backup for most of the CPE vendors so there’s definitely reason to have it duplicated in the service providers’ systems. Very few allow for end users to download their router configuration via the admin page and I know of none that encrypt that configuration before it is delivered to the end user’s computer. (It’s also relevant that the usage for those vendors that do allow end users to backup the config is vanishingly low.) If we’re looking at a TR-069 based system for managing the WiFi and router components it’s not really feasible to do a real time grab of that data since that process can take up to ~5 minutes depending on your periodic inform settings in your ACS. That’s because TR-069 is inherently a push technology (from the CPE to the ACS) rather than a pull like SNMP.
The next piece is that a DOCSIS configuration file itself, which in some cases contains these parameters, is by the standard required to be delivered via insecure protocols, namely TFTP. Newer devices have options to allow for TLS secured HTTP, but that’s very rare today in production provisioning systems, and in case the secured protocols are all still optional in the spec. In general the config file itself is stored in it’s text format on the provisioning systems or if the file is dynamically generated the user specific parameters are held in a database with the general ones coming from a template for that class of service.
Again, I’m not disagreeing with your premise but the service providers directly control a small piece of the overall process and we’re still working with standards from earlier times. Most cable operators have gotten rid of their DOCSIS 2.0 (1.0 and 1.1 as well) but it’s not uncommon to find a handful of users with (mostly customer owned) D2 devices that the provisioning and OSS systems still have to deal with. DOCSIS 3.0 devices are the majority and 3.1 devices are just now being rolled out in large numbers.
In short, not everything is quickly retrievable, much of the configuration is in fact generated by the service provider and must be maintained because the modem needs to download its configuration every time it reboots, and the vendors and associations in the provisioning and OSS space have more input than the operators themselves.
As much as it pains me to Devil’s Advocate for Comcast… Has anyone proven that they are storing this PSK in cleartext? From the original StackExchange post :
" When I went to the account web page, it showed me my password. I changed the password and it instantly showed the new password on the account web page (after refresh). "
The SNMP response is essentially cleartext , sure. But perhaps they are performing the query from a modem management network only accessible from the RF side, the transmission back to the CS backend is encrypted in flight, and the data is also encrypted at rest until retrieved and decrypted by a agent or the end user via the web portal. Nothing has been shown that I can recall reading that proves or disproves any of that.
Tom,
No, and I would hope that they were storing it in an encrypted format and then decrypting it on the fly for display in the customer portal.