Network scan tool/appliance horror stories

We're evaluating several tools at the moment, and one vendor wants to dynamically scan our network to pick up hosts - SNMP, port-scans, WMI, the works. I was curious if anyone had any particularly gruesome horror stories of scanning tools run amok.

If you have any overloaded/under-powered network gear, such as stateful firewalls and routers that do lots of NAT, you might find them very quickly, depending on how aggressive the scanning tool is. There might also be devices out there that, while possibly lightly loaded, can reach some minimally documented resource threshold under a very aggressive scan, and subsequently tip over.

Also, if you're doing IPv6, the performance metrics for many network devices can be a bit more of a moving target.

jms

It all depends on what tools they are using and how you have your system
setup.

Both NMAP and Nessus can check system\service to see if common accounts
have default or non password at all.
This can cause these accounts to be locked out.

There are other "exploits" that can cause systems\services to be DOS'd but
these normally have to be enabled.

Best to get a statement of works from them which should list all the tools
including options they will be using.

They also should be able to hand over a raw dump of ALL commands run during
the testing.

I'd almost be tempted to set up a few machines doing v6 only on the LAN, with some trivial to exploit telnet/SNMP access then invite them to scan the LAN and see if said machines are picked up.

My experience of these things a year or two ago was that most of these companies thought everyone had an internal flat IPv4 network in RFC1918 space and that was that. YMMV of course.

Paul.

I heard a story in the past year of someone that had a system get scanned and it opened a ticket with their IT department for each time they scanned them. Eventually the IT department system crashed due to the excessive number of tickets being opened by their scanning tool.

The network was properly exempted from the future scans after the system had to be recovered from backup.

- Jared

LOAD "*",8,1

^^^^^^^^^^^^^
yay

http://www.tulsaworld.com/news/article.aspx?subjectid=334&articleid=20121002_11_A1_CUTLIN325691

A > layer 7 failure. Make sure all members of your organization are aware
of your plans.

During scans at various times in the past (and depending on throttling and settings of that scan) we've seen:
1) small remote site firewalls doing site to site vpns drop a small number of packets
2) locally installed remote control service popup a 'user has been disconnected' error on PCs when port scanned
3) some devices send alerts like 'Unauthorized attempt to gain access' when their SNMP ports are hit with non-standard community strings
4) logging on some devices that causes concern for the admin of that device ("Is someone hacking my device?")
5) out of date/non-patched (yet critical) applications and/or web servers crashing/locking up (this occurred on specific nessus scans, not a generic port/snmp scan)
6) large stacks of 3750s (six or more members) have issues around CPU during certain SNMP commands (I want to say some sort of getbulk type of command)

The first four were pretty minor although #3 could generate a lot of calls to the support center. #5 was a big deal due to the nature of the application. #6 was impactful because we dropped routing neighbors for about 10 seconds but this was a couple of years ago so may have been an old IOS bug.

A particular model of ShoreTel voice switches I used to administer (running
VxWorks, IIRC) would reliably lock up hard when hit with nmap's OS/service
detection on a particular port. Required pulling the plug to restore
service.

The truly odd thing was that it didn't seem like a resource exhaustion
issue, it could be triggered with a single well-crafted probe or two.

After several long nights of painful troubleshooting with their level III
support, we came to the conclusion that if it hurts, you probably shouldn't
do it, and mitigating ACLs were put in place.

-n

On Oct 29, 2012, at 3:55 PM, "Rutis, Cameron"

6) large stacks of 3750s (six or more members) have issues around CPU during certain SNMP commands (I want to say some sort of getbulk type of command)

The first four were pretty minor although #3 could generate a lot of calls to the support center. #5 was a big deal due to the nature of the application. #6 was impactful because we dropped routing neighbors for about 10 seconds but this was a couple of years ago so may have been an old IOS bug.

Saw the same. All of our 3750 stacks (which are small) committed suicide during a trial of Foglight. We had discovery timings turned way down, but it still caused a reload on a mix of the last supposedly really stable releases of 12.x.

Not confidence inspiring. TAC was useless and suggested a v15 upgrade despite no known fix. The proposed v15 upgrade sent our lab boxes into continuous reload unless you broke the stack and manually wiped each switch. Oh, and port 28 was invisible on each switch after upgrade, and Gi2/0/28 would throw a syntax error. Wait for new releases, lather, rinse, repeat.

Total time to resolution in production was several man-weeks on our side, and a few months calendar time, all because the discovery scan revealed how great a "software company" Cisco has become.

Check your netmask on the to-be-discovered network and what the rate
of discovery is. I have seen internal systems attempt to scan and
discover nodes in a /16 and promptly set off a flood of alarms on all
PDUs (6 per rack) and plenty of other devices that thought they are
being attacked.

-andreas

I can share with you several stories personnel (both IT or vendors), who have scanned Electric Utility environments with or without permission; and hence caused multiple failures - including electro-mechanical systems and related applications. Utilities typically utilize many industrial controllers - some of which many IT personnel have no knowledge, and some are not robust enough to weather the storm.

1. Know your environment.
2. Know your tools.
3. Communicate.

We have had ncircle scans unexpectedly crash alcatel-lucent omni-switches.

Network scan tools are a great way to verify what important protocols you
left out of your control plane policing non-default policies. Had a scanner
totally clog up our 6500 core router DHCP relay (ip helper) function once.
Uggghhh, security people....

Chuck

Speaking of scan tools, does anyone have recommendations for tools to do baseline configurations on Windows systems? Looking for pre-change configuration baseline and post change configuration baseline - to identify differences implemented by the change?

Thanks.

Second that. First agree on what rate they are allowed to scan your network, then let them come back with what they find before they point other tools at the found nodes. Then inform the owners of said nodes of what is going to happen.

In a previous life I found an publicly available SQL server on a network belonging to a medical institution that I was pen testing. I pointed Nessus at it and it just died...

BR
/Joakim