have a friend who has 21 floors of a building in DFW, multiple switches, etc and they started to have latency issues this weekend where half if not all packet are being dropped to folder shares, printers, etc. Suggestions on how they can troubleshoot that? call in a company to help identify it?
-Dennis
That could be a lot of things. Without a network drawing and access
to the devices to dig further it is difficult to say.
Simplest would be to do a trace route from different sources or loop back
interfaces to the servers/computers in question and see where latency
starts spiking. this will at the very least point you to what device or
devices are possibly over utilized.
I'd start with a map of the network mark the routes (paths) that work.
Then redraw the map without those paths and mark which stations talk to
which other stations.
If that exercise discloses which equipment is broken, fix or replace it
and start over.
If it does not, and no other you-can-do-it-yourself tests or analyses
come to mind, call for expensive help.
(If they are competent, they will use an orderly analysis--that one is
my favorite--I call it sectionalization. I'm not bright enough to deal
with 21 floors. I have to sectionalize it to a particular horizontal or
vertical before I can figure where to start.)
on of the first things I'd do is check interface statistics from the
inter-connecting interfaces for errors. On Cisco switches, the command is
fairly straight forward - show interface counters errors. All of the
numbers should be low if things are operating well...if you see more than
100 errors on any given port, it is probably worth investigating.
Question - are the floors connected by fiber or by copper?
Check CPU levels on each switch, pull traffic logs of trunk ports, check
syslogs for flapping ports or weird errors.
I'd guess someone plugged something underneath their desk they shouldn't
have.
Jason
Hi,
Have I been banned?
Found a MAC address spewing stuff. looks like we have our culprit. thanks EVERYONE!
-Dennis
That was pretty quick.
But what do you mean by spewing stuff? It would help the rest of us understand for possible
future issues we may run into ourselves.....
That was pretty quick.
But what do you mean by spewing stuff? It would help the rest of us
understand for possible future issues we may run into ourselves.....
Good question. Without thinking about it I saw in my mind's eye a
situation we used to see at $EX-EMPLOYER (who was fond of the absolute
smallest-dollar-amount-per-immediate-problem "solutions") who bout toy
4-port hubs by the pallet-load.
These little gems had the endearing habit of spewing random bits onto
the wire whenever the wall-wart failed--which they frequently did.
I had MRTG graphs of every switch and router port so I could quickly
determine which leg the current culprit was on.
Never solved the problem of having two or three go bad, which, believe
it or not, complicates the issue.
But the graphs did allow me to identify the port and shut it down saving
the rest of the network.
yea, I'm working on that. will get back to you once he answers my IM
I too would be interested for my own companies.
-Dennis
Isn't it amazing that one can be so cheap it ends up biting them in the arse?
There's a difference between frugal and cheap. Being cheap comes back to you,
it's like Karma....
Great! looking forward to it......
It would be interesting to know where this message has been for an hour
and a half.
Stuck in the NSA's queues?
-r
Received: from localhost ([::1] helo=s0.nanog.org)
by s0.nanog.org with esmtp (Exim 4.68 (FreeBSD))
(envelope-from <nanog-bounces@nanog.org>)
id 1NsIqy-0007si-VK; Thu, 18 Mar 2010 16:45:49 +0000
Received: from eastrmpop110.cox.net ([68.230.240.52])
by s0.nanog.org with esmtp (Exim 4.68 (FreeBSD))
(envelope-from <larrysheldon@cox.net>) id 1NsIq7-00072X-DV
for nanog@nanog.org; Thu, 18 Mar 2010 16:44:56 +0000
Received: from eastrmimpo01.cox.net ([68.1.16.119])
by eastrmmtao107.cox.net
(InterMail vM.8.00.01.00 201-2244-105-20090324) with ESMTP id
<20100318150713.FCRZ18765.eastrmmtao107.cox.net@eastrmimpo01.cox.net>
for <nanog@nanog.org>; Thu, 18 Mar 2010 11:07:13 -0400
Received: from [192.168.1.202] ([68.229.170.168])
by eastrmimpo01.cox.net with bizsmtp
id uf7E1d00F3eLnoL02f7F7u; Thu, 18 Mar 2010 11:07:15 -0400
That _is_ interesting!
I wonder if there is a way to get to those headers from Thunderbird.
Not much else works and I didn't even think to try.
My bad.
[previous comments and header display]
That _is_ interesting!
I wonder if there is a way to get to those headers from Thunderbird.
Not much else works and I didn't even think to try.My bad.
It does work (takes a bit of poking to find them, but it does work).
My very bad.
I wonder if there is a way to get to those headers from Thunderbird.
Not much else works and I didn't even think to try.
Ctrl + U (or "View" and then "Message source").
As an aside .. we see this all the time with some of the cable providers
(we have both Cox and TWC here in Cleveland) when investigating
wrongly-placed blame for "missing" or "delayed" emails. One server at
TWC (which still has an *.adelphia.net name) in particular seemed to
hold messages for the default retry interval 100% of the time
(misconfigured greylisting?).
Cheers,
Michael Holstein
Cleveland State University