Hello,
I'm trying to troubleshoot a problem with a fractional (311 mbit/second)
gigabit-ethernet line provided to me by a metro access provider.
Specifically, it is riding a gig-e port of a 15454.
The behavior we are seeing is an occasional loss of packets, adding up to
a few percent. When doing a cisco-type ping across the link, we were
seeing a consistent 3 to 4 percent loss.
For fun, the provider brought it up to 622 mbit/second, and loss dropped
considerably, but still hangs at about 1 to 2 percent.
There is no question in my mind the issue is with the line, as we've done
a wide variety of tests to rule out the local equipment (MSFC2s, FYI).
Any clues would be exceptional.
-- Alex Rubenstein, AR97, K2AHR, alex@nac.net, latency, Al Reuben --
-- Net Access Corporation, 800-NET-ME-36, http://www.nac.net --
Hello Alex,
I'd say this sounds obvious, but may be deceptively so...
If you are taking a pipe capable of 1000 mbit, and rate-limiting it to
311 mbit, the logic used may be:
In the last 1000 msec have there been more than 311mbits? If yes: drop.
What you want is to shape the traffic, so the rule would be:
In the last 1000 msec have there been more than 311 mbits? If yes: store
until the msec period is up, then transmit.
If you are pushing 100 mbits over this link, it is entirely likely that
there will be a few sub-second burts up to 1000 mbit, and a few
sub-second drops to 0mbit.
An option for you would be to just figure out what the exact
rate-limiting rules are, and then shape it into those rules on your side
of the link -- assuming they wont change it to a shaping rule.
--Phil
Except, we're at the levels of 100 kbit/second in our tests.
I did just find CSCdr94172, which might be related.
-- Alex Rubenstein, AR97, K2AHR, alex@nac.net, latency, Al Reuben --
-- Net Access Corporation, 800-NET-ME-36, http://www.nac.net --
This may sound a bit ridiculous, but say the timer is every 0.25ms.
100kbit per 0.25ms = 400,000kbit or 400 mbit.
It is remotely possible to hit a 300 mbit limit with only 100kbits of
traffic, if the timer is sufficiently short, and your traffic is
sufficiently bursty.
Unless your traffic is Mcast, I doubt that issue is related.
Can you ask your provider how exactly they are limiting the pipe? When
dealing with 300 or so megs, I doubt they will be shaping with a policy
friendly to you, as the logistics of doing so are a bit difficult.
--Phil
Hello,
I'm trying to troubleshoot a problem with a fractional (311 mbit/second)
gigabit-ethernet line provided to me by a metro access provider.
Specifically, it is riding a gig-e port of a 15454.
The behavior we are seeing is an occasional loss of packets, adding up to
a few percent. When doing a cisco-type ping across the link, we were
seeing a consistent 3 to 4 percent loss.
Over what averaging time ? Could this be an example of the
"65 second problem", where the router stops dead for one or two seconds
out of every 65 seconds ?
Regards
Marshall Eubanks
This may sound a bit ridiculous, but say the timer is every 0.25ms.
100kbit per 0.25ms = 400,000kbit or 400 mbit.
It is remotely possible to hit a 300 mbit limit with only 100kbits of
traffic, if the timer is sufficiently short, and your traffic is
sufficiently bursty.
A cisco ping is not bursty, to the extent of hundreds of mb/s. Also, cisco
ping doesn't offer 4,000 pings/sec.
Unless your traffic is Mcast, I doubt that issue is related.
Read on; EIGRP, CDP, etc.
Can you ask your provider how exactly they are limiting the pipe? When
dealing with 300 or so megs, I doubt they will be shaping with a policy
friendly to you, as the logistics of doing so are a bit difficult.
By strapping 1 or more STS-1's to the GE iface.
-- Alex Rubenstein, AR97, K2AHR, alex@nac.net, latency, Al Reuben --
-- Net Access Corporation, 800-NET-ME-36, http://www.nac.net --
Is there a Pipeline 75 guru out there tonight that might be willing to help
me offline? This is my first experience with the Pipeline and I am feeling
really stupid about now!
Please excuse the minor operation content, but I am stumped and really need
a hand.
I am replacing my fading (but still functional) Flowpoint 128 ISDN router
with a used Pipeline 75 that I got off eBay. It appears to be fine. I have
upgraded the firmware and have spent all day trying to get this @!<>**~!!
thing to dial a call to the ISDN WAN. Both POTS work in & out bound.
Show if stat indicates the all the WAN ports are down except 'wanidle0'
which is up. The system options status window says dynamic bandwith
allocation is not installed and I can't fiqure out how to turn it on or
install it. Is this my problem?
In diagnostics, bridisplay occasionally reports
---> parameter 1,128.1 could not be loaded"
---> parameter 1,128.2 could not be loaded"
but I am unable to locate any cross reference between these parameter
numbers and the associated configuration item.
Your kind assistance is greatly appreciated.
Dennis ...
: A cisco ping is not bursty, to the extent of hundreds of mb/s. Also, cisco
: ping doesn't offer 4,000 pings/sec.
No, but you can start 6 simultaneous sessions to the router and have 5 of
them pinging the other side of the circuit while looking at the 6th
session to watch for traffic levels and errors. You'd be surprised how
much traffic you can push when you set the packets to as big as you can
and as fast as you can.
I don't know how bursty a traffic pattern this would create, though.
scott
I may be missing something but..
presumably their rate-limiting involves some form of queuing/buffering..
in which case assuming the ping is the only thing occuring, when the rate hits
the limit it will queue, delay and slow down the echo/reply
and no packets should be lost?
on the other hand, if as is i think suggested below theres no buffer ie when it
hits the limit it starts dropping then i dont think thats a good way of rate
limiting as it only works for tcp and the network really needs to provide a way
to slow the ip layer down. i cant see how that will provide a usable service...
Steve
I would definitely recommend using something like iperf, instead of ping if you want to precisely measure your link performance. Ideally you would have two machines on each side of the link to isolate the problem.
Is this link in production? We are using a gigabit ethernet to our provider. We are limited on our traffic going to Commodity traffic, but have free reign on our Internet 2 traffic. We found that we get the best results when we shape/police our traffic to stay within our contractual limits, on our side of the link. Since we are using a 6509 with a Sup1A, we had to do some tricky things to police traffic on only one vlan of an 802.1q trunk on the gigE connection. It works though. We see insignificant losses on the link.
good luck!
Peter Hill
Network Engineer
Carnegie Mellon University
Is this link in production? We are using a gigabit ethernet to our
provider. We are limited on our traffic going to Commodity traffic, but
have free reign on our Internet 2 traffic. We found that we get the best
results when we shape/police our traffic to stay within our contractual
limits, on our side of the link. Since we are using a 6509 with a Sup1A, we
had to do some tricky things to police traffic on only one vlan of an
802.1q trunk on the gigE connection. It works though. We see insignificant
losses on the link.
can you share how you are doing this?
Hybrid or integrated?
good luck!
Peter Hill
Network Engineer
Carnegie Mellon University
-- Alex Rubenstein, AR97, K2AHR, alex@nac.net, latency, Al Reuben --
-- Net Access Corporation, 800-NET-ME-36, http://www.nac.net --
It is very messy, but until we get a Supervisor/MSFC/PFC that can police on egress vlan...
We have two bgp sessions with our provider, one that distributes I2 routes, and the other, default. Each points to the other end of their respective /30 subnets on their own vlan on an 802.1q trunk (set up manually, not using VTP.
This box is connected to two separate layer 2 cores via two /29s (via 802.1q trunks). Once /29 is the "Commodity" (ie. we must pay) vlan, and has the default-information-originate. Then, the other /29 has its own ospf process (like i said, messy). We redistribute the bgp I2 routes into that ospf process, using as-path filters. Now at the two cores, when traffic for the commodity traffic comes in, it goes to the border router via one vlan, research/I2 traffic the other. We are then able to filter on the ingress vlan on the border router.
Now, for you, if don't have a situation where some of your external traffic needs rate-limited, while some can flow as fast and free as it wants, then you just need to do in-bound rate limiting, coming from your internal network to your border router.
What sucks for us, is that since we are not experiencing any congestion on our internal network, we can't take advantage of wfq or wred.
That was as simple as i could put it, if you have other questions, please ask.