Per Site QOS policy with Cisco IOS-XE

I have a question for the QOS gurus out there.

                We are having some problems with packet loss for our
smaller MPLS locations. This packet loss is due to the large speed
differential on our Hub site(150mb/s) in comparison the the branch office
locations(single T-1 to 4.5mb/s multilinks). This packet loss only seems
to impact really bursty applications like our Web Proxy. I have been
around and around with WindStream to give me some extra buffer or enable
random early detection on the smaller interfaces in my MPLS network. So
far they are unwilling to do a custom policy and none of their standard
policies have enough buffer to handle the bursts. They do FIFO tail drop
in every queue, so I can’t even choose a policy that has WRED implemented.

                I am looking for a way to solve the problem on my side. I
can create a shaper for the proxy and match off an access-list to the
smaller sites, but I am either forced to do bandwidth reservations for each
site, or have multiple sites share the same shaper. Here is an example of
what I was playing around with:

ip access-list extended ProxyT1Sites

permit tcp any host 10.x.x.x 10.x.x.x 0.0.0.255

permit tcp any host 10.x.x.x 10.x.x.x 0.0.0.255

class-map match-any ProxyShaperT1

match access-group name ProxyT1Sites

policy-map WindStream

class VOICE

  priority percent 25

  set dscp ef

class AF41

  bandwidth percent 40

  set dscp af41

  queue-limit 1024 packets

class ProxyShaperT1

  shape average 1536

  bandwidth percent 1

  set dscp af21

  queue-limit 1024 packets

class class-default

  fair-queue

  set dscp af21

  queue-limit 1024 packets

Another Idea I had was to create a bunch of shaper classes all feeding the
same child policy for priority queuing and bandwidth reservations based on
DSCP markings. I’m just not exactly sure that this is allowed or
supported. I also would run out of bandwidth allocation on the policy if I
use the true bandwidth number of 150mb/s. It is on a Gig port so I could
just take the bandwidth statement off of the interface to give myself
enough room for all of the shaper allocations.

Something like this(I am omitting the access-list that matches the branch
subnet and class map for brevity):

policy-map PerSiteShaper

class FtSmith

  shape average 1536

  bandwidth 1536
service policy Scheduler
class Dallas

  shape average 4500

  bandwidth 4500
  service policy Scheduler
class NYC

  shape average 100000

  bandwidth 100000
  service policy Scheduler
class-default
  service-policy Scheduler

policy-map Scheduler
class VOICE

  priority percent 25

  set dscp ef

class AF41

  bandwidth percent 40

  set dscp af41

  queue-limit 1024 packets

class class-default

  fair-queue

  set dscp af21

  queue-limit 1024 packets

Just looking for some ideas that do not involve building tunnels to our
remote offices. Thanks in advance,

*Wes Tribble*

I have a question for the QOS gurus out there.

cisco-nsp might be a better place to post your question. But in any
case, this option looks right:

Another Idea I had was to create a bunch of shaper classes all feeding the
same child policy for priority queuing and bandwidth reservations based on
DSCP markings. I’m just not exactly sure that this is allowed or
supported.

see http://puck.nether.net/pipermail/cisco-nsp/2007-October/044508.html
  So they just shaped at the hub towards the spoke to prevent overrunning the
  PE-CE link at the spoke. Another advantage was they didnt' waste
  hub-PE bandwidth for traffic that would be dropped at the spoke PE-CE link
  anyway.

which has nothing to do with IOS-XE but does sound like what you're
wanting to do.

Regards,
Lee

If you want to prevent a PE router from deciding which ingress packets to
drop, the only plan is to send packets to spoke sites at or below the spoke
line-rate. The only good way to do that is shaping on the hub router.

policy-map parent_shaper
class class-default
  shape average 100000000 < --- 100Mbps parent shaper.
    service-policy site_shaper

policy-map site_shaper
class t1_site
  shape average 1536000
   service-policy qos_global
class multilink_site
  shape average 3072000
    service-policy qos_global
class class-default
    service-policy qos_global

policy-map qos_global
... whatever you typically use here....

Tyler Haske

Tyler,

I would love to implement a policy similar to that one. Unfortunately, I
don't believe you can have two tiers of shaping like that in a policy.
Most of the two-tiered shaping solutions I have seen involve using a VRF to
shape to the aggregate rate and then use a second VRF to shape to the site
rate. This is to get around the three-tier policy limitations.

With that said, if you have something like that configured and working, I
would love to see the config and the "show policy-map interface" output.
That is exactly the kind of policy I was originally looking to implement,
but then I ran into those limitations.

Thanks for the reply. Great idea in concept. If only we could implement.

Wes,

If the router is running HQF code for QoS [really anything later then
12.4(20)T], it should support this kind of hierarchy. It's a common policy
I have customers implement all the time.

http://www.cisco.com/en/US/docs/ios/qos/configuration/guide/qos_frhqf_support.html

Thanks for the information Tyler, I will have to play around with that kind
of policy in my lab. What would you suggest if you are oversubscribing the
interface? With the child policy inheriting the bandwith of the parent
shaper, wouldn't I run out of bandwidth allocation before I built all the
shapers for all of my 29 sites?

Wes,

The earlier policy doesn't use bandwidth commands, hence, it doesn't
*subscribe* anything. The only thing it does is ensures that individual
sites do not exceed their shaped rate. You could add bandwidth statements
if you wanted to ensure a certain site always is guaranteed a certain
amount of bandwidth from the parent shaper. You can't oversubscribe with
the bandwidth command.

policy-map parent_shaper
class class-default
  shape average 100000000
   service-policy site_shaper

policy-map site_shaper
class t1_site
  shape average 1536000
  bandwidth percent 1
   service-policy qos_global
class multilink_site
  shape average 3072000
  bandwidth percent 2
   service-policy qos_global
class class-default
  bandwidth percent 97
   service-policy qos_global

policy-map qos_global
  ! ... whatever you want here.

This would make sure that large sites don't stare out small spoke sites for
bandwidth.

Tyler,

Tyler,

I already had a case open with TAC on this issue. This is what the CCIE
assigned to the case is saying about that type of policy:

Hi Wesley,

Yes, I’m afraid that configuration is not possible. We can only mark or
police traffic on this child policy.

You will see the following message when trying to attach the service-policy
to the interface:

We had a similar problem years ago with a frame-relay <---> IMA setup. The
hub end was a multiplexed ATM circuit with PVC's to each site's frame-relay
circuit. The IMA speed was equal to the aggregate speed of each site's
CIR. It worked great until all the sites were bursting above CIR. VoIP
call quality would go really bad when that happened (early mornings
mainly.) QoS policies could not be maintained between the frame and ATM
sides.

Sprint (support contract on the CPE) and MCI (circuits) engineers finally
decided to run PPP over the native protocols on each end and then apply QoS
to the PPP sessions. That fixed the problem. I am not sure if something
like that is possible in your case or not though as I am not familiar with
MPLS. I tried to find the config, but I no longer have it. I remember it
used virtual templates, but don't remember the specifics.

Jason

Tyler,
Thank you very much. I took off the bandwidth reservations on the child
shapers and I was able to apply to an 1841 series router in my lab. Either
my TAC engineer is off base or there is some limitatin with the ASR that
does not exist for vanilla IOS.

QUOTE:
The earlier policy doesn't use bandwidth commands, hence, it doesn't
*subscribe* anything. The only thing it does is ensures that individual
sites do not exceed their shaped rate. You could add bandwidth statements
if you wanted to ensure a certain site always is guaranteed a certain
amount of bandwidth from the parent shaper. You can't oversubscribe with
the bandwidth command.

Here is a short snippet of the show policy-map int, i cut it off after two
sites for brevity.

Service-policy output: BigShaper
    Class-map: class-default (match-any)
      31694 packets, 4932119 bytes
      30 second offered rate 129000 bps, drop rate 0 bps
      Match: any
      Queueing
      queue limit 64 packets
      (queue depth/total drops/no-buffer drops) 0/0/0
      (pkts output/bytes output) 31723/4962574
      shape (average) cir 50000000, bc 1250000, be 1250000
      target shape rate 50000000
      Service-policy : PerSiteShaper
        Class-map: LittleRock (match-any)
          0 packets, 0 bytes
          30 second offered rate 0 bps, drop rate 0 bps
          Match: access-group name LittleRockSubnets
            0 packets, 0 bytes
            30 second rate 0 bps
          Queueing
          queue limit 64 packets
          (queue depth/total drops/no-buffer drops) 0/0/0
          (pkts output/bytes output) 0/0
          shape (average) cir 4608000, bc 115200, be 115200
          target shape rate 4608000
          Service-policy : Scheduler
            queue stats for all priority classes:

              queue limit 64 packets
              (queue depth/total drops/no-buffer drops) 0/0/0
              (pkts output/bytes output) 0/0
            Class-map: VOICE (match-any)
              0 packets, 0 bytes
              30 second offered rate 0 bps, drop rate 0 bps
              Match: ip dscp ef (46)
                0 packets, 0 bytes
                30 second rate 0 bps
              Match: ip dscp cs3 (24)
                0 packets, 0 bytes
                30 second rate 0 bps
              Match: ip dscp af31 (26)
                0 packets, 0 bytes
                30 second rate 0 bps
              Priority: 50% (28 kbps), burst bytes 1500, b/w exceed drops: 0

          QoS Set
            dscp ef
              Packets marked 0
            Class-map: AF41 (match-any)
              0 packets, 0 bytes
              30 second offered rate 0 bps, drop rate 0 bps
              Match: ip dscp af41 (34)
                0 packets, 0 bytes
                30 second rate 0 bps
              Match: access-group name eCustodyClass
                0 packets, 0 bytes
                30 second rate 0 bps
              Match: access-group name BloombergClass
                0 packets, 0 bytes
                30 second rate 0 bps
              Match: access-group name LiquidPointClass
                0 packets, 0 bytes
                30 second rate 0 bps
              Queueing
              queue limit 64 packets
              (queue depth/total drops/no-buffer drops) 0/0/0
              (pkts output/bytes output) 0/0
              bandwidth 25% (14 kbps)
          QoS Set
            dscp af41
              Packets marked 0

            Class-map: class-default (match-any)
              0 packets, 0 bytes
              30 second offered rate 0 bps, drop rate 0 bps
              Match: any
              Queueing
              queue limit 64 packets
              (queue depth/total drops/no-buffer drops/flowdrops) 0/0/0/0
              (pkts output/bytes output) 0/0
              Fair-queue: per-flow queue limit 16
          QoS Set
            dscp af21
              Packets marked 0
                Exp-weight-constant: 9 (1/512)
                Mean queue depth: 0 packets
                dscp Transmitted ECN Random drop Tail/Flow
drop Minimum Maximum Mark
                          pkts/bytes marked pkts/bytes
pkts/bytes thresh thresh prob

        Class-map: Chicago (match-any)
          0 packets, 0 bytes
          30 second offered rate 0 bps, drop rate 0 bps
          Match: access-group name ChicagoSubnets
            0 packets, 0 bytes
            30 second rate 0 bps
          Queueing
          queue limit 64 packets
          (queue depth/total drops/no-buffer drops) 0/0/0
          (pkts output/bytes output) 0/0
          shape (average) cir 10000000, bc 250000, be 250000
          target shape rate 10000000

          Service-policy : Scheduler

            queue stats for all priority classes:

              queue limit 64 packets
              (queue depth/total drops/no-buffer drops) 0/0/0
              (pkts output/bytes output) 0/0

            Class-map: VOICE (match-any)
              0 packets, 0 bytes
              30 second offered rate 0 bps, drop rate 0 bps
              Match: ip dscp ef (46)
                0 packets, 0 bytes
                30 second rate 0 bps
              Match: ip dscp cs3 (24)
                0 packets, 0 bytes
                30 second rate 0 bps
              Match: ip dscp af31 (26)
                0 packets, 0 bytes
                30 second rate 0 bps
              Priority: 50% (28 kbps), burst bytes 1500, b/w exceed drops: 0

          QoS Set
            dscp ef
              Packets marked 0

            Class-map: AF41 (match-any)
              0 packets, 0 bytes
              30 second offered rate 0 bps, drop rate 0 bps
              Match: ip dscp af41 (34)
                0 packets, 0 bytes
                30 second rate 0 bps
              Match: access-group name eCustodyClass
                0 packets, 0 bytes
                30 second rate 0 bps
              Match: access-group name BloombergClass
                0 packets, 0 bytes
                30 second rate 0 bps
              Match: access-group name LiquidPointClass
                0 packets, 0 bytes
                30 second rate 0 bps
              Queueing
              queue limit 64 packets
              (queue depth/total drops/no-buffer drops) 0/0/0
              (pkts output/bytes output) 0/0
              bandwidth 25% (14 kbps)
          QoS Set
            dscp af41
              Packets marked 0

            Class-map: class-default (match-any)
              0 packets, 0 bytes
              30 second offered rate 0 bps, drop rate 0 bps
              Match: any
              Queueing
              queue limit 64 packets
              (queue depth/total drops/no-buffer drops/flowdrops) 0/0/0/0
              (pkts output/bytes output) 0/0
              Fair-queue: per-flow queue limit 16
          QoS Set
            dscp af21
              Packets marked 0
                Exp-weight-constant: 9 (1/512)
                Mean queue depth: 0 packets
                dscp Transmitted ECN Random drop Tail/Flow
drop Minimum Maximum Mark
                          pkts/bytes marked pkts/bytes
pkts/bytes thresh thresh prob

you shouldn't be surprised by this. the asr1k platform implements qos
using dedicated silicon, which means that it's less flexible but much
faster. The 1800 series routers handle everything using a single route
processor and shared dram, which means that they are more flexible but much
slower.

Nick