QoS for ADSL customers

Mailman List Mirror (Read Only)
1

Hello everyone,

We have Juniper ERX as BRAS for ADSL, its GigE interface is on an old Cisco 3508 switch with an old IOS, its gateway to the internet is a 7609, our transit internet links terminate on GigaE, Flexwan on the 7600

The links are now almost always fully utilized, we want to do some QoS to cap our ADSL downstream, to give room for the Corp. customers traffic to flow without pain.

I’m here to collect ideas, comments, advises and experiences for such situations.

Our humble approach was to collect some p2p ports and police traffic to these ports, but the traffic wasnt much, one other thing is rate-limiting per ADSL customers IPs, but that wasnt supported by management, so we thought of matching ADSL www traffic and doing exceed action is transmit, and police other IP traffic.

Doing so on the ERX wasnt a nice experience, so we’re trying to do it on the cisco.

Thanks

2

While some people will cry network neutrality and think the Yellow Pages
must sell only one size listing, some people are willing to pay for
differentiated service. Trying to classify "bad" traffic can be
done using products like Sandvine. But it may be easier to classify "premium"
traffic and mark it for special handling, and then treating everything
that isn't marked as premium traffic as best effort traffic.

But expect great wailing and gnashing of teeth over setting or changing
DSCP/TOS bits or creating different queues for different traffic. Should
DSCP bits in IP headers be treated like TTL bits which are modified by
the network. Should ISPs use anti-spoofing techniques similar to prevent
the use of arbitrary IP addresses to control DSCP/TOS values in packet
headers?

Most routers already give priority to some types of traffic, such as
routing update packets.

3

While some people will cry network neutrality and
think the Yellow Pages
must sell only one size listing, some people are
willing to pay for
differentiated service. Trying to classify "bad"
traffic can be
done using products like Sandvine. But it may be
easier to classify "premium"
traffic and mark it for special handling, and then
treating everything
that isn't marked as premium traffic as best effort
traffic.

That may be a simple method to differentiate service
between customers. considering e2e qos parameter
requirement by different network applications,
multiple service levels are required to supported in
ISP network ( both intra-ISPnetwork and
inter-ISPnetwork).

But expect great wailing and gnashing of teeth over
setting or changing
DSCP/TOS bits or creating different queues for
different traffic. Should
DSCP bits in IP headers be treated like TTL bits
which are modified by
the network. Should ISPs use anti-spoofing
techniques similar to prevent
the use of arbitrary IP addresses to control
DSCP/TOS values in packet
headers?

To Kim's situation, IP packet header based (or access
interface based) traffic classification is pratical.
If application based traffic classification is
required, tools from sandvine or packeteer may have to
be sitted between ERX1440 and Cisco7609. IMHO, ISP
network should NOT trust any TOS/DSCP set by their
customers; so, classifying and (re)tagging must be
done in PE or BRAS. On the other hand, anti-spoofing
configuration must be enabled in ERX1440 or 7609.
Anyway, I don't trust current router's ability on
content based traffic delivery.

Most routers already give priority to some types of
traffic, such as
routing update packets.

Only with routing protocol packets, it's far from what
we need for service differentiation.

Would Kim share his experience with this work?

Joe

4

The problem with waiting until the PE or BRAS to do the classification is
most access providers use traffic aggregation in the access network (e.g.
ATM/DSL, Cable, WiFi, etc). This means the interfaces on the BRAS or PE
are oversubscribed and the access network interface will experience
inbound cell/frame drops.

If you don't trust the router's ability, imagine a dslam's ability to do
it at the ATM layer.

Some networks let users tag their traffic, other networks re-tag all
traffic according the network's policies. At the moment it seems to be a
business decision. But the result is users shouldn't expect unmangled
TOS/DSCP bits over the Internet. Coordinating the IP layer QOS with the
access network/physical layer QOS is a bit of a challenge.

5

Can any one please suggest to me any commercial or none solution to cap the download stream traffic, our upstream will not recieve marked traffic from us, so what can be done ?

6

Hello.

Going back to your original question, how to keep from
saturating the network with residential users using
bittorrent/edonkey et al, while suffocating business
customers. Here goes.

Netfilter/IpTables (and a slew of commercial products I'm
sure) has a Layer 7 traffic classifier, meaning it can
identify specific file transfer applications and set a
DiffServ bit. This means it can tell between a real http
request and a edonkey transfer, even if they are both using
http. It also has rate-limiting capability. So... If you
pass all of the traffic destined for your DSL customers
through an iptables box (single point of failure) then you
can classify and rate-limit the downstream rate on a
per-application basis.

Fwiw, if you are using diffserv bits, you could push the
rate-limits down to the router with a qos policy in it
instead of doing it all in the iptables box.

References on this.. The netfilter website (for
classification info) and the Linux advanced router tools
(LART) (qos info/rate limiting)

-e

From: owner-nanog@merit.edu [mailto:owner-nanog@merit.edu]

On

Behalf Of Kim Onnel
Sent: Thursday, December 01, 2005 3:26 AM
To: NANGO
Subject: Re: QoS for ADSL customers

Can any one please suggest to me any commercial or none
solution to cap the download stream traffic, our upstream
will not recieve marked traffic from us, so what can be

done ?

  Hello everyone,
  
  We have Juniper ERX as BRAS for ADSL, its GigE
interface is on an old Cisco 3508 switch with an old IOS,

its

gateway to the internet is a 7609, our transit internet

links

terminate on GigaE, Flexwan on the 7600
  
  The links are now almost always fully utilized, we

want

to do some QoS to cap our ADSL downstream, to give room

for

the Corp. customers traffic to flow without pain.
  
  I'm here to collect ideas, comments, advises and
experiences for such situations.
  
  Our humble approach was to collect some p2p ports

and

police traffic to these ports, but the traffic wasnt much,

one other thing is rate-limiting per ADSL customers IPs,

but

that wasnt supported by management, so we thought of

matching

ADSL www traffic and doing exceed action is transmit, and
police other IP traffic.
  
  Doing so on the ERX wasnt a nice experience, so

we're

7

Our ADSL customers traffic is 3 OC3 worth of traffic, I dont think our management would buy the idea.

thanks

8

I got an off-list reply about using Nbar, but I've never
seen a class map that would match torrent.

-e

From: owner-nanog@merit.edu [mailto:owner-nanog@merit.edu]

On

Behalf Of Kim Onnel
Sent: Thursday, December 01, 2005 7:12 AM
To: Ejay Hire
Cc: NANGO
Subject: Re: QoS for ADSL customers

Our ADSL customers traffic is 3 OC3 worth of traffic, I

dont

think our management would buy the idea.

thanks

  Hello.
  
  Going back to your original question, how to keep

from

  saturating the network with residential users using
  bittorrent/edonkey et al, while suffocating business
  customers. Here goes.
  
  Netfilter/IpTables (and a slew of commercial

products I'm

  sure) has a Layer 7 traffic classifier, meaning it

can

  identify specific file transfer applications and set

a

  DiffServ bit. This means it can tell between a real

http

  request and a edonkey transfer, even if they are

both using

  http. It also has rate-limiting capability. So...

If you

  pass all of the traffic destined for your DSL

customers

  through an iptables box (single point of failure)

then you

  can classify and rate-limit the downstream rate on a

  per-application basis.
  
  Fwiw, if you are using diffserv bits, you could push

the

  rate-limits down to the router with a qos policy in

it

  instead of doing it all in the iptables box.
  
  References on this.. The netfilter website (for
  classification info) and the Linux advanced router

tools

  (LART) (qos info/rate limiting)
  
  -e
  
  > From: owner-nanog@merit.edu

[mailto:owner-nanog@merit.edu]

  On
  > Behalf Of Kim Onnel
  > Sent: Thursday, December 01, 2005 3:26 AM
  > To: NANGO
  > Subject: Re: QoS for ADSL customers
  >
  > Can any one please suggest to me any commercial or

none

  > solution to cap the download stream traffic, our

upstream

  > will not recieve marked traffic from us, so what

can be

  done ?
  >
  >
  >
  > Hello everyone,
  >
  > We have Juniper ERX as BRAS for ADSL, its

GigE

  > interface is on an old Cisco 3508 switch with an

old IOS,

  its
  > gateway to the internet is a 7609, our transit

internet

  links
  > terminate on GigaE, Flexwan on the 7600
  >
  > The links are now almost always fully

utilized, we

  want
  > to do some QoS to cap our ADSL downstream, to give

room

  for
  > the Corp. customers traffic to flow without pain.
  >
  > I'm here to collect ideas, comments, advises

and

  > experiences for such situations.
  >
  > Our humble approach was to collect some p2p

ports

  and
  > police traffic to these ports, but the traffic

wasnt much,

  
  > one other thing is rate-limiting per ADSL

customers IPs,

  but
  > that wasnt supported by management, so we thought

of

  matching
  > ADSL www traffic and doing exceed action is

transmit, and

  > police other IP traffic.
  >
  > Doing so on the ERX wasnt a nice experience,

so

9

There are a bunch of p2p and torrent custom classifier pdlm's at
http://www.cisco.com/cgi-bin/tablebuild.pl/pdlm

Quoting Ejay Hire <ejay.hire@isdn.net>:

10

There was a 3.0 PDLM release on 11/1/05 for Bittorrent traffic. See
http://www.cisco.com/cgi-bin/tablebuild.pl/pdlm

Scott

11

Sean Donelan wrote:

The problem with waiting until the PE or BRAS to do the classification is
most access providers use traffic aggregation in the access network (e.g.
ATM/DSL, Cable, WiFi, etc). This means the interfaces on the BRAS or PE
are oversubscribed and the access network interface will experience
inbound cell/frame drops.

If you don't trust the router's ability, imagine a dslam's ability to do
it at the ATM layer.

Some networks let users tag their traffic, other networks re-tag all
traffic according the network's policies. At the moment it seems to be a
business decision. But the result is users shouldn't expect unmangled
TOS/DSCP bits over the Internet. Coordinating the IP layer QOS with the
access network/physical layer QOS is a bit of a challenge.

I'm not an operator (although I used to be, at a tiny little specialist ISP), but I hope this is on-topic.

Since nearly all of your domestic customers' traffic will be TCP, in particular the bulk file-sharing traffic which I imagine is your greatest problem, although you cannot directly rate-limit their traffic _into_ your layer 2 access network, you can do so indirectly by rate-limiting their traffic within your network, which should cause their TCP traffic to throttle back in response.

This is arguably an easier and more effective way to go than QoS if all you care about is leaving enough slack capacity in your network to keep your business customers happy.

If you want to be ingenious, you could even try the approach of rate-limiting by restricting the flow of ACKs returning from your network, rather than dropping outbound packets. This could be done in a super-dumb way, by just throttling the aggregate flow of ACKs based on source-routing from your domestic IPs, or in a smarter way that was flow and sequence-number aware.

And if you are worried about using Linux / BSD boxes in production work, you could always use a pool of multiple redundant filtering boxes, with load-balancing using some carrier-class kit and automatic failover at layer 3 to hot spare boxes.

-- Neil

12

Any way you do limiting depending on any level over L3, you're going to fail in the long run (people will start to move ports around or go encrypted).

My suggestion is to do marking on comitted bitrate, ie make sure that people get a certain amount of bandwidth per IP, mark excess traffic, and drop this excess traffic when you get congestion.

Either that or give up and give your customer the bandwidth they want and need, which is the preferred and least complicated solution (apart from Layer 8 problems).

If you need to drop traffic in the core or distribution, your access speed business model is flawed.

13

Step 1: Please identify how you identify your Corp. customers.

Once you explain how you identify your Corp. customers then you can
proceed with classifying and prioritizing their traffic.

14

I still don't see the requirement for application level classification.
The original request only mentioned differenting between Corp. customers
and other customers; not the applications they are using.

15

Well, for us, it is based on IP ranges. Corporate customers are assigned
IP addresses that are outside of our residential IP pools.

16

Could IPtables control traffic with inspecting layer7
information?

As someone suggested, bandwidth allocation could be
done with TCP protocol control ( ACK dropping or so);
How can we do that? NBAR only limit the bandwidth, and
to our experience with cisco7609 it cost a lot of cpu
time!

Where can I find QoS experiemnt result and sample
configuration of ERX14xx?

Joe

17

Could IPtables control traffic with inspecting layer7
information?

Not layer 7. IPtables works on L3 & L4 (and another similar system
for linux called ebtables provides filtering at L2) but it can
be used for setting up qos depending on where from (and to) the
traffic is going and what port is being used.

For layer7 filtering on linux you need protocol proxies, and you
can use iptables to redirect all http traffic from subnet to squid
(although its not designed to be a filter, it can be used to
implement L7 filtering for http, but I'm not sure it can be used
for prioritization though).

18

There are quite a few modules for iptables that will reach
up to Layer 7, including several specifically for file
sharing applications...

And one really nifty one that makes non-passive ftp work
through NAT.

-e

From: william(at)elan.net [mailto:william@elan.net]
Sent: Tuesday, December 06, 2005 10:50 AM
To: Joe Shen
Cc: Ejay Hire; 'Kim Onnel'; 'NANGO'
Subject: RE: QoS for ADSL customers

> Could IPtables control traffic with inspecting layer7
> information?

Not layer 7. IPtables works on L3 & L4 (and another

similar system

for linux called ebtables provides filtering at L2) but it

can

be used for setting up qos depending on where from (and

to) the

traffic is going and what port is being used.

For layer7 filtering on linux you need protocol proxies,

and you

can use iptables to redirect all http traffic from subnet

to squid

(although its not designed to be a filter, it can be used

to

implement L7 filtering for http, but I'm not sure it can

be used

19

These are "action" modules - they receive the data when it matches
particular netfilter rules and then do something in place where you
could have accept or reject. But for L7 filtering you need module
that can be used in place of "source" or "destination" rules. Yes
it is possible to build those with linux (like ipset - see ipset.netfilter.org - its pretty cool), but I've not seen ones for
L7 classification - at least not public open source ...

The place to find more about iptable is http://www.netfilter.org
For iptables it is http://ebtables.sourceforge.net (this one you
need only if you're building custom linux bridge).

20

Somebody else emailed me privately link for L7 filtering with linux
(its all experimental and requires custom linux patches for now):
  Layer 7 Kernel HOWTO

Also in previous post it was supposed to be:
   For ebtables it is http://ebtables.sourceforge.net (this is
   needed if you want security when building custom linux bridge)

There are quite a few modules for iptables that will reach
up to Layer 7, including several specifically for file
sharing applications...

And one really nifty one that makes non-passive ftp work
through NAT.

These are "action" modules - they receive the data when it matches
particular netfilter rules and then do something in place where you
could have accept or reject. But for L7 filtering you need module
that can be used in place of "source" or "destination" rules. Yes
it is possible to build those with linux (like ipset - see
ipset.netfilter.org - its pretty cool), but I've not seen ones for
L7 classification - at least not public open source ...

The place to find more about iptable is http://www.netfilter.org
For iptables it is http://ebtables.sourceforge.net (this one you
need only if you're building custom linux bridge).