Can someone shed some technical light on the details of how two T1's are
bonded (typically). We've got two sets of T's at two different location
with vendor 'X' (name starts w/ an 'A') and it appears that we're really
only getting about 1 full T's worth of bandwidth and maybe 20% of the
second.
Seems like they're bonded perhaps using destination IP? It's a vendor
managed solution and I need to get some answers faster than they're
coming in. Thanks.
Matt
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Matt Bazan wrote:
Can someone shed some technical light on the details of how two T1's are
bonded (typically). We've got two sets of T's at two different location
with vendor 'X' (name starts w/ an 'A') and it appears that we're really
only getting about 1 full T's worth of bandwidth and maybe 20% of the
second.Seems like they're bonded perhaps using destination IP? It's a vendor
managed solution and I need to get some answers faster than they're
coming in. Thanks.Matt
More than likely they are not bonded t1's they are just load balanced by
the router which by default on Cisco is per session. Meaning pc1 to
t1#1, pc2to t1#2, pc3 to t1#1. If they are truly bonded with some sort
of MUX for a 3 meg port then you would not see the results you are seeing.
- --
http://www.digitalrage.org/
The Information Technology News Center
If you're treating them as two separate links (e.g. two POPs, etc.) then
that's correct, it'll be done by the routers choice of load-balancing (L3).
If you are going to the same POP (or box potentially) you can do MLPPP and
have a more effective L2 load balancing.
Otherwise, it's possible to get an iMux DSU (Digital Link is a vendor as I
recall, but there may be others) that allow that magical bonding to occur
prior to the router seeing the link. At that point, the router just sees a
bigger line coming in (some do 6xT-1 and have a 10meg ethernet output to
your router).
If you're seeing the balancing the way that you are, most likely that vendor
(I have no specific knowledge about the A-vendor) is doing usage-based
aggregation which isn't exactly a balancing act. The ones at some of my
sites are MLPPP which is a vendor-agnostic approach for the most part.
Scott
Is it AT&T?
If so, they only use Cisco Express Forwarding on the router, or so
that's at least what I was told by the level 1 techs. If packet order
reassembly is a an issue and the link is oversubscribed (IE: Heavy
VoIP/gaming use), this method isn't the greatest over others like MLPPP,
or per-flow CEF, but in 99% of circumstances it works great (and has
other advantages). Can you max out the T-1 with two or three separate
"flows" (IE: simultaneous transfers?) If so, it is possible that they
are doing per flow and not per-packet load balancing. It should be per
packet.
Call them up. Once you get screened and transferred to a Cisco guy,
fire away with your questions -- they know their stuff in my
experience. Or if is your equipment, log into the router and see if ip
load-sharing per-packet is set (assuming it is CEF), and confirm they
did the same.
Off topic, but in my experience MLPPP usually does a better job of
getting 190% of a T-1's speed with two of them. CEF usually tops out at
around 160-170% with a single flow, but will max out with as little as
two flows. I don't know why though, and haven't cared since I've never
really had a dual T-1 all to myself without any other users. 2.5
megabit seems to be the single flow norm on our AT&T Circuits at 3 AM
with no usage., 2.8-2.9 with two or three flows.
As for the technical details, here is some reading material that
explains it quite nicely.
http://www.cisco.com/univercd/cc/td/doc/product/software/ios120/120newft/120limit/120s/120s21/pplb.htm
Test file here for speed tests:
ftp://ftp1.optonline.net/test64
Matt Bazan wrote:
Can someone shed some technical light on the details of how two T1's are
bonded (typically). We've got two sets of T's at two different location
with vendor 'X' (name starts w/ an 'A') and it appears that we're really
only getting about 1 full T's worth of bandwidth and maybe 20% of the
second.Seems like they're bonded perhaps using destination IP? It's a vendor
managed solution and I need to get some answers faster than they're
coming in. Thanks.Matt
Matt Bazan wrote:
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Scott Morris wrote:
If you're treating them as two separate links (e.g. two POPs, etc.) then
that's correct, it'll be done by the routers choice of load-balancing (L3).
If you are going to the same POP (or box potentially) you can do MLPPP and
have a more effective L2 load balancing.Otherwise, it's possible to get an iMux DSU (Digital Link is a vendor as I
recall, but there may be others) that allow that magical bonding to occur
prior to the router seeing the link. At that point, the router just sees a
bigger line coming in (some do 6xT-1 and have a 10meg ethernet output to
your router).If you're seeing the balancing the way that you are, most likely that vendor
(I have no specific knowledge about the A-vendor) is doing usage-based
aggregation which isn't exactly a balancing act. The ones at some of my
sites are MLPPP which is a vendor-agnostic approach for the most part.Scott
From: owner-nanog@merit.edu [mailto:owner-nanog@merit.edu] On Behalf Of
Elijah Savage
Sent: Tuesday, January 24, 2006 7:28 PM
To: Matt Bazan
Cc: nanog@merit.edu
Subject: Re: T1 bondingMatt Bazan wrote:
Can someone shed some technical light on the details of how two T1's
are bonded (typically). We've got two sets of T's at two different
location with vendor 'X' (name starts w/ an 'A') and it appears that
we're really only getting about 1 full T's worth of bandwidth and
maybe 20% of the second.Seems like they're bonded perhaps using destination IP? It's a vendor
managed solution and I need to get some answers faster than they're
coming in. Thanks.Matt
More than likely they are not bonded t1's they are just load balanced by the
router which by default on Cisco is per session. Meaning pc1 to t1#1, pc2to
t1#2, pc3 to t1#1. If they are truly bonded with some sort of MUX for a 3
meg port then you would not see the results you are seeing.--
http://www.digitalrage.org/
The Information Technology News Center
Remember he said both t1's are coming from different vendors, which
would only leave the Mux route which is why I said what I said 
- --
http://www.digitalrage.org/
The Information Technology News Center
I'm re-reading it, and slowly, but I don't see mention of having two
different vendors. Perhaps I need to put the beer a bit further away, but
he talks about generic vendor 'x' and notes that it starts with letter 'A'
as further definition, not as two separate vendors.
*shrug*
Scott
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Scott Morris wrote:
I'm re-reading it, and slowly, but I don't see mention of having two
different vendors. Perhaps I need to put the beer a bit further away, but
he talks about generic vendor 'x' and notes that it starts with letter 'A'
as further definition, not as two separate vendors.*shrug*
Scott
From: Elijah Savage [mailto:esavage@digitalrage.org]
Sent: Tuesday, January 24, 2006 8:20 PM
To: swm@emanon.com
Cc: 'Matt Bazan'; nanog@merit.edu
Subject: Re: T1 bondingScott Morris wrote:
If you're treating them as two separate links (e.g. two POPs, etc.)
then that's correct, it'll be done by the routers choice of load-balancing(L3).
If you are going to the same POP (or box potentially) you can do MLPPP
and have a more effective L2 load balancing.Otherwise, it's possible to get an iMux DSU (Digital Link is a vendor
as I recall, but there may be others) that allow that magical bonding
to occur prior to the router seeing the link. At that point, the
router just sees a bigger line coming in (some do 6xT-1 and have a
10meg ethernet output to your router).If you're seeing the balancing the way that you are, most likely that
vendor (I have no specific knowledge about the A-vendor) is doing
usage-based aggregation which isn't exactly a balancing act. The ones
at some of my sites are MLPPP which is a vendor-agnostic approach for themost part.
Scott
From: owner-nanog@merit.edu [mailto:owner-nanog@merit.edu] On Behalf
Of Elijah Savage
Sent: Tuesday, January 24, 2006 7:28 PM
To: Matt Bazan
Cc: nanog@merit.edu
Subject: Re: T1 bondingMatt Bazan wrote:
Can someone shed some technical light on the details of how two T1's
are bonded (typically). We've got two sets of T's at two different
location with vendor 'X' (name starts w/ an 'A') and it appears that
we're really only getting about 1 full T's worth of bandwidth and
maybe 20% of the second.Seems like they're bonded perhaps using destination IP? It's a
vendor managed solution and I need to get some answers faster than
they're coming in. Thanks.Matt
More than likely they are not bonded t1's they are just load balanced
by the router which by default on Cisco is per session. Meaning pc1 to
t1#1, pc2to t1#2, pc3 to t1#1. If they are truly bonded with some sort
of MUX for a 3 meg port then you would not see the results you are seeing.--
http://www.digitalrage.org/
The Information Technology News CenterRemember he said both t1's are coming from different vendors, which would
only leave the Mux route which is why I said what I said
--
http://www.digitalrage.org/
The Information Technology News Center
Uh Scott I think it is I whom by the way is getting up right now and
going to put the rest of the beer back in the fridge. OOOOPS
- --
http://www.digitalrage.org/
The Information Technology News Center
They can be bonded via MLPPP or IMA, as stated previously. Also they can be load-balanced via EIGRP.
What are you using to test your bandwidth (IPerf is pretty handy)? I'm kinda assuming that the T1's are point to point, how far apart are the offices?
-Wil
Matt Bazan wrote: