Consumer-grade dual-homed connectivity options?

Mailman List Mirror (Read Only)
1

Not sure whether this is an appropriate place to post this, but I thought I'd give it a shot, since you're all knowledgeable folks with regard to networking things...

At home, I currently run two DSL lines. Right now, we just have two separate LANs, one connected to each line, with my wife's devices attached to one, and my devices attached to the other. For a while now, I've been thinking about setting up a load-balancing routing solution to give both of us access to both lines.

I have the opportunity to acquire a refurbed Cisco Catalyst 2960 at a ridiculously low price. I also have access to a (nominally) spare quad-core 64-bit PC with 8GB of RAM. I say "nominally" because I'm thinking about setting it up as a media center / gaming rig connected to the TV in the den. That's largely beside the point, but it bears pointing out that keeping the PC available for my other needs would be a good thing.

So.

Is it going to be a more-effective solution to drop a few bucks on the 2960 and go through the hassle of learning how to set it up (and then setting it up), or would I be better off putting a secured Linux distro (e.g. gentoo-hardened, or something) on the semi-spare PC and running the load-balancing via iproute2 and friends?

Either way, I'm looking at a learning curve, and a good amount of time fannying around getting the damn thing working -- there's a good chance I'd spend almost as much cash on the PC-based solution getting good-quality network cards, and maybe fast HDD tech (though it seems like RAM and cores would be more important than disk IO).

What are your opinions?

2

Do you control or have access to the provider side-the PPPoE server-and would both PPPoE connections hit the same PPPoE server at the provider? If so, I recommend setting up a PPP multilink with both DSL lines. The DSL provider would have to support that capability. I also recommend something like a Cisco 2691 router with two WIC-1ADSL cards. I have used this hardware for a 2xDSL multilink to my own home and it worked well.

3

I know nothing of how to do this on a Catalyst; for PCs, my own guess is that you're looking far too high-end. If the issue is relaying to the outside, I suspect that a small, dedicated Soekris or the like will do all you need -- there's no point in switching traffic faster than your DSL lines can run. I'm not doing load-balancing, but all traffic from my house to the outside world (I have a cable modem) goes through a Soekris 4801, and I can download large files from my office at 12-13M bps. Further, since the Soekris is bridging some networks, its interfaces are in promiscuous mode, so the box is seeing every packet on my home LAN. Granted, there usually isn't that much traffic, even though the house is wired for GigE -- but I suspect I'm seeing about as much speed, end to end, as the cable modem will give me.

    --Steve Bellovin, http://www.cs.columbia.edu/~smb

4

Paul Bennett wrote:

At home, I currently run two DSL lines. Right now, we just have two separate LANs, one connected to each line, with my wife's devices attached to one, and my devices attached to the other. For a while now, I've been thinking about setting up a load-balancing routing solution to give both of us access to both lines.

Have you looked at a simple dual-WAN router?

5

2x DSL not so backhoe-resistant.

I like mixing cable with dsl. Tasty disparate paths (modulo garden shears
applied to the single ingres point to your basement) if not technologies, orgs
and methodologies. Or radio + dsl, or pigeon + mule, take your pick.

Would be great if you could rate your connections somehow (ToS? packets under
1000 bytes?) and for those with high priority (voip, ssh < 10K/s != scp, etc)
spray redundant udp packets containing your data down all links, first packet to
the end point wins. Higher speed stuff just gets RR'd for aggregate
bandwidth.

Could even brute force your way through packetloss (ever try typing into an
ssh session with even 10% pl?) with redundant packets down the same links,
just use up 10K/s of bandwidth for 1K/s of desired throughput.

Nicer with the local cable co *IX'd a few ms away from the DSL endpoints. (I
suspect that higher latency differences would make this less viable). Course
there's still the issue of a single org at the endpoint - that's your SPOF,
but it's easily up more than my dsl at home here is. If it fails, use your
base connection to the other provider for internets (unfortunately your ips
for inbound connections wont be working during the outtage without more tricks
at the far end).

Does mulitlink specify any ability such as this, or is this a non existent protocol
as yet? Would anyone find it useful?

/kc

6

2x DSL not so backhoe-resistant.

I like mixing cable with dsl. Tasty disparate paths (modulo garden shears
applied to the single ingres point to your basement) if not technologies,
orgs
and methodologies. Or radio + dsl, or pigeon + mule, take your pick.

*snip*

I'm using cable and wimax in the Chicago suburbs with a dual-wan router.
Works well, would recommend to others, and so forth.

7

I use a T1/26xx for primary and a sprint datacard in a little NAT router for
secondary. The two boxes sit on the same LAN but provide different gateway
IP addresses. The sprint router does the DHCP, so things that ask for DHCP
wind up using that as the primary. Some boxes use the 26xx as default
gateway with static IP's outside the DHCP range. A smart enough box could
choose paths per conversation by playing with the next hop. If that active
path for a box fails I can just change it's default gateway to switch to the
other service. I have a routable C I use for the LAN, the sprint
connections just NAT's it anyway, the other connection is firewalled but not
NAT'd. Seems to work ok for me. Could be made fancier.

8

Back at the Toronto NANOG I bumped into someone who had an interesting solution to the multihoming problem.

What they had was a machine that would key/sequence the packets and send them out each connection (so if they had 2, it would send a copy out each).

Whichever got there first, was decapsulated and forwarded on. Any duplicates/late packets were dropped. This meant that they would always have the speed of the fastest link for either up or down.

They also had a method to load-share to bond the two (or more) links together.

It was some custom solution they built, but something I would like to see a link to or open-sourced.

- Jared

9

> Is it going to be a more-effective solution to drop a few bucks on the
2960 and go through the hassle of learning how to set it up (and then
setting it up), or would I be better off putting a secured Linux distro
(e.g. gentoo-hardened, or something) on the semi-spare PC and running the
load-balancing via iproute2 and friends?

Back at the Toronto NANOG I bumped into someone who had an interesting
solution to the multihoming problem.

What they had was a machine that would key/sequence the packets and send
them out each connection (so if they had 2, it would send a copy out each).

Whichever got there first, was decapsulated and forwarded on. Any
duplicates/late packets were dropped. This meant that they would always
have the speed of the fastest link for either up or down.

They also had a method to load-share to bond the two (or more) links
together.

It was some custom solution they built, but something I would like to see a
link to or open-sourced.

I guess that method presume some cooperating box out there on the net
somewhere to coordinate the far end?

10

At home, I currently run two DSL lines. Right now, we just have two
separate LANs, one connected to each line, with my wife's devices attached
to one, and my devices attached to the other. For a while now, I've been
thinking about setting up a load-balancing routing solution to give both
of us access to both lines.

If you decide to use an IOS-based router, you'll find most what you need here:

http://wiki.nil.com/Small_site_multihoming

Ivan Pepelnjak
blog.ioshints.info / www.ioshints.info

11

Back at the Toronto NANOG I bumped into someone who had an interesting

  >> solution to the multihoming problem.
  >>
  >> What they had was a machine that would key/sequence the packets and send
  >> them out each connection (so if they had 2, it would send a copy out each).

That's exactly what I was alluding to and you may have spoken to the person
that wrote the tool I was thinking of, as that's pretty much what I described.
(He and I both operate out of Toronto.)

  >> Whichever got there first, was decapsulated and forwarded on. Any
  >> duplicates/late packets were dropped. This meant that they would always
  >> have the speed of the fastest link for either up or down.

With similar links (my allusion to low latency between the far ends of the
upstreams across a local *IX), you really reduce jitter as well. Happy voip.

I've used it, it works, just need to get it out there. Esp out here, for my
voip because my latencies go up and down, so I'd rather have my packets go out
twice and first one wins. (I've assisted with customers that have this service
running today and have for a couple years, but I havent set it up locally here
yet as I havent had a real need for reliability til I went all VOIP. I used to
use plain mpppd across multi providers mainly for agg bw, but that's not
nearly as good as this solution for reliability.)

  >> They also had a method to load-share to bond the two (or more) links
  >> together.

As I mentioned, I think based on ToS or packet size. And can even pound through
packetloss with duplicate packets down the same link (though I dont think that's
implimented yet).

  >> It was some custom solution they built, but something I would like to see a
  >> link to or open-sourced.

Still is and still hasnt been moved into a proper wide-deploy testing and
marketing phase. I think it would be useful, but wanted to gauge your
reaction. In fact, Im not sure what the next proper step in the whole
endeavour is. If anyone is intersted in testing/using/assisting with
marketing/selling it, contact me off list and Ill describe the particulars.
Note it aint my tech, I just work closely with the developer.

12

Yes. This allowed the provider to use a variety of different technologies to reach a site, eg: IP over CATV, DSL, Fiber, Wireless, etc with built-in backup.

- Jared

13

Really? If it's connected to a switch, I'd expect it to only see
broadcast/multicast/unknown destination MACs, as well as traffic
actually flowing through the Soekris.

     -- Brett

14

Brett Frankenberger wrote:

I know nothing of how to do this on a Catalyst; for PCs, my own guess
is that you're looking far too high-end. If the issue is relaying to
the outside, I suspect that a small, dedicated Soekris or the like
will do all you need -- there's no point in switching traffic faster
than your DSL lines can run. I'm not doing load-balancing, but all
traffic from my house to the outside world (I have a cable modem)
goes through a Soekris 4801, and I can download large files from my
office at 12-13M bps. Further, since the Soekris is bridging some
networks, its interfaces are in promiscuous mode, so the box is
seeing every packet on my home LAN.

Really? If it's connected to a switch, I'd expect it to only see
broadcast/multicast/unknown destination MACs, as well as traffic
actually flowing through the Soekris.

I believe he's refering to the situation where the soekris is doing the
bridging, since the soekris only has 4 ethernet ports and two pci slots
max it's likely that if you need greater than quantity 3 plus wireless
internal interfaces that you'll need a switch. given the performance
limits of even a 5501 I tend to disagree that the switching traffic
internally in software bridge at less than line rate at 100Mb/s is a
great trade-off vs say using a cheapo gig-e switch.

15

Correct, except that my Soekris has only 3 100Mbps ports.

My house is wired with COTS GigE switches. Outbound traffic passes through the Soekris, which bridges to an older 100M bps switch. That, in turn, is connected to the cable modem and a few older devices that don't need much bandwidth and only have 100baseT ports themselves, like a wireless access point and a printer.

I have that setup for several reasons. First, I want a point from which I can monitor outbound traffic -- home "routers" and switches don't have monitoring ports. I wanted a DHCP server that supported static allocations. I contemplated (but never implemented) putting an IPsec gateway there; I still may do that. I'm about to move my IPv6 tunnel endpoint to the Soekris. I have contemplated multihoming my house, though I might conclude that that would incur too many spousal points. Finally, at one point I had a more complex topology for my home network -- certain locations in the house were separated, to permit imposition of restrictions for, shall we say, violations of the house AUP...

    --Steve Bellovin, http://www.cs.columbia.edu/~smb

16

I believe he's refering to the situation where the soekris is doing
the bridging, since the soekris only has 4 ethernet ports and two pci
slots max it's likely that if you need greater than quantity 3 plus
wireless internal interfaces that you'll need a switch. given the
performance limits of even a 5501 I tend to disagree that the
switching traffic internally in software bridge at less than line rate
at 100Mb/s is a great trade-off vs say using a cheapo gig-e switch.

i am not sure this is the forum for home networking (in fact, i am
pretty sure it's not), but wtf.

i have a 5501 with 8g flash running freebsd 8.0 on a 100/100 b-flets
looking kinda like

           .----------------.
           > >
           > b --wlan0|
           > r | 192.168.0.0/24
ext iij | i --- vr1| LAN hosts,
PPP/NAT ---|vr0--- d | DHCP Clients
  WAN | g --- vr2| pptp 200-209
           > e | ...
           > 0 --- vr3|
           > >
           `----------------'

there is a gige switch on one of the vr ports, but i currently do not
use it (lack of white gaffers' tape to hide cabling). my plan is to use
it for ethers to the mac mini by the tv and the mbps on the desktops so
that file transfers to/from the mini do not go through the soekris.

randy