Question about migrating to IPv6 with multiple upstreams.

I have an interesting situation at a business that I am working on. We currently have the office set up with redundant connections for their mission critical servers and such, and also have a (cheap) cable modem for general browsing on client machines.

The interesting part is that the client machines need to access some customer networks via the main redundant network, so we have a firewall set up to route those connections via the redundant connections, and everything else via the cheaper, faster cable modem. NAT is used on both outbound connections.

With IPv6, we are having some trouble coming up with a way to do this. Since there is no NAT, does anyone have any ideas as to how this could be accomplished?

In a nutshell: how do you have 2 upstream connections, and choose between them based on outbound destination?

thanks,
-Randy

I have an interesting situation at a business that I am working on. We currently have the office set up with redundant connections for their mission critical servers and such, and also have a (cheap) cable modem for general browsing on client machines.

The interesting part is that the client machines need to access some customer networks via the main redundant network, so we have a firewall set up to route those connections via the redundant connections, and everything else via the cheaper, faster cable modem. NAT is used on both outbound connections.

With IPv6, we are having some trouble coming up with a way to do this. Since there is no NAT, does anyone have any ideas as to how this could be accomplished?

In a nutshell: how do you have 2 upstream connections, and choose between them based on outbound destination?

*LAUGH*

really interesting and funny.

my only idea is to have a 2nd ip and 2nd gateway at all "users" workstations with explicit routes. (scales very very well, perhaps run some routing protocol? ospf? :slight_smile:

bye,
   Ingo Flaschberger

*LAUGH*

really interesting and funny.

my only idea is to have a 2nd ip and 2nd gateway at all "users"
workstations with explicit routes. (scales very very well, perhaps
run some routing
protocol? ospf? :slight_smile:

I've thought of that, but that is a management nightmare, particularly on windows machines...

I would rather just get rid of the cable modem, and expand their main connections, but the cost is more than an order of magnitude more expensive.

thanks for the reply :slight_smile:

-Randy

Juniper, *BSD (including pfsense) and Linux all do NAT66 in some form or
other, as potentially do others.

  Scott

I have an interesting situation at a business that I am working on. We
currently have the office set up with redundant connections for their
mission critical servers and such, and also have a (cheap) cable modem for
general browsing on client machines.

The interesting part is that the client machines need to access some
customer networks via the main redundant network, so we have a firewall
set up to route those connections via the redundant connections, and
everything else via the cheaper, faster cable modem. NAT is used on both
outbound connections.

With IPv6, we are having some trouble coming up with a way to do this.
Since there is no NAT, does anyone have any ideas as to how this could be
accomplished?

In a nutshell: how do you have 2 upstream connections, and choose between
them based on outbound destination?

thanks,
-Randy

Standard IP routing, the default gateway of the network can decide based
on a route entry whether to send it to the cable modem or send it to the
firewall.

If the source block is not routed via both connections it won't work without
NAT. I had this same problem trying to use my ISP's native v6 over PPPoE
and maintain a tunnel as backup since it was still pretty flaky as they were
testing it at the time ... no way a residential ISP is going to route 3rd
party blocks for all their customers, and no chance the tunnel provider was
going to route the block my ISP assigned me either ... with no NAT66 in
Tomato/ddWRT/etc it was 100% impossible to have multiple connections ...

I guess I'm a little confused on the setup. You have a firewall with a
connection to a local LAN, another connection to customer network(s), and
a third connection to the Internet via cable modem?

You have NAT setup to NAT your Local LAN out to the Internet and to the
customer network? A customer network device would use the outside IP on
the customer network connection to communicate with devices in the Local
LAN?

I think it makes more sense to me now.

Are you able to create ip6ip tunnels from your firewall/router to each
customer?

I guess I'm a little confused on the setup. You have a firewall with
a
connection to a local LAN, another connection to customer network(s),
and
a third connection to the Internet via cable modem?

You have NAT setup to NAT your Local LAN out to the Internet and to
the
customer network? A customer network device would use the outside IP
on
the customer network connection to communicate with devices in the
Local
LAN?

I think it makes more sense to me now.

             Provider1 Provider2
                    > >
                    > >
cable modem router (PI space, BGP)
    > >
    > >--- Servers
    > >
    -------Firewall-----
              >
           Clients

The clients are on rfc1918 space, or on a small chunk of a block of PI space. For normal web traffic, they get NATed as the outside cable modem IP address on the firewall. For traffic that is to specific places (customer sites), it is routed to the router. For the rfc1918 clients, they are NATed as the PI IP address on the firewall. For the clients that have fully routable PI addresses, they are simply routed normally.

Has worked quite well for a long time.

-Randy

For a fuller discussion of this scenario, you can read this draft:
http://wiki.tools.ietf.org/id/draft-ietf-v6ops-ipv6-multihoming-without-ipv6nat-00.txt

Frank

I have an interesting situation at a business that I am working on. We currently have the office set up with redundant connections for their mission critical servers and such, and also have a (cheap) cable modem for general browsing on client machines.

So basically policy routing?

The interesting part is that the client machines need to access some customer networks via the main redundant network, so we have a firewall set up to route those connections via the redundant connections, and everything else via the cheaper, faster cable modem. NAT is used on both outbound connections.

Yep that sounds like policy routing.

With IPv6, we are having some trouble coming up with a way to do this. Since there is no NAT, does anyone have any ideas as to how this could be accomplished?

Sure there is NAT, you can use prefix translation to translate your Global Address Range from the redundant ISP to the Cable ISP Global address range when leaving that interface. I've run a similar setup with 3 independent ISPs with IPv6 netblocks.

Whichever connection the traffic went out it got the right GUA mapped onto it. Note that this is 1:1 NAT and not N:1.

In my case there was no primary GUA range, I used a ULA on the LAN side of things, and mapped the corresponding GUA onto it when leaving the network. I had 3 rules, 1 for each WAN and mapped the ULA/56 to the GUA/56.

In your case you already have a primary connection of sorts, so I'd suggest using that on the LAN side and only map the other GUA onto it when it leaves the other interfaces.

The policy routing rules on your firewall can make all the routing decissions for you.

If you search google for "IPv6 network prefix translation" there will be a firewall listed that can do this somewhere in the middle of the page.

Cheers,

Seth

Prefix translation looks to be exactly what we need to do here. Thanks for all of the replies.

-Randy

The vastly better option is to obtain a prefix and ASN from ARIN and merely trade BGP with your
upstream providers.

Prefix translation comes with all the same disabilities that are present when you do this in IPv4.

In IPv4, everyone's software expects you to have a broken network (NAT) and there is lots of extra
code in all of the applications to work around this.

In iPv6, it is not unlikely that this code will eventually get removed and you will then have a high
level of application problems in your "prefix-translated" environment.

Owen

The vastly better option is to obtain a prefix and ASN from ARIN and
merely trade BGP with your
upstream providers.

This is precisely what we are doing on the main network. We just want to keep the general browsing traffic separated.

Prefix translation comes with all the same disabilities that are
present when you do this in IPv4.

In IPv4, everyone's software expects you to have a broken network
(NAT) and there is lots of extra
code in all of the applications to work around this.

In iPv6, it is not unlikely that this code will eventually get
removed and you will then have a high
level of application problems in your "prefix-translated"
environment.

I am hoping that this will eventually become a moot point for this particular installation, but as it is, the cable modem is $50/month for 15 Mb/s, and adding 15 Mb/s to the main network would cost around $3,000/month. It is really hard to justify.

If we could BGP via the cable modem, that would be great, but they won't allow it :slight_smile:

-Randy

This is precisely what we are doing on the main network. We just want to

keep the general browsing traffic separated.

If you're worried about browsing traffic and not worried about occasional
other things slipping through, set up Squid and WPAD on your network.
Direct all general internet stuff (via WPAD) out the cheap connection, the
business-critical traffic through the other traffic.

Now things that don't listen to the WPAD configuration (basically anything
but PC and Mac browsers) will go out your expensive connection. But it
sounds like a little bit of leakage wouldn't be a huge problem. You could
get a bit fancier and run DNS on the proxy server, so that the proxy uses
itself for DNS resolution rather than the corporate DNS. That would let you
do basic browsing while the corporate WAN is down.

The proxy would be the only box on the cable modem segment. It would also
need an interface on some internal LAN segment. Default route on it would
be via the cable modem, with routes to everything internal on the internal
interface. Make sure you set the cable modem IP as Squid's outbound IP, and
make sure your WPAD file doesn't use this proxy for anything internal.

Everything else inside the network would have a default route pointing at
the corporate WAN and wouldn't know anything about the cable segment.

The nice thing about this setup is that you don't have any address
translation going on and only one IP per host. You can replace Squid with
the proxy of your choice, doing as much or as little caching as you want to
do (and other things if desired, like virus scanning, deep packet
inspection, or content filtering - if your policy requires it). Make sure
you talk to your legal and/or HR about what logs should be kept or removed
from the proxy. You may also want to repress X-Forwarded-For headers to
keep your internal network addressing hidden while browsing. Also remember
to make sure the proxy is secure enough to trust as a firewall for your
corporation - or put it behind a firewall that is secure enough.

My "(cheap) cable modem for general browsing" provider wouldn't even
delegate RDNS; they'd only put PTRs in *their* servers. Swap BGP
routes with them? Swell dream.

This has become a common strategy: the cheap, fast, commodity service
for the most-of-the-time that it's working and the most-of-the-stuff
that it works for combined with the expensive and slow but reliable
and full featured service for the mission critical apps. One of these
isn't going to come with BGP and a PI prefix, and the technologies we
deploy are going to have to deal with that.

Regards,
Bill Herrin

The vastly better option is to obtain a prefix and ASN from ARIN and merely trade BGP with your
upstream providers.

My "(cheap) cable modem for general browsing" provider wouldn't even
delegate RDNS; they'd only put PTRs in *their* servers. Swap BGP
routes with them? Swell dream.

Or work around it with a free tunnel to a nearby tunnel service that does
support BGP and will give you a /48.

This has become a common strategy: the cheap, fast, commodity service
for the most-of-the-time that it's working and the most-of-the-stuff
that it works for combined with the expensive and slow but reliable
and full featured service for the mission critical apps. One of these
isn't going to come with BGP and a PI prefix, and the technologies we
deploy are going to have to deal with that.

Yep. For IPv6, there are options.

Owen

Today you're probably correct. If you want to have more than one
provider reliably you pretty much need to be doing BGP; or have some
sort of primary-backup setup to fail over from one to the other; or
give each host a global address from each provider (really not
desirable in the majority of networks).

I think in the long term telling everyone to jump into the BGP table
is not sustainable; and not operationally consistent with the majority
of SMB networks.

A better solution; and the one I think that will be adopted in the
long term as soon as vendors come into the fold, is to swap out
RFC1918 with ULA addressing, and swap out PAT with NPT; then use
policy routing to handle load balancing and failover the way most
"dual WAN" multifunction firewalls do today.

Example:

Each provider provides a 48-bit prefix;

Internally you use a ULA prefix; and setup prefix translation so that
the prefix gets swapped appropriately for each uplink interface. This
provides the benefits of "NAT" used today; without the drawback of
having to do funky port rewriting and restricting incoming traffic to
mapped assignments or UPnP.

The firewall keeps track of if a connection is up or down and keeps
policy routing up to date;

You can choose to set it up to either load balance per-flow or as a
primary and backup interface.

You can handle DNS by using RFC 2136 updates for a sub-domain with a
short TTL on a hosted server to keep names up to date in the event of
a link drop.

This doesn't require cooperation from the provider; it doesn't require
knowledge of routing protocols; and it is easy to understand for
people used to the "NAT" environment.

Last time I checked, Linux still needs some work to handle ECMP
routing for IPv6, but once that is in place; combined with patches for
Network Prefix Translation (NPT), it should be trivial to implement.

My guess is within the next year we'll see something pop up that does this.

Ray

Hi Ray,

There's a nuance here you've missed.

There are two main reasons for ULA inside the network:

1. Address stability (simplifies network management)
2. Source obfuscation (improves the depth of the security plan)

Option 1: Obfuscation desired.

ULA inside. NAT/PAT at both borders. You don't use prefix translation
here because prefix translation does little obfuscation: it has a 1:1
relationship with each individual host and still reveals the internal
routing structure.

Option 2: Stability, no obfuscation desired.

ULA inside, prefix translation at both borders.

Option 3: Neither stability nor obfuscation required.

GUA from one of the providers inside. Prefix translation to the other
provider for the connections desired out that border. Giving the hosts
real GUA addresses maximizes application compatibility.

Regards,
Bill Herrin

I try to avoid the Obfuscation argument when I can.

I've seen people try to be smart by telling Law Enforcement that they
don't keep logs and can't point to which host was a problem behind a
NAT box, only to see Law Enforcement take all the PCs instead of the
one in question. So it's always made me nervous. As for the security
value; I think it's more a privacy value than anything. But you can
accomplish almost the same thing by having those hosts use a web
proxy; which you likely want to be doing anyway so you can scan
content for threats.

I personally have no desire for it; but if someone wants to implement
it I won't stop them.