Hello Nanog,
I'm currently looking into what are the options for enabling inter-datacenter communication.
Our current solution is to use ipsec/gre tunnels traversing over the Internet. The specific needs the new solution must meet are:
- The ability to run end-to-end QOS.
What are you trying to accomplish?
Do you need to be able to pass DiffServ/DSCP tagging between sites?
- Dedicated bandwidth
- Support 1gbps transfer rates
- Enable communication between 3 locations
Okay.
The options I have looked into so far are:
- Layer 2 Ethernet (Virtual Private Line): This service seems to be offered by a lot of ISPs using various networking techniques. The price point is attractive however packets are forwarded only at best effort across the ISP's network which means the quality of the service will directly reflect the ISP's network performance.
How is this a problem? Is that concern that you never want an
interface which is (physically, to routing protocols, ...) "up" but
latent and dropping packets like whoa, from an application or
monitoring/management prospective?
You raise a valid point about oversubscription. At the same time,
this is often overhyped by marketing people, and dependent on how
ghetto your pseudowire provider is and whether or not they know how to
capacity-plan.
- Traditional Leased Line (dsX/ocX): This service seems to be more expensive then wavelength services however meets my needs.
Quite. And it limits your router options significantly while driving
up capex costs. Just say no!
- WaveLength Services (oc3-10gig): This service seems to be cheaper then traditional leased lines when comparing similar bandwidth. However, availability is limited to on-net buildings. This solution meets my needs.
Not a bad idea, but often overlooked when purchasing unprotected
long-haul waves is that you can be down for days or weeks on end,
depending on the severity of a given fiber cut. And protected waves
cost significantly more because the carrier is provisioning twice the
capacity -- sometimes in a configuration not as redundant as
advertised. This is not for the faint of heart, and best left to ISPs
who are buying from multiple vendors/cable systems and put in the
effort to engineer suitable diversity. As an end-user, a switched
service might afford you more economical route protection.
- MPLS based VPN solutions: Seems to be a good point to multipoint technology with QOS offerings. However, the price seems to be around the same as wavelength services for the amount of bandwidth we require. If the number of data centers we were looking to connect was larger then this option would be more attractive. This solution meets my needs.
(Assuming you're talking about l3vpn, as l2 can be grouped into your
first example...)
It would probably help if you'd explain the "QOS" feature set of the
offerings you're looking at.
This is a highly technically complex deployment; even at the largest
telecoms, you can count on one hand the number of staff expert in its
implementation and troubleshooting. It's also the most limiting in
terms of specific routing protocols and prefix counts supported, the
type of traffic you can pass, etc. The only benefit I can see to a
l3vpn is in the enterprise with a lot of branch offices, where it
simplifies end-site configurations and hub/spoke topology. Connecting
your three datacenters, this is obviously not an issue. These are
often the most expensive solutions too, given that their target
customers have deep pockets.
Based on my needs and what my options are I am leaning towards point to point wavelength services connecting my 3 locations in a loop like fashion.
Are there any other options I should consider?
None come to mind.
Are my descriptions of the today's possible solutions inaccurate?
More or less, though it would help if you'd explain more what you're
trying to get out of the "QOS".
Best Of Luck, and Drive Slow,
Paul Wall