Restrictions on Ethernet L2 circuits?

Hello,

Anyone with opinions on what restrictions a service provider should and
should not impose on Ethernet L2 circuits provided to business customers
wanting to connect several offices?

The service provider's MPLS core network doesn't mind what traffic flows
through the EoMPLS tunnel, but the L2 access network do mind and can be
vulnerable to several layer 2 issues. Broadcast storm control and BPDU
filter will protect the access network to a certain degree, but there
are still potential layer 2 problems that can affect the switches, for
example MAC address spoofing/flooding. Not to mention potentially
difficult troubleshooting if a business customer connects two large LANs
through the ISP's Ethernet layer 2 circuit, and then experience some
occult layer 2 problem.

Should business customers expect to be able to connect several LANs
through an Ethernet L2 ciruit and build a layer 2 network spanning
several locations? Or should the service provider implement port
security and limit the number of MAC addresses on the access ports,
forcing the customer to connect a router in both ends and segment their
network? Also, do you see a demand for multi-point layer 2 networks
(requiring VPLS), or are point-to-point layer 2 circuits sufficient to
meet market demand?

The most important argument for customers that choose Ethernet L2 over
MPLS IP-VPN is that they want full control over their routing, they
don't want the involvement from the service provider. Some customers
also argue that a flat layer 2 network spanning several locations is a
simpler and better design for them, and they don't want the "hassle"
with routers and network segmentation. But IMO the customer (and the
service provider) is far better off by segmenting their network in the
vast majority of cases. What do you think?

Regards,
Even

----- "Endresen Even" <Even.Endresen@bkk.no> schreef:

Hello,

Anyone with opinions on what restrictions a service provider should
and
should not impose on Ethernet L2 circuits provided to business
customers
wanting to connect several offices?

Although different in concept, but somehwat similair in issues regarding L2, this description from the Amsterdam IX can give some pointers as to which issues need or can be adressed to minimize L2 issues.

See: http://www.ams-ix.net/specifications-descriptions/

Regards,

Nils Kolstein
SSCPlus B.V.

Interesting questions. Here are a few thoughts from the perspective of
an education/research backbone operator that used to be IP only but has
also been offering L2 point-to-point circuits for a few years.

Should business customers expect to be able to connect several LANs
through an Ethernet L2 ciruit and build a layer 2 network spanning
several locations?

At least for our customers, that is indeed important. The most popular
application here is for a customer to connect a remote location to their
campus network, and that want to (at least be able to) use any of their
existing VLANs at the remote site.

Or should the service provider implement port security and limit the
number of MAC addresses on the access ports, forcing the customer to
connect a router in both ends and segment their network?

That would make the service less attractive, and also more complex to
set up and maintain. For point-to-point service, there is really no
reason for the network to care about customers' MAC addresses, VLAN tags
and such. As you said, EoMPLS doesn't care. (Ethernet over L2TPv3
shouldn't care either. If I had cost-effective edge routers that did
L2TPv3 encapsulation/decapsulation at line rate, I'd switch off MPLS in
our core tomorrow.)

Couldn't PBB or even Q-in-Q provide that isolation as well, at least for
point-to-point services? I must say that I don't personally have much
experience with those, because we tend to connect our customers to
EoMPLS-capable routers directly.

Also, do you see a demand for multi-point layer 2 networks (requiring
VPLS), or are point-to-point layer 2 circuits sufficient to meet
market demand?

That's a big question for us right now... we're not sure yet. I'd like to
hear others' opinions on this.

The most important argument for customers that choose Ethernet L2 over
MPLS IP-VPN is that they want full control over their routing, they
don't want the involvement from the service provider. Some customers
also argue that a flat layer 2 network spanning several locations is a
simpler and better design for them, and they don't want the "hassle"
with routers and network segmentation.

I have a good deal of sympathy for customers who think this way. Also
from the service provider point of view, I like the simplicity of the
offering - basically we're providing an emulation of a very long piece
of Ethernet cable. (My worry with multipoint L2 VPNs is that they can't
have such a simple service model.)

But IMO the customer (and the service provider) is far better off by
segmenting their network in the vast majority of cases. What do you
think?

Maybe they already have a segmented network, but don't want to segment
it based on geography/topology.

As far as I'm concerned, enterprises should just connect their various
sites to the Internet independently, and use VPN techniques if and where
necessary to provide the illusion of a unified network. In practice,
this illusion of a single large LAN (or rather, multiple
organization-wide LANs) is very important to the typical enterprise,
because so much security policy is enforced based on IP addresses. And
the typical enterprise wants a central chokepoint that all traffic must
go through, for reasons that might have to do with security, or support
costs, or with (illusions of) control.

This bridging function required to maintain the illusion of a unified
network is something that most enterprises prefer to outsource. I'd
hope that at some point, better security mechanisms and/or better VPN
technologies make these kinds of VPN services less relevant. Until that
happens, there's going to be demand for them. Of course the telcos have
known that for eons and provided many generations of expensive and
hard-to-use services to address this. Point-to-point Ethernet services
are interesting because they are relatively easy to provide for folks
like us who only really know IP (and maybe some MPLS). And the more
transparent they are, the easier it is for customers to use them.

The MEF has a set of specs for this.

http://metroethernetforum.org/

In general, it's built as a "dumb pipe" virtual circuit, IE your client
BPDUs and other IEEE 802.* signaling are ignored, as they are
encapsulated, and forwarded explicitly to a given port. What you do on
the switch that gets the deencapsualted traffic is your business.

> Or should the service provider implement port security and limit the
> number of MAC addresses on the access ports, forcing the customer to
> connect a router in both ends and segment their network?

That would make the service less attractive, and also more complex to
set up and maintain. For point-to-point service, there is really no
reason for the network to care about customers' MAC addresses, VLAN tags
and such.

*If* the customer connects directly to a router which terminates
EoMPLS, I agree. But router ports are usually expensive, which
often means that the customer connects to a switch. And switches
definitely care about MAC addresses.

Couldn't PBB or even Q-in-Q provide that isolation as well, at least for
point-to-point services? I must say that I don't personally have much
experience with those, because we tend to connect our customers to
EoMPLS-capable routers directly.

QinQ does nothing to reduce the number of MAC addresses required.
PBB can do this, but there is still not a lot of PBB equipment
available.

> Also, do you see a demand for multi-point layer 2 networks (requiring
> VPLS), or are point-to-point layer 2 circuits sufficient to meet
> market demand?

That's a big question for us right now... we're not sure yet. I'd like to
hear others' opinions on this.

There is some demand there. Whether that makes it worth it implementing
as a product is another question. Trouybleshooting multipoint is more
difficult than troubleshooting point to point circuits.

Steinar Haug, Nethelp consulting, sthaug@nethelp.no

From: Simon Leinen
Sent: Thursday, December 31, 2009 8:29 AM
Subject: Re: Restrictions on Ethernet L2 circuits?

> Should business customers expect to be able to connect several LANs
> through an Ethernet L2 ciruit and build a layer 2 network spanning
> several locations?

At least for our customers, that is indeed important. The most

popular

application here is for a customer to connect a remote location to
their
campus network, and that want to (at least be able to) use any of

their

existing VLANs at the remote site.

That is what I currently expect from a layer 2 solution. One that does
not allow VLAN tagging across the span is of much less utility.

> Or should the service provider implement port security and limit the
> number of MAC addresses on the access ports, forcing the customer to
> connect a router in both ends and segment their network?

That would make the service less attractive, and also more complex to
set up and maintain. For point-to-point service, there is really no
reason for the network to care about customers' MAC addresses, VLAN
tags
and such. As you said, EoMPLS doesn't care. (Ethernet over L2TPv3
shouldn't care either. If I had cost-effective edge routers that did
L2TPv3 encapsulation/decapsulation at line rate, I'd switch off MPLS

in

our core tomorrow.)

I don't want my provider enforcing such things on me provided it doesn't
blow up their network. If I break my stuff, I expect to own all the
pieces. I don't want them to nanny me and enforce policy that they
determine is "for my own good" but are of no consequence in maintaining
their own network. I want the pipe to be basically a long ethernet
cable. My traffic should be sufficiently isolated as not to cause a
problem in their network no matter what I do.

> Also, do you see a demand for multi-point layer 2 networks

(requiring

> VPLS), or are point-to-point layer 2 circuits sufficient to meet
> market demand?

That's a big question for us right now... we're not sure yet. I'd

like

to
hear others' opinions on this.

I once had such a solution in a network and it worked quite well. It
was the (now defunct) Yipes! NAN (National Area Network) product. We
used it for OSPF area 0 between all of our US facilities (several
offices and two production colos). It worked quite well for the amount
of traffic that went between facilities and it was stable for the
approx. two years we had it deployed. In that case we had only a single
VLAN that acted as a flat layer2 network that spanned the country with a
pair of layer 2/3 switches at each office acting as routers for each
facility area. This solution turned out to be cheaper and more
efficient than running dedicated links to each facility and getting
everything meshed over point-to-point circuts. If someone else already
has a nationwide mesh, it probably makes good sense to lease some of
that capacity than to try and build your own.

(now defunct) Yipes! NAN (National Area Network) product

They don't offer this anymore?

From: Endresen Even
Sent: Thursday, December 31, 2009 12:41 AM
Subject: Restrictions on Ethernet L2 circuits?

Hello,

Anyone with opinions on what restrictions a service provider should

and

should not impose on Ethernet L2 circuits provided to business
customers
wanting to connect several offices?

One thing I *really* miss from about a decade back is the Telseon model.
It was completely self-managed. They would place a switch at the
customer's location and if I wanted to change the bandwidth allocation
on a port, it was as easy as logging in to a management portal and
changing it. So if a port usually required only 10Meg but I needed to
do something different for a few days, I could bump the bandwidth of the
port up to 100Meg and then set it back down when I was done and I was
billed based on what each day's bandwidth setting was. We paid only for
what we had configured for each day.

The other benefit of it was that if someone else also had the same
service, we could self-provision a "logical wire" between us. So if I
wanted to peer or somehow partner with another network that also had a
Telseon switch, we simply logged in to the management portal and
configured it, jacked in to the appropriate port on our respective
switches, and it was done. It didn't even take a phone call, the
provisioning process could be done on the web.

I don't think anyone offers such a MAN service these days and I really
wish it had stayed around.

Yipes! Doesn't exist anymore. They were taken over by Reliance
Globalcom and I am not sure what their offerings are. I would expect
them to offer something similar but possibly under a different name.

Yipes is still offering services under the Yipes, name, at least in the NY Metro Area.

From: Shane Ronan
Sent: Thursday, December 31, 2009 1:39 PM
Subject: Re: Restrictions on Ethernet L2 circuits?

Yipes is still offering services under the Yipes, name, at least in

the

NY Metro Area.

I based my information on the fact that going to yipes.com takes you to
Reliance. And this

"The Yipes name is (presumably) gone forever, as the company has been
assimilated into a new global communications entity, Reliance Globalcom.
Virtually unknown in the U.S., Reliance will need to spend heavily on
branding and marketing efforts."

And this:

Yipes was acquired by Reliance Globalcom in December 2007. Reliance
Globalcom, a division of Reliance Communications, spearheads the Global
Telecom operations of India's largest Integrated Telecom Service
Provider ...

So while they might be operating still as Yipes in some areas, they are
Reliance Globalcom out here in California.

> Couldn't PBB or even Q-in-Q provide that isolation as well, at least for
> point-to-point services? I must say that I don't personally have much
> experience with those, because we tend to connect our customers to
> EoMPLS-capable routers directly.

QinQ does nothing to reduce the number of MAC addresses required.
PBB can do this, but there is still not a lot of PBB equipment
available.

PBB would help a lot, but as far as Cisco equipment is concerned, it is only supported on 7600 with ES40 line cards: http://www.cisco.com/en/US/docs/ios/cether/configuration/guide/ce_mac_evc_pbb_ps6922_TSD_Products_Configuration_Guide_Chapter.html

On my wish list is PBB support on access layer switches, like the Cisco ME3400. This would bring the benefits of tunneling out to the very edge of the network.

We have an MPLS core with a hierarchial Ethernet layer 2 access network between the core and the customer. Typically there are two or three switches between the the PE node and the customer. Even though we are building the MPLS network further out towards the edge, there will always be a layer 2 access network.

It is the switches in the access network that is my concern. We have seen some rather strange problems over the years, like customer nodes that flood MAC address tables with spoofed MAC addresses, and frames that are reflected, causing switches to continually re-learn the same MAC-addresses from different ports. MAC header encapsulation at the very edge of the network would protect the switches in the access layer, and thereby make the service providers more willing to offer more transparent products to their customers.

Regards,
Even