Router for Metro Ethernet

Before I get taken for a ride by salespeople, I figured it would be best to
ask the experts of Nanog....

My company is currently in talks to bring an ethernet circuit into our
headquarters, initially committing around 40Mbps. The ISP will be providing
ethernet handoff, but I do not want their managed router offering (Adtran
4430) since it is pricey, non-redundant and I'd rather manage it myself. My
question is about hardware. Can I assume that I can use something like a
Cisco 2000 series router with two built in fast/gig ethernet ports, without
a WIC? and since both sides are ethernet would the routing throughput be
near fast ethernet speed? This is my first dealing with metro ethernet
offerings, and I don't want to assume that the Cisco throughput rates listed
for T1/ADSL etc. are the same for a metro ethernet as the WAN.

Any and all suggestions on the hardware would be greatly appreciated. Thank
you in advance!

a PowerRouter at http://www.mikrotikrouter.com can handle several
hundred meg without issues.

Jeffrey,

We have deployed metro Ethernet in our network... some things to consider:

1) Is metro Ethernet available end to end, if not will you utilize MPLS?
2) We've deployed Juniper EX3200s, Cisco has great solutions as well... for example 2800 series router. We use Cisco as well.
3) Metro Ethernet is available in increments up to 1G, aka 1000Mbs, so I would explore cost solutions for scalability and future proofing.
4) Benchmark tests revealed near wire speed... however, this is contingent upon region, carrier, provider, locale, etc.
5) It's quick. We use it and it works!

Hope this sheds some light.

~Jay Murphy
IP Network Specialist
NM State Government
IT Services Division
PSB – IP Network Management Center
Santa Fé, New México 87505

"We move the information that moves your world."
“Good engineering demands that we understand what we’re doing and why, keep an open mind, and learn from experience.”
“Engineering is about finding the sweet spot between what's solvable and what isn't."
               Radia Perlman
 Please consider the environment before printing e-mail

We use metro E for our WAN and our internet access delivery. The 2600 series routers do not have enough horsepower to do a 40 Mb connection and eigrp. The 2811 can do 40 mb and eigrp but they start to have difficulty when you add in inspection or large ACLs. We just last week turned a 40mb metroe circuit into a 60mb and the router, a 2811, is now have constant problems. We are replacing it with a 2921. However, this router also has 2 100mb connections from local lans that it is also terminiating. For our 100mb metro e connections we use 3845s. The 100 mb service terminates into NM-GEs, which have a faster throughput than the hwics. This setup works well.
On our internet edges we use 2811s with their memory maxed. We have partial BGP routers from 2 isps. One connection is a 30mb and the other is a 25mb. no inspection is done on these but we do have stateless acls running on the inbound. these are running just fine today, but they sit at about 20% cpu all the time.
When doing a metro e connection, make sure the router/switch can do traffic shaping. If it can't, you are relying on the provider to shape your outgoing traffic, which of course will happen down the line, adding additional delay during high usage times.

You should also look at the new cisco small metro switches. They can traffic shape, do bgp and have more than one interface. one of the annoying thing about metro e(at least with qwest) is they have a tendancy to install new pe switches at your locations when you upgrade your service. this means a new connection from them and unless you have extra fiber or copper ports on your router. So to transition to the new circuit, you need to unplug your existing service first. And that means downtime, which no one likes.

Dylan

FWIW, we made the mistake of going for 3825s on a 50Mb/s policed
GigE. Running GRE/IPSec (AIM-VPN'd) and QoS, the boxes go to
100% CPU in the vicinity of 40Mb/s.

-cjp

In our case I believe we would be dealing with just static routes and a
lines of ACL. Do you think the routing protocols are your largest resource
usage in your scenario, or is it also just simple routing as well?

Jeffrey Negro, Network Engineer
Billtrust - Improving Your Billing, Improving Your Business
www.billtrust.com
609.235.1010 x137

Be careful using 3845s for 100 Mbps connections or above - Cisco rates
them at 45 Mbps (and 3825 at half of that) but last time I checked
doesn't make any promises at faster than T3. They're being
conservative about it, but one thing that really can burn the
horsepower is traffic shaping, which you need with some MetroE
carriers.

Jeffrey Negro wrote:

In our case I believe we would be dealing with just static routes and a
lines of ACL.

In that case a linux/FreeBSD router would work great.

- Kevin

question is about hardware. Can I assume that I can use something like

a

Cisco 2000 series router with two built in fast/gig ethernet ports,
without a WIC?

For Cisco, check out the ME3400 series of switches. Be sure to look at
the IOS licensing carefully to see if the features you need are there.

~JasonG

Taffic shaping and eigrp eat a lot. inspection is huge as well. I have no ida what the new zone based firewalling will do to a 2800, but after seeing it on an 1800, I know it will not be pretty. static acls should be easy if they are not really large. I wouldn't go out and grab the new CRYMU bogon list, that would kill you.
The problem is the router CAN do these things, but if you want any management on the back end you get in trouble. things like NBAR and netflow are incredibly important, but the router cannot handle all these services and the routing protocols and the traffic. If you are not doing nbar or netflow today, that doesn't mean you won't in the near future. I have been finding that getting a router that is too small puts you in a precarious position at times. You can either know where your traffic is going and have a router that drops packets, or you can run blind knowing that all those unmonitored packets are getting through.

Dylan Ebner, Network Engineer
Consulting Radiologists, Ltd.
1221 Nicollet Mall, Minneapolis, MN 55403
ph. 612.573.2236 fax. 612.573.2250
dylan.ebner@crlmed.com<mailto:dylan.ebner@crlmed.com>
www.consultingradiologists.com<http://www.consultingradiologists.com>

If your needs are simple IP routing + simple ACL, but you want line rate ethernet, a layer 3 switch might make sense.

http://www.vyatta.com/ ?

Yes, but, according to the Mikrotik web site they appear to be obsolete
and incapable of routing IPv6.

Owen

We run a 3845 at over 300 Mbps and it's less than 50% CPU....most times less
than 30%. No BGP, just OSPF.

Frank

I stand corrected on the Mikrotik... Apparently, while not well documented, they
do, indeed support IPv6 and their Wiki even includes tunnel configuration
information.

Apologies to Mikrotik (and some encouragement to add this to your main-line
documentation).

Owen

Get a used 3550 or a new 3400ME or something. Sounds likeyuou'll get by just fine using an L3 switch.

For better or worse, the Wiki *IS* their mainline documentation.

Fair enough... My point is that http://wiki.mikrotik.com/wiki/Category:Manual#list

Contains no mention whatsoever of IPv6.

If you go, for example, to the Static IP Addressing page from there, there is also
no mention of IPv6.

It would be nice if they made IPv6 easier to find in the same places you would
find the corresponding IPv4 information.

Owen

Actually, the latest version 5 adds IP6 over PPP, I don't know where you
got that they are not capable of routing IPv6. Just have to install the
V6 package.

They just added IPv6 over PPP Support in v5 too :slight_smile: