Router Suggestions

For around $11,000 right now, you can get a brand new Juniper MX204 router. Alternatively, you can get a used MX240 / MX480 with quad power supplies, redundant quad core RE’s, and 2 16X10G MIC cards for around $12,000.

My question, is there anything else worth looking at in this price range / port configuration? Open to both new and used options. Looking to take full BGP routes.

Do you want high-touch or a packet pusher? The MX204 is somewhere in the middle.

Extreme SLX9540 and Arista 7280SR will "take full tables" with some FIB compression and route caching. YMMV if they'll actually work in your application, but my experience with the 9540 has been positive in a typical leaf edge application. Street price is in the ballpark of what you're talking, and you get a few 100GbE ports to go with your 10GbE ports.

The SLX9640 or 7280R will apparently actually fit full routes in hardware, but the pricing seems to be a fair bit higher than you're talking.

All of these are pretty much packet pushers with minimal "touch". In particular, traffic control capabilities are somewhat limited aside from applying them to the port itself, and they definitely won't do "BNG" type functionality with PPPoE or tag-per-customer with shared L2 appearance at least not at any real scale.

Colton,

We recently opted for the Arista 7280R2K for peering edge. They come in at
similar price points (maybe a little more?) to the MX204 and are a bit higher
capacity.

Worth a look in.

Cheers,

Patrick

Mon, Jun 15, 2020 at 07:00:55AM -0500, Colton Conor wrote:

We've been setting up some Arista DCS-7280CR2K-30-F lately and they have been just OK. The pricing is not at all close to $12,000 though.

-Drew

Drew,

A 6 Tbps router is a little more expensive than a 2 Tbps router, yes.

I was referring to the 7280SR range not the 7280CR.

I ended up getting our SR2k's around the same price as MX204's with the help of our friendly Arista rep.
MX204's may have gotten chaper in the last year I don't know. But YMMV.

-PC

OP needs to check the licensing package for the MX204, and work out the N-year TCO.

Nick

We just got a MX204 quote and it was close to 2.5x the price you’re quoting, with apparently the minimum license needed for full tables, and Next Day replacement. So if it’s really $11K, please shoot me an email off list. Or if someone has a better place to get a decent quote for a MX204, or can clarify where this quote might have went wrong, that would be useful too.

We’re also looking at going the virtual router route where we put 2-3 servers in a HA cluster loaded up with 10Gb interfaces and running some sort of routing software. In case you didn’t catch on, I’m fairly early in running this idea through the paces, although it seems like this is a pretty common thing nowadays.

Yes I too looked into that. And it was not near that price. Please send me and email off list. I would like to know where I might find that.

As someone who has used VSR (Nokia) and VMX (Juniper) I’d suggest, good luck on your plan to use servers for this sort of routing. If you want a cheap router to handle full tables and a couple of 10G interfaces worth of throughput I’d suggest you would be a lot better off with Mikrotik’s latest hardware offering - https://mikrotik.com/product/ccr2004_1g_12s_2xs

Just my 2c

I bought three MX204 a year ago and paid maybe 50% more than the quoted 11K for hardware and standard license. On top of that I paid a significant amount for BNG features and scale licenses, but not everyone needs that.

The third MX204 was considerably cheaper (half price) because its purpose in life is to be a cold spare and a lab router. Why pay someone else for having a cold spare ready for next day replacement when you can have it yourself? Having a lab router to test config before rollout has really been a life saver.

Averaged by the three routers I may have hit close to 11K, not counting the BNG licenses.

Regards,

Baldur

Depends on network size. You can have multiple failures happening in the
same week, and you may not necessarily have all the spares to cover the
replacements at the same time, in all the locations.

But yes, I agree that if it makes sense to buy cold spares than pay for
RMA support, go for it. As long as you are still paying for TAC support
if it's something you find useful.

Mark.

We all like our edge boxes to be not susceptible to any old garbage that might come at them in form of BGP advertisements from the Internet (500 AS-PATH prepends, 1000s of communities, or straight out exotic attributes - just to name a few). So I'd recommend a vendor that has some pedigree in providing facilities to withstand these wild whims of the Internet and ability to normalize/bleach the routing information before sending it to the rest of your AS via iBGP.

Examples:
RFC 7606 - Revised Error Handling for BGP UPDATE Messages
Max as path limit
Max community limit
Max prefix per session limit
Attribute filtering
Etc...

adam

Covering them all under vendor contract doesn’t necessarily guarantee that
the vendor does, either. In general, if you can cover 10% of your hardware
failing in the same 3-day period, you’re probably not going to do much better
with vendor support.

Of course, YMMV.

Owen

In my experience, our vendors have been able to abide by their
obligations when we've had successive failures in a short period of
time, as long as our subscription is up-to-date.

I am yet to be disappointed.

Mark.

e.g. your production deployment might be in another country, and getting equipment in and out of the country could involve customs headwreck, delay and cost.

Or you might have only a handful of a specific type of device so there would be no justification getting a cold spare / lab unit.

There are lots of good reasons to pay for support, but then again there are also lots of good reasons not to pay for support. It's highly dependent on what you're trying to achieve and there's no one-size-fits-all approach.

Nick

Count your blessings… I once faced a situation where a vendor had shipped a batch of defective power supplies (10s of thousands of them). It wasn’t just my network facing successive failures
in this case, but widespread across their entire customer base… By day 2, all of their depots were depleted and day 3 involved mapping out “how non-redundant can we make the power in our
routers to cover the outages that we’re seeing without causing more outages than we solve?”

It was a genuine nightmare.

I’ve had other situations involving early failures of just released line cards and such as well.

As I said, YMMV, but I’m betting your vendor doesn’t stock a second copy of every piece of covered equipment in the local depot. They’re playing the statistical probabilities just
like anyone else stocking their own spares pool. The biggest difference is that they’re
spreading the risk across a (potentially) much wider sample size which may better normalize
the numbers.

Owen

Count your blessings…

I know that we are lucky that in the markets we operate, local depots
are available. There are other markets in Africa that may not be so
lucky. If we ever built into those markets, we'd certainly cold spare as
much as possible, as we used to in the current markets that the vendors
didn't have local depots for 10 or so years ago.

As I said, YMMV, but I’m betting your vendor doesn’t stock a second copy of every piece of covered equipment in the local depot. They’re playing the statistical probabilities just
like anyone else stocking their own spares pool. The biggest difference is that they’re
spreading the risk across a (potentially) much wider sample size which may better normalize
the numbers.

Yes, it's just like a bank - they hope not all customers come to
withdraw all their cash on the same morning.

We run a CRS 4-port 100Gbps line card that I know is not very popular
among other operators in the markets where we have them. We had one fail
in a smaller city a few weeks ago. We pay for NBD, not 24/7. A new line
card arrived promptly, the morning after. I did hold my breath, but they
managed.

But yes, this is one of those things to seriously consider before you go
standing up a network in a new market.

Mark.

Covering them all under vendor contract doesn’t necessarily guarantee that
the vendor does, either. In general, if you can cover 10% of your hardware
failing in the same 3-day period, you’re probably not going to do much better
with vendor support.

In my experience, our vendors have been able to abide by their
obligations when we’ve had successive failures in a short period of
time, as long as our subscription is up-to-date.

I am yet to be disappointed.

Count your blessings… I once faced a situation where a vendor had shipped a batch of defective power supplies (10s of thousands of them). It wasn’t just my network facing successive failures
in this case, but widespread across their entire customer base… By day 2, all of their depots were depleted and day 3 involved mapping out “how non-redundant can we make the power in our
routers to cover the outages that we’re seeing without causing more outages than we solve?”

It was a genuine nightmare.

Huh, was this in the early to mid 1990’s?

I had an incident in NYC area where one of the large (at the time) datacenter/IXPs had a power outage, and their transfer switch failed to switch over. Customers were annoyed, so they promised another test, which also failed, dropping power to the facility again… now customers were hopping mad…

The next test was just of the generator, but with all of the work they had done they had (somehow) gotten the transfer switch really confused / hardwired into an odd state. This resulted in the facility being powered by both the street power and the generator (at least for a few seconds until the generator went “Nope!”)

These were of course not synchronized, and so 120V equipment saw 0V, then 240V, then some weird harmonic, then other surprising values. … most supplies kind of dealt with this OK, but one of the really common models of router, from the largest vendor upped and died. This resulted in a few hundred dead routers and way exceeded the vendors spares strategies.

A number of customers (myself included) had 4 hour replacement contracts, which the vendor really could not meet - so we agreed to take a new, much larger/better model as a replacement.

W

We always have at least one spare, or something that could be (relatively) easily pressed into service as one.

Even in the Midwest, we’ve had times where ‘guaranteed next day replacement’ is more like 2nd or third day due to weather conditions, the carrier routing it weird, or just plain the plane didn’t come today issues. We generally laugh when they try to offer us 4 hour contracts – we know there’s 0 chance they can meet them, and they never want to refund you when you need it and they can’t.

Count your blessings…

I know that we are lucky that in the markets we operate, local depots
are available. There are other markets in Africa that may not be so
lucky. If we ever built into those markets, we'd certainly cold spare as
much as possible, as we used to in the current markets that the vendors
didn't have local depots for 10 or so years ago.

As I said, YMMV, but I’m betting your vendor doesn’t stock a second copy of every piece of covered equipment in the local depot. They’re playing the statistical probabilities just
like anyone else stocking their own spares pool. The biggest difference is that they’re
spreading the risk across a (potentially) much wider sample size which may better normalize
the numbers.

Yes, it's just like a bank - they hope not all customers come to
withdraw all their cash on the same morning.

Yep… FWIW, my experiences were in locations in the US with NFL teams and multiple depots proximate to each location. That didn’t help in these cases.

We run a CRS 4-port 100Gbps line card that I know is not very popular
among other operators in the markets where we have them. We had one fail
in a smaller city a few weeks ago. We pay for NBD, not 24/7. A new line
card arrived promptly, the morning after. I did hold my breath, but they
managed.

Yeah, that’s far less likely to be a problem than a popular line card or other component that turns out to have a bad batch. Generally, they’ll keep at least one of everything any customer has in at least one nearby depot. OTOH, I bet if you’d had two of those cards fail, you might
have been SOL on the second one for a couple of days.

Owen