Is Cisco equpiment de facto for you?

I try to follow the Tolly Group who compares products, and they

>continually show that Cisco equipment
>is a poor performer in almost any equipment compared to others, I find >that so hard to believe.....

Just a rough comment here. Tolly's business model is a sponsored test one, and Cisco is the dominant vendor in the marketplace. So when anyone has a new product they want to promote, they can hire Tolly's lab to show how a particular feature set is implemented and can often show how one model exceeds Cisco's performance on some or all benchmarks.

While Tolly never cheats, remember that the guys who "win" the test are the ones that are helping to design the test methodology, and so they are focusing on features that they know that they can beat Cisco on. Thus, the Tolly reports are true, but may not be telling the whole story or may be focusing on a particular feature more than anything else. No one is lying to you, but you may not be given the whole picture.

There's nothing wrong with having Tolly show that your product is in the same league as Cisco's, but I would not necessarily read those reports to say that Cisco is always the "loser" in any new product comparison. The real value in Tolly's reports is to give credibility to the new and up-coming products by having a mostly-independent source show that the new products are comparable with the market leaders' in some ways.

Tolly is also very very important at doing spec verification: showing that a product actually DOES what the specification sheet says it does. Remember that those numbers are often made up by product managers based on poorly-done in-house tests, so having a third-party, even a paid one, say "yes, it does do that in our test lab" is WAY more information than you get from most vendors.

Tolly provides a valuable service to our community but you need to look at a combination of benchmarks from Tolly as well as other very-unbiased sources (IDG and CMP magazines are pretty thorough) AND YOUR OWN TESTING to know what is best for you.

On the other hand, compared to the analyst firms like Gartner who don't even have test labs and generate their reports based on phone calls, Google searches, and unverified specification sheets, Tolly is a source of infinite knowledge about products.

Finally, also remember that performance is only one part of system design and deployment. Management and feature completeness are, for the operator community, often more important than whether the box can do 80 Gbps or 85 Gbps. Product selection is best done by YOU setting YOUR requirements and doing YOUR OWN testing, not by looking at point tests to see if one product version/model/device beats another in a particular small set of performance tests.

Obligatory disclaimer: we don't compete directly with Tolly, but we do get paid by the magazines to do testing and there is some overlap, so we do compete indirectly.