We're looking for a semi-portable solution to validate 10Gb customer
circuits and hitting walls surrounding PCI lanes and the amount of data
laptops can push via their busses. We'd prefer to not have techs lugging
around server equipment for these tests.
Anyone out there testing 10gbE with iPerf? If so, what are you using?
Thanks,
Dan
I have not tried doing that myself, but the only thing that would even be possible that I know of is thunderbolt.
A new MacBook Pro and one of these maybe: http://www.sonnettech.com/product/echoexpresssel_10gbeadapter.html
-Randy
why doesn't a tbird do this for you?
You really want one of these
http://www.jdsu.com/en-us/Test-and-Measurement/products/a-z-product-list/Pa
ges/tb-6000.aspx#.VGFcetZ65PI
Or it¹s larger 9000 series that can scale to 100Gb.
I gotta wonder. How reliable is iPerf over something like RFC2544 or Y.1564? Especially at those speeds?
I just picked up a couple of Accedian’s RFC2544/Y.1564 boxes to use as loopbacks to our field Exfos. We’ll probably wind up buying a few more Accedian boxes for the field where we don’t need to spend the money on an Exfo.
One of the Accedian boxes is arguably less than what you’d pay for a TB MBP and that Sonnet adapter.
I gotta wonder. How reliable is iPerf over something like RFC2544 or
Y.1564? Especially at those speeds?
Apples and oranges? iperf tests a TCP connection (over v4 or v6) and
2544 runs Ethernet tests. Yes, both test throughput, but in very
different ways. Arguably you only need one or the other -- but you work
with what you have.
Anyone out there testing 10gbE with iPerf? If so, what are you using?
As you say, server hardware is heavy to lug around -- but you can fit
quite a lot of CPU in a Mini-ITX/uATX board these days, and newer boards
even come with PCI-E 3.0 capable chipsets (Intel Z87).
Given the right NIC (Intel, if I was buying one) and a recent Linux
distro... I'd be surprised if a recent quad core CPU couldn't generate
10G based on what I've seen our servers do.
We're looking for a semi-portable solution to validate 10Gb customer
circuits and hitting walls surrounding PCI lanes and the amount of data
laptops can push via their busses. We'd prefer to not have techs lugging
around server equipment for these tests.
Maybe something like EXFO or Anritsu. Spirent and Agilent would be best, but
probably too expensive if this is the only use-case.
Anyone out there testing 10gbE with iPerf? If so, what are you using?
Not possible on iPerf. Network testing shouldn't be done with TCP, because
then you don't know what you're testing, are you testing TCP stack of the host
or network?
But writing performing UDP software is quite hard. You cannot use UDPSocket
like iperf does, it just does not work, you are lucky if you reliably test
1Gbps.
I'm not aware of modern performing UDP testing software, which is shame,
because it's possible to do today, it won't be portable and will impose
specific limitations on what hardware to use, but it'll be fast and will
deliver reliable results and quite accurate jitter measurement.
Or one of these ones for dual-10Gbit links (one for out of band management or internet?):
http://www.sonnettech.com/product/twin10g.html
I haven't tried one myself, but they're relatively cheap (for 10gig) so not that much outlay to grab one and try it (esp if you already have an Apple laptop you can test with).
I've done loads of 1Gbit testing using the entry-level MacBook Air and a Thunderbolt Gigabit Ethernet adapter though, and I disagree with Saku's statement of 'You cannot use UDPSocket like iperf does, it just does not work, you are lucky if you reliably test 1Gbps'. I find iperf testing at 1Gbit on Mac Air with Thunderbolt Eth extremely reliable (always 950+mbit/sec TCP on a good network, and easy to push right to the 1gbit limit with UDP.
Pete
Hey,
I've done loads of 1Gbit testing using the entry-level MacBook Air and a Thunderbolt Gigabit Ethernet adapter though, and I disagree with Saku's statement of 'You cannot use UDPSocket like iperf does, it just does not work, you are lucky if you reliably test 1Gbps'. I find iperf testing at 1Gbit on Mac Air with Thunderbolt Eth extremely reliable (always 950+mbit/sec TCP on a good network, and easy to push right to the 1gbit limit with UDP.
In my experience majority of people using iperf in UDP mode do not monitor for
packet loss. And once they start doing, they notice they can't go very fast.
For me 1 packet loss due to host issues is absolutely too much,I need flat 0,
and in optimal scenario, I'd get microsecond resolution jitter statistics.
Granted I've not tried on OSX, perhaps by default it has deeper buffers for
UDP. But on Linux, top of the shelf Cisco UCS server with 10GE interfaces
running RHEL, and 1Gbps is simply too much, you'll get packet loss. You can
observe the packets arriving, but they'll be registered as UDP errors on
'netstat -s', because your program isn't picking them up fast enough.
Things iperf could do to perform better
- setsockopt for deeper buffers
- recvmmsg to pick up multiple messages with single context switch
- raw sockets to reduce overhead
But ultimately you're still going to be very far from 10Gbps, which is doable,
if you'll use something like DPDK.
I find nuttcp very useful in those situations.
Be sure to use one of the recent betas, I have been using 7.2.1 for UDP
with excellent results (decent loss stats and jitter calc)
http://nuttcp.net/nuttcp/beta/nuttcp-7.2.1.c
As I understand it, it's still developed, 7.3.2 is now out.
M
i've found that the tb gigabit ethernet adapter starts dropping udp packets
at around 600-650mbit/sec on iperf (mbp, MC976xx/A model).
Nick
From: pete@fiberphone.co.nz
Subject: Re: 10Gb iPerf kit?
Date: Sun, 7 Dec 2014 09:24:41 +1300
To: nanog@nanog.org> I have not tried doing that myself, but the only thing that would even be possible that I know of is thunderbolt.
> A new MacBook Pro and one of these maybe: http://www.sonnettech.com/product/echoexpresssel_10gbeadapter.htmlOr one of these ones for dual-10Gbit links (one for out of band management or internet?):
I haven't tried one myself, but they're relatively cheap (for 10gig) so not that much outlay to grab one and try it (esp if you already have an Apple laptop you can test with).
How would you use it? with iperf still?I don't think you will go nearly close to 14.8Mpps per port this way.Unless you are talking about bandwidth testing with full sized packet frames and low pps rate.
I personally tested a 1Gbit/s port over a MBP retina 15 thunderbot gbe with BCM5701 chipset. I had only 220kpps on a single TX flow.Later I tried another adapter with a marvel yukon mini port. Had better pps rate, but nothing beyond 260kpps.
I've done loads of 1Gbit testing using the entry-level MacBook Air and a Thunderbolt Gigabit Ethernet adapter though, and I disagree with Saku's statement of 'You cannot use UDPSocket like iperf does, it just does not work, you are lucky if you reliably test 1Gbps'. I find iperf testing at 1Gbit on Mac Air with Thunderbolt Eth extremely reliable (always 950+mbit/sec TCP on a good network, and easy to push right to the 1gbit limit with UDP.
Again, with 64byte packet size? Or are you talking MTU?
With MTU size you can try whatever you want and it will seem to be reliable. A wget/ftp download of a 1GB file will provide similar results, but I dont think this is useful anyway since it won't test anything close to rfc2544 or at least an ordinary internet traffic profile with a mix of 600bytes pkg size combined with a lower rate of smaller packets (icmp/udp, ping/dns/ntp/voice/video).
I am also interested in a cheap and reliable method to test 10GbE connections. So far I haven't found something I trust.
Can't help with faster adapters, but I believe there are some underlying architectural issues here as to why the speeds are hard to achieve, and why some people can and others maybe can't achieve them.
For Carrier Ethernet, I believe most of these are covered in RFC2444 and the related RFC6815. Even with bit speeds up to spec, traffic speeds are impacted non-linearly by customer protocols including the usual suspect, TCP. This is documented in ITU-T Y.1564, clearly enough for simple folk like me. A good example for your corkboard is slide (page) 28 of the excellent 20140409-Tierney-100G-experience-Internet2-Summit.pdf, included as part of a report on 100GE performance test methodologies. Which is how I stumbled across it.
Roy
*Roy Hirst* | 425-556-5773 | 425-324-0941 cell
XKL LLC | 12020 113th Ave NE, Suite 100 | Kirkland, WA 98034 | USA
For RFC2444, please read RFC2544, and forgive the spam.
*Roy Hirst* | 425-556-5773 | 425-324-0941 cell
XKL LLC | 12020 113th Ave NE, Suite 100 | Kirkland, WA 98034 | USA
You have a very good point. My 1gig tests have always been for throughput rather than pps, and therefore was using MTU sized packets and though would be much less load on the CPU.
Pete