I'm curious what people here have found as a good standard for providing solid speedtest results to customers. All our techs have Dell laptops of various models, but we always hit 100% CPU when doing a Ookla speedtest for a server we have on site. So then if you have a customer paying for 600M or 1000M symmetric, they get mad and demand you prove it's full speed. At that point we have to roll out different people with JDSU's to test and prove it's functional where a Ookla result would substitute fine if we didn't have crummy laptops possibly. Even though from what I can see on some google results, we exceed the standards several providers call for.
Most of these complaints come from the typical "power" internet user of course that never actually uses more than 50M sustained paying for a residential connection, so running a circuit test on each turn up is uncalled for.
Anyone have any suggestions of the requirements (CPU/RAM/etc) for a laptop that can actually do symmetric gig, a rugged small inexpensive device we can roll with instead to prove, or any other weird solution involving ritual sacrifice that isn't too offensive to the eyes?
We've found that running windows in safe mode produces better results with
Ookla. And MACs usually do better as well. We've gotten >900mb/s with those
two approaches.
I'm on a Mac and launch 40 speedtests at the same time and monitor interface bandwidth
#!/bin/bash
for i in `./speedtest-cli --list | cut -f1 -d')' | head -n 40`; do ./speedtest-cli --server $i & done
I've been able to saturate 10G links with this method
-Matt
I've seen engineers even forget to account for differing behaviors
of vendors, eg: Juniper doesn't display the layer-2 header counters
This means a 920Mb/s link may actually be 100% once you add back in
ethernet framing. Remind folks that they are seeing the TCP/UDP throughput
and there is ethernet + IP headers involved.
- Jared
We use Iperf3 for customers that complain about throughput, it's
relatively low overhead compared to the Ookla HTML5 client. Same
scenario as you, we have the tech hook up their laptop to the
customer's drop and perform testing. I suspect your antivirus may be
attempting to perform real-time inspection on the http(s) traffic,
which would crush the little laptop CPU for sure.
Message me off-list and I'll send you a private Iperf3 server IP to test with.
-Matt
I have this deployed for our customers, it works well. I have yet to hear any complaints of not being able to max out a connection.
https://github.com/adolfintel/speedtest
+1 to Jared. I’ve seen people not account for this when sizing CoS as well on Juniper.
-Eddie
Hi!
Here I have http://www.speedtest.net/result/7475546550 from my notebook
right now. It is i5-2540M CPU.
First of all, NIC is much more important than CPU. Intel NIC can give
1Gbps easy, while Realtek or Broadcom probably never gives you more than
~300mbps.
Linux times faster than Windows in the same hardware config. Speedtest
very dependent on the browser, so try different and find better with
your configuration as well.
Sometimes you will need to tune TCP stack options to have >100mbps in
one TCP session.
Speedtest usually shows good results on download, but somewhy shows slow
upload speed. Nowdays it is better, but several years ago I can't get
more than 100mbps upload in same configuration of notebook and network I
have now. Real uploads was on gig speeds.
But the best is to use IPERF to do meansurements. It is really accurate.
16.07.18 20:58, Chris Gross пише:
Ookla does have a client that you can install in various OSes to remove browser issues.
I use my Lenovo Thinkpad with or any "decent" client machine and run iperf to prove the connectivity. Of course, client switch quality or firewall can be an issue.
Second the recommendation for the downloadable ookla speedtest desktop app.
-Ben
Winner winner chicken dinner. I forgot to pull "Antivirus is at fault" card from my deck. 250/675 with it installed, 920/920 when removed so now I get to pass the the issue onwards.
Thanks everyone for your replies and the responses for the adolfintel/speedtest github, I'll definitely look at it as a replacement for later.
I recently talked at the IRTF on this subject and followed up with a blog post at https://blog.apnic.net/2018/06/21/measurement-challenges-in-the-gigabit-era/. There's also an open source speed test project you may want to consider at https://github.com/Comcast/Speed-testJS.
Jason
It's a complete rabbit hole different hardware with different browsers give different readings, even not having your laptop plugged into power can cause a change in results due to dropping cpu to power save. The only reliable solution we found for field techs was the exfo ex1. Still talks to the ookla speedtest server etc. Obvious this is a well known issue and exfo has a solution.
https://www.exfo.com/en/products/field-network-testing/network-protocol-testing/ethernet-testing/ex1/
Carlos Alcantar
Race Communications / Race Team Member
Phone: +1 415 376 3314 / carlos@race.com<mailto:carlos@race.com> / http://www.race.com/>
My practice is to use iperf with packet capture on both sides. The packet
capture can then be analyzed for accurate per-second, or less, throughput,
re-transmit rates, etc. This was implemented in a corporate network in
several ways including dedicated servers (that also did other monitoring),
and bootable CDs or USB sticks that a user in a small office could run on a
standard desktop. Many interesting issues were discovered with this
technique, and a fair number of perceived issues were debunked.
Here is a wrapper to run iperf + tcpdump on each side of a connection (it
could use some automation):
https://github.com/meekj/perl-packet-tools/blob/master/run_iperf
I originally did the analysis in Perl, but that can be fairly slow when
processing 30 seconds of packets on a saturated GigE link. If anyone is
interested there is now a C++ version along with analysis code in R at:
https://github.com/meekj/iperfsum
That version currently has only one second resolution. I have a R interface
to libpcap files that could be used for analysis at any time resolution:
https://github.com/meekj/libpcapR
I have a plan to implement the complete test environment in a Docker
container at some point. I also have a collection of small, mostly
low-cost, computers that I plan to benchmark for network throughput and
data analysis time. Some of the tiny computers can saturate a GigE link but
are very slow processing the data.
Jon
It's a complete rabbit hole different hardware with different browsers give different readings, even not having your laptop plugged into power can cause a change in results due to dropping cpu to power save. The only reliable solution we found for field techs was the exfo ex1. Still talks to the ookla speedtest server etc. Obvious this is a well known issue and exfo has a solution.
https://www.exfo.com/en/products/field-network-testing/network-protocol-testing/ethernet-testing/ex1/
This is an interesting device. But the manufactures pages promote it like “Speedtest for Dummies”.
Why don’t the User Manual or Spec Sheet mention IPv6 (or even (IPv4)?
I should think technicians would want technical answers.
Cutler
Thanks, Jason.
While I might have idle curiosity of how well my link performs when I first get it, beyond that the only time I care is when I or somebody else in the house starts screaming "THE INTERTOOBZ R SLOWZ!@!".
I just had this happen to me the other night as I trying to watch possibly the worst movie ever on Amazon Prime. Maybe because I've been around a long time, my first suspicion is that something at home was slagging the ISP hop. This is often the case, but it is *maddeningly* slow to diagnose -- my popcorn was getting stale. Naively, I may have thought to hook up to one of the speed testers, but it would only show what was obvious from ping times, etc: lots of drops.
So where was the problem? I had to manually go around and start disconnecting things. Even after I thought I had found the perp several times, my popcorn's freshness suffered. And in the end, I never did triangulate out where the problem was... and by morning it had magically fixed itself. My popcorn, on the other hand, gave its all.
As we get more and more stuff on our home networks, the probability for crappy software, infected devices, piggy updates, etc multiplies. I'm sure this isn't news to $CORPRO network managers, but at home we quite literally have nothing to help us figure out and remedy these kinds of problems. Or if it turns out to *not* be our problem, that we have some reassurance when we decide to call the support desk and be put through IVR maze hell only to find out it was a local problem after all.
cheers, Mike
Hi Chris,
I'm curious what people here have found as a good standard for providing solid speedtest results to customers. All our techs have Dell laptops of various models, but we always hit 100% CPU when doing a Ookla speedtest for a server we have on site. So then if you have a customer paying for 600M or 1000M symmetric, they get mad and demand you prove it's full speed. At that point we have to roll out different people with JDSU's to test and prove it's functional where a Ookla result would substitute fine if we didn't have crummy laptops possibly. Even though from what I can see on some google results, we exceed the standards several providers call for.
Most of these complaints come from the typical "power" internet user of course that never actually uses more than 50M sustained paying for a residential connection, so running a circuit test on each turn up is uncalled for.
Anyone have any suggestions of the requirements (CPU/RAM/etc) for a laptop that can actually do symmetric gig, a rugged small inexpensive device we can roll with instead to prove, or any other weird solution involving ritual sacrifice that isn't too offensive to the eyes?
I would say, don't use a browser based speed test - how fast is your
browser? Answer: It can vary wildly!
Also there are SO many variables when testing TCP you MUST test using
UDP if you want to just test the network path. Every OS will behave
differently with TCP, also with UDP but the variance is a lot lower.
Also I recommend you test to a server on you network near to your
peering & transit edge. This way users can test up to the point where
you would have over the "The Internet" and have no further control.
Testing to a server off-net (like off-net Ookla tells me nothing in my
opinion).
Virtually any modern day laptop with a 1G NIC will saturate a 1G link
using UDP traffic in iPerf with ease. I crummy i3 netbook with 1G NIC
can do it on one core/thread.
We have several iPerf servers dotted around the network and our
engineers can test to those at any time and it works well for us.
Cheers,
James.
I guess if you use large packets this might be true. But personally,
if I'm testing network, I'm interested in latency, jitter, packet, bps
_AND_ pps goals as well, not just bps goal. And I've never seen clean
1Gbps on iperf with small packets. It just cannot be done, even if
iPerf was written half decently and it used recvmmsg, it still
wouldn't be anywhere near.
Clean 1Gbps with small packets in user space is actually very much
doable today, just you can't use UDP socket, you must use AF_PACKET on
Linux or BPF on OSX and you can write portable 1Gbps UDP
sender/receiver.
I'm very surprised we don't have iperf like program for netengs which
does this and reports latency, jitter, packet loss with binary search
for highest lossless pps/bps rates.
I started to write one with Anton Aksola in Rust (using libpnet[0]),
and implemented quite flexible protocol (server/client, client can ask
server exactly what kind of packet to construct/expect, what rate to
send/receive over JSON based protocol), so you could also use it to
ask it to DDoS your routers control-plane in lab etc. And actually got
it working, OSX+Linux ~wirarate (still needs higher end laptop to do
1.5Mpps on single core and we didn't implement multicore support). But
as both of us are trash in Rust (and every other applicable language
in this domain), we kind of dropped the project once we had sufficient
POC running on our laptops.
Someone who actually can code, could easily implement such program in
a weekend. I'm happy to share the trash we've done if someone intends
to check this box in open source world. May use it for inspiration, or
just straight up add polish and enough CLI to make it usable as-is.
I think very important quality is multiplatform with static binaries.
Because important use case is, that you can ask modestly informed
customer to copy paste one line to donwload server and copy paste
another line to have it running.
If use case is that both ends have arbitrary clued people, then there
are plenty of good solutions, like Cisco's trex[1]. But what I need is
iPerf-like program, which actually a) performs and b) reports the
correct things.
[0] https://github.com/libpnet/libpnet
[1] https://trex-tgn.cisco.com/
One of the issues I have repeatedly run into is an incorrect burst size set on shaped carriage circuits. In the specific case I have in mind now, I don't recall what the exact figures were - but the carrier side configuration was set by the carrier to be something like a 64k burst on a 20M L3 MPLS circuit. End to end speed testing results both with browsers and with iperf depended enormously on if the test was done with TCP or UDP.
Consequently the end customer was unable to get more than 2-3MBit/sec of single stream TCP traffic through the link. The carrier insisted that because we could still get 19+ MBit/sec of UDP then there was no issue. This was the same for all operating systems. The end customer certainly didn't feel that they were getting the 20Mbit circuit they were sold.
The carrier view was that as we were able to get 20 TCP streams running concurrently and max out the link that they were providing the service as ordered. After many months of testing and negotiating we were able to get the shaper burst increased temporarily, and the issue completely went away. The customer was able to get over 18MBit/sec of continuous TCP throughput on a single stream. I was told that despite this finding and admission that the burst was indeed way too small, the carrier was going to continue to provision circuits with almost no burst, because this was their "standard configuration".
The common belief seemed to be that a burst was a free upgrade for the customer. I was of the alternate view that this parameter was required to be set correctly for TCP to function properly to get their quoted CIR.
I'd be very interested in other's thoughts with regards to testing of this. It seems to me that measuring performance with UDP only means that this very critical real-world aspect of a circuit (burst size on a shaper) is not tested, and this seems to be a very common misconfiguration. In my case...seen across multiple carriers over many years and many dozens of hours spent on "faults" related to it.
[NB: I've always used the rule of thumb that the L3 burst size should be about 1/8th the Contracted Line Rate, but there seems to be no consensus whatsoever about that...certainly no agreement whatsoever within the carrier world]
Reuben