Apple Catalina Appears to Introduce Massive Jitter

Mailman List Mirror (Read Only)
1

Hi all.

I’ve been on High Sierra for several years now due to a limitation with an app that couldn’t deal with Apple’s latest rounds of system permissions since Mojave. Eventually, I gave up on waiting for them to fix it and upgraded my older Butterfly keyboard laptop to Catalina 4 weeks ago.

At the same time, I picked up the new Magic keyboard laptop 2 weeks ago which came with Catalina.

Over the past week, I’ve been troubleshooting a massive jitter issue on Catalina, just between itself and my home router. For control, I have a Windows PC (tower-top) using a wireless adapter to connect to my home network. That has no jitter at all.

I have noticed as much as 300ms+ jitter on Catalina.

I then asked a few friends around the world to run tests for me on their own Catalina installations to their local router over wi-fi, and the results are the same. Jitter so high that what should be a 1ms - 5ms latency can (for a short period) jump to 200ms+, 300ms+, 400ms+.

On the off-chance that it is an issue with the new wireless chips on the later MacBook models, one of my friends tested the same on a 2013 MacBook Pro running a beta version of Big Sur. Same story!

Another friend in South East Asia, testing on a 2018 13-inch MacBook Pro running Catalina, also had the same issue.

A Google search suggests that this is some known issue since Mojave, to do with Location Services, and some other apps, in a non-deterministic way:

For me, even after disabling all or some Location Services features, the problem remains. Is anyone else seeing this on their Catalina Mac’s while on wi-fi? If so, does anyone know what’s going on here? Ideally, this wouldn’t matter if it was just a cosmetic issue - but I do actually see physical impact to performance of network access to/from the laptop, which has all the hallmarks of high jitter and/or packet loss. An app like Zoom, which can display network performance data for a session in real-time, does indicate nominal packet loss for audio and video on this device, while other devices on the same WLAN are happy. Thoughts? Mark.

2

Is it actually jitter or is it potentially the wireless network card going into sleep mode? I have seen that type of behavior on Apple products when the cards go into low power mode although I can’t say I have noticed that on my laptop.

3

Ah yes, an example of what I am seeing:

Marks-MacBook-Pro.local (172.16.0.239) 2020-10-29T14:28:27+0200
Keys: Help Display mode Restart statistics Order of fields quit
Packets Pings
Host Loss% Snt Last Avg Best Wrst StDev

  1. 172.16.0.254 0.8% 126 3.9 34.7 2.5 232.1 54.9

Mark.

4

Not, not sleep mode, because I am actively using the device to move data to/from the Internet.

I was actually struggling to upload some files to Youtube last weekend, and had to use another computer to do it as I couldn't figure out what was going on. That it is how bad the jitter is.

It seems to be a much bigger problem for the upload direction than the download, but it, inevitably creates a symmetrical performance problem.

Mark.

5

Have you ruled out local wireless issues, such as a literal side-by-side test?

Does the problem still exist when wired?

6

I know there was a recent fix Apple did for devices talking to UBNT APs
for their handsets, perhaps there's a similar fix needed on your side?

  I have all UBNT at home for wireless and periodically have some random
issues which I can't explain, but for the most part have things tuned to ensure
there's little to no interference.

  Do you see the same when hardwired? I keep many of my devices hardwired
to avoid odd jitter issues. I also saw some older versions of the Pulse Secure
VPN add the behavior you describe, including the more uptime the slower it would
get.

  - Jared

7

Yes - all other wi-fi devices don’t exhibit this issue, including my wireless-connected PC. Only this MacBook running Catalina. The problem exists at all wi-fi AP’s around the house, all of which are easily 15m apart to cover the whole house. No problem on wire. As you can see below, my worst record latency value is 11.8ms only: Marks-MacBook-Pro.local (172.16.0.239) 2020-10-29T14:48:12+0200 Keys: Help Display mode Restart statistics Order of fields quit Packets Pings Host Loss% Snt Last Avg Best Wrst StDev 1. 172.16.0.254 0.0% 50 1.0 2.6 0.7 11.8 2.1 Mark.

8

  I know there was a recent fix Apple did for devices talking to UBNT APs
for their handsets, perhaps there's a similar fix needed on your side?

  I have all UBNT at home for wireless and periodically have some random
issues which I can't explain, but for the most part have things tuned to ensure
there's little to no interference.

I am running TP-Link AP's. Two of them are Google OnHub, which was built by TP-Link, and the 3rd one is the TP-Link Archer C6.

So they all support 802.11ac, which is where my device spends most of its time (5GHz).

No interference from my neighbors (separated by thick walls), and I am running separate channels for both frequencies per device.

Also, no other wireless device is suffering like this.

  Do you see the same when hardwired? I keep many of my devices hardwired
to avoid odd jitter issues.

No issues on the wire at all. Quite perfect.

Like you, I hard-wire all fixed devices (TV's, a/v receivers, satellite decoders, gaming consoles, energy meters, e.t.c.). The only devices on wireless are mobile devices, tablets, laptops and the Windows PC which is hooked up via wi-fi as well.

   I also saw some older versions of the Pulse Secure
VPN add the behavior you describe, including the more uptime the slower it would
get.

I use Viscosity as an SSL/VPN client. The issue is the same whether it is enabled or offline.

Mark.

9

Might be worth disabling each AP to see if there’s one out there having an issue playing nice with the MacBook. Also try different combinations of two APs working together. It’s possible the MacBook is flip flopping because the power levels are fighting each other.

Does the Mac have this issue at your local coffee shop or another establishment with Wi-Fi? You can try to rule out the AirPort card in the Mac itself.

10

Might be worth disabling each AP to see if there's one out there having an issue playing nice with the MacBook. Also try different combinations of two APs working together. It's possible the MacBook is flip flopping because the power levels are fighting each other.

Tested all that, as well as dropping Tx power levels on each of the AP's to Low so that there isn't any power coming from any other AP (despite being quite far, already).

And to confirm, when the laptop locks into an AP, it doesn't try to join another one. When in range, power is very good (between -37dB and -52dB). When I walk away, that AP becomes too far (as bad as -80dB), but the next one close by is far better (same good values as before) and laptop connects and sticks to that.

Again, only impacts Catalina. No other Apple device, or the Windows PC that is on the same WLAN.

Does the Mac have this issue at your local coffee shop or another establishment with Wi-Fi? You can try to rule out the AirPort card in the Mac itself.

Never tried, I generally work from home. If I'm out, it's faster to tether to my 4G service rather than any public wi-fi.

Mark.

11

I believe I have seen the same thing with a Mid 2015 11,4 running catalina. Not diagnosing further because I could not find a reason for it fast enough and not sure if it really had an impact at the moment…. but could you try the following

sudo sysctl net.link.generic.system.hwcksum_tx=0
sudo sysctl net.link.generic.system.hwcksum_rx=0
sudo ifconfig en0 -rxcsum

in reverse … to restore the settings

sudo sysctl net.link.generic.system.hwcksum_tx=1
sudo sysctl net.link.generic.system.hwcksum_rx=1
sudo ifconfig en0 rxcsum

If you have some specific tests to run I would be willing to run them here on Big Sur with the same laptop but I have nothing now that runs Catalina

Wireshark used to in Catalina rack up cksum errors a lot while these were all at their defaults.

12

Should also state here that net.inet.icmp.icmplim=0 and the command I have been testing from is: (ping -c 5000 -i 0.1 router)

--- router ping statistics ---
5000 packets transmitted, 5000 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 1.118/4.060/172.031/6.841 ms

13

* nanog@nanog.org (J. Hellenthal via NANOG) [Thu 29 Oct 2020, 15:10 CET]:
[disabling checksum offload]

Wireshark used to in Catalina rack up cksum errors a lot while these were all at their defaults.

This is completely expected behaviour for outgoing packets.

  -- Niels.

14

I believe I have seen the same thing with a Mid 2015 11,4 running catalina. Not diagnosing further because I could not find a reason for it fast enough and not sure if it really had an impact at the moment…. but could you try the following

sudo sysctl net.link.generic.system.hwcksum_tx=0
sudo sysctl net.link.generic.system.hwcksum_rx=0
sudo ifconfig en0 -rxcsum

Thanks, I'll have a sniff.

If you have some specific tests to run I would be willing to run them here on Big Sur with the same laptop but I have nothing now that runs Catalina

One of my mates found the same issue on Big Sur (beta) on a 2013 MacBook Pro.

Just a simple mtr test to your local home router's IP address should, over wi-fi, should show you the jitter.

Mark.

15

On the latest Catalina 10.15.7 from a MacBook Air (early 2014) via WiFi to Google Wifi Mesh router (only a single unit network):

Over 2.4Ghz through 3 interior walls:
— 192.168.86.1 ping statistics —
100 packets transmitted, 100 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 1.440/15.078/213.538/30.541 ms

Over 5Ghz through 1 door:
— 192.168.86.1 ping statistics —
100 packets transmitted, 100 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 1.557/24.614/137.233/39.015 ms

This is a real world test with several iPhones, 2.4GHz WiFi only cameras, and an iPad being used for zoom (remote school) while running these tests. If you were not aware, on a Mac you can hold the Option key while clicking on the WiFi icon to see more options as well as more information about your current WiFi connection, including the realtime Tx rate, channel, and when your Mac is searching for new networks (mine does this every few seconds, it seems).

From an experience perspective, all applications seem to work fine for us. Including uploading to YouTube (a regular event as my spouse is a teacher) and Zoom/Teams/FaceTime/your teleconference app of choice (used by all of us).

–B

16

On the latest Catalina 10.15.7 from a MacBook Air (early 2014) via WiFi to Google Wifi Mesh router (only a single unit network):

Over 2.4Ghz through 3 interior walls:
--- 192.168.86.1 ping statistics ---
100 packets transmitted, 100 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 1.440/15.078/213.538/30.541 ms

Over 5Ghz through 1 door:
--- 192.168.86.1 ping statistics ---
100 packets transmitted, 100 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 1.557/24.614/137.233/39.015 ms

This is what I'm talking about. Why would you have such massive jitter on wi-fi to your local router?

You're hitting 213ms on 2.4GHz and 137ms on 5GHz. This doesn't seem odd to you?

Never had this issue pre-Catalina (I understand it may have been introduced in Mojave, but I came from High Sierra).

This is a real world test with several iPhones, 2.4GHz WiFi only cameras, and an iPad being used for zoom (remote school) while running these tests.

Same for me. There are tons of devices on my WLAN, and only the one running Catalina behaves this way.

No drama for the rest.

If you were not aware, on a Mac you can hold the Option key while clicking on the WiFi icon to see more options as well as more information about your current WiFi connection, including the realtime Tx rate, channel, and when your Mac is searching for new networks (mine does this every few seconds, it seems).

Yes, I am aware about the on-board macOS wi-fi diagnostics, and the good ol' trusted "airport -s". But those aren't enough.

I ended up spending money on NetSpot to see what's going on. Nothing there either. All I can tell is that Catalina is the problem; how exactly, is not yet very clear.

From an experience perspective, all applications seem to work fine for us. Including uploading to YouTube (a regular event as my spouse is a teacher) and Zoom/Teams/FaceTime/your teleconference app of choice (used by all of us).

So even though apps like Zoom report mild packet loss while on wi-fi (0.1% - 0.10%), it's not a train smash.

In general, everything works fine. But it doesn't feel 100%.

For the Youtube upload, I've found it to be temperamental, but I'm making it work for the time being.

Mark.

17

Hey Mark,
Good shout with debug, same issue seen on MacBook Air with Catalina 10.15.6 beta, pings upto 150ms seen
iMac with Sierra zero jitter and usually sub 1m pings
Now need to find out why, I never noticed as wife using the MacBook Air :frowning:
I cant yet update to big sur since need lots of sad space, need to cutdown on university docs me thinks

Col

18

Thanks, Col.

So confirmed that Big Sur has the same problem, both on newer and older Mac's.

Issue seems to have surfaced somewhere around Mojave.

Most folk from various fora suggested Location Services were to blame. I turned all of mine off, no joy.

A friend of mine in Malaysia noticed an improvement when he disabled Handoff between the Mac and all his other iCloud-enabled devices. But that didn't work for me.

Mark.

19

This does seem to be solved with the checksum disable below, or at least pings down to sub 10ms on Mac book air with Catalina beta 10.15.6, why aim performs far better I don’t know. I tried to introduce load after cksum disable and it did not see ping spikes as before

How do we now explain to Apple to fix ?

Col

20

Most folk from various fora suggested Location Services were to
blame. I turned all of mine off, no joy.

you only *think* you turned off location services. as they are a vital
component of providing a good user experience ...

:frowning: