Performance degradation with "wifi: mac80211: Drop support for TX push path"

Quoted 3 lines · Maxime Bizon

Hello Alexander,

I would like to keep linux-wireless in the loop for discussions. When that's ok for you, just add [email protected] when you reply...

Quoted 15 lines

First I would like to thank you for the cleanups you are doing in mac80211, as a driver writer [1] I used to scratch my head to understand the internals.

I work for a french ISP called Free, and actively maintain the software of all router/gateway devices (couple millions deployed)

We have a very large number of old devices deployed that are >10 years old but on which we manage to run a mainline kernel. Those are ARM Kirkwook (armv5 1Ghz) based devices, wifi is Marvell 11n 2.4Ghz (mwl8k based) + QCA 11ac 5ghz (QCA998X/ath10k).

The CPU was always a bit underpowered for the 11ac card, but I could reach ~400Mbit/s in a routing test between WAN and Wifi, 100% CPU usage.

Quoted 7 lines

Since work started in 2019 on ath10k debloat, I applied that patch [2] to avoid performance degration.

Since "wifi: mac80211: Drop support for TX push path", it's not possible to do that anymore, so I did a benchmark to see the impact, the same routing test caps at 250Mbit/s. I suppose some openwrt users will be affected also.

Well, I guess calling wake_tx_queue and then transmitting (mostly) one packet is probably causing that. Now there are some ideas to improve that by using a kthreads. It's hidden in the discussion here:
https://lore.kernel.org/all/[email protected]/

My problem with doing that *now* is, that I'm working on a invasive patch set in mac80211. Which will really clean up the old logic and not just tweak some simple things.
It's mostly sorted out and written, only a few issue left till I can try it out. But I'm not getting around to wrap that up for work/private reasons for quite some weeks (really months...) now. Rebasing and then testing the already working patches for a wake_tx_queue implementation using kthreads is something I'm hoping to avoid:-)

Now, I've more or less already decided to add the kthread patch to that series, once I get it stabilized at its current level. I have some hopes, that I can reuse the reworked PS mechanism to even simplify such an patch.

But it's hard to even estimate when I'll have that patch set together, so I would have something for you to test. (Should at least be this year. Originally I was aiming for 6.4. But that was weeks ago and I've found no time to work on that till then...)

Getting the push path operational in >= 6.2 will be non-trival. You basically would have to revert drivers/mac80211 to 6.1 and maintain your own tree. Looks like a very bad idea...

When you need a fix in the near future and you have time to work on that I would look into using a kthread for wake_tx_queue. Seems to be the most promising way to solve the issue.. (Maybe I'm to optimistic, but that could even be faster...)

Of course some people on the list may have other - simpler to implement?
- ideas why it's so much slower for you to use wake_tx_queue.

Quoted 6 lines

I don't want to hinder progress of course, and I can still revert it (for now at least), but if you had an idea to keep a simpler alternative tx path without too much maintenance burden, I'd be willing to put the effort into it.

I don't think that we'll ever bring the push path back.
We'll have to figure out how to change wake_tx_queue, so you get the same - or better - performance.

Quoted 5 lines

Thanks,

[1] https://marc.info/?l=linux-wireless&m=158862152015048&w=2
[2] https://patchwork.kernel.org/project/ath10k/patch/[email protected]/

Quoted 23 lines

Quoted 22 lines

First I would like to thank you for the cleanups you are doing in mac80211, as a driver writer [1] I used to scratch my head to understand the internals.

I work for a french ISP called Free, and actively maintain the software of all router/gateway devices (couple millions deployed)

We have a very large number of old devices deployed that are >10 years old but on which we manage to run a mainline kernel. Those are ARM Kirkwook (armv5 1Ghz) based devices, wifi is Marvell 11n 2.4Ghz (mwl8k based) + QCA 11ac 5ghz (QCA998X/ath10k).

The CPU was always a bit underpowered for the 11ac card, but I could reach ~400Mbit/s in a routing test between WAN and Wifi, 100% CPU usage.

Since work started in 2019 on ath10k debloat, I applied that patch [2] to avoid performance degration.

Since "wifi: mac80211: Drop support for TX push path", it's not possible to do that anymore, so I did a benchmark to see the impact, the same routing test caps at 250Mbit/s. I suppose some openwrt users will be affected also.

Quoted 8 lines

Well, I guess calling wake_tx_queue and then transmitting (mostly) one packet is probably causing that. Now there are some ideas to improve that by using a kthreads. It's hidden in the discussion here:
https://lore.kernel.org/all/82d5623b-8d21-a8c1-e835- [email protected]/

My problem with doing that *now* is, that I'm working on a invasive patch set in mac80211. Which will really clean up the old logic and not just tweak some simple things.
It's mostly sorted out and written, only a few issue left till I can try it out. But I'm not getting around to wrap that up for work/ private reasons for quite some weeks (really months...) now. Rebasing and then testing the already working patches for a wake_tx_queue implementation using kthreads is something I'm hoping to avoid:-)

Now, I've more or less already decided to add the kthread patch to that series, once I get it stabilized at its current level. I have some hopes, that I can reuse the reworked PS mechanism to even simplify such an patch.