Skip to content

io-wq locking improvements

Jens Axboe io-uring 1 version In review

Get series with b4
b4 am [email protected]

Versions newest first

  1. v1 Latest3 patches 2 repliesIn review

Patches in v1 No applies recorded yet

#Subject and filesSizeRepliesApplied
1/3io_uring/io-wq: don't grab wq->lock for worker activationio_uring/io-wq.c+0 -30 replies
2/3io_uring/io-wq: reduce frequency of acct->lock acquisitionsio_uring/io-wq.c+34 -130 replies
3/3io_uring/io-wq: don't gate worker wake up success on wake_up_process()io_uring/io-wq.c+7 -40 replies

Reviews trailers per patch

None of the 3 patches has a Reviewed-by or Acked-by of their own; the series has one for all of them.

PatchHao Xu
SeriesR-b
1/3
2/3
3/3

Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes.

Read the v1 thread

Cover letter [PATCHSET 0/3] io-wq locking improvements

Hi,

In chatting with someone that was trying to use io_uring to read mailddirs, they found that running a test case that does:

open file, statx file, read file, close file

The culprit here is statx, and argumentation aside on whether it makes sense to statx in the first place, it does highlight that io-wq is pretty locking intensive.

This (very lightly tested [1]) patchset attempts to improve this situation, but reducing the frequency of grabbing wq->lock and acct->lock.

The first patch gets rid of wq->lock on work insertion. io-wq grabs it to iterate the free worker list, but that is not necessary.

Second patch reduces the frequency of acct->lock grabs, when we need to run the queue and process new work. We currently grab the lock and check for work, then drop it, then grab it again to process the work. That is unneccessary.

Final patch just optimizes how we activate new workers. It's not related to the locking itself, just reducing the overhead of activating a new worker.

Running the above test case on a directory with 50K files, each being between 10 and 4096 bytes, before these patches we get spend 160-170ms running the workload. With this patchset, we spend 90-100ms doing the same work. A bit of profile information is included in the patch commit messages.

Can also be found here:

https://git.kernel.dk/cgit/linux/log/?h=io_uring-wq-lock

[1] Runs the test suite just fine, with PROVE_LOCKING enabled and raw lockdep as well.

Signature · 1 line
Jens Axboe