io_uring/sqpoll: wait on sqd->wait for thread parking
Jens Axboe
maintainerPATCH 1/1 io_uring/sqpoll: wait on sqd->wait for thread parking
io_sqd_handle_event() just does a mutex unlock/lock dance when it's supposed to park, somewhat relying on full ordering with the thread trying to park it which does a similar unlock/lock dance on sqd->lock. However, with adaptive spinning on mutexes, this can waste an awful lot of time. Normally this isn't very noticeable, as parking and unparking the thread isn't a common (or fast path) occurence. However, in testing ring resizing, it's testing exactly that, as each resize will require the SQPOLL to safely park and unpark.
Have io_sq_thread_park() explicitly wait on sqd->park_pending being zero before attempting to grab the sqd->lock again.
In a resize test, this brings the runtime of SQPOLL down from about 60 seconds to a few seconds, just like the !SQPOLL tests. And saves a ton of spinning time on the mutex, on both sides.
---
io_uring/sqpoll.c +2 -1
@@ -40,6 +40,7 @@ void io_sq_thread_unpark(struct io_sq_data *sqd)if (atomic_dec_return (&sqd->park_pending))set_bit (IO_SQ_THREAD_SHOULD_PARK , &sqd->state);mutex_unlock (&sqd->lock); +wake_up (&sqd->wait); }void io_sq_thread_park (struct io_sq_data *sqd)
@@ -215,7 +216,7 @@ static bool io_sqd_handle_event(struct io_sq_data *sqd)mutex_unlock (&sqd->lock);if (signal_pending (current)) did_sig =get_signal (&ksig); -cond_resched (); +wait_event (sqd->wait, !atomic_read (&sqd->park_pending));mutex_lock (&sqd->lock); sqd->sq_cpu =raw_smp_processor_id (); }
Signature · 1 line
Jens Axboe