io_uring: gate iowait schedule on having pending requests
Jens Axboe
maintainerPATCH 1/1 io_uring: gate iowait schedule on having pending requests
A previous commit made all cqring waits marked as iowait, as a way to improve performance for short schedules with pending IO. However, for use cases that have a special reaper thread that does nothing but wait on events on the ring, this causes a cosmetic issue where we know have one core marked as being "busy" with 100% iowait.
While this isn't a grave issue, it is confusing to users. Rather than always mark us as being in iowait, gate setting of current->in_iowait to 1 by whether or not the waiting task has pending requests.
Link https://lore.kernel.org/io-uring/CAMEGJJ2RxopfNQ7GNLhr7X9=bHXKo+G5OOe0LUq=+UgLXsv1Xg@mail.gmail.com/ open here
Reported-by Oleksandr Natalenko <[email protected]>
Reported-by Phil Elwell <[email protected]>
Tested-by Andres Freund <[email protected]>
Fixes 8a796565cec3 ("io_uring: Use io_schedule* in cqring wait")
Signed-off-by Jens Axboe <[email protected]>
---
io_uring/io_uring.c +17 -6
@@ -2493,11 +2493,20 @@ int io_run_task_work_sig(struct io_ring_ctx *ctx)return 0 ; } +static bool current_pending_io (void ) +{ +struct io_uring_task *tctx = current->io_uring; + +if (!tctx) +return false; +return percpu_counter_read_positive (&tctx->inflight); +} +/* when returns >0, the caller should retry */ static inline int io_cqring_wait_schedule (struct io_ring_ctx *ctx,struct io_wait_queue *iowq) { -int token, ret; +int io_wait, ret;if (unlikely (READ_ONCE (ctx->check_cq)))return 1 ;
@@ -2511,17 +2520,19 @@ static inline int io_cqring_wait_schedule(struct io_ring_ctx *ctx,return 0 ;/* -* Use io_schedule_prepare/finish, so cpufreq can take into account -* that the task is waiting for IO - turns out to be important for low -* QD IO. +* Mark us as being in io_wait if we have pending requests, so cpufreq +* can take into account that the task is waiting for IO - turns out +* to be important for low QD IO. */ - token =io_schedule_prepare (); + io_wait = current->in_iowait; +if (current_pending_io ()) + current->in_iowait =1 ; ret =0 ;if (iowq->timeout ==KTIME_MAX )schedule ();else if (!schedule_hrtimeout (&iowq->timeout,HRTIMER_MODE_ABS )) ret = -ETIME ; -io_schedule_finish (token); + current->in_iowait = io_wait;return ret; }
Signature · 1 line
Jens Axboe