io_uring: consider cgroup setting when binding sqpoll cpu
PATCH 1/1 io_uring: consider cgroup setting when binding sqpoll cpu
Since sqthread is userspace like thread now, it should respect cgroup setting, thus we should consider current allowed cpuset when doing cpu binding for sqthread.
--- fs/io_uring.c | 41 +++++++++++++++++++++++++++++++++++++---- 1 file changed, 37 insertions(+), 4 deletions(-)
fs/io_uring.c +37 -4
@@ -79,6 +79,7 @@#include <linux/pagemap.h> #include <linux/io_uring.h> #include <linux/tracehook.h> +#include <linux/cpuset.h> #define CREATE_TRACE_POINTS #include <trace/events/io_uring.h>
@@ -6903,15 +6904,35 @@ static int io_sq_thread(void *data)struct io_ring_ctx *ctx;unsigned long timeout =0 ;char buf[TASK_COMM_LEN ]; +cpumask_var_t cpus_allowed;DEFINE_WAIT (wait);snprintf (buf,sizeof (buf),"iou-sqp-%d" , sqd->task_pid);set_task_comm (current, buf); -if (sqd->sq_cpu !=-1 ) -set_cpus_allowed_ptr (current,cpumask_of (sqd->sq_cpu)); -else -set_cpus_allowed_ptr (current, cpu_online_mask); +if (!alloc_cpumask_var (&cpus_allowed,GFP_KERNEL )) { +mutex_lock (&sqd->lock); + sqd->thread =NULL ; +list_for_each_entry (ctx, &sqd->ctx_list, sqd_list) +io_ring_set_wakeup_flag (ctx); +io_run_task_work (); +mutex_unlock (&sqd->lock); + +complete (&sqd->exited); +do_exit (1 ); + } +cpuset_cpus_allowed (current, cpus_allowed); +if (sqd->sq_cpu !=-1 ) { +if (!cpumask_test_cpu (sqd->sq_cpu, cpus_allowed)) { + sqd->sq_cpu =-1 ; +pr_warn ("sqthread %d: bound cpu not allowed\n" , current->pid); + }else { +set_cpus_allowed_ptr (current,cpumask_of (sqd->sq_cpu)); + } + } +if (sqd->sq_cpu ==-1 ) +set_cpus_allowed_ptr (current, cpus_allowed); +free_cpumask_var (cpus_allowed); current->flags |=PF_NO_SETAFFINITY ;mutex_lock (&sqd->lock);
@@ -8060,11 +8081,23 @@ static int io_sq_offload_create(struct io_ring_ctx *ctx,if (p->flags &IORING_SETUP_SQ_AFF ) {int cpu = p->sq_thread_cpu; +cpumask_var_t cpus_allowed; ret = -EINVAL ;if (cpu >= nr_cpu_ids || !cpu_online (cpu)) goto err_sqpoll; +if (!alloc_cpumask_var (&cpus_allowed,GFP_KERNEL )) { + ret = -ENOMEM ; + goto err_sqpoll; + } +cpuset_cpus_allowed (current, cpus_allowed); +if (!cpumask_test_cpu (cpu, cpus_allowed)) { +free_cpumask_var (cpus_allowed); + goto err_sqpoll; + } + sqd->sq_cpu = cpu; +free_cpumask_var (cpus_allowed); }else { sqd->sq_cpu =-1 ; }
Signature · 1 line
2.24.4
Jens Axboe
maintainerQuoted 3 lines · Hao Xu
Since sqthread is userspace like thread now, it should respect cgroup setting, thus we should consider current allowed cpuset when doing cpu binding for sqthread.
This seems a bit convoluted for what it needs to do. Surely we can just test sqd->sq_cpu directly in the task_cs()?
Signature · 1 line
Jens Axboe
Quoted 7 lines · Jens Axboe
Quoted 3 lines · Hao Xu
Since sqthread is userspace like thread now, it should respect cgroup setting, thus we should consider current allowed cpuset when doing cpu binding for sqthread.
This seems a bit convoluted for what it needs to do. Surely we can just test
sqd->sq_cpudirectly in thetask_cs()?
I didn't know task_cs() before, it seems to be a static function, which is called by cpuset_cpus_allowed(), and this one is exposed.
Quoted 1 line
Jens Axboe
maintainerQuoted 10 lines · Hao Xu
Quoted 7 lines · Jens Axboe
Quoted 3 lines · Hao Xu
Since sqthread is userspace like thread now, it should respect cgroup setting, thus we should consider current allowed cpuset when doing cpu binding for sqthread.
This seems a bit convoluted for what it needs to do. Surely we can just test
sqd->sq_cpudirectly in thetask_cs()?I didn't know
task_cs()before, it seems to be a static function, which is called bycpuset_cpus_allowed(), and this one is exposed.
But it'd be a much saner to add a helper for this rather than add all of that boiler plate code to io_uring just to check for whether or not a CPU is set in a mask.
Signature · 1 line
Jens Axboe
Quoted 7 lines · Jens Axboe
Quoted 6 lines · Hao Xu
Quoted 5 lines · Jens Axboe
Quoted 3 lines · Hao Xu
Since sqthread is userspace like thread now, it should respect cgroup setting, thus we should consider current allowed cpuset when doing cpu binding for sqthread.
Quoted 8 lines
Quoted 4 lines
Quoted 2 lines
This seems a bit convoluted for what it needs to do. Surely we can just test
sqd->sq_cpudirectly in thetask_cs()?I didn't know
task_cs()before, it seems to be a static function, which is called bycpuset_cpus_allowed(), and this one is exposed.But it'd be a much saner to add a helper for this rather than add all of that boiler plate code to
io_uringjust to check for whether or not a CPU is set in a mask.
I looked into cpuset_cpus_allowed(), it seems the code logic guarantee_online_cpus() is neccessary? It guarantees the cpumask returned are all online cpus.
Quoted 1 line
Quoted 39 lines · Hao Xu
Since sqthread is userspace like thread now, it should respect cgroup setting, thus we should consider current allowed cpuset when doing cpu binding for sqthread.
Signed-off-by Hao Xu <[email protected]>--- fs/io_uring.c | 41 +++++++++++++++++++++++++++++++++++++---- 1 file changed, 37 insertions(+), 4 deletions(-)fs/io_uring.c +6
-4@@ -79,6 +79,7 @@#include <linux/pagemap.h> #include <linux/io_uring.h> #include <linux/tracehook.h> +#include <linux/cpuset.h> #define CREATE_TRACE_POINTS #include <trace/events/io_uring.h> @@ -6903,15 +6904,35 @@ static int io_sq_thread(void *data)struct io_ring_ctx *ctx;unsigned long timeout =0 ;char buf[TASK_COMM_LEN ]; +cpumask_var_t cpus_allowed;DEFINE_WAIT (wait);snprintf (buf,sizeof (buf),"iou-sqp-%d" , sqd->task_pid);set_task_comm (current, buf); -if (sqd->sq_cpu !=-1 ) -set_cpus_allowed_ptr (current,cpumask_of (sqd->sq_cpu)); -else -set_cpus_allowed_ptr (current, cpu_online_mask); +if (!alloc_cpumask_var (&cpus_allowed,GFP_KERNEL )) { +mutex_lock (&sqd->lock); + sqd->thread =NULL ; +list_for_each_entry (ctx, &sqd->ctx_list, sqd_list)
fs/io_uring.c static int io_sq_thread(void *data) · line 6926
Quoted 10 lines
fs/io_uring.c +10
-0+io_ring_set_wakeup_flag (ctx); +io_run_task_work (); +mutex_unlock (&sqd->lock); + +complete (&sqd->exited); +do_exit (1 ); + } +cpuset_cpus_allowed (current, cpus_allowed); +if (sqd->sq_cpu !=-1 ) { +if (!cpumask_test_cpu (sqd->sq_cpu, cpus_allowed)) {
Here comes a small race with cpus_allowed updating. If this is the case, it means the cpus_allowed are changing frequently, in this situation sqthread's cpumask will be setup to cpus_allowed anyway, so it's still in the correct range.
(considering the aim of this patch is to limit sqthread to the cpuset where its cgroup allows. If cpuset of its cgroup is changing frequently we cannot bind sqthread to some cpu anyway.)
Quoted 37 lines
fs/io_uring.c +21
-0+ sqd->sq_cpu =-1 ; +pr_warn ("sqthread %d: bound cpu not allowed\n" , current->pid); + }else { +set_cpus_allowed_ptr (current,cpumask_of (sqd->sq_cpu)); + } + } +if (sqd->sq_cpu ==-1 ) +set_cpus_allowed_ptr (current, cpus_allowed); +free_cpumask_var (cpus_allowed); current->flags |=PF_NO_SETAFFINITY ;mutex_lock (&sqd->lock);@@ -8060,11 +8081,23 @@ static int io_sq_offload_create(struct io_ring_ctx *ctx,if (p->flags &IORING_SETUP_SQ_AFF ) {int cpu = p->sq_thread_cpu; +cpumask_var_t cpus_allowed; ret = -EINVAL ;if (cpu >= nr_cpu_ids || !cpu_online (cpu)) goto err_sqpoll; +if (!alloc_cpumask_var (&cpus_allowed,GFP_KERNEL )) { + ret = -ENOMEM ; + goto err_sqpoll; + } +cpuset_cpus_allowed (current, cpus_allowed); +if (!cpumask_test_cpu (cpu, cpus_allowed)) { +free_cpumask_var (cpus_allowed); + goto err_sqpoll; + } + sqd->sq_cpu = cpu; +free_cpumask_var (cpus_allowed); }else { sqd->sq_cpu =-1 ; }