io_uring/fdinfo: park SQ thread while retrieving cpu/pid
Jens Axboe
maintainerPATCH 1/1 io_uring/fdinfo: park SQ thread while retrieving cpu/pid
We could race with SQ thread exit, and if we do, we'll hit a NULL pointer dereference. Park the SQPOLL thread while getting the task cpu and pid for fdinfo, this ensures we have a stable view of it.
---
io_uring/fdinfo.c +11 -6
@@ -53,7 +53,6 @@ static __cold int io_uring_show_cred(struct seq_file *m, unsigned int id,__cold void io_uring_show_fdinfo (struct seq_file *m,struct file *f) {struct io_ring_ctx *ctx = f->private_data; -struct io_sq_data *sq =NULL ;struct io_overflow_cqe *ocqe;struct io_rings *r = ctx->rings;unsigned int sq_mask = ctx->sq_entries -1 , cq_mask = ctx->cq_entries -1 ;
@@ -64,6 +63,7 @@ __cold void io_uring_show_fdinfo(struct seq_file *m, struct file *f)unsigned int cq_shift =0 ;unsigned int sq_shift =0 ;unsigned int sq_entries, cq_entries; +int sq_pid =-1 , sq_cpu =-1 ;bool has_lock;unsigned int i;
@@ -143,13 +143,18 @@ __cold void io_uring_show_fdinfo(struct seq_file *m, struct file *f) has_lock =mutex_trylock (&ctx->uring_lock);if (has_lock && (ctx->flags &IORING_SETUP_SQPOLL )) { - sq = ctx->sq_data; -if (!sq->thread) - sq =NULL ; +struct io_sq_data *sq = ctx->sq_data; + +io_sq_thread_park (sq); +if (sq->thread) { + sq_pid =task_pid_nr (sq->thread); + sq_cpu =task_cpu (sq->thread); + } +io_sq_thread_unpark (sq); } -seq_printf (m,"SqThread:\t%d\n" , sq ?task_pid_nr (sq->thread) :-1 ); -seq_printf (m,"SqThreadCpu:\t%d\n" , sq ?task_cpu (sq->thread) :-1 ); +seq_printf (m,"SqThread:\t%d\n" , sq_pid); +seq_printf (m,"SqThreadCpu:\t%d\n" , sq_cpu);seq_printf (m,"UserFiles:\t%u\n" , ctx->nr_user_files);for (i =0 ; has_lock && i < ctx->nr_user_files; i++) {struct file *f =io_file_from_index (&ctx->file_table, i);
Signature · 1 line
Jens Axboe
A message that is not in the archive (sent off-list or not yet received): [email protected]
Jens Axboe
maintainerQuoted 55 lines · Gabriel Krisman Bertazi
Quoted 35 lines · Jens Axboe
We could race with SQ thread exit, and if we do, we'll hit a NULL pointer dereference. Park the SQPOLL thread while getting the task cpu and pid for fdinfo, this ensures we have a stable view of it.
Signed-off-by Jens Axboe <[email protected]>---
io_uring/fdinfo.c +1
-2@@ -53,7 +53,6 @@ static __cold int io_uring_show_cred(struct seq_file *m, unsigned int id,__cold void io_uring_show_fdinfo (struct seq_file *m,struct file *f) {struct io_ring_ctx *ctx = f->private_data; -struct io_sq_data *sq =NULL ;struct io_overflow_cqe *ocqe;struct io_rings *r = ctx->rings;unsigned int sq_mask = ctx->sq_entries -1 , cq_mask = ctx->cq_entries -1 ;@@ -64,6 +63,7 @@ __cold void io_uring_show_fdinfo(struct seq_file *m, struct file *f)unsigned int cq_shift =0 ;unsigned int sq_shift =0 ;unsigned int sq_entries, cq_entries; +int sq_pid =-1 , sq_cpu =-1 ;bool has_lock;unsigned int i;@@ -143,13 +143,18 @@ __cold void io_uring_show_fdinfo(struct seq_file *m, struct file *f) has_lock =mutex_trylock (&ctx->uring_lock);if (has_lock && (ctx->flags &IORING_SETUP_SQPOLL )) { - sq = ctx->sq_data;io_uring/fdinfo.c __cold void io_uring_show_fdinfo(struct seq_file *m, struct file *f) · line 153
Quoted 10 lines
io_uring/fdinfo.c +8
-2-if (!sq->thread) - sq =NULL ; +struct io_sq_data *sq = ctx->sq_data; + +io_sq_thread_park (sq); +if (sq->thread) { + sq_pid =task_pid_nr (sq->thread); + sq_cpu =task_cpu (sq->thread); + } +io_sq_thread_unpark (sq);Jens,
io_sq_thread_parkwill try to wake the sqpoll, which is, at least, unnecessary. But I'm thinking we don't want to expose the ability to schedule the sqpoll from procfs, which can be done by any unrelated process.
Quoted 5 lines
To solve the bug, it should be enough to synchronize directly on
sqd->lock, preventingsq->threadfrom going away inside the if leg. Granted, it is might take longer if the sqpoll is busy, but reading fdinfo is not supposed to be fast. Alternatively, don't callwake_processin this case?
I did think about that but just went with the exported API. But you are right, it's a bit annoying that it'd also wake the thread, in case it was idle. Probably mostly cosmetic, but we may as well just stick with grabbing the sqd mutex. I'll send a v2.
Signature · 1 line
Jens Axboe
Quoted 56 lines · Jens Axboe
Quoted 55 lines · Gabriel Krisman Bertazi
Quoted 35 lines · Jens Axboe
We could race with SQ thread exit, and if we do, we'll hit a NULL pointer dereference. Park the SQPOLL thread while getting the task cpu and pid for fdinfo, this ensures we have a stable view of it.
Signed-off-by Jens Axboe <[email protected]>---
io_uring/fdinfo.c +1
-2@@ -53,7 +53,6 @@ static __cold int io_uring_show_cred(struct seq_file *m, unsigned int id,__cold void io_uring_show_fdinfo (struct seq_file *m,struct file *f) {struct io_ring_ctx *ctx = f->private_data; -struct io_sq_data *sq =NULL ;struct io_overflow_cqe *ocqe;struct io_rings *r = ctx->rings;unsigned int sq_mask = ctx->sq_entries -1 , cq_mask = ctx->cq_entries -1 ;@@ -64,6 +63,7 @@ __cold void io_uring_show_fdinfo(struct seq_file *m, struct file *f)unsigned int cq_shift =0 ;unsigned int sq_shift =0 ;unsigned int sq_entries, cq_entries; +int sq_pid =-1 , sq_cpu =-1 ;bool has_lock;unsigned int i;@@ -143,13 +143,18 @@ __cold void io_uring_show_fdinfo(struct seq_file *m, struct file *f) has_lock =mutex_trylock (&ctx->uring_lock);if (has_lock && (ctx->flags &IORING_SETUP_SQPOLL )) { - sq = ctx->sq_data;io_uring/fdinfo.c __cold void io_uring_show_fdinfo(struct seq_file *m, struct file *f) · line 153
Quoted 10 lines
io_uring/fdinfo.c +8
-2-if (!sq->thread) - sq =NULL ; +struct io_sq_data *sq = ctx->sq_data; + +io_sq_thread_park (sq); +if (sq->thread) { + sq_pid =task_pid_nr (sq->thread); + sq_cpu =task_cpu (sq->thread); + } +io_sq_thread_unpark (sq);Jens,
io_sq_thread_parkwill try to wake the sqpoll, which is, at least, unnecessary. But I'm thinking we don't want to expose the ability to schedule the sqpoll from procfs, which can be done by any unrelated process.
Quoted 8 lines
Quoted 5 lines
To solve the bug, it should be enough to synchronize directly on
sqd->lock, preventingsq->threadfrom going away inside the if leg. Granted, it is might take longer if the sqpoll is busy, but reading fdinfo is not supposed to be fast. Alternatively, don't callwake_processin this case?I did think about that but just went with the exported API. But you are right, it's a bit annoying that it'd also wake the thread, in case it
Waking it up is not a problem but without parking sq thread won't drop the lock until it's time to sleep, which might be pretty long leaving the /proc read stuck on the lock uninterruptibly.
Aside from parking vs lock, there is a lock inversion now:
proc read | SQPOLL
|
try_lock(ring) // success |
| woken up
| lock(sqd); // success
lock(sqd); // stuck |
| try to submit requests
| -- lock(ring); // stuckQuoted 2 lines
was idle. Probably mostly cosmetic, but we may as well just stick with grabbing the sqd mutex. I'll send a v2.
Signature · 1 line
Pavel Begunkov
Jens Axboe
maintainerQuoted 70 lines · Pavel Begunkov
Quoted 54 lines · Jens Axboe
Quoted 53 lines · Gabriel Krisman Bertazi
Quoted 33 lines · Jens Axboe
We could race with SQ thread exit, and if we do, we'll hit a NULL pointer dereference. Park the SQPOLL thread while getting the task cpu and pid for fdinfo, this ensures we have a stable view of it.
Signed-off-by Jens Axboe <[email protected]>---
io_uring/fdinfo.c +1
-1@@ -53,7 +53,6 @@ static __cold int io_uring_show_cred(struct seq_file *m, unsigned int id,__cold void io_uring_show_fdinfo (struct seq_file *m,struct file *f) {struct io_ring_ctx *ctx = f->private_data; -struct io_sq_data *sq =NULL ;struct io_overflow_cqe *ocqe;struct io_rings *r = ctx->rings;unsigned int sq_mask = ctx->sq_entries -1 , cq_mask = ctx->cq_entries -1 ;@@ -64,6 +63,7 @@ __cold void io_uring_show_fdinfo(struct seq_file *m, struct file *f)unsigned int cq_shift =0 ;unsigned int sq_shift =0 ;unsigned int sq_entries, cq_entries; +int sq_pid =-1 , sq_cpu =-1 ;bool has_lock;unsigned int i; @@ -143,13 +143,18 @@ __cold void io_uring_show_fdinfo(struct seq_file *m, struct file *f)has_lock = mutex_trylock(&ctx->uring_lock); if (has_lock && (ctx->flags & IORING_SETUP_SQPOLL)) {io_uring/fdinfo.c +0
-1- sq = ctx->sq_data;io_uring/fdinfo.c
Quoted 10 lines
io_uring/fdinfo.c +8
-2-if (!sq->thread) - sq =NULL ; +struct io_sq_data *sq = ctx->sq_data; + +io_sq_thread_park (sq); +if (sq->thread) { + sq_pid =task_pid_nr (sq->thread); + sq_cpu =task_cpu (sq->thread); + } +io_sq_thread_unpark (sq);Jens,
io_sq_thread_parkwill try to wake the sqpoll, which is, at least, unnecessary. But I'm thinking we don't want to expose the ability to schedule the sqpoll from procfs, which can be done by any unrelated process.Quoted 8 lines
Quoted 5 lines
To solve the bug, it should be enough to synchronize directly on
sqd->lock, preventingsq->threadfrom going away inside the if leg. Granted, it is might take longer if the sqpoll is busy, but reading fdinfo is not supposed to be fast. Alternatively, don't callwake_processin this case?I did think about that but just went with the exported API. But you are right, it's a bit annoying that it'd also wake the thread, in case it
Waking it up is not a problem but without parking sq thread won't drop the lock until it's time to sleep, which might be pretty long leaving the /proc read stuck on the lock uninterruptibly.
Aside from parking vs lock, there is a lock inversion now:
Quoted 8 lines
proc read | SQPOLL
|
try_lock(ring) // success |
| woken up
| lock(sqd); // success
lock(sqd); // stuck |
| try to submit requests
| -- lock(ring); // stuckYeah good point, forgot we nest these opposite of what you'd expect. Honestly I think the fix here is just to turn it into a trylock. Yes that'll miss some cases where we could've gotten the pid/cpu, but doesn't seem worth caring about.
IOW, fold in this incremental.
io_uring/fdinfo.c +6 -5
@@ -145,12 +145,13 @@ __cold void io_uring_show_fdinfo(struct seq_file *m, struct file *f)if (has_lock && (ctx->flags &IORING_SETUP_SQPOLL )) {struct io_sq_data *sq = ctx->sq_data; -mutex_lock (&sq->lock); -if (sq->thread) { - sq_pid =task_pid_nr (sq->thread); - sq_cpu =task_cpu (sq->thread); +if (mutex_trylock (&sq->lock)) { +if (sq->thread) { + sq_pid =task_pid_nr (sq->thread); + sq_cpu =task_cpu (sq->thread); + } +mutex_unlock (&sq->lock); } -mutex_unlock (&sq->lock); }seq_printf (m,"SqThread:\t%d\n" , sq_pid);
Signature · 1 line
Jens Axboe
Quoted 80 lines · Jens Axboe
Quoted 70 lines · Pavel Begunkov
Quoted 54 lines · Jens Axboe
Quoted 53 lines · Gabriel Krisman Bertazi
Quoted 33 lines · Jens Axboe
We could race with SQ thread exit, and if we do, we'll hit a NULL pointer dereference. Park the SQPOLL thread while getting the task cpu and pid for fdinfo, this ensures we have a stable view of it.
Signed-off-by Jens Axboe <[email protected]>---
io_uring/fdinfo.c +1
-1@@ -53,7 +53,6 @@ static __cold int io_uring_show_cred(struct seq_file *m, unsigned int id,__cold void io_uring_show_fdinfo (struct seq_file *m,struct file *f) {struct io_ring_ctx *ctx = f->private_data; -struct io_sq_data *sq =NULL ;struct io_overflow_cqe *ocqe;struct io_rings *r = ctx->rings;unsigned int sq_mask = ctx->sq_entries -1 , cq_mask = ctx->cq_entries -1 ;@@ -64,6 +63,7 @@ __cold void io_uring_show_fdinfo(struct seq_file *m, struct file *f)unsigned int cq_shift =0 ;unsigned int sq_shift =0 ;unsigned int sq_entries, cq_entries; +int sq_pid =-1 , sq_cpu =-1 ;bool has_lock;unsigned int i; @@ -143,13 +143,18 @@ __cold void io_uring_show_fdinfo(struct seq_file *m, struct file *f)has_lock = mutex_trylock(&ctx->uring_lock); if (has_lock && (ctx->flags & IORING_SETUP_SQPOLL)) {io_uring/fdinfo.c +0
-1- sq = ctx->sq_data;io_uring/fdinfo.c
Quoted 10 lines
io_uring/fdinfo.c +8
-2-if (!sq->thread) - sq =NULL ; +struct io_sq_data *sq = ctx->sq_data; + +io_sq_thread_park (sq); +if (sq->thread) { + sq_pid =task_pid_nr (sq->thread); + sq_cpu =task_cpu (sq->thread); + } +io_sq_thread_unpark (sq);Jens,
io_sq_thread_parkwill try to wake the sqpoll, which is, at least, unnecessary. But I'm thinking we don't want to expose the ability to schedule the sqpoll from procfs, which can be done by any unrelated process.Quoted 8 lines
Quoted 5 lines
To solve the bug, it should be enough to synchronize directly on
sqd->lock, preventingsq->threadfrom going away inside the if leg. Granted, it is might take longer if the sqpoll is busy, but reading fdinfo is not supposed to be fast. Alternatively, don't callwake_processin this case?I did think about that but just went with the exported API. But you are right, it's a bit annoying that it'd also wake the thread, in case it
Waking it up is not a problem but without parking sq thread won't drop the lock until it's time to sleep, which might be pretty long leaving the /proc read stuck on the lock uninterruptibly.
Aside from parking vs lock, there is a lock inversion now:
Quoted 8 lines
proc read | SQPOLL
|
try_lock(ring)// success |
| woken up
| lock(sqd); // success
lock(sqd); // stuck |
| try to submit requests | -- lock(ring); // stuck
Quoted 6 lines
Yeah good point, forgot we nest these opposite of what you'd expect. Honestly I think the fix here is just to turn it into a trylock. Yes that'll miss some cases where we could've gotten the pid/cpu, but doesn't seem worth caring about.
IOW, fold in this incremental.
Should work, otherwise you probably can just park first.
Long term it'd be nice to make sqpoll to not hold sqd->lock during submission + polling as it currently does. Park callers sleep on the lock, but if we replace it with some struct completion the rest should be easy enough.
Quoted 24 lines
io_uring/fdinfo.c +6
-5@@ -145,12 +145,13 @@ __cold void io_uring_show_fdinfo(struct seq_file *m, struct file *f)if (has_lock && (ctx->flags &IORING_SETUP_SQPOLL )) {struct io_sq_data *sq = ctx->sq_data; -mutex_lock (&sq->lock); -if (sq->thread) { - sq_pid =task_pid_nr (sq->thread); - sq_cpu =task_cpu (sq->thread); +if (mutex_trylock (&sq->lock)) { +if (sq->thread) { + sq_pid =task_pid_nr (sq->thread); + sq_cpu =task_cpu (sq->thread); + } +mutex_unlock (&sq->lock); } -mutex_unlock (&sq->lock); }seq_printf (m,"SqThread:\t%d\n" , sq_pid);
Signature · 1 line
Pavel Begunkov
Jens Axboe
maintainerQuoted 81 lines · Pavel Begunkov
Quoted 80 lines · Jens Axboe
Quoted 78 lines · Pavel Begunkov
Quoted 54 lines · Jens Axboe
Quoted 53 lines · Gabriel Krisman Bertazi
Quoted 33 lines · Jens Axboe
We could race with SQ thread exit, and if we do, we'll hit a NULL pointer dereference. Park the SQPOLL thread while getting the task cpu and pid for fdinfo, this ensures we have a stable view of it.
Signed-off-by Jens Axboe <[email protected]>---
io_uring/fdinfo.c +1
-1@@ -53,7 +53,6 @@ static __cold int io_uring_show_cred(struct seq_file *m, unsigned int id,__cold void io_uring_show_fdinfo (struct seq_file *m,struct file *f) {struct io_ring_ctx *ctx = f->private_data; -struct io_sq_data *sq =NULL ;struct io_overflow_cqe *ocqe;struct io_rings *r = ctx->rings;unsigned int sq_mask = ctx->sq_entries -1 , cq_mask = ctx->cq_entries -1 ;@@ -64,6 +63,7 @@ __cold void io_uring_show_fdinfo(struct seq_file *m, struct file *f)unsigned int cq_shift =0 ;unsigned int sq_shift =0 ;unsigned int sq_entries, cq_entries; +int sq_pid =-1 , sq_cpu =-1 ;bool has_lock;unsigned int i; @@ -143,13 +143,18 @@ __cold void io_uring_show_fdinfo(struct seq_file *m, struct file *f)has_lock = mutex_trylock(&ctx->uring_lock); if (has_lock && (ctx->flags & IORING_SETUP_SQPOLL)) {io_uring/fdinfo.c +0
-1- sq = ctx->sq_data;io_uring/fdinfo.c
Quoted 10 lines
io_uring/fdinfo.c +8
-2-if (!sq->thread) - sq =NULL ; +struct io_sq_data *sq = ctx->sq_data; + +io_sq_thread_park (sq); +if (sq->thread) { + sq_pid =task_pid_nr (sq->thread); + sq_cpu =task_cpu (sq->thread); + } +io_sq_thread_unpark (sq);Jens,
io_sq_thread_parkwill try to wake the sqpoll, which is, at least, unnecessary. But I'm thinking we don't want to expose the ability to schedule the sqpoll from procfs, which can be done by any unrelated process.Quoted 8 lines
Quoted 5 lines
To solve the bug, it should be enough to synchronize directly on
sqd->lock, preventingsq->threadfrom going away inside the if leg. Granted, it is might take longer if the sqpoll is busy, but reading fdinfo is not supposed to be fast. Alternatively, don't callwake_processin this case?I did think about that but just went with the exported API. But you are right, it's a bit annoying that it'd also wake the thread, in case it
Waking it up is not a problem but without parking sq thread won't drop the lock until it's time to sleep, which might be pretty long leaving the /proc read stuck on the lock uninterruptibly.
Aside from parking vs lock, there is a lock inversion now:
proc read | SQPOLL
|
try_lock(ring)// success |
| woken up
| lock(sqd); // success
lock(sqd); // stuck |
| try to submit requests | -- lock(ring); // stuck
Quoted 8 lines
Quoted 6 lines
Yeah good point, forgot we nest these opposite of what you'd expect. Honestly I think the fix here is just to turn it into a trylock. Yes that'll miss some cases where we could've gotten the pid/cpu, but doesn't seem worth caring about.
IOW, fold in this incremental.
Should work, otherwise you probably can just park first.
In general I think it's better to have the observational side be less of an impact, which is why I liked not doing the parking. Sometimes apps do stupid things and monitor fdinfo/ etc continually. As it's possible to get the task/cpu anyway via other means, should be better to just skip if we don't get the lock.
Quoted 4 lines
Long term it'd be nice to make sqpoll to not hold
sqd->lockduring submission + polling as it currently does. Park callers sleep on the lock, but if we replace it with some struct completion the rest should be easy enough.
Yeah agree.
Signature · 1 line
Jens Axboe