Skip to contentSkip to conversation

io_uring/fdinfo: park SQ thread while retrieving cpu/pid

PATCH Superseded Jens Axboe 2 participants 6 messages Oct 23, 2023 – Oct 25, 2023 1 message not in the archive io_uring

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore

Jens Axboe

author maintainer
#0 lore raw

PATCH 1/1 io_uring/fdinfo: park SQ thread while retrieving cpu/pid

We could race with SQ thread exit, and if we do, we'll hit a NULL pointer dereference. Park the SQPOLL thread while getting the task cpu and pid for fdinfo, this ensures we have a stable view of it.

---

io_uring/fdinfo.c +11 -6
index c53678875416..cd2a0c6b97c4 100644
@@ -53,7 +53,6 @@ static __cold int io_uring_show_cred(struct seq_file *m, unsigned int id,
 __cold void io_uring_show_fdinfo(struct seq_file *m, struct file *f)
 {
 	struct io_ring_ctx *ctx = f->private_data;
-	struct io_sq_data *sq = NULL;
 	struct io_overflow_cqe *ocqe;
 	struct io_rings *r = ctx->rings;
 	unsigned int sq_mask = ctx->sq_entries - 1, cq_mask = ctx->cq_entries - 1;
@@ -64,6 +63,7 @@ __cold void io_uring_show_fdinfo(struct seq_file *m, struct file *f)
 	unsigned int cq_shift = 0;
 	unsigned int sq_shift = 0;
 	unsigned int sq_entries, cq_entries;
+	int sq_pid = -1, sq_cpu = -1;
 	bool has_lock;
 	unsigned int i;
 
@@ -143,13 +143,18 @@ __cold void io_uring_show_fdinfo(struct seq_file *m, struct file *f)
 	has_lock = mutex_trylock(&ctx->uring_lock);
 
 	if (has_lock && (ctx->flags & IORING_SETUP_SQPOLL)) {
-		sq = ctx->sq_data;
-		if (!sq->thread)
-			sq = NULL;
+		struct io_sq_data *sq = ctx->sq_data;
+
+		io_sq_thread_park(sq);
+		if (sq->thread) {
+			sq_pid = task_pid_nr(sq->thread);
+			sq_cpu = task_cpu(sq->thread);
+		}
+		io_sq_thread_unpark(sq);
 	}
 
-	seq_printf(m, "SqThread:\t%d\n", sq ? task_pid_nr(sq->thread) : -1);
-	seq_printf(m, "SqThreadCpu:\t%d\n", sq ? task_cpu(sq->thread) : -1);
+	seq_printf(m, "SqThread:\t%d\n", sq_pid);
+	seq_printf(m, "SqThreadCpu:\t%d\n", sq_cpu);
 	seq_printf(m, "UserFiles:\t%u\n", ctx->nr_user_files);
 	for (i = 0; has_lock && i < ctx->nr_user_files; i++) {
 		struct file *f = io_file_from_index(&ctx->file_table, i);
Signature · 1 line
Jens Axboe

A message that is not in the archive (sent off-list or not yet received): [email protected]

author maintainer
#2 lore raw
Quoted 55 lines · Gabriel Krisman Bertazi
Quoted 35 lines · Jens Axboe

We could race with SQ thread exit, and if we do, we'll hit a NULL pointer dereference. Park the SQPOLL thread while getting the task cpu and pid for fdinfo, this ensures we have a stable view of it.

---

io_uring/fdinfo.c +1 -2
index c53678875416..cd2a0c6b97c4 100644
@@ -53,7 +53,6 @@ static __cold int io_uring_show_cred(struct seq_file *m, unsigned int id,
 __cold void io_uring_show_fdinfo(struct seq_file *m, struct file *f)
 {
 	struct io_ring_ctx *ctx = f->private_data;
-	struct io_sq_data *sq = NULL;
 	struct io_overflow_cqe *ocqe;
 	struct io_rings *r = ctx->rings;
 	unsigned int sq_mask = ctx->sq_entries - 1, cq_mask = ctx->cq_entries - 1;
@@ -64,6 +63,7 @@ __cold void io_uring_show_fdinfo(struct seq_file *m, struct file *f)
 	unsigned int cq_shift = 0;
 	unsigned int sq_shift = 0;
 	unsigned int sq_entries, cq_entries;
+	int sq_pid = -1, sq_cpu = -1;
 	bool has_lock;
 	unsigned int i;
 
@@ -143,13 +143,18 @@ __cold void io_uring_show_fdinfo(struct seq_file *m, struct file *f)
 	has_lock = mutex_trylock(&ctx->uring_lock);
 
 	if (has_lock && (ctx->flags & IORING_SETUP_SQPOLL)) {
-		sq = ctx->sq_data;

io_uring/fdinfo.c __cold void io_uring_show_fdinfo(struct seq_file *m, struct file *f) · line 153

Quoted 10 lines
io_uring/fdinfo.c +8 -2
-		if (!sq->thread)
-			sq = NULL;
+		struct io_sq_data *sq = ctx->sq_data;
+
+		io_sq_thread_park(sq);
+		if (sq->thread) {
+			sq_pid = task_pid_nr(sq->thread);
+			sq_cpu = task_cpu(sq->thread);
+		}
+		io_sq_thread_unpark(sq);

Jens,

io_sq_thread_park will try to wake the sqpoll, which is, at least, unnecessary. But I'm thinking we don't want to expose the ability to schedule the sqpoll from procfs, which can be done by any unrelated process.

Quoted 5 lines

To solve the bug, it should be enough to synchronize directly on sqd->lock, preventing sq->thread from going away inside the if leg. Granted, it is might take longer if the sqpoll is busy, but reading fdinfo is not supposed to be fast. Alternatively, don't call wake_process in this case?

I did think about that but just went with the exported API. But you are right, it's a bit annoying that it'd also wake the thread, in case it was idle. Probably mostly cosmetic, but we may as well just stick with grabbing the sqd mutex. I'll send a v2.

Signature · 1 line
Jens Axboe
#3 lore raw
Quoted 56 lines · Jens Axboe
Quoted 55 lines · Gabriel Krisman Bertazi
Quoted 35 lines · Jens Axboe

We could race with SQ thread exit, and if we do, we'll hit a NULL pointer dereference. Park the SQPOLL thread while getting the task cpu and pid for fdinfo, this ensures we have a stable view of it.

---

io_uring/fdinfo.c +1 -2
index c53678875416..cd2a0c6b97c4 100644
@@ -53,7 +53,6 @@ static __cold int io_uring_show_cred(struct seq_file *m, unsigned int id,
  __cold void io_uring_show_fdinfo(struct seq_file *m, struct file *f)
  {
  	struct io_ring_ctx *ctx = f->private_data;
-	struct io_sq_data *sq = NULL;
  	struct io_overflow_cqe *ocqe;
  	struct io_rings *r = ctx->rings;
  	unsigned int sq_mask = ctx->sq_entries - 1, cq_mask = ctx->cq_entries - 1;
@@ -64,6 +63,7 @@ __cold void io_uring_show_fdinfo(struct seq_file *m, struct file *f)
  	unsigned int cq_shift = 0;
  	unsigned int sq_shift = 0;
  	unsigned int sq_entries, cq_entries;
+	int sq_pid = -1, sq_cpu = -1;
  	bool has_lock;
  	unsigned int i;
 
@@ -143,13 +143,18 @@ __cold void io_uring_show_fdinfo(struct seq_file *m, struct file *f)
  	has_lock = mutex_trylock(&ctx->uring_lock);
 
  	if (has_lock && (ctx->flags & IORING_SETUP_SQPOLL)) {
-		sq = ctx->sq_data;

io_uring/fdinfo.c __cold void io_uring_show_fdinfo(struct seq_file *m, struct file *f) · line 153

Quoted 10 lines
io_uring/fdinfo.c +8 -2
-		if (!sq->thread)
-			sq = NULL;
+		struct io_sq_data *sq = ctx->sq_data;
+
+		io_sq_thread_park(sq);
+		if (sq->thread) {
+			sq_pid = task_pid_nr(sq->thread);
+			sq_cpu = task_cpu(sq->thread);
+		}
+		io_sq_thread_unpark(sq);

Jens,

io_sq_thread_park will try to wake the sqpoll, which is, at least, unnecessary. But I'm thinking we don't want to expose the ability to schedule the sqpoll from procfs, which can be done by any unrelated process.

Quoted 8 lines
Quoted 5 lines

To solve the bug, it should be enough to synchronize directly on sqd->lock, preventing sq->thread from going away inside the if leg. Granted, it is might take longer if the sqpoll is busy, but reading fdinfo is not supposed to be fast. Alternatively, don't call wake_process in this case?

I did think about that but just went with the exported API. But you are right, it's a bit annoying that it'd also wake the thread, in case it

Waking it up is not a problem but without parking sq thread won't drop the lock until it's time to sleep, which might be pretty long leaving the /proc read stuck on the lock uninterruptibly.

Aside from parking vs lock, there is a lock inversion now:

proc read                   | SQPOLL
                            |
try_lock(ring) // success   |
                            | woken up
                            | lock(sqd); // success
lock(sqd); // stuck         |
                            | try to submit requests
                            | -- lock(ring); // stuck
Quoted 2 lines

was idle. Probably mostly cosmetic, but we may as well just stick with grabbing the sqd mutex. I'll send a v2.

Signature · 1 line
Pavel Begunkov
author maintainer
#4 lore raw
Quoted 70 lines · Pavel Begunkov
Quoted 54 lines · Jens Axboe
Quoted 53 lines · Gabriel Krisman Bertazi
Quoted 33 lines · Jens Axboe

We could race with SQ thread exit, and if we do, we'll hit a NULL pointer dereference. Park the SQPOLL thread while getting the task cpu and pid for fdinfo, this ensures we have a stable view of it.

---

io_uring/fdinfo.c +1 -1
index c53678875416..cd2a0c6b97c4 100644
@@ -53,7 +53,6 @@ static __cold int io_uring_show_cred(struct seq_file *m, unsigned int id,
  __cold void io_uring_show_fdinfo(struct seq_file *m, struct file *f)
  {
      struct io_ring_ctx *ctx = f->private_data;
-    struct io_sq_data *sq = NULL;
      struct io_overflow_cqe *ocqe;
      struct io_rings *r = ctx->rings;
      unsigned int sq_mask = ctx->sq_entries - 1, cq_mask = ctx->cq_entries - 1;
@@ -64,6 +63,7 @@ __cold void io_uring_show_fdinfo(struct seq_file *m, struct file *f)
      unsigned int cq_shift = 0;
      unsigned int sq_shift = 0;
      unsigned int sq_entries, cq_entries;
+    int sq_pid = -1, sq_cpu = -1;
      bool has_lock;
      unsigned int i;
  @@ -143,13 +143,18 @@ __cold void io_uring_show_fdinfo(struct seq_file *m, struct file *f)
      has_lock = mutex_trylock(&ctx->uring_lock);
        if (has_lock && (ctx->flags & IORING_SETUP_SQPOLL)) {
io_uring/fdinfo.c +0 -1
-        sq = ctx->sq_data;

io_uring/fdinfo.c

Quoted 10 lines
io_uring/fdinfo.c +8 -2
-        if (!sq->thread)
-            sq = NULL;
+        struct io_sq_data *sq = ctx->sq_data;
+
+        io_sq_thread_park(sq);
+        if (sq->thread) {
+            sq_pid = task_pid_nr(sq->thread);
+            sq_cpu = task_cpu(sq->thread);
+        }
+        io_sq_thread_unpark(sq);

Jens,

io_sq_thread_park will try to wake the sqpoll, which is, at least, unnecessary. But I'm thinking we don't want to expose the ability to schedule the sqpoll from procfs, which can be done by any unrelated process.

Quoted 8 lines
Quoted 5 lines

To solve the bug, it should be enough to synchronize directly on sqd->lock, preventing sq->thread from going away inside the if leg. Granted, it is might take longer if the sqpoll is busy, but reading fdinfo is not supposed to be fast. Alternatively, don't call wake_process in this case?

I did think about that but just went with the exported API. But you are right, it's a bit annoying that it'd also wake the thread, in case it

Waking it up is not a problem but without parking sq thread won't drop the lock until it's time to sleep, which might be pretty long leaving the /proc read stuck on the lock uninterruptibly.

Aside from parking vs lock, there is a lock inversion now:

Quoted 8 lines
proc read                   | SQPOLL
                            |
try_lock(ring) // success   |
                            | woken up
                            | lock(sqd); // success
lock(sqd); // stuck         |
                            | try to submit requests
                            | -- lock(ring); // stuck

Yeah good point, forgot we nest these opposite of what you'd expect. Honestly I think the fix here is just to turn it into a trylock. Yes that'll miss some cases where we could've gotten the pid/cpu, but doesn't seem worth caring about.

IOW, fold in this incremental.

io_uring/fdinfo.c +6 -5
index af1bdcc0703e..f04a43044d91 100644
@@ -145,12 +145,13 @@ __cold void io_uring_show_fdinfo(struct seq_file *m, struct file *f)
 	if (has_lock && (ctx->flags & IORING_SETUP_SQPOLL)) {
 		struct io_sq_data *sq = ctx->sq_data;
 
-		mutex_lock(&sq->lock);
-		if (sq->thread) {
-			sq_pid = task_pid_nr(sq->thread);
-			sq_cpu = task_cpu(sq->thread);
+		if (mutex_trylock(&sq->lock)) {
+			if (sq->thread) {
+				sq_pid = task_pid_nr(sq->thread);
+				sq_cpu = task_cpu(sq->thread);
+			}
+			mutex_unlock(&sq->lock);
 		}
-		mutex_unlock(&sq->lock);
 	}
 
 	seq_printf(m, "SqThread:\t%d\n", sq_pid);
Signature · 1 line
Jens Axboe
#5 lore raw
Quoted 80 lines · Jens Axboe
Quoted 70 lines · Pavel Begunkov
Quoted 54 lines · Jens Axboe
Quoted 53 lines · Gabriel Krisman Bertazi
Quoted 33 lines · Jens Axboe

We could race with SQ thread exit, and if we do, we'll hit a NULL pointer dereference. Park the SQPOLL thread while getting the task cpu and pid for fdinfo, this ensures we have a stable view of it.

---

io_uring/fdinfo.c +1 -1
index c53678875416..cd2a0c6b97c4 100644
@@ -53,7 +53,6 @@ static __cold int io_uring_show_cred(struct seq_file *m, unsigned int id,
   __cold void io_uring_show_fdinfo(struct seq_file *m, struct file *f)
   {
       struct io_ring_ctx *ctx = f->private_data;
-    struct io_sq_data *sq = NULL;
       struct io_overflow_cqe *ocqe;
       struct io_rings *r = ctx->rings;
       unsigned int sq_mask = ctx->sq_entries - 1, cq_mask = ctx->cq_entries - 1;
@@ -64,6 +63,7 @@ __cold void io_uring_show_fdinfo(struct seq_file *m, struct file *f)
       unsigned int cq_shift = 0;
       unsigned int sq_shift = 0;
       unsigned int sq_entries, cq_entries;
+    int sq_pid = -1, sq_cpu = -1;
       bool has_lock;
       unsigned int i;
   @@ -143,13 +143,18 @@ __cold void io_uring_show_fdinfo(struct seq_file *m, struct file *f)
       has_lock = mutex_trylock(&ctx->uring_lock);
         if (has_lock && (ctx->flags & IORING_SETUP_SQPOLL)) {
io_uring/fdinfo.c +0 -1
-        sq = ctx->sq_data;

io_uring/fdinfo.c

Quoted 10 lines
io_uring/fdinfo.c +8 -2
-        if (!sq->thread)
-            sq = NULL;
+        struct io_sq_data *sq = ctx->sq_data;
+
+        io_sq_thread_park(sq);
+        if (sq->thread) {
+            sq_pid = task_pid_nr(sq->thread);
+            sq_cpu = task_cpu(sq->thread);
+        }
+        io_sq_thread_unpark(sq);

Jens,

io_sq_thread_park will try to wake the sqpoll, which is, at least, unnecessary. But I'm thinking we don't want to expose the ability to schedule the sqpoll from procfs, which can be done by any unrelated process.

Quoted 8 lines
Quoted 5 lines

To solve the bug, it should be enough to synchronize directly on sqd->lock, preventing sq->thread from going away inside the if leg. Granted, it is might take longer if the sqpoll is busy, but reading fdinfo is not supposed to be fast. Alternatively, don't call wake_process in this case?

I did think about that but just went with the exported API. But you are right, it's a bit annoying that it'd also wake the thread, in case it

Waking it up is not a problem but without parking sq thread won't drop the lock until it's time to sleep, which might be pretty long leaving the /proc read stuck on the lock uninterruptibly.

Aside from parking vs lock, there is a lock inversion now:

Quoted 8 lines

proc read | SQPOLL

                             |

try_lock(ring) // success |
| woken up
| lock(sqd); // success
lock(sqd); // stuck |
| try to submit requests | -- lock(ring); // stuck

Quoted 6 lines

Yeah good point, forgot we nest these opposite of what you'd expect. Honestly I think the fix here is just to turn it into a trylock. Yes that'll miss some cases where we could've gotten the pid/cpu, but doesn't seem worth caring about.

IOW, fold in this incremental.

Should work, otherwise you probably can just park first.

Long term it'd be nice to make sqpoll to not hold sqd->lock during submission + polling as it currently does. Park callers sleep on the lock, but if we replace it with some struct completion the rest should be easy enough.

Quoted 24 lines
io_uring/fdinfo.c +6 -5
index af1bdcc0703e..f04a43044d91 100644
@@ -145,12 +145,13 @@ __cold void io_uring_show_fdinfo(struct seq_file *m, struct file *f)
  	if (has_lock && (ctx->flags & IORING_SETUP_SQPOLL)) {
  		struct io_sq_data *sq = ctx->sq_data;
 
-		mutex_lock(&sq->lock);
-		if (sq->thread) {
-			sq_pid = task_pid_nr(sq->thread);
-			sq_cpu = task_cpu(sq->thread);
+		if (mutex_trylock(&sq->lock)) {
+			if (sq->thread) {
+				sq_pid = task_pid_nr(sq->thread);
+				sq_cpu = task_cpu(sq->thread);
+			}
+			mutex_unlock(&sq->lock);
  		}
-		mutex_unlock(&sq->lock);
  	}
 
  	seq_printf(m, "SqThread:\t%d\n", sq_pid);
Signature · 1 line
Pavel Begunkov
author maintainer
#6 lore raw
Quoted 81 lines · Pavel Begunkov
Quoted 80 lines · Jens Axboe
Quoted 78 lines · Pavel Begunkov
Quoted 54 lines · Jens Axboe
Quoted 53 lines · Gabriel Krisman Bertazi
Quoted 33 lines · Jens Axboe

We could race with SQ thread exit, and if we do, we'll hit a NULL pointer dereference. Park the SQPOLL thread while getting the task cpu and pid for fdinfo, this ensures we have a stable view of it.

---

io_uring/fdinfo.c +1 -1
index c53678875416..cd2a0c6b97c4 100644
@@ -53,7 +53,6 @@ static __cold int io_uring_show_cred(struct seq_file *m, unsigned int id,
   __cold void io_uring_show_fdinfo(struct seq_file *m, struct file *f)
   {
       struct io_ring_ctx *ctx = f->private_data;
-    struct io_sq_data *sq = NULL;
       struct io_overflow_cqe *ocqe;
       struct io_rings *r = ctx->rings;
       unsigned int sq_mask = ctx->sq_entries - 1, cq_mask = ctx->cq_entries - 1;
@@ -64,6 +63,7 @@ __cold void io_uring_show_fdinfo(struct seq_file *m, struct file *f)
       unsigned int cq_shift = 0;
       unsigned int sq_shift = 0;
       unsigned int sq_entries, cq_entries;
+    int sq_pid = -1, sq_cpu = -1;
       bool has_lock;
       unsigned int i;
   @@ -143,13 +143,18 @@ __cold void io_uring_show_fdinfo(struct seq_file *m, struct file *f)
       has_lock = mutex_trylock(&ctx->uring_lock);
         if (has_lock && (ctx->flags & IORING_SETUP_SQPOLL)) {
io_uring/fdinfo.c +0 -1
-        sq = ctx->sq_data;

io_uring/fdinfo.c

Quoted 10 lines
io_uring/fdinfo.c +8 -2
-        if (!sq->thread)
-            sq = NULL;
+        struct io_sq_data *sq = ctx->sq_data;
+
+        io_sq_thread_park(sq);
+        if (sq->thread) {
+            sq_pid = task_pid_nr(sq->thread);
+            sq_cpu = task_cpu(sq->thread);
+        }
+        io_sq_thread_unpark(sq);

Jens,

io_sq_thread_park will try to wake the sqpoll, which is, at least, unnecessary. But I'm thinking we don't want to expose the ability to schedule the sqpoll from procfs, which can be done by any unrelated process.

Quoted 8 lines
Quoted 5 lines

To solve the bug, it should be enough to synchronize directly on sqd->lock, preventing sq->thread from going away inside the if leg. Granted, it is might take longer if the sqpoll is busy, but reading fdinfo is not supposed to be fast. Alternatively, don't call wake_process in this case?

I did think about that but just went with the exported API. But you are right, it's a bit annoying that it'd also wake the thread, in case it

Waking it up is not a problem but without parking sq thread won't drop the lock until it's time to sleep, which might be pretty long leaving the /proc read stuck on the lock uninterruptibly.

Aside from parking vs lock, there is a lock inversion now:

proc read | SQPOLL

                             |

try_lock(ring) // success |
| woken up
| lock(sqd); // success
lock(sqd); // stuck |
| try to submit requests | -- lock(ring); // stuck

Quoted 8 lines
Quoted 6 lines

Yeah good point, forgot we nest these opposite of what you'd expect. Honestly I think the fix here is just to turn it into a trylock. Yes that'll miss some cases where we could've gotten the pid/cpu, but doesn't seem worth caring about.

IOW, fold in this incremental.

Should work, otherwise you probably can just park first.

In general I think it's better to have the observational side be less of an impact, which is why I liked not doing the parking. Sometimes apps do stupid things and monitor fdinfo/ etc continually. As it's possible to get the task/cpu anyway via other means, should be better to just skip if we don't get the lock.

Quoted 4 lines

Long term it'd be nice to make sqpoll to not hold sqd->lock during submission + polling as it currently does. Park callers sleep on the lock, but if we replace it with some struct completion the rest should be easy enough.

Yeah agree.

Signature · 1 line
Jens Axboe