Skip to contentSkip to conversation

io_uring: don't mark S_ISBLK async work as unbounded

PATCH In review Jens Axboe 3 participants 7 messages Apr 1, 2021 – Apr 5, 2021

Subsystems (2)
lore
Share & tools
lore

Jens Axboe

author maintainer
#0 lore raw

PATCH 1/1 io_uring: don't mark S_ISBLK async work as unbounded

S_ISBLK is marked as unbounded work for async preparation, because it doesn't match S_ISREG. That is incorrect, as any read/write to a block device is also a bounded operation. Fix it up and ensure that S_ISBLK isn't marked unbounded.

---

fs/io_uring.c +1 -1
index 6d7a1b69712b..a16b7df934d1 100644
@@ -1213,7 +1213,7 @@ static void io_prep_async_work(struct io_kiocb *req)
 	if (req->flags & REQ_F_ISREG) {
 		if (def->hash_reg_file || (ctx->flags & IORING_SETUP_IOPOLL))
 			io_wq_hash_work(&req->work, file_inode(req->file));
-	} else {
+	} else if (!req->file || !S_ISBLK(file_inode(req->file)->i_mode)) {
 		if (def->unbound_nonreg_file)
 			req->work.flags |= IO_WQ_WORK_UNBOUND;
 	}
Signature · 1 line
Jens Axboe
#1 lore raw
Quoted 7 lines · Jens Axboe

S_ISBLK is marked as unbounded work for async preparation, because it doesn't match S_ISREG. That is incorrect, as any read/write to a block device is also a bounded operation. Fix it up and ensure that S_ISBLK isn't marked unbounded.

Hi Jens, I saw a (un)bounded work is for a (un)bounded worker to execute. What is the difference between bounded and unbounded?

Thanks,
Hao

Quoted 16 lines

---

fs/io_uring.c +1 -1
index 6d7a1b69712b..a16b7df934d1 100644
@@ -1213,7 +1213,7 @@ static void io_prep_async_work(struct io_kiocb *req)
  	if (req->flags & REQ_F_ISREG) {
  		if (def->hash_reg_file || (ctx->flags & IORING_SETUP_IOPOLL))
  			io_wq_hash_work(&req->work, file_inode(req->file));
-	} else {
+	} else if (!req->file || !S_ISBLK(file_inode(req->file)->i_mode)) {
  		if (def->unbound_nonreg_file)
  			req->work.flags |= IO_WQ_WORK_UNBOUND;
  	}
#2 lore raw
Quoted 10 lines · Hao Xu
Quoted 7 lines · Jens Axboe

S_ISBLK is marked as unbounded work for async preparation, because it doesn't match S_ISREG. That is incorrect, as any read/write to a block device is also a bounded operation. Fix it up and ensure that S_ISBLK isn't marked unbounded.

Hi Jens, I saw a (un)bounded work is for a (un)bounded worker to execute. What is the difference between bounded and unbounded?

Unbounded works are not bounded in execution time, i.e. they may take forever to complete. E.g. recv depends on the other end to send something, that not necessarily will ever happen.

Quoted 20 lines

Thanks,
Hao

Quoted 16 lines

---

fs/io_uring.c +1 -1
index 6d7a1b69712b..a16b7df934d1 100644
@@ -1213,7 +1213,7 @@ static void io_prep_async_work(struct io_kiocb *req)
       if (req->flags & REQ_F_ISREG) {
           if (def->hash_reg_file || (ctx->flags & IORING_SETUP_IOPOLL))
               io_wq_hash_work(&req->work, file_inode(req->file));
-    } else {
+    } else if (!req->file || !S_ISBLK(file_inode(req->file)->i_mode)) {
           if (def->unbound_nonreg_file)
               req->work.flags |= IO_WQ_WORK_UNBOUND;
       }
Signature · 1 line
Pavel Begunkov
#3 lore raw
Quoted 7 lines · Pavel Begunkov
Quoted 6 lines · Hao Xu
Quoted 5 lines · Jens Axboe

S_ISBLK is marked as unbounded work for async preparation, because it doesn't match S_ISREG. That is incorrect, as any read/write to a block device is also a bounded operation. Fix it up and ensure that S_ISBLK isn't marked unbounded.

Quoted 8 lines
Quoted 4 lines
Quoted 2 lines

Hi Jens, I saw a (un)bounded work is for a (un)bounded worker to execute. What is the difference between bounded and unbounded?

Unbounded works are not bounded in execution time, i.e. they may take forever to complete. E.g. recv depends on the other end to send something, that not necessarily will ever happen.

To elaborate a bit, one example of how it's used: because unbounded may stay for long, it always spawns a new worker thread for each of them.

If app submits SQEs as below, and send's are not actually sent for execution but stashed somewhere internally in a list, e.g. waiting for a worker thread to get free, it would just hang from the userspace perspective.

recv(fd1), recv(fd1), send(fd1), send(fd1)

Quoted 13 lines
Quoted 13 lines
Quoted 13 lines
fs/io_uring.c +1 -1
index 6d7a1b69712b..a16b7df934d1 100644
@@ -1213,7 +1213,7 @@ static void io_prep_async_work(struct io_kiocb *req)
       if (req->flags & REQ_F_ISREG) {
           if (def->hash_reg_file || (ctx->flags & IORING_SETUP_IOPOLL))
               io_wq_hash_work(&req->work, file_inode(req->file));
-    } else {
+    } else if (!req->file || !S_ISBLK(file_inode(req->file)->i_mode)) {
           if (def->unbound_nonreg_file)
               req->work.flags |= IO_WQ_WORK_UNBOUND;
       }
Signature · 1 line
Pavel Begunkov
#4 lore raw
Quoted 17 lines · Pavel Begunkov
Quoted 7 lines · Pavel Begunkov
Quoted 6 lines · Hao Xu
Quoted 5 lines · Jens Axboe

S_ISBLK is marked as unbounded work for async preparation, because it doesn't match S_ISREG. That is incorrect, as any read/write to a block device is also a bounded operation. Fix it up and ensure that S_ISBLK isn't marked unbounded.

Quoted 8 lines
Quoted 4 lines
Quoted 2 lines

Hi Jens, I saw a (un)bounded work is for a (un)bounded worker to execute. What is the difference between bounded and unbounded?

Unbounded works are not bounded in execution time, i.e. they may take forever to complete. E.g. recv depends on the other end to send something, that not necessarily will ever happen.

Quoted 9 lines

To elaborate a bit, one example of how it's used: because unbounded may stay for long, it always spawns a new worker thread for each of them.

If app submits SQEs as below, and send's are not actually sent for execution but stashed somewhere internally in a list, e.g. waiting for a worker thread to get free, it would just hang from the userspace perspective.

recv(fd1), recv(fd1), send(fd1), send(fd1)

Hi Pavel, thank you for the patient explanation, I got the meaning of
bound/unbond now, but it seems there is no difference handling bounded
and unbounded work in the current code?

Quoted 14 lines
Quoted 13 lines
Quoted 13 lines
Quoted 13 lines
fs/io_uring.c +1 -1
index 6d7a1b69712b..a16b7df934d1 100644
@@ -1213,7 +1213,7 @@ static void io_prep_async_work(struct io_kiocb *req)
       if (req->flags & REQ_F_ISREG) {
           if (def->hash_reg_file || (ctx->flags & IORING_SETUP_IOPOLL))
               io_wq_hash_work(&req->work, file_inode(req->file));
-    } else {
+    } else if (!req->file || !S_ISBLK(file_inode(req->file)->i_mode)) {
           if (def->unbound_nonreg_file)
               req->work.flags |= IO_WQ_WORK_UNBOUND;
       }
#5 lore raw
Quoted 21 lines · Hao Xu
Quoted 20 lines · Pavel Begunkov
Quoted 7 lines · Pavel Begunkov
Quoted 6 lines · Hao Xu
Quoted 5 lines · Jens Axboe

S_ISBLK is marked as unbounded work for async preparation, because it doesn't match S_ISREG. That is incorrect, as any read/write to a block device is also a bounded operation. Fix it up and ensure that S_ISBLK isn't marked unbounded.

Quoted 8 lines
Quoted 4 lines
Quoted 2 lines

Hi Jens, I saw a (un)bounded work is for a (un)bounded worker to execute. What is the difference between bounded and unbounded?

Unbounded works are not bounded in execution time, i.e. they may take forever to complete. E.g. recv depends on the other end to send something, that not necessarily will ever happen.

To elaborate a bit, one example of how it's used: because unbounded may stay for long, it always spawns a new worker thread for each of them.

Quoted 8 lines
Quoted 6 lines

If app submits SQEs as below, and send's are not actually sent for execution but stashed somewhere internally in a list, e.g. waiting for a worker thread to get free, it would just hang from the userspace perspective.

recv(fd1), recv(fd1), send(fd1), send(fd1)

Hi Pavel, thank you for the patient explanation, I got the meaning of
 bound/unbond now, but it seems there is no difference handling bounded and unbounded work in the current code?

It was used in io-wq, so see in io-wq.[c,h] files. Not sure about 5.12+, but a quick look shows:

wqe->acct[IO_WQ_ACCT_BOUND].max_workers = bounded;
...
wqe->acct[IO_WQ_ACCT_UNBOUND].max_workers =

			task_rlimit(current, RLIMIT_NPROC);
Quoted 15 lines
Quoted 14 lines
Quoted 13 lines
Quoted 13 lines
Quoted 13 lines
fs/io_uring.c +1 -1
index 6d7a1b69712b..a16b7df934d1 100644
@@ -1213,7 +1213,7 @@ static void io_prep_async_work(struct io_kiocb *req)
        if (req->flags & REQ_F_ISREG) {
            if (def->hash_reg_file || (ctx->flags & IORING_SETUP_IOPOLL))
                io_wq_hash_work(&req->work, file_inode(req->file));
-    } else {
+    } else if (!req->file || !S_ISBLK(file_inode(req->file)->i_mode)) {
            if (def->unbound_nonreg_file)
                req->work.flags |= IO_WQ_WORK_UNBOUND;
        }
Signature · 1 line
Pavel Begunkov
#6 lore raw
Quoted 31 lines · Pavel Begunkov
Quoted 21 lines · Hao Xu
Quoted 20 lines · Pavel Begunkov
Quoted 7 lines · Pavel Begunkov
Quoted 6 lines · Hao Xu
Quoted 5 lines · Jens Axboe

S_ISBLK is marked as unbounded work for async preparation, because it doesn't match S_ISREG. That is incorrect, as any read/write to a block device is also a bounded operation. Fix it up and ensure that S_ISBLK isn't marked unbounded.

Quoted 8 lines
Quoted 4 lines
Quoted 2 lines

Hi Jens, I saw a (un)bounded work is for a (un)bounded worker to execute. What is the difference between bounded and unbounded?

Unbounded works are not bounded in execution time, i.e. they may take forever to complete. E.g. recv depends on the other end to send something, that not necessarily will ever happen.

To elaborate a bit, one example of how it's used: because unbounded may stay for long, it always spawns a new worker thread for each of them.

Quoted 8 lines
Quoted 6 lines

If app submits SQEs as below, and send's are not actually sent for execution but stashed somewhere internally in a list, e.g. waiting for a worker thread to get free, it would just hang from the userspace perspective.

recv(fd1), recv(fd1), send(fd1), send(fd1)

Hi Pavel, thank you for the patient explanation, I got the meaning of
 bound/unbond now, but it seems there is no difference handling bounded and unbounded work in the current code?

Quoted 8 lines

It was used in io-wq, so see in io-wq.[c,h] files. Not sure about 5.12+, but a quick look shows:

wqe->acct[IO_WQ_ACCT_BOUND].max_workers = bounded;
...
wqe->acct[IO_WQ_ACCT_UNBOUND].max_workers =

			task_rlimit(current, RLIMIT_NPROC);

Thanks, Pavel.:)

Quoted 18 lines
Quoted 17 lines
Quoted 16 lines
Quoted 15 lines
Quoted 15 lines
Quoted 15 lines
fs/io_uring.c +1 -1
index 6d7a1b69712b..a16b7df934d1 100644
@@ -1213,7 +1213,7 @@ static void io_prep_async_work(struct io_kiocb *req)
        if (req->flags & REQ_F_ISREG) {
            if (def->hash_reg_file || (ctx->flags & IORING_SETUP_IOPOLL))
                io_wq_hash_work(&req->work, file_inode(req->file));
-    } else {
+    } else if (!req->file || !S_ISBLK(file_inode(req->file)->i_mode)) {
            if (def->unbound_nonreg_file)
                req->work.flags |= IO_WQ_WORK_UNBOUND;
        }