dm: support I/O polling

Support I/O polling if submit_bio_noacct_mq_direct returned non-empty cookie.

Signed-off-by Mikulas Patocka <[email protected]>

---
 drivers/md/dm.c |    5 +++++
 1 file changed, 5 insertions(+)

Index: linux-2.6/drivers/md/dm.c
===================================================================

drivers/md/dm.c +0 -0

@@ -1682,6 +1682,11 @@ static void __split_and_process_bio(stru
 		}

Quoted 17 lines · Mikulas Patocka

Support I/O polling if submit_bio_noacct_mq_direct returned non-empty cookie.

Signed-off-by Mikulas Patocka <[email protected]>

---
 drivers/md/dm.c |    5 +++++
 1 file changed, 5 insertions(+)

Index: linux-2.6/drivers/md/dm.c
===================================================================

drivers/md/dm.c +0 -0

@@ -1682,6 +1682,11 @@ static void __split_and_process_bio(stru
 		}
 	}
 

drivers/md/dm.c static void __split_and_process_bio(stru · line 1692

Quoted 8 lines

drivers/md/dm.c +5 -0

+	if (ci.poll_cookie != BLK_QC_T_NONE) {
+		while (atomic_read(&ci.io->io_count) > 1 &&
+		       blk_poll(ci.poll_queue, ci.poll_cookie, true)) ;
+	}
+
 	/* drop the extra reference count */
 	dec_pending(ci.io, errno_to_blk_status(error));
 }

It seems that the general idea of your design is to
1) submit *one* split bio
2) blk_poll(), waiting the previously submitted split bio complets

and then submit next split bio, repeating the above process. I'm afraid the performance may be an issue here, since the batch every time blk_poll() reaps may decrease.

Besides, the submitting routine and polling routine is bound together here, i.e., polling is always synchronous.

Quoted 21 lines · JeffleXu

Quoted 17 lines · Mikulas Patocka

Support I/O polling if submit_bio_noacct_mq_direct returned non-empty cookie.

Signed-off-by Mikulas Patocka <[email protected]>

---
 drivers/md/dm.c |    5 +++++
 1 file changed, 5 insertions(+)

Index: linux-2.6/drivers/md/dm.c
===================================================================

drivers/md/dm.c +0 -0

@@ -1682,6 +1682,11 @@ static void __split_and_process_bio(stru
 		}
 	}
 

Quoted 12 lines

drivers/md/dm.c static void __split_and_process_bio(stru · line 1692

Quoted 8 lines

drivers/md/dm.c +5 -0

+	if (ci.poll_cookie != BLK_QC_T_NONE) {
+		while (atomic_read(&ci.io->io_count) > 1 &&
+		       blk_poll(ci.poll_queue, ci.poll_cookie, true)) ;
+	}
+
 	/* drop the extra reference count */
 	dec_pending(ci.io, errno_to_blk_status(error));
 }

It seems that the general idea of your design is to
1) submit *one* split bio
2) blk_poll(), waiting the previously submitted split bio complets

No, I submit all the bios and poll for the last one.

Quoted 3 lines

and then submit next split bio, repeating the above process. I'm afraid the performance may be an issue here, since the batch every time blk_poll() reaps may decrease.

Could you benchmark it?

Quoted 2 lines

Besides, the submitting routine and polling routine is bound together here, i.e., polling is always synchronous.

__split_and_process_bio calls __split_and_process_non_flush in a loop

__split_and_process_non_flush records the poll cookie in ci.poll_cookie. When we processed all the bios, we poll for the last cookie here:

        if (ci.poll_cookie != BLK_QC_T_NONE) {
                while (atomic_read(&ci.io->io_count) > 1 &&
                       blk_poll(ci.poll_queue, ci.poll_cookie, true)) ;
        }

Quoted 7 lines

__split_and_process_non_flush records the poll cookie in ci.poll_cookie. When we processed all the bios, we poll for the last cookie here:

        if (ci.poll_cookie != BLK_QC_T_NONE) {
                while (atomic_read(&ci.io->io_count) > 1 &&
                       blk_poll(ci.poll_queue, ci.poll_cookie, true)) ;
        }

So what will happen if one bio submitted to dm device crosses the device boundary among several target devices (e.g., dm-stripe)? Please refer the following call graph.

```
submit_bio
  __submit_bio_noacct
    disk->fops->submit_bio(), calling into __split_and_process_bio(),
call __split_and_process_non_flush() once, submitting the *first* split bio
    disk->fops->submit_bio(), calling into __split_and_process_bio(),
call __split_and_process_non_flush() once, submitting the *second* split bio
    ...
```

So the loop is in __submit_bio_noacct(), rather than __split_and_process_bio(). Your design will send the first split bio, and then poll on this split bio, then send the next split bio, polling on this, go on and on...

Quoted 23 lines · JeffleXu

Quoted 7 lines

__split_and_process_non_flush records the poll cookie in ci.poll_cookie. When we processed all the bios, we poll for the last cookie here:

        if (ci.poll_cookie != BLK_QC_T_NONE) {
                while (atomic_read(&ci.io->io_count) > 1 &&
                       blk_poll(ci.poll_queue, ci.poll_cookie, true)) ;
        }

So what will happen if one bio submitted to dm device crosses the device boundary among several target devices (e.g., dm-stripe)? Please refer the following call graph.

```
submit_bio
  __submit_bio_noacct
    disk->fops->submit_bio(), calling into __split_and_process_bio(),
call __split_and_process_non_flush() once, submitting the *first* split bio
    disk->fops->submit_bio(), calling into __split_and_process_bio(),
call __split_and_process_non_flush() once, submitting the *second* split bio
    ...
```

Quoted 4 lines

So the loop is in __submit_bio_noacct(), rather than __split_and_process_bio(). Your design will send the first split bio, and then poll on this split bio, then send the next split bio, polling on this, go on and on...

No. It will send all the bios and poll for the last one.

Quoted 21 lines · JeffleXu

Quoted 17 lines · Mikulas Patocka

Support I/O polling if submit_bio_noacct_mq_direct returned non-empty cookie.

Signed-off-by Mikulas Patocka <[email protected]>

---
 drivers/md/dm.c |    5 +++++
 1 file changed, 5 insertions(+)

Index: linux-2.6/drivers/md/dm.c
===================================================================

drivers/md/dm.c +0 -0

@@ -1682,6 +1682,11 @@ static void __split_and_process_bio(stru
 		}
 	}
 

Quoted 12 lines

drivers/md/dm.c static void __split_and_process_bio(stru · line 1692

Quoted 8 lines

drivers/md/dm.c +5 -0

+	if (ci.poll_cookie != BLK_QC_T_NONE) {
+		while (atomic_read(&ci.io->io_count) > 1 &&
+		       blk_poll(ci.poll_queue, ci.poll_cookie, true)) ;
+	}
+
 	/* drop the extra reference count */
 	dec_pending(ci.io, errno_to_blk_status(error));
 }

It seems that the general idea of your design is to
1) submit *one* split bio
2) blk_poll(), waiting the previously submitted split bio complets

No, I submit all the bios and poll for the last one.

Hi, Mikulas,

Quoted 36 lines · Mikulas Patocka

Quoted 33 lines · JeffleXu

Quoted 15 lines · Mikulas Patocka

Support I/O polling if submit_bio_noacct_mq_direct returned non-empty cookie.

Signed-off-by Mikulas Patocka <[email protected]>

---
 drivers/md/dm.c |    5 +++++
 1 file changed, 5 insertions(+)

Index: linux-2.6/drivers/md/dm.c
===================================================================

drivers/md/dm.c +0 -0

@@ -1682,6 +1682,11 @@ static void __split_and_process_bio(stru
 		}

drivers/md/dm.c static void __split_and_process_bio(stru · line 1692

Quoted 10 lines

drivers/md/dm.c +5 -0

 	}
 
+	if (ci.poll_cookie != BLK_QC_T_NONE) {
+		while (atomic_read(&ci.io->io_count) > 1 &&
+		       blk_poll(ci.poll_queue, ci.poll_cookie, true)) ;
+	}
+
 	/* drop the extra reference count */
 	dec_pending(ci.io, errno_to_blk_status(error));
 }

It seems that the general idea of your design is to
1) submit *one* split bio
2) blk_poll(), waiting the previously submitted split bio complets

Quoted 1 line

No, I submit all the bios and poll for the last one.

What happens if the last bio completes first? It looks like you will call blk_poll with a cookie that already completed, and I'm pretty sure that's invalid.

Hi, Mikulas,

Quoted 36 lines · Mikulas Patocka

Quoted 33 lines · JeffleXu

Quoted 15 lines · Mikulas Patocka

Support I/O polling if submit_bio_noacct_mq_direct returned non-empty cookie.

Signed-off-by Mikulas Patocka <[email protected]>

---
 drivers/md/dm.c |    5 +++++
 1 file changed, 5 insertions(+)

Index: linux-2.6/drivers/md/dm.c
===================================================================

drivers/md/dm.c +0 -0

@@ -1682,6 +1682,11 @@ static void __split_and_process_bio(stru
 		}

drivers/md/dm.c static void __split_and_process_bio(stru · line 1692

Quoted 10 lines

drivers/md/dm.c +5 -0

 	}
 
+	if (ci.poll_cookie != BLK_QC_T_NONE) {
+		while (atomic_read(&ci.io->io_count) > 1 &&
+		       blk_poll(ci.poll_queue, ci.poll_cookie, true)) ;
+	}
+
 	/* drop the extra reference count */
 	dec_pending(ci.io, errno_to_blk_status(error));
 }

It seems that the general idea of your design is to
1) submit *one* split bio
2) blk_poll(), waiting the previously submitted split bio complets

Quoted 1 line

No, I submit all the bios and poll for the last one.

What happens if the last bio completes first? It looks like you will call blk_poll with a cookie that already completed, and I'm pretty sure that's invalid.

Thanks,
Jeff

Quoted 21 lines · JeffleXu

Quoted 17 lines · Mikulas Patocka

Support I/O polling if submit_bio_noacct_mq_direct returned non-empty cookie.

Signed-off-by Mikulas Patocka <[email protected]>

---
 drivers/md/dm.c |    5 +++++
 1 file changed, 5 insertions(+)

Index: linux-2.6/drivers/md/dm.c
===================================================================

drivers/md/dm.c +0 -0

@@ -1682,6 +1682,11 @@ static void __split_and_process_bio(stru
 		}
 	}
 

Quoted 12 lines

drivers/md/dm.c static void __split_and_process_bio(stru · line 1692

Quoted 8 lines

drivers/md/dm.c +5 -0

+	if (ci.poll_cookie != BLK_QC_T_NONE) {
+		while (atomic_read(&ci.io->io_count) > 1 &&
+		       blk_poll(ci.poll_queue, ci.poll_cookie, true)) ;
+	}
+
 	/* drop the extra reference count */
 	dec_pending(ci.io, errno_to_blk_status(error));
 }

It seems that the general idea of your design is to
1) submit *one* split bio
2) blk_poll(), waiting the previously submitted split bio complets

No, I submit all the bios and poll for the last one.

Quoted 3 lines

and then submit next split bio, repeating the above process. I'm afraid the performance may be an issue here, since the batch every time blk_poll() reaps may decrease.

Could you benchmark it?

Quoted 34 lines · Mikulas Patocka

Quoted 20 lines · JeffleXu

Quoted 17 lines · Mikulas Patocka

Support I/O polling if submit_bio_noacct_mq_direct returned non-empty cookie.

Signed-off-by Mikulas Patocka <[email protected]>

---
  drivers/md/dm.c |    5 +++++
  1 file changed, 5 insertions(+)

Index: linux-2.6/drivers/md/dm.c
===================================================================

drivers/md/dm.c +0 -0

@@ -1682,6 +1682,11 @@ static void __split_and_process_bio(stru
  		}
  	}
 

Quoted 11 lines

drivers/md/dm.c static void __split_and_process_bio(stru · line 1692

Quoted 8 lines

drivers/md/dm.c +5 -0

+	if (ci.poll_cookie != BLK_QC_T_NONE) {
+		while (atomic_read(&ci.io->io_count) > 1 &&
+		       blk_poll(ci.poll_queue, ci.poll_cookie, true)) ;
+	}
+
  	/* drop the extra reference count */
  	dec_pending(ci.io, errno_to_blk_status(error));
  }

It seems that the general idea of your design is to
1) submit *one* split bio
2) blk_poll(), waiting the previously submitted split bio complets

Quoted 6 lines

No, I submit all the bios and poll for the last one.

Quoted 3 lines

and then submit next split bio, repeating the above process. I'm afraid the performance may be an issue here, since the batch every time blk_poll() reaps may decrease.

Could you benchmark it?

I only tested dm-linear.

The configuration (dm table) of dm-linear is:
0 1048576 linear /dev/nvme0n1 0
1048576 1048576 linear /dev/nvme2n1 0
2097152 1048576 linear /dev/nvme5n1 0

fio script used is:

```
$cat fio.conf
[global]
name=iouring-sqpoll-iopoll-1
ioengine=io_uring
iodepth=128
numjobs=1
thread
rw=randread
direct=1
registerfiles=1
hipri=1
runtime=10
time_based
group_reporting
randrepeat=0
filename=/dev/mapper/testdev
bs=4k

[job-1]
cpus_allowed=14
```

IOPS (IRQ mode) | IOPS (iopoll mode (hipri=1))
--------------- | --------------------

            213k |		   19k

At least, it doesn't work well with io_uring interface.

Quoted 74 lines · JeffleXu

Quoted 34 lines · Mikulas Patocka

Quoted 23 lines · JeffleXu

Quoted 20 lines · Mikulas Patocka

Support I/O polling if submit_bio_noacct_mq_direct returned non-empty cookie.

Signed-off-by Mikulas Patocka <[email protected]>

---
  drivers/md/dm.c |    5 +++++

  1 file changed, 5 insertions(+)

Index: linux-2.6/drivers/md/dm.c
===================================================================
--- linux-2.6.orig/drivers/md/dm.c    2021-03-02 19:26:34.000000000 +0100
+++ linux-2.6/drivers/md/dm.c    2021-03-02 19:26:34.000000000 +0100

drivers/md/dm.c +2 -0

@@ -1682,6 +1682,11 @@ static void __split_and_process_bio(stru
           }
       }
   +    if (ci.poll_cookie != BLK_QC_T_NONE) {
+        while (atomic_read(&ci.io->io_count) > 1 &&
+               blk_poll(ci.poll_queue, ci.poll_cookie, true)) ;

Quoted 8 lines

drivers/md/dm.c static void __split_and_process_bio(stru · line 1692

Quoted 5 lines

drivers/md/dm.c +2 -0

+    }
+
       /* drop the extra reference count */
       dec_pending(ci.io, errno_to_blk_status(error));
   }

It seems that the general idea of your design is to
1) submit *one* split bio
2) blk_poll(), waiting the previously submitted split bio complets

Quoted 6 lines

No, I submit all the bios and poll for the last one.

Quoted 3 lines

and then submit next split bio, repeating the above process. I'm afraid the performance may be an issue here, since the batch every time blk_poll() reaps may decrease.

Could you benchmark it?

I only tested dm-linear.

The configuration (dm table) of dm-linear is:
0 1048576 linear /dev/nvme0n1 0
1048576 1048576 linear /dev/nvme2n1 0
2097152 1048576 linear /dev/nvme5n1 0

fio script used is:

```
$cat fio.conf
[global]
name=iouring-sqpoll-iopoll-1
ioengine=io_uring
iodepth=128
numjobs=1
thread
rw=randread
direct=1
registerfiles=1
hipri=1
runtime=10
time_based
group_reporting
randrepeat=0
filename=/dev/mapper/testdev
bs=4k

[job-1]
cpus_allowed=14
```

Quoted 7 lines

IOPS (IRQ mode) | IOPS (iopoll mode (hipri=1))
--------------- | --------------------
            213k |           19k

At least, it doesn't work well with io_uring interface.

Jeffle,

I ran your above fio test on a linear LV split across 3 NVMes to second your split mapping (system: 32 core Intel, 256GiB RAM) comparing io engines sync, libaio and io_uring, the latter w/ and w/o hipri (sync+libaio obviously w/o registerfiles and hipri) which resulted ok:

sync  |  libaio  |  IRQ mode (hipri=0) | iopoll (hipri=1) ------|----------|---------------------|----------------- 56.3K |
290K  |                329K |             351K I can't second your drastic hipri=1 drop here...

Heinz

Quoted 96 lines · Heinz Mauelshagen

Quoted 75 lines · JeffleXu

Quoted 34 lines · Mikulas Patocka

Quoted 23 lines · JeffleXu

Quoted 20 lines · Mikulas Patocka

Support I/O polling if submit_bio_noacct_mq_direct returned non-empty cookie.

Signed-off-by Mikulas Patocka <[email protected]>

---
  drivers/md/dm.c |    5 +++++

  1 file changed, 5 insertions(+)

Index: linux-2.6/drivers/md/dm.c
===================================================================
--- linux-2.6.orig/drivers/md/dm.c    2021-03-02 19:26:34.000000000 +0100
+++ linux-2.6/drivers/md/dm.c    2021-03-02 19:26:34.000000000 +0100

drivers/md/dm.c +2 -0

@@ -1682,6 +1682,11 @@ static void __split_and_process_bio(stru
           }
       }
   +    if (ci.poll_cookie != BLK_QC_T_NONE) {
+        while (atomic_read(&ci.io->io_count) > 1 &&
+               blk_poll(ci.poll_queue, ci.poll_cookie, true)) ;

Quoted 8 lines

drivers/md/dm.c static void __split_and_process_bio(stru · line 1692

Quoted 5 lines

drivers/md/dm.c +2 -0

+    }
+
       /* drop the extra reference count */
       dec_pending(ci.io, errno_to_blk_status(error));
   }

It seems that the general idea of your design is to
1) submit *one* split bio
2) blk_poll(), waiting the previously submitted split bio complets

Quoted 7 lines

No, I submit all the bios and poll for the last one.

Quoted 4 lines

and then submit next split bio, repeating the above process. I'm afraid
the performance may be an issue here, since the batch every time blk_poll() reaps may decrease.

Could you benchmark it?

I only tested dm-linear.

The configuration (dm table) of dm-linear is:
0 1048576 linear /dev/nvme0n1 0
1048576 1048576 linear /dev/nvme2n1 0
2097152 1048576 linear /dev/nvme5n1 0

fio script used is:

```
$cat fio.conf
[global]
name=iouring-sqpoll-iopoll-1
ioengine=io_uring
iodepth=128
numjobs=1
thread
rw=randread
direct=1
registerfiles=1
hipri=1
runtime=10
time_based
group_reporting
randrepeat=0
filename=/dev/mapper/testdev
bs=4k

[job-1]
cpus_allowed=14
```

Quoted 7 lines

IOPS (IRQ mode) | IOPS (iopoll mode (hipri=1))
--------------- | --------------------
            213k |           19k

At least, it doesn't work well with io_uring interface.

Jeffle,

I ran your above fio test on a linear LV split across 3 NVMes to second your split mapping
(system: 32 core Intel, 256GiB RAM) comparing io engines sync, libaio and io_uring,
the latter w/ and w/o hipri (sync+libaio obviously w/o registerfiles and hipri) which resulted ok:

Quoted 4 lines

sync  |  libaio  |  IRQ mode (hipri=0) | iopoll (hipri=1) ------|----------|---------------------|----------------- 56.3K |    290K  |                329K |             351K I can't second your drastic hipri=1 drop here...

Sorry, email mess.

sync   |  libaio  |  IRQ mode (hipri=0) | iopoll (hipri=1) -------|----------|---------------------|----------------- 56.3K  |    290K  |                329K |             351K

I can't second your drastic hipri=1 drop here...

Quoted 107 lines · Heinz Mauelshagen

Quoted 92 lines · Heinz Mauelshagen

Quoted 72 lines · JeffleXu

Quoted 32 lines · Mikulas Patocka

Quoted 22 lines · JeffleXu

Quoted 20 lines · Mikulas Patocka

Support I/O polling if submit_bio_noacct_mq_direct returned non-empty cookie.

Signed-off-by Mikulas Patocka <[email protected]>

---
   drivers/md/dm.c |    5 +++++
   1 file changed, 5 insertions(+)

Index: linux-2.6/drivers/md/dm.c
===================================================================
--- linux-2.6.orig/drivers/md/dm.c    2021-03-02 19:26:34.000000000 +0100
+++ linux-2.6/drivers/md/dm.c    2021-03-02 19:26:34.000000000 +0100

(quoted diff) +2 -0

@@ -1682,6 +1682,11 @@ static void __split_and_process_bio(stru
           }
       }
   +    if (ci.poll_cookie != BLK_QC_T_NONE) {
+        while (atomic_read(&ci.io->io_count) > 1 &&
+               blk_poll(ci.poll_queue, ci.poll_cookie, true)) ;

Quoted 8 lines

Review static void __split_and_process_bio(stru · line 1691

Quoted 5 lines

(quoted diff) +2 -0

+    }
+
       /* drop the extra reference count */
       dec_pending(ci.io, errno_to_blk_status(error));
   }

It seems that the general idea of your design is to
1) submit *one* split bio
2) blk_poll(), waiting the previously submitted split bio complets

Quoted 7 lines

No, I submit all the bios and poll for the last one.

Quoted 4 lines

and then submit next split bio, repeating the above process. I'm afraid
the performance may be an issue here, since the batch every time blk_poll() reaps may decrease.

Could you benchmark it?

I only tested dm-linear.

The configuration (dm table) of dm-linear is:
0 1048576 linear /dev/nvme0n1 0
1048576 1048576 linear /dev/nvme2n1 0
2097152 1048576 linear /dev/nvme5n1 0

fio script used is:

```
$cat fio.conf
[global]
name=iouring-sqpoll-iopoll-1
ioengine=io_uring
iodepth=128
numjobs=1
thread
rw=randread
direct=1
registerfiles=1
hipri=1
runtime=10
time_based
group_reporting
randrepeat=0
filename=/dev/mapper/testdev
bs=4k

[job-1]
cpus_allowed=14
```

Quoted 7 lines

IOPS (IRQ mode) | IOPS (iopoll mode (hipri=1))
--------------- | --------------------
            213k |           19k

At least, it doesn't work well with io_uring interface.

Jeffle,

I ran your above fio test on a linear LV split across 3 NVMes to second your split mapping
(system: 32 core Intel, 256GiB RAM) comparing io engines sync, libaio and io_uring,
the latter w/ and w/o hipri (sync+libaio obviously w/o registerfiles and hipri) which resulted ok:

Quoted 4 lines

sync  |  libaio  |  IRQ mode (hipri=0) | iopoll (hipri=1) ------|----------|---------------------|----------------- 56.3K |    290K  |                329K |             351K I can't second your drastic hipri=1 drop here...

Sorry, email mess.

sync   |  libaio  |  IRQ mode (hipri=0) | iopoll (hipri=1) -------|----------|---------------------|----------------- 56.3K  |    290K  |                329K |             351K

Quoted 1 line

I can't second your drastic hipri=1 drop here...

I think your result is just showcasing your powerful system's ability to poll every related HW queue.. whereas Jeffle's system is likely somehow more constrained (on a cpu level, memory, whatever).

My basis for this is that Mikulas' changes simply always return an invalid cookie (BLK_QC_T_NONE) for purposes of intelligent IO polling.

Such an implementation is completely invalid.

I discussed with Jens and he said:
"it needs to return something that f_op->iopoll() can make sense of. otherwise you have no option but to try everything."

Mike

--
dm-devel mailing list
[email protected]
https://listman.redhat.com/mailman/listinfo/dm-devel

Quoted 96 lines · Heinz Mauelshagen

Quoted 75 lines · JeffleXu

Quoted 34 lines · Mikulas Patocka

Quoted 23 lines · JeffleXu

Quoted 20 lines · Mikulas Patocka

Support I/O polling if submit_bio_noacct_mq_direct returned non-empty cookie.

Signed-off-by Mikulas Patocka <[email protected]>

---
  drivers/md/dm.c |    5 +++++

  1 file changed, 5 insertions(+)

Index: linux-2.6/drivers/md/dm.c
===================================================================
--- linux-2.6.orig/drivers/md/dm.c    2021-03-02 19:26:34.000000000 +0100
+++ linux-2.6/drivers/md/dm.c    2021-03-02 19:26:34.000000000 +0100

drivers/md/dm.c +2 -0

@@ -1682,6 +1682,11 @@ static void __split_and_process_bio(stru
           }
       }
   +    if (ci.poll_cookie != BLK_QC_T_NONE) {
+        while (atomic_read(&ci.io->io_count) > 1 &&
+               blk_poll(ci.poll_queue, ci.poll_cookie, true)) ;

Quoted 8 lines

drivers/md/dm.c static void __split_and_process_bio(stru · line 1692

Quoted 5 lines

drivers/md/dm.c +2 -0

+    }
+
       /* drop the extra reference count */
       dec_pending(ci.io, errno_to_blk_status(error));
   }

It seems that the general idea of your design is to
1) submit *one* split bio
2) blk_poll(), waiting the previously submitted split bio complets

Quoted 7 lines

No, I submit all the bios and poll for the last one.

Quoted 4 lines

and then submit next split bio, repeating the above process. I'm afraid
the performance may be an issue here, since the batch every time blk_poll() reaps may decrease.

Could you benchmark it?

I only tested dm-linear.

The configuration (dm table) of dm-linear is:
0 1048576 linear /dev/nvme0n1 0
1048576 1048576 linear /dev/nvme2n1 0
2097152 1048576 linear /dev/nvme5n1 0

fio script used is:

```
$cat fio.conf
[global]
name=iouring-sqpoll-iopoll-1
ioengine=io_uring
iodepth=128
numjobs=1
thread
rw=randread
direct=1
registerfiles=1
hipri=1
runtime=10
time_based
group_reporting
randrepeat=0
filename=/dev/mapper/testdev
bs=4k

[job-1]
cpus_allowed=14
```

Quoted 7 lines

IOPS (IRQ mode) | IOPS (iopoll mode (hipri=1))
--------------- | --------------------
            213k |           19k

At least, it doesn't work well with io_uring interface.

Jeffle,

I ran your above fio test on a linear LV split across 3 NVMes to second your split mapping
(system: 32 core Intel, 256GiB RAM) comparing io engines sync, libaio and io_uring,
the latter w/ and w/o hipri (sync+libaio obviously w/o registerfiles and hipri) which resulted ok:

Quoted 4 lines

sync  |  libaio  |  IRQ mode (hipri=0) | iopoll (hipri=1) ------|----------|---------------------|----------------- 56.3K | 290K  |                329K |             351K I can't second your drastic hipri=1 drop here...

Sorry, email mess.

sync   |  libaio  |  IRQ mode (hipri=0) | iopoll (hipri=1) -------|----------|---------------------|----------------- 56.3K  |    290K  |                329K |             351K

I can't second your drastic hipri=1 drop here...

Quoted 107 lines · Heinz Mauelshagen

Quoted 96 lines · Heinz Mauelshagen

Quoted 75 lines · JeffleXu

Quoted 34 lines · Mikulas Patocka

Quoted 23 lines · JeffleXu

Quoted 20 lines · Mikulas Patocka

Support I/O polling if submit_bio_noacct_mq_direct returned non-empty cookie.

Signed-off-by Mikulas Patocka <[email protected]>

---
  drivers/md/dm.c |    5 +++++

  1 file changed, 5 insertions(+)

Index: linux-2.6/drivers/md/dm.c
===================================================================
--- linux-2.6.orig/drivers/md/dm.c    2021-03-02 19:26:34.000000000 +0100
+++ linux-2.6/drivers/md/dm.c    2021-03-02 19:26:34.000000000 +0100

(quoted diff) +2 -0

@@ -1682,6 +1682,11 @@ static void __split_and_process_bio(stru
           }
       }
   +    if (ci.poll_cookie != BLK_QC_T_NONE) {
+        while (atomic_read(&ci.io->io_count) > 1 &&
+               blk_poll(ci.poll_queue, ci.poll_cookie, true)) ;

Quoted 8 lines

Review static void __split_and_process_bio(stru · line 1691

Quoted 5 lines

(quoted diff) +2 -0

+    }
+
       /* drop the extra reference count */
       dec_pending(ci.io, errno_to_blk_status(error));
   }

It seems that the general idea of your design is to
1) submit *one* split bio
2) blk_poll(), waiting the previously submitted split bio complets

Quoted 7 lines

No, I submit all the bios and poll for the last one.

Quoted 4 lines

and then submit next split bio, repeating the above process. I'm afraid
the performance may be an issue here, since the batch every time blk_poll() reaps may decrease.

Could you benchmark it?

I only tested dm-linear.

The configuration (dm table) of dm-linear is:
0 1048576 linear /dev/nvme0n1 0
1048576 1048576 linear /dev/nvme2n1 0
2097152 1048576 linear /dev/nvme5n1 0

fio script used is:

```
$cat fio.conf
[global]
name=iouring-sqpoll-iopoll-1
ioengine=io_uring
iodepth=128
numjobs=1
thread
rw=randread
direct=1
registerfiles=1
hipri=1
runtime=10
time_based
group_reporting
randrepeat=0
filename=/dev/mapper/testdev
bs=4k

[job-1]
cpus_allowed=14
```

Quoted 7 lines

IOPS (IRQ mode) | IOPS (iopoll mode (hipri=1))
--------------- | --------------------
            213k |           19k

At least, it doesn't work well with io_uring interface.

Jeffle,

I ran your above fio test on a linear LV split across 3 NVMes to second your split mapping
(system: 32 core Intel, 256GiB RAM) comparing io engines sync, libaio and io_uring,
the latter w/ and w/o hipri (sync+libaio obviously w/o registerfiles and hipri) which resulted ok:

Quoted 4 lines

sync  |  libaio  |  IRQ mode (hipri=0) | iopoll (hipri=1) ------|----------|---------------------|----------------- 56.3K | 290K  |                329K |             351K I can't second your drastic hipri=1 drop here...

Sorry, email mess.

Quoted 8 lines

sync   |  libaio  |  IRQ mode (hipri=0) | iopoll (hipri=1) -------|----------|---------------------|----------------- 56.3K  |    290K  |                329K |             351K

I can't second your drastic hipri=1 drop here...

Hummm, that's indeed somewhat strange...

My test environment:
- CPU: 128 cores, though only one CPU core is used since 'cpus_allowed=14' in fio configuration
- memory: 983G memory free
- NVMe: Huawai ES3510P (HWE52P434T0L005N), with 'nvme.poll_queues=3'

Maybe you didn't specify 'nvme.poll_queues=XXX'? In this case, IO still goes into IRQ mode, even you have specified 'hipri=1'?

Quoted 101 lines · Heinz Mauelshagen

Quoted 92 lines · Heinz Mauelshagen

Quoted 72 lines · JeffleXu

Quoted 32 lines · Mikulas Patocka

Quoted 22 lines · JeffleXu

Quoted 20 lines · Mikulas Patocka

Support I/O polling if submit_bio_noacct_mq_direct returned non-empty cookie.

Signed-off-by Mikulas Patocka <[email protected]>

---
   drivers/md/dm.c |    5 +++++
   1 file changed, 5 insertions(+)

Index: linux-2.6/drivers/md/dm.c
===================================================================
--- linux-2.6.orig/drivers/md/dm.c    2021-03-02 19:26:34.000000000 +0100
+++ linux-2.6/drivers/md/dm.c    2021-03-02 19:26:34.000000000 +0100

(quoted diff) +2 -0

@@ -1682,6 +1682,11 @@ static void __split_and_process_bio(stru
           }
       }
   +    if (ci.poll_cookie != BLK_QC_T_NONE) {
+        while (atomic_read(&ci.io->io_count) > 1 &&
+               blk_poll(ci.poll_queue, ci.poll_cookie, true)) ;

Quoted 8 lines

Review static void __split_and_process_bio(stru · line 1691

Quoted 5 lines

(quoted diff) +2 -0

+    }
+
       /* drop the extra reference count */
       dec_pending(ci.io, errno_to_blk_status(error));
   }

It seems that the general idea of your design is to
1) submit *one* split bio
2) blk_poll(), waiting the previously submitted split bio complets

Quoted 7 lines

No, I submit all the bios and poll for the last one.

Quoted 4 lines

and then submit next split bio, repeating the above process. I'm afraid
the performance may be an issue here, since the batch every time blk_poll() reaps may decrease.

Could you benchmark it?

I only tested dm-linear.

The configuration (dm table) of dm-linear is:
0 1048576 linear /dev/nvme0n1 0
1048576 1048576 linear /dev/nvme2n1 0
2097152 1048576 linear /dev/nvme5n1 0

fio script used is:

```
$cat fio.conf
[global]
name=iouring-sqpoll-iopoll-1
ioengine=io_uring
iodepth=128
numjobs=1
thread
rw=randread
direct=1
registerfiles=1
hipri=1
runtime=10
time_based
group_reporting
randrepeat=0
filename=/dev/mapper/testdev
bs=4k

[job-1]
cpus_allowed=14
```

Quoted 7 lines

IOPS (IRQ mode) | IOPS (iopoll mode (hipri=1))
--------------- | --------------------
            213k |           19k

At least, it doesn't work well with io_uring interface.

Jeffle,

I ran your above fio test on a linear LV split across 3 NVMes to second your split mapping
(system: 32 core Intel, 256GiB RAM) comparing io engines sync, libaio and io_uring,
the latter w/ and w/o hipri (sync+libaio obviously w/o registerfiles and hipri) which resulted ok:

Quoted 4 lines

sync  |  libaio  |  IRQ mode (hipri=0) | iopoll (hipri=1) ------|----------|---------------------|----------------- 56.3K | 290K  |                329K |             351K I can't second your drastic hipri=1 drop here...

Sorry, email mess.

Quoted 8 lines

sync   |  libaio  |  IRQ mode (hipri=0) | iopoll (hipri=1) -------|----------|---------------------|----------------- 56.3K  |    290K  |                329K |             351K

I can't second your drastic hipri=1 drop here...

Hummm, that's indeed somewhat strange...

My test environment:
- CPU: 128 cores, though only one CPU core is used since 'cpus_allowed=14' in fio configuration
- memory: 983G memory free
- NVMe: Huawai ES3510P (HWE52P434T0L005N), with 'nvme.poll_queues=3'

Maybe you didn't specify 'nvme.poll_queues=XXX'? In this case, IO still goes into IRQ mode, even you have specified 'hipri=1'?