block/dm: support bio polling
Subsystems (2)
v6 cover block/dm: support bio polling
Hi,
I've rebased Ming's latest [1] ontop of dm-5.18 [2] (which is based on for-5.18/block). End result available in dm-5.18-biopoll branch [3]
These changes add bio polling support to DM. Tested with linear and striped DM targets.
IOPS improvement was ~5% on my baremetal system with a single Intel Optane NVMe device (561K hipri=1 vs 530K hipri=0).
Ming has seen better improvement while testing within a VM:
dm-linear: hipri=1 vs hipri=0 15~20% iops improvement
dm-stripe: hipri=1 vs hipri=0 ~30% iops improvement
I'd like to merge these changes via the DM tree when the 5.18 merge window opens. The first block patch that adds ->poll_bio to block_device_operations will need review so that I can take it through the DM tree. Reason for going through the DM tree is there have been some fairly extensive changes queued in dm-5.18 that build on for-5.18/block. So I think it easiest to just add the block depenency via DM tree since DM is first consumer of ->poll_bio
Thanks,
Mike
[1] https://github.com/ming1/linux/commits/my_v5.18-dm-bio-poll
[2] https://git.kernel.org/pub/scm/linux/kernel/git/device-mapper/linux-dm.git/log/?h=dm-5.18
[3] https://git.kernel.org/pub/scm/linux/kernel/git/device-mapper/linux-dm.git/log/?h=dm-5.18-biopoll
[4] https://github.com/ming1/linux/commit/c107c30e15041ac1ce672f56809961406e2a3e52
v6: Ming switched from reusing .bi_end_io to .bi_private and added a
comment to comment-block above dm_get_bio_hlist_head as suggested by Jens.
v5: remove WARN_ONs in ->poll_bio interface patch. Fixed comment typo
along the way (found while seeing how other block_device_operations are referenced in block's code comments).
Ming Lei (2):
block: add ->poll_bio to block_device_operations
dm: support bio polling
block/blk-core.c | 14 +++-- block/genhd.c | 4 ++ drivers/md/dm-core.h | 2 + drivers/md/dm-table.c | 27 ++++++++++ drivers/md/dm.c | 143 +++++++++++++++++++++++++++++++++++++++++++++++-- include/linux/blkdev.h | 2 + 6 files changed, 184 insertions(+), 8 deletions(-)
Signature · 1 line
2.15.0
PATCH v6 2/2 dm: support bio polling
From: Ming Lei <[email protected]>
Support bio(REQ_POLLED) polling in the following approach:
1) only support io polling on normal READ/WRITE, and other abnormal IOs still fallback to IRQ mode, so the target io is exactly inside the dm io.
2) hold one refcnt on io->io_count after submitting this dm bio with REQ_POLLED
3) support dm native bio splitting, any dm io instance associated with current bio will be added into one list which head is bio->bi_private which will be recovered before ending this bio
4) implement .poll_bio() callback, call bio_poll() on the single target bio inside the dm io which is retrieved via bio->bi_bio_drv_data; call dm_io_dec_pending() after the target io is done in .poll_bio()
5) enable QUEUE_FLAG_POLL if all underlying queues enable QUEUE_FLAG_POLL, which is based on Jeffle's previous patch.
--- drivers/md/dm-core.h | 2 + drivers/md/dm-table.c | 27 ++++++++++ drivers/md/dm.c | 143 ++++++++++++++++++++++++++++++++++++++++++++++++-- 3 files changed, 169 insertions(+), 3 deletions(-)
drivers/md/dm-core.h +2 -0
@@ -235,6 +235,8 @@ struct dm_io { bool start_io_acct:1;int was_accounted;unsigned long start_time; +void *data; +struct hlist_node node;spinlock_t endio_lock;struct dm_stats_aux stats_aux;/* last member of dm_target_io is 'struct bio' */
drivers/md/dm-table.c +27 -0
@@ -1481,6 +1481,14 @@ struct dm_target *dm_table_find_target(struct dm_table *t, sector_t sector)return &t->targets[(KEYS_PER_NODE * n) + k]; } +static int device_not_poll_capable (struct dm_target *ti,struct dm_dev *dev, +sector_t start,sector_t len,void *data) +{ +struct request_queue *q =bdev_get_queue (dev->bdev); + +return !test_bit (QUEUE_FLAG_POLL , &q->queue_flags); +} +/* * type->iterate_devices() should be called when the sanity check needs to * iterate and check all underlying data devices. iterate_devices() will
@@ -1531,6 +1539,11 @@ static int count_device(struct dm_target *ti, struct dm_dev *dev,return 0 ; } +static int dm_table_supports_poll (struct dm_table *t) +{ +return !dm_table_any_dev_attr (t, device_not_poll_capable,NULL ); +} +/* * Check whether a table has no data devices attached using each * target's iterate_devices method.
@@ -2067,6 +2080,20 @@ int dm_table_set_restrictions(struct dm_table *t, struct request_queue *q,dm_update_crypto_profile (q, t);disk_update_readahead (t->md->disk); +/* +* Check for request-based device is left to +* dm_mq_init_request_queue()->blk_mq_init_allocated_queue(). +* +* For bio-based device, only set QUEUE_FLAG_POLL when all +* underlying devices supporting polling. +*/ +if (__table_type_bio_based (t->type)) { +if (dm_table_supports_poll (t)) +blk_queue_flag_set (QUEUE_FLAG_POLL , q); +else +blk_queue_flag_clear (QUEUE_FLAG_POLL , q); + } +return 0 ; }
drivers/md/dm.c +140 -3
204 lines not on this page · raw message
Signature · 1 line
2.15.0
PATCH v6 1/2 block: add ->poll_bio to block_device_operations
In reply to Mike SnitzerFrom: Ming Lei <[email protected]>
Prepare for supporting IO polling for bio-based driver.
Add ->poll_bio callback so that bio-based driver can provide their own logic for polling bio.
Also fix ->submit_bio_bio typo in comment block above __submit_bio_noacct.
--- block/blk-core.c | 14 +++++++++----- block/genhd.c | 4 ++++ include/linux/blkdev.h | 2 ++ 3 files changed, 15 insertions(+), 5 deletions(-)
block/blk-core.c +9 -5
@@ -708,7 +708,7 @@ static void __submit_bio(struct bio *bio)* * bio_list_on_stack[0] contains bios submitted by the current ->submit_bio. * bio_list_on_stack[1] contains bios that were submitted before the current -* ->submit_bio_bio, but that haven't been processed yet. +* ->submit_bio, but that haven't been processed yet. */ static void __submit_bio_noacct (struct bio *bio) {
@@ -975,7 +975,7 @@ int bio_poll(struct bio *bio, struct io_comp_batch *iob, unsigned int flags) {struct request_queue *q =bdev_get_queue (bio->bi_bdev);blk_qc_t cookie =READ_ONCE (bio->bi_cookie); -int ret; +int ret =0 ;if (cookie ==BLK_QC_T_NONE || !test_bit (QUEUE_FLAG_POLL , &q->queue_flags))
@@ -985,10 +985,14 @@ int bio_poll(struct bio *bio, struct io_comp_batch *iob, unsigned int flags)if (blk_queue_enter (q,BLK_MQ_REQ_NOWAIT ))return 0 ; -if (WARN_ON_ONCE (!queue_is_mq (q))) - ret =0 ;/* not yet implemented, should not happen */ -else +if (queue_is_mq (q)) { ret =blk_mq_poll (q, cookie, iob, flags); + }else { +struct gendisk *disk = q->disk; + +if (disk && disk->fops->poll_bio) + ret = disk->fops->poll_bio (bio, iob, flags); + }blk_queue_exit (q);return ret; }
block/genhd.c +4 -0
@@ -410,6 +410,10 @@ int __must_check device_add_disk(struct device *parent, struct gendisk *disk,struct device *ddev =disk_to_dev (disk);int ret; +/* Only makes sense for bio-based to set ->poll_bio */ +if (queue_is_mq (disk->queue) && disk->fops->poll_bio) +return -EINVAL ; +/* * The disk queue should now be all set with enough information about * the device for the elevator code to pick an adequate default
include/linux/blkdev.h +2 -0
@@ -1455,6 +1455,8 @@ enum blk_unique_id {struct block_device_operations {void (*submit_bio)(struct bio *bio); +int (*poll_bio)(struct bio *bio,struct io_comp_batch *iob, +unsigned int flags);int (*open) (struct block_device *,fmode_t );void (*release) (struct gendisk *,fmode_t );int (*rw_page)(struct block_device *,sector_t ,struct page *,unsigned int );
Signature · 6 lines
2.15.0 -- dm-devel mailing list [email protected] https://listman.redhat.com/mailman/listinfo/dm-devel
Re: [PATCH v6 1/2] block: add ->poll_bio to block_device_operations
Quoted 9 lines · Mike Snitzer
From: Ming Lei <[email protected]>
Prepare for supporting IO polling for bio-based driver.
Add ->poll_bio callback so that bio-based driver can provide their own logic for polling bio.
Also fix ->submit_bio_bio typo in comment block above
__submit_bio_noacct.
Assuming you want to take this through the dm tree:
Signature · 6 lines
Jens Axboe -- dm-devel mailing list [email protected] https://listman.redhat.com/mailman/listinfo/dm-devel
Jens Axboe
maintainerRe: [PATCH v6 2/2] dm: support bio polling
In reply to Mike SnitzerQuoted 15 lines · Mike Snitzer
From: Ming Lei <[email protected]>
Support
bio(REQ_POLLED)polling in the following approach:1) only support io polling on normal READ/WRITE, and other abnormal IOs still fallback to IRQ mode, so the target io is exactly inside the dm io.
2) hold one refcnt on
io->io_countafter submitting this dm bio withREQ_POLLED3) support dm native bio splitting, any dm io instance associated with current bio will be added into one list which head is
bio->bi_privatewhich will be recovered before ending this bio
Quoted 6 lines
4) implement .poll_bio() callback, call
bio_poll()on the single target bio inside the dm io which is retrieved viabio->bi_bio_drv_data; calldm_io_dec_pending()after the target io is done in .poll_bio()5) enable
QUEUE_FLAG_POLLif all underlying queues enableQUEUE_FLAG_POLL, which is based on Jeffle's previous patch.
It's not the prettiest thing in the world with the overlay on bi_private, but at least it's nicely documented now.
I would encourage you to actually test this on fast storage, should make a nice difference. I can run this on a gen2 optane, it's 10x the IOPS of what it was tested on and should help better highlight where it makes a difference.
If either of you would like that, then send me a fool proof recipe for what should be setup so I have a poll capable dm device.
Signature · 6 lines
Jens Axboe -- dm-devel mailing list [email protected] https://listman.redhat.com/mailman/listinfo/dm-devel
Re: [PATCH v6 2/2] dm: support bio polling
Quoted 26 lines · Jens Axboe
Quoted 15 lines · Mike Snitzer
From: Ming Lei <[email protected]>
Support
bio(REQ_POLLED)polling in the following approach:1) only support io polling on normal READ/WRITE, and other abnormal IOs still fallback to IRQ mode, so the target io is exactly inside the dm io.
2) hold one refcnt on
io->io_countafter submitting this dm bio withREQ_POLLED3) support dm native bio splitting, any dm io instance associated with current bio will be added into one list which head is
bio->bi_privatewhich will be recovered before ending this bioQuoted 6 lines
4) implement .poll_bio() callback, call
bio_poll()on the single target bio inside the dm io which is retrieved viabio->bi_bio_drv_data; calldm_io_dec_pending()after the target io is done in .poll_bio()5) enable
QUEUE_FLAG_POLLif all underlying queues enableQUEUE_FLAG_POLL, which is based on Jeffle's previous patch.It's not the prettiest thing in the world with the overlay on
bi_private, but at least it's nicely documented now.
Quoted 7 lines
I would encourage you to actually test this on fast storage, should make a nice difference. I can run this on a gen2 optane, it's 10x the IOPS of what it was tested on and should help better highlight where it makes a difference.
If either of you would like that, then send me a fool proof recipe for what should be setup so I have a poll capable dm device.
Follows steps for setup dm stripe over two nvmes, then run io_uring on the dm stripe dev.
1) dm_stripe.perl
#!/usr/bin/perl -w
# Create a striped device across any number of underlying devices. The device
# will be called "stripe_dev" and have a chunk-size of 128k.
my $chunk_size = 128 * 2;
my $dev_name = "stripe_dev";
my $num_devs = @ARGV;
my @devs = @ARGV;
my ($min_dev_size, $stripe_dev_size, $i);
if (!$num_devs) {
die("Specify at least one device\n");
}$min_dev_size = blockdev --getsz $devs[0];
for ($i = 1; $i < $num_devs; $i++) {
my $this_size = `blockdev --getsz $devs[$i]`;
$min_dev_size = ($min_dev_size < $this_size) ?
$min_dev_size : $this_size;
}$stripe_dev_size = $min_dev_size * $num_devs; $stripe_dev_size -= $stripe_dev_size % ($chunk_size * $num_devs);
$table = "0 $stripe_dev_size striped $num_devs $chunk_size";
for ($i = 0; $i < $num_devs; $i++) {
$table .= " $devs[$i] 0";
}echo $table | dmsetup create $dev_name;
2) test_poll_on_dm_stripe.sh
#!/bin/bash
RT=40
JOBS=1
HI=1
BS=4K
set -x
dmsetup remove_all
rmmod nvme
modprobe nvme poll_queues=2
sleep 2
./dm_stripe.perl /dev/nvme0n1 /dev/nvme1n1
sleep 1
DEV=/dev/mapper/stripe_dev
echo "io_uring hipri test"
fio --bs=$BS --ioengine=io_uring --fixedbufs --registerfiles \
--hipri=$HI --iodepth=64 --iodepth_batch_submit=16 --iodepth_batch_complete_min=16 \
--filename=$DEV --direct=1 --runtime=$RT --numjobs=$JOBS --rw=randread --name=test \
--group_reportingThanks,
Ming
--
dm-devel mailing list
[email protected]
https://listman.redhat.com/mailman/listinfo/dm-devel
Jens Axboe
maintainerRe: [PATCH v6 2/2] dm: support bio polling
Quoted 32 lines · Ming Lei
Quoted 31 lines · Jens Axboe
Quoted 15 lines · Mike Snitzer
From: Ming Lei <[email protected]>
Support
bio(REQ_POLLED)polling in the following approach:1) only support io polling on normal READ/WRITE, and other abnormal IOs still fallback to IRQ mode, so the target io is exactly inside the dm io.
2) hold one refcnt on
io->io_countafter submitting this dm bio withREQ_POLLED3) support dm native bio splitting, any dm io instance associated with current bio will be added into one list which head is
bio->bi_privatewhich will be recovered before ending this bioQuoted 6 lines
4) implement .poll_bio() callback, call
bio_poll()on the single target bio inside the dm io which is retrieved viabio->bi_bio_drv_data; calldm_io_dec_pending()after the target io is done in .poll_bio()5) enable
QUEUE_FLAG_POLLif all underlying queues enableQUEUE_FLAG_POLL, which is based on Jeffle's previous patch.It's not the prettiest thing in the world with the overlay on
bi_private, but at least it's nicely documented now.I would encourage you to actually test this on fast storage, should make a nice difference. I can run this on a gen2 optane, it's 10x the IOPS of what it was tested on and should help better highlight where it makes a difference.
Quoted 5 lines
Quoted 2 lines
If either of you would like that, then send me a fool proof recipe for what should be setup so I have a poll capable dm device.
Follows steps for setup dm stripe over two nvmes, then run
io_uringon the dm stripe dev.
Thanks! Much easier when I don't have to figure it out... Setup:
CPU: 12900K
Drives: 2x P5800X gen2 optane (~5M IOPS each at 512b)
Baseline kernel:
sudo taskset -c 10 t/io_uring -d128 -b512 -s31 -c16 -p1 -F1 -B1 -n1 -R1 -X1 /dev/dm-0 Added file /dev/dm-0 (submitter 0)
polled=1, fixedbufs=1/0, register_files=1, buffered=0, QD=128 Engine=io_uring, sq_ring=128, cq_ring=128
submitter=0, tid=1004
IOPS=2794K, BW=1364MiB/s, IOS/call=31/30, inflight=(124)
IOPS=2793K, BW=1363MiB/s, IOS/call=31/31, inflight=(62)
IOPS=2789K, BW=1362MiB/s, IOS/call=31/30, inflight=(124)
IOPS=2779K, BW=1357MiB/s, IOS/call=31/31, inflight=(124)
IOPS=2780K, BW=1357MiB/s, IOS/call=31/31, inflight=(62)
IOPS=2779K, BW=1357MiB/s, IOS/call=31/31, inflight=(62)
^CExiting on signal
Maximum IOPS=2794K
generating about 500K ints/sec, and using 4k blocks:
sudo taskset -c 10 t/io_uring -d128 -b4096 -s31 -c16 -p1 -F1 -B1 -n1 -R1 -X1 /dev/dm-0 Added file /dev/dm-0 (submitter 0)
polled=1, fixedbufs=1/0, register_files=1, buffered=0, QD=128 Engine=io_uring, sq_ring=128, cq_ring=128
submitter=0, tid=967
IOPS=1683K, BW=6575MiB/s, IOS/call=24/24, inflight=(93)
IOPS=1685K, BW=6584MiB/s, IOS/call=24/24, inflight=(124)
IOPS=1686K, BW=6588MiB/s, IOS/call=24/24, inflight=(124)
IOPS=1684K, BW=6581MiB/s, IOS/call=24/24, inflight=(93)
IOPS=1686K, BW=6589MiB/s, IOS/call=24/24, inflight=(124)
IOPS=1687K, BW=6593MiB/s, IOS/call=24/24, inflight=(128)
IOPS=1687K, BW=6590MiB/s, IOS/call=24/24, inflight=(93)
^CExiting on signal
Maximum IOPS=1687K
which ends up being bw limited for me, because the devices aren't linked gen4. That's about 1.4M ints/sec.
With the patched kernel, same test:
sudo taskset -c 10 t/io_uring -d128 -b512 -s31 -c16 -p1 -F1 -B1 -n1 -R1 -X1 /dev/dm-0 Added file /dev/dm-0 (submitter 0)
polled=1, fixedbufs=1/0, register_files=1, buffered=0, QD=128 Engine=io_uring, sq_ring=128, cq_ring=128
submitter=0, tid=989
IOPS=4151K, BW=2026MiB/s, IOS/call=16/15, inflight=(128)
IOPS=4159K, BW=2031MiB/s, IOS/call=15/15, inflight=(128)
IOPS=4193K, BW=2047MiB/s, IOS/call=15/15, inflight=(128)
IOPS=4191K, BW=2046MiB/s, IOS/call=15/15, inflight=(128)
IOPS=4202K, BW=2052MiB/s, IOS/call=15/15, inflight=(128)
^CExiting on signal
Maximum IOPS=4202K
with basically zero interrupts, and 4k:
sudo taskset -c 10 t/io_uring -d128 -b4096 -s31 -c16 -p1 -F1 -B1 -n1 -R1 -X1 /dev/dm-0 Added file /dev/dm-0 (submitter 0)
polled=1, fixedbufs=1/0, register_files=1, buffered=0, QD=128 Engine=io_uring, sq_ring=128, cq_ring=128
submitter=0, tid=1015
IOPS=1706K, BW=6666MiB/s, IOS/call=15/15, inflight=(128)
IOPS=1704K, BW=6658MiB/s, IOS/call=15/15, inflight=(128)
IOPS=1704K, BW=6658MiB/s, IOS/call=15/15, inflight=(128)
IOPS=1704K, BW=6658MiB/s, IOS/call=15/15, inflight=(128)
IOPS=1704K, BW=6658MiB/s, IOS/call=15/15, inflight=(128)
^CExiting on signal
Maximum IOPS=1706K
again with basically zero interrupts.
That's about a 50% improvement for polled IO. This is using 2 gen2 optanes, which are good for ~5M IOPS each. Using two threads on a single core, baseline kernel:
sudo taskset -c 10,11 t/io_uring -d128 -b512 -s31 -c16 -p1 -F1 -B1 -n2 -R1 -X1 /dev/dm-0 Added file /dev/dm-0 (submitter 0)
Added file /dev/dm-0 (submitter 1)
polled=1, fixedbufs=1/0, register_files=1, buffered=0, QD=128 Engine=io_uring, sq_ring=128, cq_ring=128
submitter=0, tid=1081
submitter=1, tid=1082
IOPS=3515K, BW=1716MiB/s, IOS/call=31/30, inflight=(124 62)
IOPS=3515K, BW=1716MiB/s, IOS/call=31/31, inflight=(62 124)
IOPS=3517K, BW=1717MiB/s, IOS/call=30/30, inflight=(113 124)
IOPS=3517K, BW=1717MiB/s, IOS/call=31/31, inflight=(62 62)
^CExiting on signal
Maximum IOPS=3517K
and patched:
udo taskset -c 10,11 t/io_uring -d128 -b512 -s31 -c16 -p1 -F1 -B1 -n2 -R1 -X1 /dev/dm-0 Added file /dev/dm-0 (submitter 0)
Added file /dev/dm-0 (submitter 1)
polled=1, fixedbufs=1/0, register_files=1, buffered=0, QD=128 Engine=io_uring, sq_ring=128, cq_ring=128
submitter=0, tid=949
submitter=1, tid=950
IOPS=4988K, BW=2435MiB/s, IOS/call=15/15, inflight=(128 128)
IOPS=4985K, BW=2434MiB/s, IOS/call=15/15, inflight=(128 128)
IOPS=4970K, BW=2426MiB/s, IOS/call=15/15, inflight=(128 128)
IOPS=4985K, BW=2434MiB/s, IOS/call=15/15, inflight=(128 128)
^CExiting on signal
Maximum IOPS=4988K
which is about a 42% improvement in IOPS.
Signature · 1 line
Jens Axboe
Re: [PATCH v6 2/2] dm: support bio polling
Quoted 33 lines · Jens Axboe
Quoted 32 lines · Ming Lei
Quoted 31 lines · Jens Axboe
Quoted 15 lines · Mike Snitzer
From: Ming Lei <[email protected]>
Support
bio(REQ_POLLED)polling in the following approach:1) only support io polling on normal READ/WRITE, and other abnormal IOs still fallback to IRQ mode, so the target io is exactly inside the dm io.
2) hold one refcnt on
io->io_countafter submitting this dm bio withREQ_POLLED3) support dm native bio splitting, any dm io instance associated with current bio will be added into one list which head is
bio->bi_privatewhich will be recovered before ending this bioQuoted 6 lines
4) implement .poll_bio() callback, call
bio_poll()on the single target bio inside the dm io which is retrieved viabio->bi_bio_drv_data; calldm_io_dec_pending()after the target io is done in .poll_bio()5) enable
QUEUE_FLAG_POLLif all underlying queues enableQUEUE_FLAG_POLL, which is based on Jeffle's previous patch.It's not the prettiest thing in the world with the overlay on
bi_private, but at least it's nicely documented now.I would encourage you to actually test this on fast storage, should make a nice difference. I can run this on a gen2 optane, it's 10x the IOPS of what it was tested on and should help better highlight where it makes a difference.
Quoted 7 lines
Quoted 5 lines
Quoted 2 lines
If either of you would like that, then send me a fool proof recipe for what should be setup so I have a poll capable dm device.
Follows steps for setup dm stripe over two nvmes, then run
io_uringon the dm stripe dev.Thanks! Much easier when I don't have to figure it out... Setup:
Jens, thanks for running the test!
Quoted 20 lines
CPU: 12900KDrives: 2x P5800X gen2 optane (~5M IOPS each at 512b)
Baseline kernel:
sudo taskset -c 10 t/io_uring -d128 -b512 -s31 -c16 -p1 -F1 -B1 -n1 -R1 -X1
/dev/dm-0Added file/dev/dm-0(submitter 0)
polled=1, fixedbufs=1/0, register_files=1, buffered=0, QD=128 Engine=io_uring, sq_ring=128, cq_ring=128
submitter=0, tid=1004
IOPS=2794K, BW=1364MiB/s, IOS/call=31/30, inflight=(124)
IOPS=2793K, BW=1363MiB/s, IOS/call=31/31, inflight=(62)
IOPS=2789K, BW=1362MiB/s, IOS/call=31/30, inflight=(124)
IOPS=2779K, BW=1357MiB/s, IOS/call=31/31, inflight=(124)
IOPS=2780K, BW=1357MiB/s, IOS/call=31/31, inflight=(62)
IOPS=2779K, BW=1357MiB/s, IOS/call=31/31, inflight=(62)
^CExiting on signal
Maximum IOPS=2794K
Quoted 1 line
generating about 500K ints/sec,
~5.6 IOs completed in each int averagely, looks irq coalesce is working.
Quoted 17 lines
and using 4k blocks:
sudo taskset -c 10 t/io_uring -d128 -b4096 -s31 -c16 -p1 -F1 -B1 -n1 -R1 -X1
/dev/dm-0Added file/dev/dm-0(submitter 0)
polled=1, fixedbufs=1/0, register_files=1, buffered=0, QD=128 Engine=io_uring, sq_ring=128, cq_ring=128
submitter=0, tid=967
IOPS=1683K, BW=6575MiB/s, IOS/call=24/24, inflight=(93)
IOPS=1685K, BW=6584MiB/s, IOS/call=24/24, inflight=(124)
IOPS=1686K, BW=6588MiB/s, IOS/call=24/24, inflight=(124)
IOPS=1684K, BW=6581MiB/s, IOS/call=24/24, inflight=(93)
IOPS=1686K, BW=6589MiB/s, IOS/call=24/24, inflight=(124)
IOPS=1687K, BW=6593MiB/s, IOS/call=24/24, inflight=(128)
IOPS=1687K, BW=6590MiB/s, IOS/call=24/24, inflight=(93)
^CExiting on signal
Maximum IOPS=1687K
Quoted 2 lines
which ends up being bw limited for me, because the devices aren't linked gen4. That's about 1.4M ints/sec.
Looks one interrupt just completes one IO with 4k bs, no irq coalesce any more. The interrupts may not run in CPU 10 I guess.
Quoted 18 lines
With the patched kernel, same test:
sudo taskset -c 10 t/io_uring -d128 -b512 -s31 -c16 -p1 -F1 -B1 -n1 -R1 -X1
/dev/dm-0Added file/dev/dm-0(submitter 0)
polled=1, fixedbufs=1/0, register_files=1, buffered=0, QD=128 Engine=io_uring, sq_ring=128, cq_ring=128
submitter=0, tid=989
IOPS=4151K, BW=2026MiB/s, IOS/call=16/15, inflight=(128)
IOPS=4159K, BW=2031MiB/s, IOS/call=15/15, inflight=(128)
IOPS=4193K, BW=2047MiB/s, IOS/call=15/15, inflight=(128)
IOPS=4191K, BW=2046MiB/s, IOS/call=15/15, inflight=(128)
IOPS=4202K, BW=2052MiB/s, IOS/call=15/15, inflight=(128)
^CExiting on signal
Maximum IOPS=4202Kwith basically zero interrupts, and 4k:
Quoted 12 lines
sudo taskset -c 10 t/io_uring -d128 -b4096 -s31 -c16 -p1 -F1 -B1 -n1 -R1 -X1
/dev/dm-0Added file/dev/dm-0(submitter 0)
polled=1, fixedbufs=1/0, register_files=1, buffered=0, QD=128 Engine=io_uring, sq_ring=128, cq_ring=128
submitter=0, tid=1015
IOPS=1706K, BW=6666MiB/s, IOS/call=15/15, inflight=(128)
IOPS=1704K, BW=6658MiB/s, IOS/call=15/15, inflight=(128)
IOPS=1704K, BW=6658MiB/s, IOS/call=15/15, inflight=(128)
IOPS=1704K, BW=6658MiB/s, IOS/call=15/15, inflight=(128)
IOPS=1704K, BW=6658MiB/s, IOS/call=15/15, inflight=(128)
^CExiting on signal
Maximum IOPS=1706K
Looks improvement on 4k is small, is it caused by pcie bw limit? What is the IOPS when running the same t/io_uring on single optane directly?
Thanks,
Ming
Jens Axboe
maintainerRe: [PATCH v6 2/2] dm: support bio polling
Quoted 3 lines · Ming Lei
Looks improvement on 4k is small, is it caused by pcie bw limit? What is the IOPS when running the same t/io_uring on single optane directly?
Yes, see what you responded to higher up:
"which ends up being bw limited for me, because the devices aren't linked gen4".
Some of them are, but the adapters are a bit janky and we often end up with gen3 links and hence limited to ~3.2GB/sec per drive. But with the bw limits even on gen4, you're roughly at about 1.5M IOPS per drive at that point, so would expect lower percentage wise gains for 4k with polling.
Signature · 1 line
Jens Axboe