Skip to contentSkip to conversation

dm: Avoid sending redundant empty flush bios to the same block device

PATCH New Yang Yang 1 participant 1 message Apr 17, 2024 Device mapper

lore
Share & tools
lore

Yang Yang

author
#0 lore raw

PATCH 1/1 dm: Avoid sending redundant empty flush bios to the same block device

__send_empty_flush() sends empty flush bios to every target in the dm_table. However, if the num_targets exceeds the number of block devices in the dm_table's device list, it could lead to multiple invocations of __send_duplicate_bios() for the same block device. Typically, a single thread sending numerous empty flush bios to one block device is redundant, as these bios are likely to be merged by the flush state machine. In scenarios where num_targets significantly outweighs the number of block devices, such behavior may result in a noteworthy decrease in performance.

This issue can be reproduced using this command line:
for i in {0..1023}; do

    echo $((8000*$i)) 8000 linear /dev/sda2 $((16384*$i))

done | dmsetup create example

With this fix, a random write with fsync workload executed with the following fio command:

  fio --group_reporting --name=benchmark --filename=/dev/mapper/example \
      --ioengine=sync --invalidate=1 --numjobs=16 --rw=randwrite \
      --blocksize=4k --size=2G --time_based --runtime=30 --fdatasync=1

results in an increase from 857 KB/s to 30.8 MB/s of the write throughput (3580% increase).

---
 drivers/md/dm-core.h          |  1 +
 drivers/md/dm-table.c         |  7 ++++
 drivers/md/dm.c               | 60 +++++++++++++++++++++++++++++++++++
 include/linux/device-mapper.h |  6 ++++
 4 files changed, 74 insertions(+)
drivers/md/dm-core.h +1 -0
index e6757a30dcca..7e3f2168289f 100644
@@ -217,6 +217,7 @@ struct dm_table {
 	/* a list of devices used by this table */
 	struct list_head devices;
 	struct rw_semaphore devices_lock;
+	unsigned short num_devices;
 
 	/* events get handed up using this callback */
 	void (*event_fn)(void *data);
drivers/md/dm-table.c +7 -0
index 41f1d731ae5a..ddc60e498afb 100644
@@ -2133,6 +2133,8 @@ void dm_table_postsuspend_targets(struct dm_table *t)
 
 int dm_table_resume_targets(struct dm_table *t)
 {
+	struct list_head *devices = dm_table_get_devices(t);
+	struct dm_dev_internal *dd;
 	unsigned int i;
 	int r = 0;
 
@@ -2159,6 +2161,11 @@ int dm_table_resume_targets(struct dm_table *t)
 			ti->type->resume(ti);
 	}
 
+	t->num_devices = 0;
+
+	list_for_each_entry(dd, devices, list)
+		dd->dm_dev->index = ++(t->num_devices);
+
 	return 0;
 }
 
drivers/md/dm.c +60 -0

93 lines not on this page · raw message

include/linux/device-mapper.h +6 -0
index 82b2195efaca..4a54b4f0a609 100644
@@ -169,6 +169,12 @@ struct dm_dev {
 	struct dax_device *dax_dev;
 	blk_mode_t mode;
 	char name[16];
+
+	/*
+	 * sequential number for each dm_dev in dm_table's devices list,
+	 * start from 1
+	 */
+	unsigned short index;
 };
 
 /*
Signature · 1 line
2.34.1