Skip to contentSkip to conversation

null_blk: Support configuring the maximum segment size

PATCH In review Bart Van Assche 3 participants 9 messages Jan 28, 2023 – Feb 2, 2023 Block layer

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 null_blk: Support configuring the maximum segment size

Add support for configuring the maximum segment size.

Add support for segments smaller than the page size.

This patch enables testing segments smaller than the page size with a driver that does not call blk_rq_map_sg().

Cc Christoph Hellwig <[email protected]>
Cc Ming Lei <[email protected]>
Cc Damien Le Moal <[email protected]>
Cc Chaitanya Kulkarni <[email protected]>
---
 drivers/block/null_blk/main.c     | 19 ++++++++++++++++---
 drivers/block/null_blk/null_blk.h |  1 +
 2 files changed, 17 insertions(+), 3 deletions(-)
drivers/block/null_blk/main.c +16 -3
index 4c601ca9552a..06eaa7ff4a86 100644
@@ -157,6 +157,10 @@ static int g_max_sectors;
 module_param_named(max_sectors, g_max_sectors, int, 0444);
 MODULE_PARM_DESC(max_sectors, "Maximum size of a command (in 512B sectors)");
 
+static unsigned int g_max_segment_size = BLK_MAX_SEGMENT_SIZE;
+module_param_named(max_segment_size, g_max_segment_size, int, 0444);
+MODULE_PARM_DESC(max_segment_size, "Maximum size of a segment in bytes");
+
 static unsigned int nr_devices = 1;
 module_param(nr_devices, uint, 0444);
 MODULE_PARM_DESC(nr_devices, "Number of devices to register");
@@ -409,6 +413,7 @@ NULLB_DEVICE_ATTR(home_node, uint, NULL);
 NULLB_DEVICE_ATTR(queue_mode, uint, NULL);
 NULLB_DEVICE_ATTR(blocksize, uint, NULL);
 NULLB_DEVICE_ATTR(max_sectors, uint, NULL);
+NULLB_DEVICE_ATTR(max_segment_size, uint, NULL);
 NULLB_DEVICE_ATTR(irqmode, uint, NULL);
 NULLB_DEVICE_ATTR(hw_queue_depth, uint, NULL);
 NULLB_DEVICE_ATTR(index, uint, NULL);
@@ -550,6 +555,7 @@ static struct configfs_attribute *nullb_device_attrs[] = {
 	&nullb_device_attr_queue_mode,
 	&nullb_device_attr_blocksize,
 	&nullb_device_attr_max_sectors,
+	&nullb_device_attr_max_segment_size,
 	&nullb_device_attr_irqmode,
 	&nullb_device_attr_hw_queue_depth,
 	&nullb_device_attr_index,
@@ -630,7 +636,8 @@ static ssize_t memb_group_features_show(struct config_item *item, char *page)
 	return snprintf(page, PAGE_SIZE,
 			"badblocks,blocking,blocksize,cache_size,"
 			"completion_nsec,discard,home_node,hw_queue_depth,"
-			"irqmode,max_sectors,mbps,memory_backed,no_sched,"
+			"irqmode,max_sectors,max_segment_size,mbps,"
+			"memory_backed,no_sched,"
 			"poll_queues,power,queue_mode,shared_tag_bitmap,size,"
 			"submit_queues,use_per_node_hctx,virt_boundary,zoned,"
 			"zone_capacity,zone_max_active,zone_max_open,"
@@ -693,6 +700,7 @@ static struct nullb_device *null_alloc_dev(void)
 	dev->queue_mode = g_queue_mode;
 	dev->blocksize = g_bs;
 	dev->max_sectors = g_max_sectors;
+	dev->max_segment_size = g_max_segment_size;
 	dev->irqmode = g_irqmode;
 	dev->hw_queue_depth = g_hw_queue_depth;
 	dev->blocking = g_blocking;
@@ -1234,6 +1242,8 @@ static int null_transfer(struct nullb *nullb, struct page *page,
 	unsigned int valid_len = len;
 	int err = 0;
 
+	WARN_ONCE(len > dev->max_segment_size, "%u > %u\n", len,
+		  dev->max_segment_size);
 	if (!is_write) {
 		if (dev->zoned)
 			valid_len = null_zone_valid_read_len(nullb,
@@ -1269,7 +1279,8 @@ static int null_handle_rq(struct nullb_cmd *cmd)
 
 	spin_lock_irq(&nullb->lock);
 	rq_for_each_segment(bvec, rq, iter) {
-		len = bvec.bv_len;
+		len = min(bvec.bv_len, nullb->dev->max_segment_size);
+		bvec.bv_len = len;
 		err = null_transfer(nullb, bvec.bv_page, len, bvec.bv_offset,
 				     op_is_write(req_op(rq)), sector,
 				     rq->cmd_flags & REQ_FUA);
@@ -1296,7 +1307,8 @@ static int null_handle_bio(struct nullb_cmd *cmd)
 
 	spin_lock_irq(&nullb->lock);
 	bio_for_each_segment(bvec, bio, iter) {
-		len = bvec.bv_len;
+		len = min(bvec.bv_len, nullb->dev->max_segment_size);
+		bvec.bv_len = len;
 		err = null_transfer(nullb, bvec.bv_page, len, bvec.bv_offset,
 				     op_is_write(bio_op(bio)), sector,
 				     bio->bi_opf & REQ_FUA);
@@ -2125,6 +2137,7 @@ static int null_add_dev(struct nullb_device *dev)
 		dev->max_sectors = queue_max_hw_sectors(nullb->q);
 	dev->max_sectors = min(dev->max_sectors, BLK_DEF_MAX_SECTORS);
 	blk_queue_max_hw_sectors(nullb->q, dev->max_sectors);
+	blk_queue_max_segment_size(nullb->q, dev->max_segment_size);
 
 	if (dev->virt_boundary)
 		blk_queue_virt_boundary(nullb->q, PAGE_SIZE - 1);
drivers/block/null_blk/null_blk.h +1 -0
index eb5972c50be8..8cb73fe05fa3 100644
@@ -102,6 +102,7 @@ struct nullb_device {
 	unsigned int queue_mode; /* block interface */
 	unsigned int blocksize; /* block size */
 	unsigned int max_sectors; /* Max sectors per command */
+	unsigned int max_segment_size; /* Max size of a single DMA segment. */
 	unsigned int irqmode; /* IRQ completion handler */
 	unsigned int hw_queue_depth; /* queue depth */
 	unsigned int index; /* index of the disk, only valid with a disk */
#1 lore raw
Quoted 67 lines · Bart Van Assche

Add support for configuring the maximum segment size.

Add support for segments smaller than the page size.

This patch enables testing segments smaller than the page size with a driver that does not call blk_rq_map_sg().

Cc Christoph Hellwig <[email protected]>
Cc Ming Lei <[email protected]>
Cc Damien Le Moal <[email protected]>
Cc Chaitanya Kulkarni <[email protected]>
---
 drivers/block/null_blk/main.c     | 19 ++++++++++++++++---
 drivers/block/null_blk/null_blk.h |  1 +
 2 files changed, 17 insertions(+), 3 deletions(-)
drivers/block/null_blk/main.c +9 -1
index 4c601ca9552a..06eaa7ff4a86 100644
@@ -157,6 +157,10 @@ static int g_max_sectors;
 module_param_named(max_sectors, g_max_sectors, int, 0444);
 MODULE_PARM_DESC(max_sectors, "Maximum size of a command (in 512B sectors)");
 
+static unsigned int g_max_segment_size = BLK_MAX_SEGMENT_SIZE;
+module_param_named(max_segment_size, g_max_segment_size, int, 0444);
+MODULE_PARM_DESC(max_segment_size, "Maximum size of a segment in bytes");
+
 static unsigned int nr_devices = 1;
 module_param(nr_devices, uint, 0444);
 MODULE_PARM_DESC(nr_devices, "Number of devices to register");
@@ -409,6 +413,7 @@ NULLB_DEVICE_ATTR(home_node, uint, NULL);
 NULLB_DEVICE_ATTR(queue_mode, uint, NULL);
 NULLB_DEVICE_ATTR(blocksize, uint, NULL);
 NULLB_DEVICE_ATTR(max_sectors, uint, NULL);
+NULLB_DEVICE_ATTR(max_segment_size, uint, NULL);
 NULLB_DEVICE_ATTR(irqmode, uint, NULL);
 NULLB_DEVICE_ATTR(hw_queue_depth, uint, NULL);
 NULLB_DEVICE_ATTR(index, uint, NULL);
@@ -550,6 +555,7 @@ static struct configfs_attribute *nullb_device_attrs[] = {
 	&nullb_device_attr_queue_mode,
 	&nullb_device_attr_blocksize,
 	&nullb_device_attr_max_sectors,
+	&nullb_device_attr_max_segment_size,
 	&nullb_device_attr_irqmode,
 	&nullb_device_attr_hw_queue_depth,
 	&nullb_device_attr_index,
@@ -630,7 +636,8 @@ static ssize_t memb_group_features_show(struct config_item *item, char *page)
 	return snprintf(page, PAGE_SIZE,
 			"badblocks,blocking,blocksize,cache_size,"
 			"completion_nsec,discard,home_node,hw_queue_depth,"
-			"irqmode,max_sectors,mbps,memory_backed,no_sched,"
+			"irqmode,max_sectors,max_segment_size,mbps,"
+			"memory_backed,no_sched,"
 			"poll_queues,power,queue_mode,shared_tag_bitmap,size,"
 			"submit_queues,use_per_node_hctx,virt_boundary,zoned,"
 			"zone_capacity,zone_max_active,zone_max_open,"
@@ -693,6 +700,7 @@ static struct nullb_device *null_alloc_dev(void)
 	dev->queue_mode = g_queue_mode;
 	dev->blocksize = g_bs;
 	dev->max_sectors = g_max_sectors;
+	dev->max_segment_size = g_max_segment_size;
 	dev->irqmode = g_irqmode;
 	dev->hw_queue_depth = g_hw_queue_depth;
 	dev->blocking = g_blocking;

drivers/block/null_blk/main.c static int null_transfer(struct nullb *nullb, struct page *page, · line 1246

Quoted 5 lines
drivers/block/null_blk/main.c +2 -0
@@ -1234,6 +1242,8 @@ static int null_transfer(struct nullb *nullb, struct page *page,
 	unsigned int valid_len = len;
 	int err = 0;
 
+	WARN_ONCE(len > dev->max_segment_size, "%u > %u\n", len,
+		  dev->max_segment_size);

Shouldn't this be an EIO return as this is not supposed to happen ?

Quoted 43 lines
drivers/block/null_blk/main.c +5 -2
 	if (!is_write) {
 		if (dev->zoned)
 			valid_len = null_zone_valid_read_len(nullb,
@@ -1269,7 +1279,8 @@ static int null_handle_rq(struct nullb_cmd *cmd)
 
 	spin_lock_irq(&nullb->lock);
 	rq_for_each_segment(bvec, rq, iter) {
-		len = bvec.bv_len;
+		len = min(bvec.bv_len, nullb->dev->max_segment_size);
+		bvec.bv_len = len;
 		err = null_transfer(nullb, bvec.bv_page, len, bvec.bv_offset,
 				     op_is_write(req_op(rq)), sector,
 				     rq->cmd_flags & REQ_FUA);
@@ -1296,7 +1307,8 @@ static int null_handle_bio(struct nullb_cmd *cmd)
 
 	spin_lock_irq(&nullb->lock);
 	bio_for_each_segment(bvec, bio, iter) {
-		len = bvec.bv_len;
+		len = min(bvec.bv_len, nullb->dev->max_segment_size);
+		bvec.bv_len = len;
 		err = null_transfer(nullb, bvec.bv_page, len, bvec.bv_offset,
 				     op_is_write(bio_op(bio)), sector,
 				     bio->bi_opf & REQ_FUA);
@@ -2125,6 +2137,7 @@ static int null_add_dev(struct nullb_device *dev)
 		dev->max_sectors = queue_max_hw_sectors(nullb->q);
 	dev->max_sectors = min(dev->max_sectors, BLK_DEF_MAX_SECTORS);
 	blk_queue_max_hw_sectors(nullb->q, dev->max_sectors);
+	blk_queue_max_segment_size(nullb->q, dev->max_segment_size);
 
 	if (dev->virt_boundary)
 		blk_queue_virt_boundary(nullb->q, PAGE_SIZE - 1);
drivers/block/null_blk/null_blk.h +1 -0
index eb5972c50be8..8cb73fe05fa3 100644
@@ -102,6 +102,7 @@ struct nullb_device {
 	unsigned int queue_mode; /* block interface */
 	unsigned int blocksize; /* block size */
 	unsigned int max_sectors; /* Max sectors per command */
+	unsigned int max_segment_size; /* Max size of a single DMA segment. */
 	unsigned int irqmode; /* IRQ completion handler */
 	unsigned int hw_queue_depth; /* queue depth */
 	unsigned int index; /* index of the disk, only valid with a disk */
Signature · 2 lines
Damien Le Moal
Western Digital Research
#2 lore raw
Quoted 5 lines · Damien Le Moal

drivers/block/null_blk/main.c static int null_transfer(struct nullb *nullb, struct page *page, · line 1246

Quoted 2 lines · Bart Van Assche
drivers/block/null_blk/main.c +2 -0
+	WARN_ONCE(len > dev->max_segment_size, "%u > %u\n", len,
+		  dev->max_segment_size);

Shouldn't this be an EIO return as this is not supposed to happen ?

Hmm ... the above WARN_ONCE() statement is intended as a precondition check. This statement is intended as a help for developers to check that the code below works fine. I'm not sure how returning EIO here would help?

Thanks

Bart.

#3 lore raw
Quoted 10 lines · Bart Van Assche
Quoted 5 lines · Damien Le Moal

drivers/block/null_blk/main.c static int null_transfer(struct nullb *nullb, struct page *page, · line 1246

Quoted 2 lines · Bart Van Assche
drivers/block/null_blk/main.c +2 -0
+	WARN_ONCE(len > dev->max_segment_size, "%u > %u\n", len,
+		  dev->max_segment_size);

Shouldn't this be an EIO return as this is not supposed to happen ?

Hmm ... the above WARN_ONCE() statement is intended as a precondition check. This statement is intended as a help for developers to check that the code below works fine. I'm not sure how returning EIO here would help?

My point was that given that null_transfer() is called like this:

	rq_for_each_segment(bvec, rq, iter) {
                len = bvec.bv_len;
                err = null_transfer(nullb, bvec.bv_page, len, bvec.bv_offset,
                                     ...);

len should never be larger than max_segment_size, no ? Unless I am missing something else...

Quoted 4 lines

Thanks

Bart.

Signature · 2 lines
Damien Le Moal
Western Digital Research
#4 lore raw
Quoted 14 lines · Damien Le Moal
Quoted 10 lines · Bart Van Assche
Quoted 5 lines · Damien Le Moal

drivers/block/null_blk/main.c static int null_transfer(struct nullb *nullb, struct page *page, · line 1246

Quoted 2 lines · Bart Van Assche
drivers/block/null_blk/main.c +2 -0
+	WARN_ONCE(len > dev->max_segment_size, "%u > %u\n", len,
+		  dev->max_segment_size);

Shouldn't this be an EIO return as this is not supposed to happen ?

Hmm ... the above WARN_ONCE() statement is intended as a precondition check. This statement is intended as a help for developers to check that the code below works fine. I'm not sure how returning EIO here would help?

My point was that given that null_transfer() is called like this:

Quoted 7 lines
	rq_for_each_segment(bvec, rq, iter) {
                 len = bvec.bv_len;
                 err = null_transfer(nullb, bvec.bv_page, len, bvec.bv_offset,
                                      ...);

len should never be larger than max_segment_size, no ? Unless I am missing something else...

Hi Damien,

null_transfer() supports the case len > dev->max_segment_size so I think that a warning is sufficient and also that we don't need to return -EIO.

Thanks,

Bart.

Reviewed-by on v1 1/1
#5 lore raw
In reply to Bart Van Assche
Quoted 7 lines · Bart Van Assche

Add support for configuring the maximum segment size.

Add support for segments smaller than the page size.

This patch enables testing segments smaller than the page size with a driver that does not call blk_rq_map_sg().

Quoted 7 lines
Cc Christoph Hellwig <[email protected]>
Cc Ming Lei <[email protected]>
Cc Damien Le Moal <[email protected]>
Cc Chaitanya Kulkarni <[email protected]>
---
  drivers/block/null_blk/main.c     | 19 ++++++++++++++++---

Looks good.

-ck

#6 lore raw
In reply to Bart Van Assche
Quoted 4 lines · Bart Van Assche

Add support for configuring the maximum segment size.

Add support for segments smaller than the page size.

Quoted 8 lines

This patch enables testing segments smaller than the page size with a driver that does not call blk_rq_map_sg().

Cc Christoph Hellwig <[email protected]>
Cc Ming Lei <[email protected]>
Cc Damien Le Moal <[email protected]>
Cc Chaitanya Kulkarni <[email protected]>

Bart, if you can send a blktest for this it will be great.

-ck

#7 lore raw
Quoted 8 lines · Chaitanya Kulkarni
Quoted 7 lines · Bart Van Assche

Add support for configuring the maximum segment size.

Add support for segments smaller than the page size.

This patch enables testing segments smaller than the page size with a driver that does not call blk_rq_map_sg().

Quoted 7 lines
Quoted 5 lines
Cc Christoph Hellwig <[email protected]>
Cc Ming Lei <[email protected]>
Cc Damien Le Moal <[email protected]>
Cc Chaitanya Kulkarni <[email protected]>

Bart, if you can send a blktest for this it will be great.

Hi Chaitanya,

This patch has been resent as patch 7/7 of the following series: "[PATCH v4 0/7] Add support for limits below the page size" (https://lore.kernel.org/linux-block/[email protected]/T/#t).

blktests for that patch series are available here: https://github.com/bvanassche/blktests/commits/master. Do you prefer that I send a pull request for these blktests now or do you perhaps prefer that I wait until the kernel patch series has been merged?

Thanks,

Bart.

#8 lore raw
Quoted 7 lines

Hi Chaitanya,

This patch has been resent as patch 7/7 of the following series: "[PATCH v4 0/7] Add support for limits below the page size"
(https://lore.kernel.org/linux-block/[email protected]/T/#t).

I see.

Quoted 9 lines

blktests for that patch series are available here:
https://github.com/bvanassche/blktests/commits/master. Do you prefer that I send a pull request for these blktests now or do you perhaps prefer that I wait until the kernel patch series has been merged?

Thanks,

Bart.

Let's wait till it is merged.

-ck