Skip to contentSkip to conversation

add a ZONE_RESET_ALL uring cmd v2

PATCH In review Christoph Hellwig 2 participants 13 messages Jul 16

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

Hi Jens,

this series adds a new ZONE_RESET_ALL uring cmd, and cleans up the io_uring discard support and the block layer zone management code a little bit in preparation for that.

This command is useful to speed up workloads that need to initialize a large number (or even a small > 1 number) of disks.

Diffstat:

 block/blk-core.c            |    6 +++-
 block/blk-zoned.c           |   30 ----------------------
 block/ioctl.c               |   60 ++++++++++++++++++++++++++++++++++----------
 include/uapi/linux/blkdev.h |    1 
 4 files changed, 53 insertions(+), 44 deletions(-)
#1 lore raw

PATCH 1/6 block: remove a dead return statement in blk_zone_plug_bio

The switch at the end of blk_zone_plug_bio always returns, so remove the dead extra return statement after it.

---
 block/blk-zoned.c | 2 --
 1 file changed, 2 deletions(-)
block/blk-zoned.c +0 -2
index ca30caec838e..034af9dfb5f9 100644
@@ -1651,8 +1651,6 @@ bool blk_zone_plug_bio(struct bio *bio, unsigned int nr_segs)
 	default:
 		return false;
 	}
-
-	return false;
 }
 EXPORT_SYMBOL_GPL(blk_zone_plug_bio);
 
Signature · 1 line
2.53.0
#3 lore raw

PATCH 2/6 block: allow REQ_NOWAIT zone management commands

In reply to Christoph Hellwig

Commit efae226c2ef1 ("block: handle zone management operations completions") moved all block layer tracking of zone management operations to the I/O completion handler. With that REQ_NOWAIT zone management operations are just fine, so allow them.

---
 block/blk-zoned.c | 9 ---------
 1 file changed, 9 deletions(-)
block/blk-zoned.c +0 -9
index 034af9dfb5f9..3b7a5f2bdf98 100644
@@ -1586,15 +1586,6 @@ static bool blk_zone_wplug_handle_zone_mgmt(struct bio *bio)
 		return true;
 	}
 
-	/*
-	 * No-wait zone management BIOs do not make much sense as the callers
-	 * issue these as blocking operations in most cases. To avoid issues
-	 * with the BIO execution potentially failing with BLK_STS_AGAIN, warn
-	 * about REQ_NOWAIT being set and ignore that flag.
-	 */
-	if (WARN_ON_ONCE(bio->bi_opf & REQ_NOWAIT))
-		bio->bi_opf &= ~REQ_NOWAIT;
-
 	return false;
 }
 
Signature · 1 line
2.53.0
maintainer Reviewed-by on v1 2/6
#4 lore raw

Re: [PATCH 2/6] block: allow REQ_NOWAIT zone management commands

Quoted 6 lines · Christoph Hellwig

Commit efae226c2ef1 ("block: handle zone management operations completions") moved all block layer tracking of zone management operations to the I/O completion handler. With that REQ_NOWAIT zone management operations are just fine, so allow them.

Signature · 2 lines
Damien Le Moal
Western Digital Research
#5 lore raw

PATCH 3/6 block: remove blk_zone_wplug_handle_zone_mgmt

In reply to Christoph Hellwig

blk_zone_wplug_handle_zone_mgmt now only checks that zone reset and zone finish operations are directed to a sequential write required zone. This has nothing to do with zone write plugs and is better handled with other bio validity checks in submit_bio_noacct.

---
 block/blk-core.c  |  9 ++++++++-
 block/blk-zoned.c | 19 -------------------
 2 files changed, 8 insertions(+), 20 deletions(-)
block/blk-core.c +8 -1
index 365641266c9e..67cfd7bd8542 100644
@@ -898,9 +898,16 @@ void submit_bio_noacct(struct bio *bio)
 			goto not_supported;
 		break;
 	case REQ_OP_ZONE_RESET:
+	case REQ_OP_ZONE_FINISH:
+		/*
+		 * Zone reset and zone finish operations do not apply to
+		 * conventional zones.
+		 */
+		if (!bdev_zone_is_seq(bio->bi_bdev, bio->bi_iter.bi_sector))
+			goto end_io;
+		break;
 	case REQ_OP_ZONE_OPEN:
 	case REQ_OP_ZONE_CLOSE:
-	case REQ_OP_ZONE_FINISH:
 	case REQ_OP_ZONE_RESET_ALL:
 		if (!bdev_is_zoned(bio->bi_bdev))
 			goto not_supported;
block/blk-zoned.c +0 -19
index 3b7a5f2bdf98..a5afb842bf35 100644
@@ -1574,21 +1574,6 @@ static void blk_zone_wplug_handle_native_zone_append(struct bio *bio)
 	disk_put_zone_wplug(zwplug);
 }
 
-static bool blk_zone_wplug_handle_zone_mgmt(struct bio *bio)
-{
-	if (bio_op(bio) != REQ_OP_ZONE_RESET_ALL &&
-	    !bdev_zone_is_seq(bio->bi_bdev, bio->bi_iter.bi_sector)) {
-		/*
-		 * Zone reset and zone finish operations do not apply to
-		 * conventional zones.
-		 */
-		bio_io_error(bio);
-		return true;
-	}
-
-	return false;
-}
-
 /**
  * blk_zone_plug_bio - Handle a zone write BIO with zone write plugging
  * @bio: The BIO being submitted
@@ -1635,10 +1620,6 @@ bool blk_zone_plug_bio(struct bio *bio, unsigned int nr_segs)
 	case REQ_OP_WRITE:
 	case REQ_OP_WRITE_ZEROES:
 		return blk_zone_wplug_handle_write(bio, nr_segs);
-	case REQ_OP_ZONE_RESET:
-	case REQ_OP_ZONE_FINISH:
-	case REQ_OP_ZONE_RESET_ALL:
-		return blk_zone_wplug_handle_zone_mgmt(bio);
 	default:
 		return false;
 	}
Signature · 1 line
2.53.0
maintainer Reviewed-by on v1 3/6
#6 lore raw

Re: [PATCH 3/6] block: remove blk_zone_wplug_handle_zone_mgmt

Quoted 6 lines · Christoph Hellwig

blk_zone_wplug_handle_zone_mgmt now only checks that zone reset and zone finish operations are directed to a sequential write required zone. This has nothing to do with zone write plugs and is better handled with other bio validity checks in submit_bio_noacct.

Signature · 2 lines
Damien Le Moal
Western Digital Research
#7 lore raw

PATCH 4/6 block: also reject zone open / close on conventional zones

In reply to Christoph Hellwig

Just like zone reset / finish, these only apply to sequential zones.

---
 block/blk-core.c | 9 +++------
 1 file changed, 3 insertions(+), 6 deletions(-)
block/blk-core.c +3 -6
index 67cfd7bd8542..196bccf27f58 100644
@@ -897,17 +897,14 @@ void submit_bio_noacct(struct bio *bio)
 		if (!q->limits.max_write_zeroes_sectors)
 			goto not_supported;
 		break;
+	case REQ_OP_ZONE_OPEN:
+	case REQ_OP_ZONE_CLOSE:
 	case REQ_OP_ZONE_RESET:
 	case REQ_OP_ZONE_FINISH:
-		/*
-		 * Zone reset and zone finish operations do not apply to
-		 * conventional zones.
-		 */
+		/* Zone management operations require sequential zones. */
 		if (!bdev_zone_is_seq(bio->bi_bdev, bio->bi_iter.bi_sector))
 			goto end_io;
 		break;
-	case REQ_OP_ZONE_OPEN:
-	case REQ_OP_ZONE_CLOSE:
 	case REQ_OP_ZONE_RESET_ALL:
 		if (!bdev_is_zoned(bio->bi_bdev))
 			goto not_supported;
Signature · 1 line
2.53.0
#9 lore raw

PATCH 5/6 block: remove most blkdev_cmd_discard arguments

In reply to Christoph Hellwig

All other arguments can be derived from cmd, so do that to simplify the calling convention.

---
 block/ioctl.c | 23 ++++++++++-------------
 1 file changed, 10 insertions(+), 13 deletions(-)
block/ioctl.c +10 -13
index 3d4ea1537457..af2a897f13f9 100644
@@ -887,14 +887,13 @@ static void bio_cmd_bio_end_io(struct bio *bio)
 	bio_put(bio);
 }
 
-static int blkdev_cmd_discard(struct io_uring_cmd *cmd,
-			      struct block_device *bdev,
-			      uint64_t start, uint64_t len, bool nowait)
+static int blkdev_cmd_discard(struct io_uring_cmd *cmd)
 {
 	struct blk_iou_cmd *bic = io_uring_cmd_to_pdu(cmd, struct blk_iou_cmd);
-	gfp_t gfp = nowait ? GFP_NOWAIT : GFP_KERNEL;
-	sector_t sector = start >> SECTOR_SHIFT;
-	sector_t nr_sects = len >> SECTOR_SHIFT;
+	struct block_device *bdev = I_BDEV(cmd->file->f_mapping->host);
+	gfp_t gfp = bic->nowait ? GFP_NOWAIT : GFP_KERNEL;
+	sector_t sector = bic->start >> SECTOR_SHIFT;
+	sector_t nr_sects = bic->len >> SECTOR_SHIFT;
 	struct bio *prev = NULL, *bio;
 	int err;
 
@@ -904,12 +903,12 @@ static int blkdev_cmd_discard(struct io_uring_cmd *cmd,
 		return -EBADF;
 	if (bdev_read_only(bdev))
 		return -EPERM;
-	err = blk_validate_byte_range(bdev, start, len);
+	err = blk_validate_byte_range(bdev, bic->start, bic->len);
 	if (err)
 		return err;
 
-	err = filemap_invalidate_pages(bdev->bd_mapping, start,
-					start + len - 1, nowait);
+	err = filemap_invalidate_pages(bdev->bd_mapping, bic->start,
+				       bic->start + bic->len - 1, bic->nowait);
 	if (err)
 		return err;
 
@@ -917,7 +916,7 @@ static int blkdev_cmd_discard(struct io_uring_cmd *cmd,
 		bio = blk_alloc_discard_bio(bdev, &sector, &nr_sects, gfp);
 		if (!bio)
 			break;
-		if (nowait) {
+		if (bic->nowait) {
 			/*
 			 * Don't allow multi-bio non-blocking submissions as
 			 * subsequent bios may fail but we won't get a direct
@@ -946,7 +945,6 @@ static int blkdev_cmd_discard(struct io_uring_cmd *cmd,
 
 int blkdev_uring_cmd(struct io_uring_cmd *cmd, unsigned int issue_flags)
 {
-	struct block_device *bdev = I_BDEV(cmd->file->f_mapping->host);
 	struct blk_iou_cmd *bic = io_uring_cmd_to_pdu(cmd, struct blk_iou_cmd);
 	u32 cmd_op = cmd->cmd_op;
 
@@ -967,8 +965,7 @@ int blkdev_uring_cmd(struct io_uring_cmd *cmd, unsigned int issue_flags)
 
 	switch (cmd_op) {
 	case BLOCK_URING_CMD_DISCARD:
-		return blkdev_cmd_discard(cmd, bdev, bic->start, bic->len,
-					  bic->nowait);
+		return blkdev_cmd_discard(cmd);
 	}
 	return -EINVAL;
 }
Signature · 1 line
2.53.0
#11 lore raw

PATCH 6/6 block: implement async io_uring zone reset all

In reply to Christoph Hellwig

Add a new BLOCK_URING_CMD_ZONE_RESET_ALL uring cmd to reset all zones for a given block device. This can be used by storage systems or file system mkfs tools to initialize multiple devices in parallel.

---
 block/ioctl.c               | 37 +++++++++++++++++++++++++++++++++++++
 include/uapi/linux/blkdev.h |  1 +
 2 files changed, 38 insertions(+)
block/ioctl.c +37 -0
index af2a897f13f9..64b4e6c0f696 100644
@@ -14,6 +14,7 @@
 #include <linux/pagemap.h>
 #include <linux/io_uring/cmd.h>
 #include <linux/blk-integrity.h>
+#include <trace/events/block.h>
 #include <uapi/linux/blkdev.h>
 #include "blk.h"
 #include "blk-crypto-internal.h"
@@ -943,6 +944,40 @@ static int blkdev_cmd_discard(struct io_uring_cmd *cmd)
 	return -EIOCBQUEUED;
 }
 
+static int blkdev_cmd_zone_reset_all(struct io_uring_cmd *cmd)
+{
+	struct blk_iou_cmd *bic = io_uring_cmd_to_pdu(cmd, struct blk_iou_cmd);
+	struct block_device *bdev = I_BDEV(cmd->file->f_mapping->host);
+	struct bio *bio;
+	int err;
+
+	if (!(file_to_blk_mode(cmd->file) & BLK_OPEN_WRITE))
+		return -EBADF;
+	if (bdev_read_only(bdev))
+		return -EPERM;
+	if (!bdev_is_zoned(bdev))
+		return -EOPNOTSUPP;
+	if (bic->start || bic->len)
+		return -EINVAL;
+
+	err = filemap_invalidate_pages(bdev->bd_mapping, 0,
+				bdev_nr_bytes(bdev) - 1, bic->nowait);
+	if (err)
+		return err;
+
+	bio = bio_alloc(bdev, 0, REQ_OP_ZONE_RESET_ALL,
+			bic->nowait ? GFP_NOWAIT : GFP_KERNEL);
+	if (!bio)
+		return -EAGAIN;
+	if (bic->nowait)
+		bio->bi_opf |= REQ_NOWAIT;
+	trace_blkdev_zone_mgmt(bio, 0);
+	bio->bi_private = cmd;
+	bio->bi_end_io = bio_cmd_bio_end_io;
+	submit_bio(bio);
+	return -EIOCBQUEUED;
+}
+
 int blkdev_uring_cmd(struct io_uring_cmd *cmd, unsigned int issue_flags)
 {
 	struct blk_iou_cmd *bic = io_uring_cmd_to_pdu(cmd, struct blk_iou_cmd);
@@ -966,6 +1001,8 @@ int blkdev_uring_cmd(struct io_uring_cmd *cmd, unsigned int issue_flags)
 	switch (cmd_op) {
 	case BLOCK_URING_CMD_DISCARD:
 		return blkdev_cmd_discard(cmd);
+	case BLOCK_URING_CMD_ZONE_RESET_ALL:
+		return blkdev_cmd_zone_reset_all(cmd);
 	}
 	return -EINVAL;
 }
include/uapi/linux/blkdev.h +1 -0
index 66373cd1a83a..57b0bbd04e2b 100644
@@ -10,5 +10,6 @@
  * It's a different number space from ioctl(), reuse the block's code 0x12.
  */
 #define BLOCK_URING_CMD_DISCARD			_IO(0x12, 0)
+#define BLOCK_URING_CMD_ZONE_RESET_ALL		_IO(0x12, 1)
 
 #endif
Signature · 1 line
2.53.0
maintainer Reviewed-by on v1 6/6
#12 lore raw

Re: [PATCH 6/6] block: implement async io_uring zone reset all

Quoted 5 lines · Christoph Hellwig

Add a new BLOCK_URING_CMD_ZONE_RESET_ALL uring cmd to reset all zones for a given block device. This can be used by storage systems or file system mkfs tools to initialize multiple devices in parallel.

Signature · 2 lines
Damien Le Moal
Western Digital Research