Skip to contentSkip to conversation

block: fix I/O errors in BLKRRPART

PATCH Superseded Minwoo Im 2 participants 3 messages Jan 5, 2021

Subsystems (3)

v4 of 6 Superseded Latest is v6 →

lore
Share & tools
lore

Minwoo Im

author
#0 lore raw

v4 cover block: fix I/O errors in BLKRRPART

Hello,

This patch fixes I/O errors during BLKRRPART ioctl() behavior right
after format operation that changed logical block size of the block device with a same file descriptor opened.

Testcase:

The following testcase is a case of NVMe namespace with the following
conditions:

- Current LBA format is lbaf=0 (512 bytes logical block size)
- LBA Format(lbaf=1) has 4096 bytes logical block size

  # Format block device logical block size 512B to 4096B                                                                                                                                                                                                                                                                                                                                       
  nvme format /dev/nvme0n1 --lbaf=1 --force

This will cause I/O errors because BLKRRPART ioctl() happened right after
the format command with same file descriptor opened in application (e.g., nvme-cli) like:

  fd = open("/dev/nvme0n1", O_RDONLY);

  nvme_format(fd, ...);
  if (ioctl(fd, BLKRRPART) < 0)
        ...

Errors:

We can see the Read command with Number of LBA(NLB) 0xffff(65535) which
was under-flowed because BLKRRPART operation requested request size based on i_blkbits of the block device which is 9 via buffer_head.

[dmesg-snip]

    [   10.771740] blk_update_request: operation not supported error, dev nvme0n1, sector 0 op 0x0:(READ) flags 0x0 phys_seg 1 prio class 0
    [   10.780262] Buffer I/O error on dev nvme0n1, logical block 0, async page read

[event-snip]

    kworker/0:1H-56      [000] ....   913.456922: nvme_setup_cmd: nvme0: disk=nvme0n1, qid=1, cmdid=216, nsid=1, flags=0x0, meta=0x0, cmd=(nvme_cmd_read slba=0, len=65535, ctrl=0x0, dsmgmt=0, reftag=0)
     ksoftirqd/0-9       [000] .Ns.   916.566351: nvme_complete_rq: nvme0: disk=nvme0n1, qid=1, cmdid=216, res=0x0, retries=0, flags=0x0, status=0x4002

The patch below fixes the I/O errors by rejecting I/O requests from the
block layer with setting a flag to request_queue until the file descriptor re-opened to be updated by __blkdev_get(). This is based on the previous discussion [1].

Since V3(RFC):
- Move flag from gendisk to request_queue for future clean-ups.

    (Christoph, [3])

Since V2(RFC):
- Cover letter with testcase and error logs attached. Removed un-related
changes: empty line. (Chaitanya, [2])
- Put blkdev with blkdev_put_no_open().

Since V1(RFC):
- Updated patch to reject I/O rather than updating i_blkbits of the block device's inode directly from driver. (Christoph, [1])

[1] https://lore.kernel.org/linux-nvme/[email protected]/T/#t
[2] https://lore.kernel.org/linux-nvme/[email protected]/T/#t
[3] https://lore.kernel.org/linux-block/[email protected]/T/#u

Thanks,

Minwoo Im (1):
block: reject I/O for same fd if block size changed

 block/blk-settings.c    |  3 +++
 block/partitions/core.c | 12 ++++++++++++
 fs/block_dev.c          |  8 ++++++++
 include/linux/blkdev.h  |  1 +
 4 files changed, 24 insertions(+)
Signature · 7 lines
2.17.1


_______________________________________________
Linux-nvme mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-nvme
author
#1 lore raw

PATCH v4 1/1 block: reject I/O for same fd if block size changed

This patch fixes I/O errors during BLKRRPART ioctl() behavior right after format operation that changed logical block size of the block device with a same file descriptor opened.

This issue can be easily reproduced with a single format command in case of NVMe (logical block size 512B to 4096B).

	nvme format /dev/nvme0n1 --lbaf=1 --force

This is because the application, nvme-cli format subcommand issues an admin command followed by BLKRRPART ioctl to re-read partition information without closing the file descriptor. If file descriptor stays opened, __blkdev_get() will not be invoked at all even logical block size has been changed.

It will cause I/O errors with invalid Read operations during the BLKRRPART ioctl due to i_blkbits mismatch. The invalid operations in BLKRRPART happens with under-flowed Number of LBA(NLB) values 0xffff(65535) because i_blkbits is still set to 9 even the logical block size has been updated to 4096. The BLKRRPART will lead buffer_head to hold 512B data which is less than the logical lock size of the block device.

The root cause, which is because i_blkbits of inode of the block device is not updated, can be solved easily by re-opening file descriptor again from application. But, that's just for application's business and kernel should reject invalid Read operations during the BLKRRPART ioctl.

This patch rejects I/O from the path of add_partitions() to avoid issuing invalid Read operations to device. It sets a flag to request_queue in blk_queue_logical_block_size to minimize caller-side updates.

---
 block/blk-settings.c    |  3 +++
 block/partitions/core.c | 12 ++++++++++++
 fs/block_dev.c          |  8 ++++++++
 include/linux/blkdev.h  |  1 +
 4 files changed, 24 insertions(+)
block/blk-settings.c +3 -0
index 43990b1d148b..48a6fc7bb5f5 100644
@@ -329,6 +329,9 @@ void blk_queue_logical_block_size(struct request_queue *q, unsigned int size)
 {
 	struct queue_limits *limits = &q->limits;
 
+	if (limits->logical_block_size != size)
+		blk_queue_flag_set(QUEUE_FLAG_LBSZ_CHANGED, q);
+
 	limits->logical_block_size = size;
 
 	if (limits->physical_block_size < size)
block/partitions/core.c +12 -0
index e7d776db803b..6f175ea18ff3 100644
@@ -612,12 +612,24 @@ static bool blk_add_partition(struct gendisk *disk, struct block_device *bdev,
 
 int blk_add_partitions(struct gendisk *disk, struct block_device *bdev)
 {
+	struct request_queue *q = bdev_get_queue(bdev);
 	struct parsed_partitions *state;
 	int ret = -EAGAIN, p, highest;
 
 	if (!disk_part_scan_enabled(disk))
 		return 0;
 
+	/*
+	 * Reject to check partition information if block size has been changed
+	 * in the runtime.  If block size of a block device has been changed,
+	 * the file descriptor should be opened agian to update the blkbits.
+	 */
+	if (test_bit(QUEUE_FLAG_LBSZ_CHANGED, &q->queue_flags)) {
+		pr_warn("%s: rejecting checking partition. fd should be opened again.\n",
+				disk->disk_name);
+		return -EBADFD;
+	}
+
 	state = check_partition(disk, bdev);
 	if (!state)
 		return 0;
fs/block_dev.c +8 -0
index 9293045e128c..8056a412a3d1 100644
@@ -130,7 +130,15 @@ EXPORT_SYMBOL(truncate_bdev_range);
 
 static void set_init_blocksize(struct block_device *bdev)
 {
+	struct request_queue *q = bdev_get_queue(bdev);
+
 	bdev->bd_inode->i_blkbits = blksize_bits(bdev_logical_block_size(bdev));
+
+	/*
+	 * Allow I/O commands for this block device.  We can say that this
+	 * block device has proper blkbits updated.
+	 */
+	blk_queue_flag_clear(QUEUE_FLAG_LBSZ_CHANGED, q);
 }
 
 int set_blocksize(struct block_device *bdev, int size)
include/linux/blkdev.h +1 -0
index 070de09425ad..6d0542434be6 100644
@@ -625,6 +625,7 @@ struct request_queue {
 #define QUEUE_FLAG_RQ_ALLOC_TIME 27	/* record rq->alloc_time_ns */
 #define QUEUE_FLAG_HCTX_ACTIVE	28	/* at least one blk-mq hctx is active */
 #define QUEUE_FLAG_NOWAIT       29	/* device supports NOWAIT */
+#define QUEUE_FLAG_LBSZ_CHANGED	30	/* logical block size changed */
 
 #define QUEUE_FLAG_MQ_DEFAULT	((1 << QUEUE_FLAG_IO_STAT) |		\
 				 (1 << QUEUE_FLAG_SAME_COMP) |		\
Signature · 7 lines
2.17.1


_______________________________________________
Linux-nvme mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-nvme