Skip to contentSkip to conversation

fix direct io device mapper errors

PATCH Applied Keith Busch 5 participants 14 messages Nov 10, 2022 – Nov 16, 2022

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

v2 cover fix direct io device mapper errors

From: Keith Busch <[email protected]>

The 6.0 kernel made some changes to the direct io interface to allow offsets in user addresses. This based on the hardware's capabilities reported in the request_queue's dma_alignment attribute.

dm-crypt, -log-writes and -integrity require direct io be aligned to the block size. Since it was only ever using the default 511 dma mask, this requirement may fail if formatted to something larger, like 4k, which will result in unexpected behavior with direct-io.

Changes since v1: Added the same fix for -integrity and -log-writes

The first three were reported successfully tested by Dmitrii Tcvetkov, but I don't have an official Tested-by: tag.

https://lore.kernel.org/linux-block/[email protected]/T/#mba1d0b13374541cdad3b669ec4257a11301d1860

Keitio errors on Busch (5):
block: make dma_alignment a stacking queue_limit
dm-crypt: provide dma_alignment limit in io_hints
block: make blk_set_default_limits() private
dm-integrity: set dma_alignment limit in io_hints
dm-log-writes: set dma_alignment limit in io_hints

 block/blk-core.c           |  1 -
 block/blk-settings.c       |  9 +++++----
 block/blk.h                |  1 +
 drivers/md/dm-crypt.c      |  1 +
 drivers/md/dm-integrity.c  |  1 +
 drivers/md/dm-log-writes.c |  1 +
 include/linux/blkdev.h     | 16 ++++++++--------
 7 files changed, 17 insertions(+), 13 deletions(-)
Signature · 1 line
2.30.2
#1 lore raw

PATCH v2 1/5 block: make dma_alignment a stacking queue_limit

From: Keith Busch <[email protected]>

Device mappers had always been getting the default 511 dma mask, but the underlying device might have a larger alignment requirement. Since this value is used to determine alloweable direct-io alignment, this needs to be a stackable limit.

---
 block/blk-core.c       |  1 -
 block/blk-settings.c   |  8 +++++---
 include/linux/blkdev.h | 15 ++++++++-------
 3 files changed, 13 insertions(+), 11 deletions(-)
block/blk-core.c +0 -1
index 17667159482e..5487912befe8 100644
@@ -425,7 +425,6 @@ struct request_queue *blk_alloc_queue(int node_id, bool alloc_srcu)
 				PERCPU_REF_INIT_ATOMIC, GFP_KERNEL))
 		goto fail_stats;
 
-	blk_queue_dma_alignment(q, 511);
 	blk_set_default_limits(&q->limits);
 	q->nr_requests = BLKDEV_DEFAULT_RQ;
 
block/blk-settings.c +5 -3
index 8bb9eef5310e..4949ed3ce7c9 100644
@@ -57,6 +57,7 @@ void blk_set_default_limits(struct queue_limits *lim)
 	lim->misaligned = 0;
 	lim->zoned = BLK_ZONED_NONE;
 	lim->zone_write_granularity = 0;
+	lim->dma_alignment = 511;
 }
 EXPORT_SYMBOL(blk_set_default_limits);
 
@@ -600,6 +601,7 @@ int blk_stack_limits(struct queue_limits *t, struct queue_limits *b,
 
 	t->io_min = max(t->io_min, b->io_min);
 	t->io_opt = lcm_not_zero(t->io_opt, b->io_opt);
+	t->dma_alignment = max(t->dma_alignment, b->dma_alignment);
 
 	/* Set non-power-of-2 compatible chunk_sectors boundary */
 	if (b->chunk_sectors)
@@ -773,7 +775,7 @@ EXPORT_SYMBOL(blk_queue_virt_boundary);
  **/
 void blk_queue_dma_alignment(struct request_queue *q, int mask)
 {
-	q->dma_alignment = mask;
+	q->limits.dma_alignment = mask;
 }
 EXPORT_SYMBOL(blk_queue_dma_alignment);
 
@@ -795,8 +797,8 @@ void blk_queue_update_dma_alignment(struct request_queue *q, int mask)
 {
 	BUG_ON(mask > PAGE_SIZE);
 
-	if (mask > q->dma_alignment)
-		q->dma_alignment = mask;
+	if (mask > q->limits.dma_alignment)
+		q->limits.dma_alignment = mask;
 }
 EXPORT_SYMBOL(blk_queue_update_dma_alignment);
 
include/linux/blkdev.h +8 -7
index 50e358a19d98..9898e34b2c2d 100644
@@ -311,6 +311,13 @@ struct queue_limits {
 	unsigned char		discard_misaligned;
 	unsigned char		raid_partial_stripes_expensive;
 	enum blk_zoned_model	zoned;
+
+	/*
+	 * Drivers that set dma_alignment to less than 511 must be prepared to
+	 * handle individual bvec's that are not a multiple of a SECTOR_SIZE
+	 * due to possible offsets.
+	 */
+	unsigned int		dma_alignment;
 };
 
 typedef int (*report_zones_cb)(struct blk_zone *zone, unsigned int idx,
@@ -456,12 +463,6 @@ struct request_queue {
 	unsigned long		nr_requests;	/* Max # of requests */
 
 	unsigned int		dma_pad_mask;
-	/*
-	 * Drivers that set dma_alignment to less than 511 must be prepared to
-	 * handle individual bvec's that are not a multiple of a SECTOR_SIZE
-	 * due to possible offsets.
-	 */
-	unsigned int		dma_alignment;
 
 #ifdef CONFIG_BLK_INLINE_ENCRYPTION
 	struct blk_crypto_profile *crypto_profile;
@@ -1324,7 +1325,7 @@ static inline sector_t bdev_zone_sectors(struct block_device *bdev)
 
 static inline int queue_dma_alignment(const struct request_queue *q)
 {
-	return q ? q->dma_alignment : 511;
+	return q ? q->limits.dma_alignment : 511;
 }
 
 static inline unsigned int bdev_dma_alignment(struct block_device *bdev)
Signature · 1 line
2.30.2
#2 lore raw

PATCH v2 2/5 dm-crypt: provide dma_alignment limit in io_hints

In reply to Keith Busch

From: Keith Busch <[email protected]>

This device mapper needs bio vectors to be sized and memory aligned to the logical block size. Set the minimum required queue limit accordingly.

---
 drivers/md/dm-crypt.c | 1 +
 1 file changed, 1 insertion(+)
drivers/md/dm-crypt.c +1 -0
index 159c6806c19b..2653516bcdef 100644
@@ -3630,6 +3630,7 @@ static void crypt_io_hints(struct dm_target *ti, struct queue_limits *limits)
 	limits->physical_block_size =
 		max_t(unsigned, limits->physical_block_size, cc->sector_size);
 	limits->io_min = max_t(unsigned, limits->io_min, cc->sector_size);
+	limits->dma_alignment = limits->logical_block_size - 1;
 }
 
 static struct target_type crypt_target = {
Signature · 1 line
2.30.2
Reviewed-by on v2 2/5
#3 lore raw

Re: [PATCHv2 2/5] dm-crypt: provide dma_alignment limit in io_hints

Quoted 11 lines · Keith Busch

From: Keith Busch <[email protected]>

This device mapper needs bio vectors to be sized and memory aligned to the logical block size. Set the minimum required queue limit accordingly.

--
dm-devel mailing list
[email protected]
https://listman.redhat.com/mailman/listinfo/dm-devel

#4 lore raw

PATCH v2 3/5 block: make blk_set_default_limits() private

In reply to Keith Busch

From: Keith Busch <[email protected]>

There are no external users of this function.

---
 block/blk-settings.c   | 1 -
 block/blk.h            | 1 +
 include/linux/blkdev.h | 1 -
 3 files changed, 1 insertion(+), 2 deletions(-)
block/blk-settings.c +0 -1
index 4949ed3ce7c9..8ac1038d0c79 100644
@@ -59,7 +59,6 @@ void blk_set_default_limits(struct queue_limits *lim)
 	lim->zone_write_granularity = 0;
 	lim->dma_alignment = 511;
 }
-EXPORT_SYMBOL(blk_set_default_limits);
 
 /**
  * blk_set_stacking_limits - set default limits for stacking devices
block/blk.h +1 -0
index d6ea0d1a6db0..a186ea20f39d 100644
@@ -331,6 +331,7 @@ void blk_rq_set_mixed_merge(struct request *rq);
 bool blk_rq_merge_ok(struct request *rq, struct bio *bio);
 enum elv_merge blk_try_merge(struct request *rq, struct bio *bio);
 
+void blk_set_default_limits(struct queue_limits *lim);
 int blk_dev_init(void);
 
 /*
include/linux/blkdev.h +0 -1
index 9898e34b2c2d..891f8cbcd043 100644
@@ -945,7 +945,6 @@ extern void blk_queue_io_min(struct request_queue *q, unsigned int min);
 extern void blk_limits_io_opt(struct queue_limits *limits, unsigned int opt);
 extern void blk_queue_io_opt(struct request_queue *q, unsigned int opt);
 extern void blk_set_queue_depth(struct request_queue *q, unsigned int depth);
-extern void blk_set_default_limits(struct queue_limits *lim);
 extern void blk_set_stacking_limits(struct queue_limits *lim);
 extern int blk_stack_limits(struct queue_limits *t, struct queue_limits *b,
 			    sector_t offset);
Signature · 1 line
2.30.2
#5 lore raw

PATCH v2 4/5 dm-integrity: set dma_alignment limit in io_hints

In reply to Keith Busch

From: Keith Busch <[email protected]>

This device mapper needs bio vectors to be sized and memory aligned to the logical block size. Set the minimum required queue limit accordingly.

---
 drivers/md/dm-integrity.c | 1 +
 1 file changed, 1 insertion(+)
drivers/md/dm-integrity.c +1 -0
index aaf2472df6e5..e1e7b205573f 100644
@@ -3370,6 +3370,7 @@ static void dm_integrity_io_hints(struct dm_target *ti, struct queue_limits *lim
 		limits->logical_block_size = ic->sectors_per_block << SECTOR_SHIFT;
 		limits->physical_block_size = ic->sectors_per_block << SECTOR_SHIFT;
 		blk_limits_io_min(limits, ic->sectors_per_block << SECTOR_SHIFT);
+		limits->dma_alignment = limits->logical_block_size - 1;
 	}
 }
 
Signature · 1 line
2.30.2
Reviewed-by on v2 4/5
#6 lore raw

Re: [PATCHv2 4/5] dm-integrity: set dma_alignment limit in io_hints

Quoted 7 lines · Keith Busch

From: Keith Busch <[email protected]>

This device mapper needs bio vectors to be sized and memory aligned to the logical block size. Set the minimum required queue limit accordingly.

--
dm-devel mailing list
[email protected]
https://listman.redhat.com/mailman/listinfo/dm-devel

#7 lore raw

PATCH v2 5/5 dm-log-writes: set dma_alignment limit in io_hints

In reply to Keith Busch

From: Keith Busch <[email protected]>

This device mapper needs bio vectors to be sized and memory aligned to the logical block size. Set the minimum required queue limit accordingly.

---
 drivers/md/dm-log-writes.c | 1 +
 1 file changed, 1 insertion(+)
drivers/md/dm-log-writes.c +1 -0
index 20fd688f72e7..178e13a5b059 100644
@@ -875,6 +875,7 @@ static void log_writes_io_hints(struct dm_target *ti, struct queue_limits *limit
 	limits->logical_block_size = bdev_logical_block_size(lc->dev->bdev);
 	limits->physical_block_size = bdev_physical_block_size(lc->dev->bdev);
 	limits->io_min = limits->physical_block_size;
+	limits->dma_alignment = limits->logical_block_size - 1;
 }
 
 #if IS_ENABLED(CONFIG_FS_DAX)
Signature · 1 line
2.30.2
Reviewed-by on v2 5/5
#8 lore raw

Re: [PATCHv2 5/5] dm-log-writes: set dma_alignment limit in io_hints

Quoted 7 lines · Keith Busch

From: Keith Busch <[email protected]>

This device mapper needs bio vectors to be sized and memory aligned to the logical block size. Set the minimum required queue limit accordingly.

--
dm-devel mailing list
[email protected]
https://listman.redhat.com/mailman/listinfo/dm-devel

#9 lore raw
In reply to Keith Busch
Quoted 25 lines · Keith Busch

From: Keith Busch <[email protected]>

The 6.0 kernel made some changes to the direct io interface to allow offsets in user addresses. This based on the hardware's capabilities reported in the request_queue's dma_alignment attribute.

dm-crypt, -log-writes and -integrity require direct io be aligned to the block size. Since it was only ever using the default 511 dma mask, this requirement may fail if formatted to something larger, like 4k, which will result in unexpected behavior with direct-io.

Changes since v1: Added the same fix for -integrity and -log-writes

The first three were reported successfully tested by Dmitrii Tcvetkov, but I don't have an official Tested-by: tag.

https://lore.kernel.org/linux-block/[email protected]/T/#mba1d0b13374541cdad3b669ec4257a11301d1860

Keitio errors on Busch (5):
block: make dma_alignment a stacking queue_limit
dm-crypt: provide dma_alignment limit in io_hints
block: make blk_set_default_limits() private
dm-integrity: set dma_alignment limit in io_hints
dm-log-writes: set dma_alignment limit in io_hints

Quoted 12 lines
 block/blk-core.c           |  1 -
 block/blk-settings.c       |  9 +++++----
 block/blk.h                |  1 +
 drivers/md/dm-crypt.c      |  1 +
 drivers/md/dm-integrity.c  |  1 +
 drivers/md/dm-log-writes.c |  1 +
 include/linux/blkdev.h     | 16 ++++++++--------
 7 files changed, 17 insertions(+), 13 deletions(-)
Signature · 1 line
2.30.2

There are other DM targets that override logical_block_size in their .io_hints hook (writecache, ebs, zoned). Have you reasoned through why those do _not_ need updating too?

Is there any risk of just introducing a finalization method in block core (that DM's .io_hints would call) that would ensure limits that are a funtion of another are always in sync? Would avoid whack-a-mole issues in the future.

Thanks,
Mike

--
dm-devel mailing list
[email protected]
https://listman.redhat.com/mailman/listinfo/dm-devel

#10 lore raw
Quoted 37 lines · Mike Snitzer
Quoted 34 lines · Keith Busch

From: Keith Busch <[email protected]>

The 6.0 kernel made some changes to the direct io interface to allow offsets in user addresses. This based on the hardware's capabilities reported in the request_queue's dma_alignment attribute.

dm-crypt, -log-writes and -integrity require direct io be aligned to the block size. Since it was only ever using the default 511 dma mask, this requirement may fail if formatted to something larger, like 4k, which will result in unexpected behavior with direct-io.

Changes since v1: Added the same fix for -integrity and -log-writes

The first three were reported successfully tested by Dmitrii Tcvetkov, but I don't have an official Tested-by: tag.

https://lore.kernel.org/linux-block/[email protected]/T/#mba1d0b13374541cdad3b669ec4257a11301d1860

Keitio errors on Busch (5):
block: make dma_alignment a stacking queue_limit
dm-crypt: provide dma_alignment limit in io_hints
block: make blk_set_default_limits() private
dm-integrity: set dma_alignment limit in io_hints
dm-log-writes: set dma_alignment limit in io_hints

 block/blk-core.c           |  1 -
 block/blk-settings.c       |  9 +++++----
 block/blk.h                |  1 +
 drivers/md/dm-crypt.c      |  1 +
 drivers/md/dm-integrity.c  |  1 +
 drivers/md/dm-log-writes.c |  1 +
 include/linux/blkdev.h     | 16 ++++++++--------
 7 files changed, 17 insertions(+), 13 deletions(-)
Quoted 7 lines
Quoted 3 lines
Signature · 1 line
2.30.2

There are other DM targets that override logical_block_size in their .io_hints hook (writecache, ebs, zoned). Have you reasoned through why those do _not_ need updating too?

Yeah, that's a good question. The ones that have a problem all make assumptions about a bio's bv_offset being logical block size aligned, and each of those is accounted for here. Everything else looks fine with respect to handling offsets.

Quoted 4 lines

Is there any risk of just introducing a finalization method in block core (that DM's .io_hints would call) that would ensure limits that are a funtion of another are always in sync? Would avoid whack-a-mole issues in the future.

I don't think we can this particular limit issue. A lot of drivers and devices can use a smaller dma_alignment than the logical block size, so the generic block layer wouldn't really know the driver's intentions for the relationship of these limits.

maintainer
#11 lore raw
Quoted 8 lines · Keith Busch
Quoted 3 lines

There are other DM targets that override logical_block_size in their .io_hints hook (writecache, ebs, zoned). Have you reasoned through why those do _not_ need updating too?

Yeah, that's a good question. The ones that have a problem all make assumptions about a bio's bv_offset being logical block size aligned, and each of those is accounted for here. Everything else looks fine with respect to handling offsets.

Unaligned bv_offset should work - because XFS is sending such bios. If you compile the kernel with memory debugging, kmalloc returns unaligned memory. XFS will allocate a buffer with kmalloc, test if it crosses a page boundary, if not, use the buffer, if yes, free the buffer and allocate a full page.

There have been device mapper problems about unaligned bv_offset in the past and I have fixed them.

Unaligned bv_length is a problem for the affected targets.

Mikulas
--
dm-devel mailing list
[email protected]
https://listman.redhat.com/mailman/listinfo/dm-devel

#12 lore raw
Quoted 19 lines · Mikulas Patocka
Quoted 8 lines · Keith Busch
Quoted 3 lines

There are other DM targets that override logical_block_size in their .io_hints hook (writecache, ebs, zoned). Have you reasoned through why those do _not_ need updating too?

Yeah, that's a good question. The ones that have a problem all make assumptions about a bio's bv_offset being logical block size aligned, and each of those is accounted for here. Everything else looks fine with respect to handling offsets.

Unaligned bv_offset should work - because XFS is sending such bios. If you compile the kernel with memory debugging, kmalloc returns unaligned memory. XFS will allocate a buffer with kmalloc, test if it crosses a page boundary, if not, use the buffer, if yes, free the buffer and allocate a full page.

Quoted 4 lines

There have been device mapper problems about unaligned bv_offset in the past and I have fixed them.

Unaligned bv_length is a problem for the affected targets.

kmalloc is physically contiguous, and bio's support multi-page bvecs for these, so the bv_len is always aligned if the kmalloc is sized correctly. The unaligned offsets become a problem with virtually contiguous buffers since individual bv lengths might not be block size aligned when bv offsets exist.

maintainer Applied
#13 lore raw
In reply to Keith Busch
Quoted 6 lines · Keith Busch

From: Keith Busch <[email protected]>

The 6.0 kernel made some changes to the direct io interface to allow offsets in user addresses. This based on the hardware's capabilities reported in the request_queue's dma_alignment attribute.

Quoted 6 lines

dm-crypt, -log-writes and -integrity require direct io be aligned to the block size. Since it was only ever using the default 511 dma mask, this requirement may fail if formatted to something larger, like 4k, which will result in unexpected behavior with direct-io.

[...]

Applied, thanks!

[1/5] block: make dma_alignment a stacking queue_limit

      commit: c964d62f5cab7b43dd0534f22a96eab386c6ec5d
[2/5] dm-crypt: provide dma_alignment limit in io_hints
      commit: 86e4d3e8d1838ca88fb9267e669c36f6c8f7c6cd
[3/5] block: make blk_set_default_limits() private
      commit: b3228254bb6e91e57f920227f72a1a7d81925d81
[4/5] dm-integrity: set dma_alignment limit in io_hints
      commit: 29aa778bb66795e6a78b1c99beadc83887827868
[5/5] dm-log-writes: set dma_alignment limit in io_hints
      commit: 50a893359cd2643ee1afc96eedc9e7084cab49fa

Best regards,

Signature · 1 line
Jens Axboe