fix direct io device mapper errors
Subsystems (2)
v2 cover fix direct io device mapper errors
From: Keith Busch <[email protected]>
The 6.0 kernel made some changes to the direct io interface to allow offsets in user addresses. This based on the hardware's capabilities reported in the request_queue's dma_alignment attribute.
dm-crypt, -log-writes and -integrity require direct io be aligned to the block size. Since it was only ever using the default 511 dma mask, this requirement may fail if formatted to something larger, like 4k, which will result in unexpected behavior with direct-io.
Changes since v1: Added the same fix for -integrity and -log-writes
The first three were reported successfully tested by Dmitrii Tcvetkov, but I don't have an official Tested-by: tag.
https://lore.kernel.org/linux-block/[email protected]/T/#mba1d0b13374541cdad3b669ec4257a11301d1860
Keitio errors on Busch (5):
block: make dma_alignment a stacking queue_limit
dm-crypt: provide dma_alignment limit in io_hints
block: make blk_set_default_limits() private
dm-integrity: set dma_alignment limit in io_hints
dm-log-writes: set dma_alignment limit in io_hints
block/blk-core.c | 1 - block/blk-settings.c | 9 +++++---- block/blk.h | 1 + drivers/md/dm-crypt.c | 1 + drivers/md/dm-integrity.c | 1 + drivers/md/dm-log-writes.c | 1 + include/linux/blkdev.h | 16 ++++++++-------- 7 files changed, 17 insertions(+), 13 deletions(-)
Signature · 1 line
2.30.2
PATCH v2 1/5 block: make dma_alignment a stacking queue_limit
From: Keith Busch <[email protected]>
Device mappers had always been getting the default 511 dma mask, but the underlying device might have a larger alignment requirement. Since this value is used to determine alloweable direct-io alignment, this needs to be a stackable limit.
--- block/blk-core.c | 1 - block/blk-settings.c | 8 +++++--- include/linux/blkdev.h | 15 ++++++++------- 3 files changed, 13 insertions(+), 11 deletions(-)
block/blk-core.c +0 -1
@@ -425,7 +425,6 @@ struct request_queue *blk_alloc_queue(int node_id, bool alloc_srcu) PERCPU_REF_INIT_ATOMIC, GFP_KERNEL)) goto fail_stats; -blk_queue_dma_alignment (q,511 );blk_set_default_limits (&q->limits); q->nr_requests =BLKDEV_DEFAULT_RQ ;
block/blk-settings.c +5 -3
@@ -57,6 +57,7 @@ void blk_set_default_limits(struct queue_limits *lim) lim->misaligned =0 ; lim->zoned =BLK_ZONED_NONE ; lim->zone_write_granularity =0 ; + lim->dma_alignment =511 ; }EXPORT_SYMBOL (blk_set_default_limits);
@@ -600,6 +601,7 @@ int blk_stack_limits(struct queue_limits *t, struct queue_limits *b, t->io_min =max (t->io_min, b->io_min); t->io_opt =lcm_not_zero (t->io_opt, b->io_opt); + t->dma_alignment =max (t->dma_alignment, b->dma_alignment);/* Set non-power-of-2 compatible chunk_sectors boundary */ if (b->chunk_sectors)
@@ -773,7 +775,7 @@ EXPORT_SYMBOL(blk_queue_virt_boundary);**/ void blk_queue_dma_alignment (struct request_queue *q,int mask) { - q->dma_alignment = mask; + q->limits.dma_alignment = mask; }EXPORT_SYMBOL (blk_queue_dma_alignment);
@@ -795,8 +797,8 @@ void blk_queue_update_dma_alignment(struct request_queue *q, int mask) {BUG_ON (mask >PAGE_SIZE ); -if (mask > q->dma_alignment) - q->dma_alignment = mask; +if (mask > q->limits.dma_alignment) + q->limits.dma_alignment = mask; }EXPORT_SYMBOL (blk_queue_update_dma_alignment);
include/linux/blkdev.h +8 -7
@@ -311,6 +311,13 @@ struct queue_limits {unsigned char discard_misaligned;unsigned char raid_partial_stripes_expensive;enum blk_zoned_model zoned; + +/* +* Drivers that set dma_alignment to less than 511 must be prepared to +* handle individual bvec's that are not a multiple of a SECTOR_SIZE +* due to possible offsets. +*/ +unsigned int dma_alignment; }; typedef int (*report_zones_cb)(struct blk_zone *zone, unsigned int idx,
@@ -456,12 +463,6 @@ struct request_queue {unsigned long nr_requests;/* Max # of requests */ unsigned int dma_pad_mask; -/* -* Drivers that set dma_alignment to less than 511 must be prepared to -* handle individual bvec's that are not a multiple of a SECTOR_SIZE -* due to possible offsets. -*/ -unsigned int dma_alignment;#ifdef CONFIG_BLK_INLINE_ENCRYPTION struct blk_crypto_profile *crypto_profile;
@@ -1324,7 +1325,7 @@ static inline sector_t bdev_zone_sectors(struct block_device *bdev)static inline int queue_dma_alignment (const struct request_queue *q) { -return q ? q->dma_alignment :511 ; +return q ? q->limits.dma_alignment :511 ; }static inline unsigned int bdev_dma_alignment (struct block_device *bdev)
Signature · 1 line
2.30.2
PATCH v2 2/5 dm-crypt: provide dma_alignment limit in io_hints
In reply to Keith BuschFrom: Keith Busch <[email protected]>
This device mapper needs bio vectors to be sized and memory aligned to the logical block size. Set the minimum required queue limit accordingly.
--- drivers/md/dm-crypt.c | 1 + 1 file changed, 1 insertion(+)
drivers/md/dm-crypt.c +1 -0
@@ -3630,6 +3630,7 @@ static void crypt_io_hints(struct dm_target *ti, struct queue_limits *limits) limits->physical_block_size =max_t (unsigned, limits->physical_block_size, cc->sector_size); limits->io_min =max_t (unsigned, limits->io_min, cc->sector_size); + limits->dma_alignment = limits->logical_block_size -1 ; }static struct target_type crypt_target = {
Signature · 1 line
2.30.2
Re: [PATCHv2 2/5] dm-crypt: provide dma_alignment limit in io_hints
Quoted 11 lines · Keith Busch
From: Keith Busch <[email protected]>
This device mapper needs bio vectors to be sized and memory aligned to the logical block size. Set the minimum required queue limit accordingly.
Fixes b1a000d3b8ec5 ("block: relax direct io memory alignment")Reportred-by Eric Biggers <[email protected]>Reported-by Dmitrii Tcvetkov <[email protected]>Signed-off-by Keith Busch <[email protected]>
--
dm-devel mailing list
[email protected]
https://listman.redhat.com/mailman/listinfo/dm-devel
PATCH v2 3/5 block: make blk_set_default_limits() private
In reply to Keith BuschFrom: Keith Busch <[email protected]>
There are no external users of this function.
--- block/blk-settings.c | 1 - block/blk.h | 1 + include/linux/blkdev.h | 1 - 3 files changed, 1 insertion(+), 2 deletions(-)
block/blk-settings.c +0 -1
@@ -59,7 +59,6 @@ void blk_set_default_limits(struct queue_limits *lim) lim->zone_write_granularity =0 ; lim->dma_alignment =511 ; } -EXPORT_SYMBOL (blk_set_default_limits);/** * blk_set_stacking_limits - set default limits for stacking devices
block/blk.h +1 -0
@@ -331,6 +331,7 @@ void blk_rq_set_mixed_merge(struct request *rq);bool blk_rq_merge_ok (struct request *rq,struct bio *bio);enum elv_merge blk_try_merge (struct request *rq,struct bio *bio); +void blk_set_default_limits (struct queue_limits *lim);int blk_dev_init (void );/*
include/linux/blkdev.h +0 -1
@@ -945,7 +945,6 @@ extern void blk_queue_io_min(struct request_queue *q, unsigned int min);extern void blk_limits_io_opt (struct queue_limits *limits,unsigned int opt);extern void blk_queue_io_opt (struct request_queue *q,unsigned int opt);extern void blk_set_queue_depth (struct request_queue *q,unsigned int depth); -extern void blk_set_default_limits (struct queue_limits *lim);extern void blk_set_stacking_limits (struct queue_limits *lim);extern int blk_stack_limits (struct queue_limits *t,struct queue_limits *b,sector_t offset);
Signature · 1 line
2.30.2
PATCH v2 4/5 dm-integrity: set dma_alignment limit in io_hints
In reply to Keith BuschFrom: Keith Busch <[email protected]>
This device mapper needs bio vectors to be sized and memory aligned to the logical block size. Set the minimum required queue limit accordingly.
--- drivers/md/dm-integrity.c | 1 + 1 file changed, 1 insertion(+)
drivers/md/dm-integrity.c +1 -0
@@ -3370,6 +3370,7 @@ static void dm_integrity_io_hints(struct dm_target *ti, struct queue_limits *lim limits->logical_block_size = ic->sectors_per_block <<SECTOR_SHIFT ; limits->physical_block_size = ic->sectors_per_block <<SECTOR_SHIFT ;blk_limits_io_min (limits, ic->sectors_per_block <<SECTOR_SHIFT ); + limits->dma_alignment = limits->logical_block_size -1 ; } }
Signature · 1 line
2.30.2
Re: [PATCHv2 4/5] dm-integrity: set dma_alignment limit in io_hints
Quoted 7 lines · Keith Busch
From: Keith Busch <[email protected]>
This device mapper needs bio vectors to be sized and memory aligned to the logical block size. Set the minimum required queue limit accordingly.
Signed-off-by Keith Busch <[email protected]>
--
dm-devel mailing list
[email protected]
https://listman.redhat.com/mailman/listinfo/dm-devel
PATCH v2 5/5 dm-log-writes: set dma_alignment limit in io_hints
In reply to Keith BuschFrom: Keith Busch <[email protected]>
This device mapper needs bio vectors to be sized and memory aligned to the logical block size. Set the minimum required queue limit accordingly.
--- drivers/md/dm-log-writes.c | 1 + 1 file changed, 1 insertion(+)
drivers/md/dm-log-writes.c +1 -0
@@ -875,6 +875,7 @@ static void log_writes_io_hints(struct dm_target *ti, struct queue_limits *limit limits->logical_block_size =bdev_logical_block_size (lc->dev->bdev); limits->physical_block_size =bdev_physical_block_size (lc->dev->bdev); limits->io_min = limits->physical_block_size; + limits->dma_alignment = limits->logical_block_size -1 ; }#if IS_ENABLED (CONFIG_FS_DAX )
Signature · 1 line
2.30.2
Re: [PATCHv2 5/5] dm-log-writes: set dma_alignment limit in io_hints
Quoted 7 lines · Keith Busch
From: Keith Busch <[email protected]>
This device mapper needs bio vectors to be sized and memory aligned to the logical block size. Set the minimum required queue limit accordingly.
Signed-off-by Keith Busch <[email protected]>
--
dm-devel mailing list
[email protected]
https://listman.redhat.com/mailman/listinfo/dm-devel
Quoted 25 lines · Keith Busch
From: Keith Busch <[email protected]>
The 6.0 kernel made some changes to the direct io interface to allow offsets in user addresses. This based on the hardware's capabilities reported in the request_queue's
dma_alignmentattribute.dm-crypt, -log-writes and -integrity require direct io be aligned to the block size. Since it was only ever using the default 511 dma mask, this requirement may fail if formatted to something larger, like 4k, which will result in unexpected behavior with direct-io.
Changes since v1: Added the same fix for -integrity and -log-writes
The first three were reported successfully tested by Dmitrii Tcvetkov, but I don't have an official Tested-by: tag.
https://lore.kernel.org/linux-block/[email protected]/T/#mba1d0b13374541cdad3b669ec4257a11301d1860
Keitio errors on Busch (5):
block: makedma_alignmenta stackingqueue_limit
dm-crypt: providedma_alignmentlimit inio_hints
block: makeblk_set_default_limits()private
dm-integrity: setdma_alignmentlimit inio_hints
dm-log-writes: setdma_alignmentlimit inio_hints
Quoted 12 lines
block/blk-core.c | 1 - block/blk-settings.c | 9 +++++---- block/blk.h | 1 + drivers/md/dm-crypt.c | 1 + drivers/md/dm-integrity.c | 1 + drivers/md/dm-log-writes.c | 1 + include/linux/blkdev.h | 16 ++++++++-------- 7 files changed, 17 insertions(+), 13 deletions(-)Signature · 1 line
2.30.2
There are other DM targets that override logical_block_size in their .io_hints hook (writecache, ebs, zoned). Have you reasoned through why those do _not_ need updating too?
Is there any risk of just introducing a finalization method in block core (that DM's .io_hints would call) that would ensure limits that are a funtion of another are always in sync? Would avoid whack-a-mole issues in the future.
Thanks,
Mike
--
dm-devel mailing list
[email protected]
https://listman.redhat.com/mailman/listinfo/dm-devel
Quoted 37 lines · Mike Snitzer
Quoted 34 lines · Keith Busch
From: Keith Busch <[email protected]>
The 6.0 kernel made some changes to the direct io interface to allow offsets in user addresses. This based on the hardware's capabilities reported in the request_queue's
dma_alignmentattribute.dm-crypt, -log-writes and -integrity require direct io be aligned to the block size. Since it was only ever using the default 511 dma mask, this requirement may fail if formatted to something larger, like 4k, which will result in unexpected behavior with direct-io.
Changes since v1: Added the same fix for -integrity and -log-writes
The first three were reported successfully tested by Dmitrii Tcvetkov, but I don't have an official Tested-by: tag.
https://lore.kernel.org/linux-block/[email protected]/T/#mba1d0b13374541cdad3b669ec4257a11301d1860
Keitio errors on Busch (5):
block: makedma_alignmenta stackingqueue_limit
dm-crypt: providedma_alignmentlimit inio_hints
block: makeblk_set_default_limits()private
dm-integrity: setdma_alignmentlimit inio_hints
dm-log-writes: setdma_alignmentlimit inio_hintsblock/blk-core.c | 1 - block/blk-settings.c | 9 +++++---- block/blk.h | 1 + drivers/md/dm-crypt.c | 1 + drivers/md/dm-integrity.c | 1 + drivers/md/dm-log-writes.c | 1 + include/linux/blkdev.h | 16 ++++++++-------- 7 files changed, 17 insertions(+), 13 deletions(-)
Quoted 7 lines
Quoted 3 lines
Signature · 1 line
2.30.2There are other DM targets that override
logical_block_sizein their .io_hints hook (writecache, ebs, zoned). Have you reasoned through why those do _not_ need updating too?
Yeah, that's a good question. The ones that have a problem all make assumptions about a bio's bv_offset being logical block size aligned, and each of those is accounted for here. Everything else looks fine with respect to handling offsets.
Quoted 4 lines
Is there any risk of just introducing a finalization method in block core (that DM's .io_hints would call) that would ensure limits that are a funtion of another are always in sync? Would avoid whack-a-mole issues in the future.
I don't think we can this particular limit issue. A lot of drivers and devices can use a smaller dma_alignment than the logical block size, so the generic block layer wouldn't really know the driver's intentions for the relationship of these limits.
Mikulas Patocka
maintainerQuoted 8 lines · Keith Busch
Quoted 3 lines
There are other DM targets that override
logical_block_sizein their .io_hints hook (writecache, ebs, zoned). Have you reasoned through why those do _not_ need updating too?Yeah, that's a good question. The ones that have a problem all make assumptions about a bio's
bv_offsetbeing logical block size aligned, and each of those is accounted for here. Everything else looks fine with respect to handling offsets.
Unaligned bv_offset should work - because XFS is sending such bios. If you compile the kernel with memory debugging, kmalloc returns unaligned memory. XFS will allocate a buffer with kmalloc, test if it crosses a page boundary, if not, use the buffer, if yes, free the buffer and allocate a full page.
There have been device mapper problems about unaligned bv_offset in the past and I have fixed them.
Unaligned bv_length is a problem for the affected targets.
Mikulas
--
dm-devel mailing list
[email protected]
https://listman.redhat.com/mailman/listinfo/dm-devel
Quoted 19 lines · Mikulas Patocka
Quoted 8 lines · Keith Busch
Quoted 3 lines
There are other DM targets that override
logical_block_sizein their .io_hints hook (writecache, ebs, zoned). Have you reasoned through why those do _not_ need updating too?Yeah, that's a good question. The ones that have a problem all make assumptions about a bio's
bv_offsetbeing logical block size aligned, and each of those is accounted for here. Everything else looks fine with respect to handling offsets.Unaligned
bv_offsetshould work - because XFS is sending such bios. If you compile the kernel with memory debugging, kmalloc returns unaligned memory. XFS will allocate a buffer with kmalloc, test if it crosses a page boundary, if not, use the buffer, if yes, free the buffer and allocate a full page.
Quoted 4 lines
There have been device mapper problems about unaligned
bv_offsetin the past and I have fixed them.Unaligned
bv_lengthis a problem for the affected targets.
kmalloc is physically contiguous, and bio's support multi-page bvecs for these, so the bv_len is always aligned if the kmalloc is sized correctly. The unaligned offsets become a problem with virtually contiguous buffers since individual bv lengths might not be block size aligned when bv offsets exist.
Quoted 6 lines · Keith Busch
From: Keith Busch <[email protected]>
The 6.0 kernel made some changes to the direct io interface to allow offsets in user addresses. This based on the hardware's capabilities reported in the request_queue's
dma_alignmentattribute.
Quoted 6 lines
dm-crypt, -log-writes and -integrity require direct io be aligned to the block size. Since it was only ever using the default 511 dma mask, this requirement may fail if formatted to something larger, like 4k, which will result in unexpected behavior with direct-io.
[...]
Applied, thanks!
[1/5] block: make dma_alignment a stacking queue_limit
commit: c964d62f5cab7b43dd0534f22a96eab386c6ec5d
[2/5] dm-crypt: provide dma_alignment limit in io_hints
commit: 86e4d3e8d1838ca88fb9267e669c36f6c8f7c6cd
[3/5] block: make blk_set_default_limits() private
commit: b3228254bb6e91e57f920227f72a1a7d81925d81
[4/5] dm-integrity: set dma_alignment limit in io_hints
commit: 29aa778bb66795e6a78b1c99beadc83887827868
[5/5] dm-log-writes: set dma_alignment limit in io_hints
commit: 50a893359cd2643ee1afc96eedc9e7084cab49faBest regards,
Signature · 1 line
Jens Axboe