bdev: use bdev_io_min() for statx DIO min IO
Subsystems (2)
RFC 1/1 bdev: use bdev_io_min() for statx DIO min IO
We currently rely on the block device logical block size for the offset alignment. While this *works* it doesn't work with performance in mind. That's exactly what the minimum_io_size attribute is for.
This would for example enhance performance for DIO on 4k IU drives which have for example an LBA format of 512 bytes for both HDDs and NVMe. Another use case is to ensure that DIO will be used with 16k IOs on existing market 16k IU drives with an LBA format of 4k or 512 bytes.
--- block/bdev.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
block/bdev.c +1 -1
@@ -1282,7 +1282,7 @@ void bdev_statx(struct inode *backing_inode, struct kstat *stat,if (request_mask &STATX_DIOALIGN ) { stat->dio_mem_align =bdev_dma_alignment (bdev) +1 ; - stat->dio_offset_align =bdev_logical_block_size (bdev); + stat->dio_offset_align = (unsigned int )bdev_io_min (bdev); stat->result_mask |=STATX_DIOALIGN ; }
Signature · 1 line
2.43.0
Quoted 8 lines · Luis Chamberlain
We currently rely on the block device logical block size for the offset alignment. While this *works* it doesn't work with performance in mind. That's exactly what the
minimum_io_sizeattribute is for.This would for example enhance performance for DIO on 4k IU drives which have for example an LBA format of 512 bytes for both HDDs and NVMe. Another use case is to ensure that DIO will be used with 16k IOs on existing market 16k IU drives with an LBA format of 4k or 512 bytes.
The minimum_io_size clearly is the minimum I/O size, not the minimal nice to have one. Changing this will break existing setups.
Quoted 5 lines · Christoph Hellwig
Quoted 4 lines · Luis Chamberlain
We currently rely on the block device logical block size for the offset alignment. While this *works* it doesn't work with performance in mind. That's exactly what the
minimum_io_sizeattribute is for.
Quoted 7 lines
Quoted 4 lines
This would for example enhance performance for DIO on 4k IU drives which have for example an LBA format of 512 bytes for both HDDs and NVMe. Another use case is to ensure that DIO will be used with 16k IOs on existing market 16k IU drives with an LBA format of 4k or 512 bytes.
The
minimum_io_sizeclearly is the minimum I/O size, not the minimal nice to have one.
I may have misread the below documentation then, because it seems to suggest this is a performance parameter, not a real minimum. Do we need to update it?
What: /sys/block/<disk>/queue/minimum_io_size Date: April 2009 Contact: Martin K. Petersen <[email protected]>
Description:
[RO] Storage devices may report a granularity or preferred minimum I/O size which is the smallest request the device can perform without incurring a performance penalty. For disk drives this is often the physical block size. For RAID arrays it is often the stripe chunk size. A properly aligned multiple of minimum_io_size is the preferred request size for workloads where a high number of I/O operations is desired.
If this is not the right place, do we need to use a new topology entry for the IU? Today the NVMe drive uses it for the NPWG which these days is the IU.
Quoted 1 line
Changing this will break existing setups.
My impression was that STATX_DIOALIGN was rather new, so any new enhancements due to the difficulties in getting DIO alignment right, this was the right place to do so.
Let me know if you have other suggestions.
Luis
Quoted 6 lines · Luis Chamberlain
Quoted 2 lines
The
minimum_io_sizeclearly is the minimum I/O size, not the minimal nice to have one.I may have misread the below documentation then, because it seems to suggest this is a performance parameter, not a real minimum. Do we need to update it?
queue_limits.min_io is corretly described and a performance hint. The statx dio_offset_align is actual minimum I/O size and alignment and not in any way related to the performance hint in minimum_io_size.
Quoted 9 lines · Christoph Hellwig
Quoted 6 lines · Luis Chamberlain
Quoted 2 lines
The
minimum_io_sizeclearly is the minimum I/O size, not the minimal nice to have one.I may have misread the below documentation then, because it seems to suggest this is a performance parameter, not a real minimum. Do we need to update it?
queue_limits.min_io is corretly described and a performance hint.
OK, great!
Quoted 2 lines
The statx
dio_offset_alignis actual minimum I/O size and alignment and not in any way related to the performance hint inminimum_io_size.
Oh, darn, I just read again 825cf206ed510 ("statx: add direct I/O alignment information") and the block layer change through commit 2d985f8c6b91b ("vfs: support STATX_DIOALIGN on block devices") and no where do I see any mention of it being a min. Should we clarify that?
And should we add a respective value for performance? I suspect userspace will want to work with optimal values, not ones which could for instance incur read-modify-write. Altough we have BLKIOMIN to get the optimal performance min IO and BLKIOOPT to get the optimal size it is not terribly clear to me that users know they should prefer to align to BLKIOMIN and use that for an DIO size for writes when possible.
Luis
Quoted 13 lines · Luis Chamberlain
Quoted 9 lines · Christoph Hellwig
Quoted 6 lines · Luis Chamberlain
Quoted 2 lines
The
minimum_io_sizeclearly is the minimum I/O size, not the minimal nice to have one.I may have misread the below documentation then, because it seems to suggest this is a performance parameter, not a real minimum. Do we need to update it?
queue_limits.min_io is corretly described and a performance hint.
OK, great!
Quoted 8 lines
Quoted 2 lines
The statx
dio_offset_alignis actual minimum I/O size and alignment and not in any way related to the performance hint inminimum_io_size.Oh, darn, I just read again
825cf206ed510("statx: add direct I/O alignment information") and the block layer change through commit2d985f8c6b91b("vfs: supportSTATX_DIOALIGNon block devices") and no where do I see any mention of it being a min. Should we clarify that?
dio_offset_align is an _alignment_ parameter, not a "size" parameter. In no way does it define either the absolute or "best performance" minimum IO size - it just defines the minimum valid alignment for the file offset that the filesystem/device supports.
It is implied that an IO of dio_offset_align bytes in length will be supported, because that is the minimum length IO that meets the offset alignment requirements defined by dio_offset_align.
Quoted 1 line
And should we add a respective value for performance?
We could, but we already have:
__u32 stx_blksize; /* Block size for filesystem I/O */
which is defined as:
stx_blksize
The "preferred" block size for efficient filesystem I/O. (Writing to a file in smaller chunks may cause an inefficient read-modify-rewrite.)
Quoted 3 lines
I suspect
userspace will want to work with optimal values, not ones which could for instance incur read-modify-write.
Yup, that's the very definition of what stx_blksize should contain.
-Dave.
Signature · 2 lines
Dave Chinner [email protected]