file-posix: alignment probing improvements
v2 cover file-posix: alignment probing improvements
v2:
- Make sure that XFS_IOC_DIOINFO takes priority over logical blocksize [Eric Biggers]
- Included a link to Eric's linux-block regression email
These patches fix alignment probing with dm-crypt and add support for the new Linux statx(STATX_DIOALIGN) interface.
Given that Linux v6.0 kernels with dm-crypt returning EIO are out there, I think we might as well rejig the alignment probing loop to handle that.
Let's also add support for the new STATX_DIOALIGN interface so probing is not required on new kernels.
Stefan Hajnoczi (2):
file-posix: fix Linux alignment probing when EIO is returned
file-posix: add statx(STATX_DIOALIGN) support
block/file-posix.c | 102 ++++++++++++++++++++++++--------------------- 1 file changed, 55 insertions(+), 47 deletions(-)
Signature · 1 line
2.38.1
PATCH v2 1/2 file-posix: fix Linux alignment probing when EIO is returned
Linux v6.0 dm-crypt returns errno EIO from unaligned O_DIRECT pread(2) calls. Alignment probing fails on dm-crypt devices because the code expects EINVAL. This is a kernel regression that is expected to be fixed upstream:
https://lore.kernel.org/linux-block/[email protected]/T/#t
Treating any errno as an "unaligned" indicator would be easy, but breaks commit 22d182e82b4b ("block/raw-posix: fix launching with failed disks"). Offline disks return EIO for correctly aligned requests and EINVAL for unaligned requests.
It's possible to make both v6.0 dm-crypt and offline disks work: look for the transition from EINVAL to EIO instead of for a single EINVAL value.
--- block/file-posix.c | 42 +++++++++++++++--------------------------- 1 file changed, 15 insertions(+), 27 deletions(-)
block/file-posix.c +15 -27
@@ -355,31 +355,6 @@ static bool raw_needs_alignment(BlockDriverState *bs)return s->force_alignment; } -/* Check if read is allowed with given memory buffer and length. -* -* This function is used to check O_DIRECT memory buffer and request alignment. -*/ -static bool raw_is_io_aligned (int fd,void *buf,size_t len) -{ -ssize_t ret =pread (fd, buf, len,0 ); - -if (ret >=0 ) { -return true; - } - -#ifdef __linux__ -/* The Linux kernel returns EINVAL for misaligned O_DIRECT reads. Ignore -* other errors (e.g. real I/O error), which could happen on a failed -* drive, since we only care about probing alignment. -*/ -if (errno !=EINVAL ) { -return true; - } -#endif - -return false; -} -static void raw_probe_alignment (BlockDriverState *bs,int fd,Error **errp) {BDRVRawState *s = bs->opaque;
@@ -426,34 +401,47 @@ static void raw_probe_alignment(BlockDriverState *bs, int fd, Error **errp)* try to detect buf_align, which cannot be detected in some cases (e.g. * Gluster). If buf_align cannot be detected, we fallback to the value of * request_alignment. +* +* The probing loop keeps track of the last errno so that the alignment of +* offline disks can be probed. On Linux pread(2) returns with errno EINVAL +* for most file descriptors when O_DIRECT alignment constraints are unmet. +* Offline disks fail correctly aligned pread(2) with EIO. Therefore it's +* possible to detect alignment on offline disks by observing when the +* errno changes from EINVAL to something else. */ if (!bs->bl.request_alignment) { +int last_errno =0 ;int i;size_t align; buf =qemu_memalign (max_align, max_align);for (i =0 ; i <ARRAY_SIZE (alignments); i++) { align = alignments[i]; -if (raw_is_io_aligned (fd, buf, align)) { +if (pread (fd, buf, align,0 ) >=0 || + (errno !=EINVAL && last_errno ==EINVAL )) {/* Fallback to safe value. */ bs->bl.request_alignment = (align !=1 ) ? align : max_align;break ; } + last_errno = errno; }qemu_vfree (buf); }if (!s->buf_align) { +int last_errno =0 ;int i;size_t align; buf =qemu_memalign (max_align,2 * max_align);for (i =0 ; i <ARRAY_SIZE (alignments); i++) { align = alignments[i]; -if (raw_is_io_aligned (fd, buf + align, max_align)) { +if (pread (fd, buf + align, max_align,0 ) >=0 || + (errno !=EINVAL && last_errno ==EINVAL )) {/* Fallback to request_alignment. */ s->buf_align = (align !=1 ) ? align : bs->bl.request_alignment;break ; } + last_errno = errno; }qemu_vfree (buf); }
Signature · 1 line
2.38.1
Re: [PATCH v2 1/2] file-posix: fix Linux alignment probing when EIO is returned
Am 03.11.2022 um 19:36 hat Stefan Hajnoczi geschrieben:
Quoted 11 lines
Linux v6.0 dm-crypt returns errno EIO from unaligned
O_DIRECTpread(2) calls. Alignment probing fails on dm-crypt devices because the code expects EINVAL. This is a kernel regression that is expected to be fixed upstream:
https://lore.kernel.org/linux-block/[email protected]/T/#tTreating any errno as an "unaligned" indicator would be easy, but breaks commit
22d182e82b4b("block/raw-posix: fix launching with failed disks"). Offline disks return EIO for correctly aligned requests and EINVAL for unaligned requests.
Quoted 7 lines
It's possible to make both v6.0 dm-crypt and offline disks work: look for the transition from EINVAL to EIO instead of for a single EINVAL value.
Fixes 22d182e82b4b ("block/raw-posix: fix launching with failed disks")Signed-off-by Stefan Hajnoczi <[email protected]>
I haven't tested it myself, but from the comments in the bug tracker and on the mailing this, I don't think this can actually distinguish the two
cases: AIUI, even with the buggy dm-crypt, you get a transition from EINVAL to EIO because everything < 512 (such as our first probed value
1) will still return EINVAL.
So even with this patch, I think we would still probe the incorrect 512 bytes when the dm-crypt bug is present.
Kevin
PATCH v2 2/2 file-posix: add statx(STATX_DIOALIGN) support
In reply to Stefan HajnocziLinux v6.1 commit 825cf206ed51 ("statx: add direct I/O alignment information") added an interface to fetch O_DIRECT alignment values for block devices and file systems.
Prefer STATX_DIOALIGN to older interfaces and probing, but keep them as fallbacks in case STATX_DIOALIGN cannot provide the information.
Testing shows the status of STATX_DIOALIGN support in Linux 6.1-rc3 appears to be:
- btrfs: no
- ext4: yes
- XFS: yes
- NVMe block devices: yes
- dm-crypt: yes
---
v2:
- Make sure that XFS_IOC_DIOINFO takes priority over logical blocksize [Eric Biggers]
--- block/file-posix.c | 60 ++++++++++++++++++++++++++++++---------------- 1 file changed, 40 insertions(+), 20 deletions(-)
block/file-posix.c +40 -20
@@ -372,29 +372,49 @@ static void raw_probe_alignment(BlockDriverState *bs, int fd, Error **errp) bs->bl.request_alignment =0 ; s->buf_align =0 ; + +#if defined(__linux__) && defined(STATX_DIOALIGN ) +struct statx stx; + +/* +* Linux 6.1 introduced an interface for both block devices and file +* systems. The system call returns with the STATX_DIOALIGN bit cleared +* when the information is unavailable. +*/ +if (statx (fd,"" ,AT_EMPTY_PATH ,STATX_DIOALIGN , &stx) ==0 && + (stx.stx_mask &STATX_DIOALIGN )) { + bs->bl.request_alignment = stx.stx_dio_offset_align; + s->buf_align = stx.stx_dio_mem_align; + } +#endif /* defined(__linux__) && defined(STATX_DIOALIGN) */ + +#ifdef __linux__ +if (!bs->bl.request_alignment) { +/* +* The XFS ioctl definitions are shipped in extra packages that might +* not always be available. Since we just need the XFS_IOC_DIOINFO ioctl +* here, we simply use our own definition instead: +*/ +struct xfs_dioattr { +uint32_t d_mem; +uint32_t d_miniosz; +uint32_t d_maxiosz; + } da; + if (ioctl(fd, _IOR('X', 30, struct xfs_dioattr), &da) >= 0) { + bs->bl.request_alignment = da.d_miniosz; +/* The kernel returns wrong information for d_mem */ +/* s->buf_align = da.d_mem; */ + } + } +#endif /* __linux__ */ +/* Let's try to use the logical blocksize for the alignment. */ -if (probe_logical_blocksize (fd, &bs->bl.request_alignment) <0 ) { - bs->bl.request_alignment =0 ; +if (!bs->bl.request_alignment) { +if (probe_logical_blocksize (fd, &bs->bl.request_alignment) <0 ) { + bs->bl.request_alignment =0 ; + } } -#ifdef __linux__ -/* -* The XFS ioctl definitions are shipped in extra packages that might -* not always be available. Since we just need the XFS_IOC_DIOINFO ioctl -* here, we simply use our own definition instead: -*/ -struct xfs_dioattr { -uint32_t d_mem; -uint32_t d_miniosz; -uint32_t d_maxiosz; - } da; - if (ioctl(fd, _IOR('X', 30, struct xfs_dioattr), &da) >= 0) { - bs->bl.request_alignment = da.d_miniosz; -/* The kernel returns wrong information for d_mem */ -/* s->buf_align = da.d_mem; */ - } -#endif -/* * If we could not get the sizes so far, we can only guess them. First try * to detect request alignment, since it is more likely to succeed. Then
Signature · 1 line
2.38.1
Re: [PATCH v2 2/2] file-posix: add statx(STATX_DIOALIGN) support
Am 03.11.2022 um 19:36 hat Stefan Hajnoczi geschrieben:
Quoted 15 lines
Linux v6.1 commit
825cf206ed51("statx: add direct I/O alignment information") added an interface to fetchO_DIRECTalignment values for block devices and file systems.Prefer
STATX_DIOALIGNto older interfaces and probing, but keep them as fallbacks in caseSTATX_DIOALIGNcannot provide the information.Testing shows the status of
STATX_DIOALIGNsupport in Linux 6.1-rc3 appears to be:
- btrfs: no
- ext4: yes
- XFS: yes
- NVMe block devices: yes
- dm-crypt: yes
Quoted 2 lines
Cc Eric Biggers <[email protected]>Signed-off-by Stefan Hajnoczi <[email protected]>
I think it's worth noting in the commit message that this essentially disables patch 1 again on Linux 6.1 because it doesn't even use the code any more that patch 1 modified to work around the dm-crypt bug.
This is only okay because we think that the final 6.1 release is going to have the bug fixed, and it's also the first version to support STATX_DIOALIGN, so you won't have both STATX_DIOALIGN and the bug in a stable kernel.
Kevin