qemu-img convert: Fix abort with unaligned image size
PATCH v2 1/2 block: Require aligned image size to avoid assertion failure
In reply to Kevin WolfUnaligned requests will automatically be aligned to bl.request_alignment and we can't extend write requests to access space beyond the end of the image without resizing the image, so if we have the WRITE permission, but not the RESIZE one, it's required that the image size is aligned.
Failing to meet this requirement could cause assertion failures like this if RESIZE permissions weren't requested:
qemu-img: block/io.c:1910: bdrv_co_write_req_prepare: Assertion `end_sector <= bs->total_sectors || child->perm & BLK_PERM_RESIZE' failed.
This was e.g. triggered by qemu-img converting to a target image with 4k request alignment when the image was only aligned to 512 bytes, but not to 4k.
Turn this into a graceful error in bdrv_check_perm() so that WRITE without RESIZE can only be taken if the image size is aligned. If a user holds both permissions and drops only RESIZE, the function will return an error, but bdrv_child_try_set_perm() will ignore the failure silently if permissions are only requested to be relaxed and just keep both permissions while returning success.
--- block.c | 16 ++++++++++++++++ 1 file changed, 16 insertions(+)
block.c +16 -0
@@ -2025,6 +2025,22 @@ static int bdrv_check_perm(BlockDriverState *bs, BlockReopenQueue *q,return -EPERM ; } +/* +* Unaligned requests will automatically be aligned to bl.request_alignment +* and without RESIZE we can't extend requests to write to space beyond the +* end of the image, so it's required that the image size is aligned. +*/ +if ((cumulative_perms &BLK_PERM_WRITE ) && + !(cumulative_perms &BLK_PERM_RESIZE )) + { +if ((bs->total_sectors *BDRV_SECTOR_SIZE ) % bs->bl.request_alignment) { +error_setg (errp,"Cannot get 'write' permission without 'resize': " +"Image size is not a multiple of request " +"alignment" ); +return -EPERM ; + } + } +/* Check this node */ if (!drv) {return 0 ;
Signature · 1 line
2.25.4
PATCH v2 2/2 file-posix: Allow byte-aligned O_DIRECT with NFS
In reply to Kevin WolfSince commit a6b257a08e3 ('file-posix: Handle undetectable alignment'), we assume that if we open a file with O_DIRECT and alignment probing returns 1, we just couldn't find out the real alignment requirement because some filesystems make the requirement only for allocated blocks. In this case, a safe default of 4k is used.
This is too strict for NFS, which does actually allow byte-aligned requests even with O_DIRECT. Because we can't distinguish both cases with generic code, let's just look at the file system magic and disable s->needs_alignment for NFS. This way, O_DIRECT can still be used on NFS for images that are not aligned to 4k.
--- block/file-posix.c | 26 +++++++++++++++++++++++++- 1 file changed, 25 insertions(+), 1 deletion(-)
block/file-posix.c +25 -1
@@ -62,10 +62,12 @@#include <sys/ioctl.h> #include <sys/param.h> #include <sys/syscall.h> +#include <sys/vfs.h> #include <linux/cdrom.h> #include <linux/fd.h> #include <linux/fs.h> #include <linux/hdreg.h> +#include <linux/magic.h> #include <scsi/sg.h> #ifdef __s390__#include <asm/dasd.h>
@@ -300,6 +302,28 @@ static int probe_physical_blocksize(int fd, unsigned int *blk_size)#endif } +/* +* Returns true if no alignment restrictions are necessary even for files +* opened with O_DIRECT. +* +* raw_probe_alignment() probes the required alignment and assume that 1 means +* the probing failed, so it falls back to a safe default of 4k. This can be +* avoided if we know that byte alignment is okay for the file. +*/ +static bool dio_byte_aligned (int fd) +{ +#ifdef __linux__ +struct statfs buf; +int ret; + + ret =fstatfs (fd, &buf); +if (ret ==0 && buf.f_type ==NFS_SUPER_MAGIC ) { +return true; + } +#endif +return false; +} +/* Check if read is allowed with given memory buffer and length. * * This function is used to check O_DIRECT memory buffer and request alignment.
@@ -629,7 +653,7 @@ static int raw_open_common(BlockDriverState *bs, QDict *options, s->has_discard = true; s->has_write_zeroes = true; -if ((bs->open_flags &BDRV_O_NOCACHE ) !=0 ) { +if ((bs->open_flags &BDRV_O_NOCACHE ) !=0 && !dio_byte_aligned (s->fd)) { s->needs_alignment = true; }
Signature · 1 line
2.25.4
v2 cover qemu-img convert: Fix abort with unaligned image size
Patch 1 fixes the assertion failure by failing gracefully when opening an image whose size isn't aligned to the required request alignment.
Patch 2 relaxes the restrictions for NFS, which actually supports byte alignment, but incorrectly gets a 4k request alignment in the file-posix block driver.
v2:
- Don't fail opening unaligned images, but requesting WRITE permission without RESIZE. This keeps qcow2 images with unaligned metadata at EOF working. [Max]
Kevin Wolf (2):
block: Require aligned image size to avoid assertion failure
file-posix: Allow byte-aligned O_DIRECT with NFS
block.c | 16 ++++++++++++++++ block/file-posix.c | 26 +++++++++++++++++++++++++- 2 files changed, 41 insertions(+), 1 deletion(-)
Signature · 1 line
2.25.4
Re: [PATCH for-5.1 v2 1/2] block: Require aligned image size to avoid assertion failure
In reply to Kevin WolfQuoted 31 lines · Kevin Wolf
Unaligned requests will automatically be aligned to bl.request_alignment and we can't extend write requests to access space beyond the end of the image without resizing the image, so if we have the WRITE permission, but not the RESIZE one, it's required that the image size is aligned.
Failing to meet this requirement could cause assertion failures like this if RESIZE permissions weren't requested:
qemu-img:
block/io.c:1910:bdrv_co_write_req_prepare: Assertion `end_sector <=bs->total_sectors||child->perm&BLK_PERM_RESIZE' failed.This was e.g. triggered by qemu-img converting to a target image with 4k request alignment when the image was only aligned to 512 bytes, but not to 4k.
Turn this into a graceful error in
bdrv_check_perm()so that WRITE without RESIZE can only be taken if the image size is aligned. If a user holds both permissions and drops only RESIZE, the function will return an error, butbdrv_child_try_set_perm()will ignore the failure silently if permissions are only requested to be relaxed and just keep both permissions while returning success.Signed-off-by Kevin Wolf <[email protected]>--- block.c | 16 ++++++++++++++++ 1 file changed, 16 insertions(+)block.c +0
-0
block.c static int bdrv_check_perm(BlockDriverState *bs, BlockReopenQueue *q, · line 2033
Quoted 9 lines
block.c +6
-0@@ -2025,6 +2025,22 @@ static int bdrv_check_perm(BlockDriverState *bs, BlockReopenQueue *q,return -EPERM ; } +/* +* Unaligned requests will automatically be aligned to bl.request_alignment +* and without RESIZE we can't extend requests to write to space beyond the +* end of the image, so it's required that the image size is aligned. +*/ + if ((cumulative_perms & BLK_PERM_WRITE) &&
What about WRITE_UNCHANGED? I think this would only matter with nodes that can have backing files (i.e., qcow2 in practice) because WRITE_UNCHANGED is only used by COR and block jobs doing something with a backing chain, so it shouldn’t matter in practice, but, well.
So, either way:
Quoted 14 lines
block.c +10
-0+ !(cumulative_perms & BLK_PERM_RESIZE)) + { +if ((bs->total_sectors *BDRV_SECTOR_SIZE ) % bs->bl.request_alignment) { +error_setg (errp,"Cannot get 'write' permission without 'resize': " +"Image size is not a multiple of request " +"alignment" ); +return -EPERM ; + } + } +/* Check this node */ if (!drv) {return 0 ;
Re: [PATCH for-5.1 v2 2/2] file-posix: Allow byte-aligned O_DIRECT with NFS
In reply to Kevin WolfQuoted 12 lines · Kevin Wolf
Since commit
a6b257a08e3('file-posix: Handle undetectable alignment'), we assume that if we open a file withO_DIRECTand alignment probing returns 1, we just couldn't find out the real alignment requirement because some filesystems make the requirement only for allocated blocks. In this case, a safe default of 4k is used.This is too strict for NFS, which does actually allow byte-aligned requests even with
O_DIRECT. Because we can't distinguish both cases with generic code, let's just look at the file system magic and disables->needs_alignmentfor NFS. This way,O_DIRECTcan still be used on NFS for images that are not aligned to 4k.
Quoted 5 lines
Signed-off-by Kevin Wolf <[email protected]>Reviewed-by Eric Blake <[email protected]>--- block/file-posix.c | 26 +++++++++++++++++++++++++- 1 file changed, 25 insertions(+), 1 deletion(-)
Re: [PATCH for-5.1 v2 1/2] block: Require aligned image size to avoid assertion failure
In reply to Max ReitzAm 17.07.2020 um 13:02 hat Max Reitz geschrieben:
Quoted 35 lines · Kevin Wolf
Quoted 34 lines
Unaligned requests will automatically be aligned to bl.request_alignment and we can't extend write requests to access space beyond the end of the image without resizing the image, so if we have the WRITE permission, but not the RESIZE one, it's required that the image size is aligned.
Failing to meet this requirement could cause assertion failures like this if RESIZE permissions weren't requested:
qemu-img:
block/io.c:1910:bdrv_co_write_req_prepare: Assertion `end_sector <=bs->total_sectors||child->perm&BLK_PERM_RESIZE' failed.This was e.g. triggered by qemu-img converting to a target image with 4k request alignment when the image was only aligned to 512 bytes, but not to 4k.
Turn this into a graceful error in
bdrv_check_perm()so that WRITE without RESIZE can only be taken if the image size is aligned. If a user holds both permissions and drops only RESIZE, the function will return an error, butbdrv_child_try_set_perm()will ignore the failure silently if permissions are only requested to be relaxed and just keep both permissions while returning success.Signed-off-by Kevin Wolf <[email protected]>--- block.c | 16 ++++++++++++++++ 1 file changed, 16 insertions(+)block.c +0
-0@@ -2025,6 +2025,22 @@ static int bdrv_check_perm(BlockDriverState *bs, BlockReopenQueue *q,return -EPERM ; }
Quoted 11 lines
block.c static int bdrv_check_perm(BlockDriverState *bs, BlockReopenQueue *q, · line 2033
Quoted 6 lines
block.c +6
-0+/* +* Unaligned requests will automatically be aligned to bl.request_alignment +* and without RESIZE we can't extend requests to write to space beyond the +* end of the image, so it's required that the image size is aligned. +*/ + if ((cumulative_perms & BLK_PERM_WRITE) &&What about
WRITE_UNCHANGED? I think this would only matter with nodes that can have backing files (i.e., qcow2 in practice) becauseWRITE_UNCHANGEDis only used by COR and block jobs doing something with a backing chain, so it shouldn’t matter in practice, but, well.
So basically just replacing the line with this?
if ((cumulative_perms & (BLK_PERM_WRITE | BDRV_PERM_WRITE_UNCHANGED)) &&
I can do that while applying if it is what you mean.
Quoted 3 lines
So, either way:
Reviewed-by Max Reitz <[email protected]>
Thanks!
Kevin
Quoted 16 lines
Quoted 14 lines
block.c +10
-0+ !(cumulative_perms & BLK_PERM_RESIZE)) + { +if ((bs->total_sectors *BDRV_SECTOR_SIZE ) % bs->bl.request_alignment) { +error_setg (errp,"Cannot get 'write' permission without 'resize': " +"Image size is not a multiple of request " +"alignment" ); +return -EPERM ; + } + } +/* Check this node */ if (!drv) {return 0 ;
Re: [PATCH for-5.1 v2 1/2] block: Require aligned image size to avoid assertion failure
Quoted 48 lines · Kevin Wolf
Am 17.07.2020 um 13:02 hat Max Reitz geschrieben:
Quoted 35 lines · Kevin Wolf
Quoted 34 lines
Unaligned requests will automatically be aligned to bl.request_alignment and we can't extend write requests to access space beyond the end of the image without resizing the image, so if we have the WRITE permission, but not the RESIZE one, it's required that the image size is aligned.
Failing to meet this requirement could cause assertion failures like this if RESIZE permissions weren't requested:
qemu-img:
block/io.c:1910:bdrv_co_write_req_prepare: Assertion `end_sector <=bs->total_sectors||child->perm&BLK_PERM_RESIZE' failed.This was e.g. triggered by qemu-img converting to a target image with 4k request alignment when the image was only aligned to 512 bytes, but not to 4k.
Turn this into a graceful error in
bdrv_check_perm()so that WRITE without RESIZE can only be taken if the image size is aligned. If a user holds both permissions and drops only RESIZE, the function will return an error, butbdrv_child_try_set_perm()will ignore the failure silently if permissions are only requested to be relaxed and just keep both permissions while returning success.Signed-off-by Kevin Wolf <[email protected]>--- block.c | 16 ++++++++++++++++ 1 file changed, 16 insertions(+)block.c +0
-0@@ -2025,6 +2025,22 @@ static int bdrv_check_perm(BlockDriverState *bs, BlockReopenQueue *q,return -EPERM ; }Quoted 11 lines
block.c static int bdrv_check_perm(BlockDriverState *bs, BlockReopenQueue *q, · line 2033
Quoted 6 lines
block.c +6
-0+/* +* Unaligned requests will automatically be aligned to bl.request_alignment +* and without RESIZE we can't extend requests to write to space beyond the +* end of the image, so it's required that the image size is aligned. +*/ + if ((cumulative_perms & BLK_PERM_WRITE) &&What about
WRITE_UNCHANGED? I think this would only matter with nodes that can have backing files (i.e., qcow2 in practice) becauseWRITE_UNCHANGEDis only used by COR and block jobs doing something with a backing chain, so it shouldn’t matter in practice, but, well.
Quoted 5 lines
So basically just replacing the line with this?
if ((cumulative_perms & (BLK_PERM_WRITE | BDRV_PERM_WRITE_UNCHANGED)) &&I can do that while applying if it is what you mean.
Yes. :)
Quoted 27 lines
Quoted 3 lines
So, either way:
Reviewed-by Max Reitz <[email protected]>Thanks!
Kevin
Quoted 16 lines
Quoted 14 lines
block.c +10
-0+ !(cumulative_perms & BLK_PERM_RESIZE)) + { +if ((bs->total_sectors *BDRV_SECTOR_SIZE ) % bs->bl.request_alignment) { +error_setg (errp,"Cannot get 'write' permission without 'resize': " +"Image size is not a multiple of request " +"alignment" ); +return -EPERM ; + } + } +/* Check this node */ if (!drv) {return 0 ;