Block layer patches
The following changes since commit 8796c64ecdfd34be394ea277aaaaa53df0c76996:
Merge remote-tracking branch 'remotes/kraxel/tags/audio-20200706-pull-request' into staging (2020-07-08 16:33:59 +0100)
are available in the Git repository at:
git://repo.or.cz/qemu/kevin.git tags/for-upstream
for you to fetch changes up to d1c824e681e423a6224c6831a493e9175fa02dc1:
qemu-img: Deprecate use of -b without -F (2020-07-09 14:14:55 +0200)
----------------------------------------------------------------
Block layer patches:
- file-posix: Mitigate file fragmentation with extent size hints
- Tighten qemu-img rules on missing backing format
- qemu-img map: Don't limit block status request size
----------------------------------------------------------------
Eric Blake (10):
qemu-img: Flush stdout before before potential stderr messages
block: Finish deprecation of 'qemu-img convert -n -o'
sheepdog: Add trivial backing_fmt support
vmdk: Add trivial backing_fmt support
qcow: Tolerate backing_fmt=
block: Error if backing file fails during creation without -u
qcow2: Deprecate use of qemu-img amend to change backing file
iotests: Specify explicit backing format where sensible
block: Add support to warn on backing file change without format
qemu-img: Deprecate use of -b without -F
Kevin Wolf (2):
qemu-img map: Don't limit block status request size
file-posix: Mitigate file fragmentation with extent size hints
Max Reitz (1):
iotests: Simplify _filter_img_create() a bit
qapi/block-core.json | 11 +++-- docs/system/deprecated.rst | 58 ++++++++++++++++++++++---- docs/tools/qemu-img.rst | 4 ++ include/block/block.h | 4 +- include/block/block_int.h | 1 + block.c | 53 +++++++++++++++++------- block/file-posix.c | 42 +++++++++++++++++++ block/qcow.c | 20 ++++++++- block/qcow2.c | 7 +++- block/sheepdog.c | 18 +++++++- block/stream.c | 2 +- block/vmdk.c | 14 +++++++ blockdev.c | 3 +- qemu-img.c | 20 +++++---- tests/qemu-iotests/017 | 2 +- tests/qemu-iotests/017.out | 2 +- tests/qemu-iotests/018 | 2 +- tests/qemu-iotests/018.out | 2 +- tests/qemu-iotests/019 | 5 ++- tests/qemu-iotests/019.out | 2 +- tests/qemu-iotests/020 | 31 ++++++++------ tests/qemu-iotests/020.out | 15 +------ tests/qemu-iotests/024 | 8 ++-- tests/qemu-iotests/024.out | 5 ++- tests/qemu-iotests/028 | 4 +- tests/qemu-iotests/028.out | 2 +- tests/qemu-iotests/030 | 26 ++++++++---- tests/qemu-iotests/034 | 2 +- tests/qemu-iotests/034.out | 2 +- tests/qemu-iotests/037 | 2 +- tests/qemu-iotests/037.out | 2 +- tests/qemu-iotests/038 | 2 +- tests/qemu-iotests/038.out | 2 +- tests/qemu-iotests/039 | 3 +- tests/qemu-iotests/039.out | 2 +- tests/qemu-iotests/040 | 47 +++++++++++++++------ tests/qemu-iotests/041 | 37 +++++++++++------ tests/qemu-iotests/042 | 4 +- tests/qemu-iotests/043 | 18 ++++---- tests/qemu-iotests/043.out | 16 +++++--- tests/qemu-iotests/046 | 2 +- tests/qemu-iotests/046.out | 2 +- tests/qemu-iotests/049.out | 8 ++-- tests/qemu-iotests/050 | 4 +- tests/qemu-iotests/050.out | 2 +- tests/qemu-iotests/051 | 2 +- tests/qemu-iotests/051.out | 2 +- tests/qemu-iotests/051.pc.out | 2 +- tests/qemu-iotests/054.out | 2 +- tests/qemu-iotests/056 | 3 +- tests/qemu-iotests/060 | 2 +- tests/qemu-iotests/060.out | 2 +- tests/qemu-iotests/061 | 10 ++--- tests/qemu-iotests/061.out | 11 ++--- tests/qemu-iotests/069 | 2 +- tests/qemu-iotests/069.out | 2 +- tests/qemu-iotests/073 | 2 +- tests/qemu-iotests/073.out | 2 +- tests/qemu-iotests/079.out | 2 +- tests/qemu-iotests/082 | 10 +++-- tests/qemu-iotests/082.out | 30 +++++++++++--- tests/qemu-iotests/085 | 4 +- tests/qemu-iotests/085.out | 6 +-- tests/qemu-iotests/089 | 2 +- tests/qemu-iotests/089.out | 2 +- tests/qemu-iotests/095 | 4 +- tests/qemu-iotests/095.out | 4 +- tests/qemu-iotests/097 | 4 +- tests/qemu-iotests/097.out | 16 ++++---- tests/qemu-iotests/098 | 2 +- tests/qemu-iotests/098.out | 8 ++-- tests/qemu-iotests/110 | 4 +- tests/qemu-iotests/110.out | 4 +- tests/qemu-iotests/111.out | 2 +- tests/qemu-iotests/112.out | 4 +- tests/qemu-iotests/114 | 12 ++++++ tests/qemu-iotests/114.out | 9 ++++ tests/qemu-iotests/122 | 34 +++++++++++----- tests/qemu-iotests/122.out | 12 ++++-- tests/qemu-iotests/126 | 4 +- tests/qemu-iotests/126.out | 4 +- tests/qemu-iotests/127 | 4 +- tests/qemu-iotests/127.out | 4 +- tests/qemu-iotests/129 | 3 +- tests/qemu-iotests/133 | 2 +- tests/qemu-iotests/133.out | 2 +- tests/qemu-iotests/139 | 2 +- tests/qemu-iotests/141 | 6 +-- tests/qemu-iotests/141.out | 4 +- tests/qemu-iotests/142 | 2 +- tests/qemu-iotests/142.out | 2 +- tests/qemu-iotests/153 | 14 +++---- tests/qemu-iotests/153.out | 35 ++++++++-------- tests/qemu-iotests/154 | 42 +++++++++---------- tests/qemu-iotests/154.out | 42 +++++++++---------- tests/qemu-iotests/155 | 12 ++++-- tests/qemu-iotests/156 | 9 ++-- tests/qemu-iotests/156.out | 6 +-- tests/qemu-iotests/158 | 2 +- tests/qemu-iotests/158.out | 2 +- tests/qemu-iotests/161 | 8 ++-- tests/qemu-iotests/161.out | 8 ++-- tests/qemu-iotests/176 | 4 +- tests/qemu-iotests/176.out | 32 +++++++-------- tests/qemu-iotests/177 | 2 +- tests/qemu-iotests/177.out | 2 +- tests/qemu-iotests/179 | 2 +- tests/qemu-iotests/179.out | 2 +- tests/qemu-iotests/189 | 2 +- tests/qemu-iotests/189.out | 2 +- tests/qemu-iotests/191 | 12 +++--- tests/qemu-iotests/191.out | 12 +++--- tests/qemu-iotests/195 | 6 +-- tests/qemu-iotests/195.out | 6 +-- tests/qemu-iotests/198 | 2 +- tests/qemu-iotests/198.out | 3 +- tests/qemu-iotests/204 | 2 +- tests/qemu-iotests/204.out | 2 +- tests/qemu-iotests/216 | 2 +- tests/qemu-iotests/224 | 4 +- tests/qemu-iotests/225 | 2 +- tests/qemu-iotests/225.out | 2 +- tests/qemu-iotests/228 | 5 ++- tests/qemu-iotests/243 | 7 ++-- tests/qemu-iotests/245 | 3 +- tests/qemu-iotests/249 | 4 +- tests/qemu-iotests/249.out | 4 +- tests/qemu-iotests/252 | 2 +- tests/qemu-iotests/257 | 3 +- tests/qemu-iotests/259.out | 2 +- tests/qemu-iotests/267 | 4 +- tests/qemu-iotests/267.out | 6 +-- tests/qemu-iotests/270 | 2 +- tests/qemu-iotests/270.out | 2 +- tests/qemu-iotests/273 | 4 +- tests/qemu-iotests/273.out | 4 +- tests/qemu-iotests/274 | 12 +++--- tests/qemu-iotests/274.out | 29 +++++++------ tests/qemu-iotests/279 | 4 +- tests/qemu-iotests/279.out | 4 +- tests/qemu-iotests/290 | 2 +- tests/qemu-iotests/290.out | 4 +- tests/qemu-iotests/301 | 88 ++++++++++++++++++++++++++++++++++++++++ tests/qemu-iotests/301.out | 59 +++++++++++++++++++++++++++ tests/qemu-iotests/common.filter | 62 ++++++++++++---------------- tests/qemu-iotests/group | 1 + 146 files changed, 906 insertions(+), 472 deletions(-) create mode 100755 tests/qemu-iotests/301 create mode 100644 tests/qemu-iotests/301.out
Quoted 11 lines · Kevin Wolf
The following changes since commit
8796c64ecdfd34be394ea277aaaaa53df0c76996:Merge remote-tracking branch 'remotes/kraxel/tags/audio-20200706-pull-request' into staging (2020-07-08 16:33:59 +0100)
are available in the Git repository at:
git://repo.or.cz/qemu/kevin.gittags/for-upstreamfor you to fetch changes up to
d1c824e681e423a6224c6831a493e9175fa02dc1:
Quoted 9 lines
qemu-img: Deprecate use of -b without -F (2020-07-09 14:14:55 +0200)
----------------------------------------------------------------
Block layer patches:- file-posix: Mitigate file fragmentation with extent size hints
- Tighten qemu-img rules on missing backing format
- qemu-img map: Don't limit block status request size
iotest 114 fails on FreeBSD and OpenBSD (and probably NetBSD but that build hasn't reported back yet): looks like a non-portable use of 'truncate' ?
TEST iotest-qcow2: 114 [fail]
QEMU --
"/home/qemu/qemu-test.FOdDM2/build/tests/qemu-iotests/../../aarch64-softmmu/qemu-system-aarch64" -nodefaults -disp
lay none -machine virt -accel qtestQEMU_IMG --
"/home/qemu/qemu-test.FOdDM2/build/tests/qemu-iotests/../../qemu-img"QEMU_IO --
"/home/qemu/qemu-test.FOdDM2/build/tests/qemu-iotests/../../qemu-io"
--cache writeback --aio threads -f qcow2QEMU_NBD --
"/home/qemu/qemu-test.FOdDM2/build/tests/qemu-iotests/../../qemu-nbd"
IMGFMT -- qcow2 (compat=1.1) IMGPROTO -- file PLATFORM -- FreeBSD/amd64 freebsd 12.1-RELEASE TEST_DIR -- /home/qemu/qemu-test.FOdDM2/build/tests/qemu-iotests/scratch SOCK_DIR -- /tmp/tmp.RQQ2LyuG
SOCKET_SCM_HELPER --
--- /home/qemu/qemu-test.FOdDM2/src/tests/qemu-iotests/114.out
2020-07-10 14:51:21.000000000 +0000 +++ /home/qemu/qemu-test.FOdDM2/build/tests/qemu-iotests/114.out.bad 2020-07-10 15:15:14.446278000 +0000
@@ -1,8 +1,11 @@
QA output created by 114
-qemu-img: warning: Deprecated use of backing file without explicit
backing format (detected format of raw)
-Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864 backing_file=TEST_DIR/t.IMGFMT.orig
+truncate: illegal option -- -
+usage: truncate [-c] -s [+|-|%|/]size[K|k|M|m|G|g|T|t] file ...
+ truncate [-c] -r rfile file ...
+qemu-img: TEST_DIR/t.IMGFMT: Could not open 'TEST_DIR/t.IMGFMT.orig':
No such file or directory
+Could not open backing image.
Formatting 'TEST_DIR/t.IMGFMT.base', fmt=IMGFMT size=67108864
-qemu-img: warning: Deprecated use of backing file without explicit
backing format
+qemu-img: Could not open '/home/qemu/qemu-test.FOdDM2/build/tests/qemu-iotests/scratch/t.qcow2.orig':
Could not open '/home/qemu/qemu-test.FOdDM2/build/tests/qemu-iotests/scratch/t.qcow2.orig':
No such file or directory
qemu-img: warning: Deprecated use of backing file without explicit
backing format (detected format of IMGFMT)
Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864
backing_file=TEST_DIR/t.IMGFMT.base backing_fmt=IMGFMT
qemu-img: warning: Deprecated use of unopened backing file without
explicit backing format, use of this image requires potentially unsafe format probing
thanks
-- PMM
Quoted 21 lines · Peter Maydell
Quoted 9 lines
qemu-img: Deprecate use of -b without -F (2020-07-09 14:14:55 +0200)----------------------------------------------------------------
Block layer patches:- file-posix: Mitigate file fragmentation with extent size hints
- Tighten qemu-img rules on missing backing format
- qemu-img map: Don't limit block status request sizeiotest 114 fails on FreeBSD and OpenBSD (and probably NetBSD but that build hasn't reported back yet): looks like a non-portable use of 'truncate' ?
(quoted diff) +0
-2@@ -1,8 +1,11 @@ QA output created by 114 -qemu-img: warning: Deprecated use of backing file without explicit backing format (detected format of raw) -Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864backing_file=TEST_DIR/t.IMGFMT.orig
Review
Quoted 4 lines
(quoted diff) +4
-0+truncate: illegal option -- - +usage: truncate [-c] -s [+|-|%|/]size[K|k|M|m|G|g|T|t] file ... + truncate [-c] -r rfile file ... +qemu-img: TEST_DIR/t.IMGFMT: Could not open 'TEST_DIR/t.IMGFMT.orig':
Yep, 'truncate' is not POSIX, but we've been lucky that most of our uses have been under '_supported_os Linux'; test 272 runs on BSD with 'truncate -s' instead of 'truncate --size'. The fix is obvious; I can post the followup to squash in if that will help.
Signature · 3 lines
Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3226 Virtualization: qemu.org | libvirt.org
Quoted 19 lines · Kevin Wolf
The following changes since commit
8796c64ecdfd34be394ea277aaaaa53df0c76996:Merge remote-tracking branch 'remotes/kraxel/tags/audio-20200706-pull-request' into staging (2020-07-08 16:33:59 +0100)are available in the Git repository at:
git://repo.or.cz/qemu/kevin.gittags/for-upstreamfor you to fetch changes up to
d1c824e681e423a6224c6831a493e9175fa02dc1:qemu-img: Deprecate use of -b without -F (2020-07-09 14:14:55 +0200)----------------------------------------------------------------
Block layer patches:- file-posix: Mitigate file fragmentation with extent size hints
- Tighten qemu-img rules on missing backing format
- qemu-img map: Don't limit block status request size
Quoted 7 lines
----------------------------------------------------------------
Eric Blake (10):
qemu-img: Flush stdout before before potential stderr messages
block: Finish deprecation of 'qemu-img convert -n -o'
sheepdog: Add trivialbacking_fmtsupport
vmdk: Add trivialbacking_fmtsupport
qcow: Tolerate backing_fmt=
I see you renumbered my test 293 to 301 for conflict resolution, but did not update the commit message to call out the updated number ;)
Since we may have to do a v3 because of the 'truncate -s' issue in 114, you could touch that up too.
Quoted 9 lines
tests/qemu-iotests/301 | 88 ++++++++++++++++++++++++++++++++++++++++ tests/qemu-iotests/301.out | 59 +++++++++++++++++++++++++++ tests/qemu-iotests/common.filter | 62 ++++++++++++---------------- tests/qemu-iotests/group | 1 + 146 files changed, 906 insertions(+), 472 deletions(-)create mode 100755 tests/qemu-iotests/301
create mode 100644 tests/qemu-iotests/301.out
Signature · 3 lines
Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3226 Virtualization: qemu.org | libvirt.org
PATCH 1/1 fixup! qemu-img: Deprecate use of -b without -F
In reply to Kevin WolfPort to BSD truncate.
--- tests/qemu-iotests/114 | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
tests/qemu-iotests/114 +1 -1
@@ -45,7 +45,7 @@ _unsupported_imgopts data_file # Intentionally specify backing file without backing format; demonstrate # the difference in warning messages when backing file could be probed. # Note that only a non-raw probe result will affect the resulting image. -truncate --size=64M "$TEST_IMG.orig" +truncate -s $((64 * 1024 * 1024)) "$TEST_IMG.orig" _make_test_img -b "$TEST_IMG.orig" 64M TEST_IMG="$TEST_IMG.base" _make_test_img 64M
Signature · 1 line
2.27.0
PATCH 1/1 fixup! qemu-img: Flush stdout before before potential stderr messages
In reply to Kevin WolfMax noticed that I didn't run iotests on -luks...
--- tests/qemu-iotests/282.out | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)
tests/qemu-iotests/282.out +3 -3
@@ -1,11 +1,11 @@ QA output created by 282 == Create non-UTF8 secret == == Throws an error because of invalid UTF-8 secret == -qemu-img: vol.img: Data from secret sec0 is not valid UTF-8 Formatting 'vol.img', fmt=luks size=4194304 key-secret=sec0 +qemu-img: vol.img: Data from secret sec0 is not valid UTF-8 == Image file should not exist after the error == == Create a stub image file and run qemu-img again == -qemu-img: vol.img: Data from secret sec0 is not valid UTF-8 Formatting 'vol.img', fmt=luks size=4194304 key-secret=sec0 +qemu-img: vol.img: Data from secret sec0 is not valid UTF-8 == Pre-existing image file should also be deleted after the error == - *** done +*** done
Signature · 1 line
2.27.0
PATCH 1/1 fixup! iotests: Specify explicit backing format where sensible
In reply to Kevin WolfMax spotted that I didn't test -qed...
--- tests/qemu-iotests/024 | 2 +- tests/qemu-iotests/024.out | 1 - tests/qemu-iotests/043 | 5 +++-- tests/qemu-iotests/043.out | 4 ---- 4 files changed, 4 insertions(+), 8 deletions(-)
tests/qemu-iotests/024 +1 -1
@@ -180,7 +180,7 @@ $QEMU_IMG rebase -f "$IMGFMT" -b "$BASE_NEW_OREL" -F $IMGFMT "$OVERLAY_WREL" popd >/dev/null # Verify the backing path is correct -TEST_IMG=$OVERLAY _img_info | grep '^backing file' +TEST_IMG=$OVERLAY _img_info | grep '^backing file:' echo
tests/qemu-iotests/024.out +0 -1
@@ -158,7 +158,6 @@ wrote 65536/65536 bytes at offset 196608 64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) backing file: t.IMGFMT.base_new (actual path: TEST_DIR/subdir/t.IMGFMT.base_new) -backing file format: IMGFMT read 65536/65536 bytes at offset 0 64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
tests/qemu-iotests/043 +3 -2
@@ -82,11 +82,12 @@ _make_test_img -b "$TEST_IMG.2.base" -F $IMGFMT $size echo echo "== finite chain of length 3 (human) ==" -_img_info --backing-chain +# Exclude backing format, since qed differs from qcow2 on what gets stored +_img_info --backing-chain | grep -v '^backing file format:' echo echo "== finite chain of length 3 (json) ==" -_img_info --backing-chain --output=json +_img_info --backing-chain --output=json | grep -v 'backing-filename-format' # success, all done echo "*** done"
tests/qemu-iotests/043.out +0 -4
@@ -25,14 +25,12 @@ file format: IMGFMT virtual size: 128 MiB (134217728 bytes) cluster_size: 65536 backing file: TEST_DIR/t.IMGFMT.2.base -backing file format: IMGFMT image: TEST_DIR/t.IMGFMT.2.base file format: IMGFMT virtual size: 128 MiB (134217728 bytes) cluster_size: 65536 backing file: TEST_DIR/t.IMGFMT.1.base -backing file format: IMGFMT image: TEST_DIR/t.IMGFMT.1.base file format: IMGFMT
@@ -42,7 +40,6 @@ cluster_size: 65536 == finite chain of length 3 (json) == [ { - "backing-filename-format": "IMGFMT", "virtual-size": 134217728, "filename": "TEST_DIR/t.IMGFMT", "cluster-size": 65536,
@@ -52,7 +49,6 @@ cluster_size: 65536 "dirty-flag": false }, { - "backing-filename-format": "IMGFMT", "virtual-size": 134217728, "filename": "TEST_DIR/t.IMGFMT.2.base", "cluster-size": 65536,
Signature · 1 line
2.27.0
Re: [PATCH] fixup! iotests: Specify explicit backing format where sensible
Am 13.07.2020 um 13:40 hat Eric Blake geschrieben:
Quoted 3 lines
Max spotted that I didn't test -qed...
Signed-off-by Eric Blake <[email protected]>
I guess I have a new name now... :-)
Kevin^WMax
Re: [PATCH] fixup! iotests: Specify explicit backing format where sensible
Quoted 6 lines · Kevin 'Max' Wolf
Am 13.07.2020 um 13:40 hat Eric Blake geschrieben:
Quoted 3 lines
Max spotted that I didn't test -qed...
Signed-off-by Eric Blake <[email protected]>I guess I have a new name now... :-)
Sorry, too many replies on the thread, and I messed up on which one I was reading!
Quoted 2 lines
Kevin^WMax
Brown Paper Bag^W^W^WEric
Signature · 3 lines
Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3226 Virtualization: qemu.org | libvirt.org