block: LUKS encryption slot management + iotest tweaks
Subsystems (2)
cover block: LUKS encryption slot management + iotest tweaks
Hi,
This series collects all patches from Maxim’s series “LUKS: encryption slot management using amend interface”, plus iotest patches that seem useful to have before it. (And then I modified some of Maxim’s patches to take advantage of those patches.)
Maxim’s series was here:
https://lists.nongnu.org/archive/html/qemu-block/2020-06/msg00330.html
I already posted patches 1 and 2 here:
https://lists.nongnu.org/archive/html/qemu-block/2020-06/msg00954.html
Change from the latter series:
- Patch 1:
- Use $SED instead of plain “sed” in one place I missed
- Older bash versions do not support readarray’s -d switch, so we have to separate the Formatting line by a \n instead of \0. Unfortunately, that means we cannot just split by \0 and be done with it, but we have to ensure we put everything before the line starting by “Formatting” into $filename_part, and we do that with a loop that iterates over the split array until it finds the “Formatting” line.
- Use multiple -e for grep instead of '(...|...|...)'
- Patch 2: Rebase fallout du to grep now using multiple -e parameters
git-backport-diff against Maxim’s series:
001/19:[down] 'iotests: Make _filter_img_create more active'
002/19:[0053] [FC] 'iotests: filter few more luks specific create options' 003/19:[down] 'iotests/common.rc: Add _require_working_luks'
004/19:[down] 'iotests.py: Add qemu_img_pipe_and_status()'
005/19:[down] 'iotests.py: Add (verify|has)_working_luks()'
006/19:[down] 'iotests: Check whether luks works'
007/19:[----] [--] 'qcrypto/core: add generic infrastructure for crypto options amendment' 008/19:[----] [--] 'qcrypto/luks: implement encryption key management' 009/19:[----] [--] 'block/amend: add 'force' option'
010/19:[----] [--] 'block/amend: separate amend and create options for qemu-img' 011/19:[0268] [FC] 'block/amend: refactor qcow2 amend options'
012/19:[----] [--] 'block/crypto: rename two functions'
013/19:[----] [--] 'block/crypto: implement the encryption key management' 014/19:[----] [--] 'block/qcow2: extend qemu-img amend interface with crypto options' 015/19:[0007] [FC] 'iotests: qemu-img tests for luks key management' 016/19:[----] [--] 'block/core: add generic infrastructure for x-blockdev-amend qmp command' 017/19:[----] [--] 'block/crypto: implement blockdev-amend'
018/19:[----] [--] 'block/qcow2: implement blockdev-amend'
019/19:[0005] [FC] 'iotests: add tests for blockdev-amend'
git-backport-diff against “iotests: Make _filter_img_create more active”:
Key:
[----] : patches are identical
[####] : number of functional differences between upstream/downstream patch [down] : patch is downstream-only
The flags [FC] indicate (F)unctional and (C)ontextual differences, respectively
001/19:[0035] [FC] 'iotests: Make _filter_img_create more active' 002/19:[0004] [FC] 'iotests: filter few more luks specific create options' 003/19:[down] 'iotests/common.rc: Add _require_working_luks' 004/19:[down] 'iotests.py: Add qemu_img_pipe_and_status()'
005/19:[down] 'iotests.py: Add (verify|has)_working_luks()'
006/19:[down] 'iotests: Check whether luks works'
007/19:[down] 'qcrypto/core: add generic infrastructure for crypto options amendment' 008/19:[down] 'qcrypto/luks: implement encryption key management' 009/19:[down] 'block/amend: add 'force' option'
010/19:[down] 'block/amend: separate amend and create options for qemu-img' 011/19:[down] 'block/amend: refactor qcow2 amend options'
012/19:[down] 'block/crypto: rename two functions'
013/19:[down] 'block/crypto: implement the encryption key management' 014/19:[down] 'block/qcow2: extend qemu-img amend interface with crypto options' 015/19:[down] 'iotests: qemu-img tests for luks key management' 016/19:[down] 'block/core: add generic infrastructure for x-blockdev-amend qmp command' 017/19:[down] 'block/crypto: implement blockdev-amend'
018/19:[down] 'block/qcow2: implement blockdev-amend'
019/19:[down] 'iotests: add tests for blockdev-amend'
Max Reitz (5):
iotests: Make _filter_img_create more active
iotests/common.rc: Add _require_working_luks
iotests.py: Add qemu_img_pipe_and_status()
iotests.py: Add (verify|has)_working_luks()
iotests: Check whether luks works
Maxim Levitsky (14):
iotests: filter few more luks specific create options qcrypto/core: add generic infrastructure for crypto options amendment qcrypto/luks: implement encryption key management block/amend: add 'force' option
block/amend: separate amend and create options for qemu-img block/amend: refactor qcow2 amend options
block/crypto: rename two functions
block/crypto: implement the encryption key management block/qcow2: extend qemu-img amend interface with crypto options
iotests: qemu-img tests for luks key management block/core: add generic infrastructure for x-blockdev-amend qmp
command
block/crypto: implement blockdev-amend
block/qcow2: implement blockdev-amend
iotests: add tests for blockdev-amend
docs/tools/qemu-img.rst | 5 +- qapi/block-core.json | 68 +++++ qapi/crypto.json | 73 +++++- qapi/job.json | 4 +- block/crypto.h | 37 +++ crypto/blockpriv.h | 8 + include/block/block.h | 1 + include/block/block_int.h | 24 +- include/crypto/block.h | 22 ++ block.c | 4 +- block/amend.c | 113 +++++++++ block/crypto.c | 206 +++++++++++++-- block/qcow2.c | 332 +++++++++++++----------- crypto/block-luks.c | 416 ++++++++++++++++++++++++++++++- crypto/block.c | 29 +++ qemu-img.c | 44 +++- block/Makefile.objs | 2 +- qemu-img-cmds.hx | 4 +- tests/qemu-iotests/049.out | 102 ++++---- tests/qemu-iotests/061.out | 12 +- tests/qemu-iotests/082.out | 185 ++++---------- tests/qemu-iotests/085.out | 38 +-- tests/qemu-iotests/087 | 1 + tests/qemu-iotests/087.out | 6 +- tests/qemu-iotests/112.out | 2 +- tests/qemu-iotests/134.out | 2 +- tests/qemu-iotests/141 | 2 +- tests/qemu-iotests/144.out | 4 +- tests/qemu-iotests/153 | 9 +- tests/qemu-iotests/158.out | 4 +- tests/qemu-iotests/178 | 1 + tests/qemu-iotests/182.out | 2 +- tests/qemu-iotests/185.out | 8 +- tests/qemu-iotests/188 | 1 + tests/qemu-iotests/188.out | 2 +- tests/qemu-iotests/189 | 1 + tests/qemu-iotests/189.out | 4 +- tests/qemu-iotests/198 | 1 + tests/qemu-iotests/198.out | 4 +- tests/qemu-iotests/206 | 1 + tests/qemu-iotests/255.out | 8 +- tests/qemu-iotests/263 | 1 + tests/qemu-iotests/263.out | 4 +- tests/qemu-iotests/274.out | 46 ++-- tests/qemu-iotests/280.out | 2 +- tests/qemu-iotests/284 | 1 + tests/qemu-iotests/284.out | 6 +- tests/qemu-iotests/293 | 208 ++++++++++++++++ tests/qemu-iotests/293.out | 99 ++++++++ tests/qemu-iotests/294 | 90 +++++++ tests/qemu-iotests/294.out | 30 +++ tests/qemu-iotests/295 | 280 +++++++++++++++++++++ tests/qemu-iotests/295.out | 40 +++ tests/qemu-iotests/296 | 234 +++++++++++++++++ tests/qemu-iotests/296.out | 33 +++ tests/qemu-iotests/common.filter | 106 ++++++-- tests/qemu-iotests/common.rc | 30 +++ tests/qemu-iotests/group | 4 + tests/qemu-iotests/iotests.py | 78 ++++-- 59 files changed, 2591 insertions(+), 493 deletions(-) create mode 100644 block/amend.c create mode 100755 tests/qemu-iotests/293 create mode 100644 tests/qemu-iotests/293.out create mode 100755 tests/qemu-iotests/294 create mode 100644 tests/qemu-iotests/294.out create mode 100755 tests/qemu-iotests/295 create mode 100644 tests/qemu-iotests/295.out create mode 100755 tests/qemu-iotests/296 create mode 100644 tests/qemu-iotests/296.out
Signature · 1 line
2.26.2
PATCH 1/19 iotests: Make _filter_img_create more active
Right now, _filter_img_create just filters out everything that looks format-dependent, and applies some filename filters. That means that we have to add another filter line every time some format gets a new creation option. This can be avoided by instead discarding everything and just keeping what we know is format-independent (format, size, backing file, encryption information[1], preallocation) or just interesting to have in the reference output (external data file path).
Furthermore, we probably want to sort these options. Format drivers are not required to define them in any specific order, so the output is effectively random (although this has never bothered us until now). We need a specific order for our reference outputs, though. Unfortunately, just using a plain "sort" would change a lot of existing reference outputs, so we have to pre-filter the option keys to keep our existing order (fmt, size, backing*, data, encryption info, preallocation).
Finally, this makes it difficult for _filter_img_create to automagically work for QMP output. Thus, this patch adds a separate _filter_img_create_for_qmp function that echos every line verbatim that does not start with "Formatting", and pipes those "Formatting" lines to _filter_img_create.
[1] Actually, the only thing that is really important is whether encryption is enabled or not. A patch by Maxim thus removes all other "encrypt.*" options from the output:
https://lists.nongnu.org/archive/html/qemu-block/2020-06/msg00339.html
But that patch needs to come later so we can get away with changing as few reference outputs in this patch here as possible.
--- tests/qemu-iotests/112.out | 2 +- tests/qemu-iotests/141 | 2 +- tests/qemu-iotests/153 | 9 ++- tests/qemu-iotests/common.filter | 109 ++++++++++++++++++++++++------- 4 files changed, 91 insertions(+), 31 deletions(-)
tests/qemu-iotests/112.out +1 -1
@@ -5,7 +5,7 @@ QA output created by 112 qemu-img: TEST_DIR/t.IMGFMT: Refcount width must be a power of two and may not exceed 64 bits Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864 qemu-img: TEST_DIR/t.IMGFMT: Refcount width must be a power of two and may not exceed 64 bits -Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864 refcount_bits=-1 +Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864 qemu-img: TEST_DIR/t.IMGFMT: Refcount width must be a power of two and may not exceed 64 bits Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864 qemu-img: TEST_DIR/t.IMGFMT: Refcount width must be a power of two and may not exceed 64 bits
tests/qemu-iotests/141 +1 -1
@@ -68,7 +68,7 @@ test_blockjob() _send_qemu_cmd $QEMU_HANDLE \ "$1" \ "$2" \ - | _filter_img_create | _filter_qmp_empty_return + | _filter_img_create_in_qmp | _filter_qmp_empty_return # We want this to return an error because the block job is still running _send_qemu_cmd $QEMU_HANDLE \
tests/qemu-iotests/153 +4 -5
@@ -167,11 +167,10 @@ done echo echo "== Creating ${TEST_IMG}.[abc] ==" | _filter_testdir -( - $QEMU_IMG create -f qcow2 "${TEST_IMG}.a" -b "${TEST_IMG}" - $QEMU_IMG create -f qcow2 "${TEST_IMG}.b" -b "${TEST_IMG}" - $QEMU_IMG create -f qcow2 "${TEST_IMG}.c" -b "${TEST_IMG}.b" -) | _filter_img_create +$QEMU_IMG create -f qcow2 "${TEST_IMG}.a" -b "${TEST_IMG}" | _filter_img_create +$QEMU_IMG create -f qcow2 "${TEST_IMG}.b" -b "${TEST_IMG}" | _filter_img_create +$QEMU_IMG create -f qcow2 "${TEST_IMG}.c" -b "${TEST_IMG}.b" \ + | _filter_img_create echo echo "== Two devices sharing the same file in backing chain =="
tests/qemu-iotests/common.filter +85 -24
123 lines not on this page · raw message
Signature · 1 line
2.26.2
Re: [PATCH 01/19] iotests: Make _filter_img_create more active
Quoted 22 lines · Max Reitz
Right now,
_filter_img_createjust filters out everything that looks format-dependent, and applies some filename filters. That means that we have to add another filter line every time some format gets a new creation option. This can be avoided by instead discarding everything and just keeping what we know is format-independent (format, size, backing file, encryption information[1], preallocation) or just interesting to have in the reference output (external data file path).Furthermore, we probably want to sort these options. Format drivers are not required to define them in any specific order, so the output is effectively random (although this has never bothered us until now). We need a specific order for our reference outputs, though. Unfortunately, just using a plain "sort" would change a lot of existing reference outputs, so we have to pre-filter the option keys to keep our existing order (fmt, size, backing*, data, encryption info, preallocation).
Finally, this makes it difficult for
_filter_img_createto automagically work for QMP output. Thus, this patch adds a separate_filter_img_create_for_qmpfunction that echos every line verbatim that does not start with "Formatting", and pipes those "Formatting" lines to_filter_img_create.
Quoted 8 lines
[1] Actually, the only thing that is really important is whether encryption is enabled or not. A patch by Maxim thus removes all other "encrypt.*" options from the output:
https://lists.nongnu.org/archive/html/qemu-block/2020-06/msg00339.html
But that patch needs to come later so we can get away with changing as few reference outputs in this patch here as possible.Signed-off-by Max Reitz <[email protected]>
I went over this patch again, and it looks OK, but I might have missed something.
Best regards,
Maxim Levitsky
Quoted 181 lines
--- tests/qemu-iotests/112.out | 2 +- tests/qemu-iotests/141 | 2 +- tests/qemu-iotests/153 | 9 ++- tests/qemu-iotests/common.filter | 109 ++++++++++++++++++++++++------- 4 files changed, 91 insertions(+), 31 deletions(-)tests/qemu-iotests/112.out +1
-1@@ -5,7 +5,7 @@ QA output created by 112 qemu-img: TEST_DIR/t.IMGFMT: Refcount width must be a power of two and may not exceed 64 bits Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864 qemu-img: TEST_DIR/t.IMGFMT: Refcount width must be a power of two and may not exceed 64 bits -Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864 refcount_bits=-1 +Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864 qemu-img: TEST_DIR/t.IMGFMT: Refcount width must be a power of two and may not exceed 64 bits Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864 qemu-img: TEST_DIR/t.IMGFMT: Refcount width must be a power of two and may not exceed 64 bitstests/qemu-iotests/141 +1
-1@@ -68,7 +68,7 @@ test_blockjob() _send_qemu_cmd $QEMU_HANDLE \ "$1" \ "$2" \ - | _filter_img_create | _filter_qmp_empty_return + | _filter_img_create_in_qmp | _filter_qmp_empty_return # We want this to return an error because the block job is still running _send_qemu_cmd $QEMU_HANDLE \tests/qemu-iotests/153 +4
-5@@ -167,11 +167,10 @@ done echo echo "== Creating ${TEST_IMG}.[abc] ==" | _filter_testdir -( - $QEMU_IMG create -f qcow2 "${TEST_IMG}.a" -b "${TEST_IMG}" - $QEMU_IMG create -f qcow2 "${TEST_IMG}.b" -b "${TEST_IMG}" - $QEMU_IMG create -f qcow2 "${TEST_IMG}.c" -b "${TEST_IMG}.b" -) | _filter_img_create +$QEMU_IMG create -f qcow2 "${TEST_IMG}.a" -b "${TEST_IMG}" | _filter_img_create +$QEMU_IMG create -f qcow2 "${TEST_IMG}.b" -b "${TEST_IMG}" | _filter_img_create +$QEMU_IMG create -f qcow2 "${TEST_IMG}.c" -b "${TEST_IMG}.b" \ + | _filter_img_create echo echo "== Two devices sharing the same file in backing chain =="tests/qemu-iotests/common.filter +85
-24@@ -122,38 +122,99 @@ _filter_actual_image_size() # replace driver-specific options in the "Formatting..." line _filter_img_create() { - data_file_filter=() - if data_file=$(_get_data_file "$TEST_IMG"); then - data_file_filter=(-e "s# data_file=$data_file##") + # Split the line into the pre-options part ($filename_part, which + # precedes ", fmt=") and the options part ($options, which starts + # with "fmt=") + # (And just echo everything before the first "^Formatting") + readarray formatting_line < <($SED -e 's/, fmt=/\n/') + + filename_part='' + options='' + lines=${#formatting_line[@]} + for ((i = 0; i < $lines; i++)); do + line=${formatting_line[i]} + unset formatting_line[i] + + filename_part="$filename_part$line" + + if echo "$line" | grep -q '^Formatting'; then + next_i=$((i + 1)) + if [ -n "${formatting_line[next_i]}" ]; then + options="fmt=${formatting_line[@]}" + fi + break + fi + done + + # Set grep_data_file to '\|data_file' to keep it; make it empty + # to drop it. + # We want to drop it if it is part of the global $IMGOPTS, and we + # want to keep it otherwise (if the test specifically wants to + # test data files). + grep_data_file=(-e data_file) + if _get_data_file "$TEST_IMG" > /dev/null; then + grep_data_file=() fi - $SED "${data_file_filter[@]}" \ + filename_filters=( -e "s#$REMOTE_TEST_DIR#TEST_DIR#g" \ -e "s#$IMGPROTO:$TEST_DIR#TEST_DIR#g" \ -e "s#$TEST_DIR#TEST_DIR#g" \ -e "s#$SOCK_DIR#SOCK_DIR#g" \ -e "s#$IMGFMT#IMGFMT#g" \ -e 's#nbd:127.0.0.1:[0-9]\\+#TEST_DIR/t.IMGFMT#g' \ - -e 's#nbd+unix:///\??socket=SOCK_DIR/nbd#TEST_DIR/t.IMGFMT#g' \ - -e "s# encryption=off##g" \ - -e "s# cluster_size=[0-9]\\+##g" \ - -e "s# table_size=[0-9]\\+##g" \ - -e "s# compat=[^ ]*##g" \ - -e "s# compat6=\\(on\\|off\\)##g" \ - -e "s# static=\\(on\\|off\\)##g" \ - -e "s# zeroed_grain=\\(on\\|off\\)##g" \ - -e "s# subformat=[^ ]*##g" \ - -e "s# adapter_type=[^ ]*##g" \ - -e "s# hwversion=[^ ]*##g" \ - -e "s# lazy_refcounts=\\(on\\|off\\)##g" \ - -e "s# block_size=[0-9]\\+##g" \ - -e "s# block_state_zero=\\(on\\|off\\)##g" \ - -e "s# log_size=[0-9]\\+##g" \ - -e "s# refcount_bits=[0-9]\\+##g" \ - -e "s# key-secret=[a-zA-Z0-9]\\+##g" \ - -e "s# iter-time=[0-9]\\+##g" \ - -e "s# force_size=\\(on\\|off\\)##g" \ - -e "s# compression_type=[a-zA-Z0-9]\\+##g" + -e 's#nbd+unix:///\??socket=SOCK_DIR/nbd#TEST_DIR/t.IMGFMT#g' + ) + + filename_part=$(echo "$filename_part" | $SED "${filename_filters[@]}") + + # Break the option line before each option (preserving pre-existing + # line breaks by replacing them by \0 and restoring them at the end), + # then filter out the options we want to keep and sort them according + # to some order that all block drivers used at the time of writing + # this function. + options=$( + echo "$options" \ + | tr '\n' '\0' \ + | $SED -e 's/ \([a-z0-9_.-]*\)=/\n\1=/g' \ + | grep -a -e '^fmt' -e '^size' -e '^backing' -e '^preallocation' \ + -e '^encrypt' "${grep_data_file[@]}" \ + | $SED "${filename_filters[@]}" \ + -e 's/^\(fmt\)/0-\1/' \ + -e 's/^\(size\)/1-\1/' \ + -e 's/^\(backing\)/2-\1/' \ + -e 's/^\(data_file\)/3-\1/' \ + -e 's/^\(encryption\)/4-\1/' \ + -e 's/^\(encrypt\.format\)/5-\1/' \ + -e 's/^\(encrypt\.key-secret\)/6-\1/' \ + -e 's/^\(encrypt\.iter-time\)/7-\1/' \ + -e 's/^\(preallocation\)/8-\1/' \ + | sort \ + | $SED -e 's/^[0-9]-//' \ + | tr '\n\0' ' \n' \ + | $SED -e 's/^ *$//' -e 's/ *$//' + ) + + if [ -n "$options" ]; then + echo "$filename_part, $options" + elif [ -n "$filename_part" ]; then + echo "$filename_part" + fi +} + +# Filter the "Formatting..." line in QMP output (leaving the QMP output +# untouched) +# (In contrast to _filter_img_create(), this function does not support +# multi-line Formatting output) +_filter_img_create_in_qmp() +{ + while read -r line; do + if echo "$line" | grep -q '^Formatting'; then + echo "$line" | _filter_img_create + else + echo "$line" + fi + done } _filter_img_create_size()
PATCH 2/19 iotests: filter few more luks specific create options
In reply to Max ReitzFrom: Maxim Levitsky <[email protected]>
This allows more tests to be able to have same output on both qcow2 luks encrypted images and raw luks images
--- tests/qemu-iotests/087.out | 6 +++--- tests/qemu-iotests/134.out | 2 +- tests/qemu-iotests/158.out | 4 ++-- tests/qemu-iotests/188.out | 2 +- tests/qemu-iotests/189.out | 4 ++-- tests/qemu-iotests/198.out | 4 ++-- tests/qemu-iotests/263.out | 4 ++-- tests/qemu-iotests/284.out | 6 +++--- tests/qemu-iotests/common.filter | 5 +---- 9 files changed, 17 insertions(+), 20 deletions(-)
tests/qemu-iotests/087.out +3 -3
@@ -34,7 +34,7 @@ QMP_VERSION === Encrypted image QCow === -Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=134217728 encryption=on encrypt.key-secret=sec0 +Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=134217728 encryption=on Testing: QMP_VERSION {"return": {}}
@@ -46,7 +46,7 @@ QMP_VERSION === Encrypted image LUKS === -Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=134217728 encrypt.format=luks encrypt.key-secret=sec0 +Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=134217728 Testing: QMP_VERSION {"return": {}}
@@ -58,7 +58,7 @@ QMP_VERSION === Missing driver === -Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=134217728 encryption=on encrypt.key-secret=sec0 +Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=134217728 encryption=on Testing: -S QMP_VERSION {"return": {}}
tests/qemu-iotests/134.out +1 -1
@@ -1,5 +1,5 @@ QA output created by 134 -Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=134217728 encryption=on encrypt.key-secret=sec0 +Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=134217728 encryption=on == reading whole image == read 134217728/134217728 bytes at offset 0
tests/qemu-iotests/158.out +2 -2
@@ -1,6 +1,6 @@ QA output created by 158 == create base == -Formatting 'TEST_DIR/t.IMGFMT.base', fmt=IMGFMT size=134217728 encryption=on encrypt.key-secret=sec0 +Formatting 'TEST_DIR/t.IMGFMT.base', fmt=IMGFMT size=134217728 encryption=on == writing whole image == wrote 134217728/134217728 bytes at offset 0
@@ -10,7 +10,7 @@ wrote 134217728/134217728 bytes at offset 0 read 134217728/134217728 bytes at offset 0 128 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) == create overlay == -Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=134217728 backing_file=TEST_DIR/t.IMGFMT.base encryption=on encrypt.key-secret=sec0 +Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=134217728 backing_file=TEST_DIR/t.IMGFMT.base encryption=on == writing part of a cluster == wrote 1024/1024 bytes at offset 0
tests/qemu-iotests/188.out +1 -1
@@ -1,5 +1,5 @@ QA output created by 188 -Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=16777216 encrypt.format=luks encrypt.key-secret=sec0 encrypt.iter-time=10 +Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=16777216 == reading whole image == read 16777216/16777216 bytes at offset 0
tests/qemu-iotests/189.out +2 -2
@@ -1,6 +1,6 @@ QA output created by 189 == create base == -Formatting 'TEST_DIR/t.IMGFMT.base', fmt=IMGFMT size=16777216 encrypt.format=luks encrypt.key-secret=sec0 encrypt.iter-time=10 +Formatting 'TEST_DIR/t.IMGFMT.base', fmt=IMGFMT size=16777216 == writing whole image == wrote 16777216/16777216 bytes at offset 0
@@ -10,7 +10,7 @@ wrote 16777216/16777216 bytes at offset 0 read 16777216/16777216 bytes at offset 0 16 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) == create overlay == -Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=16777216 backing_file=TEST_DIR/t.IMGFMT.base encrypt.format=luks encrypt.key-secret=sec1 encrypt.iter-time=10 +Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=16777216 backing_file=TEST_DIR/t.IMGFMT.base == writing part of a cluster == wrote 1024/1024 bytes at offset 0
tests/qemu-iotests/198.out +2 -2
@@ -1,12 +1,12 @@ QA output created by 198 == create base == -Formatting 'TEST_DIR/t.IMGFMT.base', fmt=IMGFMT size=16777216 encrypt.format=luks encrypt.key-secret=sec0 encrypt.iter-time=10 +Formatting 'TEST_DIR/t.IMGFMT.base', fmt=IMGFMT size=16777216 == writing whole image base == wrote 16777216/16777216 bytes at offset 0 16 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) == create overlay == -Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=16777216 backing_file=TEST_DIR/t.IMGFMT.base encrypt.format=luks encrypt.key-secret=sec1 encrypt.iter-time=10 +Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=16777216 backing_file=TEST_DIR/t.IMGFMT.base == writing whole image layer == wrote 16777216/16777216 bytes at offset 0
tests/qemu-iotests/263.out +2 -2
@@ -2,7 +2,7 @@ QA output created by 263 testing LUKS qcow2 encryption -Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576 encrypt.format=luks encrypt.key-secret=sec0 encrypt.iter-time=10 +Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576 == reading the whole image == read 1048576/1048576 bytes at offset 0 1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
@@ -21,7 +21,7 @@ read 982528/982528 bytes at offset 66048 testing legacy AES qcow2 encryption -Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576 encrypt.format=aes encrypt.key-secret=sec0 +Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576 == reading the whole image == read 1048576/1048576 bytes at offset 0 1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
tests/qemu-iotests/284.out +3 -3
@@ -2,7 +2,7 @@ QA output created by 284 testing LUKS qcow2 encryption -Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576 encrypt.format=luks encrypt.key-secret=sec0 encrypt.iter-time=10 +Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576 == cluster size 512 == checking image refcounts ==
@@ -21,7 +21,7 @@ wrote 1/1 bytes at offset 512 == rechecking image refcounts == No errors were found on the image. -Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576 encrypt.format=luks encrypt.key-secret=sec0 encrypt.iter-time=10 +Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576 == cluster size 2048 == checking image refcounts ==
@@ -40,7 +40,7 @@ wrote 1/1 bytes at offset 2048 == rechecking image refcounts == No errors were found on the image. -Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576 encrypt.format=luks encrypt.key-secret=sec0 encrypt.iter-time=10 +Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576 == cluster size 32768 == checking image refcounts ==
tests/qemu-iotests/common.filter +1 -4
@@ -178,16 +178,13 @@ _filter_img_create() | tr '\n' '\0' \ | $SED -e 's/ \([a-z0-9_.-]*\)=/\n\1=/g' \ | grep -a -e '^fmt' -e '^size' -e '^backing' -e '^preallocation' \ - -e '^encrypt' "${grep_data_file[@]}" \ + -e '^encryption' "${grep_data_file[@]}" \ | $SED "${filename_filters[@]}" \ -e 's/^\(fmt\)/0-\1/' \ -e 's/^\(size\)/1-\1/' \ -e 's/^\(backing\)/2-\1/' \ -e 's/^\(data_file\)/3-\1/' \ -e 's/^\(encryption\)/4-\1/' \ - -e 's/^\(encrypt\.format\)/5-\1/' \ - -e 's/^\(encrypt\.key-secret\)/6-\1/' \ - -e 's/^\(encrypt\.iter-time\)/7-\1/' \ -e 's/^\(preallocation\)/8-\1/' \ | sort \ | $SED -e 's/^[0-9]-//' \
Signature · 1 line
2.26.2
PATCH 3/19 iotests/common.rc: Add _require_working_luks
In reply to Max ReitzThat the luks driver is present is little indication on whether it is actually working. Without the crypto libraries linked in, it does not work. So add this function, which tries to create a luks image to see whether that actually works.
--- tests/qemu-iotests/common.rc | 27 +++++++++++++++++++++++++++ 1 file changed, 27 insertions(+)
tests/qemu-iotests/common.rc +27 -0
@@ -740,6 +740,33 @@ _unsupported_imgopts() done } +# Caution: Overwrites $TEST_DIR/t.luks +_require_working_luks() +{ + file="$TEST_DIR/t.luks" + + output=$( + $QEMU_IMG create -f luks \ + --object secret,id=sec0,data=hunter0 \ + -o key-secret=sec0 \ + -o iter-time=10 \ + "$file" \ + 1M \ + 2>&1 + ) + status=$? + + IMGFMT='luks' _rm_test_img "$file" + + if [ $status != 0 ]; then + reason=$(echo "$output" | grep "$file:" | $SED -e "s#.*$file: *##") + if [ -z "$reason" ]; then + reason="Failed to create a LUKS image" + fi + _notrun "$reason" + fi +} + # this test requires that a specified command (executable) exists # _require_command()
Signature · 1 line
2.26.2
Re: [PATCH 03/19] iotests/common.rc: Add _require_working_luks
Quoted 38 lines · Max Reitz
That the luks driver is present is little indication on whether it is actually working. Without the crypto libraries linked in, it does not work. So add this function, which tries to create a luks image to see whether that actually works.
Signed-off-by Max Reitz <[email protected]>--- tests/qemu-iotests/common.rc | 27 +++++++++++++++++++++++++++ 1 file changed, 27 insertions(+)tests/qemu-iotests/common.rc +20
-0@@ -740,6 +740,33 @@ _unsupported_imgopts() done } +# Caution: Overwrites $TEST_DIR/t.luks +_require_working_luks() +{ + file="$TEST_DIR/t.luks" + + output=$( + $QEMU_IMG create -f luks \ + --object secret,id=sec0,data=hunter0 \ + -o key-secret=sec0 \ + -o iter-time=10 \ + "$file" \ + 1M \ + 2>&1 + ) + status=$? + + IMGFMT='luks' _rm_test_img "$file" + + if [ $status != 0 ]; then + reason=$(echo "$output" | grep "$file:" | $SED -e "s#.*$file: *##")
tests/qemu-iotests/common.rc _unsupported_imgopts() · line 772
Quoted 10 lines
tests/qemu-iotests/common.rc +7
-0+ if [ -z "$reason" ]; then + reason="Failed to create a LUKS image" + fi + _notrun "$reason" + fi +} + # this test requires that a specified command (executable) exists # _require_command()
Look OK. I also tested this by building the qemu with all the patches applied,a and with --disable-nettle --disable-gcrypt
And now all my tests are skipped with this nice message:
"No crypto library supporting PBKDF in this build: Function not implemented"
Thank you very very much for implementing this!
Best regards,
Maxim Levitsky
PATCH 4/19 iotests.py: Add qemu_img_pipe_and_status()
In reply to Max ReitzThis function will be used by the next patch, which intends to check both the exit code and qemu-img's output.
--- tests/qemu-iotests/iotests.py | 34 +++++++++++++++++----------------- 1 file changed, 17 insertions(+), 17 deletions(-)
tests/qemu-iotests/iotests.py +17 -17
@@ -29,7 +29,7 @@ import struct import subprocess import sys from typing import (Any, Callable, Dict, Iterable, - List, Optional, Sequence, TypeVar) + List, Optional, Sequence, Tuple, TypeVar) import unittest # pylint: disable=import-error, wrong-import-position
@@ -90,15 +90,23 @@ luks_default_secret_object = 'secret,id=keysec0,data=' + \ luks_default_key_secret_opt = 'key-secret=keysec0' -def qemu_img(*args): - '''Run qemu-img and return the exit code''' - devnull = open('/dev/null', 'r+') - exitcode = subprocess.call(qemu_img_args + list(args), - stdin=devnull, stdout=devnull) +def qemu_img_pipe_and_status(*args: str) -> Tuple[str, int]: + """ + Run qemu-img and return both its output and its exit code + """ + subp = subprocess.Popen(qemu_img_args + list(args), + stdout=subprocess.PIPE, + stderr=subprocess.STDOUT, + universal_newlines=True) + exitcode = subp.wait() if exitcode < 0: sys.stderr.write('qemu-img received signal %i: %s\n' % (-exitcode, ' '.join(qemu_img_args + list(args)))) - return exitcode + return (subp.communicate()[0], exitcode) + +def qemu_img(*args: str) -> int: + '''Run qemu-img and return the exit code''' + return qemu_img_pipe_and_status(*args)[1] def ordered_qmp(qmsg, conv_keys=True): # Dictionaries are not ordered prior to 3.6, therefore:
@@ -140,17 +148,9 @@ def qemu_img_verbose(*args): % (-exitcode, ' '.join(qemu_img_args + list(args)))) return exitcode -def qemu_img_pipe(*args): +def qemu_img_pipe(*args: str) -> str: '''Run qemu-img and return its output''' - subp = subprocess.Popen(qemu_img_args + list(args), - stdout=subprocess.PIPE, - stderr=subprocess.STDOUT, - universal_newlines=True) - exitcode = subp.wait() - if exitcode < 0: - sys.stderr.write('qemu-img received signal %i: %s\n' - % (-exitcode, ' '.join(qemu_img_args + list(args)))) - return subp.communicate()[0] + return qemu_img_pipe_and_status(*args)[0] def qemu_img_log(*args): result = qemu_img_pipe(*args)
Signature · 1 line
2.26.2
Re: [PATCH 04/19] iotests.py: Add qemu_img_pipe_and_status()
Quoted 61 lines · Max Reitz
This function will be used by the next patch, which intends to check both the exit code and qemu-img's output.
Signed-off-by Max Reitz <[email protected]>--- tests/qemu-iotests/iotests.py | 34 +++++++++++++++++----------------- 1 file changed, 17 insertions(+), 17 deletions(-)tests/qemu-iotests/iotests.py +16
-11@@ -29,7 +29,7 @@ import struct import subprocess import sys from typing import (Any, Callable, Dict, Iterable, - List, Optional, Sequence, TypeVar) + List, Optional, Sequence, Tuple, TypeVar) import unittest # pylint: disable=import-error, wrong-import-position@@ -90,15 +90,23 @@ luks_default_secret_object = 'secret,id=keysec0,data=' + \ luks_default_key_secret_opt = 'key-secret=keysec0' -def qemu_img(*args): - '''Run qemu-img and return the exit code''' - devnull = open('/dev/null', 'r+') - exitcode = subprocess.call(qemu_img_args + list(args), - stdin=devnull, stdout=devnull) +def qemu_img_pipe_and_status(*args: str) -> Tuple[str, int]: + """ + Run qemu-img and return both its output and its exit code + """ + subp = subprocess.Popen(qemu_img_args + list(args), + stdout=subprocess.PIPE, + stderr=subprocess.STDOUT, + universal_newlines=True) + exitcode = subp.wait() if exitcode < 0: sys.stderr.write('qemu-img received signal %i: %s\n' % (-exitcode, ' '.join(qemu_img_args + list(args)))) - return exitcode + return (subp.communicate()[0], exitcode) + +def qemu_img(*args: str) -> int: + '''Run qemu-img and return the exit code''' + return qemu_img_pipe_and_status(*args)[1] def ordered_qmp(qmsg, conv_keys=True): # Dictionaries are not ordered prior to 3.6, therefore:@@ -140,17 +148,9 @@ def qemu_img_verbose(*args): % (-exitcode, ' '.join(qemu_img_args + list(args)))) return exitcode -def qemu_img_pipe(*args): +def qemu_img_pipe(*args: str) -> str: '''Run qemu-img and return its output''' - subp = subprocess.Popen(qemu_img_args + list(args), - stdout=subprocess.PIPE, - stderr=subprocess.STDOUT,
tests/qemu-iotests/iotests.py def qemu_img_verbose(*args): · line 156
Quoted 10 lines
tests/qemu-iotests/iotests.py +1
-6- universal_newlines=True) - exitcode = subp.wait() - if exitcode < 0: - sys.stderr.write('qemu-img received signal %i: %s\n' - % (-exitcode, ' '.join(qemu_img_args + list(args)))) - return subp.communicate()[0] + return qemu_img_pipe_and_status(*args)[0] def qemu_img_log(*args): result = qemu_img_pipe(*args)
You made me learn a bit about python type hints, and I don't regret it :-) Looks OK.
Best regards,
Maxim Levitsky
PATCH 5/19 iotests.py: Add (verify|has)_working_luks()
In reply to Max ReitzSimilar to _require_working_luks for bash tests, these functions can be used to check whether our luks driver can actually create images.
--- tests/qemu-iotests/iotests.py | 39 +++++++++++++++++++++++++++++++++++ 1 file changed, 39 insertions(+)
tests/qemu-iotests/iotests.py +39 -0
@@ -1052,6 +1052,45 @@ def verify_quorum(): if not supports_quorum(): notrun('quorum support missing') +def has_working_luks() -> Tuple[bool, str]: + """ + Check whether our LUKS driver can actually create images + (this extends to LUKS encryption for qcow2). + + If not, return the reason why. + """ + + img_file = f'{test_dir}/luks-test.luks' + (output, status) = \ + qemu_img_pipe_and_status('create', '-f', 'luks', + '--object', luks_default_secret_object, + '-o', luks_default_key_secret_opt, + '-o', 'iter-time=10', + img_file, '1G') + try: + os.remove(img_file) + except OSError: + pass + + if status != 0: + reason = output + for line in output.splitlines(): + if img_file + ':' in line: + reason = line.split(img_file + ':', 1)[1].strip() + break + + return (False, reason) + else: + return (True, '') + +def verify_working_luks(): + """ + Skip test suite if LUKS does not work + """ + (working, reason) = has_working_luks() + if not working: + notrun(reason) + def qemu_pipe(*args): """ Run qemu with an option to print something and exit (e.g. a help option).
Signature · 1 line
2.26.2
Re: [PATCH 05/19] iotests.py: Add (verify|has)_working_luks()
Quoted 48 lines · Max Reitz
Similar to
_require_working_luksfor bash tests, these functions can be used to check whether our luks driver can actually create images.Signed-off-by Max Reitz <[email protected]>--- tests/qemu-iotests/iotests.py | 39 +++++++++++++++++++++++++++++++++++ 1 file changed, 39 insertions(+)tests/qemu-iotests/iotests.py +32
-0@@ -1052,6 +1052,45 @@ def verify_quorum(): if not supports_quorum(): notrun('quorum support missing') +def has_working_luks() -> Tuple[bool, str]: + """ + Check whether our LUKS driver can actually create images + (this extends to LUKS encryption for qcow2). + + If not, return the reason why. + """ + + img_file = f'{test_dir}/luks-test.luks' + (output, status) = \ + qemu_img_pipe_and_status('create', '-f', 'luks', + '--object', luks_default_secret_object, + '-o', luks_default_key_secret_opt, + '-o', 'iter-time=10', + img_file, '1G') + try: + os.remove(img_file) + except OSError: + pass + + if status != 0: + reason = output + for line in output.splitlines(): + if img_file + ':' in line: + reason = line.split(img_file + ':', 1)[1].strip() + break + + return (False, reason) + else: + return (True, '') + +def verify_working_luks():
tests/qemu-iotests/iotests.py def verify_quorum(): · line 1096
Quoted 10 lines
tests/qemu-iotests/iotests.py +7
-0+ """ + Skip test suite if LUKS does not work + """ + (working, reason) = has_working_luks() + if not working: + notrun(reason) + def qemu_pipe(*args): """ Run qemu with an option to print something and exit (e.g. a help option).
It just occured to me that we can have a situation when luks driver is blacklisted (via block driver blacklist "--block-drv-whitelist=") then this test.
THe whilelist only affects the qmp it seems so this check doesn't catch it, plus you could have case when luks driver is blacklisted but qcow2 isn't
When I build qemu with
'--block-drv-whitelist='raw,qcow2' I was able to break iotests 295 296 But this is such a non issue, that I am just noting for reference.
Best regards,
Maxim Levitsky
Re: [PATCH 05/19] iotests.py: Add (verify|has)_working_luks()
Quoted 63 lines · Maxim Levitsky
Quoted 48 lines · Max Reitz
Similar to
_require_working_luksfor bash tests, these functions can be used to check whether our luks driver can actually create images.Signed-off-by Max Reitz <[email protected]>--- tests/qemu-iotests/iotests.py | 39 +++++++++++++++++++++++++++++++++++ 1 file changed, 39 insertions(+)tests/qemu-iotests/iotests.py +32
-0@@ -1052,6 +1052,45 @@ def verify_quorum(): if not supports_quorum(): notrun('quorum support missing') +def has_working_luks() -> Tuple[bool, str]: + """ + Check whether our LUKS driver can actually create images + (this extends to LUKS encryption for qcow2). + + If not, return the reason why. + """ + + img_file = f'{test_dir}/luks-test.luks' + (output, status) = \ + qemu_img_pipe_and_status('create', '-f', 'luks', + '--object', luks_default_secret_object, + '-o', luks_default_key_secret_opt, + '-o', 'iter-time=10', + img_file, '1G') + try: + os.remove(img_file) + except OSError: + pass + + if status != 0: + reason = output + for line in output.splitlines(): + if img_file + ':' in line: + reason = line.split(img_file + ':', 1)[1].strip() + break + + return (False, reason) + else: + return (True, '') + +def verify_working_luks():tests/qemu-iotests/iotests.py def verify_quorum(): · line 1096
Quoted 10 lines
tests/qemu-iotests/iotests.py +7
-0+ """ + Skip test suite if LUKS does not work + """ + (working, reason) = has_working_luks() + if not working: + notrun(reason) + def qemu_pipe(*args): """ Run qemu with an option to print something and exit (e.g. a help option).It just occured to me that we can have a situation when luks driver is blacklisted (via block driver blacklist "--block-drv-whitelist=") then this test.
Quoted 2 lines
THe whilelist only affects the qmp it seems so this check doesn't catch it, plus you could have case when luks driver is blacklisted but qcow2 isn't
Yes, the whitelist only affects qemu proper, not qemu-img.
Quoted 3 lines
When I build qemu with
'--block-drv-whitelist='raw,qcow2' I was able to break iotests 295 296 But this is such a non issue, that I am just noting for reference.
Hm. If anything, that’s actually a problem in the test infrastructure, isn’t it? For example, if you don’t whitelist qcow2, then all qcow2 tests break. Ideally, they shouldn’t. The test infrastructure should check whether the format used to run the test ($IMGFMT) is actually supported (e.g. iotests.py’s _verify_image_format() could check whether the format is actually in supported_formats()), and if it isn’t, skip the test.
OTOH, that’s pretty much the user’s fault for using an $IMGFMT that they didn’t whitelist, so I’m not sure whether we’re actually keen on fixing that.
*shrug* O:)
Max
PATCH 6/19 iotests: Check whether luks works
In reply to Max ReitzWhenever running an iotest for the luks format, we should check whether luks actually really works.
Tests that try to create luks-encrypted qcow2 images should do the same.
--- tests/qemu-iotests/087 | 1 + tests/qemu-iotests/178 | 1 + tests/qemu-iotests/188 | 1 + tests/qemu-iotests/189 | 1 + tests/qemu-iotests/198 | 1 + tests/qemu-iotests/206 | 1 + tests/qemu-iotests/263 | 1 + tests/qemu-iotests/284 | 1 + tests/qemu-iotests/common.rc | 3 +++ tests/qemu-iotests/iotests.py | 5 +++++ 10 files changed, 16 insertions(+)
tests/qemu-iotests/087 +1 -0
@@ -39,6 +39,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15 _supported_fmt qcow2 _supported_proto file _supported_os Linux +_require_working_luks do_run_qemu() {
tests/qemu-iotests/178 +1 -0
@@ -41,6 +41,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15 _supported_fmt raw qcow2 _supported_proto file _supported_os Linux +_require_working_luks echo "== Input validation ==" echo
tests/qemu-iotests/188 +1 -0
@@ -39,6 +39,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15 _supported_fmt qcow2 _supported_proto generic _supported_os Linux +_require_working_luks size=16M
tests/qemu-iotests/189 +1 -0
@@ -39,6 +39,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15 _supported_fmt qcow2 _supported_proto generic _supported_os Linux +_require_working_luks size=16M
tests/qemu-iotests/198 +1 -0
@@ -39,6 +39,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15 _supported_fmt qcow2 _supported_proto generic _supported_os Linux +_require_working_luks size=16M
tests/qemu-iotests/206 +1 -0
@@ -24,6 +24,7 @@ import iotests from iotests import imgfmt iotests.script_initialize(supported_fmts=['qcow2']) +iotests.verify_working_luks() with iotests.FilePath('t.qcow2') as disk_path, \ iotests.FilePath('t.qcow2.base') as backing_path, \
tests/qemu-iotests/263 +1 -0
@@ -40,6 +40,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15 _supported_fmt qcow2 _supported_proto generic _supported_os Linux +_require_working_luks size=1M
tests/qemu-iotests/284 +1 -0
@@ -39,6 +39,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15 _supported_fmt qcow2 _supported_proto generic _supported_os Linux +_require_working_luks size=1M
tests/qemu-iotests/common.rc +3 -0
@@ -605,6 +605,9 @@ _supported_fmt() # setting IMGFMT_GENERIC to false. for f; do if [ "$f" = "$IMGFMT" -o "$f" = "generic" -a "$IMGFMT_GENERIC" = "true" ]; then + if [ "$IMGFMT" = "luks" ]; then + _require_working_luks + fi return fi done
tests/qemu-iotests/iotests.py +5 -0
@@ -1010,12 +1010,17 @@ def _verify_image_format(supported_fmts: Sequence[str] = (), # similar to # _supported_fmt generic # for bash tests + if imgfmt == 'luks': + verify_working_luks() return not_sup = supported_fmts and (imgfmt not in supported_fmts) if not_sup or (imgfmt in unsupported_fmts): notrun('not suitable for this image format: %s' % imgfmt) + if imgfmt == 'luks': + verify_working_luks() + def _verify_protocol(supported: Sequence[str] = (), unsupported: Sequence[str] = ()) -> None: assert not (supported and unsupported)
Signature · 1 line
2.26.2
Re: [PATCH 06/19] iotests: Check whether luks works
Quoted 141 lines · Max Reitz
Whenever running an iotest for the luks format, we should check whether luks actually really works.
Tests that try to create luks-encrypted qcow2 images should do the same.
Signed-off-by Max Reitz <[email protected]>--- tests/qemu-iotests/087 | 1 + tests/qemu-iotests/178 | 1 + tests/qemu-iotests/188 | 1 + tests/qemu-iotests/189 | 1 + tests/qemu-iotests/198 | 1 + tests/qemu-iotests/206 | 1 + tests/qemu-iotests/263 | 1 + tests/qemu-iotests/284 | 1 + tests/qemu-iotests/common.rc | 3 +++ tests/qemu-iotests/iotests.py | 5 +++++ 10 files changed, 16 insertions(+)tests/qemu-iotests/087 +1
-0@@ -39,6 +39,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15 _supported_fmt qcow2 _supported_proto file _supported_os Linux +_require_working_luks do_run_qemu() {tests/qemu-iotests/178 +1
-0@@ -41,6 +41,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15 _supported_fmt raw qcow2 _supported_proto file _supported_os Linux +_require_working_luks echo "== Input validation ==" echotests/qemu-iotests/188 +1
-0@@ -39,6 +39,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15 _supported_fmt qcow2 _supported_proto generic _supported_os Linux +_require_working_luks size=16Mtests/qemu-iotests/189 +1
-0@@ -39,6 +39,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15 _supported_fmt qcow2 _supported_proto generic _supported_os Linux +_require_working_luks size=16Mtests/qemu-iotests/198 +1
-0@@ -39,6 +39,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15 _supported_fmt qcow2 _supported_proto generic _supported_os Linux +_require_working_luks size=16Mtests/qemu-iotests/206 +1
-0@@ -24,6 +24,7 @@ import iotests from iotests import imgfmt iotests.script_initialize(supported_fmts=['qcow2']) +iotests.verify_working_luks() with iotests.FilePath('t.qcow2') as disk_path, \ iotests.FilePath('t.qcow2.base') as backing_path, \tests/qemu-iotests/263 +1
-0@@ -40,6 +40,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15 _supported_fmt qcow2 _supported_proto generic _supported_os Linux +_require_working_luks size=1Mtests/qemu-iotests/284 +1
-0@@ -39,6 +39,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15 _supported_fmt qcow2 _supported_proto generic _supported_os Linux +_require_working_luks size=1Mtests/qemu-iotests/common.rc +3
-0@@ -605,6 +605,9 @@ _supported_fmt() # setting IMGFMT_GENERIC to false. for f; do if [ "$f" = "$IMGFMT" -o "$f" = "generic" -a "$IMGFMT_GENERIC" = "true" ]; then + if [ "$IMGFMT" = "luks" ]; then + _require_working_luks + fi return fi donetests/qemu-iotests/iotests.py +2
-0@@ -1010,12 +1010,17 @@ def _verify_image_format(supported_fmts: Sequence[str] = (), # similar to # _supported_fmt generic # for bash tests + if imgfmt == 'luks': + verify_working_luks() return
tests/qemu-iotests/iotests.py def _verify_image_format(supported_fmts: Sequence[str] = (), · line 1026
Quoted 10 lines
tests/qemu-iotests/iotests.py +3
-0not_sup = supported_fmts and (imgfmt not in supported_fmts) if not_sup or (imgfmt in unsupported_fmts): notrun('not suitable for this image format: %s' % imgfmt) + if imgfmt == 'luks': + verify_working_luks() + def _verify_protocol(supported: Sequence[str] = (), unsupported: Sequence[str] = ()) -> None: assert not (supported and unsupported)
If I understand correctly all raw luks tests are now under the condition that luks works, and qcow2 tests are added to this manually for the tests that use encryption. Seems fair to me.
PS: I also run the iotests with luks and qcow2, and it works fine.
Note that iotest 051 is broken (regardless of these patches)
051 fail [14:18:42] [14:18:49] output mismatch (see 051.out.bad)
home/mlevitsk/UPSTREAM/qemu/build_luks/tests/qemu-iotests/051.out.bad renamed from home/mlevitsk/UPSTREAM/qemu/src/tests/qemu-iotests/051.pc.out +2 -2
@@ -142,7 +142,7 @@ Testing: -drive if=ide QEMU X.Y.Z monitor - type 'help' for more information -(qemu) QEMU_PROG: Initialization of device ide-hd failed: Device needs media, but drive is empty +(qemu) QEMU_PROG: Device needs media, but drive is empty Testing: -drive if=virtio QEMU X.Y.Z monitor - type 'help' for more information
@@ -214,7 +214,7 @@ Testing: -drive file=TEST_DIR/t.qcow2,if=ide,readonly=on QEMU X.Y.Z monitor - type 'help' for more information -(qemu) QEMU_PROG: Initialization of device ide-hd failed: Block node is read-only +(qemu) QEMU_PROG: Block node is read-only Testing: -drive file=TEST_DIR/t.qcow2,if=virtio,readonly=on QEMU X.Y.Z monitor - type 'help' for more information
Failures: 051
Failed 1 of 1 iotests
It seem to use an pc machine type specific 'out' file, thus I might have something wrong in my enviroment.
Best regards,
Maxim Levitsky
Re: [PATCH 06/19] iotests: Check whether luks works
Quoted 190 lines · Maxim Levitsky
Quoted 141 lines · Max Reitz
Whenever running an iotest for the luks format, we should check whether luks actually really works.
Tests that try to create luks-encrypted qcow2 images should do the same.
Signed-off-by Max Reitz <[email protected]>--- tests/qemu-iotests/087 | 1 + tests/qemu-iotests/178 | 1 + tests/qemu-iotests/188 | 1 + tests/qemu-iotests/189 | 1 + tests/qemu-iotests/198 | 1 + tests/qemu-iotests/206 | 1 + tests/qemu-iotests/263 | 1 + tests/qemu-iotests/284 | 1 + tests/qemu-iotests/common.rc | 3 +++ tests/qemu-iotests/iotests.py | 5 +++++ 10 files changed, 16 insertions(+)tests/qemu-iotests/087 +1
-0@@ -39,6 +39,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15 _supported_fmt qcow2 _supported_proto file _supported_os Linux +_require_working_luks do_run_qemu() {tests/qemu-iotests/178 +1
-0@@ -41,6 +41,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15 _supported_fmt raw qcow2 _supported_proto file _supported_os Linux +_require_working_luks echo "== Input validation ==" echotests/qemu-iotests/188 +1
-0@@ -39,6 +39,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15 _supported_fmt qcow2 _supported_proto generic _supported_os Linux +_require_working_luks size=16Mtests/qemu-iotests/189 +1
-0@@ -39,6 +39,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15 _supported_fmt qcow2 _supported_proto generic _supported_os Linux +_require_working_luks size=16Mtests/qemu-iotests/198 +1
-0@@ -39,6 +39,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15 _supported_fmt qcow2 _supported_proto generic _supported_os Linux +_require_working_luks size=16Mtests/qemu-iotests/206 +1
-0@@ -24,6 +24,7 @@ import iotests from iotests import imgfmt iotests.script_initialize(supported_fmts=['qcow2']) +iotests.verify_working_luks() with iotests.FilePath('t.qcow2') as disk_path, \ iotests.FilePath('t.qcow2.base') as backing_path, \tests/qemu-iotests/263 +1
-0@@ -40,6 +40,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15 _supported_fmt qcow2 _supported_proto generic _supported_os Linux +_require_working_luks size=1Mtests/qemu-iotests/284 +1
-0@@ -39,6 +39,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15 _supported_fmt qcow2 _supported_proto generic _supported_os Linux +_require_working_luks size=1Mtests/qemu-iotests/common.rc +3
-0@@ -605,6 +605,9 @@ _supported_fmt() # setting IMGFMT_GENERIC to false. for f; do if [ "$f" = "$IMGFMT" -o "$f" = "generic" -a "$IMGFMT_GENERIC" = "true" ]; then + if [ "$IMGFMT" = "luks" ]; then + _require_working_luks + fi return fi donetests/qemu-iotests/iotests.py +2
-0@@ -1010,12 +1010,17 @@ def _verify_image_format(supported_fmts: Sequence[str] = (), # similar to # _supported_fmt generic # for bash tests + if imgfmt == 'luks': + verify_working_luks() returntests/qemu-iotests/iotests.py def _verify_image_format(supported_fmts: Sequence[str] = (), · line 1026
Quoted 10 lines
tests/qemu-iotests/iotests.py +3
-0not_sup = supported_fmts and (imgfmt not in supported_fmts) if not_sup or (imgfmt in unsupported_fmts): notrun('not suitable for this image format: %s' % imgfmt) + if imgfmt == 'luks': + verify_working_luks() + def _verify_protocol(supported: Sequence[str] = (), unsupported: Sequence[str] = ()) -> None: assert not (supported and unsupported)If I understand correctly all raw luks tests are now under the condition that luks works, and qcow2 tests are added to this manually for the tests that use encryption. Seems fair to me.
Reviewed-by Maxim Levitsky <[email protected]>PS: I also run the iotests with luks and qcow2, and it works fine.
Note that iotest 051 is broken (regardless of these patches)
051 fail [14:18:42] [14:18:49] output mismatch (see 051.out.bad)
home/mlevitsk/UPSTREAM/qemu/build_luks/tests/qemu-iotests/051.out.bad renamed from home/mlevitsk/UPSTREAM/qemu/src/tests/qemu-iotests/051.pc.out +2
-2@@ -142,7 +142,7 @@ Testing: -drive if=ide QEMU X.Y.Z monitor - type 'help' for more information -(qemu) QEMU_PROG: Initialization of device ide-hd failed: Device needs media, but drive is empty +(qemu) QEMU_PROG: Device needs media, but drive is empty Testing: -drive if=virtio QEMU X.Y.Z monitor - type 'help' for more information@@ -214,7 +214,7 @@ Testing: -drive file=TEST_DIR/t.qcow2,if=ide,readonly=on QEMU X.Y.Z monitor - type 'help' for more information -(qemu) QEMU_PROG: Initialization of device ide-hd failed: Block node is read-only +(qemu) QEMU_PROG: Block node is read-only Testing: -drive file=TEST_DIR/t.qcow2,if=virtio,readonly=on QEMU X.Y.Z monitor - type 'help' for more informationFailures: 051
Failed 1 of 1 iotests
Quoted 1 line
It seem to use an pc machine type specific 'out' file, thus I might have something wrong in my enviroment.
No, it isn’t just you. The patch to fix that is in my pull request:
https://lists.nongnu.org/archive/html/qemu-block/2020-06/msg01159.html
Max
PATCH 7/19 qcrypto/core: add generic infrastructure for crypto options amendment
In reply to Max ReitzFrom: Maxim Levitsky <[email protected]>
This will be used first to implement luks keyslot management.
block_crypto_amend_opts_init will be used to convert qemu-img cmdline to QCryptoBlockAmendOptions
Message-Id: <[email protected]>
--- qapi/crypto.json | 16 ++++++++++++++++ block/crypto.h | 3 +++ crypto/blockpriv.h | 8 ++++++++ include/crypto/block.h | 22 ++++++++++++++++++++++ block/crypto.c | 17 +++++++++++++++++ crypto/block.c | 29 +++++++++++++++++++++++++++++ 6 files changed, 95 insertions(+)
qapi/crypto.json +16 -0
@@ -309,3 +309,19 @@ 'base': 'QCryptoBlockInfoBase', 'discriminator': 'format', 'data': { 'luks': 'QCryptoBlockInfoLUKS' } } + + + +## +# @QCryptoBlockAmendOptions: +# +# The options that are available for all encryption formats +# when amending encryption settings +# +# Since: 5.1 +## +{ 'union': 'QCryptoBlockAmendOptions', + 'base': 'QCryptoBlockOptionsBase', + 'discriminator': 'format', + 'data': { + } }
block/crypto.h +3 -0
@@ -91,6 +91,9 @@QCryptoBlockCreateOptions *block_crypto_create_opts_init (QDict *opts,Error **errp); +QCryptoBlockAmendOptions * +block_crypto_amend_opts_init (QDict *opts,Error **errp); +QCryptoBlockOpenOptions *block_crypto_open_opts_init (QDict *opts,Error **errp);
crypto/blockpriv.h +8 -0
@@ -62,6 +62,14 @@ struct QCryptoBlockDriver {void *opaque, Error **errp); +int (*amend)(QCryptoBlock *block, +QCryptoBlockReadFunc readfunc, +QCryptoBlockWriteFunc writefunc, +void *opaque, +QCryptoBlockAmendOptions *options, +bool force, +Error **errp); +int (*get_info)(QCryptoBlock *block,QCryptoBlockInfo *info,Error **errp);
include/crypto/block.h +22 -0
@@ -144,6 +144,28 @@ QCryptoBlock *qcrypto_block_create(QCryptoBlockCreateOptions *options,void *opaque, Error **errp); +/** +* qcrypto_block_amend_options: +* @block: the block encryption object +* +* @readfunc: callback for reading data from the volume header +* @writefunc: callback for writing data to the volume header +* @opaque: data to pass to @readfunc and @writefunc +* @options: the new/amended encryption options +* @force: hint for the driver to allow unsafe operation +* @errp: error pointer +* +* Changes the crypto options of the encryption format +* +*/ +int qcrypto_block_amend_options (QCryptoBlock *block, +QCryptoBlockReadFunc readfunc, +QCryptoBlockWriteFunc writefunc, +void *opaque, +QCryptoBlockAmendOptions *options, +bool force, +Error **errp); +/** * qcrypto_block_calculate_payload_offset:
block/crypto.c +17 -0
@@ -202,6 +202,23 @@ block_crypto_create_opts_init(QDict *opts, Error **errp)return ret; } +QCryptoBlockAmendOptions * +block_crypto_amend_opts_init (QDict *opts,Error **errp) +{ +Visitor *v; +QCryptoBlockAmendOptions *ret; + + v =qobject_input_visitor_new_flat_confused (opts, errp); +if (!v) { +return NULL ; + } + +visit_type_QCryptoBlockAmendOptions (v,NULL , &ret, errp); + +visit_free (v); +return ret; +} + static int block_crypto_open_generic(QCryptoBlockFormat format, QemuOptsList *opts_spec,
crypto/block.c +29 -0
@@ -150,6 +150,35 @@ qcrypto_block_calculate_payload_offset(QCryptoBlockCreateOptions *create_opts,return crypto !=NULL ; } +int qcrypto_block_amend_options (QCryptoBlock *block, +QCryptoBlockReadFunc readfunc, +QCryptoBlockWriteFunc writefunc, +void *opaque, +QCryptoBlockAmendOptions *options, +bool force, +Error **errp) +{ +if (options->format != block->format) { +error_setg (errp, +"Cannot amend encryption format" ); +return -1 ; + } + +if (!block->driver->amend) { +error_setg (errp, +"Crypto format %s doesn't support format options amendment" , +QCryptoBlockFormat_str (block->format)); +return -1 ; + } + +return block->driver->amend (block, + readfunc, + writefunc, + opaque, + options, + force, + errp); +}QCryptoBlockInfo *qcrypto_block_get_info (QCryptoBlock *block,Error **errp)
Signature · 1 line
2.26.2
PATCH 8/19 qcrypto/luks: implement encryption key management
In reply to Max ReitzFrom: Maxim Levitsky <[email protected]>
Next few patches will expose that functionality to the user.
Message-Id: <[email protected]>
--- qapi/crypto.json | 59 ++++++- crypto/block-luks.c | 416 +++++++++++++++++++++++++++++++++++++++++++- 2 files changed, 469 insertions(+), 6 deletions(-)
qapi/crypto.json +57 -2
@@ -297,7 +297,6 @@ 'uuid': 'str', 'slots': [ 'QCryptoBlockInfoLUKSSlot' ] }} - ## # @QCryptoBlockInfo: #
@@ -310,7 +309,63 @@ 'discriminator': 'format', 'data': { 'luks': 'QCryptoBlockInfoLUKS' } } +## +# @QCryptoBlockLUKSKeyslotState: +# +# Defines state of keyslots that are affected by the update +# +# @active: The slots contain the given password and marked as active +# @inactive: The slots are erased (contain garbage) and marked as inactive +# +# Since: 5.1 +## +{ 'enum': 'QCryptoBlockLUKSKeyslotState', + 'data': [ 'active', 'inactive' ] } + +## +# @QCryptoBlockAmendOptionsLUKS: +# +# This struct defines the update parameters that activate/de-activate set +# of keyslots +# +# @state: the desired state of the keyslots +# +# @new-secret: The ID of a QCryptoSecret object providing the password to be +# written into added active keyslots +# +# @old-secret: Optional (for deactivation only) +# If given will deactive all keyslots that +# match password located in QCryptoSecret with this ID +# +# @iter-time: Optional (for activation only) +# Number of milliseconds to spend in +# PBKDF passphrase processing for the newly activated keyslot. +# Currently defaults to 2000. +# +# @keyslot: Optional. ID of the keyslot to activate/deactivate. +# For keyslot activation, keyslot should not be active already +# (this is unsafe to update an active keyslot), +# but possible if 'force' parameter is given. +# If keyslot is not given, first free keyslot will be written. +# +# For keyslot deactivation, this parameter specifies the exact +# keyslot to deactivate +# +# @secret: Optional. The ID of a QCryptoSecret object providing the +# password to use to retrive current master key. +# Defaults to the same secret that was used to open the image +# +# +# Since 5.1 +## +{ 'struct': 'QCryptoBlockAmendOptionsLUKS', + 'data': { 'state': 'QCryptoBlockLUKSKeyslotState', + '*new-secret': 'str', + '*old-secret': 'str', + '*keyslot': 'int', + '*iter-time': 'int', + '*secret': 'str' } } ## # @QCryptoBlockAmendOptions:
@@ -324,4 +379,4 @@ 'base': 'QCryptoBlockOptionsBase', 'discriminator': 'format', 'data': { - } } + 'luks': 'QCryptoBlockAmendOptionsLUKS' } }
crypto/block-luks.c +412 -4
510 lines not on this page · raw message
Signature · 1 line
2.26.2
PATCH 9/19 block/amend: add 'force' option
In reply to Max ReitzFrom: Maxim Levitsky <[email protected]>
'force' option will be used for some unsafe amend operations.
This includes things like erasing last keyslot in luks based formats (which destroys the data, unless the master key is backed up by external means), but that _might_ be desired result.
Message-Id: <[email protected]>
--- docs/tools/qemu-img.rst | 5 ++++- include/block/block.h | 1 + include/block/block_int.h | 1 + block.c | 4 +++- block/qcow2.c | 1 + qemu-img.c | 8 +++++++- qemu-img-cmds.hx | 4 ++-- 7 files changed, 19 insertions(+), 5 deletions(-)
docs/tools/qemu-img.rst +4 -1
@@ -253,11 +253,14 @@ Command description: .. program:: qemu-img-commands -.. option:: amend [--object OBJECTDEF] [--image-opts] [-p] [-q] [-f FMT] [-t CACHE] -o OPTIONS FILENAME +.. option:: amend [--object OBJECTDEF] [--image-opts] [-p] [-q] [-f FMT] [-t CACHE] [--force] -o OPTIONS FILENAME Amends the image format specific *OPTIONS* for the image file *FILENAME*. Not all file formats support this operation. + --force allows some unsafe operations. Currently for -f luks, it allows to + erase the last encryption key, and to overwrite an active encryption key. + .. option:: bench [-c COUNT] [-d DEPTH] [-f FMT] [--flush-interval=FLUSH_INTERVAL] [-i AIO] [-n] [--no-drain] [-o OFFSET] [--pattern=PATTERN] [-q] [-s BUFFER_SIZE] [-S STEP_SIZE] [-t CACHE] [-w] [-U] FILENAME Run a simple sequential I/O benchmark on the specified image. If ``-w`` is
include/block/block.h +1 -0
@@ -450,6 +450,7 @@ typedef void BlockDriverAmendStatusCB(BlockDriverState *bs, int64_t offset, int64_t total_work_size, void *opaque);int bdrv_amend_options (BlockDriverState *bs_new,QemuOpts *opts,BlockDriverAmendStatusCB *status_cb,void *cb_opaque, +bool force,Error **errp);/* check if a named node can be replaced when doing drive-mirror */
include/block/block_int.h +1 -0
@@ -440,6 +440,7 @@ struct BlockDriver {int (*bdrv_amend_options)(BlockDriverState *bs,QemuOpts *opts,BlockDriverAmendStatusCB *status_cb,void *cb_opaque, +bool force,Error **errp);void (*bdrv_debug_event)(BlockDriverState *bs,BlkdebugEvent event);
block.c +3 -1
@@ -6482,6 +6482,7 @@ void bdrv_remove_aio_context_notifier(BlockDriverState *bs,int bdrv_amend_options (BlockDriverState *bs,QemuOpts *opts,BlockDriverAmendStatusCB *status_cb,void *cb_opaque, +bool force,Error **errp) {if (!bs->drv) {
@@ -6493,7 +6494,8 @@ int bdrv_amend_options(BlockDriverState *bs, QemuOpts *opts, bs->drv->format_name);return -ENOTSUP ; } -return bs->drv->bdrv_amend_options (bs, opts, status_cb, cb_opaque, errp); +return bs->drv->bdrv_amend_options (bs, opts, status_cb, + cb_opaque, force, errp); }/*
block/qcow2.c +1 -0
@@ -5340,6 +5340,7 @@ static void qcow2_amend_helper_cb(BlockDriverState *bs,static int qcow2_amend_options (BlockDriverState *bs,QemuOpts *opts,BlockDriverAmendStatusCB *status_cb,void *cb_opaque, +bool force,Error **errp) {BDRVQcow2State *s = bs->opaque;
qemu-img.c +7 -1
@@ -79,6 +79,7 @@ enum {OPTION_DISABLE =273 ,OPTION_MERGE =274 ,OPTION_BITMAPS =275 , + OPTION_FORCE = 276, }; typedef enum OutputFormat {
@@ -4099,6 +4100,7 @@ static int img_amend(int argc, char **argv)BlockBackend *blk =NULL ;BlockDriverState *bs =NULL ;bool image_opts = false; +bool force = false; cache =BDRV_DEFAULT_CACHE ;for (;;) {
@@ -4106,6 +4108,7 @@ static int img_amend(int argc, char **argv) {"help", no_argument, 0, 'h'}, {"object", required_argument, 0, OPTION_OBJECT}, {"image-opts", no_argument, 0, OPTION_IMAGE_OPTS}, + {"force", no_argument, 0, OPTION_FORCE}, {0, 0, 0, 0} }; c = getopt_long(argc, argv, ":ho:f:t:pq",
@@ -4153,6 +4156,9 @@ static int img_amend(int argc, char **argv)case OPTION_IMAGE_OPTS : image_opts = true;break ; +case OPTION_FORCE : + force = true; +break ; } }
@@ -4230,7 +4236,7 @@ static int img_amend(int argc, char **argv)/* In case the driver does not call amend_status_cb() */ qemu_progress_print (0.f ,0 ); - ret =bdrv_amend_options (bs, opts, &amend_status_cb,NULL , &err); + ret =bdrv_amend_options (bs, opts, &amend_status_cb,NULL , force, &err);qemu_progress_print (100.f ,0 );if (ret <0 ) {error_report_err (err);
qemu-img-cmds.hx +2 -2
@@ -10,9 +10,9 @@ HXCOMM When amending the rST sections, please remember to copy the usage HXCOMM over to the per-command sections in docs/tools/qemu-img.rst. DEF("amend", img_amend, - "amend [--object objectdef] [--image-opts] [-p] [-q] [-f fmt] [-t cache] -o options filename") + "amend [--object objectdef] [--image-opts] [-p] [-q] [-f fmt] [-t cache] [--force] -o options filename") SRST -.. option:: amend [--object OBJECTDEF] [--image-opts] [-p] [-q] [-f FMT] [-t CACHE] -o OPTIONS FILENAME +.. option:: amend [--object OBJECTDEF] [--image-opts] [-p] [-q] [-f FMT] [-t CACHE] [--force] -o OPTIONS FILENAME ERST DEF("bench", img_bench,
Signature · 1 line
2.26.2
PATCH 10/19 block/amend: separate amend and create options for qemu-img
In reply to Max ReitzFrom: Maxim Levitsky <[email protected]>
Some options are only useful for creation
(or hard to be amended, like cluster size for qcow2), while some other options are only useful for amend, like upcoming keyslot management options for luks
Since currently only qcow2 supports amend, move all its options to a common macro and then include it in each action option list.
In future it might be useful to remove some options which are not supported anyway from amend list, which currently cause an error message if amended.
Message-Id: <[email protected]>
--- include/block/block_int.h | 4 + block/qcow2.c | 173 +++++++++++++++++++++----------------- qemu-img.c | 18 ++-- 3 files changed, 107 insertions(+), 88 deletions(-)
include/block/block_int.h +4 -0
@@ -420,6 +420,10 @@ struct BlockDriver {/* List of options for creating images, terminated by name == NULL */ QemuOptsList *create_opts; + +/* List of options for image amend */ +QemuOptsList *amend_opts; +/* * If this driver supports reopening images this contains a * NULL-terminated list of the runtime options that can be
block/qcow2.c +94 -79
189 lines not on this page · raw message
qemu-img.c +9 -9
@@ -4077,11 +4077,11 @@ static int print_amend_option_help(const char *format)return 1 ; } -/* Every driver supporting amendment must have create_opts */ -assert (drv->create_opts); +/* Every driver supporting amendment must have amend_opts */ +assert (drv->amend_opts);printf ("Creation options for '%s':\n" , format); -qemu_opts_print_help (drv->create_opts, false); +qemu_opts_print_help (drv->amend_opts, false);printf ("\nNote that not all of these options may be amendable.\n" );return 0 ; }
@@ -4091,7 +4091,7 @@ static int img_amend(int argc, char **argv)Error *err =NULL ;int c, ret =0 ;char *options =NULL ; -QemuOptsList *create_opts =NULL ; +QemuOptsList *amend_opts =NULL ;QemuOpts *opts =NULL ;const char *fmt =NULL , *filename, *cache;int flags;
@@ -4222,11 +4222,11 @@ static int img_amend(int argc, char **argv) goto out; } -/* Every driver supporting amendment must have create_opts */ -assert (bs->drv->create_opts); +/* Every driver supporting amendment must have amend_opts */ +assert (bs->drv->amend_opts); - create_opts =qemu_opts_append (create_opts, bs->drv->create_opts); - opts =qemu_opts_create (create_opts,NULL ,0 , &error_abort); + amend_opts =qemu_opts_append (amend_opts, bs->drv->amend_opts); + opts =qemu_opts_create (amend_opts,NULL ,0 , &error_abort);qemu_opts_do_parse (opts, options,NULL , &err);if (err) {error_report_err (err);
@@ -4249,7 +4249,7 @@ out: out_no_progress:blk_unref (blk);qemu_opts_del (opts); -qemu_opts_free (create_opts); +qemu_opts_free (amend_opts);g_free (options);if (ret) {
Signature · 1 line
2.26.2
PATCH 11/19 block/amend: refactor qcow2 amend options
In reply to Max ReitzFrom: Maxim Levitsky <[email protected]>
Some qcow2 create options can't be used for amend.
Remove them from the qcow2 create options and add generic logic to detect such options in qemu-img
[mreitz: Dropped some iotests reference output hunks that became
unnecessary thanks to
"iotests: Make _filter_img_create more active"]
--- block/qcow2.c | 138 +++++++++----------------------- qemu-img.c | 18 ++++- tests/qemu-iotests/049.out | 102 ++++++++++++------------ tests/qemu-iotests/061.out | 12 ++- tests/qemu-iotests/082.out | 158 ++++--------------------------------- tests/qemu-iotests/085.out | 38 ++++----- tests/qemu-iotests/144.out | 4 +- tests/qemu-iotests/182.out | 2 +- tests/qemu-iotests/185.out | 8 +- tests/qemu-iotests/255.out | 8 +- tests/qemu-iotests/274.out | 46 +++++------ tests/qemu-iotests/280.out | 2 +- 12 files changed, 183 insertions(+), 353 deletions(-)
block/qcow2.c +38 -100
@@ -3042,17 +3042,6 @@ static int qcow2_change_backing_file(BlockDriverState *bs,return qcow2_update_header (bs); } -static int qcow2_crypt_method_from_format (const char *encryptfmt) -{ -if (g_str_equal (encryptfmt,"luks" )) { -return QCOW_CRYPT_LUKS ; - }else if (g_str_equal (encryptfmt,"aes" )) { -return QCOW_CRYPT_AES ; - }else { -return -EINVAL ; - } -} -static int qcow2_set_up_encryption (BlockDriverState *bs,QCryptoBlockCreateOptions *cryptoopts,Error **errp)
@@ -5350,9 +5339,6 @@ static int qcow2_amend_options(BlockDriverState *bs, QemuOpts *opts,bool lazy_refcounts = s->use_lazy_refcounts;bool data_file_raw =data_file_is_raw (bs);const char *compat =NULL ; -uint64_t cluster_size = s->cluster_size; -bool encrypt; -int encformat;int refcount_bits = s->refcount_bits;int ret;QemuOptDesc *desc = opts->list->desc;
@@ -5377,44 +5363,12 @@ static int qcow2_amend_options(BlockDriverState *bs, QemuOpts *opts,error_setg (errp,"Unknown compatibility level %s" , compat);return -EINVAL ; } - }else if (!strcmp (desc->name,BLOCK_OPT_PREALLOC )) { -error_setg (errp,"Cannot change preallocation mode" ); -return -ENOTSUP ; }else if (!strcmp (desc->name,BLOCK_OPT_SIZE )) { new_size =qemu_opt_get_size (opts,BLOCK_OPT_SIZE ,0 ); }else if (!strcmp (desc->name,BLOCK_OPT_BACKING_FILE )) { backing_file =qemu_opt_get (opts,BLOCK_OPT_BACKING_FILE ); }else if (!strcmp (desc->name,BLOCK_OPT_BACKING_FMT )) { backing_format =qemu_opt_get (opts,BLOCK_OPT_BACKING_FMT ); - }else if (!strcmp (desc->name,BLOCK_OPT_ENCRYPT )) { - encrypt =qemu_opt_get_bool (opts,BLOCK_OPT_ENCRYPT , - !!s->crypto); - -if (encrypt != !!s->crypto) { -error_setg (errp, -"Changing the encryption flag is not supported" ); -return -ENOTSUP ; - } - }else if (!strcmp (desc->name,BLOCK_OPT_ENCRYPT_FORMAT )) { - encformat =qcow2_crypt_method_from_format ( -qemu_opt_get (opts,BLOCK_OPT_ENCRYPT_FORMAT )); - -if (encformat != s->crypt_method_header) { -error_setg (errp, -"Changing the encryption format is not supported" ); -return -ENOTSUP ; - } - }else if (g_str_has_prefix (desc->name,"encrypt." )) { -error_setg (errp, -"Changing the encryption parameters is not supported" ); -return -ENOTSUP ; - }else if (!strcmp (desc->name,BLOCK_OPT_CLUSTER_SIZE )) { - cluster_size =qemu_opt_get_size (opts,BLOCK_OPT_CLUSTER_SIZE , - cluster_size); -if (cluster_size != s->cluster_size) { -error_setg (errp,"Changing the cluster size is not supported" ); -return -ENOTSUP ; - } }else if (!strcmp (desc->name,BLOCK_OPT_LAZY_REFCOUNTS )) { lazy_refcounts =qemu_opt_get_bool (opts,BLOCK_OPT_LAZY_REFCOUNTS , lazy_refcounts);
@@ -5444,22 +5398,6 @@ static int qcow2_amend_options(BlockDriverState *bs, QemuOpts *opts, "images");return -EINVAL ; } - }else if (!strcmp (desc->name,BLOCK_OPT_COMPRESSION_TYPE )) { -const char *ct_name = -qemu_opt_get (opts,BLOCK_OPT_COMPRESSION_TYPE ); -int compression_type = -qapi_enum_parse (&Qcow2CompressionType_lookup, ct_name,-1 , -NULL ); -if (compression_type ==-1 ) { -error_setg (errp,"Unknown compression type: %s" , ct_name); -return -ENOTSUP ; - } - -if (compression_type != s->compression_type) { -error_setg (errp,"Changing the compression type " -"is not supported" ); -return -ENOTSUP ; - } }else {/* if this point is reached, this probably means a new option was * added without having it covered here */
@@ -5681,37 +5619,6 @@ void qcow2_signal_corruption(BlockDriverState *bs, bool fatal, int64_t offset, .help = "The external data file must stay valid " \"as a raw image" \ }, \ - { \ - .name = BLOCK_OPT_ENCRYPT, \ - .type = QEMU_OPT_BOOL, \ - .help ="Encrypt the image with format 'aes'. (Deprecated " \ - "in favor of " BLOCK_OPT_ENCRYPT_FORMAT "=aes)", \ - }, \ - { \ - .name = BLOCK_OPT_ENCRYPT_FORMAT, \ - .type = QEMU_OPT_STRING, \ - .help ="Encrypt the image, format choices: 'aes', 'luks'" , \ - }, \ -BLOCK_CRYPTO_OPT_DEF_KEY_SECRET ("encrypt." , \ -"ID of secret providing qcow AES key or LUKS passphrase" ), \ -BLOCK_CRYPTO_OPT_DEF_LUKS_CIPHER_ALG ("encrypt." ), \ -BLOCK_CRYPTO_OPT_DEF_LUKS_CIPHER_MODE ("encrypt." ), \ -BLOCK_CRYPTO_OPT_DEF_LUKS_IVGEN_ALG ("encrypt." ), \ -BLOCK_CRYPTO_OPT_DEF_LUKS_IVGEN_HASH_ALG ("encrypt." ), \ -BLOCK_CRYPTO_OPT_DEF_LUKS_HASH_ALG ("encrypt." ), \ - BLOCK_CRYPTO_OPT_DEF_LUKS_ITER_TIME("encrypt."), \ - { \ - .name =BLOCK_OPT_CLUSTER_SIZE , \ - .type = QEMU_OPT_SIZE, \ - .help = "qcow2 cluster size", \ - .def_value_str = stringify(DEFAULT_CLUSTER_SIZE) \ - }, \ - { \ - .name =BLOCK_OPT_PREALLOC , \ - .type = QEMU_OPT_STRING, \ - .help = "Preallocation mode (allowed values: off, " \ -"metadata, falloc, full)" \ - }, \ { \ .name = BLOCK_OPT_LAZY_REFCOUNTS, \ .type = QEMU_OPT_BOOL, \
@@ -5723,19 +5630,50 @@ void qcow2_signal_corruption(BlockDriverState *bs, bool fatal, int64_t offset, .type = QEMU_OPT_NUMBER, \ .help = "Width of a reference count entry in bits", \ .def_value_str = "16" \ - }, \ - { \ - .name = BLOCK_OPT_COMPRESSION_TYPE, \ - .type = QEMU_OPT_STRING, \ - .help ="Compression method used for image cluster " \ - "compression", \ - .def_value_str = "zlib" \ }static QemuOptsList qcow2_create_opts = { .name ="qcow2-create-opts" , .head =QTAILQ_HEAD_INITIALIZER (qcow2_create_opts.head), .desc = { + { \ + .name =BLOCK_OPT_ENCRYPT , \ + .type =QEMU_OPT_BOOL , \ + .help ="Encrypt the image with format 'aes'. (Deprecated " \ +"in favor of " BLOCK_OPT_ENCRYPT_FORMAT "=aes)" , \ + }, \ + { \ + .name =BLOCK_OPT_ENCRYPT_FORMAT , \ + .type =QEMU_OPT_STRING , \ + .help ="Encrypt the image, format choices: 'aes', 'luks'" , \ + }, \ +BLOCK_CRYPTO_OPT_DEF_KEY_SECRET ("encrypt." , \ +"ID of secret providing qcow AES key or LUKS passphrase" ), \ +BLOCK_CRYPTO_OPT_DEF_LUKS_CIPHER_ALG ("encrypt." ), \ +BLOCK_CRYPTO_OPT_DEF_LUKS_CIPHER_MODE ("encrypt." ), \ +BLOCK_CRYPTO_OPT_DEF_LUKS_IVGEN_ALG ("encrypt." ), \ +BLOCK_CRYPTO_OPT_DEF_LUKS_IVGEN_HASH_ALG ("encrypt." ), \ +BLOCK_CRYPTO_OPT_DEF_LUKS_HASH_ALG ("encrypt." ), \ +BLOCK_CRYPTO_OPT_DEF_LUKS_ITER_TIME ("encrypt." ), \ + { \ + .name =BLOCK_OPT_CLUSTER_SIZE , \ + .type =QEMU_OPT_SIZE , \ + .help ="qcow2 cluster size" , \ + .def_value_str =stringify (DEFAULT_CLUSTER_SIZE ) \ + }, \ + { \ + .name =BLOCK_OPT_PREALLOC , \ + .type =QEMU_OPT_STRING , \ + .help ="Preallocation mode (allowed values: off, " \ +"metadata, falloc, full)" \ + }, \ + { \ + .name =BLOCK_OPT_COMPRESSION_TYPE , \ + .type =QEMU_OPT_STRING , \ + .help ="Compression method used for image cluster " \ +"compression" , \ + .def_value_str ="zlib" \ + },QCOW_COMMON_OPTIONS , {/* end of list */ } }
qemu-img.c +16 -2
@@ -4080,9 +4080,8 @@ static int print_amend_option_help(const char *format)/* Every driver supporting amendment must have amend_opts */ assert (drv->amend_opts); -printf ("Creation options for '%s':\n" , format); +printf ("Amend options for '%s':\n" , format);qemu_opts_print_help (drv->amend_opts, false); -printf ("\nNote that not all of these options may be amendable.\n" );return 0 ; }
@@ -4228,7 +4227,22 @@ static int img_amend(int argc, char **argv) amend_opts =qemu_opts_append (amend_opts, bs->drv->amend_opts); opts =qemu_opts_create (amend_opts,NULL ,0 , &error_abort);qemu_opts_do_parse (opts, options,NULL , &err); +if (err) { +/* Try to parse options using the create options */ +Error *err1 =NULL ; + amend_opts =qemu_opts_append (amend_opts, bs->drv->create_opts); +qemu_opts_del (opts); + opts =qemu_opts_create (amend_opts,NULL ,0 , &error_abort); +qemu_opts_do_parse (opts, options,NULL , &err1); + +if (!err1) { +error_append_hint (&err, +"This option is only supported for image creation\n" ); + }else { +error_free (err1); + } +error_report_err (err); ret =-1 ; goto out;
tests/qemu-iotests/049.out +51 -51
225 lines not on this page · raw message
tests/qemu-iotests/061.out +8 -4
@@ -381,16 +381,20 @@ qemu-img: Lazy refcounts only supported with compatibility level 1.1 and above ( qemu-img: Lazy refcounts only supported with compatibility level 1.1 and above (use compat=1.1 or greater) qemu-img: Unknown compatibility level 0.42 qemu-img: Invalid parameter 'foo' -qemu-img: Changing the cluster size is not supported -qemu-img: Changing the encryption flag is not supported -qemu-img: Cannot change preallocation mode +qemu-img: Invalid parameter 'cluster_size' +This option is only supported for image creation +qemu-img: Invalid parameter 'encryption' +This option is only supported for image creation +qemu-img: Invalid parameter 'preallocation' +This option is only supported for image creation === Testing correct handling of unset value === Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864 Should work: Should not work: -qemu-img: Changing the cluster size is not supported +qemu-img: Invalid parameter 'cluster_size' +This option is only supported for image creation === Testing zero expansion on inactive clusters ===
tests/qemu-iotests/082.out +16 -142
296 lines not on this page · raw message
tests/qemu-iotests/085.out +19 -19
@@ -13,7 +13,7 @@ Formatting 'TEST_DIR/t.IMGFMT.2', fmt=IMGFMT size=134217728 === Create a single snapshot on virtio0 === { 'execute': 'blockdev-snapshot-sync', 'arguments': { 'device': 'virtio0', 'snapshot-file':'TEST_DIR/1-snapshot-v0.IMGFMT', 'format': 'IMGFMT' } } -Formatting 'TEST_DIR/1-snapshot-v0.qcow2', fmt=qcow2 size=134217728 backing_file=TEST_DIR/t.qcow2.1 backing_fmt=qcow2 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib +Formatting 'TEST_DIR/1-snapshot-v0.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=134217728 backing_file=TEST_DIR/t.qcow2.1 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16 {"return": {}} === Invalid command - missing device and nodename ===
@@ -30,40 +30,40 @@ Formatting 'TEST_DIR/1-snapshot-v0.qcow2', fmt=qcow2 size=134217728 backing_file === Create several transactional group snapshots === { 'execute': 'transaction', 'arguments': {'actions': [ { 'type': 'blockdev-snapshot-sync', 'data' : { 'device': 'virtio0', 'snapshot-file': 'TEST_DIR/2-snapshot-v0.IMGFMT' } }, { 'type': 'blockdev-snapshot-sync', 'data' : { 'device': 'virtio1', 'snapshot-file': 'TEST_DIR/2-snapshot-v1.IMGFMT' } } ] } } -Formatting 'TEST_DIR/2-snapshot-v0.qcow2', fmt=qcow2 size=134217728 backing_file=TEST_DIR/1-snapshot-v0.qcow2 backing_fmt=qcow2 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib -Formatting 'TEST_DIR/2-snapshot-v1.qcow2', fmt=qcow2 size=134217728 backing_file=TEST_DIR/t.qcow2.2 backing_fmt=qcow2 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib +Formatting 'TEST_DIR/2-snapshot-v0.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=134217728 backing_file=TEST_DIR/1-snapshot-v0.qcow2 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/2-snapshot-v1.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=134217728 backing_file=TEST_DIR/t.qcow2.2 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16 {"return": {}} { 'execute': 'transaction', 'arguments': {'actions': [ { 'type': 'blockdev-snapshot-sync', 'data' : { 'device': 'virtio0', 'snapshot-file': 'TEST_DIR/3-snapshot-v0.IMGFMT' } }, { 'type': 'blockdev-snapshot-sync', 'data' : { 'device': 'virtio1', 'snapshot-file': 'TEST_DIR/3-snapshot-v1.IMGFMT' } } ] } } -Formatting 'TEST_DIR/3-snapshot-v0.qcow2', fmt=qcow2 size=134217728 backing_file=TEST_DIR/2-snapshot-v0.qcow2 backing_fmt=qcow2 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib -Formatting 'TEST_DIR/3-snapshot-v1.qcow2', fmt=qcow2 size=134217728 backing_file=TEST_DIR/2-snapshot-v1.qcow2 backing_fmt=qcow2 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib +Formatting 'TEST_DIR/3-snapshot-v0.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=134217728 backing_file=TEST_DIR/2-snapshot-v0.qcow2 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/3-snapshot-v1.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=134217728 backing_file=TEST_DIR/2-snapshot-v1.qcow2 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16 {"return": {}} { 'execute': 'transaction', 'arguments': {'actions': [ { 'type': 'blockdev-snapshot-sync', 'data' : { 'device': 'virtio0', 'snapshot-file': 'TEST_DIR/4-snapshot-v0.IMGFMT' } }, { 'type': 'blockdev-snapshot-sync', 'data' : { 'device': 'virtio1', 'snapshot-file': 'TEST_DIR/4-snapshot-v1.IMGFMT' } } ] } } -Formatting 'TEST_DIR/4-snapshot-v0.qcow2', fmt=qcow2 size=134217728 backing_file=TEST_DIR/3-snapshot-v0.qcow2 backing_fmt=qcow2 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib -Formatting 'TEST_DIR/4-snapshot-v1.qcow2', fmt=qcow2 size=134217728 backing_file=TEST_DIR/3-snapshot-v1.qcow2 backing_fmt=qcow2 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib +Formatting 'TEST_DIR/4-snapshot-v0.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=134217728 backing_file=TEST_DIR/3-snapshot-v0.qcow2 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/4-snapshot-v1.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=134217728 backing_file=TEST_DIR/3-snapshot-v1.qcow2 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16 {"return": {}} { 'execute': 'transaction', 'arguments': {'actions': [ { 'type': 'blockdev-snapshot-sync', 'data' : { 'device': 'virtio0', 'snapshot-file': 'TEST_DIR/5-snapshot-v0.IMGFMT' } }, { 'type': 'blockdev-snapshot-sync', 'data' : { 'device': 'virtio1', 'snapshot-file': 'TEST_DIR/5-snapshot-v1.IMGFMT' } } ] } } -Formatting 'TEST_DIR/5-snapshot-v0.qcow2', fmt=qcow2 size=134217728 backing_file=TEST_DIR/4-snapshot-v0.qcow2 backing_fmt=qcow2 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib -Formatting 'TEST_DIR/5-snapshot-v1.qcow2', fmt=qcow2 size=134217728 backing_file=TEST_DIR/4-snapshot-v1.qcow2 backing_fmt=qcow2 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib +Formatting 'TEST_DIR/5-snapshot-v0.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=134217728 backing_file=TEST_DIR/4-snapshot-v0.qcow2 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/5-snapshot-v1.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=134217728 backing_file=TEST_DIR/4-snapshot-v1.qcow2 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16 {"return": {}} { 'execute': 'transaction', 'arguments': {'actions': [ { 'type': 'blockdev-snapshot-sync', 'data' : { 'device': 'virtio0', 'snapshot-file': 'TEST_DIR/6-snapshot-v0.IMGFMT' } }, { 'type': 'blockdev-snapshot-sync', 'data' : { 'device': 'virtio1', 'snapshot-file': 'TEST_DIR/6-snapshot-v1.IMGFMT' } } ] } } -Formatting 'TEST_DIR/6-snapshot-v0.qcow2', fmt=qcow2 size=134217728 backing_file=TEST_DIR/5-snapshot-v0.qcow2 backing_fmt=qcow2 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib -Formatting 'TEST_DIR/6-snapshot-v1.qcow2', fmt=qcow2 size=134217728 backing_file=TEST_DIR/5-snapshot-v1.qcow2 backing_fmt=qcow2 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib +Formatting 'TEST_DIR/6-snapshot-v0.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=134217728 backing_file=TEST_DIR/5-snapshot-v0.qcow2 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/6-snapshot-v1.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=134217728 backing_file=TEST_DIR/5-snapshot-v1.qcow2 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16 {"return": {}} { 'execute': 'transaction', 'arguments': {'actions': [ { 'type': 'blockdev-snapshot-sync', 'data' : { 'device': 'virtio0', 'snapshot-file': 'TEST_DIR/7-snapshot-v0.IMGFMT' } }, { 'type': 'blockdev-snapshot-sync', 'data' : { 'device': 'virtio1', 'snapshot-file': 'TEST_DIR/7-snapshot-v1.IMGFMT' } } ] } } -Formatting 'TEST_DIR/7-snapshot-v0.qcow2', fmt=qcow2 size=134217728 backing_file=TEST_DIR/6-snapshot-v0.qcow2 backing_fmt=qcow2 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib -Formatting 'TEST_DIR/7-snapshot-v1.qcow2', fmt=qcow2 size=134217728 backing_file=TEST_DIR/6-snapshot-v1.qcow2 backing_fmt=qcow2 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib +Formatting 'TEST_DIR/7-snapshot-v0.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=134217728 backing_file=TEST_DIR/6-snapshot-v0.qcow2 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/7-snapshot-v1.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=134217728 backing_file=TEST_DIR/6-snapshot-v1.qcow2 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16 {"return": {}} { 'execute': 'transaction', 'arguments': {'actions': [ { 'type': 'blockdev-snapshot-sync', 'data' : { 'device': 'virtio0', 'snapshot-file': 'TEST_DIR/8-snapshot-v0.IMGFMT' } }, { 'type': 'blockdev-snapshot-sync', 'data' : { 'device': 'virtio1', 'snapshot-file': 'TEST_DIR/8-snapshot-v1.IMGFMT' } } ] } } -Formatting 'TEST_DIR/8-snapshot-v0.qcow2', fmt=qcow2 size=134217728 backing_file=TEST_DIR/7-snapshot-v0.qcow2 backing_fmt=qcow2 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib -Formatting 'TEST_DIR/8-snapshot-v1.qcow2', fmt=qcow2 size=134217728 backing_file=TEST_DIR/7-snapshot-v1.qcow2 backing_fmt=qcow2 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib +Formatting 'TEST_DIR/8-snapshot-v0.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=134217728 backing_file=TEST_DIR/7-snapshot-v0.qcow2 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/8-snapshot-v1.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=134217728 backing_file=TEST_DIR/7-snapshot-v1.qcow2 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16 {"return": {}} { 'execute': 'transaction', 'arguments': {'actions': [ { 'type': 'blockdev-snapshot-sync', 'data' : { 'device': 'virtio0', 'snapshot-file': 'TEST_DIR/9-snapshot-v0.IMGFMT' } }, { 'type': 'blockdev-snapshot-sync', 'data' : { 'device': 'virtio1', 'snapshot-file': 'TEST_DIR/9-snapshot-v1.IMGFMT' } } ] } } -Formatting 'TEST_DIR/9-snapshot-v0.qcow2', fmt=qcow2 size=134217728 backing_file=TEST_DIR/8-snapshot-v0.qcow2 backing_fmt=qcow2 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib -Formatting 'TEST_DIR/9-snapshot-v1.qcow2', fmt=qcow2 size=134217728 backing_file=TEST_DIR/8-snapshot-v1.qcow2 backing_fmt=qcow2 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib +Formatting 'TEST_DIR/9-snapshot-v0.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=134217728 backing_file=TEST_DIR/8-snapshot-v0.qcow2 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/9-snapshot-v1.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=134217728 backing_file=TEST_DIR/8-snapshot-v1.qcow2 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16 {"return": {}} { 'execute': 'transaction', 'arguments': {'actions': [ { 'type': 'blockdev-snapshot-sync', 'data' : { 'device': 'virtio0', 'snapshot-file': 'TEST_DIR/10-snapshot-v0.IMGFMT' } }, { 'type': 'blockdev-snapshot-sync', 'data' : { 'device': 'virtio1', 'snapshot-file': 'TEST_DIR/10-snapshot-v1.IMGFMT' } } ] } } -Formatting 'TEST_DIR/10-snapshot-v0.qcow2', fmt=qcow2 size=134217728 backing_file=TEST_DIR/9-snapshot-v0.qcow2 backing_fmt=qcow2 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib -Formatting 'TEST_DIR/10-snapshot-v1.qcow2', fmt=qcow2 size=134217728 backing_file=TEST_DIR/9-snapshot-v1.qcow2 backing_fmt=qcow2 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib +Formatting 'TEST_DIR/10-snapshot-v0.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=134217728 backing_file=TEST_DIR/9-snapshot-v0.qcow2 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16 +Formatting 'TEST_DIR/10-snapshot-v1.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=134217728 backing_file=TEST_DIR/9-snapshot-v1.qcow2 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16 {"return": {}} === Create a couple of snapshots using blockdev-snapshot ===
tests/qemu-iotests/144.out +2 -2
@@ -9,7 +9,7 @@ Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=536870912 { 'execute': 'qmp_capabilities' } {"return": {}} { 'execute': 'blockdev-snapshot-sync', 'arguments': { 'device': 'virtio0', 'snapshot-file':'TEST_DIR/tmp.IMGFMT', 'format': 'IMGFMT' } } -Formatting 'TEST_DIR/tmp.qcow2', fmt=qcow2 size=536870912 backing_file=TEST_DIR/t.qcow2 backing_fmt=qcow2 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib +Formatting 'TEST_DIR/tmp.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=536870912 backing_file=TEST_DIR/t.qcow2 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16 {"return": {}} === Performing block-commit on active layer ===
@@ -31,6 +31,6 @@ Formatting 'TEST_DIR/tmp.qcow2', fmt=qcow2 size=536870912 backing_file=TEST_DIR/ === Performing Live Snapshot 2 === { 'execute': 'blockdev-snapshot-sync', 'arguments': { 'device': 'virtio0', 'snapshot-file':'TEST_DIR/tmp2.IMGFMT', 'format': 'IMGFMT' } } -Formatting 'TEST_DIR/tmp2.qcow2', fmt=qcow2 size=536870912 backing_file=TEST_DIR/t.qcow2 backing_fmt=qcow2 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib +Formatting 'TEST_DIR/tmp2.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=536870912 backing_file=TEST_DIR/t.qcow2 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16 {"return": {}} *** done
tests/qemu-iotests/182.out +1 -1
@@ -13,7 +13,7 @@ Is another process using the image [TEST_DIR/t.qcow2]? {'execute': 'blockdev-add', 'arguments': { 'node-name': 'node0', 'driver': 'file', 'filename': 'TEST_DIR/t.IMGFMT', 'locking': 'on' } } {"return": {}} {'execute': 'blockdev-snapshot-sync', 'arguments': { 'node-name': 'node0', 'snapshot-file': 'TEST_DIR/t.IMGFMT.overlay', 'snapshot-node-name': 'node1' } } -Formatting 'TEST_DIR/t.qcow2.overlay', fmt=qcow2 size=197120 backing_file=TEST_DIR/t.qcow2 backing_fmt=file cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib +Formatting 'TEST_DIR/t.qcow2.overlay', fmt=qcow2 cluster_size=65536 compression_type=zlib size=197120 backing_file=TEST_DIR/t.qcow2 backing_fmt=file lazy_refcounts=off refcount_bits=16 {"return": {}} {'execute': 'blockdev-add', 'arguments': { 'node-name': 'node1', 'driver': 'file', 'filename': 'TEST_DIR/t.IMGFMT', 'locking': 'on' } } {"return": {}}
tests/qemu-iotests/185.out +4 -4
@@ -9,14 +9,14 @@ Formatting 'TEST_DIR/t.IMGFMT.base', fmt=IMGFMT size=67108864 === Creating backing chain === { 'execute': 'blockdev-snapshot-sync', 'arguments': { 'device': 'disk', 'snapshot-file': 'TEST_DIR/t.IMGFMT.mid', 'format': 'IMGFMT', 'mode': 'absolute-paths' } } -Formatting 'TEST_DIR/t.qcow2.mid', fmt=qcow2 size=67108864 backing_file=TEST_DIR/t.qcow2.base backing_fmt=qcow2 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib +Formatting 'TEST_DIR/t.qcow2.mid', fmt=qcow2 cluster_size=65536 compression_type=zlib size=67108864 backing_file=TEST_DIR/t.qcow2.base backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16 {"return": {}} { 'execute': 'human-monitor-command', 'arguments': { 'command-line': 'qemu-io disk "write 0 4M"' } } wrote 4194304/4194304 bytes at offset 0 4 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) {"return": ""} { 'execute': 'blockdev-snapshot-sync', 'arguments': { 'device': 'disk', 'snapshot-file': 'TEST_DIR/t.IMGFMT', 'format': 'IMGFMT', 'mode': 'absolute-paths' } } -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 backing_file=TEST_DIR/t.qcow2.mid backing_fmt=qcow2 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=67108864 backing_file=TEST_DIR/t.qcow2.mid backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16 {"return": {}} === Start commit job and exit qemu ===
@@ -48,7 +48,7 @@ Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 backing_file=TEST_DIR/t.q { 'execute': 'qmp_capabilities' } {"return": {}} { 'execute': 'drive-mirror', 'arguments': { 'device': 'disk', 'target': 'TEST_DIR/t.IMGFMT.copy', 'format': 'IMGFMT', 'sync': 'full', 'speed': 65536 } } -Formatting 'TEST_DIR/t.qcow2.copy', fmt=qcow2 size=67108864 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib +Formatting 'TEST_DIR/t.qcow2.copy', fmt=qcow2 cluster_size=65536 compression_type=zlib size=67108864 lazy_refcounts=off refcount_bits=16 {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "created", "id": "disk"}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "running", "id": "disk"}} {"return": {}}
@@ -62,7 +62,7 @@ Formatting 'TEST_DIR/t.qcow2.copy', fmt=qcow2 size=67108864 cluster_size=65536 l { 'execute': 'qmp_capabilities' } {"return": {}} { 'execute': 'drive-backup', 'arguments': { 'device': 'disk', 'target': 'TEST_DIR/t.IMGFMT.copy', 'format': 'IMGFMT', 'sync': 'full', 'speed': 65536 } } -Formatting 'TEST_DIR/t.qcow2.copy', fmt=qcow2 size=67108864 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib +Formatting 'TEST_DIR/t.qcow2.copy', fmt=qcow2 cluster_size=65536 compression_type=zlib size=67108864 lazy_refcounts=off refcount_bits=16 {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "created", "id": "disk"}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "running", "id": "disk"}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "paused", "id": "disk"}}
tests/qemu-iotests/255.out +4 -4
@@ -3,9 +3,9 @@ Finishing a commit job with background reads === Create backing chain and start VM === -Formatting 'TEST_DIR/PID-t.qcow2.mid', fmt=qcow2 size=134217728 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib +Formatting 'TEST_DIR/PID-t.qcow2.mid', fmt=qcow2 cluster_size=65536 compression_type=zlib size=134217728 lazy_refcounts=off refcount_bits=16 -Formatting 'TEST_DIR/PID-t.qcow2', fmt=qcow2 size=134217728 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib +Formatting 'TEST_DIR/PID-t.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=134217728 lazy_refcounts=off refcount_bits=16 === Start background read requests ===
@@ -23,9 +23,9 @@ Closing the VM while a job is being cancelled === Create images and start VM === -Formatting 'TEST_DIR/PID-src.qcow2', fmt=qcow2 size=134217728 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib +Formatting 'TEST_DIR/PID-src.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=134217728 lazy_refcounts=off refcount_bits=16 -Formatting 'TEST_DIR/PID-dst.qcow2', fmt=qcow2 size=134217728 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib +Formatting 'TEST_DIR/PID-dst.qcow2', fmt=qcow2 cluster_size=65536 compression_type=zlib size=134217728 lazy_refcounts=off refcount_bits=16 wrote 1048576/1048576 bytes at offset 0 1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
tests/qemu-iotests/274.out +23 -23
117 lines not on this page · raw message
tests/qemu-iotests/280.out +1 -1
@@ -1,4 +1,4 @@ -Formatting 'TEST_DIR/PID-base', fmt=qcow2 size=67108864 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib +Formatting 'TEST_DIR/PID-base', fmt=qcow2 cluster_size=65536 compression_type=zlib size=67108864 lazy_refcounts=off refcount_bits=16 === Launch VM === Enabling migration QMP events on VM...
Signature · 1 line
2.26.2
PATCH 12/19 block/crypto: rename two functions
In reply to Max ReitzFrom: Maxim Levitsky <[email protected]>
rename the write_func to create_write_func, and init_func to create_init_func. This is preparation for other write_func that will be used to update the encryption keys.
No functional changes
Message-Id: <[email protected]>
--- block/crypto.c | 25 ++++++++++++------------- 1 file changed, 12 insertions(+), 13 deletions(-)
block/crypto.c +12 -13
@@ -79,12 +79,12 @@ struct BlockCryptoCreateData { }; -static ssize_t block_crypto_write_func (QCryptoBlock *block, -size_t offset, -const uint8_t *buf, -size_t buflen, -void *opaque, -Error **errp) +static ssize_t block_crypto_create_write_func (QCryptoBlock *block, +size_t offset, +const uint8_t *buf, +size_t buflen, +void *opaque, +Error **errp) {struct BlockCryptoCreateData *data = opaque;ssize_t ret;
@@ -97,11 +97,10 @@ static ssize_t block_crypto_write_func(QCryptoBlock *block,return ret; } - -static ssize_t block_crypto_init_func (QCryptoBlock *block, -size_t headerlen, -void *opaque, -Error **errp) +static ssize_t block_crypto_create_init_func (QCryptoBlock *block, +size_t headerlen, +void *opaque, +Error **errp) {struct BlockCryptoCreateData *data = opaque;Error *local_error =NULL ;
@@ -313,8 +312,8 @@ static int block_crypto_co_create_generic(BlockDriverState *bs, }; crypto =qcrypto_block_create (opts,NULL , - block_crypto_init_func, - block_crypto_write_func, + block_crypto_create_init_func, + block_crypto_create_write_func, &data, errp);
Signature · 1 line
2.26.2
PATCH 13/19 block/crypto: implement the encryption key management
In reply to Max ReitzFrom: Maxim Levitsky <[email protected]>
This implements the encryption key management using the generic code in qcrypto layer and exposes it to the user via qemu-img
This code adds another 'write_func' because the initialization write_func works directly on the underlying file, and amend works on instance of luks device.
This commit also adds a 'hack/workaround' I and Kevin Wolf (thanks) made to make the driver both support write sharing (to avoid breaking the users), and be safe against concurrent metadata update (the keyslots)
Eventually the write sharing for luks driver will be deprecated and removed together with this hack.
The hack is that we ask (as a format driver) for BLK_PERM_CONSISTENT_READ and then when we want to update the keys, we unshare that permission. So if someone else has the image open, even readonly, encryption key update will fail gracefully.
Also thanks to Daniel Berrange for the idea of unsharing read, rather that write permission which allows to avoid cases when the other user had opened the image read-only.
Message-Id: <[email protected]>
--- block/crypto.h | 34 +++++++++++++ block/crypto.c | 130 +++++++++++++++++++++++++++++++++++++++++++++++-- 2 files changed, 161 insertions(+), 3 deletions(-)
block/crypto.h +34 -0
@@ -41,6 +41,11 @@#define BLOCK_CRYPTO_OPT_LUKS_IVGEN_HASH_ALG "ivgen-hash-alg"#define BLOCK_CRYPTO_OPT_LUKS_HASH_ALG "hash-alg"#define BLOCK_CRYPTO_OPT_LUKS_ITER_TIME "iter-time" +#define BLOCK_CRYPTO_OPT_LUKS_KEYSLOT "keyslot" +#define BLOCK_CRYPTO_OPT_LUKS_STATE "state" +#define BLOCK_CRYPTO_OPT_LUKS_OLD_SECRET "old-secret" +#define BLOCK_CRYPTO_OPT_LUKS_NEW_SECRET "new-secret" +#define BLOCK_CRYPTO_OPT_DEF_LUKS_KEY_SECRET (prefix) \ BLOCK_CRYPTO_OPT_DEF_KEY_SECRET(prefix, \
@@ -88,6 +93,35 @@ .help = "Time to spend in PBKDF in milliseconds", \ } +#define BLOCK_CRYPTO_OPT_DEF_LUKS_STATE (prefix) \ + { \ + .name = prefix BLOCK_CRYPTO_OPT_LUKS_STATE, \ + .type = QEMU_OPT_STRING, \ + .help = "Select new state of affected keyslots (active/inactive)",\ + } + +#define BLOCK_CRYPTO_OPT_DEF_LUKS_KEYSLOT (prefix) \ + { \ + .name = prefix BLOCK_CRYPTO_OPT_LUKS_KEYSLOT, \ + .type = QEMU_OPT_NUMBER, \ + .help = "Select a single keyslot to modify explicitly",\ + } + +#define BLOCK_CRYPTO_OPT_DEF_LUKS_OLD_SECRET (prefix) \ + { \ + .name = prefix BLOCK_CRYPTO_OPT_LUKS_OLD_SECRET, \ + .type = QEMU_OPT_STRING, \ + .help = "Select all keyslots that match this password", \ + } + +#define BLOCK_CRYPTO_OPT_DEF_LUKS_NEW_SECRET (prefix) \ + { \ + .name = prefix BLOCK_CRYPTO_OPT_LUKS_NEW_SECRET, \ + .type = QEMU_OPT_STRING, \ + .help = "New secret to set in the matching keyslots. " \ + "Empty string to erase", \ + } +QCryptoBlockCreateOptions *block_crypto_create_opts_init (QDict *opts,Error **errp);
block/crypto.c +127 -3
170 lines not on this page · raw message
Signature · 1 line
2.26.2
PATCH 14/19 block/qcow2: extend qemu-img amend interface with crypto options
In reply to Max ReitzFrom: Maxim Levitsky <[email protected]>
Now that we have all the infrastructure in place, wire it in the qcow2 driver and expose this to the user.
Message-Id: <[email protected]>
--- block/qcow2.c | 71 +++++++++++++++++++++++++++++++++----- tests/qemu-iotests/082.out | 45 ++++++++++++++++++++++++ 2 files changed, 107 insertions(+), 9 deletions(-)
block/qcow2.c +62 -9
@@ -176,6 +176,19 @@ static ssize_t qcow2_crypto_hdr_write_func(QCryptoBlock *block, size_t offset,return ret; } +static QDict * +qcow2_extract_crypto_opts (QemuOpts *opts,const char *fmt,Error **errp) +{ +QDict *cryptoopts_qdict; +QDict *opts_qdict; + +/* Extract "encrypt." options into a qdict */ + opts_qdict =qemu_opts_to_qdict (opts,NULL ); +qdict_extract_subqdict (opts_qdict, &cryptoopts_qdict,"encrypt." ); +qobject_unref (opts_qdict); +qdict_put_str (cryptoopts_qdict,"format" , fmt); +return cryptoopts_qdict; +}/* * read qcow2 extension and fill bs
@@ -4849,16 +4862,9 @@ static BlockMeasureInfo *qcow2_measure(QemuOpts *opts, BlockDriverState *in_bs,if (has_luks) {g_autoptr (QCryptoBlockCreateOptions) create_opts =NULL ; -QDict *opts_qdict; -QDict *cryptoopts; +QDict *cryptoopts =qcow2_extract_crypto_opts (opts,"luks" , errp);size_t headerlen; - opts_qdict =qemu_opts_to_qdict (opts,NULL ); -qdict_extract_subqdict (opts_qdict, &cryptoopts,"encrypt." ); -qobject_unref (opts_qdict); - -qdict_put_str (cryptoopts,"format" ,"luks" ); - create_opts =block_crypto_create_opts_init (cryptoopts, errp);qobject_unref (cryptoopts);if (!create_opts) {
@@ -5262,6 +5268,7 @@ typedef enum Qcow2AmendOperation {QCOW2_NO_OPERATION =0 ,QCOW2_UPGRADING , +QCOW2_UPDATING_ENCRYPTION ,QCOW2_CHANGING_REFCOUNT_ORDER ,QCOW2_DOWNGRADING , } Qcow2AmendOperation;
@@ -5343,6 +5350,7 @@ static int qcow2_amend_options(BlockDriverState *bs, QemuOpts *opts,int ret;QemuOptDesc *desc = opts->list->desc;Qcow2AmendHelperCBInfo helper_cb_info; +bool encryption_update = false;while (desc && desc->name) {if (!qemu_opt_find (opts, desc->name)) {
@@ -5369,6 +5377,18 @@ static int qcow2_amend_options(BlockDriverState *bs, QemuOpts *opts, backing_file =qemu_opt_get (opts,BLOCK_OPT_BACKING_FILE ); }else if (!strcmp (desc->name,BLOCK_OPT_BACKING_FMT )) { backing_format =qemu_opt_get (opts,BLOCK_OPT_BACKING_FMT ); + }else if (g_str_has_prefix (desc->name,"encrypt." )) { +if (!s->crypto) { +error_setg (errp, +"Can't amend encryption options - encryption not present" ); +return -EINVAL ; + } +if (s->crypt_method_header !=QCOW_CRYPT_LUKS ) { +error_setg (errp, +"Only LUKS encryption options can be amended" ); +return -ENOTSUP ; + } + encryption_update = true; }else if (!strcmp (desc->name,BLOCK_OPT_LAZY_REFCOUNTS )) { lazy_refcounts =qemu_opt_get_bool (opts,BLOCK_OPT_LAZY_REFCOUNTS , lazy_refcounts);
@@ -5411,7 +5431,8 @@ static int qcow2_amend_options(BlockDriverState *bs, QemuOpts *opts, .original_status_cb = status_cb, .original_cb_opaque = cb_opaque, .total_operations = (new_version != old_version) - + (s->refcount_bits != refcount_bits) + + (s->refcount_bits != refcount_bits) + + (encryption_update == true) };/* Upgrade first (some features may require compat=1.1) */
@@ -5424,6 +5445,33 @@ static int qcow2_amend_options(BlockDriverState *bs, QemuOpts *opts, } } +if (encryption_update) { +QDict *amend_opts_dict; +QCryptoBlockAmendOptions *amend_opts; + + helper_cb_info.current_operation =QCOW2_UPDATING_ENCRYPTION ; + amend_opts_dict =qcow2_extract_crypto_opts (opts,"luks" , errp); +if (!amend_opts_dict) { +return -EINVAL ; + } + amend_opts =block_crypto_amend_opts_init (amend_opts_dict, errp); +qobject_unref (amend_opts_dict); +if (!amend_opts) { +return -EINVAL ; + } + ret =qcrypto_block_amend_options (s->crypto, + qcow2_crypto_hdr_read_func, + qcow2_crypto_hdr_write_func, + bs, + amend_opts, + force, + errp); +qapi_free_QCryptoBlockAmendOptions (amend_opts); +if (ret <0 ) { +return ret; + } + } +if (s->refcount_bits != refcount_bits) {int refcount_order =ctz32 (refcount_bits);
@@ -5683,6 +5731,11 @@ static QemuOptsList qcow2_amend_opts = { .name = "qcow2-amend-opts", .head = QTAILQ_HEAD_INITIALIZER(qcow2_amend_opts.head), .desc = { +BLOCK_CRYPTO_OPT_DEF_LUKS_STATE ("encrypt." ), +BLOCK_CRYPTO_OPT_DEF_LUKS_KEYSLOT ("encrypt." ), +BLOCK_CRYPTO_OPT_DEF_LUKS_OLD_SECRET ("encrypt." ), +BLOCK_CRYPTO_OPT_DEF_LUKS_NEW_SECRET ("encrypt." ), +BLOCK_CRYPTO_OPT_DEF_LUKS_ITER_TIME ("encrypt." ), QCOW_COMMON_OPTIONS, { /* end of list */ } }
tests/qemu-iotests/082.out +45 -0
99 lines not on this page · raw message
Signature · 1 line
2.26.2
PATCH 15/19 iotests: qemu-img tests for luks key management
In reply to Max ReitzFrom: Maxim Levitsky <[email protected]>
This commit adds two tests, which test the new amend interface of both luks raw images and qcow2 luks encrypted images.
[mreitz: Let 293 verify that LUKS works; drop $(seq) usage from 293;
drop 293 and 294 from the auto group]
--- tests/qemu-iotests/293 | 208 +++++++++++++++++++++++++++++++++++++ tests/qemu-iotests/293.out | 99 ++++++++++++++++++ tests/qemu-iotests/294 | 90 ++++++++++++++++ tests/qemu-iotests/294.out | 30 ++++++ tests/qemu-iotests/group | 2 + 5 files changed, 429 insertions(+) create mode 100755 tests/qemu-iotests/293 create mode 100644 tests/qemu-iotests/293.out create mode 100755 tests/qemu-iotests/294 create mode 100644 tests/qemu-iotests/294.out
tests/qemu-iotests/293 new file +208 -0
@@ -0,0 +1,208 @@ +#!/usr/bin/env bash +# +# Test encryption key management with luks +# Based on 134 +# +# Copyright (C) 2019 Red Hat, Inc. +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see <http://www.gnu.org/licenses/>. +# + +# creator [email protected] + +seq=`basename $0` +echo "QA output created by $seq" + +status=1 # failure is the default! + +_cleanup() +{ + _cleanup_test_img +} +trap "_cleanup; exit \$status" 0 1 2 3 15 + +# get standard environment, filters and checks +. ./common.rc +. ./common.filter + +_supported_fmt qcow2 luks +_supported_proto file #TODO +_require_working_luks + +QEMU_IO_OPTIONS=$QEMU_IO_OPTIONS_NO_FMT + +if [ "$IMGFMT" = "qcow2" ] ; then + PR="encrypt." + EXTRA_IMG_ARGS="-o encrypt.format=luks" +fi + + +# secrets: you are supposed to see the password as *******, see :-) +S0="--object secret,id=sec0,data=hunter0" +S1="--object secret,id=sec1,data=hunter1" +S2="--object secret,id=sec2,data=hunter2" +S3="--object secret,id=sec3,data=hunter3" +S4="--object secret,id=sec4,data=hunter4" +SECRETS="$S0 $S1 $S2 $S3 $S4" + +# image with given secret +IMGS0="--image-opts driver=$IMGFMT,file.filename=$TEST_IMG,${PR}key-secret=sec0" +IMGS1="--image-opts driver=$IMGFMT,file.filename=$TEST_IMG,${PR}key-secret=sec1" +IMGS2="--image-opts driver=$IMGFMT,file.filename=$TEST_IMG,${PR}key-secret=sec2" +IMGS3="--image-opts driver=$IMGFMT,file.filename=$TEST_IMG,${PR}key-secret=sec3" +IMGS4="--image-opts driver=$IMGFMT,file.filename=$TEST_IMG,${PR}key-secret=sec4" + + +echo "== creating a test image ==" +_make_test_img $S0 $EXTRA_IMG_ARGS -o ${PR}key-secret=sec0,${PR}iter-time=10 32M + +echo +echo "== test that key 0 opens the image ==" +$QEMU_IO $S0 -c "read 0 4096" $IMGS0 | _filter_qemu_io | _filter_testdir + +echo +echo "== adding a password to slot 4 ==" +$QEMU_IMG amend $SECRETS $IMGS0 -o ${PR}state=active,${PR}new-secret=sec4,${PR}iter-time=10,${PR}keyslot=4 +echo "== adding a password to slot 1 ==" +$QEMU_IMG amend $SECRETS $IMGS0 -o ${PR}state=active,${PR}new-secret=sec1,${PR}iter-time=10 +echo "== adding a password to slot 3 ==" +$QEMU_IMG amend $SECRETS $IMGS1 -o ${PR}state=active,${PR}new-secret=sec3,${PR}iter-time=10,${PR}keyslot=3 + +echo "== adding a password to slot 2 ==" +$QEMU_IMG amend $SECRETS $IMGS3 -o ${PR}state=active,${PR}new-secret=sec2,${PR}iter-time=10 + + +echo "== erase slot 4 ==" +$QEMU_IMG amend $SECRETS $IMGS1 -o ${PR}state=inactive,${PR}keyslot=4 | _filter_img_create + + +echo +echo "== all secrets should work ==" +for IMG in "$IMGS0" "$IMGS1" "$IMGS2" "$IMGS3"; do + $QEMU_IO $SECRETS -c "read 0 4096" $IMG | _filter_qemu_io | _filter_testdir +done + +echo +echo "== erase slot 0 and try it ==" +$QEMU_IMG amend $SECRETS $IMGS1 -o ${PR}state=inactive,${PR}old-secret=sec0 | _filter_img_create +$QEMU_IO $SECRETS -c "read 0 4096" $IMGS0 | _filter_qemu_io | _filter_testdir + +echo +echo "== erase slot 2 and try it ==" +$QEMU_IMG amend $SECRETS $IMGS1 -o ${PR}state=inactive,${PR}keyslot=2 | _filter_img_create +$QEMU_IO $SECRETS -c "read 0 4096" $IMGS2 | _filter_qemu_io | _filter_testdir + + +# at this point slots 1 and 3 should be active + +echo +echo "== filling 4 slots with secret 2 ==" +for ((i = 0; i < 4; i++)); do + $QEMU_IMG amend $SECRETS $IMGS3 -o ${PR}state=active,${PR}new-secret=sec2,${PR}iter-time=10 +done + +echo +echo "== adding secret 0 ==" + $QEMU_IMG amend $SECRETS $IMGS3 -o ${PR}state=active,${PR}new-secret=sec0,${PR}iter-time=10 + +echo +echo "== adding secret 3 (last slot) ==" + $QEMU_IMG amend $SECRETS $IMGS3 -o ${PR}state=active,${PR}new-secret=sec3,${PR}iter-time=10 + +echo +echo "== trying to add another slot (should fail) ==" +$QEMU_IMG amend $SECRETS $IMGS2 -o ${PR}state=active,${PR}new-secret=sec3,${PR}iter-time=10 + +echo +echo "== all secrets should work again ==" +for IMG in "$IMGS0" "$IMGS1" "$IMGS2" "$IMGS3"; do + $QEMU_IO $SECRETS -c "read 0 4096" $IMG | _filter_qemu_io | _filter_testdir +done + + +echo + +echo "== erase all keys of secret 2==" +$QEMU_IMG amend $SECRETS $IMGS1 -o ${PR}state=inactive,${PR}old-secret=sec2 + +echo "== erase all keys of secret 1==" +$QEMU_IMG amend $SECRETS $IMGS1 -o ${PR}state=inactive,${PR}old-secret=sec1 + +echo "== erase all keys of secret 0==" +$QEMU_IMG amend $SECRETS $IMGS0 -o ${PR}state=inactive,${PR}old-secret=sec0 + +echo "== erasing secret3 will fail now since it is the only secret (in 3 slots) ==" +$QEMU_IMG amend $SECRETS $IMGS3 -o ${PR}state=inactive,${PR}old-secret=sec3 + +echo +echo "== only secret3 should work now ==" +for IMG in "$IMGS0" "$IMGS1" "$IMGS2" "$IMGS3"; do + $QEMU_IO $SECRETS -c "read 0 4096" $IMG | _filter_qemu_io | _filter_testdir +done + +echo +echo "== add secret0 ==" +$QEMU_IMG amend $SECRETS $IMGS3 -o ${PR}state=active,${PR}new-secret=sec0,${PR}iter-time=10 + +echo "== erase secret3 ==" +$QEMU_IMG amend $SECRETS $IMGS0 -o ${PR}state=inactive,${PR}old-secret=sec3 + +echo +echo "== only secret0 should work now ==" +for IMG in "$IMGS0" "$IMGS1" "$IMGS2" "$IMGS3"; do + $QEMU_IO $SECRETS -c "read 0 4096" $IMG | _filter_qemu_io | _filter_testdir +done + +echo +echo "== replace secret0 with secret1 (should fail) ==" +$QEMU_IMG amend $SECRETS $IMGS0 -o ${PR}state=active,${PR}new-secret=sec1,${PR}keyslot=0 + +echo +echo "== replace secret0 with secret1 with force (should work) ==" +$QEMU_IMG amend $SECRETS $IMGS0 -o ${PR}state=active,${PR}new-secret=sec1,${PR}iter-time=10,${PR}keyslot=0 --force + +echo +echo "== only secret1 should work now ==" +for IMG in "$IMGS0" "$IMGS1" "$IMGS2" "$IMGS3"; do + $QEMU_IO $SECRETS -c "read 0 4096" $IMG | _filter_qemu_io | _filter_testdir +done + + +echo +echo "== erase last secret (should fail) ==" +$QEMU_IMG amend $SECRETS $IMGS1 -o ${PR}state=inactive,${PR}keyslot=0 +$QEMU_IMG amend $SECRETS $IMGS1 -o ${PR}state=inactive,${PR}old-secret=sec1 + + +echo "== erase non existing secrets (should fail) ==" +$QEMU_IMG amend $SECRETS $IMGS1 -o ${PR}state=inactive,${PR}old-secret=sec5 --force +$QEMU_IMG amend $SECRETS $IMGS1 -o ${PR}state=inactive,${PR}old-secret=sec0 --force +$QEMU_IMG amend $SECRETS $IMGS1 -o ${PR}state=inactive,${PR}keyslot=1 --force + +echo +echo "== erase last secret with force by slot (should work) ==" +$QEMU_IMG amend $SECRETS $IMGS1 -o ${PR}state=inactive,${PR}keyslot=0 --force + +echo +echo "== we have no secrets now, data is lost forever ==" +for IMG in "$IMGS0" "$IMGS1" "$IMGS2" "$IMGS3"; do + $QEMU_IO $SECRETS -c "read 0 4096" $IMG | _filter_qemu_io | _filter_testdir +done + +# success, all done +echo "*** done" +rm -f $seq.full +status=0 +
tests/qemu-iotests/293.out new file +99 -0
99 lines not on this page · raw message
tests/qemu-iotests/294 new file +90 -0
90 lines not on this page · raw message
tests/qemu-iotests/294.out new file +30 -0
@@ -0,0 +1,30 @@ +QA output created by 294 +== creating a test image == +Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=33554432 + +== test that key 0 opens the image == +read 4096/4096 bytes at offset 0 +4 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) + +== adding a password to slot 1 == + +== 'backup' the image header == +1+0 records in +1+0 records out + +== erase slot 0 == + +== test that key 0 doesn't open the image == +qemu-io: can't open: Invalid password, cannot unlock any keyslot + +== 'restore' the image header == +1+0 records in +1+0 records out + +== test that key 0 still doesn't open the image (key material is erased) == +qemu-io: can't open: Invalid password, cannot unlock any keyslot + +== test that key 1 still works == +read 4096/4096 bytes at offset 0 +4 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +*** done
tests/qemu-iotests/group +2 -0
@@ -301,4 +301,6 @@ 290 rw auto quick 291 rw quick 292 rw auto quick +293 rw +294 rw quick 297 meta
Signature · 1 line
2.26.2
Re: [PATCH 15/19] iotests: qemu-img tests for luks key management
Quoted 478 lines · Max Reitz
From: Maxim Levitsky <[email protected]>
This commit adds two tests, which test the new amend interface of both luks raw images and qcow2 luks encrypted images.
Signed-off-by Maxim Levitsky <[email protected]>Reviewed-by Daniel P. Berrangé <[email protected]>[mreitz: Let 293 verify that LUKS works; drop $(seq) usage from 293;
drop 293 and 294 from the auto group]Signed-off-by Max Reitz <[email protected]>--- tests/qemu-iotests/293 | 208 +++++++++++++++++++++++++++++++++++++ tests/qemu-iotests/293.out | 99 ++++++++++++++++++ tests/qemu-iotests/294 | 90 ++++++++++++++++ tests/qemu-iotests/294.out | 30 ++++++ tests/qemu-iotests/group | 2 + 5 files changed, 429 insertions(+) create mode 100755 tests/qemu-iotests/293 create mode 100644 tests/qemu-iotests/293.out create mode 100755 tests/qemu-iotests/294 create mode 100644 tests/qemu-iotests/294.outtests/qemu-iotests/293 new file +208
-0@@ -0,0 +1,208 @@ +#!/usr/bin/env bash +# +# Test encryption key management with luks +# Based on 134 +# +# Copyright (C) 2019 Red Hat, Inc. +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see <http://www.gnu.org/licenses/>;. +# + +# creator [email protected] + +seq=`basename $0` +echo "QA output created by $seq" + +status=1 # failure is the default! + +_cleanup() +{ + _cleanup_test_img +} +trap "_cleanup; exit \$status" 0 1 2 3 15 + +# get standard environment, filters and checks +. ./common.rc +. ./common.filter + +_supported_fmt qcow2 luks +_supported_proto file #TODO +_require_working_luks + +QEMU_IO_OPTIONS=$QEMU_IO_OPTIONS_NO_FMT + +if [ "$IMGFMT" = "qcow2" ] ; then + PR="encrypt." + EXTRA_IMG_ARGS="-o encrypt.format=luks" +fi + + +# secrets: you are supposed to see the password as *******, see :-) +S0="--object secret,id=sec0,data=hunter0" +S1="--object secret,id=sec1,data=hunter1" +S2="--object secret,id=sec2,data=hunter2" +S3="--object secret,id=sec3,data=hunter3" +S4="--object secret,id=sec4,data=hunter4" +SECRETS="$S0 $S1 $S2 $S3 $S4" + +# image with given secret +IMGS0="--image-opts driver=$IMGFMT,file.filename=$TEST_IMG,${PR}key-secret=sec0" +IMGS1="--image-opts driver=$IMGFMT,file.filename=$TEST_IMG,${PR}key-secret=sec1" +IMGS2="--image-opts driver=$IMGFMT,file.filename=$TEST_IMG,${PR}key-secret=sec2" +IMGS3="--image-opts driver=$IMGFMT,file.filename=$TEST_IMG,${PR}key-secret=sec3" +IMGS4="--image-opts driver=$IMGFMT,file.filename=$TEST_IMG,${PR}key-secret=sec4" + + +echo "== creating a test image ==" +_make_test_img $S0 $EXTRA_IMG_ARGS -o ${PR}key-secret=sec0,${PR}iter-time=10 32M + +echo +echo "== test that key 0 opens the image ==" +$QEMU_IO $S0 -c "read 0 4096" $IMGS0 | _filter_qemu_io | _filter_testdir + +echo +echo "== adding a password to slot 4 ==" +$QEMU_IMG amend $SECRETS $IMGS0 -o ${PR}state=active,${PR}new-secret=sec4,${PR}iter-time=10,${PR}keyslot=4 +echo "== adding a password to slot 1 ==" +$QEMU_IMG amend $SECRETS $IMGS0 -o ${PR}state=active,${PR}new-secret=sec1,${PR}iter-time=10 +echo "== adding a password to slot 3 ==" +$QEMU_IMG amend $SECRETS $IMGS1 -o ${PR}state=active,${PR}new-secret=sec3,${PR}iter-time=10,${PR}keyslot=3 + +echo "== adding a password to slot 2 ==" +$QEMU_IMG amend $SECRETS $IMGS3 -o ${PR}state=active,${PR}new-secret=sec2,${PR}iter-time=10 + + +echo "== erase slot 4 ==" +$QEMU_IMG amend $SECRETS $IMGS1 -o ${PR}state=inactive,${PR}keyslot=4 | _filter_img_create + + +echo +echo "== all secrets should work ==" +for IMG in "$IMGS0" "$IMGS1" "$IMGS2" "$IMGS3"; do + $QEMU_IO $SECRETS -c "read 0 4096" $IMG | _filter_qemu_io | _filter_testdir +done + +echo +echo "== erase slot 0 and try it ==" +$QEMU_IMG amend $SECRETS $IMGS1 -o ${PR}state=inactive,${PR}old-secret=sec0 | _filter_img_create +$QEMU_IO $SECRETS -c "read 0 4096" $IMGS0 | _filter_qemu_io | _filter_testdir + +echo +echo "== erase slot 2 and try it ==" +$QEMU_IMG amend $SECRETS $IMGS1 -o ${PR}state=inactive,${PR}keyslot=2 | _filter_img_create +$QEMU_IO $SECRETS -c "read 0 4096" $IMGS2 | _filter_qemu_io | _filter_testdir + + +# at this point slots 1 and 3 should be active + +echo +echo "== filling 4 slots with secret 2 ==" +for ((i = 0; i < 4; i++)); do + $QEMU_IMG amend $SECRETS $IMGS3 -o ${PR}state=active,${PR}new-secret=sec2,${PR}iter-time=10 +done + +echo +echo "== adding secret 0 ==" + $QEMU_IMG amend $SECRETS $IMGS3 -o ${PR}state=active,${PR}new-secret=sec0,${PR}iter-time=10 + +echo +echo "== adding secret 3 (last slot) ==" + $QEMU_IMG amend $SECRETS $IMGS3 -o ${PR}state=active,${PR}new-secret=sec3,${PR}iter-time=10 + +echo +echo "== trying to add another slot (should fail) ==" +$QEMU_IMG amend $SECRETS $IMGS2 -o ${PR}state=active,${PR}new-secret=sec3,${PR}iter-time=10 + +echo +echo "== all secrets should work again ==" +for IMG in "$IMGS0" "$IMGS1" "$IMGS2" "$IMGS3"; do + $QEMU_IO $SECRETS -c "read 0 4096" $IMG | _filter_qemu_io | _filter_testdir +done + + +echo + +echo "== erase all keys of secret 2==" +$QEMU_IMG amend $SECRETS $IMGS1 -o ${PR}state=inactive,${PR}old-secret=sec2 + +echo "== erase all keys of secret 1==" +$QEMU_IMG amend $SECRETS $IMGS1 -o ${PR}state=inactive,${PR}old-secret=sec1 + +echo "== erase all keys of secret 0==" +$QEMU_IMG amend $SECRETS $IMGS0 -o ${PR}state=inactive,${PR}old-secret=sec0 + +echo "== erasing secret3 will fail now since it is the only secret (in 3 slots) ==" +$QEMU_IMG amend $SECRETS $IMGS3 -o ${PR}state=inactive,${PR}old-secret=sec3 + +echo +echo "== only secret3 should work now ==" +for IMG in "$IMGS0" "$IMGS1" "$IMGS2" "$IMGS3"; do + $QEMU_IO $SECRETS -c "read 0 4096" $IMG | _filter_qemu_io | _filter_testdir +done + +echo +echo "== add secret0 ==" +$QEMU_IMG amend $SECRETS $IMGS3 -o ${PR}state=active,${PR}new-secret=sec0,${PR}iter-time=10 + +echo "== erase secret3 ==" +$QEMU_IMG amend $SECRETS $IMGS0 -o ${PR}state=inactive,${PR}old-secret=sec3 + +echo +echo "== only secret0 should work now ==" +for IMG in "$IMGS0" "$IMGS1" "$IMGS2" "$IMGS3"; do + $QEMU_IO $SECRETS -c "read 0 4096" $IMG | _filter_qemu_io | _filter_testdir +done + +echo +echo "== replace secret0 with secret1 (should fail) ==" +$QEMU_IMG amend $SECRETS $IMGS0 -o ${PR}state=active,${PR}new-secret=sec1,${PR}keyslot=0 + +echo +echo "== replace secret0 with secret1 with force (should work) ==" +$QEMU_IMG amend $SECRETS $IMGS0 -o ${PR}state=active,${PR}new-secret=sec1,${PR}iter-time=10,${PR}keyslot=0 --force + +echo +echo "== only secret1 should work now ==" +for IMG in "$IMGS0" "$IMGS1" "$IMGS2" "$IMGS3"; do + $QEMU_IO $SECRETS -c "read 0 4096" $IMG | _filter_qemu_io | _filter_testdir +done + + +echo +echo "== erase last secret (should fail) ==" +$QEMU_IMG amend $SECRETS $IMGS1 -o ${PR}state=inactive,${PR}keyslot=0 +$QEMU_IMG amend $SECRETS $IMGS1 -o ${PR}state=inactive,${PR}old-secret=sec1 + + +echo "== erase non existing secrets (should fail) ==" +$QEMU_IMG amend $SECRETS $IMGS1 -o ${PR}state=inactive,${PR}old-secret=sec5 --force +$QEMU_IMG amend $SECRETS $IMGS1 -o ${PR}state=inactive,${PR}old-secret=sec0 --force +$QEMU_IMG amend $SECRETS $IMGS1 -o ${PR}state=inactive,${PR}keyslot=1 --force + +echo +echo "== erase last secret with force by slot (should work) ==" +$QEMU_IMG amend $SECRETS $IMGS1 -o ${PR}state=inactive,${PR}keyslot=0 --force + +echo +echo "== we have no secrets now, data is lost forever ==" +for IMG in "$IMGS0" "$IMGS1" "$IMGS2" "$IMGS3"; do + $QEMU_IO $SECRETS -c "read 0 4096" $IMG | _filter_qemu_io | _filter_testdir +done + +# success, all done +echo "*** done" +rm -f $seq.full +status=0 +tests/qemu-iotests/293.out new file +99
-0@@ -0,0 +1,99 @@ +QA output created by 293 +== creating a test image == +Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=33554432 + +== test that key 0 opens the image == +read 4096/4096 bytes at offset 0 +4 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) + +== adding a password to slot 4 == +== adding a password to slot 1 == +== adding a password to slot 3 == +== adding a password to slot 2 == +== erase slot 4 == + +== all secrets should work == +read 4096/4096 bytes at offset 0 +4 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +read 4096/4096 bytes at offset 0 +4 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +read 4096/4096 bytes at offset 0 +4 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +read 4096/4096 bytes at offset 0 +4 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) + +== erase slot 0 and try it == +qemu-io: can't open: Invalid password, cannot unlock any keyslot + +== erase slot 2 and try it == +qemu-io: can't open: Invalid password, cannot unlock any keyslot + +== filling 4 slots with secret 2 == + +== adding secret 0 == + +== adding secret 3 (last slot) == + +== trying to add another slot (should fail) == +qemu-img: Can't add a keyslot - all keyslots are in use + +== all secrets should work again == +read 4096/4096 bytes at offset 0 +4 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +read 4096/4096 bytes at offset 0 +4 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +read 4096/4096 bytes at offset 0 +4 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +read 4096/4096 bytes at offset 0 +4 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) + +== erase all keys of secret 2== +== erase all keys of secret 1== +== erase all keys of secret 0== +== erasing secret3 will fail now since it is the only secret (in 3 slots) == +qemu-img: All the active keyslots match the (old) password that was given and erasing them will erase all the data in the image irreversibly - refusing operation + +== only secret3 should work now == +qemu-io: can't open: Invalid password, cannot unlock any keyslot +qemu-io: can't open: Invalid password, cannot unlock any keyslot +qemu-io: can't open: Invalid password, cannot unlock any keyslot +read 4096/4096 bytes at offset 0 +4 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) + +== add secret0 == +== erase secret3 == + +== only secret0 should work now == +read 4096/4096 bytes at offset 0 +4 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +qemu-io: can't open: Invalid password, cannot unlock any keyslot +qemu-io: can't open: Invalid password, cannot unlock any keyslot +qemu-io: can't open: Invalid password, cannot unlock any keyslot + +== replace secret0 with secret1 (should fail) == +qemu-img: Refusing to overwrite active keyslot 0 - please erase it first + +== replace secret0 with secret1 with force (should work) == + +== only secret1 should work now == +qemu-io: can't open: Invalid password, cannot unlock any keyslot +read 4096/4096 bytes at offset 0 +4 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +qemu-io: can't open: Invalid password, cannot unlock any keyslot +qemu-io: can't open: Invalid password, cannot unlock any keyslot + +== erase last secret (should fail) == +qemu-img: Attempt to erase the only active keyslot 0 which will erase all the data in the image irreversibly - refusing operation +qemu-img: All the active keyslots match the (old) password that was given and erasing them will erase all the data in the image irreversibly - refusing operation +== erase non existing secrets (should fail) == +qemu-img: No secret with id 'sec5' +qemu-img: No keyslots match given (old) password for erase operation + +== erase last secret with force by slot (should work) == + +== we have no secrets now, data is lost forever == +qemu-io: can't open: Invalid password, cannot unlock any keyslot +qemu-io: can't open: Invalid password, cannot unlock any keyslot +qemu-io: can't open: Invalid password, cannot unlock any keyslot +qemu-io: can't open: Invalid password, cannot unlock any keyslot +*** donetests/qemu-iotests/294 new file +90
-0@@ -0,0 +1,90 @@ +# +# Copyright (C) 2019 Red Hat, Inc. +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see <http://www.gnu.org/licenses/>;. +# + +# creator [email protected] + +seq=`basename $0` +echo "QA output created by $seq" + +status=1 # failure is the default! + +_cleanup() +{ + _cleanup_test_img +} +trap "_cleanup; exit \$status" 0 1 2 3 15 + +# get standard environment, filters and checks +. ./common.rc +. ./common.filter + +_supported_fmt luks +_supported_proto file #TODO + +QEMU_IO_OPTIONS=$QEMU_IO_OPTIONS_NO_FMT + +# you are supposed to see the password as *******, see :-) +S0="--object secret,id=sec0,data=hunter0" +S1="--object secret,id=sec1,data=hunter1" +SECRETS="$S0 $S1" + + +IMGS0="--image-opts driver=$IMGFMT,file.filename=$TEST_IMG,key-secret=sec0" +IMGS1="--image-opts driver=$IMGFMT,file.filename=$TEST_IMG,key-secret=sec1" + +echo "== creating a test image ==" +_make_test_img $S0 -o "key-secret=sec0,iter-time=10" 32M + +echo +echo "== test that key 0 opens the image ==" +$QEMU_IO $S0 -c "read 0 4096" $IMGS0 | _filter_qemu_io | _filter_testdir + +echo +echo "== adding a password to slot 1 ==" +$QEMU_IMG amend $SECRETS $IMGS0 -o state=active,new-secret=sec1,keyslot=1,iter-time=10 + +echo +echo "== 'backup' the image header ==" +dd if=$TEST_IMG_FILE of=${TEST_IMG_FILE}.bk bs=4K skip=0 count=1 + +echo +echo "== erase slot 0 ==" +$QEMU_IMG amend $SECRETS $IMGS1 -o state=inactive,keyslot=0 | _filter_img_create + +echo +echo "== test that key 0 doesn't open the image ==" +$QEMU_IO $S0 -c "read 0 4096" $IMGS0 | _filter_qemu_io | _filter_testdir + +echo +echo "== 'restore' the image header ==" +dd if=${TEST_IMG_FILE}.bk of=${TEST_IMG_FILE} bs=4K skip=0 count=1 conv=notrunc + +echo +echo "== test that key 0 still doesn't open the image (key material is erased) ==" +$QEMU_IO $SECRETS -c "read 0 4096" $IMGS0 | _filter_qemu_io | _filter_testdir + +echo +echo "== test that key 1 still works ==" +$QEMU_IO $SECRETS -c "read 0 4096" $IMGS1 | _filter_qemu_io | _filter_testdir + +echo "*** done" +rm -f $seq.full +status=0 + + +exit 0tests/qemu-iotests/294.out new file +30
-0@@ -0,0 +1,30 @@ +QA output created by 294 +== creating a test image == +Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=33554432 + +== test that key 0 opens the image == +read 4096/4096 bytes at offset 0 +4 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) + +== adding a password to slot 1 == + +== 'backup' the image header == +1+0 records in +1+0 records out + +== erase slot 0 == + +== test that key 0 doesn't open the image == +qemu-io: can't open: Invalid password, cannot unlock any keyslot + +== 'restore' the image header == +1+0 records in +1+0 records out + +== test that key 0 still doesn't open the image (key material is erased) == +qemu-io: can't open: Invalid password, cannot unlock any keyslot + +== test that key 1 still works == +read 4096/4096 bytes at offset 0 +4 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +*** donetests/qemu-iotests/group +0
-0
tests/qemu-iotests/group line 306
Quoted 6 lines
tests/qemu-iotests/group +2
-0@@ -301,4 +301,6 @@ 290 rw auto quick 291 rw quick 292 rw auto quick +293 rw +294 rw quick 297 meta
I guess now we can add these to 'auto' group?
Best regards,
Maxim Levitsky
Re: [PATCH 15/19] iotests: qemu-img tests for luks key management
Quoted 22 lines · Maxim Levitsky
Quoted 21 lines · Max Reitz
From: Maxim Levitsky <[email protected]>
This commit adds two tests, which test the new amend interface of both luks raw images and qcow2 luks encrypted images.
Signed-off-by Maxim Levitsky <[email protected]>Reviewed-by Daniel P. Berrangé <[email protected]>[mreitz: Let 293 verify that LUKS works; drop $(seq) usage from 293;
drop 293 and 294 from the auto group]Signed-off-by Max Reitz <[email protected]>--- tests/qemu-iotests/293 | 208 +++++++++++++++++++++++++++++++++++++ tests/qemu-iotests/293.out | 99 ++++++++++++++++++ tests/qemu-iotests/294 | 90 ++++++++++++++++ tests/qemu-iotests/294.out | 30 ++++++ tests/qemu-iotests/group | 2 + 5 files changed, 429 insertions(+) create mode 100755 tests/qemu-iotests/293 create mode 100644 tests/qemu-iotests/293.out create mode 100755 tests/qemu-iotests/294 create mode 100644 tests/qemu-iotests/294.out
[...]
Quoted 5 lines
Quoted 5 lines
tests/qemu-iotests/group +0
-0@@ -301,4 +301,6 @@
Quoted 8 lines
tests/qemu-iotests/group line 306
Quoted 6 lines
tests/qemu-iotests/group +2
-0290 rw auto quick 291 rw quick 292 rw auto quick +293 rw +294 rw quick 297 metaI guess now we can add these to 'auto' group?
Have you run them on all platforms? I haven’t.
(I have run the openbsd build, but I’m not even sure I had these tests in auto at that point. And that build takes a really long time.)
All in all, I don’t think I ever want to add a test to auto again, unless it’s done in a specific patch where the author guarantees they tested it everywhere.
Max
Re: [PATCH 15/19] iotests: qemu-img tests for luks key management
Quoted 40 lines · Max Reitz
Quoted 22 lines · Maxim Levitsky
Quoted 21 lines · Max Reitz
From: Maxim Levitsky <[email protected]>
This commit adds two tests, which test the new amend interface of both luks raw images and qcow2 luks encrypted images.
Signed-off-by Maxim Levitsky <[email protected]>Reviewed-by Daniel P. Berrangé <[email protected]>[mreitz: Let 293 verify that LUKS works; drop $(seq) usage from 293;
drop 293 and 294 from the auto group]Signed-off-by Max Reitz <[email protected]>--- tests/qemu-iotests/293 | 208 +++++++++++++++++++++++++++++++++++++ tests/qemu-iotests/293.out | 99 ++++++++++++++++++ tests/qemu-iotests/294 | 90 ++++++++++++++++ tests/qemu-iotests/294.out | 30 ++++++ tests/qemu-iotests/group | 2 + 5 files changed, 429 insertions(+) create mode 100755 tests/qemu-iotests/293 create mode 100644 tests/qemu-iotests/293.out create mode 100755 tests/qemu-iotests/294 create mode 100644 tests/qemu-iotests/294.out[...]
Quoted 5 lines
Quoted 5 lines
tests/qemu-iotests/group +0
-0@@ -301,4 +301,6 @@Quoted 8 lines
tests/qemu-iotests/group line 306
Quoted 6 lines
tests/qemu-iotests/group +2
-0290 rw auto quick 291 rw quick 292 rw auto quick +293 rw +294 rw quick 297 metaI guess now we can add these to 'auto' group?
Quoted 1 line
Have you run them on all platforms? I haven’t.
Me neither.
Quoted 7 lines
(I have run the openbsd build, but I’m not even sure I had these tests in auto at that point. And that build takes a really long time.)
All in all, I don’t think I ever want to add a test to auto again, unless it’s done in a specific patch where the author guarantees they tested it everywhere.
If we look at this at this angle, it makes sense.
I fully agreee.
Best regards,
Maxim Levitsky
Quoted 3 lines
Max
PATCH 16/19 block/core: add generic infrastructure for x-blockdev-amend qmp command
In reply to Max ReitzFrom: Maxim Levitsky <[email protected]>
blockdev-amend will be used similiar to blockdev-create
to allow on the fly changes of the structure of the format based block devices.
Current plan is to first support encryption keyslot management for luks based formats (raw and embedded in qcow2)
Message-Id: <[email protected]>
--- qapi/block-core.json | 42 ++++++++++++++ qapi/job.json | 4 +- include/block/block_int.h | 21 +++++-- block/amend.c | 113 ++++++++++++++++++++++++++++++++++++++ block/Makefile.objs | 2 +- 5 files changed, 174 insertions(+), 8 deletions(-) create mode 100644 block/amend.c
qapi/block-core.json +42 -0
@@ -4674,6 +4674,48 @@ 'data': { 'job-id': 'str', 'options': 'BlockdevCreateOptions' } } +## +# @BlockdevAmendOptions: +# +# Options for amending an image format +# +# @driver: Block driver of the node to amend. +# +# Since: 5.1 +## +{ 'union': 'BlockdevAmendOptions', + 'base': { + 'driver': 'BlockdevDriver' }, + 'discriminator': 'driver', + 'data': { + } } + +## +# @x-blockdev-amend: +# +# Starts a job to amend format specific options of an existing open block device +# The job is automatically finalized, but a manual job-dismiss is required. +# +# @job-id: Identifier for the newly created job. +# +# @node-name: Name of the block node to work on +# +# @options: Options (driver specific) +# +# @force: Allow unsafe operations, format specific +# For luks that allows erase of the last active keyslot +# (permanent loss of data), +# and replacement of an active keyslot +# (possible loss of data if IO error happens) +# +# Since: 5.1 +## +{ 'command': 'x-blockdev-amend', + 'data': { 'job-id': 'str', + 'node-name': 'str', + 'options': 'BlockdevAmendOptions', + '*force': 'bool' } } + ## # @BlockErrorAction: #
qapi/job.json +3 -1
@@ -19,10 +19,12 @@ # # @create: image creation job type, see "blockdev-create" (since 3.0) # +# @amend: image options amend job type, see "x-blockdev-amend" (since 5.1) +# # Since: 1.7 ## { 'enum': 'JobType', - 'data': ['commit', 'stream', 'mirror', 'backup', 'create'] } + 'data': ['commit', 'stream', 'mirror', 'backup', 'create', 'amend'] } ## # @JobStatus:
include/block/block_int.h +15 -6
@@ -141,12 +141,27 @@ struct BlockDriver {int (*bdrv_file_open)(BlockDriverState *bs,QDict *options,int flags,Error **errp);void (*bdrv_close)(BlockDriverState *bs); + + int coroutine_fn (*bdrv_co_create)(BlockdevCreateOptions *opts,Error **errp); int coroutine_fn (*bdrv_co_create_opts)(BlockDriver *drv,const char *filename,QemuOpts *opts,Error **errp); + + int coroutine_fn (*bdrv_co_amend)(BlockDriverState *bs, +BlockdevAmendOptions *opts, +bool force, +Error **errp); + +int (*bdrv_amend_options)(BlockDriverState *bs, +QemuOpts *opts, +BlockDriverAmendStatusCB *status_cb, +void *cb_opaque, +bool force, +Error **errp); +int (*bdrv_make_empty)(BlockDriverState *bs);/*
@@ -441,12 +456,6 @@ struct BlockDriver {BdrvCheckResult *result, BdrvCheckMode fix); -int (*bdrv_amend_options)(BlockDriverState *bs,QemuOpts *opts, -BlockDriverAmendStatusCB *status_cb, -void *cb_opaque, -bool force, -Error **errp); -void (*bdrv_debug_event)(BlockDriverState *bs,BlkdebugEvent event);/* TODO Better pass a option string/QDict/QemuOpts to add any rule? */
block/amend.c new file +113 -0
113 lines not on this page · raw message
block/Makefile.objs +1 -1
@@ -19,7 +19,7 @@ block-obj-$(CONFIG_WIN32) += file-win32.o win32-aio.o block-obj-$(CONFIG_POSIX) += file-posix.o block-obj-$(CONFIG_LINUX_AIO) += linux-aio.o block-obj-$(CONFIG_LINUX_IO_URING) += io_uring.o -block-obj-y += null.o mirror.o commit.o io.o create.o +block-obj-y += null.o mirror.o commit.o io.o create.o amend.o block-obj-y += throttle-groups.o block-obj-$(CONFIG_LINUX) += nvme.o
Signature · 1 line
2.26.2
PATCH 17/19 block/crypto: implement blockdev-amend
In reply to Max ReitzFrom: Maxim Levitsky <[email protected]>
Message-Id: <[email protected]>
--- qapi/block-core.json | 14 ++++++++- block/crypto.c | 72 ++++++++++++++++++++++++++++++++------------ 2 files changed, 66 insertions(+), 20 deletions(-)
qapi/block-core.json +13 -1
@@ -4674,6 +4674,18 @@ 'data': { 'job-id': 'str', 'options': 'BlockdevCreateOptions' } } +## +# @BlockdevAmendOptionsLUKS: +# +# Driver specific image amend options for LUKS. +# +# Since: 5.1 +## +{ 'struct': 'BlockdevAmendOptionsLUKS', + 'base': 'QCryptoBlockAmendOptionsLUKS', + 'data': { } +} + ## # @BlockdevAmendOptions: #
@@ -4688,7 +4700,7 @@ 'driver': 'BlockdevDriver' }, 'discriminator': 'driver', 'data': { - } } + 'luks': 'BlockdevAmendOptionsLUKS' } } ## # @x-blockdev-amend:
block/crypto.c +53 -19
104 lines not on this page · raw message
Signature · 1 line
2.26.2
PATCH 18/19 block/qcow2: implement blockdev-amend
In reply to Max ReitzFrom: Maxim Levitsky <[email protected]>
Currently the implementation only supports amending the encryption options, unlike the qemu-img version
Message-Id: <[email protected]>
--- qapi/block-core.json | 16 +++++++++++++++- block/qcow2.c | 39 +++++++++++++++++++++++++++++++++++++++ 2 files changed, 54 insertions(+), 1 deletion(-)
qapi/block-core.json +15 -1
@@ -4686,6 +4686,19 @@ 'data': { } } +## +# @BlockdevAmendOptionsQcow2: +# +# Driver specific image amend options for qcow2. +# For now, only encryption options can be amended +# +# @encrypt Encryption options to be amended +# +# Since: 5.1 +## +{ 'struct': 'BlockdevAmendOptionsQcow2', + 'data': { '*encrypt': 'QCryptoBlockAmendOptions' } } + ## # @BlockdevAmendOptions: #
@@ -4700,7 +4713,8 @@ 'driver': 'BlockdevDriver' }, 'discriminator': 'driver', 'data': { - 'luks': 'BlockdevAmendOptionsLUKS' } } + 'luks': 'BlockdevAmendOptionsLUKS', + 'qcow2': 'BlockdevAmendOptionsQcow2' } } ## # @x-blockdev-amend:
block/qcow2.c +39 -0
@@ -5585,6 +5585,44 @@ static int qcow2_amend_options(BlockDriverState *bs, QemuOpts *opts,return 0 ; } +static int coroutine_fn qcow2_co_amend(BlockDriverState *bs, +BlockdevAmendOptions *opts, +bool force, +Error **errp) +{ +BlockdevAmendOptionsQcow2 *qopts = &opts->u.qcow2; +BDRVQcow2State *s = bs->opaque; +int ret =0 ; + +if (qopts->has_encrypt) { +if (!s->crypto) { +error_setg (errp,"image is not encrypted, can't amend" ); +return -EOPNOTSUPP ; + } + +if (qopts->encrypt->format !=Q_CRYPTO_BLOCK_FORMAT_LUKS ) { +error_setg (errp, +"Amend can't be used to change the qcow2 encryption format" ); +return -EOPNOTSUPP ; + } + +if (s->crypt_method_header !=QCOW_CRYPT_LUKS ) { +error_setg (errp, +"Only LUKS encryption options can be amended for qcow2 with blockdev-amend" ); +return -EOPNOTSUPP ; + } + + ret =qcrypto_block_amend_options (s->crypto, + qcow2_crypto_hdr_read_func, + qcow2_crypto_hdr_write_func, + bs, + qopts->encrypt, + force, + errp); + } +return ret; +} +/* * If offset or size are negative, respectively, they will not be included in * the BLOCK_IMAGE_CORRUPTED event emitted.
@@ -5802,6 +5840,7 @@ BlockDriver bdrv_qcow2 = { .mutable_opts = mutable_opts, .bdrv_co_check = qcow2_co_check, .bdrv_amend_options = qcow2_amend_options, + .bdrv_co_amend = qcow2_co_amend, .bdrv_detach_aio_context = qcow2_detach_aio_context, .bdrv_attach_aio_context = qcow2_attach_aio_context,
Signature · 1 line
2.26.2
PATCH 19/19 iotests: add tests for blockdev-amend
In reply to Max ReitzFrom: Maxim Levitsky <[email protected]>
This commit adds two tests that cover the
new blockdev-amend functionality of luks and qcow2 driver
[mreitz: Let 295 verify that LUKS works; drop 295 and 296 from the auto
group]
--- tests/qemu-iotests/295 | 280 +++++++++++++++++++++++++++++++++++++ tests/qemu-iotests/295.out | 40 ++++++ tests/qemu-iotests/296 | 234 +++++++++++++++++++++++++++++++ tests/qemu-iotests/296.out | 33 +++++ tests/qemu-iotests/group | 2 + 5 files changed, 589 insertions(+) create mode 100755 tests/qemu-iotests/295 create mode 100644 tests/qemu-iotests/295.out create mode 100755 tests/qemu-iotests/296 create mode 100644 tests/qemu-iotests/296.out
tests/qemu-iotests/295 new file +280 -0
@@ -0,0 +1,280 @@ +#!/usr/bin/env python3 +# +# Test case QMP's encrypted key management +# +# Copyright (C) 2019 Red Hat, Inc. +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see <http://www.gnu.org/licenses/>. +# + +import iotests +import os +import time +import json + +test_img = os.path.join(iotests.test_dir, 'test.img') + +class Secret: + def __init__(self, index): + self._id = "keysec" + str(index) + # you are not supposed to see the password... + self._secret = "hunter" + str(index) + + def id(self): + return self._id + + def secret(self): + return self._secret + + def to_cmdline_object(self): + return [ "secret,id=" + self._id + ",data=" + self._secret] + + def to_qmp_object(self): + return { "qom_type" : "secret", "id": self.id(), + "props": { "data": self.secret() } } + +################################################################################ +class EncryptionSetupTestCase(iotests.QMPTestCase): + + # test case startup + def setUp(self): + # start the VM + self.vm = iotests.VM() + self.vm.launch() + + # create the secrets and load 'em into the VM + self.secrets = [ Secret(i) for i in range(0, 6) ] + for secret in self.secrets: + result = self.vm.qmp("object-add", **secret.to_qmp_object()) + self.assert_qmp(result, 'return', {}) + + if iotests.imgfmt == "qcow2": + self.pfx = "encrypt." + self.img_opts = [ '-o', "encrypt.format=luks" ] + else: + self.pfx = "" + self.img_opts = [] + + # test case shutdown + def tearDown(self): + # stop the VM + self.vm.shutdown() + + ########################################################################### + # create the encrypted block device + def createImg(self, file, secret): + + iotests.qemu_img( + 'create', + '--object', *secret.to_cmdline_object(), + '-f', iotests.imgfmt, + '-o', self.pfx + 'key-secret=' + secret.id(), + '-o', self.pfx + 'iter-time=10', + *self.img_opts, + file, + '1M') + + ########################################################################### + # open an encrypted block device + def openImageQmp(self, id, file, secret, read_only = False): + + encrypt_options = { + 'key-secret' : secret.id() + } + + if iotests.imgfmt == "qcow2": + encrypt_options = { + 'encrypt': { + 'format':'luks', + **encrypt_options + } + } + + result = self.vm.qmp('blockdev-add', ** + { + 'driver': iotests.imgfmt, + 'node-name': id, + 'read-only': read_only, + + **encrypt_options, + + 'file': { + 'driver': 'file', + 'filename': test_img, + } + } + ) + self.assert_qmp(result, 'return', {}) + + # close the encrypted block device + def closeImageQmp(self, id): + result = self.vm.qmp('blockdev-del', **{ 'node-name': id }) + self.assert_qmp(result, 'return', {}) + + ########################################################################### + # add a key to an encrypted block device + def addKeyQmp(self, id, new_secret, secret = None, + slot = None, force = False): + + crypt_options = { + 'state' : 'active', + 'new-secret' : new_secret.id(), + 'iter-time' : 10 + } + + if slot != None: + crypt_options['keyslot'] = slot + + + if secret != None: + crypt_options['secret'] = secret.id() + + if iotests.imgfmt == "qcow2": + crypt_options['format'] = 'luks' + crypt_options = { + 'encrypt': crypt_options + } + + args = { + 'node-name': id, + 'job-id' : 'job_add_key', + 'options' : { + 'driver' : iotests.imgfmt, + **crypt_options + }, + } + + if force == True: + args['force'] = True + + #TODO: check what jobs return + result = self.vm.qmp('x-blockdev-amend', **args) + assert result['return'] == {} + self.vm.run_job('job_add_key') + + # erase a key from an encrypted block device + def eraseKeyQmp(self, id, old_secret = None, slot = None, force = False): + + crypt_options = { + 'state' : 'inactive', + } + + if slot != None: + crypt_options['keyslot'] = slot + if old_secret != None: + crypt_options['old-secret'] = old_secret.id() + + if iotests.imgfmt == "qcow2": + crypt_options['format'] = 'luks' + crypt_options = { + 'encrypt': crypt_options + } + + args = { + 'node-name': id, + 'job-id' : 'job_erase_key', + 'options' : { + 'driver' : iotests.imgfmt, + **crypt_options + }, + } + + if force == True: + args['force'] = True + + result = self.vm.qmp('x-blockdev-amend', **args) + assert result['return'] == {} + self.vm.run_job('job_erase_key') + + ########################################################################### + # create image, and change its key + def testChangeKey(self): + + # create the image with secret0 and open it + self.createImg(test_img, self.secrets[0]); + self.openImageQmp("testdev", test_img, self.secrets[0]) + + # add key to slot 1 + self.addKeyQmp("testdev", new_secret = self.secrets[1]) + + # add key to slot 5 + self.addKeyQmp("testdev", new_secret = self.secrets[2], slot=5) + + # erase key from slot 0 + self.eraseKeyQmp("testdev", old_secret = self.secrets[0]) + + #reopen the image with secret1 + self.closeImageQmp("testdev") + self.openImageQmp("testdev", test_img, self.secrets[1]) + + # close and erase the image for good + self.closeImageQmp("testdev") + os.remove(test_img) + + # test that if we erase the old password, + # we can still change the encryption keys using 'old-secret' + def testOldPassword(self): + + # create the image with secret0 and open it + self.createImg(test_img, self.secrets[0]); + self.openImageQmp("testdev", test_img, self.secrets[0]) + + # add key to slot 1 + self.addKeyQmp("testdev", new_secret = self.secrets[1]) + + # erase key from slot 0 + self.eraseKeyQmp("testdev", old_secret = self.secrets[0]) + + # this will fail as the old password is no longer valid + self.addKeyQmp("testdev", new_secret = self.secrets[2]) + + # this will work + self.addKeyQmp("testdev", new_secret = self.secrets[2], secret = self.secrets[1]) + + # close and erase the image for good + self.closeImageQmp("testdev") + os.remove(test_img) + + def testUseForceLuke(self): + + self.createImg(test_img, self.secrets[0]); + self.openImageQmp("testdev", test_img, self.secrets[0]) + + # Add bunch of secrets + self.addKeyQmp("testdev", new_secret = self.secrets[1], slot=4) + self.addKeyQmp("testdev", new_secret = self.secrets[4], slot=2) + + # overwrite an active secret + self.addKeyQmp("testdev", new_secret = self.secrets[5], slot=2) + self.addKeyQmp("testdev", new_secret = self.secrets[5], slot=2, force=True) + + self.addKeyQmp("testdev", new_secret = self.secrets[0]) + + # Now erase all the secrets + self.eraseKeyQmp("testdev", old_secret = self.secrets[5]) + self.eraseKeyQmp("testdev", slot=4) + + # erase last keyslot + self.eraseKeyQmp("testdev", old_secret = self.secrets[0]) + self.eraseKeyQmp("testdev", old_secret = self.secrets[0], force=True) + + self.closeImageQmp("testdev") + os.remove(test_img) + + +if __name__ == '__main__': + iotests.verify_working_luks() + # Encrypted formats support + iotests.activate_logging() + iotests.main(supported_fmts = ['qcow2', 'luks'])
tests/qemu-iotests/295.out new file +40 -0
@@ -0,0 +1,40 @@ +{"execute": "job-dismiss", "arguments": {"id": "job_add_key"}} +{"return": {}} +{"execute": "job-dismiss", "arguments": {"id": "job_add_key"}} +{"return": {}} +{"execute": "job-dismiss", "arguments": {"id": "job_erase_key"}} +{"return": {}} +{"execute": "job-dismiss", "arguments": {"id": "job_add_key"}} +{"return": {}} +{"execute": "job-dismiss", "arguments": {"id": "job_erase_key"}} +{"return": {}} +Job failed: Invalid password, cannot unlock any keyslot +{"execute": "job-dismiss", "arguments": {"id": "job_add_key"}} +{"return": {}} +{"execute": "job-dismiss", "arguments": {"id": "job_add_key"}} +{"return": {}} +{"execute": "job-dismiss", "arguments": {"id": "job_add_key"}} +{"return": {}} +{"execute": "job-dismiss", "arguments": {"id": "job_add_key"}} +{"return": {}} +Job failed: Refusing to overwrite active keyslot 2 - please erase it first +{"execute": "job-dismiss", "arguments": {"id": "job_add_key"}} +{"return": {}} +{"execute": "job-dismiss", "arguments": {"id": "job_add_key"}} +{"return": {}} +{"execute": "job-dismiss", "arguments": {"id": "job_add_key"}} +{"return": {}} +{"execute": "job-dismiss", "arguments": {"id": "job_erase_key"}} +{"return": {}} +{"execute": "job-dismiss", "arguments": {"id": "job_erase_key"}} +{"return": {}} +Job failed: All the active keyslots match the (old) password that was given and erasing them will erase all the data in the image irreversibly - refusing operation +{"execute": "job-dismiss", "arguments": {"id": "job_erase_key"}} +{"return": {}} +{"execute": "job-dismiss", "arguments": {"id": "job_erase_key"}} +{"return": {}} +... +---------------------------------------------------------------------- +Ran 3 tests + +OK
tests/qemu-iotests/296 new file +234 -0
234 lines not on this page · raw message
tests/qemu-iotests/296.out new file +33 -0
@@ -0,0 +1,33 @@ +Formatting 'TEST_DIR/test.img', fmt=luks size=1048576 key-secret=keysec0 iter-time=10 + +{"execute": "job-dismiss", "arguments": {"id": "job0"}} +{"return": {}} +Job failed: Failed to get shared "consistent read" lock +{"execute": "job-dismiss", "arguments": {"id": "job0"}} +{"return": {}} +Job failed: Failed to get shared "consistent read" lock +{"execute": "job-dismiss", "arguments": {"id": "job0"}} +{"return": {}} +{"execute": "job-dismiss", "arguments": {"id": "job0"}} +{"return": {}} +qemu-img: Failed to get shared "consistent read" lock +Is another process using the image [TEST_DIR/test.img]? + +Formatting 'TEST_DIR/test.img', fmt=luks size=1048576 key-secret=keysec0 iter-time=10 + +Job failed: Block node is read-only +{"execute": "job-dismiss", "arguments": {"id": "job0"}} +{"return": {}} +Job failed: Failed to get shared "consistent read" lock +{"execute": "job-dismiss", "arguments": {"id": "job0"}} +{"return": {}} +Job failed: Failed to get shared "consistent read" lock +{"execute": "job-dismiss", "arguments": {"id": "job0"}} +{"return": {}} +{"execute": "job-dismiss", "arguments": {"id": "job0"}} +{"return": {}} +.. +---------------------------------------------------------------------- +Ran 2 tests + +OK
tests/qemu-iotests/group +2 -0
@@ -303,4 +303,6 @@ 292 rw auto quick 293 rw 294 rw quick +295 rw +296 rw 297 meta
Signature · 1 line
2.26.2
Re: [PATCH 19/19] iotests: add tests for blockdev-amend
Quoted 638 lines · Max Reitz
From: Maxim Levitsky <[email protected]>
This commit adds two tests that cover the
new blockdev-amend functionality of luks and qcow2 driverSigned-off-by Maxim Levitsky <[email protected]>Reviewed-by Daniel P. Berrangé <[email protected]>[mreitz: Let 295 verify that LUKS works; drop 295 and 296 from the auto
group]Signed-off-by Max Reitz <[email protected]>--- tests/qemu-iotests/295 | 280 +++++++++++++++++++++++++++++++++++++ tests/qemu-iotests/295.out | 40 ++++++ tests/qemu-iotests/296 | 234 +++++++++++++++++++++++++++++++ tests/qemu-iotests/296.out | 33 +++++ tests/qemu-iotests/group | 2 + 5 files changed, 589 insertions(+) create mode 100755 tests/qemu-iotests/295 create mode 100644 tests/qemu-iotests/295.out create mode 100755 tests/qemu-iotests/296 create mode 100644 tests/qemu-iotests/296.outtests/qemu-iotests/295 new file +280
-0@@ -0,0 +1,280 @@ +#!/usr/bin/env python3 +# +# Test case QMP's encrypted key management +# +# Copyright (C) 2019 Red Hat, Inc. +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see <http://www.gnu.org/licenses/>;. +# + +import iotests +import os +import time +import json + +test_img = os.path.join(iotests.test_dir, 'test.img') + +class Secret: + def __init__(self, index): + self._id = "keysec" + str(index) + # you are not supposed to see the password... + self._secret = "hunter" + str(index) + + def id(self): + return self._id + + def secret(self): + return self._secret + + def to_cmdline_object(self): + return [ "secret,id=" + self._id + ",data=" + self._secret] + + def to_qmp_object(self): + return { "qom_type" : "secret", "id": self.id(), + "props": { "data": self.secret() } } + +################################################################################ +class EncryptionSetupTestCase(iotests.QMPTestCase): + + # test case startup + def setUp(self): + # start the VM + self.vm = iotests.VM() + self.vm.launch() + + # create the secrets and load 'em into the VM + self.secrets = [ Secret(i) for i in range(0, 6) ] + for secret in self.secrets: + result = self.vm.qmp("object-add", **secret.to_qmp_object()) + self.assert_qmp(result, 'return', {}) + + if iotests.imgfmt == "qcow2": + self.pfx = "encrypt." + self.img_opts = [ '-o', "encrypt.format=luks" ] + else: + self.pfx = "" + self.img_opts = [] + + # test case shutdown + def tearDown(self): + # stop the VM + self.vm.shutdown() + + ########################################################################### + # create the encrypted block device + def createImg(self, file, secret): + + iotests.qemu_img( + 'create', + '--object', *secret.to_cmdline_object(), + '-f', iotests.imgfmt, + '-o', self.pfx + 'key-secret=' + secret.id(), + '-o', self.pfx + 'iter-time=10', + *self.img_opts, + file, + '1M') + + ########################################################################### + # open an encrypted block device + def openImageQmp(self, id, file, secret, read_only = False): + + encrypt_options = { + 'key-secret' : secret.id() + } + + if iotests.imgfmt == "qcow2": + encrypt_options = { + 'encrypt': { + 'format':'luks', + **encrypt_options + } + } + + result = self.vm.qmp('blockdev-add', ** + { + 'driver': iotests.imgfmt, + 'node-name': id, + 'read-only': read_only, + + **encrypt_options, + + 'file': { + 'driver': 'file', + 'filename': test_img, + } + } + ) + self.assert_qmp(result, 'return', {}) + + # close the encrypted block device + def closeImageQmp(self, id): + result = self.vm.qmp('blockdev-del', **{ 'node-name': id }) + self.assert_qmp(result, 'return', {}) + + ########################################################################### + # add a key to an encrypted block device + def addKeyQmp(self, id, new_secret, secret = None, + slot = None, force = False): + + crypt_options = { + 'state' : 'active', + 'new-secret' : new_secret.id(), + 'iter-time' : 10 + } + + if slot != None: + crypt_options['keyslot'] = slot + + + if secret != None: + crypt_options['secret'] = secret.id() + + if iotests.imgfmt == "qcow2": + crypt_options['format'] = 'luks' + crypt_options = { + 'encrypt': crypt_options + } + + args = { + 'node-name': id, + 'job-id' : 'job_add_key', + 'options' : { + 'driver' : iotests.imgfmt, + **crypt_options + }, + } + + if force == True: + args['force'] = True + + #TODO: check what jobs return + result = self.vm.qmp('x-blockdev-amend', **args) + assert result['return'] == {} + self.vm.run_job('job_add_key') + + # erase a key from an encrypted block device + def eraseKeyQmp(self, id, old_secret = None, slot = None, force = False): + + crypt_options = { + 'state' : 'inactive', + } + + if slot != None: + crypt_options['keyslot'] = slot + if old_secret != None: + crypt_options['old-secret'] = old_secret.id() + + if iotests.imgfmt == "qcow2": + crypt_options['format'] = 'luks' + crypt_options = { + 'encrypt': crypt_options + } + + args = { + 'node-name': id, + 'job-id' : 'job_erase_key', + 'options' : { + 'driver' : iotests.imgfmt, + **crypt_options + }, + } + + if force == True: + args['force'] = True + + result = self.vm.qmp('x-blockdev-amend', **args) + assert result['return'] == {} + self.vm.run_job('job_erase_key') + + ########################################################################### + # create image, and change its key + def testChangeKey(self): + + # create the image with secret0 and open it + self.createImg(test_img, self.secrets[0]); + self.openImageQmp("testdev", test_img, self.secrets[0]) + + # add key to slot 1 + self.addKeyQmp("testdev", new_secret = self.secrets[1]) + + # add key to slot 5 + self.addKeyQmp("testdev", new_secret = self.secrets[2], slot=5) + + # erase key from slot 0 + self.eraseKeyQmp("testdev", old_secret = self.secrets[0]) + + #reopen the image with secret1 + self.closeImageQmp("testdev") + self.openImageQmp("testdev", test_img, self.secrets[1]) + + # close and erase the image for good + self.closeImageQmp("testdev") + os.remove(test_img) + + # test that if we erase the old password, + # we can still change the encryption keys using 'old-secret' + def testOldPassword(self): + + # create the image with secret0 and open it + self.createImg(test_img, self.secrets[0]); + self.openImageQmp("testdev", test_img, self.secrets[0]) + + # add key to slot 1 + self.addKeyQmp("testdev", new_secret = self.secrets[1]) + + # erase key from slot 0 + self.eraseKeyQmp("testdev", old_secret = self.secrets[0]) + + # this will fail as the old password is no longer valid + self.addKeyQmp("testdev", new_secret = self.secrets[2]) + + # this will work + self.addKeyQmp("testdev", new_secret = self.secrets[2], secret = self.secrets[1]) + + # close and erase the image for good + self.closeImageQmp("testdev") + os.remove(test_img) + + def testUseForceLuke(self): + + self.createImg(test_img, self.secrets[0]); + self.openImageQmp("testdev", test_img, self.secrets[0]) + + # Add bunch of secrets + self.addKeyQmp("testdev", new_secret = self.secrets[1], slot=4) + self.addKeyQmp("testdev", new_secret = self.secrets[4], slot=2) + + # overwrite an active secret + self.addKeyQmp("testdev", new_secret = self.secrets[5], slot=2) + self.addKeyQmp("testdev", new_secret = self.secrets[5], slot=2, force=True) + + self.addKeyQmp("testdev", new_secret = self.secrets[0]) + + # Now erase all the secrets + self.eraseKeyQmp("testdev", old_secret = self.secrets[5]) + self.eraseKeyQmp("testdev", slot=4) + + # erase last keyslot + self.eraseKeyQmp("testdev", old_secret = self.secrets[0]) + self.eraseKeyQmp("testdev", old_secret = self.secrets[0], force=True) + + self.closeImageQmp("testdev") + os.remove(test_img) + + +if __name__ == '__main__': + iotests.verify_working_luks() + # Encrypted formats support + iotests.activate_logging() + iotests.main(supported_fmts = ['qcow2', 'luks'])tests/qemu-iotests/295.out new file +40
-0@@ -0,0 +1,40 @@ +{"execute": "job-dismiss", "arguments": {"id": "job_add_key"}} +{"return": {}} +{"execute": "job-dismiss", "arguments": {"id": "job_add_key"}} +{"return": {}} +{"execute": "job-dismiss", "arguments": {"id": "job_erase_key"}} +{"return": {}} +{"execute": "job-dismiss", "arguments": {"id": "job_add_key"}} +{"return": {}} +{"execute": "job-dismiss", "arguments": {"id": "job_erase_key"}} +{"return": {}} +Job failed: Invalid password, cannot unlock any keyslot +{"execute": "job-dismiss", "arguments": {"id": "job_add_key"}} +{"return": {}} +{"execute": "job-dismiss", "arguments": {"id": "job_add_key"}} +{"return": {}} +{"execute": "job-dismiss", "arguments": {"id": "job_add_key"}} +{"return": {}} +{"execute": "job-dismiss", "arguments": {"id": "job_add_key"}} +{"return": {}} +Job failed: Refusing to overwrite active keyslot 2 - please erase it first +{"execute": "job-dismiss", "arguments": {"id": "job_add_key"}} +{"return": {}} +{"execute": "job-dismiss", "arguments": {"id": "job_add_key"}} +{"return": {}} +{"execute": "job-dismiss", "arguments": {"id": "job_add_key"}} +{"return": {}} +{"execute": "job-dismiss", "arguments": {"id": "job_erase_key"}} +{"return": {}} +{"execute": "job-dismiss", "arguments": {"id": "job_erase_key"}} +{"return": {}} +Job failed: All the active keyslots match the (old) password that was given and erasing them will erase all the data in the image irreversibly - refusing operation +{"execute": "job-dismiss", "arguments": {"id": "job_erase_key"}} +{"return": {}} +{"execute": "job-dismiss", "arguments": {"id": "job_erase_key"}} +{"return": {}} +... +---------------------------------------------------------------------- +Ran 3 tests + +OKtests/qemu-iotests/296 new file +234
-0@@ -0,0 +1,234 @@ +#!/usr/bin/env python3 +# +# Test case for encryption key management versus image sharing +# +# Copyright (C) 2019 Red Hat, Inc. +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see <http://www.gnu.org/licenses/>;. +# + +import iotests +import os +import time +import json + +test_img = os.path.join(iotests.test_dir, 'test.img') + +class Secret: + def __init__(self, index): + self._id = "keysec" + str(index) + # you are not supposed to see the password... + self._secret = "hunter" + str(index) + + def id(self): + return self._id + + def secret(self): + return self._secret + + def to_cmdline_object(self): + return [ "secret,id=" + self._id + ",data=" + self._secret] + + def to_qmp_object(self): + return { "qom_type" : "secret", "id": self.id(), + "props": { "data": self.secret() } } + +################################################################################ + +class EncryptionSetupTestCase(iotests.QMPTestCase): + + # test case startup + def setUp(self): + + # start the VMs + self.vm1 = iotests.VM(path_suffix = 'VM1') + self.vm2 = iotests.VM(path_suffix = 'VM2') + self.vm1.launch() + self.vm2.launch() + + # create the secrets and load 'em into the VMs + self.secrets = [ Secret(i) for i in range(0, 4) ] + for secret in self.secrets: + result = self.vm1.qmp("object-add", **secret.to_qmp_object()) + self.assert_qmp(result, 'return', {}) + result = self.vm2.qmp("object-add", **secret.to_qmp_object()) + self.assert_qmp(result, 'return', {}) + + # test case shutdown + def tearDown(self): + # stop the VM + self.vm1.shutdown() + self.vm2.shutdown() + + ########################################################################### + # create the encrypted block device using qemu-img + def createImg(self, file, secret): + + output = iotests.qemu_img_pipe( + 'create', + '--object', *secret.to_cmdline_object(), + '-f', iotests.imgfmt, + '-o', 'key-secret=' + secret.id(), + '-o', 'iter-time=10', + file, + '1M') + + iotests.log(output, filters=[iotests.filter_test_dir]) + + # attempts to add a key using qemu-img + def addKey(self, file, secret, new_secret): + + image_options = { + 'key-secret' : secret.id(), + 'driver' : iotests.imgfmt, + 'file' : { + 'driver':'file', + 'filename': file, + } + } + + output = iotests.qemu_img_pipe( + 'amend', + '--object', *secret.to_cmdline_object(), + '--object', *new_secret.to_cmdline_object(), + + '-o', 'state=active', + '-o', 'new-secret=' + new_secret.id(), + '-o', 'iter-time=10', + + "json:" + json.dumps(image_options) + ) + + iotests.log(output, filters=[iotests.filter_test_dir]) + + ########################################################################### + # open an encrypted block device + def openImageQmp(self, vm, id, file, secret, + readOnly = False, reOpen = False): + + command = 'x-blockdev-reopen' if reOpen else 'blockdev-add' + + result = vm.qmp(command, ** + { + 'driver': iotests.imgfmt, + 'node-name': id, + 'read-only': readOnly, + 'key-secret' : secret.id(), + 'file': { + 'driver': 'file', + 'filename': test_img, + } + } + ) + self.assert_qmp(result, 'return', {}) + + # close the encrypted block device + def closeImageQmp(self, vm, id): + result = vm.qmp('blockdev-del', **{ 'node-name': id }) + self.assert_qmp(result, 'return', {}) + + ########################################################################### + + # add a key to an encrypted block device + def addKeyQmp(self, vm, id, new_secret): + + args = { + 'node-name': id, + 'job-id' : 'job0', + 'options' : { + 'state' : 'active', + 'driver' : iotests.imgfmt, + 'new-secret': new_secret.id(), + 'iter-time' : 10 + }, + } + + result = vm.qmp('x-blockdev-amend', **args) + assert result['return'] == {} + vm.run_job('job0') + + # test that when the image opened by two qemu processes, + # neither of them can update the image + def test1(self): + self.createImg(test_img, self.secrets[0]); + + # VM1 opens the image and adds a key + self.openImageQmp(self.vm1, "testdev", test_img, self.secrets[0]) + self.addKeyQmp(self.vm1, "testdev", new_secret = self.secrets[1]) + + + # VM2 opens the image + self.openImageQmp(self.vm2, "testdev", test_img, self.secrets[0]) + + + # neither VMs now should be able to add a key + self.addKeyQmp(self.vm1, "testdev", new_secret = self.secrets[2]) + self.addKeyQmp(self.vm2, "testdev", new_secret = self.secrets[2]) + + + # VM 1 closes the image + self.closeImageQmp(self.vm1, "testdev") + + + # now VM2 can add the key + self.addKeyQmp(self.vm2, "testdev", new_secret = self.secrets[2]) + + + # qemu-img should also not be able to add a key + self.addKey(test_img, self.secrets[0], self.secrets[2]) + + # cleanup + self.closeImageQmp(self.vm2, "testdev") + os.remove(test_img) + + + def test2(self): + self.createImg(test_img, self.secrets[0]); + + # VM1 opens the image readonly + self.openImageQmp(self.vm1, "testdev", test_img, self.secrets[0], + readOnly = True) + + # VM2 opens the image + self.openImageQmp(self.vm2, "testdev", test_img, self.secrets[0]) + + # VM1 can't add a key since image is readonly + self.addKeyQmp(self.vm1, "testdev", new_secret = self.secrets[2]) + + # VM2 can't add a key since VM is has the image opened + self.addKeyQmp(self.vm2, "testdev", new_secret = self.secrets[2]) + + + #VM1 reopens the image read-write + self.openImageQmp(self.vm1, "testdev", test_img, self.secrets[0], + reOpen = True, readOnly = False) + + # VM1 still can't add the key + self.addKeyQmp(self.vm1, "testdev", new_secret = self.secrets[2]) + + # VM2 gets away + self.closeImageQmp(self.vm2, "testdev") + + # VM1 now can add the key + self.addKeyQmp(self.vm1, "testdev", new_secret = self.secrets[2]) + + self.closeImageQmp(self.vm1, "testdev") + os.remove(test_img) + + +if __name__ == '__main__': + # support only raw luks since luks encrypted qcow2 is a proper + # format driver which doesn't allow any sharing + iotests.activate_logging() + iotests.main(supported_fmts = ['luks'])tests/qemu-iotests/296.out new file +33
-0@@ -0,0 +1,33 @@ +Formatting 'TEST_DIR/test.img', fmt=luks size=1048576 key-secret=keysec0 iter-time=10 + +{"execute": "job-dismiss", "arguments": {"id": "job0"}} +{"return": {}} +Job failed: Failed to get shared "consistent read" lock +{"execute": "job-dismiss", "arguments": {"id": "job0"}} +{"return": {}} +Job failed: Failed to get shared "consistent read" lock +{"execute": "job-dismiss", "arguments": {"id": "job0"}} +{"return": {}} +{"execute": "job-dismiss", "arguments": {"id": "job0"}} +{"return": {}} +qemu-img: Failed to get shared "consistent read" lock +Is another process using the image [TEST_DIR/test.img]? + +Formatting 'TEST_DIR/test.img', fmt=luks size=1048576 key-secret=keysec0 iter-time=10 + +Job failed: Block node is read-only +{"execute": "job-dismiss", "arguments": {"id": "job0"}} +{"return": {}} +Job failed: Failed to get shared "consistent read" lock +{"execute": "job-dismiss", "arguments": {"id": "job0"}} +{"return": {}} +Job failed: Failed to get shared "consistent read" lock +{"execute": "job-dismiss", "arguments": {"id": "job0"}} +{"return": {}} +{"execute": "job-dismiss", "arguments": {"id": "job0"}} +{"return": {}} +.. +---------------------------------------------------------------------- +Ran 2 tests + +OKtests/qemu-iotests/group +0
-0
tests/qemu-iotests/group line 308
Quoted 6 lines
tests/qemu-iotests/group +2
-0@@ -303,4 +303,6 @@ 292 rw auto quick 293 rw 294 rw quick +295 rw +296 rw 297 meta
Same here,
should we add these to auto?
Best regards,
Maxim Levitsky
Patchew URL: https://patchew.org/QEMU/[email protected]/
Patchew URL: https://patchew.org/QEMU/[email protected]/
Hi,
This series seems to have some coding style problems. See output below for more information:
Subject: [PATCH 00/19] block: LUKS encryption slot management + iotest tweaks
Type: series
Message-id: [email protected]
=== TEST SCRIPT BEGIN ===
#!/bin/bash
git rev-parse base > /dev/null || exit 0
git config --local diff.renamelimit 0
git config --local diff.renames True
git config --local diff.algorithm histogram./scripts/checkpatch.pl --mailback base..
=== TEST SCRIPT END ===
From https://github.com/patchew-project/qemu
27c77b1..0250c59 master -> master
From https://github.com/patchew-project/qemu
* [new tag] patchew/[email protected] -> patchew/[email protected]
Switched to a new branch 'test'
e117214 iotests: add tests for blockdev-amend
cae47ee block/qcow2: implement blockdev-amend
88311ff block/crypto: implement blockdev-amend
a0a3f15 block/core: add generic infrastructure for x-blockdev-amend qmp command 1384b4f iotests: qemu-img tests for luks key management
49c50ee block/qcow2: extend qemu-img amend interface with crypto options f83b5c1 block/crypto: implement the encryption key management ff40d29 block/crypto: rename two functions
67b2028 block/amend: refactor qcow2 amend options
90caec5 block/amend: separate amend and create options for qemu-img 2df2e01 block/amend: add 'force' option
340bac8 qcrypto/luks: implement encryption key management
9d2d1ed qcrypto/core: add generic infrastructure for crypto options amendment 235478b iotests: Check whether luks works
5dd17f0 iotests.py: Add (verify|has)_working_luks()
5d918f7 iotests.py: Add qemu_img_pipe_and_status()
4330ee1 iotests/common.rc: Add _require_working_luks
c365035 iotests: filter few more luks specific create options daa65e4 iotests: Make _filter_img_create more active
=== OUTPUT BEGIN ===
1/19 Checking commit daa65e49e845 (iotests: Make _filter_img_create more active)
2/19 Checking commit c3650352b6fd (iotests: filter few more luks specific create options) 3/19 Checking commit 4330ee13d808 (iotests/common.rc: Add _require_working_luks)
4/19 Checking commit 5d918f763241 (iotests.py: Add qemu_img_pipe_and_status())
5/19 Checking commit 5dd17f035980 (iotests.py: Add (verify|has)_working_luks())
6/19 Checking commit 235478b9f3d3 (iotests: Check whether luks works)
7/19 Checking commit 9d2d1ed28ae9 (qcrypto/core: add generic infrastructure for crypto options amendment) 8/19 Checking commit 340bac8a40ad (qcrypto/luks: implement encryption key management) 9/19 Checking commit 2df2e01ef83b (block/amend: add 'force' option)
10/19 Checking commit 90caec5acfcd (block/amend: separate amend and create options for qemu-img)
ERROR: Macros with multiple statements should be enclosed in a do - while loop
#34: FILE: block/qcow2.c:5652:
+#define QCOW_COMMON_OPTIONS \
+ { \
+ .name = BLOCK_OPT_SIZE, \
+ .type = QEMU_OPT_SIZE, \
+ .help = "Virtual disk size" \
+ }, \
+ { \
+ .name = BLOCK_OPT_COMPAT_LEVEL, \
+ .type = QEMU_OPT_STRING, \
+ .help = "Compatibility level (v2 [0.10] or v3 [1.1])" \
+ }, \
+ { \
+ .name = BLOCK_OPT_BACKING_FILE, \
+ .type = QEMU_OPT_STRING, \
+ .help = "File name of a base image" \
+ }, \
+ { \
+ .name = BLOCK_OPT_BACKING_FMT, \
+ .type = QEMU_OPT_STRING, \
+ .help = "Image format of the base image" \
+ }, \
+ { \
+ .name = BLOCK_OPT_DATA_FILE, \
+ .type = QEMU_OPT_STRING, \
+ .help = "File name of an external data file" \
+ }, \
+ { \
+ .name = BLOCK_OPT_DATA_FILE_RAW, \
+ .type = QEMU_OPT_BOOL, \
+ .help = "The external data file must stay valid " \
+ "as a raw image" \
+ }, \
+ { \
+ .name = BLOCK_OPT_ENCRYPT, \
+ .type = QEMU_OPT_BOOL, \
+ .help = "Encrypt the image with format 'aes'. (Deprecated " \
+ "in favor of " BLOCK_OPT_ENCRYPT_FORMAT "=aes)", \
+ }, \
+ { \
+ .name = BLOCK_OPT_ENCRYPT_FORMAT, \
+ .type = QEMU_OPT_STRING, \
+ .help = "Encrypt the image, format choices: 'aes', 'luks'", \
+ }, \
+ BLOCK_CRYPTO_OPT_DEF_KEY_SECRET("encrypt.", \
+ "ID of secret providing qcow AES key or LUKS passphrase"), \
+ BLOCK_CRYPTO_OPT_DEF_LUKS_CIPHER_ALG("encrypt."), \
+ BLOCK_CRYPTO_OPT_DEF_LUKS_CIPHER_MODE("encrypt."), \
+ BLOCK_CRYPTO_OPT_DEF_LUKS_IVGEN_ALG("encrypt."), \
+ BLOCK_CRYPTO_OPT_DEF_LUKS_IVGEN_HASH_ALG("encrypt."), \
+ BLOCK_CRYPTO_OPT_DEF_LUKS_HASH_ALG("encrypt."), \
+ BLOCK_CRYPTO_OPT_DEF_LUKS_ITER_TIME("encrypt."), \
+ { \
+ .name = BLOCK_OPT_CLUSTER_SIZE, \
+ .type = QEMU_OPT_SIZE, \
+ .help = "qcow2 cluster size", \
+ .def_value_str = stringify(DEFAULT_CLUSTER_SIZE) \
+ }, \
+ { \
+ .name = BLOCK_OPT_PREALLOC, \
+ .type = QEMU_OPT_STRING, \
+ .help = "Preallocation mode (allowed values: off, " \
+ "metadata, falloc, full)" \
+ }, \
+ { \
+ .name = BLOCK_OPT_LAZY_REFCOUNTS, \
+ .type = QEMU_OPT_BOOL, \
+ .help = "Postpone refcount updates", \
+ .def_value_str = "off" \
+ }, \
+ { \
+ .name = BLOCK_OPT_REFCOUNT_BITS, \
+ .type = QEMU_OPT_NUMBER, \
+ .help = "Width of a reference count entry in bits", \
+ .def_value_str = "16" \
+ }, \
+ { \
+ .name = BLOCK_OPT_COMPRESSION_TYPE, \
+ .type = QEMU_OPT_STRING, \
+ .help = "Compression method used for image cluster " \
+ "compression", \
+ .def_value_str = "zlib" \+ }
total: 1 errors, 0 warnings, 244 lines checked
Patch 10/19 has style problems, please review. If any of these errors are false positives report them to the maintainer, see CHECKPATCH in MAINTAINERS.
11/19 Checking commit 67b20281fba1 (block/amend: refactor qcow2 amend options)
12/19 Checking commit ff40d29caf37 (block/crypto: rename two functions)
13/19 Checking commit f83b5c15d7e2 (block/crypto: implement the encryption key management) 14/19 Checking commit 49c50ee64bcd (block/qcow2: extend qemu-img amend interface with crypto options) 15/19 Checking commit 1384b4f4917d (iotests: qemu-img tests for luks key management)
WARNING: added, moved or deleted file(s), does MAINTAINERS need updating?
#18:
new file mode 100755
total: 0 errors, 1 warnings, 433 lines checked
Patch 15/19 has style problems, please review. If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
16/19 Checking commit a0a3f156abe8 (block/core: add generic infrastructure for x-blockdev-amend qmp command)
WARNING: added, moved or deleted file(s), does MAINTAINERS need updating?
#33:
new file mode 100644
total: 0 errors, 1 warnings, 221 lines checked
Patch 16/19 has style problems, please review. If any of these errors are false positives report them to the maintainer, see CHECKPATCH in MAINTAINERS.
17/19 Checking commit 88311ff56332 (block/crypto: implement blockdev-amend) 18/19 Checking commit cae47ee034f7 (block/qcow2: implement blockdev-amend) 19/19 Checking commit e11721462ab2 (iotests: add tests for blockdev-amend)
WARNING: added, moved or deleted file(s), does MAINTAINERS need updating? #18:
new file mode 100755
total: 0 errors, 1 warnings, 593 lines checked
Patch 19/19 has style problems, please review. If any of these errors are false positives report them to the maintainer, see CHECKPATCH in MAINTAINERS.
=== OUTPUT END ===
Test command exited with code: 1
The full log is available at
http://patchew.org/logs/[email protected]/testing.checkpatch/?type=message.
---
Email generated automatically by Patchew [https://patchew.org/].
Please send your feedback to [email protected]
Quoted 13 lines · Max Reitz
Hi,
This series collects all patches from Maxim’s series “LUKS: encryption slot management using amend interface”, plus iotest patches that seem useful to have before it. (And then I modified some of Maxim’s patches to take advantage of those patches.)
Maxim’s series was here:
https://lists.nongnu.org/archive/html/qemu-block/2020-06/msg00330.htmlI already posted patches 1 and 2 here:
https://lists.nongnu.org/archive/html/qemu-block/2020-06/msg00954.html
Quoted 12 lines
Change from the latter series:
- Patch 1:
- Use $SED instead of plain “sed” in one place I missed
- Older bash versions do not support readarray’s -d switch, so we have to separate the Formatting line by a \n instead of \0. Unfortunately, that means we cannot just split by \0 and be done with it, but we have to ensure we put everything before the line starting by “Formatting” into $filename_part, and we do that with a loop that iterates over the split array until it finds the “Formatting” line.
- Use multiple -e for grep instead of '(...|...|...)'
- Patch 2: Rebase fallout du to grep now using multiple -e parameters
Thanks for reviewing, Maxim; (re-)applied to my block branch: