Skip to contentSkip to conversation

Patch Round-up for stable 4.1.1, freeze on 2019-11-12

PATCH In review Michael Roth 4 participants 62 messages Nov 5, 2019 – Nov 12, 2019

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

cover Patch Round-up for stable 4.1.1, freeze on 2019-11-12

Hi everyone,

The following new patches are queued for QEMU stable v4.1.1:

https://github.com/mdroth/qemu/commits/stable-4.1-staging

The release is tentatively planned for 2019-11-14:

https://wiki.qemu.org/Planning/4.1

Please note that the original release date was planned for 2019-11-21, but was moved up to address a number of qcow2 corruption issues:

https://lists.gnu.org/archive/html/qemu-devel/2019-10/msg07144.html

Fixes for the XFS issues noted in the thread are still pending, but will hopefully be qemu.git master in time for 4.1.1 freeze and the currently-scheduled release date for 4.2.0-rc1.

The list of still-pending patchsets being tracked for inclusion are:

qcow2: Fix data corruption on XFS

    https://lists.gnu.org/archive/html/qemu-devel/2019-11/msg00073.html
    (PULL pending)
  qcow2: Fix QCOW2_COMPRESSED_SECTOR_MASK
    https://lists.gnu.org/archive/html/qemu-devel/2019-10/msg07718.html
  qcow2-bitmap: Fix uint64_t left-shift overflow
    https://lists.gnu.org/archive/html/qemu-devel/2019-10/msg07989.html

Please respond here or CC [email protected] on any additional patches you think should be included in the release.

Thanks!

----------------------------------------------------------------
Adrian Moreno (1):
vhost-user: save features if the char dev is closed

Alberto Garcia (1):
qcow2: Fix the calculation of the maximum L2 cache size

Anthony PERARD (1):
xen-bus: Fix backend state transition on device reset

Aurelien Jarno (1):
target/alpha: fix tlb_fill trap_arg2 value for instruction fetch

Christophe Lyon (1):
target/arm: Allow reading flags from FPSCR for M-profile

David Hildenbrand (1):

      s390x/tcg: Fix VERIM with 32/64 bit elements

Eduardo Habkost (1):
pc: Don't make die-id mandatory unless necessary

Fan Yang (1):
COLO-compare: Fix incorrect if logic

Hikaru Nishida (1):
ui: Fix hanging up Cocoa display on macOS 10.15 (Catalina)

Igor Mammedov (1):
x86: do not advertise die-id in query-hotpluggbale-cpus if '-smp dies' is not set

Johannes Berg (1):
libvhost-user: fix SLAVE_SEND_FD handling

John Snow (2):
Revert "ide/ahci: Check for -ECANCELED in aio callbacks"
iotests: add testing shim for script-style python tests

Kevin Wolf (4):
coroutine: Add qemu_co_mutex_assert_locked()
qcow2: Fix corruption bug in qcow2_detect_metadata_preallocation() block/snapshot: Restrict set of snapshot nodes
iotests: Test internal snapshots with -blockdev

Markus Armbruster (1):
pr-manager: Fix invalid g_free() crash bug

Matthew Rosato (1):
s390: PCI: fix IOMMU region init

Max Filippov (1):
target/xtensa: regenerate and re-import test_mmuhifi_c3 core

Max Reitz (16):
block/file-posix: Reduce xfsctl() use
iotests: Test reverse sub-cluster qcow2 writes
vpc: Return 0 from vpc_co_create() on success
iotests: Add supported protocols to execute_test()
iotests: Restrict file Python tests to file
iotests: Restrict nbd Python tests to nbd
iotests: Test blockdev-create for vpc
curl: Keep pointer to the CURLState in CURLSocket
curl: Keep *socket until the end of curl_sock_cb()
curl: Check completion in curl_multi_do()
curl: Pass CURLSocket to curl_multi_do()
curl: Report only ready sockets
curl: Handle success in multi_check_completion
qcow2: Limit total allocation range to INT_MAX
iotests: Test large write request to qcow2 file
mirror: Do not dereference invalid pointers

Maxim Levitsky (1):
block/qcow2: Fix corruption introduced by commit 8ac0f15f335

Michael Roth (2):
make-release: pull in edk2 submodules so we can build it from tarballs roms/Makefile.edk2: don't pull in submodules when building from tarball

Michael S. Tsirkin (1):
virtio: new post_load hook

Mikhail Sennikovsky (1):
virtio-net: prevent offloads reset on migration

Paolo Bonzini (2):
dma-helpers: ensure AIO callback is invoked after cancellation
scsi: lsi: exit infinite loop while executing script (CVE-2019-12068)

Paul Durrant (1):
xen-bus: check whether the frontend is active during device reset...

Peter Lieven (1):
block/nfs: tear down aio before nfs_close

Peter Maydell (3):
target/arm: Free TCG temps in trans_VMOV_64_sp() target/arm: Don't abort on M-profile exception return in linux-user mode hw/arm/boot.c: Set NSACR.{CP11,CP10} for NS kernel boots

Philippe Mathieu-Daudé (1):
virtio-blk: Cancel the pending BH when the dataplane is reset

Sergio Lopez (1):
blockjob: update nodes head while removing all bdrv

Thomas Huth (1):
hw/core/loader: Fix possible crash in rom_copy()

Vladimir Sementsov-Ogievskiy (4):
block/backup: fix max_transfer handling for copy_range block/backup: fix backup_cow_with_offload for last cluster util/hbitmap: strict hbitmap_reset
hbitmap: handle set/reset with zero length

 block/backup.c                                     |   17 +-
 block/curl.c                                       |  125 +-
 block/file-posix.c                                 |   77 +-
 block/mirror.c                                     |   13 +-
 block/nfs.c                                        |    6 +-
 block/qcow2-cluster.c                              |   12 +-
 block/qcow2-refcount.c                             |    2 +
 block/qcow2.c                                      |    9 +-
 block/snapshot.c                                   |   26 +-
 block/vpc.c                                        |    3 +-
 blockjob.c                                         |   17 +-
 contrib/libvhost-user/libvhost-user.c              |    3 +-
 dma-helpers.c                                      |   13 +-
 hw/arm/boot.c                                      |    2 +
 hw/block/dataplane/virtio-blk.c                    |    3 +
 hw/core/loader.c                                   |    2 +-
 hw/i386/pc.c                                       |   14 +-
 hw/ide/ahci.c                                      |    3 -
 hw/ide/core.c                                      |   14 -
 hw/net/virtio-net.c                                |   27 +-
 hw/s390x/s390-pci-bus.c                            |    7 +-
 hw/scsi/lsi53c895a.c                               |   41 +-
 hw/virtio/virtio.c                                 |    7 +
 hw/xen/xen-bus.c                                   |   23 +-
 include/hw/virtio/virtio-net.h                     |    2 +
 include/hw/virtio/virtio.h                         |    6 +
 include/qemu/coroutine.h                           |   15 +
 include/qemu/hbitmap.h                             |    5 +
 net/colo-compare.c                                 |    6 +-
 net/vhost-user.c                                   |    4 +
 roms/Makefile.edk2                                 |    7 +-
 scripts/make-release                               |    8 +
 scsi/pr-manager.c                                  |    1 -
 target/alpha/helper.c                              |    4 +-
 target/arm/translate-vfp.inc.c                     |    7 +-
 target/arm/translate.c                             |   21 +-
 target/s390x/translate_vx.inc.c                    |    2 +-
 target/xtensa/core-test_mmuhifi_c3.c               |    3 +-
 target/xtensa/core-test_mmuhifi_c3/core-isa.h      |  116 +-
 .../xtensa/core-test_mmuhifi_c3/gdb-config.inc.c   |  114 +-
 .../core-test_mmuhifi_c3/xtensa-modules.inc.c      | 6384 ++++++++++----------
 tests/acceptance/pc_cpu_hotplug_props.py           |   35 +
 tests/qemu-iotests/030                             |    3 +-
 tests/qemu-iotests/040                             |    3 +-
 tests/qemu-iotests/041                             |    3 +-
 tests/qemu-iotests/044                             |    3 +-
 tests/qemu-iotests/045                             |    3 +-
 tests/qemu-iotests/055                             |    3 +-
 tests/qemu-iotests/056                             |    3 +-
 tests/qemu-iotests/057                             |    3 +-
 tests/qemu-iotests/065                             |    3 +-
 tests/qemu-iotests/096                             |    3 +-
 tests/qemu-iotests/118                             |    3 +-
 tests/qemu-iotests/124                             |    3 +-
 tests/qemu-iotests/129                             |    3 +-
 tests/qemu-iotests/132                             |    3 +-
 tests/qemu-iotests/139                             |    3 +-
 tests/qemu-iotests/147                             |    5 +-
 tests/qemu-iotests/148                             |    3 +-
 tests/qemu-iotests/151                             |    3 +-
 tests/qemu-iotests/152                             |    3 +-
 tests/qemu-iotests/155                             |    3 +-
 tests/qemu-iotests/163                             |    3 +-
 tests/qemu-iotests/165                             |    3 +-
 tests/qemu-iotests/169                             |    3 +-
 tests/qemu-iotests/196                             |    3 +-
 tests/qemu-iotests/199                             |    3 +-
 tests/qemu-iotests/205                             |    3 +-
 tests/qemu-iotests/245                             |    3 +-
 tests/qemu-iotests/265                             |   67 +
 tests/qemu-iotests/265.out                         |    6 +
 tests/qemu-iotests/266                             |  153 +
 tests/qemu-iotests/266.out                         |  137 +
 tests/qemu-iotests/267                             |  168 +
 tests/qemu-iotests/267.out                         |  182 +
 tests/qemu-iotests/270                             |   83 +
 tests/qemu-iotests/270.out                         |    9 +
 tests/qemu-iotests/common.filter                   |   11 +-
 tests/qemu-iotests/group                           |    4 +
 tests/qemu-iotests/iotests.py                      |   42 +-
 tests/test-hbitmap.c                               |    2 +-
 ui/cocoa.m                                         |   12 +
 util/hbitmap.c                                     |   12 +
 83 files changed, 4663 insertions(+), 3514 deletions(-)
 create mode 100644 tests/acceptance/pc_cpu_hotplug_props.py
 create mode 100755 tests/qemu-iotests/265
 create mode 100644 tests/qemu-iotests/265.out
 create mode 100755 tests/qemu-iotests/266
 create mode 100644 tests/qemu-iotests/266.out
 create mode 100755 tests/qemu-iotests/267
 create mode 100644 tests/qemu-iotests/267.out
 create mode 100755 tests/qemu-iotests/270
 create mode 100644 tests/qemu-iotests/270.out
#1 lore raw

PATCH 1/55 dma-helpers: ensure AIO callback is invoked after cancellation

From: Paolo Bonzini <[email protected]>

dma_aio_cancel unschedules the BH if there is one, which corresponds to the reschedule_dma case of dma_blk_cb. This can stall the DMA permanently, because dma_complete will never get invoked and therefore nobody will ever invoke the original AIO callback in dbs->common.cb.

Fix this by invoking the callback (which is ensured to happen after a bdrv_aio_cancel_async, or done manually in the dbs->bh case), and add assertions to check that the DMA state machine is indeed waiting for dma_complete or reschedule_dma, but never both.

Message-id: [email protected]

(cherry picked from commit 539343c0a47e19d5dd64d846d64d084d9793681f)
---
 dma-helpers.c | 13 +++++++++----
 1 file changed, 9 insertions(+), 4 deletions(-)
dma-helpers.c +9 -4
index 2d7e02d35e..d3871dc61e 100644
@@ -90,6 +90,7 @@ static void reschedule_dma(void *opaque)
 {
     DMAAIOCB *dbs = (DMAAIOCB *)opaque;
 
+    assert(!dbs->acb && dbs->bh);
     qemu_bh_delete(dbs->bh);
     dbs->bh = NULL;
     dma_blk_cb(dbs, 0);
@@ -111,15 +112,12 @@ static void dma_complete(DMAAIOCB *dbs, int ret)
 {
     trace_dma_complete(dbs, ret, dbs->common.cb);
 
+    assert(!dbs->acb && !dbs->bh);
     dma_blk_unmap(dbs);
     if (dbs->common.cb) {
         dbs->common.cb(dbs->common.opaque, ret);
     }
     qemu_iovec_destroy(&dbs->iov);
-    if (dbs->bh) {
-        qemu_bh_delete(dbs->bh);
-        dbs->bh = NULL;
-    }
     qemu_aio_unref(dbs);
 }
 
@@ -179,14 +177,21 @@ static void dma_aio_cancel(BlockAIOCB *acb)
 
     trace_dma_aio_cancel(dbs);
 
+    assert(!(dbs->acb && dbs->bh));
     if (dbs->acb) {
+        /* This will invoke dma_blk_cb.  */
         blk_aio_cancel_async(dbs->acb);
+        return;
     }
+
     if (dbs->bh) {
         cpu_unregister_map_client(dbs->bh);
         qemu_bh_delete(dbs->bh);
         dbs->bh = NULL;
     }
+    if (dbs->common.cb) {
+        dbs->common.cb(dbs->common.opaque, -ECANCELED);
+    }
 }
 
 static AioContext *dma_get_aio_context(BlockAIOCB *acb)
Signature · 1 line
2.17.1
#2 lore raw

PATCH 2/55 Revert "ide/ahci: Check for -ECANCELED in aio callbacks"

In reply to Michael Roth

From: John Snow <[email protected]>

This reverts commit 0d910cfeaf2076b116b4517166d5deb0fea76394.

It's not correct to just ignore an error code in a callback; we need to handle that error and possible report failure to the guest so that they don't wait indefinitely for an operation that will now never finish.

This ought to help cases reported by Nutanix where iSCSI returns a legitimate -ECANCELED for certain operations which should be propagated normally.

Message-id: [email protected]

(cherry picked from commit 8ec41c4265714255d5a138f8b538faf3583dcff6)
---
 hw/ide/ahci.c |  3 ---
 hw/ide/core.c | 14 --------------
 2 files changed, 17 deletions(-)
hw/ide/ahci.c +0 -3
index 00ba422a48..6aaf66534a 100644
@@ -1023,9 +1023,6 @@ static void ncq_cb(void *opaque, int ret)
     IDEState *ide_state = &ncq_tfs->drive->port.ifs[0];
 
     ncq_tfs->aiocb = NULL;
-    if (ret == -ECANCELED) {
-        return;
-    }
 
     if (ret < 0) {
         bool is_read = ncq_tfs->cmd == READ_FPDMA_QUEUED;
hw/ide/core.c +0 -14
index 6afadf894f..8e1624f7ce 100644
@@ -722,9 +722,6 @@ static void ide_sector_read_cb(void *opaque, int ret)
     s->pio_aiocb = NULL;
     s->status &= ~BUSY_STAT;
 
-    if (ret == -ECANCELED) {
-        return;
-    }
     if (ret != 0) {
         if (ide_handle_rw_error(s, -ret, IDE_RETRY_PIO |
                                 IDE_RETRY_READ)) {
@@ -840,10 +837,6 @@ static void ide_dma_cb(void *opaque, int ret)
     uint64_t offset;
     bool stay_active = false;
 
-    if (ret == -ECANCELED) {
-        return;
-    }
-
     if (ret == -EINVAL) {
         ide_dma_error(s);
         return;
@@ -975,10 +968,6 @@ static void ide_sector_write_cb(void *opaque, int ret)
     IDEState *s = opaque;
     int n;
 
-    if (ret == -ECANCELED) {
-        return;
-    }
-
     s->pio_aiocb = NULL;
     s->status &= ~BUSY_STAT;
 
@@ -1058,9 +1047,6 @@ static void ide_flush_cb(void *opaque, int ret)
 
     s->pio_aiocb = NULL;
 
-    if (ret == -ECANCELED) {
-        return;
-    }
     if (ret < 0) {
         /* XXX: What sector number to set here? */
         if (ide_handle_rw_error(s, -ret, IDE_RETRY_FLUSH)) {
Signature · 1 line
2.17.1
#3 lore raw

PATCH 3/55 s390x/tcg: Fix VERIM with 32/64 bit elements

In reply to Michael Roth

From: David Hildenbrand <[email protected]>

Wrong order of operands. The constant always comes last. Makes QEMU crash reliably on specific git fetch invocations.

Message-Id: <[email protected]>

Fixes 5c4b0ab460ef ("s390x/tcg: Implement VECTOR ELEMENT ROTATE AND INSERT UNDER MASK")
(cherry picked from commit 25bcb45d1b81d22634daa2b1a2d8bee746ac129b)
---
 target/s390x/translate_vx.inc.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
target/s390x/translate_vx.inc.c +1 -1
index 41d5cf869f..0caddb3958 100644
@@ -213,7 +213,7 @@ static void get_vec_element_ptr_i64(TCGv_ptr ptr, uint8_t reg, TCGv_i64 enr,
                        vec_full_reg_offset(v3), ptr, 16, 16, data, fn)
 #define gen_gvec_3i(v1, v2, v3, c, gen) \
     tcg_gen_gvec_3i(vec_full_reg_offset(v1), vec_full_reg_offset(v2), \
-                    vec_full_reg_offset(v3), c, 16, 16, gen)
+                    vec_full_reg_offset(v3), 16, 16, c, gen)
 #define gen_gvec_4(v1, v2, v3, v4, gen) \
     tcg_gen_gvec_4(vec_full_reg_offset(v1), vec_full_reg_offset(v2), \
                    vec_full_reg_offset(v3), vec_full_reg_offset(v4), \
Signature · 1 line
2.17.1
#4 lore raw

PATCH 4/55 target/alpha: fix tlb_fill trap_arg2 value for instruction fetch

In reply to Michael Roth

From: Aurelien Jarno <[email protected]>

Commit e41c94529740cc26 ("target/alpha: Convert to CPUClass::tlb_fill") slightly changed the way the trap_arg2 value is computed in case of TLB fill. The type of the variable used in the ternary operator has been changed from an int to an enum. This causes the -1 value to not be sign-extended to 64-bit in case of an instruction fetch. The trap_arg2 ends up with 0xffffffff instead of 0xffffffffffffffff. Fix that by changing the -1 into -1LL.

This fixes the execution of user space processes in qemu-system-alpha.

Fixes e41c94529740cc26
[rth: Test MMU_DATA_LOAD and MMU_DATA_STORE instead of implying them.]
(cherry picked from commit cb1de55a83eaca9ee32be9c959dca99e11f2fea8)
---
 target/alpha/helper.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)
target/alpha/helper.c +3 -1
index 93b8e788b1..d0cc623192 100644
@@ -283,7 +283,9 @@ bool alpha_cpu_tlb_fill(CPUState *cs, vaddr addr, int size,
         cs->exception_index = EXCP_MMFAULT;
         env->trap_arg0 = addr;
         env->trap_arg1 = fail;
-        env->trap_arg2 = (access_type == MMU_INST_FETCH ? -1 : access_type);
+        env->trap_arg2 = (access_type == MMU_DATA_LOAD ? 0ull :
+                          access_type == MMU_DATA_STORE ? 1ull :
+                          /* access_type == MMU_INST_FETCH */ -1ull);
         cpu_loop_exit_restore(cs, retaddr);
     }
 
Signature · 1 line
2.17.1
#5 lore raw

PATCH 5/55 pc: Don't make die-id mandatory unless necessary

In reply to Michael Roth

From: Eduardo Habkost <[email protected]>

We have this issue reported when using libvirt to hotplug CPUs:
https://bugzilla.redhat.com/show_bug.cgi?id=1741451

Basically, libvirt is not copying die-id from query-hotpluggable-cpus, but die-id is now mandatory.

We could blame libvirt and say it is not following the documented interface, because we have this buried in the QAPI schema documentation:

Quoted 5 lines

Note: currently there are 5 properties that could be present but management should be prepared to pass through other properties with device_add command to allow for future interface extension. This also requires the filed names to be kept in sync with the properties passed to -device/device_add.

But I don't think this would be reasonable from us. We can just make QEMU more flexible and let die-id to be omitted when there's no ambiguity. This will allow us to keep compatibility with existing libvirt versions.

Test case included to ensure we don't break this again.

Fixes commit 176d2cda0dee ("i386/cpu: Consolidate die-id validity in smp context")

Message-Id: <[email protected]>

(cherry picked from commit fea374e7c8079563bca7c8fac895c6a880f76adc)
---
 hw/i386/pc.c                             |  8 ++++++
 tests/acceptance/pc_cpu_hotplug_props.py | 35 ++++++++++++++++++++++++
 2 files changed, 43 insertions(+)
 create mode 100644 tests/acceptance/pc_cpu_hotplug_props.py
hw/i386/pc.c +8 -0
index 549c437050..947f81070f 100644
@@ -2403,6 +2403,14 @@ static void pc_cpu_pre_plug(HotplugHandler *hotplug_dev,
         int max_socket = (ms->smp.max_cpus - 1) /
                                 smp_threads / smp_cores / pcms->smp_dies;
 
+        /*
+         * die-id was optional in QEMU 4.0 and older, so keep it optional
+         * if there's only one die per socket.
+         */
+        if (cpu->die_id < 0 && pcms->smp_dies == 1) {
+            cpu->die_id = 0;
+        }
+
         if (cpu->socket_id < 0) {
             error_setg(errp, "CPU socket-id is not set");
             return;
tests/acceptance/pc_cpu_hotplug_props.py new file +35 -0
new file mode 100644
index 0000000000..08b7e632c6
@@ -0,0 +1,35 @@
+#
+# Ensure CPU die-id can be omitted on -device
+#
+#  Copyright (c) 2019 Red Hat Inc
+#
+# Author:
+#  Eduardo Habkost <[email protected]>
+#
+# This library is free software; you can redistribute it and/or
+# modify it under the terms of the GNU Lesser General Public
+# License as published by the Free Software Foundation; either
+# version 2 of the License, or (at your option) any later version.
+#
+# This library is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+# Lesser General Public License for more details.
+#
+# You should have received a copy of the GNU Lesser General Public
+# License along with this library; if not, see <http://www.gnu.org/licenses/>.
+#
+
+from avocado_qemu import Test
+
+class OmittedCPUProps(Test):
+    """
+    :avocado: tags=arch:x86_64
+    """
+    def test_no_die_id(self):
+        self.vm.add_args('-nodefaults', '-S')
+        self.vm.add_args('-smp', '1,sockets=2,cores=2,threads=2,maxcpus=8')
+        self.vm.add_args('-cpu', 'qemu64')
+        self.vm.add_args('-device', 'qemu64-x86_64-cpu,socket-id=1,core-id=0,thread-id=0')
+        self.vm.launch()
+        self.assertEquals(len(self.vm.command('query-cpus')), 2)
Signature · 1 line
2.17.1
#6 lore raw

PATCH 6/55 xen-bus: Fix backend state transition on device reset

In reply to Michael Roth

From: Anthony PERARD <[email protected]>

When a frontend wants to reset its state and the backend one, it starts with setting "Closing", then waits for the backend (QEMU) to do the same.

But when QEMU is setting "Closing" to its state, it triggers an event (xenstore watch) that re-execute xen_device_backend_changed() and set the backend state to "Closed". QEMU should wait for the frontend to set "Closed" before doing the same.

Before setting "Closed" to the backend_state, we are also going to check if there is a frontend. If that the case, when the backend state is set to "Closing" the frontend should react and sets its state to "Closing" then "Closed". The backend should wait for that to happen.

Fixes b6af8926fb858c4f1426e5acb2cfc1f0580ec98a

Message-Id: <[email protected]>

(cherry picked from commit cb3231460747552d70af9d546dc53d8195bcb796)
---
 hw/xen/xen-bus.c | 23 ++++++++++++++++++++---
 1 file changed, 20 insertions(+), 3 deletions(-)
hw/xen/xen-bus.c +20 -3
index 7503eea9e9..5929aa4b2e 100644
@@ -516,6 +516,23 @@ static void xen_device_backend_set_online(XenDevice *xendev, bool online)
     xen_device_backend_printf(xendev, "online", "%u", online);
 }
 
+/*
+ * Tell from the state whether the frontend is likely alive,
+ * i.e. it will react to a change of state of the backend.
+ */
+static bool xen_device_state_is_active(enum xenbus_state state)
+{
+    switch (state) {
+    case XenbusStateInitWait:
+    case XenbusStateInitialised:
+    case XenbusStateConnected:
+    case XenbusStateClosing:
+        return true;
+    default:
+        return false;
+    }
+}
+
 static void xen_device_backend_changed(void *opaque)
 {
     XenDevice *xendev = opaque;
@@ -539,11 +556,11 @@ static void xen_device_backend_changed(void *opaque)
 
     /*
      * If the toolstack (or unplug request callback) has set the backend
-     * state to Closing, but there is no active frontend (i.e. the
-     * state is not Connected) then set the backend state to Closed.
+     * state to Closing, but there is no active frontend then set the
+     * backend state to Closed.
      */
     if (xendev->backend_state == XenbusStateClosing &&
-        xendev->frontend_state != XenbusStateConnected) {
+        !xen_device_state_is_active(state)) {
         xen_device_backend_set_state(xendev, XenbusStateClosed);
     }
 
Signature · 1 line
2.17.1
#7 lore raw

PATCH 7/55 xen-bus: check whether the frontend is active during device reset...

In reply to Michael Roth

From: Paul Durrant <[email protected]>

...not the backend

Commit cb323146 "xen-bus: Fix backend state transition on device reset" contained a subtle mistake. The hunk

@@ -539,11 +556,11 @@ static void xen_device_backend_changed(void *opaque)

/*
* If the toolstack (or unplug request callback) has set the backend

-     * state to Closing, but there is no active frontend (i.e. the
-     * state is not Connected) then set the backend state to Closed.
+     * state to Closing, but there is no active frontend then set the
+     * backend state to Closed.
      */
     if (xendev->backend_state == XenbusStateClosing &&
-        xendev->frontend_state != XenbusStateConnected) {
+        !xen_device_state_is_active(state)) {
         xen_device_backend_set_state(xendev, XenbusStateClosed);
     }

mistakenly replaced the check of 'xendev->frontend_state' with a check (now in a helper function) of 'state', which actually equates to 'xendev->backend_state'.

This patch fixes the mistake.

Fixes cb3231460747552d70af9d546dc53d8195bcb796

Message-Id: <[email protected]>

(cherry picked from commit df6180bb56cd03949c2c64083da58755fed81a61)
---
 hw/xen/xen-bus.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
hw/xen/xen-bus.c +1 -1
index 5929aa4b2e..10b7e02b5c 100644
@@ -560,7 +560,7 @@ static void xen_device_backend_changed(void *opaque)
      * backend state to Closed.
      */
     if (xendev->backend_state == XenbusStateClosing &&
-        !xen_device_state_is_active(state)) {
+        !xen_device_state_is_active(xendev->frontend_state)) {
         xen_device_backend_set_state(xendev, XenbusStateClosed);
     }
 
Signature · 1 line
2.17.1
#8 lore raw

PATCH 8/55 block/file-posix: Reduce xfsctl() use

In reply to Michael Roth

From: Max Reitz <[email protected]>

This patch removes xfs_write_zeroes() and xfs_discard(). Both functions have been added just before the same feature was present through fallocate():

- fallocate() has supported PUNCH_HOLE for XFS since Linux 2.6.38 (March 2011); xfs_discard() was added in December 2010.

- fallocate() has supported ZERO_RANGE for XFS since Linux 3.15 (June 2014); xfs_write_zeroes() was added in November 2013.

Nowadays, all systems that qemu runs on should support both fallocate() features (RHEL 7's kernel does).

xfsctl() is still useful for getting the request alignment for O_DIRECT, so this patch does not remove our dependency on it completely.

Note that xfs_write_zeroes() had a bug: It calls ftruncate() when the file is shorter than the specified range (because ZERO_RANGE does not increase the file length). ftruncate() may yield and then discard data that parallel write requests have written past the EOF in the meantime. Dropping the function altogether fixes the bug.

Fixes 50ba5b2d994853b38fed10e0841b119da0f8b8e5
(cherry picked from commit b2c6f23f4a9f6d8f1b648705cd46d3713b78d6a2)
---
 block/file-posix.c | 77 +---------------------------------------------
 1 file changed, 1 insertion(+), 76 deletions(-)
block/file-posix.c +1 -76
index 4479cc7ab4..992eb4a798 100644
@@ -1445,59 +1445,6 @@ out:
     }
 }
 
-#ifdef CONFIG_XFS
-static int xfs_write_zeroes(BDRVRawState *s, int64_t offset, uint64_t bytes)
-{
-    int64_t len;
-    struct xfs_flock64 fl;
-    int err;
-
-    len = lseek(s->fd, 0, SEEK_END);
-    if (len < 0) {
-        return -errno;
-    }
-
-    if (offset + bytes > len) {
-        /* XFS_IOC_ZERO_RANGE does not increase the file length */
-        if (ftruncate(s->fd, offset + bytes) < 0) {
-            return -errno;
-        }
-    }
-
-    memset(&fl, 0, sizeof(fl));
-    fl.l_whence = SEEK_SET;
-    fl.l_start = offset;
-    fl.l_len = bytes;
-
-    if (xfsctl(NULL, s->fd, XFS_IOC_ZERO_RANGE, &fl) < 0) {
-        err = errno;
-        trace_file_xfs_write_zeroes(strerror(errno));
-        return -err;
-    }
-
-    return 0;
-}
-
-static int xfs_discard(BDRVRawState *s, int64_t offset, uint64_t bytes)
-{
-    struct xfs_flock64 fl;
-    int err;
-
-    memset(&fl, 0, sizeof(fl));
-    fl.l_whence = SEEK_SET;
-    fl.l_start = offset;
-    fl.l_len = bytes;
-
-    if (xfsctl(NULL, s->fd, XFS_IOC_UNRESVSP64, &fl) < 0) {
-        err = errno;
-        trace_file_xfs_discard(strerror(errno));
-        return -err;
-    }
-
-    return 0;
-}
-#endif
-
 static int translate_err(int err)
 {
     if (err == -ENODEV || err == -ENOSYS || err == -EOPNOTSUPP ||
@@ -1553,10 +1500,8 @@ static ssize_t handle_aiocb_write_zeroes_block(RawPosixAIOData *aiocb)
 static int handle_aiocb_write_zeroes(void *opaque)
 {
     RawPosixAIOData *aiocb = opaque;
-#if defined(CONFIG_FALLOCATE) || defined(CONFIG_XFS)
-    BDRVRawState *s = aiocb->bs->opaque;
-#endif
 #ifdef CONFIG_FALLOCATE
+    BDRVRawState *s = aiocb->bs->opaque;
     int64_t len;
 #endif
 
@@ -1564,12 +1509,6 @@ static int handle_aiocb_write_zeroes(void *opaque)
         return handle_aiocb_write_zeroes_block(aiocb);
     }
 
-#ifdef CONFIG_XFS
-    if (s->is_xfs) {
-        return xfs_write_zeroes(s, aiocb->aio_offset, aiocb->aio_nbytes);
-    }
-#endif
-
 #ifdef CONFIG_FALLOCATE_ZERO_RANGE
     if (s->has_write_zeroes) {
         int ret = do_fallocate(s->fd, FALLOC_FL_ZERO_RANGE,
@@ -1632,14 +1571,6 @@ static int handle_aiocb_write_zeroes_unmap(void *opaque)
     }
 #endif
 
-#ifdef CONFIG_XFS
-    if (s->is_xfs) {
-        /* xfs_discard() guarantees that the discarded area reads as all-zero
-         * afterwards, so we can use it here. */
-        return xfs_discard(s, aiocb->aio_offset, aiocb->aio_nbytes);
-    }
-#endif
-
     /* If we couldn't manage to unmap while guaranteed that the area reads as
      * all-zero afterwards, just write zeroes without unmapping */
     ret = handle_aiocb_write_zeroes(aiocb);
@@ -1716,12 +1647,6 @@ static int handle_aiocb_discard(void *opaque)
         ret = -errno;
 #endif
     } else {
-#ifdef CONFIG_XFS
-        if (s->is_xfs) {
-            return xfs_discard(s, aiocb->aio_offset, aiocb->aio_nbytes);
-        }
-#endif
-
 #ifdef CONFIG_FALLOCATE_PUNCH_HOLE
         ret = do_fallocate(s->fd, FALLOC_FL_PUNCH_HOLE | FALLOC_FL_KEEP_SIZE,
                            aiocb->aio_offset, aiocb->aio_nbytes);
Signature · 1 line
2.17.1
#9 lore raw

PATCH 9/55 iotests: Test reverse sub-cluster qcow2 writes

In reply to Michael Roth

From: Max Reitz <[email protected]>

This exercises the regression introduced in commit 50ba5b2d994853b38fed10e0841b119da0f8b8e5. On my machine, it has close to a 50 % false-negative rate, but that should still be sufficient to test the fix.

(cherry picked from commit ae6ef0190981a21f2d4bc8dcee7253688f14fae7)

Conflicts:
tests/qemu-iotests/group
*fix context deps on tests not in 4.1.0

---
 tests/qemu-iotests/265     | 67 ++++++++++++++++++++++++++++++++++++++
 tests/qemu-iotests/265.out |  6 ++++
 tests/qemu-iotests/group   |  1 +
 3 files changed, 74 insertions(+)
 create mode 100755 tests/qemu-iotests/265
 create mode 100644 tests/qemu-iotests/265.out
tests/qemu-iotests/265 new file +67 -0
new file mode 100755
index 0000000000..dce6f77be3
@@ -0,0 +1,67 @@
+#!/usr/bin/env bash
+#
+# Test reverse-ordered qcow2 writes on a sub-cluster level
+#
+# Copyright (C) 2019 Red Hat, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+#
+
+seq=$(basename $0)
+echo "QA output created by $seq"
+
+status=1	# failure is the default!
+
+_cleanup()
+{
+    _cleanup_test_img
+}
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+# get standard environment, filters and checks
+. ./common.rc
+. ./common.filter
+
+# qcow2-specific test
+_supported_fmt qcow2
+_supported_proto file
+_supported_os Linux
+
+echo '--- Writing to the image ---'
+
+# Reduce cluster size so we get more and quicker I/O
+IMGOPTS='cluster_size=4096' _make_test_img 1M
+(for ((kb = 1024 - 4; kb >= 0; kb -= 4)); do \
+     echo "aio_write -P 42 $((kb + 1))k 2k"; \
+ done) \
+ | $QEMU_IO "$TEST_IMG" > /dev/null
+
+echo '--- Verifying its content ---'
+
+(for ((kb = 0; kb < 1024; kb += 4)); do \
+    echo "read -P 0 ${kb}k 1k"; \
+    echo "read -P 42 $((kb + 1))k 2k"; \
+    echo "read -P 0 $((kb + 3))k 1k"; \
+ done) \
+ | $QEMU_IO "$TEST_IMG" | _filter_qemu_io | grep 'verification'
+
+# Status of qemu-io
+if [ ${PIPESTATUS[1]} = 0 ]; then
+    echo 'Content verified.'
+fi
+
+# success, all done
+echo "*** done"
+rm -f $seq.full
+status=0
tests/qemu-iotests/265.out new file +6 -0
new file mode 100644
index 0000000000..6eac620f25
@@ -0,0 +1,6 @@
+QA output created by 265
+--- Writing to the image ---
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576
+--- Verifying its content ---
+Content verified.
+*** done
tests/qemu-iotests/group +1 -0
index f13e5f2e23..468458efb1 100644
@@ -271,3 +271,4 @@
 254 rw backing quick
 255 rw quick
 256 rw quick
+265 rw auto quick
Signature · 1 line
2.17.1
#10 lore raw

PATCH 10/55 pr-manager: Fix invalid g_free() crash bug

In reply to Michael Roth

From: Markus Armbruster <[email protected]>

pr_manager_worker() passes its @opaque argument to g_free(). Wrong; it points to pr_manager_worker()'s automatic @data. Broken when commit 2f3a7ab39be converted @data from heap- to stack-allocated. Fix by deleting the g_free().

Fixes 2f3a7ab39bec4ba8022dc4d42ea641165b004e3e
(cherry picked from commit 6b9d62c2a9e83bbad73fb61406f0ff69b46ff6f3)
---
 scsi/pr-manager.c | 1 -
 1 file changed, 1 deletion(-)
scsi/pr-manager.c +0 -1
index ee43663576..0c866e8698 100644
@@ -39,7 +39,6 @@ static int pr_manager_worker(void *opaque)
     int fd = data->fd;
     int r;
 
-    g_free(data);
     trace_pr_manager_run(fd, hdr->cmdp[0], hdr->cmdp[1]);
 
     /* The reference was taken in pr_manager_execute.  */
Signature · 1 line
2.17.1
#11 lore raw

PATCH 11/55 x86: do not advertise die-id in query-hotpluggbale-cpus if '-smp dies' is not set

In reply to Michael Roth

From: Igor Mammedov <[email protected]>

Commit 176d2cda0 (i386/cpu: Consolidate die-id validity in smp context) added new 'die-id' topology property to CPUs and exposed it via QMP command query-hotpluggable-cpus, which broke -device/device_add cpu-foo for existing users that do not support die-id/dies yet. That's would be fine if it happened to new machine type only but it also happened to old machine types, which breaks migration from old QEMU to the new one, for example following CLI:

  OLD-QEMU -M pc-i440fx-4.0 -smp 1,max_cpus=2 \
           -device qemu64-x86_64-cpu,socket-id=1,core-id=0,thread-id

is not able to start with new QEMU, complaining about invalid die-id.

After discovering regression, the patch
"pc: Don't make die-id mandatory unless necessary"
makes die-id optional so old CLI would work.

However it's not enough as new QEMU still exposes die-id via query-hotpluggbale-cpus QMP command, so the users that started old machine type on new QEMU, using all properties (including die-id) received from QMP command (as required), won't be able to start old QEMU using the same properties since it doesn't support die-id.

Fix it by hiding die-id in query-hotpluggbale-cpus for all machine types in case '-smp dies' is not provided on CLI or -smp dies = 1', in which case smp_dies == 1 and APIC ID is calculated in default way (as it was before DIE support) so we won't need compat code as in both cases the topology provided to guest via CPUID is the same.

Message-Id: <[email protected]>

(cherry picked from commit c6c1bb89fb46f3b88f832e654cf5a6f7941aac51)
---
 hw/i386/pc.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)
hw/i386/pc.c +4 -2
index 947f81070f..d011733ff7 100644
@@ -2887,8 +2887,10 @@ static const CPUArchIdList *pc_possible_cpu_arch_ids(MachineState *ms)
                                  ms->smp.threads, &topo);
         ms->possible_cpus->cpus[i].props.has_socket_id = true;
         ms->possible_cpus->cpus[i].props.socket_id = topo.pkg_id;
-        ms->possible_cpus->cpus[i].props.has_die_id = true;
-        ms->possible_cpus->cpus[i].props.die_id = topo.die_id;
+        if (pcms->smp_dies > 1) {
+            ms->possible_cpus->cpus[i].props.has_die_id = true;
+            ms->possible_cpus->cpus[i].props.die_id = topo.die_id;
+        }
         ms->possible_cpus->cpus[i].props.has_core_id = true;
         ms->possible_cpus->cpus[i].props.core_id = topo.core_id;
         ms->possible_cpus->cpus[i].props.has_thread_id = true;
Signature · 1 line
2.17.1
#12 lore raw

PATCH 12/55 vpc: Return 0 from vpc_co_create() on success

In reply to Michael Roth

From: Max Reitz <[email protected]>

blockdev_create_run() directly uses .bdrv_co_create()'s return value as the job's return value. Jobs must return 0 on success, not just any nonnegative value. Therefore, using blockdev-create for VPC images may currently fail as the vpc driver may return a positive integer.

Because there is no point in returning a positive integer anywhere in the block layer (all non-negative integers are generally treated as complete success), we probably do not want to add more such cases. Therefore, fix this problem by making the vpc driver always return 0 in case of success.

(cherry picked from commit 1a37e3124407b5a145d44478d3ecbdb89c63789f)
---
 block/vpc.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
block/vpc.c +2 -1
index d4776ee8a5..3a88e28e2b 100644
@@ -885,6 +885,7 @@ static int create_dynamic_disk(BlockBackend *blk, uint8_t *buf,
         goto fail;
     }
 
+    ret = 0;
  fail:
     return ret;
 }
@@ -908,7 +909,7 @@ static int create_fixed_disk(BlockBackend *blk, uint8_t *buf,
         return ret;
     }
 
-    return ret;
+    return 0;
 }
 
 static int calculate_rounded_image_size(BlockdevCreateOptionsVpc *vpc_opts,
Signature · 1 line
2.17.1
#13 lore raw

PATCH 13/55 iotests: add testing shim for script-style python tests

In reply to Michael Roth

From: John Snow <[email protected]>

Because the new-style python tests don't use the iotests.main() test launcher, we don't turn on the debugger logging for these scripts when invoked via ./check -d.

Refactor the launcher shim into new and old style shims so that they share environmental configuration.

Two cleanup notes: debug was not actually used as a global, and there was no reason to create a class in an inner scope just to achieve default variables; we can simply create an instance of the runner with the values we want instead.

Message-id: [email protected]

(cherry picked from commit 456a2d5ac7641c7e75c76328a561b528a8607a8e)

*prereq for 88d2aa533a

---
 tests/qemu-iotests/iotests.py | 40 +++++++++++++++++++++++------------
 1 file changed, 26 insertions(+), 14 deletions(-)
tests/qemu-iotests/iotests.py +26 -14
index ce74177ab1..25c5a047b3 100644
@@ -61,7 +61,6 @@ cachemode = os.environ.get('CACHEMODE')
 qemu_default_machine = os.environ.get('QEMU_DEFAULT_MACHINE')
 
 socket_scm_helper = os.environ.get('SOCKET_SCM_HELPER', 'socket_scm_helper')
-debug = False
 
 luks_default_secret_object = 'secret,id=keysec0,data=' + \
                              os.environ.get('IMGKEYSECRET', '')
@@ -842,11 +841,22 @@ def skip_if_unsupported(required_formats=[], read_only=False):
         return func_wrapper
     return skip_test_decorator
 
-def main(supported_fmts=[], supported_oses=['linux'], supported_cache_modes=[],
-         unsupported_fmts=[]):
-    '''Run tests'''
+def execute_unittest(output, verbosity, debug):
+    runner = unittest.TextTestRunner(stream=output, descriptions=True,
+                                     verbosity=verbosity)
+    try:
+        # unittest.main() will use sys.exit(); so expect a SystemExit
+        # exception
+        unittest.main(testRunner=runner)
+    finally:
+        if not debug:
+            sys.stderr.write(re.sub(r'Ran (\d+) tests? in [\d.]+s',
+                                    r'Ran \1 tests', output.getvalue()))
 
-    global debug
+def execute_test(test_function=None,
+                 supported_fmts=[], supported_oses=['linux'],
+                 supported_cache_modes=[], unsupported_fmts=[]):
+    """Run either unittest or script-style tests."""
 
     # We are using TEST_DIR and QEMU_DEFAULT_MACHINE as proxies to
     # indicate that we're not being run via "check". There may be
@@ -878,13 +888,15 @@ def main(supported_fmts=[], supported_oses=['linux'], supported_cache_modes=[],
 
     logging.basicConfig(level=(logging.DEBUG if debug else logging.WARN))
 
-    class MyTestRunner(unittest.TextTestRunner):
-        def __init__(self, stream=output, descriptions=True, verbosity=verbosity):
-            unittest.TextTestRunner.__init__(self, stream, descriptions, verbosity)
+    if not test_function:
+        execute_unittest(output, verbosity, debug)
+    else:
+        test_function()
+
+def script_main(test_function, *args, **kwargs):
+    """Run script-style tests outside of the unittest framework"""
+    execute_test(test_function, *args, **kwargs)
 
-    # unittest.main() will use sys.exit() so expect a SystemExit exception
-    try:
-        unittest.main(testRunner=MyTestRunner)
-    finally:
-        if not debug:
-            sys.stderr.write(re.sub(r'Ran (\d+) tests? in [\d.]+s', r'Ran \1 tests', output.getvalue()))
+def main(*args, **kwargs):
+    """Run tests using the unittest framework"""
+    execute_test(None, *args, **kwargs)
Signature · 1 line
2.17.1
#14 lore raw

PATCH 14/55 iotests: Add supported protocols to execute_test()

In reply to Michael Roth

From: Max Reitz <[email protected]>

(cherry picked from commit 88d2aa533a4a1aad44a27c2e6cd5bc5fbcbce7ed)
---
 tests/qemu-iotests/iotests.py | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)
tests/qemu-iotests/iotests.py +3 -1
index 25c5a047b3..2f7edc2f33 100644
@@ -855,7 +855,8 @@ def execute_unittest(output, verbosity, debug):
 
 def execute_test(test_function=None,
                  supported_fmts=[], supported_oses=['linux'],
-                 supported_cache_modes=[], unsupported_fmts=[]):
+                 supported_cache_modes=[], unsupported_fmts=[],
+                 supported_protocols=[], unsupported_protocols=[]):
     """Run either unittest or script-style tests."""
 
     # We are using TEST_DIR and QEMU_DEFAULT_MACHINE as proxies to
@@ -869,6 +870,7 @@ def execute_test(test_function=None,
     debug = '-d' in sys.argv
     verbosity = 1
     verify_image_format(supported_fmts, unsupported_fmts)
+    verify_protocol(supported_protocols, unsupported_protocols)
     verify_platform(supported_oses)
     verify_cache_mode(supported_cache_modes)
 
Signature · 1 line
2.17.1
#15 lore raw

PATCH 15/55 iotests: Restrict file Python tests to file

In reply to Michael Roth

From: Max Reitz <[email protected]>

Most of our Python unittest-style tests only support the file protocol. You can run them with any other protocol, but the test will simply ignore your choice and use file anyway.

We should let them signal that they require the file protocol so they are skipped when you want to test some other protocol.

(cherry picked from commit 103cbc771e5660d1f5bb458be80aa9e363547ae0)

Conflicts:

	tests/qemu-iotests/257

*drop changes for tests not in 4.1.0

---
 tests/qemu-iotests/030 | 3 ++-
 tests/qemu-iotests/040 | 3 ++-
 tests/qemu-iotests/041 | 3 ++-
 tests/qemu-iotests/044 | 3 ++-
 tests/qemu-iotests/045 | 3 ++-
 tests/qemu-iotests/055 | 3 ++-
 tests/qemu-iotests/056 | 3 ++-
 tests/qemu-iotests/057 | 3 ++-
 tests/qemu-iotests/065 | 3 ++-
 tests/qemu-iotests/096 | 3 ++-
 tests/qemu-iotests/118 | 3 ++-
 tests/qemu-iotests/124 | 3 ++-
 tests/qemu-iotests/129 | 3 ++-
 tests/qemu-iotests/132 | 3 ++-
 tests/qemu-iotests/139 | 3 ++-
 tests/qemu-iotests/148 | 3 ++-
 tests/qemu-iotests/151 | 3 ++-
 tests/qemu-iotests/152 | 3 ++-
 tests/qemu-iotests/155 | 3 ++-
 tests/qemu-iotests/163 | 3 ++-
 tests/qemu-iotests/165 | 3 ++-
 tests/qemu-iotests/169 | 3 ++-
 tests/qemu-iotests/196 | 3 ++-
 tests/qemu-iotests/199 | 3 ++-
 tests/qemu-iotests/245 | 3 ++-
 25 files changed, 50 insertions(+), 25 deletions(-)
tests/qemu-iotests/030 +2 -1
index 1b69f318c6..f3766f2a81 100755
@@ -957,4 +957,5 @@ class TestSetSpeed(iotests.QMPTestCase):
         self.cancel_and_wait(resume=True)
 
 if __name__ == '__main__':
-    iotests.main(supported_fmts=['qcow2', 'qed'])
+    iotests.main(supported_fmts=['qcow2', 'qed'],
+                 supported_protocols=['file'])
tests/qemu-iotests/040 +2 -1
index aa0b1847e3..f9e603e715 100755
@@ -433,4 +433,5 @@ class TestReopenOverlay(ImageCommitTestCase):
         self.run_commit_test(self.img1, self.img0)
 
 if __name__ == '__main__':
-    iotests.main(supported_fmts=['qcow2', 'qed'])
+    iotests.main(supported_fmts=['qcow2', 'qed'],
+                 supported_protocols=['file'])
tests/qemu-iotests/041 +2 -1
index 26bf1701eb..ae6ed952c6 100755
@@ -1068,4 +1068,5 @@ class TestOrphanedSource(iotests.QMPTestCase):
         self.assert_qmp(result, 'error/class', 'GenericError')
 
 if __name__ == '__main__':
-    iotests.main(supported_fmts=['qcow2', 'qed'])
+    iotests.main(supported_fmts=['qcow2', 'qed'],
+                 supported_protocols=['file'])
tests/qemu-iotests/044 +2 -1
index 9ec3dba734..05ea1f49c5 100755
@@ -118,4 +118,5 @@ class TestRefcountTableGrowth(iotests.QMPTestCase):
         pass
 
 if __name__ == '__main__':
-    iotests.main(supported_fmts=['qcow2'])
+    iotests.main(supported_fmts=['qcow2'],
+                 supported_protocols=['file'])
tests/qemu-iotests/045 +2 -1
index d5484a0ee1..01cc038884 100755
@@ -175,4 +175,5 @@ class TestSCMFd(iotests.QMPTestCase):
             "File descriptor named '%s' not found" % fdname)
 
 if __name__ == '__main__':
-    iotests.main(supported_fmts=['raw'])
+    iotests.main(supported_fmts=['raw'],
+                 supported_protocols=['file'])
tests/qemu-iotests/055 +2 -1
index 3437c11507..c732a112d6 100755
@@ -563,4 +563,5 @@ class TestDriveCompression(iotests.QMPTestCase):
                                         target='drive1')
 
 if __name__ == '__main__':
-    iotests.main(supported_fmts=['raw', 'qcow2'])
+    iotests.main(supported_fmts=['raw', 'qcow2'],
+                 supported_protocols=['file'])
tests/qemu-iotests/056 +2 -1
index e761e465ae..98c55d8e5a 100755
@@ -335,4 +335,5 @@ class BackupTest(iotests.QMPTestCase):
         self.dismissal_failure(True)
 
 if __name__ == '__main__':
-    iotests.main(supported_fmts=['qcow2', 'qed'])
+    iotests.main(supported_fmts=['qcow2', 'qed'],
+                 supported_protocols=['file'])
tests/qemu-iotests/057 +2 -1
index 9f0a5a3057..9fbba759b6 100755
@@ -256,4 +256,5 @@ class TestSnapshotDelete(ImageSnapshotTestCase):
         self.assert_qmp(result, 'error/class', 'GenericError')
 
 if __name__ == '__main__':
-    iotests.main(supported_fmts=['qcow2'])
+    iotests.main(supported_fmts=['qcow2'],
+                 supported_protocols=['file'])
tests/qemu-iotests/065 +2 -1
index 8bac383ea7..5b21eb96bd 100755
@@ -129,4 +129,5 @@ TestQemuImgInfo = None
 TestQMP = None
 
 if __name__ == '__main__':
-    iotests.main(supported_fmts=['qcow2'])
+    iotests.main(supported_fmts=['qcow2'],
+                 supported_protocols=['file'])
tests/qemu-iotests/096 +2 -1
index a69439602d..ab9cb47822 100755
@@ -67,4 +67,5 @@ class TestLiveSnapshot(iotests.QMPTestCase):
         self.checkConfig('target')
 
 if __name__ == '__main__':
-    iotests.main(supported_fmts=['qcow2'])
+    iotests.main(supported_fmts=['qcow2'],
+                 supported_protocols=['file'])
tests/qemu-iotests/118 +2 -1
index 499c5f0901..c7d8af1a85 100755
@@ -707,4 +707,5 @@ if __name__ == '__main__':
                        iotests.qemu_default_machine)
     # Need to support image creation
     iotests.main(supported_fmts=['vpc', 'parallels', 'qcow', 'vdi', 'qcow2',
-                                 'vmdk', 'raw', 'vhdx', 'qed'])
+                                 'vmdk', 'raw', 'vhdx', 'qed'],
+                 supported_protocols=['file'])
tests/qemu-iotests/124 +2 -1
index 3440f54781..ca40ba3be2 100755
@@ -779,4 +779,5 @@ class TestIncrementalBackupBlkdebug(TestIncrementalBackupBase):
 
 
 if __name__ == '__main__':
-    iotests.main(supported_fmts=['qcow2'])
+    iotests.main(supported_fmts=['qcow2'],
+                 supported_protocols=['file'])
tests/qemu-iotests/129 +2 -1
index 9e87e1c8d9..cd6b9e9ce7 100755
@@ -83,4 +83,5 @@ class TestStopWithBlockJob(iotests.QMPTestCase):
         self.do_test_stop("block-commit", device="drive0")
 
 if __name__ == '__main__':
-    iotests.main(supported_fmts=["qcow2"])
+    iotests.main(supported_fmts=["qcow2"],
+                 supported_protocols=["file"])
tests/qemu-iotests/132 +2 -1
index f53ef6e391..0f2a106c81 100755
@@ -56,4 +56,5 @@ class TestSingleDrive(iotests.QMPTestCase):
                         'target image does not match source after mirroring')
 
 if __name__ == '__main__':
-    iotests.main(supported_fmts=['raw', 'qcow2'])
+    iotests.main(supported_fmts=['raw', 'qcow2'],
+                 supported_protocols=['file'])
tests/qemu-iotests/139 +2 -1
index 933b45121a..8dc839ec7e 100755
@@ -361,4 +361,5 @@ class TestBlockdevDel(iotests.QMPTestCase):
 
 
 if __name__ == '__main__':
-    iotests.main(supported_fmts=["qcow2"])
+    iotests.main(supported_fmts=["qcow2"],
+                 supported_protocols=["file"])
tests/qemu-iotests/148 +2 -1
index e01b061fe7..8c11c53cba 100755
@@ -137,4 +137,5 @@ class TestFifoQuorumEvents(TestQuorumEvents):
 
 if __name__ == '__main__':
     iotests.verify_quorum()
-    iotests.main(supported_fmts=["raw"])
+    iotests.main(supported_fmts=["raw"],
+                 supported_protocols=["file"])
tests/qemu-iotests/151 +2 -1
index ad7359fc8d..76ae265cc1 100755
@@ -142,4 +142,5 @@ class TestActiveMirror(iotests.QMPTestCase):
 
 
 if __name__ == '__main__':
-    iotests.main(supported_fmts=['qcow2', 'raw'])
+    iotests.main(supported_fmts=['qcow2', 'raw'],
+                 supported_protocols=['file'])
tests/qemu-iotests/152 +2 -1
index fec546d033..732bf5f062 100755
@@ -59,4 +59,5 @@ class TestUnaligned(iotests.QMPTestCase):
 
 
 if __name__ == '__main__':
-    iotests.main(supported_fmts=['raw', 'qcow2'])
+    iotests.main(supported_fmts=['raw', 'qcow2'],
+                 supported_protocols=['file'])
tests/qemu-iotests/155 +2 -1
index 63a5b5e2c0..e19485911c 100755
@@ -258,4 +258,5 @@ BaseClass = None
 MirrorBaseClass = None
 
 if __name__ == '__main__':
-    iotests.main(supported_fmts=['qcow2'])
+    iotests.main(supported_fmts=['qcow2'],
+                 supported_protocols=['file'])
tests/qemu-iotests/163 +2 -1
index 158ba5d092..081ccc8ac1 100755
@@ -170,4 +170,5 @@ class TestShrink1M(ShrinkBaseClass):
 ShrinkBaseClass = None
 
 if __name__ == '__main__':
-    iotests.main(supported_fmts=['raw', 'qcow2'])
+    iotests.main(supported_fmts=['raw', 'qcow2'],
+                 supported_protocols=['file'])
tests/qemu-iotests/165 +2 -1
index 88f62d3c6d..5650dc7c87 100755
@@ -103,4 +103,5 @@ class TestPersistentDirtyBitmap(iotests.QMPTestCase):
         self.vm.shutdown()
 
 if __name__ == '__main__':
-    iotests.main(supported_fmts=['qcow2'])
+    iotests.main(supported_fmts=['qcow2'],
+                 supported_protocols=['file'])
tests/qemu-iotests/169 +2 -1
index 7e06cc1145..8c204caf20 100755
@@ -227,4 +227,5 @@ for cmb in list(itertools.product((True, False), repeat=2)):
                      'do_test_migration_resume_source', *list(cmb))
 
 if __name__ == '__main__':
-    iotests.main(supported_fmts=['qcow2'])
+    iotests.main(supported_fmts=['qcow2'],
+                 supported_protocols=['file'])
tests/qemu-iotests/196 +2 -1
index 4116ebc92b..92fe9244f8 100755
@@ -63,4 +63,5 @@ class TestInvalidateAutoclear(iotests.QMPTestCase):
             self.assertEqual(f.read(1), b'\x00')
 
 if __name__ == '__main__':
-    iotests.main(supported_fmts=['qcow2'])
+    iotests.main(supported_fmts=['qcow2'],
+                 supported_protocols=['file'])
tests/qemu-iotests/199 +2 -1
index 651e8df5d9..a2c8ecab5a 100755
@@ -115,4 +115,5 @@ class TestDirtyBitmapPostcopyMigration(iotests.QMPTestCase):
         self.assert_qmp(result, 'return/sha256', sha256);
 
 if __name__ == '__main__':
-    iotests.main(supported_fmts=['qcow2'], supported_cache_modes=['none'])
+    iotests.main(supported_fmts=['qcow2'], supported_cache_modes=['none'],
+                 supported_protocols=['file'])
tests/qemu-iotests/245 +2 -1
index bc1ceb9792..41218d5f1d 100644
@@ -1000,4 +1000,5 @@ class TestBlockdevReopen(iotests.QMPTestCase):
         self.reopen(opts, {'backing': 'hd2'})
 
 if __name__ == '__main__':
-    iotests.main(supported_fmts=["qcow2"])
+    iotests.main(supported_fmts=["qcow2"],
+                 supported_protocols=["file"])
Signature · 1 line
2.17.1
#16 lore raw

PATCH 16/55 iotests: Restrict nbd Python tests to nbd

In reply to Michael Roth

From: Max Reitz <[email protected]>

We have two Python unittest-style tests that test NBD. As such, they should specify supported_protocols=['nbd'] so they are skipped when the user wants to test some other protocol.

Furthermore, we should restrict their choice of formats to 'raw'. The idea of a protocol/format combination is to use some format over some protocol; but we always use the raw format over NBD. It does not really matter what the NBD server uses on its end, and it is not a useful test of the respective format driver anyway.

(cherry picked from commit 7c932a1d69a6d6ac5c0b615c11d191da3bbe9aa8)
---
 tests/qemu-iotests/147 | 5 ++---
 tests/qemu-iotests/205 | 3 ++-
 2 files changed, 4 insertions(+), 4 deletions(-)
tests/qemu-iotests/147 +2 -3
index 2d84fddb01..ab8480b9a4 100755
@@ -287,6 +287,5 @@ class BuiltinNBD(NBDBlockdevAddBase):
 
 
 if __name__ == '__main__':
-    # Need to support image creation
-    iotests.main(supported_fmts=['vpc', 'parallels', 'qcow', 'vdi', 'qcow2',
-                                 'vmdk', 'raw', 'vhdx', 'qed'])
+    iotests.main(supported_fmts=['raw'],
+                 supported_protocols=['nbd'])
tests/qemu-iotests/205 +2 -1
index b8a86c446e..76f6c5fa2b 100755
@@ -153,4 +153,5 @@ class TestNbdServerRemove(iotests.QMPTestCase):
 
 
 if __name__ == '__main__':
-    iotests.main(supported_fmts=['generic'])
+    iotests.main(supported_fmts=['raw'],
+                 supported_protocols=['nbd'])
Signature · 1 line
2.17.1
#17 lore raw

PATCH 17/55 iotests: Test blockdev-create for vpc

In reply to Michael Roth

From: Max Reitz <[email protected]>

(cherry picked from commit cb73747e1a47b93d3dfdc3f769c576b053916938)
---
 tests/qemu-iotests/266     | 153 +++++++++++++++++++++++++++++++++++++
 tests/qemu-iotests/266.out | 137 +++++++++++++++++++++++++++++++++
 tests/qemu-iotests/group   |   1 +
 3 files changed, 291 insertions(+)
 create mode 100755 tests/qemu-iotests/266
 create mode 100644 tests/qemu-iotests/266.out
tests/qemu-iotests/266 new file +153 -0
new file mode 100755
index 0000000000..5b35cd67e4
@@ -0,0 +1,153 @@
+#!/usr/bin/env python
+#
+# Test VPC and file image creation
+#
+# Copyright (C) 2019 Red Hat, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+#
+
+import iotests
+from iotests import imgfmt
+
+
+def blockdev_create(vm, options):
+    result = vm.qmp_log('blockdev-create', job_id='job0', options=options,
+                        filters=[iotests.filter_qmp_testfiles])
+
+    if 'return' in result:
+        assert result['return'] == {}
+        vm.run_job('job0')
+
+
+# Successful image creation (defaults)
+def implicit_defaults(vm, file_path):
+    iotests.log("=== Successful image creation (defaults) ===")
+    iotests.log("")
+
+    # 8 heads, 964 cyls/head, 17 secs/cyl
+    # (Close to 64 MB)
+    size = 8 * 964 * 17 * 512
+
+    blockdev_create(vm, { 'driver': imgfmt,
+                          'file': 'protocol-node',
+                          'size': size })
+
+
+# Successful image creation (explicit defaults)
+def explicit_defaults(vm, file_path):
+    iotests.log("=== Successful image creation (explicit defaults) ===")
+    iotests.log("")
+
+    # 16 heads, 964 cyls/head, 17 secs/cyl
+    # (Close to 128 MB)
+    size = 16 * 964 * 17 * 512
+
+    blockdev_create(vm, { 'driver': imgfmt,
+                          'file': 'protocol-node',
+                          'size': size,
+                          'subformat': 'dynamic',
+                          'force-size': False })
+
+
+# Successful image creation (non-default options)
+def non_defaults(vm, file_path):
+    iotests.log("=== Successful image creation (non-default options) ===")
+    iotests.log("")
+
+    # Not representable in CHS (fine with force-size=True)
+    size = 1048576
+
+    blockdev_create(vm, { 'driver': imgfmt,
+                          'file': 'protocol-node',
+                          'size': size,
+                          'subformat': 'fixed',
+                          'force-size': True })
+
+
+# Size not representable in CHS with force-size=False
+def non_chs_size_without_force(vm, file_path):
+    iotests.log("=== Size not representable in CHS ===")
+    iotests.log("")
+
+    # Not representable in CHS (will not work with force-size=False)
+    size = 1048576
+
+    blockdev_create(vm, { 'driver': imgfmt,
+                          'file': 'protocol-node',
+                          'size': size,
+                          'force-size': False })
+
+
+# Zero size
+def zero_size(vm, file_path):
+    iotests.log("=== Zero size===")
+    iotests.log("")
+
+    blockdev_create(vm, { 'driver': imgfmt,
+                          'file': 'protocol-node',
+                          'size': 0 })
+
+
+# Maximum CHS size
+def maximum_chs_size(vm, file_path):
+    iotests.log("=== Maximum CHS size===")
+    iotests.log("")
+
+    blockdev_create(vm, { 'driver': imgfmt,
+                          'file': 'protocol-node',
+                          'size': 16 * 65535 * 255 * 512 })
+
+
+# Actual maximum size
+def maximum_size(vm, file_path):
+    iotests.log("=== Actual maximum size===")
+    iotests.log("")
+
+    blockdev_create(vm, { 'driver': imgfmt,
+                          'file': 'protocol-node',
+                          'size': 0xff000000 * 512,
+                          'force-size': True })
+
+
+def main():
+    for test_func in [implicit_defaults, explicit_defaults, non_defaults,
+                      non_chs_size_without_force, zero_size, maximum_chs_size,
+                      maximum_size]:
+
+        with iotests.FilePath('t.vpc') as file_path, \
+             iotests.VM() as vm:
+
+            vm.launch()
+
+            iotests.log('--- Creating empty file ---')
+            blockdev_create(vm, { 'driver': 'file',
+                                  'filename': file_path,
+                                  'size': 0 })
+
+            vm.qmp_log('blockdev-add', driver='file', filename=file_path,
+                       node_name='protocol-node',
+                       filters=[iotests.filter_qmp_testfiles])
+            iotests.log('')
+
+            print_info = test_func(vm, file_path)
+            iotests.log('')
+
+            vm.shutdown()
+            iotests.img_info_log(file_path)
+
+
+iotests.script_main(main,
+                    supported_fmts=['vpc'],
+                    supported_protocols=['file'])
tests/qemu-iotests/266.out new file +137 -0

137 lines not on this page · raw message

tests/qemu-iotests/group +1 -0
index 468458efb1..3660a741f2 100644
@@ -272,3 +272,4 @@
 255 rw quick
 256 rw quick
 265 rw auto quick
+266 rw quick
Signature · 1 line
2.17.1
#18 lore raw

PATCH 18/55 target/arm: Free TCG temps in trans_VMOV_64_sp()

In reply to Michael Roth

From: Peter Maydell <[email protected]>

The function neon_store_reg32() doesn't free the TCG temp that it is passed, so the caller must do that. We got this right in most places but forgot to free the TCG temps in trans_VMOV_64_sp().

Message-id: [email protected]

(cherry picked from commit 342d27581bd3ecdb995e4fc55fcd383cf3242888)
---
 target/arm/translate-vfp.inc.c | 2 ++
 1 file changed, 2 insertions(+)
target/arm/translate-vfp.inc.c +2 -0
index 092eb5ec53..ef45cecbea 100644
@@ -881,8 +881,10 @@ static bool trans_VMOV_64_sp(DisasContext *s, arg_VMOV_64_sp *a)
         /* gpreg to fpreg */
         tmp = load_reg(s, a->rt);
         neon_store_reg32(tmp, a->vm);
+        tcg_temp_free_i32(tmp);
         tmp = load_reg(s, a->rt2);
         neon_store_reg32(tmp, a->vm + 1);
+        tcg_temp_free_i32(tmp);
     }
 
     return true;
Signature · 1 line
2.17.1
#19 lore raw

PATCH 19/55 target/arm: Don't abort on M-profile exception return in linux-user mode

In reply to Michael Roth

From: Peter Maydell <[email protected]>

An attempt to do an exception-return (branch to one of the magic addresses) in linux-user mode for M-profile should behave like a normal branch, because linux-user mode is always going to be in 'handler' mode. This used to work, but we broke it when we added support for the M-profile security extension in commit d02a8698d7ae2bfed.

In that commit we allowed even handler-mode calls to magic return values to be checked for and dealt with by causing an EXCP_EXCEPTION_EXIT exception to be taken, because this is needed for the FNC_RETURN return-from-non-secure-function-call handling. For system mode we added a check in do_v7m_exception_exit() to make any spurious calls from Handler mode behave correctly, but forgot that linux-user mode would also be affected.

How an attempted return-from-non-secure-function-call in linux-user mode should be handled is not clear -- on real hardware it would result in return to secure code (not to the Linux kernel) which could then handle the error in any way it chose. For QEMU we take the simple approach of treating this erroneous return the same way it would be handled on a CPU without the security extensions -- treat it as a normal branch.

The upshot of all this is that for linux-user mode we should never do any of the bx_excret magic, so the code change is simple.

This ought to be a weird corner case that only affects broken guest code (because Linux user processes should never be attempting to do exception returns or NS function returns), except that the code that assigns addresses in RAM for the process and stack in our linux-user code does not attempt to avoid this magic address range, so legitimate code attempting to return to a trampoline routine on the stack can fall into this case. This change fixes those programs, but we should also look at restricting the range of memory we use for M-profile linux-user guests to the area that would be real RAM in hardware.

Message-id: [email protected]

(cherry picked from commit 5e5584c89f36b302c666bc6db535fd3f7ff35ad2)
---
 target/arm/translate.c | 21 ++++++++++++++++++++-
 1 file changed, 20 insertions(+), 1 deletion(-)
target/arm/translate.c +20 -1
index 7853462b21..24cb4ba075 100644
@@ -952,10 +952,27 @@ static inline void gen_bx(DisasContext *s, TCGv_i32 var)
     store_cpu_field(var, thumb);
 }
 
-/* Set PC and Thumb state from var. var is marked as dead.
+/*
+ * Set PC and Thumb state from var. var is marked as dead.
  * For M-profile CPUs, include logic to detect exception-return
  * branches and handle them. This is needed for Thumb POP/LDM to PC, LDR to PC,
  * and BX reg, and no others, and happens only for code in Handler mode.
+ * The Security Extension also requires us to check for the FNC_RETURN
+ * which signals a function return from non-secure state; this can happen
+ * in both Handler and Thread mode.
+ * To avoid having to do multiple comparisons in inline generated code,
+ * we make the check we do here loose, so it will match for EXC_RETURN
+ * in Thread mode. For system emulation do_v7m_exception_exit() checks
+ * for these spurious cases and returns without doing anything (giving
+ * the same behaviour as for a branch to a non-magic address).
+ *
+ * In linux-user mode it is unclear what the right behaviour for an
+ * attempted FNC_RETURN should be, because in real hardware this will go
+ * directly to Secure code (ie not the Linux kernel) which will then treat
+ * the error in any way it chooses. For QEMU we opt to make the FNC_RETURN
+ * attempt behave the way it would on a CPU without the security extension,
+ * which is to say "like a normal branch". That means we can simply treat
+ * all branches as normal with no magic address behaviour.
  */
 static inline void gen_bx_excret(DisasContext *s, TCGv_i32 var)
 {
@@ -963,10 +980,12 @@ static inline void gen_bx_excret(DisasContext *s, TCGv_i32 var)
      * s->base.is_jmp that we need to do the rest of the work later.
      */
     gen_bx(s, var);
+#ifndef CONFIG_USER_ONLY
     if (arm_dc_feature(s, ARM_FEATURE_M_SECURITY) ||
         (s->v7m_handler_mode && arm_dc_feature(s, ARM_FEATURE_M))) {
         s->base.is_jmp = DISAS_BX_EXCRET;
     }
+#endif
 }
 
 static inline void gen_bx_excret_final_code(DisasContext *s)
Signature · 1 line
2.17.1
#20 lore raw

PATCH 20/55 libvhost-user: fix SLAVE_SEND_FD handling

In reply to Michael Roth

From: Johannes Berg <[email protected]>

It doesn't look like this could possibly work properly since VHOST_USER_PROTOCOL_F_SLAVE_SEND_FD is defined to 10, but the dev->protocol_features has a bitmap. I suppose the peer this was tested with also supported VHOST_USER_PROTOCOL_F_LOG_SHMFD, in which case the test would always be false, but nevertheless the code seems wrong.

Use has_feature() to fix this.

Fixes d84599f56c82 ("libvhost-user: support host notifier")

Message-Id: <[email protected]>

(cherry picked from commit 8726b70b449896f1211f869ec4f608904f027207)
---
 contrib/libvhost-user/libvhost-user.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
contrib/libvhost-user/libvhost-user.c +2 -1
index 4b36e35a82..cb5f5770e4 100644
@@ -1097,7 +1097,8 @@ bool vu_set_queue_host_notifier(VuDev *dev, VuVirtq *vq, int fd,
 
     vmsg.fd_num = fd_num;
 
-    if ((dev->protocol_features & VHOST_USER_PROTOCOL_F_SLAVE_SEND_FD) == 0) {
+    if (!has_feature(dev->protocol_features,
+                     VHOST_USER_PROTOCOL_F_SLAVE_SEND_FD)) {
         return false;
     }
 
Signature · 1 line
2.17.1
#21 lore raw

PATCH 21/55 qcow2: Fix the calculation of the maximum L2 cache size

In reply to Michael Roth

From: Alberto Garcia <[email protected]>

The size of the qcow2 L2 cache defaults to 32 MB, which can be easily larger than the maximum amount of L2 metadata that the image can have. For example: with 64 KB clusters the user would need a qcow2 image with a virtual size of 256 GB in order to have 32 MB of L2 metadata.

Because of that, since commit b749562d9822d14ef69c9eaa5f85903010b86c30 we forbid the L2 cache to become larger than the maximum amount of L2 metadata for the image, calculated using this formula:

    uint64_t max_l2_cache = virtual_disk_size / (s->cluster_size / 8);

The problem with this formula is that the result should be rounded up to the cluster size because an L2 table on disk always takes one full cluster.

For example, a 1280 MB qcow2 image with 64 KB clusters needs exactly 160 KB of L2 metadata, but we need 192 KB on disk (3 clusters) even if the last 32 KB of those are not going to be used.

However QEMU rounds the numbers down and only creates 2 cache tables (128 KB), which is not enough for the image.

A quick test doing 4KB random writes on a 1280 MB image gives me around 500 IOPS, while with the correct cache size I get 16K IOPS.

(cherry picked from commit b70d08205b2e4044c529eefc21df2c8ab61b473b)
---
 block/qcow2.c | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)
block/qcow2.c +5 -1
index 039bdc2f7e..865839682c 100644
@@ -826,7 +826,11 @@ static void read_cache_sizes(BlockDriverState *bs, QemuOpts *opts,
     bool l2_cache_entry_size_set;
     int min_refcount_cache = MIN_REFCOUNT_CACHE_SIZE * s->cluster_size;
     uint64_t virtual_disk_size = bs->total_sectors * BDRV_SECTOR_SIZE;
-    uint64_t max_l2_cache = virtual_disk_size / (s->cluster_size / 8);
+    uint64_t max_l2_entries = DIV_ROUND_UP(virtual_disk_size, s->cluster_size);
+    /* An L2 table is always one cluster in size so the max cache size
+     * should be a multiple of the cluster size. */
+    uint64_t max_l2_cache = ROUND_UP(max_l2_entries * sizeof(uint64_t),
+                                     s->cluster_size);
 
     combined_cache_size_set = qemu_opt_get(opts, QCOW2_OPT_CACHE_SIZE);
     l2_cache_size_set = qemu_opt_get(opts, QCOW2_OPT_L2_CACHE_SIZE);
Signature · 1 line
2.17.1
#22 lore raw

PATCH 22/55 block/nfs: tear down aio before nfs_close

In reply to Michael Roth

From: Peter Lieven <[email protected]>

nfs_close is a sync call from libnfs and has its own event handler polling on the nfs FD. Avoid that both QEMU and libnfs are intefering here.

(cherry picked from commit 601dc6559725f7a614b6f893611e17ff0908e914)
---
 block/nfs.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)
block/nfs.c +4 -2
index d93241b3bb..2b7a078241 100644
@@ -390,12 +390,14 @@ static void nfs_attach_aio_context(BlockDriverState *bs,
 static void nfs_client_close(NFSClient *client)
 {
     if (client->context) {
+        qemu_mutex_lock(&client->mutex);
+        aio_set_fd_handler(client->aio_context, nfs_get_fd(client->context),
+                           false, NULL, NULL, NULL, NULL);
+        qemu_mutex_unlock(&client->mutex);
         if (client->fh) {
             nfs_close(client->context, client->fh);
             client->fh = NULL;
         }
-        aio_set_fd_handler(client->aio_context, nfs_get_fd(client->context),
-                           false, NULL, NULL, NULL, NULL);
         nfs_destroy_context(client->context);
         client->context = NULL;
     }
Signature · 1 line
2.17.1
#23 lore raw

PATCH 23/55 curl: Keep pointer to the CURLState in CURLSocket

In reply to Michael Roth

From: Max Reitz <[email protected]>

A follow-up patch will make curl_multi_do() and curl_multi_read() take a CURLSocket instead of the CURLState. They still need the latter, though, so add a pointer to it to the former.

Message-id: [email protected]

(cherry picked from commit 0487861685294660b23bc146e1ebd5304aa8bbe0)
---
 block/curl.c | 3 +++
 1 file changed, 3 insertions(+)
block/curl.c +3 -0
index d4c8e94f3e..92dc2f630e 100644
@@ -80,6 +80,7 @@ static CURLMcode __curl_multi_socket_action(CURLM *multi_handle,
 #define CURL_BLOCK_OPT_TIMEOUT_DEFAULT 5
 
 struct BDRVCURLState;
+struct CURLState;
 
 static bool libcurl_initialized;
 
@@ -97,6 +98,7 @@ typedef struct CURLAIOCB {
 
 typedef struct CURLSocket {
     int fd;
+    struct CURLState *state;
     QLIST_ENTRY(CURLSocket) next;
 } CURLSocket;
 
@@ -180,6 +182,7 @@ static int curl_sock_cb(CURL *curl, curl_socket_t fd, int action,
     if (!socket) {
         socket = g_new0(CURLSocket, 1);
         socket->fd = fd;
+        socket->state = state;
         QLIST_INSERT_HEAD(&state->sockets, socket, next);
     }
     socket = NULL;
Signature · 1 line
2.17.1
#24 lore raw

PATCH 24/55 curl: Keep *socket until the end of curl_sock_cb()

In reply to Michael Roth

From: Max Reitz <[email protected]>

This does not really change anything, but it makes the code a bit easier to follow once we use @socket as the opaque pointer for aio_set_fd_handler().

Message-id: [email protected]

(cherry picked from commit 007f339b1099af46a008dac438ca0943e31dba72)
---
 block/curl.c | 10 +++++-----
 1 file changed, 5 insertions(+), 5 deletions(-)
block/curl.c +5 -5
index 92dc2f630e..95d7b77dc0 100644
@@ -172,10 +172,6 @@ static int curl_sock_cb(CURL *curl, curl_socket_t fd, int action,
 
     QLIST_FOREACH(socket, &state->sockets, next) {
         if (socket->fd == fd) {
-            if (action == CURL_POLL_REMOVE) {
-                QLIST_REMOVE(socket, next);
-                g_free(socket);
-            }
             break;
         }
     }
@@ -185,7 +181,6 @@ static int curl_sock_cb(CURL *curl, curl_socket_t fd, int action,
         socket->state = state;
         QLIST_INSERT_HEAD(&state->sockets, socket, next);
     }
-    socket = NULL;
 
     trace_curl_sock_cb(action, (int)fd);
     switch (action) {
@@ -207,6 +202,11 @@ static int curl_sock_cb(CURL *curl, curl_socket_t fd, int action,
             break;
     }
 
+    if (action == CURL_POLL_REMOVE) {
+        QLIST_REMOVE(socket, next);
+        g_free(socket);
+    }
+
     return 0;
 }
 
Signature · 1 line
2.17.1
#25 lore raw

PATCH 25/55 curl: Check completion in curl_multi_do()

In reply to Michael Roth

From: Max Reitz <[email protected]>

While it is more likely that transfers complete after some file descriptor has data ready to read, we probably should not rely on it. Better be safe than sorry and call curl_multi_check_completion() in curl_multi_do(), too, just like it is done in curl_multi_read().

With this change, curl_multi_do() and curl_multi_read() are actually the same, so drop curl_multi_read() and use curl_multi_do() as the sole FD handler.

Message-id: [email protected]

(cherry picked from commit 948403bcb1c7e71dcbe8ab8479cf3934a0efcbb5)
---
 block/curl.c | 14 ++------------
 1 file changed, 2 insertions(+), 12 deletions(-)
block/curl.c +2 -12
index 95d7b77dc0..5838afef99 100644
@@ -139,7 +139,6 @@ typedef struct BDRVCURLState {
 
 static void curl_clean_state(CURLState *s);
 static void curl_multi_do(void *arg);
-static void curl_multi_read(void *arg);
 
 #ifdef NEED_CURL_TIMER_CALLBACK
 /* Called from curl_multi_do_locked, with s->mutex held.  */
@@ -186,7 +185,7 @@ static int curl_sock_cb(CURL *curl, curl_socket_t fd, int action,
     switch (action) {
         case CURL_POLL_IN:
             aio_set_fd_handler(s->aio_context, fd, false,
-                               curl_multi_read, NULL, NULL, state);
+                               curl_multi_do, NULL, NULL, state);
             break;
         case CURL_POLL_OUT:
             aio_set_fd_handler(s->aio_context, fd, false,
@@ -194,7 +193,7 @@ static int curl_sock_cb(CURL *curl, curl_socket_t fd, int action,
             break;
         case CURL_POLL_INOUT:
             aio_set_fd_handler(s->aio_context, fd, false,
-                               curl_multi_read, curl_multi_do, NULL, state);
+                               curl_multi_do, curl_multi_do, NULL, state);
             break;
         case CURL_POLL_REMOVE:
             aio_set_fd_handler(s->aio_context, fd, false,
@@ -416,15 +415,6 @@ static void curl_multi_do(void *arg)
 {
     CURLState *s = (CURLState *)arg;
 
-    qemu_mutex_lock(&s->s->mutex);
-    curl_multi_do_locked(s);
-    qemu_mutex_unlock(&s->s->mutex);
-}
-
-static void curl_multi_read(void *arg)
-{
-    CURLState *s = (CURLState *)arg;
-
     qemu_mutex_lock(&s->s->mutex);
     curl_multi_do_locked(s);
     curl_multi_check_completion(s->s);
Signature · 1 line
2.17.1
#26 lore raw

PATCH 26/55 curl: Pass CURLSocket to curl_multi_do()

In reply to Michael Roth

From: Max Reitz <[email protected]>

curl_multi_do_locked() currently marks all sockets as ready. That is not only inefficient, but in fact unsafe (the loop is). A follow-up patch will change that, but to do so, curl_multi_do_locked() needs to know exactly which socket is ready; and that is accomplished by this patch here.

Message-id: [email protected]

(cherry picked from commit 9dbad87d25587ff640ef878f7b6159fc368ff541)
---
 block/curl.c | 20 +++++++++++---------
 1 file changed, 11 insertions(+), 9 deletions(-)
block/curl.c +11 -9
index 5838afef99..cf2686218d 100644
@@ -185,15 +185,15 @@ static int curl_sock_cb(CURL *curl, curl_socket_t fd, int action,
     switch (action) {
         case CURL_POLL_IN:
             aio_set_fd_handler(s->aio_context, fd, false,
-                               curl_multi_do, NULL, NULL, state);
+                               curl_multi_do, NULL, NULL, socket);
             break;
         case CURL_POLL_OUT:
             aio_set_fd_handler(s->aio_context, fd, false,
-                               NULL, curl_multi_do, NULL, state);
+                               NULL, curl_multi_do, NULL, socket);
             break;
         case CURL_POLL_INOUT:
             aio_set_fd_handler(s->aio_context, fd, false,
-                               curl_multi_do, curl_multi_do, NULL, state);
+                               curl_multi_do, curl_multi_do, NULL, socket);
             break;
         case CURL_POLL_REMOVE:
             aio_set_fd_handler(s->aio_context, fd, false,
@@ -392,9 +392,10 @@ static void curl_multi_check_completion(BDRVCURLState *s)
 }
 
 /* Called with s->mutex held.  */
-static void curl_multi_do_locked(CURLState *s)
+static void curl_multi_do_locked(CURLSocket *ready_socket)
 {
     CURLSocket *socket, *next_socket;
+    CURLState *s = ready_socket->state;
     int running;
     int r;
 
@@ -413,12 +414,13 @@ static void curl_multi_do_locked(CURLState *s)
 
 static void curl_multi_do(void *arg)
 {
-    CURLState *s = (CURLState *)arg;
+    CURLSocket *socket = arg;
+    BDRVCURLState *s = socket->state->s;
 
-    qemu_mutex_lock(&s->s->mutex);
-    curl_multi_do_locked(s);
-    curl_multi_check_completion(s->s);
-    qemu_mutex_unlock(&s->s->mutex);
+    qemu_mutex_lock(&s->mutex);
+    curl_multi_do_locked(socket);
+    curl_multi_check_completion(s);
+    qemu_mutex_unlock(&s->mutex);
 }
 
 static void curl_multi_timeout_do(void *arg)
Signature · 1 line
2.17.1
#27 lore raw

PATCH 27/55 curl: Report only ready sockets

In reply to Michael Roth

From: Max Reitz <[email protected]>

Instead of reporting all sockets to cURL, only report the one that has caused curl_multi_do_locked() to be called. This lets us get rid of the QLIST_FOREACH_SAFE() list, which was actually wrong: SAFE foreaches are only safe when the current element is removed in each iteration. If it possible for the list to be concurrently modified, we cannot guarantee that only the current element will be removed. Therefore, we must not use QLIST_FOREACH_SAFE() here.

Fixes ff5ca1664af85b24a4180d595ea6873fd3deac57

Message-id: [email protected]

(cherry picked from commit 9abaf9fc474c3dd53e8e119326abc774c977c331)
---
 block/curl.c | 17 ++++++-----------
 1 file changed, 6 insertions(+), 11 deletions(-)
block/curl.c +6 -11
index cf2686218d..fd70f1ebc4 100644
@@ -392,24 +392,19 @@ static void curl_multi_check_completion(BDRVCURLState *s)
 }
 
 /* Called with s->mutex held.  */
-static void curl_multi_do_locked(CURLSocket *ready_socket)
+static void curl_multi_do_locked(CURLSocket *socket)
 {
-    CURLSocket *socket, *next_socket;
-    CURLState *s = ready_socket->state;
+    BDRVCURLState *s = socket->state->s;
     int running;
     int r;
 
-    if (!s->s->multi) {
+    if (!s->multi) {
         return;
     }
 
-    /* Need to use _SAFE because curl_multi_socket_action() may trigger
-     * curl_sock_cb() which might modify this list */
-    QLIST_FOREACH_SAFE(socket, &s->sockets, next, next_socket) {
-        do {
-            r = curl_multi_socket_action(s->s->multi, socket->fd, 0, &running);
-        } while (r == CURLM_CALL_MULTI_PERFORM);
-    }
+    do {
+        r = curl_multi_socket_action(s->multi, socket->fd, 0, &running);
+    } while (r == CURLM_CALL_MULTI_PERFORM);
 }
 
 static void curl_multi_do(void *arg)
Signature · 1 line
2.17.1
#28 lore raw

PATCH 28/55 curl: Handle success in multi_check_completion

In reply to Michael Roth

From: Max Reitz <[email protected]>

Background: As of cURL 7.59.0, it verifies that several functions are not called from within a callback. Among these functions is curl_multi_add_handle().

curl_read_cb() is a callback from cURL and not a coroutine. Waking up acb->co will lead to entering it then and there, which means the current request will settle and the caller (if it runs in the same coroutine) may then issue the next request. In such a case, we will enter curl_setup_preadv() effectively from within curl_read_cb().

Calling curl_multi_add_handle() will then fail and the new request will not be processed.

Fix this by not letting curl_read_cb() wake up acb->co. Instead, leave the whole business of settling the AIOCB objects to curl_multi_check_completion() (which is called from our timer callback and our FD handler, so not from any cURL callbacks).

Message-id: [email protected]

(cherry picked from commit bfb23b480a49114315877aacf700b49453e0f9d9)
---
 block/curl.c | 69 ++++++++++++++++++++++------------------------------
 1 file changed, 29 insertions(+), 40 deletions(-)
block/curl.c +29 -40
index fd70f1ebc4..c343c7ed3d 100644
@@ -229,7 +229,6 @@ static size_t curl_read_cb(void *ptr, size_t size, size_t nmemb, void *opaque)
 {
     CURLState *s = ((CURLState*)opaque);
     size_t realsize = size * nmemb;
-    int i;
 
     trace_curl_read_cb(realsize);
 
@@ -245,32 +244,6 @@ static size_t curl_read_cb(void *ptr, size_t size, size_t nmemb, void *opaque)
     memcpy(s->orig_buf + s->buf_off, ptr, realsize);
     s->buf_off += realsize;
 
-    for(i=0; i<CURL_NUM_ACB; i++) {
-        CURLAIOCB *acb = s->acb[i];
-
-        if (!acb)
-            continue;
-
-        if ((s->buf_off >= acb->end)) {
-            size_t request_length = acb->bytes;
-
-            qemu_iovec_from_buf(acb->qiov, 0, s->orig_buf + acb->start,
-                                acb->end - acb->start);
-
-            if (acb->end - acb->start < request_length) {
-                size_t offset = acb->end - acb->start;
-                qemu_iovec_memset(acb->qiov, offset, 0,
-                                  request_length - offset);
-            }
-
-            acb->ret = 0;
-            s->acb[i] = NULL;
-            qemu_mutex_unlock(&s->s->mutex);
-            aio_co_wake(acb->co);
-            qemu_mutex_lock(&s->s->mutex);
-        }
-    }
-
 read_end:
     /* curl will error out if we do not return this value */
     return size * nmemb;
@@ -351,13 +324,14 @@ static void curl_multi_check_completion(BDRVCURLState *s)
             break;
 
         if (msg->msg == CURLMSG_DONE) {
+            int i;
             CURLState *state = NULL;
+            bool error = msg->data.result != CURLE_OK;
+
             curl_easy_getinfo(msg->easy_handle, CURLINFO_PRIVATE,
                               (char **)&state);
 
-            /* ACBs for successful messages get completed in curl_read_cb */
-            if (msg->data.result != CURLE_OK) {
-                int i;
+            if (error) {
                 static int errcount = 100;
 
                 /* Don't lose the original error message from curl, since
@@ -369,20 +343,35 @@ static void curl_multi_check_completion(BDRVCURLState *s)
                         error_report("curl: further errors suppressed");
                     }
                 }
+            }
 
-                for (i = 0; i < CURL_NUM_ACB; i++) {
-                    CURLAIOCB *acb = state->acb[i];
+            for (i = 0; i < CURL_NUM_ACB; i++) {
+                CURLAIOCB *acb = state->acb[i];
 
-                    if (acb == NULL) {
-                        continue;
-                    }
+                if (acb == NULL) {
+                    continue;
+                }
+
+                if (!error) {
+                    /* Assert that we have read all data */
+                    assert(state->buf_off >= acb->end);
+
+                    qemu_iovec_from_buf(acb->qiov, 0,
+                                        state->orig_buf + acb->start,
+                                        acb->end - acb->start);
 
-                    acb->ret = -EIO;
-                    state->acb[i] = NULL;
-                    qemu_mutex_unlock(&s->mutex);
-                    aio_co_wake(acb->co);
-                    qemu_mutex_lock(&s->mutex);
+                    if (acb->end - acb->start < acb->bytes) {
+                        size_t offset = acb->end - acb->start;
+                        qemu_iovec_memset(acb->qiov, offset, 0,
+                                          acb->bytes - offset);
+                    }
                 }
+
+                acb->ret = error ? -EIO : 0;
+                state->acb[i] = NULL;
+                qemu_mutex_unlock(&s->mutex);
+                aio_co_wake(acb->co);
+                qemu_mutex_lock(&s->mutex);
             }
 
             curl_clean_state(state);
Signature · 1 line
2.17.1
#29 lore raw

PATCH 29/55 blockjob: update nodes head while removing all bdrv

In reply to Michael Roth

From: Sergio Lopez <[email protected]>

block_job_remove_all_bdrv() iterates through job->nodes, calling bdrv_root_unref_child() for each entry. The call to the latter may reach child_job_[can_]set_aio_ctx(), which will also attempt to traverse job->nodes, potentially finding entries that where freed on previous iterations.

To avoid this situation, update job->nodes head on each iteration to ensure that already freed entries are no longer linked to the list.

RHBZ: https://bugzilla.redhat.com/show_bug.cgi?id=1746631

Message-id: [email protected]

(cherry picked from commit d876bf676f5e7c6aa9ac64555e48cba8734ecb2f)
---
 blockjob.c | 17 +++++++++++++----
 1 file changed, 13 insertions(+), 4 deletions(-)
blockjob.c +13 -4
index 20b7f557da..74abb97bfd 100644
@@ -186,14 +186,23 @@ static const BdrvChildRole child_job = {
 
 void block_job_remove_all_bdrv(BlockJob *job)
 {
-    GSList *l;
-    for (l = job->nodes; l; l = l->next) {
+    /*
+     * bdrv_root_unref_child() may reach child_job_[can_]set_aio_ctx(),
+     * which will also traverse job->nodes, so consume the list one by
+     * one to make sure that such a concurrent access does not attempt
+     * to process an already freed BdrvChild.
+     */
+    while (job->nodes) {
+        GSList *l = job->nodes;
         BdrvChild *c = l->data;
+
+        job->nodes = l->next;
+
         bdrv_op_unblock_all(c->bs, job->blocker);
         bdrv_root_unref_child(c);
+
+        g_slist_free_1(l);
     }
-    g_slist_free(job->nodes);
-    job->nodes = NULL;
 }
 
 bool block_job_has_bdrv(BlockJob *job, BlockDriverState *bs)
Signature · 1 line
2.17.1
#30 lore raw

PATCH 30/55 block/qcow2: Fix corruption introduced by commit 8ac0f15f335

In reply to Michael Roth

From: Maxim Levitsky <[email protected]>

This fixes subtle corruption introduced by luks threaded encryption in commit 8ac0f15f335

Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=1745922

The corruption happens when we do a write that
* writes to two or more unallocated clusters at once
* doesn't fully cover the first sector
* doesn't fully cover the last sector
* uses luks encryption

In this case, when allocating the new clusters we COW both areas prior to the write and after the write, and we encrypt them.

The above mentioned commit accidentally made it so we encrypt the second COW area using the physical cluster offset of the first area.

The problem is that offset_in_cluster in do_perform_cow_encrypt can be larger that the cluster size, thus cluster_offset will no longer point to the start of the cluster at which encrypted area starts.

Next patch in this series will refactor the code to avoid all these assumptions.

In the bugreport that was triggered by rebasing a luks image to new, zero filled base, which lot of such writes, and causes some files with zero areas to contain garbage there instead.
But as described above it can happen elsewhere as well

Message-id: [email protected]

(cherry picked from commit 38e7d54bdc518b5a05a922467304bcace2396945)
---
 block/qcow2-cluster.c | 7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)
block/qcow2-cluster.c +4 -3
index cc5609e27a..760564c8fb 100644
@@ -473,9 +473,10 @@ static bool coroutine_fn do_perform_cow_encrypt(BlockDriverState *bs,
         assert((offset_in_cluster & ~BDRV_SECTOR_MASK) == 0);
         assert((bytes & ~BDRV_SECTOR_MASK) == 0);
         assert(s->crypto);
-        if (qcow2_co_encrypt(bs, cluster_offset,
-                             src_cluster_offset + offset_in_cluster,
-                             buffer, bytes) < 0) {
+        if (qcow2_co_encrypt(bs,
+                start_of_cluster(s, cluster_offset + offset_in_cluster),
+                src_cluster_offset + offset_in_cluster,
+                buffer, bytes) < 0) {
             return false;
         }
     }
Signature · 1 line
2.17.1
#31 lore raw

PATCH 31/55 coroutine: Add qemu_co_mutex_assert_locked()

In reply to Michael Roth

From: Kevin Wolf <[email protected]>

Some functions require that the caller holds a certain CoMutex for them to operate correctly. Add a function so that they can assert the lock is really held.

---
 include/qemu/coroutine.h | 15 +++++++++++++++
 1 file changed, 15 insertions(+)
include/qemu/coroutine.h +15 -0
index 9801e7f5a4..f4843b5f59 100644
@@ -167,6 +167,21 @@ void coroutine_fn qemu_co_mutex_lock(CoMutex *mutex);
  */
 void coroutine_fn qemu_co_mutex_unlock(CoMutex *mutex);
 
+/**
+ * Assert that the current coroutine holds @mutex.
+ */
+static inline coroutine_fn void qemu_co_mutex_assert_locked(CoMutex *mutex)
+{
+    /*
+     * mutex->holder doesn't need any synchronisation if the assertion holds
+     * true because the mutex protects it. If it doesn't hold true, we still
+     * don't mind if another thread takes or releases mutex behind our back,
+     * because the condition will be false no matter whether we read NULL or
+     * the pointer for any other coroutine.
+     */
+    assert(atomic_read(&mutex->locked) &&
+           mutex->holder == qemu_coroutine_self());
+}
 
 /**
  * CoQueues are a mechanism to queue coroutines in order to continue executing
Signature · 1 line
2.17.1
#32 lore raw

PATCH 32/55 qcow2: Fix corruption bug in qcow2_detect_metadata_preallocation()

In reply to Michael Roth

From: Kevin Wolf <[email protected]>

qcow2_detect_metadata_preallocation() calls qcow2_get_refcount() which requires s->lock to be taken to protect its accesses to the refcount table and refcount blocks. However, nothing in this code path actually took the lock. This could cause the same cache entry to be used by two requests at the same time, for different tables at different offsets, resulting in image corruption.

As it would be preferable to base the detection on consistent data (even though it's just heuristics), let's take the lock not only around the qcow2_get_refcount() calls, but around the whole function.

This patch takes the lock in qcow2_co_block_status() earlier and asserts in qcow2_detect_metadata_preallocation() that we hold the lock.

Fixes 69f47505ee66afaa513305de0c1895a224e52c45
(cherry picked from commit 5e9785505210e2477e590e61b1ab100d0ec22b01)
---
 block/qcow2-refcount.c | 2 ++
 block/qcow2.c          | 3 ++-
 2 files changed, 4 insertions(+), 1 deletion(-)
block/qcow2-refcount.c +2 -0
index ef965d7895..0d64bf5a5e 100644
@@ -3455,6 +3455,8 @@ int qcow2_detect_metadata_preallocation(BlockDriverState *bs)
     int64_t i, end_cluster, cluster_count = 0, threshold;
     int64_t file_length, real_allocation, real_clusters;
 
+    qemu_co_mutex_assert_locked(&s->lock);
+
     file_length = bdrv_getlength(bs->file->bs);
     if (file_length < 0) {
         return file_length;
block/qcow2.c +2 -1
index 865839682c..c0f5439dc8 100644
@@ -1899,6 +1899,8 @@ static int coroutine_fn qcow2_co_block_status(BlockDriverState *bs,
     unsigned int bytes;
     int status = 0;
 
+    qemu_co_mutex_lock(&s->lock);
+
     if (!s->metadata_preallocation_checked) {
         ret = qcow2_detect_metadata_preallocation(bs);
         s->metadata_preallocation = (ret == 1);
@@ -1906,7 +1908,6 @@ static int coroutine_fn qcow2_co_block_status(BlockDriverState *bs,
     }
 
     bytes = MIN(INT_MAX, count);
-    qemu_co_mutex_lock(&s->lock);
     ret = qcow2_get_cluster_offset(bs, offset, &bytes, &cluster_offset);
     qemu_co_mutex_unlock(&s->lock);
     if (ret < 0) {
Signature · 1 line
2.17.1
#33 lore raw

PATCH 33/55 block/backup: fix max_transfer handling for copy_range

In reply to Michael Roth

From: Vladimir Sementsov-Ogievskiy <[email protected]>

Of course, QEMU_ALIGN_UP is a typo, it should be QEMU_ALIGN_DOWN, as we are trying to find aligned size which satisfy both source and target. Also, don't ignore too small max_transfer. In this case seems safer to disable copy_range.

Fixes 9ded4a0114968e

Message-id: [email protected]

(cherry picked from commit 981fb5810aa3f68797ee6e261db338bd78857614)
---
 block/backup.c | 15 +++++++++++----
 1 file changed, 11 insertions(+), 4 deletions(-)
block/backup.c +11 -4
index b26c22c4b8..7067d1d1ad 100644
@@ -657,12 +657,19 @@ BlockJob *backup_job_create(const char *job_id, BlockDriverState *bs,
     job->cluster_size = cluster_size;
     job->copy_bitmap = copy_bitmap;
     copy_bitmap = NULL;
-    job->use_copy_range = !compress; /* compression isn't supported for it */
     job->copy_range_size = MIN_NON_ZERO(blk_get_max_transfer(job->common.blk),
                                         blk_get_max_transfer(job->target));
-    job->copy_range_size = MAX(job->cluster_size,
-                               QEMU_ALIGN_UP(job->copy_range_size,
-                                             job->cluster_size));
+    job->copy_range_size = QEMU_ALIGN_DOWN(job->copy_range_size,
+                                           job->cluster_size);
+    /*
+     * Set use_copy_range, consider the following:
+     * 1. Compression is not supported for copy_range.
+     * 2. copy_range does not respect max_transfer (it's a TODO), so we factor
+     *    that in here. If max_transfer is smaller than the job->cluster_size,
+     *    we do not use copy_range (in that case it's zero after aligning down
+     *    above).
+     */
+    job->use_copy_range = !compress && job->copy_range_size > 0;
 
     /* Required permissions are already taken with target's blk_new() */
     block_job_add_bdrv(&job->common, "target", target, 0, BLK_PERM_ALL,
Signature · 1 line
2.17.1
#34 lore raw

PATCH 34/55 block/backup: fix backup_cow_with_offload for last cluster

In reply to Michael Roth

From: Vladimir Sementsov-Ogievskiy <[email protected]>

We shouldn't try to copy bytes beyond EOF. Fix it.

Message-id: [email protected]

(cherry picked from commit 1048ddf0a32dcdaa952e581bd503d49adad527cc)
---
 block/backup.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
block/backup.c +1 -1
index 7067d1d1ad..8761f1f9a7 100644
@@ -167,7 +167,7 @@ static int coroutine_fn backup_cow_with_offload(BackupBlockJob *job,
 
     assert(QEMU_IS_ALIGNED(job->copy_range_size, job->cluster_size));
     assert(QEMU_IS_ALIGNED(start, job->cluster_size));
-    nbytes = MIN(job->copy_range_size, end - start);
+    nbytes = MIN(job->copy_range_size, MIN(end, job->len) - start);
     nr_clusters = DIV_ROUND_UP(nbytes, job->cluster_size);
     hbitmap_reset(job->copy_bitmap, start, job->cluster_size * nr_clusters);
     ret = blk_co_copy_range(blk, start, job->target, start, nbytes,
Signature · 1 line
2.17.1
#35 lore raw

PATCH 35/55 hw/arm/boot.c: Set NSACR.{CP11, CP10} for NS kernel boots

In reply to Michael Roth

From: Peter Maydell <[email protected]>

If we're booting a Linux kernel directly into Non-Secure state on a CPU which has Secure state, then make sure we set the NSACR CP11 and CP10 bits, so that Non-Secure is allowed to access the FPU. Otherwise an AArch32 kernel will UNDEF as soon as it tries to use the FPU.

It used to not matter that we didn't do this until commit fc1120a7f5f2d4b6, where we implemented actually honouring these NSACR bits.

The problem only exists for CPUs where EL3 is AArch32; the equivalent AArch64 trap bits are in CPTR_EL3 and are "0 to not trap, 1 to trap", so the reset value of the register permits NS access, unlike NSACR.

Message-id: [email protected]

(cherry picked from commit ece628fcf69cbbd4b3efb6fbd203af07609467a2)
---
 hw/arm/boot.c | 2 ++
 1 file changed, 2 insertions(+)
hw/arm/boot.c +2 -0
index c2b89b3bb9..fc4e021a38 100644
@@ -754,6 +754,8 @@ static void do_cpu_reset(void *opaque)
                     (cs != first_cpu || !info->secure_board_setup)) {
                     /* Linux expects non-secure state */
                     env->cp15.scr_el3 |= SCR_NS;
+                    /* Set NSACR.{CP11,CP10} so NS can access the FPU */
+                    env->cp15.nsacr |= 3 << 10;
                 }
             }
 
Signature · 1 line
2.17.1
#36 lore raw

PATCH 36/55 make-release: pull in edk2 submodules so we can build it from tarballs

In reply to Michael Roth

The make efi target added by 536d2173 is built from the roms/edk2 submodule, which in turn relies on additional submodules nested under roms/edk2.

The make-release script currently only pulls in top-level submodules, so these nested submodules are missing in the resulting tarball.

We could try to address this situation more generally by recursively pulling in all submodules, but this doesn't necessarily ensure the end-result will build properly (this case also required other changes).

Additionally, due to the nature of submodules, we may not always have control over how these sorts of things are dealt with, so for now we continue to handle it on a case-by-case in the make-release script.

Message-Id: <[email protected]>

(cherry picked from commit 45c61c6c23918e3b05ed9ecac5b2328ebae5f774)
---
 scripts/make-release | 8 ++++++++
 1 file changed, 8 insertions(+)
scripts/make-release +8 -0
index b4af9c9e52..a2a8cda33c 100755
@@ -20,6 +20,14 @@ git checkout "v${version}"
 git submodule update --init
 (cd roms/seabios && git describe --tags --long --dirty > .version)
 (cd roms/skiboot && ./make_version.sh > .version)
+# Fetch edk2 submodule's submodules, since it won't have access to them via
+# the tarball later.
+#
+# A more uniform way to handle this sort of situation would be nice, but we
+# don't necessarily have much control over how a submodule handles its
+# submodule dependencies, so we continue to handle these on a case-by-case
+# basis for now.
+(cd roms/edk2 && git submodule update --init)
 popd
 tar --exclude=.git -cjf ${destination}.tar.bz2 ${destination}
 rm -rf ${destination}
Signature · 1 line
2.17.1
#37 lore raw

PATCH 37/55 roms/Makefile.edk2: don't pull in submodules when building from tarball

In reply to Michael Roth

Currently the make efi target pulls submodules nested under the roms/edk2 submodule as dependencies. However, when we attempt to build from a tarball this fails since we are no longer in a git tree.

A preceding patch will pre-populate these submodules in the tarball, so assume this build dependency is only needed when building from a git tree.

Message-Id: <[email protected]>

(cherry picked from commit f3e330e3c319160ac04954399b5a10afc965098c)
---
 roms/Makefile.edk2 | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)
roms/Makefile.edk2 +6 -1
index c2f2ff59d5..33a074d3a4 100644
@@ -46,8 +46,13 @@ all: $(foreach flashdev,$(flashdevs),../pc-bios/edk2-$(flashdev).fd.bz2) \
 # files.
 .INTERMEDIATE: $(foreach flashdev,$(flashdevs),../pc-bios/edk2-$(flashdev).fd)
 
+# Fetch edk2 submodule's submodules. If it is not in a git tree, assume
+# we're building from a tarball and that they've already been fetched by
+# make-release/tarball scripts.
 submodules:
-	cd edk2 && git submodule update --init --force
+	if test -d edk2/.git; then \
+		cd edk2 && git submodule update --init --force; \
+	fi
 
 # See notes on the ".NOTPARALLEL" target and the "+" indicator in
 # "tests/uefi-test-tools/Makefile".
Signature · 1 line
2.17.1
#38 lore raw

PATCH 38/55 s390: PCI: fix IOMMU region init

In reply to Michael Roth

From: Matthew Rosato <[email protected]>

The fix in dbe9cf606c shrinks the IOMMU memory region to a size that seems reasonable on the surface, however is actually too small as it is based against a 0-mapped address space. This causes breakage with small guests as they can overrun the IOMMU window.

Let's go back to the prior method of initializing iommu for now.

Fixes dbe9cf606c ("s390x/pci: Set the iommu region size mpcifc request")

Message-Id: <[email protected]>

(cherry picked from commit 7df1dac5f1c85312474df9cb3a8fcae72303da62)
---
 hw/s390x/s390-pci-bus.c | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)
hw/s390x/s390-pci-bus.c +6 -1
index 2c6e084e2c..9a935f22b5 100644
@@ -694,10 +694,15 @@ static const MemoryRegionOps s390_msi_ctrl_ops = {
 
 void s390_pci_iommu_enable(S390PCIIOMMU *iommu)
 {
+    /*
+     * The iommu region is initialized against a 0-mapped address space,
+     * so the smallest IOMMU region we can define runs from 0 to the end
+     * of the PCI address space.
+     */
     char *name = g_strdup_printf("iommu-s390-%04x", iommu->pbdev->uid);
     memory_region_init_iommu(&iommu->iommu_mr, sizeof(iommu->iommu_mr),
                              TYPE_S390_IOMMU_MEMORY_REGION, OBJECT(&iommu->mr),
-                             name, iommu->pal - iommu->pba + 1);
+                             name, iommu->pal + 1);
     iommu->enabled = true;
     memory_region_add_subregion(&iommu->mr, 0, MEMORY_REGION(&iommu->iommu_mr));
     g_free(name);
Signature · 1 line
2.17.1
#39 lore raw

PATCH 39/55 block/snapshot: Restrict set of snapshot nodes

In reply to Michael Roth

From: Kevin Wolf <[email protected]>

Nodes involved in internal snapshots were those that were returned by bdrv_next(), inserted and not read-only. bdrv_next() in turn returns all nodes that are either the root node of a BlockBackend or monitor-owned nodes.

With the typical -drive use, this worked well enough. However, in the typical -blockdev case, the user defines one node per option, making all nodes monitor-owned nodes. This includes protocol nodes etc. which often are not snapshottable, so "savevm" only returns an error.

Change the conditions so that internal snapshot still include all nodes that have a BlockBackend attached (we definitely want to snapshot anything attached to a guest device and probably also the built-in NBD server; snapshotting block job BlockBackends is more of an accident, but a preexisting one), but other monitor-owned nodes are only included if they have no parents.

This makes internal snapshots usable again with typical -blockdev configurations.

(cherry picked from commit 05f4aced658a02b02d3e89a6c7a2281008fcf26c)
---
 block/snapshot.c | 26 +++++++++++++++++++-------
 1 file changed, 19 insertions(+), 7 deletions(-)
block/snapshot.c +19 -7
index f2f48f926a..8081616ae9 100644
@@ -31,6 +31,7 @@
 #include "qapi/qmp/qerror.h"
 #include "qapi/qmp/qstring.h"
 #include "qemu/option.h"
+#include "sysemu/block-backend.h"
 
 QemuOptsList internal_snapshot_opts = {
     .name = "snapshot",
@@ -384,6 +385,16 @@ int bdrv_snapshot_load_tmp_by_id_or_name(BlockDriverState *bs,
     return ret;
 }
 
+static bool bdrv_all_snapshots_includes_bs(BlockDriverState *bs)
+{
+    if (!bdrv_is_inserted(bs) || bdrv_is_read_only(bs)) {
+        return false;
+    }
+
+    /* Include all nodes that are either in use by a BlockBackend, or that
+     * aren't attached to any node, but owned by the monitor. */
+    return bdrv_has_blk(bs) || QLIST_EMPTY(&bs->parents);
+}
 
 /* Group operations. All block drivers are involved.
  * These functions will properly handle dataplane (take aio_context_acquire
@@ -399,7 +410,7 @@ bool bdrv_all_can_snapshot(BlockDriverState **first_bad_bs)
         AioContext *ctx = bdrv_get_aio_context(bs);
 
         aio_context_acquire(ctx);
-        if (bdrv_is_inserted(bs) && !bdrv_is_read_only(bs)) {
+        if (bdrv_all_snapshots_includes_bs(bs)) {
             ok = bdrv_can_snapshot(bs);
         }
         aio_context_release(ctx);
@@ -426,8 +437,9 @@ int bdrv_all_delete_snapshot(const char *name, BlockDriverState **first_bad_bs,
         AioContext *ctx = bdrv_get_aio_context(bs);
 
         aio_context_acquire(ctx);
-        if (bdrv_can_snapshot(bs) &&
-                bdrv_snapshot_find(bs, snapshot, name) >= 0) {
+        if (bdrv_all_snapshots_includes_bs(bs) &&
+            bdrv_snapshot_find(bs, snapshot, name) >= 0)
+        {
             ret = bdrv_snapshot_delete(bs, snapshot->id_str,
                                        snapshot->name, err);
         }
@@ -455,7 +467,7 @@ int bdrv_all_goto_snapshot(const char *name, BlockDriverState **first_bad_bs,
         AioContext *ctx = bdrv_get_aio_context(bs);
 
         aio_context_acquire(ctx);
-        if (bdrv_can_snapshot(bs)) {
+        if (bdrv_all_snapshots_includes_bs(bs)) {
             ret = bdrv_snapshot_goto(bs, name, errp);
         }
         aio_context_release(ctx);
@@ -481,7 +493,7 @@ int bdrv_all_find_snapshot(const char *name, BlockDriverState **first_bad_bs)
         AioContext *ctx = bdrv_get_aio_context(bs);
 
         aio_context_acquire(ctx);
-        if (bdrv_can_snapshot(bs)) {
+        if (bdrv_all_snapshots_includes_bs(bs)) {
             err = bdrv_snapshot_find(bs, &sn, name);
         }
         aio_context_release(ctx);
@@ -512,7 +524,7 @@ int bdrv_all_create_snapshot(QEMUSnapshotInfo *sn,
         if (bs == vm_state_bs) {
             sn->vm_state_size = vm_state_size;
             err = bdrv_snapshot_create(bs, sn);
-        } else if (bdrv_can_snapshot(bs)) {
+        } else if (bdrv_all_snapshots_includes_bs(bs)) {
             sn->vm_state_size = 0;
             err = bdrv_snapshot_create(bs, sn);
         }
@@ -538,7 +550,7 @@ BlockDriverState *bdrv_all_find_vmstate_bs(void)
         bool found;
 
         aio_context_acquire(ctx);
-        found = bdrv_can_snapshot(bs);
+        found = bdrv_all_snapshots_includes_bs(bs) && bdrv_can_snapshot(bs);
         aio_context_release(ctx);
 
         if (found) {
Signature · 1 line
2.17.1
#40 lore raw

PATCH 40/55 iotests: Test internal snapshots with -blockdev

In reply to Michael Roth

From: Kevin Wolf <[email protected]>

(cherry picked from commit 92b22e7b1789b0e5f20d245706e72eae70dbddce)
---
 tests/qemu-iotests/267           | 168 ++++++++++++++++++++++++++++
 tests/qemu-iotests/267.out       | 182 +++++++++++++++++++++++++++++++
 tests/qemu-iotests/common.filter |  11 +-
 tests/qemu-iotests/group         |   1 +
 4 files changed, 358 insertions(+), 4 deletions(-)
 create mode 100755 tests/qemu-iotests/267
 create mode 100644 tests/qemu-iotests/267.out
tests/qemu-iotests/267 new file +168 -0
new file mode 100755
index 0000000000..d37a67c012
@@ -0,0 +1,168 @@
+#!/usr/bin/env bash
+#
+# Test which nodes are involved in internal snapshots
+#
+# Copyright (C) 2019 Red Hat, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+#
+
+# creator
[email protected]
+
+seq=`basename $0`
+echo "QA output created by $seq"
+
+status=1	# failure is the default!
+
+_cleanup()
+{
+    _cleanup_test_img
+    rm -f "$TEST_DIR/nbd"
+}
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+# get standard environment, filters and checks
+. ./common.rc
+. ./common.filter
+
+_supported_fmt qcow2
+_supported_proto file
+_supported_os Linux
+
+# Internal snapshots are (currently) impossible with refcount_bits=1
+_unsupported_imgopts 'refcount_bits=1[^0-9]'
+
+do_run_qemu()
+{
+    echo Testing: "$@"
+    (
+        if ! test -t 0; then
+            while read cmd; do
+                echo $cmd
+            done
+        fi
+        echo quit
+    ) | $QEMU -nographic -monitor stdio -nodefaults "$@"
+    echo
+}
+
+run_qemu()
+{
+    do_run_qemu "$@" 2>&1 | _filter_testdir | _filter_qemu | _filter_hmp |
+        _filter_generated_node_ids | _filter_imgfmt | _filter_vmstate_size
+}
+
+size=128M
+
+run_test()
+{
+    _make_test_img $size
+    printf "savevm snap0\ninfo snapshots\nloadvm snap0\n" | run_qemu "$@" | _filter_date
+}
+
+
+echo
+echo "=== No block devices at all ==="
+echo
+
+run_test
+
+echo
+echo "=== -drive if=none ==="
+echo
+
+run_test -drive driver=file,file="$TEST_IMG",if=none
+run_test -drive driver=$IMGFMT,file="$TEST_IMG",if=none
+run_test -drive driver=$IMGFMT,file="$TEST_IMG",if=none -device virtio-blk,drive=none0
+
+echo
+echo "=== -drive if=virtio ==="
+echo
+
+run_test -drive driver=file,file="$TEST_IMG",if=virtio
+run_test -drive driver=$IMGFMT,file="$TEST_IMG",if=virtio
+
+echo
+echo "=== Simple -blockdev ==="
+echo
+
+run_test -blockdev driver=file,filename="$TEST_IMG",node-name=file
+run_test -blockdev driver=file,filename="$TEST_IMG",node-name=file \
+         -blockdev driver=$IMGFMT,file=file,node-name=fmt
+run_test -blockdev driver=file,filename="$TEST_IMG",node-name=file \
+         -blockdev driver=raw,file=file,node-name=raw \
+         -blockdev driver=$IMGFMT,file=raw,node-name=fmt
+
+echo
+echo "=== -blockdev with a filter on top ==="
+echo
+
+run_test -blockdev driver=file,filename="$TEST_IMG",node-name=file \
+         -blockdev driver=$IMGFMT,file=file,node-name=fmt \
+         -blockdev driver=copy-on-read,file=fmt,node-name=filter
+
+echo
+echo "=== -blockdev with a backing file ==="
+echo
+
+TEST_IMG="$TEST_IMG.base" _make_test_img $size
+
+IMGOPTS="backing_file=$TEST_IMG.base" \
+run_test -blockdev driver=file,filename="$TEST_IMG.base",node-name=backing-file \
+         -blockdev driver=file,filename="$TEST_IMG",node-name=file \
+         -blockdev driver=$IMGFMT,file=file,backing=backing-file,node-name=fmt
+
+IMGOPTS="backing_file=$TEST_IMG.base" \
+run_test -blockdev driver=file,filename="$TEST_IMG.base",node-name=backing-file \
+         -blockdev driver=$IMGFMT,file=backing-file,node-name=backing-fmt \
+         -blockdev driver=file,filename="$TEST_IMG",node-name=file \
+         -blockdev driver=$IMGFMT,file=file,backing=backing-fmt,node-name=fmt
+
+# A snapshot should be present on the overlay, but not the backing file
+echo Internal snapshots on overlay:
+$QEMU_IMG snapshot -l "$TEST_IMG" | _filter_date | _filter_vmstate_size
+
+echo Internal snapshots on backing file:
+$QEMU_IMG snapshot -l "$TEST_IMG.base" | _filter_date | _filter_vmstate_size
+
+echo
+echo "=== -blockdev with NBD server on the backing file ==="
+echo
+
+IMGOPTS="backing_file=$TEST_IMG.base" _make_test_img $size
+cat <<EOF |
+nbd_server_start unix:$TEST_DIR/nbd
+nbd_server_add -w backing-fmt
+savevm snap0
+info snapshots
+loadvm snap0
+EOF
+run_qemu -blockdev driver=file,filename="$TEST_IMG.base",node-name=backing-file \
+         -blockdev driver=$IMGFMT,file=backing-file,node-name=backing-fmt \
+         -blockdev driver=file,filename="$TEST_IMG",node-name=file \
+         -blockdev driver=$IMGFMT,file=file,backing=backing-fmt,node-name=fmt |
+         _filter_date
+
+# This time, a snapshot should be created on both files
+echo Internal snapshots on overlay:
+$QEMU_IMG snapshot -l "$TEST_IMG" | _filter_date | _filter_vmstate_size
+
+echo Internal snapshots on backing file:
+$QEMU_IMG snapshot -l "$TEST_IMG.base" | _filter_date | _filter_vmstate_size
+
+# success, all done
+echo "*** done"
+rm -f $seq.full
+status=0
tests/qemu-iotests/267.out new file +182 -0

182 lines not on this page · raw message

tests/qemu-iotests/common.filter +7 -4
index 35fddc746f..67fd5f4e41 100644
@@ -19,12 +19,15 @@
 # standard filters
 #
 
-# ctime(3) dates
-#
 _filter_date()
 {
-    $SED \
-        -e 's/[A-Z][a-z][a-z] [A-z][a-z][a-z]  *[0-9][0-9]* [0-9][0-9]:[0-9][0-9]:[0-9][0-9] [0-9][0-9][0-9][0-9]$/DATE/'
+    $SED -re 's/[0-9]{4}-[0-9]{2}-[0-9]{2} [0-9]{2}:[0-9]{2}:[0-9]{2}/yyyy-mm-dd hh:mm:ss/'
+}
+
+_filter_vmstate_size()
+{
+    $SED -r -e 's/[0-9. ]{5} [KMGT]iB/     SIZE/' \
+            -e 's/[0-9. ]{5} B/   SIZE/'
 }
 
 _filter_generated_node_ids()
tests/qemu-iotests/group +1 -0
index 3660a741f2..8bc2f3857d 100644
@@ -273,3 +273,4 @@
 256 rw quick
 265 rw auto quick
 266 rw quick
+267 rw auto quick snapshot
Signature · 1 line
2.17.1
#41 lore raw

PATCH 41/55 vhost-user: save features if the char dev is closed

In reply to Michael Roth

From: Adrian Moreno <[email protected]>

That way the state can be correctly restored when the device is opened again. This might happen if the backend is restarted.

Fixes 6ab79a20af3a ("do not call vhost_net_cleanup() on running net from char user event")
Cc Michael S. Tsirkin <[email protected]>

Message-Id: <[email protected]>

(cherry picked from commit c6beefd674fff8d41b90365dfccad32e53a5abcb)
---
 net/vhost-user.c | 4 ++++
 1 file changed, 4 insertions(+)
net/vhost-user.c +4 -0
index 51921de443..014199d600 100644
@@ -235,6 +235,10 @@ static void chr_closed_bh(void *opaque)
 
     s = DO_UPCAST(NetVhostUserState, nc, ncs[0]);
 
+    if (s->vhost_net) {
+        s->acked_features = vhost_net_get_acked_features(s->vhost_net);
+    }
+
     qmp_set_link(name, false, &err);
 
     qemu_chr_fe_set_handlers(&s->chr, NULL, NULL, net_vhost_user_event,
Signature · 1 line
2.17.1
#42 lore raw

PATCH 42/55 hw/core/loader: Fix possible crash in rom_copy()

In reply to Michael Roth

From: Thomas Huth <[email protected]>

Both, "rom->addr" and "addr" are derived from the binary image that can be loaded with the "-kernel" paramer. The code in rom_copy() then calculates:

    d = dest + (rom->addr - addr);

and uses "d" as destination in a memcpy() some lines later. Now with bad kernel images, it is possible that rom->addr is smaller than addr, thus "rom->addr - addr" gets negative and the memcpy() then tries to copy contents from the image to a bad memory location. This could maybe be used to inject code from a kernel image into the QEMU binary, so we better fix it with an additional sanity check here.

Message-Id: <[email protected]>

(cherry picked from commit e423455c4f23a1a828901c78fe6d03b7dde79319)
---
 hw/core/loader.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
hw/core/loader.c +1 -1
index 425bf69a99..838a34174a 100644
@@ -1242,7 +1242,7 @@ int rom_copy(uint8_t *dest, hwaddr addr, size_t size)
         if (rom->addr + rom->romsize < addr) {
             continue;
         }
-        if (rom->addr > end) {
+        if (rom->addr > end || rom->addr < addr) {
             break;
         }
 
Signature · 1 line
2.17.1
#43 lore raw

PATCH 43/55 qcow2: Limit total allocation range to INT_MAX

In reply to Michael Roth

From: Max Reitz <[email protected]>

When the COW areas are included, the size of an allocation can exceed INT_MAX. This is kind of limited by handle_alloc() in that it already caps avail_bytes at INT_MAX, but the number of clusters still reflects the original length.

This can have all sorts of effects, ranging from the storage layer write call failing to image corruption. (If there were no image corruption, then I suppose there would be data loss because the .cow_end area is forced to be empty, even though there might be something we need to COW.)

Fix all of it by limiting nb_clusters so the equivalent number of bytes will not exceed INT_MAX.

(cherry picked from commit d1b9d19f99586b33795e20a79f645186ccbc070f)
---
 block/qcow2-cluster.c | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)
block/qcow2-cluster.c +4 -1
index 760564c8fb..f8576031b6 100644
@@ -1341,6 +1341,9 @@ static int handle_alloc(BlockDriverState *bs, uint64_t guest_offset,
     nb_clusters = MIN(nb_clusters, s->l2_slice_size - l2_index);
     assert(nb_clusters <= INT_MAX);
 
+    /* Limit total allocation byte count to INT_MAX */
+    nb_clusters = MIN(nb_clusters, INT_MAX >> s->cluster_bits);
+
     /* Find L2 entry for the first involved cluster */
     ret = get_cluster_table(bs, guest_offset, &l2_slice, &l2_index);
     if (ret < 0) {
@@ -1429,7 +1432,7 @@ static int handle_alloc(BlockDriverState *bs, uint64_t guest_offset,
      * request actually writes to (excluding COW at the end)
      */
     uint64_t requested_bytes = *bytes + offset_into_cluster(s, guest_offset);
-    int avail_bytes = MIN(INT_MAX, nb_clusters << s->cluster_bits);
+    int avail_bytes = nb_clusters << s->cluster_bits;
     int nb_bytes = MIN(requested_bytes, avail_bytes);
     QCowL2Meta *old_m = *m;
 
Signature · 1 line
2.17.1
#44 lore raw

PATCH 44/55 iotests: Test large write request to qcow2 file

In reply to Michael Roth

From: Max Reitz <[email protected]>

Without HEAD^, the following happens when you attempt a large write request to a qcow2 file such that the number of bytes covered by all clusters involved in a single allocation will exceed INT_MAX:

(A) handle_alloc_space() decides to fill the whole area with zeroes and fails because bdrv_co_pwrite_zeroes() fails (the request is too large).

(B) If handle_alloc_space() does not do anything, but merge_cow() decides that the requests can be merged, it will create a too long IOV that later cannot be written.

(C) Otherwise, all parts will be written separately, so those requests will work.

In either B or C, though, qcow2_alloc_cluster_link_l2() will have an
overflow: We use an int (i) to iterate over nb_clusters, and then calculate the L2 entry based on "i << s->cluster_bits" -- which will overflow if the range covers more than INT_MAX bytes. This then leads to image corruption because the L2 entry will be wrong (it will be recognized as a compressed cluster).

Even if that were not the case, the .cow_end area would be empty (because handle_alloc() will cap avail_bytes and nb_bytes at INT_MAX, so their difference (which is the .cow_end size) will be 0).

So this test checks that on such large requests, the image will not be corrupted. Unfortunately, we cannot check whether COW will be handled correctly, because that data is discarded when it is written to null-co (but we have to use null-co, because writing 2 GB of data in a test is not quite reasonable).

(cherry picked from commit a1406a9262a087d9ec9627b88da13c4590b61dae)

Conflicts:
tests/qemu-iotests/group
*drop context dep. on tests not in 4.1

---
 tests/qemu-iotests/270     | 83 ++++++++++++++++++++++++++++++++++++++
 tests/qemu-iotests/270.out |  9 +++++
 tests/qemu-iotests/group   |  1 +
 3 files changed, 93 insertions(+)
 create mode 100755 tests/qemu-iotests/270
 create mode 100644 tests/qemu-iotests/270.out
tests/qemu-iotests/270 new file +83 -0
new file mode 100755
index 0000000000..b9a12b908c
@@ -0,0 +1,83 @@
+#!/usr/bin/env bash
+#
+# Test large write to a qcow2 image
+#
+# Copyright (C) 2019 Red Hat, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+#
+
+seq=$(basename "$0")
+echo "QA output created by $seq"
+
+status=1	# failure is the default!
+
+_cleanup()
+{
+    _cleanup_test_img
+}
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+# get standard environment, filters and checks
+. ./common.rc
+. ./common.filter
+
+# This is a qcow2 regression test
+_supported_fmt qcow2
+_supported_proto file
+_supported_os Linux
+
+# We use our own external data file and our own cluster size, and we
+# require v3 images
+_unsupported_imgopts data_file cluster_size 'compat=0.10'
+
+
+# We need a backing file so that handle_alloc_space() will not do
+# anything.  (If it were to do anything, it would simply fail its
+# write-zeroes request because the request range is too large.)
+TEST_IMG="$TEST_IMG.base" _make_test_img 4G
+$QEMU_IO -c 'write 0 512' "$TEST_IMG.base" | _filter_qemu_io
+
+# (Use .orig because _cleanup_test_img will remove that file)
+# We need a large cluster size, see below for why (above the $QEMU_IO
+# invocation)
+_make_test_img -o cluster_size=2M,data_file="$TEST_IMG.orig" \
+    -b "$TEST_IMG.base" 4G
+
+# We want a null-co as the data file, because it allows us to quickly
+# "write" 2G of data without using any space.
+# (qemu-img create does not like it, though, because null-co does not
+# support image creation.)
+$QEMU_IMG amend -o data_file="json:{'driver':'null-co',,'size':'4294967296'}" \
+    "$TEST_IMG"
+
+# This gives us a range of:
+#   2^31 - 512 + 768 - 1 = 2^31 + 255 > 2^31
+# until the beginning of the end COW block.  (The total allocation
+# size depends on the cluster size, but all that is important is that
+# it exceeds INT_MAX.)
+#
+# 2^31 - 512 is the maximum request size.  We want this to result in a
+# single allocation, and because the qcow2 driver splits allocations
+# on L2 boundaries, we need large L2 tables; hence the cluster size of
+# 2 MB.  (Anything from 256 kB should work, though, because then one L2
+# table covers 8 GB.)
+$QEMU_IO -c "write 768 $((2 ** 31 - 512))" "$TEST_IMG" | _filter_qemu_io
+
+_check_test_img
+
+# success, all done
+echo "*** done"
+rm -f $seq.full
+status=0
tests/qemu-iotests/270.out new file +9 -0
new file mode 100644
index 0000000000..c7be111014
@@ -0,0 +1,9 @@
+QA output created by 270
+Formatting 'TEST_DIR/t.IMGFMT.base', fmt=IMGFMT size=4294967296
+wrote 512/512 bytes at offset 0
+512 bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=4294967296 backing_file=TEST_DIR/t.IMGFMT.base data_file=TEST_DIR/t.IMGFMT.orig
+wrote 2147483136/2147483136 bytes at offset 768
+2 GiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+No errors were found on the image.
+*** done
tests/qemu-iotests/group +1 -0
index 8bc2f3857d..7629e8ea17 100644
@@ -274,3 +274,4 @@
 265 rw auto quick
 266 rw quick
 267 rw auto quick snapshot
+270 rw backing quick
Signature · 1 line
2.17.1
#45 lore raw

PATCH 45/55 mirror: Do not dereference invalid pointers

In reply to Michael Roth

From: Max Reitz <[email protected]>

mirror_exit_common() may be called twice (if it is called from mirror_prepare() and fails, it will be called from mirror_abort() again).

In such a case, many of the pointers in the MirrorBlockJob object will already be freed. This can be seen most reliably for s->target, which is set to NULL (and then dereferenced by blk_bs()).

Fixes 737efc1eda23b904fbe0e66b37715fb0e5c3e58b

Message-id: [email protected]

(cherry picked from commit f93c3add3a773e0e3f6277e5517583c4ad3a43c2)
---
 block/mirror.c | 13 +++++++++----
 1 file changed, 9 insertions(+), 4 deletions(-)
block/mirror.c +9 -4
index 9f5c59ece1..0e3f7923cf 100644
@@ -618,11 +618,11 @@ static int mirror_exit_common(Job *job)
 {
     MirrorBlockJob *s = container_of(job, MirrorBlockJob, common.job);
     BlockJob *bjob = &s->common;
-    MirrorBDSOpaque *bs_opaque = s->mirror_top_bs->opaque;
+    MirrorBDSOpaque *bs_opaque;
     AioContext *replace_aio_context = NULL;
-    BlockDriverState *src = s->mirror_top_bs->backing->bs;
-    BlockDriverState *target_bs = blk_bs(s->target);
-    BlockDriverState *mirror_top_bs = s->mirror_top_bs;
+    BlockDriverState *src;
+    BlockDriverState *target_bs;
+    BlockDriverState *mirror_top_bs;
     Error *local_err = NULL;
     bool abort = job->ret < 0;
     int ret = 0;
@@ -632,6 +632,11 @@ static int mirror_exit_common(Job *job)
     }
     s->prepared = true;
 
+    mirror_top_bs = s->mirror_top_bs;
+    bs_opaque = mirror_top_bs->opaque;
+    src = mirror_top_bs->backing->bs;
+    target_bs = blk_bs(s->target);
+
     if (bdrv_chain_contains(src, target_bs)) {
         bdrv_unfreeze_backing_chain(mirror_top_bs, target_bs);
     }
Signature · 1 line
2.17.1
#46 lore raw

PATCH 46/55 ui: Fix hanging up Cocoa display on macOS 10.15 (Catalina)

In reply to Michael Roth

From: Hikaru Nishida <[email protected]>

macOS API documentation says that before applicationDidFinishLaunching is called, any events will not be processed. However, some events are fired before it is called in macOS Catalina. This causes deadlock of iothread_lock in handleEvent while it will be released after the app_started_sem is posted.
This patch avoids processing events before the app_started_sem is posted to prevent this deadlock.

Message-id: [email protected]

(cherry picked from commit dff742ad27efa474ec04accdbf422c9acfd3e30e)
---
 ui/cocoa.m | 12 ++++++++++++
 1 file changed, 12 insertions(+)
ui/cocoa.m +12 -0
index c2984028c5..3026ead621 100644
@@ -132,6 +132,7 @@ NSArray * supportedImageFileTypes;
 
 static QemuSemaphore display_init_sem;
 static QemuSemaphore app_started_sem;
+static bool allow_events;
 
 // Utility functions to run specified code block with iothread lock held
 typedef void (^CodeBlock)(void);
@@ -727,6 +728,16 @@ QemuCocoaView *cocoaView;
 
 - (bool) handleEvent:(NSEvent *)event
 {
+    if(!allow_events) {
+        /*
+         * Just let OSX have all events that arrive before
+         * applicationDidFinishLaunching.
+         * This avoids a deadlock on the iothread lock, which cocoa_display_init()
+         * will not drop until after the app_started_sem is posted. (In theory
+         * there should not be any such events, but OSX Catalina now emits some.)
+         */
+        return false;
+    }
     return bool_with_iothread_lock(^{
         return [self handleEventLocked:event];
     });
@@ -1154,6 +1165,7 @@ QemuCocoaView *cocoaView;
 - (void)applicationDidFinishLaunching: (NSNotification *) note
 {
     COCOA_DEBUG("QemuCocoaAppController: applicationDidFinishLaunching\n");
+    allow_events = true;
     /* Tell cocoa_display_init to proceed */
     qemu_sem_post(&app_started_sem);
 }
Signature · 1 line
2.17.1
#47 lore raw

PATCH 47/55 virtio: new post_load hook

In reply to Michael Roth

From: "Michael S. Tsirkin" <[email protected]>

Post load hook in virtio vmsd is called early while device is processed, and when VirtIODevice core isn't fully initialized. Most device specific code isn't ready to deal with a device in such state, and behaves weirdly.

Add a new post_load hook in a device class instead. Devices should use this unless they specifically want to verify the migration stream as it's processed, e.g. for bounds checking.

Cc Mikhail Sennikovsky <[email protected]>
(cherry picked from commit 1dd713837cac8ec5a97d3b8492d72ce5ac94803c)
---
 hw/virtio/virtio.c         | 7 +++++++
 include/hw/virtio/virtio.h | 6 ++++++
 2 files changed, 13 insertions(+)
hw/virtio/virtio.c +7 -0
index a94ea18a9c..7c3822c3a0 100644
@@ -2287,6 +2287,13 @@ int virtio_load(VirtIODevice *vdev, QEMUFile *f, int version_id)
     }
     rcu_read_unlock();
 
+    if (vdc->post_load) {
+        ret = vdc->post_load(vdev);
+        if (ret) {
+            return ret;
+        }
+    }
+
     return 0;
 }
 
include/hw/virtio/virtio.h +6 -0
index b189788cb2..f9f62370e9 100644
@@ -158,6 +158,12 @@ typedef struct VirtioDeviceClass {
      */
     void (*save)(VirtIODevice *vdev, QEMUFile *f);
     int (*load)(VirtIODevice *vdev, QEMUFile *f, int version_id);
+    /* Post load hook in vmsd is called early while device is processed, and
+     * when VirtIODevice isn't fully initialized.  Devices should use this instead,
+     * unless they specifically want to verify the migration stream as it's
+     * processed, e.g. for bounds checking.
+     */
+    int (*post_load)(VirtIODevice *vdev);
     const VMStateDescription *vmsd;
 } VirtioDeviceClass;
 
Signature · 1 line
2.17.1
#48 lore raw

PATCH 48/55 virtio-net: prevent offloads reset on migration

In reply to Michael Roth

From: Mikhail Sennikovsky <[email protected]>

Currently offloads disabled by guest via the VIRTIO_NET_CTRL_GUEST_OFFLOADS_SET command are not preserved on VM migration.
Instead all offloads reported by guest features (via VIRTIO_PCI_GUEST_FEATURES) get enabled.
What happens is: first the VirtIONet::curr_guest_offloads gets restored and offloads are getting set correctly:

 #0  qemu_set_offload (nc=0x555556a11400, csum=1, tso4=0, tso6=0, ecn=0, ufo=0) at net/net.c:474
 #1  virtio_net_apply_guest_offloads (n=0x555557701ca0) at hw/net/virtio-net.c:720
 #2  virtio_net_post_load_device (opaque=0x555557701ca0, version_id=11) at hw/net/virtio-net.c:2334
 #3  vmstate_load_state (f=0x5555569dc010, vmsd=0x555556577c80 <vmstate_virtio_net_device>, opaque=0x555557701ca0, version_id=11)
     at migration/vmstate.c:168
 #4  virtio_load (vdev=0x555557701ca0, f=0x5555569dc010, version_id=11) at hw/virtio/virtio.c:2197
 #5  virtio_device_get (f=0x5555569dc010, opaque=0x555557701ca0, size=0, field=0x55555668cd00 <__compound_literal.5>) at hw/virtio/virtio.c:2036
 #6  vmstate_load_state (f=0x5555569dc010, vmsd=0x555556577ce0 <vmstate_virtio_net>, opaque=0x555557701ca0, version_id=11) at migration/vmstate.c:143
 #7  vmstate_load (f=0x5555569dc010, se=0x5555578189e0) at migration/savevm.c:829
 #8  qemu_loadvm_section_start_full (f=0x5555569dc010, mis=0x5555569eee20) at migration/savevm.c:2211
 #9  qemu_loadvm_state_main (f=0x5555569dc010, mis=0x5555569eee20) at migration/savevm.c:2395
 #10 qemu_loadvm_state (f=0x5555569dc010) at migration/savevm.c:2467
 #11 process_incoming_migration_co (opaque=0x0) at migration/migration.c:449

However later on the features are getting restored, and offloads get reset to everything supported by features:

 #0  qemu_set_offload (nc=0x555556a11400, csum=1, tso4=1, tso6=1, ecn=0, ufo=0) at net/net.c:474
 #1  virtio_net_apply_guest_offloads (n=0x555557701ca0) at hw/net/virtio-net.c:720
 #2  virtio_net_set_features (vdev=0x555557701ca0, features=5104441767) at hw/net/virtio-net.c:773
 #3  virtio_set_features_nocheck (vdev=0x555557701ca0, val=5104441767) at hw/virtio/virtio.c:2052
 #4  virtio_load (vdev=0x555557701ca0, f=0x5555569dc010, version_id=11) at hw/virtio/virtio.c:2220
 #5  virtio_device_get (f=0x5555569dc010, opaque=0x555557701ca0, size=0, field=0x55555668cd00 <__compound_literal.5>) at hw/virtio/virtio.c:2036
 #6  vmstate_load_state (f=0x5555569dc010, vmsd=0x555556577ce0 <vmstate_virtio_net>, opaque=0x555557701ca0, version_id=11) at migration/vmstate.c:143
 #7  vmstate_load (f=0x5555569dc010, se=0x5555578189e0) at migration/savevm.c:829
 #8  qemu_loadvm_section_start_full (f=0x5555569dc010, mis=0x5555569eee20) at migration/savevm.c:2211
 #9  qemu_loadvm_state_main (f=0x5555569dc010, mis=0x5555569eee20) at migration/savevm.c:2395

#10 qemu_loadvm_state (f=0x5555569dc010) at migration/savevm.c:2467 #11 process_incoming_migration_co (opaque=0x0) at migration/migration.c:449

Fix this by preserving the state in saved_guest_offloads field and pushing out offload initialization to the new post load hook.

(cherry picked from commit 7788c3f2e21e35902d45809b236791383bbb613e)
---
 hw/net/virtio-net.c            | 27 ++++++++++++++++++++++++---
 include/hw/virtio/virtio-net.h |  2 ++
 2 files changed, 26 insertions(+), 3 deletions(-)
hw/net/virtio-net.c +24 -3
index b9e1cd71cf..6adb0fe252 100644
@@ -2330,9 +2330,13 @@ static int virtio_net_post_load_device(void *opaque, int version_id)
         n->curr_guest_offloads = virtio_net_supported_guest_offloads(n);
     }
 
-    if (peer_has_vnet_hdr(n)) {
-        virtio_net_apply_guest_offloads(n);
-    }
+    /*
+     * curr_guest_offloads will be later overwritten by the
+     * virtio_set_features_nocheck call done from the virtio_load.
+     * Here we make sure it is preserved and restored accordingly
+     * in the virtio_net_post_load_virtio callback.
+     */
+    n->saved_guest_offloads = n->curr_guest_offloads;
 
     virtio_net_set_queues(n);
 
@@ -2367,6 +2371,22 @@ static int virtio_net_post_load_device(void *opaque, int version_id)
     return 0;
 }
 
+static int virtio_net_post_load_virtio(VirtIODevice *vdev)
+{
+    VirtIONet *n = VIRTIO_NET(vdev);
+    /*
+     * The actual needed state is now in saved_guest_offloads,
+     * see virtio_net_post_load_device for detail.
+     * Restore it back and apply the desired offloads.
+     */
+    n->curr_guest_offloads = n->saved_guest_offloads;
+    if (peer_has_vnet_hdr(n)) {
+        virtio_net_apply_guest_offloads(n);
+    }
+
+    return 0;
+}
+
 /* tx_waiting field of a VirtIONetQueue */
 static const VMStateDescription vmstate_virtio_net_queue_tx_waiting = {
     .name = "virtio-net-queue-tx_waiting",
@@ -2909,6 +2929,7 @@ static void virtio_net_class_init(ObjectClass *klass, void *data)
     vdc->guest_notifier_mask = virtio_net_guest_notifier_mask;
     vdc->guest_notifier_pending = virtio_net_guest_notifier_pending;
     vdc->legacy_features |= (0x1 << VIRTIO_NET_F_GSO);
+    vdc->post_load = virtio_net_post_load_virtio;
     vdc->vmsd = &vmstate_virtio_net_device;
 }
 
include/hw/virtio/virtio-net.h +2 -0
index b96f0c643f..07a9319f4b 100644
@@ -182,6 +182,8 @@ struct VirtIONet {
     char *netclient_name;
     char *netclient_type;
     uint64_t curr_guest_offloads;
+    /* used on saved state restore phase to preserve the curr_guest_offloads */
+    uint64_t saved_guest_offloads;
     AnnounceTimer announce_timer;
     bool needs_vnet_hdr_swap;
     bool mtu_bypass_backend;
Signature · 1 line
2.17.1
#49 lore raw

PATCH 49/55 COLO-compare: Fix incorrect `if` logic

In reply to Michael Roth

From: Fan Yang <[email protected]>

'colo_mark_tcp_pkt' should return 'true' when packets are the same, and 'false' otherwise. However, it returns 'true' when 'colo_compare_packet_payload' returns non-zero while 'colo_compare_packet_payload' is just a 'memcmp'. The result is that COLO-compare reports inconsistent TCP packets when they are actually the same.

Fixes f449c9e549c ("colo: compare the packet based on the tcp sequence number")
(cherry picked from commit 1e907a32b77e5d418538453df5945242e43224fa)
---
 net/colo-compare.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
net/colo-compare.c +3 -3
index 7489840bde..7ee17f2cf8 100644
@@ -319,7 +319,7 @@ static bool colo_mark_tcp_pkt(Packet *ppkt, Packet *spkt,
     *mark = 0;
 
     if (ppkt->tcp_seq == spkt->tcp_seq && ppkt->seq_end == spkt->seq_end) {
-        if (colo_compare_packet_payload(ppkt, spkt,
+        if (!colo_compare_packet_payload(ppkt, spkt,
                                         ppkt->header_size, spkt->header_size,
                                         ppkt->payload_size)) {
             *mark = COLO_COMPARE_FREE_SECONDARY | COLO_COMPARE_FREE_PRIMARY;
@@ -329,7 +329,7 @@ static bool colo_mark_tcp_pkt(Packet *ppkt, Packet *spkt,
 
     /* one part of secondary packet payload still need to be compared */
     if (!after(ppkt->seq_end, spkt->seq_end)) {
-        if (colo_compare_packet_payload(ppkt, spkt,
+        if (!colo_compare_packet_payload(ppkt, spkt,
                                         ppkt->header_size + ppkt->offset,
                                         spkt->header_size + spkt->offset,
                                         ppkt->payload_size - ppkt->offset)) {
@@ -348,7 +348,7 @@ static bool colo_mark_tcp_pkt(Packet *ppkt, Packet *spkt,
         /* primary packet is longer than secondary packet, compare
          * the same part and mark the primary packet offset
          */
-        if (colo_compare_packet_payload(ppkt, spkt,
+        if (!colo_compare_packet_payload(ppkt, spkt,
                                         ppkt->header_size + ppkt->offset,
                                         spkt->header_size + spkt->offset,
                                         spkt->payload_size - spkt->offset)) {
Signature · 1 line
2.17.1
#50 lore raw

PATCH 50/55 util/hbitmap: strict hbitmap_reset

In reply to Michael Roth

From: Vladimir Sementsov-Ogievskiy <[email protected]>

hbitmap_reset has an unobvious property: it rounds requested region up. It may provoke bugs, like in recently fixed write-blocking mode of
mirror: user calls reset on unaligned region, not keeping in mind that there are possible unrelated dirty bytes, covered by rounded-up region and information of this unrelated "dirtiness" will be lost.

Make hbitmap_reset strict: assert that arguments are aligned, allowing only one exception when @start + @count == hb->orig_size. It's needed to comfort users of hbitmap_next_dirty_area, which cares about hb->orig_size.

Message-Id: <[email protected]>

[Maintainer edit: Max's suggestions from on-list. --js]
[Maintainer edit: Eric's suggestion for aligned macro. --js]
(cherry picked from commit 48557b138383aaf69c2617ca9a88bfb394fc50ec)

*prereq for fed33bd175f663cc8c13f8a490a4f35a19756cfe

---
 include/qemu/hbitmap.h | 5 +++++
 tests/test-hbitmap.c   | 2 +-
 util/hbitmap.c         | 4 ++++
 3 files changed, 10 insertions(+), 1 deletion(-)
include/qemu/hbitmap.h +5 -0
index 4afbe6292e..1bf944ca3d 100644
@@ -132,6 +132,11 @@ void hbitmap_set(HBitmap *hb, uint64_t start, uint64_t count);
  * @count: Number of bits to reset.
  *
  * Reset a consecutive range of bits in an HBitmap.
+ * @start and @count must be aligned to bitmap granularity. The only exception
+ * is resetting the tail of the bitmap: @count may be equal to hb->orig_size -
+ * @start, in this case @count may be not aligned. The sum of @start + @count is
+ * allowed to be greater than hb->orig_size, but only if @start < hb->orig_size
+ * and @start + @count = ALIGN_UP(hb->orig_size, granularity).
  */
 void hbitmap_reset(HBitmap *hb, uint64_t start, uint64_t count);
 
tests/test-hbitmap.c +1 -1
index 592d8219db..2be56d1597 100644
@@ -423,7 +423,7 @@ static void test_hbitmap_granularity(TestHBitmapData *data,
     hbitmap_test_check(data, 0);
     hbitmap_test_set(data, 0, 3);
     g_assert_cmpint(hbitmap_count(data->hb), ==, 4);
-    hbitmap_test_reset(data, 0, 1);
+    hbitmap_test_reset(data, 0, 2);
     g_assert_cmpint(hbitmap_count(data->hb), ==, 2);
 }
 
util/hbitmap.c +4 -0
index bcc0acdc6a..71c6ba2c52 100644
@@ -476,6 +476,10 @@ void hbitmap_reset(HBitmap *hb, uint64_t start, uint64_t count)
     /* Compute range in the last layer.  */
     uint64_t first;
     uint64_t last = start + count - 1;
+    uint64_t gran = 1ULL << hb->granularity;
+
+    assert(QEMU_IS_ALIGNED(start, gran));
+    assert(QEMU_IS_ALIGNED(count, gran) || (start + count == hb->orig_size));
 
     trace_hbitmap_reset(hb, start, count,
                         start >> hb->granularity, last >> hb->granularity);
Signature · 1 line
2.17.1
#51 lore raw

PATCH 51/55 hbitmap: handle set/reset with zero length

In reply to Michael Roth

From: Vladimir Sementsov-Ogievskiy <[email protected]>

Passing zero length to these functions leads to unpredicted results. Zero-length set/reset may occur in active-mirror, on zero-length write (which is unlikely, but not guaranteed to never happen).

Let's just do nothing on zero-length request.

Message-id: [email protected]

(cherry picked from commit fed33bd175f663cc8c13f8a490a4f35a19756cfe)
---
 util/hbitmap.c | 8 ++++++++
 1 file changed, 8 insertions(+)
util/hbitmap.c +8 -0
index 71c6ba2c52..c059313b9e 100644
@@ -387,6 +387,10 @@ void hbitmap_set(HBitmap *hb, uint64_t start, uint64_t count)
     uint64_t first, n;
     uint64_t last = start + count - 1;
 
+    if (count == 0) {
+        return;
+    }
+
     trace_hbitmap_set(hb, start, count,
                       start >> hb->granularity, last >> hb->granularity);
 
@@ -478,6 +482,10 @@ void hbitmap_reset(HBitmap *hb, uint64_t start, uint64_t count)
     uint64_t last = start + count - 1;
     uint64_t gran = 1ULL << hb->granularity;
 
+    if (count == 0) {
+        return;
+    }
+
     assert(QEMU_IS_ALIGNED(start, gran));
     assert(QEMU_IS_ALIGNED(count, gran) || (start + count == hb->orig_size));
 
Signature · 1 line
2.17.1
#52 lore raw

PATCH 52/55 target/arm: Allow reading flags from FPSCR for M-profile

In reply to Michael Roth

From: Christophe Lyon <[email protected]>

rt==15 is a special case when reading the flags: it means the destination is APSR. This patch avoids rejecting vmrs apsr_nzcv, fpscr
as illegal instruction.

Message-id: [email protected]

[PMM: updated the comment]
(cherry picked from commit 2529ab43b8a05534494704e803e0332d111d8b91)
---
 target/arm/translate-vfp.inc.c | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)
target/arm/translate-vfp.inc.c +3 -2
index ef45cecbea..75406fd9db 100644
@@ -704,9 +704,10 @@ static bool trans_VMSR_VMRS(DisasContext *s, arg_VMSR_VMRS *a)
     if (arm_dc_feature(s, ARM_FEATURE_M)) {
         /*
          * The only M-profile VFP vmrs/vmsr sysreg is FPSCR.
-         * Writes to R15 are UNPREDICTABLE; we choose to undef.
+         * Accesses to R15 are UNPREDICTABLE; we choose to undef.
+         * (FPSCR -> r15 is a special case which writes to the PSR flags.)
          */
-        if (a->rt == 15 || a->reg != ARM_VFP_FPSCR) {
+        if (a->rt == 15 && (!a->l || a->reg != ARM_VFP_FPSCR)) {
             return false;
         }
     }
Signature · 1 line
2.17.1
#53 lore raw

PATCH 53/55 target/xtensa: regenerate and re-import test_mmuhifi_c3 core

In reply to Michael Roth

From: Max Filippov <[email protected]>

Overlay part of the test_mmuhifi_c3 core has GPL3 copyright headers in it. Fix that by regenerating test_mmuhifi_c3 core overlay and re-importing it.

Fixes d848ea776728 ("target/xtensa: add test_mmuhifi_c3 core")
(cherry picked from commit d5eaec84e592bb0085f84bef54d0a41e31faa99a)
---
 target/xtensa/core-test_mmuhifi_c3.c          |    3 +-
 target/xtensa/core-test_mmuhifi_c3/core-isa.h |  116 +-
 .../core-test_mmuhifi_c3/gdb-config.inc.c     |  114 +-
 .../core-test_mmuhifi_c3/xtensa-modules.inc.c | 6384 ++++++++---------
 4 files changed, 3385 insertions(+), 3232 deletions(-)
target/xtensa/core-test_mmuhifi_c3.c +1 -2
index 3a59fefa94..089ed7da5d 100644
@@ -27,8 +27,8 @@
 
 #include "qemu/osdep.h"
 #include "cpu.h"
-#include "exec/exec-all.h"
 #include "exec/gdbstub.h"
+#include "qemu-common.h"
 #include "qemu/host-utils.h"
 
 #include "core-test_mmuhifi_c3/core-isa.h"
@@ -39,7 +39,6 @@
 
 static XtensaConfig test_mmuhifi_c3 __attribute__((unused)) = {
     .name = "test_mmuhifi_c3",
-    .options = XTENSA_OPTIONS,
     .gdb_regmap = {
         .reg = {
 #include "core-test_mmuhifi_c3/gdb-config.inc.c"
target/xtensa/core-test_mmuhifi_c3/core-isa.h +103 -13

236 lines not on this page · raw message

target/xtensa/core-test_mmuhifi_c3/gdb-config.inc.c +97 -17

132 lines not on this page · raw message

target/xtensa/core-test_mmuhifi_c3/xtensa-modules.inc.c +3184 -3200

2,616 lines not on this page · raw message

The rest of this diff is not shown. View the raw message.

Signature · 1 line
2.17.1

Part of the diff is not shown here. View the raw message.

#54 lore raw

PATCH 54/55 scsi: lsi: exit infinite loop while executing script (CVE-2019-12068)

In reply to Michael Roth

From: Paolo Bonzini <[email protected]>

When executing script in lsi_execute_script(), the LSI scsi adapter emulator advances 's->dsp' index to read next opcode. This can lead to an infinite loop if the next opcode is empty. Move the existing loop exit after 10k iterations so that it covers no-op opcodes as well.

(cherry picked from commit de594e47659029316bbf9391efb79da0a1a08e08)
---
 hw/scsi/lsi53c895a.c | 41 +++++++++++++++++++++++++++--------------
 1 file changed, 27 insertions(+), 14 deletions(-)
hw/scsi/lsi53c895a.c +27 -14
index 10468c1ec1..72f7b59ab5 100644
@@ -185,6 +185,9 @@ static const char *names[] = {
 /* Flag set if this is a tagged command.  */
 #define LSI_TAG_VALID     (1 << 16)
 
+/* Maximum instructions to process. */
+#define LSI_MAX_INSN    10000
+
 typedef struct lsi_request {
     SCSIRequest *req;
     uint32_t tag;
@@ -1132,7 +1135,21 @@ static void lsi_execute_script(LSIState *s)
 
     s->istat1 |= LSI_ISTAT1_SRUN;
 again:
-    insn_processed++;
+    if (++insn_processed > LSI_MAX_INSN) {
+        /* Some windows drivers make the device spin waiting for a memory
+           location to change.  If we have been executed a lot of code then
+           assume this is the case and force an unexpected device disconnect.
+           This is apparently sufficient to beat the drivers into submission.
+         */
+        if (!(s->sien0 & LSI_SIST0_UDC)) {
+            qemu_log_mask(LOG_GUEST_ERROR,
+                          "lsi_scsi: inf. loop with UDC masked");
+        }
+        lsi_script_scsi_interrupt(s, LSI_SIST0_UDC, 0);
+        lsi_disconnect(s);
+        trace_lsi_execute_script_stop();
+        return;
+    }
     insn = read_dword(s, s->dsp);
     if (!insn) {
         /* If we receive an empty opcode increment the DSP by 4 bytes
@@ -1569,19 +1586,7 @@ again:
             }
         }
     }
-    if (insn_processed > 10000 && s->waiting == LSI_NOWAIT) {
-        /* Some windows drivers make the device spin waiting for a memory
-           location to change.  If we have been executed a lot of code then
-           assume this is the case and force an unexpected device disconnect.
-           This is apparently sufficient to beat the drivers into submission.
-         */
-        if (!(s->sien0 & LSI_SIST0_UDC)) {
-            qemu_log_mask(LOG_GUEST_ERROR,
-                          "lsi_scsi: inf. loop with UDC masked");
-        }
-        lsi_script_scsi_interrupt(s, LSI_SIST0_UDC, 0);
-        lsi_disconnect(s);
-    } else if (s->istat1 & LSI_ISTAT1_SRUN && s->waiting == LSI_NOWAIT) {
+    if (s->istat1 & LSI_ISTAT1_SRUN && s->waiting == LSI_NOWAIT) {
         if (s->dcntl & LSI_DCNTL_SSM) {
             lsi_script_dma_interrupt(s, LSI_DSTAT_SSI);
         } else {
@@ -1969,6 +1974,10 @@ static void lsi_reg_writeb(LSIState *s, int offset, uint8_t val)
     case 0x2f: /* DSP[24:31] */
         s->dsp &= 0x00ffffff;
         s->dsp |= val << 24;
+        /*
+         * FIXME: if s->waiting != LSI_NOWAIT, this will only execute one
+         * instruction.  Is this correct?
+         */
         if ((s->dmode & LSI_DMODE_MAN) == 0
             && (s->istat1 & LSI_ISTAT1_SRUN) == 0)
             lsi_execute_script(s);
@@ -1987,6 +1996,10 @@ static void lsi_reg_writeb(LSIState *s, int offset, uint8_t val)
         break;
     case 0x3b: /* DCNTL */
         s->dcntl = val & ~(LSI_DCNTL_PFF | LSI_DCNTL_STD);
+        /*
+         * FIXME: if s->waiting != LSI_NOWAIT, this will only execute one
+         * instruction.  Is this correct?
+         */
         if ((val & LSI_DCNTL_STD) && (s->istat1 & LSI_ISTAT1_SRUN) == 0)
             lsi_execute_script(s);
         break;
Signature · 1 line
2.17.1
#55 lore raw

PATCH 55/55 virtio-blk: Cancel the pending BH when the dataplane is reset

In reply to Michael Roth

From: Philippe Mathieu-Daudé <[email protected]>

When 'system_reset' is called, the main loop clear the memory region cache before the BH has a chance to execute. Later when the deferred function is called, some assumptions that were made when scheduling them are no longer true when they actually execute.

This is what happens using a virtio-blk device (fresh RHEL7.8 install):

 $ (sleep 12.3; echo system_reset; sleep 12.3; echo system_reset; sleep 1; echo q) \
   | qemu-system-x86_64 -m 4G -smp 8 -boot menu=on \
     -device virtio-blk-pci,id=image1,drive=drive_image1 \
     -drive file=/var/lib/libvirt/images/rhel78.qcow2,if=none,id=drive_image1,format=qcow2,cache=none \
     -device virtio-net-pci,netdev=net0,id=nic0,mac=52:54:00:c4:e7:84 \
     -netdev tap,id=net0,script=/bin/true,downscript=/bin/true,vhost=on \
     -monitor stdio -serial null -nographic
  (qemu) system_reset
  (qemu) system_reset
  (qemu) qemu-system-x86_64: hw/virtio/virtio.c:225: vring_get_region_caches: Assertion `caches != NULL' failed.
  Aborted

(gdb) bt
Thread 1 (Thread 0x7f109c17b680 (LWP 10939)):

  #0  0x00005604083296d1 in vring_get_region_caches (vq=0x56040a24bdd0) at hw/virtio/virtio.c:227
  #1  0x000056040832972b in vring_avail_flags (vq=0x56040a24bdd0) at hw/virtio/virtio.c:235
  #2  0x000056040832d13d in virtio_should_notify (vdev=0x56040a240630, vq=0x56040a24bdd0) at hw/virtio/virtio.c:1648
  #3  0x000056040832d1f8 in virtio_notify_irqfd (vdev=0x56040a240630, vq=0x56040a24bdd0) at hw/virtio/virtio.c:1662
  #4  0x00005604082d213d in notify_guest_bh (opaque=0x56040a243ec0) at hw/block/dataplane/virtio-blk.c:75
  #5  0x000056040883dc35 in aio_bh_call (bh=0x56040a243f10) at util/async.c:90
  #6  0x000056040883dccd in aio_bh_poll (ctx=0x560409161980) at util/async.c:118
  #7  0x0000560408842af7 in aio_dispatch (ctx=0x560409161980) at util/aio-posix.c:460
  #8  0x000056040883e068 in aio_ctx_dispatch (source=0x560409161980, callback=0x0, user_data=0x0) at util/async.c:261
  #9  0x00007f10a8fca06d in g_main_context_dispatch () at /lib64/libglib-2.0.so.0

#10 0x0000560408841445 in glib_pollfds_poll () at util/main-loop.c:215
#11 0x00005604088414bf in os_host_main_loop_wait (timeout=0) at util/main-loop.c:238 #12 0x00005604088415c4 in main_loop_wait (nonblocking=0) at util/main-loop.c:514 #13 0x0000560408416b1e in main_loop () at vl.c:1923
#14 0x000056040841e0e8 in main (argc=20, argv=0x7ffc2c3f9c58, envp=0x7ffc2c3f9d00) at vl.c:4578

Fix this by cancelling the BH when the virtio dataplane is stopped.

[This is version of the patch was modified as discussed with Philippe on the mailing list thread.
--Stefan]

Message-Id: <[email protected]>

(cherry picked from commit ebb6ff25cd888a52a64a9adc3692541c6d1d9a42)
---
 hw/block/dataplane/virtio-blk.c | 3 +++
 1 file changed, 3 insertions(+)
hw/block/dataplane/virtio-blk.c +3 -0
index 158c78f852..5fea76df85 100644
@@ -297,6 +297,9 @@ void virtio_blk_data_plane_stop(VirtIODevice *vdev)
         virtio_bus_cleanup_host_notifier(VIRTIO_BUS(qbus), i);
     }
 
+    qemu_bh_cancel(s->bh);
+    notify_guest_bh(s); /* final chance to notify guest */
+
     /* Clean up guest notifier (irq) */
     k->set_guest_notifiers(qbus->parent, nvqs, false);
 
Signature · 1 line
2.17.1
#56 lore raw
In reply to Michael Roth
Quoted 21 lines · Michael Roth

Hi everyone,

The following new patches are queued for QEMU stable v4.1.1:

https://github.com/mdroth/qemu/commits/stable-4.1-staging

The release is tentatively planned for 2019-11-14:

https://wiki.qemu.org/Planning/4.1

Please note that the original release date was planned for 2019-11-21, but was moved up to address a number of qcow2 corruption issues:

https://lists.gnu.org/archive/html/qemu-devel/2019-10/msg07144.html

Fixes for the XFS issues noted in the thread are still pending, but will hopefully be qemu.git master in time for 4.1.1 freeze and the currently-scheduled release date for 4.2.0-rc1.

The list of still-pending patchsets being tracked for inclusion are:

Quoted 7 lines

qcow2: Fix data corruption on XFS

    https://lists.gnu.org/archive/html/qemu-devel/2019-11/msg00073.html
    (PULL pending)
  qcow2: Fix QCOW2_COMPRESSED_SECTOR_MASK
    https://lists.gnu.org/archive/html/qemu-devel/2019-10/msg07718.html
  qcow2-bitmap: Fix uint64_t left-shift overflow
    https://lists.gnu.org/archive/html/qemu-devel/2019-10/msg07989.html

Thanks for tracking these, all of these are in master now. (So they should be in time.)

Max

#57 lore raw
In reply to Michael Roth
Quoted 29 lines · Michael Roth

Hi everyone,

The following new patches are queued for QEMU stable v4.1.1:

https://github.com/mdroth/qemu/commits/stable-4.1-staging

The release is tentatively planned for 2019-11-14:

https://wiki.qemu.org/Planning/4.1

Please note that the original release date was planned for 2019-11-21, but was moved up to address a number of qcow2 corruption issues:

https://lists.gnu.org/archive/html/qemu-devel/2019-10/msg07144.html

Fixes for the XFS issues noted in the thread are still pending, but will hopefully be qemu.git master in time for 4.1.1 freeze and the currently-scheduled release date for 4.2.0-rc1.

The list of still-pending patchsets being tracked for inclusion are:

qcow2: Fix data corruption on XFS

    https://lists.gnu.org/archive/html/qemu-devel/2019-11/msg00073.html
    (PULL pending)
  qcow2: Fix QCOW2_COMPRESSED_SECTOR_MASK
    https://lists.gnu.org/archive/html/qemu-devel/2019-10/msg07718.html
  qcow2-bitmap: Fix uint64_t left-shift overflow
    https://lists.gnu.org/archive/html/qemu-devel/2019-10/msg07989.html
Quoted 5 lines

Please respond here or CC [email protected] on any additional patches you think should be included in the release.

Thanks!

Extra patches we are carrying in Fedora 31. First 2 were requested as fixes for gluster 4k issues. Last one fixes tests on newer kernels

commit a6b257a08e3d72219f03e461a52152672fec0612

Author: Nir Soffer <[email protected]>
Date:   Tue Aug 13 21:21:03 2019 +0300

file-posix: Handle undetectable alignment

commit 1fa6975773bb7dcb27dd5a248c0ccfe839b83178

Author: Nir Soffer <[email protected]>
Date:   Tue Aug 27 04:05:27 2019 +0300

block: posix: Always allocate the first block

commit d4f42d8c648d7e94b408e8056483189c27cf53bf

Author: Daniel P. Berrangé <[email protected]>
Date:   Wed Aug 21 16:14:27 2019 +0100

tests: make filemonitor test more robust to event ordering

Thanks,
Cole

#58 lore raw
Quoted 30 lines · Cole Robinson
Quoted 29 lines · Michael Roth

Hi everyone,

The following new patches are queued for QEMU stable v4.1.1:

https://github.com/mdroth/qemu/commits/stable-4.1-staging

The release is tentatively planned for 2019-11-14:

https://wiki.qemu.org/Planning/4.1

Please note that the original release date was planned for 2019-11-21, but was moved up to address a number of qcow2 corruption issues:

https://lists.gnu.org/archive/html/qemu-devel/2019-10/msg07144.html

Fixes for the XFS issues noted in the thread are still pending, but will hopefully be qemu.git master in time for 4.1.1 freeze and the currently-scheduled release date for 4.2.0-rc1.

The list of still-pending patchsets being tracked for inclusion are:

qcow2: Fix data corruption on XFS

    https://lists.gnu.org/archive/html/qemu-devel/2019-11/msg00073.html
    (PULL pending)
  qcow2: Fix QCOW2_COMPRESSED_SECTOR_MASK
    https://lists.gnu.org/archive/html/qemu-devel/2019-10/msg07718.html
  qcow2-bitmap: Fix uint64_t left-shift overflow
    https://lists.gnu.org/archive/html/qemu-devel/2019-10/msg07989.html
Quoted 9 lines
Quoted 5 lines

Please respond here or CC [email protected] on any additional patches you think should be included in the release.

Thanks!

Extra patches we are carrying in Fedora 31. First 2 were requested as fixes for gluster 4k issues. Last one fixes tests on newer kernels

Sorry, wrong commit IDs. Actual commits are

commit a6b257a08e3d72219f03e461a52152672fec0612

Author: Nir Soffer <[email protected]>
Date:   Tue Aug 13 21:21:03 2019 +0300

file-posix: Handle undetectable alignment

commit 3a20013fbb26d2a1bd11ef148eefdb1508783787

Author: Nir Soffer <[email protected]>
Date:   Tue Aug 27 04:05:27 2019 +0300

block: posix: Always allocate the first block

commit bf9e0313c27d8e6ecd7f7de3d63e1cb25d8f6311

Author: Daniel P. Berrangé <[email protected]>
Date:   Wed Aug 21 16:14:27 2019 +0100

tests: make filemonitor test more robust to event ordering

Thanks,
Cole

#59 lore raw
In reply to Michael Roth

Quoting Michael Roth (2019-11-05 14:51:48)

Quoted 29 lines

Hi everyone,

The following new patches are queued for QEMU stable v4.1.1:

https://github.com/mdroth/qemu/commits/stable-4.1-staging

The release is tentatively planned for 2019-11-14:

https://wiki.qemu.org/Planning/4.1

Please note that the original release date was planned for 2019-11-21, but was moved up to address a number of qcow2 corruption issues:

https://lists.gnu.org/archive/html/qemu-devel/2019-10/msg07144.html

Fixes for the XFS issues noted in the thread are still pending, but will hopefully be qemu.git master in time for 4.1.1 freeze and the currently-scheduled release date for 4.2.0-rc1.

The list of still-pending patchsets being tracked for inclusion are:

qcow2: Fix data corruption on XFS

    https://lists.gnu.org/archive/html/qemu-devel/2019-11/msg00073.html
    (PULL pending)
  qcow2: Fix QCOW2_COMPRESSED_SECTOR_MASK
    https://lists.gnu.org/archive/html/qemu-devel/2019-10/msg07718.html
  qcow2-bitmap: Fix uint64_t left-shift overflow
    https://lists.gnu.org/archive/html/qemu-devel/2019-10/msg07989.html
Quoted 2 lines

Please respond here or CC [email protected] on any additional patches you think should be included in the release.

The following additional patches have been pushed to the staging tree:

tests: make filemonitor test more robust to event ordering
block: posix: Always allocate the first block
file-posix: Handle undetectable alignment block/file-posix: Let post-EOF fallocate serialize
block: Add bdrv_co_get_self_request()
block: Make wait/mark serialising requests public block/io: refactor padding
util/iov: improve qemu_iovec_is_zero
util/iov: introduce qemu_iovec_init_extended
qcow2-bitmap: Fix uint64_t left-shift overflow
iotests: Add peek_file* functions
iotests: Add test for 4G+ compressed qcow2 write
qcow2: Fix QCOW2_COMPRESSED_SECTOR_MASK

Thank you for the suggestions.

Quoted 213 lines

Thanks!

----------------------------------------------------------------
Adrian Moreno (1):
vhost-user: save features if the char dev is closed

Alberto Garcia (1):
qcow2: Fix the calculation of the maximum L2 cache size

Anthony PERARD (1):
xen-bus: Fix backend state transition on device reset

Aurelien Jarno (1):
target/alpha: fix tlb_fill trap_arg2 value for instruction fetch

Christophe Lyon (1):
target/arm: Allow reading flags from FPSCR for M-profile

David Hildenbrand (1):

      s390x/tcg: Fix VERIM with 32/64 bit elements

Eduardo Habkost (1):
pc: Don't make die-id mandatory unless necessary

Fan Yang (1):
COLO-compare: Fix incorrect if logic

Hikaru Nishida (1):
ui: Fix hanging up Cocoa display on macOS 10.15 (Catalina)

Igor Mammedov (1):
x86: do not advertise die-id in query-hotpluggbale-cpus if '-smp dies' is not set

Johannes Berg (1):
libvhost-user: fix SLAVE_SEND_FD handling

John Snow (2):
Revert "ide/ahci: Check for -ECANCELED in aio callbacks"
iotests: add testing shim for script-style python tests

Kevin Wolf (4):
coroutine: Add qemu_co_mutex_assert_locked()
qcow2: Fix corruption bug in qcow2_detect_metadata_preallocation() block/snapshot: Restrict set of snapshot nodes
iotests: Test internal snapshots with -blockdev

Markus Armbruster (1):
pr-manager: Fix invalid g_free() crash bug

Matthew Rosato (1):
s390: PCI: fix IOMMU region init

Max Filippov (1):
target/xtensa: regenerate and re-import test_mmuhifi_c3 core

Max Reitz (16):
block/file-posix: Reduce xfsctl() use
iotests: Test reverse sub-cluster qcow2 writes
vpc: Return 0 from vpc_co_create() on success
iotests: Add supported protocols to execute_test()
iotests: Restrict file Python tests to file
iotests: Restrict nbd Python tests to nbd
iotests: Test blockdev-create for vpc
curl: Keep pointer to the CURLState in CURLSocket
curl: Keep *socket until the end of curl_sock_cb()
curl: Check completion in curl_multi_do()
curl: Pass CURLSocket to curl_multi_do()
curl: Report only ready sockets
curl: Handle success in multi_check_completion
qcow2: Limit total allocation range to INT_MAX
iotests: Test large write request to qcow2 file
mirror: Do not dereference invalid pointers

Maxim Levitsky (1):
block/qcow2: Fix corruption introduced by commit 8ac0f15f335

Michael Roth (2):
make-release: pull in edk2 submodules so we can build it from tarballs roms/Makefile.edk2: don't pull in submodules when building from tarball

Michael S. Tsirkin (1):
virtio: new post_load hook

Mikhail Sennikovsky (1):
virtio-net: prevent offloads reset on migration

Paolo Bonzini (2):
dma-helpers: ensure AIO callback is invoked after cancellation
scsi: lsi: exit infinite loop while executing script (CVE-2019-12068)

Paul Durrant (1):
xen-bus: check whether the frontend is active during device reset...

Peter Lieven (1):
block/nfs: tear down aio before nfs_close

Peter Maydell (3):
target/arm: Free TCG temps in trans_VMOV_64_sp() target/arm: Don't abort on M-profile exception return in linux-user mode hw/arm/boot.c: Set NSACR.{CP11,CP10} for NS kernel boots

Philippe Mathieu-Daudé (1):
virtio-blk: Cancel the pending BH when the dataplane is reset

Sergio Lopez (1):
blockjob: update nodes head while removing all bdrv

Thomas Huth (1):
hw/core/loader: Fix possible crash in rom_copy()

Vladimir Sementsov-Ogievskiy (4):
block/backup: fix max_transfer handling for copy_range block/backup: fix backup_cow_with_offload for last cluster util/hbitmap: strict hbitmap_reset
hbitmap: handle set/reset with zero length

 block/backup.c                                     |   17 +-
 block/curl.c                                       |  125 +-
 block/file-posix.c                                 |   77 +-
 block/mirror.c                                     |   13 +-
 block/nfs.c                                        |    6 +-
 block/qcow2-cluster.c                              |   12 +-
 block/qcow2-refcount.c                             |    2 +
 block/qcow2.c                                      |    9 +-
 block/snapshot.c                                   |   26 +-
 block/vpc.c                                        |    3 +-
 blockjob.c                                         |   17 +-
 contrib/libvhost-user/libvhost-user.c              |    3 +-
 dma-helpers.c                                      |   13 +-
 hw/arm/boot.c                                      |    2 +
 hw/block/dataplane/virtio-blk.c                    |    3 +
 hw/core/loader.c                                   |    2 +-
 hw/i386/pc.c                                       |   14 +-
 hw/ide/ahci.c                                      |    3 -
 hw/ide/core.c                                      |   14 -
 hw/net/virtio-net.c                                |   27 +-
 hw/s390x/s390-pci-bus.c                            |    7 +-
 hw/scsi/lsi53c895a.c                               |   41 +-
 hw/virtio/virtio.c                                 |    7 +
 hw/xen/xen-bus.c                                   |   23 +-
 include/hw/virtio/virtio-net.h                     |    2 +
 include/hw/virtio/virtio.h                         |    6 +
 include/qemu/coroutine.h                           |   15 +
 include/qemu/hbitmap.h                             |    5 +
 net/colo-compare.c                                 |    6 +-
 net/vhost-user.c                                   |    4 +
 roms/Makefile.edk2                                 |    7 +-
 scripts/make-release                               |    8 +
 scsi/pr-manager.c                                  |    1 -
 target/alpha/helper.c                              |    4 +-
 target/arm/translate-vfp.inc.c                     |    7 +-
 target/arm/translate.c                             |   21 +-
 target/s390x/translate_vx.inc.c                    |    2 +-
 target/xtensa/core-test_mmuhifi_c3.c               |    3 +-
 target/xtensa/core-test_mmuhifi_c3/core-isa.h      |  116 +-
 .../xtensa/core-test_mmuhifi_c3/gdb-config.inc.c   |  114 +-
 .../core-test_mmuhifi_c3/xtensa-modules.inc.c      | 6384 ++++++++++----------
 tests/acceptance/pc_cpu_hotplug_props.py           |   35 +
 tests/qemu-iotests/030                             |    3 +-
 tests/qemu-iotests/040                             |    3 +-
 tests/qemu-iotests/041                             |    3 +-
 tests/qemu-iotests/044                             |    3 +-
 tests/qemu-iotests/045                             |    3 +-
 tests/qemu-iotests/055                             |    3 +-
 tests/qemu-iotests/056                             |    3 +-
 tests/qemu-iotests/057                             |    3 +-
 tests/qemu-iotests/065                             |    3 +-
 tests/qemu-iotests/096                             |    3 +-
 tests/qemu-iotests/118                             |    3 +-
 tests/qemu-iotests/124                             |    3 +-
 tests/qemu-iotests/129                             |    3 +-
 tests/qemu-iotests/132                             |    3 +-
 tests/qemu-iotests/139                             |    3 +-
 tests/qemu-iotests/147                             |    5 +-
 tests/qemu-iotests/148                             |    3 +-
 tests/qemu-iotests/151                             |    3 +-
 tests/qemu-iotests/152                             |    3 +-
 tests/qemu-iotests/155                             |    3 +-
 tests/qemu-iotests/163                             |    3 +-
 tests/qemu-iotests/165                             |    3 +-
 tests/qemu-iotests/169                             |    3 +-
 tests/qemu-iotests/196                             |    3 +-
 tests/qemu-iotests/199                             |    3 +-
 tests/qemu-iotests/205                             |    3 +-
 tests/qemu-iotests/245                             |    3 +-
 tests/qemu-iotests/265                             |   67 +
 tests/qemu-iotests/265.out                         |    6 +
 tests/qemu-iotests/266                             |  153 +
 tests/qemu-iotests/266.out                         |  137 +
 tests/qemu-iotests/267                             |  168 +
 tests/qemu-iotests/267.out                         |  182 +
 tests/qemu-iotests/270                             |   83 +
 tests/qemu-iotests/270.out                         |    9 +
 tests/qemu-iotests/common.filter                   |   11 +-
 tests/qemu-iotests/group                           |    4 +
 tests/qemu-iotests/iotests.py                      |   42 +-
 tests/test-hbitmap.c                               |    2 +-
 ui/cocoa.m                                         |   12 +
 util/hbitmap.c                                     |   12 +
 83 files changed, 4663 insertions(+), 3514 deletions(-)
 create mode 100644 tests/acceptance/pc_cpu_hotplug_props.py
 create mode 100755 tests/qemu-iotests/265
 create mode 100644 tests/qemu-iotests/265.out
 create mode 100755 tests/qemu-iotests/266
 create mode 100644 tests/qemu-iotests/266.out
 create mode 100755 tests/qemu-iotests/267
 create mode 100644 tests/qemu-iotests/267.out
 create mode 100755 tests/qemu-iotests/270
 create mode 100644 tests/qemu-iotests/270.out
#60 lore raw

Quoting Michael Roth (2019-11-12 12:05:14)

Quoted 35 lines

Quoting Michael Roth (2019-11-05 14:51:48)

Quoted 29 lines

Hi everyone,

The following new patches are queued for QEMU stable v4.1.1:

https://github.com/mdroth/qemu/commits/stable-4.1-staging

The release is tentatively planned for 2019-11-14:

https://wiki.qemu.org/Planning/4.1

Please note that the original release date was planned for 2019-11-21, but was moved up to address a number of qcow2 corruption issues:

https://lists.gnu.org/archive/html/qemu-devel/2019-10/msg07144.html

Fixes for the XFS issues noted in the thread are still pending, but will hopefully be qemu.git master in time for 4.1.1 freeze and the currently-scheduled release date for 4.2.0-rc1.

The list of still-pending patchsets being tracked for inclusion are:

qcow2: Fix data corruption on XFS

    https://lists.gnu.org/archive/html/qemu-devel/2019-11/msg00073.html
    (PULL pending)
  qcow2: Fix QCOW2_COMPRESSED_SECTOR_MASK
    https://lists.gnu.org/archive/html/qemu-devel/2019-10/msg07718.html
  qcow2-bitmap: Fix uint64_t left-shift overflow
    https://lists.gnu.org/archive/html/qemu-devel/2019-10/msg07989.html
Quoted 2 lines

Please respond here or CC [email protected] on any additional patches you think should be included in the release.

The following additional patches have been pushed to the staging tree:

Quoted 13 lines

tests: make filemonitor test more robust to event ordering
block: posix: Always allocate the first block
file-posix: Handle undetectable alignment block/file-posix: Let post-EOF fallocate serialize
block: Add bdrv_co_get_self_request()
block: Make wait/mark serialising requests public block/io: refactor padding
util/iov: improve qemu_iovec_is_zero
util/iov: introduce qemu_iovec_init_extended
qcow2-bitmap: Fix uint64_t left-shift overflow
iotests: Add peek_file* functions
iotests: Add test for 4G+ compressed qcow2 write
qcow2: Fix QCOW2_COMPRESSED_SECTOR_MASK

The following additional patches have been pushed to the staging tree:

mirror: Keep mirror_top_bs drained after dropping permissions block/create: Do not abort if a block driver is not available
vhost: Fix memory region section comparison
memory: Provide an equality function for MemoryRegionSections
memory: Align MemoryRegionSections fields

Quoted 217 lines

Thank you for the suggestions.

Quoted 213 lines

Thanks!

----------------------------------------------------------------
Adrian Moreno (1):
vhost-user: save features if the char dev is closed

Alberto Garcia (1):
qcow2: Fix the calculation of the maximum L2 cache size

Anthony PERARD (1):
xen-bus: Fix backend state transition on device reset

Aurelien Jarno (1):
target/alpha: fix tlb_fill trap_arg2 value for instruction fetch

Christophe Lyon (1):
target/arm: Allow reading flags from FPSCR for M-profile

David Hildenbrand (1):

      s390x/tcg: Fix VERIM with 32/64 bit elements

Eduardo Habkost (1):
pc: Don't make die-id mandatory unless necessary

Fan Yang (1):
COLO-compare: Fix incorrect if logic

Hikaru Nishida (1):
ui: Fix hanging up Cocoa display on macOS 10.15 (Catalina)

Igor Mammedov (1):
x86: do not advertise die-id in query-hotpluggbale-cpus if '-smp dies' is not set

Johannes Berg (1):
libvhost-user: fix SLAVE_SEND_FD handling

John Snow (2):
Revert "ide/ahci: Check for -ECANCELED in aio callbacks"
iotests: add testing shim for script-style python tests

Kevin Wolf (4):
coroutine: Add qemu_co_mutex_assert_locked()
qcow2: Fix corruption bug in qcow2_detect_metadata_preallocation() block/snapshot: Restrict set of snapshot nodes
iotests: Test internal snapshots with -blockdev

Markus Armbruster (1):
pr-manager: Fix invalid g_free() crash bug

Matthew Rosato (1):
s390: PCI: fix IOMMU region init

Max Filippov (1):
target/xtensa: regenerate and re-import test_mmuhifi_c3 core

Max Reitz (16):
block/file-posix: Reduce xfsctl() use
iotests: Test reverse sub-cluster qcow2 writes
vpc: Return 0 from vpc_co_create() on success
iotests: Add supported protocols to execute_test()
iotests: Restrict file Python tests to file
iotests: Restrict nbd Python tests to nbd
iotests: Test blockdev-create for vpc
curl: Keep pointer to the CURLState in CURLSocket
curl: Keep *socket until the end of curl_sock_cb()
curl: Check completion in curl_multi_do()
curl: Pass CURLSocket to curl_multi_do()
curl: Report only ready sockets
curl: Handle success in multi_check_completion
qcow2: Limit total allocation range to INT_MAX
iotests: Test large write request to qcow2 file
mirror: Do not dereference invalid pointers

Maxim Levitsky (1):
block/qcow2: Fix corruption introduced by commit 8ac0f15f335

Michael Roth (2):
make-release: pull in edk2 submodules so we can build it from tarballs roms/Makefile.edk2: don't pull in submodules when building from tarball

Michael S. Tsirkin (1):
virtio: new post_load hook

Mikhail Sennikovsky (1):
virtio-net: prevent offloads reset on migration

Paolo Bonzini (2):
dma-helpers: ensure AIO callback is invoked after cancellation
scsi: lsi: exit infinite loop while executing script (CVE-2019-12068)

Paul Durrant (1):
xen-bus: check whether the frontend is active during device reset...

Peter Lieven (1):
block/nfs: tear down aio before nfs_close

Peter Maydell (3):
target/arm: Free TCG temps in trans_VMOV_64_sp() target/arm: Don't abort on M-profile exception return in linux-user mode hw/arm/boot.c: Set NSACR.{CP11,CP10} for NS kernel boots

Philippe Mathieu-Daudé (1):
virtio-blk: Cancel the pending BH when the dataplane is reset

Sergio Lopez (1):
blockjob: update nodes head while removing all bdrv

Thomas Huth (1):
hw/core/loader: Fix possible crash in rom_copy()

Vladimir Sementsov-Ogievskiy (4):
block/backup: fix max_transfer handling for copy_range block/backup: fix backup_cow_with_offload for last cluster util/hbitmap: strict hbitmap_reset
hbitmap: handle set/reset with zero length

 block/backup.c                                     |   17 +-
 block/curl.c                                       |  125 +-
 block/file-posix.c                                 |   77 +-
 block/mirror.c                                     |   13 +-
 block/nfs.c                                        |    6 +-
 block/qcow2-cluster.c                              |   12 +-
 block/qcow2-refcount.c                             |    2 +
 block/qcow2.c                                      |    9 +-
 block/snapshot.c                                   |   26 +-
 block/vpc.c                                        |    3 +-
 blockjob.c                                         |   17 +-
 contrib/libvhost-user/libvhost-user.c              |    3 +-
 dma-helpers.c                                      |   13 +-
 hw/arm/boot.c                                      |    2 +
 hw/block/dataplane/virtio-blk.c                    |    3 +
 hw/core/loader.c                                   |    2 +-
 hw/i386/pc.c                                       |   14 +-
 hw/ide/ahci.c                                      |    3 -
 hw/ide/core.c                                      |   14 -
 hw/net/virtio-net.c                                |   27 +-
 hw/s390x/s390-pci-bus.c                            |    7 +-
 hw/scsi/lsi53c895a.c                               |   41 +-
 hw/virtio/virtio.c                                 |    7 +
 hw/xen/xen-bus.c                                   |   23 +-
 include/hw/virtio/virtio-net.h                     |    2 +
 include/hw/virtio/virtio.h                         |    6 +
 include/qemu/coroutine.h                           |   15 +
 include/qemu/hbitmap.h                             |    5 +
 net/colo-compare.c                                 |    6 +-
 net/vhost-user.c                                   |    4 +
 roms/Makefile.edk2                                 |    7 +-
 scripts/make-release                               |    8 +
 scsi/pr-manager.c                                  |    1 -
 target/alpha/helper.c                              |    4 +-
 target/arm/translate-vfp.inc.c                     |    7 +-
 target/arm/translate.c                             |   21 +-
 target/s390x/translate_vx.inc.c                    |    2 +-
 target/xtensa/core-test_mmuhifi_c3.c               |    3 +-
 target/xtensa/core-test_mmuhifi_c3/core-isa.h      |  116 +-
 .../xtensa/core-test_mmuhifi_c3/gdb-config.inc.c   |  114 +-
 .../core-test_mmuhifi_c3/xtensa-modules.inc.c      | 6384 ++++++++++----------
 tests/acceptance/pc_cpu_hotplug_props.py           |   35 +
 tests/qemu-iotests/030                             |    3 +-
 tests/qemu-iotests/040                             |    3 +-
 tests/qemu-iotests/041                             |    3 +-
 tests/qemu-iotests/044                             |    3 +-
 tests/qemu-iotests/045                             |    3 +-
 tests/qemu-iotests/055                             |    3 +-
 tests/qemu-iotests/056                             |    3 +-
 tests/qemu-iotests/057                             |    3 +-
 tests/qemu-iotests/065                             |    3 +-
 tests/qemu-iotests/096                             |    3 +-
 tests/qemu-iotests/118                             |    3 +-
 tests/qemu-iotests/124                             |    3 +-
 tests/qemu-iotests/129                             |    3 +-
 tests/qemu-iotests/132                             |    3 +-
 tests/qemu-iotests/139                             |    3 +-
 tests/qemu-iotests/147                             |    5 +-
 tests/qemu-iotests/148                             |    3 +-
 tests/qemu-iotests/151                             |    3 +-
 tests/qemu-iotests/152                             |    3 +-
 tests/qemu-iotests/155                             |    3 +-
 tests/qemu-iotests/163                             |    3 +-
 tests/qemu-iotests/165                             |    3 +-
 tests/qemu-iotests/169                             |    3 +-
 tests/qemu-iotests/196                             |    3 +-
 tests/qemu-iotests/199                             |    3 +-
 tests/qemu-iotests/205                             |    3 +-
 tests/qemu-iotests/245                             |    3 +-
 tests/qemu-iotests/265                             |   67 +
 tests/qemu-iotests/265.out                         |    6 +
 tests/qemu-iotests/266                             |  153 +
 tests/qemu-iotests/266.out                         |  137 +
 tests/qemu-iotests/267                             |  168 +
 tests/qemu-iotests/267.out                         |  182 +
 tests/qemu-iotests/270                             |   83 +
 tests/qemu-iotests/270.out                         |    9 +
 tests/qemu-iotests/common.filter                   |   11 +-
 tests/qemu-iotests/group                           |    4 +
 tests/qemu-iotests/iotests.py                      |   42 +-
 tests/test-hbitmap.c                               |    2 +-
 ui/cocoa.m                                         |   12 +
 util/hbitmap.c                                     |   12 +
 83 files changed, 4663 insertions(+), 3514 deletions(-)
 create mode 100644 tests/acceptance/pc_cpu_hotplug_props.py
 create mode 100755 tests/qemu-iotests/265
 create mode 100644 tests/qemu-iotests/265.out
 create mode 100755 tests/qemu-iotests/266
 create mode 100644 tests/qemu-iotests/266.out
 create mode 100755 tests/qemu-iotests/267
 create mode 100644 tests/qemu-iotests/267.out
 create mode 100755 tests/qemu-iotests/270
 create mode 100644 tests/qemu-iotests/270.out
#61 lore raw
In reply to Michael Roth
Quoted 248 lines · Michael Roth

Hi everyone,

The following new patches are queued for QEMU stable v4.1.1:

https://github.com/mdroth/qemu/commits/stable-4.1-staging

The release is tentatively planned for 2019-11-14:

https://wiki.qemu.org/Planning/4.1

Please note that the original release date was planned for 2019-11- 21,
but was moved up to address a number of qcow2 corruption issues:

https://lists.gnu.org/archive/html/qemu-devel/2019-10/msg07144.html

Fixes for the XFS issues noted in the thread are still pending, but will
hopefully be qemu.git master in time for 4.1.1 freeze and the currently-scheduled release date for 4.2.0-rc1.

The list of still-pending patchsets being tracked for inclusion are:

qcow2: Fix data corruption on XFS

https://lists.gnu.org/archive/html/qemu-devel/2019-11/msg00073.html
(PULL pending)
qcow2: Fix QCOW2_COMPRESSED_SECTOR_MASK

https://lists.gnu.org/archive/html/qemu-devel/2019-10/msg07718.html
qcow2-bitmap: Fix uint64_t left-shift overflow

https://lists.gnu.org/archive/html/qemu-devel/2019-10/msg07989.html

Please respond here or CC [email protected] on any additional patches
you think should be included in the release.

Thanks!

----------------------------------------------------------------
Adrian Moreno (1):
vhost-user: save features if the char dev is closed

Alberto Garcia (1):
qcow2: Fix the calculation of the maximum L2 cache size

Anthony PERARD (1):
xen-bus: Fix backend state transition on device reset

Aurelien Jarno (1):
target/alpha: fix tlb_fill trap_arg2 value for instruction
fetch

Christophe Lyon (1):
target/arm: Allow reading flags from FPSCR for M-profile

David Hildenbrand (1):

      s390x/tcg: Fix VERIM with 32/64 bit elements

Eduardo Habkost (1):
pc: Don't make die-id mandatory unless necessary

Fan Yang (1):
COLO-compare: Fix incorrect if logic

Hikaru Nishida (1):
ui: Fix hanging up Cocoa display on macOS 10.15 (Catalina)

Igor Mammedov (1):
x86: do not advertise die-id in query-hotpluggbale-cpus if '-
smp dies' is not set

Johannes Berg (1):
libvhost-user: fix SLAVE_SEND_FD handling

John Snow (2):
Revert "ide/ahci: Check for -ECANCELED in aio callbacks"
iotests: add testing shim for script-style python tests

Kevin Wolf (4):
coroutine: Add qemu_co_mutex_assert_locked()
qcow2: Fix corruption bug in
qcow2_detect_metadata_preallocation()
block/snapshot: Restrict set of snapshot nodes
iotests: Test internal snapshots with -blockdev

Markus Armbruster (1):
pr-manager: Fix invalid g_free() crash bug

Matthew Rosato (1):
s390: PCI: fix IOMMU region init

Max Filippov (1):
target/xtensa: regenerate and re-import test_mmuhifi_c3 core

Max Reitz (16):
block/file-posix: Reduce xfsctl() use
iotests: Test reverse sub-cluster qcow2 writes
vpc: Return 0 from vpc_co_create() on success
iotests: Add supported protocols to execute_test()
iotests: Restrict file Python tests to file
iotests: Restrict nbd Python tests to nbd
iotests: Test blockdev-create for vpc
curl: Keep pointer to the CURLState in CURLSocket
curl: Keep *socket until the end of curl_sock_cb()
curl: Check completion in curl_multi_do()
curl: Pass CURLSocket to curl_multi_do()
curl: Report only ready sockets
curl: Handle success in multi_check_completion
qcow2: Limit total allocation range to INT_MAX
iotests: Test large write request to qcow2 file
mirror: Do not dereference invalid pointers

Maxim Levitsky (1):
block/qcow2: Fix corruption introduced by commit 8ac0f15f335

Michael Roth (2):
make-release: pull in edk2 submodules so we can build it from
tarballs
roms/Makefile.edk2: don't pull in submodules when building from
tarball

Michael S. Tsirkin (1):
virtio: new post_load hook

Mikhail Sennikovsky (1):
virtio-net: prevent offloads reset on migration

Paolo Bonzini (2):
dma-helpers: ensure AIO callback is invoked after cancellation
scsi: lsi: exit infinite loop while executing script (CVE-2019-
12068)

Paul Durrant (1):
xen-bus: check whether the frontend is active during device
reset...

Peter Lieven (1):
block/nfs: tear down aio before nfs_close

Peter Maydell (3):
target/arm: Free TCG temps in trans_VMOV_64_sp() target/arm: Don't abort on M-profile exception return in linux-
user mode
hw/arm/boot.c: Set NSACR.{CP11,CP10} for NS kernel boots

Philippe Mathieu-Daudé (1):
virtio-blk: Cancel the pending BH when the dataplane is reset

Sergio Lopez (1):
blockjob: update nodes head while removing all bdrv

Thomas Huth (1):
hw/core/loader: Fix possible crash in rom_copy()

Vladimir Sementsov-Ogievskiy (4):
block/backup: fix max_transfer handling for copy_range block/backup: fix backup_cow_with_offload for last cluster util/hbitmap: strict hbitmap_reset
hbitmap: handle set/reset with zero length

 block/backup.c                                     |   17 +-
 block/curl.c                                       |  125 +-
 block/file-posix.c                                 |   77 +-
 block/mirror.c                                     |   13 +-
 block/nfs.c                                        |    6 +-
 block/qcow2-cluster.c                              |   12 +-
 block/qcow2-refcount.c                             |    2 +
 block/qcow2.c                                      |    9 +-
 block/snapshot.c                                   |   26 +-
 block/vpc.c                                        |    3 +-
 blockjob.c                                         |   17 +-
 contrib/libvhost-user/libvhost-user.c              |    3 +-
 dma-helpers.c                                      |   13 +-
 hw/arm/boot.c                                      |    2 +
 hw/block/dataplane/virtio-blk.c                    |    3 +
 hw/core/loader.c                                   |    2 +-
 hw/i386/pc.c                                       |   14 +-
 hw/ide/ahci.c                                      |    3 -
 hw/ide/core.c                                      |   14 -
 hw/net/virtio-net.c                                |   27 +-
 hw/s390x/s390-pci-bus.c                            |    7 +-
 hw/scsi/lsi53c895a.c                               |   41 +-
 hw/virtio/virtio.c                                 |    7 +
 hw/xen/xen-bus.c                                   |   23 +-
 include/hw/virtio/virtio-net.h                     |    2 +
 include/hw/virtio/virtio.h                         |    6 +
 include/qemu/coroutine.h                           |   15 +
 include/qemu/hbitmap.h                             |    5 +
 net/colo-compare.c                                 |    6 +-
 net/vhost-user.c                                   |    4 +
 roms/Makefile.edk2                                 |    7 +-
 scripts/make-release                               |    8 +
 scsi/pr-manager.c                                  |    1 -
 target/alpha/helper.c                              |    4 +-
 target/arm/translate-vfp.inc.c                     |    7 +-
 target/arm/translate.c                             |   21 +-
 target/s390x/translate_vx.inc.c                    |    2 +-
 target/xtensa/core-test_mmuhifi_c3.c               |    3 +-
 target/xtensa/core-test_mmuhifi_c3/core-isa.h      |  116 +-
 .../xtensa/core-test_mmuhifi_c3/gdb-config.inc.c   |  114 +-
 .../core-test_mmuhifi_c3/xtensa-modules.inc.c      | 6384
++++++++++----------
 tests/acceptance/pc_cpu_hotplug_props.py           |   35 +
 tests/qemu-iotests/030                             |    3 +-
 tests/qemu-iotests/040                             |    3 +-
 tests/qemu-iotests/041                             |    3 +-
 tests/qemu-iotests/044                             |    3 +-
 tests/qemu-iotests/045                             |    3 +-
 tests/qemu-iotests/055                             |    3 +-
 tests/qemu-iotests/056                             |    3 +-
 tests/qemu-iotests/057                             |    3 +-
 tests/qemu-iotests/065                             |    3 +-
 tests/qemu-iotests/096                             |    3 +-
 tests/qemu-iotests/118                             |    3 +-
 tests/qemu-iotests/124                             |    3 +-
 tests/qemu-iotests/129                             |    3 +-
 tests/qemu-iotests/132                             |    3 +-
 tests/qemu-iotests/139                             |    3 +-
 tests/qemu-iotests/147                             |    5 +-
 tests/qemu-iotests/148                             |    3 +-
 tests/qemu-iotests/151                             |    3 +-
 tests/qemu-iotests/152                             |    3 +-
 tests/qemu-iotests/155                             |    3 +-
 tests/qemu-iotests/163                             |    3 +-
 tests/qemu-iotests/165                             |    3 +-
 tests/qemu-iotests/169                             |    3 +-
 tests/qemu-iotests/196                             |    3 +-
 tests/qemu-iotests/199                             |    3 +-
 tests/qemu-iotests/205                             |    3 +-
 tests/qemu-iotests/245                             |    3 +-
 tests/qemu-iotests/265                             |   67 +
 tests/qemu-iotests/265.out                         |    6 +
 tests/qemu-iotests/266                             |  153 +
 tests/qemu-iotests/266.out                         |  137 +
 tests/qemu-iotests/267                             |  168 +
 tests/qemu-iotests/267.out                         |  182 +
 tests/qemu-iotests/270                             |   83 +
 tests/qemu-iotests/270.out                         |    9 +
 tests/qemu-iotests/common.filter                   |   11 +-
 tests/qemu-iotests/group                           |    4 +
 tests/qemu-iotests/iotests.py                      |   42 +-
 tests/test-hbitmap.c                               |    2 +-
 ui/cocoa.m                                         |   12 +
 util/hbitmap.c                                     |   12 +
 83 files changed, 4663 insertions(+), 3514 deletions(-)
 create mode 100644 tests/acceptance/pc_cpu_hotplug_props.py
Quoted 9 lines
 create mode 100755 tests/qemu-iotests/265
 create mode 100644 tests/qemu-iotests/265.out
 create mode 100755 tests/qemu-iotests/266
 create mode 100644 tests/qemu-iotests/266.out
 create mode 100755 tests/qemu-iotests/267
 create mode 100644 tests/qemu-iotests/267.out
 create mode 100755 tests/qemu-iotests/270
 create mode 100644 tests/qemu-iotests/270.out

For openSUSE Factory/Tumbleweed v4.1.0 qemu, we also carry these:

commit d2da5e288a2e71e82866c8fdefd41b5727300124

Author: Kevin Wolf <[email protected]>
Date:   Mon Jul 22 17:44:27 2019 +0200

mirror: Keep mirror_top_bs drained after dropping permissions

commit d90d5cae2b10efc0e8d0b3cc91ff16201853d3ba

Author: Philippe Mathieu-Daudé <[email protected]>
Date:   Thu Sep 12 00:08:49 2019 +0200

block/create: Do not abort if a block driver is not available

commit 3fc4a64cbaed2ddee4c60ddc06740b320e18ab82

Author: Dr. David Alan Gilbert <[email protected]>
Date:   Wed Aug 14 18:55:35 2019 +0100

vhost: Fix memory region section comparison

along with this supporting patch:

commit 9366cf02e4e31c2a8128904d4d8290a0fad5f888

Author: Dr. David Alan Gilbert <[email protected]>
Date:   Wed Aug 14 18:55:34 2019 +0100

memory: Provide an equality function for MemoryRegionSections

- Bruce