Skip to contentSkip to conversation

virtio,acpi: features, fixes, cleanups.

GIT PULL Applied Michael S. Tsirkin 4 participants 54 messages Jul 4, 2020 – Jul 27, 2020

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

v2 of the pull, fixing non-Linux builds.

The following changes since commit fc1bff958998910ec8d25db86cd2f53ff125f7ab:

hw/misc/pca9552: Add missing TypeInfo::class_size field (2020-06-29 21:16:10 +0100)

are available in the Git repository at:

git://git.kernel.org/pub/scm/virt/kvm/mst/qemu.git tags/for_upstream

for you to fetch changes up to f7d98068bf7a0913744dd4ac0a28e8c841df2af5:

vhost-vdpa: introduce vhost-vdpa net client (2020-07-04 11:13:50 -0400)

----------------------------------------------------------------
virtio,acpi: features, fixes, cleanups.

vdpa support
virtio-mem support
a handy script for disassembling acpi tables
misc fixes and cleanups

----------------------------------------------------------------
Andrew Jones (1):
tests/acpi: remove stale allowed tables

Cindy Lu (11):
net: introduce qemu_get_peer
vhost_net: use the function qemu_get_peer
vhost: introduce new VhostOps vhost_dev_start
vhost: implement vhost_dev_start method
vhost: introduce new VhostOps vhost_vq_get_addr
vhost: implement vhost_vq_get_addr method
vhost: introduce new VhostOps vhost_force_iommu
vhost: implement vhost_force_iommu method
vhost_net: introduce set_config & get_config
vhost-vdpa: introduce vhost-vdpa backend
vhost-vdpa: introduce vhost-vdpa net client

David Hildenbrand (22):
virtio-balloon: always indicate S_DONE when migration fails
pc: Support coldplugging of virtio-pmem-pci devices on all buses
exec: Introduce ram_block_discard_(disable|require)()
vfio: Convert to ram_block_discard_disable() accel/kvm: Convert to ram_block_discard_disable() s390x/pv: Convert to ram_block_discard_disable()
virtio-balloon: Rip out qemu_balloon_inhibit() target/i386: sev: Use ram_block_discard_disable() migration/rdma: Use ram_block_discard_disable() migration/colo: Use ram_block_discard_disable()
virtio-mem: Paravirtualized memory hot(un)plug
virtio-pci: Proxy for virtio-mem
MAINTAINERS: Add myself as virtio-mem maintainer
hmp: Handle virtio-mem when printing memory device info
numa: Handle virtio-mem in NUMA stats
pc: Support for virtio-mem-pci
virtio-mem: Allow notifiers for size changes
virtio-pci: Send qapi events when the virtio-mem size changes
virtio-mem: Migration sanity checks
virtio-mem: Add trace events
virtio-mem: Exclude unplugged memory during migration
numa: Auto-enable NUMA when any memory devices are possible

Jason Wang (3):
virtio-bus: introduce queue_enabled method
virtio-pci: implement queue_enabled method
vhost: check the existence of vhost_set_iotlb_callback

Maxime Coquelin (1):
docs: vhost-user: add Virtio status protocol feature

Michael S. Tsirkin (2):
tests: disassemble-aml.sh: generate AML in readable format Revert "tests/migration: Reduce autoconverge initial bandwidth"

Peter Xu (1):
MAINTAINERS: add VT-d entry

 configure                                   |  21 +
 qapi/misc.json                              |  64 +-
 qapi/net.json                               |  28 +-
 hw/virtio/virtio-mem-pci.h                  |  34 ++
 include/exec/memory.h                       |  41 ++
 include/hw/boards.h                         |   1 +
 include/hw/pci/pci.h                        |   1 +
 include/hw/vfio/vfio-common.h               |   4 +-
 include/hw/virtio/vhost-backend.h           |  19 +-
 include/hw/virtio/vhost-vdpa.h              |  26 +
 include/hw/virtio/vhost.h                   |   7 +
 include/hw/virtio/virtio-bus.h              |   4 +
 include/hw/virtio/virtio-mem.h              |  86 +++
 include/migration/colo.h                    |   2 +-
 include/migration/misc.h                    |   2 +
 include/net/net.h                           |   1 +
 include/net/vhost-vdpa.h                    |  22 +
 include/net/vhost_net.h                     |   5 +
 include/sysemu/balloon.h                    |   2 -
 net/clients.h                               |   2 +
 tests/qtest/bios-tables-test-allowed-diff.h |  18 -
 accel/kvm/kvm-all.c                         |   4 +-
 balloon.c                                   |  17 -
 exec.c                                      |  52 ++
 hw/arm/virt.c                               |   2 +
 hw/core/numa.c                              |  17 +-
 hw/i386/microvm.c                           |   1 +
 hw/i386/pc.c                                |  66 ++-
 hw/i386/pc_piix.c                           |   1 +
 hw/i386/pc_q35.c                            |   1 +
 hw/net/vhost_net-stub.c                     |  11 +
 hw/net/vhost_net.c                          |  44 +-
 hw/net/virtio-net.c                         |  19 +
 hw/s390x/s390-virtio-ccw.c                  |  22 +-
 hw/vfio/ap.c                                |   8 +-
 hw/vfio/ccw.c                               |  11 +-
 hw/vfio/common.c                            |  53 +-
 hw/vfio/pci.c                               |   6 +-
 hw/virtio/vhost-backend.c                   |   6 +
 hw/virtio/vhost-vdpa.c                      | 475 +++++++++++++++
 hw/virtio/vhost.c                           |  52 +-
 hw/virtio/virtio-balloon.c                  |  36 +-
 hw/virtio/virtio-mem-pci.c                  | 157 +++++
 hw/virtio/virtio-mem.c                      | 873 ++++++++++++++++++++++++++++
 hw/virtio/virtio-pci.c                      |  13 +
 hw/virtio/virtio.c                          |   6 +
 migration/migration.c                       |  15 +-
 migration/postcopy-ram.c                    |  23 -
 migration/rdma.c                            |  18 +-
 migration/savevm.c                          |  11 +-
 monitor/hmp-cmds.c                          |  16 +
 monitor/monitor.c                           |   1 +
 net/net.c                                   |  10 +
 net/vhost-vdpa.c                            | 228 ++++++++
 target/i386/sev.c                           |   7 +
 tests/qtest/migration-test.c                |   2 +-
 MAINTAINERS                                 |  18 +
 docs/interop/index.rst                      |   1 +
 docs/interop/vhost-user.rst                 |  24 +
 docs/interop/vhost-vdpa.rst                 |  17 +
 hw/i386/Kconfig                             |   1 +
 hw/virtio/Kconfig                           |  11 +
 hw/virtio/Makefile.objs                     |   3 +
 hw/virtio/trace-events                      |  10 +
 net/Makefile.objs                           |   2 +-
 qemu-options.hx                             |  12 +
 tests/data/acpi/disassemle-aml.sh           |  52 ++
 tests/data/acpi/rebuild-expected-aml.sh     |   1 +
 68 files changed, 2639 insertions(+), 187 deletions(-)
 create mode 100644 hw/virtio/virtio-mem-pci.h
 create mode 100644 include/hw/virtio/vhost-vdpa.h
 create mode 100644 include/hw/virtio/virtio-mem.h
 create mode 100644 include/net/vhost-vdpa.h
 create mode 100644 hw/virtio/vhost-vdpa.c
 create mode 100644 hw/virtio/virtio-mem-pci.c
 create mode 100644 hw/virtio/virtio-mem.c
 create mode 100644 net/vhost-vdpa.c
 create mode 100644 docs/interop/vhost-vdpa.rst
 create mode 100755 tests/data/acpi/disassemle-aml.sh
#1 lore raw

[PULL v2 07/41] accel/kvm: Convert to ram_block_discard_disable()

From: David Hildenbrand <[email protected]>

Discarding memory does not work as expected. At the time this is called, we cannot have anyone active that relies on discards to work properly.

Message-Id: <[email protected]>

---
 accel/kvm/kvm-all.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
accel/kvm/kvm-all.c +2 -2
index d54a8701d8..ab36fbfa0c 100644
@@ -40,7 +40,6 @@
 #include "trace.h"
 #include "hw/irq.h"
 #include "sysemu/sev.h"
-#include "sysemu/balloon.h"
 #include "qapi/visitor.h"
 #include "qapi/qapi-types-common.h"
 #include "qapi/qapi-visit-common.h"
@@ -2229,7 +2228,8 @@ static int kvm_init(MachineState *ms)
 
     s->sync_mmu = !!kvm_vm_check_extension(kvm_state, KVM_CAP_SYNC_MMU);
     if (!s->sync_mmu) {
-        qemu_balloon_inhibit(true);
+        ret = ram_block_discard_disable(true);
+        assert(!ret);
     }
 
     return 0;
Signature · 1 line
MST
#2 lore raw

[PULL v2 03/41] virtio-balloon: always indicate S_DONE when migration fails

In reply to Michael S. Tsirkin

From: David Hildenbrand <[email protected]>

If something goes wrong during precopy, before stopping the VM, we will never send a S_DONE indication to the VM, resulting in the hinted pages not getting released to be used by the guest OS (e.g., Linux).

Easy to reproduce:
1. Start migration (e.g., HMP "migrate -d 'exec:gzip -c > STATEFILE.gz'")
2. Cancel migration (e.g., HMP "migrate_cancel")
3. Oberve in the guest (e.g., cat /proc/meminfo) that there is basically no free memory left.

While at it, add similar locking to virtio_balloon_free_page_done() as done in virtio_balloon_free_page_stop. Locking is still weird, but that has to be sorted out separately.

There is nothing to do in the PRECOPY_NOTIFY_COMPLETE case. Add some comments regarding S_DONE handling.

Fixes c13c4153f76d ("virtio-balloon: VIRTIO_BALLOON_F_FREE_PAGE_HINT")
Cc Wei Wang <[email protected]>
Cc Alexander Duyck <[email protected]>

Message-Id: <[email protected]>

---
 hw/virtio/virtio-balloon.c | 26 ++++++++++++++++++++------
 1 file changed, 20 insertions(+), 6 deletions(-)
hw/virtio/virtio-balloon.c +20 -6
index 10507b2a43..8a84718490 100644
@@ -628,8 +628,13 @@ static void virtio_balloon_free_page_done(VirtIOBalloon *s)
 {
     VirtIODevice *vdev = VIRTIO_DEVICE(s);
 
-    s->free_page_report_status = FREE_PAGE_REPORT_S_DONE;
-    virtio_notify_config(vdev);
+    if (s->free_page_report_status != FREE_PAGE_REPORT_S_DONE) {
+        /* See virtio_balloon_free_page_stop() */
+        qemu_mutex_lock(&s->free_page_lock);
+        s->free_page_report_status = FREE_PAGE_REPORT_S_DONE;
+        qemu_mutex_unlock(&s->free_page_lock);
+        virtio_notify_config(vdev);
+    }
 }
 
 static int
@@ -653,17 +658,26 @@ virtio_balloon_free_page_report_notify(NotifierWithReturn *n, void *data)
     case PRECOPY_NOTIFY_SETUP:
         precopy_enable_free_page_optimization();
         break;
-    case PRECOPY_NOTIFY_COMPLETE:
-    case PRECOPY_NOTIFY_CLEANUP:
     case PRECOPY_NOTIFY_BEFORE_BITMAP_SYNC:
         virtio_balloon_free_page_stop(dev);
         break;
     case PRECOPY_NOTIFY_AFTER_BITMAP_SYNC:
         if (vdev->vm_running) {
             virtio_balloon_free_page_start(dev);
-        } else {
-            virtio_balloon_free_page_done(dev);
+            break;
         }
+        /*
+         * Set S_DONE before migrating the vmstate, so the guest will reuse
+         * all hinted pages once running on the destination. Fall through.
+         */
+    case PRECOPY_NOTIFY_CLEANUP:
+        /*
+         * Especially, if something goes wrong during precopy or if migration
+         * is canceled, we have to properly communicate S_DONE to the VM.
+         */
+        virtio_balloon_free_page_done(dev);
+        break;
+    case PRECOPY_NOTIFY_COMPLETE:
         break;
     default:
         virtio_error(vdev, "%s: %d reason unknown", __func__, pnd->reason);
Signature · 1 line
MST
#3 lore raw

[PULL v2 01/41] tests: disassemble-aml.sh: generate AML in readable format

In reply to Michael S. Tsirkin

On systems where the IASL tool exists, we can convert extected ACPI tables to ASL format, which is useful
for debugging and documentation purposes.
This script does this for all ACPI tables under tests/data/acpi/.

---
 tests/data/acpi/disassemle-aml.sh       | 52 +++++++++++++++++++++++++
 tests/data/acpi/rebuild-expected-aml.sh |  1 +
 2 files changed, 53 insertions(+)
 create mode 100755 tests/data/acpi/disassemle-aml.sh
tests/data/acpi/disassemle-aml.sh new file +52 -0
new file mode 100755
index 0000000000..1d8a4d0301
@@ -0,0 +1,52 @@
+#!/usr/bin/bash
+
+outdir=
+while getopts "o:" arg; do
+  case ${arg} in
+    o )
+        outdir=$OPTARG
+        ;;
+    \? )
+        echo "Usage: ./tests/data/acpi/disassemle-aml.sh [-o <output-directory>]"
+        exit 1
+        ;;
+
+  esac
+done
+
+for machine in tests/data/acpi/*
+do
+    if [[ ! -d "$machine" ]];
+    then
+        continue
+    fi
+
+    if [[ "${outdir}" ]];
+    then
+        mkdir -p "${outdir}"/${machine} || exit $?
+    fi
+    for aml in $machine/*
+    do
+        if [[ "$aml" == $machine/*.dsl ]];
+        then
+            continue
+        fi
+        if [[ "$aml" == $machine/SSDT*.* ]];
+        then
+            dsdt=${aml/SSDT*./DSDT.}
+            extra="-e ${dsdt}"
+        elif [[ "$aml" == $machine/SSDT* ]];
+        then
+            dsdt=${aml/SSDT*/DSDT};
+            extra="-e ${dsdt}"
+        else
+            extra=""
+        fi
+        asl=${aml}.dsl
+        if [[ "${outdir}" ]];
+        then
+            asl="${outdir}"/${machine}/${asl}
+        fi
+        iasl -d -p ${asl} ${extra} ${aml}
+    done
+done
tests/data/acpi/rebuild-expected-aml.sh +1 -0
index 9cbaab1a4d..76cd797d1e 100755
@@ -36,6 +36,7 @@ old_allowed_dif=`grep -v -e 'List of comma-separated changed AML files to ignore
 echo '/* List of comma-separated changed AML files to ignore */' > ${SRC_PATH}/tests/qtest/bios-tables-test-allowed-diff.h
 
 echo "The files were rebuilt and can be added to git."
+echo "You can use ${SRC_PATH}/tests/data/acpi/disassemle-aml.sh to disassemble them to ASL."
 
 if [ -z "$old_allowed_dif" ]; then
     echo "Note! Please do not commit expected files with source changes"
Signature · 1 line
MST
#4 lore raw

[PULL v2 11/41] migration/rdma: Use ram_block_discard_disable()

In reply to Michael S. Tsirkin

From: David Hildenbrand <[email protected]>

RDMA will pin all guest memory (as documented in docs/rdma.txt). We want to disable RAM block discards - however, to keep it simple use ram_block_discard_is_required() instead of inhibiting.

Note: It is not sufficient to limit disabling to pin_all. Even when only conditionally pinning 1 MB chunks, as soon as one page within such a chunk was discarded and one page not, the discarded pages will be pinned as well.

Cc "Michael S. Tsirkin" <[email protected]>
Cc Juan Quintela <[email protected]>
Cc "Dr. David Alan Gilbert" <[email protected]>

Message-Id: <[email protected]>

---
 migration/rdma.c | 18 ++++++++++++++++--
 1 file changed, 16 insertions(+), 2 deletions(-)
migration/rdma.c +16 -2
index ec45d33ba3..bbe6f36627 100644
@@ -29,6 +29,7 @@
 #include "qemu/sockets.h"
 #include "qemu/bitmap.h"
 #include "qemu/coroutine.h"
+#include "exec/memory.h"
 #include <sys/socket.h>
 #include <netdb.h>
 #include <arpa/inet.h>
@@ -4017,8 +4018,14 @@ void rdma_start_incoming_migration(const char *host_port, Error **errp)
     Error *local_err = NULL;
 
     trace_rdma_start_incoming_migration();
-    rdma = qemu_rdma_data_init(host_port, &local_err);
 
+    /* Avoid ram_block_discard_disable(), cannot change during migration. */
+    if (ram_block_discard_is_required()) {
+        error_setg(errp, "RDMA: cannot disable RAM discard");
+        return;
+    }
+
+    rdma = qemu_rdma_data_init(host_port, &local_err);
     if (rdma == NULL) {
         goto err;
     }
@@ -4067,10 +4074,17 @@ void rdma_start_outgoing_migration(void *opaque,
                             const char *host_port, Error **errp)
 {
     MigrationState *s = opaque;
-    RDMAContext *rdma = qemu_rdma_data_init(host_port, errp);
     RDMAContext *rdma_return_path = NULL;
+    RDMAContext *rdma;
     int ret = 0;
 
+    /* Avoid ram_block_discard_disable(), cannot change during migration. */
+    if (ram_block_discard_is_required()) {
+        error_setg(errp, "RDMA: cannot disable RAM discard");
+        return;
+    }
+
+    rdma = qemu_rdma_data_init(host_port, errp);
     if (rdma == NULL) {
         goto err;
     }
Signature · 1 line
MST
#5 lore raw

[PULL v2 04/41] pc: Support coldplugging of virtio-pmem-pci devices on all buses

In reply to Michael S. Tsirkin

From: David Hildenbrand <[email protected]>

E.g., with "pc-q35-4.2", trying to coldplug a virtio-pmem-pci devices results in
"virtio-pmem-pci not supported on this bus"

Reasons is, that the bus does not support hotplug and, therefore, does not have a hotplug handler. Let's allow coldplugging virtio-pmem devices on such buses. The hotplug order is only relevant for virtio-pmem-pci when the guest is already alive and the device is visible before memory_device_plug() wired up the memory device bits.

Hotplug attempts will still fail with:
"Error: Bus 'pcie.0' does not support hotplugging"

Hotunplug attempts will still fail with:
"Error: Bus 'pcie.0' does not support hotplugging"

Cc Pankaj Gupta <[email protected]>
Cc Igor Mammedov <[email protected]>
Cc Paolo Bonzini <[email protected]>
Cc Richard Henderson <[email protected]>
Cc Eduardo Habkost <[email protected]>
Cc "Michael S. Tsirkin" <[email protected]>
Cc Marcel Apfelbaum <[email protected]>

Message-Id: <[email protected]>

---
 hw/i386/pc.c | 18 ++++++++++--------
 1 file changed, 10 insertions(+), 8 deletions(-)
hw/i386/pc.c +10 -8
index 4af9679d03..58b1425c17 100644
@@ -1643,13 +1643,13 @@ static void pc_virtio_pmem_pci_pre_plug(HotplugHandler *hotplug_dev,
     HotplugHandler *hotplug_dev2 = qdev_get_bus_hotplug_handler(dev);
     Error *local_err = NULL;
 
-    if (!hotplug_dev2) {
+    if (!hotplug_dev2 && dev->hotplugged) {
         /*
          * Without a bus hotplug handler, we cannot control the plug/unplug
-         * order. This should never be the case on x86, however better add
-         * a safety net.
+         * order. We should never reach this point when hotplugging on x86,
+         * however, better add a safety net.
          */
-        error_setg(errp, "virtio-pmem-pci not supported on this bus.");
+        error_setg(errp, "virtio-pmem-pci hotplug not supported on this bus.");
         return;
     }
     /*
@@ -1658,7 +1658,7 @@ static void pc_virtio_pmem_pci_pre_plug(HotplugHandler *hotplug_dev,
      */
     memory_device_pre_plug(MEMORY_DEVICE(dev), MACHINE(hotplug_dev), NULL,
                            &local_err);
-    if (!local_err) {
+    if (!local_err && hotplug_dev2) {
         hotplug_handler_pre_plug(hotplug_dev2, dev, &local_err);
     }
     error_propagate(errp, local_err);
@@ -1676,9 +1676,11 @@ static void pc_virtio_pmem_pci_plug(HotplugHandler *hotplug_dev,
      * device bits.
      */
     memory_device_plug(MEMORY_DEVICE(dev), MACHINE(hotplug_dev));
-    hotplug_handler_plug(hotplug_dev2, dev, &local_err);
-    if (local_err) {
-        memory_device_unplug(MEMORY_DEVICE(dev), MACHINE(hotplug_dev));
+    if (hotplug_dev2) {
+        hotplug_handler_plug(hotplug_dev2, dev, &local_err);
+        if (local_err) {
+            memory_device_unplug(MEMORY_DEVICE(dev), MACHINE(hotplug_dev));
+        }
     }
     error_propagate(errp, local_err);
 }
Signature · 1 line
MST
#6 lore raw

[PULL v2 02/41] Revert "tests/migration: Reduce autoconverge initial bandwidth"

In reply to Michael S. Tsirkin

This reverts commit 6d1da867e65f ("tests/migration: Reduce autoconverge initial bandwidth") since that change makes unit tests much slower for all developers, while it's not a robust way to fix migration tests. Migration tests need to find
a more robust way to discover a reasonable bandwidth without slowing
things down for everyone.

Fixes 6d1da867e65f ("tests/migration: Reduce autoconverge initial bandwidth")
---
 tests/qtest/migration-test.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
tests/qtest/migration-test.c +1 -1
index dc3490c9fa..21ea5ba1d2 100644
@@ -1211,7 +1211,7 @@ static void test_migrate_auto_converge(void)
      * without throttling.
      */
     migrate_set_parameter_int(from, "downtime-limit", 1);
-    migrate_set_parameter_int(from, "max-bandwidth", 1000000); /* ~1Mb/s */
+    migrate_set_parameter_int(from, "max-bandwidth", 100000000); /* ~100Mb/s */
 
     /* To check remaining size after precopy */
     migrate_set_capability(from, "pause-before-switchover", true);
Signature · 1 line
MST
#7 lore raw

[PULL v2 13/41] virtio-mem: Paravirtualized memory hot(un)plug

In reply to Michael S. Tsirkin

From: David Hildenbrand <[email protected]>

This is the very basic/initial version of virtio-mem. An introduction to virtio-mem can be found in the Linux kernel driver [1]. While it can be used in the current state for hotplug of a smaller amount of memory, it will heavily benefit from resizeable memory regions in the future.

Each virtio-mem device manages a memory region (provided via a memory backend). After requested by the hypervisor ("requested-size"), the guest can try to plug/unplug blocks of memory within that region, in order to reach the requested size. Initially, and after a reboot, all memory is unplugged (except in special cases - reboot during postcopy).

The guest may only try to plug/unplug blocks of memory within the usable region size. The usable region size is a little bigger than the requested size, to give the device driver some flexibility. The usable region size will only grow, except on reboots or when all memory is requested to get unplugged. The guest can never plug more memory than requested. Unplugged memory will get zapped/discarded, similar to in a balloon device.

The block size is variable, however, it is always chosen in a way such that THP splits are avoided (e.g., 2MB). The state of each block (plugged/unplugged) is tracked in a bitmap.

As virtio-mem devices (e.g., virtio-mem-pci) will be memory devices, we now expose "VirtioMEMDeviceInfo" via "query-memory-devices".

--------------------------------------------------------------------------

There are two important follow-up items that are in the works:
1. Resizeable memory regions: Use resizeable allocations/RAM blocks to grow/shrink along with the usable region size. This avoids creating initially very big VMAs, RAM blocks, and KVM slots.
2. Protection of unplugged memory: Make sure the gust cannot actually make use of unplugged memory.

Other follow-up items that are in the works:
1. Exclude unplugged memory during migration (via precopy notifier).
2. Handle remapping of memory.
3. Support for other architectures.

--------------------------------------------------------------------------

Example usage (virtio-mem-pci is introduced in follow-up patches):

Start QEMU with two virtio-mem devices (one per NUMA node):

 $ qemu-system-x86_64 -m 4G,maxmem=20G \
  -smp sockets=2,cores=2 \
  -numa node,nodeid=0,cpus=0-1 -numa node,nodeid=1,cpus=2-3 \
  [...]
  -object memory-backend-ram,id=mem0,size=8G \
  -device virtio-mem-pci,id=vm0,memdev=mem0,node=0,requested-size=0M \
  -object memory-backend-ram,id=mem1,size=8G \
  -device virtio-mem-pci,id=vm1,memdev=mem1,node=1,requested-size=1G

Query the configuration:
(qemu) info memory-devices
Memory device [virtio-mem]: "vm0"
memaddr: 0x140000000
node: 0
requested-size: 0
size: 0
max-size: 8589934592
block-size: 2097152
memdev: /objects/mem0
Memory device [virtio-mem]: "vm1"
memaddr: 0x340000000
node: 1
requested-size: 1073741824
size: 1073741824
max-size: 8589934592
block-size: 2097152
memdev: /objects/mem1

Add some memory to node 0:
(qemu) qom-set vm0 requested-size 500M

Remove some memory from node 1:
(qemu) qom-set vm1 requested-size 200M

Query the configuration again:
(qemu) info memory-devices
Memory device [virtio-mem]: "vm0"
memaddr: 0x140000000
node: 0
requested-size: 524288000
size: 524288000
max-size: 8589934592
block-size: 2097152
memdev: /objects/mem0
Memory device [virtio-mem]: "vm1"
memaddr: 0x340000000
node: 1
requested-size: 209715200
size: 209715200
max-size: 8589934592
block-size: 2097152
memdev: /objects/mem1

[1] https://lkml.kernel.org/r/[email protected]

Cc "Michael S. Tsirkin" <[email protected]>
Cc Eric Blake <[email protected]>
Cc Markus Armbruster <[email protected]>
Cc "Dr. David Alan Gilbert" <[email protected]>
Cc Igor Mammedov <[email protected]>

Message-Id: <[email protected]>

---
 qapi/misc.json                 |  39 +-
 include/hw/virtio/virtio-mem.h |  78 ++++
 hw/virtio/virtio-mem.c         | 724 +++++++++++++++++++++++++++++++++
 hw/virtio/Kconfig              |  11 +
 hw/virtio/Makefile.objs        |   1 +
 5 files changed, 852 insertions(+), 1 deletion(-)
 create mode 100644 include/hw/virtio/virtio-mem.h
 create mode 100644 hw/virtio/virtio-mem.c
qapi/misc.json +38 -1
index a5a0beb902..65ca3edf32 100644
@@ -1356,19 +1356,56 @@
           }
 }
 
+##
+# @VirtioMEMDeviceInfo:
+#
+# VirtioMEMDevice state information
+#
+# @id: device's ID
+#
+# @memaddr: physical address in memory, where device is mapped
+#
+# @requested-size: the user requested size of the device
+#
+# @size: the (current) size of memory that the device provides
+#
+# @max-size: the maximum size of memory that the device can provide
+#
+# @block-size: the block size of memory that the device provides
+#
+# @node: NUMA node number where device is assigned to
+#
+# @memdev: memory backend linked with the region
+#
+# Since: 5.1
+##
+{ 'struct': 'VirtioMEMDeviceInfo',
+  'data': { '*id': 'str',
+            'memaddr': 'size',
+            'requested-size': 'size',
+            'size': 'size',
+            'max-size': 'size',
+            'block-size': 'size',
+            'node': 'int',
+            'memdev': 'str'
+          }
+}
+
 ##
 # @MemoryDeviceInfo:
 #
 # Union containing information about a memory device
 #
 # nvdimm is included since 2.12. virtio-pmem is included since 4.1.
+# virtio-mem is included since 5.1.
 #
 # Since: 2.1
 ##
 { 'union': 'MemoryDeviceInfo',
   'data': { 'dimm': 'PCDIMMDeviceInfo',
             'nvdimm': 'PCDIMMDeviceInfo',
-            'virtio-pmem': 'VirtioPMEMDeviceInfo'
+            'virtio-pmem': 'VirtioPMEMDeviceInfo',
+            'virtio-mem': 'VirtioMEMDeviceInfo'
           }
 }
 
include/hw/virtio/virtio-mem.h new file +78 -0

78 lines not on this page · raw message

hw/virtio/virtio-mem.c new file +724 -0

724 lines not on this page · raw message

hw/virtio/Kconfig +11 -0
index 83122424fa..0eda25c4e1 100644
@@ -47,3 +47,14 @@ config VIRTIO_PMEM
     depends on VIRTIO
     depends on VIRTIO_PMEM_SUPPORTED
     select MEM_DEVICE
+
+config VIRTIO_MEM_SUPPORTED
+    bool
+
+config VIRTIO_MEM
+    bool
+    default y
+    depends on VIRTIO
+    depends on LINUX
+    depends on VIRTIO_MEM_SUPPORTED
+    select MEM_DEVICE
hw/virtio/Makefile.objs +1 -0
index 13e75f171f..f3a65e01b7 100644
@@ -19,6 +19,7 @@ obj-$(call land,$(CONFIG_VHOST_USER_FS),$(CONFIG_VIRTIO_PCI)) += vhost-user-fs-p
 obj-$(CONFIG_VIRTIO_IOMMU) += virtio-iommu.o
 obj-$(CONFIG_VHOST_VSOCK) += vhost-vsock-common.o vhost-vsock.o
 obj-$(CONFIG_VHOST_USER_VSOCK) += vhost-vsock-common.o vhost-user-vsock.o
+obj-$(CONFIG_VIRTIO_MEM) += virtio-mem.o
 
 ifeq ($(CONFIG_VIRTIO_PCI),y)
 obj-$(CONFIG_VHOST_VSOCK) += vhost-vsock-pci.o
Signature · 1 line
MST
#8 lore raw

[PULL v2 08/41] s390x/pv: Convert to ram_block_discard_disable()

In reply to Michael S. Tsirkin

From: David Hildenbrand <[email protected]>

Discarding RAM does not work as expected with protected VMs. Let's switch to ram_block_discard_disable() for now, as we want to get rid of qemu_balloon_inhibit(). Note that it will currently never fail, but might fail in the future with new technologies (e.g., virtio-mem).

Cc Richard Henderson <[email protected]>
Cc Cornelia Huck <[email protected]>
Cc Halil Pasic <[email protected]>
Cc Christian Borntraeger <[email protected]>
Cc Janosch Frank <[email protected]>

Message-Id: <[email protected]>

---
 hw/s390x/s390-virtio-ccw.c | 22 +++++++++++++---------
 1 file changed, 13 insertions(+), 9 deletions(-)
hw/s390x/s390-virtio-ccw.c +13 -9
index b111406d56..023fd25f2b 100644
@@ -43,7 +43,6 @@
 #include "hw/qdev-properties.h"
 #include "hw/s390x/tod.h"
 #include "sysemu/sysemu.h"
-#include "sysemu/balloon.h"
 #include "hw/s390x/pv.h"
 #include "migration/blocker.h"
 
@@ -329,7 +328,7 @@ static void s390_machine_unprotect(S390CcwMachineState *ms)
     ms->pv = false;
     migrate_del_blocker(pv_mig_blocker);
     error_free_or_abort(&pv_mig_blocker);
-    qemu_balloon_inhibit(false);
+    ram_block_discard_disable(false);
 }
 
 static int s390_machine_protect(S390CcwMachineState *ms)
@@ -338,17 +337,22 @@ static int s390_machine_protect(S390CcwMachineState *ms)
     int rc;
 
    /*
-    * Ballooning on protected VMs needs support in the guest for
-    * sharing and unsharing balloon pages. Block ballooning for
-    * now, until we have a solution to make at least Linux guests
-    * either support it or fail gracefully.
+    * Discarding of memory in RAM blocks does not work as expected with
+    * protected VMs. Sharing and unsharing pages would be required. Disable
+    * it for now, until until we have a solution to make at least Linux
+    * guests either support it (e.g., virtio-balloon) or fail gracefully.
     */
-    qemu_balloon_inhibit(true);
+    rc = ram_block_discard_disable(true);
+    if (rc) {
+        error_report("protected VMs: cannot disable RAM discard");
+        return rc;
+    }
+
     error_setg(&pv_mig_blocker,
                "protected VMs are currently not migrateable.");
     rc = migrate_add_blocker(pv_mig_blocker, &local_err);
     if (rc) {
-        qemu_balloon_inhibit(false);
+        ram_block_discard_disable(false);
         error_report_err(local_err);
         error_free_or_abort(&pv_mig_blocker);
         return rc;
@@ -357,7 +361,7 @@ static int s390_machine_protect(S390CcwMachineState *ms)
     /* Create SE VM */
     rc = s390_pv_vm_enable();
     if (rc) {
-        qemu_balloon_inhibit(false);
+        ram_block_discard_disable(false);
         migrate_del_blocker(pv_mig_blocker);
         error_free_or_abort(&pv_mig_blocker);
         return rc;
Signature · 1 line
MST
#9 lore raw

[PULL v2 24/41] numa: Auto-enable NUMA when any memory devices are possible

In reply to Michael S. Tsirkin

From: David Hildenbrand <[email protected]>

Let's auto-enable it also when maxmem is specified but no slots are defined. This will result in us properly creating ACPI srat tables, indicating the maximum possible PFN to the guest OS. Based on this, e.g., Linux will enable the swiotlb properly.

This avoids having to manually force the switolb on (swiotlb=force) in Linux in case we're booting only using DMA memory (e.g., 2GB on x86-64), and virtio-mem adds memory later on that really needs the swiotlb to be used for DMA.

Let's take care of backwards compatibility if somebody has a setup that specifies "maxram" without "slots".

Cc Peter Maydell <[email protected]>
Cc Eduardo Habkost <[email protected]>
Cc Marcel Apfelbaum <[email protected]>
Cc Sergio Lopez <[email protected]>
Cc Paolo Bonzini <[email protected]>
Cc Richard Henderson <[email protected]>
Cc "Michael S. Tsirkin" <[email protected]>
Cc Igor Mammedov <[email protected]>

Message-Id: <[email protected]>

---
 include/hw/boards.h |  1 +
 hw/arm/virt.c       |  2 ++
 hw/core/numa.c      | 11 ++++++-----
 hw/i386/microvm.c   |  1 +
 hw/i386/pc.c        |  1 +
 hw/i386/pc_piix.c   |  1 +
 hw/i386/pc_q35.c    |  1 +
 7 files changed, 13 insertions(+), 5 deletions(-)
include/hw/boards.h +1 -0
index 18815d9be2..426ce5f625 100644
@@ -207,6 +207,7 @@ struct MachineClass {
     const char **valid_cpu_types;
     strList *allowed_dynamic_sysbus_devices;
     bool auto_enable_numa_with_memhp;
+    bool auto_enable_numa_with_memdev;
     void (*numa_auto_assign_ram)(MachineClass *mc, NodeInfo *nodes,
                                  int nb_nodes, ram_addr_t size);
     bool ignore_boot_device_suffixes;
hw/arm/virt.c +2 -0
index cd0834ce7f..f97be80a86 100644
@@ -2373,6 +2373,7 @@ static void virt_machine_class_init(ObjectClass *oc, void *data)
     hc->unplug = virt_machine_device_unplug_cb;
     mc->nvdimm_supported = true;
     mc->auto_enable_numa_with_memhp = true;
+    mc->auto_enable_numa_with_memdev = true;
     mc->default_ram_id = "mach-virt.ram";
 
     object_class_property_add(oc, "acpi", "OnOffAuto",
@@ -2485,6 +2486,7 @@ static void virt_machine_5_0_options(MachineClass *mc)
     virt_machine_5_1_options(mc);
     compat_props_add(mc->compat_props, hw_compat_5_0, hw_compat_5_0_len);
     mc->numa_mem_supported = true;
+    mc->auto_enable_numa_with_memdev = false;
 }
 DEFINE_VIRT_MACHINE(5, 0)
 
hw/core/numa.c +6 -5
index e9aec69afd..6a20ce7cf1 100644
@@ -688,8 +688,9 @@ void numa_complete_configuration(MachineState *ms)
     NodeInfo *numa_info = ms->numa_state->nodes;
 
     /*
-     * If memory hotplug is enabled (slots > 0) but without '-numa'
-     * options explicitly on CLI, guestes will break.
+     * If memory hotplug is enabled (slot > 0) or memory devices are enabled
+     * (ms->maxram_size > ram_size) but without '-numa' options explicitly on
+     * CLI, guests will break.
      *
      *   Windows: won't enable memory hotplug without SRAT table at all
      *
@@ -704,9 +705,9 @@ void numa_complete_configuration(MachineState *ms)
      * assume there is just one node with whole RAM.
      */
     if (ms->numa_state->num_nodes == 0 &&
-        ((ms->ram_slots > 0 &&
-        mc->auto_enable_numa_with_memhp) ||
-        mc->auto_enable_numa)) {
+        ((ms->ram_slots && mc->auto_enable_numa_with_memhp) ||
+         (ms->maxram_size > ms->ram_size && mc->auto_enable_numa_with_memdev) ||
+         mc->auto_enable_numa)) {
             NumaNodeOptions node = { };
             parse_numa_node(ms, &node, &error_abort);
             numa_info[0].node_mem = ram_size;
hw/i386/microvm.c +1 -0
index 5e931975a0..81d0888930 100644
@@ -464,6 +464,7 @@ static void microvm_class_init(ObjectClass *oc, void *data)
     mc->max_cpus = 288;
     mc->has_hotpluggable_cpus = false;
     mc->auto_enable_numa_with_memhp = false;
+    mc->auto_enable_numa_with_memdev = false;
     mc->default_cpu_type = TARGET_DEFAULT_CPU_TYPE;
     mc->nvdimm_supported = false;
     mc->default_ram_id = "microvm.ram";
hw/i386/pc.c +1 -0
index 576f2502f9..61acc9e530 100644
@@ -1975,6 +1975,7 @@ static void pc_machine_class_init(ObjectClass *oc, void *data)
     mc->get_default_cpu_node_id = x86_get_default_cpu_node_id;
     mc->possible_cpu_arch_ids = x86_possible_cpu_arch_ids;
     mc->auto_enable_numa_with_memhp = true;
+    mc->auto_enable_numa_with_memdev = true;
     mc->has_hotpluggable_cpus = true;
     mc->default_boot_order = "cad";
     mc->hot_add_cpu = pc_hot_add_cpu;
hw/i386/pc_piix.c +1 -0
index 1d832b2878..fae487f57d 100644
@@ -444,6 +444,7 @@ static void pc_i440fx_5_0_machine_options(MachineClass *m)
     m->numa_mem_supported = true;
     compat_props_add(m->compat_props, hw_compat_5_0, hw_compat_5_0_len);
     compat_props_add(m->compat_props, pc_compat_5_0, pc_compat_5_0_len);
+    m->auto_enable_numa_with_memdev = false;
 }
 
 DEFINE_I440FX_MACHINE(v5_0, "pc-i440fx-5.0", NULL,
hw/i386/pc_q35.c +1 -0
index 047ea8db28..acd6d405f0 100644
@@ -372,6 +372,7 @@ static void pc_q35_5_0_machine_options(MachineClass *m)
     m->numa_mem_supported = true;
     compat_props_add(m->compat_props, hw_compat_5_0, hw_compat_5_0_len);
     compat_props_add(m->compat_props, pc_compat_5_0, pc_compat_5_0_len);
+    m->auto_enable_numa_with_memhp = false;
 }
 
 DEFINE_Q35_MACHINE(v5_0, "pc-q35-5.0", NULL,
Signature · 1 line
MST
#10 lore raw

[PULL v2 05/41] exec: Introduce ram_block_discard_(disable|require)()

In reply to Michael S. Tsirkin

From: David Hildenbrand <[email protected]>

We want to replace qemu_balloon_inhibit() by something more generic. Especially, we want to make sure that technologies that really rely on RAM block discards to work reliably to run mutual exclusive with technologies that effectively break it.

E.g., vfio will usually pin all guest memory, turning the virtio-balloon basically useless and make the VM consume more memory than reported via the balloon. While the balloon is special already (=> no guarantees, same behavior possible afer reboots and with huge pages), this will be different, especially, with virtio-mem.

Let's implement a way such that we can make both types of technology run mutually exclusive. We'll convert existing balloon inhibitors in successive patches and add some new ones. Add the check to qemu_balloon_is_inhibited() for now. We might want to make virtio-balloon an acutal inhibitor in the future - however, that requires more thought to not break existing setups.

Cc "Michael S. Tsirkin" <[email protected]>
Cc Richard Henderson <[email protected]>
Cc Paolo Bonzini <[email protected]>

Message-Id: <[email protected]>

---
 include/exec/memory.h | 41 ++++++++++++++++++++++++++++++++++
 balloon.c             |  3 ++-
 exec.c                | 52 +++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 95 insertions(+), 1 deletion(-)
include/exec/memory.h +41 -0
index 7207025bd4..38ec38b9a8 100644
@@ -2472,6 +2472,47 @@ static inline MemOp devend_memop(enum device_endian end)
 }
 #endif
 
+/*
+ * Inhibit technologies that require discarding of pages in RAM blocks, e.g.,
+ * to manage the actual amount of memory consumed by the VM (then, the memory
+ * provided by RAM blocks might be bigger than the desired memory consumption).
+ * This *must* be set if:
+ * - Discarding parts of a RAM blocks does not result in the change being
+ *   reflected in the VM and the pages getting freed.
+ * - All memory in RAM blocks is pinned or duplicated, invaldiating any previous
+ *   discards blindly.
+ * - Discarding parts of a RAM blocks will result in integrity issues (e.g.,
+ *   encrypted VMs).
+ * Technologies that only temporarily pin the current working set of a
+ * driver are fine, because we don't expect such pages to be discarded
+ * (esp. based on guest action like balloon inflation).
+ *
+ * This is *not* to be used to protect from concurrent discards (esp.,
+ * postcopy).
+ *
+ * Returns 0 if successful. Returns -EBUSY if a technology that relies on
+ * discards to work reliably is active.
+ */
+int ram_block_discard_disable(bool state);
+
+/*
+ * Inhibit technologies that disable discarding of pages in RAM blocks.
+ *
+ * Returns 0 if successful. Returns -EBUSY if discards are already set to
+ * broken.
+ */
+int ram_block_discard_require(bool state);
+
+/*
+ * Test if discarding of memory in ram blocks is disabled.
+ */
+bool ram_block_discard_is_disabled(void);
+
+/*
+ * Test if discarding of memory in ram blocks is required to work reliably.
+ */
+bool ram_block_discard_is_required(void);
+
 #endif
 
 #endif
balloon.c +2 -1
index f104b42961..5fff79523a 100644
@@ -40,7 +40,8 @@ static int balloon_inhibit_count;
 
 bool qemu_balloon_is_inhibited(void)
 {
-    return atomic_read(&balloon_inhibit_count) > 0;
+    return atomic_read(&balloon_inhibit_count) > 0 ||
+           ram_block_discard_is_disabled();
 }
 
 void qemu_balloon_inhibit(bool state)
exec.c +52 -0

56 lines not on this page · raw message

Signature · 1 line
MST
#11 lore raw

[PULL v2 14/41] virtio-pci: Proxy for virtio-mem

In reply to Michael S. Tsirkin

From: David Hildenbrand <[email protected]>

Let's add a proxy for virtio-mem, make it a memory device, and pass-through the properties.

Cc "Michael S. Tsirkin" <[email protected]>
Cc Marcel Apfelbaum <[email protected]>
Cc "Dr. David Alan Gilbert" <[email protected]>
Cc Igor Mammedov <[email protected]>

Message-Id: <[email protected]>

---
 hw/virtio/virtio-mem-pci.h |  33 ++++++++++
 include/hw/pci/pci.h       |   1 +
 hw/virtio/virtio-mem-pci.c | 129 +++++++++++++++++++++++++++++++++++++
 hw/virtio/Makefile.objs    |   1 +
 4 files changed, 164 insertions(+)
 create mode 100644 hw/virtio/virtio-mem-pci.h
 create mode 100644 hw/virtio/virtio-mem-pci.c
hw/virtio/virtio-mem-pci.h new file +33 -0
new file mode 100644
index 0000000000..8820cd6628
@@ -0,0 +1,33 @@
+/*
+ * Virtio MEM PCI device
+ *
+ * Copyright (C) 2020 Red Hat, Inc.
+ *
+ * Authors:
+ *  David Hildenbrand <[email protected]>
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2.
+ * See the COPYING file in the top-level directory.
+ */
+
+#ifndef QEMU_VIRTIO_MEM_PCI_H
+#define QEMU_VIRTIO_MEM_PCI_H
+
+#include "hw/virtio/virtio-pci.h"
+#include "hw/virtio/virtio-mem.h"
+
+typedef struct VirtIOMEMPCI VirtIOMEMPCI;
+
+/*
+ * virtio-mem-pci: This extends VirtioPCIProxy.
+ */
+#define TYPE_VIRTIO_MEM_PCI "virtio-mem-pci-base"
+#define VIRTIO_MEM_PCI(obj) \
+        OBJECT_CHECK(VirtIOMEMPCI, (obj), TYPE_VIRTIO_MEM_PCI)
+
+struct VirtIOMEMPCI {
+    VirtIOPCIProxy parent_obj;
+    VirtIOMEM vdev;
+};
+
+#endif /* QEMU_VIRTIO_MEM_PCI_H */
include/hw/pci/pci.h +1 -0
index a4e9c33416..c1bf7d5356 100644
@@ -87,6 +87,7 @@ extern bool pci_available;
 #define PCI_DEVICE_ID_VIRTIO_VSOCK       0x1012
 #define PCI_DEVICE_ID_VIRTIO_PMEM        0x1013
 #define PCI_DEVICE_ID_VIRTIO_IOMMU       0x1014
+#define PCI_DEVICE_ID_VIRTIO_MEM         0x1015
 
 #define PCI_VENDOR_ID_REDHAT             0x1b36
 #define PCI_DEVICE_ID_REDHAT_BRIDGE      0x0001
hw/virtio/virtio-mem-pci.c new file +129 -0

129 lines not on this page · raw message

hw/virtio/Makefile.objs +1 -0
index f3a65e01b7..a986708186 100644
@@ -20,6 +20,7 @@ obj-$(CONFIG_VIRTIO_IOMMU) += virtio-iommu.o
 obj-$(CONFIG_VHOST_VSOCK) += vhost-vsock-common.o vhost-vsock.o
 obj-$(CONFIG_VHOST_USER_VSOCK) += vhost-vsock-common.o vhost-user-vsock.o
 obj-$(CONFIG_VIRTIO_MEM) += virtio-mem.o
+common-obj-$(call land,$(CONFIG_VIRTIO_MEM),$(CONFIG_VIRTIO_PCI)) += virtio-mem-pci.o
 
 ifeq ($(CONFIG_VIRTIO_PCI),y)
 obj-$(CONFIG_VHOST_VSOCK) += vhost-vsock-pci.o
Signature · 1 line
MST
#12 lore raw

[PULL v2 10/41] target/i386: sev: Use ram_block_discard_disable()

In reply to Michael S. Tsirkin

From: David Hildenbrand <[email protected]>

AMD SEV will pin all guest memory, mark discarding of RAM broken. At the time this is called, we cannot have anyone active that relies on discards to work properly - let's still implement error handling.

Cc "Michael S. Tsirkin" <[email protected]>
Cc Paolo Bonzini <[email protected]>
Cc Richard Henderson <[email protected]>
Cc Eduardo Habkost <[email protected]>

Message-Id: <[email protected]>

---
 target/i386/sev.c | 7 +++++++
 1 file changed, 7 insertions(+)
target/i386/sev.c +7 -0
index d273174ad3..f100a53231 100644
@@ -680,6 +680,12 @@ sev_guest_init(const char *id)
     uint32_t host_cbitpos;
     struct sev_user_data_status status = {};
 
+    ret = ram_block_discard_disable(true);
+    if (ret) {
+        error_report("%s: cannot disable RAM discard", __func__);
+        return NULL;
+    }
+
     sev = lookup_sev_guest_info(id);
     if (!sev) {
         error_report("%s: '%s' is not a valid '%s' object",
@@ -751,6 +757,7 @@ sev_guest_init(const char *id)
     return sev;
 err:
     sev_guest = NULL;
+    ram_block_discard_disable(false);
     return NULL;
 }
 
Signature · 1 line
MST
#13 lore raw

[PULL v2 06/41] vfio: Convert to ram_block_discard_disable()

In reply to Michael S. Tsirkin

From: David Hildenbrand <[email protected]>

VFIO is (except devices without a physical IOMMU or some mediated devices) incompatible with discarding of RAM. The kernel will pin basically all VM memory. Let's convert to ram_block_discard_disable(), which can now fail, in contrast to qemu_balloon_inhibit().

Leave "x-balloon-allowed" named as it is for now.

Cc Cornelia Huck <[email protected]>
Cc Alex Williamson <[email protected]>
Cc Christian Borntraeger <[email protected]>
Cc Tony Krowiak <[email protected]>
Cc Halil Pasic <[email protected]>
Cc Pierre Morel <[email protected]>
Cc Eric Farman <[email protected]>

Message-Id: <[email protected]>

---
 include/hw/vfio/vfio-common.h |  4 +--
 hw/vfio/ap.c                  |  8 +++---
 hw/vfio/ccw.c                 | 11 ++++----
 hw/vfio/common.c              | 53 +++++++++++++++++++----------------
 hw/vfio/pci.c                 |  6 ++--
 5 files changed, 44 insertions(+), 38 deletions(-)
include/hw/vfio/vfio-common.h +2 -2
index fd564209ac..c78f3ff559 100644
@@ -108,7 +108,7 @@ typedef struct VFIODevice {
     bool reset_works;
     bool needs_reset;
     bool no_mmap;
-    bool balloon_allowed;
+    bool ram_block_discard_allowed;
     VFIODeviceOps *ops;
     unsigned int num_irqs;
     unsigned int num_regions;
@@ -128,7 +128,7 @@ typedef struct VFIOGroup {
     QLIST_HEAD(, VFIODevice) device_list;
     QLIST_ENTRY(VFIOGroup) next;
     QLIST_ENTRY(VFIOGroup) container_next;
-    bool balloon_allowed;
+    bool ram_block_discard_allowed;
 } VFIOGroup;
 
 typedef struct VFIODMABuf {
hw/vfio/ap.c +4 -4
index 95564c17ed..b9330a8e6f 100644
@@ -105,12 +105,12 @@ static void vfio_ap_realize(DeviceState *dev, Error **errp)
     vapdev->vdev.dev = dev;
 
     /*
-     * vfio-ap devices operate in a way compatible with
-     * memory ballooning, as no pages are pinned in the host.
+     * vfio-ap devices operate in a way compatible with discarding of
+     * memory in RAM blocks, as no pages are pinned in the host.
      * This needs to be set before vfio_get_device() for vfio common to
-     * handle the balloon inhibitor.
+     * handle ram_block_discard_disable().
      */
-    vapdev->vdev.balloon_allowed = true;
+    vapdev->vdev.ram_block_discard_allowed = true;
 
     ret = vfio_get_device(vfio_group, mdevid, &vapdev->vdev, errp);
     if (ret) {
hw/vfio/ccw.c +6 -5
index 06e69d7066..ff7f369779 100644
@@ -574,12 +574,13 @@ static void vfio_ccw_get_device(VFIOGroup *group, VFIOCCWDevice *vcdev,
 
     /*
      * All vfio-ccw devices are believed to operate in a way compatible with
-     * memory ballooning, ie. pages pinned in the host are in the current
-     * working set of the guest driver and therefore never overlap with pages
-     * available to the guest balloon driver.  This needs to be set before
-     * vfio_get_device() for vfio common to handle the balloon inhibitor.
+     * discarding of memory in RAM blocks, ie. pages pinned in the host are
+     * in the current working set of the guest driver and therefore never
+     * overlap e.g., with pages available to the guest balloon driver.  This
+     * needs to be set before vfio_get_device() for vfio common to handle
+     * ram_block_discard_disable().
      */
-    vcdev->vdev.balloon_allowed = true;
+    vcdev->vdev.ram_block_discard_allowed = true;
 
     if (vfio_get_device(group, vcdev->cdev.mdevid, &vcdev->vdev, errp)) {
         goto out_err;
hw/vfio/common.c +29 -24

104 lines not on this page · raw message

hw/vfio/pci.c +3 -3
index 6838bcc4b3..d020ea9f82 100644
@@ -2789,7 +2789,7 @@ static void vfio_realize(PCIDevice *pdev, Error **errp)
     }
 
     /*
-     * Mediated devices *might* operate compatibly with memory ballooning, but
+     * Mediated devices *might* operate compatibly with discarding of RAM, but
      * we cannot know for certain, it depends on whether the mdev vendor driver
      * stays in sync with the active working set of the guest driver.  Prevent
      * the x-balloon-allowed option unless this is minimally an mdev device.
@@ -2802,7 +2802,7 @@ static void vfio_realize(PCIDevice *pdev, Error **errp)
 
     trace_vfio_mdev(vdev->vbasedev.name, is_mdev);
 
-    if (vdev->vbasedev.balloon_allowed && !is_mdev) {
+    if (vdev->vbasedev.ram_block_discard_allowed && !is_mdev) {
         error_setg(errp, "x-balloon-allowed only potentially compatible "
                    "with mdev devices");
         vfio_put_group(group);
@@ -3156,7 +3156,7 @@ static Property vfio_pci_dev_properties[] = {
                     VFIO_FEATURE_ENABLE_IGD_OPREGION_BIT, false),
     DEFINE_PROP_BOOL("x-no-mmap", VFIOPCIDevice, vbasedev.no_mmap, false),
     DEFINE_PROP_BOOL("x-balloon-allowed", VFIOPCIDevice,
-                     vbasedev.balloon_allowed, false),
+                     vbasedev.ram_block_discard_allowed, false),
     DEFINE_PROP_BOOL("x-no-kvm-intx", VFIOPCIDevice, no_kvm_intx, false),
     DEFINE_PROP_BOOL("x-no-kvm-msi", VFIOPCIDevice, no_kvm_msi, false),
     DEFINE_PROP_BOOL("x-no-kvm-msix", VFIOPCIDevice, no_kvm_msix, false),
Signature · 1 line
MST
#14 lore raw

[PULL v2 20/41] virtio-pci: Send qapi events when the virtio-mem size changes

In reply to Michael S. Tsirkin

From: David Hildenbrand <[email protected]>

Let's register the notifier and trigger the qapi event with the right device id.

MEMORY_DEVICE_SIZE_CHANGE is similar to BALLOON_CHANGE, however on a memory device level.

Don't unregister the notifier (we neither have finalize() nor unrealize() for VirtIOPCIProxy, so it's not that simple to do it) - both devices are expected to vanish at the same time.

Cc "Michael S. Tsirkin" <[email protected]>
Cc Markus Armbruster <[email protected]>
Cc "Dr. David Alan Gilbert" <[email protected]>
Cc Eric Blake <[email protected]>
Cc Igor Mammedov <[email protected]>

Message-Id: <[email protected]>

---
 qapi/misc.json             | 25 +++++++++++++++++++++++++
 hw/virtio/virtio-mem-pci.h |  1 +
 hw/virtio/virtio-mem-pci.c | 28 ++++++++++++++++++++++++++++
 monitor/monitor.c          |  1 +
 4 files changed, 55 insertions(+)
qapi/misc.json +25 -0
index 65ca3edf32..149c925246 100644
@@ -1434,6 +1434,31 @@
 ##
 { 'command': 'query-memory-devices', 'returns': ['MemoryDeviceInfo'] }
 
+##
+# @MEMORY_DEVICE_SIZE_CHANGE:
+#
+# Emitted when the size of a memory device changes. Only emitted for memory
+# devices that can actually change the size (e.g., virtio-mem due to guest
+# action).
+#
+# @id: device's ID
+# @size: the new size of memory that the device provides
+#
+# Note: this event is rate-limited.
+#
+# Since: 5.1
+#
+# Example:
+#
+# <- { "event": "MEMORY_DEVICE_SIZE_CHANGE",
+#      "data": { "id": "vm0", "size": 1073741824},
+#      "timestamp": { "seconds": 1588168529, "microseconds": 201316 } }
+#
+##
+{ 'event': 'MEMORY_DEVICE_SIZE_CHANGE',
+  'data': { '*id': 'str', 'size': 'size' } }
+
+
 ##
 # @MEM_UNPLUG_ERROR:
 #
hw/virtio/virtio-mem-pci.h +1 -0
index 8820cd6628..b51a28b275 100644
@@ -28,6 +28,7 @@ typedef struct VirtIOMEMPCI VirtIOMEMPCI;
 struct VirtIOMEMPCI {
     VirtIOPCIProxy parent_obj;
     VirtIOMEM vdev;
+    Notifier size_change_notifier;
 };
 
 #endif /* QEMU_VIRTIO_MEM_PCI_H */
hw/virtio/virtio-mem-pci.c +28 -0
index b325303b32..1a8e854123 100644
@@ -14,6 +14,7 @@
 #include "virtio-mem-pci.h"
 #include "hw/mem/memory-device.h"
 #include "qapi/error.h"
+#include "qapi/qapi-events-misc.h"
 
 static void virtio_mem_pci_realize(VirtIOPCIProxy *vpci_dev, Error **errp)
 {
@@ -74,6 +75,21 @@ static void virtio_mem_pci_fill_device_info(const MemoryDeviceState *md,
     info->type = MEMORY_DEVICE_INFO_KIND_VIRTIO_MEM;
 }
 
+static void virtio_mem_pci_size_change_notify(Notifier *notifier, void *data)
+{
+    VirtIOMEMPCI *pci_mem = container_of(notifier, VirtIOMEMPCI,
+                                         size_change_notifier);
+    DeviceState *dev = DEVICE(pci_mem);
+    const uint64_t * const size_p = data;
+    const char *id = NULL;
+
+    if (dev->id) {
+        id = g_strdup(dev->id);
+    }
+
+    qapi_event_send_memory_device_size_change(!!id, id, *size_p);
+}
+
 static void virtio_mem_pci_class_init(ObjectClass *klass, void *data)
 {
     DeviceClass *dc = DEVICE_CLASS(klass);
@@ -98,9 +114,21 @@ static void virtio_mem_pci_class_init(ObjectClass *klass, void *data)
 static void virtio_mem_pci_instance_init(Object *obj)
 {
     VirtIOMEMPCI *dev = VIRTIO_MEM_PCI(obj);
+    VirtIOMEMClass *vmc;
+    VirtIOMEM *vmem;
 
     virtio_instance_init_common(obj, &dev->vdev, sizeof(dev->vdev),
                                 TYPE_VIRTIO_MEM);
+
+    dev->size_change_notifier.notify = virtio_mem_pci_size_change_notify;
+    vmem = VIRTIO_MEM(&dev->vdev);
+    vmc = VIRTIO_MEM_GET_CLASS(vmem);
+    /*
+     * We never remove the notifier again, as we expect both devices to
+     * disappear at the same time.
+     */
+    vmc->add_size_change_notifier(vmem, &dev->size_change_notifier);
+
     object_property_add_alias(obj, VIRTIO_MEM_BLOCK_SIZE_PROP,
                               OBJECT(&dev->vdev), VIRTIO_MEM_BLOCK_SIZE_PROP);
     object_property_add_alias(obj, VIRTIO_MEM_SIZE_PROP, OBJECT(&dev->vdev),
monitor/monitor.c +1 -0
index 125494410a..19dcb8fbe3 100644
@@ -235,6 +235,7 @@ static MonitorQAPIEventConf monitor_qapi_event_conf[QAPI_EVENT__MAX] = {
     [QAPI_EVENT_QUORUM_REPORT_BAD] = { 1000 * SCALE_MS },
     [QAPI_EVENT_QUORUM_FAILURE]    = { 1000 * SCALE_MS },
     [QAPI_EVENT_VSERPORT_CHANGE]   = { 1000 * SCALE_MS },
+    [QAPI_EVENT_MEMORY_DEVICE_SIZE_CHANGE] = { 1000 * SCALE_MS },
 };
 
 /*
Signature · 1 line
MST
#15 lore raw

[PULL v2 26/41] docs: vhost-user: add Virtio status protocol feature

In reply to Michael S. Tsirkin

From: Maxime Coquelin <[email protected]>

This patch specifies the VHOST_USER_SET_STATUS and VHOST_USER_GET_STATUS requests, which are sent by the master to update and query the Virtio status in the backend.

Message-Id: <[email protected]>

---
 docs/interop/vhost-user.rst | 24 ++++++++++++++++++++++++
 1 file changed, 24 insertions(+)
docs/interop/vhost-user.rst +24 -0
index 688b7c6900..10e3e3475e 100644
@@ -816,6 +816,7 @@ Protocol features
   #define VHOST_USER_PROTOCOL_F_RESET_DEVICE         13
   #define VHOST_USER_PROTOCOL_F_INBAND_NOTIFICATIONS 14
   #define VHOST_USER_PROTOCOL_F_CONFIGURE_MEM_SLOTS  15
+  #define VHOST_USER_PROTOCOL_F_STATUS               16
 
 Master message types
 --------------------
@@ -1307,6 +1308,29 @@ Master message types
   ``VHOST_USER_ADD_MEM_REG`` message, this message is used to set and
   update the memory tables of the slave device.
 
+``VHOST_USER_SET_STATUS``
+  :id: 39
+  :equivalent ioctl: VHOST_VDPA_SET_STATUS
+  :slave payload: N/A
+  :master payload: ``u64``
+
+  When the ``VHOST_USER_PROTOCOL_F_STATUS`` protocol feature has been
+  successfully negotiated, this message is submitted by the master to
+  notify the backend with updated device status as defined in the Virtio
+  specification.
+
+``VHOST_USER_GET_STATUS``
+  :id: 40
+  :equivalent ioctl: VHOST_VDPA_GET_STATUS
+  :slave payload: ``u64``
+  :master payload: N/A
+
+  When the ``VHOST_USER_PROTOCOL_F_STATUS`` protocol feature has been
+  successfully negotiated, this message is submitted by the master to
+  query the backend for its device status as defined in the Virtio
+  specification.
+
+
 Slave message types
 -------------------
 
Signature · 1 line
MST
#16 lore raw

[PULL v2 15/41] MAINTAINERS: Add myself as virtio-mem maintainer

In reply to Michael S. Tsirkin

From: David Hildenbrand <[email protected]>

Let's make sure patches/bug reports find the right person.

Cc "Michael S. Tsirkin" <[email protected]>
Cc Peter Maydell <[email protected]>
Cc Markus Armbruster <[email protected]>

Message-Id: <[email protected]>

---
 MAINTAINERS | 9 +++++++++
 1 file changed, 9 insertions(+)
MAINTAINERS +9 -0
index dec252f38b..5f02160436 100644
@@ -1790,6 +1790,15 @@ F: hw/virtio/virtio-crypto.c
 F: hw/virtio/virtio-crypto-pci.c
 F: include/hw/virtio/virtio-crypto.h
 
+virtio-mem
+M: David Hildenbrand <[email protected]>
+S: Supported
+W: https://virtio-mem.gitlab.io/
+F: hw/virtio/virtio-mem.c
+F: hw/virtio/virtio-mem-pci.h
+F: hw/virtio/virtio-mem-pci.c
+F: include/hw/virtio/virtio-mem.h
+
 nvme
 M: Keith Busch <[email protected]>
 L: [email protected]
Signature · 1 line
MST
#17 lore raw

[PULL v2 25/41] tests/acpi: remove stale allowed tables

In reply to Michael S. Tsirkin

From: Andrew Jones <[email protected]>

Fixes 93dd625f8bf7 ("tests/acpi: update expected data files")

Message-Id: <[email protected]>

---
 tests/qtest/bios-tables-test-allowed-diff.h | 18 ------------------
 1 file changed, 18 deletions(-)
tests/qtest/bios-tables-test-allowed-diff.h +0 -18
index 8992f1f12b..dfb8523c8b 100644
@@ -1,19 +1 @@
 /* List of comma-separated changed AML files to ignore */
-"tests/data/acpi/pc/DSDT",
-"tests/data/acpi/pc/DSDT.acpihmat",
-"tests/data/acpi/pc/DSDT.bridge",
-"tests/data/acpi/pc/DSDT.cphp",
-"tests/data/acpi/pc/DSDT.dimmpxm",
-"tests/data/acpi/pc/DSDT.ipmikcs",
-"tests/data/acpi/pc/DSDT.memhp",
-"tests/data/acpi/pc/DSDT.numamem",
-"tests/data/acpi/q35/DSDT",
-"tests/data/acpi/q35/DSDT.acpihmat",
-"tests/data/acpi/q35/DSDT.bridge",
-"tests/data/acpi/q35/DSDT.cphp",
-"tests/data/acpi/q35/DSDT.dimmpxm",
-"tests/data/acpi/q35/DSDT.ipmibt",
-"tests/data/acpi/q35/DSDT.memhp",
-"tests/data/acpi/q35/DSDT.mmio64",
-"tests/data/acpi/q35/DSDT.numamem",
-"tests/data/acpi/q35/DSDT.tis",
Signature · 1 line
MST
#18 lore raw

[PULL v2 29/41] vhost_net: use the function qemu_get_peer

In reply to Michael S. Tsirkin

From: Cindy Lu <[email protected]>

user the qemu_get_peer to replace the old process

Message-Id: <[email protected]>

---
 hw/net/vhost_net.c | 16 ++++++++++------
 1 file changed, 10 insertions(+), 6 deletions(-)
hw/net/vhost_net.c +10 -6
index 6b82803fa7..4096d64aaf 100644
@@ -306,7 +306,9 @@ int vhost_net_start(VirtIODevice *dev, NetClientState *ncs,
     BusState *qbus = BUS(qdev_get_parent_bus(DEVICE(dev)));
     VirtioBusState *vbus = VIRTIO_BUS(qbus);
     VirtioBusClass *k = VIRTIO_BUS_GET_CLASS(vbus);
+    struct vhost_net *net;
     int r, e, i;
+    NetClientState *peer;
 
     if (!k->set_guest_notifiers) {
         error_report("binding does not support guest notifiers");
@@ -314,9 +316,9 @@ int vhost_net_start(VirtIODevice *dev, NetClientState *ncs,
     }
 
     for (i = 0; i < total_queues; i++) {
-        struct vhost_net *net;
 
-        net = get_vhost_net(ncs[i].peer);
+        peer = qemu_get_peer(ncs, i);
+        net = get_vhost_net(peer);
         vhost_net_set_vq_index(net, i * 2);
 
         /* Suppress the masking guest notifiers on vhost user
@@ -335,15 +337,16 @@ int vhost_net_start(VirtIODevice *dev, NetClientState *ncs,
     }
 
     for (i = 0; i < total_queues; i++) {
-        r = vhost_net_start_one(get_vhost_net(ncs[i].peer), dev);
+        peer = qemu_get_peer(ncs, i);
+        r = vhost_net_start_one(get_vhost_net(peer), dev);
 
         if (r < 0) {
             goto err_start;
         }
 
-        if (ncs[i].peer->vring_enable) {
+        if (peer->vring_enable) {
             /* restore vring enable state */
-            r = vhost_set_vring_enable(ncs[i].peer, ncs[i].peer->vring_enable);
+            r = vhost_set_vring_enable(peer, peer->vring_enable);
 
             if (r < 0) {
                 goto err_start;
@@ -355,7 +358,8 @@ int vhost_net_start(VirtIODevice *dev, NetClientState *ncs,
 
 err_start:
     while (--i >= 0) {
-        vhost_net_stop_one(get_vhost_net(ncs[i].peer), dev);
+        peer = qemu_get_peer(ncs , i);
+        vhost_net_stop_one(get_vhost_net(peer), dev);
     }
     e = k->set_guest_notifiers(qbus->parent, total_queues * 2, false);
     if (e < 0) {
Signature · 1 line
MST
#19 lore raw

[PULL v2 17/41] numa: Handle virtio-mem in NUMA stats

In reply to Michael S. Tsirkin

From: David Hildenbrand <[email protected]>

Account the memory to the configured nid.

Cc Eduardo Habkost <[email protected]>
Cc Marcel Apfelbaum <[email protected]>
Cc "Michael S. Tsirkin" <[email protected]>

Message-Id: <[email protected]>

---
 hw/core/numa.c | 6 ++++++
 1 file changed, 6 insertions(+)
hw/core/numa.c +6 -0
index 2725886d06..e9aec69afd 100644
@@ -824,6 +824,7 @@ static void numa_stat_memory_devices(NumaNodeMem node_mem[])
     MemoryDeviceInfoList *info;
     PCDIMMDeviceInfo     *pcdimm_info;
     VirtioPMEMDeviceInfo *vpi;
+    VirtioMEMDeviceInfo *vmi;
 
     for (info = info_list; info; info = info->next) {
         MemoryDeviceInfo *value = info->value;
@@ -844,6 +845,11 @@ static void numa_stat_memory_devices(NumaNodeMem node_mem[])
                 node_mem[0].node_mem += vpi->size;
                 node_mem[0].node_plugged_mem += vpi->size;
                 break;
+            case MEMORY_DEVICE_INFO_KIND_VIRTIO_MEM:
+                vmi = value->u.virtio_mem.data;
+                node_mem[vmi->node].node_mem += vmi->size;
+                node_mem[vmi->node].node_plugged_mem += vmi->size;
+                break;
             default:
                 g_assert_not_reached();
             }
Signature · 1 line
MST
#20 lore raw

[PULL v2 27/41] MAINTAINERS: add VT-d entry

In reply to Michael S. Tsirkin

From: Peter Xu <[email protected]>

Add this entry as suggested by Jason and Michael.

CC Jason Wang <[email protected]>
CC Michael S. Tsirkin <[email protected]>
CC Paolo Bonzini <[email protected]>

Message-Id: <[email protected]>

---
 MAINTAINERS | 9 +++++++++
 1 file changed, 9 insertions(+)
MAINTAINERS +9 -0
index 5f02160436..49a0d837d7 100644
@@ -2624,6 +2624,15 @@ F: tests/uefi-test-tools/
 F: .gitlab-ci.d/edk2.yml
 F: .gitlab-ci.d/edk2/
 
+VT-d Emulation
+M: Michael S. Tsirkin <[email protected]>
+M: Peter Xu <[email protected]>
+R: Jason Wang <[email protected]>
+S: Supported
+F: hw/i386/intel_iommu.c
+F: hw/i386/intel_iommu_internal.h
+F: include/hw/i386/intel_iommu.h
+
 Usermode Emulation
 ------------------
 Overall usermode emulation
Signature · 1 line
MST
#21 lore raw

[PULL v2 09/41] virtio-balloon: Rip out qemu_balloon_inhibit()

In reply to Michael S. Tsirkin

From: David Hildenbrand <[email protected]>

The only remaining special case is postcopy. It cannot handle concurrent discards yet, which would result in requesting already sent pages from the source. Special-case it in virtio-balloon instead.

Introduce migration_in_incoming_postcopy(), to find out if incoming postcopy is active.

Message-Id: <[email protected]>

---
 include/migration/misc.h   |  2 ++
 include/sysemu/balloon.h   |  2 --
 balloon.c                  | 18 ------------------
 hw/virtio/virtio-balloon.c | 10 ++++++++--
 migration/migration.c      |  7 +++++++
 migration/postcopy-ram.c   | 23 -----------------------
 6 files changed, 17 insertions(+), 45 deletions(-)
include/migration/misc.h +2 -0
index d2762257aa..34e7d75713 100644
@@ -69,6 +69,8 @@ bool migration_has_failed(MigrationState *);
 /* ...and after the device transmission */
 bool migration_in_postcopy_after_devices(MigrationState *);
 void migration_global_dump(Monitor *mon);
+/* True if incomming migration entered POSTCOPY_INCOMING_DISCARD */
+bool migration_in_incoming_postcopy(void);
 
 /* migration/block-dirty-bitmap.c */
 void dirty_bitmap_mig_init(void);
include/sysemu/balloon.h +0 -2
index aea0c44985..20a2defe3a 100644
@@ -23,7 +23,5 @@ typedef void (QEMUBalloonStatus)(void *opaque, BalloonInfo *info);
 int qemu_add_balloon_handler(QEMUBalloonEvent *event_func,
                              QEMUBalloonStatus *stat_func, void *opaque);
 void qemu_remove_balloon_handler(void *opaque);
-bool qemu_balloon_is_inhibited(void);
-void qemu_balloon_inhibit(bool state);
 
 #endif
balloon.c +0 -18
index 5fff79523a..354408c6ea 100644
@@ -36,24 +36,6 @@
 static QEMUBalloonEvent *balloon_event_fn;
 static QEMUBalloonStatus *balloon_stat_fn;
 static void *balloon_opaque;
-static int balloon_inhibit_count;
-
-bool qemu_balloon_is_inhibited(void)
-{
-    return atomic_read(&balloon_inhibit_count) > 0 ||
-           ram_block_discard_is_disabled();
-}
-
-void qemu_balloon_inhibit(bool state)
-{
-    if (state) {
-        atomic_inc(&balloon_inhibit_count);
-    } else {
-        atomic_dec(&balloon_inhibit_count);
-    }
-
-    assert(atomic_read(&balloon_inhibit_count) >= 0);
-}
 
 static bool have_balloon(Error **errp)
 {
hw/virtio/virtio-balloon.c +8 -2
index 8a84718490..ae31f0817a 100644
@@ -63,6 +63,12 @@ static bool virtio_balloon_pbp_matches(PartiallyBalloonedPage *pbp,
     return pbp->base_gpa == base_gpa;
 }
 
+static bool virtio_balloon_inhibited(void)
+{
+    /* Postcopy cannot deal with concurrent discards, so it's special. */
+    return ram_block_discard_is_disabled() || migration_in_incoming_postcopy();
+}
+
 static void balloon_inflate_page(VirtIOBalloon *balloon,
                                  MemoryRegion *mr, hwaddr mr_offset,
                                  PartiallyBalloonedPage *pbp)
@@ -336,7 +342,7 @@ static void virtio_balloon_handle_report(VirtIODevice *vdev, VirtQueue *vq)
          * accessible by another device or process, or if the guest is
          * expecting it to retain a non-zero value.
          */
-        if (qemu_balloon_is_inhibited() || dev->poison_val) {
+        if (virtio_balloon_inhibited() || dev->poison_val) {
             goto skip_element;
         }
 
@@ -421,7 +427,7 @@ static void virtio_balloon_handle_output(VirtIODevice *vdev, VirtQueue *vq)
 
             trace_virtio_balloon_handle_output(memory_region_name(section.mr),
                                                pa);
-            if (!qemu_balloon_is_inhibited()) {
+            if (!virtio_balloon_inhibited()) {
                 if (vq == s->ivq) {
                     balloon_inflate_page(s, section.mr,
                                          section.offset_within_region, &pbp);
migration/migration.c +7 -0
index 481a590f72..d365d82209 100644
@@ -1772,6 +1772,13 @@ bool migration_in_postcopy_after_devices(MigrationState *s)
     return migration_in_postcopy() && s->postcopy_after_devices;
 }
 
+bool migration_in_incoming_postcopy(void)
+{
+    PostcopyState ps = postcopy_state_get();
+
+    return ps >= POSTCOPY_INCOMING_DISCARD && ps < POSTCOPY_INCOMING_END;
+}
+
 bool migration_is_idle(void)
 {
     MigrationState *s = current_migration;
migration/postcopy-ram.c +0 -23
index a36402722b..b41a9fe2fd 100644
@@ -27,7 +27,6 @@
 #include "qemu/notify.h"
 #include "qemu/rcu.h"
 #include "sysemu/sysemu.h"
-#include "sysemu/balloon.h"
 #include "qemu/error-report.h"
 #include "trace.h"
 #include "hw/boards.h"
@@ -520,20 +519,6 @@ int postcopy_ram_incoming_init(MigrationIncomingState *mis)
     return 0;
 }
 
-/*
- * Manage a single vote to the QEMU balloon inhibitor for all postcopy usage,
- * last caller wins.
- */
-static void postcopy_balloon_inhibit(bool state)
-{
-    static bool cur_state = false;
-
-    if (state != cur_state) {
-        qemu_balloon_inhibit(state);
-        cur_state = state;
-    }
-}
-
 /*
  * At the end of a migration where postcopy_ram_incoming_init was called.
  */
@@ -565,8 +550,6 @@ int postcopy_ram_incoming_cleanup(MigrationIncomingState *mis)
         mis->have_fault_thread = false;
     }
 
-    postcopy_balloon_inhibit(false);
-
     if (enable_mlock) {
         if (os_mlock() < 0) {
             error_report("mlock: %s", strerror(errno));
@@ -1160,12 +1143,6 @@ int postcopy_ram_incoming_setup(MigrationIncomingState *mis)
     }
     memset(mis->postcopy_tmp_zero_page, '\0', mis->largest_page_size);
 
-    /*
-     * Ballooning can mark pages as absent while we're postcopying
-     * that would cause false userfaults.
-     */
-    postcopy_balloon_inhibit(true);
-
     trace_postcopy_ram_enable_notify();
 
     return 0;
Signature · 1 line
MST
#22 lore raw

[PULL v2 32/41] vhost: check the existence of vhost_set_iotlb_callback

In reply to Michael S. Tsirkin

From: Jason Wang <[email protected]>

Add the check of vhost_set_iotlb_callback before calling

Message-Id: <[email protected]>

---
 hw/virtio/vhost.c | 9 ++++++---
 1 file changed, 6 insertions(+), 3 deletions(-)
hw/virtio/vhost.c +6 -3
index 5fd25fe520..10304b583e 100644
@@ -1686,8 +1686,9 @@ int vhost_dev_start(struct vhost_dev *hdev, VirtIODevice *vdev)
         }
     }
 
-    if (vhost_dev_has_iommu(hdev)) {
-        hdev->vhost_ops->vhost_set_iotlb_callback(hdev, true);
+    if (vhost_dev_has_iommu(hdev) &&
+        hdev->vhost_ops->vhost_set_iotlb_callback) {
+            hdev->vhost_ops->vhost_set_iotlb_callback(hdev, true);
 
         /* Update used ring information for IOTLB to work correctly,
          * vhost-kernel code requires for this.*/
@@ -1730,7 +1731,9 @@ void vhost_dev_stop(struct vhost_dev *hdev, VirtIODevice *vdev)
     }
 
     if (vhost_dev_has_iommu(hdev)) {
-        hdev->vhost_ops->vhost_set_iotlb_callback(hdev, false);
+        if (hdev->vhost_ops->vhost_set_iotlb_callback) {
+            hdev->vhost_ops->vhost_set_iotlb_callback(hdev, false);
+        }
         memory_listener_unregister(&hdev->iommu_listener);
     }
     vhost_log_put(hdev, true);
Signature · 1 line
MST
#23 lore raw

[PULL v2 19/41] virtio-mem: Allow notifiers for size changes

In reply to Michael S. Tsirkin

From: David Hildenbrand <[email protected]>

We want to send qapi events in case the size of a virtio-mem device changes. This allows upper layers to always know how much memory is actually currently consumed via a virtio-mem device.

Unfortuantely, we have to report the id of our proxy device. Let's provide an easy way for our proxy device to register, so it can send the qapi events. Piggy-backing on the notifier infrastructure (although we'll only ever have one notifier registered) seems to be an easy way.

Cc "Michael S. Tsirkin" <[email protected]>
Cc "Dr. David Alan Gilbert" <[email protected]>
Cc Igor Mammedov <[email protected]>

Message-Id: <[email protected]>

---
 include/hw/virtio/virtio-mem.h |  5 +++++
 hw/virtio/virtio-mem.c         | 21 ++++++++++++++++++++-
 2 files changed, 25 insertions(+), 1 deletion(-)
include/hw/virtio/virtio-mem.h +5 -0
index 6981096f7c..b74c77cd42 100644
@@ -64,6 +64,9 @@ typedef struct VirtIOMEM {
 
     /* block size and alignment */
     uint64_t block_size;
+
+    /* notifiers to notify when "size" changes */
+    NotifierList size_change_notifiers;
 } VirtIOMEM;
 
 typedef struct VirtIOMEMClass {
@@ -73,6 +76,8 @@ typedef struct VirtIOMEMClass {
     /* public */
     void (*fill_device_info)(const VirtIOMEM *vmen, VirtioMEMDeviceInfo *vi);
     MemoryRegion *(*get_memory_region)(VirtIOMEM *vmem, Error **errp);
+    void (*add_size_change_notifier)(VirtIOMEM *vmem, Notifier *notifier);
+    void (*remove_size_change_notifier)(VirtIOMEM *vmem, Notifier *notifier);
 } VirtIOMEMClass;
 
 #endif
hw/virtio/virtio-mem.c +20 -1
index 2d6f3061a6..6b5551515d 100644
@@ -184,6 +184,7 @@ static int virtio_mem_state_change_request(VirtIOMEM *vmem, uint64_t gpa,
     } else {
         vmem->size -= size;
     }
+    notifier_list_notify(&vmem->size_change_notifiers, &vmem->size);
     return VIRTIO_MEM_RESP_ACK;
 }
 
@@ -242,7 +243,10 @@ static int virtio_mem_unplug_all(VirtIOMEM *vmem)
         return -EBUSY;
     }
     bitmap_clear(vmem->bitmap, 0, vmem->bitmap_size);
-    vmem->size = 0;
+    if (vmem->size) {
+        vmem->size = 0;
+        notifier_list_notify(&vmem->size_change_notifiers, &vmem->size);
+    }
 
     virtio_mem_resize_usable_region(vmem, vmem->requested_size, true);
     return 0;
@@ -561,6 +565,18 @@ static MemoryRegion *virtio_mem_get_memory_region(VirtIOMEM *vmem, Error **errp)
     return &vmem->memdev->mr;
 }
 
+static void virtio_mem_add_size_change_notifier(VirtIOMEM *vmem,
+                                                Notifier *notifier)
+{
+    notifier_list_add(&vmem->size_change_notifiers, notifier);
+}
+
+static void virtio_mem_remove_size_change_notifier(VirtIOMEM *vmem,
+                                                   Notifier *notifier)
+{
+    notifier_remove(notifier);
+}
+
 static void virtio_mem_get_size(Object *obj, Visitor *v, const char *name,
                                 void *opaque, Error **errp)
 {
@@ -668,6 +684,7 @@ static void virtio_mem_instance_init(Object *obj)
     VirtIOMEM *vmem = VIRTIO_MEM(obj);
 
     vmem->block_size = VIRTIO_MEM_MIN_BLOCK_SIZE;
+    notifier_list_init(&vmem->size_change_notifiers);
 
     object_property_add(obj, VIRTIO_MEM_SIZE_PROP, "size", virtio_mem_get_size,
                         NULL, NULL, NULL);
@@ -705,6 +722,8 @@ static void virtio_mem_class_init(ObjectClass *klass, void *data)
 
     vmc->fill_device_info = virtio_mem_fill_device_info;
     vmc->get_memory_region = virtio_mem_get_memory_region;
+    vmc->add_size_change_notifier = virtio_mem_add_size_change_notifier;
+    vmc->remove_size_change_notifier = virtio_mem_remove_size_change_notifier;
 }
 
 static const TypeInfo virtio_mem_info = {
Signature · 1 line
MST
#24 lore raw

[PULL v2 33/41] vhost: introduce new VhostOps vhost_dev_start

In reply to Michael S. Tsirkin

From: Cindy Lu <[email protected]>

This patch introduces new VhostOps vhost_dev_start callback which allows the vhost_net set the start/stop status to backend

Message-Id: <[email protected]>

---
 include/hw/virtio/vhost-backend.h | 2 ++
 1 file changed, 2 insertions(+)
include/hw/virtio/vhost-backend.h +2 -0
index 6f6670783f..b80f344cd6 100644
@@ -112,6 +112,7 @@ typedef int (*vhost_get_inflight_fd_op)(struct vhost_dev *dev,
 typedef int (*vhost_set_inflight_fd_op)(struct vhost_dev *dev,
                                         struct vhost_inflight *inflight);
 
+typedef int (*vhost_dev_start_op)(struct vhost_dev *dev, bool started);
 typedef struct VhostOps {
     VhostBackendType backend_type;
     vhost_backend_init vhost_backend_init;
@@ -152,6 +153,7 @@ typedef struct VhostOps {
     vhost_backend_mem_section_filter_op vhost_backend_mem_section_filter;
     vhost_get_inflight_fd_op vhost_get_inflight_fd;
     vhost_set_inflight_fd_op vhost_set_inflight_fd;
+    vhost_dev_start_op vhost_dev_start;
 } VhostOps;
 
 extern const VhostOps user_ops;
Signature · 1 line
MST
#25 lore raw

[PULL v2 12/41] migration/colo: Use ram_block_discard_disable()

In reply to Michael S. Tsirkin

From: David Hildenbrand <[email protected]>

COLO will copy all memory in a RAM block, disable discarding of RAM.

Cc "Michael S. Tsirkin" <[email protected]>
Cc Hailiang Zhang <[email protected]>
Cc Juan Quintela <[email protected]>
Cc "Dr. David Alan Gilbert" <[email protected]>

Message-Id: <[email protected]>

---
 include/migration/colo.h |  2 +-
 migration/migration.c    |  8 +++++++-
 migration/savevm.c       | 11 +++++++++--
 3 files changed, 17 insertions(+), 4 deletions(-)
include/migration/colo.h +1 -1
index 1636e6f907..768e1f04c3 100644
@@ -25,7 +25,7 @@ void migrate_start_colo_process(MigrationState *s);
 bool migration_in_colo_state(void);
 
 /* loadvm */
-void migration_incoming_enable_colo(void);
+int migration_incoming_enable_colo(void);
 void migration_incoming_disable_colo(void);
 bool migration_incoming_colo_enabled(void);
 void *colo_process_incoming_thread(void *opaque);
migration/migration.c +7 -1
index d365d82209..92e44e021e 100644
@@ -338,12 +338,18 @@ bool migration_incoming_colo_enabled(void)
 
 void migration_incoming_disable_colo(void)
 {
+    ram_block_discard_disable(false);
     migration_colo_enabled = false;
 }
 
-void migration_incoming_enable_colo(void)
+int migration_incoming_enable_colo(void)
 {
+    if (ram_block_discard_disable(true)) {
+        error_report("COLO: cannot disable RAM discard");
+        return -EBUSY;
+    }
     migration_colo_enabled = true;
+    return 0;
 }
 
 void migrate_add_address(SocketAddress *address)
migration/savevm.c +9 -2
index b979ea6e7f..6e01724605 100644
@@ -2111,8 +2111,15 @@ static int loadvm_handle_recv_bitmap(MigrationIncomingState *mis,
 
 static int loadvm_process_enable_colo(MigrationIncomingState *mis)
 {
-    migration_incoming_enable_colo();
-    return colo_init_ram_cache();
+    int ret = migration_incoming_enable_colo();
+
+    if (!ret) {
+        ret = colo_init_ram_cache();
+        if (ret) {
+            migration_incoming_disable_colo();
+        }
+    }
+    return ret;
 }
 
 /*
Signature · 1 line
MST
#26 lore raw

[PULL v2 36/41] vhost: implement vhost_vq_get_addr method

In reply to Michael S. Tsirkin

From: Cindy Lu <[email protected]>

use vhost_vq_get_addr callback to get the vq address from backend

Message-Id: <[email protected]>

---
 include/hw/virtio/vhost-backend.h |  4 ++++
 hw/virtio/vhost.c                 | 28 +++++++++++++++++++---------
 2 files changed, 23 insertions(+), 9 deletions(-)
include/hw/virtio/vhost-backend.h +4 -0
index fa84abac97..bfc24207e2 100644
@@ -118,6 +118,9 @@ typedef int (*vhost_dev_start_op)(struct vhost_dev *dev, bool started);
 typedef int (*vhost_vq_get_addr_op)(struct vhost_dev *dev,
                     struct vhost_vring_addr *addr,
                     struct vhost_virtqueue *vq);
+
+typedef int (*vhost_get_device_id_op)(struct vhost_dev *dev, uint32_t *dev_id);
+
 typedef struct VhostOps {
     VhostBackendType backend_type;
     vhost_backend_init vhost_backend_init;
@@ -160,6 +163,7 @@ typedef struct VhostOps {
     vhost_set_inflight_fd_op vhost_set_inflight_fd;
     vhost_dev_start_op vhost_dev_start;
     vhost_vq_get_addr_op  vhost_vq_get_addr;
+    vhost_get_device_id_op vhost_get_device_id;
 } VhostOps;
 
 extern const VhostOps user_ops;
hw/virtio/vhost.c +19 -9
index 32809e54b5..1e083a8976 100644
@@ -773,15 +773,25 @@ static int vhost_virtqueue_set_addr(struct vhost_dev *dev,
                                     struct vhost_virtqueue *vq,
                                     unsigned idx, bool enable_log)
 {
-    struct vhost_vring_addr addr = {
-        .index = idx,
-        .desc_user_addr = (uint64_t)(unsigned long)vq->desc,
-        .avail_user_addr = (uint64_t)(unsigned long)vq->avail,
-        .used_user_addr = (uint64_t)(unsigned long)vq->used,
-        .log_guest_addr = vq->used_phys,
-        .flags = enable_log ? (1 << VHOST_VRING_F_LOG) : 0,
-    };
-    int r = dev->vhost_ops->vhost_set_vring_addr(dev, &addr);
+    struct vhost_vring_addr addr;
+    int r;
+    memset(&addr, 0, sizeof(struct vhost_vring_addr));
+
+    if (dev->vhost_ops->vhost_vq_get_addr) {
+        r = dev->vhost_ops->vhost_vq_get_addr(dev, &addr, vq);
+        if (r < 0) {
+            VHOST_OPS_DEBUG("vhost_vq_get_addr failed");
+            return -errno;
+        }
+    } else {
+        addr.desc_user_addr = (uint64_t)(unsigned long)vq->desc;
+        addr.avail_user_addr = (uint64_t)(unsigned long)vq->avail;
+        addr.used_user_addr = (uint64_t)(unsigned long)vq->used;
+    }
+    addr.index = idx;
+    addr.log_guest_addr = vq->used_phys;
+    addr.flags = enable_log ? (1 << VHOST_VRING_F_LOG) : 0;
+    r = dev->vhost_ops->vhost_set_vring_addr(dev, &addr);
     if (r < 0) {
         VHOST_OPS_DEBUG("vhost_set_vring_addr failed");
         return -errno;
Signature · 1 line
MST
#27 lore raw

[PULL v2 22/41] virtio-mem: Add trace events

In reply to Michael S. Tsirkin

From: David Hildenbrand <[email protected]>

Let's add some trace events that might come in handy later.

Cc "Michael S. Tsirkin" <[email protected]>
Cc "Dr. David Alan Gilbert" <[email protected]>

Message-Id: <[email protected]>

---
 hw/virtio/virtio-mem.c | 10 +++++++++-
 hw/virtio/trace-events | 10 ++++++++++
 2 files changed, 19 insertions(+), 1 deletion(-)
hw/virtio/virtio-mem.c +9 -1
index c1acfb9f24..1ab9ef17fa 100644
@@ -30,6 +30,7 @@
 #include "hw/boards.h"
 #include "hw/qdev-properties.h"
 #include "config-devices.h"
+#include "trace.h"
 
 /*
  * Use QEMU_VMALLOC_ALIGN, so no THP will have to be split when unplugging
@@ -100,6 +101,7 @@ static void virtio_mem_send_response(VirtIOMEM *vmem, VirtQueueElement *elem,
     VirtIODevice *vdev = VIRTIO_DEVICE(vmem);
     VirtQueue *vq = vmem->vq;
 
+    trace_virtio_mem_send_response(le16_to_cpu(resp->type));
     iov_from_buf(elem->in_sg, elem->in_num, 0, resp, sizeof(*resp));
 
     virtqueue_push(vq, elem, sizeof(*resp));
@@ -195,6 +197,7 @@ static void virtio_mem_plug_request(VirtIOMEM *vmem, VirtQueueElement *elem,
     const uint16_t nb_blocks = le16_to_cpu(req->u.plug.nb_blocks);
     uint16_t type;
 
+    trace_virtio_mem_plug_request(gpa, nb_blocks);
     type = virtio_mem_state_change_request(vmem, gpa, nb_blocks, true);
     virtio_mem_send_response_simple(vmem, elem, type);
 }
@@ -206,6 +209,7 @@ static void virtio_mem_unplug_request(VirtIOMEM *vmem, VirtQueueElement *elem,
     const uint16_t nb_blocks = le16_to_cpu(req->u.unplug.nb_blocks);
     uint16_t type;
 
+    trace_virtio_mem_unplug_request(gpa, nb_blocks);
     type = virtio_mem_state_change_request(vmem, gpa, nb_blocks, false);
     virtio_mem_send_response_simple(vmem, elem, type);
 }
@@ -225,6 +229,7 @@ static void virtio_mem_resize_usable_region(VirtIOMEM *vmem,
         return;
     }
 
+    trace_virtio_mem_resized_usable_region(vmem->usable_region_size, newsize);
     vmem->usable_region_size = newsize;
 }
 
@@ -247,7 +252,7 @@ static int virtio_mem_unplug_all(VirtIOMEM *vmem)
         vmem->size = 0;
         notifier_list_notify(&vmem->size_change_notifiers, &vmem->size);
     }
-
+    trace_virtio_mem_unplugged_all();
     virtio_mem_resize_usable_region(vmem, vmem->requested_size, true);
     return 0;
 }
@@ -255,6 +260,7 @@ static int virtio_mem_unplug_all(VirtIOMEM *vmem)
 static void virtio_mem_unplug_all_request(VirtIOMEM *vmem,
                                           VirtQueueElement *elem)
 {
+    trace_virtio_mem_unplug_all_request();
     if (virtio_mem_unplug_all(vmem)) {
         virtio_mem_send_response_simple(vmem, elem, VIRTIO_MEM_RESP_BUSY);
     } else {
@@ -272,6 +278,7 @@ static void virtio_mem_state_request(VirtIOMEM *vmem, VirtQueueElement *elem,
         .type = cpu_to_le16(VIRTIO_MEM_RESP_ACK),
     };
 
+    trace_virtio_mem_state_request(gpa, nb_blocks);
     if (!virtio_mem_valid_range(vmem, gpa, size)) {
         virtio_mem_send_response_simple(vmem, elem, VIRTIO_MEM_RESP_ERROR);
         return;
@@ -284,6 +291,7 @@ static void virtio_mem_state_request(VirtIOMEM *vmem, VirtQueueElement *elem,
     } else {
         resp.u.state.state = cpu_to_le16(VIRTIO_MEM_STATE_MIXED);
     }
+    trace_virtio_mem_state_response(le16_to_cpu(resp.u.state.state));
     virtio_mem_send_response(vmem, elem, &resp);
 }
 
hw/virtio/trace-events +10 -0
index 6427a0047d..292fc15e29 100644
@@ -74,3 +74,13 @@ virtio_iommu_get_domain(uint32_t domain_id) "Alloc domain=%d"
 virtio_iommu_put_domain(uint32_t domain_id) "Free domain=%d"
 virtio_iommu_translate_out(uint64_t virt_addr, uint64_t phys_addr, uint32_t sid) "0x%"PRIx64" -> 0x%"PRIx64 " for sid=%d"
 virtio_iommu_report_fault(uint8_t reason, uint32_t flags, uint32_t endpoint, uint64_t addr) "FAULT reason=%d flags=%d endpoint=%d address =0x%"PRIx64
+
+# virtio-mem.c
+virtio_mem_send_response(uint16_t type) "type=%" PRIu16
+virtio_mem_plug_request(uint64_t addr, uint16_t nb_blocks) "addr=0x%" PRIx64 " nb_blocks=%" PRIu16
+virtio_mem_unplug_request(uint64_t addr, uint16_t nb_blocks) "addr=0x%" PRIx64 " nb_blocks=%" PRIu16
+virtio_mem_unplugged_all(void) ""
+virtio_mem_unplug_all_request(void) ""
+virtio_mem_resized_usable_region(uint64_t old_size, uint64_t new_size) "old_size=0x%" PRIx64 "new_size=0x%" PRIx64
+virtio_mem_state_request(uint64_t addr, uint16_t nb_blocks) "addr=0x%" PRIx64 " nb_blocks=%" PRIu16
+virtio_mem_state_response(uint16_t state) "state=%" PRIu16
Signature · 1 line
MST
#28 lore raw

[PULL v2 35/41] vhost: introduce new VhostOps vhost_vq_get_addr

In reply to Michael S. Tsirkin

From: Cindy Lu <[email protected]>

This patch introduces new VhostOps vhost_vq_get_addr_op callback to get the vring addr from the backend

Message-Id: <[email protected]>

---
 include/hw/virtio/vhost-backend.h | 6 ++++++
 1 file changed, 6 insertions(+)
include/hw/virtio/vhost-backend.h +6 -0
index b80f344cd6..fa84abac97 100644
@@ -34,6 +34,7 @@ struct vhost_vring_state;
 struct vhost_vring_addr;
 struct vhost_scsi_target;
 struct vhost_iotlb_msg;
+struct vhost_virtqueue;
 
 typedef int (*vhost_backend_init)(struct vhost_dev *dev, void *opaque);
 typedef int (*vhost_backend_cleanup)(struct vhost_dev *dev);
@@ -113,6 +114,10 @@ typedef int (*vhost_set_inflight_fd_op)(struct vhost_dev *dev,
                                         struct vhost_inflight *inflight);
 
 typedef int (*vhost_dev_start_op)(struct vhost_dev *dev, bool started);
+
+typedef int (*vhost_vq_get_addr_op)(struct vhost_dev *dev,
+                    struct vhost_vring_addr *addr,
+                    struct vhost_virtqueue *vq);
 typedef struct VhostOps {
     VhostBackendType backend_type;
     vhost_backend_init vhost_backend_init;
@@ -154,6 +159,7 @@ typedef struct VhostOps {
     vhost_get_inflight_fd_op vhost_get_inflight_fd;
     vhost_set_inflight_fd_op vhost_set_inflight_fd;
     vhost_dev_start_op vhost_dev_start;
+    vhost_vq_get_addr_op  vhost_vq_get_addr;
 } VhostOps;
 
 extern const VhostOps user_ops;
Signature · 1 line
MST
#29 lore raw

[PULL v2 16/41] hmp: Handle virtio-mem when printing memory device info

In reply to Michael S. Tsirkin

From: David Hildenbrand <[email protected]>

Print the memory device info just like for other memory devices.

Cc "Dr. David Alan Gilbert" <[email protected]>
Cc "Michael S. Tsirkin" <[email protected]>

Message-Id: <[email protected]>

---
 monitor/hmp-cmds.c | 16 ++++++++++++++++
 1 file changed, 16 insertions(+)
monitor/hmp-cmds.c +16 -0
index 2b0b58a336..2ec13e4cc3 100644
@@ -1821,6 +1821,7 @@ void hmp_info_memory_devices(Monitor *mon, const QDict *qdict)
     MemoryDeviceInfoList *info_list = qmp_query_memory_devices(&err);
     MemoryDeviceInfoList *info;
     VirtioPMEMDeviceInfo *vpi;
+    VirtioMEMDeviceInfo *vmi;
     MemoryDeviceInfo *value;
     PCDIMMDeviceInfo *di;
 
@@ -1855,6 +1856,21 @@ void hmp_info_memory_devices(Monitor *mon, const QDict *qdict)
                 monitor_printf(mon, "  size: %" PRIu64 "\n", vpi->size);
                 monitor_printf(mon, "  memdev: %s\n", vpi->memdev);
                 break;
+            case MEMORY_DEVICE_INFO_KIND_VIRTIO_MEM:
+                vmi = value->u.virtio_mem.data;
+                monitor_printf(mon, "Memory device [%s]: \"%s\"\n",
+                               MemoryDeviceInfoKind_str(value->type),
+                               vmi->id ? vmi->id : "");
+                monitor_printf(mon, "  memaddr: 0x%" PRIx64 "\n", vmi->memaddr);
+                monitor_printf(mon, "  node: %" PRId64 "\n", vmi->node);
+                monitor_printf(mon, "  requested-size: %" PRIu64 "\n",
+                               vmi->requested_size);
+                monitor_printf(mon, "  size: %" PRIu64 "\n", vmi->size);
+                monitor_printf(mon, "  max-size: %" PRIu64 "\n", vmi->max_size);
+                monitor_printf(mon, "  block-size: %" PRIu64 "\n",
+                               vmi->block_size);
+                monitor_printf(mon, "  memdev: %s\n", vmi->memdev);
+                break;
             default:
                 g_assert_not_reached();
             }
Signature · 1 line
MST
#30 lore raw

[PULL v2 38/41] vhost: implement vhost_force_iommu method

In reply to Michael S. Tsirkin

From: Cindy Lu <[email protected]>

use the vhost_force_iommu callback to force enable feature bit VIRTIO_F_IOMMU_PLATFORM

Message-Id: <[email protected]>

---
 hw/virtio/vhost.c | 5 +++++
 1 file changed, 5 insertions(+)
hw/virtio/vhost.c +5 -0
index 1e083a8976..1a1384e7a6 100644
@@ -810,6 +810,11 @@ static int vhost_dev_set_features(struct vhost_dev *dev,
     if (!vhost_dev_has_iommu(dev)) {
         features &= ~(0x1ULL << VIRTIO_F_IOMMU_PLATFORM);
     }
+    if (dev->vhost_ops->vhost_force_iommu) {
+        if (dev->vhost_ops->vhost_force_iommu(dev) == true) {
+            features |= 0x1ULL << VIRTIO_F_IOMMU_PLATFORM;
+       }
+    }
     r = dev->vhost_ops->vhost_set_features(dev, features);
     if (r < 0) {
         VHOST_OPS_DEBUG("vhost_set_features failed");
Signature · 1 line
MST
#31 lore raw

[PULL v2 23/41] virtio-mem: Exclude unplugged memory during migration

In reply to Michael S. Tsirkin

From: David Hildenbrand <[email protected]>

The content of unplugged memory is undefined and should not be migrated, ever. Exclude all unplugged memory during precopy using the precopy notifier infrastructure introduced for free page hinting in virtio-balloon.

Unplugged memory is marked as "not dirty", meaning it won't be considered for migration.

Cc "Michael S. Tsirkin" <[email protected]>
Cc "Dr. David Alan Gilbert" <[email protected]>

Message-Id: <[email protected]>

---
 include/hw/virtio/virtio-mem.h |  3 ++
 hw/virtio/virtio-mem.c         | 54 +++++++++++++++++++++++++++++++++-
 2 files changed, 56 insertions(+), 1 deletion(-)
include/hw/virtio/virtio-mem.h +3 -0
index b74c77cd42..0778224964 100644
@@ -67,6 +67,9 @@ typedef struct VirtIOMEM {
 
     /* notifiers to notify when "size" changes */
     NotifierList size_change_notifiers;
+
+    /* don't migrate unplugged memory */
+    NotifierWithReturn precopy_notifier;
 } VirtIOMEM;
 
 typedef struct VirtIOMEMClass {
hw/virtio/virtio-mem.c +53 -1

85 lines not on this page · raw message

Signature · 1 line
MST
#32 lore raw

[PULL v2 37/41] vhost: introduce new VhostOps vhost_force_iommu

In reply to Michael S. Tsirkin

From: Cindy Lu <[email protected]>

This patch introduces new VhostOps vhost_force_iommu callback to force enable features bit VIRTIO_F_IOMMU_PLATFORM.

Message-Id: <[email protected]>

---
 include/hw/virtio/vhost-backend.h | 3 +++
 1 file changed, 3 insertions(+)
include/hw/virtio/vhost-backend.h +3 -0
index bfc24207e2..e7cb8d028c 100644
@@ -121,6 +121,8 @@ typedef int (*vhost_vq_get_addr_op)(struct vhost_dev *dev,
 
 typedef int (*vhost_get_device_id_op)(struct vhost_dev *dev, uint32_t *dev_id);
 
+typedef bool (*vhost_force_iommu_op)(struct vhost_dev *dev);
+
 typedef struct VhostOps {
     VhostBackendType backend_type;
     vhost_backend_init vhost_backend_init;
@@ -164,6 +166,7 @@ typedef struct VhostOps {
     vhost_dev_start_op vhost_dev_start;
     vhost_vq_get_addr_op  vhost_vq_get_addr;
     vhost_get_device_id_op vhost_get_device_id;
+    vhost_force_iommu_op vhost_force_iommu;
 } VhostOps;
 
 extern const VhostOps user_ops;
Signature · 1 line
MST
#33 lore raw

[PULL v2 18/41] pc: Support for virtio-mem-pci

In reply to Michael S. Tsirkin

From: David Hildenbrand <[email protected]>

Let's wire it up similar to virtio-pmem. Also disallow unplug, so it's harder for users to shoot themselves into the foot.

Cc "Michael S. Tsirkin" <[email protected]>
Cc Marcel Apfelbaum <[email protected]>
Cc Paolo Bonzini <[email protected]>
Cc Richard Henderson <[email protected]>
Cc Eduardo Habkost <[email protected]>
Cc Eric Blake <[email protected]>
Cc Markus Armbruster <[email protected]>

Message-Id: <[email protected]>

---
 hw/i386/pc.c    | 49 ++++++++++++++++++++++++++++---------------------
 hw/i386/Kconfig |  1 +
 2 files changed, 29 insertions(+), 21 deletions(-)
hw/i386/pc.c +28 -21
index 58b1425c17..576f2502f9 100644
@@ -88,6 +88,7 @@
 #include "hw/net/ne2000-isa.h"
 #include "standard-headers/asm-x86/bootparam.h"
 #include "hw/virtio/virtio-pmem-pci.h"
+#include "hw/virtio/virtio-mem-pci.h"
 #include "hw/mem/memory-device.h"
 #include "sysemu/replay.h"
 #include "qapi/qmp/qerror.h"
@@ -1637,8 +1638,8 @@ static void pc_cpu_pre_plug(HotplugHandler *hotplug_dev,
     numa_cpu_pre_plug(cpu_slot, dev, errp);
 }
 
-static void pc_virtio_pmem_pci_pre_plug(HotplugHandler *hotplug_dev,
-                                        DeviceState *dev, Error **errp)
+static void pc_virtio_md_pci_pre_plug(HotplugHandler *hotplug_dev,
+                                      DeviceState *dev, Error **errp)
 {
     HotplugHandler *hotplug_dev2 = qdev_get_bus_hotplug_handler(dev);
     Error *local_err = NULL;
@@ -1649,7 +1650,8 @@ static void pc_virtio_pmem_pci_pre_plug(HotplugHandler *hotplug_dev,
          * order. We should never reach this point when hotplugging on x86,
          * however, better add a safety net.
          */
-        error_setg(errp, "virtio-pmem-pci hotplug not supported on this bus.");
+        error_setg(errp, "hotplug of virtio based memory devices not supported"
+                   " on this bus.");
         return;
     }
     /*
@@ -1664,8 +1666,8 @@ static void pc_virtio_pmem_pci_pre_plug(HotplugHandler *hotplug_dev,
     error_propagate(errp, local_err);
 }
 
-static void pc_virtio_pmem_pci_plug(HotplugHandler *hotplug_dev,
-                                    DeviceState *dev, Error **errp)
+static void pc_virtio_md_pci_plug(HotplugHandler *hotplug_dev,
+                                  DeviceState *dev, Error **errp)
 {
     HotplugHandler *hotplug_dev2 = qdev_get_bus_hotplug_handler(dev);
     Error *local_err = NULL;
@@ -1685,17 +1687,17 @@ static void pc_virtio_pmem_pci_plug(HotplugHandler *hotplug_dev,
     error_propagate(errp, local_err);
 }
 
-static void pc_virtio_pmem_pci_unplug_request(HotplugHandler *hotplug_dev,
-                                              DeviceState *dev, Error **errp)
+static void pc_virtio_md_pci_unplug_request(HotplugHandler *hotplug_dev,
+                                            DeviceState *dev, Error **errp)
 {
-    /* We don't support virtio pmem hot unplug */
-    error_setg(errp, "virtio pmem device unplug not supported.");
+    /* We don't support hot unplug of virtio based memory devices */
+    error_setg(errp, "virtio based memory devices cannot be unplugged.");
 }
 
-static void pc_virtio_pmem_pci_unplug(HotplugHandler *hotplug_dev,
-                                      DeviceState *dev, Error **errp)
+static void pc_virtio_md_pci_unplug(HotplugHandler *hotplug_dev,
+                                    DeviceState *dev, Error **errp)
 {
-    /* We don't support virtio pmem hot unplug */
+    /* We don't support hot unplug of virtio based memory devices */
 }
 
 static void pc_machine_device_pre_plug_cb(HotplugHandler *hotplug_dev,
@@ -1705,8 +1707,9 @@ static void pc_machine_device_pre_plug_cb(HotplugHandler *hotplug_dev,
         pc_memory_pre_plug(hotplug_dev, dev, errp);
     } else if (object_dynamic_cast(OBJECT(dev), TYPE_CPU)) {
         pc_cpu_pre_plug(hotplug_dev, dev, errp);
-    } else if (object_dynamic_cast(OBJECT(dev), TYPE_VIRTIO_PMEM_PCI)) {
-        pc_virtio_pmem_pci_pre_plug(hotplug_dev, dev, errp);
+    } else if (object_dynamic_cast(OBJECT(dev), TYPE_VIRTIO_PMEM_PCI) ||
+               object_dynamic_cast(OBJECT(dev), TYPE_VIRTIO_MEM_PCI)) {
+        pc_virtio_md_pci_pre_plug(hotplug_dev, dev, errp);
     }
 }
 
@@ -1717,8 +1720,9 @@ static void pc_machine_device_plug_cb(HotplugHandler *hotplug_dev,
         pc_memory_plug(hotplug_dev, dev, errp);
     } else if (object_dynamic_cast(OBJECT(dev), TYPE_CPU)) {
         pc_cpu_plug(hotplug_dev, dev, errp);
-    } else if (object_dynamic_cast(OBJECT(dev), TYPE_VIRTIO_PMEM_PCI)) {
-        pc_virtio_pmem_pci_plug(hotplug_dev, dev, errp);
+    } else if (object_dynamic_cast(OBJECT(dev), TYPE_VIRTIO_PMEM_PCI) ||
+               object_dynamic_cast(OBJECT(dev), TYPE_VIRTIO_MEM_PCI)) {
+        pc_virtio_md_pci_plug(hotplug_dev, dev, errp);
     }
 }
 
@@ -1729,8 +1733,9 @@ static void pc_machine_device_unplug_request_cb(HotplugHandler *hotplug_dev,
         pc_memory_unplug_request(hotplug_dev, dev, errp);
     } else if (object_dynamic_cast(OBJECT(dev), TYPE_CPU)) {
         pc_cpu_unplug_request_cb(hotplug_dev, dev, errp);
-    } else if (object_dynamic_cast(OBJECT(dev), TYPE_VIRTIO_PMEM_PCI)) {
-        pc_virtio_pmem_pci_unplug_request(hotplug_dev, dev, errp);
+    } else if (object_dynamic_cast(OBJECT(dev), TYPE_VIRTIO_PMEM_PCI) ||
+               object_dynamic_cast(OBJECT(dev), TYPE_VIRTIO_MEM_PCI)) {
+        pc_virtio_md_pci_unplug_request(hotplug_dev, dev, errp);
     } else {
         error_setg(errp, "acpi: device unplug request for not supported device"
                    " type: %s", object_get_typename(OBJECT(dev)));
@@ -1744,8 +1749,9 @@ static void pc_machine_device_unplug_cb(HotplugHandler *hotplug_dev,
         pc_memory_unplug(hotplug_dev, dev, errp);
     } else if (object_dynamic_cast(OBJECT(dev), TYPE_CPU)) {
         pc_cpu_unplug_cb(hotplug_dev, dev, errp);
-    } else if (object_dynamic_cast(OBJECT(dev), TYPE_VIRTIO_PMEM_PCI)) {
-        pc_virtio_pmem_pci_unplug(hotplug_dev, dev, errp);
+    } else if (object_dynamic_cast(OBJECT(dev), TYPE_VIRTIO_PMEM_PCI) ||
+               object_dynamic_cast(OBJECT(dev), TYPE_VIRTIO_MEM_PCI)) {
+        pc_virtio_md_pci_unplug(hotplug_dev, dev, errp);
     } else {
         error_setg(errp, "acpi: device unplug for not supported device"
                    " type: %s", object_get_typename(OBJECT(dev)));
@@ -1757,7 +1763,8 @@ static HotplugHandler *pc_get_hotplug_handler(MachineState *machine,
 {
     if (object_dynamic_cast(OBJECT(dev), TYPE_PC_DIMM) ||
         object_dynamic_cast(OBJECT(dev), TYPE_CPU) ||
-        object_dynamic_cast(OBJECT(dev), TYPE_VIRTIO_PMEM_PCI)) {
+        object_dynamic_cast(OBJECT(dev), TYPE_VIRTIO_PMEM_PCI) ||
+        object_dynamic_cast(OBJECT(dev), TYPE_VIRTIO_MEM_PCI)) {
         return HOTPLUG_HANDLER(machine);
     }
 
hw/i386/Kconfig +1 -0
index c93f32f657..03e347b207 100644
@@ -35,6 +35,7 @@ config PC
     select ACPI_PCI
     select ACPI_VMGENID
     select VIRTIO_PMEM_SUPPORTED
+    select VIRTIO_MEM_SUPPORTED
 
 config PC_PCI
     bool
Signature · 1 line
MST
#34 lore raw

[PULL v2 28/41] net: introduce qemu_get_peer

In reply to Michael S. Tsirkin

From: Cindy Lu <[email protected]>

This is a small function that can get the peer from given NetClientState and queue_index

Message-Id: <[email protected]>

---
 include/net/net.h | 1 +
 net/net.c         | 7 +++++++
 2 files changed, 8 insertions(+)
include/net/net.h +1 -0
index 39085d9444..e7ef42d62b 100644
@@ -176,6 +176,7 @@ void hmp_info_network(Monitor *mon, const QDict *qdict);
 void net_socket_rs_init(SocketReadState *rs,
                         SocketReadStateFinalize *finalize,
                         bool vnet_hdr);
+NetClientState *qemu_get_peer(NetClientState *nc, int queue_index);
 
 /* NIC info */
 
net/net.c +7 -0
index d1130296e1..9099a327dd 100644
@@ -325,6 +325,13 @@ void *qemu_get_nic_opaque(NetClientState *nc)
     return nic->opaque;
 }
 
+NetClientState *qemu_get_peer(NetClientState *nc, int queue_index)
+{
+    assert(nc != NULL);
+    NetClientState *ncs = nc + queue_index;
+    return ncs->peer;
+}
+
 static void qemu_cleanup_net_client(NetClientState *nc)
 {
     QTAILQ_REMOVE(&net_clients, nc, next);
Signature · 1 line
MST
#35 lore raw

[PULL v2 30/41] virtio-bus: introduce queue_enabled method

In reply to Michael S. Tsirkin

From: Jason Wang <[email protected]>

This patch introduces queue_enabled() method which allows the transport to implement its own way to report whether or not a queue is enabled.

Message-Id: <[email protected]>

---
 include/hw/virtio/virtio-bus.h | 4 ++++
 hw/virtio/virtio.c             | 6 ++++++
 2 files changed, 10 insertions(+)
include/hw/virtio/virtio-bus.h +4 -0
index 38c9399cd4..0f6f215925 100644
@@ -83,6 +83,10 @@ typedef struct VirtioBusClass {
      */
     int (*ioeventfd_assign)(DeviceState *d, EventNotifier *notifier,
                             int n, bool assign);
+    /*
+     * Whether queue number n is enabled.
+     */
+    bool (*queue_enabled)(DeviceState *d, int n);
     /*
      * Does the transport have variable vring alignment?
      * (ie can it ever call virtio_queue_set_align()?)
hw/virtio/virtio.c +6 -0
index cc9c9dc162..5bd2a2f621 100644
@@ -3286,6 +3286,12 @@ hwaddr virtio_queue_get_desc_addr(VirtIODevice *vdev, int n)
 
 bool virtio_queue_enabled(VirtIODevice *vdev, int n)
 {
+    BusState *qbus = qdev_get_parent_bus(DEVICE(vdev));
+    VirtioBusClass *k = VIRTIO_BUS_GET_CLASS(qbus);
+
+    if (k->queue_enabled) {
+        return k->queue_enabled(qbus->parent, n);
+    }
     return virtio_queue_get_desc_addr(vdev, n) != 0;
 }
 
Signature · 1 line
MST
#36 lore raw

[PULL v2 40/41] vhost-vdpa: introduce vhost-vdpa backend

In reply to Michael S. Tsirkin

From: Cindy Lu <[email protected]>

Currently we have 2 types of vhost backends in QEMU: vhost kernel and vhost-user. The above patch provides a generic device for vDPA purpose, this vDPA device exposes to user space a non-vendor-specific configuration interface for setting up a vhost HW accelerator, this patch set introduces a third vhost backend called vhost-vdpa based on the vDPA interface.

Vhost-vdpa usage:

qemu-system-x86_64 -cpu host -enable-kvm \
    ......
    -netdev type=vhost-vdpa,vhostdev=/dev/vhost-vdpa-id,id=vhost-vdpa0 \
    -device virtio-net-pci,netdev=vhost-vdpa0,page-per-vq=on \

Message-Id: <[email protected]>

---
 configure                         |  21 ++
 include/hw/virtio/vhost-backend.h |   4 +-
 include/hw/virtio/vhost-vdpa.h    |  26 ++
 include/hw/virtio/vhost.h         |   7 +
 hw/net/vhost_net.c                |  18 +-
 hw/net/virtio-net.c               |  19 ++
 hw/virtio/vhost-backend.c         |   6 +
 hw/virtio/vhost-vdpa.c            | 475 ++++++++++++++++++++++++++++++
 docs/interop/index.rst            |   1 +
 docs/interop/vhost-vdpa.rst       |  17 ++
 hw/virtio/Makefile.objs           |   1 +
 qemu-options.hx                   |  12 +
 12 files changed, 600 insertions(+), 7 deletions(-)
 create mode 100644 include/hw/virtio/vhost-vdpa.h
 create mode 100644 hw/virtio/vhost-vdpa.c
 create mode 100644 docs/interop/vhost-vdpa.rst
configure +21 -0
index 4a22dcd563..3db7f20185 100755
@@ -1575,6 +1575,10 @@ for opt do
   ;;
   --enable-vhost-user) vhost_user="yes"
   ;;
+  --disable-vhost-vdpa) vhost_vdpa="no"
+  ;;
+  --enable-vhost-vdpa) vhost_vdpa="yes"
+  ;;
   --disable-vhost-kernel) vhost_kernel="no"
   ;;
   --enable-vhost-kernel) vhost_kernel="yes"
@@ -1883,6 +1887,7 @@ disabled with --disable-FEATURE, default is enabled if available:
   vhost-crypto    vhost-user-crypto backend support
   vhost-kernel    vhost kernel backend support
   vhost-user      vhost-user backend support
+  vhost-vdpa      vhost-vdpa kernel backend support
   spice           spice
   rbd             rados block device (rbd)
   libiscsi        iscsi support
@@ -2394,6 +2399,10 @@ test "$vhost_user" = "" && vhost_user=yes
 if test "$vhost_user" = "yes" && test "$mingw32" = "yes"; then
   error_exit "vhost-user isn't available on win32"
 fi
+test "$vhost_vdpa" = "" && vhost_vdpa=$linux
+if test "$vhost_vdpa" = "yes" && test "$linux" != "yes"; then
+  error_exit "vhost-vdpa is only available on Linux"
+fi
 test "$vhost_kernel" = "" && vhost_kernel=$linux
 if test "$vhost_kernel" = "yes" && test "$linux" != "yes"; then
   error_exit "vhost-kernel is only available on Linux"
@@ -2422,6 +2431,11 @@ test "$vhost_user_fs" = "" && vhost_user_fs=$vhost_user
 if test "$vhost_user_fs" = "yes" && test "$vhost_user" = "no"; then
   error_exit "--enable-vhost-user-fs requires --enable-vhost-user"
 fi
+#vhost-vdpa backends
+test "$vhost_net_vdpa" = "" && vhost_net_vdpa=$vhost_vdpa
+if test "$vhost_net_vdpa" = "yes" && test "$vhost_vdpa" = "no"; then
+  error_exit "--enable-vhost-net-vdpa requires --enable-vhost-vdpa"
+fi
 
 # OR the vhost-kernel and vhost-user values for simplicity
 if test "$vhost_net" = ""; then
@@ -6936,6 +6950,7 @@ echo "vhost-scsi support $vhost_scsi"
 echo "vhost-vsock support $vhost_vsock"
 echo "vhost-user support $vhost_user"
 echo "vhost-user-fs support $vhost_user_fs"
+echo "vhost-vdpa support $vhost_vdpa"
 echo "Trace backends    $trace_backends"
 if have_backend "simple"; then
 echo "Trace output file $trace_file-<pid>"
@@ -7437,6 +7452,9 @@ fi
 if test "$vhost_net_user" = "yes" ; then
   echo "CONFIG_VHOST_NET_USER=y" >> $config_host_mak
 fi
+if test "$vhost_net_vdpa" = "yes" ; then
+  echo "CONFIG_VHOST_NET_VDPA=y" >> $config_host_mak
+fi
 if test "$vhost_crypto" = "yes" ; then
   echo "CONFIG_VHOST_CRYPTO=y" >> $config_host_mak
 fi
@@ -7452,6 +7470,9 @@ fi
 if test "$vhost_user" = "yes" ; then
   echo "CONFIG_VHOST_USER=y" >> $config_host_mak
 fi
+if test "$vhost_vdpa" = "yes" ; then
+  echo "CONFIG_VHOST_VDPA=y" >> $config_host_mak
+fi
 if test "$vhost_user_fs" = "yes" ; then
   echo "CONFIG_VHOST_USER_FS=y" >> $config_host_mak
 fi
include/hw/virtio/vhost-backend.h +3 -1
index e7cb8d028c..8825bd278f 100644
@@ -17,7 +17,8 @@ typedef enum VhostBackendType {
     VHOST_BACKEND_TYPE_NONE = 0,
     VHOST_BACKEND_TYPE_KERNEL = 1,
     VHOST_BACKEND_TYPE_USER = 2,
-    VHOST_BACKEND_TYPE_MAX = 3,
+    VHOST_BACKEND_TYPE_VDPA = 3,
+    VHOST_BACKEND_TYPE_MAX = 4,
 } VhostBackendType;
 
 typedef enum VhostSetConfigType {
@@ -170,6 +171,7 @@ typedef struct VhostOps {
 } VhostOps;
 
 extern const VhostOps user_ops;
+extern const VhostOps vdpa_ops;
 
 int vhost_set_backend_type(struct vhost_dev *dev,
                            VhostBackendType backend_type);
include/hw/virtio/vhost-vdpa.h new file +26 -0
new file mode 100644
index 0000000000..6455663388
@@ -0,0 +1,26 @@
+/*
+ * vhost-vdpa.h
+ *
+ * Copyright(c) 2017-2018 Intel Corporation.
+ * Copyright(c) 2020 Red Hat, Inc.
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ *
+ */
+
+#ifndef HW_VIRTIO_VHOST_VDPA_H
+#define HW_VIRTIO_VHOST_VDPA_H
+
+#include "hw/virtio/virtio.h"
+
+typedef struct vhost_vdpa {
+    int device_fd;
+    uint32_t msg_type;
+    MemoryListener listener;
+} VhostVDPA;
+
+extern AddressSpace address_space_memory;
+extern int vhost_vdpa_get_device_id(struct vhost_dev *dev,
+                                   uint32_t *device_id);
+#endif
include/hw/virtio/vhost.h +7 -0
index 085450c6f8..767a95ec0b 100644
@@ -92,6 +92,13 @@ struct vhost_dev {
     const VhostDevConfigOps *config_ops;
 };
 
+struct vhost_net {
+    struct vhost_dev dev;
+    struct vhost_virtqueue vqs[2];
+    int backend;
+    NetClientState *nc;
+};
+
 int vhost_dev_init(struct vhost_dev *hdev, void *opaque,
                    VhostBackendType backend_type,
                    uint32_t busyloop_timeout);
hw/net/vhost_net.c +12 -6
index 4561665f6b..24d555e764 100644
@@ -17,6 +17,7 @@
 #include "net/net.h"
 #include "net/tap.h"
 #include "net/vhost-user.h"
+#include "net/vhost-vdpa.h"
 
 #include "standard-headers/linux/vhost_types.h"
 #include "hw/virtio/virtio-net.h"
@@ -33,12 +34,6 @@
 #include "hw/virtio/vhost.h"
 #include "hw/virtio/virtio-bus.h"
 
-struct vhost_net {
-    struct vhost_dev dev;
-    struct vhost_virtqueue vqs[2];
-    int backend;
-    NetClientState *nc;
-};
 
 /* Features supported by host kernel. */
 static const int kernel_feature_bits[] = {
@@ -96,6 +91,11 @@ static const int *vhost_net_get_feature_bits(struct vhost_net *net)
     case NET_CLIENT_DRIVER_VHOST_USER:
         feature_bits = user_feature_bits;
         break;
+#ifdef CONFIG_VHOST_NET_VDPA
+    case NET_CLIENT_DRIVER_VHOST_VDPA:
+        feature_bits = vdpa_feature_bits;
+        break;
+#endif
     default:
         error_report("Feature bits not defined for this type: %d",
                 net->nc->info->type);
@@ -443,6 +443,12 @@ VHostNetState *get_vhost_net(NetClientState *nc)
         vhost_net = vhost_user_get_vhost_net(nc);
         assert(vhost_net);
         break;
+#endif
+#ifdef CONFIG_VHOST_NET_VDPA
+    case NET_CLIENT_DRIVER_VHOST_VDPA:
+        vhost_net = vhost_vdpa_get_vhost_net(nc);
+        assert(vhost_net);
+        break;
 #endif
     default:
         break;
hw/net/virtio-net.c +19 -0
index aff67a92df..0a54b1910a 100644
@@ -43,6 +43,7 @@
 #include "monitor/qdev.h"
 #include "hw/pci/pci.h"
 #include "net_rx_pkt.h"
+#include "hw/virtio/vhost.h"
 
 #define VIRTIO_NET_VM_VERSION    11
 
@@ -125,6 +126,8 @@ static void virtio_net_get_config(VirtIODevice *vdev, uint8_t *config)
     VirtIONet *n = VIRTIO_NET(vdev);
     struct virtio_net_config netcfg;
 
+    int ret = 0;
+    memset(&netcfg, 0 , sizeof(struct virtio_net_config));
     virtio_stw_p(vdev, &netcfg.status, n->status);
     virtio_stw_p(vdev, &netcfg.max_virtqueue_pairs, n->max_queues);
     virtio_stw_p(vdev, &netcfg.mtu, n->net_conf.mtu);
@@ -138,6 +141,15 @@ static void virtio_net_get_config(VirtIODevice *vdev, uint8_t *config)
     virtio_stl_p(vdev, &netcfg.supported_hash_types,
                  VIRTIO_NET_RSS_SUPPORTED_HASHES);
     memcpy(config, &netcfg, n->config_size);
+
+    NetClientState *nc = qemu_get_queue(n->nic);
+    if (nc->peer->info->type == NET_CLIENT_DRIVER_VHOST_VDPA) {
+        ret = vhost_net_get_config(get_vhost_net(nc->peer), (uint8_t *)&netcfg,
+                             n->config_size);
+    if (ret != -1) {
+        memcpy(config, &netcfg, n->config_size);
+    }
+    }
 }
 
 static void virtio_net_set_config(VirtIODevice *vdev, const uint8_t *config)
@@ -153,6 +165,13 @@ static void virtio_net_set_config(VirtIODevice *vdev, const uint8_t *config)
         memcpy(n->mac, netcfg.mac, ETH_ALEN);
         qemu_format_nic_info_str(qemu_get_queue(n->nic), n->mac);
     }
+
+    NetClientState *nc = qemu_get_queue(n->nic);
+    if (nc->peer->info->type == NET_CLIENT_DRIVER_VHOST_VDPA) {
+        vhost_net_set_config(get_vhost_net(nc->peer), (uint8_t *)&netcfg,
+                               0, n->config_size,
+                        VHOST_SET_CONFIG_TYPE_MASTER);
+      }
 }
 
 static bool virtio_net_started(VirtIONet *n, uint8_t status)
hw/virtio/vhost-backend.c +6 -0
index 48905383f8..782b1d67d9 100644
@@ -15,6 +15,7 @@
 #include "qemu/main-loop.h"
 #include "standard-headers/linux/vhost_types.h"
 
+#include "hw/virtio/vhost-vdpa.h"
 #ifdef CONFIG_VHOST_KERNEL
 #include <linux/vhost.h>
 #include <sys/ioctl.h>
@@ -285,6 +286,11 @@ int vhost_set_backend_type(struct vhost_dev *dev, VhostBackendType backend_type)
     case VHOST_BACKEND_TYPE_USER:
         dev->vhost_ops = &user_ops;
         break;
+#endif
+#ifdef CONFIG_VHOST_VDPA
+    case VHOST_BACKEND_TYPE_VDPA:
+        dev->vhost_ops = &vdpa_ops;
+        break;
 #endif
     default:
         error_report("Unknown vhost backend type");
hw/virtio/vhost-vdpa.c new file +475 -0

475 lines not on this page · raw message

docs/interop/index.rst +1 -0
index 049387ac6d..006f986420 100644
@@ -20,3 +20,4 @@ Contents:
    qemu-ga
    vhost-user
    vhost-user-gpu
+   vhost-vdpa
docs/interop/vhost-vdpa.rst new file +17 -0
new file mode 100644
index 0000000000..0c70ba01bc
@@ -0,0 +1,17 @@
+=====================
+Vhost-vdpa Protocol
+=====================
+
+Introduction
+=============
+vDPA(Virtual data path acceleration) device is a device that uses
+a datapath which complies with the virtio specifications with vendor
+specific control path. vDPA devices can be both physically located on
+the hardware or emulated by software.
+
+This document describes the vDPA support in qemu
+
+Here is the kernel commit here
+https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=4c8cf31885f69e86be0b5b9e6677a26797365e1d
+
+TODO : More information will add later
hw/virtio/Makefile.objs +1 -0
index a986708186..fc91719b4a 100644
@@ -5,6 +5,7 @@ obj-y += virtio.o
 obj-$(CONFIG_VHOST) += vhost.o vhost-backend.o
 common-obj-$(call lnot,$(CONFIG_VHOST)) += vhost-stub.o
 obj-$(CONFIG_VHOST_USER) += vhost-user.o
+obj-$(CONFIG_VHOST_VDPA) += vhost-vdpa.o
 
 common-obj-$(CONFIG_VIRTIO_RNG) += virtio-rng.o
 common-obj-$(CONFIG_VIRTIO_PCI) += virtio-pci.o
qemu-options.hx +12 -0
index 196f468786..fa1b19de4c 100644
@@ -2418,6 +2418,10 @@ DEF("netdev", HAS_ARG, QEMU_OPTION_netdev,
 #ifdef CONFIG_POSIX
     "-netdev vhost-user,id=str,chardev=dev[,vhostforce=on|off]\n"
     "                configure a vhost-user network, backed by a chardev 'dev'\n"
+#endif
+#ifdef __linux__
+    "-netdev vhost-vdpa,id=str,vhostdev=/path/to/dev\n"
+    "                configure a vhost-vdpa network,Establish a vhost-vdpa netdev\n"
 #endif
     "-netdev hubport,id=str,hubid=n[,netdev=nd]\n"
     "                configure a hub port on the hub with ID 'n'\n", QEMU_ARCH_ALL)
@@ -2897,6 +2901,14 @@ SRST
              -netdev type=vhost-user,id=net0,chardev=chr0 \
              -device virtio-net-pci,netdev=net0
 
+``-netdev vhost-vdpa,vhostdev=/path/to/dev``
+    Establish a vhost-vdpa netdev.
+
+    vDPA device is a device that uses a datapath which complies with
+    the virtio specifications with a vendor specific control path.
+    vDPA devices can be both physically located on the hardware or
+    emulated by software.
+
 ``-netdev hubport,id=id,hubid=hubid[,netdev=nd]``
     Create a hub port on the emulated hub with ID hubid.
 
Signature · 1 line
MST
#37 lore raw

Re: [PULL v2 40/41] vhost-vdpa: introduce vhost-vdpa backend

Quoted 9 lines · Michael S. Tsirkin

From: Cindy Lu <[email protected]>

Currently we have 2 types of vhost backends in QEMU: vhost kernel and vhost-user. The above patch provides a generic device for vDPA purpose, this vDPA device exposes to user space a non-vendor-specific configuration interface for setting up a vhost HW accelerator, this patch set introduces a third vhost backend called vhost-vdpa based on the vDPA interface.

Quoted 6 lines

Vhost-vdpa usage:

qemu-system-x86_64 -cpu host -enable-kvm \
    ......
    -netdev type=vhost-vdpa,vhostdev=/dev/vhost-vdpa-id,id=vhost-vdpa0 \
    -device virtio-net-pci,netdev=vhost-vdpa0,page-per-vq=on \

Hi; Coverity reports some issues with this change:

Quoted 5 lines
hw/virtio/vhost-vdpa.c new file +5 -0
+static int vhost_vdpa_dma_map(struct vhost_vdpa *v, hwaddr iova, hwaddr size,
+                              void *vaddr, bool readonly)
+{
+    struct vhost_msg_v2 msg;
+    int fd = v->device_fd;

hw/virtio/vhost-vdpa.c line 52

Quoted 10 lines
hw/virtio/vhost-vdpa.c new file +10 -0
+    int ret = 0;
+
+    msg.type = v->msg_type;
+    msg.iotlb.iova = iova;
+    msg.iotlb.size = size;
+    msg.iotlb.uaddr = (uint64_t)vaddr;
+    msg.iotlb.perm = readonly ? VHOST_ACCESS_RO : VHOST_ACCESS_RW;
+    msg.iotlb.type = VHOST_IOTLB_UPDATE;
+
+    if (write(fd, &msg, sizeof(msg)) != sizeof(msg)) {

Here we write the contents of the struct down the pipe, but we have not initialized all its fields; specifically, msg.reserved is not initialized and so those bytes will be random. We'll also transfer random contents from the stack in the padding, potentially.

This is CID 1420267.

The easy fix is to zero-initialize the whole struct at the start:

   struct vhost_msg_v2 msg = {};
Quoted 15 lines
hw/virtio/vhost-vdpa.c new file +15 -0
+        error_report("failed to write, fd=%d, errno=%d (%s)",
+            fd, errno, strerror(errno));
+        return -EIO ;
+    }
+
+    return ret;
+}
+
+static int vhost_vdpa_dma_unmap(struct vhost_vdpa *v, hwaddr iova,
+                                hwaddr size)
+{
+    struct vhost_msg_v2 msg;
+    int fd = v->device_fd;
+    int ret = 0;
+

hw/virtio/vhost-vdpa.c line 77

Quoted 10 lines
hw/virtio/vhost-vdpa.c new file +10 -0
+    msg.type = v->msg_type;
+    msg.iotlb.iova = iova;
+    msg.iotlb.size = size;
+    msg.iotlb.type = VHOST_IOTLB_INVALIDATE;
+
+    if (write(fd, &msg, sizeof(msg)) != sizeof(msg)) {
+        error_report("failed to write, fd=%d, errno=%d (%s)",
+            fd, errno, strerror(errno));
+        return -EIO ;
+    }

Same here (CID 1430270)

hw/virtio/vhost-vdpa.c line 60

Quoted 4 lines
hw/virtio/vhost-vdpa.c new file +4 -0
+
+    return ret;
+}
+

thanks
-- PMM

#38 lore raw

Re: [PULL v2 40/41] vhost-vdpa: introduce vhost-vdpa backend

Quoted 89 lines · Peter Maydell <[email protected]> 于2020年7月9日周四 下午11:15写道:
Quoted 6 lines · Michael S. Tsirkin

From: Cindy Lu <[email protected]>

Currently we have 2 types of vhost backends in QEMU: vhost kernel and vhost-user. The above patch provides a generic device for vDPA purpose, this vDPA device exposes to user space a non-vendor-specific

configuration

Quoted 1 line

interface for setting up a vhost HW accelerator, this patch set

introduces

Quoted 8 lines

a third vhost backend called vhost-vdpa based on the vDPA interface.

Vhost-vdpa usage:

qemu-system-x86_64 -cpu host -enable-kvm \
    ......
    -netdev type=vhost-vdpa,vhostdev=/dev/vhost-vdpa-id,id=vhost-vdpa0 \
    -device virtio-net-pci,netdev=vhost-vdpa0,page-per-vq=on \

Hi; Coverity reports some issues with this change:

Quoted 1 line

+static int vhost_vdpa_dma_map(struct vhost_vdpa *v, hwaddr iova, hwaddr

size,

Quoted 4 lines
hw/virtio/vhost-vdpa.c new file +4 -0
+                              void *vaddr, bool readonly)
+{
+    struct vhost_msg_v2 msg;
+    int fd = v->device_fd;

hw/virtio/vhost-vdpa.c line 52

Quoted 10 lines
hw/virtio/vhost-vdpa.c new file +10 -0
+    int ret = 0;
+
+    msg.type = v->msg_type;
+    msg.iotlb.iova = iova;
+    msg.iotlb.size = size;
+    msg.iotlb.uaddr = (uint64_t)vaddr;
+    msg.iotlb.perm = readonly ? VHOST_ACCESS_RO : VHOST_ACCESS_RW;
+    msg.iotlb.type = VHOST_IOTLB_UPDATE;
+
+    if (write(fd, &msg, sizeof(msg)) != sizeof(msg)) {

Here we write the contents of the struct down the pipe, but we have not initialized all its fields; specifically, msg.reserved is not initialized and so those bytes will be random. We'll also transfer random contents from the stack in the padding, potentially.

This is CID 1420267.

The easy fix is to zero-initialize the whole struct at the start:

   struct vhost_msg_v2 msg = {};
Quoted 15 lines
hw/virtio/vhost-vdpa.c new file +15 -0
+        error_report("failed to write, fd=%d, errno=%d (%s)",
+            fd, errno, strerror(errno));
+        return -EIO ;
+    }
+
+    return ret;
+}
+
+static int vhost_vdpa_dma_unmap(struct vhost_vdpa *v, hwaddr iova,
+                                hwaddr size)
+{
+    struct vhost_msg_v2 msg;
+    int fd = v->device_fd;
+    int ret = 0;
+

hw/virtio/vhost-vdpa.c line 77

Quoted 10 lines
hw/virtio/vhost-vdpa.c new file +10 -0
+    msg.type = v->msg_type;
+    msg.iotlb.iova = iova;
+    msg.iotlb.size = size;
+    msg.iotlb.type = VHOST_IOTLB_INVALIDATE;
+
+    if (write(fd, &msg, sizeof(msg)) != sizeof(msg)) {
+        error_report("failed to write, fd=%d, errno=%d (%s)",
+            fd, errno, strerror(errno));
+        return -EIO ;
+    }

Same here (CID 1430270)

hw/virtio/vhost-vdpa.c line 60

Quoted 4 lines
hw/virtio/vhost-vdpa.c new file +4 -0
+
+    return ret;
+}
+

thanks
-- PMM

Thank you Peter, I will fix this soon

#39 lore raw

[PULL v2 39/41] vhost_net: introduce set_config & get_config

In reply to Michael S. Tsirkin

From: Cindy Lu <[email protected]>

This patch introduces set_config & get_config method which allows vhost_net set/get the config to backend

Message-Id: <[email protected]>

---
 include/net/vhost_net.h |  5 +++++
 hw/net/vhost_net-stub.c | 11 +++++++++++
 hw/net/vhost_net.c      | 10 ++++++++++
 3 files changed, 26 insertions(+)
include/net/vhost_net.h +5 -0
index 77e47398c4..172b0051d8 100644
@@ -28,6 +28,11 @@ void vhost_net_cleanup(VHostNetState *net);
 uint64_t vhost_net_get_features(VHostNetState *net, uint64_t features);
 void vhost_net_ack_features(VHostNetState *net, uint64_t features);
 
+int vhost_net_get_config(struct vhost_net *net,  uint8_t *config,
+                         uint32_t config_len);
+
+int vhost_net_set_config(struct vhost_net *net, const uint8_t *data,
+                         uint32_t offset, uint32_t size, uint32_t flags);
 bool vhost_net_virtqueue_pending(VHostNetState *net, int n);
 void vhost_net_virtqueue_mask(VHostNetState *net, VirtIODevice *dev,
                               int idx, bool mask);
hw/net/vhost_net-stub.c +11 -0
index aac0e98228..a7f4252630 100644
@@ -52,6 +52,17 @@ uint64_t vhost_net_get_features(struct vhost_net *net, uint64_t features)
     return features;
 }
 
+int vhost_net_get_config(struct vhost_net *net,  uint8_t *config,
+                         uint32_t config_len)
+{
+    return 0;
+}
+int vhost_net_set_config(struct vhost_net *net, const uint8_t *data,
+                         uint32_t offset, uint32_t size, uint32_t flags)
+{
+    return 0;
+}
+
 void vhost_net_ack_features(struct vhost_net *net, uint64_t features)
 {
 }
hw/net/vhost_net.c +10 -0
index 4096d64aaf..4561665f6b 100644
@@ -110,6 +110,16 @@ uint64_t vhost_net_get_features(struct vhost_net *net, uint64_t features)
     return vhost_get_features(&net->dev, vhost_net_get_feature_bits(net),
             features);
 }
+int vhost_net_get_config(struct vhost_net *net,  uint8_t *config,
+                         uint32_t config_len)
+{
+    return vhost_dev_get_config(&net->dev, config, config_len);
+}
+int vhost_net_set_config(struct vhost_net *net, const uint8_t *data,
+                         uint32_t offset, uint32_t size, uint32_t flags)
+{
+    return vhost_dev_set_config(&net->dev, data, offset, size, flags);
+}
 
 void vhost_net_ack_features(struct vhost_net *net, uint64_t features)
 {
Signature · 1 line
MST
#40 lore raw

[PULL v2 31/41] virtio-pci: implement queue_enabled method

In reply to Michael S. Tsirkin

From: Jason Wang <[email protected]>

With version 1, we can detect whether a queue is enabled via queue_enabled.

Message-Id: <[email protected]>

---
 hw/virtio/virtio-pci.c | 13 +++++++++++++
 1 file changed, 13 insertions(+)
hw/virtio/virtio-pci.c +13 -0
index 7bc8c1c056..8554cf2a03 100644
@@ -1107,6 +1107,18 @@ static AddressSpace *virtio_pci_get_dma_as(DeviceState *d)
     return pci_get_address_space(dev);
 }
 
+static bool virtio_pci_queue_enabled(DeviceState *d, int n)
+{
+    VirtIOPCIProxy *proxy = VIRTIO_PCI(d);
+    VirtIODevice *vdev = virtio_bus_get_device(&proxy->bus);
+
+    if (virtio_vdev_has_feature(vdev, VIRTIO_F_VERSION_1)) {
+        return proxy->vqs[vdev->queue_sel].enabled;
+    }
+
+    return virtio_queue_enabled(vdev, n);
+}
+
 static int virtio_pci_add_mem_cap(VirtIOPCIProxy *proxy,
                                    struct virtio_pci_cap *cap)
 {
@@ -2064,6 +2076,7 @@ static void virtio_pci_bus_class_init(ObjectClass *klass, void *data)
     k->ioeventfd_enabled = virtio_pci_ioeventfd_enabled;
     k->ioeventfd_assign = virtio_pci_ioeventfd_assign;
     k->get_dma_as = virtio_pci_get_dma_as;
+    k->queue_enabled = virtio_pci_queue_enabled;
 }
 
 static const TypeInfo virtio_pci_bus_info = {
Signature · 1 line
MST
#41 lore raw

Re: [PULL v2 31/41] virtio-pci: implement queue_enabled method

Quoted 24 lines · Michael S. Tsirkin

From: Jason Wang <[email protected]>

With version 1, we can detect whether a queue is enabled via queue_enabled.

Message-Id: <[email protected]>

---
 hw/virtio/virtio-pci.c | 13 +++++++++++++
 1 file changed, 13 insertions(+)
hw/virtio/virtio-pci.c +1 -0
index 7bc8c1c056..8554cf2a03 100644
@@ -1107,6 +1107,18 @@ static AddressSpace *virtio_pci_get_dma_as(DeviceState *d)
     return pci_get_address_space(dev);
 }
 
+static bool virtio_pci_queue_enabled(DeviceState *d, int n)

hw/virtio/virtio-pci.c static AddressSpace *virtio_pci_get_dma_as(DeviceState *d) · line 1120

Quoted 10 lines
hw/virtio/virtio-pci.c +10 -0
+{
+    VirtIOPCIProxy *proxy = VIRTIO_PCI(d);
+    VirtIODevice *vdev = virtio_bus_get_device(&proxy->bus);
+
+    if (virtio_vdev_has_feature(vdev, VIRTIO_F_VERSION_1)) {
+        return proxy->vqs[vdev->queue_sel].enabled;
+    }
+
+    return virtio_queue_enabled(vdev, n);
+}

With "disable-legacy=off,disable-modern=true",
this changes introduces an infinite loop: virtio_queue_enabled() calls again virtio_pci_queue_enabled() that calls againvirtio_pci_queue_enabled()...

I think this should be changed like this:

hw/virtio/virtio-pci.c +0 -0
index ada1101d07..0a85c17e91 100644
@@ -1116,7 +1116,7 @@ static bool virtio_pci_queue_enabled(DeviceState

*d, int n)

         return proxy->vqs[vdev->queue_sel].enabled;
     }

- return virtio_queue_enabled(vdev, n);
+ return virtio_queue_get_desc_addr(vdev, n) != 0;
}

hw/virtio/virtio-pci.c static void virtio_pci_bus_class_init(ObjectClass *klass, void *data) · line 2082

Quoted 13 lines
hw/virtio/virtio-pci.c +2 -0
+
 static int virtio_pci_add_mem_cap(VirtIOPCIProxy *proxy,
                                    struct virtio_pci_cap *cap)
 {
@@ -2064,6 +2076,7 @@ static void virtio_pci_bus_class_init(ObjectClass *klass, void *data)
     k->ioeventfd_enabled = virtio_pci_ioeventfd_enabled;
     k->ioeventfd_assign = virtio_pci_ioeventfd_assign;
     k->get_dma_as = virtio_pci_get_dma_as;
+    k->queue_enabled = virtio_pci_queue_enabled;
 }
 
 static const TypeInfo virtio_pci_bus_info = {

Thanks,
Laurent

#42 lore raw

Re: [PULL v2 31/41] virtio-pci: implement queue_enabled method

Quoted 48 lines · Laurent Vivier
Quoted 24 lines · Michael S. Tsirkin

From: Jason Wang <[email protected]>

With version 1, we can detect whether a queue is enabled via queue_enabled.

Message-Id: <[email protected]>

---
 hw/virtio/virtio-pci.c | 13 +++++++++++++
 1 file changed, 13 insertions(+)
hw/virtio/virtio-pci.c +1 -0
index 7bc8c1c056..8554cf2a03 100644
@@ -1107,6 +1107,18 @@ static AddressSpace *virtio_pci_get_dma_as(DeviceState *d)
     return pci_get_address_space(dev);
 }
 
+static bool virtio_pci_queue_enabled(DeviceState *d, int n)

hw/virtio/virtio-pci.c static AddressSpace *virtio_pci_get_dma_as(DeviceState *d) · line 1120

Quoted 10 lines
hw/virtio/virtio-pci.c +10 -0
+{
+    VirtIOPCIProxy *proxy = VIRTIO_PCI(d);
+    VirtIODevice *vdev = virtio_bus_get_device(&proxy->bus);
+
+    if (virtio_vdev_has_feature(vdev, VIRTIO_F_VERSION_1)) {
+        return proxy->vqs[vdev->queue_sel].enabled;
+    }
+
+    return virtio_queue_enabled(vdev, n);
+}

With "disable-legacy=off,disable-modern=true",
this changes introduces an infinite loop: virtio_queue_enabled() calls again virtio_pci_queue_enabled() that calls againvirtio_pci_queue_enabled()...

I think this should be changed like this:

hw/virtio/virtio-pci.c +0 -0
index ada1101d07..0a85c17e91 100644

hw/virtio/virtio-pci.c static bool virtio_pci_queue_enabled(DeviceState · line 1122

Quoted 8 lines
hw/virtio/virtio-pci.c +1 -1
@@ -1116,7 +1116,7 @@ static bool virtio_pci_queue_enabled(DeviceState
 *d, int n)
         return proxy->vqs[vdev->queue_sel].enabled;
     }
 
-    return virtio_queue_enabled(vdev, n);
+    return virtio_queue_get_desc_addr(vdev, n) != 0;
 }
 

Thanks for the report and debugging the issue!

Maybe move

       return virtio_queue_get_desc_addr(vdev, n) != 0;

to a new API
virtio_pci_queue_enabled_legacy()

to avoid code duplication.

Could you cook up a patch pls? don't forget the Fixes: tag so people remember to backport it.

Quoted 16 lines

hw/virtio/virtio-pci.c static void virtio_pci_bus_class_init(ObjectClass *klass, void *data) · line 2082

Quoted 13 lines
hw/virtio/virtio-pci.c +2 -0
+
 static int virtio_pci_add_mem_cap(VirtIOPCIProxy *proxy,
                                    struct virtio_pci_cap *cap)
 {
@@ -2064,6 +2076,7 @@ static void virtio_pci_bus_class_init(ObjectClass *klass, void *data)
     k->ioeventfd_enabled = virtio_pci_ioeventfd_enabled;
     k->ioeventfd_assign = virtio_pci_ioeventfd_assign;
     k->get_dma_as = virtio_pci_get_dma_as;
+    k->queue_enabled = virtio_pci_queue_enabled;
 }
 
 static const TypeInfo virtio_pci_bus_info = {

Thanks,
Laurent

#43 lore raw

Re: [PULL v2 31/41] virtio-pci: implement queue_enabled method

Quoted 53 lines · Michael S. Tsirkin
Quoted 52 lines · Laurent Vivier
Quoted 24 lines · Michael S. Tsirkin

From: Jason Wang <[email protected]>

With version 1, we can detect whether a queue is enabled via queue_enabled.

Message-Id: <[email protected]>

---
 hw/virtio/virtio-pci.c | 13 +++++++++++++
 1 file changed, 13 insertions(+)
hw/virtio/virtio-pci.c +1 -0
index 7bc8c1c056..8554cf2a03 100644
@@ -1107,6 +1107,18 @@ static AddressSpace *virtio_pci_get_dma_as(DeviceState *d)
     return pci_get_address_space(dev);
 }
 
+static bool virtio_pci_queue_enabled(DeviceState *d, int n)

hw/virtio/virtio-pci.c static AddressSpace *virtio_pci_get_dma_as(DeviceState *d) · line 1120

Quoted 10 lines
hw/virtio/virtio-pci.c +10 -0
+{
+    VirtIOPCIProxy *proxy = VIRTIO_PCI(d);
+    VirtIODevice *vdev = virtio_bus_get_device(&proxy->bus);
+
+    if (virtio_vdev_has_feature(vdev, VIRTIO_F_VERSION_1)) {
+        return proxy->vqs[vdev->queue_sel].enabled;
+    }
+
+    return virtio_queue_enabled(vdev, n);
+}

With "disable-legacy=off,disable-modern=true",
this changes introduces an infinite loop: virtio_queue_enabled() calls again virtio_pci_queue_enabled() that calls againvirtio_pci_queue_enabled()...

I think this should be changed like this:

hw/virtio/virtio-pci.c +0 -0
index ada1101d07..0a85c17e91 100644
@@ -1116,7 +1116,7 @@ static bool virtio_pci_queue_enabled(DeviceState
 *d, int n)
         return proxy->vqs[vdev->queue_sel].enabled;
     }
 
Quoted 6 lines

hw/virtio/virtio-pci.c static bool virtio_pci_queue_enabled(DeviceState · line 1122

Quoted 4 lines
hw/virtio/virtio-pci.c +1 -1
-    return virtio_queue_enabled(vdev, n);
+    return virtio_queue_get_desc_addr(vdev, n) != 0;
 }
 

Thanks for the report and debugging the issue!

Ironically, the bug has been introduced because of my comment on the series:

https://patchew.org/QEMU/[email protected]/[email protected]/

Sorry for that.

Quoted 9 lines

Maybe move

       return virtio_queue_get_desc_addr(vdev, n) != 0;

to a new API
virtio_pci_queue_enabled_legacy()

to avoid code duplication.

Could you cook up a patch pls? don't forget the Fixes: tag so people remember to backport it.

Yes. I'm going to do that.

Thanks,
Laurent

#44 lore raw

[PULL v2 21/41] virtio-mem: Migration sanity checks

In reply to Michael S. Tsirkin

From: David Hildenbrand <[email protected]>

We want to make sure that certain properties don't change during migration, especially to catch user errors in a nice way. Let's migrate a temporary structure and validate that the properties didn't change.

Cc "Michael S. Tsirkin" <[email protected]>
Cc "Dr. David Alan Gilbert" <[email protected]>

Message-Id: <[email protected]>

---
 hw/virtio/virtio-mem.c | 70 ++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 70 insertions(+)
hw/virtio/virtio-mem.c +70 -0
index 6b5551515d..c1acfb9f24 100644
@@ -519,12 +519,82 @@ static int virtio_mem_post_load(void *opaque, int version_id)
     return virtio_mem_restore_unplugged(VIRTIO_MEM(opaque));
 }
 
+typedef struct VirtIOMEMMigSanityChecks {
+    VirtIOMEM *parent;
+    uint64_t addr;
+    uint64_t region_size;
+    uint64_t block_size;
+    uint32_t node;
+} VirtIOMEMMigSanityChecks;
+
+static int virtio_mem_mig_sanity_checks_pre_save(void *opaque)
+{
+    VirtIOMEMMigSanityChecks *tmp = opaque;
+    VirtIOMEM *vmem = tmp->parent;
+
+    tmp->addr = vmem->addr;
+    tmp->region_size = memory_region_size(&vmem->memdev->mr);
+    tmp->block_size = vmem->block_size;
+    tmp->node = vmem->node;
+    return 0;
+}
+
+static int virtio_mem_mig_sanity_checks_post_load(void *opaque, int version_id)
+{
+    VirtIOMEMMigSanityChecks *tmp = opaque;
+    VirtIOMEM *vmem = tmp->parent;
+    const uint64_t new_region_size = memory_region_size(&vmem->memdev->mr);
+
+    if (tmp->addr != vmem->addr) {
+        error_report("Property '%s' changed from 0x%" PRIx64 " to 0x%" PRIx64,
+                     VIRTIO_MEM_ADDR_PROP, tmp->addr, vmem->addr);
+        return -EINVAL;
+    }
+    /*
+     * Note: Preparation for resizeable memory regions. The maximum size
+     * of the memory region must not change during migration.
+     */
+    if (tmp->region_size != new_region_size) {
+        error_report("Property '%s' size changed from 0x%" PRIx64 " to 0x%"
+                     PRIx64, VIRTIO_MEM_MEMDEV_PROP, tmp->region_size,
+                     new_region_size);
+        return -EINVAL;
+    }
+    if (tmp->block_size != vmem->block_size) {
+        error_report("Property '%s' changed from 0x%" PRIx64 " to 0x%" PRIx64,
+                     VIRTIO_MEM_BLOCK_SIZE_PROP, tmp->block_size,
+                     vmem->block_size);
+        return -EINVAL;
+    }
+    if (tmp->node != vmem->node) {
+        error_report("Property '%s' changed from %" PRIu32 " to %" PRIu32,
+                     VIRTIO_MEM_NODE_PROP, tmp->node, vmem->node);
+        return -EINVAL;
+    }
+    return 0;
+}
+
+static const VMStateDescription vmstate_virtio_mem_sanity_checks = {
+    .name = "virtio-mem-device/sanity-checks",
+    .pre_save = virtio_mem_mig_sanity_checks_pre_save,
+    .post_load = virtio_mem_mig_sanity_checks_post_load,
+    .fields = (VMStateField[]) {
+        VMSTATE_UINT64(addr, VirtIOMEMMigSanityChecks),
+        VMSTATE_UINT64(region_size, VirtIOMEMMigSanityChecks),
+        VMSTATE_UINT64(block_size, VirtIOMEMMigSanityChecks),
+        VMSTATE_UINT32(node, VirtIOMEMMigSanityChecks),
+        VMSTATE_END_OF_LIST(),
+    },
+};
+
 static const VMStateDescription vmstate_virtio_mem_device = {
     .name = "virtio-mem-device",
     .minimum_version_id = 1,
     .version_id = 1,
     .post_load = virtio_mem_post_load,
     .fields = (VMStateField[]) {
+        VMSTATE_WITH_TMP(VirtIOMEM, VirtIOMEMMigSanityChecks,
+                         vmstate_virtio_mem_sanity_checks),
         VMSTATE_UINT64(usable_region_size, VirtIOMEM),
         VMSTATE_UINT64(size, VirtIOMEM),
         VMSTATE_UINT64(requested_size, VirtIOMEM),
Signature · 1 line
MST
#45 lore raw

[PULL v2 34/41] vhost: implement vhost_dev_start method

In reply to Michael S. Tsirkin

From: Cindy Lu <[email protected]>

use the vhost_dev_start callback to send the status to backend

Message-Id: <[email protected]>

---
 hw/virtio/vhost.c | 10 +++++++++-
 1 file changed, 9 insertions(+), 1 deletion(-)
hw/virtio/vhost.c +9 -1
index 10304b583e..32809e54b5 100644
@@ -1685,7 +1685,12 @@ int vhost_dev_start(struct vhost_dev *hdev, VirtIODevice *vdev)
             goto fail_log;
         }
     }
-
+    if (hdev->vhost_ops->vhost_dev_start) {
+        r = hdev->vhost_ops->vhost_dev_start(hdev, true);
+        if (r) {
+            goto fail_log;
+        }
+    }
     if (vhost_dev_has_iommu(hdev) &&
         hdev->vhost_ops->vhost_set_iotlb_callback) {
             hdev->vhost_ops->vhost_set_iotlb_callback(hdev, true);
@@ -1723,6 +1728,9 @@ void vhost_dev_stop(struct vhost_dev *hdev, VirtIODevice *vdev)
     /* should only be called after backend is connected */
     assert(hdev->vhost_ops);
 
+    if (hdev->vhost_ops->vhost_dev_start) {
+        hdev->vhost_ops->vhost_dev_start(hdev, false);
+    }
     for (i = 0; i < hdev->nvqs; ++i) {
         vhost_virtqueue_stop(hdev,
                              vdev,
Signature · 1 line
MST
#46 lore raw

[PULL v2 41/41] vhost-vdpa: introduce vhost-vdpa net client

In reply to Michael S. Tsirkin

From: Cindy Lu <[email protected]>

This patch set introduces a new net client type: vhost-vdpa. vhost-vdpa net client will set up a vDPA device which is specified by a "vhostdev" parameter.

Message-Id: <[email protected]>

---
 qapi/net.json            |  28 ++++-
 include/net/vhost-vdpa.h |  22 ++++
 net/clients.h            |   2 +
 net/net.c                |   3 +
 net/vhost-vdpa.c         | 228 +++++++++++++++++++++++++++++++++++++++
 net/Makefile.objs        |   2 +-
 6 files changed, 282 insertions(+), 3 deletions(-)
 create mode 100644 include/net/vhost-vdpa.h
 create mode 100644 net/vhost-vdpa.c
qapi/net.json +26 -2
index 9244c9af56..558d520a2f 100644
@@ -428,16 +428,39 @@
     '*vhostforce':    'bool',
     '*queues':        'int' } }
 
+##
+# @NetdevVhostVDPAOptions:
+#
+# Vhost-vdpa network backend
+#
+# vDPA device is a device that uses a datapath which complies with the virtio
+# specifications with a vendor specific control path.
+#
+# @vhostdev: path of vhost-vdpa device
+#            (default:'/dev/vhost-vdpa-0')
+#
+# @queues: number of queues to be created for multiqueue vhost-vdpa
+#          (default: 1)
+#
+# Since: 5.1
+##
+{ 'struct': 'NetdevVhostVDPAOptions',
+  'data': {
+    '*vhostdev':     'str',
+    '*queues':       'int' } }
+
 ##
 # @NetClientDriver:
 #
 # Available netdev drivers.
 #
 # Since: 2.7
+#
+# @vhost-vdpa since 5.1
 ##
 { 'enum': 'NetClientDriver',
   'data': [ 'none', 'nic', 'user', 'tap', 'l2tpv3', 'socket', 'vde',
-            'bridge', 'hubport', 'netmap', 'vhost-user' ] }
+            'bridge', 'hubport', 'netmap', 'vhost-user', 'vhost-vdpa' ] }
 
 ##
 # @Netdev:
@@ -465,7 +488,8 @@
     'bridge':   'NetdevBridgeOptions',
     'hubport':  'NetdevHubPortOptions',
     'netmap':   'NetdevNetmapOptions',
-    'vhost-user': 'NetdevVhostUserOptions' } }
+    'vhost-user': 'NetdevVhostUserOptions',
+    'vhost-vdpa': 'NetdevVhostVDPAOptions' } }
 
 ##
 # @NetFilterDirection:
include/net/vhost-vdpa.h new file +22 -0
new file mode 100644
index 0000000000..45e34b7cfc
@@ -0,0 +1,22 @@
+/*
+ * vhost-vdpa.h
+ *
+ * Copyright(c) 2017-2018 Intel Corporation.
+ * Copyright(c) 2020 Red Hat, Inc.
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ *
+ */
+
+#ifndef VHOST_VDPA_H
+#define VHOST_VDPA_H
+
+#define TYPE_VHOST_VDPA "vhost-vdpa"
+
+struct vhost_net *vhost_vdpa_get_vhost_net(NetClientState *nc);
+uint64_t vhost_vdpa_get_acked_features(NetClientState *nc);
+
+extern const int vdpa_feature_bits[];
+
+#endif /* VHOST_VDPA_H */
net/clients.h +2 -0
index a6ef267e19..92f9b59aed 100644
@@ -61,4 +61,6 @@ int net_init_netmap(const Netdev *netdev, const char *name,
 int net_init_vhost_user(const Netdev *netdev, const char *name,
                         NetClientState *peer, Error **errp);
 
+int net_init_vhost_vdpa(const Netdev *netdev, const char *name,
+                        NetClientState *peer, Error **errp);
 #endif /* QEMU_NET_CLIENTS_H */
net/net.c +3 -0
index 9099a327dd..94dc546fb2 100644
@@ -966,6 +966,9 @@ static int (* const net_client_init_fun[NET_CLIENT_DRIVER__MAX])(
 #ifdef CONFIG_VHOST_NET_USER
         [NET_CLIENT_DRIVER_VHOST_USER] = net_init_vhost_user,
 #endif
+#ifdef CONFIG_VHOST_NET_VDPA
+        [NET_CLIENT_DRIVER_VHOST_VDPA] = net_init_vhost_vdpa,
+#endif
 #ifdef CONFIG_L2TPV3
         [NET_CLIENT_DRIVER_L2TPV3]    = net_init_l2tpv3,
 #endif
net/vhost-vdpa.c new file +228 -0

228 lines not on this page · raw message

net/Makefile.objs +1 -1
index c5d076d19c..5ab45545db 100644
@@ -26,7 +26,7 @@ tap-obj-$(CONFIG_SOLARIS) = tap-solaris.o
 tap-obj-y ?= tap-stub.o
 common-obj-$(CONFIG_POSIX) += tap.o $(tap-obj-y)
 common-obj-$(CONFIG_WIN32) += tap-win32.o
-
+common-obj-$(CONFIG_VHOST_NET_VDPA) += vhost-vdpa.o
 vde.o-libs = $(VDE_LIBS)
 
 common-obj-$(CONFIG_CAN_BUS) += can/
Signature · 1 line
MST
#47 lore raw
In reply to Michael S. Tsirkin
Quoted 20 lines · Michael S. Tsirkin

v2 of the pull, fixing non-Linux builds.

The following changes since commit fc1bff958998910ec8d25db86cd2f53ff125f7ab:

hw/misc/pca9552: Add missing TypeInfo::class_size field (2020-06-29 21:16:10 +0100)

are available in the Git repository at:

git://git.kernel.org/pub/scm/virt/kvm/mst/qemu.git tags/for_upstream

for you to fetch changes up to f7d98068bf7a0913744dd4ac0a28e8c841df2af5:

vhost-vdpa: introduce vhost-vdpa net client (2020-07-04 11:13:50 -0400)

----------------------------------------------------------------
virtio,acpi: features, fixes, cleanups.

Quoted 8 lines

vdpa support
virtio-mem support
a handy script for disassembling acpi tables
misc fixes and cleanups

----------------------------------------------------------------

Hi; I'm afraid this fails to build on 32-bit:

/home/peter.maydell/qemu/hw/virtio/vhost-vdpa.c: In function 'vhost_vdpa_dma_map':
/home/peter.maydell/qemu/hw/virtio/vhost-vdpa.c:48:23: error: cast from pointer to integer of different size [-Werror=pointer-to-int-cast]

     msg.iotlb.uaddr = (uint64_t)vaddr;
                       ^

This should probably be "= (uintptr_t)vaddr".

thanks
-- PMM

#48 lore raw
Quoted 37 lines · Peter Maydell
Quoted 20 lines · Michael S. Tsirkin

v2 of the pull, fixing non-Linux builds.

The following changes since commit fc1bff958998910ec8d25db86cd2f53ff125f7ab:

hw/misc/pca9552: Add missing TypeInfo::class_size field (2020-06-29 21:16:10 +0100)

are available in the Git repository at:

git://git.kernel.org/pub/scm/virt/kvm/mst/qemu.git tags/for_upstream

for you to fetch changes up to f7d98068bf7a0913744dd4ac0a28e8c841df2af5:

vhost-vdpa: introduce vhost-vdpa net client (2020-07-04 11:13:50 -0400)

----------------------------------------------------------------
virtio,acpi: features, fixes, cleanups.

Quoted 8 lines

vdpa support
virtio-mem support
a handy script for disassembling acpi tables
misc fixes and cleanups

----------------------------------------------------------------

Hi; I'm afraid this fails to build on 32-bit:

/home/peter.maydell/qemu/hw/virtio/vhost-vdpa.c: In function 'vhost_vdpa_dma_map':
/home/peter.maydell/qemu/hw/virtio/vhost-vdpa.c:48:23: error: cast from pointer to integer of different size [-Werror=pointer-to-int-cast]

Quoted 7 lines
     msg.iotlb.uaddr = (uint64_t)vaddr;
                       ^

This should probably be "= (uintptr_t)vaddr".

thanks
-- PMM

Precisely. Sorry about missing this.
I made this change and pushed to the same tag - don't want to spam the list for a small thing like this. Can you pick this up pls? Commit 849c48004df0e123b53fe9888770cb4f6eb5e8ab now

Signature · 1 line
MST
#49 lore raw
Quoted 44 lines · Michael S. Tsirkin
Quoted 42 lines · Peter Maydell
Quoted 20 lines · Michael S. Tsirkin

v2 of the pull, fixing non-Linux builds.

The following changes since commit fc1bff958998910ec8d25db86cd2f53ff125f7ab:

hw/misc/pca9552: Add missing TypeInfo::class_size field (2020-06-29 21:16:10 +0100)

are available in the Git repository at:

git://git.kernel.org/pub/scm/virt/kvm/mst/qemu.git tags/for_upstream

for you to fetch changes up to f7d98068bf7a0913744dd4ac0a28e8c841df2af5:

vhost-vdpa: introduce vhost-vdpa net client (2020-07-04 11:13:50 -0400)

----------------------------------------------------------------
virtio,acpi: features, fixes, cleanups.

Quoted 8 lines

vdpa support
virtio-mem support
a handy script for disassembling acpi tables
misc fixes and cleanups

----------------------------------------------------------------

Hi; I'm afraid this fails to build on 32-bit:

/home/peter.maydell/qemu/hw/virtio/vhost-vdpa.c: In function 'vhost_vdpa_dma_map':
/home/peter.maydell/qemu/hw/virtio/vhost-vdpa.c:48:23: error: cast from pointer to integer of different size [-Werror=pointer-to-int-cast]

     msg.iotlb.uaddr = (uint64_t)vaddr;
                       ^

This should probably be "= (uintptr_t)vaddr".

Quoted 7 lines
Quoted 2 lines

thanks
-- PMM

Precisely. Sorry about missing this.
I made this change and pushed to the same tag - don't want to spam the list for a small thing like this. Can you pick this up pls? Commit 849c48004df0e123b53fe9888770cb4f6eb5e8ab now

Sure. (You can always just resend a new v2 cover letter without all the patches; that's what most people do for minor respins.)

Applied, thanks.

Please update the changelog at https://wiki.qemu.org/ChangeLog/5.1
for any user-visible changes.

-- PMM

#50 lore raw
Quoted 48 lines · Peter Maydell
Quoted 47 lines · Michael S. Tsirkin
Quoted 44 lines · Peter Maydell
Quoted 20 lines · Michael S. Tsirkin

v2 of the pull, fixing non-Linux builds.

The following changes since commit fc1bff958998910ec8d25db86cd2f53ff125f7ab:

hw/misc/pca9552: Add missing TypeInfo::class_size field (2020-06-29 21:16:10 +0100)

are available in the Git repository at:

git://git.kernel.org/pub/scm/virt/kvm/mst/qemu.git tags/for_upstream

for you to fetch changes up to f7d98068bf7a0913744dd4ac0a28e8c841df2af5:

vhost-vdpa: introduce vhost-vdpa net client (2020-07-04 11:13:50 -0400)

----------------------------------------------------------------
virtio,acpi: features, fixes, cleanups.

Quoted 8 lines

vdpa support
virtio-mem support
a handy script for disassembling acpi tables
misc fixes and cleanups

----------------------------------------------------------------

Hi; I'm afraid this fails to build on 32-bit:

/home/peter.maydell/qemu/hw/virtio/vhost-vdpa.c: In function 'vhost_vdpa_dma_map':
/home/peter.maydell/qemu/hw/virtio/vhost-vdpa.c:48:23: error: cast from pointer to integer of different size [-Werror=pointer-to-int-cast]

     msg.iotlb.uaddr = (uint64_t)vaddr;
                       ^

This should probably be "= (uintptr_t)vaddr".

thanks
-- PMM

Quoted 7 lines
Quoted 4 lines

Precisely. Sorry about missing this.
I made this change and pushed to the same tag - don't want to spam the list for a small thing like this. Can you pick this up pls? Commit 849c48004df0e123b53fe9888770cb4f6eb5e8ab now

Sure. (You can always just resend a new v2 cover letter without all the patches; that's what most people do for minor respins.)

As a reply-to or as a new thread?

Quoted 6 lines

Applied, thanks.

Please update the changelog at https://wiki.qemu.org/ChangeLog/5.1
for any user-visible changes.

-- PMM

#51 lore raw
Quoted 6 lines · Michael S. Tsirkin
Quoted 2 lines · Peter Maydell

Sure. (You can always just resend a new v2 cover letter without all the patches; that's what most people do for minor respins.)

As a reply-to or as a new thread?

New thread. Just like sending a complete new pullreq, only you only send the cover letter to the list, not all the patch mails. Some people send "cover letter plus the mail for the patch that changed" if they feel the changes were significant.

-- PMM

#52 lore raw
In reply to Peter Maydell
Quoted 11 lines · Peter Maydell
Quoted 4 lines · Michael S. Tsirkin

Precisely. Sorry about missing this.
I made this change and pushed to the same tag - don't want to spam the list for a small thing like this. Can you pick this up pls? Commit 849c48004df0e123b53fe9888770cb4f6eb5e8ab now

Sure. (You can always just resend a new v2 cover letter without all the patches; that's what most people do for minor respins.)

Applied, thanks.

I've just noticed that the commit that got merged was not the one you quote but 1e0a84ea49b68b7cf60e -- can you check whether anything was missed or the wrong version ?

thanks
-- PMM

#53 lore raw
Quoted 11 lines · Peter Maydell
Quoted 10 lines · Peter Maydell
Quoted 4 lines · Michael S. Tsirkin

Precisely. Sorry about missing this.
I made this change and pushed to the same tag - don't want to spam the list for a small thing like this. Can you pick this up pls? Commit 849c48004df0e123b53fe9888770cb4f6eb5e8ab now

Sure. (You can always just resend a new v2 cover letter without all the patches; that's what most people do for minor respins.)

Quoted 8 lines
Quoted 1 line

Applied, thanks.

I've just noticed that the commit that got merged was not the one you quote but 1e0a84ea49b68b7cf60e -- can you check whether anything was missed or the wrong version ?

thanks
-- PMM

Yea not a big deal:

hw/virtio/vhost-vdpa.c +1 -1
index fb7de972f3..a3d17fe0f9 100644
@@ -45,7 +45,7 @@ static int vhost_vdpa_dma_map(struct vhost_vdpa *v, hwaddr iova, hwaddr size,
     msg.type = v->msg_type;
     msg.iotlb.iova = iova;
     msg.iotlb.size = size;
-    msg.iotlb.uaddr = (uintptr_t)vaddr;
+    msg.iotlb.uaddr = (uint64_t)(uintptr_t)vaddr;
     msg.iotlb.perm = readonly ? VHOST_ACCESS_RO : VHOST_ACCESS_RW;
     msg.iotlb.type = VHOST_IOTLB_UPDATE;
 

I guess I forgot to push or something. Sorry.

Signature · 1 line
MST