Skip to contentSkip to conversation

Introduce xenpv machine for arm architecture

PATCH Superseded Vikram Garhwal 6 participants 50 messages Oct 15, 2022 – Dec 2, 2022 Arm/Amlogic Meson SoC support

v1 of 8 Superseded Latest is v8 →

lore
Share & tools
lore
#0 lore raw

cover Introduce xenpv machine for arm architecture

Hi,
This series add xenpv machine for aarch64. Motivation behind creating xenpv machine with IOREQ and TPM was to enable each guest on Xen aarch64 to have it's own unique and emulated TPM.

This series does following:
1. Moved common xen functionalities from hw/i386/xen to hw/xen/ so those can be used for aarch64.
2. We added a minimal xenpv arm machine which creates an IOREQ server and support TPM.

Please note that patch 05/12 breaks the build. Patch 06/12 fixes the build issue. If needed we can merge patch 05/12 and 06/12. For now we kept these separate to make changes easy to review.

Also, checkpatch.pl fails for 03/12 and 06/12. These fails are due to moving old code to new place which was not QEMU code style compatible. No new add code was added.

Regards,
Vikram

Stefano Stabellini (5):
hw/i386/xen/xen-hvm: move x86-specific fields out of XenIOState hw/i386/xen/xen-hvm: create arch_handle_ioreq and arch_xen_set_memory include/hw/xen/xen_common: return error from xen_create_ioreq_server hw/xen/xen-hvm-common: skip ioreq creation on ioreq registration
failure
meson.build: do not set have_xen_pci_passthrough for aarch64 targets

Vikram Garhwal (7):
hw/xen: Correct build config for xen_pt_stub hw/i386/xen/: move xen-mapcache.c to hw/xen/ hw/i386/xen: rearrange xen_hvm_init_pc
xen-hvm: move common functions to hw/xen/xen-hvm-common.c accel/xen/xen-all: export xenstore_record_dm_state hw/arm: introduce xenpv machine
meson.build: enable xenpv machine build for ARM

 accel/xen/xen-all.c              |    2 +-
 hw/arm/meson.build               |    2 +
 hw/arm/xen_arm.c                 |  163 +++++
 hw/i386/meson.build              |    1 +
 hw/i386/xen/meson.build          |    1 -
 hw/i386/xen/trace-events         |   19 -
 hw/i386/xen/xen-hvm.c            | 1084 +++---------------------------
 hw/xen/meson.build               |    9 +-
 hw/xen/trace-events              |   19 +
 hw/xen/xen-hvm-common.c          |  863 ++++++++++++++++++++++++
 hw/{i386 => }/xen/xen-mapcache.c |    0
 include/hw/arm/xen_arch_hvm.h    |   12 +
 include/hw/i386/xen_arch_hvm.h   |   11 +
 include/hw/xen/arch_hvm.h        |    5 +
 include/hw/xen/xen-hvm-common.h  |   97 +++
 include/hw/xen/xen.h             |    2 +
 include/hw/xen/xen_common.h      |   12 +-
 meson.build                      |    4 +-
 18 files changed, 1287 insertions(+), 1019 deletions(-)
 create mode 100644 hw/arm/xen_arm.c
 create mode 100644 hw/xen/xen-hvm-common.c
 rename hw/{i386 => }/xen/xen-mapcache.c (100%)
 create mode 100644 include/hw/arm/xen_arch_hvm.h
 create mode 100644 include/hw/i386/xen_arch_hvm.h
 create mode 100644 include/hw/xen/arch_hvm.h
 create mode 100644 include/hw/xen/xen-hvm-common.h
Signature · 1 line
2.17.0
#1 lore raw

PATCH 1/12 hw/xen: Correct build config for xen_pt_stub

Build fails when have_xen_pci_passthrough is disabled. This is because of incorrect build configuration for xen_pt_stub.c.

---
 hw/xen/meson.build | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
hw/xen/meson.build +1 -1
index 08dc1f6857..ae0ace3046 100644
@@ -18,7 +18,7 @@ if have_xen_pci_passthrough
     'xen_pt_msi.c',
   ))
 else
-  xen_specific_ss.add('xen_pt_stub.c')
+  xen_specific_ss.add(files('xen_pt_stub.c'))
 endif
 
 specific_ss.add_all(when: ['CONFIG_XEN', xen], if_true: xen_specific_ss)
Signature · 1 line
2.17.0
#4 lore raw

PATCH 2/12 hw/i386/xen/: move xen-mapcache.c to hw/xen/

In reply to Vikram Garhwal

xen-mapcache.c contains common functions which can be used for enabling Xen on aarch64 with IOREQ handling. Moving it out from hw/i386/xen to hw/xen to make it accessible for both aarch64 and x86.

---
 hw/i386/meson.build              | 1 +
 hw/i386/xen/meson.build          | 1 -
 hw/i386/xen/trace-events         | 5 -----
 hw/xen/meson.build               | 4 ++++
 hw/xen/trace-events              | 5 +++++
 hw/{i386 => }/xen/xen-mapcache.c | 0
 6 files changed, 10 insertions(+), 6 deletions(-)
 rename hw/{i386 => }/xen/xen-mapcache.c (100%)
hw/i386/meson.build +1 -0
index 213e2e82b3..cfdbfdcbcb 100644
@@ -33,5 +33,6 @@ subdir('kvm')
 subdir('xen')
 
 i386_ss.add_all(xenpv_ss)
+i386_ss.add_all(xen_ss)
 
 hw_arch += {'i386': i386_ss}
hw/i386/xen/meson.build +0 -1
index be84130300..2fcc46e6ca 100644
@@ -1,6 +1,5 @@
 i386_ss.add(when: 'CONFIG_XEN', if_true: files(
   'xen-hvm.c',
-  'xen-mapcache.c',
   'xen_apic.c',
   'xen_platform.c',
   'xen_pvdevice.c',
hw/i386/xen/trace-events +0 -5
index 5d6be61090..a0c89d91c4 100644
@@ -21,8 +21,3 @@ xen_map_resource_ioreq(uint32_t id, void *addr) "id: %u addr: %p"
 cpu_ioreq_config_read(void *req, uint32_t sbdf, uint32_t reg, uint32_t size, uint32_t data) "I/O=%p sbdf=0x%x reg=%u size=%u data=0x%x"
 cpu_ioreq_config_write(void *req, uint32_t sbdf, uint32_t reg, uint32_t size, uint32_t data) "I/O=%p sbdf=0x%x reg=%u size=%u data=0x%x"
 
-# xen-mapcache.c
-xen_map_cache(uint64_t phys_addr) "want 0x%"PRIx64
-xen_remap_bucket(uint64_t index) "index 0x%"PRIx64
-xen_map_cache_return(void* ptr) "%p"
-
hw/xen/meson.build +4 -0
index ae0ace3046..19d0637c46 100644
@@ -22,3 +22,7 @@ else
 endif
 
 specific_ss.add_all(when: ['CONFIG_XEN', xen], if_true: xen_specific_ss)
+
+xen_ss = ss.source_set()
+
+xen_ss.add(when: 'CONFIG_XEN', if_true: files('xen-mapcache.c'))
hw/xen/trace-events +5 -0
index 3da3fd8348..2c8f238f42 100644
@@ -41,3 +41,8 @@ xs_node_vprintf(char *path, char *value) "%s %s"
 xs_node_vscanf(char *path, char *value) "%s %s"
 xs_node_watch(char *path) "%s"
 xs_node_unwatch(char *path) "%s"
+
+# xen-mapcache.c
+xen_map_cache(uint64_t phys_addr) "want 0x%"PRIx64
+xen_remap_bucket(uint64_t index) "index 0x%"PRIx64
+xen_map_cache_return(void* ptr) "%p"
hw/xen/xen-mapcache.c renamed from hw/i386/xen/xen-mapcache.c +0 -0
similarity index 100%
rename from hw/i386/xen/xen-mapcache.c
rename to hw/xen/xen-mapcache.c
Signature · 1 line
2.17.0
#5 lore raw

Re: [PATCH v1 02/12] hw/i386/xen/: move xen-mapcache.c to hw/xen/

Quoted 56 lines · Vikram Garhwal

xen-mapcache.c contains common functions which can be used for enabling Xen on aarch64 with IOREQ handling. Moving it out from hw/i386/xen to hw/xen to make it accessible for both aarch64 and x86.

---
  hw/i386/meson.build              | 1 +
  hw/i386/xen/meson.build          | 1 -
  hw/i386/xen/trace-events         | 5 -----
  hw/xen/meson.build               | 4 ++++
  hw/xen/trace-events              | 5 +++++
  hw/{i386 => }/xen/xen-mapcache.c | 0
  6 files changed, 10 insertions(+), 6 deletions(-)

rename hw/{i386 => }/xen/xen-mapcache.c (100%)

hw/i386/meson.build +1 -0
index 213e2e82b3..cfdbfdcbcb 100644
@@ -33,5 +33,6 @@ subdir('kvm')
  subdir('xen')
 
  i386_ss.add_all(xenpv_ss)
+i386_ss.add_all(xen_ss)
 
  hw_arch += {'i386': i386_ss}
hw/i386/xen/meson.build +0 -1
index be84130300..2fcc46e6ca 100644
@@ -1,6 +1,5 @@
  i386_ss.add(when: 'CONFIG_XEN', if_true: files(
    'xen-hvm.c',
-  'xen-mapcache.c',
    'xen_apic.c',
    'xen_platform.c',
    'xen_pvdevice.c',
hw/i386/xen/trace-events +0 -5
index 5d6be61090..a0c89d91c4 100644
@@ -21,8 +21,3 @@ xen_map_resource_ioreq(uint32_t id, void *addr) "id: %u addr: %p"
  cpu_ioreq_config_read(void *req, uint32_t sbdf, uint32_t reg, uint32_t size, uint32_t data) "I/O=%p sbdf=0x%x reg=%u size=%u data=0x%x"
  cpu_ioreq_config_write(void *req, uint32_t sbdf, uint32_t reg, uint32_t size, uint32_t data) "I/O=%p sbdf=0x%x reg=%u size=%u data=0x%x"
 
-# xen-mapcache.c
-xen_map_cache(uint64_t phys_addr) "want 0x%"PRIx64
-xen_remap_bucket(uint64_t index) "index 0x%"PRIx64
-xen_map_cache_return(void* ptr) "%p"
-
hw/xen/meson.build +0 -0
index ae0ace3046..19d0637c46 100644

hw/xen/meson.build else · line 28

Quoted 7 lines
hw/xen/meson.build +4 -0
@@ -22,3 +22,7 @@ else
  endif
 
  specific_ss.add_all(when: ['CONFIG_XEN', xen], if_true: xen_specific_ss)
+
+xen_ss = ss.source_set()
+
+xen_ss.add(when: 'CONFIG_XEN', if_true: files('xen-mapcache.c'))

Curious as to why you couldn't just add this to the softmmu_ss list above?

Paul

Quoted 17 lines
hw/xen/trace-events +5 -0
index 3da3fd8348..2c8f238f42 100644
@@ -41,3 +41,8 @@ xs_node_vprintf(char *path, char *value) "%s %s"
  xs_node_vscanf(char *path, char *value) "%s %s"
  xs_node_watch(char *path) "%s"
  xs_node_unwatch(char *path) "%s"
+
+# xen-mapcache.c
+xen_map_cache(uint64_t phys_addr) "want 0x%"PRIx64
+xen_remap_bucket(uint64_t index) "index 0x%"PRIx64
+xen_map_cache_return(void* ptr) "%p"
hw/xen/xen-mapcache.c renamed from hw/i386/xen/xen-mapcache.c +0 -0
similarity index 100%
rename from hw/i386/xen/xen-mapcache.c
rename to hw/xen/xen-mapcache.c
#6 lore raw

Re: [PATCH v1 02/12] hw/i386/xen/: move xen-mapcache.c to hw/xen/

Hi Paul,

From: Paul Durrant <[email protected]>
Date: Wednesday, October 19, 2022 at 7:54 AM
To: Garhwal, Vikram <[email protected]>, [email protected] <[email protected]>
Cc: Stabellini, Stefano <[email protected]>, Michael S. Tsirkin <[email protected]>, Marcel Apfelbaum <[email protected]>, Paolo Bonzini <[email protected]>, Richard Henderson <[email protected]>, Eduardo Habkost <[email protected]>, Stefano Stabellini <[email protected]>, Anthony Perard <[email protected]>, open list:X86 Xen CPUs <[email protected]>
Subject: Re: [PATCH v1 02/12] hw/i386/xen/: move xen-mapcache.c to hw/xen/

Quoted 56 lines · Vikram Garhwal

xen-mapcache.c contains common functions which can be used for enabling Xen on aarch64 with IOREQ handling. Moving it out from hw/i386/xen to hw/xen to make it accessible for both aarch64 and x86.

---
  hw/i386/meson.build              | 1 +
  hw/i386/xen/meson.build          | 1 -
  hw/i386/xen/trace-events         | 5 -----
  hw/xen/meson.build               | 4 ++++
  hw/xen/trace-events              | 5 +++++
  hw/{i386 => }/xen/xen-mapcache.c | 0
  6 files changed, 10 insertions(+), 6 deletions(-)

rename hw/{i386 => }/xen/xen-mapcache.c (100%)

hw/i386/meson.build +1 -0
index 213e2e82b3..cfdbfdcbcb 100644
@@ -33,5 +33,6 @@ subdir('kvm')
  subdir('xen')
 
  i386_ss.add_all(xenpv_ss)
+i386_ss.add_all(xen_ss)
 
  hw_arch += {'i386': i386_ss}
hw/i386/xen/meson.build +0 -1
index be84130300..2fcc46e6ca 100644
@@ -1,6 +1,5 @@
  i386_ss.add(when: 'CONFIG_XEN', if_true: files(
    'xen-hvm.c',
-  'xen-mapcache.c',
    'xen_apic.c',
    'xen_platform.c',
    'xen_pvdevice.c',
hw/i386/xen/trace-events +0 -5
index 5d6be61090..a0c89d91c4 100644
@@ -21,8 +21,3 @@ xen_map_resource_ioreq(uint32_t id, void *addr) "id: %u addr: %p"
  cpu_ioreq_config_read(void *req, uint32_t sbdf, uint32_t reg, uint32_t size, uint32_t data) "I/O=%p sbdf=0x%x reg=%u size=%u data=0x%x"
  cpu_ioreq_config_write(void *req, uint32_t sbdf, uint32_t reg, uint32_t size, uint32_t data) "I/O=%p sbdf=0x%x reg=%u size=%u data=0x%x"
 
-# xen-mapcache.c
-xen_map_cache(uint64_t phys_addr) "want 0x%"PRIx64
-xen_remap_bucket(uint64_t index) "index 0x%"PRIx64
-xen_map_cache_return(void* ptr) "%p"
-
hw/xen/meson.build +0 -0
index ae0ace3046..19d0637c46 100644

hw/xen/meson.build else · line 28

Quoted 7 lines
hw/xen/meson.build +4 -0
@@ -22,3 +22,7 @@ else
  endif
 
  specific_ss.add_all(when: ['CONFIG_XEN', xen], if_true: xen_specific_ss)
+
+xen_ss = ss.source_set()
+
+xen_ss.add(when: 'CONFIG_XEN', if_true: files('xen-mapcache.c'))

Curious as to why you couldn't just add this to the softmmu_ss list above?
Moving it to softmmu_ss breaks the build as it builds without XEN_CONFIG. I think xen option in “softmmu_ss.add(when: ['CONFIG_XEN', xen], if_true: files” is problem here. I see below error:
/sysemu/xen-mapcache.h:16:8: error: attempt to use poisoned "CONFIG_XEN"

#ifdef CONFIG_XEN
        ^

../hw/xen/xen-mapcache.c:106:6: error: redefinition of 'xen_map_cache_init' void xen_map_cache_init(phys_offset_to_gaddr_t f, void *opaque)

Paul

Quoted 17 lines
hw/xen/trace-events +5 -0
index 3da3fd8348..2c8f238f42 100644
@@ -41,3 +41,8 @@ xs_node_vprintf(char *path, char *value) "%s %s"
  xs_node_vscanf(char *path, char *value) "%s %s"
  xs_node_watch(char *path) "%s"
  xs_node_unwatch(char *path) "%s"
+
+# xen-mapcache.c
+xen_map_cache(uint64_t phys_addr) "want 0x%"PRIx64
+xen_remap_bucket(uint64_t index) "index 0x%"PRIx64
+xen_map_cache_return(void* ptr) "%p"
hw/xen/xen-mapcache.c renamed from hw/i386/xen/xen-mapcache.c +0 -0
similarity index 100%
rename from hw/i386/xen/xen-mapcache.c
rename to hw/xen/xen-mapcache.c
#7 lore raw

PATCH 3/12 hw/i386/xen: rearrange xen_hvm_init_pc

In reply to Vikram Garhwal

In preparation to moving most of xen-hvm code to an arch-neutral location, move non IOREQ references to:
- xen_get_vmport_regs_pfn
- xen_suspend_notifier
- xen_wakeup_notifier
- xen_ram_init

towards the end of the xen_hvm_init_pc() function.

This is done to keep the common ioreq functions in one place which will be moved to new function in next patch in order to make it common to both x86 and aarch64 machines.

---
 hw/i386/xen/xen-hvm.c | 49 ++++++++++++++++++++++---------------------
 1 file changed, 25 insertions(+), 24 deletions(-)
hw/i386/xen/xen-hvm.c +25 -24
index e4293d6d66..b27484ad22 100644
@@ -1416,12 +1416,6 @@ void xen_hvm_init_pc(PCMachineState *pcms, MemoryRegion **ram_memory)
     state->exit.notify = xen_exit_notifier;
     qemu_add_exit_notifier(&state->exit);
 
-    state->suspend.notify = xen_suspend_notifier;
-    qemu_register_suspend_notifier(&state->suspend);
-
-    state->wakeup.notify = xen_wakeup_notifier;
-    qemu_register_wakeup_notifier(&state->wakeup);
-
     /*
      * Register wake-up support in QMP query-current-machine API
      */
@@ -1432,23 +1426,6 @@ void xen_hvm_init_pc(PCMachineState *pcms, MemoryRegion **ram_memory)
         goto err;
     }
 
-    rc = xen_get_vmport_regs_pfn(xen_xc, xen_domid, &ioreq_pfn);
-    if (!rc) {
-        DPRINTF("shared vmport page at pfn %lx\n", ioreq_pfn);
-        state->shared_vmport_page =
-            xenforeignmemory_map(xen_fmem, xen_domid, PROT_READ|PROT_WRITE,
-                                 1, &ioreq_pfn, NULL);
-        if (state->shared_vmport_page == NULL) {
-            error_report("map shared vmport IO page returned error %d handle=%p",
-                         errno, xen_xc);
-            goto err;
-        }
-    } else if (rc != -ENOSYS) {
-        error_report("get vmport regs pfn returned error %d, rc=%d",
-                     errno, rc);
-        goto err;
-    }
-
     /* Note: cpus is empty at this point in init */
     state->cpu_by_vcpu_id = g_new0(CPUState *, max_cpus);
 
@@ -1486,7 +1463,6 @@ void xen_hvm_init_pc(PCMachineState *pcms, MemoryRegion **ram_memory)
 #else
     xen_map_cache_init(NULL, state);
 #endif
-    xen_ram_init(pcms, ms->ram_size, ram_memory);
 
     qemu_add_vm_change_state_handler(xen_hvm_change_state_handler, state);
 
@@ -1513,6 +1489,31 @@ void xen_hvm_init_pc(PCMachineState *pcms, MemoryRegion **ram_memory)
     QLIST_INIT(&xen_physmap);
     xen_read_physmap(state);
 
+    state->suspend.notify = xen_suspend_notifier;
+    qemu_register_suspend_notifier(&state->suspend);
+
+    state->wakeup.notify = xen_wakeup_notifier;
+    qemu_register_wakeup_notifier(&state->wakeup);
+
+    rc = xen_get_vmport_regs_pfn(xen_xc, xen_domid, &ioreq_pfn);
+    if (!rc) {
+        DPRINTF("shared vmport page at pfn %lx\n", ioreq_pfn);
+        state->shared_vmport_page =
+            xenforeignmemory_map(xen_fmem, xen_domid, PROT_READ|PROT_WRITE,
+                                 1, &ioreq_pfn, NULL);
+        if (state->shared_vmport_page == NULL) {
+            error_report("map shared vmport IO page returned error %d handle=%p",
+                         errno, xen_xc);
+            goto err;
+        }
+    } else if (rc != -ENOSYS) {
+        error_report("get vmport regs pfn returned error %d, rc=%d",
+                     errno, rc);
+        goto err;
+    }
+
+    xen_ram_init(pcms, ms->ram_size, ram_memory);
+
     /* Disable ACPI build because Xen handles it */
     pcms->acpi_build_enabled = false;
 
Signature · 1 line
2.17.0
Reviewed-by on v1 3/12
#8 lore raw

Re: [PATCH v1 03/12] hw/i386/xen: rearrange xen_hvm_init_pc

Quoted 7 lines · Vikram Garhwal

In preparation to moving most of xen-hvm code to an arch-neutral location, move non IOREQ references to:
- xen_get_vmport_regs_pfn
- xen_suspend_notifier
- xen_wakeup_notifier
- xen_ram_init

Quoted 8 lines

towards the end of the xen_hvm_init_pc() function.

This is done to keep the common ioreq functions in one place which will be moved to new function in next patch in order to make it common to both x86 and aarch64 machines.

#9 lore raw

PATCH 4/12 hw/i386/xen/xen-hvm: move x86-specific fields out of XenIOState

In reply to Vikram Garhwal

From: Stefano Stabellini <[email protected]>

In preparation to moving most of xen-hvm code to an arch-neutral location, move:
- shared_vmport_page
- log_for_dirtybit
- dirty_bitmap
- suspend
- wakeup

out of XenIOState struct as these are only used on x86, especially the ones related to dirty logging.
Updated XenIOState can be used for both aarch64 and x86.

Also, remove free_phys_offset as it was unused.

---
 hw/i386/xen/xen-hvm.c | 58 ++++++++++++++++++++-----------------------
 1 file changed, 27 insertions(+), 31 deletions(-)
hw/i386/xen/xen-hvm.c +27 -31
index b27484ad22..e169de16c4 100644
@@ -73,6 +73,7 @@ struct shared_vmport_iopage {
 };
 typedef struct shared_vmport_iopage shared_vmport_iopage_t;
 #endif
+static shared_vmport_iopage_t *shared_vmport_page;
 
 static inline uint32_t xen_vcpu_eport(shared_iopage_t *shared_page, int i)
 {
@@ -95,6 +96,11 @@ typedef struct XenPhysmap {
 } XenPhysmap;
 
 static QLIST_HEAD(, XenPhysmap) xen_physmap;
+static const XenPhysmap *log_for_dirtybit;
+/* Buffer used by xen_sync_dirty_bitmap */
+static unsigned long *dirty_bitmap;
+static Notifier suspend;
+static Notifier wakeup;
 
 typedef struct XenPciDevice {
     PCIDevice *pci_dev;
@@ -105,7 +111,6 @@ typedef struct XenPciDevice {
 typedef struct XenIOState {
     ioservid_t ioservid;
     shared_iopage_t *shared_page;
-    shared_vmport_iopage_t *shared_vmport_page;
     buffered_iopage_t *buffered_io_page;
     xenforeignmemory_resource_handle *fres;
     QEMUTimer *buffered_io_timer;
@@ -125,14 +130,8 @@ typedef struct XenIOState {
     MemoryListener io_listener;
     QLIST_HEAD(, XenPciDevice) dev_list;
     DeviceListener device_listener;
-    hwaddr free_phys_offset;
-    const XenPhysmap *log_for_dirtybit;
-    /* Buffer used by xen_sync_dirty_bitmap */
-    unsigned long *dirty_bitmap;
 
     Notifier exit;
-    Notifier suspend;
-    Notifier wakeup;
 } XenIOState;
 
 /* Xen specific function for piix pci */
@@ -462,10 +461,10 @@ static int xen_remove_from_physmap(XenIOState *state,
     }
 
     QLIST_REMOVE(physmap, list);
-    if (state->log_for_dirtybit == physmap) {
-        state->log_for_dirtybit = NULL;
-        g_free(state->dirty_bitmap);
-        state->dirty_bitmap = NULL;
+    if (log_for_dirtybit == physmap) {
+        log_for_dirtybit = NULL;
+        g_free(dirty_bitmap);
+        dirty_bitmap = NULL;
     }
     g_free(physmap);
 
@@ -626,16 +625,16 @@ static void xen_sync_dirty_bitmap(XenIOState *state,
         return;
     }
 
-    if (state->log_for_dirtybit == NULL) {
-        state->log_for_dirtybit = physmap;
-        state->dirty_bitmap = g_new(unsigned long, bitmap_size);
-    } else if (state->log_for_dirtybit != physmap) {
+    if (log_for_dirtybit == NULL) {
+        log_for_dirtybit = physmap;
+        dirty_bitmap = g_new(unsigned long, bitmap_size);
+    } else if (log_for_dirtybit != physmap) {
         /* Only one range for dirty bitmap can be tracked. */
         return;
     }
 
     rc = xen_track_dirty_vram(xen_domid, start_addr >> TARGET_PAGE_BITS,
-                              npages, state->dirty_bitmap);
+                              npages, dirty_bitmap);
     if (rc < 0) {
 #ifndef ENODATA
 #define ENODATA  ENOENT
@@ -650,7 +649,7 @@ static void xen_sync_dirty_bitmap(XenIOState *state,
     }
 
     for (i = 0; i < bitmap_size; i++) {
-        unsigned long map = state->dirty_bitmap[i];
+        unsigned long map = dirty_bitmap[i];
         while (map != 0) {
             j = ctzl(map);
             map &= ~(1ul << j);
@@ -676,12 +675,10 @@ static void xen_log_start(MemoryListener *listener,
 static void xen_log_stop(MemoryListener *listener, MemoryRegionSection *section,
                          int old, int new)
 {
-    XenIOState *state = container_of(listener, XenIOState, memory_listener);
-
     if (old & ~new & (1 << DIRTY_MEMORY_VGA)) {
-        state->log_for_dirtybit = NULL;
-        g_free(state->dirty_bitmap);
-        state->dirty_bitmap = NULL;
+        log_for_dirtybit = NULL;
+        g_free(dirty_bitmap);
+        dirty_bitmap = NULL;
         /* Disable dirty bit tracking */
         xen_track_dirty_vram(xen_domid, 0, 0, NULL);
     }
@@ -1021,9 +1018,9 @@ static void handle_vmport_ioreq(XenIOState *state, ioreq_t *req)
 {
     vmware_regs_t *vmport_regs;
 
-    assert(state->shared_vmport_page);
+    assert(shared_vmport_page);
     vmport_regs =
-        &state->shared_vmport_page->vcpu_vmport_regs[state->send_vcpu];
+        &shared_vmport_page->vcpu_vmport_regs[state->send_vcpu];
     QEMU_BUILD_BUG_ON(sizeof(*req) < sizeof(*vmport_regs));
 
     current_cpu = state->cpu_by_vcpu_id[state->send_vcpu];
@@ -1468,7 +1465,6 @@ void xen_hvm_init_pc(PCMachineState *pcms, MemoryRegion **ram_memory)
 
     state->memory_listener = xen_memory_listener;
     memory_listener_register(&state->memory_listener, &address_space_memory);
-    state->log_for_dirtybit = NULL;
 
     state->io_listener = xen_io_listener;
     memory_listener_register(&state->io_listener, &address_space_io);
@@ -1489,19 +1485,19 @@ void xen_hvm_init_pc(PCMachineState *pcms, MemoryRegion **ram_memory)
     QLIST_INIT(&xen_physmap);
     xen_read_physmap(state);
 
-    state->suspend.notify = xen_suspend_notifier;
-    qemu_register_suspend_notifier(&state->suspend);
+    suspend.notify = xen_suspend_notifier;
+    qemu_register_suspend_notifier(&suspend);
 
-    state->wakeup.notify = xen_wakeup_notifier;
-    qemu_register_wakeup_notifier(&state->wakeup);
+    wakeup.notify = xen_wakeup_notifier;
+    qemu_register_wakeup_notifier(&wakeup);
 
     rc = xen_get_vmport_regs_pfn(xen_xc, xen_domid, &ioreq_pfn);
     if (!rc) {
         DPRINTF("shared vmport page at pfn %lx\n", ioreq_pfn);
-        state->shared_vmport_page =
+        shared_vmport_page =
             xenforeignmemory_map(xen_fmem, xen_domid, PROT_READ|PROT_WRITE,
                                  1, &ioreq_pfn, NULL);
-        if (state->shared_vmport_page == NULL) {
+        if (shared_vmport_page == NULL) {
             error_report("map shared vmport IO page returned error %d handle=%p",
                          errno, xen_xc);
             goto err;
Signature · 1 line
2.17.0
Reviewed-by on v1 4/12
#10 lore raw

Re: [PATCH v1 04/12] hw/i386/xen/xen-hvm: move x86-specific fields out of XenIOState

Quoted 9 lines · Vikram Garhwal

From: Stefano Stabellini <[email protected]>

In preparation to moving most of xen-hvm code to an arch-neutral location, move:
- shared_vmport_page
- log_for_dirtybit
- dirty_bitmap
- suspend
- wakeup

Quoted 8 lines

out of XenIOState struct as these are only used on x86, especially the ones related to dirty logging.
Updated XenIOState can be used for both aarch64 and x86.

Also, remove free_phys_offset as it was unused.

Reviewed-by on v1 4/12
#11 lore raw

Re: [PATCH v1 04/12] hw/i386/xen/xen-hvm: move x86-specific fields out of XenIOState

In reply to Vikram Garhwal
Quoted 9 lines · Vikram Garhwal

From: Stefano Stabellini <[email protected]>

In preparation to moving most of xen-hvm code to an arch-neutral location, move:
- shared_vmport_page
- log_for_dirtybit
- dirty_bitmap
- suspend
- wakeup

Quoted 8 lines

out of XenIOState struct as these are only used on x86, especially the ones related to dirty logging.
Updated XenIOState can be used for both aarch64 and x86.

Also, remove free_phys_offset as it was unused.

Reviewed-by on v1 4/12
#12 lore raw

Re: [PATCH v1 04/12] hw/i386/xen/xen-hvm: move x86-specific fields out of XenIOState

In reply to Vikram Garhwal
Quoted 9 lines · Vikram Garhwal

From: Stefano Stabellini <[email protected]>

In preparation to moving most of xen-hvm code to an arch-neutral location, move:
- shared_vmport_page
- log_for_dirtybit
- dirty_bitmap
- suspend
- wakeup

Quoted 8 lines

out of XenIOState struct as these are only used on x86, especially the ones related to dirty logging.
Updated XenIOState can be used for both aarch64 and x86.

Also, remove free_phys_offset as it was unused.

Signature · 1 line
Alex Bennée
#13 lore raw

PATCH 5/12 hw/i386/xen/xen-hvm: create arch_handle_ioreq and arch_xen_set_memory

In reply to Vikram Garhwal

From: Stefano Stabellini <[email protected]>

In preparation to moving most of xen-hvm code to an arch-neutral location, move the x86-specific portion of xen_set_memory to arch_xen_set_memory.

Also move handle_vmport_ioreq to arch_handle_ioreq.

NOTE: This patch breaks the build. Next patch fixes the build issue. Reason behind creating this patch is because there is lot of new code addition and pure code movement done for enabling Xen on ARM. Keeping the this patch separate is done to make it easier to review.

---
 hw/i386/xen/xen-hvm.c          | 97 ++++++++++++++++++++--------------
 include/hw/i386/xen_arch_hvm.h | 10 ++++
 include/hw/xen/arch_hvm.h      |  3 ++
 3 files changed, 70 insertions(+), 40 deletions(-)
 create mode 100644 include/hw/i386/xen_arch_hvm.h
 create mode 100644 include/hw/xen/arch_hvm.h
hw/i386/xen/xen-hvm.c +57 -40
index e169de16c4..3cd1808f9d 100644
@@ -34,6 +34,7 @@
 #include "trace.h"
 
 #include <xen/hvm/ioreq.h>
+#include "hw/xen/arch_hvm.h"
 #include <xen/hvm/e820.h>
 
 //#define DEBUG_XEN_HVM
@@ -476,10 +477,6 @@ static void xen_set_memory(struct MemoryListener *listener,
                            bool add)
 {
     XenIOState *state = container_of(listener, XenIOState, memory_listener);
-    hwaddr start_addr = section->offset_within_address_space;
-    ram_addr_t size = int128_get64(section->size);
-    bool log_dirty = memory_region_is_logging(section->mr, DIRTY_MEMORY_VGA);
-    hvmmem_type_t mem_type;
 
     if (section->mr == &ram_memory) {
         return;
@@ -492,38 +489,7 @@ static void xen_set_memory(struct MemoryListener *listener,
                                      section);
         }
     }
-
-    if (!memory_region_is_ram(section->mr)) {
-        return;
-    }
-
-    if (log_dirty != add) {
-        return;
-    }
-
-    trace_xen_client_set_memory(start_addr, size, log_dirty);
-
-    start_addr &= TARGET_PAGE_MASK;
-    size = TARGET_PAGE_ALIGN(size);
-
-    if (add) {
-        if (!memory_region_is_rom(section->mr)) {
-            xen_add_to_physmap(state, start_addr, size,
-                               section->mr, section->offset_within_region);
-        } else {
-            mem_type = HVMMEM_ram_ro;
-            if (xen_set_mem_type(xen_domid, mem_type,
-                                 start_addr >> TARGET_PAGE_BITS,
-                                 size >> TARGET_PAGE_BITS)) {
-                DPRINTF("xen_set_mem_type error, addr: "TARGET_FMT_plx"\n",
-                        start_addr);
-            }
-        }
-    } else {
-        if (xen_remove_from_physmap(state, start_addr, size) < 0) {
-            DPRINTF("physmapping does not exist at "TARGET_FMT_plx"\n", start_addr);
-        }
-    }
+    arch_xen_set_memory(state, section, add);
 }
 
 static void xen_region_add(MemoryListener *listener,
@@ -1051,9 +1017,6 @@ static void handle_ioreq(XenIOState *state, ioreq_t *req)
         case IOREQ_TYPE_COPY:
             cpu_ioreq_move(req);
             break;
-        case IOREQ_TYPE_VMWARE_PORT:
-            handle_vmport_ioreq(state, req);
-            break;
         case IOREQ_TYPE_TIMEOFFSET:
             break;
         case IOREQ_TYPE_INVALIDATE:
@@ -1063,7 +1026,7 @@ static void handle_ioreq(XenIOState *state, ioreq_t *req)
             cpu_ioreq_config(state, req);
             break;
         default:
-            hw_error("Invalid ioreq type 0x%x\n", req->type);
+            arch_handle_ioreq(state, req);
     }
     if (req->dir == IOREQ_READ) {
         trace_handle_ioreq_read(req, req->type, req->df, req->data_is_ptr,
@@ -1604,3 +1567,57 @@ void qmp_xen_set_global_dirty_log(bool enable, Error **errp)
         memory_global_dirty_log_stop(GLOBAL_DIRTY_MIGRATION);
     }
 }
+
+void arch_xen_set_memory(XenIOState *state, MemoryRegionSection *section,
+                                bool add)
+{
+    hwaddr start_addr = section->offset_within_address_space;
+    ram_addr_t size = int128_get64(section->size);
+    bool log_dirty = memory_region_is_logging(section->mr, DIRTY_MEMORY_VGA);
+    hvmmem_type_t mem_type;
+
+    if (!memory_region_is_ram(section->mr)) {
+        return;
+    }
+
+    if (log_dirty != add) {
+        return;
+    }
+
+    trace_xen_client_set_memory(start_addr, size, log_dirty);
+
+    start_addr &= TARGET_PAGE_MASK;
+    size = TARGET_PAGE_ALIGN(size);
+
+    if (add) {
+        if (!memory_region_is_rom(section->mr)) {
+            xen_add_to_physmap(state, start_addr, size,
+                               section->mr, section->offset_within_region);
+        } else {
+            mem_type = HVMMEM_ram_ro;
+            if (xen_set_mem_type(xen_domid, mem_type,
+                                 start_addr >> TARGET_PAGE_BITS,
+                                 size >> TARGET_PAGE_BITS)) {
+                DPRINTF("xen_set_mem_type error, addr: "TARGET_FMT_plx"\n",
+                        start_addr);
+            }
+        }
+    } else {
+        if (xen_remove_from_physmap(state, start_addr, size) < 0) {
+            DPRINTF("physmapping does not exist at "TARGET_FMT_plx"\n", start_addr);
+        }
+    }
+}
+
+void arch_handle_ioreq(XenIOState *state, ioreq_t *req)
+{
+    switch (req->type) {
+    case IOREQ_TYPE_VMWARE_PORT:
+            handle_vmport_ioreq(state, req);
+        break;
+    default:
+        hw_error("Invalid ioreq type 0x%x\n", req->type);
+    }
+
+    return;
+}
include/hw/i386/xen_arch_hvm.h new file +10 -0
new file mode 100644
index 0000000000..1b2c71ba4f
@@ -0,0 +1,10 @@
+#ifndef HW_XEN_ARCH_I386_HVM_H
+#define HW_XEN_ARCH_I386_HVM_H
+
+#include <xen/hvm/ioreq.h>
+
+void arch_handle_ioreq(XenIOState *state, ioreq_t *req);
+void arch_xen_set_memory(XenIOState *state,
+                         MemoryRegionSection *section,
+                         bool add);
+#endif
include/hw/xen/arch_hvm.h new file +3 -0
new file mode 100644
index 0000000000..26674648d8
@@ -0,0 +1,3 @@
+#if defined(TARGET_I386) || defined(TARGET_X86_64)
+#include "hw/i386/xen_arch_hvm.h"
+#endif
Signature · 1 line
2.17.0
Reviewed-by on v1 5/12
#14 lore raw

Re: [PATCH v1 05/12] hw/i386/xen/xen-hvm: create arch_handle_ioreq and arch_xen_set_memory

Quoted 7 lines · Vikram Garhwal

From: Stefano Stabellini <[email protected]>

In preparation to moving most of xen-hvm code to an arch-neutral location, move the x86-specific portion of xen_set_memory to arch_xen_set_memory.

Also move handle_vmport_ioreq to arch_handle_ioreq.

Quoted 7 lines

NOTE: This patch breaks the build. Next patch fixes the build issue. Reason behind creating this patch is because there is lot of new code addition and pure code movement done for enabling Xen on ARM. Keeping the this patch separate is done to make it easier to review.

Reviewed-by on v1 5/12
#15 lore raw

Re: [PATCH v1 05/12] hw/i386/xen/xen-hvm: create arch_handle_ioreq and arch_xen_set_memory

In reply to Vikram Garhwal
Quoted 11 lines · Vikram Garhwal

From: Stefano Stabellini <[email protected]>

In preparation to moving most of xen-hvm code to an arch-neutral location, move the x86-specific portion of xen_set_memory to arch_xen_set_memory.

Also move handle_vmport_ioreq to arch_handle_ioreq.

NOTE: This patch breaks the build. Next patch fixes the build issue. Reason behind creating this patch is because there is lot of new code addition and pure code movement done for enabling Xen on ARM. Keeping the this patch separate is done to make it easier to review.

But you do intend to squash the patches for the final version? We don't want to intentionally break bisection.

Otherwise:

Signature · 1 line
Alex Bennée
#16 lore raw

PATCH 6/12 xen-hvm: move common functions to hw/xen/xen-hvm-common.c

In reply to Vikram Garhwal

Extract common functionalities from hw/i386/xen/xen-hvm.c and move them to hw/xen/xen-hvm-common.c. These common functions are useful for creating an IOREQ server.

xen_hvm_init_pc() contains the arch independent code for creating and mapping a IOREQ server, connecting memory and IO listeners, initializing a xen bus and registering backends. Moved this common xen code to a new function xen_register_ioreq() which can be used by both x86 and ARM machines.

Following functions are moved to hw/xen/xen-hvm-common.c:

    xen_vcpu_eport(), xen_vcpu_ioreq(), xen_ram_alloc(), xen_set_memory(),
    xen_region_add(), xen_region_del(), xen_io_add(), xen_io_del(),
    xen_device_realize(), xen_device_unrealize(),
    cpu_get_ioreq_from_shared_memory(), cpu_get_ioreq(), do_inp(), do_outp(),
    rw_phys_req_item(), read_phys_req_item(), write_phys_req_item(),
    cpu_ioreq_pio(), cpu_ioreq_move(), cpu_ioreq_config(), handle_ioreq(),
    handle_buffered_iopage(), handle_buffered_io(), cpu_handle_ioreq(),
    xen_main_loop_prepare(), xen_hvm_change_state_handler(),
    xen_exit_notifier(), xen_map_ioreq_server(), destroy_hvm_domain() and
    xen_shutdown_fatal_error()

Removed static type from below functions:
1. xen_region_add()
2. xen_region_del()
3. xen_io_add()
4. xen_io_del()
5. xen_device_realize()
6. xen_device_unrealize()
7. xen_hvm_change_state_handler()
8. cpu_ioreq_pio()
9. xen_exit_notifier()

---
 hw/i386/xen/trace-events        |  14 -
 hw/i386/xen/xen-hvm.c           | 930 +-------------------------------
 hw/xen/meson.build              |   5 +-
 hw/xen/trace-events             |  14 +
 hw/xen/xen-hvm-common.c         | 858 +++++++++++++++++++++++++++++
 include/hw/i386/xen_arch_hvm.h  |   1 +
 include/hw/xen/xen-hvm-common.h |  97 ++++
 7 files changed, 983 insertions(+), 936 deletions(-)
 create mode 100644 hw/xen/xen-hvm-common.c
 create mode 100644 include/hw/xen/xen-hvm-common.h
hw/i386/xen/trace-events +0 -14
index a0c89d91c4..5d0a8d6dcf 100644
@@ -7,17 +7,3 @@ xen_platform_log(char *s) "xen platform: %s"
 xen_pv_mmio_read(uint64_t addr) "WARNING: read from Xen PV Device MMIO space (address 0x%"PRIx64")"
 xen_pv_mmio_write(uint64_t addr) "WARNING: write to Xen PV Device MMIO space (address 0x%"PRIx64")"
 
-# xen-hvm.c
-xen_ram_alloc(unsigned long ram_addr, unsigned long size) "requested: 0x%lx, size 0x%lx"
-xen_client_set_memory(uint64_t start_addr, unsigned long size, bool log_dirty) "0x%"PRIx64" size 0x%lx, log_dirty %i"
-handle_ioreq(void *req, uint32_t type, uint32_t dir, uint32_t df, uint32_t data_is_ptr, uint64_t addr, uint64_t data, uint32_t count, uint32_t size) "I/O=%p type=%d dir=%d df=%d ptr=%d port=0x%"PRIx64" data=0x%"PRIx64" count=%d size=%d"
-handle_ioreq_read(void *req, uint32_t type, uint32_t df, uint32_t data_is_ptr, uint64_t addr, uint64_t data, uint32_t count, uint32_t size) "I/O=%p read type=%d df=%d ptr=%d port=0x%"PRIx64" data=0x%"PRIx64" count=%d size=%d"
-handle_ioreq_write(void *req, uint32_t type, uint32_t df, uint32_t data_is_ptr, uint64_t addr, uint64_t data, uint32_t count, uint32_t size) "I/O=%p write type=%d df=%d ptr=%d port=0x%"PRIx64" data=0x%"PRIx64" count=%d size=%d"
-cpu_ioreq_pio(void *req, uint32_t dir, uint32_t df, uint32_t data_is_ptr, uint64_t addr, uint64_t data, uint32_t count, uint32_t size) "I/O=%p pio dir=%d df=%d ptr=%d port=0x%"PRIx64" data=0x%"PRIx64" count=%d size=%d"
-cpu_ioreq_pio_read_reg(void *req, uint64_t data, uint64_t addr, uint32_t size) "I/O=%p pio read reg data=0x%"PRIx64" port=0x%"PRIx64" size=%d"
-cpu_ioreq_pio_write_reg(void *req, uint64_t data, uint64_t addr, uint32_t size) "I/O=%p pio write reg data=0x%"PRIx64" port=0x%"PRIx64" size=%d"
-cpu_ioreq_move(void *req, uint32_t dir, uint32_t df, uint32_t data_is_ptr, uint64_t addr, uint64_t data, uint32_t count, uint32_t size) "I/O=%p copy dir=%d df=%d ptr=%d port=0x%"PRIx64" data=0x%"PRIx64" count=%d size=%d"
-xen_map_resource_ioreq(uint32_t id, void *addr) "id: %u addr: %p"
-cpu_ioreq_config_read(void *req, uint32_t sbdf, uint32_t reg, uint32_t size, uint32_t data) "I/O=%p sbdf=0x%x reg=%u size=%u data=0x%x"
-cpu_ioreq_config_write(void *req, uint32_t sbdf, uint32_t reg, uint32_t size, uint32_t data) "I/O=%p sbdf=0x%x reg=%u size=%u data=0x%x"
-
hw/i386/xen/xen-hvm.c +9 -921

1,008 lines not on this page · raw message

hw/xen/meson.build +4 -1
index 19d0637c46..008e036d63 100644
@@ -25,4 +25,7 @@ specific_ss.add_all(when: ['CONFIG_XEN', xen], if_true: xen_specific_ss)
 
 xen_ss = ss.source_set()
 
-xen_ss.add(when: 'CONFIG_XEN', if_true: files('xen-mapcache.c'))
+xen_ss.add(when: 'CONFIG_XEN', if_true: files(
+  'xen-mapcache.c',
+  'xen-hvm-common.c',
+))
hw/xen/trace-events +14 -0
index 2c8f238f42..02ca1183da 100644
@@ -42,6 +42,20 @@ xs_node_vscanf(char *path, char *value) "%s %s"
 xs_node_watch(char *path) "%s"
 xs_node_unwatch(char *path) "%s"
 
+# xen-hvm.c
+xen_ram_alloc(unsigned long ram_addr, unsigned long size) "requested: 0x%lx, size 0x%lx"
+xen_client_set_memory(uint64_t start_addr, unsigned long size, bool log_dirty) "0x%"PRIx64" size 0x%lx, log_dirty %i"
+handle_ioreq(void *req, uint32_t type, uint32_t dir, uint32_t df, uint32_t data_is_ptr, uint64_t addr, uint64_t data, uint32_t count, uint32_t size) "I/O=%p type=%d dir=%d df=%d ptr=%d port=0x%"PRIx64" data=0x%"PRIx64" count=%d size=%d"
+handle_ioreq_read(void *req, uint32_t type, uint32_t df, uint32_t data_is_ptr, uint64_t addr, uint64_t data, uint32_t count, uint32_t size) "I/O=%p read type=%d df=%d ptr=%d port=0x%"PRIx64" data=0x%"PRIx64" count=%d size=%d"
+handle_ioreq_write(void *req, uint32_t type, uint32_t df, uint32_t data_is_ptr, uint64_t addr, uint64_t data, uint32_t count, uint32_t size) "I/O=%p write type=%d df=%d ptr=%d port=0x%"PRIx64" data=0x%"PRIx64" count=%d size=%d"
+cpu_ioreq_pio(void *req, uint32_t dir, uint32_t df, uint32_t data_is_ptr, uint64_t addr, uint64_t data, uint32_t count, uint32_t size) "I/O=%p pio dir=%d df=%d ptr=%d port=0x%"PRIx64" data=0x%"PRIx64" count=%d size=%d"
+cpu_ioreq_pio_read_reg(void *req, uint64_t data, uint64_t addr, uint32_t size) "I/O=%p pio read reg data=0x%"PRIx64" port=0x%"PRIx64" size=%d"
+cpu_ioreq_pio_write_reg(void *req, uint64_t data, uint64_t addr, uint32_t size) "I/O=%p pio write reg data=0x%"PRIx64" port=0x%"PRIx64" size=%d"
+cpu_ioreq_move(void *req, uint32_t dir, uint32_t df, uint32_t data_is_ptr, uint64_t addr, uint64_t data, uint32_t count, uint32_t size) "I/O=%p copy dir=%d df=%d ptr=%d port=0x%"PRIx64" data=0x%"PRIx64" count=%d size=%d"
+xen_map_resource_ioreq(uint32_t id, void *addr) "id: %u addr: %p"
+cpu_ioreq_config_read(void *req, uint32_t sbdf, uint32_t reg, uint32_t size, uint32_t data) "I/O=%p sbdf=0x%x reg=%u size=%u data=0x%x"
+cpu_ioreq_config_write(void *req, uint32_t sbdf, uint32_t reg, uint32_t size, uint32_t data) "I/O=%p sbdf=0x%x reg=%u size=%u data=0x%x"
+
 # xen-mapcache.c
 xen_map_cache(uint64_t phys_addr) "want 0x%"PRIx64
 xen_remap_bucket(uint64_t index) "index 0x%"PRIx64
hw/xen/xen-hvm-common.c new file +858 -0

858 lines not on this page · raw message

include/hw/i386/xen_arch_hvm.h +1 -0
index 1b2c71ba4f..1000f8f543 100644
@@ -2,6 +2,7 @@
 #define HW_XEN_ARCH_I386_HVM_H
 
 #include <xen/hvm/ioreq.h>
+#include "hw/xen/xen-hvm-common.h"
 
 void arch_handle_ioreq(XenIOState *state, ioreq_t *req);
 void arch_xen_set_memory(XenIOState *state,
include/hw/xen/xen-hvm-common.h new file +97 -0

97 lines not on this page · raw message

Signature · 1 line
2.17.0
#17 lore raw

Re: [PATCH v1 06/12] xen-hvm: move common functions to hw/xen/xen-hvm-common.c

Hi Vikram,

Quoted 3 lines · Vikram Garhwal
hw/xen/xen-hvm-common.c new file +3 -0
+void xen_register_ioreq(XenIOState *state, unsigned int max_cpus,
+                        MemoryListener xen_memory_listener)
+{

[...]

hw/xen/xen-hvm-common.c line 852

Quoted 9 lines
hw/xen/xen-hvm-common.c new file +9 -0
+
+    xen_bus_init();
+
+    /* Initialize backend core & drivers */
+    if (xen_be_init() != 0) {
+        error_report("xen backend core setup failed");
+        goto err;
+    }
+    xen_be_register_common();

Calling xen_be_init() and xen_be_register_common() from xen_register_ioreq() sounds wrong to me. There are no dependency between the two. I think it would be better to create a new function to register backends.

Cheers,

Signature · 1 line
Julien Grall
#18 lore raw

Re: [PATCH v1 06/12] xen-hvm: move common functions to hw/xen/xen-hvm-common.c

In reply to Vikram Garhwal

On 15/10/2022 06:07, Vikram Garhwal wrote:
[snip]

hw/xen/xen-hvm-common.c line 844

Quoted 12 lines
hw/xen/xen-hvm-common.c new file +12 -0
+    qemu_add_vm_change_state_handler(xen_hvm_change_state_handler, state);
+
+    state->memory_listener = xen_memory_listener;
+    memory_listener_register(&state->memory_listener, &address_space_memory);
+
+    state->io_listener = xen_io_listener;
+    memory_listener_register(&state->io_listener, &address_space_io);
+
+    state->device_listener = xen_device_listener;
+    QLIST_INIT(&state->dev_list);
+    device_listener_register(&state->device_listener);
+

As Julien said, these do not belong here. These are the (current and legacy) PV backend setup functions; they most certainly have nothing to do with device emulation.

Paul

#19 lore raw

Re: [PATCH v1 06/12] xen-hvm: move common functions to hw/xen/xen-hvm-common.c

Thanks Paul & Julien for reviewing. I will update this in v2.

Regards,
Vikram

From: Paul Durrant <[email protected]>
Date: Wednesday, October 19, 2022 at 9:16 AM
To: Garhwal, Vikram <[email protected]>, [email protected] <[email protected]>
Cc: Stabellini, Stefano <[email protected]>, Stefano Stabellini <[email protected]>, Anthony Perard <[email protected]>, Michael S. Tsirkin <[email protected]>, Marcel Apfelbaum <[email protected]>, Paolo Bonzini <[email protected]>, Richard Henderson <[email protected]>, Eduardo Habkost <[email protected]>, open list:X86 Xen CPUs <[email protected]>
Subject: Re: [PATCH v1 06/12] xen-hvm: move common functions to hw/xen/xen-hvm-common.c
On 15/10/2022 06:07, Vikram Garhwal wrote:
[snip]

hw/xen/xen-hvm-common.c line 844

Quoted 12 lines
hw/xen/xen-hvm-common.c new file +12 -0
+    qemu_add_vm_change_state_handler(xen_hvm_change_state_handler, state);
+
+    state->memory_listener = xen_memory_listener;
+    memory_listener_register(&state->memory_listener, &address_space_memory);
+
+    state->io_listener = xen_io_listener;
+    memory_listener_register(&state->io_listener, &address_space_io);
+
+    state->device_listener = xen_device_listener;
+    QLIST_INIT(&state->dev_list);
+    device_listener_register(&state->device_listener);
+

As Julien said, these do not belong here. These are the (current and legacy) PV backend setup functions; they most certainly have nothing to do with device emulation.

Paul

#20 lore raw

PATCH 7/12 include/hw/xen/xen_common: return error from xen_create_ioreq_server

In reply to Vikram Garhwal

From: Stefano Stabellini <[email protected]>

This is done to prepare for enabling xenpv support for ARM architecture. On ARM it is possible to have a functioning xenpv machine with only the PV backends and no IOREQ server. If the IOREQ server creation fails, continue to the PV backends initialization.

---
 include/hw/xen/xen_common.h | 12 +++++++-----
 1 file changed, 7 insertions(+), 5 deletions(-)
include/hw/xen/xen_common.h +7 -5
index 77ce17d8a4..c2d2f36bde 100644
@@ -467,8 +467,8 @@ static inline void xen_unmap_pcidev(domid_t dom,
 {
 }
 
-static inline void xen_create_ioreq_server(domid_t dom,
-                                           ioservid_t *ioservid)
+static inline int xen_create_ioreq_server(domid_t dom,
+                                          ioservid_t *ioservid)
 {
 }
 
@@ -600,8 +600,8 @@ static inline void xen_unmap_pcidev(domid_t dom,
                                                   PCI_FUNC(pci_dev->devfn));
 }
 
-static inline void xen_create_ioreq_server(domid_t dom,
-                                           ioservid_t *ioservid)
+static inline int xen_create_ioreq_server(domid_t dom,
+                                          ioservid_t *ioservid)
 {
     int rc = xendevicemodel_create_ioreq_server(xen_dmod, dom,
                                                 HVM_IOREQSRV_BUFIOREQ_ATOMIC,
@@ -609,12 +609,14 @@ static inline void xen_create_ioreq_server(domid_t dom,
 
     if (rc == 0) {
         trace_xen_ioreq_server_create(*ioservid);
-        return;
+        return rc;
     }
 
     *ioservid = 0;
     use_default_ioreq_server = true;
     trace_xen_default_ioreq_server();
+
+    return rc;
 }
 
 static inline void xen_destroy_ioreq_server(domid_t dom,
Signature · 1 line
2.17.0
#21 lore raw

Re: [PATCH v1 07/12] include/hw/xen/xen_common: return error from xen_create_ioreq_server

Hi Vikram,

Quoted 17 lines · Vikram Garhwal

From: Stefano Stabellini <[email protected]>

This is done to prepare for enabling xenpv support for ARM architecture. On ARM it is possible to have a functioning xenpv machine with only the PV backends and no IOREQ server. If the IOREQ server creation fails, continue to the PV backends initialization.

---
  include/hw/xen/xen_common.h | 12 +++++++-----
  1 file changed, 7 insertions(+), 5 deletions(-)
include/hw/xen/xen_common.h +0 -0
index 77ce17d8a4..c2d2f36bde 100644

include/hw/xen/xen_common.h static inline void xen_unmap_pcidev(domid_t dom, · line 472

Quoted 8 lines
include/hw/xen/xen_common.h +2 -2
@@ -467,8 +467,8 @@ static inline void xen_unmap_pcidev(domid_t dom,
  {
  }
 
-static inline void xen_create_ioreq_server(domid_t dom,
-                                           ioservid_t *ioservid)
+static inline int xen_create_ioreq_server(domid_t dom,
+                                          ioservid_t *ioservid)
  {

I think there is a return missing here.

Quoted 19 lines
include/hw/xen/xen_common.h +3 -3
  }
 
@@ -600,8 +600,8 @@ static inline void xen_unmap_pcidev(domid_t dom,
                                                    PCI_FUNC(pci_dev->devfn));
  }
 
-static inline void xen_create_ioreq_server(domid_t dom,
-                                           ioservid_t *ioservid)
+static inline int xen_create_ioreq_server(domid_t dom,
+                                          ioservid_t *ioservid)
  {
      int rc = xendevicemodel_create_ioreq_server(xen_dmod, dom,
                                                  HVM_IOREQSRV_BUFIOREQ_ATOMIC,
@@ -609,12 +609,14 @@ static inline void xen_create_ioreq_server(domid_t dom,
 
      if (rc == 0) {
          trace_xen_ioreq_server_create(*ioservid);
-        return;
+        return rc;

include/hw/xen/xen_common.h static inline void xen_create_ioreq_server(domid_t dom, · line 622

Quoted 10 lines
include/hw/xen/xen_common.h +2 -0
      }
 
      *ioservid = 0;
      use_default_ioreq_server = true;
      trace_xen_default_ioreq_server();
+
+    return rc;
  }
 
  static inline void xen_destroy_ioreq_server(domid_t dom,

Cheers,

Signature · 1 line
Julien Grall
#22 lore raw

PATCH 8/12 hw/xen/xen-hvm-common: skip ioreq creation on ioreq registration failure

In reply to Vikram Garhwal

From: Stefano Stabellini <[email protected]>

On ARM it is possible to have a functioning xenpv machine with only the PV backends and no IOREQ server. If the IOREQ server creation fails continue to the PV backends initialization.

---
 hw/xen/xen-hvm-common.c | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)
hw/xen/xen-hvm-common.c +6 -1
index f848f9e625..7bccf595fc 100644
@@ -777,7 +777,11 @@ void xen_register_ioreq(XenIOState *state, unsigned int max_cpus,
         goto err;
     }
 
-    xen_create_ioreq_server(xen_domid, &state->ioservid);
+    rc = xen_create_ioreq_server(xen_domid, &state->ioservid);
+    if (rc) {
+        DPRINTF("xen: failed to create ioreq server\n");
+        goto no_ioreq;
+    }
 
     state->exit.notify = xen_exit_notifier;
     qemu_add_exit_notifier(&state->exit);
@@ -842,6 +846,7 @@ void xen_register_ioreq(XenIOState *state, unsigned int max_cpus,
     QLIST_INIT(&state->dev_list);
     device_listener_register(&state->device_listener);
 
+no_ioreq:
     xen_bus_init();
 
     /* Initialize backend core & drivers */
Signature · 1 line
2.17.0
#23 lore raw

Re: [PATCH v1 08/12] hw/xen/xen-hvm-common: skip ioreq creation on ioreq registration failure

Quoted 16 lines · Vikram Garhwal

From: Stefano Stabellini <[email protected]>

On ARM it is possible to have a functioning xenpv machine with only the PV backends and no IOREQ server. If the IOREQ server creation fails continue to the PV backends initialization.

---
 hw/xen/xen-hvm-common.c | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)
hw/xen/xen-hvm-common.c +0 -0
index f848f9e625..7bccf595fc 100644

hw/xen/xen-hvm-common.c void xen_register_ioreq(XenIOState *state, unsigned int max_cpus, · line 782

Quoted 7 lines
hw/xen/xen-hvm-common.c +3 -1
@@ -777,7 +777,11 @@ void xen_register_ioreq(XenIOState *state, unsigned int max_cpus,
         goto err;
     }
 
-    xen_create_ioreq_server(xen_domid, &state->ioservid);
+    rc = xen_create_ioreq_server(xen_domid, &state->ioservid);
+    if (rc) {
+        DPRINTF("xen: failed to create ioreq server\n");

This should be a warn_report to properly inform the user.

hw/xen/xen-hvm-common.c void xen_register_ioreq(XenIOState *state, unsigned int max_cpus, · line 783

Quoted 1 line
hw/xen/xen-hvm-common.c +1 -0
+        goto no_ioreq;

Maybe pushing the rest of this function into a local subroutine would reduce the amount of goto messing about. Other candidates for cleaning up/modernising:

- g_malloc to g_new0
- perror -> error_setg_errno

Quoted 12 lines
hw/xen/xen-hvm-common.c +2 -0
+    }
 
     state->exit.notify = xen_exit_notifier;
     qemu_add_exit_notifier(&state->exit);
@@ -842,6 +846,7 @@ void xen_register_ioreq(XenIOState *state, unsigned int max_cpus,
     QLIST_INIT(&state->dev_list);
     device_listener_register(&state->device_listener);
 
+no_ioreq:
     xen_bus_init();
 
     /* Initialize backend core & drivers */
Signature · 1 line
Alex Bennée
#24 lore raw

PATCH 9/12 accel/xen/xen-all: export xenstore_record_dm_state

In reply to Vikram Garhwal

xenstore_record_dm_state() will also be used in aarch64 xenpv machine.

---
 accel/xen/xen-all.c  | 2 +-
 include/hw/xen/xen.h | 2 ++
 2 files changed, 3 insertions(+), 1 deletion(-)
accel/xen/xen-all.c +1 -1
index 69aa7d018b..276625b78b 100644
@@ -100,7 +100,7 @@ void xenstore_store_pv_console_info(int i, Chardev *chr)
 }
 
 
-static void xenstore_record_dm_state(struct xs_handle *xs, const char *state)
+void xenstore_record_dm_state(struct xs_handle *xs, const char *state)
 {
     char path[50];
 
include/hw/xen/xen.h +2 -0
index afdf9c436a..31e9538a5c 100644
@@ -9,6 +9,7 @@
  */
 
 #include "exec/cpu-common.h"
+#include <xenstore.h>
 
 /* xen-machine.c */
 enum xen_mode {
@@ -31,5 +32,6 @@ qemu_irq *xen_interrupt_controller_init(void);
 void xenstore_store_pv_console_info(int i, Chardev *chr);
 
 void xen_register_framebuffer(struct MemoryRegion *mr);
+void xenstore_record_dm_state(struct xs_handle *xs, const char *state);
 
 #endif /* QEMU_HW_XEN_H */
Signature · 1 line
2.17.0
#26 lore raw

Re: [PATCH v1 09/12] accel/xen/xen-all: export xenstore_record_dm_state

In reply to Vikram Garhwal
Quoted 27 lines · Vikram Garhwal

xenstore_record_dm_state() will also be used in aarch64 xenpv machine.

---
 accel/xen/xen-all.c  | 2 +-
 include/hw/xen/xen.h | 2 ++
 2 files changed, 3 insertions(+), 1 deletion(-)
accel/xen/xen-all.c +1 -1
index 69aa7d018b..276625b78b 100644
@@ -100,7 +100,7 @@ void xenstore_store_pv_console_info(int i, Chardev *chr)
 }
 
 
-static void xenstore_record_dm_state(struct xs_handle *xs, const char *state)
+void xenstore_record_dm_state(struct xs_handle *xs, const char *state)
 {
     char path[50];
 
include/hw/xen/xen.h +0 -0
index afdf9c436a..31e9538a5c 100644

include/hw/xen/xen.h line 12

Quoted 4 lines
include/hw/xen/xen.h +1 -0
@@ -9,6 +9,7 @@
  */
 
 #include "exec/cpu-common.h"
+#include <xenstore.h>

This is breaking a bunch of the builds and generally we try and avoid adding system includes in headers (apart from osdep.h) for this reason. In fact there is a comment just above to that fact.

I think you can just add struct xs_handle to typedefs.h (or maybe just xen.h) and directly include xenstore.h in xen-all.c following the usual rules:

https://qemu.readthedocs.io/en/latest/devel/style.html#include-directives

It might be worth doing an audit to see what else is including xen.h needlessly or should be using sysemu/xen.h.

Quoted 10 lines
include/hw/xen/xen.h +1 -0
 /* xen-machine.c */
 enum xen_mode {
@@ -31,5 +32,6 @@ qemu_irq *xen_interrupt_controller_init(void);
 void xenstore_store_pv_console_info(int i, Chardev *chr);
 
 void xen_register_framebuffer(struct MemoryRegion *mr);
+void xenstore_record_dm_state(struct xs_handle *xs, const char *state);
 
 #endif /* QEMU_HW_XEN_H */
Signature · 1 line
Alex Bennée
#27 lore raw

Re: [PATCH v1 09/12] accel/xen/xen-all: export xenstore_record_dm_state

On Thu, Oct 27, 2022 at 12:24 PM Alex Bennée <[email protected]> wrote:

Hello all

Quoted 56 lines

accel/xen/xen-all.c void xenstore_store_pv_console_info(int i, Chardev

Quoted 14 lines · Vikram Garhwal

xenstore_record_dm_state() will also be used in aarch64 xenpv machine.

---
 accel/xen/xen-all.c  | 2 +-
 include/hw/xen/xen.h | 2 ++
 2 files changed, 3 insertions(+), 1 deletion(-)
accel/xen/xen-all.c +0 -0
index 69aa7d018b..276625b78b 100644
@@ -100,7 +100,7 @@ void xenstore_store_pv_console_info(int i, Chardev

*chr)

accel/xen/xen-all.c void xenstore_store_pv_console_info(int i, Chardev · line 103

Quoted 4 lines
accel/xen/xen-all.c +0 -1
 }
 
 
-static void xenstore_record_dm_state(struct xs_handle *xs, const char

*state)

include/hw/xen/xen.h line 12

Quoted 13 lines
accel/xen/xen-all.c +1 -0
+void xenstore_record_dm_state(struct xs_handle *xs, const char *state)
 {
     char path[50];
 
include/hw/xen/xen.h +1 -0
index afdf9c436a..31e9538a5c 100644
@@ -9,6 +9,7 @@
  */
 
 #include "exec/cpu-common.h"
+#include <xenstore.h>

This is breaking a bunch of the builds and generally we try and avoid adding system includes in headers (apart from osdep.h) for this reason. In fact there is a comment just above to that fact.

I think you can just add struct xs_handle to typedefs.h (or maybe just xen.h) and directly include xenstore.h in xen-all.c following the usual rules:

https://qemu.readthedocs.io/en/latest/devel/style.html#include-directives

It might be worth doing an audit to see what else is including xen.h needlessly or should be using sysemu/xen.h.

Quoted 6 lines
include/hw/xen/xen.h +0 -0
 /* xen-machine.c */
 enum xen_mode {
@@ -31,5 +32,6 @@ qemu_irq *xen_interrupt_controller_init(void);
 void xenstore_store_pv_console_info(int i, Chardev *chr);
 
Quoted 10 lines

include/hw/xen/xen.h qemu_irq *xen_interrupt_controller_init(void); · line 37

Quoted 4 lines
include/hw/xen/xen.h +1 -0
 void xen_register_framebuffer(struct MemoryRegion *mr);
+void xenstore_record_dm_state(struct xs_handle *xs, const char *state);
 
 #endif /* QEMU_HW_XEN_H */

--
Alex Bennée

For considering:
I think this patch and some other changes done in "[PATCH v1 10/12] hw/arm:
introduce xenpv machine" (the opening of Xen interfaces and calling xenstore_record_dm_state() in hw/arm/xen_arm.c:xen_init_ioreq()) could be avoided if we enable the Xen accelerator (either by passing "-M xenpv,accel=xen" or by adding mc->default_machine_opts = "accel=xen"; to hw/arm/xen_arm.c:xen_arm_machine_class_init() or by some other method). These actions are already done in accel/xen/xen-all.c:xen_init(). Please note, that I am not too familiar with that code, so there might be nuances.

Besides that, Xen accelerator will be needed for the xen-mapcache to be in use (this is needed for mapping guest memory), there are a few xen_enabled() checks spreading around that code to perform Xen specific actions.

Signature · 3 lines
Regards,

Oleksandr Tyshchenko
#28 lore raw

Re: [PATCH v1 09/12] accel/xen/xen-all: export xenstore_record_dm_state

Hi Oleksandr,

Quoted 79 lines · Oleksandr Tyshchenko

On Thu, Oct 27, 2022 at 12:24 PM Alex Bennée <[email protected]> wrote:

Hello all

    Vikram Garhwal <[email protected]> writes:

    > xenstore_record_dm_state() will also be used in aarch64 xenpv
    machine.
    >
    > Signed-off-by: Vikram Garhwal <[email protected]>
    > Signed-off-by: Stefano Stabellini <[email protected]>
    > ---
    >  accel/xen/xen-all.c  | 2 +-
    >  include/hw/xen/xen.h | 2 ++
    >  2 files changed, 3 insertions(+), 1 deletion(-)
    >
    > diff --git a/accel/xen/xen-all.c b/accel/xen/xen-all.c
    > index 69aa7d018b..276625b78b 100644
    > --- a/accel/xen/xen-all.c
    > +++ b/accel/xen/xen-all.c
    > @@ -100,7 +100,7 @@ void xenstore_store_pv_console_info(int i,
    Chardev *chr)
    >  }
    >
    >
    > -static void xenstore_record_dm_state(struct xs_handle *xs,
    const char *state)
    > +void xenstore_record_dm_state(struct xs_handle *xs, const char
    *state)
    >  {
    >      char path[50];
    >
    > diff --git a/include/hw/xen/xen.h b/include/hw/xen/xen.h
    > index afdf9c436a..31e9538a5c 100644
    > --- a/include/hw/xen/xen.h
    > +++ b/include/hw/xen/xen.h
    > @@ -9,6 +9,7 @@
    >   */
    >
    >  #include "exec/cpu-common.h"
    > +#include <xenstore.h>

    This is breaking a bunch of the builds and generally we try and avoid
    adding system includes in headers (apart from osdep.h) for this
    reason.
    In fact there is a comment just above to that fact.

    I think you can just add struct xs_handle to typedefs.h (or maybe just
    xen.h) and directly include xenstore.h in xen-all.c following the
    usual
    rules:

    https://qemu.readthedocs.io/en/latest/devel/style.html#include-directives

    It might be worth doing an audit to see what else is including xen.h
    needlessly or should be using sysemu/xen.h.

    >
    >  /* xen-machine.c */
    >  enum xen_mode {
    > @@ -31,5 +32,6 @@ qemu_irq *xen_interrupt_controller_init(void);
    >  void xenstore_store_pv_console_info(int i, Chardev *chr);
    >
    >  void xen_register_framebuffer(struct MemoryRegion *mr);
    > +void xenstore_record_dm_state(struct xs_handle *xs, const char
    *state);
    >
    >  #endif /* QEMU_HW_XEN_H */


    -- 
    Alex Bennée
Quoted 7 lines

For considering:
I think this patch and some other changes done in "[PATCH v1 10/12] hw/arm: introduce xenpv machine" (the opening of Xen interfaces and calling xenstore_record_dm_state() in hw/arm/xen_arm.c:xen_init_ioreq()) could be avoided if we enable the Xen accelerator (either by passing "-M xenpv,accel=xen" or by adding mc->default_machine_opts = "accel=xen"; to hw/arm/xen_arm.c:xen_arm_machine_class_init() or by some other method). These actions are already done in accel/xen/xen-all.c:xen_init(). Please note, that I am not too familiar with that code, so there might be nuances.

Besides that, Xen accelerator will be needed for the xen-mapcache to be in use (this is needed for mapping guest memory), there are a few xen_enabled() checks spreading around that code to perform Xen specific actions.

Unfortunately, I am not that familiar with xen as accelerator function. Let me check and get back to you.

Quoted 4 lines
Signature · 3 lines
Regards,

Oleksandr Tyshchenko
#29 lore raw

Re: [PATCH v1 09/12] accel/xen/xen-all: export xenstore_record_dm_state

In reply to Alex Bennée

Thanks, Alex, for reviewing this one. I built for all the archs and it was fine. Can you please share more about what environment builds are breaking? So, I can test the changes for v2.

Regards,
Vikram

From: Alex Bennée <[email protected]>
Date: Thursday, October 27, 2022 at 2:24 AM
To: Garhwal, Vikram <[email protected]>
Cc: [email protected] <[email protected]>, Stabellini, Stefano <[email protected]>, Stefano Stabellini <[email protected]>, Anthony Perard <[email protected]>, Paul Durrant <[email protected]>, [email protected] <[email protected]>
Subject: Re: [PATCH v1 09/12] accel/xen/xen-all: export xenstore_record_dm_state

Quoted 27 lines · Vikram Garhwal

xenstore_record_dm_state() will also be used in aarch64 xenpv machine.

---
 accel/xen/xen-all.c  | 2 +-
 include/hw/xen/xen.h | 2 ++
 2 files changed, 3 insertions(+), 1 deletion(-)
accel/xen/xen-all.c +1 -1
index 69aa7d018b..276625b78b 100644
@@ -100,7 +100,7 @@ void xenstore_store_pv_console_info(int i, Chardev *chr)
 }
 
 
-static void xenstore_record_dm_state(struct xs_handle *xs, const char *state)
+void xenstore_record_dm_state(struct xs_handle *xs, const char *state)
 {
     char path[50];
 
include/hw/xen/xen.h +0 -0
index afdf9c436a..31e9538a5c 100644

include/hw/xen/xen.h line 12

Quoted 4 lines
include/hw/xen/xen.h +1 -0
@@ -9,6 +9,7 @@
  */
 
 #include "exec/cpu-common.h"
+#include <xenstore.h>

This is breaking a bunch of the builds and generally we try and avoid adding system includes in headers (apart from osdep.h) for this reason. In fact there is a comment just above to that fact.

I think you can just add struct xs_handle to typedefs.h (or maybe just xen.h) and directly include xenstore.h in xen-all.c following the usual rules:

https://qemu.readthedocs.io/en/latest/devel/style.html#include-directives

It might be worth doing an audit to see what else is including xen.h needlessly or should be using sysemu/xen.h.

include/hw/xen/xen.h qemu_irq *xen_interrupt_controller_init(void); · line 37

Quoted 10 lines
include/hw/xen/xen.h +1 -0
 /* xen-machine.c */
 enum xen_mode {
@@ -31,5 +32,6 @@ qemu_irq *xen_interrupt_controller_init(void);
 void xenstore_store_pv_console_info(int i, Chardev *chr);
 
 void xen_register_framebuffer(struct MemoryRegion *mr);
+void xenstore_record_dm_state(struct xs_handle *xs, const char *state);
 
 #endif /* QEMU_HW_XEN_H */

--
Alex Bennée

#30 lore raw

Re: [PATCH v1 09/12] accel/xen/xen-all: export xenstore_record_dm_state

Quoted 2 lines · Vikram

Thanks, Alex, for reviewing this one. I built for all the archs and it was fine. Can you please share more about what environment builds are breaking? So, I can test the changes for v2.

My cross build environment failed:

../../configure' '--disable-docs' '--disable-tools' '--cross-prefix=aarch64-linux-gnu-' '--enable-xen' '--target-list=i386-softmmu,x86_64-softmmu,arm-softmmu,aarch64-softmmu' '--disable-tpm'

On a Debian Bullseye with:

11:30:20 [root@zen:~] # dpkg -l libxen\*
Desired=Unknown/Install/Remove/Purge/Hold

  | Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
  |/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
  ||/ Name                       Version                 Architecture Description
  +++-==========================-=======================-============-====================================================
  ii  libxen-dev:arm64           4.14.5+24-g87d90d511c-1 arm64        Public headers and libs for Xen
  ii  libxencall1:amd64          4.14.5+24-g87d90d511c-1 amd64        Xen runtime library - libxencall
  ii  libxencall1:arm64          4.14.5+24-g87d90d511c-1 arm64        Xen runtime library - libxencall
  ii  libxendevicemodel1:amd64   4.14.5+24-g87d90d511c-1 amd64        Xen runtime libraries - libxendevicemodel
  ii  libxendevicemodel1:arm64   4.14.5+24-g87d90d511c-1 arm64        Xen runtime libraries - libxendevicemodel
  ii  libxenevtchn1:amd64        4.14.5+24-g87d90d511c-1 amd64        Xen runtime libraries - libxenevtchn
  ii  libxenevtchn1:arm64        4.14.5+24-g87d90d511c-1 arm64        Xen runtime libraries - libxenevtchn
  ii  libxenforeignmemory1:amd64 4.14.5+24-g87d90d511c-1 amd64        Xen runtime libraries - libxenforeignmemory
  ii  libxenforeignmemory1:arm64 4.14.5+24-g87d90d511c-1 arm64        Xen runtime libraries - libxenforeignmemory
  ii  libxengnttab1:amd64        4.14.5+24-g87d90d511c-1 amd64        Xen runtime libraries - libxengnttab
  ii  libxengnttab1:arm64        4.14.5+24-g87d90d511c-1 arm64        Xen runtime libraries - libxengnttab
  ii  libxenhypfs1:amd64         4.14.5+24-g87d90d511c-1 amd64        Xen runtime library - libxenhypfs
  ii  libxenhypfs1:arm64         4.14.5+24-g87d90d511c-1 arm64        Xen runtime library - libxenhypfs
  ii  libxenmisc4.14:amd64       4.14.5+24-g87d90d511c-1 amd64        Xen runtime libraries - miscellaneous, versioned ABI
  ii  libxenmisc4.14:arm64       4.14.5+24-g87d90d511c-1 arm64        Xen runtime libraries - miscellaneous, versioned ABI
  ii  libxenstore3.0:amd64       4.14.5+24-g87d90d511c-1 amd64        Xen runtime libraries - libxenstore
  ii  libxenstore3.0:arm64       4.14.5+24-g87d90d511c-1 arm64        Xen runtime libraries - libxenstore
  ii  libxentoolcore1:amd64      4.14.5+24-g87d90d511c-1 amd64        Xen runtime libraries - libxentoolcore
  ii  libxentoolcore1:arm64      4.14.5+24-g87d90d511c-1 arm64        Xen runtime libraries - libxentoolcore
  ii  libxentoollog1:amd64       4.14.5+24-g87d90d511c-1 amd64        Xen runtime libraries - libxentoollog
  ii  libxentoollog1:arm64       4.14.5+24-g87d90d511c-1 arm64        Xen runtime libraries - libxentoollog

But also a bunch of cross builds on the CI system:

https://gitlab.com/stsquad/qemu/-/pipelines/677956972/failures

Quoted 74 lines

Regards,

Vikram

From: Alex Bennée <[email protected]>
Date: Thursday, October 27, 2022 at 2:24 AM
To: Garhwal, Vikram <[email protected]>
Cc: [email protected] <[email protected]>, Stabellini, Stefano <[email protected]>, Stefano Stabellini <[email protected]>, Anthony Perard <[email protected]>, Paul Durrant <[email protected]>, [email protected] <[email protected]>
Subject: Re: [PATCH v1 09/12] accel/xen/xen-all: export xenstore_record_dm_state

Quoted 27 lines · Vikram Garhwal

xenstore_record_dm_state() will also be used in aarch64 xenpv machine.

---
 accel/xen/xen-all.c  | 2 +-
 include/hw/xen/xen.h | 2 ++
 2 files changed, 3 insertions(+), 1 deletion(-)
accel/xen/xen-all.c +1 -1
index 69aa7d018b..276625b78b 100644
@@ -100,7 +100,7 @@ void xenstore_store_pv_console_info(int i, Chardev *chr)
 }
 
 
-static void xenstore_record_dm_state(struct xs_handle *xs, const char *state)
+void xenstore_record_dm_state(struct xs_handle *xs, const char *state)
 {
     char path[50];
 
include/hw/xen/xen.h +0 -0
index afdf9c436a..31e9538a5c 100644

include/hw/xen/xen.h line 12

Quoted 4 lines
include/hw/xen/xen.h +1 -0
@@ -9,6 +9,7 @@
  */
 
 #include "exec/cpu-common.h"
+#include <xenstore.h>

This is breaking a bunch of the builds and generally we try and avoid adding system includes in headers (apart from osdep.h) for this reason. In fact there is a comment just above to that fact.

I think you can just add struct xs_handle to typedefs.h (or maybe just xen.h) and directly include xenstore.h in xen-all.c following the usual rules:

https://qemu.readthedocs.io/en/latest/devel/style.html#include-directives

It might be worth doing an audit to see what else is including xen.h needlessly or should be using sysemu/xen.h.

Quoted 10 lines
include/hw/xen/xen.h +1 -0
 /* xen-machine.c */
 enum xen_mode {
@@ -31,5 +32,6 @@ qemu_irq *xen_interrupt_controller_init(void);
 void xenstore_store_pv_console_info(int i, Chardev *chr);
 
 void xen_register_framebuffer(struct MemoryRegion *mr);
+void xenstore_record_dm_state(struct xs_handle *xs, const char *state);
 
 #endif /* QEMU_HW_XEN_H */
Signature · 1 line
Alex Bennée
#31 lore raw

PATCH 10/12 hw/arm: introduce xenpv machine

In reply to Vikram Garhwal

Add a new machine xenpv which creates a IOREQ server to register/connect with Xen Hypervisor.

Xen IOREQ connection expect the TARGET_PAGE_SIZE to 4096, and the xenpv machine on ARM will have no CPU definitions. We need to define TARGET_PAGE_SIZE appropriately ourselves.

Optional: When CONFIG_TPM is enabled, it also creates a tpm-tis-device, adds a TPM emulator and connects to swtpm running on host machine via chardev socket and support TPM functionalities for a guest domain.

Extra command line for aarch64 xenpv QEMU to connect to swtpm:

    -chardev socket,id=chrtpm,path=/tmp/myvtpm2/swtpm-sock \
    -tpmdev emulator,id=tpm0,chardev=chrtpm \

swtpm implements a TPM software emulator(TPM 1.2 & TPM 2) built on libtpms and provides access to TPM functionality over socket, chardev and CUSE interface. Github repo: https://github.com/stefanberger/swtpm
Example for starting swtpm on host machine:
mkdir /tmp/vtpm2
swtpm socket --tpmstate dir=/tmp/vtpm2 \
--ctrl type=unixio,path=/tmp/vtpm2/swtpm-sock &

---
 hw/arm/meson.build            |   2 +
 hw/arm/xen_arm.c              | 163 ++++++++++++++++++++++++++++++++++
 include/hw/arm/xen_arch_hvm.h |  12 +++
 include/hw/xen/arch_hvm.h     |   2 +
 4 files changed, 179 insertions(+)
 create mode 100644 hw/arm/xen_arm.c
 create mode 100644 include/hw/arm/xen_arch_hvm.h
hw/arm/meson.build +2 -0
index 92f9f6e000..0cae024374 100644
@@ -62,5 +62,7 @@ arm_ss.add(when: 'CONFIG_FSL_IMX7', if_true: files('fsl-imx7.c', 'mcimx7d-sabre.
 arm_ss.add(when: 'CONFIG_ARM_SMMUV3', if_true: files('smmu-common.c', 'smmuv3.c'))
 arm_ss.add(when: 'CONFIG_FSL_IMX6UL', if_true: files('fsl-imx6ul.c', 'mcimx6ul-evk.c'))
 arm_ss.add(when: 'CONFIG_NRF51_SOC', if_true: files('nrf51_soc.c'))
+arm_ss.add(when: 'CONFIG_XEN', if_true: files('xen_arm.c'))
+arm_ss.add_all(xen_ss)
 
 hw_arch += {'arm': arm_ss}
hw/arm/xen_arm.c new file +163 -0

163 lines not on this page · raw message

include/hw/arm/xen_arch_hvm.h new file +12 -0
new file mode 100644
index 0000000000..f645dfec28
@@ -0,0 +1,12 @@
+#ifndef HW_XEN_ARCH_ARM_HVM_H
+#define HW_XEN_ARCH_ARM_HVM_H
+
+#include <xen/hvm/ioreq.h>
+void arch_handle_ioreq(XenIOState *state, ioreq_t *req);
+void arch_xen_set_memory(XenIOState *state,
+                         MemoryRegionSection *section,
+                         bool add);
+
+#undef TARGET_PAGE_SIZE
+#define TARGET_PAGE_SIZE 4096
+#endif
include/hw/xen/arch_hvm.h +2 -0
index 26674648d8..c7c515220d 100644
@@ -1,3 +1,5 @@
 #if defined(TARGET_I386) || defined(TARGET_X86_64)
 #include "hw/i386/xen_arch_hvm.h"
+#elif defined(TARGET_ARM) || defined(TARGET_ARM_64)
+#include "hw/arm/xen_arch_hvm.h"
 #endif
Signature · 1 line
2.17.0
#32 lore raw

Re: [PATCH v1 10/12] hw/arm: introduce xenpv machine

Hi,

There seem to be some missing patches on xen-devel (including the cover letter). Is that expected?

Quoted 2 lines · Vikram Garhwal

Add a new machine xenpv which creates a IOREQ server to register/connect with Xen Hypervisor.

I don't like the name 'xenpv' because it doesn't convey the fact that some of the HW may be emulated rather than para-virtualized. In fact one may only want to use for emulating devices.

Potential name would be 'xen-arm' or re-using 'virt' but with 'accel=xen' to select a Xen layout.

Quoted 13 lines

Xen IOREQ connection expect the TARGET_PAGE_SIZE to 4096, and the xenpv machine on ARM will have no CPU definitions. We need to define TARGET_PAGE_SIZE appropriately ourselves.

Optional: When CONFIG_TPM is enabled, it also creates a tpm-tis-device, adds a TPM emulator and connects to swtpm running on host machine via chardev socket and support TPM functionalities for a guest domain.

Extra command line for aarch64 xenpv QEMU to connect to swtpm:

     -chardev socket,id=chrtpm,path=/tmp/myvtpm2/swtpm-sock \
     -tpmdev emulator,id=tpm0,chardev=chrtpm \
Quoted 7 lines

swtpm implements a TPM software emulator(TPM 1.2 & TPM 2) built on libtpms and provides access to TPM functionality over socket, chardev and CUSE interface. Github repo: https://github.com/stefanberger/swtpm
Example for starting swtpm on host machine:
mkdir /tmp/vtpm2
swtpm socket --tpmstate dir=/tmp/vtpm2 \
--ctrl type=unixio,path=/tmp/vtpm2/swtpm-sock &

I see patches for QEMU but not Xen. How can this be tested with existing Xen? Will libxl ever create QEMU?

[...]

hw/arm/xen_arm.c line 83

Quoted 6 lines
hw/arm/xen_arm.c new file +6 -0
+static int xen_init_ioreq(XenIOState *state, unsigned int max_cpus)
+{
+    xen_dmod = xendevicemodel_open(0, 0);
+    xen_xc = xc_interface_open(0, 0, 0);
+
+    if (xen_xc == NULL) {

You are checking xen_xc but not xen_dmod. Why?

Quoted 27 lines
hw/arm/xen_arm.c new file +27 -0
+        perror("xen: can't open xen interface\n");
+        return -1;
+    }
+
+    xen_fmem = xenforeignmemory_open(0, 0);
+    if (xen_fmem == NULL) {
+        perror("xen: can't open xen fmem interface\n");
+        xc_interface_close(xen_xc);
+        return -1;
+    }
+
+    xen_register_ioreq(state, max_cpus, xen_memory_listener);
+
+    xenstore_record_dm_state(xenstore, "running");
+
+    return 0;
+}
+
+static void xen_enable_tpm(void)
+{
+/* qemu_find_tpm_be is only available when CONFIG_TPM is enabled. */
+#ifdef CONFIG_TPM
+    Error *errp = NULL;
+    DeviceState *dev;
+    SysBusDevice *busdev;
+
+    TPMBackend *be = qemu_find_tpm_be("tpm0");

hw/arm/xen_arm.c line 120

Quoted 10 lines
hw/arm/xen_arm.c new file +10 -0
+    if (be == NULL) {
+        DPRINTF("Couldn't fine the backend for tpm0\n");
+        return;
+    }
+    dev = qdev_new(TYPE_TPM_TIS_SYSBUS);
+    object_property_set_link(OBJECT(dev), "tpmdev", OBJECT(be), &errp);
+    object_property_set_str(OBJECT(dev), "tpmdev", be->id, &errp);
+    busdev = SYS_BUS_DEVICE(dev);
+    sysbus_realize_and_unref(busdev, &error_fatal);
+    sysbus_mmio_map(busdev, 0, GUEST_TPM_BASE);

I can't find where GUEST_TPM_BASE is defined. But then the guest memory layout is not expected to be stable. With your current approach, it means QEMU would need to be rebuilt for every Xen version. Is it what we want?

hw/arm/xen_arm.c line 133

Quoted 13 lines
hw/arm/xen_arm.c new file +13 -0
+
+    DPRINTF("Connected tpmdev at address 0x%lx\n", GUEST_TPM_BASE);
+#endif
+}
+
+static void xen_arm_init(MachineState *machine)
+{
+    XenArmState *xam = XEN_ARM(machine);
+
+    xam->state =  g_new0(XenIOState, 1);
+
+    if (xen_init_ioreq(xam->state, machine->smp.cpus)) {
+        return;

In another patch, you said the IOREQ would be optional. IHMO, I think this is a bad idea to register it by default because one may only want to use PV drivers. Registering IOREQ will add unnecessary overhead in Xen.

Furthermore, it means that someone selecting TPM but Xen is not built with CONFIG_IOREQ=y (BTW This is still a tech preview but there are security holes on Arm...) will not get an error. Instead, the OS will until it crashes when trying to access the TPM.

Overall I think it would be better if IOREQ is only registered when a device requires (like TPM) it *and* throw an error if there is a problem during the initialization.

Quoted 4 lines
hw/arm/xen_arm.c new file +4 -0
+    } > +
+    xen_enable_tpm();
+
+    return;

hw/arm/xen_arm.c line 148

Quoted 10 lines
hw/arm/xen_arm.c new file +10 -0
+}
+
+static void xen_arm_machine_class_init(ObjectClass *oc, void *data)
+{
+
+    MachineClass *mc = MACHINE_CLASS(oc);
+    mc->desc = "Xen Para-virtualized PC";
+    mc->init = xen_arm_init;
+    mc->max_cpus = 1;
+    machine_class_allow_dynamic_sysbus_dev(mc, TYPE_TPM_TIS_SYSBUS);

Shouldn't this be protected with #ifdef CONFIG_TPM?

Quoted 21 lines
hw/arm/xen_arm.c new file +15 -0
+}
+
+static const TypeInfo xen_arm_machine_type = {
+    .name = TYPE_XEN_ARM,
+    .parent = TYPE_MACHINE,
+    .class_init = xen_arm_machine_class_init,
+    .instance_size = sizeof(XenArmState),
+};
+
+static void xen_arm_machine_register_types(void)
+{
+    type_register_static(&xen_arm_machine_type);
+}
+
+type_init(xen_arm_machine_register_types)
include/hw/arm/xen_arch_hvm.h new file +0 -0
new file mode 100644
index 0000000000..f645dfec28

include/hw/arm/xen_arch_hvm.h line 10

Quoted 10 lines
include/hw/arm/xen_arch_hvm.h new file +10 -0
@@ -0,0 +1,12 @@
+#ifndef HW_XEN_ARCH_ARM_HVM_H
+#define HW_XEN_ARCH_ARM_HVM_H
+
+#include <xen/hvm/ioreq.h>
+void arch_handle_ioreq(XenIOState *state, ioreq_t *req);
+void arch_xen_set_memory(XenIOState *state,
+                         MemoryRegionSection *section,
+                         bool add);
+
+#undef TARGET_PAGE_SIZE

I am a bit puzzled with this #undef. In the commit message you said that there will be no CPU definition. So the implications is this should not be defined.

If it is defined, then what guarantees that all the source will use the correct value?

include/hw/arm/xen_arch_hvm.h line 11

Quoted 1 line
include/hw/arm/xen_arch_hvm.h new file +1 -0
+#define TARGET_PAGE_SIZE 4096

It would be better to use XC_PAGE_SIZE (or similar) rather than hardcoding it.

include/hw/xen/arch_hvm.h line 5

Quoted 11 lines
include/hw/arm/xen_arch_hvm.h new file +1 -0
+#endif
include/hw/xen/arch_hvm.h +2 -0
index 26674648d8..c7c515220d 100644
@@ -1,3 +1,5 @@
  #if defined(TARGET_I386) || defined(TARGET_X86_64)
  #include "hw/i386/xen_arch_hvm.h"
+#elif defined(TARGET_ARM) || defined(TARGET_ARM_64)
+#include "hw/arm/xen_arch_hvm.h"
  #endif

Cheers,

Signature · 1 line
Julien Grall
#33 lore raw

Re: [PATCH v1 10/12] hw/arm: introduce xenpv machine

Quoted 10 lines · Julien Grall

Hi,

There seem to be some missing patches on xen-devel (including the cover letter). Is that expected?

Quoted 3 lines · Vikram Garhwal

Add a new machine xenpv which creates a IOREQ server to register/connect with
Xen Hypervisor.

Quoted 6 lines

I don't like the name 'xenpv' because it doesn't convey the fact that some of the HW may be emulated rather than para-virtualized. In fact one may only want to use for emulating devices.

Potential name would be 'xen-arm' or re-using 'virt' but with 'accel=xen' to select a Xen layout.

The benefit of 'xenpv' is that it doesn't require any changes to libxl. It is even backward compatible so it could be used with an older version of Xen/libxl. Backward compatibility aside, if we come up with a different name then we'll need changes to libxl and to manage those changes. For instance, if we use 'xen-arm' that would mean we would need to handle per-arch QEMU machine names.

#34 lore raw

Re: [PATCH v1 10/12] hw/arm: introduce xenpv machine

Hi Stefano,

Quoted 11 lines · Stefano Stabellini
Quoted 10 lines · Julien Grall

Hi,

There seem to be some missing patches on xen-devel (including the cover letter). Is that expected?

Quoted 3 lines · Vikram Garhwal

Add a new machine xenpv which creates a IOREQ server to register/connect with
Xen Hypervisor.

Quoted 8 lines
Quoted 6 lines

I don't like the name 'xenpv' because it doesn't convey the fact that some of the HW may be emulated rather than para-virtualized. In fact one may only want to use for emulating devices.

Potential name would be 'xen-arm' or re-using 'virt' but with 'accel=xen' to select a Xen layout.

The benefit of 'xenpv' is that it doesn't require any changes to libxl.

I am quite surprised. Looking at the code, it seems to work more by chance than it is intentional as the code is gated by libxl__need_xenpv_qemu(). So it would not start if there were no emulated devices.

Quoted 2 lines

It is even backward compatible so it could be used with an older version of Xen/libxl.

We don't really gain much here. IOREQ is a tech preview and anyone that wants to try it should really use the latest Xen.

Quoted 4 lines

Backward compatibility aside, if we come up with a
different name then we'll need changes to libxl and to manage those changes. For instance, if we use 'xen-arm' that would mean we would need to handle per-arch QEMU machine names.

Right, so the main argument here is for simplicity in libxl
Looking at how 'xenpv' is built, this is really expected to deal with PV backend rather than emulated device. I do expect some changes as we go along to be able to add emulated device.

Furthermore, libxl is not the only toolstack out. So I am not convinced this is a good argument to keep the name the same.

Cheers,

Signature · 1 line
Julien Grall
#35 lore raw

Re: [PATCH v1 10/12] hw/arm: introduce xenpv machine

Quoted 24 lines · Julien Grall
Quoted 17 lines · Stefano Stabellini
Quoted 16 lines · Julien Grall

Hi,

There seem to be some missing patches on xen-devel (including the cover letter). Is that expected?

Quoted 3 lines · Vikram Garhwal

Add a new machine xenpv which creates a IOREQ server to register/connect with
Xen Hypervisor.

I don't like the name 'xenpv' because it doesn't convey the fact that some of
the HW may be emulated rather than para-virtualized. In fact one may only want
to use for emulating devices.

Quoted 5 lines
Quoted 3 lines

Potential name would be 'xen-arm' or re-using 'virt' but with 'accel=xen' to
select a Xen layout.

The benefit of 'xenpv' is that it doesn't require any changes to libxl.

Quoted 8 lines

I am quite surprised. Looking at the code, it seems to work more by chance than it is intentional as the code is gated by libxl__need_xenpv_qemu(). So it would not start if there were no emulated devices.

Quoted 2 lines

It is even backward compatible so it could be used with an older version of Xen/libxl.

We don't really gain much here. IOREQ is a tech preview and anyone that wants to try it should really use the latest Xen.

I think that's fair.

Quoted 6 lines
Quoted 4 lines

Backward compatibility aside, if we come up with a
different name then we'll need changes to libxl and to manage those changes. For instance, if we use 'xen-arm' that would mean we would need to handle per-arch QEMU machine names.

Right, so the main argument here is for simplicity in libxl

Yeah

Quoted 6 lines

Looking at how 'xenpv' is built, this is really expected to deal with PV backend rather than emulated device. I do expect some changes as we go along to be able to add emulated device.

Furthermore, libxl is not the only toolstack out. So I am not convinced this is a good argument to keep the name the same.

Let's think some more about the naming strategy. From a QEMU point of view we could choose any name we like (Vikram please confirm), the issue is really on the libxl side.

Today libxl understands two QEMU "machines": xenpv and xenfv (pc,accel=xen is the same as xenfv, I'll use xenfv in this email for simplicity).

xenpv is for PV guests and only provides PV backends, no emulation. It is used on both ARM and x86.

xenfv is only used on x86, and it is for HVM guests, with a full set of emulated hardware (PIIX3, etc.).

The purpose of this series is to introduce a QEMU machine that:
- works on ARM (but could maybe work on other archs as a stretch goal)
- provides PV backends
- no emulated devices by default
- also can emulate selected devices on request

Certainly it is not xenfv or pc,accel=xen because they would with more emulation by default. This is more like "xenpvh": an extension to PV with also the capability of emulating one device as requested. It is not intended to emulate a full PC and doesn't do that by default like xenfv.

If/When x86 PVH gains the ability to use QEMU as IOREQ server, I would imagine it would run a QEMU machine similar to this one.

This is also how I would imagine it would get integrated in libxl: as a xenpv + individual emulated devices. Not as HVM for ARM. The other QEMU command line arguments are inline with the xenpv command line arguments rather than xenfv command line arguments. This is why the libxl changes are small to zero to make it work today.

So, I think the following options work:

a) call it "xenpv" because it is an extension of the old xenpv machine
b) call it "xenpvh" because it is PV + few individual emulated devices

If we call it xenpv there are fewer changes in libxl. If we call it xenpvh there are more changes in libxl but we can distinguish xenpv from xenpvh (I don't see why we need it right now, but I could imagine it could turn out useful in the future.)

I would stay away from arch-specific machine names because it will make it harder on the libxl side without immediate benefits.

#36 lore raw

Re: [PATCH v1 10/12] hw/arm: introduce xenpv machine

Hi Stefano,

Quoted 74 lines · Stefano Stabellini
Quoted 24 lines · Julien Grall
Quoted 17 lines · Stefano Stabellini
Quoted 16 lines · Julien Grall

Hi,

There seem to be some missing patches on xen-devel (including the cover letter). Is that expected?

Quoted 3 lines · Vikram Garhwal

Add a new machine xenpv which creates a IOREQ server to register/connect with
Xen Hypervisor.

I don't like the name 'xenpv' because it doesn't convey the fact that some of
the HW may be emulated rather than para-virtualized. In fact one may only want
to use for emulating devices.

Quoted 5 lines
Quoted 3 lines

Potential name would be 'xen-arm' or re-using 'virt' but with 'accel=xen' to
select a Xen layout.

The benefit of 'xenpv' is that it doesn't require any changes to libxl.

Quoted 8 lines

I am quite surprised. Looking at the code, it seems to work more by chance than it is intentional as the code is gated by libxl__need_xenpv_qemu(). So it would not start if there were no emulated devices.

Quoted 2 lines

It is even backward compatible so it could be used with an older version of Xen/libxl.

We don't really gain much here. IOREQ is a tech preview and anyone that wants to try it should really use the latest Xen.

I think that's fair.

Quoted 6 lines
Quoted 4 lines

Backward compatibility aside, if we come up with a
different name then we'll need changes to libxl and to manage those changes. For instance, if we use 'xen-arm' that would mean we would need to handle per-arch QEMU machine names.

Right, so the main argument here is for simplicity in libxl

Yeah

Quoted 6 lines

Looking at how 'xenpv' is built, this is really expected to deal with PV backend rather than emulated device. I do expect some changes as we go along to be able to add emulated device.

Furthermore, libxl is not the only toolstack out. So I am not convinced this is a good argument to keep the name the same.

Let's think some more about the naming strategy. From a QEMU point of view we could choose any name we like (Vikram please confirm), the issue is really on the libxl side.

Today libxl understands two QEMU "machines": xenpv and xenfv (pc,accel=xen is the same as xenfv, I'll use xenfv in this email for simplicity).

xenpv is for PV guests and only provides PV backends, no emulation. It is used on both ARM and x86.

xenfv is only used on x86, and it is for HVM guests, with a full set of emulated hardware (PIIX3, etc.).

The purpose of this series is to introduce a QEMU machine that:
- works on ARM (but could maybe work on other archs as a stretch goal)
- provides PV backends
- no emulated devices by default
- also can emulate selected devices on request

Quoted 4 lines

Certainly it is not xenfv or pc,accel=xen because they would with more emulation by default. This is more like "xenpvh": an extension to PV with also the capability of emulating one device as requested. It is not intended to emulate a full PC and doesn't do that by default like xenfv.

The definition of "full PC" is not very clear for me. Unlike x86, Arm doesn't have legacy devices that needs to be emulated. So technically, if we emulated one network card and one block device, then we would be able potentially be able to boot an unaware OS on Xen on Arm. This would be the same as if you passthrough-ed the two devices.

In the past, I have seen interest to boot OS like Windows OS/iOS on Arm. I do expect that the addition of a Xen platform in QEMU will lead to another increase of the interest because we could expose anything to the VM. Although, it might need some tweak in Xen to allow more dynamic layout just in case an OS doesn't discover dynamically devices.

Quoted 3 lines

If/When x86 PVH gains the ability to use QEMU as IOREQ server, I would imagine it would run a QEMU machine similar to this one.

To me it would sounds odd to add emulated devices in the 'xenpv' because they would only work for PVH domain. AFAIK, QEMU doesn't know whether a domain is PV or PVH. So we would solely rely on IOREQ to return an error.

Quoted 9 lines

This is also how I would imagine it would get integrated in libxl: as a xenpv + individual emulated devices. Not as HVM for ARM. The other QEMU command line arguments are inline with the xenpv command line arguments rather than xenfv command line arguments. This is why the libxl changes are small to zero to make it work today.

So, I think the following options work:

Quoted 7 lines

a) call it "xenpv" because it is an extension of the old xenpv machine
b) call it "xenpvh" because it is PV + few individual emulated devices

If we call it xenpv there are fewer changes in libxl. If we call it xenpvh there are more changes in libxl but we can distinguish xenpv from xenpvh (I don't see why we need it right now, but I could imagine it could turn out useful in the future.)

IMHO, we need to plan for the future.

Quoted 3 lines

I would stay away from arch-specific machine names because it will make it harder on the libxl side without immediate benefits.

If the name is the only change, then I would expect this could be done with a per-arch define. So that would be a few lines change maximum.

Cheers,

Signature · 1 line
Julien Grall
#37 lore raw

Re: [PATCH v1 10/12] hw/arm: introduce xenpv machine

In reply to Julien Grall
Quoted 9 lines · Julien Grall

Hi,

There seem to be some missing patches on xen-devel (including the cover letter). Is that expected?

Quoted 2 lines · Vikram Garhwal

Add a new machine xenpv which creates a IOREQ server to register/connect with Xen Hypervisor.

Quoted 6 lines

I don't like the name 'xenpv' because it doesn't convey the fact that some of the HW may be emulated rather than para-virtualized. In fact one may only want to use for emulating devices.

Potential name would be 'xen-arm' or re-using 'virt' but with 'accel=xen' to select a Xen layout.

I don't think you can re-use the machine name and select by accelerator because the virt machine does quite a lot of other stuff this model doesn't support. However I've been calling this concept "xen-virt" or maybe the explicit "xen-virtio" because that is what it is targeting.

Signature · 1 line
Alex Bennée
#38 lore raw

Re: [PATCH v1 10/12] hw/arm: introduce xenpv machine

In reply to Julien Grall

Hi Julien,

From: Julien Grall <[email protected]>
Date: Sunday, October 16, 2022 at 10:48 AM
To: Garhwal, Vikram <[email protected]>, [email protected] <[email protected]>
Cc: Stabellini, Stefano <[email protected]>, Peter Maydell <[email protected]>, Stefano Stabellini <[email protected]>, Anthony Perard <[email protected]>, Paul Durrant <[email protected]>, open list:ARM TCG CPUs <[email protected]>, open list:X86 Xen CPUs <[email protected]>
Subject: Re: [PATCH v1 10/12] hw/arm: introduce xenpv machine
Hi,

There seem to be some missing patches on xen-devel (including the cover
letter). Is that expected?
Not sure what went wrong there. I can see all of these on QEMU-devel. Perhaps xen-devel is not in maintainer’s list for all the xen files?

Quoted 2 lines · Vikram Garhwal

Add a new machine xenpv which creates a IOREQ server to register/connect with Xen Hypervisor.

I don't like the name 'xenpv' because it doesn't convey the fact that some of the HW may be emulated rather than para-virtualized. In fact one may only want to use for emulating devices.

Potential name would be 'xen-arm' or re-using 'virt' but with 'accel=xen' to select a Xen layout.

Quoted 13 lines

Xen IOREQ connection expect the TARGET_PAGE_SIZE to 4096, and the xenpv machine on ARM will have no CPU definitions. We need to define TARGET_PAGE_SIZE appropriately ourselves.

Optional: When CONFIG_TPM is enabled, it also creates a tpm-tis-device, adds a TPM emulator and connects to swtpm running on host machine via chardev socket and support TPM functionalities for a guest domain.

Extra command line for aarch64 xenpv QEMU to connect to swtpm:

     -chardev socket,id=chrtpm,path=/tmp/myvtpm2/swtpm-sock \
     -tpmdev emulator,id=tpm0,chardev=chrtpm \
Quoted 7 lines

swtpm implements a TPM software emulator(TPM 1.2 & TPM 2) built on libtpms and provides access to TPM functionality over socket, chardev and CUSE interface. Github repo: https://github.com/stefanberger/swtpm
Example for starting swtpm on host machine:
mkdir /tmp/vtpm2
swtpm socket --tpmstate dir=/tmp/vtpm2 \
--ctrl type=unixio,path=/tmp/vtpm2/swtpm-sock &

I see patches for QEMU but not Xen. How can this be tested with existing Xen? Will libxl ever create QEMU?
Will send the patch for libxl Xen separately.

[...]

hw/arm/xen_arm.c line 83

Quoted 6 lines
hw/arm/xen_arm.c new file +6 -0
+static int xen_init_ioreq(XenIOState *state, unsigned int max_cpus)
+{
+    xen_dmod = xendevicemodel_open(0, 0);
+    xen_xc = xc_interface_open(0, 0, 0);
+
+    if (xen_xc == NULL) {

You are checking xen_xc but not xen_dmod. Why?

Quoted 27 lines
hw/arm/xen_arm.c new file +27 -0
+        perror("xen: can't open xen interface\n");
+        return -1;
+    }
+
+    xen_fmem = xenforeignmemory_open(0, 0);
+    if (xen_fmem == NULL) {
+        perror("xen: can't open xen fmem interface\n");
+        xc_interface_close(xen_xc);
+        return -1;
+    }
+
+    xen_register_ioreq(state, max_cpus, xen_memory_listener);
+
+    xenstore_record_dm_state(xenstore, "running");
+
+    return 0;
+}
+
+static void xen_enable_tpm(void)
+{
+/* qemu_find_tpm_be is only available when CONFIG_TPM is enabled. */
+#ifdef CONFIG_TPM
+    Error *errp = NULL;
+    DeviceState *dev;
+    SysBusDevice *busdev;
+
+    TPMBackend *be = qemu_find_tpm_be("tpm0");

hw/arm/xen_arm.c line 120

Quoted 10 lines
hw/arm/xen_arm.c new file +10 -0
+    if (be == NULL) {
+        DPRINTF("Couldn't fine the backend for tpm0\n");
+        return;
+    }
+    dev = qdev_new(TYPE_TPM_TIS_SYSBUS);
+    object_property_set_link(OBJECT(dev), "tpmdev", OBJECT(be), &errp);
+    object_property_set_str(OBJECT(dev), "tpmdev", be->id, &errp);
+    busdev = SYS_BUS_DEVICE(dev);
+    sysbus_realize_and_unref(busdev, &error_fatal);
+    sysbus_mmio_map(busdev, 0, GUEST_TPM_BASE);

I can't find where GUEST_TPM_BASE is defined. But then the guest memory
layout is not expected to be stable. With your current approach, it
means QEMU would need to be rebuilt for every Xen version. Is it what we
want?
I cannot think of better way to do this. Either we add the the def here or rebuild it if GUEST_TPM_BASE changes for each xen version.

hw/arm/xen_arm.c line 133

Quoted 13 lines
hw/arm/xen_arm.c new file +13 -0
+
+    DPRINTF("Connected tpmdev at address 0x%lx\n", GUEST_TPM_BASE);
+#endif
+}
+
+static void xen_arm_init(MachineState *machine)
+{
+    XenArmState *xam = XEN_ARM(machine);
+
+    xam->state =  g_new0(XenIOState, 1);
+
+    if (xen_init_ioreq(xam->state, machine->smp.cpus)) {
+        return;

In another patch, you said the IOREQ would be optional. IHMO, I think this is a bad idea to register it by default because one may only want to use PV drivers. Registering IOREQ will add unnecessary overhead in Xen.

Furthermore, it means that someone selecting TPM but Xen is not built with CONFIG_IOREQ=y (BTW This is still a tech preview but there are security holes on Arm...) will not get an error. Instead, the OS will until it crashes when trying to access the TPM.

Overall I think it would be better if IOREQ is only registered when a device requires (like TPM) it *and* throw an error if there is a problem during the initialization.

Quoted 4 lines
hw/arm/xen_arm.c new file +4 -0
+    } > +
+    xen_enable_tpm();
+
+    return;

hw/arm/xen_arm.c line 148

Quoted 10 lines
hw/arm/xen_arm.c new file +10 -0
+}
+
+static void xen_arm_machine_class_init(ObjectClass *oc, void *data)
+{
+
+    MachineClass *mc = MACHINE_CLASS(oc);
+    mc->desc = "Xen Para-virtualized PC";
+    mc->init = xen_arm_init;
+    mc->max_cpus = 1;
+    machine_class_allow_dynamic_sysbus_dev(mc, TYPE_TPM_TIS_SYSBUS);

Shouldn't this be protected with #ifdef CONFIG_TPM?

Quoted 21 lines
hw/arm/xen_arm.c new file +15 -0
+}
+
+static const TypeInfo xen_arm_machine_type = {
+    .name = TYPE_XEN_ARM,
+    .parent = TYPE_MACHINE,
+    .class_init = xen_arm_machine_class_init,
+    .instance_size = sizeof(XenArmState),
+};
+
+static void xen_arm_machine_register_types(void)
+{
+    type_register_static(&xen_arm_machine_type);
+}
+
+type_init(xen_arm_machine_register_types)
include/hw/arm/xen_arch_hvm.h new file +0 -0
new file mode 100644
index 0000000000..f645dfec28

include/hw/arm/xen_arch_hvm.h line 10

Quoted 10 lines
include/hw/arm/xen_arch_hvm.h new file +10 -0
@@ -0,0 +1,12 @@
+#ifndef HW_XEN_ARCH_ARM_HVM_H
+#define HW_XEN_ARCH_ARM_HVM_H
+
+#include <xen/hvm/ioreq.h>
+void arch_handle_ioreq(XenIOState *state, ioreq_t *req);
+void arch_xen_set_memory(XenIOState *state,
+                         MemoryRegionSection *section,
+                         bool add);
+
+#undef TARGET_PAGE_SIZE

I am a bit puzzled with this #undef. In the commit message you said that there will be no CPU definition. So the implications is this should not be defined.

If it is defined, then what guarantees that all the source will use the correct value?

include/hw/arm/xen_arch_hvm.h line 11

Quoted 1 line
include/hw/arm/xen_arch_hvm.h new file +1 -0
+#define TARGET_PAGE_SIZE 4096

It would be better to use XC_PAGE_SIZE (or similar) rather than hardcoding it.
Corrected this and sent a v2.

include/hw/xen/arch_hvm.h line 5

Quoted 11 lines
include/hw/arm/xen_arch_hvm.h new file +1 -0
+#endif
include/hw/xen/arch_hvm.h +2 -0
index 26674648d8..c7c515220d 100644
@@ -1,3 +1,5 @@
  #if defined(TARGET_I386) || defined(TARGET_X86_64)
  #include "hw/i386/xen_arch_hvm.h"
+#elif defined(TARGET_ARM) || defined(TARGET_ARM_64)
+#include "hw/arm/xen_arch_hvm.h"
  #endif

Cheers,

--
Julien Grall

#39 lore raw

Re: [PATCH v1 10/12] hw/arm: introduce xenpv machine

Quoted 1 line · Vikram

Hi Julien,

Hi Vikram,

I am having trouble to differentiate your answers from my remark. For instance...

Quoted 44 lines

From: Julien Grall <[email protected]>
Date: Sunday, October 16, 2022 at 10:48 AM
To: Garhwal, Vikram <[email protected]>, [email protected] <[email protected]>
Cc: Stabellini, Stefano <[email protected]>, Peter Maydell <[email protected]>, Stefano Stabellini <[email protected]>, Anthony Perard <[email protected]>, Paul Durrant <[email protected]>, open list:ARM TCG CPUs <[email protected]>, open list:X86 Xen CPUs <[email protected]>
Subject: Re: [PATCH v1 10/12] hw/arm: introduce xenpv machine
Hi,

There seem to be some missing patches on xen-devel (including the cover
letter). Is that expected?
Not sure what went wrong there. I can see all of these on QEMU-devel. Perhaps xen-devel is not in maintainer’s list for all the xen files?

Quoted 2 lines · Vikram Garhwal

Add a new machine xenpv which creates a IOREQ server to register/connect with Xen Hypervisor.

I don't like the name 'xenpv' because it doesn't convey the fact that some of the HW may be emulated rather than para-virtualized. In fact one may only want to use for emulating devices.

Potential name would be 'xen-arm' or re-using 'virt' but with 'accel=xen' to select a Xen layout.

Quoted 13 lines

Xen IOREQ connection expect the TARGET_PAGE_SIZE to 4096, and the xenpv machine on ARM will have no CPU definitions. We need to define TARGET_PAGE_SIZE appropriately ourselves.

Optional: When CONFIG_TPM is enabled, it also creates a tpm-tis-device, adds a TPM emulator and connects to swtpm running on host machine via chardev socket and support TPM functionalities for a guest domain.

Extra command line for aarch64 xenpv QEMU to connect to swtpm:

      -chardev socket,id=chrtpm,path=/tmp/myvtpm2/swtpm-sock \
      -tpmdev emulator,id=tpm0,chardev=chrtpm \
Quoted 7 lines

swtpm implements a TPM software emulator(TPM 1.2 & TPM 2) built on libtpms and provides access to TPM functionality over socket, chardev and CUSE interface. Github repo: https://github.com/stefanberger/swtpm
Example for starting swtpm on host machine:
mkdir /tmp/vtpm2
swtpm socket --tpmstate dir=/tmp/vtpm2 \
--ctrl type=unixio,path=/tmp/vtpm2/swtpm-sock &

Quoted 3 lines

I see patches for QEMU but not Xen. How can this be tested with existing Xen? Will libxl ever create QEMU?
Will send the patch for libxl Xen separately.

... the first two lines are my remarks and the 3rd is your answer. Can you configure your e-mail client to do proper quoting?

[...]

Quoted 7 lines

hw/arm/xen_arm.c line 120

Quoted 6 lines
hw/arm/xen_arm.c new file +6 -0
+    dev = qdev_new(TYPE_TPM_TIS_SYSBUS);
+    object_property_set_link(OBJECT(dev), "tpmdev", OBJECT(be), &errp);
+    object_property_set_str(OBJECT(dev), "tpmdev", be->id, &errp);
+    busdev = SYS_BUS_DEVICE(dev);
+    sysbus_realize_and_unref(busdev, &error_fatal);
+    sysbus_mmio_map(busdev, 0, GUEST_TPM_BASE);
Quoted 5 lines

I can't find where GUEST_TPM_BASE is defined. But then the guest memory
layout is not expected to be stable. With your current approach, it
means QEMU would need to be rebuilt for every Xen version. Is it what we
want?
I cannot think of better way to do this. Either we add the the def here or rebuild it if GUEST_TPM_BASE changes for each xen version.

The alternative would be to specify the address on the QEMU command line. The advantage is you could build a system where each guests have different layout.

Cheers,

Signature · 1 line
Julien Grall
#40 lore raw

Re: [PATCH v1 10/12] hw/arm: introduce xenpv machine

In reply to Vikram Garhwal

Vikram Garhwal <[email protected]> writes:

<snip>

Quoted 15 lines

Optional: When CONFIG_TPM is enabled, it also creates a tpm-tis-device, adds a TPM emulator and connects to swtpm running on host machine via chardev socket and support TPM functionalities for a guest domain.

Extra command line for aarch64 xenpv QEMU to connect to swtpm:

    -chardev socket,id=chrtpm,path=/tmp/myvtpm2/swtpm-sock \
    -tpmdev emulator,id=tpm0,chardev=chrtpm \

swtpm implements a TPM software emulator(TPM 1.2 & TPM 2) built on libtpms and provides access to TPM functionality over socket, chardev and CUSE interface. Github repo: https://github.com/stefanberger/swtpm
Example for starting swtpm on host machine:
mkdir /tmp/vtpm2
swtpm socket --tpmstate dir=/tmp/vtpm2 \
--ctrl type=unixio,path=/tmp/vtpm2/swtpm-sock &

<snip>

Quoted 9 lines
hw/arm/xen_arm.c new file +9 -0
+static void xen_enable_tpm(void)
+{
+/* qemu_find_tpm_be is only available when CONFIG_TPM is enabled. */
+#ifdef CONFIG_TPM
+    Error *errp = NULL;
+    DeviceState *dev;
+    SysBusDevice *busdev;
+
+    TPMBackend *be = qemu_find_tpm_be("tpm0");

hw/arm/xen_arm.c line 120

Quoted 10 lines
hw/arm/xen_arm.c new file +10 -0
+    if (be == NULL) {
+        DPRINTF("Couldn't fine the backend for tpm0\n");
+        return;
+    }
+    dev = qdev_new(TYPE_TPM_TIS_SYSBUS);
+    object_property_set_link(OBJECT(dev), "tpmdev", OBJECT(be), &errp);
+    object_property_set_str(OBJECT(dev), "tpmdev", be->id, &errp);
+    busdev = SYS_BUS_DEVICE(dev);
+    sysbus_realize_and_unref(busdev, &error_fatal);
+    sysbus_mmio_map(busdev, 0, GUEST_TPM_BASE);

I'm not sure what has gone wrong here but I'm getting:

../../hw/arm/xen_arm.c: In function ‘xen_enable_tpm’:
../../hw/arm/xen_arm.c:120:32: error: ‘GUEST_TPM_BASE’ undeclared (first use in this function); did you mean ‘GUEST_RAM_BASE’?

    120 |     sysbus_mmio_map(busdev, 0, GUEST_TPM_BASE);
        |                                ^~~~~~~~~~~~~~
        |                                GUEST_RAM_BASE

../../hw/arm/xen_arm.c:120:32: note: each undeclared identifier is reported only once for each function it appears in

In my cross build:

# Configured with: '../../configure' '--disable-docs' '--target-list=aarch64-softmmu' '--disable-kvm' '--enable-xen' '--disable-opengl' '--disable-libudev' '--enable-tpm' '--disable-xen-pci-passthrough' '--cross-prefix=aarch64-linux-gnu-' '--skip-meson'

which makes me wonder if this is a configure failure or a confusion about being able to have host swtpm implementations during emulation but needing target tpm for Xen?

Signature · 1 line
Alex Bennée
#41 lore raw

Re: [PATCH v1 10/12] hw/arm: introduce xenpv machine

Hi,

Quoted 50 lines · Alex Bennée

Vikram Garhwal <[email protected]> writes:

<snip>

Quoted 15 lines

Optional: When CONFIG_TPM is enabled, it also creates a tpm-tis-device, adds a TPM emulator and connects to swtpm running on host machine via chardev socket and support TPM functionalities for a guest domain.

Extra command line for aarch64 xenpv QEMU to connect to swtpm:

     -chardev socket,id=chrtpm,path=/tmp/myvtpm2/swtpm-sock \
     -tpmdev emulator,id=tpm0,chardev=chrtpm \

swtpm implements a TPM software emulator(TPM 1.2 & TPM 2) built on libtpms and provides access to TPM functionality over socket, chardev and CUSE interface. Github repo: https://github.com/stefanberger/swtpm
Example for starting swtpm on host machine:
mkdir /tmp/vtpm2
swtpm socket --tpmstate dir=/tmp/vtpm2 \
--ctrl type=unixio,path=/tmp/vtpm2/swtpm-sock &

<snip>

Quoted 9 lines
hw/arm/xen_arm.c new file +9 -0
+static void xen_enable_tpm(void)
+{
+/* qemu_find_tpm_be is only available when CONFIG_TPM is enabled. */
+#ifdef CONFIG_TPM
+    Error *errp = NULL;
+    DeviceState *dev;
+    SysBusDevice *busdev;
+
+    TPMBackend *be = qemu_find_tpm_be("tpm0");

hw/arm/xen_arm.c line 120

Quoted 10 lines
hw/arm/xen_arm.c new file +10 -0
+    if (be == NULL) {
+        DPRINTF("Couldn't fine the backend for tpm0\n");
+        return;
+    }
+    dev = qdev_new(TYPE_TPM_TIS_SYSBUS);
+    object_property_set_link(OBJECT(dev), "tpmdev", OBJECT(be), &errp);
+    object_property_set_str(OBJECT(dev), "tpmdev", be->id, &errp);
+    busdev = SYS_BUS_DEVICE(dev);
+    sysbus_realize_and_unref(busdev, &error_fatal);
+    sysbus_mmio_map(busdev, 0, GUEST_TPM_BASE);

I'm not sure what has gone wrong here but I'm getting:

../../hw/arm/xen_arm.c: In function ‘xen_enable_tpm’:
../../hw/arm/xen_arm.c:120:32: error: ‘GUEST_TPM_BASE’ undeclared (first use in this function); did you mean ‘GUEST_RAM_BASE’?

     120 |     sysbus_mmio_map(busdev, 0, GUEST_TPM_BASE);
         |                                ^~~~~~~~~~~~~~
         |                                GUEST_RAM_BASE

../../hw/arm/xen_arm.c:120:32: note: each undeclared identifier is reported only once for each function it appears in

Quoted 7 lines

In my cross build:

   # Configured with: '../../configure' '--disable-docs' '--target-list=aarch64-softmmu' '--disable-kvm' '--enable-xen' '--disable-opengl' '--disable-libudev' '--enable-tpm' '--disable-xen-pci-passthrough' '--cross-prefix=aarch64-linux-gnu-' '--skip-meson'

which makes me wonder if this is a configure failure or a confusion about being able to have host swtpm implementations during emulation but needing target tpm for Xen?

I was also wondering where is that value come from. Note that the memory/IRQ layout exposed to the guest is not stable.

Are we expecting the user to rebuild QEMU for every Xen versions (or possibly every guest if we ever allow dynamic layout in Xen)?

Cheers,

Signature · 1 line
Julien Grall
#42 lore raw

Re: [PATCH v1 10/12] hw/arm: introduce xenpv machine

Quoted 2 lines · Julien Grall

Hi,

Hello all.

[sorry for the possible format issues]

Quoted 70 lines · Alex Bennée
Quoted 5 lines

Vikram Garhwal <[email protected]> writes:

<snip>

Quoted 1 line

Optional: When CONFIG_TPM is enabled, it also creates a tpm-tis-device,

adds a

Quoted 1 line
Quoted 1 line

TPM emulator and connects to swtpm running on host machine via chardev

socket

Quoted 7 lines
Quoted 7 lines

and support TPM functionalities for a guest domain.

Extra command line for aarch64 xenpv QEMU to connect to swtpm:

     -chardev socket,id=chrtpm,path=/tmp/myvtpm2/swtpm-sock \
     -tpmdev emulator,id=tpm0,chardev=chrtpm \

swtpm implements a TPM software emulator(TPM 1.2 & TPM 2) built on

libtpms and

Quoted 1 line
Quoted 1 line

provides access to TPM functionality over socket, chardev and CUSE

interface.

Quoted 29 lines
Quoted 5 lines

Github repo: https://github.com/stefanberger/swtpm
Example for starting swtpm on host machine:
mkdir /tmp/vtpm2
swtpm socket --tpmstate dir=/tmp/vtpm2 \
--ctrl type=unixio,path=/tmp/vtpm2/swtpm-sock &

<snip>

Quoted 9 lines
hw/arm/xen_arm.c new file +9 -0
+static void xen_enable_tpm(void)
+{
+/* qemu_find_tpm_be is only available when CONFIG_TPM is enabled. */
+#ifdef CONFIG_TPM
+    Error *errp = NULL;
+    DeviceState *dev;
+    SysBusDevice *busdev;
+
+    TPMBackend *be = qemu_find_tpm_be("tpm0");

hw/arm/xen_arm.c line 120

Quoted 10 lines
hw/arm/xen_arm.c new file +10 -0
+    if (be == NULL) {
+        DPRINTF("Couldn't fine the backend for tpm0\n");
+        return;
+    }
+    dev = qdev_new(TYPE_TPM_TIS_SYSBUS);
+    object_property_set_link(OBJECT(dev), "tpmdev", OBJECT(be), &errp);
+    object_property_set_str(OBJECT(dev), "tpmdev", be->id, &errp);
+    busdev = SYS_BUS_DEVICE(dev);
+    sysbus_realize_and_unref(busdev, &error_fatal);
+    sysbus_mmio_map(busdev, 0, GUEST_TPM_BASE);

I'm not sure what has gone wrong here but I'm getting:

Quoted 2 lines

../../hw/arm/xen_arm.c: In function ‘xen_enable_tpm’:

   ../../hw/arm/xen_arm.c:120:32: error: ‘GUEST_TPM_BASE’ undeclared

(first use in this function); did you mean ‘GUEST_RAM_BASE’?

Quoted 4 lines
     120 |     sysbus_mmio_map(busdev, 0, GUEST_TPM_BASE);
         |                                ^~~~~~~~~~~~~~
         |                                GUEST_RAM_BASE

../../hw/arm/xen_arm.c:120:32: note: each undeclared identifier is

reported only once for each function it appears in

Quoted 4 lines

In my cross build:

   # Configured with: '../../configure' '--disable-docs'

'--target-list=aarch64-softmmu' '--disable-kvm' '--enable-xen' '--disable-opengl' '--disable-libudev' '--enable-tpm' '--disable-xen-pci-passthrough' '--cross-prefix=aarch64-linux-gnu-' '--skip-meson'

Quoted 4 lines

which makes me wonder if this is a configure failure or a confusion about being able to have host swtpm implementations during emulation but needing target tpm for Xen?

Quoted 6 lines

I was also wondering where is that value come from. Note that the memory/IRQ layout exposed to the guest is not stable.

Are we expecting the user to rebuild QEMU for every Xen versions (or possibly every guest if we ever allow dynamic layout in Xen)?

This doesn't sound ideal.

I am wondering what would be the correct way here assuming that we would likely need to have more such information in place for supporting more use-cases...
For instance, the PCI host bridge emulation in Qemu. Xen toolstack (another software layer) generates device-tree for the guest, so creates PCI Host bridge node by using reserved regions from Guest OS interface (arch-arm.h):
- GUEST_VPCI_MEM_ADDR (GUEST_VPCI_MEM_SIZE)
- GUEST_VPCI_ECAM_BASE (GUEST_VPCI_ECAM_SIZE)
- GUEST_VPCI_PREFETCH_MEM_ADDR (GUEST_VPCI_PREFETCH_MEM_SIZE)
https://xenbits.xen.org/gitweb/?p=xen.git;a=blob;f=tools/libs/light/libxl_arm.c;h=2a5e93c28403738779863aded31d2df3ba72f8c0;hb=HEAD#l833

Here in Qemu when creating a PCI Host bridge we would need to use exactly the same reserved regions which toolstack writes in the corresponding device-tree node. So how to tell Qemu about them?
1. Introduce new cmd line arguments?
2. Using Xenstore?
3. Anything else?

I am afraid this would be related to every device that we want to emulate in Qemu and for which the toolstack needs to generate device-tree node by using something defined with GUEST_*, unless I really missed something.

Quoted 7 lines

Cheers,

--
Julien Grall

Signature · 3 lines
Regards,

Oleksandr Tyshchenko
#43 lore raw

Re: [PATCH v1 10/12] hw/arm: introduce xenpv machine

In reply to Vikram Garhwal
Quoted 3 lines · Vikram Garhwal

Add a new machine xenpv which creates a IOREQ server to register/connect with Xen Hypervisor.

<snip>

Quoted 3 lines

Optional: When CONFIG_TPM is enabled, it also creates a tpm-tis-device, adds a TPM emulator and connects to swtpm running on host machine via chardev socket and support TPM functionalities for a guest domain.

<snip>

hw/arm/xen_arm.c line 148

Quoted 9 lines
hw/arm/xen_arm.c new file +9 -0
+
+static void xen_arm_machine_class_init(ObjectClass *oc, void *data)
+{
+
+    MachineClass *mc = MACHINE_CLASS(oc);
+    mc->desc = "Xen Para-virtualized PC";
+    mc->init = xen_arm_init;
+    mc->max_cpus = 1;
+    machine_class_allow_dynamic_sysbus_dev(mc, TYPE_TPM_TIS_SYSBUS);

This needs #ifdef CONFIG_TPM because while doing --disable-tpm to try and get the cross build working it then fails with:

../../hw/arm/xen_arm.c: In function ‘xen_arm_machine_class_init’:
../../hw/arm/xen_arm.c:148:48: error: ‘TYPE_TPM_TIS_SYSBUS’ undeclared (first use in this function)

  148 |     machine_class_allow_dynamic_sysbus_dev(mc, TYPE_TPM_TIS_SYSBUS);
      |                                                ^~~~~~~~~~~~~~~~~~~

../../hw/arm/xen_arm.c:148:48: note: each undeclared identifier is reported only once for each function it appears in

Signature · 1 line
Alex Bennée
#44 lore raw

PATCH 11/12 meson.build: enable xenpv machine build for ARM

In reply to Vikram Garhwal

Add CONFIG_XEN for aarch64 device to support build for ARM targets.

---
 meson.build | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
meson.build +1 -1
index b686dfef75..0027d7d195 100644
@@ -125,7 +125,7 @@ endif
 if cpu in ['x86', 'x86_64', 'arm', 'aarch64']
   # i386 emulator provides xenpv machine type for multiple architectures
   accelerator_targets += {
-    'CONFIG_XEN': ['i386-softmmu', 'x86_64-softmmu'],
+    'CONFIG_XEN': ['i386-softmmu', 'x86_64-softmmu', 'aarch64-softmmu'],
   }
 endif
 if cpu in ['x86', 'x86_64']
Signature · 1 line
2.17.0
Reviewed-by on v1 11/12
#45 lore raw

Re: [PATCH v1 11/12] meson.build: enable xenpv machine build for ARM

Quoted 1 line · Vikram Garhwal

Add CONFIG_XEN for aarch64 device to support build for ARM targets.

So to be clear a --enable-xen only build for any of these binaries essentially ends up being the same thing just with a slightly less discombobulating name?

Maybe given there is no real architecture specific stuff we should just create a neutral binary for --enable-xen (e.g. qemu-xen-backend)?

Anyway:

Quoted 20 lines
---
 meson.build | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
meson.build +1 -1
index b686dfef75..0027d7d195 100644
@@ -125,7 +125,7 @@ endif
 if cpu in ['x86', 'x86_64', 'arm', 'aarch64']
   # i386 emulator provides xenpv machine type for multiple architectures
   accelerator_targets += {
-    'CONFIG_XEN': ['i386-softmmu', 'x86_64-softmmu'],
+    'CONFIG_XEN': ['i386-softmmu', 'x86_64-softmmu', 'aarch64-softmmu'],
   }
 endif
 if cpu in ['x86', 'x86_64']
Signature · 1 line
Alex Bennée
#46 lore raw

PATCH 12/12 meson.build: do not set have_xen_pci_passthrough for aarch64 targets

In reply to Vikram Garhwal

From: Stefano Stabellini <[email protected]>

have_xen_pci_passthrough is only used for Xen x86 VMs.

---
 meson.build | 2 ++
 1 file changed, 2 insertions(+)
meson.build +2 -0
index 0027d7d195..43e70936ee 100644
@@ -1454,6 +1454,8 @@ have_xen_pci_passthrough = get_option('xen_pci_passthrough') \
            error_message: 'Xen PCI passthrough requested but Xen not enabled') \
   .require(targetos == 'linux',
            error_message: 'Xen PCI passthrough not available on this platform') \
+  .require(cpu == 'x86'  or cpu == 'x86_64',
+           error_message: 'Xen PCI passthrough not available on this platform') \
   .allowed()
 
 
Signature · 1 line
2.17.0
Reviewed-by on v1 12/12
#47 lore raw

Re: [PATCH v1 12/12] meson.build: do not set have_xen_pci_passthrough for aarch64 targets

Quoted 5 lines · Vikram Garhwal

From: Stefano Stabellini <[email protected]>

have_xen_pci_passthrough is only used for Xen x86 VMs.

I think this might want to before 11/12. Anyway:

Quoted 15 lines
---
 meson.build | 2 ++
 1 file changed, 2 insertions(+)
meson.build +2 -0
index 0027d7d195..43e70936ee 100644
@@ -1454,6 +1454,8 @@ have_xen_pci_passthrough = get_option('xen_pci_passthrough') \
            error_message: 'Xen PCI passthrough requested but Xen not enabled') \
   .require(targetos == 'linux',
            error_message: 'Xen PCI passthrough not available on this platform') \
+  .require(cpu == 'x86'  or cpu == 'x86_64',
+           error_message: 'Xen PCI passthrough not available on this platform') \
   .allowed()
Signature · 1 line
Alex Bennée
#48 lore raw
In reply to Vikram Garhwal
Quoted 10 lines · Vikram Garhwal

Hi,
This series add xenpv machine for aarch64. Motivation behind creating xenpv machine with IOREQ and TPM was to enable each guest on Xen aarch64 to have it's own unique and emulated TPM.

This series does following:
1. Moved common xen functionalities from hw/i386/xen to hw/xen/ so those can be used for aarch64.
2. We added a minimal xenpv arm machine which creates an IOREQ server and support TPM.

Now I have some CI minutes again:

https://gitlab.com/stsquad/qemu/-/pipelines/677956972/failures

which broadly break down into:

* GUEST_TPM_BASE define missing
* #include <xenstore.h> failure on builds that don't enable Xen
* CPUTLBEntryFull f; breakage (tcg bits in a non-tcg build?)

Signature · 1 line
Alex Bennée
#49 lore raw
In reply to Vikram Garhwal
Quoted 11 lines · Vikram Garhwal

Hi,
This series add xenpv machine for aarch64. Motivation behind creating xenpv machine with IOREQ and TPM was to enable each guest on Xen aarch64 to have it's own unique and emulated TPM.

This series does following:
1. Moved common xen functionalities from hw/i386/xen to hw/xen/ so those can be used for aarch64.
2. We added a minimal xenpv arm machine which creates an IOREQ server and support TPM.

Quoted 7 lines

Please note that patch 05/12 breaks the build. Patch 06/12 fixes the build issue. If needed we can merge patch 05/12 and 06/12. For now we kept these separate to make changes easy to review.

Also, checkpatch.pl fails for 03/12 and 06/12. These fails are due to moving old code to new place which was not QEMU code style compatible. No new add code was added.

I've finished my review pass. Please CC me on v2 when it's ready ;-)

Signature · 1 line
Alex Bennée