Skip to contentSkip to conversation

hide struct start_info from PVH code

PATCH In review Juergen Gross 3 participants 15 messages Jun 19, 2025 – Jun 23, 2025

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

cover hide struct start_info from PVH code

While working on xenstore-stubdom live update support I came across some really unpleasant code using struct start_info even when running in PVH mode.

Especially suspend/resume was doing some crazy stuff, including overwriting the initial struct hvm_start_info partially.

This series is hiding struct start_info from PVH code. This didn't result in any runtime error, but in future I'll need to access struct hvm_start_info some time after boot, so I need to get that fixed.

Tested especially with xl suspend/resume and xl save/restore in PV and in PVH mode.

Juergen Gross (6):
suspend: drop crazy pfn handling for PVH
setup: drop parameter from map_shared_info()
x86/pv: remove global start_info
console: drop parameter from get_console()
xenbus: drop parameter from get_xenbus()
x86/pv: make start_info_ptr PV-only

 arch/arm/setup.c     |  6 ++--
 arch/x86/setup.c     | 69 ++++++++++++--------------------------------
 console.c            | 10 +++----
 hypervisor.c         |  2 +-
 include/console.h    |  2 +-
 include/hypervisor.h | 14 ++-------
 include/xenbus.h     |  4 +--
 xenbus.c             | 13 ++++-----
 8 files changed, 39 insertions(+), 81 deletions(-)
Signature · 1 line
2.43.0
#1 lore raw

PATCH 1/6 suspend: drop crazy pfn handling for PVH

Suspend for PVH is doing some crazy stuff with the Xenstore and console PFNs:

- In arch_pre_suspend() the PFNs are "translated" to MFNs, even if that translation is a nop.
- The result is stored via start_info_ptr, which points to a struct start_info only in case of PV mode.
- After resuming or a cancelled suspend, the PFNs are retrieved again and "translated".
- The "MFNs" are stored again via start_info_ptr, which is still not pointing at a struct start_info, and from where the data is never read again after that.

Drop all of that completely.

---
 arch/x86/setup.c | 22 ----------------------
 1 file changed, 22 deletions(-)
arch/x86/setup.c +0 -22
index b613083b..e30eb065 100644
@@ -227,17 +227,6 @@ void arch_pre_suspend(void)
         virt_to_pfn(mfn_to_virt(start_info_ptr->store_mfn));
     start_info_ptr->console.domU.mfn =
         virt_to_pfn(mfn_to_virt(start_info_ptr->console.domU.mfn));
-#else
-    uint64_t store_v;
-    uint64_t console_v;
-
-    if( hvm_get_parameter(HVM_PARAM_STORE_PFN, &store_v) )
-        BUG();
-    start_info_ptr->store_mfn = store_v;
-
-    if( hvm_get_parameter(HVM_PARAM_CONSOLE_PFN, &console_v) )
-        BUG();
-    start_info_ptr->console.domU.mfn = console_v;
 #endif
     unmap_shared_info();
 
@@ -262,17 +251,6 @@ void arch_post_suspend(int canceled)
     } else {
         memcpy(&start_info, start_info_ptr, sizeof(start_info_t));
     }
-#else
-    uint64_t store_v;
-    uint64_t console_v;
-
-    if (hvm_get_parameter(HVM_PARAM_STORE_PFN, &store_v))
-        BUG();
-    start_info_ptr->store_mfn = pfn_to_mfn(store_v);
-
-    if (hvm_get_parameter(HVM_PARAM_CONSOLE_PFN, &console_v))
-        BUG();
-    start_info_ptr->console.domU.mfn = pfn_to_mfn(console_v);
 #endif
 
     HYPERVISOR_shared_info = map_shared_info((void*) start_info_ptr);
Signature · 1 line
2.43.0
Reviewed-by on v1 1/6
#2 lore raw

Re: [MINI-OS PATCH 1/6] suspend: drop crazy pfn handling for PVH

Quoted 13 lines · Juergen Gross

Suspend for PVH is doing some crazy stuff with the Xenstore and console PFNs:

- In arch_pre_suspend() the PFNs are "translated" to MFNs, even if that translation is a nop.
- The result is stored via start_info_ptr, which points to a struct start_info only in case of PV mode.
- After resuming or a cancelled suspend, the PFNs are retrieved again and "translated".
- The "MFNs" are stored again via start_info_ptr, which is still not pointing at a struct start_info, and from where the data is never read again after that.

Quoted 3 lines

Drop all of that completely.

start_info_ptr points at an hvm_start_info. Assuming I counted properly, accessing ->console/store is writing past the end of the struct.

#3 lore raw

PATCH 2/6 setup: drop parameter from map_shared_info()

In reply to Juergen Gross

The parameter of map_shared_info() is evaluated in PV mode only, and it is always the pointer to the initial struct start_info from boot time.

By setting start_info_ptr a little bit earlier the PV variant of map_shared_info() can just use start_info_ptr instead of a parameter.

---
 arch/arm/setup.c     |  2 +-
 arch/x86/setup.c     | 12 ++++++------
 hypervisor.c         |  2 +-
 include/hypervisor.h |  2 +-
 4 files changed, 9 insertions(+), 9 deletions(-)
arch/arm/setup.c +1 -1
index 3d112fae..1bcfa689 100644
@@ -38,7 +38,7 @@ void arch_init(void *dtb_pointer, uint32_t physical_offset)
     device_tree = dtb_pointer;
 
     /* Map shared_info page */
-    HYPERVISOR_shared_info = map_shared_info(NULL);
+    HYPERVISOR_shared_info = map_shared_info();
 
     get_console(NULL);
     get_xenbus(NULL);
arch/x86/setup.c +6 -6
index e30eb065..1bdf9509 100644
@@ -79,11 +79,10 @@ static inline void sse_init(void) {
 #ifdef CONFIG_PARAVIRT
 #define hpc_init()
 
-shared_info_t *map_shared_info(void *p)
+shared_info_t *map_shared_info(void)
 {
     int rc;
-    start_info_t *si = p;
-    unsigned long pa = si->shared_info;
+    unsigned long pa = start_info_ptr->shared_info;
 
     if ( (rc = HYPERVISOR_update_va_mapping((unsigned long)shared_info,
                                             __pte(pa | 7), UVMF_INVLPG)) )
@@ -185,6 +184,8 @@ arch_init(void *par)
 {
 	static char hello[] = "Bootstrapping...\n";
 
+	start_info_ptr = par;
+
 	hpc_init();
 	(void)HYPERVISOR_console_io(CONSOLEIO_write, strlen(hello), hello);
 
@@ -206,7 +207,7 @@ arch_init(void *par)
 	get_cmdline(par);
 
 	/* Grab the shared_info pointer and put it in a safe place. */
-	HYPERVISOR_shared_info = map_shared_info(par);
+	HYPERVISOR_shared_info = map_shared_info();
 
 	/* print out some useful information  */
 	print_start_of_day(par);
@@ -214,7 +215,6 @@ arch_init(void *par)
 #ifdef CONFIG_PARAVIRT
 	memcpy(&start_info, par, sizeof(start_info));
 #endif
-	start_info_ptr = (start_info_t *)par;
 
 	start_kernel();
 }
@@ -253,7 +253,7 @@ void arch_post_suspend(int canceled)
     }
 #endif
 
-    HYPERVISOR_shared_info = map_shared_info((void*) start_info_ptr);
+    HYPERVISOR_shared_info = map_shared_info();
 #ifndef CONFIG_PARAVIRT
     xen_callback_vector();
 #endif
hypervisor.c +1 -1
index ba535556..6476d658 100644
@@ -65,7 +65,7 @@ int hvm_set_parameter(int idx, uint64_t value)
     return HYPERVISOR_hvm_op(HVMOP_set_param, &xhv);
 }
 
-shared_info_t *map_shared_info(void *p)
+shared_info_t *map_shared_info(void)
 {
     struct xen_add_to_physmap xatp;
 
include/hypervisor.h +1 -1
index b852a42a..aef142d1 100644
@@ -42,7 +42,7 @@ extern union start_info_union start_info_union;
 int hvm_get_parameter(int idx, uint64_t *value);
 int hvm_set_parameter(int idx, uint64_t value);
 #endif
-shared_info_t *map_shared_info(void *p);
+shared_info_t *map_shared_info(void);
 void unmap_shared_info(void);
 void force_evtchn_callback(void);
 void do_hypervisor_callback(struct pt_regs *regs);
Signature · 1 line
2.43.0
Reviewed-by on v1 2/6
#4 lore raw

Re: [MINI-OS PATCH 2/6] setup: drop parameter from map_shared_info()

Quoted 15 lines · Juergen Gross

The parameter of map_shared_info() is evaluated in PV mode only, and it is always the pointer to the initial struct start_info from boot time.

By setting start_info_ptr a little bit earlier the PV variant of map_shared_info() can just use start_info_ptr instead of a parameter.

---

arch/x86/setup.c +0 -0
index e30eb065..1bdf9509 100644

arch/x86/setup.c arch_init(void *par) · line 187

Quoted 4 lines
arch/x86/setup.c +1 -0
@@ -185,6 +184,8 @@ arch_init(void *par)
  {
  	static char hello[] = "Bootstrapping...\n";
 
+	start_info_ptr = par;

Should start_info_ptr only be set for CONFIG_PARAVIRT?

Oh, patch 6 does that. Strangely, patch 3 and 6 are not in my inbox.

Regards,
Jason

Quoted 4 lines
arch/x86/setup.c +1 -0
+
  	hpc_init();
  	(void)HYPERVISOR_console_io(CONSOLEIO_write, strlen(hello), hello);
 
#5 lore raw

PATCH 3/6 x86/pv: remove global start_info

In reply to Juergen Gross

Add an extern declaration for start_info_ptr when running in PV mode and use that instead of the copy held in start_info. This allows to drop start_info.

---
 arch/x86/setup.c     | 17 +----------------
 console.c            |  2 +-
 include/hypervisor.h | 12 ++----------
 xenbus.c             |  2 +-
 4 files changed, 5 insertions(+), 28 deletions(-)
arch/x86/setup.c +1 -16
index 1bdf9509..4e2f0749 100644
@@ -35,17 +35,8 @@
 #include <xen/arch-x86/hvm/start_info.h>
 #include <xen/hvm/params.h>
 
-#ifdef CONFIG_PARAVIRT
-/*
- * This structure contains start-of-day info, such as pagetable base pointer,
- * address of the shared_info structure, and things like that.
- */
-union start_info_union start_info_union;
-EXPORT_SYMBOL(start_info_union);
-#endif
-
 /*
- * This pointer holds a reference to the copy of the start_info struct.
+ * This pointer holds a reference to the start_info struct.
  */
 start_info_t *start_info_ptr;
 
@@ -212,10 +203,6 @@ arch_init(void *par)
 	/* print out some useful information  */
 	print_start_of_day(par);
 
-#ifdef CONFIG_PARAVIRT
-	memcpy(&start_info, par, sizeof(start_info));
-#endif
-
 	start_kernel();
 }
 
@@ -248,8 +235,6 @@ void arch_post_suspend(int canceled)
     if (canceled) {
         start_info_ptr->store_mfn = pfn_to_mfn(start_info_ptr->store_mfn);
         start_info_ptr->console.domU.mfn = pfn_to_mfn(start_info_ptr->console.domU.mfn);
-    } else {
-        memcpy(&start_info, start_info_ptr, sizeof(start_info_t));
     }
 #endif
 
console.c +1 -1
index 0107b685..a4c50371 100644
@@ -411,7 +411,7 @@ void xencons_ring_resume(struct consfront_dev *dev)
     if ( dev )
     {
 #if CONFIG_PARAVIRT
-        get_console(&start_info);
+        get_console(start_info_ptr);
 #else
         get_console(0);
 #endif
include/hypervisor.h +2 -10
index aef142d1..f40bc8c3 100644
@@ -28,16 +28,8 @@
 
 /* hypervisor.c */
 #ifdef CONFIG_PARAVIRT
-/*
- * a placeholder for the start of day information passed up from the hypervisor
- */
-union start_info_union
-{
-    start_info_t start_info;
-    char padding[512];
-};
-extern union start_info_union start_info_union;
-#define start_info (start_info_union.start_info)
+/* A pointer to the start of day information passed up from the hypervisor. */
+extern start_info_t *start_info_ptr;
 #else
 int hvm_get_parameter(int idx, uint64_t *value);
 int hvm_set_parameter(int idx, uint64_t value);
xenbus.c +1 -1
index 686428a4..43fdebd7 100644
@@ -469,7 +469,7 @@ void resume_xenbus(int canceled)
     struct xsd_sockmsg *rep;
 
 #ifdef CONFIG_PARAVIRT
-    get_xenbus(&start_info);
+    get_xenbus(start_info_ptr);
 #else
     get_xenbus(0);
 #endif
Signature · 1 line
2.43.0
#7 lore raw

PATCH 4/6 console: drop parameter from get_console()

In reply to Juergen Gross

The parameter of get_console() is used only when running in PV mode, and it always has the same value.

Drop the parameter and use start_info_ptr directly in the PV variant of get_console().

---
 arch/arm/setup.c  |  2 +-
 arch/x86/setup.c  |  2 +-
 console.c         | 10 +++++-----
 include/console.h |  2 +-
 4 files changed, 8 insertions(+), 8 deletions(-)
arch/arm/setup.c +1 -1
index 1bcfa689..161d137d 100644
@@ -40,7 +40,7 @@ void arch_init(void *dtb_pointer, uint32_t physical_offset)
     /* Map shared_info page */
     HYPERVISOR_shared_info = map_shared_info();
 
-    get_console(NULL);
+    get_console();
     get_xenbus(NULL);
 
     gic_init();
arch/x86/setup.c +1 -1
index 4e2f0749..2b314aa2 100644
@@ -193,7 +193,7 @@ arch_init(void *par)
 
 	/* WARN: don't do printk before here, it uses information from
 	   shared_info. Use xprintk instead. */
-	get_console(par);
+	get_console();
 	get_xenbus(par);
 	get_cmdline(par);
 
console.c +5 -5
index a4c50371..48c5be43 100644
@@ -197,15 +197,15 @@ uint32_t console_evtchn;
 static struct consfront_dev* resume_xen_console(struct consfront_dev *dev);
 
 #ifdef CONFIG_PARAVIRT
-void get_console(void *p)
+void get_console(void)
 {
-    start_info_t *si = p;
+    start_info_t *si = start_info_ptr;
 
     console_ring = mfn_to_virt(si->console.domU.mfn);
     console_evtchn = si->console.domU.evtchn;
 }
 #else
-void get_console(void *p)
+void get_console(void)
 {
     uint64_t v = -1;
 
@@ -411,9 +411,9 @@ void xencons_ring_resume(struct consfront_dev *dev)
     if ( dev )
     {
 #if CONFIG_PARAVIRT
-        get_console(start_info_ptr);
+        get_console();
 #else
-        get_console(0);
+        get_console();
 #endif
         resume_xen_console(dev);
     }
include/console.h +1 -1
index d216d247..326e993b 100644
@@ -76,7 +76,7 @@ void xprintk(const char *fmt, ...) __attribute__ ((format (printf, 1, 2)));
 void xencons_rx(char *buf, unsigned len, struct pt_regs *regs);
 void xencons_tx(void);
 
-void get_console(void *p);
+void get_console(void);
 void init_console(void);
 void console_print(struct consfront_dev *dev, const char *data, int length);
 void fini_consfront(struct consfront_dev *dev);
Signature · 1 line
2.43.0
Reviewed-by on v1 4/6
#8 lore raw

Re: [MINI-OS PATCH 4/6] console: drop parameter from get_console()

Quoted 7 lines · Juergen Gross

The parameter of get_console() is used only when running in PV mode, and it always has the same value.

Drop the parameter and use start_info_ptr directly in the PV variant of get_console().

#9 lore raw

PATCH 5/6 xenbus: drop parameter from get_xenbus()

In reply to Juergen Gross

The parameter of get_xenbus() is used only when running in PV mode, and it always has the same value.

Drop the parameter and use start_info_ptr directly in the PV variant of get_xenbus().

---
 arch/arm/setup.c |  2 +-
 arch/x86/setup.c |  2 +-
 include/xenbus.h |  4 ++--
 xenbus.c         | 13 +++++--------
 4 files changed, 9 insertions(+), 12 deletions(-)
arch/arm/setup.c +1 -1
index 161d137d..e918a064 100644
@@ -41,7 +41,7 @@ void arch_init(void *dtb_pointer, uint32_t physical_offset)
     HYPERVISOR_shared_info = map_shared_info();
 
     get_console();
-    get_xenbus(NULL);
+    get_xenbus();
 
     gic_init();
 
arch/x86/setup.c +1 -1
index 2b314aa2..590f51a8 100644
@@ -194,7 +194,7 @@ arch_init(void *par)
 	/* WARN: don't do printk before here, it uses information from
 	   shared_info. Use xprintk instead. */
 	get_console();
-	get_xenbus(par);
+	get_xenbus();
 	get_cmdline(par);
 
 	/* Grab the shared_info pointer and put it in a safe place. */
include/xenbus.h +2 -2
index 542ee456..21da05e2 100644
@@ -12,7 +12,7 @@ extern struct xenstore_domain_interface *xenstore_buf;
 
 /* Initialize the XenBus system. */
 void init_xenbus(void);
-void get_xenbus(void *p);
+void get_xenbus(void);
 #else
 #define xenbus_evtchn ~0
 #define xenstore_buf NULL
@@ -20,7 +20,7 @@ void get_xenbus(void *p);
 static inline void init_xenbus(void)
 {
 }
-static inline void get_xenbus(void *p)
+static inline void get_xenbus(void)
 {
 }
 #endif
xenbus.c +5 -8
index 43fdebd7..8225ed4b 100644
@@ -74,15 +74,15 @@ uint32_t xenbus_evtchn;
 EXPORT_SYMBOL(xenbus_evtchn);
 
 #ifdef CONFIG_PARAVIRT
-void get_xenbus(void *p)
+void get_xenbus(void)
 {
-    start_info_t *si = p;
+    start_info_t *si = start_info_ptr;
 
     xenbus_evtchn = si->store_evtchn;
     xenstore_buf = mfn_to_virt(si->store_mfn);
 }
 #else
-void get_xenbus(void *p)
+void get_xenbus(void)
 {
     uint64_t v;
 
@@ -468,11 +468,8 @@ void resume_xenbus(int canceled)
     struct write_req req[2];
     struct xsd_sockmsg *rep;
 
-#ifdef CONFIG_PARAVIRT
-    get_xenbus(start_info_ptr);
-#else
-    get_xenbus(0);
-#endif
+    get_xenbus();
+
     unmask_evtchn(xenbus_evtchn);
 
     if ( !canceled )
Signature · 1 line
2.43.0
#11 lore raw

PATCH 6/6 x86/pv: make start_info_ptr PV-only

In reply to Juergen Gross

Hide start_info_ptr via CONFIG_PARAVIRT in order to avoid accesses to it form not PV specific code.

---
 arch/x86/setup.c | 16 +++++++++++-----
 1 file changed, 11 insertions(+), 5 deletions(-)
arch/x86/setup.c +11 -5
index 590f51a8..b3fc835a 100644
@@ -35,11 +35,6 @@
 #include <xen/arch-x86/hvm/start_info.h>
 #include <xen/hvm/params.h>
 
-/*
- * This pointer holds a reference to the start_info struct.
- */
-start_info_t *start_info_ptr;
-
 /*
  * Shared page for communicating with the hypervisor.
  * Events flags go here, for example.
@@ -68,6 +63,11 @@ static inline void sse_init(void) {
 #endif
 
 #ifdef CONFIG_PARAVIRT
+/*
+ * This pointer holds a reference to the start_info struct.
+ */
+start_info_t *start_info_ptr;
+
 #define hpc_init()
 
 shared_info_t *map_shared_info(void)
@@ -175,7 +175,9 @@ arch_init(void *par)
 {
 	static char hello[] = "Bootstrapping...\n";
 
+#ifdef CONFIG_PARAVIRT
 	start_info_ptr = par;
+#endif
 
 	hpc_init();
 	(void)HYPERVISOR_console_io(CONSOLEIO_write, strlen(hello), hello);
@@ -226,7 +228,11 @@ int arch_suspend(void)
      * This hypercall returns 1 if the suspend
      * was cancelled and 0 if resuming in a new domain
      */
+#ifdef CONFIG_PARAVIRT
     return HYPERVISOR_suspend(virt_to_mfn(start_info_ptr));
+#else
+    return HYPERVISOR_suspend(0);
+#endif
 }
 
 void arch_post_suspend(int canceled)
Signature · 1 line
2.43.0
#13 lore raw
In reply to Juergen Gross
Quoted 14 lines · Juergen Gross

While working on xenstore-stubdom live update support I came across some really unpleasant code using struct start_info even when running in PVH mode.

Especially suspend/resume was doing some crazy stuff, including overwriting the initial struct hvm_start_info partially.

This series is hiding struct start_info from PVH code. This didn't result in any runtime error, but in future I'll need to access struct hvm_start_info some time after boot, so I need to get that fixed.

Tested especially with xl suspend/resume and xl save/restore in PV and in PVH mode.

Quoted 7 lines

Juergen Gross (6):
suspend: drop crazy pfn handling for PVH
setup: drop parameter from map_shared_info()
x86/pv: remove global start_info
console: drop parameter from get_console()
xenbus: drop parameter from get_xenbus()
x86/pv: make start_info_ptr PV-only

Before I even try: Does this series interfere in any way with the kexec one, and hence may need re-basing on top of that (which I've pushed a minute ago)?

Jan

#14 lore raw
Quoted 23 lines · Jan Beulich
Quoted 14 lines · Juergen Gross

While working on xenstore-stubdom live update support I came across some really unpleasant code using struct start_info even when running in PVH mode.

Especially suspend/resume was doing some crazy stuff, including overwriting the initial struct hvm_start_info partially.

This series is hiding struct start_info from PVH code. This didn't result in any runtime error, but in future I'll need to access struct hvm_start_info some time after boot, so I need to get that fixed.

Tested especially with xl suspend/resume and xl save/restore in PV and in PVH mode.

Quoted 7 lines

Juergen Gross (6):
suspend: drop crazy pfn handling for PVH
setup: drop parameter from map_shared_info()
x86/pv: remove global start_info
console: drop parameter from get_console()
xenbus: drop parameter from get_xenbus()
x86/pv: make start_info_ptr PV-only

Quoted 2 lines

Before I even try: Does this series interfere in any way with the kexec one, and hence may need re-basing on top of that (which I've pushed a minute ago)?

I'm not aware of any possible conflict.

Juergen