Skip to contentSkip to conversation

Hyper-V: kexec fixes for L1VH (mshv)

PATCH Applied Jork Loeser 5 participants 13 messages Apr 27 – May 5

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

v4 cover Hyper-V: kexec fixes for L1VH (mshv)

This series fixes kexec support when Linux runs as an L1 Virtual Host (L1VH) under Hyper-V, using the MSHV driver to manage child VMs.

1-2. SynIC cleanup: the MSHV driver manages its own SynIC resources
separately from vmbus. Add proper teardown of MSHV-owned SINTs and SIRBP on kexec, scoped to only the resources MSHV owns. Use hv_vmbus_exists() to decide at runtime whether VMBus owns SIMP/SIEFP/SCONTROL (so MSHV must not touch them) or whether MSHV must manage them itself (bare root partition without VMBus). Also fix SIEFP and SIRBP address calculations to use HV_HYP_PAGE_SHIFT instead of PAGE_SHIFT, which produces wrong addresses on ARM64 with 64K pages.

3. Debugfs stats pages: unmap the VP statistics overlay pages before
kexec to avoid machine check exceptions when the new kernel reuses those physical pages.

Changes since v3:
- Dropped patches 1-3 (vmbus variable shadowing, stimer cleanup, LP/VP skip), now merged via hyperv-next.
- Patch 1: fix SIEFP and SIRBP memremap()/virt_to_phys() to use HV_HYP_PAGE_SHIFT/HV_HYP_PAGE_SIZE instead of PAGE_SHIFT/PAGE_SIZE.

Changes since v2:
- Rebased onto linux-next/master to adapt to the upstream SynIC refactor (commit 5a674ef871fe, "mshv: refactor synic init and cleanup").

Changes since v1:
- Patch 1: account for nested root partitions where VMBus is also active (not just L1VH); use a vmbus_active local variable; allocate SIRBP in L1VH allocation path for when the hypervisor doesn't pre-provision the page.

Jork Loeser (3):
mshv: limit SynIC management to MSHV-owned resources
mshv: clean up SynIC state on kexec for L1VH
mshv: unmap debugfs stats pages on kexec

 drivers/hv/hv.c           |   3 +
 drivers/hv/mshv_debugfs.c |   7 +-
 drivers/hv/mshv_synic.c   | 154 +++++++++++++++++++++++++-------------
 3 files changed, 110 insertions(+), 54 deletions(-)
Signature · 1 line
2.43.0
#1 lore raw

PATCH v4 1/3 mshv: limit SynIC management to MSHV-owned resources

The SynIC is shared between VMBus and MSHV. VMBus owns the message page (SIMP), event flags page (SIEFP), global enable (SCONTROL), and SINT2. MSHV adds SINT0, SINT5, and the event ring page (SIRBP).

Currently mshv_synic_cpu_init() redundantly enables SIMP, SIEFP, and SCONTROL that VMBus already configured, and mshv_synic_cpu_exit() disables all of them. This is wrong because MSHV can be torn down while VMBus is still active. In particular, a kexec reboot notifier tears down MSHV first. Disabling SCONTROL, SIMP, and SIEFP out from under VMBus causes its later cleanup to write SynIC MSRs while SynIC is disabled, which the hypervisor does not tolerate.

Restrict MSHV to managing only the resources it owns:
- SINT0, SINT5: mask on cleanup, unmask on init
- SIRBP: enable/disable as before
- SIMP, SIEFP, SCONTROL: leave to VMBus when it is active (L1VH and nested root partition); on a non-nested root partition VMBus does not run, so MSHV must enable/disable them

While here, fix the SIEFP and SIRBP memremap() and virt_to_phys() calls to use HV_HYP_PAGE_SHIFT/HV_HYP_PAGE_SIZE instead of PAGE_SHIFT/PAGE_SIZE. The hypervisor always uses 4K pages for SynIC register GPAs regardless of the kernel page size, so using PAGE_SHIFT produces wrong addresses on ARM64 with 64K pages.

Note that initialization order matters - VMBUS first, MSHV second, and the reverse on de-init. Ideally, we would want a dedicated SYNIC driver that replaces the cross-dependencies with a clear API and dynamic tracking. Such refactor should go into its own dedicated series, outside of this kexec fix series.

---
 drivers/hv/hv.c         |   3 +
 drivers/hv/mshv_synic.c | 150 ++++++++++++++++++++++++++--------------
 2 files changed, 103 insertions(+), 50 deletions(-)
drivers/hv/hv.c +3 -0
index ae60fd542292..ef4b1b03395d 100644
@@ -272,6 +272,9 @@ void hv_synic_free(void)
 /*
  * hv_hyp_synic_enable_regs - Initialize the Synthetic Interrupt Controller
  * with the hypervisor.
+ *
+ * Note: When MSHV is present, mshv_synic_cpu_init() intializes further
+ * registers later.
  */
 void hv_hyp_synic_enable_regs(unsigned int cpu)
 {
drivers/hv/mshv_synic.c +100 -50

224 lines not on this page · raw message

Signature · 1 line
2.43.0
Reviewed-by on v4 1/3
#2 lore raw

Re: [PATCH v4 1/3] mshv: limit SynIC management to MSHV-owned resources

Quoted 91 lines · Jork Loeser

The SynIC is shared between VMBus and MSHV. VMBus owns the message page (SIMP), event flags page (SIEFP), global enable (SCONTROL), and SINT2. MSHV adds SINT0, SINT5, and the event ring page (SIRBP).

Currently mshv_synic_cpu_init() redundantly enables SIMP, SIEFP, and SCONTROL that VMBus already configured, and mshv_synic_cpu_exit() disables all of them. This is wrong because MSHV can be torn down while VMBus is still active. In particular, a kexec reboot notifier tears down MSHV first. Disabling SCONTROL, SIMP, and SIEFP out from under VMBus causes its later cleanup to write SynIC MSRs while SynIC is disabled, which the hypervisor does not tolerate.

Restrict MSHV to managing only the resources it owns:
- SINT0, SINT5: mask on cleanup, unmask on init
- SIRBP: enable/disable as before
- SIMP, SIEFP, SCONTROL: leave to VMBus when it is active (L1VH and nested root partition); on a non-nested root partition VMBus does not run, so MSHV must enable/disable them

While here, fix the SIEFP and SIRBP memremap() and virt_to_phys() calls to use HV_HYP_PAGE_SHIFT/HV_HYP_PAGE_SIZE instead of PAGE_SHIFT/PAGE_SIZE. The hypervisor always uses 4K pages for SynIC register GPAs regardless of the kernel page size, so using PAGE_SHIFT produces wrong addresses on ARM64 with 64K pages.

Note that initialization order matters - VMBUS first, MSHV second, and the reverse on de-init. Ideally, we would want a dedicated SYNIC driver that replaces the cross-dependencies with a clear API and dynamic tracking. Such refactor should go into its own dedicated series, outside of this kexec fix series.

---
 drivers/hv/hv.c         |   3 +
 drivers/hv/mshv_synic.c | 150 ++++++++++++++++++++++++++--------------
 2 files changed, 103 insertions(+), 50 deletions(-)
drivers/hv/hv.c +3 -0
index ae60fd542292..ef4b1b03395d 100644
@@ -272,6 +272,9 @@ void hv_synic_free(void)
 /*
  * hv_hyp_synic_enable_regs - Initialize the Synthetic Interrupt Controller
  * with the hypervisor.
+ *
+ * Note: When MSHV is present, mshv_synic_cpu_init() intializes further
+ * registers later.
  */
 void hv_hyp_synic_enable_regs(unsigned int cpu)
 {
drivers/hv/mshv_synic.c +14 -3
index e2288a726fec..2db3b0192eac 100644
@@ -13,6 +13,7 @@
 #include <linux/interrupt.h>
 #include <linux/io.h>
 #include <linux/cpuhotplug.h>
+#include <linux/hyperv.h>
 #include <linux/reboot.h>
 #include <asm/mshyperv.h>
 #include <linux/acpi.h>
@@ -456,46 +457,75 @@ static int mshv_synic_cpu_init(unsigned int cpu)
 	union hv_synic_siefp siefp;
 	union hv_synic_sirbp sirbp;
 	union hv_synic_sint sint;
-	union hv_synic_scontrol sctrl;
 	struct hv_synic_pages *spages = this_cpu_ptr(synic_pages);
 	struct hv_message_page **msg_page = &spages->hyp_synic_message_page;
 	struct hv_synic_event_flags_page **event_flags_page =
 			&spages->synic_event_flags_page;
 	struct hv_synic_event_ring_page **event_ring_page =
 			&spages->synic_event_ring_page;
+	/*
+	 * VMBus owns SIMP/SIEFP/SCONTROL when it is active.
+	 * See hv_hyp_synic_enable_regs() for that initialization.
+	 */
+	bool vmbus_active = hv_vmbus_exists();
 
-	/* Setup the Synic's message page */
+	/*
+	 * Map the SYNIC message page. When VMBus is not active the
+	 * hypervisor pre-provisions the SIMP GPA but may not set
+	 * simp_enabled — enable it here.
+	 */
 	simp.as_uint64 = hv_get_non_nested_msr(HV_MSR_SIMP);
-	simp.simp_enabled = true;
+	if (!vmbus_active) {
+		simp.simp_enabled = true;
+		hv_set_non_nested_msr(HV_MSR_SIMP, simp.as_uint64);

drivers/hv/mshv_synic.c static int mshv_synic_cpu_init(unsigned int cpu) · line 487

Quoted 10 lines
drivers/hv/mshv_synic.c +2 -3
+	}
 	*msg_page = memremap(simp.base_simp_gpa << HV_HYP_PAGE_SHIFT,
 			     HV_HYP_PAGE_SIZE,
 			     MEMREMAP_WB);
 
 	if (!(*msg_page))
-		return -EFAULT;
-
-	hv_set_non_nested_msr(HV_MSR_SIMP, simp.as_uint64);
+		goto cleanup_simp;

It would be cleaner (and simpler to read), if there would be another goto label to only unset HV_MSR_SIMP instead of checking *msg_page for NULL again in the cleanup_simp label.

This applies to all the goto labels in this function.

Quoted 186 lines
drivers/hv/mshv_synic.c +84 -44
-	/* Setup the Synic's event flags page */
+	/*
+	 * Map the event flags page. Same as SIMP: enable when
+	 * VMBus is not active, already enabled by VMBus otherwise.
+	 */
 	siefp.as_uint64 = hv_get_non_nested_msr(HV_MSR_SIEFP);
-	siefp.siefp_enabled = true;
-	*event_flags_page = memremap(siefp.base_siefp_gpa << PAGE_SHIFT,
-				     PAGE_SIZE, MEMREMAP_WB);
+	if (!vmbus_active) {
+		siefp.siefp_enabled = true;
+		hv_set_non_nested_msr(HV_MSR_SIEFP, siefp.as_uint64);
+	}
+	*event_flags_page = memremap(siefp.base_siefp_gpa << HV_HYP_PAGE_SHIFT,
+				     HV_HYP_PAGE_SIZE, MEMREMAP_WB);
 
 	if (!(*event_flags_page))
-		goto cleanup;
-
-	hv_set_non_nested_msr(HV_MSR_SIEFP, siefp.as_uint64);
+		goto cleanup_siefp;
 
 	/* Setup the Synic's event ring page */
 	sirbp.as_uint64 = hv_get_non_nested_msr(HV_MSR_SIRBP);
-	sirbp.sirbp_enabled = true;
-	*event_ring_page = memremap(sirbp.base_sirbp_gpa << PAGE_SHIFT,
-				    PAGE_SIZE, MEMREMAP_WB);
 
-	if (!(*event_ring_page))
-		goto cleanup;
+	if (hv_root_partition()) {
+		*event_ring_page = memremap(sirbp.base_sirbp_gpa << HV_HYP_PAGE_SHIFT,
+					    HV_HYP_PAGE_SIZE, MEMREMAP_WB);
 
+		if (!(*event_ring_page))
+			goto cleanup_siefp;
+	} else {
+		/*
+		 * On L1VH the hypervisor does not provide a SIRBP page.
+		 * Allocate one and program its GPA into the MSR.
+		 */
+		*event_ring_page = (struct hv_synic_event_ring_page *)
+			get_zeroed_page(GFP_KERNEL);
+
+		if (!(*event_ring_page))
+			goto cleanup_siefp;
+
+		sirbp.base_sirbp_gpa = virt_to_phys(*event_ring_page)
+				>> HV_HYP_PAGE_SHIFT;
+	}
+
+	sirbp.sirbp_enabled = true;
 	hv_set_non_nested_msr(HV_MSR_SIRBP, sirbp.as_uint64);
 
 	if (mshv_sint_irq != -1)
@@ -518,28 +548,30 @@ static int mshv_synic_cpu_init(unsigned int cpu)
 	hv_set_non_nested_msr(HV_MSR_SINT0 + HV_SYNIC_DOORBELL_SINT_INDEX,
 			      sint.as_uint64);
 
-	/* Enable global synic bit */
-	sctrl.as_uint64 = hv_get_non_nested_msr(HV_MSR_SCONTROL);
-	sctrl.enable = 1;
-	hv_set_non_nested_msr(HV_MSR_SCONTROL, sctrl.as_uint64);
+	/* When VMBus is active it already enabled SCONTROL. */
+	if (!vmbus_active) {
+		union hv_synic_scontrol sctrl;
+
+		sctrl.as_uint64 = hv_get_non_nested_msr(HV_MSR_SCONTROL);
+		sctrl.enable = 1;
+		hv_set_non_nested_msr(HV_MSR_SCONTROL, sctrl.as_uint64);
+	}
 
 	return 0;
 
-cleanup:
-	if (*event_ring_page) {
-		sirbp.sirbp_enabled = false;
-		hv_set_non_nested_msr(HV_MSR_SIRBP, sirbp.as_uint64);
-		memunmap(*event_ring_page);
-	}
-	if (*event_flags_page) {
+cleanup_siefp:
+	if (*event_flags_page)
+		memunmap(*event_flags_page);
+	if (!vmbus_active) {
 		siefp.siefp_enabled = false;
 		hv_set_non_nested_msr(HV_MSR_SIEFP, siefp.as_uint64);
-		memunmap(*event_flags_page);
 	}
-	if (*msg_page) {
+cleanup_simp:
+	if (*msg_page)
+		memunmap(*msg_page);
+	if (!vmbus_active) {
 		simp.simp_enabled = false;
 		hv_set_non_nested_msr(HV_MSR_SIMP, simp.as_uint64);
-		memunmap(*msg_page);
 	}
 
 	return -EFAULT;
@@ -548,16 +580,15 @@ static int mshv_synic_cpu_init(unsigned int cpu)
 static int mshv_synic_cpu_exit(unsigned int cpu)
 {
 	union hv_synic_sint sint;
-	union hv_synic_simp simp;
-	union hv_synic_siefp siefp;
 	union hv_synic_sirbp sirbp;
-	union hv_synic_scontrol sctrl;
 	struct hv_synic_pages *spages = this_cpu_ptr(synic_pages);
 	struct hv_message_page **msg_page = &spages->hyp_synic_message_page;
 	struct hv_synic_event_flags_page **event_flags_page =
 		&spages->synic_event_flags_page;
 	struct hv_synic_event_ring_page **event_ring_page =
 		&spages->synic_event_ring_page;
+	/* VMBus owns SIMP/SIEFP/SCONTROL when it is active */
+	bool vmbus_active = hv_vmbus_exists();
 
 	/* Disable the interrupt */
 	sint.as_uint64 = hv_get_non_nested_msr(HV_MSR_SINT0 + HV_SYNIC_INTERCEPTION_SINT_INDEX);
@@ -574,28 +605,47 @@ static int mshv_synic_cpu_exit(unsigned int cpu)
 	if (mshv_sint_irq != -1)
 		disable_percpu_irq(mshv_sint_irq);
 
-	/* Disable Synic's event ring page */
+	/* Disable SYNIC event ring page owned by MSHV */
 	sirbp.as_uint64 = hv_get_non_nested_msr(HV_MSR_SIRBP);
 	sirbp.sirbp_enabled = false;
-	hv_set_non_nested_msr(HV_MSR_SIRBP, sirbp.as_uint64);
-	memunmap(*event_ring_page);
 
-	/* Disable Synic's event flags page */
-	siefp.as_uint64 = hv_get_non_nested_msr(HV_MSR_SIEFP);
-	siefp.siefp_enabled = false;
-	hv_set_non_nested_msr(HV_MSR_SIEFP, siefp.as_uint64);
+	if (hv_root_partition()) {
+		hv_set_non_nested_msr(HV_MSR_SIRBP, sirbp.as_uint64);
+		memunmap(*event_ring_page);
+	} else {
+		sirbp.base_sirbp_gpa = 0;
+		hv_set_non_nested_msr(HV_MSR_SIRBP, sirbp.as_uint64);
+		free_page((unsigned long)*event_ring_page);
+	}
+
+	/*
+	 * Release our mappings of the message and event flags pages.
+	 * When VMBus is not active, we enabled SIMP/SIEFP — disable
+	 * them. Otherwise VMBus owns the MSRs — leave them.
+	 */
 	memunmap(*event_flags_page);
+	if (!vmbus_active) {
+		union hv_synic_simp simp;
+		union hv_synic_siefp siefp;
 
-	/* Disable Synic's message page */
-	simp.as_uint64 = hv_get_non_nested_msr(HV_MSR_SIMP);
-	simp.simp_enabled = false;
-	hv_set_non_nested_msr(HV_MSR_SIMP, simp.as_uint64);
+		siefp.as_uint64 = hv_get_non_nested_msr(HV_MSR_SIEFP);
+		siefp.siefp_enabled = false;
+		hv_set_non_nested_msr(HV_MSR_SIEFP, siefp.as_uint64);
+
+		simp.as_uint64 = hv_get_non_nested_msr(HV_MSR_SIMP);
+		simp.simp_enabled = false;
+		hv_set_non_nested_msr(HV_MSR_SIMP, simp.as_uint64);
+	}
 	memunmap(*msg_page);
 
-	/* Disable global synic bit */
-	sctrl.as_uint64 = hv_get_non_nested_msr(HV_MSR_SCONTROL);
-	sctrl.enable = 0;
-	hv_set_non_nested_msr(HV_MSR_SCONTROL, sctrl.as_uint64);
+	/* When VMBus is active it owns SCONTROL — leave it. */
+	if (!vmbus_active) {
+		union hv_synic_scontrol sctrl;
+
+		sctrl.as_uint64 = hv_get_non_nested_msr(HV_MSR_SCONTROL);
+		sctrl.enable = 0;
+		hv_set_non_nested_msr(HV_MSR_SCONTROL, sctrl.as_uint64);
+	}
 
 	return 0;
 }
Signature · 1 line
2.43.0
Reviewed-by on v4 1/3
#3 lore raw

Re: [PATCH v4 1/3] mshv: limit SynIC management to MSHV-owned resources

In reply to Jork Loeser
Quoted 31 lines · Jork Loeser

The SynIC is shared between VMBus and MSHV. VMBus owns the message page (SIMP), event flags page (SIEFP), global enable (SCONTROL), and SINT2. MSHV adds SINT0, SINT5, and the event ring page (SIRBP).

Currently mshv_synic_cpu_init() redundantly enables SIMP, SIEFP, and SCONTROL that VMBus already configured, and mshv_synic_cpu_exit() disables all of them. This is wrong because MSHV can be torn down while VMBus is still active. In particular, a kexec reboot notifier tears down MSHV first. Disabling SCONTROL, SIMP, and SIEFP out from under VMBus causes its later cleanup to write SynIC MSRs while SynIC is disabled, which the hypervisor does not tolerate.

Restrict MSHV to managing only the resources it owns:
- SINT0, SINT5: mask on cleanup, unmask on init
- SIRBP: enable/disable as before
- SIMP, SIEFP, SCONTROL: leave to VMBus when it is active (L1VH and nested root partition); on a non-nested root partition VMBus does not run, so MSHV must enable/disable them

While here, fix the SIEFP and SIRBP memremap() and virt_to_phys() calls to use HV_HYP_PAGE_SHIFT/HV_HYP_PAGE_SIZE instead of PAGE_SHIFT/PAGE_SIZE. The hypervisor always uses 4K pages for SynIC register GPAs regardless of the kernel page size, so using PAGE_SHIFT produces wrong addresses on ARM64 with 64K pages.

Note that initialization order matters - VMBUS first, MSHV second, and the reverse on de-init. Ideally, we would want a dedicated SYNIC driver that replaces the cross-dependencies with a clear API and dynamic tracking. Such refactor should go into its own dedicated series, outside of this kexec fix series.

Quoted 5 lines
---
 drivers/hv/hv.c         |   3 +
 drivers/hv/mshv_synic.c | 150 ++++++++++++++++++++++++++--------------
 2 files changed, 103 insertions(+), 50 deletions(-)
#4 lore raw

RE: [PATCH v4 1/3] mshv: limit SynIC management to MSHV-owned resources

In reply to Jork Loeser

From: Jork Loeser <[email protected]> Sent: Monday, April 27, 2026 2:39 PM

Quoted 20 lines

The SynIC is shared between VMBus and MSHV. VMBus owns the message page (SIMP), event flags page (SIEFP), global enable (SCONTROL), and SINT2. MSHV adds SINT0, SINT5, and the event ring page (SIRBP).

Currently mshv_synic_cpu_init() redundantly enables SIMP, SIEFP, and SCONTROL that VMBus already configured, and mshv_synic_cpu_exit() disables all of them. This is wrong because MSHV can be torn down while VMBus is still active. In particular, a kexec reboot notifier tears down MSHV first. Disabling SCONTROL, SIMP, and SIEFP out from under VMBus causes its later cleanup to write SynIC MSRs while SynIC is disabled, which the hypervisor does not tolerate.

Restrict MSHV to managing only the resources it owns:
- SINT0, SINT5: mask on cleanup, unmask on init
- SIRBP: enable/disable as before
- SIMP, SIEFP, SCONTROL: leave to VMBus when it is active (L1VH and nested root partition); on a non-nested root partition VMBus does not run, so MSHV must enable/disable them

Quoted 5 lines

While here, fix the SIEFP and SIRBP memremap() and virt_to_phys() calls to use HV_HYP_PAGE_SHIFT/HV_HYP_PAGE_SIZE instead of PAGE_SHIFT/PAGE_SIZE. The hypervisor always uses 4K pages for SynIC register GPAs regardless of the kernel page size, so using PAGE_SHIFT produces wrong addresses on ARM64 with 64K pages.

I agree that this is a good change. But any kernel image built with CONFIG_MSHV_ROOT set must use only 4KiB pages, as enforced by the dependency in drivers/hv/Kconfig. The change makes the code explicitly match the SynIC register layout, which is good, but it doesn't actually fix a problem since root MSHV code can't run on ARM64 with 64KiB pages. My only concern is that this commit message should not imply that an ARM64/64KiB configuration is possible for the root.

Michael

Quoted 265 lines

Note that initialization order matters - VMBUS first, MSHV second, and the reverse on de-init. Ideally, we would want a dedicated SYNIC driver that replaces the cross-dependencies with a clear API and dynamic tracking. Such refactor should go into its own dedicated series, outside of this kexec fix series.

---
 drivers/hv/hv.c         |   3 +
 drivers/hv/mshv_synic.c | 150 ++++++++++++++++++++++++++--------------
 2 files changed, 103 insertions(+), 50 deletions(-)
drivers/hv/hv.c +3 -0
index ae60fd542292..ef4b1b03395d 100644
@@ -272,6 +272,9 @@ void hv_synic_free(void)
 /*
  * hv_hyp_synic_enable_regs - Initialize the Synthetic Interrupt Controller
  * with the hypervisor.
+ *
+ * Note: When MSHV is present, mshv_synic_cpu_init() intializes further
+ * registers later.
  */
 void hv_hyp_synic_enable_regs(unsigned int cpu)
 {
drivers/hv/mshv_synic.c +48 -18
index e2288a726fec..2db3b0192eac 100644
@@ -13,6 +13,7 @@
 #include <linux/interrupt.h>
 #include <linux/io.h>
 #include <linux/cpuhotplug.h>
+#include <linux/hyperv.h>
 #include <linux/reboot.h>
 #include <asm/mshyperv.h>
 #include <linux/acpi.h>
@@ -456,46 +457,75 @@ static int mshv_synic_cpu_init(unsigned int cpu)
 	union hv_synic_siefp siefp;
 	union hv_synic_sirbp sirbp;
 	union hv_synic_sint sint;
-	union hv_synic_scontrol sctrl;
 	struct hv_synic_pages *spages = this_cpu_ptr(synic_pages);
 	struct hv_message_page **msg_page = &spages->hyp_synic_message_page;
 	struct hv_synic_event_flags_page **event_flags_page =
 			&spages->synic_event_flags_page;
 	struct hv_synic_event_ring_page **event_ring_page =
 			&spages->synic_event_ring_page;
+	/*
+	 * VMBus owns SIMP/SIEFP/SCONTROL when it is active.
+	 * See hv_hyp_synic_enable_regs() for that initialization.
+	 */
+	bool vmbus_active = hv_vmbus_exists();
 
-	/* Setup the Synic's message page */
+	/*
+	 * Map the SYNIC message page. When VMBus is not active the
+	 * hypervisor pre-provisions the SIMP GPA but may not set
+	 * simp_enabled - enable it here.
+	 */
 	simp.as_uint64 = hv_get_non_nested_msr(HV_MSR_SIMP);
-	simp.simp_enabled = true;
+	if (!vmbus_active) {
+		simp.simp_enabled = true;
+		hv_set_non_nested_msr(HV_MSR_SIMP, simp.as_uint64);
+	}
 	*msg_page = memremap(simp.base_simp_gpa << HV_HYP_PAGE_SHIFT,
 			     HV_HYP_PAGE_SIZE,
 			     MEMREMAP_WB);
 
 	if (!(*msg_page))
-		return -EFAULT;
-
-	hv_set_non_nested_msr(HV_MSR_SIMP, simp.as_uint64);
+		goto cleanup_simp;
 
-	/* Setup the Synic's event flags page */
+	/*
+	 * Map the event flags page. Same as SIMP: enable when
+	 * VMBus is not active, already enabled by VMBus otherwise.
+	 */
 	siefp.as_uint64 = hv_get_non_nested_msr(HV_MSR_SIEFP);
-	siefp.siefp_enabled = true;
-	*event_flags_page = memremap(siefp.base_siefp_gpa << PAGE_SHIFT,
-				     PAGE_SIZE, MEMREMAP_WB);
+	if (!vmbus_active) {
+		siefp.siefp_enabled = true;
+		hv_set_non_nested_msr(HV_MSR_SIEFP, siefp.as_uint64);
+	}
+	*event_flags_page = memremap(siefp.base_siefp_gpa << HV_HYP_PAGE_SHIFT,
+				     HV_HYP_PAGE_SIZE, MEMREMAP_WB);
 
 	if (!(*event_flags_page))
-		goto cleanup;
-
-	hv_set_non_nested_msr(HV_MSR_SIEFP, siefp.as_uint64);
+		goto cleanup_siefp;
 
 	/* Setup the Synic's event ring page */
 	sirbp.as_uint64 = hv_get_non_nested_msr(HV_MSR_SIRBP);
-	sirbp.sirbp_enabled = true;
-	*event_ring_page = memremap(sirbp.base_sirbp_gpa << PAGE_SHIFT,
-				    PAGE_SIZE, MEMREMAP_WB);
 
-	if (!(*event_ring_page))
-		goto cleanup;
+	if (hv_root_partition()) {
+		*event_ring_page = memremap(sirbp.base_sirbp_gpa <<
 HV_HYP_PAGE_SHIFT,
+					    HV_HYP_PAGE_SIZE, MEMREMAP_WB);
 
+		if (!(*event_ring_page))
+			goto cleanup_siefp;
+	} else {
+		/*
+		 * On L1VH the hypervisor does not provide a SIRBP page.
+		 * Allocate one and program its GPA into the MSR.
+		 */
+		*event_ring_page = (struct hv_synic_event_ring_page *)
+			get_zeroed_page(GFP_KERNEL);
+
+		if (!(*event_ring_page))
+			goto cleanup_siefp;
+
+		sirbp.base_sirbp_gpa = virt_to_phys(*event_ring_page)
+				>> HV_HYP_PAGE_SHIFT;
+	}
+
+	sirbp.sirbp_enabled = true;
 	hv_set_non_nested_msr(HV_MSR_SIRBP, sirbp.as_uint64);
 
drivers/hv/mshv_synic.c +18 -17
 	if (mshv_sint_irq != -1)
@@ -518,28 +548,30 @@ static int mshv_synic_cpu_init(unsigned int cpu)
 	hv_set_non_nested_msr(HV_MSR_SINT0 + HV_SYNIC_DOORBELL_SINT_INDEX,
 			      sint.as_uint64);
 
-	/* Enable global synic bit */
-	sctrl.as_uint64 = hv_get_non_nested_msr(HV_MSR_SCONTROL);
-	sctrl.enable = 1;
-	hv_set_non_nested_msr(HV_MSR_SCONTROL, sctrl.as_uint64);
+	/* When VMBus is active it already enabled SCONTROL. */
+	if (!vmbus_active) {
+		union hv_synic_scontrol sctrl;
+
+		sctrl.as_uint64 = hv_get_non_nested_msr(HV_MSR_SCONTROL);
+		sctrl.enable = 1;
+		hv_set_non_nested_msr(HV_MSR_SCONTROL, sctrl.as_uint64);
+	}
 
 	return 0;
 
-cleanup:
-	if (*event_ring_page) {
-		sirbp.sirbp_enabled = false;
-		hv_set_non_nested_msr(HV_MSR_SIRBP, sirbp.as_uint64);
-		memunmap(*event_ring_page);
-	}
-	if (*event_flags_page) {
+cleanup_siefp:
+	if (*event_flags_page)
+		memunmap(*event_flags_page);
+	if (!vmbus_active) {
 		siefp.siefp_enabled = false;
 		hv_set_non_nested_msr(HV_MSR_SIEFP, siefp.as_uint64);
-		memunmap(*event_flags_page);
 	}
-	if (*msg_page) {
+cleanup_simp:
+	if (*msg_page)
+		memunmap(*msg_page);
+	if (!vmbus_active) {
 		simp.simp_enabled = false;
 		hv_set_non_nested_msr(HV_MSR_SIMP, simp.as_uint64);
-		memunmap(*msg_page);
 	}
 
 	return -EFAULT;
@@ -548,16 +580,15 @@ static int mshv_synic_cpu_init(unsigned int cpu)
 static int mshv_synic_cpu_exit(unsigned int cpu)
 {
 	union hv_synic_sint sint;
-	union hv_synic_simp simp;
-	union hv_synic_siefp siefp;
 	union hv_synic_sirbp sirbp;
-	union hv_synic_scontrol sctrl;
 	struct hv_synic_pages *spages = this_cpu_ptr(synic_pages);
 	struct hv_message_page **msg_page = &spages->hyp_synic_message_page;
 	struct hv_synic_event_flags_page **event_flags_page =
 		&spages->synic_event_flags_page;
 	struct hv_synic_event_ring_page **event_ring_page =
 		&spages->synic_event_ring_page;
+	/* VMBus owns SIMP/SIEFP/SCONTROL when it is active */
+	bool vmbus_active = hv_vmbus_exists();
 
 	/* Disable the interrupt */
 	sint.as_uint64 = hv_get_non_nested_msr(HV_MSR_SINT0 +

HV_SYNIC_INTERCEPTION_SINT_INDEX);

drivers/hv/mshv_synic.c +34 -16
@@ -574,28 +605,47 @@ static int mshv_synic_cpu_exit(unsigned int cpu)
 	if (mshv_sint_irq != -1)
 		disable_percpu_irq(mshv_sint_irq);
 
-	/* Disable Synic's event ring page */
+	/* Disable SYNIC event ring page owned by MSHV */
 	sirbp.as_uint64 = hv_get_non_nested_msr(HV_MSR_SIRBP);
 	sirbp.sirbp_enabled = false;
-	hv_set_non_nested_msr(HV_MSR_SIRBP, sirbp.as_uint64);
-	memunmap(*event_ring_page);
 
-	/* Disable Synic's event flags page */
-	siefp.as_uint64 = hv_get_non_nested_msr(HV_MSR_SIEFP);
-	siefp.siefp_enabled = false;
-	hv_set_non_nested_msr(HV_MSR_SIEFP, siefp.as_uint64);
+	if (hv_root_partition()) {
+		hv_set_non_nested_msr(HV_MSR_SIRBP, sirbp.as_uint64);
+		memunmap(*event_ring_page);
+	} else {
+		sirbp.base_sirbp_gpa = 0;
+		hv_set_non_nested_msr(HV_MSR_SIRBP, sirbp.as_uint64);
+		free_page((unsigned long)*event_ring_page);
+	}
+
+	/*
+	 * Release our mappings of the message and event flags pages.
+	 * When VMBus is not active, we enabled SIMP/SIEFP - disable
+	 * them. Otherwise VMBus owns the MSRs - leave them.
+	 */
 	memunmap(*event_flags_page);
+	if (!vmbus_active) {
+		union hv_synic_simp simp;
+		union hv_synic_siefp siefp;
 
-	/* Disable Synic's message page */
-	simp.as_uint64 = hv_get_non_nested_msr(HV_MSR_SIMP);
-	simp.simp_enabled = false;
-	hv_set_non_nested_msr(HV_MSR_SIMP, simp.as_uint64);
+		siefp.as_uint64 = hv_get_non_nested_msr(HV_MSR_SIEFP);
+		siefp.siefp_enabled = false;
+		hv_set_non_nested_msr(HV_MSR_SIEFP, siefp.as_uint64);
+
+		simp.as_uint64 = hv_get_non_nested_msr(HV_MSR_SIMP);
+		simp.simp_enabled = false;
+		hv_set_non_nested_msr(HV_MSR_SIMP, simp.as_uint64);
+	}
 	memunmap(*msg_page);
 
-	/* Disable global synic bit */
-	sctrl.as_uint64 = hv_get_non_nested_msr(HV_MSR_SCONTROL);
-	sctrl.enable = 0;
-	hv_set_non_nested_msr(HV_MSR_SCONTROL, sctrl.as_uint64);
+	/* When VMBus is active it owns SCONTROL - leave it. */
+	if (!vmbus_active) {
+		union hv_synic_scontrol sctrl;
+
+		sctrl.as_uint64 = hv_get_non_nested_msr(HV_MSR_SCONTROL);
+		sctrl.enable = 0;
+		hv_set_non_nested_msr(HV_MSR_SCONTROL, sctrl.as_uint64);
+	}
 
 	return 0;
 }
--

2.43.0

#5 lore raw

RE: [PATCH v4 1/3] mshv: limit SynIC management to MSHV-owned resources

Quoted 8 lines · Michael Kelley

From: Jork Loeser <[email protected]> Sent: Monday, April 27, 2026 2:39 PM

Quoted 6 lines

While here, fix the SIEFP and SIRBP memremap() and virt_to_phys() calls to use HV_HYP_PAGE_SHIFT/HV_HYP_PAGE_SIZE instead of PAGE_SHIFT/PAGE_SIZE. The hypervisor always uses 4K pages for SynIC register GPAs regardless of the kernel page size, so using PAGE_SHIFT produces wrong addresses on ARM64 with 64K pages.

Quoted 8 lines

I agree that this is a good change. But any kernel image built with CONFIG_MSHV_ROOT set must use only 4KiB pages, as enforced by the dependency in drivers/hv/Kconfig. The change makes the code explicitly match the SynIC register layout, which is good, but it doesn't actually fix a problem since root MSHV code can't run on ARM64 with 64KiB pages. My only concern is that this commit message should not imply that an ARM64/64KiB configuration is possible for the root.

Agree for root. For L1VH, I think it theoretically could (likely with other changes needing to happen elsewhere).

Best,
Jork

#6 lore raw

PATCH v4 2/3 mshv: clean up SynIC state on kexec for L1VH

In reply to Jork Loeser

The reboot notifier that tears down the SynIC cpuhp state guards the cleanup with hv_root_partition(), so on L1VH (where hv_root_partition() is false) SINT0, SINT5, and SIRBP are never cleaned up before kexec. The kexec'd kernel then inherits stale unmasked SINTs and an enabled SIRBP pointing to freed memory.

Remove the hv_root_partition() guard so the cleanup runs for all parent partitions.

---
 drivers/hv/mshv_synic.c | 3 ---
 1 file changed, 3 deletions(-)
drivers/hv/mshv_synic.c +0 -3
index 2db3b0192eac..978a1cace341 100644
@@ -723,9 +723,6 @@ mshv_unregister_doorbell(u64 partition_id, int doorbell_portid)
 static int mshv_synic_reboot_notify(struct notifier_block *nb,
 			      unsigned long code, void *unused)
 {
-	if (!hv_root_partition())
-		return 0;
-
 	cpuhp_remove_state(synic_cpuhp_online);
 	return 0;
 }
Signature · 1 line
2.43.0
Reviewed-by on v4 2/3
#7 lore raw

Re: [PATCH v4 2/3] mshv: clean up SynIC state on kexec for L1VH

Quoted 10 lines · Jork Loeser

The reboot notifier that tears down the SynIC cpuhp state guards the cleanup with hv_root_partition(), so on L1VH (where hv_root_partition() is false) SINT0, SINT5, and SIRBP are never cleaned up before kexec. The kexec'd kernel then inherits stale unmasked SINTs and an enabled SIRBP pointing to freed memory.

Remove the hv_root_partition() guard so the cleanup runs for all parent partitions.

Quoted 21 lines
---
 drivers/hv/mshv_synic.c | 3 ---
 1 file changed, 3 deletions(-)
drivers/hv/mshv_synic.c +0 -3
index 2db3b0192eac..978a1cace341 100644
@@ -723,9 +723,6 @@ mshv_unregister_doorbell(u64 partition_id, int doorbell_portid)
 static int mshv_synic_reboot_notify(struct notifier_block *nb,
 			      unsigned long code, void *unused)
 {
-	if (!hv_root_partition())
-		return 0;
-
 	cpuhp_remove_state(synic_cpuhp_online);
 	return 0;
 }
Signature · 1 line
2.43.0
Reviewed-by on v4 2/3
#8 lore raw

Re: [PATCH v4 2/3] mshv: clean up SynIC state on kexec for L1VH

In reply to Jork Loeser
Quoted 31 lines · Jork Loeser

The reboot notifier that tears down the SynIC cpuhp state guards the cleanup with hv_root_partition(), so on L1VH (where hv_root_partition() is false) SINT0, SINT5, and SIRBP are never cleaned up before kexec. The kexec'd kernel then inherits stale unmasked SINTs and an enabled SIRBP pointing to freed memory.

Remove the hv_root_partition() guard so the cleanup runs for all parent partitions.

---
 drivers/hv/mshv_synic.c | 3 ---
 1 file changed, 3 deletions(-)
drivers/hv/mshv_synic.c +0 -3
index 2db3b0192eac..978a1cace341 100644
@@ -723,9 +723,6 @@ mshv_unregister_doorbell(u64 partition_id, int doorbell_portid)
 static int mshv_synic_reboot_notify(struct notifier_block *nb,
 			      unsigned long code, void *unused)
 {
-	if (!hv_root_partition())
-		return 0;
-
 	cpuhp_remove_state(synic_cpuhp_online);
 	return 0;
 }
Signature · 1 line
2.43.0
#9 lore raw

PATCH v4 3/3 mshv: unmap debugfs stats pages on kexec

In reply to Jork Loeser

On L1VH, debugfs stats pages are overlay pages: the kernel allocates them and registers the GPAs with the hypervisor via HVCALL_MAP_STATS_PAGE2. These overlay mappings persist in the hypervisor across kexec. If the kexec'd kernel reuses those physical pages, the hypervisor's overlay semantics cause a machine check exception.

Fix this by calling mshv_debugfs_exit() from the reboot notifier, which issues HVCALL_UNMAP_STATS_PAGE for each mapped stats page before kexec. This releases the overlay bindings so the physical pages can be safely reused. Guard mshv_debugfs_exit() against being called when init failed.

---
 drivers/hv/mshv_debugfs.c | 7 ++++++-
 drivers/hv/mshv_synic.c   | 1 +
 2 files changed, 7 insertions(+), 1 deletion(-)
drivers/hv/mshv_debugfs.c +6 -1
index 418b6dc8f3c2..3c3e02237ae9 100644
@@ -674,8 +674,10 @@ int __init mshv_debugfs_init(void)
 
 	mshv_debugfs = debugfs_create_dir("mshv", NULL);
 	if (IS_ERR(mshv_debugfs)) {
+		err = PTR_ERR(mshv_debugfs);
+		mshv_debugfs = NULL;
 		pr_err("%s: failed to create debugfs directory\n", __func__);
-		return PTR_ERR(mshv_debugfs);
+		return err;
 	}
 
 	if (hv_root_partition()) {
@@ -710,6 +712,9 @@ int __init mshv_debugfs_init(void)
 
 void mshv_debugfs_exit(void)
 {
+	if (!mshv_debugfs)
+		return;
+
 	mshv_debugfs_parent_partition_remove();
 
 	if (hv_root_partition()) {
drivers/hv/mshv_synic.c +1 -0
index 978a1cace341..88170ce6b83f 100644
@@ -723,6 +723,7 @@ mshv_unregister_doorbell(u64 partition_id, int doorbell_portid)
 static int mshv_synic_reboot_notify(struct notifier_block *nb,
 			      unsigned long code, void *unused)
 {
+	mshv_debugfs_exit();
 	cpuhp_remove_state(synic_cpuhp_online);
 	return 0;
 }
Signature · 1 line
2.43.0
Reviewed-by on v4 3/3
#10 lore raw

Re: [PATCH v4 3/3] mshv: unmap debugfs stats pages on kexec

Quoted 7 lines · Jork Loeser

On L1VH, debugfs stats pages are overlay pages: the kernel allocates them and registers the GPAs with the hypervisor via HVCALL_MAP_STATS_PAGE2. These overlay mappings persist in the hypervisor across kexec. If the kexec'd kernel reuses those physical pages, the hypervisor's overlay semantics cause a machine check exception.

Quoted 7 lines

Fix this by calling mshv_debugfs_exit() from the reboot notifier, which issues HVCALL_UNMAP_STATS_PAGE for each mapped stats page before kexec. This releases the overlay bindings so the physical pages can be safely reused. Guard mshv_debugfs_exit() against being called when init failed.

Quoted 46 lines
---
 drivers/hv/mshv_debugfs.c | 7 ++++++-
 drivers/hv/mshv_synic.c   | 1 +
 2 files changed, 7 insertions(+), 1 deletion(-)
drivers/hv/mshv_debugfs.c +6 -1
index 418b6dc8f3c2..3c3e02237ae9 100644
@@ -674,8 +674,10 @@ int __init mshv_debugfs_init(void)
 
 	mshv_debugfs = debugfs_create_dir("mshv", NULL);
 	if (IS_ERR(mshv_debugfs)) {
+		err = PTR_ERR(mshv_debugfs);
+		mshv_debugfs = NULL;
 		pr_err("%s: failed to create debugfs directory\n", __func__);
-		return PTR_ERR(mshv_debugfs);
+		return err;
 	}
 
 	if (hv_root_partition()) {
@@ -710,6 +712,9 @@ int __init mshv_debugfs_init(void)
 
 void mshv_debugfs_exit(void)
 {
+	if (!mshv_debugfs)
+		return;
+
 	mshv_debugfs_parent_partition_remove();
 
 	if (hv_root_partition()) {
drivers/hv/mshv_synic.c +1 -0
index 978a1cace341..88170ce6b83f 100644
@@ -723,6 +723,7 @@ mshv_unregister_doorbell(u64 partition_id, int doorbell_portid)
 static int mshv_synic_reboot_notify(struct notifier_block *nb,
 			      unsigned long code, void *unused)
 {
+	mshv_debugfs_exit();
 	cpuhp_remove_state(synic_cpuhp_online);
 	return 0;
 }
Signature · 1 line
2.43.0
Reviewed-by on v4 3/3
#11 lore raw

Re: [PATCH v4 3/3] mshv: unmap debugfs stats pages on kexec

In reply to Jork Loeser
Quoted 24 lines · Jork Loeser

On L1VH, debugfs stats pages are overlay pages: the kernel allocates them and registers the GPAs with the hypervisor via HVCALL_MAP_STATS_PAGE2. These overlay mappings persist in the hypervisor across kexec. If the kexec'd kernel reuses those physical pages, the hypervisor's overlay semantics cause a machine check exception.

Fix this by calling mshv_debugfs_exit() from the reboot notifier, which issues HVCALL_UNMAP_STATS_PAGE for each mapped stats page before kexec. This releases the overlay bindings so the physical pages can be safely reused. Guard mshv_debugfs_exit() against being called when init failed.

---
 drivers/hv/mshv_debugfs.c | 7 ++++++-
 drivers/hv/mshv_synic.c   | 1 +
 2 files changed, 7 insertions(+), 1 deletion(-)
drivers/hv/mshv_debugfs.c +0 -0
index 418b6dc8f3c2..3c3e02237ae9 100644

drivers/hv/mshv_debugfs.c int __init mshv_debugfs_init(void) · line 679

Quoted 6 lines
drivers/hv/mshv_debugfs.c +2 -0
@@ -674,8 +674,10 @@ int __init mshv_debugfs_init(void)
 
 	mshv_debugfs = debugfs_create_dir("mshv", NULL);
 	if (IS_ERR(mshv_debugfs)) {
+		err = PTR_ERR(mshv_debugfs);
+		mshv_debugfs = NULL;
 		pr_err("%s: failed to create debugfs directory\n", __func__);

Might as well print err here.

Nevertheless:

Thanks,
Anirudh.

maintainer Applied
#12 lore raw
In reply to Jork Loeser
Quoted 34 lines · Jork Loeser

This series fixes kexec support when Linux runs as an L1 Virtual Host (L1VH) under Hyper-V, using the MSHV driver to manage child VMs.

1-2. SynIC cleanup: the MSHV driver manages its own SynIC resources
separately from vmbus. Add proper teardown of MSHV-owned SINTs and SIRBP on kexec, scoped to only the resources MSHV owns. Use hv_vmbus_exists() to decide at runtime whether VMBus owns SIMP/SIEFP/SCONTROL (so MSHV must not touch them) or whether MSHV must manage them itself (bare root partition without VMBus). Also fix SIEFP and SIRBP address calculations to use HV_HYP_PAGE_SHIFT instead of PAGE_SHIFT, which produces wrong addresses on ARM64 with 64K pages.

3. Debugfs stats pages: unmap the VP statistics overlay pages before
kexec to avoid machine check exceptions when the new kernel reuses those physical pages.

Changes since v3:
- Dropped patches 1-3 (vmbus variable shadowing, stimer cleanup, LP/VP skip), now merged via hyperv-next.
- Patch 1: fix SIEFP and SIRBP memremap()/virt_to_phys() to use HV_HYP_PAGE_SHIFT/HV_HYP_PAGE_SIZE instead of PAGE_SHIFT/PAGE_SIZE.

Changes since v2:
- Rebased onto linux-next/master to adapt to the upstream SynIC refactor (commit 5a674ef871fe, "mshv: refactor synic init and cleanup").

Changes since v1:
- Patch 1: account for nested root partitions where VMBus is also active (not just L1VH); use a vmbus_active local variable; allocate SIRBP in L1VH allocation path for when the hypervisor doesn't pre-provision the page.

Quoted 4 lines

Jork Loeser (3):
mshv: limit SynIC management to MSHV-owned resources
mshv: clean up SynIC state on kexec for L1VH
mshv: unmap debugfs stats pages on kexec

Applied to hyperv-fixes. Thanks.