Skip to contentSkip to conversation

x86/hyperv: Suspend/resume the hypercall page for hibernation

PATCH RESEND Applied Dexuan Cui 4 participants 6 messages Jan 6, 2020 – Feb 1, 2020

Subsystems (4)
lore
Share & tools
lore

Dexuan Cui

author maintainer
#0 lore raw

PATCH RESEND v6 1/1 x86/hyperv: Suspend/resume the hypercall page for hibernation

This is needed for hibernation, e.g. when we resume the old kernel, we need to disable the "current" kernel's hypercall page and then resume the old kernel's.

---

This is a RESEND of https://lkml.org/lkml/2019/11/20/68 .

Please review.

If it looks good, can you please pick it up through the tip.git tree?

 arch/x86/hyperv/hv_init.c | 48 +++++++++++++++++++++++++++++++++++++++
 1 file changed, 48 insertions(+)
arch/x86/hyperv/hv_init.c +48 -0
index caaf4dce99bf..24a62d33067c 100644
@@ -21,11 +21,15 @@
 #include <linux/hyperv.h>
 #include <linux/slab.h>
 #include <linux/cpuhotplug.h>
+#include <linux/syscore_ops.h>
 #include <clocksource/hyperv_timer.h>
 
 void *hv_hypercall_pg;
 EXPORT_SYMBOL_GPL(hv_hypercall_pg);
 
+/* Save the hypercall page temporarily for hibernation */
+static void *hv_hypercall_pg_saved;
+
 u32 *hv_vp_index;
 EXPORT_SYMBOL_GPL(hv_vp_index);
 
@@ -246,6 +250,46 @@ static int __init hv_pci_init(void)
 	return 1;
 }
 
+static int hv_suspend(void)
+{
+	union hv_x64_msr_hypercall_contents hypercall_msr;
+
+	/*
+	 * Reset hypercall page reference before reset the page,
+	 * let hypercall operations fail safely rather than
+	 * panic the kernel for using invalid hypercall page
+	 */
+	hv_hypercall_pg_saved = hv_hypercall_pg;
+	hv_hypercall_pg = NULL;
+
+	/* Reset the hypercall page */
+	rdmsrl(HV_X64_MSR_HYPERCALL, hypercall_msr.as_uint64);
+	hypercall_msr.enable = 0;
+	wrmsrl(HV_X64_MSR_HYPERCALL, hypercall_msr.as_uint64);
+
+	return 0;
+}
+
+static void hv_resume(void)
+{
+	union hv_x64_msr_hypercall_contents hypercall_msr;
+
+	/* Re-enable the hypercall page */
+	rdmsrl(HV_X64_MSR_HYPERCALL, hypercall_msr.as_uint64);
+	hypercall_msr.enable = 1;
+	hypercall_msr.guest_physical_address =
+		vmalloc_to_pfn(hv_hypercall_pg_saved);
+	wrmsrl(HV_X64_MSR_HYPERCALL, hypercall_msr.as_uint64);
+
+	hv_hypercall_pg = hv_hypercall_pg_saved;
+	hv_hypercall_pg_saved = NULL;
+}
+
+static struct syscore_ops hv_syscore_ops = {
+	.suspend = hv_suspend,
+	.resume = hv_resume,
+};
+
 /*
  * This function is to be invoked early in the boot sequence after the
  * hypervisor has been detected.
@@ -330,6 +374,8 @@ void __init hyperv_init(void)
 
 	x86_init.pci.arch_init = hv_pci_init;
 
+	register_syscore_ops(&hv_syscore_ops);
+
 	return;
 
 remove_cpuhp_state:
@@ -349,6 +395,8 @@ void hyperv_cleanup(void)
 {
 	union hv_x64_msr_hypercall_contents hypercall_msr;
 
+	unregister_syscore_ops(&hv_syscore_ops);
+
 	/* Reset our OS id */
 	wrmsrl(HV_X64_MSR_GUEST_OS_ID, 0);
 
Signature · 1 line
2.19.1
author maintainer
#1 lore raw
Quoted 13 lines

From: Dexuan Cui <[email protected]>
Sent: Monday, January 6, 2020 2:43 PM

This is needed for hibernation, e.g. when we resume the old kernel, we need to disable the "current" kernel's hypercall page and then resume the old kernel's.

---

This is a RESEND of https://lkml.org/lkml/2019/11/20/68

Quoted 7 lines

Please review.

If it looks good, can you please pick it up through the tip.git tree?

 arch/x86/hyperv/hv_init.c | 48
+++++++++++++++++++++++++++++++++++++++
 1 file changed, 48 insertions(+)

Hi, Vitaly and x86 maintainers,
Can you please take a look at this patch?

Thanks,
-- Dexuan

#2 lore raw
Quoted 15 lines · Dexuan Cui
Quoted 15 lines

From: Dexuan Cui <[email protected]>
Sent: Monday, January 6, 2020 2:43 PM

This is needed for hibernation, e.g. when we resume the old kernel, we need to disable the "current" kernel's hypercall page and then resume the old kernel's.

---

This is a RESEND of https://lkml.org/lkml/2019/11/20/68

Please review.

Quoted 8 lines
Quoted 5 lines

If it looks good, can you please pick it up through the tip.git tree?

 arch/x86/hyperv/hv_init.c | 48
+++++++++++++++++++++++++++++++++++++++
 1 file changed, 48 insertions(+)

Hi, Vitaly and x86 maintainers,
Can you please take a look at this patch?

Ping?

This patch has been floating around in it's current form for the past 2 months. I'll happily take Hyper-V patches under arch/x86/hyperv/ via the Hyper-V tree rather than tip if the x86 folks don't want to deal with them.

Signature · 2 lines
Thanks,
Sasha
author maintainer
#3 lore raw
Quoted 43 lines

From: [email protected] <[email protected]> On Behalf Of Sasha Levin
Sent: Wednesday, January 29, 2020 4:06 PM
To: Dexuan Cui <[email protected]>
Cc: [email protected]; [email protected]; [email protected]; Haiyang Zhang <[email protected]>; [email protected]; KY Srinivasan <[email protected]>; [email protected]; [email protected]; [email protected]; Stephen Hemminger <[email protected]>; [email protected]; [email protected]; Michael Kelley <[email protected]>; Sasha Levin <[email protected]>; vkuznets <[email protected]>; [email protected]
Subject: Re: [PATCH v6][RESEND] x86/hyperv: Suspend/resume the hypercall page for hibernation

Quoted 4 lines · Dexuan Cui
Quoted 4 lines

From: Dexuan Cui <[email protected]>
Sent: Monday, January 6, 2020 2:43 PM

This is needed for hibernation, e.g. when we resume the old kernel, we

need

Quoted 11 lines
Quoted 11 lines

to disable the "current" kernel's hypercall page and then resume the old kernel's.

---

This is a RESEND of https://lkml.org/lkml/2019/11/20/68

Please review.

Quoted 8 lines
Quoted 5 lines

If it looks good, can you please pick it up through the tip.git tree?

 arch/x86/hyperv/hv_init.c | 48
+++++++++++++++++++++++++++++++++++++++
 1 file changed, 48 insertions(+)

Hi, Vitaly and x86 maintainers,
Can you please take a look at this patch?

Ping?

Quoted 8 lines

This patch has been floating around in it's current form for the past 2 months. I'll happily take Hyper-V patches under arch/x86/hyperv/ via the Hyper-V tree rather than tip if the x86 folks don't want to deal with them.

--
Thanks,
Sasha

This straightforward patch is the only pending patch for v5.6 for the hibernation functionality of Linux VM on Hyper-V. It would be really great if we could merge it for v5.6-rc1. The patch is safe in that it only runs when the VM hibernates, and the hibernation functionality of Linux VM on Hyper-V never worked before. I'm pretty sure the patch can not cause any merge conflict since nobody else tries to modify the file recently.

Thanks,
-- Dexuan

#4 lore raw
In reply to Sasha Levin
Quoted 14 lines · Sasha Levin
Quoted 4 lines · Dexuan Cui
Quoted 4 lines

This is a RESEND of https://lkml.org/lkml/2019/11/20/68

Please review.

Quoted 8 lines
Quoted 5 lines

If it looks good, can you please pick it up through the tip.git tree?

 arch/x86/hyperv/hv_init.c | 48
+++++++++++++++++++++++++++++++++++++++
 1 file changed, 48 insertions(+)

Hi, Vitaly and x86 maintainers,
Can you please take a look at this patch?

Quoted 6 lines

Ping?

This patch has been floating around in it's current form for the past 2 months. I'll happily take Hyper-V patches under arch/x86/hyperv/ via the Hyper-V tree rather than tip if the x86 folks don't want to deal with them.

sorry fell through the cracks. Will take care of it.

merge bot Applied
#5 lore raw
In reply to Dexuan Cui
The following commit has been merged into the x86/urgent branch of tip:

The following commit has been merged into the x86/urgent branch of tip:

Commit-ID:     05bd330a7fd8875c423fc07d8ddcad73c10e556e
Gitweb:        https://git.kernel.org/tip/05bd330a7fd8875c423fc07d8ddcad73c10e556e
Author:        Dexuan Cui <[email protected]>
AuthorDate:    Mon, 06 Jan 2020 14:42:39 -08:00
Committer:     Thomas Gleixner <[email protected]>
CommitterDate: Sat, 01 Feb 2020 09:41:16 +01:00

x86/hyperv: Suspend/resume the hypercall page for hibernation

For hibernation the hypercall page must be disabled before the hibernation image is created so that subsequent hypercall operations fail safely. On resume the hypercall page has to be restored and reenabled to ensure proper operation of the resumed kernel.

Implement the necessary suspend/resume callbacks.

[ tglx: Decrypted changelog ]

---
 arch/x86/hyperv/hv_init.c | 50 ++++++++++++++++++++++++++++++++++++++-
 1 file changed, 50 insertions(+)
arch/x86/hyperv/hv_init.c +50 -0
index caaf4dc..b0da532 100644
@@ -21,11 +21,15 @@
 #include <linux/hyperv.h>
 #include <linux/slab.h>
 #include <linux/cpuhotplug.h>
+#include <linux/syscore_ops.h>
 #include <clocksource/hyperv_timer.h>
 
 void *hv_hypercall_pg;
 EXPORT_SYMBOL_GPL(hv_hypercall_pg);
 
+/* Storage to save the hypercall page temporarily for hibernation */
+static void *hv_hypercall_pg_saved;
+
 u32 *hv_vp_index;
 EXPORT_SYMBOL_GPL(hv_vp_index);
 
@@ -246,6 +250,48 @@ static int __init hv_pci_init(void)
 	return 1;
 }
 
+static int hv_suspend(void)
+{
+	union hv_x64_msr_hypercall_contents hypercall_msr;
+
+	/*
+	 * Reset the hypercall page as it is going to be invalidated
+	 * accross hibernation. Setting hv_hypercall_pg to NULL ensures
+	 * that any subsequent hypercall operation fails safely instead of
+	 * crashing due to an access of an invalid page. The hypercall page
+	 * pointer is restored on resume.
+	 */
+	hv_hypercall_pg_saved = hv_hypercall_pg;
+	hv_hypercall_pg = NULL;
+
+	/* Disable the hypercall page in the hypervisor */
+	rdmsrl(HV_X64_MSR_HYPERCALL, hypercall_msr.as_uint64);
+	hypercall_msr.enable = 0;
+	wrmsrl(HV_X64_MSR_HYPERCALL, hypercall_msr.as_uint64);
+
+	return 0;
+}
+
+static void hv_resume(void)
+{
+	union hv_x64_msr_hypercall_contents hypercall_msr;
+
+	/* Re-enable the hypercall page */
+	rdmsrl(HV_X64_MSR_HYPERCALL, hypercall_msr.as_uint64);
+	hypercall_msr.enable = 1;
+	hypercall_msr.guest_physical_address =
+		vmalloc_to_pfn(hv_hypercall_pg_saved);
+	wrmsrl(HV_X64_MSR_HYPERCALL, hypercall_msr.as_uint64);
+
+	hv_hypercall_pg = hv_hypercall_pg_saved;
+	hv_hypercall_pg_saved = NULL;
+}
+
+static struct syscore_ops hv_syscore_ops = {
+	.suspend	= hv_suspend,
+	.resume		= hv_resume,
+};
+
 /*
  * This function is to be invoked early in the boot sequence after the
  * hypervisor has been detected.
@@ -330,6 +376,8 @@ void __init hyperv_init(void)
 
 	x86_init.pci.arch_init = hv_pci_init;
 
+	register_syscore_ops(&hv_syscore_ops);
+
 	return;
 
 remove_cpuhp_state:
@@ -349,6 +397,8 @@ void hyperv_cleanup(void)
 {
 	union hv_x64_msr_hypercall_contents hypercall_msr;
 
+	unregister_syscore_ops(&hv_syscore_ops);
+
 	/* Reset our OS id */
 	wrmsrl(HV_X64_MSR_GUEST_OS_ID, 0);