Skip to contentSkip to conversation

Enable Linux guests on Hyper-V on ARM64

PATCH Superseded Michael Kelley 5 participants 25 messages Aug 24, 2020 – Aug 26, 2020

Subsystems (7)

v7 of 12 Superseded Latest is v12 →

lore
Share & tools
lore
#0 lore raw

v7 cover Enable Linux guests on Hyper-V on ARM64

This series enables Linux guests running on Hyper-V on ARM64 hardware. New ARM64-specific code in arch/arm64/hyperv initializes Hyper-V, including its interrupts and hypercall mechanism. Existing architecture independent drivers for Hyper-V's VMbus and synthetic devices just work when built for ARM64. Hyper-V code is built and included in the image and modules only if CONFIG_HYPERV is enabled.

The ten patches are organized as follows:
1) Add #define for vendor specific owner definition to linux/arm-smccc.h

2) Add include files that define the Hyper-V interface as described in the Hyper-V Top Level Functional Spec (TLFS), plus additional definitions specific to Linux running on Hyper-V.

3) thru 7) Add core Hyper-V support on ARM64, including hypercalls, interrupt handlers, kexec & panic handlers, and core hypervisor initialization.

8) Update the existing VMbus driver to generalize interrupt management across x86/x64 and ARM64.

9) Export screen_info so it may be used by the Hyper-V frame buffer driver built as a module. It is already exported for x86, powerpc, and alpha architectures.

10) Make CONFIG_HYPERV selectable on ARM64 in addition to x86/x64.

Some areas of Linux guests on Hyper-V on ARM64 are a work- in-progress:

* Hyper-V on ARM64 currently runs with a 4 Kbyte page size, but allows guests with 16K/64K page size. However, the Linux drivers for Hyper-V synthetic devices assume the guest page size is 4K. This patch set lays the groundwork for larger guest page sizes, but additional patches are coming to update these drivers.

* The Hyper-V vPCI driver at drivers/pci/host/pci-hyperv.c has x86/x64-specific code and is not being built for ARM64. Fixing this driver to enable vPCI devices on ARM64 will be done later.

In a few cases, terminology from the x86/x64 world has been carried over into the ARM64 code ("MSR", "TSC"). Hyper-V still uses the x86/x64 terminology and has not replaced it with something more generic, so the code uses the Hyper-V terminology. This will be fixed when Hyper-V updates the usage in the TLFS.

This patch set is based on the linux-next-20200817 tree (5.9-rc1).

Changes in v7:
* Separately upstreamed split of hyperv-tlfs.h into arch dependent and independent sections. In this patch set, update the ARM64 hyperv-tlfs.h to include architecture independent definitions. This approach eliminates a lot of lines of otherwise duplicated code on the ARM64 side.
* Break ARM64 mshyperv.h into smaller pieces. Have an initial baseline, and add code along with patches for a particular functional area. [Marc Zyngier]
* In mshyperv.h, use static inline functions instead of #defines where possible. [Arnd Bergmann]
* Use VMbus INTID obtained from ACPI DSDT instead of hardcoding. The STIMER INTID is still hardcoded because it is needed before Linux has initialized the ACPI subsystem, so it can't be obtained from the DSDT. Wedging it into the GTDT seems dubious, so was not done. [Marc Zyngier]
* Update Hyper-V page size allocation functions to use alloc_page() if PAGE_SIZE == HV_HYP_PAGE_SIZE [Arnd Bergmann]
* Various other minor changes based on feedback and to rebase to latest linux-next [Marc Zyngier and Arnd Bergmann]

Changes in v6:
* Use SMCCC hypercall interface instead of direct invocation of HVC instruction and the Hyper-V hypercall interface [Marc Zyngier]
* Reimplemented functions to alloc/free Hyper-V size pages using kmalloc/kfree since kmalloc now guarantees alignment of power of 2 size allocations [Marc Zyngier]
* Export screen_info in arm64 architecture so it can be used by the Hyper-V buffer driver built as a module
* Renamed source file arch/arm64/hyperv/hv_init.c to hv_core.c to better reflect its content
* Fixed the bit position of certain feature flags presented by Hyper-V to the guest. The bit positions on ARM64 don't match the position on x86 like originally thought.
* Minor fixups to rebase to 5.6-rc5 linux-next

Changes in v5:
* Minor fixups to rebase to 5.4-rc1 linux-next

Changes in v4:
* Moved clock-related code into an architecture independent Hyper-V clocksource driver that is already upstream. Clock related code is removed from this patch set except for the ARM64 specific interrupt handler. [Marc Zyngier]
* Separately upstreamed the split of mshyperv.h into arch independent and arch dependent portions. The arch independent portion has been removed from this patch set.
* Divided patch #2 of the series into multiple smaller patches [Marc Zyngier]
* Changed a dozen or so smaller things based on feedback [Marc Zyngier, Will Deacon]
* Added functions to alloc/free Hyper-V size pages for use by drivers for Hyper-V synthetic devices when updated to not assume guest page size and Hyper-v page size are the same

Changes in v3:
* Added initialization of hv_vp_index array like was recently added on x86 branch [KY Srinivasan]
* Changed Hyper-V ARM64 register symbols to be all uppercase instead of mixed case [KY Srinivasan]
* Separated mshyperv.h into two files, one architecture independent and one architecture dependent. After this code is upstream, will make changes to the x86 code to use the architecture independent file and remove duplication. And once we have a multi-architecture Hyper-V TLFS, will do a separate patch to split hyperv-tlfs.h in the same way. [KY Srinivasan]
* Minor tweaks to rebase to latest linux-next code

Changes in v2:
* Removed patch to implement slow_virt_to_phys() on ARM64. Use of slow_virt_to_phys() in arch independent Hyper-V drivers has been eliminated by commit 6ba34171bcbd ("Drivers: hv: vmbus: Remove use of slow_virt_to_phys()")
* Minor tweaks to rebase to latest linux-next code

---

Michael Kelley (10):
arm/arm64: smccc-1.1: Add vendor specific owner definition
arm64: hyperv: Add core Hyper-V include files
arm64: hyperv: Add hypercall and register access functions
arm64: hyperv: Add memory alloc/free functions for Hyper-V size pages
arm64: hyperv: Add interrupt handlers for VMbus and stimer
arm64: hyperv: Add kexec and panic handlers
arm64: hyperv: Initialize hypervisor on boot
Drivers: hv: vmbus: Add hooks for per-CPU IRQ
arm64: efi: Export screen_info
Drivers: hv: Enable Hyper-V code to be built on ARM64

 MAINTAINERS                          |   3 +
 arch/arm64/Kbuild                    |   1 +
 arch/arm64/hyperv/Makefile           |   2 +
 arch/arm64/hyperv/hv_core.c          | 437 +++++++++++++++++++++++++++++++++++
 arch/arm64/hyperv/mshyperv.c         | 159 +++++++++++++
 arch/arm64/include/asm/hyperv-tlfs.h |  94 ++++++++
 arch/arm64/include/asm/mshyperv.h    | 165 +++++++++++++
 arch/arm64/kernel/efi.c              |   1 +
 arch/x86/include/asm/mshyperv.h      |   4 +
 drivers/hv/Kconfig                   |   3 +-
 drivers/hv/hv.c                      |   3 +
 include/asm-generic/mshyperv.h       |   5 +
 include/linux/arm-smccc.h            |   1 +
 13 files changed, 877 insertions(+), 1 deletion(-)
 create mode 100644 arch/arm64/hyperv/Makefile
 create mode 100644 arch/arm64/hyperv/hv_core.c
 create mode 100644 arch/arm64/hyperv/mshyperv.c
 create mode 100644 arch/arm64/include/asm/hyperv-tlfs.h
 create mode 100644 arch/arm64/include/asm/mshyperv.h
Signature · 7 lines
1.8.3.1


_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
#1 lore raw

PATCH v7 1/10 arm/arm64: smccc-1.1: Add vendor specific owner definition

In the SMCCC v1.1 spec, the Owning Entity Number in the Function Identifier may have a value of 6 for a vendor specific hypervisor service call. Add this #define.

---
 include/linux/arm-smccc.h | 1 +
 1 file changed, 1 insertion(+)
include/linux/arm-smccc.h +1 -0
index 15c706f..ee286f5 100644
@@ -49,6 +49,7 @@
 #define ARM_SMCCC_OWNER_OEM		3
 #define ARM_SMCCC_OWNER_STANDARD	4
 #define ARM_SMCCC_OWNER_STANDARD_HYP	5
+#define ARM_SMCCC_OWNER_VENDOR_HYP	6
 #define ARM_SMCCC_OWNER_TRUSTED_APP	48
 #define ARM_SMCCC_OWNER_TRUSTED_APP_END	49
 #define ARM_SMCCC_OWNER_TRUSTED_OS	50
Signature · 1 line
1.8.3.1
#2 lore raw

PATCH v7 2/10 arm64: hyperv: Add core Hyper-V include files

In reply to Michael Kelley

hyperv-tlfs.h defines Hyper-V interfaces from the Hyper-V Top Level Functional Spec (TLFS), and #includes the architecture-independent part of hyperv-tlfs.h in include/asm-generic. The published TLFS is distinctly oriented to x86/x64, so the ARM64-specific hyperv-tlfs.h includes information for ARM64 that is not yet formally published. The TLFS is available here:

docs.microsoft.com/en-us/virtualization/hyper-v-on-windows/reference/tlfs

mshyperv.h defines Linux-specific structures and routines for interacting with Hyper-V on ARM64, and #includes the architecture- independent part of mshyperv.h in include/asm-generic.

---
 MAINTAINERS                          |  2 +
 arch/arm64/include/asm/hyperv-tlfs.h | 94 ++++++++++++++++++++++++++++++++++++
 arch/arm64/include/asm/mshyperv.h    | 76 +++++++++++++++++++++++++++++
 3 files changed, 172 insertions(+)
 create mode 100644 arch/arm64/include/asm/hyperv-tlfs.h
 create mode 100644 arch/arm64/include/asm/mshyperv.h
MAINTAINERS +2 -0
index 4179dfa..803bade 100644
@@ -8054,6 +8054,8 @@ T:	git git://git.kernel.org/pub/scm/linux/kernel/git/hyperv/linux.git
 F:	Documentation/ABI/stable/sysfs-bus-vmbus
 F:	Documentation/ABI/testing/debugfs-hyperv
 F:	Documentation/networking/device_drivers/ethernet/microsoft/netvsc.rst
+F:	arch/arm64/include/asm/hyperv-tlfs.h
+F:	arch/arm64/include/asm/mshyperv.h
 F:	arch/x86/hyperv
 F:	arch/x86/include/asm/hyperv-tlfs.h
 F:	arch/x86/include/asm/mshyperv.h
arch/arm64/include/asm/hyperv-tlfs.h new file +94 -0

94 lines not on this page · raw message

arch/arm64/include/asm/mshyperv.h new file +76 -0

76 lines not on this page · raw message

Signature · 7 lines
1.8.3.1


_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
maintainer
#3 lore raw

Re: [PATCH v7 02/10] arm64: hyperv: Add core Hyper-V include files

arch/arm64/include/asm/mshyperv.h line 42

Quoted 4 lines · Michael Kelley
arch/arm64/include/asm/mshyperv.h new file +4 -0
+
+#define hv_get_simp(val) (val = hv_get_vpreg(HV_REGISTER_SIPP))
+
+#define hv_get_siefp(val) (val = hv_get_vpreg(HV_REGISTER_SIFP))

Macros that modify their arguments are generally a bad idea. Since each one of these only has a few callers, could you just redefine the x86 version to use function-style calling conventions and turn them into inline functions?

Arnd

#4 lore raw

PATCH v7 3/10 arm64: hyperv: Add hypercall and register access functions

In reply to Michael Kelley

Add ARM64-specific code to make Hyper-V hypercalls and to access virtual processor synthetic registers via hypercalls. Hypercalls follow the SMC Calling Convention spec v1.1.

This code is architecture dependent and is mostly driven by architecture independent code in the VMbus driver and the Hyper-V timer clocksource driver.

This code is built only when CONFIG_HYPERV is enabled.

---
 MAINTAINERS                       |   1 +
 arch/arm64/Kbuild                 |   1 +
 arch/arm64/hyperv/Makefile        |   2 +
 arch/arm64/hyperv/hv_core.c       | 170 ++++++++++++++++++++++++++++++++++++++
 arch/arm64/include/asm/mshyperv.h |  17 ++++
 5 files changed, 191 insertions(+)
 create mode 100644 arch/arm64/hyperv/Makefile
 create mode 100644 arch/arm64/hyperv/hv_core.c
MAINTAINERS +1 -0
index 803bade..f67d208 100644
@@ -8054,6 +8054,7 @@ T:	git git://git.kernel.org/pub/scm/linux/kernel/git/hyperv/linux.git
 F:	Documentation/ABI/stable/sysfs-bus-vmbus
 F:	Documentation/ABI/testing/debugfs-hyperv
 F:	Documentation/networking/device_drivers/ethernet/microsoft/netvsc.rst
+F:	arch/arm64/hyperv
 F:	arch/arm64/include/asm/hyperv-tlfs.h
 F:	arch/arm64/include/asm/mshyperv.h
 F:	arch/x86/hyperv
arch/arm64/Kbuild +1 -0
index d646582..7a37608 100644
@@ -3,4 +3,5 @@ obj-y			+= kernel/ mm/
 obj-$(CONFIG_NET)	+= net/
 obj-$(CONFIG_KVM)	+= kvm/
 obj-$(CONFIG_XEN)	+= xen/
+obj-$(subst m,y,$(CONFIG_HYPERV))	+= hyperv/
 obj-$(CONFIG_CRYPTO)	+= crypto/
arch/arm64/hyperv/Makefile new file +2 -0
new file mode 100644
index 0000000..1697d30
@@ -0,0 +1,2 @@
+# SPDX-License-Identifier: GPL-2.0
+obj-y		:= hv_core.o
arch/arm64/hyperv/hv_core.c new file +170 -0

170 lines not on this page · raw message

arch/arm64/include/asm/mshyperv.h +17 -0
index 6b1f26c..b17d4a1 100644
@@ -26,6 +26,14 @@
 #include <linux/arm-smccc.h>
 #include <asm/hyperv-tlfs.h>
 
+/*
+ * Declare calls to get and set Hyper-V VP register values on ARM64, which
+ * requires a hypercall.
+ */
+extern void hv_set_vpreg(u32 reg, u64 value);
+extern u64 hv_get_vpreg(u32 reg);
+extern void hv_get_vpreg_128(u32 reg, struct hv_get_vp_registers_output *result);
+
 /* Access various Hyper-V synthetic registers */
 static inline void hv_set_simp(u64 val)
 {
@@ -71,6 +79,15 @@ static inline void hv_set_synint_state(u32 sint_num, u64 val)
 #define hv_get_synint_state(sint_num, val) \
 		(val = hv_get_vpreg(HV_REGISTER_SINT0 + sint_num))
 
+
+/* SMCCC hypercall parameters */
+#define HV_SMCCC_FUNC_NUMBER	1
+#define HV_FUNC_ID	ARM_SMCCC_CALL_VAL(			\
+				ARM_SMCCC_STD_CALL,		\
+				ARM_SMCCC_SMC_64,		\
+				ARM_SMCCC_OWNER_VENDOR_HYP,	\
+				HV_SMCCC_FUNC_NUMBER)
+
 #include <asm-generic/mshyperv.h>
 
 #endif
Signature · 1 line
1.8.3.1
#5 lore raw

PATCH v7 4/10 arm64: hyperv: Add memory alloc/free functions for Hyper-V size pages

In reply to Michael Kelley

Add ARM64-specific code to allocate memory with HV_HYP_PAGE_SIZE size and alignment. These are for use when pages need to be shared with Hyper-V. Separate functions are needed as the page size used by Hyper-V may not be the same as the guest page size.

This code is built only when CONFIG_HYPERV is enabled.

---
 arch/arm64/hyperv/hv_core.c    | 43 ++++++++++++++++++++++++++++++++++++++++++
 include/asm-generic/mshyperv.h |  5 +++++
 2 files changed, 48 insertions(+)
arch/arm64/hyperv/hv_core.c +43 -0
index 9b35011..14e41b7 100644
@@ -24,6 +24,49 @@
 
 
 /*
+ * Functions for allocating and freeing memory with size and
+ * alignment HV_HYP_PAGE_SIZE. These functions are needed because
+ * the guest page size may not be the same as the Hyper-V page
+ * size. We depend upon kmalloc() aligning power-of-two size
+ * allocations to the allocation size boundary, so that the
+ * allocated memory appears to Hyper-V as a page of the size
+ * it expects.
+ *
+ * These functions are used by arm64 specific code as well as
+ * arch independent Hyper-V drivers.
+ */
+
+void *hv_alloc_hyperv_page(void)
+{
+	BUILD_BUG_ON(PAGE_SIZE <  HV_HYP_PAGE_SIZE);
+
+	if (PAGE_SIZE == HV_HYP_PAGE_SIZE)
+		return (void *)__get_free_page(GFP_KERNEL);
+	else
+		return kmalloc(HV_HYP_PAGE_SIZE, GFP_KERNEL);
+}
+EXPORT_SYMBOL_GPL(hv_alloc_hyperv_page);
+
+void *hv_alloc_hyperv_zeroed_page(void)
+{
+	if (PAGE_SIZE == HV_HYP_PAGE_SIZE)
+		return (void *)__get_free_page(GFP_KERNEL | __GFP_ZERO);
+	else
+		return kzalloc(HV_HYP_PAGE_SIZE, GFP_KERNEL);
+}
+EXPORT_SYMBOL_GPL(hv_alloc_hyperv_zeroed_page);
+
+void hv_free_hyperv_page(unsigned long addr)
+{
+	if (PAGE_SIZE == HV_HYP_PAGE_SIZE)
+		free_page(addr);
+	else
+		kfree((void *)addr);
+}
+EXPORT_SYMBOL_GPL(hv_free_hyperv_page);
+
+
+/*
  * hv_do_hypercall- Invoke the specified hypercall
  */
 u64 hv_do_hypercall(u64 control, void *input, void *output)
include/asm-generic/mshyperv.h +5 -0
index c577996..d345a59 100644
@@ -101,6 +101,11 @@ static inline void vmbus_signal_eom(struct hv_message *msg, u32 old_msg_type)
 
 extern int vmbus_interrupt;
 
+void *hv_alloc_hyperv_page(void);
+void *hv_alloc_hyperv_zeroed_page(void);
+void hv_free_hyperv_page(unsigned long addr);
+
+
 #if IS_ENABLED(CONFIG_HYPERV)
 /*
  * Hypervisor's notion of virtual processor ID is different from
Signature · 1 line
1.8.3.1
#6 lore raw

PATCH v7 5/10 arm64: hyperv: Add interrupt handlers for VMbus and stimer

In reply to Michael Kelley

Add ARM64-specific code to set up and handle the interrupts generated by Hyper-V for VMbus messages and for stimer expiration.

This code is architecture dependent and is mostly driven by architecture independent code in the VMbus driver and the Hyper-V timer clocksource driver.

This code is built only when CONFIG_HYPERV is enabled.

---
 arch/arm64/hyperv/Makefile        |   2 +-
 arch/arm64/hyperv/mshyperv.c      | 133 ++++++++++++++++++++++++++++++++++++++
 arch/arm64/include/asm/mshyperv.h |  70 ++++++++++++++++++++
 3 files changed, 204 insertions(+), 1 deletion(-)
 create mode 100644 arch/arm64/hyperv/mshyperv.c
arch/arm64/hyperv/Makefile +1 -1
index 1697d30..87c31c0 100644
@@ -1,2 +1,2 @@
 # SPDX-License-Identifier: GPL-2.0
-obj-y		:= hv_core.o
+obj-y		:= hv_core.o mshyperv.o
arch/arm64/hyperv/mshyperv.c new file +133 -0

133 lines not on this page · raw message

arch/arm64/include/asm/mshyperv.h +70 -0

82 lines not on this page · raw message

Signature · 1 line
1.8.3.1
maintainer
#7 lore raw

Re: [PATCH v7 05/10] arm64: hyperv: Add interrupt handlers for VMbus and stimer

Quoted 8 lines · Michael Kelley

Add ARM64-specific code to set up and handle the interrupts generated by Hyper-V for VMbus messages and for stimer expiration.

This code is architecture dependent and is mostly driven by architecture independent code in the VMbus driver and the Hyper-V timer clocksource driver.

Quoted 7 lines

This code is built only when CONFIG_HYPERV is enabled.

---
 arch/arm64/hyperv/Makefile        |   2 +-
 arch/arm64/hyperv/mshyperv.c      | 133 ++++++++++++++++++++++++++++++++++++++
 arch/arm64/include/asm/mshyperv.h |  70 ++++++++++++++++++++

I still have the feeling that most of the code in arch/arm64/hyperv/ is
misplaced: the only callers are loadable modules in drivers/hv/, and the code is not really part of the architecture but part of the platform.

For the arm64 architecture, we have a rule that platform specific code belongs into device drivers rather than into the architecture code as we used to do in the linux-2.6 days for arch/arm/.

I don't see hyperv being virtual rather than an SoC as a differentiator either; it's still just one of many platforms. If you look at arch/arm64/xen/, you can see that they have managed to get to a much simpler implementation in comparison.

I'm not sure what the correct solution should be, but what I'd try to do here is to move every function that just considers the platform rather than the architecture somewhere into drivers/hv where it can be linked into the same modules as the existing files when building for arm64, while trying to keep architecture specific code in the header file where it can be included from those modules.

Arnd

#8 lore raw

RE: [PATCH v7 05/10] arm64: hyperv: Add interrupt handlers for VMbus and stimer

From: Arnd Bergmann <[email protected]> Sent: Monday, August 24, 2020 11:54 AM

Quoted 32 lines · Michael Kelley
Quoted 8 lines

Add ARM64-specific code to set up and handle the interrupts generated by Hyper-V for VMbus messages and for stimer expiration.

This code is architecture dependent and is mostly driven by architecture independent code in the VMbus driver and the Hyper-V timer clocksource driver.

Quoted 6 lines

This code is built only when CONFIG_HYPERV is enabled.

---
 arch/arm64/hyperv/Makefile        |   2 +-
 arch/arm64/hyperv/mshyperv.c      | 133

++++++++++++++++++++++++++++++++++++++

Quoted 1 line
 arch/arm64/include/asm/mshyperv.h |  70 ++++++++++++++++++++

I still have the feeling that most of the code in arch/arm64/hyperv/ is
misplaced: the only callers are loadable modules in drivers/hv/, and the code is not really part of the architecture but part of the platform.

For the arm64 architecture, we have a rule that platform specific code belongs into device drivers rather than into the architecture code as we used to do in the linux-2.6 days for arch/arm/.

I don't see hyperv being virtual rather than an SoC as a differentiator either; it's still just one of many platforms. If you look at arch/arm64/xen/, you can see that they have managed to get to a much simpler implementation in comparison.

Quoted 6 lines

I'm not sure what the correct solution should be, but what I'd try to do here is to move every function that just considers the platform rather than the architecture somewhere into drivers/hv where it can be linked into the same modules as the existing files when building for arm64, while trying to keep architecture specific code in the header file where it can be included from those modules.

OK. The concept of separating platform from architecture makes sense to me. The original separation of the Hyper-V code into architecture independent portions and x86-specific portions could use some tweaking now that we're dealing with n=2 architectures. With that tweaking, I can reduce the amount of Hyper-V code under arch/x86 and under arch/arm64.

On the flip side, the Hyper-V implementation on x86 and ARM64 has differences that are semi-related to the architecture. For example, on x86 Hyper-V uses synthetic MSRs for a lot of guest-hypervisor setup, while hypercalls are required on ARM64. So I'm assuming those differences will end up in code under arch/x86 and arch/arm64. Arguably, I could introduce a level of indirection (such as CONFIG_HYPERV_USE_MSRS vs. CONFIG_HYPERV_USE_HYPERCALLS) to distinguish the two behaviors. The selection would be tied to the architecture, and then code in drivers/hv can #ifdef the two cases. But I wonder if getting code out of arch/x86 and arch/arm64 is worth that additional messiness.

Looking at the Xen code in drivers/xen, it looks like a lot of the Xen functionality is implemented in hypercalls that can be consistent across architectures, though I was a bit surprised to see a dozen or so instances of #ifdef CONFIG_X86. Xen also #ifdefs on PV vs. PVHVM, which may handle some architecture differences implicitly. But I'm assuming that doing #ifdef <architecture> in the Hyper-V code in order to reduce code under arch/x86 or arch/arm64 is not the right way to go.

Michael

Quoted 2 lines

Arnd

maintainer
#9 lore raw

Re: [PATCH v7 05/10] arm64: hyperv: Add interrupt handlers for VMbus and stimer

Quoted 17 lines · Michael Kelley

From: Arnd Bergmann <[email protected]> Sent: Monday, August 24, 2020 11:54 AM

Quoted 7 lines

I'm not sure what the correct solution should be, but what I'd try to do here is to move every function that just considers the platform rather than the architecture somewhere into drivers/hv where it can be linked into the same modules as the existing files when building for arm64, while trying to keep architecture specific code in the header file where it can be included from those modules.

OK. The concept of separating platform from architecture makes sense to me. The original separation of the Hyper-V code into architecture independent portions and x86-specific portions could use some tweaking now that we're dealing with n=2 architectures. With that tweaking, I can reduce the amount of Hyper-V code under arch/x86 and under arch/arm64.

Quoted 5 lines

On the flip side, the Hyper-V implementation on x86 and ARM64 has differences that are semi-related to the architecture. For example, on x86 Hyper-V uses synthetic MSRs for a lot of guest-hypervisor setup, while hypercalls are required on ARM64. So I'm assuming those differences will end up in code under arch/x86 and arch/arm64.

Yes, that absolutely makes sense.

Quoted 6 lines

Arguably, I could introduce a level of indirection (such as CONFIG_HYPERV_USE_MSRS vs.
CONFIG_HYPERV_USE_HYPERCALLS) to distinguish the two behaviors. The selection would be tied to the architecture, and then code in drivers/hv can #ifdef the two cases. But I wonder if getting code out of arch/x86 and arch/arm64 is worth that additional messiness.

No, I think that would take it a little too far, and conflicts with the generic rule that code under drivers/* should be written to be portable even if can only run on a particular target platform.

Quoted 7 lines

Looking at the Xen code in drivers/xen, it looks like a lot of the Xen functionality is implemented in hypercalls that can be consistent across architectures, though I was a bit surprised to see a dozen or so instances of #ifdef CONFIG_X86. Xen also #ifdefs on PV vs. PVHVM, which may handle some architecture differences implicitly. But I'm assuming that doing #ifdef <architecture> in the Hyper-V code in order to reduce code under arch/x86 or arch/arm64 is not the right way to go.

In general that is true, adding a lot of #ifdefs makes code less readable and harder to test. OTOH there are cases where a single #ifdef can be useful when it avoids adding a larger amount of complexity elsewhere. Many subsystems try to restrict the #ifdef checks to header files while keeping the drivers/* code
free of them.

Arnd

#10 lore raw

PATCH v7 6/10 arm64: hyperv: Add kexec and panic handlers

In reply to Michael Kelley

Add functions to set up and remove kexec and panic handlers, and to inform Hyper-V about a guest panic. These functions are called from architecture independent code in the VMbus driver.

This code is built only when CONFIG_HYPERV is enabled.

---
 arch/arm64/hyperv/hv_core.c       | 80 +++++++++++++++++++++++++++++++++++++++
 arch/arm64/hyperv/mshyperv.c      | 26 +++++++++++++
 arch/arm64/include/asm/mshyperv.h |  2 +
 3 files changed, 108 insertions(+)
arch/arm64/hyperv/hv_core.c +80 -0
index 14e41b7..966d815 100644
@@ -211,3 +211,83 @@ void hv_get_vpreg_128(u32 msr, struct hv_get_vp_registers_output *res)
 	kfree(output);
 }
 EXPORT_SYMBOL_GPL(hv_get_vpreg_128);
+
+
+/*
+ * hyperv_report_panic - report a panic to Hyper-V.  This function uses
+ * the older version of the Hyper-V interface that admittedly doesn't
+ * pass enough information to be useful beyond just recording the
+ * occurrence of a panic. The parallel hyperv_report_panic_msg() uses the
+ * new interface that allows reporting 4 Kbytes of data, which is much
+ * more useful. Hyper-V on ARM64 always supports the newer interface, but
+ * we retain support for the older version because the sysadmin is allowed
+ * to disable the newer version via sysctl in case of information security
+ * concerns about the more verbose version.
+ */
+void hyperv_report_panic(struct pt_regs *regs, long err, bool in_die)
+{
+	static bool panic_reported;
+	u64 guest_id;
+
+	/* Don't report a panic to Hyper-V if we're not going to panic */
+	if (in_die && !panic_on_oops)
+		return;
+
+	/*
+	 * We prefer to report panic on 'die' chain as we have proper
+	 * registers to report, but if we miss it (e.g. on BUG()) we need
+	 * to report it on 'panic'.
+	 *
+	 * Calling code in the 'die' and 'panic' paths ensures that only
+	 * one CPU is running this code, so no atomicity is needed.
+	 */
+	if (panic_reported)
+		return;
+	panic_reported = true;
+
+	guest_id = hv_get_vpreg(HV_REGISTER_GUEST_OSID);
+
+	/*
+	 * Hyper-V provides the ability to store only 5 values.
+	 * Pick the passed in error value, the guest_id, and the PC.
+	 * The first two general registers are added arbitrarily.
+	 */
+	hv_set_vpreg(HV_REGISTER_CRASH_P0, err);
+	hv_set_vpreg(HV_REGISTER_CRASH_P1, guest_id);
+	hv_set_vpreg(HV_REGISTER_CRASH_P2, regs->pc);
+	hv_set_vpreg(HV_REGISTER_CRASH_P3, regs->regs[0]);
+	hv_set_vpreg(HV_REGISTER_CRASH_P4, regs->regs[1]);
+
+	/*
+	 * Let Hyper-V know there is crash data available
+	 */
+	hv_set_vpreg(HV_REGISTER_CRASH_CTL, HV_CRASH_CTL_CRASH_NOTIFY);
+}
+EXPORT_SYMBOL_GPL(hyperv_report_panic);
+
+/*
+ * hyperv_report_panic_msg - report panic message to Hyper-V
+ * @pa: physical address of the panic page containing the message
+ * @size: size of the message in the page
+ */
+void hyperv_report_panic_msg(phys_addr_t pa, size_t size)
+{
+	/*
+	 * P3 to contain the physical address of the panic page & P4 to
+	 * contain the size of the panic data in that page. Rest of the
+	 * registers are no-op when the NOTIFY_MSG flag is set.
+	 */
+	hv_set_vpreg(HV_REGISTER_CRASH_P0, 0);
+	hv_set_vpreg(HV_REGISTER_CRASH_P1, 0);
+	hv_set_vpreg(HV_REGISTER_CRASH_P2, 0);
+	hv_set_vpreg(HV_REGISTER_CRASH_P3, pa);
+	hv_set_vpreg(HV_REGISTER_CRASH_P4, size);
+
+	/*
+	 * Let Hyper-V know there is crash data available along with
+	 * the panic message.
+	 */
+	hv_set_vpreg(HV_REGISTER_CRASH_CTL,
+	       (HV_CRASH_CTL_CRASH_NOTIFY | HV_CRASH_CTL_CRASH_NOTIFY_MSG));
+}
+EXPORT_SYMBOL_GPL(hyperv_report_panic_msg);
arch/arm64/hyperv/mshyperv.c +26 -0
index be2cd2f..5bd39cb 100644
@@ -23,6 +23,8 @@
 
 static void (*vmbus_handler)(void);
 static void (*hv_stimer0_handler)(void);
+static void (*hv_kexec_handler)(void);
+static void (*hv_crash_handler)(struct pt_regs *regs);
 
 static int vmbus_irq;
 static long __percpu *vmbus_evt;
@@ -131,3 +133,27 @@ void hv_remove_stimer0_irq(int irq)
 	}
 }
 EXPORT_SYMBOL_GPL(hv_remove_stimer0_irq);
+
+void hv_setup_kexec_handler(void (*handler)(void))
+{
+	hv_kexec_handler = handler;
+}
+EXPORT_SYMBOL_GPL(hv_setup_kexec_handler);
+
+void hv_remove_kexec_handler(void)
+{
+	hv_kexec_handler = NULL;
+}
+EXPORT_SYMBOL_GPL(hv_remove_kexec_handler);
+
+void hv_setup_crash_handler(void (*handler)(struct pt_regs *regs))
+{
+	hv_crash_handler = handler;
+}
+EXPORT_SYMBOL_GPL(hv_setup_crash_handler);
+
+void hv_remove_crash_handler(void)
+{
+	hv_crash_handler = NULL;
+}
+EXPORT_SYMBOL_GPL(hv_remove_crash_handler);
arch/arm64/include/asm/mshyperv.h +2 -0
index 2ea64e54..5df96a5 100644
@@ -149,6 +149,8 @@ static inline int hv_get_vector(void)
 	return vmbus_interrupt;
 }
 
+#define hv_get_crash_ctl(val) \
+		(val = hv_get_vpreg(HV_REGISTER_CRASH_CTL))
 
 /* SMCCC hypercall parameters */
 #define HV_SMCCC_FUNC_NUMBER	1
Signature · 1 line
1.8.3.1
#11 lore raw

PATCH v7 7/10 arm64: hyperv: Initialize hypervisor on boot

In reply to Michael Kelley

Add ARM64-specific code to initialize the Hyper-V hypervisor when booting as a guest VM. Provide functions and data structures indicating hypervisor status that are needed by VMbus driver.

This code is built only when CONFIG_HYPERV is enabled.

---
 arch/arm64/hyperv/hv_core.c | 144 ++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 144 insertions(+)
arch/arm64/hyperv/hv_core.c +144 -0
index 966d815..831a69c 100644
@@ -18,10 +18,41 @@
 #include <linux/slab.h>
 #include <linux/hyperv.h>
 #include <linux/arm-smccc.h>
+#include <linux/vmalloc.h>
+#include <linux/acpi.h>
+#include <linux/module.h>
+#include <linux/cpuhotplug.h>
+#include <linux/psci.h>
 #include <asm-generic/bug.h>
 #include <asm/hyperv-tlfs.h>
 #include <asm/mshyperv.h>
+#include <asm/sysreg.h>
+#include <clocksource/hyperv_timer.h>
 
+static bool    hyperv_initialized;
+
+struct         ms_hyperv_info ms_hyperv __ro_after_init;
+EXPORT_SYMBOL_GPL(ms_hyperv);
+
+u32            *hv_vp_index;
+EXPORT_SYMBOL_GPL(hv_vp_index);
+
+u32            hv_max_vp_index;
+EXPORT_SYMBOL_GPL(hv_max_vp_index);
+
+static int hv_cpu_init(unsigned int cpu)
+{
+	u64 msr_vp_index;
+
+	msr_vp_index = hv_get_vpreg(HV_REGISTER_VPINDEX);
+
+	hv_vp_index[smp_processor_id()] = msr_vp_index;
+
+	if (msr_vp_index > hv_max_vp_index)
+		hv_max_vp_index = msr_vp_index;
+
+	return 0;
+}
 
 /*
  * Functions for allocating and freeing memory with size and
@@ -67,6 +98,107 @@ void hv_free_hyperv_page(unsigned long addr)
 
 
 /*
+ * This function is invoked via the ACPI clocksource probe mechanism. We
+ * don't actually use any values from the ACPI GTDT table, but we set up
+ * the Hyper-V synthetic clocksource and do other initialization for
+ * interacting with Hyper-V the first time.  Using early_initcall to invoke
+ * this function is too late because interrupts are already enabled at that
+ * point, and hv_init_clocksource() must run before interrupts are enabled.
+ *
+ * 1. Setup the guest ID.
+ * 2. Get features and hints info from Hyper-V
+ * 3. Setup per-cpu VP indices.
+ * 4. Initialize the Hyper-V clocksource.
+ */
+
+static int __init hyperv_init(struct acpi_table_header *table)
+{
+	struct hv_get_vp_registers_output result;
+	u32	a, b, c, d;
+	u64	guest_id;
+	int	i, cpuhp;
+
+	/*
+	 * If we're in a VM on Hyper-V, the ACPI hypervisor_id field will
+	 * have the string "MsHyperV".
+	 */
+	if (strncmp((char *)&acpi_gbl_FADT.hypervisor_id, "MsHyperV", 8))
+		return -EINVAL;
+
+	/* Setup the guest ID */
+	guest_id = generate_guest_id(0, LINUX_VERSION_CODE, 0);
+	hv_set_vpreg(HV_REGISTER_GUEST_OSID, guest_id);
+
+	/* Get the features and hints from Hyper-V */
+	hv_get_vpreg_128(HV_REGISTER_FEATURES, &result);
+	ms_hyperv.features = result.as32.a;
+	ms_hyperv.misc_features = result.as32.c;
+
+	hv_get_vpreg_128(HV_REGISTER_ENLIGHTENMENTS, &result);
+	ms_hyperv.hints = result.as32.a;
+
+	pr_info("Hyper-V: Features 0x%x, hints 0x%x, misc 0x%x\n",
+		ms_hyperv.features, ms_hyperv.hints, ms_hyperv.misc_features);
+
+	/*
+	 * If Hyper-V has crash notifications, set crash_kexec_post_notifiers
+	 * so that we will report the panic to Hyper-V before running kdump.
+	 */
+	if (ms_hyperv.misc_features & HV_FEATURE_GUEST_CRASH_MSR_AVAILABLE)
+		crash_kexec_post_notifiers = true;
+
+	/* Get information about the Hyper-V host version */
+	hv_get_vpreg_128(HV_REGISTER_HYPERVISOR_VERSION, &result);
+	a = result.as32.a;
+	b = result.as32.b;
+	c = result.as32.c;
+	d = result.as32.d;
+	pr_info("Hyper-V: Host Build %d.%d.%d.%d-%d-%d\n",
+		b >> 16, b & 0xFFFF, a,	d & 0xFFFFFF, c, d >> 24);
+
+	/* Allocate and initialize percpu VP index array */
+	hv_vp_index = kmalloc_array(num_possible_cpus(), sizeof(*hv_vp_index),
+				    GFP_KERNEL);
+	if (!hv_vp_index)
+		return -ENOMEM;
+
+	for (i = 0; i < num_possible_cpus(); i++)
+		hv_vp_index[i] = VP_INVAL;
+
+	cpuhp = cpuhp_setup_state(CPUHP_AP_ONLINE_DYN,
+			"arm64/hyperv_init:online", hv_cpu_init, NULL);
+	if (cpuhp < 0)
+		goto free_vp_index;
+
+	hv_init_clocksource();
+	if (hv_stimer_alloc())
+		goto remove_cpuhp_state;
+
+	hyperv_initialized = true;
+	return 0;
+
+remove_cpuhp_state:
+	cpuhp_remove_state(cpuhp);
+free_vp_index:
+	kfree(hv_vp_index);
+	hv_vp_index = NULL;
+	return -EINVAL;
+}
+TIMER_ACPI_DECLARE(hyperv, ACPI_SIG_GTDT, hyperv_init);
+
+/*
+ * This routine is called before kexec/kdump, it does the required cleanup.
+ */
+void hyperv_cleanup(void)
+{
+	/* Reset our OS id */
+	hv_set_vpreg(HV_REGISTER_GUEST_OSID, 0);
+
+}
+EXPORT_SYMBOL_GPL(hyperv_cleanup);
+
+
+/*
  * hv_do_hypercall- Invoke the specified hypercall
  */
 u64 hv_do_hypercall(u64 control, void *input, void *output)
@@ -291,3 +423,15 @@ void hyperv_report_panic_msg(phys_addr_t pa, size_t size)
 	       (HV_CRASH_CTL_CRASH_NOTIFY | HV_CRASH_CTL_CRASH_NOTIFY_MSG));
 }
 EXPORT_SYMBOL_GPL(hyperv_report_panic_msg);
+
+bool hv_is_hyperv_initialized(void)
+{
+	return hyperv_initialized;
+}
+EXPORT_SYMBOL_GPL(hv_is_hyperv_initialized);
+
+bool hv_is_hibernation_supported(void)
+{
+	return false;
+}
+EXPORT_SYMBOL_GPL(hv_is_hibernation_supported);
Signature · 1 line
1.8.3.1
maintainer
#12 lore raw

Re: [PATCH v7 07/10] arm64: hyperv: Initialize hypervisor on boot

Quoted 79 lines · Michael Kelley

/*
+ * This function is invoked via the ACPI clocksource probe mechanism. We
+ * don't actually use any values from the ACPI GTDT table, but we set up
+ * the Hyper-V synthetic clocksource and do other initialization for
+ * interacting with Hyper-V the first time. Using early_initcall to invoke
+ * this function is too late because interrupts are already enabled at that
+ * point, and hv_init_clocksource() must run before interrupts are enabled.
+ *
+ * 1. Setup the guest ID.
+ * 2. Get features and hints info from Hyper-V
+ * 3. Setup per-cpu VP indices.
+ * 4. Initialize the Hyper-V clocksource.
+ */

arch/arm64/hyperv/hv_core.c +65 -0
+
+static int __init hyperv_init(struct acpi_table_header *table)
+{
+       struct hv_get_vp_registers_output result;
+       u32     a, b, c, d;
+       u64     guest_id;
+       int     i, cpuhp;
+
+       /*
+        * If we're in a VM on Hyper-V, the ACPI hypervisor_id field will
+        * have the string "MsHyperV".
+        */
+       if (strncmp((char *)&acpi_gbl_FADT.hypervisor_id, "MsHyperV", 8))
+               return -EINVAL;
+
+       /* Setup the guest ID */
+       guest_id = generate_guest_id(0, LINUX_VERSION_CODE, 0);
+       hv_set_vpreg(HV_REGISTER_GUEST_OSID, guest_id);
+
+       /* Get the features and hints from Hyper-V */
+       hv_get_vpreg_128(HV_REGISTER_FEATURES, &result);
+       ms_hyperv.features = result.as32.a;
+       ms_hyperv.misc_features = result.as32.c;
+
+       hv_get_vpreg_128(HV_REGISTER_ENLIGHTENMENTS, &result);
+       ms_hyperv.hints = result.as32.a;
+
+       pr_info("Hyper-V: Features 0x%x, hints 0x%x, misc 0x%x\n",
+               ms_hyperv.features, ms_hyperv.hints, ms_hyperv.misc_features);
+
+       /*
+        * If Hyper-V has crash notifications, set crash_kexec_post_notifiers
+        * so that we will report the panic to Hyper-V before running kdump.
+        */
+       if (ms_hyperv.misc_features & HV_FEATURE_GUEST_CRASH_MSR_AVAILABLE)
+               crash_kexec_post_notifiers = true;
+
+       /* Get information about the Hyper-V host version */
+       hv_get_vpreg_128(HV_REGISTER_HYPERVISOR_VERSION, &result);
+       a = result.as32.a;
+       b = result.as32.b;
+       c = result.as32.c;
+       d = result.as32.d;
+       pr_info("Hyper-V: Host Build %d.%d.%d.%d-%d-%d\n",
+               b >> 16, b & 0xFFFF, a, d & 0xFFFFFF, c, d >> 24);
+
+       /* Allocate and initialize percpu VP index array */
+       hv_vp_index = kmalloc_array(num_possible_cpus(), sizeof(*hv_vp_index),
+                                   GFP_KERNEL);
+       if (!hv_vp_index)
+               return -ENOMEM;
+
+       for (i = 0; i < num_possible_cpus(); i++)
+               hv_vp_index[i] = VP_INVAL;
+
+       cpuhp = cpuhp_setup_state(CPUHP_AP_ONLINE_DYN,
+                       "arm64/hyperv_init:online", hv_cpu_init, NULL);
+       if (cpuhp < 0)
+               goto free_vp_index;
+
+       hv_init_clocksource();
+       if (hv_stimer_alloc())
+               goto remove_cpuhp_state;
+
+       hyperv_initialized = true;

arch/arm64/hyperv/hv_core.c void hv_free_hyperv_page(unsigned long addr) · line 187

Quoted 10 lines
arch/arm64/hyperv/hv_core.c +10 -0
+       return 0;
+
+remove_cpuhp_state:
+       cpuhp_remove_state(cpuhp);
+free_vp_index:
+       kfree(hv_vp_index);
+       hv_vp_index = NULL;
+       return -EINVAL;
+}
+TIMER_ACPI_DECLARE(hyperv, ACPI_SIG_GTDT, hyperv_init);

I think this has come up before, and I still don't consider it an acceptable hack to hook platform initialization code into the timer code.

Please split out the timer into a standalone driver in drivers/clocksource that can get reviewed by the clocksource maintainers.

Arnd

#13 lore raw

RE: [PATCH v7 07/10] arm64: hyperv: Initialize hypervisor on boot

From: Arnd Bergmann <[email protected]> Sent: Monday, August 24, 2020 11:34 AM

Quoted 95 lines · Michael Kelley
Quoted 79 lines

/*
+ * This function is invoked via the ACPI clocksource probe mechanism. We
+ * don't actually use any values from the ACPI GTDT table, but we set up
+ * the Hyper-V synthetic clocksource and do other initialization for
+ * interacting with Hyper-V the first time. Using early_initcall to invoke
+ * this function is too late because interrupts are already enabled at that
+ * point, and hv_init_clocksource() must run before interrupts are enabled.
+ *
+ * 1. Setup the guest ID.
+ * 2. Get features and hints info from Hyper-V
+ * 3. Setup per-cpu VP indices.
+ * 4. Initialize the Hyper-V clocksource.
+ */

arch/arm64/hyperv/hv_core.c +65 -0
+
+static int __init hyperv_init(struct acpi_table_header *table)
+{
+       struct hv_get_vp_registers_output result;
+       u32     a, b, c, d;
+       u64     guest_id;
+       int     i, cpuhp;
+
+       /*
+        * If we're in a VM on Hyper-V, the ACPI hypervisor_id field will
+        * have the string "MsHyperV".
+        */
+       if (strncmp((char *)&acpi_gbl_FADT.hypervisor_id, "MsHyperV", 8))
+               return -EINVAL;
+
+       /* Setup the guest ID */
+       guest_id = generate_guest_id(0, LINUX_VERSION_CODE, 0);
+       hv_set_vpreg(HV_REGISTER_GUEST_OSID, guest_id);
+
+       /* Get the features and hints from Hyper-V */
+       hv_get_vpreg_128(HV_REGISTER_FEATURES, &result);
+       ms_hyperv.features = result.as32.a;
+       ms_hyperv.misc_features = result.as32.c;
+
+       hv_get_vpreg_128(HV_REGISTER_ENLIGHTENMENTS, &result);
+       ms_hyperv.hints = result.as32.a;
+
+       pr_info("Hyper-V: Features 0x%x, hints 0x%x, misc 0x%x\n",
+               ms_hyperv.features, ms_hyperv.hints, ms_hyperv.misc_features);
+
+       /*
+        * If Hyper-V has crash notifications, set crash_kexec_post_notifiers
+        * so that we will report the panic to Hyper-V before running kdump.
+        */
+       if (ms_hyperv.misc_features & HV_FEATURE_GUEST_CRASH_MSR_AVAILABLE)
+               crash_kexec_post_notifiers = true;
+
+       /* Get information about the Hyper-V host version */
+       hv_get_vpreg_128(HV_REGISTER_HYPERVISOR_VERSION, &result);
+       a = result.as32.a;
+       b = result.as32.b;
+       c = result.as32.c;
+       d = result.as32.d;
+       pr_info("Hyper-V: Host Build %d.%d.%d.%d-%d-%d\n",
+               b >> 16, b & 0xFFFF, a, d & 0xFFFFFF, c, d >> 24);
+
+       /* Allocate and initialize percpu VP index array */
+       hv_vp_index = kmalloc_array(num_possible_cpus(), sizeof(*hv_vp_index),
+                                   GFP_KERNEL);
+       if (!hv_vp_index)
+               return -ENOMEM;
+
+       for (i = 0; i < num_possible_cpus(); i++)
+               hv_vp_index[i] = VP_INVAL;
+
+       cpuhp = cpuhp_setup_state(CPUHP_AP_ONLINE_DYN,
+                       "arm64/hyperv_init:online", hv_cpu_init, NULL);
+       if (cpuhp < 0)
+               goto free_vp_index;
+
+       hv_init_clocksource();
+       if (hv_stimer_alloc())
+               goto remove_cpuhp_state;
+
+       hyperv_initialized = true;

arch/arm64/hyperv/hv_core.c void hv_free_hyperv_page(unsigned long addr) · line 187

Quoted 10 lines
arch/arm64/hyperv/hv_core.c +10 -0
+       return 0;
+
+remove_cpuhp_state:
+       cpuhp_remove_state(cpuhp);
+free_vp_index:
+       kfree(hv_vp_index);
+       hv_vp_index = NULL;
+       return -EINVAL;
+}
+TIMER_ACPI_DECLARE(hyperv, ACPI_SIG_GTDT, hyperv_init);

I think this has come up before, and I still don't consider it an acceptable hack to hook platform initialization code into the timer code.

Quoted 2 lines

Please split out the timer into a standalone driver in drivers/clocksource that can get reviewed by the clocksource maintainers.

I see two related topics here. First, the Hyper-V clocksource driver is drivers/clocksource/hyperv_timer.c. The code is architecture independent and is used today on the x86 side and for ARM64 in this patch series. A few architecture specific calls are satisfied by code under arch/x86, and in this patch series, under arch/arm64. Is there some aspect of this driver that needs reconsideration? I just want to make sure to understand what you are getting at.

Second is the question of where/how to do Hyper-V specific initialization. I agree that hanging it off the timer initialization isn't a great approach. Should I add a Hyper-V specific initialization call at the appropriate point in the ARM64 init sequence? The x86 side has some structure for handling multiple hypervisors, and the Hyper-V initialization code naturally plugs into that structure. I'm certainly open to suggestions on the best way to handle it for ARM64.

Michael

Quoted 2 lines

Arnd

maintainer
#14 lore raw

Re: [PATCH v7 07/10] arm64: hyperv: Initialize hypervisor on boot

Quoted 10 lines · Michael Kelley

From: Arnd Bergmann <[email protected]> Sent: Monday, August 24, 2020 11:34 AM

Quoted 7 lines · Michael Kelley

I think this has come up before, and I still don't consider it an acceptable hack to hook platform initialization code into the timer code.

Please split out the timer into a standalone driver in drivers/clocksource that can get reviewed by the clocksource maintainers.

I see two related topics here.

Agreed

Quoted 7 lines

First, the Hyper-V clocksource driver is
drivers/clocksource/hyperv_timer.c. The code is architecture independent and is used today on the x86 side and for ARM64 in this patch series. A few architecture specific calls are satisfied by code under arch/x86, and in this patch series, under arch/arm64. Is there some aspect of this driver that needs reconsideration? I just want to make sure to understand what you are getting at.

For the clocksource driver, I would like to see the arm64 specific bits (the code you add in arch/arm64 that are only relevant to this driver) moved out of arch/arm64 and into drivers/clocksource, in whatever form the clocksource maintainers prefer. I would suggest having a separate file that can get linked along with the architecture-independent part of that driver.

Quoted 7 lines

Second is the question of where/how to do Hyper-V specific initialization. I agree that hanging it off the timer initialization isn't a great approach. Should I add a Hyper-V specific initialization call at the appropriate point in the ARM64 init sequence? The x86 side has some structure for handling multiple hypervisors, and the Hyper-V initialization code naturally plugs into that structure. I'm certainly open to suggestions on the best way to handle it for ARM64.

Yes, that is where I was getting at. Maybe the x86 abstraction for handling multiple hypervisors can be lifted out of arch/x86/ into common code?

Arnd

#15 lore raw

PATCH v7 8/10 Drivers: hv: vmbus: Add hooks for per-CPU IRQ

In reply to Michael Kelley

Add hooks to enable/disable a per-CPU IRQ for VMbus. These hooks are in the architecture independent setup and shutdown paths for Hyper-V, and are needed by Linux guests on Hyper-V on ARM64. The x86/x64 implementation is null because VMbus interrupts on x86/x64 don't use an IRQ.

---
 arch/x86/include/asm/mshyperv.h | 4 ++++
 drivers/hv/hv.c                 | 3 +++
 2 files changed, 7 insertions(+)
arch/x86/include/asm/mshyperv.h +4 -0
index ffc2899..dd1365c 100644
@@ -56,6 +56,10 @@ typedef int (*hyperv_fill_flush_list_func)(
 #define hv_get_raw_timer() rdtsc_ordered()
 #define hv_get_vector() HYPERVISOR_CALLBACK_VECTOR
 
+/* On x86/x64, there isn't a real IRQ to be enabled/disable */
+static inline void hv_enable_vmbus_irq(void) {}
+static inline void hv_disable_vmbus_irq(void) {}
+
 /*
  * Reference to pv_ops must be inline so objtool
  * detection of noinstr violations can work correctly.
drivers/hv/hv.c +3 -0
index 2bd44fd..7499079 100644
@@ -178,6 +178,7 @@ void hv_synic_enable_regs(unsigned int cpu)
 	hv_set_siefp(siefp.as_uint64);
 
 	/* Setup the shared SINT. */
+	hv_enable_vmbus_irq();
 	hv_get_synint_state(VMBUS_MESSAGE_SINT, shared_sint.as_uint64);
 
 	shared_sint.vector = hv_get_vector();
@@ -235,6 +236,8 @@ void hv_synic_disable_regs(unsigned int cpu)
 	hv_get_synic_state(sctrl.as_uint64);
 	sctrl.enable = 0;
 	hv_set_synic_state(sctrl.as_uint64);
+
+	hv_disable_vmbus_irq();
 }
 
 int hv_synic_cleanup(unsigned int cpu)
Signature · 1 line
1.8.3.1
#16 lore raw

PATCH v7 9/10 arm64: efi: Export screen_info

In reply to Michael Kelley

The Hyper-V frame buffer driver may be built as a module, and it needs access to screen_info. So export screen_info.

---
 arch/arm64/kernel/efi.c | 1 +
 1 file changed, 1 insertion(+)
arch/arm64/kernel/efi.c +1 -0
index d0cf596..8ff557a 100644
@@ -55,6 +55,7 @@ static __init pteval_t create_mapping_protection(efi_memory_desc_t *md)
 
 /* we will fill this structure from the stub, so don't put it in .bss */
 struct screen_info screen_info __section(.data);
+EXPORT_SYMBOL(screen_info);
 
 int __init efi_create_mapping(struct mm_struct *mm, efi_memory_desc_t *md)
 {
Signature · 1 line
1.8.3.1
maintainer Acked-by on v7 9/10
#17 lore raw

Re: [PATCH v7 09/10] arm64: efi: Export screen_info

Quoted 5 lines · Michael Kelley

The Hyper-V frame buffer driver may be built as a module, and it needs access to screen_info. So export screen_info.

Quoted 19 lines
---
 arch/arm64/kernel/efi.c | 1 +
 1 file changed, 1 insertion(+)
arch/arm64/kernel/efi.c +1 -1
index d0cf596..8ff557a 100644
@@ -55,6 +55,7 @@ static __init pteval_t create_mapping_protection(efi_memory_desc_t *md)
 
 /* we will fill this structure from the stub, so don't put it in .bss */
 struct screen_info screen_info __section(.data);
+EXPORT_SYMBOL(screen_info);
 
 int __init efi_create_mapping(struct mm_struct *mm, efi_memory_desc_t *md)
 {
--

1.8.3.1

#18 lore raw

Re: [PATCH v7 09/10] arm64: efi: Export screen_info

In reply to Michael Kelley
Quoted 13 lines · Michael Kelley

The Hyper-V frame buffer driver may be built as a module, and it needs access to screen_info. So export screen_info.

---
 arch/arm64/kernel/efi.c | 1 +
 1 file changed, 1 insertion(+)
arch/arm64/kernel/efi.c +0 -0
index d0cf596..8ff557a 100644

arch/arm64/kernel/efi.c static __init pteval_t create_mapping_protection(efi_memory_desc_t *md) · line 58

Quoted 4 lines
arch/arm64/kernel/efi.c +1 -0
@@ -55,6 +55,7 @@ static __init pteval_t create_mapping_protection(efi_memory_desc_t *md)
 
 /* we will fill this structure from the stub, so don't put it in .bss */
 struct screen_info screen_info __section(.data);
+EXPORT_SYMBOL(screen_info);

EXPORT_SYMBOL_GPL()?

I have to ask :)

thanks,

greg k-h

#19 lore raw

RE: [PATCH v7 09/10] arm64: efi: Export screen_info

From: Greg KH <[email protected]> Sent: Monday, August 24, 2020 10:35 AM

Quoted 17 lines · Michael Kelley

arch/arm64/kernel/efi.c static __init pteval_t

Quoted 13 lines

The Hyper-V frame buffer driver may be built as a module, and it needs access to screen_info. So export screen_info.

---
 arch/arm64/kernel/efi.c | 1 +
 1 file changed, 1 insertion(+)
arch/arm64/kernel/efi.c +0 -0
index d0cf596..8ff557a 100644
@@ -55,6 +55,7 @@ static __init pteval_t

create_mapping_protection(efi_memory_desc_t *md)

Quoted 1 line
Quoted 7 lines
Quoted 3 lines

/* we will fill this structure from the stub, so don't put it in .bss */ struct screen_info screen_info __section(.data);
+EXPORT_SYMBOL(screen_info);

EXPORT_SYMBOL_GPL()?

I have to ask :)

It's also just EXPORT_SYMBOL(screen_info) in the x86, PowerPC, and Alpha architectures. I know that doesn't guarantee it's right, but I'll have to defer to the appropriate subsystem maintainers for whether it really should be EXPORT_SYMBOL_GPL.

Michael

Quoted 4 lines

thanks,

greg k-h

#20 lore raw

Re: [PATCH v7 09/10] arm64: efi: Export screen_info

Quoted 22 lines · Michael Kelley

From: Greg KH <[email protected]> Sent: Monday, August 24, 2020 10:35 AM

Quoted 21 lines · Michael Kelley

arch/arm64/kernel/efi.c static __init pteval_t

Quoted 13 lines

The Hyper-V frame buffer driver may be built as a module, and it needs access to screen_info. So export screen_info.

---
 arch/arm64/kernel/efi.c | 1 +
 1 file changed, 1 insertion(+)
arch/arm64/kernel/efi.c +0 -0
index d0cf596..8ff557a 100644
@@ -55,6 +55,7 @@ static __init pteval_t

create_mapping_protection(efi_memory_desc_t *md)

Quoted 4 lines
arch/arm64/kernel/efi.c +1 -0
 /* we will fill this structure from the stub, so don't put it in .bss */
 struct screen_info screen_info __section(.data);
+EXPORT_SYMBOL(screen_info);
Quoted 8 lines
Quoted 3 lines

EXPORT_SYMBOL_GPL()?

I have to ask :)

It's also just EXPORT_SYMBOL(screen_info) in the x86, PowerPC, and Alpha architectures. I know that doesn't guarantee it's right, but I'll have to defer to the appropriate subsystem maintainers for whether it really should be EXPORT_SYMBOL_GPL.

Ok, fair enough, your original patch is fine.

thanks,

greg k-h

#21 lore raw

PATCH v7 10/10 Drivers: hv: Enable Hyper-V code to be built on ARM64

In reply to Michael Kelley

Update drivers/hv/Kconfig so CONFIG_HYPERV can be selected on ARM64, causing the Hyper-V specific code to be built.

---
 drivers/hv/Kconfig | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
drivers/hv/Kconfig +2 -1
index 79e5356..1113e49 100644
@@ -4,7 +4,8 @@ menu "Microsoft Hyper-V guest support"
 
 config HYPERV
 	tristate "Microsoft Hyper-V client drivers"
-	depends on X86 && ACPI && X86_LOCAL_APIC && HYPERVISOR_GUEST
+	depends on ACPI && \
+			((X86 && X86_LOCAL_APIC && HYPERVISOR_GUEST) || ARM64)
 	select PARAVIRT
 	select X86_HV_CALLBACK_VECTOR
 	help
Signature · 1 line
1.8.3.1
maintainer
#22 lore raw

Re: [PATCH v7 10/10] Drivers: hv: Enable Hyper-V code to be built on ARM64

Quoted 14 lines · Michael Kelley

Update drivers/hv/Kconfig so CONFIG_HYPERV can be selected on ARM64, causing the Hyper-V specific code to be built.

---
 drivers/hv/Kconfig | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
drivers/hv/Kconfig +0 -0
index 79e5356..1113e49 100644

drivers/hv/Kconfig menu "Microsoft Hyper-V guest support" · line 11

Quoted 9 lines
drivers/hv/Kconfig +2 -1
@@ -4,7 +4,8 @@ menu "Microsoft Hyper-V guest support"
 
 config HYPERV
        tristate "Microsoft Hyper-V client drivers"
-       depends on X86 && ACPI && X86_LOCAL_APIC && HYPERVISOR_GUEST
+       depends on ACPI && \
+                       ((X86 && X86_LOCAL_APIC && HYPERVISOR_GUEST) || ARM64)
        select PARAVIRT
        select X86_HV_CALLBACK_VECTOR
        help

Given the comment in a previous patch

+/*
+ * All data structures defined in the TLFS that are shared between Hyper-V
+ * and a guest VM use Little Endian byte ordering. This matches the default
+ * byte ordering of Linux running on ARM64, so no special handling is required.
+ */

shouldn't this depend on !CONFIG_CPU_BIG_ENDIAN ?

#23 lore raw

RE: [PATCH v7 10/10] Drivers: hv: Enable Hyper-V code to be built on ARM64

From: Ard Biesheuvel <[email protected]> Sent: Monday, August 24, 2020 10:24 AM

Quoted 28 lines · Michael Kelley
Quoted 14 lines

Update drivers/hv/Kconfig so CONFIG_HYPERV can be selected on ARM64, causing the Hyper-V specific code to be built.

---
 drivers/hv/Kconfig | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
drivers/hv/Kconfig +0 -0
index 79e5356..1113e49 100644

drivers/hv/Kconfig menu "Microsoft Hyper-V guest support" · line 11

Quoted 9 lines
drivers/hv/Kconfig +2 -1
@@ -4,7 +4,8 @@ menu "Microsoft Hyper-V guest support"
 
 config HYPERV
        tristate "Microsoft Hyper-V client drivers"
-       depends on X86 && ACPI && X86_LOCAL_APIC && HYPERVISOR_GUEST
+       depends on ACPI && \
+                       ((X86 && X86_LOCAL_APIC && HYPERVISOR_GUEST) || ARM64)
        select PARAVIRT
        select X86_HV_CALLBACK_VECTOR
        help

Given the comment in a previous patch

Quoted 7 lines

+/*
+ * All data structures defined in the TLFS that are shared between Hyper-V
+ * and a guest VM use Little Endian byte ordering. This matches the default
+ * byte ordering of Linux running on ARM64, so no special handling is required.
+ */

shouldn't this depend on !CONFIG_CPU_BIG_ENDIAN ?

Yes indeed. I'll add it.

Michael

#24 lore raw

Re: [PATCH v7 10/10] Drivers: hv: Enable Hyper-V code to be built on ARM64

In reply to Ard Biesheuvel
Quoted 27 lines · Ard Biesheuvel
Quoted 14 lines · Michael Kelley

Update drivers/hv/Kconfig so CONFIG_HYPERV can be selected on ARM64, causing the Hyper-V specific code to be built.

---
  drivers/hv/Kconfig | 3 ++-
  1 file changed, 2 insertions(+), 1 deletion(-)
drivers/hv/Kconfig +0 -0
index 79e5356..1113e49 100644

drivers/hv/Kconfig menu "Microsoft Hyper-V guest support" · line 11

Quoted 9 lines
drivers/hv/Kconfig +2 -1
@@ -4,7 +4,8 @@ menu "Microsoft Hyper-V guest support"
 
  config HYPERV
         tristate "Microsoft Hyper-V client drivers"
-       depends on X86 && ACPI && X86_LOCAL_APIC && HYPERVISOR_GUEST
+       depends on ACPI && \
+                       ((X86 && X86_LOCAL_APIC && HYPERVISOR_GUEST) || ARM64)
         select PARAVIRT
         select X86_HV_CALLBACK_VECTOR
         help

Given the comment in a previous patch

Quoted 7 lines

+/*
+ * All data structures defined in the TLFS that are shared between Hyper-V
+ * and a guest VM use Little Endian byte ordering. This matches the default
+ * byte ordering of Linux running on ARM64, so no special handling is required.
+ */

shouldn't this depend on !CONFIG_CPU_BIG_ENDIAN ?

or mark the data __le and have the appropriate accessor functions do the swapping.

Signature · 9 lines
Ben Dooks				http://www.codethink.co.uk/
Senior Engineer				Codethink - Providing Genius

https://www.codethink.co.uk/privacy.html

_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel