Skip to contentSkip to conversation

arm64: hyperv: Support Virtual Trust Level Boot

PATCH Superseded Roman Kisel 6 participants 53 messages Jul 26, 2024 – Aug 5, 2024

Subsystems (9)
Device tree bindings x86 ACPI PCI +5 more subsystems

v3 of 9 Superseded Latest is v9 →

lore
Share & tools
lore
#0 lore raw

v3 cover arm64: hyperv: Support Virtual Trust Level Boot

This patch set enables the Hyper-V code to boot on ARM64 inside a Virtual Trust Level. These levels are a part of the Virtual Secure Mode documented in the Top-Level Functional Specification available at
https://learn.microsoft.com/en-us/virtualization/hyper-v-on-windows/tlfs/vsm

[V3]
- Employed the SMC recently implemented in the Microsoft Hyper-V hypervisor
to detect running on Hyper-V/arm64. No dependence on ACPI/DT is needed
anymore although the source code still falls back to ACPI as the new
hypervisor might be available only in the Windows Insiders channel just
yet.
- As a part of the above, refactored detecting the hypervisor via ACPI FADT.
- There was a suggestion to explore whether it is feasible or not to express
that ACPI must be absent for the VTL mode and present for the regular guests
in the Hyper-V Kconfig file.
My current conclusion is that this will require refactoring in many places. That becomes especially convoluted on x86_64 due to the MSI and APIC dependencies. I'd ask to let us tackle that in another patch series (or chalk up to nice-have's rather than fires to put out) to separate concerns and decrease chances of breakage.
- While refactoring get_vtl(void) and the related code, fixed the hypercall output address not to overlap with the input as the Hyper-V TLFS mandates:
"The input and output parameter lists cannot overlap or cross page boundaries." See https://learn.microsoft.com/en-us/virtualization/hyper-v-on-windows/tlfs/hypercall-interface
for more.
Some might argue that should've been a topic for a separate patch series;
I'd counter that the change is well-contained (one line), has no dependencies, and makes the code legal.
- Made the VTL boot code (c)leaner as was suggested.
- Set DMA cache coherency for the VMBus.
- Updated DT bindings in the VMBus documentation (separated out into a new patch).
- Fixed vmbus_set_irq to use the API that works both for the ACPI and OF.
- Reworked setting up the vPCI MSI IRQ domain in the non-ACPI case. The logic looks a bit fiddly/ad-hoc as I couldn't find the API that would fit the bill. Added comments to explain myself.

[V2]
https://lore.kernel.org/all/[email protected]/
- Decreased number of #ifdef's
- Updated the wording in the commit messages to adhere to the guidlines
- Sending to the correct set of maintainers and mail lists

[V1]

    https://lore.kernel.org/all/[email protected]/

For validation, I built kernels for the arch'es in question with the small initrd embedded into the kernel and booted the Hyper-V VMs off of that.

Roman Kisel (7):
arm64: hyperv: Use SMC to detect hypervisor presence
Drivers: hv: Enable VTL mode for arm64
Drivers: hv: Provide arch-neutral implementation of get_vtl()
arm64: hyperv: Boot in a Virtual Trust Level
dt-bindings: bus: Add Hyper-V VMBus cache coherency and IRQs
Drivers: hv: vmbus: Get the IRQ number from DT
PCI: hv: Get vPCI MSI IRQ domain from DT

 .../bindings/bus/microsoft,vmbus.yaml         | 11 +++
 arch/arm64/hyperv/Makefile                    |  1 +
 arch/arm64/hyperv/hv_vtl.c                    | 13 ++++
 arch/arm64/hyperv/mshyperv.c                  | 40 +++++++++--
 arch/arm64/include/asm/mshyperv.h             | 12 ++++
 arch/x86/hyperv/hv_init.c                     | 34 ---------
 arch/x86/include/asm/hyperv-tlfs.h            |  7 --
 drivers/hv/Kconfig                            |  6 +-
 drivers/hv/hv_common.c                        | 47 +++++++++++-
 drivers/hv/vmbus_drv.c                        | 72 ++++++++++++++++---
 drivers/pci/controller/pci-hyperv.c           | 55 +++++++++++++-
 include/asm-generic/hyperv-tlfs.h             |  7 ++
 include/asm-generic/mshyperv.h                |  6 ++
 include/linux/hyperv.h                        |  2 +
 14 files changed, 251 insertions(+), 62 deletions(-)
 create mode 100644 arch/arm64/hyperv/hv_vtl.c

base-commit: 831bcbcead6668ebf20b64fdb27518f1362ace3a

Signature · 1 line
2.34.1
#1 lore raw

PATCH v3 1/7 arm64: hyperv: Use SMC to detect hypervisor presence

The arm64 Hyper-V startup path relies on ACPI to detect running under a Hyper-V compatible hypervisor. That doesn't work on non-ACPI systems.

Hoist the ACPI detection logic into a separate function, use the new SMC added recently to Hyper-V to use in the non-ACPI case.

---
 arch/arm64/hyperv/mshyperv.c      | 36 ++++++++++++++++++++++++++-----
 arch/arm64/include/asm/mshyperv.h |  5 +++++
 2 files changed, 36 insertions(+), 5 deletions(-)
arch/arm64/hyperv/mshyperv.c +31 -5
index b1a4de4eee29..341f98312667 100644
@@ -27,6 +27,34 @@ int hv_get_hypervisor_version(union hv_hypervisor_version_info *info)
 	return 0;
 }
 
+static bool hyperv_detect_via_acpi(void)
+{
+	if (acpi_disabled)
+		return false;
+#if IS_ENABLED(CONFIG_ACPI)
+	/* Hypervisor ID is only available in ACPI v6+. */
+	if (acpi_gbl_FADT.header.revision < 6)
+		return false;
+	return strncmp((char *)&acpi_gbl_FADT.hypervisor_id, "MsHyperV", 8) == 0;
+#else
+	return false;
+#endif
+}
+
+static bool hyperv_detect_via_smc(void)
+{
+	struct arm_smccc_res res = {};
+
+	if (arm_smccc_1_1_get_conduit() != SMCCC_CONDUIT_HVC)
+		return false;
+	arm_smccc_1_1_hvc(ARM_SMCCC_VENDOR_HYP_CALL_UID_FUNC_ID, &res);
+
+	return res.a0 == ARM_SMCCC_VENDOR_HYP_UID_HYPERV_REG_0 &&
+		res.a1 == ARM_SMCCC_VENDOR_HYP_UID_HYPERV_REG_1 &&
+		res.a2 == ARM_SMCCC_VENDOR_HYP_UID_HYPERV_REG_2 &&
+		res.a3 == ARM_SMCCC_VENDOR_HYP_UID_HYPERV_REG_3;
+}
+
 static int __init hyperv_init(void)
 {
 	struct hv_get_vp_registers_output	result;
@@ -35,13 +63,11 @@ static int __init hyperv_init(void)
 
 	/*
 	 * Allow for a kernel built with CONFIG_HYPERV to be running in
-	 * a non-Hyper-V environment, including on DT instead of ACPI.
+	 * a non-Hyper-V environment.
+	 *
 	 * In such cases, do nothing and return success.
 	 */
-	if (acpi_disabled)
-		return 0;
-
-	if (strncmp((char *)&acpi_gbl_FADT.hypervisor_id, "MsHyperV", 8))
+	if (!hyperv_detect_via_acpi() && !hyperv_detect_via_smc())
 		return 0;
 
 	/* Setup the guest ID */
arch/arm64/include/asm/mshyperv.h +5 -0
index a975e1a689dd..a7a3586f7cb1 100644
@@ -51,4 +51,9 @@ static inline u64 hv_get_msr(unsigned int reg)
 
 #include <asm-generic/mshyperv.h>
 
+#define ARM_SMCCC_VENDOR_HYP_UID_HYPERV_REG_0	0x7948734d
+#define ARM_SMCCC_VENDOR_HYP_UID_HYPERV_REG_1	0x56726570
+#define ARM_SMCCC_VENDOR_HYP_UID_HYPERV_REG_2	0
+#define ARM_SMCCC_VENDOR_HYP_UID_HYPERV_REG_3	0
+
 #endif
Signature · 1 line
2.34.1
#12 lore raw

PATCH v3 2/7 Drivers: hv: Enable VTL mode for arm64

In reply to Roman Kisel

Kconfig dependencies for arm64 guests on Hyper-V require that be ACPI enabled, and limit VTL mode to x86/x64. To enable VTL mode on arm64 as well, update the dependencies. Since VTL mode requires DeviceTree instead of ACPI, don’t require arm64 guests on Hyper-V to have ACPI.

---
 drivers/hv/Kconfig | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
drivers/hv/Kconfig +3 -3
index 862c47b191af..a5cd1365e248 100644
@@ -5,7 +5,7 @@ menu "Microsoft Hyper-V guest support"
 config HYPERV
 	tristate "Microsoft Hyper-V client drivers"
 	depends on (X86 && X86_LOCAL_APIC && HYPERVISOR_GUEST) \
-		|| (ACPI && ARM64 && !CPU_BIG_ENDIAN)
+		|| (ARM64 && !CPU_BIG_ENDIAN)
 	select PARAVIRT
 	select X86_HV_CALLBACK_VECTOR if X86
 	select OF_EARLY_FLATTREE if OF
@@ -15,7 +15,7 @@ config HYPERV
 
 config HYPERV_VTL_MODE
 	bool "Enable Linux to boot in VTL context"
-	depends on X86_64 && HYPERV
+	depends on HYPERV
 	depends on SMP
 	default n
 	help
@@ -31,7 +31,7 @@ config HYPERV_VTL_MODE
 
 	  Select this option to build a Linux kernel to run at a VTL other than
 	  the normal VTL0, which currently is only VTL2.  This option
-	  initializes the x86 platform for VTL2, and adds the ability to boot
+	  initializes the kernel to run in VTL2, and adds the ability to boot
 	  secondary CPUs directly into 64-bit context as required for VTLs other
 	  than 0.  A kernel built with this option must run at VTL2, and will
 	  not run as a normal guest.
Signature · 1 line
2.34.1
#19 lore raw

PATCH v3 3/7 Drivers: hv: Provide arch-neutral implementation of get_vtl()

In reply to Roman Kisel

To run in the VTL mode, Hyper-V drivers have to know what VTL the system boots in, and the arm64/hyperv code does not have the means to compute that.

Refactor the code to hoist the function that detects VTL, make it arch-neutral to be able to employ it to get the VTL on arm64. Fix the hypercall output address in get_vtl(void) not to overlap with the hypercall input area to adhere to the Hyper-V TLFS.

---
 arch/x86/hyperv/hv_init.c          | 34 ---------------------
 arch/x86/include/asm/hyperv-tlfs.h |  7 -----
 drivers/hv/hv_common.c             | 47 ++++++++++++++++++++++++++++--
 include/asm-generic/hyperv-tlfs.h  |  7 +++++
 include/asm-generic/mshyperv.h     |  6 ++++
 5 files changed, 58 insertions(+), 43 deletions(-)
arch/x86/hyperv/hv_init.c +0 -34
index 17a71e92a343..c350fa05ee59 100644
@@ -413,40 +413,6 @@ static void __init hv_get_partition_id(void)
 	local_irq_restore(flags);
 }
 
-#if IS_ENABLED(CONFIG_HYPERV_VTL_MODE)
-static u8 __init get_vtl(void)
-{
-	u64 control = HV_HYPERCALL_REP_COMP_1 | HVCALL_GET_VP_REGISTERS;
-	struct hv_get_vp_registers_input *input;
-	struct hv_get_vp_registers_output *output;
-	unsigned long flags;
-	u64 ret;
-
-	local_irq_save(flags);
-	input = *this_cpu_ptr(hyperv_pcpu_input_arg);
-	output = (struct hv_get_vp_registers_output *)input;
-
-	memset(input, 0, struct_size(input, element, 1));
-	input->header.partitionid = HV_PARTITION_ID_SELF;
-	input->header.vpindex = HV_VP_INDEX_SELF;
-	input->header.inputvtl = 0;
-	input->element[0].name0 = HV_X64_REGISTER_VSM_VP_STATUS;
-
-	ret = hv_do_hypercall(control, input, output);
-	if (hv_result_success(ret)) {
-		ret = output->as64.low & HV_X64_VTL_MASK;
-	} else {
-		pr_err("Failed to get VTL(error: %lld) exiting...\n", ret);
-		BUG();
-	}
-
-	local_irq_restore(flags);
-	return ret;
-}
-#else
-static inline u8 get_vtl(void) { return 0; }
-#endif
-
 /*
  * This function is to be invoked early in the boot sequence after the
  * hypervisor has been detected.
arch/x86/include/asm/hyperv-tlfs.h +0 -7
index 3787d26810c1..9ee68eb8e6ff 100644
@@ -309,13 +309,6 @@ enum hv_isolation_type {
 #define HV_MSR_STIMER0_CONFIG	(HV_X64_MSR_STIMER0_CONFIG)
 #define HV_MSR_STIMER0_COUNT	(HV_X64_MSR_STIMER0_COUNT)
 
-/*
- * Registers are only accessible via HVCALL_GET_VP_REGISTERS hvcall and
- * there is not associated MSR address.
- */
-#define	HV_X64_REGISTER_VSM_VP_STATUS	0x000D0003
-#define	HV_X64_VTL_MASK			GENMASK(3, 0)
-
 /* Hyper-V memory host visibility */
 enum hv_mem_host_visibility {
 	VMBUS_PAGE_NOT_VISIBLE		= 0,
drivers/hv/hv_common.c +45 -2

56 lines not on this page · raw message

include/asm-generic/hyperv-tlfs.h +7 -0
index 814207e7c37f..271c365973d6 100644
@@ -75,6 +75,13 @@
 /* AccessTscInvariantControls privilege */
 #define HV_ACCESS_TSC_INVARIANT			BIT(15)
 
+/*
+ * This synthetic register is only accessible via the HVCALL_GET_VP_REGISTERS
+ * hvcall, and there is no an associated MSR on x86.
+ */
+#define	HV_REGISTER_VSM_VP_STATUS	0x000D0003
+#define	HV_VTL_MASK			GENMASK(3, 0)
+
 /*
  * Group B features.
  */
include/asm-generic/mshyperv.h +6 -0
index 8fe7aaab2599..85a5b8cb1702 100644
@@ -315,4 +315,10 @@ static inline enum hv_isolation_type hv_get_isolation_type(void)
 }
 #endif /* CONFIG_HYPERV */
 
+#if IS_ENABLED(CONFIG_HYPERV_VTL_MODE)
+u8 __init get_vtl(void);
+#else
+static inline u8 get_vtl(void) { return 0; }
+#endif
+
 #endif
Signature · 1 line
2.34.1
#26 lore raw

PATCH v3 4/7 arm64: hyperv: Boot in a Virtual Trust Level

In reply to Roman Kisel

To run in the VTL mode, Hyper-V drivers have to know what VTL the system boots in, and the arm64/hyperv code does not update the variable that stores the value.

Update the variable to enable the Hyper-V drivers to boot in the VTL mode and print the VTL the code runs in.

---
 arch/arm64/hyperv/Makefile        |  1 +
 arch/arm64/hyperv/hv_vtl.c        | 13 +++++++++++++
 arch/arm64/hyperv/mshyperv.c      |  4 ++++
 arch/arm64/include/asm/mshyperv.h |  7 +++++++
 4 files changed, 25 insertions(+)
 create mode 100644 arch/arm64/hyperv/hv_vtl.c
arch/arm64/hyperv/Makefile +1 -0
index 87c31c001da9..9701a837a6e1 100644
@@ -1,2 +1,3 @@
 # SPDX-License-Identifier: GPL-2.0
 obj-y		:= hv_core.o mshyperv.o
+obj-$(CONFIG_HYPERV_VTL_MODE)	+= hv_vtl.o
arch/arm64/hyperv/hv_vtl.c new file +13 -0
new file mode 100644
index 000000000000..38642b7b6be0
@@ -0,0 +1,13 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (C) 2024, Microsoft, Inc.
+ *
+ * Author : Roman Kisel <[email protected]>
+ */
+
+#include <asm/mshyperv.h>
+
+void __init hv_vtl_init_platform(void)
+{
+	pr_info("Linux runs in Hyper-V Virtual Trust Level\n");
+}
arch/arm64/hyperv/mshyperv.c +4 -0
index 341f98312667..8fd04d6e4800 100644
@@ -98,6 +98,10 @@ static int __init hyperv_init(void)
 		return ret;
 	}
 
+	/* Find the VTL */
+	ms_hyperv.vtl = get_vtl();
+	hv_vtl_init_platform();
+
 	ms_hyperv_late_init();
 
 	hyperv_initialized = true;
arch/arm64/include/asm/mshyperv.h +7 -0
index a7a3586f7cb1..63d6bb6998fc 100644
@@ -49,6 +49,13 @@ static inline u64 hv_get_msr(unsigned int reg)
 				ARM_SMCCC_OWNER_VENDOR_HYP,	\
 				HV_SMCCC_FUNC_NUMBER)
 
+#ifdef CONFIG_HYPERV_VTL_MODE
+void __init hv_vtl_init_platform(void);
+int __init hv_vtl_early_init(void);
+#else
+static inline void __init hv_vtl_init_platform(void) {}
+#endif
+
 #include <asm-generic/mshyperv.h>
 
 #define ARM_SMCCC_VENDOR_HYP_UID_HYPERV_REG_0	0x7948734d
Signature · 1 line
2.34.1
#33 lore raw

PATCH v3 5/7 dt-bindings: bus: Add Hyper-V VMBus cache coherency and IRQs

In reply to Roman Kisel

Add dt-bindings for the Hyper-V VMBus DMA cache coherency and interrupt specification.

---
 .../devicetree/bindings/bus/microsoft,vmbus.yaml      | 11 +++++++++++
 1 file changed, 11 insertions(+)
Documentation/devicetree/bindings/bus/microsoft,vmbus.yaml +11 -0
index a8d40c766dcd..5ec69226ab85 100644
@@ -44,11 +44,22 @@ examples:
             #size-cells = <1>;
             ranges;
 
+            gic: intc@fe200000 {
+              compatible = "arm,gic-v3";
+              reg = <0x0 0xfe200000 0x0 0x10000>,   /* GIC Dist */
+                    <0x0 0xfe280000 0x0 0x200000>;  /* GICR */
+              interrupt-controller;
+              #interrupt-cells = <3>;
+            }
+
             vmbus@ff0000000 {
                 compatible = "microsoft,vmbus";
                 #address-cells = <2>;
                 #size-cells = <1>;
                 ranges = <0x0f 0xf0000000 0x0f 0xf0000000 0x10000000>;
+                dma-coherent;
+                interrupt-parent = <&gic>;
+                interrupts = <1 2 1>;
             };
         };
     };
Signature · 1 line
2.34.1
#36 lore raw

PATCH v3 6/7 Drivers: hv: vmbus: Get the IRQ number from DT

In reply to Roman Kisel

The VMBus driver uses ACPI for interrupt assignment on arm64 hence it won't function in the VTL mode where only DeviceTree can be used.

Update the VMBus driver to discover interrupt configuration via DeviceTree and indicate DMA cache coherency.

---
 drivers/hv/vmbus_drv.c | 49 ++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 49 insertions(+)
drivers/hv/vmbus_drv.c +49 -0
index 12a707ab73f8..7eee7caff5f6 100644
@@ -2306,6 +2306,34 @@ static int vmbus_acpi_add(struct platform_device *pdev)
 }
 #endif
 
+static int __maybe_unused vmbus_set_irq(struct platform_device *pdev)
+{
+	struct irq_desc *desc;
+	int irq;
+
+	irq = platform_get_irq(pdev, 0);
+	if (irq == 0) {
+		pr_err("VMBus interrupt mapping failure\n");
+		return -EINVAL;
+	}
+	if (irq < 0) {
+		pr_err("VMBus interrupt data can't be read from DeviceTree, error %d\n", irq);
+		return irq;
+	}
+
+	desc = irq_to_desc(irq);
+	if (!desc) {
+		pr_err("No interrupt descriptor for VMBus virq %d\n", irq);
+		return -ENODEV;
+	}
+
+	vmbus_irq = irq;
+	vmbus_interrupt = desc->irq_data.hwirq;
+	pr_debug("VMBus virq %d, hwirq %d\n", vmbus_irq, vmbus_interrupt);
+
+	return 0;
+}
+
 static int vmbus_device_add(struct platform_device *pdev)
 {
 	struct resource **cur_res = &hyperv_mmio;
@@ -2320,6 +2348,12 @@ static int vmbus_device_add(struct platform_device *pdev)
 	if (ret)
 		return ret;
 
+#ifndef HYPERVISOR_CALLBACK_VECTOR
+	ret = vmbus_set_irq(pdev);
+	if (ret)
+		return ret;
+#endif
+
 	for_each_of_range(&parser, &range) {
 		struct resource *res;
 
@@ -2338,6 +2372,21 @@ static int vmbus_device_add(struct platform_device *pdev)
 		cur_res = &res->sibling;
 	}
 
+	/*
+	 * Hyper-V always assumes DMA cache coherency, and the DMA subsystem
+	 * might default to 'not coherent' on some architectures.
+	 * Avoid high-cost cache coherency maintenance done by the CPU.
+	 */
+#if defined(CONFIG_ARCH_HAS_SYNC_DMA_FOR_DEVICE) || \
+	defined(CONFIG_ARCH_HAS_SYNC_DMA_FOR_CPU) || \
+	defined(CONFIG_ARCH_HAS_SYNC_DMA_FOR_CPU_ALL)
+
+	if (!of_property_read_bool(np, "dma-coherent"))
+		pr_warn("Assuming cache coherent DMA transactions, no 'dma-coherent' node supplied\n");
+	pdev->dev.dma_coherent = true;
+
+#endif
+
 	return ret;
 }
 
Signature · 1 line
2.34.1
#47 lore raw

PATCH v3 7/7 PCI: hv: Get vPCI MSI IRQ domain from DT

In reply to Roman Kisel

The hyperv-pci driver uses ACPI for MSI IRQ domain configuration on arm64. It won't be able to do that in the VTL mode where only DeviceTree can be used.

Update the hyperv-pci driver to get vPCI MSI IRQ domain in the DeviceTree case, too.

---
 drivers/hv/vmbus_drv.c              | 23 +++++++-----
 drivers/pci/controller/pci-hyperv.c | 55 +++++++++++++++++++++++++++--
 include/linux/hyperv.h              |  2 ++
 3 files changed, 69 insertions(+), 11 deletions(-)
drivers/hv/vmbus_drv.c +15 -8
index 7eee7caff5f6..038bd9be64b7 100644
@@ -45,7 +45,8 @@ struct vmbus_dynid {
 	struct hv_vmbus_device_id id;
 };
 
-static struct device  *hv_dev;
+/* VMBus Root Device */
+static struct device  *vmbus_root_device;
 
 static int hyperv_cpuhp_online;
 
@@ -80,9 +81,15 @@ static struct resource *fb_mmio;
 static struct resource *hyperv_mmio;
 static DEFINE_MUTEX(hyperv_mmio_lock);
 
+struct device *get_vmbus_root_device(void)
+{
+	return vmbus_root_device;
+}
+EXPORT_SYMBOL_GPL(get_vmbus_root_device);
+
 static int vmbus_exists(void)
 {
-	if (hv_dev == NULL)
+	if (vmbus_root_device == NULL)
 		return -ENODEV;
 
 	return 0;
@@ -861,7 +868,7 @@ static int vmbus_dma_configure(struct device *child_device)
 	 * On x86/x64 coherence is assumed and these calls have no effect.
 	 */
 	hv_setup_dma_ops(child_device,
-		device_get_dma_attr(hv_dev) == DEV_DMA_COHERENT);
+		device_get_dma_attr(vmbus_root_device) == DEV_DMA_COHERENT);
 	return 0;
 }
 
@@ -1892,7 +1899,7 @@ int vmbus_device_register(struct hv_device *child_device_obj)
 		     &child_device_obj->channel->offermsg.offer.if_instance);
 
 	child_device_obj->device.bus = &hv_bus;
-	child_device_obj->device.parent = hv_dev;
+	child_device_obj->device.parent = vmbus_root_device;
 	child_device_obj->device.release = vmbus_device_release;
 
 	child_device_obj->device.dma_parms = &child_device_obj->dma_parms;
@@ -2253,7 +2260,7 @@ static int vmbus_acpi_add(struct platform_device *pdev)
 	struct acpi_device *ancestor;
 	struct acpi_device *device = ACPI_COMPANION(&pdev->dev);
 
-	hv_dev = &device->dev;
+	vmbus_root_device = &device->dev;
 
 	/*
 	 * Older versions of Hyper-V for ARM64 fail to include the _CCA
@@ -2342,7 +2349,7 @@ static int vmbus_device_add(struct platform_device *pdev)
 	struct device_node *np = pdev->dev.of_node;
 	int ret;
 
-	hv_dev = &pdev->dev;
+	vmbus_root_device = &pdev->dev;
 
 	ret = of_range_parser_init(&parser, np);
 	if (ret)
@@ -2675,7 +2682,7 @@ static int __init hv_acpi_init(void)
 	if (ret)
 		return ret;
 
-	if (!hv_dev) {
+	if (!vmbus_root_device) {
 		ret = -ENODEV;
 		goto cleanup;
 	}
@@ -2706,7 +2713,7 @@ static int __init hv_acpi_init(void)
 
 cleanup:
 	platform_driver_unregister(&vmbus_platform_driver);
-	hv_dev = NULL;
+	vmbus_root_device = NULL;
 	return ret;
 }
 
drivers/pci/controller/pci-hyperv.c +52 -3

74 lines not on this page · raw message

include/linux/hyperv.h +2 -0
index 5e39baa7f6cb..b4aa1f579a97 100644
@@ -1346,6 +1346,8 @@ static inline void *hv_get_drvdata(struct hv_device *dev)
 	return dev_get_drvdata(&dev->device);
 }
 
+struct device *get_vmbus_root_device(void);
+
 struct hv_ring_buffer_debug_info {
 	u32 current_interrupt_mask;
 	u32 current_read_index;
Signature · 1 line
2.34.1
#34 lore raw

Re: [PATCH v3 5/7] dt-bindings: bus: Add Hyper-V VMBus cache coherency and IRQs

In reply to Roman Kisel
Quoted 3 lines · Roman Kisel

Add dt-bindings for the Hyper-V VMBus DMA cache coherency and interrupt specification.

You did not add any bindings. I don't understand this description.

Anyway, not tested.

<form letter>
Please use scripts/get_maintainers.pl to get a list of necessary people and lists to CC. It might happen, that command when run on an older kernel, gives you outdated entries. Therefore please be sure you base your patches on recent Linux kernel.

Tools like b4 or scripts/get_maintainer.pl provide you proper list of people, so fix your workflow. Tools might also fail if you work on some ancient tree (don't, instead use mainline) or work on fork of kernel (don't, instead use mainline). Just use b4 and everything should be fine, although remember about b4 prep --auto-to-cc if you added new patches to the patchset.

You missed at least devicetree list (maybe more), so this won't be tested by automated tooling. Performing review on untested code might be a waste of time.

Please kindly resend and include all necessary To/Cc entries. </form letter>

Best regards,
Krzysztof

#37 lore raw

Re: [PATCH v3 6/7] Drivers: hv: vmbus: Get the IRQ number from DT

In reply to Roman Kisel
Quoted 5 lines · Roman Kisel
drivers/hv/vmbus_drv.c +1 -0
@@ -2338,6 +2372,21 @@ static int vmbus_device_add(struct platform_device *pdev)
 		cur_res = &res->sibling;
 	}
 
+	/*

drivers/hv/vmbus_drv.c static int vmbus_device_add(struct platform_device *pdev) · line 2385

Quoted 10 lines
drivers/hv/vmbus_drv.c +10 -0
+	 * Hyper-V always assumes DMA cache coherency, and the DMA subsystem
+	 * might default to 'not coherent' on some architectures.
+	 * Avoid high-cost cache coherency maintenance done by the CPU.
+	 */
+#if defined(CONFIG_ARCH_HAS_SYNC_DMA_FOR_DEVICE) || \
+	defined(CONFIG_ARCH_HAS_SYNC_DMA_FOR_CPU) || \
+	defined(CONFIG_ARCH_HAS_SYNC_DMA_FOR_CPU_ALL)
+
+	if (!of_property_read_bool(np, "dma-coherent"))
+		pr_warn("Assuming cache coherent DMA transactions, no 'dma-coherent' node supplied\n");

Why do you need this property at all, if it is allways dma-coherent? Are you supporting dma-noncoherent somewhere?

Best regards,
Krzysztof

maintainer
#38 lore raw

Re: [PATCH v3 6/7] Drivers: hv: vmbus: Get the IRQ number from DT

Quoted 5 lines · Krzysztof Kozlowski
Quoted 4 lines · Roman Kisel
drivers/hv/vmbus_drv.c +0 -0
@@ -2338,6 +2372,21 @@ static int vmbus_device_add(struct platform_device *pdev)
 		cur_res = &res->sibling;
 	}
 
Quoted 14 lines

drivers/hv/vmbus_drv.c static int vmbus_device_add(struct platform_device *pdev) · line 2385

Quoted 11 lines
drivers/hv/vmbus_drv.c +11 -0
+	/*
+	 * Hyper-V always assumes DMA cache coherency, and the DMA subsystem
+	 * might default to 'not coherent' on some architectures.
+	 * Avoid high-cost cache coherency maintenance done by the CPU.
+	 */
+#if defined(CONFIG_ARCH_HAS_SYNC_DMA_FOR_DEVICE) || \
+	defined(CONFIG_ARCH_HAS_SYNC_DMA_FOR_CPU) || \
+	defined(CONFIG_ARCH_HAS_SYNC_DMA_FOR_CPU_ALL)
+
+	if (!of_property_read_bool(np, "dma-coherent"))
+		pr_warn("Assuming cache coherent DMA transactions, no 'dma-coherent' node supplied\n");

Why do you need this property at all, if it is allways dma-coherent? Are you supporting dma-noncoherent somewhere?

It's just a sanity check that the DT is well-formed.

Since the dma-coherent property is interpreted by common code, it's not up to hv to change the default for the platform. I'm not sure if the presence of CONFIG_ARCH_HAS_SYNC_DMA_* options is the correct check to determine that an architecture defaults to noncoherent though, as the function may be needed to do something else.

The global "dma_default_coherent' may be a better thing to check for. This is e.g. set on powerpc64, riscv and on specific mips platforms, but it's never set on arm64 as far as I can tell.

Arnd

#39 lore raw

Re: [PATCH v3 6/7] Drivers: hv: vmbus: Get the IRQ number from DT

Quoted 29 lines · Arnd Bergmann
Quoted 5 lines · Krzysztof Kozlowski
Quoted 4 lines · Roman Kisel
drivers/hv/vmbus_drv.c +0 -0
@@ -2338,6 +2372,21 @@ static int vmbus_device_add(struct platform_device *pdev)
 		cur_res = &res->sibling;
 	}
 
Quoted 14 lines

drivers/hv/vmbus_drv.c static int vmbus_device_add(struct platform_device *pdev) · line 2385

Quoted 11 lines
drivers/hv/vmbus_drv.c +11 -0
+	/*
+	 * Hyper-V always assumes DMA cache coherency, and the DMA subsystem
+	 * might default to 'not coherent' on some architectures.
+	 * Avoid high-cost cache coherency maintenance done by the CPU.
+	 */
+#if defined(CONFIG_ARCH_HAS_SYNC_DMA_FOR_DEVICE) || \
+	defined(CONFIG_ARCH_HAS_SYNC_DMA_FOR_CPU) || \
+	defined(CONFIG_ARCH_HAS_SYNC_DMA_FOR_CPU_ALL)
+
+	if (!of_property_read_bool(np, "dma-coherent"))
+		pr_warn("Assuming cache coherent DMA transactions, no 'dma-coherent' node supplied\n");

Why do you need this property at all, if it is allways dma-coherent? Are you supporting dma-noncoherent somewhere?

It's just a sanity check that the DT is well-formed.

Since the dma-coherent property is interpreted by common code, it's not up to hv to change the default for the platform. I'm not sure if the presence of CONFIG_ARCH_HAS_SYNC_DMA_* options is the correct check to determine that an architecture defaults to noncoherent though, as the function may be needed to do something else.

Quoted 3 lines

The global "dma_default_coherent' may be a better thing to check for. This is e.g. set on powerpc64, riscv and on specific mips platforms, but it's never set on arm64 as far as I can tell.

Kernel's task is not to validate the DT. Even if it was, above code works poor. What if someone adds 'dma-noncoherent'?

The job of the bindings and DT schema is to validate and check DT if is well formed.

Best regards,
Krzysztof

#35 lore raw

Re: [PATCH v3 5/7] dt-bindings: bus: Add Hyper-V VMBus cache coherency and IRQs

In reply to Krzysztof Kozlowski
Quoted 7 lines · Krzysztof Kozlowski
Quoted 3 lines · Roman Kisel

Add dt-bindings for the Hyper-V VMBus DMA cache coherency and interrupt specification.

You did not add any bindings. I don't understand this description.

My bad, extended the example for the interrupt controller node. Will fix in the next version.

Quoted 2 lines

Anyway, not tested.

Understood. I didn't realize that yml goes through testing, thought it was akin to a machine-readable portion of the documentation.

Quoted 5 lines

<form letter>
Please use scripts/get_maintainers.pl to get a list of necessary people and lists to CC. It might happen, that command when run on an older kernel, gives you outdated entries. Therefore please be sure you base your patches on recent Linux kernel.

I used the "cached" version of the get_maintainers.pl output produced for v2. The bad assumption was that adding a yml do Documentation would not affect that. Will fix, appreciate helping me so much in getting this into a good shape!

Quoted 19 lines

Tools like b4 or scripts/get_maintainer.pl provide you proper list of people, so fix your workflow. Tools might also fail if you work on some ancient tree (don't, instead use mainline) or work on fork of kernel (don't, instead use mainline). Just use b4 and everything should be fine, although remember about b4 prep --auto-to-cc if you added new patches to the patchset.

You missed at least devicetree list (maybe more), so this won't be tested by automated tooling. Performing review on untested code might be a waste of time.

Please kindly resend and include all necessary To/Cc entries. </form letter>

Best regards,
Krzysztof

Signature · 2 lines
Thank you,
Roman
#43 lore raw

Re: [PATCH v3 6/7] Drivers: hv: vmbus: Get the IRQ number from DT

In reply to Krzysztof Kozlowski
Quoted 5 lines · Krzysztof Kozlowski
Quoted 4 lines · Roman Kisel
drivers/hv/vmbus_drv.c +0 -0
@@ -2338,6 +2372,21 @@ static int vmbus_device_add(struct platform_device *pdev)
  		cur_res = &res->sibling;
  	}
 
Quoted 14 lines

drivers/hv/vmbus_drv.c static int vmbus_device_add(struct platform_device *pdev) · line 2385

Quoted 11 lines
drivers/hv/vmbus_drv.c +11 -0
+	/*
+	 * Hyper-V always assumes DMA cache coherency, and the DMA subsystem
+	 * might default to 'not coherent' on some architectures.
+	 * Avoid high-cost cache coherency maintenance done by the CPU.
+	 */
+#if defined(CONFIG_ARCH_HAS_SYNC_DMA_FOR_DEVICE) || \
+	defined(CONFIG_ARCH_HAS_SYNC_DMA_FOR_CPU) || \
+	defined(CONFIG_ARCH_HAS_SYNC_DMA_FOR_CPU_ALL)
+
+	if (!of_property_read_bool(np, "dma-coherent"))
+		pr_warn("Assuming cache coherent DMA transactions, no 'dma-coherent' node supplied\n");

Why do you need this property at all, if it is allways dma-coherent? Are you supporting dma-noncoherent somewhere?

No support for non-coherent. Wanted to do a sanity check. Had better check for dma-noncoherent and warn about that I believe.

Quoted 4 lines

Best regards,
Krzysztof

Signature · 2 lines
Thank you,
Roman
#40 lore raw

Re: [PATCH v3 6/7] Drivers: hv: vmbus: Get the IRQ number from DT

In reply to Arnd Bergmann
Quoted 21 lines · Arnd Bergmann
Quoted 5 lines · Krzysztof Kozlowski
Quoted 4 lines · Roman Kisel
drivers/hv/vmbus_drv.c +0 -0
@@ -2338,6 +2372,21 @@ static int vmbus_device_add(struct platform_device *pdev)
  		cur_res = &res->sibling;
  	}
 
Quoted 14 lines

drivers/hv/vmbus_drv.c static int vmbus_device_add(struct platform_device *pdev) · line 2385

Quoted 11 lines
drivers/hv/vmbus_drv.c +11 -0
+	/*
+	 * Hyper-V always assumes DMA cache coherency, and the DMA subsystem
+	 * might default to 'not coherent' on some architectures.
+	 * Avoid high-cost cache coherency maintenance done by the CPU.
+	 */
+#if defined(CONFIG_ARCH_HAS_SYNC_DMA_FOR_DEVICE) || \
+	defined(CONFIG_ARCH_HAS_SYNC_DMA_FOR_CPU) || \
+	defined(CONFIG_ARCH_HAS_SYNC_DMA_FOR_CPU_ALL)
+
+	if (!of_property_read_bool(np, "dma-coherent"))
+		pr_warn("Assuming cache coherent DMA transactions, no 'dma-coherent' node supplied\n");

Why do you need this property at all, if it is allways dma-coherent? Are you supporting dma-noncoherent somewhere?

Quoted 7 lines

It's just a sanity check that the DT is well-formed.

Since the dma-coherent property is interpreted by common code, it's not up to hv to change the default for the platform. I'm not sure if the presence of CONFIG_ARCH_HAS_SYNC_DMA_* options is the correct check to determine that an architecture defaults to noncoherent though, as the function may be needed to do something else.

I used the ifdef as the dma_coherent field is declared under these macros:

#if defined(CONFIG_ARCH_HAS_SYNC_DMA_FOR_DEVICE) || \
	defined(CONFIG_ARCH_HAS_SYNC_DMA_FOR_CPU) || \
	defined(CONFIG_ARCH_HAS_SYNC_DMA_FOR_CPU_ALL)
extern bool dma_default_coherent;
static inline bool dev_is_dma_coherent(struct device *dev)
{
	return dev->dma_coherent;
}

#else
#define dma_default_coherent true

static inline bool dev_is_dma_coherent(struct device *dev)

{
	return true;
}

i.e., there is no API to set dma_coherent. As I see it, the options are either warn the user if they forgot to add dma-coherent

if (!dev_is_dma_coherent(dev)) pr_warn("add dma-coherent to be faster\n"),

or warn and force the flag to true. Maybe just warn
the user I think now... The code will be cleaner (no need to emulate a-would-be set_dma_coherent) , and the user will
know how to make the system perform at its best.

Appreciate sharing the reservations about that piece!

Quoted 6 lines

The global "dma_default_coherent' may be a better thing to check for. This is e.g. set on powerpc64, riscv and on specific mips platforms, but it's never set on arm64 as far as I can tell.

Arnd

Signature · 2 lines
Thank you,
Roman
maintainer
#48 lore raw

Re: [PATCH v3 7/7] PCI: hv: Get vPCI MSI IRQ domain from DT

In reply to Roman Kisel
Quoted 19 lines · Roman Kisel

The hyperv-pci driver uses ACPI for MSI IRQ domain configuration on arm64. It won't be able to do that in the VTL mode where only DeviceTree can be used.

Update the hyperv-pci driver to get vPCI MSI IRQ domain in the DeviceTree case, too.

---
 drivers/hv/vmbus_drv.c              | 23 +++++++-----
 drivers/pci/controller/pci-hyperv.c | 55 +++++++++++++++++++++++++++--
 include/linux/hyperv.h              |  2 ++
 3 files changed, 69 insertions(+), 11 deletions(-)
drivers/hv/vmbus_drv.c +0 -0
index 7eee7caff5f6..038bd9be64b7 100644

drivers/hv/vmbus_drv.c struct vmbus_dynid { · line 49

Quoted 6 lines
drivers/hv/vmbus_drv.c +2 -1
@@ -45,7 +45,8 @@ struct vmbus_dynid {
 	struct hv_vmbus_device_id id;
 };
 
-static struct device  *hv_dev;
+/* VMBus Root Device */
+static struct device  *vmbus_root_device;

You're changing the name of the variable. That should preferably be done in a separate patch. That's going to make this patch shorter and easier to review.

drivers/hv/vmbus_drv.c static struct resource *fb_mmio; · line 88

Quoted 12 lines
drivers/hv/vmbus_drv.c +5 -0
 static int hyperv_cpuhp_online;
 
@@ -80,9 +81,15 @@ static struct resource *fb_mmio;
 static struct resource *hyperv_mmio;
 static DEFINE_MUTEX(hyperv_mmio_lock);
 
+struct device *get_vmbus_root_device(void)
+{
+	return vmbus_root_device;
+}
+EXPORT_SYMBOL_GPL(get_vmbus_root_device);

I would like you to add "hv_" prefix to this exported symbol, or rename it to "vmbus_get_root_device()".

Quoted 82 lines
drivers/hv/vmbus_drv.c +8 -7
+
 static int vmbus_exists(void)
 {
-	if (hv_dev == NULL)
+	if (vmbus_root_device == NULL)
 		return -ENODEV;
 
 	return 0;
@@ -861,7 +868,7 @@ static int vmbus_dma_configure(struct device *child_device)
 	 * On x86/x64 coherence is assumed and these calls have no effect.
 	 */
 	hv_setup_dma_ops(child_device,
-		device_get_dma_attr(hv_dev) == DEV_DMA_COHERENT);
+		device_get_dma_attr(vmbus_root_device) == DEV_DMA_COHERENT);
 	return 0;
 }
 
@@ -1892,7 +1899,7 @@ int vmbus_device_register(struct hv_device *child_device_obj)
 		     &child_device_obj->channel->offermsg.offer.if_instance);
 
 	child_device_obj->device.bus = &hv_bus;
-	child_device_obj->device.parent = hv_dev;
+	child_device_obj->device.parent = vmbus_root_device;
 	child_device_obj->device.release = vmbus_device_release;
 
 	child_device_obj->device.dma_parms = &child_device_obj->dma_parms;
@@ -2253,7 +2260,7 @@ static int vmbus_acpi_add(struct platform_device *pdev)
 	struct acpi_device *ancestor;
 	struct acpi_device *device = ACPI_COMPANION(&pdev->dev);
 
-	hv_dev = &device->dev;
+	vmbus_root_device = &device->dev;
 
 	/*
 	 * Older versions of Hyper-V for ARM64 fail to include the _CCA
@@ -2342,7 +2349,7 @@ static int vmbus_device_add(struct platform_device *pdev)
 	struct device_node *np = pdev->dev.of_node;
 	int ret;
 
-	hv_dev = &pdev->dev;
+	vmbus_root_device = &pdev->dev;
 
 	ret = of_range_parser_init(&parser, np);
 	if (ret)
@@ -2675,7 +2682,7 @@ static int __init hv_acpi_init(void)
 	if (ret)
 		return ret;
 
-	if (!hv_dev) {
+	if (!vmbus_root_device) {
 		ret = -ENODEV;
 		goto cleanup;
 	}
@@ -2706,7 +2713,7 @@ static int __init hv_acpi_init(void)
 
 cleanup:
 	platform_driver_unregister(&vmbus_platform_driver);
-	hv_dev = NULL;
+	vmbus_root_device = NULL;
 	return ret;
 }
 
drivers/pci/controller/pci-hyperv.c +5 -0
index 5992280e8110..cdecefd1f9bd 100644
@@ -50,6 +50,7 @@
 #include <linux/irqdomain.h>
 #include <linux/acpi.h>
 #include <linux/sizes.h>
+#include <linux/of_irq.h>
 #include <asm/mshyperv.h>
 
 /*
@@ -887,6 +888,35 @@ static const struct irq_domain_ops hv_pci_domain_ops = {
 	.activate = hv_pci_vec_irq_domain_activate,
 };
 
+#ifdef CONFIG_OF
+
+static struct irq_domain *hv_pci_of_irq_domain_parent(void)
+{

drivers/pci/controller/pci-hyperv.c static const struct irq_domain_ops hv_pci_domain_ops = { · line 904

Quoted 10 lines
drivers/pci/controller/pci-hyperv.c +10 -0
+	struct device_node *parent;
+	struct irq_domain *domain;
+
+	parent = of_irq_find_parent(to_platform_device(get_vmbus_root_device())->dev.of_node);
+	domain = NULL;
+	if (parent) {
+		domain = irq_find_host(parent);
+		of_node_put(parent);
+	}
+

I cannot really comment on the ARM side of things around how this system is set up. I will leave that to someone who's more familiar with the matter to review.

Thanks,
Wei.

maintainer Acked-by on v3 1/7
#2 lore raw

Re: [PATCH v3 1/7] arm64: hyperv: Use SMC to detect hypervisor presence

In reply to Roman Kisel
Quoted 9 lines · Roman Kisel

The arm64 Hyper-V startup path relies on ACPI to detect running under a Hyper-V compatible hypervisor. That doesn't work on non-ACPI systems.

Hoist the ACPI detection logic into a separate function, use the new SMC added recently to Hyper-V to use in the non-ACPI case.

The change looks sensible.

Within one minor comment fixed below:

However I would also like to get an Acked-by or reivewed-by from someone who works on the ARM64 side of things -- Saurabh, Boqun, Srivatsa, and Jinank?

Quoted 5 lines
---
 arch/arm64/hyperv/mshyperv.c      | 36 ++++++++++++++++++++++++++-----
 arch/arm64/include/asm/mshyperv.h |  5 +++++
 2 files changed, 36 insertions(+), 5 deletions(-)

[...]

arch/arm64/include/asm/mshyperv.h static inline u64 hv_get_msr(unsigned int reg) · line 55

Quoted 10 lines
arch/arm64/include/asm/mshyperv.h +2 -0
index a975e1a689dd..a7a3586f7cb1 100644
@@ -51,4 +51,9 @@ static inline u64 hv_get_msr(unsigned int reg)
 
 #include <asm-generic/mshyperv.h>
 
+#define ARM_SMCCC_VENDOR_HYP_UID_HYPERV_REG_0	0x7948734d
+#define ARM_SMCCC_VENDOR_HYP_UID_HYPERV_REG_1	0x56726570

Presumably these are ASCII codes for an identifier string, please provide comments to explain what they are.

Quoted 7 lines
arch/arm64/include/asm/mshyperv.h +3 -0
+#define ARM_SMCCC_VENDOR_HYP_UID_HYPERV_REG_2	0
+#define ARM_SMCCC_VENDOR_HYP_UID_HYPERV_REG_3	0
+
 #endif
Signature · 1 line
2.34.1
maintainer Acked-by on v3 2/7
#13 lore raw

Re: [PATCH v3 2/7] Drivers: hv: Enable VTL mode for arm64

In reply to Roman Kisel
Quoted 6 lines · Roman Kisel

Kconfig dependencies for arm64 guests on Hyper-V require that be ACPI enabled, and limit VTL mode to x86/x64. To enable VTL mode on arm64 as well, update the dependencies. Since VTL mode requires DeviceTree instead of ACPI, don’t require arm64 guests on Hyper-V to have ACPI.

maintainer
#20 lore raw

Re: [PATCH v3 3/7] Drivers: hv: Provide arch-neutral implementation of get_vtl()

In reply to Roman Kisel
Quoted 9 lines · Roman Kisel

To run in the VTL mode, Hyper-V drivers have to know what VTL the system boots in, and the arm64/hyperv code does not have the means to compute that.

Refactor the code to hoist the function that detects VTL, make it arch-neutral to be able to employ it to get the VTL on arm64. Fix the hypercall output address in get_vtl(void) not to overlap with the hypercall input area to adhere to the Hyper-V TLFS.

Can you split the fix out? That potentially can be backported.

Quoted 6 lines

---

drivers/hv/hv_common.c +3 -0
+
+#if IS_ENABLED(CONFIG_HYPERV_VTL_MODE)
+u8 __init get_vtl(void)

drivers/hv/hv_common.c u64 __weak hv_tdx_hypercall(u64 control, u64 param1, u64 param2) · line 671

Quoted 10 lines
drivers/hv/hv_common.c +10 -0
+{
+	u64 control = HV_HYPERCALL_REP_COMP_1 | HVCALL_GET_VP_REGISTERS;
+	struct hv_get_vp_registers_input *input;
+	struct hv_get_vp_registers_output *output;
+	unsigned long flags;
+	u64 ret;
+
+	local_irq_save(flags);
+	input = *this_cpu_ptr(hyperv_pcpu_input_arg);
+	output = *this_cpu_ptr(hyperv_pcpu_output_arg);

Hmm... I don't remember why the old code didn't use hyperv_pcpu_output_arg but instead reused input+OFFSET as output.

Saurabh, can you comment on this?

Thanks,
Wei.

maintainer
#27 lore raw

Re: [PATCH v3 4/7] arm64: hyperv: Boot in a Virtual Trust Level

In reply to Roman Kisel
Quoted 48 lines · Roman Kisel

To run in the VTL mode, Hyper-V drivers have to know what VTL the system boots in, and the arm64/hyperv code does not update the variable that stores the value.

Update the variable to enable the Hyper-V drivers to boot in the VTL mode and print the VTL the code runs in.

---
 arch/arm64/hyperv/Makefile        |  1 +
 arch/arm64/hyperv/hv_vtl.c        | 13 +++++++++++++
 arch/arm64/hyperv/mshyperv.c      |  4 ++++
 arch/arm64/include/asm/mshyperv.h |  7 +++++++
 4 files changed, 25 insertions(+)
 create mode 100644 arch/arm64/hyperv/hv_vtl.c
arch/arm64/hyperv/Makefile +1 -0
index 87c31c001da9..9701a837a6e1 100644
@@ -1,2 +1,3 @@
 # SPDX-License-Identifier: GPL-2.0
 obj-y		:= hv_core.o mshyperv.o
+obj-$(CONFIG_HYPERV_VTL_MODE)	+= hv_vtl.o
arch/arm64/hyperv/hv_vtl.c new file +13 -0
new file mode 100644
index 000000000000..38642b7b6be0
@@ -0,0 +1,13 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (C) 2024, Microsoft, Inc.
+ *
+ * Author : Roman Kisel <[email protected]>
+ */
+
+#include <asm/mshyperv.h>
+
+void __init hv_vtl_init_platform(void)
+{
+	pr_info("Linux runs in Hyper-V Virtual Trust Level\n");
+}
arch/arm64/hyperv/mshyperv.c +0 -0
index 341f98312667..8fd04d6e4800 100644

arch/arm64/hyperv/mshyperv.c static int __init hyperv_init(void) · line 103

Quoted 6 lines
arch/arm64/hyperv/mshyperv.c +3 -0
@@ -98,6 +98,10 @@ static int __init hyperv_init(void)
 		return ret;
 	}
 
+	/* Find the VTL */
+	ms_hyperv.vtl = get_vtl();
+	hv_vtl_init_platform();

It doesn't make sense to me because this function unconditionally prints Linux runs in Hyper-V Virtual Trust Level.

Thanks,
Wei.

Quoted 25 lines
arch/arm64/hyperv/mshyperv.c +1 -0
+
 	ms_hyperv_late_init();
 
 	hyperv_initialized = true;
arch/arm64/include/asm/mshyperv.h +7 -0
index a7a3586f7cb1..63d6bb6998fc 100644
@@ -49,6 +49,13 @@ static inline u64 hv_get_msr(unsigned int reg)
 				ARM_SMCCC_OWNER_VENDOR_HYP,	\
 				HV_SMCCC_FUNC_NUMBER)
 
+#ifdef CONFIG_HYPERV_VTL_MODE
+void __init hv_vtl_init_platform(void);
+int __init hv_vtl_early_init(void);
+#else
+static inline void __init hv_vtl_init_platform(void) {}
+#endif
+
 #include <asm-generic/mshyperv.h>
 
 #define ARM_SMCCC_VENDOR_HYP_UID_HYPERV_REG_0	0x7948734d
Signature · 1 line
2.34.1
#4 lore raw

RE: [PATCH v3 1/7] arm64: hyperv: Use SMC to detect hypervisor presence

In reply to Roman Kisel

From: Roman Kisel <[email protected]> Sent: Friday, July 26, 2024 3:59 PM

Quoted 8 lines

The arm64 Hyper-V startup path relies on ACPI to detect running under a Hyper-V compatible hypervisor. That doesn't work on non-ACPI systems.

Hoist the ACPI detection logic into a separate function, use the new SMC added recently to Hyper-V to use in the non-ACPI case.

Wording seems slightly messed up. Perhaps:

Hoist the ACPI detection logic into a separate function. Then use the new SMC added recently to Hyper-V in the non-ACPI case.

Also, the phrase "the new SMC" seems a bit off to me. The "Terms and Abbreviations" section of the SMCCC specification defines "SMC" as an instruction:

Secure Monitor Call. An Arm assembler instruction that causes an exception that is taken synchronously into EL3.

More precisely, I think you mean a SMC "function identifier" that is newly implemented by Hyper-V. And the function identifier itself isn't new; it's the Hyper-V implementation that's new.

Similar comment applies in the cover letter for this patch set, and perhaps to the Subject line of this patch.

Quoted 69 lines
---
 arch/arm64/hyperv/mshyperv.c      | 36 ++++++++++++++++++++++++++-----
 arch/arm64/include/asm/mshyperv.h |  5 +++++
 2 files changed, 36 insertions(+), 5 deletions(-)
arch/arm64/hyperv/mshyperv.c +31 -5
index b1a4de4eee29..341f98312667 100644
@@ -27,6 +27,34 @@ int hv_get_hypervisor_version(union hv_hypervisor_version_info *info)
 	return 0;
 }
 
+static bool hyperv_detect_via_acpi(void)
+{
+	if (acpi_disabled)
+		return false;
+#if IS_ENABLED(CONFIG_ACPI)
+	/* Hypervisor ID is only available in ACPI v6+. */
+	if (acpi_gbl_FADT.header.revision < 6)
+		return false;
+	return strncmp((char *)&acpi_gbl_FADT.hypervisor_id, "MsHyperV", 8) == 0;
+#else
+	return false;
+#endif
+}
+
+static bool hyperv_detect_via_smc(void)
+{
+	struct arm_smccc_res res = {};
+
+	if (arm_smccc_1_1_get_conduit() != SMCCC_CONDUIT_HVC)
+		return false;
+	arm_smccc_1_1_hvc(ARM_SMCCC_VENDOR_HYP_CALL_UID_FUNC_ID, &res);
+
+	return res.a0 == ARM_SMCCC_VENDOR_HYP_UID_HYPERV_REG_0 &&
+		res.a1 == ARM_SMCCC_VENDOR_HYP_UID_HYPERV_REG_1 &&
+		res.a2 == ARM_SMCCC_VENDOR_HYP_UID_HYPERV_REG_2 &&
+		res.a3 == ARM_SMCCC_VENDOR_HYP_UID_HYPERV_REG_3;
+}
+
 static int __init hyperv_init(void)
 {
 	struct hv_get_vp_registers_output	result;
@@ -35,13 +63,11 @@ static int __init hyperv_init(void)
 
 	/*
 	 * Allow for a kernel built with CONFIG_HYPERV to be running in
-	 * a non-Hyper-V environment, including on DT instead of ACPI.
+	 * a non-Hyper-V environment.
+	 *
 	 * In such cases, do nothing and return success.
 	 */
-	if (acpi_disabled)
-		return 0;
-
-	if (strncmp((char *)&acpi_gbl_FADT.hypervisor_id, "MsHyperV", 8))
+	if (!hyperv_detect_via_acpi() && !hyperv_detect_via_smc())
 		return 0;
 
 	/* Setup the guest ID */
arch/arm64/include/asm/mshyperv.h +0 -0
index a975e1a689dd..a7a3586f7cb1 100644

arch/arm64/include/asm/mshyperv.h static inline u64 hv_get_msr(unsigned int reg) · line 58

Quoted 8 lines
arch/arm64/include/asm/mshyperv.h +5 -0
@@ -51,4 +51,9 @@ static inline u64 hv_get_msr(unsigned int reg)
 
 #include <asm-generic/mshyperv.h>
 
+#define ARM_SMCCC_VENDOR_HYP_UID_HYPERV_REG_0	0x7948734d
+#define ARM_SMCCC_VENDOR_HYP_UID_HYPERV_REG_1	0x56726570
+#define ARM_SMCCC_VENDOR_HYP_UID_HYPERV_REG_2	0
+#define ARM_SMCCC_VENDOR_HYP_UID_HYPERV_REG_3	0
+

Section 6.2 of the SMCCC specification says that the "Call UID Query" returns a UUID. The above #defines look like an ASCII string is being returned. Arguably the ASCII string can be treated as a set of 128 bits just like a UUID, but it doesn't meet the spirit of the spec. Can Hyper-V be changed to return a real UUID? While the distinction probably won't make a material difference here, we've had problems in the past with Hyper-V doing slightly weird things that later caused unexpected trouble. Please just get it right. :-)

Michael

Quoted 4 lines
arch/arm64/include/asm/mshyperv.h +0 -1
 #endif
--

2.34.1

#14 lore raw

RE: [PATCH v3 2/7] Drivers: hv: Enable VTL mode for arm64

In reply to Roman Kisel

From: Roman Kisel <[email protected]> Sent: Friday, July 26, 2024 3:59 PM

Quoted 43 lines

Kconfig dependencies for arm64 guests on Hyper-V require that be ACPI enabled, and limit VTL mode to x86/x64. To enable VTL mode on arm64 as well, update the dependencies. Since VTL mode requires DeviceTree instead of ACPI, don't require arm64 guests on Hyper-V to have ACPI.

---
 drivers/hv/Kconfig | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
drivers/hv/Kconfig +3 -4
index 862c47b191af..a5cd1365e248 100644
@@ -5,7 +5,7 @@ menu "Microsoft Hyper-V guest support"
 config HYPERV
 	tristate "Microsoft Hyper-V client drivers"
 	depends on (X86 && X86_LOCAL_APIC && HYPERVISOR_GUEST) \
-		|| (ACPI && ARM64 && !CPU_BIG_ENDIAN)
+		|| (ARM64 && !CPU_BIG_ENDIAN)
 	select PARAVIRT
 	select X86_HV_CALLBACK_VECTOR if X86
 	select OF_EARLY_FLATTREE if OF
@@ -15,7 +15,7 @@ config HYPERV
 
 config HYPERV_VTL_MODE
 	bool "Enable Linux to boot in VTL context"
-	depends on X86_64 && HYPERV
+	depends on HYPERV
 	depends on SMP
 	default n
 	help
@@ -31,7 +31,7 @@ config HYPERV_VTL_MODE
 
 	  Select this option to build a Linux kernel to run at a VTL other than
 	  the normal VTL0, which currently is only VTL2.  This option
-	  initializes the x86 platform for VTL2, and adds the ability to boot
+	  initializes the kernel to run in VTL2, and adds the ability to boot
 	  secondary CPUs directly into 64-bit context as required for VTLs other
 	  than 0.  A kernel built with this option must run at VTL2, and will
 	  not run as a normal guest.
--
Quoted 2 lines

2.34.1

In v2 of this patch, I suggested [1] making a couple additional minor changes so that kernels built *without* HYPER_VTL_MODE would still require ACPI. Did that suggestion not work out? If that's the case, I'm curious about what goes wrong.

Michael

[1] https://lore.kernel.org/all/SN6PR02MB4157E15EFE263BBA3D8DFC51D4EC2@SN6PR02MB4157.namprd02.prod.outlook.com/

#22 lore raw

RE: [PATCH v3 3/7] Drivers: hv: Provide arch-neutral implementation of get_vtl()

In reply to Roman Kisel

From: Roman Kisel <[email protected]> Sent: Friday, July 26, 2024 3:59 PM

Quoted 106 lines

To run in the VTL mode, Hyper-V drivers have to know what VTL the system boots in, and the arm64/hyperv code does not have the means to compute that.

Refactor the code to hoist the function that detects VTL, make it arch-neutral to be able to employ it to get the VTL on arm64. Fix the hypercall output address in get_vtl(void) not to overlap with the hypercall input area to adhere to the Hyper-V TLFS.

---
 arch/x86/hyperv/hv_init.c          | 34 ---------------------
 arch/x86/include/asm/hyperv-tlfs.h |  7 -----
 drivers/hv/hv_common.c             | 47 ++++++++++++++++++++++++++++--
 include/asm-generic/hyperv-tlfs.h  |  7 +++++
 include/asm-generic/mshyperv.h     |  6 ++++
 5 files changed, 58 insertions(+), 43 deletions(-)
arch/x86/hyperv/hv_init.c +0 -34
index 17a71e92a343..c350fa05ee59 100644
@@ -413,40 +413,6 @@ static void __init hv_get_partition_id(void)
 	local_irq_restore(flags);
 }
 
-#if IS_ENABLED(CONFIG_HYPERV_VTL_MODE)
-static u8 __init get_vtl(void)
-{
-	u64 control = HV_HYPERCALL_REP_COMP_1 | HVCALL_GET_VP_REGISTERS;
-	struct hv_get_vp_registers_input *input;
-	struct hv_get_vp_registers_output *output;
-	unsigned long flags;
-	u64 ret;
-
-	local_irq_save(flags);
-	input = *this_cpu_ptr(hyperv_pcpu_input_arg);
-	output = (struct hv_get_vp_registers_output *)input;
-
-	memset(input, 0, struct_size(input, element, 1));
-	input->header.partitionid = HV_PARTITION_ID_SELF;
-	input->header.vpindex = HV_VP_INDEX_SELF;
-	input->header.inputvtl = 0;
-	input->element[0].name0 = HV_X64_REGISTER_VSM_VP_STATUS;
-
-	ret = hv_do_hypercall(control, input, output);
-	if (hv_result_success(ret)) {
-		ret = output->as64.low & HV_X64_VTL_MASK;
-	} else {
-		pr_err("Failed to get VTL(error: %lld) exiting...\n", ret);
-		BUG();
-	}
-
-	local_irq_restore(flags);
-	return ret;
-}
-#else
-static inline u8 get_vtl(void) { return 0; }
-#endif
-
 /*
  * This function is to be invoked early in the boot sequence after the
  * hypervisor has been detected.
arch/x86/include/asm/hyperv-tlfs.h +0 -7
index 3787d26810c1..9ee68eb8e6ff 100644
@@ -309,13 +309,6 @@ enum hv_isolation_type {
 #define HV_MSR_STIMER0_CONFIG	(HV_X64_MSR_STIMER0_CONFIG)
 #define HV_MSR_STIMER0_COUNT	(HV_X64_MSR_STIMER0_COUNT)
 
-/*
- * Registers are only accessible via HVCALL_GET_VP_REGISTERS hvcall and
- * there is not associated MSR address.
- */
-#define	HV_X64_REGISTER_VSM_VP_STATUS	0x000D0003
-#define	HV_X64_VTL_MASK			GENMASK(3, 0)
-
 /* Hyper-V memory host visibility */
 enum hv_mem_host_visibility {
 	VMBUS_PAGE_NOT_VISIBLE		= 0,
drivers/hv/hv_common.c +5 -2
index 9c452bfbd571..7d6c1523b0b5 100644
@@ -339,8 +339,8 @@ int __init hv_common_init(void)
 	hyperv_pcpu_input_arg = alloc_percpu(void  *);
 	BUG_ON(!hyperv_pcpu_input_arg);
 
-	/* Allocate the per-CPU state for output arg for root */
-	if (hv_root_partition) {
+	/* Allocate the per-CPU state for output arg for root or a VTL */
+	if (hv_root_partition || IS_ENABLED(CONFIG_HYPERV_VTL_MODE)) {
 		hyperv_pcpu_output_arg = alloc_percpu(void *);
 		BUG_ON(!hyperv_pcpu_output_arg);
 	}
@@ -656,3 +656,46 @@ u64 __weak hv_tdx_hypercall(u64 control, u64 param1, u64
 param2)
 	return HV_STATUS_INVALID_PARAMETER;
 }
 EXPORT_SYMBOL_GPL(hv_tdx_hypercall);
+
+#if IS_ENABLED(CONFIG_HYPERV_VTL_MODE)
+u8 __init get_vtl(void)

drivers/hv/hv_common.c u64 __weak hv_tdx_hypercall(u64 control, u64 param1, u64 · line 671

Quoted 10 lines
drivers/hv/hv_common.c +10 -0
+{
+	u64 control = HV_HYPERCALL_REP_COMP_1 | HVCALL_GET_VP_REGISTERS;
+	struct hv_get_vp_registers_input *input;
+	struct hv_get_vp_registers_output *output;
+	unsigned long flags;
+	u64 ret;
+
+	local_irq_save(flags);
+	input = *this_cpu_ptr(hyperv_pcpu_input_arg);
+	output = *this_cpu_ptr(hyperv_pcpu_output_arg);

Rather than use the hyperv_pcpu_output_arg here, it's OK to use a different area of the hyperv_pcpu_input_arg page. For example,

	output = (void *)input + HV_HYP_PAGE_SIZE/2;

The TLFS does not require that the input and output be in separate pages.

While using the hyperv_pcpu_output_arg is conceptually a bit cleaner, doing so requires allocating a 4K page per CPU that is not otherwise used. The VTL 2 code wants to be frugal with memory, and this seems like a good step in that direction. :-)

The hyperv_pcpu_output_arg was added for the cases where up to a full page is needed for input and output on the same hypercall. So far, the only case of that is when running in the root partition.

Quoted 35 lines
drivers/hv/hv_common.c +30 -0
+
+	memset(input, 0, struct_size(input, element, 1));
+	input->header.partitionid = HV_PARTITION_ID_SELF;
+	input->header.vpindex = HV_VP_INDEX_SELF;
+	input->header.inputvtl = 0;
+	input->element[0].name0 = HV_REGISTER_VSM_VP_STATUS;
+
+	ret = hv_do_hypercall(control, input, output);
+	if (hv_result_success(ret)) {
+		ret = output->as64.low & HV_VTL_MASK;
+	} else {
+		pr_err("Failed to get VTL(error: %lld) exiting...\n", ret);
+
+		/*
+		 * This is a dead end, something fundamental is broken.
+		 *
+		 * There is no sensible way of continuing as the Hyper-V drivers
+		 * transitively depend via the vmbus driver on knowing which VTL
+		 * they run in to establish communication with the host. The kernel
+		 * is going to be worse off if continued booting than a panicked one,
+		 * just hung and stuck, producing second-order failures, with neither
+		 * a way to recover nor to provide expected services.
+		 */
+		BUG();
+	}
+
+	local_irq_restore(flags);
+	return ret;
+}
+#endif
include/asm-generic/hyperv-tlfs.h +0 -0
index 814207e7c37f..271c365973d6 100644

include/asm-generic/hyperv-tlfs.h line 80

Quoted 6 lines
include/asm-generic/hyperv-tlfs.h +3 -0
@@ -75,6 +75,13 @@
 /* AccessTscInvariantControls privilege */
 #define HV_ACCESS_TSC_INVARIANT			BIT(15)
 
+/*
+ * This synthetic register is only accessible via the HVCALL_GET_VP_REGISTERS
+ * hvcall, and there is no an associated MSR on x86.

s/there is no an associated/there is no associated/

include/asm-generic/hyperv-tlfs.h line 83

Quoted 3 lines
include/asm-generic/hyperv-tlfs.h +3 -0
+ */
+#define	HV_REGISTER_VSM_VP_STATUS	0x000D0003
+#define	HV_VTL_MASK			GENMASK(3, 0)

Further down in hyperv-tlfs.h is a section devoted to register definitions. It seems like this definition should go there in numeric order, which is after HV_REGISTER_STIMER0_COUNT.

Michael

Quoted 23 lines
include/asm-generic/hyperv-tlfs.h +1 -0
+
 /*
  * Group B features.
  */
include/asm-generic/mshyperv.h +6 -0
index 8fe7aaab2599..85a5b8cb1702 100644
@@ -315,4 +315,10 @@ static inline enum hv_isolation_type
 hv_get_isolation_type(void)
 }
 #endif /* CONFIG_HYPERV */
 
+#if IS_ENABLED(CONFIG_HYPERV_VTL_MODE)
+u8 __init get_vtl(void);
+#else
+static inline u8 get_vtl(void) { return 0; }
+#endif
+

#endif
--
2.34.1

#29 lore raw

RE: [PATCH v3 4/7] arm64: hyperv: Boot in a Virtual Trust Level

In reply to Roman Kisel

From: Roman Kisel <[email protected]> Sent: Friday, July 26, 2024 3:59 PM

Quoted 65 lines

To run in the VTL mode, Hyper-V drivers have to know what VTL the system boots in, and the arm64/hyperv code does not update the variable that stores the value.

Update the variable to enable the Hyper-V drivers to boot in the VTL mode and print the VTL the code runs in.

---
 arch/arm64/hyperv/Makefile        |  1 +
 arch/arm64/hyperv/hv_vtl.c        | 13 +++++++++++++
 arch/arm64/hyperv/mshyperv.c      |  4 ++++
 arch/arm64/include/asm/mshyperv.h |  7 +++++++
 4 files changed, 25 insertions(+)
 create mode 100644 arch/arm64/hyperv/hv_vtl.c
arch/arm64/hyperv/Makefile +1 -0
index 87c31c001da9..9701a837a6e1 100644
@@ -1,2 +1,3 @@
 # SPDX-License-Identifier: GPL-2.0
 obj-y		:= hv_core.o mshyperv.o
+obj-$(CONFIG_HYPERV_VTL_MODE)	+= hv_vtl.o
arch/arm64/hyperv/hv_vtl.c new file +13 -0
new file mode 100644
index 000000000000..38642b7b6be0
@@ -0,0 +1,13 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (C) 2024, Microsoft, Inc.
+ *
+ * Author : Roman Kisel <[email protected]>
+ */
+
+#include <asm/mshyperv.h>
+
+void __init hv_vtl_init_platform(void)
+{
+	pr_info("Linux runs in Hyper-V Virtual Trust Level\n");
+}
arch/arm64/hyperv/mshyperv.c +4 -0
index 341f98312667..8fd04d6e4800 100644
@@ -98,6 +98,10 @@ static int __init hyperv_init(void)
 		return ret;
 	}
 
+	/* Find the VTL */
+	ms_hyperv.vtl = get_vtl();
+	hv_vtl_init_platform();
+
 	ms_hyperv_late_init();
 
 	hyperv_initialized = true;
arch/arm64/include/asm/mshyperv.h +0 -0
index a7a3586f7cb1..63d6bb6998fc 100644

arch/arm64/include/asm/mshyperv.h static inline u64 hv_get_msr(unsigned int reg) · line 54

Quoted 6 lines
arch/arm64/include/asm/mshyperv.h +3 -0
@@ -49,6 +49,13 @@ static inline u64 hv_get_msr(unsigned int reg)
 				ARM_SMCCC_OWNER_VENDOR_HYP,	\
 				HV_SMCCC_FUNC_NUMBER)
 
+#ifdef CONFIG_HYPERV_VTL_MODE
+void __init hv_vtl_init_platform(void);
+int __init hv_vtl_early_init(void);

This declaration is spurious since you removed the function.

Michael

Quoted 10 lines
arch/arm64/include/asm/mshyperv.h +4 -1
+#else
+static inline void __init hv_vtl_init_platform(void) {}
+#endif
+
 #include <asm-generic/mshyperv.h>
 
 #define ARM_SMCCC_VENDOR_HYP_UID_HYPERV_REG_0	0x7948734d
--

2.34.1

#41 lore raw

RE: [PATCH v3 6/7] Drivers: hv: vmbus: Get the IRQ number from DT

In reply to Roman Kisel

From: Roman Kisel <[email protected]> Sent: Monday, July 29, 2024 9:51 AM

Quoted 24 lines · Arnd Bergmann
Quoted 21 lines · Krzysztof Kozlowski
Quoted 19 lines · Roman Kisel
Quoted 5 lines
drivers/hv/vmbus_drv.c +1 -0
@@ -2338,6 +2372,21 @@ static int vmbus_device_add(struct platform_device *pdev)
  		cur_res = &res->sibling;
  	}
 
+	/*

drivers/hv/vmbus_drv.c static int vmbus_device_add(struct platform_device *pdev) · line 2385

Quoted 10 lines
drivers/hv/vmbus_drv.c +10 -0
+	 * Hyper-V always assumes DMA cache coherency, and the DMA subsystem
+	 * might default to 'not coherent' on some architectures.
+	 * Avoid high-cost cache coherency maintenance done by the CPU.
+	 */
+#if defined(CONFIG_ARCH_HAS_SYNC_DMA_FOR_DEVICE) || \
+	defined(CONFIG_ARCH_HAS_SYNC_DMA_FOR_CPU) || \
+	defined(CONFIG_ARCH_HAS_SYNC_DMA_FOR_CPU_ALL)
+
+	if (!of_property_read_bool(np, "dma-coherent"))
+		pr_warn("Assuming cache coherent DMA transactions, no 'dma-coherent' node supplied\n");

Why do you need this property at all, if it is allways dma-coherent? Are you supporting dma-noncoherent somewhere?

Quoted 1 line
Quoted 1 line

It's just a sanity check that the DT is well-formed.

In my view, this chunk of code can be dropped entirely. The guest should believe what the Hyper-V host tells it via DT, and that includes operating in non-coherent mode. There might be some future case where non-coherent DMA is correct. In such a case, we don't want to have to come back and remove an overly aggressive sanity test from Linux kernel code.

As Arnd noted, the dma-coherent (or dma-noncoherent) property should be interpreted and applied to the device by common code. If that's not working for some reason in this case, we should investigate why not.

Note that the ACPI code for VMBus does the same thing -- it believes and uses whatever the _CCA property says. The exception is that there are deployed version of Hyper-V that don't set _CCA at all, contrary to the ACPI spec. So there's a hack in vmbus_acpi_add() to work around this case and force coherent_dma. But that's the only place where the current Hyper-V assumption of coherence comes into play. I sincerely hope Hyper-V ensures that the DT correctly includes dma-coherent from the start, and that we don't have to replicate the hack on the DT side.

Michael

Quoted 47 lines
Quoted 6 lines

Since the dma-coherent property is interpreted by common code, it's not up to hv to change the default for the platform. I'm not sure if the presence of CONFIG_ARCH_HAS_SYNC_DMA_* options is the correct check to determine that an architecture defaults to noncoherent though, as the function may be needed to do something else.

I used the ifdef as the dma_coherent field is declared under these macros:

#if defined(CONFIG_ARCH_HAS_SYNC_DMA_FOR_DEVICE) || \
	defined(CONFIG_ARCH_HAS_SYNC_DMA_FOR_CPU) || \
	defined(CONFIG_ARCH_HAS_SYNC_DMA_FOR_CPU_ALL)
extern bool dma_default_coherent;
static inline bool dev_is_dma_coherent(struct device *dev)
{
	return dev->dma_coherent;
}

#else
#define dma_default_coherent true

static inline bool dev_is_dma_coherent(struct device *dev)

{
	return true;
}

i.e., there is no API to set dma_coherent. As I see it, the options are either warn the user if they forgot to add dma-coherent

if (!dev_is_dma_coherent(dev)) pr_warn("add dma-coherent to be faster\n"),

or warn and force the flag to true. Maybe just warn
the user I think now... The code will be cleaner (no need to emulate a-would-be set_dma_coherent) , and the user will
know how to make the system perform at its best.

Appreciate sharing the reservations about that piece!

Quoted 6 lines

The global "dma_default_coherent' may be a better thing to check for. This is e.g. set on powerpc64, riscv and on specific mips platforms, but it's never set on arm64 as far as I can tell.

Arnd

--
Thank you,
Roman

#51 lore raw

RE: [PATCH v3 7/7] PCI: hv: Get vPCI MSI IRQ domain from DT

In reply to Roman Kisel

From: Roman Kisel <[email protected]> Sent: Friday, July 26, 2024 3:59 PM

Quoted 9 lines

The hyperv-pci driver uses ACPI for MSI IRQ domain configuration on arm64. It won't be able to do that in the VTL mode where only DeviceTree can be used.

Update the hyperv-pci driver to get vPCI MSI IRQ domain in the DeviceTree case, too.

Overall, this makes sense to me, and I think it works. As you noted in the cover letter for the patch series, it's a bit messy. But see my two comments below.

Quoted 105 lines
---
 drivers/hv/vmbus_drv.c              | 23 +++++++-----
 drivers/pci/controller/pci-hyperv.c | 55 +++++++++++++++++++++++++++--
 include/linux/hyperv.h              |  2 ++
 3 files changed, 69 insertions(+), 11 deletions(-)
drivers/hv/vmbus_drv.c +15 -8
index 7eee7caff5f6..038bd9be64b7 100644
@@ -45,7 +45,8 @@ struct vmbus_dynid {
 	struct hv_vmbus_device_id id;
 };
 
-static struct device  *hv_dev;
+/* VMBus Root Device */
+static struct device  *vmbus_root_device;
 
 static int hyperv_cpuhp_online;
 
@@ -80,9 +81,15 @@ static struct resource *fb_mmio;
 static struct resource *hyperv_mmio;
 static DEFINE_MUTEX(hyperv_mmio_lock);
 
+struct device *get_vmbus_root_device(void)
+{
+	return vmbus_root_device;
+}
+EXPORT_SYMBOL_GPL(get_vmbus_root_device);
+
 static int vmbus_exists(void)
 {
-	if (hv_dev == NULL)
+	if (vmbus_root_device == NULL)
 		return -ENODEV;
 
 	return 0;
@@ -861,7 +868,7 @@ static int vmbus_dma_configure(struct device *child_device)
 	 * On x86/x64 coherence is assumed and these calls have no effect.
 	 */
 	hv_setup_dma_ops(child_device,
-		device_get_dma_attr(hv_dev) == DEV_DMA_COHERENT);
+		device_get_dma_attr(vmbus_root_device) == DEV_DMA_COHERENT);
 	return 0;
 }
 
@@ -1892,7 +1899,7 @@ int vmbus_device_register(struct hv_device *child_device_obj)
 		     &child_device_obj->channel->offermsg.offer.if_instance);
 
 	child_device_obj->device.bus = &hv_bus;
-	child_device_obj->device.parent = hv_dev;
+	child_device_obj->device.parent = vmbus_root_device;
 	child_device_obj->device.release = vmbus_device_release;
 
 	child_device_obj->device.dma_parms = &child_device_obj->dma_parms;
@@ -2253,7 +2260,7 @@ static int vmbus_acpi_add(struct platform_device *pdev)
 	struct acpi_device *ancestor;
 	struct acpi_device *device = ACPI_COMPANION(&pdev->dev);
 
-	hv_dev = &device->dev;
+	vmbus_root_device = &device->dev;
 
 	/*
 	 * Older versions of Hyper-V for ARM64 fail to include the _CCA
@@ -2342,7 +2349,7 @@ static int vmbus_device_add(struct platform_device *pdev)
 	struct device_node *np = pdev->dev.of_node;
 	int ret;
 
-	hv_dev = &pdev->dev;
+	vmbus_root_device = &pdev->dev;
 
 	ret = of_range_parser_init(&parser, np);
 	if (ret)
@@ -2675,7 +2682,7 @@ static int __init hv_acpi_init(void)
 	if (ret)
 		return ret;
 
-	if (!hv_dev) {
+	if (!vmbus_root_device) {
 		ret = -ENODEV;
 		goto cleanup;
 	}
@@ -2706,7 +2713,7 @@ static int __init hv_acpi_init(void)
 
 cleanup:
 	platform_driver_unregister(&vmbus_platform_driver);
-	hv_dev = NULL;
+	vmbus_root_device = NULL;
 	return ret;
 }
 
drivers/pci/controller/pci-hyperv.c +1 -0
index 5992280e8110..cdecefd1f9bd 100644
@@ -50,6 +50,7 @@
 #include <linux/irqdomain.h>
 #include <linux/acpi.h>
 #include <linux/sizes.h>
+#include <linux/of_irq.h>
 #include <asm/mshyperv.h>
 
 /*
@@ -887,6 +888,35 @@ static const struct irq_domain_ops hv_pci_domain_ops = {
 	.activate = hv_pci_vec_irq_domain_activate,

drivers/pci/controller/pci-hyperv.c static const struct irq_domain_ops hv_pci_domain_ops = { · line 898

Quoted 10 lines
drivers/pci/controller/pci-hyperv.c +8 -0
 };
 
+#ifdef CONFIG_OF
+
+static struct irq_domain *hv_pci_of_irq_domain_parent(void)
+{
+	struct device_node *parent;
+	struct irq_domain *domain;
+
+	parent = of_irq_find_parent(to_platform_device(get_vmbus_root_device())->dev.of_node);

I think the above can be simplified to:

	parent = of_irq_find_parent(get_vmbus_root_device()->of_node);

Converting the vmbus_root_device to the platform device, and then accessing the "dev" field of the platform device puts you back where you started with the vmbus_root_device. See the code in vmbus_device_add() where the vmbus_root_device is set to the dev field of the platform device.

Quoted 8 lines
drivers/pci/controller/pci-hyperv.c +8 -0
+	domain = NULL;
+	if (parent) {
+		domain = irq_find_host(parent);
+		of_node_put(parent);
+	}
+
+	/*
+	 * `domain == NULL` shouldn't happen.

drivers/pci/controller/pci-hyperv.c static const struct irq_domain_ops hv_pci_domain_ops = { · line 916

Quoted 10 lines
drivers/pci/controller/pci-hyperv.c +10 -0
+	 *
+	 * If somehow the code does end up in that state, treat this as a configuration
+	 * issue rather than a hard error, emit a warning, and let the code proceed.
+	 * The NULL parent domain is an acceptable option for the `irq_domain_create_hierarchy`
+	 * function called later.
+	 */
+	if (!domain)
+		WARN_ONCE(1, "No interrupt-parent found, check the DeviceTree data.\n");
+	return domain;
+}

Here's a thought, which may or may not be a good one: Push some or all the functionality of hv_pci_of_irq_domain_parent() into vmbus_device_add(). In the simplest case, have vmbus_device_add() store the of_node (which is already the "np" local variable) in a global static variable, and provide hv_get_vmbus_of_node() instead of get_vmbus_root_device().

The next step to consider would be to compute the parent in vmbus_device_add(), and provide hv_get_vmbus_parent_of_node() instead of hv_get_vmbus_of_node(). One difference is that vmbus_device_add() runs for both x86 and arm64, while hv_pci_of_irq_domain_parent() is for arm64 only. The parent node may not exist on x86, but maybe that isn't really a problem.

Pushing everything into vmbus_device_add() would entail doing the irq_find_host(parent) as well, and the accessor function would just return the IRQ domain. In that case, hv_pci_of_irq_domain_parent() can go away. The domain might be NULL on x86, but that's OK because the accessor function won't be called on x86.

Maybe there's a snag that prevents this idea from working well, particularly on x86 where the domain will be NULL. But if it works, it seems slightly less messy to me, though that is a judgment call. I'll leave it to you to decide, and I'm OK either way. :-)

Michael

Quoted 55 lines
drivers/pci/controller/pci-hyperv.c +25 -3
+
+#endif
+
 static int hv_pci_irqchip_init(void)
 {
 	static struct hv_pci_chip_data *chip_data;
@@ -906,10 +936,29 @@ static int hv_pci_irqchip_init(void)
 	 * IRQ domain once enabled, should not be removed since there is no
 	 * way to ensure that all the corresponding devices are also gone and
 	 * no interrupts will be generated.
+	 *
+	 * In the ACPI case, the parent IRQ domain is supplied by the ACPI
+	 * subsystem, and it is the default GSI domain pointing to the GIC.
+	 * Neither is available outside of the ACPI subsystem, cannot avoid
+	 * the messy ifdef below.
+	 * There is apparently no such default in the OF subsystem, and
+	 * `hv_pci_of_irq_domain_parent` finds the parent IRQ domain that
+	 * points to the GIC as well.
+	 * None of these two cases reaches for the MSI parent domain.
 	 */
-	hv_msi_gic_irq_domain = acpi_irq_create_hierarchy(0, HV_PCI_MSI_SPI_NR,
-							  fn, &hv_pci_domain_ops,
-							  chip_data);
+#ifdef CONFIG_ACPI
+	if (!acpi_disabled)
+		hv_msi_gic_irq_domain = acpi_irq_create_hierarchy(0, HV_PCI_MSI_SPI_NR,
+			fn, &hv_pci_domain_ops,
+			chip_data);
+#endif
+#if defined(CONFIG_OF)
+	if (!hv_msi_gic_irq_domain)
+		hv_msi_gic_irq_domain = irq_domain_create_hierarchy(
+			hv_pci_of_irq_domain_parent(), 0, HV_PCI_MSI_SPI_NR,
+			fn, &hv_pci_domain_ops,
+			chip_data);
+#endif
 
 	if (!hv_msi_gic_irq_domain) {
 		pr_err("Failed to create Hyper-V arm64 vPCI MSI IRQ domain\n");
include/linux/hyperv.h +2 -1
index 5e39baa7f6cb..b4aa1f579a97 100644
@@ -1346,6 +1346,8 @@ static inline void *hv_get_drvdata(struct hv_device *dev)
 	return dev_get_drvdata(&dev->device);
 }
 
+struct device *get_vmbus_root_device(void);
+
 struct hv_ring_buffer_debug_info {
 	u32 current_interrupt_mask;
 	u32 current_read_index;
--

2.34.1

#8 lore raw

Re: [PATCH v3 1/7] arm64: hyperv: Use SMC to detect hypervisor presence

In reply to Roman Kisel
Quoted 39 lines · Roman Kisel

The arm64 Hyper-V startup path relies on ACPI to detect running under a Hyper-V compatible hypervisor. That doesn't work on non-ACPI systems.

Hoist the ACPI detection logic into a separate function, use the new SMC added recently to Hyper-V to use in the non-ACPI case.

---
 arch/arm64/hyperv/mshyperv.c      | 36 ++++++++++++++++++++++++++-----
 arch/arm64/include/asm/mshyperv.h |  5 +++++
 2 files changed, 36 insertions(+), 5 deletions(-)
arch/arm64/hyperv/mshyperv.c +17 -0
index b1a4de4eee29..341f98312667 100644
@@ -27,6 +27,34 @@ int hv_get_hypervisor_version(union hv_hypervisor_version_info *info)
 	return 0;
 }
 
+static bool hyperv_detect_via_acpi(void)
+{
+	if (acpi_disabled)
+		return false;
+#if IS_ENABLED(CONFIG_ACPI)
+	/* Hypervisor ID is only available in ACPI v6+. */
+	if (acpi_gbl_FADT.header.revision < 6)
+		return false;
+	return strncmp((char *)&acpi_gbl_FADT.hypervisor_id, "MsHyperV", 8) == 0;
+#else
+	return false;
+#endif
+}
+
+static bool hyperv_detect_via_smc(void)
+{
+	struct arm_smccc_res res = {};

arch/arm64/hyperv/mshyperv.c int hv_get_hypervisor_version(union hv_hypervisor_version_info *info) · line 56

Quoted 10 lines
arch/arm64/hyperv/mshyperv.c +10 -0
+
+	if (arm_smccc_1_1_get_conduit() != SMCCC_CONDUIT_HVC)
+		return false;
+	arm_smccc_1_1_hvc(ARM_SMCCC_VENDOR_HYP_CALL_UID_FUNC_ID, &res);
+
+	return res.a0 == ARM_SMCCC_VENDOR_HYP_UID_HYPERV_REG_0 &&
+		res.a1 == ARM_SMCCC_VENDOR_HYP_UID_HYPERV_REG_1 &&
+		res.a2 == ARM_SMCCC_VENDOR_HYP_UID_HYPERV_REG_2 &&
+		res.a3 == ARM_SMCCC_VENDOR_HYP_UID_HYPERV_REG_3;
+}

As you mentioned in the cover letter this is supported in latest Hyper-V hypervisor, can we add a comment about it, specifying the exact version in it would be great.

If someone attempts to build non-ACPI kernel on older Hyper-V what is the behaviour of this function, do we need to safeguard or handle that case ?

- Saurabh

#15 lore raw

Re: [PATCH v3 2/7] Drivers: hv: Enable VTL mode for arm64

In reply to Michael Kelley
Quoted 36 lines · Michael Kelley

From: Roman Kisel <[email protected]> Sent: Friday, July 26, 2024 3:59 PM

Quoted 35 lines

Kconfig dependencies for arm64 guests on Hyper-V require that be ACPI enabled, and limit VTL mode to x86/x64. To enable VTL mode on arm64 as well, update the dependencies. Since VTL mode requires DeviceTree instead of ACPI, don't require arm64 guests on Hyper-V to have ACPI.

---
 drivers/hv/Kconfig | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
drivers/hv/Kconfig +2 -2
index 862c47b191af..a5cd1365e248 100644
@@ -5,7 +5,7 @@ menu "Microsoft Hyper-V guest support"
 config HYPERV
 	tristate "Microsoft Hyper-V client drivers"
 	depends on (X86 && X86_LOCAL_APIC && HYPERVISOR_GUEST) \
-		|| (ACPI && ARM64 && !CPU_BIG_ENDIAN)
+		|| (ARM64 && !CPU_BIG_ENDIAN)
 	select PARAVIRT
 	select X86_HV_CALLBACK_VECTOR if X86
 	select OF_EARLY_FLATTREE if OF
@@ -15,7 +15,7 @@ config HYPERV
 
 config HYPERV_VTL_MODE
 	bool "Enable Linux to boot in VTL context"
-	depends on X86_64 && HYPERV
+	depends on HYPERV
 	depends on SMP
 	default n
 	help
@@ -31,7 +31,7 @@ config HYPERV_VTL_MODE
 
Quoted 15 lines
Quoted 10 lines
drivers/hv/Kconfig +1 -2
 	  Select this option to build a Linux kernel to run at a VTL other than
 	  the normal VTL0, which currently is only VTL2.  This option
-	  initializes the x86 platform for VTL2, and adds the ability to boot
+	  initializes the kernel to run in VTL2, and adds the ability to boot
 	  secondary CPUs directly into 64-bit context as required for VTLs other
 	  than 0.  A kernel built with this option must run at VTL2, and will
 	  not run as a normal guest.
--

2.34.1

In v2 of this patch, I suggested [1] making a couple additional minor changes so that kernels built *without* HYPER_VTL_MODE would still require ACPI. Did that suggestion not work out? If that's the case, I'm curious about what goes wrong.

Hi Michael/Roman,
I was considering making HYPERV_VTL_MODE depend on CONFIG_OF. That should address above concern as well. Do you see any potential issue with it.

- Saurabh

#21 lore raw

Re: [PATCH v3 3/7] Drivers: hv: Provide arch-neutral implementation of get_vtl()

In reply to Wei Liu
Quoted 33 lines · Wei Liu
Quoted 9 lines · Roman Kisel

To run in the VTL mode, Hyper-V drivers have to know what VTL the system boots in, and the arm64/hyperv code does not have the means to compute that.

Refactor the code to hoist the function that detects VTL, make it arch-neutral to be able to employ it to get the VTL on arm64. Fix the hypercall output address in get_vtl(void) not to overlap with the hypercall input area to adhere to the Hyper-V TLFS.

Can you split the fix out? That potentially can be backported.

Quoted 6 lines

---

drivers/hv/hv_common.c +3 -0
+
+#if IS_ENABLED(CONFIG_HYPERV_VTL_MODE)
+u8 __init get_vtl(void)

drivers/hv/hv_common.c u64 __weak hv_tdx_hypercall(u64 control, u64 param1, u64 param2) · line 671

Quoted 10 lines
drivers/hv/hv_common.c +10 -0
+{
+	u64 control = HV_HYPERCALL_REP_COMP_1 | HVCALL_GET_VP_REGISTERS;
+	struct hv_get_vp_registers_input *input;
+	struct hv_get_vp_registers_output *output;
+	unsigned long flags;
+	u64 ret;
+
+	local_irq_save(flags);
+	input = *this_cpu_ptr(hyperv_pcpu_input_arg);
+	output = *this_cpu_ptr(hyperv_pcpu_output_arg);

Hmm... I don't remember why the old code didn't use hyperv_pcpu_output_arg but instead reused input+OFFSET as output.

Quoted 1 line

Saurabh, can you comment on this?

This was done to optimize memory usage. Michael has provided more details on this in his review of the patch today.

#31 lore raw

Re: [PATCH v3 4/7] arm64: hyperv: Boot in a Virtual Trust Level

In reply to Roman Kisel
Quoted 63 lines · Roman Kisel

To run in the VTL mode, Hyper-V drivers have to know what VTL the system boots in, and the arm64/hyperv code does not update the variable that stores the value.

Update the variable to enable the Hyper-V drivers to boot in the VTL mode and print the VTL the code runs in.

---
 arch/arm64/hyperv/Makefile        |  1 +
 arch/arm64/hyperv/hv_vtl.c        | 13 +++++++++++++
 arch/arm64/hyperv/mshyperv.c      |  4 ++++
 arch/arm64/include/asm/mshyperv.h |  7 +++++++
 4 files changed, 25 insertions(+)
 create mode 100644 arch/arm64/hyperv/hv_vtl.c
arch/arm64/hyperv/Makefile +1 -0
index 87c31c001da9..9701a837a6e1 100644
@@ -1,2 +1,3 @@
 # SPDX-License-Identifier: GPL-2.0
 obj-y		:= hv_core.o mshyperv.o
+obj-$(CONFIG_HYPERV_VTL_MODE)	+= hv_vtl.o
arch/arm64/hyperv/hv_vtl.c new file +13 -0
new file mode 100644
index 000000000000..38642b7b6be0
@@ -0,0 +1,13 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (C) 2024, Microsoft, Inc.
+ *
+ * Author : Roman Kisel <[email protected]>
+ */
+
+#include <asm/mshyperv.h>
+
+void __init hv_vtl_init_platform(void)
+{
+	pr_info("Linux runs in Hyper-V Virtual Trust Level\n");
+}
arch/arm64/hyperv/mshyperv.c +4 -0
index 341f98312667..8fd04d6e4800 100644
@@ -98,6 +98,10 @@ static int __init hyperv_init(void)
 		return ret;
 	}
 
+	/* Find the VTL */
+	ms_hyperv.vtl = get_vtl();
+	hv_vtl_init_platform();
+
 	ms_hyperv_late_init();
 
 	hyperv_initialized = true;
arch/arm64/include/asm/mshyperv.h +0 -0
index a7a3586f7cb1..63d6bb6998fc 100644

arch/arm64/include/asm/mshyperv.h static inline u64 hv_get_msr(unsigned int reg) · line 58

Quoted 10 lines
arch/arm64/include/asm/mshyperv.h +7 -0
@@ -49,6 +49,13 @@ static inline u64 hv_get_msr(unsigned int reg)
 				ARM_SMCCC_OWNER_VENDOR_HYP,	\
 				HV_SMCCC_FUNC_NUMBER)
 
+#ifdef CONFIG_HYPERV_VTL_MODE
+void __init hv_vtl_init_platform(void);
+int __init hv_vtl_early_init(void);
+#else
+static inline void __init hv_vtl_init_platform(void) {}
+#endif
+

These functions are defined in x86 header as well. We can move it to generic header.

- Saurabh

#44 lore raw

Re: [PATCH v3 6/7] Drivers: hv: vmbus: Get the IRQ number from DT

In reply to Roman Kisel
Quoted 26 lines · Roman Kisel

The VMBus driver uses ACPI for interrupt assignment on arm64 hence it won't function in the VTL mode where only DeviceTree can be used.

Update the VMBus driver to discover interrupt configuration via DeviceTree and indicate DMA cache coherency.

---
 drivers/hv/vmbus_drv.c | 49 ++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 49 insertions(+)
drivers/hv/vmbus_drv.c +6 -0
index 12a707ab73f8..7eee7caff5f6 100644
@@ -2306,6 +2306,34 @@ static int vmbus_acpi_add(struct platform_device *pdev)
 }
 #endif
 
+static int __maybe_unused vmbus_set_irq(struct platform_device *pdev)
+{
+	struct irq_desc *desc;
+	int irq;
+
+	irq = platform_get_irq(pdev, 0);

drivers/hv/vmbus_drv.c static int vmbus_acpi_add(struct platform_device *pdev) · line 2324

Quoted 10 lines
drivers/hv/vmbus_drv.c +10 -0
+	if (irq == 0) {
+		pr_err("VMBus interrupt mapping failure\n");
+		return -EINVAL;
+	}
+	if (irq < 0) {
+		pr_err("VMBus interrupt data can't be read from DeviceTree, error %d\n", irq);
+		return irq;
+	}
+
+	desc = irq_to_desc(irq);

irq_to_desc is not an exported symbol if CONFIG_SPARSE_IRQ is enabled. This will break the builds for HYPERV as module.

- Saurabh

#45 lore raw

RE: [PATCH v3 6/7] Drivers: hv: vmbus: Get the IRQ number from DT

From: Saurabh Singh Sengar <[email protected]> Sent: Monday, August 5, 2024 1:30 AM

Quoted 33 lines · Roman Kisel
Quoted 31 lines

The VMBus driver uses ACPI for interrupt assignment on arm64 hence it won't function in the VTL mode where only DeviceTree can be used.

Update the VMBus driver to discover interrupt configuration via DeviceTree and indicate DMA cache coherency.

---
 drivers/hv/vmbus_drv.c | 49 ++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 49 insertions(+)
drivers/hv/vmbus_drv.c +11 -0
index 12a707ab73f8..7eee7caff5f6 100644
@@ -2306,6 +2306,34 @@ static int vmbus_acpi_add(struct platform_device *pdev)
 }
 #endif
 
+static int __maybe_unused vmbus_set_irq(struct platform_device *pdev)
+{
+	struct irq_desc *desc;
+	int irq;
+
+	irq = platform_get_irq(pdev, 0);
+	if (irq == 0) {
+		pr_err("VMBus interrupt mapping failure\n");
+		return -EINVAL;
+	}
+	if (irq < 0) {
Quoted 9 lines

drivers/hv/vmbus_drv.c static int vmbus_acpi_add(struct platform_device *pdev) · line 2324

Quoted 5 lines
drivers/hv/vmbus_drv.c +5 -0
+		pr_err("VMBus interrupt data can't be read from DeviceTree, error %d\n", irq);
+		return irq;
+	}
+
+	desc = irq_to_desc(irq);

irq_to_desc is not an exported symbol if CONFIG_SPARSE_IRQ is enabled. This will break the builds for HYPERV as module.

Instead, use irq_get_irq_data(), then irqd_to_hwirq().

Michael

#49 lore raw

Re: [PATCH v3 7/7] PCI: hv: Get vPCI MSI IRQ domain from DT

In reply to Wei Liu
Quoted 23 lines · Wei Liu
Quoted 22 lines · Roman Kisel

The hyperv-pci driver uses ACPI for MSI IRQ domain configuration on arm64. It won't be able to do that in the VTL mode where only DeviceTree can be used.

Update the hyperv-pci driver to get vPCI MSI IRQ domain in the DeviceTree case, too.

---
  drivers/hv/vmbus_drv.c              | 23 +++++++-----
  drivers/pci/controller/pci-hyperv.c | 55 +++++++++++++++++++++++++++--
  include/linux/hyperv.h              |  2 ++
  3 files changed, 69 insertions(+), 11 deletions(-)
drivers/hv/vmbus_drv.c +0 -0
index 7eee7caff5f6..038bd9be64b7 100644
@@ -45,7 +45,8 @@ struct vmbus_dynid {
  	struct hv_vmbus_device_id id;
  };
 
Quoted 8 lines

drivers/hv/vmbus_drv.c struct vmbus_dynid { · line 49

Quoted 3 lines
drivers/hv/vmbus_drv.c +2 -1
-static struct device  *hv_dev;
+/* VMBus Root Device */
+static struct device  *vmbus_root_device;

You're changing the name of the variable. That should preferably be done in a separate patch. That's going to make this patch shorter and easier to review.

Will fix in v4, thanks!

Quoted 115 lines

drivers/hv/vmbus_drv.c static struct resource *fb_mmio; · line 88

Quoted 12 lines
drivers/hv/vmbus_drv.c +5 -0
  static int hyperv_cpuhp_online;
 
@@ -80,9 +81,15 @@ static struct resource *fb_mmio;
  static struct resource *hyperv_mmio;
  static DEFINE_MUTEX(hyperv_mmio_lock);
 
+struct device *get_vmbus_root_device(void)
+{
+	return vmbus_root_device;
+}
+EXPORT_SYMBOL_GPL(get_vmbus_root_device);

I would like you to add "hv_" prefix to this exported symbol, or rename it to "vmbus_get_root_device()".

Quoted 82 lines
drivers/hv/vmbus_drv.c +8 -7
+
  static int vmbus_exists(void)
  {
-	if (hv_dev == NULL)
+	if (vmbus_root_device == NULL)
  		return -ENODEV;
 
  	return 0;
@@ -861,7 +868,7 @@ static int vmbus_dma_configure(struct device *child_device)
  	 * On x86/x64 coherence is assumed and these calls have no effect.
  	 */
  	hv_setup_dma_ops(child_device,
-		device_get_dma_attr(hv_dev) == DEV_DMA_COHERENT);
+		device_get_dma_attr(vmbus_root_device) == DEV_DMA_COHERENT);
  	return 0;
  }
 
@@ -1892,7 +1899,7 @@ int vmbus_device_register(struct hv_device *child_device_obj)
  		     &child_device_obj->channel->offermsg.offer.if_instance);
 
  	child_device_obj->device.bus = &hv_bus;
-	child_device_obj->device.parent = hv_dev;
+	child_device_obj->device.parent = vmbus_root_device;
  	child_device_obj->device.release = vmbus_device_release;
 
  	child_device_obj->device.dma_parms = &child_device_obj->dma_parms;
@@ -2253,7 +2260,7 @@ static int vmbus_acpi_add(struct platform_device *pdev)
  	struct acpi_device *ancestor;
  	struct acpi_device *device = ACPI_COMPANION(&pdev->dev);
 
-	hv_dev = &device->dev;
+	vmbus_root_device = &device->dev;
 
  	/*
  	 * Older versions of Hyper-V for ARM64 fail to include the _CCA
@@ -2342,7 +2349,7 @@ static int vmbus_device_add(struct platform_device *pdev)
  	struct device_node *np = pdev->dev.of_node;
  	int ret;
 
-	hv_dev = &pdev->dev;
+	vmbus_root_device = &pdev->dev;
 
  	ret = of_range_parser_init(&parser, np);
  	if (ret)
@@ -2675,7 +2682,7 @@ static int __init hv_acpi_init(void)
  	if (ret)
  		return ret;
 
-	if (!hv_dev) {
+	if (!vmbus_root_device) {
  		ret = -ENODEV;
  		goto cleanup;
  	}
@@ -2706,7 +2713,7 @@ static int __init hv_acpi_init(void)
 
  cleanup:
  	platform_driver_unregister(&vmbus_platform_driver);
-	hv_dev = NULL;
+	vmbus_root_device = NULL;
  	return ret;
  }
 
drivers/pci/controller/pci-hyperv.c +5 -0
index 5992280e8110..cdecefd1f9bd 100644
@@ -50,6 +50,7 @@
  #include <linux/irqdomain.h>
  #include <linux/acpi.h>
  #include <linux/sizes.h>
+#include <linux/of_irq.h>
  #include <asm/mshyperv.h>
 
  /*
@@ -887,6 +888,35 @@ static const struct irq_domain_ops hv_pci_domain_ops = {
  	.activate = hv_pci_vec_irq_domain_activate,
  };
 
+#ifdef CONFIG_OF
+
+static struct irq_domain *hv_pci_of_irq_domain_parent(void)
+{

drivers/pci/controller/pci-hyperv.c static const struct irq_domain_ops hv_pci_domain_ops = { · line 904

Quoted 10 lines
drivers/pci/controller/pci-hyperv.c +10 -0
+	struct device_node *parent;
+	struct irq_domain *domain;
+
+	parent = of_irq_find_parent(to_platform_device(get_vmbus_root_device())->dev.of_node);
+	domain = NULL;
+	if (parent) {
+		domain = irq_find_host(parent);
+		of_node_put(parent);
+	}
+

I cannot really comment on the ARM side of things around how this system is set up. I will leave that to someone who's more familiar with the matter to review.

Thanks,
Wei.

Signature · 2 lines
Thank you,
Roman
#3 lore raw

Re: [PATCH v3 1/7] arm64: hyperv: Use SMC to detect hypervisor presence

In reply to Wei Liu
Quoted 33 lines · Wei Liu
Quoted 9 lines · Roman Kisel

The arm64 Hyper-V startup path relies on ACPI to detect running under a Hyper-V compatible hypervisor. That doesn't work on non-ACPI systems.

Hoist the ACPI detection logic into a separate function, use the new SMC added recently to Hyper-V to use in the non-ACPI case.

The change looks sensible.

Within one minor comment fixed below:

However I would also like to get an Acked-by or reivewed-by from someone who works on the ARM64 side of things -- Saurabh, Boqun, Srivatsa, and Jinank?

Quoted 5 lines
---
  arch/arm64/hyperv/mshyperv.c      | 36 ++++++++++++++++++++++++++-----
  arch/arm64/include/asm/mshyperv.h |  5 +++++
  2 files changed, 36 insertions(+), 5 deletions(-)

[...]

Quoted 6 lines
arch/arm64/include/asm/mshyperv.h +0 -0
index a975e1a689dd..a7a3586f7cb1 100644
@@ -51,4 +51,9 @@ static inline u64 hv_get_msr(unsigned int reg)
 
Quoted 8 lines

arch/arm64/include/asm/mshyperv.h static inline u64 hv_get_msr(unsigned int reg) · line 55

Quoted 4 lines
arch/arm64/include/asm/mshyperv.h +2 -0
  #include <asm-generic/mshyperv.h>
 
+#define ARM_SMCCC_VENDOR_HYP_UID_HYPERV_REG_0	0x7948734d
+#define ARM_SMCCC_VENDOR_HYP_UID_HYPERV_REG_1	0x56726570

Presumably these are ASCII codes for an identifier string, please provide comments to explain what they are.

Michael posted a suggestion to change that altogether; will elaborate on the topic in the thread with Michael.

Quoted 7 lines
Quoted 7 lines
arch/arm64/include/asm/mshyperv.h +3 -0
+#define ARM_SMCCC_VENDOR_HYP_UID_HYPERV_REG_2	0
+#define ARM_SMCCC_VENDOR_HYP_UID_HYPERV_REG_3	0
+
  #endif
Signature · 1 line
2.34.1
Signature · 2 lines
Thank you,
Roman
#28 lore raw

Re: [PATCH v3 4/7] arm64: hyperv: Boot in a Virtual Trust Level

In reply to Wei Liu
Quoted 52 lines · Wei Liu
Quoted 51 lines · Roman Kisel

To run in the VTL mode, Hyper-V drivers have to know what VTL the system boots in, and the arm64/hyperv code does not update the variable that stores the value.

Update the variable to enable the Hyper-V drivers to boot in the VTL mode and print the VTL the code runs in.

---
  arch/arm64/hyperv/Makefile        |  1 +
  arch/arm64/hyperv/hv_vtl.c        | 13 +++++++++++++
  arch/arm64/hyperv/mshyperv.c      |  4 ++++
  arch/arm64/include/asm/mshyperv.h |  7 +++++++
  4 files changed, 25 insertions(+)

create mode 100644 arch/arm64/hyperv/hv_vtl.c

arch/arm64/hyperv/Makefile +1 -0
index 87c31c001da9..9701a837a6e1 100644
@@ -1,2 +1,3 @@
  # SPDX-License-Identifier: GPL-2.0
  obj-y		:= hv_core.o mshyperv.o
+obj-$(CONFIG_HYPERV_VTL_MODE)	+= hv_vtl.o
arch/arm64/hyperv/hv_vtl.c new file +13 -0
new file mode 100644
index 000000000000..38642b7b6be0
@@ -0,0 +1,13 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (C) 2024, Microsoft, Inc.
+ *
+ * Author : Roman Kisel <[email protected]>
+ */
+
+#include <asm/mshyperv.h>
+
+void __init hv_vtl_init_platform(void)
+{
+	pr_info("Linux runs in Hyper-V Virtual Trust Level\n");
+}
arch/arm64/hyperv/mshyperv.c +0 -0
index 341f98312667..8fd04d6e4800 100644
@@ -98,6 +98,10 @@ static int __init hyperv_init(void)
  		return ret;
  	}
 
Quoted 7 lines

arch/arm64/hyperv/mshyperv.c static int __init hyperv_init(void) · line 103

Quoted 3 lines
arch/arm64/hyperv/mshyperv.c +3 -0
+	/* Find the VTL */
+	ms_hyperv.vtl = get_vtl();
+	hv_vtl_init_platform();

It doesn't make sense to me because this function unconditionally prints Linux runs in Hyper-V Virtual Trust Level.

Thought to structure this as the similar parts of the VTL support are. Will remove, thank you!

Quoted 28 lines

Thanks,
Wei.

Quoted 25 lines
arch/arm64/hyperv/mshyperv.c +1 -0
+
  	ms_hyperv_late_init();
 
  	hyperv_initialized = true;
arch/arm64/include/asm/mshyperv.h +7 -0
index a7a3586f7cb1..63d6bb6998fc 100644
@@ -49,6 +49,13 @@ static inline u64 hv_get_msr(unsigned int reg)
  				ARM_SMCCC_OWNER_VENDOR_HYP,	\
  				HV_SMCCC_FUNC_NUMBER)
 
+#ifdef CONFIG_HYPERV_VTL_MODE
+void __init hv_vtl_init_platform(void);
+int __init hv_vtl_early_init(void);
+#else
+static inline void __init hv_vtl_init_platform(void) {}
+#endif
+
  #include <asm-generic/mshyperv.h>
 
  #define ARM_SMCCC_VENDOR_HYP_UID_HYPERV_REG_0	0x7948734d
Signature · 1 line
2.34.1
Signature · 2 lines
Thank you,
Roman
#9 lore raw

Re: [PATCH v3 1/7] arm64: hyperv: Use SMC to detect hypervisor presence

In reply to Saurabh Singh Sengar
Quoted 45 lines · Saurabh Singh Sengar
Quoted 44 lines · Roman Kisel

The arm64 Hyper-V startup path relies on ACPI to detect running under a Hyper-V compatible hypervisor. That doesn't work on non-ACPI systems.

Hoist the ACPI detection logic into a separate function, use the new SMC added recently to Hyper-V to use in the non-ACPI case.

---
  arch/arm64/hyperv/mshyperv.c      | 36 ++++++++++++++++++++++++++-----
  arch/arm64/include/asm/mshyperv.h |  5 +++++
  2 files changed, 36 insertions(+), 5 deletions(-)
arch/arm64/hyperv/mshyperv.c +22 -0
index b1a4de4eee29..341f98312667 100644
@@ -27,6 +27,34 @@ int hv_get_hypervisor_version(union hv_hypervisor_version_info *info)
  	return 0;
  }
 
+static bool hyperv_detect_via_acpi(void)
+{
+	if (acpi_disabled)
+		return false;
+#if IS_ENABLED(CONFIG_ACPI)
+	/* Hypervisor ID is only available in ACPI v6+. */
+	if (acpi_gbl_FADT.header.revision < 6)
+		return false;
+	return strncmp((char *)&acpi_gbl_FADT.hypervisor_id, "MsHyperV", 8) == 0;
+#else
+	return false;
+#endif
+}
+
+static bool hyperv_detect_via_smc(void)
+{
+	struct arm_smccc_res res = {};
+
+	if (arm_smccc_1_1_get_conduit() != SMCCC_CONDUIT_HVC)
+		return false;
+	arm_smccc_1_1_hvc(ARM_SMCCC_VENDOR_HYP_CALL_UID_FUNC_ID, &res);
+
Quoted 9 lines

arch/arm64/hyperv/mshyperv.c int hv_get_hypervisor_version(union hv_hypervisor_version_info *info) · line 56

Quoted 5 lines
arch/arm64/hyperv/mshyperv.c +5 -0
+	return res.a0 == ARM_SMCCC_VENDOR_HYP_UID_HYPERV_REG_0 &&
+		res.a1 == ARM_SMCCC_VENDOR_HYP_UID_HYPERV_REG_1 &&
+		res.a2 == ARM_SMCCC_VENDOR_HYP_UID_HYPERV_REG_2 &&
+		res.a3 == ARM_SMCCC_VENDOR_HYP_UID_HYPERV_REG_3;
+}

As you mentioned in the cover letter this is supported in latest Hyper-V hypervisor, can we add a comment about it, specifying the exact version in it would be great.

I can add a comment about that, thought that would look as too much detail to refer to a version of the Windows insiders build in the comments in this code. Another option would be to entrench the logic in if statements which felt gross as there is a fallback.

Quoted 2 lines

If someone attempts to build non-ACPI kernel on older Hyper-V what is the behaviour of this function, do we need to safeguard or handle that case ?

The function won't panic if that's what you're asking about, i.e. safe for runtime. That won't break the build either as it relies on the SMCCC spec, and that uses the smc or hvc instructions (the code does expect hvc to be the conduit and checks for that being the case). The hypervisor doesn't inject the exception in the guest for the unknown call, just returns SMCCC_RET_NOT_SUPPORTED in the first output register (the hypervisor got a unit-test for that, too).

That said, I think the logic is solid. Appreciate your question and your help! Will be glad to discuss other concerns should you have any.

Quoted 2 lines

- Saurabh

Signature · 2 lines
Thank you,
Roman
#16 lore raw

Re: [PATCH v3 2/7] Drivers: hv: Enable VTL mode for arm64

In reply to Saurabh Singh Sengar
Quoted 53 lines · Saurabh Singh Sengar
Quoted 36 lines · Michael Kelley

From: Roman Kisel <[email protected]> Sent: Friday, July 26, 2024 3:59 PM

Quoted 35 lines

Kconfig dependencies for arm64 guests on Hyper-V require that be ACPI enabled, and limit VTL mode to x86/x64. To enable VTL mode on arm64 as well, update the dependencies. Since VTL mode requires DeviceTree instead of ACPI, don't require arm64 guests on Hyper-V to have ACPI.

---
  drivers/hv/Kconfig | 6 +++---
  1 file changed, 3 insertions(+), 3 deletions(-)
drivers/hv/Kconfig +2 -2
index 862c47b191af..a5cd1365e248 100644
@@ -5,7 +5,7 @@ menu "Microsoft Hyper-V guest support"
  config HYPERV
  	tristate "Microsoft Hyper-V client drivers"
  	depends on (X86 && X86_LOCAL_APIC && HYPERVISOR_GUEST) \
-		|| (ACPI && ARM64 && !CPU_BIG_ENDIAN)
+		|| (ARM64 && !CPU_BIG_ENDIAN)
  	select PARAVIRT
  	select X86_HV_CALLBACK_VECTOR if X86
  	select OF_EARLY_FLATTREE if OF
@@ -15,7 +15,7 @@ config HYPERV
 
  config HYPERV_VTL_MODE
  	bool "Enable Linux to boot in VTL context"
-	depends on X86_64 && HYPERV
+	depends on HYPERV
  	depends on SMP
  	default n
  	help
@@ -31,7 +31,7 @@ config HYPERV_VTL_MODE
 
Quoted 15 lines
Quoted 10 lines
drivers/hv/Kconfig +1 -2
  	  Select this option to build a Linux kernel to run at a VTL other than
  	  the normal VTL0, which currently is only VTL2.  This option
-	  initializes the x86 platform for VTL2, and adds the ability to boot
+	  initializes the kernel to run in VTL2, and adds the ability to boot
  	  secondary CPUs directly into 64-bit context as required for VTLs other
  	  than 0.  A kernel built with this option must run at VTL2, and will
  	  not run as a normal guest.
--

2.34.1

In v2 of this patch, I suggested [1] making a couple additional minor changes so that kernels built *without* HYPER_VTL_MODE would still require ACPI. Did that suggestion not work out? If that's the case, I'm curious about what goes wrong.

Quoted 4 lines

Hi Michael/Roman,
I was considering making HYPERV_VTL_MODE depend on CONFIG_OF. That should address above concern as well. Do you see any potential issue with it.

Michael,

I ran into a pretty gnarly recursive dependencies which in all fairness might stem from not being fluent enough in the Kconfig language. Any help of how to approach implementing your idea would be greatly appreciated!

Saurabh,

I could try out the idea you're offering if you folks are fine with that. Or, we could let this be for the time being and grapple with that in a separate patch series :)

Quoted 1 line

- Saurabh

Signature · 2 lines
Thank you,
Roman
#10 lore raw

Re: [PATCH v3 1/7] arm64: hyperv: Use SMC to detect hypervisor presence

In reply to Roman Kisel
Quoted 56 lines · Roman Kisel
Quoted 44 lines · Saurabh Singh Sengar
Quoted 43 lines · Roman Kisel

The arm64 Hyper-V startup path relies on ACPI to detect running under a Hyper-V compatible hypervisor. That doesn't work on non-ACPI systems.

Hoist the ACPI detection logic into a separate function, use the new SMC added recently to Hyper-V to use in the non-ACPI case.

---
 arch/arm64/hyperv/mshyperv.c      | 36 ++++++++++++++++++++++++++-----
 arch/arm64/include/asm/mshyperv.h |  5 +++++
 2 files changed, 36 insertions(+), 5 deletions(-)
arch/arm64/hyperv/mshyperv.c +22 -0
index b1a4de4eee29..341f98312667 100644
@@ -27,6 +27,34 @@ int hv_get_hypervisor_version(union hv_hypervisor_version_info *info)
 	return 0;
 }
+static bool hyperv_detect_via_acpi(void)
+{
+	if (acpi_disabled)
+		return false;
+#if IS_ENABLED(CONFIG_ACPI)
+	/* Hypervisor ID is only available in ACPI v6+. */
+	if (acpi_gbl_FADT.header.revision < 6)
+		return false;
+	return strncmp((char *)&acpi_gbl_FADT.hypervisor_id, "MsHyperV", 8) == 0;
+#else
+	return false;
+#endif
+}
+
+static bool hyperv_detect_via_smc(void)
+{
+	struct arm_smccc_res res = {};
+
+	if (arm_smccc_1_1_get_conduit() != SMCCC_CONDUIT_HVC)
+		return false;
+	arm_smccc_1_1_hvc(ARM_SMCCC_VENDOR_HYP_CALL_UID_FUNC_ID, &res);
+
Quoted 9 lines

arch/arm64/hyperv/mshyperv.c int hv_get_hypervisor_version(union hv_hypervisor_version_info *info) · line 56

Quoted 5 lines
arch/arm64/hyperv/mshyperv.c +5 -0
+	return res.a0 == ARM_SMCCC_VENDOR_HYP_UID_HYPERV_REG_0 &&
+		res.a1 == ARM_SMCCC_VENDOR_HYP_UID_HYPERV_REG_1 &&
+		res.a2 == ARM_SMCCC_VENDOR_HYP_UID_HYPERV_REG_2 &&
+		res.a3 == ARM_SMCCC_VENDOR_HYP_UID_HYPERV_REG_3;
+}

As you mentioned in the cover letter this is supported in latest Hyper-V hypervisor, can we add a comment about it, specifying the exact version in it would be great.

Quoted 4 lines

I can add a comment about that, thought that would look as too much detail to refer to a version of the Windows insiders build in the comments in this code. Another option would be to entrench the logic in if statements which felt gross as there is a fallback.

I'll leave the decision to your judgment.

Quoted 10 lines
Quoted 3 lines

If someone attempts to build non-ACPI kernel on older Hyper-V what is the behaviour of this function, do we need to safeguard or handle that case ?

The function won't panic if that's what you're asking about, i.e. safe for runtime. That won't break the build either as it relies on the SMCCC spec, and that uses the smc or hvc instructions (the code does expect hvc to be the conduit and checks for that being the case). The hypervisor doesn't inject the exception in the guest for the unknown call, just returns SMCCC_RET_NOT_SUPPORTED in the first output register (the hypervisor got a unit-test for that, too).

Looks good, have you considered checking for SMCCC_RET_NOT_SUPPORTED ?

- Saurabh

#32 lore raw

Re: [PATCH v3 4/7] arm64: hyperv: Boot in a Virtual Trust Level

In reply to Saurabh Singh Sengar
Quoted 67 lines · Saurabh Singh Sengar
Quoted 66 lines · Roman Kisel

To run in the VTL mode, Hyper-V drivers have to know what VTL the system boots in, and the arm64/hyperv code does not update the variable that stores the value.

Update the variable to enable the Hyper-V drivers to boot in the VTL mode and print the VTL the code runs in.

---
  arch/arm64/hyperv/Makefile        |  1 +
  arch/arm64/hyperv/hv_vtl.c        | 13 +++++++++++++
  arch/arm64/hyperv/mshyperv.c      |  4 ++++
  arch/arm64/include/asm/mshyperv.h |  7 +++++++
  4 files changed, 25 insertions(+)

create mode 100644 arch/arm64/hyperv/hv_vtl.c

arch/arm64/hyperv/Makefile +1 -0
index 87c31c001da9..9701a837a6e1 100644
@@ -1,2 +1,3 @@
  # SPDX-License-Identifier: GPL-2.0
  obj-y		:= hv_core.o mshyperv.o
+obj-$(CONFIG_HYPERV_VTL_MODE)	+= hv_vtl.o
arch/arm64/hyperv/hv_vtl.c new file +13 -0
new file mode 100644
index 000000000000..38642b7b6be0
@@ -0,0 +1,13 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (C) 2024, Microsoft, Inc.
+ *
+ * Author : Roman Kisel <[email protected]>
+ */
+
+#include <asm/mshyperv.h>
+
+void __init hv_vtl_init_platform(void)
+{
+	pr_info("Linux runs in Hyper-V Virtual Trust Level\n");
+}
arch/arm64/hyperv/mshyperv.c +4 -0
index 341f98312667..8fd04d6e4800 100644
@@ -98,6 +98,10 @@ static int __init hyperv_init(void)
  		return ret;
  	}
 
+	/* Find the VTL */
+	ms_hyperv.vtl = get_vtl();
+	hv_vtl_init_platform();
+
  	ms_hyperv_late_init();
 
  	hyperv_initialized = true;
arch/arm64/include/asm/mshyperv.h +0 -0
index a7a3586f7cb1..63d6bb6998fc 100644
@@ -49,6 +49,13 @@ static inline u64 hv_get_msr(unsigned int reg)
  				ARM_SMCCC_OWNER_VENDOR_HYP,	\
  				HV_SMCCC_FUNC_NUMBER)
 
Quoted 10 lines

arch/arm64/include/asm/mshyperv.h static inline u64 hv_get_msr(unsigned int reg) · line 58

Quoted 7 lines
arch/arm64/include/asm/mshyperv.h +7 -0
+#ifdef CONFIG_HYPERV_VTL_MODE
+void __init hv_vtl_init_platform(void);
+int __init hv_vtl_early_init(void);
+#else
+static inline void __init hv_vtl_init_platform(void) {}
+#endif
+

These functions are defined in x86 header as well. We can move it to generic header.

Will do, thanks!

Quoted 1 line

- Saurabh

Signature · 2 lines
Thank you,
Roman
#46 lore raw

Re: [PATCH v3 6/7] Drivers: hv: vmbus: Get the IRQ number from DT

In reply to Michael Kelley
Quoted 44 lines · Michael Kelley

From: Saurabh Singh Sengar <[email protected]> Sent: Monday, August 5, 2024 1:30 AM

Quoted 33 lines · Roman Kisel
Quoted 31 lines

The VMBus driver uses ACPI for interrupt assignment on arm64 hence it won't function in the VTL mode where only DeviceTree can be used.

Update the VMBus driver to discover interrupt configuration via DeviceTree and indicate DMA cache coherency.

---
  drivers/hv/vmbus_drv.c | 49 ++++++++++++++++++++++++++++++++++++++++++
  1 file changed, 49 insertions(+)
drivers/hv/vmbus_drv.c +11 -0
index 12a707ab73f8..7eee7caff5f6 100644
@@ -2306,6 +2306,34 @@ static int vmbus_acpi_add(struct platform_device *pdev)
  }
  #endif
 
+static int __maybe_unused vmbus_set_irq(struct platform_device *pdev)
+{
+	struct irq_desc *desc;
+	int irq;
+
+	irq = platform_get_irq(pdev, 0);
+	if (irq == 0) {
+		pr_err("VMBus interrupt mapping failure\n");
+		return -EINVAL;
+	}
+	if (irq < 0) {
Quoted 9 lines

drivers/hv/vmbus_drv.c static int vmbus_acpi_add(struct platform_device *pdev) · line 2324

Quoted 5 lines
drivers/hv/vmbus_drv.c +5 -0
+		pr_err("VMBus interrupt data can't be read from DeviceTree, error %d\n", irq);
+		return irq;
+	}
+
+	desc = irq_to_desc(irq);

irq_to_desc is not an exported symbol if CONFIG_SPARSE_IRQ is enabled. This will break the builds for HYPERV as module.

Quoted 2 lines

Instead, use irq_get_irq_data(), then irqd_to_hwirq().

Couldn't appreciate enough your indispensable advice, folks!

Quoted 1 line

Michael

Signature · 2 lines
Thank you,
Roman
#11 lore raw

Re: [PATCH v3 1/7] arm64: hyperv: Use SMC to detect hypervisor presence

In reply to Saurabh Singh Sengar
Quoted 75 lines · Saurabh Singh Sengar
Quoted 56 lines · Roman Kisel
Quoted 44 lines · Saurabh Singh Sengar
Quoted 43 lines · Roman Kisel

The arm64 Hyper-V startup path relies on ACPI to detect running under a Hyper-V compatible hypervisor. That doesn't work on non-ACPI systems.

Hoist the ACPI detection logic into a separate function, use the new SMC added recently to Hyper-V to use in the non-ACPI case.

---
  arch/arm64/hyperv/mshyperv.c      | 36 ++++++++++++++++++++++++++-----
  arch/arm64/include/asm/mshyperv.h |  5 +++++
  2 files changed, 36 insertions(+), 5 deletions(-)
arch/arm64/hyperv/mshyperv.c +22 -0
index b1a4de4eee29..341f98312667 100644
@@ -27,6 +27,34 @@ int hv_get_hypervisor_version(union hv_hypervisor_version_info *info)
  	return 0;
  }
+static bool hyperv_detect_via_acpi(void)
+{
+	if (acpi_disabled)
+		return false;
+#if IS_ENABLED(CONFIG_ACPI)
+	/* Hypervisor ID is only available in ACPI v6+. */
+	if (acpi_gbl_FADT.header.revision < 6)
+		return false;
+	return strncmp((char *)&acpi_gbl_FADT.hypervisor_id, "MsHyperV", 8) == 0;
+#else
+	return false;
+#endif
+}
+
+static bool hyperv_detect_via_smc(void)
+{
+	struct arm_smccc_res res = {};
+
+	if (arm_smccc_1_1_get_conduit() != SMCCC_CONDUIT_HVC)
+		return false;
+	arm_smccc_1_1_hvc(ARM_SMCCC_VENDOR_HYP_CALL_UID_FUNC_ID, &res);
+
Quoted 9 lines

arch/arm64/hyperv/mshyperv.c int hv_get_hypervisor_version(union hv_hypervisor_version_info *info) · line 56

Quoted 5 lines
arch/arm64/hyperv/mshyperv.c +5 -0
+	return res.a0 == ARM_SMCCC_VENDOR_HYP_UID_HYPERV_REG_0 &&
+		res.a1 == ARM_SMCCC_VENDOR_HYP_UID_HYPERV_REG_1 &&
+		res.a2 == ARM_SMCCC_VENDOR_HYP_UID_HYPERV_REG_2 &&
+		res.a3 == ARM_SMCCC_VENDOR_HYP_UID_HYPERV_REG_3;
+}

As you mentioned in the cover letter this is supported in latest Hyper-V hypervisor, can we add a comment about it, specifying the exact version in it would be great.

Quoted 4 lines

I can add a comment about that, thought that would look as too much detail to refer to a version of the Windows insiders build in the comments in this code. Another option would be to entrench the logic in if statements which felt gross as there is a fallback.

I'll leave the decision to your judgment.

Quoted 10 lines
Quoted 3 lines

If someone attempts to build non-ACPI kernel on older Hyper-V what is the behaviour of this function, do we need to safeguard or handle that case ?

The function won't panic if that's what you're asking about, i.e. safe for runtime. That won't break the build either as it relies on the SMCCC spec, and that uses the smc or hvc instructions (the code does expect hvc to be the conduit and checks for that being the case). The hypervisor doesn't inject the exception in the guest for the unknown call, just returns SMCCC_RET_NOT_SUPPORTED in the first output register (the hypervisor got a unit-test for that, too).

Quoted 2 lines

Looks good, have you considered checking for SMCCC_RET_NOT_SUPPORTED ?

No, I have not. Let me think out loud here... a0 is compared to what must be return from the hypervisor the UID. That constant is an all-1 32 or 64 bit pattern, high unlikely to see that as a part of the UID due to low entropy as I understand. I might've added the check though for the better code readability, and because we have this e-mail thread going on, looks like I must :) Let me do that in v4, thanks!

Quoted 1 line

- Saurabh

Signature · 2 lines
Thank you,
Roman
#50 lore raw

Re: [PATCH v3 7/7] PCI: hv: Get vPCI MSI IRQ domain from DT

In reply to Roman Kisel
Quoted 33 lines · Roman Kisel
Quoted 15 lines · Wei Liu
Quoted 14 lines · Roman Kisel

The hyperv-pci driver uses ACPI for MSI IRQ domain configuration on arm64. It won't be able to do that in the VTL mode where only DeviceTree can be used.

Update the hyperv-pci driver to get vPCI MSI IRQ domain in the DeviceTree case, too.

---
  drivers/hv/vmbus_drv.c              | 23 +++++++----- drivers/pci/controller/pci-hyperv.c | 55 +++++++++++++++++++++++++++-- include/linux/hyperv.h              |  2 ++

  3 files changed, 69 insertions(+), 11 deletions(-)
Quoted 15 lines

drivers/hv/vmbus_drv.c struct vmbus_dynid { · line 48

Quoted 10 lines
drivers/hv/vmbus_drv.c +2 -1
index 7eee7caff5f6..038bd9be64b7 100644
@@ -45,7 +45,8 @@ struct vmbus_dynid {
       struct hv_vmbus_device_id id;
   };
-static struct device  *hv_dev;
+/* VMBus Root Device */
+static struct device  *vmbus_root_device;

You're changing the name of the variable. That should preferably be done in a separate patch. That's going to make this patch shorter and easier to review.

Quoted 15 lines

Will fix in v4, thanks!

Quoted 13 lines

drivers/hv/vmbus_drv.c static struct resource *fb_mmio; · line 87

Quoted 9 lines
drivers/hv/vmbus_drv.c +5 -0
   static int hyperv_cpuhp_online;
@@ -80,9 +81,15 @@ static struct resource *fb_mmio;
   static struct resource *hyperv_mmio;
   static DEFINE_MUTEX(hyperv_mmio_lock);
+struct device *get_vmbus_root_device(void)
+{
+    return vmbus_root_device;
+}
+EXPORT_SYMBOL_GPL(get_vmbus_root_device);

I would like you to add "hv_" prefix to this exported symbol, or rename it to "vmbus_get_root_device()".

Will do in v4, missed this suggestion in my first reply.

Quoted 94 lines
Quoted 93 lines
Quoted 79 lines
drivers/hv/vmbus_drv.c +7 -6
+
   static int vmbus_exists(void)
   {
-    if (hv_dev == NULL)
+    if (vmbus_root_device == NULL)
           return -ENODEV;
       return 0;
@@ -861,7 +868,7 @@ static int vmbus_dma_configure(struct device 
 *child_device)
        * On x86/x64 coherence is assumed and these calls have no effect.
        */
       hv_setup_dma_ops(child_device,
-        device_get_dma_attr(hv_dev) == DEV_DMA_COHERENT);
+        device_get_dma_attr(vmbus_root_device) == DEV_DMA_COHERENT);
       return 0;
   }
@@ -1892,7 +1899,7 @@ int vmbus_device_register(struct hv_device 
 *child_device_obj)
                &child_device_obj->channel->offermsg.offer.if_instance);
       child_device_obj->device.bus = &hv_bus;
-    child_device_obj->device.parent = hv_dev;
+    child_device_obj->device.parent = vmbus_root_device;
       child_device_obj->device.release = vmbus_device_release;
       child_device_obj->device.dma_parms = &child_device_obj->dma_parms;
@@ -2253,7 +2260,7 @@ static int vmbus_acpi_add(struct 
 platform_device *pdev)
       struct acpi_device *ancestor;
       struct acpi_device *device = ACPI_COMPANION(&pdev->dev);
-    hv_dev = &device->dev;
+    vmbus_root_device = &device->dev;
       /*
        * Older versions of Hyper-V for ARM64 fail to include the _CCA
@@ -2342,7 +2349,7 @@ static int vmbus_device_add(struct 
 platform_device *pdev)
       struct device_node *np = pdev->dev.of_node;
       int ret;
-    hv_dev = &pdev->dev;
+    vmbus_root_device = &pdev->dev;
       ret = of_range_parser_init(&parser, np);
       if (ret)
@@ -2675,7 +2682,7 @@ static int __init hv_acpi_init(void)
       if (ret)
           return ret;
-    if (!hv_dev) {
+    if (!vmbus_root_device) {
           ret = -ENODEV;
           goto cleanup;
       }
@@ -2706,7 +2713,7 @@ static int __init hv_acpi_init(void)

  cleanup:
      platform_driver_unregister(&vmbus_platform_driver); -    hv_dev = NULL;
+    vmbus_root_device = NULL;
      return ret;
  }

drivers/pci/controller/pci-hyperv.c +9 -0
index 5992280e8110..cdecefd1f9bd 100644
@@ -50,6 +50,7 @@
   #include <linux/irqdomain.h>
   #include <linux/acpi.h>
   #include <linux/sizes.h>
+#include <linux/of_irq.h>
   #include <asm/mshyperv.h>
   /*
@@ -887,6 +888,35 @@ static const struct irq_domain_ops 
 hv_pci_domain_ops = {
       .activate = hv_pci_vec_irq_domain_activate,
   };
+#ifdef CONFIG_OF
+
+static struct irq_domain *hv_pci_of_irq_domain_parent(void)
+{
+    struct device_node *parent;
+    struct irq_domain *domain;
+
+    parent = 

of_irq_find_parent(to_platform_device(get_vmbus_root_device())-

Quoted 7 lines
Quoted 1 line

dev.of_node);

+    domain = NULL;
+    if (parent) {
+        domain = irq_find_host(parent);
+        of_node_put(parent);
+    }
+

I cannot really comment on the ARM side of things around how this system is set up. I will leave that to someone who's more familiar with the matter to review.

Thanks,
Wei.

Signature · 2 lines
Thank you,
Roman
#23 lore raw

Re: [PATCH v3 3/7] Drivers: hv: Provide arch-neutral implementation of get_vtl()

In reply to Michael Kelley
Quoted 124 lines · Michael Kelley

From: Roman Kisel <[email protected]> Sent: Friday, July 26, 2024 3:59 PM

Quoted 106 lines

To run in the VTL mode, Hyper-V drivers have to know what VTL the system boots in, and the arm64/hyperv code does not have the means to compute that.

Refactor the code to hoist the function that detects VTL, make it arch-neutral to be able to employ it to get the VTL on arm64. Fix the hypercall output address in get_vtl(void) not to overlap with the hypercall input area to adhere to the Hyper-V TLFS.

---
  arch/x86/hyperv/hv_init.c          | 34 ---------------------
  arch/x86/include/asm/hyperv-tlfs.h |  7 -----
  drivers/hv/hv_common.c             | 47 ++++++++++++++++++++++++++++--
  include/asm-generic/hyperv-tlfs.h  |  7 +++++
  include/asm-generic/mshyperv.h     |  6 ++++
  5 files changed, 58 insertions(+), 43 deletions(-)
arch/x86/hyperv/hv_init.c +0 -34
index 17a71e92a343..c350fa05ee59 100644
@@ -413,40 +413,6 @@ static void __init hv_get_partition_id(void)
  	local_irq_restore(flags);
  }
 
-#if IS_ENABLED(CONFIG_HYPERV_VTL_MODE)
-static u8 __init get_vtl(void)
-{
-	u64 control = HV_HYPERCALL_REP_COMP_1 | HVCALL_GET_VP_REGISTERS;
-	struct hv_get_vp_registers_input *input;
-	struct hv_get_vp_registers_output *output;
-	unsigned long flags;
-	u64 ret;
-
-	local_irq_save(flags);
-	input = *this_cpu_ptr(hyperv_pcpu_input_arg);
-	output = (struct hv_get_vp_registers_output *)input;
-
-	memset(input, 0, struct_size(input, element, 1));
-	input->header.partitionid = HV_PARTITION_ID_SELF;
-	input->header.vpindex = HV_VP_INDEX_SELF;
-	input->header.inputvtl = 0;
-	input->element[0].name0 = HV_X64_REGISTER_VSM_VP_STATUS;
-
-	ret = hv_do_hypercall(control, input, output);
-	if (hv_result_success(ret)) {
-		ret = output->as64.low & HV_X64_VTL_MASK;
-	} else {
-		pr_err("Failed to get VTL(error: %lld) exiting...\n", ret);
-		BUG();
-	}
-
-	local_irq_restore(flags);
-	return ret;
-}
-#else
-static inline u8 get_vtl(void) { return 0; }
-#endif
-
  /*
   * This function is to be invoked early in the boot sequence after the
   * hypervisor has been detected.
arch/x86/include/asm/hyperv-tlfs.h +0 -7
index 3787d26810c1..9ee68eb8e6ff 100644
@@ -309,13 +309,6 @@ enum hv_isolation_type {
  #define HV_MSR_STIMER0_CONFIG	(HV_X64_MSR_STIMER0_CONFIG)
  #define HV_MSR_STIMER0_COUNT	(HV_X64_MSR_STIMER0_COUNT)
 
-/*
- * Registers are only accessible via HVCALL_GET_VP_REGISTERS hvcall and
- * there is not associated MSR address.
- */
-#define	HV_X64_REGISTER_VSM_VP_STATUS	0x000D0003
-#define	HV_X64_VTL_MASK			GENMASK(3, 0)
-
  /* Hyper-V memory host visibility */
  enum hv_mem_host_visibility {
  	VMBUS_PAGE_NOT_VISIBLE		= 0,
drivers/hv/hv_common.c +5 -2
index 9c452bfbd571..7d6c1523b0b5 100644
@@ -339,8 +339,8 @@ int __init hv_common_init(void)
  	hyperv_pcpu_input_arg = alloc_percpu(void  *);
  	BUG_ON(!hyperv_pcpu_input_arg);
 
-	/* Allocate the per-CPU state for output arg for root */
-	if (hv_root_partition) {
+	/* Allocate the per-CPU state for output arg for root or a VTL */
+	if (hv_root_partition || IS_ENABLED(CONFIG_HYPERV_VTL_MODE)) {
  		hyperv_pcpu_output_arg = alloc_percpu(void *);
  		BUG_ON(!hyperv_pcpu_output_arg);
  	}
@@ -656,3 +656,46 @@ u64 __weak hv_tdx_hypercall(u64 control, u64 param1, u64
 param2)
  	return HV_STATUS_INVALID_PARAMETER;
  }
  EXPORT_SYMBOL_GPL(hv_tdx_hypercall);
+
+#if IS_ENABLED(CONFIG_HYPERV_VTL_MODE)
+u8 __init get_vtl(void)

drivers/hv/hv_common.c u64 __weak hv_tdx_hypercall(u64 control, u64 param1, u64 · line 671

Quoted 10 lines
drivers/hv/hv_common.c +10 -0
+{
+	u64 control = HV_HYPERCALL_REP_COMP_1 | HVCALL_GET_VP_REGISTERS;
+	struct hv_get_vp_registers_input *input;
+	struct hv_get_vp_registers_output *output;
+	unsigned long flags;
+	u64 ret;
+
+	local_irq_save(flags);
+	input = *this_cpu_ptr(hyperv_pcpu_input_arg);
+	output = *this_cpu_ptr(hyperv_pcpu_output_arg);

Rather than use the hyperv_pcpu_output_arg here, it's OK to use a different area of the hyperv_pcpu_input_arg page. For example,

	output = (void *)input + HV_HYP_PAGE_SIZE/2;
Quoted 8 lines

The TLFS does not require that the input and output be in separate pages.

While using the hyperv_pcpu_output_arg is conceptually a bit cleaner, doing so requires allocating a 4K page per CPU that is not otherwise used. The VTL 2 code wants to be frugal with memory, and this seems like a good step in that direction. :-)

I agree on the both counts: the code looks conceptually cleaner now and VTL2 wants to be frugal with memory, esp that the output hypercall page is per-CPU so we have O(n) as the CPU count increases. Still, the output page will be needed for VTL2 (say to get/set registers just as done here). That said, with this patch we can achieve both the conceptual cleanliness and being ready to grow more on the primitives being built out in the VTL support patches.

Quoted 42 lines

The hyperv_pcpu_output_arg was added for the cases where up to a full page is needed for input and output on the same hypercall. So far, the only case of that is when running in the root partition.

Quoted 38 lines
drivers/hv/hv_common.c +30 -0
+
+	memset(input, 0, struct_size(input, element, 1));
+	input->header.partitionid = HV_PARTITION_ID_SELF;
+	input->header.vpindex = HV_VP_INDEX_SELF;
+	input->header.inputvtl = 0;
+	input->element[0].name0 = HV_REGISTER_VSM_VP_STATUS;
+
+	ret = hv_do_hypercall(control, input, output);
+	if (hv_result_success(ret)) {
+		ret = output->as64.low & HV_VTL_MASK;
+	} else {
+		pr_err("Failed to get VTL(error: %lld) exiting...\n", ret);
+
+		/*
+		 * This is a dead end, something fundamental is broken.
+		 *
+		 * There is no sensible way of continuing as the Hyper-V drivers
+		 * transitively depend via the vmbus driver on knowing which VTL
+		 * they run in to establish communication with the host. The kernel
+		 * is going to be worse off if continued booting than a panicked one,
+		 * just hung and stuck, producing second-order failures, with neither
+		 * a way to recover nor to provide expected services.
+		 */
+		BUG();
+	}
+
+	local_irq_restore(flags);
+	return ret;
+}
+#endif
include/asm-generic/hyperv-tlfs.h +0 -0
index 814207e7c37f..271c365973d6 100644
@@ -75,6 +75,13 @@
  /* AccessTscInvariantControls privilege */
  #define HV_ACCESS_TSC_INVARIANT			BIT(15)
 
Quoted 6 lines

include/asm-generic/hyperv-tlfs.h line 80

Quoted 3 lines
include/asm-generic/hyperv-tlfs.h +3 -0
+/*
+ * This synthetic register is only accessible via the HVCALL_GET_VP_REGISTERS
+ * hvcall, and there is no an associated MSR on x86.

s/there is no an associated/there is no associated/

Much appreciated, will fix this!

Quoted 8 lines

include/asm-generic/hyperv-tlfs.h line 83

Quoted 3 lines
include/asm-generic/hyperv-tlfs.h +3 -0
+ */
+#define	HV_REGISTER_VSM_VP_STATUS	0x000D0003
+#define	HV_VTL_MASK			GENMASK(3, 0)

Further down in hyperv-tlfs.h is a section devoted to register definitions. It seems like this definition should go there in numeric order, which is after HV_REGISTER_STIMER0_COUNT.

Agreed, will fix, thanks!

Quoted 25 lines

Michael

Quoted 23 lines
include/asm-generic/hyperv-tlfs.h +1 -0
+
  /*
   * Group B features.
   */
include/asm-generic/mshyperv.h +6 -0
index 8fe7aaab2599..85a5b8cb1702 100644
@@ -315,4 +315,10 @@ static inline enum hv_isolation_type
 hv_get_isolation_type(void)
  }
  #endif /* CONFIG_HYPERV */
 
+#if IS_ENABLED(CONFIG_HYPERV_VTL_MODE)
+u8 __init get_vtl(void);
+#else
+static inline u8 get_vtl(void) { return 0; }
+#endif
+

#endif
--
2.34.1

Signature · 2 lines
Thank you,
Roman
#30 lore raw

Re: [PATCH v3 4/7] arm64: hyperv: Boot in a Virtual Trust Level

In reply to Michael Kelley
Quoted 69 lines · Michael Kelley

From: Roman Kisel <[email protected]> Sent: Friday, July 26, 2024 3:59 PM

Quoted 68 lines

To run in the VTL mode, Hyper-V drivers have to know what VTL the system boots in, and the arm64/hyperv code does not update the variable that stores the value.

Update the variable to enable the Hyper-V drivers to boot in the VTL mode and print the VTL the code runs in.

---
  arch/arm64/hyperv/Makefile        |  1 +
  arch/arm64/hyperv/hv_vtl.c        | 13 +++++++++++++
  arch/arm64/hyperv/mshyperv.c      |  4 ++++
  arch/arm64/include/asm/mshyperv.h |  7 +++++++
  4 files changed, 25 insertions(+)

create mode 100644 arch/arm64/hyperv/hv_vtl.c

arch/arm64/hyperv/Makefile +1 -0
index 87c31c001da9..9701a837a6e1 100644
@@ -1,2 +1,3 @@
  # SPDX-License-Identifier: GPL-2.0
  obj-y		:= hv_core.o mshyperv.o
+obj-$(CONFIG_HYPERV_VTL_MODE)	+= hv_vtl.o
arch/arm64/hyperv/hv_vtl.c new file +13 -0
new file mode 100644
index 000000000000..38642b7b6be0
@@ -0,0 +1,13 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (C) 2024, Microsoft, Inc.
+ *
+ * Author : Roman Kisel <[email protected]>
+ */
+
+#include <asm/mshyperv.h>
+
+void __init hv_vtl_init_platform(void)
+{
+	pr_info("Linux runs in Hyper-V Virtual Trust Level\n");
+}
arch/arm64/hyperv/mshyperv.c +4 -0
index 341f98312667..8fd04d6e4800 100644
@@ -98,6 +98,10 @@ static int __init hyperv_init(void)
  		return ret;
  	}
 
+	/* Find the VTL */
+	ms_hyperv.vtl = get_vtl();
+	hv_vtl_init_platform();
+
  	ms_hyperv_late_init();
 
  	hyperv_initialized = true;
arch/arm64/include/asm/mshyperv.h +0 -0
index a7a3586f7cb1..63d6bb6998fc 100644
@@ -49,6 +49,13 @@ static inline u64 hv_get_msr(unsigned int reg)
  				ARM_SMCCC_OWNER_VENDOR_HYP,	\
  				HV_SMCCC_FUNC_NUMBER)
 
Quoted 6 lines

arch/arm64/include/asm/mshyperv.h static inline u64 hv_get_msr(unsigned int reg) · line 54

Quoted 3 lines
arch/arm64/include/asm/mshyperv.h +3 -0
+#ifdef CONFIG_HYPERV_VTL_MODE
+void __init hv_vtl_init_platform(void);
+int __init hv_vtl_early_init(void);

This declaration is spurious since you removed the function.

Will clean this up, thanks!

Quoted 12 lines

Michael

Quoted 10 lines
arch/arm64/include/asm/mshyperv.h +4 -1
+#else
+static inline void __init hv_vtl_init_platform(void) {}
+#endif
+
  #include <asm-generic/mshyperv.h>
 
  #define ARM_SMCCC_VENDOR_HYP_UID_HYPERV_REG_0	0x7948734d
--

2.34.1

Signature · 2 lines
Thank you,
Roman
#42 lore raw

Re: [PATCH v3 6/7] Drivers: hv: vmbus: Get the IRQ number from DT

In reply to Michael Kelley
Quoted 38 lines · Michael Kelley

From: Roman Kisel <[email protected]> Sent: Monday, July 29, 2024 9:51 AM

Quoted 24 lines · Arnd Bergmann
Quoted 21 lines · Krzysztof Kozlowski
Quoted 19 lines · Roman Kisel
Quoted 5 lines
drivers/hv/vmbus_drv.c +1 -0
@@ -2338,6 +2372,21 @@ static int vmbus_device_add(struct platform_device *pdev)
   		cur_res = &res->sibling;
   	}
 
+	/*

drivers/hv/vmbus_drv.c static int vmbus_device_add(struct platform_device *pdev) · line 2385

Quoted 10 lines
drivers/hv/vmbus_drv.c +10 -0
+	 * Hyper-V always assumes DMA cache coherency, and the DMA subsystem
+	 * might default to 'not coherent' on some architectures.
+	 * Avoid high-cost cache coherency maintenance done by the CPU.
+	 */
+#if defined(CONFIG_ARCH_HAS_SYNC_DMA_FOR_DEVICE) || \
+	defined(CONFIG_ARCH_HAS_SYNC_DMA_FOR_CPU) || \
+	defined(CONFIG_ARCH_HAS_SYNC_DMA_FOR_CPU_ALL)
+
+	if (!of_property_read_bool(np, "dma-coherent"))
+		pr_warn("Assuming cache coherent DMA transactions, no 'dma-coherent' node supplied\n");

Why do you need this property at all, if it is allways dma-coherent? Are you supporting dma-noncoherent somewhere?

Quoted 1 line
Quoted 1 line

It's just a sanity check that the DT is well-formed.

In my view, this chunk of code can be dropped entirely. The guest should believe what the Hyper-V host tells it via DT, and that includes operating in non-coherent mode. There might be some future case where non-coherent DMA is correct. In such a case, we don't want to have to come back and remove an overly aggressive sanity test from Linux kernel code.

As Arnd noted, the dma-coherent (or dma-noncoherent) property should be interpreted and applied to the device by common code. If that's not working for some reason in this case, we should investigate why not.

Quoted 9 lines

Note that the ACPI code for VMBus does the same thing -- it believes and uses whatever the _CCA property says. The exception is that there are deployed version of Hyper-V that don't set _CCA at all, contrary to the ACPI spec. So there's a hack in vmbus_acpi_add() to work around this case and force coherent_dma. But that's the only place where the current Hyper-V assumption of coherence comes into play. I sincerely hope Hyper-V ensures that the DT correctly includes dma-coherent from the start, and that we don't have to replicate the hack on the DT side.

I was replicating the _CCA hack diligently a bit much too much, agreed. This great conversation really gives me reassurance that the code doesn't have to be paranoid, and I can happily remove this if statement.

Quoted 49 lines

Michael

Quoted 47 lines
Quoted 6 lines

Since the dma-coherent property is interpreted by common code, it's not up to hv to change the default for the platform. I'm not sure if the presence of CONFIG_ARCH_HAS_SYNC_DMA_* options is the correct check to determine that an architecture defaults to noncoherent though, as the function may be needed to do something else.

I used the ifdef as the dma_coherent field is declared under these macros:

#if defined(CONFIG_ARCH_HAS_SYNC_DMA_FOR_DEVICE) || \
	defined(CONFIG_ARCH_HAS_SYNC_DMA_FOR_CPU) || \
	defined(CONFIG_ARCH_HAS_SYNC_DMA_FOR_CPU_ALL)
extern bool dma_default_coherent;
static inline bool dev_is_dma_coherent(struct device *dev)
{
	return dev->dma_coherent;
}

#else
#define dma_default_coherent true

static inline bool dev_is_dma_coherent(struct device *dev)

{
	return true;
}

i.e., there is no API to set dma_coherent. As I see it, the options are either warn the user if they forgot to add dma-coherent

if (!dev_is_dma_coherent(dev)) pr_warn("add dma-coherent to be faster\n"),

or warn and force the flag to true. Maybe just warn
the user I think now... The code will be cleaner (no need to emulate a-would-be set_dma_coherent) , and the user will
know how to make the system perform at its best.

Appreciate sharing the reservations about that piece!

Quoted 6 lines

The global "dma_default_coherent' may be a better thing to check for. This is e.g. set on powerpc64, riscv and on specific mips platforms, but it's never set on arm64 as far as I can tell.

Arnd

--
Thank you,
Roman

Signature · 2 lines
Thank you,
Roman
#52 lore raw

Re: [PATCH v3 7/7] PCI: hv: Get vPCI MSI IRQ domain from DT

In reply to Michael Kelley
Quoted 132 lines · Michael Kelley

From: Roman Kisel <[email protected]> Sent: Friday, July 26, 2024 3:59 PM

Quoted 9 lines

The hyperv-pci driver uses ACPI for MSI IRQ domain configuration on arm64. It won't be able to do that in the VTL mode where only DeviceTree can be used.

Update the hyperv-pci driver to get vPCI MSI IRQ domain in the DeviceTree case, too.

Overall, this makes sense to me, and I think it works. As you noted in the cover letter for the patch series, it's a bit messy. But see my two comments below.

Quoted 105 lines
---
  drivers/hv/vmbus_drv.c              | 23 +++++++-----
  drivers/pci/controller/pci-hyperv.c | 55 +++++++++++++++++++++++++++--
  include/linux/hyperv.h              |  2 ++
  3 files changed, 69 insertions(+), 11 deletions(-)
drivers/hv/vmbus_drv.c +15 -8
index 7eee7caff5f6..038bd9be64b7 100644
@@ -45,7 +45,8 @@ struct vmbus_dynid {
  	struct hv_vmbus_device_id id;
  };
 
-static struct device  *hv_dev;
+/* VMBus Root Device */
+static struct device  *vmbus_root_device;
 
  static int hyperv_cpuhp_online;
 
@@ -80,9 +81,15 @@ static struct resource *fb_mmio;
  static struct resource *hyperv_mmio;
  static DEFINE_MUTEX(hyperv_mmio_lock);
 
+struct device *get_vmbus_root_device(void)
+{
+	return vmbus_root_device;
+}
+EXPORT_SYMBOL_GPL(get_vmbus_root_device);
+
  static int vmbus_exists(void)
  {
-	if (hv_dev == NULL)
+	if (vmbus_root_device == NULL)
  		return -ENODEV;
 
  	return 0;
@@ -861,7 +868,7 @@ static int vmbus_dma_configure(struct device *child_device)
  	 * On x86/x64 coherence is assumed and these calls have no effect.
  	 */
  	hv_setup_dma_ops(child_device,
-		device_get_dma_attr(hv_dev) == DEV_DMA_COHERENT);
+		device_get_dma_attr(vmbus_root_device) == DEV_DMA_COHERENT);
  	return 0;
  }
 
@@ -1892,7 +1899,7 @@ int vmbus_device_register(struct hv_device *child_device_obj)
  		     &child_device_obj->channel->offermsg.offer.if_instance);
 
  	child_device_obj->device.bus = &hv_bus;
-	child_device_obj->device.parent = hv_dev;
+	child_device_obj->device.parent = vmbus_root_device;
  	child_device_obj->device.release = vmbus_device_release;
 
  	child_device_obj->device.dma_parms = &child_device_obj->dma_parms;
@@ -2253,7 +2260,7 @@ static int vmbus_acpi_add(struct platform_device *pdev)
  	struct acpi_device *ancestor;
  	struct acpi_device *device = ACPI_COMPANION(&pdev->dev);
 
-	hv_dev = &device->dev;
+	vmbus_root_device = &device->dev;
 
  	/*
  	 * Older versions of Hyper-V for ARM64 fail to include the _CCA
@@ -2342,7 +2349,7 @@ static int vmbus_device_add(struct platform_device *pdev)
  	struct device_node *np = pdev->dev.of_node;
  	int ret;
 
-	hv_dev = &pdev->dev;
+	vmbus_root_device = &pdev->dev;
 
  	ret = of_range_parser_init(&parser, np);
  	if (ret)
@@ -2675,7 +2682,7 @@ static int __init hv_acpi_init(void)
  	if (ret)
  		return ret;
 
-	if (!hv_dev) {
+	if (!vmbus_root_device) {
  		ret = -ENODEV;
  		goto cleanup;
  	}
@@ -2706,7 +2713,7 @@ static int __init hv_acpi_init(void)
 
  cleanup:
  	platform_driver_unregister(&vmbus_platform_driver);
-	hv_dev = NULL;
+	vmbus_root_device = NULL;
  	return ret;
  }
 
drivers/pci/controller/pci-hyperv.c +1 -0
index 5992280e8110..cdecefd1f9bd 100644
@@ -50,6 +50,7 @@
  #include <linux/irqdomain.h>
  #include <linux/acpi.h>
  #include <linux/sizes.h>
+#include <linux/of_irq.h>
  #include <asm/mshyperv.h>
 
  /*
@@ -887,6 +888,35 @@ static const struct irq_domain_ops hv_pci_domain_ops = {
  	.activate = hv_pci_vec_irq_domain_activate,

drivers/pci/controller/pci-hyperv.c static const struct irq_domain_ops hv_pci_domain_ops = { · line 898

Quoted 10 lines
drivers/pci/controller/pci-hyperv.c +8 -0
  };
 
+#ifdef CONFIG_OF
+
+static struct irq_domain *hv_pci_of_irq_domain_parent(void)
+{
+	struct device_node *parent;
+	struct irq_domain *domain;
+
+	parent = of_irq_find_parent(to_platform_device(get_vmbus_root_device())->dev.of_node);

I think the above can be simplified to:

Quoted 7 lines
	parent = of_irq_find_parent(get_vmbus_root_device()->of_node);

Converting the vmbus_root_device to the platform device, and then accessing the "dev" field of the platform device puts you back where you started with the vmbus_root_device. See the code in vmbus_device_add() where the vmbus_root_device is set to the dev field of the platform device.

Appreciate your help with making this code looks better! Will try your suggestion out!

Quoted 38 lines
Quoted 8 lines
drivers/pci/controller/pci-hyperv.c +8 -0
+	domain = NULL;
+	if (parent) {
+		domain = irq_find_host(parent);
+		of_node_put(parent);
+	}
+
+	/*
+	 * `domain == NULL` shouldn't happen.

drivers/pci/controller/pci-hyperv.c static const struct irq_domain_ops hv_pci_domain_ops = { · line 916

Quoted 10 lines
drivers/pci/controller/pci-hyperv.c +10 -0
+	 *
+	 * If somehow the code does end up in that state, treat this as a configuration
+	 * issue rather than a hard error, emit a warning, and let the code proceed.
+	 * The NULL parent domain is an acceptable option for the `irq_domain_create_hierarchy`
+	 * function called later.
+	 */
+	if (!domain)
+		WARN_ONCE(1, "No interrupt-parent found, check the DeviceTree data.\n");
+	return domain;
+}

Here's a thought, which may or may not be a good one: Push some or all the functionality of hv_pci_of_irq_domain_parent() into vmbus_device_add(). In the simplest case, have vmbus_device_add() store the of_node (which is already the "np" local variable) in a global static variable, and provide hv_get_vmbus_of_node() instead of get_vmbus_root_device().

The next step to consider would be to compute the parent in vmbus_device_add(), and provide hv_get_vmbus_parent_of_node() instead of hv_get_vmbus_of_node(). One difference is that vmbus_device_add() runs for both x86 and arm64, while hv_pci_of_irq_domain_parent() is for arm64 only. The parent node may not exist on x86, but maybe that isn't really a problem.

Pushing everything into vmbus_device_add() would entail doing the irq_find_host(parent) as well, and the accessor function would just return the IRQ domain. In that case, hv_pci_of_irq_domain_parent() can go away. The domain might be NULL on x86, but that's OK because the accessor function won't be called on x86.

Quoted 5 lines

Maybe there's a snag that prevents this idea from working well, particularly on x86 where the domain will be NULL. But if it works, it seems slightly less messy to me, though that is a judgment call. I'll leave it to you to decide, and I'm OK either way. :-)

I'll explore your great idea, thanks for sharing! Since v1 this code has improved quite a bit thanks to your suggestions :)

Quoted 57 lines

Michael

Quoted 55 lines
drivers/pci/controller/pci-hyperv.c +25 -3
+
+#endif
+
  static int hv_pci_irqchip_init(void)
  {
  	static struct hv_pci_chip_data *chip_data;
@@ -906,10 +936,29 @@ static int hv_pci_irqchip_init(void)
  	 * IRQ domain once enabled, should not be removed since there is no
  	 * way to ensure that all the corresponding devices are also gone and
  	 * no interrupts will be generated.
+	 *
+	 * In the ACPI case, the parent IRQ domain is supplied by the ACPI
+	 * subsystem, and it is the default GSI domain pointing to the GIC.
+	 * Neither is available outside of the ACPI subsystem, cannot avoid
+	 * the messy ifdef below.
+	 * There is apparently no such default in the OF subsystem, and
+	 * `hv_pci_of_irq_domain_parent` finds the parent IRQ domain that
+	 * points to the GIC as well.
+	 * None of these two cases reaches for the MSI parent domain.
  	 */
-	hv_msi_gic_irq_domain = acpi_irq_create_hierarchy(0, HV_PCI_MSI_SPI_NR,
-							  fn, &hv_pci_domain_ops,
-							  chip_data);
+#ifdef CONFIG_ACPI
+	if (!acpi_disabled)
+		hv_msi_gic_irq_domain = acpi_irq_create_hierarchy(0, HV_PCI_MSI_SPI_NR,
+			fn, &hv_pci_domain_ops,
+			chip_data);
+#endif
+#if defined(CONFIG_OF)
+	if (!hv_msi_gic_irq_domain)
+		hv_msi_gic_irq_domain = irq_domain_create_hierarchy(
+			hv_pci_of_irq_domain_parent(), 0, HV_PCI_MSI_SPI_NR,
+			fn, &hv_pci_domain_ops,
+			chip_data);
+#endif
 
  	if (!hv_msi_gic_irq_domain) {
  		pr_err("Failed to create Hyper-V arm64 vPCI MSI IRQ domain\n");
include/linux/hyperv.h +2 -1
index 5e39baa7f6cb..b4aa1f579a97 100644
@@ -1346,6 +1346,8 @@ static inline void *hv_get_drvdata(struct hv_device *dev)
  	return dev_get_drvdata(&dev->device);
  }
 
+struct device *get_vmbus_root_device(void);
+
  struct hv_ring_buffer_debug_info {
  	u32 current_interrupt_mask;
  	u32 current_read_index;
--

2.34.1

Signature · 2 lines
Thank you,
Roman
#5 lore raw

Re: [PATCH v3 1/7] arm64: hyperv: Use SMC to detect hypervisor presence

In reply to Michael Kelley
Quoted 23 lines · Michael Kelley

From: Roman Kisel <[email protected]> Sent: Friday, July 26, 2024 3:59 PM

Quoted 8 lines

The arm64 Hyper-V startup path relies on ACPI to detect running under a Hyper-V compatible hypervisor. That doesn't work on non-ACPI systems.

Hoist the ACPI detection logic into a separate function, use the new SMC added recently to Hyper-V to use in the non-ACPI case.

Wording seems slightly messed up. Perhaps:

Hoist the ACPI detection logic into a separate function. Then use the new SMC added recently to Hyper-V in the non-ACPI case.

Also, the phrase "the new SMC" seems a bit off to me. The "Terms and Abbreviations" section of the SMCCC specification defines "SMC" as an instruction:

Secure Monitor Call. An Arm assembler instruction that causes an exception that is taken synchronously into EL3.

Quoted 7 lines

More precisely, I think you mean a SMC "function identifier" that is newly implemented by Hyper-V. And the function identifier itself isn't new; it's the Hyper-V implementation that's new.

Similar comment applies in the cover letter for this patch set, and perhaps to the Subject line of this patch.

Will fix the wording, appreciate your help with bringing this into the best shape possible!

Quoted 72 lines
Quoted 72 lines
---
  arch/arm64/hyperv/mshyperv.c      | 36 ++++++++++++++++++++++++++-----
  arch/arm64/include/asm/mshyperv.h |  5 +++++
  2 files changed, 36 insertions(+), 5 deletions(-)
arch/arm64/hyperv/mshyperv.c +31 -5
index b1a4de4eee29..341f98312667 100644
@@ -27,6 +27,34 @@ int hv_get_hypervisor_version(union hv_hypervisor_version_info *info)
  	return 0;
  }
 
+static bool hyperv_detect_via_acpi(void)
+{
+	if (acpi_disabled)
+		return false;
+#if IS_ENABLED(CONFIG_ACPI)
+	/* Hypervisor ID is only available in ACPI v6+. */
+	if (acpi_gbl_FADT.header.revision < 6)
+		return false;
+	return strncmp((char *)&acpi_gbl_FADT.hypervisor_id, "MsHyperV", 8) == 0;
+#else
+	return false;
+#endif
+}
+
+static bool hyperv_detect_via_smc(void)
+{
+	struct arm_smccc_res res = {};
+
+	if (arm_smccc_1_1_get_conduit() != SMCCC_CONDUIT_HVC)
+		return false;
+	arm_smccc_1_1_hvc(ARM_SMCCC_VENDOR_HYP_CALL_UID_FUNC_ID, &res);
+
+	return res.a0 == ARM_SMCCC_VENDOR_HYP_UID_HYPERV_REG_0 &&
+		res.a1 == ARM_SMCCC_VENDOR_HYP_UID_HYPERV_REG_1 &&
+		res.a2 == ARM_SMCCC_VENDOR_HYP_UID_HYPERV_REG_2 &&
+		res.a3 == ARM_SMCCC_VENDOR_HYP_UID_HYPERV_REG_3;
+}
+
  static int __init hyperv_init(void)
  {
  	struct hv_get_vp_registers_output	result;
@@ -35,13 +63,11 @@ static int __init hyperv_init(void)
 
  	/*
  	 * Allow for a kernel built with CONFIG_HYPERV to be running in
-	 * a non-Hyper-V environment, including on DT instead of ACPI.
+	 * a non-Hyper-V environment.
+	 *
  	 * In such cases, do nothing and return success.
  	 */
-	if (acpi_disabled)
-		return 0;
-
-	if (strncmp((char *)&acpi_gbl_FADT.hypervisor_id, "MsHyperV", 8))
+	if (!hyperv_detect_via_acpi() && !hyperv_detect_via_smc())
  		return 0;
 
  	/* Setup the guest ID */
arch/arm64/include/asm/mshyperv.h +0 -0
index a975e1a689dd..a7a3586f7cb1 100644
@@ -51,4 +51,9 @@ static inline u64 hv_get_msr(unsigned int reg)
 
  #include <asm-generic/mshyperv.h>
 
Quoted 15 lines

arch/arm64/include/asm/mshyperv.h static inline u64 hv_get_msr(unsigned int reg) · line 58

Quoted 5 lines
arch/arm64/include/asm/mshyperv.h +5 -0
+#define ARM_SMCCC_VENDOR_HYP_UID_HYPERV_REG_0	0x7948734d
+#define ARM_SMCCC_VENDOR_HYP_UID_HYPERV_REG_1	0x56726570
+#define ARM_SMCCC_VENDOR_HYP_UID_HYPERV_REG_2	0
+#define ARM_SMCCC_VENDOR_HYP_UID_HYPERV_REG_3	0
+

Section 6.2 of the SMCCC specification says that the "Call UID Query" returns a UUID. The above #defines look like an ASCII string is being returned. Arguably the ASCII string can be treated as a set of 128 bits just like a UUID, but it doesn't meet the spirit of the spec. Can Hyper-V be changed to return a real UUID? While the distinction probably won't make a material difference here, we've had problems in the past with Hyper-V doing slightly weird things that later caused unexpected trouble. Please just get it right. :-)

The above values don't violate anything in the spec, and I think it would be hard to give an example of what can be broken in the world by using the above values. I do understand what you're saying when you mention the UIDs & the spirit of the spec. Put on the quantitative footing, the Shannon entropy of these values is much lower than that of an UID. A cursory search in the kernel tree does turn up other UIDs that don't look too random.

As that is implemented only in the non-release versions, hardly someone has taken a dependency on the specific values in their production code. I guess that can be changed without causing any disturbance to the customers, will report of any concerns though.

Quoted 6 lines

Michael

Quoted 4 lines
arch/arm64/include/asm/mshyperv.h +0 -1
  #endif
--

2.34.1

Signature · 2 lines
Thank you,
Roman
#17 lore raw

RE: [PATCH v3 2/7] Drivers: hv: Enable VTL mode for arm64

In reply to Roman Kisel

From: Roman Kisel <[email protected]> Sent: Monday, August 5, 2024 8:24 AM

Quoted 58 lines · Saurabh Singh Sengar
Quoted 2 lines · Michael Kelley
Quoted 1 line

From: Roman Kisel <[email protected]> Sent: Friday, July 26, 2024 3:59

PM

Quoted 51 lines
Quoted 35 lines
Quoted 35 lines

Kconfig dependencies for arm64 guests on Hyper-V require that be ACPI enabled, and limit VTL mode to x86/x64. To enable VTL mode on arm64 as well, update the dependencies. Since VTL mode requires DeviceTree instead of ACPI, don't require arm64 guests on Hyper-V to have ACPI.

---
  drivers/hv/Kconfig | 6 +++---
  1 file changed, 3 insertions(+), 3 deletions(-)
drivers/hv/Kconfig +2 -2
index 862c47b191af..a5cd1365e248 100644
@@ -5,7 +5,7 @@ menu "Microsoft Hyper-V guest support"
  config HYPERV
  	tristate "Microsoft Hyper-V client drivers"
  	depends on (X86 && X86_LOCAL_APIC && HYPERVISOR_GUEST) \
-		|| (ACPI && ARM64 && !CPU_BIG_ENDIAN)
+		|| (ARM64 && !CPU_BIG_ENDIAN)
  	select PARAVIRT
  	select X86_HV_CALLBACK_VECTOR if X86
  	select OF_EARLY_FLATTREE if OF
@@ -15,7 +15,7 @@ config HYPERV
 
  config HYPERV_VTL_MODE
  	bool "Enable Linux to boot in VTL context"
-	depends on X86_64 && HYPERV
+	depends on HYPERV
  	depends on SMP
  	default n
  	help
@@ -31,7 +31,7 @@ config HYPERV_VTL_MODE
 
Quoted 15 lines
Quoted 10 lines
drivers/hv/Kconfig +1 -2
  	  Select this option to build a Linux kernel to run at a VTL other than
  	  the normal VTL0, which currently is only VTL2.  This option
-	  initializes the x86 platform for VTL2, and adds the ability to boot
+	  initializes the kernel to run in VTL2, and adds the ability to boot
  	  secondary CPUs directly into 64-bit context as required for VTLs other
  	  than 0.  A kernel built with this option must run at VTL2, and will
  	  not run as a normal guest.
--

2.34.1

In v2 of this patch, I suggested [1] making a couple additional minor changes so that kernels built *without* HYPER_VTL_MODE would still require ACPI. Did that suggestion not work out? If that's the case, I'm curious about what goes wrong.

Quoted 2 lines

Hi Michael/Roman,
I was considering making HYPERV_VTL_MODE depend on CONFIG_OF. That should

Quoted 9 lines

address

Quoted 2 lines

above concern as well. Do you see any potential issue with it.

Michael,

I ran into a pretty gnarly recursive dependencies which in all fairness might stem from not being fluent enough in the Kconfig language. Any help of how to approach implementing your idea would be greatly appreciated!

This is what I had in mind:

drivers/hv/Kconfig +3 -2
@@ -5,7 +5,8 @@ menu "Microsoft Hyper-V guest support"
 config HYPERV
        tristate "Microsoft Hyper-V client drivers"
        depends on (X86 && X86_LOCAL_APIC && HYPERVISOR_GUEST) \
-               || (ACPI && ARM64 && !CPU_BIG_ENDIAN)
+               || (ARM64 && !CPU_BIG_ENDIAN)
+       depends on (ACPI || HYPERV_VTL_MODE)
        select PARAVIRT
        select X86_HV_CALLBACK_VECTOR if X86
        select OF_EARLY_FLATTREE if OF
@@ -15,7 +16,7 @@ config HYPERV
 
 config HYPERV_VTL_MODE
        bool "Enable Linux to boot in VTL context"
-       depends on X86_64 && HYPERV
+       depends on X86_64
        depends on SMP
        default n
        help

HYPERV_VTL_MODE can now be selected independently of HYPERV. The existing code should be such that even if someone is building a random config and gets HYPERV_VTL_MODE without HYPERV, the kernel will build and run in a non-Hyper-V environment and isn't broken somehow.

For HYPERV to be selected, either ACPI must already be selected, or HYPERV_VTL_MODE must already be selected. So "normal" kernels are still enforced to require ACPI. But if building with HYPERV_VTL_MODE, then ACPI is optional.

Saurabh's idea of adding "depends on OF" to HYPERV_VTL_MODE should also work with these changes.

I haven't fully tested the above with all the relevant combinations on both x86 and arm64, but I think the logic makes sense.

Michael

#24 lore raw

RE: [PATCH v3 3/7] Drivers: hv: Provide arch-neutral implementation of get_vtl()

In reply to Roman Kisel

From: Roman Kisel <[email protected]> Sent: Monday, August 5, 2024 9:20 AM

Quoted 134 lines · Michael Kelley
Quoted 1 line

From: Roman Kisel <[email protected]> Sent: Friday, July 26, 2024 3:59

PM

Quoted 122 lines
Quoted 105 lines

To run in the VTL mode, Hyper-V drivers have to know what VTL the system boots in, and the arm64/hyperv code does not have the means to compute that.

Refactor the code to hoist the function that detects VTL, make it arch-neutral to be able to employ it to get the VTL on arm64. Fix the hypercall output address in get_vtl(void) not to overlap with the hypercall input area to adhere to the Hyper-V TLFS.

---
  arch/x86/hyperv/hv_init.c          | 34 ---------------------
  arch/x86/include/asm/hyperv-tlfs.h |  7 -----
  drivers/hv/hv_common.c             | 47 ++++++++++++++++++++++++++++--
  include/asm-generic/hyperv-tlfs.h  |  7 +++++
  include/asm-generic/mshyperv.h     |  6 ++++
  5 files changed, 58 insertions(+), 43 deletions(-)
arch/x86/hyperv/hv_init.c +0 -34
index 17a71e92a343..c350fa05ee59 100644
@@ -413,40 +413,6 @@ static void __init hv_get_partition_id(void)
  	local_irq_restore(flags);
  }
 
-#if IS_ENABLED(CONFIG_HYPERV_VTL_MODE)
-static u8 __init get_vtl(void)
-{
-	u64 control = HV_HYPERCALL_REP_COMP_1 | HVCALL_GET_VP_REGISTERS;
-	struct hv_get_vp_registers_input *input;
-	struct hv_get_vp_registers_output *output;
-	unsigned long flags;
-	u64 ret;
-
-	local_irq_save(flags);
-	input = *this_cpu_ptr(hyperv_pcpu_input_arg);
-	output = (struct hv_get_vp_registers_output *)input;
-
-	memset(input, 0, struct_size(input, element, 1));
-	input->header.partitionid = HV_PARTITION_ID_SELF;
-	input->header.vpindex = HV_VP_INDEX_SELF;
-	input->header.inputvtl = 0;
-	input->element[0].name0 = HV_X64_REGISTER_VSM_VP_STATUS;
-
-	ret = hv_do_hypercall(control, input, output);
-	if (hv_result_success(ret)) {
-		ret = output->as64.low & HV_X64_VTL_MASK;
-	} else {
-		pr_err("Failed to get VTL(error: %lld) exiting...\n", ret);
-		BUG();
-	}
-
-	local_irq_restore(flags);
-	return ret;
-}
-#else
-static inline u8 get_vtl(void) { return 0; }
-#endif
-
  /*
   * This function is to be invoked early in the boot sequence after the
   * hypervisor has been detected.
arch/x86/include/asm/hyperv-tlfs.h +0 -7
index 3787d26810c1..9ee68eb8e6ff 100644
@@ -309,13 +309,6 @@ enum hv_isolation_type {
  #define HV_MSR_STIMER0_CONFIG	(HV_X64_MSR_STIMER0_CONFIG)
  #define HV_MSR_STIMER0_COUNT	(HV_X64_MSR_STIMER0_COUNT)
 
-/*
- * Registers are only accessible via HVCALL_GET_VP_REGISTERS hvcall and
- * there is not associated MSR address.
- */
-#define	HV_X64_REGISTER_VSM_VP_STATUS	0x000D0003
-#define	HV_X64_VTL_MASK			GENMASK(3, 0)
-
  /* Hyper-V memory host visibility */
  enum hv_mem_host_visibility {
  	VMBUS_PAGE_NOT_VISIBLE		= 0,
drivers/hv/hv_common.c +5 -2
index 9c452bfbd571..7d6c1523b0b5 100644
@@ -339,8 +339,8 @@ int __init hv_common_init(void)
  	hyperv_pcpu_input_arg = alloc_percpu(void  *);
  	BUG_ON(!hyperv_pcpu_input_arg);
 
-	/* Allocate the per-CPU state for output arg for root */
-	if (hv_root_partition) {
+	/* Allocate the per-CPU state for output arg for root or a VTL */
+	if (hv_root_partition || IS_ENABLED(CONFIG_HYPERV_VTL_MODE)) {
  		hyperv_pcpu_output_arg = alloc_percpu(void *);
  		BUG_ON(!hyperv_pcpu_output_arg);
  	}
@@ -656,3 +656,46 @@ u64 __weak hv_tdx_hypercall(u64 control, u64 param1, u64 param2)
  	return HV_STATUS_INVALID_PARAMETER;
  }
  EXPORT_SYMBOL_GPL(hv_tdx_hypercall);
+
+#if IS_ENABLED(CONFIG_HYPERV_VTL_MODE)
+u8 __init get_vtl(void)

drivers/hv/hv_common.c u64 __weak hv_tdx_hypercall(u64 control, u64 param1, u64 param2) · line 671

Quoted 10 lines
drivers/hv/hv_common.c +10 -0
+{
+	u64 control = HV_HYPERCALL_REP_COMP_1 | HVCALL_GET_VP_REGISTERS;
+	struct hv_get_vp_registers_input *input;
+	struct hv_get_vp_registers_output *output;
+	unsigned long flags;
+	u64 ret;
+
+	local_irq_save(flags);
+	input = *this_cpu_ptr(hyperv_pcpu_input_arg);
+	output = *this_cpu_ptr(hyperv_pcpu_output_arg);

Rather than use the hyperv_pcpu_output_arg here, it's OK to use a different area of the hyperv_pcpu_input_arg page. For example,

	output = (void *)input + HV_HYP_PAGE_SIZE/2;
Quoted 8 lines

The TLFS does not require that the input and output be in separate pages.

While using the hyperv_pcpu_output_arg is conceptually a bit cleaner, doing so requires allocating a 4K page per CPU that is not otherwise used. The VTL 2 code wants to be frugal with memory, and this seems like a good step in that direction. :-)

Quoted 8 lines

I agree on the both counts: the code looks conceptually cleaner now and VTL2 wants to be frugal with memory, esp that the output hypercall page is per-CPU so we have O(n) as the CPU count increases. Still, the output page will be needed for VTL2 (say to get/set registers just as done here). That said, with this patch we can achieve both the conceptual cleanliness and being ready to grow more on the primitives being built out in the VTL support patches.

Could you elaborate further on why the output page is needed for VTL2? The get/set register hypercalls can operate with just the input page (again, splitting it into two halves for input and output args) as long as the number of registers acted on by a single hypercall isn't more than a few dozen.

If you really *do* need the output page in VTL2 for other reasons that I'm not aware of, then my suggestion isn't relevant and there's no memory to be saved.

Michael

#6 lore raw

RE: [PATCH v3 1/7] arm64: hyperv: Use SMC to detect hypervisor presence

In reply to Roman Kisel

From: Roman Kisel <[email protected]> Sent: Monday, August 5, 2024 9:51 AM

[snip]

Quoted 24 lines
Quoted 9 lines
Quoted 9 lines
arch/arm64/include/asm/mshyperv.h +0 -0
index a975e1a689dd..a7a3586f7cb1 100644
@@ -51,4 +51,9 @@ static inline u64 hv_get_msr(unsigned int reg)
 
  #include <asm-generic/mshyperv.h>
 
Quoted 15 lines

arch/arm64/include/asm/mshyperv.h static inline u64 hv_get_msr(unsigned int reg) · line 58

Quoted 5 lines
arch/arm64/include/asm/mshyperv.h +5 -0
+#define ARM_SMCCC_VENDOR_HYP_UID_HYPERV_REG_0	0x7948734d
+#define ARM_SMCCC_VENDOR_HYP_UID_HYPERV_REG_1	0x56726570
+#define ARM_SMCCC_VENDOR_HYP_UID_HYPERV_REG_2	0
+#define ARM_SMCCC_VENDOR_HYP_UID_HYPERV_REG_3	0
+

Section 6.2 of the SMCCC specification says that the "Call UID Query" returns a UUID. The above #defines look like an ASCII string is being returned. Arguably the ASCII string can be treated as a set of 128 bits just like a UUID, but it doesn't meet the spirit of the spec. Can Hyper-V be changed to return a real UUID? While the distinction probably won't make a material difference here, we've had problems in the past with Hyper-V doing slightly weird things that later caused unexpected trouble. Please just get it right. :-)

Quoted 3 lines

The above values don't violate anything in the spec, and I think it would be hard to give an example of what can be broken in the world by using the above values.

Agreed. However, note that UUIDs *do* have some internal structure. See https://www.uuidtools.com/decode, for example. The UUID above is:

4d734879-7065-7256-0000-000000000000

It has a version digit of "7", which is "unknown". I'm no expert on UUIDs, but apparently the "7" isn't necessarily invalid, though perhaps a bit unexpected.

Quoted 10 lines

I do understand what you're saying when you
mention the UIDs & the spirit of the spec. Put on the quantitative footing, the Shannon entropy of these values is much lower than that of an UID. A cursory search in the kernel tree does turn up other UIDs that don't look too random.

As that is implemented only in the non-release versions, hardly someone has taken a dependency on the specific values in their production code. I guess that can be changed without causing any disturbance to the customers, will report of any concerns though.

My last thought is that when dealing with the open source world, and with published standards, it's usually best to do what's normal and expected, rather than trying to make the case that something unexpected is allowed by the spec. Doing the latter can come back to bite you in completely unexpected ways. :-)

With that, I won't make any further comments on the topic. You do whatever you can do in working with Hyper-V. Either way, it won't be a blocker to my giving "Reviewed-by" on the next version of the patch.

Michael

#7 lore raw

Re: [PATCH v3 1/7] arm64: hyperv: Use SMC to detect hypervisor presence

Quoted 32 lines · Michael Kelley

From: Roman Kisel <[email protected]> Sent: Monday, August 5, 2024 9:51 AM

[snip]

Quoted 24 lines
Quoted 9 lines
Quoted 9 lines
arch/arm64/include/asm/mshyperv.h +0 -0
index a975e1a689dd..a7a3586f7cb1 100644
@@ -51,4 +51,9 @@ static inline u64 hv_get_msr(unsigned int reg)
 
   #include <asm-generic/mshyperv.h>
 
Quoted 15 lines

arch/arm64/include/asm/mshyperv.h static inline u64 hv_get_msr(unsigned int reg) · line 58

Quoted 5 lines
arch/arm64/include/asm/mshyperv.h +5 -0
+#define ARM_SMCCC_VENDOR_HYP_UID_HYPERV_REG_0	0x7948734d
+#define ARM_SMCCC_VENDOR_HYP_UID_HYPERV_REG_1	0x56726570
+#define ARM_SMCCC_VENDOR_HYP_UID_HYPERV_REG_2	0
+#define ARM_SMCCC_VENDOR_HYP_UID_HYPERV_REG_3	0
+

Section 6.2 of the SMCCC specification says that the "Call UID Query" returns a UUID. The above #defines look like an ASCII string is being returned. Arguably the ASCII string can be treated as a set of 128 bits just like a UUID, but it doesn't meet the spirit of the spec. Can Hyper-V be changed to return a real UUID? While the distinction probably won't make a material difference here, we've had problems in the past with Hyper-V doing slightly weird things that later caused unexpected trouble. Please just get it right. :-)

Quoted 3 lines

The above values don't violate anything in the spec, and I think it would be hard to give an example of what can be broken in the world by using the above values.

Quoted 9 lines

Agreed. However, note that UUIDs *do* have some internal structure. See https://www.uuidtools.com/decode, for example. The UUID above is:

4d734879-7065-7256-0000-000000000000

It has a version digit of "7", which is "unknown". I'm no expert on UUIDs, but apparently the "7" isn't necessarily invalid, though perhaps a bit unexpected.

Understood! I made an assumption that's just an array of random bytes. Thank you very much for explaining that to me :)

Quoted 11 lines
Quoted 10 lines

I do understand what you're saying when you
mention the UIDs & the spirit of the spec. Put on the quantitative footing, the Shannon entropy of these values is much lower than that of an UID. A cursory search in the kernel tree does turn up other UIDs that don't look too random.

As that is implemented only in the non-release versions, hardly someone has taken a dependency on the specific values in their production code. I guess that can be changed without causing any disturbance to the customers, will report of any concerns though.

Quoted 6 lines

My last thought is that when dealing with the open source world, and with published standards, it's usually best to do what's normal and expected, rather than trying to make the case that something unexpected is allowed by the spec. Doing the latter can come back to bite you in completely unexpected ways. :-)

Agreed, thanks for the discussion! I've discussed changing the values, and a pull request to the hypervisor has been put up for that. So far, going well.

Quoted 5 lines

With that, I won't make any further comments on the topic. You do whatever you can do in working with Hyper-V. Either way, it won't be a blocker to my giving "Reviewed-by" on the next version of the patch.

Will be a badge of honor to me! Again, appreciate your words of advise and caution, and helping in bringing these patches into the best shape possible!

Quoted 1 line

Michael

Signature · 2 lines
Thank you,
Roman
#25 lore raw

Re: [PATCH v3 3/7] Drivers: hv: Provide arch-neutral implementation of get_vtl()

In reply to Michael Kelley
Quoted 150 lines · Michael Kelley

From: Roman Kisel <[email protected]> Sent: Monday, August 5, 2024 9:20 AM

Quoted 134 lines · Michael Kelley
Quoted 1 line

From: Roman Kisel <[email protected]> Sent: Friday, July 26, 2024 3:59

PM

Quoted 122 lines
Quoted 105 lines

To run in the VTL mode, Hyper-V drivers have to know what VTL the system boots in, and the arm64/hyperv code does not have the means to compute that.

Refactor the code to hoist the function that detects VTL, make it arch-neutral to be able to employ it to get the VTL on arm64. Fix the hypercall output address in get_vtl(void) not to overlap with the hypercall input area to adhere to the Hyper-V TLFS.

---
   arch/x86/hyperv/hv_init.c          | 34 ---------------------
   arch/x86/include/asm/hyperv-tlfs.h |  7 -----
   drivers/hv/hv_common.c             | 47 ++++++++++++++++++++++++++++--
   include/asm-generic/hyperv-tlfs.h  |  7 +++++
   include/asm-generic/mshyperv.h     |  6 ++++
   5 files changed, 58 insertions(+), 43 deletions(-)
arch/x86/hyperv/hv_init.c +0 -34
index 17a71e92a343..c350fa05ee59 100644
@@ -413,40 +413,6 @@ static void __init hv_get_partition_id(void)
   	local_irq_restore(flags);
   }
 
-#if IS_ENABLED(CONFIG_HYPERV_VTL_MODE)
-static u8 __init get_vtl(void)
-{
-	u64 control = HV_HYPERCALL_REP_COMP_1 | HVCALL_GET_VP_REGISTERS;
-	struct hv_get_vp_registers_input *input;
-	struct hv_get_vp_registers_output *output;
-	unsigned long flags;
-	u64 ret;
-
-	local_irq_save(flags);
-	input = *this_cpu_ptr(hyperv_pcpu_input_arg);
-	output = (struct hv_get_vp_registers_output *)input;
-
-	memset(input, 0, struct_size(input, element, 1));
-	input->header.partitionid = HV_PARTITION_ID_SELF;
-	input->header.vpindex = HV_VP_INDEX_SELF;
-	input->header.inputvtl = 0;
-	input->element[0].name0 = HV_X64_REGISTER_VSM_VP_STATUS;
-
-	ret = hv_do_hypercall(control, input, output);
-	if (hv_result_success(ret)) {
-		ret = output->as64.low & HV_X64_VTL_MASK;
-	} else {
-		pr_err("Failed to get VTL(error: %lld) exiting...\n", ret);
-		BUG();
-	}
-
-	local_irq_restore(flags);
-	return ret;
-}
-#else
-static inline u8 get_vtl(void) { return 0; }
-#endif
-
   /*
    * This function is to be invoked early in the boot sequence after the
    * hypervisor has been detected.
arch/x86/include/asm/hyperv-tlfs.h +0 -7
index 3787d26810c1..9ee68eb8e6ff 100644
@@ -309,13 +309,6 @@ enum hv_isolation_type {
   #define HV_MSR_STIMER0_CONFIG	(HV_X64_MSR_STIMER0_CONFIG)
   #define HV_MSR_STIMER0_COUNT	(HV_X64_MSR_STIMER0_COUNT)
 
-/*
- * Registers are only accessible via HVCALL_GET_VP_REGISTERS hvcall and
- * there is not associated MSR address.
- */
-#define	HV_X64_REGISTER_VSM_VP_STATUS	0x000D0003
-#define	HV_X64_VTL_MASK			GENMASK(3, 0)
-
   /* Hyper-V memory host visibility */
   enum hv_mem_host_visibility {
   	VMBUS_PAGE_NOT_VISIBLE		= 0,
drivers/hv/hv_common.c +5 -2
index 9c452bfbd571..7d6c1523b0b5 100644
@@ -339,8 +339,8 @@ int __init hv_common_init(void)
   	hyperv_pcpu_input_arg = alloc_percpu(void  *);
   	BUG_ON(!hyperv_pcpu_input_arg);
 
-	/* Allocate the per-CPU state for output arg for root */
-	if (hv_root_partition) {
+	/* Allocate the per-CPU state for output arg for root or a VTL */
+	if (hv_root_partition || IS_ENABLED(CONFIG_HYPERV_VTL_MODE)) {
   		hyperv_pcpu_output_arg = alloc_percpu(void *);
   		BUG_ON(!hyperv_pcpu_output_arg);
   	}
@@ -656,3 +656,46 @@ u64 __weak hv_tdx_hypercall(u64 control, u64 param1, u64 param2)
   	return HV_STATUS_INVALID_PARAMETER;
   }
   EXPORT_SYMBOL_GPL(hv_tdx_hypercall);
+
+#if IS_ENABLED(CONFIG_HYPERV_VTL_MODE)
+u8 __init get_vtl(void)

drivers/hv/hv_common.c u64 __weak hv_tdx_hypercall(u64 control, u64 param1, u64 param2) · line 671

Quoted 10 lines
drivers/hv/hv_common.c +10 -0
+{
+	u64 control = HV_HYPERCALL_REP_COMP_1 | HVCALL_GET_VP_REGISTERS;
+	struct hv_get_vp_registers_input *input;
+	struct hv_get_vp_registers_output *output;
+	unsigned long flags;
+	u64 ret;
+
+	local_irq_save(flags);
+	input = *this_cpu_ptr(hyperv_pcpu_input_arg);
+	output = *this_cpu_ptr(hyperv_pcpu_output_arg);

Rather than use the hyperv_pcpu_output_arg here, it's OK to use a different area of the hyperv_pcpu_input_arg page. For example,

	output = (void *)input + HV_HYP_PAGE_SIZE/2;
Quoted 8 lines

The TLFS does not require that the input and output be in separate pages.

While using the hyperv_pcpu_output_arg is conceptually a bit cleaner, doing so requires allocating a 4K page per CPU that is not otherwise used. The VTL 2 code wants to be frugal with memory, and this seems like a good step in that direction. :-)

Quoted 8 lines

I agree on the both counts: the code looks conceptually cleaner now and VTL2 wants to be frugal with memory, esp that the output hypercall page is per-CPU so we have O(n) as the CPU count increases. Still, the output page will be needed for VTL2 (say to get/set registers just as done here). That said, with this patch we can achieve both the conceptual cleanliness and being ready to grow more on the primitives being built out in the VTL support patches.

Could you elaborate further on why the output page is needed for VTL2? The get/set register hypercalls can operate with just the input page (again, splitting it into two halves for input and output args) as long as the number of registers acted on by a single hypercall isn't more than a few dozen.

Quoted 3 lines

If you really *do* need the output page in VTL2 for other reasons that I'm not aware of, then my suggestion isn't relevant and there's no memory to be saved.

VTL2 might potentially use any hypercalls being in some sense an exclave of the hypervisor living inside the guest quite similarly to the VBS/VTL1/SecureKernel.

The tradeoff here would be to save a page per processor at the cost of specializing the hypercall issuing code that would use a part of the input page to save memory (quite likely limiting which hypercalls can be used), or use the common implementation at the cost of spending one more page per processor. Less code means less maintenance usually so seems beneficial to choose the latter option although at the cost of using more memory.

Quoted 2 lines

Michael

Signature · 2 lines
Thank you,
Roman
#18 lore raw

RE: [PATCH v3 2/7] Drivers: hv: Enable VTL mode for arm64

In reply to Michael Kelley
Quoted 73 lines
Quoted 58 lines · Saurabh Singh Sengar
Quoted 2 lines · Michael Kelley
Quoted 1 line

From: Roman Kisel <[email protected]> Sent: Friday, July 26, 2024 3:59

PM

Quoted 51 lines
Quoted 35 lines
Quoted 35 lines

Kconfig dependencies for arm64 guests on Hyper-V require that be ACPI enabled, and limit VTL mode to x86/x64. To enable VTL mode on arm64 as well, update the dependencies. Since VTL mode requires DeviceTree instead of ACPI, don't require arm64 guests on Hyper-V to have ACPI.

---
  drivers/hv/Kconfig | 6 +++---
  1 file changed, 3 insertions(+), 3 deletions(-)
drivers/hv/Kconfig +2 -2
index 862c47b191af..a5cd1365e248 100644
@@ -5,7 +5,7 @@ menu "Microsoft Hyper-V guest support"
  config HYPERV
  	tristate "Microsoft Hyper-V client drivers"
  	depends on (X86 && X86_LOCAL_APIC && HYPERVISOR_GUEST) \
-		|| (ACPI && ARM64 && !CPU_BIG_ENDIAN)
+		|| (ARM64 && !CPU_BIG_ENDIAN)
  	select PARAVIRT
  	select X86_HV_CALLBACK_VECTOR if X86
  	select OF_EARLY_FLATTREE if OF
@@ -15,7 +15,7 @@ config HYPERV
 
  config HYPERV_VTL_MODE
  	bool "Enable Linux to boot in VTL context"
-	depends on X86_64 && HYPERV
+	depends on HYPERV
  	depends on SMP
  	default n
  	help
@@ -31,7 +31,7 @@ config HYPERV_VTL_MODE
 
Quoted 15 lines
Quoted 10 lines
drivers/hv/Kconfig +1 -2
  	  Select this option to build a Linux kernel to run at a VTL other than
  	  the normal VTL0, which currently is only VTL2.  This option
-	  initializes the x86 platform for VTL2, and adds the ability to boot
+	  initializes the kernel to run in VTL2, and adds the ability to boot
  	  secondary CPUs directly into 64-bit context as required for VTLs other
  	  than 0.  A kernel built with this option must run at VTL2, and will
  	  not run as a normal guest.
--

2.34.1

In v2 of this patch, I suggested [1] making a couple additional minor changes so that kernels built *without* HYPER_VTL_MODE would still require ACPI. Did that suggestion not work out? If that's the case, I'm curious about what goes wrong.

Quoted 2 lines

Hi Michael/Roman,
I was considering making HYPERV_VTL_MODE depend on CONFIG_OF. That should

Quoted 9 lines

address

Quoted 2 lines

above concern as well. Do you see any potential issue with it.

Michael,

I ran into a pretty gnarly recursive dependencies which in all fairness might stem from not being fluent enough in the Kconfig language. Any help of how to approach implementing your idea would be greatly appreciated!

This is what I had in mind:

drivers/hv/Kconfig +0 -0
@@ -5,7 +5,8 @@ menu "Microsoft Hyper-V guest support"
Quoted 1 line

config HYPERV

> tristate "Microsoft Hyper-V client drivers"
> depends on (X86 && X86_LOCAL_APIC && HYPERVISOR_GUEST) \

drivers/hv/Kconfig menu "Microsoft Hyper-V guest support" · line 9

Quoted 3 lines
drivers/hv/Kconfig +2 -1
-               || (ACPI && ARM64 && !CPU_BIG_ENDIAN)
+               || (ARM64 && !CPU_BIG_ENDIAN)
+       depends on (ACPI || HYPERV_VTL_MODE)

> select PARAVIRT
> select X86_HV_CALLBACK_VECTOR if X86
> select OF_EARLY_FLATTREE if OF

Quoted 3 lines
drivers/hv/Kconfig +0 -0
@@ -15,7 +16,7 @@ config HYPERV
 

config HYPERV_VTL_MODE

> bool "Enable Linux to boot in VTL context"

drivers/hv/Kconfig config HYPERV · line 19

Quoted 2 lines
drivers/hv/Kconfig +1 -1
-       depends on X86_64 && HYPERV
+       depends on X86_64

> depends on SMP
> default n
> help

Quoted 7 lines

HYPERV_VTL_MODE can now be selected independently of HYPERV. The existing code should be such that even if someone is building a random config and gets HYPERV_VTL_MODE without HYPERV, the kernel will build and run in a non-Hyper-V environment and isn't broken somehow.

Quoted 5 lines

For HYPERV to be selected, either ACPI must already be selected, or HYPERV_VTL_MODE must already be selected. So "normal" kernels are still enforced to require ACPI. But if building with HYPERV_VTL_MODE, then ACPI is optional.

Thanks a ton! Let me try this out for the (arch; VTL) build matrix :)

Quoted 5 lines

Saurabh's idea of adding "depends on OF" to HYPERV_VTL_MODE should also work with these changes.

I haven't fully tested the above with all the relevant combinations on both x86 and arm64, but I think the logic makes sense.