Skip to content

arm64: hyperv: Support Virtual Trust Level Boot

Roman Kisel kvmarm linux-acpi linux-arch linux-arm-kernel linux-devicetree linux-hyperv linux-pci lkml 9 versions – In review

Get series with b4
b4 am [email protected]

Versions newest first · linked by author and title

  1. v9 Latest11 patches 2 repliesIn review
  2. v811 patches 7 repliesSuperseded
  3. v711 patches 2 repliesSuperseded
Earlier versions · 6
VersionPostedPatchesRepliesStatus
v611 patches17 repliesSuperseded
v511 patches41 repliesSuperseded
v46 patches22 repliesSuperseded
v37 patches45 repliesSuperseded
v26 patches29 repliesSuperseded
v16 patches8 repliesSuperseded

Patches in v9 No applies recorded yet

#Subject and filesSizeRepliesApplied
1/11arm64: kvm, smccc: Introduce and use API for getting hypervisor UUIDarch/arm64/kvm/hypercalls.c drivers/firmware/smccc/kvm_guest.c drivers/firmware/smccc/smccc.c +1 more+85 -160 replies
2/11arm64: hyperv: Use SMCCC to detect hypervisor presencearch/arm64/hyperv/mshyperv.c+45 -50 replies
3/11Drivers: hv: Enable VTL mode for arm64drivers/hv/Kconfig+3 -30 replies
4/11Drivers: hv: Provide arch-neutral implementation of get_vtl()arch/x86/hyperv/hv_init.c drivers/hv/hv_common.c include/asm-generic/mshyperv.h +1 more+38 -350 replies
5/11arm64: hyperv: Initialize the Virtual Trust Level fieldarch/arm64/hyperv/mshyperv.c+1 -00 replies
6/11arm64, x86: hyperv: Report the VTL the system boots inarch/arm64/hyperv/mshyperv.c arch/x86/hyperv/hv_vtl.c+8 -10 replies
7/11dt-bindings: microsoft,vmbus: Add interrupt and DMA coherence propertiesDocumentation/devicetree/bindings/bus/microsoft,vmbus.yaml+14 -20 replies
8/11Drivers: hv: vmbus: Get the IRQ number from DeviceTreedrivers/hv/vmbus_drv.c+30 -00 replies
9/11Drivers: hv: vmbus: Introduce hv_get_vmbus_root_device()drivers/hv/vmbus_drv.c include/linux/hyperv.h+17 -80 replies
10/11ACPI: irq: Introduce acpi_get_gsi_dispatcher()drivers/acpi/irq.c include/linux/acpi.h+18 -30 replies
11/11PCI: hv: Get vPCI MSI IRQ domain from DeviceTreedrivers/pci/controller/pci-hyperv.c+64 -61 reply

Reviews trailers per patch

All 11 patches have a Reviewed-by or Acked-by.

PatchArnd BergmannMichael KelleyTianyu LanKrzysztof KozlowskiRafael J. WysockiBjorn Helgaas
1/11A-b R-b
2/11R-b
3/11R-b
4/11R-b R-b
5/11R-b
6/11R-b
7/11R-b
8/11R-b
9/11R-b
10/11R-b A-b
11/11R-b A-b

Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes. Scroll the table sideways for all 6 reviewers.

Read the v9 thread

Cover letter [PATCH hyperv-next v9 00/11] arm64: hyperv: Support Virtual Trust Level Boot

This patch set allows the Hyper-V code to boot on ARM64 inside a Virtual Trust Level. These levels are a part of the Virtual Secure Mode documented in the Top-Level Functional Specification available at
https://learn.microsoft.com/en-us/virtualization/hyper-v-on-windows/tlfs/vsm.

The OpenHCL paravisor https://github.com/microsoft/openvmm/tree/main/openhcl
can serve as a practical application of these patches on ARM64.

For validation, I built kernels for the {x86_64, ARM64} x {VTL0, VTL2} set with a small initrd embedded into the kernel and booted VMs managed by Hyper-V and OpenVMM off of that.

Starting from V5, the patch series includes a non-functional change to KVM on arm64 which I tested as well.

I've kept the Acked-by tags given by Arnd and Bjorn. These patches (1 and 11) have changed very slightly since then (V5 and V6), no functional changes:
in patch 1, removed macro's in favour of functions as Marc suggested to get rid of "sparse" warnings, and in patch 11, fixed building as a module. Please let me know if I should have not kept the tags.

[V9]
- Fix building the pci-hyperv driver as a module. ** Thank you, Micheal! **

- Removed an overzealous check for the ACPI IRQ model on arm64 running with Hyper-V -- only GIC is supported. That check belongs to the lower layer code, and I took it from the ACPI driver while open-coding getting the parent IRQ domain.

[V8]
https://lore.kernel.org/linux-hyperv/[email protected]/
- Use the Linux derfined macro __ASSEMBLY__ instead of the gcc's pre-defined macro __ASSEMBLER__ to follow the kernel coding style.
** Thank you, Marc! **

[V7]
https://lore.kernel.org/linux-hyperv/[email protected]/
- Used another approach not to increase the number of warnings produced when building with CHECK_ENDIAN.
** Thank you, Arnd! **

- Adjusted the function parameter formatting to match the rest of the code. ** Thank you, Bjorn! **

- Removed the now unused local variable.
** Thank you, kernel robot! **

- Fixed the description in the VMBus DT binding patch. ** Thank you, Krzysztof! **

- Adjusted the function names and comments to better reflect what they do, used the suggested approach to handling UUIDs to make code more readable and maintainable on big-endian.
- Replaced ifdeffery with a stub function to make the code more readable. ** Thank you, Mark! **

- Fixed the Kconfig not to build the VTL mode code on 32-bit kernels. ** Thank you, Michael! **

- Fixed the indentation and the comment style. ** Thank you, Rafael! **

[V6]
https://lore.kernel.org/linux-hyperv/[email protected]/
- Use more intuitive Kconfig update.
- Remove ifdef for getting IRQ number
** Thank you, Arnd! **

- Simplify code for finding the parent IRQ domain. ** Thank you, Bjorn! **

- Remove a superfluous check.
** Thank you, Dan! **

- Make the commit title and descrtiption legible.
- Don't set additionalProperties to true. ** Thank you, Krzysztof! **

- Fix spelling in the commit title and description.
- Trade-offs for options in Kconfig.
- Export the new symbol as hyperv-pci can be built as a module. ** Thank you, Michael! **

- Simplify code for getting IRQ number.
** Thank you, Rob! **

- Add comment to clarify when running in VTL mode is reported. ** Thank you, Wei! **

[V5]
https://lore.kernel.org/linux-hyperv/[email protected]/
- Provide and use a common SMCCC-based infra for the arm64 hypervisor guests to detect hypervisor presence.
** Thank you, Arnd! **

- Fix line wraps to follow the rest of the code.
- Open-code getting IRQ domain parent in the ACPI case to make the code better.
** Thank you, Bjorn! **

- Test the binding with the latest dtschema.
- Clean up the commit title and description.
- Use proper defines for known constants.
** Thank you, Krzysztof! **

- Extend comment on why ACPI v6 is checked for.
- Reorder patches to make sure that even with partial series application the compilation succeeds.
- Report VTL the kernel runs in.
- Use "X86_64" in Kconfig rather than "X86".
- Extract a non-functional change for hv_get_vmbus_root_device() into a separate patch.
** Thank you, Michael! **

[V4]
https://lore.kernel.org/linux-hyperv/[email protected]/
- Fixed wording to match acronyms defined in the "Terms and Abbreviations" section of the SMCCC specification throughout the patch series. **Thank you, Michael!**

- Replaced the hypervisor ID containing ASCII with an UUID as required by the specification.
**Thank you, Michael!**

- Added an explicit check for SMCCC_RET_NOT_SUPPORTED when discovering the hypervisor presence to make the backward compatibility obvious. **Thank you, Saurabh!**

- Split the fix for get_vtl(void) out to make it easier to backport.
- Refactored the configuration options as requested to eliminate the risk of building non-functional kernels with randomly selected options. **Thank you, Michael!**

- Refactored the changes not to introduce an additional file with a one-line function.
**Thank you, Wei!**

- Fixed change description for the VMBus DeviceTree changes, used scripts/get_maintainers.pl on the latest kernel to get the up-to-date list of maintainers as requested.
**Thank you, Krzysztof!**

- Removed the added (paranoidal+superfluous) checks for DMA coherence in the VMBus driver and instead relied on the DMA and the OF subsystem code. **Thank you, Arnd, Krzysztof, Michael!**

- Used another set of APIs for discovering the hardware interrupt number in the VMBus driver to be able to build the driver as a module. **Thank you, Michael, Saurabh!**

- Renamed the newly introduced get_vmbus_root_device(void) function to hv_get_vmbus_root_device(void) as requested.
**Thank you, Wei!**

- Applied the suggested small-scale refactoring to simplify changes to the Hyper-V PCI driver. Taking the offered liberty of doing the large scale refactoring in another patch series.
**Thank you, Michael!**

- Added a fix for the issue discovered internally where the CPU would not get the interrupt from a PCI device attached to VTL2 as the shared peripheral interrupt number (SPI) was not offset by 32 (the first valid SPI number). **Thank you, Brian!**

[V3]
https://lore.kernel.org/lkml/[email protected]/
- Employed the SMCCC function recently implemented in the Microsoft Hyper-V hypervisor to detect running on Hyper-V/arm64. No dependence on ACPI/DT is needed anymore although the source code still falls back to ACPI as the new hypervisor might be available only in the Windows Insiders channel just
yet.
- As a part of the above, refactored detecting the hypervisor via ACPI FADT.
- There was a suggestion to explore whether it is feasible or not to express
that ACPI must be absent for the VTL mode and present for the regular guests
in the Hyper-V Kconfig file.
My current conclusion is that this will require refactoring in many places. That becomes especially convoluted on x86_64 due to the MSI and APIC dependencies. I'd ask to let us tackle that in another patch series (or chalk up to nice-have's rather than fires to put out) to separate concerns and decrease chances of breakage.
- While refactoring get_vtl(void) and the related code, fixed the hypercall output address not to overlap with the input as the Hyper-V TLFS mandates:
"The input and output parameter lists cannot overlap or cross page boundaries." See https://learn.microsoft.com/en-us/virtualization/hyper-v-on-windows/tlfs/hypercall-interface
for more.
Some might argue that should've been a topic for a separate patch series;
I'd counter that the change is well-contained (one line), has no dependencies, and makes the code legal.
- Made the VTL boot code (c)leaner as was suggested.
- Set DMA cache coherency for the VMBus.
- Updated DT bindings in the VMBus documentation (separated out into a new patch).
- Fixed vmbus_set_irq to use the API that works both for the ACPI and OF.
- Reworked setting up the vPCI MSI IRQ domain in the non-ACPI case. The logic looks a bit fiddly/ad-hoc as I couldn't find the API that would fit the bill. Added comments to explain myself.

[V2]
https://lore.kernel.org/all/[email protected]/
- Decreased number of #ifdef's
- Updated the wording in the commit messages to adhere to the guidlines
- Sending to the correct set of maintainers and mail lists

[V1]

    https://lore.kernel.org/all/[email protected]/

Roman Kisel (11):
arm64: kvm, smccc: Introduce and use API for getting hypervisor UUID
arm64: hyperv: Use SMCCC to detect hypervisor presence
Drivers: hv: Enable VTL mode for arm64
Drivers: hv: Provide arch-neutral implementation of get_vtl()
arm64: hyperv: Initialize the Virtual Trust Level field arm64, x86: hyperv: Report the VTL the system boots in
dt-bindings: microsoft,vmbus: Add interrupt and DMA coherence
properties
Drivers: hv: vmbus: Get the IRQ number from DeviceTree
Drivers: hv: vmbus: Introduce hv_get_vmbus_root_device()
ACPI: irq: Introduce acpi_get_gsi_dispatcher()
PCI: hv: Get vPCI MSI IRQ domain from DeviceTree

 .../bindings/bus/microsoft,vmbus.yaml         | 16 ++++-
 arch/arm64/hyperv/mshyperv.c                  | 53 ++++++++++++--
 arch/arm64/kvm/hypercalls.c                   | 10 +--
 arch/x86/hyperv/hv_init.c                     | 34 ---------
 arch/x86/hyperv/hv_vtl.c                      |  7 +-
 drivers/acpi/irq.c                            | 16 ++++-
 drivers/firmware/smccc/kvm_guest.c            | 10 +--
 drivers/firmware/smccc/smccc.c                | 17 +++++
 drivers/hv/Kconfig                            |  6 +-
 drivers/hv/hv_common.c                        | 31 ++++++++
 drivers/hv/vmbus_drv.c                        | 53 +++++++++++---
 drivers/pci/controller/pci-hyperv.c           | 70 +++++++++++++++++--
 include/asm-generic/mshyperv.h                |  6 ++
 include/hyperv/hvgdk_mini.h                   |  2 +-
 include/linux/acpi.h                          |  5 +-
 include/linux/arm-smccc.h                     | 64 +++++++++++++++--
 include/linux/hyperv.h                        |  2 +
 17 files changed, 323 insertions(+), 79 deletions(-)

base-commit: 628cc040b3a2980df6032766e8ef0688e981ab95

Signature · 1 line
2.43.0