x86/hyperv/hv_vtl: Use a wakeup mailbox to boot secondary CPUs
Subsystems (5)
v7 cover x86/hyperv/hv_vtl: Use a wakeup mailbox to boot secondary CPUs
Hi,
Many thanks to Boris, Rafael, Rob, and Dexuan for their valuable feedback! The main change in this version is the removal of the patch that moved the ACPI mailbox code from the x86 ACPI subsystem to a generic location. Users with DeviceTree-based firmware who wish use the ACPI wakeup mailbox need to select CONFIG_ACPI=y.
I dropped the Reviewed-by and Acked-by tags from patches 1, 7, and 8 as I made non-trivial changes to them. Once reviewed, maybe the patchset is ready to be merged?
I made minor changes to the cover letter to reflect the contents of this updated version.
Thanks in advance for your feedback!
...
This patchset adds functionality to use the ACPI wakeup mailbox to boot secondary CPUs in Hyper-V VTL level 2 TDX guests with DeviceTree-based virtual firmware. Although this is the target use case, the use of the mailbox depends solely on it being enumerated in the DeviceTree graph.
On x86 platforms, secondary CPUs are typically booted using INIT assert, de-assert followed by Start-Up IPI messages. Virtual machines can also use hypercalls to bring up secondary CPUs to a desired execution state. These two mechanisms require support from the hypervisor. Confidential computing VMs in a TDX environment cannot use this mechanism because the hypervisor is considered an untrusted entity.
Linux already supports the ACPI Multiprocessor Wakeup Structure in which the guest platform firmware boots the secondary CPUs and transfers control to the kernel using a mailbox. This mechanism does not need involvement of the VMM. It can be used in a Hyper-V VTL level 2 TDX guest.
Currently, this mechanism can only be used on x86 platforms with firmware that supports ACPI. There are platforms that use DeviceTree (e.g., OpenHCL [2]) instead of ACPI to describe the hardware.
Provided that the wakeup mailbox enumerated in a DeviceTree-based platform firmware is implemented as described in the ACPI specification, the kernel can use the existing ACPI code for both DeviceTree and ACPI systems. The DeviceTree firmware does not need to use any ACPI table to enumerate the mailbox.
This patchset is structured as follows:
* Expose functions to reuse the code handling the ACPI Multiprocessor Wakeup Structure outside of ACPI code. (patch 1)
* Define DeviceTree bindings to enumerate a mailbox as described in the ACPI specification. (patch 2)
* Find and set up the wakeup mailbox if enumerated in the DeviceTree graph. (patch 3)
* Prepare Hyper-V VTL2 TDX guests to use the Wakeup Mailbox to boot secondary CPUs when available. (patches 4-9)
I have tested this patchset on a Hyper-V host with VTL2 OpenHCL, QEMU, and physical hardware.
Changes in v7:
- Dropped the patch that relocated the ACPI wakeup mailbox to an generic
location. (Boris)
- Instead, added function declarations to use the wakeup mailbox from
outside ACPI code. Also added stubs for !CONFIG_ACPI.
- Link to v6: https://lore.kernel.org/r/[email protected]
Changes in v6:
- Fixed a build error with !CONFIG_X86_MAILBOX_WAKEUP and
CONFIG_HYPER_VTL_MODE.
- Added Acked-by tags from Rafael. Thanks!
- Added Reviewed-by tags from Dexuan and Rob. Thanks!
- Corrected typos and function names in the changelog.
- Link to v5: https://lore.kernel.org/r/[email protected]
Changes in v5:
- Referred in the DeviceTree binding documentation the section and
section of the ACPI specification that defines the wakeup mailbox.
- Moved the dependency on CONFIG_OF to patch 4, where the flattened
DeviceTree is parsed for the mailbox.
- Fixed a warning from yamllint regarding line lengths.
- Link to v4: https://lore.kernel.org/r/[email protected]
Changes in v4:
- Added Reviewed-by: tags from Michael Kelley. Thanks!
- Relocated the common wakeup code from acpi/madt_wakeup.c to a new
smpwakeup.c to be used in DeviceTree- and ACPI-based systems.
- Dropped the x86 CPU bindings as they are not a good fit to document
firmware features.
- Dropped the code that parsed and validated of the enable-method
property for cpu@N nodes in x86. Instead, unconditionally parse and use
the wakeup mailbox when found.
- Updated the wakeup mailbox schema to avoid redefing the structure and
operation of the mailbox. Instead, refer to the ACPI specification.
Also clarified that the enumeration of the mailbox is done separately.
- Prefixed helper functions of wakeup code with acpi_.
- Link to v3: https://lore.kernel.org/r/[email protected]
Changes in v3:
- Only move out of the acpi directory acpi_wakeup_cpu() and its
accessory variables. Use helper functions to access the mailbox as
needed. This also fixed the warnings about unused code with CONFIG_
ACPI=n that Michael reported.
- Major rework of the DeviceTree bindings and schema. Now there is a reserved-memory binding for the mailbox as well as a new x86 CPU
bindings. Both have compatible properties.
- Rework of the code parsing the DeviceTree bindings for the mailbox.
Now configuring the mailbox depends solely on its enumeration in the DeviceTree and not on Hyper-V VTL2 TDX guest.
- Do not make reserving the first 1MB of memory optional. It is not
needed and may introduce bugs.
- Prepare Hyper-V VTL2 guests to unconditionally use the mailbox in TDX environments. If the mailbox is not available, booting secondary CPUs
will fail gracefully.
- Link to v2: https://lore.kernel.org/r/[email protected]
Changes in v2:
- Fix the cover letter's summary phrase.
- Fix the DT binding document to pass validation.
- Change the DT binding document to be ACPI independent.
- Move ACPI-only functions into the #ifdef CONFIG_ACPI.
- Change dtb_parse_mp_wake() to return mailbox physical address.
- Rework the hv_is_private_mmio_tdx().
- Remove unrelated real mode change from the patch that marks mailbox
page private.
- Check hv_isolation_type_tdx() instead of wakeup_mailbox_addr in hv_vtl_init_platform() because wakeup_mailbox_addr is not parsed yet.
- Add memory range support to reserve_real_mode.
- Remove realmode_reserve callback and use the memory range.
- Move setting the real_mode_header to hv_vtl_init_platform.
- Update comments and commit messages.
- Minor style changes.
- Link to v1: https://lore.kernel.org/r/[email protected]
[1]. https://lore.kernel.org/all/aNxGzWMoM_oQ6n1N@liuwe-devbox-ubuntu-v2.lamzopl0uupeniq2etz1fddiyg.xx.internal.cloudapp.net/
[2]. https://openvmm.dev/guide/user_guide/openhcl.html
--
2.43.0
---
Ricardo Neri (5):
x86/acpi: Add functions to setup and access the wakeup mailbox
dt-bindings: reserved-memory: Wakeup Mailbox for Intel processors x86/dt: Parse the Wakeup Mailbox for Intel processors x86/acpi: Add a helper get the address of the wakeup mailbox x86/hyperv/vtl: Use the wakeup mailbox to boot secondary CPUs
Yunhong Jiang (4):
x86/hyperv/vtl: Set real_mode_header in hv_vtl_init_platform() x86/realmode: Make the location of the trampoline configurable x86/hyperv/vtl: Setup the 64-bit trampoline for TDX guests x86/hyperv/vtl: Mark the wakeup mailbox page as private
.../reserved-memory/intel,wakeup-mailbox.yaml | 50 ++++++++++++++++++++++ arch/x86/hyperv/hv_vtl.c | 36 ++++++++++++++-- arch/x86/include/asm/acpi.h | 16 +++++++ arch/x86/include/asm/x86_init.h | 3 ++ arch/x86/kernel/acpi/madt_wakeup.c | 16 +++++++ arch/x86/kernel/devicetree.c | 47 ++++++++++++++++++++ arch/x86/kernel/x86_init.c | 3 ++ arch/x86/realmode/init.c | 7 ++- 8 files changed, 170 insertions(+), 8 deletions(-)
---
base-commit: 6f85aad74a70d17919a64ecd93037aa51c08698d
change-id: 20250602-rneri-wakeup-mailbox-328efe72803f
Best regards,
Signature · 1 line
Ricardo Neri <[email protected]>
PATCH v7 1/9 x86/acpi: Add functions to setup and access the wakeup mailbox
Systems that describe hardware using DeviceTree graphs may enumerate and implement the wakeup mailbox as defined in the ACPI specification but do not otherwise depend on ACPI. Expose functions to setup and access the location of the wakeup mailbox from outside ACPI code.
The function acpi_setup_mp_wakeup_mailbox() stores the physical address of the mailbox and updates the wakeup_secondary_cpu_64() APIC callback.
The function acpi_madt_multiproc_wakeup_mailbox() returns a pointer to the mailbox.
---
Changes in v7:
- Moved function declarations to arch/x86/include/asm/acpi.h
- Added stubs for !CONFIG_ACPI.
- Do not use these new functions in madt_wakeup.c.
- Dropped Acked-by and Reviewed-by tags from Rafael and Dexuan as this patch changed.
Changes in v6:
- Fixed grammar error in the subject of the patch. (Rafael)
- Added Acked-by tag from Rafael. Thanks!
- Added Reviewed-by tag from Dexuan. Thanks!
Changes in v5:
- None
Changes in v4:
- Squashed the two first patches of the series into one, both introduce helper functions. (Rafael)
- Renamed setup_mp_wakeup_mailbox() as acpi_setup_mp_wakeup_mailbox(). (Rafael)
- Dropped the function prototype for !CONFIG_X86_64. (Rafael)
Changes in v3:
- Introduced this patch.
Changes in v2:
- N/A
--- arch/x86/include/asm/acpi.h | 10 ++++++++++ arch/x86/kernel/acpi/madt_wakeup.c | 11 +++++++++++ 2 files changed, 21 insertions(+)
arch/x86/include/asm/acpi.h +10 -0
@@ -182,6 +182,9 @@ void __iomem *x86_acpi_os_ioremap(acpi_physical_address phys, acpi_size size);#define acpi_os_ioremap acpi_os_ioremap#endif +void acpi_setup_mp_wakeup_mailbox (u64 addr); +struct acpi_madt_multiproc_wakeup_mailbox *acpi_get_mp_wakeup_mailbox (void ); +#else /* !CONFIG_ACPI */ #define acpi_lapic 0
@@ -200,6 +203,13 @@ static inline u64 x86_default_get_root_pointer(void) return 0; } +static inline void acpi_setup_mp_wakeup_mailbox (u64 addr) { } + +static inline struct acpi_madt_multiproc_wakeup_mailbox *acpi_get_mp_wakeup_mailbox (void ) +{ +return NULL ; +} +#endif /* !CONFIG_ACPI */ #define ARCH_HAS_POWER_INIT 1
arch/x86/kernel/acpi/madt_wakeup.c +11 -0
@@ -247,3 +247,14 @@ int __init acpi_parse_mp_wake(union acpi_subtable_headers *header,return 0 ; } + +void __init acpi_setup_mp_wakeup_mailbox (u64 mailbox_paddr) +{ + acpi_mp_wake_mailbox_paddr = mailbox_paddr; +apic_update_callback (wakeup_secondary_cpu_64, acpi_wakeup_cpu); +} + +struct acpi_madt_multiproc_wakeup_mailbox *acpi_get_mp_wakeup_mailbox (void ) +{ +return acpi_mp_wake_mailbox; +}
Signature · 1 line
2.43.0
Re: [PATCH v7 1/9] x86/acpi: Add functions to setup and access the wakeup mailbox
Quoted 6 lines · Ricardo Neri
Systems that describe hardware using DeviceTree graphs may enumerate and implement the wakeup mailbox as defined in the ACPI specification but do not otherwise depend on ACPI. Expose functions to setup and access the location of the wakeup mailbox from outside ACPI code.
Quoted 7 lines
The function
acpi_setup_mp_wakeup_mailbox()stores the physical address of the mailbox and updates thewakeup_secondary_cpu_64()APIC callback.The function
acpi_madt_multiproc_wakeup_mailbox()returns a pointer to the mailbox.Signed-off-by Ricardo Neri <[email protected]>
Quoted 84 lines
---
Changes in v7:
- Moved function declarations toarch/x86/include/asm/acpi.h
- Added stubs for !CONFIG_ACPI.
- Do not use these new functions inmadt_wakeup.c.
- Dropped Acked-by and Reviewed-by tags from Rafael and Dexuan as this patch changed.Changes in v6:
- Fixed grammar error in the subject of the patch. (Rafael)
- Added Acked-by tag from Rafael. Thanks!
- Added Reviewed-by tag from Dexuan. Thanks!Changes in v5:
- NoneChanges in v4:
- Squashed the two first patches of the series into one, both introduce helper functions. (Rafael)
- Renamedsetup_mp_wakeup_mailbox()asacpi_setup_mp_wakeup_mailbox(). (Rafael)
- Dropped the function prototype for !CONFIG_X86_64. (Rafael)Changes in v3:
- Introduced this patch.Changes in v2:
- N/A--- arch/x86/include/asm/acpi.h | 10 ++++++++++ arch/x86/kernel/acpi/madt_wakeup.c | 11 +++++++++++ 2 files changed, 21 insertions(+)arch/x86/include/asm/acpi.h +10
-0@@ -182,6 +182,9 @@ void __iomem *x86_acpi_os_ioremap(acpi_physical_address phys, acpi_size size);#define acpi_os_ioremap acpi_os_ioremap#endif +void acpi_setup_mp_wakeup_mailbox (u64 addr); +struct acpi_madt_multiproc_wakeup_mailbox *acpi_get_mp_wakeup_mailbox (void ); +#else /* !CONFIG_ACPI */ #define acpi_lapic 0@@ -200,6 +203,13 @@ static inline u64 x86_default_get_root_pointer(void) return 0; } +static inline void acpi_setup_mp_wakeup_mailbox (u64 addr) { } + +static inline struct acpi_madt_multiproc_wakeup_mailbox *acpi_get_mp_wakeup_mailbox (void ) +{ +return NULL ; +} +#endif /* !CONFIG_ACPI */ #define ARCH_HAS_POWER_INIT 1arch/x86/kernel/acpi/madt_wakeup.c +11
-1@@ -247,3 +247,14 @@ int __init acpi_parse_mp_wake(union acpi_subtable_headers *header,return 0 ; } + +void __init acpi_setup_mp_wakeup_mailbox (u64 mailbox_paddr) +{ + acpi_mp_wake_mailbox_paddr = mailbox_paddr; +apic_update_callback (wakeup_secondary_cpu_64, acpi_wakeup_cpu); +} + +struct acpi_madt_multiproc_wakeup_mailbox *acpi_get_mp_wakeup_mailbox (void ) +{ +return acpi_mp_wake_mailbox; +} --2.43.0
Re: [PATCH v7 1/9] x86/acpi: Add functions to setup and access the wakeup mailbox
Quoted 11 lines · Rafael J. Wysocki
Quoted 9 lines · Ricardo Neri
Systems that describe hardware using DeviceTree graphs may enumerate and implement the wakeup mailbox as defined in the ACPI specification but do not otherwise depend on ACPI. Expose functions to setup and access the location of the wakeup mailbox from outside ACPI code.
The function
acpi_setup_mp_wakeup_mailbox()stores the physical address of the mailbox and updates thewakeup_secondary_cpu_64()APIC callback.
Quoted 6 lines
Quoted 4 lines
The function
acpi_madt_multiproc_wakeup_mailbox()returns a pointer to the mailbox.Signed-off-by Ricardo Neri <[email protected]>
Thanks Rafael!
PATCH v7 2/9 dt-bindings: reserved-memory: Wakeup Mailbox for Intel processors
In reply to Ricardo NeriAdd DeviceTree bindings to enumerate the wakeup mailbox used in platform firmware for Intel processors.
x86 platforms commonly boot secondary CPUs using an INIT assert, de-assert followed by Start-Up IPI messages. The wakeup mailbox can be used when this mechanism is unavailable.
The wakeup mailbox offers more control to the operating system to boot secondary CPUs than a spin-table. It allows the reuse of the same wakeup vector for all CPUs while maintaining control over which CPUs to boot and when. While it is possible to achieve the same level of control using a spin-table, it would require specifying a separate cpu-release-addr for each secondary CPU.
The operation and structure of the mailbox are described in the Multiprocessor Wakeup Structure defined in the ACPI specification. Note that this structure does not specify how to publish the mailbox to the operating system (ACPI-based platform firmware uses a separate table). No ACPI table is needed in DeviceTree-based firmware to enumerate the mailbox.
Nodes that want to refer to the reserved memory usually define a memory-region property. /cpus/cpu* nodes would want to refer to the mailbox, but they do not have such property defined in the DeviceTree specification. Moreover, it would imply that there is a memory region per CPU. Instead, add a compatible property that the operating system can use to discover the mailbox.
---
Changes in v7:
- Fixed Acked-by tag from Rafael to include the "(Intel)" suffix.
Changes in v6:
- Reworded the changelog for clarity.
- Added Acked-by tag from Rafael. Thanks!
- Added Reviewed-by tag from Rob. Thanks!
- Added Reviewed-by tag from Dexuan. Thanks!
Changes in v5:
- Specified the version and section of the ACPI spec in which the wakeup mailbox is defined. (Rafael)
- Fixed a warning from yamllint about line lengths of URLs.
Changes in v4:
- Removed redefinitions of the mailbox and instead referred to ACPI specification as per discussion on LKML.
- Clarified that DeviceTree-based firmware do not require the use of ACPI tables to enumerate the mailbox. (Rob)
- Described the need of using a compatible property.
- Dropped the alignment property. (Krzysztof, Rafael)
- Used a real address for the mailbox node. (Krzysztof)
Changes in v3:
- Implemented the mailbox as a reserved-memory node. Add to it a compatible property. (Krzysztof)
- Explained the relationship between the mailbox and the enable-mehod property of the CPU nodes.
- Expanded the documentation of the binding.
Changes in v2:
- Added more details to the description of the binding.
- Added requirement a new requirement for cpu@N nodes to add an enable-method.
--- .../reserved-memory/intel,wakeup-mailbox.yaml | 50 ++++++++++++++++++++++ 1 file changed, 50 insertions(+)
Documentation/devicetree/bindings/reserved-memory/intel,wakeup-mailbox.yaml new file +50 -0
@@ -0,0 +1,50 @@ +# SPDX-License-Identifier: GPL-2.0-only OR BSD-2-Clause +%YAML 1.2 +--- +$id: http://devicetree.org/schemas/reserved-memory/intel,wakeup-mailbox.yaml# +$schema: http://devicetree.org/meta-schemas/core.yaml# + +title: Wakeup Mailbox for Intel processors + +description: | + The Wakeup Mailbox provides a mechanism for the operating system to wake up + secondary CPUs on Intel processors. It is an alternative to the INIT-!INIT- + SIPI sequence used on most x86 systems. + + The structure and operation of the mailbox is described in the Multiprocessor + Wakeup Structure of the ACPI specification version 6.6 section 5.2.12.19 [1]. + + The implementation of the mailbox in platform firmware is described in the + Intel TDX Virtual Firmware Design Guide section 4.3.5 [2]. + + 1: https://uefi.org/specs/ACPI/6.6/05_ACPI_Software_Programming_Model.html#multiprocessor-wakeup-structure + 2: https://www.intel.com/content/www/us/en/content-details/733585/intel-tdx-virtual-firmware-design-guide.html + + +maintainers: + - Ricardo Neri <[email protected]> + +allOf: + - $ref: reserved-memory.yaml + +properties: + compatible: + const: intel,wakeup-mailbox + +required: + - compatible + - reg + +unevaluatedProperties: false + +examples: + - | + reserved-memory { + #address-cells = <2>; + #size-cells = <1>; + + wakeup-mailbox@ffff0000 { + compatible = "intel,wakeup-mailbox"; + reg = <0x0 0xffff0000 0x1000>; + }; + };
Signature · 1 line
2.43.0
PATCH v7 3/9 x86/dt: Parse the Wakeup Mailbox for Intel processors
In reply to Ricardo NeriThe Wakeup Mailbox is a mechanism to boot secondary CPUs on systems that do not want or cannot use the INIT + StartUp IPI messages.
The platform firmware is expected to implement the mailbox as described in the Multiprocessor Wakeup Structure of the ACPI specification. It is also expected to publish the mailbox to the operating system as described in the corresponding DeviceTree schema that accompanies the documentation of the Linux kernel.
Reuse the existing functionality to set the memory location of the mailbox and update the wakeup_secondary_cpu_64() APIC callback. Make this functionality available to DeviceTree-based systems by making CONFIG_X86_ MAILBOX_WAKEUP depend on either CONFIG_OF or CONFIG_ACPI_MADT_WAKEUP.
do_boot_cpu() uses wakeup_secondary_cpu_64() when set. It will be set if a wakeup mailbox is enumerated via an ACPI table or a DeviceTree node. For cases in which this behavior is not desired, this APIC callback can be updated later during boot using platform-specific hooks.
---
Changes in v7:
- #included asm/acpi.h to reflect the updated declaration of the needed functions.
- (Kept Reviewed-by tag from Dexuan, as this single change is trivial.)
Changes in v6:
- Added Reviewed-by tag from Dexuan. Thanks!
Changes in v5:
- Made CONFIG_X86_MAILBOX_WAKEUP depend on CONFIG_OF or CONFIG_ACPI_ MADT_WAKEUP.
Changes in v4:
- Look for the wakeup mailbox unconditionally, regardless of whether cpu@N nodes have an enable-method property.
- Add a reference to the ACPI specification. (Rafael)
Changes in v3:
- Added extra sanity checks when parsing the mailbox node.
- Probe the mailbox using its compatible property
- Setup the Wakeup Mailbox if the enable-method is found in the CPU nodes.
- Cleaned up unneeded ifdeffery.
- Clarified the mechanisms used to override the wakeup_secondary_64() callback to not use the mailbox when not desired. (Michael)
- Edited the commit message for clarity.
Changes in v2:
- Disabled CPU offlining.
- Modified dtb_parse_mp_wake() to return the address of the mailbox.
--- arch/x86/kernel/devicetree.c | 47 ++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 47 insertions(+)
arch/x86/kernel/devicetree.c +47 -0
@@ -18,6 +18,7 @@#include <linux/of_pci.h> #include <linux/initrd.h> +#include <asm/acpi.h> #include <asm/irqdomain.h> #include <asm/hpet.h> #include <asm/apic.h>
@@ -125,6 +126,51 @@ static void __init dtb_setup_hpet(void)#endif } +#if defined(CONFIG_X86_64 ) && defined(CONFIG_SMP ) + +#define WAKEUP_MAILBOX_SIZE 0x1000 +#define WAKEUP_MAILBOX_ALIGN 0x1000 + +/** dtb_wakeup_mailbox_setup() - Parse the wakeup mailbox from the device tree +* +* Look for the presence of a wakeup mailbox in the DeviceTree. The mailbox is +* expected to follow the structure and operation described in the Multiprocessor +* Wakeup Structure of the ACPI specification. +*/ +static void __init dtb_wakeup_mailbox_setup (void ) +{ +struct device_node *node; +struct resource res; + + node =of_find_compatible_node (NULL ,NULL ,"intel,wakeup-mailbox" ); +if (!node) +return ; + +if (of_address_to_resource (node,0 , &res)) + goto done; + +/* The mailbox is a 4KB-aligned region.*/ +if (res.start & (WAKEUP_MAILBOX_ALIGN -1 )) + goto done; + +/* The mailbox has a size of 4KB. */ +if (res.end - res.start +1 !=WAKEUP_MAILBOX_SIZE ) + goto done; + +/* Not supported when the mailbox is used. */ +cpu_hotplug_disable_offlining (); + +acpi_setup_mp_wakeup_mailbox (res.start); +done: +of_node_put (node); +} +#else /* !CONFIG_X86_64 || !CONFIG_SMP */ +static inline int dtb_wakeup_mailbox_setup (void ) +{ +return -EOPNOTSUPP ; +} +#endif /* CONFIG_X86_64 && CONFIG_SMP */ +#ifdef CONFIG_X86_LOCAL_APIC static void __init dtb_cpu_setup (void )
@@ -287,6 +333,7 @@ static void __init x86_dtb_parse_smp_config(void)dtb_setup_hpet ();dtb_apic_setup (); +dtb_wakeup_mailbox_setup (); }void __init x86_flattree_get_config (void )
Signature · 1 line
2.43.0
PATCH v7 4/9 x86/hyperv/vtl: Set real_mode_header in hv_vtl_init_platform()
In reply to Ricardo NeriFrom: Yunhong Jiang <[email protected]>
Hyper-V VTL clears x86_platform.realmode_{init(), reserve()} in hv_vtl_init_platform() whereas it sets real_mode_header later in hv_vtl_early_init(). There is no need to deal with the settings of real mode memory in two places. Also, both functions are called much earlier than x86_platform.realmode_init() (via an early_initcall), where the real_mode_header is needed.
Set real_mode_header in hv_vtl_init_platform() to keep all code dealing with memory for the real mode trampoline in one place. Besides making the code more readable, it prepares it for a subsequent changeset in which the behavior needs to change to support Hyper-V VTL guests in TDX a environment.
---
Changes in v7:
- None
Changes in v6:
- Corrected reference to hv_vtl_init_platform() in the changelog. (Dexuan)
- Added Reviewed-by tag from Dexuan. Thanks!
Changes in v5:
- None
Changes in v4:
- Added Reviewed-by tag from Michael. Thanks!
Changes in v3:
- Edited the commit message for clarity.
Changes in v2:
- Introduced this patch.
--- arch/x86/hyperv/hv_vtl.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
arch/x86/hyperv/hv_vtl.c +1 -1
@@ -65,6 +65,7 @@ void __init hv_vtl_init_platform(void) x86_platform.realmode_reserve = x86_init_noop; x86_platform.realmode_init = x86_init_noop; + real_mode_header = &hv_vtl_real_mode_header; x86_init.irqs.pre_vector_init = x86_init_noop; x86_init.timers.timer_init = x86_init_noop; x86_init.resources.probe_roms = x86_init_noop;
@@ -244,7 +245,6 @@ int __init hv_vtl_early_init(void)panic ("XSAVE has to be disabled as it is not supported by this module.\n" "Please add 'noxsave' to the kernel command line.\n" ); - real_mode_header = &hv_vtl_real_mode_header;apic_update_callback (wakeup_secondary_cpu_64, hv_vtl_wakeup_secondary_cpu);return 0 ;
Signature · 1 line
2.43.0
PATCH v7 5/9 x86/realmode: Make the location of the trampoline configurable
In reply to Ricardo NeriFrom: Yunhong Jiang <[email protected]>
x86 CPUs boot in real mode. This mode uses a 1MB address space. The trampoline must reside below this 1MB memory boundary.
There are platforms in which the firmware boots the secondary CPUs, switches them to long mode and transfers control to the kernel. An example of such a mechanism is the ACPI Multiprocessor Wakeup Structure.
In this scenario there is no restriction on locating the trampoline under 1MB memory. Moreover, certain platforms (for example, Hyper-V VTL guests) may not have memory available for allocation below 1MB.
Add a new member to struct x86_init_resources to specify the upper bound for the location of the trampoline memory. Preserve the default upper bound of 1MB to conserve the current behavior.
---
Changes in v7:
- None
Changes in v6:
- Added Reviewed-by tag from Dexuan. Thanks!
Changes in v5:
- None
Changes in v4:
- Added Reviewed-by tag from Michael. Thanks!
Changes in v3:
- Edited the commit message for clarity.
- Minor tweaks to comments.
- Removed the option to not reserve the first 1MB of memory as it is not needed.
Changes in v2:
- Added this patch using code that Thomas suggested:
https://lore.kernel.org/lkml/87a5ho2q6x.ffs@tglx/
--- arch/x86/include/asm/x86_init.h | 3 +++ arch/x86/kernel/x86_init.c | 3 +++ arch/x86/realmode/init.c | 7 +++---- 3 files changed, 9 insertions(+), 4 deletions(-)
arch/x86/include/asm/x86_init.h +3 -0
@@ -31,12 +31,15 @@ struct x86_init_mpparse {* platform * @memory_setup: platform specific memory setup * @dmi_setup: platform specific DMI setup +* @realmode_limit: platform specific address limit for the real mode trampoline +* (default 1M) */ struct x86_init_resources {void (*probe_roms)(void );void (*reserve_resources)(void );char *(*memory_setup)(void );void (*dmi_setup)(void ); +unsigned long realmode_limit; };/**
arch/x86/kernel/x86_init.c +3 -0
@@ -9,6 +9,7 @@#include <linux/export.h> #include <linux/pci.h> #include <linux/acpi.h> +#include <linux/sizes.h> #include <asm/acpi.h> #include <asm/bios_ebda.h>
@@ -69,6 +70,8 @@ struct x86_init_ops x86_init __initdata = { .reserve_resources = reserve_standard_io_resources, .memory_setup = e820__memory_setup_default, .dmi_setup = dmi_setup, +/* Has to be under 1M so we can execute real-mode AP code. */ + .realmode_limit = SZ_1M, }, .mpparse = {
arch/x86/realmode/init.c +3 -4
@@ -46,7 +46,7 @@ void load_trampoline_pgtable(void)void __init reserve_real_mode (void ) { -phys_addr_t mem; +phys_addr_t mem, limit = x86_init.resources.realmode_limit;size_t size =real_mode_size_needed ();if (!size)
@@ -54,10 +54,9 @@ void __init reserve_real_mode(void)WARN_ON (slab_is_available ()); -/* Has to be under 1M so we can execute real-mode AP code. */ - mem =memblock_phys_alloc_range (size,PAGE_SIZE ,0 ,1 <<20 ); + mem =memblock_phys_alloc_range (size,PAGE_SIZE ,0 , limit);if (!mem) -pr_info ("No sub-1M memory is available for the trampoline\n" ); +pr_info ("No memory below %pa for the real-mode trampoline\n" , &limit);else set_real_mode_mem (mem);
Signature · 1 line
2.43.0
PATCH v7 6/9 x86/hyperv/vtl: Setup the 64-bit trampoline for TDX guests
In reply to Ricardo NeriFrom: Yunhong Jiang <[email protected]>
The hypervisor is an untrusted entity for TDX guests. It cannot be used to boot secondary CPUs - neither via hypercalls nor the INIT assert, de-assert, plus Start-Up IPI messages.
Instead, the platform virtual firmware boots the secondary CPUs and puts them in a state to transfer control to the kernel. This mechanism uses the wakeup mailbox described in the Multiprocessor Wakeup Structure of the ACPI specification. The entry point to the kernel is trampoline_start64.
Allocate and setup the trampoline using the default x86_platform callbacks.
The platform firmware configures the secondary CPUs in long mode. It is no longer necessary to locate the trampoline under 1MB memory. After handoff from firmware, the trampoline code switches briefly to 32-bit addressing mode, which has an addressing limit of 4GB. Set the upper bound of the trampoline memory accordingly.
---
Changes in v7:
- None
Changes in v6:
- Added Reviewed-by tag from Dexuan. Thanks!
Changes in v5:
- None
Changes in v4:
- Added Reviewed-by tag from Michael. Thanks!
Changes in v3:
- Added a note regarding there is no need to check for a present paravisor.
- Edited commit message for clarity.
Changes in v2:
- Dropped the function hv_reserve_real_mode(). Instead, used the new members realmode_limit and reserve_bios members of x86_init to set the upper bound of the trampoline memory. (Thomas)
--- arch/x86/hyperv/hv_vtl.c | 11 ++++++++--- 1 file changed, 8 insertions(+), 3 deletions(-)
arch/x86/hyperv/hv_vtl.c +8 -3
@@ -63,9 +63,14 @@ void __init hv_vtl_init_platform(void)*/ pr_info ("Linux runs in Hyper-V Virtual Trust Level %d\n" , ms_hyperv.vtl); - x86_platform.realmode_reserve = x86_init_noop; - x86_platform.realmode_init = x86_init_noop; - real_mode_header = &hv_vtl_real_mode_header; +/* There is no paravisor present if we are here. */ +if (hv_isolation_type_tdx ()) { + x86_init.resources.realmode_limit =SZ_4G ; + }else { + x86_platform.realmode_reserve = x86_init_noop; + x86_platform.realmode_init = x86_init_noop; + real_mode_header = &hv_vtl_real_mode_header; + } x86_init.irqs.pre_vector_init = x86_init_noop; x86_init.timers.timer_init = x86_init_noop; x86_init.resources.probe_roms = x86_init_noop;
Signature · 1 line
2.43.0
PATCH v7 7/9 x86/acpi: Add a helper get the address of the wakeup mailbox
In reply to Ricardo NeriA Hyper-V VTL level 2 guest in a TDX environment needs to map the physical page of the ACPI Multiprocessor Wakeup Structure as private (encrypted). It needs to know the physical address of this structure. Add a helper function to retrieve the address.
---
Changes in v7:
- Moved the added function to arch/x86/kernel/acpi/madt_wakeup.c
- Dropped Reviewed-by tags from Dexuan and Michael as this patch changed.
Changes in v6:
- Added Reviewed-by tag from Dexuan. Thanks!
Changes in v5:
- None
Changes in v4:
- Renamed function to acpi_get_mp_wakeup_mailbox_paddr().
- Added Reviewed-by tag from Michael. Thanks!
Changes in v3:
- Introduced this patch
Changes in v2:
- N/A
--- arch/x86/include/asm/acpi.h | 6 ++++++ arch/x86/kernel/acpi/madt_wakeup.c | 5 +++++ 2 files changed, 11 insertions(+)
arch/x86/include/asm/acpi.h +6 -0
@@ -184,6 +184,7 @@ void __iomem *x86_acpi_os_ioremap(acpi_physical_address phys, acpi_size size);void acpi_setup_mp_wakeup_mailbox (u64 addr);struct acpi_madt_multiproc_wakeup_mailbox *acpi_get_mp_wakeup_mailbox (void ); +u64 acpi_get_mp_wakeup_mailbox_paddr (void );#else /* !CONFIG_ACPI */
@@ -210,6 +211,11 @@ static inline struct acpi_madt_multiproc_wakeup_mailbox *acpi_get_mp_wakeup_mail return NULL; } +static inline u64 acpi_get_mp_wakeup_mailbox_paddr (void ) +{ +return 0 ; +} +#endif /* !CONFIG_ACPI */ #define ARCH_HAS_POWER_INIT 1
arch/x86/kernel/acpi/madt_wakeup.c +5 -0
@@ -258,3 +258,8 @@ struct acpi_madt_multiproc_wakeup_mailbox *acpi_get_mp_wakeup_mailbox(void) {return acpi_mp_wake_mailbox; } + +u64 acpi_get_mp_wakeup_mailbox_paddr (void ) +{ +return acpi_mp_wake_mailbox_paddr; +}
Signature · 1 line
2.43.0
Re: [PATCH v7 7/9] x86/acpi: Add a helper get the address of the wakeup mailbox
Quoted 8 lines · Ricardo Neri
A Hyper-V VTL level 2 guest in a TDX environment needs to map the physical page of the ACPI Multiprocessor Wakeup Structure as private (encrypted). It needs to know the physical address of this structure. Add a helper function to retrieve the address.
Suggested-by Michael Kelley <[email protected]>Signed-off-by Ricardo Neri <[email protected]>
Quoted 67 lines
---
Changes in v7:
- Moved the added function toarch/x86/kernel/acpi/madt_wakeup.c
- Dropped Reviewed-by tags from Dexuan and Michael as this patch changed.Changes in v6:
- Added Reviewed-by tag from Dexuan. Thanks!Changes in v5:
- NoneChanges in v4:
- Renamed function toacpi_get_mp_wakeup_mailbox_paddr().
- Added Reviewed-by tag from Michael. Thanks!Changes in v3:
- Introduced this patchChanges in v2:
- N/A--- arch/x86/include/asm/acpi.h | 6 ++++++ arch/x86/kernel/acpi/madt_wakeup.c | 5 +++++ 2 files changed, 11 insertions(+)arch/x86/include/asm/acpi.h +6
-0@@ -184,6 +184,7 @@ void __iomem *x86_acpi_os_ioremap(acpi_physical_address phys, acpi_size size);void acpi_setup_mp_wakeup_mailbox (u64 addr);struct acpi_madt_multiproc_wakeup_mailbox *acpi_get_mp_wakeup_mailbox (void ); +u64 acpi_get_mp_wakeup_mailbox_paddr (void );#else /* !CONFIG_ACPI */ @@ -210,6 +211,11 @@ static inline struct acpi_madt_multiproc_wakeup_mailbox *acpi_get_mp_wakeup_mail return NULL; } +static inline u64 acpi_get_mp_wakeup_mailbox_paddr (void ) +{ +return 0 ; +} +#endif /* !CONFIG_ACPI */ #define ARCH_HAS_POWER_INIT 1arch/x86/kernel/acpi/madt_wakeup.c +5
-1@@ -258,3 +258,8 @@ struct acpi_madt_multiproc_wakeup_mailbox *acpi_get_mp_wakeup_mailbox(void) {return acpi_mp_wake_mailbox; } + +u64 acpi_get_mp_wakeup_mailbox_paddr (void ) +{ +return acpi_mp_wake_mailbox_paddr; +} --2.43.0
PATCH v7 8/9 x86/hyperv/vtl: Mark the wakeup mailbox page as private
In reply to Ricardo NeriFrom: Yunhong Jiang <[email protected]>
The current code maps MMIO devices as shared (decrypted) by default in a confidential computing VM.
In a TDX environment, secondary CPUs are booted using the Multiprocessor Wakeup Structure defined in the ACPI specification. The virtual firmware and the operating system function in the guest context, without intervention from the VMM. Map the physical memory of the mailbox as private. Use the is_private_mmio() callback.
---
Changed since v7:
- Dropped check for !CONFIG_X86_MAILBOX_WAKEUP. The symbol is no longer valid and now we have a stub for !CONFIG_ACPI.
- Dropped Reviewed-by tags from Dexuan and Michael as this patch changed.
Changes in v6:
- Fixed a compile error with !CONFIG_X86_MAILBOX_WAKEUP.
- Added Reviewed-by tag from Dexuan. Thanks!
Changes in v5:
- None
Changes in v4:
- Updated to use the renamed function acpi_get_mp_wakeup_mailbox_paddr().
- Added Reviewed-by tag from Michael. Thanks!
Changes in v3:
- Use the new helper function get_mp_wakeup_mailbox_paddr().
- Edited the commit message for clarity.
Changes in v2:
- Added the helper function within_page() to improve readability
- Override the is_private_mmio() callback when detecting a TDX environment. The address of the mailbox is checked in hv_is_private_mmio_tdx().
--- arch/x86/hyperv/hv_vtl.c | 15 +++++++++++++++ 1 file changed, 15 insertions(+)
arch/x86/hyperv/hv_vtl.c +15 -0
@@ -6,6 +6,7 @@* Saurabh Sengar <[email protected]> */ +#include <asm/acpi.h> #include <asm/apic.h> #include <asm/boot.h> #include <asm/desc.h>
@@ -54,6 +55,18 @@ static void __noreturn hv_vtl_restart(char __maybe_unused *cmd)hv_vtl_emergency_restart (); } +static inline bool within_page (u64 addr,u64 start) +{ +return addr >= start && addr < (start +PAGE_SIZE ); +} + +static bool hv_vtl_is_private_mmio_tdx (u64 addr) +{ +u64 mb_addr =acpi_get_mp_wakeup_mailbox_paddr (); + +return mb_addr &&within_page (addr, mb_addr); +} +void __init hv_vtl_init_platform (void ) {/*
@@ -66,6 +79,8 @@ void __init hv_vtl_init_platform(void)/* There is no paravisor present if we are here. */ if (hv_isolation_type_tdx ()) { x86_init.resources.realmode_limit =SZ_4G ; + x86_platform.hyper.is_private_mmio = hv_vtl_is_private_mmio_tdx; + }else { x86_platform.realmode_reserve = x86_init_noop; x86_platform.realmode_init = x86_init_noop;
Signature · 1 line
2.43.0
kernel test robot
build botRe: [PATCH v7 8/9] x86/hyperv/vtl: Mark the wakeup mailbox page as private
kernel test robot noticed the following build warnings:
Hi Ricardo,
kernel test robot noticed the following build warnings:
[auto build test WARNING on 6f85aad74a70d17919a64ecd93037aa51c08698d]
url: https://github.com/intel-lab-lkp/linux/commits/Ricardo-Neri/x86-acpi-Add-functions-to-setup-and-access-the-wakeup-mailbox/20251118-011935 base: 6f85aad74a70d17919a64ecd93037aa51c08698d patch link: https://lore.kernel.org/r/20251117-rneri-wakeup-mailbox-v7-8-4a8b82ab7c2c%40linux.intel.com patch subject: [PATCH v7 8/9] x86/hyperv/vtl: Mark the wakeup mailbox page as private config: x86_64-buildonly-randconfig-001-20251118 (https://download.01.org/0day-ci/archive/20251118/[email protected]/config) compiler: clang version 20.1.8 (https://github.com/llvm/llvm-project 87f0227cb60147a26a1eeb4fb06e3b505e9c7261) reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20251118/[email protected]/reproduce)
If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <[email protected]>
| Closes: https://lore.kernel.org/oe-kbuild-all/[email protected]/
All warnings (new ones prefixed by >>):
In file included from arch/x86/hyperv/hv_vtl.c:9:
In file included from arch/x86/include/asm/acpi.h:11:
In file included from arch/x86/include/asm/numa.h:8:
>> arch/x86/include/asm/topology.h:159:47: warning: declaration of 'struct cpuinfo_x86' will not be visible outside of this function [-Wvisibility]
159 | const char *get_topology_cpu_type_name(struct cpuinfo_x86 *c);
| ^
arch/x86/include/asm/topology.h:160:57: warning: declaration of 'struct cpuinfo_x86' will not be visible outside of this function [-Wvisibility]
160 | enum x86_topology_cpu_type get_topology_cpu_type(struct cpuinfo_x86 *c);
| ^2 warnings generated.
vim +159 arch/x86/include/asm/topology.h
090610ba704a66 Thomas Gleixner 2024-02-13 158
45239ba39a5279 Pawan Gupta 2024-10-25 @159 const char *get_topology_cpu_type_name(struct cpuinfo_x86 *c); 45239ba39a5279 Pawan Gupta 2024-10-25 160 enum x86_topology_cpu_type get_topology_cpu_type(struct cpuinfo_x86 *c); 45239ba39a5279 Pawan Gupta 2024-10-25 161
Signature · 2 lines
0-DAY CI Kernel Test Service https://github.com/intel/lkp-tests/wiki
Re: [PATCH v7 8/9] x86/hyperv/vtl: Mark the wakeup mailbox page as private
Quoted 24 lines · kernel test robot
Hi Ricardo,
kernel test robot noticed the following build warnings:
[auto build test WARNING on
6f85aad74a70d17919a64ecd93037aa51c08698d]url: https://github.com/intel-lab-lkp/linux/commits/Ricardo-Neri/x86-acpi-Add-functions-to-setup-and-access-the-wakeup-mailbox/20251118-011935 base: 6f85aad74a70d17919a64ecd93037aa51c08698d patch link: https://lore.kernel.org/r/20251117-rneri-wakeup-mailbox-v7-8-4a8b82ab7c2c%40linux.intel.com patch subject: [PATCH v7 8/9] x86/hyperv/vtl: Mark the wakeup mailbox page as private config: x86_64-buildonly-randconfig-001-20251118 (https://download.01.org/0day-ci/archive/20251118/[email protected]/config) compiler: clang version 20.1.8 (https://github.com/llvm/llvm-project 87f0227cb60147a26a1eeb4fb06e3b505e9c7261) reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20251118/[email protected]/reproduce)If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <[email protected]>
| Closes: https://lore.kernel.org/oe-kbuild-all/[email protected]/All warnings (new ones prefixed by >>):
In file included from
arch/x86/hyperv/hv_vtl.c:9:
In file included fromarch/x86/include/asm/acpi.h:11:
In file included fromarch/x86/include/asm/numa.h:8:
Quoted 7 lines
Quoted 1 line
Quoted 1 line
arch/x86/include/asm/topology.h:159:47: warning: declaration of 'structcpuinfo_x86' will not be visible outside of this function [-Wvisibility]159 | const char *get_topology_cpu_type_name(struct cpuinfo_x86 *c); | ^ arch/x86/include/asm/topology.h:160:57: warning: declaration of 'struct cpuinfo_x86' will not be visible outside of this function [-Wvisibility] 160 | enum x86_topology_cpu_type get_topology_cpu_type(struct cpuinfo_x86 *c); | ^2 warnings generated.
This is fixed with: https://lore.kernel.org/r/20251117-rneri-topology-cpuinfo-bug-v1-1-a905bb5f91e2@linux.intel.com
kernel test robot
build botRe: [PATCH v7 8/9] x86/hyperv/vtl: Mark the wakeup mailbox page as private
In reply to Ricardo Nerikernel test robot noticed the following build errors:
Hi Ricardo,
kernel test robot noticed the following build errors:
[auto build test ERROR on 6f85aad74a70d17919a64ecd93037aa51c08698d]
url: https://github.com/intel-lab-lkp/linux/commits/Ricardo-Neri/x86-acpi-Add-functions-to-setup-and-access-the-wakeup-mailbox/20251118-011935 base: 6f85aad74a70d17919a64ecd93037aa51c08698d patch link: https://lore.kernel.org/r/20251117-rneri-wakeup-mailbox-v7-8-4a8b82ab7c2c%40linux.intel.com patch subject: [PATCH v7 8/9] x86/hyperv/vtl: Mark the wakeup mailbox page as private config: x86_64-allyesconfig (https://download.01.org/0day-ci/archive/20251119/[email protected]/config) compiler: clang version 20.1.8 (https://github.com/llvm/llvm-project 87f0227cb60147a26a1eeb4fb06e3b505e9c7261) reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20251119/[email protected]/reproduce)
If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <[email protected]>
| Closes: https://lore.kernel.org/oe-kbuild-all/[email protected]/
All errors (new ones prefixed by >>):
In file included from arch/x86/hyperv/hv_vtl.c:9:
In file included from arch/x86/include/asm/acpi.h:11:
In file included from arch/x86/include/asm/numa.h:8:
>> arch/x86/include/asm/topology.h:264:26: error: expected ';' after top level declarator
264 | extern bool __read_mostly sysctl_sched_itmt_enabled;
| ^
| ;
In file included from arch/x86/hyperv/hv_vtl.c:9:
>> arch/x86/include/asm/acpi.h:179:42: error: unknown type name 'acpi_physical_address'
179 | extern void __iomem * (*acpi_os_ioremap)(acpi_physical_address phys,
| ^
>> arch/x86/include/asm/acpi.h:180:7: error: unknown type name 'acpi_size'
180 | acpi_size size);
| ^
arch/x86/include/asm/acpi.h:181:35: error: unknown type name 'acpi_physical_address'
181 | void __iomem *x86_acpi_os_ioremap(acpi_physical_address phys, acpi_size size);
| ^
arch/x86/include/asm/acpi.h:181:63: error: unknown type name 'acpi_size'
181 | void __iomem *x86_acpi_os_ioremap(acpi_physical_address phys, acpi_size size);
| ^
In file included from arch/x86/hyperv/hv_vtl.c:10:
In file included from arch/x86/include/asm/apic.h:6:
In file included from include/linux/static_call.h:135:
In file included from include/linux/cpu.h:17:
In file included from include/linux/node.h:18:
In file included from include/linux/device.h:32:
In file included from include/linux/device/driver.h:21:
In file included from include/linux/module.h:20:
In file included from include/linux/elf.h:6:
In file included from arch/x86/include/asm/elf.h:10:
In file included from arch/x86/include/asm/ia32.h:7:
In file included from include/linux/compat.h:17:
In file included from include/linux/fs.h:34:
In file included from include/linux/percpu-rwsem.h:7:
In file included from include/linux/rcuwait.h:6:
In file included from include/linux/sched/signal.h:6:
include/linux/signal.h:98:11: warning: array index 3 is past the end of the array (that has type 'unsigned long[1]') [-Warray-bounds]
98 | return (set->sig[3] | set->sig[2] |
| ^ ~
arch/x86/include/asm/signal.h:24:2: note: array 'sig' declared here
24 | unsigned long sig[_NSIG_WORDS];
| ^
In file included from arch/x86/hyperv/hv_vtl.c:10:
In file included from arch/x86/include/asm/apic.h:6:
In file included from include/linux/static_call.h:135:
In file included from include/linux/cpu.h:17:
In file included from include/linux/node.h:18:
In file included from include/linux/device.h:32:
In file included from include/linux/device/driver.h:21:
In file included from include/linux/module.h:20:
In file included from include/linux/elf.h:6:
In file included from arch/x86/include/asm/elf.h:10:
In file included from arch/x86/include/asm/ia32.h:7:
In file included from include/linux/compat.h:17:
In file included from include/linux/fs.h:34:
In file included from include/linux/percpu-rwsem.h:7:
In file included from include/linux/rcuwait.h:6:
In file included from include/linux/sched/signal.h:6:
include/linux/signal.h:98:25: warning: array index 2 is past the end of the array (that has type 'unsigned long[1]') [-Warray-bounds]
98 | return (set->sig[3] | set->sig[2] |
| ^ ~
arch/x86/include/asm/signal.h:24:2: note: array 'sig' declared here
24 | unsigned long sig[_NSIG_WORDS];
| ^
In file included from arch/x86/hyperv/hv_vtl.c:10:
In file included from arch/x86/include/asm/apic.h:6:
In file included from include/linux/static_call.h:135:
In file included from include/linux/cpu.h:17:
In file included from include/linux/node.h:18:
In file included from include/linux/device.h:32:
In file included from include/linux/device/driver.h:21:
In file included from include/linux/module.h:20:
In file included from include/linux/elf.h:6:
In file included from arch/x86/include/asm/elf.h:10:
In file included from arch/x86/include/asm/ia32.h:7:
In file included from include/linux/compat.h:17:
In file included from include/linux/fs.h:34:
In file included from include/linux/percpu-rwsem.h:7:
In file included from include/linux/rcuwait.h:6:
In file included from include/linux/sched/signal.h:6:
include/linux/signal.h:99:4: warning: array index 1 is past the end of the array (that has type 'unsigned long[1]') [-Warray-bounds]
99 | set->sig[1] | set->sig[0]) == 0;
| ^ ~
arch/x86/include/asm/signal.h:24:2: note: array 'sig' declared here
24 | unsigned long sig[_NSIG_WORDS];
| ^
In file included from arch/x86/hyperv/hv_vtl.c:10:
In file included from arch/x86/include/asm/apic.h:6:
In file included from include/linux/static_call.h:135:
In file included from include/linux/cpu.h:17:
In file included from include/linux/node.h:18:
In file included from include/linux/device.h:32:
In file included from include/linux/device/driver.h:21:
In file included from include/linux/module.h:20:
In file included from include/linux/elf.h:6:
In file included from arch/x86/include/asm/elf.h:10:
In file included from arch/x86/include/asm/ia32.h:7:
In file included from include/linux/compat.h:17:
In file included from include/linux/fs.h:34:
In file included from include/linux/percpu-rwsem.h:7:
In file included from include/linux/rcuwait.h:6:
In file included from include/linux/sched/signal.h:6:
include/linux/signal.h:101:11: warning: array index 1 is past the end of the array (that has type 'unsigned long[1]') [-Warray-bounds]
101 | return (set->sig[1] | set->sig[0]) == 0;
| ^ ~
arch/x86/include/asm/signal.h:24:2: note: array 'sig' declared here
24 | unsigned long sig[_NSIG_WORDS];
| ^ In file included from arch/x86/hyperv/hv_vtl.c:10:
In file included from arch/x86/include/asm/apic.h:6:
In file included from include/linux/static_call.h:135:
In file included from include/linux/cpu.h:17:
vim +/acpi_physical_address +179 arch/x86/include/asm/acpi.h
e7b66d16fe4172 Juergen Gross 2018-10-10 176 9221222c717dbd Juergen Gross 2024-08-09 177 #ifdef CONFIG_XEN_PV 9221222c717dbd Juergen Gross 2024-08-09 178 /* A Xen PV domain needs a special acpi_os_ioremap() handling. */ 9221222c717dbd Juergen Gross 2024-08-09 @179 extern void __iomem * (*acpi_os_ioremap)(acpi_physical_address phys, 9221222c717dbd Juergen Gross 2024-08-09 @180 acpi_size size); 9221222c717dbd Juergen Gross 2024-08-09 181 void __iomem *x86_acpi_os_ioremap(acpi_physical_address phys, acpi_size size); 9221222c717dbd Juergen Gross 2024-08-09 182 #define acpi_os_ioremap acpi_os_ioremap 9221222c717dbd Juergen Gross 2024-08-09 183 #endif
9221222c717dbd Juergen Gross 2024-08-09 184
Signature · 2 lines
0-DAY CI Kernel Test Service https://github.com/intel/lkp-tests/wiki
Re: [PATCH v7 8/9] x86/hyperv/vtl: Mark the wakeup mailbox page as private
Quoted 21 lines · kernel test robot
Hi Ricardo,
kernel test robot noticed the following build errors:
[auto build test ERROR on
6f85aad74a70d17919a64ecd93037aa51c08698d]url: https://github.com/intel-lab-lkp/linux/commits/Ricardo-Neri/x86-acpi-Add-functions-to-setup-and-access-the-wakeup-mailbox/20251118-011935 base: 6f85aad74a70d17919a64ecd93037aa51c08698d patch link: https://lore.kernel.org/r/20251117-rneri-wakeup-mailbox-v7-8-4a8b82ab7c2c%40linux.intel.com patch subject: [PATCH v7 8/9] x86/hyperv/vtl: Mark the wakeup mailbox page as private config: x86_64-allyesconfig (https://download.01.org/0day-ci/archive/20251119/[email protected]/config) compiler: clang version 20.1.8 (https://github.com/llvm/llvm-project 87f0227cb60147a26a1eeb4fb06e3b505e9c7261) reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20251119/[email protected]/reproduce)If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <[email protected]>
| Closes: https://lore.kernel.org/oe-kbuild-all/[email protected]/All errors (new ones prefixed by >>):
Quoted 7 lines
In file included from
arch/x86/hyperv/hv_vtl.c:9:
In file included fromarch/x86/include/asm/acpi.h:11:
In file included fromarch/x86/include/asm/numa.h:8:Quoted 1 line
Quoted 1 line
arch/x86/include/asm/topology.h:264:26: error: expected ';' after top level declarator264 | extern bool __read_mostly sysctl_sched_itmt_enabled; | ^ | ;
The patch in https://lore.kernel.org/all/20251117-rneri-topology-cpuinfo-bug-v1-1-a905bb5f91e2@linux.intel.com/
fixes this error too.
Quoted 117 lines
In file included from
arch/x86/hyperv/hv_vtl.c:9:Quoted 1 line
Quoted 1 line
arch/x86/include/asm/acpi.h:179:42: error: unknown type name 'acpi_physical_address'179 | extern void
__iomem* (*acpi_os_ioremap)(acpi_physical_address phys,
| ^Quoted 1 line
Quoted 1 line
arch/x86/include/asm/acpi.h:180:7: error: unknown type name 'acpi_size'180 | acpi_size size); | ^ arch/x86/include/asm/acpi.h:181:35: error: unknown type name 'acpi_physical_address' 181 | void __iomem *x86_acpi_os_ioremap(acpi_physical_address phys, acpi_size size); | ^ arch/x86/include/asm/acpi.h:181:63: error: unknown type name 'acpi_size' 181 | void __iomem *x86_acpi_os_ioremap(acpi_physical_address phys, acpi_size size); | ^ In file included from arch/x86/hyperv/hv_vtl.c:10: In file included from arch/x86/include/asm/apic.h:6: In file included from include/linux/static_call.h:135: In file included from include/linux/cpu.h:17: In file included from include/linux/node.h:18: In file included from include/linux/device.h:32: In file included from include/linux/device/driver.h:21: In file included from include/linux/module.h:20: In file included from include/linux/elf.h:6: In file included from arch/x86/include/asm/elf.h:10: In file included from arch/x86/include/asm/ia32.h:7: In file included from include/linux/compat.h:17: In file included from include/linux/fs.h:34: In file included from include/linux/percpu-rwsem.h:7: In file included from include/linux/rcuwait.h:6: In file included from include/linux/sched/signal.h:6: include/linux/signal.h:98:11: warning: array index 3 is past the end of the array (that has type 'unsigned long[1]') [-Warray-bounds] 98 | return (set->sig[3] | set->sig[2] | | ^ ~ arch/x86/include/asm/signal.h:24:2: note: array 'sig' declared here 24 | unsigned long sig[_NSIG_WORDS]; | ^ In file included from arch/x86/hyperv/hv_vtl.c:10: In file included from arch/x86/include/asm/apic.h:6: In file included from include/linux/static_call.h:135: In file included from include/linux/cpu.h:17: In file included from include/linux/node.h:18: In file included from include/linux/device.h:32: In file included from include/linux/device/driver.h:21: In file included from include/linux/module.h:20: In file included from include/linux/elf.h:6: In file included from arch/x86/include/asm/elf.h:10: In file included from arch/x86/include/asm/ia32.h:7: In file included from include/linux/compat.h:17: In file included from include/linux/fs.h:34: In file included from include/linux/percpu-rwsem.h:7: In file included from include/linux/rcuwait.h:6: In file included from include/linux/sched/signal.h:6: include/linux/signal.h:98:25: warning: array index 2 is past the end of the array (that has type 'unsigned long[1]') [-Warray-bounds] 98 | return (set->sig[3] | set->sig[2] | | ^ ~ arch/x86/include/asm/signal.h:24:2: note: array 'sig' declared here 24 | unsigned long sig[_NSIG_WORDS]; | ^ In file included from arch/x86/hyperv/hv_vtl.c:10: In file included from arch/x86/include/asm/apic.h:6: In file included from include/linux/static_call.h:135: In file included from include/linux/cpu.h:17: In file included from include/linux/node.h:18: In file included from include/linux/device.h:32: In file included from include/linux/device/driver.h:21: In file included from include/linux/module.h:20: In file included from include/linux/elf.h:6: In file included from arch/x86/include/asm/elf.h:10: In file included from arch/x86/include/asm/ia32.h:7: In file included from include/linux/compat.h:17: In file included from include/linux/fs.h:34: In file included from include/linux/percpu-rwsem.h:7: In file included from include/linux/rcuwait.h:6: In file included from include/linux/sched/signal.h:6: include/linux/signal.h:99:4: warning: array index 1 is past the end of the array (that has type 'unsigned long[1]') [-Warray-bounds] 99 | set->sig[1] | set->sig[0]) == 0; | ^ ~ arch/x86/include/asm/signal.h:24:2: note: array 'sig' declared here 24 | unsigned long sig[_NSIG_WORDS]; | ^ In file included from arch/x86/hyperv/hv_vtl.c:10: In file included from arch/x86/include/asm/apic.h:6: In file included from include/linux/static_call.h:135: In file included from include/linux/cpu.h:17: In file included from include/linux/node.h:18: In file included from include/linux/device.h:32: In file included from include/linux/device/driver.h:21: In file included from include/linux/module.h:20: In file included from include/linux/elf.h:6: In file included from arch/x86/include/asm/elf.h:10: In file included from arch/x86/include/asm/ia32.h:7: In file included from include/linux/compat.h:17: In file included from include/linux/fs.h:34: In file included from include/linux/percpu-rwsem.h:7: In file included from include/linux/rcuwait.h:6: In file included from include/linux/sched/signal.h:6: include/linux/signal.h:101:11: warning: array index 1 is past the end of the array (that has type 'unsigned long[1]') [-Warray-bounds] 101 | return (set->sig[1] | set->sig[0]) == 0; | ^ ~ arch/x86/include/asm/signal.h:24:2: note: array 'sig' declared here 24 | unsigned long sig[_NSIG_WORDS]; | ^In file included from
arch/x86/hyperv/hv_vtl.c:10:
In file included fromarch/x86/include/asm/apic.h:6:
In file included frominclude/linux/static_call.h:135:
In file included frominclude/linux/cpu.h:17:vim +/acpi_physical_address +179
arch/x86/include/asm/acpi.he7b66d16fe4172 Juergen Gross 2018-10-10 176 9221222c717dbd Juergen Gross 2024-08-09 177 #ifdef CONFIG_XEN_PV 9221222c717dbd Juergen Gross 2024-08-09 178 /* A Xen PV domain needs a special acpi_os_ioremap() handling. */ 9221222c717dbd Juergen Gross 2024-08-09 @179 extern void __iomem * (*acpi_os_ioremap)(acpi_physical_address phys, 9221222c717dbd Juergen Gross 2024-08-09 @180 acpi_size size); 9221222c717dbd Juergen Gross 2024-08-09 181 void __iomem *x86_acpi_os_ioremap(acpi_physical_address phys, acpi_size size); 9221222c717dbd Juergen Gross 2024-08-09 182 #define acpi_os_ioremap acpi_os_ioremap 9221222c717dbd Juergen Gross 2024-08-09 183 #endif
Quoted 1 line
9221222c717dbdJuergen Gross 2024-08-09 184
These errors are caused by a missing header in this patch. I will fix this issue.
PATCH v7 9/9 x86/hyperv/vtl: Use the wakeup mailbox to boot secondary CPUs
In reply to Ricardo NeriThe hypervisor is an untrusted entity for TDX guests. It cannot be used to boot secondary CPUs. The function hv_vtl_wakeup_secondary_cpu() cannot be used.
Instead, the virtual firmware boots the secondary CPUs and places them in a state to transfer control to the kernel using the wakeup mailbox. The firmware enumerates the mailbox via either an ACPI table or a DeviceTree node.
If the wakeup mailbox is present, the kernel updates the APIC callback wakeup_secondary_cpu_64() to use it.
---
Changes in v7:
- None
Changes in v6:
- Added Reviewed-by tag from Dexuan. Thanks!
Changes in v5:
- None
Changes in v4:
- Added Reviewed-by tag from Michael. Thanks!
Changes in v3:
- Unconditionally use the wakeup mailbox in a TDX confidential VM. (Michael).
- Edited the commit message for clarity.
Changes in v2:
- None
--- arch/x86/hyperv/hv_vtl.c | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-)
arch/x86/hyperv/hv_vtl.c +9 -1
@@ -265,7 +265,15 @@ int __init hv_vtl_early_init(void)panic ("XSAVE has to be disabled as it is not supported by this module.\n" "Please add 'noxsave' to the kernel command line.\n" ); -apic_update_callback (wakeup_secondary_cpu_64, hv_vtl_wakeup_secondary_cpu); +/* +* TDX confidential VMs do not trust the hypervisor and cannot use it to +* boot secondary CPUs. Instead, they will be booted using the wakeup +* mailbox if detected during boot. See setup_arch(). +* +* There is no paravisor present if we are here. +*/ +if (!hv_isolation_type_tdx ()) +apic_update_callback (wakeup_secondary_cpu_64, hv_vtl_wakeup_secondary_cpu);return 0 ; }
Signature · 1 line
2.43.0
Quoted 7 lines · Ricardo Neri
Hi,
Many thanks to Boris, Rafael, Rob, and Dexuan for their valuable feedback! The main change in this version is the removal of the patch that moved the ACPI mailbox code from the x86 ACPI subsystem to a generic location. Users with DeviceTree-based firmware who wish use the ACPI wakeup mailbox need to select
CONFIG_ACPI=y.
Also, this patchset uncovered, IMHO, missing definitions in arch/x86/asm/ topology.h. I posted a patch for that effect here:
https://lore.kernel.org/all/20251117-rneri-topology-cpuinfo-bug-v1-1-a905bb5f91e2@linux.intel.com/
I can repost this patchset along with the patch above if it makes things easier.
BR,
Ricardo