arch/x86, x86/hyperv: Few fixes for the AP startup

PATCH In review Roman Kisel 4 participants 9 messages May 7, 2025 – May 8, 2025

Subsystems (5)

x86 ACPI Hyper-V/Azure core and drivers Microsoft MANA RDMA driver +1 more subsystems

lore

Share & tools

lore

Roman Kisel

author

2025-05-07 18:22 UTC #0 lore raw

cover arch/x86, x86/hyperv: Few fixes for the AP startup

This patchset combines two patches that depend on each other and were not applying cleanly:
1. Fix APIC ID and VP index confusion in hv_snp_boot_ap():

    https://lore.kernel.org/linux-hyperv/[email protected]/
  2. Provide the CPU number in the wakeup AP callback:
    https://lore.kernel.org/linux-hyperv/[email protected]/

I rebased the patches on top of the latest hyperv-next tree and updated the second patch that broke the linux-next build. That fix that, I made one non-functional change:
updated the signature of numachip_wakeup_secondary() to match the parameter list of wakeup_secondary_cpu().

Roman Kisel (2):
x86/hyperv: Fix APIC ID and VP index confusion in hv_snp_boot_ap() arch/x86: Provide the CPU number in the wakeup AP callback

 arch/x86/coco/sev/core.c             | 13 ++-----
 arch/x86/hyperv/hv_init.c            | 33 +++++++++++++++++
 arch/x86/hyperv/hv_vtl.c             | 54 ++++------------------------
 arch/x86/hyperv/ivm.c                | 11 ++++--
 arch/x86/include/asm/apic.h          |  8 ++---
 arch/x86/include/asm/mshyperv.h      |  7 ++--
 arch/x86/kernel/acpi/madt_wakeup.c   |  2 +-
 arch/x86/kernel/apic/apic_noop.c     |  8 ++++-
 arch/x86/kernel/apic/apic_numachip.c |  2 +-
 arch/x86/kernel/apic/x2apic_uv_x.c   |  2 +-
 arch/x86/kernel/smpboot.c            | 10 +++---
 include/hyperv/hvgdk_mini.h          |  2 +-
 12 files changed, 76 insertions(+), 76 deletions(-)

base-commit: 9b0844d87b1407681b78130429f798beb366f43f

Signature · 1 line

2.43.0

Roman Kisel

author

2025-05-07 18:22 UTC #1 lore raw

PATCH 1/2 x86/hyperv: Fix APIC ID and VP index confusion in hv_snp_boot_ap()

To start an application processor in SNP-isolated guest, a hypercall is used that takes a virtual processor index. The hv_snp_boot_ap() function uses that START_VP hypercall but passes as VP index to it what it receives as a wakeup_secondary_cpu_64 callback: the APIC ID.

As those two aren't generally interchangeable, that may lead to hung APs if the VP index and the APIC ID don't match up.

Update the parameter names to avoid confusion as to what the parameter is. Use the APIC ID to the VP index conversion to provide the correct input to the hypercall.

Cc [email protected]

Fixes 44676bb9d566 ("x86/hyperv: Add smp support for SEV-SNP guest")

Signed-off-by Roman Kisel <[email protected]>

Reviewed-by Michael Kelley <[email protected]>

---
 arch/x86/hyperv/hv_init.c       | 33 +++++++++++++++++++++++++
 arch/x86/hyperv/hv_vtl.c        | 44 +++++----------------------------
 arch/x86/hyperv/ivm.c           | 22 +++++++++++++++--
 arch/x86/include/asm/mshyperv.h |  6 +++--
 include/hyperv/hvgdk_mini.h     |  2 +-
 5 files changed, 64 insertions(+), 43 deletions(-)

arch/x86/hyperv/hv_init.c +33 -0

index 3b569291dfed..9a8fc144e195 100644
@@ -672,3 +672,36 @@ bool hv_is_hyperv_initialized(void)
 	return hypercall_msr.enable;
 }
 EXPORT_SYMBOL_GPL(hv_is_hyperv_initialized);
+
+int hv_apicid_to_vp_index(u32 apic_id)
+{
+	u64 control;
+	u64 status;
+	unsigned long irq_flags;
+	struct hv_get_vp_from_apic_id_in *input;
+	u32 *output, ret;
+
+	local_irq_save(irq_flags);
+
+	input = *this_cpu_ptr(hyperv_pcpu_input_arg);
+	memset(input, 0, sizeof(*input));
+	input->partition_id = HV_PARTITION_ID_SELF;
+	input->apic_ids[0] = apic_id;
+
+	output = *this_cpu_ptr(hyperv_pcpu_output_arg);
+
+	control = HV_HYPERCALL_REP_COMP_1 | HVCALL_GET_VP_INDEX_FROM_APIC_ID;
+	status = hv_do_hypercall(control, input, output);
+	ret = output[0];
+
+	local_irq_restore(irq_flags);
+
+	if (!hv_result_success(status)) {
+		pr_err("failed to get vp index from apic id %d, status %#llx\n",
+		       apic_id, status);
+		return -EINVAL;
+	}
+
+	return ret;
+}
+EXPORT_SYMBOL_GPL(hv_apicid_to_vp_index);

arch/x86/hyperv/hv_vtl.c +6 ~~-38~~

65 lines not on this page · raw message

arch/x86/hyperv/ivm.c +20 -2

index 77bf05f06b9e..0cc239cdb4da 100644
@@ -9,6 +9,7 @@
 #include <linux/bitfield.h>
 #include <linux/types.h>
 #include <linux/slab.h>
+#include <linux/cpu.h>
 #include <asm/svm.h>
 #include <asm/sev.h>
 #include <asm/io.h>

@@ -288,7 +289,7 @@ static void snp_cleanup_vmsa(struct sev_es_save_area *vmsa)
 		free_page((unsigned long)vmsa);
 }
 
-int hv_snp_boot_ap(u32 cpu, unsigned long start_ip)
+int hv_snp_boot_ap(u32 apic_id, unsigned long start_ip)
 {
 	struct sev_es_save_area *vmsa = (struct sev_es_save_area *)
 		__get_free_page(GFP_KERNEL | __GFP_ZERO);

@@ -297,10 +298,27 @@ int hv_snp_boot_ap(u32 cpu, unsigned long start_ip)
 	u64 ret, retry = 5;
 	struct hv_enable_vp_vtl *start_vp_input;
 	unsigned long flags;
+	int cpu, vp_index;
 
 	if (!vmsa)
 		return -ENOMEM;
 
+	/* Find the Hyper-V VP index which might be not the same as APIC ID */
+	vp_index = hv_apicid_to_vp_index(apic_id);
+	if (vp_index < 0 || vp_index > ms_hyperv.max_vp_index)
+		return -EINVAL;
+
+	/*
+	 * Find the Linux CPU number for addressing the per-CPU data, and it
+	 * might not be the same as APIC ID.
+	 */
+	for_each_present_cpu(cpu) {
+		if (arch_match_cpu_phys_id(cpu, apic_id))
+			break;
+	}
+	if (cpu >= nr_cpu_ids)
+		return -EINVAL;
+
 	native_store_gdt(&gdtr);
 
 	vmsa->gdtr.base = gdtr.address;

@@ -348,7 +366,7 @@ int hv_snp_boot_ap(u32 cpu, unsigned long start_ip)
 	start_vp_input = (struct hv_enable_vp_vtl *)ap_start_input_arg;
 	memset(start_vp_input, 0, sizeof(*start_vp_input));
 	start_vp_input->partition_id = -1;
-	start_vp_input->vp_index = cpu;
+	start_vp_input->vp_index = vp_index;
 	start_vp_input->target_vtl.target_vtl = ms_hyperv.vtl;
 	*(u64 *)&start_vp_input->vp_context = __pa(vmsa) | 1;

arch/x86/include/asm/mshyperv.h +4 -2

index bab5ccfc60a7..0b9a3a307d06 100644
@@ -268,11 +268,11 @@ int hv_unmap_ioapic_interrupt(int ioapic_id, struct hv_interrupt_entry *entry);
 #ifdef CONFIG_AMD_MEM_ENCRYPT
 bool hv_ghcb_negotiate_protocol(void);
 void __noreturn hv_ghcb_terminate(unsigned int set, unsigned int reason);
-int hv_snp_boot_ap(u32 cpu, unsigned long start_ip);
+int hv_snp_boot_ap(u32 apic_id, unsigned long start_ip);
 #else
 static inline bool hv_ghcb_negotiate_protocol(void) { return false; }
 static inline void hv_ghcb_terminate(unsigned int set, unsigned int reason) {}
-static inline int hv_snp_boot_ap(u32 cpu, unsigned long start_ip) { return 0; }
+static inline int hv_snp_boot_ap(u32 apic_id, unsigned long start_ip) { return 0; }
 #endif
 
 #if defined(CONFIG_AMD_MEM_ENCRYPT) || defined(CONFIG_INTEL_TDX_GUEST)

@@ -306,6 +306,7 @@ static __always_inline u64 hv_raw_get_msr(unsigned int reg)
 {
 	return __rdmsr(reg);
 }
+int hv_apicid_to_vp_index(u32 apic_id);
 
 #else /* CONFIG_HYPERV */
 static inline void hyperv_init(void) {}

@@ -327,6 +328,7 @@ static inline void hv_set_msr(unsigned int reg, u64 value) { }
 static inline u64 hv_get_msr(unsigned int reg) { return 0; }
 static inline void hv_set_non_nested_msr(unsigned int reg, u64 value) { }
 static inline u64 hv_get_non_nested_msr(unsigned int reg) { return 0; }
+static inline int hv_apicid_to_vp_index(u32 apic_id) { return -EINVAL; }
 #endif /* CONFIG_HYPERV */

include/hyperv/hvgdk_mini.h +1 -1

index cf0923dc727d..2d431b53f587 100644
@@ -475,7 +475,7 @@ union hv_vp_assist_msr_contents {	 /* HV_REGISTER_VP_ASSIST_PAGE */
 #define HVCALL_CREATE_PORT				0x0095
 #define HVCALL_CONNECT_PORT				0x0096
 #define HVCALL_START_VP					0x0099
-#define HVCALL_GET_VP_ID_FROM_APIC_ID			0x009a
+#define HVCALL_GET_VP_INDEX_FROM_APIC_ID			0x009a
 #define HVCALL_FLUSH_GUEST_PHYSICAL_ADDRESS_SPACE	0x00af
 #define HVCALL_FLUSH_GUEST_PHYSICAL_ADDRESS_LIST	0x00b0
 #define HVCALL_SIGNAL_EVENT_DIRECT			0x00c0

Signature · 1 line

2.43.0

Roman Kisel

author

2025-05-07 18:22 UTC #2 lore raw

PATCH 2/2 arch/x86: Provide the CPU number in the wakeup AP callback

In reply to Roman Kisel

When starting APs, confidential guests and paravisor guests need to know the CPU number, and the pattern of using the linear search has emerged in several places. With N processors that leads to the O(N^2) time complexity.

Provide the CPU number in the AP wake up callback so that one can get the CPU number in constant time.

Signed-off-by Roman Kisel <[email protected]>

Reviewed-by Tom Lendacky <[email protected]>

Reviewed-by Thomas Gleixner <[email protected]>

Reviewed-by Michael Kelley <[email protected]>

---
 arch/x86/coco/sev/core.c             | 13 ++-----------
 arch/x86/hyperv/hv_vtl.c             | 12 ++----------
 arch/x86/hyperv/ivm.c                | 15 ++-------------
 arch/x86/include/asm/apic.h          |  8 ++++----
 arch/x86/include/asm/mshyperv.h      |  5 +++--
 arch/x86/kernel/acpi/madt_wakeup.c   |  2 +-
 arch/x86/kernel/apic/apic_noop.c     |  8 +++++++-
 arch/x86/kernel/apic/apic_numachip.c |  2 +-
 arch/x86/kernel/apic/x2apic_uv_x.c   |  2 +-
 arch/x86/kernel/smpboot.c            | 10 +++++-----
 10 files changed, 28 insertions(+), 49 deletions(-)

arch/x86/coco/sev/core.c +2 ~~-11~~

index b0c1a7a57497..7780d55d1833 100644
@@ -1177,7 +1177,7 @@ static void snp_cleanup_vmsa(struct sev_es_save_area *vmsa, int apic_id)
 		free_page((unsigned long)vmsa);
 }
 
-static int wakeup_cpu_via_vmgexit(u32 apic_id, unsigned long start_ip)
+static int wakeup_cpu_via_vmgexit(u32 apic_id, unsigned long start_ip, unsigned int cpu)
 {
 	struct sev_es_save_area *cur_vmsa, *vmsa;
 	struct ghcb_state state;

@@ -1185,7 +1185,7 @@ static int wakeup_cpu_via_vmgexit(u32 apic_id, unsigned long start_ip)
 	unsigned long flags;
 	struct ghcb *ghcb;
 	u8 sipi_vector;
-	int cpu, ret;
+	int ret;
 	u64 cr4;
 
 	/*

@@ -1206,15 +1206,6 @@ static int wakeup_cpu_via_vmgexit(u32 apic_id, unsigned long start_ip)
 
 	/* Override start_ip with known protected guest start IP */
 	start_ip = real_mode_header->sev_es_trampoline_start;
-
-	/* Find the logical CPU for the APIC ID */
-	for_each_present_cpu(cpu) {
-		if (arch_match_cpu_phys_id(cpu, apic_id))
-			break;
-	}
-	if (cpu >= nr_cpu_ids)
-		return -EINVAL;
-
 	cur_vmsa = per_cpu(sev_vmsa, cpu);
 
 	/*

arch/x86/hyperv/hv_vtl.c +2 ~~-10~~

index 2f32ac1ae40e..3d149a2ca4c8 100644
@@ -211,17 +211,9 @@ static int hv_vtl_bringup_vcpu(u32 target_vp_index, int cpu, u64 eip_ignored)
 	return ret;
 }
 
-static int hv_vtl_wakeup_secondary_cpu(u32 apicid, unsigned long start_eip)
+static int hv_vtl_wakeup_secondary_cpu(u32 apicid, unsigned long start_eip, unsigned int cpu)
 {
-	int vp_index, cpu;
-
-	/* Find the logical CPU for the APIC ID */
-	for_each_present_cpu(cpu) {
-		if (arch_match_cpu_phys_id(cpu, apicid))
-			break;
-	}
-	if (cpu >= nr_cpu_ids)
-		return -EINVAL;
+	int vp_index;
 
 	pr_debug("Bringing up CPU with APIC ID %d in VTL2...\n", apicid);
 	vp_index = hv_apicid_to_vp_index(apicid);

arch/x86/hyperv/ivm.c +2 ~~-13~~

index 0cc239cdb4da..e21557b24d19 100644
@@ -289,7 +289,7 @@ static void snp_cleanup_vmsa(struct sev_es_save_area *vmsa)
 		free_page((unsigned long)vmsa);
 }
 
-int hv_snp_boot_ap(u32 apic_id, unsigned long start_ip)
+int hv_snp_boot_ap(u32 apic_id, unsigned long start_ip, unsigned int cpu)
 {
 	struct sev_es_save_area *vmsa = (struct sev_es_save_area *)
 		__get_free_page(GFP_KERNEL | __GFP_ZERO);

@@ -298,7 +298,7 @@ int hv_snp_boot_ap(u32 apic_id, unsigned long start_ip)
 	u64 ret, retry = 5;
 	struct hv_enable_vp_vtl *start_vp_input;
 	unsigned long flags;
-	int cpu, vp_index;
+	int vp_index;
 
 	if (!vmsa)
 		return -ENOMEM;

@@ -308,17 +308,6 @@ int hv_snp_boot_ap(u32 apic_id, unsigned long start_ip)
 	if (vp_index < 0 || vp_index > ms_hyperv.max_vp_index)
 		return -EINVAL;
 
-	/*
-	 * Find the Linux CPU number for addressing the per-CPU data, and it
-	 * might not be the same as APIC ID.
-	 */
-	for_each_present_cpu(cpu) {
-		if (arch_match_cpu_phys_id(cpu, apic_id))
-			break;
-	}
-	if (cpu >= nr_cpu_ids)
-		return -EINVAL;
-
 	native_store_gdt(&gdtr);
 
 	vmsa->gdtr.base = gdtr.address;

arch/x86/include/asm/apic.h +4 -4

index c903d358405d..eaf43d446203 100644
@@ -313,9 +313,9 @@ struct apic {
 	u32	(*get_apic_id)(u32 id);
 
 	/* wakeup_secondary_cpu */
-	int	(*wakeup_secondary_cpu)(u32 apicid, unsigned long start_eip);
+	int	(*wakeup_secondary_cpu)(u32 apicid, unsigned long start_eip, unsigned int cpu);
 	/* wakeup secondary CPU using 64-bit wakeup point */
-	int	(*wakeup_secondary_cpu_64)(u32 apicid, unsigned long start_eip);
+	int	(*wakeup_secondary_cpu_64)(u32 apicid, unsigned long start_eip, unsigned int cpu);
 
 	char	*name;
 };

@@ -333,8 +333,8 @@ struct apic_override {
 	void	(*send_IPI_self)(int vector);
 	u64	(*icr_read)(void);
 	void	(*icr_write)(u32 low, u32 high);
-	int	(*wakeup_secondary_cpu)(u32 apicid, unsigned long start_eip);
-	int	(*wakeup_secondary_cpu_64)(u32 apicid, unsigned long start_eip);
+	int	(*wakeup_secondary_cpu)(u32 apicid, unsigned long start_eip, unsigned int cpu);
+	int	(*wakeup_secondary_cpu_64)(u32 apicid, unsigned long start_eip, unsigned int cpu);
 };
 
 /*

arch/x86/include/asm/mshyperv.h +3 -2

index 0b9a3a307d06..5ec92e3e2e37 100644
@@ -268,11 +268,12 @@ int hv_unmap_ioapic_interrupt(int ioapic_id, struct hv_interrupt_entry *entry);
 #ifdef CONFIG_AMD_MEM_ENCRYPT
 bool hv_ghcb_negotiate_protocol(void);
 void __noreturn hv_ghcb_terminate(unsigned int set, unsigned int reason);
-int hv_snp_boot_ap(u32 apic_id, unsigned long start_ip);
+int hv_snp_boot_ap(u32 apic_id, unsigned long start_ip, unsigned int cpu);
 #else
 static inline bool hv_ghcb_negotiate_protocol(void) { return false; }
 static inline void hv_ghcb_terminate(unsigned int set, unsigned int reason) {}
-static inline int hv_snp_boot_ap(u32 apic_id, unsigned long start_ip) { return 0; }
+static inline int hv_snp_boot_ap(u32 apic_id, unsigned long start_ip,
+		unsigned int cpu) { return 0; }
 #endif
 
 #if defined(CONFIG_AMD_MEM_ENCRYPT) || defined(CONFIG_INTEL_TDX_GUEST)

arch/x86/kernel/acpi/madt_wakeup.c +1 -1

index f36f28405dcc..6d7603511f52 100644
@@ -126,7 +126,7 @@ static int __init acpi_mp_setup_reset(u64 reset_vector)
 	return 0;
 }
 
-static int acpi_wakeup_cpu(u32 apicid, unsigned long start_ip)
+static int acpi_wakeup_cpu(u32 apicid, unsigned long start_ip, unsigned int cpu)
 {
 	if (!acpi_mp_wake_mailbox_paddr) {
 		pr_warn_once("No MADT mailbox: cannot bringup secondary CPUs. Booting with kexec?\n");

arch/x86/kernel/apic/apic_noop.c +7 -1

index b5bb7a2e8340..58abb941c45b 100644
@@ -27,7 +27,13 @@ static void noop_send_IPI_allbutself(int vector) { }
 static void noop_send_IPI_all(int vector) { }
 static void noop_send_IPI_self(int vector) { }
 static void noop_apic_icr_write(u32 low, u32 id) { }
-static int noop_wakeup_secondary_cpu(u32 apicid, unsigned long start_eip) { return -1; }
+
+static int noop_wakeup_secondary_cpu(u32 apicid, unsigned long start_eip,
+	unsigned int cpu)
+{
+	return -1;
+}
+
 static u64 noop_apic_icr_read(void) { return 0; }
 static u32 noop_get_apic_id(u32 apicid) { return 0; }
 static void noop_apic_eoi(void) { }

arch/x86/kernel/apic/apic_numachip.c +1 -1

index 16410f087b7a..333536b89bde 100644
@@ -56,7 +56,7 @@ static void numachip2_apic_icr_write(int apicid, unsigned int val)
 	numachip2_write32_lcsr(NUMACHIP2_APIC_ICR, (apicid << 12) | val);
 }
 
-static int numachip_wakeup_secondary(u32 phys_apicid, unsigned long start_rip)
+static int numachip_wakeup_secondary(u32 phys_apicid, unsigned long start_rip, unsigned int cpu)
 {
 	numachip_apic_icr_write(phys_apicid, APIC_DM_INIT);
 	numachip_apic_icr_write(phys_apicid, APIC_DM_STARTUP |

arch/x86/kernel/apic/x2apic_uv_x.c +1 -1

index 7fef504ca508..15209f220e1f 100644
@@ -667,7 +667,7 @@ static __init void build_uv_gr_table(void)
 	}
 }
 
-static int uv_wakeup_secondary(u32 phys_apicid, unsigned long start_rip)
+static int uv_wakeup_secondary(u32 phys_apicid, unsigned long start_rip, unsigned int cpu)
 {
 	unsigned long val;
 	int pnode;

arch/x86/kernel/smpboot.c +5 -5

index d6cf1e23c2a3..d52e9238e9fd 100644
@@ -695,7 +695,7 @@ static void send_init_sequence(u32 phys_apicid)
 /*
  * Wake up AP by INIT, INIT, STARTUP sequence.
  */
-static int wakeup_secondary_cpu_via_init(u32 phys_apicid, unsigned long start_eip)
+static int wakeup_secondary_cpu_via_init(u32 phys_apicid, unsigned long start_eip, unsigned int cpu)
 {
 	unsigned long send_status = 0, accept_status = 0;
 	int num_starts, j, maxlvt;

@@ -842,7 +842,7 @@ int common_cpu_up(unsigned int cpu, struct task_struct *idle)
  * Returns zero if startup was successfully sent, else error code from
  * ->wakeup_secondary_cpu.
  */
-static int do_boot_cpu(u32 apicid, int cpu, struct task_struct *idle)
+static int do_boot_cpu(u32 apicid, unsigned int cpu, struct task_struct *idle)
 {
 	unsigned long start_ip = real_mode_header->trampoline_start;
 	int ret;

@@ -896,11 +896,11 @@ static int do_boot_cpu(u32 apicid, int cpu, struct task_struct *idle)
 	 * - Use an INIT boot APIC message
 	 */
 	if (apic->wakeup_secondary_cpu_64)
-		ret = apic->wakeup_secondary_cpu_64(apicid, start_ip);
+		ret = apic->wakeup_secondary_cpu_64(apicid, start_ip, cpu);
 	else if (apic->wakeup_secondary_cpu)
-		ret = apic->wakeup_secondary_cpu(apicid, start_ip);
+		ret = apic->wakeup_secondary_cpu(apicid, start_ip, cpu);
 	else
-		ret = wakeup_secondary_cpu_via_init(apicid, start_ip);
+		ret = wakeup_secondary_cpu_via_init(apicid, start_ip, cpu);
 
 	/* If the wakeup mechanism failed, cleanup the warm reset vector */
 	if (ret)

Signature · 1 line

2.43.0

Rafael J. Wysocki

maintainer Acked-by on v1 2/2

2025-05-07 20:09 UTC #3 lore raw

Re: [PATCH hyperv-next 2/2] arch/x86: Provide the CPU number in the wakeup AP callback

Quoted 6 lines · Roman Kisel

When starting APs, confidential guests and paravisor guests need to know the CPU number, and the pattern of using the linear search has emerged in several places. With N processors that leads to the O(N^2) time complexity.

Quoted 8 lines

Provide the CPU number in the AP wake up callback so that one can get the CPU number in constant time.
Suggested-by Michael Kelley <[email protected]>
Signed-off-by Roman Kisel <[email protected]>
Reviewed-by Tom Lendacky <[email protected]>
Reviewed-by Thomas Gleixner <[email protected]>
Reviewed-by Michael Kelley <[email protected]>

For the ACPI bits

Acked-by Rafael J. Wysocki <[email protected]>

and I'm assuming that the x86 folks will take care of this.

Thanks!

Quoted 259 lines

---
 arch/x86/coco/sev/core.c             | 13 ++-----------
 arch/x86/hyperv/hv_vtl.c             | 12 ++----------
 arch/x86/hyperv/ivm.c                | 15 ++-------------
 arch/x86/include/asm/apic.h          |  8 ++++----
 arch/x86/include/asm/mshyperv.h      |  5 +++--
 arch/x86/kernel/acpi/madt_wakeup.c   |  2 +-
 arch/x86/kernel/apic/apic_noop.c     |  8 +++++++-
 arch/x86/kernel/apic/apic_numachip.c |  2 +-
 arch/x86/kernel/apic/x2apic_uv_x.c   |  2 +-
 arch/x86/kernel/smpboot.c            | 10 +++++-----
 10 files changed, 28 insertions(+), 49 deletions(-)

arch/x86/coco/sev/core.c +2 ~~-11~~

index b0c1a7a57497..7780d55d1833 100644
@@ -1177,7 +1177,7 @@ static void snp_cleanup_vmsa(struct sev_es_save_area *vmsa, int apic_id)
                free_page((unsigned long)vmsa);
 }
 
-static int wakeup_cpu_via_vmgexit(u32 apic_id, unsigned long start_ip)
+static int wakeup_cpu_via_vmgexit(u32 apic_id, unsigned long start_ip, unsigned int cpu)
 {
        struct sev_es_save_area *cur_vmsa, *vmsa;
        struct ghcb_state state;

@@ -1185,7 +1185,7 @@ static int wakeup_cpu_via_vmgexit(u32 apic_id, unsigned long start_ip)
        unsigned long flags;
        struct ghcb *ghcb;
        u8 sipi_vector;
-       int cpu, ret;
+       int ret;
        u64 cr4;
 
        /*

@@ -1206,15 +1206,6 @@ static int wakeup_cpu_via_vmgexit(u32 apic_id, unsigned long start_ip)
 
        /* Override start_ip with known protected guest start IP */
        start_ip = real_mode_header->sev_es_trampoline_start;
-
-       /* Find the logical CPU for the APIC ID */
-       for_each_present_cpu(cpu) {
-               if (arch_match_cpu_phys_id(cpu, apic_id))
-                       break;
-       }
-       if (cpu >= nr_cpu_ids)
-               return -EINVAL;
-
        cur_vmsa = per_cpu(sev_vmsa, cpu);
 
        /*

arch/x86/hyperv/hv_vtl.c +2 ~~-10~~

index 2f32ac1ae40e..3d149a2ca4c8 100644
@@ -211,17 +211,9 @@ static int hv_vtl_bringup_vcpu(u32 target_vp_index, int cpu, u64 eip_ignored)
        return ret;
 }
 
-static int hv_vtl_wakeup_secondary_cpu(u32 apicid, unsigned long start_eip)
+static int hv_vtl_wakeup_secondary_cpu(u32 apicid, unsigned long start_eip, unsigned int cpu)
 {
-       int vp_index, cpu;
-
-       /* Find the logical CPU for the APIC ID */
-       for_each_present_cpu(cpu) {
-               if (arch_match_cpu_phys_id(cpu, apicid))
-                       break;
-       }
-       if (cpu >= nr_cpu_ids)
-               return -EINVAL;
+       int vp_index;
 
        pr_debug("Bringing up CPU with APIC ID %d in VTL2...\n", apicid);
        vp_index = hv_apicid_to_vp_index(apicid);

arch/x86/hyperv/ivm.c +2 ~~-13~~

index 0cc239cdb4da..e21557b24d19 100644
@@ -289,7 +289,7 @@ static void snp_cleanup_vmsa(struct sev_es_save_area *vmsa)
                free_page((unsigned long)vmsa);
 }
 
-int hv_snp_boot_ap(u32 apic_id, unsigned long start_ip)
+int hv_snp_boot_ap(u32 apic_id, unsigned long start_ip, unsigned int cpu)
 {
        struct sev_es_save_area *vmsa = (struct sev_es_save_area *)
                __get_free_page(GFP_KERNEL | __GFP_ZERO);

@@ -298,7 +298,7 @@ int hv_snp_boot_ap(u32 apic_id, unsigned long start_ip)
        u64 ret, retry = 5;
        struct hv_enable_vp_vtl *start_vp_input;
        unsigned long flags;
-       int cpu, vp_index;
+       int vp_index;
 
        if (!vmsa)
                return -ENOMEM;

@@ -308,17 +308,6 @@ int hv_snp_boot_ap(u32 apic_id, unsigned long start_ip)
        if (vp_index < 0 || vp_index > ms_hyperv.max_vp_index)
                return -EINVAL;
 
-       /*
-        * Find the Linux CPU number for addressing the per-CPU data, and it
-        * might not be the same as APIC ID.
-        */
-       for_each_present_cpu(cpu) {
-               if (arch_match_cpu_phys_id(cpu, apic_id))
-                       break;
-       }
-       if (cpu >= nr_cpu_ids)
-               return -EINVAL;
-
        native_store_gdt(&gdtr);
 
        vmsa->gdtr.base = gdtr.address;

arch/x86/include/asm/apic.h +4 -4

index c903d358405d..eaf43d446203 100644
@@ -313,9 +313,9 @@ struct apic {
        u32     (*get_apic_id)(u32 id);
 
        /* wakeup_secondary_cpu */
-       int     (*wakeup_secondary_cpu)(u32 apicid, unsigned long start_eip);
+       int     (*wakeup_secondary_cpu)(u32 apicid, unsigned long start_eip, unsigned int cpu);
        /* wakeup secondary CPU using 64-bit wakeup point */
-       int     (*wakeup_secondary_cpu_64)(u32 apicid, unsigned long start_eip);
+       int     (*wakeup_secondary_cpu_64)(u32 apicid, unsigned long start_eip, unsigned int cpu);
 
        char    *name;
 };

@@ -333,8 +333,8 @@ struct apic_override {
        void    (*send_IPI_self)(int vector);
        u64     (*icr_read)(void);
        void    (*icr_write)(u32 low, u32 high);
-       int     (*wakeup_secondary_cpu)(u32 apicid, unsigned long start_eip);
-       int     (*wakeup_secondary_cpu_64)(u32 apicid, unsigned long start_eip);
+       int     (*wakeup_secondary_cpu)(u32 apicid, unsigned long start_eip, unsigned int cpu);
+       int     (*wakeup_secondary_cpu_64)(u32 apicid, unsigned long start_eip, unsigned int cpu);
 };
 
 /*

arch/x86/include/asm/mshyperv.h +3 -2

index 0b9a3a307d06..5ec92e3e2e37 100644
@@ -268,11 +268,12 @@ int hv_unmap_ioapic_interrupt(int ioapic_id, struct hv_interrupt_entry *entry);
 #ifdef CONFIG_AMD_MEM_ENCRYPT
 bool hv_ghcb_negotiate_protocol(void);
 void __noreturn hv_ghcb_terminate(unsigned int set, unsigned int reason);
-int hv_snp_boot_ap(u32 apic_id, unsigned long start_ip);
+int hv_snp_boot_ap(u32 apic_id, unsigned long start_ip, unsigned int cpu);
 #else
 static inline bool hv_ghcb_negotiate_protocol(void) { return false; }
 static inline void hv_ghcb_terminate(unsigned int set, unsigned int reason) {}
-static inline int hv_snp_boot_ap(u32 apic_id, unsigned long start_ip) { return 0; }
+static inline int hv_snp_boot_ap(u32 apic_id, unsigned long start_ip,
+               unsigned int cpu) { return 0; }
 #endif
 
 #if defined(CONFIG_AMD_MEM_ENCRYPT) || defined(CONFIG_INTEL_TDX_GUEST)

arch/x86/kernel/acpi/madt_wakeup.c +1 -1

index f36f28405dcc..6d7603511f52 100644
@@ -126,7 +126,7 @@ static int __init acpi_mp_setup_reset(u64 reset_vector)
        return 0;
 }
 
-static int acpi_wakeup_cpu(u32 apicid, unsigned long start_ip)
+static int acpi_wakeup_cpu(u32 apicid, unsigned long start_ip, unsigned int cpu)
 {
        if (!acpi_mp_wake_mailbox_paddr) {
                pr_warn_once("No MADT mailbox: cannot bringup secondary CPUs. Booting with kexec?\n");

arch/x86/kernel/apic/apic_noop.c +7 -1

index b5bb7a2e8340..58abb941c45b 100644
@@ -27,7 +27,13 @@ static void noop_send_IPI_allbutself(int vector) { }
 static void noop_send_IPI_all(int vector) { }
 static void noop_send_IPI_self(int vector) { }
 static void noop_apic_icr_write(u32 low, u32 id) { }
-static int noop_wakeup_secondary_cpu(u32 apicid, unsigned long start_eip) { return -1; }
+
+static int noop_wakeup_secondary_cpu(u32 apicid, unsigned long start_eip,
+       unsigned int cpu)
+{
+       return -1;
+}
+
 static u64 noop_apic_icr_read(void) { return 0; }
 static u32 noop_get_apic_id(u32 apicid) { return 0; }
 static void noop_apic_eoi(void) { }

arch/x86/kernel/apic/apic_numachip.c +1 -1

index 16410f087b7a..333536b89bde 100644
@@ -56,7 +56,7 @@ static void numachip2_apic_icr_write(int apicid, unsigned int val)
        numachip2_write32_lcsr(NUMACHIP2_APIC_ICR, (apicid << 12) | val);
 }
 
-static int numachip_wakeup_secondary(u32 phys_apicid, unsigned long start_rip)
+static int numachip_wakeup_secondary(u32 phys_apicid, unsigned long start_rip, unsigned int cpu)
 {
        numachip_apic_icr_write(phys_apicid, APIC_DM_INIT);
        numachip_apic_icr_write(phys_apicid, APIC_DM_STARTUP |

arch/x86/kernel/apic/x2apic_uv_x.c +1 -1

index 7fef504ca508..15209f220e1f 100644
@@ -667,7 +667,7 @@ static __init void build_uv_gr_table(void)
        }
 }
 
-static int uv_wakeup_secondary(u32 phys_apicid, unsigned long start_rip)
+static int uv_wakeup_secondary(u32 phys_apicid, unsigned long start_rip, unsigned int cpu)
 {
        unsigned long val;
        int pnode;

arch/x86/kernel/smpboot.c +5 -6

index d6cf1e23c2a3..d52e9238e9fd 100644
@@ -695,7 +695,7 @@ static void send_init_sequence(u32 phys_apicid)
 /*
  * Wake up AP by INIT, INIT, STARTUP sequence.
  */
-static int wakeup_secondary_cpu_via_init(u32 phys_apicid, unsigned long start_eip)
+static int wakeup_secondary_cpu_via_init(u32 phys_apicid, unsigned long start_eip, unsigned int cpu)
 {
        unsigned long send_status = 0, accept_status = 0;
        int num_starts, j, maxlvt;

@@ -842,7 +842,7 @@ int common_cpu_up(unsigned int cpu, struct task_struct *idle)
  * Returns zero if startup was successfully sent, else error code from
  * ->wakeup_secondary_cpu.
  */
-static int do_boot_cpu(u32 apicid, int cpu, struct task_struct *idle)
+static int do_boot_cpu(u32 apicid, unsigned int cpu, struct task_struct *idle)
 {
        unsigned long start_ip = real_mode_header->trampoline_start;
        int ret;

@@ -896,11 +896,11 @@ static int do_boot_cpu(u32 apicid, int cpu, struct task_struct *idle)
         * - Use an INIT boot APIC message
         */
        if (apic->wakeup_secondary_cpu_64)
-               ret = apic->wakeup_secondary_cpu_64(apicid, start_ip);
+               ret = apic->wakeup_secondary_cpu_64(apicid, start_ip, cpu);
        else if (apic->wakeup_secondary_cpu)
-               ret = apic->wakeup_secondary_cpu(apicid, start_ip);
+               ret = apic->wakeup_secondary_cpu(apicid, start_ip, cpu);
        else
-               ret = wakeup_secondary_cpu_via_init(apicid, start_ip);
+               ret = wakeup_secondary_cpu_via_init(apicid, start_ip, cpu);
 
        /* If the wakeup mechanism failed, cleanup the warm reset vector */
        if (ret)
--

2.43.0

Wei Liu

maintainer

2025-05-08 01:10 UTC #4 lore raw

Re: [PATCH hyperv-next 2/2] arch/x86: Provide the CPU number in the wakeup AP callback

Quoted 16 lines · Rafael J. Wysocki

Quoted 6 lines · Roman Kisel
When starting APs, confidential guests and paravisor guests need to know the CPU number, and the pattern of using the linear search has emerged in several places. With N processors that leads to the O(N^2) time complexity.
Quoted 8 lines
Provide the CPU number in the AP wake up callback so that one can get the CPU number in constant time.
Suggested-by Michael Kelley <[email protected]>
Signed-off-by Roman Kisel <[email protected]>
Reviewed-by Tom Lendacky <[email protected]>
Reviewed-by Thomas Gleixner <[email protected]>
Reviewed-by Michael Kelley <[email protected]>

Quoted 6 lines

For the ACPI bits
Acked-by Rafael J. Wysocki <[email protected]>
and I'm assuming that the x86 folks will take care of this.

Thank you Rafael. I will take this patch via the hyperv-next tree.

Michael Kelley

2025-05-08 04:23 UTC #5 lore raw

In reply to Roman Kisel

From: Roman Kisel <[email protected]> Sent: Wednesday, May 7, 2025 11:22 AM

Quoted 8 lines

This patchset combines two patches that depend on each other and were not applying cleanly:
1. Fix APIC ID and VP index confusion in hv_snp_boot_ap():
    https://lore.kernel.org/linux-hyperv/[email protected]/ 
  2. Provide the CPU number in the wakeup AP callback:
    https://lore.kernel.org/linux-hyperv/[email protected]/ 

Quoted 8 lines

I rebased the patches on top of the latest hyperv-next tree and updated the second patch that broke the linux-next build. That fix that, I made one non-functional change:
updated the signature of numachip_wakeup_secondary() to match the parameter list of wakeup_secondary_cpu().
Roman Kisel (2):
x86/hyperv: Fix APIC ID and VP index confusion in hv_snp_boot_ap() arch/x86: Provide the CPU number in the wakeup AP callback

I think this works. It's unfortunate that Patch 1 adds 11 lines of code/comments that Patch 2 then deletes, which seems like undesirable churn. I was expecting adding the "cpu" parameter to come first, which then makes fixing the hv_snp_boot_ap() problem more straightforward. But looking more closely, hv_snp_boot_ap() already has a parameter erroneously named "cpu", so adding the correct "cpu" parameter isn't transparent. Hence the order you've chosen is probably the best resolution for a messy situation. :-)

Michael

Quoted 19 lines

 arch/x86/coco/sev/core.c             | 13 ++-----
 arch/x86/hyperv/hv_init.c            | 33 +++++++++++++++++
 arch/x86/hyperv/hv_vtl.c             | 54 ++++------------------------
 arch/x86/hyperv/ivm.c                | 11 ++++--
 arch/x86/include/asm/apic.h          |  8 ++---
 arch/x86/include/asm/mshyperv.h      |  7 ++--
 arch/x86/kernel/acpi/madt_wakeup.c   |  2 +-
 arch/x86/kernel/apic/apic_noop.c     |  8 ++++-
 arch/x86/kernel/apic/apic_numachip.c |  2 +-
 arch/x86/kernel/apic/x2apic_uv_x.c   |  2 +-
 arch/x86/kernel/smpboot.c            | 10 +++---
 include/hyperv/hvgdk_mini.h          |  2 +-
 12 files changed, 76 insertions(+), 76 deletions(-)

base-commit: 9b0844d87b1407681b78130429f798beb366f43f --
2.43.0

Roman Kisel

author

2025-05-08 15:09 UTC #6 lore raw

Quoted 1 line · Michael Kelley

From: Roman Kisel <[email protected]> Sent: Wednesday, May 7, 2025 11:22 AM

[...]>

Quoted 7 lines

I think this works. It's unfortunate that Patch 1 adds 11 lines of code/comments that Patch 2 then deletes, which seems like undesirable churn. I was expecting adding the "cpu" parameter to come first, which then makes fixing the hv_snp_boot_ap() problem more straightforward. But looking more closely, hv_snp_boot_ap() already has a parameter erroneously named "cpu", so adding the correct "cpu" parameter isn't transparent. Hence the order you've chosen is probably the best resolution for a messy situation. :-)

Thanks, Michael :) Looked as a good trade-off, went ahead with it. Will be happy to address any concerns of that folks might have!

Quoted 23 lines

Michael

Quoted 19 lines

  arch/x86/coco/sev/core.c             | 13 ++-----
  arch/x86/hyperv/hv_init.c            | 33 +++++++++++++++++
  arch/x86/hyperv/hv_vtl.c             | 54 ++++------------------------
  arch/x86/hyperv/ivm.c                | 11 ++++--
  arch/x86/include/asm/apic.h          |  8 ++---
  arch/x86/include/asm/mshyperv.h      |  7 ++--
  arch/x86/kernel/acpi/madt_wakeup.c   |  2 +-
  arch/x86/kernel/apic/apic_noop.c     |  8 ++++-
  arch/x86/kernel/apic/apic_numachip.c |  2 +-
  arch/x86/kernel/apic/x2apic_uv_x.c   |  2 +-
  arch/x86/kernel/smpboot.c            | 10 +++---
  include/hyperv/hvgdk_mini.h          |  2 +-
  12 files changed, 76 insertions(+), 76 deletions(-)

base-commit: 9b0844d87b1407681b78130429f798beb366f43f --
2.43.0

Signature · 2 lines

Thank you,
Roman

Wei Liu

maintainer

2025-05-08 18:39 UTC #7 lore raw

In reply to Roman Kisel

Quoted 7 lines · Roman Kisel

This patchset combines two patches that depend on each other and were not applying cleanly:
1. Fix APIC ID and VP index confusion in hv_snp_boot_ap():
    https://lore.kernel.org/linux-hyperv/[email protected]/
  2. Provide the CPU number in the wakeup AP callback:
    https://lore.kernel.org/linux-hyperv/[email protected]/

Quoted 8 lines

I rebased the patches on top of the latest hyperv-next tree and updated the second patch that broke the linux-next build. That fix that, I made one non-functional change:
updated the signature of numachip_wakeup_secondary() to match the parameter list of wakeup_secondary_cpu().
Roman Kisel (2):
x86/hyperv: Fix APIC ID and VP index confusion in hv_snp_boot_ap() arch/x86: Provide the CPU number in the wakeup AP callback

I queue these up.

Just so you know I'm experimenting a new setup. These have been applied to hyperv-next-staging. It will take some time for them to propagate to hyperv-next.

Thanks,
Wei.

Quoted 20 lines

 arch/x86/coco/sev/core.c             | 13 ++-----
 arch/x86/hyperv/hv_init.c            | 33 +++++++++++++++++
 arch/x86/hyperv/hv_vtl.c             | 54 ++++------------------------
 arch/x86/hyperv/ivm.c                | 11 ++++--
 arch/x86/include/asm/apic.h          |  8 ++---
 arch/x86/include/asm/mshyperv.h      |  7 ++--
 arch/x86/kernel/acpi/madt_wakeup.c   |  2 +-
 arch/x86/kernel/apic/apic_noop.c     |  8 ++++-
 arch/x86/kernel/apic/apic_numachip.c |  2 +-
 arch/x86/kernel/apic/x2apic_uv_x.c   |  2 +-
 arch/x86/kernel/smpboot.c            | 10 +++---
 include/hyperv/hvgdk_mini.h          |  2 +-
 12 files changed, 76 insertions(+), 76 deletions(-)

base-commit: 9b0844d87b1407681b78130429f798beb366f43f

Signature · 1 line

2.43.0

Roman Kisel

author

2025-05-08 19:43 UTC #8 lore raw

Quoted 17 lines · Wei Liu

Quoted 7 lines · Roman Kisel
This patchset combines two patches that depend on each other and were not applying cleanly:
1. Fix APIC ID and VP index confusion in hv_snp_boot_ap():
     https://lore.kernel.org/linux-hyperv/[email protected]/
   2. Provide the CPU number in the wakeup AP callback:
     https://lore.kernel.org/linux-hyperv/[email protected]/
Quoted 8 lines
I rebased the patches on top of the latest hyperv-next tree and updated the second patch that broke the linux-next build. That fix that, I made one non-functional change:
updated the signature of numachip_wakeup_secondary() to match the parameter list of wakeup_secondary_cpu().
Roman Kisel (2):
x86/hyperv: Fix APIC ID and VP index confusion in hv_snp_boot_ap() arch/x86: Provide the CPU number in the wakeup AP callback

Quoted 5 lines

I queue these up.
Just so you know I'm experimenting a new setup. These have been applied to hyperv-next-staging. It will take some time for them to propagate to hyperv-next.

Thank you very much! That's exciting news about the new staging setup!!

Quoted 24 lines

Thanks,
Wei.

Quoted 20 lines

  arch/x86/coco/sev/core.c             | 13 ++-----
  arch/x86/hyperv/hv_init.c            | 33 +++++++++++++++++
  arch/x86/hyperv/hv_vtl.c             | 54 ++++------------------------
  arch/x86/hyperv/ivm.c                | 11 ++++--
  arch/x86/include/asm/apic.h          |  8 ++---
  arch/x86/include/asm/mshyperv.h      |  7 ++--
  arch/x86/kernel/acpi/madt_wakeup.c   |  2 +-
  arch/x86/kernel/apic/apic_noop.c     |  8 ++++-
  arch/x86/kernel/apic/apic_numachip.c |  2 +-
  arch/x86/kernel/apic/x2apic_uv_x.c   |  2 +-
  arch/x86/kernel/smpboot.c            | 10 +++---
  include/hyperv/hvgdk_mini.h          |  2 +-
  12 files changed, 76 insertions(+), 76 deletions(-)

base-commit: 9b0844d87b1407681b78130429f798beb366f43f

Signature · 1 line

2.43.0

Signature · 2 lines

Thank you,
Roman