Skip to contentSkip to conversation

Add support for AMD IOMMU GAPPI

PATCH Superseded Sairaj Kodilkar 4 participants 18 messages Aug 21 – Oct 20

Subsystems (7)
Documentation x86 KVM KVM x86 +3 more subsystems

v4 of 7 Superseded Latest is v7 →

lore
Share & tools
lore
#0 lore raw

v4 cover Add support for AMD IOMMU GAPPI

Introduction
============
On newer generations of AMD processors, IOMMU AVIC/x2AVIC guest-mode interrupt remapping can use Guest APIC Physical Processor Interrupt (GAPPI) as an alternative host-notification path when device interrupts target a vCPU that is not running (IRTE[IsRun] = 0).

With AVIC enabled, the IOMMU posts device interrupts into the guest virtual APIC backing page. When the vCPU is not running, KVM must additionally be notified so it can schedule the vCPU.

The legacy notification path is the GA log (GALOG): the IOMMU appends vCPU tags to a shared GA log buffer and raises a single GA log interrupt. KVM registers a notifier and scans the buffer to decide which vCPUs to wake. Under heavy interrupt load this adds latency and can overflow the buffer because all wakeups funnel through one interrupt and one shared log.

Guest APIC Physical Processor Interrupt (GAPPI), defined in section 2.2.5.4 of the AMD I/O Virtualization Technology (IOMMU) Specification [1], is an alternative. With GAPPI enabled, the IOMMU still updates the guest vAPIC backing page IRR, but may deliver a physical APIC interrupt directly to IRTE[Destination], using IRTE[GATag][7:0] as the vector. This distributes host wakeup notifications across CPUs instead of centralizing them in a log buffer.

This series programs guest-mode IRTEs accordingly: IRTE[Destination] carries the target host physical APIC ID, IRTE[GATag] is set to POSTED_INTR_WAKEUP_VECTOR, and IRTE[GAPPIDis] / IRTE[GALogIntr] are set based on whether KVM requests host wakeup. GAPPI is selected at boot via the amd_iommu=gappi kernel parameter on capable hardware, otherwise the existing GA log path is unchanged.

SVM/AMD IOMMU interface changes
===============================
The first four patches refactor the SVM/AMD IOMMU interface ahead of GAPPI.

The cpu field is renamed to apicid because it carries the host physical APIC ID for IRTE[Destination], not a Linux CPU number.

The ga_log_intr boolean is renamed to wakeup_intr (and the synthetic AVIC_PHYSICAL_ID_ENTRY_GA_LOG_INTR shadow bit to AVIC_PHYSICAL_ID_ENTRY_WAKEUP_INTR). wakeup_intr describes KVM's intent (request host wakeup while the vCPU is not running), not a specific hardware mechanism.

A separate is_running boolean is added to IOMMU interface because GAPPI requires a valid apicid in IRTE[Destination] even when the vCPU is not running. The prior encoding (apicid >= 0 means running, apicid == -1 means not running) no longer works once apicid carries the GAPPI destination while IRTE[IsRun] is clear. The IOMMU driver keys IRTE[IsRun] and destination programming off this explicit boolean instead of inferring running state from apicid.

KVM GAPPI wakeup scheme
=======================
SVM follows the Intel posted-interrupt wakeup model already used by VMX. Each pCPU maintains a list of blocked vCPUs that may be woken by a GAPPI delivery to that CPU. When a vCPU blocks while waiting for a device interrupt, SVM enqueues it on the wakeup list of the pCPU on which it was previously running (gappi_cpu) and passes that pCPU's physical APIC ID to the IOMMU to program IRTE[Destination]. The rationale is that the vCPU is likely to run again on the same pCPU, which is common when vCPUs are pinned; targeting GAPPI notifications there reduces unnecessary VMEXITs from GAPPI deliveries on other CPUs. When the vCPU is scheduled in again, it is removed from the list and IRTE[Destination] is updated to the current pCPU.

List maintenance is moved from VMX layer to common KVM layer so that both VMX and SVM layer can share the same list handling code.

Note
====
kvm_pi_enable_wakeup_handler() and kvm_pi_disable_wakeup_handler() take the target pCPU explicitly instead of using vcpu->cpu, because on SVM the two can differ. avic_vcpu_load() returns early while the vCPU is blocking and leaves the target wakeup CPU untouched, but the generic layer still refreshes vcpu->cpu on every load. Using vcpu->cpu would then unlink the vCPU under the wrong per-pCPU lock, so SVM tracks the pCPU it last programmed in svm->gappi_cpu and passes that instead.

Changes since v3
================
https://lore.kernel.org/all/[email protected]/

Patch 3:
- Make amd_iommu_activate_guest_mode() as wrapper around __amd_iommu_activate_guest_mode() to in order to hardcode is_running flag to "true" for external calls

Patch 5:
- Newly introduced, moves wakeup list handling from VMX to the KVM layer

Patch 6:
- Use common list handling functions introduced in patch 5.
- Remove list addition during pi_update_irte by moving list handling

    functions before list_empty(&svm->ir_list) check.

Changes since v2
================
https://lore.kernel.org/linux-iommu/[email protected]/

Patch[1-6]
- Expand commit messages to explain GAPPI, the interface changes, and the per-CPU wakeup list scheme [Sean].

Patch[1-3]
- Split the monolithic SVM/IOMMU API refactor into four preparatory patches [Sean]
- Rename posted_intr to wakeup_intr to reflect host wakeup intent, not guest interrupt posting [Sean]
- Pass vCPU running status with a extra parameter (is_running) instead of flags.

Patch[4,5]
- Move ga_tag=POSTED_INTR_WAKEUP_VECTOR setting from IOMMU to SVM layer.

Changes since V1:
================
https://lore.kernel.org/all/[email protected]/

Patch4
- Disable interrupts while holding wakeup list lock inside [sashiko] avic_add_vcpu_to_gappi_wakeup_list and avic_remove_vcpu_from_gappi_wakeup_list
- Unregister posted_intr_wakeup_handler during module unload [sashiko]

Patch5
- Disable GAPPI feature during kexec and suspend path [sashiko]

------
[1] https://docs.amd.com/v/u/en-US/48882_3.11_IOMMU_PUB

------
Base: 89516a1c6837156f43525ef49fdbd781145942f5

Sairaj Kodilkar (7):
iommu/amd: KVM: SVM: Rename cpu to apicid in IOMMU interface iommu/amd: KVM: SVM: Rename ga_log_intr to wakeup_intr in IOMMU
interface
iommu/amd: KVM: SVM: Add explicit vCPU running state to IOMMU
interface
iommu/amd: Program guest-mode IRTEs for GAPPI wakeup when IRTE[IsRun]

    = 0

KVM: VMX: Factor out wakeup list handling code to KVM
KVM: SVM: Add support for AMD IOMMU Guest APIC Physical Processor
Interrupt (GAPPI)
iommu/amd: Provide kernel command line option to enable GAPPI

 .../admin-guide/kernel-parameters.txt         |   3 +-
 arch/x86/include/asm/irq_remapping.h          |   5 +-
 arch/x86/include/asm/kvm-x86-ops.h            |   1 +
 arch/x86/include/asm/kvm_host.h               |   1 +
 arch/x86/include/asm/svm.h                    |   9 +-
 arch/x86/kvm/svm/avic.c                       | 109 +++++++++++++----
 arch/x86/kvm/svm/svm.c                        |   3 +
 arch/x86/kvm/svm/svm.h                        |   5 +
 arch/x86/kvm/vmx/common.h                     |   3 -
 arch/x86/kvm/vmx/main.c                       |   6 +
 arch/x86/kvm/vmx/posted_intr.c                |  82 +------------
 arch/x86/kvm/vmx/tdx.c                        |   1 -
 arch/x86/kvm/vmx/vmx.c                        |  14 +--
 arch/x86/kvm/vmx/x86_ops.h                    |   1 +
 arch/x86/kvm/x86.c                            | 111 ++++++++++++++++++
 drivers/iommu/amd/amd_iommu_types.h           |   6 +-
 drivers/iommu/amd/init.c                      |  31 ++++-
 drivers/iommu/amd/iommu.c                     |  65 ++++++----
 include/linux/amd-iommu.h                     |  16 ++-
 include/linux/kvm_host.h                      |   6 +
 20 files changed, 323 insertions(+), 155 deletions(-)
Signature · 1 line
2.34.1
#1 lore raw

PATCH v4 1/7 iommu/amd: KVM: SVM: Rename cpu to apicid in IOMMU interface

The cpu field passed from SVM to the AMD IOMMU driver does not hold a linux CPU number. It carries the host physical APIC ID used to program the IRTE Destination field when the target vCPU is running (IRTE[IsRun] = 1).

Rename the field to apicid everywhere in the IOMMU interface so the name matches the hardware semantics.

No functional change is intended.

---
 arch/x86/include/asm/irq_remapping.h |  2 +-
 arch/x86/kvm/svm/avic.c              | 12 +++++-----
 drivers/iommu/amd/iommu.c            | 35 ++++++++++++++--------------
 include/linux/amd-iommu.h            |  8 +++----
 4 files changed, 29 insertions(+), 28 deletions(-)
arch/x86/include/asm/irq_remapping.h +1 -1
index 37b94f484ef3..263f4df429d8 100644
@@ -35,7 +35,7 @@ struct amd_iommu_pi_data {
 	u64 vapic_addr;		/* Physical address of the vCPU's vAPIC. */
 	u32 ga_tag;
 	u32 vector;		/* Guest vector of the interrupt */
-	int cpu;
+	int apicid;
 	bool ga_log_intr;
 	bool is_guest_mode;
 	void *ir_data;
arch/x86/kvm/svm/avic.c +6 -6
index cdd5a6dc646f..bccc5d7ed207 100644
@@ -949,9 +949,9 @@ int avic_pi_update_irte(struct kvm_kernel_irqfd *irqfd, struct kvm *kvm,
 		 */
 		entry = svm->avic_physical_id_entry;
 		if (entry & AVIC_PHYSICAL_ID_ENTRY_IS_RUNNING_MASK) {
-			pi_data.cpu = entry & AVIC_PHYSICAL_ID_ENTRY_HOST_PHYSICAL_ID_MASK;
+			pi_data.apicid = entry & AVIC_PHYSICAL_ID_ENTRY_HOST_PHYSICAL_ID_MASK;
 		} else {
-			pi_data.cpu = -1;
+			pi_data.apicid = -1;
 			pi_data.ga_log_intr = entry & AVIC_PHYSICAL_ID_ENTRY_GA_LOG_INTR;
 		}
 
@@ -1004,7 +1004,7 @@ enum avic_vcpu_action {
 	AVIC_START_BLOCKING	= BIT(1),
 };
 
-static void avic_update_iommu_vcpu_affinity(struct kvm_vcpu *vcpu, int cpu,
+static void avic_update_iommu_vcpu_affinity(struct kvm_vcpu *vcpu, int apicid,
 					    enum avic_vcpu_action action)
 {
 	bool ga_log_intr = (action & AVIC_START_BLOCKING);
@@ -1024,9 +1024,9 @@ static void avic_update_iommu_vcpu_affinity(struct kvm_vcpu *vcpu, int cpu,
 		void *data = irqfd->irq_bypass_data;
 
 		if (!(action & AVIC_TOGGLE_ON_OFF))
-			WARN_ON_ONCE(amd_iommu_update_ga(data, cpu, ga_log_intr));
-		else if (cpu >= 0)
-			WARN_ON_ONCE(amd_iommu_activate_guest_mode(data, cpu, ga_log_intr));
+			WARN_ON_ONCE(amd_iommu_update_ga(data, apicid, ga_log_intr));
+		else if (apicid >= 0)
+			WARN_ON_ONCE(amd_iommu_activate_guest_mode(data, apicid, ga_log_intr));
 		else
 			WARN_ON_ONCE(amd_iommu_deactivate_guest_mode(data));
 	}
drivers/iommu/amd/iommu.c +18 -17
index 57dc8fabc7d9..c0cf7799c56e 100644
@@ -3959,14 +3959,14 @@ static const struct irq_domain_ops amd_ir_domain_ops = {
 	.deactivate = irq_remapping_deactivate,
 };
 
-static void __amd_iommu_update_ga(struct irte_ga *entry, int cpu,
+static void __amd_iommu_update_ga(struct irte_ga *entry, int apicid,
 				  bool ga_log_intr)
 {
-	if (cpu >= 0) {
+	if (apicid >= 0) {
 		entry->lo.fields_vapic.destination =
-					APICID_TO_IRTE_DEST_LO(cpu);
+					APICID_TO_IRTE_DEST_LO(apicid);
 		entry->hi.fields.destination =
-					APICID_TO_IRTE_DEST_HI(cpu);
+					APICID_TO_IRTE_DEST_HI(apicid);
 		entry->lo.fields_vapic.is_run = true;
 		entry->lo.fields_vapic.ga_log_intr = false;
 	} else {
@@ -3979,20 +3979,21 @@ static void __amd_iommu_update_ga(struct irte_ga *entry, int cpu,
  * Update the pCPU information for an IRTE that is configured to post IRQs to
  * a vCPU, without issuing an IOMMU invalidation for the IRTE.
  *
- * If the vCPU is associated with a pCPU (@cpu >= 0), configure the Destination
- * with the pCPU's APIC ID, set IsRun, and clear GALogIntr.  If the vCPU isn't
- * associated with a pCPU (@cpu < 0), clear IsRun and set/clear GALogIntr based
- * on input from the caller (e.g. KVM only requests GALogIntr when the vCPU is
- * blocking and requires a notification wake event).  I.e. treat vCPUs that are
- * associated with a pCPU as running.  This API is intended to be used when a
- * vCPU is scheduled in/out (or stops running for any reason), to do a fast
- * update of IsRun, GALogIntr, and (conditionally) Destination.
+ * If the vCPU is associated with a pCPU (@apicid >= 0), configure the
+ * Destination with the pCPU's APIC ID, set IsRun, and clear GALogIntr.  If the
+ * vCPU isn't associated with a pCPU (@apicid < 0), clear IsRun and set/clear
+ * GALogIntr based on input from the caller (e.g. KVM only requests GALogIntr
+ * when the vCPU is blocking and requires a notification wake event).  I.e.
+ * treat vCPUs that are associated with a pCPU as running.  This API is
+ * intended to be used when a vCPU is scheduled in/out (or stops running for
+ * any reason), to do a fast update of IsRun, GALogIntr, and (conditionally)
+ * Destination.
  *
  * Per the IOMMU spec, the Destination, IsRun, and GATag fields are not cached
  * and thus don't require an invalidation to ensure the IOMMU consumes fresh
  * information.
  */
-int amd_iommu_update_ga(void *data, int cpu, bool ga_log_intr)
+int amd_iommu_update_ga(void *data, int apicid, bool ga_log_intr)
 {
 	struct amd_ir_data *ir_data = (struct amd_ir_data *)data;
 	struct irte_ga *entry = (struct irte_ga *) ir_data->entry;
@@ -4006,14 +4007,14 @@ int amd_iommu_update_ga(void *data, int cpu, bool ga_log_intr)
 	if (!ir_data->iommu)
 		return -ENODEV;
 
-	__amd_iommu_update_ga(entry, cpu, ga_log_intr);
+	__amd_iommu_update_ga(entry, apicid, ga_log_intr);
 
 	return __modify_irte_ga(ir_data->iommu, ir_data->irq_2_irte.devid,
 				ir_data->irq_2_irte.index, entry);
 }
 EXPORT_SYMBOL(amd_iommu_update_ga);
 
-int amd_iommu_activate_guest_mode(void *data, int cpu, bool ga_log_intr)
+int amd_iommu_activate_guest_mode(void *data, int apicid, bool ga_log_intr)
 {
 	struct amd_ir_data *ir_data = (struct amd_ir_data *)data;
 	struct irte_ga *entry = (struct irte_ga *) ir_data->entry;
@@ -4036,7 +4037,7 @@ int amd_iommu_activate_guest_mode(void *data, int cpu, bool ga_log_intr)
 	entry->hi.fields.vector            = ir_data->ga_vector;
 	entry->lo.fields_vapic.ga_tag      = ir_data->ga_tag;
 
-	__amd_iommu_update_ga(entry, cpu, ga_log_intr);
+	__amd_iommu_update_ga(entry, apicid, ga_log_intr);
 
 	return modify_irte_ga(ir_data->iommu, ir_data->irq_2_irte.devid,
 			      ir_data->irq_2_irte.index, entry);
@@ -4107,7 +4108,7 @@ static int amd_ir_set_vcpu_affinity(struct irq_data *data, void *info)
 		ir_data->ga_vector = pi_data->vector;
 		ir_data->ga_tag = pi_data->ga_tag;
 		if (pi_data->is_guest_mode)
-			ret = amd_iommu_activate_guest_mode(ir_data, pi_data->cpu,
+			ret = amd_iommu_activate_guest_mode(ir_data, pi_data->apicid,
 							    pi_data->ga_log_intr);
 		else
 			ret = amd_iommu_deactivate_guest_mode(ir_data);
include/linux/amd-iommu.h +4 -4
index edcee9f5335a..2c6232aefafa 100644
@@ -30,8 +30,8 @@ static inline void amd_iommu_detect(void) { }
 /* IOMMU AVIC Function */
 extern int amd_iommu_register_ga_log_notifier(int (*notifier)(u32));
 
-extern int amd_iommu_update_ga(void *data, int cpu, bool ga_log_intr);
-extern int amd_iommu_activate_guest_mode(void *data, int cpu, bool ga_log_intr);
+extern int amd_iommu_update_ga(void *data, int apicid, bool ga_log_intr);
+extern int amd_iommu_activate_guest_mode(void *data, int apicid, bool ga_log_intr);
 extern int amd_iommu_deactivate_guest_mode(void *data);
 
 #else /* defined(CONFIG_AMD_IOMMU) && defined(CONFIG_IRQ_REMAP) */
@@ -42,12 +42,12 @@ amd_iommu_register_ga_log_notifier(int (*notifier)(u32))
 	return 0;
 }
 
-static inline int amd_iommu_update_ga(void *data, int cpu, bool ga_log_intr)
+static inline int amd_iommu_update_ga(void *data, int apicid, bool ga_log_intr)
 {
 	return 0;
 }
 
-static inline int amd_iommu_activate_guest_mode(void *data, int cpu, bool ga_log_intr)
+static inline int amd_iommu_activate_guest_mode(void *data, int apicid, bool ga_log_intr)
 {
 	return 0;
 }
Signature · 1 line
2.34.1
maintainer Reviewed-by on v4 1/7
#2 lore raw

Re: [PATCH v4 1/7] iommu/amd: KVM: SVM: Rename cpu to apicid in IOMMU interface

Quoted 11 lines · Sairaj Kodilkar

The cpu field passed from SVM to the AMD IOMMU driver does not hold a linux CPU number. It carries the host physical APIC ID used to program the IRTE Destination field when the target vCPU is running (IRTE[IsRun] = 1).

Rename the field to apicid everywhere in the IOMMU interface so the name matches the hardware semantics.

No functional change is intended.

-Vasant

#3 lore raw

PATCH v4 2/7 iommu/amd: KVM: SVM: Rename ga_log_intr to wakeup_intr in IOMMU interface

In reply to Sairaj Kodilkar

When AVIC is enabled, device interrupts are posted into the guest virtual APIC backing page regardless of vCPU running state. When the vCPU is not running, KVM may additionally need the IOMMU to notify the host so the vCPU can be woken, historically via the GA log mechanism.

The existing ga_log_intr name is tied to one specific hardware mechanism. Guest APIC Physical Processor Interrupt (GAPPI) provides an alternative host-notification path when the vCPU is not running, i.e., IRTE[IsRun] = 0. The KVM-side intent is the same in both cases: request host wakeup notification while the vCPU is blocked.

Rename ga_log_intr to wakeup_intr in the IOMMU API to describe KVM's intent rather than a single hardware knob. The IOMMU driver still maps wakeup_intr to IRTE[GALogIntr] in this patch; GAPPI support is added in following patches.

Similarly rename the synthetic AVIC physical ID table shadow bit from AVIC_PHYSICAL_ID_ENTRY_GA_LOG_INTR to AVIC_PHYSICAL_ID_ENTRY_WAKEUP_INTR.

No functional change is intended.

---
 arch/x86/include/asm/irq_remapping.h |  2 +-
 arch/x86/include/asm/svm.h           |  9 +++++----
 arch/x86/kvm/svm/avic.c              | 26 +++++++++++++-------------
 drivers/iommu/amd/iommu.c            | 16 ++++++++--------
 include/linux/amd-iommu.h            |  8 ++++----
 5 files changed, 31 insertions(+), 30 deletions(-)
arch/x86/include/asm/irq_remapping.h +1 -1
index 263f4df429d8..789e3e154600 100644
@@ -36,7 +36,7 @@ struct amd_iommu_pi_data {
 	u32 ga_tag;
 	u32 vector;		/* Guest vector of the interrupt */
 	int apicid;
-	bool ga_log_intr;
+	bool wakeup_intr;
 	bool is_guest_mode;
 	void *ir_data;
 };
arch/x86/include/asm/svm.h +5 -4
index bcfeb5e7c0ed..add8b7274cc2 100644
@@ -259,11 +259,12 @@ struct __attribute__ ((__packed__)) vmcb_control_area {
 #define AVIC_LOGICAL_ID_ENTRY_VALID_MASK		(1 << 31)
 
 /*
- * GA_LOG_INTR is a synthetic flag that's never propagated to hardware-visible
- * tables.  GA_LOG_INTR is set if the vCPU needs device posted IRQs to generate
- * GA log interrupts to wake the vCPU (because it's blocking or about to block).
+ * WAKEUP_INTR is a synthetic flag that's never propagated to hardware-visible
+ * tables.  WAKEUP_INTR is set if the vCPU needs device posted IRQs to generate
+ * GA log/GAPPI interrupts to wake the vCPU (because it's blocking or about to
+ * block).
  */
-#define AVIC_PHYSICAL_ID_ENTRY_GA_LOG_INTR		BIT_ULL(61)
+#define AVIC_PHYSICAL_ID_ENTRY_WAKEUP_INTR		BIT_ULL(61)
 
 #define AVIC_PHYSICAL_ID_ENTRY_HOST_PHYSICAL_ID_MASK	GENMASK_ULL(11, 0)
 #define AVIC_PHYSICAL_ID_ENTRY_BACKING_PAGE_MASK	GENMASK_ULL(51, 12)
arch/x86/kvm/svm/avic.c +13 -13
index bccc5d7ed207..8e87a7f2f64f 100644
@@ -952,7 +952,7 @@ int avic_pi_update_irte(struct kvm_kernel_irqfd *irqfd, struct kvm *kvm,
 			pi_data.apicid = entry & AVIC_PHYSICAL_ID_ENTRY_HOST_PHYSICAL_ID_MASK;
 		} else {
 			pi_data.apicid = -1;
-			pi_data.ga_log_intr = entry & AVIC_PHYSICAL_ID_ENTRY_GA_LOG_INTR;
+			pi_data.wakeup_intr = entry & AVIC_PHYSICAL_ID_ENTRY_WAKEUP_INTR;
 		}
 
 		ret = irq_set_vcpu_affinity(host_irq, &pi_data);
@@ -991,15 +991,15 @@ enum avic_vcpu_action {
 	 * No unique action is required to deal with a vCPU that stops/starts
 	 * running.  A vCPU that starts running by definition stops blocking as
 	 * well, and a vCPU that stops running can't have been blocking, i.e.
-	 * doesn't need to toggle GALogIntr.
+	 * doesn't need to toggle host wakeup notification.
 	 */
 	AVIC_START_RUNNING	= 0,
 	AVIC_STOP_RUNNING	= 0,
 
 	/*
-	 * When a vCPU starts blocking, KVM needs to set the GALogIntr flag
-	 * int all associated IRTEs so that KVM can wake the vCPU if an IRQ is
-	 * sent to the vCPU.
+	 * When a vCPU starts blocking, KVM needs to request host wakeup
+	 * notification in all associated IRTEs so that KVM can wake the vCPU
+	 * if an IRQ is sent to the vCPU.
 	 */
 	AVIC_START_BLOCKING	= BIT(1),
 };
@@ -1007,7 +1007,7 @@ enum avic_vcpu_action {
 static void avic_update_iommu_vcpu_affinity(struct kvm_vcpu *vcpu, int apicid,
 					    enum avic_vcpu_action action)
 {
-	bool ga_log_intr = (action & AVIC_START_BLOCKING);
+	bool wakeup_intr = (action & AVIC_START_BLOCKING);
 	struct vcpu_svm *svm = to_svm(vcpu);
 	struct kvm_kernel_irqfd *irqfd;
 
@@ -1024,9 +1024,9 @@ static void avic_update_iommu_vcpu_affinity(struct kvm_vcpu *vcpu, int apicid,
 		void *data = irqfd->irq_bypass_data;
 
 		if (!(action & AVIC_TOGGLE_ON_OFF))
-			WARN_ON_ONCE(amd_iommu_update_ga(data, apicid, ga_log_intr));
+			WARN_ON_ONCE(amd_iommu_update_ga(data, apicid, wakeup_intr));
 		else if (apicid >= 0)
-			WARN_ON_ONCE(amd_iommu_activate_guest_mode(data, apicid, ga_log_intr));
+			WARN_ON_ONCE(amd_iommu_activate_guest_mode(data, apicid, wakeup_intr));
 		else
 			WARN_ON_ONCE(amd_iommu_deactivate_guest_mode(data));
 	}
@@ -1063,7 +1063,7 @@ static void __avic_vcpu_load(struct kvm_vcpu *vcpu, int cpu,
 	WARN_ON_ONCE(entry & AVIC_PHYSICAL_ID_ENTRY_IS_RUNNING_MASK);
 
 	entry &= ~(AVIC_PHYSICAL_ID_ENTRY_HOST_PHYSICAL_ID_MASK |
-		   AVIC_PHYSICAL_ID_ENTRY_GA_LOG_INTR);
+		   AVIC_PHYSICAL_ID_ENTRY_WAKEUP_INTR);
 	entry |= (h_physical_id & AVIC_PHYSICAL_ID_ENTRY_HOST_PHYSICAL_ID_MASK);
 	entry |= AVIC_PHYSICAL_ID_ENTRY_IS_RUNNING_MASK;
 
@@ -1125,7 +1125,7 @@ static void __avic_vcpu_put(struct kvm_vcpu *vcpu, enum avic_vcpu_action action)
 
 	avic_update_iommu_vcpu_affinity(vcpu, -1, action);
 
-	WARN_ON_ONCE(entry & AVIC_PHYSICAL_ID_ENTRY_GA_LOG_INTR);
+	WARN_ON_ONCE(entry & AVIC_PHYSICAL_ID_ENTRY_WAKEUP_INTR);
 
 	/*
 	 * Keep the previous APIC ID in the entry so that a rogue doorbell from
@@ -1137,11 +1137,11 @@ static void __avic_vcpu_put(struct kvm_vcpu *vcpu, enum avic_vcpu_action action)
 		WRITE_ONCE(kvm_svm->avic_physical_id_table[vcpu->vcpu_id], entry);
 
 	/*
-	 * Note!  Don't set AVIC_PHYSICAL_ID_ENTRY_GA_LOG_INTR in the table as
+	 * Note!  Don't set AVIC_PHYSICAL_ID_ENTRY_WAKEUP_INTR in the table as
 	 * it's a synthetic flag that usurps an unused should-be-zero bit.
 	 */
 	if (action & AVIC_START_BLOCKING)
-		entry |= AVIC_PHYSICAL_ID_ENTRY_GA_LOG_INTR;
+		entry |= AVIC_PHYSICAL_ID_ENTRY_WAKEUP_INTR;
 
 	svm->avic_physical_id_entry = entry;
 
@@ -1173,7 +1173,7 @@ void avic_vcpu_put(struct kvm_vcpu *vcpu)
 		 * The vCPU was preempted while blocking, ensure its IRTEs are
 		 * configured to generate GA Log Interrupts.
 		 */
-		if (!(WARN_ON_ONCE(!(entry & AVIC_PHYSICAL_ID_ENTRY_GA_LOG_INTR))))
+		if (!(WARN_ON_ONCE(!(entry & AVIC_PHYSICAL_ID_ENTRY_WAKEUP_INTR))))
 			return;
 	}
 
drivers/iommu/amd/iommu.c +8 -8
index c0cf7799c56e..1284f37e44c7 100644
@@ -3960,7 +3960,7 @@ static const struct irq_domain_ops amd_ir_domain_ops = {
 };
 
 static void __amd_iommu_update_ga(struct irte_ga *entry, int apicid,
-				  bool ga_log_intr)
+				  bool wakeup_intr)
 {
 	if (apicid >= 0) {
 		entry->lo.fields_vapic.destination =
@@ -3971,7 +3971,7 @@ static void __amd_iommu_update_ga(struct irte_ga *entry, int apicid,
 		entry->lo.fields_vapic.ga_log_intr = false;
 	} else {
 		entry->lo.fields_vapic.is_run = false;
-		entry->lo.fields_vapic.ga_log_intr = ga_log_intr;
+		entry->lo.fields_vapic.ga_log_intr = wakeup_intr;
 	}
 }
 
@@ -3982,7 +3982,7 @@ static void __amd_iommu_update_ga(struct irte_ga *entry, int apicid,
  * If the vCPU is associated with a pCPU (@apicid >= 0), configure the
  * Destination with the pCPU's APIC ID, set IsRun, and clear GALogIntr.  If the
  * vCPU isn't associated with a pCPU (@apicid < 0), clear IsRun and set/clear
- * GALogIntr based on input from the caller (e.g. KVM only requests GALogIntr
+ * GALogIntr based on input from the caller (e.g. KVM only requests wakeup_intr
  * when the vCPU is blocking and requires a notification wake event).  I.e.
  * treat vCPUs that are associated with a pCPU as running.  This API is
  * intended to be used when a vCPU is scheduled in/out (or stops running for
@@ -3993,7 +3993,7 @@ static void __amd_iommu_update_ga(struct irte_ga *entry, int apicid,
  * and thus don't require an invalidation to ensure the IOMMU consumes fresh
  * information.
  */
-int amd_iommu_update_ga(void *data, int apicid, bool ga_log_intr)
+int amd_iommu_update_ga(void *data, int apicid, bool wakeup_intr)
 {
 	struct amd_ir_data *ir_data = (struct amd_ir_data *)data;
 	struct irte_ga *entry = (struct irte_ga *) ir_data->entry;
@@ -4007,14 +4007,14 @@ int amd_iommu_update_ga(void *data, int apicid, bool ga_log_intr)
 	if (!ir_data->iommu)
 		return -ENODEV;
 
-	__amd_iommu_update_ga(entry, apicid, ga_log_intr);
+	__amd_iommu_update_ga(entry, apicid, wakeup_intr);
 
 	return __modify_irte_ga(ir_data->iommu, ir_data->irq_2_irte.devid,
 				ir_data->irq_2_irte.index, entry);
 }
 EXPORT_SYMBOL(amd_iommu_update_ga);
 
-int amd_iommu_activate_guest_mode(void *data, int apicid, bool ga_log_intr)
+int amd_iommu_activate_guest_mode(void *data, int apicid, bool wakeup_intr)
 {
 	struct amd_ir_data *ir_data = (struct amd_ir_data *)data;
 	struct irte_ga *entry = (struct irte_ga *) ir_data->entry;
@@ -4037,7 +4037,7 @@ int amd_iommu_activate_guest_mode(void *data, int apicid, bool ga_log_intr)
 	entry->hi.fields.vector            = ir_data->ga_vector;
 	entry->lo.fields_vapic.ga_tag      = ir_data->ga_tag;
 
-	__amd_iommu_update_ga(entry, apicid, ga_log_intr);
+	__amd_iommu_update_ga(entry, apicid, wakeup_intr);
 
 	return modify_irte_ga(ir_data->iommu, ir_data->irq_2_irte.devid,
 			      ir_data->irq_2_irte.index, entry);
@@ -4109,7 +4109,7 @@ static int amd_ir_set_vcpu_affinity(struct irq_data *data, void *info)
 		ir_data->ga_tag = pi_data->ga_tag;
 		if (pi_data->is_guest_mode)
 			ret = amd_iommu_activate_guest_mode(ir_data, pi_data->apicid,
-							    pi_data->ga_log_intr);
+							    pi_data->wakeup_intr);
 		else
 			ret = amd_iommu_deactivate_guest_mode(ir_data);
 	} else {
include/linux/amd-iommu.h +4 -4
index 2c6232aefafa..e20c909edc56 100644
@@ -30,8 +30,8 @@ static inline void amd_iommu_detect(void) { }
 /* IOMMU AVIC Function */
 extern int amd_iommu_register_ga_log_notifier(int (*notifier)(u32));
 
-extern int amd_iommu_update_ga(void *data, int apicid, bool ga_log_intr);
-extern int amd_iommu_activate_guest_mode(void *data, int apicid, bool ga_log_intr);
+extern int amd_iommu_update_ga(void *data, int apicid, bool wakeup_intr);
+extern int amd_iommu_activate_guest_mode(void *data, int apicid, bool wakeup_intr);
 extern int amd_iommu_deactivate_guest_mode(void *data);
 
 #else /* defined(CONFIG_AMD_IOMMU) && defined(CONFIG_IRQ_REMAP) */
@@ -42,12 +42,12 @@ amd_iommu_register_ga_log_notifier(int (*notifier)(u32))
 	return 0;
 }
 
-static inline int amd_iommu_update_ga(void *data, int apicid, bool ga_log_intr)
+static inline int amd_iommu_update_ga(void *data, int apicid, bool wakeup_intr)
 {
 	return 0;
 }
 
-static inline int amd_iommu_activate_guest_mode(void *data, int apicid, bool ga_log_intr)
+static inline int amd_iommu_activate_guest_mode(void *data, int apicid, bool wakeup_intr)
 {
 	return 0;
 }
Signature · 1 line
2.34.1
maintainer Reviewed-by on v4 2/7
#4 lore raw

Re: [PATCH v4 2/7] iommu/amd: KVM: SVM: Rename ga_log_intr to wakeup_intr in IOMMU interface

Quoted 16 lines · Sairaj Kodilkar

When AVIC is enabled, device interrupts are posted into the guest virtual APIC backing page regardless of vCPU running state. When the vCPU is not running, KVM may additionally need the IOMMU to notify the host so the vCPU can be woken, historically via the GA log mechanism.

The existing ga_log_intr name is tied to one specific hardware mechanism. Guest APIC Physical Processor Interrupt (GAPPI) provides an alternative host-notification path when the vCPU is not running, i.e., IRTE[IsRun] = 0. The KVM-side intent is the same in both cases: request host wakeup notification while the vCPU is blocked.

Rename ga_log_intr to wakeup_intr in the IOMMU API to describe KVM's intent rather than a single hardware knob. The IOMMU driver still maps wakeup_intr to IRTE[GALogIntr] in this patch; GAPPI support is added in following patches.

Quoted 6 lines

Similarly rename the synthetic AVIC physical ID table shadow bit from AVIC_PHYSICAL_ID_ENTRY_GA_LOG_INTR to AVIC_PHYSICAL_ID_ENTRY_WAKEUP_INTR.

No functional change is intended.

-Vasant

#5 lore raw

PATCH v4 3/7 iommu/amd: KVM: SVM: Add explicit vCPU running state to IOMMU interface

In reply to Sairaj Kodilkar

Guest APIC Physical Processor Interrupt (GAPPI) is an AMD IOMMU mechanism for notifying the host when a device interrupt targets a non-running vCPU (IRTE[IsRun] = 0). Unlike the GA log path, GAPPI delivers a physical APIC interrupt directly to the host CPU described by IRTE[Destination], with the vector carried in IRTE[GATag], while still posting the interrupt into the guest virtual APIC backing page as usual.

In GAPPI mode, the IOMMU delivers the notification to the host CPU using the physical APIC ID in IRTE[Destination] and the vector in IRTE[GATag]. KVM must therefore supply a valid apicid even when the vCPU is not running. The prior interface inferred running state from apicid: apicid >= 0 meant running and apicid == -1 meant not running. That encoding breaks once apicid carries the GAPPI destination while the vCPU is not running; the IOMMU driver can no longer derive vCPU running state from apicid alone and needs an explicit indication from KVM.

Add an explicit is_running indication to the IOMMU interface so that SVM can pass vCPU running state to the IOMMU.

SVM calls amd_iommu_activate_guest_mode() only when the vCPU is running, so keep is_running out of the exported prototype and make it a thin wrapper that hardcodes is_running = true. The IOMMU internal caller, amd_ir_set_vcpu_affinity(), calls __amd_iommu_activate_guest_mode() directly and passes the running state supplied by KVM, i.e. it handles both a running and a non-running vCPU.

No functional change is intended.

---
 arch/x86/include/asm/irq_remapping.h |  1 +
 arch/x86/kvm/svm/avic.c              |  7 ++++--
 drivers/iommu/amd/iommu.c            | 34 ++++++++++++++++------------
 include/linux/amd-iommu.h            |  6 +++--
 4 files changed, 30 insertions(+), 18 deletions(-)
arch/x86/include/asm/irq_remapping.h +1 -0
index 789e3e154600..af0cdfa32c44 100644
@@ -37,6 +37,7 @@ struct amd_iommu_pi_data {
 	u32 vector;		/* Guest vector of the interrupt */
 	int apicid;
 	bool wakeup_intr;
+	bool is_running;
 	bool is_guest_mode;
 	void *ir_data;
 };
arch/x86/kvm/svm/avic.c +5 -2
index 8e87a7f2f64f..dd497530d365 100644
@@ -950,9 +950,11 @@ int avic_pi_update_irte(struct kvm_kernel_irqfd *irqfd, struct kvm *kvm,
 		entry = svm->avic_physical_id_entry;
 		if (entry & AVIC_PHYSICAL_ID_ENTRY_IS_RUNNING_MASK) {
 			pi_data.apicid = entry & AVIC_PHYSICAL_ID_ENTRY_HOST_PHYSICAL_ID_MASK;
+			pi_data.is_running = true;
 		} else {
 			pi_data.apicid = -1;
 			pi_data.wakeup_intr = entry & AVIC_PHYSICAL_ID_ENTRY_WAKEUP_INTR;
+			pi_data.is_running = false;
 		}
 
 		ret = irq_set_vcpu_affinity(host_irq, &pi_data);
@@ -1008,6 +1010,7 @@ static void avic_update_iommu_vcpu_affinity(struct kvm_vcpu *vcpu, int apicid,
 					    enum avic_vcpu_action action)
 {
 	bool wakeup_intr = (action & AVIC_START_BLOCKING);
+	bool is_running = apicid >= 0;
 	struct vcpu_svm *svm = to_svm(vcpu);
 	struct kvm_kernel_irqfd *irqfd;
 
@@ -1024,8 +1027,8 @@ static void avic_update_iommu_vcpu_affinity(struct kvm_vcpu *vcpu, int apicid,
 		void *data = irqfd->irq_bypass_data;
 
 		if (!(action & AVIC_TOGGLE_ON_OFF))
-			WARN_ON_ONCE(amd_iommu_update_ga(data, apicid, wakeup_intr));
-		else if (apicid >= 0)
+			WARN_ON_ONCE(amd_iommu_update_ga(data, apicid, wakeup_intr, is_running));
+		else if (is_running)
 			WARN_ON_ONCE(amd_iommu_activate_guest_mode(data, apicid, wakeup_intr));
 		else
 			WARN_ON_ONCE(amd_iommu_deactivate_guest_mode(data));
drivers/iommu/amd/iommu.c +20 -14
index 1284f37e44c7..a5f89e93ee41 100644
@@ -3960,9 +3960,9 @@ static const struct irq_domain_ops amd_ir_domain_ops = {
 };
 
 static void __amd_iommu_update_ga(struct irte_ga *entry, int apicid,
-				  bool wakeup_intr)
+				  bool wakeup_intr, bool is_running)
 {
-	if (apicid >= 0) {
+	if (is_running) {
 		entry->lo.fields_vapic.destination =
 					APICID_TO_IRTE_DEST_LO(apicid);
 		entry->hi.fields.destination =
@@ -3979,12 +3979,11 @@ static void __amd_iommu_update_ga(struct irte_ga *entry, int apicid,
  * Update the pCPU information for an IRTE that is configured to post IRQs to
  * a vCPU, without issuing an IOMMU invalidation for the IRTE.
  *
- * If the vCPU is associated with a pCPU (@apicid >= 0), configure the
- * Destination with the pCPU's APIC ID, set IsRun, and clear GALogIntr.  If the
- * vCPU isn't associated with a pCPU (@apicid < 0), clear IsRun and set/clear
- * GALogIntr based on input from the caller (e.g. KVM only requests wakeup_intr
- * when the vCPU is blocking and requires a notification wake event).  I.e.
- * treat vCPUs that are associated with a pCPU as running.  This API is
+ * If the vCPU is scheduled to run on pCPU (@is_running = 1), configure the
+ * Destination with the pCPU's APIC ID, set IsRun, and clear GALogIntr. If the
+ * vCPU is scheduled out (@is_running = 0), clear IsRun and set/clear GALogIntr
+ * based on input from the caller (e.g. KVM only requests wakeup_intr when the
+ * vCPU is blocking and requires a notification wake event). This API is
  * intended to be used when a vCPU is scheduled in/out (or stops running for
  * any reason), to do a fast update of IsRun, GALogIntr, and (conditionally)
  * Destination.
@@ -3993,7 +3992,7 @@ static void __amd_iommu_update_ga(struct irte_ga *entry, int apicid,
  * and thus don't require an invalidation to ensure the IOMMU consumes fresh
  * information.
  */
-int amd_iommu_update_ga(void *data, int apicid, bool wakeup_intr)
+int amd_iommu_update_ga(void *data, int apicid, bool wakeup_intr, bool is_running)
 {
 	struct amd_ir_data *ir_data = (struct amd_ir_data *)data;
 	struct irte_ga *entry = (struct irte_ga *) ir_data->entry;
@@ -4007,14 +4006,15 @@ int amd_iommu_update_ga(void *data, int apicid, bool wakeup_intr)
 	if (!ir_data->iommu)
 		return -ENODEV;
 
-	__amd_iommu_update_ga(entry, apicid, wakeup_intr);
+	__amd_iommu_update_ga(entry, apicid, wakeup_intr, is_running);
 
 	return __modify_irte_ga(ir_data->iommu, ir_data->irq_2_irte.devid,
 				ir_data->irq_2_irte.index, entry);
 }
 EXPORT_SYMBOL(amd_iommu_update_ga);
 
-int amd_iommu_activate_guest_mode(void *data, int apicid, bool wakeup_intr)
+static int __amd_iommu_activate_guest_mode(void *data, int apicid, bool wakeup_intr,
+					   bool is_running)
 {
 	struct amd_ir_data *ir_data = (struct amd_ir_data *)data;
 	struct irte_ga *entry = (struct irte_ga *) ir_data->entry;
@@ -4037,11 +4037,16 @@ int amd_iommu_activate_guest_mode(void *data, int apicid, bool wakeup_intr)
 	entry->hi.fields.vector            = ir_data->ga_vector;
 	entry->lo.fields_vapic.ga_tag      = ir_data->ga_tag;
 
-	__amd_iommu_update_ga(entry, apicid, wakeup_intr);
+	__amd_iommu_update_ga(entry, apicid, wakeup_intr, is_running);
 
 	return modify_irte_ga(ir_data->iommu, ir_data->irq_2_irte.devid,
 			      ir_data->irq_2_irte.index, entry);
 }
+
+int amd_iommu_activate_guest_mode(void *data, int apicid, bool wakeup_intr)
+{
+	return __amd_iommu_activate_guest_mode(data, apicid, wakeup_intr, true);
+}
 EXPORT_SYMBOL(amd_iommu_activate_guest_mode);
 
 int amd_iommu_deactivate_guest_mode(void *data)
@@ -4108,8 +4113,9 @@ static int amd_ir_set_vcpu_affinity(struct irq_data *data, void *info)
 		ir_data->ga_vector = pi_data->vector;
 		ir_data->ga_tag = pi_data->ga_tag;
 		if (pi_data->is_guest_mode)
-			ret = amd_iommu_activate_guest_mode(ir_data, pi_data->apicid,
-							    pi_data->wakeup_intr);
+			ret = __amd_iommu_activate_guest_mode(ir_data, pi_data->apicid,
+							      pi_data->wakeup_intr,
+							      pi_data->is_running);
 		else
 			ret = amd_iommu_deactivate_guest_mode(ir_data);
 	} else {
include/linux/amd-iommu.h +4 -2
index e20c909edc56..a5f1bd6db0ee 100644
@@ -30,7 +30,8 @@ static inline void amd_iommu_detect(void) { }
 /* IOMMU AVIC Function */
 extern int amd_iommu_register_ga_log_notifier(int (*notifier)(u32));
 
-extern int amd_iommu_update_ga(void *data, int apicid, bool wakeup_intr);
+extern int amd_iommu_update_ga(void *data, int apicid, bool wakeup_intr,
+			       bool is_running);
 extern int amd_iommu_activate_guest_mode(void *data, int apicid, bool wakeup_intr);
 extern int amd_iommu_deactivate_guest_mode(void *data);
 
@@ -42,7 +43,8 @@ amd_iommu_register_ga_log_notifier(int (*notifier)(u32))
 	return 0;
 }
 
-static inline int amd_iommu_update_ga(void *data, int apicid, bool wakeup_intr)
+static inline int amd_iommu_update_ga(void *data, int apicid, bool wakeup_intr,
+				      bool is_running)
 {
 	return 0;
 }
Signature · 1 line
2.34.1
maintainer Reviewed-by on v4 3/7
#6 lore raw

Re: [PATCH v4 3/7] iommu/amd: KVM: SVM: Add explicit vCPU running state to IOMMU interface

Quoted 26 lines · Sairaj Kodilkar

Guest APIC Physical Processor Interrupt (GAPPI) is an AMD IOMMU mechanism for notifying the host when a device interrupt targets a non-running vCPU (IRTE[IsRun] = 0). Unlike the GA log path, GAPPI delivers a physical APIC interrupt directly to the host CPU described by IRTE[Destination], with the vector carried in IRTE[GATag], while still posting the interrupt into the guest virtual APIC backing page as usual.

In GAPPI mode, the IOMMU delivers the notification to the host CPU using the physical APIC ID in IRTE[Destination] and the vector in IRTE[GATag]. KVM must therefore supply a valid apicid even when the vCPU is not running. The prior interface inferred running state from apicid: apicid >= 0 meant running and apicid == -1 meant not running. That encoding breaks once apicid carries the GAPPI destination while the vCPU is not running; the IOMMU driver can no longer derive vCPU running state from apicid alone and needs an explicit indication from KVM.

Add an explicit is_running indication to the IOMMU interface so that SVM can pass vCPU running state to the IOMMU.

SVM calls amd_iommu_activate_guest_mode() only when the vCPU is running, so keep is_running out of the exported prototype and make it a thin wrapper that hardcodes is_running = true. The IOMMU internal caller, amd_ir_set_vcpu_affinity(), calls __amd_iommu_activate_guest_mode() directly and passes the running state supplied by KVM, i.e. it handles both a running and a non-running vCPU.

Quoted 3 lines

No functional change is intended.

-Vasant

#7 lore raw

PATCH v4 4/7 iommu/amd: Program guest-mode IRTEs for GAPPI wakeup when IRTE[IsRun] = 0

In reply to Sairaj Kodilkar

Guest APIC Physical Processor Interrupt (GAPPI) is an alternative to the GA log for notifying the host when a device interrupt targets a non-running vCPU (IRTE[IsRun] = 0). Per the AMD IOMMU specification [1] (section 2.2.5.4), with GAPPI enabled the IOMMU delivers a physical APIC interrupt to the CPU described by IRTE[Destination], using IRTE[GATag][7:0] as the vector, while still updating the guest vAPIC backing page IRR as usual.

The AMD IOMMU also allows suppressing GAPPI interrupts using the IRTE[GAPPIDis] bit when the GAPPIDisSup feature is available in extended feature register 2. The AMD IOMMU driver sets this bit when the wakeup_intr flag is not set by KVM.

Note: amd_iommu_gappi is currently false; a later patch enables it via the kernel command line.

[1] https://docs.amd.com/v/u/en-US/48882_3.11_IOMMU_PUB

---
 drivers/iommu/amd/amd_iommu_types.h |  4 +++-
 drivers/iommu/amd/init.c            |  3 +++
 drivers/iommu/amd/iommu.c           | 30 +++++++++++++++++++----------
 include/linux/amd-iommu.h           |  6 ++++++
 4 files changed, 32 insertions(+), 11 deletions(-)
drivers/iommu/amd/amd_iommu_types.h +3 -1
index f9f718087893..26d7a9796e64 100644
@@ -113,6 +113,7 @@
 /* Extended Feature 2 Bits */
 #define FEATURE_SEVSNPIO_SUP	BIT_ULL(1)
 #define FEATURE_GCR3TRPMODE	BIT_ULL(3)
+#define FEATURE_GAPPIDISSUP	BIT_ULL(4)
 #define FEATURE_SNPAVICSUP	GENMASK_ULL(7, 5)
 #define FEATURE_SNPAVICSUP_GAM(x) \
 	(FIELD_GET(FEATURE_SNPAVICSUP, x) == 0x1)
@@ -1004,7 +1005,8 @@ union irte_ga_lo {
 		    no_fault	: 1,
 		    /* ------ */
 		    ga_log_intr	: 1,
-		    rsvd1	: 3,
+		    rsvd1	: 2,
+		    gappi_dis	: 1,
 		    is_run	: 1,
 		    /* ------ */
 		    guest_mode	: 1,
drivers/iommu/amd/init.c +3 -0
index 3bdb380d23e9..2e1889f8a9e4 100644
@@ -160,6 +160,9 @@ u8 amd_iommu_hpt_level;
 /* Guest page table level */
 int amd_iommu_gpt_level = PAGE_MODE_4_LEVEL;
 
+bool amd_iommu_gappi;
+EXPORT_SYMBOL(amd_iommu_gappi);
+
 int amd_iommu_guest_ir = AMD_IOMMU_GUEST_IR_VAPIC;
 static int amd_iommu_xt_mode = IRQ_REMAP_XAPIC_MODE;
 
drivers/iommu/amd/iommu.c +20 -10
index a5f89e93ee41..d89e3ce97e57 100644
@@ -3970,8 +3970,18 @@ static void __amd_iommu_update_ga(struct irte_ga *entry, int apicid,
 		entry->lo.fields_vapic.is_run = true;
 		entry->lo.fields_vapic.ga_log_intr = false;
 	} else {
-		entry->lo.fields_vapic.is_run = false;
-		entry->lo.fields_vapic.ga_log_intr = wakeup_intr;
+		if (amd_iommu_gappi) {
+			entry->lo.fields_vapic.gappi_dis = !wakeup_intr &&
+						check_feature2(FEATURE_GAPPIDISSUP);
+			entry->lo.fields_vapic.is_run = false;
+			entry->lo.fields_vapic.destination =
+						APICID_TO_IRTE_DEST_LO(apicid);
+			entry->hi.fields.destination =
+						APICID_TO_IRTE_DEST_HI(apicid);
+		} else {
+			entry->lo.fields_vapic.is_run = false;
+			entry->lo.fields_vapic.ga_log_intr = wakeup_intr;
+		}
 	}
 }
 
@@ -3982,15 +3992,15 @@ static void __amd_iommu_update_ga(struct irte_ga *entry, int apicid,
  * If the vCPU is scheduled to run on pCPU (@is_running = 1), configure the
  * Destination with the pCPU's APIC ID, set IsRun, and clear GALogIntr. If the
  * vCPU is scheduled out (@is_running = 0), clear IsRun and set/clear GALogIntr
- * based on input from the caller (e.g. KVM only requests wakeup_intr when the
- * vCPU is blocking and requires a notification wake event). This API is
- * intended to be used when a vCPU is scheduled in/out (or stops running for
- * any reason), to do a fast update of IsRun, GALogIntr, and (conditionally)
- * Destination.
+ * and GAPPIDis based on input from the caller (e.g. KVM only requests
+ * wakeup_intr when the vCPU is blocking and requires a notification wake
+ * event). This API is intended to be used when a vCPU is scheduled in/out (or
+ * stops running for any reason), to do a fast update of IsRun, GALogIntr,
+ * GAPPIDis and (conditionally) Destination.
  *
- * Per the IOMMU spec, the Destination, IsRun, and GATag fields are not cached
- * and thus don't require an invalidation to ensure the IOMMU consumes fresh
- * information.
+ * Per the IOMMU spec, the Destination, IsRun, GATag and GAPPIDis fields are
+ * not cached and thus don't require an invalidation to ensure the IOMMU
+ * consumes fresh information.
  */
 int amd_iommu_update_ga(void *data, int apicid, bool wakeup_intr, bool is_running)
 {
include/linux/amd-iommu.h +6 -0
index a5f1bd6db0ee..de228140ce6e 100644
@@ -35,6 +35,9 @@ extern int amd_iommu_update_ga(void *data, int apicid, bool wakeup_intr,
 extern int amd_iommu_activate_guest_mode(void *data, int apicid, bool wakeup_intr);
 extern int amd_iommu_deactivate_guest_mode(void *data);
 
+/* IOMMU AVIC Flag */
+extern bool amd_iommu_gappi;
+
 #else /* defined(CONFIG_AMD_IOMMU) && defined(CONFIG_IRQ_REMAP) */
 
 static inline int
@@ -58,6 +61,9 @@ static inline int amd_iommu_deactivate_guest_mode(void *data)
 {
 	return 0;
 }
+
+#define amd_iommu_gappi false
+
 #endif /* defined(CONFIG_AMD_IOMMU) && defined(CONFIG_IRQ_REMAP) */
 
 int amd_iommu_get_num_iommus(void);
Signature · 1 line
2.34.1
maintainer Reviewed-by on v4 4/7
#8 lore raw

Re: [PATCH v4 4/7] iommu/amd: Program guest-mode IRTEs for GAPPI wakeup when IRTE[IsRun] = 0

Quoted 13 lines · Sairaj Kodilkar

Guest APIC Physical Processor Interrupt (GAPPI) is an alternative to the GA log for notifying the host when a device interrupt targets a non-running vCPU (IRTE[IsRun] = 0). Per the AMD IOMMU specification [1] (section 2.2.5.4), with GAPPI enabled the IOMMU delivers a physical APIC interrupt to the CPU described by IRTE[Destination], using IRTE[GATag][7:0] as the vector, while still updating the guest vAPIC backing page IRR as usual.

The AMD IOMMU also allows suppressing GAPPI interrupts using the IRTE[GAPPIDis] bit when the GAPPIDisSup feature is available in extended feature register 2. The AMD IOMMU driver sets this bit when the wakeup_intr flag is not set by KVM.

Quoted 6 lines

Note: amd_iommu_gappi is currently false; a later patch enables it via the kernel command line.

[1] https://docs.amd.com/v/u/en-US/48882_3.11_IOMMU_PUB

-Vasant

#9 lore raw

PATCH 1/1 iommu/amd: Reset IRTE notification bits on every GA update

In reply to Sairaj Kodilkar

From: Jinmeng Zhou <[email protected]>

__amd_iommu_update_ga() clears GALogIntr when the vCPU starts running, but leaves GAPPIDis untouched, so a GAPPIDis value programmed while the vCPU was scheduled out is carried over into the running IRTE. GAPPIDis is recomputed every time the vCPU is put, so the stale value is only ever visible while the vCPU is running. GALogIntr and GAPPIDis are never both in use, as GAPPI vs. GA log is selected when the IOMMU is initialized.

Clear both bits up front and let the not-running path set whichever one the current wakeup policy needs, so that each update fully derives the notification bits from the new state. The only resulting difference is that GAPPIDis is now clear, rather than stale, in IRTEs with IsRun set. Update the kernel-doc accordingly.

---
This is a suggested cleanup on top of v4 4/7; please feel free to squash it into that patch for the next revision.

 drivers/iommu/amd/iommu.c | 22 ++++++++++++++--------
 1 file changed, 14 insertions(+), 8 deletions(-)
drivers/iommu/amd/iommu.c +14 -8
index d89e3ce97e57..3d4d44be1ecb 100644
@@ -3962,13 +3962,19 @@ static const struct irq_domain_ops amd_ir_domain_ops = {
 static void __amd_iommu_update_ga(struct irte_ga *entry, int apicid,
 				  bool wakeup_intr, bool is_running)
 {
+	/*
+	 * Derive the notification bits from the new state on every update so
+	 * that an in-place update doesn't carry over bits from the old state.
+	 */
+	entry->lo.fields_vapic.ga_log_intr = false;
+	entry->lo.fields_vapic.gappi_dis = false;
+
 	if (is_running) {
 		entry->lo.fields_vapic.destination =
 					APICID_TO_IRTE_DEST_LO(apicid);
 		entry->hi.fields.destination =
 					APICID_TO_IRTE_DEST_HI(apicid);
 		entry->lo.fields_vapic.is_run = true;
-		entry->lo.fields_vapic.ga_log_intr = false;
 	} else {
 		if (amd_iommu_gappi) {
 			entry->lo.fields_vapic.gappi_dis = !wakeup_intr &&
@@ -3990,13 +3996,13 @@ static void __amd_iommu_update_ga(struct irte_ga *entry, int apicid,
  * a vCPU, without issuing an IOMMU invalidation for the IRTE.
  *
  * If the vCPU is scheduled to run on pCPU (@is_running = 1), configure the
- * Destination with the pCPU's APIC ID, set IsRun, and clear GALogIntr. If the
- * vCPU is scheduled out (@is_running = 0), clear IsRun and set/clear GALogIntr
- * and GAPPIDis based on input from the caller (e.g. KVM only requests
- * wakeup_intr when the vCPU is blocking and requires a notification wake
- * event). This API is intended to be used when a vCPU is scheduled in/out (or
- * stops running for any reason), to do a fast update of IsRun, GALogIntr,
- * GAPPIDis and (conditionally) Destination.
+ * Destination with the pCPU's APIC ID, set IsRun, and clear GALogIntr and
+ * GAPPIDis. If the vCPU is scheduled out (@is_running = 0), clear IsRun and
+ * set/clear GALogIntr and GAPPIDis based on input from the caller (e.g. KVM
+ * only requests wakeup_intr when the vCPU is blocking and requires a
+ * notification wake event). This API is intended to be used when a vCPU is
+ * scheduled in/out (or stops running for any reason), to do a fast update of
+ * IsRun, GALogIntr, GAPPIDis and (conditionally) Destination.
  *
  * Per the IOMMU spec, the Destination, IsRun, GATag and GAPPIDis fields are
  * not cached and thus don't require an invalidation to ensure the IOMMU
Signature · 1 line
2.39.5
#10 lore raw

PATCH v4 5/7 KVM: VMX: Factor out wakeup list handling code to KVM

In reply to Sairaj Kodilkar

VMX maintains a per-pCPU list of blocked vCPUs so that they can be woken when a device interrupt arrives while the vCPU is not running. The vCPU is queued on the list of the pCPU it last ran on and its notification vector is switched to POSTED_INTR_WAKEUP_VECTOR, so the handler for that vector walks the local list and wakes the vCPUs that have a notification pending.

Guest APIC Physical Processor Interrupt (GAPPI) needs the same infrastructure on AMD. When a device interrupt targets a non-running vCPU (IRTE[IsRun] = 0), the IOMMU delivers a physical APIC interrupt to IRTE[Destination] using IRTE[GATag][7:0] as the vector, which KVM programs to POSTED_INTR_WAKEUP_VECTOR. SVM therefore requires the same per-pCPU list of blocked vCPUs that VMX already has. Hence Move the per-pCPU list, its lock, the wakeup handler and the per-CPU initialization from VMX to common KVM x86 code.

Intel and AMD determine whether a queued vCPU has a notification pending in different ways. Hence, introduce a new kvm_x86_ops callback vcpu_irq_pending(), which the wakeup handler uses to decide whether a vCPU needs to be woken.

No functional change is intended.

---
 arch/x86/include/asm/kvm-x86-ops.h |   1 +
 arch/x86/include/asm/kvm_host.h    |   1 +
 arch/x86/kvm/vmx/common.h          |   3 -
 arch/x86/kvm/vmx/main.c            |   6 ++
 arch/x86/kvm/vmx/posted_intr.c     |  82 +--------------------
 arch/x86/kvm/vmx/tdx.c             |   1 -
 arch/x86/kvm/vmx/vmx.c             |  14 ++--
 arch/x86/kvm/vmx/x86_ops.h         |   1 +
 arch/x86/kvm/x86.c                 | 111 +++++++++++++++++++++++++++++
 include/linux/kvm_host.h           |   6 ++
 10 files changed, 135 insertions(+), 91 deletions(-)
arch/x86/include/asm/kvm-x86-ops.h +1 -0
index 3776cf5382a2..9adc88cc9724 100644
@@ -30,6 +30,7 @@ KVM_X86_OP(vcpu_reset)
 KVM_X86_OP(prepare_switch_to_guest)
 KVM_X86_OP(vcpu_load)
 KVM_X86_OP(vcpu_put)
+KVM_X86_OP_OPTIONAL(vcpu_irq_pending)
 KVM_X86_OP(update_exception_bitmap)
 KVM_X86_OP(get_msr)
 KVM_X86_OP(set_msr)
arch/x86/include/asm/kvm_host.h +1 -0
index f14009f25a3b..1ac547b77696 100644
@@ -1787,6 +1787,7 @@ struct kvm_x86_ops {
 	void (*prepare_switch_to_guest)(struct kvm_vcpu *vcpu);
 	void (*vcpu_load)(struct kvm_vcpu *vcpu, int cpu);
 	void (*vcpu_put)(struct kvm_vcpu *vcpu);
+	bool (*vcpu_irq_pending)(struct kvm_vcpu *vcpu);
 
 	/*
 	 * Mask of DEBUGCTL bits that are owned by the host, i.e. that need to
arch/x86/kvm/vmx/common.h +0 -3
index 412d0829d7a2..d6a654e05fcf 100644
@@ -34,9 +34,6 @@ struct vcpu_vt {
 	/* Posted interrupt descriptor */
 	struct pi_desc pi_desc;
 
-	/* Used if this vCPU is waiting for PI notification wakeup. */
-	struct list_head pi_wakeup_list;
-
 	union vmx_exit_reason exit_reason;
 
 	unsigned long	exit_qualification;
arch/x86/kvm/vmx/main.c +6 -0
index dbebddf648be..8a037febb60e 100644
@@ -108,6 +108,11 @@ static void vt_vcpu_load(struct kvm_vcpu *vcpu, int cpu)
 	vmx_vcpu_load(vcpu, cpu);
 }
 
+static bool vt_vcpu_irq_pending(struct kvm_vcpu *vcpu)
+{
+	return vmx_vcpu_irq_pending(vcpu);
+}
+
 static void vt_update_cpu_dirty_logging(struct kvm_vcpu *vcpu)
 {
 	/*
@@ -896,6 +901,7 @@ struct kvm_x86_ops vt_x86_ops __initdata = {
 	.prepare_switch_to_guest = vt_op(prepare_switch_to_guest),
 	.vcpu_load = vt_op(vcpu_load),
 	.vcpu_put = vt_op(vcpu_put),
+	.vcpu_irq_pending = vt_op(vcpu_irq_pending),
 
 	.HOST_OWNED_DEBUGCTL = VMX_HOST_OWNED_DEBUGCTL_BITS,
 
arch/x86/kvm/vmx/posted_intr.c +2 -80

114 lines not on this page · raw message

arch/x86/kvm/vmx/tdx.c +0 -1
index 04ce321ebdf3..b8fc0419efba 100644
@@ -687,7 +687,6 @@ int tdx_vcpu_create(struct kvm_vcpu *vcpu)
 
 	fpstate_set_confidential(&vcpu->arch.guest_fpu);
 	vcpu->arch.apic->guest_apic_protected = true;
-	INIT_LIST_HEAD(&tdx->vt.pi_wakeup_list);
 
 	vcpu->arch.efer = EFER_SCE | EFER_LME | EFER_LMA | EFER_NX;
 
arch/x86/kvm/vmx/vmx.c +7 -7
index b9103de01428..9bf088877815 100644
@@ -1567,6 +1567,13 @@ void vmx_vcpu_put(struct kvm_vcpu *vcpu)
 	vmx_prepare_switch_to_host(to_vmx(vcpu));
 }
 
+bool vmx_vcpu_irq_pending(struct kvm_vcpu *vcpu)
+{
+	struct vcpu_vt *vt = to_vt(vcpu);
+
+	return pi_test_on(&vt->pi_desc);
+}
+
 static void vmx_switch_loaded_vmcs(struct kvm_vcpu *vcpu,
 				   struct loaded_vmcs *vmcs)
 {
@@ -7686,8 +7693,6 @@ int vmx_vcpu_create(struct kvm_vcpu *vcpu)
 	BUILD_BUG_ON(offsetof(struct vcpu_vmx, vcpu) != 0);
 	vmx = to_vmx(vcpu);
 
-	INIT_LIST_HEAD(&vmx->vt.pi_wakeup_list);
-
 	err = -ENOMEM;
 
 	vmx->vpid = allocate_vpid();
@@ -8475,8 +8480,6 @@ void vmx_migrate_timers(struct kvm_vcpu *vcpu)
 
 void vmx_hardware_unsetup(void)
 {
-	kvm_set_posted_intr_wakeup_handler(NULL);
-
 	if (nested)
 		nested_vmx_hardware_unsetup();
 }
@@ -8780,7 +8783,6 @@ __init int vmx_hardware_setup(void)
 			return r;
 	}
 
-	kvm_set_posted_intr_wakeup_handler(pi_wakeup_handler);
 
 	/*
 	 * On Intel CPUs that lack self-snoop feature, letting the guest control
@@ -8850,8 +8852,6 @@ int __init vmx_init(void)
 
 	for_each_possible_cpu(cpu) {
 		INIT_LIST_HEAD(&per_cpu(loaded_vmcss_on_cpu, cpu));
-
-		pi_init_cpu(cpu);
 	}
 
 	vmx_check_vmcs12_offsets();
arch/x86/kvm/vmx/x86_ops.h +1 -0
index d09abeac2b56..1c5cc50f9c60 100644
@@ -26,6 +26,7 @@ void vmx_vcpu_free(struct kvm_vcpu *vcpu);
 void vmx_vcpu_reset(struct kvm_vcpu *vcpu, bool init_event);
 void vmx_vcpu_load(struct kvm_vcpu *vcpu, int cpu);
 void vmx_vcpu_put(struct kvm_vcpu *vcpu);
+bool vmx_vcpu_irq_pending(struct kvm_vcpu *vcpu);
 int vmx_handle_exit(struct kvm_vcpu *vcpu, fastpath_t exit_fastpath);
 void vmx_handle_exit_irqoff(struct kvm_vcpu *vcpu);
 int vmx_skip_emulated_instruction(struct kvm_vcpu *vcpu);
arch/x86/kvm/x86.c +111 -0

151 lines not on this page · raw message

include/linux/kvm_host.h +6 -0
index 4c14aee1fb06..e5105cbe5230 100644
@@ -398,6 +398,9 @@ struct kvm_vcpu {
 	 */
 	struct kvm_memory_slot *last_used_slot;
 	u64 last_used_slot_gen;
+
+	/* Used if this vCPU is waiting for PI notification wakeup. */
+	struct list_head pi_wakeup_list;
 };
 
 /*
@@ -2609,4 +2612,7 @@ long kvm_arch_vcpu_pre_fault_memory(struct kvm_vcpu *vcpu,
 				    struct kvm_pre_fault_memory *range);
 #endif
 
+void kvm_pi_enable_wakeup_handler(struct kvm_vcpu *vcpu, int cpu);
+void kvm_pi_disable_wakeup_handler(struct kvm_vcpu *vcpu, int cpu);
+
 #endif
Signature · 1 line
2.34.1
#11 lore raw

PATCH v4 6/7 KVM: SVM: Add support for AMD IOMMU Guest APIC Physical Processor Interrupt (GAPPI)

In reply to Sairaj Kodilkar

With AVIC guest-mode interrupt remapping, device interrupts are posted into the guest vAPIC backing page by the IOMMU. When the vCPU is not running (IRTE[IsRun] = 0), KVM must still be notified to schedule it. The legacy path uses the GA log.

GAPPI (Guest APIC Physical Processor Interrupt) is an alternative to the GA log mechanism provided by the AMD IOMMU. With GAPPI enabled, the IOMMU still updates the vAPIC backing page IRR, but the host wakeup notification is delivered as a physical APIC interrupt to IRTE[Destination], using IRTE[GATag][7:0] as the vector (POSTED_INTR_WAKEUP_VECTOR).

SVM follows the Intel posted-interrupt wakeup model. Each pCPU maintains a list of blocked vCPUs that may be woken by a GAPPI delivery to that CPU. When a vCPU blocks while waiting for an interrupt, SVM enqueues it on the wakeup list of the pCPU on which it was previously running and passes that same pCPU's physical APIC ID to the IOMMU to program IRTE[Destination]. The rationale is that the vCPU is likely to run again on the same pCPU, which is common when vCPUs are pinned; targeting GAPPI notifications there reduces unnecessary VMEXITs from GAPPI deliveries on other CPUs. SVM registers the GAPPI handler via kvm_set_posted_intr_wakeup_handler(). On delivery, it walks the local vCPU list and wakes vCPUs with a pending IRR.

All GAPPI logic is gated on amd_iommu_gappi. Without it, KVM and the IOMMU falls back to the legacy GA log mechanism for vCPU wakeup.

---
 arch/x86/kvm/svm/avic.c | 72 ++++++++++++++++++++++++++++++++++++-----
 arch/x86/kvm/svm/svm.c  |  3 ++
 arch/x86/kvm/svm/svm.h  |  5 +++
 3 files changed, 72 insertions(+), 8 deletions(-)
arch/x86/kvm/svm/avic.c +64 -8
index dd497530d365..18ac24ef40e1 100644
@@ -874,6 +874,8 @@ int avic_init_vcpu(struct vcpu_svm *svm)
 	INIT_LIST_HEAD(&svm->ir_list);
 	raw_spin_lock_init(&svm->ir_list_lock);
 
+	svm->gappi_cpu = -1;
+
 	if (!enable_apicv || !irqchip_in_kernel(vcpu->kvm))
 		return 0;
 
@@ -886,6 +888,20 @@ int avic_init_vcpu(struct vcpu_svm *svm)
 	return ret;
 }
 
+void avic_destroy_vcpu(struct vcpu_svm *svm)
+{
+	if (amd_iommu_gappi && svm->gappi_cpu != -1) {
+		unsigned long flags;
+
+		local_irq_save(flags);
+
+		kvm_pi_disable_wakeup_handler(&svm->vcpu, svm->gappi_cpu);
+		svm->gappi_cpu = -1;
+
+		local_irq_restore(flags);
+	}
+}
+
 void avic_apicv_post_state_restore(struct kvm_vcpu *vcpu)
 {
 	avic_handle_dfr_update(vcpu);
@@ -923,8 +939,6 @@ int avic_pi_update_irte(struct kvm_kernel_irqfd *irqfd, struct kvm *kvm,
 		 * if AVIC is enabled/uninhibited in the future.
 		 */
 		struct amd_iommu_pi_data pi_data = {
-			.ga_tag = AVIC_GATAG(to_kvm_svm(kvm)->avic_vm_id,
-					     vcpu->vcpu_idx),
 			.is_guest_mode = kvm_vcpu_apicv_active(vcpu),
 			.vapic_addr = avic_get_backing_page_address(to_svm(vcpu)),
 			.vector = vector,
@@ -947,6 +961,12 @@ int avic_pi_update_irte(struct kvm_kernel_irqfd *irqfd, struct kvm *kvm,
 		 * scheduled out, KVM will update the pCPU info when the vCPU
 		 * is awakened and/or scheduled in.  See also avic_vcpu_load().
 		 */
+		if (amd_iommu_gappi)
+			pi_data.ga_tag = POSTED_INTR_WAKEUP_VECTOR;
+		else
+			pi_data.ga_tag = AVIC_GATAG(to_kvm_svm(kvm)->avic_vm_id,
+						    vcpu->vcpu_idx);
+
 		entry = svm->avic_physical_id_entry;
 		if (entry & AVIC_PHYSICAL_ID_ENTRY_IS_RUNNING_MASK) {
 			pi_data.apicid = entry & AVIC_PHYSICAL_ID_ENTRY_HOST_PHYSICAL_ID_MASK;
@@ -955,6 +975,19 @@ int avic_pi_update_irte(struct kvm_kernel_irqfd *irqfd, struct kvm *kvm,
 			pi_data.apicid = -1;
 			pi_data.wakeup_intr = entry & AVIC_PHYSICAL_ID_ENTRY_WAKEUP_INTR;
 			pi_data.is_running = false;
+
+			if (amd_iommu_gappi) {
+				if (svm->gappi_cpu != -1)
+					pi_data.apicid = kvm_cpu_get_apicid(svm->gappi_cpu);
+				else
+					/*
+					 * If user calls KVM_IRQFD before vCPU
+					 * run, use CPU 0. We don't need to add
+					 * vCPU to the wakeup list as it is
+					 * never been loaded.
+					 */
+					pi_data.apicid = kvm_cpu_get_apicid(0);
+			}
 		}
 
 		ret = irq_set_vcpu_affinity(host_irq, &pi_data);
@@ -1007,7 +1040,7 @@ enum avic_vcpu_action {
 };
 
 static void avic_update_iommu_vcpu_affinity(struct kvm_vcpu *vcpu, int apicid,
-					    enum avic_vcpu_action action)
+					    int cpu, enum avic_vcpu_action action)
 {
 	bool wakeup_intr = (action & AVIC_START_BLOCKING);
 	bool is_running = apicid >= 0;
@@ -1016,6 +1049,20 @@ static void avic_update_iommu_vcpu_affinity(struct kvm_vcpu *vcpu, int apicid,
 
 	lockdep_assert_held(&svm->ir_list_lock);
 
+	if (amd_iommu_gappi && is_running) {
+		if (svm->gappi_cpu != -1)
+			/*
+			 * Handle initial state when vCPU is loaded for the
+			 * first time without any IRQ affinity.
+			 */
+			kvm_pi_disable_wakeup_handler(vcpu, svm->gappi_cpu);
+
+		svm->gappi_cpu = cpu; /* Store cpu number as target for GAPPI */
+	} else if (amd_iommu_gappi) {
+		apicid = kvm_cpu_get_apicid(svm->gappi_cpu);
+		kvm_pi_enable_wakeup_handler(vcpu, svm->gappi_cpu);
+	}
+
 	/*
 	 * Here, we go through the per-vcpu ir_list to update all existing
 	 * interrupt remapping table entry targeting this vcpu.
@@ -1083,7 +1130,7 @@ static void __avic_vcpu_load(struct kvm_vcpu *vcpu, int cpu,
 
 	WRITE_ONCE(kvm_svm->avic_physical_id_table[vcpu->vcpu_id], entry);
 
-	avic_update_iommu_vcpu_affinity(vcpu, h_physical_id, action);
+	avic_update_iommu_vcpu_affinity(vcpu, h_physical_id, cpu, action);
 
 	raw_spin_unlock_irqrestore(&svm->ir_list_lock, flags);
 }
@@ -1126,7 +1173,7 @@ static void __avic_vcpu_put(struct kvm_vcpu *vcpu, enum avic_vcpu_action action)
 	 */
 	raw_spin_lock_irqsave(&svm->ir_list_lock, flags);
 
-	avic_update_iommu_vcpu_affinity(vcpu, -1, action);
+	avic_update_iommu_vcpu_affinity(vcpu, -1, -1, action);
 
 	WARN_ON_ONCE(entry & AVIC_PHYSICAL_ID_ENTRY_WAKEUP_INTR);
 
@@ -1174,7 +1221,7 @@ void avic_vcpu_put(struct kvm_vcpu *vcpu)
 
 		/*
 		 * The vCPU was preempted while blocking, ensure its IRTEs are
-		 * configured to generate GA Log Interrupts.
+		 * configured to request host wakeup notification.
 		 */
 		if (!(WARN_ON_ONCE(!(entry & AVIC_PHYSICAL_ID_ENTRY_WAKEUP_INTR))))
 			return;
@@ -1299,6 +1346,13 @@ static bool __init avic_want_avic_enabled(void)
 	return true;
 }
 
+bool avic_vcpu_irq_pending(struct kvm_vcpu *vcpu)
+{
+	struct vcpu_svm *svm = to_svm(vcpu);
+
+	return kvm_lapic_find_highest_irr(&svm->vcpu) >= 0;
+}
+
 /*
  * Note:
  * - The module param avic enable both xAPIC and x2APIC mode.
@@ -1342,6 +1396,8 @@ bool __init avic_hardware_setup(void)
 
 void avic_hardware_unsetup(void)
 {
-	if (avic)
-		amd_iommu_register_ga_log_notifier(NULL);
+	if (!avic)
+		return;
+
+	amd_iommu_register_ga_log_notifier(NULL);
 }
arch/x86/kvm/svm/svm.c +3 -0
index e02a38da5296..0883cbf24b6d 100644
@@ -1356,6 +1356,8 @@ static void svm_vcpu_free(struct kvm_vcpu *vcpu)
 
 	WARN_ON_ONCE(!list_empty(&svm->ir_list));
 
+	avic_destroy_vcpu(svm);
+
 	svm_leave_nested(vcpu);
 	svm_free_nested(svm);
 
@@ -5292,6 +5294,7 @@ struct kvm_x86_ops svm_x86_ops __initdata = {
 	.vcpu_put = svm_vcpu_put,
 	.vcpu_blocking = avic_vcpu_blocking,
 	.vcpu_unblocking = avic_vcpu_unblocking,
+	.vcpu_irq_pending = avic_vcpu_irq_pending,
 
 	.update_exception_bitmap = svm_update_exception_bitmap,
 	.get_feature_msr = svm_get_feature_msr,
arch/x86/kvm/svm/svm.h +5 -0
index 5137416be593..d74777f287c8 100644
@@ -362,6 +362,9 @@ struct vcpu_svm {
 
 	/* Guest GIF value, used when vGIF is not enabled */
 	bool guest_gif;
+
+	/* GAPPI related fields */
+	int gappi_cpu;
 };
 
 struct svm_cpu_data {
@@ -909,8 +912,10 @@ void avic_init_vmcb(struct vcpu_svm *svm, struct vmcb *vmcb);
 int avic_incomplete_ipi_interception(struct kvm_vcpu *vcpu);
 int avic_unaccelerated_access_interception(struct kvm_vcpu *vcpu);
 int avic_init_vcpu(struct vcpu_svm *svm);
+void avic_destroy_vcpu(struct vcpu_svm *svm);
 void avic_vcpu_load(struct kvm_vcpu *vcpu, int cpu);
 void avic_vcpu_put(struct kvm_vcpu *vcpu);
+bool avic_vcpu_irq_pending(struct kvm_vcpu *vcpu);
 void avic_apicv_post_state_restore(struct kvm_vcpu *vcpu);
 void avic_refresh_apicv_exec_ctrl(struct kvm_vcpu *vcpu);
 int avic_pi_update_irte(struct kvm_kernel_irqfd *irqfd, struct kvm *kvm,
Signature · 1 line
2.34.1
maintainer
#12 lore raw

Re: [PATCH v4 6/7] KVM: SVM: Add support for AMD IOMMU Guest APIC Physical Processor Interrupt (GAPPI)

Quoted 37 lines · Sairaj Kodilkar

With AVIC guest-mode interrupt remapping, device interrupts are posted into the guest vAPIC backing page by the IOMMU. When the vCPU is not running (IRTE[IsRun] = 0), KVM must still be notified to schedule it. The legacy path uses the GA log.

GAPPI (Guest APIC Physical Processor Interrupt) is an alternative to the GA log mechanism provided by the AMD IOMMU. With GAPPI enabled, the IOMMU still updates the vAPIC backing page IRR, but the host wakeup notification is delivered as a physical APIC interrupt to IRTE[Destination], using IRTE[GATag][7:0] as the vector (POSTED_INTR_WAKEUP_VECTOR).

SVM follows the Intel posted-interrupt wakeup model. Each pCPU maintains a list of blocked vCPUs that may be woken by a GAPPI delivery to that CPU. When a vCPU blocks while waiting for an interrupt, SVM enqueues it on the wakeup list of the pCPU on which it was previously running and passes that same pCPU's physical APIC ID to the IOMMU to program IRTE[Destination]. The rationale is that the vCPU is likely to run again on the same pCPU, which is common when vCPUs are pinned; targeting GAPPI notifications there reduces unnecessary VMEXITs from GAPPI deliveries on other CPUs. SVM registers the GAPPI handler via kvm_set_posted_intr_wakeup_handler(). On delivery, it walks the local vCPU list and wakes vCPUs with a pending IRR.

All GAPPI logic is gated on amd_iommu_gappi. Without it, KVM and the IOMMU falls back to the legacy GA log mechanism for vCPU wakeup.

---
 arch/x86/kvm/svm/avic.c | 72 ++++++++++++++++++++++++++++++++++++-----
 arch/x86/kvm/svm/svm.c  |  3 ++
 arch/x86/kvm/svm/svm.h  |  5 +++
 3 files changed, 72 insertions(+), 8 deletions(-)
arch/x86/kvm/svm/avic.c +0 -0
index dd497530d365..18ac24ef40e1 100644

arch/x86/kvm/svm/avic.c int avic_init_vcpu(struct vcpu_svm *svm) · line 877

Quoted 4 lines
arch/x86/kvm/svm/avic.c +1 -0
@@ -874,6 +874,8 @@ int avic_init_vcpu(struct vcpu_svm *svm)
 	INIT_LIST_HEAD(&svm->ir_list);
 	raw_spin_lock_init(&svm->ir_list_lock);
 
+	svm->gappi_cpu = -1;

Can we initialize it to CPU0?

arch/x86/kvm/svm/avic.c int avic_init_vcpu(struct vcpu_svm *svm) · line 893

Quoted 11 lines
arch/x86/kvm/svm/avic.c +4 -0
+
 	if (!enable_apicv || !irqchip_in_kernel(vcpu->kvm))
 		return 0;
 
@@ -886,6 +888,20 @@ int avic_init_vcpu(struct vcpu_svm *svm)
 	return ret;
 }
 
+void avic_destroy_vcpu(struct vcpu_svm *svm)
+{
+	if (amd_iommu_gappi && svm->gappi_cpu != -1) {

Redundant amd_iommu_gappi check?

arch/x86/kvm/svm/avic.c int avic_init_vcpu(struct vcpu_svm *svm) · line 899

Quoted 6 lines
arch/x86/kvm/svm/avic.c +6 -0
+		unsigned long flags;
+
+		local_irq_save(flags);
+
+		kvm_pi_disable_wakeup_handler(&svm->vcpu, svm->gappi_cpu);
+		svm->gappi_cpu = -1;

Redundant assignement?

-Vasant

#13 lore raw

Re: [PATCH v4 6/7] KVM: SVM: Add support for AMD IOMMU Guest APIC Physical Processor Interrupt (GAPPI)

Quoted 43 lines · Vasant Hegde
Quoted 40 lines · Sairaj Kodilkar

With AVIC guest-mode interrupt remapping, device interrupts are posted into the guest vAPIC backing page by the IOMMU. When the vCPU is not running (IRTE[IsRun] = 0), KVM must still be notified to schedule it. The legacy path uses the GA log.

GAPPI (Guest APIC Physical Processor Interrupt) is an alternative to the GA log mechanism provided by the AMD IOMMU. With GAPPI enabled, the IOMMU still updates the vAPIC backing page IRR, but the host wakeup notification is delivered as a physical APIC interrupt to IRTE[Destination], using IRTE[GATag][7:0] as the vector (POSTED_INTR_WAKEUP_VECTOR).

SVM follows the Intel posted-interrupt wakeup model. Each pCPU maintains a list of blocked vCPUs that may be woken by a GAPPI delivery to that CPU. When a vCPU blocks while waiting for an interrupt, SVM enqueues it on the wakeup list of the pCPU on which it was previously running and passes that same pCPU's physical APIC ID to the IOMMU to program IRTE[Destination]. The rationale is that the vCPU is likely to run again on the same pCPU, which is common when vCPUs are pinned; targeting GAPPI notifications there reduces unnecessary VMEXITs from GAPPI deliveries on other CPUs. SVM registers the GAPPI handler via kvm_set_posted_intr_wakeup_handler(). On delivery, it walks the local vCPU list and wakes vCPUs with a pending IRR.

All GAPPI logic is gated on amd_iommu_gappi. Without it, KVM and the IOMMU falls back to the legacy GA log mechanism for vCPU wakeup.

---
 arch/x86/kvm/svm/avic.c | 72 ++++++++++++++++++++++++++++++++++++-----
 arch/x86/kvm/svm/svm.c  |  3 ++
 arch/x86/kvm/svm/svm.h  |  5 +++
 3 files changed, 72 insertions(+), 8 deletions(-)
arch/x86/kvm/svm/avic.c +0 -0
index dd497530d365..18ac24ef40e1 100644
@@ -874,6 +874,8 @@ int avic_init_vcpu(struct vcpu_svm *svm)
 	INIT_LIST_HEAD(&svm->ir_list);
 	raw_spin_lock_init(&svm->ir_list_lock);
 
Quoted 3 lines

arch/x86/kvm/svm/avic.c int avic_init_vcpu(struct vcpu_svm *svm) · line 877

Quoted 1 line
arch/x86/kvm/svm/avic.c +1 -0
+	svm->gappi_cpu = -1;

Can we initialize it to CPU0?

Yeah, I think we can do this, I was worried about passing gappi_cpu=0 to kvm_pi_disable_wakeup_handler() when vCPU is not in list but I think it should be safe because of list_empty() check.

Quoted 8 lines
Quoted 8 lines
arch/x86/kvm/svm/avic.c +1 -0
+
 	if (!enable_apicv || !irqchip_in_kernel(vcpu->kvm))
 		return 0;
 
@@ -886,6 +888,20 @@ int avic_init_vcpu(struct vcpu_svm *svm)
 	return ret;
 }
 
Quoted 5 lines

arch/x86/kvm/svm/avic.c int avic_init_vcpu(struct vcpu_svm *svm) · line 893

Quoted 3 lines
arch/x86/kvm/svm/avic.c +3 -0
+void avic_destroy_vcpu(struct vcpu_svm *svm)
+{
+	if (amd_iommu_gappi && svm->gappi_cpu != -1) {

Redundant amd_iommu_gappi check?

I am not sure if we should call kvm_pi_disable_wakeup_handler() outside amd_iommu_gappi check. Although it is safe to do so, I want to have this call inside amd_iommu_gappi check.

Thanks
Sairaj.

Quoted 12 lines

arch/x86/kvm/svm/avic.c int avic_init_vcpu(struct vcpu_svm *svm) · line 899

Quoted 7 lines
arch/x86/kvm/svm/avic.c +6 -0
+		unsigned long flags;
+
+		local_irq_save(flags);
+
+		kvm_pi_disable_wakeup_handler(&svm->vcpu, svm->gappi_cpu);
+		svm->gappi_cpu = -1;

Redundant assignement?

-Vasant

#14 lore raw

PATCH v4 7/7 iommu/amd: Provide kernel command line option to enable GAPPI

In reply to Sairaj Kodilkar

Enable GAPPI when the IOMMU GAPPISup extended feature bit is set and the kernel is booted with the amd_iommu=gappi kernel parameter.

Document the option in Documentation/admin-guide/kernel-parameters.txt.

---
 .../admin-guide/kernel-parameters.txt         |  3 +-
 drivers/iommu/amd/amd_iommu_types.h           |  2 ++
 drivers/iommu/amd/init.c                      | 28 +++++++++++++++++--
 3 files changed, 29 insertions(+), 4 deletions(-)
Documentation/admin-guide/kernel-parameters.txt +2 -1
index 97007f4f69d4..7bea1bbaa0e0 100644
@@ -455,7 +455,8 @@ Kernel parameters
 				          to 4 KiB.
 			v2_pgsizes_only - Limit page-sizes used for v1 page-tables
 				          to 4KiB/2Mib/1GiB.
-
+			gappi           - Use GAPPI instead of GALOG for vCPU
+				          wakeup notification.
 
 	amd_iommu_dump=	[HW,X86-64]
 			Enable AMD IOMMU driver option to dump the ACPI table
drivers/iommu/amd/amd_iommu_types.h +2 -0
index 26d7a9796e64..de0b23306944 100644
@@ -94,6 +94,7 @@
 #define FEATURE_X2APIC		BIT_ULL(2)
 #define FEATURE_NX		BIT_ULL(3)
 #define FEATURE_GT		BIT_ULL(4)
+#define FEATURE_GAPPI		BIT_ULL(5)
 #define FEATURE_IA		BIT_ULL(6)
 #define FEATURE_GA		BIT_ULL(7)
 #define FEATURE_HE		BIT_ULL(8)
@@ -191,6 +192,7 @@
 #define CONTROL_EPH_EN		45
 #define CONTROL_XT_EN		50
 #define CONTROL_INTCAPXT_EN	51
+#define CONTROL_GAPPI_EN	55
 #define CONTROL_GCR3TRPMODE	58
 #define CONTROL_IRTCACHEDIS	59
 #define CONTROL_SNPAVIC_EN	61
drivers/iommu/amd/init.c +25 -3
index 2e1889f8a9e4..7d2b1c24f6ec 100644
@@ -465,6 +465,9 @@ static void iommu_disable(struct amd_iommu *iommu)
 	iommu_feature_disable(iommu, CONTROL_GALOG_EN);
 	iommu_feature_disable(iommu, CONTROL_GAINT_EN);
 
+	/* Disable IOMMU GAPPI */
+	iommu_feature_disable(iommu, CONTROL_GAPPI_EN);
+
 	/* Disable IOMMU PPR logging */
 	iommu_feature_disable(iommu, CONTROL_PPRLOG_EN);
 	iommu_feature_disable(iommu, CONTROL_PPRINT_EN);
@@ -2999,6 +3002,13 @@ static void enable_iommus_vapic(void)
 	struct amd_iommu *iommu;
 
 	for_each_iommu(iommu) {
+		/*
+		 * Disable GAPPI, do not check amd_iommu_gappi as it may be
+		 * false in new kexec kernel even though previous kernel has
+		 * enabled it.
+		 */
+		iommu_feature_disable(iommu, CONTROL_GAPPI_EN);
+
 		/*
 		 * Disable GALog if already running. It could have been enabled
 		 * in the previous boot before kdump.
@@ -3038,10 +3048,19 @@ static void enable_iommus_vapic(void)
 		return;
 	}
 
+	if (amd_iommu_gappi &&
+	    !(check_feature(FEATURE_GAPPI) &&
+	      AMD_IOMMU_GUEST_IR_VAPIC(amd_iommu_guest_ir))) {
+		pr_warn("GAPPI is not supported.\n");
+		amd_iommu_gappi = false;
+	}
+
 	/* Enabling GAM and SNPAVIC support */
 	for_each_iommu(iommu) {
-		if (iommu_init_ga_log(iommu) ||
-		    iommu_ga_log_enable(iommu))
+		if (amd_iommu_gappi)
+			iommu_feature_enable(iommu, CONTROL_GAPPI_EN);
+		else if (iommu_init_ga_log(iommu) ||
+			 iommu_ga_log_enable(iommu))
 			return;
 
 		iommu_feature_enable(iommu, CONTROL_GAM_EN);
@@ -3050,7 +3069,8 @@ static void enable_iommus_vapic(void)
 	}
 
 	amd_iommu_irq_ops.capability |= (1 << IRQ_POSTING_CAP);
-	pr_info("Virtual APIC enabled\n");
+	pr_info("Virtual APIC enabled with %s\n",
+		amd_iommu_gappi ? "GAPPI" : "GALOG");
 #endif
 }
 
@@ -3741,6 +3761,8 @@ static int __init parse_amd_iommu_options(char *str)
 		} else if (strncmp(str, "v2_pgsizes_only", 15) == 0) {
 			pr_info("Restricting V1 page-sizes to 4KiB/2MiB/1GiB");
 			amd_iommu_pgsize_bitmap = AMD_IOMMU_PGSIZES_V2;
+		} else if (strncmp(str, "gappi", 5) == 0) {
+			amd_iommu_gappi = true;
 		} else {
 			pr_notice("Unknown option - '%s'\n", str);
 		}
Signature · 1 line
2.34.1
reviewer
#15 lore raw

Re: [PATCH v4 7/7] iommu/amd: Provide kernel command line option to enable GAPPI

Documentation/admin-guide/kernel-parameters.txt Kernel parameters · line 459

Quoted 11 lines · Sairaj Kodilkar
Documentation/admin-guide/kernel-parameters.txt +2 -1
index 97007f4f69d4..7bea1bbaa0e0 100644
@@ -455,7 +455,8 @@ Kernel parameters
 				          to 4 KiB.
 			v2_pgsizes_only - Limit page-sizes used for v1 page-tables
 				          to 4KiB/2Mib/1GiB.
-
+			gappi           - Use GAPPI instead of GALOG for vCPU
+				          wakeup notification.

Is GALOG documented somewhere? Should it be?
I can't find it...

thanks.

Signature · 1 line
~Randy
#16 lore raw

Re: [PATCH v4 7/7] iommu/amd: Provide kernel command line option to enable GAPPI

Quoted 17 lines · Randy Dunlap

Documentation/admin-guide/kernel-parameters.txt Kernel parameters · line 459

Quoted 11 lines · Sairaj Kodilkar
Documentation/admin-guide/kernel-parameters.txt +2 -1
index 97007f4f69d4..7bea1bbaa0e0 100644
@@ -455,7 +455,8 @@ Kernel parameters
 				          to 4 KiB.
 			v2_pgsizes_only - Limit page-sizes used for v1 page-tables
 				          to 4KiB/2Mib/1GiB.
-
+			gappi           - Use GAPPI instead of GALOG for vCPU
+				          wakeup notification.

Is GALOG documented somewhere? Should it be?
I can't find it...

Its not documented. Maybe we can add small documentation in arch/x86/iommu.rst

Thanks
Sairaj

Quoted 2 lines

thanks.

#17 lore raw
In reply to Sairaj Kodilkar
Quoted 132 lines · Sairaj Kodilkar

Introduction
============
On newer generations of AMD processors, IOMMU AVIC/x2AVIC guest-mode interrupt remapping can use Guest APIC Physical Processor Interrupt (GAPPI) as an alternative host-notification path when device interrupts target a vCPU that is not running (IRTE[IsRun] = 0).

With AVIC enabled, the IOMMU posts device interrupts into the guest virtual APIC backing page. When the vCPU is not running, KVM must additionally be notified so it can schedule the vCPU.

The legacy notification path is the GA log (GALOG): the IOMMU appends vCPU tags to a shared GA log buffer and raises a single GA log interrupt. KVM registers a notifier and scans the buffer to decide which vCPUs to wake. Under heavy interrupt load this adds latency and can overflow the buffer because all wakeups funnel through one interrupt and one shared log.

Guest APIC Physical Processor Interrupt (GAPPI), defined in section 2.2.5.4 of the AMD I/O Virtualization Technology (IOMMU) Specification [1], is an alternative. With GAPPI enabled, the IOMMU still updates the guest vAPIC backing page IRR, but may deliver a physical APIC interrupt directly to IRTE[Destination], using IRTE[GATag][7:0] as the vector. This distributes host wakeup notifications across CPUs instead of centralizing them in a log buffer.

This series programs guest-mode IRTEs accordingly: IRTE[Destination] carries the target host physical APIC ID, IRTE[GATag] is set to POSTED_INTR_WAKEUP_VECTOR, and IRTE[GAPPIDis] / IRTE[GALogIntr] are set based on whether KVM requests host wakeup. GAPPI is selected at boot via the amd_iommu=gappi kernel parameter on capable hardware, otherwise the existing GA log path is unchanged.

SVM/AMD IOMMU interface changes
===============================
The first four patches refactor the SVM/AMD IOMMU interface ahead of GAPPI.

The cpu field is renamed to apicid because it carries the host physical APIC ID for IRTE[Destination], not a Linux CPU number.

The ga_log_intr boolean is renamed to wakeup_intr (and the synthetic AVIC_PHYSICAL_ID_ENTRY_GA_LOG_INTR shadow bit to AVIC_PHYSICAL_ID_ENTRY_WAKEUP_INTR). wakeup_intr describes KVM's intent (request host wakeup while the vCPU is not running), not a specific hardware mechanism.

A separate is_running boolean is added to IOMMU interface because GAPPI requires a valid apicid in IRTE[Destination] even when the vCPU is not running. The prior encoding (apicid >= 0 means running, apicid == -1 means not running) no longer works once apicid carries the GAPPI destination while IRTE[IsRun] is clear. The IOMMU driver keys IRTE[IsRun] and destination programming off this explicit boolean instead of inferring running state from apicid.

KVM GAPPI wakeup scheme
=======================
SVM follows the Intel posted-interrupt wakeup model already used by VMX. Each pCPU maintains a list of blocked vCPUs that may be woken by a GAPPI delivery to that CPU. When a vCPU blocks while waiting for a device interrupt, SVM enqueues it on the wakeup list of the pCPU on which it was previously running (gappi_cpu) and passes that pCPU's physical APIC ID to the IOMMU to program IRTE[Destination]. The rationale is that the vCPU is likely to run again on the same pCPU, which is common when vCPUs are pinned; targeting GAPPI notifications there reduces unnecessary VMEXITs from GAPPI deliveries on other CPUs. When the vCPU is scheduled in again, it is removed from the list and IRTE[Destination] is updated to the current pCPU.

List maintenance is moved from VMX layer to common KVM layer so that both VMX and SVM layer can share the same list handling code.

Note
====
kvm_pi_enable_wakeup_handler() and kvm_pi_disable_wakeup_handler() take the target pCPU explicitly instead of using vcpu->cpu, because on SVM the two can differ. avic_vcpu_load() returns early while the vCPU is blocking and leaves the target wakeup CPU untouched, but the generic layer still refreshes vcpu->cpu on every load. Using vcpu->cpu would then unlink the vCPU under the wrong per-pCPU lock, so SVM tracks the pCPU it last programmed in svm->gappi_cpu and passes that instead.

Changes since v3
================
https://lore.kernel.org/all/[email protected]/

Patch 3:
- Make amd_iommu_activate_guest_mode() as wrapper around __amd_iommu_activate_guest_mode() to in order to hardcode is_running flag to "true" for external calls

Patch 5:
- Newly introduced, moves wakeup list handling from VMX to the KVM layer

Patch 6:
- Use common list handling functions introduced in patch 5.
- Remove list addition during pi_update_irte by moving list handling

    functions before list_empty(&svm->ir_list) check.

Changes since v2
================
https://lore.kernel.org/linux-iommu/[email protected]/

Patch[1-6]
- Expand commit messages to explain GAPPI, the interface changes, and the per-CPU wakeup list scheme [Sean].

Patch[1-3]
- Split the monolithic SVM/IOMMU API refactor into four preparatory patches [Sean]
- Rename posted_intr to wakeup_intr to reflect host wakeup intent, not guest interrupt posting [Sean]
- Pass vCPU running status with a extra parameter (is_running) instead of flags.

Patch[4,5]
- Move ga_tag=POSTED_INTR_WAKEUP_VECTOR setting from IOMMU to SVM layer.

Changes since V1:
================
https://lore.kernel.org/all/[email protected]/

Patch4
- Disable interrupts while holding wakeup list lock inside [sashiko] avic_add_vcpu_to_gappi_wakeup_list and avic_remove_vcpu_from_gappi_wakeup_list
- Unregister posted_intr_wakeup_handler during module unload [sashiko]

Patch5
- Disable GAPPI feature during kexec and suspend path [sashiko]

Quoted 5 lines

------
[1] https://docs.amd.com/v/u/en-US/48882_3.11_IOMMU_PUB

------
Base: 89516a1c6837156f43525ef49fdbd781145942f5

Apologies for wrong base commit, Please use following base commit

Base: 8cd9520d35a6c38db6567e97dd93b1f11f185dc6

Quoted 36 lines

Sairaj Kodilkar (7):
iommu/amd: KVM: SVM: Rename cpu to apicid in IOMMU interface iommu/amd: KVM: SVM: Rename ga_log_intr to wakeup_intr in IOMMU
interface
iommu/amd: KVM: SVM: Add explicit vCPU running state to IOMMU
interface
iommu/amd: Program guest-mode IRTEs for GAPPI wakeup when IRTE[IsRun]

    = 0

KVM: VMX: Factor out wakeup list handling code to KVM
KVM: SVM: Add support for AMD IOMMU Guest APIC Physical Processor
Interrupt (GAPPI)
iommu/amd: Provide kernel command line option to enable GAPPI

 .../admin-guide/kernel-parameters.txt         |   3 +-
 arch/x86/include/asm/irq_remapping.h          |   5 +-
 arch/x86/include/asm/kvm-x86-ops.h            |   1 +
 arch/x86/include/asm/kvm_host.h               |   1 +
 arch/x86/include/asm/svm.h                    |   9 +-
 arch/x86/kvm/svm/avic.c                       | 109 +++++++++++++----
 arch/x86/kvm/svm/svm.c                        |   3 +
 arch/x86/kvm/svm/svm.h                        |   5 +
 arch/x86/kvm/vmx/common.h                     |   3 -
 arch/x86/kvm/vmx/main.c                       |   6 +
 arch/x86/kvm/vmx/posted_intr.c                |  82 +------------
 arch/x86/kvm/vmx/tdx.c                        |   1 -
 arch/x86/kvm/vmx/vmx.c                        |  14 +--
 arch/x86/kvm/vmx/x86_ops.h                    |   1 +
 arch/x86/kvm/x86.c                            | 111 ++++++++++++++++++
 drivers/iommu/amd/amd_iommu_types.h           |   6 +-
 drivers/iommu/amd/init.c                      |  31 ++++-
 drivers/iommu/amd/iommu.c                     |  65 ++++++----
 include/linux/amd-iommu.h                     |  16 ++-
 include/linux/kvm_host.h                      |   6 +
 20 files changed, 323 insertions(+), 155 deletions(-)