KVM: X86: Cleanups on dest_mode and headers

PATCH Superseded Peter Xu 3 participants 13 messages Dec 3, 2019 – Dec 4, 2019

Subsystems (4)

x86 KVM KVM x86 KVM x86 Hyper-V

lore

Share & tools

lore

Peter Xu

author

2019-12-03 16:59 UTC #0 lore raw

v4 cover KVM: X86: Cleanups on dest_mode and headers

v4:
- address all comments from Vitaly, adding r-bs properly
- added one more trivial patch:
"KVM: X86: Conert the last users of "shorthand = 0" to use macros"

v3:
- address all the comments from both Vitaly and Sean
- since at it, added patches:
"KVM: X86: Fix kvm_bitmap_or_dest_vcpus() to use irq shorthand" "KVM: X86: Drop KVM_APIC_SHORT_MASK and KVM_APIC_DEST_MASK"

Each patch explains itself.

Please have a look, thanks.

Peter Xu (6):
KVM: X86: Fix kvm_bitmap_or_dest_vcpus() to use irq shorthand
KVM: X86: Move irrelevant declarations out of ioapic.h
KVM: X86: Use APIC_DEST_* macros properly in kvm_lapic_irq.dest_mode
KVM: X86: Drop KVM_APIC_SHORT_MASK and KVM_APIC_DEST_MASK
KVM: X86: Fix callers of kvm_apic_match_dest() to use correct macros
KVM: X86: Conert the last users of "shorthand = 0" to use macros

 arch/x86/include/asm/kvm_host.h |  5 +++++
 arch/x86/kvm/hyperv.c           |  1 +
 arch/x86/kvm/ioapic.c           | 24 +++++++++++++++---------
 arch/x86/kvm/ioapic.h           |  6 ------
 arch/x86/kvm/irq.h              |  3 +++
 arch/x86/kvm/irq_comm.c         | 12 +++++++-----
 arch/x86/kvm/lapic.c            |  5 +----
 arch/x86/kvm/lapic.h            |  7 ++++---
 arch/x86/kvm/svm.c              |  4 ++--
 arch/x86/kvm/x86.c              |  4 ++--
 10 files changed, 40 insertions(+), 31 deletions(-)

Signature · 1 line

2.21.0

Peter Xu

author

2019-12-03 16:59 UTC #1 lore raw

PATCH v4 1/6 KVM: X86: Fix kvm_bitmap_or_dest_vcpus() to use irq shorthand

The 3rd parameter of kvm_apic_match_dest() is the irq shorthand, rather than the irq delivery mode.

Fixes 7ee30bc132c6 ("KVM: x86: deliver KVM IOAPIC scan request to target vCPUs")

Reviewed-by Vitaly Kuznetsov <[email protected]>

Signed-off-by Peter Xu <[email protected]>

---
 arch/x86/kvm/lapic.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

arch/x86/kvm/lapic.c +1 -1

index cf9177b4a07f..1eabe58bb6d5 100644
@@ -1151,7 +1151,7 @@ void kvm_bitmap_or_dest_vcpus(struct kvm *kvm, struct kvm_lapic_irq *irq,
 			if (!kvm_apic_present(vcpu))
 				continue;
 			if (!kvm_apic_match_dest(vcpu, NULL,
-						 irq->delivery_mode,
+						 irq->shorthand,
 						 irq->dest_id,
 						 irq->dest_mode))
 				continue;

Signature · 1 line

2.21.0

Peter Xu

author

2019-12-03 16:59 UTC #2 lore raw

PATCH v4 2/6 KVM: X86: Move irrelevant declarations out of ioapic.h

In reply to Peter Xu

kvm_apic_match_dest() is declared in both ioapic.h and lapic.h. Remove the declaration in ioapic.h.

kvm_apic_compare_prio() is declared in ioapic.h but defined in lapic.c. Move the declaration to lapic.h.

kvm_irq_delivery_to_apic() is declared in ioapic.h but defined in irq_comm.c. Move the declaration to irq.h.

hyperv.c needs to use kvm_irq_delivery_to_apic(). Include irq.h in hyperv.c.

Reviewed-by Vitaly Kuznetsov <[email protected]>

Signed-off-by Peter Xu <[email protected]>

---
 arch/x86/kvm/hyperv.c | 1 +
 arch/x86/kvm/ioapic.h | 6 ------
 arch/x86/kvm/irq.h    | 3 +++
 arch/x86/kvm/lapic.h  | 2 +-
 4 files changed, 5 insertions(+), 7 deletions(-)

arch/x86/kvm/hyperv.c +1 -0

index 23ff65504d7e..c7d4640b7b1c 100644
@@ -33,6 +33,7 @@
 #include <trace/events/kvm.h>
 
 #include "trace.h"
+#include "irq.h"
 
 #define KVM_HV_MAX_SPARSE_VCPU_SET_BITS DIV_ROUND_UP(KVM_MAX_VCPUS, 64)

arch/x86/kvm/ioapic.h +0 -6

index ea1a4e0297da..2fb2e3c80724 100644
@@ -116,9 +116,6 @@ static inline int ioapic_in_kernel(struct kvm *kvm)
 }
 
 void kvm_rtc_eoi_tracking_restore_one(struct kvm_vcpu *vcpu);
-bool kvm_apic_match_dest(struct kvm_vcpu *vcpu, struct kvm_lapic *source,
-		int short_hand, unsigned int dest, int dest_mode);
-int kvm_apic_compare_prio(struct kvm_vcpu *vcpu1, struct kvm_vcpu *vcpu2);
 void kvm_ioapic_update_eoi(struct kvm_vcpu *vcpu, int vector,
 			int trigger_mode);
 int kvm_ioapic_init(struct kvm *kvm);

@@ -126,9 +123,6 @@ void kvm_ioapic_destroy(struct kvm *kvm);
 int kvm_ioapic_set_irq(struct kvm_ioapic *ioapic, int irq, int irq_source_id,
 		       int level, bool line_status);
 void kvm_ioapic_clear_all(struct kvm_ioapic *ioapic, int irq_source_id);
-int kvm_irq_delivery_to_apic(struct kvm *kvm, struct kvm_lapic *src,
-			     struct kvm_lapic_irq *irq,
-			     struct dest_map *dest_map);
 void kvm_get_ioapic(struct kvm *kvm, struct kvm_ioapic_state *state);
 void kvm_set_ioapic(struct kvm *kvm, struct kvm_ioapic_state *state);
 void kvm_ioapic_scan_entry(struct kvm_vcpu *vcpu,

arch/x86/kvm/irq.h +3 -0

index 7c6233d37c64..f173ab6b407e 100644
@@ -113,5 +113,8 @@ int apic_has_pending_timer(struct kvm_vcpu *vcpu);
 
 int kvm_setup_default_irq_routing(struct kvm *kvm);
 int kvm_setup_empty_irq_routing(struct kvm *kvm);
+int kvm_irq_delivery_to_apic(struct kvm *kvm, struct kvm_lapic *src,
+			     struct kvm_lapic_irq *irq,
+			     struct dest_map *dest_map);
 
 #endif

arch/x86/kvm/lapic.h +1 -1

index 39925afdfcdc..0b9bbadd1f3c 100644
@@ -83,7 +83,7 @@ int kvm_lapic_reg_read(struct kvm_lapic *apic, u32 offset, int len,
 		       void *data);
 bool kvm_apic_match_dest(struct kvm_vcpu *vcpu, struct kvm_lapic *source,
 			   int short_hand, unsigned int dest, int dest_mode);
-
+int kvm_apic_compare_prio(struct kvm_vcpu *vcpu1, struct kvm_vcpu *vcpu2);
 bool __kvm_apic_update_irr(u32 *pir, void *regs, int *max_irr);
 bool kvm_apic_update_irr(struct kvm_vcpu *vcpu, u32 *pir, int *max_irr);
 void kvm_apic_update_ppr(struct kvm_vcpu *vcpu);

Signature · 1 line

2.21.0

Peter Xu

author

2019-12-03 16:59 UTC #3 lore raw

PATCH v4 3/6 KVM: X86: Use APIC_DEST_* macros properly in kvm_lapic_irq.dest_mode

In reply to Peter Xu

We were using either APIC_DEST_PHYSICAL|APIC_DEST_LOGICAL or 0|1 to fill in kvm_lapic_irq.dest_mode. It's fine only because in most cases when we check against dest_mode it's against APIC_DEST_PHYSICAL (which equals to 0). However, that's not consistent. We'll have problem when we want to start checking against APIC_DEST_LOGICAL, which does not equals to 1.

This patch firstly introduces kvm_lapic_irq_dest_mode() helper to take any boolean of destination mode and return the APIC_DEST_* macro. Then, it replaces the 0|1 settings of irq.dest_mode with the helper.

Signed-off-by Peter Xu <[email protected]>

---
 arch/x86/include/asm/kvm_host.h | 5 +++++
 arch/x86/kvm/ioapic.c           | 9 ++++++---
 arch/x86/kvm/irq_comm.c         | 7 ++++---
 arch/x86/kvm/x86.c              | 2 +-
 4 files changed, 16 insertions(+), 7 deletions(-)

arch/x86/include/asm/kvm_host.h +5 -0

index b79cd6aa4075..f815c97b1b57 100644
@@ -1022,6 +1022,11 @@ struct kvm_lapic_irq {
 	bool msi_redir_hint;
 };
 
+static inline u16 kvm_lapic_irq_dest_mode(bool dest_mode)
+{
+	return dest_mode ? APIC_DEST_LOGICAL : APIC_DEST_PHYSICAL;
+}
+
 struct kvm_x86_ops {
 	int (*cpu_has_kvm_support)(void);          /* __init */
 	int (*disabled_by_bios)(void);             /* __init */

arch/x86/kvm/ioapic.c +6 -3

index 9fd2dd89a1c5..e623a4f8d27e 100644
@@ -331,7 +331,8 @@ static void ioapic_write_indirect(struct kvm_ioapic *ioapic, u32 val)
 			irq.vector = e->fields.vector;
 			irq.delivery_mode = e->fields.delivery_mode << 8;
 			irq.dest_id = e->fields.dest_id;
-			irq.dest_mode = e->fields.dest_mode;
+			irq.dest_mode =
+			    kvm_lapic_irq_dest_mode(!!e->fields.dest_mode);
 			bitmap_zero(&vcpu_bitmap, 16);
 			kvm_bitmap_or_dest_vcpus(ioapic->kvm, &irq,
 						 &vcpu_bitmap);

@@ -343,7 +344,9 @@ static void ioapic_write_indirect(struct kvm_ioapic *ioapic, u32 val)
 				 * keep ioapic_handled_vectors synchronized.
 				 */
 				irq.dest_id = old_dest_id;
-				irq.dest_mode = old_dest_mode;
+				irq.dest_mode =
+				    kvm_lapic_irq_dest_mode(
+					!!e->fields.dest_mode);
 				kvm_bitmap_or_dest_vcpus(ioapic->kvm, &irq,
 							 &vcpu_bitmap);
 			}

@@ -369,7 +372,7 @@ static int ioapic_service(struct kvm_ioapic *ioapic, int irq, bool line_status)
 
 	irqe.dest_id = entry->fields.dest_id;
 	irqe.vector = entry->fields.vector;
-	irqe.dest_mode = entry->fields.dest_mode;
+	irqe.dest_mode = kvm_lapic_irq_dest_mode(!!entry->fields.dest_mode);
 	irqe.trig_mode = entry->fields.trig_mode;
 	irqe.delivery_mode = entry->fields.delivery_mode << 8;
 	irqe.level = 1;

arch/x86/kvm/irq_comm.c +4 -3

index 8ecd48d31800..22108ed66a76 100644
@@ -52,8 +52,8 @@ int kvm_irq_delivery_to_apic(struct kvm *kvm, struct kvm_lapic *src,
 	unsigned long dest_vcpu_bitmap[BITS_TO_LONGS(KVM_MAX_VCPUS)];
 	unsigned int dest_vcpus = 0;
 
-	if (irq->dest_mode == 0 && irq->dest_id == 0xff &&
-			kvm_lowest_prio_delivery(irq)) {
+	if (irq->dest_mode == APIC_DEST_PHYSICAL &&
+	    irq->dest_id == 0xff && kvm_lowest_prio_delivery(irq)) {
 		printk(KERN_INFO "kvm: apic: phys broadcast and lowest prio\n");
 		irq->delivery_mode = APIC_DM_FIXED;
 	}

@@ -114,7 +114,8 @@ void kvm_set_msi_irq(struct kvm *kvm, struct kvm_kernel_irq_routing_entry *e,
 		irq->dest_id |= MSI_ADDR_EXT_DEST_ID(e->msi.address_hi);
 	irq->vector = (e->msi.data &
 			MSI_DATA_VECTOR_MASK) >> MSI_DATA_VECTOR_SHIFT;
-	irq->dest_mode = (1 << MSI_ADDR_DEST_MODE_SHIFT) & e->msi.address_lo;
+	irq->dest_mode = kvm_lapic_irq_dest_mode(
+	    !!((1 << MSI_ADDR_DEST_MODE_SHIFT) & e->msi.address_lo));
 	irq->trig_mode = (1 << MSI_DATA_TRIGGER_SHIFT) & e->msi.data;
 	irq->delivery_mode = e->msi.data & 0x700;
 	irq->msi_redir_hint = ((e->msi.address_lo

arch/x86/kvm/x86.c +1 -1

index 3ed167e039e5..3b00d662dc14 100644
@@ -7356,7 +7356,7 @@ static void kvm_pv_kick_cpu_op(struct kvm *kvm, unsigned long flags, int apicid)
 	struct kvm_lapic_irq lapic_irq;
 
 	lapic_irq.shorthand = 0;
-	lapic_irq.dest_mode = 0;
+	lapic_irq.dest_mode = APIC_DEST_PHYSICAL;
 	lapic_irq.level = 0;
 	lapic_irq.dest_id = apicid;
 	lapic_irq.msi_redir_hint = false;

Signature · 1 line

2.21.0

Sean Christopherson

2019-12-03 22:07 UTC #4 lore raw

Re: [PATCH v4 3/6] KVM: X86: Use APIC_DEST_* macros properly in kvm_lapic_irq.dest_mode

Quoted 24 lines · Peter Xu

We were using either APIC_DEST_PHYSICAL|APIC_DEST_LOGICAL or 0|1 to fill in kvm_lapic_irq.dest_mode. It's fine only because in most cases when we check against dest_mode it's against APIC_DEST_PHYSICAL (which equals to 0). However, that's not consistent. We'll have problem when we want to start checking against APIC_DEST_LOGICAL, which does not equals to 1.
This patch firstly introduces kvm_lapic_irq_dest_mode() helper to take any boolean of destination mode and return the APIC_DEST_* macro. Then, it replaces the 0|1 settings of irq.dest_mode with the helper.
Signed-off-by Peter Xu <[email protected]>
---
 arch/x86/include/asm/kvm_host.h | 5 +++++
 arch/x86/kvm/ioapic.c           | 9 ++++++---
 arch/x86/kvm/irq_comm.c         | 7 ++++---
 arch/x86/kvm/x86.c              | 2 +-
 4 files changed, 16 insertions(+), 7 deletions(-)
arch/x86/include/asm/kvm_host.h +0 -0
index b79cd6aa4075..f815c97b1b57 100644

arch/x86/include/asm/kvm_host.h struct kvm_lapic_irq { · line 1027

Quoted 6 lines

arch/x86/include/asm/kvm_host.h +3 -0

@@ -1022,6 +1022,11 @@ struct kvm_lapic_irq {
 	bool msi_redir_hint;
 };
 
+static inline u16 kvm_lapic_irq_dest_mode(bool dest_mode)
+{
+	return dest_mode ? APIC_DEST_LOGICAL : APIC_DEST_PHYSICAL;

IMO this belongs in ioapic.c as it's specifically provided for converting an I/O APIC redirection entry into a local APIC destination mode. Without the I/O APIC context, %true==APIC_DEST_LOGICAL looks like a completely arbitrary decision. And if it's in ioapic.c, it can take the union of a bool, which avoids the casting and shortens the callers. E.g.:

static u64 ioapic_to_lapic_dest_mode(union kvm_ioapic_redirect_entry *e)

{
	return e->fields.dest_mode ?  APIC_DEST_LOGICAL : APIC_DEST_PHYSICAL;
}

The other option would be to use the same approach as delivery_mode and open code the shift.

Quoted 80 lines

arch/x86/include/asm/kvm_host.h +2 -0

+}
+
 struct kvm_x86_ops {
 	int (*cpu_has_kvm_support)(void);          /* __init */
 	int (*disabled_by_bios)(void);             /* __init */

arch/x86/kvm/ioapic.c +6 -3

index 9fd2dd89a1c5..e623a4f8d27e 100644
@@ -331,7 +331,8 @@ static void ioapic_write_indirect(struct kvm_ioapic *ioapic, u32 val)
 			irq.vector = e->fields.vector;
 			irq.delivery_mode = e->fields.delivery_mode << 8;
 			irq.dest_id = e->fields.dest_id;
-			irq.dest_mode = e->fields.dest_mode;
+			irq.dest_mode =
+			    kvm_lapic_irq_dest_mode(!!e->fields.dest_mode);
 			bitmap_zero(&vcpu_bitmap, 16);
 			kvm_bitmap_or_dest_vcpus(ioapic->kvm, &irq,
 						 &vcpu_bitmap);

@@ -343,7 +344,9 @@ static void ioapic_write_indirect(struct kvm_ioapic *ioapic, u32 val)
 				 * keep ioapic_handled_vectors synchronized.
 				 */
 				irq.dest_id = old_dest_id;
-				irq.dest_mode = old_dest_mode;
+				irq.dest_mode =
+				    kvm_lapic_irq_dest_mode(
+					!!e->fields.dest_mode);
 				kvm_bitmap_or_dest_vcpus(ioapic->kvm, &irq,
 							 &vcpu_bitmap);
 			}

@@ -369,7 +372,7 @@ static int ioapic_service(struct kvm_ioapic *ioapic, int irq, bool line_status)
 
 	irqe.dest_id = entry->fields.dest_id;
 	irqe.vector = entry->fields.vector;
-	irqe.dest_mode = entry->fields.dest_mode;
+	irqe.dest_mode = kvm_lapic_irq_dest_mode(!!entry->fields.dest_mode);
 	irqe.trig_mode = entry->fields.trig_mode;
 	irqe.delivery_mode = entry->fields.delivery_mode << 8;
 	irqe.level = 1;

arch/x86/kvm/irq_comm.c +4 -3

index 8ecd48d31800..22108ed66a76 100644
@@ -52,8 +52,8 @@ int kvm_irq_delivery_to_apic(struct kvm *kvm, struct kvm_lapic *src,
 	unsigned long dest_vcpu_bitmap[BITS_TO_LONGS(KVM_MAX_VCPUS)];
 	unsigned int dest_vcpus = 0;
 
-	if (irq->dest_mode == 0 && irq->dest_id == 0xff &&
-			kvm_lowest_prio_delivery(irq)) {
+	if (irq->dest_mode == APIC_DEST_PHYSICAL &&
+	    irq->dest_id == 0xff && kvm_lowest_prio_delivery(irq)) {
 		printk(KERN_INFO "kvm: apic: phys broadcast and lowest prio\n");
 		irq->delivery_mode = APIC_DM_FIXED;
 	}

@@ -114,7 +114,8 @@ void kvm_set_msi_irq(struct kvm *kvm, struct kvm_kernel_irq_routing_entry *e,
 		irq->dest_id |= MSI_ADDR_EXT_DEST_ID(e->msi.address_hi);
 	irq->vector = (e->msi.data &
 			MSI_DATA_VECTOR_MASK) >> MSI_DATA_VECTOR_SHIFT;
-	irq->dest_mode = (1 << MSI_ADDR_DEST_MODE_SHIFT) & e->msi.address_lo;
+	irq->dest_mode = kvm_lapic_irq_dest_mode(
+	    !!((1 << MSI_ADDR_DEST_MODE_SHIFT) & e->msi.address_lo));
 	irq->trig_mode = (1 << MSI_DATA_TRIGGER_SHIFT) & e->msi.data;
 	irq->delivery_mode = e->msi.data & 0x700;
 	irq->msi_redir_hint = ((e->msi.address_lo

arch/x86/kvm/x86.c +1 -1

index 3ed167e039e5..3b00d662dc14 100644
@@ -7356,7 +7356,7 @@ static void kvm_pv_kick_cpu_op(struct kvm *kvm, unsigned long flags, int apicid)
 	struct kvm_lapic_irq lapic_irq;
 
 	lapic_irq.shorthand = 0;
-	lapic_irq.dest_mode = 0;
+	lapic_irq.dest_mode = APIC_DEST_PHYSICAL;
 	lapic_irq.level = 0;
 	lapic_irq.dest_id = apicid;
 	lapic_irq.msi_redir_hint = false;

Signature · 1 line

2.21.0

Peter Xu

author

2019-12-03 22:15 UTC #5 lore raw

Re: [PATCH v4 3/6] KVM: X86: Use APIC_DEST_* macros properly in kvm_lapic_irq.dest_mode

Quoted 38 lines · Sean Christopherson

Quoted 24 lines · Peter Xu
We were using either APIC_DEST_PHYSICAL|APIC_DEST_LOGICAL or 0|1 to fill in kvm_lapic_irq.dest_mode. It's fine only because in most cases when we check against dest_mode it's against APIC_DEST_PHYSICAL (which equals to 0). However, that's not consistent. We'll have problem when we want to start checking against APIC_DEST_LOGICAL, which does not equals to 1.
This patch firstly introduces kvm_lapic_irq_dest_mode() helper to take any boolean of destination mode and return the APIC_DEST_* macro. Then, it replaces the 0|1 settings of irq.dest_mode with the helper.
Signed-off-by Peter Xu <[email protected]>
---
 arch/x86/include/asm/kvm_host.h | 5 +++++
 arch/x86/kvm/ioapic.c           | 9 ++++++---
 arch/x86/kvm/irq_comm.c         | 7 ++++---
 arch/x86/kvm/x86.c              | 2 +-
 4 files changed, 16 insertions(+), 7 deletions(-)
arch/x86/include/asm/kvm_host.h +0 -0
index b79cd6aa4075..f815c97b1b57 100644
arch/x86/include/asm/kvm_host.h struct kvm_lapic_irq { · line 1027
Quoted 6 lines
arch/x86/include/asm/kvm_host.h +3 -0
@@ -1022,6 +1022,11 @@ struct kvm_lapic_irq {
 	bool msi_redir_hint;
 };
 
+static inline u16 kvm_lapic_irq_dest_mode(bool dest_mode)
+{
+	return dest_mode ? APIC_DEST_LOGICAL : APIC_DEST_PHYSICAL;
IMO this belongs in ioapic.c as it's specifically provided for converting an I/O APIC redirection entry into a local APIC destination mode. Without the I/O APIC context, %true==APIC_DEST_LOGICAL looks like a completely arbitrary decision. And if it's in ioapic.c, it can take the union of a bool, which avoids the casting and shortens the callers. E.g.:

Quoted 7 lines

static u64 ioapic_to_lapic_dest_mode(union kvm_ioapic_redirect_entry *e)
{
	return e->fields.dest_mode ?  APIC_DEST_LOGICAL : APIC_DEST_PHYSICAL;
}
The other option would be to use the same approach as delivery_mode and open code the shift.

It's also used for MSI address encodings, please see below [1].

The thing is that no matter how external protocols define destination mode, it's always a boolean no matter where it resides, or at which bit.

Thanks,

[...]

Quoted 7 lines

arch/x86/include/asm/kvm_host.h void kvm_set_msi_irq(struct kvm *kvm, struct kvm_kernel_irq_routing_entry *e, · line 118

Quoted 7 lines

arch/x86/include/asm/kvm_host.h +2 -1

@@ -114,7 +114,8 @@ void kvm_set_msi_irq(struct kvm *kvm, struct kvm_kernel_irq_routing_entry *e,
 		irq->dest_id |= MSI_ADDR_EXT_DEST_ID(e->msi.address_hi);
 	irq->vector = (e->msi.data &
 			MSI_DATA_VECTOR_MASK) >> MSI_DATA_VECTOR_SHIFT;
-	irq->dest_mode = (1 << MSI_ADDR_DEST_MODE_SHIFT) & e->msi.address_lo;
+	irq->dest_mode = kvm_lapic_irq_dest_mode(
+	    !!((1 << MSI_ADDR_DEST_MODE_SHIFT) & e->msi.address_lo));

[1]

Quoted 3 lines

arch/x86/include/asm/kvm_host.h +0 -0

 	irq->trig_mode = (1 << MSI_DATA_TRIGGER_SHIFT) & e->msi.data;
 	irq->delivery_mode = e->msi.data & 0x700;
 	irq->msi_redir_hint = ((e->msi.address_lo

Signature · 1 line

Peter Xu

Sean Christopherson

2019-12-03 22:20 UTC #6 lore raw

Re: [PATCH v4 3/6] KVM: X86: Use APIC_DEST_* macros properly in kvm_lapic_irq.dest_mode

Quoted 25 lines · Peter Xu

Quoted 24 lines · Sean Christopherson
arch/x86/include/asm/kvm_host.h struct kvm_lapic_irq { · line 1027
Quoted 11 lines · Peter Xu
arch/x86/include/asm/kvm_host.h +3 -0
index b79cd6aa4075..f815c97b1b57 100644
@@ -1022,6 +1022,11 @@ struct kvm_lapic_irq {
 	bool msi_redir_hint;
 };
 
+static inline u16 kvm_lapic_irq_dest_mode(bool dest_mode)
+{
+	return dest_mode ? APIC_DEST_LOGICAL : APIC_DEST_PHYSICAL;
IMO this belongs in ioapic.c as it's specifically provided for converting an I/O APIC redirection entry into a local APIC destination mode. Without the I/O APIC context, %true==APIC_DEST_LOGICAL looks like a completely arbitrary decision. And if it's in ioapic.c, it can take the union of a bool, which avoids the casting and shortens the callers. E.g.:
static u64 ioapic_to_lapic_dest_mode(union kvm_ioapic_redirect_entry *e)
{
	return e->fields.dest_mode ?  APIC_DEST_LOGICAL : APIC_DEST_PHYSICAL;
}

Quoted 4 lines

Quoted 2 lines
The other option would be to use the same approach as delivery_mode and open code the shift.
It's also used for MSI address encodings, please see below [1].

Boooh. How about naming the param "logical_dest_mode" or something else with "logical" in the name so that the correctness of the function itself is apparent?

Peter Xu

author

2019-12-04 18:48 UTC #7 lore raw

Re: [PATCH v4 3/6] KVM: X86: Use APIC_DEST_* macros properly in kvm_lapic_irq.dest_mode

Quoted 26 lines · Sean Christopherson

Quoted 25 lines · Peter Xu
Quoted 24 lines · Sean Christopherson
arch/x86/include/asm/kvm_host.h struct kvm_lapic_irq { · line 1027
Quoted 11 lines · Peter Xu
arch/x86/include/asm/kvm_host.h +3 -0
index b79cd6aa4075..f815c97b1b57 100644
@@ -1022,6 +1022,11 @@ struct kvm_lapic_irq {
 	bool msi_redir_hint;
 };
 
+static inline u16 kvm_lapic_irq_dest_mode(bool dest_mode)
+{
+	return dest_mode ? APIC_DEST_LOGICAL : APIC_DEST_PHYSICAL;
IMO this belongs in ioapic.c as it's specifically provided for converting an I/O APIC redirection entry into a local APIC destination mode. Without the I/O APIC context, %true==APIC_DEST_LOGICAL looks like a completely arbitrary decision. And if it's in ioapic.c, it can take the union of a bool, which avoids the casting and shortens the callers. E.g.:
static u64 ioapic_to_lapic_dest_mode(union kvm_ioapic_redirect_entry *e)
{
	return e->fields.dest_mode ?  APIC_DEST_LOGICAL : APIC_DEST_PHYSICAL;
}

Quoted 8 lines

Quoted 4 lines
Quoted 2 lines
The other option would be to use the same approach as delivery_mode and open code the shift.
It's also used for MSI address encodings, please see below [1].
Boooh. How about naming the param "logical_dest_mode" or something else with "logical" in the name so that the correctness of the function itself is apparent?

Ok, will do. Thanks,

Signature · 1 line

Peter Xu

Peter Xu

author

2019-12-03 16:59 UTC #8 lore raw

PATCH v4 4/6 KVM: X86: Drop KVM_APIC_SHORT_MASK and KVM_APIC_DEST_MASK

In reply to Peter Xu

We have both APIC_SHORT_MASK and KVM_APIC_SHORT_MASK defined for the shorthand mask. Similarly, we have both APIC_DEST_MASK and KVM_APIC_DEST_MASK defined for the destination mode mask.

Drop the KVM_APIC_* macros and replace the only user of them to use the APIC_DEST_* macros instead. At the meantime, move APIC_SHORT_MASK and APIC_DEST_MASK from lapic.c to lapic.h.

Reviewed-by Vitaly Kuznetsov <[email protected]>

Signed-off-by Peter Xu <[email protected]>

---
 arch/x86/kvm/lapic.c | 3 ---
 arch/x86/kvm/lapic.h | 5 +++--
 arch/x86/kvm/svm.c   | 4 ++--
 3 files changed, 5 insertions(+), 7 deletions(-)

arch/x86/kvm/lapic.c +0 -3

index 1eabe58bb6d5..805c18178bbf 100644
@@ -56,9 +56,6 @@
 #define APIC_VERSION			(0x14UL | ((KVM_APIC_LVT_NUM - 1) << 16))
 #define LAPIC_MMIO_LENGTH		(1 << 12)
 /* followed define is not in apicdef.h */
-#define APIC_SHORT_MASK			0xc0000
-#define APIC_DEST_NOSHORT		0x0
-#define APIC_DEST_MASK			0x800
 #define MAX_APIC_VECTOR			256
 #define APIC_VECTORS_PER_REG		32

arch/x86/kvm/lapic.h +3 -2

index 0b9bbadd1f3c..5a9f29ed9a4b 100644
@@ -10,8 +10,9 @@
 #define KVM_APIC_SIPI		1
 #define KVM_APIC_LVT_NUM	6
 
-#define KVM_APIC_SHORT_MASK	0xc0000
-#define KVM_APIC_DEST_MASK	0x800
+#define APIC_SHORT_MASK			0xc0000
+#define APIC_DEST_NOSHORT		0x0
+#define APIC_DEST_MASK			0x800
 
 #define APIC_BUS_CYCLE_NS       1
 #define APIC_BUS_FREQUENCY      (1000000000ULL / APIC_BUS_CYCLE_NS)

arch/x86/kvm/svm.c +2 -2

index 362e874297e4..65a27a7e9cb1 100644
@@ -4519,9 +4519,9 @@ static int avic_incomplete_ipi_interception(struct vcpu_svm *svm)
 		 */
 		kvm_for_each_vcpu(i, vcpu, kvm) {
 			bool m = kvm_apic_match_dest(vcpu, apic,
-						     icrl & KVM_APIC_SHORT_MASK,
+						     icrl & APIC_SHORT_MASK,
 						     GET_APIC_DEST_FIELD(icrh),
-						     icrl & KVM_APIC_DEST_MASK);
+						     icrl & APIC_DEST_MASK);
 
 			if (m && !avic_vcpu_is_running(vcpu))
 				kvm_vcpu_wake_up(vcpu);

Signature · 1 line

2.21.0

Peter Xu

author

2019-12-03 16:59 UTC #9 lore raw

PATCH v4 5/6 KVM: X86: Fix callers of kvm_apic_match_dest() to use correct macros

In reply to Peter Xu

Callers of kvm_apic_match_dest() should always pass in APIC_DEST_* macros for either dest_mode and short_hand parameters. Fix up all the callers of kvm_apic_match_dest() that are not following the rule.

Reported-by Sean Christopherson <[email protected]>

Reported-by Vitaly Kuznetsov <[email protected]>

Reviewed-by Vitaly Kuznetsov <[email protected]>

Signed-off-by Peter Xu <[email protected]>

---
 arch/x86/kvm/ioapic.c   | 11 +++++++----
 arch/x86/kvm/irq_comm.c |  3 ++-
 2 files changed, 9 insertions(+), 5 deletions(-)

arch/x86/kvm/ioapic.c +7 -4

index e623a4f8d27e..f53daeaaeb37 100644
@@ -108,8 +108,9 @@ static void __rtc_irq_eoi_tracking_restore_one(struct kvm_vcpu *vcpu)
 	union kvm_ioapic_redirect_entry *e;
 
 	e = &ioapic->redirtbl[RTC_GSI];
-	if (!kvm_apic_match_dest(vcpu, NULL, 0,	e->fields.dest_id,
-				e->fields.dest_mode))
+	if (!kvm_apic_match_dest(vcpu, NULL, APIC_DEST_NOSHORT,
+				 e->fields.dest_id,
+				 kvm_lapic_irq_dest_mode(!!e->fields.dest_mode)))
 		return;
 
 	new_val = kvm_apic_pending_eoi(vcpu, e->fields.vector);

@@ -250,8 +251,10 @@ void kvm_ioapic_scan_entry(struct kvm_vcpu *vcpu, ulong *ioapic_handled_vectors)
 		if (e->fields.trig_mode == IOAPIC_LEVEL_TRIG ||
 		    kvm_irq_has_notifier(ioapic->kvm, KVM_IRQCHIP_IOAPIC, index) ||
 		    index == RTC_GSI) {
-			if (kvm_apic_match_dest(vcpu, NULL, 0,
-			             e->fields.dest_id, e->fields.dest_mode) ||
+			u16 dm = kvm_lapic_irq_dest_mode(!!e->fields.dest_mode);
+
+			if (kvm_apic_match_dest(vcpu, NULL, APIC_DEST_NOSHORT,
+						e->fields.dest_id, dm) ||
 			    kvm_apic_pending_eoi(vcpu, e->fields.vector))
 				__set_bit(e->fields.vector,
 					  ioapic_handled_vectors);

arch/x86/kvm/irq_comm.c +2 -1

index 22108ed66a76..7d083f71fc8e 100644
@@ -417,7 +417,8 @@ void kvm_scan_ioapic_routes(struct kvm_vcpu *vcpu,
 
 			kvm_set_msi_irq(vcpu->kvm, entry, &irq);
 
-			if (irq.level && kvm_apic_match_dest(vcpu, NULL, 0,
+			if (irq.level &&
+			    kvm_apic_match_dest(vcpu, NULL, APIC_DEST_NOSHORT,
 						irq.dest_id, irq.dest_mode))
 				__set_bit(irq.vector, ioapic_handled_vectors);
 		}

Signature · 1 line

2.21.0

Peter Xu

author

2019-12-03 16:59 UTC #10 lore raw

PATCH v4 6/6 KVM: X86: Conert the last users of "shorthand = 0" to use macros

In reply to Peter Xu

Change the last users of "shorthand = 0" to use APIC_DEST_NOSHORT.

Signed-off-by Peter Xu <[email protected]>

---
 arch/x86/kvm/ioapic.c   | 4 ++--
 arch/x86/kvm/irq_comm.c | 2 +-
 arch/x86/kvm/x86.c      | 2 +-
 3 files changed, 4 insertions(+), 4 deletions(-)

arch/x86/kvm/ioapic.c +2 -2

index f53daeaaeb37..77538fd77dc2 100644
@@ -330,7 +330,7 @@ static void ioapic_write_indirect(struct kvm_ioapic *ioapic, u32 val)
 		if (e->fields.delivery_mode == APIC_DM_FIXED) {
 			struct kvm_lapic_irq irq;
 
-			irq.shorthand = 0;
+			irq.shorthand = APIC_DEST_NOSHORT;
 			irq.vector = e->fields.vector;
 			irq.delivery_mode = e->fields.delivery_mode << 8;
 			irq.dest_id = e->fields.dest_id;

@@ -379,7 +379,7 @@ static int ioapic_service(struct kvm_ioapic *ioapic, int irq, bool line_status)
 	irqe.trig_mode = entry->fields.trig_mode;
 	irqe.delivery_mode = entry->fields.delivery_mode << 8;
 	irqe.level = 1;
-	irqe.shorthand = 0;
+	irqe.shorthand = APIC_DEST_NOSHORT;
 	irqe.msi_redir_hint = false;
 
 	if (irqe.trig_mode == IOAPIC_EDGE_TRIG)

arch/x86/kvm/irq_comm.c +1 -1

index 7d083f71fc8e..9d711c2451c7 100644
@@ -121,7 +121,7 @@ void kvm_set_msi_irq(struct kvm *kvm, struct kvm_kernel_irq_routing_entry *e,
 	irq->msi_redir_hint = ((e->msi.address_lo
 		& MSI_ADDR_REDIRECTION_LOWPRI) > 0);
 	irq->level = 1;
-	irq->shorthand = 0;
+	irq->shorthand = APIC_DEST_NOSHORT;
 }
 EXPORT_SYMBOL_GPL(kvm_set_msi_irq);

arch/x86/kvm/x86.c +1 -1

index 3b00d662dc14..f6d778436e15 100644
@@ -7355,7 +7355,7 @@ static void kvm_pv_kick_cpu_op(struct kvm *kvm, unsigned long flags, int apicid)
 {
 	struct kvm_lapic_irq lapic_irq;
 
-	lapic_irq.shorthand = 0;
+	lapic_irq.shorthand = APIC_DEST_NOSHORT;
 	lapic_irq.dest_mode = APIC_DEST_PHYSICAL;
 	lapic_irq.level = 0;
 	lapic_irq.dest_id = apicid;

Signature · 1 line

2.21.0

Vitaly Kuznetsov

maintainer Reviewed-by on v4 6/6

2019-12-04 16:12 UTC #11 lore raw

Re: [PATCH v4 6/6] KVM: X86: Conert the last users of "shorthand = 0" to use macros

Quoted 49 lines · Peter Xu

Change the last users of "shorthand = 0" to use APIC_DEST_NOSHORT.

Signed-off-by Peter Xu <[email protected]>

---
 arch/x86/kvm/ioapic.c   | 4 ++--
 arch/x86/kvm/irq_comm.c | 2 +-
 arch/x86/kvm/x86.c      | 2 +-
 3 files changed, 4 insertions(+), 4 deletions(-)

arch/x86/kvm/ioapic.c +2 -2

index f53daeaaeb37..77538fd77dc2 100644
@@ -330,7 +330,7 @@ static void ioapic_write_indirect(struct kvm_ioapic *ioapic, u32 val)
 		if (e->fields.delivery_mode == APIC_DM_FIXED) {
 			struct kvm_lapic_irq irq;
 
-			irq.shorthand = 0;
+			irq.shorthand = APIC_DEST_NOSHORT;
 			irq.vector = e->fields.vector;
 			irq.delivery_mode = e->fields.delivery_mode << 8;
 			irq.dest_id = e->fields.dest_id;

@@ -379,7 +379,7 @@ static int ioapic_service(struct kvm_ioapic *ioapic, int irq, bool line_status)
 	irqe.trig_mode = entry->fields.trig_mode;
 	irqe.delivery_mode = entry->fields.delivery_mode << 8;
 	irqe.level = 1;
-	irqe.shorthand = 0;
+	irqe.shorthand = APIC_DEST_NOSHORT;
 	irqe.msi_redir_hint = false;
 
 	if (irqe.trig_mode == IOAPIC_EDGE_TRIG)

arch/x86/kvm/irq_comm.c +1 -1

index 7d083f71fc8e..9d711c2451c7 100644
@@ -121,7 +121,7 @@ void kvm_set_msi_irq(struct kvm *kvm, struct kvm_kernel_irq_routing_entry *e,
 	irq->msi_redir_hint = ((e->msi.address_lo
 		& MSI_ADDR_REDIRECTION_LOWPRI) > 0);
 	irq->level = 1;
-	irq->shorthand = 0;
+	irq->shorthand = APIC_DEST_NOSHORT;
 }
 EXPORT_SYMBOL_GPL(kvm_set_msi_irq);

arch/x86/kvm/x86.c +0 -0

index 3b00d662dc14..f6d778436e15 100644

arch/x86/kvm/x86.c static void kvm_pv_kick_cpu_op(struct kvm *kvm, unsigned long flags, int apicid) · line 7361

Quoted 8 lines

arch/x86/kvm/x86.c +1 -1

@@ -7355,7 +7355,7 @@ static void kvm_pv_kick_cpu_op(struct kvm *kvm, unsigned long flags, int apicid)
 {
 	struct kvm_lapic_irq lapic_irq;
 
-	lapic_irq.shorthand = 0;
+	lapic_irq.shorthand = APIC_DEST_NOSHORT;
 	lapic_irq.dest_mode = APIC_DEST_PHYSICAL;
 	lapic_irq.level = 0;
 	lapic_irq.dest_id = apicid;

Reviewed-by Vitaly Kuznetsov <[email protected]>

And, while on it, and if you're not yet tired I just noticed that kvm_apic_match_dest()'s parameter is called 'short_hand' while everywhere else we use 'shorthand'. Value the consistency we should :-)

Signature · 1 line

Vitaly

Peter Xu

author

2019-12-04 18:53 UTC #12 lore raw

Re: [PATCH v4 6/6] KVM: X86: Conert the last users of "shorthand = 0" to use macros

Quoted 62 lines · Vitaly Kuznetsov

Quoted 49 lines · Peter Xu

Change the last users of "shorthand = 0" to use APIC_DEST_NOSHORT.

Signed-off-by Peter Xu <[email protected]>

---
 arch/x86/kvm/ioapic.c   | 4 ++--
 arch/x86/kvm/irq_comm.c | 2 +-
 arch/x86/kvm/x86.c      | 2 +-
 3 files changed, 4 insertions(+), 4 deletions(-)

arch/x86/kvm/ioapic.c +2 -2

index f53daeaaeb37..77538fd77dc2 100644
@@ -330,7 +330,7 @@ static void ioapic_write_indirect(struct kvm_ioapic *ioapic, u32 val)
 		if (e->fields.delivery_mode == APIC_DM_FIXED) {
 			struct kvm_lapic_irq irq;
 
-			irq.shorthand = 0;
+			irq.shorthand = APIC_DEST_NOSHORT;
 			irq.vector = e->fields.vector;
 			irq.delivery_mode = e->fields.delivery_mode << 8;
 			irq.dest_id = e->fields.dest_id;

@@ -379,7 +379,7 @@ static int ioapic_service(struct kvm_ioapic *ioapic, int irq, bool line_status)
 	irqe.trig_mode = entry->fields.trig_mode;
 	irqe.delivery_mode = entry->fields.delivery_mode << 8;
 	irqe.level = 1;
-	irqe.shorthand = 0;
+	irqe.shorthand = APIC_DEST_NOSHORT;
 	irqe.msi_redir_hint = false;
 
 	if (irqe.trig_mode == IOAPIC_EDGE_TRIG)

arch/x86/kvm/irq_comm.c +1 -1

index 7d083f71fc8e..9d711c2451c7 100644
@@ -121,7 +121,7 @@ void kvm_set_msi_irq(struct kvm *kvm, struct kvm_kernel_irq_routing_entry *e,
 	irq->msi_redir_hint = ((e->msi.address_lo
 		& MSI_ADDR_REDIRECTION_LOWPRI) > 0);
 	irq->level = 1;
-	irq->shorthand = 0;
+	irq->shorthand = APIC_DEST_NOSHORT;
 }
 EXPORT_SYMBOL_GPL(kvm_set_msi_irq);

arch/x86/kvm/x86.c +0 -0

index 3b00d662dc14..f6d778436e15 100644

arch/x86/kvm/x86.c static void kvm_pv_kick_cpu_op(struct kvm *kvm, unsigned long flags, int apicid) · line 7361

Quoted 8 lines

arch/x86/kvm/x86.c +1 -1

@@ -7355,7 +7355,7 @@ static void kvm_pv_kick_cpu_op(struct kvm *kvm, unsigned long flags, int apicid)
 {
 	struct kvm_lapic_irq lapic_irq;
 
-	lapic_irq.shorthand = 0;
+	lapic_irq.shorthand = APIC_DEST_NOSHORT;
 	lapic_irq.dest_mode = APIC_DEST_PHYSICAL;
 	lapic_irq.level = 0;
 	lapic_irq.dest_id = apicid;

Reviewed-by Vitaly Kuznetsov <[email protected]>

Quoted 3 lines

And, while on it, and if you're not yet tired I just noticed that kvm_apic_match_dest()'s parameter is called 'short_hand' while everywhere else we use 'shorthand'. Value the consistency we should :-)

I'll aquash your suggested change into previous patch ("KVM: X86: Fix callers of kvm_apic_match_dest() to use correct macros") and keep your r-b for that, otherwise I'm afraid the change could be too trivial to stand as itself..

Thanks,

Signature · 1 line

Peter Xu