Skip to contentSkip to conversation

irqchip/gic-v4: GICv4.1 architecture support

PATCH Superseded Marc Zyngier 3 participants 62 messages Sep 23, 2019 – Sep 30, 2019

Subsystems (5)

v1 of 6 Superseded Latest is v6 →

lore
Share & tools
lore

Marc Zyngier

author maintainer
#0 lore raw

cover irqchip/gic-v4: GICv4.1 architecture support

[Apologies for posting this during the merge window, I'm fighting against
another deadline which is incompatible with the -rc1 time frame]

This rather long series expands the existing GICv4 support to deal with the new GICv4.1 architecture, which comes with a set of major improvements compared to v4.0:

- One architectural doorbell per vcpu, instead of one doorbell per VLPI

- Doorbell entirely managed by the HW, with an "at most once" delivery guarantee per non-residency phase and only when requested by the hypervisor

- A shared memory scheme between ITSs and redistributors, allowing for an optimised residency sequence (the use of VMOVP becomes less frequent)

- Support for direct virtual SGI delivery (the injection path still involves the hypervisor), at the cost of losing the active state on SGIs. It shouldn't be a big deal, but some guest operating systems might notice (Linux definitely won't care)

On the other hand, public documentation is not available yet, so that's a bit annoying..

The series is roughly organised in 5 parts:

(1) A bunch of reworks to make the checking of some features more straightforward,
(2) VPE table allocation, new flavours of VMAPP/VMOVP commands
(3) v4.1 doorbell management
(4) Virtual SGI support
(5) Plumbing of virtual SGIs in KVM

Note that patch #1 has been previously posted, and forms the base of this work. This has had *very little* testing on a fairly early model, and some behaviours are unexpected (the VSGI feature clearly misbehaves). But it is in a shape that should make it possible to review and even debug. Consider this an opportunity to influence the shape of this code at an early stage! ;-)

Marc Zyngier (35):
KVM: arm64: vgic-v4: Move the GICv4 residency flow to be driven by
vcpu_load/put
irqchip/gic-v3-its: Factor out wait_for_syncr primitive irqchip/gic-v3-its: Allow LPI invalidation via the DirectLPI interface irqchip/gic-v3: Detect GICv4.1 supporting RVPEID irqchip/gic-v3: Add GICv4.1 VPEID size discovery irqchip/gic-v3-its: Make is_v4 use a TYPER copy irqchip/gic-v3-its: Kill its->ite_size and use TYPER copy instead irqchip/gic-v3-its: Kill its->device_ids and use TYPER copy instead irqchip/gic-v3-its: Add get_vlpi_map() helper irqchip/gic-v4.1: VPE table (aka GICR_VPROPBASER) allocation irqchip/gic-v4.1: Implement the v4.1 flavour of VMAPP irqchip/gic-v4.1: Don't use the VPE proxy if RVPEID is set irqchip/gic-v4.1: Implement the v4.1 flavour of VMOVP irqchip/gic-v4.1: Plumb skeletal VPE irqchip irqchip/gic-v4.1: Add mask/unmask doorbell callbacks irqchip/gic-v4.1: Add VPE residency callback irqchip/gic-v4.1: Add VPE eviction callback irqchip/gic-v4.1: Add VPE INVALL callback irqchip/gic-v4.1: Suppress per-VLPI doorbell irqchip/gic-v4.1: Allow direct invalidation of VLPIs irqchip/gic-v4.1: Advertise support v4.1 to KVM irqchip/gic-v4.1: Map the ITS SGIR register page irqchip/gic-v4.1: Plumb skeletal VSGI irqchip irqchip/gic-v4.1: Add initial SGI configuration irqchip/gic-v4.1: Plumb mask/unmask SGI callbacks irqchip/gic-v4.1: Plumb get/set_irqchip_state SGI callbacks irqchip/gic-v4.1: Plumb set_vcpu_affinity SGI callbacks irqchip/gic-v4.1: Move doorbell management to the GICv4 abstraction
layer
irqchip/gic-v4.1: Add VSGI allocation/teardown irqchip/gic-v4.1: Add VSGI property setup irqchip/gic-v4.1: Eagerly vmap vPEs
KVM: arm64: GICv4.1: Let doorbells be auto-enabled
KVM: arm64: GICv4.1: Add direct injection capability to SGI registers
KVM: arm64: GICv4.1: Configure SGIs as HW interrupts
KVM: arm64: GICv4.1: Expose HW-based SGIs in debugfs

 arch/arm/include/asm/arch_gicv3.h      |    2 +
 arch/arm64/include/asm/arch_gicv3.h    |    1 +
 drivers/irqchip/irq-gic-v3-its.c       | 1099 +++++++++++++++++++++---
 drivers/irqchip/irq-gic-v3.c           |   25 +-
 drivers/irqchip/irq-gic-v4.c           |  143 ++-
 include/kvm/arm_vgic.h                 |    5 +-
 include/linux/irqchip/arm-gic-common.h |    2 +
 include/linux/irqchip/arm-gic-v3.h     |   75 +-
 include/linux/irqchip/arm-gic-v4.h     |   45 +-
 virt/kvm/arm/arm.c                     |   12 +-
 virt/kvm/arm/vgic/vgic-debug.c         |   14 +-
 virt/kvm/arm/vgic/vgic-mmio-v3.c       |   15 +-
 virt/kvm/arm/vgic/vgic-mmio.c          |   88 +-
 virt/kvm/arm/vgic/vgic-v3.c            |    5 +
 virt/kvm/arm/vgic/vgic-v4.c            |  106 ++-
 virt/kvm/arm/vgic/vgic.c               |    4 -
 virt/kvm/arm/vgic/vgic.h               |    2 -
 17 files changed, 1466 insertions(+), 177 deletions(-)
Signature · 1 line
2.20.1
author maintainer
#1 lore raw

PATCH 1/35 KVM: arm64: vgic-v4: Move the GICv4 residency flow to be driven by vcpu_load/put

When the VHE code was reworked, a lot of the vgic stuff was moved around, but the GICv4 residency code did stay untouched, meaning that we come in and out of residency on each flush/sync, which is obviously suboptimal.

To address this, let's move things around a bit:

- Residency entry (flush) moves to vcpu_load
- Residency exit (sync) moves to vcpu_put
- On blocking (entry to WFI), we "put"
- On unblocking (exit from WFI, we "load"

Because these can nest (load/block/put/load/unblock/put, for example), we now have per-VPE tracking of the residency state.

Additionally, vgic_v4_put gains a "need doorbell" parameter, which only gets set to true when blocking because of a WFI. This allows a finer control of the doorbell, which now also gets disabled as soon as it gets signaled.

---
 drivers/irqchip/irq-gic-v4.c       |  7 +++-
 include/kvm/arm_vgic.h             |  4 +--
 include/linux/irqchip/arm-gic-v4.h |  2 ++
 virt/kvm/arm/arm.c                 | 12 ++++---
 virt/kvm/arm/vgic/vgic-v3.c        |  4 +++
 virt/kvm/arm/vgic/vgic-v4.c        | 55 ++++++++++++++----------------
 virt/kvm/arm/vgic/vgic.c           |  4 ---
 virt/kvm/arm/vgic/vgic.h           |  2 --
 8 files changed, 48 insertions(+), 42 deletions(-)
drivers/irqchip/irq-gic-v4.c +6 -1
index 563e87ed0766..45969927cc81 100644
@@ -141,12 +141,17 @@ static int its_send_vpe_cmd(struct its_vpe *vpe, struct its_cmd_info *info)
 int its_schedule_vpe(struct its_vpe *vpe, bool on)
 {
 	struct its_cmd_info info;
+	int ret;
 
 	WARN_ON(preemptible());
 
 	info.cmd_type = on ? SCHEDULE_VPE : DESCHEDULE_VPE;
 
-	return its_send_vpe_cmd(vpe, &info);
+	ret = its_send_vpe_cmd(vpe, &info);
+	if (!ret)
+		vpe->resident = on;
+
+	return ret;
 }
 
 int its_invall_vpe(struct its_vpe *vpe)
include/kvm/arm_vgic.h +2 -2
index af4f09c02bf1..4dc58d7a0010 100644
@@ -396,7 +396,7 @@ int kvm_vgic_v4_set_forwarding(struct kvm *kvm, int irq,
 int kvm_vgic_v4_unset_forwarding(struct kvm *kvm, int irq,
 				 struct kvm_kernel_irq_routing_entry *irq_entry);
 
-void kvm_vgic_v4_enable_doorbell(struct kvm_vcpu *vcpu);
-void kvm_vgic_v4_disable_doorbell(struct kvm_vcpu *vcpu);
+int vgic_v4_load(struct kvm_vcpu *vcpu);
+int vgic_v4_put(struct kvm_vcpu *vcpu, bool need_db);
 
 #endif /* __KVM_ARM_VGIC_H */
include/linux/irqchip/arm-gic-v4.h +2 -0
index e6b155713b47..ab1396afe08a 100644
@@ -35,6 +35,8 @@ struct its_vpe {
 	/* Doorbell interrupt */
 	int			irq;
 	irq_hw_number_t		vpe_db_lpi;
+	/* VPE resident */
+	bool			resident;
 	/* VPE proxy mapping */
 	int			vpe_proxy_event;
 	/*
virt/kvm/arm/arm.c +8 -4
index 86c6aa1cb58e..bd2afcf9a13f 100644
@@ -322,20 +322,24 @@ void kvm_arch_vcpu_blocking(struct kvm_vcpu *vcpu)
 	/*
 	 * If we're about to block (most likely because we've just hit a
 	 * WFI), we need to sync back the state of the GIC CPU interface
-	 * so that we have the lastest PMR and group enables. This ensures
+	 * so that we have the latest PMR and group enables. This ensures
 	 * that kvm_arch_vcpu_runnable has up-to-date data to decide
 	 * whether we have pending interrupts.
+	 *
+	 * For the same reason, we want to tell GICv4 that we need
+	 * doorbells to be signalled, should an interrupt become pending.
 	 */
 	preempt_disable();
 	kvm_vgic_vmcr_sync(vcpu);
+	vgic_v4_put(vcpu, true);
 	preempt_enable();
-
-	kvm_vgic_v4_enable_doorbell(vcpu);
 }
 
 void kvm_arch_vcpu_unblocking(struct kvm_vcpu *vcpu)
 {
-	kvm_vgic_v4_disable_doorbell(vcpu);
+	preempt_disable();
+	vgic_v4_load(vcpu);
+	preempt_enable();
 }
 
 int kvm_arch_vcpu_init(struct kvm_vcpu *vcpu)
virt/kvm/arm/vgic/vgic-v3.c +4 -0
index 8d69f007dd0c..48307a9eb1d8 100644
@@ -664,6 +664,8 @@ void vgic_v3_load(struct kvm_vcpu *vcpu)
 
 	if (has_vhe())
 		__vgic_v3_activate_traps(vcpu);
+
+	WARN_ON(vgic_v4_load(vcpu));
 }
 
 void vgic_v3_vmcr_sync(struct kvm_vcpu *vcpu)
@@ -676,6 +678,8 @@ void vgic_v3_vmcr_sync(struct kvm_vcpu *vcpu)
 
 void vgic_v3_put(struct kvm_vcpu *vcpu)
 {
+	WARN_ON(vgic_v4_put(vcpu, false));
+
 	vgic_v3_vmcr_sync(vcpu);
 
 	kvm_call_hyp(__vgic_v3_save_aprs, vcpu);
virt/kvm/arm/vgic/vgic-v4.c +26 -29

93 lines not on this page · raw message

virt/kvm/arm/vgic/vgic.c +0 -4
index 45a870cb63f5..99b02ca730a8 100644
@@ -857,8 +857,6 @@ void kvm_vgic_sync_hwstate(struct kvm_vcpu *vcpu)
 {
 	struct vgic_cpu *vgic_cpu = &vcpu->arch.vgic_cpu;
 
-	WARN_ON(vgic_v4_sync_hwstate(vcpu));
-
 	/* An empty ap_list_head implies used_lrs == 0 */
 	if (list_empty(&vcpu->arch.vgic_cpu.ap_list_head))
 		return;
@@ -882,8 +880,6 @@ static inline void vgic_restore_state(struct kvm_vcpu *vcpu)
 /* Flush our emulation state into the GIC hardware before entering the guest. */
 void kvm_vgic_flush_hwstate(struct kvm_vcpu *vcpu)
 {
-	WARN_ON(vgic_v4_flush_hwstate(vcpu));
-
 	/*
 	 * If there are no virtual interrupts active or pending for this
 	 * VCPU, then there is no work to do and we can bail out without
virt/kvm/arm/vgic/vgic.h +0 -2
index 83066a81b16a..c7fefd6b1c80 100644
@@ -316,7 +316,5 @@ void vgic_its_invalidate_cache(struct kvm *kvm);
 bool vgic_supports_direct_msis(struct kvm *kvm);
 int vgic_v4_init(struct kvm *kvm);
 void vgic_v4_teardown(struct kvm *kvm);
-int vgic_v4_sync_hwstate(struct kvm_vcpu *vcpu);
-int vgic_v4_flush_hwstate(struct kvm_vcpu *vcpu);
 
 #endif
Signature · 1 line
2.20.1
author maintainer
#2 lore raw

PATCH 2/35 irqchip/gic-v3-its: Factor out wait_for_syncr primitive

In reply to Marc Zyngier

Waiting for a redistributor to have performed an operation is a common thing to do, and the idiom is already spread around. As we're going to make even more use of this, let's have a primitive that does just that.

---
 drivers/irqchip/irq-gic-v3-its.c | 15 +++++++++------
 1 file changed, 9 insertions(+), 6 deletions(-)
drivers/irqchip/irq-gic-v3-its.c +9 -6
index 62e54f1a248b..58cb233cf138 100644
@@ -1075,6 +1075,12 @@ static void lpi_write_config(struct irq_data *d, u8 clr, u8 set)
 		dsb(ishst);
 }
 
+static void wait_for_syncr(void __iomem *rdbase)
+{
+	while (gic_read_lpir(rdbase + GICR_SYNCR) & 1)
+		cpu_relax();
+}
+
 static void lpi_update_config(struct irq_data *d, u8 clr, u8 set)
 {
 	struct its_device *its_dev = irq_data_get_irq_chip_data(d);
@@ -2757,8 +2763,7 @@ static void its_vpe_db_proxy_move(struct its_vpe *vpe, int from, int to)
 
 		rdbase = per_cpu_ptr(gic_rdists->rdist, from)->rd_base;
 		gic_write_lpir(vpe->vpe_db_lpi, rdbase + GICR_CLRLPIR);
-		while (gic_read_lpir(rdbase + GICR_SYNCR) & 1)
-			cpu_relax();
+		wait_for_syncr(rdbase);
 
 		return;
 	}
@@ -2914,8 +2919,7 @@ static void its_vpe_send_inv(struct irq_data *d)
 
 		rdbase = per_cpu_ptr(gic_rdists->rdist, vpe->col_idx)->rd_base;
 		gic_write_lpir(vpe->vpe_db_lpi, rdbase + GICR_INVLPIR);
-		while (gic_read_lpir(rdbase + GICR_SYNCR) & 1)
-			cpu_relax();
+		wait_for_syncr(rdbase);
 	} else {
 		its_vpe_send_cmd(vpe, its_send_inv);
 	}
@@ -2957,8 +2961,7 @@ static int its_vpe_set_irqchip_state(struct irq_data *d,
 			gic_write_lpir(vpe->vpe_db_lpi, rdbase + GICR_SETLPIR);
 		} else {
 			gic_write_lpir(vpe->vpe_db_lpi, rdbase + GICR_CLRLPIR);
-			while (gic_read_lpir(rdbase + GICR_SYNCR) & 1)
-				cpu_relax();
+			wait_for_syncr(rdbase);
 		}
 	} else {
 		if (state)
Signature · 1 line
2.20.1
author maintainer
#4 lore raw

PATCH 3/35 irqchip/gic-v3-its: Allow LPI invalidation via the DirectLPI interface

In reply to Marc Zyngier

We currently don't make much use of the DirectLPI feature, and it would be beneficial to do this more, if only because it becomes a mandatory feature for GICv4.1.

---
 drivers/irqchip/irq-gic-v3-its.c | 51 +++++++++++++++++++++++---------
 1 file changed, 37 insertions(+), 14 deletions(-)
drivers/irqchip/irq-gic-v3-its.c +37 -14
index 58cb233cf138..c94eb287393b 100644
@@ -175,6 +175,12 @@ static DEFINE_IDA(its_vpeid_ida);
 #define gic_data_rdist_rd_base()	(gic_data_rdist()->rd_base)
 #define gic_data_rdist_vlpi_base()	(gic_data_rdist_rd_base() + SZ_128K)
 
+static inline u32 its_get_event_id(struct irq_data *d)
+{
+	struct its_device *its_dev = irq_data_get_irq_chip_data(d);
+	return d->hwirq - its_dev->event_map.lpi_base;
+}
+
 static struct its_collection *dev_event_to_col(struct its_device *its_dev,
 					       u32 event)
 {
@@ -183,6 +189,13 @@ static struct its_collection *dev_event_to_col(struct its_device *its_dev,
 	return its->collections + its_dev->event_map.col_map[event];
 }
 
+static struct its_collection *irq_to_col(struct irq_data *d)
+{
+	struct its_device *its_dev = irq_data_get_irq_chip_data(d);
+
+	return dev_event_to_col(its_dev, its_get_event_id(d));
+}
+
 static struct its_collection *valid_col(struct its_collection *col)
 {
 	if (WARN_ON_ONCE(col->target_address & GENMASK_ULL(15, 0)))
@@ -1031,12 +1044,6 @@ static void its_send_vinvall(struct its_node *its, struct its_vpe *vpe)
  * irqchip functions - assumes MSI, mostly.
  */
 
-static inline u32 its_get_event_id(struct irq_data *d)
-{
-	struct its_device *its_dev = irq_data_get_irq_chip_data(d);
-	return d->hwirq - its_dev->event_map.lpi_base;
-}
-
 static void lpi_write_config(struct irq_data *d, u8 clr, u8 set)
 {
 	irq_hw_number_t hwirq;
@@ -1081,12 +1088,28 @@ static void wait_for_syncr(void __iomem *rdbase)
 		cpu_relax();
 }
 
+static void direct_lpi_inv(struct irq_data *d)
+{
+	struct its_collection *col;
+	void __iomem *rdbase;
+
+	/* Target the redistributor this LPI is currently routed to */
+	col = irq_to_col(d);
+	rdbase = per_cpu_ptr(gic_rdists->rdist, col->col_id)->rd_base;
+	gic_write_lpir(d->hwirq, rdbase + GICR_INVLPIR);
+
+	wait_for_syncr(rdbase);
+}
+
 static void lpi_update_config(struct irq_data *d, u8 clr, u8 set)
 {
 	struct its_device *its_dev = irq_data_get_irq_chip_data(d);
 
 	lpi_write_config(d, clr, set);
-	its_send_inv(its_dev, its_get_event_id(d));
+	if (gic_rdists->has_direct_lpi && !irqd_is_forwarded_to_vcpu(d))
+		direct_lpi_inv(d);
+	else
+		its_send_inv(its_dev, its_get_event_id(d));
 }
 
 static void its_vlpi_set_doorbell(struct irq_data *d, bool enable)
@@ -2912,15 +2935,15 @@ static void its_vpe_send_cmd(struct its_vpe *vpe,
 
 static void its_vpe_send_inv(struct irq_data *d)
 {
-	struct its_vpe *vpe = irq_data_get_irq_chip_data(d);
-
 	if (gic_rdists->has_direct_lpi) {
-		void __iomem *rdbase;
-
-		rdbase = per_cpu_ptr(gic_rdists->rdist, vpe->col_idx)->rd_base;
-		gic_write_lpir(vpe->vpe_db_lpi, rdbase + GICR_INVLPIR);
-		wait_for_syncr(rdbase);
+		/*
+		 * Don't mess about. Generating the invalidation is easily
+		 * done by using the parent irq_data, just like below.
+		 */
+		direct_lpi_inv(d->parent_data);
 	} else {
+		struct its_vpe *vpe = irq_data_get_irq_chip_data(d);
+
 		its_vpe_send_cmd(vpe, its_send_inv);
 	}
 }
Signature · 1 line
2.20.1
author maintainer
#9 lore raw

PATCH 4/35 irqchip/gic-v3: Detect GICv4.1 supporting RVPEID

In reply to Marc Zyngier

GICv4.1 supports the RVPEID ("Residency per vPE ID"), which allows for a much efficient way of making virtual CPUs resident (to allow direct injection of interrupts).

The functionnality needs to be discovered on each and every redistributor in the system, and disabled if the settings are inconsistent.

---
 drivers/irqchip/irq-gic-v3.c       | 21 ++++++++++++++++++---
 include/linux/irqchip/arm-gic-v3.h |  2 ++
 2 files changed, 20 insertions(+), 3 deletions(-)
drivers/irqchip/irq-gic-v3.c +18 -3
index 422664ac5f53..0b545e2c3498 100644
@@ -849,8 +849,21 @@ static int __gic_update_rdist_properties(struct redist_region *region,
 					 void __iomem *ptr)
 {
 	u64 typer = gic_read_typer(ptr + GICR_TYPER);
+
 	gic_data.rdists.has_vlpis &= !!(typer & GICR_TYPER_VLPIS);
-	gic_data.rdists.has_direct_lpi &= !!(typer & GICR_TYPER_DirectLPIS);
+
+	/* RVPEID implies some form of DirectLPI, no matter what the doc says... :-/ */
+	gic_data.rdists.has_rvpeid &= !!(typer & GICR_TYPER_RVPEID);
+	gic_data.rdists.has_direct_lpi &= (!!(typer & GICR_TYPER_DirectLPIS) |
+					   gic_data.rdists.has_rvpeid);
+
+	/* Detect non-sensical configurations */
+	if (WARN_ON_ONCE(gic_data.rdists.has_rvpeid && !gic_data.rdists.has_vlpis)) {
+		gic_data.rdists.has_direct_lpi = false;
+		gic_data.rdists.has_vlpis = false;
+		gic_data.rdists.has_rvpeid = false;
+	}
+
 	gic_data.ppi_nr = min(GICR_TYPER_NR_PPIS(typer), gic_data.ppi_nr);
 
 	return 1;
@@ -863,9 +876,10 @@ static void gic_update_rdist_properties(void)
 	if (WARN_ON(gic_data.ppi_nr == UINT_MAX))
 		gic_data.ppi_nr = 0;
 	pr_info("%d PPIs implemented\n", gic_data.ppi_nr);
-	pr_info("%sVLPI support, %sdirect LPI support\n",
+	pr_info("%sVLPI support, %sdirect LPI support, %sRVPEID support\n",
 		!gic_data.rdists.has_vlpis ? "no " : "",
-		!gic_data.rdists.has_direct_lpi ? "no " : "");
+		!gic_data.rdists.has_direct_lpi ? "no " : "",
+		!gic_data.rdists.has_rvpeid ? "no " : "");
 }
 
 /* Check whether it's single security state view */
@@ -1546,6 +1560,7 @@ static int __init gic_init_bases(void __iomem *dist_base,
 						 &gic_data);
 	irq_domain_update_bus_token(gic_data.domain, DOMAIN_BUS_WIRED);
 	gic_data.rdists.rdist = alloc_percpu(typeof(*gic_data.rdists.rdist));
+	gic_data.rdists.has_rvpeid = true;
 	gic_data.rdists.has_vlpis = true;
 	gic_data.rdists.has_direct_lpi = true;
 
include/linux/irqchip/arm-gic-v3.h +2 -0
index 5cc10cf7cb3e..b34e0c113697 100644
@@ -234,6 +234,7 @@
 #define GICR_TYPER_VLPIS		(1U << 1)
 #define GICR_TYPER_DirectLPIS		(1U << 3)
 #define GICR_TYPER_LAST			(1U << 4)
+#define GICR_TYPER_RVPEID		(1U << 7)
 
 #define GIC_V3_REDIST_SIZE		0x20000
 
@@ -613,6 +614,7 @@ struct rdists {
 	u64			flags;
 	u32			gicd_typer;
 	bool			has_vlpis;
+	bool			has_rvpeid;
 	bool			has_direct_lpi;
 };
 
Signature · 1 line
2.20.1
author maintainer
#14 lore raw

PATCH 5/35 irqchip/gic-v3: Add GICv4.1 VPEID size discovery

In reply to Marc Zyngier

While GICv4.0 mandates 16 bit worth of VPEIDs, GICv4.1 allows smaller implementations to be built. Add the required glue to dynamically compute the limit.

---
 drivers/irqchip/irq-gic-v3-its.c   | 11 ++++++++++-
 drivers/irqchip/irq-gic-v3.c       |  3 +++
 include/linux/irqchip/arm-gic-v3.h |  5 +++++
 3 files changed, 18 insertions(+), 1 deletion(-)
drivers/irqchip/irq-gic-v3-its.c +10 -1
index c94eb287393b..17b77a0b9d97 100644
@@ -119,7 +119,16 @@ struct its_node {
 #define ITS_ITT_ALIGN		SZ_256
 
 /* The maximum number of VPEID bits supported by VLPI commands */
-#define ITS_MAX_VPEID_BITS	(16)
+#define ITS_MAX_VPEID_BITS						\
+	({								\
+		int nvpeid = 16;					\
+		if (gic_rdists->has_rvpeid &&				\
+		    gic_rdists->gicd_typer2 & GICD_TYPER2_VIL)		\
+			nvpeid = 1 + (gic_rdists->gicd_typer2 &		\
+				      GICD_TYPER2_VID);			\
+									\
+		nvpeid;							\
+	})
 #define ITS_MAX_VPEID		(1 << (ITS_MAX_VPEID_BITS))
 
 /* Convert page order to size in bytes */
drivers/irqchip/irq-gic-v3.c +3 -0
index 0b545e2c3498..fb6360161d6c 100644
@@ -1556,6 +1556,9 @@ static int __init gic_init_bases(void __iomem *dist_base,
 
 	pr_info("%d SPIs implemented\n", GIC_LINE_NR - 32);
 	pr_info("%d Extended SPIs implemented\n", GIC_ESPI_NR);
+
+	gic_data.rdists.gicd_typer2 = readl_relaxed(gic_data.dist_base + GICD_TYPER2);
+
 	gic_data.domain = irq_domain_create_tree(handle, &gic_irq_domain_ops,
 						 &gic_data);
 	irq_domain_update_bus_token(gic_data.domain, DOMAIN_BUS_WIRED);
include/linux/irqchip/arm-gic-v3.h +5 -0
index b34e0c113697..71730b9def0c 100644
@@ -13,6 +13,7 @@
 #define GICD_CTLR			0x0000
 #define GICD_TYPER			0x0004
 #define GICD_IIDR			0x0008
+#define GICD_TYPER2			0x000C
 #define GICD_STATUSR			0x0010
 #define GICD_SETSPI_NSR			0x0040
 #define GICD_CLRSPI_NSR			0x0048
@@ -89,6 +90,9 @@
 #define GICD_TYPER_ESPIS(typer)						\
 	(((typer) & GICD_TYPER_ESPI) ? GICD_TYPER_SPIS((typer) >> 27) : 0)
 
+#define GICD_TYPER2_VIL			(1U << 7)
+#define GICD_TYPER2_VID			GENMASK(4, 0)
+
 #define GICD_IROUTER_SPI_MODE_ONE	(0U << 31)
 #define GICD_IROUTER_SPI_MODE_ANY	(1U << 31)
 
@@ -613,6 +617,7 @@ struct rdists {
 	void			*prop_table_va;
 	u64			flags;
 	u32			gicd_typer;
+	u32			gicd_typer2;
 	bool			has_vlpis;
 	bool			has_rvpeid;
 	bool			has_direct_lpi;
Signature · 6 lines
2.20.1

_______________________________________________
kvmarm mailing list
[email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
author maintainer
#17 lore raw

PATCH 6/35 irqchip/gic-v3-its: Make is_v4 use a TYPER copy

In reply to Marc Zyngier

Instead of caching the GICv4 compatibility in a discrete way, cache the TYPER register instead, which can then be used to implement the same functionnality. This will get used more extensively in subsequent patches.

---
 drivers/irqchip/irq-gic-v3-its.c | 24 +++++++++++++-----------
 1 file changed, 13 insertions(+), 11 deletions(-)
drivers/irqchip/irq-gic-v3-its.c +13 -11
index 17b77a0b9d97..731726540efa 100644
@@ -102,6 +102,7 @@ struct its_node {
 	struct its_collection	*collections;
 	struct fwnode_handle	*fwnode_handle;
 	u64			(*get_msi_base)(struct its_device *its_dev);
+	u64			typer;
 	u64			cbaser_save;
 	u32			ctlr_save;
 	struct list_head	its_device_list;
@@ -112,10 +113,11 @@ struct its_node {
 	int			numa_node;
 	unsigned int		msi_domain_flags;
 	u32			pre_its_base; /* for Socionext Synquacer */
-	bool			is_v4;
 	int			vlpi_redist_offset;
 };
 
+#define is_v4(its)		(!!((its)->typer & GITS_TYPER_VLPIS))
+
 #define ITS_ITT_ALIGN		SZ_256
 
 /* The maximum number of VPEID bits supported by VLPI commands */
@@ -1028,7 +1030,7 @@ static void its_send_vmovp(struct its_vpe *vpe)
 
 	/* Emit VMOVPs */
 	list_for_each_entry(its, &its_nodes, entry) {
-		if (!its->is_v4)
+		if (!is_v4(its))
 			continue;
 
 		if (!vpe->its_vm->vlpi_count[its->list_nr])
@@ -1439,7 +1441,7 @@ static int its_irq_set_vcpu_affinity(struct irq_data *d, void *vcpu_info)
 	struct its_cmd_info *info = vcpu_info;
 
 	/* Need a v4 ITS */
-	if (!its_dev->its->is_v4)
+	if (!is_v4(its_dev->its))
 		return -EINVAL;
 
 	/* Unmap request? */
@@ -2403,7 +2405,7 @@ static bool its_alloc_vpe_table(u32 vpe_id)
 	list_for_each_entry(its, &its_nodes, entry) {
 		struct its_baser *baser;
 
-		if (!its->is_v4)
+		if (!is_v4(its))
 			continue;
 
 		baser = its_get_baser(its, GITS_BASER_TYPE_VCPU);
@@ -2891,7 +2893,7 @@ static void its_vpe_invall(struct its_vpe *vpe)
 	struct its_node *its;
 
 	list_for_each_entry(its, &its_nodes, entry) {
-		if (!its->is_v4)
+		if (!is_v4(its))
 			continue;
 
 		if (its_list_map && !vpe->its_vm->vlpi_count[its->list_nr])
@@ -3158,7 +3160,7 @@ static int its_vpe_irq_domain_activate(struct irq_domain *domain,
 	vpe->col_idx = cpumask_first(cpu_online_mask);
 
 	list_for_each_entry(its, &its_nodes, entry) {
-		if (!its->is_v4)
+		if (!is_v4(its))
 			continue;
 
 		its_send_vmapp(its, vpe, true);
@@ -3184,7 +3186,7 @@ static void its_vpe_irq_domain_deactivate(struct irq_domain *domain,
 		return;
 
 	list_for_each_entry(its, &its_nodes, entry) {
-		if (!its->is_v4)
+		if (!is_v4(its))
 			continue;
 
 		its_send_vmapp(its, vpe, false);
@@ -3622,12 +3624,12 @@ static int __init its_probe_one(struct resource *res,
 	INIT_LIST_HEAD(&its->entry);
 	INIT_LIST_HEAD(&its->its_device_list);
 	typer = gic_read_typer(its_base + GITS_TYPER);
+	its->typer = typer;
 	its->base = its_base;
 	its->phys_base = res->start;
 	its->ite_size = GITS_TYPER_ITT_ENTRY_SIZE(typer);
 	its->device_ids = GITS_TYPER_DEVBITS(typer);
-	its->is_v4 = !!(typer & GITS_TYPER_VLPIS);
-	if (its->is_v4) {
+	if (is_v4(its)) {
 		if (!(typer & GITS_TYPER_VMOVP)) {
 			err = its_compute_its_list_map(res, its_base);
 			if (err < 0)
@@ -3694,7 +3696,7 @@ static int __init its_probe_one(struct resource *res,
 	gits_write_cwriter(0, its->base + GITS_CWRITER);
 	ctlr = readl_relaxed(its->base + GITS_CTLR);
 	ctlr |= GITS_CTLR_ENABLE;
-	if (its->is_v4)
+	if (is_v4(its))
 		ctlr |= GITS_CTLR_ImDe;
 	writel_relaxed(ctlr, its->base + GITS_CTLR);
 
@@ -4019,7 +4021,7 @@ int __init its_init(struct fwnode_handle *handle, struct rdists *rdists,
 		return err;
 
 	list_for_each_entry(its, &its_nodes, entry)
-		has_v4 |= its->is_v4;
+		has_v4 |= is_v4(its);
 
 	if (has_v4 & rdists->has_vlpis) {
 		if (its_init_vpe_domain() ||
Signature · 1 line
2.20.1
author maintainer
#18 lore raw

PATCH 7/35 irqchip/gic-v3-its: Kill its->ite_size and use TYPER copy instead

In reply to Marc Zyngier

Now that we have a copy of TYPER in the ITS structure, rely on this to provide the same service as its->ite_size, which gets axed. Errata workarounds are now updating the cached fields instead of requiring a separate field in the ITS structure.

---
 drivers/irqchip/irq-gic-v3-its.c   | 8 ++++----
 include/linux/irqchip/arm-gic-v3.h | 2 +-
 2 files changed, 5 insertions(+), 5 deletions(-)
drivers/irqchip/irq-gic-v3-its.c +4 -4
index 731726540efa..94a0885a57b6 100644
@@ -6,6 +6,7 @@
 
 #include <linux/acpi.h>
 #include <linux/acpi_iort.h>
+#include <linux/bitfield.h>
 #include <linux/bitmap.h>
 #include <linux/cpu.h>
 #include <linux/crash_dump.h>
@@ -108,7 +109,6 @@ struct its_node {
 	struct list_head	its_device_list;
 	u64			flags;
 	unsigned long		list_nr;
-	u32			ite_size;
 	u32			device_ids;
 	int			numa_node;
 	unsigned int		msi_domain_flags;
@@ -2444,7 +2444,7 @@ static struct its_device *its_create_device(struct its_node *its, u32 dev_id,
 	 * sized as a power of two (and you need at least one bit...).
 	 */
 	nr_ites = max(2, nvecs);
-	sz = nr_ites * its->ite_size;
+	sz = nr_ites * (FIELD_GET(GITS_TYPER_ITT_ENTRY_SIZE, its->typer) + 1);
 	sz = max(sz, ITS_ITT_ALIGN) + ITS_ITT_ALIGN - 1;
 	itt = kzalloc_node(sz, GFP_KERNEL, its->numa_node);
 	if (alloc_lpis) {
@@ -3258,7 +3258,8 @@ static bool __maybe_unused its_enable_quirk_qdf2400_e0065(void *data)
 	struct its_node *its = data;
 
 	/* On QDF2400, the size of the ITE is 16Bytes */
-	its->ite_size = 16;
+	its->typer &= ~GITS_TYPER_ITT_ENTRY_SIZE;
+	its->typer |= FIELD_PREP(GITS_TYPER_ITT_ENTRY_SIZE, 16 - 1);
 
 	return true;
 }
@@ -3627,7 +3628,6 @@ static int __init its_probe_one(struct resource *res,
 	its->typer = typer;
 	its->base = its_base;
 	its->phys_base = res->start;
-	its->ite_size = GITS_TYPER_ITT_ENTRY_SIZE(typer);
 	its->device_ids = GITS_TYPER_DEVBITS(typer);
 	if (is_v4(its)) {
 		if (!(typer & GITS_TYPER_VMOVP)) {
include/linux/irqchip/arm-gic-v3.h +1 -1
index 71730b9def0c..df8994f23a00 100644
@@ -339,7 +339,7 @@
 #define GITS_TYPER_PLPIS		(1UL << 0)
 #define GITS_TYPER_VLPIS		(1UL << 1)
 #define GITS_TYPER_ITT_ENTRY_SIZE_SHIFT	4
-#define GITS_TYPER_ITT_ENTRY_SIZE(r)	((((r) >> GITS_TYPER_ITT_ENTRY_SIZE_SHIFT) & 0xf) + 1)
+#define GITS_TYPER_ITT_ENTRY_SIZE	GENMASK_ULL(7, 4)
 #define GITS_TYPER_IDBITS_SHIFT		8
 #define GITS_TYPER_DEVBITS_SHIFT	13
 #define GITS_TYPER_DEVBITS(r)		((((r) >> GITS_TYPER_DEVBITS_SHIFT) & 0x1f) + 1)
Signature · 1 line
2.20.1
author maintainer
#19 lore raw

PATCH 8/35 irqchip/gic-v3-its: Kill its->device_ids and use TYPER copy instead

In reply to Marc Zyngier

Now that we have a copy of TYPER in the ITS structure, rely on this to provide the same service as its->device_ids, which gets axed. Errata workarounds are now updating the cached fields instead of requiring a separate field in the ITS structure.

---
 drivers/irqchip/irq-gic-v3-its.c   | 24 +++++++++++++-----------
 include/linux/irqchip/arm-gic-v3.h |  2 +-
 2 files changed, 14 insertions(+), 12 deletions(-)
drivers/irqchip/irq-gic-v3-its.c +13 -11
index 94a0885a57b6..fbe7b0c14b9e 100644
@@ -109,7 +109,6 @@ struct its_node {
 	struct list_head	its_device_list;
 	u64			flags;
 	unsigned long		list_nr;
-	u32			device_ids;
 	int			numa_node;
 	unsigned int		msi_domain_flags;
 	u32			pre_its_base; /* for Socionext Synquacer */
@@ -117,6 +116,7 @@ struct its_node {
 };
 
 #define is_v4(its)		(!!((its)->typer & GITS_TYPER_VLPIS))
+#define device_ids(its)		(FIELD_GET(GITS_TYPER_DEVBITS, (its)->typer) + 1)
 
 #define ITS_ITT_ALIGN		SZ_256
 
@@ -1947,9 +1947,9 @@ static bool its_parse_indirect_baser(struct its_node *its,
 	if (new_order >= MAX_ORDER) {
 		new_order = MAX_ORDER - 1;
 		ids = ilog2(PAGE_ORDER_TO_SIZE(new_order) / (int)esz);
-		pr_warn("ITS@%pa: %s Table too large, reduce ids %u->%u\n",
+		pr_warn("ITS@%pa: %s Table too large, reduce ids %llu->%u\n",
 			&its->phys_base, its_base_type_string[type],
-			its->device_ids, ids);
+			device_ids(its), ids);
 	}
 
 	*order = new_order;
@@ -1995,7 +1995,7 @@ static int its_alloc_tables(struct its_node *its)
 		case GITS_BASER_TYPE_DEVICE:
 			indirect = its_parse_indirect_baser(its, baser,
 							    psz, &order,
-							    its->device_ids);
+							    device_ids(its));
 			break;
 
 		case GITS_BASER_TYPE_VCPU:
@@ -2386,7 +2386,7 @@ static bool its_alloc_device_table(struct its_node *its, u32 dev_id)
 
 	/* Don't allow device id that exceeds ITS hardware limit */
 	if (!baser)
-		return (ilog2(dev_id) < its->device_ids);
+		return (ilog2(dev_id) < device_ids(its));
 
 	return its_alloc_table_entry(its, baser, dev_id);
 }
@@ -3237,8 +3237,9 @@ static bool __maybe_unused its_enable_quirk_cavium_22375(void *data)
 {
 	struct its_node *its = data;
 
-	/* erratum 22375: only alloc 8MB table size */
-	its->device_ids = 0x14;		/* 20 bits, 8MB */
+	/* erratum 22375: only alloc 8MB table size (20 bits) */
+	its->typer &= ~GITS_TYPER_DEVBITS;
+	its->typer |= FIELD_PREP(GITS_TYPER_DEVBITS, 20 - 1);
 	its->flags |= ITS_FLAGS_WORKAROUND_CAVIUM_22375;
 
 	return true;
@@ -3293,8 +3294,10 @@ static bool __maybe_unused its_enable_quirk_socionext_synquacer(void *data)
 		its->get_msi_base = its_irq_get_msi_base_pre_its;
 
 		ids = ilog2(pre_its_window[1]) - 2;
-		if (its->device_ids > ids)
-			its->device_ids = ids;
+		if (device_ids(its) > ids) {
+			its->typer &= ~GITS_TYPER_DEVBITS;
+			its->typer |= FIELD_PREP(GITS_TYPER_DEVBITS, ids - 1);
+		}
 
 		/* the pre-ITS breaks isolation, so disable MSI remapping */
 		its->msi_domain_flags &= ~IRQ_DOMAIN_FLAG_MSI_REMAP;
@@ -3527,7 +3530,7 @@ static int its_init_vpe_domain(void)
 	}
 
 	/* Use the last possible DevID */
-	devid = GENMASK(its->device_ids - 1, 0);
+	devid = GENMASK(device_ids(its) - 1, 0);
 	vpe_proxy.dev = its_create_device(its, devid, entries, false);
 	if (!vpe_proxy.dev) {
 		kfree(vpe_proxy.vpes);
@@ -3628,7 +3631,6 @@ static int __init its_probe_one(struct resource *res,
 	its->typer = typer;
 	its->base = its_base;
 	its->phys_base = res->start;
-	its->device_ids = GITS_TYPER_DEVBITS(typer);
 	if (is_v4(its)) {
 		if (!(typer & GITS_TYPER_VMOVP)) {
 			err = its_compute_its_list_map(res, its_base);
include/linux/irqchip/arm-gic-v3.h +1 -1
index df8994f23a00..8c6be56da7e9 100644
@@ -342,7 +342,7 @@
 #define GITS_TYPER_ITT_ENTRY_SIZE	GENMASK_ULL(7, 4)
 #define GITS_TYPER_IDBITS_SHIFT		8
 #define GITS_TYPER_DEVBITS_SHIFT	13
-#define GITS_TYPER_DEVBITS(r)		((((r) >> GITS_TYPER_DEVBITS_SHIFT) & 0x1f) + 1)
+#define GITS_TYPER_DEVBITS		GENMASK_ULL(17, 13)
 #define GITS_TYPER_PTA			(1UL << 19)
 #define GITS_TYPER_HCC_SHIFT		24
 #define GITS_TYPER_HCC(r)		(((r) >> GITS_TYPER_HCC_SHIFT) & 0xff)
Signature · 1 line
2.20.1
author maintainer
#20 lore raw

PATCH 9/35 irqchip/gic-v3-its: Add get_vlpi_map() helper

In reply to Marc Zyngier

Obtaining the mapping information for a VLPI is something quite common, and the GICv4.1 code is going to make even more use of it. Expose it as a separate helper.

---
 drivers/irqchip/irq-gic-v3-its.c | 27 ++++++++++++++++-----------
 1 file changed, 16 insertions(+), 11 deletions(-)
drivers/irqchip/irq-gic-v3-its.c +16 -11
index fbe7b0c14b9e..9bad3887f6ce 100644
@@ -1054,20 +1054,26 @@ static void its_send_vinvall(struct its_node *its, struct its_vpe *vpe)
 /*
  * irqchip functions - assumes MSI, mostly.
  */
+static struct its_vlpi_map *get_vlpi_map(struct irq_data *d)
+{
+	struct its_device *its_dev = irq_data_get_irq_chip_data(d);
+	u32 event = its_get_event_id(d);
+
+	if (!irqd_is_forwarded_to_vcpu(d))
+		return NULL;
+
+	return &its_dev->event_map.vlpi_maps[event];
+}
 
 static void lpi_write_config(struct irq_data *d, u8 clr, u8 set)
 {
+	struct its_vlpi_map *map = get_vlpi_map(d);
 	irq_hw_number_t hwirq;
 	void *va;
 	u8 *cfg;
 
-	if (irqd_is_forwarded_to_vcpu(d)) {
-		struct its_device *its_dev = irq_data_get_irq_chip_data(d);
-		u32 event = its_get_event_id(d);
-		struct its_vlpi_map *map;
-
-		va = page_address(its_dev->event_map.vm->vprop_page);
-		map = &its_dev->event_map.vlpi_maps[event];
+	if (map) {
+		va = page_address(map->vm->vprop_page);
 		hwirq = map->vintid;
 
 		/* Remember the updated property */
@@ -1360,19 +1366,18 @@ static int its_vlpi_map(struct irq_data *d, struct its_cmd_info *info)
 static int its_vlpi_get(struct irq_data *d, struct its_cmd_info *info)
 {
 	struct its_device *its_dev = irq_data_get_irq_chip_data(d);
-	u32 event = its_get_event_id(d);
+	struct its_vlpi_map *map = get_vlpi_map(d);
 	int ret = 0;
 
 	mutex_lock(&its_dev->event_map.vlpi_lock);
 
-	if (!its_dev->event_map.vm ||
-	    !its_dev->event_map.vlpi_maps[event].vm) {
+	if (!its_dev->event_map.vm || !map->vm) {
 		ret = -EINVAL;
 		goto out;
 	}
 
 	/* Copy our mapping information to the incoming request */
-	*info->map = its_dev->event_map.vlpi_maps[event];
+	*info->map = *map;
 
 out:
 	mutex_unlock(&its_dev->event_map.vlpi_lock);
Signature · 1 line
2.20.1
author maintainer
#21 lore raw

PATCH 10/35 irqchip/gic-v4.1: VPE table (aka GICR_VPROPBASER) allocation

In reply to Marc Zyngier

GICv4.1 defines a new VPE table that is potentially shared between both the ITSs and the redistributors, following complicated affinity rules.

To make things more confusing, the programming of this table at the redistributor level is reusing the GICv4.0 GICR_VPROPBASER register for something completely different.

The code flow is somewhat complexified by the need to respect the affinities required by the HW, meaning that tables can either be inherited from a previously discovered ITS or redistributor.

---
 arch/arm/include/asm/arch_gicv3.h   |   2 +
 arch/arm64/include/asm/arch_gicv3.h |   1 +
 drivers/irqchip/irq-gic-v3-its.c    | 301 +++++++++++++++++++++++++++-
 include/linux/irqchip/arm-gic-v3.h  |  33 ++-
 4 files changed, 330 insertions(+), 7 deletions(-)
arch/arm/include/asm/arch_gicv3.h +2 -0
index 0555f14cc8be..20bacab1d791 100644
@@ -10,6 +10,7 @@
 #ifndef __ASSEMBLY__
 
 #include <linux/io.h>
+#include <linux/io-64-nonatomic-lo-hi.h>
 #include <asm/barrier.h>
 #include <asm/cacheflush.h>
 #include <asm/cp15.h>
@@ -327,6 +328,7 @@ static inline u64 __gic_readq_nonatomic(const volatile void __iomem *addr)
 /*
  * GITS_VPROPBASER - hi and lo bits may be accessed independently.
  */
+#define gits_read_vpropbaser(c)		__gic_readq_nonatomic(c)
 #define gits_write_vpropbaser(v, c)	__gic_writeq_nonatomic(v, c)
 
 /*
arch/arm64/include/asm/arch_gicv3.h +1 -0
index 89e4c8b79349..4750fc8030c3 100644
@@ -141,6 +141,7 @@ static inline u32 gic_read_rpr(void)
 #define gicr_read_pendbaser(c)		readq_relaxed(c)
 
 #define gits_write_vpropbaser(v, c)	writeq_relaxed(v, c)
+#define gits_read_vpropbaser(c)		readq_relaxed(c)
 
 #define gits_write_vpendbaser(v, c)	writeq_relaxed(v, c)
 #define gits_read_vpendbaser(c)		readq_relaxed(c)
drivers/irqchip/irq-gic-v3-its.c +298 -3

367 lines not on this page · raw message

include/linux/irqchip/arm-gic-v3.h +29 -4
index 8c6be56da7e9..f1d6de53e09b 100644
@@ -102,6 +102,11 @@
 
 #define GIC_V3_DIST_SIZE		0x10000
 
+#define GIC_PAGE_SIZE_4K		0ULL
+#define GIC_PAGE_SIZE_16K		1ULL
+#define GIC_PAGE_SIZE_64K		2ULL
+#define GIC_PAGE_SIZE_MASK		3ULL
+
 /*
  * Re-Distributor registers, offsets from RD_base
  */
@@ -239,6 +244,8 @@
 #define GICR_TYPER_DirectLPIS		(1U << 3)
 #define GICR_TYPER_LAST			(1U << 4)
 #define GICR_TYPER_RVPEID		(1U << 7)
+#define GICR_TYPER_COMMON_LPI_AFF	GENMASK_ULL(25, 24)
+#define GICR_TYPER_AFFINITY		GENMASK_ULL(63, 32)
 
 #define GIC_V3_REDIST_SIZE		0x20000
 
@@ -277,6 +284,18 @@
 #define GICR_VPROPBASER_RaWaWt	GIC_BASER_CACHEABILITY(GICR_VPROPBASER, INNER, RaWaWt)
 #define GICR_VPROPBASER_RaWaWb	GIC_BASER_CACHEABILITY(GICR_VPROPBASER, INNER, RaWaWb)
 
+/*
+ * GICv4.1 VPROPBASER reinvention. A subtle mix between the old
+ * VPROPBASER and ITS_BASER. Just not quite any of the two.
+ */
+#define GICR_VPROPBASER_4_1_VALID	(1ULL << 63)
+#define GICR_VPROPBASER_4_1_ENTRY_SIZE	GENMASK_ULL(61, 59)
+#define GICR_VPROPBASER_4_1_INDIRECT	(1ULL << 55)
+#define GICR_VPROPBASER_4_1_PAGE_SIZE	GENMASK_ULL(54, 53)
+#define GICR_VPROPBASER_4_1_Z		(1ULL << 52)
+#define GICR_VPROPBASER_4_1_ADDR	GENMASK_ULL(51, 12)
+#define GICR_VPROPBASER_4_1_SIZE	GENMASK_ULL(6, 0)
+
 #define GICR_VPENDBASER			0x0078
 
 #define GICR_VPENDBASER_SHAREABILITY_SHIFT		(10)
@@ -314,6 +333,7 @@
 #define GITS_CTLR			0x0000
 #define GITS_IIDR			0x0004
 #define GITS_TYPER			0x0008
+#define GITS_MPIDR			0x0018
 #define GITS_CBASER			0x0080
 #define GITS_CWRITER			0x0088
 #define GITS_CREADR			0x0090
@@ -347,6 +367,8 @@
 #define GITS_TYPER_HCC_SHIFT		24
 #define GITS_TYPER_HCC(r)		(((r) >> GITS_TYPER_HCC_SHIFT) & 0xff)
 #define GITS_TYPER_VMOVP		(1ULL << 37)
+#define GITS_TYPER_VMAPP		(1ULL << 40)
+#define GITS_TYPER_SVPET		GENMASK_ULL(42, 41)
 
 #define GITS_IIDR_REV_SHIFT		12
 #define GITS_IIDR_REV_MASK		(0xf << GITS_IIDR_REV_SHIFT)
@@ -417,10 +439,11 @@
 #define GITS_BASER_InnerShareable					\
 	GIC_BASER_SHAREABILITY(GITS_BASER, InnerShareable)
 #define GITS_BASER_PAGE_SIZE_SHIFT	(8)
-#define GITS_BASER_PAGE_SIZE_4K		(0ULL << GITS_BASER_PAGE_SIZE_SHIFT)
-#define GITS_BASER_PAGE_SIZE_16K	(1ULL << GITS_BASER_PAGE_SIZE_SHIFT)
-#define GITS_BASER_PAGE_SIZE_64K	(2ULL << GITS_BASER_PAGE_SIZE_SHIFT)
-#define GITS_BASER_PAGE_SIZE_MASK	(3ULL << GITS_BASER_PAGE_SIZE_SHIFT)
+#define __GITS_BASER_PSZ(sz)		(GIC_PAGE_SIZE_ ## sz << GITS_BASER_PAGE_SIZE_SHIFT)
+#define GITS_BASER_PAGE_SIZE_4K		__GITS_BASER_PSZ(4K)
+#define GITS_BASER_PAGE_SIZE_16K	__GITS_BASER_PSZ(16K)
+#define GITS_BASER_PAGE_SIZE_64K	__GITS_BASER_PSZ(64K)
+#define GITS_BASER_PAGE_SIZE_MASK	__GITS_BASER_PSZ(MASK)
 #define GITS_BASER_PAGES_MAX		256
 #define GITS_BASER_PAGES_SHIFT		(0)
 #define GITS_BASER_NR_PAGES(r)		(((r) & 0xff) + 1)
@@ -610,8 +633,10 @@ struct rdists {
 	struct {
 		void __iomem	*rd_base;
 		struct page	*pend_page;
+		struct page	*vpe_l1_page;
 		phys_addr_t	phys_base;
 		bool		lpi_enabled;
+		cpumask_t	*vpe_table_mask;
 	} __percpu		*rdist;
 	phys_addr_t		prop_table_pa;
 	void			*prop_table_va;
Signature · 1 line
2.20.1
author maintainer
#30 lore raw

PATCH 11/35 irqchip/gic-v4.1: Implement the v4.1 flavour of VMAPP

In reply to Marc Zyngier

The ITS VMAPP command gains some new fields with GICv4.1:
- a default doorbell, which allows a single doorbell to be used for all the VLPIs routed to a given VPE
- a pointer to the configuration table (instead of having it in a register that gets context switched)
- a flag indicating whether this is the first map or the last unmap for this particulat VPE
- a flag indicating whether the pending table is known to be zeroed, or not

Plumb in the new fields in the VMAPP builder, and add the map/unmap refcounting so that the ITS can do the right thing.

---
 drivers/irqchip/irq-gic-v3-its.c   | 60 +++++++++++++++++++++++++++---
 include/linux/irqchip/arm-gic-v4.h | 18 +++++++--
 2 files changed, 69 insertions(+), 9 deletions(-)
drivers/irqchip/irq-gic-v3-its.c +55 -5
index 58c7c40f5ad7..8d6deb75b973 100644
@@ -422,6 +422,27 @@ static void its_encode_vpt_size(struct its_cmd_block *cmd, u8 vpt_size)
 	its_mask_encode(&cmd->raw_cmd[3], vpt_size, 4, 0);
 }
 
+static void its_encode_vconf_addr(struct its_cmd_block *cmd, u64 vconf_pa)
+{
+	its_mask_encode(&cmd->raw_cmd[0], vconf_pa >> 16, 51, 16);
+}
+
+static void its_encode_alloc(struct its_cmd_block *cmd, bool alloc)
+{
+	its_mask_encode(&cmd->raw_cmd[0], alloc, 8, 8);
+}
+
+static void its_encode_ptz(struct its_cmd_block *cmd, bool ptz)
+{
+	its_mask_encode(&cmd->raw_cmd[0], ptz, 9, 9);
+}
+
+static void its_encode_vmapp_default_db(struct its_cmd_block *cmd,
+					u32 vpe_db_lpi)
+{
+	its_mask_encode(&cmd->raw_cmd[1], vpe_db_lpi, 31, 0);
+}
+
 static inline void its_fixup_cmd(struct its_cmd_block *cmd)
 {
 	/* Let's fixup BE commands */
@@ -605,19 +626,45 @@ static struct its_vpe *its_build_vmapp_cmd(struct its_node *its,
 					   struct its_cmd_block *cmd,
 					   struct its_cmd_desc *desc)
 {
-	unsigned long vpt_addr;
+	unsigned long vpt_addr, vconf_addr;
 	u64 target;
-
-	vpt_addr = virt_to_phys(page_address(desc->its_vmapp_cmd.vpe->vpt_page));
-	target = desc->its_vmapp_cmd.col->target_address + its->vlpi_redist_offset;
+	bool alloc;
 
 	its_encode_cmd(cmd, GITS_CMD_VMAPP);
 	its_encode_vpeid(cmd, desc->its_vmapp_cmd.vpe->vpe_id);
 	its_encode_valid(cmd, desc->its_vmapp_cmd.valid);
+
+	if (!desc->its_vmapp_cmd.valid) {
+		if (is_v4_1(its)) {
+			alloc = !atomic_dec_return(&desc->its_vmapp_cmd.vpe->vmapp_count);
+			its_encode_alloc(cmd, alloc);
+		}
+
+		goto out;
+	}
+
+	vpt_addr = virt_to_phys(page_address(desc->its_vmapp_cmd.vpe->vpt_page));
+	target = desc->its_vmapp_cmd.col->target_address + its->vlpi_redist_offset;
+
 	its_encode_target(cmd, target);
 	its_encode_vpt_addr(cmd, vpt_addr);
 	its_encode_vpt_size(cmd, LPI_NRBITS - 1);
 
+	if (!is_v4_1(its))
+		goto out;
+
+	vconf_addr = virt_to_phys(page_address(desc->its_vmapp_cmd.vpe->its_vm->vprop_page));
+
+	alloc = atomic_inc_and_test(&desc->its_vmapp_cmd.vpe->vmapp_count);
+
+	its_encode_alloc(cmd, alloc);
+
+	/* We can only signal PTZ when alloc==1. Why do we have two bits? */
+	its_encode_ptz(cmd, alloc);
+	its_encode_vconf_addr(cmd, vconf_addr);
+	its_encode_vmapp_default_db(cmd, desc->its_vmapp_cmd.vpe->vpe_db_lpi);
+
+out:
 	its_fixup_cmd(cmd);
 
 	return valid_vpe(its, desc->its_vmapp_cmd.vpe);
@@ -3344,7 +3391,10 @@ static int its_vpe_init(struct its_vpe *vpe)
 
 	vpe->vpe_id = vpe_id;
 	vpe->vpt_page = vpt_page;
-	vpe->vpe_proxy_event = -1;
+	if (gic_rdists->has_rvpeid)
+		atomic_set(&vpe->vmapp_count, 0);
+	else
+		vpe->vpe_proxy_event = -1;
 
 	return 0;
 }
include/linux/irqchip/arm-gic-v4.h +14 -4
index ab1396afe08a..6213ced6f199 100644
@@ -37,8 +37,20 @@ struct its_vpe {
 	irq_hw_number_t		vpe_db_lpi;
 	/* VPE resident */
 	bool			resident;
-	/* VPE proxy mapping */
-	int			vpe_proxy_event;
+	union {
+		/* GICv4.0 implementations */
+		struct {
+			/* VPE proxy mapping */
+			int	vpe_proxy_event;
+			/* Implementation Defined Area Invalid */
+			bool	idai;
+		};
+		/* GICv4.1 implementations */
+		struct {
+			atomic_t vmapp_count;
+		};
+	};
+
 	/*
 	 * This collection ID is used to indirect the target
 	 * redistributor for this VPE. The ID itself isn't involved in
@@ -47,8 +59,6 @@ struct its_vpe {
 	u16			col_idx;
 	/* Unique (system-wide) VPE identifier */
 	u16			vpe_id;
-	/* Implementation Defined Area Invalid */
-	bool			idai;
 	/* Pending VLPIs on schedule out? */
 	bool			pending_last;
 };
Signature · 1 line
2.20.1
author maintainer
#31 lore raw

PATCH 12/35 irqchip/gic-v4.1: Don't use the VPE proxy if RVPEID is set

In reply to Marc Zyngier

The infamous VPE proxy device isn't used with GICv4.1 because:
- we can invalidate any LPI from the DirectLPI MMIO interface
- the ITS and redistributors understand the life cycle of the doorbell, so we don't need to enable/disable it all the time

So let's escape early from the proxy related functions.

---
 drivers/irqchip/irq-gic-v3-its.c | 23 ++++++++++++++++++++++-
 1 file changed, 22 insertions(+), 1 deletion(-)
drivers/irqchip/irq-gic-v3-its.c +22 -1
index 8d6deb75b973..a9976bc81d9a 100644
@@ -3065,7 +3065,7 @@ static const struct irq_domain_ops its_domain_ops = {
 /*
  * This is insane.
  *
- * If a GICv4 doesn't implement Direct LPIs (which is extremely
+ * If a GICv4.0 doesn't implement Direct LPIs (which is extremely
  * likely), the only way to perform an invalidate is to use a fake
  * device to issue an INV command, implying that the LPI has first
  * been mapped to some event on that device. Since this is not exactly
@@ -3073,9 +3073,18 @@ static const struct irq_domain_ops its_domain_ops = {
  * only issue an UNMAP if we're short on available slots.
  *
  * Broken by design(tm).
+ *
+ * GICv4.1 actually mandates that we're able to invalidate by writing to a
+ * MMIO register. It doesn't implement the whole of DirectLPI, but that's
+ * good enough. And most of the time, we don't even have to invalidate
+ * anything, so that's actually pretty good!
  */
 static void its_vpe_db_proxy_unmap_locked(struct its_vpe *vpe)
 {
+	/* GICv4.1 doesn't use a proxy, so nothing to do here */
+	if (gic_rdists->has_rvpeid)
+		return;
+
 	/* Already unmapped? */
 	if (vpe->vpe_proxy_event == -1)
 		return;
@@ -3098,6 +3107,10 @@ static void its_vpe_db_proxy_unmap_locked(struct its_vpe *vpe)
 
 static void its_vpe_db_proxy_unmap(struct its_vpe *vpe)
 {
+	/* GICv4.1 doesn't use a proxy, so nothing to do here */
+	if (gic_rdists->has_rvpeid)
+		return;
+
 	if (!gic_rdists->has_direct_lpi) {
 		unsigned long flags;
 
@@ -3109,6 +3122,10 @@ static void its_vpe_db_proxy_unmap(struct its_vpe *vpe)
 
 static void its_vpe_db_proxy_map_locked(struct its_vpe *vpe)
 {
+	/* GICv4.1 doesn't use a proxy, so nothing to do here */
+	if (gic_rdists->has_rvpeid)
+		return;
+
 	/* Already mapped? */
 	if (vpe->vpe_proxy_event != -1)
 		return;
@@ -3131,6 +3148,10 @@ static void its_vpe_db_proxy_move(struct its_vpe *vpe, int from, int to)
 	unsigned long flags;
 	struct its_collection *target_col;
 
+	/* GICv4.1 doesn't use a proxy, so nothing to do here */
+	if (gic_rdists->has_rvpeid)
+		return;
+
 	if (gic_rdists->has_direct_lpi) {
 		void __iomem *rdbase;
 
Signature · 1 line
2.20.1
author maintainer
#32 lore raw

PATCH 13/35 irqchip/gic-v4.1: Implement the v4.1 flavour of VMOVP

In reply to Marc Zyngier

With GICv4.1, VMOVP is extended to allow a default doorbell to be specified, as well as a validity bit for this doorbell. As an added bonus, VMOPVP isn't required anymore of moving a VPE between redistributors that share the same affinity.

Let's add this support to the VMOVP builder, and make sure we don't issuer the command if we don't really need to.

---
 drivers/irqchip/irq-gic-v3-its.c | 40 ++++++++++++++++++++++++++------
 1 file changed, 33 insertions(+), 7 deletions(-)
drivers/irqchip/irq-gic-v3-its.c +33 -7
index a9976bc81d9a..c71aa0046bf0 100644
@@ -443,6 +443,17 @@ static void its_encode_vmapp_default_db(struct its_cmd_block *cmd,
 	its_mask_encode(&cmd->raw_cmd[1], vpe_db_lpi, 31, 0);
 }
 
+static void its_encode_vmovp_default_db(struct its_cmd_block *cmd,
+					u32 vpe_db_lpi)
+{
+	its_mask_encode(&cmd->raw_cmd[3], vpe_db_lpi, 31, 0);
+}
+
+static void its_encode_db(struct its_cmd_block *cmd, bool db)
+{
+	its_mask_encode(&cmd->raw_cmd[2], db, 63, 63);
+}
+
 static inline void its_fixup_cmd(struct its_cmd_block *cmd)
 {
 	/* Let's fixup BE commands */
@@ -729,6 +740,11 @@ static struct its_vpe *its_build_vmovp_cmd(struct its_node *its,
 	its_encode_vpeid(cmd, desc->its_vmovp_cmd.vpe->vpe_id);
 	its_encode_target(cmd, target);
 
+	if (is_v4_1(its)) {
+		its_encode_db(cmd, true);
+		its_encode_vmovp_default_db(cmd, desc->its_vmovp_cmd.vpe->vpe_db_lpi);
+	}
+
 	its_fixup_cmd(cmd);
 
 	return valid_vpe(its, desc->its_vmovp_cmd.vpe);
@@ -3178,7 +3194,7 @@ static int its_vpe_set_affinity(struct irq_data *d,
 				bool force)
 {
 	struct its_vpe *vpe = irq_data_get_irq_chip_data(d);
-	int cpu = cpumask_first(mask_val);
+	int from, cpu = cpumask_first(mask_val);
 
 	/*
 	 * Changing affinity is mega expensive, so let's be as lazy as
@@ -3186,14 +3202,24 @@ static int its_vpe_set_affinity(struct irq_data *d,
 	 * into the proxy device, we need to move the doorbell
 	 * interrupt to its new location.
 	 */
-	if (vpe->col_idx != cpu) {
-		int from = vpe->col_idx;
+	if (vpe->col_idx == cpu)
+		goto out;
 
-		vpe->col_idx = cpu;
-		its_send_vmovp(vpe);
-		its_vpe_db_proxy_move(vpe, from, cpu);
-	}
+	from = vpe->col_idx;
+	vpe->col_idx = cpu;
+
+	/*
+	 * GICv4.1 allows us to skip VMOVP if moving to a cpu whose RD
+	 * is sharing its VPE table with the current one.
+	 */
+	if (gic_data_rdist_cpu(cpu)->vpe_table_mask &&
+	    cpumask_test_cpu(from, gic_data_rdist_cpu(cpu)->vpe_table_mask))
+		goto out;
 
+	its_send_vmovp(vpe);
+	its_vpe_db_proxy_move(vpe, from, cpu);
+
+out:
 	irq_data_update_effective_affinity(d, cpumask_of(cpu));
 
 	return IRQ_SET_MASK_OK_DONE;
Signature · 6 lines
2.20.1

_______________________________________________
kvmarm mailing list
[email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
author maintainer
#33 lore raw

PATCH 14/35 irqchip/gic-v4.1: Plumb skeletal VPE irqchip

In reply to Marc Zyngier

Just like for GICv4.0, each VPE has its own doorbell interrupt, and thus an irqchip that manages them. Since the doorbell management is quite different on GICv4.1, let's introduce an almost empty irqchip the will get populated over the next new patches.

---
 drivers/irqchip/irq-gic-v3-its.c | 32 +++++++++++++++++++++++++++++++-
 1 file changed, 31 insertions(+), 1 deletion(-)
drivers/irqchip/irq-gic-v3-its.c +31 -1
index c71aa0046bf0..4098ef6d030a 100644
@@ -3403,6 +3403,32 @@ static struct irq_chip its_vpe_irq_chip = {
 	.irq_set_vcpu_affinity	= its_vpe_set_vcpu_affinity,
 };
 
+static int its_vpe_4_1_set_vcpu_affinity(struct irq_data *d, void *vcpu_info)
+{
+	struct its_cmd_info *info = vcpu_info;
+
+	switch (info->cmd_type) {
+	case SCHEDULE_VPE:
+		return 0;
+
+	case DESCHEDULE_VPE:
+		return 0;
+
+	case INVALL_VPE:
+		return 0;
+
+	default:
+		return -EINVAL;
+	}
+}
+
+static struct irq_chip its_vpe_4_1_irq_chip = {
+	.name			= "GICv4.1-vpe",
+	.irq_eoi		= irq_chip_eoi_parent,
+	.irq_set_affinity	= its_vpe_set_affinity,
+	.irq_set_vcpu_affinity	= its_vpe_4_1_set_vcpu_affinity,
+};
+
 static int its_vpe_id_alloc(void)
 {
 	return ida_simple_get(&its_vpeid_ida, 0, ITS_MAX_VPEID, GFP_KERNEL);
@@ -3483,6 +3509,7 @@ static void its_vpe_irq_domain_free(struct irq_domain *domain,
 static int its_vpe_irq_domain_alloc(struct irq_domain *domain, unsigned int virq,
 				    unsigned int nr_irqs, void *args)
 {
+	struct irq_chip *irqchip = &its_vpe_irq_chip;
 	struct its_vm *vm = args;
 	unsigned long *bitmap;
 	struct page *vprop_page;
@@ -3510,6 +3537,9 @@ static int its_vpe_irq_domain_alloc(struct irq_domain *domain, unsigned int virq
 	vm->nr_db_lpis = nr_ids;
 	vm->vprop_page = vprop_page;
 
+	if (gic_rdists->has_rvpeid)
+		irqchip = &its_vpe_4_1_irq_chip;
+
 	for (i = 0; i < nr_irqs; i++) {
 		vm->vpes[i]->vpe_db_lpi = base + i;
 		err = its_vpe_init(vm->vpes[i]);
@@ -3520,7 +3550,7 @@ static int its_vpe_irq_domain_alloc(struct irq_domain *domain, unsigned int virq
 		if (err)
 			break;
 		irq_domain_set_hwirq_and_chip(domain, virq + i, i,
-					      &its_vpe_irq_chip, vm->vpes[i]);
+					      irqchip, vm->vpes[i]);
 		set_bit(i, bitmap);
 	}
 
Signature · 1 line
2.20.1
author maintainer
#34 lore raw

PATCH 15/35 irqchip/gic-v4.1: Add mask/unmask doorbell callbacks

In reply to Marc Zyngier

masking/unmasking doorbells on GICv4.1 relies on a new INVDB command, which broadcasts the invalidation to all RDs.

Implement the new command as well as the masking callbacks, and plug the whole thing into the v4.1 VPE irqchip.

---
 drivers/irqchip/irq-gic-v3-its.c   | 60 ++++++++++++++++++++++++++++++
 include/linux/irqchip/arm-gic-v3.h |  3 +-
 2 files changed, 62 insertions(+), 1 deletion(-)
drivers/irqchip/irq-gic-v3-its.c +60 -0
index 4098ef6d030a..36653db372bc 100644
@@ -309,6 +309,10 @@ struct its_cmd_desc {
 			u16 seq_num;
 			u16 its_list;
 		} its_vmovp_cmd;
+
+		struct {
+			struct its_vpe *vpe;
+		} its_invdb_cmd;
 	};
 };
 
@@ -750,6 +754,21 @@ static struct its_vpe *its_build_vmovp_cmd(struct its_node *its,
 	return valid_vpe(its, desc->its_vmovp_cmd.vpe);
 }
 
+static struct its_vpe *its_build_invdb_cmd(struct its_node *its,
+					   struct its_cmd_block *cmd,
+					   struct its_cmd_desc *desc)
+{
+	if (WARN_ON(!is_v4_1(its)))
+		return NULL;
+
+	its_encode_cmd(cmd, GITS_CMD_INVDB);
+	its_encode_vpeid(cmd, desc->its_invdb_cmd.vpe->vpe_id);
+
+	its_fixup_cmd(cmd);
+
+	return valid_vpe(its, desc->its_invdb_cmd.vpe);
+}
+
 static u64 its_cmd_ptr_to_offset(struct its_node *its,
 				 struct its_cmd_block *ptr)
 {
@@ -1117,6 +1136,14 @@ static void its_send_vinvall(struct its_node *its, struct its_vpe *vpe)
 	its_send_single_vcommand(its, its_build_vinvall_cmd, &desc);
 }
 
+static void its_send_invdb(struct its_node *its, struct its_vpe *vpe)
+{
+	struct its_cmd_desc desc;
+
+	desc.its_invdb_cmd.vpe = vpe;
+	its_send_single_vcommand(its, its_build_invdb_cmd, &desc);
+}
+
 /*
  * irqchip functions - assumes MSI, mostly.
  */
@@ -3403,6 +3430,37 @@ static struct irq_chip its_vpe_irq_chip = {
 	.irq_set_vcpu_affinity	= its_vpe_set_vcpu_affinity,
 };
 
+static void its_vpe_4_1_send_inv(struct irq_data *d)
+{
+	struct its_vpe *vpe = irq_data_get_irq_chip_data(d);
+	struct its_node *its;
+
+	/*
+	 * GICv4.1 wants doorbells to be invalidated using the
+	 * INVDB command in order to be broadcast to all RDs. Send
+	 * it to the first valid ITS, and let the HW do its magic.
+	 */
+	list_for_each_entry(its, &its_nodes, entry) {
+		if (!is_v4_1(its))
+			continue;
+
+		its_send_invdb(its, vpe);
+		break;
+	}
+}
+
+static void its_vpe_4_1_mask_irq(struct irq_data *d)
+{
+	lpi_write_config(d->parent_data, LPI_PROP_ENABLED, 0);
+	its_vpe_4_1_send_inv(d);
+}
+
+static void its_vpe_4_1_unmask_irq(struct irq_data *d)
+{
+	lpi_write_config(d->parent_data, 0, LPI_PROP_ENABLED);
+	its_vpe_4_1_send_inv(d);
+}
+
 static int its_vpe_4_1_set_vcpu_affinity(struct irq_data *d, void *vcpu_info)
 {
 	struct its_cmd_info *info = vcpu_info;
@@ -3424,6 +3482,8 @@ static int its_vpe_4_1_set_vcpu_affinity(struct irq_data *d, void *vcpu_info)
 
 static struct irq_chip its_vpe_4_1_irq_chip = {
 	.name			= "GICv4.1-vpe",
+	.irq_mask		= its_vpe_4_1_mask_irq,
+	.irq_unmask		= its_vpe_4_1_unmask_irq,
 	.irq_eoi		= irq_chip_eoi_parent,
 	.irq_set_affinity	= its_vpe_set_affinity,
 	.irq_set_vcpu_affinity	= its_vpe_4_1_set_vcpu_affinity,
include/linux/irqchip/arm-gic-v3.h +2 -1
index f1d6de53e09b..8157737053e4 100644
@@ -484,8 +484,9 @@
 #define GITS_CMD_VMAPTI			GITS_CMD_GICv4(GITS_CMD_MAPTI)
 #define GITS_CMD_VMOVI			GITS_CMD_GICv4(GITS_CMD_MOVI)
 #define GITS_CMD_VSYNC			GITS_CMD_GICv4(GITS_CMD_SYNC)
-/* VMOVP is the odd one, as it doesn't have a physical counterpart */
+/* VMOVP and INVDB are the odd ones, as they dont have a physical counterpart */
 #define GITS_CMD_VMOVP			GITS_CMD_GICv4(2)
+#define GITS_CMD_INVDB			GITS_CMD_GICv4(0xe)
 
 /*
  * ITS error numbers
Signature · 1 line
2.20.1
author maintainer
#35 lore raw

PATCH 16/35 irqchip/gic-v4.1: Add VPE residency callback

In reply to Marc Zyngier

Making a VPE resident on GICv4.1 is pretty simple, as it is just a single write to the local redistributor. We just need extra information about which groups to enable, which the KVM code will have to provide.

---
 drivers/irqchip/irq-gic-v3-its.c   | 17 +++++++++++++++++
 include/linux/irqchip/arm-gic-v3.h |  9 +++++++++
 include/linux/irqchip/arm-gic-v4.h |  5 +++++
 3 files changed, 31 insertions(+)
drivers/irqchip/irq-gic-v3-its.c +17 -0
index 36653db372bc..72d80daa7230 100644
@@ -3461,12 +3461,29 @@ static void its_vpe_4_1_unmask_irq(struct irq_data *d)
 	its_vpe_4_1_send_inv(d);
 }
 
+static void its_vpe_4_1_schedule(struct its_vpe *vpe,
+				 struct its_cmd_info *info)
+{
+	void __iomem *vlpi_base = gic_data_rdist_vlpi_base();
+	u64 val = 0;
+
+	/* Schedule the VPE */
+	val |= GICR_VPENDBASER_Valid;
+	val |= info->g0en ? GICR_VPENDBASER_4_1_VGRP0EN : 0;
+	val |= info->g1en ? GICR_VPENDBASER_4_1_VGRP1EN : 0;
+	val |= FIELD_PREP(GICR_VPENDBASER_4_1_VPEID, vpe->vpe_id);
+
+	gits_write_vpendbaser(val, vlpi_base + GICR_VPENDBASER);
+}
+
 static int its_vpe_4_1_set_vcpu_affinity(struct irq_data *d, void *vcpu_info)
 {
+	struct its_vpe *vpe = irq_data_get_irq_chip_data(d);
 	struct its_cmd_info *info = vcpu_info;
 
 	switch (info->cmd_type) {
 	case SCHEDULE_VPE:
+		its_vpe_4_1_schedule(vpe, info);
 		return 0;
 
 	case DESCHEDULE_VPE:
include/linux/irqchip/arm-gic-v3.h +9 -0
index 8157737053e4..6fd89d77b2b2 100644
@@ -327,6 +327,15 @@
 #define GICR_VPENDBASER_IDAI		(1ULL << 62)
 #define GICR_VPENDBASER_Valid		(1ULL << 63)
 
+/*
+ * GICv4.1 VPENDBASER, used for VPE residency. On top of these fields,
+ * also use the above Valid, PendingLast and Dirty.
+ */
+#define GICR_VPENDBASER_4_1_DB		(1ULL << 62)
+#define GICR_VPENDBASER_4_1_VGRP0EN	(1ULL << 59)
+#define GICR_VPENDBASER_4_1_VGRP1EN	(1ULL << 58)
+#define GICR_VPENDBASER_4_1_VPEID	GENMASK_ULL(15, 0)
+
 /*
  * ITS registers, offsets from ITS_base
  */
include/linux/irqchip/arm-gic-v4.h +5 -0
index 6213ced6f199..edbaa37fd3f1 100644
@@ -98,6 +98,11 @@ struct its_cmd_info {
 	union {
 		struct its_vlpi_map	*map;
 		u8			config;
+		bool			req_db;
+		struct {
+			bool		g0en;
+			bool		g1en;
+		};
 	};
 };
 
Signature · 1 line
2.20.1
author maintainer
#36 lore raw

PATCH 17/35 irqchip/gic-v4.1: Add VPE eviction callback

In reply to Marc Zyngier

When descheduling a VPE, special care must be taken to tell the GIC about whether we want to receive a doorbell or not. This is a major improvement on GICv4.0, where the doorbell had to be separately enabled/disabled.

---
 drivers/irqchip/irq-gic-v3-its.c | 53 +++++++++++++++++++++++++-------
 1 file changed, 42 insertions(+), 11 deletions(-)
drivers/irqchip/irq-gic-v3-its.c +42 -11
index 72d80daa7230..3079ce74def6 100644
@@ -2483,7 +2483,7 @@ static int __init allocate_lpi_tables(void)
 	return 0;
 }
 
-static u64 its_clear_vpend_valid(void __iomem *vlpi_base)
+static u64 its_clear_vpend_valid(void __iomem *vlpi_base, u64 clr, u64 set)
 {
 	u32 count = 1000000;	/* 1s! */
 	bool clean;
@@ -2491,6 +2491,8 @@ static u64 its_clear_vpend_valid(void __iomem *vlpi_base)
 
 	val = gits_read_vpendbaser(vlpi_base + GICR_VPENDBASER);
 	val &= ~GICR_VPENDBASER_Valid;
+	val &= ~clr;
+	val |= set;
 	gits_write_vpendbaser(val, vlpi_base + GICR_VPENDBASER);
 
 	do {
@@ -2503,6 +2505,11 @@ static u64 its_clear_vpend_valid(void __iomem *vlpi_base)
 		}
 	} while (!clean && count);
 
+	if (unlikely(val & GICR_VPENDBASER_Dirty)) {
+		pr_err_ratelimited("ITS virtual pending table not cleaning\n");
+		val |= GICR_VPENDBASER_PendingLast;
+	}
+
 	return val;
 }
 
@@ -2611,7 +2618,7 @@ static void its_cpu_init_lpis(void)
 		 * ancient programming gets left in and has possibility of
 		 * corrupting memory.
 		 */
-		val = its_clear_vpend_valid(vlpi_base);
+		val = its_clear_vpend_valid(vlpi_base, 0, 0);
 		WARN_ON(val & GICR_VPENDBASER_Dirty);
 	}
 
@@ -3289,16 +3296,10 @@ static void its_vpe_deschedule(struct its_vpe *vpe)
 	void __iomem *vlpi_base = gic_data_rdist_vlpi_base();
 	u64 val;
 
-	val = its_clear_vpend_valid(vlpi_base);
+	val = its_clear_vpend_valid(vlpi_base, 0, 0);
 
-	if (unlikely(val & GICR_VPENDBASER_Dirty)) {
-		pr_err_ratelimited("ITS virtual pending table not cleaning\n");
-		vpe->idai = false;
-		vpe->pending_last = true;
-	} else {
-		vpe->idai = !!(val & GICR_VPENDBASER_IDAI);
-		vpe->pending_last = !!(val & GICR_VPENDBASER_PendingLast);
-	}
+	vpe->idai = !!(val & GICR_VPENDBASER_IDAI);
+	vpe->pending_last = !!(val & GICR_VPENDBASER_PendingLast);
 }
 
 static void its_vpe_invall(struct its_vpe *vpe)
@@ -3476,6 +3477,35 @@ static void its_vpe_4_1_schedule(struct its_vpe *vpe,
 	gits_write_vpendbaser(val, vlpi_base + GICR_VPENDBASER);
 }
 
+static void its_vpe_4_1_deschedule(struct its_vpe *vpe,
+				   struct its_cmd_info *info)
+{
+	void __iomem *vlpi_base = gic_data_rdist_vlpi_base();
+	u64 val;
+
+	if (info->req_db) {
+		/*
+		 * vPE is going to block: make the vPE non-resident with
+		 * PendingLast clear and DB set. The GIC guarantees that if
+		 * we read-back PendingLast clear, then a doorbell will be
+		 * delivered when an interrupt comes.
+		 */
+		val = its_clear_vpend_valid(vlpi_base,
+					    GICR_VPENDBASER_PendingLast,
+					    GICR_VPENDBASER_4_1_DB);
+		vpe->pending_last = !!(val & GICR_VPENDBASER_PendingLast);
+	} else {
+		/*
+		 * We're not blocking, so just make the vPE non-resident
+		 * with PendingLast set, indicating that we'll be back.
+		 */
+		val = its_clear_vpend_valid(vlpi_base,
+					    0,
+					    GICR_VPENDBASER_PendingLast);
+		vpe->pending_last = true;
+	}
+}
+
 static int its_vpe_4_1_set_vcpu_affinity(struct irq_data *d, void *vcpu_info)
 {
 	struct its_vpe *vpe = irq_data_get_irq_chip_data(d);
@@ -3487,6 +3517,7 @@ static int its_vpe_4_1_set_vcpu_affinity(struct irq_data *d, void *vcpu_info)
 		return 0;
 
 	case DESCHEDULE_VPE:
+		its_vpe_4_1_deschedule(vpe, info);
 		return 0;
 
 	case INVALL_VPE:
Signature · 1 line
2.20.1
author maintainer
#37 lore raw

PATCH 18/35 irqchip/gic-v4.1: Add VPE INVALL callback

In reply to Marc Zyngier

GICv4.1 redistributors have a VPE-aware INVALL register. Progress! We can now emulate a guest-requested INVALL without emiting a VINVALL command.

---
 drivers/irqchip/irq-gic-v3-its.c   | 14 ++++++++++++++
 include/linux/irqchip/arm-gic-v3.h |  3 +++
 2 files changed, 17 insertions(+)
drivers/irqchip/irq-gic-v3-its.c +14 -0
index 3079ce74def6..eda35e66237f 100644
@@ -3506,6 +3506,19 @@ static void its_vpe_4_1_deschedule(struct its_vpe *vpe,
 	}
 }
 
+static void its_vpe_4_1_invall(struct its_vpe *vpe)
+{
+	void __iomem *rdbase;
+	u64 val;
+
+	val  = GICR_INVLPIR_V;
+	val |= FIELD_PREP(GICR_INVLPIR_VPEID, vpe->vpe_id);
+
+	/* Target the redistributor this vPE is currently known on */
+	rdbase = per_cpu_ptr(gic_rdists->rdist, vpe->col_idx)->rd_base;
+	gic_write_lpir(val, rdbase + GICR_INVALLR);
+}
+
 static int its_vpe_4_1_set_vcpu_affinity(struct irq_data *d, void *vcpu_info)
 {
 	struct its_vpe *vpe = irq_data_get_irq_chip_data(d);
@@ -3521,6 +3534,7 @@ static int its_vpe_4_1_set_vcpu_affinity(struct irq_data *d, void *vcpu_info)
 		return 0;
 
 	case INVALL_VPE:
+		its_vpe_4_1_invall(vpe);
 		return 0;
 
 	default:
include/linux/irqchip/arm-gic-v3.h +3 -0
index 6fd89d77b2b2..b69f60792554 100644
@@ -247,6 +247,9 @@
 #define GICR_TYPER_COMMON_LPI_AFF	GENMASK_ULL(25, 24)
 #define GICR_TYPER_AFFINITY		GENMASK_ULL(63, 32)
 
+#define GICR_INVLPIR_VPEID		GENMASK_ULL(47, 32)
+#define GICR_INVLPIR_V			GENMASK_ULL(63, 63)
+
 #define GIC_V3_REDIST_SIZE		0x20000
 
 #define LPI_PROP_GROUP1			(1 << 1)
Signature · 1 line
2.20.1
author maintainer
#38 lore raw

PATCH 19/35 irqchip/gic-v4.1: Suppress per-VLPI doorbell

In reply to Marc Zyngier

Since GICv4.1 gives us a per-VPE doorbell, avoid programming anything else on VMOVI/VMAPI/VMAPTI and on any other action that would have otherwise resulted in a per-VLPI doorbell to be programmed.

---
 drivers/irqchip/irq-gic-v3-its.c | 11 +++++++++--
 1 file changed, 9 insertions(+), 2 deletions(-)
drivers/irqchip/irq-gic-v3-its.c +9 -2
index eda35e66237f..b791c9beddf2 100644
@@ -691,7 +691,7 @@ static struct its_vpe *its_build_vmapti_cmd(struct its_node *its,
 {
 	u32 db;
 
-	if (desc->its_vmapti_cmd.db_enabled)
+	if (!is_v4_1(its) && desc->its_vmapti_cmd.db_enabled)
 		db = desc->its_vmapti_cmd.vpe->vpe_db_lpi;
 	else
 		db = 1023;
@@ -714,7 +714,7 @@ static struct its_vpe *its_build_vmovi_cmd(struct its_node *its,
 {
 	u32 db;
 
-	if (desc->its_vmovi_cmd.db_enabled)
+	if (!is_v4_1(its) && desc->its_vmovi_cmd.db_enabled)
 		db = desc->its_vmovi_cmd.vpe->vpe_db_lpi;
 	else
 		db = 1023;
@@ -1227,6 +1227,13 @@ static void its_vlpi_set_doorbell(struct irq_data *d, bool enable)
 	struct its_device *its_dev = irq_data_get_irq_chip_data(d);
 	u32 event = its_get_event_id(d);
 
+	/*
+	 * GICv4.1 does away with the per-LPI nonsense, nothing to do
+	 * here.
+	 */
+	if (is_v4_1(its_dev->its))
+		return;
+
 	if (its_dev->event_map.vlpi_maps[event].db_enabled == enable)
 		return;
 
Signature · 6 lines
2.20.1

_______________________________________________
kvmarm mailing list
[email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
author maintainer
#39 lore raw

PATCH 20/35 irqchip/gic-v4.1: Allow direct invalidation of VLPIs

In reply to Marc Zyngier

Just like for INVALL, GICv4.1 has grown a VPE-aware INVLPI register. Let's plumb it in and make use of the DirectLPI code in that case.

---
 drivers/irqchip/irq-gic-v3-its.c   | 19 +++++++++++++++++--
 include/linux/irqchip/arm-gic-v3.h |  1 +
 2 files changed, 18 insertions(+), 2 deletions(-)
drivers/irqchip/irq-gic-v3-its.c +17 -2
index b791c9beddf2..34595a7fcccb 100644
@@ -1200,13 +1200,27 @@ static void wait_for_syncr(void __iomem *rdbase)
 
 static void direct_lpi_inv(struct irq_data *d)
 {
+	struct its_vlpi_map *map = get_vlpi_map(d);
 	struct its_collection *col;
 	void __iomem *rdbase;
+	u64 val;
+
+	if (map) {
+		struct its_device *its_dev = irq_data_get_irq_chip_data(d);
+
+		WARN_ON(!is_v4_1(its_dev->its));
+
+		val  = GICR_INVLPIR_V;
+		val |= FIELD_PREP(GICR_INVLPIR_VPEID, map->vpe->vpe_id);
+		val |= FIELD_PREP(GICR_INVLPIR_INTID, map->vintid);
+	} else {
+		val = d->hwirq;
+	}
 
 	/* Target the redistributor this LPI is currently routed to */
 	col = irq_to_col(d);
 	rdbase = per_cpu_ptr(gic_rdists->rdist, col->col_id)->rd_base;
-	gic_write_lpir(d->hwirq, rdbase + GICR_INVLPIR);
+	gic_write_lpir(val, rdbase + GICR_INVLPIR);
 
 	wait_for_syncr(rdbase);
 }
@@ -1216,7 +1230,8 @@ static void lpi_update_config(struct irq_data *d, u8 clr, u8 set)
 	struct its_device *its_dev = irq_data_get_irq_chip_data(d);
 
 	lpi_write_config(d, clr, set);
-	if (gic_rdists->has_direct_lpi && !irqd_is_forwarded_to_vcpu(d))
+	if (gic_rdists->has_direct_lpi &&
+	    (is_v4_1(its_dev->its) || !irqd_is_forwarded_to_vcpu(d)))
 		direct_lpi_inv(d);
 	else
 		its_send_inv(its_dev, its_get_event_id(d));
include/linux/irqchip/arm-gic-v3.h +1 -0
index b69f60792554..5f3278cbf247 100644
@@ -247,6 +247,7 @@
 #define GICR_TYPER_COMMON_LPI_AFF	GENMASK_ULL(25, 24)
 #define GICR_TYPER_AFFINITY		GENMASK_ULL(63, 32)
 
+#define GICR_INVLPIR_INTID		GENMASK_ULL(31, 0)
 #define GICR_INVLPIR_VPEID		GENMASK_ULL(47, 32)
 #define GICR_INVLPIR_V			GENMASK_ULL(63, 63)
 
Signature · 6 lines
2.20.1

_______________________________________________
kvmarm mailing list
[email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
author maintainer
#43 lore raw

PATCH 21/35 irqchip/gic-v4.1: Advertise support v4.1 to KVM

In reply to Marc Zyngier

Tell KVM that we support v4.1. Nothing uses this information so far.

---
 drivers/irqchip/irq-gic-v3-its.c       | 9 ++++++++-
 drivers/irqchip/irq-gic-v3.c           | 1 +
 include/linux/irqchip/arm-gic-common.h | 2 ++
 3 files changed, 11 insertions(+), 1 deletion(-)
drivers/irqchip/irq-gic-v3-its.c +8 -1
index 34595a7fcccb..633cb10ec64b 100644
@@ -4573,6 +4573,7 @@ int __init its_init(struct fwnode_handle *handle, struct rdists *rdists,
 	struct device_node *of_node;
 	struct its_node *its;
 	bool has_v4 = false;
+	bool has_v4_1 = false;
 	int err;
 
 	gic_rdists = rdists;
@@ -4593,8 +4594,14 @@ int __init its_init(struct fwnode_handle *handle, struct rdists *rdists,
 	if (err)
 		return err;
 
-	list_for_each_entry(its, &its_nodes, entry)
+	list_for_each_entry(its, &its_nodes, entry) {
 		has_v4 |= is_v4(its);
+		has_v4_1 |= is_v4_1(its);
+	}
+
+	/* Don't bother with inconsistent systems */
+	if (WARN_ON(!has_v4_1 && rdists->has_rvpeid))
+		rdists->has_rvpeid = false;
 
 	if (has_v4 & rdists->has_vlpis) {
 		if (its_init_vpe_domain() ||
drivers/irqchip/irq-gic-v3.c +1 -0
index fb6360161d6c..47b9522516fe 100644
@@ -1737,6 +1737,7 @@ static void __init gic_of_setup_kvm_info(struct device_node *node)
 		gic_v3_kvm_info.vcpu = r;
 
 	gic_v3_kvm_info.has_v4 = gic_data.rdists.has_vlpis;
+	gic_v3_kvm_info.has_v4_1 = gic_data.rdists.has_rvpeid;
 	gic_set_kvm_info(&gic_v3_kvm_info);
 }
 
include/linux/irqchip/arm-gic-common.h +2 -0
index b9850f5f1906..fa8c0455c352 100644
@@ -32,6 +32,8 @@ struct gic_kvm_info {
 	struct resource vctrl;
 	/* vlpi support */
 	bool		has_v4;
+	/* rvpeid support */
+	bool		has_v4_1;
 };
 
 const struct gic_kvm_info *gic_get_kvm_info(void);
Signature · 6 lines
2.20.1

_______________________________________________
kvmarm mailing list
[email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
author maintainer
#44 lore raw

PATCH 22/35 irqchip/gic-v4.1: Map the ITS SGIR register page

In reply to Marc Zyngier

One of the new features of GICv4.1 is to allow virtual SGIs to be directly signaled to a VPE. For that, the ITS has grown a new 64kB page containing only a single register that is used to signal a SGI to a given VPE.

Add a second mapping covering this new 64kB range, and take this opportunity to limit the original mapping to 64kB, which is enough to cover the span of the ITS registers.

---
 drivers/irqchip/irq-gic-v3-its.c | 15 +++++++++++++--
 1 file changed, 13 insertions(+), 2 deletions(-)
drivers/irqchip/irq-gic-v3-its.c +13 -2
index 633cb10ec64b..88933b3fd61d 100644
@@ -97,6 +97,7 @@ struct its_node {
 	struct mutex		dev_alloc_lock;
 	struct list_head	entry;
 	void __iomem		*base;
+	void __iomem		*sgir_base;
 	phys_addr_t		phys_base;
 	struct its_cmd_block	*cmd_base;
 	struct its_cmd_block	*cmd_write;
@@ -4159,7 +4160,7 @@ static int __init its_probe_one(struct resource *res,
 	struct page *page;
 	int err;
 
-	its_base = ioremap(res->start, resource_size(res));
+	its_base = ioremap(res->start, SZ_64K);
 	if (!its_base) {
 		pr_warn("ITS@%pa: Unable to map ITS registers\n", &res->start);
 		return -ENOMEM;
@@ -4210,6 +4211,13 @@ static int __init its_probe_one(struct resource *res,
 
 		if (is_v4_1(its)) {
 			u32 svpet = FIELD_GET(GITS_TYPER_SVPET, typer);
+
+			its->sgir_base = ioremap(res->start + SZ_128K, SZ_64K);
+			if (!its->sgir_base) {
+				err = -ENOMEM;
+				goto out_free_its;
+			}
+
 			its->mpidr = readl_relaxed(its_base + GITS_MPIDR);
 
 			pr_info("ITS@%pa: Using GICv4.1 mode %08x %08x\n",
@@ -4223,7 +4231,7 @@ static int __init its_probe_one(struct resource *res,
 				get_order(ITS_CMD_QUEUE_SZ));
 	if (!page) {
 		err = -ENOMEM;
-		goto out_free_its;
+		goto out_unmap_sgir;
 	}
 	its->cmd_base = (void *)page_address(page);
 	its->cmd_write = its->cmd_base;
@@ -4290,6 +4298,9 @@ static int __init its_probe_one(struct resource *res,
 	its_free_tables(its);
 out_free_cmd:
 	free_pages((unsigned long)its->cmd_base, get_order(ITS_CMD_QUEUE_SZ));
+out_unmap_sgir:
+	if (its->sgir_base)
+		iounmap(its->sgir_base);
 out_free_its:
 	kfree(its);
 out_unmap:
Signature · 1 line
2.20.1
author maintainer
#45 lore raw

PATCH 23/35 irqchip/gic-v4.1: Plumb skeletal VSGI irqchip

In reply to Marc Zyngier

Since GICv4.1 has the capability to inject 16 SGIs into each VPE, and that I'm keen not to invent too many specific interfaces to manupulate these interrupts, let's pretend that each of these SGIs is an actual Linux interrupt.

For that matter, let's introduce a minimal irqchip and irqdomain setup that will get fleshed up in the following patches.

---
 drivers/irqchip/irq-gic-v3-its.c   | 68 +++++++++++++++++++++++++++++-
 drivers/irqchip/irq-gic-v4.c       |  8 +++-
 include/linux/irqchip/arm-gic-v4.h |  9 +++-
 3 files changed, 81 insertions(+), 4 deletions(-)
drivers/irqchip/irq-gic-v3-its.c +67 -1
index 88933b3fd61d..69c26be709be 100644
@@ -3574,6 +3574,67 @@ static struct irq_chip its_vpe_4_1_irq_chip = {
 	.irq_set_vcpu_affinity	= its_vpe_4_1_set_vcpu_affinity,
 };
 
+static int its_sgi_set_affinity(struct irq_data *d,
+				const struct cpumask *mask_val,
+				bool force)
+{
+	return -EINVAL;
+}
+
+static struct irq_chip its_sgi_irq_chip = {
+	.name			= "GICv4.1-sgi",
+	.irq_set_affinity	= its_sgi_set_affinity,
+};
+
+static int its_sgi_irq_domain_alloc(struct irq_domain *domain,
+				    unsigned int virq, unsigned int nr_irqs,
+				    void *args)
+{
+	struct its_vpe *vpe = args;
+	int i;
+
+	/* Yes, we do want 16 SGIs */
+	WARN_ON(nr_irqs != 16);
+
+	for (i = 0; i < 16; i++) {
+		vpe->sgi_config[i].priority = 0;
+		vpe->sgi_config[i].enabled = false;
+		vpe->sgi_config[i].group = false;
+
+		irq_domain_set_hwirq_and_chip(domain, virq + i, i,
+					      &its_sgi_irq_chip, vpe);
+		irq_set_status_flags(virq + i, IRQ_DISABLE_UNLAZY);
+	}
+
+	return 0;
+}
+
+static void its_sgi_irq_domain_free(struct irq_domain *domain,
+				    unsigned int virq,
+				    unsigned int nr_irqs)
+{
+	/* Nothing to do */
+}
+
+static int its_sgi_irq_domain_activate(struct irq_domain *domain,
+				       struct irq_data *d, bool reserve)
+{
+	return 0;
+}
+
+static void its_sgi_irq_domain_deactivate(struct irq_domain *domain,
+					  struct irq_data *d)
+{
+	/* Nothing to do */
+}
+
+static struct irq_domain_ops its_sgi_domain_ops = {
+	.alloc		= its_sgi_irq_domain_alloc,
+	.free		= its_sgi_irq_domain_free,
+	.activate	= its_sgi_irq_domain_activate,
+	.deactivate	= its_sgi_irq_domain_deactivate,
+};
+
 static int its_vpe_id_alloc(void)
 {
 	return ida_simple_get(&its_vpeid_ida, 0, ITS_MAX_VPEID, GFP_KERNEL);
@@ -4615,8 +4676,13 @@ int __init its_init(struct fwnode_handle *handle, struct rdists *rdists,
 		rdists->has_rvpeid = false;
 
 	if (has_v4 & rdists->has_vlpis) {
+		struct irq_domain_ops *sgi_ops = NULL;
+
+		if (has_v4_1)
+			sgi_ops = &its_sgi_domain_ops;
+
 		if (its_init_vpe_domain() ||
-		    its_init_v4(parent_domain, &its_vpe_domain_ops)) {
+		    its_init_v4(parent_domain, &its_vpe_domain_ops, sgi_ops)) {
 			rdists->has_vlpis = false;
 			pr_err("ITS: Disabling GICv4 support\n");
 		}
drivers/irqchip/irq-gic-v4.c +6 -2
index 45969927cc81..c01910d53f9e 100644
@@ -85,6 +85,7 @@
 
 static struct irq_domain *gic_domain;
 static const struct irq_domain_ops *vpe_domain_ops;
+static const struct irq_domain_ops *sgi_domain_ops;
 
 int its_alloc_vcpu_irqs(struct its_vm *vm)
 {
@@ -216,12 +217,15 @@ int its_prop_update_vlpi(int irq, u8 config, bool inv)
 	return irq_set_vcpu_affinity(irq, &info);
 }
 
-int its_init_v4(struct irq_domain *domain, const struct irq_domain_ops *ops)
+int its_init_v4(struct irq_domain *domain,
+		const struct irq_domain_ops *vpe_ops,
+		const struct irq_domain_ops *sgi_ops)
 {
 	if (domain) {
 		pr_info("ITS: Enabling GICv4 support\n");
 		gic_domain = domain;
-		vpe_domain_ops = ops;
+		vpe_domain_ops = vpe_ops;
+		sgi_domain_ops = sgi_ops;
 		return 0;
 	}
 
include/linux/irqchip/arm-gic-v4.h +8 -1
index edbaa37fd3f1..03bbd0aed2e2 100644
@@ -47,6 +47,11 @@ struct its_vpe {
 		};
 		/* GICv4.1 implementations */
 		struct {
+			struct {
+				u8	priority;
+				bool	enabled;
+				bool	group;
+			}			sgi_config[16];
 			atomic_t vmapp_count;
 		};
 	};
@@ -116,6 +121,8 @@ int its_unmap_vlpi(int irq);
 int its_prop_update_vlpi(int irq, u8 config, bool inv);
 
 struct irq_domain_ops;
-int its_init_v4(struct irq_domain *domain, const struct irq_domain_ops *ops);
+int its_init_v4(struct irq_domain *domain,
+		const struct irq_domain_ops *vpe_ops,
+		const struct irq_domain_ops *sgi_ops);
 
 #endif
Signature · 1 line
2.20.1
author maintainer
#46 lore raw

PATCH 24/35 irqchip/gic-v4.1: Add initial SGI configuration

In reply to Marc Zyngier

The GICv4.1 ITS has yet another new command (VSGI) which allows a VPE-targeted SGI to be configured (or have its pending state cleared). Add support for this command and plumb it into the activate irqdomain callback so that it is ready to be used.

---
 drivers/irqchip/irq-gic-v3-its.c   | 88 ++++++++++++++++++++++++++++++
 include/linux/irqchip/arm-gic-v3.h |  3 +-
 2 files changed, 90 insertions(+), 1 deletion(-)
drivers/irqchip/irq-gic-v3-its.c +88 -0
index 69c26be709be..5234b9eef8ad 100644
@@ -314,6 +314,15 @@ struct its_cmd_desc {
 		struct {
 			struct its_vpe *vpe;
 		} its_invdb_cmd;
+
+		struct {
+			struct its_vpe *vpe;
+			u8 sgi;
+			u8 priority;
+			bool enable;
+			bool group;
+			bool clear;
+		} its_vsgi_cmd;
 	};
 };
 
@@ -459,6 +468,31 @@ static void its_encode_db(struct its_cmd_block *cmd, bool db)
 	its_mask_encode(&cmd->raw_cmd[2], db, 63, 63);
 }
 
+static void its_encode_sgi_intid(struct its_cmd_block *cmd, u8 sgi)
+{
+	its_mask_encode(&cmd->raw_cmd[0], sgi, 35, 32);
+}
+
+static void its_encode_sgi_priority(struct its_cmd_block *cmd, u8 prio)
+{
+	its_mask_encode(&cmd->raw_cmd[0], prio >> 4, 23, 20);
+}
+
+static void its_encode_sgi_group(struct its_cmd_block *cmd, bool grp)
+{
+	its_mask_encode(&cmd->raw_cmd[0], grp, 10, 10);
+}
+
+static void its_encode_sgi_clear(struct its_cmd_block *cmd, bool clr)
+{
+	its_mask_encode(&cmd->raw_cmd[0], clr, 9, 9);
+}
+
+static void its_encode_sgi_enable(struct its_cmd_block *cmd, bool en)
+{
+	its_mask_encode(&cmd->raw_cmd[0], en, 8, 8);
+}
+
 static inline void its_fixup_cmd(struct its_cmd_block *cmd)
 {
 	/* Let's fixup BE commands */
@@ -770,6 +804,26 @@ static struct its_vpe *its_build_invdb_cmd(struct its_node *its,
 	return valid_vpe(its, desc->its_invdb_cmd.vpe);
 }
 
+static struct its_vpe *its_build_vsgi_cmd(struct its_node *its,
+					  struct its_cmd_block *cmd,
+					  struct its_cmd_desc *desc)
+{
+	if (WARN_ON(!is_v4_1(its)))
+		return NULL;
+
+	its_encode_cmd(cmd, GITS_CMD_VSGI);
+	its_encode_vpeid(cmd, desc->its_vsgi_cmd.vpe->vpe_id);
+	its_encode_sgi_intid(cmd, desc->its_vsgi_cmd.sgi);
+	its_encode_sgi_priority(cmd, desc->its_vsgi_cmd.priority);
+	its_encode_sgi_group(cmd, desc->its_vsgi_cmd.group);
+	its_encode_sgi_clear(cmd, desc->its_vsgi_cmd.clear);
+	its_encode_sgi_enable(cmd, desc->its_vsgi_cmd.enable);
+
+	its_fixup_cmd(cmd);
+
+	return valid_vpe(its, desc->its_vsgi_cmd.vpe);
+}
+
 static u64 its_cmd_ptr_to_offset(struct its_node *its,
 				 struct its_cmd_block *ptr)
 {
@@ -3574,6 +3628,38 @@ static struct irq_chip its_vpe_4_1_irq_chip = {
 	.irq_set_vcpu_affinity	= its_vpe_4_1_set_vcpu_affinity,
 };
 
+static struct its_node *find_4_1_its(void)
+{
+	static struct its_node *its = NULL;
+
+	if (!its) {
+		list_for_each_entry(its, &its_nodes, entry) {
+			if (is_v4_1(its))
+				return its;
+		}
+
+		/* Oops? */
+		its = NULL;
+	}
+
+	return its;
+}
+
+static void its_configure_sgi(struct irq_data *d, bool clear)
+{
+	struct its_vpe *vpe = irq_data_get_irq_chip_data(d);
+	struct its_cmd_desc desc;
+
+	desc.its_vsgi_cmd.vpe = vpe;
+	desc.its_vsgi_cmd.sgi = d->hwirq;
+	desc.its_vsgi_cmd.priority = vpe->sgi_config[d->hwirq].priority;
+	desc.its_vsgi_cmd.enable = vpe->sgi_config[d->hwirq].enabled;
+	desc.its_vsgi_cmd.group = vpe->sgi_config[d->hwirq].group;
+	desc.its_vsgi_cmd.clear = clear;
+
+	its_send_single_vcommand(find_4_1_its(), its_build_vsgi_cmd, &desc);
+}
+
 static int its_sgi_set_affinity(struct irq_data *d,
 				const struct cpumask *mask_val,
 				bool force)
@@ -3619,6 +3705,8 @@ static void its_sgi_irq_domain_free(struct irq_domain *domain,
 static int its_sgi_irq_domain_activate(struct irq_domain *domain,
 				       struct irq_data *d, bool reserve)
 {
+	/* Write out the initial SGI configuration */
+	its_configure_sgi(d, false);
 	return 0;
 }
 
include/linux/irqchip/arm-gic-v3.h +2 -1
index 5f3278cbf247..c73176d3ab2b 100644
@@ -497,8 +497,9 @@
 #define GITS_CMD_VMAPTI			GITS_CMD_GICv4(GITS_CMD_MAPTI)
 #define GITS_CMD_VMOVI			GITS_CMD_GICv4(GITS_CMD_MOVI)
 #define GITS_CMD_VSYNC			GITS_CMD_GICv4(GITS_CMD_SYNC)
-/* VMOVP and INVDB are the odd ones, as they dont have a physical counterpart */
+/* VMOVP, VSGI and INVDB are the odd ones, as they dont have a physical counterpart */
 #define GITS_CMD_VMOVP			GITS_CMD_GICv4(2)
+#define GITS_CMD_VSGI			GITS_CMD_GICv4(3)
 #define GITS_CMD_INVDB			GITS_CMD_GICv4(0xe)
 
 /*
Signature · 6 lines
2.20.1

_______________________________________________
kvmarm mailing list
[email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
author maintainer
#49 lore raw

PATCH 25/35 irqchip/gic-v4.1: Plumb mask/unmask SGI callbacks

In reply to Marc Zyngier

Implement mask/unmask for virtual SGIs by calling into the configuration helper.

---
 drivers/irqchip/irq-gic-v3-its.c | 18 ++++++++++++++++++
 1 file changed, 18 insertions(+)
drivers/irqchip/irq-gic-v3-its.c +18 -0
index 5234b9eef8ad..727a890f72ae 100644
@@ -3660,6 +3660,22 @@ static void its_configure_sgi(struct irq_data *d, bool clear)
 	its_send_single_vcommand(find_4_1_its(), its_build_vsgi_cmd, &desc);
 }
 
+static void its_sgi_mask_irq(struct irq_data *d)
+{
+	struct its_vpe *vpe = irq_data_get_irq_chip_data(d);
+
+	vpe->sgi_config[d->hwirq].enabled = false;
+	its_configure_sgi(d, false);
+}
+
+static void its_sgi_unmask_irq(struct irq_data *d)
+{
+	struct its_vpe *vpe = irq_data_get_irq_chip_data(d);
+
+	vpe->sgi_config[d->hwirq].enabled = true;
+	its_configure_sgi(d, false);
+}
+
 static int its_sgi_set_affinity(struct irq_data *d,
 				const struct cpumask *mask_val,
 				bool force)
@@ -3669,6 +3685,8 @@ static int its_sgi_set_affinity(struct irq_data *d,
 
 static struct irq_chip its_sgi_irq_chip = {
 	.name			= "GICv4.1-sgi",
+	.irq_mask		= its_sgi_mask_irq,
+	.irq_unmask		= its_sgi_unmask_irq,
 	.irq_set_affinity	= its_sgi_set_affinity,
 };
 
Signature · 6 lines
2.20.1

_______________________________________________
kvmarm mailing list
[email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
author maintainer
#50 lore raw

PATCH 26/35 irqchip/gic-v4.1: Plumb get/set_irqchip_state SGI callbacks

In reply to Marc Zyngier

To implement the get/set_irqchip_state callbacks (limited to the PENDING state), we have to use a particular set of hacks:

- Reading the pending state is done by using a pair of new redistributor registers (GICR_VSGIR, GICR_VSGIPENDR), which allow the 16 interrupts state to be retrieved.
- Setting the pending state is done by generating it as we'd otherwise do for a guest (writing to GITS_SGIR)
- Clearing the pending state is done by emiting a VSGI command with the "clear" bit set.

---
 drivers/irqchip/irq-gic-v3-its.c   | 56 ++++++++++++++++++++++++++++++
 include/linux/irqchip/arm-gic-v3.h | 14 ++++++++
 2 files changed, 70 insertions(+)
drivers/irqchip/irq-gic-v3-its.c +56 -0
index 727a890f72ae..5e67dfe1c4b1 100644
@@ -3683,11 +3683,67 @@ static int its_sgi_set_affinity(struct irq_data *d,
 	return -EINVAL;
 }
 
+static int its_sgi_set_irqchip_state(struct irq_data *d,
+				     enum irqchip_irq_state which,
+				     bool state)
+{
+	if (which != IRQCHIP_STATE_PENDING)
+		return -EINVAL;
+
+	if (state) {
+		struct its_vpe *vpe = irq_data_get_irq_chip_data(d);
+		struct its_node *its = find_4_1_its();
+		u64 val;
+
+		val  = FIELD_PREP(GITS_SGIR_VPEID, vpe->vpe_id);
+		val |= FIELD_PREP(GITS_SGIR_VINTID, d->hwirq);
+		writeq_relaxed(val, its->sgir_base + GITS_SGIR - SZ_128K);
+	} else {
+		its_configure_sgi(d, true);
+	}
+
+	return 0;
+}
+
+static int its_sgi_get_irqchip_state(struct irq_data *d,
+				     enum irqchip_irq_state which, bool *val)
+{
+	struct its_vpe *vpe = irq_data_get_irq_chip_data(d);
+	void __iomem *base = gic_data_rdist_cpu(vpe->col_idx)->rd_base + SZ_128K;
+	u32 count = 1000000;	/* 1s! */
+	u32 status;
+
+	if (which != IRQCHIP_STATE_PENDING)
+		return -EINVAL;
+
+	writel_relaxed(vpe->vpe_id, base + GICR_VSGIR);
+	do {
+		status = readl_relaxed(base + GICR_VSGIPENDR);
+		if (!(status & GICR_VSGIPENDR_BUSY))
+			goto out;
+
+		count--;
+		if (!count) {
+			pr_err_ratelimited("Unable to get SGI status\n");
+			goto out;
+		}
+		cpu_relax();
+		udelay(1);
+	} while(count);
+
+out:
+	*val = !!(status & (1 << d->hwirq));
+
+	return 0;
+}
+
 static struct irq_chip its_sgi_irq_chip = {
 	.name			= "GICv4.1-sgi",
 	.irq_mask		= its_sgi_mask_irq,
 	.irq_unmask		= its_sgi_unmask_irq,
 	.irq_set_affinity	= its_sgi_set_affinity,
+	.irq_set_irqchip_state	= its_sgi_set_irqchip_state,
+	.irq_get_irqchip_state	= its_sgi_get_irqchip_state,
 };
 
 static int its_sgi_irq_domain_alloc(struct irq_domain *domain,
include/linux/irqchip/arm-gic-v3.h +14 -0
index c73176d3ab2b..cb8563554ed2 100644
@@ -340,6 +340,15 @@
 #define GICR_VPENDBASER_4_1_VGRP1EN	(1ULL << 58)
 #define GICR_VPENDBASER_4_1_VPEID	GENMASK_ULL(15, 0)
 
+#define GICR_VSGIR			0x0080
+
+#define GICR_VSGIR_VPEID		GENMASK(15, 0)
+
+#define GICR_VSGIPENDR			0x0088
+
+#define GICR_VSGIPENDR_BUSY		(1U << 31)
+#define GICR_VSGIPENDR_PENDING		GENMASK(15, 0)
+
 /*
  * ITS registers, offsets from ITS_base
  */
@@ -363,6 +372,11 @@
 
 #define GITS_TRANSLATER			0x10040
 
+#define GITS_SGIR			0x20020
+
+#define GITS_SGIR_VPEID			GENMASK_ULL(47, 32)
+#define GITS_SGIR_VINTID		GENMASK_ULL(7, 0)
+
 #define GITS_CTLR_ENABLE		(1U << 0)
 #define GITS_CTLR_ImDe			(1U << 1)
 #define	GITS_CTLR_ITS_NUMBER_SHIFT	4
Signature · 6 lines
2.20.1

_______________________________________________
kvmarm mailing list
[email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
author maintainer
#51 lore raw

PATCH 27/35 irqchip/gic-v4.1: Plumb set_vcpu_affinity SGI callbacks

In reply to Marc Zyngier

As for VLPIs, there is a number of configuration bits that cannot be directly communicated through the normal irqchip API, and we have to use our good old friend set_vcpu_affinity.

This is used to configure group and priority for a given vSGI.

---
 drivers/irqchip/irq-gic-v3-its.c   | 18 ++++++++++++++++++
 include/linux/irqchip/arm-gic-v4.h |  5 +++++
 2 files changed, 23 insertions(+)
drivers/irqchip/irq-gic-v3-its.c +18 -0
index 5e67dfe1c4b1..4aae9582182b 100644
@@ -3737,6 +3737,23 @@ static int its_sgi_get_irqchip_state(struct irq_data *d,
 	return 0;
 }
 
+static int its_sgi_set_vcpu_affinity(struct irq_data *d, void *vcpu_info)
+{
+	struct its_vpe *vpe = irq_data_get_irq_chip_data(d);
+	struct its_cmd_info *info = vcpu_info;
+
+	switch (info->cmd_type) {
+	case PROP_UPDATE_SGI:
+		vpe->sgi_config[d->hwirq].priority = info->priority;
+		vpe->sgi_config[d->hwirq].group = info->group;
+		its_configure_sgi(d, false);
+		return 0;
+
+	default:
+		return -EINVAL;
+	}
+}
+
 static struct irq_chip its_sgi_irq_chip = {
 	.name			= "GICv4.1-sgi",
 	.irq_mask		= its_sgi_mask_irq,
@@ -3744,6 +3761,7 @@ static struct irq_chip its_sgi_irq_chip = {
 	.irq_set_affinity	= its_sgi_set_affinity,
 	.irq_set_irqchip_state	= its_sgi_set_irqchip_state,
 	.irq_get_irqchip_state	= its_sgi_get_irqchip_state,
+	.irq_set_vcpu_affinity	= its_sgi_set_vcpu_affinity,
 };
 
 static int its_sgi_irq_domain_alloc(struct irq_domain *domain,
include/linux/irqchip/arm-gic-v4.h +5 -0
index 03bbd0aed2e2..6185926e4582 100644
@@ -96,6 +96,7 @@ enum its_vcpu_info_cmd_type {
 	SCHEDULE_VPE,
 	DESCHEDULE_VPE,
 	INVALL_VPE,
+	PROP_UPDATE_SGI,
 };
 
 struct its_cmd_info {
@@ -108,6 +109,10 @@ struct its_cmd_info {
 			bool		g0en;
 			bool		g1en;
 		};
+		struct {
+			u8		priority;
+			bool		group;
+		};
 	};
 };
 
Signature · 6 lines
2.20.1

_______________________________________________
kvmarm mailing list
[email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
author maintainer
#52 lore raw

PATCH 28/35 irqchip/gic-v4.1: Move doorbell management to the GICv4 abstraction layer

In reply to Marc Zyngier

In order to hide some of the differences between v4.0 and v4.1, move the doorbell management out of the KVM code, and into the GICv4-specific layer. This allows the calling code to ask for the doorbell when blocking, and otherwise to leave the doorbell permanently disabled.

This matches the v4.1 code perfectly, and only results in a minor refactoring of the v4.0 code.

---
 drivers/irqchip/irq-gic-v4.c       | 49 +++++++++++++++++++++++++++---
 include/kvm/arm_vgic.h             |  1 +
 include/linux/irqchip/arm-gic-v4.h |  3 +-
 virt/kvm/arm/vgic/vgic-v3.c        |  1 +
 virt/kvm/arm/vgic/vgic-v4.c        | 34 +++++++++------------
 5 files changed, 63 insertions(+), 25 deletions(-)
drivers/irqchip/irq-gic-v4.c +45 -4
index c01910d53f9e..60b51d6683aa 100644
@@ -87,6 +87,11 @@ static struct irq_domain *gic_domain;
 static const struct irq_domain_ops *vpe_domain_ops;
 static const struct irq_domain_ops *sgi_domain_ops;
 
+static bool has_v4_1(void)
+{
+	return !!sgi_domain_ops;
+}
+
 int its_alloc_vcpu_irqs(struct its_vm *vm)
 {
 	int vpe_base_irq, i;
@@ -139,18 +144,54 @@ static int its_send_vpe_cmd(struct its_vpe *vpe, struct its_cmd_info *info)
 	return irq_set_vcpu_affinity(vpe->irq, info);
 }
 
-int its_schedule_vpe(struct its_vpe *vpe, bool on)
+int its_make_vpe_non_resident(struct its_vpe *vpe, bool db)
 {
-	struct its_cmd_info info;
+	struct irq_desc *desc = irq_to_desc(vpe->irq);
+	struct its_cmd_info info = { };
 	int ret;
 
 	WARN_ON(preemptible());
 
-	info.cmd_type = on ? SCHEDULE_VPE : DESCHEDULE_VPE;
+	info.cmd_type = DESCHEDULE_VPE;
+	if (has_v4_1()) {
+		/* GICv4.1 can directly deal with doorbells */
+		info.req_db = db;
+	} else {
+		/* Undo the nested disable_irq() calls... */
+		while (db && irqd_irq_disabled(&desc->irq_data))
+			enable_irq(vpe->irq);
+	}
+
+	ret = its_send_vpe_cmd(vpe, &info);
+	if (!ret)
+		vpe->resident = false;
+
+	return ret;
+}
+
+int its_make_vpe_resident(struct its_vpe *vpe)
+{
+	struct its_cmd_info info = { };
+	int ret;
+
+	WARN_ON(preemptible());
+
+	info.cmd_type = SCHEDULE_VPE;
+	if (has_v4_1()) {
+		/*
+		 * FIXME: This needs to come from the distributor, and
+		 * not be blindly set out of thin air...
+		 */
+		info.g0en = true;
+		info.g1en = true;
+	} else {
+		/* Disabled the doorbell, as we're about to enter the guest */
+		disable_irq_nosync(vpe->irq);
+	}
 
 	ret = its_send_vpe_cmd(vpe, &info);
 	if (!ret)
-		vpe->resident = on;
+		vpe->resident = true;
 
 	return ret;
 }
include/kvm/arm_vgic.h +1 -0
index 4dc58d7a0010..86cb95171eec 100644
@@ -70,6 +70,7 @@ struct vgic_global {
 
 	/* Hardware has GICv4? */
 	bool			has_gicv4;
+	bool			has_gicv4_1;
 
 	/* GIC system register CPU interface */
 	struct static_key_false gicv3_cpuif;
include/linux/irqchip/arm-gic-v4.h +2 -1
index 6185926e4582..084890a5c2bf 100644
@@ -118,7 +118,8 @@ struct its_cmd_info {
 
 int its_alloc_vcpu_irqs(struct its_vm *vm);
 void its_free_vcpu_irqs(struct its_vm *vm);
-int its_schedule_vpe(struct its_vpe *vpe, bool on);
+int its_make_vpe_resident(struct its_vpe *vpe);
+int its_make_vpe_non_resident(struct its_vpe *vpe, bool db);
 int its_invall_vpe(struct its_vpe *vpe);
 int its_map_vlpi(int irq, struct its_vlpi_map *map);
 int its_get_vlpi(int irq, struct its_vlpi_map *map);
virt/kvm/arm/vgic/vgic-v3.c +1 -0
index 48307a9eb1d8..07a33b3e71c5 100644
@@ -595,6 +595,7 @@ int vgic_v3_probe(const struct gic_kvm_info *info)
 	/* GICv4 support? */
 	if (info->has_v4) {
 		kvm_vgic_global_state.has_gicv4 = gicv4_enable;
+		kvm_vgic_global_state.has_gicv4_1 = info->has_v4_1;
 		kvm_info("GICv4 support %sabled\n",
 			 gicv4_enable ? "en" : "dis");
 	}
virt/kvm/arm/vgic/vgic-v4.c +14 -20
index 7e1f3202968a..50f84f4ce903 100644
@@ -67,10 +67,10 @@
  * it. And if we've migrated our vcpu from one CPU to another, we must
  * tell the ITS (so that the messages reach the right redistributor).
  * This is done in two steps: first issue a irq_set_affinity() on the
- * irq corresponding to the vcpu, then call its_schedule_vpe(). You
- * must be in a non-preemptible context. On exit, another call to
- * its_schedule_vpe() tells the redistributor that we're done with the
- * vcpu.
+ * irq corresponding to the vcpu, then call its_make_vpe_resident().
+ * You must be in a non-preemptible context. On exit, a call to
+ * its_make_vpe_non_resident() tells the redistributor that we're done
+ * with the vcpu.
  *
  * Finally, the doorbell handling: Each vcpu is allocated an interrupt
  * which will fire each time a VLPI is made pending whilst the vcpu is
@@ -86,7 +86,8 @@ static irqreturn_t vgic_v4_doorbell_handler(int irq, void *info)
 	struct kvm_vcpu *vcpu = info;
 
 	/* We got the message, no need to fire again */
-	if (!irqd_irq_disabled(&irq_to_desc(irq)->irq_data))
+	if (!kvm_vgic_global_state.has_gicv4_1 &&
+	    !irqd_irq_disabled(&irq_to_desc(irq)->irq_data))
 		disable_irq_nosync(irq);
 
 	vcpu->arch.vgic_cpu.vgic_v3.its_vpe.pending_last = true;
@@ -199,19 +200,11 @@ void vgic_v4_teardown(struct kvm *kvm)
 int vgic_v4_put(struct kvm_vcpu *vcpu, bool need_db)
 {
 	struct its_vpe *vpe = &vcpu->arch.vgic_cpu.vgic_v3.its_vpe;
-	struct irq_desc *desc = irq_to_desc(vpe->irq);
 
 	if (!vgic_supports_direct_msis(vcpu->kvm) || !vpe->resident)
 		return 0;
 
-	/*
-	 * If blocking, a doorbell is required. Undo the nested
-	 * disable_irq() calls...
-	 */
-	while (need_db && irqd_irq_disabled(&desc->irq_data))
-		enable_irq(vpe->irq);
-
-	return its_schedule_vpe(vpe, false);
+	return its_make_vpe_non_resident(vpe, need_db);
 }
 
 int vgic_v4_load(struct kvm_vcpu *vcpu)
@@ -232,18 +225,19 @@ int vgic_v4_load(struct kvm_vcpu *vcpu)
 	if (err)
 		return err;
 
-	/* Disabled the doorbell, as we're about to enter the guest */
-	disable_irq_nosync(vpe->irq);
-
-	err = its_schedule_vpe(vpe, true);
+	err = its_make_vpe_resident(vpe);
 	if (err)
 		return err;
 
 	/*
 	 * Now that the VPE is resident, let's get rid of a potential
-	 * doorbell interrupt that would still be pending.
+	 * doorbell interrupt that would still be pending. This is a
+	 * GICv4.0 only "feature"...
 	 */
-	return irq_set_irqchip_state(vpe->irq, IRQCHIP_STATE_PENDING, false);
+	if (!kvm_vgic_global_state.has_gicv4_1)
+		err = irq_set_irqchip_state(vpe->irq, IRQCHIP_STATE_PENDING, false);
+
+	return err;
 }
 
 static struct vgic_its *vgic_get_its(struct kvm *kvm,
Signature · 6 lines
2.20.1

_______________________________________________
kvmarm mailing list
[email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
author maintainer
#53 lore raw

PATCH 29/35 irqchip/gic-v4.1: Add VSGI allocation/teardown

In reply to Marc Zyngier

Allocate per-VPE SGIs when initializing the GIC-specific part of the VPE data structure.

---
 drivers/irqchip/irq-gic-v4.c       | 68 +++++++++++++++++++++++++++++-
 include/linux/irqchip/arm-gic-v4.h |  2 +
 2 files changed, 69 insertions(+), 1 deletion(-)
drivers/irqchip/irq-gic-v4.c +67 -1
index 60b51d6683aa..a29a063861bc 100644
@@ -92,6 +92,47 @@ static bool has_v4_1(void)
 	return !!sgi_domain_ops;
 }
 
+static int its_alloc_vcpu_sgis(struct its_vpe *vpe, int idx)
+{
+	char *name;
+	int sgi_base;
+
+	if (!has_v4_1())
+		return 0;
+
+	name = kasprintf(GFP_KERNEL, "GICv4-sgi-%d", task_pid_nr(current));
+	if (!name)
+		goto err;
+
+	vpe->fwnode = irq_domain_alloc_named_id_fwnode(name, idx);
+	if (!vpe->fwnode)
+		goto err;
+
+	kfree(name);
+	name = NULL;
+
+	vpe->sgi_domain = irq_domain_create_linear(vpe->fwnode, 16,
+						   sgi_domain_ops, vpe);
+	if (!vpe->sgi_domain)
+		goto err;
+
+	sgi_base = __irq_domain_alloc_irqs(vpe->sgi_domain, -1, 16,
+					       NUMA_NO_NODE, vpe,
+					       false, NULL);
+	if (sgi_base <= 0)
+		goto err;
+
+	return 0;
+
+err:
+	if (vpe->sgi_domain)
+		irq_domain_remove(vpe->sgi_domain);
+	if (vpe->fwnode)
+		irq_domain_free_fwnode(vpe->fwnode);
+	kfree(name);
+	return -ENOMEM;
+}
+
 int its_alloc_vcpu_irqs(struct its_vm *vm)
 {
 	int vpe_base_irq, i;
@@ -118,8 +159,13 @@ int its_alloc_vcpu_irqs(struct its_vm *vm)
 	if (vpe_base_irq <= 0)
 		goto err;
 
-	for (i = 0; i < vm->nr_vpes; i++)
+	for (i = 0; i < vm->nr_vpes; i++) {
+		int ret;
 		vm->vpes[i]->irq = vpe_base_irq + i;
+		ret = its_alloc_vcpu_sgis(vm->vpes[i], i);
+		if (ret)
+			goto err;
+	}
 
 	return 0;
 
@@ -132,8 +178,28 @@ int its_alloc_vcpu_irqs(struct its_vm *vm)
 	return -ENOMEM;
 }
 
+static void its_free_sgi_irqs(struct its_vm *vm)
+{
+	int i;
+
+	if (!has_v4_1())
+		return;
+
+	for (i = 0; i < vm->nr_vpes; i++) {
+		unsigned int irq = irq_find_mapping(vm->vpes[i]->sgi_domain, 0);
+
+		if (WARN_ON(!irq))
+			continue;
+
+		irq_domain_free_irqs(irq, 16);
+		irq_domain_remove(vm->vpes[i]->sgi_domain);
+		irq_domain_free_fwnode(vm->vpes[i]->fwnode);
+	}
+}
+
 void its_free_vcpu_irqs(struct its_vm *vm)
 {
+	its_free_sgi_irqs(vm);
 	irq_domain_free_irqs(vm->vpes[0]->irq, vm->nr_vpes);
 	irq_domain_remove(vm->domain);
 	irq_domain_free_fwnode(vm->fwnode);
include/linux/irqchip/arm-gic-v4.h +2 -0
index 084890a5c2bf..5578cbe7430b 100644
@@ -47,6 +47,8 @@ struct its_vpe {
 		};
 		/* GICv4.1 implementations */
 		struct {
+			struct fwnode_handle	*fwnode;
+			struct irq_domain	*sgi_domain;
 			struct {
 				u8	priority;
 				bool	enabled;
Signature · 6 lines
2.20.1

_______________________________________________
kvmarm mailing list
[email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
author maintainer
#54 lore raw

PATCH 30/35 irqchip/gic-v4.1: Add VSGI property setup

In reply to Marc Zyngier

Add the SGI configuration entry point for KVM to use.

---
 drivers/irqchip/irq-gic-v4.c       | 13 +++++++++++++
 include/linux/irqchip/arm-gic-v4.h |  1 +
 2 files changed, 14 insertions(+)
drivers/irqchip/irq-gic-v4.c +13 -0
index a29a063861bc..b937e51a9178 100644
@@ -324,6 +324,19 @@ int its_prop_update_vlpi(int irq, u8 config, bool inv)
 	return irq_set_vcpu_affinity(irq, &info);
 }
 
+int its_prop_update_vsgi(int irq, u8 priority, bool group)
+{
+	struct its_cmd_info info = {
+		.cmd_type = PROP_UPDATE_SGI,
+		{
+			.priority	= priority,
+			.group		= group,
+		},
+	};
+
+	return irq_set_vcpu_affinity(irq, &info);
+}
+
 int its_init_v4(struct irq_domain *domain,
 		const struct irq_domain_ops *vpe_ops,
 		const struct irq_domain_ops *sgi_ops)
include/linux/irqchip/arm-gic-v4.h +1 -0
index 5578cbe7430b..b894796df9ab 100644
@@ -127,6 +127,7 @@ int its_map_vlpi(int irq, struct its_vlpi_map *map);
 int its_get_vlpi(int irq, struct its_vlpi_map *map);
 int its_unmap_vlpi(int irq);
 int its_prop_update_vlpi(int irq, u8 config, bool inv);
+int its_prop_update_vsgi(int irq, u8 priority, bool group);
 
 struct irq_domain_ops;
 int its_init_v4(struct irq_domain *domain,
Signature · 6 lines
2.20.1

_______________________________________________
kvmarm mailing list
[email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
author maintainer
#55 lore raw

PATCH 31/35 irqchip/gic-v4.1: Eagerly vmap vPEs

In reply to Marc Zyngier

Now that we have HW-accelerated SGIs being delivered to VPEs, it becomes required to map the VPEs on all ITSs instead of relying on the lazy approach that we would use when using the ITS-list mechanism.

---
 drivers/irqchip/irq-gic-v3-its.c | 39 +++++++++++++++++++++++++-------
 1 file changed, 31 insertions(+), 8 deletions(-)
drivers/irqchip/irq-gic-v3-its.c +31 -8
index 4aae9582182b..a1e8c4c2598a 100644
@@ -1417,12 +1417,31 @@ static int its_irq_set_irqchip_state(struct irq_data *d,
 	return 0;
 }
 
+/*
+ * Two favourable cases:
+ *
+ * (a) Either we have a GICv4.1, and all vPEs have to be mapped at all times
+ *     for vSGI delivery
+ *
+ * (b) Or the ITSs do not use a list map, meaning that VMOVP is cheap enough
+ *     and we're better off mapping all VPEs always
+ *
+ * If neither (a) nor (b) is true, then we map VLPIs on demand.
+ *
+ */
+static bool gic_requires_eager_mapping(void)
+{
+	if (!its_list_map || gic_rdists->has_rvpeid)
+		return true;
+
+	return false;
+}
+
 static void its_map_vm(struct its_node *its, struct its_vm *vm)
 {
 	unsigned long flags;
 
-	/* Not using the ITS list? Everything is always mapped. */
-	if (!its_list_map)
+	if (gic_requires_eager_mapping())
 		return;
 
 	raw_spin_lock_irqsave(&vmovp_lock, flags);
@@ -1456,7 +1475,7 @@ static void its_unmap_vm(struct its_node *its, struct its_vm *vm)
 	unsigned long flags;
 
 	/* Not using the ITS list? Everything is always mapped. */
-	if (!its_list_map)
+	if (gic_requires_eager_mapping())
 		return;
 
 	raw_spin_lock_irqsave(&vmovp_lock, flags);
@@ -3957,8 +3976,12 @@ static int its_vpe_irq_domain_activate(struct irq_domain *domain,
 	struct its_vpe *vpe = irq_data_get_irq_chip_data(d);
 	struct its_node *its;
 
-	/* If we use the list map, we issue VMAPP on demand... */
-	if (its_list_map)
+	/*
+	 * If we use the list map, we issue VMAPP on demand... Unless
+	 * we're on a GICv4.1 and we eagerly map the VPE on all ITSs
+	 * so that VSGIs can work.
+	 */
+	if (!gic_requires_eager_mapping())
 		return 0;
 
 	/* Map the VPE to the first possible CPU */
@@ -3984,10 +4007,10 @@ static void its_vpe_irq_domain_deactivate(struct irq_domain *domain,
 	struct its_node *its;
 
 	/*
-	 * If we use the list map, we unmap the VPE once no VLPIs are
-	 * associated with the VM.
+	 * If we use the list map on GICv4.0, we unmap the VPE once no
+	 * VLPIs are associated with the VM.
 	 */
-	if (its_list_map)
+	if (!gic_requires_eager_mapping())
 		return;
 
 	list_for_each_entry(its, &its_nodes, entry) {
Signature · 6 lines
2.20.1

_______________________________________________
kvmarm mailing list
[email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
author maintainer
#58 lore raw

PATCH 32/35 KVM: arm64: GICv4.1: Let doorbells be auto-enabled

In reply to Marc Zyngier

As GICv4.1 understands the life cycle of doorbells (instead of just randomly firing them at the most inconvenient time), just enable them at irq_request time, and be done with it.

---
 virt/kvm/arm/vgic/vgic-v4.c | 9 ++++++++-
 1 file changed, 8 insertions(+), 1 deletion(-)
virt/kvm/arm/vgic/vgic-v4.c +8 -1
index 50f84f4ce903..e61c7a149515 100644
@@ -141,6 +141,7 @@ int vgic_v4_init(struct kvm *kvm)
 
 	kvm_for_each_vcpu(i, vcpu, kvm) {
 		int irq = dist->its_vm.vpes[i]->irq;
+		unsigned long irq_flags = DB_IRQ_FLAGS;
 
 		/*
 		 * Don't automatically enable the doorbell, as we're
@@ -148,8 +149,14 @@ int vgic_v4_init(struct kvm *kvm)
 		 * blocked. Also disable the lazy disabling, as the
 		 * doorbell could kick us out of the guest too
 		 * early...
+		 *
+		 * On GICv4.1, the doorbell is managed in HW and must
+		 * be left enabled.
 		 */
-		irq_set_status_flags(irq, DB_IRQ_FLAGS);
+		if (kvm_vgic_global_state.has_gicv4_1)
+			irq_flags &= ~IRQ_NOAUTOEN;
+		irq_set_status_flags(irq, irq_flags);
+
 		ret = request_irq(irq, vgic_v4_doorbell_handler,
 				  0, "vcpu", vcpu);
 		if (ret) {
Signature · 6 lines
2.20.1

_______________________________________________
kvmarm mailing list
[email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
author maintainer
#59 lore raw

PATCH 33/35 KVM: arm64: GICv4.1: Add direct injection capability to SGI registers

In reply to Marc Zyngier

Most of the GICv3 emulation code that deals with SGIs now has to be aware of the v4.1 capabilities in order to benefit from it.

Add such support, keyed on the interrupt having the hw flag set and being a SGI.

---
 virt/kvm/arm/vgic/vgic-mmio-v3.c | 15 +++++-
 virt/kvm/arm/vgic/vgic-mmio.c    | 88 ++++++++++++++++++++++++++++++--
 2 files changed, 96 insertions(+), 7 deletions(-)
virt/kvm/arm/vgic/vgic-mmio-v3.c +13 -2
index 7dfd15dbb308..d73f4ffd7d36 100644
@@ -6,6 +6,7 @@
 #include <linux/irqchip/arm-gic-v3.h>
 #include <linux/kvm.h>
 #include <linux/kvm_host.h>
+#include <linux/interrupt.h>
 #include <kvm/iodev.h>
 #include <kvm/arm_vgic.h>
 
@@ -939,8 +940,18 @@ void vgic_v3_dispatch_sgi(struct kvm_vcpu *vcpu, u64 reg, bool allow_group1)
 		 * generate interrupts of either group.
 		 */
 		if (!irq->group || allow_group1) {
-			irq->pending_latch = true;
-			vgic_queue_irq_unlock(vcpu->kvm, irq, flags);
+			if (!irq->hw) {
+				irq->pending_latch = true;
+				vgic_queue_irq_unlock(vcpu->kvm, irq, flags);
+			} else {
+				/* HW SGI? Ask the GIC to inject it */
+				int err;
+				err = irq_set_irqchip_state(irq->host_irq,
+							    IRQCHIP_STATE_PENDING,
+							    true);
+				WARN_RATELIMIT(err, "IRQ %d", irq->host_irq);
+				raw_spin_unlock_irqrestore(&irq->irq_lock, flags);
+			}
 		} else {
 			raw_spin_unlock_irqrestore(&irq->irq_lock, flags);
 		}
virt/kvm/arm/vgic/vgic-mmio.c +83 -5

151 lines not on this page · raw message

Signature · 1 line
2.20.1
author maintainer
#60 lore raw

PATCH 34/35 KVM: arm64: GICv4.1: Configure SGIs as HW interrupts

In reply to Marc Zyngier

To enable HW delivery of SGIs, set their hw flag to true, link them to the corresponding Linux interrupt, which is then activated.

This configures the SGIs at the ITS level, and put the show on the road!

---
 virt/kvm/arm/vgic/vgic-v4.c | 42 +++++++++++++++++++++++++++++++++++++
 1 file changed, 42 insertions(+)
virt/kvm/arm/vgic/vgic-v4.c +42 -0
index e61c7a149515..0ff6dac16cba 100644
@@ -97,6 +97,46 @@ static irqreturn_t vgic_v4_doorbell_handler(int irq, void *info)
 	return IRQ_HANDLED;
 }
 
+static void vgic_v4_sync_sgi_config(struct its_vpe *vpe, struct vgic_irq *irq)
+{
+	vpe->sgi_config[irq->intid].enabled	= irq->enabled;
+	vpe->sgi_config[irq->intid].group 	= irq->group;
+	vpe->sgi_config[irq->intid].priority	= irq->priority;
+}
+
+static void vgic_v4_init_vsgis(struct kvm_vcpu *vcpu)
+{
+	struct its_vpe *vpe = &vcpu->arch.vgic_cpu.vgic_v3.its_vpe;
+	int i;
+
+	if (!kvm_vgic_global_state.has_gicv4_1)
+		return;
+
+	/*
+	 * With GICv4.1, every virtual SGI can be directly injected. So
+	 * let's pretend that they are HW interrupts, tied to a host
+	 * IRQ. The SGI code will do its magic.
+	 */
+	for (i = 0; i < VGIC_NR_SGIS; i++) {
+		struct vgic_irq *irq = vgic_get_irq(vcpu->kvm, vcpu, i);
+		struct irq_desc *desc;
+		int ret;
+
+		irq->hw = true;
+		irq->host_irq = irq_find_mapping(vpe->sgi_domain, i);
+		vgic_v4_sync_sgi_config(vpe, irq);
+		/*
+		 * SGIs are initialised as disabled. Enable them if
+		 * required by the rest of the VGIC init code.
+		 */
+		desc = irq_to_desc(irq->host_irq);
+		ret = irq_domain_activate_irq(irq_desc_get_irq_data(desc),
+					      false);
+		WARN_ON(ret);
+		vgic_put_irq(vcpu->kvm, irq);
+	}
+}
+
 /**
  * vgic_v4_init - Initialize the GICv4 data structures
  * @kvm:	Pointer to the VM being initialized
@@ -168,6 +208,8 @@ int vgic_v4_init(struct kvm *kvm)
 			dist->its_vm.nr_vpes = i;
 			break;
 		}
+
+		vgic_v4_init_vsgis(vcpu);
 	}
 
 	if (ret)
Signature · 1 line
2.20.1
author maintainer
#61 lore raw

PATCH 35/35 KVM: arm64: GICv4.1: Expose HW-based SGIs in debugfs

In reply to Marc Zyngier

The vgic-state debugfs file could do with showing the pending state of the HW-backed SGIs. Plug it into the low-level code.

---
 virt/kvm/arm/vgic/vgic-debug.c | 14 +++++++++++++-
 1 file changed, 13 insertions(+), 1 deletion(-)
virt/kvm/arm/vgic/vgic-debug.c +13 -1
index cc12fe9b2df3..98f3242a34a6 100644
@@ -178,6 +178,8 @@ static void print_irq_state(struct seq_file *s, struct vgic_irq *irq,
 			    struct kvm_vcpu *vcpu)
 {
 	char *type;
+	bool pending;
+
 	if (irq->intid < VGIC_NR_SGIS)
 		type = "SGI";
 	else if (irq->intid < VGIC_NR_PRIVATE_IRQS)
@@ -190,6 +192,16 @@ static void print_irq_state(struct seq_file *s, struct vgic_irq *irq,
 	if (irq->intid ==0 || irq->intid == VGIC_NR_PRIVATE_IRQS)
 		print_header(s, irq, vcpu);
 
+	pending = irq->pending_latch;
+	if (irq->hw && vgic_irq_is_sgi(irq->intid)) {
+		int err;
+
+		err = irq_get_irqchip_state(irq->host_irq,
+					    IRQCHIP_STATE_PENDING,
+					    &pending);
+		WARN_ON_ONCE(err);
+	}
+		
 	seq_printf(s, "       %s %4d "
 		      "    %2d "
 		      "%d%d%d%d%d%d%d "
@@ -201,7 +213,7 @@ static void print_irq_state(struct seq_file *s, struct vgic_irq *irq,
 		      "\n",
 			type, irq->intid,
 			(irq->target_vcpu) ? irq->target_vcpu->vcpu_id : -1,
-			irq->pending_latch,
+		   	pending,
 			irq->line_level,
 			irq->active,
 			irq->enabled,
Signature · 1 line
2.20.1
Reviewed-by on v1 2/35
#3 lore raw

Re: [PATCH 02/35] irqchip/gic-v3-its: Factor out wait_for_syncr primitive

In reply to Marc Zyngier
Quoted 6 lines · Marc Zyngier

Waiting for a redistributor to have performed an operation is a common thing to do, and the idiom is already spread around. As we're going to make even more use of this, let's have a primitive that does just that.

Quoted 54 lines
---
 drivers/irqchip/irq-gic-v3-its.c | 15 +++++++++------
 1 file changed, 9 insertions(+), 6 deletions(-)
drivers/irqchip/irq-gic-v3-its.c +9 -6
index 62e54f1a248b..58cb233cf138 100644
@@ -1075,6 +1075,12 @@ static void lpi_write_config(struct irq_data *d, u8 clr, u8 set)
 		dsb(ishst);
 }
 
+static void wait_for_syncr(void __iomem *rdbase)
+{
+	while (gic_read_lpir(rdbase + GICR_SYNCR) & 1)
+		cpu_relax();
+}
+
 static void lpi_update_config(struct irq_data *d, u8 clr, u8 set)
 {
 	struct its_device *its_dev = irq_data_get_irq_chip_data(d);
@@ -2757,8 +2763,7 @@ static void its_vpe_db_proxy_move(struct its_vpe *vpe, int from, int to)
 
 		rdbase = per_cpu_ptr(gic_rdists->rdist, from)->rd_base;
 		gic_write_lpir(vpe->vpe_db_lpi, rdbase + GICR_CLRLPIR);
-		while (gic_read_lpir(rdbase + GICR_SYNCR) & 1)
-			cpu_relax();
+		wait_for_syncr(rdbase);
 
 		return;
 	}
@@ -2914,8 +2919,7 @@ static void its_vpe_send_inv(struct irq_data *d)
 
 		rdbase = per_cpu_ptr(gic_rdists->rdist, vpe->col_idx)->rd_base;
 		gic_write_lpir(vpe->vpe_db_lpi, rdbase + GICR_INVLPIR);
-		while (gic_read_lpir(rdbase + GICR_SYNCR) & 1)
-			cpu_relax();
+		wait_for_syncr(rdbase);
 	} else {
 		its_vpe_send_cmd(vpe, its_send_inv);
 	}
@@ -2957,8 +2961,7 @@ static int its_vpe_set_irqchip_state(struct irq_data *d,
 			gic_write_lpir(vpe->vpe_db_lpi, rdbase + GICR_SETLPIR);
 		} else {
 			gic_write_lpir(vpe->vpe_db_lpi, rdbase + GICR_CLRLPIR);
-			while (gic_read_lpir(rdbase + GICR_SYNCR) & 1)
-				cpu_relax();
+			wait_for_syncr(rdbase);
 		}
 	} else {
 		if (state)
Signature · 1 line
2.20.1
#10 lore raw

Re: [PATCH 04/35] irqchip/gic-v3: Detect GICv4.1 supporting RVPEID

In reply to Marc Zyngier
Quoted 18 lines · Marc Zyngier

GICv4.1 supports the RVPEID ("Residency per vPE ID"), which allows for a much efficient way of making virtual CPUs resident (to allow direct injection of interrupts).

The functionnality needs to be discovered on each and every redistributor in the system, and disabled if the settings are inconsistent.

---
 drivers/irqchip/irq-gic-v3.c       | 21 ++++++++++++++++++---
 include/linux/irqchip/arm-gic-v3.h |  2 ++
 2 files changed, 20 insertions(+), 3 deletions(-)
drivers/irqchip/irq-gic-v3.c +0 -0
index 422664ac5f53..0b545e2c3498 100644

drivers/irqchip/irq-gic-v3.c static int __gic_update_rdist_properties(struct redist_region *region, · line 855

Quoted 8 lines
drivers/irqchip/irq-gic-v3.c +3 -1
@@ -849,8 +849,21 @@ static int __gic_update_rdist_properties(struct redist_region *region,
 					 void __iomem *ptr)
 {
 	u64 typer = gic_read_typer(ptr + GICR_TYPER);
+
 	gic_data.rdists.has_vlpis &= !!(typer & GICR_TYPER_VLPIS);
-	gic_data.rdists.has_direct_lpi &= !!(typer & GICR_TYPER_DirectLPIS);
+
+	/* RVPEID implies some form of DirectLPI, no matter what the doc says... :-/ */

I think the doc says, RVPEID is *always* 1 for GICv4.1 (and presumably beyond) and when RVPEID==1 then DirectLPI is *always* 0 - but that's OK because for GICv4.1 support for direct LPIs is mandatory.

drivers/irqchip/irq-gic-v3.c static int __gic_update_rdist_properties(struct redist_region *region, · line 861

Quoted 6 lines
drivers/irqchip/irq-gic-v3.c +6 -0
+	gic_data.rdists.has_rvpeid &= !!(typer & GICR_TYPER_RVPEID);
+	gic_data.rdists.has_direct_lpi &= (!!(typer & GICR_TYPER_DirectLPIS) |
+					   gic_data.rdists.has_rvpeid);
+
+	/* Detect non-sensical configurations */
+	if (WARN_ON_ONCE(gic_data.rdists.has_rvpeid && !gic_data.rdists.has_vlpis)) {

How feasible is the following suitation? All the redistributors in the system has vlpis=0, and only the first redistributor has rvpeid=1 (with the remaining ones rvpeid=0). If we evaluate this WARN_ON_ONCE on each call to __gic_update_rdist_properties we end up without direct LPI support, however if we evaluated this after iterating through all the redistributors then we'd end up with direct LPI support and a non-essential WARN.

Should we do the WARN after iterating through all the redistributors once we know what the final values of these flags will be, perhaps in gic_update_rdist_properties?

Quoted 52 lines
drivers/irqchip/irq-gic-v3.c +9 -2
+		gic_data.rdists.has_direct_lpi = false;
+		gic_data.rdists.has_vlpis = false;
+		gic_data.rdists.has_rvpeid = false;
+	}
+
 	gic_data.ppi_nr = min(GICR_TYPER_NR_PPIS(typer), gic_data.ppi_nr);
 
 	return 1;
@@ -863,9 +876,10 @@ static void gic_update_rdist_properties(void)
 	if (WARN_ON(gic_data.ppi_nr == UINT_MAX))
 		gic_data.ppi_nr = 0;
 	pr_info("%d PPIs implemented\n", gic_data.ppi_nr);
-	pr_info("%sVLPI support, %sdirect LPI support\n",
+	pr_info("%sVLPI support, %sdirect LPI support, %sRVPEID support\n",
 		!gic_data.rdists.has_vlpis ? "no " : "",
-		!gic_data.rdists.has_direct_lpi ? "no " : "");
+		!gic_data.rdists.has_direct_lpi ? "no " : "",
+		!gic_data.rdists.has_rvpeid ? "no " : "");
 }
 
 /* Check whether it's single security state view */
@@ -1546,6 +1560,7 @@ static int __init gic_init_bases(void __iomem *dist_base,
 						 &gic_data);
 	irq_domain_update_bus_token(gic_data.domain, DOMAIN_BUS_WIRED);
 	gic_data.rdists.rdist = alloc_percpu(typeof(*gic_data.rdists.rdist));
+	gic_data.rdists.has_rvpeid = true;
 	gic_data.rdists.has_vlpis = true;
 	gic_data.rdists.has_direct_lpi = true;
 
include/linux/irqchip/arm-gic-v3.h +2 -0
index 5cc10cf7cb3e..b34e0c113697 100644
@@ -234,6 +234,7 @@
 #define GICR_TYPER_VLPIS		(1U << 1)
 #define GICR_TYPER_DirectLPIS		(1U << 3)
 #define GICR_TYPER_LAST			(1U << 4)
+#define GICR_TYPER_RVPEID		(1U << 7)
 
 #define GIC_V3_REDIST_SIZE		0x20000
 
@@ -613,6 +614,7 @@ struct rdists {
 	u64			flags;
 	u32			gicd_typer;
 	bool			has_vlpis;
+	bool			has_rvpeid;
 	bool			has_direct_lpi;
 };
 
Signature · 1 line
2.20.1
#15 lore raw

Re: [PATCH 05/35] irqchip/gic-v3: Add GICv4.1 VPEID size discovery

In reply to Marc Zyngier
Quoted 1 line · Marc Zyngier

While GICv4.0 mandates 16 bit worth of VPEIDs, GICv4.1 allows smaller

s/VPEIDs/vPEIDs/

Quoted 15 lines

implementations to be built. Add the required glue to dynamically compute the limit.

---
 drivers/irqchip/irq-gic-v3-its.c   | 11 ++++++++++-
 drivers/irqchip/irq-gic-v3.c       |  3 +++
 include/linux/irqchip/arm-gic-v3.h |  5 +++++
 3 files changed, 18 insertions(+), 1 deletion(-)
drivers/irqchip/irq-gic-v3-its.c +0 -0
index c94eb287393b..17b77a0b9d97 100644

drivers/irqchip/irq-gic-v3-its.c struct its_node { · line 125

Quoted 8 lines
drivers/irqchip/irq-gic-v3-its.c +4 -1
@@ -119,7 +119,16 @@ struct its_node {
 #define ITS_ITT_ALIGN		SZ_256
 
 /* The maximum number of VPEID bits supported by VLPI commands */
-#define ITS_MAX_VPEID_BITS	(16)
+#define ITS_MAX_VPEID_BITS						\
+	({								\
+		int nvpeid = 16;					\
+		if (gic_rdists->has_rvpeid &&				\

We use rvpeid as a way of determining if this is a GICv4.1, are there any other means of determining this? If we use it in this way, is there any benefit to having a has_gicv4_1 type of flag instead?

Also for 'insane' configurations we set has_rvpeid to false, thus preventing this feature. Does it make sense to do that?

GICD_TYPER2 is reserved in GICv4, however I understand this reads as RES0, can we just rely on that instead? (We read it below anyway).

Quoted 36 lines
drivers/irqchip/irq-gic-v3-its.c +6 -0
+		    gic_rdists->gicd_typer2 & GICD_TYPER2_VIL)		\
+			nvpeid = 1 + (gic_rdists->gicd_typer2 &		\
+				      GICD_TYPER2_VID);			\
+									\
+		nvpeid;							\
+	})
 #define ITS_MAX_VPEID		(1 << (ITS_MAX_VPEID_BITS))
 
 /* Convert page order to size in bytes */
drivers/irqchip/irq-gic-v3.c +3 -0
index 0b545e2c3498..fb6360161d6c 100644
@@ -1556,6 +1556,9 @@ static int __init gic_init_bases(void __iomem *dist_base,
 
 	pr_info("%d SPIs implemented\n", GIC_LINE_NR - 32);
 	pr_info("%d Extended SPIs implemented\n", GIC_ESPI_NR);
+
+	gic_data.rdists.gicd_typer2 = readl_relaxed(gic_data.dist_base + GICD_TYPER2);
+
 	gic_data.domain = irq_domain_create_tree(handle, &gic_irq_domain_ops,
 						 &gic_data);
 	irq_domain_update_bus_token(gic_data.domain, DOMAIN_BUS_WIRED);
include/linux/irqchip/arm-gic-v3.h +1 -0
index b34e0c113697..71730b9def0c 100644
@@ -13,6 +13,7 @@
 #define GICD_CTLR			0x0000
 #define GICD_TYPER			0x0004
 #define GICD_IIDR			0x0008
+#define GICD_TYPER2			0x000C
 #define GICD_STATUSR			0x0010
 #define GICD_SETSPI_NSR			0x0040
 #define GICD_CLRSPI_NSR			0x0048

include/linux/irqchip/arm-gic-v3.h line 94

Quoted 5 lines
include/linux/irqchip/arm-gic-v3.h +2 -0
@@ -89,6 +90,9 @@
 #define GICD_TYPER_ESPIS(typer)						\
 	(((typer) & GICD_TYPER_ESPI) ? GICD_TYPER_SPIS((typer) >> 27) : 0)
 
+#define GICD_TYPER2_VIL			(1U << 7)
+#define GICD_TYPER2_VID			GENMASK(4, 0)

Given that the 4th bit is reserved for future expansion and values greater than 0xF are reserved, is there value in changing this to GENMASK(3, 0)?

Quoted 15 lines
include/linux/irqchip/arm-gic-v3.h +2 -0
+
 #define GICD_IROUTER_SPI_MODE_ONE	(0U << 31)
 #define GICD_IROUTER_SPI_MODE_ANY	(1U << 31)
 
@@ -613,6 +617,7 @@ struct rdists {
 	void			*prop_table_va;
 	u64			flags;
 	u32			gicd_typer;
+	u32			gicd_typer2;
 	bool			has_vlpis;
 	bool			has_rvpeid;
 	bool			has_direct_lpi;
Signature · 1 line
2.20.1
author maintainer
#11 lore raw

Re: [PATCH 04/35] irqchip/gic-v3: Detect GICv4.1 supporting RVPEID

In reply to Andrew Murray
Quoted 23 lines · Andrew Murray
Quoted 22 lines · Marc Zyngier

GICv4.1 supports the RVPEID ("Residency per vPE ID"), which allows for a much efficient way of making virtual CPUs resident (to allow direct injection of interrupts).

The functionnality needs to be discovered on each and every redistributor in the system, and disabled if the settings are inconsistent.

---
 drivers/irqchip/irq-gic-v3.c       | 21 ++++++++++++++++++---
 include/linux/irqchip/arm-gic-v3.h |  2 ++
 2 files changed, 20 insertions(+), 3 deletions(-)
drivers/irqchip/irq-gic-v3.c +1 -0
index 422664ac5f53..0b545e2c3498 100644
@@ -849,8 +849,21 @@ static int __gic_update_rdist_properties(struct redist_region *region,
 					 void __iomem *ptr)
 {
 	u64 typer = gic_read_typer(ptr + GICR_TYPER);
+
Quoted 8 lines

drivers/irqchip/irq-gic-v3.c static int __gic_update_rdist_properties(struct redist_region *region, · line 855

Quoted 4 lines
drivers/irqchip/irq-gic-v3.c +2 -1
 	gic_data.rdists.has_vlpis &= !!(typer & GICR_TYPER_VLPIS);
-	gic_data.rdists.has_direct_lpi &= !!(typer & GICR_TYPER_DirectLPIS);
+
+	/* RVPEID implies some form of DirectLPI, no matter what the doc says... :-/ */

I think the doc says, RVPEID is *always* 1 for GICv4.1 (and presumably beyond) and when RVPEID==1 then DirectLPI is *always* 0 - but that's OK because for GICv4.1 support for direct LPIs is mandatory.

Well, v4.1 support for DirectLPI is pretty patchy. It has just enough features to make it useful.

Quoted 15 lines

drivers/irqchip/irq-gic-v3.c static int __gic_update_rdist_properties(struct redist_region *region, · line 861

Quoted 7 lines
drivers/irqchip/irq-gic-v3.c +6 -0
+	gic_data.rdists.has_rvpeid &= !!(typer & GICR_TYPER_RVPEID);
+	gic_data.rdists.has_direct_lpi &= (!!(typer & GICR_TYPER_DirectLPIS) |
+					   gic_data.rdists.has_rvpeid);
+
+	/* Detect non-sensical configurations */
+	if (WARN_ON_ONCE(gic_data.rdists.has_rvpeid && !gic_data.rdists.has_vlpis)) {

How feasible is the following suitation? All the redistributors in the system has vlpis=0, and only the first redistributor has rvpeid=1 (with the remaining ones rvpeid=0).If we evaluate this WARN_ON_ONCE on each call to __gic_update_rdist_properties we end up without direct LPI support, however if we evaluated this after iterating through all the redistributors then we'd end up with direct LPI support and a non-essential WARN.

Quoted 3 lines

Should we do the WARN after iterating through all the redistributors once we know what the final values of these flags will be, perhaps in gic_update_rdist_properties?

What does it gains us? The moment we've detected any inconsistency, any use of DirectLPI or VLPIs is a big nono, because the redistributors care not designed to communicate with each other, and we might as well do that early. Frankly, the HW should have stayed in someone's lab. The only reason I have that code in is to detect the FVP model being misconfigured, which is pretty easy to do.

	M.
Signature · 1 line
Jazz is not dead, it just smells funny...
#12 lore raw

Re: [PATCH 04/35] irqchip/gic-v3: Detect GICv4.1 supporting RVPEID

Quoted 51 lines · Marc Zyngier
Quoted 23 lines · Andrew Murray
Quoted 22 lines · Marc Zyngier

GICv4.1 supports the RVPEID ("Residency per vPE ID"), which allows for a much efficient way of making virtual CPUs resident (to allow direct injection of interrupts).

The functionnality needs to be discovered on each and every redistributor in the system, and disabled if the settings are inconsistent.

---
 drivers/irqchip/irq-gic-v3.c       | 21 ++++++++++++++++++---
 include/linux/irqchip/arm-gic-v3.h |  2 ++
 2 files changed, 20 insertions(+), 3 deletions(-)
drivers/irqchip/irq-gic-v3.c +1 -0
index 422664ac5f53..0b545e2c3498 100644
@@ -849,8 +849,21 @@ static int __gic_update_rdist_properties(struct redist_region *region,
 					 void __iomem *ptr)
 {
 	u64 typer = gic_read_typer(ptr + GICR_TYPER);
+
Quoted 8 lines

drivers/irqchip/irq-gic-v3.c static int __gic_update_rdist_properties(struct redist_region *region, · line 855

Quoted 4 lines
drivers/irqchip/irq-gic-v3.c +2 -1
 	gic_data.rdists.has_vlpis &= !!(typer & GICR_TYPER_VLPIS);
-	gic_data.rdists.has_direct_lpi &= !!(typer & GICR_TYPER_DirectLPIS);
+
+	/* RVPEID implies some form of DirectLPI, no matter what the doc says... :-/ */

I think the doc says, RVPEID is *always* 1 for GICv4.1 (and presumably beyond) and when RVPEID==1 then DirectLPI is *always* 0 - but that's OK because for GICv4.1 support for direct LPIs is mandatory.

Well, v4.1 support for DirectLPI is pretty patchy. It has just enough features to make it useful.

Quoted 15 lines

drivers/irqchip/irq-gic-v3.c static int __gic_update_rdist_properties(struct redist_region *region, · line 861

Quoted 7 lines
drivers/irqchip/irq-gic-v3.c +6 -0
+	gic_data.rdists.has_rvpeid &= !!(typer & GICR_TYPER_RVPEID);
+	gic_data.rdists.has_direct_lpi &= (!!(typer & GICR_TYPER_DirectLPIS) |
+					   gic_data.rdists.has_rvpeid);
+
+	/* Detect non-sensical configurations */
+	if (WARN_ON_ONCE(gic_data.rdists.has_rvpeid && !gic_data.rdists.has_vlpis)) {

How feasible is the following suitation? All the redistributors in the system has vlpis=0, and only the first redistributor has rvpeid=1 (with the remaining ones rvpeid=0).If we evaluate this WARN_ON_ONCE on each call to __gic_update_rdist_properties we end up without direct LPI support, however if we evaluated this after iterating through all the redistributors then we'd end up with direct LPI support and a non-essential WARN.

Quoted 5 lines
Quoted 3 lines

Should we do the WARN after iterating through all the redistributors once we know what the final values of these flags will be, perhaps in gic_update_rdist_properties?

What does it gains us?

It prevents an unnecessary WARN.

If the first redistributor has rvpeid=1, vlpis=0, direct_lpi=1, and the others have rvpeid=0, vlpis=0, direct_lpi=0. At the end of iteration, without the WARN if statement, you end up wth rvpeid=0, vlpis=0, direct_lpi=0. I.e. it's done the right thing. In this use-case the WARN doesn't achieve anything other than give the user a pointless WARN. If the WARN was moved to after iteration then the WARN wouldn't fire.

I have no idea how likely this use-case is.

Quoted 6 lines

The moment we've detected any inconsistency, any
use of DirectLPI or VLPIs is a big nono, because the redistributors care not designed to communicate with each other, and we might as well do that early. Frankly, the HW should have stayed in someone's lab. The only reason I have that code in is to detect the FVP model being misconfigured, which is pretty easy to do

Thanks,

Andrew Murray

Quoted 4 lines
	M.
Signature · 1 line
Jazz is not dead, it just smells funny...
author maintainer
#16 lore raw

Re: [PATCH 05/35] irqchip/gic-v3: Add GICv4.1 VPEID size discovery

In reply to Andrew Murray
Quoted 22 lines · Andrew Murray
Quoted 1 line · Marc Zyngier

While GICv4.0 mandates 16 bit worth of VPEIDs, GICv4.1 allows smaller

s/VPEIDs/vPEIDs/

Quoted 17 lines

implementations to be built. Add the required glue to dynamically compute the limit.

---
 drivers/irqchip/irq-gic-v3-its.c   | 11 ++++++++++-
 drivers/irqchip/irq-gic-v3.c       |  3 +++
 include/linux/irqchip/arm-gic-v3.h |  5 +++++
 3 files changed, 18 insertions(+), 1 deletion(-)
drivers/irqchip/irq-gic-v3-its.c +0 -0
index c94eb287393b..17b77a0b9d97 100644
@@ -119,7 +119,16 @@ struct its_node {
 #define ITS_ITT_ALIGN		SZ_256
 
Quoted 10 lines

drivers/irqchip/irq-gic-v3-its.c struct its_node { · line 125

Quoted 6 lines
drivers/irqchip/irq-gic-v3-its.c +4 -1
 /* The maximum number of VPEID bits supported by VLPI commands */
-#define ITS_MAX_VPEID_BITS	(16)
+#define ITS_MAX_VPEID_BITS						\
+	({								\
+		int nvpeid = 16;					\
+		if (gic_rdists->has_rvpeid &&				\

We use rvpeid as a way of determining if this is a GICv4.1, are there any other means of determining this? If we use it in this way, is there any benefit to having a has_gicv4_1 type of flag instead?

RVPEID *is* the way to discover a GICv4.1. To be clear, if we adopted the ARM ARM nomenclature to describe extensions, GICv4.1 would be called GIC-RVPEID, and that'd be it.

Quoted 2 lines

Also for 'insane' configurations we set has_rvpeid to false, thus preventing this feature. Does it make sense to do that?

It makes perfect sense. RVPEID *and* VLPI are set to false, and we don't do *any* direct injection, because it simply cannot work.

Quoted 2 lines

GICD_TYPER2 is reserved in GICv4, however I understand this reads as RES0, can we just rely on that instead? (We read it below anyway).

Yes. In general for the GIC, any RESERVED register is RAZ/WI.

Quoted 40 lines
Quoted 40 lines
drivers/irqchip/irq-gic-v3-its.c +6 -0
+		    gic_rdists->gicd_typer2 & GICD_TYPER2_VIL)		\
+			nvpeid = 1 + (gic_rdists->gicd_typer2 &		\
+				      GICD_TYPER2_VID);			\
+									\
+		nvpeid;							\
+	})
 #define ITS_MAX_VPEID		(1 << (ITS_MAX_VPEID_BITS))
 
 /* Convert page order to size in bytes */
drivers/irqchip/irq-gic-v3.c +3 -0
index 0b545e2c3498..fb6360161d6c 100644
@@ -1556,6 +1556,9 @@ static int __init gic_init_bases(void __iomem *dist_base,
 
 	pr_info("%d SPIs implemented\n", GIC_LINE_NR - 32);
 	pr_info("%d Extended SPIs implemented\n", GIC_ESPI_NR);
+
+	gic_data.rdists.gicd_typer2 = readl_relaxed(gic_data.dist_base + GICD_TYPER2);
+
 	gic_data.domain = irq_domain_create_tree(handle, &gic_irq_domain_ops,
 						 &gic_data);
 	irq_domain_update_bus_token(gic_data.domain, DOMAIN_BUS_WIRED);
include/linux/irqchip/arm-gic-v3.h +1 -0
index b34e0c113697..71730b9def0c 100644
@@ -13,6 +13,7 @@
 #define GICD_CTLR			0x0000
 #define GICD_TYPER			0x0004
 #define GICD_IIDR			0x0008
+#define GICD_TYPER2			0x000C
 #define GICD_STATUSR			0x0010
 #define GICD_SETSPI_NSR			0x0040
 #define GICD_CLRSPI_NSR			0x0048
@@ -89,6 +90,9 @@
 #define GICD_TYPER_ESPIS(typer)						\
 	(((typer) & GICD_TYPER_ESPI) ? GICD_TYPER_SPIS((typer) >> 27) : 0)
 
Quoted 5 lines

include/linux/irqchip/arm-gic-v3.h line 94

Quoted 2 lines
include/linux/irqchip/arm-gic-v3.h +2 -0
+#define GICD_TYPER2_VIL			(1U << 7)
+#define GICD_TYPER2_VID			GENMASK(4, 0)

Given that the 4th bit is reserved for future expansion and values greater than 0xF are reserved, is there value in changing this to GENMASK(3, 0)?

No, I'd rather leave the field to match the specification, and discard values that go beyond 16 in the ITS_MAX_VPEID_BITS macro.

Thanks,

	M.
Signature · 1 line
Jazz is not dead, it just smells funny...
author maintainer
#13 lore raw

Re: [PATCH 04/35] irqchip/gic-v3: Detect GICv4.1 supporting RVPEID

In reply to Andrew Murray
Quoted 58 lines · Andrew Murray
Quoted 51 lines · Marc Zyngier
Quoted 23 lines · Andrew Murray
Quoted 22 lines · Marc Zyngier

GICv4.1 supports the RVPEID ("Residency per vPE ID"), which allows for a much efficient way of making virtual CPUs resident (to allow direct injection of interrupts).

The functionnality needs to be discovered on each and every redistributor in the system, and disabled if the settings are inconsistent.

---
 drivers/irqchip/irq-gic-v3.c       | 21 ++++++++++++++++++---
 include/linux/irqchip/arm-gic-v3.h |  2 ++
 2 files changed, 20 insertions(+), 3 deletions(-)
drivers/irqchip/irq-gic-v3.c +1 -0
index 422664ac5f53..0b545e2c3498 100644
@@ -849,8 +849,21 @@ static int __gic_update_rdist_properties(struct redist_region *region,
 					 void __iomem *ptr)
 {
 	u64 typer = gic_read_typer(ptr + GICR_TYPER);
+
Quoted 8 lines

drivers/irqchip/irq-gic-v3.c static int __gic_update_rdist_properties(struct redist_region *region, · line 855

Quoted 4 lines
drivers/irqchip/irq-gic-v3.c +2 -1
 	gic_data.rdists.has_vlpis &= !!(typer & GICR_TYPER_VLPIS);
-	gic_data.rdists.has_direct_lpi &= !!(typer & GICR_TYPER_DirectLPIS);
+
+	/* RVPEID implies some form of DirectLPI, no matter what the doc says... :-/ */

I think the doc says, RVPEID is *always* 1 for GICv4.1 (and presumably beyond) and when RVPEID==1 then DirectLPI is *always* 0 - but that's OK because for GICv4.1 support for direct LPIs is mandatory.

Well, v4.1 support for DirectLPI is pretty patchy. It has just enough features to make it useful.

Quoted 15 lines

drivers/irqchip/irq-gic-v3.c static int __gic_update_rdist_properties(struct redist_region *region, · line 861

Quoted 7 lines
drivers/irqchip/irq-gic-v3.c +6 -0
+	gic_data.rdists.has_rvpeid &= !!(typer & GICR_TYPER_RVPEID);
+	gic_data.rdists.has_direct_lpi &= (!!(typer & GICR_TYPER_DirectLPIS) |
+					   gic_data.rdists.has_rvpeid);
+
+	/* Detect non-sensical configurations */
+	if (WARN_ON_ONCE(gic_data.rdists.has_rvpeid && !gic_data.rdists.has_vlpis)) {

How feasible is the following suitation? All the redistributors in the system has vlpis=0, and only the first redistributor has rvpeid=1 (with the remaining ones rvpeid=0).If we evaluate this WARN_ON_ONCE on each call to __gic_update_rdist_properties we end up without direct LPI support, however if we evaluated this after iterating through all the redistributors then we'd end up with direct LPI support and a non-essential WARN.

Quoted 5 lines
Quoted 3 lines

Should we do the WARN after iterating through all the redistributors once we know what the final values of these flags will be, perhaps in gic_update_rdist_properties?

What does it gains us?

Quoted 8 lines

It prevents an unnecessary WARN.

If the first redistributor has rvpeid=1, vlpis=0, direct_lpi=1, and the others have rvpeid=0, vlpis=0, direct_lpi=0. At the end of iteration, without the WARN if statement, you end up wth rvpeid=0, vlpis=0, direct_lpi=0. I.e. it's done the right thing. In this use-case the WARN doesn't achieve anything other than give the user a pointless WARN. If the WARN was moved to after iteration then the WARN wouldn't fire.

But it definitely *should* fire. RVPEID+!VLPI is terminally broken. What's the use of RVPEID if you cannot directly inject anything? To me
blunt: any difference of HW configuration for any redistributor is an error. They should all be identical (no, I'm not planning to deal with the GIC equivalent OF BL).

Quoted 1 line

I have no idea how likely this use-case is.

There is no use case. Such a configuration shouldn't exist. I'm considering calling panic instead.

	M.
Signature · 1 line
Jazz is not dead, it just smells funny...
reviewer
#22 lore raw

Re: [PATCH 10/35] irqchip/gic-v4.1: VPE table (aka GICR_VPROPBASER) allocation

In reply to Marc Zyngier

Hi Marc,

Some questions about this patch, mostly to confirm that I would understand things here correctly.

Quoted 14 lines · Marc Zyngier

GICv4.1 defines a new VPE table that is potentially shared between both the ITSs and the redistributors, following complicated affinity rules.

To make things more confusing, the programming of this table at the redistributor level is reusing the GICv4.0 GICR_VPROPBASER register for something completely different.

The code flow is somewhat complexified by the need to respect the affinities required by the HW, meaning that tables can either be inherited from a previously discovered ITS or redistributor.

---

[...]

Quoted 67 lines
drivers/irqchip/irq-gic-v3-its.c +59 -0
@@ -1962,6 +1965,65 @@ static bool its_parse_indirect_baser(struct its_node *its,
  	return indirect;
  }
 
+static u32 compute_common_aff(u64 val)
+{
+	u32 aff, clpiaff;
+
+	aff = FIELD_GET(GICR_TYPER_AFFINITY, val);
+	clpiaff = FIELD_GET(GICR_TYPER_COMMON_LPI_AFF, val);
+
+	return aff & ~(GENMASK(31, 0) >> (clpiaff * 8));
+}
+
+static u32 compute_its_aff(struct its_node *its)
+{
+	u64 val;
+	u32 svpet;
+
+	/*
+	 * Reencode the ITS SVPET and MPIDR as a GICR_TYPER, and compute
+	 * the resulting affinity. We then use that to see if this match
+	 * our own affinity.
+	 */
+	svpet = FIELD_GET(GITS_TYPER_SVPET, its->typer);
+	val  = FIELD_PREP(GICR_TYPER_COMMON_LPI_AFF, svpet);
+	val |= FIELD_PREP(GICR_TYPER_AFFINITY, its->mpidr);
+	return compute_common_aff(val);
+}
+
+static struct its_node *find_sibbling_its(struct its_node *cur_its)
+{
+	struct its_node *its;
+	u32 aff;
+
+	if (!FIELD_GET(GITS_TYPER_SVPET, cur_its->typer))
+		return NULL;
+
+	aff = compute_its_aff(cur_its);
+
+	list_for_each_entry(its, &its_nodes, entry) {
+		u64 baser;
+
+		if (!is_v4_1(its) || its == cur_its)
+			continue;
+
+		if (!FIELD_GET(GITS_TYPER_SVPET, its->typer))
+			continue;
+
+		if (aff != compute_its_aff(its))
+			continue;
+
+		/* GICv4.1 guarantees that the vPE table is GITS_BASER2 */
+		baser = its->tables[2].val;
+		if (!(baser & GITS_BASER_VALID))
+			continue;
+
+		return its;
+	}
+
+	return NULL;
+}
+
  static void its_free_tables(struct its_node *its)
  {
  	int i;

drivers/irqchip/irq-gic-v3-its.c static int its_alloc_tables(struct its_node *its) · line 2073

Quoted 8 lines
drivers/irqchip/irq-gic-v3-its.c +5 -0
@@ -2004,6 +2066,15 @@ static int its_alloc_tables(struct its_node *its)
  			break;
 
  		case GITS_BASER_TYPE_VCPU:
+			if (is_v4_1(its)) {
+				struct its_node *sibbling;
+
+				if ((sibbling = find_sibbling_its(its))) {
+					its->tables[2] = sibbling->tables[2];

This means thst the vPE table is shared between ITSs which are under the same affinity group?
Don't we need to set GITS_BASER (by its_setup_baser()) here to tell this ITS where the vPE table lacates?

Quoted 35 lines
drivers/irqchip/irq-gic-v3-its.c +28 -0
+					continue;
+				}
+			}
+
  			indirect = its_parse_indirect_baser(its, baser,
  							    psz, &order,
  							    ITS_MAX_VPEID_BITS);
@@ -2025,6 +2096,212 @@ static int its_alloc_tables(struct its_node *its)
  	return 0;
  }
 
+static u64 inherit_vpe_l1_table_from_its(void)
+{
+	struct its_node *its;
+	u64 val;
+	u32 aff;
+
+	val = gic_read_typer(gic_data_rdist_rd_base() + GICR_TYPER);
+	aff = compute_common_aff(val);
+
+	list_for_each_entry(its, &its_nodes, entry) {
+		u64 baser;
+
+		if (!is_v4_1(its))
+			continue;
+
+		if (!FIELD_GET(GITS_TYPER_SVPET, its->typer))
+			continue;
+
+		if (aff != compute_its_aff(its))
+			continue;
+
+		/* GICv4.1 guarantees that the vPE table is GITS_BASER2 */
+		baser = its->tables[2].val;
+		if (!(baser & GITS_BASER_VALID))

drivers/irqchip/irq-gic-v3-its.c static int its_alloc_tables(struct its_node *its) · line 2132

Quoted 10 lines
drivers/irqchip/irq-gic-v3-its.c +10 -0
+			continue;
+
+		/* We have a winner! */
+		val  = GICR_VPROPBASER_4_1_VALID;
+		if (baser & GITS_BASER_INDIRECT)
+			val |= GICR_VPROPBASER_4_1_INDIRECT;
+		val |= FIELD_PREP(GICR_VPROPBASER_4_1_PAGE_SIZE,
+				  FIELD_GET(GITS_BASER_PAGE_SIZE_MASK, baser));
+		val |= FIELD_PREP(GICR_VPROPBASER_4_1_ADDR,
+				  GITS_BASER_ADDR_48_to_52(baser) >> 12);

I remember the spec says,
GITS_BASER2 "points to" the ITS *vPE table*, which provides mappings from the vPEID to target Redistributor and associated vpt address. In GICv4.1 GICR_VPROPBASER "points to" the *vPE Configuration table*, which stores the locations of each vPE's LPI configuration and pending table.

And the code here says, if we can find an ITS (which is under the same CommonLPIAff group with this Redistributor) has already been probed and allocated an vPE table, then use this vPE table as this Redist's vPE Configuration table.
So I infer that in GICv4.1, *vPE table* and *vPE Configuration table* are actually the same concept? And this table is shared between ITSs and Redists which are under the same affinity group?
Please fix me if I have any misunderstanding.

Quoted 72 lines
drivers/irqchip/irq-gic-v3-its.c +72 -0
+		val |= FIELD_PREP(GICR_VPROPBASER_SHAREABILITY_MASK,
+				  FIELD_GET(GITS_BASER_SHAREABILITY_MASK, baser));
+		val |= FIELD_PREP(GICR_VPROPBASER_INNER_CACHEABILITY_MASK,
+				  FIELD_GET(GITS_BASER_INNER_CACHEABILITY_MASK, baser));
+		val |= FIELD_PREP(GICR_VPROPBASER_4_1_SIZE, GITS_BASER_NR_PAGES(baser) - 1);
+
+		return val;
+	}
+
+	return 0;
+}
+
+static u64 inherit_vpe_l1_table_from_rd(cpumask_t **mask)
+{
+	u32 aff;
+	u64 val;
+	int cpu;
+
+	val = gic_read_typer(gic_data_rdist_rd_base() + GICR_TYPER);
+	aff = compute_common_aff(val);
+
+	for_each_possible_cpu(cpu) {
+		void __iomem *base = gic_data_rdist_cpu(cpu)->rd_base;
+		u32 tmp;
+
+		if (!base || cpu == smp_processor_id())
+			continue;
+
+		val = gic_read_typer(base + GICR_TYPER);
+		tmp = compute_common_aff(val);
+		if (tmp != aff)
+			continue;
+
+		/*
+		 * At this point, we have a victim. This particular CPU
+		 * has already booted, and has an affinity that matches
+		 * ours wrt CommonLPIAff. Let's use its own VPROPBASER.
+		 * Make sure we don't write the Z bit in that case.
+		 */
+		val = gits_read_vpropbaser(base + SZ_128K + GICR_VPROPBASER);
+		val &= ~GICR_VPROPBASER_4_1_Z;
+
+		*mask = gic_data_rdist_cpu(cpu)->vpe_table_mask;
+
+		return val;
+	}
+
+	return 0;
+}
+
+static int allocate_vpe_l1_table(void)
+{
+	void __iomem *vlpi_base = gic_data_rdist_vlpi_base();
+	u64 val, esz, gpsz, npg, pa;
+	unsigned int psz = SZ_64K;
+	unsigned int np, epp;
+	struct page *page;
+
+	if (!gic_rdists->has_rvpeid)
+		return 0;
+
+	/*
+	 * if VPENDBASER.Valid is set, disable any previously programmed
+	 * VPE by setting PendingLast while clearing Valid. This has the
+	 * effect of making sure no doorbell will be generated and we can
+	 * then safely clear VPROPBASER.Valid.
+	 */
+	if (gits_read_vpendbaser(vlpi_base + GICR_VPENDBASER) & GICR_VPENDBASER_Valid)
+		gits_write_vpendbaser(GICR_VPENDBASER_PendingLast,
+				      vlpi_base + GICR_VPENDBASER);
+
+	/*

drivers/irqchip/irq-gic-v3-its.c static int its_alloc_tables(struct its_node *its) · line 2214

Quoted 10 lines
drivers/irqchip/irq-gic-v3-its.c +10 -0
+	 * If we can inherit the configuration from another RD, let's do
+	 * so. Otherwise, we have to go through the allocation process. We
+	 * assume that all RDs have the exact same requirements, as
+	 * nothing will work otherwise.
+	 */
+	val = inherit_vpe_l1_table_from_rd(&gic_data_rdist()->vpe_table_mask);
+	if (val & GICR_VPROPBASER_4_1_VALID)
+		goto out;
+
+	gic_data_rdist()->vpe_table_mask = kzalloc(sizeof(cpumask_t), GFP_KERNEL);

I think we should check the allocation failure here.

Quoted 24 lines
drivers/irqchip/irq-gic-v3-its.c +24 -0
+
+	val = inherit_vpe_l1_table_from_its();
+	if (val & GICR_VPROPBASER_4_1_VALID)
+		goto out;
+
+	/* First probe the page size */
+	val = FIELD_PREP(GICR_VPROPBASER_4_1_PAGE_SIZE, GIC_PAGE_SIZE_64K);
+	gits_write_vpropbaser(val, vlpi_base + GICR_VPROPBASER);
+	val = gits_read_vpropbaser(vlpi_base + GICR_VPROPBASER);
+	gpsz = FIELD_GET(GICR_VPROPBASER_4_1_PAGE_SIZE, val);
+
+	switch (gpsz) {
+	default:
+		gpsz = GIC_PAGE_SIZE_4K;
+		/* fall through */
+	case GIC_PAGE_SIZE_4K:
+		psz = SZ_4K;
+		break;
+	case GIC_PAGE_SIZE_16K:
+		psz = SZ_16K;
+		break;
+	case GIC_PAGE_SIZE_64K:
+		psz = SZ_64K;
+		break;

drivers/irqchip/irq-gic-v3-its.c static int its_alloc_tables(struct its_node *its) · line 2248

Quoted 10 lines
drivers/irqchip/irq-gic-v3-its.c +10 -0
+	}
+
+	/*
+	 * Start populating the register from scratch, including RO fields
+	 * (which we want to print in debug cases...)
+	 */
+	val = 0;
+	val |= FIELD_PREP(GICR_VPROPBASER_4_1_PAGE_SIZE, gpsz);
+	esz = FIELD_GET(GICR_VPROPBASER_4_1_ENTRY_SIZE, val);
+	val |= FIELD_PREP(GICR_VPROPBASER_4_1_ENTRY_SIZE, esz);

'esz' seems always be 0 here?

Quoted 75 lines
drivers/irqchip/irq-gic-v3-its.c +60 -1
+
+	/* How many entries per GIC page? */
+	esz++;
+	epp = psz / (esz * SZ_8);
+
+	/*
+	 * If we need more than just a single L1 page, flag the table
+	 * as indirect and compute the number of required L1 pages.
+	 */
+	if (epp < ITS_MAX_VPEID) {
+		int nl2;
+
+		gic_rdists->flags |= RDIST_FLAGS_VPE_INDIRECT;
+		val |= GICR_VPROPBASER_4_1_INDIRECT;
+
+		/* Number of L2 pages required to cover the VPEID space */
+		nl2 = DIV_ROUND_UP(ITS_MAX_VPEID, epp);
+
+		/* Number of L1 pages to point to the L2 pages */
+		npg = DIV_ROUND_UP(nl2 * SZ_8, psz);
+	} else {
+		npg = 1;
+	}
+
+	val |= FIELD_PREP(GICR_VPROPBASER_4_1_SIZE, npg);
+
+	/* Right, that's the number of CPU pages we need for L1 */
+	np = DIV_ROUND_UP(npg * psz, PAGE_SIZE);
+
+	pr_debug("np = %d, npg = %lld, psz = %d, epp = %d, esz = %lld\n",
+		 np, npg, psz, epp, esz);
+	page = alloc_pages(GFP_KERNEL | __GFP_ZERO, get_order(np * PAGE_SIZE));
+	if (!page)
+		return -ENOMEM;
+
+	gic_data_rdist()->vpe_l1_page = page;
+	pa = virt_to_phys(page_address(page));
+	WARN_ON(!IS_ALIGNED(pa, psz));
+
+	val |= FIELD_PREP(GICR_VPROPBASER_4_1_ADDR, pa >> 12);
+	val |= GICR_VPROPBASER_RaWb;
+	val |= GICR_VPROPBASER_InnerShareable;
+	val |= GICR_VPROPBASER_4_1_Z;
+	val |= GICR_VPROPBASER_4_1_VALID;
+
+out:
+	gits_write_vpropbaser(val, vlpi_base + GICR_VPROPBASER);
+	cpumask_set_cpu(smp_processor_id(), gic_data_rdist()->vpe_table_mask);
+
+	pr_debug("CPU%d: VPROPBASER = %llx %*pbl\n",
+		 smp_processor_id(), val,
+		 cpumask_pr_args(gic_data_rdist()->vpe_table_mask));
+
+	return 0;
+}
+
  static int its_alloc_collections(struct its_node *its)
  {
  	int i;
@@ -2224,7 +2501,7 @@ static void its_cpu_init_lpis(void)
  	val |= GICR_CTLR_ENABLE_LPIS;
  	writel_relaxed(val, rbase + GICR_CTLR);
 
-	if (gic_rdists->has_vlpis) {
+	if (gic_rdists->has_vlpis && !gic_rdists->has_rvpeid) {
  		void __iomem *vlpi_base = gic_data_rdist_vlpi_base();
 
  		/*
@@ -2248,6 +2525,16 @@ static void its_cpu_init_lpis(void)
  		WARN_ON(val & GICR_VPENDBASER_Dirty);
  	}
 
+	if (allocate_vpe_l1_table()) {
+		/*
+		 * If the allocation has failed, we're in massive trouble.

drivers/irqchip/irq-gic-v3-its.c static void its_cpu_init_lpis(void) · line 2540

Quoted 10 lines
drivers/irqchip/irq-gic-v3-its.c +7 -0
+		 * Disable direct injection, and pray that no VM was
+		 * already running...
+		 */
+		gic_rdists->has_rvpeid = false;
+		gic_rdists->has_vlpis = false;
+	}
+
  	/* Make sure the GIC has seen the above */
  	dsb(sy);
  out:

Thanks,
zenghui

author maintainer
#23 lore raw

Re: [PATCH 10/35] irqchip/gic-v4.1: VPE table (aka GICR_VPROPBASER) allocation

Quoted 92 lines · Zenghui Yu

Hi Marc,

Some questions about this patch, mostly to confirm that I would understand things here correctly.

Quoted 14 lines · Marc Zyngier

GICv4.1 defines a new VPE table that is potentially shared between both the ITSs and the redistributors, following complicated affinity rules.

To make things more confusing, the programming of this table at the redistributor level is reusing the GICv4.0 GICR_VPROPBASER register for something completely different.

The code flow is somewhat complexified by the need to respect the affinities required by the HW, meaning that tables can either be inherited from a previously discovered ITS or redistributor.

---

[...]

Quoted 69 lines
drivers/irqchip/irq-gic-v3-its.c +59 -0
@@ -1962,6 +1965,65 @@ static bool its_parse_indirect_baser(struct its_node *its,
  	return indirect;
  }
 
+static u32 compute_common_aff(u64 val)
+{
+	u32 aff, clpiaff;
+
+	aff = FIELD_GET(GICR_TYPER_AFFINITY, val);
+	clpiaff = FIELD_GET(GICR_TYPER_COMMON_LPI_AFF, val);
+
+	return aff & ~(GENMASK(31, 0) >> (clpiaff * 8));
+}
+
+static u32 compute_its_aff(struct its_node *its)
+{
+	u64 val;
+	u32 svpet;
+
+	/*
+	 * Reencode the ITS SVPET and MPIDR as a GICR_TYPER, and compute
+	 * the resulting affinity. We then use that to see if this match
+	 * our own affinity.
+	 */
+	svpet = FIELD_GET(GITS_TYPER_SVPET, its->typer);
+	val  = FIELD_PREP(GICR_TYPER_COMMON_LPI_AFF, svpet);
+	val |= FIELD_PREP(GICR_TYPER_AFFINITY, its->mpidr);
+	return compute_common_aff(val);
+}
+
+static struct its_node *find_sibbling_its(struct its_node *cur_its)
+{
+	struct its_node *its;
+	u32 aff;
+
+	if (!FIELD_GET(GITS_TYPER_SVPET, cur_its->typer))
+		return NULL;
+
+	aff = compute_its_aff(cur_its);
+
+	list_for_each_entry(its, &its_nodes, entry) {
+		u64 baser;
+
+		if (!is_v4_1(its) || its == cur_its)
+			continue;
+
+		if (!FIELD_GET(GITS_TYPER_SVPET, its->typer))
+			continue;
+
+		if (aff != compute_its_aff(its))
+			continue;
+
+		/* GICv4.1 guarantees that the vPE table is GITS_BASER2 */
+		baser = its->tables[2].val;
+		if (!(baser & GITS_BASER_VALID))
+			continue;
+
+		return its;
+	}
+
+	return NULL;
+}
+
  static void its_free_tables(struct its_node *its)
  {
  	int i;
@@ -2004,6 +2066,15 @@ static int its_alloc_tables(struct its_node *its)
  			break;
 
Quoted 9 lines

drivers/irqchip/irq-gic-v3-its.c static int its_alloc_tables(struct its_node *its) · line 2073

Quoted 6 lines
drivers/irqchip/irq-gic-v3-its.c +5 -0
  		case GITS_BASER_TYPE_VCPU:
+			if (is_v4_1(its)) {
+				struct its_node *sibbling;
+
+				if ((sibbling = find_sibbling_its(its))) {
+					its->tables[2] = sibbling->tables[2];

This means thst the vPE table is shared between ITSs which are under the same affinity group?

That's indeed what this is trying to do...

Quoted 2 lines

Don't we need to set GITS_BASER (by its_setup_baser()) here to tell this ITS where the vPE table lacates?

Absolutely. This is a bug. I didn't spot it as my model only has a single ITS.

Quoted 54 lines
Quoted 36 lines
drivers/irqchip/irq-gic-v3-its.c +28 -0
+					continue;
+				}
+			}
+
  			indirect = its_parse_indirect_baser(its, baser,
  							    psz, &order,
  							    ITS_MAX_VPEID_BITS);
@@ -2025,6 +2096,212 @@ static int its_alloc_tables(struct its_node *its)
  	return 0;
  }
 
+static u64 inherit_vpe_l1_table_from_its(void)
+{
+	struct its_node *its;
+	u64 val;
+	u32 aff;
+
+	val = gic_read_typer(gic_data_rdist_rd_base() + GICR_TYPER);
+	aff = compute_common_aff(val);
+
+	list_for_each_entry(its, &its_nodes, entry) {
+		u64 baser;
+
+		if (!is_v4_1(its))
+			continue;
+
+		if (!FIELD_GET(GITS_TYPER_SVPET, its->typer))
+			continue;
+
+		if (aff != compute_its_aff(its))
+			continue;
+
+		/* GICv4.1 guarantees that the vPE table is GITS_BASER2 */
+		baser = its->tables[2].val;
+		if (!(baser & GITS_BASER_VALID))

drivers/irqchip/irq-gic-v3-its.c static int its_alloc_tables(struct its_node *its) · line 2132

Quoted 10 lines
drivers/irqchip/irq-gic-v3-its.c +10 -0
+			continue;
+
+		/* We have a winner! */
+		val  = GICR_VPROPBASER_4_1_VALID;
+		if (baser & GITS_BASER_INDIRECT)
+			val |= GICR_VPROPBASER_4_1_INDIRECT;
+		val |= FIELD_PREP(GICR_VPROPBASER_4_1_PAGE_SIZE,
+				  FIELD_GET(GITS_BASER_PAGE_SIZE_MASK, baser));
+		val |= FIELD_PREP(GICR_VPROPBASER_4_1_ADDR,
+				  GITS_BASER_ADDR_48_to_52(baser) >> 12);

I remember the spec says,
GITS_BASER2 "points to" the ITS *vPE table*, which provides mappings from the vPEID to target Redistributor and associated vpt address. In GICv4.1 GICR_VPROPBASER "points to" the *vPE Configuration table*, which stores the locations of each vPE's LPI configuration and pending table.

Quoted 8 lines

And the code here says, if we can find an ITS (which is under the same CommonLPIAff group with this Redistributor) has already been probed and allocated an vPE table, then use this vPE table as this Redist's vPE Configuration table.
So I infer that in GICv4.1, *vPE table* and *vPE Configuration table* are actually the same concept? And this table is shared between ITSs and Redists which are under the same affinity group?
Please fix me if I have any misunderstanding.

Indeed. The whole idea is that ITSs and RDs can share a vPE table if they are in the same affinity group (such as a socket, for example). This is what is missing from v4.0 where the ITS knows about vPEs, but doesn't know about residency. With that in place, the HW is able to do a lot more for us.

Quoted 77 lines
Quoted 77 lines
drivers/irqchip/irq-gic-v3-its.c +76 -0
+		val |= FIELD_PREP(GICR_VPROPBASER_SHAREABILITY_MASK,
+				  FIELD_GET(GITS_BASER_SHAREABILITY_MASK, baser));
+		val |= FIELD_PREP(GICR_VPROPBASER_INNER_CACHEABILITY_MASK,
+				  FIELD_GET(GITS_BASER_INNER_CACHEABILITY_MASK, baser));
+		val |= FIELD_PREP(GICR_VPROPBASER_4_1_SIZE, GITS_BASER_NR_PAGES(baser) - 1);
+
+		return val;
+	}
+
+	return 0;
+}
+
+static u64 inherit_vpe_l1_table_from_rd(cpumask_t **mask)
+{
+	u32 aff;
+	u64 val;
+	int cpu;
+
+	val = gic_read_typer(gic_data_rdist_rd_base() + GICR_TYPER);
+	aff = compute_common_aff(val);
+
+	for_each_possible_cpu(cpu) {
+		void __iomem *base = gic_data_rdist_cpu(cpu)->rd_base;
+		u32 tmp;
+
+		if (!base || cpu == smp_processor_id())
+			continue;
+
+		val = gic_read_typer(base + GICR_TYPER);
+		tmp = compute_common_aff(val);
+		if (tmp != aff)
+			continue;
+
+		/*
+		 * At this point, we have a victim. This particular CPU
+		 * has already booted, and has an affinity that matches
+		 * ours wrt CommonLPIAff. Let's use its own VPROPBASER.
+		 * Make sure we don't write the Z bit in that case.
+		 */
+		val = gits_read_vpropbaser(base + SZ_128K + GICR_VPROPBASER);
+		val &= ~GICR_VPROPBASER_4_1_Z;
+
+		*mask = gic_data_rdist_cpu(cpu)->vpe_table_mask;
+
+		return val;
+	}
+
+	return 0;
+}
+
+static int allocate_vpe_l1_table(void)
+{
+	void __iomem *vlpi_base = gic_data_rdist_vlpi_base();
+	u64 val, esz, gpsz, npg, pa;
+	unsigned int psz = SZ_64K;
+	unsigned int np, epp;
+	struct page *page;
+
+	if (!gic_rdists->has_rvpeid)
+		return 0;
+
+	/*
+	 * if VPENDBASER.Valid is set, disable any previously programmed
+	 * VPE by setting PendingLast while clearing Valid. This has the
+	 * effect of making sure no doorbell will be generated and we can
+	 * then safely clear VPROPBASER.Valid.
+	 */
+	if (gits_read_vpendbaser(vlpi_base + GICR_VPENDBASER) & GICR_VPENDBASER_Valid)
+		gits_write_vpendbaser(GICR_VPENDBASER_PendingLast,
+				      vlpi_base + GICR_VPENDBASER);
+
+	/*
+	 * If we can inherit the configuration from another RD, let's do
+	 * so. Otherwise, we have to go through the allocation process. We
+	 * assume that all RDs have the exact same requirements, as
+	 * nothing will work otherwise.
Quoted 8 lines

drivers/irqchip/irq-gic-v3-its.c static int its_alloc_tables(struct its_node *its) · line 2214

Quoted 6 lines
drivers/irqchip/irq-gic-v3-its.c +6 -0
+	 */
+	val = inherit_vpe_l1_table_from_rd(&gic_data_rdist()->vpe_table_mask);
+	if (val & GICR_VPROPBASER_4_1_VALID)
+		goto out;
+
+	gic_data_rdist()->vpe_table_mask = kzalloc(sizeof(cpumask_t), GFP_KERNEL);

I think we should check the allocation failure here.

Sure.

Quoted 29 lines
Quoted 29 lines
drivers/irqchip/irq-gic-v3-its.c +28 -0
+
+	val = inherit_vpe_l1_table_from_its();
+	if (val & GICR_VPROPBASER_4_1_VALID)
+		goto out;
+
+	/* First probe the page size */
+	val = FIELD_PREP(GICR_VPROPBASER_4_1_PAGE_SIZE, GIC_PAGE_SIZE_64K);
+	gits_write_vpropbaser(val, vlpi_base + GICR_VPROPBASER);
+	val = gits_read_vpropbaser(vlpi_base + GICR_VPROPBASER);
+	gpsz = FIELD_GET(GICR_VPROPBASER_4_1_PAGE_SIZE, val);
+
+	switch (gpsz) {
+	default:
+		gpsz = GIC_PAGE_SIZE_4K;
+		/* fall through */
+	case GIC_PAGE_SIZE_4K:
+		psz = SZ_4K;
+		break;
+	case GIC_PAGE_SIZE_16K:
+		psz = SZ_16K;
+		break;
+	case GIC_PAGE_SIZE_64K:
+		psz = SZ_64K;
+		break;
+	}
+
+	/*
+	 * Start populating the register from scratch, including RO fields
Quoted 8 lines

drivers/irqchip/irq-gic-v3-its.c static int its_alloc_tables(struct its_node *its) · line 2248

Quoted 6 lines
drivers/irqchip/irq-gic-v3-its.c +6 -0
+	 * (which we want to print in debug cases...)
+	 */
+	val = 0;
+	val |= FIELD_PREP(GICR_VPROPBASER_4_1_PAGE_SIZE, gpsz);
+	esz = FIELD_GET(GICR_VPROPBASER_4_1_ENTRY_SIZE, val);
+	val |= FIELD_PREP(GICR_VPROPBASER_4_1_ENTRY_SIZE, esz);

'esz' seems always be 0 here?

Yup, that's broken. Not sure what I had in mind here. esz (element size) should be extracted before resetting val, and ORed back in the register copy for debug purposes.

Quoted 73 lines
Quoted 73 lines
drivers/irqchip/irq-gic-v3-its.c +57 -1
+
+	/* How many entries per GIC page? */
+	esz++;
+	epp = psz / (esz * SZ_8);
+
+	/*
+	 * If we need more than just a single L1 page, flag the table
+	 * as indirect and compute the number of required L1 pages.
+	 */
+	if (epp < ITS_MAX_VPEID) {
+		int nl2;
+
+		gic_rdists->flags |= RDIST_FLAGS_VPE_INDIRECT;
+		val |= GICR_VPROPBASER_4_1_INDIRECT;
+
+		/* Number of L2 pages required to cover the VPEID space */
+		nl2 = DIV_ROUND_UP(ITS_MAX_VPEID, epp);
+
+		/* Number of L1 pages to point to the L2 pages */
+		npg = DIV_ROUND_UP(nl2 * SZ_8, psz);
+	} else {
+		npg = 1;
+	}
+
+	val |= FIELD_PREP(GICR_VPROPBASER_4_1_SIZE, npg);
+
+	/* Right, that's the number of CPU pages we need for L1 */
+	np = DIV_ROUND_UP(npg * psz, PAGE_SIZE);
+
+	pr_debug("np = %d, npg = %lld, psz = %d, epp = %d, esz = %lld\n",
+		 np, npg, psz, epp, esz);
+	page = alloc_pages(GFP_KERNEL | __GFP_ZERO, get_order(np * PAGE_SIZE));
+	if (!page)
+		return -ENOMEM;
+
+	gic_data_rdist()->vpe_l1_page = page;
+	pa = virt_to_phys(page_address(page));
+	WARN_ON(!IS_ALIGNED(pa, psz));
+
+	val |= FIELD_PREP(GICR_VPROPBASER_4_1_ADDR, pa >> 12);
+	val |= GICR_VPROPBASER_RaWb;
+	val |= GICR_VPROPBASER_InnerShareable;
+	val |= GICR_VPROPBASER_4_1_Z;
+	val |= GICR_VPROPBASER_4_1_VALID;
+
+out:
+	gits_write_vpropbaser(val, vlpi_base + GICR_VPROPBASER);
+	cpumask_set_cpu(smp_processor_id(), gic_data_rdist()->vpe_table_mask);
+
+	pr_debug("CPU%d: VPROPBASER = %llx %*pbl\n",
+		 smp_processor_id(), val,
+		 cpumask_pr_args(gic_data_rdist()->vpe_table_mask));
+
+	return 0;
+}
+
  static int its_alloc_collections(struct its_node *its)
  {
  	int i;
@@ -2224,7 +2501,7 @@ static void its_cpu_init_lpis(void)
  	val |= GICR_CTLR_ENABLE_LPIS;
  	writel_relaxed(val, rbase + GICR_CTLR);
 
-	if (gic_rdists->has_vlpis) {
+	if (gic_rdists->has_vlpis && !gic_rdists->has_rvpeid) {
  		void __iomem *vlpi_base = gic_data_rdist_vlpi_base();
 
  		/*
@@ -2248,6 +2525,16 @@ static void its_cpu_init_lpis(void)
  		WARN_ON(val & GICR_VPENDBASER_Dirty);
  	}
 
Quoted 13 lines

drivers/irqchip/irq-gic-v3-its.c static void its_cpu_init_lpis(void) · line 2540

Quoted 13 lines
drivers/irqchip/irq-gic-v3-its.c +10 -0
+	if (allocate_vpe_l1_table()) {
+		/*
+		 * If the allocation has failed, we're in massive trouble.
+		 * Disable direct injection, and pray that no VM was
+		 * already running...
+		 */
+		gic_rdists->has_rvpeid = false;
+		gic_rdists->has_vlpis = false;
+	}
+
  	/* Make sure the GIC has seen the above */
  	dsb(sy);
  out:

Thanks for having had a look!

        M.
Signature · 1 line
Jazz is not dead, it just smells funny...
author maintainer
#24 lore raw

Re: [PATCH 10/35] irqchip/gic-v4.1: VPE table (aka GICR_VPROPBASER) allocation

Quoted 106 lines · Marc Zyngier
Quoted 95 lines · Zenghui Yu

Hi Marc,

Some questions about this patch, mostly to confirm that I would understand things here correctly.

Quoted 14 lines · Marc Zyngier

GICv4.1 defines a new VPE table that is potentially shared between both the ITSs and the redistributors, following complicated affinity rules.

To make things more confusing, the programming of this table at the redistributor level is reusing the GICv4.0 GICR_VPROPBASER register for something completely different.

The code flow is somewhat complexified by the need to respect the affinities required by the HW, meaning that tables can either be inherited from a previously discovered ITS or redistributor.

---

[...]

Quoted 72 lines
drivers/irqchip/irq-gic-v3-its.c +59 -0
@@ -1962,6 +1965,65 @@ static bool its_parse_indirect_baser(struct 
 its_node *its,
  	return indirect;
  }
 
+static u32 compute_common_aff(u64 val)
+{
+	u32 aff, clpiaff;
+
+	aff = FIELD_GET(GICR_TYPER_AFFINITY, val);
+	clpiaff = FIELD_GET(GICR_TYPER_COMMON_LPI_AFF, val);
+
+	return aff & ~(GENMASK(31, 0) >> (clpiaff * 8));
+}
+
+static u32 compute_its_aff(struct its_node *its)
+{
+	u64 val;
+	u32 svpet;
+
+	/*
+	 * Reencode the ITS SVPET and MPIDR as a GICR_TYPER, and compute
+	 * the resulting affinity. We then use that to see if this match
+	 * our own affinity.
+	 */
+	svpet = FIELD_GET(GITS_TYPER_SVPET, its->typer);
+	val  = FIELD_PREP(GICR_TYPER_COMMON_LPI_AFF, svpet);
+	val |= FIELD_PREP(GICR_TYPER_AFFINITY, its->mpidr);
+	return compute_common_aff(val);
+}
+
+static struct its_node *find_sibbling_its(struct its_node 
 *cur_its)
+{
+	struct its_node *its;
+	u32 aff;
+
+	if (!FIELD_GET(GITS_TYPER_SVPET, cur_its->typer))
+		return NULL;
+
+	aff = compute_its_aff(cur_its);
+
+	list_for_each_entry(its, &its_nodes, entry) {
+		u64 baser;
+
+		if (!is_v4_1(its) || its == cur_its)
+			continue;
+
+		if (!FIELD_GET(GITS_TYPER_SVPET, its->typer))
+			continue;
+
+		if (aff != compute_its_aff(its))
+			continue;
+
+		/* GICv4.1 guarantees that the vPE table is GITS_BASER2 */
+		baser = its->tables[2].val;
+		if (!(baser & GITS_BASER_VALID))
+			continue;
+
+		return its;
+	}
+
+	return NULL;
+}
+
  static void its_free_tables(struct its_node *its)
  {
  	int i;
(quoted diff) +0 -0
@@ -2004,6 +2066,15 @@ static int its_alloc_tables(struct its_node 
 *its)
  			break;
 
Quoted 9 lines

drivers/irqchip/irq-gic-v3-its.c static int its_alloc_tables(struct its_node *its) · line 2073

Quoted 6 lines
drivers/irqchip/irq-gic-v3-its.c +5 -0
  		case GITS_BASER_TYPE_VCPU:
+			if (is_v4_1(its)) {
+				struct its_node *sibbling;
+
+				if ((sibbling = find_sibbling_its(its))) {
+					its->tables[2] = sibbling->tables[2];

This means thst the vPE table is shared between ITSs which are under the same affinity group?

Quoted 7 lines

That's indeed what this is trying to do...

Quoted 2 lines

Don't we need to set GITS_BASER (by its_setup_baser()) here to tell this ITS where the vPE table lacates?

Absolutely. This is a bug. I didn't spot it as my model only has a single ITS.

FWIW, I've pushed out an updated branch with this fixed as well as the rest of the comments you had.

Thanks again,

        M.
Signature · 1 line
Jazz is not dead. It just smells funny...
reviewer
#5 lore raw

Re: [PATCH 03/35] irqchip/gic-v3-its: Allow LPI invalidation via the DirectLPI interface

In reply to Marc Zyngier

Hi Marc,

I get one kernel panic with this patch on D05.

(I don't have the GICv4.1 board at the moment. I have to wait for the
appropriate HW to do more tests.)

Quoted 27 lines · Marc Zyngier

We currently don't make much use of the DirectLPI feature, and it would be beneficial to do this more, if only because it becomes a mandatory feature for GICv4.1.

---
  drivers/irqchip/irq-gic-v3-its.c | 51 +++++++++++++++++++++++---------
  1 file changed, 37 insertions(+), 14 deletions(-)
drivers/irqchip/irq-gic-v3-its.c +6 -0
index 58cb233cf138..c94eb287393b 100644
@@ -175,6 +175,12 @@ static DEFINE_IDA(its_vpeid_ida);
  #define gic_data_rdist_rd_base()	(gic_data_rdist()->rd_base)
  #define gic_data_rdist_vlpi_base()	(gic_data_rdist_rd_base() + SZ_128K)
 
+static inline u32 its_get_event_id(struct irq_data *d)
+{
+	struct its_device *its_dev = irq_data_get_irq_chip_data(d);
+	return d->hwirq - its_dev->event_map.lpi_base;
+}
+
  static struct its_collection *dev_event_to_col(struct its_device *its_dev,
  					       u32 event)
  {

drivers/irqchip/irq-gic-v3-its.c static struct its_collection *dev_event_to_col(struct its_device *its_dev, · line 198

Quoted 10 lines
drivers/irqchip/irq-gic-v3-its.c +7 -0
@@ -183,6 +189,13 @@ static struct its_collection *dev_event_to_col(struct its_device *its_dev,
  	return its->collections + its_dev->event_map.col_map[event];
  }
 
+static struct its_collection *irq_to_col(struct irq_data *d)
+{
+	struct its_device *its_dev = irq_data_get_irq_chip_data(d);
+
+	return dev_event_to_col(its_dev, its_get_event_id(d));
+}
+

irq_to_col uses device's event_map and col_map to get the target collection, yes it works well with device's LPI.
But direct_lpi_inv also pass doorbells to it...

We don't allocate doorbells for any devices, instead for each vPE.

And see below,

Quoted 53 lines
drivers/irqchip/irq-gic-v3-its.c +17 -9
  static struct its_collection *valid_col(struct its_collection *col)
  {
  	if (WARN_ON_ONCE(col->target_address & GENMASK_ULL(15, 0)))
@@ -1031,12 +1044,6 @@ static void its_send_vinvall(struct its_node *its, struct its_vpe *vpe)
   * irqchip functions - assumes MSI, mostly.
   */
 
-static inline u32 its_get_event_id(struct irq_data *d)
-{
-	struct its_device *its_dev = irq_data_get_irq_chip_data(d);
-	return d->hwirq - its_dev->event_map.lpi_base;
-}
-
  static void lpi_write_config(struct irq_data *d, u8 clr, u8 set)
  {
  	irq_hw_number_t hwirq;
@@ -1081,12 +1088,28 @@ static void wait_for_syncr(void __iomem *rdbase)
  		cpu_relax();
  }
 
+static void direct_lpi_inv(struct irq_data *d)
+{
+	struct its_collection *col;
+	void __iomem *rdbase;
+
+	/* Target the redistributor this LPI is currently routed to */
+	col = irq_to_col(d);
+	rdbase = per_cpu_ptr(gic_rdists->rdist, col->col_id)->rd_base;
+	gic_write_lpir(d->hwirq, rdbase + GICR_INVLPIR);
+
+	wait_for_syncr(rdbase);
+}
+
  static void lpi_update_config(struct irq_data *d, u8 clr, u8 set)
  {
  	struct its_device *its_dev = irq_data_get_irq_chip_data(d);
 
  	lpi_write_config(d, clr, set);
-	its_send_inv(its_dev, its_get_event_id(d));
+	if (gic_rdists->has_direct_lpi && !irqd_is_forwarded_to_vcpu(d))
+		direct_lpi_inv(d);
+	else
+		its_send_inv(its_dev, its_get_event_id(d));
  }
 
  static void its_vlpi_set_doorbell(struct irq_data *d, bool enable)
@@ -2912,15 +2935,15 @@ static void its_vpe_send_cmd(struct its_vpe *vpe,
 
  static void its_vpe_send_inv(struct irq_data *d)
  {
-	struct its_vpe *vpe = irq_data_get_irq_chip_data(d);
-
  	if (gic_rdists->has_direct_lpi) {

drivers/irqchip/irq-gic-v3-its.c static void its_vpe_send_cmd(struct its_vpe *vpe, · line 2943

Quoted 10 lines
drivers/irqchip/irq-gic-v3-its.c +5 -5
-		void __iomem *rdbase;
-
-		rdbase = per_cpu_ptr(gic_rdists->rdist, vpe->col_idx)->rd_base;
-		gic_write_lpir(vpe->vpe_db_lpi, rdbase + GICR_INVLPIR);
-		wait_for_syncr(rdbase);
+		/*
+		 * Don't mess about. Generating the invalidation is easily
+		 * done by using the parent irq_data, just like below.
+		 */
+		direct_lpi_inv(d->parent_data);

"GICv4-vpe"'s parent is "GICv3", not "ITS". What do we expect with irq_data_get_irq_chip_data(parent's irq_data)?

I noticed it when running this series on D05 (with GICv4.0 and DirectLPI support), panic call trace attached below.
I think we really need a fix here.

Thanks,
zenghui

drivers/irqchip/irq-gic-v3-its.c static void its_vpe_send_cmd(struct its_vpe *vpe, · line 2949

Quoted 7 lines
drivers/irqchip/irq-gic-v3-its.c +2 -0
  	} else {
+		struct its_vpe *vpe = irq_data_get_irq_chip_data(d);
+
  		its_vpe_send_cmd(vpe, its_send_inv);
  	}
  }

---8<

[ 2046.193227] Unable to handle kernel paging request at virtual address 0000002045fa4d92
[ 2046.201350] Mem abort info:
[ 2046.204132]   ESR = 0x96000005
[ 2046.207174]   Exception class = DABT (current EL), IL = 32 bits
[ 2046.213081]   SET = 0, FnV = 0
[ 2046.216123]   EA = 0, S1PTW = 0
[ 2046.219251] Data abort info:
[ 2046.222119]   ISV = 0, ISS = 0x00000005
[ 2046.225942]   CM = 0, WnR = 0
[ 2046.228898] user pgtable: 4k pages, 48-bit VAs, pgdp=0000001fa85d7000
[ 2046.235326] [0000002045fa4d92] pgd=0000001fb185d003, pud=0000000000000000
[ 2046.242103] Internal error: Oops: 96000005 [#1] PREEMPT SMP
[ 2046.247664] Modules linked in: openvswitch nsh nf_conncount nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter sunrpc vfat fat aes_ce_blk crypto_simd cryptd aesel vhost_net tun vhost tap ip_tables ext4 mbcache jbd2 sd_mod marvell hisi_sas_v2_hw hisi_sas_main ipmi_si libsas scsi_transport_sas hns_mdio hnae br_netfilter bridge dm_mod stp llc nvme nvme_core xt_sctp sctp libcrc32c nbd
[ 2046.307974] CPU: 3 PID: 15094 Comm: CPU 0/KVM Kdump: loaded Not tainted 5.3.0gic-v4.1-devel+ #2
[ 2046.316658] Hardware name: Huawei TaiShan 2280 /BC11SPCD, BIOS 1.58 10/29/2018
[ 2046.323867] pstate: 60000085 (nZCv daIf -PAN -UAO)
[ 2046.328646] pc : direct_lpi_inv+0x7c/0xe0
[ 2046.332643] lr : its_vpe_send_inv.isra.24+0x50/0x60
[ 2046.337508] sp : ffff000034cab780
[ 2046.340810] x29: ffff000034cab780 x28: ffff000010e7c000
[ 2046.346110] x27: 0000000000000002 x26: ffff0000113718c0
[ 2046.351410] x25: ffff00001136f000 x24: ffff80af8a3f7420
[ 2046.356710] x23: 0000000000000001 x22: 0000000000000001
[ 2046.362010] x21: ffff80af95d96518 x20: ffff802fb78bf380
[ 2046.367309] x19: ffff80af95d96548 x18: 0000000000000000
[ 2046.372609] x17: 0000000000000000 x16: 0000000000000000
[ 2046.377908] x15: 0000000000000000 x14: 0000000000000000
[ 2046.383207] x13: 0000000000000000 x12: 0000000000000088
[ 2046.388506] x11: 0000000000000002 x10: ffff00001156a000
[ 2046.393805] x9 : 0000000000000000 x8 : ffff00001136f848
[ 2046.399105] x7 : ffff00001018c6a8 x6 : 0000001fba060000
[ 2046.404405] x5 : ffff000012c40000 x4 : 0000000045fa26c9
[ 2046.409704] x3 : ffff801fbb429800 x2 : 00000000000026c9
[ 2046.415004] x1 : ffff00001136f8a8 x0 : ffff000012163000
[ 2046.420303] Call trace:
[ 2046.422739]  direct_lpi_inv+0x7c/0xe0
[ 2046.426388]  its_vpe_send_inv.isra.24+0x50/0x60
[ 2046.430906]  its_vpe_unmask_irq+0x34/0x40
[ 2046.434904]  unmask_irq.part.5+0x30/0x50
[ 2046.438814]  irq_enable+0x78/0x98
[ 2046.442116]  __irq_startup+0x88/0x90
[ 2046.445680]  irq_startup+0x7c/0x140
[ 2046.449156]  __enable_irq+0x80/0x90
[ 2046.452632]  enable_irq+0x58/0xb0
[ 2046.455934]  its_make_vpe_non_resident+0xf0/0x108
[ 2046.460626]  vgic_v4_put+0x64/0x70
[ 2046.464015]  kvm_arch_vcpu_blocking+0x34/0x68
[ 2046.468360]  kvm_vcpu_block+0x50/0x598
[ 2046.472097]  kvm_handle_wfx+0x118/0x3d8
[ 2046.475920]  handle_exit+0x14c/0x1c8
[ 2046.479484]  kvm_arch_vcpu_ioctl_run+0x338/0xab8
[ 2046.484088]  kvm_vcpu_ioctl+0x3c8/0xa60
[ 2046.487912]  do_vfs_ioctl+0xc4/0x7f0
[ 2046.491475]  ksys_ioctl+0x8c/0xa0
[ 2046.494778]  __arm64_sys_ioctl+0x28/0x38
[ 2046.498689]  el0_svc_common.constprop.0+0x80/0x1b8
[ 2046.503467]  el0_svc_handler+0x34/0x90
[ 2046.507204]  el0_svc+0x8/0xc
[ 2046.510076] Code: d0006f41 f940d865 9122a021 d000dee0 (786478c3)
[ 2046.516158] ---[ end trace f6392171e61487dc ]---
[ 2046.520763] Kernel panic - not syncing: Fatal exception
[ 2046.525976] SMP: stopping secondary CPUs
[ 2046.529913] Kernel Offset: disabled
[ 2046.533391] CPU features: 0x0002,20006008
[ 2046.537387] Memory Limit: none
reviewer
#25 lore raw

Re: [PATCH 10/35] irqchip/gic-v4.1: VPE table (aka GICR_VPROPBASER) allocation

In reply to Marc Zyngier

Hi Marc,

Two more questions below.

Quoted 41 lines · Marc Zyngier
Quoted 40 lines · Zenghui Yu

Hi Marc,

Some questions about this patch, mostly to confirm that I would understand things here correctly.

Quoted 14 lines · Marc Zyngier

GICv4.1 defines a new VPE table that is potentially shared between both the ITSs and the redistributors, following complicated affinity rules.

To make things more confusing, the programming of this table at the redistributor level is reusing the GICv4.0 GICR_VPROPBASER register for something completely different.

The code flow is somewhat complexified by the need to respect the affinities required by the HW, meaning that tables can either be inherited from a previously discovered ITS or redistributor.

---

[...]

Quoted 17 lines
drivers/irqchip/irq-gic-v3-its.c +13 -0
@@ -1962,6 +1965,65 @@ static bool its_parse_indirect_baser(struct its_node *its,
   	return indirect;
   }
 
+static u32 compute_common_aff(u64 val)
+{
+	u32 aff, clpiaff;
+
+	aff = FIELD_GET(GICR_TYPER_AFFINITY, val);
+	clpiaff = FIELD_GET(GICR_TYPER_COMMON_LPI_AFF, val);
+
+	return aff & ~(GENMASK(31, 0) >> (clpiaff * 8));
+}
+
+static u32 compute_its_aff(struct its_node *its)
+{
+	u64 val;
Quoted 8 lines
Quoted 8 lines

drivers/irqchip/irq-gic-v3-its.c static bool its_parse_indirect_baser(struct its_node *its, · line 1988

Quoted 8 lines
drivers/irqchip/irq-gic-v3-its.c +8 -0
+	u32 svpet;
+
+	/*
+	 * Reencode the ITS SVPET and MPIDR as a GICR_TYPER, and compute
+	 * the resulting affinity. We then use that to see if this match
+	 * our own affinity.
+	 */
+	svpet = FIELD_GET(GITS_TYPER_SVPET, its->typer);

The spec says, ITS does not share vPE table with Redistributors when SVPET==0. It seems that we miss this rule and simply regard SVPET as GICR_TYPER_COMMON_LPI_AFF here. Am I wrong?

Quoted 125 lines
Quoted 44 lines
Quoted 44 lines
drivers/irqchip/irq-gic-v3-its.c +38 -0
+	val  = FIELD_PREP(GICR_TYPER_COMMON_LPI_AFF, svpet);
+	val |= FIELD_PREP(GICR_TYPER_AFFINITY, its->mpidr);
+	return compute_common_aff(val);
+}
+
+static struct its_node *find_sibbling_its(struct its_node *cur_its)
+{
+	struct its_node *its;
+	u32 aff;
+
+	if (!FIELD_GET(GITS_TYPER_SVPET, cur_its->typer))
+		return NULL;
+
+	aff = compute_its_aff(cur_its);
+
+	list_for_each_entry(its, &its_nodes, entry) {
+		u64 baser;
+
+		if (!is_v4_1(its) || its == cur_its)
+			continue;
+
+		if (!FIELD_GET(GITS_TYPER_SVPET, its->typer))
+			continue;
+
+		if (aff != compute_its_aff(its))
+			continue;
+
+		/* GICv4.1 guarantees that the vPE table is GITS_BASER2 */
+		baser = its->tables[2].val;
+		if (!(baser & GITS_BASER_VALID))
+			continue;
+
+		return its;
+	}
+
+	return NULL;
+}
+
   static void its_free_tables(struct its_node *its)
   {
   	int i;
@@ -2004,6 +2066,15 @@ static int its_alloc_tables(struct its_node *its)
   			break;
 
Quoted 9 lines

drivers/irqchip/irq-gic-v3-its.c static int its_alloc_tables(struct its_node *its) · line 2073

Quoted 6 lines
drivers/irqchip/irq-gic-v3-its.c +5 -0
   		case GITS_BASER_TYPE_VCPU:
+			if (is_v4_1(its)) {
+				struct its_node *sibbling;
+
+				if ((sibbling = find_sibbling_its(its))) {
+					its->tables[2] = sibbling->tables[2];

This means thst the vPE table is shared between ITSs which are under the same affinity group?

That's indeed what this is trying to do...

Quoted 2 lines

Don't we need to set GITS_BASER (by its_setup_baser()) here to tell this ITS where the vPE table lacates?

Absolutely. This is a bug. I didn't spot it as my model only has a single ITS.

Quoted 54 lines
Quoted 36 lines
drivers/irqchip/irq-gic-v3-its.c +28 -0
+					continue;
+				}
+			}
+
   			indirect = its_parse_indirect_baser(its, baser,
   							    psz, &order,
   							    ITS_MAX_VPEID_BITS);
@@ -2025,6 +2096,212 @@ static int its_alloc_tables(struct its_node *its)
   	return 0;
   }
 
+static u64 inherit_vpe_l1_table_from_its(void)
+{
+	struct its_node *its;
+	u64 val;
+	u32 aff;
+
+	val = gic_read_typer(gic_data_rdist_rd_base() + GICR_TYPER);
+	aff = compute_common_aff(val);
+
+	list_for_each_entry(its, &its_nodes, entry) {
+		u64 baser;
+
+		if (!is_v4_1(its))
+			continue;
+
+		if (!FIELD_GET(GITS_TYPER_SVPET, its->typer))
+			continue;
+
+		if (aff != compute_its_aff(its))
+			continue;
+
+		/* GICv4.1 guarantees that the vPE table is GITS_BASER2 */
+		baser = its->tables[2].val;
+		if (!(baser & GITS_BASER_VALID))

drivers/irqchip/irq-gic-v3-its.c static int its_alloc_tables(struct its_node *its) · line 2132

Quoted 10 lines
drivers/irqchip/irq-gic-v3-its.c +10 -0
+			continue;
+
+		/* We have a winner! */
+		val  = GICR_VPROPBASER_4_1_VALID;
+		if (baser & GITS_BASER_INDIRECT)
+			val |= GICR_VPROPBASER_4_1_INDIRECT;
+		val |= FIELD_PREP(GICR_VPROPBASER_4_1_PAGE_SIZE,
+				  FIELD_GET(GITS_BASER_PAGE_SIZE_MASK, baser));
+		val |= FIELD_PREP(GICR_VPROPBASER_4_1_ADDR,
+				  GITS_BASER_ADDR_48_to_52(baser) >> 12);

I remember the spec says,
GITS_BASER2 "points to" the ITS *vPE table*, which provides mappings from the vPEID to target Redistributor and associated vpt address. In GICv4.1 GICR_VPROPBASER "points to" the *vPE Configuration table*, which stores the locations of each vPE's LPI configuration and pending table.

Quoted 8 lines

And the code here says, if we can find an ITS (which is under the same CommonLPIAff group with this Redistributor) has already been probed and allocated an vPE table, then use this vPE table as this Redist's vPE Configuration table.
So I infer that in GICv4.1, *vPE table* and *vPE Configuration table* are actually the same concept? And this table is shared between ITSs and Redists which are under the same affinity group?
Please fix me if I have any misunderstanding.

Quoted 5 lines

Indeed. The whole idea is that ITSs and RDs can share a vPE table if they are in the same affinity group (such as a socket, for example). This is what is missing from v4.0 where the ITS knows about vPEs, but doesn't know about residency. With that in place, the HW is able to do a lot more for us.

Thanks for your education!

I really want to know *how* does GICv4.1 ITS know about the vPE residency (in hardware level)?

I can understand that Redistributor can easily achieve it by VPENDBASER's Valid and vPEID field. And it's necessary for ITS to know the residency so that it can determine whether it's appropriate to trigger default doorbell for vPE. But I have no knowledge about how can it be achieved in hardware details.

Thanks in advance for your pointer. :)

Zenghui

author maintainer
#6 lore raw

Re: [PATCH 03/35] irqchip/gic-v3-its: Allow LPI invalidation via the DirectLPI interface

In reply to Zenghui Yu

Hi Zenghui,

Quoted 3 lines · Zenghui Yu

Hi Marc,

I get one kernel panic with this patch on D05.

Ah, surprise! I haven't had time to test this on a D05 yet, such in a hurry to push the damn thing out of the building...

Quoted 47 lines

(I don't have the GICv4.1 board at the moment. I have to wait for the
appropriate HW to do more tests.)

Quoted 27 lines · Marc Zyngier

We currently don't make much use of the DirectLPI feature, and it would be beneficial to do this more, if only because it becomes a mandatory feature for GICv4.1.

---
  drivers/irqchip/irq-gic-v3-its.c | 51 +++++++++++++++++++++++---------
  1 file changed, 37 insertions(+), 14 deletions(-)
drivers/irqchip/irq-gic-v3-its.c +6 -0
index 58cb233cf138..c94eb287393b 100644
@@ -175,6 +175,12 @@ static DEFINE_IDA(its_vpeid_ida);
  #define gic_data_rdist_rd_base()	(gic_data_rdist()->rd_base)
  #define gic_data_rdist_vlpi_base()	(gic_data_rdist_rd_base() + SZ_128K)
 
+static inline u32 its_get_event_id(struct irq_data *d)
+{
+	struct its_device *its_dev = irq_data_get_irq_chip_data(d);
+	return d->hwirq - its_dev->event_map.lpi_base;
+}
+
  static struct its_collection *dev_event_to_col(struct its_device *its_dev,
  					       u32 event)
  {

drivers/irqchip/irq-gic-v3-its.c static struct its_collection *dev_event_to_col(struct its_device *its_dev, · line 198

Quoted 10 lines
drivers/irqchip/irq-gic-v3-its.c +7 -0
@@ -183,6 +189,13 @@ static struct its_collection *dev_event_to_col(struct its_device *its_dev,
  	return its->collections + its_dev->event_map.col_map[event];
  }
 
+static struct its_collection *irq_to_col(struct irq_data *d)
+{
+	struct its_device *its_dev = irq_data_get_irq_chip_data(d);
+
+	return dev_event_to_col(its_dev, its_get_event_id(d));
+}
+

irq_to_col uses device's event_map and col_map to get the target collection, yes it works well with device's LPI.
But direct_lpi_inv also pass doorbells to it...

Quoted 1 line

We don't allocate doorbells for any devices, instead for each vPE.

Hmm. Yes, you're right. It looks like I've been carried away on this one. I'll have a look.

Quoted 61 lines

And see below,

Quoted 58 lines
drivers/irqchip/irq-gic-v3-its.c +17 -14
  static struct its_collection *valid_col(struct its_collection *col)
  {
  	if (WARN_ON_ONCE(col->target_address & GENMASK_ULL(15, 0)))
@@ -1031,12 +1044,6 @@ static void its_send_vinvall(struct its_node *its, struct its_vpe *vpe)
   * irqchip functions - assumes MSI, mostly.
   */
 
-static inline u32 its_get_event_id(struct irq_data *d)
-{
-	struct its_device *its_dev = irq_data_get_irq_chip_data(d);
-	return d->hwirq - its_dev->event_map.lpi_base;
-}
-
  static void lpi_write_config(struct irq_data *d, u8 clr, u8 set)
  {
  	irq_hw_number_t hwirq;
@@ -1081,12 +1088,28 @@ static void wait_for_syncr(void __iomem *rdbase)
  		cpu_relax();
  }
 
+static void direct_lpi_inv(struct irq_data *d)
+{
+	struct its_collection *col;
+	void __iomem *rdbase;
+
+	/* Target the redistributor this LPI is currently routed to */
+	col = irq_to_col(d);
+	rdbase = per_cpu_ptr(gic_rdists->rdist, col->col_id)->rd_base;
+	gic_write_lpir(d->hwirq, rdbase + GICR_INVLPIR);
+
+	wait_for_syncr(rdbase);
+}
+
  static void lpi_update_config(struct irq_data *d, u8 clr, u8 set)
  {
  	struct its_device *its_dev = irq_data_get_irq_chip_data(d);
 
  	lpi_write_config(d, clr, set);
-	its_send_inv(its_dev, its_get_event_id(d));
+	if (gic_rdists->has_direct_lpi && !irqd_is_forwarded_to_vcpu(d))
+		direct_lpi_inv(d);
+	else
+		its_send_inv(its_dev, its_get_event_id(d));
  }
 
  static void its_vlpi_set_doorbell(struct irq_data *d, bool enable)
@@ -2912,15 +2935,15 @@ static void its_vpe_send_cmd(struct its_vpe *vpe,
 
  static void its_vpe_send_inv(struct irq_data *d)
  {
-	struct its_vpe *vpe = irq_data_get_irq_chip_data(d);
-
  	if (gic_rdists->has_direct_lpi) {
-		void __iomem *rdbase;
-
-		rdbase = per_cpu_ptr(gic_rdists->rdist, vpe->col_idx)->rd_base;
-		gic_write_lpir(vpe->vpe_db_lpi, rdbase + GICR_INVLPIR);
-		wait_for_syncr(rdbase);
Quoted 8 lines

drivers/irqchip/irq-gic-v3-its.c static void its_vpe_send_cmd(struct its_vpe *vpe, · line 2943

Quoted 5 lines
drivers/irqchip/irq-gic-v3-its.c +5 -0
+		/*
+		 * Don't mess about. Generating the invalidation is easily
+		 * done by using the parent irq_data, just like below.
+		 */
+		direct_lpi_inv(d->parent_data);

"GICv4-vpe"'s parent is "GICv3", not "ITS". What do we expect with irq_data_get_irq_chip_data(parent's irq_data)?

Yup, terrible mix up. d->parent_data comes from the fact that we want to invalidate the LPI and not d->hwirq (which is the VPEID). But doing so, we also confuse direct_lpi_inv(), which expects to find meaningful data (the its_dev) as chip data (and the irq_to_col doesn't help either).

To sum it up, the whole thing is busted, I'll have a brown paper bag, thank you very much... :-(. Let me work on a fix.

Thanks,

	M.
Signature · 1 line
Jazz is not dead, it just smells funny...
author maintainer
#26 lore raw

Re: [PATCH 10/35] irqchip/gic-v4.1: VPE table (aka GICR_VPROPBASER) allocation

In reply to Zenghui Yu
Quoted 50 lines · Zenghui Yu

Hi Marc,

Two more questions below.

Quoted 45 lines · Marc Zyngier
Quoted 44 lines · Zenghui Yu

Hi Marc,

Some questions about this patch, mostly to confirm that I would understand things here correctly.

Quoted 14 lines · Marc Zyngier

GICv4.1 defines a new VPE table that is potentially shared between both the ITSs and the redistributors, following complicated affinity rules.

To make things more confusing, the programming of this table at the redistributor level is reusing the GICv4.0 GICR_VPROPBASER register for something completely different.

The code flow is somewhat complexified by the need to respect the affinities required by the HW, meaning that tables can either be inherited from a previously discovered ITS or redistributor.

---

[...]

Quoted 21 lines
drivers/irqchip/irq-gic-v3-its.c +17 -0
@@ -1962,6 +1965,65 @@ static bool its_parse_indirect_baser(struct its_node *its,
   	return indirect;
   }
 
+static u32 compute_common_aff(u64 val)
+{
+	u32 aff, clpiaff;
+
+	aff = FIELD_GET(GICR_TYPER_AFFINITY, val);
+	clpiaff = FIELD_GET(GICR_TYPER_COMMON_LPI_AFF, val);
+
+	return aff & ~(GENMASK(31, 0) >> (clpiaff * 8));
+}
+
+static u32 compute_its_aff(struct its_node *its)
+{
+	u64 val;
+	u32 svpet;
+
+	/*
+	 * Reencode the ITS SVPET and MPIDR as a GICR_TYPER, and compute
Quoted 8 lines
Quoted 4 lines
Quoted 4 lines

drivers/irqchip/irq-gic-v3-its.c static bool its_parse_indirect_baser(struct its_node *its, · line 1988

Quoted 4 lines
drivers/irqchip/irq-gic-v3-its.c +4 -0
+	 * the resulting affinity. We then use that to see if this match
+	 * our own affinity.
+	 */
+	svpet = FIELD_GET(GITS_TYPER_SVPET, its->typer);

The spec says, ITS does not share vPE table with Redistributors when SVPET==0. It seems that we miss this rule and simply regard SVPET as GICR_TYPER_COMMON_LPI_AFF here. Am I wrong?

Correct. I missed the case where the ITS doesn't share anything. That's pretty unlikely though (you loose all the benefit of v4.1, and I don't really see how you'd make it work reliably).

Quoted 132 lines
Quoted 126 lines
Quoted 45 lines
Quoted 45 lines
drivers/irqchip/irq-gic-v3-its.c +38 -0
+	val  = FIELD_PREP(GICR_TYPER_COMMON_LPI_AFF, svpet);
+	val |= FIELD_PREP(GICR_TYPER_AFFINITY, its->mpidr);
+	return compute_common_aff(val);
+}
+
+static struct its_node *find_sibbling_its(struct its_node *cur_its)
+{
+	struct its_node *its;
+	u32 aff;
+
+	if (!FIELD_GET(GITS_TYPER_SVPET, cur_its->typer))
+		return NULL;
+
+	aff = compute_its_aff(cur_its);
+
+	list_for_each_entry(its, &its_nodes, entry) {
+		u64 baser;
+
+		if (!is_v4_1(its) || its == cur_its)
+			continue;
+
+		if (!FIELD_GET(GITS_TYPER_SVPET, its->typer))
+			continue;
+
+		if (aff != compute_its_aff(its))
+			continue;
+
+		/* GICv4.1 guarantees that the vPE table is GITS_BASER2 */
+		baser = its->tables[2].val;
+		if (!(baser & GITS_BASER_VALID))
+			continue;
+
+		return its;
+	}
+
+	return NULL;
+}
+
   static void its_free_tables(struct its_node *its)
   {
   	int i;
@@ -2004,6 +2066,15 @@ static int its_alloc_tables(struct its_node *its)
   			break;
 
Quoted 9 lines

drivers/irqchip/irq-gic-v3-its.c static int its_alloc_tables(struct its_node *its) · line 2073

Quoted 6 lines
drivers/irqchip/irq-gic-v3-its.c +5 -0
   		case GITS_BASER_TYPE_VCPU:
+			if (is_v4_1(its)) {
+				struct its_node *sibbling;
+
+				if ((sibbling = find_sibbling_its(its))) {
+					its->tables[2] = sibbling->tables[2];

This means thst the vPE table is shared between ITSs which are under the same affinity group?

That's indeed what this is trying to do...

Quoted 2 lines

Don't we need to set GITS_BASER (by its_setup_baser()) here to tell this ITS where the vPE table lacates?

Absolutely. This is a bug. I didn't spot it as my model only has a single ITS.

Quoted 54 lines
Quoted 36 lines
drivers/irqchip/irq-gic-v3-its.c +28 -0
+					continue;
+				}
+			}
+
   			indirect = its_parse_indirect_baser(its, baser,
   							    psz, &order,
   							    ITS_MAX_VPEID_BITS);
@@ -2025,6 +2096,212 @@ static int its_alloc_tables(struct its_node *its)
   	return 0;
   }
 
+static u64 inherit_vpe_l1_table_from_its(void)
+{
+	struct its_node *its;
+	u64 val;
+	u32 aff;
+
+	val = gic_read_typer(gic_data_rdist_rd_base() + GICR_TYPER);
+	aff = compute_common_aff(val);
+
+	list_for_each_entry(its, &its_nodes, entry) {
+		u64 baser;
+
+		if (!is_v4_1(its))
+			continue;
+
+		if (!FIELD_GET(GITS_TYPER_SVPET, its->typer))
+			continue;
+
+		if (aff != compute_its_aff(its))
+			continue;
+
+		/* GICv4.1 guarantees that the vPE table is GITS_BASER2 */
+		baser = its->tables[2].val;
+		if (!(baser & GITS_BASER_VALID))

drivers/irqchip/irq-gic-v3-its.c static int its_alloc_tables(struct its_node *its) · line 2132

Quoted 10 lines
drivers/irqchip/irq-gic-v3-its.c +10 -0
+			continue;
+
+		/* We have a winner! */
+		val  = GICR_VPROPBASER_4_1_VALID;
+		if (baser & GITS_BASER_INDIRECT)
+			val |= GICR_VPROPBASER_4_1_INDIRECT;
+		val |= FIELD_PREP(GICR_VPROPBASER_4_1_PAGE_SIZE,
+				  FIELD_GET(GITS_BASER_PAGE_SIZE_MASK, baser));
+		val |= FIELD_PREP(GICR_VPROPBASER_4_1_ADDR,
+				  GITS_BASER_ADDR_48_to_52(baser) >> 12);

I remember the spec says,
GITS_BASER2 "points to" the ITS *vPE table*, which provides mappings from the vPEID to target Redistributor and associated vpt address. In GICv4.1 GICR_VPROPBASER "points to" the *vPE Configuration table*, which stores the locations of each vPE's LPI configuration and pending table.

Quoted 8 lines

And the code here says, if we can find an ITS (which is under the same CommonLPIAff group with this Redistributor) has already been probed and allocated an vPE table, then use this vPE table as this Redist's vPE Configuration table.
So I infer that in GICv4.1, *vPE table* and *vPE Configuration table* are actually the same concept? And this table is shared between ITSs and Redists which are under the same affinity group?
Please fix me if I have any misunderstanding.

Quoted 5 lines

Indeed. The whole idea is that ITSs and RDs can share a vPE table if they are in the same affinity group (such as a socket, for example). This is what is missing from v4.0 where the ITS knows about vPEs, but doesn't know about residency. With that in place, the HW is able to do a lot more for us.

Quoted 4 lines

Thanks for your education!

I really want to know *how* does GICv4.1 ITS know about the vPE residency (in hardware level)?

Hey, I'm a SW guy, I don't design the hardware! ;-)

Quoted 5 lines

I can understand that Redistributor can easily achieve it by VPENDBASER's Valid and vPEID field. And it's necessary for ITS to know the residency so that it can determine whether it's appropriate to trigger default doorbell for vPE. But I have no knowledge about how can it be achieved in hardware details.

My take is that the RD and the ITS can communicate over the shared table, hence my remark above about SVPET==0. But as I said, I'm not a HW guy.

Thanks,

	M.
Signature · 1 line
Jazz is not dead, it just smells funny...
author maintainer
#7 lore raw

Re: [PATCH 03/35] irqchip/gic-v3-its: Allow LPI invalidation via the DirectLPI interface

In reply to Zenghui Yu
Quoted 3 lines · Zenghui Yu

Hi Marc,

I get one kernel panic with this patch on D05.

Can you please try this (completely untested for now) on top of the whole series? I'll otherwise give it a go next week.

Thanks,

	M.
drivers/irqchip/irq-gic-v3-its.c +8 -7
index bc55327406b7..9d24236d1257 100644
@@ -3461,15 +3461,16 @@ static void its_vpe_send_cmd(struct its_vpe *vpe,
 
 static void its_vpe_send_inv(struct irq_data *d)
 {
+	struct its_vpe *vpe = irq_data_get_irq_chip_data(d);
+
 	if (gic_rdists->has_direct_lpi) {
-		/*
-		 * Don't mess about. Generating the invalidation is easily
-		 * done by using the parent irq_data, just like below.
-		 */
-		direct_lpi_inv(d->parent_data);
-	} else {
-		struct its_vpe *vpe = irq_data_get_irq_chip_data(d);
+		void __iomem *rdbase;
 
+		/* Target the redistributor this VPE is currently known on */
+		rdbase = per_cpu_ptr(gic_rdists->rdist, vpe->col_idx)->rd_base;
+		gic_write_lpir(d->parent_data->hwirq, rdbase + GICR_INVLPIR);
+		wait_for_syncr(rdbase);
+	} else {
 		its_vpe_send_cmd(vpe, its_send_inv);
 	}
 }
Signature · 1 line
Jazz is not dead, it just smells funny...
author maintainer
#27 lore raw

Re: [PATCH 10/35] irqchip/gic-v4.1: VPE table (aka GICR_VPROPBASER) allocation

In reply to Marc Zyngier
Quoted 60 lines · Marc Zyngier
Quoted 50 lines · Zenghui Yu

Hi Marc,

Two more questions below.

Quoted 45 lines · Marc Zyngier
Quoted 44 lines · Zenghui Yu

Hi Marc,

Some questions about this patch, mostly to confirm that I would understand things here correctly.

Quoted 14 lines · Marc Zyngier

GICv4.1 defines a new VPE table that is potentially shared between both the ITSs and the redistributors, following complicated affinity rules.

To make things more confusing, the programming of this table at the redistributor level is reusing the GICv4.0 GICR_VPROPBASER register for something completely different.

The code flow is somewhat complexified by the need to respect the affinities required by the HW, meaning that tables can either be inherited from a previously discovered ITS or redistributor.

---

[...]

Quoted 21 lines
drivers/irqchip/irq-gic-v3-its.c +17 -0
@@ -1962,6 +1965,65 @@ static bool its_parse_indirect_baser(struct its_node *its,
   	return indirect;
   }
 
+static u32 compute_common_aff(u64 val)
+{
+	u32 aff, clpiaff;
+
+	aff = FIELD_GET(GICR_TYPER_AFFINITY, val);
+	clpiaff = FIELD_GET(GICR_TYPER_COMMON_LPI_AFF, val);
+
+	return aff & ~(GENMASK(31, 0) >> (clpiaff * 8));
+}
+
+static u32 compute_its_aff(struct its_node *its)
+{
+	u64 val;
+	u32 svpet;
+
+	/*
+	 * Reencode the ITS SVPET and MPIDR as a GICR_TYPER, and compute
Quoted 8 lines
Quoted 4 lines
Quoted 4 lines

drivers/irqchip/irq-gic-v3-its.c static bool its_parse_indirect_baser(struct its_node *its, · line 1988

Quoted 4 lines
drivers/irqchip/irq-gic-v3-its.c +4 -0
+	 * the resulting affinity. We then use that to see if this match
+	 * our own affinity.
+	 */
+	svpet = FIELD_GET(GITS_TYPER_SVPET, its->typer);

The spec says, ITS does not share vPE table with Redistributors when SVPET==0. It seems that we miss this rule and simply regard SVPET as GICR_TYPER_COMMON_LPI_AFF here. Am I wrong?

Quoted 3 lines

Correct. I missed the case where the ITS doesn't share anything. That's pretty unlikely though (you loose all the benefit of v4.1, and I don't really see how you'd make it work reliably).

Actually, this is already handled...

Quoted 14 lines
Quoted 14 lines
Quoted 14 lines
Quoted 14 lines
Quoted 4 lines
drivers/irqchip/irq-gic-v3-its.c +2 -0
+	val  = FIELD_PREP(GICR_TYPER_COMMON_LPI_AFF, svpet);
+	val |= FIELD_PREP(GICR_TYPER_AFFINITY, its->mpidr);

drivers/irqchip/irq-gic-v3-its.c static bool its_parse_indirect_baser(struct its_node *its, · line 2000

Quoted 10 lines
drivers/irqchip/irq-gic-v3-its.c +10 -0
+	return compute_common_aff(val);
+}
+
+static struct its_node *find_sibbling_its(struct its_node *cur_its)
+{
+	struct its_node *its;
+	u32 aff;
+
+	if (!FIELD_GET(GITS_TYPER_SVPET, cur_its->typer))
+		return NULL;

... here. If SVPET is 0, there is no sibling, and we'll allocate a VPE table as usual.

Thanks,

	M.
Signature · 1 line
Jazz is not dead, it just smells funny...
reviewer
#29 lore raw

Re: [PATCH 10/35] irqchip/gic-v4.1: VPE table (aka GICR_VPROPBASER) allocation

In reply to Marc Zyngier
Quoted 22 lines · Marc Zyngier
Quoted 8 lines · Zenghui Yu
Quoted 6 lines · Marc Zyngier

Indeed. The whole idea is that ITSs and RDs can share a vPE table if they are in the same affinity group (such as a socket, for example). This is what is missing from v4.0 where the ITS knows about vPEs, but doesn't know about residency. With that in place, the HW is able to do a lot more for us.

Quoted 4 lines

Thanks for your education!

I really want to know *how* does GICv4.1 ITS know about the vPE residency (in hardware level)?

Hey, I'm a SW guy, I don't design the hardware! ;-)

Quoted 5 lines

I can understand that Redistributor can easily achieve it by VPENDBASER's Valid and vPEID field. And it's necessary for ITS to know the residency so that it can determine whether it's appropriate to trigger default doorbell for vPE. But I have no knowledge about how can it be achieved in hardware details.

Quoted 3 lines

My take is that the RD and the ITS can communicate over the shared table, hence my remark above about SVPET==0. But as I said, I'm not a HW guy.

;-) I should have asked our GIC engineers for these things.

Thanks,
zenghui

reviewer
#28 lore raw

Re: [PATCH 10/35] irqchip/gic-v4.1: VPE table (aka GICR_VPROPBASER) allocation

In reply to Marc Zyngier
Quoted 69 lines · Marc Zyngier
Quoted 60 lines · Marc Zyngier
Quoted 50 lines · Zenghui Yu

Hi Marc,

Two more questions below.

Quoted 45 lines · Marc Zyngier
Quoted 44 lines · Zenghui Yu

Hi Marc,

Some questions about this patch, mostly to confirm that I would understand things here correctly.

Quoted 14 lines · Marc Zyngier

GICv4.1 defines a new VPE table that is potentially shared between both the ITSs and the redistributors, following complicated affinity rules.

To make things more confusing, the programming of this table at the redistributor level is reusing the GICv4.0 GICR_VPROPBASER register for something completely different.

The code flow is somewhat complexified by the need to respect the affinities required by the HW, meaning that tables can either be inherited from a previously discovered ITS or redistributor.

---

[...]

Quoted 21 lines
drivers/irqchip/irq-gic-v3-its.c +17 -0
@@ -1962,6 +1965,65 @@ static bool its_parse_indirect_baser(struct its_node *its,
    	return indirect;
    }
 
+static u32 compute_common_aff(u64 val)
+{
+	u32 aff, clpiaff;
+
+	aff = FIELD_GET(GICR_TYPER_AFFINITY, val);
+	clpiaff = FIELD_GET(GICR_TYPER_COMMON_LPI_AFF, val);
+
+	return aff & ~(GENMASK(31, 0) >> (clpiaff * 8));
+}
+
+static u32 compute_its_aff(struct its_node *its)
+{
+	u64 val;
+	u32 svpet;
+
+	/*
+	 * Reencode the ITS SVPET and MPIDR as a GICR_TYPER, and compute
Quoted 8 lines
Quoted 4 lines
Quoted 4 lines

drivers/irqchip/irq-gic-v3-its.c static bool its_parse_indirect_baser(struct its_node *its, · line 1988

Quoted 4 lines
drivers/irqchip/irq-gic-v3-its.c +4 -0
+	 * the resulting affinity. We then use that to see if this match
+	 * our own affinity.
+	 */
+	svpet = FIELD_GET(GITS_TYPER_SVPET, its->typer);

The spec says, ITS does not share vPE table with Redistributors when SVPET==0. It seems that we miss this rule and simply regard SVPET as GICR_TYPER_COMMON_LPI_AFF here. Am I wrong?

Quoted 3 lines

Correct. I missed the case where the ITS doesn't share anything. That's pretty unlikely though (you loose all the benefit of v4.1, and I don't really see how you'd make it work reliably).

Actually, this is already handled...

Quoted 2 lines
Quoted 2 lines
Quoted 15 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
drivers/irqchip/irq-gic-v3-its.c +4 -0
+	val  = FIELD_PREP(GICR_TYPER_COMMON_LPI_AFF, svpet);
+	val |= FIELD_PREP(GICR_TYPER_AFFINITY, its->mpidr);
+	return compute_common_aff(val);
+}
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines

drivers/irqchip/irq-gic-v3-its.c static bool its_parse_indirect_baser(struct its_node *its, · line 2000

Quoted 8 lines
drivers/irqchip/irq-gic-v3-its.c +8 -0
+
+static struct its_node *find_sibbling_its(struct its_node *cur_its)
+{
+	struct its_node *its;
+	u32 aff;
+
+	if (!FIELD_GET(GITS_TYPER_SVPET, cur_its->typer))
+		return NULL;

... here. If SVPET is 0, there is no sibling, and we'll allocate a VPE table as usual.

Yes, I see. So we can safely encode the non-zero SVPET as COMMON_LPI_AFF in a GICR_TYPER when computing the affinity.

Thanks,
zenghui

reviewer
#8 lore raw

Re: [PATCH 03/35] irqchip/gic-v3-its: Allow LPI invalidation via the DirectLPI interface

In reply to Marc Zyngier
Quoted 7 lines · Marc Zyngier
Quoted 3 lines · Zenghui Yu

Hi Marc,

I get one kernel panic with this patch on D05.

Can you please try this (completely untested for now) on top of the whole series? I'll otherwise give it a go next week.

Yes, it helps. At least I don't see panic any more. Without this change, the host would get crashed as long as the VM is started.

Thanks,
zenghui

Quoted 24 lines

Thanks,

	M.
drivers/irqchip/irq-gic-v3-its.c +2 -7
index bc55327406b7..9d24236d1257 100644
@@ -3461,15 +3461,16 @@ static void its_vpe_send_cmd(struct its_vpe *vpe,
 
  static void its_vpe_send_inv(struct irq_data *d)
  {
+	struct its_vpe *vpe = irq_data_get_irq_chip_data(d);
+
  	if (gic_rdists->has_direct_lpi) {
-		/*
-		 * Don't mess about. Generating the invalidation is easily
-		 * done by using the parent irq_data, just like below.
-		 */
-		direct_lpi_inv(d->parent_data);
-	} else {
-		struct its_vpe *vpe = irq_data_get_irq_chip_data(d);

drivers/irqchip/irq-gic-v3-its.c static void its_vpe_send_cmd(struct its_vpe *vpe, · line 3476

Quoted 10 lines
drivers/irqchip/irq-gic-v3-its.c +6 -0
+		void __iomem *rdbase;
 
+		/* Target the redistributor this VPE is currently known on */
+		rdbase = per_cpu_ptr(gic_rdists->rdist, vpe->col_idx)->rd_base;
+		gic_write_lpir(d->parent_data->hwirq, rdbase + GICR_INVLPIR);
+		wait_for_syncr(rdbase);
+	} else {
  		its_vpe_send_cmd(vpe, its_send_inv);
  	}
  }

_______________________________________________
kvmarm mailing list
[email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm

reviewer
#40 lore raw

Re: [PATCH 20/35] irqchip/gic-v4.1: Allow direct invalidation of VLPIs

In reply to Marc Zyngier
Quoted 26 lines · Marc Zyngier

Just like for INVALL, GICv4.1 has grown a VPE-aware INVLPI register. Let's plumb it in and make use of the DirectLPI code in that case.

---
  drivers/irqchip/irq-gic-v3-its.c   | 19 +++++++++++++++++--
  include/linux/irqchip/arm-gic-v3.h |  1 +
  2 files changed, 18 insertions(+), 2 deletions(-)
drivers/irqchip/irq-gic-v3-its.c +7 -0
index b791c9beddf2..34595a7fcccb 100644
@@ -1200,13 +1200,27 @@ static void wait_for_syncr(void __iomem *rdbase)
 
  static void direct_lpi_inv(struct irq_data *d)
  {
+	struct its_vlpi_map *map = get_vlpi_map(d);
  	struct its_collection *col;
  	void __iomem *rdbase;
+	u64 val;
+
+	if (map) {
+		struct its_device *its_dev = irq_data_get_irq_chip_data(d);
+
+		WARN_ON(!is_v4_1(its_dev->its));

drivers/irqchip/irq-gic-v3-its.c static void wait_for_syncr(void __iomem *rdbase) · line 1221

Quoted 10 lines
drivers/irqchip/irq-gic-v3-its.c +7 -0
+
+		val  = GICR_INVLPIR_V;
+		val |= FIELD_PREP(GICR_INVLPIR_VPEID, map->vpe->vpe_id);
+		val |= FIELD_PREP(GICR_INVLPIR_INTID, map->vintid);
+	} else {
+		val = d->hwirq;
+	}
 
  	/* Target the redistributor this LPI is currently routed to */
  	col = irq_to_col(d);

I think irq_to_col() may not work when GICv4.1 VLPIs are involved in.

irq_to_col() gives us col_map[event] as the target redistributor, but the correct one for VLPIs should be vlpi_maps[event]->vpe->col_idx. These two are not always pointing to the same physical RD. For example, if guest issues a MOVI against a VLPI, we will update the corresponding vlpi_map->vpe and issue a VMOVI on ITS... but leave the col_map[event] unchanged.

col_map[event] usually describes the physical LPI's CPU affinity, but when this physical LPI serves as something which the VLPI is backed by, we take really little care of it. Did I miss something here?

Thanks,
zenghui

Quoted 29 lines
drivers/irqchip/irq-gic-v3-its.c +3 -2
  	rdbase = per_cpu_ptr(gic_rdists->rdist, col->col_id)->rd_base;
-	gic_write_lpir(d->hwirq, rdbase + GICR_INVLPIR);
+	gic_write_lpir(val, rdbase + GICR_INVLPIR);
 
  	wait_for_syncr(rdbase);
  }
@@ -1216,7 +1230,8 @@ static void lpi_update_config(struct irq_data *d, u8 clr, u8 set)
  	struct its_device *its_dev = irq_data_get_irq_chip_data(d);
 
  	lpi_write_config(d, clr, set);
-	if (gic_rdists->has_direct_lpi && !irqd_is_forwarded_to_vcpu(d))
+	if (gic_rdists->has_direct_lpi &&
+	    (is_v4_1(its_dev->its) || !irqd_is_forwarded_to_vcpu(d)))
  		direct_lpi_inv(d);
  	else
  		its_send_inv(its_dev, its_get_event_id(d));
include/linux/irqchip/arm-gic-v3.h +1 -0
index b69f60792554..5f3278cbf247 100644
@@ -247,6 +247,7 @@
  #define GICR_TYPER_COMMON_LPI_AFF	GENMASK_ULL(25, 24)
  #define GICR_TYPER_AFFINITY		GENMASK_ULL(63, 32)
 
+#define GICR_INVLPIR_INTID		GENMASK_ULL(31, 0)
  #define GICR_INVLPIR_VPEID		GENMASK_ULL(47, 32)
  #define GICR_INVLPIR_V			GENMASK_ULL(63, 63)
 
reviewer
#47 lore raw

Re: [PATCH 24/35] irqchip/gic-v4.1: Add initial SGI configuration

In reply to Marc Zyngier

Hi Marc,

Quoted 15 lines · Marc Zyngier

The GICv4.1 ITS has yet another new command (VSGI) which allows a VPE-targeted SGI to be configured (or have its pending state cleared). Add support for this command and plumb it into the activate irqdomain callback so that it is ready to be used.

---
  drivers/irqchip/irq-gic-v3-its.c   | 88 ++++++++++++++++++++++++++++++
  include/linux/irqchip/arm-gic-v3.h |  3 +-
  2 files changed, 90 insertions(+), 1 deletion(-)
drivers/irqchip/irq-gic-v3-its.c +0 -0
index 69c26be709be..5234b9eef8ad 100644

[...]

Quoted 24 lines
drivers/irqchip/irq-gic-v3-its.c +20 -0
@@ -3574,6 +3628,38 @@ static struct irq_chip its_vpe_4_1_irq_chip = {
  	.irq_set_vcpu_affinity	= its_vpe_4_1_set_vcpu_affinity,
  };
 
+static struct its_node *find_4_1_its(void)
+{
+	static struct its_node *its = NULL;
+
+	if (!its) {
+		list_for_each_entry(its, &its_nodes, entry) {
+			if (is_v4_1(its))
+				return its;
+		}
+
+		/* Oops? */
+		its = NULL;
+	}
+
+	return its;
+}
+
+static void its_configure_sgi(struct irq_data *d, bool clear)
+{
+	struct its_vpe *vpe = irq_data_get_irq_chip_data(d);

drivers/irqchip/irq-gic-v3-its.c static struct irq_chip its_vpe_4_1_irq_chip = { · line 3660

Quoted 10 lines
drivers/irqchip/irq-gic-v3-its.c +10 -0
+	struct its_cmd_desc desc;
+
+	desc.its_vsgi_cmd.vpe = vpe;
+	desc.its_vsgi_cmd.sgi = d->hwirq;
+	desc.its_vsgi_cmd.priority = vpe->sgi_config[d->hwirq].priority;
+	desc.its_vsgi_cmd.enable = vpe->sgi_config[d->hwirq].enabled;
+	desc.its_vsgi_cmd.group = vpe->sgi_config[d->hwirq].group;
+	desc.its_vsgi_cmd.clear = clear;
+
+	its_send_single_vcommand(find_4_1_its(), its_build_vsgi_cmd, &desc);

I can't follow the logic in find_4_1_its(). We simply use the first ITS with GICv4.1 support, but what if the vPE is not mapped on this ITS? We will fail the valid_vpe() check when building this command and will have no effect on HW in the end?

Thanks,
zenghui

Quoted 20 lines
drivers/irqchip/irq-gic-v3-its.c +4 -0
+}
+
  static int its_sgi_set_affinity(struct irq_data *d,
  				const struct cpumask *mask_val,
  				bool force)
@@ -3619,6 +3705,8 @@ static void its_sgi_irq_domain_free(struct irq_domain *domain,
  static int its_sgi_irq_domain_activate(struct irq_domain *domain,
  				       struct irq_data *d, bool reserve)
  {
+	/* Write out the initial SGI configuration */
+	its_configure_sgi(d, false);
  	return 0;
  }
 
include/linux/irqchip/arm-gic-v3.h +0 -0
index 5f3278cbf247..c73176d3ab2b 100644

include/linux/irqchip/arm-gic-v3.h line 505

Quoted 10 lines
include/linux/irqchip/arm-gic-v3.h +2 -1
@@ -497,8 +497,9 @@
  #define GITS_CMD_VMAPTI			GITS_CMD_GICv4(GITS_CMD_MAPTI)
  #define GITS_CMD_VMOVI			GITS_CMD_GICv4(GITS_CMD_MOVI)
  #define GITS_CMD_VSYNC			GITS_CMD_GICv4(GITS_CMD_SYNC)
-/* VMOVP and INVDB are the odd ones, as they dont have a physical counterpart */
+/* VMOVP, VSGI and INVDB are the odd ones, as they dont have a physical counterpart */
  #define GITS_CMD_VMOVP			GITS_CMD_GICv4(2)
+#define GITS_CMD_VSGI			GITS_CMD_GICv4(3)
  #define GITS_CMD_INVDB			GITS_CMD_GICv4(0xe)
 
  /*

_______________________________________________
kvmarm mailing list
[email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm

reviewer
#48 lore raw

Re: [PATCH 24/35] irqchip/gic-v4.1: Add initial SGI configuration

Quoted 51 lines · Zenghui Yu

Hi Marc,

Quoted 15 lines · Marc Zyngier

The GICv4.1 ITS has yet another new command (VSGI) which allows a VPE-targeted SGI to be configured (or have its pending state cleared). Add support for this command and plumb it into the activate irqdomain callback so that it is ready to be used.

---
  drivers/irqchip/irq-gic-v3-its.c   | 88 ++++++++++++++++++++++++++++++ include/linux/irqchip/arm-gic-v3.h |  3 +-

  2 files changed, 90 insertions(+), 1 deletion(-)
drivers/irqchip/irq-gic-v3-its.c +0 -0
index 69c26be709be..5234b9eef8ad 100644

[...]

Quoted 30 lines
drivers/irqchip/irq-gic-v3-its.c +27 -0
@@ -3574,6 +3628,38 @@ static struct irq_chip its_vpe_4_1_irq_chip = {
       .irq_set_vcpu_affinity    = its_vpe_4_1_set_vcpu_affinity,
   };
+static struct its_node *find_4_1_its(void)
+{
+    static struct its_node *its = NULL;
+
+    if (!its) {
+        list_for_each_entry(its, &its_nodes, entry) {
+            if (is_v4_1(its))
+                return its;
+        }
+
+        /* Oops? */
+        its = NULL;
+    }
+
+    return its;
+}
+
+static void its_configure_sgi(struct irq_data *d, bool clear)
+{
+    struct its_vpe *vpe = irq_data_get_irq_chip_data(d);
+    struct its_cmd_desc desc;
+
+    desc.its_vsgi_cmd.vpe = vpe;
+    desc.its_vsgi_cmd.sgi = d->hwirq;
+    desc.its_vsgi_cmd.priority = vpe->sgi_config[d->hwirq].priority;
+    desc.its_vsgi_cmd.enable = vpe->sgi_config[d->hwirq].enabled;
+    desc.its_vsgi_cmd.group = vpe->sgi_config[d->hwirq].group;
Quoted 8 lines

drivers/irqchip/irq-gic-v3-its.c static struct irq_chip its_vpe_4_1_irq_chip = { · line 3660

Quoted 3 lines
drivers/irqchip/irq-gic-v3-its.c +3 -0
+    desc.its_vsgi_cmd.clear = clear;
+
+    its_send_single_vcommand(find_4_1_its(), its_build_vsgi_cmd, &desc);

I can't follow the logic in find_4_1_its().  We simply use the first ITS with GICv4.1 support, but what if the vPE is not mapped on this ITS? We will fail the valid_vpe() check when building this command and will have no effect on HW in the end?

I guess I find the answer in patch#31 ("Eagerly vmap vPEs").

I missed the important point in GICv4.1 that we have to map vPEs at all times for VSGI delivery, and we currently choose to map vPEs on all ITSs when requesting per vPE irq (instead of mapping them on demand, before mapping VLPI, which could happen at a pretty late stage).
So it's OK to use the first GICv4.1 ITS when configuring VSGI for this specified vPE, given that it is already mapped on all ITSs.

Thanks,
zenghui

reviewer
#56 lore raw

Re: [PATCH 31/35] irqchip/gic-v4.1: Eagerly vmap vPEs

In reply to Marc Zyngier
Quoted 18 lines · Marc Zyngier

Now that we have HW-accelerated SGIs being delivered to VPEs, it becomes required to map the VPEs on all ITSs instead of relying on the lazy approach that we would use when using the ITS-list mechanism.

---
  drivers/irqchip/irq-gic-v3-its.c | 39 +++++++++++++++++++++++++-------
  1 file changed, 31 insertions(+), 8 deletions(-)
drivers/irqchip/irq-gic-v3-its.c +0 -0
index 4aae9582182b..a1e8c4c2598a 100644
@@ -1417,12 +1417,31 @@ static int its_irq_set_irqchip_state(struct irq_data *d,
  	return 0;
  }
 

drivers/irqchip/irq-gic-v3-its.c static int its_irq_set_irqchip_state(struct irq_data *d, · line 1429

Quoted 10 lines
drivers/irqchip/irq-gic-v3-its.c +10 -0
+/*
+ * Two favourable cases:
+ *
+ * (a) Either we have a GICv4.1, and all vPEs have to be mapped at all times
+ *     for vSGI delivery
+ *
+ * (b) Or the ITSs do not use a list map, meaning that VMOVP is cheap enough
+ *     and we're better off mapping all VPEs always
+ *
+ * If neither (a) nor (b) is true, then we map VLPIs on demand.
                                                 ^^^^^

vPEs

Quoted 49 lines
drivers/irqchip/irq-gic-v3-its.c +18 -5
+ *
+ */
+static bool gic_requires_eager_mapping(void)
+{
+	if (!its_list_map || gic_rdists->has_rvpeid)
+		return true;
+
+	return false;
+}
+
  static void its_map_vm(struct its_node *its, struct its_vm *vm)
  {
  	unsigned long flags;
 
-	/* Not using the ITS list? Everything is always mapped. */
-	if (!its_list_map)
+	if (gic_requires_eager_mapping())
  		return;
 
  	raw_spin_lock_irqsave(&vmovp_lock, flags);
@@ -1456,7 +1475,7 @@ static void its_unmap_vm(struct its_node *its, struct its_vm *vm)
  	unsigned long flags;
 
  	/* Not using the ITS list? Everything is always mapped. */
-	if (!its_list_map)
+	if (gic_requires_eager_mapping())
  		return;
 
  	raw_spin_lock_irqsave(&vmovp_lock, flags);
@@ -3957,8 +3976,12 @@ static int its_vpe_irq_domain_activate(struct irq_domain *domain,
  	struct its_vpe *vpe = irq_data_get_irq_chip_data(d);
  	struct its_node *its;
 
-	/* If we use the list map, we issue VMAPP on demand... */
-	if (its_list_map)
+	/*
+	 * If we use the list map, we issue VMAPP on demand... Unless
+	 * we're on a GICv4.1 and we eagerly map the VPE on all ITSs
+	 * so that VSGIs can work.
+	 */
+	if (!gic_requires_eager_mapping())
  		return 0;
 
  	/* Map the VPE to the first possible CPU */
@@ -3984,10 +4007,10 @@ static void its_vpe_irq_domain_deactivate(struct irq_domain *domain,
  	struct its_node *its;
 
  	/*

drivers/irqchip/irq-gic-v3-its.c static void its_vpe_irq_domain_deactivate(struct irq_domain *domain, · line 4016

Quoted 10 lines
drivers/irqchip/irq-gic-v3-its.c +3 -3
-	 * If we use the list map, we unmap the VPE once no VLPIs are
-	 * associated with the VM.
+	 * If we use the list map on GICv4.0, we unmap the VPE once no
+	 * VLPIs are associated with the VM.
  	 */
-	if (its_list_map)
+	if (!gic_requires_eager_mapping())
  		return;
 
  	list_for_each_entry(its, &its_nodes, entry) {

Thanks,
zenghui

_______________________________________________
kvmarm mailing list
[email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm

author maintainer
#41 lore raw

Re: [PATCH 20/35] irqchip/gic-v4.1: Allow direct invalidation of VLPIs

In reply to Zenghui Yu
Quoted 45 lines · Zenghui Yu
Quoted 24 lines · Marc Zyngier

Just like for INVALL, GICv4.1 has grown a VPE-aware INVLPI register. Let's plumb it in and make use of the DirectLPI code in that case.

---
  drivers/irqchip/irq-gic-v3-its.c   | 19 +++++++++++++++++--
  include/linux/irqchip/arm-gic-v3.h |  1 +
  2 files changed, 18 insertions(+), 2 deletions(-)
drivers/irqchip/irq-gic-v3-its.c +7 -0
index b791c9beddf2..34595a7fcccb 100644
@@ -1200,13 +1200,27 @@ static void wait_for_syncr(void __iomem 
 *rdbase)
  static void direct_lpi_inv(struct irq_data *d)
  {
+	struct its_vlpi_map *map = get_vlpi_map(d);
  	struct its_collection *col;
  	void __iomem *rdbase;
+	u64 val;
+
+	if (map) {
+		struct its_device *its_dev = irq_data_get_irq_chip_data(d);
+
+		WARN_ON(!is_v4_1(its_dev->its));

drivers/irqchip/irq-gic-v3-its.c static void wait_for_syncr(void __iomem · line 1221

Quoted 10 lines
drivers/irqchip/irq-gic-v3-its.c +7 -0
+
+		val  = GICR_INVLPIR_V;
+		val |= FIELD_PREP(GICR_INVLPIR_VPEID, map->vpe->vpe_id);
+		val |= FIELD_PREP(GICR_INVLPIR_INTID, map->vintid);
+	} else {
+		val = d->hwirq;
+	}
 
  	/* Target the redistributor this LPI is currently routed to */
  	col = irq_to_col(d);

I think irq_to_col() may not work when GICv4.1 VLPIs are involved in.

irq_to_col() gives us col_map[event] as the target redistributor, but the correct one for VLPIs should be vlpi_maps[event]->vpe->col_idx. These two are not always pointing to the same physical RD. For example, if guest issues a MOVI against a VLPI, we will update the corresponding vlpi_map->vpe and issue a VMOVI on ITS... but leave the col_map[event] unchanged.

Quoted 3 lines

col_map[event] usually describes the physical LPI's CPU affinity, but when this physical LPI serves as something which the VLPI is backed by, we take really little care of it. Did I miss something here?

You didn't miss anything, and this is indeed another pretty bad bug. The collection mapping is completely unused when the LPI becomes a VLPI, and it is only the vpe->col_id that matters (which gets updated on VMOVP).

This shows that irq_to_col() is the wrong abstraction, and what we're interested is something that is more like 'irq to cpuid', allowing us to directly point to the right distributor.

Please see the patch I just pushed[1], which does that.

Thanks,

        M.

[1] https://git.kernel.org/pub/scm/linux/kernel/git/maz/arm-platforms.git/commit/?h=irq/gic-v4.1-devel&id=aff363113eb26b6840136b69c2c7db2ea691db20

Signature · 1 line
Jazz is not dead. It just smells funny...
author maintainer
#57 lore raw

Re: [PATCH 31/35] irqchip/gic-v4.1: Eagerly vmap vPEs

In reply to Zenghui Yu
Quoted 31 lines · Zenghui Yu
Quoted 21 lines · Marc Zyngier

Now that we have HW-accelerated SGIs being delivered to VPEs, it becomes required to map the VPEs on all ITSs instead of relying on the lazy approach that we would use when using the ITS-list mechanism.

---
  drivers/irqchip/irq-gic-v3-its.c | 39 
+++++++++++++++++++++++++-------
  1 file changed, 31 insertions(+), 8 deletions(-)
drivers/irqchip/irq-gic-v3-its.c +3 -0
index 4aae9582182b..a1e8c4c2598a 100644
@@ -1417,12 +1417,31 @@ static int its_irq_set_irqchip_state(struct 
 irq_data *d,
  	return 0;
  }
  +/*
+ * Two favourable cases:
+ *
+ * (a) Either we have a GICv4.1, and all vPEs have to be mapped at 
Quoted 8 lines

all times
+ * for vSGI delivery
+ *
+ * (b) Or the ITSs do not use a list map, meaning that VMOVP is cheap enough
+ * and we're better off mapping all VPEs always
+ *
+ * If neither (a) nor (b) is true, then we map VLPIs on demand.

                                                 ^^^^^
Quoted 1 line

vPEs

Yes, well caught.

Thanks,

        M.
Signature · 5 lines
Jazz is not dead. It just smells funny...
_______________________________________________
kvmarm mailing list
[email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
reviewer
#42 lore raw

Re: [PATCH 20/35] irqchip/gic-v4.1: Allow direct invalidation of VLPIs

In reply to Marc Zyngier
Quoted 58 lines · Marc Zyngier
Quoted 44 lines · Zenghui Yu
Quoted 23 lines · Marc Zyngier

Just like for INVALL, GICv4.1 has grown a VPE-aware INVLPI register. Let's plumb it in and make use of the DirectLPI code in that case.

---
  drivers/irqchip/irq-gic-v3-its.c   | 19 +++++++++++++++++-- include/linux/irqchip/arm-gic-v3.h |  1 +

  2 files changed, 18 insertions(+), 2 deletions(-)
drivers/irqchip/irq-gic-v3-its.c +7 -0
index b791c9beddf2..34595a7fcccb 100644
@@ -1200,13 +1200,27 @@ static void wait_for_syncr(void __iomem *rdbase)
   static void direct_lpi_inv(struct irq_data *d)
   {
+    struct its_vlpi_map *map = get_vlpi_map(d);
       struct its_collection *col;
       void __iomem *rdbase;
+    u64 val;
+
+    if (map) {
+        struct its_device *its_dev = irq_data_get_irq_chip_data(d);
+
+        WARN_ON(!is_v4_1(its_dev->its));

drivers/irqchip/irq-gic-v3-its.c static void wait_for_syncr(void __iomem *rdbase) · line 1220

Quoted 10 lines
drivers/irqchip/irq-gic-v3-its.c +7 -0
+
+        val  = GICR_INVLPIR_V;
+        val |= FIELD_PREP(GICR_INVLPIR_VPEID, map->vpe->vpe_id);
+        val |= FIELD_PREP(GICR_INVLPIR_INTID, map->vintid);
+    } else {
+        val = d->hwirq;
+    }
 
       /* Target the redistributor this LPI is currently routed to */
       col = irq_to_col(d);

I think irq_to_col() may not work when GICv4.1 VLPIs are involved in.

irq_to_col() gives us col_map[event] as the target redistributor, but the correct one for VLPIs should be vlpi_maps[event]->vpe->col_idx. These two are not always pointing to the same physical RD. For example, if guest issues a MOVI against a VLPI, we will update the corresponding vlpi_map->vpe and issue a VMOVI on ITS... but leave the col_map[event] unchanged.

Quoted 3 lines

col_map[event] usually describes the physical LPI's CPU affinity, but when this physical LPI serves as something which the VLPI is backed by, we take really little care of it.  Did I miss something here?

You didn't miss anything, and this is indeed another pretty bad bug. The collection mapping is completely unused when the LPI becomes a VLPI, and it is only the vpe->col_id that matters (which gets updated on VMOVP).

This shows that irq_to_col() is the wrong abstraction, and what we're interested is something that is more like 'irq to cpuid', allowing us to directly point to the right distributor.

Quoted 7 lines

Please see the patch I just pushed[1], which does that.

Thanks,

        M.

[1] https://git.kernel.org/pub/scm/linux/kernel/git/maz/arm-platforms.git/commit/?h=irq/gic-v4.1-devel&id=aff363113eb26b6840136b69c2c7db2ea691db20

Yes, I think this makes sense.

Thanks,
zenghui