Skip to contentSkip to conversation

irqchip/gic: Generalize use of HW-based retriggering

PATCH In review Marc Zyngier 2 participants 8 messages Sep 3, 2020 – Sep 5, 2020

Subsystems (3)
lore
Share & tools
lore

Marc Zyngier

author maintainer
#0 lore raw

v2 cover irqchip/gic: Generalize use of HW-based retriggering

Valentin recently pointed out that that relying on SW-based retrigger with any of the GIC interrupt controllers is both inefficient and slightly broken, as it messes the GIC's own state machine.

For this to work with the hierarchical irqchip model that the GIC uses, we need check_irq_resend grow an understanding of hierarchies, something it has been lacking for some time.

The 3 other patches simply implement the HW resend callbacks, and finally prevent any SW resend for the GICs.

Unless anyone objects, I plan to take this into 5.10.

* From v1:
- Dropped most of the patches in favour of a teaching the core kernel to use irq_chip_retriger_hierarchy()

Marc Zyngier (2):
genirq: Walk the irq_data hierarchy when resending an interrupt irqchip/git-v3-its: Implement irq_retrigger callback for
device-triggered LPIs

Valentin Schneider (2):
irqchip/gic-v2, v3: Implement irq_chip->irq_retrigger() irqchip/gic-v2, v3: Prevent SW resends entirely

 drivers/irqchip/irq-gic-v3-its.c |  6 ++++++
 drivers/irqchip/irq-gic-v3.c     | 12 +++++++++++-
 drivers/irqchip/irq-gic.c        | 12 +++++++++++-
 kernel/irq/resend.c              | 15 +++++++++++++--
 4 files changed, 41 insertions(+), 4 deletions(-)
Signature · 1 line
2.28.0
author maintainer
#1 lore raw

PATCH v2 1/4 genirq: Walk the irq_data hierarchy when resending an interrupt

On resending an interrupt, we only check the topmost irqchip for a irq_retrigger callback. However, this callback could be implemented at a lower level. Use irq_chip_retrigger_hierarchy() in this case.

---
 kernel/irq/resend.c | 15 +++++++++++++--
 1 file changed, 13 insertions(+), 2 deletions(-)
kernel/irq/resend.c +13 -2
index c48ce19a257f..8ccd32a0cc80 100644
@@ -86,6 +86,18 @@ static int irq_sw_resend(struct irq_desc *desc)
 }
 #endif
 
+static int try_retrigger(struct irq_desc *desc)
+{
+	if (desc->irq_data.chip->irq_retrigger)
+		return desc->irq_data.chip->irq_retrigger(&desc->irq_data);
+
+#ifdef CONFIG_IRQ_DOMAIN_HIERARCHY
+	return irq_chip_retrigger_hierarchy(&desc->irq_data);
+#else
+	return 0;
+#endif
+}
+
 /*
  * IRQ resend
  *
@@ -113,8 +125,7 @@ int check_irq_resend(struct irq_desc *desc, bool inject)
 
 	desc->istate &= ~IRQS_PENDING;
 
-	if (!desc->irq_data.chip->irq_retrigger ||
-	    !desc->irq_data.chip->irq_retrigger(&desc->irq_data))
+	if (!try_retrigger(desc))
 		err = irq_sw_resend(desc);
 
 	/* If the retrigger was successfull, mark it with the REPLAY bit */
Signature · 1 line
2.28.0
#2 lore raw

Re: [PATCH v2 1/4] genirq: Walk the irq_data hierarchy when resending an interrupt

Hi Marc,

Quoted 4 lines · Marc Zyngier

On resending an interrupt, we only check the topmost irqchip for a irq_retrigger callback. However, this callback could be implemented at a lower level. Use irq_chip_retrigger_hierarchy() in this case.

Rookie wording question here; re-reading this I'm questioning which way is up.

From an irq_data hierarchy PoV, the topmost chip (i.e. last ->parent) should be the root irqchip. However, the irq_desc we get from irq_to_desc() ought to hold the irq_data for the lowermost irqchip in that irq_data hierarchy.

Is it that here by "topmost" you instead mean topmost of the irqchip stack on top of the root (IOW furthest away from the root)?

Quoted 38 lines
---
 kernel/irq/resend.c | 15 +++++++++++++--
 1 file changed, 13 insertions(+), 2 deletions(-)
kernel/irq/resend.c +13 -2
index c48ce19a257f..8ccd32a0cc80 100644
@@ -86,6 +86,18 @@ static int irq_sw_resend(struct irq_desc *desc)
 }
 #endif
 
+static int try_retrigger(struct irq_desc *desc)
+{
+	if (desc->irq_data.chip->irq_retrigger)
+		return desc->irq_data.chip->irq_retrigger(&desc->irq_data);
+
+#ifdef CONFIG_IRQ_DOMAIN_HIERARCHY
+	return irq_chip_retrigger_hierarchy(&desc->irq_data);
+#else
+	return 0;
+#endif
+}
+
 /*
  * IRQ resend
  *
@@ -113,8 +125,7 @@ int check_irq_resend(struct irq_desc *desc, bool inject)
 
      desc->istate &= ~IRQS_PENDING;
 
-	if (!desc->irq_data.chip->irq_retrigger ||
-	    !desc->irq_data.chip->irq_retrigger(&desc->irq_data))
+	if (!try_retrigger(desc))
              err = irq_sw_resend(desc);
 
      /* If the retrigger was successfull, mark it with the REPLAY bit */
author maintainer
#3 lore raw

Re: [PATCH v2 1/4] genirq: Walk the irq_data hierarchy when resending an interrupt

Hi Valentin,

Quoted 13 lines · Valentin Schneider

Hi Marc,

Quoted 4 lines · Marc Zyngier

On resending an interrupt, we only check the topmost irqchip for a irq_retrigger callback. However, this callback could be implemented at a lower level. Use irq_chip_retrigger_hierarchy() in this case.

Rookie wording question here; re-reading this I'm questioning which way is up.

Quoted 7 lines

From an irq_data hierarchy PoV, the topmost chip (i.e. last ->parent) should be the root irqchip. However, the irq_desc we get from irq_to_desc() ought to hold the irq_data for the lowermost irqchip in that irq_data hierarchy.

Is it that here by "topmost" you instead mean topmost of the irqchip stack on top of the root (IOW furthest away from the root)?

That's indeed what I mean, but I agree that the terminology is confusing, and often used inconsistently (by me included).

<random>
Maybe considering the irqchip stack along a vertical axis is the wrong thing to do, and that looking at it as a volume would be marginally better?

How about innermost (close to the CPU) vs outermost (close to the device)?
</random>

Thanks,

	M.
Signature · 1 line
Without deviation from the norm, progress is not possible.
Reviewed-by
#4 lore raw

Re: [PATCH v2 1/4] genirq: Walk the irq_data hierarchy when resending an interrupt

Quoted 9 lines · Marc Zyngier

<random>
Maybe considering the irqchip stack along a vertical axis is the wrong thing to do, and that looking at it as a volume would be marginally better?

How about innermost (close to the CPU) vs outermost (close to the device)?
</random>

I guess this is fairly subjective, but the inner/outer thing does click with me.

I think the "issue" with the top / bottom wording is that existing data structures (domain / irq_data hierarchy) bias my interpretation of it, but it's upside down from the irq_chip stack representation. That's not the case of inner / outer where all I can think of is the actual chip layout (i.e. as an image of the distance from the CPUs).

Anyway, that's enough psychoanalysis from me, the patches look fine - I also reran my quick rtcwake test on GICv2 (tests the WAKEUP_ARMED path). Feel free to add

to 1/4 & 3/4.

Quoted 3 lines

Thanks,

      M.
author maintainer
#5 lore raw

PATCH v2 2/4 irqchip/gic-v2, v3: Implement irq_chip->irq_retrigger()

In reply to Marc Zyngier

From: Valentin Schneider <[email protected]>

While digging around IRQCHIP_EOI_IF_HANDLED and irq/resend.c, it has come to my attention that the IRQ resend situation seems a bit precarious for the GIC(s).

When marking an IRQ with IRQS_PENDING, handle_fasteoi_irq() will bail out and issue an irq_eoi(). Should the IRQ in question be re-enabled, check_irq_resend() will trigger a SW resend, which will go through the flow handler again and issue *another* irq_eoi() on the *same* IRQ activation. This is something the GIC spec clearly describes as a bad idea:
any EOI must match a previous ACK.

Implement irq_chip.irq_retrigger() for the GIC chips by setting the GIC pending bit of the relevant IRQ. After being called by check_irq_resend(), this will eventually trigger a *new* interrupt which we will handle as usual.

---
 drivers/irqchip/irq-gic-v3.c | 7 +++++++
 drivers/irqchip/irq-gic.c    | 6 ++++++
 2 files changed, 13 insertions(+)
drivers/irqchip/irq-gic-v3.c +7 -0
index 324f280ff606..b507bc7c5cda 100644
@@ -1207,6 +1207,11 @@ static int gic_set_affinity(struct irq_data *d, const struct cpumask *mask_val,
 #define gic_smp_init()		do { } while(0)
 #endif
 
+static int gic_retrigger(struct irq_data *data)
+{
+	return !gic_irq_set_irqchip_state(data, IRQCHIP_STATE_PENDING, true);
+}
+
 #ifdef CONFIG_CPU_PM
 static int gic_cpu_pm_notifier(struct notifier_block *self,
 			       unsigned long cmd, void *v)
@@ -1242,6 +1247,7 @@ static struct irq_chip gic_chip = {
 	.irq_eoi		= gic_eoi_irq,
 	.irq_set_type		= gic_set_type,
 	.irq_set_affinity	= gic_set_affinity,
+	.irq_retrigger          = gic_retrigger,
 	.irq_get_irqchip_state	= gic_irq_get_irqchip_state,
 	.irq_set_irqchip_state	= gic_irq_set_irqchip_state,
 	.irq_nmi_setup		= gic_irq_nmi_setup,
@@ -1258,6 +1264,7 @@ static struct irq_chip gic_eoimode1_chip = {
 	.irq_eoi		= gic_eoimode1_eoi_irq,
 	.irq_set_type		= gic_set_type,
 	.irq_set_affinity	= gic_set_affinity,
+	.irq_retrigger          = gic_retrigger,
 	.irq_get_irqchip_state	= gic_irq_get_irqchip_state,
 	.irq_set_irqchip_state	= gic_irq_set_irqchip_state,
 	.irq_set_vcpu_affinity	= gic_irq_set_vcpu_affinity,
drivers/irqchip/irq-gic.c +6 -0
index a27ba2cc1dce..e92ee2b6d7a5 100644
@@ -347,6 +347,11 @@ static int gic_set_affinity(struct irq_data *d, const struct cpumask *mask_val,
 }
 #endif
 
+static int gic_retrigger(struct irq_data *data)
+{
+	return !gic_irq_set_irqchip_state(data, IRQCHIP_STATE_PENDING, true);
+}
+
 static void __exception_irq_entry gic_handle_irq(struct pt_regs *regs)
 {
 	u32 irqstat, irqnr;
@@ -417,6 +422,7 @@ static const struct irq_chip gic_chip = {
 	.irq_unmask		= gic_unmask_irq,
 	.irq_eoi		= gic_eoi_irq,
 	.irq_set_type		= gic_set_type,
+	.irq_retrigger          = gic_retrigger,
 	.irq_get_irqchip_state	= gic_irq_get_irqchip_state,
 	.irq_set_irqchip_state	= gic_irq_set_irqchip_state,
 	.flags			= IRQCHIP_SET_TYPE_MASKED |
Signature · 1 line
2.28.0
author maintainer
#6 lore raw

PATCH v2 3/4 irqchip/git-v3-its: Implement irq_retrigger callback for device-triggered LPIs

In reply to Marc Zyngier

It is pretty easy to provide a retrigger callback for the ITS, as it we already have the required support in terms of irq_set_irqchip_state().

Note that this only works for device-generated LPIs, and not the GICv4 doorbells, which should never have to be retriggered anyway.

---
 drivers/irqchip/irq-gic-v3-its.c | 6 ++++++
 1 file changed, 6 insertions(+)
drivers/irqchip/irq-gic-v3-its.c +6 -0
index 95f097448f97..2808545a963e 100644
@@ -1720,6 +1720,11 @@ static int its_irq_set_irqchip_state(struct irq_data *d,
 	return 0;
 }
 
+static int its_irq_retrigger(struct irq_data *d)
+{
+	return !its_irq_set_irqchip_state(d, IRQCHIP_STATE_PENDING, true);
+}
+
 /*
  * Two favourable cases:
  *
@@ -1971,6 +1976,7 @@ static struct irq_chip its_irq_chip = {
 	.irq_set_affinity	= its_set_affinity,
 	.irq_compose_msi_msg	= its_irq_compose_msi_msg,
 	.irq_set_irqchip_state	= its_irq_set_irqchip_state,
+	.irq_retrigger		= its_irq_retrigger,
 	.irq_set_vcpu_affinity	= its_irq_set_vcpu_affinity,
 };
 
Signature · 1 line
2.28.0
author maintainer
#7 lore raw

PATCH v2 4/4 irqchip/gic-v2, v3: Prevent SW resends entirely

In reply to Marc Zyngier

From: Valentin Schneider <[email protected]>

The GIC irqchips can now use a HW resend when a retrigger is invoked by check_irq_resend(). However, should the HW resend fail, check_irq_resend() will still attempt to trigger a SW resend, which is still a bad idea for the GICs.

Prevent this from happening by setting IRQD_HANDLE_ENFORCE_IRQCTX on all GIC IRQs. Technically per-cpu IRQs do not need this, as their flow handlers never set IRQS_PENDING, but this aligns all IRQs wrt context enforcement:
this also forces all GIC IRQ handling to happen in IRQ context (as defined by in_irq()).

---
 drivers/irqchip/irq-gic-v3.c | 5 ++++-
 drivers/irqchip/irq-gic.c    | 6 +++++-
 2 files changed, 9 insertions(+), 2 deletions(-)
drivers/irqchip/irq-gic-v3.c +4 -1
index b507bc7c5cda..4e9387aafed8 100644
@@ -1279,6 +1279,7 @@ static int gic_irq_domain_map(struct irq_domain *d, unsigned int irq,
 			      irq_hw_number_t hw)
 {
 	struct irq_chip *chip = &gic_chip;
+	struct irq_data *irqd = irq_desc_get_irq_data(irq_to_desc(irq));
 
 	if (static_branch_likely(&supports_deactivate_key))
 		chip = &gic_eoimode1_chip;
@@ -1296,7 +1297,7 @@ static int gic_irq_domain_map(struct irq_domain *d, unsigned int irq,
 		irq_domain_set_info(d, irq, hw, chip, d->host_data,
 				    handle_fasteoi_irq, NULL, NULL);
 		irq_set_probe(irq);
-		irqd_set_single_target(irq_desc_get_irq_data(irq_to_desc(irq)));
+		irqd_set_single_target(irqd);
 		break;
 
 	case LPI_RANGE:
@@ -1310,6 +1311,8 @@ static int gic_irq_domain_map(struct irq_domain *d, unsigned int irq,
 		return -EPERM;
 	}
 
+	/* Prevents SW retriggers which mess up the ACK/EOI ordering */
+	irqd_set_handle_enforce_irqctx(irqd);
 	return 0;
 }
 
drivers/irqchip/irq-gic.c +5 -1
index e92ee2b6d7a5..b59bcef69bf3 100644
@@ -975,6 +975,7 @@ static int gic_irq_domain_map(struct irq_domain *d, unsigned int irq,
 				irq_hw_number_t hw)
 {
 	struct gic_chip_data *gic = d->host_data;
+	struct irq_data *irqd = irq_desc_get_irq_data(irq_to_desc(irq));
 
 	if (hw < 32) {
 		irq_set_percpu_devid(irq);
@@ -984,8 +985,11 @@ static int gic_irq_domain_map(struct irq_domain *d, unsigned int irq,
 		irq_domain_set_info(d, irq, hw, &gic->chip, d->host_data,
 				    handle_fasteoi_irq, NULL, NULL);
 		irq_set_probe(irq);
-		irqd_set_single_target(irq_desc_get_irq_data(irq_to_desc(irq)));
+		irqd_set_single_target(irqd);
 	}
+
+	/* Prevents SW retriggers which mess up the ACK/EOI ordering */
+	irqd_set_handle_enforce_irqctx(irqd);
 	return 0;
 }
 
Signature · 1 line
2.28.0