Skip to contentSkip to conversation

genirq: Expand doc for PENDING and REPLAY flags

PATCH Superseded James Gowans 3 participants 6 messages May 30, 2023 – Jun 5, 2023 IRQ

v1 of 3 Superseded Latest is v3 →

lore
Share & tools
lore
#0 lore raw

PATCH 1/2 genirq: Expand doc for PENDING and REPLAY flags

Adding a bit more info about what the flags are used for may help future code readers.

Cc Thomas Gleixner <[email protected]>
Cc Marc Zyngier <[email protected]>
Cc Liao Chang <[email protected]>
---
 kernel/irq/internals.h | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)
kernel/irq/internals.h +5 -2
index 5fdc0b557579..c443a0ddc07e 100644
@@ -47,9 +47,12 @@ enum {
  *				  detection
  * IRQS_POLL_INPROGRESS		- polling in progress
  * IRQS_ONESHOT			- irq is not unmasked in primary handler
- * IRQS_REPLAY			- irq is replayed
+ * IRQS_REPLAY			- irq has been resent and will not be resent
+ * 				  again until the handler has run and cleared
+ * 				  this flag.
  * IRQS_WAITING			- irq is waiting
- * IRQS_PENDING			- irq is pending and replayed later
+ * IRQS_PENDING			- irq needs to be resent and should be resent
+ * 				  at the next available opportunity.
  * IRQS_SUSPENDED		- irq is suspended
  * IRQS_NMI			- irq line is used to deliver NMIs
  * IRQS_SYSFS			- descriptor has been added to sysfs
Signature · 1 line
2.25.1
#1 lore raw

PATCH 2/2 genirq: fasteoi resends interrupt on concurrent invoke

Update the generic handle_fasteoi_irq to cater for the case when the next interrupt comes in while the previous handler is still running. Currently when that happens the irq_may_run() early out causes the next IRQ to be lost. Change the behaviour to mark the interrupt as pending and re-send the interrupt when handle_fasteoi_irq sees that the pending flag has been set. This is largely inspired by handle_edge_irq.

Generally it should not be possible for the next interrupt to arrive while the previous handler is still running: the next interrupt should only arrive after the EOI message has been sent and the previous handler has returned. However, there is a race where if the interrupt affinity is changed while the previous handler is running, then the next interrupt can arrive at a different CPU while the previous handler is still running. In that case there will be a concurrent invoke and the early out will be taken.

For example:

           CPU 0             |          CPU 1
-----------------------------|-----------------------------
interrupt start              |
  handle_fasteoi_irq         | set_affinity(CPU 1)
    handler                  |
    ...                      | interrupt start
    ...                      |   handle_fasteoi_irq -> early out
  handle_fasteoi_irq return  | interrupt end
interrupt end                |

This issue was observed specifically on an arm64 system with a GIC-v3 handling MSIs; GIC-v3 uses the handle_fasteoi_irq handler. The issue is that the global ITS is responsible for affinity but does not know whether interrupts are pending/running, only the CPU-local redistributor handles the EOI. Hence when the affinity is changed in the ITS, the new CPU's redistributor does not know that the original CPU is still running the handler.

Implementation notes:

It is believed that it's NOT necessary to mask the interrupt in handle_fasteoi_irq() the way that handle_edge_irq() does. This is because handle_edge_irq() caters for controllers which are too simple to gate interrupts from the same source, so the kernel explicitly masks the interrupt if it re-occurs [0].

[0] https://lore.kernel.org/all/[email protected]/

Cc Thomas Gleixner <[email protected]>
Cc Marc Zyngier <[email protected]>
Cc KarimAllah Raslan <[email protected]>
Cc Yipeng Zou <[email protected]>
Cc Zhang Jianhua <[email protected]>
---
 kernel/irq/chip.c | 15 ++++++++++++++-
 1 file changed, 14 insertions(+), 1 deletion(-)
kernel/irq/chip.c +14 -1
index 49e7bc871fec..42f33e77c16b 100644
@@ -692,8 +692,15 @@ void handle_fasteoi_irq(struct irq_desc *desc)
 
 	raw_spin_lock(&desc->lock);
 
-	if (!irq_may_run(desc))
+	/*
+	 * When an affinity change races with IRQ delivery, the next interrupt
+	 * can arrive on the new CPU before the original CPU has completed
+	 * handling the previous one. Mark it as pending and return EOI.
+	 */
+	if (!irq_may_run(desc)) {
+		desc->istate |= IRQS_PENDING;
 		goto out;
+	}
 
 	desc->istate &= ~(IRQS_REPLAY | IRQS_WAITING);
 
@@ -715,6 +722,12 @@ void handle_fasteoi_irq(struct irq_desc *desc)
 
 	cond_unmask_eoi_irq(desc, chip);
 
+	/*
+	 * When the race descibed above happens, this will resend the interrupt.
+	 */
+	if (unlikely(desc->istate & IRQS_PENDING))
+		check_irq_resend(desc, false);
+
 	raw_spin_unlock(&desc->lock);
 	return;
 out:
Signature · 1 line
2.25.1
#2 lore raw

Re: [PATCH 2/2] genirq: fasteoi resends interrupt on concurrent invoke

Hi--

Quoted 26 lines · James Gowans
---
 kernel/irq/chip.c | 15 ++++++++++++++-
 1 file changed, 14 insertions(+), 1 deletion(-)
kernel/irq/chip.c +8 -1
index 49e7bc871fec..42f33e77c16b 100644
@@ -692,8 +692,15 @@ void handle_fasteoi_irq(struct irq_desc *desc)
 
 	raw_spin_lock(&desc->lock);
 
-	if (!irq_may_run(desc))
+	/*
+	 * When an affinity change races with IRQ delivery, the next interrupt
+	 * can arrive on the new CPU before the original CPU has completed
+	 * handling the previous one. Mark it as pending and return EOI.
+	 */
+	if (!irq_may_run(desc)) {
+		desc->istate |= IRQS_PENDING;
 		goto out;
+	}
 
 	desc->istate &= ~(IRQS_REPLAY | IRQS_WAITING);
 

kernel/irq/chip.c void handle_fasteoi_irq(struct irq_desc *desc) · line 726

Quoted 5 lines
kernel/irq/chip.c +2 -0
@@ -715,6 +722,12 @@ void handle_fasteoi_irq(struct irq_desc *desc)
 
 	cond_unmask_eoi_irq(desc, chip);
 
+	/*
+	 * When the race descibed above happens, this will resend the interrupt.

described

Quoted 7 lines
kernel/irq/chip.c +4 -0
+	 */
+	if (unlikely(desc->istate & IRQS_PENDING))
+		check_irq_resend(desc, false);
+
 	raw_spin_unlock(&desc->lock);
 	return;
 out:
Signature · 1 line
~Randy
#3 lore raw

Re: [PATCH 2/2] genirq: fasteoi resends interrupt on concurrent invoke

In reply to James Gowans
Quoted 8 lines · James Gowans

Update the generic handle_fasteoi_irq to cater for the case when the next interrupt comes in while the previous handler is still running. Currently when that happens the irq_may_run() early out causes the next IRQ to be lost. Change the behaviour to mark the interrupt as pending and re-send the interrupt when handle_fasteoi_irq sees that the pending flag has been set. This is largely inspired by handle_edge_irq.

Quoted 4 lines

Generally it should not be possible for the next interrupt to arrive while the previous handler is still running: the next interrupt should only arrive after the EOI message has been sent and the previous handler has returned.

There is no such message with LPIs. I pointed that out previously.

Quoted 18 lines

However, there is a race where if the interrupt affinity is changed while the previous handler is running, then the next interrupt can arrive at a different CPU while the previous handler is still running. In that case there will be a concurrent invoke and the early out will be taken.

For example:

           CPU 0             |          CPU 1
-----------------------------|-----------------------------
interrupt start              |
  handle_fasteoi_irq         | set_affinity(CPU 1)
    handler                  |
    ...                      | interrupt start
    ...                      |   handle_fasteoi_irq -> early out
  handle_fasteoi_irq return  | interrupt end
interrupt end                |
Quoted 7 lines

This issue was observed specifically on an arm64 system with a GIC-v3 handling MSIs; GIC-v3 uses the handle_fasteoi_irq handler. The issue is that the global ITS is responsible for affinity but does not know whether interrupts are pending/running, only the CPU-local redistributor handles the EOI. Hence when the affinity is changed in the ITS, the new CPU's redistributor does not know that the original CPU is still running the handler.

Similar to your previous patch, you don't explain *why* the interrupt gets delivered when it is an LPI, and not for any of the other GICv3 interrupt types. That's an important point.

Quoted 46 lines

Implementation notes:

It is believed that it's NOT necessary to mask the interrupt in handle_fasteoi_irq() the way that handle_edge_irq() does. This is because handle_edge_irq() caters for controllers which are too simple to gate interrupts from the same source, so the kernel explicitly masks the interrupt if it re-occurs [0].

[0] https://lore.kernel.org/all/[email protected]/

Cc Thomas Gleixner <[email protected]>
Cc Marc Zyngier <[email protected]>
Cc KarimAllah Raslan <[email protected]>
Cc Yipeng Zou <[email protected]>
Cc Zhang Jianhua <[email protected]>
---
 kernel/irq/chip.c | 15 ++++++++++++++-
 1 file changed, 14 insertions(+), 1 deletion(-)
kernel/irq/chip.c +8 -1
index 49e7bc871fec..42f33e77c16b 100644
@@ -692,8 +692,15 @@ void handle_fasteoi_irq(struct irq_desc *desc)
 
 	raw_spin_lock(&desc->lock);
 
-	if (!irq_may_run(desc))
+	/*
+	 * When an affinity change races with IRQ delivery, the next interrupt
+	 * can arrive on the new CPU before the original CPU has completed
+	 * handling the previous one. Mark it as pending and return EOI.
+	 */
+	if (!irq_may_run(desc)) {
+		desc->istate |= IRQS_PENDING;
 		goto out;
+	}
 
 	desc->istate &= ~(IRQS_REPLAY | IRQS_WAITING);
 
@@ -715,6 +722,12 @@ void handle_fasteoi_irq(struct irq_desc *desc)
 
 	cond_unmask_eoi_irq(desc, chip);

kernel/irq/chip.c void handle_fasteoi_irq(struct irq_desc *desc) · line 733

Quoted 10 lines
kernel/irq/chip.c +6 -0
 
+	/*
+	 * When the race descibed above happens, this will resend the interrupt.
+	 */
+	if (unlikely(desc->istate & IRQS_PENDING))
+		check_irq_resend(desc, false);
+
 	raw_spin_unlock(&desc->lock);
 	return;
 out:

While I'm glad that you eventually decided to use the resend mechanism instead of spinning on the "old" CPU, I still think imposing this behaviour on all users without any discrimination is wrong.

Look at what it does if an interrupt is a wake-up source. You'd pointlessly requeue the interrupt (bonus points if the irqchip doesn't provide a HW-based retrigger mechanism).

I still maintain that this change should only be applied for the particular interrupts that *require* it, and not as a blanket change affecting everything under the sun. I have proposed such a change in the past, feel free to use it or roll your own.

In the meantime, I strongly oppose this change as proposed.

Thanks,

	M.
Signature · 1 line
Without deviation from the norm, progress is not possible.
#4 lore raw

Re: [PATCH 2/2] genirq: fasteoi resends interrupt on concurrent invoke

Quoted 6 lines · Marc Zyngier
Quoted 4 lines

Generally it should not be possible for the next interrupt to arrive while the previous handler is still running: the next interrupt should only arrive after the EOI message has been sent and the previous handler has returned.

There is no such message with LPIs. I pointed that out previously.

Arg, thanks, I'll re-word this to:

"Generally it should not be possible for the next interrupt to arrive while the previous handler is still running: the CPU will not preempt an interrupt with another from the same source or same priority."

I hope that's more accurate?

Quoted 11 lines
Quoted 7 lines

This issue was observed specifically on an arm64 system with a GIC-v3 handling MSIs; GIC-v3 uses the handle_fasteoi_irq handler. The issue is that the global ITS is responsible for affinity but does not know whether interrupts are pending/running, only the CPU-local redistributor handles the EOI. Hence when the affinity is changed in the ITS, the new CPU's redistributor does not know that the original CPU is still running the handler.

Similar to your previous patch, you don't explain *why* the interrupt gets delivered when it is an LPI, and not for any of the other GICv3 interrupt types. That's an important point.

Right, you pointed out the issue with this sentence too and I missed updating it. :-/ How about:

"This issue was observed specifically on an arm64 system with a GIC-v3 handling MSIs; GIC-v3 uses the handle_fasteoi_irq handler. The issue is that the GIC-v3's physical LPIs do not have a global active state. If LPIs had an active state, then it would not be be able to be retriggered until the first CPU had issued a deactivation"

Quoted 16 lines

kernel/irq/chip.c void handle_fasteoi_irq(struct irq_desc *desc) · line 733

Quoted 11 lines
kernel/irq/chip.c +6 -0
+     /*
+      * When the race descibed above happens, this will resend the interrupt.
+      */
+     if (unlikely(desc->istate & IRQS_PENDING))
+             check_irq_resend(desc, false);
+
      raw_spin_unlock(&desc->lock);
      return;
 out:

While I'm glad that you eventually decided to use the resend mechanism instead of spinning on the "old" CPU, I still think imposing this behaviour on all users without any discrimination is wrong.

Quoted 8 lines

Look at what it does if an interrupt is a wake-up source. You'd pointlessly requeue the interrupt (bonus points if the irqchip doesn't provide a HW-based retrigger mechanism).

I still maintain that this change should only be applied for the particular interrupts that *require* it, and not as a blanket change affecting everything under the sun. I have proposed such a change in the past, feel free to use it or roll your own.

Thanks for the example of where this blanket functionality wouldn't be desired - I'll re-work this to introduce and use
the IRQD_RESEND_WHEN_IN_PROGRESS flag as you originally suggested.

Just one more thing before I post V3: are you okay with doing the resend here *after* the handler finished running, and using the IRQ_PENDING flag to know to resend it? Or would you like it to be resent in the !irq_may_run(desc) block as you suggested?

I have a slight preference to do it after, only when we know it's ready to be run again, and hence not needed to modify check_irq_resend() to cater for multiple retries.

JG

#5 lore raw

Re: [PATCH 2/2] genirq: fasteoi resends interrupt on concurrent invoke

Hi Marc,

Quoted 17 lines · James Gowans

While I'm glad that you eventually decided to use the resend mechanism

Quoted 11 lines

instead of spinning on the "old" CPU, I still think imposing this behaviour on all users without any discrimination is wrong.

Look at what it does if an interrupt is a wake-up source. You'd pointlessly requeue the interrupt (bonus points if the irqchip doesn't provide a HW-based retrigger mechanism).

I still maintain that this change should only be applied for the particular interrupts that *require* it, and not as a blanket change affecting everything under the sun. I have proposed such a change in the past, feel free to use it or roll your own.

Thanks for the example of where this blanket functionality wouldn't be desired - I'll re-work this to introduce and use
the IRQD_RESEND_WHEN_IN_PROGRESS flag as you originally suggested.

Quoted 8 lines

Just one more thing before I post V3: are you okay with doing the resend here *after* the handler finished running, and using the IRQ_PENDING flag to know to resend it? Or would you like it to be resent in the !irq_may_run(desc) block as you suggested?

I have a slight preference to do it after, only when we know it's ready to be run again, and hence not needed to modify check_irq_resend() to cater for multiple retries.

Hoping/assuming that you're okay with the keeping the resend at the end of the function, here's V3:
https://lore.kernel.org/lkml/[email protected]/

It's now very close to what to suggested originally. :-)

One thing: I'm not totally sure that it's necessary or correct to set this flag on its_vpe_irq_domain_alloc() too - on my system (r6g.metal Graviton 2 host) that doesn't seem to be run. My guess is that it's for a GIC-v4 system which supports posted interrupt which generally get delivery directly to the vCPU if running, but sometimes need to be delivered to the hypervisor (for example if vCPU not running). I can try test this on r7g.metal which might have GIC-v4 to be sure...

JG