A problem about interrupt when booting a captured kernel

DISCUSSION Yang Yingliang 3 participants 10 messages Nov 13, 2015 – Nov 19, 2015

Share & tools

Yang Yingliang

author

2015-11-13 10:09 UTC #0 lore raw

Hi, Marc

The kexec will boot a captured kernel while the kernel panic. But
it boots failed if the kernel panic in handler function of PPI. The reason is that the PPI has not been 'eoi', other interrupts can not be handled when booting the captured kernel.

The kexec will call irq_eoi to end the irqs that have
IRQD_IRQ_INPROGRESS flag. But PPIs don't have this flag, so it won't be ended.

Three ways to solve this problem we can think :
1. Is there a way to reset gic like its_reset ?
2. Can we add some flag for calling irq_eoi ?
3. Just 'eoi' all PPIs without checking flags in kexec.

Please give some advice.

Thanks
Yang

Marc Zyngier

2015-11-16 19:26 UTC #1 lore raw

Quoted 12 lines · Yang Yingliang

Hi, Marc
The kexec will boot a captured kernel while the kernel panic. But
it boots failed if the kernel panic in handler function of PPI. The reason is that the PPI has not been 'eoi', other interrupts can not be handled when booting the captured kernel.
The kexec will call irq_eoi to end the irqs that have
IRQD_IRQ_INPROGRESS flag. But PPIs don't have this flag, so it won't be ended.

Quoted 6 lines

Three ways to solve this problem we can think :
1. Is there a way to reset gic like its_reset ?
2. Can we add some flag for calling irq_eoi ?
3. Just 'eoi' all PPIs without checking flags in kexec.
Please give some advice.

Good timing. Please see:

http://lists.infradead.org/pipermail/linux-arm-kernel/2015-November/385383.html

and let me know if that fixes things for you.

Thanks,

M.

Signature · 1 line

Jazz is not dead. It just smells funny...

AKASHI Takahiro

2015-11-17 01:05 UTC #2 lore raw

Marc,
(Cc: Mark)

Quoted 18 lines · Marc Zyngier

Quoted 17 lines · Yang Yingliang
Hi, Marc
The kexec will boot a captured kernel while the kernel panic. But
it boots failed if the kernel panic in handler function of PPI. The reason is that the PPI has not been 'eoi', other interrupts can not be handled when booting the captured kernel.
The kexec will call irq_eoi to end the irqs that have
IRQD_IRQ_INPROGRESS flag. But PPIs don't have this flag, so it won't be ended.
Three ways to solve this problem we can think :
1. Is there a way to reset gic like its_reset ?
2. Can we add some flag for calling irq_eoi ?
3. Just 'eoi' all PPIs without checking flags in kexec.

Quoted 5 lines

Quoted 1 line
Please give some advice.
Good timing. Please see:
http://lists.infradead.org/pipermail/linux-arm-kernel/2015-November/385383.html

I removed machine_kexec_mask_interrupts() from my arm64 kdump patch series[1] due to the past discussions[2].

Is it the time that I should resurrect the code?

[1] http://lists.infradead.org/pipermail/linux-arm-kernel/2015-November/383581.html
[2] http://lists.infradead.org/pipermail/linux-arm-kernel/2015-August/362414.html

Thanks,
-Takahiro AKASHI

Quoted 6 lines

and let me know if that fixes things for you.
Thanks,
	M.

Marc Zyngier

2015-11-17 09:07 UTC #3 lore raw

Quoted 24 lines · On Tue, 17 Nov 2015 10:05:23 +0900 AKASHI Takahiro…

Marc,
(Cc: Mark)
Quoted 20 lines · Marc Zyngier
Quoted 18 lines · Yang Yingliang
Hi, Marc
The kexec will boot a captured kernel while the kernel panic. But
it boots failed if the kernel panic in handler function of PPI. The reason is that the PPI has not been 'eoi', other interrupts can not be handled when booting the captured kernel.
The kexec will call irq_eoi to end the irqs that have
IRQD_IRQ_INPROGRESS flag. But PPIs don't have this flag, so it won't be ended.
Three ways to solve this problem we can think :
1. Is there a way to reset gic like its_reset ?
2. Can we add some flag for calling irq_eoi ?
3. Just 'eoi' all PPIs without checking flags in kexec.
Please give some advice.

Quoted 8 lines

Quoted 3 lines
Good timing. Please see:
http://lists.infradead.org/pipermail/linux-arm-kernel/2015-November/385383.html
I removed machine_kexec_mask_interrupts() from my arm64 kdump patch series[1] due to the past discussions[2].
Is it the time that I should resurrect the code?

Probably.

Thanks,

M.

Signature · 1 line

Jazz is not dead. It just smells funny.

AKASHI Takahiro

2015-11-18 08:03 UTC #4 lore raw

Quoted 29 lines · Marc Zyngier

Quoted 26 lines · On Tue, 17 Nov 2015 10:05:23 +0900 AKASHI Takahiro…
Marc,
(Cc: Mark)
Quoted 22 lines · Marc Zyngier
Quoted 12 lines · Yang Yingliang
Hi, Marc
The kexec will boot a captured kernel while the kernel panic. But
it boots failed if the kernel panic in handler function of PPI. The reason is that the PPI has not been 'eoi', other interrupts can not be handled when booting the captured kernel.
The kexec will call irq_eoi to end the irqs that have
IRQD_IRQ_INPROGRESS flag. But PPIs don't have this flag, so it won't be ended.
Quoted 6 lines
Three ways to solve this problem we can think :
1. Is there a way to reset gic like its_reset ?
2. Can we add some flag for calling irq_eoi ?
3. Just 'eoi' all PPIs without checking flags in kexec.
Please give some advice.
Good timing. Please see:

Quoted 8 lines

Quoted 6 lines
Quoted 1 line
http://lists.infradead.org/pipermail/linux-arm-kernel/2015-November/385383.html
I removed machine_kexec_mask_interrupts() from my arm64 kdump patch series[1] due to the past discussions[2].
Is it the time that I should resurrect the code?
Probably.

Let me make sure.
Have your patches here, especially #1, also addressed the issue that you pointed out in [1]?

[1] http://lists.infradead.org/pipermail/linux-arm-kernel/2015-April/338969.html

-Takahiro AKASHI

Quoted 4 lines

Thanks,
	M.

Marc Zyngier

2015-11-18 11:29 UTC #5 lore raw

Quoted 37 lines · AKASHI Takahiro

Quoted 36 lines · Marc Zyngier
Quoted 32 lines · On Tue, 17 Nov 2015 10:05:23 +0900 AKASHI Takahiro…
Marc,
(Cc: Mark)
Quoted 18 lines · Marc Zyngier
Quoted 17 lines · Yang Yingliang
Hi, Marc
The kexec will boot a captured kernel while the kernel panic. But
it boots failed if the kernel panic in handler function of PPI. The reason is that the PPI has not been 'eoi', other interrupts can not be handled when booting the captured kernel.
The kexec will call irq_eoi to end the irqs that have
IRQD_IRQ_INPROGRESS flag. But PPIs don't have this flag, so it won't be ended.
Three ways to solve this problem we can think :
1. Is there a way to reset gic like its_reset ?
2. Can we add some flag for calling irq_eoi ?
3. Just 'eoi' all PPIs without checking flags in kexec.
Quoted 5 lines
Quoted 1 line
Please give some advice.
Good timing. Please see:
http://lists.infradead.org/pipermail/linux-arm-kernel/2015-November/385383.html
I removed machine_kexec_mask_interrupts() from my arm64 kdump patch series[1] due to the past discussions[2].
Is it the time that I should resurrect the code?

Quoted 7 lines

Quoted 1 line
Probably.
Let me make sure.
Have your patches here, especially #1, also addressed the issue that you pointed out in [1]?
[1] http://lists.infradead.org/pipermail/linux-arm-kernel/2015-April/338969.html

Patch #1 still has the current behaviour (deactivate any in flight interrupt), but does it in a more complete way. The additional 3 patches make sure that the GIC (any version) is not affected by having active interrupts when getting initialized.

These are the two sides of the same coin, basically. I'm still not fond of what we do in machine_kexec_mask_interrupts(), but we can't rely on the secondary kernel doing the right thing either.

Thanks,

M.

Signature · 1 line

Jazz is not dead. It just smells funny...

Yang Yingliang

author

2015-11-17 03:48 UTC #6 lore raw

In reply to Marc Zyngier

Quoted 18 lines · Marc Zyngier

Quoted 17 lines · Yang Yingliang
Hi, Marc
The kexec will boot a captured kernel while the kernel panic. But
it boots failed if the kernel panic in handler function of PPI. The reason is that the PPI has not been 'eoi', other interrupts can not be handled when booting the captured kernel.
The kexec will call irq_eoi to end the irqs that have
IRQD_IRQ_INPROGRESS flag. But PPIs don't have this flag, so it won't be ended.
Three ways to solve this problem we can think :
1. Is there a way to reset gic like its_reset ?
2. Can we add some flag for calling irq_eoi ?
3. Just 'eoi' all PPIs without checking flags in kexec.

Quoted 7 lines

Quoted 1 line
Please give some advice.
Good timing. Please see:
http://lists.infradead.org/pipermail/linux-arm-kernel/2015-November/385383.html
and let me know if that fixes things for you.

I tried patch #1, deactivating interrupt can not fix the problem. But EOI the interrupt can.

I think the reason is GICC status is not cleared.
So if it means, in hardware, we should clear the GICC status while deactivating the.

Thanks
Yang

Quoted 5 lines

Thanks,
	M.

Marc Zyngier

2015-11-17 09:16 UTC #7 lore raw

Quoted 23 lines · On Tue, 17 Nov 2015 11:48:07 +0800 Yang Yingliang <[email protected]>…

Hi
Quoted 20 lines · Marc Zyngier
Quoted 18 lines · Yang Yingliang
Hi, Marc
The kexec will boot a captured kernel while the kernel panic. But
it boots failed if the kernel panic in handler function of PPI. The reason is that the PPI has not been 'eoi', other interrupts can not be handled when booting the captured kernel.
The kexec will call irq_eoi to end the irqs that have
IRQD_IRQ_INPROGRESS flag. But PPIs don't have this flag, so it won't be ended.
Three ways to solve this problem we can think :
1. Is there a way to reset gic like its_reset ?
2. Can we add some flag for calling irq_eoi ?
3. Just 'eoi' all PPIs without checking flags in kexec.
Please give some advice.

Quoted 8 lines

Quoted 5 lines
Good timing. Please see:
http://lists.infradead.org/pipermail/linux-arm-kernel/2015-November/385383.html
and let me know if that fixes things for you.
I tried patch #1, deactivating interrupt can not fix the problem. But EOI the interrupt can.

You have noticed that this patch is for ARM, not arm64, right?

Quoted 3 lines

I think the reason is GICC status is not cleared.
So if it means, in hardware, we should clear the GICC status while deactivating the.

Writing to the EOI register or writing to the ACTIVE register have the exact same result as far as kexec is concerned (unless your GIC is not compliant with the architecture - in which case I can't help you).

Does applying the other patches help?

Thanks,

M.

Signature · 1 line

Jazz is not dead. It just smells funny.

Yang Yingliang

author

2015-11-19 03:42 UTC #8 lore raw

Quoted 28 lines · Marc Zyngier

Quoted 25 lines · On Tue, 17 Nov 2015 11:48:07 +0800 Yang Yingliang <[email protected]>…
Hi
Quoted 22 lines · Marc Zyngier
Quoted 12 lines · Yang Yingliang
Hi, Marc
The kexec will boot a captured kernel while the kernel panic. But
it boots failed if the kernel panic in handler function of PPI. The reason is that the PPI has not been 'eoi', other interrupts can not be handled when booting the captured kernel.
The kexec will call irq_eoi to end the irqs that have
IRQD_IRQ_INPROGRESS flag. But PPIs don't have this flag, so it won't be ended.
Quoted 6 lines
Three ways to solve this problem we can think :
1. Is there a way to reset gic like its_reset ?
2. Can we add some flag for calling irq_eoi ?
3. Just 'eoi' all PPIs without checking flags in kexec.
Please give some advice.
Good timing. Please see:

Quoted 8 lines

Quoted 6 lines
Quoted 3 lines
http://lists.infradead.org/pipermail/linux-arm-kernel/2015-November/385383.html
and let me know if that fixes things for you.
I tried patch #1, deactivating interrupt can not fix the problem. But EOI the interrupt can.
You have noticed that this patch is for ARM, not arm64, right?

Yes, I noticed.

Quoted 10 lines

Quoted 4 lines
I think the reason is GICC status is not cleared.
So if it means, in hardware, we should clear the GICC status while deactivating the.
Writing to the EOI register or writing to the ACTIVE register have the exact same result as far as kexec is concerned (unless your GIC is not compliant with the architecture - in which case I can't help you).
Does applying the other patches help?

No, it doesn't help.

Quoted 5 lines

Thanks,
	M.

Marc Zyngier

2015-11-19 08:40 UTC #9 lore raw

Quoted 47 lines · Yang Yingliang

Quoted 28 lines · Marc Zyngier
Quoted 25 lines · On Tue, 17 Nov 2015 11:48:07 +0800 Yang Yingliang <[email protected]>…
Hi
Quoted 22 lines · Marc Zyngier
Quoted 12 lines · Yang Yingliang
Hi, Marc
The kexec will boot a captured kernel while the kernel panic. But
it boots failed if the kernel panic in handler function of PPI. The reason is that the PPI has not been 'eoi', other interrupts can not be handled when booting the captured kernel.
The kexec will call irq_eoi to end the irqs that have
IRQD_IRQ_INPROGRESS flag. But PPIs don't have this flag, so it won't be ended.
Quoted 6 lines
Three ways to solve this problem we can think :
1. Is there a way to reset gic like its_reset ?
2. Can we add some flag for calling irq_eoi ?
3. Just 'eoi' all PPIs without checking flags in kexec.
Please give some advice.
Good timing. Please see:
Quoted 8 lines
Quoted 6 lines
Quoted 3 lines
http://lists.infradead.org/pipermail/linux-arm-kernel/2015-November/385383.html
and let me know if that fixes things for you.
I tried patch #1, deactivating interrupt can not fix the problem. But EOI the interrupt can.
You have noticed that this patch is for ARM, not arm64, right?
Yes, I noticed.
Quoted 5 lines
Quoted 4 lines
I think the reason is GICC status is not cleared.
So if it means, in hardware, we should clear the GICC status while deactivating the.

Quoted 7 lines

Quoted 5 lines
Writing to the EOI register or writing to the ACTIVE register have the exact same result as far as kexec is concerned (unless your GIC is not compliant with the architecture - in which case I can't help you).
Does applying the other patches help?
No, it doesn't help.

That's pretty annoying. It means that on your implementation, clearing the ACTIVE bit doesn't deactivate an interrupt. Great! This probably also break KVM's guest timer in 4.3.

Using EOI won't help either because you don't know whether to do a priority drop (which has to happen in reverse order w.r.t. the read of IAR) or a deactivate (and no matter how many flag you add, you still won't know).

Can you please get in touch with your HW folks so that they can explain:
1) why it doesn't work,
2) if there is any workaround.

Thanks,

M.

Signature · 1 line

Jazz is not dead. It just smells funny...