x86: Select HARDIRQS_SW_RESEND on x86
RFC v2 1/1 x86: Select HARDIRQS_SW_RESEND on x86
Modern x86 laptops are starting to use GPIO pins as interrupts more and more, e.g. touchpads and touchscreens have almost all moved away from PS/2 and USB to using I2C with a GPIO pin as interrupt. Modern x86 laptops also have almost all moved to using s2idle instead of using the system S3 ACPI power state to suspend.
The Intel and AMD pinctrl drivers do not define irq_retrigger handlers for the irqchips they register, this is causing edge triggered interrupts which happen while suspended using s2idle to get lost.
One specific example of this is the lid switch on some devices, lid switches used to be handled by the embedded-controller, but now the lid open/closed sensor is sometimes directly connected to a GPIO pin. On most devices the ACPI code for this looks like this:
Method (_E00, ...) {
Notify (LID0, 0x80) // Status Change
}Where _E00 is an ACPI event handler for changes on both edges of the GPIO connected to the lid sensor, this event handler is then combined with an _LID method which directly reads the pin. When the device is resumed by opening the lid, the GPIO interrupt will wake the system, but because the pinctrl irqchip doesn't have an irq_retrigger handler, the Notify will not happen. This is not a problem in the case the _LID method directly reads the GPIO, because the drivers/acpi/button.c code will call _LID on resume anyways.
But some devices have an event handler for the GPIO connected to the lid sensor which looks like this:
Method (_E00, ...) {
if (LID_GPIO == One)
LIDS = One
else
LIDS = Zero
Notify (LID0, 0x80) // Status Change
}And the _LID method returns the cached LIDS value, since on open we do not re-run the edge-interrupt handler when we re-enable IRQS on resume (because of the missing irq_retrigger handler), _LID now will keep reporting closed, as LIDS was never changed to reflect the open status, this causes userspace to re-resume the laptop again shortly after opening the lid.
The Intel GPIO controllers do not allow implementing irq_retrigger without emulating it in software, at which point we are better of just using the generic HARDIRQS_SW_RESEND mechanism rather then re-implementing software emulation for this separately in aprox. 14 different pinctrl drivers.
This commit selects HARDIRQS_SW_RESEND solving the problem of edge-triggered GPIO interrupts not being re-triggered on resume when they were triggered during suspend (s2idle) and/or when they were the cause of the wakeup.
---
I'm sending this out as a RFC since I'm not %100 sure this is the best solution and it seems like a somewhat big change to make.
Also maybe we should add a Cc: [email protected] ??? This seems like somewhat a big change for that but it does solve some real issues...
---
Changes in v2:
-v2 is really a resend because I forgot to add the pinctrl people to the Cc -While at it also fix some typos in the commit message
--- arch/x86/Kconfig | 1 + 1 file changed, 1 insertion(+)
arch/x86/Kconfig +1 -0
@@ -118,6 +118,7 @@ config X86 select GENERIC_IRQ_PROBE select GENERIC_IRQ_RESERVATION_MODE select GENERIC_IRQ_SHOW + select HARDIRQS_SW_RESEND select GENERIC_PENDING_IRQ if SMP select GENERIC_SMP_IDLE_THREAD select GENERIC_STRNCPY_FROM_USER
Signature · 1 line
2.24.1
Hans,
Quoted 5 lines · Hans de Goede
The Intel GPIO controllers do not allow implementing
irq_retriggerwithout emulating it in software, at which point we are better of just using the genericHARDIRQS_SW_RESENDmechanism rather then re-implementing software emulation for this separately in aprox. 14 different pinctrl drivers.
Indeed.
Quoted 2 lines
I'm sending this out as a RFC since I'm not %100 sure this is the best solution and it seems like a somewhat big change to make.
It's not that bad. The only affected interrupt chips on x86 should be secondary interrupt chips like the GPIO controller.
ioapic/msi/... have irq_retrigger() functionality, so it won't do the software resend.
I just need to stare at the legacy PIC and the virt stuff.
Quoted 2 lines
Also maybe we should add a Cc: [email protected] ??? This seems like somewhat a big change for that but it does solve some real issues...
Yes. Let me stare at the couple of weird irqchips which might get surprised. I'll teach them not to do that :)
Thanks,
tglx
Hi Thomas,
Quoted 20 lines · Thomas Gleixner
Hans,
Quoted 5 lines · Hans de Goede
The Intel GPIO controllers do not allow implementing
irq_retriggerwithout emulating it in software, at which point we are better of just using the genericHARDIRQS_SW_RESENDmechanism rather then re-implementing software emulation for this separately in aprox. 14 different pinctrl drivers.Indeed.
Quoted 2 lines
I'm sending this out as a RFC since I'm not %100 sure this is the best solution and it seems like a somewhat big change to make.
It's not that bad. The only affected interrupt chips on x86 should be secondary interrupt chips like the GPIO controller.
ioapic/msi/... have
irq_retrigger()functionality, so it won't do the software resend.
Quoted 7 lines
I just need to stare at the legacy PIC and the virt stuff.
Quoted 2 lines
Also maybe we should add a Cc: [email protected] ??? This seems like somewhat a big change for that but it does solve some real issues...
Yes. Let me stare at the couple of weird irqchips which might get surprised. I'll teach them not to do that :)
I know that you are very busy, still I'm wondering is there any progress on this ?
Regards,
Hans
Quoted 10 lines · Hans de Goede
Quoted 7 lines
I just need to stare at the legacy PIC and the virt stuff.
Quoted 2 lines
Also maybe we should add a Cc: [email protected] ??? This seems like somewhat a big change for that but it does solve some real issues...
Yes. Let me stare at the couple of weird irqchips which might get surprised. I'll teach them not to do that :)
I know that you are very busy, still I'm wondering is there any progress on this ?
Bah. That fell through the cracks, but actually I looked at this due to the PCI-E AER wreckage. So yes, this is fine, but we want:
https://lkml.kernel.org/r/[email protected]
https://lkml.kernel.org/r/[email protected]
if we want to backport this to stable.
Thanks,
tglx
Hi,
Quoted 12 lines · Thomas Gleixner
Quoted 10 lines · Hans de Goede
Quoted 7 lines
I just need to stare at the legacy PIC and the virt stuff.
Quoted 2 lines
Also maybe we should add a Cc: [email protected] ??? This seems like somewhat a big change for that but it does solve some real issues...
Yes. Let me stare at the couple of weird irqchips which might get surprised. I'll teach them not to do that :)
I know that you are very busy, still I'm wondering is there any progress on this ?
Quoted 7 lines
Bah. That fell through the cracks, but actually I looked at this due to the PCI-E AER wreckage. So yes, this is fine, but we want:
https://lkml.kernel.org/r/[email protected]
https://lkml.kernel.org/r/[email protected]if we want to backport this to stable.
So far I have seen a few, but not a lot of devices which need this, so I'm not 100% sure what to do here.
Do you consider this change safe / suitable for stable if those 2 patches are backported and applied first?
Regards,
Hans
Quoted 19 lines · Hans de Goede
Quoted 18 lines · Thomas Gleixner
Quoted 10 lines · Hans de Goede
Quoted 7 lines
I just need to stare at the legacy PIC and the virt stuff.
Quoted 2 lines
Also maybe we should add a Cc: [email protected] ??? This seems like somewhat a big change for that but it does solve some real issues...
Yes. Let me stare at the couple of weird irqchips which might get surprised. I'll teach them not to do that :)
I know that you are very busy, still I'm wondering is there any progress on this ?
Bah. That fell through the cracks, but actually I looked at this due to the PCI-E AER wreckage. So yes, this is fine, but we want:
https://lkml.kernel.org/r/[email protected]
https://lkml.kernel.org/r/[email protected]
Quoted 7 lines
Quoted 1 line
if we want to backport this to stable.
So far I have seen a few, but not a lot of devices which need this, so I'm not 100% sure what to do here.
Do you consider this change safe / suitable for stable if those 2 patches are backported and applied first?
I think so. The two patches are on my list for backports anyway, but I wanted to give them some time to simmer.
Thanks,
tglx
Hi,
Quoted 22 lines · Thomas Gleixner
Quoted 21 lines · Hans de Goede
Quoted 19 lines · Thomas Gleixner
Quoted 10 lines · Hans de Goede
Quoted 7 lines
I just need to stare at the legacy PIC and the virt stuff.
Quoted 2 lines
Also maybe we should add a Cc: [email protected] ??? This seems like somewhat a big change for that but it does solve some real issues...
Yes. Let me stare at the couple of weird irqchips which might get surprised. I'll teach them not to do that :)
I know that you are very busy, still I'm wondering is there any progress on this ?
Bah. That fell through the cracks, but actually I looked at this due to the PCI-E AER wreckage. So yes, this is fine, but we want:
https://lkml.kernel.org/r/[email protected] https://lkml.kernel.org/r/[email protected]if we want to backport this to stable.
Quoted 8 lines
Quoted 5 lines
So far I have seen a few, but not a lot of devices which need this, so I'm not 100% sure what to do here.
Do you consider this change safe / suitable for stable if those 2 patches are backported and applied first?
I think so. The two patches are on my list for backports anyway, but I wanted to give them some time to simmer.
OK, I'll submit this patch for stable then once your backports have landed.
Regards,
Hans
The following commit has been merged into the irq/core branch of tip:
The following commit has been merged into the irq/core branch of tip:
Commit-ID: 17e5888e4e180b45af7bafe7f3a86440d42717f3 Gitweb: https://git.kernel.org/tip/17e5888e4e180b45af7bafe7f3a86440d42717f3 Author: Hans de Goede <[email protected]> AuthorDate: Thu, 23 Jan 2020 22:02:42 +01:00 Committer: Thomas Gleixner <[email protected]> CommitterDate: Wed, 11 Mar 2020 22:39:39 +01:00
x86: Select HARDIRQS_SW_RESEND on x86
Modern x86 laptops are starting to use GPIO pins as interrupts more and more, e.g. touchpads and touchscreens have almost all moved away from PS/2 and USB to using I2C with a GPIO pin as interrupt. Modern x86 laptops also have almost all moved to using s2idle instead of using the system S3 ACPI power state to suspend.
The Intel and AMD pinctrl drivers do not define irq_retrigger handlers for the irqchips they register, this is causing edge triggered interrupts which happen while suspended using s2idle to get lost.
One specific example of this is the lid switch on some devices, lid switches used to be handled by the embedded-controller, but now the lid open/closed sensor is sometimes directly connected to a GPIO pin. On most devices the ACPI code for this looks like this:
Method (_E00, ...) {
Notify (LID0, 0x80) // Status Change
}Where _E00 is an ACPI event handler for changes on both edges of the GPIO connected to the lid sensor, this event handler is then combined with an _LID method which directly reads the pin. When the device is resumed by opening the lid, the GPIO interrupt will wake the system, but because the pinctrl irqchip doesn't have an irq_retrigger handler, the Notify will not happen. This is not a problem in the case the _LID method directly reads the GPIO, because the drivers/acpi/button.c code will call _LID on resume anyways.
But some devices have an event handler for the GPIO connected to the lid sensor which looks like this:
Method (_E00, ...) {
if (LID_GPIO == One)
LIDS = One
else
LIDS = Zero
Notify (LID0, 0x80) // Status Change
}And the _LID method returns the cached LIDS value, since on open we do not re-run the edge-interrupt handler when we re-enable IRQS on resume (because of the missing irq_retrigger handler), _LID now will keep reporting closed, as LIDS was never changed to reflect the open status, this causes userspace to re-resume the laptop again shortly after opening the lid.
The Intel GPIO controllers do not allow implementing irq_retrigger without emulating it in software, at which point we are better of just using the generic HARDIRQS_SW_RESEND mechanism rather then re-implementing software emulation for this separately in aprox. 14 different pinctrl drivers.
Select HARDIRQS_SW_RESEND to solve the problem of edge-triggered GPIO interrupts not being re-triggered on resume when they were triggered during suspend (s2idle) and/or when they were the cause of the wakeup.
This requires
008f1d60fe25 ("x86/apic/vector: Force interupt handler invocation to irq context") c16816acd086 ("genirq: Add protection against unsafe usage of generic_handle_irq()")
to protect the APIC based interrupts from being wreckaged by a software resend.
--- arch/x86/Kconfig | 1 + 1 file changed, 1 insertion(+)
arch/x86/Kconfig +1 -0
@@ -128,6 +128,7 @@ config X86 select GENERIC_GETTIMEOFDAY select GENERIC_VDSO_TIME_NS select GUP_GET_PTE_LOW_HIGH if X86_PAE + select HARDIRQS_SW_RESEND select HARDLOCKUP_CHECK_TIMESTAMP if X86_64 select HAVE_ACPI_APEI if ACPI select HAVE_ACPI_APEI_NMI if ACPI
Quoted 4 lines · On Wed, Mar 11, 2020 at 10:42 PM tip-bot2 for Hans de Goede…
select
GENERIC_GETTIMEOFDAY
selectGENERIC_VDSO_TIME_NS
selectGUP_GET_PTE_LOW_HIGHifX86_PAE
+ selectHARDIRQS_SW_RESEND
Just help me understand the semantics of this thing...
According to the text in KConfig:
# Tasklet based software resend for pending interrupts on enable_irq() config HARDIRQS_SW_RESEND
bool
According to
commit a4633adcdbc15ac51afcd0e1395de58cee27cf92
[PATCH] genirq: add genirq sw IRQ-retrigger
Enable platforms that do not have a hardware-assisted
hardirq-resend mechanism
to resend them via a softirq-driven IRQ emulation mechanism.
so when enable_irq() is called, if the IRQ is already asserted, it will be distributed in the form of a software irq?
OK I give up I don't understand the semantics of this thing.
Maybe it's because I think of a register where the IRQ line is just a level IRQ bit thing that stays high as long as the IRQ is not handled.
So I suppose it is for any type of transient IRQ such as edge triggered that happened before the system came back online entirely and now the only remnant of it is a bit in the irchip status register?
I see that ARM and ARM64 simply just select this. What happens if you do that and why is x86 not selecting it in general?
Explain it to me and I promise to patch kernel/irq/Kconfig with the explanation so you don't have to give it again.
Yours,
Linus Walleij
Hi,
Quoted 30 lines · Linus Walleij
Quoted 4 lines · On Wed, Mar 11, 2020 at 10:42 PM tip-bot2 for Hans de Goede…
select
GENERIC_GETTIMEOFDAY
selectGENERIC_VDSO_TIME_NS
selectGUP_GET_PTE_LOW_HIGHifX86_PAE
+ selectHARDIRQS_SW_RESENDJust help me understand the semantics of this thing...
According to the text in KConfig:
# Tasklet based software resend for pending interrupts on
enable_irq()configHARDIRQS_SW_RESEND
boolAccording to
commita4633adcdbc15ac51afcd0e1395de58cee27cf92[PATCH] genirq: add genirq sw IRQ-retrigger
Enable platforms that do not have a hardware-assisted
hardirq-resend mechanism
to resend them via a softirq-driven IRQ emulation mechanism.so when
enable_irq()is called, if the IRQ is already asserted, it will be distributed in the form of a software irq?OK I give up I don't understand the semantics of this thing.
Quoted 8 lines
Maybe it's because I think of a register where the IRQ line is just a level IRQ bit thing that stays high as long as the IRQ is not handled.
So I suppose it is for any type of transient IRQ such as edge triggered that happened before the system came back online entirely and now the only remnant of it is a bit in the irchip status register?
The way I understand it is like this:
1. We have an edge triggered IRQ from a peripheral to a GPIO controller
2. We have a level triggered IRQ from the GPIO controller to the "root" IRQ controller.
3. With modern x86 suspend, we do not really put the entire system in a firmware-controller suspend state, instead the CPU is halted until any IRQ happens; and there is a power-management micro-controller which shuts various things down while the CPU is halted leading to similar power consumption as old S3 suspend.
The combination of these 3 means that we must ack the edge triggered IRQ at the GPIO controller level even while suspended (we briefly wake up for this) to make the level-triggered IRQ coming from the GPIO controller low so that we can go back to sleep.
When this happens we record in the kernel IRQ tracking data for the edge-triggered IRQ tied to the GPIO (there are no "remnants" of it in any chuip registers), that the IRQ needs to be replayed on resume. Some IRQ controllers allow writing a register to retrigger the IRQ without the level on the external GPIO actually changing.
Some IRQ controllers do not allow this, in this case we need to emulate this retriggering in software, this is what the HARDIRQS_SW_RESEND option is for, this handles this in a generic way, so that we do not have to add emulation to every IRQ-chip driver where the hardware lacks a "retrigger" register.
Quoted 2 lines
I see that ARM and ARM64 simply just select this. What happens if you do that and why is x86 not selecting it in general?
Erm, "selecting it in general" (well at least on x86) is what this patch is doing. But I guess you mean why is this not selected on all architectures? If I've understood tglx correctly the HARDIRQS_SW_RESEND option is incompatible with some irq-chip drivers. I think it was incompatible with using nested threaded handlers are some such...
tglx can probably explain this way better then I can :)
Regards,
Hans
On Thu, Mar 12, 2020 at 2:49 PM Hans de Goede <[email protected]> wrote:
[Me]
Quoted 5 lines
Quoted 2 lines
I see that ARM and ARM64 simply just select this. What happens if you do that and why is x86 not selecting it in general?
Erm, "selecting it in general" (well at least on x86) is what this patch is doing.
Sorry that I was unclear, what I meant to say is why wasn't this done ages ago since so many important architectures seem to have it enabled by default.
I suppose the reason would be something like "firmware/BIOS should handle that for us" and recently that has started to break apart and x86 platforms started to be more like ARM?
Yours,
Linus Walleij
Hi,
Quoted 9 lines · Linus Walleij
On Thu, Mar 12, 2020 at 2:49 PM Hans de Goede <[email protected]> wrote:
[Me]
Quoted 5 lines
Quoted 2 lines
I see that ARM and ARM64 simply just select this. What happens if you do that and why is x86 not selecting it in general?
Erm, "selecting it in general" (well at least on x86) is what this patch is doing.
Quoted 7 lines
Sorry that I was unclear, what I meant to say is why wasn't this done ages ago since so many important architectures seem to have it enabled by default.
I suppose the reason would be something like "
firmware/BIOSshould handle that for us" and recently that has started to break apart and x86 platforms started to be more like ARM?
That (x86 becoming more like ARM, sorta, kinda) as well as that turning it on on x86 was not safe until Thomas wrote the 2 patches which are marked as dependencies in the commit message for this patch.
Regards,
Hans
Quoted 14 lines · Linus Walleij
On Wed, Mar 11, 2020 at 10:42 PM tip-bot2 for Hans de Goede Just help me understand the semantics of this thing...
According to the text in KConfig:
# Tasklet based software resend for pending interrupts on
enable_irq()configHARDIRQS_SW_RESEND
boolAccording to
commita4633adcdbc15ac51afcd0e1395de58cee27cf92[PATCH] genirq: add genirq sw IRQ-retrigger
Quoted 8 lines
Enable platforms that do not have a hardware-assisted
hardirq-resend mechanism
to resend them via a softirq-driven IRQ emulation mechanism.so when
enable_irq()is called, if the IRQ is already asserted, it will be distributed in the form of a software irq?OK I give up I don't understand the semantics of this thing.
Level type interrupts are "resending" in hardware as long as the device interrupt is still asserted.
The problem are edge interrupts.
When an edge interrupt is disabled via disable_irq() the core does not mask the chip because if the device raises an interrupt not all interrupt chips latch that and forward it to the CPU on unmask, i.e. some interrupt chips simply ignore an etch when the line is masked.
So when the device raises an edge while the interrupt is disabled the core still handles the hardware interrupt and:
- masks the interrupt line
- sets the pending bit
- does not invoke the device handler
On enable_irq() the pending bit is checked and if set the interrupt is tried to be retriggered or resent, but only if it's edge type.
So if the interrupt chip provides a irq_retrigger() callback the core uses that and only if this fails or is not available it resorts to software "resend" which means queueing it for execution in tasklet context.
Quoted 2 lines
I see that ARM and ARM64 simply just select this. What happens if you do that and why is x86 not selecting it in general?
irq resending on X86 is not really problem free for interrupts which are directly connect to the local APIC. The only way which is halfways safe is the hardware retrigger. See
https://lkml.kernel.org/r/[email protected] https://lkml.kernel.org/r/[email protected]
for the gory details. The GPIO interrupts which hang off behind some slow bus or are multiplexed in other ways are not affected by this hardware design induced madness.
As I don't know how many other architectures have trainwrecked interrupt delivery mechanisms (IA64 definitely does), I'm more than reluctant to inflict this on the world unconditionally.
Hope that helps.
Thanks,
tglx