tpm_tis: Disable interrupts on ThinkPad T490s
Subsystems (2)
PATCH 1/1 tpm_tis: Disable interrupts on ThinkPad T490s
There is a misconfiguration in the bios of the gpio pin used for the interrupt in the T490s. When interrupts are enabled in the tpm_tis driver code this results in an interrupt storm. This was initially reported when we attempted to enable the interrupt code in the tpm_tis driver, which previously wasn't setting a flag to enable it. Due to the reports of the interrupt storm that code was reverted and we went back to polling instead of using interrupts. Now that we know the T490s problem is a firmware issue, add code to check if the system is a T490s and disable interrupts if that is the case. This will allow us to enable interrupts for everyone else. If the user has a fixed bios they can force the enabling of interrupts with tpm_tis.interrupts=1 on the kernel command line.
--- drivers/char/tpm/tpm_tis.c | 29 +++++++++++++++++++++++++++-- 1 file changed, 27 insertions(+), 2 deletions(-)
drivers/char/tpm/tpm_tis.c +27 -2
@@ -27,6 +27,7 @@#include <linux/of.h> #include <linux/of_device.h> #include <linux/kernel.h> +#include <linux/dmi.h> #include "tpm.h" #include "tpm_tis_core.h"
@@ -49,8 +50,8 @@ static inline struct tpm_tis_tcg_phy *to_tpm_tis_tcg_phy(struct tpm_tis_data *da return container_of(data, struct tpm_tis_tcg_phy, priv); } -static bool interrupts = true; -module_param (interrupts, bool,0444 ); +static int interrupts =-1 ; +module_param (interrupts, int,0444 );MODULE_PARM_DESC (interrupts,"Enable interrupts" );static bool itpm;
@@ -63,6 +64,28 @@ module_param(force, bool, 0444);MODULE_PARM_DESC (force,"Force device probe rather than using ACPI entry" );#endif +static int tpm_tis_disable_irq (const struct dmi_system_id *d) +{ +if (interrupts ==-1 ) { +pr_notice ("tpm_tis: %s detected: disabling interrupts.\n" , d->ident); + interrupts =0 ; + } + +return 0 ; +} + +static const struct dmi_system_id tpm_tis_dmi_table[] = { + { + .callback = tpm_tis_disable_irq, + .ident ="ThinkPad T490s" , + .matches = { +DMI_MATCH (DMI_SYS_VENDOR ,"LENOVO" ), +DMI_MATCH (DMI_PRODUCT_VERSION ,"ThinkPad T490s" ), + }, + }, + {} +}; +#if defined(CONFIG_PNP ) && defined(CONFIG_ACPI )static int has_hid (struct acpi_device *dev,const char *hid) {
@@ -192,6 +215,8 @@ static int tpm_tis_init(struct device *dev, struct tpm_info *tpm_info)int irq =-1 ;int rc; +dmi_check_system (tpm_tis_dmi_table); + rc =check_acpi_tpm2 (dev);if (rc)return rc;
Signature · 1 line
2.28.0
James should this get tacked on the end of your patchset?
Regards,
Jerry
Quoted 13 lines · Jerry Snitselaar
There is a misconfiguration in the bios of the gpio pin used for the interrupt in the T490s. When interrupts are enabled in the
tpm_tisdriver code this results in an interrupt storm. This was initially reported when we attempted to enable the interrupt code in thetpm_tisdriver, which previously wasn't setting a flag to enable it. Due to the reports of the interrupt storm that code was reverted and we went back to polling instead of using interrupts. Now that we know the T490s problem is a firmware issue, add code to check if the system is a T490s and disable interrupts if that is the case. This will allow us to enable interrupts for everyone else. If the user has a fixed bios they can force the enabling of interrupts with tpm_tis.interrupts=1 on the kernel command line.
I think an implication of this is that systems haven't been well-tested with interrupts enabled. In general when we've found a firmware issue in one place it ends up happening elsewhere as well, so it wouldn't surprise me if there are other machines that will also be unhappy with interrupts enabled. Would it be possible to automatically detect this case (eg, if we get more than a certain number of interrupts in a certain timeframe immediately after enabling the interrupt) and automatically fall back to polling in that case? It would also mean that users with fixed firmware wouldn't need to pass a parameter.
Hi,
Quoted 15 lines · Matthew Garrett
Quoted 13 lines · Jerry Snitselaar
There is a misconfiguration in the bios of the gpio pin used for the interrupt in the T490s. When interrupts are enabled in the
tpm_tisdriver code this results in an interrupt storm. This was initially reported when we attempted to enable the interrupt code in thetpm_tisdriver, which previously wasn't setting a flag to enable it. Due to the reports of the interrupt storm that code was reverted and we went back to polling instead of using interrupts. Now that we know the T490s problem is a firmware issue, add code to check if the system is a T490s and disable interrupts if that is the case. This will allow us to enable interrupts for everyone else. If the user has a fixed bios they can force the enabling of interrupts with tpm_tis.interrupts=1 on the kernel command line.
Quoted 10 lines
I think an implication of this is that systems haven't been well-tested with interrupts enabled. In general when we've found a firmware issue in one place it ends up happening elsewhere as well, so it wouldn't surprise me if there are other machines that will also be unhappy with interrupts enabled. Would it be possible to automatically detect this case (eg, if we get more than a certain number of interrupts in a certain timeframe immediately after enabling the interrupt) and automatically fall back to polling in that case? It would also mean that users with fixed firmware wouldn't need to pass a parameter.
IIRC then at least on the T490 the irq storm caused systems to not boot in some cases. I guess if we detect the storm and disable the irq we might fix that... OTOH this problem seems to only hit a certain generation of Thinkpads so with some luck the denylist should not be too big and the denylist approach should work.
Regards,
Hans
Matthew Garrett @ 2020-10-15 15:39 MST:
Quoted 15 lines · Jerry Snitselaar
Quoted 13 lines
There is a misconfiguration in the bios of the gpio pin used for the interrupt in the T490s. When interrupts are enabled in the
tpm_tisdriver code this results in an interrupt storm. This was initially reported when we attempted to enable the interrupt code in thetpm_tisdriver, which previously wasn't setting a flag to enable it. Due to the reports of the interrupt storm that code was reverted and we went back to polling instead of using interrupts. Now that we know the T490s problem is a firmware issue, add code to check if the system is a T490s and disable interrupts if that is the case. This will allow us to enable interrupts for everyone else. If the user has a fixed bios they can force the enabling of interrupts with tpm_tis.interrupts=1 on the kernel command line.
Quoted 10 lines
I think an implication of this is that systems haven't been well-tested with interrupts enabled. In general when we've found a firmware issue in one place it ends up happening elsewhere as well, so it wouldn't surprise me if there are other machines that will also be unhappy with interrupts enabled. Would it be possible to automatically detect this case (eg, if we get more than a certain number of interrupts in a certain timeframe immediately after enabling the interrupt) and automatically fall back to polling in that case? It would also mean that users with fixed firmware wouldn't need to pass a parameter.
I believe Matthew is correct here. I found another system today with completely different vendor for both the system and the tpm chip. In addition another Lenovo model, the L490, has the issue.
This initial attempt at a solution like Matthew suggested works on the system I found today, but I imagine it is all sorts of wrong. In the 2 systems where I've seen it, there are about 100000 interrupts in around 1.5 seconds, and then the irq code shuts down the interrupt because they aren't being handled.
drivers/char/tpm/tpm_tis_core.c +37 -5
@@ -27,6 +27,11 @@#include "tpm.h" #include "tpm_tis_core.h" +static unsigned int time_start =0 ; +static bool storm_check = true; +static bool storm_killed = false; +static u32 irqs_fired =0 ; +static void tpm_tis_clkrun_enable (struct tpm_chip *chip,bool value);static void tpm_tis_enable_interrupt (struct tpm_chip *chip,u8 mask)
@@ -464,25 +469,31 @@ static int tpm_tis_send_data(struct tpm_chip *chip, const u8 *buf, size_t len)return rc; } -static void disable_interrupts (struct tpm_chip *chip) +static void __disable_interrupts (struct tpm_chip *chip) {struct tpm_tis_data *priv =dev_get_drvdata (&chip->dev);u32 intmask;int rc; -if (priv->irq ==0 ) -return ; - rc =tpm_tis_read32 (priv,TPM_INT_ENABLE (priv->locality), &intmask);if (rc <0 ) intmask =0 ; intmask &= ~TPM_GLOBAL_INT_ENABLE ; rc =tpm_tis_write32 (priv,TPM_INT_ENABLE (priv->locality), intmask); + chip->flags &= ~TPM_CHIP_FLAG_IRQ ; +} + +static void disable_interrupts (struct tpm_chip *chip) +{ +struct tpm_tis_data *priv =dev_get_drvdata (&chip->dev); +if (priv->irq ==0 ) +return ; + +__disable_interrupts (chip);devm_free_irq (chip->dev.parent, priv->irq, chip); priv->irq =0 ; - chip->flags &= ~TPM_CHIP_FLAG_IRQ ; }/*
@@ -528,6 +539,12 @@ static int tpm_tis_send(struct tpm_chip *chip, u8 *buf, size_t len)int rc, irq;struct tpm_tis_data *priv =dev_get_drvdata (&chip->dev); +if (unlikely (storm_killed)) { +devm_free_irq (chip->dev.parent, priv->irq, chip); + priv->irq =0 ; + storm_killed = false; + } +if (!(chip->flags &TPM_CHIP_FLAG_IRQ ) || priv->irq_tested)return tpm_tis_send_main (chip, buf, len);
@@ -748,6 +765,21 @@ static irqreturn_t tis_int_handler(int dummy, void *dev_id)u32 interrupt;int i, rc; +if (storm_check) { + irqs_fired++; + +if (!time_start) { + time_start =jiffies_to_msecs (jiffies); + }else if ((irqs_fired >1000 ) && (jiffies_to_msecs (jiffies) - jiffies <500 )) { +__disable_interrupts (chip); + storm_check = false; + storm_killed = true; +return IRQ_HANDLED ; + }else if ((jiffies_to_msecs (jiffies) - time_start >500 ) && (irqs_fired <1000 )) { + storm_check = false; + } + } + rc =tpm_tis_read32 (priv,TPM_INT_STATUS (priv->locality), &interrupt);if (rc <0 )return IRQ_NONE ;
Hi,
Quoted 33 lines · Jerry Snitselaar
Matthew Garrett @ 2020-10-15 15:39 MST:
Quoted 15 lines · Jerry Snitselaar
Quoted 13 lines
There is a misconfiguration in the bios of the gpio pin used for the interrupt in the T490s. When interrupts are enabled in the
tpm_tisdriver code this results in an interrupt storm. This was initially reported when we attempted to enable the interrupt code in thetpm_tisdriver, which previously wasn't setting a flag to enable it. Due to the reports of the interrupt storm that code was reverted and we went back to polling instead of using interrupts. Now that we know the T490s problem is a firmware issue, add code to check if the system is a T490s and disable interrupts if that is the case. This will allow us to enable interrupts for everyone else. If the user has a fixed bios they can force the enabling of interrupts with tpm_tis.interrupts=1 on the kernel command line.Quoted 10 lines
I think an implication of this is that systems haven't been well-tested with interrupts enabled. In general when we've found a firmware issue in one place it ends up happening elsewhere as well, so it wouldn't surprise me if there are other machines that will also be unhappy with interrupts enabled. Would it be possible to automatically detect this case (eg, if we get more than a certain number of interrupts in a certain timeframe immediately after enabling the interrupt) and automatically fall back to polling in that case? It would also mean that users with fixed firmware wouldn't need to pass a parameter.
I believe Matthew is correct here. I found another system today with completely different vendor for both the system and the tpm chip. In addition another Lenovo model, the L490, has the issue.
Quoted 5 lines
This initial attempt at a solution like Matthew suggested works on the system I found today, but I imagine it is all sorts of wrong. In the 2 systems where I've seen it, there are about 100000 interrupts in around 1.5 seconds, and then the irq code shuts down the interrupt because they aren't being handled.
Is that with your patch? The IRQ should be silenced as soon as devm_free_irq(chip->dev.parent, priv->irq, chip); is called.
Depending on if we can get your storm-detection to work or not, we might also choose to just never try to use the IRQ (at least on x86 systems). AFAIK the TPM is never used for high-throughput stuff so the polling overhead should not be a big deal (and I'm getting the feeling that Windows always polls).
Regards,
Hans
Quoted 89 lines
drivers/char/tpm/tpm_tis_core.c +37
-5@@ -27,6 +27,11 @@#include "tpm.h" #include "tpm_tis_core.h" +static unsigned int time_start =0 ; +static bool storm_check = true; +static bool storm_killed = false; +static u32 irqs_fired =0 ; +static void tpm_tis_clkrun_enable (struct tpm_chip *chip,bool value);static void tpm_tis_enable_interrupt (struct tpm_chip *chip,u8 mask)@@ -464,25 +469,31 @@ static int tpm_tis_send_data(struct tpm_chip *chip, const u8 *buf, size_t len)return rc; } -static void disable_interrupts (struct tpm_chip *chip) +static void __disable_interrupts (struct tpm_chip *chip) {struct tpm_tis_data *priv =dev_get_drvdata (&chip->dev);u32 intmask;int rc; -if (priv->irq ==0 ) -return ; - rc =tpm_tis_read32 (priv,TPM_INT_ENABLE (priv->locality), &intmask);if (rc <0 ) intmask =0 ; intmask &= ~TPM_GLOBAL_INT_ENABLE ; rc =tpm_tis_write32 (priv,TPM_INT_ENABLE (priv->locality), intmask); + chip->flags &= ~TPM_CHIP_FLAG_IRQ ; +} + +static void disable_interrupts (struct tpm_chip *chip) +{ +struct tpm_tis_data *priv =dev_get_drvdata (&chip->dev); +if (priv->irq ==0 ) +return ; + +__disable_interrupts (chip);devm_free_irq (chip->dev.parent, priv->irq, chip); priv->irq =0 ; - chip->flags &= ~TPM_CHIP_FLAG_IRQ ; }/* @@ -528,6 +539,12 @@ static int tpm_tis_send(struct tpm_chip *chip, u8 *buf, size_t len)int rc, irq;struct tpm_tis_data *priv =dev_get_drvdata (&chip->dev); +if (unlikely (storm_killed)) { +devm_free_irq (chip->dev.parent, priv->irq, chip); + priv->irq =0 ; + storm_killed = false; + } +if (!(chip->flags &TPM_CHIP_FLAG_IRQ ) || priv->irq_tested)return tpm_tis_send_main (chip, buf, len);@@ -748,6 +765,21 @@ static irqreturn_t tis_int_handler(int dummy, void *dev_id)u32 interrupt;int i, rc; +if (storm_check) { + irqs_fired++; + +if (!time_start) { + time_start =jiffies_to_msecs (jiffies); + }else if ((irqs_fired >1000 ) && (jiffies_to_msecs (jiffies) - jiffies <500 )) { +__disable_interrupts (chip); + storm_check = false; + storm_killed = true; +return IRQ_HANDLED ; + }else if ((jiffies_to_msecs (jiffies) - time_start >500 ) && (irqs_fired <1000 )) { + storm_check = false; + } + } + rc =tpm_tis_read32 (priv,TPM_INT_STATUS (priv->locality), &interrupt);if (rc <0 )return IRQ_NONE ;
Hans de Goede @ 2020-11-19 07:42 MST:
Quoted 36 lines
Hi,
Quoted 33 lines · Jerry Snitselaar
Matthew Garrett @ 2020-10-15 15:39 MST:
Quoted 15 lines · Jerry Snitselaar
Quoted 13 lines
There is a misconfiguration in the bios of the gpio pin used for the interrupt in the T490s. When interrupts are enabled in the
tpm_tisdriver code this results in an interrupt storm. This was initially reported when we attempted to enable the interrupt code in thetpm_tisdriver, which previously wasn't setting a flag to enable it. Due to the reports of the interrupt storm that code was reverted and we went back to polling instead of using interrupts. Now that we know the T490s problem is a firmware issue, add code to check if the system is a T490s and disable interrupts if that is the case. This will allow us to enable interrupts for everyone else. If the user has a fixed bios they can force the enabling of interrupts with tpm_tis.interrupts=1 on the kernel command line.Quoted 10 lines
I think an implication of this is that systems haven't been well-tested with interrupts enabled. In general when we've found a firmware issue in one place it ends up happening elsewhere as well, so it wouldn't surprise me if there are other machines that will also be unhappy with interrupts enabled. Would it be possible to automatically detect this case (eg, if we get more than a certain number of interrupts in a certain timeframe immediately after enabling the interrupt) and automatically fall back to polling in that case? It would also mean that users with fixed firmware wouldn't need to pass a parameter.
I believe Matthew is correct here. I found another system today with completely different vendor for both the system and the tpm chip. In addition another Lenovo model, the L490, has the issue.
Quoted 9 lines
Quoted 5 lines
This initial attempt at a solution like Matthew suggested works on the system I found today, but I imagine it is all sorts of wrong. In the 2 systems where I've seen it, there are about 100000 interrupts in around 1.5 seconds, and then the irq code shuts down the interrupt because they aren't being handled.
Is that with your patch? The IRQ should be silenced as soon as
devm_free_irq(chip->dev.parent, priv->irq, chip); is called.
No that is just with James' patchset that enables interrupts for tpm_tis. It looks like the irq is firing, but the tpm's int_status register is clear, so the handler immediately returns IRQ_NONE. After that happens 100000 times the core irq code shuts down the irq, but it isn't released so I could still see the stats in /proc/interrupts. With my attempt below on top of that patchset it releases the irq. I had to stick the check prior to it checking the int_status register because it is cleared and the handler returns, and I couldn't do the devm_free_irq directly in tis_int_handler, so I tried sticking it in tpm_tis_send where the other odd irq testing code was already located. I'm not sure if there is another place that would work better for calling the devm_free_irq.
Quoted 6 lines
Depending on if we can get your storm-detection to work or not, we might also choose to just never try to use the IRQ (at least on x86 systems). AFAIK the TPM is never used for high-throughput stuff so the polling overhead should not be a big deal (and I'm getting the feeling that Windows always polls).
I was wondering about Windows as well. In addition to the Lenovo systems which the T490s had Nuvoton tpm, the system I found yesterday was a development system we have from a partner with an Infineon tpm. Dan Williams has seen it internally at Intel as well on some system.
Quoted 95 lines
Regards,
Hans
Quoted 89 lines
drivers/char/tpm/tpm_tis_core.c +37
-5@@ -27,6 +27,11 @@#include "tpm.h" #include "tpm_tis_core.h" +static unsigned int time_start =0 ; +static bool storm_check = true; +static bool storm_killed = false; +static u32 irqs_fired =0 ; +static void tpm_tis_clkrun_enable (struct tpm_chip *chip,bool value);static void tpm_tis_enable_interrupt (struct tpm_chip *chip,u8 mask)@@ -464,25 +469,31 @@ static int tpm_tis_send_data(struct tpm_chip *chip, const u8 *buf, size_t len)return rc; } -static void disable_interrupts (struct tpm_chip *chip) +static void __disable_interrupts (struct tpm_chip *chip) {struct tpm_tis_data *priv =dev_get_drvdata (&chip->dev);u32 intmask;int rc; -if (priv->irq ==0 ) -return ; - rc =tpm_tis_read32 (priv,TPM_INT_ENABLE (priv->locality), &intmask);if (rc <0 ) intmask =0 ; intmask &= ~TPM_GLOBAL_INT_ENABLE ; rc =tpm_tis_write32 (priv,TPM_INT_ENABLE (priv->locality), intmask); + chip->flags &= ~TPM_CHIP_FLAG_IRQ ; +} + +static void disable_interrupts (struct tpm_chip *chip) +{ +struct tpm_tis_data *priv =dev_get_drvdata (&chip->dev); +if (priv->irq ==0 ) +return ; + +__disable_interrupts (chip);devm_free_irq (chip->dev.parent, priv->irq, chip); priv->irq =0 ; - chip->flags &= ~TPM_CHIP_FLAG_IRQ ; }/* @@ -528,6 +539,12 @@ static int tpm_tis_send(struct tpm_chip *chip, u8 *buf, size_t len)int rc, irq;struct tpm_tis_data *priv =dev_get_drvdata (&chip->dev); +if (unlikely (storm_killed)) { +devm_free_irq (chip->dev.parent, priv->irq, chip); + priv->irq =0 ; + storm_killed = false; + } +if (!(chip->flags &TPM_CHIP_FLAG_IRQ ) || priv->irq_tested)return tpm_tis_send_main (chip, buf, len);@@ -748,6 +765,21 @@ static irqreturn_t tis_int_handler(int dummy, void *dev_id)u32 interrupt;int i, rc; +if (storm_check) { + irqs_fired++; + +if (!time_start) { + time_start =jiffies_to_msecs (jiffies); + }else if ((irqs_fired >1000 ) && (jiffies_to_msecs (jiffies) - jiffies <500 )) { +__disable_interrupts (chip); + storm_check = false; + storm_killed = true; +return IRQ_HANDLED ; + }else if ((jiffies_to_msecs (jiffies) - time_start >500 ) && (irqs_fired <1000 )) { + storm_check = false; + } + } + rc =tpm_tis_read32 (priv,TPM_INT_STATUS (priv->locality), &interrupt);if (rc <0 )return IRQ_NONE ;
Hi,
Quoted 49 lines · Jerry Snitselaar
Hans de Goede @ 2020-11-19 07:42 MST:
Quoted 36 lines
Hi,
Quoted 33 lines · Jerry Snitselaar
Matthew Garrett @ 2020-10-15 15:39 MST:
Quoted 15 lines · Jerry Snitselaar
Quoted 13 lines
There is a misconfiguration in the bios of the gpio pin used for the interrupt in the T490s. When interrupts are enabled in the
tpm_tisdriver code this results in an interrupt storm. This was initially reported when we attempted to enable the interrupt code in thetpm_tisdriver, which previously wasn't setting a flag to enable it. Due to the reports of the interrupt storm that code was reverted and we went back to polling instead of using interrupts. Now that we know the T490s problem is a firmware issue, add code to check if the system is a T490s and disable interrupts if that is the case. This will allow us to enable interrupts for everyone else. If the user has a fixed bios they can force the enabling of interrupts with tpm_tis.interrupts=1 on the kernel command line.Quoted 10 lines
I think an implication of this is that systems haven't been well-tested with interrupts enabled. In general when we've found a firmware issue in one place it ends up happening elsewhere as well, so it wouldn't surprise me if there are other machines that will also be unhappy with interrupts enabled. Would it be possible to automatically detect this case (eg, if we get more than a certain number of interrupts in a certain timeframe immediately after enabling the interrupt) and automatically fall back to polling in that case? It would also mean that users with fixed firmware wouldn't need to pass a parameter.
I believe Matthew is correct here. I found another system today with completely different vendor for both the system and the tpm chip. In addition another Lenovo model, the L490, has the issue.
Quoted 9 lines
Quoted 5 lines
This initial attempt at a solution like Matthew suggested works on the system I found today, but I imagine it is all sorts of wrong. In the 2 systems where I've seen it, there are about 100000 interrupts in around 1.5 seconds, and then the irq code shuts down the interrupt because they aren't being handled.
Is that with your patch? The IRQ should be silenced as soon as
devm_free_irq(chip->dev.parent, priv->irq, chip); is called.
Quoted 5 lines
No that is just with James' patchset that enables interrupts for
tpm_tis. It looks like the irq is firing, but the tpm'sint_statusregister is clear, so the handler immediately returnsIRQ_NONE. After that happens 100000 times the core irq code shuts down the irq, but it isn't released so I could still see the stats in/proc/interrupts.
I see, yes I have seen this behavior on the X1C8 with a pre-production BIOS.
Quoted 4 lines
With
my attempt below on top of that patchset it releases the irq. I had to stick the check prior to it checking theint_statusregister because it is cleared and the handler returns,
Ack.
Quoted 5 lines
and I couldn't do the
devm_free_irq
directly intis_int_handler, so I tried sticking it intpm_tis_sendwhere the other odd irq testing code was already located. I'm not sure if there is another place that would work better for calling thedevm_free_irq.
Adding it together with the other IRQ testing related code seems fine to me.
Quoted 11 lines
Quoted 6 lines
Depending on if we can get your storm-detection to work or not, we might also choose to just never try to use the IRQ (at least on x86 systems). AFAIK the TPM is never used for high-throughput stuff so the polling overhead should not be a big deal (and I'm getting the feeling that Windows always polls).
I was wondering about Windows as well. In addition to the Lenovo systems which the T490s had Nuvoton tpm, the system I found yesterday was a development system we have from a partner with an Infineon tpm. Dan Williams has seen it internally at Intel as well on some system.
Sounds to me like Windows never uses the IRQ, so we get the fun of finding all the untested firmware bugs.
So if we cannot get this detection code to work reliable, maybe we should just not use the IRQ at all, at least on X86 + ACPI systems?
Anyways lets try this storm-detection thing first, but I have the feeling we should not spend too much time on this. Just outright disabling IRQ support might be better.
REgards,
Hans
Jarkko Sakkinen
maintainerQuoted 51 lines · Hans de Goede
Hi,
Quoted 33 lines · Jerry Snitselaar
Matthew Garrett @ 2020-10-15 15:39 MST:
Quoted 15 lines · Jerry Snitselaar
Quoted 13 lines
There is a misconfiguration in the bios of the gpio pin used for the interrupt in the T490s. When interrupts are enabled in the
tpm_tisdriver code this results in an interrupt storm. This was initially reported when we attempted to enable the interrupt code in thetpm_tisdriver, which previously wasn't setting a flag to enable it. Due to the reports of the interrupt storm that code was reverted and we went back to polling instead of using interrupts. Now that we know the T490s problem is a firmware issue, add code to check if the system is a T490s and disable interrupts if that is the case. This will allow us to enable interrupts for everyone else. If the user has a fixed bios they can force the enabling of interrupts with tpm_tis.interrupts=1 on the kernel command line.Quoted 10 lines
I think an implication of this is that systems haven't been well-tested with interrupts enabled. In general when we've found a firmware issue in one place it ends up happening elsewhere as well, so it wouldn't surprise me if there are other machines that will also be unhappy with interrupts enabled. Would it be possible to automatically detect this case (eg, if we get more than a certain number of interrupts in a certain timeframe immediately after enabling the interrupt) and automatically fall back to polling in that case? It would also mean that users with fixed firmware wouldn't need to pass a parameter.
I believe Matthew is correct here. I found another system today with completely different vendor for both the system and the tpm chip. In addition another Lenovo model, the L490, has the issue.
Quoted 5 lines
This initial attempt at a solution like Matthew suggested works on the system I found today, but I imagine it is all sorts of wrong. In the 2 systems where I've seen it, there are about 100000 interrupts in around 1.5 seconds, and then the irq code shuts down the interrupt because they aren't being handled.
Is that with your patch? The IRQ should be silenced as soon as
devm_free_irq(chip->dev.parent, priv->irq, chip); is called.Depending on if we can get your storm-detection to work or not, we might also choose to just never try to use the IRQ (at least on x86 systems). AFAIK the TPM is never used for high-throughput stuff so the polling overhead should not be a big deal (and I'm getting the feeling that Windows always polls).
Quoted 3 lines
Regards,
Hans
Yeah, this is what I've been wondering for a while. Why could not we just strip off IRQ code? Why does it matter?
/Jarkko
Jarkko Sakkinen
maintainerQuoted 52 lines · Jarkko Sakkinen
Quoted 51 lines · Hans de Goede
Hi,
Quoted 33 lines · Jerry Snitselaar
Matthew Garrett @ 2020-10-15 15:39 MST:
Quoted 15 lines · Jerry Snitselaar
Quoted 13 lines
There is a misconfiguration in the bios of the gpio pin used for the interrupt in the T490s. When interrupts are enabled in the
tpm_tisdriver code this results in an interrupt storm. This was initially reported when we attempted to enable the interrupt code in thetpm_tisdriver, which previously wasn't setting a flag to enable it. Due to the reports of the interrupt storm that code was reverted and we went back to polling instead of using interrupts. Now that we know the T490s problem is a firmware issue, add code to check if the system is a T490s and disable interrupts if that is the case. This will allow us to enable interrupts for everyone else. If the user has a fixed bios they can force the enabling of interrupts with tpm_tis.interrupts=1 on the kernel command line.Quoted 10 lines
I think an implication of this is that systems haven't been well-tested with interrupts enabled. In general when we've found a firmware issue in one place it ends up happening elsewhere as well, so it wouldn't surprise me if there are other machines that will also be unhappy with interrupts enabled. Would it be possible to automatically detect this case (eg, if we get more than a certain number of interrupts in a certain timeframe immediately after enabling the interrupt) and automatically fall back to polling in that case? It would also mean that users with fixed firmware wouldn't need to pass a parameter.
I believe Matthew is correct here. I found another system today with completely different vendor for both the system and the tpm chip. In addition another Lenovo model, the L490, has the issue.
Quoted 5 lines
This initial attempt at a solution like Matthew suggested works on the system I found today, but I imagine it is all sorts of wrong. In the 2 systems where I've seen it, there are about 100000 interrupts in around 1.5 seconds, and then the irq code shuts down the interrupt because they aren't being handled.
Is that with your patch? The IRQ should be silenced as soon as
devm_free_irq(chip->dev.parent, priv->irq, chip); is called.Depending on if we can get your storm-detection to work or not, we might also choose to just never try to use the IRQ (at least on x86 systems). AFAIK the TPM is never used for high-throughput stuff so the polling overhead should not be a big deal (and I'm getting the feeling that Windows always polls).
Quoted 6 lines
Quoted 3 lines
Regards,
Hans
Yeah, this is what I've been wondering for a while. Why could not we just strip off IRQ code? Why does it matter?
And we DO NOT use interrupts in tpm_crb and nobody has ever complained.
/Jarkko
Jarkko Sakkinen
maintainerQuoted 45 lines · Jerry Snitselaar
Matthew Garrett @ 2020-10-15 15:39 MST:
Quoted 15 lines · Jerry Snitselaar
Quoted 13 lines
There is a misconfiguration in the bios of the gpio pin used for the interrupt in the T490s. When interrupts are enabled in the
tpm_tisdriver code this results in an interrupt storm. This was initially reported when we attempted to enable the interrupt code in thetpm_tisdriver, which previously wasn't setting a flag to enable it. Due to the reports of the interrupt storm that code was reverted and we went back to polling instead of using interrupts. Now that we know the T490s problem is a firmware issue, add code to check if the system is a T490s and disable interrupts if that is the case. This will allow us to enable interrupts for everyone else. If the user has a fixed bios they can force the enabling of interrupts with tpm_tis.interrupts=1 on the kernel command line.Quoted 10 lines
I think an implication of this is that systems haven't been well-tested with interrupts enabled. In general when we've found a firmware issue in one place it ends up happening elsewhere as well, so it wouldn't surprise me if there are other machines that will also be unhappy with interrupts enabled. Would it be possible to automatically detect this case (eg, if we get more than a certain number of interrupts in a certain timeframe immediately after enabling the interrupt) and automatically fall back to polling in that case? It would also mean that users with fixed firmware wouldn't need to pass a parameter.
I believe Matthew is correct here. I found another system today with completely different vendor for both the system and the tpm chip. In addition another Lenovo model, the L490, has the issue.
This initial attempt at a solution like Matthew suggested works on the system I found today, but I imagine it is all sorts of wrong. In the 2 systems where I've seen it, there are about 100000 interrupts in around 1.5 seconds, and then the irq code shuts down the interrupt because they aren't being handled.
drivers/char/tpm/tpm_tis_core.c +0
-0
drivers/char/tpm/tpm_tis_core.c line 33
Quoted 7 lines
drivers/char/tpm/tpm_tis_core.c +4
-0@@ -27,6 +27,11 @@#include "tpm.h" #include "tpm_tis_core.h" +static unsigned int time_start =0 ; +static bool storm_check = true; +static bool storm_killed = false; +static u32 irqs_fired =0 ;
Maybe kstat_irqs() would be a better idea than ad hoc stats.
Quoted 42 lines
drivers/char/tpm/tpm_tis_core.c +12
-5+static void tpm_tis_clkrun_enable (struct tpm_chip *chip,bool value);static void tpm_tis_enable_interrupt (struct tpm_chip *chip,u8 mask)@@ -464,25 +469,31 @@ static int tpm_tis_send_data(struct tpm_chip *chip, const u8 *buf, size_t len)return rc; } -static void disable_interrupts (struct tpm_chip *chip) +static void __disable_interrupts (struct tpm_chip *chip) {struct tpm_tis_data *priv =dev_get_drvdata (&chip->dev);u32 intmask;int rc; -if (priv->irq ==0 ) -return ; - rc =tpm_tis_read32 (priv,TPM_INT_ENABLE (priv->locality), &intmask);if (rc <0 ) intmask =0 ; intmask &= ~TPM_GLOBAL_INT_ENABLE ; rc =tpm_tis_write32 (priv,TPM_INT_ENABLE (priv->locality), intmask); + chip->flags &= ~TPM_CHIP_FLAG_IRQ ; +} + +static void disable_interrupts (struct tpm_chip *chip) +{ +struct tpm_tis_data *priv =dev_get_drvdata (&chip->dev); +if (priv->irq ==0 ) +return ; + +__disable_interrupts (chip);devm_free_irq (chip->dev.parent, priv->irq, chip); priv->irq =0 ; - chip->flags &= ~TPM_CHIP_FLAG_IRQ ; }/*
drivers/char/tpm/tpm_tis_core.c static int tpm_tis_send(struct tpm_chip *chip, u8 *buf, size_t len) · line 546
Quoted 8 lines
drivers/char/tpm/tpm_tis_core.c +5
-0@@ -528,6 +539,12 @@ static int tpm_tis_send(struct tpm_chip *chip, u8 *buf, size_t len)int rc, irq;struct tpm_tis_data *priv =dev_get_drvdata (&chip->dev); +if (unlikely (storm_killed)) { +devm_free_irq (chip->dev.parent, priv->irq, chip); + priv->irq =0 ; + storm_killed = false; + }
OK this kind of bad solution because if tpm_tis_send() is not called, then IRQ is never freed. AFAIK, devres_* do not sleep but use spin lock, i.e. you could render out both storm_check and storm_killed.
Quoted 18 lines
drivers/char/tpm/tpm_tis_core.c +9
-0+if (!(chip->flags &TPM_CHIP_FLAG_IRQ ) || priv->irq_tested)return tpm_tis_send_main (chip, buf, len);@@ -748,6 +765,21 @@ static irqreturn_t tis_int_handler(int dummy, void *dev_id)u32 interrupt;int i, rc; +if (storm_check) { + irqs_fired++; + +if (!time_start) { + time_start =jiffies_to_msecs (jiffies); + }else if ((irqs_fired >1000 ) && (jiffies_to_msecs (jiffies) - jiffies <500 )) { +__disable_interrupts (chip); + storm_check = false;
drivers/char/tpm/tpm_tis_core.c static irqreturn_t tis_int_handler(int dummy, void *dev_id) · line 785
Quoted 10 lines
drivers/char/tpm/tpm_tis_core.c +7
-0+ storm_killed = true; +return IRQ_HANDLED ; + }else if ((jiffies_to_msecs (jiffies) - time_start >500 ) && (irqs_fired <1000 )) { + storm_check = false; + } + } + rc =tpm_tis_read32 (priv,TPM_INT_STATUS (priv->locality), &interrupt);if (rc <0 )return IRQ_NONE ;
/Jarkko
Jarkko Sakkinen @ 2020-11-23 20:26 MST:
Quoted 49 lines · Jerry Snitselaar
Quoted 48 lines
Matthew Garrett @ 2020-10-15 15:39 MST:
Quoted 15 lines · Jerry Snitselaar
Quoted 13 lines
There is a misconfiguration in the bios of the gpio pin used for the interrupt in the T490s. When interrupts are enabled in the
tpm_tisdriver code this results in an interrupt storm. This was initially reported when we attempted to enable the interrupt code in thetpm_tisdriver, which previously wasn't setting a flag to enable it. Due to the reports of the interrupt storm that code was reverted and we went back to polling instead of using interrupts. Now that we know the T490s problem is a firmware issue, add code to check if the system is a T490s and disable interrupts if that is the case. This will allow us to enable interrupts for everyone else. If the user has a fixed bios they can force the enabling of interrupts with tpm_tis.interrupts=1 on the kernel command line.Quoted 10 lines
I think an implication of this is that systems haven't been well-tested with interrupts enabled. In general when we've found a firmware issue in one place it ends up happening elsewhere as well, so it wouldn't surprise me if there are other machines that will also be unhappy with interrupts enabled. Would it be possible to automatically detect this case (eg, if we get more than a certain number of interrupts in a certain timeframe immediately after enabling the interrupt) and automatically fall back to polling in that case? It would also mean that users with fixed firmware wouldn't need to pass a parameter.
I believe Matthew is correct here. I found another system today with completely different vendor for both the system and the tpm chip. In addition another Lenovo model, the L490, has the issue.
This initial attempt at a solution like Matthew suggested works on the system I found today, but I imagine it is all sorts of wrong. In the 2 systems where I've seen it, there are about 100000 interrupts in around 1.5 seconds, and then the irq code shuts down the interrupt because they aren't being handled.
drivers/char/tpm/tpm_tis_core.c +0
-0@@ -27,6 +27,11 @@#include "tpm.h" #include "tpm_tis_core.h"
Quoted 7 lines
drivers/char/tpm/tpm_tis_core.c line 33
Quoted 4 lines
drivers/char/tpm/tpm_tis_core.c +4
-0+static unsigned int time_start =0 ; +static bool storm_check = true; +static bool storm_killed = false; +static u32 irqs_fired =0 ;Maybe
kstat_irqs()would be a better idea than ad hoc stats.
Thanks, yes that would be better.
Quoted 45 lines
Quoted 45 lines
drivers/char/tpm/tpm_tis_core.c +12
-5+static void tpm_tis_clkrun_enable (struct tpm_chip *chip,bool value);static void tpm_tis_enable_interrupt (struct tpm_chip *chip,u8 mask)@@ -464,25 +469,31 @@ static int tpm_tis_send_data(struct tpm_chip *chip, const u8 *buf, size_t len)return rc; } -static void disable_interrupts (struct tpm_chip *chip) +static void __disable_interrupts (struct tpm_chip *chip) {struct tpm_tis_data *priv =dev_get_drvdata (&chip->dev);u32 intmask;int rc; -if (priv->irq ==0 ) -return ; - rc =tpm_tis_read32 (priv,TPM_INT_ENABLE (priv->locality), &intmask);if (rc <0 ) intmask =0 ; intmask &= ~TPM_GLOBAL_INT_ENABLE ; rc =tpm_tis_write32 (priv,TPM_INT_ENABLE (priv->locality), intmask); + chip->flags &= ~TPM_CHIP_FLAG_IRQ ; +} + +static void disable_interrupts (struct tpm_chip *chip) +{ +struct tpm_tis_data *priv =dev_get_drvdata (&chip->dev); +if (priv->irq ==0 ) +return ; + +__disable_interrupts (chip);devm_free_irq (chip->dev.parent, priv->irq, chip); priv->irq =0 ; - chip->flags &= ~TPM_CHIP_FLAG_IRQ ; }/* @@ -528,6 +539,12 @@ static int tpm_tis_send(struct tpm_chip *chip, u8 *buf, size_t len)int rc, irq;struct tpm_tis_data *priv =dev_get_drvdata (&chip->dev);
Quoted 10 lines
drivers/char/tpm/tpm_tis_core.c static int tpm_tis_send(struct tpm_chip *chip, u8 *buf, size_t len) · line 546
Quoted 5 lines
drivers/char/tpm/tpm_tis_core.c +5
-0+if (unlikely (storm_killed)) { +devm_free_irq (chip->dev.parent, priv->irq, chip); + priv->irq =0 ; + storm_killed = false; + }OK this kind of bad solution because if
tpm_tis_send()is not called, then IRQ is never freed. AFAIK, devres_* do not sleep but use spin lock, i.e. you could render out bothstorm_checkandstorm_killed.
Is there a way to flag it for freeing later while in an interrupt context? I'm not sure where to clean it up since devm_free_irq can't be called in tis_int_handler.
Before diving further into that though, does anyone else have an opinion on ripping out the irq code, and just using polling? We've been only polling since 2015 anyways.
Quoted 30 lines
Quoted 18 lines
drivers/char/tpm/tpm_tis_core.c +9
-0+if (!(chip->flags &TPM_CHIP_FLAG_IRQ ) || priv->irq_tested)return tpm_tis_send_main (chip, buf, len);@@ -748,6 +765,21 @@ static irqreturn_t tis_int_handler(int dummy, void *dev_id)u32 interrupt;int i, rc; +if (storm_check) { + irqs_fired++; + +if (!time_start) { + time_start =jiffies_to_msecs (jiffies); + }else if ((irqs_fired >1000 ) && (jiffies_to_msecs (jiffies) - jiffies <500 )) { +__disable_interrupts (chip); + storm_check = false;drivers/char/tpm/tpm_tis_core.c static irqreturn_t tis_int_handler(int dummy, void *dev_id) · line 785
Quoted 10 lines
drivers/char/tpm/tpm_tis_core.c +7
-0+ storm_killed = true; +return IRQ_HANDLED ; + }else if ((jiffies_to_msecs (jiffies) - time_start >500 ) && (irqs_fired <1000 )) { + storm_check = false; + } + } + rc =tpm_tis_read32 (priv,TPM_INT_STATUS (priv->locality), &interrupt);if (rc <0 )return IRQ_NONE ;/Jarkko
James Bottomley
maintainerQuoted 3 lines · Jerry Snitselaar
Before diving further into that though, does anyone else have an opinion on ripping out the irq code, and just using polling? We've been only polling since 2015 anyways.
Well only a biased one, obviously: polling causes large amounts of busy waiting, which is a waste of CPU resources and does increase the time it takes us to do TPM operations ... not a concern if you're doing long computation ones, like signatures, but it is a problem for short operations like bulk updates of PCRs. The other potential issue, as we saw with atmel is that if you prod the chip too often (which you have to do with polling) you risk upsetting it. We've spent ages trying to tune the polling parameters to balance reduction of busy wait with chip upset and still, apparently, not quite got it right. If the TPM has a functioning IRQ then it gets us out of the whole polling mess entirely. The big question is how many chips that report an IRQ actually have a malfunctioning one?
James
Jarkko Sakkinen
maintainerQuoted 18 lines · James Bottomley
Quoted 3 lines · Jerry Snitselaar
Before diving further into that though, does anyone else have an opinion on ripping out the irq code, and just using polling? We've been only polling since 2015 anyways.
Well only a biased one, obviously: polling causes large amounts of busy waiting, which is a waste of CPU resources and does increase the time it takes us to do TPM operations ... not a concern if you're doing long computation ones, like signatures, but it is a problem for short operations like bulk updates of PCRs. The other potential issue, as we saw with atmel is that if you prod the chip too often (which you have to do with polling) you risk upsetting it. We've spent ages trying to tune the polling parameters to balance reduction of busy wait with chip upset and still, apparently, not quite got it right. If the TPM has a functioning IRQ then it gets us out of the whole polling mess entirely. The big question is how many chips that report an IRQ actually have a malfunctioning one?
Quoted 1 line
James
Do we have a way to know is Windows TPM code using IRQ's?
/Jarkko
Hi,
Quoted 118 lines · Jerry Snitselaar
Jarkko Sakkinen @ 2020-11-23 20:26 MST:
Quoted 49 lines · Jerry Snitselaar
Quoted 48 lines
Matthew Garrett @ 2020-10-15 15:39 MST:
Quoted 15 lines · Jerry Snitselaar
Quoted 13 lines
There is a misconfiguration in the bios of the gpio pin used for the interrupt in the T490s. When interrupts are enabled in the
tpm_tisdriver code this results in an interrupt storm. This was initially reported when we attempted to enable the interrupt code in thetpm_tisdriver, which previously wasn't setting a flag to enable it. Due to the reports of the interrupt storm that code was reverted and we went back to polling instead of using interrupts. Now that we know the T490s problem is a firmware issue, add code to check if the system is a T490s and disable interrupts if that is the case. This will allow us to enable interrupts for everyone else. If the user has a fixed bios they can force the enabling of interrupts with tpm_tis.interrupts=1 on the kernel command line.Quoted 10 lines
I think an implication of this is that systems haven't been well-tested with interrupts enabled. In general when we've found a firmware issue in one place it ends up happening elsewhere as well, so it wouldn't surprise me if there are other machines that will also be unhappy with interrupts enabled. Would it be possible to automatically detect this case (eg, if we get more than a certain number of interrupts in a certain timeframe immediately after enabling the interrupt) and automatically fall back to polling in that case? It would also mean that users with fixed firmware wouldn't need to pass a parameter.
I believe Matthew is correct here. I found another system today with completely different vendor for both the system and the tpm chip. In addition another Lenovo model, the L490, has the issue.
This initial attempt at a solution like Matthew suggested works on the system I found today, but I imagine it is all sorts of wrong. In the 2 systems where I've seen it, there are about 100000 interrupts in around 1.5 seconds, and then the irq code shuts down the interrupt because they aren't being handled.
drivers/char/tpm/tpm_tis_core.c +0
-0@@ -27,6 +27,11 @@#include "tpm.h" #include "tpm_tis_core.h" Quoted 7 lines
drivers/char/tpm/tpm_tis_core.c line 33
Quoted 4 lines
drivers/char/tpm/tpm_tis_core.c +4
-0+static unsigned int time_start =0 ; +static bool storm_check = true; +static bool storm_killed = false; +static u32 irqs_fired =0 ;Maybe
kstat_irqs()would be a better idea than ad hoc stats.Thanks, yes that would be better.
Quoted 45 lines
Quoted 45 lines
drivers/char/tpm/tpm_tis_core.c +12
-5+static void tpm_tis_clkrun_enable (struct tpm_chip *chip,bool value);static void tpm_tis_enable_interrupt (struct tpm_chip *chip,u8 mask)@@ -464,25 +469,31 @@ static int tpm_tis_send_data(struct tpm_chip *chip, const u8 *buf, size_t len)return rc; } -static void disable_interrupts (struct tpm_chip *chip) +static void __disable_interrupts (struct tpm_chip *chip) {struct tpm_tis_data *priv =dev_get_drvdata (&chip->dev);u32 intmask;int rc; -if (priv->irq ==0 ) -return ; - rc =tpm_tis_read32 (priv,TPM_INT_ENABLE (priv->locality), &intmask);if (rc <0 ) intmask =0 ; intmask &= ~TPM_GLOBAL_INT_ENABLE ; rc =tpm_tis_write32 (priv,TPM_INT_ENABLE (priv->locality), intmask); + chip->flags &= ~TPM_CHIP_FLAG_IRQ ; +} + +static void disable_interrupts (struct tpm_chip *chip) +{ +struct tpm_tis_data *priv =dev_get_drvdata (&chip->dev); +if (priv->irq ==0 ) +return ; + +__disable_interrupts (chip);devm_free_irq (chip->dev.parent, priv->irq, chip); priv->irq =0 ; - chip->flags &= ~TPM_CHIP_FLAG_IRQ ; }/* @@ -528,6 +539,12 @@ static int tpm_tis_send(struct tpm_chip *chip, u8 *buf, size_t len)int rc, irq;struct tpm_tis_data *priv =dev_get_drvdata (&chip->dev);Quoted 10 lines
drivers/char/tpm/tpm_tis_core.c static int tpm_tis_send(struct tpm_chip *chip, u8 *buf, size_t len) · line 546
Quoted 5 lines
drivers/char/tpm/tpm_tis_core.c +5
-0+if (unlikely (storm_killed)) { +devm_free_irq (chip->dev.parent, priv->irq, chip); + priv->irq =0 ; + storm_killed = false; + }OK this kind of bad solution because if
tpm_tis_send()is not called, then IRQ is never freed. AFAIK, devres_* do not sleep but use spin lock, i.e. you could render out bothstorm_checkandstorm_killed.
Quoted 3 lines
Is there a way to flag it for freeing later while in an interrupt context? I'm not sure where to clean it up since
devm_free_irqcan't be called intis_int_handler.
You could add a workqueue work-struct just for this and queue that up to do the free when you detect the storm. That will then run pretty much immediately, avoiding the storm going on for (much) longer.
Quoted 3 lines
Before diving further into that though, does anyone else have an opinion on ripping out the irq code, and just using polling? We've been only polling since 2015 anyways.
Given James Bottomley's reply I guess it would be worthwhile to get the storm detection to work.
Regards,
Hans
Quoted 32 lines
Quoted 31 lines
Quoted 19 lines
drivers/char/tpm/tpm_tis_core.c +9
-0+if (!(chip->flags &TPM_CHIP_FLAG_IRQ ) || priv->irq_tested)return tpm_tis_send_main (chip, buf, len);@@ -748,6 +765,21 @@ static irqreturn_t tis_int_handler(int dummy, void *dev_id)u32 interrupt;int i, rc; +if (storm_check) { + irqs_fired++; + +if (!time_start) { + time_start =jiffies_to_msecs (jiffies); + }else if ((irqs_fired >1000 ) && (jiffies_to_msecs (jiffies) - jiffies <500 )) { +__disable_interrupts (chip); + storm_check = false;drivers/char/tpm/tpm_tis_core.c static irqreturn_t tis_int_handler(int dummy, void *dev_id) · line 785
Quoted 10 lines
drivers/char/tpm/tpm_tis_core.c +7
-0+ storm_killed = true; +return IRQ_HANDLED ; + }else if ((jiffies_to_msecs (jiffies) - time_start >500 ) && (irqs_fired <1000 )) { + storm_check = false; + } + } + rc =tpm_tis_read32 (priv,TPM_INT_STATUS (priv->locality), &interrupt);if (rc <0 )return IRQ_NONE ;/Jarkko
Jarkko Sakkinen
maintainerQuoted 121 lines · Hans de Goede
Hi,
Quoted 118 lines · Jerry Snitselaar
Jarkko Sakkinen @ 2020-11-23 20:26 MST:
Quoted 49 lines · Jerry Snitselaar
Quoted 48 lines
Matthew Garrett @ 2020-10-15 15:39 MST:
Quoted 15 lines · Jerry Snitselaar
Quoted 13 lines
There is a misconfiguration in the bios of the gpio pin used for the interrupt in the T490s. When interrupts are enabled in the
tpm_tisdriver code this results in an interrupt storm. This was initially reported when we attempted to enable the interrupt code in thetpm_tisdriver, which previously wasn't setting a flag to enable it. Due to the reports of the interrupt storm that code was reverted and we went back to polling instead of using interrupts. Now that we know the T490s problem is a firmware issue, add code to check if the system is a T490s and disable interrupts if that is the case. This will allow us to enable interrupts for everyone else. If the user has a fixed bios they can force the enabling of interrupts with tpm_tis.interrupts=1 on the kernel command line.Quoted 10 lines
I think an implication of this is that systems haven't been well-tested with interrupts enabled. In general when we've found a firmware issue in one place it ends up happening elsewhere as well, so it wouldn't surprise me if there are other machines that will also be unhappy with interrupts enabled. Would it be possible to automatically detect this case (eg, if we get more than a certain number of interrupts in a certain timeframe immediately after enabling the interrupt) and automatically fall back to polling in that case? It would also mean that users with fixed firmware wouldn't need to pass a parameter.
I believe Matthew is correct here. I found another system today with completely different vendor for both the system and the tpm chip. In addition another Lenovo model, the L490, has the issue.
This initial attempt at a solution like Matthew suggested works on the system I found today, but I imagine it is all sorts of wrong. In the 2 systems where I've seen it, there are about 100000 interrupts in around 1.5 seconds, and then the irq code shuts down the interrupt because they aren't being handled.
drivers/char/tpm/tpm_tis_core.c +0
-0@@ -27,6 +27,11 @@#include "tpm.h" #include "tpm_tis_core.h" Quoted 7 lines
drivers/char/tpm/tpm_tis_core.c line 33
Quoted 4 lines
drivers/char/tpm/tpm_tis_core.c +4
-0+static unsigned int time_start =0 ; +static bool storm_check = true; +static bool storm_killed = false; +static u32 irqs_fired =0 ;Maybe
kstat_irqs()would be a better idea than ad hoc stats.Thanks, yes that would be better.
Quoted 55 lines
Quoted 42 lines
drivers/char/tpm/tpm_tis_core.c +12
-5+static void tpm_tis_clkrun_enable (struct tpm_chip *chip,bool value);static void tpm_tis_enable_interrupt (struct tpm_chip *chip,u8 mask)@@ -464,25 +469,31 @@ static int tpm_tis_send_data(struct tpm_chip *chip, const u8 *buf, size_t len)return rc; } -static void disable_interrupts (struct tpm_chip *chip) +static void __disable_interrupts (struct tpm_chip *chip) {struct tpm_tis_data *priv =dev_get_drvdata (&chip->dev);u32 intmask;int rc; -if (priv->irq ==0 ) -return ; - rc =tpm_tis_read32 (priv,TPM_INT_ENABLE (priv->locality), &intmask);if (rc <0 ) intmask =0 ; intmask &= ~TPM_GLOBAL_INT_ENABLE ; rc =tpm_tis_write32 (priv,TPM_INT_ENABLE (priv->locality), intmask); + chip->flags &= ~TPM_CHIP_FLAG_IRQ ; +} + +static void disable_interrupts (struct tpm_chip *chip) +{ +struct tpm_tis_data *priv =dev_get_drvdata (&chip->dev); +if (priv->irq ==0 ) +return ; + +__disable_interrupts (chip);devm_free_irq (chip->dev.parent, priv->irq, chip); priv->irq =0 ; - chip->flags &= ~TPM_CHIP_FLAG_IRQ ; }/* drivers/char/tpm/tpm_tis_core.c static int tpm_tis_send(struct tpm_chip *chip, u8 *buf, size_t len) · line 546
Quoted 8 lines
drivers/char/tpm/tpm_tis_core.c +5
-0@@ -528,6 +539,12 @@ static int tpm_tis_send(struct tpm_chip *chip, u8 *buf, size_t len)int rc, irq;struct tpm_tis_data *priv =dev_get_drvdata (&chip->dev); +if (unlikely (storm_killed)) { +devm_free_irq (chip->dev.parent, priv->irq, chip); + priv->irq =0 ; + storm_killed = false; + }OK this kind of bad solution because if
tpm_tis_send()is not called, then IRQ is never freed. AFAIK, devres_* do not sleep but use spin lock, i.e. you could render out bothstorm_checkandstorm_killed.
Quoted 7 lines
Quoted 3 lines
Is there a way to flag it for freeing later while in an interrupt context? I'm not sure where to clean it up since
devm_free_irqcan't be called intis_int_handler.You could add a workqueue work-struct just for this and queue that up to do the free when you detect the storm. That will then run pretty much immediately, avoiding the storm going on for (much) longer.
That's sounds feasible.
Quoted 6 lines
Quoted 3 lines
Before diving further into that though, does anyone else have an opinion on ripping out the irq code, and just using polling? We've been only polling since 2015 anyways.
Given James Bottomley's reply I guess it would be worthwhile to get the storm detection to work.
OK, agreed. I take my words back from a response few minutes ago :-)
Quoted 3 lines
Regards,
Hans
/Jarkko
Hi All,
Quoted 132 lines · Jarkko Sakkinen
Quoted 121 lines · Hans de Goede
Hi,
Quoted 118 lines · Jerry Snitselaar
Jarkko Sakkinen @ 2020-11-23 20:26 MST:
Quoted 49 lines · Jerry Snitselaar
Quoted 48 lines
Matthew Garrett @ 2020-10-15 15:39 MST:
Quoted 15 lines · Jerry Snitselaar
Quoted 13 lines
There is a misconfiguration in the bios of the gpio pin used for the interrupt in the T490s. When interrupts are enabled in the
tpm_tisdriver code this results in an interrupt storm. This was initially reported when we attempted to enable the interrupt code in thetpm_tisdriver, which previously wasn't setting a flag to enable it. Due to the reports of the interrupt storm that code was reverted and we went back to polling instead of using interrupts. Now that we know the T490s problem is a firmware issue, add code to check if the system is a T490s and disable interrupts if that is the case. This will allow us to enable interrupts for everyone else. If the user has a fixed bios they can force the enabling of interrupts with tpm_tis.interrupts=1 on the kernel command line.Quoted 10 lines
I think an implication of this is that systems haven't been well-tested with interrupts enabled. In general when we've found a firmware issue in one place it ends up happening elsewhere as well, so it wouldn't surprise me if there are other machines that will also be unhappy with interrupts enabled. Would it be possible to automatically detect this case (eg, if we get more than a certain number of interrupts in a certain timeframe immediately after enabling the interrupt) and automatically fall back to polling in that case? It would also mean that users with fixed firmware wouldn't need to pass a parameter.
I believe Matthew is correct here. I found another system today with completely different vendor for both the system and the tpm chip. In addition another Lenovo model, the L490, has the issue.
This initial attempt at a solution like Matthew suggested works on the system I found today, but I imagine it is all sorts of wrong. In the 2 systems where I've seen it, there are about 100000 interrupts in around 1.5 seconds, and then the irq code shuts down the interrupt because they aren't being handled.
drivers/char/tpm/tpm_tis_core.c +0
-0@@ -27,6 +27,11 @@#include "tpm.h" #include "tpm_tis_core.h" Quoted 7 lines
drivers/char/tpm/tpm_tis_core.c line 33
Quoted 4 lines
drivers/char/tpm/tpm_tis_core.c +4
-0+static unsigned int time_start =0 ; +static bool storm_check = true; +static bool storm_killed = false; +static u32 irqs_fired =0 ;Maybe
kstat_irqs()would be a better idea than ad hoc stats.Thanks, yes that would be better.
Quoted 55 lines
Quoted 42 lines
drivers/char/tpm/tpm_tis_core.c +12
-5+static void tpm_tis_clkrun_enable (struct tpm_chip *chip,bool value);static void tpm_tis_enable_interrupt (struct tpm_chip *chip,u8 mask)@@ -464,25 +469,31 @@ static int tpm_tis_send_data(struct tpm_chip *chip, const u8 *buf, size_t len)return rc; } -static void disable_interrupts (struct tpm_chip *chip) +static void __disable_interrupts (struct tpm_chip *chip) {struct tpm_tis_data *priv =dev_get_drvdata (&chip->dev);u32 intmask;int rc; -if (priv->irq ==0 ) -return ; - rc =tpm_tis_read32 (priv,TPM_INT_ENABLE (priv->locality), &intmask);if (rc <0 ) intmask =0 ; intmask &= ~TPM_GLOBAL_INT_ENABLE ; rc =tpm_tis_write32 (priv,TPM_INT_ENABLE (priv->locality), intmask); + chip->flags &= ~TPM_CHIP_FLAG_IRQ ; +} + +static void disable_interrupts (struct tpm_chip *chip) +{ +struct tpm_tis_data *priv =dev_get_drvdata (&chip->dev); +if (priv->irq ==0 ) +return ; + +__disable_interrupts (chip);devm_free_irq (chip->dev.parent, priv->irq, chip); priv->irq =0 ; - chip->flags &= ~TPM_CHIP_FLAG_IRQ ; }/* drivers/char/tpm/tpm_tis_core.c static int tpm_tis_send(struct tpm_chip *chip, u8 *buf, size_t len) · line 546
Quoted 8 lines
drivers/char/tpm/tpm_tis_core.c +5
-0@@ -528,6 +539,12 @@ static int tpm_tis_send(struct tpm_chip *chip, u8 *buf, size_t len)int rc, irq;struct tpm_tis_data *priv =dev_get_drvdata (&chip->dev); +if (unlikely (storm_killed)) { +devm_free_irq (chip->dev.parent, priv->irq, chip); + priv->irq =0 ; + storm_killed = false; + }OK this kind of bad solution because if
tpm_tis_send()is not called, then IRQ is never freed. AFAIK, devres_* do not sleep but use spin lock, i.e. you could render out bothstorm_checkandstorm_killed.Quoted 7 lines
Quoted 3 lines
Is there a way to flag it for freeing later while in an interrupt context? I'm not sure where to clean it up since
devm_free_irqcan't be called intis_int_handler.You could add a workqueue work-struct just for this and queue that up to do the free when you detect the storm. That will then run pretty much immediately, avoiding the storm going on for (much) longer.
That's sounds feasible.
Quoted 8 lines
Quoted 6 lines
Quoted 3 lines
Before diving further into that though, does anyone else have an opinion on ripping out the irq code, and just using polling? We've been only polling since 2015 anyways.
Given James Bottomley's reply I guess it would be worthwhile to get the storm detection to work.
OK, agreed. I take my words back from a response few minutes ago :-)
:)
To be clear, I think we should give the storm detection a go. Especially given the problems which James has seen with polling on some TPMs.
But if that turns out to not be feasible I agree we should just either disable IRQs by default on standard x86 platforms, or just remove the IRQ support all together.
Regards,
Hans
Jarkko Sakkinen
maintainerQuoted 144 lines · Hans de Goede
Hi All,
Quoted 132 lines · Jarkko Sakkinen
Quoted 121 lines · Hans de Goede
Hi,
Quoted 118 lines · Jerry Snitselaar
Jarkko Sakkinen @ 2020-11-23 20:26 MST:
Quoted 49 lines · Jerry Snitselaar
Quoted 48 lines
Matthew Garrett @ 2020-10-15 15:39 MST:
Quoted 15 lines · Jerry Snitselaar
Quoted 13 lines
There is a misconfiguration in the bios of the gpio pin used for the interrupt in the T490s. When interrupts are enabled in the
tpm_tisdriver code this results in an interrupt storm. This was initially reported when we attempted to enable the interrupt code in thetpm_tisdriver, which previously wasn't setting a flag to enable it. Due to the reports of the interrupt storm that code was reverted and we went back to polling instead of using interrupts. Now that we know the T490s problem is a firmware issue, add code to check if the system is a T490s and disable interrupts if that is the case. This will allow us to enable interrupts for everyone else. If the user has a fixed bios they can force the enabling of interrupts with tpm_tis.interrupts=1 on the kernel command line.Quoted 10 lines
I think an implication of this is that systems haven't been well-tested with interrupts enabled. In general when we've found a firmware issue in one place it ends up happening elsewhere as well, so it wouldn't surprise me if there are other machines that will also be unhappy with interrupts enabled. Would it be possible to automatically detect this case (eg, if we get more than a certain number of interrupts in a certain timeframe immediately after enabling the interrupt) and automatically fall back to polling in that case? It would also mean that users with fixed firmware wouldn't need to pass a parameter.
I believe Matthew is correct here. I found another system today with completely different vendor for both the system and the tpm chip. In addition another Lenovo model, the L490, has the issue.
This initial attempt at a solution like Matthew suggested works on the system I found today, but I imagine it is all sorts of wrong. In the 2 systems where I've seen it, there are about 100000 interrupts in around 1.5 seconds, and then the irq code shuts down the interrupt because they aren't being handled.
drivers/char/tpm/tpm_tis_core.c +0
-0@@ -27,6 +27,11 @@#include "tpm.h" #include "tpm_tis_core.h" Quoted 7 lines
drivers/char/tpm/tpm_tis_core.c line 33
Quoted 4 lines
drivers/char/tpm/tpm_tis_core.c +4
-0+static unsigned int time_start =0 ; +static bool storm_check = true; +static bool storm_killed = false; +static u32 irqs_fired =0 ;Maybe
kstat_irqs()would be a better idea than ad hoc stats.Thanks, yes that would be better.
Quoted 55 lines
Quoted 42 lines
drivers/char/tpm/tpm_tis_core.c +12
-5+static void tpm_tis_clkrun_enable (struct tpm_chip *chip,bool value);static void tpm_tis_enable_interrupt (struct tpm_chip *chip,u8 mask)@@ -464,25 +469,31 @@ static int tpm_tis_send_data(struct tpm_chip *chip, const u8 *buf, size_t len)return rc; } -static void disable_interrupts (struct tpm_chip *chip) +static void __disable_interrupts (struct tpm_chip *chip) {struct tpm_tis_data *priv =dev_get_drvdata (&chip->dev);u32 intmask;int rc; -if (priv->irq ==0 ) -return ; - rc =tpm_tis_read32 (priv,TPM_INT_ENABLE (priv->locality), &intmask);if (rc <0 ) intmask =0 ; intmask &= ~TPM_GLOBAL_INT_ENABLE ; rc =tpm_tis_write32 (priv,TPM_INT_ENABLE (priv->locality), intmask); + chip->flags &= ~TPM_CHIP_FLAG_IRQ ; +} + +static void disable_interrupts (struct tpm_chip *chip) +{ +struct tpm_tis_data *priv =dev_get_drvdata (&chip->dev); +if (priv->irq ==0 ) +return ; + +__disable_interrupts (chip);devm_free_irq (chip->dev.parent, priv->irq, chip); priv->irq =0 ; - chip->flags &= ~TPM_CHIP_FLAG_IRQ ; }/* drivers/char/tpm/tpm_tis_core.c static int tpm_tis_send(struct tpm_chip *chip, u8 *buf, size_t len) · line 546
Quoted 8 lines
drivers/char/tpm/tpm_tis_core.c +5
-0@@ -528,6 +539,12 @@ static int tpm_tis_send(struct tpm_chip *chip, u8 *buf, size_t len)int rc, irq;struct tpm_tis_data *priv =dev_get_drvdata (&chip->dev); +if (unlikely (storm_killed)) { +devm_free_irq (chip->dev.parent, priv->irq, chip); + priv->irq =0 ; + storm_killed = false; + }OK this kind of bad solution because if
tpm_tis_send()is not called, then IRQ is never freed. AFAIK, devres_* do not sleep but use spin lock, i.e. you could render out bothstorm_checkandstorm_killed.Quoted 7 lines
Quoted 3 lines
Is there a way to flag it for freeing later while in an interrupt context? I'm not sure where to clean it up since
devm_free_irqcan't be called intis_int_handler.You could add a workqueue work-struct just for this and queue that up to do the free when you detect the storm. That will then run pretty much immediately, avoiding the storm going on for (much) longer.
That's sounds feasible.
Quoted 8 lines
Quoted 6 lines
Quoted 3 lines
Before diving further into that though, does anyone else have an opinion on ripping out the irq code, and just using polling? We've been only polling since 2015 anyways.
Given James Bottomley's reply I guess it would be worthwhile to get the storm detection to work.
OK, agreed. I take my words back from a response few minutes ago :-)
Quoted 8 lines
:)
To be clear, I think we should give the storm detection a go. Especially given the problems which James has seen with polling on some TPMs.
But if that turns out to not be feasible I agree we should just either disable IRQs by default on standard x86 platforms, or just remove the IRQ support all together.
Just for completeness: one option is also to whitelist IRQ's.
Quoted 3 lines
Regards,
Hans
/Jarkko
Jarkko Sakkinen
maintainerQuoted 118 lines · Jerry Snitselaar
Jarkko Sakkinen @ 2020-11-23 20:26 MST:
Quoted 49 lines · Jerry Snitselaar
Quoted 48 lines
Matthew Garrett @ 2020-10-15 15:39 MST:
Quoted 15 lines · Jerry Snitselaar
Quoted 13 lines
There is a misconfiguration in the bios of the gpio pin used for the interrupt in the T490s. When interrupts are enabled in the
tpm_tisdriver code this results in an interrupt storm. This was initially reported when we attempted to enable the interrupt code in thetpm_tisdriver, which previously wasn't setting a flag to enable it. Due to the reports of the interrupt storm that code was reverted and we went back to polling instead of using interrupts. Now that we know the T490s problem is a firmware issue, add code to check if the system is a T490s and disable interrupts if that is the case. This will allow us to enable interrupts for everyone else. If the user has a fixed bios they can force the enabling of interrupts with tpm_tis.interrupts=1 on the kernel command line.Quoted 10 lines
I think an implication of this is that systems haven't been well-tested with interrupts enabled. In general when we've found a firmware issue in one place it ends up happening elsewhere as well, so it wouldn't surprise me if there are other machines that will also be unhappy with interrupts enabled. Would it be possible to automatically detect this case (eg, if we get more than a certain number of interrupts in a certain timeframe immediately after enabling the interrupt) and automatically fall back to polling in that case? It would also mean that users with fixed firmware wouldn't need to pass a parameter.
I believe Matthew is correct here. I found another system today with completely different vendor for both the system and the tpm chip. In addition another Lenovo model, the L490, has the issue.
This initial attempt at a solution like Matthew suggested works on the system I found today, but I imagine it is all sorts of wrong. In the 2 systems where I've seen it, there are about 100000 interrupts in around 1.5 seconds, and then the irq code shuts down the interrupt because they aren't being handled.
drivers/char/tpm/tpm_tis_core.c +0
-0@@ -27,6 +27,11 @@#include "tpm.h" #include "tpm_tis_core.h" Quoted 7 lines
drivers/char/tpm/tpm_tis_core.c line 33
Quoted 4 lines
drivers/char/tpm/tpm_tis_core.c +4
-0+static unsigned int time_start =0 ; +static bool storm_check = true; +static bool storm_killed = false; +static u32 irqs_fired =0 ;Maybe
kstat_irqs()would be a better idea than ad hoc stats.Thanks, yes that would be better.
Quoted 45 lines
Quoted 45 lines
drivers/char/tpm/tpm_tis_core.c +12
-5+static void tpm_tis_clkrun_enable (struct tpm_chip *chip,bool value);static void tpm_tis_enable_interrupt (struct tpm_chip *chip,u8 mask)@@ -464,25 +469,31 @@ static int tpm_tis_send_data(struct tpm_chip *chip, const u8 *buf, size_t len)return rc; } -static void disable_interrupts (struct tpm_chip *chip) +static void __disable_interrupts (struct tpm_chip *chip) {struct tpm_tis_data *priv =dev_get_drvdata (&chip->dev);u32 intmask;int rc; -if (priv->irq ==0 ) -return ; - rc =tpm_tis_read32 (priv,TPM_INT_ENABLE (priv->locality), &intmask);if (rc <0 ) intmask =0 ; intmask &= ~TPM_GLOBAL_INT_ENABLE ; rc =tpm_tis_write32 (priv,TPM_INT_ENABLE (priv->locality), intmask); + chip->flags &= ~TPM_CHIP_FLAG_IRQ ; +} + +static void disable_interrupts (struct tpm_chip *chip) +{ +struct tpm_tis_data *priv =dev_get_drvdata (&chip->dev); +if (priv->irq ==0 ) +return ; + +__disable_interrupts (chip);devm_free_irq (chip->dev.parent, priv->irq, chip); priv->irq =0 ; - chip->flags &= ~TPM_CHIP_FLAG_IRQ ; }/* @@ -528,6 +539,12 @@ static int tpm_tis_send(struct tpm_chip *chip, u8 *buf, size_t len)int rc, irq;struct tpm_tis_data *priv =dev_get_drvdata (&chip->dev);Quoted 10 lines
drivers/char/tpm/tpm_tis_core.c static int tpm_tis_send(struct tpm_chip *chip, u8 *buf, size_t len) · line 546
Quoted 5 lines
drivers/char/tpm/tpm_tis_core.c +5
-0+if (unlikely (storm_killed)) { +devm_free_irq (chip->dev.parent, priv->irq, chip); + priv->irq =0 ; + storm_killed = false; + }OK this kind of bad solution because if
tpm_tis_send()is not called, then IRQ is never freed. AFAIK, devres_* do not sleep but use spin lock, i.e. you could render out bothstorm_checkandstorm_killed.
Quoted 7 lines
Is there a way to flag it for freeing later while in an interrupt context? I'm not sure where to clean it up since
devm_free_irqcan't be called intis_int_handler.Before diving further into that though, does anyone else have an opinion on ripping out the irq code, and just using polling? We've been only polling since 2015 anyways.
Given these all these issues, it's quite obvious that Windows side is just polling. I'll ack a patch that removes the IRQ code for sure.
/Jarkko
Hi,
Quoted 13 lines · Jerry Snitselaar
There is a misconfiguration in the bios of the gpio pin used for the interrupt in the T490s. When interrupts are enabled in the
tpm_tisdriver code this results in an interrupt storm. This was initially reported when we attempted to enable the interrupt code in thetpm_tisdriver, which previously wasn't setting a flag to enable it. Due to the reports of the interrupt storm that code was reverted and we went back to polling instead of using interrupts. Now that we know the T490s problem is a firmware issue, add code to check if the system is a T490s and disable interrupts if that is the case. This will allow us to enable interrupts for everyone else. If the user has a fixed bios they can force the enabling of interrupts with tpm_tis.interrupts=1 on the kernel command line.
Quoted 6 lines
Cc Peter Huewe <[email protected]>Cc Jason Gunthorpe <[email protected]>Cc Hans de Goede <[email protected]>Reviewed-by James Bottomley <[email protected]>Signed-off-by Jerry Snitselaar <[email protected]>
Patch looks good to me:
Regards,
Hans
Quoted 66 lines
--- drivers/char/tpm/tpm_tis.c | 29 +++++++++++++++++++++++++++-- 1 file changed, 27 insertions(+), 2 deletions(-)drivers/char/tpm/tpm_tis.c +27
-2@@ -27,6 +27,7 @@#include <linux/of.h> #include <linux/of_device.h> #include <linux/kernel.h> +#include <linux/dmi.h> #include "tpm.h" #include "tpm_tis_core.h" @@ -49,8 +50,8 @@ static inline struct tpm_tis_tcg_phy *to_tpm_tis_tcg_phy(struct tpm_tis_data *da return container_of(data, struct tpm_tis_tcg_phy, priv); } -static bool interrupts = true; -module_param (interrupts, bool,0444 ); +static int interrupts =-1 ; +module_param (interrupts, int,0444 );MODULE_PARM_DESC (interrupts,"Enable interrupts" );static bool itpm;@@ -63,6 +64,28 @@ module_param(force, bool, 0444);MODULE_PARM_DESC (force,"Force device probe rather than using ACPI entry" );#endif +static int tpm_tis_disable_irq (const struct dmi_system_id *d) +{ +if (interrupts ==-1 ) { +pr_notice ("tpm_tis: %s detected: disabling interrupts.\n" , d->ident); + interrupts =0 ; + } + +return 0 ; +} + +static const struct dmi_system_id tpm_tis_dmi_table[] = { + { + .callback = tpm_tis_disable_irq, + .ident ="ThinkPad T490s" , + .matches = { +DMI_MATCH (DMI_SYS_VENDOR ,"LENOVO" ), +DMI_MATCH (DMI_PRODUCT_VERSION ,"ThinkPad T490s" ), + }, + }, + {} +}; +#if defined(CONFIG_PNP ) && defined(CONFIG_ACPI )static int has_hid (struct acpi_device *dev,const char *hid) {@@ -192,6 +215,8 @@ static int tpm_tis_init(struct device *dev, struct tpm_info *tpm_info)int irq =-1 ;int rc; +dmi_check_system (tpm_tis_dmi_table); + rc =check_acpi_tpm2 (dev);if (rc)return rc;
Quoted 13 lines · Jerry Snitselaar
There is a misconfiguration in the bios of the gpio pin used for the interrupt in the T490s. When interrupts are enabled in the
tpm_tisdriver code this results in an interrupt storm. This was initially reported when we attempted to enable the interrupt code in thetpm_tisdriver, which previously wasn't setting a flag to enable it. Due to the reports of the interrupt storm that code was reverted and we went back to polling instead of using interrupts. Now that we know the T490s problem is a firmware issue, add code to check if the system is a T490s and disable interrupts if that is the case. This will allow us to enable interrupts for everyone else. If the user has a fixed bios they can force the enabling of interrupts with tpm_tis.interrupts=1 on the kernel command line.
Quoted 6 lines
Cc Peter Huewe <[email protected]>Cc Jason Gunthorpe <[email protected]>Cc Hans de Goede <[email protected]>Reviewed-by James Bottomley <[email protected]>Signed-off-by Jerry Snitselaar <[email protected]>
I'll apply this and make it available in linux-next.
/Jarkko
Quoted 69 lines
--- drivers/char/tpm/tpm_tis.c | 29 +++++++++++++++++++++++++++-- 1 file changed, 27 insertions(+), 2 deletions(-)drivers/char/tpm/tpm_tis.c +27
-2@@ -27,6 +27,7 @@#include <linux/of.h> #include <linux/of_device.h> #include <linux/kernel.h> +#include <linux/dmi.h> #include "tpm.h" #include "tpm_tis_core.h" @@ -49,8 +50,8 @@ static inline struct tpm_tis_tcg_phy *to_tpm_tis_tcg_phy(struct tpm_tis_data *da return container_of(data, struct tpm_tis_tcg_phy, priv); } -static bool interrupts = true; -module_param (interrupts, bool,0444 ); +static int interrupts =-1 ; +module_param (interrupts, int,0444 );MODULE_PARM_DESC (interrupts,"Enable interrupts" );static bool itpm;@@ -63,6 +64,28 @@ module_param(force, bool, 0444);MODULE_PARM_DESC (force,"Force device probe rather than using ACPI entry" );#endif +static int tpm_tis_disable_irq (const struct dmi_system_id *d) +{ +if (interrupts ==-1 ) { +pr_notice ("tpm_tis: %s detected: disabling interrupts.\n" , d->ident); + interrupts =0 ; + } + +return 0 ; +} + +static const struct dmi_system_id tpm_tis_dmi_table[] = { + { + .callback = tpm_tis_disable_irq, + .ident ="ThinkPad T490s" , + .matches = { +DMI_MATCH (DMI_SYS_VENDOR ,"LENOVO" ), +DMI_MATCH (DMI_PRODUCT_VERSION ,"ThinkPad T490s" ), + }, + }, + {} +}; +#if defined(CONFIG_PNP ) && defined(CONFIG_ACPI )static int has_hid (struct acpi_device *dev,const char *hid) {@@ -192,6 +215,8 @@ static int tpm_tis_init(struct device *dev, struct tpm_info *tpm_info)int irq =-1 ;int rc; +dmi_check_system (tpm_tis_dmi_table); + rc =check_acpi_tpm2 (dev);if (rc)return rc;Signature · 1 line
2.28.0
Jarkko Sakkinen
AppliedQuoted 21 lines · Jarkko Sakkinen
Quoted 13 lines · Jerry Snitselaar
There is a misconfiguration in the bios of the gpio pin used for the interrupt in the T490s. When interrupts are enabled in the
tpm_tisdriver code this results in an interrupt storm. This was initially reported when we attempted to enable the interrupt code in thetpm_tisdriver, which previously wasn't setting a flag to enable it. Due to the reports of the interrupt storm that code was reverted and we went back to polling instead of using interrupts. Now that we know the T490s problem is a firmware issue, add code to check if the system is a T490s and disable interrupts if that is the case. This will allow us to enable interrupts for everyone else. If the user has a fixed bios they can force the enabling of interrupts with tpm_tis.interrupts=1 on the kernel command line.Quoted 6 lines
Cc Peter Huewe <[email protected]>Cc Jason Gunthorpe <[email protected]>Cc Hans de Goede <[email protected]>Reviewed-by James Bottomley <[email protected]>Signed-off-by Jerry Snitselaar <[email protected]>
Quoted 3 lines
Reviewed-by Jarkko Sakkinen <[email protected]>I'll apply this and make it available in linux-next.
Applied.
Thank you.
/Jarkko