Skip to contentSkip to conversation

Document amd_pmc delay_suspend quirk

PATCH In review Daniel Gibson 3 participants 10 messages Jun 24 – Aug 10

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

cover Document amd_pmc delay_suspend quirk

Recently I submitted patches that work around an EC bug in some Lenovo IdeaPad laptops by adding a quirk to amd_pmc that is enabled automatically for detected devices and can be enforced (to test on possibly affected devices that aren't detected yet) with the delay_suspend module parameter. The quirk makes the kernel sleep for 2.5 seconds before actually suspending.

Now that those patches are in mainline and their commit IDs are known, I submit the documentation for them, which refers the commit IDs.

There's also another small change to the same documentation file that clarifies how to identify kernel log lines with information from the reset register, because so far it was only mentioned that this information is logged, but not how to find it.

See also https://bugzilla.kernel.org/show_bug.cgi?id=221383 and
https://lore.kernel.org/platform-driver-x86/[email protected]/T/#u

These documentation changes were last discussed here:
https://lore.kernel.org/platform-driver-x86/[email protected]/t/#u

Daniel Gibson (2):
Documentation/arch/x86/amd-debugging: Add example for reset register Documentation/arch/x86/amd-debugging: Add section about delay_suspend

 Documentation/arch/x86/amd-debugging.rst | 30 ++++++++++++++++++++++++
 1 file changed, 30 insertions(+)
Signature · 1 line
2.48.1
#1 lore raw

PATCH 1/2 Documentation/arch/x86/amd-debugging: Add example for reset register

The amd debugging documentation didn't state how to identify kernel log lines with information from the reset register about the cause of a previous random reboot.

The added example rectifies this.

---
 Documentation/arch/x86/amd-debugging.rst | 5 +++++
 1 file changed, 5 insertions(+)
Documentation/arch/x86/amd-debugging.rst +5 -0
index d92bf59d62c7..3176a1240fee 100644
@@ -366,3 +366,8 @@ There are 6 classes of reasons for the reboot:
 This information is read by the kernel at bootup and printed into
 the syslog. When a random reboot occurs this message can be helpful
 to determine the next component to debug.
+
+For example, if bit 19 was set, you will get a message like this in the log on
+next bootup::
+
+  x86/amd: Previous system reset reason [0x00080000]: software wrote 0x6 to reset control register 0xCF9
Signature · 1 line
2.48.1
Reviewed-by on v1 1/2
#2 lore raw

Re: [PATCH 1/2] Documentation/arch/x86/amd-debugging: Add example for reset register

Quoted 8 lines · Daniel Gibson

The amd debugging documentation didn't state how to identify kernel log lines with information from the reset register about the cause of a previous random reboot.

The added example rectifies this.

Quoted 17 lines
---
  Documentation/arch/x86/amd-debugging.rst | 5 +++++
  1 file changed, 5 insertions(+)
Documentation/arch/x86/amd-debugging.rst +5 -0
index d92bf59d62c7..3176a1240fee 100644
@@ -366,3 +366,8 @@ There are 6 classes of reasons for the reboot:
  This information is read by the kernel at bootup and printed into
  the syslog. When a random reboot occurs this message can be helpful
  to determine the next component to debug.
+
+For example, if bit 19 was set, you will get a message like this in the log on
+next bootup::
+
+  x86/amd: Previous system reset reason [0x00080000]: software wrote 0x6 to reset control register 0xCF9
#3 lore raw

PATCH 2/2 Documentation/arch/x86/amd-debugging: Add section about delay_suspend

In reply to Daniel Gibson

Some Lenovo IdeaPad Slim 3 devices and similar with AMD CPUs (so far observed with Zen3 and Zen3+ CPUs) have a nonfunctional keyboard and lid switch after s2idle.

It helps to delay suspend by 2.5 seconds so the EC has some time to do whatever it needs to get done before suspend.

Devices known to be affected are matched automatically, others can enforce the delay with a amd_pmc module parameter.

This is now documented in amd-debugging.rst

---
 Documentation/arch/x86/amd-debugging.rst | 25 ++++++++++++++++++++++++
 1 file changed, 25 insertions(+)
Documentation/arch/x86/amd-debugging.rst +25 -0
index 3176a1240fee..3725adb42673 100644
@@ -249,6 +249,31 @@ state entry.
 
 `commit 40b8c14936bd2 ("drm/amd/display: Disable unneeded hpd interrupts during dm_init") <https://git.kernel.org/torvalds/c/40b8c14936bd2>`_
 
+Keyboard and Lid Switch stop working after resume
+-------------------------------------------------
+On various variants of the Lenovo IdeaPad Slim 3 with Barcelo and Rembrandt CPUs
+the lid switch and keyboard, or at least the Fn/Multimedia keys, stopped working
+after resume, until the next reboot.
+
+This was caused by buggy firmware having timing probles, the EC needed some idle
+time right before the CPU cores are suspended, or it got into an inconsistent state.
+
+For laptops that are known to be affected this workaround is enabled
+automatically, to test this workaround on other machines you can set the
+``delay_suspend`` parameter of the ``amd_pmc`` module.
+
+If you need to set the ``delay_suspend`` parameter to fix your machine, please
+report this at [email protected] for it to be added to the
+list of devices that need this workaround, so in future kernel versions it's
+enabled automatically.
+
+Note that the current workaround isn't perfect: On some devices the problems still
+happen if resume is triggered by a timer (wakealarm).
+
+`commit 9b9e60dd31da0 ("platform/x86/amd/pmc: Delay suspend for some Lenovo Laptops") <https://git.kernel.org/torvalds/c/9b9e60dd31da0>`_
+
+`commit 428b9fd2dce50 ("platform/x86/amd/pmc: Add delay_suspend module parameter") <https://git.kernel.org/torvalds/c/428b9fd2dce50>`_
+
 Runtime power consumption issues
 ================================
 
Signature · 1 line
2.48.1
#4 lore raw

Re: [PATCH 2/2] Documentation/arch/x86/amd-debugging: Add section about delay_suspend

Hi,

Quoted 11 lines · Daniel Gibson

Some Lenovo IdeaPad Slim 3 devices and similar with AMD CPUs (so far observed with Zen3 and Zen3+ CPUs) have a nonfunctional keyboard and lid switch after s2idle.

It helps to delay suspend by 2.5 seconds so the EC has some time to do whatever it needs to get done before suspend.

Devices known to be affected are matched automatically, others can enforce the delay with a amd_pmc module parameter.

This is now documented in amd-debugging.rst

any chance to get this merged?
As the delay_suspend parameter for the amd_pmc module are in all the current stable kernels >= 6.6 now (6.6.147, 6.12.100 etc) it would be great if the documentation was also available.

Cheers,
Daniel

Quoted 42 lines
---
  Documentation/arch/x86/amd-debugging.rst | 25 ++++++++++++++++++++++++
  1 file changed, 25 insertions(+)
Documentation/arch/x86/amd-debugging.rst +25 -0
index 3176a1240fee..3725adb42673 100644
@@ -249,6 +249,31 @@ state entry.
 
  `commit 40b8c14936bd2 ("drm/amd/display: Disable unneeded hpd interrupts during dm_init") <https://git.kernel.org/torvalds/c/40b8c14936bd2>`_
 
+Keyboard and Lid Switch stop working after resume
+-------------------------------------------------
+On various variants of the Lenovo IdeaPad Slim 3 with Barcelo and Rembrandt CPUs
+the lid switch and keyboard, or at least the Fn/Multimedia keys, stopped working
+after resume, until the next reboot.
+
+This was caused by buggy firmware having timing probles, the EC needed some idle
+time right before the CPU cores are suspended, or it got into an inconsistent state.
+
+For laptops that are known to be affected this workaround is enabled
+automatically, to test this workaround on other machines you can set the
+``delay_suspend`` parameter of the ``amd_pmc`` module.
+
+If you need to set the ``delay_suspend`` parameter to fix your machine, please
+report this at [email protected] for it to be added to the
+list of devices that need this workaround, so in future kernel versions it's
+enabled automatically.
+
+Note that the current workaround isn't perfect: On some devices the problems still
+happen if resume is triggered by a timer (wakealarm).
+
+`commit 9b9e60dd31da0 ("platform/x86/amd/pmc: Delay suspend for some Lenovo Laptops") <https://git.kernel.org/torvalds/c/9b9e60dd31da0>`_
+
+`commit 428b9fd2dce50 ("platform/x86/amd/pmc: Add delay_suspend module parameter") <https://git.kernel.org/torvalds/c/428b9fd2dce50>`_
+
  Runtime power consumption issues
  ================================
 
maintainer
#5 lore raw

Re: [PATCH 2/2] Documentation/arch/x86/amd-debugging: Add section about delay_suspend

Quoted 4 lines · Daniel Gibson

any chance to get this merged?
As the delay_suspend parameter for the amd_pmc module are in all the current stable kernels >= 6.6 now (6.6.147, 6.12.100 etc) it would be great if the documentation was also available.

I'm not sure this belongs here. More likely on some wiki page or so, but not in a document which talks about techniques how to debug AMD machines.

Signature · 4 lines
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette
#6 lore raw

Re: [PATCH 2/2] Documentation/arch/x86/amd-debugging: Add section about delay_suspend

Quoted 9 lines · Borislav Petkov
Quoted 4 lines · Daniel Gibson

any chance to get this merged?
As the delay_suspend parameter for the amd_pmc module are in all the current stable kernels >= 6.6 now (6.6.147, 6.12.100 etc) it would be great if the documentation was also available.

I'm not sure this belongs here. More likely on some wiki page or so, but not in a document which talks about techniques how to debug AMD machines.

That document mainly talks about s2idle issues and lists issues that have been quirked in the past, so IMHO it fits there?

Also, Mario Limonciello (of AMD) suggested documenting it there in:
https://lore.kernel.org/platform-driver-x86/[email protected]/

However, if there's a specific Wiki you think would be appropriate to document this in, I should probably document it there as well.

Cheers,
Daniel

maintainer
#7 lore raw

Re: [PATCH 2/2] Documentation/arch/x86/amd-debugging: Add section about delay_suspend

Quoted 2 lines · Daniel Gibson

That document mainly talks about s2idle issues and lists issues that have been quirked in the past, so IMHO it fits there?

Did you not read the first sentence of that document?

Quoted 2 lines

Also, Mario Limonciello (of AMD) suggested documenting it there in:
https://lore.kernel.org/platform-driver-x86/[email protected]/

Maybe he needs to read the first sentence of that document too.

Quoted 2 lines

However, if there's a specific Wiki you think would be appropriate to document this in, I should probably document it there as well.

You have a pretty good commit message in

428b9fd2dce5 ("platform/x86/amd/pmc: Add delay_suspend module parameter")

You have a bugzilla entry, you have a mailing list thread, you have a module param documentation.

How much more do you need to document this? People should be able to use search engines or AI, right?

The problem with adding some hints as to how suspend can be fixed on some laptops is that we'll open the floodgates. Do you know how many laptops need quirks and fixes for suspend?

In any case, this doc is not for that.

Sorry.

Signature · 4 lines
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette
#8 lore raw

Re: [PATCH 2/2] Documentation/arch/x86/amd-debugging: Add section about delay_suspend

Quoted 24 lines · Borislav Petkov
Quoted 2 lines · Daniel Gibson

That document mainly talks about s2idle issues and lists issues that have been quirked in the past, so IMHO it fits there?

Did you not read the first sentence of that document?

Quoted 2 lines

Also, Mario Limonciello (of AMD) suggested documenting it there in:
https://lore.kernel.org/platform-driver-x86/[email protected]/

Maybe he needs to read the first sentence of that document too.

Quoted 2 lines

However, if there's a specific Wiki you think would be appropriate to document this in, I should probably document it there as well.

You have a pretty good commit message in

428b9fd2dce5 ("platform/x86/amd/pmc: Add delay_suspend module parameter")

You have a bugzilla entry, you have a mailing list thread, you have a module param documentation.

How much more do you need to document this? People should be able to use search engines or AI, right?

Quoted 8 lines

The problem with adding some hints as to how suspend can be fixed on some laptops is that we'll open the floodgates. Do you know how many laptops need quirks and fixes for suspend?

In any case, this doc is not for that.

Sorry.

I guess I have a different opinion here. If someone very technical comes to me and says I want to debug an issue with suspend or shutdown I want to give them a single link that has all the information they can use to learn.

I thought this could fit in well at 18.7, and then even if a search engine or AI tool stumbles on that page it can learn about the debugging parameter too.

#9 lore raw

Re: [PATCH 2/2] Documentation/arch/x86/amd-debugging: Add section about delay_suspend

Quoted 39 lines · Mario Limonciello
Quoted 28 lines · Borislav Petkov
Quoted 3 lines · Daniel Gibson

That document mainly talks about s2idle issues and lists issues that have
been quirked in the past, so IMHO it fits there?

Did you not read the first sentence of that document?

Quoted 3 lines

Also, Mario Limonciello (of AMD) suggested documenting it there in:
https://lore.kernel.org/platform-driver-x86/
[email protected]/

Maybe he needs to read the first sentence of that document too.

Quoted 2 lines

However, if there's a specific Wiki you think would be appropriate to document this in, I should probably document it there as well.

You have a pretty good commit message in

   428b9fd2dce5 ("platform/x86/amd/pmc: Add delay_suspend module parameter")

You have a bugzilla entry, you have a mailing list thread, you have a module
param documentation.

How much more do you need to document this? People should be able to use search engines or AI, right?

Quoted 9 lines

The problem with adding some hints as to how suspend can be fixed on some laptops is that we'll open the floodgates. Do you know how many laptops need
quirks and fixes for suspend?

In any case, this doc is not for that.

Sorry.

Quoted 8 lines

I guess I have a different opinion here.  If someone very technical comes to me and says I want to debug an issue with suspend or shutdown I want to give them a single link that has all the information they can use to learn.

I thought this could fit in well at 18.7, and then even if a search engine or AI tool stumbles on that page it can learn about the debugging parameter too.

The assumption that commit messages or bug reports are documentation sure explains why Linux official documentation is so lacking and users (have to) rely on AI and Reddit and distro forums and the like, where like half of the information is outdated or even misunderstood hearsay.

Even as a very technical users with strong coding skills (compared to normal users, not to kernel developers), using git blame to figure out the context of kernel module parameter's one-line documentation is very annoying.

I thought that AMD debugging document was a step in the right direction (and wondered if I just haven't found the proper documentation in similar cases in the past), but seems like it's just a glitch in the matrix…