x86/MCE/AMD: fix warning about sleep-in-atomic at early boot
Subsystems (4)
PATCH 1/1 x86/MCE/AMD: fix warning about sleep-in-atomic at early boot
Function smca_configure() is called only for current cpu thus rdmsr_safe_on_cpu() could be replaced with atomic rdmsr_safe().
BUG: sleeping function called from invalid context at kernel/sched/completion.c:99 in_atomic(): 1, irqs_disabled(): 1, pid: 0, name: swapper/1 CPU: 1 PID: 0 Comm: swapper/1 Not tainted 4.19.79-16 #1 Hardware name: GIGABYTE R181-Z90-00/MZ91-FS0-00, BIOS R11 10/25/2019 Call Trace: dump_stack+0x5c/0x7b ___might_sleep+0xec/0x110 wait_for_completion+0x39/0x160 ? __rdmsr_safe_on_cpu+0x45/0x60 rdmsr_safe_on_cpu+0xae/0xf0 ? wrmsr_on_cpus+0x20/0x20 ? machine_check_poll+0xfd/0x1f0 ? mce_amd_feature_init+0x190/0x2d0 mce_amd_feature_init+0x190/0x2d0 mcheck_cpu_init+0x11a/0x460 identify_cpu+0x3e2/0x560 identify_secondary_cpu+0x13/0x80 smp_store_cpu_info+0x45/0x50 start_secondary+0xaa/0x200 secondary_startup_64+0xa4/0xb0
Except warning in kernel log everything works fine.
--- arch/x86/kernel/cpu/mce/amd.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
arch/x86/kernel/cpu/mce/amd.c +1 -1
@@ -269,7 +269,7 @@ static void smca_configure(unsigned int bank, unsigned int cpu)if (smca_banks[bank].hwid)return ; -if (rdmsr_safe_on_cpu (cpu,MSR_AMD64_SMCA_MCx_IPID (bank), &low, &high)) { +if (rdmsr_safe (MSR_AMD64_SMCA_MCx_IPID (bank), &low, &high)) {pr_warn ("Failed to read MCA_IPID for bank %d\n" , bank);return ; }
Borislav Petkov
maintainerQuoted 6 lines · Konstantin Khlebnikov
Function
smca_configure()is called only for current cpu thusrdmsr_safe_on_cpu()could be replaced with atomicrdmsr_safe().BUG: sleeping function called from invalid context at kernel/sched/completion.c:99 in_atomic(): 1, irqs_disabled(): 1, pid: 0, name: swapper/1 CPU: 1 PID: 0 Comm: swapper/1 Not tainted 4.19.79-16 #1
^^^^^^^^^^
I'm assuming you hit this on latest upstream too?
Quoted 31 lines
Hardware name: GIGABYTE R181-Z90-00/MZ91-FS0-00, BIOS R11 10/25/2019 Call Trace: dump_stack+0x5c/0x7b ___might_sleep+0xec/0x110 wait_for_completion+0x39/0x160 ? __rdmsr_safe_on_cpu+0x45/0x60 rdmsr_safe_on_cpu+0xae/0xf0 ? wrmsr_on_cpus+0x20/0x20 ? machine_check_poll+0xfd/0x1f0 ? mce_amd_feature_init+0x190/0x2d0 mce_amd_feature_init+0x190/0x2d0 mcheck_cpu_init+0x11a/0x460 identify_cpu+0x3e2/0x560 identify_secondary_cpu+0x13/0x80 smp_store_cpu_info+0x45/0x50 start_secondary+0xaa/0x200 secondary_startup_64+0xa4/0xb0Except warning in kernel log everything works fine.
Fixes 5896820e0aa3 ("x86/mce/AMD, EDAC/mce_amd: Define and use tables for known SMCA IP types")Signed-off-by Konstantin Khlebnikov <[email protected]>--- arch/x86/kernel/cpu/mce/amd.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)arch/x86/kernel/cpu/mce/amd.c +0
-0
arch/x86/kernel/cpu/mce/amd.c static void smca_configure(unsigned int bank, unsigned int cpu) · line 272
Quoted 5 lines
arch/x86/kernel/cpu/mce/amd.c +1
-1@@ -269,7 +269,7 @@ static void smca_configure(unsigned int bank, unsigned int cpu)if (smca_banks[bank].hwid)return ; -if (rdmsr_safe_on_cpu (cpu,MSR_AMD64_SMCA_MCx_IPID (bank), &low, &high)) { +if (rdmsr_safe (MSR_AMD64_SMCA_MCx_IPID (bank), &low, &high)) {
Yazen, any objections?
Signature · 5 lines
Regards/Gruss,
Boris.
ECO tip #101: Trim your mails when you reply.
--Ghannam, Yazen
Reviewed-byQuoted 52 lines
-----Original Message-----
From: Borislav Petkov <[email protected]>
Sent: Thursday, October 31, 2019 10:30 AM
To: Konstantin Khlebnikov <[email protected]>; Ghannam, Yazen <[email protected]>
Cc: Tony Luck <[email protected]>; [email protected]; [email protected]; [email protected]
Subject: Re: [PATCH] x86/MCE/AMD: fix warning about sleep-in-atomic at early bootQuoted 6 lines · Konstantin Khlebnikov
Function
smca_configure()is called only for current cpu thusrdmsr_safe_on_cpu()could be replaced with atomicrdmsr_safe().BUG: sleeping function called from invalid context at kernel/sched/completion.c:99 in_atomic(): 1, irqs_disabled(): 1, pid: 0, name: swapper/1 CPU: 1 PID: 0 Comm: swapper/1 Not tainted 4.19.79-16 #1^^^^^^^^^^I'm assuming you hit this on latest upstream too?
Quoted 34 lines
Hardware name: GIGABYTE R181-Z90-00/MZ91-FS0-00, BIOS R11 10/25/2019 Call Trace: dump_stack+0x5c/0x7b ___might_sleep+0xec/0x110 wait_for_completion+0x39/0x160 ? __rdmsr_safe_on_cpu+0x45/0x60 rdmsr_safe_on_cpu+0xae/0xf0 ? wrmsr_on_cpus+0x20/0x20 ? machine_check_poll+0xfd/0x1f0 ? mce_amd_feature_init+0x190/0x2d0 mce_amd_feature_init+0x190/0x2d0 mcheck_cpu_init+0x11a/0x460 identify_cpu+0x3e2/0x560 identify_secondary_cpu+0x13/0x80 smp_store_cpu_info+0x45/0x50 start_secondary+0xaa/0x200 secondary_startup_64+0xa4/0xb0Except warning in kernel log everything works fine.
Fixes 5896820e0aa3 ("x86/mce/AMD, EDAC/mce_amd: Define and use tables for known SMCA IP types")Signed-off-by Konstantin Khlebnikov <[email protected]>--- arch/x86/kernel/cpu/mce/amd.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)arch/x86/kernel/cpu/mce/amd.c +0
-0@@ -269,7 +269,7 @@ static void smca_configure(unsigned int bank, unsigned int cpu)if (smca_banks[bank].hwid)return ;
Quoted 5 lines
arch/x86/kernel/cpu/mce/amd.c static void smca_configure(unsigned int bank, unsigned int cpu) · line 272
Quoted 2 lines
arch/x86/kernel/cpu/mce/amd.c +1
-1-if (rdmsr_safe_on_cpu (cpu,MSR_AMD64_SMCA_MCx_IPID (bank), &low, &high)) { +if (rdmsr_safe (MSR_AMD64_SMCA_MCx_IPID (bank), &low, &high)) {Yazen, any objections?
This looks good to me.
We can go further and remove the "cpu" parameter from this entire function. But that can be another patch.
Thanks,
Yazen
Quoted 10 lines · Borislav Petkov
Quoted 6 lines · Konstantin Khlebnikov
Function
smca_configure()is called only for current cpu thusrdmsr_safe_on_cpu()could be replaced with atomicrdmsr_safe().BUG: sleeping function called from invalid context at kernel/sched/completion.c:99 in_atomic(): 1, irqs_disabled(): 1, pid: 0, name: swapper/1 CPU: 1 PID: 0 Comm: swapper/1 Not tainted 4.19.79-16 #1^^^^^^^^^^I'm assuming you hit this on latest upstream too?
I tried 5.4 once but there was no warning.
Code in 4.19 and in mainline almost the same.
Probably hardware needs full power cycle to reset state or something else.
Quoted 40 lines
Quoted 32 lines
Hardware name: GIGABYTE R181-Z90-00/MZ91-FS0-00, BIOS R11 10/25/2019 Call Trace: dump_stack+0x5c/0x7b ___might_sleep+0xec/0x110 wait_for_completion+0x39/0x160 ? __rdmsr_safe_on_cpu+0x45/0x60 rdmsr_safe_on_cpu+0xae/0xf0 ? wrmsr_on_cpus+0x20/0x20 ? machine_check_poll+0xfd/0x1f0 ? mce_amd_feature_init+0x190/0x2d0 mce_amd_feature_init+0x190/0x2d0 mcheck_cpu_init+0x11a/0x460 identify_cpu+0x3e2/0x560 identify_secondary_cpu+0x13/0x80 smp_store_cpu_info+0x45/0x50 start_secondary+0xaa/0x200 secondary_startup_64+0xa4/0xb0Except warning in kernel log everything works fine.
Fixes 5896820e0aa3 ("x86/mce/AMD, EDAC/mce_amd: Define and use tables for known SMCA IP types")Signed-off-by Konstantin Khlebnikov <[email protected]>--- arch/x86/kernel/cpu/mce/amd.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)arch/x86/kernel/cpu/mce/amd.c +0
-0arch/x86/kernel/cpu/mce/amd.c static void smca_configure(unsigned int bank, unsigned int cpu) · line 272
Quoted 5 lines
arch/x86/kernel/cpu/mce/amd.c +1
-1@@ -269,7 +269,7 @@ static void smca_configure(unsigned int bank, unsigned int cpu)if (smca_banks[bank].hwid)return ; -if (rdmsr_safe_on_cpu (cpu,MSR_AMD64_SMCA_MCx_IPID (bank), &low, &high)) { +if (rdmsr_safe (MSR_AMD64_SMCA_MCx_IPID (bank), &low, &high)) {Yazen, any objections?
Borislav Petkov
maintainerQuoted 2 lines · Konstantin Khlebnikov
I tried 5.4 once but there was no warning.
Code in 4.19 and in mainline almost the same.
Yes, but early boot code has changed a lot since 4.19. If you can't trigger it on 5.4, then I'll drop the BUG splat from your commit message and change it to talk about replacing the IPI-sending function, which is a good cleanup in itself.
Thx.
Signature · 4 lines
Regards/Gruss,
Boris.
https://people.kernel.org/tglx/notes-about-netiquetteBorislav Petkov
maintainerQuoted 8 lines · Borislav Petkov
Quoted 2 lines · Konstantin Khlebnikov
I tried 5.4 once but there was no warning.
Code in 4.19 and in mainline almost the same.Yes, but early boot code has changed a lot since 4.19. If you can't trigger it on 5.4, then I'll drop the BUG splat from your commit message and change it to talk about replacing the IPI-sending function, which is a good cleanup in itself.
Ok, I was able to trigger it myself:
[ 0.822602] BUG: sleeping function called from invalid context at kernel/sched/completion.c:99 [ 0.822602] in_atomic(): 1, irqs_disabled(): 1, non_block: 0, pid: 0, name: swapper/1 [ 0.822602] no locks held by swapper/1/0. [ 0.822602] irq event stamp: 0 [ 0.822602] hardirqs last enabled at (0): [<0000000000000000>] 0x0 [ 0.822602] hardirqs last disabled at (0): [<ffffffff8106dda9>] copy_process+0x8b9/0x1ca0 [ 0.822602] softirqs last enabled at (0): [<ffffffff8106dda9>] copy_process+0x8b9/0x1ca0 [ 0.822602] softirqs last disabled at (0): [<0000000000000000>] 0x0 [ 0.822602] Preemption disabled at: [ 0.822602] [<ffffffff8104703b>] start_secondary+0x3b/0x190 [ 0.822602] CPU: 1 PID: 0 Comm: swapper/1 Not tainted 5.5.0-rc2+ #1 [ 0.822602] Hardware name: GIGABYTE MZ01-CE1-00/MZ01-CE1-00, BIOS F02 08/29/2018 [ 0.822602] Call Trace: [ 0.822602] dump_stack+0x71/0xa0 [ 0.822602] ___might_sleep.cold.92+0xf7/0x11f [ 0.822602] wait_for_completion+0x3c/0x180 [ 0.822602] ? generic_exec_single+0xca/0x100 [ 0.822602] rdmsr_safe_on_cpu+0xe8/0x100 [ 0.822602] ? wrmsr_on_cpus+0x20/0x20 [ 0.822602] mce_amd_feature_init+0x2ab/0x590 [ 0.822602] mcheck_cpu_init+0x17a/0x4d0 [ 0.822602] identify_cpu+0x3f0/0x750 [ 0.822602] identify_secondary_cpu+0x13/0x80 [ 0.822602] smp_store_cpu_info+0x45/0x50 [ 0.822602] start_secondary+0x50/0x190 [ 0.822602] secondary_startup_64+0xa4/0xb0
Rerouting patch...
Thx.
Signature · 4 lines
Regards/Gruss,
Boris.
https://people.kernel.org/tglx/notes-about-netiquette[tip: ras/core] x86/MCE/AMD: Do not use rdmsr_safe_on_cpu() in smca_configure()
In reply to Konstantin KhlebnikovThe following commit has been merged into the ras/core branch of tip:
The following commit has been merged into the ras/core branch of tip:
Commit-ID: 17ba1939e3676417590ec6f7555608e137e1719a Gitweb: https://git.kernel.org/tip/17ba1939e3676417590ec6f7555608e137e1719a Author: Konstantin Khlebnikov <[email protected]> AuthorDate: Thu, 31 Oct 2019 16:04:48 +03:00 Committer: Borislav Petkov <[email protected]> CommitterDate: Mon, 09 Dec 2019 14:54:59 +01:00
x86/MCE/AMD: Do not use rdmsr_safe_on_cpu() in smca_configure()
The function smca_configure() is called only on the current CPU therefore replace rdmsr_safe_on_cpu() with atomic rdmsr_safe() and avoid the IPI.
[ bp: Cleanup commit message. ]
--- arch/x86/kernel/cpu/mce/amd.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
arch/x86/kernel/cpu/mce/amd.c +1 -1
@@ -269,7 +269,7 @@ static void smca_configure(unsigned int bank, unsigned int cpu)if (smca_banks[bank].hwid)return ; -if (rdmsr_safe_on_cpu (cpu,MSR_AMD64_SMCA_MCx_IPID (bank), &low, &high)) { +if (rdmsr_safe (MSR_AMD64_SMCA_MCx_IPID (bank), &low, &high)) {pr_warn ("Failed to read MCA_IPID for bank %d\n" , bank);return ; }
[tip: ras/urgent] x86/MCE/AMD: Do not use rdmsr_safe_on_cpu() in smca_configure()
In reply to Konstantin KhlebnikovThe following commit has been merged into the ras/urgent branch of tip:
The following commit has been merged into the ras/urgent branch of tip:
Commit-ID: 246ff09f89e54fdf740a8d496176c86743db3ec7 Gitweb: https://git.kernel.org/tip/246ff09f89e54fdf740a8d496176c86743db3ec7 Author: Konstantin Khlebnikov <[email protected]> AuthorDate: Thu, 31 Oct 2019 16:04:48 +03:00 Committer: Borislav Petkov <[email protected]> CommitterDate: Tue, 17 Dec 2019 09:39:33 +01:00
x86/MCE/AMD: Do not use rdmsr_safe_on_cpu() in smca_configure()
... because interrupts are disabled that early and sending IPIs can deadlock:
BUG: sleeping function called from invalid context at kernel/sched/completion.c:99 in_atomic(): 1, irqs_disabled(): 1, non_block: 0, pid: 0, name: swapper/1 no locks held by swapper/1/0. irq event stamp: 0 hardirqs last enabled at (0): [<0000000000000000>] 0x0 hardirqs last disabled at (0): [<ffffffff8106dda9>] copy_process+0x8b9/0x1ca0 softirqs last enabled at (0): [<ffffffff8106dda9>] copy_process+0x8b9/0x1ca0 softirqs last disabled at (0): [<0000000000000000>] 0x0 Preemption disabled at: [<ffffffff8104703b>] start_secondary+0x3b/0x190 CPU: 1 PID: 0 Comm: swapper/1 Not tainted 5.5.0-rc2+ #1 Hardware name: GIGABYTE MZ01-CE1-00/MZ01-CE1-00, BIOS F02 08/29/2018 Call Trace: dump_stack ___might_sleep.cold.92 wait_for_completion ? generic_exec_single rdmsr_safe_on_cpu ? wrmsr_on_cpus mce_amd_feature_init mcheck_cpu_init identify_cpu identify_secondary_cpu smp_store_cpu_info start_secondary secondary_startup_64
The function smca_configure() is called only on the current CPU anyway, therefore replace rdmsr_safe_on_cpu() with atomic rdmsr_safe() and avoid the IPI.
[ bp: Update commit message. ]
--- arch/x86/kernel/cpu/mce/amd.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
arch/x86/kernel/cpu/mce/amd.c +1 -1
@@ -269,7 +269,7 @@ static void smca_configure(unsigned int bank, unsigned int cpu)if (smca_banks[bank].hwid)return ; -if (rdmsr_safe_on_cpu (cpu,MSR_AMD64_SMCA_MCx_IPID (bank), &low, &high)) { +if (rdmsr_safe (MSR_AMD64_SMCA_MCx_IPID (bank), &low, &high)) {pr_warn ("Failed to read MCA_IPID for bank %d\n" , bank);return ; }