Skip to contentSkip to conversation

4.18.1-stable review

PATCH Byron Stanoszek 2 participants 4 messages Aug 15, 2018 – Aug 16, 2018 CPU hotplug

lore
Share & tools
lore
#0 lore raw

Hi Greg & Thomas,

I'd like to report a regression in Linux 4.18.1 regarding the L1TF patches.

The kernel no longer thinks I have SMT enabled in the BIOS. This works fine in 4.18.0.

Not sure if this matters, but in my particular 4-core system, my third core is broken (core #2). So I must boot using "maxcpus=2" and then online the other cores & SMT threads at startup using:

echo 1 > /sys/devices/system/cpu/cpu3/online
echo 1 > /sys/devices/system/cpu/cpu4/online
echo 1 > /sys/devices/system/cpu/cpu5/online
echo 1 > /sys/devices/system/cpu/cpu7/online

In 4.18.0, dmesg shows:

smpboot: Booting Node 0 Processor 3 APIC 0x6
smpboot: Booting Node 0 Processor 4 APIC 0x1
smpboot: Booting Node 0 Processor 5 APIC 0x3
smpboot: Booting Node 0 Processor 7 APIC 0x7

In 4.18.1, dmesg shows:

smpboot: Booting Node 0 Processor 3 APIC 0x6
smpboot: Booting Node 0 Processor 4 APIC 0x1
smpboot: CPU 4 is now offline
smpboot: Booting Node 0 Processor 5 APIC 0x3
smpboot: CPU 5 is now offline
smpboot: Booting Node 0 Processor 7 APIC 0x7
smpboot: CPU 7 is now offline

and I get an "Operation cancelled" error in the shell when trying to online 4, 5, and 7.

In 4.18.1, /sys/devices/system/cpu/smt/control says "notsupported".

- - -

A possible second regression is the following:

My CPU normally runs at 3600 MHz. I usually run my CPU at 2800 MHz to keep from overheating under full load (it is a fanless system). I do this by running "echo 1 > /sys/class/thermal/cooling_device5/cur_state", and confirm with "cat /proc/cpuinfo" (shows 2800).

This works in 4.18.0 but not in 4.18.1. I get no error from the "echo" command (and the state reads back as "1"), but the CPU remains running at 3600 MHz.

Thanks,
-Byron

#1 lore raw
Quoted 42 lines · Byron Stanoszek

Hi Greg & Thomas,

I'd like to report a regression in Linux 4.18.1 regarding the L1TF patches.

The kernel no longer thinks I have SMT enabled in the BIOS. This works fine in 4.18.0.

Not sure if this matters, but in my particular 4-core system, my third core is broken (core #2). So I must boot using "maxcpus=2" and then online the other cores & SMT threads at startup using:

echo 1 > /sys/devices/system/cpu/cpu3/online
echo 1 > /sys/devices/system/cpu/cpu4/online
echo 1 > /sys/devices/system/cpu/cpu5/online
echo 1 > /sys/devices/system/cpu/cpu7/online

In 4.18.0, dmesg shows:

smpboot: Booting Node 0 Processor 3 APIC 0x6
smpboot: Booting Node 0 Processor 4 APIC 0x1
smpboot: Booting Node 0 Processor 5 APIC 0x3
smpboot: Booting Node 0 Processor 7 APIC 0x7

In 4.18.1, dmesg shows:

smpboot: Booting Node 0 Processor 3 APIC 0x6
smpboot: Booting Node 0 Processor 4 APIC 0x1
smpboot: CPU 4 is now offline
smpboot: Booting Node 0 Processor 5 APIC 0x3
smpboot: CPU 5 is now offline
smpboot: Booting Node 0 Processor 7 APIC 0x7
smpboot: CPU 7 is now offline

and I get an "Operation cancelled" error in the shell when trying to online 4, 5, and 7.

In 4.18.1, /sys/devices/system/cpu/smt/control says "notsupported".

- - -

A possible second regression is the following:

Quoted 7 lines

My CPU normally runs at 3600 MHz. I usually run my CPU at 2800 MHz to keep from overheating under full load (it is a fanless system). I do this by running "echo 1 > /sys/class/thermal/cooling_device5/cur_state", and confirm with "cat /proc/cpuinfo" (shows 2800).

This works in 4.18.0 but not in 4.18.1. I get no error from the "echo" command (and the state reads back as "1"), but the CPU remains running at 3600 MHz.

How about Linus's tree at the moment, is it ok there?

thanks,

greg k-h

#2 lore raw
Quoted 48 lines · Greg Kroah-Hartman
Quoted 47 lines · Byron Stanoszek

Hi Greg & Thomas,

I'd like to report a regression in Linux 4.18.1 regarding the L1TF patches.

The kernel no longer thinks I have SMT enabled in the BIOS. This works fine in 4.18.0.

Not sure if this matters, but in my particular 4-core system, my third core is broken (core #2). So I must boot using "maxcpus=2" and then online the other cores & SMT threads at startup using:

echo 1 > /sys/devices/system/cpu/cpu3/online
echo 1 > /sys/devices/system/cpu/cpu4/online
echo 1 > /sys/devices/system/cpu/cpu5/online
echo 1 > /sys/devices/system/cpu/cpu7/online

In 4.18.0, dmesg shows:

smpboot: Booting Node 0 Processor 3 APIC 0x6
smpboot: Booting Node 0 Processor 4 APIC 0x1
smpboot: Booting Node 0 Processor 5 APIC 0x3
smpboot: Booting Node 0 Processor 7 APIC 0x7

In 4.18.1, dmesg shows:

smpboot: Booting Node 0 Processor 3 APIC 0x6
smpboot: Booting Node 0 Processor 4 APIC 0x1
smpboot: CPU 4 is now offline
smpboot: Booting Node 0 Processor 5 APIC 0x3
smpboot: CPU 5 is now offline
smpboot: Booting Node 0 Processor 7 APIC 0x7
smpboot: CPU 7 is now offline

and I get an "Operation cancelled" error in the shell when trying to online 4, 5, and 7.

In 4.18.1, /sys/devices/system/cpu/smt/control says "notsupported".

- - -

A possible second regression is the following:

My CPU normally runs at 3600 MHz. I usually run my CPU at 2800 MHz to keep from overheating under full load (it is a fanless system). I do this by running "echo 1 > /sys/class/thermal/cooling_device5/cur_state", and confirm with "cat /proc/cpuinfo" (shows 2800).

Quoted 9 lines
Quoted 2 lines

This works in 4.18.0 but not in 4.18.1. I get no error from the "echo" command (and the state reads back as "1"), but the CPU remains running at 3600 MHz.

How about Linus's tree at the moment, is it ok there?

thanks,

greg k-h

It also fails in Linus's tree. Seems like this logic is to blame:

/*
* If SMT was disabled by BIOS, detect it here, after the CPUs have been
* brought online. This ensures the smt/l1tf sysfs entries are consistent
* with reality. cpu_smt_available is set to true during the bringup of non
* boot CPUs when a SMT sibling is detected. Note, this may overwrite
* cpu_smt_control's previous setting.
*/
void __init cpu_smt_check_topology(void)

{
        if (!cpu_smt_available)
                cpu_smt_control = CPU_SMT_NOT_SUPPORTED;
}

SMT is enabled in my BIOS, but because I booted with maxcpus=2, the init code never officially booted any SMT thread yet--just primary threads. I suspect the line 'cpu_smt_available = true;' in kernel/cpu.c function cpu_smt_allowed() is never being reached.

It is then impossible to boot any SMT thread after init is done, since 'cpu_smt_available' is false.

The following test patch makes everything work for me on both mainline and 4.18.1, but we might as well throw out 'cpu_smt_available' altogether then (or find another way to set it appropriately). Just because we didn't boot any SMT threads at init shouldn't mean that SMT is disabled by the BIOS.

---

x86/l1tf: Fix booting with low maxcpus=# causes SMT to be disabled

If maxcpus=# is given on the kernel command line where # is too low for any SMT CPU (thread) to be booted during init, then the kernel thinks SMT is disabled by the BIOS. SMT threads are then unable to be manually brought online later after init.

Set 'cpu_smt_available' early on in init instead, if topology_smt_supported() returns true.

Fixes 958f338e96f8 ("Merge branch 'l1tf-final' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip")
Cc Thomas Gleixner <[email protected]>
---
 kernel/cpu.c | 10 ++--------
 1 file changed, 2 insertions(+), 8 deletions(-)
kernel/cpu.c +2 -8
index ed44d7d34c2d..bf9be8b8c0a0 100644
@@ -378,6 +378,8 @@ void __init cpu_smt_check_topology_early(void)
 {
 	if (!topology_smt_supported())
 		cpu_smt_control = CPU_SMT_NOT_SUPPORTED;
+	else
+		cpu_smt_available = true;
 }
 
 /*
@@ -405,14 +407,6 @@ static inline bool cpu_smt_allowed(unsigned int cpu)
 	if (topology_is_primary_thread(cpu))
 		return true;
 
-	/*
-	 * If the CPU is not a 'primary' thread and the booted_once bit is
-	 * set then the processor has SMT support. Store this information
-	 * for the late check of SMT support in cpu_smt_check_topology().
-	 */
-	if (per_cpu(cpuhp_state, cpu).booted_once)
-		cpu_smt_available = true;
-
 	if (cpu_smt_control == CPU_SMT_ENABLED)
 		return true;
 
Signature · 1 line
2.18.0
#3 lore raw
Quoted 100 lines · Byron Stanoszek
Quoted 48 lines · Greg Kroah-Hartman
Quoted 47 lines · Byron Stanoszek

Hi Greg & Thomas,

I'd like to report a regression in Linux 4.18.1 regarding the L1TF patches.

The kernel no longer thinks I have SMT enabled in the BIOS. This works fine in 4.18.0.

Not sure if this matters, but in my particular 4-core system, my third core is broken (core #2). So I must boot using "maxcpus=2" and then online the other cores & SMT threads at startup using:

echo 1 > /sys/devices/system/cpu/cpu3/online
echo 1 > /sys/devices/system/cpu/cpu4/online
echo 1 > /sys/devices/system/cpu/cpu5/online
echo 1 > /sys/devices/system/cpu/cpu7/online

In 4.18.0, dmesg shows:

smpboot: Booting Node 0 Processor 3 APIC 0x6
smpboot: Booting Node 0 Processor 4 APIC 0x1
smpboot: Booting Node 0 Processor 5 APIC 0x3
smpboot: Booting Node 0 Processor 7 APIC 0x7

In 4.18.1, dmesg shows:

smpboot: Booting Node 0 Processor 3 APIC 0x6
smpboot: Booting Node 0 Processor 4 APIC 0x1
smpboot: CPU 4 is now offline
smpboot: Booting Node 0 Processor 5 APIC 0x3
smpboot: CPU 5 is now offline
smpboot: Booting Node 0 Processor 7 APIC 0x7
smpboot: CPU 7 is now offline

and I get an "Operation cancelled" error in the shell when trying to online 4, 5, and 7.

In 4.18.1, /sys/devices/system/cpu/smt/control says "notsupported".

- - -

A possible second regression is the following:

My CPU normally runs at 3600 MHz. I usually run my CPU at 2800 MHz to keep from overheating under full load (it is a fanless system). I do this by running "echo 1 > /sys/class/thermal/cooling_device5/cur_state", and confirm with "cat /proc/cpuinfo" (shows 2800).

Quoted 9 lines
Quoted 2 lines

This works in 4.18.0 but not in 4.18.1. I get no error from the "echo" command (and the state reads back as "1"), but the CPU remains running at 3600 MHz.

How about Linus's tree at the moment, is it ok there?

thanks,

greg k-h

It also fails in Linus's tree. Seems like this logic is to blame:

/*
* If SMT was disabled by BIOS, detect it here, after the CPUs have been
* brought online. This ensures the smt/l1tf sysfs entries are consistent
* with reality. cpu_smt_available is set to true during the bringup of non
* boot CPUs when a SMT sibling is detected. Note, this may overwrite
* cpu_smt_control's previous setting.
*/
void __init cpu_smt_check_topology(void)

{
        if (!cpu_smt_available)
                cpu_smt_control = CPU_SMT_NOT_SUPPORTED;
}

SMT is enabled in my BIOS, but because I booted with maxcpus=2, the init code never officially booted any SMT thread yet--just primary threads. I suspect the line 'cpu_smt_available = true;' in kernel/cpu.c function cpu_smt_allowed() is never being reached.

It is then impossible to boot any SMT thread after init is done, since 'cpu_smt_available' is false.

The following test patch makes everything work for me on both mainline and 4.18.1, but we might as well throw out 'cpu_smt_available' altogether then (or find another way to set it appropriately). Just because we didn't boot any SMT threads at init shouldn't mean that SMT is disabled by the BIOS.

---

x86/l1tf: Fix booting with low maxcpus=# causes SMT to be disabled

If maxcpus=# is given on the kernel command line where # is too low for any SMT CPU (thread) to be booted during init, then the kernel thinks SMT is disabled by the BIOS. SMT threads are then unable to be manually brought online later after init.

Set 'cpu_smt_available' early on in init instead, if topology_smt_supported() returns true.

Quoted 6 lines
Fixes 958f338e96f8 ("Merge branch 'l1tf-final' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip")
Cc Thomas Gleixner <[email protected]>
---
 kernel/cpu.c | 10 ++--------
 1 file changed, 2 insertions(+), 8 deletions(-)

Thomas is out for at least the rest of this week, so it might be a bit before he sees this :(

greg k-h