Skip to contentSkip to conversation

clarify how insecure CPU is

PATCH In review Pavel Machek 3 participants 10 messages Jan 8, 2018 – Mar 4, 2018 x86

lore
Share & tools
lore

Pavel Machek

author
#0 lore raw

PATCH 1/1 clarify how insecure CPU is

First, what is going on with X86_BUG_AMD_E400 and X86_BUG_AMD_APIC_C1E ? They seem to refer to the same bug, perhaps comment should mention that? (Do we need two flags for one bug?)

Next, maybe X86_BUG_CPU_INSECURE is a bit too generic? This seems to address "Meltdown" problem, but not "Spectre". Should it be limited to PPro and newer Intel CPUs?

Should another erratum be added for "Spectre"? This is present even on AMD CPUs, but should not be present in 486, maybe Pentium, and some Atom chips?

Plus... is this reasonable interface?

bugs : cpu_insecure

I believe we should

a) have something more descriptive than 'cpu_insecure', like 'mem_always_r' (because poor user has no chance to know if it is Meltdown, Spectre, or something else)

b) have has_meltdown : yes/no, because otherwise poor userspace can not tell if CPU is actually bug-free, or if the kernel is just too old to know about specific bug. With all the backport, this is quite important.

Best regards,
Pavel

arch/x86/include/asm/cpufeatures.h +2 -2
index 07cdd17..d46958e 100644
@@ -340,7 +340,7 @@
 #define X86_BUG_NULL_SEG		X86_BUG(10) /* Nulling a selector preserves the base */
 #define X86_BUG_SWAPGS_FENCE		X86_BUG(11) /* SWAPGS without input dep on GS */
 #define X86_BUG_MONITOR			X86_BUG(12) /* IPI required to wake up remote CPU */
-#define X86_BUG_AMD_E400		X86_BUG(13) /* CPU is among the affected by Erratum 400 */
-#define X86_BUG_CPU_INSECURE		X86_BUG(14) /* CPU is insecure and needs kernel page table isolation */
+#define X86_BUG_AMD_E400		X86_BUG(13) /* CPU is among the affected by Erratum 400, check for X86_BUG_AMD_APIC_C1E */
+#define X86_BUG_CPU_INSECURE		X86_BUG(14) /* CPU always allows reading mapped memory, aka "Meltdown", kernel page table isolation needed */
 
 #endif /* _ASM_X86_CPUFEATURES_H */
Signature · 2 lines
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html
#1 lore raw
Quoted 9 lines · Pavel Machek

First, what is going on with X86_BUG_AMD_E400 and X86_BUG_AMD_APIC_C1E ? They seem to refer to the same bug, perhaps comment should mention that? (Do we need two flags for one bug?)

Next, maybe X86_BUG_CPU_INSECURE is a bit too generic? This seems to address "Meltdown" problem, but not "Spectre". Should it be limited to PPro and newer Intel CPUs?

Quoted 7 lines

Should another erratum be added for "Spectre"? This is present even on AMD CPUs, but should not be present in 486, maybe Pentium, and some Atom chips?

Plus... is this reasonable interface?

bugs : cpu_insecure

We've renamed it to meltdown already and added spectre_v1/v2 bits for the rest of the mess.

author
#2 lore raw
Quoted 15 lines · Thomas Gleixner
Quoted 13 lines · Pavel Machek

First, what is going on with X86_BUG_AMD_E400 and X86_BUG_AMD_APIC_C1E ? They seem to refer to the same bug, perhaps comment should mention that? (Do we need two flags for one bug?)

Next, maybe X86_BUG_CPU_INSECURE is a bit too generic? This seems to address "Meltdown" problem, but not "Spectre". Should it be limited to PPro and newer Intel CPUs?

Should another erratum be added for "Spectre"? This is present even on AMD CPUs, but should not be present in 486, maybe Pentium, and some Atom chips?

Quoted 6 lines
Quoted 3 lines

Plus... is this reasonable interface?

bugs : cpu_insecure

We've renamed it to meltdown already and added spectre_v1/v2 bits for the rest of the mess.

Could you explain (best with code comment) what is going on with X86_BUG_AMD_E400 and X86_BUG_AMD_APIC_C1E ? They seem to refer to the same bug.

Plus, as I explained: "bugs: meltdown, spectre" seems to be bad idea, as userland application can not easily tell between "no bug" and "bug not known to kernel".

Best regards,
Pavel

Signature · 2 lines
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html
#3 lore raw
Quoted 18 lines · Pavel Machek
Quoted 17 lines · Thomas Gleixner
Quoted 15 lines · Pavel Machek

First, what is going on with X86_BUG_AMD_E400 and X86_BUG_AMD_APIC_C1E ? They seem to refer to the same bug, perhaps comment should mention that? (Do we need two flags for one bug?)

Next, maybe X86_BUG_CPU_INSECURE is a bit too generic? This seems to address "Meltdown" problem, but not "Spectre". Should it be limited to PPro and newer Intel CPUs?

Should another erratum be added for "Spectre"? This is present even on AMD CPUs, but should not be present in 486, maybe Pentium, and some Atom chips?

Plus... is this reasonable interface?

Quoted 8 lines
Quoted 4 lines
Quoted 1 line

bugs : cpu_insecure

We've renamed it to meltdown already and added spectre_v1/v2 bits for the rest of the mess.

Could you explain (best with code comment) what is going on with X86_BUG_AMD_E400 and X86_BUG_AMD_APIC_C1E ? They seem to refer to the same bug.

Sorry, that;s really not the time for this.

Quoted 3 lines

Plus, as I explained: "bugs: meltdown, spectre" seems to be bad idea, as userland application can not easily tell between "no bug" and "bug not known to kernel".

https://lkml.kernel.org/r/[email protected]
https://lkml.kernel.org/r/[email protected]

Thanks,

tglx

author
#4 lore raw
Quoted 22 lines · Thomas Gleixner
Quoted 20 lines · Pavel Machek
Quoted 19 lines · Thomas Gleixner
Quoted 16 lines · Pavel Machek

First, what is going on with X86_BUG_AMD_E400 and X86_BUG_AMD_APIC_C1E ? They seem to refer to the same bug, perhaps comment should mention that? (Do we need two flags for one bug?)

Next, maybe X86_BUG_CPU_INSECURE is a bit too generic? This seems to address "Meltdown" problem, but not "Spectre". Should it be limited to PPro and newer Intel CPUs?

Should another erratum be added for "Spectre"? This is present even on AMD CPUs, but should not be present in 486, maybe Pentium, and some Atom chips?

Plus... is this reasonable interface?

bugs : cpu_insecure

Quoted 8 lines
Quoted 6 lines
Quoted 2 lines

We've renamed it to meltdown already and added spectre_v1/v2 bits for the rest of the mess.

Could you explain (best with code comment) what is going on with X86_BUG_AMD_E400 and X86_BUG_AMD_APIC_C1E ? They seem to refer to the same bug.

Sorry, that;s really not the time for this.

Ok, is there better time now? The code is a bit confusing...
Pavel

Signature · 2 lines
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html
#5 lore raw
Quoted 26 lines · Pavel Machek
Quoted 25 lines · Thomas Gleixner
Quoted 23 lines · Pavel Machek
Quoted 21 lines · Thomas Gleixner
Quoted 9 lines · Pavel Machek

First, what is going on with X86_BUG_AMD_E400 and X86_BUG_AMD_APIC_C1E ? They seem to refer to the same bug, perhaps comment should mention that? (Do we need two flags for one bug?)

Next, maybe X86_BUG_CPU_INSECURE is a bit too generic? This seems to address "Meltdown" problem, but not "Spectre". Should it be limited to PPro and newer Intel CPUs?

Quoted 7 lines

Should another erratum be added for "Spectre"? This is present even on AMD CPUs, but should not be present in 486, maybe Pentium, and some Atom chips?

Plus... is this reasonable interface?

bugs : cpu_insecure

We've renamed it to meltdown already and added spectre_v1/v2 bits for the rest of the mess.

Quoted 7 lines
Quoted 5 lines
Quoted 3 lines

Could you explain (best with code comment) what is going on with X86_BUG_AMD_E400 and X86_BUG_AMD_APIC_C1E ? They seem to refer to the same bug.

Sorry, that;s really not the time for this.

Ok, is there better time now? The code is a bit confusing...

What's confusing? Here are the relevant code snippets in invocation order.

	/*
	 * Check whether the machine is affected by erratum 400. This is
	 * used to select the proper idle routine and to enable the check
	 * whether the machine is affected in arch_post_acpi_init(), which
	 * sets the X86_BUG_AMD_APIC_C1E bug depending on the MSR check.
	 */
	if (cpu_has_amd_erratum(c, amd_erratum_400))
		set_cpu_bug(c, X86_BUG_AMD_E400);

This sets the errate 400 bug bit to tell subsequent code that the CPU might be affected by that erratum.

void select_idle_routine(const struct cpuinfo_x86 *c) {

	if (boot_cpu_has_bug(X86_BUG_AMD_E400)) {
		pr_info("using AMD E400 aware idle routine\n");
		x86_idle = amd_e400_idle;

Selects the idle routine depending on the bug flag

void __init arch_post_acpi_subsys_init(void)

{
	u32 lo, hi;

	if (!boot_cpu_has_bug(X86_BUG_AMD_E400))
		return;

	/*
	 * AMD E400 detection needs to happen after ACPI has been enabled. If
	 * the machine is affected K8_INTP_C1E_ACTIVE_MASK bits are set in
	 * MSR_K8_INT_PENDING_MSG.
	 */
	rdmsr(MSR_K8_INT_PENDING_MSG, lo, hi);
	if (!(lo & K8_INTP_C1E_ACTIVE_MASK))
		return;

Late detection whether C1E which halts TSC and APIC is enabled. This needs to be done after ACPI is initialized.

/*
* AMD Erratum 400 aware idle routine. We handle it the same way as C3 power
* states (local apic timer and TSC stop).
*/
static void amd_e400_idle(void)

{
	/*
	 * We cannot use static_cpu_has_bug() here because X86_BUG_AMD_APIC_C1E
	 * gets set after static_cpu_has() places have been converted via
	 * alternatives.
	 */
	if (!boot_cpu_has_bug(X86_BUG_AMD_APIC_C1E)) {
		default_idle();
		return;
	}

The actual idle routine. If the C1E bug flag is not set, CPU is not affected, use default idle, otherwise handle it like other C-States which stop TSC and APIC.

The comments are clear enough, but Feel free to send patches which enhance them if you think thats necessary.

Thanks,

tglx

author
#6 lore raw

Hi!

Quoted 86 lines
Quoted 20 lines
Quoted 20 lines
Quoted 20 lines
Quoted 19 lines
Quoted 9 lines

First, what is going on with X86_BUG_AMD_E400 and X86_BUG_AMD_APIC_C1E ? They seem to refer to the same bug, perhaps comment should mention that? (Do we need two flags for one bug?)

Next, maybe X86_BUG_CPU_INSECURE is a bit too generic? This seems to address "Meltdown" problem, but not "Spectre". Should it be limited to PPro and newer Intel CPUs?

Quoted 7 lines

Should another erratum be added for "Spectre"? This is present even on AMD CPUs, but should not be present in 486, maybe Pentium, and some Atom chips?

Plus... is this reasonable interface?

bugs : cpu_insecure

We've renamed it to meltdown already and added spectre_v1/v2 bits for the rest of the mess.

Quoted 7 lines
Quoted 5 lines
Quoted 3 lines

Could you explain (best with code comment) what is going on with X86_BUG_AMD_E400 and X86_BUG_AMD_APIC_C1E ? They seem to refer to the same bug.

Sorry, that;s really not the time for this.

Ok, is there better time now? The code is a bit confusing...

What's confusing? Here are the relevant code snippets in invocation order.

	/*
	 * Check whether the machine is affected by erratum 400. This is
	 * used to select the proper idle routine and to enable the check
	 * whether the machine is affected in arch_post_acpi_init(), which
	 * sets the X86_BUG_AMD_APIC_C1E bug depending on the MSR check.
	 */
	if (cpu_has_amd_erratum(c, amd_erratum_400))
		set_cpu_bug(c, X86_BUG_AMD_E400);

This sets the errate 400 bug bit to tell subsequent code that the CPU might be affected by that erratum.

void select_idle_routine(const struct cpuinfo_x86 *c) {

	if (boot_cpu_has_bug(X86_BUG_AMD_E400)) {
		pr_info("using AMD E400 aware idle routine\n");
		x86_idle = amd_e400_idle;

Selects the idle routine depending on the bug flag

void __init arch_post_acpi_subsys_init(void)

{
	u32 lo, hi;

	if (!boot_cpu_has_bug(X86_BUG_AMD_E400))
		return;

	/*
	 * AMD E400 detection needs to happen after ACPI has been enabled. If
	 * the machine is affected K8_INTP_C1E_ACTIVE_MASK bits are set in
	 * MSR_K8_INT_PENDING_MSG.
	 */
	rdmsr(MSR_K8_INT_PENDING_MSG, lo, hi);
	if (!(lo & K8_INTP_C1E_ACTIVE_MASK))
		return;

Late detection whether C1E which halts TSC and APIC is enabled. This needs to be done after ACPI is initialized.

/*
* AMD Erratum 400 aware idle routine. We handle it the same way as C3 power
* states (local apic timer and TSC stop).
*/
static void amd_e400_idle(void)

{
	/*
	 * We cannot use static_cpu_has_bug() here because X86_BUG_AMD_APIC_C1E
	 * gets set after static_cpu_has() places have been converted via
	 * alternatives.
	 */
	if (!boot_cpu_has_bug(X86_BUG_AMD_APIC_C1E)) {
		default_idle();
		return;
	}
Quoted 7 lines

The actual idle routine. If the C1E bug flag is not set, CPU is not affected, use default idle, otherwise handle it like other C-States which stop TSC and APIC.

The comments are clear enough, but Feel free to send patches which enhance them if you think thats necessary.

Thanks for explanation. Might this be good idea?

Signed-off-by Pavel Machek <[email protected]>
arch/x86/include/asm/cpufeatures.h +2 -2
index f41079d..4901742 100644
@@ -341,7 +341,7 @@
 #define X86_BUG_FDIV			X86_BUG(1) /* FPU FDIV */
 #define X86_BUG_COMA			X86_BUG(2) /* Cyrix 6x86 coma */
 #define X86_BUG_AMD_TLB_MMATCH		X86_BUG(3) /* "tlb_mmatch" AMD Erratum 383 */
-#define X86_BUG_AMD_APIC_C1E		X86_BUG(4) /* "apic_c1e" AMD Erratum 400 */
+#define X86_BUG_AMD_APIC_C1E		X86_BUG(4) /* System is affected AMD Erratum 400, special idle routine is needed */
 #define X86_BUG_11AP			X86_BUG(5) /* Bad local APIC aka 11AP */
 #define X86_BUG_FXSAVE_LEAK		X86_BUG(6) /* FXSAVE leaks FOP/FIP/FOP */
 #define X86_BUG_CLFLUSH_MONITOR		X86_BUG(7) /* AAI65, CLFLUSH required before MONITOR */

@@ -356,7 +356,7 @@
 #define X86_BUG_NULL_SEG		X86_BUG(10) /* Nulling a selector preserves the base */
 #define X86_BUG_SWAPGS_FENCE		X86_BUG(11) /* SWAPGS without input dep on GS */
 #define X86_BUG_MONITOR			X86_BUG(12) /* IPI required to wake up remote CPU */
-#define X86_BUG_AMD_E400		X86_BUG(13) /* CPU is among the affected by Erratum 400 */
+#define X86_BUG_AMD_E400		X86_BUG(13) /* System may be affected by Erratum 400, X86_BUG_AMD_APIC_C1E might be needed  */
 #define X86_BUG_CPU_MELTDOWN		X86_BUG(14) /* CPU is affected by meltdown attack and needs kernel page table isolation */
 #define X86_BUG_SPECTRE_V1		X86_BUG(15) /* CPU is affected by Spectre variant 1 attack with conditional branches */
 #define X86_BUG_SPECTRE_V2		X86_BUG(16) /* CPU is affected by Spectre variant 2 attack with indirect branches */
Signature · 2 lines
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html
#7 lore raw
Quoted 14 lines · Pavel Machek
arch/x86/include/asm/cpufeatures.h +1 -1
index f41079d..4901742 100644
@@ -341,7 +341,7 @@
 #define X86_BUG_FDIV			X86_BUG(1) /* FPU FDIV */
 #define X86_BUG_COMA			X86_BUG(2) /* Cyrix 6x86 coma */
 #define X86_BUG_AMD_TLB_MMATCH		X86_BUG(3) /* "tlb_mmatch" AMD Erratum 383 */
-#define X86_BUG_AMD_APIC_C1E		X86_BUG(4) /* "apic_c1e" AMD Erratum 400 */
+#define X86_BUG_AMD_APIC_C1E		X86_BUG(4) /* System is affected AMD Erratum 400, special idle routine is needed */
 #define X86_BUG_11AP			X86_BUG(5) /* Bad local APIC aka 11AP */
 #define X86_BUG_FXSAVE_LEAK		X86_BUG(6) /* FXSAVE leaks FOP/FIP/FOP */
 #define X86_BUG_CLFLUSH_MONITOR		X86_BUG(7) /* AAI65, CLFLUSH required before MONITOR */

arch/x86/include/asm/cpufeatures.h line 359

Quoted 5 lines
arch/x86/include/asm/cpufeatures.h +1 -1
@@ -356,7 +356,7 @@
 #define X86_BUG_NULL_SEG		X86_BUG(10) /* Nulling a selector preserves the base */
 #define X86_BUG_SWAPGS_FENCE		X86_BUG(11) /* SWAPGS without input dep on GS */
 #define X86_BUG_MONITOR			X86_BUG(12) /* IPI required to wake up remote CPU */
-#define X86_BUG_AMD_E400		X86_BUG(13) /* CPU is among the affected by Erratum 400 */
+#define X86_BUG_AMD_E400		X86_BUG(13) /* System may be affected by Erratum 400, X86_BUG_AMD_APIC_C1E might be needed  */

Not "might be needed" - "X86_BUG_AMD_APIC_C1E will be set if platform is affected".

And then you don't need the above comment change. And you can't remove "apic_c1e" there because it is magical.

Signature · 5 lines
Regards/Gruss,
    Boris.

SUSE Linux GmbH, GF: Felix Imendörffer, Jane Smithard, Graham Norton, HRB 21284 (AG Nürnberg)
--
author
#8 lore raw
Quoted 15 lines · Borislav Petkov
Quoted 14 lines · Pavel Machek
arch/x86/include/asm/cpufeatures.h +1 -1
index f41079d..4901742 100644
@@ -341,7 +341,7 @@
 #define X86_BUG_FDIV			X86_BUG(1) /* FPU FDIV */
 #define X86_BUG_COMA			X86_BUG(2) /* Cyrix 6x86 coma */
 #define X86_BUG_AMD_TLB_MMATCH		X86_BUG(3) /* "tlb_mmatch" AMD Erratum 383 */
-#define X86_BUG_AMD_APIC_C1E		X86_BUG(4) /* "apic_c1e" AMD Erratum 400 */
+#define X86_BUG_AMD_APIC_C1E		X86_BUG(4) /* System is affected AMD Erratum 400, special idle routine is needed */
 #define X86_BUG_11AP			X86_BUG(5) /* Bad local APIC aka 11AP */
 #define X86_BUG_FXSAVE_LEAK		X86_BUG(6) /* FXSAVE leaks FOP/FIP/FOP */
 #define X86_BUG_CLFLUSH_MONITOR		X86_BUG(7) /* AAI65, CLFLUSH required before MONITOR */

@@ -356,7 +356,7 @@
Quoted 8 lines

arch/x86/include/asm/cpufeatures.h line 359

Quoted 5 lines
arch/x86/include/asm/cpufeatures.h +1 -1
 #define X86_BUG_NULL_SEG		X86_BUG(10) /* Nulling a selector preserves the base */
 #define X86_BUG_SWAPGS_FENCE		X86_BUG(11) /* SWAPGS without input dep on GS */
 #define X86_BUG_MONITOR			X86_BUG(12) /* IPI required to wake up remote CPU */
-#define X86_BUG_AMD_E400		X86_BUG(13) /* CPU is among the affected by Erratum 400 */
+#define X86_BUG_AMD_E400		X86_BUG(13) /* System may be affected by Erratum 400, X86_BUG_AMD_APIC_C1E might be needed  */

Not "might be needed" - "X86_BUG_AMD_APIC_C1E will be set if platform is affected".

That's not what Thomas was explaining to me.

Quoted 2 lines

And then you don't need the above comment change. And you can't remove "apic_c1e" there because it is magical.

So.. what's magical about it, why do we need two bits, and why is that not explained in the header file?

Please go through the email thread, I'm trying to understand what is going on here, and no, the comments in the header are not helpful.

Signature · 2 lines
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html
#9 lore raw
Quoted 4 lines · Pavel Machek
Quoted 2 lines

Not "might be needed" - "X86_BUG_AMD_APIC_C1E will be set if platform is affected".

That's not what Thomas was explaining to me.

It is in the comment he pasted:

* Check whether the machine is affected by erratum 400. This is
* used to select the proper idle routine and to enable the check
* whether the machine is affected in arch_post_acpi_init(), which
* sets the X86_BUG_AMD_APIC_C1E bug depending on the MSR check.

Quoted 2 lines

So.. what's magical about it, why do we need two bits, and why is that not explained in the header file?

Lemme enable line numbers so that you can find it:

arch/x86/include/asm/cpufeatures.h:
19 /*

 20  * Note: If the comment begins with a quoted string, that string is used
 21  * in /proc/cpuinfo instead of the macro name.  If the string is "",
 22  * this feature bit is not displayed in /proc/cpuinfo at all.
Quoted 1 line

Please go through the email thread,

No, you read Thomas' mail again.

Quoted 1 line

I'm trying to understand what is going on here,

Nothing's going on, it works as designed.

X86_BUG_AMD_E400 marks all CPUs which could be affected by erratum 400 and X86_BUG_AMD_APIC_C1E is the bit we set when we detect that the CPU is *actually* affected because we need to do the detection late, after ACPI has been initialized.

A CPU might be affected by the erratum - bit X86_BUG_AMD_E400 - but if the BIOS doesn't enter C1E, then the erratum doesn't come to manifest itself, i.e., we don't set X86_BUG_AMD_APIC_C1E.

If it is still not clear, read the erratum 400 description in the revision guide.

The code works perfectly fine.

Unless you're experiencing a problem with it. Then I'm all ears.

Signature · 5 lines
Regards/Gruss,
    Boris.

SUSE Linux GmbH, GF: Felix Imendörffer, Jane Smithard, Graham Norton, HRB 21284 (AG Nürnberg)
--