CR4 fixes (some are urgent)

PATCH In review Andy Lutomirski 4 participants 8 messages Sep 28, 2016 – Sep 29, 2016

Subsystems (5)

Stable branch x86 Hibernation Xen hypervisor x86 +1 more subsystems

lore

Share & tools

lore

Andy Lutomirski

author

2016-09-28 19:34 UTC #0 lore raw

cover CR4 fixes (some are urgent)

Hi Ingo, etc:

Patches 1 and 2 fix boot regressions. Patch 1 needs to be backported. Patch 2 doesn't because the regression is new in 4.8 (although it's probably unobservable because the regression fixed in patch 1 also prevented bootup on affected CPUs).

Patch 3 cleans the whole mess up. It's probably not 4.8 material, but I'm sending it as part of this series for ease of review. If you like, you could apply patches 1 and 2 to x86/urgent, merge into x86/asm, and apply patch 3 on top.

Andy Lutomirski (3):
x86/init: Fix cr4_init_shadow() on CR4-less machines x86/boot: Fix another __read_cr4() case on 486 x86/asm: Get rid of __read_cr4_safe()

 arch/x86/include/asm/paravirt.h       |  4 ----
 arch/x86/include/asm/paravirt_types.h |  1 -
 arch/x86/include/asm/special_insns.h  | 22 +++++++---------------
 arch/x86/kernel/paravirt.c            |  1 -
 arch/x86/kernel/process_32.c          |  2 +-
 arch/x86/kernel/setup.c               |  9 +++++++--
 arch/x86/power/cpu.c                  |  2 +-
 arch/x86/xen/enlighten.c              |  1 -
 8 files changed, 16 insertions(+), 26 deletions(-)

Signature · 1 line

2.7.4

Andy Lutomirski

author

2016-09-28 19:34 UTC #1 lore raw

PATCH 1/3 x86/init: Fix cr4_init_shadow() on CR4-less machines

cr4_init_shadow() will panic on 486-like machines without CR4. Fix it using __read_cr4_safe().

Cc [email protected]

Reported-by [email protected]

Fixes 1e02ce4cccdc ("x86: Store a per-cpu shadow copy of CR4")

Signed-off-by Andy Lutomirski <[email protected]>

---
 arch/x86/include/asm/tlbflush.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

arch/x86/include/asm/tlbflush.h +1 -1

index 6fa85944af83..dee8a70382ba 100644
@@ -81,7 +81,7 @@ DECLARE_PER_CPU_SHARED_ALIGNED(struct tlb_state, cpu_tlbstate);
 /* Initialize cr4 shadow for this CPU. */
 static inline void cr4_init_shadow(void)
 {
-	this_cpu_write(cpu_tlbstate.cr4, __read_cr4());
+	this_cpu_write(cpu_tlbstate.cr4, __read_cr4_safe());
 }
 
 /* Set in this cpu's CR4. */

Signature · 1 line

2.7.4

Borislav Petkov

maintainer Reviewed-by on v1 1/3

2016-09-29 11:44 UTC #2 lore raw

Re: [PATCH 1/3] x86/init: Fix cr4_init_shadow() on CR4-less machines

Quoted 17 lines · Andy Lutomirski

cr4_init_shadow() will panic on 486-like machines without CR4. Fix it using __read_cr4_safe().
Cc [email protected]
Reported-by [email protected]
Fixes 1e02ce4cccdc ("x86: Store a per-cpu shadow copy of CR4")
Signed-off-by Andy Lutomirski <[email protected]>
---
 arch/x86/include/asm/tlbflush.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
arch/x86/include/asm/tlbflush.h +0 -0
index 6fa85944af83..dee8a70382ba 100644

arch/x86/include/asm/tlbflush.h DECLARE_PER_CPU_SHARED_ALIGNED(struct tlb_state, cpu_tlbstate); · line 87

Quoted 8 lines

arch/x86/include/asm/tlbflush.h +1 -1

@@ -81,7 +81,7 @@ DECLARE_PER_CPU_SHARED_ALIGNED(struct tlb_state, cpu_tlbstate);
 /* Initialize cr4 shadow for this CPU. */
 static inline void cr4_init_shadow(void)
 {
-	this_cpu_write(cpu_tlbstate.cr4, __read_cr4());
+	this_cpu_write(cpu_tlbstate.cr4, __read_cr4_safe());
 }
 
 /* Set in this cpu's CR4. */

Reviewed-by Borislav Petkov <[email protected]>

Signature · 5 lines

Regards/Gruss,
    Boris.

ECO tip #101: Trim your mails when you reply.
--

tip-bot for Andy Lutomirski

merge bot

2016-09-29 18:24 UTC #3 lore raw

[tip:x86/urgent] x86/init: Fix cr4_init_shadow() on CR4-less machines

In reply to Andy Lutomirski

x86/init: Fix cr4_init_shadow() on CR4-less machines

Commit-ID:  e1bfc11c5a6f40222a698a818dc269113245820e
Gitweb:     http://git.kernel.org/tip/e1bfc11c5a6f40222a698a818dc269113245820e
Author:     Andy Lutomirski <[email protected]>
AuthorDate: Wed, 28 Sep 2016 12:34:14 -0700
Committer:  Ingo Molnar <[email protected]>
CommitDate: Thu, 29 Sep 2016 19:08:30 +0200

x86/init: Fix cr4_init_shadow() on CR4-less machines

cr4_init_shadow() will panic on 486-like machines without CR4. Fix it using __read_cr4_safe().

Reported-by [email protected]

Signed-off-by Andy Lutomirski <[email protected]>

Reviewed-by Borislav Petkov <[email protected]>

Cc Borislav Petkov <[email protected]>

Cc Brian Gerst <[email protected]>

Cc Denys Vlasenko <[email protected]>

Cc H. Peter Anvin <[email protected]>

Cc Josh Poimboeuf <[email protected]>

Cc Linus Torvalds <[email protected]>

Cc Peter Zijlstra <[email protected]>

Cc Thomas Gleixner <[email protected]>

Cc [email protected]

Fixes 1e02ce4cccdc ("x86: Store a per-cpu shadow copy of CR4")

Link http://lkml.kernel.org/r/43a20f81fb504013bf613913dc25574b45336a61.1475091074.git.luto@kernel.org open here

Signed-off-by Ingo Molnar <[email protected]>

---
 arch/x86/include/asm/tlbflush.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

arch/x86/include/asm/tlbflush.h +1 -1

index 6fa8594..dee8a70 100644
@@ -81,7 +81,7 @@ DECLARE_PER_CPU_SHARED_ALIGNED(struct tlb_state, cpu_tlbstate);
 /* Initialize cr4 shadow for this CPU. */
 static inline void cr4_init_shadow(void)
 {
-	this_cpu_write(cpu_tlbstate.cr4, __read_cr4());
+	this_cpu_write(cpu_tlbstate.cr4, __read_cr4_safe());
 }
 
 /* Set in this cpu's CR4. */

Andy Lutomirski

author

2016-09-28 19:34 UTC #4 lore raw

PATCH 2/3 x86/boot: Fix another __read_cr4() case on 486

In reply to Andy Lutomirski

The condition for reading CR4 was wrong: there are some CPUs with CPUID but not CR4. Rather than trying to make the condition exact, using __read_cr4_safe().

Reported-by [email protected]

Fixes 18bc7bd523e0 ("x86/boot: Synchronize trampoline_cr4_features and mmu_cr4_features directly")

Signed-off-by Andy Lutomirski <[email protected]>

---
 arch/x86/kernel/setup.c | 10 +++++++---
 1 file changed, 7 insertions(+), 3 deletions(-)

arch/x86/kernel/setup.c +7 -3

index 0fa60f5f5a16..5930a4d191b4 100644
@@ -1137,9 +1137,13 @@ void __init setup_arch(char **cmdline_p)
 	 * auditing all the early-boot CR4 manipulation would be needed to
 	 * rule it out.
 	 */
-	if (boot_cpu_data.cpuid_level >= 0)
-		/* A CPU has %cr4 if and only if it has CPUID. */
-		mmu_cr4_features = __read_cr4();
+	if (boot_cpu_data.cpuid_level >= 0) {
+		/*
+		 * CPUs without CPUID don't have CR4.  CPUs with CPUID
+		 * usually have CR4.
+		 */
+		mmu_cr4_features = __read_cr4_safe();
+	}
 
 	memblock_set_current_limit(get_max_mapped());

Signature · 1 line

2.7.4

Borislav Petkov

maintainer

2016-09-29 11:49 UTC #5 lore raw

Re: [PATCH 2/3] x86/boot: Fix another __read_cr4() case on 486

Quoted 19 lines · Andy Lutomirski

The condition for reading CR4 was wrong: there are some CPUs with CPUID but not CR4. Rather than trying to make the condition exact, using __read_cr4_safe().
Reported-by [email protected]
Fixes 18bc7bd523e0 ("x86/boot: Synchronize trampoline_cr4_features and mmu_cr4_features directly")
Signed-off-by Andy Lutomirski <[email protected]>
---
 arch/x86/kernel/setup.c | 10 +++++++---
 1 file changed, 7 insertions(+), 3 deletions(-)
arch/x86/kernel/setup.c +0 -0
index 0fa60f5f5a16..5930a4d191b4 100644
@@ -1137,9 +1137,13 @@ void __init setup_arch(char **cmdline_p)
 	 * auditing all the early-boot CR4 manipulation would be needed to
 	 * rule it out.
 	 */

arch/x86/kernel/setup.c void __init setup_arch(char **cmdline_p) · line 1146

Quoted 10 lines

arch/x86/kernel/setup.c +7 -3

-	if (boot_cpu_data.cpuid_level >= 0)
-		/* A CPU has %cr4 if and only if it has CPUID. */
-		mmu_cr4_features = __read_cr4();
+	if (boot_cpu_data.cpuid_level >= 0) {
+		/*
+		 * CPUs without CPUID don't have CR4.  CPUs with CPUID
+		 * usually have CR4.
+		 */
+		mmu_cr4_features = __read_cr4_safe();
+	}

Why are we even doing the CPUID check instead of unconditionally doing __read_cr4_safe()?

The safe variant will give 0 on !CR4 machines.

Signature · 5 lines

Regards/Gruss,
    Boris.

ECO tip #101: Trim your mails when you reply.
--

Andy Lutomirski

2016-09-29 19:09 UTC #6 lore raw

Re: [PATCH 2/3] x86/boot: Fix another __read_cr4() case on 486

Quoted 34 lines · Borislav Petkov

Quoted 19 lines · Andy Lutomirski
The condition for reading CR4 was wrong: there are some CPUs with CPUID but not CR4. Rather than trying to make the condition exact, using __read_cr4_safe().
Reported-by [email protected]
Fixes 18bc7bd523e0 ("x86/boot: Synchronize trampoline_cr4_features and mmu_cr4_features directly")
Signed-off-by Andy Lutomirski <[email protected]>
---
 arch/x86/kernel/setup.c | 10 +++++++---
 1 file changed, 7 insertions(+), 3 deletions(-)
arch/x86/kernel/setup.c +0 -0
index 0fa60f5f5a16..5930a4d191b4 100644
@@ -1137,9 +1137,13 @@ void __init setup_arch(char **cmdline_p)
       * auditing all the early-boot CR4 manipulation would be needed to
       * rule it out.
       */
arch/x86/kernel/setup.c void __init setup_arch(char **cmdline_p) · line 1146
Quoted 10 lines
arch/x86/kernel/setup.c +7 -3
-     if (boot_cpu_data.cpuid_level >= 0)
-             /* A CPU has %cr4 if and only if it has CPUID. */
-             mmu_cr4_features = __read_cr4();
+     if (boot_cpu_data.cpuid_level >= 0) {
+             /*
+              * CPUs without CPUID don't have CR4.  CPUs with CPUID
+              * usually have CR4.
+              */
+             mmu_cr4_features = __read_cr4_safe();
+     }
Why are we even doing the CPUID check instead of unconditionally doing __read_cr4_safe()?

Quoted 1 line

The safe variant will give 0 on !CR4 machines.

Will do.

Andy Lutomirski

author

2016-09-28 19:34 UTC #7 lore raw

PATCH 3/3 x86/asm: Get rid of __read_cr4_safe()

In reply to Andy Lutomirski

We use __read_cr4() vs __read_cr4_safe() inconsistently. On CR4-less CPUs, all CR4 bits are effectively clear, so we can make the code simpler and more robust by making __read_cr4() always fix up faults on 32-bit kernels.

This may fix some bugs on old 486-like CPUs, but I don't have any easy way to test that.

Cc [email protected]

Signed-off-by Andy Lutomirski <[email protected]>

---
 arch/x86/include/asm/paravirt.h       |  4 ----
 arch/x86/include/asm/paravirt_types.h |  1 -
 arch/x86/include/asm/special_insns.h  | 22 +++++++---------------
 arch/x86/include/asm/tlbflush.h       |  2 +-
 arch/x86/kernel/paravirt.c            |  1 -
 arch/x86/kernel/process_32.c          |  2 +-
 arch/x86/kernel/setup.c               |  5 +++--
 arch/x86/power/cpu.c                  |  2 +-
 arch/x86/xen/enlighten.c              |  1 -
 9 files changed, 13 insertions(+), 27 deletions(-)

arch/x86/include/asm/paravirt.h +0 -4

index 2970d22d7766..91b6f4eed3fd 100644
@@ -80,10 +80,6 @@ static inline unsigned long __read_cr4(void)
 {
 	return PVOP_CALL0(unsigned long, pv_cpu_ops.read_cr4);
 }
-static inline unsigned long __read_cr4_safe(void)
-{
-	return PVOP_CALL0(unsigned long, pv_cpu_ops.read_cr4_safe);
-}
 
 static inline void __write_cr4(unsigned long x)
 {

arch/x86/include/asm/paravirt_types.h +0 -1

index 7fa9e7740ba3..fcf243f077ac 100644
@@ -108,7 +108,6 @@ struct pv_cpu_ops {
 	unsigned long (*read_cr0)(void);
 	void (*write_cr0)(unsigned long);
 
-	unsigned long (*read_cr4_safe)(void);
 	unsigned long (*read_cr4)(void);
 	void (*write_cr4)(unsigned long);

arch/x86/include/asm/special_insns.h +7 ~~-15~~

index 587d7914ea4b..19a2224f9e16 100644
@@ -59,22 +59,19 @@ static inline void native_write_cr3(unsigned long val)
 static inline unsigned long native_read_cr4(void)
 {
 	unsigned long val;
-	asm volatile("mov %%cr4,%0\n\t" : "=r" (val), "=m" (__force_order));
-	return val;
-}
-
-static inline unsigned long native_read_cr4_safe(void)
-{
-	unsigned long val;
-	/* This could fault if %cr4 does not exist. In x86_64, a cr4 always
-	 * exists, so it will never fail. */
 #ifdef CONFIG_X86_32
+	/*
+	 * This could fault if CR4 does not exist.  Non-existent CR4
+	 * is functionally equivalent to CR4 == 0.  Keep it simple and pretend
+	 * that CR4 == 0 on CPUs that don't have CR4.
+	 */
 	asm volatile("1: mov %%cr4, %0\n"
 		     "2:\n"
 		     _ASM_EXTABLE(1b, 2b)
 		     : "=r" (val), "=m" (__force_order) : "0" (0));
 #else
-	val = native_read_cr4();
+	/* CR4 always exists on x86_64. */
+	asm volatile("mov %%cr4,%0\n\t" : "=r" (val), "=m" (__force_order));
 #endif
 	return val;
 }

@@ -182,11 +179,6 @@ static inline unsigned long __read_cr4(void)
 	return native_read_cr4();
 }
 
-static inline unsigned long __read_cr4_safe(void)
-{
-	return native_read_cr4_safe();
-}
-
 static inline void __write_cr4(unsigned long x)
 {
 	native_write_cr4(x);

arch/x86/include/asm/tlbflush.h +1 -1

index dee8a70382ba..6fa85944af83 100644
@@ -81,7 +81,7 @@ DECLARE_PER_CPU_SHARED_ALIGNED(struct tlb_state, cpu_tlbstate);
 /* Initialize cr4 shadow for this CPU. */
 static inline void cr4_init_shadow(void)
 {
-	this_cpu_write(cpu_tlbstate.cr4, __read_cr4_safe());
+	this_cpu_write(cpu_tlbstate.cr4, __read_cr4());
 }
 
 /* Set in this cpu's CR4. */

arch/x86/kernel/paravirt.c +0 -1

index 1acfd76e3e26..ed03b1c5b1b9 100644
@@ -332,7 +332,6 @@ __visible struct pv_cpu_ops pv_cpu_ops = {
 	.read_cr0 = native_read_cr0,
 	.write_cr0 = native_write_cr0,
 	.read_cr4 = native_read_cr4,
-	.read_cr4_safe = native_read_cr4_safe,
 	.write_cr4 = native_write_cr4,
 #ifdef CONFIG_X86_64
 	.read_cr8 = native_read_cr8,

arch/x86/kernel/process_32.c +1 -1

index d86be29c38c7..e5da9ba220d1 100644
@@ -101,7 +101,7 @@ void __show_regs(struct pt_regs *regs, int all)
 	cr0 = read_cr0();
 	cr2 = read_cr2();
 	cr3 = read_cr3();
-	cr4 = __read_cr4_safe();
+	cr4 = __read_cr4();
 	printk(KERN_DEFAULT "CR0: %08lx CR2: %08lx CR3: %08lx CR4: %08lx\n",
 			cr0, cr2, cr3, cr4);

arch/x86/kernel/setup.c +3 -2

index 5930a4d191b4..09d245e8e2da 100644
@@ -1140,9 +1140,10 @@ void __init setup_arch(char **cmdline_p)
 	if (boot_cpu_data.cpuid_level >= 0) {
 		/*
 		 * CPUs without CPUID don't have CR4.  CPUs with CPUID
-		 * usually have CR4.
+		 * usually have CR4.  On CPUID-less CPUs, this avoids
+		 * an unnecessary fixup, mainly for historical reasons.
 		 */
-		mmu_cr4_features = __read_cr4_safe();
+		mmu_cr4_features = __read_cr4();
 	}
 
 	memblock_set_current_limit(get_max_mapped());

arch/x86/power/cpu.c +1 -1

index b12c26e2e309..53cace2ec0e2 100644
@@ -130,7 +130,7 @@ static void __save_processor_state(struct saved_context *ctxt)
 	ctxt->cr0 = read_cr0();
 	ctxt->cr2 = read_cr2();
 	ctxt->cr3 = read_cr3();
-	ctxt->cr4 = __read_cr4_safe();
+	ctxt->cr4 = __read_cr4();
 #ifdef CONFIG_X86_64
 	ctxt->cr8 = read_cr8();
 #endif

arch/x86/xen/enlighten.c +0 -1

index b86ebb1a9a7f..e2cf8fcea6bb 100644
@@ -1237,7 +1237,6 @@ static const struct pv_cpu_ops xen_cpu_ops __initconst = {
 	.write_cr0 = xen_write_cr0,
 
 	.read_cr4 = native_read_cr4,
-	.read_cr4_safe = native_read_cr4_safe,
 	.write_cr4 = xen_write_cr4,
 
 #ifdef CONFIG_X86_64

Signature · 1 line

2.7.4