TSX update

PATCH In review Pawan Gupta 3 participants 8 messages Mar 10, 2022 – Apr 6, 2022

Subsystems (2)

Stable branch x86

lore

Share & tools

lore

Pawan Gupta

author

2022-03-10 21:59 UTC #0 lore raw

v2 cover TSX update

v2:
- Added patch to disable TSX development mode (Andrew, Boris)
- Rebased to v5.17-rc7

v1: https://lore.kernel.org/lkml/5bd785a1d6ea0b572250add0c6617b4504bc24d1.1644440311.git.pawan.kumar.gupta@linux.intel.com/

Hi,

After a recent microcode update some Intel processors will always abort Transactional Synchronization Extensions (TSX) transactions [*]. On these processors a new CPUID bit, CPUID.07H.0H.EDX[11](RTM_ALWAYS_ABORT), will be enumerated to indicate that the loaded microcode is forcing Restricted Transactional Memory (RTM) abort. If the processor enumerated support for RTM previously, the CPUID enumeration bits for TSX (CPUID.RTM and CPUID.HLE) continue to be set by default after the microcode update.

First patch in this series clears CPUID.RTM and CPUID.HLE so that userspace doesn't enumerate TSX feature.

Microcode also added support to re-enable TSX for development purpose, doing so is not recommended for production deployments, because MD_CLEAR flows for the mitigation of TSX Asynchronous Abort (TAA) may not be effective on these processors when TSX is enabled with updated microcode.

Second patch unconditionally disables this TSX development mode, in case it was enabled by the software running before kernel boot.

Thanks,
Pawan

[*] Intel Transactional Synchronization Extension (Intel TSX) Disable Update for Selected Processors

    https://cdrdv2.intel.com/v1/dl/getContent/643557

Pawan Gupta (2):
x86/tsx: Use MSR_TSX_CTRL to clear CPUID bits
x86/tsx: Disable TSX development mode at boot

 arch/x86/include/asm/msr-index.h       |  4 +-
 arch/x86/kernel/cpu/cpu.h              |  1 +
 arch/x86/kernel/cpu/intel.c            |  5 ++
 arch/x86/kernel/cpu/tsx.c              | 88 ++++++++++++++++++++++++--
 tools/arch/x86/include/asm/msr-index.h |  4 +-
 5 files changed, 91 insertions(+), 11 deletions(-)

base-commit: ffb217a13a2eaf6d5bd974fc83036a53ca69f1e2

Signature · 1 line

2.25.1

Pawan Gupta

author

2022-03-10 22:01 UTC #1 lore raw

PATCH v2 1/2 x86/tsx: Use MSR_TSX_CTRL to clear CPUID bits

tsx_clear_cpuid() uses MSR_TSX_FORCE_ABORT to clear CPUID.RTM and CPUID.HLE. Not all CPUs support MSR_TSX_FORCE_ABORT, alternatively use MSR_IA32_TSX_CTRL when supported.

[ bp: Document how and why TSX gets disabled. ]

Fixes 293649307ef9 ("x86/tsx: Clear CPUID bits when TSX always force aborts")

Reported-by kernel test robot <[email protected]>

Signed-off-by Pawan Gupta <[email protected]>

Signed-off-by Borislav Petkov <[email protected]>

Tested-by Neelima Krishnan <[email protected]>

Cc <[email protected]>

---
 arch/x86/kernel/cpu/intel.c |  1 +
 arch/x86/kernel/cpu/tsx.c   | 54 ++++++++++++++++++++++++++++++++-----
 2 files changed, 48 insertions(+), 7 deletions(-)

arch/x86/kernel/cpu/intel.c +1 -0

index 8321c43554a1..8abf995677a4 100644
@@ -722,6 +722,7 @@ static void init_intel(struct cpuinfo_x86 *c)
 	else if (tsx_ctrl_state == TSX_CTRL_DISABLE)
 		tsx_disable();
 	else if (tsx_ctrl_state == TSX_CTRL_RTM_ALWAYS_ABORT)
+		/* See comment over that function for more details. */
 		tsx_clear_cpuid();
 
 	split_lock_init();

arch/x86/kernel/cpu/tsx.c +47 -7

79 lines not on this page · raw message

Signature · 1 line

2.25.1

Pawan Gupta

author

2022-03-10 22:02 UTC #2 lore raw

PATCH v2 2/2 x86/tsx: Disable TSX development mode at boot

In reply to Pawan Gupta

A microcode update on some Intel processors causes all TSX transactions to always abort by default [*]. Microcode also added functionality to re-enable TSX for development purpose. With this microcode loaded, if tsx=on was passed on the cmdline, and TSX development mode was already enabled before the kernel boot, it may make the system vulnerable to TSX Asynchronous Abort (TAA).

To be on safer side, unconditionally disable TSX development mode at boot. If needed, a user can enable it using msr-tools.

[*] Intel Transactional Synchronization Extension (Intel TSX) Disable Update for Selected Processors

    https://cdrdv2.intel.com/v1/dl/getContent/643557

Signed-off-by Pawan Gupta <[email protected]>

Cc <[email protected]>

---
 arch/x86/include/asm/msr-index.h       |  4 +--
 arch/x86/kernel/cpu/cpu.h              |  1 +
 arch/x86/kernel/cpu/intel.c            |  4 +++
 arch/x86/kernel/cpu/tsx.c              | 34 ++++++++++++++++++++++++++
 tools/arch/x86/include/asm/msr-index.h |  4 +--
 5 files changed, 43 insertions(+), 4 deletions(-)

arch/x86/include/asm/msr-index.h +2 -2

index a4a39c3e0f19..0c2610cde6ea 100644
@@ -128,9 +128,9 @@
 #define TSX_CTRL_RTM_DISABLE		BIT(0)	/* Disable RTM feature */
 #define TSX_CTRL_CPUID_CLEAR		BIT(1)	/* Disable TSX enumeration */
 
-/* SRBDS support */
 #define MSR_IA32_MCU_OPT_CTRL		0x00000123
-#define RNGDS_MITG_DIS			BIT(0)
+#define RNGDS_MITG_DIS			BIT(0)	/* SRBDS support */
+#define RTM_ALLOW			BIT(1)	/* TSX development mode */
 
 #define MSR_IA32_SYSENTER_CS		0x00000174
 #define MSR_IA32_SYSENTER_ESP		0x00000175

arch/x86/kernel/cpu/cpu.h +1 -0

index ee6f23f7587d..628d18062372 100644
@@ -58,6 +58,7 @@ extern void __init tsx_init(void);
 extern void tsx_enable(void);
 extern void tsx_disable(void);
 extern void tsx_clear_cpuid(void);
+extern bool tsx_dev_mode_disable(void);
 #else
 static inline void tsx_init(void) { }
 #endif /* CONFIG_CPU_SUP_INTEL */

arch/x86/kernel/cpu/intel.c +4 -0

index 8abf995677a4..46cb5a18bd97 100644
@@ -717,6 +717,10 @@ static void init_intel(struct cpuinfo_x86 *c)
 
 	init_intel_misc_features(c);
 
+	/* Boot CPU is handled in tsx_init() */
+	if (c->cpu_index != boot_cpu_data.cpu_index)
+		tsx_dev_mode_disable();
+
 	if (tsx_ctrl_state == TSX_CTRL_ENABLE)
 		tsx_enable();
 	else if (tsx_ctrl_state == TSX_CTRL_DISABLE)

arch/x86/kernel/cpu/tsx.c +34 -0

index 2835fa89fc6f..513e479bca2e 100644
@@ -142,11 +142,45 @@ void tsx_clear_cpuid(void)
 	}
 }
 
+/*
+ * Disable TSX development mode
+ *
+ * When the microcode released in Feb 2022 is applied, TSX will be disabled by
+ * default on some processors. MSR 0x122 (TSX_CTRL) and MSR 0x123
+ * (IA32_MCU_OPT_CTRL) can be used to re-enable TSX for development, doing so is
+ * not recommended for production deployments. In particular, applying MD_CLEAR
+ * flows for mitigation of the Intel TSX Asynchronous Abort (TAA) transient
+ * execution attack may not be effective on these processors when Intel TSX is
+ * enabled with updated microcode.
+ */
+bool tsx_dev_mode_disable(void)
+{
+	u64 mcu_opt_ctrl;
+
+	/* Check if RTM_ALLOW exists */
+	if (!boot_cpu_has_bug(X86_BUG_TAA) || !tsx_ctrl_is_supported() ||
+	    !boot_cpu_has(X86_FEATURE_SRBDS_CTRL))
+		return false;
+
+	rdmsrl(MSR_IA32_MCU_OPT_CTRL, mcu_opt_ctrl);
+
+	if (mcu_opt_ctrl & RTM_ALLOW) {
+		mcu_opt_ctrl &= ~RTM_ALLOW;
+		wrmsrl(MSR_IA32_MCU_OPT_CTRL, mcu_opt_ctrl);
+		return true;
+	}
+
+	return false;
+}
+
 void __init tsx_init(void)
 {
 	char arg[5] = {};
 	int ret;
 
+	if (tsx_dev_mode_disable())
+		setup_force_cpu_cap(X86_FEATURE_RTM_ALWAYS_ABORT);
+
 	/*
 	 * Hardware will always abort a TSX transaction when CPUID
 	 * RTM_ALWAYS_ABORT is set. In this case, it is better not to enumerate

tools/arch/x86/include/asm/msr-index.h +2 -2

index a4a39c3e0f19..0c2610cde6ea 100644
@@ -128,9 +128,9 @@
 #define TSX_CTRL_RTM_DISABLE		BIT(0)	/* Disable RTM feature */
 #define TSX_CTRL_CPUID_CLEAR		BIT(1)	/* Disable TSX enumeration */
 
-/* SRBDS support */
 #define MSR_IA32_MCU_OPT_CTRL		0x00000123
-#define RNGDS_MITG_DIS			BIT(0)
+#define RNGDS_MITG_DIS			BIT(0)	/* SRBDS support */
+#define RTM_ALLOW			BIT(1)	/* TSX development mode */
 
 #define MSR_IA32_SYSENTER_CS		0x00000174
 #define MSR_IA32_SYSENTER_ESP		0x00000175

Signature · 1 line

2.25.1

Pawan Gupta

author

2022-03-22 23:33 UTC #7 lore raw

In reply to Pawan Gupta

Hi Boris and others,

Quoted 32 lines · Pawan Gupta

v2:
- Added patch to disable TSX development mode (Andrew, Boris)
- Rebased to v5.17-rc7
v1: https://lore.kernel.org/lkml/5bd785a1d6ea0b572250add0c6617b4504bc24d1.1644440311.git.pawan.kumar.gupta@linux.intel.com/
Hi,
After a recent microcode update some Intel processors will always abort Transactional Synchronization Extensions (TSX) transactions [*]. On these processors a new CPUID bit, CPUID.07H.0H.EDX[11](RTM_ALWAYS_ABORT), will be enumerated to indicate that the loaded microcode is forcing Restricted Transactional Memory (RTM) abort. If the processor enumerated support for RTM previously, the CPUID enumeration bits for TSX (CPUID.RTM and CPUID.HLE) continue to be set by default after the microcode update.
First patch in this series clears CPUID.RTM and CPUID.HLE so that userspace doesn't enumerate TSX feature.
Microcode also added support to re-enable TSX for development purpose, doing so is not recommended for production deployments, because MD_CLEAR flows for the mitigation of TSX Asynchronous Abort (TAA) may not be effective on these processors when TSX is enabled with updated microcode.
Second patch unconditionally disables this TSX development mode, in case it was enabled by the software running before kernel boot.
Thanks,
Pawan

Quoted 6 lines

[*] Intel Transactional Synchronization Extension (Intel TSX) Disable Update for Selected Processors
https://cdrdv2.intel.com/v1/dl/getContent/643557
Pawan Gupta (2):
x86/tsx: Use MSR_TSX_CTRL to clear CPUID bits
x86/tsx: Disable TSX development mode at boot

I am hoping to get some feedback on v2. Are these patches looking okay?

Thanks,
Pawan

Borislav Petkov

maintainer

2022-03-29 16:24 UTC #3 lore raw

Re: [PATCH v2 2/2] x86/tsx: Disable TSX development mode at boot

In reply to Pawan Gupta

Quoted 17 lines · Pawan Gupta

A microcode update on some Intel processors causes all TSX transactions to always abort by default [*]. Microcode also added functionality to re-enable TSX for development purpose. With this microcode loaded, if tsx=on was passed on the cmdline, and TSX development mode was already enabled before the kernel boot, it may make the system vulnerable to TSX Asynchronous Abort (TAA).
To be on safer side, unconditionally disable TSX development mode at boot. If needed, a user can enable it using msr-tools.
[*] Intel Transactional Synchronization Extension (Intel TSX) Disable Update for Selected Processors
    https://cdrdv2.intel.com/v1/dl/getContent/643557
Suggested-by Andrew Cooper <[email protected]>
Suggested-by Borislav Petkov <[email protected]>
Signed-off-by Pawan Gupta <[email protected]>
Cc <[email protected]>

Quoted 7 lines

---
 arch/x86/include/asm/msr-index.h       |  4 +--
 arch/x86/kernel/cpu/cpu.h              |  1 +
 arch/x86/kernel/cpu/intel.c            |  4 +++
 arch/x86/kernel/cpu/tsx.c              | 34 ++++++++++++++++++++++++++
 tools/arch/x86/include/asm/msr-index.h |  4 +--
 5 files changed, 43 insertions(+), 4 deletions(-)

Does this a lot more encapsulated version work too?

---
From: Pawan Gupta <[email protected]>
Date: Thu, 10 Mar 2022 14:02:09 -0800
Subject: [PATCH] x86/tsx: Disable TSX development mode at boot

A microcode update on some Intel processors causes all TSX transactions to always abort by default[*]. Microcode also added functionality to re-enable TSX for development purposes. With this microcode loaded, if tsx=on was passed on the cmdline, and TSX development mode was already enabled before the kernel boot, it may make the system vulnerable to TSX Asynchronous Abort (TAA).

To be on safer side, unconditionally disable TSX development mode during boot. If a viable use case appears, this can be revisited later.

[*]: Intel TSX Disable Update for Selected Processors, doc ID: 643557

[ bp: Drop unstable web link, massage heavily. ]

Signed-off-by Pawan Gupta <[email protected]>

Signed-off-by Borislav Petkov <[email protected]>

Cc <[email protected]>

Link https://lore.kernel.org/r/347bd844da3a333a9793c6687d4e4eb3b2419a3e.1646943780.git.pawan.kumar.gupta@linux.intel.com open here

---
 arch/x86/include/asm/msr-index.h       |  4 +--
 arch/x86/kernel/cpu/common.c           |  2 ++
 arch/x86/kernel/cpu/cpu.h              |  5 ++-
 arch/x86/kernel/cpu/intel.c            |  8 -----
 arch/x86/kernel/cpu/tsx.c              | 50 ++++++++++++++++++++++++--
 tools/arch/x86/include/asm/msr-index.h |  4 +--
 6 files changed, 55 insertions(+), 18 deletions(-)

arch/x86/include/asm/msr-index.h +2 -2

index 9f1741ac4769..adce6a17770b 100644
@@ -128,9 +128,9 @@
 #define TSX_CTRL_RTM_DISABLE		BIT(0)	/* Disable RTM feature */
 #define TSX_CTRL_CPUID_CLEAR		BIT(1)	/* Disable TSX enumeration */
 
-/* SRBDS support */
 #define MSR_IA32_MCU_OPT_CTRL		0x00000123
-#define RNGDS_MITG_DIS			BIT(0)
+#define RNGDS_MITG_DIS			BIT(0)	/* SRBDS support */
+#define RTM_ALLOW			BIT(1)	/* TSX development mode */
 
 #define MSR_IA32_SYSENTER_CS		0x00000174
 #define MSR_IA32_SYSENTER_ESP		0x00000175

arch/x86/kernel/cpu/common.c +2 -0

index 64deb7727d00..4616753d1510 100644
@@ -1798,6 +1798,8 @@ void identify_secondary_cpu(struct cpuinfo_x86 *c)
 	validate_apic_and_package_id(c);
 	x86_spec_ctrl_setup_ap();
 	update_srbds_msr();
+
+	tsx_ap_init();
 }
 
 static __init int setup_noclflush(char *arg)

arch/x86/kernel/cpu/cpu.h +2 -3

index ee6f23f7587d..2a8e584fc991 100644
@@ -55,11 +55,10 @@ enum tsx_ctrl_states {
 extern __ro_after_init enum tsx_ctrl_states tsx_ctrl_state;
 
 extern void __init tsx_init(void);
-extern void tsx_enable(void);
-extern void tsx_disable(void);
-extern void tsx_clear_cpuid(void);
+void tsx_ap_init(void);
 #else
 static inline void tsx_init(void) { }
+static inline void tsx_ap_init(void) { }
 #endif /* CONFIG_CPU_SUP_INTEL */
 
 extern void get_cpu_cap(struct cpuinfo_x86 *c);

arch/x86/kernel/cpu/intel.c +0 -8

index 8abf995677a4..f7a5370a9b3b 100644
@@ -717,14 +717,6 @@ static void init_intel(struct cpuinfo_x86 *c)
 
 	init_intel_misc_features(c);
 
-	if (tsx_ctrl_state == TSX_CTRL_ENABLE)
-		tsx_enable();
-	else if (tsx_ctrl_state == TSX_CTRL_DISABLE)
-		tsx_disable();
-	else if (tsx_ctrl_state == TSX_CTRL_RTM_ALWAYS_ABORT)
-		/* See comment over that function for more details. */
-		tsx_clear_cpuid();
-
 	split_lock_init();
 	bus_lock_init();

arch/x86/kernel/cpu/tsx.c +47 -3

82 lines not on this page · raw message

tools/arch/x86/include/asm/msr-index.h +2 -2

index a4a39c3e0f19..0c2610cde6ea 100644
@@ -128,9 +128,9 @@
 #define TSX_CTRL_RTM_DISABLE		BIT(0)	/* Disable RTM feature */
 #define TSX_CTRL_CPUID_CLEAR		BIT(1)	/* Disable TSX enumeration */
 
-/* SRBDS support */
 #define MSR_IA32_MCU_OPT_CTRL		0x00000123
-#define RNGDS_MITG_DIS			BIT(0)
+#define RNGDS_MITG_DIS			BIT(0)	/* SRBDS support */
+#define RTM_ALLOW			BIT(1)	/* TSX development mode */
 
 #define MSR_IA32_SYSENTER_CS		0x00000174
 #define MSR_IA32_SYSENTER_ESP		0x00000175

Signature · 7 lines

2.35.1

-- 
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette

Pawan Gupta

author

2022-03-29 22:47 UTC #4 lore raw

Re: [PATCH v2 2/2] x86/tsx: Disable TSX development mode at boot

Quoted 26 lines · Borislav Petkov

Quoted 17 lines · Pawan Gupta
A microcode update on some Intel processors causes all TSX transactions to always abort by default [*]. Microcode also added functionality to re-enable TSX for development purpose. With this microcode loaded, if tsx=on was passed on the cmdline, and TSX development mode was already enabled before the kernel boot, it may make the system vulnerable to TSX Asynchronous Abort (TAA).
To be on safer side, unconditionally disable TSX development mode at boot. If needed, a user can enable it using msr-tools.
[*] Intel Transactional Synchronization Extension (Intel TSX) Disable Update for Selected Processors
    https://cdrdv2.intel.com/v1/dl/getContent/643557
Suggested-by Andrew Cooper <[email protected]>
Suggested-by Borislav Petkov <[email protected]>
Signed-off-by Pawan Gupta <[email protected]>
Cc <[email protected]>
Quoted 7 lines
---
 arch/x86/include/asm/msr-index.h       |  4 +--
 arch/x86/kernel/cpu/cpu.h              |  1 +
 arch/x86/kernel/cpu/intel.c            |  4 +++
 arch/x86/kernel/cpu/tsx.c              | 34 ++++++++++++++++++++++++++
 tools/arch/x86/include/asm/msr-index.h |  4 +--
 5 files changed, 43 insertions(+), 4 deletions(-)

Quoted 1 line

Does this a lot more encapsulated version work too?

It look good to me.

Thanks,
Pawan

Pawan Gupta

author

2022-03-30 05:27 UTC #5 lore raw

Re: [PATCH v2 2/2] x86/tsx: Disable TSX development mode at boot

In reply to Borislav Petkov

Quoted 26 lines · Borislav Petkov

Quoted 17 lines · Pawan Gupta
A microcode update on some Intel processors causes all TSX transactions to always abort by default [*]. Microcode also added functionality to re-enable TSX for development purpose. With this microcode loaded, if tsx=on was passed on the cmdline, and TSX development mode was already enabled before the kernel boot, it may make the system vulnerable to TSX Asynchronous Abort (TAA).
To be on safer side, unconditionally disable TSX development mode at boot. If needed, a user can enable it using msr-tools.
[*] Intel Transactional Synchronization Extension (Intel TSX) Disable Update for Selected Processors
    https://cdrdv2.intel.com/v1/dl/getContent/643557
Suggested-by Andrew Cooper <[email protected]>
Suggested-by Borislav Petkov <[email protected]>
Signed-off-by Pawan Gupta <[email protected]>
Cc <[email protected]>
Quoted 7 lines
---
 arch/x86/include/asm/msr-index.h       |  4 +--
 arch/x86/kernel/cpu/cpu.h              |  1 +
 arch/x86/kernel/cpu/intel.c            |  4 +++
 arch/x86/kernel/cpu/tsx.c              | 34 ++++++++++++++++++++++++++
 tools/arch/x86/include/asm/msr-index.h |  4 +--
 5 files changed, 43 insertions(+), 4 deletions(-)

Quoted 1 line

Does this a lot more encapsulated version work too?

Neelima is testing this patch, she will share the results tomorrow.

Thanks,
Pawan

Krishnan, Neelima

2022-04-06 20:53 UTC #6 lore raw

RE: [PATCH v2 2/2] x86/tsx: Disable TSX development mode at boot

-----Original Message-----
From: Pawan Gupta <[email protected]>
Sent: Tuesday, March 29, 2022 10:28 PM
To: Borislav Petkov <[email protected]>
Cc: Thomas Gleixner <[email protected]>; Ingo Molnar <[email protected]>; Dave Hansen <[email protected]>; [email protected]; H. Peter Anvin <[email protected]>; Andi Kleen <[email protected]>; Luck, Tony <[email protected]>; [email protected]; [email protected]; Krishnan, Neelima <[email protected]>; [email protected]; Cooper, Andrew <[email protected]>; Poimboe, Josh <[email protected]>
Subject: Re: [PATCH v2 2/2] x86/tsx: Disable TSX development mode at boot

Quoted 26 lines · Borislav Petkov

Quoted 17 lines · Pawan Gupta
A microcode update on some Intel processors causes all TSX transactions to always abort by default [*]. Microcode also added functionality to re-enable TSX for development purpose. With this microcode loaded, if tsx=on was passed on the cmdline, and TSX development mode was already enabled before the kernel boot, it may make the system vulnerable to TSX Asynchronous Abort (TAA).
To be on safer side, unconditionally disable TSX development mode at boot. If needed, a user can enable it using msr-tools.
[*] Intel Transactional Synchronization Extension (Intel TSX) Disable Update for Selected Processors
    https://cdrdv2.intel.com/v1/dl/getContent/643557
Suggested-by Andrew Cooper <[email protected]>
Suggested-by Borislav Petkov <[email protected]>
Signed-off-by Pawan Gupta <[email protected]>
Cc <[email protected]>
Quoted 7 lines
---
 arch/x86/include/asm/msr-index.h       |  4 +--
 arch/x86/kernel/cpu/cpu.h              |  1 +
 arch/x86/kernel/cpu/intel.c            |  4 +++
 arch/x86/kernel/cpu/tsx.c              | 34 ++++++++++++++++++++++++++
 tools/arch/x86/include/asm/msr-index.h |  4 +--
 5 files changed, 43 insertions(+), 4 deletions(-)

Quoted 3 lines

Does this a lot more encapsulated version work too?
Neelima is testing this patch, she will share the results tomorrow.

Following up on this email thread, I did some basic functional validation of the patch[1].
Initially I ran into the bug where the mitigation was getting disabled in one CPU after a suspend/resume [2].
But after applying the patch [1] on latest upstream, with the fix for restoring speculation related MSRs during s3 resume [2], my tests are passing.

Quick summary of testcases executed:
Testcase 1: Verify RTM_ALLOW was getting reset after kexec reboot
Testcase2: Verify TSX_CTRL_MSR is restored after system goes to S3 suspend state

[1] https://lore.kernel.org/lkml/[email protected]/
[2] https://github.com/torvalds/linux/commit/e2a1256b17b16f9b9adf1b6fea56819e7b68e463

Thanks
Neelima