Skip to contentSkip to conversation

Fix TSC ADJUST breakage causing TSC failure

PATCH New Mike Travis 2 participants 5 messages Oct 2, 2018

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

cover Fix TSC ADJUST breakage causing TSC failure

Fix a breakage caused by enabling early tsc initialization which bypasses a check that disables the forcing of TSC ADJUST to 0 for chassis 0. This is common on systems where all the chassis start up asynchronously so which chassis should have a TSC ADJUST value of 0 is not predictable.

The solution is to add a check earlier than this early tsc init to disable the potential of it incorrectly adjusting TSC ADJUST values that are already correctly initialized.

* Patch 1 adds an early callable function (after efi_init) that will check if this system might be a UV system.

* Patch 2 adds code to tsc_early_init() which disables adjusting the TSC ADJUST value if it's a UV system. This allows the later tsc_init function to test the tsc_async_resets flag that indicates the system chassis start up asynchronously, so which chassis should have a TSC ADJUST value of 0 is not predictable. Further references are in the patch.

#1 lore raw

PATCH 1/2 x86/platform/uv: Add is_early_uv_system check

Introduce is_early_uv_system() which uses efi.uv_systab to decide early in the boot process whether we're on a UV system.

This is needed to skip other early setup/init code that might break the UV platform if done too early such as before necessary ACPI tables parsing takes place.

---
 arch/x86/include/asm/uv/uv.h |    6 ++++++
 1 file changed, 6 insertions(+)
arch/x86/include/asm/uv/uv.h +6 -0
@@ -10,8 +10,13 @@ struct cpumask;
 struct mm_struct;
 
 #ifdef CONFIG_X86_UV
+#include <linux/efi.h>
 
 extern enum uv_system_type get_uv_system_type(void);
+static inline bool is_early_uv_system(void)
+{
+	return !((efi.uv_systab == EFI_INVALID_TABLE_ADDR) || !efi.uv_systab);
+}
 extern int is_uv_system(void);
 extern int is_uv_hubless(void);
 extern void uv_cpu_init(void);
@@ -23,6 +28,7 @@ extern const struct cpumask *uv_flush_tl
 #else	/* X86_UV */
 
 static inline enum uv_system_type get_uv_system_type(void) { return UV_NONE; }
+static inline bool is_early_uv_system(void)	{ return 0; }
 static inline int is_uv_system(void)	{ return 0; }
 static inline int is_uv_hubless(void)	{ return 0; }
 static inline void uv_cpu_init(void)	{ }
#2 lore raw

[tip:x86/urgent] x86/platform/uv: Provide is_early_uv_system()

x86/platform/uv: Provide is_early_uv_system()
Commit-ID:  20a8378aa9dd108a01cb0e695599f5257a885c4b
Gitweb:     https://git.kernel.org/tip/20a8378aa9dd108a01cb0e695599f5257a885c4b
Author:     Mike Travis <[email protected]>
AuthorDate: Tue, 2 Oct 2018 13:01:45 -0500
Committer:  Thomas Gleixner <[email protected]>
CommitDate: Tue, 2 Oct 2018 21:29:16 +0200

x86/platform/uv: Provide is_early_uv_system()

Introduce is_early_uv_system() which uses efi.uv_systab to decide early in the boot process whether the kernel runs on a UV system.

This is needed to skip other early setup/init code that might break the UV platform if done too early such as before necessary ACPI tables parsing takes place.

Cc "H. Peter Anvin" <[email protected]>
Cc Russ Anderson <[email protected]>
Cc Dimitri Sivanich <[email protected]>
Cc Borislav Petkov <[email protected]>
Cc Kate Stewart <[email protected]>
Cc Greg Kroah-Hartman <[email protected]>
Cc Philippe Ombredanne <[email protected]>
Cc Pavel Tatashin <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Len Brown <[email protected]>
Cc Dou Liyang <[email protected]>
Cc Xiaoming Gao <[email protected]>
Cc Rajvi Jingar <[email protected]>
---
 arch/x86/include/asm/uv/uv.h | 6 ++++++
 1 file changed, 6 insertions(+)
arch/x86/include/asm/uv/uv.h +6 -0
index a80c0673798f..e60c45fd3679 100644
@@ -10,8 +10,13 @@ struct cpumask;
 struct mm_struct;
 
 #ifdef CONFIG_X86_UV
+#include <linux/efi.h>
 
 extern enum uv_system_type get_uv_system_type(void);
+static inline bool is_early_uv_system(void)
+{
+	return !((efi.uv_systab == EFI_INVALID_TABLE_ADDR) || !efi.uv_systab);
+}
 extern int is_uv_system(void);
 extern int is_uv_hubless(void);
 extern void uv_cpu_init(void);
@@ -23,6 +28,7 @@ extern const struct cpumask *uv_flush_tlb_others(const struct cpumask *cpumask,
 #else	/* X86_UV */
 
 static inline enum uv_system_type get_uv_system_type(void) { return UV_NONE; }
+static inline bool is_early_uv_system(void)	{ return 0; }
 static inline int is_uv_system(void)	{ return 0; }
 static inline int is_uv_hubless(void)	{ return 0; }
 static inline void uv_cpu_init(void)	{ }
#3 lore raw

PATCH 2/2 x86/tsc: Fix UV TSC initialization

In reply to Mike Travis

Fix regression introduced by

commit cf7a63ef4e02 ("x86/tsc: Calibrate tsc only once")

as it added a call to tsc_early_init() which initializes the TSC ADJUST values before acpi_boot_table_init(). In the case of UV systems, that is a necessary step thats calls uv_system_init(). This informs tsc_sanitize_first_cpu() that we're on a platform with async TSC resets as documented in

commit 341102c3ef29 ("x86/tsc: Add option that TSC on Socket 0 being non-zero is valid")

Fix by skipping the early tsc initialization on UV systems and let TSC init tests take place later in tsc_init().

Fixes cf7a63ef4e02 ("x86/tsc: Calibrate tsc only once")
---
 arch/x86/kernel/tsc.c |    4 ++++
 1 file changed, 4 insertions(+)
arch/x86/kernel/tsc.c +4 -0
@@ -26,6 +26,7 @@
 #include <asm/apic.h>
 #include <asm/intel-family.h>
 #include <asm/i8259.h>
+#include <asm/uv/uv.h>
 
 unsigned int __read_mostly cpu_khz;	/* TSC clocks / usec, not used here */
 EXPORT_SYMBOL(cpu_khz);
@@ -1433,6 +1434,9 @@ void __init tsc_early_init(void)
 {
 	if (!boot_cpu_has(X86_FEATURE_TSC))
 		return;
+	/* Don't change UV TSC multi-chassis synchronization */
+	if (is_early_uv_system())
+		return;
 	if (!determine_cpu_tsc_frequencies(true))
 		return;
 	loops_per_jiffy = get_loops_per_jiffy();
#4 lore raw

[tip:x86/urgent] x86/tsc: Fix UV TSC initialization

x86/tsc: Fix UV TSC initialization
Commit-ID:  2647c43c7f3ba4b752bfce261d53b16e2f5bc9e3
Gitweb:     https://git.kernel.org/tip/2647c43c7f3ba4b752bfce261d53b16e2f5bc9e3
Author:     Mike Travis <[email protected]>
AuthorDate: Tue, 2 Oct 2018 13:01:46 -0500
Committer:  Thomas Gleixner <[email protected]>
CommitDate: Tue, 2 Oct 2018 21:29:16 +0200

x86/tsc: Fix UV TSC initialization

The recent rework of the TSC calibration code introduced a regression on UV systems as it added a call to tsc_early_init() which initializes the TSC ADJUST values before acpi_boot_table_init(). In the case of UV systems, that is a necessary step that calls uv_system_init(). This informs tsc_sanitize_first_cpu() that the kernel runs on a platform with async TSC resets as documented in commit 341102c3ef29 ("x86/tsc: Add option that TSC on Socket 0 being non-zero is valid")

Fix it by skipping the early tsc initialization on UV systems and let TSC init tests take place later in tsc_init().

Fixes cf7a63ef4e02 ("x86/tsc: Calibrate tsc only once")
Cc "H. Peter Anvin" <[email protected]>
Cc Russ Anderson <[email protected]>
Cc Dimitri Sivanich <[email protected]>
Cc Borislav Petkov <[email protected]>
Cc Kate Stewart <[email protected]>
Cc Greg Kroah-Hartman <[email protected]>
Cc Philippe Ombredanne <[email protected]>
Cc Pavel Tatashin <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Len Brown <[email protected]>
Cc Dou Liyang <[email protected]>
Cc Xiaoming Gao <[email protected]>
Cc Rajvi Jingar <[email protected]>
---
 arch/x86/kernel/tsc.c | 4 ++++
 1 file changed, 4 insertions(+)
arch/x86/kernel/tsc.c +4 -0
index 6490f618e096..b52bd2b6cdb4 100644
@@ -26,6 +26,7 @@
 #include <asm/apic.h>
 #include <asm/intel-family.h>
 #include <asm/i8259.h>
+#include <asm/uv/uv.h>
 
 unsigned int __read_mostly cpu_khz;	/* TSC clocks / usec, not used here */
 EXPORT_SYMBOL(cpu_khz);
@@ -1433,6 +1434,9 @@ void __init tsc_early_init(void)
 {
 	if (!boot_cpu_has(X86_FEATURE_TSC))
 		return;
+	/* Don't change UV TSC multi-chassis synchronization */
+	if (is_early_uv_system())
+		return;
 	if (!determine_cpu_tsc_frequencies(true))
 		return;
 	loops_per_jiffy = get_loops_per_jiffy();