Skip to contentSkip to conversation

x86/tsc: add X86_FEATURE_TSC_KNOWN_FREQ flag and hardware related changes

PATCH In review Bin Gao 3 participants 10 messages Nov 15, 2016 – Nov 18, 2016

Subsystems (2)
lore
Share & tools
lore

Bin Gao

author
#0 lore raw

cover x86/tsc: add X86_FEATURE_TSC_KNOWN_FREQ flag and hardware related changes

This patch series adds X86_FEATURE_TSC_KNOWN_FREQ flag and modifies Intel Atom SoC related TSC codes to use the new TSC_KNOWN_FREQ flag and the existed TSC_RELIABLE flag.

Bin Gao (4):
x86/tsc: add X86_FEATURE_TSC_KNOWN_FREQ flag
x86/tsc: mark TSC frequency determined by CPUID as known x86/tsc: mark Intel ATOM_GOLDMONT TSC reliable
x86/tsc: set TSC_KNOWN_FREQ and TSC_RELIABLE flags on Intel Atom SoCs

 arch/x86/include/asm/cpufeatures.h  |  1 +
 arch/x86/kernel/tsc.c               | 25 ++++++++++++++++++++++---
 arch/x86/kernel/tsc_msr.c           | 18 ++++++++++++++++++
 arch/x86/platform/intel-mid/mfld.c  |  9 +++++++--
 arch/x86/platform/intel-mid/mrfld.c |  8 ++++++--
 5 files changed, 54 insertions(+), 7 deletions(-)
Signature · 1 line
1.9.1
author
#1 lore raw

PATCH 1/4 x86/tsc: add X86_FEATURE_TSC_KNOWN_FREQ flag

The X86_FEATURE_TSC_RELIABLE flag in Linux kernel implies both reliable (at runtime) and trustable (at calibration). But reliable running and trustable calibration are logically irrelevant. Per Thomas Gleixner's suggestion we would like to split this flag into two separate flags:
X86_FEATURE_TSC_RELIABLE - running reliably X86_FEATURE_TSC_KNOWN_FREQ - frequency is known (no calibration required) These two flags allow Linux kernel to act differently based on processor/SoC's capability, i.e. no watchdog on TSC if TSC is reliable, and no calibration if TSC frequency is known.

Current Linux kernel already gurantees calibration is skipped for processors that can report TSC frequency by CPUID or MSR. However, the delayed calibration is still not skipped for these CPUID/MSR capable processors. The new flag X86_FEATURE_TSC_KNOWN_FREQ added by this patch will gurantee the delayed calibration is skipped.

---
 arch/x86/include/asm/cpufeatures.h |  1 +
 arch/x86/kernel/tsc.c              | 11 ++++++++---
 2 files changed, 9 insertions(+), 3 deletions(-)
arch/x86/include/asm/cpufeatures.h +1 -0
index a396292..7f6a5f8 100644
@@ -106,6 +106,7 @@
 #define X86_FEATURE_APERFMPERF	( 3*32+28) /* APERFMPERF */
 #define X86_FEATURE_EAGER_FPU	( 3*32+29) /* "eagerfpu" Non lazy FPU restore */
 #define X86_FEATURE_NONSTOP_TSC_S3 ( 3*32+30) /* TSC doesn't stop in S3 state */
+#define X86_FEATURE_TSC_KNOWN_FREQ ( 3*32+31) /* TSC has known frequency */
 
 /* Intel-defined CPU features, CPUID level 0x00000001 (ecx), word 4 */
 #define X86_FEATURE_XMM3	( 4*32+ 0) /* "pni" SSE-3 */
arch/x86/kernel/tsc.c +8 -3
index 46b2f41..3ba146e 100644
@@ -1283,10 +1283,15 @@ static int __init init_tsc_clocksource(void)
 		clocksource_tsc.flags |= CLOCK_SOURCE_SUSPEND_NONSTOP;
 
 	/*
-	 * Trust the results of the earlier calibration on systems
-	 * exporting a reliable TSC.
+	 * When TSC frequency is known (generally got by MSR or CPUID), we skip
+	 * the refined calibration and directly register it as a clocksource.
+	 *
+	 * We still keep the TSC_RELIABLE flag here to avoid regression -
+	 * it will be removed after all the conversion for other code paths
+	 * connected to this flag is done.
 	 */
-	if (boot_cpu_has(X86_FEATURE_TSC_RELIABLE)) {
+	if (boot_cpu_has(X86_FEATURE_TSC_RELIABLE) ||
+		boot_cpu_has(X86_FEATURE_TSC_KNOWN_FREQ)) {
 		clocksource_register_khz(&clocksource_tsc, tsc_khz);
 		return 0;
 	}
Signature · 1 line
1.9.1
#2 lore raw

[tip:x86/timers] x86/tsc: Add X86_FEATURE_TSC_KNOWN_FREQ flag

x86/tsc: Add X86_FEATURE_TSC_KNOWN_FREQ flag
Commit-ID:  47c95a46d0fae07762f0a38aa3709ae63f307048
Gitweb:     http://git.kernel.org/tip/47c95a46d0fae07762f0a38aa3709ae63f307048
Author:     Bin Gao <[email protected]>
AuthorDate: Tue, 15 Nov 2016 12:27:21 -0800
Committer:  Thomas Gleixner <[email protected]>
CommitDate: Fri, 18 Nov 2016 10:58:30 +0100

x86/tsc: Add X86_FEATURE_TSC_KNOWN_FREQ flag

The X86_FEATURE_TSC_RELIABLE flag in Linux kernel implies both reliable (at runtime) and trustable (at calibration). But reliable running and trustable calibration independent of each other.

Add a new flag X86_FEATURE_TSC_KNOWN_FREQ, which denotes that the frequency is known (via MSR/CPUID). This flag is only meant to skip the long term calibration on systems which have a known frequency.

Add X86_FEATURE_TSC_KNOWN_FREQ to the skip the delayed calibration and leave X86_FEATURE_TSC_RELIABLE in place.

After converting the existing users of X86_FEATURE_TSC_RELIABLE to use either both flags or just X86_FEATURE_TSC_KNOWN_FREQ we can seperate the functionality.

---
 arch/x86/include/asm/cpufeatures.h |  1 +
 arch/x86/kernel/tsc.c              | 11 ++++++++---
 2 files changed, 9 insertions(+), 3 deletions(-)
arch/x86/include/asm/cpufeatures.h +1 -0
index a396292..7f6a5f8 100644
@@ -106,6 +106,7 @@
 #define X86_FEATURE_APERFMPERF	( 3*32+28) /* APERFMPERF */
 #define X86_FEATURE_EAGER_FPU	( 3*32+29) /* "eagerfpu" Non lazy FPU restore */
 #define X86_FEATURE_NONSTOP_TSC_S3 ( 3*32+30) /* TSC doesn't stop in S3 state */
+#define X86_FEATURE_TSC_KNOWN_FREQ ( 3*32+31) /* TSC has known frequency */
 
 /* Intel-defined CPU features, CPUID level 0x00000001 (ecx), word 4 */
 #define X86_FEATURE_XMM3	( 4*32+ 0) /* "pni" SSE-3 */
arch/x86/kernel/tsc.c +8 -3
index 46b2f41..d2c4ee4 100644
@@ -1283,10 +1283,15 @@ static int __init init_tsc_clocksource(void)
 		clocksource_tsc.flags |= CLOCK_SOURCE_SUSPEND_NONSTOP;
 
 	/*
-	 * Trust the results of the earlier calibration on systems
-	 * exporting a reliable TSC.
+	 * When TSC frequency is known (retrieved via MSR or CPUID), we skip
+	 * the refined calibration and directly register it as a clocksource.
+	 *
+	 * We still keep the TSC_RELIABLE flag here to avoid regressions -
+	 * it will be removed after all the conversion for other code paths
+	 * connected to this flag is done.
 	 */
-	if (boot_cpu_has(X86_FEATURE_TSC_RELIABLE)) {
+	if (boot_cpu_has(X86_FEATURE_TSC_RELIABLE) ||
+		boot_cpu_has(X86_FEATURE_TSC_KNOWN_FREQ)) {
 		clocksource_register_khz(&clocksource_tsc, tsc_khz);
 		return 0;
 	}
author
#3 lore raw

PATCH 2/4 x86/tsc: mark TSC frequency determined by CPUID as known

In reply to Bin Gao

Unlike TSC calibration where we determin TSC frequency by another timer with known frequency, CPUs/SoCs with CPUID leaf 0x15 come with a known frequency and will report the frequency to software via CPUID instruction. This hardware provided frequency is the "real" frequency of TSC so we set the X86_FEATURE_TSC_KNOWN_FREQ flag to skip the whole software calibration process.

We had a 24 hours test on one of the CPUID 0x15 capable platforms. With PIT calibrated frequency, we got more than 3 seconds drift whereas with CPUID determined frequency we only got less than 0.5 second drift. This makes us believe that we should prefer CPUID determined frequency over software calibrated frequency.

---
 arch/x86/kernel/tsc.c | 7 +++++++
 1 file changed, 7 insertions(+)
arch/x86/kernel/tsc.c +7 -0
index 3ba146e..f1a7fb5 100644
@@ -702,6 +702,13 @@ unsigned long native_calibrate_tsc(void)
 		}
 	}
 
+	/*
+	 * TSC frequency determined by CPUID is a "hardware reported"
+	 * frequency and is the most accurate one so far we have. This
+	 * is considered a known frequency.
+	 */
+	setup_force_cpu_cap(X86_FEATURE_TSC_KNOWN_FREQ);
+
 	return crystal_khz * ebx_numerator / eax_denominator;
 }
 
Signature · 1 line
1.9.1
#4 lore raw

[tip:x86/timers] x86/tsc: Mark TSC frequency determined by CPUID as known

x86/tsc: Mark TSC frequency determined by CPUID as known
Commit-ID:  4ca4df0b7eb06df264b2919759957f6d6ea1822e
Gitweb:     http://git.kernel.org/tip/4ca4df0b7eb06df264b2919759957f6d6ea1822e
Author:     Bin Gao <[email protected]>
AuthorDate: Tue, 15 Nov 2016 12:27:22 -0800
Committer:  Thomas Gleixner <[email protected]>
CommitDate: Fri, 18 Nov 2016 10:58:30 +0100

x86/tsc: Mark TSC frequency determined by CPUID as known

CPUs/SoCs with CPUID leaf 0x15 come with a known frequency and will report the frequency to software via CPUID instruction. This hardware provided frequency is the "real" frequency of TSC.

Set the X86_FEATURE_TSC_KNOWN_FREQ flag for such systems to skip the software calibration process.

A 24 hours test on one of the CPUID 0x15 capable platforms was conducted. PIT calibrated frequency resulted in more than 3 seconds drift whereas the CPUID determined frequency showed less than 0.5 second drift.

---
 arch/x86/kernel/tsc.c | 7 +++++++
 1 file changed, 7 insertions(+)
arch/x86/kernel/tsc.c +7 -0
index d2c4ee4..e58c319 100644
@@ -702,6 +702,13 @@ unsigned long native_calibrate_tsc(void)
 		}
 	}
 
+	/*
+	 * TSC frequency determined by CPUID is a "hardware reported"
+	 * frequency and is the most accurate one so far we have. This
+	 * is considered a known frequency.
+	 */
+	setup_force_cpu_cap(X86_FEATURE_TSC_KNOWN_FREQ);
+
 	return crystal_khz * ebx_numerator / eax_denominator;
 }
 
author
#5 lore raw

PATCH 3/4 x86/tsc: mark Intel ATOM_GOLDMONT TSC reliable

In reply to Bin Gao

On Intel GOLDMONT Atom SoC TSC is the only reliable clocksource. We mark TSC reliable to avoid watchdog on it.

---
 arch/x86/kernel/tsc.c | 7 +++++++
 1 file changed, 7 insertions(+)
arch/x86/kernel/tsc.c +7 -0
index f1a7fb5..354b302 100644
@@ -709,6 +709,13 @@ unsigned long native_calibrate_tsc(void)
 	 */
 	setup_force_cpu_cap(X86_FEATURE_TSC_KNOWN_FREQ);
 
+	/*
+	 * For Atom SoCs TSC is the only reliable clocksource.
+	 * Mark TSC reliable so no watchdog on it.
+	 */
+	if (boot_cpu_data.x86_model == INTEL_FAM6_ATOM_GOLDMONT)
+		setup_force_cpu_cap(X86_FEATURE_TSC_RELIABLE);
+
 	return crystal_khz * ebx_numerator / eax_denominator;
 }
 
Signature · 1 line
1.9.1
#6 lore raw

[tip:x86/timers] x86/tsc: Mark Intel ATOM_GOLDMONT TSC reliable

x86/tsc: Mark Intel ATOM_GOLDMONT TSC reliable
Commit-ID:  4635fdc696a8e89eead3ea1712ae6ada38538d40
Gitweb:     http://git.kernel.org/tip/4635fdc696a8e89eead3ea1712ae6ada38538d40
Author:     Bin Gao <[email protected]>
AuthorDate: Tue, 15 Nov 2016 12:27:23 -0800
Committer:  Thomas Gleixner <[email protected]>
CommitDate: Fri, 18 Nov 2016 10:58:30 +0100

x86/tsc: Mark Intel ATOM_GOLDMONT TSC reliable

On Intel GOLDMONT Atom SoC TSC is the only available clocksource, so there is no way to do software calibration or have a watchdog clocksource for it. Software calibration is already disabled via the TSC_KNOWN_FREQ flag, but the watchdog requirement still persists, so such systems cannot switch to high resolution/nohz mode.

Mark it reliable, so it becomes usable. Hardware teams confirmed that this is safe on that SoC.

---
 arch/x86/kernel/tsc.c | 7 +++++++
 1 file changed, 7 insertions(+)
arch/x86/kernel/tsc.c +7 -0
index e58c319..f4dfdaa 100644
@@ -709,6 +709,13 @@ unsigned long native_calibrate_tsc(void)
 	 */
 	setup_force_cpu_cap(X86_FEATURE_TSC_KNOWN_FREQ);
 
+	/*
+	 * For Atom SoCs TSC is the only reliable clocksource.
+	 * Mark TSC reliable so no watchdog on it.
+	 */
+	if (boot_cpu_data.x86_model == INTEL_FAM6_ATOM_GOLDMONT)
+		setup_force_cpu_cap(X86_FEATURE_TSC_RELIABLE);
+
 	return crystal_khz * ebx_numerator / eax_denominator;
 }
 
author
#7 lore raw

PATCH 4/4 x86/tsc: set TSC_KNOWN_FREQ and TSC_RELIABLE flags on Intel Atom SoCs

In reply to Bin Gao

TSC on Intel Atom SoCs capable of determining TSC frequency by MSR is reliable and the frequency is known (because it's provided by HW). On these platforms PIT/HPET is generally not available so calibration won't work at all and also TSC is the only reliable clocksource. So we set both X86_FEATURE_TSC_KNOWN_FREQ and X86_FEATURE_TSC_RELIABLE flags to make sure the calibration is skipped and no watchdog on TSC.

---
 arch/x86/kernel/tsc_msr.c           | 18 ++++++++++++++++++
 arch/x86/platform/intel-mid/mfld.c  |  9 +++++++--
 arch/x86/platform/intel-mid/mrfld.c |  8 ++++++--
 3 files changed, 31 insertions(+), 4 deletions(-)
arch/x86/kernel/tsc_msr.c +18 -0
index 0fe720d..c0f137c 100644
@@ -100,5 +100,23 @@ unsigned long cpu_khz_from_msr(void)
 #ifdef CONFIG_X86_LOCAL_APIC
 	lapic_timer_frequency = (freq * 1000) / HZ;
 #endif
+
+	/*
+	 * TSC frequency determined by MSR is always considered "known"
+	 * because it is reported by HW.
+	 * Another fact is that on MSR capable platforms, PIT/HPET is
+	 * generally not available so calibration won't work at all.
+	 */
+	setup_force_cpu_cap(X86_FEATURE_TSC_KNOWN_FREQ);
+
+	/*
+	 * Unfortunately there is no a HW way to report TSC is reliable.
+	 * We were told by silicon design team that TSC on Atom SoCs are
+	 * always "reliable". TSC is also the only reliable clocksource
+	 * on these SoCs (HPET is either not present or not functional)
+	 * so marke TSC reliable to avoid watchdog on it.
+	 */
+	setup_force_cpu_cap(X86_FEATURE_TSC_RELIABLE);
+
 	return res;
 }
arch/x86/platform/intel-mid/mfld.c +7 -2
index 1eb47b6..e793fe5 100644
@@ -49,8 +49,13 @@ static unsigned long __init mfld_calibrate_tsc(void)
 	fast_calibrate = ratio * fsb;
 	pr_debug("read penwell tsc %lu khz\n", fast_calibrate);
 	lapic_timer_frequency = fsb * 1000 / HZ;
-	/* mark tsc clocksource as reliable */
-	set_cpu_cap(&boot_cpu_data, X86_FEATURE_TSC_RELIABLE);
+
+	/*
+	 * TSC on Intel Atom SoCs is reliable and of known frequency.
+	 * See tsc_msr.c for details.
+	 */
+	setup_force_cpu_cap(X86_FEATURE_TSC_KNOWN_FREQ);
+	setup_force_cpu_cap(X86_FEATURE_TSC_RELIABLE);
 
 	return fast_calibrate;
 }
arch/x86/platform/intel-mid/mrfld.c +6 -2
index 59253db..e0607c7 100644
@@ -78,8 +78,12 @@ static unsigned long __init tangier_calibrate_tsc(void)
 	pr_debug("Setting lapic_timer_frequency = %d\n",
 			lapic_timer_frequency);
 
-	/* mark tsc clocksource as reliable */
-	set_cpu_cap(&boot_cpu_data, X86_FEATURE_TSC_RELIABLE);
+	/*
+	 * TSC on Intel Atom SoCs is reliable and of known frequency.
+	 * See tsc_msr.c for details.
+	 */
+	setup_force_cpu_cap(X86_FEATURE_TSC_KNOWN_FREQ);
+	setup_force_cpu_cap(X86_FEATURE_TSC_RELIABLE);
 
 	return fast_calibrate;
 }
Signature · 1 line
1.9.1
#8 lore raw

Re: [PATCH 4/4] x86/tsc: set TSC_KNOWN_FREQ and TSC_RELIABLE flags on Intel Atom SoCs

Quoted 11 lines · Bin Gao

TSC on Intel Atom SoCs capable of determining TSC frequency by MSR is reliable and the frequency is known (because it's provided by HW). On these platforms PIT/HPET is generally not available so calibration won't work at all and also TSC is the only reliable clocksource. So we set both X86_FEATURE_TSC_KNOWN_FREQ and X86_FEATURE_TSC_RELIABLE flags to make sure the calibration is skipped and no watchdog on TSC.

arch/x86/kernel/tsc_msr.c +4 -0
+	/*
+	 * TSC frequency determined by MSR is always considered "known"
+	 * because it is reported by HW.
+	 * Another fact is that on MSR capable platforms, PIT/HPET is

arch/x86/kernel/tsc_msr.c unsigned long cpu_khz_from_msr(void) · line 117

Quoted 10 lines
arch/x86/kernel/tsc_msr.c +10 -0
+	 * generally not available so calibration won't work at all.
+	 */
+	setup_force_cpu_cap(X86_FEATURE_TSC_KNOWN_FREQ);
+
+	/*
+	 * Unfortunately there is no a HW way to report TSC is reliable.
+	 * We were told by silicon design team that TSC on Atom SoCs are
+	 * always "reliable". TSC is also the only reliable clocksource
+	 * on these SoCs (HPET is either not present or not functional)
+	 * so marke TSC reliable to avoid watchdog on it.

minor nit:

s/there is no a HW way/
there is no HW way

Thanks,

Ingo

#9 lore raw

[tip:x86/timers] x86/tsc: Set TSC_KNOWN_FREQ and TSC_RELIABLE flags on Intel Atom SoCs

In reply to Bin Gao
x86/tsc: Set TSC_KNOWN_FREQ and TSC_RELIABLE flags on Intel Atom SoCs
Commit-ID:  f3a02ecebed7df7d5d68898628dea7a3bfcf03e3
Gitweb:     http://git.kernel.org/tip/f3a02ecebed7df7d5d68898628dea7a3bfcf03e3
Author:     Bin Gao <[email protected]>
AuthorDate: Tue, 15 Nov 2016 12:27:24 -0800
Committer:  Thomas Gleixner <[email protected]>
CommitDate: Fri, 18 Nov 2016 10:58:31 +0100

x86/tsc: Set TSC_KNOWN_FREQ and TSC_RELIABLE flags on Intel Atom SoCs

TSC on Intel Atom SoCs capable of determining TSC frequency by MSR is reliable and the frequency is known (provided by HW).

On these platforms PIT/HPET is generally not available so calibration won't work at all and there is no other clocksource to act as a watchdog for the TSC, so we have no other choice than to trust it.

Set both X86_FEATURE_TSC_KNOWN_FREQ and X86_FEATURE_TSC_RELIABLE flags to make sure the calibration is skipped and no watchdog is required.

---
 arch/x86/kernel/tsc_msr.c           | 19 +++++++++++++++++++
 arch/x86/platform/intel-mid/mfld.c  |  9 +++++++--
 arch/x86/platform/intel-mid/mrfld.c |  8 ++++++--
 3 files changed, 32 insertions(+), 4 deletions(-)
arch/x86/kernel/tsc_msr.c +19 -0
index 0fe720d..19afdbd 100644
@@ -100,5 +100,24 @@ unsigned long cpu_khz_from_msr(void)
 #ifdef CONFIG_X86_LOCAL_APIC
 	lapic_timer_frequency = (freq * 1000) / HZ;
 #endif
+
+	/*
+	 * TSC frequency determined by MSR is always considered "known"
+	 * because it is reported by HW.
+	 * Another fact is that on MSR capable platforms, PIT/HPET is
+	 * generally not available so calibration won't work at all.
+	 */
+	setup_force_cpu_cap(X86_FEATURE_TSC_KNOWN_FREQ);
+
+	/*
+	 * Unfortunately there is no way for hardware to tell whether the
+	 * TSC is reliable.  We were told by silicon design team that TSC
+	 * on Atom SoCs are always "reliable". TSC is also the only
+	 * reliable clocksource on these SoCs (HPET is either not present
+	 * or not functional) so mark TSC reliable which removes the
+	 * requirement for a watchdog clocksource.
+	 */
+	setup_force_cpu_cap(X86_FEATURE_TSC_RELIABLE);
+
 	return res;
 }
arch/x86/platform/intel-mid/mfld.c +7 -2
index 1eb47b6..e793fe5 100644
@@ -49,8 +49,13 @@ static unsigned long __init mfld_calibrate_tsc(void)
 	fast_calibrate = ratio * fsb;
 	pr_debug("read penwell tsc %lu khz\n", fast_calibrate);
 	lapic_timer_frequency = fsb * 1000 / HZ;
-	/* mark tsc clocksource as reliable */
-	set_cpu_cap(&boot_cpu_data, X86_FEATURE_TSC_RELIABLE);
+
+	/*
+	 * TSC on Intel Atom SoCs is reliable and of known frequency.
+	 * See tsc_msr.c for details.
+	 */
+	setup_force_cpu_cap(X86_FEATURE_TSC_KNOWN_FREQ);
+	setup_force_cpu_cap(X86_FEATURE_TSC_RELIABLE);
 
 	return fast_calibrate;
 }
arch/x86/platform/intel-mid/mrfld.c +6 -2
index 59253db..e0607c7 100644
@@ -78,8 +78,12 @@ static unsigned long __init tangier_calibrate_tsc(void)
 	pr_debug("Setting lapic_timer_frequency = %d\n",
 			lapic_timer_frequency);
 
-	/* mark tsc clocksource as reliable */
-	set_cpu_cap(&boot_cpu_data, X86_FEATURE_TSC_RELIABLE);
+	/*
+	 * TSC on Intel Atom SoCs is reliable and of known frequency.
+	 * See tsc_msr.c for details.
+	 */
+	setup_force_cpu_cap(X86_FEATURE_TSC_KNOWN_FREQ);
+	setup_force_cpu_cap(X86_FEATURE_TSC_RELIABLE);
 
 	return fast_calibrate;
 }