x86/tsc: add X86_FEATURE_TSC_KNOWN_FREQ flag and hardware related changes
Subsystems (2)
cover x86/tsc: add X86_FEATURE_TSC_KNOWN_FREQ flag and hardware related changes
This patch series adds X86_FEATURE_TSC_KNOWN_FREQ flag and modifies Intel Atom SoC related TSC codes to use the new TSC_KNOWN_FREQ flag and the existed TSC_RELIABLE flag.
Bin Gao (4):
x86/tsc: add X86_FEATURE_TSC_KNOWN_FREQ flag
x86/tsc: mark TSC frequency determined by CPUID as known x86/tsc: mark Intel ATOM_GOLDMONT TSC reliable
x86/tsc: set TSC_KNOWN_FREQ and TSC_RELIABLE flags on Intel Atom SoCs
arch/x86/include/asm/cpufeatures.h | 1 + arch/x86/kernel/tsc.c | 25 ++++++++++++++++++++++--- arch/x86/kernel/tsc_msr.c | 18 ++++++++++++++++++ arch/x86/platform/intel-mid/mfld.c | 9 +++++++-- arch/x86/platform/intel-mid/mrfld.c | 8 ++++++-- 5 files changed, 54 insertions(+), 7 deletions(-)
Signature · 1 line
1.9.1
PATCH 1/4 x86/tsc: add X86_FEATURE_TSC_KNOWN_FREQ flag
The X86_FEATURE_TSC_RELIABLE flag in Linux kernel implies both reliable (at runtime) and trustable (at calibration). But reliable running and trustable calibration are logically irrelevant. Per Thomas Gleixner's suggestion we would like to split this flag into two separate flags:X86_FEATURE_TSC_RELIABLE - running reliably X86_FEATURE_TSC_KNOWN_FREQ - frequency is known (no calibration required) These two flags allow Linux kernel to act differently based on processor/SoC's capability, i.e. no watchdog on TSC if TSC is reliable, and no calibration if TSC frequency is known.
Current Linux kernel already gurantees calibration is skipped for processors that can report TSC frequency by CPUID or MSR. However, the delayed calibration is still not skipped for these CPUID/MSR capable processors. The new flag X86_FEATURE_TSC_KNOWN_FREQ added by this patch will gurantee the delayed calibration is skipped.
--- arch/x86/include/asm/cpufeatures.h | 1 + arch/x86/kernel/tsc.c | 11 ++++++++--- 2 files changed, 9 insertions(+), 3 deletions(-)
arch/x86/include/asm/cpufeatures.h +1 -0
@@ -106,6 +106,7 @@#define X86_FEATURE_APERFMPERF ( 3*32+28)/* APERFMPERF */ #define X86_FEATURE_EAGER_FPU ( 3*32+29)/* "eagerfpu" Non lazy FPU restore */ #define X86_FEATURE_NONSTOP_TSC_S3 ( 3*32+30)/* TSC doesn't stop in S3 state */ +#define X86_FEATURE_TSC_KNOWN_FREQ ( 3*32+31)/* TSC has known frequency */ /* Intel-defined CPU features, CPUID level 0x00000001 (ecx), word 4 */ #define X86_FEATURE_XMM3 ( 4*32+ 0)/* "pni" SSE-3 */
arch/x86/kernel/tsc.c +8 -3
@@ -1283,10 +1283,15 @@ static int __init init_tsc_clocksource(void) clocksource_tsc.flags |=CLOCK_SOURCE_SUSPEND_NONSTOP ;/* -* Trust the results of the earlier calibration on systems -* exporting a reliable TSC. +* When TSC frequency is known (generally got by MSR or CPUID), we skip +* the refined calibration and directly register it as a clocksource. +* +* We still keep the TSC_RELIABLE flag here to avoid regression - +* it will be removed after all the conversion for other code paths +* connected to this flag is done. */ -if (boot_cpu_has (X86_FEATURE_TSC_RELIABLE )) { +if (boot_cpu_has (X86_FEATURE_TSC_RELIABLE ) || +boot_cpu_has (X86_FEATURE_TSC_KNOWN_FREQ )) {clocksource_register_khz (&clocksource_tsc, tsc_khz);return 0 ; }
Signature · 1 line
1.9.1
tip-bot for Bin Gao
merge bot[tip:x86/timers] x86/tsc: Add X86_FEATURE_TSC_KNOWN_FREQ flag
x86/tsc: Add X86_FEATURE_TSC_KNOWN_FREQ flag
Commit-ID: 47c95a46d0fae07762f0a38aa3709ae63f307048 Gitweb: http://git.kernel.org/tip/47c95a46d0fae07762f0a38aa3709ae63f307048 Author: Bin Gao <[email protected]> AuthorDate: Tue, 15 Nov 2016 12:27:21 -0800 Committer: Thomas Gleixner <[email protected]> CommitDate: Fri, 18 Nov 2016 10:58:30 +0100
x86/tsc: Add X86_FEATURE_TSC_KNOWN_FREQ flag
The X86_FEATURE_TSC_RELIABLE flag in Linux kernel implies both reliable (at runtime) and trustable (at calibration). But reliable running and trustable calibration independent of each other.
Add a new flag X86_FEATURE_TSC_KNOWN_FREQ, which denotes that the frequency is known (via MSR/CPUID). This flag is only meant to skip the long term calibration on systems which have a known frequency.
Add X86_FEATURE_TSC_KNOWN_FREQ to the skip the delayed calibration and leave X86_FEATURE_TSC_RELIABLE in place.
After converting the existing users of X86_FEATURE_TSC_RELIABLE to use either both flags or just X86_FEATURE_TSC_KNOWN_FREQ we can seperate the functionality.
--- arch/x86/include/asm/cpufeatures.h | 1 + arch/x86/kernel/tsc.c | 11 ++++++++--- 2 files changed, 9 insertions(+), 3 deletions(-)
arch/x86/include/asm/cpufeatures.h +1 -0
@@ -106,6 +106,7 @@#define X86_FEATURE_APERFMPERF ( 3*32+28)/* APERFMPERF */ #define X86_FEATURE_EAGER_FPU ( 3*32+29)/* "eagerfpu" Non lazy FPU restore */ #define X86_FEATURE_NONSTOP_TSC_S3 ( 3*32+30)/* TSC doesn't stop in S3 state */ +#define X86_FEATURE_TSC_KNOWN_FREQ ( 3*32+31)/* TSC has known frequency */ /* Intel-defined CPU features, CPUID level 0x00000001 (ecx), word 4 */ #define X86_FEATURE_XMM3 ( 4*32+ 0)/* "pni" SSE-3 */
arch/x86/kernel/tsc.c +8 -3
@@ -1283,10 +1283,15 @@ static int __init init_tsc_clocksource(void) clocksource_tsc.flags |=CLOCK_SOURCE_SUSPEND_NONSTOP ;/* -* Trust the results of the earlier calibration on systems -* exporting a reliable TSC. +* When TSC frequency is known (retrieved via MSR or CPUID), we skip +* the refined calibration and directly register it as a clocksource. +* +* We still keep the TSC_RELIABLE flag here to avoid regressions - +* it will be removed after all the conversion for other code paths +* connected to this flag is done. */ -if (boot_cpu_has (X86_FEATURE_TSC_RELIABLE )) { +if (boot_cpu_has (X86_FEATURE_TSC_RELIABLE ) || +boot_cpu_has (X86_FEATURE_TSC_KNOWN_FREQ )) {clocksource_register_khz (&clocksource_tsc, tsc_khz);return 0 ; }
PATCH 2/4 x86/tsc: mark TSC frequency determined by CPUID as known
In reply to Bin GaoUnlike TSC calibration where we determin TSC frequency by another timer with known frequency, CPUs/SoCs with CPUID leaf 0x15 come with a known frequency and will report the frequency to software via CPUID instruction. This hardware provided frequency is the "real" frequency of TSC so we set the X86_FEATURE_TSC_KNOWN_FREQ flag to skip the whole software calibration process.
We had a 24 hours test on one of the CPUID 0x15 capable platforms. With PIT calibrated frequency, we got more than 3 seconds drift whereas with CPUID determined frequency we only got less than 0.5 second drift. This makes us believe that we should prefer CPUID determined frequency over software calibrated frequency.
--- arch/x86/kernel/tsc.c | 7 +++++++ 1 file changed, 7 insertions(+)
arch/x86/kernel/tsc.c +7 -0
@@ -702,6 +702,13 @@ unsigned long native_calibrate_tsc(void) } } +/* +* TSC frequency determined by CPUID is a "hardware reported" +* frequency and is the most accurate one so far we have. This +* is considered a known frequency. +*/ +setup_force_cpu_cap (X86_FEATURE_TSC_KNOWN_FREQ ); +return crystal_khz * ebx_numerator / eax_denominator; }
Signature · 1 line
1.9.1
tip-bot for Bin Gao
merge bot[tip:x86/timers] x86/tsc: Mark TSC frequency determined by CPUID as known
x86/tsc: Mark TSC frequency determined by CPUID as known
Commit-ID: 4ca4df0b7eb06df264b2919759957f6d6ea1822e Gitweb: http://git.kernel.org/tip/4ca4df0b7eb06df264b2919759957f6d6ea1822e Author: Bin Gao <[email protected]> AuthorDate: Tue, 15 Nov 2016 12:27:22 -0800 Committer: Thomas Gleixner <[email protected]> CommitDate: Fri, 18 Nov 2016 10:58:30 +0100
x86/tsc: Mark TSC frequency determined by CPUID as known
CPUs/SoCs with CPUID leaf 0x15 come with a known frequency and will report the frequency to software via CPUID instruction. This hardware provided frequency is the "real" frequency of TSC.
Set the X86_FEATURE_TSC_KNOWN_FREQ flag for such systems to skip the software calibration process.
A 24 hours test on one of the CPUID 0x15 capable platforms was conducted. PIT calibrated frequency resulted in more than 3 seconds drift whereas the CPUID determined frequency showed less than 0.5 second drift.
--- arch/x86/kernel/tsc.c | 7 +++++++ 1 file changed, 7 insertions(+)
arch/x86/kernel/tsc.c +7 -0
@@ -702,6 +702,13 @@ unsigned long native_calibrate_tsc(void) } } +/* +* TSC frequency determined by CPUID is a "hardware reported" +* frequency and is the most accurate one so far we have. This +* is considered a known frequency. +*/ +setup_force_cpu_cap (X86_FEATURE_TSC_KNOWN_FREQ ); +return crystal_khz * ebx_numerator / eax_denominator; }
PATCH 3/4 x86/tsc: mark Intel ATOM_GOLDMONT TSC reliable
In reply to Bin GaoOn Intel GOLDMONT Atom SoC TSC is the only reliable clocksource. We mark TSC reliable to avoid watchdog on it.
--- arch/x86/kernel/tsc.c | 7 +++++++ 1 file changed, 7 insertions(+)
arch/x86/kernel/tsc.c +7 -0
@@ -709,6 +709,13 @@ unsigned long native_calibrate_tsc(void)*/ setup_force_cpu_cap (X86_FEATURE_TSC_KNOWN_FREQ ); +/* +* For Atom SoCs TSC is the only reliable clocksource. +* Mark TSC reliable so no watchdog on it. +*/ +if (boot_cpu_data.x86_model ==INTEL_FAM6_ATOM_GOLDMONT ) +setup_force_cpu_cap (X86_FEATURE_TSC_RELIABLE ); +return crystal_khz * ebx_numerator / eax_denominator; }
Signature · 1 line
1.9.1
tip-bot for Bin Gao
merge bot[tip:x86/timers] x86/tsc: Mark Intel ATOM_GOLDMONT TSC reliable
x86/tsc: Mark Intel ATOM_GOLDMONT TSC reliable
Commit-ID: 4635fdc696a8e89eead3ea1712ae6ada38538d40 Gitweb: http://git.kernel.org/tip/4635fdc696a8e89eead3ea1712ae6ada38538d40 Author: Bin Gao <[email protected]> AuthorDate: Tue, 15 Nov 2016 12:27:23 -0800 Committer: Thomas Gleixner <[email protected]> CommitDate: Fri, 18 Nov 2016 10:58:30 +0100
x86/tsc: Mark Intel ATOM_GOLDMONT TSC reliable
On Intel GOLDMONT Atom SoC TSC is the only available clocksource, so there is no way to do software calibration or have a watchdog clocksource for it. Software calibration is already disabled via the TSC_KNOWN_FREQ flag, but the watchdog requirement still persists, so such systems cannot switch to high resolution/nohz mode.
Mark it reliable, so it becomes usable. Hardware teams confirmed that this is safe on that SoC.
--- arch/x86/kernel/tsc.c | 7 +++++++ 1 file changed, 7 insertions(+)
arch/x86/kernel/tsc.c +7 -0
@@ -709,6 +709,13 @@ unsigned long native_calibrate_tsc(void)*/ setup_force_cpu_cap (X86_FEATURE_TSC_KNOWN_FREQ ); +/* +* For Atom SoCs TSC is the only reliable clocksource. +* Mark TSC reliable so no watchdog on it. +*/ +if (boot_cpu_data.x86_model ==INTEL_FAM6_ATOM_GOLDMONT ) +setup_force_cpu_cap (X86_FEATURE_TSC_RELIABLE ); +return crystal_khz * ebx_numerator / eax_denominator; }
PATCH 4/4 x86/tsc: set TSC_KNOWN_FREQ and TSC_RELIABLE flags on Intel Atom SoCs
In reply to Bin GaoTSC on Intel Atom SoCs capable of determining TSC frequency by MSR is reliable and the frequency is known (because it's provided by HW). On these platforms PIT/HPET is generally not available so calibration won't work at all and also TSC is the only reliable clocksource. So we set both X86_FEATURE_TSC_KNOWN_FREQ and X86_FEATURE_TSC_RELIABLE flags to make sure the calibration is skipped and no watchdog on TSC.
--- arch/x86/kernel/tsc_msr.c | 18 ++++++++++++++++++ arch/x86/platform/intel-mid/mfld.c | 9 +++++++-- arch/x86/platform/intel-mid/mrfld.c | 8 ++++++-- 3 files changed, 31 insertions(+), 4 deletions(-)
arch/x86/kernel/tsc_msr.c +18 -0
@@ -100,5 +100,23 @@ unsigned long cpu_khz_from_msr(void)#ifdef CONFIG_X86_LOCAL_APIC lapic_timer_frequency = (freq *1000 ) /HZ ;#endif + +/* +* TSC frequency determined by MSR is always considered "known" +* because it is reported by HW. +* Another fact is that on MSR capable platforms, PIT/HPET is +* generally not available so calibration won't work at all. +*/ +setup_force_cpu_cap (X86_FEATURE_TSC_KNOWN_FREQ ); + +/* +* Unfortunately there is no a HW way to report TSC is reliable. +* We were told by silicon design team that TSC on Atom SoCs are +* always "reliable". TSC is also the only reliable clocksource +* on these SoCs (HPET is either not present or not functional) +* so marke TSC reliable to avoid watchdog on it. +*/ +setup_force_cpu_cap (X86_FEATURE_TSC_RELIABLE ); +return res; }
arch/x86/platform/intel-mid/mfld.c +7 -2
@@ -49,8 +49,13 @@ static unsigned long __init mfld_calibrate_tsc(void) fast_calibrate = ratio * fsb;pr_debug ("read penwell tsc %lu khz\n" , fast_calibrate); lapic_timer_frequency = fsb *1000 /HZ ; -/* mark tsc clocksource as reliable */ -set_cpu_cap (&boot_cpu_data,X86_FEATURE_TSC_RELIABLE ); + +/* +* TSC on Intel Atom SoCs is reliable and of known frequency. +* See tsc_msr.c for details. +*/ +setup_force_cpu_cap (X86_FEATURE_TSC_KNOWN_FREQ ); +setup_force_cpu_cap (X86_FEATURE_TSC_RELIABLE );return fast_calibrate; }
arch/x86/platform/intel-mid/mrfld.c +6 -2
@@ -78,8 +78,12 @@ static unsigned long __init tangier_calibrate_tsc(void)pr_debug ("Setting lapic_timer_frequency = %d\n" , lapic_timer_frequency); -/* mark tsc clocksource as reliable */ -set_cpu_cap (&boot_cpu_data,X86_FEATURE_TSC_RELIABLE ); +/* +* TSC on Intel Atom SoCs is reliable and of known frequency. +* See tsc_msr.c for details. +*/ +setup_force_cpu_cap (X86_FEATURE_TSC_KNOWN_FREQ ); +setup_force_cpu_cap (X86_FEATURE_TSC_RELIABLE );return fast_calibrate; }
Signature · 1 line
1.9.1
Re: [PATCH 4/4] x86/tsc: set TSC_KNOWN_FREQ and TSC_RELIABLE flags on Intel Atom SoCs
Quoted 11 lines · Bin Gao
TSC on Intel Atom SoCs capable of determining TSC frequency by MSR is reliable and the frequency is known (because it's provided by HW). On these platforms PIT/HPET is generally not available so calibration won't work at all and also TSC is the only reliable clocksource. So we set both
X86_FEATURE_TSC_KNOWN_FREQandX86_FEATURE_TSC_RELIABLEflags to make sure the calibration is skipped and no watchdog on TSC.arch/x86/kernel/tsc_msr.c +4
-0+/* +* TSC frequency determined by MSR is always considered "known" +* because it is reported by HW. +* Another fact is that on MSR capable platforms, PIT/HPET is
arch/x86/kernel/tsc_msr.c unsigned long cpu_khz_from_msr(void) · line 117
Quoted 10 lines
arch/x86/kernel/tsc_msr.c +10
-0+* generally not available so calibration won't work at all. +*/ +setup_force_cpu_cap (X86_FEATURE_TSC_KNOWN_FREQ ); + +/* +* Unfortunately there is no a HW way to report TSC is reliable. +* We were told by silicon design team that TSC on Atom SoCs are +* always "reliable". TSC is also the only reliable clocksource +* on these SoCs (HPET is either not present or not functional) +* so marke TSC reliable to avoid watchdog on it.
minor nit:
s/there is no a HW way/
there is no HW way
Thanks,
Ingo
tip-bot for Bin Gao
merge bot[tip:x86/timers] x86/tsc: Set TSC_KNOWN_FREQ and TSC_RELIABLE flags on Intel Atom SoCs
In reply to Bin Gaox86/tsc: Set TSC_KNOWN_FREQ and TSC_RELIABLE flags on Intel Atom SoCs
Commit-ID: f3a02ecebed7df7d5d68898628dea7a3bfcf03e3 Gitweb: http://git.kernel.org/tip/f3a02ecebed7df7d5d68898628dea7a3bfcf03e3 Author: Bin Gao <[email protected]> AuthorDate: Tue, 15 Nov 2016 12:27:24 -0800 Committer: Thomas Gleixner <[email protected]> CommitDate: Fri, 18 Nov 2016 10:58:31 +0100
x86/tsc: Set TSC_KNOWN_FREQ and TSC_RELIABLE flags on Intel Atom SoCs
TSC on Intel Atom SoCs capable of determining TSC frequency by MSR is reliable and the frequency is known (provided by HW).
On these platforms PIT/HPET is generally not available so calibration won't work at all and there is no other clocksource to act as a watchdog for the TSC, so we have no other choice than to trust it.
Set both X86_FEATURE_TSC_KNOWN_FREQ and X86_FEATURE_TSC_RELIABLE flags to make sure the calibration is skipped and no watchdog is required.
--- arch/x86/kernel/tsc_msr.c | 19 +++++++++++++++++++ arch/x86/platform/intel-mid/mfld.c | 9 +++++++-- arch/x86/platform/intel-mid/mrfld.c | 8 ++++++-- 3 files changed, 32 insertions(+), 4 deletions(-)
arch/x86/kernel/tsc_msr.c +19 -0
@@ -100,5 +100,24 @@ unsigned long cpu_khz_from_msr(void)#ifdef CONFIG_X86_LOCAL_APIC lapic_timer_frequency = (freq *1000 ) /HZ ;#endif + +/* +* TSC frequency determined by MSR is always considered "known" +* because it is reported by HW. +* Another fact is that on MSR capable platforms, PIT/HPET is +* generally not available so calibration won't work at all. +*/ +setup_force_cpu_cap (X86_FEATURE_TSC_KNOWN_FREQ ); + +/* +* Unfortunately there is no way for hardware to tell whether the +* TSC is reliable. We were told by silicon design team that TSC +* on Atom SoCs are always "reliable". TSC is also the only +* reliable clocksource on these SoCs (HPET is either not present +* or not functional) so mark TSC reliable which removes the +* requirement for a watchdog clocksource. +*/ +setup_force_cpu_cap (X86_FEATURE_TSC_RELIABLE ); +return res; }
arch/x86/platform/intel-mid/mfld.c +7 -2
@@ -49,8 +49,13 @@ static unsigned long __init mfld_calibrate_tsc(void) fast_calibrate = ratio * fsb;pr_debug ("read penwell tsc %lu khz\n" , fast_calibrate); lapic_timer_frequency = fsb *1000 /HZ ; -/* mark tsc clocksource as reliable */ -set_cpu_cap (&boot_cpu_data,X86_FEATURE_TSC_RELIABLE ); + +/* +* TSC on Intel Atom SoCs is reliable and of known frequency. +* See tsc_msr.c for details. +*/ +setup_force_cpu_cap (X86_FEATURE_TSC_KNOWN_FREQ ); +setup_force_cpu_cap (X86_FEATURE_TSC_RELIABLE );return fast_calibrate; }
arch/x86/platform/intel-mid/mrfld.c +6 -2
@@ -78,8 +78,12 @@ static unsigned long __init tangier_calibrate_tsc(void)pr_debug ("Setting lapic_timer_frequency = %d\n" , lapic_timer_frequency); -/* mark tsc clocksource as reliable */ -set_cpu_cap (&boot_cpu_data,X86_FEATURE_TSC_RELIABLE ); +/* +* TSC on Intel Atom SoCs is reliable and of known frequency. +* See tsc_msr.c for details. +*/ +setup_force_cpu_cap (X86_FEATURE_TSC_KNOWN_FREQ ); +setup_force_cpu_cap (X86_FEATURE_TSC_RELIABLE );return fast_calibrate; }