x86/asm: Remove some static_cpu_has() usage
Subsystems (10)
Borislav Petkov
maintainercover x86/asm: Remove some static_cpu_has() usage
From: Borislav Petkov <[email protected]>
Hi,
this came about after Nadav asked about static_cpu_has() and when it should be used. The resulting converts a bunch of slow-paths to use the plain boot_cpu_has() tester which is straight-forward and doesn't require to be patched in by alternatives, leaving static_cpu_has() only to the fast paths, where it should be used anyway.
Thx.
Borislav Petkov (5):
x86/asm: Clarify static_cpu_has()'s intended use
x86: Convert some slow-path static_cpu_has() callers to boot_cpu_has() x86/kvm: Convert some slow-path static_cpu_has() callers to
boot_cpu_has()
x86/mm: Convert some slow-path static_cpu_has() callers to
boot_cpu_has()
drivers/cpufreq: Convert some slow-path static_cpu_has() callers to
boot_cpu_has()arch/x86/include/asm/cpufeature.h | 9 ++++++--- arch/x86/include/asm/fpu/internal.h | 7 +++---- arch/x86/kernel/apic/apic_numachip.c | 2 +- arch/x86/kernel/cpu/aperfmperf.c | 6 +++--- arch/x86/kernel/cpu/common.c | 2 +- arch/x86/kernel/cpu/mce/inject.c | 2 +- arch/x86/kernel/cpu/proc.c | 10 +++++----- arch/x86/kernel/ldt.c | 14 +++++++------- arch/x86/kernel/paravirt.c | 2 +- arch/x86/kernel/process.c | 4 ++-- arch/x86/kernel/reboot.c | 2 +- arch/x86/kernel/vm86_32.c | 2 +- arch/x86/kvm/svm.c | 10 +++++----- arch/x86/kvm/vmx/vmx.c | 4 ++-- arch/x86/mm/dump_pagetables.c | 4 ++-- arch/x86/mm/pgtable.c | 4 ++-- arch/x86/mm/pti.c | 2 +- drivers/cpufreq/amd_freq_sensitivity.c | 2 +- drivers/cpufreq/intel_pstate.c | 18 +++++++++--------- drivers/cpufreq/powernow-k8.c | 2 +- 20 files changed, 55 insertions(+), 53 deletions(-)
Signature · 1 line
2.21.0
Borislav Petkov
maintainerPATCH 1/5 x86/asm: Clarify static_cpu_has()'s intended use
From: Borislav Petkov <[email protected]>
Clarify when one should use static_cpu_has() and when one should use boot_cpu_has().
--- arch/x86/include/asm/cpufeature.h | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-)
arch/x86/include/asm/cpufeature.h +6 -3
@@ -156,9 +156,12 @@ extern void clear_cpu_cap(struct cpuinfo_x86 *c, unsigned int bit);#else /* -* Static testing of CPU features. Used the same as boot_cpu_has(). -* These will statically patch the target code for additional -* performance. +* Static testing of CPU features. Used the same as boot_cpu_has(). It +* statically patches the target code for additional performance. Use +* static_cpu_has() only in fast paths, where every cycle counts. Which +* means that the boot_cpu_has() variant is already fast enough for the +* majority of cases and you should stick to using it as it is generally +* only two instructions: a RIP-relative MOV and a TEST. */ static __always_inline bool _static_cpu_has(u16 bit) {
Signature · 1 line
2.21.0
Peter Zijlstra
maintainerRe: [PATCH 1/5] x86/asm: Clarify static_cpu_has()'s intended use
Quoted 22 lines · Borislav Petkov
From: Borislav Petkov <[email protected]>
Clarify when one should use
static_cpu_has()and when one should useboot_cpu_has().Requested-by Nadav Amit <[email protected]>Signed-off-by Borislav Petkov <[email protected]>--- arch/x86/include/asm/cpufeature.h | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-)arch/x86/include/asm/cpufeature.h +0
-2@@ -156,9 +156,12 @@ extern void clear_cpu_cap(struct cpuinfo_x86 *c, unsigned int bit);#else /* -* Static testing of CPU features. Used the same as boot_cpu_has(). -* These will statically patch the target code for additional
arch/x86/include/asm/cpufeature.h extern void clear_cpu_cap(struct cpuinfo_x86 *c, unsigned int bit); · line 167
Quoted 10 lines
arch/x86/include/asm/cpufeature.h +6
-1-* performance. +* Static testing of CPU features. Used the same as boot_cpu_has(). It +* statically patches the target code for additional performance. Use +* static_cpu_has() only in fast paths, where every cycle counts. Which +* means that the boot_cpu_has() variant is already fast enough for the +* majority of cases and you should stick to using it as it is generally +* only two instructions: a RIP-relative MOV and a TEST. */ static __always_inline bool _static_cpu_has(u16 bit) {
Should we introduce cpu_has() ?
I'm sure it boot_cpu_has() is an awesome name, but in like 99.9% of the cases we don't give a crap about which actual CPU has the feature set. We also don't actually support asymmetric cpu features anyway.
tip-bot for Borislav Petkov
merge bot[tip:x86/asm] x86/asm: Clarify static_cpu_has()'s intended use
In reply to Borislav Petkovx86/asm: Clarify static_cpu_has()'s intended use
Commit-ID: bfdd5a67c8cb02c147c6b012543e84cb1f5759ba Gitweb: https://git.kernel.org/tip/bfdd5a67c8cb02c147c6b012543e84cb1f5759ba Author: Borislav Petkov <[email protected]> AuthorDate: Fri, 29 Mar 2019 19:35:24 +0100 Committer: Borislav Petkov <[email protected]> CommitDate: Mon, 8 Apr 2019 12:02:55 +0200
x86/asm: Clarify static_cpu_has()'s intended use
Clarify when one should use static_cpu_has() and when one should use boot_cpu_has().
--- arch/x86/include/asm/cpufeature.h | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-)
arch/x86/include/asm/cpufeature.h +6 -3
@@ -155,9 +155,12 @@ extern void clear_cpu_cap(struct cpuinfo_x86 *c, unsigned int bit);#else /* -* Static testing of CPU features. Used the same as boot_cpu_has(). -* These will statically patch the target code for additional -* performance. +* Static testing of CPU features. Used the same as boot_cpu_has(). It +* statically patches the target code for additional performance. Use +* static_cpu_has() only in fast paths, where every cycle counts. Which +* means that the boot_cpu_has() variant is already fast enough for the +* majority of cases and you should stick to using it as it is generally +* only two instructions: a RIP-relative MOV and a TEST. */ static __always_inline bool _static_cpu_has(u16 bit) {
Borislav Petkov
maintainerPATCH 2/5 x86: Convert some slow-path static_cpu_has() callers to boot_cpu_has()
In reply to Borislav PetkovFrom: Borislav Petkov <[email protected]>
Using static_cpu_has() is pointless on those paths, convert them to the boot_cpu_has() variant.
No functional changes.
--- arch/x86/include/asm/fpu/internal.h | 7 +++---- arch/x86/kernel/apic/apic_numachip.c | 2 +- arch/x86/kernel/cpu/aperfmperf.c | 6 +++--- arch/x86/kernel/cpu/common.c | 2 +- arch/x86/kernel/cpu/mce/inject.c | 2 +- arch/x86/kernel/cpu/proc.c | 10 +++++----- arch/x86/kernel/ldt.c | 14 +++++++------- arch/x86/kernel/paravirt.c | 2 +- arch/x86/kernel/process.c | 4 ++-- arch/x86/kernel/reboot.c | 2 +- arch/x86/kernel/vm86_32.c | 2 +- 11 files changed, 26 insertions(+), 27 deletions(-)
arch/x86/include/asm/fpu/internal.h +3 -4
@@ -253,7 +253,7 @@ static inline void copy_xregs_to_kernel_booting(struct xregs_state *xstate)WARN_ON (system_state !=SYSTEM_BOOTING ); -if (static_cpu_has (X86_FEATURE_XSAVES )) +if (boot_cpu_has (X86_FEATURE_XSAVES ))XSTATE_OP (XSAVES , xstate, lmask, hmask, err);else XSTATE_OP (XSAVE , xstate, lmask, hmask, err);
@@ -275,7 +275,7 @@ static inline void copy_kernel_to_xregs_booting(struct xregs_state *xstate)WARN_ON (system_state !=SYSTEM_BOOTING ); -if (static_cpu_has (X86_FEATURE_XSAVES )) +if (boot_cpu_has (X86_FEATURE_XSAVES ))XSTATE_OP (XRSTORS , xstate, lmask, hmask, err);else XSTATE_OP (XRSTOR , xstate, lmask, hmask, err);
@@ -497,8 +497,7 @@ static inline void fpregs_activate(struct fpu *fpu)* - switch_fpu_finish() restores the new state as * necessary. */ -static inline void -switch_fpu_prepare (struct fpu *old_fpu,int cpu) +static inline void switch_fpu_prepare (struct fpu *old_fpu,int cpu) {if (static_cpu_has (X86_FEATURE_FPU ) && old_fpu->initialized) {if (!copy_fpregs_to_fpstate (old_fpu))
arch/x86/kernel/apic/apic_numachip.c +1 -1
@@ -175,7 +175,7 @@ static void fixup_cpu_id(struct cpuinfo_x86 *c, int node)this_cpu_write (cpu_llc_id, node);/* Account for nodes per socket in multi-core-module processors */ -if (static_cpu_has (X86_FEATURE_NODEID_MSR )) { +if (boot_cpu_has (X86_FEATURE_NODEID_MSR )) {rdmsrl (MSR_FAM10H_NODE_ID , val); nodes = ((val >>3 ) &7 ) +1 ; }
arch/x86/kernel/cpu/aperfmperf.c +3 -3
@@ -83,7 +83,7 @@ unsigned int aperfmperf_get_khz(int cpu)if (!cpu_khz)return 0 ; -if (!static_cpu_has (X86_FEATURE_APERFMPERF )) +if (!boot_cpu_has (X86_FEATURE_APERFMPERF ))return 0 ;aperfmperf_snapshot_cpu (cpu,ktime_get (), true);
@@ -99,7 +99,7 @@ void arch_freq_prepare_all(void)if (!cpu_khz)return ; -if (!static_cpu_has (X86_FEATURE_APERFMPERF )) +if (!boot_cpu_has (X86_FEATURE_APERFMPERF ))return ;for_each_online_cpu (cpu)
@@ -115,7 +115,7 @@ unsigned int arch_freq_get_on_cpu(int cpu)if (!cpu_khz)return 0 ; -if (!static_cpu_has (X86_FEATURE_APERFMPERF )) +if (!boot_cpu_has (X86_FEATURE_APERFMPERF ))return 0 ;if (aperfmperf_snapshot_cpu (cpu,ktime_get (), true))
arch/x86/kernel/cpu/common.c +1 -1
@@ -1668,7 +1668,7 @@ static void setup_getcpu(int cpu)unsigned long cpudata =vdso_encode_cpunode (cpu,early_cpu_to_node (cpu));struct desc_struct d = { }; -if (static_cpu_has (X86_FEATURE_RDTSCP )) +if (boot_cpu_has (X86_FEATURE_RDTSCP ))write_rdtscp_aux (cpudata);/* Store CPU and node number in limit. */
arch/x86/kernel/cpu/mce/inject.c +1 -1
@@ -526,7 +526,7 @@ static void do_inject(void)* only on the node base core. Refer to D18F3x44[NbMcaToMstCpuEn] for * Fam10h and later BKDGs. */ -if (static_cpu_has (X86_FEATURE_AMD_DCM ) && +if (boot_cpu_has (X86_FEATURE_AMD_DCM ) && b ==4 && boot_cpu_data.x86 <0x17 ) {toggle_nb_mca_mst_cpu (amd_get_nb_id (cpu));
arch/x86/kernel/cpu/proc.c +5 -5
@@ -35,11 +35,11 @@ static void show_cpuinfo_misc(struct seq_file *m, struct cpuinfo_x86 *c)"fpu_exception\t: %s\n" "cpuid level\t: %d\n" "wp\t\t: yes\n" , -static_cpu_has_bug (X86_BUG_FDIV ) ?"yes" :"no" , -static_cpu_has_bug (X86_BUG_F00F ) ?"yes" :"no" , -static_cpu_has_bug (X86_BUG_COMA ) ?"yes" :"no" , -static_cpu_has (X86_FEATURE_FPU ) ?"yes" :"no" , -static_cpu_has (X86_FEATURE_FPU ) ?"yes" :"no" , +boot_cpu_has_bug (X86_BUG_FDIV ) ?"yes" :"no" , +boot_cpu_has_bug (X86_BUG_F00F ) ?"yes" :"no" , +boot_cpu_has_bug (X86_BUG_COMA ) ?"yes" :"no" , +boot_cpu_has (X86_FEATURE_FPU ) ?"yes" :"no" , +boot_cpu_has (X86_FEATURE_FPU ) ?"yes" :"no" , c->cpuid_level); }#else
arch/x86/kernel/ldt.c +7 -7
@@ -113,7 +113,7 @@ static void do_sanity_check(struct mm_struct *mm,* tables. */ WARN_ON (!had_kernel_mapping); -if (static_cpu_has (X86_FEATURE_PTI )) +if (boot_cpu_has (X86_FEATURE_PTI ))WARN_ON (!had_user_mapping); }else {/*
@@ -121,7 +121,7 @@ static void do_sanity_check(struct mm_struct *mm,* Sync the pgd to the usermode tables. */ WARN_ON (had_kernel_mapping); -if (static_cpu_has (X86_FEATURE_PTI )) +if (boot_cpu_has (X86_FEATURE_PTI ))WARN_ON (had_user_mapping); } }
@@ -156,7 +156,7 @@ static void map_ldt_struct_to_user(struct mm_struct *mm) k_pmd =pgd_to_pmd_walk (k_pgd,LDT_BASE_ADDR ); u_pmd =pgd_to_pmd_walk (u_pgd,LDT_BASE_ADDR ); -if (static_cpu_has (X86_FEATURE_PTI ) && !mm->context.ldt) +if (boot_cpu_has (X86_FEATURE_PTI ) && !mm->context.ldt)set_pmd (u_pmd, *k_pmd); }
@@ -181,7 +181,7 @@ static void map_ldt_struct_to_user(struct mm_struct *mm) {pgd_t *pgd =pgd_offset (mm,LDT_BASE_ADDR ); -if (static_cpu_has (X86_FEATURE_PTI ) && !mm->context.ldt) +if (boot_cpu_has (X86_FEATURE_PTI ) && !mm->context.ldt)set_pgd (kernel_to_user_pgdp (pgd), *pgd); }
@@ -208,7 +208,7 @@ map_ldt_struct(struct mm_struct *mm, struct ldt_struct *ldt, int slot)spinlock_t *ptl;int i, nr_pages; -if (!static_cpu_has (X86_FEATURE_PTI )) +if (!boot_cpu_has (X86_FEATURE_PTI ))return 0 ;/*
@@ -271,7 +271,7 @@ static void unmap_ldt_struct(struct mm_struct *mm, struct ldt_struct *ldt)return ;/* LDT map/unmap is only required for PTI */ -if (!static_cpu_has (X86_FEATURE_PTI )) +if (!boot_cpu_has (X86_FEATURE_PTI ))return ; nr_pages =DIV_ROUND_UP (ldt->nr_entries *LDT_ENTRY_SIZE ,PAGE_SIZE );
@@ -311,7 +311,7 @@ static void free_ldt_pgtables(struct mm_struct *mm)unsigned long start =LDT_BASE_ADDR ;unsigned long end =LDT_END_ADDR ; -if (!static_cpu_has (X86_FEATURE_PTI )) +if (!boot_cpu_has (X86_FEATURE_PTI ))return ;tlb_gather_mmu (&tlb, mm, start, end);
arch/x86/kernel/paravirt.c +1 -1
@@ -121,7 +121,7 @@ DEFINE_STATIC_KEY_TRUE(virt_spin_lock_key);void __init native_pv_lock_init (void ) { -if (!static_cpu_has (X86_FEATURE_HYPERVISOR )) +if (!boot_cpu_has (X86_FEATURE_HYPERVISOR ))static_branch_disable (&virt_spin_lock_key); }
arch/x86/kernel/process.c +2 -2
@@ -236,7 +236,7 @@ static int get_cpuid_mode(void)static int set_cpuid_mode (struct task_struct *task,unsigned long cpuid_enabled) { -if (!static_cpu_has (X86_FEATURE_CPUID_FAULT )) +if (!boot_cpu_has (X86_FEATURE_CPUID_FAULT ))return -ENODEV ;if (cpuid_enabled)
@@ -666,7 +666,7 @@ static int prefer_mwait_c1_over_halt(const struct cpuinfo_x86 *c)if (c->x86_vendor !=X86_VENDOR_INTEL )return 0 ; -if (!cpu_has (c,X86_FEATURE_MWAIT ) ||static_cpu_has_bug (X86_BUG_MONITOR )) +if (!cpu_has (c,X86_FEATURE_MWAIT ) ||boot_cpu_has_bug (X86_BUG_MONITOR ))return 0 ;return 1 ;
arch/x86/kernel/reboot.c +1 -1
@@ -108,7 +108,7 @@ void __noreturn machine_real_restart(unsigned int type)write_cr3 (real_mode_header->trampoline_pgd);/* Exiting long mode will fail if CR4.PCIDE is set. */ -if (static_cpu_has (X86_FEATURE_PCID )) +if (boot_cpu_has (X86_FEATURE_PCID ))cr4_clear_bits (X86_CR4_PCIDE );#endif
arch/x86/kernel/vm86_32.c +1 -1
@@ -369,7 +369,7 @@ static long do_sys_vm86(struct vm86plus_struct __user *user_vm86, bool plus)preempt_disable (); tsk->thread.sp0 +=16 ; -if (static_cpu_has (X86_FEATURE_SEP )) { +if (boot_cpu_has (X86_FEATURE_SEP )) { tsk->thread.sysenter_cs =0 ;refresh_sysenter_cs (&tsk->thread); }
Re: [PATCH 2/5] x86: Convert some slow-path static_cpu_has() callers to boot_cpu_has()
Quoted 5 lines · Borislav Petkov
From: Borislav Petkov <[email protected]>
Using
static_cpu_has()is pointless on those paths, convert them to theboot_cpu_has()variant.
Oh, nice. That name much better conveys what it does.
Juergen Gross
maintainerRe: [PATCH 2/5] x86: Convert some slow-path static_cpu_has() callers to boot_cpu_has()
In reply to Borislav PetkovQuoted 9 lines · Borislav Petkov
From: Borislav Petkov <[email protected]>
Using
static_cpu_has()is pointless on those paths, convert them to theboot_cpu_has()variant.No functional changes.
Reported-by Nadav Amit <[email protected]>Signed-off-by Borislav Petkov <[email protected]>
For the paravirt part: Reviewed-by: Juergen Gross <[email protected]>
Juergen
Juergen Gross
maintainerRe: [PATCH 2/5] x86: Convert some slow-path static_cpu_has() callers to boot_cpu_has()
In reply to Borislav PetkovQuoted 9 lines · Borislav Petkov
From: Borislav Petkov <[email protected]>
Using
static_cpu_has()is pointless on those paths, convert them to theboot_cpu_has()variant.No functional changes.
Reported-by Nadav Amit <[email protected]>Signed-off-by Borislav Petkov <[email protected]>
For the paravirt part: Reviewed-by: Juergen Gross <[email protected]>
Juergen
tip-bot for Borislav Petkov
merge bot[tip:x86/asm] x86: Convert some slow-path static_cpu_has() callers to boot_cpu_has()
In reply to Borislav Petkovx86: Convert some slow-path static_cpu_has() callers to boot_cpu_has()
Commit-ID: 67e87d43b794a8886b5d075b3e0fdd0c615a595f Gitweb: https://git.kernel.org/tip/67e87d43b794a8886b5d075b3e0fdd0c615a595f Author: Borislav Petkov <[email protected]> AuthorDate: Fri, 29 Mar 2019 19:52:59 +0100 Committer: Borislav Petkov <[email protected]> CommitDate: Mon, 8 Apr 2019 12:13:34 +0200
x86: Convert some slow-path static_cpu_has() callers to boot_cpu_has()
Using static_cpu_has() is pointless on those paths, convert them to the boot_cpu_has() variant.
No functional changes.
--- arch/x86/include/asm/fpu/internal.h | 7 +++---- arch/x86/kernel/apic/apic_numachip.c | 2 +- arch/x86/kernel/cpu/aperfmperf.c | 6 +++--- arch/x86/kernel/cpu/common.c | 2 +- arch/x86/kernel/cpu/mce/inject.c | 2 +- arch/x86/kernel/cpu/proc.c | 10 +++++----- arch/x86/kernel/ldt.c | 14 +++++++------- arch/x86/kernel/paravirt.c | 2 +- arch/x86/kernel/process.c | 4 ++-- arch/x86/kernel/reboot.c | 2 +- arch/x86/kernel/vm86_32.c | 2 +- 11 files changed, 26 insertions(+), 27 deletions(-)
arch/x86/include/asm/fpu/internal.h +3 -4
@@ -253,7 +253,7 @@ static inline void copy_xregs_to_kernel_booting(struct xregs_state *xstate)WARN_ON (system_state !=SYSTEM_BOOTING ); -if (static_cpu_has (X86_FEATURE_XSAVES )) +if (boot_cpu_has (X86_FEATURE_XSAVES ))XSTATE_OP (XSAVES , xstate, lmask, hmask, err);else XSTATE_OP (XSAVE , xstate, lmask, hmask, err);
@@ -275,7 +275,7 @@ static inline void copy_kernel_to_xregs_booting(struct xregs_state *xstate)WARN_ON (system_state !=SYSTEM_BOOTING ); -if (static_cpu_has (X86_FEATURE_XSAVES )) +if (boot_cpu_has (X86_FEATURE_XSAVES ))XSTATE_OP (XRSTORS , xstate, lmask, hmask, err);else XSTATE_OP (XRSTOR , xstate, lmask, hmask, err);
@@ -497,8 +497,7 @@ static inline void fpregs_activate(struct fpu *fpu)* - switch_fpu_finish() restores the new state as * necessary. */ -static inline void -switch_fpu_prepare (struct fpu *old_fpu,int cpu) +static inline void switch_fpu_prepare (struct fpu *old_fpu,int cpu) {if (static_cpu_has (X86_FEATURE_FPU ) && old_fpu->initialized) {if (!copy_fpregs_to_fpstate (old_fpu))
arch/x86/kernel/apic/apic_numachip.c +1 -1
@@ -175,7 +175,7 @@ static void fixup_cpu_id(struct cpuinfo_x86 *c, int node)this_cpu_write (cpu_llc_id, node);/* Account for nodes per socket in multi-core-module processors */ -if (static_cpu_has (X86_FEATURE_NODEID_MSR )) { +if (boot_cpu_has (X86_FEATURE_NODEID_MSR )) {rdmsrl (MSR_FAM10H_NODE_ID , val); nodes = ((val >>3 ) &7 ) +1 ; }
arch/x86/kernel/cpu/aperfmperf.c +3 -3
@@ -83,7 +83,7 @@ unsigned int aperfmperf_get_khz(int cpu)if (!cpu_khz)return 0 ; -if (!static_cpu_has (X86_FEATURE_APERFMPERF )) +if (!boot_cpu_has (X86_FEATURE_APERFMPERF ))return 0 ;aperfmperf_snapshot_cpu (cpu,ktime_get (), true);
@@ -99,7 +99,7 @@ void arch_freq_prepare_all(void)if (!cpu_khz)return ; -if (!static_cpu_has (X86_FEATURE_APERFMPERF )) +if (!boot_cpu_has (X86_FEATURE_APERFMPERF ))return ;for_each_online_cpu (cpu)
@@ -115,7 +115,7 @@ unsigned int arch_freq_get_on_cpu(int cpu)if (!cpu_khz)return 0 ; -if (!static_cpu_has (X86_FEATURE_APERFMPERF )) +if (!boot_cpu_has (X86_FEATURE_APERFMPERF ))return 0 ;if (aperfmperf_snapshot_cpu (cpu,ktime_get (), true))
arch/x86/kernel/cpu/common.c +1 -1
@@ -1668,7 +1668,7 @@ static void setup_getcpu(int cpu)unsigned long cpudata =vdso_encode_cpunode (cpu,early_cpu_to_node (cpu));struct desc_struct d = { }; -if (static_cpu_has (X86_FEATURE_RDTSCP )) +if (boot_cpu_has (X86_FEATURE_RDTSCP ))write_rdtscp_aux (cpudata);/* Store CPU and node number in limit. */
arch/x86/kernel/cpu/mce/inject.c +1 -1
@@ -528,7 +528,7 @@ static void do_inject(void)* only on the node base core. Refer to D18F3x44[NbMcaToMstCpuEn] for * Fam10h and later BKDGs. */ -if (static_cpu_has (X86_FEATURE_AMD_DCM ) && +if (boot_cpu_has (X86_FEATURE_AMD_DCM ) && b ==4 && boot_cpu_data.x86 <0x17 ) {toggle_nb_mca_mst_cpu (amd_get_nb_id (cpu));
arch/x86/kernel/cpu/proc.c +5 -5
@@ -35,11 +35,11 @@ static void show_cpuinfo_misc(struct seq_file *m, struct cpuinfo_x86 *c)"fpu_exception\t: %s\n" "cpuid level\t: %d\n" "wp\t\t: yes\n" , -static_cpu_has_bug (X86_BUG_FDIV ) ?"yes" :"no" , -static_cpu_has_bug (X86_BUG_F00F ) ?"yes" :"no" , -static_cpu_has_bug (X86_BUG_COMA ) ?"yes" :"no" , -static_cpu_has (X86_FEATURE_FPU ) ?"yes" :"no" , -static_cpu_has (X86_FEATURE_FPU ) ?"yes" :"no" , +boot_cpu_has_bug (X86_BUG_FDIV ) ?"yes" :"no" , +boot_cpu_has_bug (X86_BUG_F00F ) ?"yes" :"no" , +boot_cpu_has_bug (X86_BUG_COMA ) ?"yes" :"no" , +boot_cpu_has (X86_FEATURE_FPU ) ?"yes" :"no" , +boot_cpu_has (X86_FEATURE_FPU ) ?"yes" :"no" , c->cpuid_level); }#else
arch/x86/kernel/ldt.c +7 -7
@@ -113,7 +113,7 @@ static void do_sanity_check(struct mm_struct *mm,* tables. */ WARN_ON (!had_kernel_mapping); -if (static_cpu_has (X86_FEATURE_PTI )) +if (boot_cpu_has (X86_FEATURE_PTI ))WARN_ON (!had_user_mapping); }else {/*
@@ -121,7 +121,7 @@ static void do_sanity_check(struct mm_struct *mm,* Sync the pgd to the usermode tables. */ WARN_ON (had_kernel_mapping); -if (static_cpu_has (X86_FEATURE_PTI )) +if (boot_cpu_has (X86_FEATURE_PTI ))WARN_ON (had_user_mapping); } }
@@ -156,7 +156,7 @@ static void map_ldt_struct_to_user(struct mm_struct *mm) k_pmd =pgd_to_pmd_walk (k_pgd,LDT_BASE_ADDR ); u_pmd =pgd_to_pmd_walk (u_pgd,LDT_BASE_ADDR ); -if (static_cpu_has (X86_FEATURE_PTI ) && !mm->context.ldt) +if (boot_cpu_has (X86_FEATURE_PTI ) && !mm->context.ldt)set_pmd (u_pmd, *k_pmd); }
@@ -181,7 +181,7 @@ static void map_ldt_struct_to_user(struct mm_struct *mm) {pgd_t *pgd =pgd_offset (mm,LDT_BASE_ADDR ); -if (static_cpu_has (X86_FEATURE_PTI ) && !mm->context.ldt) +if (boot_cpu_has (X86_FEATURE_PTI ) && !mm->context.ldt)set_pgd (kernel_to_user_pgdp (pgd), *pgd); }
@@ -208,7 +208,7 @@ map_ldt_struct(struct mm_struct *mm, struct ldt_struct *ldt, int slot)spinlock_t *ptl;int i, nr_pages; -if (!static_cpu_has (X86_FEATURE_PTI )) +if (!boot_cpu_has (X86_FEATURE_PTI ))return 0 ;/*
@@ -271,7 +271,7 @@ static void unmap_ldt_struct(struct mm_struct *mm, struct ldt_struct *ldt)return ;/* LDT map/unmap is only required for PTI */ -if (!static_cpu_has (X86_FEATURE_PTI )) +if (!boot_cpu_has (X86_FEATURE_PTI ))return ; nr_pages =DIV_ROUND_UP (ldt->nr_entries *LDT_ENTRY_SIZE ,PAGE_SIZE );
@@ -311,7 +311,7 @@ static void free_ldt_pgtables(struct mm_struct *mm)unsigned long start =LDT_BASE_ADDR ;unsigned long end =LDT_END_ADDR ; -if (!static_cpu_has (X86_FEATURE_PTI )) +if (!boot_cpu_has (X86_FEATURE_PTI ))return ;tlb_gather_mmu (&tlb, mm, start, end);
arch/x86/kernel/paravirt.c +1 -1
@@ -121,7 +121,7 @@ DEFINE_STATIC_KEY_TRUE(virt_spin_lock_key);void __init native_pv_lock_init (void ) { -if (!static_cpu_has (X86_FEATURE_HYPERVISOR )) +if (!boot_cpu_has (X86_FEATURE_HYPERVISOR ))static_branch_disable (&virt_spin_lock_key); }
arch/x86/kernel/process.c +2 -2
@@ -236,7 +236,7 @@ static int get_cpuid_mode(void)static int set_cpuid_mode (struct task_struct *task,unsigned long cpuid_enabled) { -if (!static_cpu_has (X86_FEATURE_CPUID_FAULT )) +if (!boot_cpu_has (X86_FEATURE_CPUID_FAULT ))return -ENODEV ;if (cpuid_enabled)
@@ -666,7 +666,7 @@ static int prefer_mwait_c1_over_halt(const struct cpuinfo_x86 *c)if (c->x86_vendor !=X86_VENDOR_INTEL )return 0 ; -if (!cpu_has (c,X86_FEATURE_MWAIT ) ||static_cpu_has_bug (X86_BUG_MONITOR )) +if (!cpu_has (c,X86_FEATURE_MWAIT ) ||boot_cpu_has_bug (X86_BUG_MONITOR ))return 0 ;return 1 ;
arch/x86/kernel/reboot.c +1 -1
@@ -108,7 +108,7 @@ void __noreturn machine_real_restart(unsigned int type)write_cr3 (real_mode_header->trampoline_pgd);/* Exiting long mode will fail if CR4.PCIDE is set. */ -if (static_cpu_has (X86_FEATURE_PCID )) +if (boot_cpu_has (X86_FEATURE_PCID ))cr4_clear_bits (X86_CR4_PCIDE );#endif
arch/x86/kernel/vm86_32.c +1 -1
@@ -369,7 +369,7 @@ static long do_sys_vm86(struct vm86plus_struct __user *user_vm86, bool plus)preempt_disable (); tsk->thread.sp0 +=16 ; -if (static_cpu_has (X86_FEATURE_SEP )) { +if (boot_cpu_has (X86_FEATURE_SEP )) { tsk->thread.sysenter_cs =0 ;refresh_sysenter_cs (&tsk->thread); }
Borislav Petkov
maintainerPATCH 2/5 x86: Convert some slow-path static_cpu_has() callers to boot_cpu_has()
In reply to Borislav PetkovFrom: Borislav Petkov <[email protected]>
Using static_cpu_has() is pointless on those paths, convert them to the boot_cpu_has() variant.
No functional changes.
--- arch/x86/include/asm/fpu/internal.h | 7 +++---- arch/x86/kernel/apic/apic_numachip.c | 2 +- arch/x86/kernel/cpu/aperfmperf.c | 6 +++--- arch/x86/kernel/cpu/common.c | 2 +- arch/x86/kernel/cpu/mce/inject.c | 2 +- arch/x86/kernel/cpu/proc.c | 10 +++++----- arch/x86/kernel/ldt.c | 14 +++++++------- arch/x86/kernel/paravirt.c | 2 +- arch/x86/kernel/process.c | 4 ++-- arch/x86/kernel/reboot.c | 2 +- arch/x86/kernel/vm86_32.c | 2 +- 11 files changed, 26 insertions(+), 27 deletions(-)
arch/x86/include/asm/fpu/internal.h +3 -4
@@ -253,7 +253,7 @@ static inline void copy_xregs_to_kernel_booting(struct xregs_state *xstate)WARN_ON (system_state !=SYSTEM_BOOTING ); -if (static_cpu_has (X86_FEATURE_XSAVES )) +if (boot_cpu_has (X86_FEATURE_XSAVES ))XSTATE_OP (XSAVES , xstate, lmask, hmask, err);else XSTATE_OP (XSAVE , xstate, lmask, hmask, err);
@@ -275,7 +275,7 @@ static inline void copy_kernel_to_xregs_booting(struct xregs_state *xstate)WARN_ON (system_state !=SYSTEM_BOOTING ); -if (static_cpu_has (X86_FEATURE_XSAVES )) +if (boot_cpu_has (X86_FEATURE_XSAVES ))XSTATE_OP (XRSTORS , xstate, lmask, hmask, err);else XSTATE_OP (XRSTOR , xstate, lmask, hmask, err);
@@ -497,8 +497,7 @@ static inline void fpregs_activate(struct fpu *fpu)* - switch_fpu_finish() restores the new state as * necessary. */ -static inline void -switch_fpu_prepare (struct fpu *old_fpu,int cpu) +static inline void switch_fpu_prepare (struct fpu *old_fpu,int cpu) {if (static_cpu_has (X86_FEATURE_FPU ) && old_fpu->initialized) {if (!copy_fpregs_to_fpstate (old_fpu))
arch/x86/kernel/apic/apic_numachip.c +1 -1
@@ -175,7 +175,7 @@ static void fixup_cpu_id(struct cpuinfo_x86 *c, int node)this_cpu_write (cpu_llc_id, node);/* Account for nodes per socket in multi-core-module processors */ -if (static_cpu_has (X86_FEATURE_NODEID_MSR )) { +if (boot_cpu_has (X86_FEATURE_NODEID_MSR )) {rdmsrl (MSR_FAM10H_NODE_ID , val); nodes = ((val >>3 ) &7 ) +1 ; }
arch/x86/kernel/cpu/aperfmperf.c +3 -3
@@ -83,7 +83,7 @@ unsigned int aperfmperf_get_khz(int cpu)if (!cpu_khz)return 0 ; -if (!static_cpu_has (X86_FEATURE_APERFMPERF )) +if (!boot_cpu_has (X86_FEATURE_APERFMPERF ))return 0 ;aperfmperf_snapshot_cpu (cpu,ktime_get (), true);
@@ -99,7 +99,7 @@ void arch_freq_prepare_all(void)if (!cpu_khz)return ; -if (!static_cpu_has (X86_FEATURE_APERFMPERF )) +if (!boot_cpu_has (X86_FEATURE_APERFMPERF ))return ;for_each_online_cpu (cpu)
@@ -115,7 +115,7 @@ unsigned int arch_freq_get_on_cpu(int cpu)if (!cpu_khz)return 0 ; -if (!static_cpu_has (X86_FEATURE_APERFMPERF )) +if (!boot_cpu_has (X86_FEATURE_APERFMPERF ))return 0 ;if (aperfmperf_snapshot_cpu (cpu,ktime_get (), true))
arch/x86/kernel/cpu/common.c +1 -1
@@ -1668,7 +1668,7 @@ static void setup_getcpu(int cpu)unsigned long cpudata =vdso_encode_cpunode (cpu,early_cpu_to_node (cpu));struct desc_struct d = { }; -if (static_cpu_has (X86_FEATURE_RDTSCP )) +if (boot_cpu_has (X86_FEATURE_RDTSCP ))write_rdtscp_aux (cpudata);/* Store CPU and node number in limit. */
arch/x86/kernel/cpu/mce/inject.c +1 -1
@@ -526,7 +526,7 @@ static void do_inject(void)* only on the node base core. Refer to D18F3x44[NbMcaToMstCpuEn] for * Fam10h and later BKDGs. */ -if (static_cpu_has (X86_FEATURE_AMD_DCM ) && +if (boot_cpu_has (X86_FEATURE_AMD_DCM ) && b ==4 && boot_cpu_data.x86 <0x17 ) {toggle_nb_mca_mst_cpu (amd_get_nb_id (cpu));
arch/x86/kernel/cpu/proc.c +5 -5
@@ -35,11 +35,11 @@ static void show_cpuinfo_misc(struct seq_file *m, struct cpuinfo_x86 *c)"fpu_exception\t: %s\n" "cpuid level\t: %d\n" "wp\t\t: yes\n" , -static_cpu_has_bug (X86_BUG_FDIV ) ?"yes" :"no" , -static_cpu_has_bug (X86_BUG_F00F ) ?"yes" :"no" , -static_cpu_has_bug (X86_BUG_COMA ) ?"yes" :"no" , -static_cpu_has (X86_FEATURE_FPU ) ?"yes" :"no" , -static_cpu_has (X86_FEATURE_FPU ) ?"yes" :"no" , +boot_cpu_has_bug (X86_BUG_FDIV ) ?"yes" :"no" , +boot_cpu_has_bug (X86_BUG_F00F ) ?"yes" :"no" , +boot_cpu_has_bug (X86_BUG_COMA ) ?"yes" :"no" , +boot_cpu_has (X86_FEATURE_FPU ) ?"yes" :"no" , +boot_cpu_has (X86_FEATURE_FPU ) ?"yes" :"no" , c->cpuid_level); }#else
arch/x86/kernel/ldt.c +7 -7
@@ -113,7 +113,7 @@ static void do_sanity_check(struct mm_struct *mm,* tables. */ WARN_ON (!had_kernel_mapping); -if (static_cpu_has (X86_FEATURE_PTI )) +if (boot_cpu_has (X86_FEATURE_PTI ))WARN_ON (!had_user_mapping); }else {/*
@@ -121,7 +121,7 @@ static void do_sanity_check(struct mm_struct *mm,* Sync the pgd to the usermode tables. */ WARN_ON (had_kernel_mapping); -if (static_cpu_has (X86_FEATURE_PTI )) +if (boot_cpu_has (X86_FEATURE_PTI ))WARN_ON (had_user_mapping); } }
@@ -156,7 +156,7 @@ static void map_ldt_struct_to_user(struct mm_struct *mm) k_pmd =pgd_to_pmd_walk (k_pgd,LDT_BASE_ADDR ); u_pmd =pgd_to_pmd_walk (u_pgd,LDT_BASE_ADDR ); -if (static_cpu_has (X86_FEATURE_PTI ) && !mm->context.ldt) +if (boot_cpu_has (X86_FEATURE_PTI ) && !mm->context.ldt)set_pmd (u_pmd, *k_pmd); }
@@ -181,7 +181,7 @@ static void map_ldt_struct_to_user(struct mm_struct *mm) {pgd_t *pgd =pgd_offset (mm,LDT_BASE_ADDR ); -if (static_cpu_has (X86_FEATURE_PTI ) && !mm->context.ldt) +if (boot_cpu_has (X86_FEATURE_PTI ) && !mm->context.ldt)set_pgd (kernel_to_user_pgdp (pgd), *pgd); }
@@ -208,7 +208,7 @@ map_ldt_struct(struct mm_struct *mm, struct ldt_struct *ldt, int slot)spinlock_t *ptl;int i, nr_pages; -if (!static_cpu_has (X86_FEATURE_PTI )) +if (!boot_cpu_has (X86_FEATURE_PTI ))return 0 ;/*
@@ -271,7 +271,7 @@ static void unmap_ldt_struct(struct mm_struct *mm, struct ldt_struct *ldt)return ;/* LDT map/unmap is only required for PTI */ -if (!static_cpu_has (X86_FEATURE_PTI )) +if (!boot_cpu_has (X86_FEATURE_PTI ))return ; nr_pages =DIV_ROUND_UP (ldt->nr_entries *LDT_ENTRY_SIZE ,PAGE_SIZE );
@@ -311,7 +311,7 @@ static void free_ldt_pgtables(struct mm_struct *mm)unsigned long start =LDT_BASE_ADDR ;unsigned long end =LDT_END_ADDR ; -if (!static_cpu_has (X86_FEATURE_PTI )) +if (!boot_cpu_has (X86_FEATURE_PTI ))return ;tlb_gather_mmu (&tlb, mm, start, end);
arch/x86/kernel/paravirt.c +1 -1
@@ -121,7 +121,7 @@ DEFINE_STATIC_KEY_TRUE(virt_spin_lock_key);void __init native_pv_lock_init (void ) { -if (!static_cpu_has (X86_FEATURE_HYPERVISOR )) +if (!boot_cpu_has (X86_FEATURE_HYPERVISOR ))static_branch_disable (&virt_spin_lock_key); }
arch/x86/kernel/process.c +2 -2
@@ -236,7 +236,7 @@ static int get_cpuid_mode(void)static int set_cpuid_mode (struct task_struct *task,unsigned long cpuid_enabled) { -if (!static_cpu_has (X86_FEATURE_CPUID_FAULT )) +if (!boot_cpu_has (X86_FEATURE_CPUID_FAULT ))return -ENODEV ;if (cpuid_enabled)
@@ -666,7 +666,7 @@ static int prefer_mwait_c1_over_halt(const struct cpuinfo_x86 *c)if (c->x86_vendor !=X86_VENDOR_INTEL )return 0 ; -if (!cpu_has (c,X86_FEATURE_MWAIT ) ||static_cpu_has_bug (X86_BUG_MONITOR )) +if (!cpu_has (c,X86_FEATURE_MWAIT ) ||boot_cpu_has_bug (X86_BUG_MONITOR ))return 0 ;return 1 ;
arch/x86/kernel/reboot.c +1 -1
@@ -108,7 +108,7 @@ void __noreturn machine_real_restart(unsigned int type)write_cr3 (real_mode_header->trampoline_pgd);/* Exiting long mode will fail if CR4.PCIDE is set. */ -if (static_cpu_has (X86_FEATURE_PCID )) +if (boot_cpu_has (X86_FEATURE_PCID ))cr4_clear_bits (X86_CR4_PCIDE );#endif
arch/x86/kernel/vm86_32.c +1 -1
@@ -369,7 +369,7 @@ static long do_sys_vm86(struct vm86plus_struct __user *user_vm86, bool plus)preempt_disable (); tsk->thread.sp0 +=16 ; -if (static_cpu_has (X86_FEATURE_SEP )) { +if (boot_cpu_has (X86_FEATURE_SEP )) { tsk->thread.sysenter_cs =0 ;refresh_sysenter_cs (&tsk->thread); }
Signature · 1 line
2.21.0
Borislav Petkov
maintainerPATCH 3/5 x86/kvm: Convert some slow-path static_cpu_has() callers to boot_cpu_has()
In reply to Borislav PetkovFrom: Borislav Petkov <[email protected]>
Using static_cpu_has() is pointless on those paths, convert them to the boot_cpu_has() variant.
No functional changes.
--- arch/x86/kvm/svm.c | 10 +++++----- arch/x86/kvm/vmx/vmx.c | 4 ++-- 2 files changed, 7 insertions(+), 7 deletions(-)
arch/x86/kvm/svm.c +5 -5
@@ -835,7 +835,7 @@ static void svm_init_erratum_383(void)int err;u64 val; -if (!static_cpu_has_bug (X86_BUG_AMD_TLB_MMATCH )) +if (!boot_cpu_has_bug (X86_BUG_AMD_TLB_MMATCH ))return ;/* Use _safe variants to not break nested virtualization */
@@ -889,7 +889,7 @@ static int has_svm(void)static void svm_hardware_disable (void ) {/* Make sure we clean up behind us */ -if (static_cpu_has (X86_FEATURE_TSCRATEMSR )) +if (boot_cpu_has (X86_FEATURE_TSCRATEMSR ))wrmsrl (MSR_AMD64_TSC_RATIO ,TSC_RATIO_DEFAULT );cpu_svm_disable ();
@@ -931,7 +931,7 @@ static int svm_hardware_enable(void)wrmsrl (MSR_VM_HSAVE_PA ,page_to_pfn (sd->save_area) <<PAGE_SHIFT ); -if (static_cpu_has (X86_FEATURE_TSCRATEMSR )) { +if (boot_cpu_has (X86_FEATURE_TSCRATEMSR )) {wrmsrl (MSR_AMD64_TSC_RATIO ,TSC_RATIO_DEFAULT );__this_cpu_write (current_tsc_ratio,TSC_RATIO_DEFAULT ); }
@@ -2252,7 +2252,7 @@ static void svm_vcpu_load(struct kvm_vcpu *vcpu, int cpu)for (i =0 ; i <NR_HOST_SAVE_USER_MSRS ; i++)rdmsrl (host_save_user_msrs[i], svm->host_user_msrs[i]); -if (static_cpu_has (X86_FEATURE_TSCRATEMSR )) { +if (boot_cpu_has (X86_FEATURE_TSCRATEMSR )) {u64 tsc_ratio = vcpu->arch.tsc_scaling_ratio;if (tsc_ratio !=__this_cpu_read (current_tsc_ratio)) {__this_cpu_write (current_tsc_ratio, tsc_ratio);
@@ -2260,7 +2260,7 @@ static void svm_vcpu_load(struct kvm_vcpu *vcpu, int cpu) } }/* This assumes that the kernel never uses MSR_TSC_AUX */ -if (static_cpu_has (X86_FEATURE_RDTSCP )) +if (boot_cpu_has (X86_FEATURE_RDTSCP ))wrmsrl (MSR_TSC_AUX , svm->tsc_aux);if (sd->current_vmcb != svm->vmcb) {
arch/x86/kvm/vmx/vmx.c +2 -2
@@ -6423,7 +6423,7 @@ static void vmx_vcpu_run(struct kvm_vcpu *vcpu)if (vcpu->guest_debug &KVM_GUESTDBG_SINGLESTEP )vmx_set_interrupt_shadow (vcpu,0 ); -if (static_cpu_has (X86_FEATURE_PKU ) && +if (boot_cpu_has (X86_FEATURE_PKU ) &&kvm_read_cr4_bits (vcpu,X86_CR4_PKE ) && vcpu->arch.pkru != vmx->host_pkru)__write_pkru (vcpu->arch.pkru);
@@ -6512,7 +6512,7 @@ static void vmx_vcpu_run(struct kvm_vcpu *vcpu)* back on host, so it is safe to read guest PKRU from current * XSAVE. */ -if (static_cpu_has (X86_FEATURE_PKU ) && +if (boot_cpu_has (X86_FEATURE_PKU ) &&kvm_read_cr4_bits (vcpu,X86_CR4_PKE )) { vcpu->arch.pkru =__read_pkru ();if (vcpu->arch.pkru != vmx->host_pkru)
Signature · 1 line
2.21.0
Paolo Bonzini
maintainerRe: [PATCH 3/5] x86/kvm: Convert some slow-path static_cpu_has() callers to boot_cpu_has()
Quoted 32 lines · Borislav Petkov
arch/x86/kvm/svm.c +2
-2@@ -2252,7 +2252,7 @@ static void svm_vcpu_load(struct kvm_vcpu *vcpu, int cpu)for (i =0 ; i <NR_HOST_SAVE_USER_MSRS ; i++)rdmsrl (host_save_user_msrs[i], svm->host_user_msrs[i]); -if (static_cpu_has (X86_FEATURE_TSCRATEMSR )) { +if (boot_cpu_has (X86_FEATURE_TSCRATEMSR )) {u64 tsc_ratio = vcpu->arch.tsc_scaling_ratio;if (tsc_ratio !=__this_cpu_read (current_tsc_ratio)) {__this_cpu_write (current_tsc_ratio, tsc_ratio);@@ -2260,7 +2260,7 @@ static void svm_vcpu_load(struct kvm_vcpu *vcpu, int cpu) } }/* This assumes that the kernel never uses MSR_TSC_AUX */ -if (static_cpu_has (X86_FEATURE_RDTSCP )) +if (boot_cpu_has (X86_FEATURE_RDTSCP ))wrmsrl (MSR_TSC_AUX , svm->tsc_aux);if (sd->current_vmcb != svm->vmcb) {arch/x86/kvm/vmx/vmx.c +1
-1@@ -6423,7 +6423,7 @@ static void vmx_vcpu_run(struct kvm_vcpu *vcpu)if (vcpu->guest_debug &KVM_GUESTDBG_SINGLESTEP )vmx_set_interrupt_shadow (vcpu,0 ); -if (static_cpu_has (X86_FEATURE_PKU ) && +if (boot_cpu_has (X86_FEATURE_PKU ) &&kvm_read_cr4_bits (vcpu,X86_CR4_PKE ) && vcpu->arch.pkru != vmx->host_pkru)__write_pkru (vcpu->arch.pkru);
arch/x86/kvm/vmx/vmx.c static void vmx_vcpu_run(struct kvm_vcpu *vcpu) · line 6517
Quoted 7 lines
arch/x86/kvm/vmx/vmx.c +1
-1@@ -6512,7 +6512,7 @@ static void vmx_vcpu_run(struct kvm_vcpu *vcpu)* back on host, so it is safe to read guest PKRU from current * XSAVE. */ -if (static_cpu_has (X86_FEATURE_PKU ) && +if (boot_cpu_has (X86_FEATURE_PKU ) &&kvm_read_cr4_bits (vcpu,X86_CR4_PKE )) { vcpu->arch.pkru =__read_pkru ();
These are not slow path.
Paolo
Borislav Petkov
maintainerRe: [PATCH 3/5] x86/kvm: Convert some slow-path static_cpu_has() callers to boot_cpu_has()
Quoted 1 line · Paolo Bonzini
These are not slow path.
Those functions do a *lot* of stuff like a bunch of MSR reads which are tens of cycles each at least.
I don't think a RIP-relative MOV and a BT:
movq boot_cpu_data+20(%rip), %rax # MEM[(const long unsigned int *)&boot_cpu_data + 20B], _45
btq $59, %rax #, _45are at all noticeable.
On latest AMD and Intel uarch those are 2-4 cycles, according to
https://agner.org/optimize/instruction_tables.ods
Signature · 4 lines
Regards/Gruss,
Boris.
Good mailing practices for 400: avoid top-posting and trim the reply.Paolo Bonzini
maintainerRe: [PATCH 3/5] x86/kvm: Convert some slow-path static_cpu_has() callers to boot_cpu_has()
Quoted 5 lines · Borislav Petkov
Quoted 1 line · Paolo Bonzini
These are not slow path.
Those functions do a *lot* of stuff like a bunch of MSR reads which are tens of cycles each at least.
The MSR reads and writes are not done in the common case. Also, you cannot really expect boot_cpu_data to be in L1 in these functions since they run after the guest---or if they do, each L1 line you fill in with host data is one line you "steal" from the guest.
Paolo
Quoted 11 lines
I don't think a RIP-relative MOV and a BT:
movq boot_cpu_data+20(%rip), %rax # MEM[(const long unsigned int *)&boot_cpu_data + 20B], _45 btq $59, %rax #, _45are at all noticeable.
On latest AMD and Intel uarch those are 2-4 cycles, according to
Borislav Petkov
maintainerRe: [PATCH 3/5] x86/kvm: Convert some slow-path static_cpu_has() callers to boot_cpu_has()
Quoted 4 lines · Paolo Bonzini
The MSR reads and writes are not done in the common case. Also, you cannot really expect
boot_cpu_datato be in L1 in these functions since they run after the guest---or if they do, each L1 line you fill in with host data is one line you "steal" from the guest.
Ok, fair enough. I'll drop this patch.
Signature · 4 lines
Regards/Gruss,
Boris.
Good mailing practices for 400: avoid top-posting and trim the reply.Borislav Petkov
maintainerPATCH 4/5 x86/mm: Convert some slow-path static_cpu_has() callers to boot_cpu_has()
In reply to Borislav PetkovFrom: Borislav Petkov <[email protected]>
Using static_cpu_has() is pointless on those paths, convert them to the boot_cpu_has() variant.
No functional changes.
--- arch/x86/mm/dump_pagetables.c | 4 ++-- arch/x86/mm/pgtable.c | 4 ++-- arch/x86/mm/pti.c | 2 +- 3 files changed, 5 insertions(+), 5 deletions(-)
arch/x86/mm/dump_pagetables.c +2 -2
@@ -577,7 +577,7 @@ void ptdump_walk_pgd_level(struct seq_file *m, pgd_t *pgd)void ptdump_walk_pgd_level_debugfs (struct seq_file *m,pgd_t *pgd,bool user) {#ifdef CONFIG_PAGE_TABLE_ISOLATION -if (user &&static_cpu_has (X86_FEATURE_PTI )) +if (user &&boot_cpu_has (X86_FEATURE_PTI )) pgd =kernel_to_user_pgdp (pgd);#endif ptdump_walk_pgd_level_core (m, pgd, false, false);
@@ -590,7 +590,7 @@ void ptdump_walk_user_pgd_level_checkwx(void)pgd_t *pgd =INIT_PGD ;if (!(__supported_pte_mask & _PAGE_NX) || - !static_cpu_has (X86_FEATURE_PTI )) + !boot_cpu_has (X86_FEATURE_PTI ))return ;pr_info ("x86/mm: Checking user space page tables\n" );
arch/x86/mm/pgtable.c +2 -2
@@ -190,7 +190,7 @@ static void pgd_dtor(pgd_t *pgd)* when PTI is enabled. We need them to map the per-process LDT into the * user-space page-table. */ -#define PREALLOCATED_USER_PMDS (static_cpu_has(X86_FEATURE_PTI) ? \ +#define PREALLOCATED_USER_PMDS (boot_cpu_has(X86_FEATURE_PTI) ? \ KERNEL_PGD_PTRS : 0)#define MAX_PREALLOCATED_USER_PMDS KERNEL_PGD_PTRS
@@ -292,7 +292,7 @@ static void pgd_mop_up_pmds(struct mm_struct *mm, pgd_t *pgdp)#ifdef CONFIG_PAGE_TABLE_ISOLATION -if (!static_cpu_has (X86_FEATURE_PTI )) +if (!boot_cpu_has (X86_FEATURE_PTI ))return ; pgdp =kernel_to_user_pgdp (pgdp);
arch/x86/mm/pti.c +1 -1
@@ -626,7 +626,7 @@ static void pti_set_kernel_image_nonglobal(void)*/ void __init pti_init (void ) { -if (!static_cpu_has (X86_FEATURE_PTI )) +if (!boot_cpu_has (X86_FEATURE_PTI ))return ;pr_info ("enabled\n" );
Signature · 1 line
2.21.0
tip-bot for Borislav Petkov
merge bot[tip:x86/asm] x86/mm: Convert some slow-path static_cpu_has() callers to boot_cpu_has()
x86/mm: Convert some slow-path static_cpu_has() callers to boot_cpu_has()
Commit-ID: 28e3ace70c3d2ea47a62dffe046011d1b74ee839 Gitweb: https://git.kernel.org/tip/28e3ace70c3d2ea47a62dffe046011d1b74ee839 Author: Borislav Petkov <[email protected]> AuthorDate: Fri, 29 Mar 2019 20:00:38 +0100 Committer: Borislav Petkov <[email protected]> CommitDate: Mon, 8 Apr 2019 12:14:16 +0200
x86/mm: Convert some slow-path static_cpu_has() callers to boot_cpu_has()
Using static_cpu_has() is pointless on those paths, convert them to the boot_cpu_has() variant.
No functional changes.
--- arch/x86/mm/dump_pagetables.c | 4 ++-- arch/x86/mm/pgtable.c | 4 ++-- arch/x86/mm/pti.c | 2 +- 3 files changed, 5 insertions(+), 5 deletions(-)
arch/x86/mm/dump_pagetables.c +2 -2
@@ -577,7 +577,7 @@ void ptdump_walk_pgd_level(struct seq_file *m, pgd_t *pgd)void ptdump_walk_pgd_level_debugfs (struct seq_file *m,pgd_t *pgd,bool user) {#ifdef CONFIG_PAGE_TABLE_ISOLATION -if (user &&static_cpu_has (X86_FEATURE_PTI )) +if (user &&boot_cpu_has (X86_FEATURE_PTI )) pgd =kernel_to_user_pgdp (pgd);#endif ptdump_walk_pgd_level_core (m, pgd, false, false);
@@ -590,7 +590,7 @@ void ptdump_walk_user_pgd_level_checkwx(void)pgd_t *pgd =INIT_PGD ;if (!(__supported_pte_mask & _PAGE_NX) || - !static_cpu_has (X86_FEATURE_PTI )) + !boot_cpu_has (X86_FEATURE_PTI ))return ;pr_info ("x86/mm: Checking user space page tables\n" );
arch/x86/mm/pgtable.c +2 -2
@@ -190,7 +190,7 @@ static void pgd_dtor(pgd_t *pgd)* when PTI is enabled. We need them to map the per-process LDT into the * user-space page-table. */ -#define PREALLOCATED_USER_PMDS (static_cpu_has(X86_FEATURE_PTI) ? \ +#define PREALLOCATED_USER_PMDS (boot_cpu_has(X86_FEATURE_PTI) ? \ KERNEL_PGD_PTRS : 0)#define MAX_PREALLOCATED_USER_PMDS KERNEL_PGD_PTRS
@@ -292,7 +292,7 @@ static void pgd_mop_up_pmds(struct mm_struct *mm, pgd_t *pgdp)#ifdef CONFIG_PAGE_TABLE_ISOLATION -if (!static_cpu_has (X86_FEATURE_PTI )) +if (!boot_cpu_has (X86_FEATURE_PTI ))return ; pgdp =kernel_to_user_pgdp (pgdp);
arch/x86/mm/pti.c +1 -1
@@ -626,7 +626,7 @@ void pti_set_kernel_image_nonglobal(void)*/ void __init pti_init (void ) { -if (!static_cpu_has (X86_FEATURE_PTI )) +if (!boot_cpu_has (X86_FEATURE_PTI ))return ;pr_info ("enabled\n" );
Borislav Petkov
maintainerPATCH 5/5 drivers/cpufreq: Convert some slow-path static_cpu_has() callers to boot_cpu_has()
In reply to Borislav PetkovFrom: Borislav Petkov <[email protected]>
Using static_cpu_has() is pointless on those paths, convert them to the boot_cpu_has() variant.
No functional changes.
--- drivers/cpufreq/amd_freq_sensitivity.c | 2 +- drivers/cpufreq/intel_pstate.c | 18 +++++++++--------- drivers/cpufreq/powernow-k8.c | 2 +- 3 files changed, 11 insertions(+), 11 deletions(-)
drivers/cpufreq/amd_freq_sensitivity.c +1 -1
@@ -124,7 +124,7 @@ static int __init amd_freq_sensitivity_init(void) PCI_DEVICE_ID_AMD_KERNCZ_SMBUS, NULL);if (!pcidev) { -if (!static_cpu_has (X86_FEATURE_PROC_FEEDBACK )) +if (!boot_cpu_has (X86_FEATURE_PROC_FEEDBACK ))return -ENODEV ; }
drivers/cpufreq/intel_pstate.c +9 -9
@@ -522,7 +522,7 @@ static s16 intel_pstate_get_epb(struct cpudata *cpu_data)u64 epb;int ret; -if (!static_cpu_has (X86_FEATURE_EPB )) +if (!boot_cpu_has (X86_FEATURE_EPB ))return -ENXIO ; ret =rdmsrl_on_cpu (cpu_data->cpu,MSR_IA32_ENERGY_PERF_BIAS , &epb);
@@ -536,7 +536,7 @@ static s16 intel_pstate_get_epp(struct cpudata *cpu_data, u64 hwp_req_data) {s16 epp; -if (static_cpu_has (X86_FEATURE_HWP_EPP )) { +if (boot_cpu_has (X86_FEATURE_HWP_EPP )) {/* * When hwp_req_data is 0, means that caller didn't read * MSR_HWP_REQUEST, so need to read and get EPP.
@@ -561,7 +561,7 @@ static int intel_pstate_set_epb(int cpu, s16 pref)u64 epb;int ret; -if (!static_cpu_has (X86_FEATURE_EPB )) +if (!boot_cpu_has (X86_FEATURE_EPB ))return -ENXIO ; ret =rdmsrl_on_cpu (cpu,MSR_IA32_ENERGY_PERF_BIAS , &epb);
@@ -609,7 +609,7 @@ static int intel_pstate_get_energy_pref_index(struct cpudata *cpu_data)if (epp <0 )return epp; -if (static_cpu_has (X86_FEATURE_HWP_EPP )) { +if (boot_cpu_has (X86_FEATURE_HWP_EPP )) {if (epp ==HWP_EPP_PERFORMANCE )return 1 ;if (epp <=HWP_EPP_BALANCE_PERFORMANCE )
@@ -618,7 +618,7 @@ static int intel_pstate_get_energy_pref_index(struct cpudata *cpu_data)return 3 ; else return 4; - }else if (static_cpu_has (X86_FEATURE_EPB )) { + }else if (boot_cpu_has (X86_FEATURE_EPB )) {/* * Range: * 0x00-0x03 : Performance
@@ -646,7 +646,7 @@ static int intel_pstate_set_energy_pref_index(struct cpudata *cpu_data,mutex_lock (&intel_pstate_limits_lock); -if (static_cpu_has (X86_FEATURE_HWP_EPP )) { +if (boot_cpu_has (X86_FEATURE_HWP_EPP )) {u64 value; ret =rdmsrl_on_cpu (cpu_data->cpu,MSR_HWP_REQUEST , &value);
@@ -821,7 +821,7 @@ static void intel_pstate_hwp_set(unsigned int cpu) epp = cpu_data->epp_powersave; } update_epp: -if (static_cpu_has (X86_FEATURE_HWP_EPP )) { +if (boot_cpu_has (X86_FEATURE_HWP_EPP )) { value &= ~GENMASK_ULL (31 ,24 ); value |= (u64 )epp <<24 ; }else {
@@ -846,7 +846,7 @@ static void intel_pstate_hwp_force_min_perf(int cpu) value |=HWP_MIN_PERF (min_perf);/* Set EPP/EPB to min */ -if (static_cpu_has (X86_FEATURE_HWP_EPP )) +if (boot_cpu_has (X86_FEATURE_HWP_EPP )) value |=HWP_ENERGY_PERF_PREFERENCE (HWP_EPP_POWERSAVE );else intel_pstate_set_epb (cpu,HWP_EPP_BALANCE_POWERSAVE );
@@ -1194,7 +1194,7 @@ static void __init intel_pstate_sysfs_expose_params(void)static void intel_pstate_hwp_enable (struct cpudata *cpudata) {/* First disable HWP notification interrupt as we don't process them */ -if (static_cpu_has (X86_FEATURE_HWP_NOTIFY )) +if (boot_cpu_has (X86_FEATURE_HWP_NOTIFY ))wrmsrl_on_cpu (cpudata->cpu,MSR_HWP_INTERRUPT ,0x00 );wrmsrl_on_cpu (cpudata->cpu,MSR_PM_ENABLE ,0x1 );
drivers/cpufreq/powernow-k8.c +1 -1
@@ -1178,7 +1178,7 @@ static int powernowk8_init(void)unsigned int i, supported_cpus =0 ;int ret; -if (static_cpu_has (X86_FEATURE_HW_PSTATE )) { +if (boot_cpu_has (X86_FEATURE_HW_PSTATE )) {__request_acpi_cpufreq ();return -ENODEV ; }
Signature · 1 line
2.21.0
Re: [PATCH 5/5] drivers/cpufreq: Convert some slow-path static_cpu_has() callers to boot_cpu_has()
Quoted 6 lines · Borislav Petkov
From: Borislav Petkov <[email protected]>
Using
static_cpu_has()is pointless on those paths, convert them to theboot_cpu_has()variant.
Quoted 8 lines
No functional changes.
Signed-off-by Borislav Petkov <[email protected]>Cc Len Brown <[email protected]>Cc "Rafael J. Wysocki" <[email protected]>Cc Srinivas Pandruvada <[email protected]>Cc Viresh Kumar <[email protected]>
Quoted 120 lines
--- drivers/cpufreq/amd_freq_sensitivity.c | 2 +- drivers/cpufreq/intel_pstate.c | 18 +++++++++--------- drivers/cpufreq/powernow-k8.c | 2 +- 3 files changed, 11 insertions(+), 11 deletions(-)drivers/cpufreq/amd_freq_sensitivity.c +1
-1@@ -124,7 +124,7 @@ static int __init amd_freq_sensitivity_init(void) PCI_DEVICE_ID_AMD_KERNCZ_SMBUS, NULL);if (!pcidev) { -if (!static_cpu_has (X86_FEATURE_PROC_FEEDBACK )) +if (!boot_cpu_has (X86_FEATURE_PROC_FEEDBACK ))return -ENODEV ; }drivers/cpufreq/intel_pstate.c +9
-9@@ -522,7 +522,7 @@ static s16 intel_pstate_get_epb(struct cpudata *cpu_data)u64 epb;int ret; -if (!static_cpu_has (X86_FEATURE_EPB )) +if (!boot_cpu_has (X86_FEATURE_EPB ))return -ENXIO ; ret =rdmsrl_on_cpu (cpu_data->cpu,MSR_IA32_ENERGY_PERF_BIAS , &epb);@@ -536,7 +536,7 @@ static s16 intel_pstate_get_epp(struct cpudata *cpu_data, u64 hwp_req_data) {s16 epp; -if (static_cpu_has (X86_FEATURE_HWP_EPP )) { +if (boot_cpu_has (X86_FEATURE_HWP_EPP )) {/* * When hwp_req_data is 0, means that caller didn't read * MSR_HWP_REQUEST, so need to read and get EPP. @@ -561,7 +561,7 @@ static int intel_pstate_set_epb(int cpu, s16 pref)u64 epb;int ret; -if (!static_cpu_has (X86_FEATURE_EPB )) +if (!boot_cpu_has (X86_FEATURE_EPB ))return -ENXIO ; ret =rdmsrl_on_cpu (cpu,MSR_IA32_ENERGY_PERF_BIAS , &epb);@@ -609,7 +609,7 @@ static int intel_pstate_get_energy_pref_index(struct cpudata *cpu_data)if (epp <0 )return epp; -if (static_cpu_has (X86_FEATURE_HWP_EPP )) { +if (boot_cpu_has (X86_FEATURE_HWP_EPP )) {if (epp ==HWP_EPP_PERFORMANCE )return 1 ;if (epp <=HWP_EPP_BALANCE_PERFORMANCE )@@ -618,7 +618,7 @@ static int intel_pstate_get_energy_pref_index(struct cpudata *cpu_data)return 3 ; else return 4; - }else if (static_cpu_has (X86_FEATURE_EPB )) { + }else if (boot_cpu_has (X86_FEATURE_EPB )) {/* * Range: * 0x00-0x03 : Performance @@ -646,7 +646,7 @@ static int intel_pstate_set_energy_pref_index(struct cpudata *cpu_data,mutex_lock (&intel_pstate_limits_lock); -if (static_cpu_has (X86_FEATURE_HWP_EPP )) { +if (boot_cpu_has (X86_FEATURE_HWP_EPP )) {u64 value; ret =rdmsrl_on_cpu (cpu_data->cpu,MSR_HWP_REQUEST , &value);@@ -821,7 +821,7 @@ static void intel_pstate_hwp_set(unsigned int cpu) epp = cpu_data->epp_powersave; } update_epp: -if (static_cpu_has (X86_FEATURE_HWP_EPP )) { +if (boot_cpu_has (X86_FEATURE_HWP_EPP )) { value &= ~GENMASK_ULL (31 ,24 ); value |= (u64 )epp <<24 ; }else {@@ -846,7 +846,7 @@ static void intel_pstate_hwp_force_min_perf(int cpu) value |=HWP_MIN_PERF (min_perf);/* Set EPP/EPB to min */ -if (static_cpu_has (X86_FEATURE_HWP_EPP )) +if (boot_cpu_has (X86_FEATURE_HWP_EPP )) value |=HWP_ENERGY_PERF_PREFERENCE (HWP_EPP_POWERSAVE );else intel_pstate_set_epb (cpu,HWP_EPP_BALANCE_POWERSAVE );@@ -1194,7 +1194,7 @@ static void __init intel_pstate_sysfs_expose_params(void)static void intel_pstate_hwp_enable (struct cpudata *cpudata) {/* First disable HWP notification interrupt as we don't process them */ -if (static_cpu_has (X86_FEATURE_HWP_NOTIFY )) +if (boot_cpu_has (X86_FEATURE_HWP_NOTIFY ))wrmsrl_on_cpu (cpudata->cpu,MSR_HWP_INTERRUPT ,0x00 );wrmsrl_on_cpu (cpudata->cpu,MSR_PM_ENABLE ,0x1 );drivers/cpufreq/powernow-k8.c +1
-2@@ -1178,7 +1178,7 @@ static int powernowk8_init(void)unsigned int i, supported_cpus =0 ;int ret; -if (static_cpu_has (X86_FEATURE_HW_PSTATE )) { +if (boot_cpu_has (X86_FEATURE_HW_PSTATE )) {__request_acpi_cpufreq ();return -ENODEV ; } --2.21.0
Rafael J. Wysocki
maintainerRe: [PATCH 5/5] drivers/cpufreq: Convert some slow-path static_cpu_has() callers to boot_cpu_has()
Quoted 10 lines · Rafael J. Wysocki
Quoted 8 lines · Borislav Petkov
From: Borislav Petkov <[email protected]>
Using
static_cpu_has()is pointless on those paths, convert them to theboot_cpu_has()variant.No functional changes.
Quoted 8 lines
Quoted 6 lines
Signed-off-by Borislav Petkov <[email protected]>Cc Len Brown <[email protected]>Cc "Rafael J. Wysocki" <[email protected]>Cc Srinivas Pandruvada <[email protected]>Cc Viresh Kumar <[email protected]>
Or I can queue it up if you prefer.
Borislav Petkov
maintainerRe: [PATCH 5/5] drivers/cpufreq: Convert some slow-path static_cpu_has() callers to boot_cpu_has()
Quoted 1 line · Rafael J. Wysocki
Or I can queue it up if you prefer.
Doesn't matter to me - it doesn't have any dependencies to previous patches so however you like.
Thx.
Signature · 4 lines
Regards/Gruss,
Boris.
Good mailing practices for 400: avoid top-posting and trim the reply.Rafael J. Wysocki
maintainerRe: [PATCH 5/5] drivers/cpufreq: Convert some slow-path static_cpu_has() callers to boot_cpu_has()
Quoted 6 lines · Borislav Petkov
Quoted 1 line · Rafael J. Wysocki
Or I can queue it up if you prefer.
Doesn't matter to me - it doesn't have any dependencies to previous patches so however you like.
OK, I'll take it.