x86/fpu: Fix MXCSR handling and SSE component definition
Subsystems (5)
cover x86/fpu: Fix MXCSR handling and SSE component definition
Hi all,
Andrei's report [1] triggered reviewing the MXCSR-related code. My responded change [2] has been revised along with some additional fixes in this series.
While Dave Hansen alluded another version [3] for the fix, as I couldn't locate it so far, sending patches as one of proposals in this regard.
== Regression ==
Recently the XSTATE copy functions were unitized together [4]. At a glance, this change appears to relapse the ptrace write on the MXCSR state when the non-compacted format is used in the kernel.
But, this regression appears to root in the XSAVES-enabling code [5] that introduced the XSTATE conversion along with the MXCSR mistreatment.
== MXCSR Hindsight ==
MXCSR is architecturally part of the SSE component. The MXCSR association of XSTATE_BV depends on the XSAVE format.
The change [5], however, presumed MXCSR as part of the X87 component and made the kernel referencing XSTATE_BV regardless of the format.
== Patches ==
* Fix the MXCSR conversion code along with adding the test case.
* Then, fixing MXCSR, one of the other call sites is also updated to exclude legacy states.
* The hard-coded legacy state offset and size are adjusted in the end.
These patches can be also found in this repository:
git://github.com/intel/amx-linux.git mxcsr
Thanks,
Chang
[1] https://lore.kernel.org/lkml/CANaxB-wkcNKWjyNGFuMn6f6H2DQSGwwQjUgg1eATdUgmM-Kg+A@mail.gmail.com/
[2] https://lore.kernel.org/lkml/[email protected]/
[3] https://lore.kernel.org/lkml/[email protected]/
[4] Commit 43be46e89698 ("x86/fpu: Sanitize xstateregs_set()")
[5] Commit 91c3dba7dbc1 ("x86/fpu/xstate: Fix PTRACE frames for XSAVES")
Chang S. Bae (4):
x86/fpu: Fix the MXCSR state reshuffling between userspace and kernel
buffers
selftests/x86/mxcsr: Test the MXCSR state write via ptrace x86/fpu: Clarify the XSTATE clearing only for extended components x86/fpu: Correct the legacy state offset and size information
arch/x86/include/asm/fpu/api.h | 2 +- arch/x86/kernel/fpu/xstate.c | 98 +++++++++---- arch/x86/kvm/x86.c | 4 +- tools/testing/selftests/x86/Makefile | 2 +- tools/testing/selftests/x86/mxcsr.c | 200 +++++++++++++++++++++++++++ 5 files changed, 273 insertions(+), 33 deletions(-) create mode 100644 tools/testing/selftests/x86/mxcsr.c
base-commit: 82eedfedea446ae448f0975f1db7d1631cd24330
Signature · 1 line
2.17.1
PATCH 1/4 x86/fpu: Fix the MXCSR state reshuffling between userspace and kernel buffers
== Hardware Background ==
The MXCSR state, as part of the SSE component, has different treatments with XSAVE*/XRSTOR* between the two XSAVE formats:
- When the MXCSR state is XSAVEd in the non-compacted format, the feature bit in XSTATE_BV pertains to the XMM registers. XRSTOR restores the MXCSR state without referencing XSTATE_BV.
- But, on XSAVE* with the compacted format, the SSE bit is set in XSTATE_BV if MXCSR is not in the init state on XSAVE*. Then, on XRSTOR*, the state is restored only when the SSE bit is set in XSTATE_BV.
== Regression ==
The XSTATE copy routine between userspace and kernel buffers used to be separate for different XSAVE formats. Commit 43be46e89698 ("x86/fpu:
Sanitize xstateregs_set()") combined them together.
This change appears to be a regression on XSAVES-less systems. But, the merged code is based on the original conversion code with commit 91c3dba7dbc1 ("x86/fpu/xstate: Fix PTRACE frames for XSAVES").
That has such oversight as:
- Mistreating MXCSR as part of the FP state instead of the SSE component.
- Taking the SSE bit in XSTATE_BV always for all the SSE states.
== Correction ==
Update the XSTATE conversion code:
- Refactor the copy routine for legacy states. Treat MXCSR as part of SSE.
- Copying MXCSR, reference XSTATE_BV only with the compacted format.
- Also, flip the SSE bit in XSTATE_BV accordingly to the format.
--- arch/x86/kernel/fpu/xstate.c | 70 +++++++++++++++++++++++++----------- 1 file changed, 49 insertions(+), 21 deletions(-)
arch/x86/kernel/fpu/xstate.c +49 -21
@@ -1064,6 +1064,7 @@ void __copy_xstate_to_uabi_buf(struct membuf to, struct fpstate *fpstate, u32 pkru_val, enum xstate_copy_mode copy_mode) {const unsigned int off_mxcsr = offsetof(struct fxregs_state , mxcsr); +bool compacted =cpu_feature_enabled (X86_FEATURE_XCOMPACTED );struct xregs_state *xinit = &init_fpstate.regs.xsave;struct xregs_state *xsave = &fpstate->regs.xsave;struct xstate_header header;
@@ -1093,8 +1094,13 @@ void __copy_xstate_to_uabi_buf(struct membuf to, struct fpstate *fpstate,copy_feature (header.xfeatures &XFEATURE_MASK_FP , &to, &xsave->i387, &xinit->i387, off_mxcsr); -/* Copy MXCSR when SSE or YMM are set in the feature mask */ -copy_feature (header.xfeatures & (XFEATURE_MASK_SSE |XFEATURE_MASK_YMM ), +/* +* Copy MXCSR depending on the XSAVE format. If compacted, +* reference the feature mask. Otherwise, check if any of related +* features is valid. +*/ +copy_feature (compacted ? header.xfeatures &XFEATURE_MASK_SSE : + fpstate->user_xfeatures & (XFEATURE_MASK_SSE |XFEATURE_MASK_YMM ), &to, &xsave->i387.mxcsr, &xinit->i387.mxcsr,MXCSR_AND_FLAGS_SIZE );
@@ -1199,6 +1205,11 @@ static int copy_from_buffer(void *dst, unsigned int offset, unsigned int size,static int copy_uabi_to_xstate (struct fpstate *fpstate,const void *kbuf,const void __user *ubuf) { +const unsigned int off_stspace = offsetof(struct fxregs_state , st_space); +const unsigned int off_xmm = offsetof(struct fxregs_state , xmm_space); +const unsigned int off_mxcsr = offsetof(struct fxregs_state , mxcsr); +bool compacted =cpu_feature_enabled (X86_FEATURE_XCOMPACTED ); +struct fxregs_state *fxsave = &fpstate->regs.fxsave;struct xregs_state *xsave = &fpstate->regs.xsave;unsigned int offset, size;struct xstate_header hdr;
@@ -1212,38 +1223,48 @@ static int copy_uabi_to_xstate(struct fpstate *fpstate, const void *kbuf,if (validate_user_xstate_header (&hdr, fpstate))return -EINVAL ; -/* Validate MXCSR when any of the related features is in use */ - mask =XFEATURE_MASK_FP |XFEATURE_MASK_SSE |XFEATURE_MASK_YMM ; -if (hdr.xfeatures & mask) { +if (hdr.xfeatures &XFEATURE_MASK_FP ) { +if (copy_from_buffer (fxsave,0 , off_mxcsr, kbuf, ubuf)) +return -EINVAL ; +if (copy_from_buffer (fxsave->st_space, off_stspace,sizeof (fxsave->st_space), + kbuf, ubuf)) +return -EINVAL ; + } + +/* Validate MXCSR when any of the related features is valid. */ + mask =XFEATURE_MASK_SSE |XFEATURE_MASK_YMM ; +if (fpstate->user_xfeatures & mask) {u32 mxcsr[2 ]; - offset = offsetof(struct fxregs_state , mxcsr); -if (copy_from_buffer (mxcsr, offset,sizeof (mxcsr), kbuf, ubuf)) +if (copy_from_buffer (mxcsr, off_mxcsr,sizeof (mxcsr), kbuf, ubuf))return -EFAULT ;/* Reserved bits in MXCSR must be zero. */ if (mxcsr[0 ] & ~mxcsr_feature_mask)return -EINVAL ; -/* SSE and YMM require MXCSR even when FP is not in use. */ -if (!(hdr.xfeatures &XFEATURE_MASK_FP )) { - xsave->i387.mxcsr = mxcsr[0 ]; - xsave->i387.mxcsr_mask = mxcsr[1 ]; - } +/* +* Copy MXCSR regardless of the feature mask as userspace +* uses the uncompacted format. +*/ + fxsave->mxcsr = mxcsr[0 ]; + fxsave->mxcsr_mask = mxcsr[1 ]; } -for (i =0 ; i <XFEATURE_MAX ; i++) { - mask =BIT_ULL (i); +if (hdr.xfeatures &XFEATURE_MASK_SSE ) { +if (copy_from_buffer (fxsave->xmm_space, off_xmm,sizeof (fxsave->xmm_space), + kbuf, ubuf)) +return -EINVAL ; + } -if (hdr.xfeatures & mask) { -void *dst =__raw_xsave_addr (xsave, i); +for_each_extended_xfeature (i, hdr.xfeatures) { +void *dst =__raw_xsave_addr (xsave, i); - offset = xstate_offsets[i]; - size = xstate_sizes[i]; + offset = xstate_offsets[i]; + size = xstate_sizes[i]; -if (copy_from_buffer (dst, offset, size, kbuf, ubuf)) -return -EFAULT ; - } +if (copy_from_buffer (dst, offset, size, kbuf, ubuf)) +return -EFAULT ; }/*
@@ -1256,6 +1277,13 @@ static int copy_uabi_to_xstate(struct fpstate *fpstate, const void *kbuf,* Add back in the features that came in from userspace: */ xsave->header.xfeatures |= hdr.xfeatures; +/* +* Convert the SSE bit in the feature mask as it implies +* differently between the formats. It indicates the MXCSR state +* if compacted; otherwise, it pertains to XMM registers. +*/ +if (compacted && fxsave->mxcsr !=MXCSR_DEFAULT ) + xsave->header.xfeatures |=XFEATURE_MASK_SSE ;return 0 ; }
Signature · 1 line
2.17.1
PATCH 2/4 selftests/x86/mxcsr: Test the MXCSR state write via ptrace
In reply to Chang S. BaeThe ptrace buffer is in the non-compacted format. The MXCSR state should be written to the target thread when either SSE or AVX component is enabled.
Write an MXCSR value to the target and read back. Then it is validated with the XRSTOR/XSAVE result on the current.
---
If this is acceptable, I will also follow up to move some of the helper functions to a .h file from this and other test cases because duplicating what is shareable should be avoided.
--- tools/testing/selftests/x86/Makefile | 2 +- tools/testing/selftests/x86/mxcsr.c | 200 +++++++++++++++++++++++++++ 2 files changed, 201 insertions(+), 1 deletion(-) create mode 100644 tools/testing/selftests/x86/mxcsr.c
tools/testing/selftests/x86/Makefile +1 -1
@@ -13,7 +13,7 @@ CAN_BUILD_WITH_NOPIE := $(shell ./check_cc.sh "$(CC)" trivial_program.c -no-pie) TARGETS_C_BOTHBITS := single_step_syscall sysret_ss_attrs syscall_nt test_mremap_vdso \ check_initial_reg_state sigreturn iopl ioperm \ test_vsyscall mov_ss_trap \ - syscall_arg_fault fsgsbase_restore sigaltstack + syscall_arg_fault fsgsbase_restore sigaltstack mxcsr TARGETS_C_32BIT_ONLY := entry_from_vm86 test_syscall_vdso unwind_vdso \ test_FCMOV test_FCOMI test_FISTTP \ vdso_restorer
tools/testing/selftests/x86/mxcsr.c new file +200 -0
200 lines not on this page · raw message
Signature · 1 line
2.17.1
PATCH 3/4 x86/fpu: Clarify the XSTATE clearing only for extended components
In reply to Chang S. BaeCommit 087df48c298c ("x86/fpu: Replace KVMs xstate component clearing") refactored the MPX state clearing code.
But, legacy states are not warranted in this routine. Rename the function to clarify that only extended components are acceptable.
--- arch/x86/include/asm/fpu/api.h | 2 +- arch/x86/kernel/fpu/xstate.c | 7 +++++-- arch/x86/kvm/x86.c | 4 ++-- 3 files changed, 8 insertions(+), 5 deletions(-)
arch/x86/include/asm/fpu/api.h +1 -1
@@ -130,7 +130,7 @@ static inline void fpstate_free(struct fpu *fpu) { }#endif /* fpstate-related functions which are exported to KVM */ -extern void fpstate_clear_xstate_component (struct fpstate *fps,unsigned int xfeature); +extern void fpstate_clear_extended_xstate (struct fpstate *fps,unsigned int xfeature);extern u64 xstate_get_guest_group_perm (void );
arch/x86/kernel/fpu/xstate.c +5 -2
@@ -1371,14 +1371,17 @@ void xrstors(struct xregs_state *xstate, u64 mask) }#if IS_ENABLED (CONFIG_KVM ) -void fpstate_clear_xstate_component (struct fpstate *fps,unsigned int xfeature) +void fpstate_clear_extended_xstate (struct fpstate *fps,unsigned int xfeature) {void *addr =get_xsave_addr (&fps->regs.xsave, xfeature); +if (xfeature <=XFEATURE_SSE ) +return ; +if (addr)memset (addr,0 , xstate_sizes[xfeature]); } -EXPORT_SYMBOL_GPL (fpstate_clear_xstate_component); +EXPORT_SYMBOL_GPL (fpstate_clear_extended_xstate);#endif #ifdef CONFIG_X86_64
arch/x86/kvm/x86.c +2 -2
@@ -11760,8 +11760,8 @@ void kvm_vcpu_reset(struct kvm_vcpu *vcpu, bool init_event)if (init_event)kvm_put_guest_fpu (vcpu); -fpstate_clear_xstate_component (fpstate,XFEATURE_BNDREGS ); -fpstate_clear_xstate_component (fpstate,XFEATURE_BNDCSR ); +fpstate_clear_extended_xstate (fpstate,XFEATURE_BNDREGS ); +fpstate_clear_extended_xstate (fpstate,XFEATURE_BNDCSR );if (init_event)kvm_load_guest_fpu (vcpu);
Signature · 1 line
2.17.1
Sean Christopherson
maintainerRe: [PATCH 3/4] x86/fpu: Clarify the XSTATE clearing only for extended components
Quoted 4 lines · Chang S. Bae
Commit
087df48c298c("x86/fpu: Replace KVMs xstate component clearing") refactored the MPX state clearing code.But, legacy states are not warranted in this routine.
Why not? I could probably figure it out eventually, but that info should be provided in the changelog.
Quoted 1 line
Rename the function to clarify that only extended components are acceptable.
The function rename isn't the interesting part of the patch, explicitly disallowing "legacy" states is what's interesting. The shortlog+changelog should reflect that.
Quoted 28 lines
Signed-off-by Chang S. Bae <[email protected]>--- arch/x86/include/asm/fpu/api.h | 2 +- arch/x86/kernel/fpu/xstate.c | 7 +++++-- arch/x86/kvm/x86.c | 4 ++-- 3 files changed, 8 insertions(+), 5 deletions(-)arch/x86/include/asm/fpu/api.h +1
-1@@ -130,7 +130,7 @@ static inline void fpstate_free(struct fpu *fpu) { }#endif /* fpstate-related functions which are exported to KVM */ -extern void fpstate_clear_xstate_component (struct fpstate *fps,unsigned int xfeature); +extern void fpstate_clear_extended_xstate (struct fpstate *fps,unsigned int xfeature);extern u64 xstate_get_guest_group_perm (void );arch/x86/kernel/fpu/xstate.c +0
-0
arch/x86/kernel/fpu/xstate.c void xrstors(struct xregs_state *xstate, u64 mask) · line 1378
Quoted 9 lines
arch/x86/kernel/fpu/xstate.c +2
-1@@ -1371,14 +1371,17 @@ void xrstors(struct xregs_state *xstate, u64 mask) }#if IS_ENABLED (CONFIG_KVM ) -void fpstate_clear_xstate_component (struct fpstate *fps,unsigned int xfeature) +void fpstate_clear_extended_xstate (struct fpstate *fps,unsigned int xfeature) {void *addr =get_xsave_addr (&fps->regs.xsave, xfeature); +if (xfeature <=XFEATURE_SSE )
This should WARN_ON_ONCE(), silently doing nothing in the presence of buggy code isn't much better than clobbering state.
Quoted 15 lines
arch/x86/kernel/fpu/xstate.c +3
-1+return ; +if (addr) memset(addr, 0, xstate_sizes[xfeature]); } -EXPORT_SYMBOL_GPL (fpstate_clear_xstate_component); +EXPORT_SYMBOL_GPL (fpstate_clear_extended_xstate);#endif #ifdef CONFIG_X86_64 arch/x86/kvm/x86.c +0
-0
arch/x86/kvm/x86.c void kvm_vcpu_reset(struct kvm_vcpu *vcpu, bool init_event) · line 11764
Quoted 7 lines
arch/x86/kvm/x86.c +2
-2@@ -11760,8 +11760,8 @@ void kvm_vcpu_reset(struct kvm_vcpu *vcpu, bool init_event)if (init_event)kvm_put_guest_fpu (vcpu); -fpstate_clear_xstate_component (fpstate,XFEATURE_BNDREGS ); -fpstate_clear_xstate_component (fpstate,XFEATURE_BNDCSR ); +fpstate_clear_extended_xstate (fpstate,XFEATURE_BNDREGS ); +fpstate_clear_extended_xstate (fpstate,XFEATURE_BNDCSR );
From a KVM perspective, I strongly prefer the existing name. The "component" part makes it very clear that the helper clears a single component, whereas it's not obvious at first glances that the version without "component" only affects the specified feature.
The obvious alternative is something like fpstate_clear_extended_xstate_component(), but I don't really see the point. "xstate" is "extended state" after all, which is partly why I find fpstate_clear_extended_xstate() confusing; it makes me think the helper is for some fancy "extended extended state".
PATCH 4/4 x86/fpu: Correct the legacy state offset and size information
In reply to Chang S. BaeMXCSR is architecturally part of the SSE state. But, the kernel code presumes it as part of the FP component. Adjust the offset and size for these legacy states.
Notably, each legacy component area is not contiguous, unlike extended components. Add a warning message when these offset and size are referenced.
--- arch/x86/kernel/fpu/xstate.c | 21 +++++++++++++++------ 1 file changed, 15 insertions(+), 6 deletions(-)
arch/x86/kernel/fpu/xstate.c +15 -6
@@ -143,8 +143,13 @@ static unsigned int xfeature_get_offset(u64 xcomp_bv, int xfeature)* offsets. */ if (!cpu_feature_enabled (X86_FEATURE_XCOMPACTED ) || - xfeature <=XFEATURE_SSE ) + xfeature <=XFEATURE_SSE ) { +if (xfeature <=XFEATURE_SSE ) +pr_warn ("The legacy state (%d) is discontiguously located.\n" , + xfeature); +return xstate_offsets[xfeature]; + }/* * Compacted format offsets depend on the actual content of the
@@ -217,14 +222,18 @@ static void __init setup_xstate_cache(void)* The FP xstates and SSE xstates are legacy states. They are always * in the fixed offsets in the xsave area in either compacted form * or standard form. +* +* But, while MXCSR is part of the SSE state, it is located in +* between the FP states. Note that it is erroneous assuming that +* each legacy area is contiguous. */ xstate_offsets[XFEATURE_FP ] =0 ; - xstate_sizes[XFEATURE_FP ] = offsetof(struct fxregs_state , - xmm_space); + xstate_sizes[XFEATURE_FP ] = offsetof(struct fxregs_state , mxcsr) + + sizeof_field(struct fxregs_state, st_space); - xstate_offsets[XFEATURE_SSE ] = xstate_sizes[XFEATURE_FP ]; - xstate_sizes[XFEATURE_SSE] = sizeof_field(struct fxregs_state, - xmm_space); + xstate_offsets[XFEATURE_SSE ] = offsetof(struct fxregs_state , mxcsr); + xstate_sizes[XFEATURE_SSE ] =MXCSR_AND_FLAGS_SIZE + + sizeof_field(struct fxregs_state, xmm_space);for_each_extended_xfeature (i, fpu_kernel_cfg.max_features) {cpuid_count (XSTATE_CPUID , i, &eax, &ebx, &ecx, &edx);
Signature · 1 line
2.17.1