x86/fpu/xstate: Add more diagnostic information on inconsistent xstate sizes
RFC PATCH 1/1 x86/fpu/xstate: Add more diagnostic information on inconsistent xstate sizes
A warning is emitted when xstate sizes are found inconsistent. But the warning message doesn't show enough information to diagnose the issue.
Provide more detailed xstate size information to help debug the issue. As a hypothetical example, on a platform that may report incorrect xstate size in CPUID.0xd.1:EBX, the diagnostic information after the warning will show:
[ 0.000000] x86/fpu: max_features=0x407 [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: xstate_offset[10]: 832, xstate_sizes[10]: 8 [ 0.000000] x86/fpu: total size: 840 bytes [ 0.000000] x86/fpu: XCR0=0x7, IA32_XSS=0x400 [ 0.000000] x86/fpu: kernel_size from CPUID.0xd.0x1:EBX: 576 bytes
XCR0 | IA32_XSS is 0x407 which is consistent with max_features. CPUID.0xd.0x1:EBX should report the size of the xsave area containing xstate components corresponding to bits set in XCR0 | IA32_XSS. But it only reports 576 bytes of xsave area which doesn't count xstate sizes for AVX (offset 2 and 256 bytes) and PASID (offset 10 and 8 bytes). This confirms that the platform reports xstate size incorrectly through the CPUID bits.
--- arch/x86/kernel/fpu/xstate.c | 33 +++++++++++++++++++++++++++++++-- 1 file changed, 31 insertions(+), 2 deletions(-)
arch/x86/kernel/fpu/xstate.c +31 -2
@@ -602,8 +602,37 @@ static bool __init paranoid_xstate_size_valid(unsigned int kernel_size) } } size =xstate_calculate_size (fpu_kernel_cfg.max_features, compacted); -XSTATE_WARN_ON (size != kernel_size, -"size %u != kernel_size %u\n" , size, kernel_size); +if (size != kernel_size) { +u64 xcr0, ia32_xss; + +XSTATE_WARN_ON (1 ,"size %u != kernel_size %u\n" , + size, kernel_size); + +/* Show more information to help diagnose the size issue. */ +pr_info ("x86/fpu: max_features=0x%llx\n" , + fpu_kernel_cfg.max_features); +print_xstate_offset_size (); +pr_info ("x86/fpu: total size: %u bytes\n" , size); + xcr0 =xgetbv (XCR_XFEATURE_ENABLED_MASK ); +if (compacted) { +rdmsrl (MSR_IA32_XSS , ia32_xss); +pr_info ("x86/fpu: XCR0=0x%llx, IA32_XSS=0x%llx\n" , + xcr0, ia32_xss); + }else { +pr_info ("x86/fpu: XCR0=0x%llx\n" , xcr0); + } +/* +* In compact case, CPUID.0xd.0x1:EBX reports the size of +* the XSAVE size containing all the state components +* corresponding to bits set in XCR0 | IA32_XSS. +* +* Otherwise, CPUID.0xd.0x0:EBX reports the size of an XSAVE +* area containing all the *user* state components +* corresponding to bits set in XCR0. +*/ +pr_info ("x86/fpu: kernel_size from CPUID.0xd.0x%x:EBX: %u bytes\n" , + compacted ?1 :0 , kernel_size); + }return size == kernel_size; }
Signature · 1 line
2.37.1
Quoted 44 lines · Fenghua Yu
A warning is emitted when xstate sizes are found inconsistent. But the warning message doesn't show enough information to diagnose the issue.
Provide more detailed xstate size information to help debug the issue. As a hypothetical example, on a platform that may report incorrect xstate size in CPUID.0xd.1:EBX, the diagnostic information after the warning will show:
[ 0.000000] x86/fpu: max_features=0x407 [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: xstate_offset[10]: 832, xstate_sizes[10]: 8 [ 0.000000] x86/fpu: total size: 840 bytes [ 0.000000] x86/fpu: XCR0=0x7, IA32_XSS=0x400 [ 0.000000] x86/fpu: kernel_size from CPUID.0xd.0x1:EBX: 576 bytesXCR0 |
IA32_XSSis0x407which is consistent withmax_features. CPUID.0xd.0x1:EBX should report the size of the xsave area containing xstate components corresponding to bits set in XCR0 |IA32_XSS. But it only reports 576 bytes of xsave area which doesn't count xstate sizes for AVX (offset 2 and 256 bytes) and PASID (offset 10 and 8 bytes). This confirms that the platform reports xstate size incorrectly through the CPUID bits.Suggest-by Dave Hansen <[email protected]>Tested-by Chintan M Patel <[email protected]>Signed-off-by Fenghua Yu <[email protected]>--- arch/x86/kernel/fpu/xstate.c | 33 +++++++++++++++++++++++++++++++-- 1 file changed, 31 insertions(+), 2 deletions(-)arch/x86/kernel/fpu/xstate.c +4
-2@@ -602,8 +602,37 @@ static bool __init paranoid_xstate_size_valid(unsigned int kernel_size) } } size =xstate_calculate_size (fpu_kernel_cfg.max_features, compacted); -XSTATE_WARN_ON (size != kernel_size, -"size %u != kernel_size %u\n" , size, kernel_size); +if (size != kernel_size) { +u64 xcr0, ia32_xss; + + XSTATE_WARN_ON(1, "size %u != kernel_size %u\n",
arch/x86/kernel/fpu/xstate.c static bool __init paranoid_xstate_size_valid(unsigned int kernel_size) · line 618
Quoted 10 lines
arch/x86/kernel/fpu/xstate.c +10
-0+ size, kernel_size); + +/* Show more information to help diagnose the size issue. */ +pr_info ("x86/fpu: max_features=0x%llx\n" , + fpu_kernel_cfg.max_features); +print_xstate_offset_size (); +pr_info ("x86/fpu: total size: %u bytes\n" , size); + xcr0 =xgetbv (XCR_XFEATURE_ENABLED_MASK ); +if (compacted) { +rdmsrl (MSR_IA32_XSS , ia32_xss);
This shouldn't be directly read here because of the LBR state component.
See the function comment:
* Independent XSAVE features allocate their own buffers and are not
* covered by these checks. Only the size of the buffer for task->fpu
* is checked here.
But, isn't that max_features bitmask pretty much about it?
Quoted 6 lines
arch/x86/kernel/fpu/xstate.c +6
-0+pr_info ("x86/fpu: XCR0=0x%llx, IA32_XSS=0x%llx\n" , + xcr0, ia32_xss); + }else { +pr_info ("x86/fpu: XCR0=0x%llx\n" , xcr0); + } +/*
arch/x86/kernel/fpu/xstate.c static bool __init paranoid_xstate_size_valid(unsigned int kernel_size) · line 634
Quoted 10 lines
arch/x86/kernel/fpu/xstate.c +10
-0+* In compact case, CPUID.0xd.0x1:EBX reports the size of +* the XSAVE size containing all the state components +* corresponding to bits set in XCR0 | IA32_XSS. +* +* Otherwise, CPUID.0xd.0x0:EBX reports the size of an XSAVE +* area containing all the *user* state components +* corresponding to bits set in XCR0. +*/ +pr_info ("x86/fpu: kernel_size from CPUID.0xd.0x%x:EBX: %u bytes\n" , + compacted ?1 :0 , kernel_size);
Include Thiago who asked this.
Thanks,
Chang
Hi, Chang,
Quoted 59 lines · Chang S. Bae
Quoted 45 lines · Fenghua Yu
A warning is emitted when xstate sizes are found inconsistent. But the warning message doesn't show enough information to diagnose the issue.
Provide more detailed xstate size information to help debug the issue. As a hypothetical example, on a platform that may report incorrect xstate size in CPUID.0xd.1:EBX, the diagnostic information after the warning will show:
[ 0.000000] x86/fpu: max_features=0x407
[ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: xstate_offset[10]: 832, xstate_sizes[10]: 8 [ 0.000000] x86/fpu: total size: 840 bytes
[ 0.000000] x86/fpu: XCR0=0x7, IA32_XSS=0x400
[ 0.000000] x86/fpu:kernel_sizefrom CPUID.0xd.0x1:EBX: 576 bytesXCR0 |
IA32_XSSis0x407which is consistent withmax_features. CPUID.0xd.0x1:EBX should report the size of the xsave area containing xstate components corresponding to bits set in XCR0 |IA32_XSS. But it only reports 576 bytes of xsave area which doesn't count xstate sizes for AVX (offset 2 and 256 bytes) and PASID (offset 10 and 8 bytes). This confirms that the platform reports xstate size incorrectly through the CPUID bits.Suggest-by Dave Hansen <[email protected]>Tested-by Chintan M Patel <[email protected]>Signed-off-by Fenghua Yu <[email protected]>---
arch/x86/kernel/fpu/xstate.c| 33 +++++++++++++++++++++++++++++++--1 file changed, 31 insertions(+), 2 deletions(-)arch/x86/kernel/fpu/xstate.c +4
-2@@ -602,8 +602,37 @@ static bool __init paranoid_xstate_size_valid(unsigned int kernel_size) } } size = xstate_calculate_size(fpu_kernel_cfg.max_features, compacted); - XSTATE_WARN_ON(size != kernel_size, - "size %u != kernel_size %u\n", size, kernel_size); + if (size != kernel_size) { + u64 xcr0, ia32_xss; + + XSTATE_WARN_ON(1, "size %u != kernel_size %u\n",arch/x86/kernel/fpu/xstate.c static bool __init · line 619
Quoted 10 lines
arch/x86/kernel/fpu/xstate.c +10
-0+ size, kernel_size); + + /* Show more information to help diagnose the size issue. */ + pr_info("x86/fpu: max_features=0x%llx\n", + fpu_kernel_cfg.max_features); + print_xstate_offset_size(); + pr_info("x86/fpu: total size: %u bytes\n", size); + xcr0 = xgetbv(XCR_XFEATURE_ENABLED_MASK); + if (compacted) { + rdmsrl(MSR_IA32_XSS, ia32_xss);This shouldn't be directly read here because of the LBR state component.
Quoted 7 lines
See the function comment:
* Independent XSAVE features allocate their own buffers and are not
* covered by these checks. Only the size of the buffer fortask->fpu
* is checked here.But, isn't that
max_featuresbitmask pretty much about it?
How about getting IA32_XSS from xfeatures_mask_supervisor()? That's how to get kernel_size by setting IA32_XSS without independent features in get_xsave_compacted_size().
Thanks.
-Fenghua
Quoted 33 lines · Fenghua Yu
Quoted 32 lines · Chang S. Bae
Quoted 16 lines · Fenghua Yu
arch/x86/kernel/fpu/xstate.c +4
-2@@ -602,8 +602,37 @@ static bool __init paranoid_xstate_size_valid(unsigned int kernel_size) } } size = xstate_calculate_size(fpu_kernel_cfg.max_features, compacted); - XSTATE_WARN_ON(size != kernel_size, - "size %u != kernel_size %u\n", size, kernel_size); + if (size != kernel_size) { + u64 xcr0, ia32_xss; + + XSTATE_WARN_ON(1, "size %u != kernel_size %u\n",arch/x86/kernel/fpu/xstate.c static bool __init · line 619
Quoted 10 lines
arch/x86/kernel/fpu/xstate.c +10
-0+ size, kernel_size); + + /* Show more information to help diagnose the size issue. */ + pr_info("x86/fpu: max_features=0x%llx\n", + fpu_kernel_cfg.max_features); + print_xstate_offset_size(); + pr_info("x86/fpu: total size: %u bytes\n", size); + xcr0 = xgetbv(XCR_XFEATURE_ENABLED_MASK); + if (compacted) { + rdmsrl(MSR_IA32_XSS, ia32_xss);This shouldn't be directly read here because of the LBR state component.
See the function comment:
Quoted 7 lines
Quoted 5 lines
* Independent XSAVE features allocate their own buffers and are not
* covered by these checks. Only the size of the buffer fortask->fpu
* is checked here.But, isn't that
max_featuresbitmask pretty much about it?How about getting
IA32_XSSfromxfeatures_mask_supervisor()? That's how to getkernel_sizeby settingIA32_XSSwithout independent features inget_xsave_compacted_size()
I think what it tests here is comparing the sizes between the kernel code and microcode calculations on the same input, which is the max_features bitmask.
We know that the kernel code calculates the size based on it and also takes it to write down there -- XCR0 and IA32_XSS. Then, showing that bitmask looks to be enough I thought, no?
I still expect some acknowledgment of what is coded here for the kernel calculation details.
Thanks,
Chang
Hi, Chang,
Quoted 38 lines · Chang S. Bae
Quoted 37 lines · Fenghua Yu
Quoted 36 lines · Chang S. Bae
Quoted 16 lines · Fenghua Yu
arch/x86/kernel/fpu/xstate.c +4
-2@@ -602,8 +602,37 @@ static bool __init paranoid_xstate_size_valid(unsigned int kernel_size) } } size = xstate_calculate_size(fpu_kernel_cfg.max_features, compacted); - XSTATE_WARN_ON(size != kernel_size, - "size %u != kernel_size %u\n", size, kernel_size); + if (size != kernel_size) { + u64 xcr0, ia32_xss; + + XSTATE_WARN_ON(1, "size %u != kernel_size %u\n",arch/x86/kernel/fpu/xstate.c static bool __init · line 619
Quoted 10 lines
arch/x86/kernel/fpu/xstate.c +10
-0+ size, kernel_size); + + /* Show more information to help diagnose the size issue. */ + pr_info("x86/fpu: max_features=0x%llx\n", + fpu_kernel_cfg.max_features); + print_xstate_offset_size(); + pr_info("x86/fpu: total size: %u bytes\n", size); + xcr0 = xgetbv(XCR_XFEATURE_ENABLED_MASK); + if (compacted) { + rdmsrl(MSR_IA32_XSS, ia32_xss);This shouldn't be directly read here because of the LBR state component.
See the function comment:
* Independent XSAVE features allocate their own buffers and are not
* covered by these checks. Only the size of the buffer fortask->fpu
* is checked here.
Quoted 6 lines
Quoted 3 lines
Quoted 1 line
But, isn't that
max_featuresbitmask pretty much about it?How about getting
IA32_XSSfromxfeatures_mask_supervisor()? That's how to getkernel_sizeby settingIA32_XSSwithout independent features inget_xsave_compacted_size()I think what it tests here is comparing the sizes between the kernel code and microcode calculations on the same input, which is the
max_featuresbitmask.We know that the kernel code calculates the size based on it and also takes it to write down there -- XCR0 and
IA32_XSS. Then, showing that bitmask looks to be enough I thought, no?
First of all, max_features is shown already.
Kernel_size from CPUID.0xd.0x1:EBX takes XCR0 | IA32_XSS as input. Platform may take wrong XCR0 or IA32_XSS and get wrong kernel_size. The purpose of this patch is to provide more debug info to help debug platform/kernel issue. So instead of a whole max_features, xgetbv() to get XCR0 and xfeatures_mask_supervisor() to get IA32_XSS provides more debug info in case platform may have issue in XCR0 or IA32_XSS.
In other words, splitting max_features into XCR0 and IA32_XSS and showing them individually provide more useful debug info than one single max_features value.
Does it make sense?
Quoted 2 lines
I still expect some acknowledgment of what is coded here for the kernel calculation details.
The kernel calculation is shown in
+ print_xstate_offset_size();
+ pr_info("x86/fpu: total size: %u bytes\n", size);
Isn't that detailed enough to show offset and size of each xstate and sum of sizes?
After that,
+ pr_info("x86/fpu: kernel_size from CPUID.0xd.0x%x:EBX: %u bytes\n",
+ compacted ? 1 : 0, kernel_size);
shows how kernel_size is calculated from CPUID?
Using the above debug info, a real platform CPUID issue is shown clearly.
What other details are needed?
Thanks.
-Fenghua
Quoted 2 lines · Fenghua Yu
First of all,
max_featuresis shown already.
Yes.
Quoted 5 lines
Kernel_sizefrom CPUID.0xd.0x1:EBX takes XCR0 |IA32_XSSas input. Platform may take wrong XCR0 orIA32_XSSand get wrongkernel_size. The > purpose of this patch is to provide more debug info to help debug platform/kernel issue. So instead of a wholemax_features,xgetbv()to get XCR0 andxfeatures_mask_supervisor()to getIA32_XSSprovides more debug info in case platform may have issue in XCR0 orIA32_XSS.In other words, splitting
max_featuresinto XCR0 andIA32_XSSand showing them individually provide more useful debug info than one singlemax_featuresvalue. > Does it make sense?
Hmm, I don't get it. I don't think whether the microcode takes those register values wrong or miscalculates the size does matter here.
print_xstate_offset_size() or something can decode the mask and readily shows off how it was calculated here. Then, probably that's it.
Quoted 8 lines
Quoted 1 line
I still expect some acknowledgment of what is coded here for the kernel calculation details.
The kernel calculation is shown in
+print_xstate_offset_size();
+pr_info("x86/fpu: total size: %u bytes\n", size);Isn't that detailed enough to show offset and size of each xstate and sum of sizes?
Quoted 8 lines
After that,
+ pr_info("x86/fpu:kernel_sizefrom CPUID.0xd.0x%x:EBX: %u bytes\n", + compacted ? 1 : 0,kernel_size);
shows howkernel_sizeis calculated from CPUID?Using the above debug info, a real platform CPUID issue is shown clearly.
What other details are needed?
I recall it was also asked to show which features are off or mismatched as compared to the CPU calculation. I'm not so sure about it.
Thanks,
Chang
Quoted 5 lines · Fenghua Yu
In other words, splitting
max_featuresinto XCR0 andIA32_XSSand showing them individually provide more useful debug info than one singlemax_featuresvalue.Does it make sense?
Not to me.
Quoted 10 lines
Quoted 2 lines
I still expect some acknowledgment of what is coded here for the kernel calculation details.
The kernel calculation is shown in
+print_xstate_offset_size();
+pr_info("x86/fpu: total size: %u bytes\n", size);Isn't that detailed enough to show offset and size of each xstate and sum of sizes?
Quoted 8 lines
After that,
+ pr_info("x86/fpu:kernel_sizefrom CPUID.0xd.0x%x:EBX: %u bytes\n", + compacted ? 1 : 0,kernel_size);
shows howkernel_sizeis calculated from CPUID?Using the above debug info, a real platform CPUID issue is shown clearly.
What other details are needed?
I was kinda hoping this would be a simple, non-controversial patch that would get us better debugging info the next time that the microcode or a bad VMM screws up. This patch isn't turning out to be as simple as I hoped.
I was wrong. Let's just drop this.