Skip to contentSkip to conversation

xstate: make use_xsave non-init

PATCH Superseded Roger Pau Monne 2 participants 6 messages Jul 1, 2019

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 xstate: make use_xsave non-init

LLVM code generation can attempt to load from a variable in the next condition of an expression under certain circumstances, thus attempting to load use_xsave regardless of the value of the bsp variable, which leads to a page fault when the init section has already been unmapped.

Fix this by making use_xsave non-init, thus preventing the page fault. The LLVM bug with the discussion about this issue can be found at:

https://bugs.llvm.org/show_bug.cgi?id=39707

---
Cc: Jan Beulich <[email protected]>
Cc: Andrew Cooper <[email protected]>
Cc: Wei Liu <[email protected]>

---
 xen/arch/x86/xstate.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
xen/arch/x86/xstate.c +1 -1
index 3293ef834f..6a7353109b 100644
@@ -577,7 +577,7 @@ unsigned int xstate_ctxt_size(u64 xcr0)
 /* Collect the information of processor's extended state */
 void xstate_init(struct cpuinfo_x86 *c)
 {
-    static bool __initdata use_xsave = true;
+    static bool use_xsave = true;
     boolean_param("xsave", use_xsave);
 
     bool bsp = c == &boot_cpu_data;
Signature · 7 lines
2.20.1 (Apple Git-117)


_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
#1 lore raw

xen/arch/x86/xstate.c unsigned int xstate_ctxt_size(u64 xcr0) · line 580

Quoted 8 lines · Roger Pau Monne
xen/arch/x86/xstate.c +1 -1
@@ -577,7 +577,7 @@ unsigned int xstate_ctxt_size(u64 xcr0)
  /* Collect the information of processor's extended state */
  void xstate_init(struct cpuinfo_x86 *c)
  {
-    static bool __initdata use_xsave = true;
+    static bool use_xsave = true;

Please attach at least a brief comment here, such that people won't consider the __initdata missing.

Of course I'd actually prefer the annotation to stay here in the gcc case. Iirc there was one other case where there was such an issue; I don't recall whether there it too got dealt with by removing an annotation. How about we introduce an annotation that expands to nothing in the clang case, but continues to provide the same functionality for gcc? That would then also clarify the reason for it being in any particular place (I guess there are going to be more) without the need for further commentary.

Jan

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

#2 lore raw
Quoted 12 lines · Jan Beulich

xen/arch/x86/xstate.c unsigned int xstate_ctxt_size(u64 xcr0) · line 580

Quoted 8 lines · Roger Pau Monne
xen/arch/x86/xstate.c +1 -1
@@ -577,7 +577,7 @@ unsigned int xstate_ctxt_size(u64 xcr0)
  /* Collect the information of processor's extended state */
  void xstate_init(struct cpuinfo_x86 *c)
  {
-    static bool __initdata use_xsave = true;
+    static bool use_xsave = true;

Please attach at least a brief comment here, such that people won't consider the __initdata missing.

Sure.

Quoted 5 lines

Of course I'd actually prefer the annotation to stay here in the gcc case. Iirc there was one other case where there was such an issue; I don't recall whether there it too got dealt with by removing an annotation.

Yes, in that other case the annotation was just removed, it's 43fa95ae [0]

Quoted 6 lines

How about we introduce an
annotation that expands to nothing in the clang case, but continues to provide the same functionality for gcc? That would then also clarify the reason for it being in any particular place (I guess there are going to be more) without the need for further commentary.

IMO that's a little bit dangerous, from the LLVM bug report it seems like LLVM behaviour is not a bug, and hence I wouldn't be surprised if newer versions of gcc also exhibit the same issue.

Thanks, Roger.

[0] http://xenbits.xen.org/gitweb/?p=xen.git;a=commit;h=43fa95ae6a64132b8ebe3025bd187ab9df68677b

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

#3 lore raw
Quoted 31 lines · Roger Pau Monné
Quoted 12 lines · Jan Beulich

xen/arch/x86/xstate.c unsigned int xstate_ctxt_size(u64 xcr0) · line 580

Quoted 8 lines · Roger Pau Monne
xen/arch/x86/xstate.c +1 -1
@@ -577,7 +577,7 @@ unsigned int xstate_ctxt_size(u64 xcr0)
   /* Collect the information of processor's extended state */
   void xstate_init(struct cpuinfo_x86 *c)
   {
-    static bool __initdata use_xsave = true;
+    static bool use_xsave = true;

Please attach at least a brief comment here, such that people won't consider the __initdata missing.

Sure.

Quoted 5 lines

Of course I'd actually prefer the annotation to stay here in the gcc case. Iirc there was one other case where there was such an issue; I don't recall whether there it too got dealt with by removing an annotation.

Yes, in that other case the annotation was just removed, it's 43fa95ae [0]

Quoted 6 lines

How about we introduce an
annotation that expands to nothing in the clang case, but continues to provide the same functionality for gcc? That would then also clarify the reason for it being in any particular place (I guess there are going to be more) without the need for further commentary.

Quoted 3 lines

IMO that's a little bit dangerous, from the LLVM bug report it seems like LLVM behaviour is not a bug, and hence I wouldn't be surprised if newer versions of gcc also exhibit the same issue.

Okay, then let's go the route you suggested, just with a comment added to prevent re-addition of the annotation. I wonder whether we ought to do an audit of the code to find possible further offenders. It doesn't look very desirable to me to find the instances one by one by someone observing crashes.

Hmm, having looked at the older commit I'm again starting to wonder whether dropping the annotations is the right course of action. Did we consider adding volatile to the variable back then? That should make the compiler not pull ahead the memory access(es), and the downsides of this should be pretty limited for init-only items.

Jan
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

#4 lore raw
Quoted 42 lines · Jan Beulich
Quoted 31 lines · Roger Pau Monné
Quoted 12 lines · Jan Beulich

xen/arch/x86/xstate.c unsigned int xstate_ctxt_size(u64 xcr0) · line 580

Quoted 8 lines · Roger Pau Monne
xen/arch/x86/xstate.c +1 -1
@@ -577,7 +577,7 @@ unsigned int xstate_ctxt_size(u64 xcr0)
   /* Collect the information of processor's extended state */
   void xstate_init(struct cpuinfo_x86 *c)
   {
-    static bool __initdata use_xsave = true;
+    static bool use_xsave = true;

Please attach at least a brief comment here, such that people won't consider the __initdata missing.

Sure.

Quoted 5 lines

Of course I'd actually prefer the annotation to stay here in the gcc case. Iirc there was one other case where there was such an issue; I don't recall whether there it too got dealt with by removing an annotation.

Yes, in that other case the annotation was just removed, it's 43fa95ae [0]

Quoted 6 lines

How about we introduce an
annotation that expands to nothing in the clang case, but continues to provide the same functionality for gcc? That would then also clarify the reason for it being in any particular place (I guess there are going to be more) without the need for further commentary.

Quoted 3 lines

IMO that's a little bit dangerous, from the LLVM bug report it seems like LLVM behaviour is not a bug, and hence I wouldn't be surprised if newer versions of gcc also exhibit the same issue.

Okay, then let's go the route you suggested, just with a comment added to prevent re-addition of the annotation. I wonder whether we ought to do an audit of the code to find possible further offenders. It doesn't look very desirable to me to find the instances one by one by someone observing crashes.

Quoted 5 lines

Hmm, having looked at the older commit I'm again starting to wonder whether dropping the annotations is the right course of action. Did we consider adding volatile to the variable back then? That should make the compiler not pull ahead the memory access(es), and the downsides of this should be pretty limited for init-only items.

IIRC at some point I suggested using ACCESS_ONCE to read the init opt_bootscrub variable. Adding the volatile keyword to the __initdata macro is not going to work as-is, because we would also need to fixup the existing declarations of the variables.

A more simple (and maybe easier to enforce rule) might be to forbid the usage of init data/text in non-init functions, and that would likely be easier to enforce automatically by some analysis tool.

Thanks, Roger.

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

#5 lore raw
Quoted 49 lines · Roger Pau Monné
Quoted 42 lines · Jan Beulich
Quoted 31 lines · Roger Pau Monné
Quoted 12 lines · Jan Beulich

xen/arch/x86/xstate.c unsigned int xstate_ctxt_size(u64 xcr0) · line 580

Quoted 8 lines · Roger Pau Monne
xen/arch/x86/xstate.c +1 -1
@@ -577,7 +577,7 @@ unsigned int xstate_ctxt_size(u64 xcr0)
    /* Collect the information of processor's extended state */
    void xstate_init(struct cpuinfo_x86 *c)
    {
-    static bool __initdata use_xsave = true;
+    static bool use_xsave = true;

Please attach at least a brief comment here, such that people won't consider the __initdata missing.

Sure.

Quoted 5 lines

Of course I'd actually prefer the annotation to stay here in the gcc case. Iirc there was one other case where there was such an issue; I don't recall whether there it too got dealt with by removing an annotation.

Yes, in that other case the annotation was just removed, it's 43fa95ae [0]

Quoted 6 lines

How about we introduce an
annotation that expands to nothing in the clang case, but continues to provide the same functionality for gcc? That would then also clarify the reason for it being in any particular place (I guess there are going to be more) without the need for further commentary.

Quoted 3 lines

IMO that's a little bit dangerous, from the LLVM bug report it seems like LLVM behaviour is not a bug, and hence I wouldn't be surprised if newer versions of gcc also exhibit the same issue.

Okay, then let's go the route you suggested, just with a comment added to prevent re-addition of the annotation. I wonder whether we ought to do an audit of the code to find possible further offenders. It doesn't look very desirable to me to find the instances one by one by someone observing crashes.

Quoted 5 lines

Hmm, having looked at the older commit I'm again starting to wonder whether dropping the annotations is the right course of action. Did we consider adding volatile to the variable back then? That should make the compiler not pull ahead the memory access(es), and the downsides of this should be pretty limited for init-only items.

Quoted 4 lines

IIRC at some point I suggested using ACCESS_ONCE to read the init opt_bootscrub variable. Adding the volatile keyword to the __initdata macro is not going to work as-is, because we would also need to fixup the existing declarations of the variables.

ACCESS_ONCE() is not good here because you need to then catch all _uses_ of a variable. Dealing with it by changing the declaration/ definition makes it a one (or at most two) place change. I wouldn't be worried about a header file also needing to change.

Quoted 3 lines

A more simple (and maybe easier to enforce rule) might be to forbid the usage of init data/text in non-init functions, and that would likely be easier to enforce automatically by some analysis tool.

If you or anyone else is up for integrating such a tool with the build system - fine. Right now we have none, and I don't want such issues to be caught be solely dependent on the issue getting noticed the latest during review.

That said, I'm not actually convinced unconditionally forbidding such references (like Linux does, requiring further annotations to override the ban) is the best model. There are clearly cases where having such references is appropriate and useful. But I can accept that Linux'es model is overall more safe than our current effectively non-existent one.

Jan

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel