Skip to contentSkip to conversation

um: Fix misaligned stack in stub_exe

PATCH Superseded David Gow 3 participants 4 messages Oct 17, 2024 – Oct 21, 2024 User-mode Linux (UML)

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore

David Gow

author
#0 lore raw

PATCH 1/1 um: Fix misaligned stack in stub_exe

The stub_exe could segfault when built with some compilers (e.g. gcc 13.2.0), as SSE instructions which relied on stack alignment could be generated, but the stack was misaligned.

This seems to be due to the __start entry point being run with a 16-byte aligned stack, but the x86_64 SYSV ABI wanting the stack to be so aligned _before_ a function call (so it is misaligned when the function is entered due to the return address being pushed). The function prologue then realigns it. Because the entry point is never _called_, and hence there is no return address, the prologue is therefore actually misaligning it, and causing the generated movaps instructions to SIGSEGV. This results in the following error:
start_userspace : expected SIGSTOP, got status = 139

Don't generate this prologue for __start by using
__attribute__((naked)), which resolves the issue.

Fixes 32e8eaf263d9 ("um: use execveat to create userspace MMs")

---

See the discussion here:
https://lore.kernel.org/linux-um/[email protected]/T/#m90c1c5b6c34ebaaa043b402e97009c5825fd158a

---
 arch/um/kernel/skas/stub_exe.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
arch/um/kernel/skas/stub_exe.c +1 -1
index 04f75c577f1a..722ce6267476 100644
@@ -79,7 +79,7 @@ noinline static void real_init(void)
 	__builtin_unreachable();
 }
 
-void _start(void)
+__attribute__((naked)) void _start(void)
 {
 	char *alloc;
 
Signature · 1 line
2.47.0.rc1.288.g06298d1525-goog
#1 lore raw
kernel test robot noticed the following build errors:

Hi David,

kernel test robot noticed the following build errors:

[auto build test ERROR on uml/next]
[also build test ERROR on next-20241018]
[cannot apply to linus/master v6.12-rc3]
[If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]

url:    https://github.com/intel-lab-lkp/linux/commits/David-Gow/um-Fix-misaligned-stack-in-stub_exe/20241018-071148
base:   https://git.kernel.org/pub/scm/linux/kernel/git/uml/linux next
patch link:    https://lore.kernel.org/r/20241017231007.1500497-2-davidgow%40google.com
patch subject: [PATCH] um: Fix misaligned stack in stub_exe
config: um-allnoconfig (https://download.01.org/0day-ci/archive/20241019/[email protected]/config)
compiler: clang version 17.0.6 (https://github.com/llvm/llvm-project 6009708b4367171ccdbf4b5905cb6a803753fe18)
reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20241019/[email protected]/reproduce)

If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <[email protected]>
| Closes: https://lore.kernel.org/oe-kbuild-all/[email protected]/

All errors (new ones prefixed by >>):

>> arch/um/kernel/skas/stub_exe.c:84:2: error: non-ASM statement in naked function is not supported
      84 |         char *alloc;
         |         ^
   arch/um/kernel/skas/stub_exe.c:82:16: note: attribute is here
      82 | __attribute__((naked)) void _start(void)
         |                ^

1 error generated.

vim +84 arch/um/kernel/skas/stub_exe.c

32e8eaf263d9be Benjamin Berg 2024-09-19  81  
ae9ffda02a9904 David Gow     2024-10-18  82  __attribute__((naked)) void _start(void)
32e8eaf263d9be Benjamin Berg 2024-09-19  83  {

32e8eaf263d9be Benjamin Berg 2024-09-19 @84 char *alloc;

Signature · 2 lines
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki
author
#2 lore raw
In reply to David Gow
Quoted 15 lines · David Gow

The stub_exe could segfault when built with some compilers (e.g. gcc 13.2.0), as SSE instructions which relied on stack alignment could be generated, but the stack was misaligned.

This seems to be due to the __start entry point being run with a 16-byte aligned stack, but the x86_64 SYSV ABI wanting the stack to be so aligned _before_ a function call (so it is misaligned when the function is entered due to the return address being pushed). The function prologue then realigns it. Because the entry point is never _called_, and hence there is no return address, the prologue is therefore actually misaligning it, and causing the generated movaps instructions to SIGSEGV. This results in the following error:
start_userspace : expected SIGSTOP, got status = 139

Quoted 6 lines

Don't generate this prologue for __start by using
__attribute__((naked)), which resolves the issue.

Fixes 32e8eaf263d9 ("um: use execveat to create userspace MMs")

---

Okay, it turns out this breaks clang:
arch/um/kernel/skas/stub_exe.c:84:2: error: non-ASM statement in naked function is not supported

And, looking into it, gcc's docs are not encouraging here either:
https://gcc.gnu.org/onlinedocs/gcc/x86-Function-Attributes.html

This attribute allows the compiler to construct the requisite function declaration, while allowing the body of the function to be assembly code. The specified function will not have prologue/epilogue sequences generated by the compiler. Only basic asm statements can safely be included in naked functions (see Basic Asm — Assembler Instructions Without Operands). While using extended asm or a mixture of basic asm and C code may appear to work, they cannot be depended upon to work reliably and are not supported.

My gut feeling is that the "correct" way of doing this is to use an actual crt implementation for __start. I managed to get it working with nolibc on x86_54, but the sheer amount of hackery involved was not exactly encouraging. There are a lot of conflicts between the different headers for a start.

The other "correct" way would be to rewrite __start in assembly, which is annoying as it'd be architecture specific, so need a separate 32-bit version.

The less-correct-but-working-here way, which I'm tempted by for now, is to get rid of __attribute__((naked)) and just use __attribute__((force_arg_align_pointer)). That's probably the best way of fixing the stack issue, but obviously won't fix any other issues which could arise from __start playing loose with the rules.

My current plan is to send out a v2 with force_arg_align_pointer next week, unless someone has a more brilliant idea.

Cheers,
-- David

Quoted 24 lines

See the discussion here:
https://lore.kernel.org/linux-um/[email protected]/T/#m90c1c5b6c34ebaaa043b402e97009c5825fd158a

---
 arch/um/kernel/skas/stub_exe.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
arch/um/kernel/skas/stub_exe.c +1 -2
index 04f75c577f1a..722ce6267476 100644
@@ -79,7 +79,7 @@ noinline static void real_init(void)
        __builtin_unreachable();
 }
 
-void _start(void)
+__attribute__((naked)) void _start(void)
 {
        char *alloc;
 
--

2.47.0.rc1.288.g06298d1525-goog

maintainer
#3 lore raw
Quoted 4 lines · David Gow

Okay, it turns out this breaks clang:
arch/um/kernel/skas/stub_exe.c:84:2: error: non-ASM statement in naked function is not supported

Fun :)

Interesting too, I've _definitely_ got some code elsewhere, that's usually compiled with clang, using __attribute__((naked,noinline)). With a quite old clang version, I believe. Oh well, whatever.

Quoted 11 lines

And, looking into it, gcc's docs are not encouraging here either:
https://gcc.gnu.org/onlinedocs/gcc/x86-Function-Attributes.html

This attribute allows the compiler to construct the requisite function declaration, while allowing the body of the function to be assembly code. The specified function will not have prologue/epilogue sequences generated by the compiler. Only basic asm statements can safely be included in naked functions (see Basic Asm — Assembler Instructions Without Operands). While using extended asm or a mixture of basic asm and C code may appear to work, they cannot be depended upon to work reliably and are not supported.

Right ...

Quoted 5 lines

My gut feeling is that the "correct" way of doing this is to use an actual crt implementation for __start. I managed to get it working with nolibc on x86_54, but the sheer amount of hackery involved was not exactly encouraging. There are a lot of conflicts between the different headers for a start.

Yeah, that doesn't seem like a lot of fun.

Quoted 3 lines

The other "correct" way would be to rewrite __start in assembly, which is annoying as it'd be architecture specific, so need a separate 32-bit version.

That'd probably be less bad than it sounds, since all the syscall magic macros etc. in there are already architecture specific.

Quoted 8 lines

The less-correct-but-working-here way, which I'm tempted by for now, is to get rid of __attribute__((naked)) and just use __attribute__((force_arg_align_pointer)). That's probably the best way of fixing the stack issue, but obviously won't fix any other issues which could arise from __start playing loose with the rules.

My current plan is to send out a v2 with force_arg_align_pointer next week, unless someone has a more brilliant idea.

Sounds good to me. We'll find out if we have to iterate more ;)

johannes