Skip to contentSkip to conversation

exec: don't force_sigsegv processes with a pending fatal signal

PATCH RESEND Superseded Ivan Delalande 3 participants 5 messages Jul 31, 2018 – Aug 3, 2018

Subsystems (2)

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore
#0 lore raw

PATCH RESEND 1/1 exec: don't force_sigsegv processes with a pending fatal signal

We were seeing unexplained segfaults in coreutils processes and other basic utilities that we tracked down to binfmt_elf failing to load segments for ld.so. Digging further, the actual problem seems to occur when a process gets sigkilled while it is still being loaded by the kernel. In our case when _do_page_fault goes for a retry it will return early as it first checks for fatal_signal_pending(), so load_elf_interp also returns with error and as a result search_binary_handler will force_sigsegv() which is pretty confusing as nothing actually failed here.

Fixes 19d860a140be ("handle suicide on late failure exits in execve() in search_binary_handler()")

Reference: https://lkml.org/lkml/2013/2/14/5

---
 fs/exec.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
fs/exec.c +2 -1
index bdd0eacefdf5..6e8007edbb2d 100644
@@ -1656,7 +1656,8 @@ int search_binary_handler(struct linux_binprm *bprm)
 		if (retval < 0 && !bprm->mm) {
 			/* we got to flush_old_exec() and failed after it */
 			read_unlock(&binfmt_lock);
-			force_sigsegv(SIGSEGV, current);
+			if (!fatal_signal_pending(current))
+				force_sigsegv(SIGSEGV, current);
 			return retval;
 		}
 		if (retval != -ENOEXEC || !bprm->file) {
Signature · 1 line
2.18.0
#1 lore raw

Hi Ivan,

2018-07-31 1:56 GMT+01:00 Ivan Delalande <[email protected]>:

Quoted 10 lines

We were seeing unexplained segfaults in coreutils processes and other basic utilities that we tracked down to binfmt_elf failing to load segments for ld.so. Digging further, the actual problem seems to occur when a process gets sigkilled while it is still being loaded by the kernel. In our case when _do_page_fault goes for a retry it will return early as it first checks for fatal_signal_pending(), so load_elf_interp also returns with error and as a result search_binary_handler will force_sigsegv() which is pretty confusing as nothing actually failed here.

Quoted 3 lines
Fixes 19d860a140be ("handle suicide on late failure exits in execve() in search_binary_handler()")

Reference: https://lkml.org/lkml/2013/2/14/5

+Cc: Oleg Nesterov <[email protected]>
+Cc: Andy Lutomirski <[email protected]>

Quoted 9 lines
---
 fs/exec.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
fs/exec.c +0 -0
index bdd0eacefdf5..6e8007edbb2d 100644

fs/exec.c int search_binary_handler(struct linux_binprm *bprm) · line 1660

Quoted 6 lines
fs/exec.c +2 -1
@@ -1656,7 +1656,8 @@ int search_binary_handler(struct linux_binprm *bprm)
                if (retval < 0 && !bprm->mm) {
                        /* we got to flush_old_exec() and failed after it */
                        read_unlock(&binfmt_lock);
-                       force_sigsegv(SIGSEGV, current);
+                       if (!fatal_signal_pending(current))
+                               force_sigsegv(SIGSEGV, current);

I would suggest to add something like:
: if (print_fatal_signals)
: pr_info("load_binary() failed: %d\n", retval);

It was interesting to catch that it actually segfaults during loading, probably will save someone a couple of minutes too ;-)

fs/exec.c int search_binary_handler(struct linux_binprm *bprm) · line 1663

Quoted 3 lines
fs/exec.c +0 -0
                        return retval;
                }
                if (retval != -ENOEXEC || !bprm->file) {

Thanks,
Dmitry

#2 lore raw

2018-08-02 20:53 GMT+01:00 Dmitry Safonov <[email protected]>:

Quoted 17 lines

Hi Ivan,

2018-07-31 1:56 GMT+01:00 Ivan Delalande <[email protected]>:

Quoted 13 lines

We were seeing unexplained segfaults in coreutils processes and other basic utilities that we tracked down to binfmt_elf failing to load segments for ld.so. Digging further, the actual problem seems to occur when a process gets sigkilled while it is still being loaded by the kernel. In our case when _do_page_fault goes for a retry it will return early as it first checks for fatal_signal_pending(), so load_elf_interp also returns with error and as a result search_binary_handler will force_sigsegv() which is pretty confusing as nothing actually failed here.

Fixes 19d860a140be ("handle suicide on late failure exits in execve() in search_binary_handler()")

Reference: https://lkml.org/lkml/2013/2/14/5

Review

Quoted 2 lines
(quoted diff) +2 -0
+Cc: Oleg Nesterov <[email protected]>
+Cc: Andy Lutomirski <[email protected]>

Also worth to add to commit message an example of previously user-visible message in dmesg:

[ 1545.889604] potentially unexpected fatal signal 11.
[ 1545.889614] CPU: 2 PID: 7462 Comm: grep Tainted: P           O   3.18.28 #1
[ 1545.889617] Hardware name: Celestica D4040/D4040, BIOS 5.6.5 08/18/2016
[ 1545.889621] task: ffff880011282280 ti: ffff880100938000 task.ti:
ffff880100938000
[ 1545.889624] RIP: 0023:[<00000000f760eb70>]  [<00000000f760eb70>] 0xf760eb70
[ 1545.889641] RSP: 002b:00000000fffa3454  EFLAGS: 00000296
[ 1545.889644] RAX: fffffffffffffff2 RBX: 00000000f7a5c3e8 RCX: 00000000f7a5c718
[ 1545.889647] RDX: 00000000f7a5b230 RSI: 00000000f7a5c718 RDI: 00000000f757c000
[ 1545.889650] RBP: 00000000f7a5c3e8 R08: 0000000000000000 R09: 0000000000000000
[ 1545.889653] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
[ 1545.889656] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
[ 1545.889659] FS:  0000000000000000(0000) GS:ffff88017fb00000(0000)
knlGS:0000000000000000
[ 1545.889662] CS:  0010 DS: 002b ES: 002b CR0: 000000008005003b
[ 1545.889665] CR2: 00000000f77d7838 CR3: 000000010bb90000 CR4: 00000000001007e0

(which now will be suppressed if there was a fatal signal)

Quoted 21 lines
Quoted 10 lines
---
 fs/exec.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
fs/exec.c +0 -0
index bdd0eacefdf5..6e8007edbb2d 100644

fs/exec.c int search_binary_handler(struct linux_binprm *bprm) · line 1660

Quoted 6 lines
fs/exec.c +2 -1
@@ -1656,7 +1656,8 @@ int search_binary_handler(struct linux_binprm *bprm)
                if (retval < 0 && !bprm->mm) {
                        /* we got to flush_old_exec() and failed after it */
                        read_unlock(&binfmt_lock);
-                       force_sigsegv(SIGSEGV, current);
+                       if (!fatal_signal_pending(current))
+                               force_sigsegv(SIGSEGV, current);

I would suggest to add something like:
: if (print_fatal_signals)
: pr_info("load_binary() failed: %d\n", retval);

Quoted 2 lines

It was interesting to catch that it actually segfaults during loading, probably will save someone a couple of minutes too ;-)

Not sure if it's easy to trigger, but it might require a ratelimit too..

Quoted 4 lines
Quoted 4 lines
fs/exec.c +0 -0
                        return retval;
                }
                if (retval != -ENOEXEC || !bprm->file) {
Signature · 1 line
             Dmitry
#3 lore raw
In reply to Dmitry Safonov
Quoted 18 lines · Dmitry Safonov

Hi Ivan,

2018-07-31 1:56 GMT+01:00 Ivan Delalande <[email protected]>:

Quoted 13 lines

We were seeing unexplained segfaults in coreutils processes and other basic utilities that we tracked down to binfmt_elf failing to load segments for ld.so. Digging further, the actual problem seems to occur when a process gets sigkilled while it is still being loaded by the kernel. In our case when _do_page_fault goes for a retry it will return early as it first checks for fatal_signal_pending(), so load_elf_interp also returns with error and as a result search_binary_handler will force_sigsegv() which is pretty confusing as nothing actually failed here.

Fixes 19d860a140be ("handle suicide on late failure exits in execve() in search_binary_handler()")

Reference: https://lkml.org/lkml/2013/2/14/5

Review

Quoted 2 lines
(quoted diff) +2 -0
+Cc: Oleg Nesterov <[email protected]>
+Cc: Andy Lutomirski <[email protected]>

Thanks...

and sorry, I fail to understand the problem and what/how this patch tries to fix.

Hmm. After I read the next email from Dmitry it seems to me that the whole purpose of this patch is to avoid print_fatal_signal()? If yes, the changelog should clearly explain this.

Quoted 9 lines

fs/exec.c int search_binary_handler(struct linux_binprm *bprm) · line 1660

Quoted 9 lines
fs/exec.c +2 -1
@@ -1656,7 +1656,8 @@ int search_binary_handler(struct linux_binprm *bprm)
                if (retval < 0 && !bprm->mm) {
                        /* we got to flush_old_exec() and failed after it */
                        read_unlock(&binfmt_lock);
-                       force_sigsegv(SIGSEGV, current);
+                       if (!fatal_signal_pending(current))
+                               force_sigsegv(SIGSEGV, current);

I won't argue, but may be force_sigsegv() should check fatal_signal_pending() itself. setup_rt_frame() can too fail if fatal_signal_pending() by the same reason.

Oleg.

#4 lore raw

Hi,

Quoted 21 lines · Oleg Nesterov
Quoted 15 lines · Dmitry Safonov

2018-07-31 1:56 GMT+01:00 Ivan Delalande <[email protected]>:

Quoted 13 lines

We were seeing unexplained segfaults in coreutils processes and other basic utilities that we tracked down to binfmt_elf failing to load segments for ld.so. Digging further, the actual problem seems to occur when a process gets sigkilled while it is still being loaded by the kernel. In our case when _do_page_fault goes for a retry it will return early as it first checks for fatal_signal_pending(), so load_elf_interp also returns with error and as a result search_binary_handler will force_sigsegv() which is pretty confusing as nothing actually failed here.

Fixes 19d860a140be ("handle suicide on late failure exits in execve() in search_binary_handler()")

Reference: https://lkml.org/lkml/2013/2/14/5

Review

Quoted 2 lines
(quoted diff) +2 -0
+Cc: Oleg Nesterov <[email protected]>
+Cc: Andy Lutomirski <[email protected]>

Thanks...

Quoted 5 lines

and sorry, I fail to understand the problem and what/how this patch tries to fix.

Hmm. After I read the next email from Dmitry it seems to me that the whole purpose of this patch is to avoid print_fatal_signal()? If yes, the changelog should clearly explain this.

Sorry about that, yes this is purely to avoid printing the segfault messages for these processes when they were in fact killed. I'll definitely send a v2 to clarify that, and probably add the helpful message Dimitry suggested as well.

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
fs/exec.c +0 -0
@@ -1656,7 +1656,8 @@ int search_binary_handler(struct linux_binprm *bprm)
                if (retval < 0 && !bprm->mm) {
                        /* we got to flush_old_exec() and failed after it */
Quoted 8 lines
Quoted 4 lines

fs/exec.c int search_binary_handler(struct linux_binprm *bprm) · line 1660

Quoted 4 lines
fs/exec.c +2 -1
                        read_unlock(&binfmt_lock);
-                       force_sigsegv(SIGSEGV, current);
+                       if (!fatal_signal_pending(current))
+                               force_sigsegv(SIGSEGV, current);

I won't argue, but may be force_sigsegv() should check fatal_signal_pending() itself. setup_rt_frame() can too fail if fatal_signal_pending() by the same reason.

I'm not sure, I think it would feel out of place in force_sigsegv() as other callers might not expect this check in different contexts. I could add a similar call to fatal_signal_pending() in signal_setup_done() though, if you think we can hit the same problem from setup_rt_frame().

Thanks,

Signature · 2 lines
Ivan Delalande
Arista Networks