Skip to contentSkip to conversation

execve updates for v6.12-rc1

GIT PULL Merged Kees Cook 5 participants 8 messages Sep 16, 2024 – Sep 28, 2024 Regressions

lore
Share & tools
lore

Kees Cook

author
#0 lore raw

Hi Linus,

Please pull these execve updates for v6.12-rc1. Note there is a trivial merge conflict between this and mm, which was resolved in -next with:
https://lore.kernel.org/linux-next/[email protected]/

Thanks!

-Kees

The following changes since commit de9c2c66ad8e787abec7c9d7eff4f8c3cdd28aed:

Linux 6.11-rc2 (2024-08-04 13:50:53 -0700)

are available in the Git repository at:

https://git.kernel.org/pub/scm/linux/kernel/git/kees/linux.git tags/execve-v6.12-rc1

for you to fetch changes up to 44f65d900698278a8451988abe0d5ca37fd46882:

binfmt_elf: mseal address zero (2024-08-14 09:56:48 -0700)

----------------------------------------------------------------
execve updates for v6.12-rc1

- binfmt_elf: Dump smaller VMAs first in ELF cores (Brian Mak)

- binfmt_elf: mseal address zero (Jeff Xu)

- binfmt_elf, coredump: Log the reason of the failed core dumps (Roman Kisel)

----------------------------------------------------------------
Brian Mak (1):
binfmt_elf: Dump smaller VMAs first in ELF cores

Jeff Xu (1):
binfmt_elf: mseal address zero

Roman Kisel (2):
coredump: Standartize and fix logging
binfmt_elf, coredump: Log the reason of the failed core dumps

 fs/binfmt_elf.c          |  53 +++++++++++----
 fs/coredump.c            | 166 ++++++++++++++++++++++++++++++++++-------------
 include/linux/coredump.h |  30 ++++++++-
 include/linux/mm.h       |  10 +++
 kernel/signal.c          |  21 +++++-
 mm/mseal.c               |   2 +-
 6 files changed, 220 insertions(+), 62 deletions(-)
Signature · 1 line
Kees Cook
merge bot Merged
#1 lore raw
The pull request you sent on Mon, 16 Sep 2024 01:39:39 -0700:

The pull request you sent on Mon, 16 Sep 2024 01:39:39 -0700:

Quoted 1 line

https://git.kernel.org/pub/scm/linux/kernel/git/kees/linux.git tags/execve-v6.12-rc1

has been merged into torvalds/linux.git:
https://git.kernel.org/torvalds/c/667495de218c25e909c6b33ed647b592a8a71a02

Thank you!

Signature · 2 lines
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/prtracker.html
#2 lore raw
In reply to Kees Cook
Quoted 25 lines · Kees Cook

Hi Linus,

Please pull these execve updates for v6.12-rc1. Note there is a trivial merge conflict between this and mm, which was resolved in -next with:
https://lore.kernel.org/linux-next/[email protected]/

Thanks!

-Kees

The following changes since commit de9c2c66ad8e787abec7c9d7eff4f8c3cdd28aed:

   Linux 6.11-rc2 (2024-08-04 13:50:53 -0700)

are available in the Git repository at:

https://git.kernel.org/pub/scm/linux/kernel/git/kees/linux.git tags/execve-v6.12-rc1

for you to fetch changes up to 44f65d900698278a8451988abe0d5ca37fd46882:

   binfmt_elf: mseal address zero (2024-08-14 09:56:48 -0700)

----------------------------------------------------------------
execve updates for v6.12-rc1

Quoted 6 lines

- binfmt_elf: Dump smaller VMAs first in ELF cores (Brian Mak)

- binfmt_elf: mseal address zero (Jeff Xu)

- binfmt_elf, coredump: Log the reason of the failed core dumps (Roman Kisel)

Hi,

This last commit seems to introduce a regression for me, creating a completely unkillable process (but idle/0% CPU) that is stuck here:

$ sudo cat /proc/2453/stack
[<0>] do_exit+0xee/0xac0
[<0>] do_group_exit+0x34/0x90
[<0>] get_signal+0xa63/0xa70
[<0>] arch_do_signal_or_restart+0x42/0x260
[<0>] irqentry_exit_to_user_mode+0x1e0/0x250
[<0>] irqentry_exit+0x43/0x50
[<0>] exc_page_fault+0x94/0x1d0
[<0>] asm_exc_page_fault+0x27/0x30
$ cat /proc/2453/status
...
State:  I (idle)
...
TracerPid:      0
...
Kthread:        0
VmPeak:     2240 kB
VmSize:     2240 kB
VmLck:         0 kB
VmPin:         0 kB
VmHWM:       568 kB
VmRSS:       568 kB
RssAnon:             136 kB
RssFile:             432 kB
RssShmem:              0 kB
VmData:      420 kB
VmStk:       132 kB
VmExe:      1644 kB
VmLib:        16 kB
VmPTE:        60 kB
VmSwap:        0 kB
HugetlbPages:          0 kB
CoreDumping:    1
THP_enabled:    1
untag_mask:     0xffffffffffffffff
Threads:        1
SigQ:   0/62622
SigPnd: 0000000000000100
ShdPnd: 0000000000000100
SigBlk: 0000000000000000
SigIgn: 0000000000000000
SigCgt: 00000000000020db
...

The process is so unkillable I can't even shut my laptop down without holding the power button for 5 seconds -- apart from that, everything works correctly.

Bisection ended up here:

# first bad commit: [fb97d2eb542faf19a8725afbd75cbc2518903210] binfmt_elf, coredump: Log the reason of the failed core dumps

I have to admit I don't immediately see what's wrong with the patch.

Vegard

#3 lore raw
Quoted 5 lines · Vegard Nossum
# first bad commit: [fb97d2eb542faf19a8725afbd75cbc2518903210]
binfmt_elf, coredump: Log the reason of the failed core dumps

I have to admit I don't immediately see what's wrong with the patch.

That commit looks entirely broken.

I *suspect* that the problem is the crazy "get_task_comm()" in that takes the task lock inside coredump_report_failure().

But honestly, I'm not going to bother even trying to debug this. The whole notion was broken. People who have problems with truncated core-files should be looking at their debuggers, not asking the kernel for help.

Linus

#4 lore raw
Quoted 9 lines · Linus Torvalds
Quoted 5 lines · Vegard Nossum
# first bad commit: [fb97d2eb542faf19a8725afbd75cbc2518903210]
binfmt_elf, coredump: Log the reason of the failed core dumps

I have to admit I don't immediately see what's wrong with the patch.

That commit looks entirely broken.

Quoted 7 lines

I *suspect* that the problem is the crazy "get_task_comm()" in that takes the task lock inside coredump_report_failure().

But honestly, I'm not going to bother even trying to debug this. The whole notion was broken. People who have problems with truncated core-files should be looking at their debuggers, not asking the kernel for help.

One of the common causes for coredump truncation is weird interactions between io_uring and the coredump code. (AKA kernel bugs).

That is something you can't ask your debugger to tell you.

So from 10,000 feet I think the idea is sane.

Eric

#5 lore raw
Quoted 7 lines · Eric W. Biederman

One of the common causes for coredump truncation is weird interactions between io_uring and the coredump code. (AKA kernel bugs).

That is something you can't ask your debugger to tell you.

So from 10,000 feet I think the idea is sane.

What? No. Adding printk's to chase kernel bugs is certainly a time-honored tradition. But we don't leave them in the kernel sources for posterity.

And none of the coredumpo failure reports had anything to do with io_uring bugs anyway. They were literally "print out when disk filled up or core dumps weren't enabled".

If you didn't get a core dump because the kernel didn't have core dumps configured, we shouldn't print out some babying kernel message about that.

None of this has anything to do with io_uring or kernel bugs.

Linus

#6 lore raw
Quoted 5 lines · Linus Torvalds
Quoted 4 lines · Eric W. Biederman

One of the common causes for coredump truncation is weird interactions between io_uring and the coredump code. (AKA kernel bugs).

Quoted 7 lines
Quoted 3 lines

That is something you can't ask your debugger to tell you.

So from 10,000 feet I think the idea is sane.

What? No. Adding printk's to chase kernel bugs is certainly a time-honored tradition. But we don't leave them in the kernel sources for posterity.

No argument from me there. We certainly don't leave them enabled by default. Although in truth most of the failures the coredump code can experience are cases that should never happen in normal operation.

Quoted 3 lines

And none of the coredumpo failure reports had anything to do with io_uring bugs anyway. They were literally "print out when disk filled up or core dumps weren't enabled".

dump_interrupted was instrumented. That is what io_uring was triggering. In fact dump_interrupted still has problems with I think dumping to a pipe.

Quoted 3 lines

If you didn't get a core dump because the kernel didn't have core dumps configured, we shouldn't print out some babying kernel message about that.

Some of them are certainly silly, or excessive.

Quoted 1 line

None of this has anything to do with io_uring or kernel bugs.

I respectfully disagree.

A huge part of the problem is that when io_uring triggers dump_interrupted it is so subtle people don't have a clue what is going on. Not that I am saying it is necessarily io_uring that is just the one I have debugged and tried to sort out. Other kernel subsystems could have similar weird interactions, but io_uring where it plays with TIF_NOTIFY_SIGNAL has caused problems in the past.

I don't vouch for this implementation or think it is necessarily the right way to get better information out, but the coredump code is very much a black box that is quite difficult for people to work with.

What I know is that recently truncated core dumps have been on peoples radar enough that we received two separate patches from two different organizations to do something about them. That says to me that this an actual problem that people are experiencing, not some theoretical thing.

I am all for reverting code that doesn't work, and for looking for better solutions, but simply saying to people their pain is not a real problem. That seems terribly wrong.

Eric