seccomp: restore knotif->state when SECCOMP_ADDFD_FLAG_SEND is interrupted
Subsystems (3)
PATCH 1/1 seccomp: restore knotif->state when SECCOMP_ADDFD_FLAG_SEND is interrupted
In seccomp_notify_addfd(), when SECCOMP_ADDFD_FLAG_SEND is set, knotif->state is transitioned from SECCOMP_NOTIFY_SENT to SECCOMP_NOTIFY_REPLIED before dropping filter->notify_lock and waiting in wait_for_completion_interruptible(&kaddfd.completion).
If wait_for_completion_interruptible() is interrupted by a signal before the target task processes the kaddfd entry, seccomp_notify_addfd() removes kaddfd.list from knotif->addfd via list_del(&kaddfd.list), but leaves knotif->state set to SECCOMP_NOTIFY_REPLIED.
As a consequence:
1. When the target task wakes in seccomp_do_user_notification(), it observes knotif->state == SECCOMP_NOTIFY_REPLIED with an empty addfd list and zeroed knotif->val / knotif->error / knotif->flags, causing the supervised syscall to prematurely complete with return value 0 without installing the file descriptor.
2. Any subsequent retry of SECCOMP_IOCTL_NOTIF_SEND or SECCOMP_IOCTL_NOTIF_ADDFD by the supervisor fails with -EINPROGRESS because knotif->state is no longer SECCOMP_NOTIFY_SENT.
Restore knotif->state = SECCOMP_NOTIFY_SENT when removing an unhandled SECCOMP_ADDFD_FLAG_SEND request from knotif->addfd.
---
kernel/seccomp.c +5 -2
@@ -1808,10 +1808,13 @@ static long seccomp_notify_addfd(struct seccomp_filter *filter,* We need to check again if the addfd request has been handled, * and if not, we will remove it from the queue. */ -if (list_empty (&kaddfd.list)) +if (list_empty (&kaddfd.list)) { ret = kaddfd.ret; -else + }else {list_del (&kaddfd.list); +if (addfd.flags &SECCOMP_ADDFD_FLAG_SEND ) + knotif->state =SECCOMP_NOTIFY_SENT ; + } out_unlock:mutex_unlock (&filter->notify_lock);
Signature · 1 line
2.43.0
Quoted 45 lines · Hui Peng
In
seccomp_notify_addfd(), whenSECCOMP_ADDFD_FLAG_SENDis set,knotif->stateis transitioned fromSECCOMP_NOTIFY_SENTtoSECCOMP_NOTIFY_REPLIEDbefore droppingfilter->notify_lockand waiting inwait_for_completion_interruptible(&kaddfd.completion).If
wait_for_completion_interruptible()is interrupted by a signal before the target task processes the kaddfd entry,seccomp_notify_addfd()removes kaddfd.list fromknotif->addfdvialist_del(&kaddfd.list), but leavesknotif->stateset toSECCOMP_NOTIFY_REPLIED.As a consequence:
1. When the target task wakes inseccomp_do_user_notification(), it observesknotif->state==SECCOMP_NOTIFY_REPLIEDwith an empty addfd list and zeroedknotif->val/knotif->error/knotif->flags, causing the supervised syscall to prematurely complete with return value 0 without installing the file descriptor.
2. Any subsequent retry ofSECCOMP_IOCTL_NOTIF_SENDorSECCOMP_IOCTL_NOTIF_ADDFDby the supervisor fails with-EINPROGRESSbecauseknotif->stateis no longerSECCOMP_NOTIFY_SENT.Restore
knotif->state=SECCOMP_NOTIFY_SENTwhen removing an unhandledSECCOMP_ADDFD_FLAG_SENDrequest fromknotif->addfd.Fixes 7cf97b125455 ("seccomp: Introduce addfd ioctl to seccomp user notifier")Assisted-by LLMSigned-off-by Hui Peng <[email protected]>---
kernel/seccomp.c +5
-2@@ -1808,10 +1808,13 @@ static long seccomp_notify_addfd(struct seccomp_filter *filter,* We need to check again if the addfd request has been handled, * and if not, we will remove it from the queue. */ -if (list_empty (&kaddfd.list)) +if (list_empty (&kaddfd.list)) { ret = kaddfd.ret; -else + }else {list_del (&kaddfd.list); +if (addfd.flags &SECCOMP_ADDFD_FLAG_SEND ) + knotif->state =SECCOMP_NOTIFY_SENT ; + }
out_unlock:
Quoted 2 lines
mutex_unlock(&filter->notify_lock);
Akpm?
Also, Reviewed-by: Bradley Morgan <[email protected]>
--- Thanks!
"I'm not a very positive person" - Linus torvalds
Kees Cook
maintainerQuoted 9 lines · Hui Peng
In
seccomp_notify_addfd(), whenSECCOMP_ADDFD_FLAG_SENDis set,knotif->stateis transitioned fromSECCOMP_NOTIFY_SENTtoSECCOMP_NOTIFY_REPLIEDbefore droppingfilter->notify_lockand waiting inwait_for_completion_interruptible(&kaddfd.completion).If
wait_for_completion_interruptible()is interrupted by a signal before the target task processes the kaddfd entry,seccomp_notify_addfd()removes kaddfd.list fromknotif->addfdvialist_del(&kaddfd.list), but leavesknotif->stateset toSECCOMP_NOTIFY_REPLIED.
Are you able to construct a regression test for this so we can catch this race if it ever gets exposed again?
Signature · 1 line
Kees Cook
PATCH v2 1/1 seccomp: restore knotif->state when SECCOMP_ADDFD_FLAG_SEND is interrupted
In seccomp_notify_addfd(), when SECCOMP_ADDFD_FLAG_SEND is set, knotif->state is transitioned from SECCOMP_NOTIFY_SENT to SECCOMP_NOTIFY_REPLIED before dropping filter->notify_lock and waiting in wait_for_completion_interruptible(&kaddfd.completion).
If wait_for_completion_interruptible() is interrupted by a signal before the target task processes the kaddfd entry, seccomp_notify_addfd() removes kaddfd.list from knotif->addfd via list_del(&kaddfd.list), but leaves knotif->state set to SECCOMP_NOTIFY_REPLIED.
As a consequence:
1. When the target task wakes in seccomp_do_user_notification(), it observes knotif->state == SECCOMP_NOTIFY_REPLIED with an empty addfd list and zeroed knotif->val / knotif->error / knotif->flags, causing the supervised syscall to prematurely complete with return value 0 without installing the file descriptor.
2. Any subsequent retry of SECCOMP_IOCTL_NOTIF_SEND or SECCOMP_IOCTL_NOTIF_ADDFD by the supervisor fails with -EINPROGRESS (or -ENOENT after the target exits) because knotif->state is no longer SECCOMP_NOTIFY_SENT.
Restore knotif->state = SECCOMP_NOTIFY_SENT when removing an unhandled SECCOMP_ADDFD_FLAG_SEND request from knotif->addfd, and add a kselftest (user_notification_addfd_send_interrupted) to cover this race.
---
Changes in v2:
- Added kselftest regression test (user_notification_addfd_send_interrupted) in tools/testing/selftests/seccomp/seccomp_bpf.c per Kees Cook.
- Collected Reviewed-by tag from Bradley Morgan.
kernel/seccomp.c | 7 +- tools/testing/selftests/seccomp/seccomp_bpf.c | 98 +++++++++++++++++++ 2 files changed, 103 insertions(+), 2 deletions(-)
kernel/seccomp.c +5 -2
@@ -1808,10 +1808,13 @@ static long seccomp_notify_addfd(struct seccomp_filter *filter,* We need to check again if the addfd request has been handled, * and if not, we will remove it from the queue. */ -if (list_empty (&kaddfd.list)) +if (list_empty (&kaddfd.list)) { ret = kaddfd.ret; -else + }else {list_del (&kaddfd.list); +if (addfd.flags &SECCOMP_ADDFD_FLAG_SEND ) + knotif->state =SECCOMP_NOTIFY_SENT ; + } out_unlock:mutex_unlock (&filter->notify_lock);
Signature · 1 line
2.47.3
PATCH v3 1/1 seccomp: restore knotif->state when SECCOMP_ADDFD_FLAG_SEND is interrupted
In reply to Kees CookIn seccomp_notify_addfd(), when SECCOMP_ADDFD_FLAG_SEND is set in addfd.flags, knotif->state is optimistically transitioned from SECCOMP_NOTIFY_SENT to SECCOMP_NOTIFY_REPLIED before waking the tracee and waiting on kaddfd.completion:
if (addfd.flags & SECCOMP_ADDFD_FLAG_SEND) {
knotif->state = SECCOMP_NOTIFY_REPLIED;
...
}If wait_for_completion_interruptible(&kaddfd.completion) is interrupted by a signal before the tracee dequeues the kaddfd request (!list_empty(&kaddfd.list)), seccomp_notify_addfd() removes kaddfd from knotif->addfd and returns -EINTR to the supervisor without having installed the file descriptor.
However, knotif->state is left as SECCOMP_NOTIFY_REPLIED (with knotif->error == 0 and knotif->val == 0). This causes two problems:
1. When the tracee runs in do_user_notif(), it checks
if (knotif.state != SECCOMP_NOTIFY_REPLIED) after calling seccomp_handle_addfd(). Because knotif->state is already SECCOMP_NOTIFY_REPLIED, the tracee exits the notification wait loop prematurely and returns 0 from the trapped syscall without the file descriptor ever having been installed.
2. If the supervisor retries SECCOMP_IOCTL_NOTIF_ADDFD or SECCOMP_IOCTL_NOTIF_SEND before the tracee runs, it fails with -EINPROGRESS because knotif->state is no longer SECCOMP_NOTIFY_SENT.
Fix this by restoring knotif->state back to SECCOMP_NOTIFY_SENT if SECCOMP_ADDFD_FLAG_SEND was set and kaddfd was not consumed before the interrupted wait. Also add a seccomp_bpf selftest (user_notification_addfd_send_interrupted) covering this race.
---
Changes in v3:
- Actually include the tools/testing/selftests/seccomp/seccomp_bpf.c regression test in the patch diff (v2 accidentally omitted the selftest hunk), with detailed comments explaining the race and test setup.
Changes in v2:
- Add user_notification_addfd_send_interrupted regression test to tools/testing/selftests/seccomp/seccomp_bpf.c as requested by Kees Cook.
kernel/seccomp.c | 2 + tools/testing/selftests/seccomp/seccomp_bpf.c | 161 ++++++++++++++++++ 2 files changed, 163 insertions(+)
kernel/seccomp.c +5 -2
@@ -1808,10 +1808,13 @@ static long seccomp_notify_addfd(struct seccomp_filter *filter,* We need to check again if the addfd request has been handled, * and if not, we will remove it from the queue. */ -if (list_empty (&kaddfd.list)) +if (list_empty (&kaddfd.list)) { ret = kaddfd.ret; -else + }else {list_del (&kaddfd.list); +if (addfd.flags &SECCOMP_ADDFD_FLAG_SEND ) + knotif->state =SECCOMP_NOTIFY_SENT ; + } out_unlock:mutex_unlock (&filter->notify_lock);
tools/testing/selftests/seccomp/seccomp_bpf.c +167 -0
173 lines not on this page · raw message
Signature · 1 line
2.49.0
Quoted 6 lines · Kees Cook
Quoted 5 lines · Hui Peng
In
seccomp_notify_addfd(), whenSECCOMP_ADDFD_FLAG_SENDis set,knotif->stateis transitioned fromSECCOMP_NOTIFY_SENTtoSECCOMP_NOTIFY_REPLIEDbefore droppingfilter->notify_lockand waiting inwait_for_completion_interruptible(&kaddfd.completion).
Quoted 7 lines
Quoted 4 lines
If
wait_for_completion_interruptible()is interrupted by a signal before the target task processes the kaddfd entry,seccomp_notify_addfd()removes kaddfd.list fromknotif->addfdvialist_del(&kaddfd.list), but leavesknotif->stateset toSECCOMP_NOTIFY_REPLIED.Are you able to construct a regression test for this so we can catch this race if it ever gets exposed again?
I'd disregard this guy and treat this as, at best, an AI-generated bug report.
He sent 115 patches across 40 subsystems in 2 hours.
https://lore.kernel.org/all/?q=f%3Ahui+peng
Unfortunately I feel we are only at the beginning of this kind of thing :(
Quoted 4 lines
--
Kees Cook
--
Cheers, Lorenzo
Quoted 16 lines · Lorenzo Stoakes (ARM)
Quoted 7 lines · Kees Cook
Quoted 6 lines · Hui Peng
In
seccomp_notify_addfd(), whenSECCOMP_ADDFD_FLAG_SENDis set,knotif->stateis transitioned fromSECCOMP_NOTIFY_SENTtoSECCOMP_NOTIFY_REPLIEDbefore droppingfilter->notify_lockand waiting inwait_for_completion_interruptible(&kaddfd.completion).If
wait_for_completion_interruptible()is interrupted by a signalbefore
Quoted 6 lines
Quoted 3 lines
the target task processes the kaddfd entry,
seccomp_notify_addfd()removes kaddfd.list fromknotif->addfdvialist_del(&kaddfd.list), but leavesknotif->stateset toSECCOMP_NOTIFY_REPLIED.Are you able to construct a regression test for this so we can catch this race if it ever gets exposed again?
Quoted 8 lines
I'd disregard this guy and treat this as, at best, an AI-generated bug report.
He sent 115 patches across 40 subsystems in 2 hours.
https://lore.kernel.org/all/?q=f%3Ahui+peng
Unfortunately I feel we are only at the beginning of this kind of thing :(
Bruh, wasted my time on review. Mad..
I do believe it LGTM though, thanks Lorenzo for the tip.
Quoted 10 lines
Quoted 5 lines
--
Kees Cook--
Cheers, Lorenzo
--- Thanks!
"I'm not a very positive person" - Linus torvalds
Quoted 30 lines · Hui Peng
In
seccomp_notify_addfd(), whenSECCOMP_ADDFD_FLAG_SENDis set in addfd.flags,knotif->stateis optimistically transitioned fromSECCOMP_NOTIFY_SENTtoSECCOMP_NOTIFY_REPLIEDbefore waking the tracee and waiting on kaddfd.completion:if (addfd.flags & SECCOMP_ADDFD_FLAG_SEND) { knotif->state = SECCOMP_NOTIFY_REPLIED; ... }If
wait_for_completion_interruptible(&kaddfd.completion)is interrupted by a signal before the tracee dequeues the kaddfd request (!list_empty(&kaddfd.list)),seccomp_notify_addfd()removes kaddfd fromknotif->addfdand returns-EINTRto the supervisor without having installed the file descriptor.However,
knotif->stateis left asSECCOMP_NOTIFY_REPLIED(withknotif->error== 0 andknotif->val== 0). This causes two problems:1. When the tracee runs in
do_user_notif(), it checks
if (knotif.state !=SECCOMP_NOTIFY_REPLIED) after callingseccomp_handle_addfd(). Becauseknotif->stateis alreadySECCOMP_NOTIFY_REPLIED, the tracee exits the notification wait loop prematurely and returns 0 from the trapped syscall without the file descriptor ever having been installed.2. If the supervisor retries
SECCOMP_IOCTL_NOTIF_ADDFDorSECCOMP_IOCTL_NOTIF_SENDbefore the tracee runs, it fails with-EINPROGRESSbecauseknotif->stateis no longerSECCOMP_NOTIFY_SENT.
Quoted 6 lines
Fix this by restoring
knotif->stateback toSECCOMP_NOTIFY_SENTifSECCOMP_ADDFD_FLAG_SENDwas set and kaddfd was not consumed before the interrupted wait. Also add aseccomp_bpfselftest (user_notification_addfd_send_interrupted) covering this race.Fixes 0ae71c1990f3 ("seccomp: Add a flag to atomically addfd and perform send")
Oh my godd you
1: removed me from CC
2: removed my R-B tag!
What is your AI doing!
NAK. I can't review something like this!
Add a assisted by tag and CC me and I'll look at it
Quoted 204 lines
Signed-off-by Hui Peng <[email protected]>---
Changes in v3:
- Actually include thetools/testing/selftests/seccomp/seccomp_bpf.cregression test in the patch diff (v2 accidentally omitted the selftest hunk), with detailed comments explaining the race and test setup.Changes in v2:
- Adduser_notification_addfd_send_interruptedregression test totools/testing/selftests/seccomp/seccomp_bpf.cas requested by Kees Cook.
kernel/seccomp.c| 2 +tools/testing/selftests/seccomp/seccomp_bpf.c| 161 ++++++++++++++++++2 files changed, 163 insertions(+)kernel/seccomp.c +5
-2@@ -1808,10 +1808,13 @@ static long seccomp_notify_addfd(struct seccomp_filter *filter,* We need to check again if the addfd request has been handled, * and if not, we will remove it from the queue. */ -if (list_empty (&kaddfd.list)) +if (list_empty (&kaddfd.list)) { ret = kaddfd.ret; -else + }else {list_del (&kaddfd.list); +if (addfd.flags &SECCOMP_ADDFD_FLAG_SEND ) + knotif->state =SECCOMP_NOTIFY_SENT ; + }
out_unlock:mutex_unlock(&filter->notify_lock);tools/testing/selftests/seccomp/seccomp_bpf.c +160
-0@@ -4368,6 +4368,173 @@ TEST(user_notification_addfd_rlimit)close (memfd); } +static void sigusr1_handler (int signo) +{ +} + +/* +* Verify that when SECCOMP_IOCTL_NOTIF_ADDFD with SECCOMP_ADDFD_FLAG_SEND is +* interrupted by a signal before the tracee dequeues the addfd request, +* knotif->state is restored from SECCOMP_NOTIFY_REPLIED back to +* SECCOMP_NOTIFY_SENT so that: +* 1. The woken tracee sees knotif->state == SECCOMP_NOTIFY_SENT in +* do_user_notif() and goes back to sleep instead of prematurely +* returning 0 from the trapped syscall without the FD installed. +* 2. The supervisor can retry SECCOMP_IOCTL_NOTIF_ADDFD (or +* SECCOMP_IOCTL_NOTIF_SEND) instead of failing with -EINPROGRESS. +* +* To deterministically hit the race window where the supervisor sleeps in +* wait_for_completion_interruptible(&kaddfd.completion) after waking the +* tracee (complete(&knotif->ready)) but before the tracee runs +* seccomp_handle_addfd(), pin all processes to a single CPU and enforce a +* strict 3-tier scheduling priority hierarchy on that CPU: +* - Supervisor: SCHED_FIFO priority 99 (highest) +* - Signal helper (sig_pid): SCHED_FIFO priority 50 (middle) +* - Tracee (pid): SCHED_IDLE (lowest) +*/ +TEST (user_notification_addfd_send_interrupted) +{ +/* +* Save parent_pid before user_notif_syscall(__NR_getppid, ...) installs +* the seccomp filter on the calling process; children inherit that +* filter, so sig_pid must not call getppid(). +*/ +pid_t pid, sig_pid, parent_pid =getpid (); +long ret; +int status, listener, memfd; +struct seccomp_notif_addfd addfd = {}; +struct seccomp_notif req = {}; +struct sigaction sa = {}; +struct sched_param sp_tracee_idle = { .sched_priority =0 }; +struct sched_param sp_supervisor_fifo = { .sched_priority =99 }; +struct sched_param sp_sig_helper_fifo = { .sched_priority =50 }; +struct timespec delay = { .tv_nsec =15000000 }; +cpu_set_t cpuset; +int cpu; + +/* Pin the supervisor (and its future child processes) to one CPU. */ + cpu =sched_getcpu (); +if (cpu >=0 ) { +CPU_ZERO (&cpuset); +CPU_SET (cpu, &cpuset); +sched_setaffinity (0 ,sizeof (cpuset), &cpuset); + } + + sa.sa_handler = sigusr1_handler; +ASSERT_EQ (sigaction (SIGUSR1 , &sa,NULL ),0 ); + + memfd =memfd_create ("test" ,0 ); +ASSERT_GE (memfd,0 ); + + ret =prctl (PR_SET_NO_NEW_PRIVS ,1 ,0 ,0 ,0 ); + ASSERT_EQ(0, ret) { +TH_LOG ("Kernel does not support PR_SET_NO_NEW_PRIVS!" ); + } + +/* +* Follow the convention of other user_notification_* tests in this +* file by trapping __NR_getppid: because the filter is installed on +* the supervisor before fork(), the trapped syscall must be a +* side-effect-free syscall that the supervisor itself never invokes. +* Even though getppid() does not normally return an FD, +* SECCOMP_ADDFD_FLAG_SEND replaces the trapped syscall's return value +* with the newly installed FD number (42). +*/ + listener =user_notif_syscall (__NR_getppid, +SECCOMP_FILTER_FLAG_NEW_LISTENER ); +ASSERT_GE (listener,0 ); + + pid =fork (); +ASSERT_GE (pid,0 ); + +if (pid ==0 ) { +/* +* Tracee: invoke __NR_getppid as a dummy trigger syscall to +* trap into do_user_notif(). Verify that the syscall returns +* the injected FD number (42) and that FD 42 is open. +*/ + ret =syscall (__NR_getppid); +exit (ret !=42 ||fcntl (42 ,F_GETFD ) <0 ); + } + +/* Wait for the tracee to trap in do_user_notif(). */ +ASSERT_EQ (ioctl (listener,SECCOMP_IOCTL_NOTIF_RECV , &req),0 ); + +/* +* Demote the tracee to SCHED_IDLE and promote the supervisor to +* SCHED_FIFO(99) on the same CPU. +*/ +sched_setscheduler (pid,SCHED_IDLE , &sp_tracee_idle); +if (sched_setscheduler (0 ,SCHED_FIFO , &sp_supervisor_fifo) !=0 ) { +close (listener); +close (memfd); +kill (pid,SIGKILL ); +waitpid (pid,NULL ,0 ); +SKIP (return,"SCHED_FIFO requires CAP_SYS_NICE" ); + } + + addfd.srcfd = memfd; + addfd.newfd_flags =O_CLOEXEC ; + addfd.newfd =42 ; + addfd.id = req.id; + addfd.flags =SECCOMP_ADDFD_FLAG_SETFD |SECCOMP_ADDFD_FLAG_SEND ; + +/* +* Fork a signal helper on the same CPU and set it to SCHED_FIFO(50). +* Because the supervisor is currently running at SCHED_FIFO(99) on +* this CPU, sig_pid is queued on the runqueue but cannot run until the +* supervisor blocks inside the kernel. +* +* When the supervisor invokes ioctl(SECCOMP_IOCTL_NOTIF_ADDFD) below: +* 1. seccomp_notify_addfd() sets knotif->state = SECCOMP_NOTIFY_REPLIED, +* wakes the tracee (SCHED_IDLE), and blocks in +* wait_for_completion_interruptible(&kaddfd.completion). +* 2. The CPU scheduler immediately runs sig_pid (SCHED_FIFO 50) +* ahead of the woken tracee (SCHED_IDLE). +* 3. sig_pid sends SIGUSR1 to parent_pid, waking the supervisor +* (SCHED_FIFO 99), which immediately preempts sig_pid, aborts the +* wait with -ERESTARTSYS (-EINTR), removes kaddfd from +* knotif->addfd, and restores knotif->state = SECCOMP_NOTIFY_SENT +* before the tracee has executed a single instruction. +*/ + sig_pid =fork (); +ASSERT_GE (sig_pid,0 ); +if (sig_pid ==0 ) { +sched_setscheduler (0 ,SCHED_FIFO , &sp_sig_helper_fifo); +kill (parent_pid,SIGUSR1 ); +_exit (0 ); + } + +EXPECT_EQ (ioctl (listener,SECCOMP_IOCTL_NOTIF_ADDFD , &addfd),-1 ); +EXPECT_EQ (errno,EINTR ); +EXPECT_EQ (waitpid (sig_pid, &status,0 ), sig_pid); + +/* +* Restore normal scheduling and sleep briefly so the woken tracee +* runs in do_user_notif(). With knotif->state restored to +* SECCOMP_NOTIFY_SENT, the tracee must loop back to sleep waiting for +* the notification reply rather than returning 0 from __NR_getppid. +*/ +sched_setscheduler (0 ,SCHED_OTHER , &sp_tracee_idle); +sched_setscheduler (pid,SCHED_OTHER , &sp_tracee_idle); +nanosleep (&delay,NULL ); + +/* +* Retry SECCOMP_IOCTL_NOTIF_ADDFD. Because knotif->state is +* SECCOMP_NOTIFY_SENT, the retry succeeds (returns 42) instead of +* failing with -EINPROGRESS, installs FD 42 into the tracee, and wakes +* the tracee to complete the syscall with return value 42. +*/ +EXPECT_EQ (ioctl (listener,SECCOMP_IOCTL_NOTIF_ADDFD , &addfd),42 ); + +EXPECT_EQ (waitpid (pid, &status,0 ), pid);
tools/testing/selftests/seccomp/seccomp_bpf.c TEST(user_notification_addfd_rlimit) · line 4540
Quoted 10 lines
tools/testing/selftests/seccomp/seccomp_bpf.c +7
-0+EXPECT_EQ (true,WIFEXITED (status)); +EXPECT_EQ (0 ,WEXITSTATUS (status)); + +close (listener); +close (memfd); +} +#ifndef SECCOMP_USER_NOTIF_FD_SYNC_WAKE_UP #define SECCOMP_USER_NOTIF_FD_SYNC_WAKE_UP (1UL << 0)#define SECCOMP_IOCTL_NOTIF_SET_FLAGS SECCOMP_IOW(4, __u64)
--- Thanks!
"I'm not a very positive person" - Linus torvalds
PATCH v4 1/1 seccomp: restore knotif->state when SECCOMP_ADDFD_FLAG_SEND is interrupted
In seccomp_notify_addfd(), when SECCOMP_ADDFD_FLAG_SEND is set in addfd.flags, knotif->state is optimistically transitioned from SECCOMP_NOTIFY_SENT to SECCOMP_NOTIFY_REPLIED before waking the tracee and waiting on kaddfd.completion:
if (addfd.flags & SECCOMP_ADDFD_FLAG_SEND) {
knotif->state = SECCOMP_NOTIFY_REPLIED;
...
}If wait_for_completion_interruptible(&kaddfd.completion) is interrupted by a signal before the tracee dequeues the kaddfd request (!list_empty(&kaddfd.list)), seccomp_notify_addfd() removes kaddfd from knotif->addfd and returns -EINTR to the supervisor without having installed the file descriptor.
However, knotif->state is left as SECCOMP_NOTIFY_REPLIED (with knotif->error == 0 and knotif->val == 0). This causes two problems:
1. When the tracee runs in do_user_notif(), it checks
if (knotif.state != SECCOMP_NOTIFY_REPLIED) after calling seccomp_handle_addfd(). Because knotif->state is already SECCOMP_NOTIFY_REPLIED, the tracee exits the notification wait loop prematurely and returns 0 from the trapped syscall without the file descriptor ever having been installed.
2. If the supervisor retries SECCOMP_IOCTL_NOTIF_ADDFD or SECCOMP_IOCTL_NOTIF_SEND before the tracee runs, it fails with -EINPROGRESS because knotif->state is no longer SECCOMP_NOTIFY_SENT.
Fix this by restoring knotif->state back to SECCOMP_NOTIFY_SENT if SECCOMP_ADDFD_FLAG_SEND was set and kaddfd was not consumed before the interrupted wait. Also add a seccomp_bpf selftest (user_notification_addfd_send_interrupted) covering this race.
Tested in QEMU against Linux 7.3.0-rc3 by exercising kernel/seccomp.c and verifying the fix with KASAN enabled.
---
Changes in v4:
- Restore Reviewed-by: Bradley Morgan <[email protected]> and Cc:
[email protected] tags from v1 (and restore Bradley Morgan to Cc), and add Assisted-by: LLM tag.
- Fix Fixes: tag SHA and subject (0ae71c7720e3 ("seccomp: Support atomic "addfd + send reply"")).
- Fix truncated word "interrupted" in the Subject line.
Changes in v3:
- Actually include the tools/testing/selftests/seccomp/seccomp_bpf.c regression test in the patch diff (v2 accidentally omitted the selftest hunk), with detailed comments explaining the race and test setup.
Changes in v2:
- Add user_notification_addfd_send_interrupted regression test to tools/testing/selftests/seccomp/seccomp_bpf.c as requested by Kees Cook.
kernel/seccomp.c | 2 + tools/testing/selftests/seccomp/seccomp_bpf.c | 161 ++++++++++++++++++ 2 files changed, 163 insertions(+)
kernel/seccomp.c +5 -2
@@ -1808,10 +1808,13 @@ static long seccomp_notify_addfd(struct seccomp_filter *filter,* We need to check again if the addfd request has been handled, * and if not, we will remove it from the queue. */ -if (list_empty (&kaddfd.list)) +if (list_empty (&kaddfd.list)) { ret = kaddfd.ret; -else + }else {list_del (&kaddfd.list); +if (addfd.flags &SECCOMP_ADDFD_FLAG_SEND ) + knotif->state =SECCOMP_NOTIFY_SENT ; + } out_unlock:mutex_unlock (&filter->notify_lock);
tools/testing/selftests/seccomp/seccomp_bpf.c +167 -0
173 lines not on this page · raw message
Signature · 1 line
2.49.0
Kees Cook
maintainerQuoted 4 lines · Hui Peng
Fix this by restoring
knotif->stateback toSECCOMP_NOTIFY_SENTifSECCOMP_ADDFD_FLAG_SENDwas set and kaddfd was not consumed before the interrupted wait. Also add aseccomp_bpfselftest (user_notification_addfd_send_interrupted) covering this race.
Yeah, this fix is the same restoration logic as seccomp_notify_recv() uses (although that may actually need "if (state == SECCOMP_NOTIFY_SENT) ..." added). It does make it clear there is a missing state in the state machine. SECCOMP_NOTIFY_REPLIES means both "A reply is reserved" and "A reply has been delivered". But the hidden state is maintained:
"REPLIED + queued ADDFD+SEND" == in flight, and "REPLIED + no queued addfd" == delivered.
Thank you for the selftest addition, the sched trick is nice!
Quoted 2 lines
Tested in QEMU against Linux 7.3.0-rc3 by exercising
kernel/seccomp.cand verifying the fix with KASAN enabled.
I've verified this now too. It's kind of a ugly problem because unlucky timing makes it look like the returned fd is fd 0. :(
tools/testing/selftests/seccomp/seccomp_bpf.c TEST(user_notification_addfd_rlimit) · line 4467
Quoted 6 lines
[...]
tools/testing/selftests/seccomp/seccomp_bpf.c +5
-0+/* +* Demote the tracee to SCHED_IDLE and promote the supervisor to +* SCHED_FIFO(99) on the same CPU. +*/ +sched_setscheduler (pid,SCHED_IDLE , &sp_tracee_idle);
Unchecked return value?
tools/testing/selftests/seccomp/seccomp_bpf.c TEST(user_notification_addfd_rlimit) · line 4474
Quoted 7 lines
tools/testing/selftests/seccomp/seccomp_bpf.c +7
-0+if (sched_setscheduler (0 ,SCHED_FIFO , &sp_supervisor_fifo) !=0 ) { +close (listener); +close (memfd); +kill (pid,SIGKILL ); +waitpid (pid,NULL ,0 ); +SKIP (return,"SCHED_FIFO requires CAP_SYS_NICE" ); + }
Instead of this you may want to look at FIXTURE_TEARDOWN to clean up (though it's not strictly needed since the harness is run in a subprocess so all these go away on test exit). The one thing that might be worth doing is making sure sched_setscheduler(0, SCHED_OTHER, ...) happens ASAP or the test could block everything on a single CPU machine/VM/CI.
tools/testing/selftests/seccomp/seccomp_bpf.c TEST(user_notification_addfd_rlimit) · line 4503
Quoted 5 lines
[...]
tools/testing/selftests/seccomp/seccomp_bpf.c +4
-0+ sig_pid =fork (); +ASSERT_GE (sig_pid,0 ); +if (sig_pid ==0 ) { +sched_setscheduler (0 ,SCHED_FIFO , &sp_sig_helper_fifo);
Missed return value check here too.
Quoted 6 lines
tools/testing/selftests/seccomp/seccomp_bpf.c +6
-0+kill (parent_pid,SIGUSR1 ); +_exit (0 ); + } + +EXPECT_EQ (ioctl (listener,SECCOMP_IOCTL_NOTIF_ADDFD , &addfd),-1 ); +EXPECT_EQ (errno,EINTR );
tools/testing/selftests/seccomp/seccomp_bpf.c TEST(user_notification_addfd_rlimit) · line 4519
Quoted 10 lines
tools/testing/selftests/seccomp/seccomp_bpf.c +10
-0+EXPECT_EQ (waitpid (sig_pid, &status,0 ), sig_pid); + +/* +* Restore normal scheduling and sleep briefly so the woken tracee +* runs in do_user_notif(). With knotif->state restored to +* SECCOMP_NOTIFY_SENT, the tracee must loop back to sleep waiting for +* the notification reply rather than returning 0 from __NR_getppid. +*/ +sched_setscheduler (0 ,SCHED_OTHER , &sp_tracee_idle); +sched_setscheduler (pid,SCHED_OTHER , &sp_tracee_idle);
Need to check these too...
Quoted 23 lines
tools/testing/selftests/seccomp/seccomp_bpf.c +18
-0+nanosleep (&delay,NULL ); + +/* +* Retry SECCOMP_IOCTL_NOTIF_ADDFD. Because knotif->state is +* SECCOMP_NOTIFY_SENT, the retry succeeds (returns 42) instead of +* failing with -EINPROGRESS, installs FD 42 into the tracee, and wakes +* the tracee to complete the syscall with return value 42. +*/ +EXPECT_EQ (ioctl (listener,SECCOMP_IOCTL_NOTIF_ADDFD , &addfd),42 ); + +EXPECT_EQ (waitpid (pid, &status,0 ), pid); +EXPECT_EQ (true,WIFEXITED (status)); +EXPECT_EQ (0 ,WEXITSTATUS (status)); + +close (listener); +close (memfd); +} +#ifndef SECCOMP_USER_NOTIF_FD_SYNC_WAKE_UP #define SECCOMP_USER_NOTIF_FD_SYNC_WAKE_UP (1UL << 0)#define SECCOMP_IOCTL_NOTIF_SET_FLAGS SECCOMP_IOW(4, __u64)Signature · 1 line
2.49.0
Thank you for the test, it really helps see the shape of the issue.
-Kees
Signature · 1 line
Kees Cook
PATCH v5 1/1 seccomp: restore knotif->state when SECCOMP_ADDFD_FLAG_SEND is interrupted
In seccomp_notify_addfd(), when SECCOMP_ADDFD_FLAG_SEND is set in addfd.flags, knotif->state is optimistically transitioned from SECCOMP_NOTIFY_SENT to SECCOMP_NOTIFY_REPLIED before waking the tracee and waiting on kaddfd.completion:
if (addfd.flags & SECCOMP_ADDFD_FLAG_SEND) {
knotif->state = SECCOMP_NOTIFY_REPLIED;
...
}If wait_for_completion_interruptible(&kaddfd.completion) is interrupted by a signal before the tracee dequeues the kaddfd request (!list_empty(&kaddfd.list)), seccomp_notify_addfd() removes kaddfd from knotif->addfd and returns -EINTR to the supervisor without having installed the file descriptor.
However, knotif->state is left as SECCOMP_NOTIFY_REPLIED (with knotif->error == 0 and knotif->val == 0). This causes two problems:
1. When the tracee runs in do_user_notif(), it checks
if (knotif.state != SECCOMP_NOTIFY_REPLIED) after calling seccomp_handle_addfd(). Because knotif->state is already SECCOMP_NOTIFY_REPLIED, the tracee exits the notification wait loop prematurely and returns 0 from the trapped syscall without the file descriptor ever having been installed.
2. If the supervisor retries SECCOMP_IOCTL_NOTIF_ADDFD or SECCOMP_IOCTL_NOTIF_SEND before the tracee runs, it fails with -EINPROGRESS because knotif->state is no longer SECCOMP_NOTIFY_SENT.
Fix this by restoring knotif->state back to SECCOMP_NOTIFY_SENT if SECCOMP_ADDFD_FLAG_SEND was set and kaddfd was not consumed before the interrupted wait. Also add a seccomp_bpf selftest (user_notification_addfd_send_interrupted) covering this race.
Tested in QEMU against Linux 7.3.0-rc3 using the user_notification_addfd_send_interrupted selftest added in this patch (./seccomp_bpf -t user_notification_addfd_send_interrupted): on the unfixed kernel the test fails because the retry ioctl(listener, SECCOMP_IOCTL_NOTIF_ADDFD, &addfd) returns -EINPROGRESS (-1) and the tracee prematurely exits do_user_notif() with return value 0 instead of FD 42, whereas with this patch applied the test passes.
---
Changes in v5:
- Check the return values of all sched_setscheduler() calls in user_notification_addfd_send_interrupted, as requested by Kees Cook.
- Restore SCHED_OTHER on the supervisor and tracee immediately after ioctl(SECCOMP_IOCTL_NOTIF_ADDFD) returns (and on fork() failure) so the test cannot remain SCHED_FIFO on a single-CPU VM/CI runner, as suggested by Kees Cook.
- Rename pid and struct sched_param variables (tracee_pid, sp_zero, sp_fifo_high, sp_fifo_mid) for clarity.
Changes in v4:
- Restore Reviewed-by: Bradley Morgan <[email protected]> and Cc:
[email protected] tags from v1 (and restore Bradley Morgan to Cc), and add Assisted-by: LLM tag.
- Fix Fixes: tag SHA and subject (0ae71c7720e3 ("seccomp: Support atomic "addfd + send reply"")).
- Fix truncated word "interrupted" in the Subject line.
Changes in v3:
- Actually include the tools/testing/selftests/seccomp/seccomp_bpf.c regression test in the patch diff (v2 accidentally omitted the selftest hunk), with detailed comments explaining the race and test setup.
Changes in v2:
- Add user_notification_addfd_send_interrupted regression test to tools/testing/selftests/seccomp/seccomp_bpf.c as requested by Kees Cook.
kernel/seccomp.c | 5 +- tools/testing/selftests/seccomp/seccomp_bpf.c | 184 ++++++++++++++++++ 2 files changed, 187 insertions(+), 2 deletions(-)
kernel/seccomp.c +5 -2
@@ -1808,10 +1808,13 @@ static long seccomp_notify_addfd(struct seccomp_filter *filter,* We need to check again if the addfd request has been handled, * and if not, we will remove it from the queue. */ -if (list_empty (&kaddfd.list)) +if (list_empty (&kaddfd.list)) { ret = kaddfd.ret; -else + }else {list_del (&kaddfd.list); +if (addfd.flags &SECCOMP_ADDFD_FLAG_SEND ) + knotif->state =SECCOMP_NOTIFY_SENT ; + } out_unlock:mutex_unlock (&filter->notify_lock);
tools/testing/selftests/seccomp/seccomp_bpf.c +178 -0
184 lines not on this page · raw message
Signature · 1 line
2.49.0
Kees Cook
maintainerQuoted 11 lines · Hui Peng
In
seccomp_notify_addfd(), whenSECCOMP_ADDFD_FLAG_SENDis set in addfd.flags,knotif->stateis optimistically transitioned fromSECCOMP_NOTIFY_SENTtoSECCOMP_NOTIFY_REPLIEDbefore waking the tracee and waiting on kaddfd.completion:if (addfd.flags & SECCOMP_ADDFD_FLAG_SEND) { knotif->state = SECCOMP_NOTIFY_REPLIED; ... }[...]
I renamed the empty signal handler and added a comment, but otherwise, applied to for-next/seccomp, thanks!
[1/1] seccomp: restore knotif->state when SECCOMP_ADDFD_FLAG_SEND is interrupted
https://git.kernel.org/kees/c/832b9b176be0
Take care,
Signature · 1 line
Kees Cook