Add SCM_PIDFD and SO_PEERPIDFD
Subsystems (12)
v4 cover Add SCM_PIDFD and SO_PEERPIDFD
1. Implement SCM_PIDFD, a new type of CMSG type analogical to SCM_CREDENTIALS, but it contains pidfd instead of plain pid, which allows programmers not to care about PID reuse problem.
2. Add SO_PEERPIDFD which allows to get pidfd of peer socket holder pidfd. This thing is direct analog of SO_PEERCRED which allows to get plain PID.
3. Add SCM_PIDFD / SO_PEERPIDFD kselftest
Idea comes from UAPI kernel group:
https://uapi-group.org/kernel-features/
Big thanks to Christian Brauner and Lennart Poettering for productive discussions about this and Luca Boccassi for testing and reviewing this.
=== Motivation behind this patchset
Eric Dumazet raised a question:
Quoted 2 lines
It seems that we already can use
pidfd_open()(since linux-5.3), and pass the resulting fd inaf_unixSCM_RIGHTSmessage ?
Yes, it's possible, but it means that from the receiver side we need to trust the sent pidfd (in SCM_RIGHTS),
or always use combination of SCM_RIGHTS+SCM_CREDENTIALS, then we can extract pidfd from SCM_RIGHTS,
then acquire plain pid from pidfd and after compare it with the pid from SCM_CREDENTIALS.
A few comments from other folks regarding this.
Quoted 25 lines · Christian Brauner
Let me try and provide some of the missing background.
There are a range of use-cases where we would like to authenticate a client through sockets without being susceptible to PID recycling attacks. Currently, we can't do this as the race isn't fully fixable. We can only apply mitigations.
What this patchset will allows us to do is to get a pidfd without the client having to send us an fd explicitly via
SCM_RIGHTS. As that's already possibly as you correctly point out.But for protocols like polkit this is quite important. Every message is standalone and we would need to force a complete protocol change where we would need to require that every client allocate and send a pidfd via
SCM_RIGHTS. That would also mean patching through all polkit users.For something like systemd-journald where we provide logging facilities and want to add metadata to the log we would also immensely benefit from being able to get a receiver-side controlled pidfd.
With the message type we envisioned we don't need to change the sender at all and can be safe against pid recycling.
Quoted 28 lines · Lennart Poettering
So yes, this is of course possible, but it would mean the pidfd would have to be transported as part of the user protocol, explicitly sent by the sender. (Moreover, the receiver after receiving the pidfd would then still have to somehow be able to prove that the pidfd it just received actually refers to the peer's process and not some random process. – this part is actually solvable in userspace, but ugly)
The big thing is simply that we want that the pidfd is associated *implicity* with each
AF_UNIXconnection, not explicitly. A lot of userspace already relies on this, both in the authentication area (polkit) as well as in the logging area (systemd-journald). Right now using the PID field from SO_PEERCREDS/SCM_CREDENTIALS is racy though and very hard to get right. Making this available as pidfd too, would solve this raciness, without otherwise changing semantics of it all:
receivers can still enable the creds stuff as they wish, and the data is then implicitly appended to the connections/datagrams the sender initiates.Or to turn this around: things like polkit are typically used to authenticate arbitrary dbus methods calls: some service implements a dbus method call, and when an unprivileged client then issues that call, it will take the client's info, go to polkit and ask it if this is ok. If we wanted to send the pidfd as part of the protocol we basically would have to extend every single method call to contain the client's pidfd along with it as an additional argument, which would be a massive undertaking: it would change the prototypes of basically *all* methods a service defines… And that's just ugly.
Quoted 6 lines
Note that Alex' patch set doesn't expose anything that wasn't exposed before, or attach, propagate what wasn't before. All it does, is make the field already available anyway (the struct ucred .pid field) available also in a better way (as a pidfd), to solve a variety of races, with no effect on the protocol actually spoken within the
AF_UNIXtransport. It's a seamless improvement of the status quo.
===
This patch series is on top of net-next tree with pidfd.file.api.v6.4 tag (from git://git.kernel.org/pub/scm/linux/kernel/git/brauner/linux.git) merged in.
Git tree:
https://github.com/mihalicyn/linux/tree/scm_pidfd
Alexander Mikhalitsyn (4):
scm: add SO_PASSPIDFD and SCM_PIDFD
net: socket: add sockopts blacklist for BPF cgroup hook
net: core: add getsockopt SO_PEERPIDFD
selftests: net: add SCM_PIDFD / SO_PEERPIDFD test
arch/alpha/include/uapi/asm/socket.h | 3 + arch/mips/include/uapi/asm/socket.h | 3 + arch/parisc/include/uapi/asm/socket.h | 3 + arch/sparc/include/uapi/asm/socket.h | 3 + include/linux/net.h | 1 + include/linux/socket.h | 1 + include/net/scm.h | 39 +- include/uapi/asm-generic/socket.h | 3 + net/core/sock.c | 44 ++ net/mptcp/sockopt.c | 1 + net/socket.c | 45 +- net/unix/af_unix.c | 18 +- tools/include/uapi/asm-generic/socket.h | 3 + tools/testing/selftests/net/.gitignore | 1 + tools/testing/selftests/net/af_unix/Makefile | 2 +- .../testing/selftests/net/af_unix/scm_pidfd.c | 430 ++++++++++++++++++ 16 files changed, 589 insertions(+), 11 deletions(-) create mode 100644 tools/testing/selftests/net/af_unix/scm_pidfd.c
Signature · 1 line
2.34.1
PATCH v4 1/4 scm: add SO_PASSPIDFD and SCM_PIDFD
Implement SCM_PIDFD, a new type of CMSG type analogical to SCM_CREDENTIALS, but it contains pidfd instead of plain pid, which allows programmers not to care about PID reuse problem.
Idea comes from UAPI kernel group:
https://uapi-group.org/kernel-features/
Big thanks to Christian Brauner and Lennart Poettering for productive discussions about this.
---
v4:
- fixed silent fd_install if writting of CMSG to the userspace fails (pointed by Christian)
v2:
According to review comments from Kuniyuki Iwashima and Christian Brauner:
- use pidfd_create(..) retval as a result
- whitespace change
--- arch/alpha/include/uapi/asm/socket.h | 2 ++ arch/mips/include/uapi/asm/socket.h | 2 ++ arch/parisc/include/uapi/asm/socket.h | 2 ++ arch/sparc/include/uapi/asm/socket.h | 2 ++ include/linux/net.h | 1 + include/linux/socket.h | 1 + include/net/scm.h | 39 +++++++++++++++++++++++-- include/uapi/asm-generic/socket.h | 2 ++ net/core/sock.c | 11 +++++++ net/mptcp/sockopt.c | 1 + net/unix/af_unix.c | 18 ++++++++---- tools/include/uapi/asm-generic/socket.h | 2 ++ 12 files changed, 76 insertions(+), 7 deletions(-)
arch/alpha/include/uapi/asm/socket.h +2 -0
@@ -137,6 +137,8 @@#define SO_RCVMARK 75 +#define SO_PASSPIDFD 76 +#if !defined(__KERNEL__)#if __BITS_PER_LONG ==64
arch/mips/include/uapi/asm/socket.h +2 -0
@@ -148,6 +148,8 @@#define SO_RCVMARK 75 +#define SO_PASSPIDFD 76 +#if !defined(__KERNEL__)#if __BITS_PER_LONG ==64
arch/parisc/include/uapi/asm/socket.h +2 -0
@@ -129,6 +129,8 @@#define SO_RCVMARK 0x4049 +#define SO_PASSPIDFD 0x404A +#if !defined(__KERNEL__)#if __BITS_PER_LONG ==64
arch/sparc/include/uapi/asm/socket.h +2 -0
@@ -130,6 +130,8 @@#define SO_RCVMARK 0x0054 +#define SO_PASSPIDFD 0x0055 +#if !defined(__KERNEL__)
include/linux/net.h +1 -0
@@ -43,6 +43,7 @@ struct net;#define SOCK_PASSSEC 4#define SOCK_SUPPORT_ZC 5#define SOCK_CUSTOM_SOCKOPT 6 +#define SOCK_PASSPIDFD 7#ifndef ARCH_HAS_SOCKET_TYPES /**
include/linux/socket.h +1 -0
@@ -177,6 +177,7 @@ static inline size_t msg_data_left(struct msghdr *msg)#define SCM_RIGHTS 0x01/* rw: access rights (array of int) */ #define SCM_CREDENTIALS 0x02/* rw: struct ucred */ #define SCM_SECURITY 0x03/* rw: security label */ +#define SCM_PIDFD 0x04/* ro: pidfd (int) */ struct ucred {__u32 pid;
include/net/scm.h +37 -2
@@ -120,12 +120,44 @@ static inline bool scm_has_secdata(struct socket *sock) }#endif /* CONFIG_SECURITY_NETWORK */ +static __inline__void scm_pidfd_recv (struct msghdr *msg,struct scm_cookie *scm) +{ +struct file *pidfd_file =NULL ; +int pidfd; + +/* +* put_cmsg() doesn't return an error if CMSG is truncated, +* that's why we need to opencode these checks here. +*/ +if ((msg->msg_controllen <=sizeof (struct cmsghdr )) || + (msg->msg_controllen -sizeof (struct cmsghdr )) <sizeof (int )) { + msg->msg_flags |=MSG_CTRUNC ; +return ; + } + +WARN_ON_ONCE (!scm->pid); + pidfd =pidfd_prepare (scm->pid,0 , &pidfd_file); + +if (put_cmsg (msg,SOL_SOCKET ,SCM_PIDFD ,sizeof (int ), &pidfd)) { +if (pidfd_file) { +put_unused_fd (pidfd); +fput (pidfd_file); + } + +return ; + } + +if (pidfd_file) +fd_install (pidfd, pidfd_file); +} +static __inline__void scm_recv (struct socket *sock,struct msghdr *msg,struct scm_cookie *scm,int flags) {if (!msg->msg_control) { -if (test_bit (SOCK_PASSCRED , &sock->flags) || scm->fp || -scm_has_secdata (sock)) +if (test_bit (SOCK_PASSCRED , &sock->flags) || +test_bit (SOCK_PASSPIDFD , &sock->flags) || + scm->fp ||scm_has_secdata (sock)) msg->msg_flags |=MSG_CTRUNC ;scm_destroy (scm);return ;
@@ -141,6 +173,9 @@ static __inline__ void scm_recv(struct socket *sock, struct msghdr *msg,put_cmsg (msg,SOL_SOCKET ,SCM_CREDENTIALS ,sizeof (ucreds), &ucreds); } +if (test_bit (SOCK_PASSPIDFD , &sock->flags)) +scm_pidfd_recv (msg, scm); +scm_destroy_cred (scm);scm_passec (sock, msg, scm);
include/uapi/asm-generic/socket.h +2 -0
@@ -132,6 +132,8 @@#define SO_RCVMARK 75 +#define SO_PASSPIDFD 76 +#if !defined(__KERNEL__)#if __BITS_PER_LONG ==64 || (defined(__x86_64__) && defined(__ILP32__))
net/core/sock.c +11 -0
@@ -1246,6 +1246,13 @@ int sk_setsockopt(struct sock *sk, int level, int optname,clear_bit (SOCK_PASSCRED , &sock->flags);break ; +case SO_PASSPIDFD : +if (valbool) +set_bit (SOCK_PASSPIDFD , &sock->flags); +else +clear_bit (SOCK_PASSPIDFD , &sock->flags); +break ; +case SO_TIMESTAMP_OLD :case SO_TIMESTAMP_NEW :case SO_TIMESTAMPNS_OLD :
@@ -1737,6 +1744,10 @@ int sk_getsockopt(struct sock *sk, int level, int optname, v.val = !!test_bit (SOCK_PASSCRED , &sock->flags);break ; +case SO_PASSPIDFD : + v.val = !!test_bit (SOCK_PASSPIDFD , &sock->flags); +break ; +case SO_PEERCRED : {struct ucred peercred;
net/mptcp/sockopt.c +1 -0
@@ -355,6 +355,7 @@ static int mptcp_setsockopt_sol_socket(struct mptcp_sock *msk, int optname,case SO_BROADCAST :case SO_BSDCOMPAT :case SO_PASSCRED : +case SO_PASSPIDFD :case SO_PASSSEC :case SO_RXQ_OVFL :case SO_WIFI_STATUS :
net/unix/af_unix.c +13 -5
@@ -1361,7 +1361,8 @@ static int unix_dgram_connect(struct socket *sock, struct sockaddr *addr,if (err) goto out; -if (test_bit (SOCK_PASSCRED , &sock->flags) && +if ((test_bit (SOCK_PASSCRED , &sock->flags) || +test_bit (SOCK_PASSPIDFD , &sock->flags)) && !unix_sk (sk)->addr) { err =unix_autobind (sk);if (err)
@@ -1469,7 +1470,8 @@ static int unix_stream_connect(struct socket *sock, struct sockaddr *uaddr,if (err) goto out; -if (test_bit (SOCK_PASSCRED , &sock->flags) && !u->addr) { +if ((test_bit (SOCK_PASSCRED , &sock->flags) || +test_bit (SOCK_PASSPIDFD , &sock->flags)) && !u->addr) { err =unix_autobind (sk);if (err) goto out;
@@ -1670,6 +1672,8 @@ static void unix_sock_inherit_flags(const struct socket *old, {if (test_bit (SOCK_PASSCRED , &old->flags))set_bit (SOCK_PASSCRED , &new->flags); +if (test_bit (SOCK_PASSPIDFD , &old->flags)) +set_bit (SOCK_PASSPIDFD , &new->flags);if (test_bit (SOCK_PASSSEC , &old->flags))set_bit (SOCK_PASSSEC , &new->flags); }
@@ -1819,8 +1823,10 @@ static bool unix_passcred_enabled(const struct socket *sock, const struct sock *other) {return test_bit (SOCK_PASSCRED , &sock->flags) || +test_bit (SOCK_PASSPIDFD , &sock->flags) || !other->sk_socket || -test_bit (SOCK_PASSCRED , &other->sk_socket->flags); +test_bit (SOCK_PASSCRED , &other->sk_socket->flags) || +test_bit (SOCK_PASSPIDFD , &other->sk_socket->flags); }/*
@@ -1922,7 +1928,8 @@ static int unix_dgram_sendmsg(struct socket *sock, struct msghdr *msg, goto out; } -if (test_bit (SOCK_PASSCRED , &sock->flags) && !u->addr) { +if ((test_bit (SOCK_PASSCRED , &sock->flags) || +test_bit (SOCK_PASSPIDFD , &sock->flags)) && !u->addr) { err =unix_autobind (sk);if (err) goto out;
@@ -2824,7 +2831,8 @@ static int unix_stream_read_generic(struct unix_stream_read_state *state,/* Never glue messages from different writers */ if (!unix_skb_scm_eq (skb, &scm))break ; - }else if (test_bit (SOCK_PASSCRED , &sock->flags)) { + }else if (test_bit (SOCK_PASSCRED , &sock->flags) || +test_bit (SOCK_PASSPIDFD , &sock->flags)) {/* Copy credentials */ scm_set_cred (&scm,UNIXCB (skb).pid,UNIXCB (skb).uid,UNIXCB (skb).gid);unix_set_secdata (&scm, skb);
tools/include/uapi/asm-generic/socket.h +2 -0
@@ -121,6 +121,8 @@#define SO_RCVMARK 75 +#define SO_PASSPIDFD 76 +#if !defined(__KERNEL__)#if __BITS_PER_LONG ==64 || (defined(__x86_64__) && defined(__ILP32__))
Signature · 1 line
2.34.1
Re: [PATCH net-next v4 1/4] scm: add SO_PASSPIDFD and SCM_PIDFD
Quoted 137 lines · Alexander Mikhalitsyn
Implement
SCM_PIDFD, a new type of CMSG type analogical toSCM_CREDENTIALS, but it contains pidfd instead of plain pid, which allows programmers not to care about PID reuse problem.Idea comes from UAPI kernel group:
https://uapi-group.org/kernel-features/Big thanks to Christian Brauner and Lennart Poettering for productive discussions about this.
Cc "David S. Miller" <[email protected]>Cc Eric Dumazet <[email protected]>Cc Jakub Kicinski <[email protected]>Cc Paolo Abeni <[email protected]>Cc Leon Romanovsky <[email protected]>Cc David Ahern <[email protected]>Cc Arnd Bergmann <[email protected]>Cc Kees Cook <[email protected]>Cc Christian Brauner <[email protected]>Cc Kuniyuki Iwashima <[email protected]>Cc Lennart Poettering <[email protected]>Cc Luca Boccassi <[email protected]>Tested-by Luca Boccassi <[email protected]>Reviewed-by Kuniyuki Iwashima <[email protected]>Signed-off-by Alexander Mikhalitsyn <[email protected]>---
v4:
- fixed silentfd_installif writting of CMSG to the userspace fails (pointed by Christian)
v2:
According to review comments from Kuniyuki Iwashima and Christian Brauner:
- usepidfd_create(..)retval as a result
- whitespace change--- arch/alpha/include/uapi/asm/socket.h | 2 ++ arch/mips/include/uapi/asm/socket.h | 2 ++ arch/parisc/include/uapi/asm/socket.h | 2 ++ arch/sparc/include/uapi/asm/socket.h | 2 ++ include/linux/net.h | 1 + include/linux/socket.h | 1 + include/net/scm.h | 39 +++++++++++++++++++++++-- include/uapi/asm-generic/socket.h | 2 ++ net/core/sock.c | 11 +++++++ net/mptcp/sockopt.c | 1 + net/unix/af_unix.c | 18 ++++++++---- tools/include/uapi/asm-generic/socket.h | 2 ++ 12 files changed, 76 insertions(+), 7 deletions(-)arch/alpha/include/uapi/asm/socket.h +2
-0@@ -137,6 +137,8 @@#define SO_RCVMARK 75 +#define SO_PASSPIDFD 76 +#if !defined(__KERNEL__)#if __BITS_PER_LONG ==64 arch/mips/include/uapi/asm/socket.h +2
-0@@ -148,6 +148,8 @@#define SO_RCVMARK 75 +#define SO_PASSPIDFD 76 +#if !defined(__KERNEL__)#if __BITS_PER_LONG ==64 arch/parisc/include/uapi/asm/socket.h +2
-0@@ -129,6 +129,8 @@#define SO_RCVMARK 0x4049 +#define SO_PASSPIDFD 0x404A +#if !defined(__KERNEL__)#if __BITS_PER_LONG ==64 arch/sparc/include/uapi/asm/socket.h +2
-0@@ -130,6 +130,8 @@#define SO_RCVMARK 0x0054 +#define SO_PASSPIDFD 0x0055 +#if !defined(__KERNEL__)include/linux/net.h +1
-0@@ -43,6 +43,7 @@ struct net;#define SOCK_PASSSEC 4#define SOCK_SUPPORT_ZC 5#define SOCK_CUSTOM_SOCKOPT 6 +#define SOCK_PASSPIDFD 7#ifndef ARCH_HAS_SOCKET_TYPES /** include/linux/socket.h +1
-0@@ -177,6 +177,7 @@ static inline size_t msg_data_left(struct msghdr *msg)#define SCM_RIGHTS 0x01/* rw: access rights (array of int) */ #define SCM_CREDENTIALS 0x02/* rw: struct ucred */ #define SCM_SECURITY 0x03/* rw: security label */ +#define SCM_PIDFD 0x04/* ro: pidfd (int) */ struct ucred {__u32 pid;include/net/scm.h +3
-0@@ -120,12 +120,44 @@ static inline bool scm_has_secdata(struct socket *sock) }#endif /* CONFIG_SECURITY_NETWORK */ +static __inline__void scm_pidfd_recv (struct msghdr *msg,struct scm_cookie *scm) +{ +struct file *pidfd_file =NULL ;
include/net/scm.h static inline bool scm_has_secdata(struct socket *sock) · line 135
Quoted 10 lines
include/net/scm.h +10
-0+int pidfd; + +/* +* put_cmsg() doesn't return an error if CMSG is truncated, +* that's why we need to opencode these checks here. +*/ +if ((msg->msg_controllen <=sizeof (struct cmsghdr )) || + (msg->msg_controllen -sizeof (struct cmsghdr )) <sizeof (int )) { + msg->msg_flags |=MSG_CTRUNC ; +return ;
Hm, curious about this: We mark the message as truncated for SCM_PIDFD but if the same conditions were to apply for SCM_PASSCRED we don't mark the message as truncated. Am I reading this correct? And is so, you please briefly explain this difference?
include/net/scm.h static inline bool scm_has_secdata(struct socket *sock) · line 141
Quoted 6 lines
include/net/scm.h +6
-0+ } + +WARN_ON_ONCE (!scm->pid); + pidfd =pidfd_prepare (scm->pid,0 , &pidfd_file); + +if (put_cmsg (msg,SOL_SOCKET ,SCM_PIDFD ,sizeof (int ), &pidfd)) {
If the put_cmsg() of the pidfd fails userspace needs to be able to detect this. Otherwise they can't distinguish between the SCM_PIDFD value being zero because the put_cmsg() failed or put_cmsg() succeeded and the allocated fd nr was 0.
Looking at put_cmsg() it looks to me that userspace will receive a SCM_PIDFD message only if the put_cmsg() is completely successful. IIUC, then this change is fine.
Re: [PATCH net-next v4 1/4] scm: add SO_PASSPIDFD and SCM_PIDFD
Quoted 146 lines · Christian Brauner
Quoted 144 lines · Alexander Mikhalitsyn
Implement
SCM_PIDFD, a new type of CMSG type analogical toSCM_CREDENTIALS, but it contains pidfd instead of plain pid, which allows programmers not to care about PID reuse problem.Idea comes from UAPI kernel group:
https://uapi-group.org/kernel-features/Big thanks to Christian Brauner and Lennart Poettering for productive discussions about this.
Cc "David S. Miller" <[email protected]>Cc Eric Dumazet <[email protected]>Cc Jakub Kicinski <[email protected]>Cc Paolo Abeni <[email protected]>Cc Leon Romanovsky <[email protected]>Cc David Ahern <[email protected]>Cc Arnd Bergmann <[email protected]>Cc Kees Cook <[email protected]>Cc Christian Brauner <[email protected]>Cc Kuniyuki Iwashima <[email protected]>Cc Lennart Poettering <[email protected]>Cc Luca Boccassi <[email protected]>Tested-by Luca Boccassi <[email protected]>Reviewed-by Kuniyuki Iwashima <[email protected]>Signed-off-by Alexander Mikhalitsyn <[email protected]>---
v4:
- fixed silentfd_installif writting of CMSG to the userspace fails (pointed by Christian)
v2:
According to review comments from Kuniyuki Iwashima and Christian Brauner:
- usepidfd_create(..)retval as a result
- whitespace change--- arch/alpha/include/uapi/asm/socket.h | 2 ++ arch/mips/include/uapi/asm/socket.h | 2 ++ arch/parisc/include/uapi/asm/socket.h | 2 ++ arch/sparc/include/uapi/asm/socket.h | 2 ++ include/linux/net.h | 1 + include/linux/socket.h | 1 + include/net/scm.h | 39 +++++++++++++++++++++++-- include/uapi/asm-generic/socket.h | 2 ++ net/core/sock.c | 11 +++++++ net/mptcp/sockopt.c | 1 + net/unix/af_unix.c | 18 ++++++++---- tools/include/uapi/asm-generic/socket.h | 2 ++ 12 files changed, 76 insertions(+), 7 deletions(-)arch/alpha/include/uapi/asm/socket.h +2
-0@@ -137,6 +137,8 @@#define SO_RCVMARK 75 +#define SO_PASSPIDFD 76 +#if !defined(__KERNEL__)#if __BITS_PER_LONG ==64 arch/mips/include/uapi/asm/socket.h +2
-0@@ -148,6 +148,8 @@#define SO_RCVMARK 75 +#define SO_PASSPIDFD 76 +#if !defined(__KERNEL__)#if __BITS_PER_LONG ==64 arch/parisc/include/uapi/asm/socket.h +2
-0@@ -129,6 +129,8 @@#define SO_RCVMARK 0x4049 +#define SO_PASSPIDFD 0x404A +#if !defined(__KERNEL__)#if __BITS_PER_LONG ==64 arch/sparc/include/uapi/asm/socket.h +2
-0@@ -130,6 +130,8 @@#define SO_RCVMARK 0x0054 +#define SO_PASSPIDFD 0x0055 +#if !defined(__KERNEL__)include/linux/net.h +1
-0@@ -43,6 +43,7 @@ struct net;#define SOCK_PASSSEC 4#define SOCK_SUPPORT_ZC 5#define SOCK_CUSTOM_SOCKOPT 6 +#define SOCK_PASSPIDFD 7#ifndef ARCH_HAS_SOCKET_TYPES /** include/linux/socket.h +1
-0@@ -177,6 +177,7 @@ static inline size_t msg_data_left(struct msghdr *msg)#define SCM_RIGHTS 0x01/* rw: access rights (array of int) */ #define SCM_CREDENTIALS 0x02/* rw: struct ucred */ #define SCM_SECURITY 0x03/* rw: security label */ +#define SCM_PIDFD 0x04/* ro: pidfd (int) */ struct ucred {__u32 pid;include/net/scm.h +10
-0@@ -120,12 +120,44 @@ static inline bool scm_has_secdata(struct socket *sock) }#endif /* CONFIG_SECURITY_NETWORK */ +static __inline__void scm_pidfd_recv (struct msghdr *msg,struct scm_cookie *scm) +{ +struct file *pidfd_file =NULL ; +int pidfd; + +/* +* put_cmsg() doesn't return an error if CMSG is truncated, +* that's why we need to opencode these checks here. +*/ + if ((msg->msg_controllen <= sizeof(struct cmsghdr)) ||
Quoted 8 lines
include/net/scm.h static inline bool scm_has_secdata(struct socket *sock) · line 135
Quoted 3 lines
include/net/scm.h +3
-0+ (msg->msg_controllen - sizeof(struct cmsghdr)) < sizeof(int)) { + msg->msg_flags |=MSG_CTRUNC ; +return ;Hm, curious about this: We mark the message as truncated for
SCM_PIDFDbut if the same conditions were to apply forSCM_PASSCREDwe don't mark the message as truncated. Am I reading this correct? And is so, you please briefly explain this difference?
Hi, Christian!
For SCM_CREDENTIALS we mark it too. Inside the put_cmsg function:
https://github.com/torvalds/linux/blob/6a8f57ae2eb07ab39a6f0ccad60c760743051026/net/core/scm.c#L225
The reason why I'm open-coding these checks is that I want to know that the message
doesn't fit into the userspace buffer before doing pidfd_prepare and other stuff and becauseput_cmsg is not returning an error when message doesn't fit in the userspace buffer and
we won't be able to properly do pidfd cleanup (put struct pid and fd index).
Quoted 12 lines
include/net/scm.h static inline bool scm_has_secdata(struct socket *sock) · line 141
Quoted 7 lines
include/net/scm.h +6
-0+ } + +WARN_ON_ONCE (!scm->pid); + pidfd =pidfd_prepare (scm->pid,0 , &pidfd_file); + +if (put_cmsg (msg,SOL_SOCKET ,SCM_PIDFD ,sizeof (int ), &pidfd)) {If the
put_cmsg()of the pidfd fails userspace needs to be able to detect this. Otherwise they can't distinguish between theSCM_PIDFDvalue being zero because theput_cmsg()failed orput_cmsg()succeeded and the allocated fd nr was 0.
If pidfd_prepare fails then userspace will receive SCM_PIDFD message with negative pidfd value.
Quoted 4 lines
Looking at
put_cmsg()it looks to me that userspace will receive aSCM_PIDFDmessage only if theput_cmsg()is completely successful. IIUC, then this change is fine.
Kind regards,
Alex
Re: [PATCH net-next v4 1/4] scm: add SO_PASSPIDFD and SCM_PIDFD
Quoted 182 lines · Aleksandr Mikhalitsyn
Quoted 146 lines · Christian Brauner
Quoted 144 lines · Alexander Mikhalitsyn
Implement
SCM_PIDFD, a new type of CMSG type analogical toSCM_CREDENTIALS, but it contains pidfd instead of plain pid, which allows programmers not to care about PID reuse problem.Idea comes from UAPI kernel group:
https://uapi-group.org/kernel-features/Big thanks to Christian Brauner and Lennart Poettering for productive discussions about this.
Cc "David S. Miller" <[email protected]>Cc Eric Dumazet <[email protected]>Cc Jakub Kicinski <[email protected]>Cc Paolo Abeni <[email protected]>Cc Leon Romanovsky <[email protected]>Cc David Ahern <[email protected]>Cc Arnd Bergmann <[email protected]>Cc Kees Cook <[email protected]>Cc Christian Brauner <[email protected]>Cc Kuniyuki Iwashima <[email protected]>Cc Lennart Poettering <[email protected]>Cc Luca Boccassi <[email protected]>Tested-by Luca Boccassi <[email protected]>Reviewed-by Kuniyuki Iwashima <[email protected]>Signed-off-by Alexander Mikhalitsyn <[email protected]>---
v4:
- fixed silentfd_installif writting of CMSG to the userspace fails (pointed by Christian)
v2:
According to review comments from Kuniyuki Iwashima and Christian Brauner:
- usepidfd_create(..)retval as a result
- whitespace change--- arch/alpha/include/uapi/asm/socket.h | 2 ++ arch/mips/include/uapi/asm/socket.h | 2 ++ arch/parisc/include/uapi/asm/socket.h | 2 ++ arch/sparc/include/uapi/asm/socket.h | 2 ++ include/linux/net.h | 1 + include/linux/socket.h | 1 + include/net/scm.h | 39 +++++++++++++++++++++++-- include/uapi/asm-generic/socket.h | 2 ++ net/core/sock.c | 11 +++++++ net/mptcp/sockopt.c | 1 + net/unix/af_unix.c | 18 ++++++++---- tools/include/uapi/asm-generic/socket.h | 2 ++ 12 files changed, 76 insertions(+), 7 deletions(-)arch/alpha/include/uapi/asm/socket.h +2
-0@@ -137,6 +137,8 @@#define SO_RCVMARK 75 +#define SO_PASSPIDFD 76 +#if !defined(__KERNEL__)#if __BITS_PER_LONG ==64 arch/mips/include/uapi/asm/socket.h +2
-0@@ -148,6 +148,8 @@#define SO_RCVMARK 75 +#define SO_PASSPIDFD 76 +#if !defined(__KERNEL__)#if __BITS_PER_LONG ==64 arch/parisc/include/uapi/asm/socket.h +2
-0@@ -129,6 +129,8 @@#define SO_RCVMARK 0x4049 +#define SO_PASSPIDFD 0x404A +#if !defined(__KERNEL__)#if __BITS_PER_LONG ==64 arch/sparc/include/uapi/asm/socket.h +2
-0@@ -130,6 +130,8 @@#define SO_RCVMARK 0x0054 +#define SO_PASSPIDFD 0x0055 +#if !defined(__KERNEL__)include/linux/net.h +1
-0@@ -43,6 +43,7 @@ struct net;#define SOCK_PASSSEC 4#define SOCK_SUPPORT_ZC 5#define SOCK_CUSTOM_SOCKOPT 6 +#define SOCK_PASSPIDFD 7#ifndef ARCH_HAS_SOCKET_TYPES /** include/linux/socket.h +1
-0@@ -177,6 +177,7 @@ static inline size_t msg_data_left(struct msghdr *msg)#define SCM_RIGHTS 0x01/* rw: access rights (array of int) */ #define SCM_CREDENTIALS 0x02/* rw: struct ucred */ #define SCM_SECURITY 0x03/* rw: security label */ +#define SCM_PIDFD 0x04/* ro: pidfd (int) */ struct ucred {__u32 pid;include/net/scm.h +10
-0@@ -120,12 +120,44 @@ static inline bool scm_has_secdata(struct socket *sock) }#endif /* CONFIG_SECURITY_NETWORK */ +static __inline__void scm_pidfd_recv (struct msghdr *msg,struct scm_cookie *scm) +{ +struct file *pidfd_file =NULL ; +int pidfd; + +/* +* put_cmsg() doesn't return an error if CMSG is truncated, +* that's why we need to opencode these checks here. +*/ + if ((msg->msg_controllen <= sizeof(struct cmsghdr)) ||Quoted 8 lines
include/net/scm.h static inline bool scm_has_secdata(struct socket *sock) · line 135
Quoted 3 lines
include/net/scm.h +3
-0+ (msg->msg_controllen - sizeof(struct cmsghdr)) < sizeof(int)) { + msg->msg_flags |=MSG_CTRUNC ; +return ;Hm, curious about this: We mark the message as truncated for
SCM_PIDFDbut if the same conditions were to apply forSCM_PASSCREDwe don't mark the message as truncated. Am I reading this correct? And is so, you please briefly explain this difference?Hi, Christian!
For
SCM_CREDENTIALSwe mark it too. Inside theput_cmsgfunction:
https://github.com/torvalds/linux/blob/6a8f57ae2eb07ab39a6f0ccad60c760743051026/net/core/scm.c#L225The reason why I'm open-coding these checks is that I want to know that the message
doesn't fit into the userspace buffer before doingpidfd_prepareand other stuff and becauseput_cmsgis not returning an error when message doesn't fit in the userspace buffer and
we won't be able to properly do pidfd cleanup (put struct pid and fd index).Quoted 12 lines
include/net/scm.h static inline bool scm_has_secdata(struct socket *sock) · line 141
Quoted 7 lines
include/net/scm.h +6
-0+ } + +WARN_ON_ONCE (!scm->pid); + pidfd =pidfd_prepare (scm->pid,0 , &pidfd_file); + +if (put_cmsg (msg,SOL_SOCKET ,SCM_PIDFD ,sizeof (int ), &pidfd)) {If the
put_cmsg()of the pidfd fails userspace needs to be able to detect this. Otherwise they can't distinguish between theSCM_PIDFDvalue being zero because theput_cmsg()failed orput_cmsg()succeeded and the allocated fd nr was 0.
Quoted 2 lines
If
pidfd_preparefails then userspace will receiveSCM_PIDFDmessage with negative pidfd value.
So we discussed this a bit offline and I think there's still an issue. If put_cmsg() fails
if (msg->msg_control_is_user) {
struct cmsghdr __user *cm = msg->msg_control_user;
check_object_size(data, cmlen - sizeof(*cm), true);
if (!user_write_access_begin(cm, cmlen))
goto efault;
// This succeeds so cm->cmsg_len == sizeof(int)
unsafe_put_user(cmlen, &cm->cmsg_len, efault_end);
// This succeeds so cm->cmsg_level == SOL_SOCKET
unsafe_put_user(level, &cm->cmsg_level, efault_end);
// This succeeds so cm->cmsg_type == SCM_PIDFD
unsafe_put_user(type, &cm->cmsg_type, efault_end);
// This fails and leaves all bits set to 0
unsafe_copy_to_user(CMSG_USER_DATA(cm), data,
cmlen - sizeof(*cm), efault_end);
user_write_access_end();so now we hit
if (put_cmsg(msg, SOL_SOCKET, SCM_PIDFD, sizeof(int), &pidfd)) {
if (pidfd_file) {
put_unused_fd(pidfd);
fput(pidfd_file);
}
return;
}and return early. Afaict, userspace would now receive:
if (cmsg && cmsg->cmsg_len == CMSG_LEN(sizeof(int)) &&
cmsg->cmsg_level == SOL_SOCKET &&
cmsg->cmsg_type == SCM_PIDFD) {
memcpy(&pidfd, CMSG_DATA(cmsg), sizeof(int));
// pidfd is now 0 which is a valid fd number
// it'll likely refer to /dev/stdin or whatever and so
// will fail or, worst case, 0 refers to another pidfd :)
pidfd_send_signal(pidfd, SIGKILL);so we need to address this. So one way I think that would solve this is:
net/core/scm.c +1 -1
@@ -236,9 +236,9 @@ int put_cmsg(struct msghdr * msg, int level, int type, int len, void *data)unsafe_put_user (cmlen, &cm->cmsg_len, efault_end);unsafe_put_user (level, &cm->cmsg_level, efault_end); -unsafe_put_user (type, &cm->cmsg_type, efault_end);unsafe_copy_to_user (CMSG_USER_DATA (cm), data, cmlen -sizeof (*cm), efault_end); +unsafe_put_user (type, &cm->cmsg_type, efault_end);user_write_access_end (); }else {struct cmsghdr *cm = msg->msg_control;
such that we only copy cm->cmsg_type after we transfered the data.
Re: [PATCH net-next v4 1/4] scm: add SO_PASSPIDFD and SCM_PIDFD
Quoted 253 lines · Christian Brauner
Quoted 182 lines · Aleksandr Mikhalitsyn
Quoted 146 lines · Christian Brauner
Quoted 144 lines · Alexander Mikhalitsyn
Implement
SCM_PIDFD, a new type of CMSG type analogical toSCM_CREDENTIALS, but it contains pidfd instead of plain pid, which allows programmers not to care about PID reuse problem.Idea comes from UAPI kernel group:
https://uapi-group.org/kernel-features/Big thanks to Christian Brauner and Lennart Poettering for productive discussions about this.
Cc "David S. Miller" <[email protected]>Cc Eric Dumazet <[email protected]>Cc Jakub Kicinski <[email protected]>Cc Paolo Abeni <[email protected]>Cc Leon Romanovsky <[email protected]>Cc David Ahern <[email protected]>Cc Arnd Bergmann <[email protected]>Cc Kees Cook <[email protected]>Cc Christian Brauner <[email protected]>Cc Kuniyuki Iwashima <[email protected]>Cc Lennart Poettering <[email protected]>Cc Luca Boccassi <[email protected]>Tested-by Luca Boccassi <[email protected]>Reviewed-by Kuniyuki Iwashima <[email protected]>Signed-off-by Alexander Mikhalitsyn <[email protected]>---
v4:
- fixed silentfd_installif writting of CMSG to the userspace fails (pointed by Christian)
v2:
According to review comments from Kuniyuki Iwashima and Christian Brauner:
- usepidfd_create(..)retval as a result
- whitespace change--- arch/alpha/include/uapi/asm/socket.h | 2 ++ arch/mips/include/uapi/asm/socket.h | 2 ++ arch/parisc/include/uapi/asm/socket.h | 2 ++ arch/sparc/include/uapi/asm/socket.h | 2 ++ include/linux/net.h | 1 + include/linux/socket.h | 1 + include/net/scm.h | 39 +++++++++++++++++++++++-- include/uapi/asm-generic/socket.h | 2 ++ net/core/sock.c | 11 +++++++ net/mptcp/sockopt.c | 1 + net/unix/af_unix.c | 18 ++++++++---- tools/include/uapi/asm-generic/socket.h | 2 ++ 12 files changed, 76 insertions(+), 7 deletions(-)arch/alpha/include/uapi/asm/socket.h +2
-0@@ -137,6 +137,8 @@#define SO_RCVMARK 75 +#define SO_PASSPIDFD 76 +#if !defined(__KERNEL__)#if __BITS_PER_LONG ==64 arch/mips/include/uapi/asm/socket.h +2
-0@@ -148,6 +148,8 @@#define SO_RCVMARK 75 +#define SO_PASSPIDFD 76 +#if !defined(__KERNEL__)#if __BITS_PER_LONG ==64 arch/parisc/include/uapi/asm/socket.h +2
-0@@ -129,6 +129,8 @@#define SO_RCVMARK 0x4049 +#define SO_PASSPIDFD 0x404A +#if !defined(__KERNEL__)#if __BITS_PER_LONG ==64 arch/sparc/include/uapi/asm/socket.h +2
-0@@ -130,6 +130,8 @@#define SO_RCVMARK 0x0054 +#define SO_PASSPIDFD 0x0055 +#if !defined(__KERNEL__)include/linux/net.h +1
-0@@ -43,6 +43,7 @@ struct net;#define SOCK_PASSSEC 4#define SOCK_SUPPORT_ZC 5#define SOCK_CUSTOM_SOCKOPT 6 +#define SOCK_PASSPIDFD 7#ifndef ARCH_HAS_SOCKET_TYPES /** include/linux/socket.h +1
-0@@ -177,6 +177,7 @@ static inline size_t msg_data_left(struct msghdr *msg)#define SCM_RIGHTS 0x01/* rw: access rights (array of int) */ #define SCM_CREDENTIALS 0x02/* rw: struct ucred */ #define SCM_SECURITY 0x03/* rw: security label */ +#define SCM_PIDFD 0x04/* ro: pidfd (int) */ struct ucred {__u32 pid;include/net/scm.h +10
-0@@ -120,12 +120,44 @@ static inline bool scm_has_secdata(struct socket *sock) }#endif /* CONFIG_SECURITY_NETWORK */ +static __inline__void scm_pidfd_recv (struct msghdr *msg,struct scm_cookie *scm) +{ +struct file *pidfd_file =NULL ; +int pidfd; + +/* +* put_cmsg() doesn't return an error if CMSG is truncated, +* that's why we need to opencode these checks here. +*/ + if ((msg->msg_controllen <= sizeof(struct cmsghdr)) ||Quoted 8 lines
include/net/scm.h static inline bool scm_has_secdata(struct socket *sock) · line 135
Quoted 3 lines
include/net/scm.h +3
-0+ (msg->msg_controllen - sizeof(struct cmsghdr)) < sizeof(int)) { + msg->msg_flags |=MSG_CTRUNC ; +return ;Hm, curious about this: We mark the message as truncated for
SCM_PIDFDbut if the same conditions were to apply forSCM_PASSCREDwe don't mark the message as truncated. Am I reading this correct? And is so, you please briefly explain this difference?Hi, Christian!
For
SCM_CREDENTIALSwe mark it too. Inside theput_cmsgfunction:
https://github.com/torvalds/linux/blob/6a8f57ae2eb07ab39a6f0ccad60c760743051026/net/core/scm.c#L225The reason why I'm open-coding these checks is that I want to know that the message
doesn't fit into the userspace buffer before doingpidfd_prepareand other stuff and becauseput_cmsgis not returning an error when message doesn't fit in the userspace buffer and
we won't be able to properly do pidfd cleanup (put struct pid and fd index).Quoted 12 lines
include/net/scm.h static inline bool scm_has_secdata(struct socket *sock) · line 141
Quoted 7 lines
include/net/scm.h +6
-0+ } + +WARN_ON_ONCE (!scm->pid); + pidfd =pidfd_prepare (scm->pid,0 , &pidfd_file); + +if (put_cmsg (msg,SOL_SOCKET ,SCM_PIDFD ,sizeof (int ), &pidfd)) {If the
put_cmsg()of the pidfd fails userspace needs to be able to detect this. Otherwise they can't distinguish between theSCM_PIDFDvalue being zero because theput_cmsg()failed orput_cmsg()succeeded and the allocated fd nr was 0.Quoted 2 lines
If
pidfd_preparefails then userspace will receiveSCM_PIDFDmessage with negative pidfd value.So we discussed this a bit offline and I think there's still an issue. If
put_cmsg()failsif (msg->msg_control_is_user) { struct cmsghdr __user *cm = msg->msg_control_user; check_object_size(data, cmlen - sizeof(*cm), true); if (!user_write_access_begin(cm, cmlen)) goto efault; // This succeeds so cm->cmsg_len == sizeof(int) unsafe_put_user(cmlen, &cm->cmsg_len, efault_end); // This succeeds so cm->cmsg_level == SOL_SOCKET unsafe_put_user(level, &cm->cmsg_level, efault_end); // This succeeds so cm->cmsg_type == SCM_PIDFD unsafe_put_user(type, &cm->cmsg_type, efault_end); // This fails and leaves all bits set to 0 unsafe_copy_to_user(CMSG_USER_DATA(cm), data, cmlen - sizeof(*cm), efault_end); user_write_access_end();so now we hit
if (put_cmsg(msg, SOL_SOCKET, SCM_PIDFD, sizeof(int), &pidfd)) { if (pidfd_file) { put_unused_fd(pidfd); fput(pidfd_file); } return; }and return early. Afaict, userspace would now receive:
if (cmsg && cmsg->cmsg_len == CMSG_LEN(sizeof(int)) && cmsg->cmsg_level == SOL_SOCKET && cmsg->cmsg_type == SCM_PIDFD) { memcpy(&pidfd, CMSG_DATA(cmsg), sizeof(int)); // pidfd is now 0 which is a valid fd number // it'll likely refer to /dev/stdin or whatever and so // will fail or, worst case, 0 refers to another pidfd :) pidfd_send_signal(pidfd, SIGKILL);so we need to address this. So one way I think that would solve this is:
net/core/scm.c +1
-1@@ -236,9 +236,9 @@ int put_cmsg(struct msghdr * msg, int level, int type, int len, void *data)unsafe_put_user (cmlen, &cm->cmsg_len, efault_end);unsafe_put_user (level, &cm->cmsg_level, efault_end); -unsafe_put_user (type, &cm->cmsg_type, efault_end);unsafe_copy_to_user (CMSG_USER_DATA (cm), data, cmlen -sizeof (*cm), efault_end); +unsafe_put_user (type, &cm->cmsg_type, efault_end);user_write_access_end (); }else {struct cmsghdr *cm = msg->msg_control;
Quoted 1 line
such that we only copy
cm->cmsg_typeafter we transfered the data.
This looks wrong to me.
if put_cmsg() returns -EFAULT, then msg->msg_control and msg->msg_controllen were not changed.
So the user application should not attempt to read this part of the control buffer, this could contain garbage.
Re: [PATCH net-next v4 1/4] scm: add SO_PASSPIDFD and SCM_PIDFD
Quoted 258 lines · Eric Dumazet
Quoted 253 lines · Christian Brauner
Quoted 182 lines · Aleksandr Mikhalitsyn
Quoted 146 lines · Christian Brauner
Quoted 144 lines · Alexander Mikhalitsyn
Implement
SCM_PIDFD, a new type of CMSG type analogical toSCM_CREDENTIALS, but it contains pidfd instead of plain pid, which allows programmers not to care about PID reuse problem.Idea comes from UAPI kernel group:
https://uapi-group.org/kernel-features/Big thanks to Christian Brauner and Lennart Poettering for productive discussions about this.
Cc "David S. Miller" <[email protected]>Cc Eric Dumazet <[email protected]>Cc Jakub Kicinski <[email protected]>Cc Paolo Abeni <[email protected]>Cc Leon Romanovsky <[email protected]>Cc David Ahern <[email protected]>Cc Arnd Bergmann <[email protected]>Cc Kees Cook <[email protected]>Cc Christian Brauner <[email protected]>Cc Kuniyuki Iwashima <[email protected]>Cc Lennart Poettering <[email protected]>Cc Luca Boccassi <[email protected]>Tested-by Luca Boccassi <[email protected]>Reviewed-by Kuniyuki Iwashima <[email protected]>Signed-off-by Alexander Mikhalitsyn <[email protected]>---
v4:
- fixed silentfd_installif writting of CMSG to the userspace fails (pointed by Christian)
v2:
According to review comments from Kuniyuki Iwashima and Christian Brauner:
- usepidfd_create(..)retval as a result
- whitespace change--- arch/alpha/include/uapi/asm/socket.h | 2 ++ arch/mips/include/uapi/asm/socket.h | 2 ++ arch/parisc/include/uapi/asm/socket.h | 2 ++ arch/sparc/include/uapi/asm/socket.h | 2 ++ include/linux/net.h | 1 + include/linux/socket.h | 1 + include/net/scm.h | 39 +++++++++++++++++++++++-- include/uapi/asm-generic/socket.h | 2 ++ net/core/sock.c | 11 +++++++ net/mptcp/sockopt.c | 1 + net/unix/af_unix.c | 18 ++++++++---- tools/include/uapi/asm-generic/socket.h | 2 ++ 12 files changed, 76 insertions(+), 7 deletions(-)arch/alpha/include/uapi/asm/socket.h +2
-0@@ -137,6 +137,8 @@#define SO_RCVMARK 75 +#define SO_PASSPIDFD 76 +#if !defined(__KERNEL__)#if __BITS_PER_LONG ==64 arch/mips/include/uapi/asm/socket.h +2
-0@@ -148,6 +148,8 @@#define SO_RCVMARK 75 +#define SO_PASSPIDFD 76 +#if !defined(__KERNEL__)#if __BITS_PER_LONG ==64 arch/parisc/include/uapi/asm/socket.h +2
-0@@ -129,6 +129,8 @@#define SO_RCVMARK 0x4049 +#define SO_PASSPIDFD 0x404A +#if !defined(__KERNEL__)#if __BITS_PER_LONG ==64 arch/sparc/include/uapi/asm/socket.h +2
-0@@ -130,6 +130,8 @@#define SO_RCVMARK 0x0054 +#define SO_PASSPIDFD 0x0055 +#if !defined(__KERNEL__)include/linux/net.h +1
-0@@ -43,6 +43,7 @@ struct net;#define SOCK_PASSSEC 4#define SOCK_SUPPORT_ZC 5#define SOCK_CUSTOM_SOCKOPT 6 +#define SOCK_PASSPIDFD 7#ifndef ARCH_HAS_SOCKET_TYPES /** include/linux/socket.h +1
-0@@ -177,6 +177,7 @@ static inline size_t msg_data_left(struct msghdr *msg)#define SCM_RIGHTS 0x01/* rw: access rights (array of int) */ #define SCM_CREDENTIALS 0x02/* rw: struct ucred */ #define SCM_SECURITY 0x03/* rw: security label */ +#define SCM_PIDFD 0x04/* ro: pidfd (int) */ struct ucred {__u32 pid;include/net/scm.h +10
-0@@ -120,12 +120,44 @@ static inline bool scm_has_secdata(struct socket *sock) }#endif /* CONFIG_SECURITY_NETWORK */ +static __inline__void scm_pidfd_recv (struct msghdr *msg,struct scm_cookie *scm) +{ +struct file *pidfd_file =NULL ; +int pidfd; + +/* +* put_cmsg() doesn't return an error if CMSG is truncated, +* that's why we need to opencode these checks here. +*/ + if ((msg->msg_controllen <= sizeof(struct cmsghdr)) ||Quoted 8 lines
include/net/scm.h static inline bool scm_has_secdata(struct socket *sock) · line 135
Quoted 3 lines
include/net/scm.h +3
-0+ (msg->msg_controllen - sizeof(struct cmsghdr)) < sizeof(int)) { + msg->msg_flags |=MSG_CTRUNC ; +return ;Hm, curious about this: We mark the message as truncated for
SCM_PIDFDbut if the same conditions were to apply forSCM_PASSCREDwe don't mark the message as truncated. Am I reading this correct? And is so, you please briefly explain this difference?Hi, Christian!
For
SCM_CREDENTIALSwe mark it too. Inside theput_cmsgfunction:
https://github.com/torvalds/linux/blob/6a8f57ae2eb07ab39a6f0ccad60c760743051026/net/core/scm.c#L225The reason why I'm open-coding these checks is that I want to know that the message
doesn't fit into the userspace buffer before doingpidfd_prepareand other stuff and becauseput_cmsgis not returning an error when message doesn't fit in the userspace buffer and
we won't be able to properly do pidfd cleanup (put struct pid and fd index).Quoted 12 lines
include/net/scm.h static inline bool scm_has_secdata(struct socket *sock) · line 141
Quoted 7 lines
include/net/scm.h +6
-0+ } + +WARN_ON_ONCE (!scm->pid); + pidfd =pidfd_prepare (scm->pid,0 , &pidfd_file); + +if (put_cmsg (msg,SOL_SOCKET ,SCM_PIDFD ,sizeof (int ), &pidfd)) {If the
put_cmsg()of the pidfd fails userspace needs to be able to detect this. Otherwise they can't distinguish between theSCM_PIDFDvalue being zero because theput_cmsg()failed orput_cmsg()succeeded and the allocated fd nr was 0.Quoted 2 lines
If
pidfd_preparefails then userspace will receiveSCM_PIDFDmessage with negative pidfd value.So we discussed this a bit offline and I think there's still an issue. If
put_cmsg()failsif (msg->msg_control_is_user) { struct cmsghdr __user *cm = msg->msg_control_user; check_object_size(data, cmlen - sizeof(*cm), true); if (!user_write_access_begin(cm, cmlen)) goto efault; // This succeeds so cm->cmsg_len == sizeof(int) unsafe_put_user(cmlen, &cm->cmsg_len, efault_end); // This succeeds so cm->cmsg_level == SOL_SOCKET unsafe_put_user(level, &cm->cmsg_level, efault_end); // This succeeds so cm->cmsg_type == SCM_PIDFD unsafe_put_user(type, &cm->cmsg_type, efault_end); // This fails and leaves all bits set to 0 unsafe_copy_to_user(CMSG_USER_DATA(cm), data, cmlen - sizeof(*cm), efault_end); user_write_access_end();so now we hit
if (put_cmsg(msg, SOL_SOCKET, SCM_PIDFD, sizeof(int), &pidfd)) { if (pidfd_file) { put_unused_fd(pidfd); fput(pidfd_file); } return; }and return early. Afaict, userspace would now receive:
if (cmsg && cmsg->cmsg_len == CMSG_LEN(sizeof(int)) && cmsg->cmsg_level == SOL_SOCKET && cmsg->cmsg_type == SCM_PIDFD) { memcpy(&pidfd, CMSG_DATA(cmsg), sizeof(int)); // pidfd is now 0 which is a valid fd number // it'll likely refer to /dev/stdin or whatever and so // will fail or, worst case, 0 refers to another pidfd :) pidfd_send_signal(pidfd, SIGKILL);so we need to address this. So one way I think that would solve this is:
net/core/scm.c +1
-1@@ -236,9 +236,9 @@ int put_cmsg(struct msghdr * msg, int level, int type, int len, void *data)unsafe_put_user (cmlen, &cm->cmsg_len, efault_end);unsafe_put_user (level, &cm->cmsg_level, efault_end); -unsafe_put_user (type, &cm->cmsg_type, efault_end);unsafe_copy_to_user (CMSG_USER_DATA (cm), data, cmlen -sizeof (*cm), efault_end); +unsafe_put_user (type, &cm->cmsg_type, efault_end);user_write_access_end (); }else {struct cmsghdr *cm = msg->msg_control;Quoted 1 line
such that we only copy
cm->cmsg_typeafter we transfered the data.This looks wrong to me.
Quoted 5 lines
if
put_cmsg()returns-EFAULT, thenmsg->msg_controlandmsg->msg_controllenwere not changed.So the user application should not attempt to read this part of the control buffer, this could contain garbage.
Thanks for the review, Eric. That's reassuring.
I've done a bit of container related networking before but I'm fumbling my way through the reviews here. So any additional reviews here would be very helpful.
Re: [PATCH net-next v4 1/4] scm: add SO_PASSPIDFD and SCM_PIDFD
In reply to Alexander MikhalitsynQuoted 28 lines · Alexander Mikhalitsyn
Implement
SCM_PIDFD, a new type of CMSG type analogical toSCM_CREDENTIALS, but it contains pidfd instead of plain pid, which allows programmers not to care about PID reuse problem.Idea comes from UAPI kernel group:
https://uapi-group.org/kernel-features/Big thanks to Christian Brauner and Lennart Poettering for productive discussions about this.
Cc "David S. Miller" <[email protected]>Cc Eric Dumazet <[email protected]>Cc Jakub Kicinski <[email protected]>Cc Paolo Abeni <[email protected]>Cc Leon Romanovsky <[email protected]>Cc David Ahern <[email protected]>Cc Arnd Bergmann <[email protected]>Cc Kees Cook <[email protected]>Cc Christian Brauner <[email protected]>Cc Kuniyuki Iwashima <[email protected]>Cc Lennart Poettering <[email protected]>Cc Luca Boccassi <[email protected]>Tested-by Luca Boccassi <[email protected]>Reviewed-by Kuniyuki Iwashima <[email protected]>Signed-off-by Alexander Mikhalitsyn <[email protected]>
Quoted 3 lines
---
v4:
- fixed silentfd_installif writting of CMSG to the userspace fails (pointed by Christian)
I don't have a lot more to add to this,
PATCH v4 2/4 net: socket: add sockopts blacklist for BPF cgroup hook
In reply to Alexander MikhalitsynDuring work on SO_PEERPIDFD, it was discovered (thanks to Christian), that bpf cgroup hook can cause FD leaks when used with sockopts which install FDs into the process fdtable.
After some offlist discussion it was proposed to add a blacklist of socket options those can cause troubles when BPF cgroup hook is enabled.
--- net/socket.c | 38 +++++++++++++++++++++++++++++++++++--- 1 file changed, 35 insertions(+), 3 deletions(-)
net/socket.c +35 -3
@@ -108,6 +108,8 @@#include <linux/ptp_clock_kernel.h> #include <trace/events/sock.h> +#include <linux/sctp.h> +#ifdef CONFIG_NET_RX_BUSY_POLL unsigned int sysctl_net_busy_read__read_mostly ;unsigned int sysctl_net_busy_poll__read_mostly ;
@@ -2227,6 +2229,36 @@ static bool sock_use_custom_sol_socket(const struct socket *sock)return test_bit (SOCK_CUSTOM_SOCKOPT , &sock->flags); } +#ifdef CONFIG_CGROUP_BPF +static bool sockopt_installs_fd (int level,int optname) +{ +/* +* These options do fd_install(), and if BPF_CGROUP_RUN_PROG_GETSOCKOPT +* hook returns an error after success of the original handler +* sctp_getsockopt(...), userspace will receive an error from getsockopt +* syscall and will be not aware that fd was successfully installed into fdtable. +* +* Let's prevent bpf cgroup hook from running on them. +*/ +if (level ==SOL_SCTP ) { +switch (optname) { +case SCTP_SOCKOPT_PEELOFF : +case SCTP_SOCKOPT_PEELOFF_FLAGS : +return true; +default : +return false; + } + } + +return false; +} +#else /* CONFIG_CGROUP_BPF */ +static inline bool sockopt_installs_fd (int level,int optname) +{ +return false; +} +#endif /* CONFIG_CGROUP_BPF */ +/* * Set a socket option. Because we don't know the option lengths we have * to pass the user mode parameter for the protocols to sort out.
@@ -2250,7 +2282,7 @@ int __sys_setsockopt(int fd, int level, int optname, char __user *user_optval,if (err) goto out_put; -if (!in_compat_syscall ()) +if (!in_compat_syscall () && !sockopt_installs_fd (level, optname)) err =BPF_CGROUP_RUN_PROG_SETSOCKOPT (sock->sk, &level, &optname, user_optval, &optlen, &kernel_optval);
@@ -2304,7 +2336,7 @@ int __sys_getsockopt(int fd, int level, int optname, char __user *optval,if (err) goto out_put; -if (!in_compat_syscall ()) +if (!in_compat_syscall () && !sockopt_installs_fd (level, optname)) max_optlen =BPF_CGROUP_GETSOCKOPT_MAX_OPTLEN (optlen);if (level ==SOL_SOCKET )
@@ -2315,7 +2347,7 @@ int __sys_getsockopt(int fd, int level, int optname, char __user *optval, err = sock->ops->getsockopt (sock, level, optname, optval, optlen); -if (!in_compat_syscall ()) +if (!in_compat_syscall () && !sockopt_installs_fd (level, optname)) err =BPF_CGROUP_RUN_PROG_GETSOCKOPT (sock->sk, level, optname, optval, optlen, max_optlen, err);
Signature · 1 line
2.34.1
Re: [PATCH net-next v4 2/4] net: socket: add sockopts blacklist for BPF cgroup hook
Quoted 6 lines · Alexander Mikhalitsyn
During work on
SO_PEERPIDFD, it was discovered (thanks to Christian), that bpf cgroup hook can cause FD leaks when used with sockopts which install FDs into the process fdtable.After some offlist discussion it was proposed to add a blacklist of
We try to replace this word by either denylist or blocklist, even in changelogs.
Quoted 2 lines
socket options those can cause troubles when BPF cgroup hook is enabled.
Can we find the appropriate Fixes: tag to help stable teams ?
Quoted 9 lines
Cc "David S. Miller" <[email protected]>Cc Eric Dumazet <[email protected]>Cc Jakub Kicinski <[email protected]>Cc Paolo Abeni <[email protected]>Cc Leon Romanovsky <[email protected]>Cc David Ahern <[email protected]>Cc Arnd Bergmann <[email protected]>Cc Kees Cook <[email protected]>Cc Christian Brauner <[email protected]>
Quoted 8 lines
Cc Kuniyuki Iwashima <[email protected]>Cc Lennart Poettering <[email protected]>Suggested-by Daniel Borkmann <[email protected]>Suggested-by Christian Brauner <[email protected]>Signed-off-by Alexander Mikhalitsyn <[email protected]>
Thanks.
Re: [PATCH net-next v4 2/4] net: socket: add sockopts blacklist for BPF cgroup hook
Quoted 11 lines · Eric Dumazet
Quoted 6 lines · Alexander Mikhalitsyn
During work on
SO_PEERPIDFD, it was discovered (thanks to Christian), that bpf cgroup hook can cause FD leaks when used with sockopts which install FDs into the process fdtable.After some offlist discussion it was proposed to add a blacklist of
We try to replace this word by either denylist or blocklist, even in changelogs.
Hi Eric,
Oh, I'm sorry about that. :( Sure.
Quoted 5 lines
Quoted 3 lines
socket options those can cause troubles when BPF cgroup hook is enabled.
Can we find the appropriate Fixes: tag to help stable teams ?
Sure, I will add next time.
I think it's better to add Stanislav Fomichev to CC.
Kind regards,
Alex
Quoted 20 lines
Quoted 10 lines
Cc "David S. Miller" <[email protected]>Cc Eric Dumazet <[email protected]>Cc Jakub Kicinski <[email protected]>Cc Paolo Abeni <[email protected]>Cc Leon Romanovsky <[email protected]>Cc David Ahern <[email protected]>Cc Arnd Bergmann <[email protected]>Cc Kees Cook <[email protected]>Cc Christian Brauner <[email protected]>Quoted 8 lines
Cc Kuniyuki Iwashima <[email protected]>Cc Lennart Poettering <[email protected]>Suggested-by Daniel Borkmann <[email protected]>Suggested-by Christian Brauner <[email protected]>Signed-off-by Alexander Mikhalitsyn <[email protected]>Thanks.
Re: [PATCH net-next v4 2/4] net: socket: add sockopts blacklist for BPF cgroup hook
Quoted 22 lines · Aleksandr Mikhalitsyn
Quoted 11 lines · Eric Dumazet
Quoted 6 lines · Alexander Mikhalitsyn
During work on
SO_PEERPIDFD, it was discovered (thanks to Christian), that bpf cgroup hook can cause FD leaks when used with sockopts which install FDs into the process fdtable.After some offlist discussion it was proposed to add a blacklist of
We try to replace this word by either denylist or blocklist, even in changelogs.
Hi Eric,
Oh, I'm sorry about that. :( Sure.
Quoted 4 lines
Quoted 3 lines
socket options those can cause troubles when BPF cgroup hook is enabled.
Quoted 7 lines
Quoted 1 line
Can we find the appropriate Fixes: tag to help stable teams ?
Sure, I will add next time.
Fixes 0d01da6afc54 ("bpf: implement getsockopt and setsockopt hooks")I think it's better to add Stanislav Fomichev to CC.
Can we use 'struct proto' bpf_bypass_getsockopt instead? We already use it for tcp zerocopy, I'm assuming it should work in this case as well?
Quoted 23 lines
Kind regards,
AlexQuoted 20 lines
Quoted 10 lines
Cc "David S. Miller" <[email protected]>Cc Eric Dumazet <[email protected]>Cc Jakub Kicinski <[email protected]>Cc Paolo Abeni <[email protected]>Cc Leon Romanovsky <[email protected]>Cc David Ahern <[email protected]>Cc Arnd Bergmann <[email protected]>Cc Kees Cook <[email protected]>Cc Christian Brauner <[email protected]>Quoted 8 lines
Cc Kuniyuki Iwashima <[email protected]>Cc Lennart Poettering <[email protected]>Suggested-by Daniel Borkmann <[email protected]>Suggested-by Christian Brauner <[email protected]>Signed-off-by Alexander Mikhalitsyn <[email protected]>Thanks.
Re: [PATCH net-next v4 2/4] net: socket: add sockopts blacklist for BPF cgroup hook
Quoted 28 lines · Stanislav Fomichev
Quoted 26 lines · Aleksandr Mikhalitsyn
Quoted 11 lines · Eric Dumazet
Quoted 6 lines · Alexander Mikhalitsyn
During work on
SO_PEERPIDFD, it was discovered (thanks to Christian), that bpf cgroup hook can cause FD leaks when used with sockopts which install FDs into the process fdtable.After some offlist discussion it was proposed to add a blacklist of
We try to replace this word by either denylist or blocklist, even in changelogs.
Hi Eric,
Oh, I'm sorry about that. :( Sure.
Quoted 5 lines
Quoted 3 lines
socket options those can cause troubles when BPF cgroup hook is enabled.
Can we find the appropriate Fixes: tag to help stable teams ?
Sure, I will add next time.
Quoted 7 lines
Quoted 3 lines
Fixes 0d01da6afc54 ("bpf: implement getsockopt and setsockopt hooks")I think it's better to add Stanislav Fomichev to CC.
Can we use 'struct proto'
bpf_bypass_getsockoptinstead? We already use it for tcp zerocopy, I'm assuming it should work in this case as well?
Jakub reminded me of the other things I wanted to ask here bug forgot:
- setsockopt is probably not needed, right? setsockopt hook triggers before the kernel and shouldn't leak anything
- for getsockopt, instead of bypassing bpf completely, should we instead ignore the error from the bpf program? that would still preserve the observability aspect
- or maybe we can even have a per-proto bpf_getsockopt_cleanup call that gets called whenever bpf returns an error to make sure protocols have a chance to handle that condition (and free the fd)
Quoted 23 lines
Quoted 23 lines
Kind regards,
AlexQuoted 20 lines
Quoted 10 lines
Cc "David S. Miller" <[email protected]>Cc Eric Dumazet <[email protected]>Cc Jakub Kicinski <[email protected]>Cc Paolo Abeni <[email protected]>Cc Leon Romanovsky <[email protected]>Cc David Ahern <[email protected]>Cc Arnd Bergmann <[email protected]>Cc Kees Cook <[email protected]>Cc Christian Brauner <[email protected]>Quoted 8 lines
Cc Kuniyuki Iwashima <[email protected]>Cc Lennart Poettering <[email protected]>Suggested-by Daniel Borkmann <[email protected]>Suggested-by Christian Brauner <[email protected]>Signed-off-by Alexander Mikhalitsyn <[email protected]>Thanks.
Re: [PATCH net-next v4 2/4] net: socket: add sockopts blacklist for BPF cgroup hook
Quoted 37 lines · Stanislav Fomichev
Quoted 28 lines · Stanislav Fomichev
Quoted 26 lines · Aleksandr Mikhalitsyn
Quoted 11 lines · Eric Dumazet
Quoted 6 lines · Alexander Mikhalitsyn
During work on
SO_PEERPIDFD, it was discovered (thanks to Christian), that bpf cgroup hook can cause FD leaks when used with sockopts which install FDs into the process fdtable.After some offlist discussion it was proposed to add a blacklist of
We try to replace this word by either denylist or blocklist, even in changelogs.
Hi Eric,
Oh, I'm sorry about that. :( Sure.
Quoted 5 lines
Quoted 3 lines
socket options those can cause troubles when BPF cgroup hook is enabled.
Can we find the appropriate Fixes: tag to help stable teams ?
Sure, I will add next time.
Quoted 7 lines
Quoted 3 lines
Fixes 0d01da6afc54 ("bpf: implement getsockopt and setsockopt hooks")I think it's better to add Stanislav Fomichev to CC.
Can we use 'struct proto'
bpf_bypass_getsockoptinstead? We already use it for tcp zerocopy, I'm assuming it should work in this case as well?
Quoted 6 lines
Jakub reminded me of the other things I wanted to ask here bug forgot:
- setsockopt is probably not needed, right? setsockopt hook triggers before the kernel and shouldn't leak anything
- for getsockopt, instead of bypassing bpf completely, should we instead ignore the error from the bpf program? that would still preserve
That's fine by me as well.
It'd be great if the net folks could tell Alex how they would want this handled.
Quoted 1 line
the observability aspect
Please see for more details
https://lore.kernel.org/lkml/20230411-nudelsalat-spreu-3038458f25c4@brauner
Quoted 3 lines
- or maybe we can even have a per-proto
bpf_getsockopt_cleanupcall that gets called whenever bpf returns an error to make sure protocols have a chance to handle that condition (and free the fd)
Installing an fd into an fdtable makes it visible to userspace at which point calling close_fd() is doable but an absolute last resort and generally a good indicator of misdesign. If the bpf hook wants to make decisions based on the file then it should receive a struct file, not an fd.
Re: [PATCH net-next v4 2/4] net: socket: add sockopts blacklist for BPF cgroup hook
Quoted 46 lines · Christian Brauner
Quoted 37 lines · Stanislav Fomichev
Quoted 28 lines · Stanislav Fomichev
Quoted 26 lines · Aleksandr Mikhalitsyn
Quoted 11 lines · Eric Dumazet
Quoted 6 lines · Alexander Mikhalitsyn
During work on
SO_PEERPIDFD, it was discovered (thanks to Christian), that bpf cgroup hook can cause FD leaks when used with sockopts which install FDs into the process fdtable.After some offlist discussion it was proposed to add a blacklist of
We try to replace this word by either denylist or blocklist, even in changelogs.
Hi Eric,
Oh, I'm sorry about that. :( Sure.
Quoted 5 lines
Quoted 3 lines
socket options those can cause troubles when BPF cgroup hook is enabled.
Can we find the appropriate Fixes: tag to help stable teams ?
Sure, I will add next time.
Quoted 7 lines
Quoted 3 lines
Fixes 0d01da6afc54 ("bpf: implement getsockopt and setsockopt hooks")I think it's better to add Stanislav Fomichev to CC.
Can we use 'struct proto'
bpf_bypass_getsockoptinstead? We already use it for tcp zerocopy, I'm assuming it should work in this case as well?Quoted 6 lines
Jakub reminded me of the other things I wanted to ask here bug forgot:
- setsockopt is probably not needed, right? setsockopt hook triggers before the kernel and shouldn't leak anything
- for getsockopt, instead of bypassing bpf completely, should we instead ignore the error from the bpf program? that would still preserve
Quoted 4 lines
That's fine by me as well.
It'd be great if the net folks could tell Alex how they would want this handled.
Doing the bypass seems fine with me for now. If we ever decide that fd-based optvals are worth inspecting in bpf, we can lift that bypass.
Quoted 4 lines
Quoted 1 line
the observability aspect
Please see for more details
https://lore.kernel.org/lkml/20230411-nudelsalat-spreu-3038458f25c4@brauner
Thanks for the context. Yeah, sockopts are being used for a lot of interesting things :-(
Quoted 9 lines
Quoted 3 lines
- or maybe we can even have a per-proto
bpf_getsockopt_cleanupcall that gets called whenever bpf returns an error to make sure protocols have a chance to handle that condition (and free the fd)Installing an fd into an fdtable makes it visible to userspace at which point calling
close_fd()is doable but an absolute last resort and generally a good indicator of misdesign. If the bpf hook wants to make decisions based on the file then it should receive a struct file, not an fd.
SG! Then let's not over-complicate it for now and do a simple bypass.
Martin KaFai Lau
reviewerhandling unsupported optlen in cgroup bpf getsockopt: (was [PATCH net-next v4 2/4] net: socket: add sockopts blacklist for BPF cgroup hook)
In reply to Stanislav FomichevQuoted 37 lines · Stanislav Fomichev
Quoted 28 lines · Stanislav Fomichev
Quoted 26 lines · Aleksandr Mikhalitsyn
Quoted 11 lines · Eric Dumazet
Quoted 6 lines · Alexander Mikhalitsyn
During work on
SO_PEERPIDFD, it was discovered (thanks to Christian), that bpf cgroup hook can cause FD leaks when used with sockopts which install FDs into the process fdtable.After some offlist discussion it was proposed to add a blacklist of
We try to replace this word by either denylist or blocklist, even in changelogs.
Hi Eric,
Oh, I'm sorry about that. :( Sure.
Quoted 5 lines
Quoted 3 lines
socket options those can cause troubles when BPF cgroup hook is enabled.
Can we find the appropriate Fixes: tag to help stable teams ?
Sure, I will add next time.
Quoted 7 lines
Quoted 3 lines
Fixes 0d01da6afc54 ("bpf: implement getsockopt and setsockopt hooks")I think it's better to add Stanislav Fomichev to CC.
Can we use 'struct proto'
bpf_bypass_getsockoptinstead? We already use it for tcp zerocopy, I'm assuming it should work in this case as well?
Quoted 7 lines
Jakub reminded me of the other things I wanted to ask here bug forgot:
- setsockopt is probably not needed, right? setsockopt hook triggers before the kernel and shouldn't leak anything
- for getsockopt, instead of bypassing bpf completely, should we instead ignore the error from the bpf program? that would still preserve the observability aspect
stealing this thread to discuss the optlen issue which may make sense to bypass also.
There has been issue with optlen. Other than this older post related to optlen > PAGE_SIZE: https://lore.kernel.org/bpf/[email protected]/, the recent one related to optlen that we have seen is NETLINK_LIST_MEMBERSHIPS. The userspace passed in optlen == 0 and the kernel put the expected optlen (> 0) and 'return 0;' to userspace. The userspace intention is to learn the expected optlen. This makes 'ctx.optlen > max_optlen' and __cgroup_bpf_run_filter_getsockopt() ends up returning -EFAULT to the userspace even the bpf prog has not changed anything.
Does it make sense to also bypass the bpf prog when 'ctx.optlen > max_optlen' for now (and this can use a separate patch which as usual requires a bpf selftests)?
In the future, does it make sense to have a specific cgroup-bpf-prog (a specific attach type?) that only uses bpf_dynptr kfunc to access the optval such that it can enforce read-only for some optname and potentially also track if bpf-prog has written a new optval? The bpf-prog can only return 1 (OK) and only allows using bpf_set_retval() instead. Likely there is still holes but could be a seed of thought to continue polishing the idea.
Quoted 4 lines
- or maybe we can even have a per-proto
bpf_getsockopt_cleanupcall that gets called whenever bpf returns an error to make sure protocols have a chance to handle that condition (and free the fd)
Re: handling unsupported optlen in cgroup bpf getsockopt: (was [PATCH net-next v4 2/4] net: socket: add sockopts blacklist for BPF cgroup hook)
Quoted 49 lines · Martin KaFai Lau
Quoted 37 lines · Stanislav Fomichev
Quoted 28 lines · Stanislav Fomichev
Quoted 26 lines · Aleksandr Mikhalitsyn
Quoted 11 lines · Eric Dumazet
Quoted 6 lines · Alexander Mikhalitsyn
During work on
SO_PEERPIDFD, it was discovered (thanks to Christian), that bpf cgroup hook can cause FD leaks when used with sockopts which install FDs into the process fdtable.After some offlist discussion it was proposed to add a blacklist of
We try to replace this word by either denylist or blocklist, even in changelogs.
Hi Eric,
Oh, I'm sorry about that. :( Sure.
Quoted 5 lines
Quoted 3 lines
socket options those can cause troubles when BPF cgroup hook is enabled.
Can we find the appropriate Fixes: tag to help stable teams ?
Sure, I will add next time.
Quoted 7 lines
Quoted 3 lines
Fixes 0d01da6afc54 ("bpf: implement getsockopt and setsockopt hooks")I think it's better to add Stanislav Fomichev to CC.
Can we use 'struct proto'
bpf_bypass_getsockoptinstead? We already use it for tcp zerocopy, I'm assuming it should work in this case as well?Quoted 7 lines
Jakub reminded me of the other things I wanted to ask here bug forgot:
- setsockopt is probably not needed, right? setsockopt hook triggers before the kernel and shouldn't leak anything
- for getsockopt, instead of bypassing bpf completely, should we instead ignore the error from the bpf program? that would still preserve the observability aspectstealing this thread to discuss the optlen issue which may make sense to bypass also.
Quoted 9 lines
There has been issue with optlen. Other than this older post related to optlen >
PAGE_SIZE:
https://lore.kernel.org/bpf/[email protected]/,
the recent one related to optlen that we have seen isNETLINK_LIST_MEMBERSHIPS. The userspace passed in optlen == 0 and the kernel put the expected optlen (> 0) and 'return 0;' to userspace. The userspace intention is to learn the expected optlen. This makes 'ctx.optlen >max_optlen' and__cgroup_bpf_run_filter_getsockopt()ends up returning-EFAULTto the userspace even the bpf prog has not changed anything.
(ignoring -EFAULT issue) this seems like it needs to be
if (optval && (ctx.optlen > max_optlen || ctx.optlen < 0)) {
/* error */
}?
Quoted 3 lines
Does it make sense to also bypass the bpf prog when 'ctx.optlen >
max_optlen' for now (and this can use a separate patch which as usual requires a bpf selftests)?
Yeah, makes sense. Replacing this -EFAULT with WARN_ON_ONCE or something seems like the way to go. It caused too much trouble already :-(
Should I prepare a patch or do you want to take a stab at it?
Quoted 6 lines
In the future, does it make sense to have a specific cgroup-bpf-prog (a specific attach type?) that only uses
bpf_dynptrkfunc to access the optval such that it can enforce read-only for some optname and potentially also track if bpf-prog has written a new optval? The bpf-prog can only return 1 (OK) and only allows usingbpf_set_retval()instead. Likely there is still holes but could be a seed of thought to continue polishing the idea.
Ack, let's think about it.
Maybe we should re-evaluate 'getsockopt-happens-after-the-kernel' idea as well? If we can have a sleepable hook that can copy_from_user/copy_to_user, and we have a mostly working bpf_getsockopt (after your refactoring), I don't see why we need to continue the current scheme of triggering after the kernel?
Quoted 6 lines
Quoted 4 lines
- or maybe we can even have a per-proto
bpf_getsockopt_cleanupcall that gets called whenever bpf returns an error to make sure protocols have a chance to handle that condition (and free the fd)
Re: handling unsupported optlen in cgroup bpf getsockopt: (was [PATCH net-next v4 2/4] net: socket: add sockopts blacklist for BPF cgroup hook)
Quoted 84 lines · Stanislav Fomichev
Quoted 49 lines · Martin KaFai Lau
Quoted 37 lines · Stanislav Fomichev
Quoted 28 lines · Stanislav Fomichev
Quoted 26 lines · Aleksandr Mikhalitsyn
Quoted 11 lines · Eric Dumazet
Quoted 6 lines · Alexander Mikhalitsyn
During work on
SO_PEERPIDFD, it was discovered (thanks to Christian), that bpf cgroup hook can cause FD leaks when used with sockopts which install FDs into the process fdtable.After some offlist discussion it was proposed to add a blacklist of
We try to replace this word by either denylist or blocklist, even in changelogs.
Hi Eric,
Oh, I'm sorry about that. :( Sure.
Quoted 5 lines
Quoted 3 lines
socket options those can cause troubles when BPF cgroup hook is enabled.
Can we find the appropriate Fixes: tag to help stable teams ?
Sure, I will add next time.
Quoted 7 lines
Quoted 3 lines
Fixes 0d01da6afc54 ("bpf: implement getsockopt and setsockopt hooks")I think it's better to add Stanislav Fomichev to CC.
Can we use 'struct proto'
bpf_bypass_getsockoptinstead? We already use it for tcp zerocopy, I'm assuming it should work in this case as well?Quoted 7 lines
Jakub reminded me of the other things I wanted to ask here bug forgot:
- setsockopt is probably not needed, right? setsockopt hook triggers before the kernel and shouldn't leak anything
- for getsockopt, instead of bypassing bpf completely, should we instead ignore the error from the bpf program? that would still preserve the observability aspectstealing this thread to discuss the optlen issue which may make sense to bypass also.
Quoted 9 lines
There has been issue with optlen. Other than this older post related to optlen >
PAGE_SIZE:
https://lore.kernel.org/bpf/[email protected]/,
the recent one related to optlen that we have seen isNETLINK_LIST_MEMBERSHIPS. The userspace passed in optlen == 0 and the kernel put the expected optlen (> 0) and 'return 0;' to userspace. The userspace intention is to learn the expected optlen. This makes 'ctx.optlen >max_optlen' and__cgroup_bpf_run_filter_getsockopt()ends up returning-EFAULTto the userspace even the bpf prog has not changed anything.(ignoring
-EFAULTissue) this seems like it needs to beif (optval && (ctx.optlen > max_optlen || ctx.optlen < 0)) { /* error */ }?
Quoted 3 lines
Does it make sense to also bypass the bpf prog when 'ctx.optlen >
max_optlen' for now (and this can use a separate patch which as usual requires a bpf selftests)?Yeah, makes sense. Replacing this
-EFAULTwithWARN_ON_ONCEor something seems like the way to go. It caused too much trouble already :-(Should I prepare a patch or do you want to take a stab at it?
Quoted 6 lines
In the future, does it make sense to have a specific cgroup-bpf-prog (a specific attach type?) that only uses
bpf_dynptrkfunc to access the optval such that it can enforce read-only for some optname and potentially also track if bpf-prog has written a new optval? The bpf-prog can only return 1 (OK) and only allows usingbpf_set_retval()instead. Likely there is still holes but could be a seed of thought to continue polishing the idea.
Quoted 7 lines
Ack, let's think about it.
Maybe we should re-evaluate 'getsockopt-happens-after-the-kernel' idea as well? If we can have a sleepable hook that can copy_from_user/copy_to_user, and we have a mostly working
bpf_getsockopt(after your refactoring), I don't see why we need to continue the current scheme of triggering after the kernel?
Since a sleepable hook would cause some restrictions, perhaps, we could introduce something like the promise pattern. In our case here, BPF program call an async version of copy_from_user()/copy_to_user() to return a promise.
Quoted 7 lines
Quoted 7 lines
Quoted 5 lines
- or maybe we can even have a per-proto
bpf_getsockopt_cleanupcall that gets called whenever bpf returns an error to make sure protocols have a chance to handle that condition (and free the fd)
Re: handling unsupported optlen in cgroup bpf getsockopt: (was [PATCH net-next v4 2/4] net: socket: add sockopts blacklist for BPF cgroup hook)
Quoted 96 lines · Kui-Feng Lee
Quoted 84 lines · Stanislav Fomichev
Quoted 49 lines · Martin KaFai Lau
Quoted 37 lines · Stanislav Fomichev
Quoted 28 lines · Stanislav Fomichev
Quoted 26 lines · Aleksandr Mikhalitsyn
Quoted 11 lines · Eric Dumazet
Quoted 6 lines · Alexander Mikhalitsyn
During work on
SO_PEERPIDFD, it was discovered (thanks to Christian), that bpf cgroup hook can cause FD leaks when used with sockopts which install FDs into the process fdtable.After some offlist discussion it was proposed to add a blacklist of
We try to replace this word by either denylist or blocklist, even in changelogs.
Hi Eric,
Oh, I'm sorry about that. :( Sure.
Quoted 5 lines
Quoted 3 lines
socket options those can cause troubles when BPF cgroup hook is enabled.
Can we find the appropriate Fixes: tag to help stable teams ?
Sure, I will add next time.
Quoted 7 lines
Quoted 3 lines
Fixes 0d01da6afc54 ("bpf: implement getsockopt and setsockopt hooks")I think it's better to add Stanislav Fomichev to CC.
Can we use 'struct proto'
bpf_bypass_getsockoptinstead? We already use it for tcp zerocopy, I'm assuming it should work in this case as well?Quoted 7 lines
Jakub reminded me of the other things I wanted to ask here bug forgot:
- setsockopt is probably not needed, right? setsockopt hook triggers before the kernel and shouldn't leak anything
- for getsockopt, instead of bypassing bpf completely, should we instead ignore the error from the bpf program? that would still preserve the observability aspectstealing this thread to discuss the optlen issue which may make sense to bypass also.
Quoted 9 lines
There has been issue with optlen. Other than this older post related to optlen >
PAGE_SIZE:
https://lore.kernel.org/bpf/[email protected]/,
the recent one related to optlen that we have seen isNETLINK_LIST_MEMBERSHIPS. The userspace passed in optlen == 0 and the kernel put the expected optlen (> 0) and 'return 0;' to userspace. The userspace intention is to learn the expected optlen. This makes 'ctx.optlen >max_optlen' and__cgroup_bpf_run_filter_getsockopt()ends up returning-EFAULTto the userspace even the bpf prog has not changed anything.(ignoring
-EFAULTissue) this seems like it needs to beif (optval && (ctx.optlen > max_optlen || ctx.optlen < 0)) { /* error */ }?
Quoted 3 lines
Does it make sense to also bypass the bpf prog when 'ctx.optlen >
max_optlen' for now (and this can use a separate patch which as usual requires a bpf selftests)?Yeah, makes sense. Replacing this
-EFAULTwithWARN_ON_ONCEor something seems like the way to go. It caused too much trouble already :-(Should I prepare a patch or do you want to take a stab at it?
Quoted 6 lines
In the future, does it make sense to have a specific cgroup-bpf-prog (a specific attach type?) that only uses
bpf_dynptrkfunc to access the optval such that it can enforce read-only for some optname and potentially also track if bpf-prog has written a new optval? The bpf-prog can only return 1 (OK) and only allows usingbpf_set_retval()instead. Likely there is still holes but could be a seed of thought to continue polishing the idea.Quoted 7 lines
Ack, let's think about it.
Maybe we should re-evaluate 'getsockopt-happens-after-the-kernel' idea as well? If we can have a sleepable hook that can copy_from_user/copy_to_user, and we have a mostly working
bpf_getsockopt(after your refactoring), I don't see why we need to continue the current scheme of triggering after the kernel?
Quoted 4 lines
Since a sleepable hook would cause some restrictions, perhaps, we could introduce something like the promise pattern. In our case here, BPF program call an async version of copy_from_user()/copy_to_user() to return a promise.
Having a promise might work. This is essentially what we already do with sockets/etc with acquire/release pattern.
What are the sleepable restrictions you're hinting about? I feel like with the sleepable bpf, we can also remove all the temporary buffer management / extra copies which sounds like a win to me. (we have this ugly heuristics with BPF_SOCKOPT_KERN_BUF_SIZE) The program can allocate temporary buffers if needed..
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 4 lines
- or maybe we can even have a per-proto
bpf_getsockopt_cleanupcall that gets called whenever bpf returns an error to make sure protocols have a chance to handle that condition (and free the fd)
Re: handling unsupported optlen in cgroup bpf getsockopt: (was [PATCH net-next v4 2/4] net: socket: add sockopts blacklist for BPF cgroup hook)
Quoted 102 lines · Stanislav Fomichev
Quoted 96 lines · Kui-Feng Lee
Quoted 84 lines · Stanislav Fomichev
Quoted 49 lines · Martin KaFai Lau
Quoted 37 lines · Stanislav Fomichev
Quoted 28 lines · Stanislav Fomichev
Quoted 26 lines · Aleksandr Mikhalitsyn
Quoted 11 lines · Eric Dumazet
Quoted 6 lines · Alexander Mikhalitsyn
During work on
SO_PEERPIDFD, it was discovered (thanks to Christian), that bpf cgroup hook can cause FD leaks when used with sockopts which install FDs into the process fdtable.After some offlist discussion it was proposed to add a blacklist of
We try to replace this word by either denylist or blocklist, even in changelogs.
Hi Eric,
Oh, I'm sorry about that. :( Sure.
Quoted 5 lines
Quoted 3 lines
socket options those can cause troubles when BPF cgroup hook is enabled.
Can we find the appropriate Fixes: tag to help stable teams ?
Sure, I will add next time.
Quoted 7 lines
Quoted 3 lines
Fixes 0d01da6afc54 ("bpf: implement getsockopt and setsockopt hooks")I think it's better to add Stanislav Fomichev to CC.
Can we use 'struct proto'
bpf_bypass_getsockoptinstead? We already use it for tcp zerocopy, I'm assuming it should work in this case as well?Quoted 7 lines
Jakub reminded me of the other things I wanted to ask here bug forgot:
- setsockopt is probably not needed, right? setsockopt hook triggers before the kernel and shouldn't leak anything
- for getsockopt, instead of bypassing bpf completely, should we instead ignore the error from the bpf program? that would still preserve the observability aspectstealing this thread to discuss the optlen issue which may make sense to bypass also.
Quoted 9 lines
There has been issue with optlen. Other than this older post related to optlen >
PAGE_SIZE:
https://lore.kernel.org/bpf/[email protected]/,
the recent one related to optlen that we have seen isNETLINK_LIST_MEMBERSHIPS. The userspace passed in optlen == 0 and the kernel put the expected optlen (> 0) and 'return 0;' to userspace. The userspace intention is to learn the expected optlen. This makes 'ctx.optlen >max_optlen' and__cgroup_bpf_run_filter_getsockopt()ends up returning-EFAULTto the userspace even the bpf prog has not changed anything.(ignoring
-EFAULTissue) this seems like it needs to beif (optval && (ctx.optlen > max_optlen || ctx.optlen < 0)) { /* error */ }?
Quoted 3 lines
Does it make sense to also bypass the bpf prog when 'ctx.optlen >
max_optlen' for now (and this can use a separate patch which as usual requires a bpf selftests)?Yeah, makes sense. Replacing this
-EFAULTwithWARN_ON_ONCEor something seems like the way to go. It caused too much trouble already :-(Should I prepare a patch or do you want to take a stab at it?
Quoted 6 lines
In the future, does it make sense to have a specific cgroup-bpf-prog (a specific attach type?) that only uses
bpf_dynptrkfunc to access the optval such that it can enforce read-only for some optname and potentially also track if bpf-prog has written a new optval? The bpf-prog can only return 1 (OK) and only allows usingbpf_set_retval()instead. Likely there is still holes but could be a seed of thought to continue polishing the idea.Quoted 7 lines
Ack, let's think about it.
Maybe we should re-evaluate 'getsockopt-happens-after-the-kernel' idea as well? If we can have a sleepable hook that can copy_from_user/copy_to_user, and we have a mostly working
bpf_getsockopt(after your refactoring), I don't see why we need to continue the current scheme of triggering after the kernel?Quoted 4 lines
Since a sleepable hook would cause some restrictions, perhaps, we could introduce something like the promise pattern. In our case here, BPF program call an async version of copy_from_user()/copy_to_user() to return a promise.
Quoted 4 lines
Having a promise might work. This is essentially what we already do with sockets/etc with acquire/release pattern.
What are the sleepable restrictions you're hinting about? I feel like
AFAIK, a sleepable program can use only some types of maps; for example, array, hash, ringbuf,... etc. Other types of maps are unavailable to sleepable programs; for example, sockmap, sockhash.
Quoted 4 lines
with the sleepable bpf, we can also remove all the temporary buffer management / extra copies which sounds like a win to me. (we have this ugly heuristics with
BPF_SOCKOPT_KERN_BUF_SIZE) The program can allocate temporary buffers if needed..
Agree!
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 5 lines
- or maybe we can even have a per-proto
bpf_getsockopt_cleanupcall that gets called whenever bpf returns an error to make sure protocols have a chance to handle that condition (and free the fd)
Re: handling unsupported optlen in cgroup bpf getsockopt: (was [PATCH net-next v4 2/4] net: socket: add sockopts blacklist for BPF cgroup hook)
Quoted 106 lines · Kui-Feng Lee
Quoted 102 lines · Stanislav Fomichev
Quoted 100 lines · Kui-Feng Lee
Quoted 84 lines · Stanislav Fomichev
Quoted 49 lines · Martin KaFai Lau
Quoted 37 lines · Stanislav Fomichev
Quoted 28 lines · Stanislav Fomichev
Quoted 26 lines · Aleksandr Mikhalitsyn
Quoted 11 lines · Eric Dumazet
Quoted 6 lines · Alexander Mikhalitsyn
During work on
SO_PEERPIDFD, it was discovered (thanks to Christian), that bpf cgroup hook can cause FD leaks when used with sockopts which install FDs into the process fdtable.After some offlist discussion it was proposed to add a blacklist of
We try to replace this word by either denylist or blocklist, even in changelogs.
Hi Eric,
Oh, I'm sorry about that. :( Sure.
Quoted 5 lines
Quoted 3 lines
socket options those can cause troubles when BPF cgroup hook is enabled.
Can we find the appropriate Fixes: tag to help stable teams ?
Sure, I will add next time.
Quoted 7 lines
Quoted 3 lines
Fixes 0d01da6afc54 ("bpf: implement getsockopt and setsockopt hooks")I think it's better to add Stanislav Fomichev to CC.
Can we use 'struct proto'
bpf_bypass_getsockoptinstead? We already use it for tcp zerocopy, I'm assuming it should work in this case as well?Quoted 7 lines
Jakub reminded me of the other things I wanted to ask here bug forgot:
- setsockopt is probably not needed, right? setsockopt hook triggers before the kernel and shouldn't leak anything
- for getsockopt, instead of bypassing bpf completely, should we instead ignore the error from the bpf program? that would still preserve the observability aspectstealing this thread to discuss the optlen issue which may make sense to bypass also.
Quoted 9 lines
There has been issue with optlen. Other than this older post related to optlen >
PAGE_SIZE:
https://lore.kernel.org/bpf/[email protected]/,
the recent one related to optlen that we have seen isNETLINK_LIST_MEMBERSHIPS. The userspace passed in optlen == 0 and the kernel put the expected optlen (> 0) and 'return 0;' to userspace. The userspace intention is to learn the expected optlen. This makes 'ctx.optlen >max_optlen' and__cgroup_bpf_run_filter_getsockopt()ends up returning-EFAULTto the userspace even the bpf prog has not changed anything.(ignoring
-EFAULTissue) this seems like it needs to beif (optval && (ctx.optlen > max_optlen || ctx.optlen < 0)) { /* error */ }?
Quoted 3 lines
Does it make sense to also bypass the bpf prog when 'ctx.optlen >
max_optlen' for now (and this can use a separate patch which as usual requires a bpf selftests)?Yeah, makes sense. Replacing this
-EFAULTwithWARN_ON_ONCEor something seems like the way to go. It caused too much trouble already :-(Should I prepare a patch or do you want to take a stab at it?
Quoted 6 lines
In the future, does it make sense to have a specific cgroup-bpf-prog (a specific attach type?) that only uses
bpf_dynptrkfunc to access the optval such that it can enforce read-only for some optname and potentially also track if bpf-prog has written a new optval? The bpf-prog can only return 1 (OK) and only allows usingbpf_set_retval()instead. Likely there is still holes but could be a seed of thought to continue polishing the idea.Quoted 7 lines
Ack, let's think about it.
Maybe we should re-evaluate 'getsockopt-happens-after-the-kernel' idea as well? If we can have a sleepable hook that can copy_from_user/copy_to_user, and we have a mostly working
bpf_getsockopt(after your refactoring), I don't see why we need to continue the current scheme of triggering after the kernel?Since a sleepable hook would cause some restrictions, perhaps, we could introduce something like the promise pattern. In our case here, BPF program call an async version of copy_from_user()/copy_to_user() to return a promise.
Quoted 7 lines
Quoted 4 lines
Having a promise might work. This is essentially what we already do with sockets/etc with acquire/release pattern.
What are the sleepable restrictions you're hinting about? I feel like
AFAIK, a sleepable program can use only some types of maps; for example, array, hash, ringbuf,... etc. Other types of maps are unavailable to sleepable programs; for example, sockmap, sockhash.
Sure, but it doesn't have to stay that way. (hypothetically) If we think that sleepable makes sense, we can try to expand the scope.
Quoted 14 lines
Quoted 4 lines
with the sleepable bpf, we can also remove all the temporary buffer management / extra copies which sounds like a win to me. (we have this ugly heuristics with
BPF_SOCKOPT_KERN_BUF_SIZE) The program can allocate temporary buffers if needed..Agree!
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 5 lines
- or maybe we can even have a per-proto
bpf_getsockopt_cleanupcall that gets called whenever bpf returns an error to make sure protocols have a chance to handle that condition (and free the fd)
Re: handling unsupported optlen in cgroup bpf getsockopt: (was [PATCH net-next v4 2/4] net: socket: add sockopts blacklist for BPF cgroup hook)
In reply to Stanislav FomichevQuoted 97 lines · Stanislav Fomichev
Quoted 96 lines · Kui-Feng Lee
Quoted 91 lines · Stanislav Fomichev
Quoted 49 lines · Martin KaFai Lau
Quoted 37 lines · Stanislav Fomichev
Quoted 28 lines · Stanislav Fomichev
Quoted 26 lines · Aleksandr Mikhalitsyn
Quoted 11 lines · Eric Dumazet
Quoted 6 lines · Alexander Mikhalitsyn
During work on
SO_PEERPIDFD, it was discovered (thanks to Christian), that bpf cgroup hook can cause FD leaks when used with sockopts which install FDs into the process fdtable.After some offlist discussion it was proposed to add a blacklist of
We try to replace this word by either denylist or blocklist, even in changelogs.
Hi Eric,
Oh, I'm sorry about that. :( Sure.
Quoted 5 lines
Quoted 3 lines
socket options those can cause troubles when BPF cgroup hook is enabled.
Can we find the appropriate Fixes: tag to help stable teams ?
Sure, I will add next time.
Quoted 7 lines
Quoted 3 lines
Fixes 0d01da6afc54 ("bpf: implement getsockopt and setsockopt hooks")I think it's better to add Stanislav Fomichev to CC.
Can we use 'struct proto'
bpf_bypass_getsockoptinstead? We already use it for tcp zerocopy, I'm assuming it should work in this case as well?Quoted 7 lines
Jakub reminded me of the other things I wanted to ask here bug forgot:
- setsockopt is probably not needed, right? setsockopt hook triggers before the kernel and shouldn't leak anything
- for getsockopt, instead of bypassing bpf completely, should we instead ignore the error from the bpf program? that would still preserve the observability aspectstealing this thread to discuss the optlen issue which may make sense to bypass also.
Quoted 9 lines
There has been issue with optlen. Other than this older post related to optlen >
PAGE_SIZE:
https://lore.kernel.org/bpf/[email protected]/,
the recent one related to optlen that we have seen isNETLINK_LIST_MEMBERSHIPS. The userspace passed in optlen == 0 and the kernel put the expected optlen (> 0) and 'return 0;' to userspace. The userspace intention is to learn the expected optlen. This makes 'ctx.optlen >max_optlen' and__cgroup_bpf_run_filter_getsockopt()ends up returning-EFAULTto the userspace even the bpf prog has not changed anything.(ignoring
-EFAULTissue) this seems like it needs to beif (optval && (ctx.optlen > max_optlen || ctx.optlen < 0)) { /* error */ }?
Quoted 3 lines
Does it make sense to also bypass the bpf prog when 'ctx.optlen >
max_optlen' for now (and this can use a separate patch which as usual requires a bpf selftests)?Yeah, makes sense. Replacing this
-EFAULTwithWARN_ON_ONCEor something seems like the way to go. It caused too much trouble already :-(Should I prepare a patch or do you want to take a stab at it?
Quoted 6 lines
In the future, does it make sense to have a specific cgroup-bpf-prog (a specific attach type?) that only uses
bpf_dynptrkfunc to access the optval such that it can enforce read-only for some optname and potentially also track if bpf-prog has written a new optval? The bpf-prog can only return 1 (OK) and only allows usingbpf_set_retval()instead. Likely there is still holes but could be a seed of thought to continue polishing the idea.Ack, let's think about it.
Maybe we should re-evaluate 'getsockopt-happens-after-the-kernel' idea as well? If we can have a sleepable hook that can copy_from_user/copy_to_user, and we have a mostly working
bpf_getsockopt(after your refactoring), I don't see why we need to continue the current scheme of triggering after the kernel?
Quoted 7 lines
Quoted 4 lines
Since a sleepable hook would cause some restrictions, perhaps, we could introduce something like the promise pattern. In our case here, BPF program call an async version of copy_from_user()/copy_to_user() to return a promise.
Having a promise might work. This is essentially what we already do with sockets/etc with acquire/release pattern.
Would you mind to give me some context of the socket things?
Quoted 13 lines
What are the sleepable restrictions you're hinting about? I feel like with the sleepable bpf, we can also remove all the temporary buffer management / extra copies which sounds like a win to me. (we have this ugly heuristics with
BPF_SOCKOPT_KERN_BUF_SIZE) The program can allocate temporary buffers if needed..Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 4 lines
- or maybe we can even have a per-proto
bpf_getsockopt_cleanupcall that gets called whenever bpf returns an error to make sure protocols have a chance to handle that condition (and free the fd)
Re: handling unsupported optlen in cgroup bpf getsockopt: (was [PATCH net-next v4 2/4] net: socket: add sockopts blacklist for BPF cgroup hook)
Quoted 106 lines · Kui-Feng Lee
Quoted 102 lines · Stanislav Fomichev
Quoted 100 lines · Kui-Feng Lee
Quoted 84 lines · Stanislav Fomichev
Quoted 49 lines · Martin KaFai Lau
Quoted 37 lines · Stanislav Fomichev
Quoted 28 lines · Stanislav Fomichev
Quoted 26 lines · Aleksandr Mikhalitsyn
Quoted 11 lines · Eric Dumazet
Quoted 6 lines · Alexander Mikhalitsyn
During work on
SO_PEERPIDFD, it was discovered (thanks to Christian), that bpf cgroup hook can cause FD leaks when used with sockopts which install FDs into the process fdtable.After some offlist discussion it was proposed to add a blacklist of
We try to replace this word by either denylist or blocklist, even in changelogs.
Hi Eric,
Oh, I'm sorry about that. :( Sure.
Quoted 5 lines
Quoted 3 lines
socket options those can cause troubles when BPF cgroup hook is enabled.
Can we find the appropriate Fixes: tag to help stable teams ?
Sure, I will add next time.
Quoted 7 lines
Quoted 3 lines
Fixes 0d01da6afc54 ("bpf: implement getsockopt and setsockopt hooks")I think it's better to add Stanislav Fomichev to CC.
Can we use 'struct proto'
bpf_bypass_getsockoptinstead? We already use it for tcp zerocopy, I'm assuming it should work in this case as well?Quoted 7 lines
Jakub reminded me of the other things I wanted to ask here bug forgot:
- setsockopt is probably not needed, right? setsockopt hook triggers before the kernel and shouldn't leak anything
- for getsockopt, instead of bypassing bpf completely, should we instead ignore the error from the bpf program? that would still preserve the observability aspectstealing this thread to discuss the optlen issue which may make sense to bypass also.
Quoted 9 lines
There has been issue with optlen. Other than this older post related to optlen >
PAGE_SIZE:
https://lore.kernel.org/bpf/[email protected]/,
the recent one related to optlen that we have seen isNETLINK_LIST_MEMBERSHIPS. The userspace passed in optlen == 0 and the kernel put the expected optlen (> 0) and 'return 0;' to userspace. The userspace intention is to learn the expected optlen. This makes 'ctx.optlen >max_optlen' and__cgroup_bpf_run_filter_getsockopt()ends up returning-EFAULTto the userspace even the bpf prog has not changed anything.(ignoring
-EFAULTissue) this seems like it needs to beif (optval && (ctx.optlen > max_optlen || ctx.optlen < 0)) { /* error */ }?
Quoted 3 lines
Does it make sense to also bypass the bpf prog when 'ctx.optlen >
max_optlen' for now (and this can use a separate patch which as usual requires a bpf selftests)?Yeah, makes sense. Replacing this
-EFAULTwithWARN_ON_ONCEor something seems like the way to go. It caused too much trouble already :-(Should I prepare a patch or do you want to take a stab at it?
Quoted 6 lines
In the future, does it make sense to have a specific cgroup-bpf-prog (a specific attach type?) that only uses
bpf_dynptrkfunc to access the optval such that it can enforce read-only for some optname and potentially also track if bpf-prog has written a new optval? The bpf-prog can only return 1 (OK) and only allows usingbpf_set_retval()instead. Likely there is still holes but could be a seed of thought to continue polishing the idea.Quoted 7 lines
Ack, let's think about it.
Maybe we should re-evaluate 'getsockopt-happens-after-the-kernel' idea as well? If we can have a sleepable hook that can copy_from_user/copy_to_user, and we have a mostly working
bpf_getsockopt(after your refactoring), I don't see why we need to continue the current scheme of triggering after the kernel?Since a sleepable hook would cause some restrictions, perhaps, we could introduce something like the promise pattern. In our case here, BPF program call an async version of copy_from_user()/copy_to_user() to return a promise.
Quoted 4 lines
Quoted 2 lines
Having a promise might work. This is essentially what we already do with sockets/etc with acquire/release pattern.
Would you mind to give me some context of the socket things?
I'm mostly referring to the infra around KF_ACQUIRE/KF_RELEASE where the verifier already has some resource tracking functionality. We can probably extend it to verify that the program does copy_to_user equivalent at the end of the run (or somehow specifically marks that it's not needed).
Quoted 13 lines
Quoted 13 lines
What are the sleepable restrictions you're hinting about? I feel like with the sleepable bpf, we can also remove all the temporary buffer management / extra copies which sounds like a win to me. (we have this ugly heuristics with
BPF_SOCKOPT_KERN_BUF_SIZE) The program can allocate temporary buffers if needed..Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 4 lines
- or maybe we can even have a per-proto
bpf_getsockopt_cleanupcall that gets called whenever bpf returns an error to make sure protocols have a chance to handle that condition (and free the fd)
Jakub Kicinski
maintainerRe: [PATCH net-next v4 2/4] net: socket: add sockopts blacklist for BPF cgroup hook
In reply to Aleksandr MikhalitsynQuoted 5 lines · On Thu, 13 Apr 2023 16:38:35 +0200 Aleksandr Mikhalitsyn wrote:
Sure, I will add next time.
Fixes 0d01da6afc54 ("bpf: implement getsockopt and setsockopt hooks")I think it's better to add Stanislav Fomichev to CC.
This should go in separately as a fix, right? Not in a -next series.
Also the whole set as is does not build on top of net-next :(
PATCH v4 3/4 net: core: add getsockopt SO_PEERPIDFD
In reply to Alexander MikhalitsynAdd SO_PEERPIDFD which allows to get pidfd of peer socket holder pidfd. This thing is direct analog of SO_PEERCRED which allows to get plain PID.
---
v4:
- return -ESRCH if sk->sk_peer_pid is NULL from getsockopt() syscall
- return errors from pidfd_prepare() as it is from getsockopt() syscall
v3:
- fixed possible fd leak (thanks to Christian Brauner)
v2:
According to review comments from Kuniyuki Iwashima and Christian Brauner:
- use pidfd_create(..) retval as a result
- whitespace change
--- arch/alpha/include/uapi/asm/socket.h | 1 + arch/mips/include/uapi/asm/socket.h | 1 + arch/parisc/include/uapi/asm/socket.h | 1 + arch/sparc/include/uapi/asm/socket.h | 1 + include/uapi/asm-generic/socket.h | 1 + net/core/sock.c | 33 +++++++++++++++++++++++++ net/socket.c | 7 ++++++ tools/include/uapi/asm-generic/socket.h | 1 + 8 files changed, 46 insertions(+)
arch/alpha/include/uapi/asm/socket.h +1 -0
@@ -138,6 +138,7 @@#define SO_RCVMARK 75#define SO_PASSPIDFD 76 +#define SO_PEERPIDFD 77#if !defined(__KERNEL__)
arch/mips/include/uapi/asm/socket.h +1 -0
@@ -149,6 +149,7 @@#define SO_RCVMARK 75#define SO_PASSPIDFD 76 +#define SO_PEERPIDFD 77#if !defined(__KERNEL__)
arch/parisc/include/uapi/asm/socket.h +1 -0
@@ -130,6 +130,7 @@#define SO_RCVMARK 0x4049#define SO_PASSPIDFD 0x404A +#define SO_PEERPIDFD 0x404B#if !defined(__KERNEL__)
arch/sparc/include/uapi/asm/socket.h +1 -0
@@ -131,6 +131,7 @@#define SO_RCVMARK 0x0054#define SO_PASSPIDFD 0x0055 +#define SO_PEERPIDFD 0x0056#if !defined(__KERNEL__)
include/uapi/asm-generic/socket.h +1 -0
@@ -133,6 +133,7 @@#define SO_RCVMARK 75#define SO_PASSPIDFD 76 +#define SO_PEERPIDFD 77#if !defined(__KERNEL__)
net/core/sock.c +33 -0
@@ -1763,6 +1763,39 @@ int sk_getsockopt(struct sock *sk, int level, int optname, goto lenout; } +case SO_PEERPIDFD : + { +struct pid *peer_pid; +struct file *pidfd_file =NULL ; +int pidfd; + +if (len >sizeof (pidfd)) + len =sizeof (pidfd); + +spin_lock (&sk->sk_peer_lock); + peer_pid =get_pid (sk->sk_peer_pid); +spin_unlock (&sk->sk_peer_lock); + +if (!peer_pid) +return -ESRCH ; + + pidfd =pidfd_prepare (peer_pid,0 , &pidfd_file); +put_pid (peer_pid); +if (pidfd <0 ) +return pidfd; + +if (copy_to_sockptr (optval, &pidfd, len) || +copy_to_sockptr (optlen, &len,sizeof (int ))) { +put_unused_fd (pidfd); +fput (pidfd_file); + +return -EFAULT ; + } + +fd_install (pidfd, pidfd_file); +return 0 ; + } +case SO_PEERGROUPS : {const struct cred *cred;
net/socket.c +7 -0
@@ -2248,6 +2248,13 @@ static bool sockopt_installs_fd(int level, int optname)default :return false; } + }else if (level ==SOL_SOCKET ) { +switch (optname) { +case SO_PEERPIDFD : +return true; +default : +return false; + } }return false;
tools/include/uapi/asm-generic/socket.h +1 -0
@@ -122,6 +122,7 @@#define SO_RCVMARK 75#define SO_PASSPIDFD 76 +#define SO_PEERPIDFD 77#if !defined(__KERNEL__)
Signature · 1 line
2.34.1
Re: [PATCH net-next v4 3/4] net: core: add getsockopt SO_PEERPIDFD
Quoted 20 lines · Alexander Mikhalitsyn
Add
SO_PEERPIDFDwhich allows to get pidfd of peer socket holder pidfd. This thing is direct analog ofSO_PEERCREDwhich allows to get plain PID.Cc "David S. Miller" <[email protected]>Cc Eric Dumazet <[email protected]>Cc Jakub Kicinski <[email protected]>Cc Paolo Abeni <[email protected]>Cc Leon Romanovsky <[email protected]>Cc David Ahern <[email protected]>Cc Arnd Bergmann <[email protected]>Cc Kees Cook <[email protected]>Cc Christian Brauner <[email protected]>Cc Kuniyuki Iwashima <[email protected]>Cc Lennart Poettering <[email protected]>Cc Luca Boccassi <[email protected]>Tested-by Luca Boccassi <[email protected]>Signed-off-by Alexander Mikhalitsyn <[email protected]>
Quoted 4 lines
---
v4:
- return-ESRCHifsk->sk_peer_pidis NULL fromgetsockopt()syscall
- return errors frompidfd_prepare()as it is fromgetsockopt()syscall
I see no obvious issues anymore,
PATCH v4 4/4 selftests: net: add SCM_PIDFD / SO_PEERPIDFD test
In reply to Alexander MikhalitsynBasic test to check consistency between:
- SCM_CREDENTIALS and SCM_PIDFD
- SO_PEERCRED and SO_PEERPIDFD
---
v3:
- started using kselftest lib (thanks to Kuniyuki Iwashima for suggestion/review)
- now test covers abstract sockets too and SOCK_DGRAM sockets
--- tools/testing/selftests/net/.gitignore | 1 + tools/testing/selftests/net/af_unix/Makefile | 2 +- .../testing/selftests/net/af_unix/scm_pidfd.c | 430 ++++++++++++++++++ 3 files changed, 432 insertions(+), 1 deletion(-) create mode 100644 tools/testing/selftests/net/af_unix/scm_pidfd.c
tools/testing/selftests/net/.gitignore +1 -0
@@ -26,6 +26,7 @@ reuseport_bpf_cpu reuseport_bpf_numa reuseport_dualstack rxtimestamp +scm_pidfd sk_bind_sendto_listen sk_connect_zero_addr socket
tools/testing/selftests/net/af_unix/Makefile +1 -1
@@ -1,3 +1,3 @@ -TEST_GEN_PROGS := diag_uid test_unix_oob unix_connect +TEST_GEN_PROGS := diag_uid test_unix_oob unix_connect scm_pidfd include ../../lib.mk
tools/testing/selftests/net/af_unix/scm_pidfd.c new file +430 -0
430 lines not on this page · raw message
Signature · 1 line
2.34.1