libbpf: call dup2() syscall directly
Andrii Nakryiko
maintainerPATCH 1/1 libbpf: call dup2() syscall directly
We've ran into issues with using dup2() API in production setting, where libbpf is linked into large production environment and ends up calling uninteded custom implementations of dup2(). These custom implementations don't provide atomic FD replacement guarantees of dup2() syscall, leading to subtle and hard to debug issues.
To prevent this in the future and guarantee that no libc implementation will do their own custom non-atomic dup2() implementation, call dup2() syscall directly with syscall(SYS_dup2).
Note that some architectures don't seem to provide dup2 and have dup3 instead. Try to detect and pick best syscall.
--- tools/lib/bpf/libbpf_internal.h | 12 +++++++++++- 1 file changed, 11 insertions(+), 1 deletion(-)
tools/lib/bpf/libbpf_internal.h +11 -1
@@ -15,6 +15,7 @@#include <linux/err.h> #include <fcntl.h> #include <unistd.h> +#include <sys/syscall.h> #include <libelf.h> #include "relo_core.h"
@@ -555,6 +556,15 @@ static inline int ensure_good_fd(int fd)return fd; } +static inline int sys_dup2 (int oldfd,int newfd) +{ +#ifdef __NR_dup2 +return syscall (__NR_dup2, oldfd, newfd); +#else +return syscall (__NR_dup3, oldfd, newfd,0 ); +#endif +} +/* Point *fixed_fd* to the same file that *tmp_fd* points to. * Regardless of success, *tmp_fd* is closed. * Whatever *fixed_fd* pointed to is closed silently.
@@ -563,7 +573,7 @@ static inline int reuse_fd(int fixed_fd, int tmp_fd) {int err; - err =dup2 (tmp_fd, fixed_fd); + err =sys_dup2 (tmp_fd, fixed_fd); err = err <0 ? -errno :0 ;close (tmp_fd);/* clean up temporary FD */ return err;
Signature · 1 line
2.34.1
Quoted 4 lines · Andrii Nakryiko
We've ran into issues with using
dup2()API in production setting, where libbpf is linked into large production environment and ends up calling uninteded custom implementations ofdup2(). These custom implementations
typo: unintended
Quoted 9 lines
don't provide atomic FD replacement guarantees of
dup2()syscall, leading to subtle and hard to debug issues.To prevent this in the future and guarantee that no libc implementation will do their own custom non-atomic
dup2()implementation, calldup2()syscall directly withsyscall(SYS_dup2).Note that some architectures don't seem to provide dup2 and have dup3 instead. Try to detect and pick best syscall.
I wonder whether we can just always use dup3().
Quoted 1 line
Signed-off-by Andrii Nakryiko <[email protected]>
[...]
Quoted 8 lines · Song Liu
Quoted 4 lines · Andrii Nakryiko
We've ran into issues with using
dup2()API in production setting, where libbpf is linked into large production environment and ends up calling uninteded custom implementations ofdup2(). These custom implementationstypo: unintended
oops, but probably doesn't warrant respinning
Quoted 4 lines
Quoted 4 lines
don't provide atomic FD replacement guarantees of
dup2()syscall, leading to subtle and hard to debug issues.
Quoted 8 lines
Quoted 6 lines
To prevent this in the future and guarantee that no libc implementation will do their own custom non-atomic
dup2()implementation, calldup2()syscall directly withsyscall(SYS_dup2).Note that some architectures don't seem to provide dup2 and have dup3 instead. Try to detect and pick best syscall.
I wonder whether we can just always use
dup3().
dup3() (according to my git foo) was added in 4.17, which is more modern than some other usable BPF, so I don't want to just randomly bump the minimal supported (by libbpf) kernel for something like this.
Quoted 6 lines
Quoted 2 lines
Signed-off-by Andrii Nakryiko <[email protected]>Acked-by Song Liu <[email protected]>[...]
Quoted 21 lines · Andrii Nakryiko
Quoted 8 lines · Song Liu
Quoted 4 lines · Andrii Nakryiko
We've ran into issues with using
dup2()API in production setting, where libbpf is linked into large production environment and ends up calling uninteded custom implementations ofdup2(). These custom implementationstypo: unintended
oops, but probably doesn't warrant respinning
Quoted 8 lines
Quoted 8 lines
don't provide atomic FD replacement guarantees of
dup2()syscall, leading to subtle and hard to debug issues.To prevent this in the future and guarantee that no libc implementation will do their own custom non-atomic
dup2()implementation, calldup2()syscall directly withsyscall(SYS_dup2).
Quoted 9 lines
Quoted 4 lines
Quoted 2 lines
Note that some architectures don't seem to provide dup2 and have dup3 instead. Try to detect and pick best syscall.
I wonder whether we can just always use
dup3().
dup3()(according to my git foo) was added in 4.17, which is more modern than some other usable BPF, so I don't want to just randomly bump the minimal supported (by libbpf) kernel for something like this.
Btw, this #ifdef check is the same as what glibc does for its implementation of dup2() (except for fd equality check which isn't necessary for libbpf), see [0]
[0] https://github.com/bminor/glibc/blob/master/sysdeps/unix/sysv/linux/dup2.c
Quoted 6 lines
Quoted 6 lines
Quoted 2 lines
Signed-off-by Andrii Nakryiko <[email protected]>Acked-by Song Liu <[email protected]>[...]
Song Liu
reviewerQuoted 27 lines · Andrii Nakryiko
Quoted 24 lines · Andrii Nakryiko
Quoted 8 lines · Song Liu
Quoted 4 lines · Andrii Nakryiko
We've ran into issues with using
dup2()API in production setting, where libbpf is linked into large production environment and ends up calling uninteded custom implementations ofdup2(). These custom implementationstypo: unintended
oops, but probably doesn't warrant respinning
Quoted 11 lines
Quoted 10 lines
don't provide atomic FD replacement guarantees of
dup2()syscall, leading to subtle and hard to debug issues.To prevent this in the future and guarantee that no libc implementation will do their own custom non-atomic
dup2()implementation, calldup2()syscall directly withsyscall(SYS_dup2).Note that some architectures don't seem to provide dup2 and have dup3 instead. Try to detect and pick best syscall.
Quoted 6 lines
Quoted 6 lines
Quoted 1 line
I wonder whether we can just always use
dup3().
dup3()(according to my git foo) was added in 4.17, which is more modern than some other usable BPF, so I don't want to just randomly bump the minimal supported (by libbpf) kernel for something like this.
I believe dup3() was added in 3.7.
Quoted 6 lines
Btw, this #ifdef check is the same as what glibc does for its implementation of
dup2()(except for fd equality check which isn't necessary for libbpf), see [0][0] https://github.com/bminor/glibc/blob/master/sysdeps/unix/sysv/linux/dup2.c
Yep, this looks good.
Thanks,
Song
Quoted 30 lines · Song Liu
Quoted 27 lines · Andrii Nakryiko
Quoted 24 lines · Andrii Nakryiko
Quoted 8 lines · Song Liu
Quoted 4 lines · Andrii Nakryiko
We've ran into issues with using
dup2()API in production setting, where libbpf is linked into large production environment and ends up calling uninteded custom implementations ofdup2(). These custom implementationstypo: unintended
oops, but probably doesn't warrant respinning
Quoted 11 lines
Quoted 10 lines
don't provide atomic FD replacement guarantees of
dup2()syscall, leading to subtle and hard to debug issues.To prevent this in the future and guarantee that no libc implementation will do their own custom non-atomic
dup2()implementation, calldup2()syscall directly withsyscall(SYS_dup2).Note that some architectures don't seem to provide dup2 and have dup3 instead. Try to detect and pick best syscall.
Quoted 8 lines
Quoted 6 lines
Quoted 6 lines
Quoted 1 line
I wonder whether we can just always use
dup3().
dup3()(according to my git foo) was added in 4.17, which is more modern than some other usable BPF, so I don't want to just randomly bump the minimal supported (by libbpf) kernel for something like this.I believe
dup3()was added in 3.7.
True, my git-foo isn't careful enough, 4.17 is when dup3 kernel refactoring happened. bpf() syscall was added in 3.17, right? In that case, yep, I could have just gone with __NR_dup3 directly, I suppose, but this version should work well anyways, so I wouldn't bother changing it.
Quoted 12 lines
Quoted 7 lines
Btw, this #ifdef check is the same as what glibc does for its implementation of
dup2()(except for fd equality check which isn't necessary for libbpf), see [0][0] https://github.com/bminor/glibc/blob/master/sysdeps/unix/sysv/linux/dup2.c
Yep, this looks good.
Thanks,
Song
Quoted 6 lines · Andrii Nakryiko
We've ran into issues with using
dup2()API in production setting, where libbpf is linked into large production environment and ends up calling uninteded custom implementations ofdup2(). These custom implementations don't provide atomic FD replacement guarantees ofdup2()syscall, leading to subtle and hard to debug issues.
Quoted 8 lines
To prevent this in the future and guarantee that no libc implementation will do their own custom non-atomic
dup2()implementation, calldup2()syscall directly withsyscall(SYS_dup2).Note that some architectures don't seem to provide dup2 and have dup3 instead. Try to detect and pick best syscall.
Signed-off-by Andrii Nakryiko <[email protected]>
This patch was applied to bpf/bpf-next.git (master) by Alexei Starovoitov <[email protected]>:
Hello:
This patch was applied to bpf/bpf-next.git (master) by Alexei Starovoitov <[email protected]>:
Quoted 11 lines · On Fri, 19 Jan 2024 13:02:01 -0800 you wrote:
We've ran into issues with using
dup2()API in production setting, where libbpf is linked into large production environment and ends up calling uninteded custom implementations ofdup2(). These custom implementations don't provide atomic FD replacement guarantees ofdup2()syscall, leading to subtle and hard to debug issues.To prevent this in the future and guarantee that no libc implementation will do their own custom non-atomic
dup2()implementation, calldup2()syscall directly withsyscall(SYS_dup2).[...]
Here is the summary with links:
- [bpf-next] libbpf: call dup2() syscall directly
https://git.kernel.org/bpf/bpf-next/c/bc308d011ab8
You are awesome, thank you!
Signature · 2 lines
Deet-doot-dot, I am a bot. https://korg.docs.kernel.org/patchwork/pwbot.html