Skip to contentSkip to conversation

libbpf: call dup2() syscall directly

PATCH Applied Andrii Nakryiko 5 participants 8 messages Jan 19, 2024 – Jan 23, 2024

Subsystems (2)
lore
Share & tools
lore

Andrii Nakryiko

author maintainer
#0 lore raw

PATCH 1/1 libbpf: call dup2() syscall directly

We've ran into issues with using dup2() API in production setting, where libbpf is linked into large production environment and ends up calling uninteded custom implementations of dup2(). These custom implementations don't provide atomic FD replacement guarantees of dup2() syscall, leading to subtle and hard to debug issues.

To prevent this in the future and guarantee that no libc implementation will do their own custom non-atomic dup2() implementation, call dup2() syscall directly with syscall(SYS_dup2).

Note that some architectures don't seem to provide dup2 and have dup3 instead. Try to detect and pick best syscall.

---
 tools/lib/bpf/libbpf_internal.h | 12 +++++++++++-
 1 file changed, 11 insertions(+), 1 deletion(-)
tools/lib/bpf/libbpf_internal.h +11 -1
index 27e4e320e1a6..58c547d473e0 100644
@@ -15,6 +15,7 @@
 #include <linux/err.h>
 #include <fcntl.h>
 #include <unistd.h>
+#include <sys/syscall.h>
 #include <libelf.h>
 #include "relo_core.h"
 
@@ -555,6 +556,15 @@ static inline int ensure_good_fd(int fd)
 	return fd;
 }
 
+static inline int sys_dup2(int oldfd, int newfd)
+{
+#ifdef __NR_dup2
+	return syscall(__NR_dup2, oldfd, newfd);
+#else
+	return syscall(__NR_dup3, oldfd, newfd, 0);
+#endif
+}
+
 /* Point *fixed_fd* to the same file that *tmp_fd* points to.
  * Regardless of success, *tmp_fd* is closed.
  * Whatever *fixed_fd* pointed to is closed silently.
@@ -563,7 +573,7 @@ static inline int reuse_fd(int fixed_fd, int tmp_fd)
 {
 	int err;
 
-	err = dup2(tmp_fd, fixed_fd);
+	err = sys_dup2(tmp_fd, fixed_fd);
 	err = err < 0 ? -errno : 0;
 	close(tmp_fd); /* clean up temporary FD */
 	return err;
Signature · 1 line
2.34.1
reviewer Acked-by on v1 1/1
#1 lore raw
Quoted 4 lines · Andrii Nakryiko

We've ran into issues with using dup2() API in production setting, where libbpf is linked into large production environment and ends up calling uninteded custom implementations of dup2(). These custom implementations

typo: unintended

Quoted 9 lines

don't provide atomic FD replacement guarantees of dup2() syscall, leading to subtle and hard to debug issues.

To prevent this in the future and guarantee that no libc implementation will do their own custom non-atomic dup2() implementation, call dup2() syscall directly with syscall(SYS_dup2).

Note that some architectures don't seem to provide dup2 and have dup3 instead. Try to detect and pick best syscall.

I wonder whether we can just always use dup3().

Quoted 1 line

[...]

#2 lore raw
Quoted 8 lines · Song Liu
Quoted 4 lines · Andrii Nakryiko

We've ran into issues with using dup2() API in production setting, where libbpf is linked into large production environment and ends up calling uninteded custom implementations of dup2(). These custom implementations

typo: unintended

oops, but probably doesn't warrant respinning

Quoted 4 lines
Quoted 4 lines

don't provide atomic FD replacement guarantees of dup2() syscall, leading to subtle and hard to debug issues.

Quoted 8 lines
Quoted 6 lines

To prevent this in the future and guarantee that no libc implementation will do their own custom non-atomic dup2() implementation, call dup2() syscall directly with syscall(SYS_dup2).

Note that some architectures don't seem to provide dup2 and have dup3 instead. Try to detect and pick best syscall.

I wonder whether we can just always use dup3().

dup3() (according to my git foo) was added in 4.17, which is more modern than some other usable BPF, so I don't want to just randomly bump the minimal supported (by libbpf) kernel for something like this.

Quoted 6 lines
Quoted 2 lines

[...]

#3 lore raw
Quoted 21 lines · Andrii Nakryiko
Quoted 8 lines · Song Liu
Quoted 4 lines · Andrii Nakryiko

We've ran into issues with using dup2() API in production setting, where libbpf is linked into large production environment and ends up calling uninteded custom implementations of dup2(). These custom implementations

typo: unintended

oops, but probably doesn't warrant respinning

Quoted 8 lines
Quoted 8 lines

don't provide atomic FD replacement guarantees of dup2() syscall, leading to subtle and hard to debug issues.

To prevent this in the future and guarantee that no libc implementation will do their own custom non-atomic dup2() implementation, call dup2() syscall directly with syscall(SYS_dup2).

Quoted 9 lines
Quoted 4 lines
Quoted 2 lines

Note that some architectures don't seem to provide dup2 and have dup3 instead. Try to detect and pick best syscall.

I wonder whether we can just always use dup3().

dup3() (according to my git foo) was added in 4.17, which is more modern than some other usable BPF, so I don't want to just randomly bump the minimal supported (by libbpf) kernel for something like this.

Btw, this #ifdef check is the same as what glibc does for its implementation of dup2() (except for fd equality check which isn't necessary for libbpf), see [0]

[0] https://github.com/bminor/glibc/blob/master/sysdeps/unix/sysv/linux/dup2.c

Quoted 6 lines
Quoted 6 lines
Quoted 2 lines

[...]

reviewer
#4 lore raw
Quoted 27 lines · Andrii Nakryiko
Quoted 24 lines · Andrii Nakryiko
Quoted 8 lines · Song Liu
Quoted 4 lines · Andrii Nakryiko

We've ran into issues with using dup2() API in production setting, where libbpf is linked into large production environment and ends up calling uninteded custom implementations of dup2(). These custom implementations

typo: unintended

oops, but probably doesn't warrant respinning

Quoted 11 lines
Quoted 10 lines

don't provide atomic FD replacement guarantees of dup2() syscall, leading to subtle and hard to debug issues.

To prevent this in the future and guarantee that no libc implementation will do their own custom non-atomic dup2() implementation, call dup2() syscall directly with syscall(SYS_dup2).

Note that some architectures don't seem to provide dup2 and have dup3 instead. Try to detect and pick best syscall.

Quoted 6 lines
Quoted 6 lines
Quoted 1 line

I wonder whether we can just always use dup3().

dup3() (according to my git foo) was added in 4.17, which is more modern than some other usable BPF, so I don't want to just randomly bump the minimal supported (by libbpf) kernel for something like this.

I believe dup3() was added in 3.7.

Quoted 6 lines

Btw, this #ifdef check is the same as what glibc does for its implementation of dup2() (except for fd equality check which isn't necessary for libbpf), see [0]

[0] https://github.com/bminor/glibc/blob/master/sysdeps/unix/sysv/linux/dup2.c

Yep, this looks good.

Thanks,
Song

#5 lore raw
Quoted 30 lines · Song Liu
Quoted 27 lines · Andrii Nakryiko
Quoted 24 lines · Andrii Nakryiko
Quoted 8 lines · Song Liu
Quoted 4 lines · Andrii Nakryiko

We've ran into issues with using dup2() API in production setting, where libbpf is linked into large production environment and ends up calling uninteded custom implementations of dup2(). These custom implementations

typo: unintended

oops, but probably doesn't warrant respinning

Quoted 11 lines
Quoted 10 lines

don't provide atomic FD replacement guarantees of dup2() syscall, leading to subtle and hard to debug issues.

To prevent this in the future and guarantee that no libc implementation will do their own custom non-atomic dup2() implementation, call dup2() syscall directly with syscall(SYS_dup2).

Note that some architectures don't seem to provide dup2 and have dup3 instead. Try to detect and pick best syscall.

Quoted 8 lines
Quoted 6 lines
Quoted 6 lines
Quoted 1 line

I wonder whether we can just always use dup3().

dup3() (according to my git foo) was added in 4.17, which is more modern than some other usable BPF, so I don't want to just randomly bump the minimal supported (by libbpf) kernel for something like this.

I believe dup3() was added in 3.7.

True, my git-foo isn't careful enough, 4.17 is when dup3 kernel refactoring happened. bpf() syscall was added in 3.17, right? In that case, yep, I could have just gone with __NR_dup3 directly, I suppose, but this version should work well anyways, so I wouldn't bother changing it.

Quoted 12 lines
Quoted 7 lines

Btw, this #ifdef check is the same as what glibc does for its implementation of dup2() (except for fd equality check which isn't necessary for libbpf), see [0]

[0] https://github.com/bminor/glibc/blob/master/sysdeps/unix/sysv/linux/dup2.c

Yep, this looks good.

Thanks,
Song

reviewer Acked-by on v1 1/1
#6 lore raw
In reply to Andrii Nakryiko
Quoted 6 lines · Andrii Nakryiko

We've ran into issues with using dup2() API in production setting, where libbpf is linked into large production environment and ends up calling uninteded custom implementations of dup2(). These custom implementations don't provide atomic FD replacement guarantees of dup2() syscall, leading to subtle and hard to debug issues.

Quoted 8 lines

To prevent this in the future and guarantee that no libc implementation will do their own custom non-atomic dup2() implementation, call dup2() syscall directly with syscall(SYS_dup2).

Note that some architectures don't seem to provide dup2 and have dup3 instead. Try to detect and pick best syscall.

#7 lore raw
In reply to Andrii Nakryiko
This patch was applied to bpf/bpf-next.git (master) by Alexei Starovoitov <[email protected]>:

Hello:

This patch was applied to bpf/bpf-next.git (master) by Alexei Starovoitov <[email protected]>:

Quoted 11 lines · On Fri, 19 Jan 2024 13:02:01 -0800 you wrote:

We've ran into issues with using dup2() API in production setting, where libbpf is linked into large production environment and ends up calling uninteded custom implementations of dup2(). These custom implementations don't provide atomic FD replacement guarantees of dup2() syscall, leading to subtle and hard to debug issues.

To prevent this in the future and guarantee that no libc implementation will do their own custom non-atomic dup2() implementation, call dup2() syscall directly with syscall(SYS_dup2).

[...]

Here is the summary with links:
- [bpf-next] libbpf: call dup2() syscall directly

    https://git.kernel.org/bpf/bpf-next/c/bc308d011ab8

You are awesome, thank you!

Signature · 2 lines
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/patchwork/pwbot.html