bpf: add BPF_F_SEALED flag to seal BPF links at creation
PATCH 1/1 bpf: add BPF_F_SEALED flag to seal BPF links at creation
Introduce a new BPF_F_SEALED flag for BPF_LINK_CREATE that creates the link permanently sealed. A sealed link can never have its program replaced via BPF_LINK_UPDATE, can never be detached via BPF_LINK_DETACH, and holds an extra self-reference that is never released, so the link and its program attachment persist until the machine reboots, even after user space closes every fd referring to it. There is no way to unseal a link.
The sealed state is tracked by a new bool field on struct bpf_link.
--- include/linux/bpf.h | 2 ++ include/uapi/linux/bpf.h | 1 + kernel/bpf/syscall.c | 41 +++++++++++++++++++++++++++++++--- tools/include/uapi/linux/bpf.h | 1 + 4 files changed, 42 insertions(+), 3 deletions(-)
include/linux/bpf.h +2 -0
@@ -1934,6 +1934,8 @@ struct bpf_link {* link's semantics is determined by target attach hook */ bool sleepable; +/* set once by BPF_F_SEALED; blocks update/detach, pins link until reboot */ +bool sealed; };struct bpf_link_ops {
include/uapi/linux/bpf.h +1 -0
@@ -1251,6 +1251,7 @@ enum bpf_perf_event_type {#define BPF_F_AFTER (1U << 4)#define BPF_F_ID (1U << 5)#define BPF_F_PREORDER (1U << 6) +#define BPF_F_SEALED (1U << 7)#define BPF_F_LINK BPF_F_LINK/* 1 << 13 */ /* If BPF_F_STRICT_ALIGNMENT is used in BPF_PROG_LOAD command, the
kernel/bpf/syscall.c +38 -3
80 lines not on this page · raw message
tools/include/uapi/linux/bpf.h +1 -0
@@ -1251,6 +1251,7 @@ enum bpf_perf_event_type {#define BPF_F_AFTER (1U << 4)#define BPF_F_ID (1U << 5)#define BPF_F_PREORDER (1U << 6) +#define BPF_F_SEALED (1U << 7)#define BPF_F_LINK BPF_F_LINK/* 1 << 13 */ /* If BPF_F_STRICT_ALIGNMENT is used in BPF_PROG_LOAD command, the
base-commit: 28e911d61d66b92a3bded8b54622ed3cd2795bf6
Signature · 1 line
2.53.0
Quoted 10 lines · David Windsor
Introduce a new
BPF_F_SEALEDflag forBPF_LINK_CREATEthat creates the link permanently sealed. A sealed link can never have its program replaced viaBPF_LINK_UPDATE, can never be detached viaBPF_LINK_DETACH, and holds an extra self-reference that is never released, so the link and its program attachment persist until the machine reboots, even after user space closes every fd referring to it. There is no way to unseal a link.The sealed state is tracked by a new bool field on struct
bpf_link.
Quoted 8 lines
Signed-off-by David Windsor <[email protected]>--- include/linux/bpf.h | 2 ++ include/uapi/linux/bpf.h | 1 + kernel/bpf/syscall.c | 41 +++++++++++++++++++++++++++++++--- tools/include/uapi/linux/bpf.h | 1 + 4 files changed, 42 insertions(+), 3 deletions(-)
There is deliberately no selftest for this, as running any such selftest would necessarily create a sealed link, which cannot be cleaned up.
Kumar Kartikeya Dwivedi
maintainerQuoted 20 lines · David Windsor
Quoted 10 lines · David Windsor
Introduce a new
BPF_F_SEALEDflag forBPF_LINK_CREATEthat creates the link permanently sealed. A sealed link can never have its program replaced viaBPF_LINK_UPDATE, can never be detached viaBPF_LINK_DETACH, and holds an extra self-reference that is never released, so the link and its program attachment persist until the machine reboots, even after user space closes every fd referring to it. There is no way to unseal a link.The sealed state is tracked by a new bool field on struct
bpf_link.Quoted 8 lines
Signed-off-by David Windsor <[email protected]>--- include/linux/bpf.h | 2 ++ include/uapi/linux/bpf.h | 1 + kernel/bpf/syscall.c | 41 +++++++++++++++++++++++++++++++--- tools/include/uapi/linux/bpf.h | 1 + 4 files changed, 42 insertions(+), 3 deletions(-)
Quoted 3 lines
There is deliberately no selftest for this, as running any such selftest would necessarily create a sealed link, which cannot be cleaned up.
Let's wait for more comments, esp. on whether this is useful or the right path forward, but I'd still prefer adding selftests to verify various aspects of the behavior if we proceed.
I think the point about a user being able to create many sealed links is definitely valid too, so it might make sense to keep this gated behind more privileged capabilities (if not already).
For tests, we might have to force decrement the link refcount (after verifying it stays elevated after close(2)) from the test. It might involve some creative use of a kfunc in bpf_testmod that can do the cleanup for us to avoid wedging the system on repeated invocations of the test. It wouldn't be available on a real system. It depends on whether we deem it important enough that leaking the link on VMs running tests is important enough. You'd probably have to run it thousands of time to exhaust memory unless the VM is provisioned with a tiny amount of memory.
Anyway, let's get into all that after seeing how others feel about the concept.
[email protected]
AI reviewThank you for your contribution! Sashiko AI review found 3 potential issue(s) to consider: - [High] The BPF_F_SEALED flag is applied via a file descriptor lookup after the FD has been exposed to…
Thank you for your contribution! Sashiko AI review found 3 potential issue(s) to consider:
- [High] The BPF_F_SEALED flag is applied via a file descriptor lookup after the FD has been exposed to user space, creating a TOCTOU race that allows cross-object sabotage and unauthorized link sealing.
- [High] The implementation of BPF_F_SEALED introduces a permanent memory leak, enabling a Denial of Service (DoS) via kernel resource exhaustion.
- [Medium] Concurrent executions of link_seal_fd on the same link can leak multiple self-references due to a missing atomic read-modify-write.
--
--- Patch [1]: [PATCH bpf-next] bpf: add BPF_F_SEALED flag to seal BPF links at creation --- commit f9d98fb378be461fdf9509562339b45a07b8d912
Author: David Windsor <[email protected]>
bpf: add BPF_F_SEALED flag to seal BPF links at creation
This commit introduces a new BPF_F_SEALED flag for BPF_LINK_CREATE that creates a permanently sealed link. A sealed link cannot have its program replaced, cannot be detached, and holds an extra self-reference so it persists until reboot.
Quoted 4 lines
kernel/bpf/syscall.c +0
-0
[ ... ]
Quoted 4 lines
kernel/bpf/syscall.c +0
-0@@ -5776,17 +5777,41 @@ static int bpf_map_do_batch(const union bpf_attr *attr,return err; }
kernel/bpf/syscall.c static int bpf_map_do_batch(const union bpf_attr *attr, · line 5789
Quoted 10 lines
kernel/bpf/syscall.c +10
-0+/* Seal the just-created link: take a self-reference that is never released. */ +static void link_seal_fd (int fd) +{ +struct bpf_link *link; + + link =bpf_link_get_from_fd (fd); +if (IS_ERR (link)) +return ; + +if (!READ_ONCE (link->sealed)) {
[Severity: Medium]
Is this flag check safe against concurrent execution?
If the file descriptor is raced via dup2(), multiple threads might enter link_seal_fd() for the same victim link concurrently. Since the READ_ONCE() and WRITE_ONCE() are not atomic, could multiple threads observe false and increment the link reference count multiple times?
kernel/bpf/syscall.c static int bpf_map_do_batch(const union bpf_attr *attr, · line 5791
Quoted 2 lines
kernel/bpf/syscall.c +2
-0+bpf_link_inc (link); +WRITE_ONCE (link->sealed, true);
[Severity: High]
Does taking a self-reference here without any release path allow users to exhaust kernel memory?
Since link_detach() explicitly returns -EPERM for sealed links, a user could repeatedly create sealed links and close the fds, leaking the bpf_link and bpf_prog indefinitely. Is there an administrative way to reclaim this memory?
Quoted 4 lines
kernel/bpf/syscall.c +4
-0+ } + +bpf_link_put_direct (link); +}
[ ... ]
kernel/bpf/syscall.c static int link_create(union bpf_attr *attr, bpfptr_t uattr) · line 5910
Quoted 7 lines
kernel/bpf/syscall.c +3
-0@@ -5880,6 +5905,9 @@ static int link_create(union bpf_attr *attr, bpfptr_t uattr) out:if (ret <0 )bpf_prog_put (prog); +out_seal: +if (ret >=0 && seal) +link_seal_fd (ret);
[Severity: High]
Does looking up the link from the file descriptor here create a race condition?
By the time link_create() reaches this point, attach functions called earlier have already installed the fd into the process's file descriptor table.
Could a concurrent thread use dup2() to overwrite this fd with one pointing to a different victim bpf_link before link_seal_fd() executes? This would cause the system to permanently seal the victim link instead of the newly created one.
Quoted 2 lines
kernel/bpf/syscall.c +0
-0return ret; }
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=1
Quoted 8 lines · David Windsor
Introduce a new
BPF_F_SEALEDflag forBPF_LINK_CREATEthat creates the link permanently sealed. A sealed link can never have its program replaced viaBPF_LINK_UPDATE, can never be detached viaBPF_LINK_DETACH, and holds an extra self-reference that is never released, so the link and its program attachment persist until the machine reboots, even after user space closes every fd referring to it. There is no way to unseal a link.The sealed state is tracked by a new bool field on struct
bpf_link.
Why do you want BPF_F_SEALED? What's your case for it?
I think 'sealed' should be set before allocating the FD to avoid the race issues reported by Sashiko.
Thanks,
Leon
Quoted 1 line
[...]
Kumar Kartikeya Dwivedi
maintainerQuoted 8 lines · Leon Hwang
Quoted 7 lines · David Windsor
Introduce a new
BPF_F_SEALEDflag forBPF_LINK_CREATEthat creates the link permanently sealed. A sealed link can never have its program replaced viaBPF_LINK_UPDATE, can never be detached viaBPF_LINK_DETACH, and holds an extra self-reference that is never released, so the link and its program attachment persist until the machine reboots, even after user space closes every fd referring to it. There is no way to unseal a link.
Quoted 8 lines
Quoted 1 line
The sealed state is tracked by a new bool field on struct
bpf_link.Why do you want
BPF_F_SEALED? What's your case for it?I think 'sealed' should be set before allocating the FD to avoid the race issues reported by Sashiko.
Agreed, then we would not need READ_ONCE() on the sealed bit either since the it stays immutable through the lifetime of the link.
Quoted 4 lines
Thanks,
LeonQuoted 1 line
[...]
Quoted 9 lines · Leon Hwang
Quoted 7 lines · David Windsor
Introduce a new
BPF_F_SEALEDflag forBPF_LINK_CREATEthat creates the link permanently sealed. A sealed link can never have its program replaced viaBPF_LINK_UPDATE, can never be detached viaBPF_LINK_DETACH, and holds an extra self-reference that is never released, so the link and its program attachment persist until the machine reboots, even after user space closes every fd referring to it. There is no way to unseal a link.
Quoted 5 lines
Quoted 1 line
The sealed state is tracked by a new bool field on struct
bpf_link.Why do you want
BPF_F_SEALED? What's your case for it?
For the BPF LSM use case, mainly. There is some language in the commitmsg about programs not being able to be unloaded until reboot, but I will be more explicit in v2.
There was a presentation ("Securing BPF LSMs against tampering") [1] at LSFMM+BPF about it.
[1]: https://lwn.net/Articles/1082111/
Quoted 3 lines
I think 'sealed' should be set before allocating the FD to avoid the race issues reported by Sashiko.
Will do this in v2.
Quoted 4 lines
Thanks,
LeonQuoted 1 line
[...]
Quoted 17 lines · David Windsor
Quoted 9 lines · Leon Hwang
Quoted 7 lines · David Windsor
Introduce a new
BPF_F_SEALEDflag forBPF_LINK_CREATEthat creates the link permanently sealed. A sealed link can never have its program replaced viaBPF_LINK_UPDATE, can never be detached viaBPF_LINK_DETACH, and holds an extra self-reference that is never released, so the link and its program attachment persist until the machine reboots, even after user space closes every fd referring to it. There is no way to unseal a link.Quoted 5 lines
Quoted 1 line
The sealed state is tracked by a new bool field on struct
bpf_link.Why do you want
BPF_F_SEALED? What's your case for it?
Quoted 8 lines
For the BPF LSM use case, mainly. There is some language in the commitmsg about programs not being able to be unloaded until reboot, but I will be more explicit in v2.
There was a presentation ("Securing BPF LSMs against tampering") [1] at LSFMM+BPF about it.
I had to skim the articles to recall that it was me proposing something like this :)
Anyways, as Kumar mentioned, a) link has to be crated as sealed before FD is exposed, it's straightforward, and b) I do think that it would be too easy to have syzbots of the world to accidentally or not create tons of sealed links, so I'd require CAP_SYS_ADMIN for this, which I assume won't be a problem for intended use cases?
But also, yeah, I wonder what Christian and other BPF LSM users think about this.
pw-bot: cr
Quoted 11 lines
Quoted 4 lines
I think 'sealed' should be set before allocating the FD to avoid the race issues reported by Sashiko.
Will do this in v2.
Quoted 4 lines
Thanks,
LeonQuoted 1 line
[...]
Quoted 4 lines · Andrii Nakryiko
I had to skim the articles to recall that it was me proposing something like this :)
I wasn't actually there, but read about it on lwn. Happy to add your Suggested-By if that makes sense here.
Quoted 6 lines
Anyways, as Kumar mentioned, a) link has to be crated as sealed before FD is exposed, it's straightforward, and b) I do think that it would be too easy to have syzbots of the world to accidentally or not create tons of sealed links, so I'd require
CAP_SYS_ADMINfor this, which I assume won't be a problem for intended use cases?
CAP_SYS_ADMIN makes sense.
Quoted 3 lines
But also, yeah, I wonder what Christian and other BPF LSM users think about this.
I'll send v2 with the above changes.
The overall intent here is to follow up with another series for map sealing. Currently, we're passing BPF_F_SEALED as a flag for BPF_LINK_CREATE, but I wonder if we should decompose BPF_F_SEALED into eg BPF_F_LINK_SEALED and BPF_F_MAPS_SEALED?
Link sealing is useful on its own, though, for preserving bpf-lsm programs around even after all user fd's have been closed.
Kumar Kartikeya Dwivedi
maintainer+Cc Christian since he was involved in the discussion at LSF/MM/BPF.
Quoted 145 lines · David Windsor
Introduce a new
BPF_F_SEALEDflag forBPF_LINK_CREATEthat creates the link permanently sealed. A sealed link can never have its program replaced viaBPF_LINK_UPDATE, can never be detached viaBPF_LINK_DETACH, and holds an extra self-reference that is never released, so the link and its program attachment persist until the machine reboots, even after user space closes every fd referring to it. There is no way to unseal a link.The sealed state is tracked by a new bool field on struct
bpf_link.Signed-off-by David Windsor <[email protected]>--- include/linux/bpf.h | 2 ++ include/uapi/linux/bpf.h | 1 + kernel/bpf/syscall.c | 41 +++++++++++++++++++++++++++++++--- tools/include/uapi/linux/bpf.h | 1 + 4 files changed, 42 insertions(+), 3 deletions(-)include/linux/bpf.h +2
-0@@ -1934,6 +1934,8 @@ struct bpf_link {* link's semantics is determined by target attach hook */ bool sleepable; +/* set once by BPF_F_SEALED; blocks update/detach, pins link until reboot */ +bool sealed; };struct bpf_link_ops {include/uapi/linux/bpf.h +1
-0@@ -1251,6 +1251,7 @@ enum bpf_perf_event_type {#define BPF_F_AFTER (1U << 4)#define BPF_F_ID (1U << 5)#define BPF_F_PREORDER (1U << 6) +#define BPF_F_SEALED (1U << 7)#define BPF_F_LINK BPF_F_LINK/* 1 << 13 */ /* If BPF_F_STRICT_ALIGNMENT is used in BPF_PROG_LOAD command, the kernel/bpf/syscall.c +38
-3@@ -3411,6 +3411,7 @@ static void bpf_link_show_fdinfo(struct seq_file *m, struct file *filp)seq_printf (m,"link_type:\t<%u>\n" , type); }seq_printf (m,"link_id:\t%u\n" , link->id); +seq_printf (m,"sealed:\t%d\n" ,READ_ONCE (link->sealed) ?1 :0 );rcu_read_lock (); prog =READ_ONCE (link->prog);@@ -5776,17 +5777,41 @@ static int bpf_map_do_batch(const union bpf_attr *attr,return err; } +/* Seal the just-created link: take a self-reference that is never released. */ +static void link_seal_fd (int fd) +{ +struct bpf_link *link; + + link =bpf_link_get_from_fd (fd); +if (IS_ERR (link)) +return ; + +if (!READ_ONCE (link->sealed)) { +bpf_link_inc (link); +WRITE_ONCE (link->sealed, true); + } + +bpf_link_put_direct (link); +} +#define BPF_LINK_CREATE_LAST_FIELD link_create.uprobe_multi.path_fdstatic int link_create (union bpf_attr *attr,bpfptr_t uattr) {struct bpf_prog *prog; +bool seal;int ret;if (CHECK_ATTR (BPF_LINK_CREATE ))return -EINVAL ; -if (attr->link_create.attach_type ==BPF_STRUCT_OPS ) -return bpf_struct_ops_link_create (attr); +/* Strip BPF_F_SEALED before per-type flag validation. */ + seal = attr->link_create.flags &BPF_F_SEALED ; + attr->link_create.flags &= ~BPF_F_SEALED ; + +if (attr->link_create.attach_type ==BPF_STRUCT_OPS ) { + ret =bpf_struct_ops_link_create (attr); + goto out_seal; + } prog =bpf_prog_get (attr->link_create.prog_fd);if (IS_ERR (prog))@@ -5880,6 +5905,9 @@ static int link_create(union bpf_attr *attr, bpfptr_t uattr) out:if (ret <0 )bpf_prog_put (prog); +out_seal: +if (ret >=0 && seal) +link_seal_fd (ret);return ret; }@@ -5932,6 +5960,11 @@ static int link_update(union bpf_attr *attr)if (IS_ERR (link))return PTR_ERR (link); +if (READ_ONCE (link->sealed)) { + ret = -EPERM ; + goto out_put_link; + } +if (link->ops->update_map) { ret =link_update_map (link, attr); goto out_put_link;@@ -5984,7 +6017,9 @@ static int link_detach(union bpf_attr *attr)if (IS_ERR (link))return PTR_ERR (link); -if (link->ops->detach) +if (READ_ONCE (link->sealed)) + ret = -EPERM ; +else if (link->ops->detach) ret = link->ops->detach (link);else ret = -EOPNOTSUPP ;tools/include/uapi/linux/bpf.h +1
-0@@ -1251,6 +1251,7 @@ enum bpf_perf_event_type {#define BPF_F_AFTER (1U << 4)#define BPF_F_ID (1U << 5)#define BPF_F_PREORDER (1U << 6) +#define BPF_F_SEALED (1U << 7)#define BPF_F_LINK BPF_F_LINK/* 1 << 13 */ /* If BPF_F_STRICT_ALIGNMENT is used in BPF_PROG_LOAD command, the base-commit:
28e911d61d66b92a3bded8b54622ed3cd2795bf6