Skip to contentSkip to conversation

bpf: Release module BTF IDR before module unload

PATCH Superseded Kumar Kartikeya Dwivedi 3 participants 6 messages Mar 12 BPF

v1 of 3 Superseded Latest is v3 →

lore
Share & tools
lore

Kumar Kartikeya Dwivedi

author maintainer
#0 lore raw

PATCH 1/1 bpf: Release module BTF IDR before module unload

Gregory reported in [0] that the global_map_resize test when run in repeatedly ends up failing during program load. This stems from the fact that BTF reference has not dropped to zero after the previous run's module is unloaded, and the older module's BTF is still discoverable and visible. Later, in libbpf, load_module_btfs() will find the ID for this stale BTF, open its fd, and then it will be used during program load where later steps taking module reference using btf_try_get_module() fail since the underlying module for the BTF is gone.

Logically, once a module is unloaded, it's associated BTF artifacts should become hidden. The BTF object inside the kernel may still remain alive as long its reference counts are alive, but it should no longer be discoverable.

To fix this, let us call btf_free_id() from the MODULE_STATE_GOING case for the module unload to free the BTF associated IDR entry, and disable its discovery once module unload returns to user space. If a race happens during unload, the outcome is non-deterministic anyway. However, user space should be able to rely on the guarantee that once it has synchronously established a successful module unload, no more stale artifacts associated with this module can be obtained subsequently.

Note that we must be careful to not invoke btf_free_id() in btf_put() when btf_is_module() is true now. There could be a window where the module unload drops a non-terminal reference, frees the IDR, but the same ID gets reused and the second unconditional btf_free_id() ends up releasing an unrelated entry.

We can be sure that by the time we reach final btf_put() for btf_is_module() case, the btf_free_id() is already done, since the module itself holds the BTF reference, and it will call this function for the BTF before dropping its own reference.

[0]: https://lore.kernel.org/bpf/[email protected]

Fixes 36e68442d1af ("bpf: Load and verify kernel module BTFs")
---
 kernel/bpf/btf.c | 15 ++++++++++++++-
 1 file changed, 14 insertions(+), 1 deletion(-)
kernel/bpf/btf.c +14 -1
index 4872d2a6c42d..c44fb9c9386f 100644
@@ -1883,10 +1883,17 @@ void btf_get(struct btf *btf)
 	refcount_inc(&btf->refcnt);
 }
 
+bool btf_is_module(const struct btf *btf);
+
 void btf_put(struct btf *btf)
 {
 	if (btf && refcount_dec_and_test(&btf->refcnt)) {
-		btf_free_id(btf);
+		/*
+		 * IDR for module BTFs is freed when module's BTF reference is
+		 * dropped, which _will_ happen before the final put.
+		 */
+		if (!btf_is_module(btf))
+			btf_free_id(btf);
 		call_rcu(&btf->rcu, btf_free_rcu);
 	}
 }
@@ -8382,6 +8389,12 @@ static int btf_module_notify(struct notifier_block *nb, unsigned long op,
 			if (btf_mod->module != module)
 				continue;
 
+			/*
+			 * For modules, we do the freeing of BTF IDR as soon as
+			 * module goes away to disable BTF discovery, since the
+			 * btf_try_get_module() on such BTFs will fail.
+			 */
+			btf_free_id(btf_mod->btf);
 			list_del(&btf_mod->list);
 			if (btf_mod->sysfs_attr)
 				sysfs_remove_bin_file(btf_kobj, btf_mod->sysfs_attr);

--
2.52.0

reviewer Reviewed-by on v1 1/1
#1 lore raw
Quoted 33 lines · Kumar Kartikeya Dwivedi

Gregory reported in [0] that the global_map_resize test when run in repeatedly ends up failing during program load. This stems from the fact that BTF reference has not dropped to zero after the previous run's module is unloaded, and the older module's BTF is still discoverable and visible. Later, in libbpf, load_module_btfs() will find the ID for this stale BTF, open its fd, and then it will be used during program load where later steps taking module reference using btf_try_get_module() fail since the underlying module for the BTF is gone.

Logically, once a module is unloaded, it's associated BTF artifacts should become hidden. The BTF object inside the kernel may still remain alive as long its reference counts are alive, but it should no longer be discoverable.

To fix this, let us call btf_free_id() from the MODULE_STATE_GOING case for the module unload to free the BTF associated IDR entry, and disable its discovery once module unload returns to user space. If a race happens during unload, the outcome is non-deterministic anyway. However, user space should be able to rely on the guarantee that once it has synchronously established a successful module unload, no more stale artifacts associated with this module can be obtained subsequently.

Note that we must be careful to not invoke btf_free_id() in btf_put() when btf_is_module() is true now. There could be a window where the module unload drops a non-terminal reference, frees the IDR, but the same ID gets reused and the second unconditional btf_free_id() ends up releasing an unrelated entry.

We can be sure that by the time we reach final btf_put() for btf_is_module() case, the btf_free_id() is already done, since the module itself holds the BTF reference, and it will call this function for the BTF before dropping its own reference.

Quoted 7 lines

[0]: https://lore.kernel.org/bpf/[email protected]

Fixes 36e68442d1af ("bpf: Load and verify kernel module BTFs")

---

kernel/bpf/btf.c void btf_get(struct btf *btf) · line 1886

Quoted 12 lines
 kernel/bpf/btf.c | 15 ++++++++++++++-
 1 file changed, 14 insertions(+), 1 deletion(-)
kernel/bpf/btf.c +1 -0
index 4872d2a6c42d..c44fb9c9386f 100644
@@ -1883,10 +1883,17 @@ void btf_get(struct btf *btf)
 	refcount_inc(&btf->refcnt);
 }
 
+bool btf_is_module(const struct btf *btf);

Nit: Maybe just move the entire btf_is_module function to here? It's a one-liner anyway.

Quoted 29 lines
kernel/bpf/btf.c +13 -2
+
 void btf_put(struct btf *btf)
 {
 	if (btf && refcount_dec_and_test(&btf->refcnt)) {
-		btf_free_id(btf);
+		/*
+		 * IDR for module BTFs is freed when module's BTF reference is
+		 * dropped, which _will_ happen before the final put.
+		 */
+		if (!btf_is_module(btf))
+			btf_free_id(btf);
 		call_rcu(&btf->rcu, btf_free_rcu);
 	}
 }
@@ -8382,6 +8389,12 @@ static int btf_module_notify(struct notifier_block *nb, unsigned long op,
 			if (btf_mod->module != module)
 				continue;
 
+			/*
+			 * For modules, we do the freeing of BTF IDR as soon as
+			 * module goes away to disable BTF discovery, since the
+			 * btf_try_get_module() on such BTFs will fail.
+			 */
+			btf_free_id(btf_mod->btf);
 			list_del(&btf_mod->list);
 			if (btf_mod->sysfs_attr)
 				sysfs_remove_bin_file(btf_kobj, btf_mod->sysfs_attr);
--

2.52.0

#2 lore raw
In reply to Kumar Kartikeya Dwivedi
Quoted 23 lines · Kumar Kartikeya Dwivedi

Gregory reported in [0] that the global_map_resize test when run in repeatedly ends up failing during program load. This stems from the fact that BTF reference has not dropped to zero after the previous run's module is unloaded, and the older module's BTF is still discoverable and visible. Later, in libbpf, load_module_btfs() will find the ID for this stale BTF, open its fd, and then it will be used during program load where later steps taking module reference using btf_try_get_module() fail since the underlying module for the BTF is gone.

Logically, once a module is unloaded, it's associated BTF artifacts should become hidden. The BTF object inside the kernel may still remain alive as long its reference counts are alive, but it should no longer be discoverable.

To fix this, let us call btf_free_id() from the MODULE_STATE_GOING case for the module unload to free the BTF associated IDR entry, and disable its discovery once module unload returns to user space. If a race happens during unload, the outcome is non-deterministic anyway. However, user space should be able to rely on the guarantee that once it has synchronously established a successful module unload, no more stale artifacts associated with this module can be obtained subsequently.

Quoted 5 lines

Note that we must be careful to not invoke btf_free_id() in btf_put() when btf_is_module() is true now. There could be a window where the module unload drops a non-terminal reference, frees the IDR, but the same ID gets reused and the second unconditional btf_free_id() ends up releasing an unrelated entry.

what if we just check module_is_live() state for module BTFs during GET_FD_BY_ID() operation and return -ENOENT? We already have refcount_inc_not_zero(&btf->refcnt), it's just another "it's still there, but not valid to be taken ahold by user space" condition

Quoted 55 lines

We can be sure that by the time we reach final btf_put() for btf_is_module() case, the btf_free_id() is already done, since the module itself holds the BTF reference, and it will call this function for the BTF before dropping its own reference.

[0]: https://lore.kernel.org/bpf/[email protected]

Fixes 36e68442d1af ("bpf: Load and verify kernel module BTFs")
---
 kernel/bpf/btf.c | 15 ++++++++++++++-
 1 file changed, 14 insertions(+), 1 deletion(-)
kernel/bpf/btf.c +14 -2
index 4872d2a6c42d..c44fb9c9386f 100644
@@ -1883,10 +1883,17 @@ void btf_get(struct btf *btf)
        refcount_inc(&btf->refcnt);
 }
 
+bool btf_is_module(const struct btf *btf);
+
 void btf_put(struct btf *btf)
 {
        if (btf && refcount_dec_and_test(&btf->refcnt)) {
-               btf_free_id(btf);
+               /*
+                * IDR for module BTFs is freed when module's BTF reference is
+                * dropped, which _will_ happen before the final put.
+                */
+               if (!btf_is_module(btf))
+                       btf_free_id(btf);
                call_rcu(&btf->rcu, btf_free_rcu);
        }
 }
@@ -8382,6 +8389,12 @@ static int btf_module_notify(struct notifier_block *nb, unsigned long op,
                        if (btf_mod->module != module)
                                continue;
 
+                       /*
+                        * For modules, we do the freeing of BTF IDR as soon as
+                        * module goes away to disable BTF discovery, since the
+                        * btf_try_get_module() on such BTFs will fail.
+                        */
+                       btf_free_id(btf_mod->btf);
                        list_del(&btf_mod->list);
                        if (btf_mod->sysfs_attr)
                                sysfs_remove_bin_file(btf_kobj, btf_mod->sysfs_attr);
--

2.52.0

author maintainer
#3 lore raw
Quoted 32 lines · Andrii Nakryiko
Quoted 23 lines · Kumar Kartikeya Dwivedi

Gregory reported in [0] that the global_map_resize test when run in repeatedly ends up failing during program load. This stems from the fact that BTF reference has not dropped to zero after the previous run's module is unloaded, and the older module's BTF is still discoverable and visible. Later, in libbpf, load_module_btfs() will find the ID for this stale BTF, open its fd, and then it will be used during program load where later steps taking module reference using btf_try_get_module() fail since the underlying module for the BTF is gone.

Logically, once a module is unloaded, it's associated BTF artifacts should become hidden. The BTF object inside the kernel may still remain alive as long its reference counts are alive, but it should no longer be discoverable.

To fix this, let us call btf_free_id() from the MODULE_STATE_GOING case for the module unload to free the BTF associated IDR entry, and disable its discovery once module unload returns to user space. If a race happens during unload, the outcome is non-deterministic anyway. However, user space should be able to rely on the guarantee that once it has synchronously established a successful module unload, no more stale artifacts associated with this module can be obtained subsequently.

Quoted 5 lines

Note that we must be careful to not invoke btf_free_id() in btf_put() when btf_is_module() is true now. There could be a window where the module unload drops a non-terminal reference, frees the IDR, but the same ID gets reused and the second unconditional btf_free_id() ends up releasing an unrelated entry.

Quoted 4 lines

what if we just check module_is_live() state for module BTFs during GET_FD_BY_ID() operation and return -ENOENT? We already have refcount_inc_not_zero(&btf->refcnt), it's just another "it's still there, but not valid to be taken ahold by user space" condition

We will need the module pointer to be able to do module_is_live(), but in this case the module is already gone.
I was achieving the same thing in my first proposal, I didn't share the diff, but TL;DR was setting a bool in btf->module_unloaded and checking for it after refcount_inc_not_zero.
Set btf->module_unloaded from MODULE_STATE_GOING.

I don't have much of a preference for the above or the current diff.

Quoted 56 lines
Quoted 56 lines

We can be sure that by the time we reach final btf_put() for btf_is_module() case, the btf_free_id() is already done, since the module itself holds the BTF reference, and it will call this function for the BTF before dropping its own reference.

[0]: https://lore.kernel.org/bpf/[email protected]

Fixes 36e68442d1af ("bpf: Load and verify kernel module BTFs")
---
 kernel/bpf/btf.c | 15 ++++++++++++++-
 1 file changed, 14 insertions(+), 1 deletion(-)
kernel/bpf/btf.c +14 -2
index 4872d2a6c42d..c44fb9c9386f 100644
@@ -1883,10 +1883,17 @@ void btf_get(struct btf *btf)
        refcount_inc(&btf->refcnt);
 }
 
+bool btf_is_module(const struct btf *btf);
+
 void btf_put(struct btf *btf)
 {
        if (btf && refcount_dec_and_test(&btf->refcnt)) {
-               btf_free_id(btf);
+               /*
+                * IDR for module BTFs is freed when module's BTF reference is
+                * dropped, which _will_ happen before the final put.
+                */
+               if (!btf_is_module(btf))
+                       btf_free_id(btf);
                call_rcu(&btf->rcu, btf_free_rcu);
        }
 }
@@ -8382,6 +8389,12 @@ static int btf_module_notify(struct notifier_block *nb, unsigned long op,
                        if (btf_mod->module != module)
                                continue;
 
+                       /*
+                        * For modules, we do the freeing of BTF IDR as soon as
+                        * module goes away to disable BTF discovery, since the
+                        * btf_try_get_module() on such BTFs will fail.
+                        */
+                       btf_free_id(btf_mod->btf);
                        list_del(&btf_mod->list);
                        if (btf_mod->sysfs_attr)
                                sysfs_remove_bin_file(btf_kobj, btf_mod->sysfs_attr);
--

2.52.0

#4 lore raw
Quoted 39 lines · Kumar Kartikeya Dwivedi
Quoted 32 lines · Andrii Nakryiko
Quoted 23 lines · Kumar Kartikeya Dwivedi

Gregory reported in [0] that the global_map_resize test when run in repeatedly ends up failing during program load. This stems from the fact that BTF reference has not dropped to zero after the previous run's module is unloaded, and the older module's BTF is still discoverable and visible. Later, in libbpf, load_module_btfs() will find the ID for this stale BTF, open its fd, and then it will be used during program load where later steps taking module reference using btf_try_get_module() fail since the underlying module for the BTF is gone.

Logically, once a module is unloaded, it's associated BTF artifacts should become hidden. The BTF object inside the kernel may still remain alive as long its reference counts are alive, but it should no longer be discoverable.

To fix this, let us call btf_free_id() from the MODULE_STATE_GOING case for the module unload to free the BTF associated IDR entry, and disable its discovery once module unload returns to user space. If a race happens during unload, the outcome is non-deterministic anyway. However, user space should be able to rely on the guarantee that once it has synchronously established a successful module unload, no more stale artifacts associated with this module can be obtained subsequently.

Quoted 5 lines

Note that we must be careful to not invoke btf_free_id() in btf_put() when btf_is_module() is true now. There could be a window where the module unload drops a non-terminal reference, frees the IDR, but the same ID gets reused and the second unconditional btf_free_id() ends up releasing an unrelated entry.

Quoted 4 lines

what if we just check module_is_live() state for module BTFs during GET_FD_BY_ID() operation and return -ENOENT? We already have refcount_inc_not_zero(&btf->refcnt), it's just another "it's still there, but not valid to be taken ahold by user space" condition

Quoted 8 lines

We will need the module pointer to be able to do module_is_live(), but in this case the module is already gone.
I was achieving the same thing in my first proposal, I didn't share the diff, but TL;DR was setting a bool in btf->module_unloaded and checking for it after refcount_inc_not_zero.
Set btf->module_unloaded from MODULE_STATE_GOING.

I don't have much of a preference for the above or the current diff.

The part I don't particularly like is that we have two different cases when ID is removed, based on whether BTF is module BTF or not. What if we add btf->id = 0; after idr_remove within spinlock area in btf_free_id() and then call btf_Free_id() unconditionally both on module unload and in normal btf_put path?

Quoted 57 lines
Quoted 57 lines
Quoted 57 lines

We can be sure that by the time we reach final btf_put() for btf_is_module() case, the btf_free_id() is already done, since the module itself holds the BTF reference, and it will call this function for the BTF before dropping its own reference.

[0]: https://lore.kernel.org/bpf/[email protected]

Fixes 36e68442d1af ("bpf: Load and verify kernel module BTFs")
---
 kernel/bpf/btf.c | 15 ++++++++++++++-
 1 file changed, 14 insertions(+), 1 deletion(-)
kernel/bpf/btf.c +14 -2
index 4872d2a6c42d..c44fb9c9386f 100644
@@ -1883,10 +1883,17 @@ void btf_get(struct btf *btf)
        refcount_inc(&btf->refcnt);
 }
 
+bool btf_is_module(const struct btf *btf);
+
 void btf_put(struct btf *btf)
 {
        if (btf && refcount_dec_and_test(&btf->refcnt)) {
-               btf_free_id(btf);
+               /*
+                * IDR for module BTFs is freed when module's BTF reference is
+                * dropped, which _will_ happen before the final put.
+                */
+               if (!btf_is_module(btf))
+                       btf_free_id(btf);
                call_rcu(&btf->rcu, btf_free_rcu);
        }
 }
@@ -8382,6 +8389,12 @@ static int btf_module_notify(struct notifier_block *nb, unsigned long op,
                        if (btf_mod->module != module)
                                continue;
 
+                       /*
+                        * For modules, we do the freeing of BTF IDR as soon as
+                        * module goes away to disable BTF discovery, since the
+                        * btf_try_get_module() on such BTFs will fail.
+                        */
+                       btf_free_id(btf_mod->btf);
                        list_del(&btf_mod->list);
                        if (btf_mod->sysfs_attr)
                                sysfs_remove_bin_file(btf_kobj, btf_mod->sysfs_attr);
--

2.52.0

author maintainer
#5 lore raw
Quoted 49 lines · Andrii Nakryiko
Quoted 46 lines · Kumar Kartikeya Dwivedi
Quoted 32 lines · Andrii Nakryiko
Quoted 23 lines · Kumar Kartikeya Dwivedi

Gregory reported in [0] that the global_map_resize test when run in repeatedly ends up failing during program load. This stems from the fact that BTF reference has not dropped to zero after the previous run's module is unloaded, and the older module's BTF is still discoverable and visible. Later, in libbpf, load_module_btfs() will find the ID for this stale BTF, open its fd, and then it will be used during program load where later steps taking module reference using btf_try_get_module() fail since the underlying module for the BTF is gone.

Logically, once a module is unloaded, it's associated BTF artifacts should become hidden. The BTF object inside the kernel may still remain alive as long its reference counts are alive, but it should no longer be discoverable.

To fix this, let us call btf_free_id() from the MODULE_STATE_GOING case for the module unload to free the BTF associated IDR entry, and disable its discovery once module unload returns to user space. If a race happens during unload, the outcome is non-deterministic anyway. However, user space should be able to rely on the guarantee that once it has synchronously established a successful module unload, no more stale artifacts associated with this module can be obtained subsequently.

Quoted 5 lines

Note that we must be careful to not invoke btf_free_id() in btf_put() when btf_is_module() is true now. There could be a window where the module unload drops a non-terminal reference, frees the IDR, but the same ID gets reused and the second unconditional btf_free_id() ends up releasing an unrelated entry.

Quoted 4 lines

what if we just check module_is_live() state for module BTFs during GET_FD_BY_ID() operation and return -ENOENT? We already have refcount_inc_not_zero(&btf->refcnt), it's just another "it's still there, but not valid to be taken ahold by user space" condition

We will need the module pointer to be able to do module_is_live(), but in this case the module is already gone.
I was achieving the same thing in my first proposal, I didn't share the diff, but TL;DR was setting a bool in btf->module_unloaded and checking for it after refcount_inc_not_zero.
Set btf->module_unloaded from MODULE_STATE_GOING.

Quoted 8 lines
Quoted 1 line

I don't have much of a preference for the above or the current diff.

The part I don't particularly like is that we have two different cases when ID is removed, based on whether BTF is module BTF or not. What if we add btf->id = 0; after idr_remove within spinlock area in btf_free_id() and then call btf_Free_id() unconditionally both on module unload and in normal btf_put path?

Yeah, that should work too. id = 0 should make the function idempotent since 0 is not a valid IDR. Let me give it a spin and resend.

Quoted 1 line
Quoted 1 line

[...]