Skip to contentSkip to conversation

kernel/module.c: don't allow modprobe to hang forever on a module load

PATCH In review Konstantin Khorenko 3 participants 4 messages Nov 13, 2019 – Nov 15, 2019

lore
Share & tools
lore
#0 lore raw

cover kernel/module.c: don't allow modprobe to hang forever on a module load

After commit 6e6de3dee51a ("kernel/module.c: Only return -EEXIST for modules that have finished loading")
the simple test leads to hanged modprobe process (INTERRUPTIBLE):

1. Make sure nft_ct module is not used by your firewall rules.
2. Run 3 copies of

   # export i=0; while true; if [[ $(($i % 100)) -eq 0 ]] ; then \
     echo "i=$i"; fi; do modprobe nft_ct; i=$(($i + 1)); done

3. Run 2 copies of

   # while true; do rmmod nft_ct; done

Hanged "modprobe" process will appear in ~ 10 seconds.

   # cat /proc/30184/stack
   [<0>] load_module+0x53f/0x2060
   [<0>] __do_sys_finit_module+0xd2/0x100
   [<0>] do_syscall_64+0x5b/0x1c0
   [<0>] entry_SYSCALL_64_after_hwframe+0x44/0xa9

Konstantin Khorenko (1):
kernel/module.c: wakeup processes in module_wq on module unload

 kernel/module.c | 2 ++
 1 file changed, 2 insertions(+)
Signature · 1 line
2.15.1
#1 lore raw

PATCH 1/1 kernel/module.c: wakeup processes in module_wq on module unload

Fix the race between load and unload a kernel module.

sys_delete_module()
try_stop_module()
mod->state = _GOING
add_unformed_module()
old = find_module_all() (old->state == _GOING =>
wait_event_interruptible())

During pre-condition finished_loading() rets 0 schedule() (never gets waken up later)
free_module()
mod->state = _UNFORMED
list_del_rcu(&mod->list)
(dels mod from "modules" list)

return

The race above leads to modprobe hanging forever on loading a module.

Error paths on loading module call wake_up_all(&module_wq) after freeing module, so let's do the same on straight module unload.

Fixes 6e6de3dee51a ("kernel/module.c: Only return -EEXIST

for modules that have finished loading")

---
 kernel/module.c | 2 ++
 1 file changed, 2 insertions(+)
kernel/module.c +2 -0
index ff2d7359a418..cb09a5f37a5f 100644
@@ -1033,6 +1033,8 @@ SYSCALL_DEFINE2(delete_module, const char __user *, name_user,
 	strlcpy(last_unloaded_module, mod->name, sizeof(last_unloaded_module));
 
 	free_module(mod);
+	/* someone could wait for the module in add_unformed_module() */
+	wake_up_all(&module_wq);
 	return 0;
 out:
 	mutex_unlock(&module_mutex);
Signature · 1 line
2.15.1
Reviewed-by on v1 1/1
#2 lore raw

Re: [PATCH 1/1] kernel/module.c: wakeup processes in module_wq on module unload

Quoted 40 lines · Konstantin Khorenko

Fix the race between load and unload a kernel module.

sys_delete_module()
try_stop_module()
mod->state = _GOING
add_unformed_module()
old = find_module_all() (old->state == _GOING =>
wait_event_interruptible())

During pre-condition finished_loading() rets 0 schedule() (never gets waken up later)
free_module()
mod->state = _UNFORMED
list_del_rcu(&mod->list)
(dels mod from "modules" list)

return

The race above leads to modprobe hanging forever on loading a module.

Error paths on loading module call wake_up_all(&module_wq) after freeing module, so let's do the same on straight module unload.

Fixes 6e6de3dee51a ("kernel/module.c: Only return -EEXIST

for modules that have finished loading")

---
 kernel/module.c | 2 ++
 1 file changed, 2 insertions(+)
kernel/module.c +0 -0
index ff2d7359a418..cb09a5f37a5f 100644

kernel/module.c SYSCALL_DEFINE2(delete_module, const char __user *, name_user, · line 1037

Quoted 5 lines
kernel/module.c +2 -0
@@ -1033,6 +1033,8 @@ SYSCALL_DEFINE2(delete_module, const char __user *, name_user,
 	strlcpy(last_unloaded_module, mod->name, sizeof(last_unloaded_module));
 
 	free_module(mod);
+	/* someone could wait for the module in add_unformed_module() */
+	wake_up_all(&module_wq);

free_module() acts on that *this* module so it makes sense to include the wake_up_all(&module_wq) call in delete_module. IIRC I did try a module load & unload stress test but it looks like my testing wasn't hard enough :(

In any case,

P.

Quoted 4 lines
kernel/module.c +0 -0
 	return 0;
 out:
 	mutex_unlock(&module_mutex);
#3 lore raw

Re: [PATCH 1/1] kernel/module.c: wakeup processes in module_wq on module unload

In reply to Konstantin Khorenko

+++ Konstantin Khorenko [13/11/19 12:29 +0300]:

Quoted 24 lines

Fix the race between load and unload a kernel module.

sys_delete_module()
try_stop_module()
mod->state = _GOING
add_unformed_module()
old = find_module_all() (old->state == _GOING =>
wait_event_interruptible())

During pre-condition finished_loading() rets 0 schedule() (never gets waken up later)
free_module()
mod->state = _UNFORMED
list_del_rcu(&mod->list)
(dels mod from "modules" list)

return

The race above leads to modprobe hanging forever on loading a module.

Quoted 7 lines

Error paths on loading module call wake_up_all(&module_wq) after freeing module, so let's do the same on straight module unload.

Fixes 6e6de3dee51a ("kernel/module.c: Only return -EEXIST

for modules that have finished loading")

Thanks Konstantin for catching this. I've applied this to modules-next.

Jessica