Skip to contentSkip to conversation

netfilter: nfnetlink: avoid deadlock due to synchronous request_module

PATCH Applied Florian Westphal 2 participants 2 messages Jul 2, 2019 – Jul 3, 2019

Subsystems (2)
lore
Share & tools
lore

Florian Westphal

author maintainer
#0 lore raw

PATCH 1/1 netfilter: nfnetlink: avoid deadlock due to synchronous request_module

Thomas and Juliana report a deadlock when running:

(rmmod nf_conntrack_netlink/xfrm_user)

conntrack -e NEW -E &
modprobe -v xfrm_user

They provided following analysis:

conntrack -e NEW -E

    netlink_bind()
        netlink_lock_table() -> increases "nl_table_users"
            nfnetlink_bind()
            # does not unlock the table as it's locked by netlink_bind()
                __request_module()
                    call_usermodehelper_exec()

This triggers "modprobe nf_conntrack_netlink" from kernel, netlink_bind() won't return until modprobe process is done.

"modprobe xfrm_user":

    xfrm_user_init()
        register_pernet_subsys()
            -> grab pernet_ops_rwsem
                ..
                netlink_table_grab()
                    calls schedule() as "nl_table_users" is non-zero

so modprobe is blocked because netlink_bind() increased nl_table_users while also holding pernet_ops_rwsem.

"modprobe nf_conntrack_netlink" runs and inits nf_conntrack_netlink:

    ctnetlink_init()
        register_pernet_subsys()
            -> blocks on "pernet_ops_rwsem" thanks to xfrm_user module

both modprobe processes wait on one another -- neither can make progress.

Switch netlink_bind() to "nowait" modprobe -- this releases the netlink table lock, which then allows both modprobe instances to complete.

---
 net/netfilter/nfnetlink.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
net/netfilter/nfnetlink.c +1 -1
index 92077d459109..4abbb452cf6c 100644
@@ -578,7 +578,7 @@ static int nfnetlink_bind(struct net *net, int group)
 	ss = nfnetlink_get_subsys(type << 8);
 	rcu_read_unlock();
 	if (!ss)
-		request_module("nfnetlink-subsys-%d", type);
+		request_module_nowait("nfnetlink-subsys-%d", type);
 	return 0;
 }
 #endif
Signature · 1 line
2.21.0
maintainer Applied
#1 lore raw
Quoted 36 lines · Florian Westphal

Thomas and Juliana report a deadlock when running:

(rmmod nf_conntrack_netlink/xfrm_user)

conntrack -e NEW -E &
modprobe -v xfrm_user

They provided following analysis:

conntrack -e NEW -E

    netlink_bind()
        netlink_lock_table() -> increases "nl_table_users"
            nfnetlink_bind()
            # does not unlock the table as it's locked by netlink_bind()
                __request_module()
                    call_usermodehelper_exec()

This triggers "modprobe nf_conntrack_netlink" from kernel, netlink_bind() won't return until modprobe process is done.

"modprobe xfrm_user":

    xfrm_user_init()
        register_pernet_subsys()
            -> grab pernet_ops_rwsem
                ..
                netlink_table_grab()
                    calls schedule() as "nl_table_users" is non-zero

so modprobe is blocked because netlink_bind() increased nl_table_users while also holding pernet_ops_rwsem.

"modprobe nf_conntrack_netlink" runs and inits nf_conntrack_netlink:

    ctnetlink_init()
        register_pernet_subsys()
            -> blocks on "pernet_ops_rwsem" thanks to xfrm_user module
Quoted 5 lines

both modprobe processes wait on one another -- neither can make progress.

Switch netlink_bind() to "nowait" modprobe -- this releases the netlink table lock, which then allows both modprobe instances to complete.

Applied, thanks.