Skip to contentSkip to conversation

Netfilter/IPVS fixes for net

PATCH Applied Pablo Neira Ayuso 2 participants 9 messages Nov 2, 2022 – Nov 3, 2022

Subsystems (4)
lore
Share & tools
lore

Pablo Neira Ayuso

author maintainer
#0 lore raw

cover Netfilter/IPVS fixes for net

Hi,

The following patchset contains Netfilter/IPVS fixes for net:

1) netlink socket notifier might win race to release objects that are already pending to be released via commit release path, reported by syzbot.

2) No need to postpone flow rule release to commit release path, this triggered the syzbot report, complementary fix to previous patch.

3) Use explicit signed chars in IPVS to unbreak arm, from Jason A. Donenfeld.

4) Missing check for proc entry creation failure in IPVS, from Zhengchao Shao.

5) Incorrect error path handling when BPF NAT fails to register, from Chen Zhongjin.

6) Prevent huge memory allocation in ipset hash types, from Jozsef Kadlecsik.

Except the incorrect BPF NAT error path which is broken in 6.1-rc, anything else has been broken for several releases.

Please, pull these changes from:

git://git.kernel.org/pub/scm/linux/kernel/git/netfilter/nf.git

Thanks.

----------------------------------------------------------------

The following changes since commit 363a5328f4b0517e59572118ccfb7c626d81dca9:

net: tun: fix bugs for oversize packet when napi frags enabled (2022-10-31 20:04:55 -0700)

are available in the Git repository at:

git://git.kernel.org/pub/scm/linux/kernel/git/netfilter/nf.git HEAD

for you to fetch changes up to 510841da1fcc16f702440ab58ef0b4d82a9056b7:

netfilter: ipset: enforce documented limit to prevent allocating huge memory (2022-11-02 19:22:23 +0100)

----------------------------------------------------------------
Chen Zhongjin (1):
netfilter: nf_nat: Fix possible memory leak in nf_nat_init()

Jason A. Donenfeld (1):
ipvs: use explicitly signed chars

Jozsef Kadlecsik (1):
netfilter: ipset: enforce documented limit to prevent allocating huge memory

Pablo Neira Ayuso (2):
netfilter: nf_tables: netlink notifier might race to release objects
netfilter: nf_tables: release flow rule object from commit path

Zhengchao Shao (2):
ipvs: fix WARNING in __ip_vs_cleanup_batch()
ipvs: fix WARNING in ip_vs_app_net_cleanup()

 net/netfilter/ipset/ip_set_hash_gen.h | 30 ++++++------------------------
 net/netfilter/ipvs/ip_vs_app.c        | 10 ++++++++--
 net/netfilter/ipvs/ip_vs_conn.c       | 30 +++++++++++++++++++++++-------
 net/netfilter/nf_nat_core.c           | 11 ++++++++++-
 net/netfilter/nf_tables_api.c         |  8 +++++---
 5 files changed, 52 insertions(+), 37 deletions(-)
author maintainer
#1 lore raw

PATCH 1/7 netfilter: nf_tables: netlink notifier might race to release objects

commit release path is invoked via call_rcu and it runs lockless to release the objects after rcu grace period. The netlink notifier handler might win race to remove objects that the transaction context is still referencing from the commit release path.

Call rcu_barrier() to ensure pending rcu callbacks run to completion if the list of transactions to be destroyed is not empty.

Fixes 6001a930ce03 ("netfilter: nftables: introduce table ownership")
Reported-by [email protected]
---
 net/netfilter/nf_tables_api.c | 2 ++
 1 file changed, 2 insertions(+)
net/netfilter/nf_tables_api.c +2 -0
index 58d9cbc9ccdc..2197118aa7b0 100644
@@ -10030,6 +10030,8 @@ static int nft_rcv_nl_event(struct notifier_block *this, unsigned long event,
 	nft_net = nft_pernet(net);
 	deleted = 0;
 	mutex_lock(&nft_net->commit_mutex);
+	if (!list_empty(&nf_tables_destroy_list))
+		rcu_barrier();
 again:
 	list_for_each_entry(table, &nft_net->tables, list) {
 		if (nft_table_has_owner(table) &&
Signature · 1 line
2.30.2
#2 lore raw

Re: [PATCH net 1/7] netfilter: nf_tables: netlink notifier might race to release objects

This series was applied to netdev/net.git (master) by Pablo Neira Ayuso <[email protected]>:

Hello:

This series was applied to netdev/net.git (master) by Pablo Neira Ayuso <[email protected]>:

Quoted 9 lines · On Wed, 2 Nov 2022 19:46:53 +0100 you wrote:

commit release path is invoked via call_rcu and it runs lockless to release the objects after rcu grace period. The netlink notifier handler might win race to remove objects that the transaction context is still referencing from the commit release path.

Call rcu_barrier() to ensure pending rcu callbacks run to completion if the list of transactions to be destroyed is not empty.

[...]

Here is the summary with links:
- [net,1/7] netfilter: nf_tables: netlink notifier might race to release objects

    https://git.kernel.org/netdev/net/c/d4bc8271db21
  - [net,2/7] netfilter: nf_tables: release flow rule object from commit path
    https://git.kernel.org/netdev/net/c/26b5934ff419
  - [net,3/7] ipvs: use explicitly signed chars
    https://git.kernel.org/netdev/net/c/5c26159c97b3
  - [net,4/7] ipvs: fix WARNING in __ip_vs_cleanup_batch()
    https://git.kernel.org/netdev/net/c/3d00c6a0da8d
  - [net,5/7] ipvs: fix WARNING in ip_vs_app_net_cleanup()
    https://git.kernel.org/netdev/net/c/5663ed63adb9
  - [net,6/7] netfilter: nf_nat: Fix possible memory leak in nf_nat_init()
    https://git.kernel.org/netdev/net/c/cbc1dd5b659f
  - [net,7/7] netfilter: ipset: enforce documented limit to prevent allocating huge memory
    https://git.kernel.org/netdev/net/c/510841da1fcc

You are awesome, thank you!

Signature · 2 lines
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/patchwork/pwbot.html
author maintainer
#3 lore raw

PATCH 2/7 netfilter: nf_tables: release flow rule object from commit path

In reply to Pablo Neira Ayuso

No need to postpone this to the commit release path, since no packets are walking over this object, this is accessed from control plane only. This helped uncovered UAF triggered by races with the netlink notifier.

Fixes 9dd732e0bdf5 ("netfilter: nf_tables: memleak flow rule from commit path")
Reported-by [email protected]
---
 net/netfilter/nf_tables_api.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
net/netfilter/nf_tables_api.c +3 -3
index 2197118aa7b0..76bd4d03dbda 100644
@@ -8465,9 +8465,6 @@ static void nft_commit_release(struct nft_trans *trans)
 		nf_tables_chain_destroy(&trans->ctx);
 		break;
 	case NFT_MSG_DELRULE:
-		if (trans->ctx.chain->flags & NFT_CHAIN_HW_OFFLOAD)
-			nft_flow_rule_destroy(nft_trans_flow_rule(trans));
-
 		nf_tables_rule_destroy(&trans->ctx, nft_trans_rule(trans));
 		break;
 	case NFT_MSG_DELSET:
@@ -8973,6 +8970,9 @@ static int nf_tables_commit(struct net *net, struct sk_buff *skb)
 			nft_rule_expr_deactivate(&trans->ctx,
 						 nft_trans_rule(trans),
 						 NFT_TRANS_COMMIT);
+
+			if (trans->ctx.chain->flags & NFT_CHAIN_HW_OFFLOAD)
+				nft_flow_rule_destroy(nft_trans_flow_rule(trans));
 			break;
 		case NFT_MSG_NEWSET:
 			nft_clear(net, nft_trans_set(trans));
Signature · 1 line
2.30.2
author maintainer
#4 lore raw

PATCH 3/7 ipvs: use explicitly signed chars

In reply to Pablo Neira Ayuso

From: "Jason A. Donenfeld" <[email protected]>

The char type with no explicit sign is sometimes signed and sometimes unsigned. This code will break on platforms such as arm, where char is unsigned. So mark it here as explicitly signed, so that the todrop_counter decrement and subsequent comparison is correct.

Fixes 1da177e4c3f4 ("Linux-2.6.12-rc2")
---
 net/netfilter/ipvs/ip_vs_conn.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
net/netfilter/ipvs/ip_vs_conn.c +2 -2
index 8c04bb57dd6f..7c4866c04343 100644
@@ -1265,8 +1265,8 @@ static inline int todrop_entry(struct ip_vs_conn *cp)
 	 * The drop rate array needs tuning for real environments.
 	 * Called from timer bh only => no locking
 	 */
-	static const char todrop_rate[9] = {0, 1, 2, 3, 4, 5, 6, 7, 8};
-	static char todrop_counter[9] = {0};
+	static const signed char todrop_rate[9] = {0, 1, 2, 3, 4, 5, 6, 7, 8};
+	static signed char todrop_counter[9] = {0};
 	int i;
 
 	/* if the conn entry hasn't lasted for 60 seconds, don't drop it.
Signature · 1 line
2.30.2
author maintainer
#5 lore raw

PATCH 4/7 ipvs: fix WARNING in __ip_vs_cleanup_batch()

In reply to Pablo Neira Ayuso

From: Zhengchao Shao <[email protected]>

During the initialization of ip_vs_conn_net_init(), if file ip_vs_conn or ip_vs_conn_sync fails to be created, the initialization is successful by default. Therefore, the ip_vs_conn or ip_vs_conn_sync file doesn't be found during the remove.

The following is the stack information:
name 'ip_vs_conn_sync'

WARNING: CPU: 3 PID: 9 at fs/proc/generic.c:712
remove_proc_entry+0x389/0x460
Modules linked in:
Workqueue: netns cleanup_net
RIP: 0010:remove_proc_entry+0x389/0x460
Call Trace:
<TASK>
__ip_vs_cleanup_batch+0x7d/0x120
ops_exit_list+0x125/0x170
cleanup_net+0x4ea/0xb00
process_one_work+0x9bf/0x1710
worker_thread+0x665/0x1080
kthread+0x2e4/0x3a0
ret_from_fork+0x1f/0x30
</TASK>
Fixes 61b1ab4583e2 ("IPVS: netns, add basic init per netns.")
---
 net/netfilter/ipvs/ip_vs_conn.c | 26 +++++++++++++++++++++-----
 1 file changed, 21 insertions(+), 5 deletions(-)
net/netfilter/ipvs/ip_vs_conn.c +21 -5
index 7c4866c04343..13534e02346c 100644
@@ -1447,20 +1447,36 @@ int __net_init ip_vs_conn_net_init(struct netns_ipvs *ipvs)
 {
 	atomic_set(&ipvs->conn_count, 0);
 
-	proc_create_net("ip_vs_conn", 0, ipvs->net->proc_net,
-			&ip_vs_conn_seq_ops, sizeof(struct ip_vs_iter_state));
-	proc_create_net("ip_vs_conn_sync", 0, ipvs->net->proc_net,
-			&ip_vs_conn_sync_seq_ops,
-			sizeof(struct ip_vs_iter_state));
+#ifdef CONFIG_PROC_FS
+	if (!proc_create_net("ip_vs_conn", 0, ipvs->net->proc_net,
+			     &ip_vs_conn_seq_ops,
+			     sizeof(struct ip_vs_iter_state)))
+		goto err_conn;
+
+	if (!proc_create_net("ip_vs_conn_sync", 0, ipvs->net->proc_net,
+			     &ip_vs_conn_sync_seq_ops,
+			     sizeof(struct ip_vs_iter_state)))
+		goto err_conn_sync;
+#endif
+
 	return 0;
+
+#ifdef CONFIG_PROC_FS
+err_conn_sync:
+	remove_proc_entry("ip_vs_conn", ipvs->net->proc_net);
+err_conn:
+	return -ENOMEM;
+#endif
 }
 
 void __net_exit ip_vs_conn_net_cleanup(struct netns_ipvs *ipvs)
 {
 	/* flush all the connection entries first */
 	ip_vs_conn_flush(ipvs);
+#ifdef CONFIG_PROC_FS
 	remove_proc_entry("ip_vs_conn", ipvs->net->proc_net);
 	remove_proc_entry("ip_vs_conn_sync", ipvs->net->proc_net);
+#endif
 }
 
 int __init ip_vs_conn_init(void)
Signature · 1 line
2.30.2
author maintainer
#6 lore raw

PATCH 5/7 ipvs: fix WARNING in ip_vs_app_net_cleanup()

In reply to Pablo Neira Ayuso

From: Zhengchao Shao <[email protected]>

During the initialization of ip_vs_app_net_init(), if file ip_vs_app fails to be created, the initialization is successful by default. Therefore, the ip_vs_app file doesn't be found during the remove in ip_vs_app_net_cleanup(). It will cause WRNING.

The following is the stack information:
name 'ip_vs_app'

WARNING: CPU: 1 PID: 9 at fs/proc/generic.c:712 remove_proc_entry+0x389/0x460
Modules linked in:
Workqueue: netns cleanup_net
RIP: 0010:remove_proc_entry+0x389/0x460
Call Trace:
<TASK>
ops_exit_list+0x125/0x170
cleanup_net+0x4ea/0xb00
process_one_work+0x9bf/0x1710
worker_thread+0x665/0x1080
kthread+0x2e4/0x3a0
ret_from_fork+0x1f/0x30
</TASK>
Fixes 457c4cbc5a3d ("[NET]: Make /proc/net per network namespace")
---
 net/netfilter/ipvs/ip_vs_app.c | 10 ++++++++--
 1 file changed, 8 insertions(+), 2 deletions(-)
net/netfilter/ipvs/ip_vs_app.c +8 -2
index f9b16f2b2219..fdacbc3c15be 100644
@@ -599,13 +599,19 @@ static const struct seq_operations ip_vs_app_seq_ops = {
 int __net_init ip_vs_app_net_init(struct netns_ipvs *ipvs)
 {
 	INIT_LIST_HEAD(&ipvs->app_list);
-	proc_create_net("ip_vs_app", 0, ipvs->net->proc_net, &ip_vs_app_seq_ops,
-			sizeof(struct seq_net_private));
+#ifdef CONFIG_PROC_FS
+	if (!proc_create_net("ip_vs_app", 0, ipvs->net->proc_net,
+			     &ip_vs_app_seq_ops,
+			     sizeof(struct seq_net_private)))
+		return -ENOMEM;
+#endif
 	return 0;
 }
 
 void __net_exit ip_vs_app_net_cleanup(struct netns_ipvs *ipvs)
 {
 	unregister_ip_vs_app(ipvs, NULL /* all */);
+#ifdef CONFIG_PROC_FS
 	remove_proc_entry("ip_vs_app", ipvs->net->proc_net);
+#endif
 }
Signature · 1 line
2.30.2
author maintainer
#7 lore raw

PATCH 6/7 netfilter: nf_nat: Fix possible memory leak in nf_nat_init()

In reply to Pablo Neira Ayuso

From: Chen Zhongjin <[email protected]>

In nf_nat_init(), register_nf_nat_bpf() can fail and return directly without any error handling.
Then nf_nat_bysource will leak and registering of &nat_net_ops, &follow_master_nat and nf_nat_hook won't be reverted.

This leaves wild ops in linkedlists and when another module tries to call register_pernet_operations() or nf_ct_helper_expectfn_register() it triggers page fault:

 BUG: unable to handle page fault for address: fffffbfff81b964c
 RIP: 0010:register_pernet_operations+0x1b9/0x5f0
 Call Trace:
 <TASK>
  register_pernet_subsys+0x29/0x40
  ebtables_init+0x58/0x1000 [ebtables]
  ...
Fixes 820dc0523e05 ("net: netfilter: move bpf_ct_set_nat_info kfunc in nf_nat_bpf.c")
---
 net/netfilter/nf_nat_core.c | 11 ++++++++++-
 1 file changed, 10 insertions(+), 1 deletion(-)
net/netfilter/nf_nat_core.c +10 -1
index 18319a6e6806..e29e4ccb5c5a 100644
@@ -1152,7 +1152,16 @@ static int __init nf_nat_init(void)
 	WARN_ON(nf_nat_hook != NULL);
 	RCU_INIT_POINTER(nf_nat_hook, &nat_hook);
 
-	return register_nf_nat_bpf();
+	ret = register_nf_nat_bpf();
+	if (ret < 0) {
+		RCU_INIT_POINTER(nf_nat_hook, NULL);
+		nf_ct_helper_expectfn_unregister(&follow_master_nat);
+		synchronize_net();
+		unregister_pernet_subsys(&nat_net_ops);
+		kvfree(nf_nat_bysource);
+	}
+
+	return ret;
 }
 
 static void __exit nf_nat_cleanup(void)
Signature · 1 line
2.30.2
author maintainer
#8 lore raw

PATCH 7/7 netfilter: ipset: enforce documented limit to prevent allocating huge memory

In reply to Pablo Neira Ayuso

From: Jozsef Kadlecsik <[email protected]>

Daniel Xu reported that the hash:net,iface type of the ipset subsystem does not limit adding the same network with different interfaces to a set, which can lead to huge memory usage or allocation failure.

The quick reproducer is

$ ipset create ACL.IN.ALL_PERMIT hash:net,iface hashsize 1048576 timeout 0
$ for i in $(seq 0 100); do /sbin/ipset add ACL.IN.ALL_PERMIT 0.0.0.0/0,kaf_$i timeout 0 -exist; done

The backtrace when vmalloc fails:

        [Tue Oct 25 00:13:08 2022] ipset: vmalloc error: size 1073741848, exceeds total pages
        <...>
        [Tue Oct 25 00:13:08 2022] Call Trace:
        [Tue Oct 25 00:13:08 2022]  <TASK>
        [Tue Oct 25 00:13:08 2022]  dump_stack_lvl+0x48/0x60
        [Tue Oct 25 00:13:08 2022]  warn_alloc+0x155/0x180
        [Tue Oct 25 00:13:08 2022]  __vmalloc_node_range+0x72a/0x760
        [Tue Oct 25 00:13:08 2022]  ? hash_netiface4_add+0x7c0/0xb20
        [Tue Oct 25 00:13:08 2022]  ? __kmalloc_large_node+0x4a/0x90
        [Tue Oct 25 00:13:08 2022]  kvmalloc_node+0xa6/0xd0
        [Tue Oct 25 00:13:08 2022]  ? hash_netiface4_resize+0x99/0x710
        <...>

The fix is to enforce the limit documented in the ipset(8) manpage:

Quoted 3 lines

The internal restriction of the hash:net,iface set type is that the same network prefix cannot be stored with more than 64 different interfaces in a single set.

Fixes ccf0a4b7fc68 ("netfilter: ipset: Add bucketsize parameter to all hash types")
---
 net/netfilter/ipset/ip_set_hash_gen.h | 30 ++++++---------------------
 1 file changed, 6 insertions(+), 24 deletions(-)
net/netfilter/ipset/ip_set_hash_gen.h +6 -24
index 6e391308431d..3adc291d9ce1 100644
@@ -42,31 +42,8 @@
 #define AHASH_MAX_SIZE			(6 * AHASH_INIT_SIZE)
 /* Max muber of elements in the array block when tuned */
 #define AHASH_MAX_TUNED			64
-
 #define AHASH_MAX(h)			((h)->bucketsize)
 
-/* Max number of elements can be tuned */
-#ifdef IP_SET_HASH_WITH_MULTI
-static u8
-tune_bucketsize(u8 curr, u32 multi)
-{
-	u32 n;
-
-	if (multi < curr)
-		return curr;
-
-	n = curr + AHASH_INIT_SIZE;
-	/* Currently, at listing one hash bucket must fit into a message.
-	 * Therefore we have a hard limit here.
-	 */
-	return n > curr && n <= AHASH_MAX_TUNED ? n : curr;
-}
-#define TUNE_BUCKETSIZE(h, multi)	\
-	((h)->bucketsize = tune_bucketsize((h)->bucketsize, multi))
-#else
-#define TUNE_BUCKETSIZE(h, multi)
-#endif
-
 /* A hash bucket */
 struct hbucket {
 	struct rcu_head rcu;	/* for call_rcu */
@@ -936,7 +913,12 @@ mtype_add(struct ip_set *set, void *value, const struct ip_set_ext *ext,
 		goto set_full;
 	/* Create a new slot */
 	if (n->pos >= n->size) {
-		TUNE_BUCKETSIZE(h, multi);
+#ifdef IP_SET_HASH_WITH_MULTI
+		if (h->bucketsize >= AHASH_MAX_TUNED)
+			goto set_full;
+		else if (h->bucketsize < multi)
+			h->bucketsize += AHASH_INIT_SIZE;
+#endif
 		if (n->size >= AHASH_MAX(h)) {
 			/* Trigger rehashing */
 			mtype_data_next(&h->next, d);
Signature · 1 line
2.30.2