netfilter: fix ordering of jumpstack allocation and table update
Subsystems (2)
PATCH 1/1 netfilter: fix ordering of jumpstack allocation and table update
During kernel stability testing on an SMP ARMv7 system, Yalin Wang reported the following panic from the netfilter code:
1fe0: 0000001c 5e2d3b10 4007e779 4009e110 60000010 00000032 ff565656 ff545454 [<c06c48dc>] (ipt_do_table+0x448/0x584) from [<c0655ef0>] (nf_iterate+0x48/0x7c) [<c0655ef0>] (nf_iterate+0x48/0x7c) from [<c0655f7c>] (nf_hook_slow+0x58/0x104) [<c0655f7c>] (nf_hook_slow+0x58/0x104) from [<c0683bbc>] (ip_local_deliver+0x88/0xa8) [<c0683bbc>] (ip_local_deliver+0x88/0xa8) from [<c0683718>] (ip_rcv_finish+0x418/0x43c) [<c0683718>] (ip_rcv_finish+0x418/0x43c) from [<c062b1c4>] (__netif_receive_skb+0x4cc/0x598) [<c062b1c4>] (__netif_receive_skb+0x4cc/0x598) from [<c062b314>] (process_backlog+0x84/0x158) [<c062b314>] (process_backlog+0x84/0x158) from [<c062de84>] (net_rx_action+0x70/0x1dc) [<c062de84>] (net_rx_action+0x70/0x1dc) from [<c0088230>] (__do_softirq+0x11c/0x27c) [<c0088230>] (__do_softirq+0x11c/0x27c) from [<c008857c>] (do_softirq+0x44/0x50) [<c008857c>] (do_softirq+0x44/0x50) from [<c0088614>] (local_bh_enable_ip+0x8c/0xd0) [<c0088614>] (local_bh_enable_ip+0x8c/0xd0) from [<c06b0330>] (inet_stream_connect+0x164/0x298) [<c06b0330>] (inet_stream_connect+0x164/0x298) from [<c061d68c>] (sys_connect+0x88/0xc8) [<c061d68c>] (sys_connect+0x88/0xc8) from [<c000e340>] (ret_fast_syscall+0x0/0x30)
Code: 2a000021 e59d2028 e59de01c e59f011c (e7824103) ---[ end trace da227214a82491bd ]--- Kernel panic - not syncing: Fatal exception in interrupt
This comes about because CPU1 is executing xt_replace_table in response to a setsockopt syscall, resulting in:
ret = xt_jumpstack_alloc(newinfo); --> newinfo->jumpstack = kzalloc(size, GFP_KERNEL); [...] table->private = newinfo; newinfo->initial_entries = private->initial_entries;
Meanwhile, CPU0 is handling the network receive path and ends up in ipt_do_table, resulting in:
private = table->private; [...] jumpstack = (struct ipt_entry **)private->jumpstack[cpu];
On weakly ordered memory architectures, the writes to table->private and newinfo->jumpstack from CPU1 can be observed out of order by CPU0. Furthermore, on architectures which don't respect ordering of address dependencies (i.e. Alpha), the reads from CPU0 can also be re-ordered.
This patch adds an smp_wmb() before the assignment to table->private (which is essentially publishing newinfo) to ensure that all writes to newinfo will be observed before plugging it into the table structure. A dependent-read barrier is also added on the consumer side, to ensure the same ordering requirement are also respected there.
--- net/ipv4/netfilter/ip_tables.c | 5 +++++ net/netfilter/x_tables.c | 7 ++++++- 2 files changed, 11 insertions(+), 1 deletion(-)
net/ipv4/netfilter/ip_tables.c +5 -0
@@ -327,6 +327,11 @@ ipt_do_table(struct sk_buff *skb, addend =xt_write_recseq_begin (); private = table->private; cpu =smp_processor_id (); +/* +* Ensure we load private members after we've fetched the base +* pointer. +*/ +smp_read_barrier_depends (); table_base = private->entries[cpu]; jumpstack = (struct ipt_entry **)private->jumpstack[cpu]; stackptr =per_cpu_ptr (private->stackptr, cpu);
net/netfilter/x_tables.c +6 -1
@@ -845,8 +845,13 @@ xt_replace_table(struct xt_table *table,return NULL ; } - table->private = newinfo; newinfo->initial_entries = private->initial_entries; +/* +* Ensure contents of newinfo are visible before assigning to +* private. +*/ +smp_wmb (); + table->private = newinfo;/* * Even though table entries have now been swapped, other CPU's
Signature · 1 line
1.8.2.2
Quoted 15 lines · Will Deacon
During kernel stability testing on an SMP ARMv7 system, Yalin Wang reported the following panic from the netfilter code:
On weakly ordered memory architectures, the writes to
table->privateandnewinfo->jumpstackfrom CPU1 can be observed out of order by CPU0. Furthermore, on architectures which don't respect ordering of address dependencies (i.e. Alpha), the reads from CPU0 can also be re-ordered.This patch adds an
smp_wmb()before the assignment totable->private(which is essentially publishing newinfo) to ensure that all writes to newinfo will be observed before plugging it into the table structure. A dependent-read barrier is also added on the consumer side, to ensure the same ordering requirement are also respected there.
Quoted 6 lines
Cc Pablo Neira Ayuso <[email protected]>Cc Paul E. McKenney <[email protected]>Reported-by Wang, Yalin <[email protected]>Tested-by Wang, Yalin <[email protected]>Signed-off-by Will Deacon <[email protected]>---
Nice catch !
Pablo Neira Ayuso
maintainerQuoted 46 lines · Will Deacon
During kernel stability testing on an SMP ARMv7 system, Yalin Wang reported the following panic from the netfilter code:
1fe0:
0000001c5e2d3b104007e7794009e11060000010 00000032ff565656ff545454[<c06c48dc>] (ipt_do_table+0x448/0x584) from [<c0655ef0>] (nf_iterate+0x48/0x7c) [<c0655ef0>] (nf_iterate+0x48/0x7c) from [<c0655f7c>] (nf_hook_slow+0x58/0x104) [<c0655f7c>] (nf_hook_slow+0x58/0x104) from [<c0683bbc>] (ip_local_deliver+0x88/0xa8) [<c0683bbc>] (ip_local_deliver+0x88/0xa8) from [<c0683718>] (ip_rcv_finish+0x418/0x43c) [<c0683718>] (ip_rcv_finish+0x418/0x43c) from [<c062b1c4>] (__netif_receive_skb+0x4cc/0x598) [<c062b1c4>] (__netif_receive_skb+0x4cc/0x598) from [<c062b314>] (process_backlog+0x84/0x158) [<c062b314>] (process_backlog+0x84/0x158) from [<c062de84>] (net_rx_action+0x70/0x1dc) [<c062de84>] (net_rx_action+0x70/0x1dc) from [<c0088230>] (__do_softirq+0x11c/0x27c) [<c0088230>] (__do_softirq+0x11c/0x27c) from [<c008857c>] (do_softirq+0x44/0x50) [<c008857c>] (do_softirq+0x44/0x50) from [<c0088614>] (local_bh_enable_ip+0x8c/0xd0) [<c0088614>] (local_bh_enable_ip+0x8c/0xd0) from [<c06b0330>] (inet_stream_connect+0x164/0x298) [<c06b0330>] (inet_stream_connect+0x164/0x298) from [<c061d68c>] (sys_connect+0x88/0xc8) [<c061d68c>] (sys_connect+0x88/0xc8) from [<c000e340>] (ret_fast_syscall+0x0/0x30)Code: 2a000021 e59d2028 e59de01c e59f011c (e7824103) ---[ end trace da227214a82491bd ]--- Kernel panic - not syncing: Fatal exception in interruptThis comes about because CPU1 is executing
xt_replace_tablein response to a setsockopt syscall, resulting in:ret = xt_jumpstack_alloc(newinfo); --> newinfo->jumpstack = kzalloc(size, GFP_KERNEL); [...] table->private = newinfo; newinfo->initial_entries = private->initial_entries;Meanwhile, CPU0 is handling the network receive path and ends up in
ipt_do_table, resulting in:private = table->private; [...] jumpstack = (struct ipt_entry **)private->jumpstack[cpu];On weakly ordered memory architectures, the writes to
table->privateandnewinfo->jumpstackfrom CPU1 can be observed out of order by CPU0. Furthermore, on architectures which don't respect ordering of address dependencies (i.e. Alpha), the reads from CPU0 can also be re-ordered.
Quoted 5 lines
This patch adds an
smp_wmb()before the assignment totable->private(which is essentially publishing newinfo) to ensure that all writes to newinfo will be observed before plugging it into the table structure. A dependent-read barrier is also added on the consumer side, to ensure the same ordering requirement are also respected there.
We also need fixes for net/ipv6/netfilter/ip6_tables.c and net/ipv4/netfilter/arp_tables.c as well. Could you extend this patch and resend?
Thanks!
Hi Pablo,
Quoted 3 lines · Pablo Neira Ayuso
Quoted 2 lines · Will Deacon
During kernel stability testing on an SMP ARMv7 system, Yalin Wang reported the following panic from the netfilter code:
[...]
Quoted 9 lines
Quoted 5 lines
This patch adds an
smp_wmb()before the assignment totable->private(which is essentially publishing newinfo) to ensure that all writes to newinfo will be observed before plugging it into the table structure. A dependent-read barrier is also added on the consumer side, to ensure the same ordering requirement are also respected there.We also need fixes for
net/ipv6/netfilter/ip6_tables.candnet/ipv4/netfilter/arp_tables.cas well. Could you extend this patch and resend?
Sure, I can try, but that's going to require a bit of time to sit down and look at the shared data, access order, dependencies etc. I'm currently preparing for Edinburgh, so it might be a while before I get a chance to extend this.
Will
Quoted 11 lines · Will Deacon
Hi Pablo,
Quoted 3 lines
We also need fixes for
net/ipv6/netfilter/ip6_tables.candnet/ipv4/netfilter/arp_tables.cas well. Could you extend this patch and resend?Sure, I can try, but that's going to require a bit of time to sit down and look at the shared data, access order, dependencies etc. I'm currently preparing for Edinburgh, so it might be a while before I get a chance to extend this.
That's basically same code copy/pasted, so it should be relatively easy.
Will Deacon
Tested-by Wang, YalinQuoted 8 lines · Eric Dumazet
Quoted 7 lines · Will Deacon
Hi Pablo,
Quoted 3 lines
We also need fixes for
net/ipv6/netfilter/ip6_tables.candnet/ipv4/netfilter/arp_tables.cas well. Could you extend this patch and resend?
Quoted 6 lines
Quoted 4 lines
Sure, I can try, but that's going to require a bit of time to sit down and look at the shared data, access order, dependencies etc. I'm currently preparing for Edinburgh, so it might be a while before I get a chance to extend this.
That's basically same code copy/pasted, so it should be relatively easy.
Ok, I took a look and I think I see what you mean: there are just some additional consumers of the tables, so there aren't any additional writers afaict (at least, net/bridge/netfilter/ebtables.c uses rw locks so we don't have an issue there).
Attempt at an updated patch below.
Cheers,
Will
--->8
Quoted 1 line
From
128bff68debba70c3cec4b4ac01050d7e864aa83Mon Sep 17 00:00:00 2001
From: Will Deacon <[email protected]>
Date: Thu, 17 Oct 2013 13:34:13 +0100
Subject: [PATCH v2] netfilter: fix ordering of jumpstack allocation and table
update
During kernel stability testing on an SMP ARMv7 system, Yalin Wang reported the following panic from the netfilter code:
1fe0: 0000001c 5e2d3b10 4007e779 4009e110 60000010 00000032 ff565656 ff545454 [<c06c48dc>] (ipt_do_table+0x448/0x584) from [<c0655ef0>] (nf_iterate+0x48/0x7c) [<c0655ef0>] (nf_iterate+0x48/0x7c) from [<c0655f7c>] (nf_hook_slow+0x58/0x104) [<c0655f7c>] (nf_hook_slow+0x58/0x104) from [<c0683bbc>] (ip_local_deliver+0x88/0xa8) [<c0683bbc>] (ip_local_deliver+0x88/0xa8) from [<c0683718>] (ip_rcv_finish+0x418/0x43c) [<c0683718>] (ip_rcv_finish+0x418/0x43c) from [<c062b1c4>] (__netif_receive_skb+0x4cc/0x598) [<c062b1c4>] (__netif_receive_skb+0x4cc/0x598) from [<c062b314>] (process_backlog+0x84/0x158) [<c062b314>] (process_backlog+0x84/0x158) from [<c062de84>] (net_rx_action+0x70/0x1dc) [<c062de84>] (net_rx_action+0x70/0x1dc) from [<c0088230>] (__do_softirq+0x11c/0x27c) [<c0088230>] (__do_softirq+0x11c/0x27c) from [<c008857c>] (do_softirq+0x44/0x50) [<c008857c>] (do_softirq+0x44/0x50) from [<c0088614>] (local_bh_enable_ip+0x8c/0xd0) [<c0088614>] (local_bh_enable_ip+0x8c/0xd0) from [<c06b0330>] (inet_stream_connect+0x164/0x298) [<c06b0330>] (inet_stream_connect+0x164/0x298) from [<c061d68c>] (sys_connect+0x88/0xc8) [<c061d68c>] (sys_connect+0x88/0xc8) from [<c000e340>] (ret_fast_syscall+0x0/0x30)
Code: 2a000021 e59d2028 e59de01c e59f011c (e7824103) ---[ end trace da227214a82491bd ]--- Kernel panic - not syncing: Fatal exception in interrupt
This comes about because CPU1 is executing xt_replace_table in response to a setsockopt syscall, resulting in:
ret = xt_jumpstack_alloc(newinfo); --> newinfo->jumpstack = kzalloc(size, GFP_KERNEL); [...] table->private = newinfo; newinfo->initial_entries = private->initial_entries;
Meanwhile, CPU0 is handling the network receive path and ends up in ipt_do_table, resulting in:
private = table->private; [...] jumpstack = (struct ipt_entry **)private->jumpstack[cpu];
On weakly ordered memory architectures, the writes to table->private and newinfo->jumpstack from CPU1 can be observed out of order by CPU0. Furthermore, on architectures which don't respect ordering of address dependencies (i.e. Alpha), the reads from CPU0 can also be re-ordered.
This patch adds an smp_wmb() before the assignment to table->private (which is essentially publishing newinfo) to ensure that all writes to newinfo will be observed before plugging it into the table structure. A dependent-read barrier is also added on the consumer sides, to ensure the same ordering requirements are also respected there.
--- net/ipv4/netfilter/arp_tables.c | 5 +++++ net/ipv4/netfilter/ip_tables.c | 5 +++++ net/ipv6/netfilter/ip6_tables.c | 5 +++++ net/netfilter/x_tables.c | 7 ++++++- 4 files changed, 21 insertions(+), 1 deletion(-)
net/ipv4/netfilter/arp_tables.c +5 -0
@@ -271,6 +271,11 @@ unsigned int arpt_do_table(struct sk_buff *skb,local_bh_disable (); addend =xt_write_recseq_begin (); private = table->private; +/* +* Ensure we load private-> members after we've fetched the base +* pointer. +*/ +smp_read_barrier_depends (); table_base = private->entries[smp_processor_id ()]; e =get_entry (table_base, private->hook_entry[hook]);
net/ipv4/netfilter/ip_tables.c +5 -0
@@ -327,6 +327,11 @@ ipt_do_table(struct sk_buff *skb, addend =xt_write_recseq_begin (); private = table->private; cpu =smp_processor_id (); +/* +* Ensure we load private-> members after we've fetched the base +* pointer. +*/ +smp_read_barrier_depends (); table_base = private->entries[cpu]; jumpstack = (struct ipt_entry **)private->jumpstack[cpu]; stackptr =per_cpu_ptr (private->stackptr, cpu);
net/ipv6/netfilter/ip6_tables.c +5 -0
@@ -349,6 +349,11 @@ ip6t_do_table(struct sk_buff *skb,local_bh_disable (); addend =xt_write_recseq_begin (); private = table->private; +/* +* Ensure we load private-> members after we've fetched the base +* pointer. +*/ +smp_read_barrier_depends (); cpu =smp_processor_id (); table_base = private->entries[cpu]; jumpstack = (struct ip6t_entry **)private->jumpstack[cpu];
net/netfilter/x_tables.c +6 -1
@@ -845,8 +845,13 @@ xt_replace_table(struct xt_table *table,return NULL ; } - table->private = newinfo; newinfo->initial_entries = private->initial_entries; +/* +* Ensure contents of newinfo are visible before assigning to +* private. +*/ +smp_wmb (); + table->private = newinfo;/* * Even though table entries have now been swapped, other CPU's
Signature · 1 line
1.8.3.2
Quoted 14 lines · Will Deacon
Quoted 13 lines · Eric Dumazet
Quoted 11 lines · Will Deacon
Hi Pablo,
Quoted 3 lines
We also need fixes for
net/ipv6/netfilter/ip6_tables.candnet/ipv4/netfilter/arp_tables.cas well. Could you extend this patch and resend?Sure, I can try, but that's going to require a bit of time to sit down and look at the shared data, access order, dependencies etc. I'm currently preparing for Edinburgh, so it might be a while before I get a chance to extend this.
Quoted 8 lines
Quoted 1 line
That's basically same code copy/pasted, so it should be relatively easy.
Ok, I took a look and I think I see what you mean: there are just some additional consumers of the tables, so there aren't any additional writers afaict (at least,
net/bridge/netfilter/ebtables.cuses rw locks so we don't have an issue there).Attempt at an updated patch below.
Applied, thanks Will.