3.3-rc snmp6 panic
Subsystems (3)
I've been seeing an occasional panic when I shut down my router since I put 3.3 on there. It happens about once a week, always during shutdown. It wedges before I can get a good capture of the trace. This is the best I've captured so far.. https://twitpic.com/8wh5l5
(apologies in advance for blurriness)
Quoted 1 line
From comparing the Code: line, and the objdump output, the code it's
choking on in mld_sendpack seems to be a skb_dst macro in the NF_HOOK..
err = NF_HOOK(NFPROTO_IPV6, NF_INET_LOCAL_OUT, skb, NULL, skb->dev,
2dd9: 4c 8b 43 20 mov 0x20(%rbx),%r8
2ddd: e9 00 00 00 00 jmpq 2de2 <mld_sendpack+0x1b2>
static inline struct dst_entry *skb_dst(const struct sk_buff *skb)
{
/* If refdst was not refcounted, check we still are in a
* rcu_read_lock section
*/
WARN_ON((skb->_skb_refdst & SKB_DST_NOREF) &&
2de2: 48 8b 43 58 mov 0x58(%rbx),%rax
2de6: a8 01 test $0x1,%al
2de8: 0f 85 d2 01 00 00 jne 2fc0 <mld_sendpack+0x390>
!rcu_read_lock_held() &&
!rcu_read_lock_bh_held());
return (struct dst_entry *)(skb->_skb_refdst & SKB_DST_PTRMASK);
2dee: 48 83 e0 fe and $0xfffffffffffffffe,%rax
2df2: 48 89 df mov %rbx,%rdi
2df5: ff 50 58 callq *0x58(%rax) <----- BOOMThis machine is running an snmpd, for my mrtg setup, so the teardown of that service is probably what's triggering it. But I can start/stop it in a loop as much as I want without it happening, so maybe the kernel needs to accumulate some state from it for a while first ?
Anyone have any ideas what's happening here ?
Dave
Quoted 30 lines · Dave Jones
I've been seeing an occasional panic when I shut down my router since I put 3.3 on there. It happens about once a week, always during shutdown. It wedges before I can get a good capture of the trace. This is the best I've captured so far.. https://twitpic.com/8wh5l5
(apologies in advance for blurriness)From comparing the Code: line, and the objdump output, the code it's choking on in
mld_sendpackseems to be askb_dstmacro in theNF_HOOK..err = NF_HOOK(NFPROTO_IPV6, NF_INET_LOCAL_OUT, skb, NULL, skb->dev, 2dd9: 4c 8b 43 20 mov 0x20(%rbx),%r8 2ddd: e9 00 00 00 00 jmpq 2de2 <mld_sendpack+0x1b2> static inline struct dst_entry *skb_dst(const struct sk_buff *skb) { /* If refdst was not refcounted, check we still are in a * rcu_read_lock section */ WARN_ON((skb->_skb_refdst & SKB_DST_NOREF) && 2de2: 48 8b 43 58 mov 0x58(%rbx),%rax 2de6: a8 01 test $0x1,%al 2de8: 0f 85 d2 01 00 00 jne 2fc0 <mld_sendpack+0x390> !rcu_read_lock_held() && !rcu_read_lock_bh_held()); return (struct dst_entry *)(skb->_skb_refdst & SKB_DST_PTRMASK); 2dee: 48 83 e0 fe and $0xfffffffffffffffe,%rax 2df2: 48 89 df mov %rbx,%rdi 2df5: ff 50 58 callq *0x58(%rax) <----- BOOM
Quoted 7 lines
This machine is running an snmpd, for my mrtg setup, so the teardown of that service is probably what's triggering it. But I can start/stop it in a loop as much as I want without it happening, so maybe the kernel needs to accumulate some state from it for a while first ?
Anyone have any ideas what's happening here ?
Hmm, please try :
net/ipv6/route.c +1 -1
@@ -1077,7 +1077,7 @@ struct dst_entry *icmp6_dst_alloc(struct net_device *dev,struct net *net =dev_net (dev);if (unlikely (!idev)) -return NULL ; +return ERR_PTR (-ENODEV ); rt =ip6_dst_alloc (&net->ipv6.ip6_dst_ops, dev,0 );if (unlikely (!rt)) {
Quoted 36 lines · Eric Dumazet
Quoted 35 lines · Dave Jones
I've been seeing an occasional panic when I shut down my router since I put 3.3 on there. It happens about once a week, always during shutdown. It wedges before I can get a good capture of the trace. This is the best I've captured so far.. https://twitpic.com/8wh5l5
(apologies in advance for blurriness)From comparing the Code: line, and the objdump output, the code it's choking on in
mld_sendpackseems to be askb_dstmacro in theNF_HOOK..err = NF_HOOK(NFPROTO_IPV6, NF_INET_LOCAL_OUT, skb, NULL, skb->dev, 2dd9: 4c 8b 43 20 mov 0x20(%rbx),%r8 2ddd: e9 00 00 00 00 jmpq 2de2 <mld_sendpack+0x1b2> static inline struct dst_entry *skb_dst(const struct sk_buff *skb) { /* If refdst was not refcounted, check we still are in a * rcu_read_lock section */ WARN_ON((skb->_skb_refdst & SKB_DST_NOREF) && 2de2: 48 8b 43 58 mov 0x58(%rbx),%rax 2de6: a8 01 test $0x1,%al 2de8: 0f 85 d2 01 00 00 jne 2fc0 <mld_sendpack+0x390> !rcu_read_lock_held() && !rcu_read_lock_bh_held()); return (struct dst_entry *)(skb->_skb_refdst & SKB_DST_PTRMASK); 2dee: 48 83 e0 fe and $0xfffffffffffffffe,%rax 2df2: 48 89 df mov %rbx,%rdi 2df5: ff 50 58 callq *0x58(%rax) <----- BOOMThis machine is running an snmpd, for my mrtg setup, so the teardown of that service is probably what's triggering it. But I can start/stop it in a loop as much as I want without it happening, so maybe the kernel needs to accumulate some state from it for a while first ?
Quoted 4 lines
Quoted 2 lines
Anyone have any ideas what's happening here ?
Hmm, please try :
Since patch is trivial, here is the official submission
Thanks !
[PATCH] ipv6: fix icmp6_dst_alloc()
commit 87a115783 ( ipv6: Move xfrm_lookup() call down into icmp6_dst_alloc().) forgot to convert one error path, leading to crashes in mld_sendpack()
Many thanks to Dave Jones for providing a very complete bug report.
--- net/ipv6/route.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
net/ipv6/route.c +1 -1
@@ -1077,7 +1077,7 @@ struct dst_entry *icmp6_dst_alloc(struct net_device *dev,struct net *net =dev_net (dev);if (unlikely (!idev)) -return NULL ; +return ERR_PTR (-ENODEV ); rt =ip6_dst_alloc (&net->ipv6.ip6_dst_ops, dev,0 );if (unlikely (!rt)) {
David Miller
maintainerFrom: Eric Dumazet <[email protected]>
Date: Thu, 15 Mar 2012 00:13:11 -0700
Quoted 10 lines
[PATCH] ipv6: fix
icmp6_dst_alloc()commit
87a115783( ipv6: Movexfrm_lookup()call down intoicmp6_dst_alloc().) forgot to convert one error path, leading to crashes inmld_sendpack()Many thanks to Dave Jones for providing a very complete bug report.
Reported-by Dave Jones <[email protected]>Signed-off-by Eric Dumazet <[email protected]>
Applied and queued up for -stable, thanks.
Quoted 8 lines · Eric Dumazet
Quoted 6 lines
This machine is running an snmpd, for my mrtg setup, so the teardown of that service is probably what's triggering it. But I can start/stop it in a loop as much as I want without it happening, so maybe the kernel needs to accumulate some state from it for a while first ?
Anyone have any ideas what's happening here ?
Hmm, please try :
Will do. Given I can't reproduce it quickly though, it might be a while before I can provide feedback on whether it does the trick :)
thanks,
Dave