xfrm: warning on negative dst refcount

PATCH Superseded Steffen Klassert 3 participants 7 messages Jun 4, 2010

Subsystems (3)

Networking Network drivers Networking IPsec

lore

Share & tools

lore

Steffen Klassert

author maintainer

2010-06-04 10:40 UTC #0 lore raw

I see the warning below frequently when I'm running iperf on a IPsec connection. It seems that dst_pop() drops a refcount on a noref dst_entry. I was able to fix this by changing dst_pop() to a new function skb_dst_pop() which uses skb_dst_drop(skb) instead of dst_release(dst) to drop the reference if necessary. I'll send the patch that fixed the issue for me in repy to this mail.
I don't know that much about the noref work, so I'm not sure whether this is the right fix, but I got rid of the warning at least.

Steffen

Jun  4 10:21:24 mainline kernel: [ 1334.203913] WARNING: at /home/secunet/git/linux-sinafe-2.6/net/core/dst.c:276 xfrm_output_resume+0x2d3/0x35e()
Jun  4 10:21:24 mainline kernel: [ 1334.203915] Hardware name:         
Jun  4 10:21:24 mainline kernel: [ 1334.203916] Modules linked in: authenc esp4 xfrm4_mode_tunnel aes_x86_64 aes_generic cbc sha1_generic xfrm_user ipv6 acpi_cpufreq mperf cpufreq_userspace cpufreq_stats cpufreq_ondemand freq_table cpufreq_conservative cpufreq_powersave container fan video output sbs sbshc battery af_packet ac fuse loop option usb_wwan usbserial sr_mod cdrom iTCO_wdt ehci_hcd thermal uhci_hcd serio_raw psmouse tpm_tis tpm tpm_bios iTCO_vendor_support pcspkr processor thermal_sys evdev usbcore button ata_generic
Jun  4 10:21:24 mainline kernel: [ 1334.203944] Pid: 3337, comm: dd Tainted: G        W   2.6.35-rc1+ #276
Jun  4 10:21:24 mainline kernel: [ 1334.203946] Call Trace:
Jun  4 10:21:24 mainline kernel: [ 1334.203947]  <IRQ>  [<ffffffff81261fbb>] ? xfrm_output_resume+0x2d3/0x35e
Jun  4 10:21:24 mainline kernel: [ 1334.203952]  [<ffffffff81261fbb>] ? xfrm_output_resume+0x2d3/0x35e
Jun  4 10:21:24 mainline kernel: [ 1334.203955]  [<ffffffff8102700d>] ? warn_slowpath_common+0x78/0x8d
Jun  4 10:21:24 mainline kernel: [ 1334.203958]  [<ffffffff81261fbb>] ? xfrm_output_resume+0x2d3/0x35e
Jun  4 10:21:24 mainline kernel: [ 1334.203961]  [<ffffffff8122bbe0>] ? ip_queue_xmit+0x2bc/0x304
Jun  4 10:21:24 mainline kernel: [ 1334.203964]  [<ffffffff8123b2db>] ? tcp_transmit_skb+0x6d2/0x701
Jun  4 10:21:24 mainline kernel: [ 1334.203967]  [<ffffffff81233df9>] ? tcp_cong_avoid+0xe/0x1d
Jun  4 10:21:24 mainline kernel: [ 1334.203970]  [<ffffffff81235bdf>] ? tcp_fin+0x74/0x17c
Jun  4 10:21:24 mainline kernel: [ 1334.203972]  [<ffffffff81236796>] ? tcp_data_queue+0x2fc/0xb64
Jun  4 10:21:24 mainline kernel: [ 1334.203975]  [<ffffffff81239d61>] ? tcp_rcv_state_process+0x927/0x99b
Jun  4 10:21:24 mainline kernel: [ 1334.203978]  [<ffffffff8123f563>] ? tcp_v4_do_rcv+0x18b/0x1d0
Jun  4 10:21:24 mainline kernel: [ 1334.203981]  [<ffffffff81241160>] ? tcp_v4_rcv+0x485/0x721
Jun  4 10:21:24 mainline kernel: [ 1334.203985]  [<ffffffff81227154>] ? ip_local_deliver+0xda/0x165
Jun  4 10:21:24 mainline kernel: [ 1334.203988]  [<ffffffff812276f4>] ? ip_rcv+0x515/0x53d
Jun  4 10:21:24 mainline kernel: [ 1334.203991]  [<ffffffff81209192>] ? process_backlog+0x99/0x174
Jun  4 10:21:24 mainline kernel: [ 1334.203994]  [<ffffffff81208e41>] ? net_rx_action+0xab/0x153
Jun  4 10:21:24 mainline kernel: [ 1334.203997]  [<ffffffff8102b8f5>] ? __do_softirq+0x8d/0x107
Jun  4 10:21:24 mainline kernel: [ 1334.204000]  [<ffffffff81002aec>] ? call_softirq+0x1c/0x28
Jun  4 10:21:24 mainline kernel: [ 1334.204003]  [<ffffffff810043a5>] ? do_softirq+0x31/0x64
Jun  4 10:21:24 mainline kernel: [ 1334.204005]  [<ffffffff8102b826>] ? irq_exit+0x36/0x78
Jun  4 10:21:24 mainline kernel: [ 1334.204008]  [<ffffffff81003c4f>] ? do_IRQ+0xa7/0xbd
Jun  4 10:21:24 mainline kernel: [ 1334.204011]  [<ffffffff8127dad3>] ? ret_from_intr+0x0/0xa
Jun  4 10:21:24 mainline kernel: [ 1334.204012]  <EOI>  [<ffffffff8127d810>] ? _raw_spin_unlock_irqrestore+0x4/0x5
Jun  4 10:21:24 mainline kernel: [ 1334.204018]  [<ffffffff8108e618>] ? pipe_write+0x413/0x447
Jun  4 10:21:24 mainline kernel: [ 1334.204021]  [<ffffffff81087aab>] ? do_sync_write+0xb3/0xf7
Jun  4 10:21:24 mainline kernel: [ 1334.204024]  [<ffffffff810c7007>] ? kmsg_read+0x44/0x4e
Jun  4 10:21:24 mainline kernel: [ 1334.204027]  [<ffffffff8127c26c>] ? schedule+0x525/0x5db
Jun  4 10:21:24 mainline kernel: [ 1334.204030]  [<ffffffff81088120>] ? vfs_write+0xad/0x149
Jun  4 10:21:24 mainline kernel: [ 1334.204033]  [<ffffffff81088669>] ? sys_write+0x45/0x6e
Jun  4 10:21:24 mainline kernel: [ 1334.204035]  [<ffffffff81001ceb>] ? system_call_fastpath+0x16/0x1b
Jun  4 10:21:24 mainline kernel: [ 1334.204037] ---[ end trace ab41f96596a62e77 ]---

Steffen Klassert

author maintainer

2010-06-04 10:41 UTC #1 lore raw

PATCH 1/1 net: check for refcount if pop a stacked dst_entry

xfrm triggers a warning if dst_pop() drops a refcount on a noref dst. This patch changes dst_pop() to skb_dst_pop(). skb_dst_pop() drops the refcnt only on a refcounted dst.

Signed-off-by Steffen Klassert <[email protected]>

---
 include/net/dst.h      |    6 +++---
 net/xfrm/xfrm_output.c |    2 +-
 2 files changed, 4 insertions(+), 4 deletions(-)

include/net/dst.h +3 -3

index 612069b..acd1538 100644
@@ -250,11 +250,11 @@ static inline void skb_tunnel_rx(struct sk_buff *skb, struct net_device *dev)
  * Linux networking.  Thus, destinations are stackable.
  */
 
-static inline struct dst_entry *dst_pop(struct dst_entry *dst)
+static inline struct dst_entry *skb_dst_pop(struct sk_buff *skb)
 {
-	struct dst_entry *child = dst_clone(dst->child);
+	struct dst_entry *child = dst_clone(skb_dst(skb)->child);
 
-	dst_release(dst);
+	skb_dst_drop(skb);
 	return child;
 }

net/xfrm/xfrm_output.c +1 -1

index 6a32915..db62a06 100644
@@ -95,7 +95,7 @@ resume:
 			goto error_nolock;
 		}
 
-		dst = dst_pop(dst);
+		dst = skb_dst_pop(skb);
 		if (!dst) {
 			XFRM_INC_STATS(net, LINUX_MIB_XFRMOUTERROR);
 			err = -EHOSTUNREACH;

Signature · 1 line

1.5.6.5

Eric Dumazet

2010-06-04 10:51 UTC #2 lore raw

Re: [PATCH] net: check for refcount if pop a stacked dst_entry

Quoted 30 lines · Steffen Klassert

xfrm triggers a warning if dst_pop() drops a refcount on a noref dst. This patch changes dst_pop() to skb_dst_pop(). skb_dst_pop() drops the refcnt only on a refcounted dst.

Signed-off-by Steffen Klassert <[email protected]>

---
 include/net/dst.h      |    6 +++---
 net/xfrm/xfrm_output.c |    2 +-
 2 files changed, 4 insertions(+), 4 deletions(-)

include/net/dst.h +3 -3

index 612069b..acd1538 100644
@@ -250,11 +250,11 @@ static inline void skb_tunnel_rx(struct sk_buff *skb, struct net_device *dev)
  * Linux networking.  Thus, destinations are stackable.
  */
 
-static inline struct dst_entry *dst_pop(struct dst_entry *dst)
+static inline struct dst_entry *skb_dst_pop(struct sk_buff *skb)
 {
-	struct dst_entry *child = dst_clone(dst->child);
+	struct dst_entry *child = dst_clone(skb_dst(skb)->child);
 
 
 
 
-	dst_release(dst);
+	skb_dst_drop(skb);

Quoted 2 lines

return child;
}

Hmm, this might fix the thing, but we probably can do it without the dst_clone(), if you replace the

skb_dst_set(skb, dst);

skb_dst_set_noref(skb, dst);

in xfrm_output_one() ?

Quoted 14 lines

net/xfrm/xfrm_output.c +1 -1

index 6a32915..db62a06 100644
@@ -95,7 +95,7 @@ resume:
 			goto error_nolock;
 		}
 
-		dst = dst_pop(dst);
+		dst = skb_dst_pop(skb);
 		if (!dst) {
 			XFRM_INC_STATS(net, LINUX_MIB_XFRMOUTERROR);
 			err = -EHOSTUNREACH;

Steffen Klassert

author maintainer

2010-06-04 11:24 UTC #3 lore raw

Re: [PATCH] net: check for refcount if pop a stacked dst_entry

Quoted 4 lines · Eric Dumazet

Hmm, this might fix the thing, but we probably can do it without the dst_clone(), if you replace the

Quoted 8 lines

skb_dst_set(skb, dst);
by
skb_dst_set_noref(skb, dst);
in xfrm_output_one() ?

Yes, this should work too. I'll update the patch to use skb_dst_set_noref() in xfrm_output_one() and remove the dst_clone() in skb_dst_pop().

Steffen Klassert

author maintainer

2010-06-04 11:58 UTC #4 lore raw

PATCH v2 1/1 net: check for refcount if pop a stacked dst_entry

In reply to Eric Dumazet

xfrm triggers a warning if dst_pop() drops a refcount on a noref dst. This patch changes dst_pop() to skb_dst_pop(). skb_dst_pop() drops the refcnt only on a refcounted dst. Also we don't clone the child dst_entry, so it is not refcounted and we can use skb_dst_set_noref() in xfrm_output_one().

Signed-off-by Steffen Klassert <[email protected]>

---
 include/net/dst.h      |    6 +++---
 net/xfrm/xfrm_output.c |    4 ++--
 2 files changed, 5 insertions(+), 5 deletions(-)

include/net/dst.h +3 -3

index 612069b..81d1413 100644
@@ -250,11 +250,11 @@ static inline void skb_tunnel_rx(struct sk_buff *skb, struct net_device *dev)
  * Linux networking.  Thus, destinations are stackable.
  */
 
-static inline struct dst_entry *dst_pop(struct dst_entry *dst)
+static inline struct dst_entry *skb_dst_pop(struct sk_buff *skb)
 {
-	struct dst_entry *child = dst_clone(dst->child);
+	struct dst_entry *child = skb_dst(skb)->child;
 
-	dst_release(dst);
+	skb_dst_drop(skb);
 	return child;
 }

net/xfrm/xfrm_output.c +2 -2

index 6a32915..a3cca0a 100644
@@ -95,13 +95,13 @@ resume:
 			goto error_nolock;
 		}
 
-		dst = dst_pop(dst);
+		dst = skb_dst_pop(skb);
 		if (!dst) {
 			XFRM_INC_STATS(net, LINUX_MIB_XFRMOUTERROR);
 			err = -EHOSTUNREACH;
 			goto error_nolock;
 		}
-		skb_dst_set(skb, dst);
+		skb_dst_set_noref(skb, dst);
 		x = dst->xfrm;
 	} while (x && !(x->outer_mode->flags & XFRM_MODE_FLAG_TUNNEL));

Signature · 1 line

1.5.6.5

Eric Dumazet

2010-06-04 12:06 UTC #5 lore raw

Re: [PATCH v2] net: check for refcount if pop a stacked dst_entry

Quoted 9 lines · Steffen Klassert

xfrm triggers a warning if dst_pop() drops a refcount on a noref dst. This patch changes dst_pop() to skb_dst_pop(). skb_dst_pop() drops the refcnt only on a refcounted dst. Also we don't clone the child dst_entry, so it is not refcounted and we can use skb_dst_set_noref() in xfrm_output_one().
Signed-off-by Steffen Klassert <[email protected]>
---

Thanks a lot Steffen !

Signed-off-by Eric Dumazet <[email protected]>

David Miller

maintainer Applied

2010-06-04 22:58 UTC #6 lore raw

Re: [PATCH v2] net: check for refcount if pop a stacked dst_entry

From: Eric Dumazet <[email protected]>
Date: Fri, 04 Jun 2010 14:06:47 +0200

Quoted 8 lines · Steffen Klassert

Quoted 7 lines
xfrm triggers a warning if dst_pop() drops a refcount on a noref dst. This patch changes dst_pop() to skb_dst_pop(). skb_dst_pop() drops the refcnt only on a refcounted dst. Also we don't clone the child dst_entry, so it is not refcounted and we can use skb_dst_set_noref() in xfrm_output_one().

Quoted 6 lines

Quoted 2 lines
Signed-off-by Steffen Klassert <[email protected]>
---
Thanks a lot Steffen !
Signed-off-by Eric Dumazet <[email protected]>

Applied, thanks guys!