New: kernel panic when receiving an ipsec packet
Subsystems (3)
Quoted 18 lines · On Wed, 22 Feb 2012 10:24:19 GMT [email protected] wrote:
https://bugzilla.kernel.org/show_bug.cgi?id=42809
Summary: kernel panic when receiving an ipsec packet Product: Networking Version: 2.5 Kernel Version: 2.6.32.54 Platform: All OS/Version: Linux Tree: Mainline Status: NEW Severity: high Priority: P1 Component: IPV4 AssignedTo: [email protected] ReportedBy: [email protected] Regression: No
Quoted 5 lines
As soon as I receive an ipsec packet (NETKEY of course) I get a kernel panic. Even magicsysrq keys do not work anymore.
O.S. Debian Squeeze amd64. I tried both Strongswan and Openswan.
That is a really old kernel now. Could you try with later kernel?
Thanks for CCing netdev, I was thinking about doing the same.
Il 22/02/2012 18:20, Stephen Hemminger ha scritto:
Quoted 1 line
That is a really old kernel now. Could you try with later kernel?
I can (a few minutes during night) but I really need a fix for 2.6.32. This is a production server with dozens of Xen virtual machines and a heavily patched kernel (xen dom0, layer 7, imq, esfq, advanced routing etc). I did test a "vanilla" kernel last night (not really vanilla, but a clean official debian kernel) and I had the same problem.
Thanks,
Niccolò
Hi,
The bug is still present in latest 3.2.7 vanilla kernel. I wasted the whole day debugging that damn thing and I finally discovered the root cause.
The problem is with my Traverse Solos multi-port ADSL2+ PCI card[1] (which has open source drivers included in the kernel) when using RFC 2684 routed.
I have two adsl lines, the first one connected using RFC 2684 routed, the second one using PPPoA.
If I create a vpn toward the PPPoA line it works flawlessly, while if I create a vpn toward the RFC 2684 routed line the whole system hangs in a kernel panic (with both 2.6.32.54 and 3.2.7). I really don't know how to fix it and I need to setup that damn ipsec vpn :(
This is the bug on bugzilla.kernel.org:
https://bugzilla.kernel.org/show_bug.cgi?id=42809
Niccolò
[1]http://www.traverse.com.au/productview.php?product_id=116
Quoted 13 lines · Niccolò Belli
Hi,
The bug is still present in latest 3.2.7 vanilla kernel. I wasted the whole day debugging that damn thing and I finally discovered the root cause. The problem is with my Traverse Solos multi-port ADSL2+ PCI card[1] (which has open source drivers included in the kernel) when using RFC 2684 routed.
I have two adsl lines, the first one connected using RFC 2684 routed, the second one using PPPoA.
If I create a vpn toward the PPPoA line it works flawlessly, while if I create a vpn toward the RFC 2684 routed line the whole system hangs in a kernel panic (with both 2.6.32.54 and 3.2.7).
I really don't know how to fix it and I need to setup that damn ipsec vpn :(
Quoted 7 lines
This is the bug on bugzilla.kernel.org:
https://bugzilla.kernel.org/show_bug.cgi?id=42809Niccolò
[1]http://www.traverse.com.au/productview.php?product_id=116
Which driver handles this Traverse Solos card ?
Quoted 1 line · Eric Dumazet
Which driver handles this Traverse Solos card ?
solos_pci, which also requires the atm module.
(I'm just trying to help here; I'm not affected by the bug.)
Quoted 1 line · Eric Dumazet
Which driver handles this Traverse Solos card ?
If br2684_push() is used, it seems it lacks proper call to skb_reset_mac_header(skb) in paths where eth_type_trans() is not called.
Later in xfrm4_mode_tunnel_input() we crash because we assume skb_mac_header() is valid.
Il 23/02/2012 02:38, Eric Dumazet ha scritto:
Quoted 1 line
Which driver handles this Traverse Solos card ?
drivers/atm/solos-pci.c
~# lsmod | grep solos
solos_pci 20009 2 atm 32378 7 pppoatm,br2684,solos_pci
Niccolò
Il 23/02/2012 02:38, Eric Dumazet ha scritto:
Quoted 1 line
Which driver handles this Traverse Solos card ?
drivers/atm/solos-pci.c
~# lsmod | grep solos
solos_pci 20009 2 atm 32378 7 pppoatm,br2684,solos_pci
Niccolò
Quoted 9 lines · Niccolò Belli
Il 23/02/2012 02:38, Eric Dumazet ha scritto:
Quoted 1 line
Which driver handles this Traverse Solos card ?
drivers/atm/solos-pci.c~# lsmod | grep solos
solos_pci 20009 2 atm 32378 7 pppoatm,br2684,solos_pci
Thanks !
Please try following patch.
net/ipv4/xfrm4_mode_tunnel.c +5 -3
@@ -84,9 +84,11 @@ static int xfrm4_mode_tunnel_input(struct xfrm_state *x, struct sk_buff *skb)if (!(x->props.flags &XFRM_STATE_NOECN ))ipip_ecn_decapsulate (skb); - old_mac =skb_mac_header (skb); -skb_set_mac_header (skb, -skb->mac_len); -memmove (skb_mac_header (skb), old_mac, skb->mac_len); +if (skb_mac_header_was_set (skb)) { + old_mac =skb_mac_header (skb); +skb_set_mac_header (skb, -skb->mac_len); +memmove (skb_mac_header (skb), old_mac, skb->mac_len); + }skb_reset_network_header (skb); err =0 ;
Il 23/02/2012 03:06, Eric Dumazet ha scritto:
Quoted 10 lines
Please try following patch.
net/ipv4/xfrm4_mode_tunnel.c +0
-0@@ -84,9 +84,11 @@ static int xfrm4_mode_tunnel_input(struct xfrm_state *x, struct sk_buff *skb)if (!(x->props.flags&XFRM_STATE_NOECN ))ipip_ecn_decapsulate (skb);
net/ipv4/xfrm4_mode_tunnel.c static int xfrm4_mode_tunnel_input(struct xfrm_state *x, struct sk_buff *skb) · line 93
Quoted 10 lines
net/ipv4/xfrm4_mode_tunnel.c +5
-3- old_mac =skb_mac_header (skb); -skb_set_mac_header (skb, -skb->mac_len); -memmove (skb_mac_header (skb), old_mac, skb->mac_len); +if (skb_mac_header_was_set (skb)) { + old_mac =skb_mac_header (skb); +skb_set_mac_header (skb, -skb->mac_len); +memmove (skb_mac_header (skb), old_mac, skb->mac_len); + }skb_reset_network_header (skb); err =0 ;
Wow, thanks! It will try it as soon as possible tomorrow, going sleeping now :)
Niccolò
Il 23/02/2012 03:06, Eric Dumazet ha scritto:
Quoted 13 lines
Thanks !
Please try following patch.
net/ipv4/xfrm4_mode_tunnel.c +0
-1@@ -84,9 +84,11 @@ static int xfrm4_mode_tunnel_input(struct xfrm_state *x, struct sk_buff *skb)if (!(x->props.flags&XFRM_STATE_NOECN ))ipip_ecn_decapsulate (skb); - old_mac =skb_mac_header (skb);
net/ipv4/xfrm4_mode_tunnel.c static int xfrm4_mode_tunnel_input(struct xfrm_state *x, struct sk_buff *skb) · line 94
Quoted 10 lines
net/ipv4/xfrm4_mode_tunnel.c +5
-2-skb_set_mac_header (skb, -skb->mac_len); -memmove (skb_mac_header (skb), old_mac, skb->mac_len); +if (skb_mac_header_was_set (skb)) { + old_mac =skb_mac_header (skb); +skb_set_mac_header (skb, -skb->mac_len); +memmove (skb_mac_header (skb), old_mac, skb->mac_len); + }skb_reset_network_header (skb); err =0 ;
Your patch does solve the problem, thanks!
Niccolò
Bug#660804: [Bug 42809] New: kernel panic when receiving an ipsec packet
In reply to Eric DumazetQuoted 9 lines · On Thu, 23 Feb 2012 02:54:39 +0100 Eric Dumazet <[email protected]> wrote:
Quoted 1 line · Eric Dumazet
Which driver handles this Traverse Solos card ?
If
br2684_push()is used, it seems it lacks proper call toskb_reset_mac_header(skb)in paths whereeth_type_trans()is not called.Later in
xfrm4_mode_tunnel_input()we crash because we assumeskb_mac_header()is valid.
when br2684_push() doesnt call eth_type_trans() the underlying packet doesnt have a mac address header -- just an llc header that says 'ip packet is next'.
Eric Dumazet
Tested-by Niccolò BelliQuoted 3 lines
Your patch does solve the problem, thanks!
Thanks for testing.
Here is the official patch I submit for review then.
[PATCH] ipsec: be careful of non existing mac headers
Nicollo Belli reported ipsec crashes in case we handle a frame without mac header (atm in his case)
Before copying mac header, better make sure it is present.
Bugzilla reference: https://bugzilla.kernel.org/show_bug.cgi?id=42809
--- net/ipv4/xfrm4_mode_beet.c | 9 +++++---- net/ipv4/xfrm4_mode_tunnel.c | 10 ++++++---- net/ipv6/xfrm6_mode_beet.c | 9 +++++---- net/ipv6/xfrm6_mode_tunnel.c | 10 ++++++---- 4 files changed, 22 insertions(+), 16 deletions(-)
net/ipv4/xfrm4_mode_beet.c +5 -4
@@ -111,10 +111,11 @@ static int xfrm4_beet_input(struct xfrm_state *x, struct sk_buff *skb)skb_push (skb,sizeof (*iph));skb_reset_network_header (skb); -memmove (skb->data - skb->mac_len,skb_mac_header (skb), - skb->mac_len); -skb_set_mac_header (skb, -skb->mac_len); - +if (skb_mac_header_was_set (skb)) { +memmove (skb->data - skb->mac_len,skb_mac_header (skb), + skb->mac_len); +skb_set_mac_header (skb, -skb->mac_len); + }xfrm4_beet_make_header (skb); iph =ip_hdr (skb);
net/ipv4/xfrm4_mode_tunnel.c +6 -4
@@ -66,7 +66,6 @@ static int xfrm4_mode_tunnel_output(struct xfrm_state *x, struct sk_buff *skb)static int xfrm4_mode_tunnel_input (struct xfrm_state *x,struct sk_buff *skb) { -const unsigned char *old_mac;int err = -EINVAL ;if (XFRM_MODE_SKB_CB (skb)->protocol !=IPPROTO_IPIP )
@@ -84,9 +83,12 @@ static int xfrm4_mode_tunnel_input(struct xfrm_state *x, struct sk_buff *skb)if (!(x->props.flags &XFRM_STATE_NOECN ))ipip_ecn_decapsulate (skb); - old_mac =skb_mac_header (skb); -skb_set_mac_header (skb, -skb->mac_len); -memmove (skb_mac_header (skb), old_mac, skb->mac_len); +if (skb_mac_header_was_set (skb)) { +const unsigned char *old_mac =skb_mac_header (skb); + +skb_set_mac_header (skb, -skb->mac_len); +memmove (skb_mac_header (skb), old_mac, skb->mac_len); + }skb_reset_network_header (skb); err =0 ;
net/ipv6/xfrm6_mode_beet.c +5 -4
@@ -80,7 +80,6 @@ static int xfrm6_beet_output(struct xfrm_state *x, struct sk_buff *skb)static int xfrm6_beet_input (struct xfrm_state *x,struct sk_buff *skb) {struct ipv6hdr *ip6h; -const unsigned char *old_mac;int size =sizeof (struct ipv6hdr );int err;
@@ -91,10 +90,12 @@ static int xfrm6_beet_input(struct xfrm_state *x, struct sk_buff *skb)__skb_push (skb, size);skb_reset_network_header (skb); - old_mac =skb_mac_header (skb); -skb_set_mac_header (skb, -skb->mac_len); -memmove (skb_mac_header (skb), old_mac, skb->mac_len); +if (skb_mac_header_was_set (skb)) { +const unsigned char *old_mac =skb_mac_header (skb); +skb_set_mac_header (skb, -skb->mac_len); +memmove (skb_mac_header (skb), old_mac, skb->mac_len); + }xfrm6_beet_make_header (skb); ip6h =ipv6_hdr (skb);
net/ipv6/xfrm6_mode_tunnel.c +6 -4
@@ -63,7 +63,6 @@ static int xfrm6_mode_tunnel_output(struct xfrm_state *x, struct sk_buff *skb)static int xfrm6_mode_tunnel_input (struct xfrm_state *x,struct sk_buff *skb) {int err = -EINVAL ; -const unsigned char *old_mac;if (XFRM_MODE_SKB_CB (skb)->protocol !=IPPROTO_IPV6 ) goto out;
@@ -80,9 +79,12 @@ static int xfrm6_mode_tunnel_input(struct xfrm_state *x, struct sk_buff *skb)if (!(x->props.flags &XFRM_STATE_NOECN ))ipip6_ecn_decapsulate (skb); - old_mac =skb_mac_header (skb); -skb_set_mac_header (skb, -skb->mac_len); -memmove (skb_mac_header (skb), old_mac, skb->mac_len); +if (skb_mac_header_was_set (skb)) { +const unsigned char *old_mac =skb_mac_header (skb); + +skb_set_mac_header (skb, -skb->mac_len); +memmove (skb_mac_header (skb), old_mac, skb->mac_len); + }skb_reset_network_header (skb); err =0 ;
Quoted 4 lines · Eric Dumazet
[PATCH] ipsec: be careful of non existing mac headers
Nicollo Belli reported ipsec crashes in case we handle a frame without mac header (atm in his case)
Oops sorry for your name being mangled in changelog, its Niccolò as correctly spelled in the "Reported-by" and "Tested-by" tags
Il 23/02/2012 15:39, Eric Dumazet ha scritto:
Quoted 2 lines
Oops sorry for your name being mangled in changelog, its Niccolò as correctly spelled in the "Reported-by" and "Tested-by" tags
Don't worry and thanks for your help. I'm currently running the official patch you submitted for review.
Niccolò
David Miller
maintainerFrom: Eric Dumazet <[email protected]>
Date: Thu, 23 Feb 2012 15:36:26 +0100
Quoted 5 lines
[PATCH] ipsec: be careful of non existing mac headers
Nicollo Belli reported ipsec crashes in case we handle a frame without mac header (atm in his case)
Quoted 7 lines
Before copying mac header, better make sure it is present.
Bugzilla reference: https://bugzilla.kernel.org/show_bug.cgi?id=42809
Reported-by Niccolò Belli <[email protected]>Tested-by Niccolò Belli <[email protected]>Signed-off-by Eric Dumazet <[email protected]>
Three instances of the same piece of code, maybe a helper function is appropriate at that point? :-) You might even get ambitious and add a big comment to that helper function explaining the situation.
Quoted 3 lines · David Miller
Three instances of the same piece of code, maybe a helper function is appropriate at that point? :-) You might even get ambitious and add a big comment to that helper function explaining the situation.
I did have this idea but refrained because this kind of things is harder to backport to stable kernels.
I'll make a cleanup in net-next if you agree ?
David Miller
maintainerFrom: Eric Dumazet <[email protected]>
Date: Thu, 23 Feb 2012 21:17:23 +0100
Quoted 9 lines · David Miller
Quoted 3 lines
Three instances of the same piece of code, maybe a helper function is appropriate at that point? :-) You might even get ambitious and add a big comment to that helper function explaining the situation.
I did have this idea but refrained because this kind of things is harder to backport to stable kernels.
I'll make a cleanup in net-next if you agree ?
I think the work to backport is equal whether you use a helper function or not. Heck, use an inline function so you don't have to worry about module exports or any details like that.
Besides, I'm the one who likely has to backport this thing, so... :-)
Quoted 5 lines · David Miller
I think the work to backport is equal whether you use a helper function or not. Heck, use an inline function so you don't have to worry about module exports or any details like that.
Besides, I'm the one who likely has to backport this thing, so... :-)
Great, I'll send an updated version in a couple of minutes ;)
PATCH v2 1/1 ipsec: be careful of non existing mac headers
In reply to David MillerNiccolo Belli reported ipsec crashes in case we handle a frame without mac header (atm in his case)
Before copying mac header, better make sure it is present.
Bugzilla reference: https://bugzilla.kernel.org/show_bug.cgi?id=42809
---
V2: added skb_mac_header_rebuild() helper as David suggested.
include/linux/skbuff.h | 10 ++++++++++ net/ipv4/xfrm4_mode_beet.c | 5 +---- net/ipv4/xfrm4_mode_tunnel.c | 6 ++---- net/ipv6/xfrm6_mode_beet.c | 6 +----- net/ipv6/xfrm6_mode_tunnel.c | 6 ++---- 5 files changed, 16 insertions(+), 17 deletions(-)
include/linux/skbuff.h +10 -0
@@ -1465,6 +1465,16 @@ static inline void skb_set_mac_header(struct sk_buff *skb, const int offset) }#endif /* NET_SKBUFF_DATA_USES_OFFSET */ +static inline void skb_mac_header_rebuild (struct sk_buff *skb) +{ +if (skb_mac_header_was_set (skb)) { +const unsigned char *old_mac =skb_mac_header (skb); + +skb_set_mac_header (skb, -skb->mac_len); +memmove (skb_mac_header (skb), old_mac, skb->mac_len); + } +} +static inline int skb_checksum_start_offset (const struct sk_buff *skb) {return skb->csum_start -skb_headroom (skb);
net/ipv4/xfrm4_mode_beet.c +1 -4
@@ -110,10 +110,7 @@ static int xfrm4_beet_input(struct xfrm_state *x, struct sk_buff *skb)skb_push (skb,sizeof (*iph));skb_reset_network_header (skb); - -memmove (skb->data - skb->mac_len,skb_mac_header (skb), - skb->mac_len); -skb_set_mac_header (skb, -skb->mac_len); +skb_mac_header_rebuild (skb);xfrm4_beet_make_header (skb);
net/ipv4/xfrm4_mode_tunnel.c +2 -4
@@ -66,7 +66,6 @@ static int xfrm4_mode_tunnel_output(struct xfrm_state *x, struct sk_buff *skb)static int xfrm4_mode_tunnel_input (struct xfrm_state *x,struct sk_buff *skb) { -const unsigned char *old_mac;int err = -EINVAL ;if (XFRM_MODE_SKB_CB (skb)->protocol !=IPPROTO_IPIP )
@@ -84,10 +83,9 @@ static int xfrm4_mode_tunnel_input(struct xfrm_state *x, struct sk_buff *skb)if (!(x->props.flags &XFRM_STATE_NOECN ))ipip_ecn_decapsulate (skb); - old_mac =skb_mac_header (skb); -skb_set_mac_header (skb, -skb->mac_len); -memmove (skb_mac_header (skb), old_mac, skb->mac_len);skb_reset_network_header (skb); +skb_mac_header_rebuild (skb); + err =0 ; out:
net/ipv6/xfrm6_mode_beet.c +1 -5
@@ -80,7 +80,6 @@ static int xfrm6_beet_output(struct xfrm_state *x, struct sk_buff *skb)static int xfrm6_beet_input (struct xfrm_state *x,struct sk_buff *skb) {struct ipv6hdr *ip6h; -const unsigned char *old_mac;int size =sizeof (struct ipv6hdr );int err;
@@ -90,10 +89,7 @@ static int xfrm6_beet_input(struct xfrm_state *x, struct sk_buff *skb)__skb_push (skb, size);skb_reset_network_header (skb); - - old_mac =skb_mac_header (skb); -skb_set_mac_header (skb, -skb->mac_len); -memmove (skb_mac_header (skb), old_mac, skb->mac_len); +skb_mac_header_rebuild (skb);xfrm6_beet_make_header (skb);
net/ipv6/xfrm6_mode_tunnel.c +2 -4
@@ -63,7 +63,6 @@ static int xfrm6_mode_tunnel_output(struct xfrm_state *x, struct sk_buff *skb)static int xfrm6_mode_tunnel_input (struct xfrm_state *x,struct sk_buff *skb) {int err = -EINVAL ; -const unsigned char *old_mac;if (XFRM_MODE_SKB_CB (skb)->protocol !=IPPROTO_IPV6 ) goto out;
@@ -80,10 +79,9 @@ static int xfrm6_mode_tunnel_input(struct xfrm_state *x, struct sk_buff *skb)if (!(x->props.flags &XFRM_STATE_NOECN ))ipip6_ecn_decapsulate (skb); - old_mac =skb_mac_header (skb); -skb_set_mac_header (skb, -skb->mac_len); -memmove (skb_mac_header (skb), old_mac, skb->mac_len);skb_reset_network_header (skb); +skb_mac_header_rebuild (skb); + err =0 ; out:
David Miller
maintainerRe: [PATCH V2] ipsec: be careful of non existing mac headers
From: Eric Dumazet <[email protected]>
Date: Thu, 23 Feb 2012 21:55:02 +0100
Quoted 10 lines
Niccolo Belli reported ipsec crashes in case we handle a frame without mac header (atm in his case)
Before copying mac header, better make sure it is present.
Bugzilla reference: https://bugzilla.kernel.org/show_bug.cgi?id=42809
Reported-by Niccolò Belli <[email protected]>Tested-by Niccolò Belli <[email protected]>Signed-off-by Eric Dumazet <[email protected]>
Applied and queued up for -stable, thanks.
BTW, if anyone wants to see what's queued up for -stable I have a public patchwork bundle for it:
http://patchwork.ozlabs.org/bundle/davem/stable/
Just FYI.