net: Convert skb->csum_(start|offset) integrity BUG_ON() to WARN_ON() & drop
Subsystems (3)
PATCH 1/1 net: Convert skb->csum_(start|offset) integrity BUG_ON() to WARN_ON() & drop
skb_checksum_help() verifies the integrity of skb->csum_start and skb->csum_offset with BUG_ON()s.
They have been hit with IPoIB which uses a 64K MTU. If a TCP retransmission gets partially ACKed and collapsed multiple times it is possible for the headroom to grow beyond 64K which will overflow the 16bit skb->csum_start.
This in turn will trigger the BUG_ON() in skb_checksum_help(). Convert these to WARN_ON() and drop the packet.
--- net/core/dev.c | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-)
net/core/dev.c +5 -2
@@ -2047,11 +2047,14 @@ int skb_checksum_help(struct sk_buff *skb) } offset =skb_checksum_start_offset (skb); -BUG_ON (offset >=skb_headlen (skb)); +if (WARN_ON (offset >=skb_headlen (skb))) +return -ERANGE ; + csum =skb_checksum (skb, offset, skb->len - offset,0 ); offset += skb->csum_offset; -BUG_ON (offset +sizeof (__sum16) >skb_headlen (skb)); +if (WARN_ON (offset +sizeof (__sum16) >skb_headlen (skb))) +return -ERANGE ;if (skb_cloned (skb) && !skb_clone_writable (skb, offset +sizeof (__sum16))) {
Quoted 4 lines · Thomas Graf
They have been hit with IPoIB which uses a 64K MTU. If a TCP retransmission gets partially ACKed and collapsed multiple times it is possible for the headroom to grow beyond 64K which will overflow the 16bit
skb->csum_start.
On the subject of fixing this, I considered:
a) Reallocate the headroom in tcp_trim_head() if it would overflow. I disregarded this idea because replacing the old skb with the new skb on the write queue for such a rare situation seems overkill.
b) No longer collapse if the new skb would result in a a headroom + data that exceeds 64K. This seems to be the most trivial fix.
c) Increase size of csum_start or store checksum_start_offset differently.
Other ideas?
David Miller
maintainerFrom: Thomas Graf <[email protected]>
Date: Wed, 13 Feb 2013 23:48:43 +0000
Quoted 13 lines · Thomas Graf
Quoted 4 lines
They have been hit with IPoIB which uses a 64K MTU. If a TCP retransmission gets partially ACKed and collapsed multiple times it is possible for the headroom to grow beyond 64K which will overflow the 16bit
skb->csum_start.On the subject of fixing this, I considered:
a) Reallocate the headroom in
tcp_trim_head()if it would overflow. I disregarded this idea because replacing the old skb with the new skb on the write queue for such a rare situation seems overkill.
Quoted 8 lines
b) No longer collapse if the new skb would result in a a headroom + data that exceeds 64K. This seems to be the most trivial fix.
c) Increase size of
csum_startor storechecksum_start_offsetdifferently.Other ideas?
"b" is a good idea.
Let's not paper over this, this BUG_ON() is really a BUG_ON() meaning "FIX ME NOW" :-)
Quoted 2 lines · David Miller
From: Thomas Graf <[email protected]>
Date: Wed, 13 Feb 2013 23:48:43 +0000
[...]
Quoted 3 lines
Quoted 3 lines
b) No longer collapse if the new skb would result in a a headroom + data that exceeds 64K. This seems to be the most trivial fix.
[...]
Quoted 3 lines
Quoted 1 line
Other ideas?
"b" is a good idea.
OK, patch to do so being tested by original reporter.
Quoted 2 lines
Let's not paper over this, this
BUG_ON()is really aBUG_ON()meaning "FIX ME NOW" :-)
Maybe it's my general dislike of BUG_ON() in the processing path, especially if the bug condition can be influenced remotely. It looks absolutely doable to trigger the previously mentioned partial acking & collapsing on purpose by a malicious receiver even with an MTU of 1500. I believe we should avoid total DoS in future similar situations that we don't think of yet.
Quoted 6 lines · Thomas Graf
Maybe it's my general dislike of
BUG_ON()in the processing path, especially if the bug condition can be influenced remotely. It looks absolutely doable to trigger the previously mentioned partial acking & collapsing on purpose by a malicious receiver even with an MTU of 1500. I believe we should avoid total DoS in future similar situations that we don't think of yet.
It seems not possible to avoid bugs, being a BUG_ON() or a out of bound memory access or whatever. We must fix them eventually.
In this case, it seems we must limit payload to
65535 - MAX_TCP_HEADER
It would make tcp_xmit_size_goal() a bit shorter.
net/ipv4/tcp.c +1 -4
@@ -793,10 +793,7 @@ static unsigned int tcp_xmit_size_goal(struct sock *sk, u32 mss_now, xmit_size_goal = mss_now;if (large_allowed &&sk_can_gso (sk)) { - xmit_size_goal = ((sk->sk_gso_max_size -1 ) - -inet_csk (sk)->icsk_af_ops->net_header_len - -inet_csk (sk)->icsk_ext_hdr_len - - tp->tcp_header_len); + xmit_size_goal = sk->sk_gso_max_size -1 -MAX_TCP_HEADER ;/* TSQ : try to have two TSO segments in flight */ xmit_size_goal = min_t(u32, xmit_size_goal,
Quoted 2 lines · Eric Dumazet
It seems not possible to avoid bugs, being a
BUG_ON()or a out of bound memory access or whatever. We must fix them eventually.
Of course, I never intended to not fix this but I still think leaving the BUG_ON() is wrong ;-)
Quoted 12 lines
In this case, it seems we must limit payload to
65535 -
MAX_TCP_HEADERIt would make
tcp_xmit_size_goal()a bit shorter.net/ipv4/tcp.c +0
-0@@ -793,10 +793,7 @@ static unsigned int tcp_xmit_size_goal(struct sock *sk, u32 mss_now, xmit_size_goal = mss_now;
net/ipv4/tcp.c static unsigned int tcp_xmit_size_goal(struct sock *sk, u32 mss_now, · line 799
Quoted 10 lines
net/ipv4/tcp.c +1
-4if (large_allowed &&sk_can_gso (sk)) { - xmit_size_goal = ((sk->sk_gso_max_size -1 ) - -inet_csk (sk)->icsk_af_ops->net_header_len - -inet_csk (sk)->icsk_ext_hdr_len - - tp->tcp_header_len); + xmit_size_goal = sk->sk_gso_max_size -1 -MAX_TCP_HEADER ;/* TSQ : try to have two TSO segments in flight */ xmit_size_goal = min_t(u32, xmit_size_goal,
I don't think this would help. The allocated skb data will still exceed 64K and thus after trimming the acked data and collapsing the header might be stored after the 64K mark.
We would have to limit the skb tailroom to 64K upon allocation. That would mean we would waste some of the additional space that kmalloc() might have given us:
net/core/skbuff.c +2 -0
@@ -241,6 +241,8 @@ struct sk_buff *__alloc_skb(unsigned int size, gfp_t gfp_mas* to allow max possible filling before reallocation. */ size =SKB_WITH_OVERHEAD (ksize (data)); +/* ensure that all offsets based on skb->head fit into 16bits */ + size =min_t (int, size,65535 );prefetchw (data + size);/*
Or if that is not ideal, avoiding the collapse that causes the overflow would also help:
net/ipv4/tcp_output.c +6 -0
@@ -2301,6 +2301,12 @@ static void tcp_retrans_try_collapse(struct sock *sk, struct sk_buff *to,if (after (TCP_SKB_CB (skb)->end_seq,tcp_wnd_end (tp)))break ; +/* Never collapse if the resulting headroom + data exceeds +* 64K as that is the maximum csum_start can cover. +*/ +if (skb_headroom (to) + to->len + skb->len >65535 ) +break ; +tcp_collapse_retrans (sk, to); } }
Quoted 27 lines · Thomas Graf
Quoted 2 lines · Eric Dumazet
It seems not possible to avoid bugs, being a
BUG_ON()or a out of bound memory access or whatever. We must fix them eventually.Of course, I never intended to not fix this but I still think leaving the
BUG_ON()is wrong ;-)Quoted 20 lines
In this case, it seems we must limit payload to
65535 -
MAX_TCP_HEADERIt would make
tcp_xmit_size_goal()a bit shorter.net/ipv4/tcp.c +1
-4@@ -793,10 +793,7 @@ static unsigned int tcp_xmit_size_goal(struct sock *sk, u32 mss_now, xmit_size_goal = mss_now;if (large_allowed &&sk_can_gso (sk)) { - xmit_size_goal = ((sk->sk_gso_max_size -1 ) - -inet_csk (sk)->icsk_af_ops->net_header_len - -inet_csk (sk)->icsk_ext_hdr_len - - tp->tcp_header_len); + xmit_size_goal = sk->sk_gso_max_size -1 -MAX_TCP_HEADER ;
Quoted 7 lines
net/ipv4/tcp.c static unsigned int tcp_xmit_size_goal(struct sock *sk, u32 mss_now, · line 799
Quoted 2 lines
net/ipv4/tcp.c +0
-0/* TSQ : try to have two TSO segments in flight */ xmit_size_goal = min_t(u32, xmit_size_goal,I don't think this would help. The allocated skb data will still exceed 64K and thus after trimming the acked data and collapsing the header might be stored after the 64K mark.
OK, I now understand your issue.
Quoted 9 lines
We would have to limit the skb tailroom to 64K upon allocation. That would mean we would waste some of the additional space that
kmalloc()might have given us:net/core/skbuff.c +0
-0
net/core/skbuff.c struct sk_buff *__alloc_skb(unsigned int size, gfp_t gfp_mas · line 247
Quoted 7 lines
net/core/skbuff.c +2
-0@@ -241,6 +241,8 @@ struct sk_buff *__alloc_skb(unsigned int size, gfp_t gfp_mas* to allow max possible filling before reallocation. */ size =SKB_WITH_OVERHEAD (ksize (data)); +/* ensure that all offsets based on skb->head fit into 16bits */ + size =min_t (int, size,65535 );prefetchw (data + size);
This part is certainly not good.
alloc_skb() is generic and some users really want more than 64K
Quoted 13 lines
net/core/skbuff.c +0
-0/* Or if that is not ideal, avoiding the collapse that causes the overflow would also help:
net/ipv4/tcp_output.c +0
-0@@ -2301,6 +2301,12 @@ static void tcp_retrans_try_collapse(struct sock *sk, struct sk_buff *to,if (after (TCP_SKB_CB (skb)->end_seq,tcp_wnd_end (tp)))
net/ipv4/tcp_output.c static void tcp_retrans_try_collapse(struct sock *sk, struct sk_buff *to, · line 2311
Quoted 10 lines
net/ipv4/tcp_output.c +6
-0break ; +/* Never collapse if the resulting headroom + data exceeds +* 64K as that is the maximum csum_start can cover. +*/ +if (skb_headroom (to) + to->len + skb->len >65535 ) +break ; +tcp_collapse_retrans (sk, to);
Definitely better.
David Miller
maintainerFrom: Thomas Graf <[email protected]>
Date: Thu, 14 Feb 2013 10:18:53 +0000
Quoted 7 lines · David Miller
Maybe it's my general dislike of
BUG_ON()in the processing path, especially if the bug condition can be influenced remotely. It looks absolutely doable to trigger the previously mentioned partial acking & collapsing on purpose by a malicious receiver even with an MTU of 1500. I believe we should avoid total DoS in future similar situations that we don't think of yet.
I heard that people can very effectively protect themselves from DoS's by not using Infiniband. :-)