tcp_cubic: fix divide error when SYN flood
Subsystems (3)
PATCH 1/1 tcp_cubic: fix divide error when SYN flood
From: Liu Yu <[email protected]>
commit b9f47a3aaeab (tcp_cubic: limit delayed_ack ratio to prevent divide error) try to prevent divide error, but it still has a little chance that delayed_ack can reach zero. In case machine sufferring continuous SYN flood, the argument cnt could be big, and so that ratio+cnt could get overflow and may happen to be zero. If so, min(ratio, ACK_RATIO_LIMIT) will calculate to be zero.
The crash log may like this:
..
<6>[27536.083145] possible SYN flooding on port 8080. Sending cookies. <6>[27596.092124] possible SYN flooding on port 8080. Sending cookies. <6>[27656.109832] possible SYN flooding on port 8080. Sending cookies. <0>[27676.940730] divide error: 0000 [#1] SMP <0>[27676.987890] last sysfs file: /sys/class/scsi_host/host0/proc_name
..
---net/ipv4/tcp_cubic.c | 4 ++--
1 files changed, 2 insertions(+), 2 deletions(-)
net/ipv4/tcp_cubic.c +0 -0
@@ -404,12 +404,12 @@ static void bictcp_acked(struct sock *sk, u32 cnt, s32 rtt_us)
u32 delay;
if (icsk->icsk_ca_state == TCP_CA_Open) {
- u32 ratio = ca->delayed_ack;
+ u64 ratio = ca->delayed_ack;
ratio -= ca->delayed_ack >> ACK_RATIO_SHIFT;
ratio += cnt;
- ca->delayed_ack = min(ratio, ACK_RATIO_LIMIT);
+ ca->delayed_ack = min_t(u64, ratio, ACK_RATIO_LIMIT); }
/* Some calls are for duplicates without timetamps */
Signature · 1 line
1.7.1
Quoted 33 lines · Liu Yu
From: Liu Yu <[email protected]>
commit
b9f47a3aaeab(tcp_cubic: limitdelayed_ackratio to prevent divide error) try to prevent divide error, but it still has a little chance thatdelayed_ackcan reach zero. In case machine sufferring continuous SYN flood, the argument cnt could be big, and so that ratio+cnt could get overflow and may happen to be zero. If so,min(ratio, ACK_RATIO_LIMIT)will calculate to be zero.The crash log may like this:
..<6>[27536.083145] possible SYN flooding on port 8080. Sending cookies. <6>[27596.092124] possible SYN flooding on port 8080. Sending cookies. <6>[27656.109832] possible SYN flooding on port 8080. Sending cookies. <0>[27676.940730] divide error: 0000 [#1] SMP <0>[27676.987890] last sysfs file: /sys/class/scsi_host/host0/proc_name..
CC Stephen Hemminger <[email protected]>Signed-off-by Liu Yu <[email protected]>---
net/ipv4/tcp_cubic.c| 4 ++--1 files changed, 2 insertions(+), 2 deletions(-)net/ipv4/tcp_cubic.c +0
-1@@ -404,12 +404,12 @@ static void bictcp_acked(struct sock *sk, u32 cnt, s32 rtt_us)u32 delay;if (icsk->icsk_ca_state == TCP_CA_Open) { -u32 ratio = ca->delayed_ack;
net/ipv4/tcp_cubic.c static void bictcp_acked(struct sock *sk, u32 cnt, s32 rtt_us) · line 415
Quoted 10 lines
net/ipv4/tcp_cubic.c +2
-1+u64 ratio = ca->delayed_ack; ratio -= ca->delayed_ack >>ACK_RATIO_SHIFT ; ratio += cnt; - ca->delayed_ack =min (ratio,ACK_RATIO_LIMIT ); + ca->delayed_ack =min_t (u64, ratio,ACK_RATIO_LIMIT ); }/* Some calls are for duplicates without timetamps */
Hi Liu
Your patch is mangled. Check Documentation/email-clients.txt
I do not see how a SYN (flood or not) can be cause of the crash you have.
Please provide full stack trace.
Thanks
Eric Dumazet said, at 2014/4/22 10:28:
Quoted 47 lines · Liu Yu
Quoted 33 lines
From: Liu Yu <[email protected]>
commit
b9f47a3aaeab(tcp_cubic: limitdelayed_ackratio to prevent divide error) try to prevent divide error, but it still has a little chance thatdelayed_ackcan reach zero. In case machine sufferring continuous SYN flood, the argument cnt could be big, and so that ratio+cnt could get overflow and may happen to be zero. If so,min(ratio, ACK_RATIO_LIMIT)will calculate to be zero.The crash log may like this:
..<6>[27536.083145] possible SYN flooding on port 8080. Sending cookies. <6>[27596.092124] possible SYN flooding on port 8080. Sending cookies. <6>[27656.109832] possible SYN flooding on port 8080. Sending cookies. <0>[27676.940730] divide error: 0000 [#1] SMP <0>[27676.987890] last sysfs file: /sys/class/scsi_host/host0/proc_name..
CC Stephen Hemminger <[email protected]>Signed-off-by Liu Yu <[email protected]>---
net/ipv4/tcp_cubic.c| 4 ++--1 files changed, 2 insertions(+), 2 deletions(-)net/ipv4/tcp_cubic.c +0
-1@@ -404,12 +404,12 @@ static void bictcp_acked(struct sock *sk, u32 cnt, s32 rtt_us)u32 delay;if (icsk->icsk_ca_state == TCP_CA_Open) { -u32 ratio = ca->delayed_ack;net/ipv4/tcp_cubic.c static void bictcp_acked(struct sock *sk, u32 cnt, s32 rtt_us) · line 415
Quoted 10 lines
net/ipv4/tcp_cubic.c +2
-1+u64 ratio = ca->delayed_ack; ratio -= ca->delayed_ack >>ACK_RATIO_SHIFT ; ratio += cnt; - ca->delayed_ack =min (ratio,ACK_RATIO_LIMIT ); + ca->delayed_ack =min_t (u64, ratio,ACK_RATIO_LIMIT ); }/* Some calls are for duplicates without timetamps */ Hi Liu
Quoted 7 lines
Your patch is mangled. Check
Documentation/email-clients.txtI do not see how a SYN (flood or not) can be cause of the crash you have.
Please provide full stack trace.
Hi Eric
below is the stack
<6>[27292.140097] possible SYN flooding on port 8080. Sending cookies. <6>[27352.212394] possible SYN flooding on port 8080. Sending cookies. <6>[27412.211454] possible SYN flooding on port 8080. Sending cookies. <6>[27472.212504] possible SYN flooding on port 8080. Sending cookies. <6>[27536.083145] possible SYN flooding on port 8080. Sending cookies. <6>[27596.092124] possible SYN flooding on port 8080. Sending cookies. <6>[27656.109832] possible SYN flooding on port 80. Sending cookies. <0>[27676.940730] divide error: 0000 [#1] SMP <0>[27676.987890] last sysfs file: /sys/class/scsi_host/host0/proc_name <4>[27677.060792] CPU 0 0 <6>[27677.198382] RIP: 0010:[<ffffffff8176940d>] [<ffffffff8176940d>] bictcp_cong_avoid+0x14d/0x2a0 <6>[27677.301544] RSP: 0018:ffff8800282039e0 EFLAGS: 00010246 <6>[27677.365088] RAX: 0000000000000e80 RBX: ffff88010eefd340 RCX: 0000000000000000 <6>[27677.450468] RDX: 0000000000000000 RSI: ffff88010eefd6b8 RDI: 0000000000000367 <6>[27677.535850] RBP: ffff8800282039f0 R08: 00000000000001d1 R09: 0000000000000004 <6>[27677.621232] R10: 0000000000000001 R11: 0000000000000000 R12: 0000000042e1612d <6>[27677.706612] R13: 0000000000000506 R14: 0000000042e1612d R15: 0000000000000004 <6>[27677.791993] FS: 0000000000000000(0000) GS:ffff880028200000(0000) knlGS:0000000000000000 <6>[27677.888815] CS: 0010 DS: 0018 ES: 0018 CR0: 000000008005003b <6>[27677.957556] CR2: 00007f9d7a94b000 CR3: 0000000001001000 CR4: 00000000000406f0 <6>[27678.042938] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 <6>[27678.128317] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 <4>[27678.213700] Process swapper (pid: 0, threadinfo ffffffff81c3e000, task ffffffff81c413e0) <0>[27678.310520] Stack: <4>[27678.334544] ffff88010eefd340 ffff88010eefd340 ffff880028203a10 ffffffff8173b286 <4>[27678.421276] <0> 0000000042e1612d 0000000000000006 ffff880028203ad0 ffffffff81740970 <4>[27678.513416] <0> ffff88010eefd3f0 0000000000000000 0000015128203b10 42e1612d42e160c5 <0>[27678.607743] Call Trace: <0>[27678.636965] <IRQ> <4>[27678.662136] [<ffffffff8173b286>] tcp_cong_avoid+0x16/0x30 <4>[27678.727754] [<ffffffff81740970>] tcp_ack+0xd50/0x1270 <4>[27678.789219] [<ffffffff81741400>] ? tcp_validate_incoming+0x220/0x350 <4>[27678.866280] [<ffffffff81742863>] tcp_rcv_established+0x203/0x660 <4>[27678.939181] [<ffffffff81748daf>] tcp_v4_do_rcv+0x11f/0x290 <4>[27679.005844] [<ffffffff8170d51f>] ? nf_iterate+0x5f/0x90 <4>[27679.069384] [<ffffffff8174ae73>] tcp_v4_rcv+0x583/0x6e0 <4>[27679.132925] [<ffffffff8170d5be>] ? nf_hook_slow+0x6e/0xe0 <4>[27679.198553] [<ffffffff8172c080>] ? ip_local_deliver_finish+0x0/0x120 <4>[27679.275610] [<ffffffff8172c0df>] ip_local_deliver_finish+0x5f/0x120 <4>[27679.351632] [<ffffffff8172c66b>] ip_local_deliver+0x3b/0x90 <4>[27679.419333] [<ffffffff8172be52>] ip_rcv_finish+0x152/0x380 <4>[27679.485995] [<ffffffff8172c531>] ip_rcv+0x251/0x350 <4>[27679.545376] [<ffffffff816f17c7>] __netif_receive_skb+0x267/0x31
Quoted 43 lines · Liu Yu
Hi Eric
below is the stack
<6>[27292.140097] possible SYN flooding on port 8080. Sending cookies. <6>[27352.212394] possible SYN flooding on port 8080. Sending cookies. <6>[27412.211454] possible SYN flooding on port 8080. Sending cookies. <6>[27472.212504] possible SYN flooding on port 8080. Sending cookies. <6>[27536.083145] possible SYN flooding on port 8080. Sending cookies. <6>[27596.092124] possible SYN flooding on port 8080. Sending cookies. <6>[27656.109832] possible SYN flooding on port 80. Sending cookies. <0>[27676.940730] divide error: 0000 [#1] SMP <0>[27676.987890] last sysfs file: /sys/class/scsi_host/host0/proc_name <4>[27677.060792] CPU 0 0 <6>[27677.198382] RIP: 0010:[<ffffffff8176940d>] [<ffffffff8176940d>] bictcp_cong_avoid+0x14d/0x2a0 <6>[27677.301544] RSP: 0018:ffff8800282039e0 EFLAGS: 00010246 <6>[27677.365088] RAX: 0000000000000e80 RBX: ffff88010eefd340 RCX: 0000000000000000 <6>[27677.450468] RDX: 0000000000000000 RSI: ffff88010eefd6b8 RDI: 0000000000000367 <6>[27677.535850] RBP: ffff8800282039f0 R08: 00000000000001d1 R09: 0000000000000004 <6>[27677.621232] R10: 0000000000000001 R11: 0000000000000000 R12: 0000000042e1612d <6>[27677.706612] R13: 0000000000000506 R14: 0000000042e1612d R15: 0000000000000004 <6>[27677.791993] FS: 0000000000000000(0000) GS:ffff880028200000(0000) knlGS:0000000000000000 <6>[27677.888815] CS: 0010 DS: 0018 ES: 0018 CR0: 000000008005003b <6>[27677.957556] CR2: 00007f9d7a94b000 CR3: 0000000001001000 CR4: 00000000000406f0 <6>[27678.042938] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 <6>[27678.128317] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 <4>[27678.213700] Process swapper (pid: 0, threadinfo ffffffff81c3e000, task ffffffff81c413e0) <0>[27678.310520] Stack: <4>[27678.334544] ffff88010eefd340 ffff88010eefd340 ffff880028203a10 ffffffff8173b286 <4>[27678.421276] <0> 0000000042e1612d 0000000000000006 ffff880028203ad0 ffffffff81740970 <4>[27678.513416] <0> ffff88010eefd3f0 0000000000000000 0000015128203b10 42e1612d42e160c5 <0>[27678.607743] Call Trace: <0>[27678.636965] <IRQ> <4>[27678.662136] [<ffffffff8173b286>] tcp_cong_avoid+0x16/0x30 <4>[27678.727754] [<ffffffff81740970>] tcp_ack+0xd50/0x1270 <4>[27678.789219] [<ffffffff81741400>] ? tcp_validate_incoming+0x220/0x350 <4>[27678.866280] [<ffffffff81742863>] tcp_rcv_established+0x203/0x660 <4>[27678.939181] [<ffffffff81748daf>] tcp_v4_do_rcv+0x11f/0x290 <4>[27679.005844] [<ffffffff8170d51f>] ? nf_iterate+0x5f/0x90
Quoted 8 lines
<4>[27679.069384] [<ffffffff8174ae73>] tcp_v4_rcv+0x583/0x6e0 <4>[27679.132925] [<ffffffff8170d5be>] ? nf_hook_slow+0x6e/0xe0 <4>[27679.198553] [<ffffffff8172c080>] ? ip_local_deliver_finish+0x0/0x120 <4>[27679.275610] [<ffffffff8172c0df>] ip_local_deliver_finish+0x5f/0x120 <4>[27679.351632] [<ffffffff8172c66b>] ip_local_deliver+0x3b/0x90 <4>[27679.419333] [<ffffffff8172be52>] ip_rcv_finish+0x152/0x380 <4>[27679.485995] [<ffffffff8172c531>] ip_rcv+0x251/0x350 <4>[27679.545376] [<ffffffff816f17c7>] __netif_receive_skb+0x267/0x31
Sorry, could you provide kernel version as well ?
There is a bug somewhere else, as tcp_cong_avoid() should not have a 'negative' acked param.
Thanks
please ignore this.
sorry for bad format.
Liu Yu said, at 2014/4/22 9:53:
Quoted 45 lines
From: Liu Yu <[email protected]>
commit
b9f47a3aaeab(tcp_cubic: limitdelayed_ackratio to prevent divide error) try to prevent divide error, but it still has a little chance thatdelayed_ackcan reach zero. In case machine sufferring continuous SYN flood, the argument cnt could be big, and so that ratio+cnt could get overflow and may happen to be zero. If so,min(ratio, ACK_RATIO_LIMIT)will calculate to be zero.The crash log may like this:
..<6>[27536.083145] possible SYN flooding on port 8080. Sending cookies. <6>[27596.092124] possible SYN flooding on port 8080. Sending cookies. <6>[27656.109832] possible SYN flooding on port 8080. Sending cookies. <0>[27676.940730] divide error: 0000 [#1] SMP <0>[27676.987890] last sysfs file: /sys/class/scsi_host/host0/proc_name..
CC Stephen Hemminger <[email protected]>Signed-off-by Liu Yu <[email protected]>---
net/ipv4/tcp_cubic.c| 4 ++--1 files changed, 2 insertions(+), 2 deletions(-)net/ipv4/tcp_cubic.c +2
-2@@ -404,12 +404,12 @@ static void bictcp_acked(struct sock *sk, u32 cnt, s32 rtt_us)u32 delay;if (icsk->icsk_ca_state == TCP_CA_Open) { -u32 ratio = ca->delayed_ack; +u64 ratio = ca->delayed_ack; ratio -= ca->delayed_ack >>ACK_RATIO_SHIFT ; ratio += cnt; - ca->delayed_ack =min (ratio,ACK_RATIO_LIMIT ); + ca->delayed_ack =min_t (u64, ratio,ACK_RATIO_LIMIT ); }/* Some calls are for duplicates without timetamps */