2.6.23-rc[68]-mm: network hangs
Subsystems (2)
Hi,
Quoted 1 line
From time to time, I experience some complete network hangs:
Suddenly, all network connections become unresponsive. Even "ping 127.0.0.1" doesn't work. SysRq-w does not show any blocked processus.
When such hang happen, I have to reboot (shutdown does work).
This is not easily reproducible: it happens several minutes after boot (could be 45 minutes or 2 hours). I do not use heavy networking apps (like P2P). My typical usage is a Gnome desktop with browser, mailer, IM, video or audio streaming.
I have a single PC connected to a DSL router via ethernet (so no LAN with NFS or CIFS).
This happens with 2.6.23-rc8-mm2 and 2.6.23-rc6-mm1. I can't remember when I first see this problem. Maybe 2 months ago.
I attached the output of "strace ping 127.0.0.1". How can I collect some more data when this problem happens ?
~~
laurent
execve("/bin/ping", ["ping", "127.0.0.1"], [/* 42 vars */]) = 0
brk(0) = 0x93ac000access("/etc/ld.so.nohwcap", F_OK) = -1 ENOENT (No such file or directory)mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb7f9b000access("/etc/ld.so.preload", R_OK) = -1 ENOENT (No such file or directory)open("/etc/ld.so.cache", O_RDONLY) = 3fstat64(3, {st_mode=S_IFREG|0644, st_size=57464, ...}) = 0mmap2(NULL, 57464, PROT_READ, MAP_PRIVATE, 3, 0) = 0xb7f8c000
close(3) = 0access("/etc/ld.so.nohwcap", F_OK) = -1 ENOENT (No such file or directory)open("/lib/tls/i686/cmov/libresolv.so.2", O_RDONLY) = 3read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\0!\0\000"..., 512) = 512 fstat64(3, {st_mode=S_IFREG|0644, st_size=67408, ...}) = 0mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb7f8b000 mmap2(NULL, 75976, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb7f78000
mmap2(0xb7f87000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0xf) = 0xb7f87000 mmap2(0xb7f89000, 6344, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xb7f89000
close(3) = 0access("/etc/ld.so.nohwcap", F_OK) = -1 ENOENT (No such file or directory)open("/lib/tls/i686/cmov/libc.so.6", O_RDONLY) = 3
read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\0`\1\000"..., 512) = 512fstat64(3, {st_mode=S_IFREG|0644, st_size=1307104, ...}) = 0mmap2(NULL, 1312164, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb7e37000mmap2(0xb7f72000, 12288, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x13b) = 0xb7f72000mmap2(0xb7f75000, 9636, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xb7f75000
close(3) = 0mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb7e36000
set_thread_area({entry_number:-1 -> 6, base_addr:0xb7e366c0, limit:1048575, seg_32bit:1, contents:0, read_exec_only:0, limit_in_pages:1, seg_not_present:0, useable:1}) = 0mprotect(0xb7f72000, 4096, PROT_READ) = 0munmap(0xb7f8c000, 57464) = 0socket(PF_INET, SOCK_RAW, IPPROTO_ICMP) = 3getuid32() = 0
setuid32(0) = 0socket(PF_INET, SOCK_DGRAM, IPPROTO_IP) = 4connect(4, {sa_family=AF_INET, sin_port=htons(1025), sin_addr=inet_addr("127.0.0.1")}, 16) = 0getsockname(4, {sa_family=AF_INET, sin_port=htons(44103), sin_addr=inet_addr("127.0.0.1")}, [16]) = 0
close(4) = 0
setsockopt(3, SOL_RAW, ICMP_FILTER, ~(ICMP_ECHOREPLY|ICMP_DEST_UNREACH|ICMP_SOURCE_QUENCH|ICMP_REDIRECT|ICMP_TIME_EXCEEDED|ICMP_PARAMETERPROB), 4) = 0setsockopt(3, SOL_IP, IP_RECVERR, [1], 4) = 0setsockopt(3, SOL_SOCKET, SO_SNDBUF, [324], 4) = 0setsockopt(3, SOL_SOCKET, SO_RCVBUF, [65536], 4) = 0getsockopt(3, SOL_SOCKET, SO_RCVBUF, [131072], [4]) = 0
brk(0) = 0x93ac000
brk(0x93cd000) = 0x93cd000fstat64(1, {st_mode=S_IFCHR|0620, st_rdev=makedev(136, 0), ...}) = 0mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb7f9a000
write(1, "PING 127.0.0.1 (127.0.0.1) 56(84"..., 49) = 49setsockopt(3, SOL_SOCKET, SO_TIMESTAMP, [1], 4) = 0setsockopt(3, SOL_SOCKET, SO_SNDTIMEO, "\1\0\0\0\0\0\0\0", 8) = 0setsockopt(3, SOL_SOCKET, SO_RCVTIMEO, "\1\0\0\0\0\0\0\0", 8) = 0getpid() = 9353rt_sigaction(SIGINT, {0x804b5b0, [], SA_INTERRUPT}, NULL, 8) = 0rt_sigaction(SIGALRM, {0x804b5b0, [], SA_INTERRUPT}, NULL, 8) = 0rt_sigaction(SIGQUIT, {0x804b5c0, [], SA_INTERRUPT}, NULL, 8) = 0gettimeofday({1190968694, 793502}, NULL) = 0ioctl(1, SNDCTL_TMR_TIMEBASE or TCGETS, {B38400 opost isig icanon echo ...}) = 0ioctl(1, TIOCGWINSZ, {ws_row=52, ws_col=157, ws_xpixel=0, ws_ypixel=0}) = 0gettimeofday({1190968694, 793762}, NULL) = 0gettimeofday({1190968694, 793824}, NULL) = 0
sendmsg(3, {msg_name(16)={sa_family=AF_INET, sin_port=htons(0), sin_addr=inet_addr("127.0.0.1")}, msg_iov(1)=[{"\10\0$\266\211$\0\1v\275\374F\340\34\f\0\10\t\n\v\f\r\16"..., 64}], msg_controllen=0, msg_flags=0}, 0) = 64recvmsg(3, 0xbf982e34, 0) = -1 EAGAIN (Resource temporarily unavailable)gettimeofday({1190968695, 793022}, NULL) = 0gettimeofday({1190968695, 793096}, NULL) = 0
sendmsg(3, {msg_name(16)={sa_family=AF_INET, sin_port=htons(0), sin_addr=inet_addr("127.0.0.1")}, msg_iov(1)=[{"\10\0\373\267\211$\0\2w\275\374F\10\32\f\0\10\t\n\v\f\r"..., 64}], msg_controllen=0, msg_flags=0}, 0) = 64recvmsg(3, 0xbf982e34, 0) = -1 EAGAIN (Resource temporarily unavailable)gettimeofday({1190968696, 793050}, NULL) = 0gettimeofday({1190968696, 793123}, NULL) = 0
sendmsg(3, {msg_name(16)={sa_family=AF_INET, sin_port=htons(0), sin_addr=inet_addr("127.0.0.1")}, msg_iov(1)=[{"\10\0\337\266\211$\0\3x\275\374F#\32\f\0\10\t\n\v\f\r\16"..., 64}], msg_controllen=0, msg_flags=0}, 0) = 64recvmsg(3, 0xbf982e34, 0) = -1 EAGAIN (Resource temporarily unavailable)gettimeofday({1190968697, 793184}, NULL) = 0gettimeofday({1190968697, 793259}, NULL) = 0
sendmsg(3, {msg_name(16)={sa_family=AF_INET, sin_port=htons(0), sin_addr=inet_addr("127.0.0.1")}, msg_iov(1)=[{"\10\0V\265\211$\0\4y\275\374F\253\32\f\0\10\t\n\v\f\r\16"..., 64}], msg_controllen=0, msg_flags=0}, 0) = 64recvmsg(3, 0xbf982e34, 0) = -1 EAGAIN (Resource temporarily unavailable)gettimeofday({1190968698, 793171}, NULL) = 0gettimeofday({1190968698, 793244}, NULL) = 0
sendmsg(3, {msg_name(16)={sa_family=AF_INET, sin_port=htons(0), sin_addr=inet_addr("127.0.0.1")}, msg_iov(1)=[{"\10\0d\264\211$\0\5z\275\374F\234\32\f\0\10\t\n\v\f\r\16"..., 64}], msg_controllen=0, msg_flags=0}, 0) = 64recvmsg(3, 0xbf982e34, 0) = -1 EAGAIN (Resource temporarily unavailable)gettimeofday({1190968699, 793234}, NULL) = 0gettimeofday({1190968699, 793306}, NULL) = 0
sendmsg(3, {msg_name(16)={sa_family=AF_INET, sin_port=htons(0), sin_addr=inet_addr("127.0.0.1")}, msg_iov(1)=[{"\10\0%\263\211$\0\6{\275\374F\332\32\f\0\10\t\n\v\f\r\16"..., 64}], msg_controllen=0, msg_flags=0}, 0) = 64recvmsg(3, 0xbf982e34, 0) = -1 EAGAIN (Resource temporarily unavailable)gettimeofday({1190968700, 793319}, NULL) = 0gettimeofday({1190968700, 793393}, NULL) = 0
sendmsg(3, {msg_name(16)={sa_family=AF_INET, sin_port=htons(0), sin_addr=inet_addr("127.0.0.1")}, msg_iov(1)=[{"\10\0\315\261\211$\0\7|\275\374F1\33\f\0\10\t\n\v\f\r\16"..., 64}], msg_controllen=0, msg_flags=0}, 0) = 64recvmsg(3, 0xbf982e34, 0) = -1 EAGAIN (Resource temporarily unavailable)gettimeofday({1190968701, 793371}, NULL) = 0gettimeofday({1190968701, 793445}, NULL) = 0
sendmsg(3, {msg_name(16)={sa_family=AF_INET, sin_port=htons(0), sin_addr=inet_addr("127.0.0.1")}, msg_iov(1)=[{"\10\0\230\260\211$\0\10}\275\374Fe\33\f\0\10\t\n\v\f\r"..., 64}], msg_controllen=0, msg_flags=0}, 0) = 64recvmsg(3, 0xbf982e34, 0) = -1 EINTR (Interrupted system call)
--- SIGINT (Interrupt) @ 0 (0) ---sigreturn() = ? (mask now [])write(1, "\n", 1) = 1write(1, "--- 127.0.0.1 ping statistics --"..., 34) = 34write(1, "8 packets transmitted, 0 receive"..., 65) = 65write(1, "\n", 1) = 1exit_group(1) = ?
Quoted 14 lines · On Fri, 28 Sep 2007 21:48:18 +0200 Laurent Riffard <[email protected]>…
Hi,
Quoted 1 line
From time to time, I experience some complete network hangs:
Suddenly, all network connections become unresponsive. Even "ping 127.0.0.1" doesn't work. SysRq-w does not show any blocked processus.
When such hang happen, I have to reboot (shutdown does work).
This is not easily reproducible: it happens several minutes after boot (could be 45 minutes or 2 hours). I do not use heavy networking apps (like P2P). My typical usage is a Gnome desktop with browser, mailer, IM, video or audio streaming.
Quoted 9 lines
I have a single PC connected to a DSL router via ethernet (so no LAN with NFS or CIFS).
This happens with 2.6.23-rc8-mm2 and 2.6.23-rc6-mm1. I can't remember when I first see this problem. Maybe 2 months ago.
I attached the output of "strace ping 127.0.0.1". How can I collect some more data when this problem happens ?
I have a second report of this from Uwe Bugla (who has been banished from all vger lists for various naughtinesses).
Similar story - after a few hours his network router (which is using ppp in some fashion) craps out and dhcp queries all time out.
I'd be suspecting a ppp bug in net-2.6.24.
Quoted 26 lines · Andrew Morton
Quoted 14 lines · On Fri, 28 Sep 2007 21:48:18 +0200 Laurent Riffard <[email protected]>…
Hi,
Quoted 1 line
From time to time, I experience some complete network hangs:
Suddenly, all network connections become unresponsive. Even "ping 127.0.0.1" doesn't work. SysRq-w does not show any blocked processus.
When such hang happen, I have to reboot (shutdown does work).
This is not easily reproducible: it happens several minutes after boot (could be 45 minutes or 2 hours). I do not use heavy networking apps (like P2P). My typical usage is a Gnome desktop with browser, mailer, IM, video or audio streaming.
Quoted 9 lines
I have a single PC connected to a DSL router via ethernet (so no LAN with NFS or CIFS).
This happens with 2.6.23-rc8-mm2 and 2.6.23-rc6-mm1. I can't remember when I first see this problem. Maybe 2 months ago.
I attached the output of "strace ping 127.0.0.1". How can I collect some more data when this problem happens ?
Quoted 7 lines
I have a second report of this from Uwe Bugla (who has been banished from all vger lists for various naughtinesses).
Similar story - after a few hours his network router (which is using ppp in some fashion) craps out and dhcp queries all time out.
I'd be suspecting a ppp bug in net-2.6.24.
mmm... I don't use PPP at all, nor dhcp.
My DSL box act as a NAT-router. Its local adress is 192.168.0.254 and my PC is at 192.192.0.9 (fixed IP).
/--- telephone line
|
+----------+
| ADSL box |
+----------+
192.168.0.254
|
192.168.0.9
+-----------+
| PC |
+-----------+Firstly, I suspected a bug in the DSL box, so I reset it but it did not help. I have to reboot the PC (don't need to reset the DSL box).
Could a router problem prevent "ping 127.0.0.1" from working ?
[this becomes somewhat off-topic for LKML, sorry]
Here's my routing table:
linux-2.6-mm$ LANG=C netstat -rn
Kernel IP routing table
Destination Gateway Genmask Flags MSS Window irtt Iface 192.168.0.0 0.0.0.0 255.255.255.0 U 0 0 0 eth0 169.254.0.0 0.0.0.0 255.255.0.0 U 0 0 0 eth0 0.0.0.0 192.168.0.254 0.0.0.0 UG 0 0 0 eth0
It seems it lacks a route for 127.0.0.1 through lo, isn't it? In this case, if eth0 hangs, 127.0.0.1 hangs too, no?
So, it could be an ethernet driver bug. I'm using ne2k-pci.
Signature · 1 line
laurent
David Miller
maintainerFrom: Laurent Riffard <[email protected]>
Date: Sat, 29 Sep 2007 11:22:06 +0200
Quoted 1 line
Could a router problem prevent "ping 127.0.0.1" from working ?
Two things that are new and could cause these problems:
1) We dynamically allocate the loopback device now.
2) We have the network namespace stuff.
Another change in this area is that we do routing cache garbage collection from a workqueue instead of a timer but that was a pretty straightforward transformation so it is not high on my list of "suspects".