Skip to contentSkip to conversation

New: ssh connections hang with 2.6.26-rc5

Bugme-new Andrew Morton 8 participants 10 messages Jun 13, 2008 – Jun 18, 2008 6 messages not in the archive

Subsystems (3)
lore
Share & tools
lore

A message that is not in the archive (sent off-list or not yet received): [email protected]/

#1 lore raw

(switched to email. Please respond via emailed reply-to-all, not via the bugzilla web interface).

Quoted 35 lines · On Fri, 13 Jun 2008 02:39:17 -0700 (PDT) [email protected] wrote:

http://bugzilla.kernel.org/show_bug.cgi?id=10903

           Summary: ssh connections hang with 2.6.26-rc5
           Product: Networking
           Version: 2.5
     KernelVersion: 2.6.26-rc5
          Platform: All
        OS/Version: Linux
              Tree: Mainline
            Status: NEW
          Severity: normal
          Priority: P1
         Component: Other
        AssignedTo: [email protected]
        ReportedBy: [email protected]

Latest working kernel version: 2.6.25-2
Earliest failing kernel version: 2.6.26-rc5
Distribution: Debian (Lenny + Sid)
Hardware Environment: amd64 (Dell Latitude D630)
Software Environment: KDE
Problem Description:

With kernel version 2.6.26-rc5, the ssh connections to remote servers randomly hang (no error message). No amelioration despite the activation of "ServerAliveInterval" on both sides.

Steps to reproduce:

Connect to a remote ssh server and do some stuff. After some time, the connection will hang.

Please ask for details.

#2 lore raw
Quoted 34 lines · Andrew Morton

(switched to email. Please respond via emailed reply-to-all, not via the bugzilla web interface).

Quoted 28 lines · On Fri, 13 Jun 2008 02:39:17 -0700 (PDT) [email protected] wrote:

http://bugzilla.kernel.org/show_bug.cgi?id=10903

           Summary: ssh connections hang with 2.6.26-rc5
           Product: Networking
           Version: 2.5
     KernelVersion: 2.6.26-rc5
          Platform: All
        OS/Version: Linux
              Tree: Mainline
            Status: NEW
          Severity: normal
          Priority: P1
         Component: Other
        AssignedTo: [email protected]
        ReportedBy: [email protected]

Latest working kernel version: 2.6.25-2
Earliest failing kernel version: 2.6.26-rc5
Distribution: Debian (Lenny + Sid)
Hardware Environment: amd64 (Dell Latitude D630)
Software Environment: KDE
Problem Description:

With kernel version 2.6.26-rc5, the ssh connections to remote servers randomly
hang (no error message). No amelioration despite the activation of "ServerAliveInterval" on both sides.

Thanks for reporting. Could you please clarify couple of things:

Does this only happen with a particular server/servers?
Any middleboxes in between (NAT, firewall, etc.)?
Do all ssh connections hang simultaneously?
How long have you waited until concluding that TCP is "hung"? Is TSO enabled (ethtool -k)? Have you tried without it?
It wouldn't hurt to include info about eth hw too (e.g., lspci), though it might turn unneeded at some point of time but it might save an email round-trip.

TCP can appear to hang due to vast number of reasons. Only recent changes that are suspectable is the DEFERRED_ACCEPT thing which is already reverted in the very latest Linus' tree (even -rc6 is too old for that) and few FRTO fixes (you can exclude FRTO by turning /proc/sys/net/ipv4/tcp_frto sysctl to 0 but it seems quite unlikely to change anything); your problem might well come from something else and TCP hang is just a symptom of other problem downstream.

So please gather this information (at least for the relevant connections):

$ netstat -pn
$ cat /proc/net/tcp

...Also a tcpdump might be handy (though I don't know yet).

...Depending on your privacy needs, you may want obfuscate ip addresses that are revealed by all of those logs (ie., if you don't want to reveal with whom you're communicating with, ssh payload is encrypted anyway).

Quoted 6 lines
Quoted 6 lines

Steps to reproduce:

Connect to a remote ssh server and do some stuff. After some time, the connection will hang.

Please ask for details.

(I'll be away nearly a month after Tuesday, so I probably won't have much time to resolve this issue but I hope I've some time to take a look before I leave).

Signature · 1 line
 i.
#3 lore raw

Le samedi 14 juin 2008 22:45:41 Ilpo Järvinen, vous avez écrit :

Quoted 3 lines · Andrew Morton
Quoted 2 lines

(switched to email. Please respond via emailed reply-to-all, not via the bugzilla web interface).

OK.

Quoted 1 line
Quoted 1 line

On Fri, 13 Jun 2008 02:39:17 -0700 (PDT) [email protected]

Quoted 24 lines · wrote:
Quoted 24 lines
Quoted 24 lines

http://bugzilla.kernel.org/show_bug.cgi?id=10903

           Summary: ssh connections hang with 2.6.26-rc5
           Product: Networking
           Version: 2.5
     KernelVersion: 2.6.26-rc5
          Platform: All
        OS/Version: Linux
              Tree: Mainline
            Status: NEW
          Severity: normal
          Priority: P1
         Component: Other
        AssignedTo: [email protected]
        ReportedBy: [email protected]

Latest working kernel version: 2.6.25-2
Earliest failing kernel version: 2.6.26-rc5
Distribution: Debian (Lenny + Sid)
Hardware Environment: amd64 (Dell Latitude D630)
Software Environment: KDE
Problem Description:

Quoted 6 lines
Quoted 4 lines
Quoted 4 lines

With kernel version 2.6.26-rc5, the ssh connections to remote servers randomly
hang (no error message). No amelioration despite the activation of "ServerAliveInterval" on both sides.

Thanks for reporting. Could you please clarify couple of things:

Hi.

I will try to, with my time and knowledge.

Quoted 1 line

Does this only happen with a particular server/servers?

I have only tried with two of my home servers. One runs 2.6.22-4-686 and the other 2.6.18-6-vserver-686.

Quoted 1 line

Any middleboxes in between (NAT, firewall, etc.)?

There is a ADSL router which "provides" internet to the servers by NAT. I have tried from "inside" the house (so in the same subnet) and from outside: it hangs in both cases.

The common point is my use of "iwl3945" : I have always tried the ssh connections through WiFi.

Quoted 1 line

Do all ssh connections hang simultaneously?

Well... It is hard to say. As far as I have seen, no. When I get one hang, I can successfully connect to the same server.

Quoted 1 line

How long have you waited until concluding that TCP is "hung"?

Well. The "ServerAliveInterval" option of openssh now leads to "Received disconnect from $IP: 2: Timeout, your session not responding." after the hang. So the openssh server notices that my session is not responding and so cuts the connection.

Quoted 1 line

Is TSO enabled (ethtool -k)? Have you tried without it?

Doesn't seem:

----

# ethtool -k wlan0
Offload parameters for wlan0:
Cannot get device rx csum settings: Operation not supported
Cannot get device tx csum settings: Operation not supported
Cannot get device scatter-gather settings: Operation not supported
Cannot get device tcp segmentation offload settings: Operation not supported
Cannot get device udp large send offload settings: Operation not supported
Cannot get device generic segmentation offload settings: Operation not 
supported
no offload info available
----
Quoted 3 lines

It wouldn't hurt to include info about eth hw too (e.g., lspci), though it might turn unneeded at some point of time but it might save an email round-trip.

lspci attached.

Quoted 7 lines

TCP can appear to hang due to vast number of reasons. Only recent changes that are suspectable is the DEFERRED_ACCEPT thing which is already reverted in the very latest Linus' tree (even -rc6 is too old for that) and few FRTO fixes (you can exclude FRTO by turning /proc/sys/net/ipv4/tcp_frto sysctl to 0 but it seems quite unlikely to change anything); your problem might well come from something else and TCP hang is just a symptom of other problem downstream.

I can't understand everything, but what I can say is that with the exact same software, I get no hangs with 2.6.25-2 but I get some with 2.6.26-rc5.

Quoted 4 lines

So please gather this information (at least for the relevant connections):

$ netstat -pn
$ cat /proc/net/tcp

Attached.

Quoted 1 line

...Also a tcpdump might be handy (though I don't know yet).

Well. It seems that there is another bug here: everytime I tried a

# tcpdump -w /tmp/tcpdump.wlan0 -i wlan0

I got a CPU lockup (or similar, can't know exactly, but keyboard blocked and nothing doable).

Quoted 9 lines

...Depending on your privacy needs, you may want obfuscate ip addresses that are revealed by all of those logs (ie., if you don't want to reveal with whom you're communicating with, ssh payload is encrypted anyway).

(...)

(I'll be away nearly a month after Tuesday, so I probably won't have much time to resolve this issue but I hope I've some time to take a look before I leave).

We'll see ;)

Regards,

OdyX

Signature · 3 lines
Didier Raboud, proud Debian user.
CH-1802 Corseaux
[email protected]

00:00.0 Host bridge: Intel Corporation Mobile PM965/GM965/GL960 Memory Controller Hub (rev 0c) 00:02.0 VGA compatible controller: Intel Corporation Mobile GM965/GL960 Integrated Graphics Controller (rev 0c) 00:02.1 Display controller: Intel Corporation Mobile GM965/GL960 Integrated Graphics Controller (rev 0c) 00:1a.0 USB Controller: Intel Corporation 82801H (ICH8 Family) USB UHCI Controller #4 (rev 02) 00:1a.1 USB Controller: Intel Corporation 82801H (ICH8 Family) USB UHCI Controller #5 (rev 02) 00:1a.7 USB Controller: Intel Corporation 82801H (ICH8 Family) USB2 EHCI Controller #2 (rev 02) 00:1b.0 Audio device: Intel Corporation 82801H (ICH8 Family) HD Audio Controller (rev 02) 00:1c.0 PCI bridge: Intel Corporation 82801H (ICH8 Family) PCI Express Port 1 (rev 02) 00:1c.1 PCI bridge: Intel Corporation 82801H (ICH8 Family) PCI Express Port 2 (rev 02) 00:1c.5 PCI bridge: Intel Corporation 82801H (ICH8 Family) PCI Express Port 6 (rev 02) 00:1d.0 USB Controller: Intel Corporation 82801H (ICH8 Family) USB UHCI Controller #1 (rev 02) 00:1d.1 USB Controller: Intel Corporation 82801H (ICH8 Family) USB UHCI Controller #2 (rev 02) 00:1d.2 USB Controller: Intel Corporation 82801H (ICH8 Family) USB UHCI Controller #3 (rev 02) 00:1d.7 USB Controller: Intel Corporation 82801H (ICH8 Family) USB2 EHCI Controller #1 (rev 02) 00:1e.0 PCI bridge: Intel Corporation 82801 Mobile PCI Bridge (rev f2)
00:1f.0 ISA bridge: Intel Corporation 82801HEM (ICH8M) LPC Interface Controller (rev 02) 00:1f.1 IDE interface: Intel Corporation 82801HBM/HEM (ICH8M/ICH8M-E) IDE Controller (rev 02) 00:1f.2 SATA controller: Intel Corporation 82801HBM/HEM (ICH8M/ICH8M-E) SATA AHCI Controller (rev 02) 00:1f.3 SMBus: Intel Corporation 82801H (ICH8 Family) SMBus Controller (rev 02)
03:01.0 CardBus bridge: O2 Micro, Inc. Cardbus bridge (rev 21)
03:01.4 FireWire (IEEE 1394): O2 Micro, Inc. Firewire (IEEE 1394) (rev 02)
09:00.0 Ethernet controller: Broadcom Corporation NetXtreme BCM5755M Gigabit Ethernet PCI Express (rev 02) 0c:00.0 Network controller: Intel Corporation PRO/Wireless 3945ABG Network Connection (rev 02)

Active Internet connections (w/o servers)

Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name
tcp        0      0 192.168.1.11:57063      128.178.222.18:143      ESTABLISHED 11314/klauncher0EyU
tcp        0      0 192.168.1.11:54596      192.168.1.3:9000        ESTABLISHED 11125/klauncher0EyU
tcp        0      0 192.168.1.11:54591      192.168.1.3:9000        TIME_WAIT   -               
tcp        0      0 192.168.1.11:54587      192.168.1.3:9000        TIME_WAIT   -               
tcp        0      0 192.168.1.11:54589      192.168.1.3:9000        TIME_WAIT   -               
tcp        0      0 192.168.1.11:54583      192.168.1.3:9000        TIME_WAIT   -               
tcp        0      0 192.168.1.11:54584      192.168.1.3:9000        TIME_WAIT   -               
tcp        0      0 192.168.1.11:47334      217.150.254.106:5223    ESTABLISHED 4437/kopete     
tcp        0      0 192.168.1.11:54593      192.168.1.3:9000        TIME_WAIT   -               
tcp        0      0 192.168.1.11:36357      192.168.1.2:22          TIME_WAIT   -               
tcp        0      0 192.168.1.11:54592      192.168.1.3:9000        TIME_WAIT   -               
tcp        0      0 192.168.1.11:54588      192.168.1.3:9000        TIME_WAIT   -               
tcp        0      0 192.168.1.11:53858      195.144.11.40:143       ESTABLISHED 4799/klauncher0EyU
tcp        0      0 192.168.1.11:54586      192.168.1.3:9000        TIME_WAIT   -               
tcp        0      0 192.168.1.11:54595      192.168.1.3:9000        ESTABLISHED 11129/klauncher0EyU
tcp        0      0 192.168.1.11:38449      212.25.25.37:143        ESTABLISHED 4801/klauncher0EyU
tcp        0      0 192.168.1.11:40539      192.168.1.3:9000        TIME_WAIT   -               
tcp        0      0 192.168.1.11:54590      192.168.1.3:9000        TIME_WAIT   -               
tcp        0      0 192.168.1.11:54594      192.168.1.3:9000        TIME_WAIT   -               
tcp        0      0 192.168.1.11:33066      84.16.68.124:143        ESTABLISHED 4800/klauncher0EyU
tcp        0      0 192.168.1.11:38525      192.168.1.3:22          ESTABLISHED 13828/ssh       
tcp        0      0 192.168.182.18:60514    213.134.45.154:443      ESTABLISHED 4666/klauncher0EyU
tcp        0      0 192.168.1.11:54585      192.168.1.3:9000        TIME_WAIT   -               
Active UNIX domain sockets (w/o servers)
Proto RefCnt Flags       Type       State         I-Node   PID/Program name    Path
unix  3      [ ]         DGRAM                    20699    -                   /var/run/wpa_supplicant/wlan0
unix  2      [ ]         DGRAM                    20625    -                   /var/run/wpa_supplicant-global
unix  2      [ ]         DGRAM                    20701    -                   /var/run/NetworkManager/wpa_ctrl_3855-19
unix  2      [ ]         DGRAM                    3393     -                   @/org/kernel/udev/udevd
unix  9      [ ]         DGRAM                    7553     -                   /dev/log
unix  2      [ ]         DGRAM                    8303     -                   @/org/freedesktop/hal/udev_event
unix  3      [ ]         STREAM     CONNECTED     41908    4366/klauncher [kde /tmp/ksocket-didier/klauncher0EyUjc.slave-socket
unix  3      [ ]         STREAM     CONNECTED     41906    4808/klauncher0EyUj 
unix  3      [ ]         STREAM     CONNECTED     41564    4363/dcopserver [kd /tmp/.ICE-unix/dcop4363-1213534701
unix  3      [ ]         STREAM     CONNECTED     41563    13658/klauncher0EyU 
unix  3      [ ]         STREAM     CONNECTED     41562    4467/kontact        /tmp/ksocket-didier/kontact28e1Ub.slave-socket
unix  3      [ ]         STREAM     CONNECTED     41560    13658/klauncher0EyU 
unix  3      [ ]         STREAM     CONNECTED     41552    4467/kontact        /tmp/ksocket-didier/kontactxiTI2a.slave-socket
unix  3      [ ]         STREAM     CONNECTED     41550    13657/klauncher0EyU 
unix  3      [ ]         STREAM     CONNECTED     38904    4363/dcopserver [kd /tmp/.ICE-unix/dcop4363-1213534701
unix  3      [ ]         STREAM     CONNECTED     38903    13293/klauncher0EyU 
unix  3      [ ]         STREAM     CONNECTED     38902    4462/konqueror [kde /tmp/ksocket-didier/konqueroroMrdRa.slave-socket
unix  3      [ ]         STREAM     CONNECTED     38900    13293/klauncher0EyU 
unix  3      [ ]         STREAM     CONNECTED     28499    -                   /tmp/.X11-unix/X0
unix  3      [ ]         STREAM     CONNECTED     28498    4366/klauncher [kde 
unix  3      [ ]         STREAM     CONNECTED     26499    4363/dcopserver [kd /tmp/.ICE-unix/dcop4363-1213534701
unix  3      [ ]         STREAM     CONNECTED     26498    11314/klauncher0EyU 
unix  3      [ ]         STREAM     CONNECTED     26486    4467/kontact        /tmp/ksocket-didier/kontactoijRSa.slave-socket
unix  3      [ ]         STREAM     CONNECTED     26485    11314/klauncher0EyU 
unix  3      [ ]         STREAM     CONNECTED     25452    4363/dcopserver [kd /tmp/.ICE-unix/dcop4363-1213534701
unix  3      [ ]         STREAM     CONNECTED     25451    11129/klauncher0EyU 
unix  3      [ ]         STREAM     CONNECTED     25450    4462/konqueror [kde /tmp/ksocket-didier/konqueror3DG8Oa.slave-socket
unix  3      [ ]         STREAM     CONNECTED     25448    11129/klauncher0EyU 
unix  3      [ ]         STREAM     CONNECTED     25432    4363/dcopserver [kd /tmp/.ICE-unix/dcop4363-1213534701
unix  3      [ ]         STREAM     CONNECTED     25431    11125/klauncher0EyU 
unix  3      [ ]         STREAM     CONNECTED     25429    4462/konqueror [kde /tmp/ksocket-didier/konquerorCg3z9b.slave-socket
unix  3      [ ]         STREAM     CONNECTED     25425    11125/klauncher0EyU 
unix  3      [ ]         STREAM     CONNECTED     22199    4363/dcopserver [kd /tmp/.ICE-unix/dcop4363-1213534701
unix  3      [ ]         STREAM     CONNECTED     22198    4801/klauncher0EyU  
unix  3      [ ]         STREAM     CONNECTED     22174    4363/dcopserver [kd /tmp/.ICE-unix/dcop4363-1213534701
unix  3      [ ]         STREAM     CONNECTED     22173    4799/klauncher0EyU  
unix  3      [ ]         STREAM     CONNECTED     22156    4363/dcopserver [kd /tmp/.ICE-unix/dcop4363-1213534701
unix  3      [ ]         STREAM     CONNECTED     22155    4800/klauncher0EyU  
unix  3      [ ]         STREAM     CONNECTED     21407    4380/ksmserver [kde /tmp/.ICE-unix/4380
unix  3      [ ]         STREAM     CONNECTED     21406    10664/kmix [kdeinit 
unix  3      [ ]         STREAM     CONNECTED     21405    -                   /tmp/.X11-unix/X0
unix  3      [ ]         STREAM     CONNECTED     21402    10664/kmix [kdeinit 
unix  3      [ ]         STREAM     CONNECTED     21399    4363/dcopserver [kd /tmp/.ICE-unix/dcop4363-1213534701
unix  3      [ ]         STREAM     CONNECTED     21398    10664/kmix [kdeinit 
unix  2      [ ]         DGRAM                    20736    -                   
unix  3      [ ]         STREAM     CONNECTED     19277    4380/ksmserver [kde /tmp/.ICE-unix/4380
unix  3      [ ]         STREAM     CONNECTED     19276    10280/kio_uiserver  
unix  3      [ ]         STREAM     CONNECTED     19275    -                   /tmp/.X11-unix/X0
unix  3      [ ]         STREAM     CONNECTED     19272    10280/kio_uiserver  
unix  3      [ ]         STREAM     CONNECTED     19265    4363/dcopserver [kd /tmp/.ICE-unix/dcop4363-1213534701
unix  3      [ ]         STREAM     CONNECTED     19264    10280/kio_uiserver  
unix  3      [ ]         STREAM     CONNECTED     12984    4363/dcopserver [kd /tmp/.ICE-unix/dcop4363-1213534701
unix  3      [ ]         STREAM     CONNECTED     12983    4666/klauncher0EyU  
unix  3      [ ]         STREAM     CONNECTED     12761    4467/kontact        /tmp/ksocket-didier/kontactsTY3Bb.slave-socket
unix  3      [ ]         STREAM     CONNECTED     12687    4801/klauncher0EyU  
unix  3      [ ]         STREAM     CONNECTED     12759    4467/kontact        /tmp/ksocket-didier/kontactYyQzta.slave-socket
unix  3      [ ]         STREAM     CONNECTED     12681    4800/klauncher0EyU  
unix  3      [ ]         STREAM     CONNECTED     12764    4467/kontact        /tmp/ksocket-didier/kontactUfNfjb.slave-socket
unix  3      [ ]         STREAM     CONNECTED     12678    4799/klauncher0EyU  
unix  3      [ ]         STREAM     CONNECTED     11713    4368/kded [kdeinit] /tmp/ksocket-didier/kdedN1sj7b.slave-socket
unix  3      [ ]         STREAM     CONNECTED     11712    4666/klauncher0EyU  
unix  3      [ ]         STREAM     CONNECTED     11480    4380/ksmserver [kde /tmp/.ICE-unix/4380
unix  3      [ ]         STREAM     CONNECTED     11479    4626/kttsd          
unix  3      [ ]         STREAM     CONNECTED     11476    -                   /tmp/.X11-unix/X0
unix  3      [ ]         STREAM     CONNECTED     11475    4626/kttsd          
unix  3      [ ]         STREAM     CONNECTED     11472    4363/dcopserver [kd /tmp/.ICE-unix/dcop4363-1213534701
unix  3      [ ]         STREAM     CONNECTED     11471    4626/kttsd          
unix  3      [ ]         STREAM     CONNECTED     11448    4380/ksmserver [kde /tmp/.ICE-unix/4380
unix  3      [ ]         STREAM     CONNECTED     11447    4621/kwalletmanager 
unix  3      [ ]         STREAM     CONNECTED     11440    -                   /tmp/.X11-unix/X0
unix  3      [ ]         STREAM     CONNECTED     11439    4621/kwalletmanager 
unix  3      [ ]         STREAM     CONNECTED     11436    4363/dcopserver [kd /tmp/.ICE-unix/dcop4363-1213534701
unix  3      [ ]         STREAM     CONNECTED     11435    4621/kwalletmanager 
unix  3      [ ]         STREAM     CONNECTED     11399    4612/ispell         
unix  3      [ ]         STREAM     CONNECTED     11398    4467/kontact        
unix  3      [ ]         STREAM     CONNECTED     11397    4612/ispell         
unix  3      [ ]         STREAM     CONNECTED     11396    4467/kontact        
unix  3      [ ]         STREAM     CONNECTED     11395    4467/kontact        
unix  3      [ ]         STREAM     CONNECTED     11394    4612/ispell         
unix  3      [ ]         STREAM     CONNECTED     11392    4370/gam_server     @/tmp/fam-didier-
unix  3      [ ]         STREAM     CONNECTED     11391    4467/kontact        
unix  3      [ ]         STREAM     CONNECTED     10985    4427/artsd          /tmp/ksocket-didier/Tamino-114b-48551204
unix  3      [ ]         STREAM     CONNECTED     10984    4504/knotify [kdein 
unix  3      [ ]         STREAM     CONNECTED     10899    -                   /var/run/sdp
unix  3      [ ]         STREAM     CONNECTED     10898    4514/kbluetooth     
unix  3      [ ]         STREAM     CONNECTED     10890    -                   /var/run/dbus/system_bus_socket
unix  3      [ ]         STREAM     CONNECTED     10889    4514/kbluetooth     
unix  3      [ ]         STREAM     CONNECTED     10888    4363/dcopserver [kd /tmp/.ICE-unix/dcop4363-1213534701
unix  3      [ ]         STREAM     CONNECTED     10887    4467/kontact        
unix  3      [ ]         STREAM     CONNECTED     10884    4363/dcopserver [kd /tmp/.ICE-unix/dcop4363-1213534701
unix  3      [ ]         STREAM     CONNECTED     10883    4462/konqueror [kde 
unix  3      [ ]         STREAM     CONNECTED     10882    4363/dcopserver [kd /tmp/.ICE-unix/dcop4363-1213534701
unix  3      [ ]         STREAM     CONNECTED     10881    4467/kontact        
unix  3      [ ]         STREAM     CONNECTED     10878    4363/dcopserver [kd /tmp/.ICE-unix/dcop4363-1213534701
unix  3      [ ]         STREAM     CONNECTED     10877    4467/kontact        
unix  3      [ ]         STREAM     CONNECTED     10872    4363/dcopserver [kd /tmp/.ICE-unix/dcop4363-1213534701
unix  3      [ ]         STREAM     CONNECTED     10871    4467/kontact        
unix  3      [ ]         STREAM     CONNECTED     10859    4363/dcopserver [kd /tmp/.ICE-unix/dcop4363-1213534701
unix  3      [ ]         STREAM     CONNECTED     10858    4467/kontact        
unix  3      [ ]         STREAM     CONNECTED     10840    4380/ksmserver [kde /tmp/.ICE-unix/4380
unix  3      [ ]         STREAM     CONNECTED     10839    4514/kbluetooth     
unix  3      [ ]         STREAM     CONNECTED     10838    -                   /tmp/.X11-unix/X0
unix  3      [ ]         STREAM     CONNECTED     10837    4514/kbluetooth     
unix  3      [ ]         STREAM     CONNECTED     10834    4363/dcopserver [kd /tmp/.ICE-unix/dcop4363-1213534701
unix  3      [ ]         STREAM     CONNECTED     10833    4514/kbluetooth     
unix  3      [ ]         STREAM     CONNECTED     10809    -                   /var/run/dbus/system_bus_socket
unix  3      [ ]         STREAM     CONNECTED     10808    4475/python         
unix  3      [ ]         STREAM     CONNECTED     10805    4380/ksmserver [kde /tmp/.ICE-unix/4380
unix  3      [ ]         STREAM     CONNECTED     10804    4475/python         
unix  3      [ ]         STREAM     CONNECTED     10768    -                   /tmp/.X11-unix/X0
unix  3      [ ]         STREAM     CONNECTED     10767    4475/python         
unix  3      [ ]         STREAM     CONNECTED     10741    4380/ksmserver [kde /tmp/.ICE-unix/4380
unix  3      [ ]         STREAM     CONNECTED     10740    4504/knotify [kdein 
unix  3      [ ]         STREAM     CONNECTED     10723    -                   /tmp/.X11-unix/X0
unix  3      [ ]         STREAM     CONNECTED     10722    4504/knotify [kdein 
unix  3      [ ]         STREAM     CONNECTED     10718    4363/dcopserver [kd /tmp/.ICE-unix/dcop4363-1213534701
unix  3      [ ]         STREAM     CONNECTED     10717    4504/knotify [kdein 
unix  3      [ ]         STREAM     CONNECTED     10656    4363/dcopserver [kd /tmp/.ICE-unix/dcop4363-1213534701
unix  3      [ ]         STREAM     CONNECTED     10655    4467/kontact        
unix  3      [ ]         STREAM     CONNECTED     10652    -                   /var/run/dbus/system_bus_socket
unix  3      [ ]         STREAM     CONNECTED     10651    4486/kpowersave [kd 
unix  3      [ ]         STREAM     CONNECTED     10646    4380/ksmserver [kde /tmp/.ICE-unix/4380
unix  3      [ ]         STREAM     CONNECTED     10645    4488/klipper [kdein 
unix  3      [ ]         STREAM     CONNECTED     10642    -                   /tmp/.X11-unix/X0
unix  3      [ ]         STREAM     CONNECTED     10641    4488/klipper [kdein 
unix  3      [ ]         STREAM     CONNECTED     10640    4363/dcopserver [kd /tmp/.ICE-unix/dcop4363-1213534701
unix  3      [ ]         STREAM     CONNECTED     10639    4488/klipper [kdein 
unix  3      [ ]         STREAM     CONNECTED     10628    4380/ksmserver [kde /tmp/.ICE-unix/4380
unix  3      [ ]         STREAM     CONNECTED     10627    4486/kpowersave [kd 
unix  3      [ ]         STREAM     CONNECTED     10624    -                   /tmp/.X11-unix/X0
unix  3      [ ]         STREAM     CONNECTED     10623    4486/kpowersave [kd 
unix  3      [ ]         STREAM     CONNECTED     10622    4363/dcopserver [kd /tmp/.ICE-unix/dcop4363-1213534701
unix  3      [ ]         STREAM     CONNECTED     10621    4486/kpowersave [kd 
unix  3      [ ]         STREAM     CONNECTED     10612    4370/gam_server     @/tmp/fam-didier-
unix  3      [ ]         STREAM     CONNECTED     10611    4482/korgac         
unix  3      [ ]         STREAM     CONNECTED     10608    -                   /tmp/.X11-unix/X0
unix  3      [ ]         STREAM     CONNECTED     10607    4368/kded [kdeinit] 
unix  3      [ ]         STREAM     CONNECTED     10603    -                   /var/run/dbus/system_bus_socket
unix  3      [ ]         STREAM     CONNECTED     10602    4483/knetworkmanage 
unix  3      [ ]         STREAM     CONNECTED     10591    4380/ksmserver [kde /tmp/.ICE-unix/4380
unix  3      [ ]         STREAM     CONNECTED     10590    4483/knetworkmanage 
unix  3      [ ]         STREAM     CONNECTED     10587    -                   /tmp/.X11-unix/X0
unix  3      [ ]         STREAM     CONNECTED     10586    4483/knetworkmanage 
unix  3      [ ]         STREAM     CONNECTED     10585    4363/dcopserver [kd /tmp/.ICE-unix/dcop4363-1213534701
unix  3      [ ]         STREAM     CONNECTED     10584    4483/knetworkmanage 
unix  3      [ ]         STREAM     CONNECTED     10573    4380/ksmserver [kde /tmp/.ICE-unix/4380
unix  3      [ ]         STREAM     CONNECTED     10572    4482/korgac         
unix  3      [ ]         STREAM     CONNECTED     10569    -                   /tmp/.X11-unix/X0
unix  3      [ ]         STREAM     CONNECTED     10568    4482/korgac         
unix  3      [ ]         STREAM     CONNECTED     10567    4363/dcopserver [kd /tmp/.ICE-unix/dcop4363-1213534701
unix  3      [ ]         STREAM     CONNECTED     10566    4482/korgac         
unix  3      [ ]         STREAM     CONNECTED     10562    4481/ispell         
unix  3      [ ]         STREAM     CONNECTED     10561    4433/katapult       
unix  3      [ ]         STREAM     CONNECTED     10560    4481/ispell         
unix  3      [ ]         STREAM     CONNECTED     10559    4433/katapult       
unix  3      [ ]         STREAM     CONNECTED     10558    4433/katapult       
unix  3      [ ]         STREAM     CONNECTED     10557    4481/ispell         
unix  3      [ ]         STREAM     CONNECTED     10486    4380/ksmserver [kde /tmp/.ICE-unix/4380
unix  3      [ ]         STREAM     CONNECTED     10485    4467/kontact        
unix  3      [ ]         STREAM     CONNECTED     10482    -                   /tmp/.X11-unix/X0
unix  3      [ ]         STREAM     CONNECTED     10481    4467/kontact        
unix  3      [ ]         STREAM     CONNECTED     10480    4363/dcopserver [kd /tmp/.ICE-unix/dcop4363-1213534701
unix  3      [ ]         STREAM     CONNECTED     10479    4467/kontact        
unix  3      [ ]         STREAM     CONNECTED     10471    4363/dcopserver [kd /tmp/.ICE-unix/dcop4363-1213534701
unix  3      [ ]         STREAM     CONNECTED     10470    4462/konqueror [kde 
unix  3      [ ]         STREAM     CONNECTED     10463    4380/ksmserver [kde /tmp/.ICE-unix/4380
unix  3      [ ]         STREAM     CONNECTED     10462    4462/konqueror [kde 
unix  3      [ ]         STREAM     CONNECTED     10461    -                   /tmp/.X11-unix/X0
unix  3      [ ]         STREAM     CONNECTED     10460    4462/konqueror [kde 
unix  3      [ ]         STREAM     CONNECTED     10433    4370/gam_server     @/tmp/fam-didier-
unix  3      [ ]         STREAM     CONNECTED     10432    4437/kopete         
unix  3      [ ]         STREAM     CONNECTED     10352    4380/ksmserver [kde /tmp/.ICE-unix/4380
unix  3      [ ]         STREAM     CONNECTED     10351    4437/kopete         
unix  3      [ ]         STREAM     CONNECTED     10349    -                   /tmp/.X11-unix/X0
unix  3      [ ]         STREAM     CONNECTED     10348    4437/kopete         
unix  3      [ ]         STREAM     CONNECTED     10346    4363/dcopserver [kd /tmp/.ICE-unix/dcop4363-1213534701
unix  3      [ ]         STREAM     CONNECTED     10345    4437/kopete         
unix  3      [ ]         STREAM     CONNECTED     10329    4363/dcopserver [kd /tmp/.ICE-unix/dcop4363-1213534701
unix  3      [ ]         STREAM     CONNECTED     10328    4433/katapult       
unix  3      [ ]         STREAM     CONNECTED     10321    4380/ksmserver [kde /tmp/.ICE-unix/4380
unix  3      [ ]         STREAM     CONNECTED     10320    4433/katapult       
unix  3      [ ]         STREAM     CONNECTED     10319    -                   /tmp/.X11-unix/X0
unix  3      [ ]         STREAM     CONNECTED     10318    4433/katapult       
unix  3      [ ]         STREAM     CONNECTED     10310    4380/ksmserver [kde /tmp/.ICE-unix/4380
unix  3      [ ]         STREAM     CONNECTED     10309    4432/yakuake        
unix  3      [ ]         STREAM     CONNECTED     10306    -                   /tmp/.X11-unix/X0
unix  3      [ ]         STREAM     CONNECTED     10305    4432/yakuake        
unix  3      [ ]         STREAM     CONNECTED     10304    4363/dcopserver [kd /tmp/.ICE-unix/dcop4363-1213534701
unix  3      [ ]         STREAM     CONNECTED     10303    4432/yakuake        
unix  3      [ ]         STREAM     CONNECTED     10292    4380/ksmserver [kde /tmp/.ICE-unix/4380
unix  3      [ ]         STREAM     CONNECTED     10291    4430/kgpg           
unix  3      [ ]         STREAM     CONNECTED     10288    -                   /tmp/.X11-unix/X0
unix  3      [ ]         STREAM     CONNECTED     10287    4430/kgpg           
unix  3      [ ]         STREAM     CONNECTED     10286    4363/dcopserver [kd /tmp/.ICE-unix/dcop4363-1213534701
unix  3      [ ]         STREAM     CONNECTED     10285    4430/kgpg           
unix  3      [ ]         STREAM     CONNECTED     10216    4380/ksmserver [kde /tmp/.ICE-unix/4380
unix  3      [ ]         STREAM     CONNECTED     10215    4412/kaccess [kdein 
unix  3      [ ]         STREAM     CONNECTED     10212    -                   /tmp/.X11-unix/X0
unix  3      [ ]         STREAM     CONNECTED     10211    4412/kaccess [kdein 
unix  3      [ ]         STREAM     CONNECTED     10210    4363/dcopserver [kd /tmp/.ICE-unix/dcop4363-1213534701
unix  3      [ ]         STREAM     CONNECTED     10209    4412/kaccess [kdein 
unix  3      [ ]         STREAM     CONNECTED     10184    4363/dcopserver [kd /tmp/.ICE-unix/dcop4363-1213534701
unix  3      [ ]         STREAM     CONNECTED     10183    4368/kded [kdeinit] 
unix  3      [ ]         STREAM     CONNECTED     10130    4363/dcopserver [kd /tmp/.ICE-unix/dcop4363-1213534701
unix  3      [ ]         STREAM     CONNECTED     10129    4368/kded [kdeinit] 
unix  3      [ ]         STREAM     CONNECTED     10095    4380/ksmserver [kde /tmp/.ICE-unix/4380
unix  3      [ ]         STREAM     CONNECTED     10094    4395/kweatherservic 
unix  3      [ ]         STREAM     CONNECTED     10091    -                   /tmp/.X11-unix/X0
unix  3      [ ]         STREAM     CONNECTED     10090    4395/kweatherservic 
unix  3      [ ]         STREAM     CONNECTED     10087    4363/dcopserver [kd /tmp/.ICE-unix/dcop4363-1213534701
unix  3      [ ]         STREAM     CONNECTED     10086    4395/kweatherservic 
unix  3      [ ]         STREAM     CONNECTED     9883     4392/sh             
unix  3      [ ]         STREAM     CONNECTED     9882     4385/kicker [kdeini 
unix  3      [ ]         STREAM     CONNECTED     9881     4392/sh             
unix  3      [ ]         STREAM     CONNECTED     9880     4385/kicker [kdeini 
unix  3      [ ]         STREAM     CONNECTED     9879     4385/kicker [kdeini 
unix  3      [ ]         STREAM     CONNECTED     9878     4392/sh             
unix  3      [ ]         STREAM     CONNECTED     9855     4370/gam_server     @/tmp/fam-didier-
unix  3      [ ]         STREAM     CONNECTED     9854     4385/kicker [kdeini 
unix  3      [ ]         STREAM     CONNECTED     9852     -                   /var/run/dbus/system_bus_socket
unix  3      [ ]         STREAM     CONNECTED     9851     4368/kded [kdeinit] 
unix  3      [ ]         STREAM     CONNECTED     9799     4380/ksmserver [kde /tmp/.ICE-unix/4380
unix  3      [ ]         STREAM     CONNECTED     9798     4385/kicker [kdeini 
unix  3      [ ]         STREAM     CONNECTED     9796     -                   /tmp/.X11-unix/X0
unix  3      [ ]         STREAM     CONNECTED     9794     4385/kicker [kdeini 
unix  3      [ ]         STREAM     CONNECTED     9791     4363/dcopserver [kd /tmp/.ICE-unix/dcop4363-1213534701
unix  3      [ ]         STREAM     CONNECTED     9790     4385/kicker [kdeini 
unix  3      [ ]         STREAM     CONNECTED     9785     4370/gam_server     @/tmp/fam-didier-
unix  3      [ ]         STREAM     CONNECTED     9784     4383/kdesktop [kdei 
unix  3      [ ]         STREAM     CONNECTED     9773     4380/ksmserver [kde /tmp/.ICE-unix/4380
unix  3      [ ]         STREAM     CONNECTED     9772     4383/kdesktop [kdei 
unix  3      [ ]         STREAM     CONNECTED     9771     -                   /tmp/.X11-unix/X0
unix  3      [ ]         STREAM     CONNECTED     9770     4383/kdesktop [kdei 
unix  3      [ ]         STREAM     CONNECTED     9765     4363/dcopserver [kd /tmp/.ICE-unix/dcop4363-1213534701
unix  3      [ ]         STREAM     CONNECTED     9764     4383/kdesktop [kdei 
unix  3      [ ]         STREAM     CONNECTED     9757     4380/ksmserver [kde /tmp/.ICE-unix/4380
unix  3      [ ]         STREAM     CONNECTED     9756     4381/kwin [kdeinit] 
unix  3      [ ]         STREAM     CONNECTED     9754     4363/dcopserver [kd /tmp/.ICE-unix/dcop4363-1213534701
unix  3      [ ]         STREAM     CONNECTED     9753     4381/kwin [kdeinit] 
unix  3      [ ]         STREAM     CONNECTED     9748     4380/ksmserver [kde /tmp/.ICE-unix/4380
unix  3      [ ]         STREAM     CONNECTED     9747     4381/kwin [kdeinit] 
unix  3      [ ]         STREAM     CONNECTED     9746     -                   /tmp/.X11-unix/X0
unix  3      [ ]         STREAM     CONNECTED     9745     4381/kwin [kdeinit] 
unix  3      [ ]         STREAM     CONNECTED     9738     4363/dcopserver [kd /tmp/.ICE-unix/dcop4363-1213534701
unix  3      [ ]         STREAM     CONNECTED     9737     4380/ksmserver [kde 
unix  3      [ ]         STREAM     CONNECTED     9732     -                   /tmp/.X11-unix/X0
unix  3      [ ]         STREAM     CONNECTED     9731     4380/ksmserver [kde 
unix  3      [ ]         STREAM     CONNECTED     9722     4380/ksmserver [kde /tmp/ksocket-didier/kdeinit__0
unix  3      [ ]         STREAM     CONNECTED     9721     4378/kwrapper       
unix  3      [ ]         STREAM     CONNECTED     9667     4370/gam_server     @/tmp/fam-didier-
unix  3      [ ]         STREAM     CONNECTED     9666     4368/kded [kdeinit] 
unix  3      [ ]         STREAM     CONNECTED     9648     -                   /tmp/.X11-unix/X0
unix  3      [ ]         STREAM     CONNECTED     9647     4368/kded [kdeinit] 
unix  3      [ ]         STREAM     CONNECTED     9646     4363/dcopserver [kd /tmp/.ICE-unix/dcop4363-1213534701
unix  3      [ ]         STREAM     CONNECTED     9645     4368/kded [kdeinit] 
unix  3      [ ]         STREAM     CONNECTED     9634     -                   /tmp/.X11-unix/X0
unix  3      [ ]         STREAM     CONNECTED     9633     4360/kdeinit Runnin 
unix  3      [ ]         STREAM     CONNECTED     9626     4363/dcopserver [kd /tmp/.ICE-unix/dcop4363-1213534701
unix  3      [ ]         STREAM     CONNECTED     9625     4366/klauncher [kde 
unix  3      [ ]         STREAM     CONNECTED     9619     4366/klauncher [kde 
unix  3      [ ]         STREAM     CONNECTED     9618     4360/kdeinit Runnin 
unix  3      [ ]         STREAM     CONNECTED     9525     -                   /tmp/.X11-unix/X0
unix  3      [ ]         STREAM     CONNECTED     9524     4321/dbus-launch    
unix  3      [ ]         STREAM     CONNECTED     9523     4322/dbus-daemon    
unix  3      [ ]         STREAM     CONNECTED     9522     4322/dbus-daemon    
unix  5      [ ]         STREAM     CONNECTED     9509     -                   /tmp/.X11-unix/X0
unix  3      [ ]         STREAM     CONNECTED     9508     4321/dbus-launch    
unix  2      [ ]         DGRAM                    9445     -                   
unix  3      [ ]         STREAM     CONNECTED     9442     -                   /var/run/dbus/system_bus_socket
unix  3      [ ]         STREAM     CONNECTED     9441     -                   
unix  3      [ ]         STREAM     CONNECTED     9232     -                   /var/run/dbus/system_bus_socket
unix  3      [ ]         STREAM     CONNECTED     9231     -                   
unix  3      [ ]         STREAM     CONNECTED     8690     -                   /var/run/dbus/system_bus_socket
unix  3      [ ]         STREAM     CONNECTED     8689     -                   
unix  2      [ ]         DGRAM                    8685     -                   
unix  3      [ ]         STREAM     CONNECTED     8610     -                   @/var/run/hald/dbus-dV87ZJjdJn
unix  3      [ ]         STREAM     CONNECTED     8608     -                   
unix  3      [ ]         STREAM     CONNECTED     8607     -                   /var/run/dbus/system_bus_socket
unix  3      [ ]         STREAM     CONNECTED     8606     -                   
unix  3      [ ]         STREAM     CONNECTED     8609     -                   @/var/run/hald/dbus-dV87ZJjdJn
unix  3      [ ]         STREAM     CONNECTED     8604     -                   
unix  3      [ ]         STREAM     CONNECTED     8495     -                   /var/run/dbus/system_bus_socket
unix  3      [ ]         STREAM     CONNECTED     8494     -                   
unix  3      [ ]         STREAM     CONNECTED     8493     -                   @/var/run/hald/dbus-dV87ZJjdJn
unix  3      [ ]         STREAM     CONNECTED     8492     -                   
unix  3      [ ]         STREAM     CONNECTED     8490     -                   /var/run/acpid.socket
unix  3      [ ]         STREAM     CONNECTED     8489     -                   
unix  3      [ ]         STREAM     CONNECTED     8484     -                   @/var/run/hald/dbus-dV87ZJjdJn
unix  3      [ ]         STREAM     CONNECTED     8483     -                   
unix  3      [ ]         STREAM     CONNECTED     8462     -                   @/var/run/hald/dbus-dV87ZJjdJn
unix  3      [ ]         STREAM     CONNECTED     8461     -                   
unix  3      [ ]         STREAM     CONNECTED     8412     -                   /var/run/acpid.socket
unix  3      [ ]         STREAM     CONNECTED     8411     -                   
unix  8      [ ]         STREAM     CONNECTED     9233     -                   /tmp/.X11-unix/X0
unix  3      [ ]         STREAM     CONNECTED     8391     -                   
unix  2      [ ]         DGRAM                    8206     -                   
unix  3      [ ]         STREAM     CONNECTED     8188     -                   /var/run/dbus/system_bus_socket
unix  3      [ ]         STREAM     CONNECTED     8187     -                   
unix  2      [ ]         DGRAM                    8171     -                   
unix  3      [ ]         STREAM     CONNECTED     8161     -                   @/var/run/hald/dbus-2dFKypkmSh
unix  3      [ ]         STREAM     CONNECTED     8160     -                   
unix  3      [ ]         STREAM     CONNECTED     8136     -                   /var/run/dbus/system_bus_socket
unix  3      [ ]         STREAM     CONNECTED     8135     -                   
unix  3      [ ]         STREAM     CONNECTED     7998     -                   /var/run/dbus/system_bus_socket
unix  3      [ ]         STREAM     CONNECTED     7997     -                   
unix  2      [ ]         DGRAM                    7996     -                   
unix  3      [ ]         STREAM     CONNECTED     7985     -                   
unix  3      [ ]         STREAM     CONNECTED     7984     -                   
unix  2      [ ]         DGRAM                    7812     -                   
  sl  local_address rem_address   st tx_queue rx_queue tr tm->when retrnsmt   uid  timeout inode                                                     
   0: 00000000:1F4A 00000000:0000 0A 00000000:00000000 00:00000000 00000000  1000        0 31010 1 ffff81007b962d80 750 0 0 2 -1                     
   1: 00000000:008B 00000000:0000 0A 00000000:00000000 00:00000000 00000000     0        0 8628 1 ffff81007b963400 750 0 0 2 -1                      
   2: 00000000:006F 00000000:0000 0A 00000000:00000000 00:00000000 00000000     0        0 7080 1 ffff81007d9693c0 750 0 0 2 -1                      
   3: 0100007F:0277 00000000:0000 0A 00000000:00000000 00:00000000 00000000     0        0 8079 1 ffff81007d9686c0 750 0 0 2 -1                      
   4: 0100007F:0019 00000000:0000 0A 00000000:00000000 00:00000000 00000000     0        0 8059 1 ffff81007d968040 750 0 0 2 -1                      
   5: 00000000:01BD 00000000:0000 0A 00000000:00000000 00:00000000 00000000     0        0 8627 1 ffff81007d968d40 750 0 0 2 -1                      
   6: 0B01A8C0:DEE7 12DEB280:008F 01 00000000:00000000 00:00000000 00000000  1000        0 26497 1 ffff8100515af4c0 60 10 30 4 -1                    
   7: 0B01A8C0:D544 0301A8C0:2328 06 00000000:00000000 03:00000EAE 00000000     0        0 0 2 ffff810051163780                                      
   8: 0B01A8C0:D53F 0301A8C0:2328 06 00000000:00000000 03:000004F8 00000000     0        0 0 2 ffff8100514d6980                                      
   9: 0B01A8C0:D53D 0301A8C0:2328 06 00000000:00000000 03:000002BA 00000000     0        0 0 2 ffff8100514d6e80                                      
  10: 0B01A8C0:B8E6 6AFE96D9:1467 01 00000000:00000000 00:00000000 00000000  1000        0 31018 1 ffff810042ac0900 63 10 26 4 -1                    
  11: 0B01A8C0:D541 0301A8C0:2328 06 00000000:00000000 03:000008F2 00000000     0        0 0 2 ffff810051163880                                      
  12: 0B01A8C0:D54A 0301A8C0:2328 01 00000000:00000000 00:00000000 00000000  1000        0 43377 1 ffff8100515aee40 54 10 4 5 -1                     
  13: 0B01A8C0:D540 0301A8C0:2328 06 00000000:00000000 03:000006F2 00000000     0        0 0 2 ffff810051163380                                      
  14: 0B01A8C0:D262 280B90C3:008F 01 00000000:00000000 00:00000000 00000000  1000        0 22172 1 ffff81004b8c7700 64 10 30 4 -1                    
  15: 0B01A8C0:D543 0301A8C0:2328 06 00000000:00000000 03:00000CEB 00000000     0        0 0 2 ffff8100514d6280                                      
  16: 0B01A8C0:D546 0301A8C0:2328 06 00000000:00000000 03:000010E9 00000000     0        0 0 2 ffff8100514d6380                                      
  17: 0B01A8C0:9631 251919D4:008F 01 00000000:00000000 00:00000000 00000000  1000        0 22194 1 ffff81004b8c7080 67 10 28 4 -1                    
  18: 0B01A8C0:D545 0301A8C0:2328 06 00000000:00000000 03:00000EEB 00000000     0        0 0 2 ffff8100514d6a80                                      
  19: 0B01A8C0:D54B 0301A8C0:2328 01 00000000:00000000 00:00000000 00000000  1000        0 43424 1 ffff810042ac0f80 54 10 4 5 -1                     
  20: 0B01A8C0:D53E 0301A8C0:2328 06 00000000:00000000 03:000002F2 00000000     0        0 0 2 ffff8100514d6780                                      
  21: 0B01A8C0:D549 0301A8C0:2328 06 00000000:00000000 03:000016DD 00000000     0        0 0 2 ffff8100514d6b80                                      
  22: 0B01A8C0:D542 0301A8C0:2328 06 00000000:00000000 03:00000AE8 00000000     0        0 0 2 ffff810051163280                                      
  23: 0B01A8C0:D547 0301A8C0:2328 06 00000000:00000000 03:000012E2 00000000     0        0 0 2 ffff8100514d6480                                      
  24: 0B01A8C0:812A 7C441054:008F 01 00000000:00000000 00:00000000 00000000  1000        0 22154 1 ffff8100510da0c0 60 10 30 4 3                     
  25: 0B01A8C0:D548 0301A8C0:2328 06 00000000:00000000 03:000014DF 00000000     0        0 0 2 ffff8100514d6c80                                      
  26: 0B01A8C0:967D 0301A8C0:0016 01 00000000:00000000 02:000AD441 00000000  1000        0 42496 2 ffff81007b962080 54 10 25 4 -1                    
  27: 12B6A8C0:EC62 9A2D86D5:01BB 01 00000000:00000000 00:00000000 00000000  1000        0 12944 1 ffff8100515ae140 77 10 4 5 -1                     
#4 lore raw
Quoted 46 lines · Didier Raboud

Le samedi 14 juin 2008 22:45:41 Ilpo Järvinen, vous avez écrit :

Quoted 3 lines · Andrew Morton
Quoted 2 lines

(switched to email. Please respond via emailed reply-to-all, not via the bugzilla web interface).

OK.

Quoted 1 line
Quoted 1 line

On Fri, 13 Jun 2008 02:39:17 -0700 (PDT) [email protected]

Quoted 24 lines · wrote:
Quoted 24 lines
Quoted 24 lines

http://bugzilla.kernel.org/show_bug.cgi?id=10903

           Summary: ssh connections hang with 2.6.26-rc5
           Product: Networking
           Version: 2.5
     KernelVersion: 2.6.26-rc5
          Platform: All
        OS/Version: Linux
              Tree: Mainline
            Status: NEW
          Severity: normal
          Priority: P1
         Component: Other
        AssignedTo: [email protected]
        ReportedBy: [email protected]

Latest working kernel version: 2.6.25-2
Earliest failing kernel version: 2.6.26-rc5
Distribution: Debian (Lenny + Sid)
Hardware Environment: amd64 (Dell Latitude D630)
Software Environment: KDE
Problem Description:

Quoted 6 lines
Quoted 4 lines
Quoted 4 lines

With kernel version 2.6.26-rc5, the ssh connections to remote servers randomly
hang (no error message). No amelioration despite the activation of "ServerAliveInterval" on both sides.

Thanks for reporting. Could you please clarify couple of things:

Hi.

I will try to, with my time and knowledge.

Quoted 1 line

Does this only happen with a particular server/servers?

Quoted 8 lines

I have only tried with two of my home servers. One runs 2.6.22-4-686 and the other 2.6.18-6-vserver-686.

Quoted 1 line

Any middleboxes in between (NAT, firewall, etc.)?

There is a ADSL router which "provides" internet to the servers by NAT. I have tried from "inside" the house (so in the same subnet) and from
outside: it hangs in both cases.

...Ok. Those have some timeouts for idle sessions, so if one has active or keepalived sessions that shouldn't be a problem.

Quoted 7 lines

The common point is my use of "iwl3945" : I have always tried the ssh connections through WiFi.

Quoted 1 line

Do all ssh connections hang simultaneously?

Well... It is hard to say. As far as I have seen, no. When I get one hang, I can successfully connect to the same server.

It's quite likely that the hangs are independent, but it was worth of confirming still.

Quoted 24 lines
Quoted 1 line

How long have you waited until concluding that TCP is "hung"?

Well. The "ServerAliveInterval" option of openssh now leads to "Received disconnect from $IP: 2: Timeout, your session not responding." after the hang. So the openssh server notices that my session is not responding and so cuts the connection.

Quoted 1 line

Is TSO enabled (ethtool -k)? Have you tried without it?

Doesn't seem:

----

# ethtool -k wlan0
Offload parameters for wlan0:
Cannot get device rx csum settings: Operation not supported
Cannot get device tx csum settings: Operation not supported
Cannot get device scatter-gather settings: Operation not supported
Cannot get device tcp segmentation offload settings: Operation not supported
Cannot get device udp large send offload settings: Operation not supported
Cannot get device generic segmentation offload settings: Operation not 
supported
no offload info available
----
Quoted 5 lines
Quoted 3 lines

It wouldn't hurt to include info about eth hw too (e.g., lspci), though it might turn unneeded at some point of time but it might save an email round-trip.

lspci attached.

...Thanks for all the details. I especially appreciated the kernel versions of the servers since TCP has two end hosts (and I forgot to ask)... :-)

Quoted 11 lines
Quoted 7 lines

TCP can appear to hang due to vast number of reasons. Only recent changes that are suspectable is the DEFERRED_ACCEPT thing which is already reverted in the very latest Linus' tree (even -rc6 is too old for that) and few FRTO fixes (you can exclude FRTO by turning /proc/sys/net/ipv4/tcp_frto sysctl to 0 but it seems quite unlikely to change anything); your problem might well come from something else and TCP hang is just a symptom of other problem downstream.

I can't understand everything, but what I can say is that with the exact same software, I get no hangs with 2.6.25-2 but I get some with 2.6.26-rc5.

Yes, I understand that... I was just trying to bring up above what has changed between those kernels :-). ...Quite few TCP related changes actually (there were also some TSO related changes but they're not significant in your case).

Quoted 6 lines
Quoted 4 lines

So please gather this information (at least for the relevant connections):

$ netstat -pn
$ cat /proc/net/tcp

Attached.

I probably wasn't specific enough. ...I meant that you would get this once one of the ssh sessions gets stuck (do it right after you notice that the session is stuck, that should get the info before the connection is cut down). This info should be collected from both ends (on client and server).

Quoted 3 lines
Quoted 1 line

...Also a tcpdump might be handy (though I don't know yet).

Well. It seems that there is another bug here: everytime I tried a

Ah, lets try to figure that one out as well...

Quoted 4 lines
# tcpdump -w /tmp/tcpdump.wlan0 -i wlan0

I got a CPU lockup (or similar, can't know exactly, but keyboard blocked and nothing doable).

You probably run it under X, no? Please switch beforehand to some other vt (a textual one) then (Ctrl-Alt-Fn, where n < 6) and then log in and running that command there and see if you get some output into screen there. If you see something (e.g., a sudden OOPS message or some other warning printed) when it locks up, the easiest things is to take a shot with a digicam (or write it down somewhere else) and send that shot (or those details) to us please.

...Once you have a tcpdump, I can probably figure at least something out (though it might still just point to the right direction rather than exposing the actual cause).

Signature · 1 line
 i.

A message that is not in the archive (sent off-list or not yet received): [email protected]

#6 lore raw
Quoted 40 lines · Didier Raboud

Le lundi 16 juin 2008 15:21:25 Ilpo J�rvinen, vous avez �crit�:

Quoted 30 lines
Quoted 30 lines
Quoted 29 lines
Quoted 28 lines
Quoted 28 lines

http://bugzilla.kernel.org/show_bug.cgi?id=10903

           Summary: ssh connections hang with 2.6.26-rc5
           Product: Networking
           Version: 2.5
     KernelVersion: 2.6.26-rc5
          Platform: All
        OS/Version: Linux
              Tree: Mainline
            Status: NEW
          Severity: normal
          Priority: P1
         Component: Other
        AssignedTo: [email protected]
        ReportedBy: [email protected]

Latest working kernel version: 2.6.25-2
Earliest failing kernel version: 2.6.26-rc5
Distribution: Debian (Lenny + Sid)
Hardware Environment: amd64 (Dell Latitude D630)
Software Environment: KDE
Problem Description:

With kernel version 2.6.26-rc5, the ssh connections to remote servers randomly
hang (no error message). No amelioration despite the activation of "ServerAliveInterval" on both sides.

(...)

Quoted 8 lines
Quoted 8 lines

The common point is my use of "iwl3945" : I have always tried the ssh connections through WiFi.

Quoted 5 lines

So please gather this information (at least for the relevant connections):

$ netstat -pn
$ cat /proc/net/tcp
Quoted 7 lines

I used a script which logged both every 15 seconds on both sides (in "screen" on server side). I then triggered a hang with several "seq 1 100" in the ssh session. The logs are in the attached debug_ssh_hang.tar.gz . The hang appeared between 234327 and 234342 on the client side (so somewhere between 234324, 234339 and 234354 on the server side).

I hope it'll help.

Thanks, a quick look didn't indicate anything similar to one other report from H�kon... I'd probably need a tcpdump.

Quoted 8 lines
Quoted 7 lines

You probably run it under X, no? Please switch beforehand to some other vt (a textual one) then (Ctrl-Alt-Fn, where n < 6) and then log in and running that command there and see if you get some output into screen there. If you see something (e.g., a sudden OOPS message or some other warning printed) when it locks up, the easiest things is to take a shot with a digicam (or write it down somewhere else) and send that shot (or those details) to us please.

Quoted 5 lines

I tried the following in vt1 (under the new 2.6.26-rc6 with kdm stopped):

# tcpdump -i wlan0 -w /tmp/tcpdump.wlan0

and I got the attached "soft lockup".

Thanks, I guess the folks at linux-wireless can continue in resolving this one with you (I'm out of time anyway now).

Quoted 5 lines
Quoted 3 lines

...Once you have a tcpdump, I can probably figure at least something out (though it might still just point to the right direction rather than exposing the actual cause).

I can't get one... :)

...Ok.

Signature · 1 line
 i.

A message that is not in the archive (sent off-list or not yet received): Pine.LNX.4.64.0806180152410.32420-x/[email protected]

#8 lore raw
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

http://bugzilla.kernel.org/show_bug.cgi?id=10903

Summary: ssh connections hang with 2.6.26-rc5

Andrew Prince reported a similar problem and said he bisected it to davem's 608961a5eca8d3c6bd07172febc27b5559408c5d ("mac80211: Use skb_header_cloned() on TX path.") which made no sense to me so I marked the report as 'to investigate when I have more time'.

Quoted 8 lines
Quoted 8 lines
Quoted 7 lines

You probably run it under X, no? Please switch beforehand to some other vt (a textual one) then (Ctrl-Alt-Fn, where n < 6) and then log in and running that command there and see if you get some output into screen there. If you see something (e.g., a sudden OOPS message or some other warning printed) when it locks up, the easiest things is to take a shot with a digicam (or write it down somewhere else) and send that shot (or those details) to us please.

Quoted 5 lines
Quoted 5 lines

I tried the following in vt1 (under the new 2.6.26-rc6 with kdm stopped):

# tcpdump -i wlan0 -w /tmp/tcpdump.wlan0

and I got the attached "soft lockup".

Attached where? I can't find it in the bug either.

johannes

maintainer
#9 lore raw

RnJvbTogSm9oYW5uZXMgQmVyZyA8am9oYW5uZXNAc2lwc29sdXRpb25zLm5ldD4NCkRhdGU6IFdl ZCwgMTggSnVuIDIwMDggMDk6MjQ6NDcgKzAyMDANCg0KPiANCj4gPiA+ID4gPiA+ID4gPiBodHRw Oi8vYnVnemlsbGEua2VybmVsLm9yZy9zaG93X2J1Zy5jZ2k/aWQ9MTA5MDMNCj4gPiA+ID4gPiA+ ID4gPg0KPiA+ID4gPiA+ID4gPiA+ICAgICAgICAgICAgU3VtbWFyeTogc3NoIGNvbm5lY3Rpb25z IGhhbmcgd2l0aCAyLjYuMjYtcmM1DQo+IA0KPiBBbmRyZXcgUHJpbmNlIHJlcG9ydGVkIGEgc2lt aWxhciBwcm9ibGVtIGFuZCBzYWlkIGhlIGJpc2VjdGVkIGl0ICB0bw0KPiBkYXZlbSdzIO+7vzYw ODk2MWE1ZWNhOGQzYzZiZDA3MTcyZmViYzI3YjU1NTk0MDhjNWQgKCJtYWM4MDIxMTogVXNlDQo+ IHNrYl9oZWFkZXJfY2xvbmVkKCkgb24gVFggcGF0aC4iKSB3aGljaCBtYWRlIG5vIHNlbnNlIHRv IG1lIHNvIEkgbWFya2VkDQo+IHRoZSByZXBvcnQgYXMgJ3RvIGludmVzdGlnYXRlIHdoZW4gSSBo YXZlIG1vcmUgdGltZScuDQoNClRoYXQncyB1c2VmdWwgaW5mb3JtYXRpb24uICBUaGUga2VybmVs IGJ1Z3ppbGxhIGVudHJ5IGlzIGZvcg0KdGhlIGl3bDM5NDUgZHJpdmVyLCBzbyB0aGF0IG1hdGNo ZXMgdXAgYWNjdXJhdGVseSB3aXRoIHRoaXMNCnRvby4NCg0KSWYgd2UgY2FuJ3QgZmlndXJlIG91 dCB3aGF0J3MgZ29pbmcgb24gaGVyZSBzb29uIChsaWtlLCBpbiBsZXNzIHRoYW4gYQ0KZGF5KSB3 ZSBzaG91bGQgcmV2ZXJ0IHRoYXQgY2hhbmdlc2V0Lg0KDQpBY3R1YWxseSwgSSB0aGluayBJIHNl ZSBob3cgdGhlIGNoYW5nZXNldCBtaWdodCBiZSB3cm9uZy4gIEkgdGhpbmsNCnRoZSBlbmNyeXB0 aW9uIGxheWVyIG9mIG1hYzgwMjExIGFzc3VtZXMgaXQgY2FuIHdyaXRlIG92ZXIgdGhlDQpkYXRh IGFyZWEgb2YgdGhlIFNLQiBpdCdzIHdvcmtpbmcgb24sIG5vdCBqdXN0IHRoZSBoZWFkZXJzLg0K DQpPbmNlIHRoaXMgaGFwcGVucywgYW55IHJldHJhbnNtaXRzIGRvbmUgYnkgU0tCIHdpbGwgZmFp bCBiZWNhdXNlIHRoZQ0KbWFzdGVyIHBhY2tldCBkYXRhIG9uIFRDUCdzIHJldHJhbnNtaXQgcXVl dWUgaXMgbm93IHRoaXMgZW5jcnlwdGVkDQpnYXJiYWdlLg0K

A message that is not in the archive (sent off-list or not yet received): 20080618.010528.05757230.davem-fT/PcQaiUtIeIZ0/[email protected]

#11 lore raw

From: David Miller <[email protected]>
Date: Wed, 18 Jun 2008 01:05:28 -0700 (PDT)

Quoted 10 lines

If we can't figure out what's going on here soon (like, in less than a day) we should revert that changeset.

Actually, I think I see how the changeset might be wrong. I think the encryption layer of mac80211 assumes it can write over the data area of the SKB it's working on, not just the headers.

Once this happens, any retransmits done by SKB will fail because the master packet data on TCP's retransmit queue is now this encrypted garbage.

After some discussion about this with Johannes on IRC, we are absolutely convinced this is exactly the problem.

I intend to send the following revert to Linus tonight so we can close this:

--------------------
Revert "mac80211: Use skb_header_cloned() on TX path."

This reverts commit 608961a5eca8d3c6bd07172febc27b5559408c5d.

The problem is that the mac80211 stack not only needs to be able to muck with the link-level headers, it also might need to mangle all of the packet data if doing sw wireless encryption.

This fixes kernel bugzilla #10903. Thanks to Didier Raboud (for the bugzilla report), Andrew Prince (for bisecting), Johannes Berg (for bringing this bisection analysis to my attention), and Ilpo (for trying to analyze this purely from the TCP side).

In 2.6.27 we can take another stab at this, by using something like skb_cow_data() when the TX path of mac80211 ends up with a non-NULL tx->key. The ESP protocol code in the IPSEC stack can be used as a model for implementation.

---
 net/mac80211/tx.c |    4 ++--
 1 files changed, 2 insertions(+), 2 deletions(-)
net/mac80211/tx.c +2 -2
index 1d7dd54..28d8bd5 100644
@@ -1562,13 +1562,13 @@ int ieee80211_subif_start_xmit(struct sk_buff *skb,
 	 * be cloned. This could happen, e.g., with Linux bridge code passing
 	 * us broadcast frames. */
 
-	if (head_need > 0 || skb_header_cloned(skb)) {
+	if (head_need > 0 || skb_cloned(skb)) {
 #if 0
 		printk(KERN_DEBUG "%s: need to reallocate buffer for %d bytes "
 		       "of headroom\n", dev->name, head_need);
 #endif
 
-		if (skb_header_cloned(skb))
+		if (skb_cloned(skb))
 			I802_DEBUG_INC(local->tx_expand_skb_head_cloned);
 		else
 			I802_DEBUG_INC(local->tx_expand_skb_head);
Signature · 1 line
1.5.5.1.308.g1fbb5

A message that is not in the archive (sent off-list or not yet received): 1213773887.3803.158.camel-YfaajirXv214zXjbi5bjpg@public.gmane.org

#13 lore raw

Le mercredi 18 juin 2008 09:24:47 Johannes Berg, vous avez écrit :

Quoted 9 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

http://bugzilla.kernel.org/show_bug.cgi?id=10903

Summary: ssh connections hang with 2.6.26-rc5

Andrew Prince reported a similar problem and said he bisected it to davem's 608961a5eca8d3c6bd07172febc27b5559408c5d ("mac80211: Use skb_header_cloned() on TX path.") which made no sense to me so I marked the report as 'to investigate when I have more time'.

Quoted 5 lines
Quoted 1 line
Quoted 1 line

and I got the attached "soft lockup".

Attached where? I can't find it in the bug either.

johannes

Hi,

for unknown reasons, my mail doesn't appear in the mailing list archives and the big attachment has visibly been stripped.

You'll find the "soft lockup" there:

http://raboud.homelinux.org/~didier/kernel/tcpdump_oops.jpg

Regards,

Didier

Signature · 3 lines
Didier Raboud, proud Debian user.
CH-1802 Corseaux
[email protected]

A message that is not in the archive (sent off-list or not yet received): 200806181334.10654.didier-efQUOpnRmEvQT0dZR+AlfA@public.gmane.org

#15 lore raw
Quoted 16 lines · On Wednesday 18 June 2008 13:34:06 Didier Raboud wrote:

Le mercredi 18 juin 2008 09:24:47 Johannes Berg, vous avez écrit :

Quoted 9 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

http://bugzilla.kernel.org/show_bug.cgi?id=10903

Summary: ssh connections hang with 2.6.26-rc5

Andrew Prince reported a similar problem and said he bisected it to davem's 608961a5eca8d3c6bd07172febc27b5559408c5d ("mac80211: Use skb_header_cloned() on TX path.") which made no sense to me so I marked the report as 'to investigate when I have more time'.

Quoted 5 lines
Quoted 1 line
Quoted 1 line

and I got the attached "soft lockup".

Attached where? I can't find it in the bug either.

johannes

Quoted 8 lines

Hi,

for unknown reasons, my mail doesn't appear in the mailing list archives and the big attachment has visibly been stripped.

You'll find the "soft lockup" there:

http://raboud.homelinux.org/~didier/kernel/tcpdump_oops.jpg

The soft lockup most likely is a followup-oops to a previous one that locked up the machine.
Can you try to reproduce and capture the screen before waiting 61 seconds for the watchdog to trigger. There should be another oops before that (you see the last two lines of it on this picture)

Signature · 5 lines
Greetings Michael.
--
To unsubscribe from this list: send the line "unsubscribe linux-wireless" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html