Skip to contentSkip to conversation

automount in status D

DISCUSSION Carsten Aulbert 3 participants 6 messages Jun 27, 2008 Kernel automounter

lore
Share & tools
lore
#0 lore raw

Hi all,

we have a weird problem here and I fear I need to reboot the box (painful, many users involved), but maybe there is a way out:

I'm automounting many nodes and usually everything seems to work nicely. However, I'm seeing this here right now:

# ps aux|grep auto.node
root      2022  0.0  0.0  30304  1240 ?        Ds   Jun19   0:21 /usr/sbin/automount --pid-file=/var/run/autofs/_atlas_node.pid --timeout=5 /atlas/node yp auto.node_local
root     11489  0.0  0.0  30516  1108 ?        D    Jun26   0:00 /usr/sbin/automount --pid-file=/var/run/autofs/_atlas_node.pid --timeout=5 /atlas/node yp auto.node_local
root     28610  0.0  0.0   2736   656 pts/31   S+   10:03   0:00 grep auto.node

As you can see there are two automounter processes in D (or Ds) state sitting there for the same mount point (the PID-file contains 2022).

My questions:

Any idea how this happened?

Any idea how to get this resolved (possibly without a reboot)?

TIA

Carsten

PS:

# ypcat auto.node_local
-fstype=nfs,nfsvers=3,soft,intr,rsize=32768,wsize=32768,tcp       &:/local
maintainer
#1 lore raw
Quoted 14 lines · Carsten Aulbert

Hi all,

we have a weird problem here and I fear I need to reboot the box (painful, many users involved), but maybe there is a way out:

I'm automounting many nodes and usually everything seems to work nicely. However, I'm seeing this here right now:

# ps aux|grep auto.node
root      2022  0.0  0.0  30304  1240 ?        Ds   Jun19   0:21 /usr/sbin/automount --pid-file=/var/run/autofs/_atlas_node.pid --timeout=5 /atlas/node yp auto.node_local
root     11489  0.0  0.0  30516  1108 ?        D    Jun26   0:00 /usr/sbin/automount --pid-file=/var/run/autofs/_atlas_node.pid --timeout=5 /atlas/node yp auto.node_local
root     28610  0.0  0.0   2736   656 pts/31   S+   10:03   0:00 grep auto.node
Quoted 5 lines

As you can see there are two automounter processes in D (or Ds) state sitting there for the same mount point (the PID-file contains 2022).

My questions:

Any idea how this happened?

It's been a long time since I've seen this reported. What distro and kernel?

Quoted 2 lines

Any idea how to get this resolved (possibly without a reboot)?

There is nothing you can do if the process is stuck in an uninterruptible state in the kernel.

Ian

#2 lore raw
Quoted 5 lines · Ian Kent
Quoted 1 line

Any idea how this happened?

It's been a long time since I've seen this reported. What distro and kernel?

Sorry, Debian Etch with self compiled kernel:

h2:~# uname -a
Linux h2 2.6.24.4-storage #1 SMP Sun Apr 20 09:33:42 CEST 2008 x86_64
x86_64 x86_64 GNU/Linux
h2:~# apt-cache show autofs|grep Version
Version: 4.1.4-13
Quoted 4 lines
Quoted 1 line

Any idea how to get this resolved (possibly without a reboot)?

There is nothing you can do if the process is stuck in an uninterruptible state in the kernel.

Well, as bad as I feared it to be :(

Thanks already, reboot is already announced to the users. Anything I might help you with to corner that bug?

Cheers

Carsten

Signature · 9 lines
Dr. Carsten Aulbert - Max Planck Institut für Gravitationsphysik
Callinstraße 38, 30167 Hannover, Germany
Fon: +49 511 762 17185, Fax: +49 511 762 17193
http://www.top500.org/system/9234 | http://www.top500.org/connfam/6/list/31

_______________________________________________
autofs mailing list
[email protected]
http://linux.kernel.org/mailman/listinfo/autofs
maintainer
#3 lore raw
Quoted 18 lines · Carsten Aulbert
Quoted 5 lines · Ian Kent
Quoted 1 line

Any idea how this happened?

It's been a long time since I've seen this reported. What distro and kernel?

Sorry, Debian Etch with self compiled kernel:

h2:~# uname -a
Linux h2 2.6.24.4-storage #1 SMP Sun Apr 20 09:33:42 CEST 2008 x86_64
x86_64 x86_64 GNU/Linux
h2:~# apt-cache show autofs|grep Version
Version: 4.1.4-13
Quoted 2 lines
Quoted 1 line

Any idea how to get this resolved (possibly without a reboot)?

Quoted 7 lines
Quoted 2 lines

There is nothing you can do if the process is stuck in an uninterruptible state in the kernel.

Well, as bad as I feared it to be :(

Thanks already, reboot is already announced to the users. Anything I might help you with to corner that bug?

Aggh .. it's been soooo long since I've heard of this happening I can't remember.

One thing that is interesting is that the autofs4 kernel module doesn't do an uninterruptible sleep "anywhere". It hasn't since long before 2.6.24. So if user space can't recover by doing a control-c (maybe even a couple of times) then it's possibly not actually autofs.

Ian

#4 lore raw

Hi Ian,

Quoted 4 lines · Ian Kent

One thing that is interesting is that the autofs4 kernel module doesn't do an uninterruptible sleep "anywhere". It hasn't since long before 2.6.24. So if user space can't recover by doing a control-c (maybe even a couple of times) then it's possibly not actually autofs.

Well, when I tried to umount it manually (with options -f -l -i) stopped indefinitely when doing a readlink on the mount point itself and never returned from its own account.

I was able to CTRL-C this one but not getting the mount point removed. A nasty side effect was that df never returned either.

I did a bit more log reading and it seems that the node got a mount as well as a umount request yesterday evening and from the nodes perspective everything looked fine.

In the end I guess this falls into the category of: Hopefully it won't happen again too soon ;)

If you come up with an idea, please let me know. I'll stay on the list, reading quietly.

Cheers

Carsten

#5 lore raw
In reply to Ian Kent
Quoted 22 lines · Ian Kent
Quoted 21 lines · Carsten Aulbert
Quoted 5 lines · Ian Kent
Quoted 1 line

Any idea how this happened?

It's been a long time since I've seen this reported. What distro and kernel?

Sorry, Debian Etch with self compiled kernel:

h2:~# uname -a
Linux h2 2.6.24.4-storage #1 SMP Sun Apr 20 09:33:42 CEST 2008 x86_64
x86_64 x86_64 GNU/Linux
h2:~# apt-cache show autofs|grep Version
Version: 4.1.4-13
Quoted 4 lines
Quoted 1 line

Any idea how to get this resolved (possibly without a reboot)?

There is nothing you can do if the process is stuck in an uninterruptible state in the kernel.

Quoted 7 lines
Quoted 4 lines

Well, as bad as I feared it to be :(

Thanks already, reboot is already announced to the users. Anything I might help you with to corner that bug?

Aggh .. it's been soooo long since I've heard of this happening I can't remember.

A sysrq-t would be a good starting point.

-Jeff