Skip to contentSkip to conversation

nfs3 lockd: cannot monitor errors

DISCUSSION steve 2 participants 5 messages Mar 3, 2012 – Mar 6, 2012

Subsystems (2)
lore
Share & tools
lore

steve

author
#0 lore raw

Hi
We recently switched to nfs3 from nfs4 and now we're getting lots of lockd errors. We can remove the error by mounting with:
-o local_lock=posix

We had to switch to be able to use the posix acl we had set on the share.

What problems may we face by setting the local_lock? Thanks,
Steve

author
#1 lore raw
Quoted 11 lines · steve

Hi
We recently switched to nfs3 from nfs4 and now we're getting lots of lockd errors. We can remove the error by mounting with:
-o local_lock=posix

We had to switch to be able to use the posix acl we had set on the share.

What problems may we face by setting the local_lock? Thanks,
Steve

Hi
Sorry to bump
This is openSUSE 12.1. All the references to lockd probs seem to come from 10 or so years ago.

Could anyone give me a one liner as to where to start looking? I've gone through all the usual channels.

Cheers,
Steve

Server:

Mar  5 16:24:46 hh3 kernel: [16760.656609] lockd: cannot monitor hh6

ps aux | grep rpc

root      1214  0.0  0.1   2356   656 ?        Ss   Mar05   0:00 /sbin/rpcbind
root      1821  0.0  0.0      0     0 ?        S<   Mar05   0:00 [rpciod]
root      4649  0.0  0.0   2416   416 ?        Ss   Mar05   0:00 /usr/sbin/rpc.idmapd
root      4668  0.0  0.3   4092  1508 ?        Ss   Mar05   0:00 rpc.gssd
root      5192  0.0  0.2   3840  1096 ?        Ss   Mar05   0:00 /usr/sbin/rpc.svcgssd
root      5206  0.0  0.1   3032   568 ?        Ss   Mar05   0:00 /usr/sbin/rpc.mountd --no-nfs-version 4
statd     5243  0.0  0.1   2584   652 ?        Ss   Mar05   0:00 /usr/sbin/rpc.statd --no-notify

rpcinfo

   program version netid     address                service    owner
    100000    4    tcp6      ::.0.111               portmapper superuser
    100000    3    tcp6      ::.0.111               portmapper superuser
    100000    4    udp6      ::.0.111               portmapper superuser
    100000    3    udp6      ::.0.111               portmapper superuser
    100000    4    tcp       0.0.0.0.0.111          portmapper superuser
    100000    3    tcp       0.0.0.0.0.111          portmapper superuser
    100000    2    tcp       0.0.0.0.0.111          portmapper superuser
    100000    4    udp       0.0.0.0.0.111          portmapper superuser
    100000    3    udp       0.0.0.0.0.111          portmapper superuser
    100000    2    udp       0.0.0.0.0.111          portmapper superuser
    100000    4    local     /var/run/rpcbind.sock  portmapper superuser
    100000    3    local     /var/run/rpcbind.sock  portmapper superuser
    100005    1    udp       0.0.0.0.178.110        mountd     superuser
    100005    1    tcp       0.0.0.0.136.30         mountd     superuser
    100005    1    udp6      ::.145.115             mountd     superuser
    100005    1    tcp6      ::.220.209             mountd     superuser
    100005    2    udp       0.0.0.0.163.232        mountd     superuser
    100003    2    tcp       0.0.0.0.8.1            nfs        superuser
    100003    3    tcp       0.0.0.0.8.1            nfs        superuser
    100227    2    tcp       0.0.0.0.8.1            nfs_acl    superuser
    100227    3    tcp       0.0.0.0.8.1            nfs_acl    superuser
    100003    2    udp       0.0.0.0.8.1            nfs        superuser
    100003    3    udp       0.0.0.0.8.1            nfs        superuser
    100227    2    udp       0.0.0.0.8.1            nfs_acl    superuser
    100227    3    udp       0.0.0.0.8.1            nfs_acl    superuser
    100005    2    tcp       0.0.0.0.210.206        mountd     superuser
    100005    2    udp6      ::.150.240             mountd     superuser
    100003    2    tcp6      ::.8.1                 nfs        superuser
    100003    3    tcp6      ::.8.1                 nfs        superuser
    100227    2    tcp6      ::.8.1                 nfs_acl    superuser
    100227    3    tcp6      ::.8.1                 nfs_acl    superuser
    100003    2    udp6      ::.8.1                 nfs        superuser
    100003    3    udp6      ::.8.1                 nfs        superuser
    100227    2    udp6      ::.8.1                 nfs_acl    superuser
    100227    3    udp6      ::.8.1                 nfs_acl    superuser
    100021    1    udp       0.0.0.0.141.165        nlockmgr   superuser
    100021    3    udp       0.0.0.0.141.165        nlockmgr   superuser
    100021    4    udp       0.0.0.0.141.165        nlockmgr   superuser
    100021    1    tcp       0.0.0.0.173.52         nlockmgr   superuser
    100021    3    tcp       0.0.0.0.173.52         nlockmgr   superuser
    100021    4    tcp       0.0.0.0.173.52         nlockmgr   superuser
    100021    1    udp6      ::.208.135             nlockmgr   superuser
    100021    3    udp6      ::.208.135             nlockmgr   superuser
    100021    4    udp6      ::.208.135             nlockmgr   superuser
    100021    1    tcp6      ::.175.226             nlockmgr   superuser
    100021    3    tcp6      ::.175.226             nlockmgr   superuser
    100021    4    tcp6      ::.175.226             nlockmgr   superuser
    100005    2    tcp6      ::.200.118             mountd     superuser
    100005    3    udp       0.0.0.0.217.9          mountd     superuser
    100005    3    tcp       0.0.0.0.185.184        mountd     superuser
    100005    3    udp6      ::.226.152             mountd     superuser
    100005    3    tcp6      ::.204.113             mountd     superuser
    100024    1    udp       0.0.0.0.131.53         status     103
    100024    1    tcp       0.0.0.0.130.61         status     103
    100024    1    udp6      ::.140.153             status     103
    100024    1    tcp6      ::.179.31              status     103
#2 lore raw
Quoted 17 lines · steve
Quoted 11 lines · steve

Hi
We recently switched to nfs3 from nfs4 and now we're getting lots of lockd errors. We can remove the error by mounting with:
-o local_lock=posix

We had to switch to be able to use the posix acl we had set on the share.

What problems may we face by setting the local_lock? Thanks,
Steve

Hi
Sorry to bump
This is openSUSE 12.1. All the references to lockd probs seem to come from 10 or so years ago.

Quoted 7 lines

Could anyone give me a one liner as to where to start looking? I've gone through all the usual channels.

Cheers,
Steve

Server:

Mar  5 16:24:46 hh3 kernel: [16760.656609] lockd: cannot monitor hh6

This error message means "hh3" cannot monitor "hh6". Usually that's a sign that rpc.statd on hh3 is having trouble getting a clean DNS lookup of hh6. Maybe enabling debugging on statd would produce a little more diagnostic information.

Quoted 74 lines

ps aux | grep rpc

root      1214  0.0  0.1   2356   656 ?        Ss   Mar05   0:00 /sbin/rpcbind
root      1821  0.0  0.0      0     0 ?        S<   Mar05   0:00 [rpciod]
root      4649  0.0  0.0   2416   416 ?        Ss   Mar05   0:00 /usr/sbin/rpc.idmapd
root      4668  0.0  0.3   4092  1508 ?        Ss   Mar05   0:00 rpc.gssd
root      5192  0.0  0.2   3840  1096 ?        Ss   Mar05   0:00 /usr/sbin/rpc.svcgssd
root      5206  0.0  0.1   3032   568 ?        Ss   Mar05   0:00 /usr/sbin/rpc.mountd --no-nfs-version 4
statd     5243  0.0  0.1   2584   652 ?        Ss   Mar05   0:00 /usr/sbin/rpc.statd --no-notify

rpcinfo

  program version netid     address                service    owner
   100000    4    tcp6      ::.0.111               portmapper superuser
   100000    3    tcp6      ::.0.111               portmapper superuser
   100000    4    udp6      ::.0.111               portmapper superuser
   100000    3    udp6      ::.0.111               portmapper superuser
   100000    4    tcp       0.0.0.0.0.111          portmapper superuser
   100000    3    tcp       0.0.0.0.0.111          portmapper superuser
   100000    2    tcp       0.0.0.0.0.111          portmapper superuser
   100000    4    udp       0.0.0.0.0.111          portmapper superuser
   100000    3    udp       0.0.0.0.0.111          portmapper superuser
   100000    2    udp       0.0.0.0.0.111          portmapper superuser
   100000    4    local     /var/run/rpcbind.sock  portmapper superuser
   100000    3    local     /var/run/rpcbind.sock  portmapper superuser
   100005    1    udp       0.0.0.0.178.110        mountd     superuser
   100005    1    tcp       0.0.0.0.136.30         mountd     superuser
   100005    1    udp6      ::.145.115             mountd     superuser
   100005    1    tcp6      ::.220.209             mountd     superuser
   100005    2    udp       0.0.0.0.163.232        mountd     superuser
   100003    2    tcp       0.0.0.0.8.1            nfs        superuser
   100003    3    tcp       0.0.0.0.8.1            nfs        superuser
   100227    2    tcp       0.0.0.0.8.1            nfs_acl    superuser
   100227    3    tcp       0.0.0.0.8.1            nfs_acl    superuser
   100003    2    udp       0.0.0.0.8.1            nfs        superuser
   100003    3    udp       0.0.0.0.8.1            nfs        superuser
   100227    2    udp       0.0.0.0.8.1            nfs_acl    superuser
   100227    3    udp       0.0.0.0.8.1            nfs_acl    superuser
   100005    2    tcp       0.0.0.0.210.206        mountd     superuser
   100005    2    udp6      ::.150.240             mountd     superuser
   100003    2    tcp6      ::.8.1                 nfs        superuser
   100003    3    tcp6      ::.8.1                 nfs        superuser
   100227    2    tcp6      ::.8.1                 nfs_acl    superuser
   100227    3    tcp6      ::.8.1                 nfs_acl    superuser
   100003    2    udp6      ::.8.1                 nfs        superuser
   100003    3    udp6      ::.8.1                 nfs        superuser
   100227    2    udp6      ::.8.1                 nfs_acl    superuser
   100227    3    udp6      ::.8.1                 nfs_acl    superuser
   100021    1    udp       0.0.0.0.141.165        nlockmgr   superuser
   100021    3    udp       0.0.0.0.141.165        nlockmgr   superuser
   100021    4    udp       0.0.0.0.141.165        nlockmgr   superuser
   100021    1    tcp       0.0.0.0.173.52         nlockmgr   superuser
   100021    3    tcp       0.0.0.0.173.52         nlockmgr   superuser
   100021    4    tcp       0.0.0.0.173.52         nlockmgr   superuser
   100021    1    udp6      ::.208.135             nlockmgr   superuser
   100021    3    udp6      ::.208.135             nlockmgr   superuser
   100021    4    udp6      ::.208.135             nlockmgr   superuser
   100021    1    tcp6      ::.175.226             nlockmgr   superuser
   100021    3    tcp6      ::.175.226             nlockmgr   superuser
   100021    4    tcp6      ::.175.226             nlockmgr   superuser
   100005    2    tcp6      ::.200.118             mountd     superuser
   100005    3    udp       0.0.0.0.217.9          mountd     superuser
   100005    3    tcp       0.0.0.0.185.184        mountd     superuser
   100005    3    udp6      ::.226.152             mountd     superuser
   100005    3    tcp6      ::.204.113             mountd     superuser
   100024    1    udp       0.0.0.0.131.53         status     103
   100024    1    tcp       0.0.0.0.130.61         status     103
   100024    1    udp6      ::.140.153             status     103
   100024    1    tcp6      ::.179.31              status     103

--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

Signature · 2 lines
Chuck Lever
chuck[dot]lever[at]oracle[dot]com
author
#3 lore raw
Quoted 22 lines · Chuck Lever
Quoted 19 lines · steve
Quoted 11 lines · steve

Hi
We recently switched to nfs3 from nfs4 and now we're getting lots of lockd errors. We can remove the error by mounting with:
-o local_lock=posix

We had to switch to be able to use the posix acl we had set on the share.

What problems may we face by setting the local_lock? Thanks,
Steve

Hi
Sorry to bump
This is openSUSE 12.1. All the references to lockd probs seem to come from 10 or so years ago.

Could anyone give me a one liner as to where to start looking? I've gone through all the usual channels.

Quoted 8 lines
Quoted 5 lines

Cheers,
Steve

Server:

Mar  5 16:24:46 hh3 kernel: [16760.656609] lockd: cannot monitor hh6

This error message means "hh3" cannot monitor "hh6". Usually that's a sign that rpc.statd on hh3 is having trouble getting a clean DNS lookup of hh6. Maybe enabling debugging on statd would produce a little more diagnostic information.

Hi Chuck
Thanks for the reply. You've got me on the right track. Running at both ends in the foreground is rock solid and instantaneous:

server: hh3, 192.168.1.3
hh3:/home/steve # rpc.statd -Fd
rpc.statd: Version 1.2.5 starting
rpc.statd: Flags: No-Daemon Log-STDERR TI-RPC
sm-notify: Version 1.2.5 starting
sm-notify: Already notifying clients; Exiting!
rpc.statd: Local NSM state number: 459
rpc.statd: Effective UID, GID: 103, 65534
rpc.statd: Waiting for client connections
rpc.statd: from_local: updating local if addr list
rpc.statd: from_local: checked 5 local if addrs; incoming address not found rpc.statd: check_default: access by 192.168.1.12 ALLOWED rpc.statd: Received SM_NOTIFY from hh6, state: 59
rpc.statd: SM_NOTIFY from hh6 while not monitoring any hosts rpc.statd: Waiting for client connections
rpc.statd: from_local: updating local if addr list
rpc.statd: from_local: incoming address matches local interface address rpc.statd: check_default: access by 127.0.0.1 ALLOWED
rpc.statd: Received SM_MON for 192.168.1.12 from hh3
rpc.statd: get_nameinfo: failed to resolve address: Name or service not known

client: hh6, 192.168.1.12
rpc.statd: MONITORING 192.168.1.12 for hh3
rpc.statd: Waiting for client connections
rpc.statd -Fd
rpc.statd: Version 1.2.5 starting
rpc.statd: Flags: No-Daemon Log-STDERR TI-RPC
sm-notify: Version 1.2.5 starting
sm-notify: Already notifying clients; Exiting! rpc.statd: Adding record for hh3.hh3.site to the monitor list... rpc.statd: Loaded 1 previously monitored hosts rpc.statd: Local NSM state number: 59
rpc.statd: Effective UID, GID: 103, 65534
rpc.statd: Waiting for client connections

Dropping to daemon makes the errors reappear with slow file transfer.

I've also nailed the Thunar file manager under XFCE which seems to be making calls to cifs via Kerberos each time we request a file ??. With Nautilus it's fine. The nfs/server principal is called once at the start of the session with no cifs requests.

Just one quick question, does the client server statd output look OK? Thanks,
Steve

#4 lore raw
Quoted 47 lines · steve
Quoted 22 lines · Chuck Lever
Quoted 19 lines · steve
Quoted 11 lines · steve

Hi
We recently switched to nfs3 from nfs4 and now we're getting lots of lockd errors. We can remove the error by mounting with:
-o local_lock=posix

We had to switch to be able to use the posix acl we had set on the share.

What problems may we face by setting the local_lock? Thanks,
Steve

Hi
Sorry to bump
This is openSUSE 12.1. All the references to lockd probs seem to come from 10 or so years ago.

Could anyone give me a one liner as to where to start looking? I've gone through all the usual channels.

Quoted 8 lines
Quoted 5 lines

Cheers,
Steve

Server:

Mar  5 16:24:46 hh3 kernel: [16760.656609] lockd: cannot monitor hh6

This error message means "hh3" cannot monitor "hh6". Usually that's a sign that rpc.statd on hh3 is having trouble getting a clean DNS lookup of hh6. Maybe enabling debugging on statd would produce a little more diagnostic information.

Hi Chuck
Thanks for the reply. You've got me on the right track. Running at both ends in the foreground is rock solid and instantaneous:

server: hh3, 192.168.1.3
hh3:/home/steve # rpc.statd -Fd
rpc.statd: Version 1.2.5 starting
rpc.statd: Flags: No-Daemon Log-STDERR TI-RPC
sm-notify: Version 1.2.5 starting
sm-notify: Already notifying clients; Exiting!
rpc.statd: Local NSM state number: 459
rpc.statd: Effective UID, GID: 103, 65534
rpc.statd: Waiting for client connections
rpc.statd: from_local: updating local if addr list
rpc.statd: from_local: checked 5 local if addrs; incoming address not found rpc.statd: check_default: access by 192.168.1.12 ALLOWED

Quoted 8 lines

rpc.statd: Received SM_NOTIFY from hh6, state: 59
rpc.statd: SM_NOTIFY from hh6 while not monitoring any hosts rpc.statd: Waiting for client connections
rpc.statd: from_local: updating local if addr list
rpc.statd: from_local: incoming address matches local interface address rpc.statd: check_default: access by 127.0.0.1 ALLOWED
rpc.statd: Received SM_MON for 192.168.1.12 from hh3
rpc.statd: get_nameinfo: failed to resolve address: Name or service not known

This is probably why you get the "lockd" failed to monitor" message. If the NFS server can't resolve "hh6.hh6.site" then it can't monitor it.

Quoted 14 lines

client: hh6, 192.168.1.12
rpc.statd: MONITORING 192.168.1.12 for hh3
rpc.statd: Waiting for client connections
rpc.statd -Fd
rpc.statd: Version 1.2.5 starting
rpc.statd: Flags: No-Daemon Log-STDERR TI-RPC
sm-notify: Version 1.2.5 starting
sm-notify: Already notifying clients; Exiting! rpc.statd: Adding record for hh3.hh3.site to the monitor list... rpc.statd: Loaded 1 previously monitored hosts rpc.statd: Local NSM state number: 59
rpc.statd: Effective UID, GID: 103, 65534
rpc.statd: Waiting for client connections

Quoted 5 lines

Dropping to daemon makes the errors reappear with slow file transfer.

I've also nailed the Thunar file manager under XFCE which seems to be making calls to cifs via Kerberos each time we request a file ??. With Nautilus it's fine. The nfs/server principal is called once at the start of the session with no cifs requests.

Just one quick question, does the client server statd output look OK?

At first blush, it looks you definitely have some DNS configuration problems. statd can only work when the forward and reverse DNS maps for both peers match each other.

Signature · 2 lines
Chuck Lever
chuck[dot]lever[at]oracle[dot]com