nfs is stupid ("getfh failed")
Two systems that worked fine for weeks, both running 2.4.[7,8] kernels. The server is running 2.4.8 and exporting a reiserfs filesystem via nfs. Or it was, anyway. The server was shut down and brought back up (power failure). The client was then
rebooted.
server# cat /etc/exports
/export 192.168.1.*(rw,no_root_squash)
/export/home 192.168.1.*(rw,no_root_squash)
client# mount /export
mount: 192.168.1.1:/export failed, reason given by server: Permission denied
server# tail /var/log/messages
Sep 6 09:37:43 gateway rpc.mountd: authenticated mount request from 192.168.1.133:933 for /export (/export) Sep 6 09:37:43 gateway rpc.mountd: getfh failed: Operation not permitted
... so, rebooting two working systems seems to kill NFS. Any ideas why?
On a related topic, will Linux ever have a better file-service protocol?
From: "Neil Brown" <[email protected]>
Quoted 1 line
This is not allowed, and makes no sense.
It apparently is (or was, anyway) allowed, because it worked until the server was rebooted. Fluke?
Are you saying that, if I export "/export", I can mount "/export/home" from a client machine? That's nice.
Quoted 1 line
Simply remove the second line and your problems should go away.
Thanks. I actually switched to two explicit exports; /export/files and /export/home, which works.
Quoted 4 lines
Quoted 1 line
On a related topic, will Linux ever have a better file-service protocol?
This is an unanswerable question. The future of Linux is completely undefined until it happens.
Just wondering if there's been any talk, plans, etc. of an alternative for NFS.
Quoted 1 line
What exactly do you mean by "better" anyway?
Better security, better performance.
Thanks,
-M
Quoted 4 lines · Michael Rothwell
server# tail
/var/log/messagesSep 6 09:37:43 gateway rpc.mountd: authenticated mount request from 192.168.1.133:933 for /export (/export) Sep 6 09:37:43 gateway rpc.mountd: getfh failed: Operation not permitted
I'm seeing this message quite often with one Linux 2.4.7 system automounting another. As long as A has B's filesystem mounted, all is ok. Then A times out, unmounts, and later wants to remount B's filesystem. Then, sometimes, I see a message much like yours.
It doesn't seem to need a reboot to cause this problem, and the fix I have found is to kill and restart the NFS server: /etc/init.d/nfs restart.
I have no idea why it happens, or why restarting nfsd or mountd fixes it.
-- Jamie
Quoted 9 lines · Michael Rothwell
Quoted 9 lines
From: "Neil Brown" <[email protected]>
> This is not allowed, and makes no sense.
It apparently is (or was, anyway) allowed, because it worked until the server was rebooted. Fluke?
Are you saying that, if I export "/export", I can mount "/export/home" from a client machine? That's nice.
Yes. At least that's how Solaris, Irix, HP/UX, and several others do it. The general rule is something along the lines of "if /a is exported, anything that is a subdirectory of /a is accessible via NFS, so long as it is on the same device (partition - actually major/minor pair)". So, if /export/home in your case lives in the same file system as /export, it is accessible via NFS as well. Now if /export/home is it's own mount point (i.e. a separate file system), then it needs to be exported explicitly.
Quoted 5 lines
Quoted 5 lines
> Simply remove the second line and your problems should go away.
Thanks. I actually switched to two explicit exports; /export/files and /export/home, which works.
This works as well, with the caveat that you cannot then mount /export itself on client machines.
tw
Signature · 1 line
[email protected]
Quoted 2 lines · Michael Rothwell
Just wondering if there's been any talk, plans, etc. of an alternative for NFS.
there's coda.
Signature · 3 lines
J o h a n K u l l s t a m [[email protected]] Don't Fear the Penguin!
Quoted 6 lines
Quoted 2 lines · Michael Rothwell
Just wondering if there's been any talk, plans, etc. of an alternative for NFS.
there's coda.
You probably want inter-mezzo rather than coda for that kind of file service. Coda is a research project, inter-mezzo is kind of "distilled coda and afs", making it a real fs. http://www.inter-mezzo.org
Quoted 10 lines · Alan Cox
Quoted 6 lines
Quoted 2 lines · Michael Rothwell
Just wondering if there's been any talk, plans, etc. of an alternative for NFS.
there's coda.
You probably want inter-mezzo rather than coda for that kind of file service. Coda is a research project, inter-mezzo is kind of "distilled coda and afs", making it a real fs. http://www.inter-mezzo.org
And it currently looks "Linux only", which kind of defeats IMHO the purpose.
In a heterogenous environment there is NFS (and SMB aka CIFS aka <name of the day>, of course. The horrors! The horrors! :-) ) and nothing much else.
Regards
Henning
Signature · 5 lines
Dipl.-Inf. (Univ.) Henning P. Schmiedehausen -- Geschaeftsfuehrer INTERMETA - Gesellschaft fuer Mehrwertdienste mbH [email protected] Am Schwabachgrund 22 Fon.: 09131 / 50654-0 [email protected] D-91054 Buckenhof Fax.: 09131 / 50654-20
Quoted 2 lines · Henning P. Schmiedehausen
And it currently looks "Linux only", which kind of defeats IMHO the purpose.
Considering I use only Linux at home, that's fine with me. :) For Windows interaction, I use samba. I have not even booted my Sparc into Solaris in ... oh, about a year.
Is intermezzo usable? Reliable? Any horror stories? Success stories?
-M
Quoted 3 lines · [email protected] (Neil Brown) writes:
SUNs "cachefs" concept can be used to improve read performance by caching a lot more of server-data on the client. That could be implemented for Linux, but I don't know of anyone with serious plans.
cachefs sucks. It doesn't seem to cache stat(2) information. Doing ls -F in a ~100-entries directory takes several seconds over a link with 50ms round-trip time.
//Marcus
Signature · 4 lines
---------------------------------+---------------------------------
Marcus Sundberg | Phone: +46 707 452062
Embedded Systems Consultant | Email: [email protected]
Cendio Systems AB | http://www.cendio.comQuoted 9 lines · Neil Brown
Quoted 9 lines
I'm seeing this message quite often with one Linux 2.4.7 system automounting another. As long as A has B's filesystem mounted, all is ok. Then A times out, unmounts, and later wants to remount B's filesystem. Then, sometimes, I see a message much like yours.
It doesn't seem to need a reboot to cause this problem, and the fix I have found is to kill and restart the NFS server:
/etc/init.d/nfsrestart.
Quoted 8 lines
Quoted 1 line
I have no idea why it happens, or why restarting nfsd or mountd fixes it.
Show me your
/etc/exports....If you export a directory and a subdirectory of that directory - both on the same filesysem, you can get this.
If you export a directory to both an IP address (or subnet) and a hostname (or wildcard or netgroup) this can also happen.
Yes, I'm doing the second of those. No alternative -- I need write access from some hosts, and read access to all the rest (who are dynamically allocated) on the subnet.
It's clearly a bug in the NFS server then.
-- Jamie
/etc/exports:
# I want ro granted to all hosts in the 172.30.* subnet, but rw
# granted to the machines in @aquarius_outside which is within that # subnet. Doesn't work, whichever order I write the lines. # (The machines in @aquarius_hosts always get read only access). # # Solution, though I don't like it: write the host names explicitly. # # The 192.168.64.192/19 subnetwork is for temporarily assigned IPs on # the Aquarius test network. /kickstart is used for kickstart # installations on machines plugged in temporarily.
/home 172.30.0.0/16(ro,root_squash) \ galatea.degree2.com(rw,root_squash) \ ariel.degree2.com(rw,root_squash) \ @aquarius_hosts(rw,root_squash) \ @aquarius_outside(rw,root_squash)
/kickstart 172.30.0.0/16(ro,root_squash) \ @aquarius_hosts(ro,root_squash) \ @aquarius_outside(ro,root_squash) \ 192.168.64.192/255.255.255.224(ro,root_squash)
Quoted 8 lines · [email protected] (Neil Brown) writes:
Quoted 3 lines · On September 10, [email protected] wrote:
cachefs sucks. It doesn't seem to cache stat(2) information. Doing ls -F in a ~100-entries directory takes several seconds over a link with 50ms round-trip time.
Well, I said "concept" not "implementation", but I suspect that Solaris cachefs does cache stat information. Maybe you just need to increase the timeouts for the attribute cache.
Considering that I did several ls'es on the order of milliseconds apart I doubt that would help...
//Marcus
Signature · 4 lines
---------------------------------+---------------------------------
Marcus Sundberg | Phone: +46 707 452062
Embedded Systems Consultant | Email: [email protected]
Cendio Systems AB | http://www.cendio.comQuoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
" " == Marcus Sundberg <[email protected]> writes:
> [email protected] (Neil Brown) writes:
>> On September 10, [email protected] wrote:
>> > cachefs sucks. It doesn't seem to cache stat(2) information. >> > Doing ls -F in a ~100-entries directory takes several seconds >> > over a link with 50ms round-trip time.
>>
>> Well, I said "concept" not "implementation", but I suspect that >> Solaris cachefs does cache stat information. Maybe you just >> need to increase the timeouts for the attribute cache.
> Considering that I did several ls'es on the order of > milliseconds apart I doubt that would help...
The NFS close-to-open cache consistency requirement forces them to compare the attribute cache to the server every time someone does a call to open(). This is true whether or not one uses cachefs.
After the file has actually been opened, you can call fstat() as many times as you like. The cached attributes to be used, and they will then be checked on the server only after the cache times out.
Cheers,
Trond