Skip to contentSkip to conversation

Stale NFS file handle

DISCUSSION Filipe Brandenburger 3 participants 5 messages Mar 23, 2005

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

Hello,

I have a problem where I'm getting "Stale NFS file handle" errors when a file is updated. I can easily reproduce the problem if I run a sequence of commands in two different hosts.

My environment is:

1) Server: Netapp FAS940
2) Client 1: Linux RedHat 9 with kernel 2.4.21-4.ELsmp (kernel of RHAS3)
3) Client 2: exactly the same as client 1.

The file system is mounted on both clients with the options rsize=8192,wsize=8192,timeo=28,intr, additionally it's mounted read-only on client 2 (it also gives me stale file handle if it's mounted read-write, so it doesn't really matter).

My test setup is:

On client 2, I setup a loop to read a file:

# while :; do cat test.txt; done >/dev/null

Then, on client 1, I create a new file and rename it over the original file:

# date >new.txt; mv -f new.txt test.txt

Whenever I execute this on client 1, I get the following error message on client 2:

cat: test.txt: Stale NFS file handle

Why is this happening? Is there a way to fix this problem? I tried the mount options "noac" and "nocto" on client 2, and used "mount -o remount" on it, after that the output of "mount" returned these options, and it didn't solve the issue.

Is there a way to solve this issue without changing applications that access this file? Because although my test environment consists of only "cat" and "mv", my real production environment is of proprietary applications, that are harder to fix, "cat" and "mv" was only the way I used to reproduce the problem in a controlled environment...

Thanks a lot,
Filipe

-------------------------------------------------------
This SF.net email is sponsored by: 2005 Windows Mobile Application Contest Submit applications for Windows Mobile(tm)-based Pocket PCs or Smartphones for the chance to win $25,000 and application distribution. Enter today at
http://ads.osdn.com/?ad_id=6882&alloc_id=15148&op=click
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs

#1 lore raw
Quoted 9 lines · Filipe Brandenburger

I have a problem where I'm getting "Stale NFS file handle" errors when a file is updated. I can easily reproduce the problem if I run a sequence of commands in two different hosts.

My environment is:

1) Server: Netapp FAS940
2) Client 1: Linux RedHat 9 with kernel 2.4.21-4.ELsmp (kernel of RHAS3)
3) Client 2: exactly the same as client 1.

Sometime back there was an Netapp issue that was causing ESTALES with mostly 2.4 kernels... I upgraded the OS on our toater and the problem when away....

steved.

-------------------------------------------------------
This SF.net email is sponsored by: 2005 Windows Mobile Application Contest Submit applications for Windows Mobile(tm)-based Pocket PCs or Smartphones for the chance to win $25,000 and application distribution. Enter today at
http://ads.osdn.com/?ad_id=6882&alloc_id=15148&op=click
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs

#2 lore raw

on den 23.03.2005 Klokka 08:12 (-0500) skreiv Steve Dickson:

Quoted 5 lines
Quoted 4 lines · Filipe Brandenburger

I have a problem where I'm getting "Stale NFS file handle" errors when a file is updated. I can easily reproduce the problem if I run a sequence of commands in two different hosts.

Quoted 8 lines
Quoted 5 lines

My environment is:

1) Server: Netapp FAS940
2) Client 1: Linux RedHat 9 with kernel 2.4.21-4.ELsmp (kernel of RHAS3)
3) Client 2: exactly the same as client 1.

Sometime back there was an Netapp issue that was causing ESTALES with mostly 2.4 kernels... I upgraded the OS on our toater and the problem when away....

... yeah, but this was a pretty obvious case of user error.

He was running

while :; do cat test.txt; done >/dev/null

on a client, then deleting the file on the server. Even if the call to open() is successful, you both can and will get ESTALEs on the subsequent call to read().

Cheers,
Trond

Signature · 12 lines
Trond Myklebust <[email protected]>



-------------------------------------------------------
This SF.net email is sponsored by: 2005 Windows Mobile Application Contest
Submit applications for Windows Mobile(tm)-based Pocket PCs or Smartphones
for the chance to win $25,000 and application distribution. Enter today at
http://ads.osdn.com/?ad_id=6882&alloc_id=15148&op=click
_______________________________________________
NFS maillist  -  [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs
#3 lore raw

* Wed, 23 Mar 2005 08:57:15 -0500, Trond Myklebust <[email protected]>:

Quoted 7 lines

He was running

while :; do cat test.txt; done >/dev/null

on a client, then deleting the file on the server. Even if the call to open() is successful, you both can and will get ESTALEs on the subsequent call to read().

Ok,

But then, how do you suggest I should change applications to do it? The applications that publish content to the NFS run on one host and are based on rsync, the applications that deliver content are web servers (Apache) reading from this same NFS on another pool of hosts (these are the ones that get the ESTALE error).

Where is the problem? On the applications that publish? Should they open the file and update it in-place instead of creating a new one and renaming? I don't think so! This would lead to content that is a mix of the old and the new, that is corrupt.

Or is the webserver? Should the application protect itself from ESTALE errors and retry? Somehow that seems wrong to me also. Then I would have to change all applications that read this content to do it. Why doesn't the NFS client recover from this kind of errors?

If it's really not possible to change it on the NFS client (the kernel), what workaround would you suggest me to use?

Thanks,
Filipe

-------------------------------------------------------
This SF.net email is sponsored by Microsoft Mobile & Embedded DevCon 2005 Attend MEDC 2005 May 9-12 in Vegas. Learn more about the latest Windows Embedded(r) & Windows Mobile(tm) platforms, applications & content. Register by 3/29 & save $300 http://ads.osdn.com/?ad_id=6883&alloc_id=15149&op=click
_______________________________________________
NFS maillist - [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs

#4 lore raw

on den 23.03.2005 Klokka 14:15 (-0300) skreiv Filipe Brandenburger:

Quoted 2 lines

If it's really not possible to change it on the NFS client (the kernel), what workaround would you suggest me to use?

Rename the file, then delete it once you know that the clients no longer have it open. That's the obvious and standard way of dealing with this sort of problem over NFS.

Cheers,
Trond

Signature · 12 lines
Trond Myklebust <[email protected]>



-------------------------------------------------------
This SF.net email is sponsored by Microsoft Mobile & Embedded DevCon 2005
Attend MEDC 2005 May 9-12 in Vegas. Learn more about the latest Windows
Embedded(r) & Windows Mobile(tm) platforms, applications & content.  Register
by 3/29 & save $300 http://ads.osdn.com/?ad_id=6883&alloc_id=15149&op=click
_______________________________________________
NFS maillist  -  [email protected]
https://lists.sourceforge.net/lists/listinfo/nfs