umount fails with device busy

Hi guys,

I get the device busy error when I try to umount an NFS file system although there is no process with an open file on it (at least fuser -m /mnt/point and lsof +f -- /mnt/point don't find anything).

To be more precise this only happens, when I run the umount right after stopping an application which has its executables and work directory on this file system. Running umount again some seconds later works fine.

I do not have any submounts under this mount point. There are no packet transmit or receive errors nor any NFS/RPC errors only a very small number of RPC retransmitts. So it seems that I have a stable environment. Client and server is SLES9 with 2.6.5-7.202.7-smp/X86_64.

I enabled NFS debug output and the only noticeable thing is, that some NFS operations are still ongoing after the application processes have already terminated:

  Aug 23 08:25:23 marvin kernel: NFS reply getattr
  Aug 23 08:25:23 marvin kernel: NFS: nfs_update_inode(0:12/149296 ct=1 info=0x6)
  Aug 23 08:25:23 marvin kernel: NFS: (0:12/149296) revalidation complete
  Aug 23 08:25:23 marvin kernel: nfs: flush(0:12/149296)
  Aug 23 08:25:23 marvin kernel: NFS: dentry_delete(bin/stop, 0)
  Aug 23 08:25:23 marvin kernel: NFS: revalidating (0:12/2)
  Aug 23 08:25:23 marvin kernel: NFS call  getattr
  Aug 23 08:25:23 marvin kernel: NFS reply getattr
  Aug 23 08:25:23 marvin kernel: NFS: nfs_update_inode(0:12/2 ct=1 info=0x6)
  Aug 23 08:25:23 marvin kernel: NFS: (0:12/2) revalidation complete
  Aug 23 08:25:25 marvin kernel: NFS call  fsstat
  Aug 23 08:25:25 marvin kernel: NFS reply statfs: 0
  Aug 23 08:25:35 marvin kernel: NFS call  fsstat
  Aug 23 08:25:35 marvin kernel: NFS reply statfs: 0
  Aug 23 08:25:38 marvin kernel: NFS: 6368 initiated commit call
  Aug 23 08:25:38 marvin kernel: NFS: 6369 initiated commit call
  Aug 23 08:25:38 marvin kernel: NFS: 6370 initiated commit call
  Aug 23 08:25:38 marvin kernel: NFS: 6371 initiated commit call
  Aug 23 08:25:38 marvin kernel: NFS: 6368 nfs_commit_done (status 0)
  Aug 23 08:25:38 marvin kernel: NFS: commit (0:12/121805 3207@0)NFS: dentry_delete(log/mux_marvin.trc, 0)
  Aug 23 08:25:38 marvin kernel:  OK
  Aug 23 08:25:38 marvin kernel: NFS: 6369 nfs_commit_done (status 0)
  Aug 23 08:25:38 marvin kernel: NFS: commit (0:12/149304 2238@0)NFS: dentry_delete(log/pw_marvin_1.trc, 0)
  Aug 23 08:25:38 marvin kernel:  OK
  Aug 23 08:25:38 marvin kernel: NFS: 6370 nfs_commit_done (status 0)
  Aug 23 08:25:38 marvin kernel: NFS: commit (0:12/149297 2238@0)NFS: dentry_delete(log/pw_marvin_0.trc, 0)
  Aug 23 08:25:38 marvin kernel:  OK
  Aug 23 08:25:38 marvin kernel: NFS: 6371 nfs_commit_done (status 0)
  Aug 23 08:25:38 marvin kernel: NFS: commit (0:12/149295 965@0)NFS: dentry_delete(net.trc, 0)
  Aug 23 08:25:38 marvin kernel:  OK
  Aug 23 08:25:45 marvin kernel: NFS call  fsstat
  Aug 23 08:25:45 marvin kernel: NFS reply statfs: 0
  Aug 23 08:25:55 marvin kernel: NFS call  fsstat

When I run umount during these ops I get an device busy. Any help would be very appreciated!

Quoted 57 lines · [email protected] wrote:

Hi guys,

I get the device busy error when I try to umount an NFS file system although there is no process with an open file on it (at least fuser -m /mnt/point and lsof +f -- /mnt/point don't find anything).

To be more precise this only happens, when I run the umount right after stopping an application which has its executables and work directory on this file system. Running umount again some seconds later works fine. I do not have any submounts under this mount point. There are no packet transmit or receive errors nor any NFS/RPC errors only a very small number of RPC retransmitts. So it seems that I have a stable environment. Client and server is SLES9 with 2.6.5-7.202.7-smp/X86_64.
I enabled NFS debug output and the only noticeable thing is, that some NFS operations are still ongoing after the application processes have already terminated:

 Aug 23 08:25:23 marvin kernel: NFS reply getattr
 Aug 23 08:25:23 marvin kernel: NFS: nfs_update_inode(0:12/149296 ct=1 
 info=0x6)
 Aug 23 08:25:23 marvin kernel: NFS: (0:12/149296) revalidation complete
 Aug 23 08:25:23 marvin kernel: nfs: flush(0:12/149296)
 Aug 23 08:25:23 marvin kernel: NFS: dentry_delete(bin/stop, 0)
 Aug 23 08:25:23 marvin kernel: NFS: revalidating (0:12/2)
 Aug 23 08:25:23 marvin kernel: NFS call  getattr
 Aug 23 08:25:23 marvin kernel: NFS reply getattr
 Aug 23 08:25:23 marvin kernel: NFS: nfs_update_inode(0:12/2 ct=1 
 info=0x6)
 Aug 23 08:25:23 marvin kernel: NFS: (0:12/2) revalidation complete
 Aug 23 08:25:25 marvin kernel: NFS call  fsstat
 Aug 23 08:25:25 marvin kernel: NFS reply statfs: 0
 Aug 23 08:25:35 marvin kernel: NFS call  fsstat
 Aug 23 08:25:35 marvin kernel: NFS reply statfs: 0
 Aug 23 08:25:38 marvin kernel: NFS: 6368 initiated commit call
 Aug 23 08:25:38 marvin kernel: NFS: 6369 initiated commit call
 Aug 23 08:25:38 marvin kernel: NFS: 6370 initiated commit call
 Aug 23 08:25:38 marvin kernel: NFS: 6371 initiated commit call
 Aug 23 08:25:38 marvin kernel: NFS: 6368 nfs_commit_done (status 0)
 Aug 23 08:25:38 marvin kernel: NFS: commit (0:12/121805 3207@0)NFS: 
 dentry_delete(log/mux_marvin.trc, 0)
 Aug 23 08:25:38 marvin kernel:  OK
 Aug 23 08:25:38 marvin kernel: NFS: 6369 nfs_commit_done (status 0)
 Aug 23 08:25:38 marvin kernel: NFS: commit (0:12/149304 2238@0)NFS: 
 dentry_delete(log/pw_marvin_1.trc, 0)
 Aug 23 08:25:38 marvin kernel:  OK
 Aug 23 08:25:38 marvin kernel: NFS: 6370 nfs_commit_done (status 0)
 Aug 23 08:25:38 marvin kernel: NFS: commit (0:12/149297 2238@0)NFS: 
 dentry_delete(log/pw_marvin_0.trc, 0)
 Aug 23 08:25:38 marvin kernel:  OK
 Aug 23 08:25:38 marvin kernel: NFS: 6371 nfs_commit_done (status 0)
 Aug 23 08:25:38 marvin kernel: NFS: commit (0:12/149295 965@0)NFS: 
 dentry_delete(net.trc, 0)
 Aug 23 08:25:38 marvin kernel:  OK
 Aug 23 08:25:45 marvin kernel: NFS call  fsstat
 Aug 23 08:25:45 marvin kernel: NFS reply statfs: 0
 Aug 23 08:25:55 marvin kernel: NFS call  fsstat

Quoted 2 lines

When I run umount during these ops I get an device busy. Any help would be very appreciated!

This is normal and expected behavior. One problem may be that your server is slow, and thus there are RPCs left outstanding for a bit on your client after your application exits. The COMMIT calls from that trace suggest that there is dirty data the client is writing back to the server.

Quoted 1 line

This is normal and expected behavior. One problem may be that your server is slow, and thus there are RPCs left outstanding for a bit on your client after your application exits. The COMMIT calls from that trace suggest that there is dirty data the client is writing back to the server.

But I had expected that umount blocks until the commit calls are finished as it is with local file systems?

Quoted 5 lines · Zoot

Quoted 5 lines

This is normal and expected behavior. One problem may be that your server is slow, and thus there are RPCs left outstanding for a bit on your client after your application exits. The COMMIT calls from that trace suggest that there is dirty data the client is writing back to the server.

Quoted 2 lines

But I had expected that umount blocks until the commit calls are finished as it is with local file systems?

Sorry, it doesn't. It has no way of knowing how long those RPCs will take.

There is a "lazy unmount" facility that removes the NFS share from your client's name space and handles the outstanding RPCs in the background.
I'm not sure if it's working, but you could try "umount -l".

Quoted 59 lines · [email protected] wrote:

Hi guys,

I get the device busy error when I try to umount an NFS file system although there is no process with an open file on it (at least fuser -m /mnt/point and lsof +f -- /mnt/point don't find anything).

To be more precise this only happens, when I run the umount right after stopping an application which has its executables and work directory on this file system. Running umount again some seconds later works fine.
I do not have any submounts under this mount point. There are no packet transmit or receive errors nor any NFS/RPC errors only a very small number of RPC retransmitts. So it seems that I have a stable environment. Client and server is SLES9 with 2.6.5-7.202.7-smp/X86_64. I enabled NFS debug output and the only noticeable thing is, that some NFS operations are still ongoing after the application processes have already terminated:

  Aug 23 08:25:23 marvin kernel: NFS reply getattr
  Aug 23 08:25:23 marvin kernel: NFS: nfs_update_inode(0:12/149296 
ct=1 info=0x6)
  Aug 23 08:25:23 marvin kernel: NFS: (0:12/149296) revalidation 
complete
  Aug 23 08:25:23 marvin kernel: nfs: flush(0:12/149296)
  Aug 23 08:25:23 marvin kernel: NFS: dentry_delete(bin/stop, 0)
  Aug 23 08:25:23 marvin kernel: NFS: revalidating (0:12/2)
  Aug 23 08:25:23 marvin kernel: NFS call  getattr
  Aug 23 08:25:23 marvin kernel: NFS reply getattr
  Aug 23 08:25:23 marvin kernel: NFS: nfs_update_inode(0:12/2 ct=1 
info=0x6)
  Aug 23 08:25:23 marvin kernel: NFS: (0:12/2) revalidation complete
  Aug 23 08:25:25 marvin kernel: NFS call  fsstat
  Aug 23 08:25:25 marvin kernel: NFS reply statfs: 0
  Aug 23 08:25:35 marvin kernel: NFS call  fsstat
  Aug 23 08:25:35 marvin kernel: NFS reply statfs: 0
  Aug 23 08:25:38 marvin kernel: NFS: 6368 initiated commit call
  Aug 23 08:25:38 marvin kernel: NFS: 6369 initiated commit call
  Aug 23 08:25:38 marvin kernel: NFS: 6370 initiated commit call
  Aug 23 08:25:38 marvin kernel: NFS: 6371 initiated commit call
  Aug 23 08:25:38 marvin kernel: NFS: 6368 nfs_commit_done (status 0)
  Aug 23 08:25:38 marvin kernel: NFS: commit (0:12/121805 3207@0)NFS: 
dentry_delete(log/mux_marvin.trc, 0)
  Aug 23 08:25:38 marvin kernel:  OK
  Aug 23 08:25:38 marvin kernel: NFS: 6369 nfs_commit_done (status 0)
  Aug 23 08:25:38 marvin kernel: NFS: commit (0:12/149304 2238@0)NFS: 
dentry_delete(log/pw_marvin_1.trc, 0)
  Aug 23 08:25:38 marvin kernel:  OK
  Aug 23 08:25:38 marvin kernel: NFS: 6370 nfs_commit_done (status 0)
  Aug 23 08:25:38 marvin kernel: NFS: commit (0:12/149297 2238@0)NFS: 
dentry_delete(log/pw_marvin_0.trc, 0)
  Aug 23 08:25:38 marvin kernel:  OK
  Aug 23 08:25:38 marvin kernel: NFS: 6371 nfs_commit_done (status 0)
  Aug 23 08:25:38 marvin kernel: NFS: commit (0:12/149295 965@0)NFS: 
dentry_delete(net.trc, 0)
  Aug 23 08:25:38 marvin kernel:  OK
  Aug 23 08:25:45 marvin kernel: NFS call  fsstat
  Aug 23 08:25:45 marvin kernel: NFS reply statfs: 0
  Aug 23 08:25:55 marvin kernel: NFS call  fsstat

Quoted 2 lines

When I run umount during these ops I get an device busy. Any help would be very appreciated!

This is normal and expected behavior. One problem may be that your server is slow, and thus there are RPCs left outstanding for a bit on your client after your application exits. The COMMIT calls from that trace suggest that there is dirty data the client is writing back to the server.

Quoted 3 lines

Quoted 1 line

When I run umount during these ops I get an device busy. Any help would be very appreciated!

This is normal and expected behavior. One problem may be that your server is slow, and thus there are RPCs left outstanding for a bit on your client after your application exits. The COMMIT calls from that trace suggest that there is dirty data the client is writing back to the server.

It seems to me that this should not be the expected behavior unless the file system is mounted "nocto". Is it?

Quoted 11 lines

Quoted 8 lines

Quoted 2 lines

When I run umount during these ops I get an device busy. Any help would be very appreciated!

This is normal and expected behavior. One problem may be that your server is slow, and thus there are RPCs left outstanding for a bit on your client after your application exits. The COMMIT calls from that trace suggest that there is dirty data the client is writing back to the server.

It seems to me that this should not be the expected behavior unless the file system is mounted "nocto". Is it?

I'm a little puzzled myself about what dirty data there might be left after the application quits. However, I'll be there is an mmap() lurking somewhere in the background...

Data that was dirtied via a mapped file is not subject to the writeback part of close-to-open.

Quoted 7 lines · Chuck Lever

Quoted 6 lines

Quoted 3 lines

Quoted 1 line

When I run umount during these ops I get an device busy. Any help would be very appreciated!

This is normal and expected behavior. One problem may be that your server is slow, and thus there are RPCs left outstanding for a bit on your client after your application exits. The COMMIT calls from that trace suggest that there is dirty data the client is writing back to the server.

It seems to me that this should not be the expected behavior unless the file system is mounted "nocto". Is it?

Quoted 3 lines

I'm a little puzzled myself about what dirty data there might be left after the application quits. However, I'll be there is an mmap() lurking somewhere in the background...

Data that was dirtied via a mapped file is not subject to the writeback part of close-to-open.

I don't think that I agree with this last statement. Although it can not be implemented using the normal close system call, something should trigger the flush of any dirty pages and wait for them to complete.

Quoted 20 lines · Peter Staubach

Quoted 18 lines · Chuck Lever

Quoted 11 lines

Quoted 8 lines

Quoted 2 lines

When I run umount during these ops I get an device busy. Any help would be very appreciated!

This is normal and expected behavior. One problem may be that your server is slow, and thus there are RPCs left outstanding for a bit on your client after your application exits. The COMMIT calls from that trace suggest that there is dirty data the client is writing back to the server.

It seems to me that this should not be the expected behavior unless the file system is mounted "nocto". Is it?

I'm a little puzzled myself about what dirty data there might be left after the application quits. However, I'll be there is an mmap() lurking somewhere in the background...

Data that was dirtied via a mapped file is not subject to the writeback part of close-to-open.

Quoted 4 lines

I don't think that I agree with this last statement. Although it can not be implemented using the normal close system call, something should trigger the flush of any dirty pages and wait for them to complete.

This is the way Linux implements dirty mmaps on NFS because that's the way Linus wants it. I didn't mean to imply that's the way it *should* work, merely the way it *does* work in Linux.

Quoted 11 lines

Quoted 8 lines

Quoted 2 lines

When I run umount during these ops I get an device busy. Any help would be very appreciated!

This is normal and expected behavior. One problem may be that your server is slow, and thus there are RPCs left outstanding for a bit on your client after your application exits. The COMMIT calls from that trace suggest that there is dirty data the client is writing back to the server.

It seems to me that this should not be the expected behavior unless the file system is mounted "nocto". Is it?

I'm a little puzzled myself about what dirty data there might be left after the application quits. However, I'll be there is an mmap() lurking somewhere in the background...

Data that was dirtied via a mapped file is not subject to the writeback part of close-to-open.

Quoted 60 lines · Chuck Lever

Quoted 59 lines · [email protected] wrote:

Hi guys,

I get the device busy error when I try to umount an NFS file system although there is no process with an open file on it (at least fuser -m /mnt/point and lsof +f -- /mnt/point don't find anything).

To be more precise this only happens, when I run the umount right after stopping an application which has its executables and work directory on this file system. Running umount again some seconds later works fine.
I do not have any submounts under this mount point. There are no packet transmit or receive errors nor any NFS/RPC errors only a very small number of RPC retransmitts. So it seems that I have a stable environment. Client and server is SLES9 with 2.6.5-7.202.7-smp/X86_64. I enabled NFS debug output and the only noticeable thing is, that some NFS operations are still ongoing after the application processes have already terminated:

 Aug 23 08:25:23 marvin kernel: NFS reply getattr
 Aug 23 08:25:23 marvin kernel: NFS: nfs_update_inode(0:12/149296 
ct=1 info=0x6)
 Aug 23 08:25:23 marvin kernel: NFS: (0:12/149296) revalidation 
complete
 Aug 23 08:25:23 marvin kernel: nfs: flush(0:12/149296)
 Aug 23 08:25:23 marvin kernel: NFS: dentry_delete(bin/stop, 0)
 Aug 23 08:25:23 marvin kernel: NFS: revalidating (0:12/2)
 Aug 23 08:25:23 marvin kernel: NFS call  getattr
 Aug 23 08:25:23 marvin kernel: NFS reply getattr
 Aug 23 08:25:23 marvin kernel: NFS: nfs_update_inode(0:12/2 ct=1 
info=0x6)
 Aug 23 08:25:23 marvin kernel: NFS: (0:12/2) revalidation complete
 Aug 23 08:25:25 marvin kernel: NFS call  fsstat
 Aug 23 08:25:25 marvin kernel: NFS reply statfs: 0
 Aug 23 08:25:35 marvin kernel: NFS call  fsstat
 Aug 23 08:25:35 marvin kernel: NFS reply statfs: 0
 Aug 23 08:25:38 marvin kernel: NFS: 6368 initiated commit call
 Aug 23 08:25:38 marvin kernel: NFS: 6369 initiated commit call
 Aug 23 08:25:38 marvin kernel: NFS: 6370 initiated commit call
 Aug 23 08:25:38 marvin kernel: NFS: 6371 initiated commit call
 Aug 23 08:25:38 marvin kernel: NFS: 6368 nfs_commit_done (status 0)
 Aug 23 08:25:38 marvin kernel: NFS: commit (0:12/121805 3207@0)NFS: 
dentry_delete(log/mux_marvin.trc, 0)
 Aug 23 08:25:38 marvin kernel:  OK
 Aug 23 08:25:38 marvin kernel: NFS: 6369 nfs_commit_done (status 0)
 Aug 23 08:25:38 marvin kernel: NFS: commit (0:12/149304 2238@0)NFS: 
dentry_delete(log/pw_marvin_1.trc, 0)
 Aug 23 08:25:38 marvin kernel:  OK
 Aug 23 08:25:38 marvin kernel: NFS: 6370 nfs_commit_done (status 0)
 Aug 23 08:25:38 marvin kernel: NFS: commit (0:12/149297 2238@0)NFS: 
dentry_delete(log/pw_marvin_0.trc, 0)
 Aug 23 08:25:38 marvin kernel:  OK
 Aug 23 08:25:38 marvin kernel: NFS: 6371 nfs_commit_done (status 0)
 Aug 23 08:25:38 marvin kernel: NFS: commit (0:12/149295 965@0)NFS: 
dentry_delete(net.trc, 0)
 Aug 23 08:25:38 marvin kernel:  OK
 Aug 23 08:25:45 marvin kernel: NFS call  fsstat
 Aug 23 08:25:45 marvin kernel: NFS reply statfs: 0
 Aug 23 08:25:55 marvin kernel: NFS call  fsstat

Quoted 8 lines

Quoted 2 lines

When I run umount during these ops I get an device busy. Any help would be very appreciated!

This is normal and expected behavior. One problem may be that your server is slow, and thus there are RPCs left outstanding for a bit on your client after your application exits. The COMMIT calls from that trace suggest that there is dirty data the client is writing back to the server.

It seems to me that this should not be the expected behavior unless the file system is mounted "nocto". Is it?

   Thanx...

      ps

Quoted 72 lines · Peter Staubach

Quoted 70 lines · Chuck Lever

Quoted 59 lines · [email protected] wrote:

Hi guys,

I get the device busy error when I try to umount an NFS file system although there is no process with an open file on it (at least fuser -m /mnt/point and lsof +f -- /mnt/point don't find anything).

To be more precise this only happens, when I run the umount right after stopping an application which has its executables and work directory on this file system. Running umount again some seconds later works fine.
I do not have any submounts under this mount point. There are no packet transmit or receive errors nor any NFS/RPC errors only a very small number of RPC retransmitts. So it seems that I have a stable environment. Client and server is SLES9 with 2.6.5-7.202.7-smp/X86_64. I enabled NFS debug output and the only noticeable thing is, that some NFS operations are still ongoing after the application processes have already terminated:

 Aug 23 08:25:23 marvin kernel: NFS reply getattr
 Aug 23 08:25:23 marvin kernel: NFS: nfs_update_inode(0:12/149296 
ct=1 info=0x6)
 Aug 23 08:25:23 marvin kernel: NFS: (0:12/149296) revalidation 
complete
 Aug 23 08:25:23 marvin kernel: nfs: flush(0:12/149296)
 Aug 23 08:25:23 marvin kernel: NFS: dentry_delete(bin/stop, 0)
 Aug 23 08:25:23 marvin kernel: NFS: revalidating (0:12/2)
 Aug 23 08:25:23 marvin kernel: NFS call  getattr
 Aug 23 08:25:23 marvin kernel: NFS reply getattr
 Aug 23 08:25:23 marvin kernel: NFS: nfs_update_inode(0:12/2 ct=1 
info=0x6)
 Aug 23 08:25:23 marvin kernel: NFS: (0:12/2) revalidation complete
 Aug 23 08:25:25 marvin kernel: NFS call  fsstat
 Aug 23 08:25:25 marvin kernel: NFS reply statfs: 0
 Aug 23 08:25:35 marvin kernel: NFS call  fsstat
 Aug 23 08:25:35 marvin kernel: NFS reply statfs: 0
 Aug 23 08:25:38 marvin kernel: NFS: 6368 initiated commit call
 Aug 23 08:25:38 marvin kernel: NFS: 6369 initiated commit call
 Aug 23 08:25:38 marvin kernel: NFS: 6370 initiated commit call
 Aug 23 08:25:38 marvin kernel: NFS: 6371 initiated commit call
 Aug 23 08:25:38 marvin kernel: NFS: 6368 nfs_commit_done (status 0)
 Aug 23 08:25:38 marvin kernel: NFS: commit (0:12/121805 3207@0)NFS: 
dentry_delete(log/mux_marvin.trc, 0)
 Aug 23 08:25:38 marvin kernel:  OK
 Aug 23 08:25:38 marvin kernel: NFS: 6369 nfs_commit_done (status 0)
 Aug 23 08:25:38 marvin kernel: NFS: commit (0:12/149304 2238@0)NFS: 
dentry_delete(log/pw_marvin_1.trc, 0)
 Aug 23 08:25:38 marvin kernel:  OK
 Aug 23 08:25:38 marvin kernel: NFS: 6370 nfs_commit_done (status 0)
 Aug 23 08:25:38 marvin kernel: NFS: commit (0:12/149297 2238@0)NFS: 
dentry_delete(log/pw_marvin_0.trc, 0)
 Aug 23 08:25:38 marvin kernel:  OK
 Aug 23 08:25:38 marvin kernel: NFS: 6371 nfs_commit_done (status 0)
 Aug 23 08:25:38 marvin kernel: NFS: commit (0:12/149295 965@0)NFS: 
dentry_delete(net.trc, 0)
 Aug 23 08:25:38 marvin kernel:  OK
 Aug 23 08:25:45 marvin kernel: NFS call  fsstat
 Aug 23 08:25:45 marvin kernel: NFS reply statfs: 0
 Aug 23 08:25:55 marvin kernel: NFS call  fsstat

Quoted 3 lines

When I run umount during these ops I get an device busy. Any help would be very appreciated!

This is normal and expected behavior. One problem may be that your server is slow, and thus there are RPCs left outstanding for a bit on your client after your application exits. The COMMIT calls from that trace suggest that there is dirty data the client is writing back to the server.

Quoted 7 lines

It seems to me that this should not be the expected behavior unless the file system is mounted "nocto". Is it?

   Thanx...

      ps

No, mount options are: nfsvers=3,soft,tcp,rsize=32768,wsize=32768

Quoted 81 lines · Zoot

Quoted 79 lines · Peter Staubach

Quoted 61 lines · Chuck Lever

Quoted 59 lines · [email protected] wrote:

Hi guys,

I get the device busy error when I try to umount an NFS file system although there is no process with an open file on it (at least fuser -m /mnt/point and lsof +f -- /mnt/point don't find anything).

To be more precise this only happens, when I run the umount right after stopping an application which has its executables and work directory on this file system. Running umount again some seconds later works fine.
I do not have any submounts under this mount point. There are no packet transmit or receive errors nor any NFS/RPC errors only a very small number of RPC retransmitts. So it seems that I have a stable environment. Client and server is SLES9 with 2.6.5-7.202.7-smp/X86_64. I enabled NFS debug output and the only noticeable thing is, that some NFS operations are still ongoing after the application processes have already terminated:

Aug 23 08:25:23 marvin kernel: NFS reply getattr
Aug 23 08:25:23 marvin kernel: NFS: nfs_update_inode(0:12/149296 
ct=1 info=0x6)
Aug 23 08:25:23 marvin kernel: NFS: (0:12/149296) revalidation 
complete
Aug 23 08:25:23 marvin kernel: nfs: flush(0:12/149296)
Aug 23 08:25:23 marvin kernel: NFS: dentry_delete(bin/stop, 0)
Aug 23 08:25:23 marvin kernel: NFS: revalidating (0:12/2)
Aug 23 08:25:23 marvin kernel: NFS call  getattr
Aug 23 08:25:23 marvin kernel: NFS reply getattr
Aug 23 08:25:23 marvin kernel: NFS: nfs_update_inode(0:12/2 ct=1 
info=0x6)
Aug 23 08:25:23 marvin kernel: NFS: (0:12/2) revalidation complete
Aug 23 08:25:25 marvin kernel: NFS call  fsstat
Aug 23 08:25:25 marvin kernel: NFS reply statfs: 0
Aug 23 08:25:35 marvin kernel: NFS call  fsstat
Aug 23 08:25:35 marvin kernel: NFS reply statfs: 0
Aug 23 08:25:38 marvin kernel: NFS: 6368 initiated commit call
Aug 23 08:25:38 marvin kernel: NFS: 6369 initiated commit call
Aug 23 08:25:38 marvin kernel: NFS: 6370 initiated commit call
Aug 23 08:25:38 marvin kernel: NFS: 6371 initiated commit call
Aug 23 08:25:38 marvin kernel: NFS: 6368 nfs_commit_done (status 0)
Aug 23 08:25:38 marvin kernel: NFS: commit (0:12/121805 3207@0)NFS: 
dentry_delete(log/mux_marvin.trc, 0)
Aug 23 08:25:38 marvin kernel:  OK
Aug 23 08:25:38 marvin kernel: NFS: 6369 nfs_commit_done (status 0)
Aug 23 08:25:38 marvin kernel: NFS: commit (0:12/149304 2238@0)NFS: 
dentry_delete(log/pw_marvin_1.trc, 0)
Aug 23 08:25:38 marvin kernel:  OK
Aug 23 08:25:38 marvin kernel: NFS: 6370 nfs_commit_done (status 0)
Aug 23 08:25:38 marvin kernel: NFS: commit (0:12/149297 2238@0)NFS: 
dentry_delete(log/pw_marvin_0.trc, 0)
Aug 23 08:25:38 marvin kernel:  OK
Aug 23 08:25:38 marvin kernel: NFS: 6371 nfs_commit_done (status 0)
Aug 23 08:25:38 marvin kernel: NFS: commit (0:12/149295 965@0)NFS: 
dentry_delete(net.trc, 0)
Aug 23 08:25:38 marvin kernel:  OK
Aug 23 08:25:45 marvin kernel: NFS call  fsstat
Aug 23 08:25:45 marvin kernel: NFS reply statfs: 0
Aug 23 08:25:55 marvin kernel: NFS call  fsstat

Quoted 9 lines

Quoted 3 lines

When I run umount during these ops I get an device busy. Any help would be very appreciated!

This is normal and expected behavior. One problem may be that your server is slow, and thus there are RPCs left outstanding for a bit on your client after your application exits. The COMMIT calls from that trace suggest that there is dirty data the client is writing back to the server.

It seems to me that this should not be the expected behavior unless the file system is mounted "nocto". Is it?

Thanx...

ps

Quoted 2 lines

No, mount options are: nfsvers=3,soft,tcp,rsize=32768,wsize=32768

"soft"?

Dangerous.

ps