2.6.2: fsync() over NFS hanging unless other I/O going on
[Cc: to linux-kernel because this might be more general than just NFS]
I've been able to reproduce the problem I've been talking about before - fsync() over NFS hanging. The weird thing is that it only happens on an otherwise complete idle system.
Server and client are both PIV 3.0 Ghz machines, 1 GB RAM running 2.6.2. Client mounts the server with
mount -o nfsvers=3,rsize=32768,wsize=32768 jenny-eth1:/export/swr /mnt
I have a "dd" like app that writes a file over NFS then fsync()s it. The fsync() hangs in the kernel, in fs/nfs/write.c:nfs_sync_file() for up to half a minute, calling nfs_wait_for_requests().
I added some debug code to nfs_sync_file(), which shows the hang (13:03:35 -> 13:04:02) :
Feb 6 13:03:35 meghan kernel: ENTER nfs_sync_file idx_start=0 npages=0 how=2 Feb 6 13:03:35 meghan kernel: Call Trace: Feb 6 13:03:35 meghan kernel: [<f8a06404>] nfs_sync_file+0x48/0x153 [nfs] Feb 6 13:03:35 meghan kernel: [<f8a047de>] nfs_writepages+0xe9/0xf0 [nfs] Feb 6 13:03:35 meghan kernel: [<c013e4bb>] do_writepages+0x1e/0x38 Feb 6 13:03:35 meghan kernel: [<c0138eb3>] __filemap_fdatawrite+0xb6/0xb8 Feb 6 13:03:35 meghan kernel: [<c0138ecc>] filemap_fdatawrite+0x17/0x1b Feb 6 13:03:35 meghan kernel: [<c0155cb5>] sys_fsync+0x85/0xcf Feb 6 13:03:35 meghan kernel: [<c010920f>] syscall_call+0x7/0xb Feb 6 13:03:35 meghan kernel: Feb 6 13:03:35 meghan kernel: nfs_sync_file: waiting for requests Feb 6 13:04:02 meghan kernel: nfs_sync_file: DONE waiting for requests error 7020 Feb 6 13:04:02 meghan kernel: nfs_sync_file: waiting for requests Feb 6 13:04:02 meghan kernel: nfs_sync_file: DONE waiting for requests error 0 Feb 6 13:04:02 meghan kernel: nfs_sync_file: flush file Feb 6 13:04:02 meghan kernel: nfs_sync_file: DONE flushing file error 0 Feb 6 13:04:02 meghan kernel: nfs_sync_file: commit file Feb 6 13:04:02 meghan kernel: nfs_sync_file: DONE committing file error 50000 Feb 6 13:04:02 meghan kernel: nfs_sync_file: waiting for requests Feb 6 13:04:04 meghan kernel: nfs_sync_file: DONE waiting for requests error 11924 Feb 6 13:04:04 meghan kernel: nfs_sync_file: waiting for requests Feb 6 13:04:04 meghan kernel: nfs_sync_file: DONE waiting for requests error 0 Feb 6 13:04:04 meghan kernel: nfs_sync_file: flush file Feb 6 13:04:04 meghan kernel: nfs_sync_file: DONE flushing file error 0 Feb 6 13:04:04 meghan kernel: nfs_sync_file: commit file Feb 6 13:04:04 meghan kernel: nfs_sync_file: DONE committing file error 0 Feb 6 13:04:04 meghan kernel: EXIT nfs_sync_file
Now the weird thing is, that if I start the following in another shell on the same server:
# cd /tmp # while :; do echo hoi > lala; done
.. then I cannot reproduce the hang. Note that that is just a write on a local filesystem and doesn't have anything to do with NFS at all ?
Mike.