Memory leaks in rpc.mountd
Subsystems (2)
Hi all,
I'm currently experiencing some large memory leaks on an NFS server when copying files to it from an NFS client. This happens for both NFSv3 and NFSv4, am currently using an NFSv4 setup.
Installed software and kernel versions below...
SERVER:
Kernel 2.6.32.9CONFIG_NFS_FS=y
# CONFIG_NFS_V3 is not setCONFIG_NFS_V4=y
# CONFIG_NFS_V4_1 is not set
# CONFIG_ROOT_NFS is not setCONFIG_NFSD=yCONFIG_NFSD_V3=y
# CONFIG_NFSD_V3_ACL is not setCONFIG_NFSD_V4=yCONFIG_NFS_COMMON=y
libtirpc-0.2.0
rpcbind-0.2.0
nfs-utils-1.2.2
CLIENT:
Kernel 2.6.34CONFIG_NFS_FS=y
# CONFIG_NFS_V3 is not setCONFIG_NFS_V4=y
# CONFIG_NFS_V4_1 is not setCONFIG_NFSD=yCONFIG_NFSD_V3=y
# CONFIG_NFSD_V3_ACL is not setCONFIG_NFSD_V4=yCONFIG_NFS_COMMON=y
libtirpc-0.2.0
rpcbind-0.2.0
nfs-utils-1.2.2
Output of kmemleak report...
unreferenced object 0xf4b2ec60 (size 32):
comm "rpc.mountd", pid 1814, jiffies 4294759177
hex dump (first 32 bytes):
ab 82 c0 3b c4 54 4f e9 a5 c9 ee db 43 71 fc 38 ...;.TO.....Cq.8
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
backtrace:
[<c12e7d79>] kmemleak_alloc+0x59/0xc0
[<c106ac5a>] __kmalloc+0xfa/0x110
[<c105787f>] kmemdup+0x1f/0x70
[<c113dfb1>] svc_export_parse+0x621/0x770
[<c12db79d>] cache_do_downcall+0x2d/0x40
[<c12dc2f1>] cache_write+0x91/0xe0
[<c12dc392>] cache_write_procfs+0x22/0x30
[<c10a6e34>] proc_reg_write+0x44/0x60
[<c106fbcc>] vfs_write+0x9c/0x160
[<c106fd4d>] sys_write+0x3d/0x70
[<c1002c48>] sysenter_do_call+0x12/0x26
[<ffffffff>] 0xffffffffThe test case is copying a 1.5GB file from the nfs client to the server.
It literally takes only a minute for the server's 1.2GB of memory to be almost totally exhausted down to around 5MB. The copying process hangs, then resumes many times along the way until it does successfully finish around 4 minutes later. The consumed memory is then not freed and the server's memory remains depleted to almost nothing.
The memory is never completely exhausted so it never starts to swap ?
Reading from server works fine with no memory loss, albeit a much slower transfer rate.
This started happening after a kernel upgrade on the server from 2.6.20.7 to 2.6.29.5, but is still occurring with the latest kernels. Hope the above info helps, I'm willing to try any patches that may get thrown my way.
Cheers,
Rick
Quoted 38 lines · Rick Harris
Hi all,
I'm currently experiencing some large memory leaks on an NFS server when copying files to it from an NFS client.
This happens for both NFSv3 and NFSv4, am currently using an NFSv4 setup. Installed software and kernel versions below...SERVER:
Kernel 2.6.32.9CONFIG_NFS_FS=y
#CONFIG_NFS_V3is not setCONFIG_NFS_V4=y
#CONFIG_NFS_V4_1is not set
#CONFIG_ROOT_NFSis not setCONFIG_NFSD=yCONFIG_NFSD_V3=y
#CONFIG_NFSD_V3_ACLis not setCONFIG_NFSD_V4=yCONFIG_NFS_COMMON=ylibtirpc-0.2.0
rpcbind-0.2.0
nfs-utils-1.2.2CLIENT:
Kernel 2.6.34CONFIG_NFS_FS=y
#CONFIG_NFS_V3is not setCONFIG_NFS_V4=y
#CONFIG_NFS_V4_1is not setCONFIG_NFSD=yCONFIG_NFSD_V3=y
#CONFIG_NFSD_V3_ACLis not setCONFIG_NFSD_V4=yCONFIG_NFS_COMMON=y
Quoted 8 lines
libtirpc-0.2.0
rpcbind-0.2.0
nfs-utils-1.2.2Output of kmemleak report...
unreferenced object
0xf4b2ec60(size 32):
If I understand right, it's saying that it only found the one unreferenced object? svc_export_parse should be called relatively rarely, so it's unlikely to explain your symptoms anyway.
Quoted 19 lines
comm "rpc.mountd", pid 1814, jiffies 4294759177
hex dump (first 32 bytes):ab 82 c0 3b c4 54 4f e9 a5 c9 ee db 43 71 fc 38 ...;.TO.....Cq.8 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ backtrace: [<c12e7d79>] kmemleak_alloc+0x59/0xc0 [<c106ac5a>] __kmalloc+0xfa/0x110 [<c105787f>] kmemdup+0x1f/0x70 [<c113dfb1>] svc_export_parse+0x621/0x770 [<c12db79d>] cache_do_downcall+0x2d/0x40 [<c12dc2f1>] cache_write+0x91/0xe0 [<c12dc392>] cache_write_procfs+0x22/0x30 [<c10a6e34>] proc_reg_write+0x44/0x60 [<c106fbcc>] vfs_write+0x9c/0x160 [<c106fd4d>] sys_write+0x3d/0x70 [<c1002c48>] sysenter_do_call+0x12/0x26 [<ffffffff>] 0xffffffff
Quoted 3 lines
The test case is copying a 1.5GB file from the nfs client to the server. It literally takes only a minute for the server's 1.2GB of memory to be almost totally exhausted down to around 5MB.
How are you measuring the amount of free memory?
Quoted 4 lines
The copying process hangs, then resumes many times along the way until it does successfully finish around 4 minutes later. The consumed memory is then not freed and the server's memory remains depleted to almost nothing.
What kind of drive are you exporting, over what network?
--b.
Quoted 18 lines
The memory is never completely exhausted so it never starts to swap ?
Reading from server works fine with no memory loss, albeit a much slower transfer rate.
This started happening after a kernel upgrade on the server from 2.6.20.7 to 2.6.29.5, but is still occurring with the latest kernels.
Hope the above info helps, I'm willing to try any patches that may get thrown my way.Cheers,
Rick
--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html