nfs: init req_lock in nfs_alloc_inode
PATCH 1/1 nfs: init req_lock in nfs_alloc_inode
Seems like req_lock is never initialized. CONFIG_DEBUG_SPINLOCK reported:
BUG: spinlock bad magic on CPU#0, mount/1073 lock: c00000007fdca108, .magic: 00000000, .owner: /24576, .owner_cpu: 0 Call Trace: [C00000007E913750] [C0000000000107B4] .show_stack+0x54/0x1f0 (unreliable) [C00000007E913800] [C0000000001D2234] .spin_bug+0xa4/0x120 [C00000007E913890] [C0000000001D247C] ._raw_spin_lock+0xdc/0x1d0 [C00000007E913930] [C0000000004DF990] ._spin_lock+0x10/0x30 [C00000007E9139B0] [C00000000017623C] .nfs_sync_mapping_wait+0xac/0x310 [C00000007E913AA0] [C000000000176F18] .nfs_sync_mapping_range+0x98/0x110 [C00000007E913B80] [C00000000016CAA0] .nfs_getattr+0x40/0xf0 [C00000007E913C20] [C0000000000BE668] .vfs_getattr+0x38/0x70 [C00000007E913CA0] [C0000000000BF05C] .vfs_fstat+0x3c/0x70 [C00000007E913D30] [C0000000000BF0B8] .sys_fstat64+0x28/0x60 [C00000007E913E30] [C000000000008608] syscall_exit+0x0/0x40
---
Trond, is your MAINTAINERS entry up to date? Seems like you mostly post from @netapp.com these days.
fs/nfs/inode.c +1 -0
@@ -1123,6 +1123,7 @@ struct inode *nfs_alloc_inode(struct sup nfsi->flags =0UL ; nfsi->cache_validity =0UL ; nfsi->cache_change_attribute = jiffies; + nfsi->req_lock =SPIN_LOCK_UNLOCKED ;#ifdef CONFIG_NFS_V3_ACL nfsi->acl_access =ERR_PTR (-EAGAIN ); nfsi->acl_default =ERR_PTR (-EAGAIN );
fs/nfs/inode.c struct inode *nfs_alloc_inode(struct sup · line 1126
Quoted 1 line · On Fri, 16 Feb 2007 11:05:32 -0600 [email protected] (Olof Johansson) wrote:
fs/nfs/inode.c +1
-0+ nfsi->req_lock =SPIN_LOCK_UNLOCKED ;
This can confound lockdep. Please use spin_lock_init().
Quoted 31 lines · On Fri, 16 Feb 2007 11:05:32 -0600 [email protected] (Olof Johansson) wrote:
Seems like
req_lockis never initialized.CONFIG_DEBUG_SPINLOCKreported:BUG: spinlock bad magic on CPU#0, mount/1073 lock: c00000007fdca108, .magic: 00000000, .owner: /24576, .owner_cpu: 0 Call Trace: [C00000007E913750] [C0000000000107B4] .show_stack+0x54/0x1f0 (unreliable) [C00000007E913800] [C0000000001D2234] .spin_bug+0xa4/0x120 [C00000007E913890] [C0000000001D247C] ._raw_spin_lock+0xdc/0x1d0 [C00000007E913930] [C0000000004DF990] ._spin_lock+0x10/0x30 [C00000007E9139B0] [C00000000017623C] .nfs_sync_mapping_wait+0xac/0x310 [C00000007E913AA0] [C000000000176F18] .nfs_sync_mapping_range+0x98/0x110 [C00000007E913B80] [C00000000016CAA0] .nfs_getattr+0x40/0xf0 [C00000007E913C20] [C0000000000BE668] .vfs_getattr+0x38/0x70 [C00000007E913CA0] [C0000000000BF05C] .vfs_fstat+0x3c/0x70 [C00000007E913D30] [C0000000000BF0B8] .sys_fstat64+0x28/0x60 [C00000007E913E30] [C000000000008608] syscall_exit+0x0/0x40Signed-off-by Olof Johansson <[email protected]>---
Trond, is your
MAINTAINERSentry up to date? Seems like you mostly post from @netapp.com these days.Index:
linux-2.6/fs/nfs/inode.c
===================================================================
fs/nfs/inode.c struct inode *nfs_alloc_inode(struct sup · line 1129
Quoted 7 lines
fs/nfs/inode.c +1
-0@@ -1123,6 +1123,7 @@ struct inode *nfs_alloc_inode(struct sup nfsi->flags =0UL ; nfsi->cache_validity =0UL ; nfsi->cache_change_attribute = jiffies; + nfsi->req_lock =SPIN_LOCK_UNLOCKED ;#ifdef CONFIG_NFS_V3_ACL nfsi->acl_access =ERR_PTR (-EAGAIN ); nfsi->acl_default =ERR_PTR (-EAGAIN );
req_lock is initialsied in fs/nfs/inode.c:init_once().
What kernel version were you using?
Quoted 1 line · Andrew Morton
req_lockis initialsied in fs/nfs/inode.c:init_once().
Oh, it is indeed. Grmbl.
Quoted 1 line
What kernel version were you using?
I've reproduced this on a base 2.6.20 g5_defconfig + NFS root and serial console options on a G5 here.
The steps I have used are:
* Boot with NFS root, default mount options
* mount /dev/sda3 /mnt
... that's all. I have not seen it happen without NFS root, even with quite active NFS activity. So it seems to be a factor.
I'll continue debugging tomorrow.
-Olof
Quoted 18 lines · Olof Johansson
Seems like
req_lockis never initialized.CONFIG_DEBUG_SPINLOCKreported:BUG: spinlock bad magic on CPU#0, mount/1073 lock: c00000007fdca108, .magic: 00000000, .owner: /24576, .owner_cpu: 0 Call Trace: [C00000007E913750] [C0000000000107B4] .show_stack+0x54/0x1f0 (unreliable) [C00000007E913800] [C0000000001D2234] .spin_bug+0xa4/0x120 [C00000007E913890] [C0000000001D247C] ._raw_spin_lock+0xdc/0x1d0 [C00000007E913930] [C0000000004DF990] ._spin_lock+0x10/0x30 [C00000007E9139B0] [C00000000017623C] .nfs_sync_mapping_wait+0xac/0x310 [C00000007E913AA0] [C000000000176F18] .nfs_sync_mapping_range+0x98/0x110 [C00000007E913B80] [C00000000016CAA0] .nfs_getattr+0x40/0xf0 [C00000007E913C20] [C0000000000BE668] .vfs_getattr+0x38/0x70 [C00000007E913CA0] [C0000000000BF05C] .vfs_fstat+0x3c/0x70 [C00000007E913D30] [C0000000000BF0B8] .sys_fstat64+0x28/0x60 [C00000007E913E30] [C000000000008608] syscall_exit+0x0/0x40
Quoted 6 lines
Signed-off-by Olof Johansson <[email protected]>---
Trond, is your
MAINTAINERSentry up to date? Seems like you mostly post from @netapp.com these days.
I tend to be easier to get hold of via the fys.uio.no address, since that isn't hidden behind a VPN. I use the netapp.com address when posting patches etc since that is sort of required by the developers certificate of origin.
fs/nfs/inode.c struct inode *nfs_alloc_inode(struct sup · line 1129
Quoted 12 lines
Index:
linux-2.6/fs/nfs/inode.c
===================================================================fs/nfs/inode.c +1
-0@@ -1123,6 +1123,7 @@ struct inode *nfs_alloc_inode(struct sup nfsi->flags =0UL ; nfsi->cache_validity =0UL ; nfsi->cache_change_attribute = jiffies; + nfsi->req_lock =SPIN_LOCK_UNLOCKED ;#ifdef CONFIG_NFS_V3_ACL nfsi->acl_access =ERR_PTR (-EAGAIN ); nfsi->acl_default =ERR_PTR (-EAGAIN );
As Andrew implied, this spin lock should already be initialised by the slab allocator in the "init_once" callback. Is this a vanilla kernel, or do you have any extra patches applied?
Cheers
Trond
Quoted 7 lines · Trond Myklebust
Quoted 2 lines
Trond, is your
MAINTAINERSentry up to date? Seems like you mostly post from @netapp.com these days.I tend to be easier to get hold of via the fys.uio.no address, since that isn't hidden behind a VPN. I use the netapp.com address when posting patches etc since that is sort of required by the developers certificate of origin.
Ok, no problem. Just figured I'd ask.
Quoted 8 lines
Quoted 8 lines
Index:
linux-2.6/fs/nfs/inode.c
===================================================================fs/nfs/inode.c +0
-0@@ -1123,6 +1123,7 @@ struct inode *nfs_alloc_inode(struct sup nfsi->flags =0UL ; nfsi->cache_validity =0UL ; nfsi->cache_change_attribute = jiffies;
Quoted 8 lines
fs/nfs/inode.c struct inode *nfs_alloc_inode(struct sup · line 1129
Quoted 4 lines
fs/nfs/inode.c +1
-0+ nfsi->req_lock =SPIN_LOCK_UNLOCKED ;#ifdef CONFIG_NFS_V3_ACL nfsi->acl_access =ERR_PTR (-EAGAIN ); nfsi->acl_default =ERR_PTR (-EAGAIN );As Andrew implied, this spin lock should already be initialised by the slab allocator in the "
init_once" callback. Is this a vanilla kernel, or do you have any extra patches applied?
Plain 2.6.20 on an old dual g5 with g5_defconfig + NFS_ROOT + serial port enabled. To see the error instead of lockup I enable CONFIG_DEBUG_SPINLOCK.
In my original reproduction, I had to boot with nfs root, and try to mount my sata drive (/dev/sda3). This is with a static /dev, no udev. Seems like it happens when trying to mount any block device that's located on NFS.
Since this is what nfs_sync_mapping_wait does:
long nfs_sync_mapping_wait(struct address_space *mapping, struct
writeback_control *wbc, int how) {
struct inode *inode = mapping->host;
struct nfs_inode *nfsi = NFS_I(inode);
[...]
spin_lock(&nfsi->req_lock);[...]
I added this and it pops when mounting:
@@ -421,6 +421,10 @@ int nfs_getattr(struct vfsmount *mnt, st
int need_atime = NFS_I(inode)->cache_validity & NFS_INO_INVALID_ATIME;
int err;+ if (inode->i_mapping->host != inode) {
+ printk("inode %p host %p\n", inode, inode->i_mapping->host);
+ printk("inode_nfs %p host_nfs %p\n", NFS_I(inode), NFS_I(inode->i_mapping->host));
+ }
/* Flush out writes to the server in order to update c/mtime */
nfs_sync_mapping_range(inode->i_mapping, 0, 0, FLUSH_NOCOMMIT);I don't claim to know VFS internals, but doesn't it make sense that the device node is backed against the actual device, not an NFS inode? And if so, NFS can't expect to do nfs_sync_mapping_range() on it, or at least not dereference ->host and use it as an NFS inode, right?
What I'm not sure I understand is why it disappears in the first place when I add the spin lock init -- I never even see the i_mapping->host pointer being allocated as an nfs inode. Maybe I just messed that one up somehow.
-Olof
Quoted 28 lines · Olof Johansson
In my original reproduction, I had to boot with nfs root, and try to mount my sata drive (
/dev/sda3). This is with a static /dev, no udev. Seems like it happens when trying to mount any block device that's located on NFS.Since this is what
nfs_sync_mapping_waitdoes:long nfs_sync_mapping_wait(struct
address_space*mapping, structwriteback_control *wbc, int how) { struct inode *inode = mapping->host; struct nfs_inode *nfsi = NFS_I(inode); [...] spin_lock(&nfsi->req_lock);[...]
I added this and it pops when mounting:
(quoted diff) +4
-0@@ -421,6 +421,10 @@ int nfs_getattr(struct vfsmount *mnt, st int need_atime = NFS_I(inode)->cache_validity & NFS_INO_INVALID_ATIME; int err; + if (inode->i_mapping->host != inode) { + printk("inode %p host %p\n", inode, inode->i_mapping->host); + printk("inode_nfs %p host_nfs %p\n", NFS_I(inode), NFS_I(inode->i_mapping->host)); + } /* Flush out writes to the server in order to update c/mtime */ nfs_sync_mapping_range(inode->i_mapping, 0, 0, FLUSH_NOCOMMIT);
Quoted 4 lines
I don't claim to know VFS internals, but doesn't it make sense that the device node is backed against the actual device, not an NFS inode? And if so, NFS can't expect to do
nfs_sync_mapping_range()on it, or at least not dereference ->host and use it as an NFS inode, right?
NFS still has to manage the inode attributes and handle permissions. It is only when you open the device that the VFS takes over (see the call to init_special_inode() in nfs_fhget()).
Quoted 4 lines
What I'm not sure I understand is why it disappears in the first place when I add the spin lock init -- I never even see the
i_mapping->hostpointer being allocated as an nfs inode. Maybe I just messed that one up somehow.
Looks like we need a check in nfs_getattr() for a regular file. It makes no sense to call nfs_sync_mapping_range() on anything else. I think that should fix your problem: it will stop the NFS client from interfering with dirty pages on that inode's mapping.
Cheers
Trond
---
fs/nfs/inode.c | 3 ++- 1 files changed, 2 insertions(+), 1 deletions(-)
fs/nfs/inode.c +2 -1
@@ -429,7 +429,8 @@ int nfs_getattr(struct vfsmount *mnt, struct dentry *dentry, struct kstat *stat)int err;/* Flush out writes to the server in order to update c/mtime */ -nfs_sync_mapping_range (inode->i_mapping,0 ,0 ,FLUSH_NOCOMMIT ); +if (S_ISREG (inode->i_mode)) +nfs_sync_mapping_range (inode->i_mapping,0 ,0 ,FLUSH_NOCOMMIT );/* * We may force a getattr if the user cares about atime.
Olof Johansson
Acked-byQuoted 4 lines · Trond Myklebust
Looks like we need a check in
nfs_getattr()for a regular file. It makes no sense to callnfs_sync_mapping_range()on anything else. I think that should fix your problem: it will stop the NFS client from interfering with dirty pages on that inode's mapping.
Yep, that works here. I also verified that my previous patch really didn't change the behaviour. I wonder why it did once. Probably just pure luck with memory contents.
Quoted 5 lines
From: Trond Myklebust <[email protected]>
Date: Tue, 20 Feb 2007 19:28:07 -0500
Subject: No SubjectSigned-off-by Trond Myklebust <[email protected]>
-Olof