Skip to contentSkip to conversation

nfs: init req_lock in nfs_alloc_inode

PATCH In review Olof Johansson 3 participants 8 messages Feb 16, 2007 – Feb 22, 2007 NFS client

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 nfs: init req_lock in nfs_alloc_inode

Seems like req_lock is never initialized. CONFIG_DEBUG_SPINLOCK reported:

BUG: spinlock bad magic on CPU#0, mount/1073
 lock: c00000007fdca108, .magic: 00000000, .owner: /24576, .owner_cpu: 0
Call Trace:
[C00000007E913750] [C0000000000107B4] .show_stack+0x54/0x1f0 (unreliable)
[C00000007E913800] [C0000000001D2234] .spin_bug+0xa4/0x120
[C00000007E913890] [C0000000001D247C] ._raw_spin_lock+0xdc/0x1d0
[C00000007E913930] [C0000000004DF990] ._spin_lock+0x10/0x30
[C00000007E9139B0] [C00000000017623C] .nfs_sync_mapping_wait+0xac/0x310
[C00000007E913AA0] [C000000000176F18] .nfs_sync_mapping_range+0x98/0x110
[C00000007E913B80] [C00000000016CAA0] .nfs_getattr+0x40/0xf0
[C00000007E913C20] [C0000000000BE668] .vfs_getattr+0x38/0x70
[C00000007E913CA0] [C0000000000BF05C] .vfs_fstat+0x3c/0x70
[C00000007E913D30] [C0000000000BF0B8] .sys_fstat64+0x28/0x60
[C00000007E913E30] [C000000000008608] syscall_exit+0x0/0x40

---

Trond, is your MAINTAINERS entry up to date? Seems like you mostly post from @netapp.com these days.

fs/nfs/inode.c +1 -0
Index: linux-2.6/fs/nfs/inode.c
===================================================================
@@ -1123,6 +1123,7 @@ struct inode *nfs_alloc_inode(struct sup
 	nfsi->flags = 0UL;
 	nfsi->cache_validity = 0UL;
 	nfsi->cache_change_attribute = jiffies;
+	nfsi->req_lock = SPIN_LOCK_UNLOCKED;
 #ifdef CONFIG_NFS_V3_ACL
 	nfsi->acl_access = ERR_PTR(-EAGAIN);
 	nfsi->acl_default = ERR_PTR(-EAGAIN);
#1 lore raw

fs/nfs/inode.c struct inode *nfs_alloc_inode(struct sup · line 1126

Quoted 1 line · On Fri, 16 Feb 2007 11:05:32 -0600 [email protected] (Olof Johansson) wrote:
fs/nfs/inode.c +1 -0
+	nfsi->req_lock = SPIN_LOCK_UNLOCKED;

This can confound lockdep. Please use spin_lock_init().

#2 lore raw
In reply to Olof Johansson
Quoted 31 lines · On Fri, 16 Feb 2007 11:05:32 -0600 [email protected] (Olof Johansson) wrote:

Seems like req_lock is never initialized. CONFIG_DEBUG_SPINLOCK reported:

BUG: spinlock bad magic on CPU#0, mount/1073
 lock: c00000007fdca108, .magic: 00000000, .owner: /24576, .owner_cpu: 0
Call Trace:
[C00000007E913750] [C0000000000107B4] .show_stack+0x54/0x1f0 (unreliable)
[C00000007E913800] [C0000000001D2234] .spin_bug+0xa4/0x120
[C00000007E913890] [C0000000001D247C] ._raw_spin_lock+0xdc/0x1d0
[C00000007E913930] [C0000000004DF990] ._spin_lock+0x10/0x30
[C00000007E9139B0] [C00000000017623C] .nfs_sync_mapping_wait+0xac/0x310
[C00000007E913AA0] [C000000000176F18] .nfs_sync_mapping_range+0x98/0x110
[C00000007E913B80] [C00000000016CAA0] .nfs_getattr+0x40/0xf0
[C00000007E913C20] [C0000000000BE668] .vfs_getattr+0x38/0x70
[C00000007E913CA0] [C0000000000BF05C] .vfs_fstat+0x3c/0x70
[C00000007E913D30] [C0000000000BF0B8] .sys_fstat64+0x28/0x60
[C00000007E913E30] [C000000000008608] syscall_exit+0x0/0x40

---

Trond, is your MAINTAINERS entry up to date? Seems like you mostly post from @netapp.com these days.

Index: linux-2.6/fs/nfs/inode.c
===================================================================

fs/nfs/inode.c struct inode *nfs_alloc_inode(struct sup · line 1129

Quoted 7 lines
fs/nfs/inode.c +1 -0
@@ -1123,6 +1123,7 @@ struct inode *nfs_alloc_inode(struct sup
 	nfsi->flags = 0UL;
 	nfsi->cache_validity = 0UL;
 	nfsi->cache_change_attribute = jiffies;
+	nfsi->req_lock = SPIN_LOCK_UNLOCKED;
 #ifdef CONFIG_NFS_V3_ACL
 	nfsi->acl_access = ERR_PTR(-EAGAIN);
 	nfsi->acl_default = ERR_PTR(-EAGAIN);

req_lock is initialsied in fs/nfs/inode.c:init_once().

What kernel version were you using?

#3 lore raw
Quoted 1 line · Andrew Morton

req_lock is initialsied in fs/nfs/inode.c:init_once().

Oh, it is indeed. Grmbl.

Quoted 1 line

What kernel version were you using?

I've reproduced this on a base 2.6.20 g5_defconfig + NFS root and serial console options on a G5 here.

The steps I have used are:

* Boot with NFS root, default mount options
* mount /dev/sda3 /mnt

... that's all. I have not seen it happen without NFS root, even with quite active NFS activity. So it seems to be a factor.

I'll continue debugging tomorrow.

-Olof

#4 lore raw
In reply to Olof Johansson
Quoted 18 lines · Olof Johansson

Seems like req_lock is never initialized. CONFIG_DEBUG_SPINLOCK reported:

BUG: spinlock bad magic on CPU#0, mount/1073
 lock: c00000007fdca108, .magic: 00000000, .owner: /24576, .owner_cpu: 0
Call Trace:
[C00000007E913750] [C0000000000107B4] .show_stack+0x54/0x1f0 (unreliable)
[C00000007E913800] [C0000000001D2234] .spin_bug+0xa4/0x120
[C00000007E913890] [C0000000001D247C] ._raw_spin_lock+0xdc/0x1d0
[C00000007E913930] [C0000000004DF990] ._spin_lock+0x10/0x30
[C00000007E9139B0] [C00000000017623C] .nfs_sync_mapping_wait+0xac/0x310
[C00000007E913AA0] [C000000000176F18] .nfs_sync_mapping_range+0x98/0x110
[C00000007E913B80] [C00000000016CAA0] .nfs_getattr+0x40/0xf0
[C00000007E913C20] [C0000000000BE668] .vfs_getattr+0x38/0x70
[C00000007E913CA0] [C0000000000BF05C] .vfs_fstat+0x3c/0x70
[C00000007E913D30] [C0000000000BF0B8] .sys_fstat64+0x28/0x60
[C00000007E913E30] [C000000000008608] syscall_exit+0x0/0x40
Quoted 6 lines

---

Trond, is your MAINTAINERS entry up to date? Seems like you mostly post from @netapp.com these days.

I tend to be easier to get hold of via the fys.uio.no address, since that isn't hidden behind a VPN. I use the netapp.com address when posting patches etc since that is sort of required by the developers certificate of origin.

fs/nfs/inode.c struct inode *nfs_alloc_inode(struct sup · line 1129

Quoted 12 lines

Index: linux-2.6/fs/nfs/inode.c
===================================================================

fs/nfs/inode.c +1 -0
@@ -1123,6 +1123,7 @@ struct inode *nfs_alloc_inode(struct sup
 	nfsi->flags = 0UL;
 	nfsi->cache_validity = 0UL;
 	nfsi->cache_change_attribute = jiffies;
+	nfsi->req_lock = SPIN_LOCK_UNLOCKED;
 #ifdef CONFIG_NFS_V3_ACL
 	nfsi->acl_access = ERR_PTR(-EAGAIN);
 	nfsi->acl_default = ERR_PTR(-EAGAIN);

As Andrew implied, this spin lock should already be initialised by the slab allocator in the "init_once" callback. Is this a vanilla kernel, or do you have any extra patches applied?

Cheers
Trond

#5 lore raw
Quoted 7 lines · Trond Myklebust
Quoted 2 lines

Trond, is your MAINTAINERS entry up to date? Seems like you mostly post from @netapp.com these days.

I tend to be easier to get hold of via the fys.uio.no address, since that isn't hidden behind a VPN. I use the netapp.com address when posting patches etc since that is sort of required by the developers certificate of origin.

Ok, no problem. Just figured I'd ask.

Quoted 8 lines
Quoted 8 lines

Index: linux-2.6/fs/nfs/inode.c
===================================================================

fs/nfs/inode.c +0 -0
@@ -1123,6 +1123,7 @@ struct inode *nfs_alloc_inode(struct sup
 	nfsi->flags = 0UL;
 	nfsi->cache_validity = 0UL;
 	nfsi->cache_change_attribute = jiffies;
Quoted 8 lines

fs/nfs/inode.c struct inode *nfs_alloc_inode(struct sup · line 1129

Quoted 4 lines
fs/nfs/inode.c +1 -0
+	nfsi->req_lock = SPIN_LOCK_UNLOCKED;
 #ifdef CONFIG_NFS_V3_ACL
 	nfsi->acl_access = ERR_PTR(-EAGAIN);
 	nfsi->acl_default = ERR_PTR(-EAGAIN);

As Andrew implied, this spin lock should already be initialised by the slab allocator in the "init_once" callback. Is this a vanilla kernel, or do you have any extra patches applied?

Plain 2.6.20 on an old dual g5 with g5_defconfig + NFS_ROOT + serial port enabled. To see the error instead of lockup I enable CONFIG_DEBUG_SPINLOCK.

In my original reproduction, I had to boot with nfs root, and try to mount my sata drive (/dev/sda3). This is with a static /dev, no udev. Seems like it happens when trying to mount any block device that's located on NFS.

Since this is what nfs_sync_mapping_wait does:

long nfs_sync_mapping_wait(struct address_space *mapping, struct

		 	   writeback_control *wbc, int how) {
        struct inode *inode = mapping->host;
        struct nfs_inode *nfsi = NFS_I(inode);
[...]
        spin_lock(&nfsi->req_lock);

[...]

I added this and it pops when mounting:

@@ -421,6 +421,10 @@ int nfs_getattr(struct vfsmount *mnt, st

        int need_atime = NFS_I(inode)->cache_validity & NFS_INO_INVALID_ATIME;
        int err;

+ if (inode->i_mapping->host != inode) {
+ printk("inode %p host %p\n", inode, inode->i_mapping->host);
+ printk("inode_nfs %p host_nfs %p\n", NFS_I(inode), NFS_I(inode->i_mapping->host));
+ }

        /* Flush out writes to the server in order to update c/mtime */
        nfs_sync_mapping_range(inode->i_mapping, 0, 0, FLUSH_NOCOMMIT);

I don't claim to know VFS internals, but doesn't it make sense that the device node is backed against the actual device, not an NFS inode? And if so, NFS can't expect to do nfs_sync_mapping_range() on it, or at least not dereference ->host and use it as an NFS inode, right?

What I'm not sure I understand is why it disappears in the first place when I add the spin lock init -- I never even see the i_mapping->host pointer being allocated as an nfs inode. Maybe I just messed that one up somehow.

-Olof

#6 lore raw
Quoted 28 lines · Olof Johansson

In my original reproduction, I had to boot with nfs root, and try to mount my sata drive (/dev/sda3). This is with a static /dev, no udev. Seems like it happens when trying to mount any block device that's located on NFS.

Since this is what nfs_sync_mapping_wait does:

long nfs_sync_mapping_wait(struct address_space *mapping, struct

		 	   writeback_control *wbc, int how) {
        struct inode *inode = mapping->host;
        struct nfs_inode *nfsi = NFS_I(inode);
[...]
        spin_lock(&nfsi->req_lock);

[...]

I added this and it pops when mounting:

(quoted diff) +4 -0
@@ -421,6 +421,10 @@ int nfs_getattr(struct vfsmount *mnt, st
        int need_atime = NFS_I(inode)->cache_validity & NFS_INO_INVALID_ATIME;
        int err;
 
+       if (inode->i_mapping->host != inode) {
+               printk("inode %p host %p\n", inode, inode->i_mapping->host);
+               printk("inode_nfs %p host_nfs %p\n", NFS_I(inode), NFS_I(inode->i_mapping->host));
+       }
        /* Flush out writes to the server in order to update c/mtime */
        nfs_sync_mapping_range(inode->i_mapping, 0, 0, FLUSH_NOCOMMIT);
 
Quoted 4 lines

I don't claim to know VFS internals, but doesn't it make sense that the device node is backed against the actual device, not an NFS inode? And if so, NFS can't expect to do nfs_sync_mapping_range() on it, or at least not dereference ->host and use it as an NFS inode, right?

NFS still has to manage the inode attributes and handle permissions. It is only when you open the device that the VFS takes over (see the call to init_special_inode() in nfs_fhget()).

Quoted 4 lines

What I'm not sure I understand is why it disappears in the first place when I add the spin lock init -- I never even see the i_mapping->host pointer being allocated as an nfs inode. Maybe I just messed that one up somehow.

Looks like we need a check in nfs_getattr() for a regular file. It makes no sense to call nfs_sync_mapping_range() on anything else. I think that should fix your problem: it will stop the NFS client from interfering with dirty pages on that inode's mapping.

Cheers
Trond

---

 fs/nfs/inode.c |    3 ++-
 1 files changed, 2 insertions(+), 1 deletions(-)
fs/nfs/inode.c +2 -1
index af53c02..93d046c 100644
@@ -429,7 +429,8 @@ int nfs_getattr(struct vfsmount *mnt, struct dentry *dentry, struct kstat *stat)
 	int err;
 
 	/* Flush out writes to the server in order to update c/mtime */
-	nfs_sync_mapping_range(inode->i_mapping, 0, 0, FLUSH_NOCOMMIT);
+	if (S_ISREG(inode->i_mode))
+		nfs_sync_mapping_range(inode->i_mapping, 0, 0, FLUSH_NOCOMMIT);
 
 	/*
 	 * We may force a getattr if the user cares about atime.
author Acked-by
#7 lore raw
Quoted 4 lines · Trond Myklebust

Looks like we need a check in nfs_getattr() for a regular file. It makes no sense to call nfs_sync_mapping_range() on anything else. I think that should fix your problem: it will stop the NFS client from interfering with dirty pages on that inode's mapping.

Yep, that works here. I also verified that my previous patch really didn't change the behaviour. I wonder why it did once. Probably just pure luck with memory contents.

Quoted 5 lines

From: Trond Myklebust <[email protected]>
Date: Tue, 20 Feb 2007 19:28:07 -0500
Subject: No Subject

-Olof