Skip to contentSkip to conversation

getting rid of filp search in fs_may_remount_ro()

DISCUSSION Dave Hansen 2 participants 4 messages Dec 31, 2007 – Jan 2, 2008 2 messages not in the archive VFS

lore
Share & tools
lore

A message that is not in the archive (sent off-list or not yet received): [email protected]

A message that is not in the archive (sent off-list or not yet received): [email protected]

#2 lore raw
Quoted 3 lines · Christoph Hellwig

Btw, I just noticed in current -mm fs_may_remount_ro() is still around and not replaced by ther per-sb writers count. That surely sounds like some kind of mismerge..

I was actually leaving that for later. Getting rid of the filp search is a great benefit of the r/o bind patches, but it isn't strictly necessary and it doesn't really hurt anything to keep it.

The reason that it was contentious was that we need some way to be able to do an sb-to-mount mapping. When remounting the sb, we need to determine whether *any* of the mounts of that sb have any writers.

We don't currently have any mechanisms to do direct lookups from sb to mount. The only alternative I can see right now is to walk over all tasks, then walk over all vfs namespaces, and walk each mount tree to see if any mounts are of the sb we're looking for. This needs to be done while already holding the mnt_writers[] locks so that no new mnt writers can come in.

*THAT* is going to be a heavyweight operation. I need to go look in detail at how the mount trees are kept, and we'll need some kind of mechanism to keep track of which vfs namespaces we've looked at during the search so we don't search them twice.

Can you think of a simpler way to do it?

-- Dave

#3 lore raw
Quoted 20 lines · Dave Hansen
Quoted 3 lines · Christoph Hellwig

Btw, I just noticed in current -mm fs_may_remount_ro() is still around and not replaced by ther per-sb writers count. That surely sounds like some kind of mismerge..

I was actually leaving that for later. Getting rid of the filp search is a great benefit of the r/o bind patches, but it isn't strictly necessary and it doesn't really hurt anything to keep it.

The reason that it was contentious was that we need some way to be able to do an sb-to-mount mapping. When remounting the sb, we need to determine whether *any* of the mounts of that sb have any writers.

We don't currently have any mechanisms to do direct lookups from sb to mount. The only alternative I can see right now is to walk over all tasks, then walk over all vfs namespaces, and walk each mount tree to see if any mounts are of the sb we're looking for. This needs to be done while already holding the mnt_writers[] locks so that no new mnt writers can come in.

Quoted 6 lines

*THAT* is going to be a heavyweight operation. I need to go look in detail at how the mount trees are kept, and we'll need some kind of mechanism to keep track of which vfs namespaces we've looked at during the search so we don't search them twice.

Can you think of a simpler way to do it?

Here's one blatantly untested idea I have. The idea is to keep track if anyone might be writing to a mnt. We keep track on a flag in the mnt. When we set the flag, we increment a counter in the sb and decrement when the flag is cleared.

We can't simply look at mnt->__mnt_writers because there might be "checked-out" writers in the mnt_writers[] array. We also have to keep new writers from coming in while we do this, so we use the spinlocks in the mnt_writers[] array for exclusion. This is a pretty heavyweight lock, but it only gets used at rw->ro transitions.

-- Dave

fs/file_table.c renamed from fs/file_table.c.orig +6 -20
@@ -376,26 +376,12 @@
 
 int fs_may_remount_ro(struct super_block *sb)
 {
-	struct file *file;
-
-	/* Check that no files are currently opened for writing. */
-	file_list_lock();
-	list_for_each_entry(file, &sb->s_files, f_u.fu_list) {
-		struct inode *inode = file->f_path.dentry->d_inode;
-
-		/* File with pending delete? */
-		if (inode->i_nlink == 0)
-			goto too_bad;
-
-		/* Writeable file? */
-		if (S_ISREG(inode->i_mode) && (file->f_mode & FMODE_WRITE))
-			goto too_bad;
-	}
-	file_list_unlock();
-	return 1; /* Tis' cool bro. */
-too_bad:
-	file_list_unlock();
-	return 0;
+	int ret = 1;
+	lock_mnt_writers();
+	if (atomic_read(&sb->__s_mnt_writers))
+		ret = 0;
+	unlock_mnt_writers();
+	return ret;
 }
 
 void __init files_init(unsigned long mempages)
fs/namespace.c renamed from fs/namespace.c.orig +40 -1

70 lines not on this page · raw message

include/linux/fs.h renamed from include/linux/fs.h.orig +3 -0
@@ -1005,6 +1005,9 @@
 #ifdef CONFIG_SECURITY
 	void                    *s_security;
 #endif
+	atomic_t		__s_mnt_writers;/* how many mounts of this sb might
+						 * have writers.  Only stable with
+						 * all mnt_writers[] locks held */
 	struct xattr_handler	**s_xattr;
 
 	struct list_head	s_inodes;	/* all inodes */
include/linux/mount.h renamed from include/linux/mount.h.orig +3 -0
@@ -33,6 +33,7 @@
 
 #define MNT_SHRINKABLE	0x100
 #define MNT_IMBALANCED_WRITE_COUNT	0x200 /* just for debugging */
+#define MNT_MAY_HAVE_WRITERS		0x400 /* did this ever have a writer? */
 
 #define MNT_SHARED	0x1000	/* if the vfsmount is a shared mount */
 #define MNT_UNBINDABLE	0x2000	/* if the vfsmount is a unbindable mount */
@@ -80,6 +81,8 @@
 
 extern int mnt_want_write(struct vfsmount *mnt);
 extern void mnt_drop_write(struct vfsmount *mnt);
+extern void lock_mnt_writers(void);
+extern void unlock_mnt_writers(void);
 extern void mntput_no_expire(struct vfsmount *mnt);
 extern void mnt_pin(struct vfsmount *mnt);
 extern void mnt_unpin(struct vfsmount *mnt);
#4 lore raw
Quoted 10 lines · Dave Hansen

Here's one blatantly untested idea I have. The idea is to keep track if anyone might be writing to a mnt. We keep track on a flag in the mnt. When we set the flag, we increment a counter in the sb and decrement when the flag is cleared.

We can't simply look at mnt->__mnt_writers because there might be "checked-out" writers in the mnt_writers[] array. We also have to keep new writers from coming in while we do this, so we use the spinlocks in the mnt_writers[] array for exclusion. This is a pretty heavyweight lock, but it only gets used at rw->ro transitions.

This idea looks good to me, but if you're going to a final version please just inline fs_may_remount_ro into its only caller.

#5 lore raw
In reply to Dave Hansen
Quoted 7 lines · Dave Hansen
Quoted 3 lines

Btw, I just noticed in current -mm fs_may_remount_ro() is still around and not replaced by ther per-sb writers count. That surely sounds like some kind of mismerge..

I was actually leaving that for later. Getting rid of the filp search is a great benefit of the r/o bind patches, but it isn't strictly necessary and it doesn't really hurt anything to keep it.

Well, it's one of the primary arguments for the expensive infrastructure behind the per-mount r/o patches, as checkin the counter here means we can now do exact tracking of the writable status of the superblock and thus avoid the remount r/o races.

But there's another argument for it which is unrelated to the per-mount r/o patches. fs_may_remount_ro is the last non-trivial user of sb->s_files and the file_list_lock. Peter Zijlstra identified the latter as scalabilty problem, and I'd really prefer to get rid of the list and the lock instead of hacking around these issues. Removing sb->s_files will also allow removing a list_head from struct file with some additional hackery to the tty core which should be a nice memory safer on big systems.

Quoted 11 lines

The reason that it was contentious was that we need some way to be able to do an sb-to-mount mapping. When remounting the sb, we need to determine whether *any* of the mounts of that sb have any writers.

We don't currently have any mechanisms to do direct lookups from sb to mount. The only alternative I can see right now is to walk over all tasks, then walk over all vfs namespaces, and walk each mount tree to see if any mounts are of the sb we're looking for. This needs to be done while already holding the mnt_writers[] locks so that no new mnt writers can come in.

Quoted 6 lines

*THAT* is going to be a heavyweight operation. I need to go look in detail at how the mount trees are kept, and we'll need some kind of mechanism to keep track of which vfs namespaces we've looked at during the search so we don't search them twice.

Can you think of a simpler way to do it?

Well, the first revisions of the patches added a list of vfsmounts to the superblock, and I wasn't quite happy with that because I though we could get around that easily. Given that we can't get around it easily I'd be happy to take my comments back.