Skip to content

change sb_writers to use percpu_rw_semaphore

Oleg Nesterov linux-fsdevel lkml 3 versions – In review

Get series with b4
b4 am [email protected]

Versions newest first · linked by author and title

  1. v3 Latest8 patches 4 repliesIn review
  2. v28 patches 17 repliesSuperseded
  3. v14 patches 12 repliesSuperseded

Patches in v3 No applies recorded yet

#Subject and filesSizeRepliesApplied
1/8introduce __sb_writers_{acquired,release}() helpersfs/btrfs/transaction.c fs/xfs/xfs_aops.c include/linux/fs.h+9 -100 replies
2/8fix the broken lockdep logic in __sb_start_write()fs/super.c+40 -330 replies
3/8document rwsem_release() in sb_wait_write()fs/super.c+9 -30 replies
4/8percpu-rwsem: introduce percpu_down_read_trylock()include/linux/percpu-rwsem.h kernel/locking/percpu-rwsem.c+14 -00 replies
5/8percpu-rwsem: introduce percpu_rwsem_release() and percpu_rwsem_acquire()include/linux/percpu-rwsem.h+19 -00 replies
6/8percpu-rwsem: kill CONFIG_PERCPU_RWSEMarch/Kconfig init/Kconfig kernel/locking/Makefile +1 more+1 -70 replies
7/8shift percpu_counter_destroy() into destroy_super_work()fs/super.c include/linux/fs.h+21 -50 replies
8/8change sb_writers to use percpu_rw_semaphorefs/super.c include/linux/fs.h+36 -940 replies

Reviews trailers per patch

5 of 8 patches have a Reviewed-by or Acked-by of their own; the series has one for all of them.

PatchJan Kara
SeriesR-b
1/8R-b
2/8R-b
3/8R-b
4/8
5/8
6/8
7/8R-b
8/8R-b

Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes.

Read the v3 thread

Cover letter [PATCH v3 0/8] change sb_writers to use percpu_rw_semaphore
Quoted 3 lines · Jan Kara

Regarding the routing, ideally Al Viro should take these as a VFS maintainer.

Al, could you take these patches?

Only cosmetic changes in V3 to address the comments from Jan, I preserved his acks.

In case you missed all the spam I sent before, let me repeat that the awful (and currently unneeded) 7/8 will be reverted later. We need it to ensure that other percpu_rw_semaphore changes routed via another tree won't break fs/super.c. After that we will add rcu_sync_dtor(s_writers->rw_sem) into deactivate_locked_super() and revert this horror.

3/8 documents the lockdep problems we currently have. This is fixed by the patch below but it depends on xfs ILOCK fixes from Dave, so I will send it later. Plus another patch which removes the "trylock" hack in __sb_start_write().

Oleg.

 arch/Kconfig                  |    1 -
 fs/btrfs/transaction.c        |    8 +--
 fs/super.c                    |  184 +++++++++++++++++++---------------------
 fs/xfs/xfs_aops.c             |    6 +-
 include/linux/fs.h            |   23 +++---
 include/linux/percpu-rwsem.h  |   20 +++++
 init/Kconfig                  |    1 -
 kernel/locking/Makefile       |    3 +-
 kernel/locking/percpu-rwsem.c |   13 +++
 lib/Kconfig                   |    3 -
 10 files changed, 136 insertions(+), 126 deletions(-)

--------------------------------------------------------------------------
[PATCH v3 9/8] don't fool lockdep in freeze_super() and thaw_super() paths

sb_wait_write()->percpu_rwsem_release() fools lockdep to avoid the false-positives. Now that xfs was fixed by Dave we can remove it and change freeze_super() and thaw_super() to run with s_writers.rw_sem locks held; we add two trivial helpers for that, sb_freeze_release() and sb_freeze_acquire().

Signed-off-by Oleg Nesterov <[email protected]>
Reviewed-by Jan Kara <[email protected]>
---
 fs/super.c |   37 +++++++++++++++++++++++++------------
 1 files changed, 25 insertions(+), 12 deletions(-)
fs/super.c +25 -12
index 4350ff4..91c9756 100644
@@ -1213,25 +1213,34 @@ EXPORT_SYMBOL(__sb_start_write);
 static void sb_wait_write(struct super_block *sb, int level)
 {
 	percpu_down_write(sb->s_writers.rw_sem + level-1);
-	/*
-	 * We are going to return to userspace and forget about this lock, the
-	 * ownership goes to the caller of thaw_super() which does unlock.
-	 *
-	 * FIXME: we should do this before return from freeze_super() after we
-	 * called sync_filesystem(sb) and s_op->freeze_fs(sb), and thaw_super()
-	 * should re-acquire these locks before s_op->unfreeze_fs(sb). However
-	 * this leads to lockdep false-positives, so currently we do the early
-	 * release right after acquire.
-	 */
-	percpu_rwsem_release(sb->s_writers.rw_sem + level-1, 0, _THIS_IP_);
 }
 
-static void sb_freeze_unlock(struct super_block *sb)
+/*
+ * We are going to return to userspace and forget about these locks, the
+ * ownership goes to the caller of thaw_super() which does unlock().
+ */
+static void sb_freeze_release(struct super_block *sb)
+{
+	int level;
+
+	for (level = SB_FREEZE_LEVELS - 1; level >= 0; level--)
+		percpu_rwsem_release(sb->s_writers.rw_sem + level, 0, _THIS_IP_);
+}
+
+/*
+ * Tell lockdep we are holding these locks before we call ->unfreeze_fs(sb).
+ */
+static void sb_freeze_acquire(struct super_block *sb)
 {
 	int level;
 
 	for (level = 0; level < SB_FREEZE_LEVELS; ++level)
 		percpu_rwsem_acquire(sb->s_writers.rw_sem + level, 0, _THIS_IP_);
+}
+
+static void sb_freeze_unlock(struct super_block *sb)
+{
+	int level;
 
 	for (level = SB_FREEZE_LEVELS - 1; level >= 0; level--)
 		percpu_up_write(sb->s_writers.rw_sem + level);

@@ -1327,6 +1336,7 @@ int freeze_super(struct super_block *sb)
 	 * sees write activity when frozen is set to SB_FREEZE_COMPLETE.
 	 */
 	sb->s_writers.frozen = SB_FREEZE_COMPLETE;
+	sb_freeze_release(sb);
 	up_write(&sb->s_umount);
 	return 0;
 }

@@ -1353,11 +1363,14 @@ int thaw_super(struct super_block *sb)
 		goto out;
 	}
 
+	sb_freeze_acquire(sb);
+
 	if (sb->s_op->unfreeze_fs) {
 		error = sb->s_op->unfreeze_fs(sb);
 		if (error) {
 			printk(KERN_ERR
 				"VFS:Filesystem thaw failed\n");
+			sb_freeze_release(sb);
 			up_write(&sb->s_umount);
 			return error;
 		}
Signature · 1 line
1.5.5.1

Read the cover letter in its thread