Skip to contentSkip to conversation

btrfs: scrub: wip, pause on fs freeze

PATCH In review David Sterba 7 participants 12 messages Jul 8, 2025 – Jul 20, 2025

Subsystems (2)
lore
Share & tools
lore

David Sterba

author maintainer
#0 lore raw

PATCH 1/1 btrfs: scrub: wip, pause on fs freeze

Implement sb->freeze_super that can instruct our threads to pause themselves. In case of (read-write) scrub this means to undo mnt_want_write, implemented as sb_start_write()/sb_end_write(). The freeze_super callback is necessary otherwise the call sb_want_write() inside the generic implementation hangs.

This works with concurrent scrub running and 'fsfreeze --freeze', not with process freezing (like with suspend).

References: https://lwn.net/Articles/1018341/

---
 fs/btrfs/fs.h    |  2 ++
 fs/btrfs/scrub.c | 21 +++++++++++++++++++++
 fs/btrfs/super.c | 36 ++++++++++++++++++++++++++++++++----
 3 files changed, 55 insertions(+), 4 deletions(-)
fs/btrfs/fs.h +2 -0
index 8cc07cc70b12..005828a6ab17 100644
@@ -137,6 +137,8 @@ enum {
 	BTRFS_FS_QUOTA_OVERRIDE,
 	/* Used to record internally whether fs has been frozen */
 	BTRFS_FS_FROZEN,
+	/* Started freezing, pause your progress. */
+	BTRFS_FS_FREEZING,
 	/*
 	 * Indicate that balance has been set up from the ioctl and is in the
 	 * main phase. The fs_info::balance_ctl is initialized.
fs/btrfs/scrub.c +21 -0
index 6776e6ab8d10..9a6bce6ea191 100644
@@ -2250,6 +2250,27 @@ static int scrub_simple_mirror(struct scrub_ctx *sctx,
 			ret = -ECANCELED;
 			break;
 		}
+
+		/* Freezing? */
+		if (test_bit(BTRFS_FS_FREEZING, &fs_info->flags)) {
+			atomic_inc(&fs_info->scrubs_paused);
+			smp_mb();
+			wake_up(&fs_info->scrub_pause_wait);
+
+			if (!sctx->readonly)
+				sb_end_write(fs_info->sb);
+
+			try_to_freeze();
+			wait_on_bit(&fs_info->flags, BTRFS_FS_FREEZING, TASK_UNINTERRUPTIBLE);
+
+			if (!sctx->readonly)
+				sb_start_write(fs_info->sb);
+
+			atomic_dec(&fs_info->scrubs_paused);
+			smp_mb();
+			wake_up(&fs_info->scrub_pause_wait);
+		}
+
 		/* Paused? */
 		if (atomic_read(&fs_info->scrub_pause_req)) {
 			/* Push queued extents */
fs/btrfs/super.c +32 -4
index e4ce2754cfde..c049d145db66 100644
@@ -2279,7 +2279,33 @@ static long btrfs_control_ioctl(struct file *file, unsigned int cmd,
 	return ret;
 }
 
-static int btrfs_freeze(struct super_block *sb)
+static int btrfs_freeze_super(struct super_block *sb, enum freeze_holder who,
+			      const void *freeze_owner)
+{
+	struct btrfs_fs_info *fs_info = btrfs_sb(sb);
+	int ret;
+
+	set_bit(BTRFS_FS_FREEZING, &fs_info->flags);
+	ret = freeze_super(sb, who, freeze_owner);
+	if (ret < 0)
+		clear_bit(BTRFS_FS_FREEZING, &fs_info->flags);
+	return ret;
+}
+
+static int btrfs_thaw_super(struct super_block *sb, enum freeze_holder who,
+			    const void *freeze_owner)
+{
+	struct btrfs_fs_info *fs_info = btrfs_sb(sb);
+	int ret;
+
+	ret = thaw_super(sb, who, freeze_owner);
+	clear_bit(BTRFS_FS_FREEZING, &fs_info->flags);
+	smp_mb();
+	wake_up_bit(&fs_info->flags, BTRFS_FS_FREEZING);
+	return ret;
+}
+
+static int btrfs_freeze_fs(struct super_block *sb)
 {
 	struct btrfs_fs_info *fs_info = btrfs_sb(sb);
 
@@ -2345,7 +2371,7 @@ static int check_dev_super(struct btrfs_device *dev)
 	return ret;
 }
 
-static int btrfs_unfreeze(struct super_block *sb)
+static int btrfs_unfreeze_fs(struct super_block *sb)
 {
 	struct btrfs_fs_info *fs_info = btrfs_sb(sb);
 	struct btrfs_device *device;
@@ -2426,8 +2452,10 @@ static const struct super_operations btrfs_super_ops = {
 	.destroy_inode	= btrfs_destroy_inode,
 	.free_inode	= btrfs_free_inode,
 	.statfs		= btrfs_statfs,
-	.freeze_fs	= btrfs_freeze,
-	.unfreeze_fs	= btrfs_unfreeze,
+	.freeze_super   = btrfs_freeze_super,
+	.thaw_super     = btrfs_thaw_super,
+	.freeze_fs	= btrfs_freeze_fs,
+	.unfreeze_fs	= btrfs_unfreeze_fs,
 	.nr_cached_objects = btrfs_nr_cached_objects,
 	.free_cached_objects = btrfs_free_cached_objects,
 };
Signature · 1 line
2.49.0
#1 lore raw
Quoted 37 lines · David Sterba

Implement sb->freeze_super that can instruct our threads to pause themselves. In case of (read-write) scrub this means to undo mnt_want_write, implemented as sb_start_write()/sb_end_write(). The freeze_super callback is necessary otherwise the call sb_want_write() inside the generic implementation hangs.

This works with concurrent scrub running and 'fsfreeze --freeze', not with process freezing (like with suspend).

References: https://lwn.net/Articles/1018341/

---

fs/btrfs/fs.h    |  2 ++
fs/btrfs/scrub.c | 21 +++++++++++++++++++++
fs/btrfs/super.c | 36 ++++++++++++++++++++++++++++++++----
3 files changed, 55 insertions(+), 4 deletions(-)
fs/btrfs/fs.h +2 -0
index 8cc07cc70b12..005828a6ab17 100644
@@ -137,6 +137,8 @@ enum {
 BTRFS_FS_QUOTA_OVERRIDE,
 /* Used to record internally whether fs has been frozen */
 BTRFS_FS_FROZEN,
+ /* Started freezing, pause your progress. */
+ BTRFS_FS_FREEZING,
 /*
 * Indicate that balance has been set up from the ioctl and is in the
 * main phase. The fs_info::balance_ctl is initialized.
fs/btrfs/scrub.c +0 -0
index 6776e6ab8d10..9a6bce6ea191 100644

fs/btrfs/scrub.c static int scrub_simple_mirror(struct scrub_ctx *sctx, · line 2257

Quoted 8 lines
fs/btrfs/scrub.c +5 -0
@@ -2250,6 +2250,27 @@ static int scrub_simple_mirror(struct scrub_ctx *sctx,
 ret = -ECANCELED;
 break;
 }
+
+ /* Freezing? */
+ if (test_bit(BTRFS_FS_FREEZING, &fs_info->flags)) {
+ atomic_inc(&fs_info->scrubs_paused);
+ smp_mb();

The memory barrier before wake_up seems not needed

Quoted 84 lines
fs/btrfs/scrub.c +16 -0
+ wake_up(&fs_info->scrub_pause_wait);
+
+ if (!sctx->readonly)
+ sb_end_write(fs_info->sb);
+
+ try_to_freeze();
+ wait_on_bit(&fs_info->flags, BTRFS_FS_FREEZING, TASK_UNINTERRUPTIBLE);
+
+ if (!sctx->readonly)
+ sb_start_write(fs_info->sb);
+
+ atomic_dec(&fs_info->scrubs_paused);
+ smp_mb();
+ wake_up(&fs_info->scrub_pause_wait);
+ }
+
 /* Paused? */
 if (atomic_read(&fs_info->scrub_pause_req)) {
 /* Push queued extents */
fs/btrfs/super.c +32 -4
index e4ce2754cfde..c049d145db66 100644
@@ -2279,7 +2279,33 @@ static long btrfs_control_ioctl(struct file *file, unsigned int cmd,
 return ret;
 }
 
-static int btrfs_freeze(struct super_block *sb)
+static int btrfs_freeze_super(struct super_block *sb, enum freeze_holder who,
+      const void *freeze_owner)
+{
+ struct btrfs_fs_info *fs_info = btrfs_sb(sb);
+ int ret;
+
+ set_bit(BTRFS_FS_FREEZING, &fs_info->flags);
+ ret = freeze_super(sb, who, freeze_owner);
+ if (ret < 0)
+ clear_bit(BTRFS_FS_FREEZING, &fs_info->flags);
+ return ret;
+}
+
+static int btrfs_thaw_super(struct super_block *sb, enum freeze_holder who,
+    const void *freeze_owner)
+{
+ struct btrfs_fs_info *fs_info = btrfs_sb(sb);
+ int ret;
+
+ ret = thaw_super(sb, who, freeze_owner);
+ clear_bit(BTRFS_FS_FREEZING, &fs_info->flags);
+ smp_mb();
+ wake_up_bit(&fs_info->flags, BTRFS_FS_FREEZING);
+ return ret;
+}
+
+static int btrfs_freeze_fs(struct super_block *sb)
 {
 struct btrfs_fs_info *fs_info = btrfs_sb(sb);
 
@@ -2345,7 +2371,7 @@ static int check_dev_super(struct btrfs_device *dev)
 return ret;
 }
 
-static int btrfs_unfreeze(struct super_block *sb)
+static int btrfs_unfreeze_fs(struct super_block *sb)
 {
 struct btrfs_fs_info *fs_info = btrfs_sb(sb);
 struct btrfs_device *device;
@@ -2426,8 +2452,10 @@ static const struct super_operations btrfs_super_ops = {
 .destroy_inode = btrfs_destroy_inode,
 .free_inode = btrfs_free_inode,
 .statfs = btrfs_statfs,
- .freeze_fs = btrfs_freeze,
- .unfreeze_fs = btrfs_unfreeze,
+ .freeze_super   = btrfs_freeze_super,
+ .thaw_super     = btrfs_thaw_super,
+ .freeze_fs = btrfs_freeze_fs,
+ .unfreeze_fs = btrfs_unfreeze_fs,
 .nr_cached_objects = btrfs_nr_cached_objects,
 .free_cached_objects = btrfs_free_cached_objects,
 };
Signature · 1 line
2.49.0
#2 lore raw
Quoted 40 lines · Alan Huang
Quoted 39 lines · David Sterba

Implement sb->freeze_super that can instruct our threads to pause themselves. In case of (read-write) scrub this means to undo mnt_want_write, implemented as sb_start_write()/sb_end_write(). The freeze_super callback is necessary otherwise the call sb_want_write() inside the generic implementation hangs.

This works with concurrent scrub running and 'fsfreeze --freeze', not with process freezing (like with suspend).

References: https://lwn.net/Articles/1018341/

---

fs/btrfs/fs.h    |  2 ++
fs/btrfs/scrub.c | 21 +++++++++++++++++++++
fs/btrfs/super.c | 36 ++++++++++++++++++++++++++++++++----
3 files changed, 55 insertions(+), 4 deletions(-)
fs/btrfs/fs.h +2 -0
index 8cc07cc70b12..005828a6ab17 100644
@@ -137,6 +137,8 @@ enum {
 BTRFS_FS_QUOTA_OVERRIDE,
 /* Used to record internally whether fs has been frozen */
 BTRFS_FS_FROZEN,
+ /* Started freezing, pause your progress. */
+ BTRFS_FS_FREEZING,
 /*
 * Indicate that balance has been set up from the ioctl and is in the
 * main phase. The fs_info::balance_ctl is initialized.
fs/btrfs/scrub.c +0 -0
index 6776e6ab8d10..9a6bce6ea191 100644
@@ -2250,6 +2250,27 @@ static int scrub_simple_mirror(struct scrub_ctx *sctx,
 ret = -ECANCELED;
 break;
Quoted 8 lines

fs/btrfs/scrub.c static int scrub_simple_mirror(struct scrub_ctx *sctx, · line 2257

Quoted 6 lines
fs/btrfs/scrub.c +5 -0
 }
+
+ /* Freezing? */
+ if (test_bit(BTRFS_FS_FREEZING, &fs_info->flags)) {
+ atomic_inc(&fs_info->scrubs_paused);
+ smp_mb();

The memory barrier before wake_up seems not needed

Yeah, possibly smp_mb__after_atomic(), I took the simplest approach not to get distracted by reasoning about barriers.

#3 lore raw
Quoted 51 lines · David Sterba
Quoted 40 lines · Alan Huang
Quoted 39 lines · David Sterba

Implement sb->freeze_super that can instruct our threads to pause themselves. In case of (read-write) scrub this means to undo mnt_want_write, implemented as sb_start_write()/sb_end_write(). The freeze_super callback is necessary otherwise the call sb_want_write() inside the generic implementation hangs.

This works with concurrent scrub running and 'fsfreeze --freeze', not with process freezing (like with suspend).

References: https://lwn.net/Articles/1018341/

---

fs/btrfs/fs.h    |  2 ++
fs/btrfs/scrub.c | 21 +++++++++++++++++++++
fs/btrfs/super.c | 36 ++++++++++++++++++++++++++++++++----
3 files changed, 55 insertions(+), 4 deletions(-)
fs/btrfs/fs.h +2 -0
index 8cc07cc70b12..005828a6ab17 100644
@@ -137,6 +137,8 @@ enum {
 BTRFS_FS_QUOTA_OVERRIDE,
 /* Used to record internally whether fs has been frozen */
 BTRFS_FS_FROZEN,
+ /* Started freezing, pause your progress. */
+ BTRFS_FS_FREEZING,
 /*
 * Indicate that balance has been set up from the ioctl and is in the
 * main phase. The fs_info::balance_ctl is initialized.
fs/btrfs/scrub.c +0 -0
index 6776e6ab8d10..9a6bce6ea191 100644
@@ -2250,6 +2250,27 @@ static int scrub_simple_mirror(struct scrub_ctx *sctx,
 ret = -ECANCELED;
 break;
Quoted 8 lines

fs/btrfs/scrub.c static int scrub_simple_mirror(struct scrub_ctx *sctx, · line 2257

Quoted 6 lines
fs/btrfs/scrub.c +5 -0
 }
+
+ /* Freezing? */
+ if (test_bit(BTRFS_FS_FREEZING, &fs_info->flags)) {
+ atomic_inc(&fs_info->scrubs_paused);
+ smp_mb();

The memory barrier before wake_up seems not needed

Quoted 2 lines

Yeah, possibly smp_mb__after_atomic(), I took the simplest approach not to get distracted by reasoning about barriers.

smp_mb__after_atomic() is only needed before wake_up_bit().

No memory barrier is needed before wake_up()

#4 lore raw
In reply to David Sterba
Quoted 5 lines · David Sterba

Implement sb->freeze_super that can instruct our threads to pause themselves. In case of (read-write) scrub this means to undo mnt_want_write, implemented as sb_start_write()/sb_end_write(). The freeze_super callback is necessary otherwise the call sb_want_write() inside the generic implementation hangs.

I don't this this is really going to work.

The main problem is out of btrfs, it's all about the s_writer.rw_sem.

If we have a running scrub, it holds the mnt_want_write_file(), which is a read lock on the rw_sem.

Then we start freezing, which will call sb_wait_write(), which will do a write lock on the rw_sem, waiting for the scrub to finish.

However the ->freeze() callback is only called when freeze_super() got the write lock on the rw_sem.

At that stage, it's already too late.

Thanks,
Qu>

Quoted 118 lines

This works with concurrent scrub running and 'fsfreeze --freeze', not with process freezing (like with suspend).

References: https://lwn.net/Articles/1018341/

---
  fs/btrfs/fs.h    |  2 ++
  fs/btrfs/scrub.c | 21 +++++++++++++++++++++
  fs/btrfs/super.c | 36 ++++++++++++++++++++++++++++++++----
  3 files changed, 55 insertions(+), 4 deletions(-)
fs/btrfs/fs.h +2 -0
index 8cc07cc70b12..005828a6ab17 100644
@@ -137,6 +137,8 @@ enum {
  	BTRFS_FS_QUOTA_OVERRIDE,
  	/* Used to record internally whether fs has been frozen */
  	BTRFS_FS_FROZEN,
+	/* Started freezing, pause your progress. */
+	BTRFS_FS_FREEZING,
  	/*
  	 * Indicate that balance has been set up from the ioctl and is in the
  	 * main phase. The fs_info::balance_ctl is initialized.
fs/btrfs/scrub.c +21 -0
index 6776e6ab8d10..9a6bce6ea191 100644
@@ -2250,6 +2250,27 @@ static int scrub_simple_mirror(struct scrub_ctx *sctx,
  			ret = -ECANCELED;
  			break;
  		}
+
+		/* Freezing? */
+		if (test_bit(BTRFS_FS_FREEZING, &fs_info->flags)) {
+			atomic_inc(&fs_info->scrubs_paused);
+			smp_mb();
+			wake_up(&fs_info->scrub_pause_wait);
+
+			if (!sctx->readonly)
+				sb_end_write(fs_info->sb);
+
+			try_to_freeze();
+			wait_on_bit(&fs_info->flags, BTRFS_FS_FREEZING, TASK_UNINTERRUPTIBLE);
+
+			if (!sctx->readonly)
+				sb_start_write(fs_info->sb);
+
+			atomic_dec(&fs_info->scrubs_paused);
+			smp_mb();
+			wake_up(&fs_info->scrub_pause_wait);
+		}
+
  		/* Paused? */
  		if (atomic_read(&fs_info->scrub_pause_req)) {
  			/* Push queued extents */
fs/btrfs/super.c +32 -4
index e4ce2754cfde..c049d145db66 100644
@@ -2279,7 +2279,33 @@ static long btrfs_control_ioctl(struct file *file, unsigned int cmd,
  	return ret;
  }
 
-static int btrfs_freeze(struct super_block *sb)
+static int btrfs_freeze_super(struct super_block *sb, enum freeze_holder who,
+			      const void *freeze_owner)
+{
+	struct btrfs_fs_info *fs_info = btrfs_sb(sb);
+	int ret;
+
+	set_bit(BTRFS_FS_FREEZING, &fs_info->flags);
+	ret = freeze_super(sb, who, freeze_owner);
+	if (ret < 0)
+		clear_bit(BTRFS_FS_FREEZING, &fs_info->flags);
+	return ret;
+}
+
+static int btrfs_thaw_super(struct super_block *sb, enum freeze_holder who,
+			    const void *freeze_owner)
+{
+	struct btrfs_fs_info *fs_info = btrfs_sb(sb);
+	int ret;
+
+	ret = thaw_super(sb, who, freeze_owner);
+	clear_bit(BTRFS_FS_FREEZING, &fs_info->flags);
+	smp_mb();
+	wake_up_bit(&fs_info->flags, BTRFS_FS_FREEZING);
+	return ret;
+}
+
+static int btrfs_freeze_fs(struct super_block *sb)
  {
  	struct btrfs_fs_info *fs_info = btrfs_sb(sb);
 
@@ -2345,7 +2371,7 @@ static int check_dev_super(struct btrfs_device *dev)
  	return ret;
  }
 
-static int btrfs_unfreeze(struct super_block *sb)
+static int btrfs_unfreeze_fs(struct super_block *sb)
  {
  	struct btrfs_fs_info *fs_info = btrfs_sb(sb);
  	struct btrfs_device *device;
@@ -2426,8 +2452,10 @@ static const struct super_operations btrfs_super_ops = {
  	.destroy_inode	= btrfs_destroy_inode,
  	.free_inode	= btrfs_free_inode,
  	.statfs		= btrfs_statfs,
-	.freeze_fs	= btrfs_freeze,
-	.unfreeze_fs	= btrfs_unfreeze,
+	.freeze_super   = btrfs_freeze_super,
+	.thaw_super     = btrfs_thaw_super,
+	.freeze_fs	= btrfs_freeze_fs,
+	.unfreeze_fs	= btrfs_unfreeze_fs,
  	.nr_cached_objects = btrfs_nr_cached_objects,
  	.free_cached_objects = btrfs_free_cached_objects,
  };
#5 lore raw
Quoted 13 lines · Qu Wenruo
Quoted 5 lines · David Sterba

Implement sb->freeze_super that can instruct our threads to pause themselves. In case of (read-write) scrub this means to undo mnt_want_write, implemented as sb_start_write()/sb_end_write(). The freeze_super callback is necessary otherwise the call sb_want_write() inside the generic implementation hangs.

I don't this this is really going to work.

The main problem is out of btrfs, it's all about the s_writer.rw_sem.

Quoted 8 lines

If we have a running scrub, it holds the mnt_want_write_file(), which is a read lock on the rw_sem.

Then we start freezing, which will call sb_wait_write(), which will do a write lock on the rw_sem, waiting for the scrub to finish.

However the ->freeze() callback is only called when freeze_super() got the write lock on the rw_sem.

Note there are 2 callbacks for freeze, sb::freeze_super and sb::freeze_fs.

ioctl_fsfreeze
if fs->freeze_super
call fs_freeze_super()
else

    freeze_super()
      sb_wait_write()
      if (sb->freeze_fs)
        call sb->freeze_fs()

What you describe matches the 'else' branch as this unconditionally blocks on the sb_wait_write and never gets to our callback. This is what I observed too.

To fix that I've added the sb::freeze_super so it can do something to avoid the lockup in sb_wait_write(), because we must at some point call freeze_super().

Schematically it goes like this:

Freeze thread Scrub

btrfs_ioctl_scrub
mnt_want_write_file (effectively sb_want_write())
ioctl_fsfreeze
fs->freeze_super

    set fs->flags FREEZING
    freeze_super
      sb_wait_freeze()
      (BLOCKS)
                                              (chunk loop)
                                                scrub_simple_mirror()
                                                  if fs->flags
						    sb_end_write()
						      wait_on_bit fs->flags FREEZING
						      (BLOCKS)
      (UNBLOCKS)
      ...

Unfreezing is basically the same in reverse:

ioctl_fsthaw
fs->thaw_super

    thaw_super()
    unset fs->flags FREEZING
                                                     (UNBLOCKS)
						     sb_want_write()
						     ...

So if you see a problem in that please let me know, I've tested the patch and it worked but I might me missing something.

#6 lore raw
Quoted 23 lines · David Sterba
Quoted 13 lines · Qu Wenruo
Quoted 5 lines · David Sterba

Implement sb->freeze_super that can instruct our threads to pause themselves. In case of (read-write) scrub this means to undo mnt_want_write, implemented as sb_start_write()/sb_end_write(). The freeze_super callback is necessary otherwise the call sb_want_write() inside the generic implementation hangs.

I don't this this is really going to work.

The main problem is out of btrfs, it's all about the s_writer.rw_sem.

Quoted 8 lines

If we have a running scrub, it holds the mnt_want_write_file(), which is a read lock on the rw_sem.

Then we start freezing, which will call sb_wait_write(), which will do a write lock on the rw_sem, waiting for the scrub to finish.

However the ->freeze() callback is only called when freeze_super() got the write lock on the rw_sem.

Quoted 6 lines

Note there are 2 callbacks for freeze, sb::freeze_super and sb::freeze_fs.

ioctl_fsfreeze
if fs->freeze_super
call fs_freeze_super()

Oh I forgot you implemented a new callback, ->freeze_super() to do the extra btrfs specific handling.

But this means, we're exposing ourselves to all the extra VFS handling,
I'm not sure if this is the preferred way, nor if the extra btrfs handling is good enough to cover what's done inside freeze_super() function (I guess not though).

The only user of ->freeze_super() callback is GFS2, and I didn't think we have enough locking/protection compared to either GFS2 nor VFS.

In that case, I still don't believe it's the correct way to go.

Add fsdevel list to see if it's even recommended to use ->freeze_super() callback.

Thanks,
Qu

Quoted 48 lines
   else
     freeze_super()
       sb_wait_write()
       if (sb->freeze_fs)
         call sb->freeze_fs()

What you describe matches the 'else' branch as this unconditionally blocks on the sb_wait_write and never gets to our callback. This is what I observed too.

To fix that I've added the sb::freeze_super so it can do something to avoid the lockup in sb_wait_write(), because we must at some point call freeze_super().

Schematically it goes like this:

Freeze thread Scrub

btrfs_ioctl_scrub
mnt_want_write_file (effectively sb_want_write())
ioctl_fsfreeze
fs->freeze_super

     set fs->flags FREEZING
     freeze_super
       sb_wait_freeze()
       (BLOCKS)
                                               (chunk loop)
                                                 scrub_simple_mirror()
                                                   if fs->flags
						    sb_end_write()
						      wait_on_bit fs->flags FREEZING
						      (BLOCKS)
       (UNBLOCKS)
       ...

Unfreezing is basically the same in reverse:

ioctl_fsthaw
fs->thaw_super

     thaw_super()
     unset fs->flags FREEZING
                                                      (UNBLOCKS)
						     sb_want_write()
						     ...

So if you see a problem in that please let me know, I've tested the patch and it worked but I might me missing something.

#7 lore raw
Quoted 29 lines · Qu Wenruo
Quoted 26 lines · David Sterba
Quoted 13 lines · Qu Wenruo
Quoted 5 lines · David Sterba

Implement sb->freeze_super that can instruct our threads to pause themselves. In case of (read-write) scrub this means to undo mnt_want_write, implemented as sb_start_write()/sb_end_write(). The freeze_super callback is necessary otherwise the call sb_want_write() inside the generic implementation hangs.

I don't this this is really going to work.

The main problem is out of btrfs, it's all about the s_writer.rw_sem.

Quoted 8 lines

If we have a running scrub, it holds the mnt_want_write_file(), which is a read lock on the rw_sem.

Then we start freezing, which will call sb_wait_write(), which will do a write lock on the rw_sem, waiting for the scrub to finish.

However the ->freeze() callback is only called when freeze_super() got the write lock on the rw_sem.

Note there are 2 callbacks for freeze, sb::freeze_super and sb::freeze_fs.

Quoted 8 lines
Quoted 3 lines

ioctl_fsfreeze
   if fs->freeze_super
     call fs_freeze_super()

Oh I forgot you implemented a new callback, ->freeze_super() to do the extra btrfs specific handling.

But this means, we're exposing ourselves to all the extra VFS handling,
I'm not sure if this is the preferred way, nor if the extra btrfs handling is good enough to cover what's done inside freeze_super() function (I guess not though).

My bad, it's calling back into freeze_super() function, so it's fine.

The remaining concern is just should we do this by using ->freeze_super() callback, or do the f2fs' way by poking into s_writer.frozen.

I'd prefer the later one, as that's already exposed to filesystems and we do not need extra callback functions especially when it's not widely used.

Thanks,
Qu

Quoted 61 lines

The only user of ->freeze_super() callback is GFS2, and I didn't think we have enough locking/protection compared to either GFS2 nor VFS.

In that case, I still don't believe it's the correct way to go.

Add fsdevel list to see if it's even recommended to use ->freeze_super() callback.

Thanks,
Qu

Quoted 48 lines

   else
     freeze_super()
       sb_wait_write()
       if (sb->freeze_fs)
         call sb->freeze_fs()

What you describe matches the 'else' branch as this unconditionally blocks on the sb_wait_write and never gets to our callback. This is what I observed too.

To fix that I've added the sb::freeze_super so it can do something to avoid the lockup in sb_wait_write(), because we must at some point call freeze_super().

Schematically it goes like this:

            Freeze thread                         Scrub

                                      btrfs_ioctl_scrub mnt_want_write_file (effectively sb_want_write()) ioctl_fsfreeze
   fs->freeze_super
     set fs->flags FREEZING
     freeze_super
       sb_wait_freeze()
       (BLOCKS)
                                               (chunk loop) scrub_simple_mirror() if fs->flags sb_end_write()
                              wait_on_bit fs->flags FREEZING (BLOCKS)
       (UNBLOCKS)
       ...

Unfreezing is basically the same in reverse:

ioctl_fsthaw
   fs->thaw_super
     thaw_super()
     unset fs->flags FREEZING
                                                      (UNBLOCKS) sb_want_write()
                             ...

So if you see a problem in that please let me know, I've tested the patch and it worked but I might me missing something.

#8 lore raw
Quoted 39 lines · Qu Wenruo
Quoted 36 lines · Qu Wenruo
Quoted 23 lines · David Sterba
Quoted 13 lines · Qu Wenruo
Quoted 5 lines · David Sterba

Implement sb->freeze_super that can instruct our threads to pause themselves. In case of (read-write) scrub this means to undo mnt_want_write, implemented as sb_start_write()/sb_end_write(). The freeze_super callback is necessary otherwise the call sb_want_write() inside the generic implementation hangs.

I don't this this is really going to work.

The main problem is out of btrfs, it's all about the s_writer.rw_sem.

Quoted 8 lines

If we have a running scrub, it holds the mnt_want_write_file(), which is a read lock on the rw_sem.

Then we start freezing, which will call sb_wait_write(), which will do a write lock on the rw_sem, waiting for the scrub to finish.

However the ->freeze() callback is only called when freeze_super() got the write lock on the rw_sem.

Quoted 6 lines

Note there are 2 callbacks for freeze, sb::freeze_super and sb::freeze_fs.

ioctl_fsfreeze
   if fs->freeze_super
     call fs_freeze_super()

Oh I forgot you implemented a new callback, ->freeze_super() to do the extra btrfs specific handling.

Quoted 6 lines
Quoted 4 lines

But this means, we're exposing ourselves to all the extra VFS handling, I'm not sure if this is the preferred way, nor if the extra btrfs handling is good enough to cover what's done inside freeze_super() function (I guess not though).

My bad, it's calling back into freeze_super() function, so it's fine.

You have to expose yourself to all the VFS handling and you must call freeze_super() at some point. You can't just do freeze on your own. You have roughly the following classes of freeze calls:

(1) initiated from userspace
(1.1) initiated from the block layer => upcall into the filesystem (1.2) initiated from the filesystem
(2) initiated from the kernel
(2.1) initiated from the filesystem
(2.2) suspend/hibernation

All of that requires synchronization and you cannot do this without going through the VFS layer's locking safely. At least not without causing royal pain for yourself and the block and VFS layer.

(Fwiw, I would still really like to see support for fs_holder_ops so btrfs freezing works correctly when called from the block layer.)

Quoted 7 lines

The remaining concern is just should we do this by using ->freeze_super() callback, or do the f2fs' way by poking into s_writer.frozen.

I'd prefer the later one, as that's already exposed to filesystems and we do not need extra callback functions especially when it's not widely used.

This would be simpler. You should probably add a helper that you can share with gfs2 to sample the state instead of raw accesses to s_writers->frozen. I find the latter a bit ugly and error prone if we ever change this. I'd rather have it encapsulated in the VFS layer as best as we can.

#9 lore raw
Quoted 13 lines · Christian Brauner
Quoted 1 line

My bad, it's calling back into freeze_super() function, so it's fine.

You have to expose yourself to all the VFS handling and you must call freeze_super() at some point. You can't just do freeze on your own. You have roughly the following classes of freeze calls:

(1) initiated from userspace
(1.1) initiated from the block layer => upcall into the filesystem (1.2) initiated from the filesystem
(2) initiated from the kernel
(2.1) initiated from the filesystem
(2.2) suspend/hibernation

Quoted 3 lines

All of that requires synchronization and you cannot do this without going through the VFS layer's locking safely. At least not without causing royal pain for yourself and the block and VFS layer.

You weren't CCed on the initial patch
https://lore.kernel.org/linux-btrfs/[email protected]/
so you can see we're not avoiding VFS, freeze_super() is called.

The problem I am solving is that we need to pass the information about freezing being started before freeze_super() is called, or alternatively by something that is set there before sb_wait_write() call.

Quoted 4 lines
Quoted 4 lines

The remaining concern is just should we do this by using ->freeze_super() callback, or do the f2fs' way by poking into s_writer.frozen.

Quoted 8 lines
Quoted 2 lines

I'd prefer the later one, as that's already exposed to filesystems and we do not need extra callback functions especially when it's not widely used.

This would be simpler. You should probably add a helper that you can share with gfs2 to sample the state instead of raw accesses to s_writers->frozen. I find the latter a bit ugly and error prone if we ever change this. I'd rather have it encapsulated in the VFS layer as best as we can.

I chose to use separate bit rather than using s_writers->frozen, which is what Qu prefers. I disgree with that on the API level as we probably should not do such direct check. A helper would be best, semantically teh same as the added filesystem bit.

#10 lore raw
Quoted 1 line · David Sterba

On Mon, Jul 14, 2025 at 12:27:37PM +0200, Christian Brauner wrote:

[...]

Quoted 14 lines
Quoted 5 lines
Quoted 5 lines

The remaining concern is just should we do this by using ->freeze_super() callback, or do the f2fs' way by poking into s_writer.frozen.

Quoted 8 lines
Quoted 2 lines

I'd prefer the later one, as that's already exposed to filesystems and we do not need extra callback functions especially when it's not widely used.

This would be simpler. You should probably add a helper that you can share with gfs2 to sample the state instead of raw accesses to s_writers->frozen. I find the latter a bit ugly and error prone if we ever change this. I'd rather have it encapsulated in the VFS layer as best as we can.

Quoted 4 lines

I chose to use separate bit rather than using s_writers->frozen, which is what Qu prefers. I disgree with that on the API level as we probably should not do such direct check. A helper would be best, semantically teh same as the added filesystem bit.

My biggest complain about the new ->freeze_super() callback is, it's very confusing to have both ->freeze_super() and ->freeze_fs().

Currently only GFS2 is utilizing that, and I'm not sure if it can be removed or not, as it's some gfs specific lock handling.

In our btrfs case, it's really just a single bit that can also be done by checking s_writer.frozen, thus I don't think doing the extra callbacks just for that bit is really worthy.

Talking about the extra helpers to handle the access, it's valid, although we may have several different helpers, as each fs has different checks. This makes the helper almost the same to the raw access to s_writers->frozen.

E.g. f2fs' gc and quota check if the fs is being/already frozen, thus it's the same as the btrfs's one.

But there are other call sites checking if the freezing is fully completed, and xfs even checks on the specific frozen stage.

So if one day we really change the s_writer->frozen stages, it will touch quite some fs codes anyway.

Thanks,
Qu

#11 lore raw
In reply to David Sterba

I just tested this patch on real hardware (laptop), and it didn't work.

This is what I did:
- I have btrfs on my root filesystem
- It is btrfs-raid with 2 copies of everything
- I started scrub, and in the same time did suspend

Suspend didn't work.

I did experiment twice: with /sys/power/freeze_filesystems set to 0 and to 1. In both cases suspend didn't work.

( /sys/power/freeze_filesystems was introduced in
https://lore.kernel.org/linux-fsdevel/[email protected]/
)

--
Askar Safin