fs: don't return -EINVAL for successful nested thaw
Subsystems (2)
PATCH 1/1 fs: don't return -EINVAL for successful nested thaw
Commit 7366f8b6fc6a ("fs: handle freezing from multiple devices") replaced the freeze_holders bitmask with per-holder counters to allow nested freezes. In the bitmask version, a thaw that released a shared hold while another holder remained returned 0. Since the rework, thaw_super_locked() drops the freeze reference via freeze_dec() but then returns -EINVAL when other freezers remain, misinforming the
caller: the thaw did succeed, the superblock just stays frozen for the remaining holders.
This breaks bdev-initiated freezing. When a filesystem is frozen with FIFREEZE and additionally frozen via bdev_freeze() -- which nests by design, see fs_bdev_freeze() -- the subsequent bdev_thaw() receives -EINVAL from the holder op although its freeze reference was dropped, and therefore keeps bd_fsfreeze_count elevated. Then device-mapper's unlock_fs() ignores bdev_thaw()'s return value, so nothing rebalances the count. After the user's FITHAW and umount, the block device can never be mounted again:
dm-1: Can't mount, blockdev is frozen
There is no way for userspace to drop the leaked count; only destroying the block device (or a reboot) recovers the device.
Reproducer (any kernel since v6.8):
dmsetup create dut --table "0 $(blockdev --getsz "$DEV") linear $DEV 0"
mkfs.ext4 /dev/mapper/dut
mount /dev/mapper/dut /mnt
fsfreeze --freeze /mnt # freeze_ucount == 1
dmsetup suspend dut # bd_fsfreeze_count == 1, ucount == 2
dmsetup resume dut # ucount 2 -> 1, but thaw_super()
# returns -EINVAL, so bdev_thaw()
# keeps bd_fsfreeze_count at 1
fsfreeze --unfreeze /mnt # filesystem thaws fine
umount /mnt
mount /dev/mapper/dut /mnt # EBUSY, foreverThe same happens with fsfreeze held across an LVM snapshot of the origin volume.
fs_bdev_thaw()'s documentation already describes the intended
semantics: "If this function returns zero it doesn't mean that the filesystem is unfrozen as it may have been frozen multiple times". Restore them by returning 0 when a nested thaw drops its hold while other freezers remain. Thawing without holding a freeze still fails with -EINVAL as may_unfreeze() rejects that case before the reference count is touched.
--- fs/super.c | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-)
fs/super.c +6 -3
@@ -2369,11 +2369,14 @@ static int thaw_super_locked(struct super_block *sb, enum freeze_holder who, goto out_unlock;/* -* All freezers share a single active reference. -* So just unlock in case there are any left. +* All freezers share a single active reference. If other freezers +* remain, drop our hold and report success; the superblock stays +* frozen until the last holder thaws it. */ -if (freeze_dec (sb, who)) +if (freeze_dec (sb, who)) { + error =0 ; goto out_unlock; + }if (sb_rdonly (sb)) { sb->s_writers.frozen =SB_UNFROZEN ;
Signature · 1 line
2.55.0
Quoted 5 lines · Moritz Tanner
This breaks bdev-initiated freezing. When a filesystem is frozen with FIFREEZE and additionally frozen via
bdev_freeze()-- which nests by design, seefs_bdev_freeze()-- the subsequentbdev_thaw()receives-EINVALfrom the holder op although its freeze reference was dropped, and therefore keepsbd_fsfreeze_countelevated.
The other thaw order is the same defect seen from userspace: while the bdev freeze is still held, fsfreeze -u returns EINVAL although it did drop its reference, and the filesystem stays frozen until the block layer holder thaws it.
In-kernel holders get the spurious error as well, whenever a userspace freeze is held at the same time: xfs_scrub's xchk_fsthaw() carries the comment "This should always succeed, we have a kernel freeze", and f2fs_ioc_gc_range() hands the error to userspace.
Quoted 13 lines · Moritz Tanner
Reproducer (any kernel since v6.8):
dmsetup create dut --table "0 $(blockdev --getsz "$DEV") linear $DEV 0" mkfs.ext4 /dev/mapper/dut mount /dev/mapper/dut /mnt fsfreeze --freeze /mnt # freeze_ucount == 1 dmsetup suspend dut # bd_fsfreeze_count == 1, ucount == 2 dmsetup resume dut # ucount 2 -> 1, but thaw_super() # returns -EINVAL, so bdev_thaw() # keeps bd_fsfreeze_count at 1 fsfreeze --unfreeze /mnt # filesystem thaws fine umount /mnt mount /dev/mapper/dut /mnt # EBUSY, forever
Can you wire this up in xfstests or blktests?
Christian Brauner
maintainerQuoted 40 lines · Moritz Tanner
Commit
7366f8b6fc6a("fs: handle freezing from multiple devices") replaced thefreeze_holdersbitmask with per-holder counters to allow nested freezes. In the bitmask version, a thaw that released a shared hold while another holder remained returned 0. Since the rework,thaw_super_locked()drops the freeze reference viafreeze_dec()but then returns-EINVALwhen other freezers remain, misinforming the
caller: the thaw did succeed, the superblock just stays frozen for the remaining holders.This breaks bdev-initiated freezing. When a filesystem is frozen with FIFREEZE and additionally frozen via
bdev_freeze()-- which nests by design, seefs_bdev_freeze()-- the subsequentbdev_thaw()receives-EINVALfrom the holder op although its freeze reference was dropped, and therefore keepsbd_fsfreeze_countelevated. Then device-mapper'sunlock_fs()ignores bdev_thaw()'s return value, so nothing rebalances the count. After the user's FITHAW and umount, the block device can never be mounted again:dm-1: Can't mount, blockdev is frozen
There is no way for userspace to drop the leaked count; only destroying the block device (or a reboot) recovers the device.
Reproducer (any kernel since v6.8):
dmsetup create dut --table "0 $(blockdev --getsz "$DEV") linear $DEV 0" mkfs.ext4 /dev/mapper/dut mount /dev/mapper/dut /mnt fsfreeze --freeze /mnt # freeze_ucount == 1 dmsetup suspend dut # bd_fsfreeze_count == 1, ucount == 2 dmsetup resume dut # ucount 2 -> 1, but thaw_super() # returns -EINVAL, so bdev_thaw() # keeps bd_fsfreeze_count at 1 fsfreeze --unfreeze /mnt # filesystem thaws fine umount /mnt mount /dev/mapper/dut /mnt # EBUSY, foreverThe same happens with fsfreeze held across an LVM snapshot of the origin volume.
Quoted 7 lines
fs_bdev_thaw()'s documentation already describes the intended
semantics: "If this function returns zero it doesn't mean that the filesystem is unfrozen as it may have been frozen multiple times". Restore them by returning 0 when a nested thaw drops its hold while other freezers remain. Thawing without holding a freeze still fails with-EINVALasmay_unfreeze()rejects that case before the reference count is touched.
This was contentions when we discussed this back then because you can also run into situations where userspace thinks it's safe to access the filesystem because they think the thaw succeeded. But anyway, I think we can try this change at least.
Quoted 1 line · Moritz Tanner
fs: don't return
-EINVALfor successful nested thaw
Applied to the vfs.fixes branch of the vfs/vfs.git tree. Patches in the vfs.fixes branch should appear in linux-next soon.
Please report any outstanding bugs that were missed during review in a new review to the original patch series allowing us to drop it.
It's encouraged to provide Acked-bys and Reviewed-bys even though the patch has now been applied. If possible patch trailers will be updated.
Note that commit hashes shown below are subject to change due to rebase, trailer updates or similar. If in doubt, please check the listed branch.
tree: https://git.kernel.org/pub/scm/linux/kernel/git/vfs/vfs.git branch: vfs.fixes
[1/1] fs: don't return -EINVAL for successful nested thaw
https://git.kernel.org/vfs/vfs/c/fe967191e585
Quoted 48 lines · Moritz Tanner
Commit
7366f8b6fc6a("fs: handle freezing from multiple devices") replaced thefreeze_holdersbitmask with per-holder counters to allow nested freezes. In the bitmask version, a thaw that released a shared hold while another holder remained returned 0. Since the rework,thaw_super_locked()drops the freeze reference viafreeze_dec()but then returns-EINVALwhen other freezers remain, misinforming the
caller: the thaw did succeed, the superblock just stays frozen for the remaining holders.This breaks bdev-initiated freezing. When a filesystem is frozen with FIFREEZE and additionally frozen via
bdev_freeze()-- which nests by design, seefs_bdev_freeze()-- the subsequentbdev_thaw()receives-EINVALfrom the holder op although its freeze reference was dropped, and therefore keepsbd_fsfreeze_countelevated. Then device-mapper'sunlock_fs()ignores bdev_thaw()'s return value, so nothing rebalances the count. After the user's FITHAW and umount, the block device can never be mounted again:dm-1: Can't mount, blockdev is frozen
There is no way for userspace to drop the leaked count; only destroying the block device (or a reboot) recovers the device.
Reproducer (any kernel since v6.8):
dmsetup create dut --table "0 $(blockdev --getsz "$DEV") linear $DEV 0" mkfs.ext4 /dev/mapper/dut mount /dev/mapper/dut /mnt fsfreeze --freeze /mnt # freeze_ucount == 1 dmsetup suspend dut # bd_fsfreeze_count == 1, ucount == 2 dmsetup resume dut # ucount 2 -> 1, but thaw_super() # returns -EINVAL, so bdev_thaw() # keeps bd_fsfreeze_count at 1 fsfreeze --unfreeze /mnt # filesystem thaws fine umount /mnt mount /dev/mapper/dut /mnt # EBUSY, foreverThe same happens with fsfreeze held across an LVM snapshot of the origin volume.
fs_bdev_thaw()'s documentation already describes the intended
semantics: "If this function returns zero it doesn't mean that the filesystem is unfrozen as it may have been frozen multiple times". Restore them by returning 0 when a nested thaw drops its hold while other freezers remain. Thawing without holding a freeze still fails with-EINVALasmay_unfreeze()rejects that case before the reference count is touched.
Quoted 3 lines
Fixes 7366f8b6fc6a ("fs: handle freezing from multiple devices")Cc <[email protected]> # needs adjustments for < 6.17 (no may_unfreeze())Signed-off-by Moritz Tanner <[email protected]>
Ouch. Good catch. Feel free to add:
Honza
Quoted 29 lines
--- fs/super.c | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-)fs/super.c +6
-3@@ -2369,11 +2369,14 @@ static int thaw_super_locked(struct super_block *sb, enum freeze_holder who, goto out_unlock;/* -* All freezers share a single active reference. -* So just unlock in case there are any left. +* All freezers share a single active reference. If other freezers +* remain, drop our hold and report success; the superblock stays +* frozen until the last holder thaws it. */ -if (freeze_dec (sb, who)) +if (freeze_dec (sb, who)) { + error =0 ; goto out_unlock; + }if (sb_rdonly (sb)) { sb->s_writers.frozen =SB_UNFROZEN ;Signature · 1 line
2.55.0
Signature · 2 lines
Jan Kara <[email protected]> SUSE Labs, CR