btrfs: enhanced logic for stray single device
cover btrfs: enhanced logic for stray single device
This patch series comprises two preparatory patches (patches 1 and 3 below), one bug fix (patch 2), and one logic hardening patch (patch 4) designed to address issues with stray single devices.
The objective is to ensure that single devices are not left lingering in the device list unless they are properly mounted.
Anand Jain (4):
btrfs: declare btrfs_free_stale_devices non-static
btrfs: forget stray device on failed open
btrfs: refactor btrfs_free_stale_devices to free single stray device
btrfs: validate device_list at scan for stray free
fs/btrfs/super.c | 3 +++ fs/btrfs/volumes.c | 18 ++++++++++++++---- fs/btrfs/volumes.h | 2 ++ 3 files changed, 19 insertions(+), 4 deletions(-)
Signature · 1 line
2.38.1
PATCH 1/4 btrfs: declare btrfs_free_stale_devices non-static
When the open fails, to remove the stray device, we need btrfs_free_stale_devices() function. Make it non-static.
--- fs/btrfs/volumes.c | 2 +- fs/btrfs/volumes.h | 1 + 2 files changed, 2 insertions(+), 1 deletion(-)
fs/btrfs/volumes.c +1 -1
@@ -515,7 +515,7 @@ btrfs_get_bdev_and_sb(const char *device_path, blk_mode_t flags, void *holder,* -EBUSY if @devt is a mounted device. * -ENOENT if @devt does not match any device in the list. */ -static int btrfs_free_stale_devices (dev_t devt,struct btrfs_device *skip_device) +int btrfs_free_stale_devices (dev_t devt,struct btrfs_device *skip_device) {struct btrfs_fs_devices *fs_devices, *tmp_fs_devices;struct btrfs_device *device, *tmp_device;
fs/btrfs/volumes.h +1 -0
@@ -681,6 +681,7 @@ int btrfs_open_devices(struct btrfs_fs_devices *fs_devices, blk_mode_t flags, void *holder);struct btrfs_device *btrfs_scan_one_device (const char *path,blk_mode_t flags,bool mount_arg_dev); +int btrfs_free_stale_devices (dev_t devt,struct btrfs_device *skip_device);int btrfs_forget_devices (dev_t devt);void btrfs_close_devices (struct btrfs_fs_devices *fs_devices);void btrfs_free_extra_devids (struct btrfs_fs_devices *fs_devices);
Signature · 1 line
2.38.1
PATCH 2/4 btrfs: forget stray device on failed open
In reply to Anand JainIf the physical device of a flakey dm device is tried mounting it fails to open the device with handle, and leaves behind a stray single device in the device list.
Remove it if the open fails and if it is a single device. As we don't register a single device in the device list unless it is mounted.
--- fs/btrfs/super.c | 3 +++ 1 file changed, 3 insertions(+)
fs/btrfs/super.c +3 -0
@@ -1820,6 +1820,9 @@ static int btrfs_get_tree_super(struct fs_context *fc) fs_info->fs_devices = fs_devices; ret =btrfs_open_devices (fs_devices, mode, &btrfs_fs_type); +if (ret && fs_devices->total_devices ==1 ) +btrfs_free_stale_devices (device->devt,NULL ); +mutex_unlock (&uuid_mutex);if (ret)return ret;
Signature · 1 line
2.38.1
Re: [PATCH 2/4] btrfs: forget stray device on failed open
Quoted 17 lines · Anand Jain
If the physical device of a flakey dm device is tried mounting it fails to open the device with handle, and leaves behind a stray single device in the device list.
Remove it if the open fails and if it is a single device. As we don't register a single device in the device list unless it is mounted.
Signed-off-by Anand Jain <[email protected]>--- fs/btrfs/super.c | 3 +++ 1 file changed, 3 insertions(+)fs/btrfs/super.c +0
-0
fs/btrfs/super.c static int btrfs_get_tree_super(struct fs_context *fc) · line 1825
Quoted 6 lines
fs/btrfs/super.c +3
-0@@ -1820,6 +1820,9 @@ static int btrfs_get_tree_super(struct fs_context *fc) fs_info->fs_devices = fs_devices; ret =btrfs_open_devices (fs_devices, mode, &btrfs_fs_type); +if (ret && fs_devices->total_devices ==1 ) +btrfs_free_stale_devices (device->devt,NULL ); +
It feels like we need to do this free no matter what the mount error is after this point, not just in this one place.
Quoted 6 lines
fs/btrfs/super.c +0
-0mutex_unlock (&uuid_mutex);if (ret)return ret;Signature · 1 line
2.38.1
Re: [PATCH 2/4] btrfs: forget stray device on failed open
Quoted 20 lines · Boris Burkov
Quoted 19 lines · Anand Jain
If the physical device of a flakey dm device is tried mounting it fails to open the device with handle, and leaves behind a stray single device in the device list.
Remove it if the open fails and if it is a single device. As we don't register a single device in the device list unless it is mounted.
Signed-off-by Anand Jain <[email protected]>--- fs/btrfs/super.c | 3 +++ 1 file changed, 3 insertions(+)fs/btrfs/super.c +0
-0@@ -1820,6 +1820,9 @@ static int btrfs_get_tree_super(struct fs_context *fc) fs_info->fs_devices = fs_devices;
Quoted 8 lines
fs/btrfs/super.c static int btrfs_get_tree_super(struct fs_context *fc) · line 1825
Quoted 4 lines
fs/btrfs/super.c +3
-0ret =btrfs_open_devices (fs_devices, mode, &btrfs_fs_type); +if (ret && fs_devices->total_devices ==1 ) +btrfs_free_stale_devices (device->devt,NULL ); +It feels like we need to do this free no matter what the mount error is after this point, not just in this one place.
Indeed. At goto error, btrfs_close_devices() is called, in turn calls free_fs_devices() for single-device filesystems.
Thanks, Anand
Quoted 6 lines
Quoted 6 lines
fs/btrfs/super.c +0
-0mutex_unlock (&uuid_mutex);if (ret)return ret;Signature · 1 line
2.38.1
PATCH 3/4 btrfs: refactor btrfs_free_stale_devices to free single stray device
In reply to Anand JainRefactor the function btrfs_free_stale_devices() to search for devices with a single device and unmounted, freeing it.
This a preparation harden the reliance of tempfsid on a stray-free single device, allowing temp fsid activation on a device.
--- fs/btrfs/super.c | 2 +- fs/btrfs/volumes.c | 16 ++++++++++++---- fs/btrfs/volumes.h | 3 ++- 3 files changed, 15 insertions(+), 6 deletions(-)
fs/btrfs/super.c +1 -1
@@ -1821,7 +1821,7 @@ static int btrfs_get_tree_super(struct fs_context *fc) ret =btrfs_open_devices (fs_devices, mode, &btrfs_fs_type);if (ret && fs_devices->total_devices ==1 ) -btrfs_free_stale_devices (device->devt,NULL ); +btrfs_free_stale_devices (device->devt,NULL , false);mutex_unlock (&uuid_mutex);if (ret)
fs/btrfs/volumes.c +12 -4
@@ -515,7 +515,8 @@ btrfs_get_bdev_and_sb(const char *device_path, blk_mode_t flags, void *holder,* -EBUSY if @devt is a mounted device. * -ENOENT if @devt does not match any device in the list. */ -int btrfs_free_stale_devices (dev_t devt,struct btrfs_device *skip_device) +int btrfs_free_stale_devices (dev_t devt,struct btrfs_device *skip_device, +bool free_stray_single) {struct btrfs_fs_devices *fs_devices, *tmp_fs_devices;struct btrfs_device *device, *tmp_device;
@@ -529,6 +530,12 @@ int btrfs_free_stale_devices(dev_t devt, struct btrfs_device *skip_device)list_for_each_entry_safe (fs_devices, tmp_fs_devices, &fs_uuids, fs_list) {mutex_lock (&fs_devices->device_list_mutex); + +if (free_stray_single && fs_devices->total_devices !=1 ) { +mutex_unlock (&fs_devices->device_list_mutex); +continue ; + } +list_for_each_entry_safe (device, tmp_device, &fs_devices->devices, dev_list) {if (skip_device && skip_device == device)
@@ -1307,7 +1314,7 @@ int btrfs_forget_devices(dev_t devt)int ret;mutex_lock (&uuid_mutex); - ret =btrfs_free_stale_devices (devt,NULL ); + ret =btrfs_free_stale_devices (devt,NULL , false);mutex_unlock (&uuid_mutex);return ret;
@@ -1416,7 +1423,8 @@ struct btrfs_device *btrfs_scan_one_device(const char *path, blk_mode_t flags, path,MAJOR (bdev_handle->bdev->bd_dev), MINOR(bdev_handle->bdev->bd_dev)); -btrfs_free_stale_devices (bdev_handle->bdev->bd_dev,NULL ); +btrfs_free_stale_devices (bdev_handle->bdev->bd_dev,NULL , + false); device =NULL ; goto free_disk_super;
@@ -1424,7 +1432,7 @@ struct btrfs_device *btrfs_scan_one_device(const char *path, blk_mode_t flags, device =device_list_add (path, disk_super, &new_device_added);if (!IS_ERR (device) && new_device_added) -btrfs_free_stale_devices (device->devt, device); +btrfs_free_stale_devices (device->devt, device, false); free_disk_super:btrfs_release_disk_super (disk_super);
fs/btrfs/volumes.h +2 -1
@@ -681,7 +681,8 @@ int btrfs_open_devices(struct btrfs_fs_devices *fs_devices, blk_mode_t flags, void *holder);struct btrfs_device *btrfs_scan_one_device (const char *path,blk_mode_t flags,bool mount_arg_dev); -int btrfs_free_stale_devices (dev_t devt,struct btrfs_device *skip_device); +int btrfs_free_stale_devices (dev_t devt,struct btrfs_device *skip_device, +bool free_stray_single);int btrfs_forget_devices (dev_t devt);void btrfs_close_devices (struct btrfs_fs_devices *fs_devices);void btrfs_free_extra_devids (struct btrfs_fs_devices *fs_devices);
Signature · 1 line
2.38.1
PATCH 4/4 btrfs: validate device_list at scan for stray free
In reply to Anand JainTempfsid assumes all registered single devices in the fs_devicies list are to be mounted; otherwise, they won't be in the btrfs_device list.
We recently fixed a related bug caused by leaving failed-open device in the list. This triggered tempfsid activation upon subsequent mounts of the same fsid wrongly.
To prevent this, scan the entire device list at mount for any stray device and free them in btrfs_scan_one_device().
--- fs/btrfs/volumes.c | 2 ++ 1 file changed, 2 insertions(+)
fs/btrfs/volumes.c +2 -0
@@ -1382,6 +1382,8 @@ struct btrfs_device *btrfs_scan_one_device(const char *path, blk_mode_t flags,lockdep_assert_held (&uuid_mutex); +btrfs_free_stale_devices (0 ,NULL , true); +/* * we would like to check all the supers, but that would make * a btrfs mount succeed after a mkfs from a different FS.
Signature · 1 line
2.38.1
Re: [PATCH 4/4] btrfs: validate device_list at scan for stray free
Quoted 9 lines · Anand Jain
Tempfsid assumes all registered single devices in the
fs_devicieslist are to be mounted; otherwise, they won't be in thebtrfs_devicelist.We recently fixed a related bug caused by leaving failed-open device in the list. This triggered tempfsid activation upon subsequent mounts of the same fsid wrongly.
To prevent this, scan the entire device list at mount for any stray device and free them in
btrfs_scan_one_device().
Is this an additional precaution on top of maintaining an invariant on every umount/failed mount that we have freed stale devices of single device fs-es? Or is it fundamentally impossible for us to enforce that invariant?
It feels like overkill to hack up free_stale_devices in this way, compared to just ensuring that we manage cleaning up single devices fs-es correctly when we are in a cleanup context. If this is practically the best way to ensure we don't get caught with our pants down by a random stale device, then I suppose it's fine.
A total aside I just thought of:
I think it might also make sense to consider adding logic to look for single device fs-es with a device->bdev that is stale from the block layer's perspective, and somehow marking those in a way that tempfsid cares about. That would help with things that like that case where we delete the block dev out from under a mounted fs and mount it a second time with tempfsid after it's recreated. Not a huge deal, as we've already discussed, though.
Quoted 22 lines
Signed-off-by Anand Jain <[email protected]>--- fs/btrfs/volumes.c | 2 ++ 1 file changed, 2 insertions(+)fs/btrfs/volumes.c +2
-0@@ -1382,6 +1382,8 @@ struct btrfs_device *btrfs_scan_one_device(const char *path, blk_mode_t flags,lockdep_assert_held (&uuid_mutex); +btrfs_free_stale_devices (0 ,NULL , true); +/* * we would like to check all the supers, but that would make * a btrfs mount succeed after a mkfs from a different FS. Signature · 1 line
2.38.1
Re: [PATCH 4/4] btrfs: validate device_list at scan for stray free
Quoted 8 lines · Boris Burkov
Quoted 7 lines · Anand Jain
Tempfsid assumes all registered single devices in the
fs_devicieslist are to be mounted; otherwise, they won't be in thebtrfs_devicelist.We recently fixed a related bug caused by leaving failed-open device in the list. This triggered tempfsid activation upon subsequent mounts of the same fsid wrongly.
Quoted 8 lines
Quoted 2 lines
To prevent this, scan the entire device list at mount for any stray device and free them in
btrfs_scan_one_device().Is this an additional precaution on top of maintaining an invariant on every umount/failed mount that we have freed stale devices of single device fs-es? Or is it fundamentally impossible for us to enforce that invariant?
Hmm. That's the ultimate goal: maintaining such an invariant. However, there are bugs. So, this is the place where we can detect whether we are successful. If we aren't, then we can work around it by freeing the stale device and avoiding bad consequences.
I think I should also include a warning message when we detect and free, so that it can be reviewed for the proper fix.
Does that seem reasonable?
Quoted 11 lines
It feels like overkill to hack up
free_stale_devicesin this way, compared to just ensuring that we manage cleaning up single devices fs-es correctly when we are in a cleanup context. If this is practically the best way to ensure we don't get caught with our pants down by a random stale device, then I suppose it's fine.A total aside I just thought of:
I think it might also make sense to consider adding logic to look for single device fs-es with adevice->bdevthat is stale from the block layer's perspective, and somehow marking those in a way that tempfsid cares about.
How would we know if the block layer considers a certain device's block device (device->bdev) as stale?
If you mention a Write IO failure, we already put the filesystem in read-only mode if that happens. But, we can't close the device due to the pending write. (Some operating systems have an option to call panic, which dumps the memory to the coredump device and reboots.).
Quoted 4 lines
That would help with things that like that case where we delete the block dev out from under a mounted fs and mount it a second time with tempfsid after it's recreated. Not a huge deal, as we've already discussed, though.
Yeah, thanks for brainstorming. Basically, we need a way to distinguish between the same physical-device with multiple nodes and different physical-devices with the same filesystem.
Thanks, Anand
Quoted 23 lines
Quoted 23 lines
Signed-off-by Anand Jain <[email protected]>--- fs/btrfs/volumes.c | 2 ++ 1 file changed, 2 insertions(+)fs/btrfs/volumes.c +2
-0@@ -1382,6 +1382,8 @@ struct btrfs_device *btrfs_scan_one_device(const char *path, blk_mode_t flags,lockdep_assert_held (&uuid_mutex); +btrfs_free_stale_devices (0 ,NULL , true); +/* * we would like to check all the supers, but that would make * a btrfs mount succeed after a mkfs from a different FS. Signature · 1 line
2.38.1
Re: [PATCH 4/4] btrfs: validate device_list at scan for stray free
Boris,
Gentle ping?
Thanks.
Quoted 80 lines · Anand Jain
Quoted 8 lines · Boris Burkov
Quoted 7 lines · Anand Jain
Tempfsid assumes all registered single devices in the
fs_devicieslist are to be mounted; otherwise, they won't be in thebtrfs_devicelist.We recently fixed a related bug caused by leaving failed-open device in the list. This triggered tempfsid activation upon subsequent mounts of the same fsid wrongly.
Quoted 8 lines
Quoted 2 lines
To prevent this, scan the entire device list at mount for any stray device and free them in
btrfs_scan_one_device().Is this an additional precaution on top of maintaining an invariant on every umount/failed mount that we have freed stale devices of single device fs-es? Or is it fundamentally impossible for us to enforce that invariant?
Hmm. That's the ultimate goal: maintaining such an invariant. However, there are bugs. So, this is the place where we can detect whether we are successful. If we aren't, then we can work around it by freeing the stale device and avoiding bad consequences.
I think I should also include a warning message when we detect and free, so that it can be reviewed for the proper fix.
Does that seem reasonable?Quoted 11 lines
It feels like overkill to hack up
free_stale_devicesin this way, compared to just ensuring that we manage cleaning up single devices fs-es correctly when we are in a cleanup context. If this is practically the best way to ensure we don't get caught with our pants down by a random stale device, then I suppose it's fine.A total aside I just thought of:
I think it might also make sense to consider adding logic to look for single device fs-es with adevice->bdevthat is stale from the block layer's perspective, and somehow marking those in a way that tempfsid cares about.How would we know if the block layer considers a certain device's block device (
device->bdev) as stale?If you mention a Write IO failure, we already put the filesystem in read-only mode if that happens. But, we can't close the device due to the pending write. (Some operating systems have an option to call panic, which dumps the memory to the coredump device and reboots.).
Quoted 4 lines
That would help with things that like that case where we delete the block dev out from under a mounted fs and mount it a second time with tempfsid after it's recreated. Not a huge deal, as we've already discussed, though.
Yeah, thanks for brainstorming. Basically, we need a way to distinguish between the same physical-device with multiple nodes and different physical-devices with the same filesystem.
Thanks, Anand
Quoted 22 lines
Quoted 22 lines
Signed-off-by Anand Jain <[email protected]>---
fs/btrfs/volumes.c| 2 ++1 file changed, 2 insertions(+)fs/btrfs/volumes.c +2
-0@@ -1382,6 +1382,8 @@ struct btrfs_device *btrfs_scan_one_device(const char *path, blk_mode_t flags, lockdep_assert_held(&uuid_mutex); + btrfs_free_stale_devices(0, NULL, true); + /** we would like to check all the supers, but that would make * a btrfs mount succeed after a mkfs from a different FS. Signature · 1 line
2.38.1
Gentle ping for any reviews?. Thx.
Quoted 18 lines · Anand Jain
This patch series comprises two preparatory patches (patches 1 and 3 below), one bug fix (patch 2), and one logic hardening patch (patch 4) designed to address issues with stray single devices.
The objective is to ensure that single devices are not left lingering in the device list unless they are properly mounted.
Anand Jain (4):
btrfs: declarebtrfs_free_stale_devicesnon-static
btrfs: forget stray device on failed open
btrfs: refactorbtrfs_free_stale_devicesto free single stray device
btrfs: validatedevice_listat scan for stray freefs/btrfs/super.c | 3 +++ fs/btrfs/volumes.c | 18 ++++++++++++++---- fs/btrfs/volumes.h | 2 ++ 3 files changed, 19 insertions(+), 4 deletions(-)