Misc cleanups around device addition
cover Misc cleanups around device addition
Here are 4 minor cleanups around btrfs_init_new_device. First one converts a readonly usage of device_lists to using RCU. Second patch slightly simplifies the logic around locking when seed device is used. Third one removes a leftover from rewrite of btrfs_free_stale_devices. And the final one replaces an opencoded filemap_write_and_wait on the bdev inode with the more consistent sync_blockdev.
All these have passed xfstest and don't constitute functional changes.
Nikolay Borisov (4):
btrfs: Use rcu when iterating devices in btrfs_init_new_device
btrfs: Refactor locked condition in btrfs_init_new_device
btrfs: Remove redundant code from btrfs_free_stale_devices
btrfs: Don't opencode sync_blockdev
fs/btrfs/volumes.c | 20 +++++++++----------- 1 file changed, 9 insertions(+), 11 deletions(-)
--
2.17.1
PATCH 1/4 btrfs: Use rcu when iterating devices in btrfs_init_new_device
When adding a new device there's a mandatory check to see if a device is being duplicated to the filesystem it's added to. Since this is a read-only operations not necessary to take device_list_mutex and can simply make do with an rcu-readlock. No semantics changes.
--- fs/btrfs/volumes.c | 9 ++++----- 1 file changed, 4 insertions(+), 5 deletions(-)
fs/btrfs/volumes.c +4 -5
@@ -2502,16 +2502,15 @@ int btrfs_init_new_device(struct btrfs_fs_info *fs_info, const char *device_pathfilemap_write_and_wait (bdev->bd_inode->i_mapping); -mutex_lock (&fs_devices->device_list_mutex); -list_for_each_entry (device, &fs_devices->devices, dev_list) { +rcu_read_lock (); +list_for_each_entry_rcu (device, &fs_devices->devices, dev_list) {if (device->bdev == bdev) { ret = -EEXIST ; -mutex_unlock ( - &fs_devices->device_list_mutex); +rcu_read_unlock (); goto error; } } -mutex_unlock (&fs_devices->device_list_mutex); +rcu_read_unlock (); device =btrfs_alloc_device (fs_info,NULL ,NULL );if (IS_ERR (device)) {
Signature · 1 line
2.17.1
Re: [PATCH 1/4] btrfs: Use rcu when iterating devices in btrfs_init_new_device
Quoted 4 lines · Nikolay Borisov
When adding a new device there's a mandatory check to see if a device is being duplicated to the filesystem it's added to. Since this is a read-only operations not necessary to take
device_list_mutexand can simply make do with an rcu-readlock. No semantics changes.
Hmm what are the actual locking rules for the device list? For instance looking at add_missing_dev() in volumes.c addition to the list is unprotected (both from read_one_chunk() and read_one_dev()). Others (i.e. btrfs_init_new_device()) do a list_add_rcu(). clone_fs_devices() takes the device_list_mutex and so on.
Thanks,
Johannes
Re: [PATCH 1/4] btrfs: Use rcu when iterating devices in btrfs_init_new_device
Quoted 10 lines · Johannes Thumshirn
Quoted 4 lines · Nikolay Borisov
When adding a new device there's a mandatory check to see if a device is being duplicated to the filesystem it's added to. Since this is a read-only operations not necessary to take
device_list_mutexand can simply make do with an rcu-readlock. No semantics changes.Hmm what are the actual locking rules for the device list? For instance looking at
add_missing_dev()involumes.caddition to the list is unprotected (both fromread_one_chunk()andread_one_dev()). Others (i.e.btrfs_init_new_device()) do alist_add_rcu().clone_fs_devices()takes thedevice_list_mutexand so on.
Bummer, the locking rules are supposed to be those documented atop volume.c, namely:
* fs_devices::device_list_mutex (per-fs, with RCU)
18 * ------------------------------------------------
17 * protects updates to fs_devices::devices, ie. adding and
deleting
16 *
15 * simple list traversal with read-only actions can be done with
RCU protection
14 *
13 * may be used to exclude some operations from running
concurrently without any
12 * modifications to the list (see write_all_supers)
However, your observations seem correct and rather annoying. Let me go and try and figure this out...
Quoted 4 lines
Thanks,
Johannes
Re: [PATCH 1/4] btrfs: Use rcu when iterating devices in btrfs_init_new_device
Quoted 29 lines · Nikolay Borisov
Quoted 10 lines · Johannes Thumshirn
Quoted 4 lines · Nikolay Borisov
When adding a new device there's a mandatory check to see if a device is being duplicated to the filesystem it's added to. Since this is a read-only operations not necessary to take
device_list_mutexand can simply make do with an rcu-readlock. No semantics changes.Hmm what are the actual locking rules for the device list? For instance looking at
add_missing_dev()involumes.caddition to the list is unprotected (both fromread_one_chunk()andread_one_dev()). Others (i.e.btrfs_init_new_device()) do alist_add_rcu().clone_fs_devices()takes thedevice_list_mutexand so on.Bummer, the locking rules are supposed to be those documented atop
volume.c, namely:*
fs_devices::device_list_mutex(per-fs, with RCU)18 * ------------------------------------------------
17 * protects updates to
fs_devices::devices, ie. adding and
deleting
16 *15 * simple list traversal with read-only actions can be done with
RCU protection
14 *
Quoted 8 lines
13 * may be used to exclude some operations from running
concurrently without any
12 * modifications to the list (seewrite_all_supers)However, your observations seem correct and rather annoying. Let me go and try and figure this out...
For device list it's important to know from which context is the list used.
On unmoutned filesystems, the devices can be added by scanning ioctl.
On mounted filesystem, before the mount is finished, the devices cannot be concurrently used (single-threaded context) and uuid_mutex is temporarily protecting the devices agains scan ioctl.
On finished mount device list must be protected by device_list_mutex from the same filesystem changes (ioctls, superblock write). Device scanning is a no-op here, but still needs to use the uuid_mutex.
Enter RCU. For performance reasons we don't want to take device_list_mutex for read-only operations like show_devname or lookups of device id, where it's not critical that the returned information is stale.
The mentioned helpers are used by various functions and the context is not obvious or documented, but it should be clear once the caller chain is known.
I can turn that into comments but please let me know if this is sufficient as explanation or if you need more.
Re: [PATCH 1/4] btrfs: Use rcu when iterating devices in btrfs_init_new_device
Quoted 48 lines · David Sterba
Quoted 29 lines · Nikolay Borisov
Quoted 10 lines · Johannes Thumshirn
Quoted 4 lines · Nikolay Borisov
When adding a new device there's a mandatory check to see if a device is being duplicated to the filesystem it's added to. Since this is a read-only operations not necessary to take
device_list_mutexand can simply make do with an rcu-readlock. No semantics changes.Hmm what are the actual locking rules for the device list? For instance looking at
add_missing_dev()involumes.caddition to the list is unprotected (both fromread_one_chunk()andread_one_dev()). Others (i.e.btrfs_init_new_device()) do alist_add_rcu().clone_fs_devices()takes thedevice_list_mutexand so on.Bummer, the locking rules are supposed to be those documented atop
volume.c, namely:*
fs_devices::device_list_mutex(per-fs, with RCU)18 * ------------------------------------------------
17 * protects updates to
fs_devices::devices, ie. adding and
deleting
16 *15 * simple list traversal with read-only actions can be done with
RCU protection
14 *Quoted 8 lines
13 * may be used to exclude some operations from running
concurrently without any
12 * modifications to the list (seewrite_all_supers)However, your observations seem correct and rather annoying. Let me go and try and figure this out...
For device list it's important to know from which context is the list used.
On unmoutned filesystems, the devices can be added by scanning ioctl.
On mounted filesystem, before the mount is finished, the devices cannot be concurrently used (single-threaded context) and
uuid_mutexis temporarily protecting the devices agains scan ioctl.
Quoted 8 lines
On finished mount device list must be protected by
device_list_mutexfrom the same filesystem changes (ioctls, superblock write). Device scanning is a no-op here, but still needs to use theuuid_mutex.Enter RCU. For performance reasons we don't want to take
device_list_mutexfor read-only operations likeshow_devnameor lookups of device id, where it's not critical that the returned information is stale.
OK I got confused by a) the mix of unlocked, device_list_mutex and uuid_mutex use and b) sometimes we use plain list_* operations and sometimes rcu list ones.
Quoted 7 lines
The mentioned helpers are used by various functions and the context is not obvious or documented, but it should be clear once the caller chain is known.
I can turn that into comments but please let me know if this is sufficient as explanation or if you need more.
This would be great yes.
Re: [PATCH 1/4] btrfs: Use rcu when iterating devices in btrfs_init_new_device
In reply to David SterbaQuoted 57 lines · David Sterba
Quoted 29 lines · Nikolay Borisov
Quoted 10 lines · Johannes Thumshirn
Quoted 4 lines · Nikolay Borisov
When adding a new device there's a mandatory check to see if a device is being duplicated to the filesystem it's added to. Since this is a read-only operations not necessary to take
device_list_mutexand can simply make do with an rcu-readlock. No semantics changes.Hmm what are the actual locking rules for the device list? For instance looking at
add_missing_dev()involumes.caddition to the list is unprotected (both fromread_one_chunk()andread_one_dev()). Others (i.e.btrfs_init_new_device()) do alist_add_rcu().clone_fs_devices()takes thedevice_list_mutexand so on.Bummer, the locking rules are supposed to be those documented atop
volume.c, namely:*
fs_devices::device_list_mutex(per-fs, with RCU)18 * ------------------------------------------------
17 * protects updates to
fs_devices::devices, ie. adding and
deleting
16 *15 * simple list traversal with read-only actions can be done with
RCU protection
14 *Quoted 8 lines
13 * may be used to exclude some operations from running
concurrently without any
12 * modifications to the list (seewrite_all_supers)However, your observations seem correct and rather annoying. Let me go and try and figure this out...
For device list it's important to know from which context is the list used.
On unmoutned filesystems, the devices can be added by scanning ioctl.
On mounted filesystem, before the mount is finished, the devices cannot be concurrently used (single-threaded context) and
uuid_mutexis temporarily protecting the devices agains scan ioctl.On finished mount device list must be protected by
device_list_mutexfrom the same filesystem changes (ioctls, superblock write). Device scanning is a no-op here, but still needs to use theuuid_mutex.Enter RCU. For performance reasons we don't want to take
device_list_mutexfor read-only operations likeshow_devnameor lookups of device id, where it's not critical that the returned information is stale.
Quoted 7 lines
The mentioned helpers are used by various functions and the context is not obvious or documented, but it should be clear once the caller chain is known.
I can turn that into comments but please let me know if this is sufficient as explanation or if you need more.
Yes having those different contexts spelled out is really beneficial.
PATCH 2/4 btrfs: Refactor locked condition in btrfs_init_new_device
In reply to Nikolay BorisovInvert unlocked to locked and exploit the fact it can only ever be modified if we are adding a new device to a seed filesystem. This allows to simplify the check in error: label. No semantics changes.
--- fs/btrfs/volumes.c | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-)
fs/btrfs/volumes.c +4 -3
@@ -2484,7 +2484,7 @@ int btrfs_init_new_device(struct btrfs_fs_info *fs_info, const char *device_pathu64 orig_super_num_devices;int seeding_dev =0 ;int ret =0 ; -bool unlocked = false; +bool locked = false;if (sb_rdonly (sb) && !fs_devices->seeding)return -EROFS ;
@@ -2498,6 +2498,7 @@ int btrfs_init_new_device(struct btrfs_fs_info *fs_info, const char *device_path seeding_dev =1 ;down_write (&sb->s_umount);mutex_lock (&uuid_mutex); + locked = true; }filemap_write_and_wait (bdev->bd_inode->i_mapping);
@@ -2629,7 +2630,7 @@ int btrfs_init_new_device(struct btrfs_fs_info *fs_info, const char *device_pathif (seeding_dev) {mutex_unlock (&uuid_mutex);up_write (&sb->s_umount); - unlocked = true; + locked = false;if (ret)/* transaction commit */ return ret;
@@ -2690,7 +2691,7 @@ int btrfs_init_new_device(struct btrfs_fs_info *fs_info, const char *device_pathbtrfs_free_device (device); error:blkdev_put (bdev,FMODE_EXCL ); -if (seeding_dev && !unlocked) { +if (locked) {mutex_unlock (&uuid_mutex);up_write (&sb->s_umount); }
Signature · 1 line
2.17.1
Re: [PATCH 2/4] btrfs: Refactor locked condition in btrfs_init_new_device
Quoted 5 lines · Nikolay Borisov
Invert unlocked to locked and exploit the fact it can only ever be modified if we are adding a new device to a seed filesystem. This allows to simplify the check in error: label. No semantics changes.
Signed-off-by Nikolay Borisov <[email protected]>
PATCH 3/4 btrfs: Remove redundant code from btrfs_free_stale_devices
In reply to Nikolay BorisovFollowing the refactor of btrfs_free_stale_devices in 7bcb8164ad94 ("btrfs: use device_list_mutex when removing stale devices") fs_devices are freed after they have been iterated by the inner list_for_each so the UAF fixed by introducing the break in fd649f10c3d2 ("btrfs: Fix use-after-free when cleaning up fs_devs with a single stale device") is no longer necessary. Just remove it altogether. No functional changes.
--- fs/btrfs/volumes.c | 2 -- 1 file changed, 2 deletions(-)
fs/btrfs/volumes.c +0 -2
@@ -588,8 +588,6 @@ static int btrfs_free_stale_devices(const char *path,btrfs_free_device (device); ret =0 ; -if (fs_devices->num_devices ==0 ) -break ; }mutex_unlock (&fs_devices->device_list_mutex);
Signature · 1 line
2.17.1
Re: [PATCH 3/4] btrfs: Remove redundant code from btrfs_free_stale_devices
Quoted 19 lines · Nikolay Borisov
Following the refactor of
btrfs_free_stale_devicesin7bcb8164ad94("btrfs: usedevice_list_mutexwhen removing stale devices")fs_devicesare freed after they have been iterated by the innerlist_for_eachso the UAF fixed by introducing the break infd649f10c3d2("btrfs: Fix use-after-free when cleaning upfs_devswith a single stale device") is no longer necessary. Just remove it altogether. No functional changes.Signed-off-by Nikolay Borisov <[email protected]>--- fs/btrfs/volumes.c | 2 -- 1 file changed, 2 deletions(-)fs/btrfs/volumes.c +0
-0
fs/btrfs/volumes.c static int btrfs_free_stale_devices(const char *path, · line 593
Quoted 8 lines
fs/btrfs/volumes.c +0
-2@@ -588,8 +588,6 @@ static int btrfs_free_stale_devices(const char *path,btrfs_free_device (device); ret =0 ; -if (fs_devices->num_devices ==0 ) -break ; }mutex_unlock (&fs_devices->device_list_mutex);
Thanks, Anand
PATCH 4/4 btrfs: Don't opencode sync_blockdev
In reply to Nikolay BorisovInstead of opencoding filemap_write_and_wait simply call syncblockdev as it makes it abundandtly clear what's going on and why this is used. No semantics changes.
--- fs/btrfs/volumes.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
fs/btrfs/volumes.c +1 -1
@@ -2499,7 +2499,7 @@ int btrfs_init_new_device(struct btrfs_fs_info *fs_info, const char *device_path locked = true; } -filemap_write_and_wait (bdev->bd_inode->i_mapping); +sync_blockdev (bdev);rcu_read_lock ();list_for_each_entry_rcu (device, &fs_devices->devices, dev_list) {
Signature · 1 line
2.17.1
Re: [PATCH 4/4] btrfs: Don't opencode sync_blockdev
Quoted 1 line · Nikolay Borisov
Instead of opencoding
filemap_write_and_waitsimply call syncblockdev as
sync_blockdev() ~^
Quoted 1 line
it makes it abundandtly clear what's going on and why this is used. No
abundantly ~^
Otherwise:
Re: [PATCH 4/4] btrfs: Don't opencode sync_blockdev
In reply to Nikolay BorisovQuoted 14 lines · Nikolay Borisov
Instead of opencoding
filemap_write_and_waitsimply call syncblockdev as it makes it abundandtly clear what's going on and why this is used. No semantics changes.Signed-off-by Nikolay Borisov <[email protected]>--- fs/btrfs/volumes.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)fs/btrfs/volumes.c +0
-0
fs/btrfs/volumes.c int btrfs_init_new_device(struct btrfs_fs_info *fs_info, const char *device_path · line 2502
Quoted 5 lines
fs/btrfs/volumes.c +1
-1@@ -2499,7 +2499,7 @@ int btrfs_init_new_device(struct btrfs_fs_info *fs_info, const char *device_path locked = true; } -filemap_write_and_wait (bdev->bd_inode->i_mapping); +sync_blockdev (bdev);
Thanks, Anand
Quoted 4 lines
fs/btrfs/volumes.c +0
-0rcu_read_lock ();list_for_each_entry_rcu (device, &fs_devices->devices, dev_list) {
Quoted 15 lines · Nikolay Borisov
Here are 4 minor cleanups around
btrfs_init_new_device. First one converts a readonly usage ofdevice_liststo using RCU. Second patch slightly simplifies the logic around locking when seed device is used. Third one removes a leftover from rewrite ofbtrfs_free_stale_devices. And the final one replaces an opencodedfilemap_write_and_waiton the bdev inode with the more consistentsync_blockdev.All these have passed xfstest and don't constitute functional changes.
Nikolay Borisov (4):
btrfs: Use rcu when iterating devices inbtrfs_init_new_device
btrfs: Refactor locked condition inbtrfs_init_new_device
btrfs: Remove redundant code frombtrfs_free_stale_devices
btrfs: Don't opencodesync_blockdev
Quoted 6 lines
fs/btrfs/volumes.c | 20 +++++++++----------- 1 file changed, 9 insertions(+), 11 deletions(-)--
2.17.1
Ping
Quoted 7 lines · Nikolay Borisov
Here are 4 minor cleanups around
btrfs_init_new_device. First one converts a readonly usage ofdevice_liststo using RCU. Second patch slightly simplifies the logic around locking when seed device is used. Third one removes a leftover from rewrite ofbtrfs_free_stale_devices. And the final one replaces an opencodedfilemap_write_and_waiton the bdev inode with the more consistentsync_blockdev.
Quoted 7 lines
All these have passed xfstest and don't constitute functional changes.
Nikolay Borisov (4):
btrfs: Use rcu when iterating devices inbtrfs_init_new_device
btrfs: Refactor locked condition inbtrfs_init_new_device
btrfs: Remove redundant code frombtrfs_free_stale_devices
btrfs: Don't opencodesync_blockdev
Added to for-next, thanks. I've updated patch 1 with explanation why RCU is safe.