btrfs: implement blk_holder_ops call backs
cover btrfs: implement blk_holder_ops call backs
Although test case generic/730 is going to be skipped for btrfs due to the missing shutdown support, it still exposed a problem that btrfs has no implementation for blk_holder_ops, thus even if lower level driver wants to notify btrfs there is a device that is going to be removed, btrfs can do nothing but ignore such critical info.
This series will implement all 4 callbacks for blk_holder_ops, 3 of them are straightforward, just call the full-fs version of sync/freeze/unfreeze.
The trickier one is the mark_dead(), all other fses with shutdown support will shutdown the fs, making all operations to return error, even if there is a cached page.
For btrfs we do not yet have full shutdown support, but at least we can notify such problem through dmesg, then check if the fs can still maintain its RW operations.
If not then mark the fs error, so no more new writes will happen, preventing further data loss.
Now btrfs will output something like this if the only device of a btrfs is removed, instead of aborting transaction later due to IO error:
BTRFS info (device sda): devid 1 device sda path /dev/sda is going to be removed
BTRFS: error (device sda) in btrfs_dev_mark_dead:294: errno=-5 IO failure (btrfs can no longer maintain read-write due to missing device(s)) BTRFS info (device sda state E): forced readonly
Future full-fs shutdown will depend on this feature to properly pass generic/730.
Qu Wenruo (2):
btrfs: use fs_info as the block device holder
btrfs: add a simple dead device detection mechanism
fs/btrfs/dev-replace.c | 2 +- fs/btrfs/fs.h | 2 - fs/btrfs/super.c | 7 ++-- fs/btrfs/super.h | 2 + fs/btrfs/volumes.c | 90 ++++++++++++++++++++++++++++++++++++++++-- fs/btrfs/volumes.h | 6 +++ 6 files changed, 99 insertions(+), 10 deletions(-)
Signature · 1 line
2.49.0
PATCH 1/2 btrfs: use fs_info as the block device holder
Currently btrfs uses "btrfs_fs_type" as the bdev holder for all opened device, which means all btrfses shares the same holder value.
That's only fine when there is no blk_holder_ops provided, but we're going to implement blk_holder_ops soon, so replace the "btrfs_fs_type" holder usage, and replace it with a proper fs_info instead.
This means we can remove the btrfs_fs_info::bdev_holder completely.
--- fs/btrfs/dev-replace.c | 2 +- fs/btrfs/fs.h | 2 -- fs/btrfs/super.c | 3 +-- fs/btrfs/volumes.c | 4 ++-- 4 files changed, 4 insertions(+), 7 deletions(-)
fs/btrfs/dev-replace.c +1 -1
@@ -250,7 +250,7 @@ static int btrfs_init_dev_replace_tgtdev(struct btrfs_fs_info *fs_info, } bdev_file =bdev_file_open_by_path (device_path,BLK_OPEN_WRITE , - fs_info->bdev_holder,NULL ); + fs_info,NULL );if (IS_ERR (bdev_file)) {btrfs_err (fs_info,"target device %s is invalid!" , device_path);return PTR_ERR (bdev_file);
fs/btrfs/fs.h +0 -2
@@ -715,8 +715,6 @@ struct btrfs_fs_info {u32 data_chunk_allocations;u32 metadata_ratio; -void *bdev_holder; -/* Private scrub information */ struct mutex scrub_lock;atomic_t scrubs_running;
fs/btrfs/super.c +1 -2
@@ -1865,7 +1865,7 @@ static int btrfs_get_tree_super(struct fs_context *fc) fs_devices = device->fs_devices; fs_info->fs_devices = fs_devices; - ret =btrfs_open_devices (fs_devices, mode, &btrfs_fs_type); + ret =btrfs_open_devices (fs_devices, mode, fs_info);mutex_unlock (&uuid_mutex);if (ret)return ret;
@@ -1905,7 +1905,6 @@ static int btrfs_get_tree_super(struct fs_context *fc) }else {snprintf (sb->s_id,sizeof (sb->s_id),"%pg" , bdev);shrinker_debugfs_rename (sb->s_shrink,"sb-btrfs:%s" , sb->s_id); -btrfs_sb (sb)->bdev_holder = &btrfs_fs_type; ret =btrfs_fill_super (sb, fs_devices);if (ret) {deactivate_locked_super (sb);
fs/btrfs/volumes.c +2 -2
@@ -2705,7 +2705,7 @@ int btrfs_init_new_device(struct btrfs_fs_info *fs_info, const char *device_pathreturn -EROFS ; bdev_file =bdev_file_open_by_path (device_path,BLK_OPEN_WRITE , - fs_info->bdev_holder,NULL ); + fs_info,NULL );if (IS_ERR (bdev_file))return PTR_ERR (bdev_file);
@@ -7174,7 +7174,7 @@ static struct btrfs_fs_devices *open_seed_devices(struct btrfs_fs_info *fs_info,if (IS_ERR (fs_devices))return fs_devices; - ret =open_fs_devices (fs_devices,BLK_OPEN_READ , fs_info->bdev_holder); + ret =open_fs_devices (fs_devices,BLK_OPEN_READ , fs_info);if (ret) {free_fs_devices (fs_devices);return ERR_PTR (ret);
Signature · 1 line
2.49.0
PATCH 2/2 btrfs: add a simple dead device detection mechanism
In reply to Qu WenruoCurrently btrfs doesn't have a real mechanism to detect a dead device at runtime.
This makes btrfs to treat intentionally or unintentionally removed device as usual, making test case generic/730 to fail as btrfs still return the cached data from page cache.
(The root cause is btrfs has no shutdown support for test cases requiring shutdown)
Add a very basic and simple dead device detection mechanism for btrfs, it utilized the blk_holder_ops for all opened btrfs devices, if a device is removed unexpectedly or is going to be removed, btrfs will:
- Output a warning or info line for that device
- Mark that device as missing
- If the fs can not maintain RW operations mark the fs as error
Furthermore to grab the btrfs_device just from a block device, introduce one extra member, btrfs_dev_lookup_args::devt.
Unlike other members, if this member is provided it's enough to uniquely locate a btrfs_device without any other members populated.
--- fs/btrfs/dev-replace.c | 2 +- fs/btrfs/super.c | 4 +- fs/btrfs/super.h | 2 + fs/btrfs/volumes.c | 88 +++++++++++++++++++++++++++++++++++++++++- fs/btrfs/volumes.h | 6 +++ 5 files changed, 97 insertions(+), 5 deletions(-)
fs/btrfs/dev-replace.c +1 -1
@@ -250,7 +250,7 @@ static int btrfs_init_dev_replace_tgtdev(struct btrfs_fs_info *fs_info, } bdev_file =bdev_file_open_by_path (device_path,BLK_OPEN_WRITE , - fs_info,NULL ); + fs_info, &btrfs_bdev_ops);if (IS_ERR (bdev_file)) {btrfs_err (fs_info,"target device %s is invalid!" , device_path);return PTR_ERR (bdev_file);
fs/btrfs/super.c +2 -2
@@ -2273,7 +2273,7 @@ static long btrfs_control_ioctl(struct file *file, unsigned int cmd,return ret; } -static int btrfs_freeze (struct super_block *sb) +int btrfs_freeze (struct super_block *sb) {struct btrfs_fs_info *fs_info =btrfs_sb (sb);
@@ -2339,7 +2339,7 @@ static int check_dev_super(struct btrfs_device *dev)return ret; } -static int btrfs_unfreeze (struct super_block *sb) +int btrfs_unfreeze (struct super_block *sb) {struct btrfs_fs_info *fs_info =btrfs_sb (sb);struct btrfs_device *device;
fs/btrfs/super.h +2 -0
@@ -14,6 +14,8 @@ bool btrfs_check_options(const struct btrfs_fs_info *info,unsigned long long *mount_opt, unsigned long flags);int btrfs_sync_fs (struct super_block *sb,int wait); +int btrfs_freeze (struct super_block *sb); +int btrfs_unfreeze (struct super_block *sb);char *btrfs_get_subvol_name_from_objectid (struct btrfs_fs_info *fs_info,u64 subvol_objectid);void btrfs_set_free_space_cache_settings (struct btrfs_fs_info *fs_info);
fs/btrfs/volumes.c +86 -2
112 lines not on this page · raw message
fs/btrfs/volumes.h +6 -0
@@ -607,6 +607,7 @@ struct btrfs_raid_attr { };extern const struct btrfs_raid_attr btrfs_raid_array[BTRFS_NR_RAID_TYPES ]; +extern const struct blk_holder_ops btrfs_bdev_ops;struct btrfs_chunk_map {struct rb_node rb_node;
@@ -652,6 +653,11 @@ struct btrfs_dev_lookup_args {u64 devid;u8 *uuid;u8 *fsid; +/* +* If @devt is set (non-zero), then args will be ignored since the +* non-zero dev_t can locate the device. +*/ +dev_t devt;bool missing; };
Signature · 1 line
2.49.0
Quoted 27 lines · Qu Wenruo
Although test case generic/730 is going to be skipped for btrfs due to the missing shutdown support, it still exposed a problem that btrfs has no implementation for
blk_holder_ops, thus even if lower level driver wants to notify btrfs there is a device that is going to be removed, btrfs can do nothing but ignore such critical info.This series will implement all 4 callbacks for
blk_holder_ops, 3 of them are straightforward, just call the full-fs version of sync/freeze/unfreeze.The trickier one is the
mark_dead(), all other fses with shutdown support will shutdown the fs, making all operations to return error, even if there is a cached page.For btrfs we do not yet have full shutdown support, but at least we can notify such problem through dmesg, then check if the fs can still maintain its RW operations.
If not then mark the fs error, so no more new writes will happen, preventing further data loss.Now btrfs will output something like this if the only device of a btrfs is removed, instead of aborting transaction later due to IO error:
BTRFS info (device sda): devid 1 device sda path
/dev/sdais going to be removed
BTRFS: error (device sda) in btrfs_dev_mark_dead:294: errno=-5 IO failure (btrfs can no longer maintain read-write due to missing device(s)) BTRFS info (device sda state E): forced readonly
Quoted 6 lines
Future full-fs shutdown will depend on this feature to properly pass generic/730.
Qu Wenruo (2):
btrfs: usefs_infoas the block device holder
btrfs: add a simple dead device detection mechanism
The series is triggering crash at generic/085.
It looks like it may not be a good idea to use fs_info directly as the block device holder, under heavy mount/unmount and per-device freeze/unfreeze workload, this can lead to various use-after-free problems.
Will get this reworked.
Thanks,
Qu
Quoted 9 lines
fs/btrfs/dev-replace.c | 2 +- fs/btrfs/fs.h | 2 - fs/btrfs/super.c | 7 ++-- fs/btrfs/super.h | 2 + fs/btrfs/volumes.c | 90 ++++++++++++++++++++++++++++++++++++++++-- fs/btrfs/volumes.h | 6 +++ 6 files changed, 99 insertions(+), 10 deletions(-)