Add support for storage element depopulation
Subsystems (7)
Damien Le Moal
maintainerv12 cover Add support for storage element depopulation
Jens,
These patches define a new set of block device operations for generically using from the block layer the storage element depopulation feature of zoned block devices. Support for this feature is added to the SCSI disk driver and an emulation of this feature added to the zloop driver.
The management operations can be accessed using block layer API, which is intended for file systems (e.g. zonefs and XFS), as well as using ioctls for users using zoned disks directly from user space.
Changes from v11:
- Addressed two silly mistakes pointed out by Sashiko in patch 8 (missing EIO error return) and patch 12 (dead code left over from previous version).
Changes from v10:
- Addressed sashiko comment on patch 8 and 12.
- Added review tags
Changes from v9:
- Addressed Sashiko comments on patch 6. This change redesigns the ordered zone append handling to simplify the code and avoid locking difficulties.
- Addressed Sashiko comments on patch 8.
Changes from v8:
- Addressed Sashiko comment on patch 7 by adding patch 6. Patch 7 is unmodified.
- Addressed Sashiko comments on patch 8 and patch 12
- Added review tags to unmodified pacthes
Changes from v7:
A significant amount of changes in this version to address most of the Sashiko review comments and to simplify the code. In more details, the changes are:
- Added patches 3, 6, and 9.
- Dropped former patch 5 as the use of irq-disabling spinlocks in zloop is not necessary.
- Reworked patch 4 by migrating the idepop operation wait loops to the device drivers. This simplifies the implementation in zloop without making the sd driver implementation more difficult. Also fixed problems with errors during restoration and zone write plugs state.
- Reworked patch 7 and 11 to be consistent with the changes in patch 4.
Changes from v6:
- Addressed the nit pointed out by Christoph in patch 4
- Added review tags
Changes from v5:
- Rebased on latest block/for-next
- Addressed Christoph's comments on patch 3
- Added patch 5 to address a potential deadlock in zloop (signaled by sashiko)
- Reworked the restoration of storage elements in patch 6 to avoid an inconsistent state of storage elements in case of error. Also added protection against disk removal during storage element removal and restoration execution.
- Added review tags
Changes from v4:
- Rebased on latest block/for-next
- Added review tags to patch 1 and 2
- Reworked patch 3, 4 and 5 to address Christoph's and sashiko comments
Changes from v3:
- Reworked patch 1 to only provide the disk_zone_is_offline() helper function. Failing of read operations to offline zones is handled directly by the zloop driver and the new patch 7 is added to handle failing early read operations to offline zones of ZBC SCSI disks.
- Added patch 2
- Addressed further valid comments from Sashiko
The biggest changes are in patch 2 to save the number of storage elements of a disk in struct gendisk and to serialize the execution of the storage element management operations using the zone revalidation mutex.
- Rebased on latest block/for-next
Changes from v2:
- Addressed further valid comments from Sashiko
- Rebased on latest block/for-next
Changes from v1:
- Addressed the many valid comments that Sashiko had
- Rebased on latest block/for-next
Damien Le Moal (13):
block: introduce disk_zone_is_offline()
block: refactor blk_revalidate_disk_zones()
block: allow dropping all zone write plugs on revalidation
block: introduce storage element management
block: add storage element management ioctls
zloop: fix ordered zone append handling
zloop: fix zone reporting consistency
zloop: add storage element emulation
zloop: add degrade_element control command
scsi: scsi_proto: refine the definition of storage element health
scsi: sd_zbc: always revalidate zones for disks supporting head depopulation
scsi: sd_zbc: define storage element management operations
scsi: sd: fail read commands to offline zones early
.../admin-guide/blockdev/zoned_loop.rst | 36 +- block/blk-zoned.c | 450 ++++++++- block/blk.h | 14 + block/genhd.c | 4 + block/ioctl.c | 4 + drivers/block/zloop.c | 919 ++++++++++++++++-- drivers/scsi/sd.c | 23 +- drivers/scsi/sd.h | 5 + drivers/scsi/sd_zbc.c | 524 +++++++++- include/linux/blkdev.h | 64 ++ include/scsi/scsi_proto.h | 6 +- include/uapi/linux/blkzoned.h | 106 ++ include/uapi/linux/fs.h | 1 + 13 files changed, 2027 insertions(+), 129 deletions(-)
Signature · 1 line
2.56.1
Damien Le Moal
maintainerPATCH v12 1/13 block: introduce disk_zone_is_offline()
Introduce the helper function disk_zone_is_offline() for testing if a zone of a gendisk has the offline condition. The zone to test is identified using a sector number that must belong to the target zone to check.
--- block/blk-zoned.c | 18 ++++++++++++++++++ include/linux/blkdev.h | 9 +++++++++ 2 files changed, 27 insertions(+)
block/blk-zoned.c +18 -0
@@ -292,6 +292,24 @@ static bool disk_zone_is_seq(struct gendisk *disk, sector_t sector)return !blk_zstate_is_conv (zs); } +/** +* disk_zone_is_offline - check if a sector belongs to an offline zone +* @disk: gendisk to check +* @sector: sector number +* +* Check if @sector on @disk is contained in an offline zone. +* This function can be used only when the bdev queue was entered (BIOs +* submitted but not yet completed) or when holding zone_revalidate_mutex. +*/ +bool disk_zone_is_offline (struct gendisk *disk,sector_t sector) +{ +if (!blk_queue_is_zoned (disk->queue)) +return false; + +return disk_zone_get_cond (disk, sector) ==BLK_ZONE_COND_OFFLINE ; +} +EXPORT_SYMBOL_GPL (disk_zone_is_offline); +/** * bdev_zone_is_seq - check if a sector belongs to a sequential write zone * @bdev: block device to check
include/linux/blkdev.h +9 -0
@@ -940,6 +940,9 @@ static inline unsigned int disk_zone_capacity(struct gendisk *disk,return disk->last_zone_capacity;return disk->zone_capacity; } + +bool disk_zone_is_offline (struct gendisk *disk,sector_t sector); +static inline unsigned int bdev_zone_capacity (struct block_device *bdev,sector_t pos) {
@@ -954,6 +957,12 @@ static inline unsigned int disk_nr_zones(struct gendisk *disk)return 0 ; } +static inline bool disk_zone_is_offline (struct gendisk *disk, +sector_t sector) +{ +return false; +} +static inline bool bdev_zone_is_seq (struct block_device *bdev,sector_t sector) {return false;
Signature · 1 line
2.56.1
Damien Le Moal
maintainerPATCH v12 2/13 block: refactor blk_revalidate_disk_zones()
In reply to Damien Le MoalSplit out most of the code of blk_revalidate_disk_zones() into the internal function disk_revalidate_zones(). This will allow calling this new helper together with other code under the zone revalidation mutex. No functional change.
--- block/blk-zoned.c | 63 +++++++++++++++++++++++++---------------------- 1 file changed, 34 insertions(+), 29 deletions(-)
block/blk-zoned.c +34 -29
@@ -2552,20 +2552,7 @@ static int blk_revalidate_zone_cb(struct blk_zone *zone, unsigned int idx,return ret; } -/** -* blk_revalidate_disk_zones - (re)allocate and initialize zone write plugs -* @disk: Target disk -* -* Helper function for low-level device drivers to check, (re) allocate and -* initialize resources used for managing zoned disks. This function should -* normally be called by blk-mq based drivers when a zoned gendisk is probed -* and when the zone configuration of the gendisk changes (e.g. after a format). -* Before calling this function, the device driver must already have set the -* device zone size (chunk_sector limit) and the max zone append limit. -* BIO based drivers can also use this function as long as the device queue -* can be safely frozen. -*/ -int blk_revalidate_disk_zones (struct gendisk *disk) +static int disk_revalidate_zones (struct gendisk *disk) {struct blk_revalidate_zone_args args = { };struct blk_report_zones_args rep_args = {
@@ -2576,19 +2563,10 @@ int blk_revalidate_disk_zones(struct gendisk *disk)int retries =2 ;int ret; -if (WARN_ON_ONCE (!blk_queue_is_zoned (disk->queue))) -return -EIO ; - -/* -* Serialize calls to this function so that we can safely look at and -* eventually change the disk zone information. -*/ -mutex_lock (&disk->zone_revalidate_mutex); - again: ret =disk_revalidate_capacity (disk, &args);if (ret) - goto unlock; +return ret;/* * Allocate zone resources if they are needed and we have not done
@@ -2601,14 +2579,14 @@ int blk_revalidate_disk_zones(struct gendisk *disk) ret =disk_alloc_zone_resources (disk, args.capacity);if (ret) {memalloc_noio_restore (noio_flag); - goto unlock; +return ret; } } ret =disk_init_revalidate_args (disk, &args);if (ret) {memalloc_noio_restore (noio_flag); - goto unlock; +return ret; } ret = disk->fops->report_zones (disk,0 ,UINT_MAX , &rep_args);
@@ -2625,8 +2603,6 @@ int blk_revalidate_disk_zones(struct gendisk *disk)if (ret) goto free_args; -mutex_unlock (&disk->zone_revalidate_mutex); -return 0 ; free_args:
@@ -2642,7 +2618,36 @@ int blk_revalidate_disk_zones(struct gendisk *disk) }pr_warn ("%s: failed to revalidate zones\n" , disk->disk_name); -unlock: + +return ret; +} + +/** +* blk_revalidate_disk_zones - (re)allocate and initialize zone write plugs +* @disk: Target disk +* +* Helper function for low-level device drivers to check, (re) allocate and +* initialize resources used for managing zoned disks. This function should +* normally be called by blk-mq based drivers when a zoned gendisk is probed +* and when the zone configuration of the gendisk changes (e.g. after a format). +* Before calling this function, the device driver must already have set the +* device zone size (chunk_sector limit) and the max zone append limit. +* BIO based drivers can also use this function as long as the device queue +* can be safely frozen. +*/ +int blk_revalidate_disk_zones (struct gendisk *disk) +{ +int ret; + +if (WARN_ON_ONCE (!blk_queue_is_zoned (disk->queue))) +return -EIO ; + +/* +* Serialize calls to this function so that we can safely look at and +* eventually change the disk zone information. +*/ +mutex_lock (&disk->zone_revalidate_mutex); + ret =disk_revalidate_zones (disk);mutex_unlock (&disk->zone_revalidate_mutex);return ret;
Signature · 1 line
2.56.1
Damien Le Moal
maintainerPATCH v12 3/13 block: allow dropping all zone write plugs on revalidation
In reply to Damien Le MoalIn preparation for supporting storage element depopulation, modify the function disk_revalidate_zones() to take an additional boolean parameter drop_all_zwplugs to force removing all zone write plugs from a disk hash table. Removing all zone write plugs is done in the same manner as for a capacity change, using the function disk_drop_zone_wplug() applied to all hashed zone write plugs.
Calls to disk_revalidate_zones() with the argument drop_all_zwplugs set to true will be used when restoring depopulated storage elements as such operation clears all sequential zones, modifying them to be empty.
--- block/blk-zoned.c | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-)
block/blk-zoned.c +5 -2
@@ -2552,7 +2552,7 @@ static int blk_revalidate_zone_cb(struct blk_zone *zone, unsigned int idx,return ret; } -static int disk_revalidate_zones (struct gendisk *disk) +static int disk_revalidate_zones (struct gendisk *disk,bool drop_all_zwplugs) {struct blk_revalidate_zone_args args = { };struct blk_report_zones_args rep_args = {
@@ -2563,6 +2563,9 @@ static int disk_revalidate_zones(struct gendisk *disk)int retries =2 ;int ret; +if (drop_all_zwplugs) +disk_for_all_zone_wplugs (disk, disk_drop_zone_wplug,NULL ); + again: ret =disk_revalidate_capacity (disk, &args);if (ret)
@@ -2647,7 +2650,7 @@ int blk_revalidate_disk_zones(struct gendisk *disk)* eventually change the disk zone information. */ mutex_lock (&disk->zone_revalidate_mutex); - ret =disk_revalidate_zones (disk); + ret =disk_revalidate_zones (disk, false);mutex_unlock (&disk->zone_revalidate_mutex);return ret;
Signature · 1 line
2.56.1
Damien Le Moal
maintainerPATCH v12 4/13 block: introduce storage element management
In reply to Damien Le MoalRecent SCSI (SBC) and ATA (ACS) standards define the storage element depopulation feature. This feature is intended for managing hard-disks heads, either combined read-write heads or pairs of read and write heads, allowing to keep disks with defective heads longer in production by allowing "depopulating" (removing) defective heads.
This feature comes in two different flavors:
- A destructive version which removes a head and reformats the disk at a lower capacity point, restoring a fully functional contiguous LBA address space.
- A data preserving version restricted to host-managed zoned disks, which marks the zones served by a removed head as offline or read-only.
In preparation for using the data-preserving flavor of the depopulation feature in file systems natively supporting zoned block devices, introduce a set of storage element management operations and functions to define generic calls into block device drivers for managing storage elements.
The set of operations is defined with struct blk_se_ops and includes three operations:
- report_elements: get information on a device storage elements state reported with the new struct blk_se.
- remove_element: depopulate a defective storage element, blocking until the operation completes.
- restore_elements: restore depopulated storage elements, blocking until the operation completes.
Each operation is called from the functions bdev_report_storage_elements(), bdev_remove_storage_element() and bdev_restore_storage_elements() respectively.
Removing a healthy storage element from a device is possible and useful for testing. The storage element restoration operation restore_elements allows repopulating such healthy element. Repopulating defective storage elements is generally not allowed by devices.
The remove_element and restore_elements operations are executed under the zone revalidation lock so that they are serialized against each other and also serialized against user initiated device revalidation, thus allowing to always have a consistent view of the device zone conditions after executing these operations. In case of failure, zone revalidation is still executed, which may give an unstable view of the zone states. In such case, stable zone states will be recovered with zone access error handling or later zone revalidation.
The device drivers of zoned block devices can indicate support for the storage element depopulation feature by specifying the storage element management operations with the se_ops field of struct block_device_operations.
--- block/blk-zoned.c | 205 ++++++++++++++++++++++++++++++++++ block/blk.h | 5 + block/genhd.c | 4 + include/linux/blkdev.h | 55 +++++++++ include/uapi/linux/blkzoned.h | 65 +++++++++++ 5 files changed, 334 insertions(+)
block/blk-zoned.c +205 -0
@@ -2736,3 +2736,208 @@ int queue_zone_wplugs_show(void *data, struct seq_file *m) }#endif + +/** +* bdev_report_storage_elements - report the storage elements of a zoned +* block device +* @bdev: the zoned block device to check +* @elements: array of storage element descriptors to fill +* @nr_elements: maximum size of @elements +* +* Fill at most @nr_elements storage element descriptors in the array +* @elements. The number of storage elements filled in the array is returned +* using @nr_elements. If @elements is NULL, only @nr_elements is returned to +* indicate the maximum number of storage elements of @bdev. This function may +* sleep. +* +* Returns 0 on success and a negative error code on failure. +*/ +int bdev_report_storage_elements (struct block_device *bdev, +struct blk_se *elements, +unsigned int *nr_elements) +{ +struct gendisk *disk = bdev->bd_disk; + +if (test_bit (GD_DEAD , &disk->state)) +return -ENODEV ; + +if (!blk_queue_is_zoned (disk->queue) || !disk->nr_storage_elements) +return -EOPNOTSUPP ; + +if (!nr_elements) +return -EINVAL ; + +if (!elements) { + *nr_elements = disk->nr_storage_elements; +return 0 ; + } + + *nr_elements =min (*nr_elements, disk->nr_storage_elements); +if (!*nr_elements) +return -EINVAL ; + +return disk->fops->se_ops->report_elements (disk, elements, + nr_elements); +} +EXPORT_SYMBOL_GPL (bdev_report_storage_elements); + +/** +* bdev_remove_storage_element - Remove (depopulate) a storage element of a +* zoned block device +* @bdev: block device to operate on +* @element_id: ID of the storage element to remove +* +* Remove (depopulate) the storage element identified by @element_id from the +* block device @bdev. The caller is responsible for taking care of any +* necessary dirty page and device write cache flush and invalidation of cached +* data. This function will block until the storage element is removed or an +* error occurs. +* +* Returns 0 on success and a negative error code on failure. +*/ +int bdev_remove_storage_element (struct block_device *bdev, +unsigned int element_id) +{ +struct gendisk *disk = bdev->bd_disk; +unsigned int memflags; +int ret, err; + +if (test_bit (GD_DEAD , &disk->state)) +return -ENODEV ; + +if (!blk_queue_is_zoned (disk->queue) || !disk->nr_storage_elements) +return -EOPNOTSUPP ; + +/* Zero is not a valid storage element ID. */ +if (!element_id) +return -EINVAL ; + +/* +* Serialize against concurrent revalidation and concurrent storage +* element management operations so that we can see consistent zone +* condition changes. +*/ +mutex_lock (&disk->zone_revalidate_mutex); + +/* +* Freeze and unfreeze the queue to flush any outstanding command. +* The caller is responsible for not queuing up more I/Os by higher +* level means. +*/ + memflags =blk_mq_freeze_queue (disk->queue); +blk_mq_unfreeze_queue (disk->queue, memflags); + +/* Initiate the storage element removal and wait for its completion. */ + ret = disk->fops->se_ops->remove_element (disk, element_id); + +/* +* Always revalidate zones, even in the case of an error as we do not +* know what condition the zones of the block device are in. +*/ + err =disk_revalidate_zones (disk, false); +if (err && !ret) + ret = err; + +mutex_unlock (&disk->zone_revalidate_mutex); + +return ret; +} +EXPORT_SYMBOL_GPL (bdev_remove_storage_element); + +/** +* bdev_restore_storage_elements - Restore all depopulated storage elements of +* a zoned block device +* @bdev: block device to operate on +* +* Restore all storage elements of @bdev that have been depopulated. Not all +* elements may be restored by this operation. This operation is destructive and +* will result in all sequential zones to be transitioned to the empty condition. +* The caller is responsible for taking care of any necessary invalidation of +* cached data. This function will block until the storage elements are restored +* or an error occurs. +* +* Returns 0 on success and a negative error code on failure. +*/ +int bdev_restore_storage_elements (struct block_device *bdev) +{ +struct gendisk *disk = bdev->bd_disk; +unsigned int memflags; +int ret, err; + +if (test_bit (GD_DEAD , &disk->state)) +return -ENODEV ; + +if (!blk_queue_is_zoned (disk->queue) || !disk->nr_storage_elements) +return -EOPNOTSUPP ; + +/* +* Serialize against concurrent revalidation and concurrent storage +* element management operations so that we can see consistent zone +* condition changes. +*/ +mutex_lock (&disk->zone_revalidate_mutex); + +/* +* Freeze and unfreeze the queue to flush any outstanding commands. +* The caller is responsible for not queuing up more I/O by higher +* level means. +*/ + memflags =blk_mq_freeze_queue (disk->queue); +blk_mq_unfreeze_queue (disk->queue, memflags); + +/* +* Initiate the storage elements restoration and wait for its +* completion. +*/ + ret = disk->fops->se_ops->restore_elements (disk); + +/* +* Always revalidate zones, even in the case of an error as we do not +* know what condition the zones of the block device are in. Note that +* since all sequential zones should have been transitioned to the +* empty condition, we can drop all zone write plugs. If an error +* occurred and not all sequential zones are empty, the revalidation +* will reinstate the write plugs for these zones. +*/ + err =disk_revalidate_zones (disk, true); +if (err && !ret) + ret = err; + +mutex_unlock (&disk->zone_revalidate_mutex); + +return ret; +} +EXPORT_SYMBOL_GPL (bdev_restore_storage_elements); + +int disk_init_storage_element_mgmt (struct gendisk *disk) +{ +int ret; + +/* +* Storage element operations are for zoned disks only and are +* optional... +*/ +if (!blk_queue_is_zoned (disk->queue) || !disk->fops->se_ops) +return 0 ; + +/* ...but if defined, all operation callbacks must be defined. */ +if (!disk->fops->se_ops->report_elements || + !disk->fops->se_ops->remove_element || + !disk->fops->se_ops->restore_elements) { +pr_err ("%s: invalid storage element operations\n" , + disk->disk_name); +return -EINVAL ; + } + +/* +* Get the number of storage elements of the disk. If this fails, +* do not report an error as the failure may be because the zoned disk +* does not support storage element management. +*/ + ret = disk->fops->se_ops->report_elements (disk,NULL , + &disk->nr_storage_elements); +if (ret) + disk->nr_storage_elements =0 ; + +return 0 ; +}
block/blk.h +5 -0
@@ -543,6 +543,7 @@ static inline void ioc_clear_queue(struct request_queue *q)#endif /* CONFIG_BLK_ICQ */ #ifdef CONFIG_BLK_DEV_ZONED +int disk_init_storage_element_mgmt (struct gendisk *disk);void disk_init_zone_resources (struct gendisk *disk);void disk_release_zone_resources (struct gendisk *disk);static inline bool bio_zone_write_plugging (struct bio *bio)
@@ -595,6 +596,10 @@ int blkdev_zone_mgmt_ioctl(struct block_device *bdev, blk_mode_t mode, unsigned int cmd, unsigned long arg);bool bdev_zone_mgmt_allowed (struct block_device *bdev,sector_t sector);#else /* CONFIG_BLK_DEV_ZONED */ +static inline int disk_init_storage_element_mgmt (struct gendisk *disk) +{ +return 0 ; +}static inline void disk_init_zone_resources (struct gendisk *disk) { }
block/genhd.c +4 -0
@@ -435,6 +435,10 @@ static int __add_disk(struct device *parent, struct gendisk *disk,if (WARN_ON_ONCE (bdev_nr_sectors (disk->part0) >BLK_DEV_MAX_SECTORS ))return -EINVAL ; + ret =disk_init_storage_element_mgmt (disk); +if (ret) +return ret; +if (queue_is_mq (disk->queue)) {/* * ->submit_bio and ->poll_bio are bypassed for blk-mq drivers.
include/linux/blkdev.h +55 -0
85 lines not on this page · raw message
include/uapi/linux/blkzoned.h +65 -0
69 lines not on this page · raw message
Signature · 1 line
2.56.1
Damien Le Moal
maintainerPATCH v12 5/13 block: add storage element management ioctls
In reply to Damien Le MoalDefine the new ioctl commands BLKREPORTSTORELEMS, BLKREMOVESTORELEM, and BLKRESTORESTORELEMS to provide users accessing zoned block devices directly with an interface to the storage elements management operations of the block layer.
The ioctl BLKREPORTSTORELEMS allows to get the total number of storage elements of a zoned block device and to get an array of storage element descriptors (struct blk_se) describing the current state of the device storage elements.
The ioctl BLKREMOVESTORELEM can be used to remove (depopulate) a storage element, in particular storage elements that have a degraded status (i.e. BLK_SE_STS_DEGRADED). But healthy elements (BLK_SE_STS_OK) can also be removed, and later restored.
Finally, the BLKRESTORESTORELEMS ioctl interfaces with bdev_restore_storage_elements() to restore the depopulated storage elements of a zoned device.
--- block/blk-zoned.c | 163 ++++++++++++++++++++++++++++++++++ block/blk.h | 9 ++ block/ioctl.c | 4 + include/uapi/linux/blkzoned.h | 41 +++++++++ include/uapi/linux/fs.h | 1 + 5 files changed, 218 insertions(+)
block/blk-zoned.c +163 -0
@@ -18,6 +18,7 @@#include <linux/mempool.h> #include <linux/kthread.h> #include <linux/freezer.h> +#include <linux/uaccess.h> #include <trace/events/block.h>
@@ -2781,6 +2782,53 @@ int bdev_report_storage_elements(struct block_device *bdev, }EXPORT_SYMBOL_GPL (bdev_report_storage_elements); +static int blkdev_ioctl_report_storage_elements (struct block_device *bdev, +void __user *argp) +{ +struct gendisk *disk = bdev->bd_disk; +struct blk_se_report *rep; +int ret =0 ; + +/* +* Allocate the report header with an array of elements, assuming that +* we need the maximum number of storage elements. In practice, we may +* need less than that, but this potentially unnecessary preallocation +* simplifies the code. +*/ + rep =kzalloc_flex (*rep, elements, disk->nr_storage_elements); +if (!rep) +return -ENOMEM ; + +if (copy_from_user (rep, argp,sizeof (struct blk_se_report ))) { + ret = -EFAULT ; + goto free_rep; + } + +if (rep->reserved) { + ret = -EINVAL ; + goto free_rep; + } + + rep->nr_reported_elements = +min (rep->nr_elements, disk->nr_storage_elements); + rep->nr_elements = disk->nr_storage_elements; + +if (rep->nr_reported_elements) { + ret =bdev_report_storage_elements (bdev, rep->elements, + &rep->nr_reported_elements); +if (ret) + goto free_rep; + } + +if (copy_to_user (argp, rep, +struct_size (rep, elements, rep->nr_reported_elements))) + ret = -EFAULT ; + +free_rep: +kfree (rep); +return ret; +} +/** * bdev_remove_storage_element - Remove (depopulate) a storage element of a * zoned block device
@@ -2844,6 +2892,62 @@ int bdev_remove_storage_element(struct block_device *bdev, }EXPORT_SYMBOL_GPL (bdev_remove_storage_element); +static int blkdev_ioctl_remove_storage_element (struct block_device *bdev, +blk_mode_t mode, +unsigned int __user *argp) +{ +unsigned int element_id; +int ret; + +/* +* Note: we require an O_EXCL open here so that a user cannot trigger +* a storage element removal from user space under a mounted file system. +* An FS that wants to manage storage element can do so using the +* function bdev_remove_storage_element() from kernel context. +*/ +if (!(mode &BLK_OPEN_EXCL )) +return -EBADF ; +if (!(mode &BLK_OPEN_WRITE )) +return -EBADF ; +if (bdev_read_only (bdev)) +return -EPERM ; + +if (get_user (element_id, argp)) +return -EFAULT ; + +/* +* Sync the device dirty data, flush the volatile write cache, and +* invalidate all cached data so that reads do not return old data for +* zones that went offline. +* Since a user can only successfully write to sequential zones of a +* zoned devices using direct IOs, the sync should only handle dirty +* data from buffered writes to conventional zones, for devices that +* have such zones. +*/ + ret =sync_blockdev (bdev); +if (ret) +return ret; + +inode_lock (bdev->bd_mapping->host); +filemap_invalidate_lock (bdev->bd_mapping); + + ret =blkdev_issue_flush (bdev->bd_disk->part0); +if (ret) + goto unlock; + + ret =truncate_bdev_range (bdev, mode,0 ,bdev_nr_bytes (bdev) -1 ); +if (ret) + goto unlock; + + ret =bdev_remove_storage_element (bdev, element_id); + +unlock: +filemap_invalidate_unlock (bdev->bd_mapping); +inode_unlock (bdev->bd_mapping->host); + +return ret; +} +/** * bdev_restore_storage_elements - Restore all depopulated storage elements of * a zoned block device
@@ -2909,6 +3013,42 @@ int bdev_restore_storage_elements(struct block_device *bdev) }EXPORT_SYMBOL_GPL (bdev_restore_storage_elements); +static int blkdev_ioctl_restore_storage_elements (struct block_device *bdev, +blk_mode_t mode) +{ +int ret; + +/* +* Note: because restoring storage elements is a destructive operation +* equivalent to reformatting the device, we require an O_EXCL open here +* so that a user cannot trigger with another active user (e.g. a +* mounted file system). +*/ +if (!(mode &BLK_OPEN_EXCL )) +return -EBADF ; +if (!(mode &BLK_OPEN_WRITE )) +return -EBADF ; +if (bdev_read_only (bdev)) +return -EPERM ; + +inode_lock (bdev->bd_mapping->host); +filemap_invalidate_lock (bdev->bd_mapping); + +/* +* Storage element restoration is a destructive operation that will +* reset all zones. So invalidate all cached data that we may have so +* that old data cannot be read once the operation completes. +*/ + ret =truncate_bdev_range (bdev, mode,0 ,bdev_nr_bytes (bdev) -1 ); +if (!ret) + ret =bdev_restore_storage_elements (bdev); + +filemap_invalidate_unlock (bdev->bd_mapping); +inode_unlock (bdev->bd_mapping->host); + +return ret; +} +int disk_init_storage_element_mgmt (struct gendisk *disk) {int ret;
@@ -2941,3 +3081,26 @@ int disk_init_storage_element_mgmt(struct gendisk *disk)return 0 ; } + +int blk_ioctl_zone_storage_elements (struct block_device *bdev, +blk_mode_t mode,unsigned int cmd, +void __user *argp) +{ +struct gendisk *disk = bdev->bd_disk; + +if (!disk->nr_storage_elements) +return -ENOTTY ; + +switch (cmd) { +case BLKREPORTSTORELEMS : +return blkdev_ioctl_report_storage_elements (bdev, argp); +case BLKREMOVESTORELEM : +return blkdev_ioctl_remove_storage_element (bdev, mode, argp); +case BLKRESTORESTORELEMS : +return blkdev_ioctl_restore_storage_elements (bdev, mode); +default : +break ; + } + +return -ENOTTY ; +}
block/blk.h +9 -0
@@ -595,6 +595,9 @@ int blkdev_report_zones_ioctl(struct block_device *bdev, unsigned int cmd,int blkdev_zone_mgmt_ioctl (struct block_device *bdev,blk_mode_t mode,unsigned int cmd,unsigned long arg);bool bdev_zone_mgmt_allowed (struct block_device *bdev,sector_t sector); +int blk_ioctl_zone_storage_elements (struct block_device *bdev, +blk_mode_t mode,unsigned int cmd, +void __user *argp);#else /* CONFIG_BLK_DEV_ZONED */ static inline int disk_init_storage_element_mgmt (struct gendisk *disk) {
@@ -646,6 +649,12 @@ static inline bool bdev_zone_mgmt_allowed(struct block_device *bdev, {return false; } +static inline int blk_ioctl_zone_storage_elements (struct block_device *bdev, +blk_mode_t mode,unsigned int cmd, +void __user *argp) +{ +return -ENOTTY ; +}#endif /* CONFIG_BLK_DEV_ZONED */ struct block_device *bdev_alloc (struct gendisk *disk,u8 partno);
block/ioctl.c +4 -0
@@ -679,6 +679,10 @@ static int blkdev_common_ioctl(struct block_device *bdev, blk_mode_t mode,return put_uint (argp,bdev_zone_sectors (bdev));case BLKGETNRZONES :return put_uint (argp,bdev_nr_zones (bdev)); +case BLKREPORTSTORELEMS : +case BLKREMOVESTORELEM : +case BLKRESTORESTORELEMS : +return blk_ioctl_zone_storage_elements (bdev, mode, cmd, argp);case BLKROGET :return put_int (argp,bdev_read_only (bdev) !=0 );case BLKSSZGET :/* get block device logical block size */
include/uapi/linux/blkzoned.h +41 -0
45 lines not on this page · raw message
include/uapi/linux/fs.h +1 -0
@@ -309,6 +309,7 @@ struct file_attr {/* 130-136 and 142 are used by zoned block device ioctls (uapi/linux/blkzoned.h) */ /* 137-141 are used by blk-crypto ioctls (uapi/linux/blk-crypto.h) */ #define BLKTRACESETUP2 _IOWR(0x12, 142, struct blk_user_trace_setup2) +/* 143-145 are used by storage element management for zoned block devices. */ #define BMAP_IOCTL 1/* obsolete - kept for compatibility */ #define FIBMAP _IO(0x00,1)/* bmap access */
Signature · 1 line
2.56.1
Damien Le Moal
maintainerPATCH v12 6/13 zloop: fix ordered zone append handling
In reply to Damien Le MoalFor oredered zone append commands, the function zloop_set_zone_append_sector() increments a zone write pointer without checking if the zone is already open (implicitly or explicitly). This check is done only when the command is later executed using a work item, and the zone is only then implicitly open if needed. This creates a short time window where a report zone can show an empty zone with a write pointer that is not equal to the zone start sector, which is incorrect.
Calling zloop_do_open_zone() in zloop_set_zone_append_sector() is required but introduces locking complexity as the zone mutex cannot be taken in the context of zloop_queue_rq(). Instead introduce the ordered workqueue append_workqueue to handle zone append commands when ordered zone append is enabled. This ensure that all zone append operations are executed in the order of there submission. This removes the need for setting the command sector in the context of zloop_queue_rq() and we can rely on the reguar command execution done by zloop_rw(). With this change, zloop_seq_write_prep() is simplified and zloop_set_zone_append_sector() is removed entirely.
This change does reduce the command execution concurrency and impact performance with ordered zone append enabled for file systems using zone append. This is however not a critical option and the default unordered zone append handling is unaffected.
--- drivers/block/zloop.c | 113 +++++++++++++++++------------------------- 1 file changed, 45 insertions(+), 68 deletions(-)
drivers/block/zloop.c +45 -68
@@ -129,6 +129,7 @@ struct zloop_device {struct gendisk *disk;struct workqueue_struct *workqueue; +struct workqueue_struct *append_workqueue;bool buffered_io;bool zone_append;bool ordered_zone_append;
@@ -590,32 +591,24 @@ static int zloop_seq_write_prep(struct zloop_cmd *cmd)* when submitted. */ if (is_append) { -/* -* If ordered zone append is in use, we already checked and set -* the target sector in zloop_queue_rq(). -*/ -if (!zlo->ordered_zone_append) { -if (zone->cond ==BLK_ZONE_COND_FULL || - zone->wp + nr_sectors > zone_end) { - ret = -EIO ; - goto out_unlock; - } - cmd->sector = zone->wp; - } - }else { -if (cmd->sector != zone->wp) { -if (zone->wp ==ULLONG_MAX ) -pr_err ("%pg: Zone %u: write to %s zone: sect %llu\n" , - zlo->disk->part0, zone_no, -blk_zone_cond_str (zone->cond), - cmd->sector); -else -pr_err ("%pg: Zone %u: unaligned write: sect %llu, wp %llu\n" , - zlo->disk->part0, zone_no, cmd->sector, - zone->wp); +if (zone->cond ==BLK_ZONE_COND_FULL || + zone->wp + nr_sectors > zone_end) { ret = -EIO ; goto out_unlock; } + cmd->sector = zone->wp; + }else if (cmd->sector != zone->wp) { +if (zone->wp ==ULLONG_MAX ) +pr_err ("%pg: Zone %u: write to %s zone: sect %llu\n" , + zlo->disk->part0, zone_no, +blk_zone_cond_str (zone->cond), + cmd->sector); +else +pr_err ("%pg: Zone %u: unaligned write: sect %llu, wp %llu\n" , + zlo->disk->part0, zone_no, cmd->sector, + zone->wp); + ret = -EIO ; + goto out_unlock; }/* Implicitly open the target zone. */
@@ -625,15 +618,13 @@ static int zloop_seq_write_prep(struct zloop_cmd *cmd) }/* -* Advance the write pointer, unless ordered zone append is in use. If -* the write fails, the write pointer position will be corrected when -* the next I/O starts execution. +* Advance the write pointer. If the write fails, the write pointer +* position will be corrected when the next I/O starts execution. */ -if (!is_append || !zlo->ordered_zone_append) { - zone->wp += nr_sectors; -if (zone->wp == zone_end) -zloop_mark_full (zlo, zone); - } + zone->wp += nr_sectors; +if (zone->wp == zone_end) +zloop_mark_full (zlo, zone); + out_unlock:spin_unlock (&zone->wp_lock);return ret;
@@ -885,32 +876,6 @@ static void zloop_complete_rq(struct request *rq)blk_mq_end_request (rq, sts); } -static bool zloop_set_zone_append_sector (struct request *rq) -{ -struct zloop_device *zlo = rq->q->queuedata; -unsigned int zone_no =rq_zone_no (rq); -struct zloop_zone *zone = &zlo->zones[zone_no]; -sector_t zone_end = zone->start + zlo->zone_capacity; -sector_t nr_sectors =blk_rq_sectors (rq); - -spin_lock (&zone->wp_lock); - -if (zone->cond ==BLK_ZONE_COND_FULL || - zone->wp + nr_sectors > zone_end) { -spin_unlock (&zone->wp_lock); -return false; - } - - rq->__sector = zone->wp; - zone->wp +=blk_rq_sectors (rq); -if (zone->wp >= zone_end) -zloop_mark_full (zlo, zone); - -spin_unlock (&zone->wp_lock); - -return true; -} -static blk_status_t zloop_queue_rq (struct blk_mq_hw_ctx *hctx,const struct blk_mq_queue_data *bd) {
@@ -923,20 +888,13 @@ static blk_status_t zloop_queue_rq(struct blk_mq_hw_ctx *hctx,return BLK_STS_IOERR ; } -/* -* If we need to strongly order zone append operations, set the request -* sector to the zone write pointer location now instead of when the -* command work runs. -*/ -if (zlo->ordered_zone_append &&req_op (rq) ==REQ_OP_ZONE_APPEND ) { -if (!zloop_set_zone_append_sector (rq)) -return BLK_STS_IOERR ; - } -blk_mq_start_request (rq);INIT_WORK (&cmd->work, zloop_cmd_workfn); -queue_work (zlo->workqueue, &cmd->work); +if (zlo->ordered_zone_append &&req_op (rq) ==REQ_OP_ZONE_APPEND ) +queue_work (zlo->append_workqueue, &cmd->work); +else +queue_work (zlo->workqueue, &cmd->work);return BLK_STS_OK ; }
@@ -1029,6 +987,8 @@ static void zloop_free_disk(struct gendisk *disk)fput (zlo->data_dir);destroy_workqueue (zlo->workqueue); +if (zlo->append_workqueue) +destroy_workqueue (zlo->append_workqueue);kfree (zlo->base_dir);kvfree (zlo); }
@@ -1285,6 +1245,21 @@ static int zloop_ctl_add(struct zloop_options *opts) goto out_free_idr; } +/* +* Use an ordered workqueue for strongly ordered zone append operations +* so that we can keep processing the operations in the same order as +* they are submitted. +*/ +if (zlo->ordered_zone_append) { + zlo->append_workqueue = +alloc_ordered_workqueue ("zloop%d_append" ,WQ_FREEZABLE , + zlo->id); +if (!zlo->append_workqueue) { + ret = -ENOMEM ; + goto out_destroy_workqueue; + } + } +if (opts->base_dir) zlo->base_dir =kstrdup (opts->base_dir,GFP_KERNEL );else
@@ -1391,6 +1366,8 @@ static int zloop_ctl_add(struct zloop_options *opts)kfree (zlo->base_dir); out_destroy_workqueue:destroy_workqueue (zlo->workqueue); +if (zlo->append_workqueue) +destroy_workqueue (zlo->append_workqueue); out_free_idr:mutex_lock (&zloop_ctl_mutex);idr_remove (&zloop_index_idr, zlo->id);
Signature · 1 line
2.56.1
Damien Le Moal
maintainerPATCH v12 7/13 zloop: fix zone reporting consistency
In reply to Damien Le MoalIn zloop_report_zones(), filling the write pointer and condition of a zone descriptor separately outside of the zone wp_lock can lead to an inconsistent zone report with a write pointer potentially not being consistent with the condition being reported. Avoid such inconsistency by filling both fields while holding the zone wp_lock.
--- drivers/block/zloop.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/block/zloop.c +1 -1
@@ -950,8 +950,8 @@ static int zloop_report_zones(struct gendisk *disk, sector_t sector, blkz.len = zlo->zone_size;spin_lock (&zone->wp_lock); blkz.wp = zone->wp; -spin_unlock (&zone->wp_lock); blkz.cond = zone->cond; +spin_unlock (&zone->wp_lock);if (test_bit (ZLOOP_ZONE_CONV , &zone->flags)) { blkz.type =BLK_ZONE_TYPE_CONVENTIONAL ; blkz.capacity = zlo->zone_size;
Signature · 1 line
2.56.1
Damien Le Moal
maintainerPATCH v12 8/13 zloop: add storage element emulation
In reply to Damien Le MoalEmulate the storage element depopulation feature of SMR disks in the zloop driver. This emulation is controlled using the new stor_elements option. This option can take several values:
- ZLOOP_STOR_ELEMENTS_NONE (0): no emulation (default)
- ZLOOP_STOR_ELEMENTS_RDWR (1): emulate all access storage elements (e.g. read+write heads)
- ZLOOP_STOR_ELEMENTS_PAIRS (2): emulate fractional access storage elements (e.g. pairs of read and write heads)
If enabled with the value 1 or 2, the number of storage elements, or of pairs of fractional access storage elements, is automatically calculated based on the number of zones of the device so that we have at least 2 and at most 32 storage elements or pairs of fractional access storage elements.
The mapping of zones to storage elements is defined simply as the modulo of a zone number with the number of storage elements for the all access storage elements emulation and with the number of storage element pairs for the fractional access storage element emulation case. E.g, removing a storage element from a set of 4 all access storage elements will offline 1 zone every 4 zones.
The storage element management operations are specified with the structure zloop_se_ops (struct blk_se_ops).
--- .../admin-guide/blockdev/zoned_loop.rst | 11 +- drivers/block/zloop.c | 712 +++++++++++++++++- 2 files changed, 696 insertions(+), 27 deletions(-)
Documentation/admin-guide/blockdev/zoned_loop.rst +10 -1
@@ -61,7 +61,7 @@ The options available for the add command can be listed by reading the /dev/zloop-control device:: $ cat /dev/zloop-control - add id=%d,capacity_mb=%u,zone_size_mb=%u,zone_capacity_mb=%u,conv_zones=%u,max_open_zones=%u,base_dir=%s,nr_queues=%u,queue_depth=%u,buffered_io,zone_append=%u,ordered_zone_append,discard_write_cache + add id=%d,capacity_mb=%u,zone_size_mb=%u,zone_capacity_mb=%u,conv_zones=%u,max_open_zones=%u,base_dir=%s,nr_queues=%u,queue_depth=%u,buffered_io,zone_append=%u,ordered_zone_append,discard_write_cache,stor_elements=%u remove id=%d In more details, the options that can be used with the "add" command are as
@@ -113,6 +113,15 @@ discard_write_cache Discard all data that was not explicitly persisted using a each zone file to the size recorded during the last flush operation. This simulates power fail events where uncommitted data is lost. +stor_elements Control storage element emulation. The default value is + 0, indicating no emulation. A value of 1 indicates that + storage elements are emulated as full access physical + elements, which is equivalent to combined read+write heads + of a hard-disk. A value of 2 enables storage element + emulation as fractional access physical storage elements, + which is equivalent to pairs of read and write heads of a + hard-disk. The health state of storage elements is not + preserved across a zloop device remove and add. =================== ========================================================= 3) Deleting a Zoned Device
drivers/block/zloop.c +686 -26
955 lines not on this page · raw message
Signature · 1 line
2.56.1
[email protected]
AI reviewRe: [PATCH v12 08/13] zloop: add storage element emulation
Thank you for your contribution! Sashiko AI review found 1 potential issue to consider: - [Medium] zloop: serialize zloop_ctl_remove() with storage element ops --
Thank you for your contribution! Sashiko AI review found 1 potential issue to consider:
- [Medium] zloop: serialize zloop_ctl_remove() with storage element ops --
commit b2e9213b7a22038d3fa09a4110dd9d6ac106045b
Author: Damien Le Moal <[email protected]>
zloop: add storage element emulation
This commit adds storage element depopulation and restoration emulation to the zloop driver, controlled via a new stor_elements option and exposed through zloop_se_ops.
Quoted 4 lines
drivers/block/zloop.c +0
-0
[ ... ]
Quoted 14 lines
drivers/block/zloop.c +6
-0@@ -1388,6 +2022,12 @@ static void zloop_forget_cache(struct zloop_device *zlo)pr_info ("%pg: discarding volatile write cache\n" , zlo->disk->part0); +/* +* Serialize with storage element removal or restoration that may be +* finishing running during device removal. +*/ +mutex_lock (&zlo->stor_elements_lock); +for (i =0 ; i < zlo->nr_zones; i++) {struct zloop_zone *zone = &zlo->zones[i];struct file *file = zone->file;
drivers/block/zloop.c static void zloop_forget_cache(struct zloop_device *zlo) · line 2063
Quoted 6 lines
drivers/block/zloop.c +2
-0@@ -1418,6 +2058,8 @@ static void zloop_forget_cache(struct zloop_device *zlo)vfs_truncate (&file->f_path, (old_wp - zone->start) <<SECTOR_SHIFT ); } + +mutex_unlock (&zlo->stor_elements_lock); }
[Severity: Medium]
Can zloop_ctl_remove() finish and return to userspace while zloop_execute_restore_elements() is still truncating zone backing files when discard_write_cache is disabled?
zloop_forget_cache() is only called from zloop_ctl_remove() when zlo->discard_write_cache is set:
drivers/block/zloop.c:zloop_ctl_remove() {
...
del_gendisk(zlo->disk);
if (zlo->discard_write_cache)
zloop_forget_cache(zlo);
put_disk(zlo->disk);
...
}When discard_write_cache is false (the default), nothing in zloop_ctl_remove() acquires zlo->stor_elements_lock to wait for an in-flight storage element operation that passed the Zlo_deleting check just before removal started:
CPU0 CPU1zloop_restore_elements()
msleep(5000)
mutex_lock(&zlo->stor_elements_lock)
/* zlo->state != Zlo_deleting */
zloop_ctl_remove()
idr_remove(&zloop_index_idr, ...)
WRITE_ONCE(zlo->state, Zlo_deleting)
del_gendisk(zlo->disk)
put_disk(zlo->disk)
return 0; zloop_execute_restore_elements()
zloop_restore_zone()
vfs_truncate(...) <-- runs after device removal completed
If userspace re-adds the device with the same ID or accesses base_dir after remove returns, the ongoing truncations from the old instance race with the new accesses and corrupt the zone files. Acquiring zlo->stor_elements_lock in zloop_ctl_remove() after del_gendisk() would serialize removal regardless of discard_write_cache.
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=8
Damien Le Moal
maintainerPATCH v12 9/13 zloop: add degrade_element control command
In reply to Damien Le MoalAllow users to mark storage elements of a zloop device as degraded using the new "degrade_element" control command. The element to degrade is indicated using the element_id option. Example:
echo "degrade_element id=0,element_id=2" > /dev/zloop-control
If the element ID identifies an all access storage element, both read and write operations targeting a zone served by the degraded element are failed. In the case of a partial access storage element, any write access to a zone whose write element is degraded will fail, and read accesses to a zone whose read element is degraded will result in an error.
The zoned_loop.rst documentation file is also updated to explain this new feature.
--- .../admin-guide/blockdev/zoned_loop.rst | 25 ++++- drivers/block/zloop.c | 100 +++++++++++++++++- 2 files changed, 121 insertions(+), 4 deletions(-)
Documentation/admin-guide/blockdev/zoned_loop.rst +23 -2
@@ -9,7 +9,8 @@ Zoned Loop Block Device 1) Overview 2) Creating a Zoned Device 3) Deleting a Zoned Device - 4) Example + 4) Degrading a storage element + 5) Example 1) Overview
@@ -63,6 +64,7 @@ The options available for the add command can be listed by reading the $ cat /dev/zloop-control add id=%d,capacity_mb=%u,zone_size_mb=%u,zone_capacity_mb=%u,conv_zones=%u,max_open_zones=%u,base_dir=%s,nr_queues=%u,queue_depth=%u,buffered_io,zone_append=%u,ordered_zone_append,discard_write_cache,stor_elements=%u remove id=%d + degrade_element id=%d,element_id=%u In more details, the options that can be used with the "add" command are as follows.
@@ -144,7 +146,26 @@ the zoned device will not be created. To fully delete a zoned device, after executing the remove operation, the device base directory containing the backing files of the device zones must be deleted. -4) Example +4) Degrading a storage element +------------------------------ + +When storage element emulation is enabled, storage elements can be set in a +degraded state to emulate defective storage elements (e.g. defective heads of a +hard disk). This is done by issuing the "degrade_element" command to +/dev/zloop-control, specifying the ID of the target device and the ID of the +storage element to degrade:: + + $ echo "degrade_element id=X,element_id=Y" > /dev/zloop-control + +Depending on the type of the storage element specified, a read or write access +to sectors of a zone served by a degraded storage element will result in a +failure. For partial access storage elements, write accesses are always failed +if the write element serving the zone is degraded. Likewise, read accesses are +failed if the read element serving the zone is degraded. When storage elements +are emulated as all access physical elements, both read and write operations are +failed when the storage element serving the zones being accessed is degraded. + +5) Example ---------- The following sequence of commands creates a 2GB zoned device with zones of 64
drivers/block/zloop.c +98 -2
184 lines not on this page · raw message
Signature · 1 line
2.56.1
Damien Le Moal
maintainerPATCH v12 10/13 scsi: scsi_proto: refine the definition of storage element health
In reply to Damien Le MoalA disk may report the health of a storage element as being within or outside of the manufacturer's specification limits using a range of values for the health field. However, in scsi_proto.h, we only define the start of each range using the SCSI_PHYS_ELEM_HEALTH_WITHIN_SPEC_LIMITS and SCSI_PHYS_ELEM_HEALTH_OUTSIDE_SPEC_LIMITS enum value.
Define the full range by renaming SCSI_PHYS_ELEM_HEALTH_WITHIN_SPEC_LIMITS to SCSI_PHYS_ELEM_HEALTH_WITHIN_SPEC_LIMITS_MIN and by defining the healthy range end with SCSI_PHYS_ELEM_HEALTH_WITHIN_SPEC_LIMITS_MAX. The same treatment is done for the outside range with the renaming of SCSI_PHYS_ELEM_HEALTH_OUTSIDE_SPEC_LIMITS to SCSI_PHYS_ELEM_HEALTH_OUTSIDE_SPEC_LIMITS_MIN, and the definition of the range upper value with SCSI_PHYS_ELEM_HEALTH_OUTSIDE_SPEC_LIMITS_MAX.
--- include/scsi/scsi_proto.h | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-)
include/scsi/scsi_proto.h +4 -2
@@ -408,9 +408,11 @@ enum scsi_phys_element_type {/* SCSI physical element health. */ enum scsi_phys_element_health {SCSI_PHYS_ELEM_HEALTH_NOT_REPORTED =0x00 , -SCSI_PHYS_ELEM_HEALTH_WITHIN_SPEC_LIMITS =0x01 , +SCSI_PHYS_ELEM_HEALTH_WITHIN_SPEC_LIMITS_MIN =0x01 , +SCSI_PHYS_ELEM_HEALTH_WITHIN_SPEC_LIMITS_MAX =0x63 ,SCSI_PHYS_ELEM_HEALTH_AT_SPEC_LIMITS =0x64 , -SCSI_PHYS_ELEM_HEALTH_OUTSIDE_SPEC_LIMITS =0x65 , +SCSI_PHYS_ELEM_HEALTH_OUTSIDE_SPEC_LIMITS_MIN =0x65 , +SCSI_PHYS_ELEM_HEALTH_OUTSIDE_SPEC_LIMITS_MAX =0xCF ,SCSI_PHYS_ELEM_HEALTH_DEPOP_REVOKE_ERR =0xFB ,SCSI_PHYS_ELEM_HEALTH_DEPOP_REVOKE_IN_PROGRESS =0xFC ,SCSI_PHYS_ELEM_HEALTH_DEPOP_ERR =0xFD ,
Signature · 1 line
2.56.1
Damien Le Moal
maintainerPATCH v12 11/13 scsi: sd_zbc: always revalidate zones for disks supporting head depopulation
In reply to Damien Le Moalsd_zbc_revalidate_zones() skips revalidating the zones of a ZBC device if the zone size and total number of zones of the disk has not changed. This is to avoid the sometimes slow report zones for the entire disk that blk_revalidate_disk_zones() will generate.
For ZBC devices that support the storage element depopulation feature, a revalidation after executing a passthrough REMOVE ELEMENT AND TRUNCATE command will thus correctly trigger a zone revalidation as the disk capacity and number of zones changed.
However, for devices supporting the data preserving head depopulation variant of the depopulation feature (REMOVE ELEMENT AND MODIFY ZONES command), a disk capacity and number of zones does not change after a head is depopulated. Only the conditions of the zones served by the head that was depopulated change. These zones are transitioned to either read-only or offline.
While the block layer storage element management functions always call blk_revalidate_disk_zones() to catch these changes, a user scan, libata device rescan after EH or a depopulation or restoration action executed through SG_IO would not. In these cases, not calling blk_revalidate_disk_zones() prevents the block layer from taking appropriate actions on the zone write plugs of the disk.
Avoid any issue with the block layer view of the zone conditions by not skipping the call to blk_revalidate_disk_zones() for disks that support both the GET PHYSICAL ELEMENTS STATUS and REMOVE ELEMENT AND MODIFY ZONES commands. This check is done from sd_zbc_read_zones() using the helper function sd_zbc_check_depop_supported(). The new SCSI disk flags depop_report_supported and depop_modify_zones_supported are defined to remember the result of this check.
This change will slow down a little SMR disk revalidation. However, user forced revalidations are normally rare, and other revalidations triggered by udev only happen when the disk device files is not in use anymore (closed). So this slow down should in practice not be an issue.
--- drivers/scsi/sd.h | 2 ++ drivers/scsi/sd_zbc.c | 47 +++++++++++++++++++++++++++++++++++++++++-- 2 files changed, 47 insertions(+), 2 deletions(-)
drivers/scsi/sd.h +2 -0
@@ -156,6 +156,8 @@ struct scsi_disk { unsigned ignore_medium_access_errors : 1; unsigned rscs : 1; /* reduced stream control support */ unsigned use_atomic_write_boundary : 1; + unsigned depop_report_supported : 1; + unsigned depop_modify_zones_supported : 1; };#define to_scsi_disk (obj) container_of(obj, struct scsi_disk, disk_dev)
drivers/scsi/sd_zbc.c +45 -2
81 lines not on this page · raw message
Signature · 1 line
2.56.1
Damien Le Moal
maintainerPATCH v12 12/13 scsi: sd_zbc: define storage element management operations
In reply to Damien Le MoalDefine the storage element management operations using struct blk_se_ops. These operations are valid only on SMR disks supporting the storage element depopulation featu, with support for the GET PHYSICAL ELEMENTS STATUS command being mandatory. The support for the modify_zones operation is already indicated by the depop_modify_zones_supported SCSI disk flag. sd_zbc_check_depop_supported() is modified to add detection for the command RESTORE ELEMENTS AND REBUILD, which is indicated with the new SCSI disk flag depop_restore_supported.
Both the remove_element and restore_elements operations will block until the operation completes or an error is detected.
--- drivers/scsi/sd.c | 3 + drivers/scsi/sd.h | 3 + drivers/scsi/sd_zbc.c | 479 +++++++++++++++++++++++++++++++++++++++++- 3 files changed, 484 insertions(+), 1 deletion(-)
drivers/scsi/sd.c +3 -0
@@ -3939,6 +3939,9 @@ static const struct block_device_operations sd_fops = { .get_unique_id = sd_get_unique_id, .free_disk = scsi_disk_free_disk, .pr_ops = &sd_pr_ops, +#ifdef CONFIG_BLK_DEV_ZONED + .se_ops = &sd_zbc_se_ops, +#endif };/**
drivers/scsi/sd.h +3 -0
@@ -158,6 +158,7 @@ struct scsi_disk { unsigned use_atomic_write_boundary : 1; unsigned depop_report_supported : 1; unsigned depop_modify_zones_supported : 1; + unsigned depop_restore_supported : 1; };#define to_scsi_disk (obj) container_of(obj, struct scsi_disk, disk_dev)
@@ -244,6 +245,8 @@ unsigned int sd_zbc_complete(struct scsi_cmnd *cmd, unsigned int good_bytes,int sd_zbc_report_zones (struct gendisk *disk,sector_t sector,unsigned int nr_zones,struct blk_report_zones_args *args); +extern const struct blk_se_ops sd_zbc_se_ops; +#else /* CONFIG_BLK_DEV_ZONED */ static inline int sd_zbc_read_zones(struct scsi_disk *sdkp,
drivers/scsi/sd_zbc.c +478 -1
515 lines not on this page · raw message
Signature · 1 line
2.56.1
Damien Le Moal
maintainerPATCH v12 13/13 scsi: sd: fail read commands to offline zones early
In reply to Damien Le MoalIssuing read commands to offline zones of a ZBC disk will result in failures. So do not issue such command and fail them early in sd_setup_read_write_cmnd(). This change also checks write commands, which is fine but should actually never happen as the block layer zone write plugging code already fails write BIOs to offline zones early in the submission path.
As the code comment explains, since disk_zone_is_offline() uses the block layer cached zone condition, this is inherently racy with zone revalidation updating the zone condition. However, this will never prevent valid read operations as the only case where we could proceed is if the read is received after a restoration of the disk storage element completes but before the zones are revalidated. And in that case, all zones are empty and thus have no data to read.
--- drivers/scsi/sd.c | 20 +++++++++++++++++++- 1 file changed, 19 insertions(+), 1 deletion(-)
drivers/scsi/sd.c +19 -1
@@ -1424,7 +1424,8 @@ static blk_status_t sd_setup_read_write_cmnd(struct scsi_cmnd *cmd) {struct request *rq =scsi_cmd_to_rq (cmd);struct scsi_device *sdp = cmd->device; -struct scsi_disk *sdkp =scsi_disk (rq->q->disk); +struct gendisk *disk = rq->q->disk; +struct scsi_disk *sdkp =scsi_disk (disk);sector_t lba =sectors_to_logical (sdp,blk_rq_pos (rq));sector_t threshold;unsigned int nr_blocks =sectors_to_logical (sdp,blk_rq_sectors (rq));
@@ -1456,6 +1457,23 @@ static blk_status_t sd_setup_read_write_cmnd(struct scsi_cmnd *cmd) goto fail; } +/* +* The block layer already prevents writes to offline zones of zoned +* block devices. But there are no checks for reads, so check this here. +* Since disk_zone_is_offline() uses the block layer cached zone +* condition, this is inherently racy with zone revalidation updating +* the zone condition. However, this will never prevent valid read +* operations as the only case where we could proceed is if the read is +* received after a restoration of the disk storage element completes +* but before the zones are revalidated. And in that case, all zones are +* empty and thus have no data to read. +* Note that we are quiet about the error (no message) to avoid flooding +* the kernel log. +*/ +if (sdkp->device->type ==TYPE_ZBC && +disk_zone_is_offline (disk,blk_rq_pos (rq))) + goto fail; +/* * Some SD card readers can't handle accesses which touch the * last one or two logical blocks. Split accesses as needed.
Signature · 1 line
2.56.1