configurable block error injection
Subsystems (6)
Hi all,
this series adds a new configurable block error injection facility. We already have a few to inject block errors, but unfortunately most of them are either not very useful or hard to use, or both:
- The fail_make_request failure injection point can't distinguish different commands, different ranges in the file and can only injection plain I/O errors.
- the should_fail_bio 'dynamic' failure injection has all the same issues as fail_make_request
- dm-error can only fail all command in the table using BLK_STS_IOERR and requires setting up a new block device
- dm-flakey and dm-dust allow all kinds of configurability, but still don't have good error selection, no good support for non-read/write commands and are limited to the dm table alignment requirements, which for zoned devices enforces setting them up for an entire zone. They also once again require setting up a stacked block device, which is really annoying in harnesses like xfstests
This series adds a new debugfs-based block layer error injection that allows to configure what operations and ranges the injection applied to, and what status to return. It also allows to configure a failure ratio similar to the xfs errortag injection.
As part of this the should_fail_bio is removed, as the should_fail_bio function really gets in the way in it's current form, and the injection of an errno which always gets turned into BLK_STS_IOERR doesn't make much sense. This required adjusting the bpf test suite as it attached to this function as it assumed it to be present.
Diffstat:
Documentation/block/error-injection.rst | 59 ++ Documentation/block/index.rst | 1 block/Kconfig | 6 block/Makefile | 1 block/blk-core.c | 128 ++--- block/blk-merge.c | 5 block/blk-mq.c | 3 block/blk-sysfs.c | 4 block/blk.h | 24 block/error-injection.c | 329 +++++++++++++ block/genhd.c | 4 include/linux/blk_types.h | 2 include/linux/blkdev.h | 5 lib/Kconfig.debug | 6 tools/testing/selftests/bpf/prog_tests/kprobe_multi_test.c | 7 15 files changed, 507 insertions(+), 77 deletions(-)
PATCH 1/9 block: remove ALLOW_ERROR_INJECTION for should_fail_bio
Allow error injection for should_fail_bio is a bit misguided. It allows inserting an errno, which is then ignored, but it forced and out of line call for something that should not exist when error injection is disabled.
Remove the error injection flag in preparation for adding better block layer error injection, and switch the bpf test to use a MM error injection site instead.
--- block/blk-core.c | 1 - tools/testing/selftests/bpf/prog_tests/kprobe_multi_test.c | 7 ++++--- 2 files changed, 4 insertions(+), 4 deletions(-)
block/blk-core.c +0 -1
@@ -545,7 +545,6 @@ int should_fail_bio(struct bio *bio)return -EIO ;return 0 ; } -ALLOW_ERROR_INJECTION (should_fail_bio,ERRNO );/* * Check whether this bio extends beyond the end of the device or partition.
tools/testing/selftests/bpf/prog_tests/kprobe_multi_test.c +4 -3
@@ -588,12 +588,13 @@ static void test_attach_override(void) goto cleanup; } -/* The should_fail_bio function is on error injection list, +/* The __filemap_add_folio function is on error injection list, * attach should succeed. */ link =bpf_program__attach_kprobe_multi_opts (skel->progs.test_override, -"should_fail_bio" ,NULL ); -if (!ASSERT_OK_PTR (link,"override_attached_should_fail_bio" )) +"__filemap_add_folio," , +NULL ); +if (!ASSERT_OK_PTR (link,"override_attached___filemap_add_folio," )) goto cleanup;bpf_link__destroy (link);
Signature · 1 line
2.53.0
PATCH 2/9 block: consolidate the calls to should_fail_bio
In reply to Christoph HellwigDelay the point of error injection a bit so that we have a single site in blk-core.c after more of the submission side checks and blkcg throttling. This allows to make should_fail_bio static in blk-core.c.
--- block/blk-core.c | 6 +++++- block/blk-merge.c | 5 +---- block/blk.h | 1 - 3 files changed, 6 insertions(+), 6 deletions(-)
block/blk-core.c +5 -1
@@ -539,7 +539,7 @@ static inline void bio_check_ro(struct bio *bio) } } -int should_fail_bio (struct bio *bio) +static int should_fail_bio (struct bio *bio) {if (should_fail_request (bdev_whole (bio->bi_bdev), bio->bi_iter.bi_size))return -EIO ;
@@ -723,6 +723,10 @@ static void __submit_bio_noacct_mq(struct bio *bio)void submit_bio_noacct_nocheck (struct bio *bio,bool split) { +if (should_fail_bio (bio)) { +bio_io_error (bio); +return ; + }blk_cgroup_bio_start (bio);if (!bio_flagged (bio,BIO_TRACE_COMPLETION )) {
block/blk-merge.c +1 -4
@@ -130,11 +130,8 @@ struct bio *bio_submit_split_bioset(struct bio *bio, unsigned int split_sectors,trace_block_split (split, bio->bi_iter.bi_sector);WARN_ON_ONCE (bio_zone_write_plugging (bio)); -if (should_fail_bio (bio)) -bio_io_error (bio); -else if (!blk_throtl_bio (bio)) +if (!blk_throtl_bio (bio))submit_bio_noacct_nocheck (bio, true); -return split; }EXPORT_SYMBOL_GPL (bio_submit_split_bioset);
block/blk.h +0 -1
@@ -646,7 +646,6 @@ extern const struct address_space_operations def_blk_aops;int disk_register_independent_access_ranges (struct gendisk *disk);void disk_unregister_independent_access_ranges (struct gendisk *disk); -int should_fail_bio (struct bio *bio);#ifdef CONFIG_FAIL_MAKE_REQUEST bool should_fail_request (struct block_device *part,unsigned int bytes);#else /* CONFIG_FAIL_MAKE_REQUEST */
Signature · 1 line
2.53.0
PATCH 3/9 block: refactor should_fail_bio and should_fail_request
In reply to Christoph HellwigMove the bdev flag checks into a helper and the blk-mq clone insers so that we can do only a single actual error injection for I/O to partitions instead of doing it twice.
--- block/blk-core.c | 28 ++++++++++++++-------------- block/blk-mq.c | 3 ++- block/blk.h | 5 ++--- include/linux/blk_types.h | 2 -- 4 files changed, 18 insertions(+), 20 deletions(-)
block/blk-core.c +14 -14
@@ -502,10 +502,9 @@ static int __init setup_fail_make_request(char *str) }__setup ("fail_make_request=" , setup_fail_make_request); -bool should_fail_request (struct block_device *part,unsigned int bytes) +bool should_fail_request (unsigned int bytes) { -return bdev_test_flag (part,BD_MAKE_IT_FAIL ) && -should_fail (&fail_make_request, bytes); +return should_fail (&fail_make_request, bytes); }static int __init fail_make_request_debugfs (void )
@@ -539,11 +538,13 @@ static inline void bio_check_ro(struct bio *bio) } } -static int should_fail_bio (struct bio *bio) +static inline bool may_fail_bio (struct bio *bio) { -if (should_fail_request (bdev_whole (bio->bi_bdev), bio->bi_iter.bi_size)) -return -EIO ; -return 0 ; +if (!IS_ENABLED (CONFIG_FAIL_MAKE_REQUEST )) +return false; +return bdev_test_flag (bio->bi_bdev,BD_MAKE_IT_FAIL ) || + (bio_flagged (bio,BIO_REMAPPED ) && +bdev_test_flag (bdev_whole (bio->bi_bdev),BD_MAKE_IT_FAIL )); }/*
@@ -577,8 +578,6 @@ static int blk_partition_remap(struct bio *bio) {struct block_device *p = bio->bi_bdev; -if (unlikely (should_fail_request (p, bio->bi_iter.bi_size))) -return -EIO ;if (bio_sectors (bio)) { bio->bi_iter.bi_sector += p->bd_start_sect; trace_block_bio_remap(bio, p->bd_dev,
@@ -723,10 +722,13 @@ static void __submit_bio_noacct_mq(struct bio *bio)void submit_bio_noacct_nocheck (struct bio *bio,bool split) { -if (should_fail_bio (bio)) { -bio_io_error (bio); -return ; +if (unlikely (may_fail_bio (bio))) { +if (should_fail_request (bio->bi_iter.bi_size)) { +bio_io_error (bio); +return ; + } } +blk_cgroup_bio_start (bio);if (!bio_flagged (bio,BIO_TRACE_COMPLETION )) {
@@ -799,8 +801,6 @@ void submit_bio_noacct(struct bio *bio) goto not_supported; } -if (should_fail_bio (bio)) - goto end_io;bio_check_ro (bio);if (!bio_flagged (bio,BIO_REMAPPED )) {if (unlikely (bio_check_eod (bio)))
block/blk-mq.c +2 -1
@@ -3275,7 +3275,8 @@ blk_status_t blk_insert_cloned_request(struct request *rq)return BLK_STS_IOERR ; } -if (q->disk &&should_fail_request (q->disk->part0,blk_rq_bytes (rq))) +if (q->disk &&bdev_test_flag (q->disk->part0,BD_MAKE_IT_FAIL ) && +should_fail_request (blk_rq_bytes (rq)))return BLK_STS_IOERR ; ret =blk_crypto_rq_get_keyslot (rq);
block/blk.h +2 -3
@@ -647,10 +647,9 @@ int disk_register_independent_access_ranges(struct gendisk *disk);void disk_unregister_independent_access_ranges (struct gendisk *disk);#ifdef CONFIG_FAIL_MAKE_REQUEST -bool should_fail_request (struct block_device *part,unsigned int bytes); +bool should_fail_request (unsigned int bytes);#else /* CONFIG_FAIL_MAKE_REQUEST */ -static inline bool should_fail_request (struct block_device *part, -unsigned int bytes) +static inline bool should_fail_request (unsigned int bytes) {return false; }
include/linux/blk_types.h +0 -2
@@ -51,9 +51,7 @@ struct block_device {#define BD_WRITE_HOLDER (1u<<9)#define BD_HAS_SUBMIT_BIO (1u<<10)#define BD_RO_WARNED (1u<<11) -#ifdef CONFIG_FAIL_MAKE_REQUEST #define BD_MAKE_IT_FAIL (1u<<12) -#endif dev_t bd_dev;struct address_space *bd_mapping;/* page cache */
Signature · 1 line
2.53.0
PATCH 4/9 block: move the FAIL_MAKE_REQUEST symbol from lib/ to block/
In reply to Christoph HellwigKeep the Kconfig symbol together with the code that it guards.
--- block/Kconfig | 6 ++++++ lib/Kconfig.debug | 6 ------ 2 files changed, 6 insertions(+), 6 deletions(-)
block/Kconfig +6 -0
@@ -209,6 +209,12 @@ config BLK_INLINE_ENCRYPTION_FALLBACK by falling back to the kernel crypto API when inline encryption hardware is not present. +config FAIL_MAKE_REQUEST + bool "Fault-injection capability for disk IO" + depends on FAULT_INJECTION + help + Provide fault-injection capability for disk IO. + source "block/partitions/Kconfig" config BLK_PM
lib/Kconfig.debug +0 -6
@@ -2116,12 +2116,6 @@ config FAULT_INJECTION_USERCOPY Provides fault-injection capability to inject failures in usercopy functions (copy_from_user(), get_user(), ...). -config FAIL_MAKE_REQUEST - bool "Fault-injection capability for disk IO" - depends on FAULT_INJECTION && BLOCK - help - Provide fault-injection capability for disk IO. - config FAIL_IO_TIMEOUT bool "Fault-injection capability for faking disk interrupts" depends on FAULT_INJECTION && BLOCK
Signature · 1 line
2.53.0
PATCH 5/9 block: add a macro to initialize the status table
In reply to Christoph HellwigPrepare for adding a new value to the error table by adding a macro to fill it.
--- block/blk-core.c | 45 +++++++++++++++++++++++++-------------------- 1 file changed, 25 insertions(+), 20 deletions(-)
block/blk-core.c +25 -20
@@ -132,39 +132,44 @@ inline const char *blk_op_str(enum req_op op) }EXPORT_SYMBOL_GPL (blk_op_str); +#define ENT (_tag, _errno, _desc) \ +[BLK_STS_##_tag] = { \ + .errno = _errno, \ + .name = _desc, \ +}static const struct {int errno;const char *name; } blk_errors[] = { - [BLK_STS_OK ] = {0 ,"" }, - [BLK_STS_NOTSUPP ] = { -EOPNOTSUPP ,"operation not supported" }, - [BLK_STS_TIMEOUT ] = { -ETIMEDOUT ,"timeout" }, - [BLK_STS_NOSPC ] = { -ENOSPC ,"critical space allocation" }, - [BLK_STS_TRANSPORT ] = { -ENOLINK ,"recoverable transport" }, - [BLK_STS_TARGET ] = { -EREMOTEIO ,"critical target" }, - [BLK_STS_RESV_CONFLICT ] = { -EBADE ,"reservation conflict" }, - [BLK_STS_MEDIUM ] = { -ENODATA ,"critical medium" }, - [BLK_STS_PROTECTION ] = { -EILSEQ ,"protection" }, - [BLK_STS_RESOURCE ] = { -ENOMEM ,"kernel resource" }, - [BLK_STS_DEV_RESOURCE ] = { -EBUSY ,"device resource" }, - [BLK_STS_AGAIN ] = { -EAGAIN ,"nonblocking retry" }, - [BLK_STS_OFFLINE ] = { -ENODEV ,"device offline" }, +ENT (OK ,0 ,"" ), +ENT (NOTSUPP , -EOPNOTSUPP ,"operation not supported" ), +ENT (TIMEOUT , -ETIMEDOUT ,"timeout" ), +ENT (NOSPC , -ENOSPC ,"critical space allocation" ), +ENT (TRANSPORT , -ENOLINK ,"recoverable transport" ), +ENT (TARGET , -EREMOTEIO ,"critical target" ), +ENT (RESV_CONFLICT , -EBADE ,"reservation conflict" ), +ENT (MEDIUM , -ENODATA ,"critical medium" ), +ENT (PROTECTION , -EILSEQ ,"protection" ), +ENT (RESOURCE , -ENOMEM ,"kernel resource" ), +ENT (DEV_RESOURCE , -EBUSY ,"device resource" ), +ENT (AGAIN , -EAGAIN ,"nonblocking retry" ), +ENT (OFFLINE , -ENODEV ,"device offline" ),/* device mapper special case, should not leak out: */ - [BLK_STS_DM_REQUEUE ] = { -EREMCHG ,"dm internal retry" }, +ENT (DM_REQUEUE , -EREMCHG ,"dm internal retry" ),/* zone device specific errors */ - [BLK_STS_ZONE_OPEN_RESOURCE ] = { -ETOOMANYREFS ,"open zones exceeded" }, - [BLK_STS_ZONE_ACTIVE_RESOURCE ] = { -EOVERFLOW ,"active zones exceeded" }, +ENT (ZONE_OPEN_RESOURCE , -ETOOMANYREFS ,"open zones exceeded" ), +ENT (ZONE_ACTIVE_RESOURCE , -EOVERFLOW ,"active zones exceeded" ),/* Command duration limit device-side timeout */ - [BLK_STS_DURATION_LIMIT ] = { -ETIME ,"duration limit exceeded" }, - - [BLK_STS_INVAL ] = { -EINVAL ,"invalid" }, +ENT (DURATION_LIMIT , -ETIME ,"duration limit exceeded" ), +ENT (INVAL , -EINVAL ,"invalid" ),/* everything else not covered above: */ - [BLK_STS_IOERR ] = { -EIO ,"I/O" }, +ENT (IOERR , -EIO ,"I/O" ), }; +#undef ENTblk_status_t errno_to_blk_status (int errno) {
Signature · 1 line
2.53.0
PATCH 6/9 block: add a "tag" for block status codes
In reply to Christoph HellwigThe full name of the status codes is not good for user interfaces as it can contain white spaces. Add the name of the status code without the BLK_STS_ prefix as a tag so that it can be used for user interfaces.
--- block/blk-core.c | 24 ++++++++++++++++++++++++ block/blk.h | 2 ++ 2 files changed, 26 insertions(+)
block/blk-core.c +24 -0
@@ -135,10 +135,12 @@ EXPORT_SYMBOL_GPL(blk_op_str);#define ENT (_tag, _errno, _desc) \ [BLK_STS_##_tag] = { \ .errno = _errno, \ + .tag = __stringify(_tag), \ .name = _desc, \ }static const struct {int errno; +const char *tag;const char *name; } blk_errors[] = {ENT (OK ,0 ,"" ),
@@ -203,6 +205,28 @@ const char *blk_status_to_str(blk_status_t status)return blk_errors[idx].name; } +const char *blk_status_to_tag (blk_status_t status) +{ +int idx = (__force int )status; + +if (WARN_ON_ONCE (idx >=ARRAY_SIZE (blk_errors))) +return "<null>" ; +return blk_errors[idx].tag; +} + +blk_status_t tag_to_blk_status (const char *tag) +{ +int i; + +for (i =0 ; i <ARRAY_SIZE (blk_errors); i++) { +if (blk_errors[i].tag && + !strcmp (blk_errors[i].tag, tag)) +return (__force blk_status_t )i; + } + +return BLK_STS_OK ; +} +/** * blk_sync_queue - cancel any pending callbacks on a queue * @q: the queue
block/blk.h +2 -0
@@ -50,6 +50,8 @@ struct blk_flush_queue *blk_alloc_flush_queue(int node, int cmd_size,void blk_free_flush_queue (struct blk_flush_queue *q);const char *blk_status_to_str (blk_status_t status); +const char *blk_status_to_tag (blk_status_t status); +blk_status_t tag_to_blk_status (const char *tag);bool __blk_mq_unfreeze_queue (struct request_queue *q,bool force_atomic);bool blk_queue_start_drain (struct request_queue *q);
Signature · 1 line
2.53.0
PATCH 7/9 block: add a str_to_blk_op helper
In reply to Christoph HellwigAdd a helper to find the REQ_OP_XYZ constant from the "XYZ" string. This will be used for the error injection debugfs interface.
--- block/blk-core.c | 13 +++++++++++++ block/blk.h | 1 + 2 files changed, 14 insertions(+)
block/blk-core.c +13 -0
@@ -132,6 +132,19 @@ inline const char *blk_op_str(enum req_op op) }EXPORT_SYMBOL_GPL (blk_op_str); +enum req_op str_to_blk_op (const char *op) +{ +int i; + +for (i =0 ; i <ARRAY_SIZE (blk_op_name); i++) { +if (blk_op_name[i] && + !strcmp (blk_op_name[i], op)) +return i; + } + +return REQ_OP_LAST ; +} +#define ENT (_tag, _errno, _desc) \ [BLK_STS_##_tag] = { \ .errno = _errno, \
block/blk.h +1 -0
@@ -52,6 +52,7 @@ void blk_free_flush_queue(struct blk_flush_queue *q);const char *blk_status_to_str (blk_status_t status);const char *blk_status_to_tag (blk_status_t status);blk_status_t tag_to_blk_status (const char *tag); +enum req_op str_to_blk_op (const char *op);bool __blk_mq_unfreeze_queue (struct request_queue *q,bool force_atomic);bool blk_queue_start_drain (struct request_queue *q);
Signature · 1 line
2.53.0
PATCH 8/9 block: add configurable error injection
In reply to Christoph HellwigAdd a new block error injection interface that allows to inject specific status code for specific ranges.
--- Documentation/block/error-injection.rst | 59 +++++ Documentation/block/index.rst | 1 + block/Makefile | 1 + block/blk-core.c | 2 + block/blk-sysfs.c | 4 + block/blk.h | 15 ++ block/error-injection.c | 299 ++++++++++++++++++++++++ block/genhd.c | 4 + include/linux/blkdev.h | 5 + 9 files changed, 390 insertions(+) create mode 100644 Documentation/block/error-injection.rst create mode 100644 block/error-injection.c
Documentation/block/error-injection.rst new file +59 -0
@@ -0,0 +1,59 @@ +.. SPDX-License-Identifier: GPL-2.0 + +============================ +Configurable Error Injection +============================ + +Overview +-------- + +Configurable error injection allows injecting specific block layer status codes +for ranges of a block device. Error can be injected unconditional, or with a +given probability. + +To use configurable error injection, CONFIG_FAIL_MAKE_REQUEST must be enabled. + +The only interface is the error_injection debugfs file, which is created for +each registered gendisk. Writes to this file are used to create or delete rules +and reads return a list of the current error injection sites. + +Options +------- + +The following options specify the operations: + +=================== ======================================================= +add add a new rule +removeall remove all existing rules +=================== ======================================================= + +The following options specify the details of the rule for the add operation: + +=================== ======================================================= +op=%s block layer operation this rule applies to, e.g. READ + or WRITE. + Mandatory. +start=%u First block layer sector the rule applies to. + Optional, defaults to 0. +nr_sectors=%u Number of sectors this rule applies. + Optional, defaults to the remainder of the device. +status=%s Status to return. + Mandatory. +chance=%u Only return a failure with a likelihood of 1/chance. + Optional, defaults to 1 (always). +=================== ======================================================= + +Example +------- + +Return BLK_STS_IOERR for one in 10 reads of sector 0 of /dev/nvme0n1: + + $ echo 'add,op=READ,start=0,status=IOERR,chance=10' > /sys/kernel/debug/block/nvme0n1/error_injection + +Return BLK_STS_MEDIUM for every write to /dev/nvme0n1: + + $ echo 'add,op=WRITE,start=0,status=MEDIUM' > /sys/kernel/debug/block/nvme0n1/error_injection + +Remove all rules for /dev/nvme0n1: + + $ echo 'removeall' > /sys/kernel/debug/block/nvme0n1/error_injection
Documentation/block/index.rst +1 -0
@@ -22,3 +22,4 @@ Block switching-sched writeback_cache_control ublk + error-injection
block/Makefile +1 -0
@@ -11,6 +11,7 @@ obj-y := bdev.o fops.o bio.o elevator.o blk-core.o blk-sysfs.o \ genhd.o ioprio.o badblocks.o partitions/ blk-rq-qos.o \ disk-events.o blk-ia-ranges.o early-lookup.o +obj-$(CONFIG_FAIL_MAKE_REQUEST) += error-injection.o obj-$(CONFIG_BLK_DEV_BSG_COMMON) += bsg.o obj-$(CONFIG_BLK_DEV_BSGLIB) += bsg-lib.o obj-$(CONFIG_BLK_CGROUP) += blk-cgroup.o
block/blk-core.c +2 -0
@@ -765,6 +765,8 @@ static void __submit_bio_noacct_mq(struct bio *bio)void submit_bio_noacct_nocheck (struct bio *bio,bool split) {if (unlikely (may_fail_bio (bio))) { +if (blk_error_inject (bio)) +return ;if (should_fail_request (bio->bi_iter.bi_size)) {bio_io_error (bio);return ;
block/blk-sysfs.c +4 -0
@@ -933,6 +933,8 @@ static void blk_debugfs_remove(struct gendisk *disk)blk_debugfs_lock_nomemsave (q);blk_trace_shutdown (q); +if (IS_ENABLED (CONFIG_FAIL_MAKE_REQUEST )) +blk_error_injection_exit (disk);debugfs_remove_recursive (q->debugfs_dir); q->debugfs_dir =NULL ; q->sched_debugfs_dir =NULL ;
@@ -963,6 +965,8 @@ int blk_register_queue(struct gendisk *disk) memflags =blk_debugfs_lock (q); q->debugfs_dir =debugfs_create_dir (disk->disk_name, blk_debugfs_root); +if (IS_ENABLED (CONFIG_FAIL_MAKE_REQUEST )) +blk_error_injection_init (disk);if (queue_is_mq (q))blk_mq_debugfs_register (q);blk_debugfs_unlock (q, memflags);
block/blk.h +15 -0
@@ -781,4 +781,19 @@ static inline void blk_debugfs_unlock(struct request_queue *q, memalloc_noio_restore(memflags); } +void blk_error_injection_init (struct gendisk *disk); +void blk_error_injection_exit (struct gendisk *disk); + +bool __blk_error_inject (struct bio *bio); +static inline bool blk_error_inject (struct bio *bio) +{ +#ifdef CONFIG_FAIL_MAKE_REQUEST +struct gendisk *disk = bio->bi_bdev->bd_disk; + +if (!list_empty_careful (&disk->error_injection_list)) +return __blk_error_inject (bio); +#endif +return false; +} +#endif /* BLK_INTERNAL_H */
block/error-injection.c new file +299 -0
299 lines not on this page · raw message
block/genhd.c +4 -0
@@ -1485,6 +1485,10 @@ struct gendisk *__alloc_disk_node(struct request_queue *q, int node_id,lockdep_init_map (&disk->lockdep_map,"(bio completion)" , lkclass,0 );#ifdef CONFIG_BLOCK_HOLDER_DEPRECATED INIT_LIST_HEAD (&disk->slave_bdevs); +#endif +#ifdef CONFIG_FAIL_MAKE_REQUEST +mutex_init (&disk->error_injection_lock); +INIT_LIST_HEAD (&disk->error_injection_list);#endif mutex_init (&disk->rqos_state_mutex);kobject_init (&disk->queue_kobj, &blk_queue_ktype);
include/linux/blkdev.h +5 -0
@@ -227,6 +227,11 @@ struct gendisk {*/ struct blk_independent_access_ranges *ia_ranges; +#ifdef CONFIG_FAIL_MAKE_REQUEST +struct mutex error_injection_lock; +struct list_head error_injection_list; +#endif +struct mutex rqos_state_mutex;/* rqos state change mutex */ };
Signature · 1 line
2.53.0
Re: [PATCH 8/9] block: add configurable error injection
Quoted 16 lines · Christoph Hellwig
block/error-injection.c new file +16
-0+static int error_inject_add (struct gendisk *disk,enum req_op op, +sector_t start,u64 nr_sectors,blk_status_t status, +unsigned int chance) +{ +struct blk_error_inject *inj; + +if (op ==REQ_OP_LAST ) +return -EINVAL ; +if (status ==BLK_STS_OK ) +return -EINVAL ; +if (U64_MAX - nr_sectors < start) +return -EINVAL ; + +if (!nr_sectors) + nr_sectors =U64_MAX ; +
...
block/error-injection.c line 80
Quoted 4 lines
block/error-injection.c new file +4
-0+ + inj->op = op; + inj->start = start; + inj->end = start + nr_sectors -1 ;
When nr_sectors is 0, it is reset to U64_MAX so overflows if start > 1. I think you want to remove overriding nr_sectors to U64_MAX and do:
if (!nr_sectors) inj->end = U64_MAX; else if (U64_MAX - nr_sectors < start ) return -EINVAL; else inj->end = start + nr_sectors - 1;
block/error-injection.c line 92
Quoted 12 lines
block/error-injection.c new file +12
-0+ inj->status = status; + inj->chance = chance; + +/* +* Add to the front of the list so that newer entries can partially +* override other entries. This also intentional allows duplicate +* entries as there is no real reason to reject them. +*/ +mutex_lock (&disk->error_injection_lock); +if (!disk_live (disk)) { +mutex_unlock (&disk->error_injection_lock); +return -EINVAL ;
I think we've leaked 'inj' in this error case.
Review
Quoted 2 lines
(quoted diff) +2
-0+ } + list_add(&inj->entry, &disk->error_injection_list);
The __blk_error_inject interates this list with "list_for_each_entry_rcu", so shouldn't this be list_add_rcu to match?
Quoted 5 lines
block/error-injection.c new file +5
-0+mutex_unlock (&disk->error_injection_lock); + +bdev_set_flag (disk->part0,BD_MAKE_IT_FAIL ); +return 0 ; +}
<snip>
block/error-injection.c line 139
Quoted 6 lines
block/error-injection.c new file +6
-0+static const match_table_t opt_tokens = { + { Opt_add,"add" , }, + { Opt_removeall,"removeall" , }, + { Opt_op,"op=%s" , }, + { Opt_start,"start=%u" }, + { Opt_nr_sectors,"nr_sectors=%u" },
Shouldn't start and nr_sectors use %llu?
Quoted 3 lines
block/error-injection.c new file +3
-0+static ssize_t blk_error_injection_write (struct file *file, +const char __user *ubuf,size_t count,loff_t *pos) +{
...
block/error-injection.c line 189
Quoted 4 lines
block/error-injection.c new file +4
-0+ options =memdup_user_nul (ubuf, count); +if (!options) +return -ENOMEM ; +
On failure, memdup_user_nul returns an ERR_PTR rather than NULL.
if (IS_ERR(options)) return PTR_ERR(options);
block/error-injection.c line 242
Quoted 3 lines
block/error-injection.c new file +3
-0+case Removeall: +if (option_mask & ~Opt_removeall) +return -EINVAL ;
Leaking "options"? Should this be:
if (option_mask & ~Opt_removeall) {
ret = -EINVAL;
goto out_free_options;
}?
Quoted 12 lines
block/error-injection.c new file +12
-0+error_inject_removall (disk); +break ; +default : + ret = -EINVAL ; + } + +if (!ret) + ret = count; +out_free_options: +kfree (options); +return ret; +}
Re: [PATCH 8/9] block: add configurable error injection
Quoted 1 line · Keith Busch
When
nr_sectorsis 0, it is reset toU64_MAXso overflows if start > 1.
Yeah.
Quoted 8 lines
I think you want to remove overriding
nr_sectorstoU64_MAXand do:if (!nr_sectors) inj->end = U64_MAX; else if (U64_MAX - nr_sectors < start ) return -EINVAL; else inj->end = start + nr_sectors - 1;
I ended up ordering a bit differently for better readability, but yes.
Quoted 6 lines
block/error-injection.c line 92
Quoted 4 lines
block/error-injection.c new file +4
-0+mutex_lock (&disk->error_injection_lock); +if (!disk_live (disk)) { +mutex_unlock (&disk->error_injection_lock); +return -EINVAL ;I think we've leaked 'inj' in this error case.
Yes.
Quoted 6 lines
Review
Quoted 3 lines
(quoted diff) +2
-0+ } + list_add(&inj->entry, &disk->error_injection_list);The
__blk_error_injectinterates this list with "list_for_each_entry_rcu", so shouldn't this belist_add_rcuto match?
Yes.
Quoted 8 lines
block/error-injection.c line 139
Quoted 6 lines
block/error-injection.c new file +6
-0+static const match_table_t opt_tokens = { + { Opt_add,"add" , }, + { Opt_removeall,"removeall" , }, + { Opt_op,"op=%s" , }, + { Opt_start,"start=%u" }, + { Opt_nr_sectors,"nr_sectors=%u" },Shouldn't start and
nr_sectorsuse %llu?
lib/parser.c doesn't use those prefixes, it's a bit weird.
Quoted 8 lines
block/error-injection.c line 189
Quoted 3 lines
block/error-injection.c new file +3
-0+if (!options) +return -ENOMEM ; +On failure,
memdup_user_nulreturns anERR_PTRrather than NULL.if (IS_ERR(options)) return PTR_ERR(options);
Aarg, annoying. Because memdup_user does return NULL :(
Quoted 7 lines
block/error-injection.c line 242
Quoted 4 lines
block/error-injection.c new file +3
-0+case Removeall: +if (option_mask & ~Opt_removeall) +return -EINVAL ;Leaking "options"? Should this be:
Quoted 6 lines
if (option_mask & ~Opt_removeall) { ret = -EINVAL; goto out_free_options; }?
Yes.
Randy Dunlap
reviewerRe: [PATCH 8/9] block: add configurable error injection
In reply to Christoph HellwigQuoted 7 lines · Christoph Hellwig
Documentation/block/error-injection.rst new file +1
-0@@ -0,0 +1,59 @@ +.. SPDX-License-Identifier: GPL-2.0
Documentation/block/error-injection.rst line 11
Quoted 10 lines
Documentation/block/error-injection.rst new file +10
-0+ +============================ +Configurable Error Injection +============================ + +Overview +-------- + +Configurable error injection allows injecting specific block layer status codes +for ranges of a block device. Error can be injected unconditional, or with a
Errors can be injected unconditionally or with a
Quoted 7 lines
Documentation/block/error-injection.rst new file +7
-0+given probability. + +To use configurable error injection, CONFIG_FAIL_MAKE_REQUEST must be enabled. + +The only interface is the error_injection debugfs file, which is created for +each registered gendisk. Writes to this file are used to create or delete rules +and reads return a list of the current error injection sites.
[snip]
Signature · 1 line
~Randy
PATCH 9/9 block: move the fail request code
In reply to Christoph HellwigKeep all error injection in one place, and out of line for the main I/O submission fast path.
--- block/blk-core.c | 37 ++----------------------------------- block/error-injection.c | 30 ++++++++++++++++++++++++++++++ 2 files changed, 32 insertions(+), 35 deletions(-)
block/blk-core.c +2 -35
@@ -29,7 +29,6 @@#include <linux/swap.h> #include <linux/writeback.h> #include <linux/task_io_accounting_ops.h> -#include <linux/fault-inject.h> #include <linux/list_sort.h> #include <linux/delay.h> #include <linux/ratelimit.h>
@@ -534,32 +533,6 @@ bool blk_get_queue(struct request_queue *q) }EXPORT_SYMBOL (blk_get_queue); -#ifdef CONFIG_FAIL_MAKE_REQUEST - -static DECLARE_FAULT_ATTR (fail_make_request); - -static int __init setup_fail_make_request (char *str) -{ -return setup_fault_attr (&fail_make_request, str); -} -__setup ("fail_make_request=" , setup_fail_make_request); - -bool should_fail_request (unsigned int bytes) -{ -return should_fail (&fail_make_request, bytes); -} - -static int __init fail_make_request_debugfs (void ) -{ -struct dentry *dir =fault_create_debugfs_attr ("fail_make_request" , -NULL , &fail_make_request); - -return PTR_ERR_OR_ZERO (dir); -} - -late_initcall (fail_make_request_debugfs); -#endif /* CONFIG_FAIL_MAKE_REQUEST */ -static inline void bio_check_ro (struct bio *bio) {if (op_is_write (bio_op (bio)) &&bdev_read_only (bio->bi_bdev)) {
@@ -764,14 +737,8 @@ static void __submit_bio_noacct_mq(struct bio *bio)void submit_bio_noacct_nocheck (struct bio *bio,bool split) { -if (unlikely (may_fail_bio (bio))) { -if (blk_error_inject (bio)) -return ; -if (should_fail_request (bio->bi_iter.bi_size)) { -bio_io_error (bio); -return ; - } - } +if (unlikely (may_fail_bio (bio)) &&blk_error_inject (bio)) +return ;blk_cgroup_bio_start (bio);
block/error-injection.c +30 -0
@@ -4,6 +4,7 @@*/ #include <linux/debugfs.h> #include <linux/blkdev.h> +#include <linux/fault-inject.h> #include <linux/parser.h> #include <linux/seq_file.h> #include "blk.h"
@@ -47,6 +48,13 @@ bool __blk_error_inject(struct bio *bio) } }rcu_read_unlock (); + +/* legacy I/O error injection */ +if (should_fail_request (bio->bi_iter.bi_size)) { +bio_io_error (bio); +return true; + } +return false; }
@@ -297,3 +305,25 @@ void blk_error_injection_exit(struct gendisk *disk) {error_inject_removall (disk); } + +static DECLARE_FAULT_ATTR (fail_make_request); + +bool should_fail_request (unsigned int bytes) +{ +return should_fail (&fail_make_request, bytes); +} + +static int __init setup_fail_make_request (char *str) +{ +return setup_fault_attr (&fail_make_request, str); +} +__setup ("fail_make_request=" , setup_fail_make_request); + +static int __init fail_make_request_debugfs (void ) +{ +struct dentry *dir =fault_create_debugfs_attr ("fail_make_request" , +NULL , &fail_make_request); + +return PTR_ERR_OR_ZERO (dir); +} +late_initcall (fail_make_request_debugfs);
Signature · 1 line
2.53.0
Keith Busch
Reviewed-byQuoted 5 lines · Christoph Hellwig
Hi all,
this series adds a new configurable block error injection facility. We already have a few to inject block errors, but unfortunately most of them are either not very useful or hard to use, or both:
Looks great! I just have some comments on patch 8/9, but for the rest:
Quoted 20 lines · Christoph Hellwig
Hi all,
this series adds a new configurable block error injection facility. We already have a few to inject block errors, but unfortunately most of them are either not very useful or hard to use, or both:
- The
fail_make_requestfailure injection point can't distinguish different commands, different ranges in the file and can only injection plain I/O errors.
- theshould_fail_bio'dynamic' failure injection has all the same issues asfail_make_request
- dm-error can only fail all command in the table usingBLK_STS_IOERRand requires setting up a new block device
- dm-flakey and dm-dust allow all kinds of configurability, but still don't have good error selection, no good support for non-read/write commands and are limited to the dm table alignment requirements, which for zoned devices enforces setting them up for an entire zone. They also once again require setting up a stacked block device, which is really annoying in harnesses like xfstests
Quoted 4 lines
This series adds a new debugfs-based block layer error injection that allows to configure what operations and ranges the injection applied to, and what status to return. It also allows to configure a failure ratio similar to the xfs errortag injection.
I wonder if the block layer would be interested in moving block error injection off the should_fail() fault injection framework and extending the ALLOW_ERROR_INJECTION annotation instead and offloading all the debugfs configuration logic (block/error-injection.c) into eBPF?
I talked about moderr [1] at LPC 2025. It's a simple error injection tool in eBPF for the module subsystem. The suggested direction there was to generalize the tool to ideally to no tool at all, and leverage bpftrace to describe the error injection conditions a given subsystem needs to be tested under. That would let blktests, for example, absorb that and simplify the configuration logic this series adds in the kernel for debugfs.
A previous attempt to add inline error injection [2] was rejected as too intrusive / source-polluting; the eBPF approach solves that, since the injection logic lives in a standalone tool/script rather than in the kernel sources.
What do you guys think?
[1] https://lpc.events/event/19/contributions/2204/
[2] https://lore.kernel.org/all/[email protected]/
Quoted 4 lines · Daniel Gomez
I wonder if the block layer would be interested in moving block error injection off the
should_fail()fault injection framework and extending theALLOW_ERROR_INJECTIONannotation instead and offloading all the debugfs configuration logic (block/error-injection.c) into eBPF?
I've looked into plain ALLOW_ERROR_INJECTION-based injection and it is not very useful. I didn't even now eBPF could use it, but I looked into other eBPF injections and at least for my uses cases it was a bit of a mess. I'd have to allow access to certain bio fields and would have create a stable UAPI for commands and status using the fake BTF struct access which really would not be a good idea here as we need to be able to change internals. Additionally having fully BTF-enabled toolchains in test VMs is not great either.
I've also not actually found any good map type for range lookups, which is kinda essential here.
Quoted 7 lines
I talked about moderr [1] at LPC 2025. It's a simple error injection tool in eBPF for the module subsystem. The suggested direction there was to generalize the tool to ideally to no tool at all, and leverage bpftrace to describe the error injection conditions a given subsystem needs to be tested under. That would let blktests, for example, absorb that and simplify the configuration logic this series adds in the kernel for debugfs.
I don't think pulling in ebpftrace for simple error injection is a winning proposition..
Quoted 3 lines
A previous attempt to add inline error injection [2] was rejected as too intrusive / source-polluting;
I'm not sure a single hand waivy comment counts as rejection, although I'm not a huge fan of setup_fault_attr - it makes a mess of debugfs and creates a lot of boilerplate for a single not very much configurable call site. That might be ok for something like the make_request case (although I think we can do better as shown in this patch), but for making random functions fail it is a lot of overhead. These injections points also are not anywhere near stable enough to be exposed.
Quoted 8 lines · Christoph Hellwig
Quoted 4 lines · Daniel Gomez
I wonder if the block layer would be interested in moving block error injection off the
should_fail()fault injection framework and extending theALLOW_ERROR_INJECTIONannotation instead and offloading all the debugfs configuration logic (block/error-injection.c) into eBPF?I've looked into plain ALLOW_ERROR_INJECTION-based injection and it is not very useful. I didn't even now eBPF could use it,
For context: Josef Bacik introduced it first for BPF only (BPF_ALLOW_ERROR_INJECTION). Masami Hiramatsu then generalized it into the error injection framework, renaming it to ALLOW_ERROR_INJECTION and adding the fail_function debugfs interface (which calls should_fail()). So annotating a function with ALLOW_ERROR_INJECTION gives you both backends at once: debugfs (fail_function) and eBPF (bpf_override_return()).
Quoted 3 lines
but I
looked into other eBPF injections and at least for my uses cases it was a bit of a mess.
Agreed, and I had not looked closely enough at the series before proposing the wrong primitives. ALLOW_ERROR_INJECTION / bpf_override_return() are not sufficient here: bpf_override_return() cannot set bio->bi_status or call bio_endio(), which I think is the key operation here.
IIU the series correctly, and oversimplifying: when injection is enabled and a bio matches, the block layer completes the bio inline with the chosen blk_status_t (the status= rule from debugfs) via bio_endio_status(). The submission path returns to the caller immediately, with the bio already in the error state. Nothing is ever sent to the device, but the completion path sees the injected error.
submit_bio() -> submit_bio_noacct() -> submit_bio_noacct_nocheck()
+-- Path 1: no match -> continue normal IO submission
+-- Path 2: match (diskN) -> blk_error_inject()
-> bio_endio_status(bio, inj->status)
-> bio_endio()
// error injected. bio completedSo this is bio mutating + a bio_endio() call, not a return override. That can't be solved with ALLOW_ERROR_INJECTION. But we can use BPF_PROG_TYPE_STRUCT_OPS instead: the kernel keeps ownership of the bio_endio_status() call, and the eBPF program only drives the policy, ie. which blk_status_t to return for a given bio, based on whatever heuristics it implements.
Quoted 1 line
I'd have to allow access to certain bio
With struct_ops the bio is passed to the ebpf side as read-only, bio fields can be read to decide the policy but cannot write them. Is read-only access to bio fields itself a concern?
Quoted 3 lines
fields and would have create a stable UAPI for commands and status using the fake BTF struct access which really would not be a good idea here as we need to be able to change internals.
That should not be a problem at all. With CO-RE (compile once, run everywhere) the program resolves the bio field offsets against the BTF of the kernel it loads on, so it adapts dynamically if the layout changes. The contract is just the struct_ops callback signature: a struct bio * argument and a blk_status_t return. And that doesn't imply any UAPI commitment AFAIK.
Quoted 2 lines
Additionally
having fully BTF-enabled toolchains in test VMs is not great either.
Are you referring to the old BCC toolchain requirements [1]? This is solved in CO-RE [2]. The toolchain (Clang/LLVM, pahole) stays on the build host; the test VM only needs the prebuilt BPF object, libbpf at runtime, and the kernel's own BTF (CONFIG_DEBUG_INFO_BTF). No compiler or BTF toolchain is required inside the VM. Clang/LLVM 10+ is enough to build CO-RE libbpf tools [3].
Quoted 3 lines
I've also not actually found any good map type for range lookups, which is kinda essential here.
Are you referring to the bio sector range comparison in __blk_error_inject()? I don't think that needs to be delegated to a BPF map (Documentation/bpf/maps.rst). The ebpf side has direct access to the bio fields, so it can apply the same sector/op filtering __blk_error_inject() does today. That match is already a linear list walk, so the ebpf program just runs the same [start, end] condition check.
In summary: what do you think of evolving this series
into eBPF, but BPF_PROG_TYPE_STRUCT_OPS instead of ALLOW_ERROR_INJECTION/bpf_override_return()?. The configurable debugfs injection is a nice enhancement over what we had, but the matching is static (op/start/nr_sectors/status/chance) per gendisk. A struct_ops hook would expand the failure model to anything derivable from the bio (plus any state the program keeps), with the kernel still owning the bio_endio_status() mutation and only the policy moving out of the tree. The benefit is the flexibility to express the policy without hard-coding the model in the kernel.
The open question is whether programmable injection is something we want to support in-tree, or whether more debugfs knobs suffice. Thoughts?
Quoted 6 lines · Daniel Gomez
IIU the series correctly, and oversimplifying: when injection is enabled and a bio matches, the block layer completes the bio inline with the chosen
blk_status_t(the status= rule from debugfs) viabio_endio_status(). The submission path returns to the caller immediately, with the bio already in the error state. Nothing is ever sent to the device, but the completion path sees the injected error.
Yes.
Quoted 5 lines
Quoted 1 line
I'd have to allow access to certain bio
With
struct_opsthe bio is passed to the ebpf side as read-only, bio fields can be read to decide the policy but cannot write them. Is read-only access to bio fields itself a concern?
Yes, for the reason state below.
Quoted 11 lines
Quoted 4 lines
fields and would have create a stable UAPI for commands and status using the fake BTF struct access which really would not be a good idea here as we need to be able to change internals.
That should not be a problem at all. With CO-RE (compile once, run everywhere) the program resolves the bio field offsets against the BTF of the kernel it loads on, so it adapts dynamically if the layout changes. The contract is just the
struct_opscallback signature: a struct bio * argument and ablk_status_treturn. And that doesn't imply any UAPI commitment AFAIK.
That assumes the fields even exists in this form. And we definitively need to do things about the iter going ahead. In other words we have to build up a huge abstraction here first.
Quoted 9 lines
Quoted 2 lines
Additionally
having fully BTF-enabled toolchains in test VMs is not great either.Are you referring to the old BCC toolchain requirements [1]? This is solved in CO-RE [2]. The toolchain (Clang/LLVM, pahole) stays on the build host; the test VM only needs the prebuilt BPF object, libbpf at runtime, and the kernel's own BTF (
CONFIG_DEBUG_INFO_BTF). No compiler or BTF toolchain is required inside the VM. Clang/LLVM 10+ is enough to build CO-RE libbpf tools [3].
libbpf and BTF support is what I mean. And of course whatever code you write to communicate with it as a simple shell script won't work any more.
Quoted 2 lines
Are you referring to the bio sector range comparison in
__blk_error_inject()?
Yes.
Quoted 6 lines
I don't think that needs to be delegated to a
BPF map (Documentation/bpf/maps.rst). The ebpf side has direct access to the bio fields, so it can apply the same sector/op filtering__blk_error_inject()does today. That match is already a linear list walk, so the ebpf program just runs the same [start, end] condition check.
Which means I need to write eBPF code for every specific range I want to inject, which makes the thing very hard to use.
Quoted 3 lines
In summary: what do you think of evolving this series into eBPF, but
BPF_PROG_TYPE_STRUCT_OPSinstead of ALLOW_ERROR_INJECTION/bpf_override_return()?.
As said before, I prototyped it and it was a mess. Having about 300 lines of simple code that can be directly used from a shell script seems much preferable.