ublk: don't dispatch to canceled io commands
Subsystems (5)
Ming Lei
maintainerv3 cover ublk: don't dispatch to canceled io commands
Hi,
Josef reported three ways in which ublk dispatches a request to a canceled io command, which oopses on the NULL io->cmd in ublk_queue_cmd() [1]:
1) STOP_DEV, then START_DEV
2) partial FETCH: a task fetches some tags and exits, then another task fetches the rest
3) user recovery: one queue's task exits before all queues are ready
This series fixes them in a simpler way, without extra locking in the I/O path:
- patch 1: a FETCH round which saw a cancel stays canceling until the server is gone (2, 3)
- patch 2-3: an io command is marked cancelable before it is published
- patch 4-5: the release resets the FETCH round, also without a disk and under ub->mutex
- patch 6: don't warn when the cancel callback finds io->cmd taken by a cancel from the control path
- patch 7: STOP_DEV holds a reference on /dev/ublkcN while it cancels, so no START_DEV, FETCH or new server can slip in (1)
- patch 8 (Josef): START_DEV and END_USER_RECOVERY refuse to go live after an io command was canceled (2, 3)
- patch 9-11: selftests
- patch 12 (Li RongQing): ublk_report_zones() passes a per-chunk index to disk_report_zone(); use the index in the whole report
Tested with all ublk selftests under lockdep on for-7.4/block; v1 was also tested under KASAN.
V3:
- rebase on for-7.4/block
- add Li RongQing's "ublk: use global index for reported zones" (patch 12)
V2:
- add Josef's "ublk: refuse to go live after an io command was canceled", so routes 2 and 3 get -EBUSY instead of a live device failing every request
- split the cancel callback's WARN change out of the STOP_DEV patch (patch 6)
- document in ublk.rst what a server sees after STOP_DEV (Josef)
- kublk: stop a batch thread when PREP_IO_CMDS is aborted, instead of asserting and leaving kublk add waiting forever (Josef)
- selftest: partial_fetch and recovery expect -EBUSY; add a batch STOP_DEV race mode with kublk
- commit log fixes (patch 4, patch 7)
- add Josef's Tested-by
The release work deciding "no disk" before ub->mutex (Josef's point 4) needs a START_DEV for a server which has already exited: no new server can open /dev/ublkcN until that work is done, and patch 8 refuses such a START_DEV with -EBUSY, so it gets no extra patch.
V2: https://lore.kernel.org/linux-block/[email protected]/
V1: https://lore.kernel.org/linux-block/[email protected]/
[1] https://lore.kernel.org/linux-block/[email protected]/
Thanks,
Ming
Josef Bacik (1):
ublk: refuse to go live after an io command was canceled
Li RongQing (1):
ublk: use global index for reported zones
Ming Lei (10):
ublk: keep a canceled FETCH round canceling until the server is gone
ublk: mark the io command cancelable before publishing it
ublk: mark the batch fetch command cancelable before linking it
ublk: reset the FETCH round in release also without a disk
ublk: reset the FETCH round under ub->mutex
ublk: allow a NULL io->cmd in the cancel callback
ublk: let STOP_DEV cancel the server's commands before its release
selftests: ublk: move the control command helpers into ctrl.c
selftests: ublk: stop a batch thread when PREP_IO_CMDS fails
selftests: ublk: add test for going live over canceled io commands
Documentation/block/ublk.rst | 17 +- drivers/block/ublk_drv.c | 246 ++++- tools/testing/selftests/ublk/.gitignore | 1 + tools/testing/selftests/ublk/Makefile | 6 +- tools/testing/selftests/ublk/batch.c | 13 +- tools/testing/selftests/ublk/ctrl.c | 268 +++++ tools/testing/selftests/ublk/kublk.c | 271 ----- tools/testing/selftests/ublk/kublk.h | 32 + .../testing/selftests/ublk/test_generic_18.sh | 69 ++ .../selftests/ublk/ublk_cancel_ready.c | 957 ++++++++++++++++++ 10 files changed, 1551 insertions(+), 329 deletions(-) create mode 100644 tools/testing/selftests/ublk/ctrl.c create mode 100755 tools/testing/selftests/ublk/test_generic_18.sh create mode 100644 tools/testing/selftests/ublk/ublk_cancel_ready.c
base-commit: 69ae59173a5668d015ab2733c8628ed542f00ccc
Signature · 1 line
2.55.0
Ming Lei
maintainerPATCH v3 1/12 ublk: keep a canceled FETCH round canceling until the server is gone
A cancel completes a fetched io command and sets io->cmd to NULL, but the io still counts as ready. Only ubq->canceling stops ublk_queue_rq() from using the NULL io->cmd. Two paths clear that flag too early:
partial FETCH round recovery, two queues
------------------- --------------------
task A: FETCH tags 0-2 q0 ready: q0->canceling = false
task A exits: q0 task exits:
start_cancel(): mark queues start_cancel(): ub->canceling
tags 0-2: io->cmd = NULL is still set -> q0 not marked
task B: FETCH tag 3 q0: io->cmd = NULL
queue ready: canceling = false q1 ready
START_DEV END_USER_RECOVERY
read -> ublk_queue_cmd(NULL) read on q0 -> ublk_queue_cmd(NULL)A FETCH round runs from the open of /dev/ublkcN until ublk_reset_ch_dev() resets the queues. A canceled command can't be fetched again in the same round. So the ready check only needs one
fact: did this round see a cancel? ub->canceling already records it:ublk_start_cancel() sets it under cancel_mutex before any command is taken. It is only cleared too early, when all queues are ready.
Fix:
- clear ub->canceling only in ublk_reset_ch_dev()
- when a queue gets ready, clear ubq->canceling only if ub->canceling is not set, and check it under cancel_mutex
The check and the marking are ordered by cancel_mutex:
check first: queue cleared -> start_cancel() marks it again
-> command taken
cancel first: check sees ub->canceling -> queue stays canceling
While ub->canceling is set, no queue clears its flag. So the "all queues marked" shortcut in ublk_start_cancel() is correct again.
UBLK_F_BATCH_IO follows the same rule. It does not dispatch through io->cmd, so it never hit the oops, but a round which lost a fetch command is treated the same way: it has to start over.
--- drivers/block/ublk_drv.c | 54 ++++++++++++++++++++++++---------------- 1 file changed, 33 insertions(+), 21 deletions(-)
drivers/block/ublk_drv.c +33 -21
@@ -334,6 +334,11 @@ struct ublk_device {u16 nr_queue_ready;bool unprivileged_daemons;struct mutex cancel_mutex; +/* +* A cancel started in this FETCH round. Set by ublk_set_canceling(), +* cleared only by ublk_reset_ch_dev() when a new round starts. While +* it is set, no queue clears its ->canceling. +*/ bool canceling;pid_t ublksrv_tgid;struct delayed_work exit_work;
@@ -2415,6 +2420,11 @@ static void ublk_reset_ch_dev(struct ublk_device *ub)spin_unlock (&ubq->cancel_lock); } +/* a new FETCH round starts, the queues stay canceling until ready */ +mutex_lock (&ub->cancel_mutex); + ub->canceling = false; +mutex_unlock (&ub->cancel_mutex); +/* set to NULL, otherwise new tasks cannot mmap io_cmd_buf */ ub->mm =NULL ; ub->nr_queue_ready =0 ;
@@ -3024,11 +3034,19 @@ static void ublk_reset_io_flags(struct ublk_queue *ubq, struct ublk_io *io) }/* reset per-queue io flags */ -static void ublk_queue_reset_io_flags (struct ublk_queue *ubq) +static void ublk_queue_reset_io_flags (struct ublk_device *ub, +struct ublk_queue *ubq) { -spin_lock (&ubq->cancel_lock); - ubq->canceling = false; -spin_unlock (&ubq->cancel_lock); +/* +* A cancel in this FETCH round took a command which still counts as +* ready, so the queue has to stay canceling. ub->canceling is set +* under cancel_mutex before any command is taken: either we see it +* here, or the cancel marks this queue again later. +*/ +mutex_lock (&ub->cancel_mutex); +if (!ub->canceling) + ubq->canceling = false; +mutex_unlock (&ub->cancel_mutex); ubq->fail_io = false; }
@@ -3050,24 +3068,17 @@ static void ublk_mark_io_ready(struct ublk_device *ub, u16 q_id, ub->nr_queue_ready++;/* -* Reset queue flags as soon as this queue is ready. -* This clears the canceling flag, allowing batch FETCH commands -* to succeed during recovery without waiting for all queues. +* Reset queue flags as soon as this queue is ready. Unless +* this round saw a cancel, this clears the canceling flag, +* allowing batch FETCH commands to succeed during recovery +* without waiting for all queues. */ -ublk_queue_reset_io_flags (ubq); +ublk_queue_reset_io_flags (ub, ubq); } -/* Check if all queues are ready */ -if (ublk_dev_ready (ub)) { -/* -* All queues ready - clear device-level canceling flag -* and wake ublk_dev_ready() waiters. -*/ -mutex_lock (&ub->cancel_mutex); - ub->canceling = false; -mutex_unlock (&ub->cancel_mutex); +/* All queues ready - wake ublk_dev_ready() waiters */ +if (ublk_dev_ready (ub))wake_up_var (&ub->nr_queue_ready); - } }static inline int ublk_check_cmd_op (u32 cmd_op)
@@ -4433,9 +4444,10 @@ static bool ublk_validate_user_pid(struct ublk_device *ub, pid_t ublksrv_pid)/* * Wait until all queues have fetched their I/O commands, and return with -* ub->mutex held and readiness guaranteed: then every queue's ->canceling -* is cleared. Ready may regress between wakeup and mutex_lock() (F_BATCH -* UNPREP, daemon death), so re-check it under the mutex and wait again. +* ub->mutex held and readiness guaranteed. The queues stay canceling if +* this round saw a cancel, see ublk_queue_reset_io_flags(). Ready may +* regress between wakeup and mutex_lock() (F_BATCH UNPREP, daemon death), +* so re-check it under the mutex and wait again. */ static int ublk_wait_dev_ready_and_lock (struct ublk_device *ub) {
Signature · 1 line
2.55.0
Ming Lei
maintainerPATCH v3 2/12 ublk: mark the io command cancelable before publishing it
In reply to Ming LeiFETCH, COMMIT_AND_FETCH and NEED_GET_DATA publish the command in io->cmd before marking it cancelable. A cancel from the control path (STOP_DEV, QUIESCE_DEV) can complete it in between:
issue path control-path cancel
io->cmd = C
take C, io_uring_cmd_done(C):
C not marked, nothing to unlink
ublk_prep_cancel(C)
the completed C is linked on the cancelable list
ring exit: the cancel walk hits it → GPF (KASAN, with a delay added)
Mark the command before ublk_fill_io_cmd() publishes it. The handlers run with uring_lock held, as io_uring's cancel walk does, so marking takes no lock and the walk can't see a marked command before it is published. smp_wmb() orders the mark before the io->cmd store; the cancel loads cmd with READ_ONCE(io->cmd) before reading cmd->flags.
A marked command has to be completed by io_uring_cmd_done(), so a failed FETCH and the inline UBLK_IO_RES_OK of NEED_GET_DATA now do that. The CQE is the same.
--- drivers/block/ublk_drv.c | 27 +++++++++++++++++++++------ 1 file changed, 21 insertions(+), 6 deletions(-)
drivers/block/ublk_drv.c +21 -6
@@ -2800,7 +2800,8 @@ static void ublk_cancel_cmd(struct ublk_queue *ubq, u16 tag, done = !!(io->flags &UBLK_IO_FLAG_CANCELED );if (!done) { io->flags |=UBLK_IO_FLAG_CANCELED ; - cmd = io->cmd; +/* dependency ordered against smp_wmb() in ublk_prep_cancel() */ + cmd =READ_ONCE (io->cmd); io->cmd =NULL ; }spin_unlock (&ubq->cancel_lock);
@@ -3162,6 +3163,12 @@ ublk_fill_io_cmd(struct ublk_io *io, struct io_uring_cmd *cmd)return req; } +/* +* Call before ublk_fill_io_cmd() publishes @cmd in io->cmd: a control-path +* cancel may complete any command found there, and io_uring_cmd_done() only +* takes it off the cancelable list if it is marked already. The handlers +* hold uring_lock, so marking takes no lock. +*/ static inline void ublk_prep_cancel (struct io_uring_cmd *cmd,unsigned int issue_flags,struct ublk_queue *ubq,u16 tag)
@@ -3175,6 +3182,8 @@ static inline void ublk_prep_cancel(struct io_uring_cmd *cmd, pdu->ubq = ubq; pdu->tag = tag;io_uring_cmd_mark_cancelable (cmd, issue_flags); +/* pairs with the cancel loading cmd from io->cmd, then cmd->flags */ +smp_wmb (); }static void ublk_io_release (void *priv)
@@ -3422,11 +3431,11 @@ static int ublk_ch_uring_cmd_local(struct io_uring_cmd *cmd, ret =ublk_check_fetch_buf (ub, addr);if (ret) goto out; +/* before ublk_fetch() publishes io->cmd, see ublk_prep_cancel() */ +ublk_prep_cancel (cmd, issue_flags, ubq, tag); ret =ublk_fetch (cmd, ub, io, addr, q_id);if (ret) - goto out; - -ublk_prep_cancel (cmd, issue_flags, ubq, tag); + goto out_done;return -EIOCBQUEUED ; }
@@ -3470,6 +3479,7 @@ static int ublk_ch_uring_cmd_local(struct io_uring_cmd *cmd,if (ret) goto out; io->res = result; +ublk_prep_cancel (cmd, issue_flags, ubq, tag); req =ublk_fill_io_cmd (io, cmd);ublk_apply_io_buf (ub, io, cmd, addr, &auto_buf, &buf_idx);if (buf_idx !=UBLK_INVALID_BUF_IDX )
@@ -3488,19 +3498,24 @@ static int ublk_ch_uring_cmd_local(struct io_uring_cmd *cmd,* uring_cmd active first and prepare for handling new requeued * request */ +ublk_prep_cancel (cmd, issue_flags, ubq, tag); req =ublk_fill_io_cmd (io, cmd); io->buf.addr = addr;if (likely (ublk_get_data (ubq, io, req))) {__ublk_prep_compl_io_cmd (io, req); -return UBLK_IO_RES_OK ; + ret =UBLK_IO_RES_OK ; + goto out_done; }break ;default : goto out; } -ublk_prep_cancel (cmd, issue_flags, ubq, tag);return -EIOCBQUEUED ; + out_done: +/* marked cancelable: complete through io_uring_cmd_done() */ +io_uring_cmd_done (cmd, ret, issue_flags); +return -EIOCBQUEUED ; out:pr_devel ("%s: complete: cmd op %d, tag %d ret %x io_flags %x\n" , __func__, cmd_op, tag, ret, io ? io->flags :0 );
Signature · 1 line
2.55.0
Ming Lei
maintainerPATCH v3 3/12 ublk: mark the batch fetch command cancelable before linking it
In reply to Ming LeiUBLK_U_IO_FETCH_IO_CMDS has the same order problem as the per-io
commands: ublk_batch_attach() links the fetch command into fcmd_head and marks it cancelable only after dropping evts_lock. A cancel from the control path (ublk_batch_cancel_queue()) can take it in between and complete it, and the later mark puts a completed request on io_uring's cancelable list.
Mark it before linking it. evts_lock orders the mark before the link, which is where the cancel finds it.
Batch commands are not bounced to task work, so they can run from io-wq without uring_lock, and io_uring's cancel walk can now find a fetch command before it is linked. Two things make that safe:
- initialize fcmd->node: it came from kzalloc(), so list_empty() saw a linked node and ublk_batch_cancel_cmd() would list_del_init() NULL pointers
- on the -ENODEV path, complete the command with io_uring_cmd_done() before freeing fcmd: that removes it from the cancelable list under uring_lock, after which nothing can see fcmd
Also use data->cmd instead of fcmd->cmd after dropping evts_lock: once fcmd is linked and not the active one, a control-path cancel may complete and free it.
--- drivers/block/ublk_drv.c | 24 ++++++++++++++++++------ 1 file changed, 18 insertions(+), 6 deletions(-)
drivers/block/ublk_drv.c +18 -6
@@ -815,6 +815,8 @@ ublk_batch_alloc_fcmd(struct io_uring_cmd *cmd)if (fcmd) { fcmd->cmd = cmd; fcmd->buf_group =READ_ONCE (cmd->sqe->buf_index); +/* a cancel may look at it before it is linked */ +INIT_LIST_HEAD (&fcmd->node); }return fcmd; }
@@ -3914,6 +3916,15 @@ static int ublk_batch_attach(struct ublk_queue *ubq,bool free = false;struct ublk_uring_cmd_pdu *pdu =ublk_get_uring_cmd_pdu (data->cmd); +/* +* Mark it cancelable before linking it into fcmd_head, where a cancel +* from the control path can take and complete it: see +* ublk_prep_cancel(). evts_lock orders the mark before the link. +*/ + pdu->ubq = ubq; + pdu->fcmd = fcmd; +io_uring_cmd_mark_cancelable (fcmd->cmd, data->issue_flags); +spin_lock (&ubq->evts_lock);if (unlikely (ubq->force_abort || ubq->canceling)) { free = true;
@@ -3924,14 +3935,12 @@ static int ublk_batch_attach(struct ublk_queue *ubq,spin_unlock (&ubq->evts_lock);if (unlikely (free)) { +/* off the cancelable list first, then nothing can see fcmd */ +io_uring_cmd_done (data->cmd, -ENODEV , data->issue_flags);ublk_batch_free_fcmd (fcmd); -return -ENODEV ; +return -EIOCBQUEUED ; } - pdu->ubq = ubq; - pdu->fcmd = fcmd; -io_uring_cmd_mark_cancelable (fcmd->cmd, data->issue_flags); -if (!new_fcmd) goto out;
@@ -3939,9 +3948,12 @@ static int ublk_batch_attach(struct ublk_queue *ubq,* If the two fetch commands are originated from same io_ring_ctx, * run batch dispatch directly. Otherwise, schedule task work for * doing it. +* +* Use data->cmd, not fcmd->cmd: once fcmd is linked and not active, +* a cancel from the control path may complete and free it. */ if (io_uring_cmd_ctx_handle (new_fcmd->cmd) == -io_uring_cmd_ctx_handle (fcmd->cmd)) { +io_uring_cmd_ctx_handle (data->cmd)) { data->cmd = new_fcmd->cmd;ublk_batch_dispatch (ubq, data, new_fcmd); }else {
Signature · 1 line
2.55.0
Ming Lei
maintainerPATCH v3 4/12 ublk: reset the FETCH round in release also without a disk
In reply to Ming LeiThe release of /dev/ublkcN resets the queues only when the device has a disk. Without one it skips ublk_reset_ch_dev():
server fetches, never starts; old server exits
or STOP_DEV removed the disk release: no disk → skip reset
ios stay ACTIVE / CANCELED, nr_io_ready stays full
new server opens /dev/ublkcN FETCH: -EBUSY (still ready), or
-EINVAL after a partial round → the device can only be deleted
All uring_cmds are done when the release work runs, with or without a disk, so the reset is safe either way. Do it in both cases. Then a new server can fetch and start the device again, as the STOP_DEV entry in Documentation/block/ublk.rst says ("ublk device is ready for the new process").
--- drivers/block/ublk_drv.c | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-)
drivers/block/ublk_drv.c +5 -5
@@ -2551,12 +2551,13 @@ static void ublk_ch_release_work_fn(struct work_struct *work) }/* -* disk isn't attached yet, either device isn't live, or it has -* been removed already, so we needn't to do anything +* No disk: the device isn't live, or it has been removed already. +* There are no requests to abort, but the round still has to be +* reset, so that a new server can fetch and start the device. */ disk =ublk_get_disk (ub);if (!disk) - goto out; + goto reset;/* * All uring_cmd are done now, so abort any request outstanding to
@@ -2617,10 +2618,9 @@ static void ublk_ch_release_work_fn(struct work_struct *work) unlock:mutex_unlock (&ub->mutex);ublk_put_disk (disk); - +reset:/* all uring_cmd has been done now, reset device & ubq */ ublk_reset_ch_dev (ub); -out:clear_bit (UB_STATE_OPEN , &ub->state);/* put the reference grabbed in ublk_ch_release() */
Signature · 1 line
2.55.0
Ming Lei
maintainerPATCH v3 5/12 ublk: reset the FETCH round under ub->mutex
In reply to Ming LeiThe release work resets the FETCH round without ub->mutex, while START_DEV checks it under ub->mutex:
START_DEV (ub->mutex) release work
all queues ready? yes
ublksrv_tgid matches? yes
ublk_reset_ch_dev():
io->cmd = NULL, nr_queue_ready = 0 go live: a request then reaches ublk_queue_cmd() with a NULL io->cmd
Do the reset under ub->mutex. Then START_DEV sees the round either ready or reset. The work already takes ub->mutex when there is a disk, after it has aborted the requests; nothing holding ub->mutex waits for it.
--- drivers/block/ublk_drv.c | 16 ++++++++++------ 1 file changed, 10 insertions(+), 6 deletions(-)
drivers/block/ublk_drv.c +10 -6
@@ -2556,8 +2556,10 @@ static void ublk_ch_release_work_fn(struct work_struct *work)* reset, so that a new server can fetch and start the device. */ disk =ublk_get_disk (ub); -if (!disk) +if (!disk) { +mutex_lock (&ub->mutex); goto reset; + }/* * All uring_cmd are done now, so abort any request outstanding to
@@ -2589,7 +2591,7 @@ static void ublk_ch_release_work_fn(struct work_struct *work)/* double check after grabbing lock */ if (!ub->ub_disk) - goto unlock; + goto reset;/* * Transition the device to the nosrv state. What exactly this
@@ -2615,12 +2617,14 @@ static void ublk_ch_release_work_fn(struct work_struct *work)WRITE_ONCE (ublk_get_queue (ub, i)->fail_io, true); } } -unlock: -mutex_unlock (&ub->mutex); -ublk_put_disk (disk); reset: -/* all uring_cmd has been done now, reset device & ubq */ +/* +* All uring_cmd has been done now, reset device & ubq. Under +* ub->mutex, so START_DEV sees the round either ready or reset. +*/ ublk_reset_ch_dev (ub); +mutex_unlock (&ub->mutex); +ublk_put_disk (disk);clear_bit (UB_STATE_OPEN , &ub->state);/* put the reference grabbed in ublk_ch_release() */
Signature · 1 line
2.55.0
Ming Lei
maintainerPATCH v3 6/12 ublk: allow a NULL io->cmd in the cancel callback
In reply to Ming LeiA cancel from the control path, like STOP_DEV's ublk_cancel_dev() after dropping ub->mutex, takes the command and clears io->cmd under cancel_lock, then completes it with io_uring_cmd_done(), which takes uring_lock to remove it from the cancelable list. The server's io_uring cancel runs ublk_uring_cmd_cancel_fn() under uring_lock in between:
control path cancel io_uring cancel (uring_lock)
io->cmd = NULL, CANCELED
io_uring_cmd_done()
waits for uring_lock cmd still on the list
ublk_uring_cmd_cancel_fn(cmd)
io->cmd == NULL: WARN
ublk_cancel_cmd() then sees CANCELED and leaves the command alone, so nothing is wrong; only warn if io->cmd holds another command.
--- drivers/block/ublk_drv.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-)
drivers/block/ublk_drv.c +4 -1
@@ -2896,6 +2896,7 @@ static void ublk_uring_cmd_cancel_fn(struct io_uring_cmd *cmd, {struct ublk_uring_cmd_pdu *pdu =ublk_get_uring_cmd_pdu (cmd);struct ublk_queue *ubq = pdu->ubq; +struct io_uring_cmd *cur;struct task_struct *task;struct ublk_io *io;
@@ -2912,7 +2913,9 @@ static void ublk_uring_cmd_cancel_fn(struct io_uring_cmd *cmd,ublk_start_cancel (ubq->dev); -WARN_ON_ONCE (io->cmd != cmd); +/* NULL if a cancel from the control path took it meanwhile */ + cur =READ_ONCE (io->cmd); +WARN_ON_ONCE (cur && cur != cmd);ublk_cancel_cmd (ubq, pdu->tag, issue_flags); }
Signature · 1 line
2.55.0
Ming Lei
maintainerPATCH v3 7/12 ublk: let STOP_DEV cancel the server's commands before its release
In reply to Ming LeiSTOP_DEV cancels the fetched commands after dropping ub->mutex, so START_DEV and FETCH can run in between:
STOP_DEV START_DEV / FETCH
lock ub->mutex
ublk_stop_dev_unlocked() (not started: does nothing)
unlock ub->mutex
START_DEV goes live, schedules scan FETCH publishes a new command
cancel_work_sync(scan) cancels the new disk's scan
ublk_cancel_dev() takes the new command: NULL io->cmd
on a queue which isn't canceling
A START_DEV after STOP_DEV also goes live over the canceled commands.
Hold a reference on the server's /dev/ublkcN while canceling:
STOP_DEV
lock ub->mutex
ublk_stop_dev_unlocked()
cancel_work_sync(scan)
server attached: take a reference on its file, set STOPPING
unlock ub->mutex
ublk_cancel_dev()
drop the reference
- STOPPING makes FETCH and PREP get UBLK_IO_RES_ABORT and START_DEV -EBUSY; a START_DEV or END_USER_RECOVERY waiting for the device to get ready wakes up too; the reset in the server's release clears it
- the release can't run before the reference is dropped, so no new server can attach, and the cancel only meets this server's commands
- the cancel still runs after the unlock: io_uring_cmd_done() may take uring_lock, under which FETCH takes ub->mutex
ublk_ch_open() saves the file in ->ch_file and ublk_ch_release() clears it, under cancel_mutex. The release runs after the last reference is gone, so take ours with file_ref_get(). Drop it with __fput_sync(): if it is the last one, fput() would defer the release to this task, which DEL_DEV then makes wait for the device to be freed.
A server which opened the device but hasn't fetched is stopped too. Document what a server sees after STOP_DEV in ublk.rst.
--- Documentation/block/ublk.rst | 7 ++++ drivers/block/ublk_drv.c | 80 ++++++++++++++++++++++++++++++++---- 2 files changed, 79 insertions(+), 8 deletions(-)
Documentation/block/ublk.rst +7 -0
@@ -126,6 +126,13 @@ managing and controlling ublk devices with help of several control commands: ublk server will release resources (such as destroying I/O handler threads & io_uring). + If a server has ``/dev/ublkc*`` open, its fetched I/O commands complete with + ``UBLK_IO_RES_ABORT``. Until it closes ``/dev/ublkc*``, new + ``UBLK_IO_FETCH_REQ`` and ``UBLK_U_IO_PREP_IO_CMDS`` commands complete with + ``UBLK_IO_RES_ABORT`` too, and ``UBLK_CMD_START_DEV`` fails with ``-EBUSY``, + even if the server hadn't fetched anything. The device can be started again + by a server that opens ``/dev/ublkc*`` after that. + - ``UBLK_CMD_DEL_DEV`` Remove ``/dev/ublkc*``. When this command returns, the allocated ublk device
drivers/block/ublk_drv.c +72 -8
149 lines not on this page · raw message
Signature · 1 line
2.55.0
Ming Lei
maintainerPATCH v3 8/12 ublk: refuse to go live after an io command was canceled
In reply to Ming LeiFrom: Josef Bacik <[email protected]>
Since commit "ublk: keep a canceled FETCH round canceling until the server is gone", a device whose FETCH round saw a cancel keeps its queues canceling until the server goes away, but START_DEV and END_USER_RECOVERY still bring it up. Without UBLK_F_USER_RECOVERY, or with UBLK_F_USER_RECOVERY_FAIL_IO, every request of the new disk fails. With UBLK_F_USER_RECOVERY, requests are requeued and never kicked: after START_DEV the partition scan hangs under disk->open_mutex, and after END_USER_RECOVERY every read parks while the command returned 0. The server cannot fetch the canceled commands again, so the device can't serve I/O until it restarts anyway.
Return -EBUSY from START_DEV and END_USER_RECOVERY while ub->canceling is set. In ublk_ctrl_start_dev() check it and publish ub->ub_disk in one cancel_mutex section, and have ublk_start_cancel() read the disk in its cancel_mutex section. Today ublk_start_cancel() samples the disk before taking the mutex, so a server dying during its own START_DEV can mark the queues without quiescing a disk START_DEV published in between, with its first I/O past the canceling check. Now either START_DEV sees the cancel, or the cancel sees the disk and quiesces it before marking. The END_USER_RECOVERY check is best effort: the disk exists there, and a cancel after it is the ordinary death of the new server, which ublk_start_cancel() handles by quiescing and marking.
--- Documentation/block/ublk.rst | 10 ++++++++-- drivers/block/ublk_drv.c | 38 ++++++++++++++++++++++++++++++++++-- 2 files changed, 44 insertions(+), 4 deletions(-)
Documentation/block/ublk.rst +8 -2
@@ -118,7 +118,11 @@ managing and controlling ublk devices with help of several control commands: After the server prepares userspace resources (such as creating I/O handler threads & io_uring for handling ublk IO), this command is sent to the driver for allocating & exposing ``/dev/ublkb*``. Parameters set via - ``UBLK_CMD_SET_PARAMS`` are applied for creating the device. + ``UBLK_CMD_SET_PARAMS`` are applied for creating the device. The command + fails with ``-EBUSY`` if an I/O command fetched by the current server + was canceled, because its io_uring is gone. The server can't fetch it + again, and the device can be started again once the server has closed + ``/dev/ublkc*``. - ``UBLK_CMD_STOP_DEV``
@@ -202,7 +206,9 @@ managing and controlling ublk devices with help of several control commands: command is accepted after ublk device is quiesced and a new process has opened ``/dev/ublkc*`` and get all ublk queues be ready. When this command returns, ublk device is unquiesced and new I/O requests are passed to the - new process. + new process. It fails with ``-EBUSY`` if an I/O command of the new + process was canceled already. The recovery can be started over once the + new process has closed ``/dev/ublkc*``. - user recovery feature description
drivers/block/ublk_drv.c +36 -2
@@ -2759,9 +2759,11 @@ static void ublk_abort_queue(struct ublk_device *ub, struct ublk_queue *ubq)static void ublk_start_cancel (struct ublk_device *ub) { -struct gendisk *disk =ublk_get_disk (ub); +struct gendisk *disk; +/* sync with ublk_ctrl_start_dev() publishing the disk */ mutex_lock (&ub->cancel_mutex); + disk =ublk_get_disk (ub);if (ub->canceling) goto out;
@@ -4575,6 +4577,7 @@ static int ublk_ctrl_start_dev(struct ublk_device *ub, .dma_alignment = 3, };struct gendisk *disk; +bool canceled;int ret = -EINVAL ;if (ublksrv_pid <=0 )
@@ -4665,8 +4668,24 @@ static int ublk_ctrl_start_dev(struct ublk_device *ub, disk->fops = &ub_fops; disk->private_data = ub; +/* +* A command of this FETCH round was canceled and can't be fetched +* again, don't bring up a disk over it. Check and publish the disk +* in one cancel_mutex section: either this sees ub->canceling, or +* ublk_start_cancel() sees the disk and quiesces it before marking +* the queues. +*/ +mutex_lock (&ub->cancel_mutex); + canceled = ub->canceling; +if (!canceled) + ub->ub_disk = disk; +mutex_unlock (&ub->cancel_mutex); +if (canceled) { +put_disk (disk); + ret = -EBUSY ; + goto out_unlock; + } ub->dev_info.ublksrv_pid = ub->ublksrv_tgid; - ub->ub_disk = disk;ublk_apply_params (ub);
@@ -5243,6 +5262,7 @@ static int ublk_ctrl_end_recovery(struct ublk_device *ub, const struct ublksrv_ctrl_cmd *header) {int ublksrv_pid = (int )header->data[0 ]; +bool canceled;int ret = -EINVAL ; pr_devel("%s: Waiting for all FETCH_REQs, dev id %d...\n", __func__,
@@ -5266,6 +5286,20 @@ static int ublk_ctrl_end_recovery(struct ublk_device *ub, ret = -EBUSY ; goto out_unlock; } + +/* +* As in ublk_ctrl_start_dev(), a canceled command can't be fetched +* again. Best effort: the disk exists here, and a cancel after this +* check is the ordinary death of the new server, which +* ublk_start_cancel() handles by quiescing and marking. +*/ +mutex_lock (&ub->cancel_mutex); + canceled = ub->canceling; +mutex_unlock (&ub->cancel_mutex); +if (canceled) { + ret = -EBUSY ; + goto out_unlock; + } ub->dev_info.ublksrv_pid = ub->ublksrv_tgid; ub->dev_info.state =UBLK_S_DEV_LIVE ; pr_devel("%s: new ublksrv_pid %d, dev id %d\n",
Signature · 1 line
2.55.0
Ming Lei
maintainerPATCH v3 9/12 selftests: ublk: move the control command helpers into ctrl.c
In reply to Ming LeiMove the ublk_ctrl_*() helpers and ublk_setup_ring() out of kublk.c, so a small test program can send control commands without copying them. kublk still links every *.c file, ctrl.c included.
ublk_ctrl_deinit() now also exits the control ring, and ublk_ctrl_init() checks the allocation and closes /dev/ublk-control when the ring setup fails. kublk calls them only around its own lifetime, but a test which opens a control handle per thread must not leak.
No functional change for kublk.
--- tools/testing/selftests/ublk/ctrl.c | 268 ++++++++++++++++++++++++++ tools/testing/selftests/ublk/kublk.c | 271 --------------------------- tools/testing/selftests/ublk/kublk.h | 32 ++++ 3 files changed, 300 insertions(+), 271 deletions(-) create mode 100644 tools/testing/selftests/ublk/ctrl.c
tools/testing/selftests/ublk/ctrl.c new file +268 -0
@@ -0,0 +1,268 @@ +// SPDX-License-Identifier: GPL-2.0 + +/* ublk control commands, shared by kublk and ublk_cancel_ready */ + +#include "kublk.h" + +static void ublk_ctrl_init_cmd (struct ublk_dev *dev, +struct io_uring_sqe *sqe, +struct ublk_ctrl_cmd_data *data) +{ +struct ublksrv_ctrl_dev_info *info = &dev->dev_info; +struct ublksrv_ctrl_cmd *cmd = (struct ublksrv_ctrl_cmd *)ublk_get_sqe_cmd (sqe); + + sqe->fd = dev->ctrl_fd; + sqe->opcode =IORING_OP_URING_CMD ; + sqe->ioprio =0 ; + +if (data->flags &CTRL_CMD_HAS_BUF ) { + cmd->addr = data->addr; + cmd->len = data->len; + } + +if (data->flags &CTRL_CMD_HAS_DATA ) + cmd->data[0 ] = data->data[0 ]; + + cmd->dev_id = info->dev_id; + cmd->queue_id =-1 ; + +ublk_set_sqe_cmd_op (sqe, data->cmd_op); + +io_uring_sqe_set_data (sqe, cmd); +} + +int __ublk_ctrl_cmd (struct ublk_dev *dev, +struct ublk_ctrl_cmd_data *data) +{ +struct io_uring_sqe *sqe; +struct io_uring_cqe *cqe; +int ret = -EINVAL ; + + sqe =io_uring_get_sqe (&dev->ring); +if (!sqe) { +ublk_err ("%s: can't get sqe ret %d\n" , __func__, ret); +return ret; + } + +ublk_ctrl_init_cmd (dev, sqe, data); + + ret =io_uring_submit (&dev->ring); +if (ret <0 ) { +ublk_err ("uring submit ret %d\n" , ret); +return ret; + } + + ret =io_uring_wait_cqe (&dev->ring, &cqe); +if (ret <0 ) { +ublk_err ("wait cqe: %s\n" ,strerror (-ret)); +return ret; + } +io_uring_cqe_seen (&dev->ring, cqe); + +return cqe->res; +} + +void ublk_ctrl_deinit (struct ublk_dev *dev) +{ +io_uring_queue_exit (&dev->ring); +close (dev->ctrl_fd); +free (dev); +} + +struct ublk_dev *ublk_ctrl_init (void ) +{ +struct ublk_dev *dev = (struct ublk_dev *)calloc (1 ,sizeof (*dev)); +struct ublksrv_ctrl_dev_info *info; +int ret; + +if (!dev) +return NULL ; + info = &dev->dev_info; + dev->ctrl_fd =open (CTRL_DEV ,O_RDWR ); +if (dev->ctrl_fd <0 ) { +free (dev); +return NULL ; + } + + info->max_io_buf_bytes =UBLK_IO_MAX_BYTES ; + + ret =ublk_setup_ring (&dev->ring,UBLK_CTRL_RING_DEPTH , +UBLK_CTRL_RING_DEPTH ,IORING_SETUP_SQE128 ); +if (ret <0 ) { +ublk_err ("queue_init: %s\n" ,strerror (-ret)); +close (dev->ctrl_fd); +free (dev); +return NULL ; + } + dev->nr_fds =1 ; + +return dev; +} + +int ublk_ctrl_stop_dev (struct ublk_dev *dev) +{ +struct ublk_ctrl_cmd_data data = { + .cmd_op =UBLK_U_CMD_STOP_DEV , + }; + +return __ublk_ctrl_cmd (dev, &data); +} + +int ublk_ctrl_try_stop_dev (struct ublk_dev *dev) +{ +struct ublk_ctrl_cmd_data data = { + .cmd_op =UBLK_U_CMD_TRY_STOP_DEV , + }; + +return __ublk_ctrl_cmd (dev, &data); +} + +int ublk_ctrl_start_dev (struct ublk_dev *dev, +int daemon_pid) +{ +struct ublk_ctrl_cmd_data data = { + .cmd_op =UBLK_U_CMD_START_DEV , + .flags =CTRL_CMD_HAS_DATA , + }; + + dev->dev_info.ublksrv_pid = data.data[0 ] = daemon_pid; + +return __ublk_ctrl_cmd (dev, &data); +} + +int ublk_ctrl_start_user_recovery (struct ublk_dev *dev) +{ +struct ublk_ctrl_cmd_data data = { + .cmd_op =UBLK_U_CMD_START_USER_RECOVERY , + }; + +return __ublk_ctrl_cmd (dev, &data); +} + +int ublk_ctrl_end_user_recovery (struct ublk_dev *dev,int daemon_pid) +{ +struct ublk_ctrl_cmd_data data = { + .cmd_op =UBLK_U_CMD_END_USER_RECOVERY , + .flags =CTRL_CMD_HAS_DATA , + }; + + dev->dev_info.ublksrv_pid = data.data[0 ] = daemon_pid; + +return __ublk_ctrl_cmd (dev, &data); +} + +int ublk_ctrl_add_dev (struct ublk_dev *dev) +{ +struct ublk_ctrl_cmd_data data = { + .cmd_op =UBLK_U_CMD_ADD_DEV , + .flags =CTRL_CMD_HAS_BUF , + .addr = (__u64) (uintptr_t) &dev->dev_info, + .len =sizeof (struct ublksrv_ctrl_dev_info ), + }; + +return __ublk_ctrl_cmd (dev, &data); +} + +int ublk_ctrl_del_dev (struct ublk_dev *dev) +{ +struct ublk_ctrl_cmd_data data = { + .cmd_op =UBLK_U_CMD_DEL_DEV , + .flags =0 , + }; + +return __ublk_ctrl_cmd (dev, &data); +} + +int ublk_ctrl_get_info (struct ublk_dev *dev) +{ +struct ublk_ctrl_cmd_data data = { + .cmd_op =UBLK_U_CMD_GET_DEV_INFO , + .flags =CTRL_CMD_HAS_BUF , + .addr = (__u64) (uintptr_t) &dev->dev_info, + .len =sizeof (struct ublksrv_ctrl_dev_info ), + }; + +return __ublk_ctrl_cmd (dev, &data); +} + +int ublk_ctrl_set_params (struct ublk_dev *dev, +struct ublk_params *params) +{ +struct ublk_ctrl_cmd_data data = { + .cmd_op =UBLK_U_CMD_SET_PARAMS , + .flags =CTRL_CMD_HAS_BUF , + .addr = (__u64 ) (uintptr_t ) params, + .len =sizeof (*params), + }; + params->len =sizeof (*params); +return __ublk_ctrl_cmd (dev, &data); +} + +int ublk_ctrl_get_params (struct ublk_dev *dev, +struct ublk_params *params) +{ +struct ublk_ctrl_cmd_data data = { + .cmd_op =UBLK_U_CMD_GET_PARAMS , + .flags =CTRL_CMD_HAS_BUF , + .addr = (__u64 )params, + .len =sizeof (*params), + }; + + params->len =sizeof (*params); + +return __ublk_ctrl_cmd (dev, &data); +} + +int ublk_ctrl_get_features (struct ublk_dev *dev, +__u64 *features) +{ +struct ublk_ctrl_cmd_data data = { + .cmd_op =UBLK_U_CMD_GET_FEATURES , + .flags =CTRL_CMD_HAS_BUF , + .addr = (__u64 ) (uintptr_t ) features, + .len =sizeof (*features), + }; + +return __ublk_ctrl_cmd (dev, &data); +} + +int ublk_ctrl_update_size (struct ublk_dev *dev, +__u64 nr_sects) +{ +struct ublk_ctrl_cmd_data data = { + .cmd_op =UBLK_U_CMD_UPDATE_SIZE , + .flags =CTRL_CMD_HAS_DATA , + }; + + data.data[0 ] = nr_sects; +return __ublk_ctrl_cmd (dev, &data); +} + +int ublk_ctrl_quiesce_dev (struct ublk_dev *dev,unsigned int timeout_ms) +{ +struct ublk_ctrl_cmd_data data = { + .cmd_op =UBLK_U_CMD_QUIESCE_DEV , + .flags =CTRL_CMD_HAS_DATA , + }; + + data.data[0 ] = timeout_ms; +return __ublk_ctrl_cmd (dev, &data); +} + +int ublk_ctrl_reg_buf (struct ublk_dev *dev,void *addr,size_t size, +__u32 flags) +{ +struct ublk_shmem_buf_reg buf_reg = { + .addr = (unsigned long )addr, + .len = size, + .flags = flags, + }; +struct ublk_ctrl_cmd_data data = { + .cmd_op =UBLK_U_CMD_REG_BUF , + .flags =CTRL_CMD_HAS_BUF , + .addr = (unsigned long )&buf_reg, + .len =sizeof (buf_reg), + }; + +return __ublk_ctrl_cmd (dev, &data); +}
tools/testing/selftests/ublk/kublk.c +0 -271
295 lines not on this page · raw message
tools/testing/selftests/ublk/kublk.h +32 -0
@@ -294,6 +294,38 @@ struct ublk_dev {extern int ublk_queue_io_cmd (struct ublk_thread *t,struct ublk_io *io); +static inline int ublk_setup_ring (struct io_uring *r,int depth, +int cq_depth,unsigned int flags) +{ +struct io_uring_params p; + +memset (&p,0 ,sizeof (p)); + p.flags = flags |IORING_SETUP_CQSIZE ; + p.cq_entries = cq_depth; + +return io_uring_queue_init_params (depth, r, &p); +} + +/* ctrl.c: control commands */ +struct ublk_dev *ublk_ctrl_init (void ); +void ublk_ctrl_deinit (struct ublk_dev *dev); +int __ublk_ctrl_cmd (struct ublk_dev *dev,struct ublk_ctrl_cmd_data *data); +int ublk_ctrl_add_dev (struct ublk_dev *dev); +int ublk_ctrl_del_dev (struct ublk_dev *dev); +int ublk_ctrl_get_info (struct ublk_dev *dev); +int ublk_ctrl_set_params (struct ublk_dev *dev,struct ublk_params *params); +int ublk_ctrl_get_params (struct ublk_dev *dev,struct ublk_params *params); +int ublk_ctrl_get_features (struct ublk_dev *dev,__u64 *features); +int ublk_ctrl_start_dev (struct ublk_dev *dev,int daemon_pid); +int ublk_ctrl_stop_dev (struct ublk_dev *dev); +int ublk_ctrl_try_stop_dev (struct ublk_dev *dev); +int ublk_ctrl_start_user_recovery (struct ublk_dev *dev); +int ublk_ctrl_end_user_recovery (struct ublk_dev *dev,int daemon_pid); +int ublk_ctrl_update_size (struct ublk_dev *dev,__u64 nr_sects); +int ublk_ctrl_quiesce_dev (struct ublk_dev *dev,unsigned int timeout_ms); +int ublk_ctrl_reg_buf (struct ublk_dev *dev,void *addr,size_t size, +__u32 flags); +static inline int __ublk_use_batch_io (__u64 flags) {return flags &UBLK_F_BATCH_IO ;
Signature · 1 line
2.55.0
Ming Lei
maintainerPATCH v3 10/12 selftests: ublk: stop a batch thread when PREP_IO_CMDS fails
In reply to Ming LeiSince STOP_DEV turns the server away until it closes /dev/ublkcN, UBLK_U_IO_PREP_IO_CMDS can complete with UBLK_IO_RES_ABORT when it races STOP_DEV. kublk asserts that PREP succeeds, so the daemon aborts before it reports to kublk add, which then waits for the start event forever.
Stop the thread like a failed FETCH_IO_CMDS does. The daemon's START_DEV then fails, and kublk add gets the error.
--- tools/testing/selftests/ublk/batch.c | 13 ++++++++++--- 1 file changed, 10 insertions(+), 3 deletions(-)
tools/testing/selftests/ublk/batch.c +10 -3
@@ -410,9 +410,16 @@ static void ublk_batch_compl_commit_cmd(struct ublk_thread *t, {unsigned short buf_idx =user_data_to_tag (cqe->user_data); -if (op ==_IOC_NR (UBLK_U_IO_PREP_IO_CMDS )) -ublk_assert (cqe->res ==0 ); -else if (op ==_IOC_NR (UBLK_U_IO_COMMIT_IO_CMDS )) { +if (op ==_IOC_NR (UBLK_U_IO_PREP_IO_CMDS )) { +/* e.g. UBLK_IO_RES_ABORT after STOP_DEV: stop like a FETCH */ +if (cqe->res <0 ) { +ublk_dbg (UBLK_DBG_IO_CMD ,"%s: prep failed %d\n" , + __func__, cqe->res); + t->state |=UBLKS_T_STOPPING ; + }else { +ublk_assert (cqe->res ==0 ); + } + }else if (op ==_IOC_NR (UBLK_U_IO_COMMIT_IO_CMDS )) {int nr_elem =user_data_to_tgt_data (cqe->user_data);ublk_assert (cqe->res == t->commit_buf_elem_size * nr_elem);
Signature · 1 line
2.55.0
Ming Lei
maintainerPATCH v3 11/12 selftests: ublk: add test for going live over canceled io commands
In reply to Ming LeiAdd ublk_cancel_ready, a small liburing program which sends control commands with kublk's helpers from ctrl.c, and test_generic_18.sh, which runs its modes:
mode what it does expected
stop_start STOP_DEV before START_DEV START_DEV -EBUSY
partial_fetch task A fetches tags 0-2 and START_DEV -EBUSY
dies, then task B fetches tag 3
recovery in user recovery, queue 0's END_USER_RECOVERY
task dies before queue 1 is -EBUSY
ready
stop_restart STOP_DEV on a new device, then START_DEV 0,
a server starts it reads succeed
stop_attached STOP_DEV after open, before FETCH ABORT and
FETCH, then a new server START_DEV -EBUSY,
then a new one works
stop_live_restart STOP_DEV on a live device, then FETCH and START_DEV
a new server starts it 0, reads succeed
race_start STOP_DEV and START_DEV at the START_DEV 0 and
same time, 50 times reads complete,
or -EBUSY
race_fetch STOP_DEV while a server opens START_DEV 0 and
and fetches, then START_DEV, reads succeed,
50 times or -EBUSY
race_async_fetch IOSQE_ASYNC FETCH while STOP_DEV no crash (KASAN
cancels, then close the ring finds the bug)
race_batch_stop STOP_DEV while kublk's batch kublk add returns,
server prepares and fetches, started or not20 times
If the device goes live, the test reads it, so an unfixed kernel shows the oops in ublk_queue_cmd(). In stop_live_restart, an unfixed kernel doesn't reset the FETCH round in ublk_ch_release() once the disk is gone, so the new server's FETCH gets -EBUSY.
A dying task is a child process which fetches and then calls exec(). exec() cancels the task's uring_cmds before it returns, so the cancel is done once the child is reaped, with no sleep. kublk can't be used:
its failure injection kills the whole process, which closes /dev/ublkcN and starts a new round.
--- tools/testing/selftests/ublk/.gitignore | 1 + tools/testing/selftests/ublk/Makefile | 6 +- .../testing/selftests/ublk/test_generic_18.sh | 69 ++ .../selftests/ublk/ublk_cancel_ready.c | 957 ++++++++++++++++++ 4 files changed, 1031 insertions(+), 2 deletions(-) create mode 100755 tools/testing/selftests/ublk/test_generic_18.sh create mode 100644 tools/testing/selftests/ublk/ublk_cancel_ready.c
tools/testing/selftests/ublk/.gitignore +1 -0
@@ -3,3 +3,4 @@ /tools kublk metadata_size +ublk_cancel_ready
tools/testing/selftests/ublk/Makefile +4 -2
@@ -19,6 +19,7 @@ TEST_PROGS += test_generic_12.sh TEST_PROGS += test_generic_13.sh TEST_PROGS += test_generic_16.sh TEST_PROGS += test_generic_17.sh +TEST_PROGS += test_generic_18.sh TEST_PROGS += test_batch_01.sh TEST_PROGS += test_batch_02.sh
@@ -74,13 +75,14 @@ TEST_PROGS += test_stress_09.sh TEST_FILES := settings -TEST_GEN_PROGS_EXTENDED = kublk metadata_size -STANDALONE_UTILS := metadata_size.c +TEST_GEN_PROGS_EXTENDED = kublk metadata_size ublk_cancel_ready +STANDALONE_UTILS := metadata_size.c ublk_cancel_ready.c LOCAL_HDRS += $(wildcard *.h) include ../lib.mk $(OUTPUT)/kublk: $(filter-out $(STANDALONE_UTILS),$(wildcard *.c)) +$(OUTPUT)/ublk_cancel_ready: ublk_cancel_ready.c ctrl.c check: shellcheck -x -f gcc *.sh
tools/testing/selftests/ublk/test_generic_18.sh new file +69 -0
69 lines not on this page · raw message
tools/testing/selftests/ublk/ublk_cancel_ready.c new file +957 -0
957 lines not on this page · raw message
Signature · 1 line
2.55.0
Ming Lei
maintainerPATCH v3 12/12 ublk: use global index for reported zones
In reply to Ming LeiFrom: Li RongQing <[email protected]>
ublk_report_zones() may split a zone report into multiple requests when the number of zones exceeds the temporary buffer capacity. The index passed to disk_report_zone() must refer to the zone position in the whole report, rather than the position within the current request.
Currently, the per-request loop index is used as the callback index, causing each chunk after the first one to restart from zero. This can make userspace callbacks receive duplicate zone indices and may result in blk_revalidate_disk_zones() updating incorrect entries in its zone state array.
Use the number of zones already reported as the base index for each request so that zone indices remain continuous across all chunks.
--- drivers/block/ublk_drv.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/block/ublk_drv.c +1 -1
@@ -696,7 +696,7 @@ static int ublk_report_zones(struct gendisk *disk, sector_t sector,if (!zone->len)break ; - ret =disk_report_zone (disk, zone, i, args); + ret =disk_report_zone (disk, zone, done_zones, args);if (ret) goto out;
Signature · 1 line
2.55.0
Ming Lei
maintainerQuoted 30 lines · Ming Lei
Hi,
Josef reported three ways in which ublk dispatches a request to a canceled io command, which oopses on the NULL
io->cmdinublk_queue_cmd()[1]:1)
STOP_DEV, thenSTART_DEV
2) partial FETCH: a task fetches some tags and exits, then another task fetches the rest
3) user recovery: one queue's task exits before all queues are readyThis series fixes them in a simpler way, without extra locking in the I/O path:
- patch 1: a FETCH round which saw a cancel stays canceling until the server is gone (2, 3)
- patch 2-3: an io command is marked cancelable before it is published
- patch 4-5: the release resets the FETCH round, also without a disk and underub->mutex
- patch 6: don't warn when the cancel callback findsio->cmdtaken by a cancel from the control path
- patch 7:STOP_DEVholds a reference on/dev/ublkcNwhile it cancels, so noSTART_DEV, FETCH or new server can slip in (1)
- patch 8 (Josef):START_DEVandEND_USER_RECOVERYrefuse to go live after an io command was canceled (2, 3)
- patch 9-11: selftests
- patch 12 (Li RongQing):ublk_report_zones()passes a per-chunk index todisk_report_zone(); use the index in the whole report
Quoted 7 lines
Tested with all ublk selftests under lockdep on for-7.4/block; v1 was also tested under KASAN.
V3:
- rebase on for-7.4/block
- add Li RongQing's "ublk: use global index for reported zones" (patch 12)
Hi Jens,
Ping...
Thanks,
Quoted 10 lines · Ming Lei
Josef reported three ways in which ublk dispatches a request to a canceled io command, which oopses on the NULL
io->cmdinublk_queue_cmd()[1]:1)
STOP_DEV, thenSTART_DEV
2) partial FETCH: a task fetches some tags and exits, then another task fetches the rest
3) user recovery: one queue's task exits before all queues are ready[...]
Applied, thanks!
[01/12] ublk: keep a canceled FETCH round canceling until the server is gone
commit: 4c454b5b2634d982caf797fbe477770c3edfc983
[02/12] ublk: mark the io command cancelable before publishing it
commit: a261f0cacacc022e4fab5b8e38c59f74ce4942e8
[03/12] ublk: mark the batch fetch command cancelable before linking it
commit: b59e96011d22e322fbf4d5d1d1124de538c278ed
[04/12] ublk: reset the FETCH round in release also without a disk
commit: f06ca594eafcdf58a399a0f92480631192e06032
[05/12] ublk: reset the FETCH round under ub->mutex
commit: 84824afdb5a4dc127fbd7036142d7611c5e53651
[06/12] ublk: allow a NULL io->cmd in the cancel callback
commit: 4915dc0a1834457520ef0592bfd454b26901614f
[07/12] ublk: let STOP_DEV cancel the server's commands before its release
commit: c8b349f9daf6d88ff21033028167d00e819dafb6
[08/12] ublk: refuse to go live after an io command was canceled
commit: 7bf71ecab1c225d4f7d0717ec482638a03527be2
[09/12] selftests: ublk: move the control command helpers into ctrl.c
commit: cdf6e5de86a91da6e4d621260bc70e99e2f3a25e
[10/12] selftests: ublk: stop a batch thread when PREP_IO_CMDS fails
commit: 9703575564aa7c74d6746e4ed2d3f8a99af7590c
[11/12] selftests: ublk: add test for going live over canceled io commands
commit: bcab0b98566b8f73f8e398d4f12a088b81885459
[12/12] ublk: use global index for reported zones
commit: 80897c7e5ae5c56f37a2eb004328edcb4da7cdd1Best regards,
Signature · 1 line
Jens Axboe