ublk: restore auto buf unregister refcount optimization
Subsystems (2)
PATCH 1/1 ublk: restore auto buf unregister refcount optimization
Commit 1ceeedb59749 ("ublk: optimize UBLK_IO_UNREGISTER_IO_BUF on daemon task") optimized ublk request buffer unregistration to use a non-atomic reference count decrement when performed on the ublk_io's daemon task. The optimization applied to auto buffer unregistration, which happens as part of handling UBLK_IO_COMMIT_AND_FETCH_REQ on the daemon task. However, commit b749965edda8 ("ublk: remove ublk_commit_and_fetch()") reordered the ublk_sub_req_ref() for the completed request before the io_buffer_unregister_bvec() call. As a result, task_registered_buffers is already 0 when io_buffer_unregister_bvec() calls ublk_io_release() and the non-atomic refcount optimization doesn't apply.
Move the io_buffer_unregister_bvec() call back to before ublk_need_complete_req() to restore the reference counting optimization.
--- drivers/block/ublk_drv.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
drivers/block/ublk_drv.c +2 -2
@@ -3243,15 +3243,15 @@ static int ublk_ch_uring_cmd_local(struct io_uring_cmd *cmd,if (ret) goto out; io->res = result; req =ublk_fill_io_cmd (io, cmd); ret =ublk_config_io_buf (ub, io, cmd, addr, &buf_idx); +if (buf_idx !=UBLK_INVALID_BUF_IDX ) +io_buffer_unregister_bvec (cmd, buf_idx, issue_flags); compl =ublk_need_complete_req (ub, io);/* can't touch 'ublk_io' any more */ -if (buf_idx !=UBLK_INVALID_BUF_IDX ) -io_buffer_unregister_bvec (cmd, buf_idx, issue_flags);if (req_op (req) ==REQ_OP_ZONE_APPEND ) req->__sector = addr;if (compl)__ublk_complete_rq (req, io,ublk_dev_need_map_io (ub),NULL );
Signature · 1 line
2.45.2
Quoted 31 lines · Caleb Sander Mateos
Commit
1ceeedb59749("ublk: optimizeUBLK_IO_UNREGISTER_IO_BUFon daemon task") optimized ublk request buffer unregistration to use a non-atomic reference count decrement when performed on the ublk_io's daemon task. The optimization applied to auto buffer unregistration, which happens as part of handlingUBLK_IO_COMMIT_AND_FETCH_REQon the daemon task. However, commitb749965edda8("ublk: removeublk_commit_and_fetch()") reordered theublk_sub_req_ref()for the completed request before theio_buffer_unregister_bvec()call. As a result,task_registered_buffersis already 0 whenio_buffer_unregister_bvec()callsublk_io_release()and the non-atomic refcount optimization doesn't apply.
Move theio_buffer_unregister_bvec()call back to beforeublk_need_complete_req()to restore the reference counting optimization.Signed-off-by Caleb Sander Mateos <[email protected]>Fixes b749965edda8 ("ublk: remove ublk_commit_and_fetch()")--- drivers/block/ublk_drv.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)drivers/block/ublk_drv.c +1
-0@@ -3243,15 +3243,15 @@ static int ublk_ch_uring_cmd_local(struct io_uring_cmd *cmd,if (ret) goto out; io->res = result; req =ublk_fill_io_cmd (io, cmd); ret =ublk_config_io_buf (ub, io, cmd, addr, &buf_idx); +if (buf_idx !=UBLK_INVALID_BUF_IDX )
drivers/block/ublk_drv.c static int ublk_ch_uring_cmd_local(struct io_uring_cmd *cmd, · line 3256
Quoted 10 lines
drivers/block/ublk_drv.c +1
-2+io_buffer_unregister_bvec (cmd, buf_idx, issue_flags); compl =ublk_need_complete_req (ub, io);/* can't touch 'ublk_io' any more */ -if (buf_idx !=UBLK_INVALID_BUF_IDX ) -io_buffer_unregister_bvec (cmd, buf_idx, issue_flags);if (req_op (req) ==REQ_OP_ZONE_APPEND ) req->__sector = addr;if (compl)__ublk_complete_rq (req, io,ublk_dev_need_map_io (ub),NULL );
I also noticed that the "can't touch 'ublk_io' any more" comment doesn't make much sense, as __ublk_complete_rq() still accesses (and even mutates) the struct ublk_io. Am I misunderstanding the comment? It looks like this might be a race condition for UBLK_U_IO_COMMIT_IO_CMDS, as __ublk_complete_rq() is called without holding the ublk_io spinlock.
Thanks,
Caleb
Quoted 36 lines · Caleb Sander Mateos
Quoted 34 lines · Caleb Sander Mateos
Commit
1ceeedb59749("ublk: optimizeUBLK_IO_UNREGISTER_IO_BUFon daemon task") optimized ublk request buffer unregistration to use a non-atomic reference count decrement when performed on the ublk_io's daemon task. The optimization applied to auto buffer unregistration, which happens as part of handlingUBLK_IO_COMMIT_AND_FETCH_REQon the daemon task. However, commitb749965edda8("ublk: removeublk_commit_and_fetch()") reordered theublk_sub_req_ref()for the completed request before theio_buffer_unregister_bvec()call. As a result,task_registered_buffersis already 0 whenio_buffer_unregister_bvec()callsublk_io_release()and the non-atomic refcount optimization doesn't apply.
Move theio_buffer_unregister_bvec()call back to beforeublk_need_complete_req()to restore the reference counting optimization.Signed-off-by Caleb Sander Mateos <[email protected]>Fixes b749965edda8 ("ublk: remove ublk_commit_and_fetch()")--- drivers/block/ublk_drv.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)drivers/block/ublk_drv.c +2
-0@@ -3243,15 +3243,15 @@ static int ublk_ch_uring_cmd_local(struct io_uring_cmd *cmd,if (ret) goto out; io->res = result; req =ublk_fill_io_cmd (io, cmd); ret =ublk_config_io_buf (ub, io, cmd, addr, &buf_idx); +if (buf_idx !=UBLK_INVALID_BUF_IDX ) +io_buffer_unregister_bvec (cmd, buf_idx, issue_flags); compl =ublk_need_complete_req (ub, io);
Quoted 11 lines
drivers/block/ublk_drv.c static int ublk_ch_uring_cmd_local(struct io_uring_cmd *cmd, · line 3256
Quoted 7 lines
drivers/block/ublk_drv.c +0
-2/* can't touch 'ublk_io' any more */ -if (buf_idx !=UBLK_INVALID_BUF_IDX ) -io_buffer_unregister_bvec (cmd, buf_idx, issue_flags);if (req_op (req) ==REQ_OP_ZONE_APPEND ) req->__sector = addr;if (compl)__ublk_complete_rq (req, io,ublk_dev_need_map_io (ub),NULL );I also noticed that the "can't touch '
ublk_io' any more" comment doesn't make much sense, as__ublk_complete_rq()still accesses (and even mutates) the structublk_io. Am I misunderstanding the comment?
Yes, it can be removed, originally this code block may be reused forBATCH_IO, but finally it doesn't work toward this way, so can you remove
it in this patch given it is introduced in b749965edda8 ("ublk: remove ublk_commit_and_fetch()")?
Quoted 3 lines
It looks like this might be a race condition for
UBLK_U_IO_COMMIT_IO_CMDS, as__ublk_complete_rq()is called without holding theublk_iospinlock.
It is actually fine for __ublk_complete_rq() to manipulate io->res lockless:
1) UBLK_IO_FLAG_OWNED_BY_SRV is cleared & checked with io->lock, so any new UBLK_U_IO_COMMIT_IO_CMDS will be failed
2) the current IO request isn't completed yet, so new io command handling won't be started.
Thanks,
Ming
Quoted 13 lines · Caleb Sander Mateos
Commit
1ceeedb59749("ublk: optimizeUBLK_IO_UNREGISTER_IO_BUFon daemon task") optimized ublk request buffer unregistration to use a non-atomic reference count decrement when performed on the ublk_io's daemon task. The optimization applied to auto buffer unregistration, which happens as part of handlingUBLK_IO_COMMIT_AND_FETCH_REQon the daemon task. However, commitb749965edda8("ublk: removeublk_commit_and_fetch()") reordered theublk_sub_req_ref()for the completed request before theio_buffer_unregister_bvec()call. As a result,task_registered_buffersis already 0 whenio_buffer_unregister_bvec()callsublk_io_release()and the non-atomic refcount optimization doesn't apply.
Move theio_buffer_unregister_bvec()call back to beforeublk_need_complete_req()to restore the reference counting optimization.
Quoted 2 lines
Signed-off-by Caleb Sander Mateos <[email protected]>Fixes b749965edda8 ("ublk: remove ublk_commit_and_fetch()")
Thanks,
Ming
Quoted 13 lines · Caleb Sander Mateos
Commit
1ceeedb59749("ublk: optimizeUBLK_IO_UNREGISTER_IO_BUFon daemon task") optimized ublk request buffer unregistration to use a non-atomic reference count decrement when performed on the ublk_io's daemon task. The optimization applied to auto buffer unregistration, which happens as part of handlingUBLK_IO_COMMIT_AND_FETCH_REQon the daemon task. However, commitb749965edda8("ublk: removeublk_commit_and_fetch()") reordered theublk_sub_req_ref()for the completed request before theio_buffer_unregister_bvec()call. As a result,task_registered_buffersis already 0 whenio_buffer_unregister_bvec()callsublk_io_release()and the non-atomic refcount optimization doesn't apply.
Move theio_buffer_unregister_bvec()call back to beforeublk_need_complete_req()to restore the reference counting optimization.
Quoted 1 line
[...]
Applied, thanks!
[1/1] ublk: restore auto buf unregister refcount optimization
commit: ad5f2e2908c9b79a86529281a48e94d644d43dc7
Best regards,
Signature · 1 line
Jens Axboe