Skip to content

rbd: switch to blk-mq

Hannes Reinecke ceph-devel linux-block 1 version In review

Get series with b4
b4 am [email protected]

Versions newest first

  1. v1 Latest15 patches 9 repliesIn review

Patches in v1 No applies recorded yet

#Subject and filesSizeRepliesApplied
1/15rbd: lock object request listdrivers/block/rbd.c+28 -81 reply
2/15rbd: use READ_ONCE() when checking the mapping sizedrivers/block/rbd.c+6 -62 replies
3/15rbd: reorder rbd_img_advance()drivers/block/rbd.c+17 -130 replies
4/15rbd: reorder switch statement in rbd_advance_read()drivers/block/rbd.c+19 -120 replies
5/15rbd: reorder switch statement in rbd_advance_write()drivers/block/rbd.c+25 -110 replies
6/15rbd: add 'done' state for rbd_obj_advance_copyup()drivers/block/rbd.c+16 -90 replies
7/15rbd: use callback for image request completiondrivers/block/rbd.c+33 -281 reply
8/15rbd: add debugging statements for the state machinedrivers/block/rbd.c+15 -20 replies
9/15rbd: count pending object requests in-linedrivers/block/rbd.c+30 -112 replies
10/15rbd: kill 'work_result'drivers/block/rbd.c+2 -30 replies
11/15rbd: drop state_mutex in __rbd_img_handle_request()drivers/block/rbd.c+9 -171 reply
12/15rbd: kill img_request krefdrivers/block/rbd.c+5 -190 replies
13/15rbd: schedule image_request after preparationdrivers/block/rbd.c+15 -371 reply
14/15rbd: embed image request as blk_mq request payloaddrivers/block/rbd.c+37 -190 replies
15/15rbd: switch to blk-mqdrivers/block/rbd.c+1 -11 reply

Reviews trailers per patch

No Reviewed-by, Acked-by or Tested-by trailers for v1 yet.

Read the v1 thread

Cover letter [PATCH 00/15] rbd: switch to blk-mq

Hi all,

this patchset implements multiqueue support for rbd, which gives a nice performance benefit (I measured up to 25% on my grid). To achieve this several steps are required:
1) drop the 'state_mutex' in rbd_img_advance.
To do so the state machines had to be reordered so ensure no race windows are left when invoking asynchronous methods.
2) Embed image request into the block request to save a memory allocation in the hot path
3) Enable one queue per CPU to enhance parallelism.

I also took the opportunity to clean up the state machines, by adding a 'done' step and ensuring that the step is always set correctly upon exit. This allows for better debugging as now the final states must always be set when destroying an object.

As usual, comments and reviews are welcome.

Hannes Reinecke (15):
rbd: lock object request list
rbd: use READ_ONCE() when checking the mapping size
rbd: reorder rbd_img_advance()
rbd: reorder switch statement in rbd_advance_read()
rbd: reorder switch statement in rbd_advance_write()
rbd: add 'done' state for rbd_obj_advance_copyup()
rbd: use callback for image request completion
rbd: add debugging statements for the state machine
rbd: count pending object requests in-line
rbd: kill 'work_result'
rbd: drop state_mutex in __rbd_img_handle_request()
rbd: kill img_request kref
rbd: schedule image_request after preparation
rbd: embed image request as blk_mq request payload
rbd: switch to blk-mq

 drivers/block/rbd.c | 418 ++++++++++++++++++++++++++++++----------------------
 1 file changed, 240 insertions(+), 178 deletions(-)
Signature · 1 line
2.16.4