task_work optimization
Get series with b4
b4 am [email protected]Versions newest first · linked by author and title
- v2 Latest8 patches 1 replyIn review
- v18 patches 13 repliesSuperseded
Patches in v2 No applies recorded yet
| # | Subject and files | Size | Replies | Applied |
|---|---|---|---|---|
| 1/8 | io-wq: code clean for io_wq_add_work_after()fs/ | +2 | 0 replies | |
| 2/8 | io-wq: add helper to merge two wq_listsfs/ | +21 | 0 replies | |
| 3/8 | io_uring: add a limited tw list for irq completion workfs/ | +33 | 1 reply | |
| 4/8 | io_uring: add helper for task work execution codefs/ | +20 | 0 replies | |
| 5/8 | io_uring: split io_req_complete_post() and add a helperfs/ | +10 | 0 replies | |
| 6/8 | io_uring: move up io_put_kbuf() and io_put_rw_kbuf()fs/ | +21 | 0 replies | |
| 7/8 | io_uring: add tw_ctx for io_uring_taskfs/ | +6 | 0 replies | |
| 8/8 | io_uring: batch completion in prior_task_listfs/ | +31 | 0 replies |
Reviews trailers per patch
No Reviewed-by, Acked-by or Tested-by trailers for v2 yet.
Cover letter [PATCH v2 0/8] task_work optimization
The main patches are 3/8 and 8/8. 3/8 is to set a new task list and complete its task works prior to the normal task works in old task list.
8/8 is an optimization of batching completion of task works in the new task list if they all have same ctx which is the normal situation, the benefit is we now batch them regardless uring_lock.
Tested this patchset by manually replace __io_queue_sqe() by io_req_task_complete() to construct 'heavy' task works. Then test with fio:
ioengine=io_uring
thread=1
bs=4k
direct=1
rw=randread
time_based=1
runtime=600
randrepeat=0
group_reporting=1
filename=/dev/nvme0n1
Tried various iodepth.
The peak IOPS for this patch is 314K, while the old one is 249K. For avg latency, difference shows when iodepth grow:
depth and avg latency(usec):
depth new old 1 22.80 23.77 2 23.48 24.54 4 24.26 25.57 8 29.21 32.89 16 53.61 63.50 32 106.29 131.34 64 217.21 256.33 128 421.59 513.87
256 815.15 1050.99
without this patchset
iodepth=1
clat percentiles (usec):
| 1.00th=[ 7], 5.00th=[ 7], 10.00th=[ 8], 20.00th=[ 8], | 30.00th=[ 8], 40.00th=[ 8], 50.00th=[ 8], 60.00th=[ 8], | 70.00th=[ 8], 80.00th=[ 8], 90.00th=[ 82], 95.00th=[ 97], | 99.00th=[ 99], 99.50th=[ 99], 99.90th=[ 100], 99.95th=[ 101], | 99.99th=[ 126] iodepth=2 clat percentiles (usec): | 1.00th=[ 7], 5.00th=[ 7], 10.00th=[ 7], 20.00th=[ 8], | 30.00th=[ 8], 40.00th=[ 8], 50.00th=[ 8], 60.00th=[ 9], | 70.00th=[ 10], 80.00th=[ 10], 90.00th=[ 83], 95.00th=[ 97], | 99.00th=[ 100], 99.50th=[ 102], 99.90th=[ 126], 99.95th=[ 145], | 99.99th=[ 971] iodepth=4 clat percentiles (usec): | 1.00th=[ 7], 5.00th=[ 7], 10.00th=[ 8], 20.00th=[ 8], | 30.00th=[ 8], 40.00th=[ 9], 50.00th=[ 9], 60.00th=[ 10], | 70.00th=[ 11], 80.00th=[ 13], 90.00th=[ 86], 95.00th=[ 98], | 99.00th=[ 105], 99.50th=[ 115], 99.90th=[ 139], 99.95th=[ 149], | 99.99th=[ 169] iodepth=8 clat percentiles (usec): | 1.00th=[ 7], 5.00th=[ 8], 10.00th=[ 9], 20.00th=[ 12], | 30.00th=[ 13], 40.00th=[ 16], 50.00th=[ 18], 60.00th=[ 20], | 70.00th=[ 22], 80.00th=[ 27], 90.00th=[ 95], 95.00th=[ 105], | 99.00th=[ 121], 99.50th=[ 131], 99.90th=[ 157], 99.95th=[ 167], | 99.99th=[ 206] iodepth=16 clat percentiles (usec): | 1.00th=[ 25], 5.00th=[ 33], 10.00th=[ 37], 20.00th=[ 41], | 30.00th=[ 44], 40.00th=[ 46], 50.00th=[ 49], 60.00th=[ 51], | 70.00th=[ 55], 80.00th=[ 63], 90.00th=[ 125], 95.00th=[ 137], | 99.00th=[ 155], 99.50th=[ 165], 99.90th=[ 198], 99.95th=[ 235], | 99.99th=[ 1844] iodepth=32 clat percentiles (usec): | 1.00th=[ 92], 5.00th=[ 98], 10.00th=[ 102], 20.00th=[ 106], | 30.00th=[ 110], 40.00th=[ 112], 50.00th=[ 116], 60.00th=[ 120], | 70.00th=[ 128], 80.00th=[ 141], 90.00th=[ 192], 95.00th=[ 204], | 99.00th=[ 227], 99.50th=[ 235], 99.90th=[ 260], 99.95th=[ 273], | 99.99th=[ 322] iodepth=64 clat percentiles (usec): | 1.00th=[ 221], 5.00th=[ 227], 10.00th=[ 231], 20.00th=[ 233], | 30.00th=[ 237], 40.00th=[ 239], 50.00th=[ 241], 60.00th=[ 243], | 70.00th=[ 247], 80.00th=[ 253], 90.00th=[ 318], 95.00th=[ 330], | 99.00th=[ 351], 99.50th=[ 359], 99.90th=[ 388], 99.95th=[ 400], | 99.99th=[ 529] iodepth=128 clat percentiles (usec): | 1.00th=[ 465], 5.00th=[ 478], 10.00th=[ 482], 20.00th=[ 486], | 30.00th=[ 490], 40.00th=[ 490], 50.00th=[ 494], 60.00th=[ 498], | 70.00th=[ 506], 80.00th=[ 553], 90.00th=[ 578], 95.00th=[ 586], | 99.00th=[ 635], 99.50th=[ 652], 99.90th=[ 676], 99.95th=[ 717], | 99.99th=[ 2278] iodepth=256 clat percentiles (usec): | 1.00th=[ 979], 5.00th=[ 988], 10.00th=[ 996], 20.00th=[ 1012], | 30.00th=[ 1020], 40.00th=[ 1037], 50.00th=[ 1037], 60.00th=[ 1045], | 70.00th=[ 1057], 80.00th=[ 1090], 90.00th=[ 1123], 95.00th=[ 1139], | 99.00th=[ 1205], 99.50th=[ 1237], 99.90th=[ 1254], 99.95th=[ 1270], | 99.99th=[ 1385]
with this patchset
iodepth=1
clat percentiles (usec):
| 1.00th=[ 7], 5.00th=[ 7], 10.00th=[ 7], 20.00th=[ 7], | 30.00th=[ 7], 40.00th=[ 7], 50.00th=[ 8], 60.00th=[ 8], | 70.00th=[ 8], 80.00th=[ 8], 90.00th=[ 82], 95.00th=[ 97], | 99.00th=[ 99], 99.50th=[ 99], 99.90th=[ 100], 99.95th=[ 101], | 99.99th=[ 125] iodepth=2 clat percentiles (usec): | 1.00th=[ 6], 5.00th=[ 7], 10.00th=[ 7], 20.00th=[ 7], | 30.00th=[ 7], 40.00th=[ 7], 50.00th=[ 8], 60.00th=[ 8], | 70.00th=[ 9], 80.00th=[ 11], 90.00th=[ 83], 95.00th=[ 97], | 99.00th=[ 100], 99.50th=[ 102], 99.90th=[ 127], 99.95th=[ 141], | 99.99th=[ 668] iodepth=4 clat percentiles (usec): | 1.00th=[ 6], 5.00th=[ 6], 10.00th=[ 7], 20.00th=[ 7], | 30.00th=[ 7], 40.00th=[ 8], 50.00th=[ 8], 60.00th=[ 9], | 70.00th=[ 10], 80.00th=[ 12], 90.00th=[ 85], 95.00th=[ 97], | 99.00th=[ 104], 99.50th=[ 115], 99.90th=[ 141], 99.95th=[ 149], | 99.99th=[ 194] iodepth=8 clat percentiles (usec): | 1.00th=[ 6], 5.00th=[ 7], 10.00th=[ 7], 20.00th=[ 9], | 30.00th=[ 11], 40.00th=[ 12], 50.00th=[ 14], 60.00th=[ 15], | 70.00th=[ 18], 80.00th=[ 22], 90.00th=[ 93], 95.00th=[ 103], | 99.00th=[ 120], 99.50th=[ 130], 99.90th=[ 157], 99.95th=[ 167], | 99.99th=[ 208] iodepth=16 clat percentiles (usec): | 1.00th=[ 16], 5.00th=[ 24], 10.00th=[ 28], 20.00th=[ 32], | 30.00th=[ 34], 40.00th=[ 37], 50.00th=[ 39], 60.00th=[ 41], | 70.00th=[ 44], 80.00th=[ 51], 90.00th=[ 117], 95.00th=[ 128], | 99.00th=[ 147], 99.50th=[ 159], 99.90th=[ 194], 99.95th=[ 235], | 99.99th=[ 1909] iodepth=32 clat percentiles (usec): | 1.00th=[ 72], 5.00th=[ 78], 10.00th=[ 81], 20.00th=[ 84], | 30.00th=[ 86], 40.00th=[ 88], 50.00th=[ 90], 60.00th=[ 93], | 70.00th=[ 96], 80.00th=[ 114], 90.00th=[ 169], 95.00th=[ 182], | 99.00th=[ 202], 99.50th=[ 212], 99.90th=[ 239], 99.95th=[ 253], | 99.99th=[ 302] iodepth=64 clat percentiles (usec): | 1.00th=[ 178], 5.00th=[ 184], 10.00th=[ 186], 20.00th=[ 192], | 30.00th=[ 196], 40.00th=[ 200], 50.00th=[ 204], 60.00th=[ 206], | 70.00th=[ 210], 80.00th=[ 221], 90.00th=[ 281], 95.00th=[ 293], | 99.00th=[ 318], 99.50th=[ 330], 99.90th=[ 355], 99.95th=[ 367], | 99.99th=[ 437] iodepth=128 clat percentiles (usec): | 1.00th=[ 379], 5.00th=[ 388], 10.00th=[ 392], 20.00th=[ 396], | 30.00th=[ 396], 40.00th=[ 400], 50.00th=[ 404], 60.00th=[ 408], | 70.00th=[ 424], 80.00th=[ 437], 90.00th=[ 482], 95.00th=[ 498], | 99.00th=[ 529], 99.50th=[ 537], 99.90th=[ 570], 99.95th=[ 635], | 99.99th=[ 2311] iodepth=256 clat percentiles (usec): | 1.00th=[ 783], 5.00th=[ 783], 10.00th=[ 791], 20.00th=[ 791], | 30.00th=[ 791], 40.00th=[ 799], 50.00th=[ 799], 60.00th=[ 799], | 70.00th=[ 807], 80.00th=[ 816], 90.00th=[ 881], 95.00th=[ 889], | 99.00th=[ 914], 99.50th=[ 930], 99.90th=[ 979], 99.95th=[ 996], | 99.99th=[ 1237]
Hao Xu (8):
io-wq: code clean for io_wq_add_work_after()
io-wq: add helper to merge two wq_lists
io_uring: add a limited tw list for irq completion work
io_uring: add helper for task work execution code
io_uring: split io_req_complete_post() and add a helper
io_uring: move up io_put_kbuf() and io_put_rw_kbuf()
io_uring: add tw_ctx for io_uring_task
io_uring: batch completion in prior_task_list
fs/io-wq.h | 27 ++++++-- fs/io_uring.c | 171 ++++++++++++++++++++++++++++++++++---------------- 2 files changed, 139 insertions(+), 59 deletions(-)
Signature · 1 line
2.24.4