Skip to contentSkip to conversation

fio: iodepth_batch_submit is broken on libaio

DISCUSSION Ming Lei 3 participants 3 messages Sep 27, 2019

lore
Share & tools
lore

Ming Lei

author
#0 lore raw

Hi Jens,

Looks fio's 'iodepth_batch_submit' is broken.

When running the attached fio script[1], I observed that 'nr' passed to io_submit() is 1, except that 64 is only passed to the 1st io_submit().

Then plug merging can't work at all.

[root@ktest-08 tools]# cat bpf
#include <uapi/linux/aio_abi.h> tracepoint:syscalls:sys_enter_io_submit { @nr = hist(args->nr); } [root@ktest-08 tools]# bpftrace ./bpf
Attaching 1 probe...
^C

@nr:

[1]               902349 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@|
[2, 4)                 0 |                                                    |
[4, 8)                 0 |                                                    |
[8, 16)                0 |                                                    |
[16, 32)               0 |                                                    |
[32, 64)               0 |                                                    |
[64, 128)              1 |                                                    |

[1] fio script:

fio --readwrite=write --filesize=4g --filename=/dev/sda \
	--runtime=10s --time_based \
	--ioengine=libaio --direct=1 --bs=4k --iodepth=64 \
	--iodepth_batch_submit=64 \
	--numjobs=64 \
	--name=foo

Thanks,
Ming

#1 lore raw
Quoted 25 lines · Ming Lei

Hi Jens,

Looks fio's 'iodepth_batch_submit' is broken.

When running the attached fio script[1], I observed that 'nr' passed to io_submit() is 1, except that 64 is only passed to the 1st io_submit().

Then plug merging can't work at all.

[root@ktest-08 tools]# cat bpf
#include <uapi/linux/aio_abi.h> tracepoint:syscalls:sys_enter_io_submit { @nr = hist(args->nr); } [root@ktest-08 tools]# bpftrace ./bpf
Attaching 1 probe...
^C

@nr:

[1]               902349 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@|
[2, 4)                 0 |                                                    |
[4, 8)                 0 |                                                    |
[8, 16)                0 |                                                    |
[16, 32)               0 |                                                    |
[32, 64)               0 |                                                    |
[64, 128)              1 |                                                    |
Quoted 7 lines

[1] fio script:

fio --readwrite=write --filesize=4g --filename=/dev/sda \
	--runtime=10s --time_based \
	--ioengine=libaio --direct=1 --bs=4k --iodepth=64 \
	--iodepth_batch_submit=64 \
	--numjobs=64 \
	--name=foo

Try adding iodepth_batch_complete as well.

Signature · 1 line
Jens Axboe
#2 lore raw
Quoted 35 lines · Jens Axboe
Quoted 25 lines · Ming Lei

Hi Jens,

Looks fio's 'iodepth_batch_submit' is broken.

When running the attached fio script[1], I observed that 'nr' passed to io_submit() is 1, except that 64 is only passed to the 1st io_submit().

Then plug merging can't work at all.

[root@ktest-08 tools]# cat bpf
#include <uapi/linux/aio_abi.h> tracepoint:syscalls:sys_enter_io_submit { @nr = hist(args->nr); } [root@ktest-08 tools]# bpftrace ./bpf
Attaching 1 probe...
^C

@nr:

[1]               902349 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@|
[2, 4)                 0 |                                                    |
[4, 8)                 0 |                                                    |
[8, 16)                0 |                                                    |
[16, 32)               0 |                                                    |
[32, 64)               0 |                                                    |
[64, 128)              1 |                                                    |
Quoted 7 lines

[1] fio script:

fio --readwrite=write --filesize=4g --filename=/dev/sda \
      --runtime=10s --time_based \
      --ioengine=libaio --direct=1 --bs=4k --iodepth=64 \
      --iodepth_batch_submit=64 \
      --numjobs=64 \
      --name=foo
Quoted 1 line

Try adding iodepth_batch_complete as well.

Yeah, it works, thanks!

Signature · 1 line
Ming Lei