nvme: fio workload hangs on 16m random read
Hi All,
My CI system has found an issue for the following fio workload:
root@debian:~# insmod /mnt/v6.14-rc5-build/drivers/nvme/host/nvme-core.ko root@debian:~# insmod /mnt/v6.14-rc5-build/drivers/nvme/host/nvme.ko poll_queues=1
"fio" "--group_reporting" "--name=default" "--filename=/dev/nvme0n1" "--readwrite=randread" "--blocksize=16777216" "--direct=1" "--numjobs=1" "--ioengine=io_uring" "--iodepth=1" "--hipri=1" "--time_based=1" "--runtime=3000"
fio will hang, some times immediately, some times after a few minutes.
I observed this on v6.14-rc1 and v6.14-rc5. It is not present in v6.13.
Can any of you reproduce this? I was able to reproduce this on real hardware and in qemu.
Best regards,
Andreas Hindborg
Keith Busch
maintainerQuoted 12 lines · Andreas Hindborg
Hi All,
My CI system has found an issue for the following fio workload:
root@debian:~# insmod /mnt/v6.14-rc5-build/drivers/nvme/host/nvme-core.ko root@debian:~# insmod /mnt/v6.14-rc5-build/drivers/nvme/host/nvme.ko poll_queues=1"fio" "--group_reporting" "--name=default" "--filename=/dev/nvme0n1" "--readwrite=randread" "--blocksize=16777216" "--direct=1" "--numjobs=1" "--ioengine=io_uring" "--iodepth=1" "--hipri=1" "--time_based=1" "--runtime=3000"
Quoted 6 lines
fio will hang, some times immediately, some times after a few minutes.
I observed this on v6.14-rc1 and v6.14-rc5. It is not present in v6.13.
Can any of you reproduce this? I was able to reproduce this on real hardware and in qemu.
Could you try adding this patch to your kernel?
Quoted 15 lines · Keith Busch
Quoted 14 lines · Andreas Hindborg
Hi All,
My CI system has found an issue for the following fio workload:
root@debian:~# insmod /mnt/v6.14-rc5-build/drivers/nvme/host/nvme-core.ko root@debian:~# insmod /mnt/v6.14-rc5-build/drivers/nvme/host/nvme.ko poll_queues=1"fio" "--group_reporting" "--name=default" "--filename=/dev/nvme0n1" "--readwrite=randread" "--blocksize=16777216" "--direct=1" "--numjobs=1" "--ioengine=io_uring" "--iodepth=1" "--hipri=1" "--time_based=1" "--runtime=3000"
fio will hang, some times immediately, some times after a few minutes.
Quoted 8 lines
Quoted 4 lines
I observed this on v6.14-rc1 and v6.14-rc5. It is not present in v6.13.
Can any of you reproduce this? I was able to reproduce this on real hardware and in qemu.
Could you try adding this patch to your kernel?
Thanks Keith, at first glance that seems to solve the issue. Will restart my CI with this applied.
Best regards,
Andreas Hindborg