pmem: set QUEUE_FLAG_NOWAIT
Subsystems (3)
v2 cover pmem: set QUEUE_FLAG_NOWAIT
Hi,
Set the QUEUE_FLAG_NOWAIT. Following are the performance numbers with io_uring fio engine for random read, note that device has been populated fully with randwrite workload before taking these numbers :-
linux-block (pmem-nowait-on) # grep IOPS pmem*fio | column -t
pmem-nowait-off-1.fio: read: IOPS=3683k, BW=14.0GiB/s pmem-nowait-off-2.fio: read: IOPS=3819k, BW=14.6GiB/s pmem-nowait-off-3.fio: read: IOPS=3999k, BW=15.3GiB/s
pmem-nowait-on-1.fio: read: IOPS=5837k, BW=22.3GiB/s pmem-nowait-on-2.fio: read: IOPS=5936k, BW=22.6GiB/s pmem-nowait-on-3.fio: read: IOPS=5945k, BW=22.7GiB/s
linux-block (pmem-nowait-on) # grep cpu pmem*fio | column -t
pmem-nowait-off-1.fio: cpu : usr=7.09%, sys=29.65%, ctx=198742065 pmem-nowait-off-2.fio: cpu : usr=6.89%, sys=30.56%, ctx=205817652 pmem-nowait-off-3.fio: cpu : usr=6.86%, sys=30.94%, ctx=222627094
pmem-nowait-on-1.fio: cpu : usr=10.58%, sys=88.44%, ctx=27181 pmem-nowait-on-2.fio: cpu : usr=10.50%, sys=87.75%, ctx=25746 pmem-nowait-on-3.fio: cpu : usr=10.67%, sys=88.60%, ctx=28261
linux-block (pmem-nowait-on) # grep slat pmem*fio | column -t
pmem-nowait-off-1.fio: slat (nsec): min=432, max=50847k, avg=9324.69 pmem-nowait-off-2.fio: slat (nsec): min=441, max=52557k, avg=9132.45 pmem-nowait-off-3.fio: slat (nsec): min=430, max=36113k, avg=9132.63
pmem-nowait-on-1.fio: slat (nsec): min=1393, max=68090k, avg=7615.31 pmem-nowait-on-2.fio: slat (nsec): min=1222, max=44137k, avg=7493.77 pmem-nowait-on-3.fio: slat (nsec): min=1493, max=40100k, avg=7486.36
Please let me know if further testing is needed I've ran fio verification job in order to make verify these changes.
-ck
V2:-
Unconditionally set the QUEUE_FLAG_NOWAIT in pmem_attach_disk() along with the other queue flags.
Chaitanya Kulkarni (1):
pmem: set QUEUE_FLAG_NOWAIT
drivers/nvdimm/pmem.c | 1 + 1 file changed, 1 insertion(+)
linux-block (pmem-nowait-on) # ./test-pmem.sh
++ unload_mod
++ rmmod nd_pmem
++ rmmod nd_btt
++ git checkout for-next
Switched to branch 'for-next'
Your branch is ahead of 'origin/for-next' by 155 commits.
(use "git push" to publish your local commits)
++ git log -1
commit e50c5e801b5a9e1797eb5a157eac1b5e50084486 (HEAD -> for-next)
Merge: e6dfe861227b e98acd815ebf
Author: Chaitanya Kulkarni <[email protected]>
Date: Mon Jul 31 14:48:39 2023 -0700
Merge branch 'for-next' of git://git.kernel.dk/linux-block into for-next
++ makej M=drivers/nvdimm
CC [M] drivers/nvdimm/pmem.o
LD [M] drivers/nvdimm/nd_pmem.o
MODPOST drivers/nvdimm/Module.symvers
LD [M] drivers/nvdimm/nd_pmem.ko
++ load_mod
++ insmod drivers/nvdimm/nd_btt.ko
++ insmod drivers/nvdimm/nd_pmem.ko
++ sleep 1
++ test_pmem nowait-off
++ sleep 1
++ fio fio/verify.fio --ioengine=io_uring --size=896M --filename=/dev/pmem0
write-and-verify: (g=0): rw=randwrite, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=io_uring, iodepth=16 fio-3.34
Starting 1 process
Jobs: 1 (f=1)
write-and-verify: (groupid=0, jobs=1): err= 0: pid=4662: Mon Jul 31 15:24:07 2023
read: IOPS=265k, BW=1036MiB/s (1087MB/s)(566MiB/546msec)
slat (nsec): min=501, max=31820, avg=2733.20, stdev=1179.52
clat (nsec): min=17022, max=96063, avg=56544.09, stdev=5848.22
lat (usec): min=20, max=101, avg=59.28, stdev= 6.06
clat percentiles (nsec):
| 1.00th=[44288], 5.00th=[46848], 10.00th=[48896], 20.00th=[51456],
| 30.00th=[53504], 40.00th=[55552], 50.00th=[56576], 60.00th=[58112],
| 70.00th=[59648], 80.00th=[61184], 90.00th=[63744], 95.00th=[66048],
| 99.00th=[72192], 99.50th=[74240], 99.90th=[80384], 99.95th=[82432],
| 99.99th=[88576]
write: IOPS=209k, BW=818MiB/s (857MB/s)(896MiB/1096msec); 0 zone resets
slat (nsec): min=1352, max=113484, avg=4293.77, stdev=1425.01
clat (usec): min=25, max=285, avg=71.81, stdev= 9.33
lat (usec): min=31, max=288, avg=76.10, stdev= 9.56
clat percentiles (usec):
| 1.00th=[ 45], 5.00th=[ 61], 10.00th=[ 65], 20.00th=[ 68],
| 30.00th=[ 70], 40.00th=[ 71], 50.00th=[ 72], 60.00th=[ 73],
| 70.00th=[ 75], 80.00th=[ 77], 90.00th=[ 80], 95.00th=[ 84],
| 99.00th=[ 102], 99.50th=[ 113], 99.90th=[ 169], 99.95th=[ 180],
| 99.99th=[ 219]
bw ( KiB/s): min=152408, max=857568, per=73.07%, avg=611669.33, stdev=398064.54, samples=3
iops : min=38102, max=214392, avg=152917.33, stdev=99516.13, samples=3
lat (usec) : 20=0.01%, 50=6.40%, 100=92.93%, 250=0.67%, 500=0.01%
cpu : usr=35.49%, sys=63.60%, ctx=2561, majf=0, minf=3973
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=100.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.1%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=144875,229376,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=16Run status group 0 (all jobs):
READ: bw=1036MiB/s (1087MB/s), 1036MiB/s-1036MiB/s (1087MB/s-1087MB/s), io=566MiB (593MB), run=546-546msec
WRITE: bw=818MiB/s (857MB/s), 818MiB/s-818MiB/s (857MB/s-857MB/s), io=896MiB (940MB), run=1096-1096msec
Disk stats (read/write):
pmem0: ios=0/0, merge=0/0, ticks=0/0, in_queue=0, util=0.00%
++ fio fio/randwrite.fio --ioengine=io_uring --size=896M --filename=/dev/pmem0
RANDWRITE: (g=0): rw=randwrite, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=io_uring, iodepth=2
...
fio-3.34
Starting 48 processes
Jobs: 48 (f=48): [w(48)][100.0%][w=20.6GiB/s][w=5389k IOPS][eta 00m:00s]
RANDWRITE: (groupid=0, jobs=48): err= 0: pid=4681: Mon Jul 31 15:25:07 2023
write: IOPS=5147k, BW=19.6GiB/s (21.1GB/s)(1178GiB/60002msec); 0 zone resets
slat (nsec): min=380, max=57147k, avg=6895.36, stdev=28357.20
clat (nsec): min=130, max=57173k, avg=11233.63, stdev=44260.60
lat (nsec): min=1944, max=57174k, avg=18128.99, stdev=53168.60
clat percentiles (usec):
| 1.00th=[ 3], 5.00th=[ 6], 10.00th=[ 6], 20.00th=[ 6],
| 30.00th=[ 7], 40.00th=[ 7], 50.00th=[ 8], 60.00th=[ 9],
| 70.00th=[ 10], 80.00th=[ 12], 90.00th=[ 21], 95.00th=[ 32],
| 99.00th=[ 58], 99.50th=[ 74], 99.90th=[ 135], 99.95th=[ 186],
| 99.99th=[ 742]
bw ( MiB/s): min= 5793, max=30898, per=100.00%, avg=20121.35, stdev=146.64, samples=5712
iops : min=1483165, max=7909996, avg=5151064.04, stdev=37540.43, samples=5712
lat (nsec) : 250=0.01%, 500=0.01%, 750=0.01%, 1000=0.01%
lat (usec) : 2=0.33%, 4=2.22%, 10=71.71%, 20=15.22%, 50=9.07%
lat (usec) : 100=1.23%, 250=0.18%, 500=0.02%, 750=0.01%, 1000=0.01%
lat (msec) : 2=0.01%, 4=0.01%, 10=0.01%, 20=0.01%, 50=0.01%
lat (msec) : 100=0.01%
cpu : usr=9.36%, sys=31.21%, ctx=286817971, majf=0, minf=607
IO depths : 1=0.1%, 2=100.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=0,308833813,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=2Run status group 0 (all jobs):
WRITE: bw=19.6GiB/s (21.1GB/s), 19.6GiB/s-19.6GiB/s (21.1GB/s-21.1GB/s), io=1178GiB (1265GB), run=60002-60002msec
Disk stats (read/write):
pmem0: ios=0/0, merge=0/0, ticks=0/0, in_queue=0, util=0.00%
++ for i in 1 2 3
++ fio fio/randread.fio --ioengine=io_uring --size=896M --filename=/dev/pmem0 --output=pmem-nowait-off-1.fio ++ for i in 1 2 3[r(48)][100.0%][r=14.7GiB/s][r=3864k IOPS][eta 00m:00s]
++ fio fio/randread.fio --ioengine=io_uring --size=896M --filename=/dev/pmem0 --output=pmem-nowait-off-2.fio ++ for i in 1 2 3[r(48)][100.0%][r=15.7GiB/s][r=4116k IOPS][eta 00m:00s]
++ fio fio/randread.fio --ioengine=io_uring --size=896M --filename=/dev/pmem0 --output=pmem-nowait-off-3.fio ++ unload_mod8): [r(48)][100.0%][r=15.1GiB/s][r=3966k IOPS][eta 00m:00s]
++ rmmod nd_pmem
++ rmmod nd_btt
++ git checkout pmem-nowait-on
Switched to branch 'pmem-nowait-on'
++ git log -1
commit 24573494ec05e8d7bb7acb82e4a0e400297272aa (HEAD -> pmem-nowait-on)
Author: Chaitanya Kulkarni <[email protected]> Date: Fri May 12 03:24:54 2023 -0700
pmem: set QUEUE_FLAG_NOWAIT
Set the QUEUE_FLAG_NOWAIT. Following are the performance numbers with io_uring fio engine for random read, note that device has been populated fully with randwrite workload before taking these numbers :-
++ rmmod nd_pmem
rmmod: ERROR: Module nd_pmem is not currently loaded
++ makej M=drivers/nvdimm
CC [M] drivers/nvdimm/pmem.o
LD [M] drivers/nvdimm/nd_pmem.o
MODPOST drivers/nvdimm/Module.symvers
LD [M] drivers/nvdimm/nd_pmem.ko
++ load_mod
++ insmod drivers/nvdimm/nd_btt.ko
++ insmod drivers/nvdimm/nd_pmem.ko
++ sleep 1
++ test_pmem nowait-on
++ sleep 1
++ fio fio/verify.fio --ioengine=io_uring --size=896M --filename=/dev/pmem0
write-and-verify: (g=0): rw=randwrite, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=io_uring, iodepth=16 fio-3.34
Starting 1 process
Jobs: 1 (f=1)
write-and-verify: (groupid=0, jobs=1): err= 0: pid=5662: Mon Jul 31 15:28:13 2023
read: IOPS=448k, BW=1750MiB/s (1835MB/s)(567MiB/324msec)
slat (nsec): min=1092, max=56347, avg=1349.16, stdev=838.70
clat (nsec): min=741, max=152570, avg=33482.76, stdev=9895.86
lat (usec): min=2, max=154, avg=34.83, stdev=10.20
clat percentiles (usec):
| 1.00th=[ 30], 5.00th=[ 31], 10.00th=[ 31], 20.00th=[ 31],
| 30.00th=[ 31], 40.00th=[ 31], 50.00th=[ 32], 60.00th=[ 32],
| 70.00th=[ 32], 80.00th=[ 32], 90.00th=[ 35], 95.00th=[ 49],
| 99.00th=[ 87], 99.50th=[ 96], 99.90th=[ 113], 99.95th=[ 122],
| 99.99th=[ 139]
write: IOPS=207k, BW=810MiB/s (849MB/s)(896MiB/1106msec); 0 zone resets
slat (nsec): min=2135, max=81445, avg=4433.09, stdev=1568.20
clat (nsec): min=441, max=210380, avg=72445.60, stdev=10663.97
lat (usec): min=3, max=254, avg=76.88, stdev=11.22
clat percentiles (usec):
| 1.00th=[ 56], 5.00th=[ 60], 10.00th=[ 62], 20.00th=[ 65],
| 30.00th=[ 68], 40.00th=[ 70], 50.00th=[ 72], 60.00th=[ 75],
| 70.00th=[ 77], 80.00th=[ 79], 90.00th=[ 83], 95.00th=[ 86],
| 99.00th=[ 120], 99.50th=[ 135], 99.90th=[ 161], 99.95th=[ 169],
| 99.99th=[ 192]
bw ( KiB/s): min=764336, max=876176, per=98.88%, avg=820256.00, stdev=79082.82, samples=2
iops : min=191084, max=219044, avg=205064.00, stdev=19770.71, samples=2
lat (nsec) : 500=0.01%, 750=0.01%
lat (usec) : 4=0.01%, 10=0.01%, 20=0.01%, 50=36.90%, 100=62.08%
lat (usec) : 250=1.01%
cpu : usr=37.58%, sys=62.28%, ctx=3, majf=0, minf=3983
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=100.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.1%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=145180,229376,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=16Run status group 0 (all jobs):
READ: bw=1750MiB/s (1835MB/s), 1750MiB/s-1750MiB/s (1835MB/s-1835MB/s), io=567MiB (595MB), run=324-324msec
WRITE: bw=810MiB/s (849MB/s), 810MiB/s-810MiB/s (849MB/s-849MB/s), io=896MiB (940MB), run=1106-1106msec
Disk stats (read/write):
pmem0: ios=0/0, merge=0/0, ticks=0/0, in_queue=0, util=0.00%
++ fio fio/randwrite.fio --ioengine=io_uring --size=896M --filename=/dev/pmem0
RANDWRITE: (g=0): rw=randwrite, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=io_uring, iodepth=2
...
fio-3.34
Starting 48 processes
Jobs: 48 (f=48): [w(48)][100.0%][w=39.7GiB/s][w=10.4M IOPS][eta 00m:00s]
RANDWRITE: (groupid=0, jobs=48): err= 0: pid=5665: Mon Jul 31 15:29:13 2023
write: IOPS=9926k, BW=37.9GiB/s (40.7GB/s)(2272GiB/60001msec); 0 zone resets
slat (nsec): min=1192, max=32084k, avg=4269.53, stdev=21566.61
clat (nsec): min=501, max=32087k, avg=5022.97, stdev=23279.45
lat (nsec): min=1854, max=32098k, avg=9292.50, stdev=31761.04
clat percentiles (nsec):
| 1.00th=[ 2544], 5.00th=[ 2992], 10.00th=[ 3280], 20.00th=[ 3600],
| 30.00th=[ 3920], 40.00th=[ 4256], 50.00th=[ 4576], 60.00th=[ 4960],
| 70.00th=[ 5408], 80.00th=[ 5984], 90.00th=[ 7008], 95.00th=[ 7968],
| 99.00th=[10944], 99.50th=[14528], 99.90th=[24192], 99.95th=[33536],
| 99.99th=[63744]
bw ( MiB/s): min=26941, max=42586, per=100.00%, avg=38794.41, stdev=73.04, samples=5712
iops : min=6897082, max=10902168, avg=9931366.71, stdev=18699.20, samples=5712
lat (nsec) : 750=0.01%, 1000=0.01%
lat (usec) : 2=0.01%, 4=32.93%, 10=65.63%, 20=1.26%, 50=0.16%
lat (usec) : 100=0.01%, 250=0.01%, 500=0.01%, 750=0.01%, 1000=0.01%
lat (msec) : 2=0.01%, 4=0.01%, 10=0.01%, 20=0.01%, 50=0.01%
cpu : usr=18.08%, sys=80.64%, ctx=30470, majf=0, minf=587
IO depths : 1=0.1%, 2=100.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=0,595577284,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=2Run status group 0 (all jobs):
WRITE: bw=37.9GiB/s (40.7GB/s), 37.9GiB/s-37.9GiB/s (40.7GB/s-40.7GB/s), io=2272GiB (2439GB), run=60001-60001msec
Disk stats (read/write):
pmem0: ios=0/0, merge=0/0, ticks=0/0, in_queue=0, util=0.00%
++ for i in 1 2 3
++ fio fio/randread.fio --ioengine=io_uring --size=896M --filename=/dev/pmem0 --output=pmem-nowait-on-1.fio ++ for i in 1 2 3[r(48)][100.0%][r=22.7GiB/s][r=5960k IOPS][eta 00m:00s]
++ fio fio/randread.fio --ioengine=io_uring --size=896M --filename=/dev/pmem0 --output=pmem-nowait-on-2.fio ++ for i in 1 2 3[r(48)][100.0%][r=22.7GiB/s][r=5953k IOPS][eta 00m:00s]
++ fio fio/randread.fio --ioengine=io_uring --size=896M --filename=/dev/pmem0 --output=pmem-nowait-on-3.fio linux-block (pmem-nowait-on) # ][r=22.7GiB/s][r=5946k IOPS][eta 00m:00s]
linux-block (pmem-nowait-on) #
linux-block (pmem-nowait-on) # grep IOPS pmem*fio | column -t
pmem-nowait-off-1.fio: read: IOPS=3683k, BW=14.0GiB/s (15.1GB/s)(843GiB/60002msec) pmem-nowait-off-2.fio: read: IOPS=3819k, BW=14.6GiB/s (15.6GB/s)(874GiB/60002msec) pmem-nowait-off-3.fio: read: IOPS=3999k, BW=15.3GiB/s (16.4GB/s)(915GiB/60002msec)
pmem-nowait-on-1.fio: read: IOPS=5837k, BW=22.3GiB/s (23.9GB/s)(1336GiB/60002msec) pmem-nowait-on-2.fio: read: IOPS=5936k, BW=22.6GiB/s (24.3GB/s)(1359GiB/60002msec) pmem-nowait-on-3.fio: read: IOPS=5945k, BW=22.7GiB/s (24.3GB/s)(1361GiB/60002msec)
linux-block (pmem-nowait-on) # grep cpu pmem*fio | column -t
pmem-nowait-off-1.fio: cpu : usr=7.09%, sys=29.65%, ctx=198742065 pmem-nowait-off-2.fio: cpu : usr=6.89%, sys=30.56%, ctx=205817652 pmem-nowait-off-3.fio: cpu : usr=6.86%, sys=30.94%, ctx=222627094
pmem-nowait-on-1.fio: cpu : usr=10.58%, sys=88.44%, ctx=27181, pmem-nowait-on-2.fio: cpu : usr=10.50%, sys=87.75%, ctx=25746, pmem-nowait-on-3.fio: cpu : usr=10.67%, sys=88.60%, ctx=28261, linux-block (pmem-nowait-on) # grep slat pmem*fio | column -t pmem-nowait-off-1.fio: slat (nsec): min=432, max=50847k, avg=9324.69 pmem-nowait-off-2.fio: slat (nsec): min=441, max=52557k, avg=9132.45 pmem-nowait-off-3.fio: slat (nsec): min=430, max=36113k, avg=9132.63
pmem-nowait-on-1.fio: slat (nsec): min=1393, max=68090k, avg=7615.31 pmem-nowait-on-2.fio: slat (nsec): min=1222, max=44137k, avg=7493.77 pmem-nowait-on-3.fio: slat (nsec): min=1493, max=40100k, avg=7486.36
linux-block (pmem-nowait-on) #
Signature · 1 line
2.40.0
PATCH v2 1/1 pmem: set QUEUE_FLAG_NOWAIT
Set the QUEUE_FLAG_NOWAIT. Following are the performance numbers with io_uring fio engine for random read, note that device has been populated fully with randwrite workload before taking these numbers :-
linux-block (pmem-nowait-on) # grep IOPS pmem*fio | column -t
pmem-nowait-off-1.fio: read: IOPS=3683k, BW=14.0GiB/s pmem-nowait-off-2.fio: read: IOPS=3819k, BW=14.6GiB/s pmem-nowait-off-3.fio: read: IOPS=3999k, BW=15.3GiB/s
pmem-nowait-on-1.fio: read: IOPS=5837k, BW=22.3GiB/s pmem-nowait-on-2.fio: read: IOPS=5936k, BW=22.6GiB/s pmem-nowait-on-3.fio: read: IOPS=5945k, BW=22.7GiB/s
linux-block (pmem-nowait-on) # grep cpu pmem*fio | column -t
pmem-nowait-off-1.fio: cpu : usr=7.09%, sys=29.65%, ctx=198742065 pmem-nowait-off-2.fio: cpu : usr=6.89%, sys=30.56%, ctx=205817652 pmem-nowait-off-3.fio: cpu : usr=6.86%, sys=30.94%, ctx=222627094
pmem-nowait-on-1.fio: cpu : usr=10.58%, sys=88.44%, ctx=27181 pmem-nowait-on-2.fio: cpu : usr=10.50%, sys=87.75%, ctx=25746 pmem-nowait-on-3.fio: cpu : usr=10.67%, sys=88.60%, ctx=28261
linux-block (pmem-nowait-on) # grep slat pmem*fio | column -t
pmem-nowait-off-1.fio: slat (nsec): min=432, max=50847k, avg=9324.69 pmem-nowait-off-2.fio: slat (nsec): min=441, max=52557k, avg=9132.45 pmem-nowait-off-3.fio: slat (nsec): min=430, max=36113k, avg=9132.63
pmem-nowait-on-1.fio: slat (nsec): min=1393, max=68090k, avg=7615.31 pmem-nowait-on-2.fio: slat (nsec): min=1222, max=44137k, avg=7493.77 pmem-nowait-on-3.fio: slat (nsec): min=1493, max=40100k, avg=7486.36
--- drivers/nvdimm/pmem.c | 1 + 1 file changed, 1 insertion(+)
drivers/nvdimm/pmem.c +1 -0
@@ -543,6 +543,7 @@ static int pmem_attach_disk(struct device *dev,blk_queue_max_hw_sectors (q,UINT_MAX );blk_queue_flag_set (QUEUE_FLAG_NONROT , q);blk_queue_flag_set (QUEUE_FLAG_SYNCHRONOUS , q); +blk_queue_flag_set (QUEUE_FLAG_NOWAIT , q);if (pmem->pfn_flags &PFN_MAP )blk_queue_flag_set (QUEUE_FLAG_DAX , q);
Signature · 1 line
2.40.0
Quoted 3 lines · Chaitanya Kulkarni
Set the
QUEUE_FLAG_NOWAIT. Following are the performance numbers withio_uringfio engine for random read, note that device has been populated fully with randwrite workload before taking these numbers :-
I am slightly embarrassed to have to ask this question, but what are the implications of setting this queue flag? Is the submit_bio routine expected to never block? Is the I/O expected to be performed asynchronously? If either of those is the case, then pmem does not qualify.
-Jeff
Quoted 44 lines
linux-block (pmem-nowait-on) # grep IOPS pmem*fio | column -t
pmem-nowait-off-1.fio: read: IOPS=3683k, BW=14.0GiB/s pmem-nowait-off-2.fio: read: IOPS=3819k, BW=14.6GiB/s pmem-nowait-off-3.fio: read: IOPS=3999k, BW=15.3GiB/spmem-nowait-on-1.fio: read: IOPS=5837k, BW=22.3GiB/s pmem-nowait-on-2.fio: read: IOPS=5936k, BW=22.6GiB/s pmem-nowait-on-3.fio: read: IOPS=5945k, BW=22.7GiB/slinux-block (pmem-nowait-on) # grep cpu pmem*fio | column -t
pmem-nowait-off-1.fio: cpu : usr=7.09%, sys=29.65%, ctx=198742065 pmem-nowait-off-2.fio: cpu : usr=6.89%, sys=30.56%, ctx=205817652 pmem-nowait-off-3.fio: cpu : usr=6.86%, sys=30.94%, ctx=222627094pmem-nowait-on-1.fio: cpu : usr=10.58%, sys=88.44%, ctx=27181 pmem-nowait-on-2.fio: cpu : usr=10.50%, sys=87.75%, ctx=25746 pmem-nowait-on-3.fio: cpu : usr=10.67%, sys=88.60%, ctx=28261linux-block (pmem-nowait-on) # grep slat pmem*fio | column -t
pmem-nowait-off-1.fio: slat (nsec): min=432, max=50847k, avg=9324.69 pmem-nowait-off-2.fio: slat (nsec): min=441, max=52557k, avg=9132.45 pmem-nowait-off-3.fio: slat (nsec): min=430, max=36113k, avg=9132.63pmem-nowait-on-1.fio: slat (nsec): min=1393, max=68090k, avg=7615.31 pmem-nowait-on-2.fio: slat (nsec): min=1222, max=44137k, avg=7493.77 pmem-nowait-on-3.fio: slat (nsec): min=1493, max=40100k, avg=7486.36Signed-off-by Chaitanya Kulkarni <[email protected]>--- drivers/nvdimm/pmem.c | 1 + 1 file changed, 1 insertion(+)drivers/nvdimm/pmem.c +1
-0@@ -543,6 +543,7 @@ static int pmem_attach_disk(struct device *dev,blk_queue_max_hw_sectors (q,UINT_MAX );blk_queue_flag_set (QUEUE_FLAG_NONROT , q);blk_queue_flag_set (QUEUE_FLAG_SYNCHRONOUS , q); +blk_queue_flag_set (QUEUE_FLAG_NOWAIT , q);if (pmem->pfn_flags &PFN_MAP )blk_queue_flag_set (QUEUE_FLAG_DAX , q);
Quoted 3 lines · Jeff Moyer
I am slightly embarrassed to have to ask this question, but what are the implications of setting this queue flag? Is the
submit_bioroutine expected to never block?
Yes, at least not significantly.
Quoted 2 lines
Is the I/O expected to be performed
asynchronously?
Not nessecarily if it is fast enough..
Hi, Christoph,
Quoted 6 lines · Christoph Hellwig
Quoted 3 lines · Jeff Moyer
I am slightly embarrassed to have to ask this question, but what are the implications of setting this queue flag? Is the
submit_bioroutine expected to never block?Yes, at least not significantly.
If there happens to be poisoned memory, the write path can make an acpi device specific method call. That involves taking a mutex (see the call chain starting at acpi_ex_enter_interpreter()). I'm not sure how long a DSM takes, but I doubt it's fast.
Quoted 4 lines
Quoted 2 lines
Is the I/O expected to be performed
asynchronously?Not nessecarily if it is fast enough..
Clear as mud. :) It's a memcpy on potentially "slow" memory. So, the amount of time spent copying depends on the speed of the cpu, the media and the size of the I/O. I don't know off-hand what the upper bound would be on today's pmem.
-Jeff
Quoted 5 lines · Jeff Moyer
Hi, Christoph,
Quoted 1 line · Christoph Hellwig
On Tue, Aug 01, 2023 at 11:23:36AM -0400, Jeff Moyer wrote:
Quoted 8 lines
Quoted 4 lines
Quoted 3 lines
I am slightly embarrassed to have to ask this question, but what are the implications of setting this queue flag? Is the
submit_bioroutine expected to never block?Yes, at least not significantly.
If there happens to be poisoned memory, the write path can make an acpi device specific method call. That involves taking a mutex (see the call chain starting at
acpi_ex_enter_interpreter()). I'm not sure how long a DSM takes, but I doubt it's fast.
for this case I can add bio->bio_opf & REQ_NOWAIT check and return with bio_wouldblock_error() that will take care of blocking case e.g. something like this untested as a prep patch for write path :-
linux-block (pmem-nowait-on) # git diff
drivers/nvdimm/pmem.c +0 -0
@@ -179,12 +179,16 @@ static blk_status_t pmem_do_read(struct
pmem_device *pmem,
static blk_status_t pmem_do_write(struct pmem_device *pmem, struct page *page, unsigned int page_off,
- sector_t sector, unsigned int len)
+ sector_t sector, unsigned int len, struct bio *bio)
{
phys_addr_t pmem_off = to_offset(pmem, sector);
void *pmem_addr = pmem->virt_addr + pmem_off;
if (unlikely(is_bad_pmem(&pmem->bb, sector, len))) {
+ if (bio && bio->bi_opf & REQ_NOWAIT) {
+ bio_wouldblock_error(bio);
+ return BLK_STS_AGAIN;
+ }
blk_status_t rc = pmem_clear_poison(pmem, pmem_off, len);
if (rc != BLK_STS_OK)
@@ -217,7 +221,7 @@ static void pmem_submit_bio(struct bio *bio)
bio_for_each_segment(bvec, bio, iter) {
if (op_is_write(bio_op(bio)))
rc = pmem_do_write(pmem, bvec.bv_page,
bvec.bv_offset,
- iter.bi_sector, bvec.bv_len); + iter.bi_sector, bvec.bv_len, bio);
else
rc = pmem_do_read(pmem, bvec.bv_page,
bvec.bv_offset,
iter.bi_sector, bvec.bv_len);
@@ -297,7 +301,7 @@ static int pmem_dax_zero_page_range(struct dax_device *dax_dev, pgoff_t pgoff,
return blk_status_to_errno(pmem_do_write(pmem, ZERO_PAGE(0), 0, PFN_PHYS(pgoff) >> SECTOR_SHIFT,
- PAGE_SIZE));
+ PAGE_SIZE, NULL));
}
static long pmem_dax_direct_access(struct dax_device *dax_dev,
Quoted 7 lines
Quoted 3 lines
Quoted 2 lines
Is the I/O expected to be performed
asynchronously?Not nessecarily if it is fast enough..
Clear as mud. :) It's a memcpy on potentially "slow" memory. So, the amount of time spent copying depends on the speed of the cpu, the media and the size of the I/O. I don't know off-hand what the upper bound would be on today's pmem.
Above scenario depends on the system and I'm not sure if we can generalize this
for all the deployments. In case we end up generalizing above scenario then we
can always add a modparam to disable nowait so user has total control whether
to enable or disable this an incremental patch ..
For the deployments those are not suffering from the "mempcy on potentially slow memory" this patch shows clear performance win with enabling NOWAIT for io_uring ..
-ck
Given that pmem simply loops over an arbitrarily large bio I think we also need a threshold for which to allow nowait I/O. While it won't block for giant I/Os, doing all of them in the submitter context isn't exactly the idea behind the nowait I/O.
I'm not really sure what a good theshold would be, though.
Btw, please also always add linux-block to the Cc list for block driver patches that are even the slightest bit about the block layer interface.
Quoted 6 lines · Christoph Hellwig
Given that pmem simply loops over an arbitrarily large bio I think we also need a threshold for which to allow nowait I/O. While it won't block for giant I/Os, doing all of them in the submitter context isn't exactly the idea behind the nowait I/O.
I'm not really sure what a good theshold would be, though.
There's no mention of the latency of the submission side in the documentation for RWF_NOWAIT. The man page says "Do not wait for data which is not immediately available." Data in pmem *is* immediately available. If we wanted to enforce a latency threshold for submission, it would have to be configurable and, ideally, a part of the API. I don't think it's something we should even try to guarantee, though, unless application writers are asking for it.
So, I think with the change to return -EAGAIN for writes to poisoned memory, this patch is probably ok.
Chaitanya, could you send a v2?
Thanks,
Jeff
Jens Axboe
maintainerQuoted 4 lines · Christoph Hellwig
Given that pmem simply loops over an arbitrarily large bio I think we also need a threshold for which to allow nowait I/O. While it won't block for giant I/Os, doing all of them in the submitter context isn't exactly the idea behind the nowait I/O.
You can do a LOT of looping over a giant bio and still come out way ahead compared to needing to punt to a different thread. So I do think it's the right choice. But I'm making assumptions here on what it looks like, as I haven't seen the patch...
Quoted 3 lines
Btw, please also always add linux-block to the Cc list for block driver patches that are even the slightest bit about the block layer interface.
Indeed. Particularly for these nowait changes, as some of them have been pretty broken in the past.
Signature · 1 line
Jens Axboe
Quoted 4 lines · Jens Axboe
You can do a LOT of looping over a giant bio and still come out way ahead compared to needing to punt to a different thread. So I do think it's the right choice. But I'm making assumptions here on what it looks like, as I haven't seen the patch...
"a LOT" is relative. A GB or two will block the submission thread for quite a while.
Quoted 10 lines · Christoph Hellwig
Given that pmem simply loops over an arbitrarily large bio I think we also need a threshold for which to allow nowait I/O. While it won't block for giant I/Os, doing all of them in the submitter context isn't exactly the idea behind the nowait I/O.
I'm not really sure what a good theshold would be, though.
Btw, please also always add linux-block to the Cc list for block driver patches that are even the slightest bit about the block layer interface.
will do ..
-ck
Quoted 10 lines · Jens Axboe
Quoted 4 lines · Christoph Hellwig
Given that pmem simply loops over an arbitrarily large bio I think we also need a threshold for which to allow nowait I/O. While it won't block for giant I/Os, doing all of them in the submitter context isn't exactly the idea behind the nowait I/O.
You can do a LOT of looping over a giant bio and still come out way ahead compared to needing to punt to a different thread. So I do think it's the right choice. But I'm making assumptions here on what it looks like, as I haven't seen the patch...
will send out the V3 soon and CC you and linux-block ...
Quoted 6 lines
Quoted 3 lines
Btw, please also always add linux-block to the Cc list for block driver patches that are even the slightest bit about the block layer interface.
Indeed. Particularly for these nowait changes, as some of them have been pretty broken in the past.
yes, didn't forgot, lately dealing with bunch of stupidity, I'll send zram and null_blk nowait changes soon with your comments fixed ..
-ck
Quoted 7 lines · Jeff Moyer
Quoted 5 lines · Christoph Hellwig
Given that pmem simply loops over an arbitrarily large bio I think we also need a threshold for which to allow nowait I/O. While it won't block for giant I/Os, doing all of them in the submitter context isn't exactly the idea behind the nowait I/O.
Quoted 8 lines
Quoted 1 line
I'm not really sure what a good theshold would be, though.
There's no mention of the latency of the submission side in the documentation for
RWF_NOWAIT. The man page says "Do not wait for data which is not immediately available." Data in pmem *is* immediately available. If we wanted to enforce a latency threshold for submission, it would have to be configurable and, ideally, a part of the API. I don't think it's something we should even try to guarantee, though, unless application writers are asking for it.
I need to see the usecase from application writter who cannot come with a solution so kernel have to provide this interface, I'll be happy to do that ..
Quoted 2 lines
So, I think with the change to return
-EAGAINfor writes to poisoned memory, this patch is probably ok.
I believe you mean the same one I've provided earlier incremental ..
Quoted 1 line
Chaitanya, could you send a v2?
sure ...
-ck
Quoted 5 lines · Chaitanya Kulkarni
Quoted 2 lines · Jeff Moyer
So, I think with the change to return
-EAGAINfor writes to poisoned memory, this patch is probably ok.I believe you mean the same one I've provided earlier incremental ..
Yes, sorry if that wasn't clear.
Quoted 3 lines
Quoted 1 line
Chaitanya, could you send a v2?
sure ...
and I guess I should have said v3. ;-)
Cheers,
Jeff