Skip to contentSkip to conversation

pmem: set QUEUE_FLAG_NOWAIT

PATCH In review Chaitanya Kulkarni 5 participants 14 messages Jul 31, 2023 – Aug 3, 2023

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

v2 cover pmem: set QUEUE_FLAG_NOWAIT

Hi,

Set the QUEUE_FLAG_NOWAIT. Following are the performance numbers with io_uring fio engine for random read, note that device has been populated fully with randwrite workload before taking these numbers :-

linux-block (pmem-nowait-on) # grep IOPS pmem*fio | column -t

pmem-nowait-off-1.fio:  read:  IOPS=3683k,  BW=14.0GiB/s
pmem-nowait-off-2.fio:  read:  IOPS=3819k,  BW=14.6GiB/s
pmem-nowait-off-3.fio:  read:  IOPS=3999k,  BW=15.3GiB/s
pmem-nowait-on-1.fio:   read:  IOPS=5837k,  BW=22.3GiB/s
pmem-nowait-on-2.fio:   read:  IOPS=5936k,  BW=22.6GiB/s
pmem-nowait-on-3.fio:   read:  IOPS=5945k,  BW=22.7GiB/s

linux-block (pmem-nowait-on) # grep cpu pmem*fio | column -t

pmem-nowait-off-1.fio:  cpu  :  usr=7.09%,   sys=29.65%,  ctx=198742065
pmem-nowait-off-2.fio:  cpu  :  usr=6.89%,   sys=30.56%,  ctx=205817652
pmem-nowait-off-3.fio:  cpu  :  usr=6.86%,   sys=30.94%,  ctx=222627094
pmem-nowait-on-1.fio:   cpu  :  usr=10.58%,  sys=88.44%,  ctx=27181   
pmem-nowait-on-2.fio:   cpu  :  usr=10.50%,  sys=87.75%,  ctx=25746   
pmem-nowait-on-3.fio:   cpu  :  usr=10.67%,  sys=88.60%,  ctx=28261   

linux-block (pmem-nowait-on) # grep slat pmem*fio | column -t

pmem-nowait-off-1.fio:  slat  (nsec):  min=432,   max=50847k,  avg=9324.69
pmem-nowait-off-2.fio:  slat  (nsec):  min=441,   max=52557k,  avg=9132.45
pmem-nowait-off-3.fio:  slat  (nsec):  min=430,   max=36113k,  avg=9132.63
pmem-nowait-on-1.fio:   slat  (nsec):  min=1393,  max=68090k,  avg=7615.31
pmem-nowait-on-2.fio:   slat  (nsec):  min=1222,  max=44137k,  avg=7493.77
pmem-nowait-on-3.fio:   slat  (nsec):  min=1493,  max=40100k,  avg=7486.36

Please let me know if further testing is needed I've ran fio verification job in order to make verify these changes.

-ck

V2:-

Unconditionally set the QUEUE_FLAG_NOWAIT in pmem_attach_disk() along with the other queue flags.

Chaitanya Kulkarni (1):
pmem: set QUEUE_FLAG_NOWAIT

 drivers/nvdimm/pmem.c | 1 +
 1 file changed, 1 insertion(+)

linux-block (pmem-nowait-on) # ./test-pmem.sh
++ unload_mod
++ rmmod nd_pmem
++ rmmod nd_btt
++ git checkout for-next
Switched to branch 'for-next'
Your branch is ahead of 'origin/for-next' by 155 commits.
(use "git push" to publish your local commits)
++ git log -1
commit e50c5e801b5a9e1797eb5a157eac1b5e50084486 (HEAD -> for-next)
Merge: e6dfe861227b e98acd815ebf
Author: Chaitanya Kulkarni <[email protected]>
Date: Mon Jul 31 14:48:39 2023 -0700

Merge branch 'for-next' of git://git.kernel.dk/linux-block into for-next
++ makej M=drivers/nvdimm
CC [M] drivers/nvdimm/pmem.o
LD [M] drivers/nvdimm/nd_pmem.o
MODPOST drivers/nvdimm/Module.symvers
LD [M] drivers/nvdimm/nd_pmem.ko
++ load_mod
++ insmod drivers/nvdimm/nd_btt.ko
++ insmod drivers/nvdimm/nd_pmem.ko
++ sleep 1
++ test_pmem nowait-off
++ sleep 1
++ fio fio/verify.fio --ioengine=io_uring --size=896M --filename=/dev/pmem0
write-and-verify: (g=0): rw=randwrite, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=io_uring, iodepth=16 fio-3.34
Starting 1 process
Jobs: 1 (f=1)
write-and-verify: (groupid=0, jobs=1): err= 0: pid=4662: Mon Jul 31 15:24:07 2023
read: IOPS=265k, BW=1036MiB/s (1087MB/s)(566MiB/546msec)

    slat (nsec): min=501, max=31820, avg=2733.20, stdev=1179.52
    clat (nsec): min=17022, max=96063, avg=56544.09, stdev=5848.22
     lat (usec): min=20, max=101, avg=59.28, stdev= 6.06
    clat percentiles (nsec):
     |  1.00th=[44288],  5.00th=[46848], 10.00th=[48896], 20.00th=[51456],
     | 30.00th=[53504], 40.00th=[55552], 50.00th=[56576], 60.00th=[58112],
     | 70.00th=[59648], 80.00th=[61184], 90.00th=[63744], 95.00th=[66048],
     | 99.00th=[72192], 99.50th=[74240], 99.90th=[80384], 99.95th=[82432],
     | 99.99th=[88576]
  write: IOPS=209k, BW=818MiB/s (857MB/s)(896MiB/1096msec); 0 zone resets
    slat (nsec): min=1352, max=113484, avg=4293.77, stdev=1425.01
    clat (usec): min=25, max=285, avg=71.81, stdev= 9.33
     lat (usec): min=31, max=288, avg=76.10, stdev= 9.56
    clat percentiles (usec):
     |  1.00th=[   45],  5.00th=[   61], 10.00th=[   65], 20.00th=[   68],
     | 30.00th=[   70], 40.00th=[   71], 50.00th=[   72], 60.00th=[   73],
     | 70.00th=[   75], 80.00th=[   77], 90.00th=[   80], 95.00th=[   84],
     | 99.00th=[  102], 99.50th=[  113], 99.90th=[  169], 99.95th=[  180],
     | 99.99th=[  219]
   bw (  KiB/s): min=152408, max=857568, per=73.07%, avg=611669.33, stdev=398064.54, samples=3
   iops        : min=38102, max=214392, avg=152917.33, stdev=99516.13, samples=3
  lat (usec)   : 20=0.01%, 50=6.40%, 100=92.93%, 250=0.67%, 500=0.01%
  cpu          : usr=35.49%, sys=63.60%, ctx=2561, majf=0, minf=3973
  IO depths    : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=100.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.1%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=144875,229376,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16

Run status group 0 (all jobs):
READ: bw=1036MiB/s (1087MB/s), 1036MiB/s-1036MiB/s (1087MB/s-1087MB/s), io=566MiB (593MB), run=546-546msec
WRITE: bw=818MiB/s (857MB/s), 818MiB/s-818MiB/s (857MB/s-857MB/s), io=896MiB (940MB), run=1096-1096msec

Disk stats (read/write):
pmem0: ios=0/0, merge=0/0, ticks=0/0, in_queue=0, util=0.00%
++ fio fio/randwrite.fio --ioengine=io_uring --size=896M --filename=/dev/pmem0
RANDWRITE: (g=0): rw=randwrite, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=io_uring, iodepth=2
...
fio-3.34
Starting 48 processes
Jobs: 48 (f=48): [w(48)][100.0%][w=20.6GiB/s][w=5389k IOPS][eta 00m:00s]
RANDWRITE: (groupid=0, jobs=48): err= 0: pid=4681: Mon Jul 31 15:25:07 2023
write: IOPS=5147k, BW=19.6GiB/s (21.1GB/s)(1178GiB/60002msec); 0 zone resets

    slat (nsec): min=380, max=57147k, avg=6895.36, stdev=28357.20
    clat (nsec): min=130, max=57173k, avg=11233.63, stdev=44260.60
     lat (nsec): min=1944, max=57174k, avg=18128.99, stdev=53168.60
    clat percentiles (usec):
     |  1.00th=[    3],  5.00th=[    6], 10.00th=[    6], 20.00th=[    6],
     | 30.00th=[    7], 40.00th=[    7], 50.00th=[    8], 60.00th=[    9],
     | 70.00th=[   10], 80.00th=[   12], 90.00th=[   21], 95.00th=[   32],
     | 99.00th=[   58], 99.50th=[   74], 99.90th=[  135], 99.95th=[  186],
     | 99.99th=[  742]
   bw (  MiB/s): min= 5793, max=30898, per=100.00%, avg=20121.35, stdev=146.64, samples=5712
   iops        : min=1483165, max=7909996, avg=5151064.04, stdev=37540.43, samples=5712
  lat (nsec)   : 250=0.01%, 500=0.01%, 750=0.01%, 1000=0.01%
  lat (usec)   : 2=0.33%, 4=2.22%, 10=71.71%, 20=15.22%, 50=9.07%
  lat (usec)   : 100=1.23%, 250=0.18%, 500=0.02%, 750=0.01%, 1000=0.01%
  lat (msec)   : 2=0.01%, 4=0.01%, 10=0.01%, 20=0.01%, 50=0.01%
  lat (msec)   : 100=0.01%
  cpu          : usr=9.36%, sys=31.21%, ctx=286817971, majf=0, minf=607
  IO depths    : 1=0.1%, 2=100.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=0,308833813,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=2

Run status group 0 (all jobs):
WRITE: bw=19.6GiB/s (21.1GB/s), 19.6GiB/s-19.6GiB/s (21.1GB/s-21.1GB/s), io=1178GiB (1265GB), run=60002-60002msec

Disk stats (read/write):
pmem0: ios=0/0, merge=0/0, ticks=0/0, in_queue=0, util=0.00%
++ for i in 1 2 3
++ fio fio/randread.fio --ioengine=io_uring --size=896M --filename=/dev/pmem0 --output=pmem-nowait-off-1.fio ++ for i in 1 2 3[r(48)][100.0%][r=14.7GiB/s][r=3864k IOPS][eta 00m:00s]
++ fio fio/randread.fio --ioengine=io_uring --size=896M --filename=/dev/pmem0 --output=pmem-nowait-off-2.fio ++ for i in 1 2 3[r(48)][100.0%][r=15.7GiB/s][r=4116k IOPS][eta 00m:00s]
++ fio fio/randread.fio --ioengine=io_uring --size=896M --filename=/dev/pmem0 --output=pmem-nowait-off-3.fio ++ unload_mod8): [r(48)][100.0%][r=15.1GiB/s][r=3966k IOPS][eta 00m:00s]
++ rmmod nd_pmem
++ rmmod nd_btt
++ git checkout pmem-nowait-on
Switched to branch 'pmem-nowait-on'
++ git log -1
commit 24573494ec05e8d7bb7acb82e4a0e400297272aa (HEAD -> pmem-nowait-on)

Author: Chaitanya Kulkarni <[email protected]>
Date:   Fri May 12 03:24:54 2023 -0700

pmem: set QUEUE_FLAG_NOWAIT

Set the QUEUE_FLAG_NOWAIT. Following are the performance numbers with io_uring fio engine for random read, note that device has been populated fully with randwrite workload before taking these numbers :-

++ rmmod nd_pmem
rmmod: ERROR: Module nd_pmem is not currently loaded
++ makej M=drivers/nvdimm
CC [M] drivers/nvdimm/pmem.o
LD [M] drivers/nvdimm/nd_pmem.o
MODPOST drivers/nvdimm/Module.symvers
LD [M] drivers/nvdimm/nd_pmem.ko
++ load_mod
++ insmod drivers/nvdimm/nd_btt.ko
++ insmod drivers/nvdimm/nd_pmem.ko
++ sleep 1
++ test_pmem nowait-on
++ sleep 1
++ fio fio/verify.fio --ioengine=io_uring --size=896M --filename=/dev/pmem0
write-and-verify: (g=0): rw=randwrite, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=io_uring, iodepth=16 fio-3.34
Starting 1 process
Jobs: 1 (f=1)
write-and-verify: (groupid=0, jobs=1): err= 0: pid=5662: Mon Jul 31 15:28:13 2023
read: IOPS=448k, BW=1750MiB/s (1835MB/s)(567MiB/324msec)

    slat (nsec): min=1092, max=56347, avg=1349.16, stdev=838.70
    clat (nsec): min=741, max=152570, avg=33482.76, stdev=9895.86
     lat (usec): min=2, max=154, avg=34.83, stdev=10.20
    clat percentiles (usec):
     |  1.00th=[   30],  5.00th=[   31], 10.00th=[   31], 20.00th=[   31],
     | 30.00th=[   31], 40.00th=[   31], 50.00th=[   32], 60.00th=[   32],
     | 70.00th=[   32], 80.00th=[   32], 90.00th=[   35], 95.00th=[   49],
     | 99.00th=[   87], 99.50th=[   96], 99.90th=[  113], 99.95th=[  122],
     | 99.99th=[  139]
  write: IOPS=207k, BW=810MiB/s (849MB/s)(896MiB/1106msec); 0 zone resets
    slat (nsec): min=2135, max=81445, avg=4433.09, stdev=1568.20
    clat (nsec): min=441, max=210380, avg=72445.60, stdev=10663.97
     lat (usec): min=3, max=254, avg=76.88, stdev=11.22
    clat percentiles (usec):
     |  1.00th=[   56],  5.00th=[   60], 10.00th=[   62], 20.00th=[   65],
     | 30.00th=[   68], 40.00th=[   70], 50.00th=[   72], 60.00th=[   75],
     | 70.00th=[   77], 80.00th=[   79], 90.00th=[   83], 95.00th=[   86],
     | 99.00th=[  120], 99.50th=[  135], 99.90th=[  161], 99.95th=[  169],
     | 99.99th=[  192]
   bw (  KiB/s): min=764336, max=876176, per=98.88%, avg=820256.00, stdev=79082.82, samples=2
   iops        : min=191084, max=219044, avg=205064.00, stdev=19770.71, samples=2
  lat (nsec)   : 500=0.01%, 750=0.01%
  lat (usec)   : 4=0.01%, 10=0.01%, 20=0.01%, 50=36.90%, 100=62.08%
  lat (usec)   : 250=1.01%
  cpu          : usr=37.58%, sys=62.28%, ctx=3, majf=0, minf=3983
  IO depths    : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=100.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.1%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=145180,229376,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16

Run status group 0 (all jobs):
READ: bw=1750MiB/s (1835MB/s), 1750MiB/s-1750MiB/s (1835MB/s-1835MB/s), io=567MiB (595MB), run=324-324msec
WRITE: bw=810MiB/s (849MB/s), 810MiB/s-810MiB/s (849MB/s-849MB/s), io=896MiB (940MB), run=1106-1106msec

Disk stats (read/write):
pmem0: ios=0/0, merge=0/0, ticks=0/0, in_queue=0, util=0.00%
++ fio fio/randwrite.fio --ioengine=io_uring --size=896M --filename=/dev/pmem0
RANDWRITE: (g=0): rw=randwrite, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=io_uring, iodepth=2
...
fio-3.34
Starting 48 processes
Jobs: 48 (f=48): [w(48)][100.0%][w=39.7GiB/s][w=10.4M IOPS][eta 00m:00s]
RANDWRITE: (groupid=0, jobs=48): err= 0: pid=5665: Mon Jul 31 15:29:13 2023
write: IOPS=9926k, BW=37.9GiB/s (40.7GB/s)(2272GiB/60001msec); 0 zone resets

    slat (nsec): min=1192, max=32084k, avg=4269.53, stdev=21566.61
    clat (nsec): min=501, max=32087k, avg=5022.97, stdev=23279.45
     lat (nsec): min=1854, max=32098k, avg=9292.50, stdev=31761.04
    clat percentiles (nsec):
     |  1.00th=[ 2544],  5.00th=[ 2992], 10.00th=[ 3280], 20.00th=[ 3600],
     | 30.00th=[ 3920], 40.00th=[ 4256], 50.00th=[ 4576], 60.00th=[ 4960],
     | 70.00th=[ 5408], 80.00th=[ 5984], 90.00th=[ 7008], 95.00th=[ 7968],
     | 99.00th=[10944], 99.50th=[14528], 99.90th=[24192], 99.95th=[33536],
     | 99.99th=[63744]
   bw (  MiB/s): min=26941, max=42586, per=100.00%, avg=38794.41, stdev=73.04, samples=5712
   iops        : min=6897082, max=10902168, avg=9931366.71, stdev=18699.20, samples=5712
  lat (nsec)   : 750=0.01%, 1000=0.01%
  lat (usec)   : 2=0.01%, 4=32.93%, 10=65.63%, 20=1.26%, 50=0.16%
  lat (usec)   : 100=0.01%, 250=0.01%, 500=0.01%, 750=0.01%, 1000=0.01%
  lat (msec)   : 2=0.01%, 4=0.01%, 10=0.01%, 20=0.01%, 50=0.01%
  cpu          : usr=18.08%, sys=80.64%, ctx=30470, majf=0, minf=587
  IO depths    : 1=0.1%, 2=100.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=0,595577284,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=2

Run status group 0 (all jobs):
WRITE: bw=37.9GiB/s (40.7GB/s), 37.9GiB/s-37.9GiB/s (40.7GB/s-40.7GB/s), io=2272GiB (2439GB), run=60001-60001msec

Disk stats (read/write):
pmem0: ios=0/0, merge=0/0, ticks=0/0, in_queue=0, util=0.00%
++ for i in 1 2 3
++ fio fio/randread.fio --ioengine=io_uring --size=896M --filename=/dev/pmem0 --output=pmem-nowait-on-1.fio ++ for i in 1 2 3[r(48)][100.0%][r=22.7GiB/s][r=5960k IOPS][eta 00m:00s]
++ fio fio/randread.fio --ioengine=io_uring --size=896M --filename=/dev/pmem0 --output=pmem-nowait-on-2.fio ++ for i in 1 2 3[r(48)][100.0%][r=22.7GiB/s][r=5953k IOPS][eta 00m:00s]
++ fio fio/randread.fio --ioengine=io_uring --size=896M --filename=/dev/pmem0 --output=pmem-nowait-on-3.fio linux-block (pmem-nowait-on) # ][r=22.7GiB/s][r=5946k IOPS][eta 00m:00s]
linux-block (pmem-nowait-on) #
linux-block (pmem-nowait-on) # grep IOPS pmem*fio | column -t

pmem-nowait-off-1.fio:  read:  IOPS=3683k,  BW=14.0GiB/s  (15.1GB/s)(843GiB/60002msec)
pmem-nowait-off-2.fio:  read:  IOPS=3819k,  BW=14.6GiB/s  (15.6GB/s)(874GiB/60002msec)
pmem-nowait-off-3.fio:  read:  IOPS=3999k,  BW=15.3GiB/s  (16.4GB/s)(915GiB/60002msec)
pmem-nowait-on-1.fio:   read:  IOPS=5837k,  BW=22.3GiB/s  (23.9GB/s)(1336GiB/60002msec)
pmem-nowait-on-2.fio:   read:  IOPS=5936k,  BW=22.6GiB/s  (24.3GB/s)(1359GiB/60002msec)
pmem-nowait-on-3.fio:   read:  IOPS=5945k,  BW=22.7GiB/s  (24.3GB/s)(1361GiB/60002msec)

linux-block (pmem-nowait-on) # grep cpu pmem*fio | column -t

pmem-nowait-off-1.fio:  cpu  :  usr=7.09%,   sys=29.65%,  ctx=198742065
pmem-nowait-off-2.fio:  cpu  :  usr=6.89%,   sys=30.56%,  ctx=205817652
pmem-nowait-off-3.fio:  cpu  :  usr=6.86%,   sys=30.94%,  ctx=222627094
pmem-nowait-on-1.fio:   cpu  :  usr=10.58%,  sys=88.44%,  ctx=27181,   
pmem-nowait-on-2.fio:   cpu  :  usr=10.50%,  sys=87.75%,  ctx=25746,   
pmem-nowait-on-3.fio:   cpu  :  usr=10.67%,  sys=88.60%,  ctx=28261,   
linux-block (pmem-nowait-on) # grep slat  pmem*fio | column -t
pmem-nowait-off-1.fio:  slat  (nsec):  min=432,   max=50847k,  avg=9324.69
pmem-nowait-off-2.fio:  slat  (nsec):  min=441,   max=52557k,  avg=9132.45
pmem-nowait-off-3.fio:  slat  (nsec):  min=430,   max=36113k,  avg=9132.63
pmem-nowait-on-1.fio:   slat  (nsec):  min=1393,  max=68090k,  avg=7615.31
pmem-nowait-on-2.fio:   slat  (nsec):  min=1222,  max=44137k,  avg=7493.77
pmem-nowait-on-3.fio:   slat  (nsec):  min=1493,  max=40100k,  avg=7486.36

linux-block (pmem-nowait-on) #

Signature · 1 line
2.40.0
#1 lore raw

PATCH v2 1/1 pmem: set QUEUE_FLAG_NOWAIT

Set the QUEUE_FLAG_NOWAIT. Following are the performance numbers with io_uring fio engine for random read, note that device has been populated fully with randwrite workload before taking these numbers :-

linux-block (pmem-nowait-on) # grep IOPS pmem*fio | column -t

pmem-nowait-off-1.fio:  read:  IOPS=3683k,  BW=14.0GiB/s
pmem-nowait-off-2.fio:  read:  IOPS=3819k,  BW=14.6GiB/s
pmem-nowait-off-3.fio:  read:  IOPS=3999k,  BW=15.3GiB/s
pmem-nowait-on-1.fio:   read:  IOPS=5837k,  BW=22.3GiB/s
pmem-nowait-on-2.fio:   read:  IOPS=5936k,  BW=22.6GiB/s
pmem-nowait-on-3.fio:   read:  IOPS=5945k,  BW=22.7GiB/s

linux-block (pmem-nowait-on) # grep cpu pmem*fio | column -t

pmem-nowait-off-1.fio:  cpu  :  usr=7.09%,   sys=29.65%,  ctx=198742065
pmem-nowait-off-2.fio:  cpu  :  usr=6.89%,   sys=30.56%,  ctx=205817652
pmem-nowait-off-3.fio:  cpu  :  usr=6.86%,   sys=30.94%,  ctx=222627094
pmem-nowait-on-1.fio:   cpu  :  usr=10.58%,  sys=88.44%,  ctx=27181   
pmem-nowait-on-2.fio:   cpu  :  usr=10.50%,  sys=87.75%,  ctx=25746   
pmem-nowait-on-3.fio:   cpu  :  usr=10.67%,  sys=88.60%,  ctx=28261   

linux-block (pmem-nowait-on) # grep slat pmem*fio | column -t

pmem-nowait-off-1.fio:  slat  (nsec):  min=432,   max=50847k,  avg=9324.69
pmem-nowait-off-2.fio:  slat  (nsec):  min=441,   max=52557k,  avg=9132.45
pmem-nowait-off-3.fio:  slat  (nsec):  min=430,   max=36113k,  avg=9132.63
pmem-nowait-on-1.fio:   slat  (nsec):  min=1393,  max=68090k,  avg=7615.31
pmem-nowait-on-2.fio:   slat  (nsec):  min=1222,  max=44137k,  avg=7493.77
pmem-nowait-on-3.fio:   slat  (nsec):  min=1493,  max=40100k,  avg=7486.36
---
 drivers/nvdimm/pmem.c | 1 +
 1 file changed, 1 insertion(+)
drivers/nvdimm/pmem.c +1 -0
index 46e094e56159..ddd485c377eb 100644
@@ -543,6 +543,7 @@ static int pmem_attach_disk(struct device *dev,
 	blk_queue_max_hw_sectors(q, UINT_MAX);
 	blk_queue_flag_set(QUEUE_FLAG_NONROT, q);
 	blk_queue_flag_set(QUEUE_FLAG_SYNCHRONOUS, q);
+	blk_queue_flag_set(QUEUE_FLAG_NOWAIT, q);
 	if (pmem->pfn_flags & PFN_MAP)
 		blk_queue_flag_set(QUEUE_FLAG_DAX, q);
 
Signature · 1 line
2.40.0
#2 lore raw
Quoted 3 lines · Chaitanya Kulkarni

Set the QUEUE_FLAG_NOWAIT. Following are the performance numbers with io_uring fio engine for random read, note that device has been populated fully with randwrite workload before taking these numbers :-

I am slightly embarrassed to have to ask this question, but what are the implications of setting this queue flag? Is the submit_bio routine expected to never block? Is the I/O expected to be performed asynchronously? If either of those is the case, then pmem does not qualify.

-Jeff

Quoted 44 lines

linux-block (pmem-nowait-on) # grep IOPS pmem*fio | column -t

pmem-nowait-off-1.fio:  read:  IOPS=3683k,  BW=14.0GiB/s
pmem-nowait-off-2.fio:  read:  IOPS=3819k,  BW=14.6GiB/s
pmem-nowait-off-3.fio:  read:  IOPS=3999k,  BW=15.3GiB/s
pmem-nowait-on-1.fio:   read:  IOPS=5837k,  BW=22.3GiB/s
pmem-nowait-on-2.fio:   read:  IOPS=5936k,  BW=22.6GiB/s
pmem-nowait-on-3.fio:   read:  IOPS=5945k,  BW=22.7GiB/s

linux-block (pmem-nowait-on) # grep cpu pmem*fio | column -t

pmem-nowait-off-1.fio:  cpu  :  usr=7.09%,   sys=29.65%,  ctx=198742065
pmem-nowait-off-2.fio:  cpu  :  usr=6.89%,   sys=30.56%,  ctx=205817652
pmem-nowait-off-3.fio:  cpu  :  usr=6.86%,   sys=30.94%,  ctx=222627094
pmem-nowait-on-1.fio:   cpu  :  usr=10.58%,  sys=88.44%,  ctx=27181   
pmem-nowait-on-2.fio:   cpu  :  usr=10.50%,  sys=87.75%,  ctx=25746   
pmem-nowait-on-3.fio:   cpu  :  usr=10.67%,  sys=88.60%,  ctx=28261   

linux-block (pmem-nowait-on) # grep slat pmem*fio | column -t

pmem-nowait-off-1.fio:  slat  (nsec):  min=432,   max=50847k,  avg=9324.69
pmem-nowait-off-2.fio:  slat  (nsec):  min=441,   max=52557k,  avg=9132.45
pmem-nowait-off-3.fio:  slat  (nsec):  min=430,   max=36113k,  avg=9132.63
pmem-nowait-on-1.fio:   slat  (nsec):  min=1393,  max=68090k,  avg=7615.31
pmem-nowait-on-2.fio:   slat  (nsec):  min=1222,  max=44137k,  avg=7493.77
pmem-nowait-on-3.fio:   slat  (nsec):  min=1493,  max=40100k,  avg=7486.36
---
 drivers/nvdimm/pmem.c | 1 +
 1 file changed, 1 insertion(+)
drivers/nvdimm/pmem.c +1 -0
index 46e094e56159..ddd485c377eb 100644
@@ -543,6 +543,7 @@ static int pmem_attach_disk(struct device *dev,
 	blk_queue_max_hw_sectors(q, UINT_MAX);
 	blk_queue_flag_set(QUEUE_FLAG_NONROT, q);
 	blk_queue_flag_set(QUEUE_FLAG_SYNCHRONOUS, q);
+	blk_queue_flag_set(QUEUE_FLAG_NOWAIT, q);
 	if (pmem->pfn_flags & PFN_MAP)
 		blk_queue_flag_set(QUEUE_FLAG_DAX, q);
#3 lore raw
Quoted 3 lines · Jeff Moyer

I am slightly embarrassed to have to ask this question, but what are the implications of setting this queue flag? Is the submit_bio routine expected to never block?

Yes, at least not significantly.

Quoted 2 lines

Is the I/O expected to be performed
asynchronously?

Not nessecarily if it is fast enough..

#4 lore raw

Hi, Christoph,

Quoted 6 lines · Christoph Hellwig
Quoted 3 lines · Jeff Moyer

I am slightly embarrassed to have to ask this question, but what are the implications of setting this queue flag? Is the submit_bio routine expected to never block?

Yes, at least not significantly.

If there happens to be poisoned memory, the write path can make an acpi device specific method call. That involves taking a mutex (see the call chain starting at acpi_ex_enter_interpreter()). I'm not sure how long a DSM takes, but I doubt it's fast.

Quoted 4 lines
Quoted 2 lines

Is the I/O expected to be performed
asynchronously?

Not nessecarily if it is fast enough..

Clear as mud. :) It's a memcpy on potentially "slow" memory. So, the amount of time spent copying depends on the speed of the cpu, the media and the size of the I/O. I don't know off-hand what the upper bound would be on today's pmem.

-Jeff

#5 lore raw
Quoted 5 lines · Jeff Moyer

Hi, Christoph,

Quoted 1 line · Christoph Hellwig

On Tue, Aug 01, 2023 at 11:23:36AM -0400, Jeff Moyer wrote:

Quoted 8 lines
Quoted 4 lines
Quoted 3 lines

I am slightly embarrassed to have to ask this question, but what are the implications of setting this queue flag? Is the submit_bio routine expected to never block?

Yes, at least not significantly.

If there happens to be poisoned memory, the write path can make an acpi device specific method call. That involves taking a mutex (see the call chain starting at acpi_ex_enter_interpreter()). I'm not sure how long a DSM takes, but I doubt it's fast.

for this case I can add bio->bio_opf & REQ_NOWAIT check and return with bio_wouldblock_error() that will take care of blocking case e.g. something like this untested as a prep patch for write path :-

linux-block (pmem-nowait-on) # git diff

drivers/nvdimm/pmem.c +0 -0
index ddd485c377eb..eff100f74357 100644
@@ -179,12 +179,16 @@ static blk_status_t pmem_do_read(struct 

pmem_device *pmem,

 static blk_status_t pmem_do_write(struct pmem_device *pmem, struct page *page, unsigned int page_off,
-                       sector_t sector, unsigned int len)
+                       sector_t sector, unsigned int len, struct bio *bio)
 {
        phys_addr_t pmem_off = to_offset(pmem, sector);
        void *pmem_addr = pmem->virt_addr + pmem_off;

        if (unlikely(is_bad_pmem(&pmem->bb, sector, len))) {
+               if (bio && bio->bi_opf & REQ_NOWAIT) {
+                       bio_wouldblock_error(bio);
+                       return BLK_STS_AGAIN;
+               }
                blk_status_t rc = pmem_clear_poison(pmem, pmem_off, len);

                if (rc != BLK_STS_OK)
@@ -217,7 +221,7 @@ static void pmem_submit_bio(struct bio *bio)
        bio_for_each_segment(bvec, bio, iter) {
                if (op_is_write(bio_op(bio)))
                        rc = pmem_do_write(pmem, bvec.bv_page,
bvec.bv_offset,
-                               iter.bi_sector, bvec.bv_len); +                               iter.bi_sector, bvec.bv_len, bio);
                else
                        rc = pmem_do_read(pmem, bvec.bv_page,
bvec.bv_offset,
                                iter.bi_sector, bvec.bv_len);
@@ -297,7 +301,7 @@ static int pmem_dax_zero_page_range(struct dax_device *dax_dev, pgoff_t pgoff,

        return blk_status_to_errno(pmem_do_write(pmem, ZERO_PAGE(0), 0, PFN_PHYS(pgoff) >> SECTOR_SHIFT,
-                                  PAGE_SIZE));
+                                  PAGE_SIZE, NULL));
 }

 static long pmem_dax_direct_access(struct dax_device *dax_dev,

Quoted 7 lines
Quoted 3 lines
Quoted 2 lines

Is the I/O expected to be performed
asynchronously?

Not nessecarily if it is fast enough..

Clear as mud. :) It's a memcpy on potentially "slow" memory. So, the amount of time spent copying depends on the speed of the cpu, the media and the size of the I/O. I don't know off-hand what the upper bound would be on today's pmem.

Above scenario depends on the system and I'm not sure if we can generalize this
for all the deployments. In case we end up generalizing above scenario then we
can always add a modparam to disable nowait so user has total control whether
to enable or disable this an incremental patch ..

For the deployments those are not suffering from the "mempcy on potentially slow memory" this patch shows clear performance win with enabling NOWAIT for io_uring ..

-ck

#6 lore raw

Given that pmem simply loops over an arbitrarily large bio I think we also need a threshold for which to allow nowait I/O. While it won't block for giant I/Os, doing all of them in the submitter context isn't exactly the idea behind the nowait I/O.

I'm not really sure what a good theshold would be, though.

Btw, please also always add linux-block to the Cc list for block driver patches that are even the slightest bit about the block layer interface.

#7 lore raw
Quoted 6 lines · Christoph Hellwig

Given that pmem simply loops over an arbitrarily large bio I think we also need a threshold for which to allow nowait I/O. While it won't block for giant I/Os, doing all of them in the submitter context isn't exactly the idea behind the nowait I/O.

I'm not really sure what a good theshold would be, though.

There's no mention of the latency of the submission side in the documentation for RWF_NOWAIT. The man page says "Do not wait for data which is not immediately available." Data in pmem *is* immediately available. If we wanted to enforce a latency threshold for submission, it would have to be configurable and, ideally, a part of the API. I don't think it's something we should even try to guarantee, though, unless application writers are asking for it.

So, I think with the change to return -EAGAIN for writes to poisoned memory, this patch is probably ok.

Chaitanya, could you send a v2?

Thanks,
Jeff

#8 lore raw
Quoted 7 lines · Jeff Moyer
Quoted 5 lines · Christoph Hellwig

Given that pmem simply loops over an arbitrarily large bio I think we also need a threshold for which to allow nowait I/O. While it won't block for giant I/Os, doing all of them in the submitter context isn't exactly the idea behind the nowait I/O.

Quoted 8 lines
Quoted 1 line

I'm not really sure what a good theshold would be, though.

There's no mention of the latency of the submission side in the documentation for RWF_NOWAIT. The man page says "Do not wait for data which is not immediately available." Data in pmem *is* immediately available. If we wanted to enforce a latency threshold for submission, it would have to be configurable and, ideally, a part of the API. I don't think it's something we should even try to guarantee, though, unless application writers are asking for it.

I need to see the usecase from application writter who cannot come with a solution so kernel have to provide this interface, I'll be happy to do that ..

Quoted 2 lines

So, I think with the change to return -EAGAIN for writes to poisoned memory, this patch is probably ok.

I believe you mean the same one I've  provided earlier incremental ..

Quoted 1 line

Chaitanya, could you send a v2?

sure ...

-ck

#9 lore raw
Quoted 5 lines · Chaitanya Kulkarni
Quoted 2 lines · Jeff Moyer

So, I think with the change to return -EAGAIN for writes to poisoned memory, this patch is probably ok.

I believe you mean the same one I've  provided earlier incremental ..

Yes, sorry if that wasn't clear.

Quoted 3 lines
Quoted 1 line

Chaitanya, could you send a v2?

sure ...

and I guess I should have said v3. ;-)

Cheers,
Jeff

maintainer
#10 lore raw
In reply to Christoph Hellwig
Quoted 4 lines · Christoph Hellwig

Given that pmem simply loops over an arbitrarily large bio I think we also need a threshold for which to allow nowait I/O. While it won't block for giant I/Os, doing all of them in the submitter context isn't exactly the idea behind the nowait I/O.

You can do a LOT of looping over a giant bio and still come out way ahead compared to needing to punt to a different thread. So I do think it's the right choice. But I'm making assumptions here on what it looks like, as I haven't seen the patch...

Quoted 3 lines

Btw, please also always add linux-block to the Cc list for block driver patches that are even the slightest bit about the block layer interface.

Indeed. Particularly for these nowait changes, as some of them have been pretty broken in the past.

Signature · 1 line
Jens Axboe
#11 lore raw
Quoted 4 lines · Jens Axboe

You can do a LOT of looping over a giant bio and still come out way ahead compared to needing to punt to a different thread. So I do think it's the right choice. But I'm making assumptions here on what it looks like, as I haven't seen the patch...

"a LOT" is relative. A GB or two will block the submission thread for quite a while.

#12 lore raw
In reply to Jens Axboe
Quoted 10 lines · Jens Axboe
Quoted 4 lines · Christoph Hellwig

Given that pmem simply loops over an arbitrarily large bio I think we also need a threshold for which to allow nowait I/O. While it won't block for giant I/Os, doing all of them in the submitter context isn't exactly the idea behind the nowait I/O.

You can do a LOT of looping over a giant bio and still come out way ahead compared to needing to punt to a different thread. So I do think it's the right choice. But I'm making assumptions here on what it looks like, as I haven't seen the patch...

will send out the V3 soon and CC you and linux-block ...

Quoted 6 lines
Quoted 3 lines

Btw, please also always add linux-block to the Cc list for block driver patches that are even the slightest bit about the block layer interface.

Indeed. Particularly for these nowait changes, as some of them have been pretty broken in the past.

yes, didn't forgot, lately dealing with bunch of stupidity, I'll send zram and null_blk nowait changes soon with your comments fixed ..

-ck

#13 lore raw
In reply to Christoph Hellwig
Quoted 10 lines · Christoph Hellwig

Given that pmem simply loops over an arbitrarily large bio I think we also need a threshold for which to allow nowait I/O. While it won't block for giant I/Os, doing all of them in the submitter context isn't exactly the idea behind the nowait I/O.

I'm not really sure what a good theshold would be, though.

Btw, please also always add linux-block to the Cc list for block driver patches that are even the slightest bit about the block layer interface.

will do ..

-ck