md: avoid counter operation conflicts
v2 cover md: avoid counter operation conflicts
Changes in v2:
- Add message of performance in cover.
- Fix a problem of error code initialization in patch 6.
- Fix a problem of sleeping during rcu_read_lock() in patch 9.
- Change base-commit from md-6.9 to md-6.10
Currently, active_aligned_reads and nr_pending used as counters are atomic types. Therefore, when inc/dec in a multi-core results in conflicts and READ I/O becomes slow. To improve performance, use "percpu_ref" counters that can avoid conflicts and maintain consistency.
Switch modes of percpu_ref to achieve both consistency and conflict avoidance. During normal operations such as inc/dec, it operates as percpu mode. When consistency is required, it operates as atomic mode.
The operations that require consistency are as follows:
- Zero check for the counter
- All operations in RAID 1/10
Patches 1, 3, 6 change active_aligned_reads, and patches 2, 4, 5, 7 to 11 change nr_pending. nr_pending temporarily switch from percpu mode to atomic mode in patch 7. This is to reduce the amount of changes from patches 8 to 10.
Finally, nr_pending switch from atomic mode to percpu mode in patch 11.
We applied the patch to base-commit and used fio to compare IOPS.
CPU: AMD EPYC 7313P (3.0GHz, 16cores)
DISK: ramdisk x 3 (modprobe brd rd_nr=3)
RAID: level 5
fio config: bs=4k, rw=randread, iodepth=128, numjobs=16
without patch: 3.64 MIOPS
with patch : 3.84 MIOPS
Keisuke TADA (11):
add infra for active_aligned_reads changes
add infra for nr_pending changes
workaround for inconsistency of config state in takeover minimize execution of zero check for nr_pending match the type of variables to percpu_ref
avoid conflicts in active_aligned_reads operations change the type of nr_pending from atomic_t to percpu_ref add atomic mode switching in RAID 1/10
add atomic mode switching when removing disk
add atomic mode switching when I/O completion avoid conflicts in nr_pending operations
drivers/md/md-bitmap.c | 2 +- drivers/md/md.c | 48 ++++++++++++++++++--- drivers/md/md.h | 62 +++++++++++++++++++++++---- drivers/md/raid1.c | 37 +++++++++++------ drivers/md/raid10.c | 60 ++++++++++++++++----------- drivers/md/raid5-cache.c | 4 +- drivers/md/raid5.c | 90 +++++++++++++++++++++++++++------------- drivers/md/raid5.h | 17 +++++++- 8 files changed, 238 insertions(+), 82 deletions(-)
base-commit: 9d1110f99c253ccef82e480bfe9f38a12eb797a7 --
2.34.1
Quoted 23 lines · tada keisuke
Changes in v2:
- Add message of performance in cover.
- Fix a problem of error code initialization in patch 6.
- Fix a problem of sleeping duringrcu_read_lock()in patch 9.
- Change base-commit from md-6.9 to md-6.10Currently,
active_aligned_readsandnr_pendingused as counters are atomic types. Therefore, when inc/dec in a multi-core results in conflicts and READ I/O becomes slow. To improve performance, use "percpu_ref" counters that can avoid conflicts and maintain consistency.Switch modes of
percpu_refto achieve both consistency and conflict avoidance. During normal operations such as inc/dec, it operates as percpu mode. When consistency is required, it operates as atomic mode.
The operations that require consistency are as follows:
- Zero check for the counter
- All operations in RAID 1/10Patches 1, 3, 6 change
active_aligned_reads, and patches 2, 4, 5, 7 to 11 changenr_pending.nr_pendingtemporarily switch from percpu mode to atomic mode in patch 7. This is to reduce the amount of changes from patches 8 to 10.
Finally,nr_pendingswitch from atomic mode to percpu mode in patch 11.
Quoted 8 lines
We applied the patch to base-commit and used fio to compare IOPS.
CPU: AMD EPYC 7313P (3.0GHz, 16cores)
DISK: ramdisk x 3 (modprobe brd rd_nr=3)
RAID: level 5
fio config: bs=4k, rw=randread, iodepth=128, numjobs=16without patch: 3.64 MIOPS
with patch : 3.84 MIOPS
Hi Tada
Thanks for the patch set. Have you done tests with nvme/ssd or hdd? It's better to see the results with real disks.
Best Regards
Xiao
Quoted 31 lines
Keisuke TADA (11):
add infra foractive_aligned_readschanges
add infra fornr_pendingchanges
workaround for inconsistency of config state in takeover minimize execution of zero check fornr_pendingmatch the type of variables topercpu_ref
avoid conflicts inactive_aligned_readsoperations change the type ofnr_pendingfromatomic_ttopercpu_refadd atomic mode switching in RAID 1/10
add atomic mode switching when removing disk
add atomic mode switching when I/O completion avoid conflicts innr_pendingoperationsdrivers/md/md-bitmap.c | 2 +- drivers/md/md.c | 48 ++++++++++++++++++--- drivers/md/md.h | 62 +++++++++++++++++++++++---- drivers/md/raid1.c | 37 +++++++++++------ drivers/md/raid10.c | 60 ++++++++++++++++----------- drivers/md/raid5-cache.c | 4 +- drivers/md/raid5.c | 90 +++++++++++++++++++++++++++------------- drivers/md/raid5.h | 17 +++++++- 8 files changed, 238 insertions(+), 82 deletions(-)base-commit:
9d1110f99c253ccef82e480bfe9f38a12eb797a7--
2.34.1