bcache: reduce front IO latency during GC

In performance benchmarks on disks with bcache using the Linux 6.6 kernel, we observe noticeable IO latency increase during btree garbage collection. The increase ranges from high tens to hundreds of milliseconds, depending on the size of the cache device. Further investigation reveals that it is the same issue reported in [1], where the large number of nodes processed in each incremental GC cycle causes the front IO latency.

Building upon the approach suggested in [1], this patch decomposes the incremental GC process into more but smaller cycles. In contrast to [1], this implementation adopts a simpler strategy by setting a lower limit of 10 nodes per cycle to reduce front IO delay and introducing a fixed 10ms sleep per cycle when front IO is in progress. Furthermore, when garbage collection statistics are available, the number of nodes processed per cycle is dynamically rescaled based on the average GC frequency to ensure GC completes well within the next subsequent scheduled interval.

Testing with a 750GB NVMe cache and 256KB bucket size using the following fio configuration demonstrates that our patch reduces front IO latency during GC without significantly increasing GC duration.

ioengine=libaio
direct=1
bs=4k
size=900G
iodepth=10
readwrite=randwrite
log_avg_msec=10

Before:

time-ms,latency-ns,,,

12170, 285016, 1, 0, 0
12183, 296581, 1, 0, 0
12207, 6542725, 1, 0, 0
12242, 24483604, 1, 0, 0
12250, 1895628, 1, 0, 0
12260, 284854, 1, 0, 0
12270, 275513, 1, 0, 0

/sys/block/bcache0/bcache/cache/internal/btree_gc_average_duration_ms:2880 /sys/block/bcache0/bcache/cache/internal/btree_gc_average_frequency_sec:133 /sys/block/bcache0/bcache/cache/internal/btree_gc_last_sec:121 /sys/block/bcache0/bcache/cache/internal/btree_gc_max_duration_ms:3456

After:

12690, 378494, 1, 0, 0
12700, 413934, 1, 0, 0
12710, 661217, 1, 0, 0
12727, 354510, 1, 0, 0
12730, 1100768, 1, 0, 0
12742, 382484, 1, 0, 0
12750, 532679, 1, 0, 0
12760, 572758, 1, 0, 0
12773, 283416, 1, 0, 0

/sys/block/bcache0/bcache/cache/internal/btree_gc_average_duration_ms:3619 /sys/block/bcache0/bcache/cache/internal/btree_gc_average_frequency_sec:58 /sys/block/bcache0/bcache/cache/internal/btree_gc_last_sec:23 /sys/block/bcache0/bcache/cache/internal/btree_gc_max_duration_ms:3866

[1] https://lore.kernel.org/all/[email protected]/

---

Changes in v3:
- Use 64-bit integer division macros to fix build error on i386 architecture.

Robert Pang (1):
bcache: process fewer btree nodes in incremental GC cycles

 drivers/md/bcache/btree.c | 37 ++++++++++++++++++-------------------
 drivers/md/bcache/util.h  |  7 +++++++
 2 files changed, 25 insertions(+), 19 deletions(-)

2.49.0.777.g153de2bbd5-goog