bcache: reduce gc latency by processing less nodes and sleep less time
RFC PATCH 1/1 bcache: reduce gc latency by processing less nodes and sleep less time
From: Coly Li <[email protected]>
When bcache device is busy for high I/O loads, there are two methods to reduce the garbage collection latency,
- Process less nodes in eac loop of incremental garbage collection in btree_gc_recurse().
- Sleep less time between two full garbage collection in bch_btree_gc().
This patch introduces to hleper routines to provide different garbage collection nodes number and sleep intervel time.
- btree_gc_min_nodes()
If there is no front end I/O, return 128 nodes to process in each incremental loop, otherwise only 10 nodes are returned. Then front I/O is able to access the btree earlier.
- btree_gc_sleep_ms()
If there is no synchronized wait for bucket allocation, sleep 100 ms between two incremental GC loop. Othersize only sleep 10 ms before incremental GC loop. Then a faster GC may provide available buckets earlier, to avoid most of bcache working threads from being starved by buckets allocation.
The idea is inspired by works from Mingzhe Zou and Robert Pang, but much simpler and the expected behavior is more predictable.
--- drivers/md/bcache/alloc.c | 4 ++++ drivers/md/bcache/bcache.h | 1 + drivers/md/bcache/btree.c | 47 +++++++++++++++++++------------------- 3 files changed, 28 insertions(+), 24 deletions(-)
drivers/md/bcache/alloc.c +4 -0
@@ -412,7 +412,11 @@ long bch_bucket_alloc(struct cache *ca, unsigned int reserve, bool wait) TASK_UNINTERRUPTIBLE);mutex_unlock (&ca->set->bucket_lock); + +atomic_inc (&ca->set->bucket_wait_cnt);schedule (); +atomic_dec (&ca->set->bucket_wait_cnt); +mutex_lock (&ca->set->bucket_lock); }while (!fifo_pop (&ca->free[RESERVE_NONE ], r) && !fifo_pop (&ca->free[reserve], r));
drivers/md/bcache/bcache.h +1 -0
@@ -607,6 +607,7 @@ struct cache_set {*/ atomic_t prio_blocked;wait_queue_head_t bucket_wait; +atomic_t bucket_wait_cnt;/* * For any bio we don't skip we subtract the number of sectors from
drivers/md/bcache/btree.c +23 -24
76 lines not on this page · raw message
Signature · 1 line
2.47.2
A message that is not in the archive (sent off-list or not yet received): [email protected]
Hi Coly,
Thank you for your new patch. I have reviewed the patch, and it is much simpler and cleaner indeed.
In addition, I have run the 24-hour stress tests and can confirm that it brings noticeable performance improvement. For our 4kb writethrough workload, your patch further reduces the median (P50) latency during garbage collection from about 20 ms in the previous approach [1] to 4 ms [2]. And I did not observe any issues during my testing.
This is a great step forward. I look forward to seeing this patch submitted soon. And please feel free to add me to the sign-off list if it fits.
Best regards,
Robert Pang
[1] https://gist.github.com/robert-pang/a22b7c5dee2600be3260f4db57e5776d
[2] https://gist.github.com/robert-pang/05b17921a83d59afc8aab28b5d9e9e0d
Quoted 48 lines · Coly Li
Hi Robert,
Your patch breaks the
emwa_add()method to maintain the gc stats numbers. So I have to look for another method but try to get similar benchmark results as yours did.Hi Mingzhe,
The dynamic sleep interval and gc nodes patch is kind of over complicated IMHO. So I compose a simplified one based on the idea from you and Robert.
Can you all help to review and test this RFC patch? Hope it may work out. Thanks for your help in advance.
Coly Li
Quoted 29 lines
2025年8月29日 00:16,[email protected] 写道:
When bcache device is busy for high I/O loads, there are two methods to reduce the garbage collection latency,
- Process less nodes in eac loop of incremental garbage collection inbtree_gc_recurse().
- Sleep less time between two full garbage collection inbch_btree_gc().This patch introduces to hleper routines to provide different garbage collection nodes number and sleep intervel time.
-btree_gc_min_nodes()
If there is no front end I/O, return 128 nodes to process in each incremental loop, otherwise only 10 nodes are returned. Then front I/O is able to access the btree earlier.
-btree_gc_sleep_ms()
If there is no synchronized wait for bucket allocation, sleep 100 ms between two incremental GC loop. Othersize only sleep 10 ms before incremental GC loop. Then a faster GC may provide available buckets earlier, to avoid most of bcache working threads from being starved by buckets allocation.The idea is inspired by works from Mingzhe Zou and Robert Pang, but much simpler and the expected behavior is more predictable.
Signed-off-by Coly Li <[email protected]>Cc Robert Pang <[email protected]>Cc Mingzhe Zou <[email protected]>---
[snipped]