Skip to content

bcache: track active bypass writes to fix read miss race

Ankit Kapoor linux-bcache lkml 1 version In review

Get series with b4
b4 am [email protected]

Versions newest first

  1. v3 Latest2 patches 1 replyIn review

Patches in v3 No applies recorded yet

#Subject and filesSizeRepliesApplied
1/2bcache: track active bypass writes to fix read miss racedrivers/md/bcache/bcache.h drivers/md/bcache/request.c drivers/md/bcache/super.c+202 -31 reply
2/2bcache: inspect active bypass writes lock-free via RCUdrivers/md/bcache/bcache.h drivers/md/bcache/request.c drivers/md/bcache/super.c+56 -250 replies

Reviews trailers per patch

No Reviewed-by, Acked-by or Tested-by trailers for v3 yet.

Read the v3 thread

Cover letter [PATCH v3 0/2] bcache: track active bypass writes to fix read miss race

This is the v3 patch series to fix a race condition between read cache misses and bypass writes in bcache.

This patch series fixes the race condition through two complementary mechanisms split across two logical patches:

1. [PATCH v3 1/2]: bcache: track active bypass writes to fix read miss race
- Dynamically tracks active bypass writes across the backing device using 32MB chunks and refcounted pages (a single 4KB page covers 32GB of disk).
- On a read cache miss, bcache checks for overlapping active bypass writes and forces the read to bypass the cache if a collision is detected.
- Serializes btree key invalidation so it executes in cached_dev_write_complete() after the payload reaches the disk, preventing early invalidations from opening a race window.
- Resilient memory management: counter pages are backed by a dedicated 16-page mempool to guarantee forward progress under severe memory pressure, and the top-level array is allocated via kvcalloc() to reliably support very large backing devices.

2. [PATCH v3 2/2]: bcache: inspect active bypass writes lock-free via RCU
- Optimizes the cache miss read path by inspecting active bypass counters lock-free under RCU (rcu_read_lock / rcu_dereference), removing spinlock contention from latency-sensitive reads.
- Writers continue to use localized page-level spinlocks to synchronize counter updates and mempool allocations.

Changes since v2:

Patch 1 (Core Tracking):
- Upgraded chunk counters from u16 to u32 to prevent any possibility of counter overflow under extreme IO concurrency.
- Replaced GFP_KERNEL/fallback allocations with a dedicated 16-page mempool (mempool_alloc with GFP_NOIO), guaranteeing that bypass tracking never fails under memory pressure and eliminating the need for untracked fallbacks.
- Used kvcalloc() for the top-level page array to support massive backing devices without requiring large contiguous physical allocations.
- Removed debug sysfs counters and tracepoints to keep the patch footprint minimal and focused on the core synchronization. These can be added in a follow-up patch.

Patch 2 (RCU Optimization):
- Introduced this entirely new modular commit to separate the lock-free read path optimization from the core race synchronization logic.

Memory Consumption:
Note: An additional 16-page mempool standby reserve is maintained (4 KB/page for Spinlock, ~4.1 KB/page for RCU).

Idle Memory Consumption (Zero active bypass writes)

Backing Disk Size  | Spinlock         | RCU
1 TB               | 0.5 KB           | 0.5 KB
10 TB              | 5.0 KB           | 5.0 KB
100 TB             | 50.0 KB          | 50.0 KB

Peak Memory Consumption (All tracking pages allocated)

Backing Disk Size  | Spinlock         | RCU
1 TB               | 128.5 KB         | ~140.0 KB
10 TB              | 1.28 MB          | ~1.4 MB
100 TB             | 12.5 MB          | ~14.0 MB

Setup:
- CPU: 32 vCPU, Intel Cascade Lake x86_64 (n2-standard-32 GCP VM)
- Memory: 128 GB RAM
- OS: Linux 6.12.94 (Google COS)
- Storage: Google Cloud Extreme PD (1000 GB) + Local SSD (375 GB)

Performance Results

FIO config:
rw=randrw, bs=(R) 4096B-4096B, (W) 128KiB-128KiB, (T) 128KiB-128KiB, ioengine=libaio, iodepth=32

NVMe SSD - 10 GB Working Set with 16 FIO jobs (1 active tracking page)

Metric                 | Baseline | Patch 1 (Spinlock) | Patch 2 (RCU)
Read IOPS              | 20,945   | 20,832             | 20,839
Write IOPS             | 8,978    | 8,936              | 8,935
Total IOPS             | 29,923   | 29,769             | 29,775
Avg Read Lat (ns)      | 33,292   | 33,858             | 33,871
Avg Write Lat (ns)     | 36,478   | 35,752             | 35,693
Agg Kernel CPU (sys %)*| 48.62%   | 50.80%             | 48.46%
NVMe Utilization       | 70.38%   | 44.29%             | 44.44%

*Note: Kernel CPU reported as aggregate across cores (~3% per core).

NVMe SSD - 320 GB Working Set with 16 FIO jobs (10 tracking pages)

Metric                 | Baseline | Patch 1 (Spinlock) | Patch 2 (RCU)
Read IOPS              | 20,968   | 20,955             | 20,959
Write IOPS             | 8,983    | 8,982              | 8,983
Total IOPS             | 29,951   | 29,937             | 29,942
Avg Read Lat (ns)      | 33,484   | 33,509             | 33,524
Avg Write Lat (ns)     | 35,932   | 35,926             | 35,884
Agg Kernel CPU (sys %)*| 52.54%   | 50.84%             | 52.15%
NVMe Utilization       | 68.11%   | 67.26%             | 69.00%

*Note: Kernel CPU reported as aggregate across cores (~3% per core).

Analysis
Active bypass write tracking shows no measurable performance degradation on real NVMe storage hardware. While a minor NVMe utilization dip is observed on small working sets (10 GB), device utilization remains completely normal on larger, realistic workloads (320 GB).

Comparing RCU and Spinlock

Because physical NVMe hardware latency masks microsecond-level software differences, we also benchmarked against a zero-latency null block device (null_blk) at 64 jobs to remove the hardware bottleneck and isolate true block-layer locking overhead.

Null Device (null_blk) - 10 GB Working Set with 64 FIO jobs

Metric                 | Baseline | Patch 1 (Spinlock) | Patch 2 (RCU)
Read IOPS              | 64,296   | 58,394             | 58,962
Write IOPS             | 27,551   | 25,012             | 25,267
Total IOPS             | 91,847   | 83,406             | 84,229
Avg Read Lat (ns)      | 45,463   | 50,104             | 49,678
Avg Write Lat (ns)     | 45,362   | 50,451             | 50,025
Agg Kernel CPU (sys %)*| 2119.08% | 2830.73%           | 2838.42%
NVMe Utilization       | 0.00%    | 0.00%              | 0.00%

*Note: Kernel CPU reported as aggregate across 64 threads (~44%/core).

Why Patch 2 (RCU Read-Path Optimization)?
While the absolute throughput improvement on our 32-core test system is modest (~1.0% / +823 IOPS and ~426 ns lower latency in a 64-job null_blk stress test), the primary motivation for Patch 2 is to maintain a lock-free read path and improve scalability on higher-core-count servers.

Thanks,
Ankit

Ankit Kapoor (2):
bcache: track active bypass writes to fix read miss race
bcache: inspect active bypass writes lock-free via RCU

 drivers/md/bcache/bcache.h  |  43 ++++++++++
 drivers/md/bcache/request.c | 151 +++++++++++++++++++++++++++++++++++-
 drivers/md/bcache/super.c   |  42 ++++++++++
 3 files changed, 233 insertions(+), 3 deletions(-)
Signature · 1 line
2.55.0.679.g6767b8d81c-goog