bcache: track active bypass writes to fix read miss race
Get series with b4
b4 am [email protected]Versions newest first
- v3 Latest2 patches 1 replyIn review
Patches in v3 No applies recorded yet
| # | Subject and files | Size | Replies | Applied |
|---|---|---|---|---|
| 1/2 | bcache: track active bypass writes to fix read miss racedrivers/ drivers/ drivers/ | +202 | 1 reply | |
| 2/2 | bcache: inspect active bypass writes lock-free via RCUdrivers/ drivers/ drivers/ | +56 | 0 replies |
Reviews trailers per patch
No Reviewed-by, Acked-by or Tested-by trailers for v3 yet.
Cover letter [PATCH v3 0/2] bcache: track active bypass writes to fix read miss race
This is the v3 patch series to fix a race condition between read cache misses and bypass writes in bcache.
This patch series fixes the race condition through two complementary mechanisms split across two logical patches:
1. [PATCH v3 1/2]: bcache: track active bypass writes to fix read miss race
- Dynamically tracks active bypass writes across the backing device using 32MB chunks and refcounted pages (a single 4KB page covers 32GB of disk).
- On a read cache miss, bcache checks for overlapping active bypass writes and forces the read to bypass the cache if a collision is detected.
- Serializes btree key invalidation so it executes in cached_dev_write_complete() after the payload reaches the disk, preventing early invalidations from opening a race window.
- Resilient memory management: counter pages are backed by a dedicated 16-page mempool to guarantee forward progress under severe memory pressure, and the top-level array is allocated via kvcalloc() to reliably support very large backing devices.
2. [PATCH v3 2/2]: bcache: inspect active bypass writes lock-free via RCU
- Optimizes the cache miss read path by inspecting active bypass counters lock-free under RCU (rcu_read_lock / rcu_dereference), removing spinlock contention from latency-sensitive reads.
- Writers continue to use localized page-level spinlocks to synchronize counter updates and mempool allocations.
Changes since v2:
Patch 1 (Core Tracking):
- Upgraded chunk counters from u16 to u32 to prevent any possibility of counter overflow under extreme IO concurrency.
- Replaced GFP_KERNEL/fallback allocations with a dedicated 16-page mempool (mempool_alloc with GFP_NOIO), guaranteeing that bypass tracking never fails under memory pressure and eliminating the need for untracked fallbacks.
- Used kvcalloc() for the top-level page array to support massive backing devices without requiring large contiguous physical allocations.
- Removed debug sysfs counters and tracepoints to keep the patch footprint minimal and focused on the core synchronization. These can be added in a follow-up patch.
Patch 2 (RCU Optimization):
- Introduced this entirely new modular commit to separate the lock-free read path optimization from the core race synchronization logic.
Memory Consumption:
Note: An additional 16-page mempool standby reserve is maintained (4 KB/page for Spinlock, ~4.1 KB/page for RCU).
Idle Memory Consumption (Zero active bypass writes)
Backing Disk Size | Spinlock | RCU 1 TB | 0.5 KB | 0.5 KB 10 TB | 5.0 KB | 5.0 KB 100 TB | 50.0 KB | 50.0 KB
Peak Memory Consumption (All tracking pages allocated)
Backing Disk Size | Spinlock | RCU 1 TB | 128.5 KB | ~140.0 KB 10 TB | 1.28 MB | ~1.4 MB 100 TB | 12.5 MB | ~14.0 MB
Setup:
- CPU: 32 vCPU, Intel Cascade Lake x86_64 (n2-standard-32 GCP VM)
- Memory: 128 GB RAM
- OS: Linux 6.12.94 (Google COS)
- Storage: Google Cloud Extreme PD (1000 GB) + Local SSD (375 GB)
Performance Results
FIO config:
rw=randrw, bs=(R) 4096B-4096B, (W) 128KiB-128KiB, (T) 128KiB-128KiB, ioengine=libaio, iodepth=32
NVMe SSD - 10 GB Working Set with 16 FIO jobs (1 active tracking page)
Metric | Baseline | Patch 1 (Spinlock) | Patch 2 (RCU) Read IOPS | 20,945 | 20,832 | 20,839 Write IOPS | 8,978 | 8,936 | 8,935 Total IOPS | 29,923 | 29,769 | 29,775 Avg Read Lat (ns) | 33,292 | 33,858 | 33,871 Avg Write Lat (ns) | 36,478 | 35,752 | 35,693 Agg Kernel CPU (sys %)*| 48.62% | 50.80% | 48.46% NVMe Utilization | 70.38% | 44.29% | 44.44%
*Note: Kernel CPU reported as aggregate across cores (~3% per core).
NVMe SSD - 320 GB Working Set with 16 FIO jobs (10 tracking pages)
Metric | Baseline | Patch 1 (Spinlock) | Patch 2 (RCU) Read IOPS | 20,968 | 20,955 | 20,959 Write IOPS | 8,983 | 8,982 | 8,983 Total IOPS | 29,951 | 29,937 | 29,942 Avg Read Lat (ns) | 33,484 | 33,509 | 33,524 Avg Write Lat (ns) | 35,932 | 35,926 | 35,884 Agg Kernel CPU (sys %)*| 52.54% | 50.84% | 52.15% NVMe Utilization | 68.11% | 67.26% | 69.00%
*Note: Kernel CPU reported as aggregate across cores (~3% per core).
Analysis
Active bypass write tracking shows no measurable performance degradation on real NVMe storage hardware. While a minor NVMe utilization dip is observed on small working sets (10 GB), device utilization remains completely normal on larger, realistic workloads (320 GB).
Comparing RCU and Spinlock
Because physical NVMe hardware latency masks microsecond-level software differences, we also benchmarked against a zero-latency null block device (null_blk) at 64 jobs to remove the hardware bottleneck and isolate true block-layer locking overhead.
Null Device (null_blk) - 10 GB Working Set with 64 FIO jobs
Metric | Baseline | Patch 1 (Spinlock) | Patch 2 (RCU) Read IOPS | 64,296 | 58,394 | 58,962 Write IOPS | 27,551 | 25,012 | 25,267 Total IOPS | 91,847 | 83,406 | 84,229 Avg Read Lat (ns) | 45,463 | 50,104 | 49,678 Avg Write Lat (ns) | 45,362 | 50,451 | 50,025 Agg Kernel CPU (sys %)*| 2119.08% | 2830.73% | 2838.42% NVMe Utilization | 0.00% | 0.00% | 0.00%
*Note: Kernel CPU reported as aggregate across 64 threads (~44%/core).
Why Patch 2 (RCU Read-Path Optimization)?
While the absolute throughput improvement on our 32-core test system is modest (~1.0% / +823 IOPS and ~426 ns lower latency in a 64-job null_blk stress test), the primary motivation for Patch 2 is to maintain a lock-free read path and improve scalability on higher-core-count servers.
Thanks,
Ankit
Ankit Kapoor (2):
bcache: track active bypass writes to fix read miss race
bcache: inspect active bypass writes lock-free via RCU
drivers/md/bcache/bcache.h | 43 ++++++++++ drivers/md/bcache/request.c | 151 +++++++++++++++++++++++++++++++++++- drivers/md/bcache/super.c | 42 ++++++++++ 3 files changed, 233 insertions(+), 3 deletions(-)
Signature · 1 line
2.55.0.679.g6767b8d81c-goog