Skip to content

nvme-pci: adaptive interrupt coalescing

Jun Zeng linux-nvme lkml 1 version In review

Get series with b4
b4 am [email protected]

Versions newest first

  1. v1 Latest1 patch 14 repliesIn review

Patches in v1 No applies recorded yet

#Subject and filesSizeRepliesApplied
1/1nvme-pci: adaptive interrupt coalescingdrivers/nvme/host/pci.c+539 -110 replies

Reviews trailers per patch

No Reviewed-by, Acked-by or Tested-by trailers for v1 yet.

Read the v1 thread

Cover letter [PATCH v1 0/1] nvme-pci: adaptive interrupt coalescing

Background
==========

Nowadays, with the high performance PCIe 5.0 NVMe drives been used more increasingly and widely, it's important to maintain and fully realize NVMe drive's performance at the system level in scenarios where multiple NVMe drives are used simultaneously.

Problem
=======

With PCIe 5.0 NVMe drives, the interrupt count can reach very high levels in burst when running IOPS workloads (e.g., 4k/8k randread) with multiple queues across many NVMe drives. These high interrupts generated instantaneously may put pressure on current CPU handling mechanism.

Specifically, IOPS with small IO (e.g., 4k/8k randread) does not meet expectations when running with multiple queues among many NVMe drives (e.g., 6-8 NVMe drives per socket), even with each queue and job assigned to separate CPU cores.

The IOPS for small IO can be significantly improved when enabling interrupt coalescing in this case. However, in some cases (e.g., big IO for bandwidth or latency-sensitive tests with small IO and small queue depth), interrupt coalescing has side effects.

Solution
========

Monitor the IOPS and average queue depth of inflight IO periodically on each NVMe drive and enable/disable interrupt coalescing automatically without user intervention. This feature is enabled by default and can be disabled/enabled through debugfs during runtime.

Test Results
============

1. IOPS improved from < 20M to 55M for 4k randread with fio job=8, qd=128 across 16 PCIe Gen5 NVMe drives on Intel Xeon GNR CPUs.
- Latency not affected for 4k randread with job=1/2/4/8,

     qd=1/2/4/8/16/32
   - Bandwidth not affected for 32k/64k/128k read/write with
     job=1/2/4/8, qd=128

2. Tested with several mainstream PCIe Gen5 SSD vendors and Intel CPUs (including AMD) with improved 4K random read IOPS.

Notes
=====

1. Interrupt coalescing is a per-drive feature, not per-queue feature. Each queue has a separate interrupt, but coalescing is controlled globally per drive. We introduce thresholds for IO_DELTA, AVG_QUEUE_DEPTH, and TOTAL_PENDING_IO to judge workload status.

2. Tunable parameters exposed via debugfs at /sys/kernel/debug/nvme_pci/:
- disable_adaptive_interrupt_coalescing: 0=enabled, 1=disabled
- high_io_delta: IO count threshold per cycle (default: 45000)
- avg_qd_threshold: average queue depth threshold (default: 32)
- total_inflight_io_threshold: total inflight IO threshold

     (default: 256)

3. The default threshold values are derived from empirical testing on various PCIe Gen5 NVMe drives.

Jun Zeng (1):
nvme-pci: adaptive interrupt coalescing

 drivers/nvme/host/pci.c | 540 +++++++++++++++++++++++++++++++++++++++-
 1 file changed, 539 insertions(+), 1 deletion(-)

base-commit: 5d6919055dec134de3c40167a490f33c74c12581

Signature · 1 line
2.43.0