Skip to content

nvme-tcp: queue stalls under high load

Hannes Reinecke linux-nvme 1 version In review

Get series with b4
b4 am [email protected]

Versions newest first

  1. v1 Latest3 patches 20 repliesIn review

Patches in v1 No applies recorded yet

#Subject and filesSizeRepliesApplied
1/3nvme-tcp: spurious I/O timeout under high loaddrivers/nvme/host/tcp.c+7 -011 replies
2/3nvme-tcp: Check for write space before queueing requestsdrivers/nvme/host/tcp.c+3 -03 replies
3/3nvme-tcp: send quota for nvme_tcp_send_all()drivers/nvme/host/tcp.c+2 -13 replies

Reviews trailers per patch

No Reviewed-by, Acked-by or Tested-by trailers for v1 yet.

Read the v1 thread

Cover letter [PATCH 0/3] nvme-tcp: queue stalls under high load

Hi all,

one of our partners registered queue stalls and I/O timeouts under high load. Analysis revealed that we see an extremely 'choppy' I/O behaviour when running large transfers on systems on low-performance links (eg 1GigE networks).
We had a system with 30 queues trying to transfer 128M requests; simple calculation shows that transferring a _single_ request on all queues will take up to 38 seconds, thereby timing out the last request before it got sent.
As a solution I first fixed up the timeout handler to reset the timeout if the request is still queued or in the process of being send. The second path modifies the send path to only allow for new requests if we have enough space on the TX queue, and finally break up the send loop to avoid system stalls when sending large request.

As usual, comments and reviews are welcome.

Hannes Reinecke (3):
nvme-tcp: spurious I/O timeout under high load
nvme-tcp: Check for write space before queueing requests
nvme-tcp: send quota for nvme_tcp_send_all()

 drivers/nvme/host/tcp.c | 13 ++++++++++++-
 1 file changed, 12 insertions(+), 1 deletion(-)
Signature · 1 line
2.29.2