Skip to content

RFC Support for cancel command (TP4097a)

Maurizio Lombardi linux-nvme 3 versions – New

Get series with b4
b4 am [email protected]

Versions newest first · linked by author and title

  1. v3 RFC Latest11 patches 0 repliesNew
  2. v2 RFC7 patches 0 repliesSuperseded
  3. v15 patches 4 repliesSuperseded

Patches in v3 No applies recorded yet

#Subject and filesSizeRepliesApplied
1/11nvme: add definitions for cancel commanddrivers/nvme/host/constants.c drivers/nvme/host/core.c include/linux/nvme.h+24 -00 replies
2/11nvme-core: reserve tags for cancel commandsdrivers/nvme/host/core.c drivers/nvme/host/nvme.h+15 -10 replies
3/11nvme-core: add a function to submit a cancel commanddrivers/nvme/host/core.c drivers/nvme/host/nvme.h+63 -00 replies
4/11nvme-tcp: use the cancel command to perform an abort if target supports itdrivers/nvme/host/core.c drivers/nvme/host/nvme.h drivers/nvme/host/tcp.c+64 -50 replies
5/11nvme-rdma: use the cancel command to perform an abort if target supports itdrivers/nvme/host/rdma.c+46 -40 replies
6/11nvmet: emulate basic support for cancel commandsdrivers/nvme/target/Makefile drivers/nvme/target/admin-cmd.c drivers/nvme/target/core.c +2 more+72 -10 replies
7/11nvmet: put all nvmet_req.execute calls behind a function namedrivers/nvme/target/fc.c drivers/nvme/target/loop.c drivers/nvme/target/nvmet.h +2 more+8 -70 replies
8/11nvmet: add delay debugfs file to nvmet_ctrldrivers/nvme/target/Kconfig drivers/nvme/target/debugfs.c drivers/nvme/target/nvmet.h+53 -00 replies
9/11nvmet: delay requestsdrivers/nvme/target/core.c drivers/nvme/target/nvmet.h+38 -00 replies
10/11nvmet: command trackingdrivers/nvme/target/core.c drivers/nvme/target/nvmet.h+32 -10 replies
11/11nvmet: cancel implementationdrivers/nvme/target/Makefile drivers/nvme/target/admin-cmd.c drivers/nvme/target/core.c +2 more+34 -140 replies

Reviews trailers per patch

No Reviewed-by, Acked-by or Tested-by trailers for v3 yet.

Read the v3 thread

Cover letter [PATCH RFC v3 00/11] Support for cancel command (TP4097a)

This patchset adds support for the cancel commands

The nvme host driver could take advantage of Cancel (defined by TP 4097a) to abort a command that timed out instead of resetting the controller.

The abort command's implementation is mandated by the specs but the target isn't necessarily required to perform any meaningful action (as it happens, for example, with the Linux nvmet implementation, where the abort command does nothing at all).
On the other hand, the Cancel command is optional, therefore if the target claims to support it, it's reasonable to expect its implementation to perform a useful action.

This patchset modifies the tcp and rdma host drivers' timeout handlers to check if the target supports the Cancel command; if it does, it will use it to try to abort the command that timed out. If multiple commands time out at the same time, the driver will use a Cancel command with the 'multiple commands' flag to attempt to abort all stuck commands as a last attempt to avoid a controller reset.

If Cancel isn't supported, if the allocation of the command fails, or if any failure is encountered, the driver will fall back to the standard controller reset mechanism.

Example of dmesg output:

nvme nvme0: I/O tag 3 (3003) type 4 opcode 0x1 (I/O Cmd) QID 5 timeout nvme nvme0: I/O tag 4 (2004) type 4 opcode 0x1 (I/O Cmd) QID 5 timeout nvme nvme0: I/O tag 5 (4005) type 4 opcode 0x1 (I/O Cmd) QID 5 timeout nvme nvme0: Cancel status: 0x0 imm abrts = 1 def abrts = 0 nvme nvme0: I/O tag 6 (5006) type 4 opcode 0x1 (I/O Cmd) QID 5 timeout nvme nvme0: Cancel status: 0x0 imm abrts = 63 def abrts = 0 nvme nvme0: I/O tag 3 (9003) type 4 opcode 0x1 (I/O Cmd) QID 6 timeout nvme nvme0: I/O tag 4 (a004) type 4 opcode 0x1 (I/O Cmd) QID 6 timeout nvme nvme0: I/O tag 5 (a005) type 4 opcode 0x1 (I/O Cmd) QID 6 timeout nvme nvme0: Cancel status: 0x0 imm abrts = 1 def abrts = 0 nvme nvme0: I/O tag 73 (7049) type 4 opcode 0x1 (I/O Cmd) QID 6 timeout nvme nvme0: Cancel status: 0x0 imm abrts = 5 def abrts = 0

This also adds a target side Cancel implementation written by Chris Leech, intended for testing. It adds a new Kconfig option CONFIG_NVME_TARGET_DELAY_REQUESTS which depends on debugfs. When this config option is enabled, a delay attribute is added under the kernel/debug/nvmet/test-nqn/ctrl1/ directory.
When a controller is created, it will read as "0 0"

$ cat delay    
0 0    

The first number is a count of how many requests will be delayed. The second is a delay time in ms.

V3: In the host driver, instead of counting outstanding Cancel
commands using atomic_inc()/atomic_dec(), use two flags in the queue structure. (Suggested by Sagi Grimberg). Added a target side implementation.

V2: Reserve 2 commands per queue for cancel commands (the number is fixed but in future it can be made configurable with trivial changes to the code).

Chris Leech (5):
nvmet: put all nvmet_req.execute calls behind a function name
nvmet: add delay debugfs file to nvmet_ctrl
nvmet: delay requests
nvmet: command tracking
nvmet: cancel implementation

Maurizio Lombardi (6):
nvme: add definitions for cancel command
nvme-core: reserve tags for cancel commands
nvme-core: add a function to submit a cancel command
nvme-tcp: use the cancel command to perform an abort if target
supports it
nvme-rdma: use the cancel command to perform an abort if target
supports it
nvmet: emulate basic support for cancel commands

 drivers/nvme/host/constants.c       |  1 +
 drivers/nvme/host/core.c            | 83 ++++++++++++++++++++++++++++-
 drivers/nvme/host/nvme.h            |  4 ++
 drivers/nvme/host/rdma.c            | 50 +++++++++++++++--
 drivers/nvme/host/tcp.c             | 62 +++++++++++++++++++--
 drivers/nvme/target/Kconfig         |  9 ++++
 drivers/nvme/target/Makefile        |  1 +
 drivers/nvme/target/admin-cmd.c     |  3 ++
 drivers/nvme/target/core.c          | 67 ++++++++++++++++++++++-
 drivers/nvme/target/debugfs.c       | 40 ++++++++++++++
 drivers/nvme/target/fc.c            |  4 +-
 drivers/nvme/target/io-cmd-cancel.c | 77 ++++++++++++++++++++++++++
 drivers/nvme/target/loop.c          |  2 +-
 drivers/nvme/target/nvmet.h         | 19 +++++++
 drivers/nvme/target/rdma.c          |  4 +-
 drivers/nvme/target/tcp.c           |  4 +-
 include/linux/nvme.h                | 22 ++++++++
 17 files changed, 434 insertions(+), 18 deletions(-)
 create mode 100644 drivers/nvme/target/io-cmd-cancel.c
Signature · 1 line
2.43.5