Skip to content

ibmvfc: NVMe/FC support over IBM Virtual FC

Tyrel Datwyler linux-scsi linuxppc-dev lkml 1 version Applied

Status from this message: Applied to 7.3/scsi-staging, thanks!

Get series with b4
b4 am [email protected]

Versions newest first

  1. v2 Latest33 patches 10 repliesApplied

Changes in v2 from the cover letter

--------------------------------------------
- Added four bug-fix patches (1-4) ahead of any NVMe/FC changes to ensure the series is bisect-safe. These address issues identified in the v1
review: the mempool_free/kfree mismatch (previously flagged on multiple patches), the rport_add_work_q teardown race, the NULL event dereference in the implicit logout and delete path, and the MAD send-failure deadlock due to the locked done handler being invoked with host_lock already held.
- Fixed the NULL pointer dereference in ibmvfc_nvme_unregister() and ibmvfc_nvme_unregister_remoteport() by adding proper NULL guards before dereferencing the local and remote port pointers (v1 patch 19/20).
- Fixed self-deadlock in ibmvfc_send_event() error path for MAD commands:
on send failure the driver now falls back to the unlocked evt->_done handler instead of the locked variant (patch 1 in v2).
- Fixed NVMe targets being entirely ignored by the TGT_DEL and TGT_DEL_FAILED work-loop actions; both the SCSI and NVMe target lists are now walked in the relevant state machine helpers (v1 patches 18/21).
- Fixed the purge path omission: NVMe FCP and LS requests are now properly failed during host reset (patch 33, expanded from v1 patch 29 which only handled the transport reset case).

Patches in v2 Applied

#Subject and filesSizeRepliesApplied
1/33ibmvfc: don't call locked done variant for MADs on send failuredrivers/scsi/ibmvscsi/ibmvfc.c+3 -10 replies
2/33ibmvfc: flush rport_add_work_q during driver teardowndrivers/scsi/ibmvscsi/ibmvfc.c+1 -00 replies
3/33ibmvfc: check for NULL evt in implicit LOGO and target delete pathdrivers/scsi/ibmvscsi/ibmvfc.c+8 -00 replies
4/33ibmvfc: free ibmvfc_target allocations with mempool_freedrivers/scsi/ibmvscsi/ibmvfc.c+1 -10 replies
5/33ibmvfc: move target list from host to protocol specific channel groupsdrivers/scsi/ibmvscsi/ibmvfc.c drivers/scsi/ibmvscsi/ibmvfc.h+28 -280 replies
6/33ibmvfc: add NVMe/FC protocol interface definitionsdrivers/scsi/ibmvscsi/ibmvfc.h+119 -340 replies
7/33ibmvfc: split NVMe support into separate source file and add transport stubsdrivers/scsi/ibmvscsi/Makefile drivers/scsi/ibmvscsi/ibmvfc-core.c drivers/scsi/ibmvscsi/ibmvfc-nvme.c +2 more+156 -40 replies
8/33ibmvfc: initialize NVMe channel configuration during driver probedrivers/scsi/ibmvscsi/ibmvfc-core.c drivers/scsi/ibmvscsi/ibmvfc.h+8 -10 replies
9/33ibmvfc: alloc/dealloc sub-queues for nvme channelsdrivers/scsi/ibmvscsi/ibmvfc-core.c+14 -10 replies
10/33ibmvfc: add logic for protocol specific fabric loginsdrivers/scsi/ibmvscsi/ibmvfc-core.c drivers/scsi/ibmvscsi/ibmvfc.h+206 -352 replies
11/33ibmvfc: add wrapper to get vhost associated with a channel structdrivers/scsi/ibmvscsi/ibmvfc.h+10 -01 reply
12/33ibmvfc: add helper for creating protocol specific discovery eventdrivers/scsi/ibmvscsi/ibmvfc-core.c+27 -120 replies
13/33ibmvfc: add helper to check NVMe/FC support with active channelsdrivers/scsi/ibmvscsi/ibmvfc-core.c+6 -00 replies
14/33ibmvfc: allocate and free NVMe channel group discover bufferdrivers/scsi/ibmvscsi/ibmvfc-core.c+7 -10 replies
15/33ibmvfc: send NVMe target discovery MADdrivers/scsi/ibmvscsi/ibmvfc-core.c+43 -70 replies
16/33ibmvfc: add NVMe/FC Implicit Logout and Move Login supportdrivers/scsi/ibmvscsi/ibmvfc-core.c+22 -110 replies
17/33ibmvfc: add NVMe/FC Port Login supportdrivers/scsi/ibmvscsi/ibmvfc-core.c+9 -60 replies
18/33ibmvfc: add NVMe/FC Process Login supportdrivers/scsi/ibmvscsi/ibmvfc-core.c drivers/scsi/ibmvscsi/ibmvfc.h+45 -160 replies
19/33ibmvfc: add NVMe/FC Query Target supportdrivers/scsi/ibmvscsi/ibmvfc-core.c+9 -60 replies
20/33ibmvfc: allocate targets based on protocoldrivers/scsi/ibmvscsi/ibmvfc-core.c+17 -50 replies
21/33ibmvfc: delete NVMe/FC targets as well as SCSIdrivers/scsi/ibmvscsi/ibmvfc-core.c+26 -20 replies
22/33ibmvfc: update state machine to process NVMe/FC targetsdrivers/scsi/ibmvscsi/ibmvfc-core.c+42 -00 replies
23/33ibmvfc: implement NVMe/FC stubs for local/remote port registrationdrivers/scsi/ibmvscsi/ibmvfc-nvme.c drivers/scsi/ibmvscsi/ibmvfc-nvme.h drivers/scsi/ibmvscsi/ibmvfc.h+84 -40 replies
24/33ibmvfc: register local nvme fc port after fabric logindrivers/scsi/ibmvscsi/ibmvfc-core.c drivers/scsi/ibmvscsi/ibmvfc-nvme.c+19 -30 replies
25/33ibmvfc: process NVMe/FC rports in work threaddrivers/scsi/ibmvscsi/ibmvfc-core.c+93 -14 replies
26/33ibmvfc: extend ibmvfc_debug visibility to ibmvfc-nvme.hdrivers/scsi/ibmvscsi/ibmvfc-core.c drivers/scsi/ibmvscsi/ibmvfc-nvme.h+4 -10 replies
27/33ibmvfc: declare global function definitionsdrivers/scsi/ibmvscsi/ibmvfc-core.c drivers/scsi/ibmvscsi/ibmvfc.h+23 -160 replies
28/33ibmvfc: implement LLDD callbacks for mapping nvme-fc queuesdrivers/scsi/ibmvscsi/ibmvfc-nvme.c drivers/scsi/ibmvscsi/ibmvfc-nvme.h+43 -20 replies
29/33ibmvfc: implement nvme-fc LS submission transport callbackdrivers/scsi/ibmvscsi/ibmvfc-core.c drivers/scsi/ibmvscsi/ibmvfc-nvme.c drivers/scsi/ibmvscsi/ibmvfc-nvme.h +1 more+100 -00 replies
30/33ibmvfc: implement nvme-fc IO command submission callbackdrivers/scsi/ibmvscsi/ibmvfc-core.c drivers/scsi/ibmvscsi/ibmvfc-nvme.c drivers/scsi/ibmvscsi/ibmvfc.h+142 -22 replies
31/33ibmvfc: implement nvme-fc LS abort handling callbackdrivers/scsi/ibmvscsi/ibmvfc-nvme.c+64 -00 replies
32/33ibmvfc: implement nvme-fc FCP abort callbackdrivers/scsi/ibmvscsi/ibmvfc-nvme.c+64 -00 replies
33/33ibmvfc: fail nvme-fc fcp-io and ls requests during transport resetdrivers/scsi/ibmvscsi/ibmvfc-core.c+45 -180 replies

Reviews trailers per patch

No Reviewed-by, Acked-by or Tested-by trailers for v2 yet.

Read the v2 thread

Cover letter [PATCH v2 00/33] ibmvfc: NVMe/FC support over IBM Virtual FC

This series adds NVMe/FC initiator support to the ibmvfc driver, enabling IBM POWER virtual machines to discover and use NVMe namespaces presented by the IBM Virtual I/O Server (VIOS) over the existing NPIV transport.

The ibmvfc driver communicates with the VIOS via a CRQ-based protocol. With this series the VIOS can present both SCSI/FCP and NVMe/FC targets through parallel sets of protocol-specific MAD opcodes, fabric login flows, and sub-CRQ channels.

The series is organized into three phases:

Patches 1-5: Bug fixes and preparatory refactoring
Four pre-existing bugs are fixed before any NVMe/FC work is introduced:
a deadlock in the MAD send-failure path (locked done variant called with host_lock already held), a race during driver teardown where rport_add_work_q work items can outlive the FC host, a NULL event dereference in ibmvfc_tgt_implicit_logout_and_del, and an allocator mismatch where mempool-allocated ibmvfc_target structs are freed via kfree rather than mempool_free. Patch 5 moves the target list and count from struct ibmvfc_host into struct ibmvfc_channels as the structural prerequisite for independent per-protocol target tracking.

Patches 6-27: Protocol interface and driver scaffolding
Patch 6 extends ibmvfc.h with NVMe/FC protocol definitions: MAD opcodes, capability flags, the v3 command layout, the fabric login MAD, async sub-CRQ event format, and updated channel enquiry/setup fields. Patch 7 splits ibmvfc.c into ibmvfc-core.c and the new ibmvfc-nvme.c/h, registers an nvme_fc_port_template with stub callbacks, and adds NVMe module parameters. The remaining patches build the NVMe/FC plumbing: NVMe channel-group initialization, sub-CRQ lifecycle management, protocol- specific fabric login flow, target discovery, PLOGI/PRLI/query-target, implicit logout, move-login, protocol-driven target allocation, NVMe target deletion, state machine updates, and local/remote port registration with the NVMe-FC transport layer.

Patches 28-33: NVMe-FC LLDD callbacks and I/O path
Implements the full nvme_fc_port_template: create_queue/delete_queue map NVMe controller queues to sub-CRQ handles; ls_req submits FC-LS frames via the ibmvfc passthru MAD; fcp_io builds and submits NVMe FCP commands via the NVMe sub-CRQ and completes them via nvme_fc_rcv_fcp_rsp(); ls_abort and fcp_abort cancel outstanding requests via NVMF cancel MADs. Patch 33 extends the purge path to fail outstanding NVMe FCP and LS requests during host reset and link-down events.

Changes since v1 (29 patches -> 33 patches)
--------------------------------------------
- Added four bug-fix patches (1-4) ahead of any NVMe/FC changes to ensure the series is bisect-safe. These address issues identified in the v1
review: the mempool_free/kfree mismatch (previously flagged on multiple patches), the rport_add_work_q teardown race, the NULL event dereference in the implicit logout and delete path, and the MAD send-failure deadlock due to the locked done handler being invoked with host_lock already held.
- Fixed the NULL pointer dereference in ibmvfc_nvme_unregister() and ibmvfc_nvme_unregister_remoteport() by adding proper NULL guards before dereferencing the local and remote port pointers (v1 patch 19/20).
- Fixed self-deadlock in ibmvfc_send_event() error path for MAD commands:
on send failure the driver now falls back to the unlocked evt->_done handler instead of the locked variant (patch 1 in v2).
- Fixed NVMe targets being entirely ignored by the TGT_DEL and TGT_DEL_FAILED work-loop actions; both the SCSI and NVMe target lists are now walked in the relevant state machine helpers (v1 patches 18/21).
- Fixed the purge path omission: NVMe FCP and LS requests are now properly failed during host reset (patch 33, expanded from v1 patch 29 which only handled the transport reset case).

Tyrel Datwyler (33):
ibmvfc: don't call locked done variant for MADs on send failure
ibmvfc: flush rport_add_work_q during driver teardown
ibmvfc: check for NULL evt in implicit LOGO and target delete path
ibmvfc: free ibmvfc_target allocations with mempool_free
ibmvfc: move target list from host to protocol specific channel groups
ibmvfc: add NVMe/FC protocol interface definitions
ibmvfc: split NVMe support into separate source file and add transport stubs
ibmvfc: initialize NVMe channel configuration during probe
ibmvfc: alloc/dealloc sub-queues for nvme channels
ibmvfc: add logic for protocol specific fabric login
ibmvfc: add wrapper to get vhost associated with a channel group
ibmvfc: add helper for creating protocol specific discover target events
ibmvfc: add helper to check NVMe/FC support with active queues
ibmvfc: allocate and free NVMe channel group discovery buffers
ibmvfc: send NVMe target discovery MAD
ibmvfc: add NVMe/FC Implicit Logout and Move Login support
ibmvfc: add NVMe/FC Port Login support
ibmvfc: add NVMe/FC Process Login support
ibmvfc: add NVMe/FC Query Target support
ibmvfc: allocate targets based on protocol
ibmvfc: delete NVMe/FC targets as well as SCSI
ibmvfc: update state machine to process NVMe/FC targets
ibmvfc: implement NVMe/FC stubs for local/remote port registration
ibmvfc: register local nvme fc port after fabric login
ibmvfc: process NVMe/FC rports in work thread
ibmvfc: extend ibmvfc_debug visibility to ibmvfc-nvme.c
ibmvfc: declare global function definitions
ibmvfc: implement LLDD callbacks for mapping nvme-fc queues
ibmvfc: implement nvme-fc LS submission transport callback
ibmvfc: implement nvme-fc IO command submission callback
ibmvfc: implement nvme-fc LS abort handling callback
ibmvfc: implement nvme-fc FCP abort callback
ibmvfc: fail nvme-fc fcp-io and ls requests during host reset

 drivers/scsi/ibmvscsi/Makefile                     |   2 +
 drivers/scsi/ibmvscsi/{ibmvfc.c => ibmvfc-core.c}  | ...
 drivers/scsi/ibmvscsi/ibmvfc-nvme.c                | ...
 drivers/scsi/ibmvscsi/ibmvfc-nvme.h                | ...
 drivers/scsi/ibmvscsi/ibmvfc.h                     | ...
 5 files changed
Signature · 1 line
2.55.0