ibmvfc: Add NVMe-FC support
Subsystems (3)
Tyrel Datwyler
maintainercover ibmvfc: Add NVMe-FC support
This patch series adds NVMe-FC protocol support to the ibmvfc driver, enabling IBM POWER virtual Fibre Channel adapters to handle both SCSI and NVMe storage traffic through a unified driver architecture.
The implementation leverages the existing multi-queue infrastructure and extends the driver's protocol handling to support NVMe-FC alongside traditional SCSI FCP. Key architectural changes include:
- Refactoring the driver into protocol-agnostic core (ibmvfc-core.c) and protocol-specific modules (ibmvfc-nvme.c for NVMe-FC)
- Extending the target discovery and management state machine to handle NVMe/FC targets with appropriate protocol-specific login sequences
- Implementing the nvme-fc LLDD (Low-Level Device Driver) callbacks for queue management, command submission, and abort handling
- Adding support for NVMe-FC specific MAD (Management Adapter Data) operations including target discovery, port login, process login, and implicit logout/move login
- Implementing NVMe FCP and Link Services (LS) command submission and abort paths using the existing sub-CRQ infrastructure
The series is organized to introduce changes incrementally:
Patches 1-8: Infrastructure and refactoring
- Move target lists to protocol-specific channel groups
- Split NVMe support into separate source file
- Add protocol interface definitions and helper functions
- Initialize NVMe channel configuration
Patches 9-16: NVMe/FC MAD operations and target management
- Implement discovery, port login, process login, query target
- Add target allocation based on protocol
- Update state machine for NVMe/FC target processing
- Handle NVMe/FC target deletion
Patches 17-20: Local and remote port registration
- Implement nvme-fc local port registration after fabric login
- Process NVMe/FC rports in work thread
- Extend debug visibility for NVMe components
Patches 21-26: LLDD callback implementation
- Implement queue mapping callbacks
- Add LS and FCP command submission paths
- Implement NVMe FCP and LS abort handling
Patches 27-29: Error handling and debugging
- Fail NVMe requests during transport reset
Tyrel Datwyler (29):
ibmvfc: move target list from host to protocol specific channel groups
ibmvfc: add NVMe/FC protocol interface definitions
ibmvfc: split NVMe support into separate source file and add transport
stubs
ibmvfc: initialize NVMe channel configuration during driver probe
ibmvfc: alloc/dealloc sub-queues for nvme channels
ibmvfc: add logic for protocol specific fabric logins
ibmvfc: add wrapper to get vhost associated with a channel struct
ibmvfc: add helper for creating protocol specific discovery event
ibmvfc: add helper to check NVMe/FC support with active channels
ibmvfc: allocate and free NVMe channel group discover buffer
ibmvfc: send NVMe target discovery MAD
ibmvfc: add NVMe/FC Implicit Logout and Move Login support
ibmvfc: add NVMe/FC Port Login support
ibmvfc: add NVMe/FC Process Login support
ibmvfc: add NVMe/FC Query Target support
ibmvfc: allocate targets based on protocol
ibmvfc: delete NVMe/FC targets as well as SCSI
ibmvfc: update state machine to process NVMe/FC targets
ibmvfc: implement NVMe/FC stubs for local/remote port registration
ibmvfc: register local nvme fc port after fabric login
ibmvfc: process NVMe/FC rports in work thread
ibmvfc: extend ibmvfc_debug visibility to ibmvfc-nvme.h
ibmvfc: declare global function definitions
ibmvfc: implement LLDD callbacks for mapping nvme-fc queues
ibmvfc: implement nvme-fc LS submission transport callback
ibmvfc: implement nvme-fc IO command submission callback
ibmvfc: implement nvme-fc LS abort handling callback
ibmvfc: implement nvme-fc FCP abort callback
ibmvfc: fail nvme-fc fcp-io and ls requests during transport reset
drivers/scsi/ibmvscsi/Makefile | 2 +
.../scsi/ibmvscsi/{ibmvfc.c => ibmvfc-core.c} | 709 ++++++++++++++----
drivers/scsi/ibmvscsi/ibmvfc-nvme.c | 565 ++++++++++++++
drivers/scsi/ibmvscsi/ibmvfc-nvme.h | 48 ++
drivers/scsi/ibmvscsi/ibmvfc.h | 209 +++++-
5 files changed, 1360 insertions(+), 173 deletions(-)
rename drivers/scsi/ibmvscsi/{ibmvfc.c => ibmvfc-core.c} (89%)
create mode 100644 drivers/scsi/ibmvscsi/ibmvfc-nvme.c
create mode 100644 drivers/scsi/ibmvscsi/ibmvfc-nvme.hSignature · 1 line
2.54.0
Tyrel Datwyler
maintainerPATCH 1/29 ibmvfc: move target list from host to protocol specific channel groups
Prepare the driver for protocol-specific target management by moving the target list and target count out of struct ibmvfc_host and into struct ibmvfc_channels.
Today the driver only maintains a single SCSI target list, but NVMe/FC support will require separate target tracking for each protocol-specific channel group. Update the existing target iteration, allocation, and discovery paths to use the SCSI channel group's target list instead of a host-wide list.
This is a preparatory refactoring only. No functional change is intended for existing SCSI operation.
--- drivers/scsi/ibmvscsi/ibmvfc.c | 52 +++++++++++++++++----------------- drivers/scsi/ibmvscsi/ibmvfc.h | 4 +-- 2 files changed, 28 insertions(+), 28 deletions(-)
drivers/scsi/ibmvscsi/ibmvfc.c +26 -26
@@ -682,7 +682,7 @@ static void ibmvfc_link_down(struct ibmvfc_host *vhost,ENTER ;scsi_block_requests (vhost->host); -list_for_each_entry (tgt, &vhost->targets, queue) +list_for_each_entry (tgt, &vhost->scsi_scrqs.targets, queue)ibmvfc_del_tgt (tgt);ibmvfc_set_host_state (vhost, state);ibmvfc_set_host_action (vhost,IBMVFC_HOST_ACTION_TGT_DEL );
@@ -715,7 +715,7 @@ static void ibmvfc_init_host(struct ibmvfc_host *vhost)memset (vhost->async_crq.msgs.async,0 ,PAGE_SIZE ); vhost->async_crq.cur =0 ; -list_for_each_entry (tgt, &vhost->targets, queue) { +list_for_each_entry (tgt, &vhost->scsi_scrqs.targets, queue) {if (vhost->client_migrated) tgt->need_login =1 ;else
@@ -1232,7 +1232,7 @@ static struct ibmvfc_target *__ibmvfc_get_target(struct scsi_target *starget)struct ibmvfc_host *vhost =shost_priv (shost);struct ibmvfc_target *tgt; -list_for_each_entry (tgt, &vhost->targets, queue) +list_for_each_entry (tgt, &vhost->scsi_scrqs.targets, queue)if (tgt->target_id == starget->id) {kref_get (&tgt->kref);return tgt;
@@ -1832,7 +1832,7 @@ static void ibmvfc_relogin(struct scsi_device *sdev)unsigned long flags;spin_lock_irqsave (vhost->host->host_lock, flags); -list_for_each_entry (tgt, &vhost->targets, queue) { +list_for_each_entry (tgt, &vhost->scsi_scrqs.targets, queue) {if (rport == tgt->rport) {ibmvfc_del_tgt (tgt);break ;
@@ -2130,7 +2130,7 @@ static int ibmvfc_bsg_plogi(struct ibmvfc_host *vhost, unsigned int port_id)ENTER ;spin_lock_irqsave (vhost->host->host_lock, flags); -list_for_each_entry (tgt, &vhost->targets, queue) { +list_for_each_entry (tgt, &vhost->scsi_scrqs.targets, queue) {if (tgt->scsi_id == port_id) { issue_login =0 ;break ;
@@ -3102,7 +3102,7 @@ static void ibmvfc_terminate_rport_io(struct fc_rport *rport)spin_lock_irqsave (shost->host_lock, flags); found =0 ; -list_for_each_entry (tgt, &vhost->targets, queue) { +list_for_each_entry (tgt, &vhost->scsi_scrqs.targets, queue) {if (tgt->scsi_id == rport->port_id) { found++;break ;
@@ -3242,7 +3242,7 @@ static void ibmvfc_handle_async(struct ibmvfc_async_crq *crq,case IBMVFC_AE_ELS_LOGO :case IBMVFC_AE_ELS_PRLO :case IBMVFC_AE_ELS_PLOGI : -list_for_each_entry (tgt, &vhost->targets, queue) { +list_for_each_entry (tgt, &vhost->scsi_scrqs.targets, queue) {if (!crq->scsi_id && !crq->wwpn && !crq->node_name)break ;if (crq->scsi_id &&cpu_to_be64 (tgt->scsi_id) != crq->scsi_id)
@@ -4863,14 +4863,14 @@ static int ibmvfc_alloc_target(struct ibmvfc_host *vhost,/* Look to see if we already have a target allocated for this SCSI ID or WWPN */ spin_lock_irqsave (vhost->host->host_lock, flags); -list_for_each_entry (tgt, &vhost->targets, queue) { +list_for_each_entry (tgt, &vhost->scsi_scrqs.targets, queue) {if (tgt->wwpn == wwpn) { wtgt = tgt;break ; } } -list_for_each_entry (tgt, &vhost->targets, queue) { +list_for_each_entry (tgt, &vhost->scsi_scrqs.targets, queue) {if (tgt->scsi_id == scsi_id) { stgt = tgt;break ;
@@ -4927,7 +4927,7 @@ static int ibmvfc_alloc_target(struct ibmvfc_host *vhost,ibmvfc_init_tgt (tgt, ibmvfc_tgt_implicit_logout);spin_lock_irqsave (vhost->host->host_lock, flags); tgt->cancel_key = vhost->task_set++; -list_add_tail (&tgt->queue, &vhost->targets); +list_add_tail (&tgt->queue, &vhost->scsi_scrqs.targets); unlock_out:spin_unlock_irqrestore (vhost->host->host_lock, flags);
@@ -4945,7 +4945,7 @@ static int ibmvfc_alloc_targets(struct ibmvfc_host *vhost) {int i, rc; -for (i =0 , rc =0 ; !rc && i < vhost->num_targets; i++) +for (i =0 , rc =0 ; !rc && i < vhost->scsi_scrqs.num_targets; i++) rc =ibmvfc_alloc_target (vhost, &vhost->scsi_scrqs.disc_buf[i]);return rc;
@@ -4966,8 +4966,8 @@ static void ibmvfc_discover_targets_done(struct ibmvfc_event *evt)switch (mad_status) {case IBMVFC_MAD_SUCCESS :ibmvfc_dbg (vhost,"Discover Targets succeeded\n" ); - vhost->num_targets =min_t (u32,be32_to_cpu (rsp->num_written), - max_targets); + vhost->scsi_scrqs.num_targets =min_t (u32,be32_to_cpu (rsp->num_written), + max_targets);ibmvfc_set_host_action (vhost,IBMVFC_HOST_ACTION_ALLOC_TGTS );break ;case IBMVFC_MAD_FAILED :
@@ -5383,7 +5383,7 @@ static int ibmvfc_dev_init_to_do(struct ibmvfc_host *vhost) {struct ibmvfc_target *tgt; -list_for_each_entry (tgt, &vhost->targets, queue) { +list_for_each_entry (tgt, &vhost->scsi_scrqs.targets, queue) {if (tgt->action ==IBMVFC_TGT_ACTION_INIT || tgt->action ==IBMVFC_TGT_ACTION_INIT_WAIT )return 1 ;
@@ -5403,7 +5403,7 @@ static int ibmvfc_dev_logo_to_do(struct ibmvfc_host *vhost) {struct ibmvfc_target *tgt; -list_for_each_entry (tgt, &vhost->targets, queue) { +list_for_each_entry (tgt, &vhost->scsi_scrqs.targets, queue) {if (tgt->action ==IBMVFC_TGT_ACTION_LOGOUT_RPORT || tgt->action ==IBMVFC_TGT_ACTION_LOGOUT_RPORT_WAIT )return 1 ;
@@ -5433,10 +5433,10 @@ static int __ibmvfc_work_to_do(struct ibmvfc_host *vhost)case IBMVFC_HOST_ACTION_QUERY_TGTS :if (vhost->discovery_threads == disc_threads)return 0 ; -list_for_each_entry (tgt, &vhost->targets, queue) +list_for_each_entry (tgt, &vhost->scsi_scrqs.targets, queue)if (tgt->action ==IBMVFC_TGT_ACTION_INIT )return 1 ; -list_for_each_entry (tgt, &vhost->targets, queue) +list_for_each_entry (tgt, &vhost->scsi_scrqs.targets, queue)if (tgt->action ==IBMVFC_TGT_ACTION_INIT_WAIT )return 0 ;return 1 ;
@@ -5444,10 +5444,10 @@ static int __ibmvfc_work_to_do(struct ibmvfc_host *vhost)case IBMVFC_HOST_ACTION_TGT_DEL_FAILED :if (vhost->discovery_threads == disc_threads)return 0 ; -list_for_each_entry (tgt, &vhost->targets, queue) +list_for_each_entry (tgt, &vhost->scsi_scrqs.targets, queue)if (tgt->action ==IBMVFC_TGT_ACTION_LOGOUT_RPORT )return 1 ; -list_for_each_entry (tgt, &vhost->targets, queue) +list_for_each_entry (tgt, &vhost->scsi_scrqs.targets, queue)if (tgt->action ==IBMVFC_TGT_ACTION_LOGOUT_RPORT_WAIT )return 0 ;return 1 ;
@@ -5635,12 +5635,12 @@ static void ibmvfc_do_work(struct ibmvfc_host *vhost) vhost->job_step (vhost);break ;case IBMVFC_HOST_ACTION_QUERY : -list_for_each_entry (tgt, &vhost->targets, queue) +list_for_each_entry (tgt, &vhost->scsi_scrqs.targets, queue)ibmvfc_init_tgt (tgt, ibmvfc_tgt_query_target);ibmvfc_set_host_action (vhost,IBMVFC_HOST_ACTION_QUERY_TGTS );break ;case IBMVFC_HOST_ACTION_QUERY_TGTS : -list_for_each_entry (tgt, &vhost->targets, queue) { +list_for_each_entry (tgt, &vhost->scsi_scrqs.targets, queue) {if (tgt->action ==IBMVFC_TGT_ACTION_INIT ) { tgt->job_step (tgt);break ;
@@ -5652,7 +5652,7 @@ static void ibmvfc_do_work(struct ibmvfc_host *vhost)break ;case IBMVFC_HOST_ACTION_TGT_DEL :case IBMVFC_HOST_ACTION_TGT_DEL_FAILED : -list_for_each_entry (tgt, &vhost->targets, queue) { +list_for_each_entry (tgt, &vhost->scsi_scrqs.targets, queue) {if (tgt->action ==IBMVFC_TGT_ACTION_LOGOUT_RPORT ) { tgt->job_step (tgt);break ;
@@ -5664,7 +5664,7 @@ static void ibmvfc_do_work(struct ibmvfc_host *vhost)return ; } -list_for_each_entry (tgt, &vhost->targets, queue) { +list_for_each_entry (tgt, &vhost->scsi_scrqs.targets, queue) {if (tgt->action ==IBMVFC_TGT_ACTION_DEL_RPORT ) {tgt_dbg (tgt,"Deleting rport\n" ); rport = tgt->rport;
@@ -5739,7 +5739,7 @@ static void ibmvfc_do_work(struct ibmvfc_host *vhost)spin_lock_irqsave (vhost->host->host_lock, flags);break ;case IBMVFC_HOST_ACTION_TGT_INIT : -list_for_each_entry (tgt, &vhost->targets, queue) { +list_for_each_entry (tgt, &vhost->scsi_scrqs.targets, queue) {if (tgt->action ==IBMVFC_TGT_ACTION_INIT ) { tgt->job_step (tgt);break ;
@@ -6276,7 +6276,7 @@ static void ibmvfc_rport_add_thread(struct work_struct *work)if (vhost->state !=IBMVFC_ACTIVE )break ; -list_for_each_entry (tgt, &vhost->targets, queue) { +list_for_each_entry (tgt, &vhost->scsi_scrqs.targets, queue) {if (tgt->add_rport) { did_work =1 ; tgt->add_rport =0 ;
@@ -6341,7 +6341,7 @@ static int ibmvfc_probe(struct vio_dev *vdev, const struct vio_device_id *id) shost->nr_hw_queues = mq_enabled ?min (max_scsi_queues, nr_scsi_hw_queues) :1 ; vhost =shost_priv (shost); -INIT_LIST_HEAD (&vhost->targets); +INIT_LIST_HEAD (&vhost->scsi_scrqs.targets);INIT_LIST_HEAD (&vhost->purge);sprintf (vhost->name,IBMVFC_NAME ); vhost->host = shost;
drivers/scsi/ibmvscsi/ibmvfc.h +2 -2
@@ -828,6 +828,8 @@ struct ibmvfc_channels {unsigned int active_queues;unsigned int desired_queues;unsigned int max_queues; +int num_targets; +struct list_head targets;int disc_buf_sz;struct ibmvfc_discover_targets_entry *disc_buf;dma_addr_t disc_buf_dma;
@@ -871,8 +873,6 @@ struct ibmvfc_host {#define IBMVFC_TRACE_SIZE (sizeof(struct ibmvfc_trace_entry) * IBMVFC_NUM_TRACE_ENTRIES)struct ibmvfc_trace_entry *trace;atomic_t trace_index; -int num_targets; -struct list_head targets;struct list_head purge;struct device *dev;struct dma_pool *sg_pool;
Signature · 1 line
2.54.0
[email protected]
AI reviewRe: [PATCH 01/29] ibmvfc: move target list from host to protocol specific channel groups
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
Pre-existing issues:
- [High] The driver schedules a background work item (vhost->rport_add_work_q) but never explicitly flushes or cancels it during host removal or module unloading, leading to a leaked work struct and a potential use-after-free. --
commit 455b96a57ea40cd18201b8e8603264b5cfb84e0c
Author: Tyrel Datwyler <[email protected]>
ibmvfc: move target list from host to protocol specific channel groups
Prepare the driver for protocol-specific target management by moving the target list and target count out of struct ibmvfc_host and into struct ibmvfc_channels.
Quoted 4 lines
drivers/scsi/ibmvscsi/ibmvfc.c +0
-0
[ ... ]
drivers/scsi/ibmvscsi/ibmvfc.c static void ibmvfc_rport_add_thread(struct work_struct *work) · line 6279
Quoted 6 lines
drivers/scsi/ibmvscsi/ibmvfc.c +1
-1@@ -6276,7 +6276,7 @@ static void ibmvfc_rport_add_thread(struct work_struct *work)if (vhost->state !=IBMVFC_ACTIVE )break ; -list_for_each_entry (tgt, &vhost->targets, queue) { +list_for_each_entry (tgt, &vhost->scsi_scrqs.targets, queue) {
[Severity: High]
This isn't a bug introduced by this patch, but does the driver leak the rport_add_work_q on module removal?
In ibmvfc_remove(), the vhost structure is freed via scsi_host_put() after teardown:
drivers/scsi/ibmvscsi/ibmvfc.c:ibmvfc_remove() {
...
kthread_stop(vhost->work_thread);
fc_remove_host(vhost->host);
scsi_remove_host(vhost->host);
...
scsi_host_put(vhost->host);
}However, the teardown sequence never explicitly flushes or cancels vhost->rport_add_work_q.
If the work thread is actively running or pending on the system workqueue:
drivers/scsi/ibmvscsi/ibmvfc.c:ibmvfc_rport_add_thread() {
struct ibmvfc_host *vhost = container_of(work, struct ibmvfc_host,
rport_add_work_q);
...
spin_lock_irqsave(vhost->host->host_lock, flags);
...
}Could it execute after the vhost is deallocated, leading to a use-after-free when accessing the lock or other memory?
Quoted 3 lines
drivers/scsi/ibmvscsi/ibmvfc.c +0
-0if (tgt->add_rport) { did_work =1 ; tgt->add_rport =0 ;
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=1
Tyrel Datwyler
maintainerPATCH 2/29 ibmvfc: add NVMe/FC protocol interface definitions
In reply to Tyrel DatwylerAdd the protocol definitions for client-VIOS interface updates needed to support NVMe/FC over the ibmvfc NPIV transport.
Extend the ibmvfc interface with:
- NVMe/FC-specific capability bits and opcodes
- protocol-specific channel and queue definitions
- updated channel enquiry/setup fields for NVMe queues
- v3 command layout support for protocol-specific payloads
These changes provide the common header and interface plumbing needed by later patches that add NVMe/FC login, discovery, remote-port handling, and I/O submission.
--- drivers/scsi/ibmvscsi/ibmvfc.h | 153 +++++++++++++++++++++++++-------- 1 file changed, 119 insertions(+), 34 deletions(-)
drivers/scsi/ibmvscsi/ibmvfc.h +119 -34
@@ -13,6 +13,8 @@#include <linux/list.h> #include <linux/types.h> #include <scsi/viosrp.h> +#include <linux/nvme.h> +#include <linux/nvme-fc.h> #define IBMVFC_NAME "ibmvfc"#define IBMVFC_DRIVER_VERSION "1.0.11"
@@ -92,6 +94,7 @@ enum ibmvfc_cmd_status_flags {IBMVFC_FC_SCSI_ERROR =0x0008 ,IBMVFC_HW_EVENT_LOGGED =0x0010 ,IBMVFC_VIOS_LOGGED =0x0020 , + IBMVFC_FC_NVME_STATUS = 0x0040, }; enum ibmvfc_fabric_mapped_errors {
@@ -124,20 +127,37 @@ enum ibmvfc_vios_errors { IBMVFC_COMMAND_FAILED = 0x8000, }; +enum ibmvfc_fc_nvme_errors { +IBMVFC_NVMS_VALID_ERSP =0x0001 , +IBMVFC_NVMS_VALID_NODMA_CQE =0x0002 , +}; +enum ibmvfc_mad_types {IBMVFC_NPIV_LOGIN =0x0001 , -IBMVFC_DISC_TARGETS =0x0002 , +IBMVFC_DISC_TARGETS =0x0002 , +IBMVFC_DISC_NVMF_TARGETS =0x0003 ,IBMVFC_PORT_LOGIN =0x0004 , -IBMVFC_PROCESS_LOGIN =0x0008 , -IBMVFC_QUERY_TARGET =0x0010 , +IBMVFC_NVMF_PORT_LOGIN =0x0005 , +IBMVFC_PROCESS_LOGIN =0x0008 , +IBMVFC_NVMF_PROCESS_LOGIN =0x0009 , +IBMVFC_QUERY_TARGET =0x0010 , +IBMVFC_NVMF_QUERY_TARGET =0x0011 ,IBMVFC_MOVE_LOGIN =0x0020 , -IBMVFC_IMPLICIT_LOGOUT =0x0040 , -IBMVFC_PASSTHRU =0x0200 , -IBMVFC_TMF_MAD =0x0100 , -IBMVFC_NPIV_LOGOUT =0x0800 , -IBMVFC_CHANNEL_ENQUIRY =0x1000 , -IBMVFC_CHANNEL_SETUP =0x2000 , -IBMVFC_CONNECTION_INFO =0x4000 , +IBMVFC_NVMF_MOVE_LOGIN =0x0021 , +IBMVFC_IMPLICIT_LOGOUT =0x0040 , +IBMVFC_NVMF_IMPLICIT_LOGOUT =0x0041 , +IBMVFC_RNID =0x0080 , +IBMVFC_NVMF_RNID =0x0081 , +IBMVFC_TMF_MAD =0x0100 , +IBMVFC_NVMF_TMF_MAD =0x0101 , +IBMVFC_PASSTHRU =0x0200 , +IBMVFC_NVMF_PASSTHRU =0x0201 , +IBMVFC_NPIV_LOGOUT =0x0800 , +IBMVFC_CHANNEL_ENQUIRY =0x1000 , +IBMVFC_CHANNEL_SETUP =0x2000 , +IBMVFC_CONNECTION_INFO =0x4000 , +IBMVFC_FABRIC_LOGIN =0x8000 , +IBMVFC_NVMF_FABRIC_LOGIN =0x8001 , };struct ibmvfc_mad_common {
@@ -175,11 +195,16 @@ struct ibmvfc_npiv_login {#define IBMVFC_FLUSH_ON_HALT 0x02__be32 max_cmds;__be64 capabilities; -#define IBMVFC_CAN_MIGRATE 0x01 -#define IBMVFC_CAN_USE_CHANNELS 0x02 -#define IBMVFC_CAN_HANDLE_FPIN 0x04 -#define IBMVFC_CAN_USE_MAD_VERSION 0x08 -#define IBMVFC_CAN_SEND_VF_WWPN 0x10 +#define IBMVFC_CAN_MIGRATE 0x001 +#define IBMVFC_CAN_USE_CHANNELS 0x002 +#define IBMVFC_CAN_HANDLE_FPIN 0x004 +#define IBMVFC_CAN_USE_MAD_VERSION 0x008 +#define IBMVFC_CAN_SEND_VF_WWPN 0x010 +#define IBMVFC_YES_NVMEOF 0x020 +#define IBMVFC_YES_SCSI 0x040 +#define IBMVFC_CAN_USE_WWPN_ALL 0x080 +#define IBMVFC_USE_ASYNC_SUBQ 0x100 +#define IBMVFC_CAN_USE_NOOP_CMD 0x200__be64 node_name;struct srp_direct_buf async;u8 partition_name[IBMVFC_MAX_NAME ];
@@ -219,13 +244,18 @@ struct ibmvfc_npiv_login_resp {__be16 error;__be32 flags;#define IBMVFC_NATIVE_FC 0x01 -__be32 reserved; +__be32 possible_nports;__be64 capabilities; -#define IBMVFC_CAN_FLUSH_ON_HALT 0x08 -#define IBMVFC_CAN_SUPPRESS_ABTS 0x10 -#define IBMVFC_MAD_VERSION_CAP 0x20 -#define IBMVFC_HANDLE_VF_WWPN 0x40 -#define IBMVFC_CAN_SUPPORT_CHANNELS 0x80 +#define IBMVFC_CAN_FLUSH_ON_HALT 0x0008 +#define IBMVFC_CAN_SUPPRESS_ABTS 0x0010 +#define IBMVFC_MAD_VERSION_CAP 0x0020 +#define IBMVFC_HANDLE_VF_WWPN 0x0040 +#define IBMVFC_CAN_SUPPORT_CHANNELS 0x0080 +#define IBMVFC_SUPPORT_NVMEOF 0x0100 +#define IBMVFC_SUPPORT_SCSI 0x0200 +#define IBMVFC_SUPPORT_WWPN_ALL 0x0400 +#define IBMVFC_ASYNC_SUBQ 0x0800 +#define IBMVFC_SUPPORT_NOOP_CMD 0x1000__be32 max_cmds;__be32 scsi_id_sz;__be64 max_dma_len;
@@ -238,7 +268,7 @@ struct ibmvfc_npiv_login_resp {u8 port_loc_code[IBMVFC_MAX_NAME ];u8 drc_name[IBMVFC_MAX_NAME ];struct ibmvfc_service_parms service_parms; -__be64 reserved2; +__be64 reserved; } __packed__aligned(8) ;union ibmvfc_npiv_login_data {
@@ -246,6 +276,17 @@ union ibmvfc_npiv_login_data {struct ibmvfc_npiv_login_resp resp; } __packed__aligned(8) ; +struct ibmvfc_fabric_login_mad { +struct ibmvfc_mad_common common; +__be64 flags; +__be64 capabilities; +__be64 nport_id; +__be16 status; +__be16 error; +__be32 reserved; +__be64 reserved2[16 ]; +} __packed__aligned(8) ; +struct ibmvfc_discover_targets_entry {__be32 scsi_id;__be32 pad;
@@ -287,6 +328,7 @@ enum ibmvfc_fc_type {IBMVFC_FABRIC_BUSY =0x04 ,IBMVFC_PORT_BUSY =0x05 ,IBMVFC_BASIC_REJECT =0x06 , + IBMVFC_FC4_LS_REJECT = 0x07, }; enum ibmvfc_gs_explain {
@@ -377,20 +419,27 @@ struct ibmvfc_query_tgt {struct ibmvfc_implicit_logout {struct ibmvfc_mad_common common;__be64 old_scsi_id; -__be64 reserved[2 ]; +__be64 reserved[8 ]; +__be64 target_wwpn; } __packed__aligned(8) ;struct ibmvfc_tmf {struct ibmvfc_mad_common common;__be64 scsi_id; -struct scsi_lun lun; +union { +struct scsi_lun lun; +__be64 assoc_id; + };__be32 flags; -#define IBMVFC_TMF_ABORT_TASK 0x02 -#define IBMVFC_TMF_ABORT_TASK_SET 0x04 -#define IBMVFC_TMF_LUN_RESET 0x10 -#define IBMVFC_TMF_TGT_RESET 0x20 -#define IBMVFC_TMF_LUA_VALID 0x40 -#define IBMVFC_TMF_SUPPRESS_ABTS 0x80 +#define IBMVFC_TMF_ABORT_TASK 0x002 +#define IBMVFC_TMF_ABORT_TASK_SET 0x004 +#define IBMVFC_TMF_LUN_RESET 0x010 +#define IBMVFC_TMF_TGT_RESET 0x020 +#define IBMVFC_TMF_LUA_VALID 0x040 +#define IBMVFC_TMF_SUPPRESS_ABTS 0x080 +#define IBMVFC_TMF_NVMF_ASSOC 0x100 +#define IBMVFC_TMF_NVMF_TARGET 0x200 +#define IBMVFC_TMF_BITMASK_VALID 0x400__be32 cancel_key;__be32 my_cancel_key;__be32 pad;
@@ -446,6 +495,8 @@ enum ibmvfc_cmd_flags {IBMVFC_WRITE =0x0008 ,IBMVFC_TMF =0x0080 ,IBMVFC_CLASS_3_ERR =0x0100 , +IBMVFC_NVMEOF_PROTOCOL =0x0200 , + IBMVFC_NVMF_SLER = 0x0400, }; enum ibmvfc_fc_task_attr {
@@ -493,7 +544,7 @@ struct ibmvfc_cmd {__be64 tgt_scsi_id;__be64 tag;__be64 target_wwpn; -__be64 reserved3; +__be64 assoc_id; union { struct { struct ibmvfc_fcp_cmd_iu iu;
@@ -504,6 +555,15 @@ struct ibmvfc_cmd {struct ibmvfc_fcp_cmd_iu iu;struct ibmvfc_fcp_rsp rsp; } v2; +struct { +__be64 reserved5[4 ]; +struct ibmvfc_fcp_cmd_iu iu; +struct ibmvfc_fcp_rsp rsp; + } v3scsi; +struct { +__be64 reserved[4 ]; +struct nvme_fc_cmd_iu iu; + } v3nvme; }; } __packed__aligned(8) ;
@@ -522,6 +582,9 @@ struct ibmvfc_passthru_iu {__be32 flags;#define IBMVFC_FC_ELS 0x01#define IBMVFC_FC_CT_IU 0x02 +#define IBMVFC_PT_PRLI 0x04 +#define IBMVFC_FC4_LS_OTH 0x08 +#define IBMVFC_FC4_LS_DSC_CTRL 0x10__be32 cancel_key;#define IBMVFC_PASSTHRU_CANCEL_KEY 0x80000000#define IBMVFC_INTERNAL_CANCEL_KEY 0x80000001
@@ -559,7 +622,7 @@ struct ibmvfc_channel_setup_mad {struct srp_direct_buf buffer; } __packed__aligned(8) ; -#define IBMVFC_MAX_CHANNELS 502 +#define IBMVFC_MAX_CHANNELS 501struct ibmvfc_channel_setup {__be32 flags;
@@ -574,6 +637,7 @@ struct ibmvfc_channel_setup {struct srp_direct_buf buffer;__be64 reserved2[5 ];__be64 channel_handles[IBMVFC_MAX_CHANNELS ]; +__be64 async_sub_crq_handle; } __packed__aligned(8) ;struct ibmvfc_connection_info {
@@ -620,7 +684,8 @@ struct ibmvfc_trace_entry {enum ibmvfc_crq_formats {IBMVFC_CMD_FORMAT =0x01 , -IBMVFC_ASYNC_EVENT =0x02 , +IBMVFC_ASYNC_EVENT =0x02 , +IBMVFC_NOOP =0x03 ,IBMVFC_MAD_FORMAT =0x04 , };
@@ -639,6 +704,9 @@ enum ibmvfc_async_event {IBMVFC_AE_RESUME =0x0800 ,IBMVFC_AE_ADAPTER_FAILED =0x1000 ,IBMVFC_AE_FPIN =0x2000 , +IBMVFC_NVME_DISCONNECT =0x4000 , +IBMVFC_NVMEOF_PROTO_AVAIL =0x40000000 , + IBMVFC_SCSI_PROTO_AVAIL = 0x80000000, }; struct ibmvfc_async_desc {
@@ -683,13 +751,30 @@ struct ibmvfc_async_crq {volatile __be64 scsi_id;volatile __be64 wwpn;volatile __be64 node_name; -__be64 reserved; +__be64 assoc_id; +} __packed__aligned(8) ; + +struct ibmvfc_async_sub_crq { +volatile u8 valid; +u8 flags; +#define IBMVFC_ASYNC_ID_IS_ASSOC_ID 0x01 +u8 link_state; +u8 fpin_status; +__be16 event; +__be16 pad; +__be64 wwpn; +__be64 nport_id; +union { +__be64 node_name; +__be64 assoc_id; + } id; } __packed__aligned(8) ;union ibmvfc_iu {struct ibmvfc_mad_common mad_common;struct ibmvfc_npiv_login_mad npiv_login;struct ibmvfc_npiv_logout_mad npiv_logout; +struct ibmvfc_fabric_login_mad fabric_login;struct ibmvfc_discover_targets discover_targets;struct ibmvfc_port_login plogi;struct ibmvfc_process_login prli;
Signature · 1 line
2.54.0
[email protected]
AI reviewRe: [PATCH 02/29] ibmvfc: add NVMe/FC protocol interface definitions
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [High] Unconditional size expansion of `struct ibmvfc_implicit_logout` breaks backwards compatibility…
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [High] Unconditional size expansion of struct ibmvfc_implicit_logout breaks backwards compatibility with older VIOS versions. --
commit c4f31afebbe28cbb64ba03689b7e7ed327cb6d9e
Author: Tyrel Datwyler <[email protected]>
ibmvfc: add NVMe/FC protocol interface definitions
This commit adds the protocol definitions for client-VIOS interface updates needed to support NVMe/FC over the ibmvfc NPIV transport. It extends the ibmvfc interface with NVMe/FC-specific capabilities, queue definitions, and command layout support.
Quoted 4 lines
drivers/scsi/ibmvscsi/ibmvfc.h +0
-0
[ ... ]
drivers/scsi/ibmvscsi/ibmvfc.h struct ibmvfc_query_tgt { · line 424
Quoted 8 lines
drivers/scsi/ibmvscsi/ibmvfc.h +2
-1@@ -377,20 +419,27 @@ struct ibmvfc_query_tgt {struct ibmvfc_implicit_logout {struct ibmvfc_mad_common common;__be64 old_scsi_id; -__be64 reserved[2 ]; +__be64 reserved[8 ]; +__be64 target_wwpn; } __packed__aligned(8) ;
[Severity: High]
Will this unconditional size expansion break backwards compatibility with older Virtual I/O Server (VIOS) firmware?
Since the driver calculates the MAD length dynamically using sizeof():
drivers/scsi/ibmvscsi/ibmvfc.c:__ibmvfc_tgt_get_implicit_logout_evt() {
...
mad->common.length = cpu_to_be16(sizeof(*mad));
...
}This changes the length sent for version 1 IBMVFC_IMPLICIT_LOGOUT operations from 40 bytes to 96 bytes. Could older VIOS versions that strictly validate MAD lengths against the version 1 specification reject this payload, breaking target teardown and error recovery?
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=2
Tyrel Datwyler
maintainerPATCH 3/29 ibmvfc: split NVMe support into separate source file and add transport stubs
In reply to Tyrel DatwylerRename ibmvfc.c to ibmvfc-core.c as first step in decoupling each protocol from the core driver logic. Add ibmvfc-nvme.[ch] files, and register an nvme_fc_port_template with empty callback stubs.
Add empty registration functions definitions for local and remote ports.
No functional NVMe/FC support is added yet.
---
drivers/scsi/ibmvscsi/Makefile | 2 +
.../scsi/ibmvscsi/{ibmvfc.c => ibmvfc-core.c} | 19 +++-
drivers/scsi/ibmvscsi/ibmvfc-nvme.c | 87 +++++++++++++++++++
drivers/scsi/ibmvscsi/ibmvfc-nvme.h | 32 +++++++
drivers/scsi/ibmvscsi/ibmvfc.h | 10 ++-
5 files changed, 146 insertions(+), 4 deletions(-)
rename drivers/scsi/ibmvscsi/{ibmvfc.c => ibmvfc-core.c} (99%)
create mode 100644 drivers/scsi/ibmvscsi/ibmvfc-nvme.c
create mode 100644 drivers/scsi/ibmvscsi/ibmvfc-nvme.hdrivers/scsi/ibmvscsi/Makefile +2 -0
@@ -1,3 +1,5 @@ # SPDX-License-Identifier: GPL-2.0-only +ibmvfc-objs := ibmvfc-core.o ibmvfc-nvme.o + obj-$(CONFIG_SCSI_IBMVSCSI) += ibmvscsi.o obj-$(CONFIG_SCSI_IBMVFC) += ibmvfc.o
drivers/scsi/ibmvscsi/ibmvfc-core.c renamed from drivers/scsi/ibmvscsi/ibmvfc.c +17 -2
@@ -1,10 +1,11 @@ -// SPDX-License-Identifier: GPL-2.0-or-later +/* SPDX-License-Identifier: GPL-2.0-or-later */ +/* * ibmvfc.c -- driver for IBM Power Virtual Fibre Channel Adapter * * Written By: Brian King <[email protected]>, IBM Corporation * -* Copyright (C) IBM Corporation, 2008 +* Copyright (C) IBM Corporation, 2008-2026 */ #include <linux/module.h>
@@ -46,6 +47,9 @@ static unsigned int cls3_error = IBMVFC_CLS3_ERROR;static unsigned int mq_enabled =IBMVFC_MQ ;static unsigned int nr_scsi_hw_queues =IBMVFC_SCSI_HW_QUEUES ;static unsigned int nr_scsi_channels =IBMVFC_SCSI_CHANNELS ; +static unsigned int nvme_enabled =IBMVFC_NVME ; +static unsigned int nr_nvme_hw_queues =IBMVFC_NVME_HW_QUEUES ; +static unsigned int nr_nvme_channels =IBMVFC_NVME_CHANNELS ;static unsigned int mig_channels_only =IBMVFC_MIG_NO_SUB_TO_CRQ ;static unsigned int mig_no_less_channels =IBMVFC_MIG_NO_N_TO_M ;
@@ -74,6 +78,16 @@ module_param_named(mig_no_less_channels, mig_no_less_channels, uint, S_IRUGO); MODULE_PARM_DESC(mig_no_less_channels, "Prevent migration to system with less channels. " "[Default=" __stringify(IBMVFC_MIG_NO_N_TO_M) "]"); +module_param_named (nvme, nvme_enabled, uint,S_IRUGO ); +MODULE_PARM_DESC(nvme, "Enable NVMe over FC support. " + "[Default=" __stringify(IBMVFC_NVME) "]"); +module_param_named (nvme_host_queues, nr_nvme_hw_queues, uint,S_IRUGO ); +MODULE_PARM_DESC(nvme_host_queues, "Number of NVMeoF Host submission queues. " + "[Default=" __stringify(IBMVFC_NVME_HW_QUEUES) "]"); +module_param_named (nvme_hw_channels, nr_nvme_channels, uint,S_IRUGO ); +MODULE_PARM_DESC(nvme_hw_channels, "Number of hw NVMeoF channels to request. " + "[Default=" __stringify(IBMVFC_NVME_CHANNELS) "]"); +module_param_named (init_timeout, init_timeout, uint,S_IRUGO |S_IWUSR ); MODULE_PARM_DESC(init_timeout, "Initialization timeout in seconds. " "[Default=" __stringify(IBMVFC_INIT_TIMEOUT) "]");
@@ -468,6 +482,7 @@ static const struct { { IBMVFC_FABRIC_BUSY, "fabric busy" }, { IBMVFC_PORT_BUSY, "port busy" }, { IBMVFC_BASIC_REJECT, "basic reject" }, + { IBMVFC_FC4_LS_REJECT, "fc4 ls reject" }, };static const char *unknown_fc_type ="unknown fc type" ;
drivers/scsi/ibmvscsi/ibmvfc-nvme.c new file +87 -0
87 lines not on this page · raw message
drivers/scsi/ibmvscsi/ibmvfc-nvme.h new file +32 -0
@@ -0,0 +1,32 @@ +/* SPDX-License-Identifier: GPL-2.0-or-later */ + +/* +* ibmvfc-nvme.h -- IBM Power Virtual Fibre Channel NVMeoF HBA driver +* +* Written By: Tyrel Datwyler <[email protected]>, IBM Corporation +* +* Copyright (C) IBM Corporation, 2026 +*/ + +#ifndef _IBMVFC_NVME_H +#define _IBMVFC_NVME_H + +#include <uapi/scsi/fc/fc_fs.h> +#include <uapi/scsi/fc/fc_els.h> +#include <linux/nvme-fc-driver.h> + +#include "ibmvfc.h" + +#define IBMVFC_NVME 0 +#define IBMVFC_NVME_HW_QUEUES 8 +#define IBMVFC_MAX_NVME_QUEUES 16 +#define IBMVFC_NVME_CHANNELS 8 + +struct ibmvfc_host ; +struct ibmvfc_target ; + +int ibmvfc_nvme_register_remoteport (struct ibmvfc_target *tgt); +void ibmvfc_nvme_unregister_remoteport (struct ibmvfc_target *tgt); +int ibmvfc_nvme_register (struct ibmvfc_host *vhost); +void ibmvfc_nvme_unregister (struct ibmvfc_host *vhost); +#endif
drivers/scsi/ibmvscsi/ibmvfc.h +8 -2
@@ -1,22 +1,27 @@/* SPDX-License-Identifier: GPL-2.0-or-later */ +/* * ibmvfc.h -- driver for IBM Power Virtual Fibre Channel Adapter * * Written By: Brian King <[email protected]>, IBM Corporation * -* Copyright (C) IBM Corporation, 2008 +* Copyright (C) IBM Corporation, 2008-2026 */ #ifndef _IBMVFC_H#define _IBMVFC_H +#include <linux/interrupt.h> #include <linux/list.h> #include <linux/types.h> +#include <scsi/scsi_device.h> #include <scsi/viosrp.h> #include <linux/nvme.h> #include <linux/nvme-fc.h> -#define IBMVFC_NAME "ibmvfc" +#include "ibmvfc-nvme.h" + +#define IBMVFC_NAME "ibmvfc"#define IBMVFC_DRIVER_VERSION "1.0.11"#define IBMVFC_DRIVER_DATE "(April 12, 2013)"
@@ -984,6 +989,7 @@ struct ibmvfc_host { unsigned int mq_enabled:1; unsigned int using_channels:1; unsigned int do_enquiry:1; + unsigned int nvme_enabled:1; unsigned int aborting_passthru:1; unsigned int scan_complete:1;int scan_timeout;
Signature · 1 line
2.54.0
[email protected]
AI reviewRe: [PATCH 03/29] ibmvfc: split NVMe support into separate source file and add transport stubs
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [High] The driver misuses the NVMe-FC transport private data API by overwriting the `fcp_req->private`…
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [High] The driver misuses the NVMe-FC transport private data API by overwriting the fcp_req->private pointer instead of storing data within the allocated memory, leading to a kernel panic during AEN teardown. --
commit ff3f397be994beea2b3c34de8ca81ef76dd5fe89
Author: Tyrel Datwyler <[email protected]>
ibmvfc: split NVMe support into separate source file and add transport stubs
Rename ibmvfc.c to ibmvfc-core.c as first step in decoupling each protocol from the core driver logic. Add ibmvfc-nvme.[ch] files, and register an nvme_fc_port_template with empty callback stubs.
Quoted 5 lines
drivers/scsi/ibmvscsi/ibmvfc-nvme.c new file +0
-0
[ ... ]
Quoted 8 lines
drivers/scsi/ibmvscsi/ibmvfc-nvme.c new file +8
-0+static struct nvme_fc_port_template ibmvfc_nvme_fc_transport = { + .localport_delete = ibmvfc_nvme_localport_delete, + .remoteport_delete = ibmvfc_nvme_remoteport_delete, + .create_queue =NULL , + .delete_queue =NULL , + .ls_req = ibmvfc_nvme_ls_req, + .ls_abort = ibmvfc_nvme_ls_abort, + .fcp_io = ibmvfc_nvme_fcp_io,
drivers/scsi/ibmvscsi/ibmvfc-nvme.c line 68
Quoted 10 lines
drivers/scsi/ibmvscsi/ibmvfc-nvme.c new file +10
-0+ .fcp_abort = ibmvfc_nvme_fcp_abort, + .map_queues = NULL, + .max_hw_queues = IBMVFC_NVME_HW_QUEUES, + .max_sgl_segments = 1024, + .max_dif_sgl_segments = 64, + .dma_boundary = 0xFFFFFFFF, + .local_priv_sz = sizeof(struct ibmvfc_host *), + .remote_priv_sz = sizeof(struct ibmvfc_target *), + .lsrqst_priv_sz = sizeof(struct ibmvfc_event *), + .fcprqst_priv_sz = sizeof(struct ibmvfc_event *),
[Severity: High]
Will this misconfigure the transport private data API?
The NVMe-FC core allocates an 8-byte private data area for AEN requests using kzalloc() in nvme_fc_init_aen_ops() based on the fcprqst_priv_sz requested here.
If the driver later overwrites the fcp_req->private pointer directly with a statically allocated ibmvfc_event pointer (e.g., fcp_req->private = evt;) instead of storing it inside the allocated memory space, this will lead to a kernel panic.
During controller teardown (e.g., device removal or transport error recovery), nvme_fc_term_aen_ops() calls kfree(aen_op->fcp_req.private). Because the pointer would have been overwritten, kfree() is called on the non-kmalloc'd event pointer, resulting in a SLUB bug and memory leak.
Quoted 1 line
+};
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=3
Tyrel Datwyler
maintainerPATCH 4/29 ibmvfc: initialize NVMe channel configuration during driver probe
In reply to Tyrel DatwylerInitialize the host's NVMe channel-group state during probe.
Set up the NVMe channel list head, desired queue count, maximum queue count, protocol identifier, and enablement state alongside the existing SCSI channel-group initialization in ibmvfc_probe().
This prepares the driver with a NVMe/FC channel group that can will be used by later patches for NVMe queue allocation, discovery buffers, target management, and IO submission.
--- drivers/scsi/ibmvscsi/ibmvfc-core.c | 8 +++++++- drivers/scsi/ibmvscsi/ibmvfc.h | 1 + 2 files changed, 8 insertions(+), 1 deletion(-)
drivers/scsi/ibmvscsi/ibmvfc-core.c +7 -1
@@ -6337,7 +6337,8 @@ static int ibmvfc_probe(struct vio_dev *vdev, const struct vio_device_id *id)struct device *dev = &vdev->dev;int rc = -ENOMEM ;unsigned int online_cpus =num_online_cpus (); -unsigned int max_scsi_queues =min ((unsigned int )IBMVFC_MAX_SCSI_QUEUES , online_cpus); + unsigned int max_scsi_queues = min_t(unsigned int, IBMVFC_MAX_SCSI_QUEUES, online_cpus); + unsigned int max_nvme_queues = min_t(unsigned int, IBMVFC_MAX_NVME_QUEUES, online_cpus);ENTER ; shost =scsi_host_alloc (&driver_template,sizeof (*vhost));
@@ -6357,6 +6358,7 @@ static int ibmvfc_probe(struct vio_dev *vdev, const struct vio_device_id *id) vhost =shost_priv (shost);INIT_LIST_HEAD (&vhost->scsi_scrqs.targets); +INIT_LIST_HEAD (&vhost->nvme_scrqs.targets);INIT_LIST_HEAD (&vhost->purge);sprintf (vhost->name,IBMVFC_NAME ); vhost->host = shost;
@@ -6371,6 +6373,10 @@ static int ibmvfc_probe(struct vio_dev *vdev, const struct vio_device_id *id) vhost->scsi_scrqs.protocol =IBMVFC_PROTO_SCSI ; vhost->using_channels =0 ; vhost->do_enquiry =1 ; + vhost->nvme_enabled = mq_enabled ? nvme_enabled :0 ; + vhost->nvme_scrqs.desired_queues =min (max_nvme_queues, nr_nvme_channels); + vhost->nvme_scrqs.max_queues = max_nvme_queues; + vhost->nvme_scrqs.protocol =IBMVFC_PROTO_NVME ; vhost->scan_timeout =0 ;strcpy (vhost->partition_name,"UNKNOWN" );
drivers/scsi/ibmvscsi/ibmvfc.h +1 -0
@@ -970,6 +970,7 @@ struct ibmvfc_host {struct ibmvfc_queue crq;struct ibmvfc_queue async_crq;struct ibmvfc_channels scsi_scrqs; +struct ibmvfc_channels nvme_scrqs;struct ibmvfc_npiv_login login_info;union ibmvfc_npiv_login_data *login_buf;dma_addr_t login_buf_dma;
Signature · 1 line
2.54.0
[email protected]
AI reviewRe: [PATCH 04/29] ibmvfc: initialize NVMe channel configuration during driver probe
Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider: - [Medium] The user-configured module parameter for the number of NVMe hardware submission queues…
Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
- [Medium] The user-configured module parameter for the number of NVMe hardware submission queues (nvme_host_queues) is ignored during NVMe queue initialization, preventing users from capping the maximum number of NVMe host queues.
- [Medium] Bitfield truncation of unsigned integer module parameters can silently disable NVMe or create an inconsistent state.
--
commit 8b0247aa812866f78c7701f707162de73d472772
Author: Tyrel Datwyler <[email protected]>
ibmvfc: initialize NVMe channel configuration during driver probe
Initialize the host's NVMe channel-group state during probe. This prepares the driver with a NVMe/FC channel group that can will be used by later patches for NVMe queue allocation, discovery buffers, target management, and IO submission.
Quoted 13 lines
drivers/scsi/ibmvscsi/ibmvfc-core.c +2
-1@@ -6337,7 +6337,8 @@ static int ibmvfc_probe(struct vio_dev *vdev, const struct vio_device_id *id)struct device *dev = &vdev->dev;int rc = -ENOMEM ;unsigned int online_cpus =num_online_cpus (); -unsigned int max_scsi_queues =min ((unsigned int )IBMVFC_MAX_SCSI_QUEUES , online_cpus); + unsigned int max_scsi_queues = min_t(unsigned int, IBMVFC_MAX_SCSI_QUEUES, online_cpus); + unsigned int max_nvme_queues = min_t(unsigned int, IBMVFC_MAX_NVME_QUEUES, online_cpus);ENTER ; shost =scsi_host_alloc (&driver_template,sizeof (*vhost));
[ ... ]
drivers/scsi/ibmvscsi/ibmvfc-core.c static int ibmvfc_probe(struct vio_dev *vdev, const struct vio_device_id *id) · line 6376
Quoted 5 lines
drivers/scsi/ibmvscsi/ibmvfc-core.c +1
-0@@ -6371,6 +6373,10 @@ static int ibmvfc_probe(struct vio_dev *vdev, const struct vio_device_id *id) vhost->scsi_scrqs.protocol =IBMVFC_PROTO_SCSI ; vhost->using_channels =0 ; vhost->do_enquiry =1 ; + vhost->nvme_enabled = mq_enabled ? nvme_enabled :0 ;
[Severity: Medium]
Could this result in unintended bitfield truncation if an even value is passed via the module parameter?
Since vhost->nvme_enabled is a 1-bit bitfield, assigning an even value directly will truncate it to 0, unintentionally disabling NVMe.
Additionally, since the ternary operator checks the un-normalized mq_enabled integer directly, couldn't this create an inconsistent state where the ternary evaluates to true, but the underlying vhost->mq_enabled field was truncated to 0?
drivers/scsi/ibmvscsi/ibmvfc-core.c static int ibmvfc_probe(struct vio_dev *vdev, const struct vio_device_id *id) · line 6378
Quoted 2 lines
drivers/scsi/ibmvscsi/ibmvfc-core.c +2
-0+ vhost->nvme_scrqs.desired_queues =min (max_nvme_queues, nr_nvme_channels); + vhost->nvme_scrqs.max_queues = max_nvme_queues;
[Severity: Medium]
Should the maximum queue calculations be constrained by the nr_nvme_hw_queues module parameter?
The NVMe queue initialization currently ignores the user-configured nr_nvme_hw_queues limit, unconditionally defaulting to the max_nvme_queues limit based on IBMVFC_MAX_NVME_QUEUES and online CPUs.
Quoted 4 lines
drivers/scsi/ibmvscsi/ibmvfc-core.c +1
-0+ vhost->nvme_scrqs.protocol =IBMVFC_PROTO_NVME ; vhost->scan_timeout =0 ;strcpy (vhost->partition_name,"UNKNOWN" );
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=4
Tyrel Datwyler
maintainerPATCH 5/29 ibmvfc: alloc/dealloc sub-queues for nvme channels
In reply to Tyrel DatwylerAllocate, register, deregister, and release NVMe subordinate CRQs alongside the existing SCSI sub-CRQs.
Update the CRQ reset and re-enable paths to tear down and recreate NVMe sub-queues, extend sub-CRQ initialization to allocate NVMe channels when enabled, and release NVMe channel resources during adapter teardown.
This keeps the NVMe queue lifecycle aligned with the existing SCSI queue lifecycle so both protocols are reset consistently across probe, remove, and connection recovery.
--- drivers/scsi/ibmvscsi/ibmvfc-core.c | 15 ++++++++++++++- 1 file changed, 14 insertions(+), 1 deletion(-)
drivers/scsi/ibmvscsi/ibmvfc-core.c +14 -1
@@ -945,6 +945,7 @@ static int ibmvfc_reenable_crq_queue(struct ibmvfc_host *vhost)unsigned long flags;ibmvfc_dereg_sub_crqs (vhost, &vhost->scsi_scrqs); +ibmvfc_dereg_sub_crqs (vhost, &vhost->nvme_scrqs);/* Re-enable the CRQ */ do {
@@ -964,6 +965,7 @@ static int ibmvfc_reenable_crq_queue(struct ibmvfc_host *vhost)spin_unlock_irqrestore (vhost->host->host_lock, flags);ibmvfc_reg_sub_crqs (vhost, &vhost->scsi_scrqs); +ibmvfc_reg_sub_crqs (vhost, &vhost->nvme_scrqs);return rc; }
@@ -983,6 +985,7 @@ static int ibmvfc_reset_crq(struct ibmvfc_host *vhost)struct ibmvfc_queue *crq = &vhost->crq;ibmvfc_dereg_sub_crqs (vhost, &vhost->scsi_scrqs); +ibmvfc_dereg_sub_crqs (vhost, &vhost->nvme_scrqs);/* Close the CRQ */ do {
@@ -1016,6 +1019,7 @@ static int ibmvfc_reset_crq(struct ibmvfc_host *vhost)spin_unlock_irqrestore (vhost->host->host_lock, flags);ibmvfc_reg_sub_crqs (vhost, &vhost->scsi_scrqs); +ibmvfc_reg_sub_crqs (vhost, &vhost->nvme_scrqs);return rc; }
@@ -6109,6 +6113,13 @@ static void ibmvfc_init_sub_crqs(struct ibmvfc_host *vhost)ibmvfc_reg_sub_crqs (vhost, &vhost->scsi_scrqs); +if (vhost->nvme_enabled) { +if (ibmvfc_alloc_channels (vhost, &vhost->nvme_scrqs)) + vhost->nvme_enabled =0 ; +else +ibmvfc_reg_sub_crqs (vhost, &vhost->nvme_scrqs); + } +LEAVE ; }
@@ -6137,8 +6148,10 @@ static void ibmvfc_release_sub_crqs(struct ibmvfc_host *vhost)return ;ibmvfc_dereg_sub_crqs (vhost, &vhost->scsi_scrqs); -ibmvfc_release_channels (vhost, &vhost->scsi_scrqs); + +ibmvfc_dereg_sub_crqs (vhost, &vhost->nvme_scrqs); +ibmvfc_release_channels (vhost, &vhost->nvme_scrqs);LEAVE ; }
Signature · 1 line
2.54.0
[email protected]
AI reviewRe: [PATCH 05/29] ibmvfc: alloc/dealloc sub-queues for nvme channels
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [High] Failure to register NVMe sub-queues silently disables SCSI multiqueue support due to a shared…
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [High] Failure to register NVMe sub-queues silently disables SCSI multiqueue support due to a shared state flag. --
commit 6eb0f2ef7c51067f6c97ea8c13211afa2e19dcc4
Author: Tyrel Datwyler <[email protected]>
ibmvfc: alloc/dealloc sub-queues for nvme channels
This patch allocates, registers, deregisters, and releases NVMe subordinate CRQs alongside the existing SCSI sub-CRQs. It integrates the NVMe queue lifecycle with the existing SCSI queue lifecycle to ensure both protocols are reset consistently.
Quoted 4 lines
drivers/scsi/ibmvscsi/ibmvfc-core.c +0
-0
[ ... ]
drivers/scsi/ibmvscsi/ibmvfc-core.c static void ibmvfc_init_sub_crqs(struct ibmvfc_host *vhost) · line 6120
Quoted 9 lines
drivers/scsi/ibmvscsi/ibmvfc-core.c +5
-0@@ -6109,6 +6113,13 @@ static void ibmvfc_init_sub_crqs(struct ibmvfc_host *vhost)ibmvfc_reg_sub_crqs (vhost, &vhost->scsi_scrqs); +if (vhost->nvme_enabled) { +if (ibmvfc_alloc_channels (vhost, &vhost->nvme_scrqs)) + vhost->nvme_enabled =0 ; +else +ibmvfc_reg_sub_crqs (vhost, &vhost->nvme_scrqs);
[Severity: High]
Does a failure to register NVMe sub-queues unintentionally disable SCSI multiqueue support here?
If ibmvfc_reg_sub_crqs() fails while registering the NVMe channels, it handles the error by unregistering the partial channels and setting a shared host flag:
ibmvfc_reg_sub_crqs() {
...
if (ibmvfc_register_channel(vhost, channels, i)) {
for (j = i; j > 0; j--)
...
vhost->do_enquiry = 0;
return;
}
}Since ibmvfc_reg_sub_crqs() is now called for NVMe right after successfully registering the SCSI sub-queues, setting vhost->do_enquiry to 0 upon an NVMe registration failure will cause the driver to skip channel enquiry during the NPIV login sequence.
Will this cause the successfully registered SCSI channels to be silently ignored if the hypervisor rejects the NVMe sub-crqs allocation?
Quoted 5 lines
drivers/scsi/ibmvscsi/ibmvfc-core.c +2
-0+ } +LEAVE ; }
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=5
Tyrel Datwyler
maintainerPATCH 6/29 ibmvfc: add logic for protocol specific fabric logins
In reply to Tyrel DatwylerAdd support for the protocol-specific fabric login flow introduced by the updated client/VIOS interface.
After NPIV login, a VIOS that advertises protocol-specific support requires separate fabric login MADs for SCSI and NVMe/FC. Track whether SCSI and NVMe/FC fabric login are needed, extend channel enquiry/setup handling to negotiate both SCSI and NVMe queue counts, and issue the appropriate fabric login MADs before target discovery begins.
Also update command layout selection so the driver uses the v3 command format when the VIOS advertises NVMe/FC-capable framing.
--- drivers/scsi/ibmvscsi/ibmvfc-core.c | 230 ++++++++++++++++++++++++---- drivers/scsi/ibmvscsi/ibmvfc.h | 11 +- 2 files changed, 206 insertions(+), 35 deletions(-)
drivers/scsi/ibmvscsi/ibmvfc-core.c +199 -31
@@ -210,7 +210,9 @@ static int ibmvfc_check_caps(struct ibmvfc_host *vhost, unsigned long cap_flags)static struct ibmvfc_fcp_cmd_iu *ibmvfc_get_fcp_iu (struct ibmvfc_host *vhost,struct ibmvfc_cmd *vfc_cmd) { -if (ibmvfc_check_caps (vhost,IBMVFC_HANDLE_VF_WWPN )) +if (ibmvfc_check_caps (vhost,IBMVFC_SUPPORT_NVMEOF )) +return &vfc_cmd->v3scsi.iu; +else if (ibmvfc_check_caps (vhost,IBMVFC_HANDLE_VF_WWPN ))return &vfc_cmd->v2.iu;else return &vfc_cmd->v1.iu;
@@ -219,7 +221,9 @@ static struct ibmvfc_fcp_cmd_iu *ibmvfc_get_fcp_iu(struct ibmvfc_host *vhost,static struct ibmvfc_fcp_rsp *ibmvfc_get_fcp_rsp (struct ibmvfc_host *vhost,struct ibmvfc_cmd *vfc_cmd) { -if (ibmvfc_check_caps (vhost,IBMVFC_HANDLE_VF_WWPN )) +if (ibmvfc_check_caps (vhost,IBMVFC_SUPPORT_NVMEOF )) +return &vfc_cmd->v3scsi.rsp; +else if (ibmvfc_check_caps (vhost,IBMVFC_HANDLE_VF_WWPN ))return &vfc_cmd->v2.rsp;else return &vfc_cmd->v1.rsp;
@@ -961,6 +965,8 @@ static int ibmvfc_reenable_crq_queue(struct ibmvfc_host *vhost)spin_lock (vhost->crq.q_lock); vhost->do_enquiry =1 ; vhost->using_channels =0 ; + vhost->do_scsi_login =0 ; + vhost->do_nvme_login =0 ;spin_unlock (vhost->crq.q_lock);spin_unlock_irqrestore (vhost->host->host_lock, flags);
@@ -1000,6 +1006,8 @@ static int ibmvfc_reset_crq(struct ibmvfc_host *vhost) vhost->logged_in =0 ; vhost->do_enquiry =1 ; vhost->using_channels =0 ; + vhost->do_scsi_login =0 ; + vhost->do_nvme_login =0 ;/* Clean out the queue */ memset (crq->msgs.crq,0 ,PAGE_SIZE );
@@ -1512,7 +1520,7 @@ static void ibmvfc_set_login_info(struct ibmvfc_host *vhost) max_cmds = scsi_qdepth +IBMVFC_NUM_INTERNAL_REQ ;if (mq_enabled) max_cmds += (scsi_qdepth +IBMVFC_NUM_INTERNAL_SUBQ_REQ ) * - vhost->scsi_scrqs.desired_queues; + (vhost->scsi_scrqs.desired_queues + vhost->nvme_scrqs.desired_queues);memset (login_info,0 ,sizeof (*login_info));
@@ -1530,8 +1538,14 @@ static void ibmvfc_set_login_info(struct ibmvfc_host *vhost) login_info->max_cmds =cpu_to_be32 (max_cmds); login_info->capabilities =cpu_to_be64 (IBMVFC_CAN_MIGRATE |IBMVFC_CAN_SEND_VF_WWPN ); -if (vhost->mq_enabled || vhost->using_channels) +if (vhost->mq_enabled || vhost->using_channels) { login_info->capabilities |=cpu_to_be64 (IBMVFC_CAN_USE_CHANNELS ); +if (vhost->nvme_enabled) { + login_info->capabilities |=cpu_to_be64 (IBMVFC_YES_NVMEOF ); + login_info->capabilities |=cpu_to_be64 (IBMVFC_YES_SCSI ); + login_info->capabilities |=cpu_to_be64 (IBMVFC_CAN_USE_WWPN_ALL ); + } + } login_info->async.va =cpu_to_be64 (vhost->async_crq.msg_token); login_info->async.len = cpu_to_be32(async_crq->size *
@@ -1954,11 +1968,13 @@ static struct ibmvfc_cmd *ibmvfc_init_vfc_cmd(struct ibmvfc_event *evt, struct ssize_t offset;memset (vfc_cmd,0 ,sizeof (*vfc_cmd)); -if (ibmvfc_check_caps (vhost,IBMVFC_HANDLE_VF_WWPN )) { +if (ibmvfc_check_caps (vhost,IBMVFC_SUPPORT_NVMEOF )) + offset = offsetof(struct ibmvfc_cmd, v3scsi.rsp); +else if (ibmvfc_check_caps (vhost,IBMVFC_HANDLE_VF_WWPN )) offset = offsetof(struct ibmvfc_cmd, v2.rsp); - vfc_cmd->target_wwpn =cpu_to_be64 (rport->port_name); - }else +else offset = offsetof(struct ibmvfc_cmd, v1.rsp); + vfc_cmd->target_wwpn =cpu_to_be64 (rport->port_name); vfc_cmd->resp.va =cpu_to_be64 (be64_to_cpu (evt->crq.ioba) + offset); vfc_cmd->resp.len =cpu_to_be32 (sizeof (*rsp)); vfc_cmd->frame_type =cpu_to_be32 (IBMVFC_SCSI_FCP_TYPE );
@@ -5042,14 +5058,141 @@ static void ibmvfc_discover_targets(struct ibmvfc_host *vhost)ibmvfc_link_down (vhost,IBMVFC_LINK_DEAD ); } +static void ibmvfc_fabric_login_nvme_done (struct ibmvfc_event *evt) +{ +struct ibmvfc_host *vhost = evt->vhost; +struct ibmvfc_fabric_login_mad *rsp = &evt->xfer_iu->fabric_login; +u32 mad_status =be16_to_cpu (rsp->common.status); +int level =IBMVFC_DEFAULT_LOG_LEVEL ; + +switch (mad_status) { +case IBMVFC_MAD_SUCCESS : +ibmvfc_dbg (vhost,"NVMe fabric login succeeded\n" ); +break ; +case IBMVFC_MAD_FAILED : +if (ibmvfc_retry_cmd (be16_to_cpu (rsp->status),be16_to_cpu (rsp->error))) + level +=ibmvfc_retry_host_init (vhost); +else +ibmvfc_link_down (vhost,IBMVFC_LINK_DEAD ); +ibmvfc_log (vhost, level,"NVMe fabric login failed: %s (%x:%x)\n" , +ibmvfc_get_cmd_error (be16_to_cpu (rsp->status),be16_to_cpu (rsp->error)), +be16_to_cpu (rsp->status),be16_to_cpu (rsp->error)); +ibmvfc_free_event (evt); +return ; +case IBMVFC_MAD_DRIVER_FAILED : +ibmvfc_free_event (evt); +return ; +default : +dev_err (vhost->dev,"Invalid NVMe fabric login response: 0x%x\n" , mad_status); +ibmvfc_link_down (vhost,IBMVFC_LINK_DEAD ); +break ; + } + +ibmvfc_free_event (evt); + +ibmvfc_set_host_action (vhost,IBMVFC_HOST_ACTION_QUERY ); +wake_up (&vhost->work_wait_q); +} + +static void ibmvfc_fabric_login_nvme (struct ibmvfc_host *vhost) +{ +struct ibmvfc_fabric_login_mad *mad; +struct ibmvfc_event *evt =ibmvfc_get_reserved_event (&vhost->crq); + +if (!evt) { +ibmvfc_retry_host_init (vhost); +return ; + } + +ibmvfc_init_event (evt, ibmvfc_fabric_login_nvme_done,IBMVFC_MAD_FORMAT ); + mad = &evt->iu.fabric_login; +memset (mad,0 ,sizeof (*mad)); + mad->common.version =cpu_to_be32 (1 ); + mad->common.opcode =cpu_to_be32 (IBMVFC_NVMF_FABRIC_LOGIN ); + mad->common.length =cpu_to_be16 (sizeof (*mad)); + +ibmvfc_set_host_action (vhost,IBMVFC_HOST_ACTION_INIT_WAIT ); + +if (!ibmvfc_send_event (evt, vhost, default_timeout)) +ibmvfc_dbg (vhost,"Send NVMe fabric login\n" ); +else +ibmvfc_link_down (vhost,IBMVFC_LINK_DEAD ); +} + +static void ibmvfc_fabric_login_scsi_done (struct ibmvfc_event *evt) +{ +struct ibmvfc_host *vhost = evt->vhost; +struct ibmvfc_fabric_login_mad *rsp = &evt->xfer_iu->fabric_login; +u32 mad_status =be16_to_cpu (rsp->common.status); +int level =IBMVFC_DEFAULT_LOG_LEVEL ; + +switch (mad_status) { +case IBMVFC_MAD_SUCCESS : +ibmvfc_dbg (vhost,"SCSI fabric login succeeded\n" ); +break ; +case IBMVFC_MAD_FAILED : +if (ibmvfc_retry_cmd (be16_to_cpu (rsp->status),be16_to_cpu (rsp->error))) + level +=ibmvfc_retry_host_init (vhost); +else +ibmvfc_link_down (vhost,IBMVFC_LINK_DEAD ); +ibmvfc_log (vhost, level,"SCSI fabric login failed: %s (%x:%x)\n" , +ibmvfc_get_cmd_error (be16_to_cpu (rsp->status),be16_to_cpu (rsp->error)), +be16_to_cpu (rsp->status),be16_to_cpu (rsp->error)); +ibmvfc_free_event (evt); +return ; +case IBMVFC_MAD_DRIVER_FAILED : +ibmvfc_free_event (evt); +return ; +default : +dev_err (vhost->dev,"Invalid SCSI fabric login response: 0x%x\n" , mad_status); +ibmvfc_link_down (vhost,IBMVFC_LINK_DEAD ); +break ; + } + +ibmvfc_free_event (evt); + +if (vhost->do_nvme_login) { +ibmvfc_fabric_login_nvme (vhost); + }else { +ibmvfc_set_host_action (vhost,IBMVFC_HOST_ACTION_QUERY ); +wake_up (&vhost->work_wait_q); + } +} + +static void ibmvfc_fabric_login_scsi (struct ibmvfc_host *vhost) +{ +struct ibmvfc_fabric_login_mad *mad; +struct ibmvfc_event *evt =ibmvfc_get_reserved_event (&vhost->crq); + +if (!evt) { +ibmvfc_retry_host_init (vhost); +return ; + } + +ibmvfc_init_event (evt, ibmvfc_fabric_login_scsi_done,IBMVFC_MAD_FORMAT ); + mad = &evt->iu.fabric_login; +memset (mad,0 ,sizeof (*mad)); + mad->common.version =cpu_to_be32 (1 ); + mad->common.opcode =cpu_to_be32 (IBMVFC_FABRIC_LOGIN ); + mad->common.length =cpu_to_be16 (sizeof (*mad)); + +ibmvfc_set_host_action (vhost,IBMVFC_HOST_ACTION_INIT_WAIT ); + +if (!ibmvfc_send_event (evt, vhost, default_timeout)) +ibmvfc_dbg (vhost,"Send SCSI fabric login\n" ); +else +ibmvfc_link_down (vhost,IBMVFC_LINK_DEAD ); +} +static void ibmvfc_channel_setup_done (struct ibmvfc_event *evt) {struct ibmvfc_host *vhost = evt->vhost;struct ibmvfc_channel_setup *setup = vhost->channel_setup_buf; -struct ibmvfc_channels *scrqs = &vhost->scsi_scrqs; +struct ibmvfc_channels *scsi = &vhost->scsi_scrqs; +struct ibmvfc_channels *nvme = &vhost->nvme_scrqs;u32 mad_status =be16_to_cpu (evt->xfer_iu->channel_setup.common.status);int level =IBMVFC_DEFAULT_LOG_LEVEL ; -int flags, active_queues, i; +int flags, i;ibmvfc_free_event (evt);
@@ -5058,22 +5201,28 @@ static void ibmvfc_channel_setup_done(struct ibmvfc_event *evt)ibmvfc_dbg (vhost,"Channel Setup succeeded\n" ); flags =be32_to_cpu (setup->flags); vhost->do_enquiry =0 ; - active_queues =be32_to_cpu (setup->num_scsi_subq_channels); - scrqs->active_queues = active_queues; + scsi->active_queues =be32_to_cpu (setup->num_scsi_subq_channels); + nvme->active_queues =be32_to_cpu (setup->num_nvme_subq_channels);if (flags &IBMVFC_CHANNELS_CANCELED ) {ibmvfc_dbg (vhost,"Channels Canceled\n" ); vhost->using_channels =0 ; - }else { -if (active_queues) - vhost->using_channels =1 ; -for (i =0 ; i < active_queues; i++) - scrqs->scrqs[i].vios_cookie = -be64_to_cpu (setup->channel_handles[i]); - -ibmvfc_dbg (vhost,"Using %u channels\n" , - vhost->scsi_scrqs.active_queues); +break ; } + +if (scsi->active_queues || nvme->active_queues) + vhost->using_channels =1 ; +for (i =0 ; i < scsi->active_queues; i++) + scsi->scrqs[i].vios_cookie = +be64_to_cpu (setup->channel_handles[i]); +for (i =0 ; i < nvme->active_queues; i++) + nvme->scrqs[i].vios_cookie = +be64_to_cpu (setup->channel_handles[scsi->active_queues + i]); + +ibmvfc_dbg (vhost,"Using %u SCSI channels\n" , + scsi->active_queues); +ibmvfc_dbg (vhost,"Using %u NVMe channels\n" , + nvme->active_queues);break ;case IBMVFC_MAD_FAILED : level +=ibmvfc_retry_host_init (vhost);
@@ -5088,8 +5237,12 @@ static void ibmvfc_channel_setup_done(struct ibmvfc_event *evt)return ; } -ibmvfc_set_host_action (vhost,IBMVFC_HOST_ACTION_QUERY ); -wake_up (&vhost->work_wait_q); +if (vhost->do_scsi_login) { +ibmvfc_fabric_login_scsi (vhost); + }else { +ibmvfc_set_host_action (vhost,IBMVFC_HOST_ACTION_QUERY ); +wake_up (&vhost->work_wait_q); + } }static void ibmvfc_channel_setup (struct ibmvfc_host *vhost)
@@ -5097,9 +5250,12 @@ static void ibmvfc_channel_setup(struct ibmvfc_host *vhost)struct ibmvfc_channel_setup_mad *mad;struct ibmvfc_channel_setup *setup_buf = vhost->channel_setup_buf;struct ibmvfc_event *evt =ibmvfc_get_reserved_event (&vhost->crq); -struct ibmvfc_channels *scrqs = &vhost->scsi_scrqs; -unsigned int num_channels = -min (scrqs->desired_queues, vhost->max_vios_scsi_channels); +struct ibmvfc_channels *scsi = &vhost->scsi_scrqs; +struct ibmvfc_channels *nvme = &vhost->nvme_scrqs; +unsigned int scsi_channels = +min (scsi->desired_queues, vhost->max_vios_scsi_channels); +unsigned int nvme_channels = +min (nvme->desired_queues, vhost->max_vios_nvme_channels);int level =IBMVFC_DEFAULT_LOG_LEVEL ;int i;
@@ -5110,12 +5266,17 @@ static void ibmvfc_channel_setup(struct ibmvfc_host *vhost) }memset (setup_buf,0 ,sizeof (*setup_buf)); -if (num_channels ==0 ) +if (!scsi_channels && !nvme_channels) setup_buf->flags =cpu_to_be32 (IBMVFC_CANCEL_CHANNELS );else { - setup_buf->num_scsi_subq_channels =cpu_to_be32 (num_channels); -for (i =0 ; i < num_channels; i++) - setup_buf->channel_handles[i] =cpu_to_be64 (scrqs->scrqs[i].cookie); + setup_buf->num_scsi_subq_channels =cpu_to_be32 (scsi_channels); + setup_buf->num_nvme_subq_channels =cpu_to_be32 (nvme_channels); +for (i =0 ; i < scsi_channels; i++) + setup_buf->channel_handles[i] = +cpu_to_be64 (scsi->scrqs[i].cookie); +for (i =0 ; i < nvme_channels; i++) + setup_buf->channel_handles[scsi_channels + i] = +cpu_to_be64 (nvme->scrqs[i].cookie); }ibmvfc_init_event (evt, ibmvfc_channel_setup_done,IBMVFC_MAD_FORMAT );
@@ -5146,6 +5307,7 @@ static void ibmvfc_channel_enquiry_done(struct ibmvfc_event *evt)case IBMVFC_MAD_SUCCESS :ibmvfc_dbg (vhost,"Channel Enquiry succeeded\n" ); vhost->max_vios_scsi_channels =be32_to_cpu (rsp->num_scsi_subq_channels); + vhost->max_vios_nvme_channels =be32_to_cpu (rsp->num_nvme_subq_channels);ibmvfc_free_event (evt);break ;case IBMVFC_MAD_FAILED :
@@ -5280,8 +5442,14 @@ static void ibmvfc_npiv_login_done(struct ibmvfc_event *evt) vhost->host->can_queue =be32_to_cpu (rsp->max_cmds) -IBMVFC_NUM_INTERNAL_REQ ; vhost->host->max_sectors = npiv_max_sectors; -if (ibmvfc_check_caps (vhost,IBMVFC_CAN_SUPPORT_CHANNELS ) && vhost->do_enquiry) { -ibmvfc_channel_enquiry (vhost); + +if (ibmvfc_check_caps (vhost,IBMVFC_CAN_SUPPORT_CHANNELS )) { +if (ibmvfc_check_caps (vhost,IBMVFC_SUPPORT_SCSI )) + vhost->do_scsi_login =1 ; +if (ibmvfc_check_caps (vhost,IBMVFC_SUPPORT_NVMEOF )) + vhost->do_nvme_login =1 ; +if (vhost->do_enquiry) +ibmvfc_channel_enquiry (vhost); }else { vhost->do_enquiry =0 ;ibmvfc_set_host_action (vhost,IBMVFC_HOST_ACTION_QUERY );
drivers/scsi/ibmvscsi/ibmvfc.h +7 -4
@@ -617,9 +617,9 @@ struct ibmvfc_channel_enquiry {#define IBMVFC_SUPPORT_VARIABLE_SUBQ_MSG 0x02#define IBMVFC_NO_N_TO_M_CHANNELS_SUPPORT 0x04__be32 num_scsi_subq_channels; -__be32 num_nvmeof_subq_channels; +__be32 num_nvme_subq_channels;__be32 num_scsi_vas_channels; -__be32 num_nvmeof_vas_channels; +__be32 num_nvme_vas_channels; } __packed__aligned(8) ;struct ibmvfc_channel_setup_mad {
@@ -636,9 +636,9 @@ struct ibmvfc_channel_setup {#define IBMVFC_CHANNELS_CANCELED 0x04__be32 reserved;__be32 num_scsi_subq_channels; -__be32 num_nvmeof_subq_channels; +__be32 num_nvme_subq_channels;__be32 num_scsi_vas_channels; -__be32 num_nvmeof_vas_channels; +__be32 num_nvme_vas_channels;struct srp_direct_buf buffer;__be64 reserved2[5 ];__be64 channel_handles[IBMVFC_MAX_CHANNELS ];
@@ -979,6 +979,7 @@ struct ibmvfc_host {int log_level;struct mutex passthru_mutex;unsigned int max_vios_scsi_channels; +unsigned int max_vios_nvme_channels;int task_set;int init_retries;int discovery_threads;
@@ -991,6 +992,8 @@ struct ibmvfc_host { unsigned int using_channels:1; unsigned int do_enquiry:1; unsigned int nvme_enabled:1; + unsigned int do_scsi_login:1; + unsigned int do_nvme_login:1; unsigned int aborting_passthru:1; unsigned int scan_complete:1;int scan_timeout;
Signature · 1 line
2.54.0
[email protected]
AI reviewRe: [PATCH 06/29] ibmvfc: add logic for protocol specific fabric logins
Thank you for your contribution! Sashiko AI review found 6 potential issue(s) to consider:
Thank you for your contribution! Sashiko AI review found 6 potential issue(s) to consider:
New issues:
- [High] Host initialization stalls indefinitely if do_enquiry is false during NPIV login completion.
- [High] NVMe fabric login is entirely skipped if SCSI fabric login is not required.
- [High] Concurrent target discovery MADs cause a data race and state machine failure, dropping targets.
- [High] Stale do_scsi_login and do_nvme_login flags persist across NPIV logins, causing unsupported MAD requests.
Pre-existing issues:
- [High] Missing bounds check on VIOS-provided queue counts causes arbitrary out-of-bounds heap write.
- [High] Integer overflow in max_cmds leads to SCSI command queue depth under-allocation and negative can_queue bounds. --
commit d57e35c4c58c31909c3240ab400f9b9c8c0b9a8b
Author: Tyrel Datwyler <[email protected]>
ibmvfc: add logic for protocol specific fabric logins
This commit adds support for the protocol-specific fabric login flow introduced by the updated client/VIOS interface. It tracks whether SCSI and NVMe/FC fabric logins are needed and negotiates queue counts for both protocols. The driver now issues appropriate fabric login MADs before target discovery begins.
Quoted 4 lines
drivers/scsi/ibmvscsi/ibmvfc-core.c +0
-0
[ ... ]
drivers/scsi/ibmvscsi/ibmvfc-core.c static void ibmvfc_set_login_info(struct ibmvfc_host *vhost) · line 1525
Quoted 8 lines
drivers/scsi/ibmvscsi/ibmvfc-core.c +1
-1@@ -1512,7 +1520,7 @@ static void ibmvfc_set_login_info(struct ibmvfc_host *vhost) max_cmds = scsi_qdepth +IBMVFC_NUM_INTERNAL_REQ ;if (mq_enabled) max_cmds += (scsi_qdepth +IBMVFC_NUM_INTERNAL_SUBQ_REQ ) * - vhost->scsi_scrqs.desired_queues; + (vhost->scsi_scrqs.desired_queues + vhost->nvme_scrqs.desired_queues);memset (login_info,0 ,sizeof (*login_info));
[Severity: High]
This is a pre-existing issue, but does max_cmds overflow here?
The max_cmds variable is a u16. When mq_enabled is true, adding the nvme desired queues effectively doubles the multiplier. On systems with high core counts, this calculation could silently overflow the 65,535 limit of the u16.
If the wrapped value is smaller than IBMVFC_NUM_INTERNAL_REQ, can_queue becomes negative, which could induce failures during SCSI command allocations.
[ ... ]
Quoted 1 line
drivers/scsi/ibmvscsi/ibmvfc-core.c +0
-0@@ -5058,22 +5201,28 @@ static void ibmvfc_channel_setup_done(struct ibmvfc_event *evt)
[ ... ]
Quoted 18 lines
drivers/scsi/ibmvscsi/ibmvfc-core.c +3
-11- active_queues =be32_to_cpu (setup->num_scsi_subq_channels); - scrqs->active_queues = active_queues; + scsi->active_queues =be32_to_cpu (setup->num_scsi_subq_channels); + nvme->active_queues =be32_to_cpu (setup->num_nvme_subq_channels);if (flags &IBMVFC_CHANNELS_CANCELED ) {ibmvfc_dbg (vhost,"Channels Canceled\n" ); vhost->using_channels =0 ; - }else { -if (active_queues) - vhost->using_channels =1 ; -for (i =0 ; i < active_queues; i++) - scrqs->scrqs[i].vios_cookie = -be64_to_cpu (setup->channel_handles[i]); - -ibmvfc_dbg (vhost,"Using %u channels\n" , - vhost->scsi_scrqs.active_queues); +break ;
drivers/scsi/ibmvscsi/ibmvfc-core.c static void ibmvfc_channel_setup_done(struct ibmvfc_event *evt) · line 5220
Quoted 10 lines
drivers/scsi/ibmvscsi/ibmvfc-core.c +9
-0} + +if (scsi->active_queues || nvme->active_queues) + vhost->using_channels =1 ; +for (i =0 ; i < scsi->active_queues; i++) + scsi->scrqs[i].vios_cookie = +be64_to_cpu (setup->channel_handles[i]); +for (i =0 ; i < nvme->active_queues; i++) + nvme->scrqs[i].vios_cookie = +be64_to_cpu (setup->channel_handles[scsi->active_queues + i]);
[Severity: High]
This isn't a bug introduced by this patch, but do these loops need bounds checking on the VIOS-provided queue counts?
The driver assigns the untrusted num_scsi_subq_channels and num_nvme_subq_channels values directly to active_queues. The code then uses these values as boundary limits in the for loops to write into the scrqs arrays.
Could a buggy or compromised VIOS returning a massive queue count cause an out-of-bounds array write into the kernel heap?
[ ... ]
drivers/scsi/ibmvscsi/ibmvfc-core.c static void ibmvfc_channel_setup_done(struct ibmvfc_event *evt) · line 5246
Quoted 13 lines
drivers/scsi/ibmvscsi/ibmvfc-core.c +6
-2@@ -5088,8 +5237,12 @@ static void ibmvfc_channel_setup_done(struct ibmvfc_event *evt)return ; } -ibmvfc_set_host_action (vhost,IBMVFC_HOST_ACTION_QUERY ); -wake_up (&vhost->work_wait_q); +if (vhost->do_scsi_login) { +ibmvfc_fabric_login_scsi (vhost); + }else { +ibmvfc_set_host_action (vhost,IBMVFC_HOST_ACTION_QUERY ); +wake_up (&vhost->work_wait_q); + } }
[Severity: High]
Does this bypass NVMe fabric login entirely if SCSI fabric login is not required?
If a VIOS advertises IBMVFC_SUPPORT_NVMEOF but not IBMVFC_SUPPORT_SCSI, vhost->do_scsi_login will be false while vhost->do_nvme_login is true. Because vhost->do_scsi_login is false, execution falls into the else block and advances the state to QUERY.
Since NVMe fabric login is only chained sequentially from the SCSI login completion handler, does this prevent NVMe target discovery entirely on NVMe-only setups?
[ ... ]
drivers/scsi/ibmvscsi/ibmvfc-core.c static void ibmvfc_npiv_login_done(struct ibmvfc_event *evt) · line 5450
Quoted 12 lines
drivers/scsi/ibmvscsi/ibmvfc-core.c +6
-2@@ -5280,8 +5442,14 @@ static void ibmvfc_npiv_login_done(struct ibmvfc_event *evt) vhost->host->can_queue =be32_to_cpu (rsp->max_cmds) -IBMVFC_NUM_INTERNAL_REQ ; vhost->host->max_sectors = npiv_max_sectors; -if (ibmvfc_check_caps (vhost,IBMVFC_CAN_SUPPORT_CHANNELS ) && vhost->do_enquiry) { -ibmvfc_channel_enquiry (vhost); + +if (ibmvfc_check_caps (vhost,IBMVFC_CAN_SUPPORT_CHANNELS )) { +if (ibmvfc_check_caps (vhost,IBMVFC_SUPPORT_SCSI )) + vhost->do_scsi_login =1 ; +if (ibmvfc_check_caps (vhost,IBMVFC_SUPPORT_NVMEOF )) + vhost->do_nvme_login =1 ;
[Severity: High]
Can stale do_scsi_login and do_nvme_login flags persist across NPIV logins here?
The flags are conditionally set based on NPIV login capabilities, but they are never cleared before checking. If a VIOS drops these capabilities in a subsequent NPIV login (for example, following a partition migration to a VIOS with different support), the flags might erroneously remain set to 1.
Could this cause the driver to issue unsupported IBMVFC_FABRIC_LOGIN MADs to the new VIOS?
drivers/scsi/ibmvscsi/ibmvfc-core.c static void ibmvfc_npiv_login_done(struct ibmvfc_event *evt) · line 5455
Quoted 5 lines
drivers/scsi/ibmvscsi/ibmvfc-core.c +2
-0+if (vhost->do_enquiry) +ibmvfc_channel_enquiry (vhost); }else { vhost->do_enquiry =0 ;ibmvfc_set_host_action (vhost,IBMVFC_HOST_ACTION_QUERY );
[Severity: High]
If vhost->do_enquiry is false, does the host initialization stall indefinitely?
The structural change placed the do_enquiry check inside the capabilities check block. If IBMVFC_CAN_SUPPORT_CHANNELS is true but vhost->do_enquiry is false (e.g. following a link flap), the driver sets the login flags and completes execution of the if block.
Because it bypasses the else block, it skips setting IBMVFC_HOST_ACTION_QUERY and waking the work queue. Does this leave the host state machine permanently stalled in IBMVFC_HOST_ACTION_INIT_WAIT with no outstanding events to advance it?
[Severity: High]
Since both protocols are now supported, do concurrent target discovery MADs cause a data race?
Looking at ibmvfc_discover_targets(), it appears to issue both SCSI and NVMe target discovery MADs to the VIOS concurrently:
ibmvfc_discover_targets()
evt = ibmvfc_get_disc_event(&vhost->scsi_scrqs);
ibmvfc_send_event(evt, vhost, default_timeout);
evt = ibmvfc_get_disc_event(&vhost->nvme_scrqs);
ibmvfc_send_event(evt, vhost, default_timeout);When the first MAD completes, ibmvfc_discover_targets_done() immediately advances the host action to ALLOC_TGTS and wakes the allocation worker thread.
Does the worker thread iterate over both discovery buffers while the second MAD is still in flight? Additionally, when the slower MAD completes, its request to set the host action is ignored because the state is no longer INIT_WAIT, potentially dropping the targets from the slower discovery operation.
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=6
Tyrel Datwyler
maintainerPATCH 7/29 ibmvfc: add wrapper to get vhost associated with a channel struct
In reply to Tyrel DatwylerAdd ibmvfc_channels_to_vhost() to recover the parent struct ibmvfc_host from a protocol-specific struct ibmvfc_channels.
Later patches need to operate on either the SCSI or NVMe channel group and still access host-wide state such as the primary CRQ, device, and logging context. Centralize that mapping in a helper instead of open- coding container lookups at each call site.
--- drivers/scsi/ibmvscsi/ibmvfc.h | 10 ++++++++++ 1 file changed, 10 insertions(+)
drivers/scsi/ibmvscsi/ibmvfc.h +10 -0
@@ -1011,6 +1011,16 @@ struct ibmvfc_host {wait_queue_head_t work_wait_q; }; +static inline struct ibmvfc_host *ibmvfc_channels_to_vhost (struct ibmvfc_channels *channels) +{ +if (channels->protocol ==IBMVFC_PROTO_SCSI ) + return container_of(channels, struct ibmvfc_host, scsi_scrqs); +else if (channels->protocol ==IBMVFC_PROTO_NVME ) + return container_of(channels, struct ibmvfc_host, nvme_scrqs); + +return NULL ; +} +#define DBG_CMD (CMD ) do { if (ibmvfc_debug) CMD; } while (0)#define tgt_dbg (t, fmt, ...) \
Signature · 1 line
2.54.0
Tyrel Datwyler
maintainerPATCH 8/29 ibmvfc: add helper for creating protocol specific discovery event
In reply to Tyrel DatwylerRefactor discover-target event creation so it can be shared by both SCSI and NVMe/FC discovery.
Introduce a helper that takes a protocol-specific channel group, selects the correct discover-target opcode, and maps the corresponding discovery buffer into the MAD.
This is a preparatory cleanup for issuing protocol-specific discovery MADs in later patches.
--- drivers/scsi/ibmvscsi/ibmvfc-core.c | 39 ++++++++++++++++++++--------- 1 file changed, 27 insertions(+), 12 deletions(-)
drivers/scsi/ibmvscsi/ibmvfc-core.c +27 -12
@@ -5023,6 +5023,32 @@ static void ibmvfc_discover_targets_done(struct ibmvfc_event *evt)wake_up (&vhost->work_wait_q); } +static struct ibmvfc_event *ibmvfc_get_disc_event (struct ibmvfc_channels *channels) +{ +struct ibmvfc_discover_targets *mad; +struct ibmvfc_host *vhost =ibmvfc_channels_to_vhost (channels); +struct ibmvfc_event *evt =ibmvfc_get_reserved_event (&vhost->crq); + +if (!evt) +return NULL ; + +ibmvfc_init_event (evt, ibmvfc_discover_targets_done,IBMVFC_MAD_FORMAT ); + mad = &evt->iu.discover_targets; +memset (mad,0 ,sizeof (*mad)); + mad->common.version =cpu_to_be32 (1 ); +if (channels->protocol ==IBMVFC_PROTO_SCSI ) + mad->common.opcode =cpu_to_be32 (IBMVFC_DISC_TARGETS ); +else + mad->common.opcode =cpu_to_be32 (IBMVFC_DISC_NVMF_TARGETS ); + mad->common.length =cpu_to_be16 (sizeof (*mad)); + mad->bufflen =cpu_to_be32 (channels->disc_buf_sz); + mad->buffer.va =cpu_to_be64 (channels->disc_buf_dma); + mad->buffer.len =cpu_to_be32 (channels->disc_buf_sz); + mad->flags =cpu_to_be32 (IBMVFC_DISC_TGT_PORT_ID_WWPN_LIST ); + +return evt; +} +/** * ibmvfc_discover_targets - Send Discover Targets MAD * @vhost: ibmvfc host struct
@@ -5030,8 +5056,7 @@ static void ibmvfc_discover_targets_done(struct ibmvfc_event *evt)**/ static void ibmvfc_discover_targets (struct ibmvfc_host *vhost) { -struct ibmvfc_discover_targets *mad; -struct ibmvfc_event *evt =ibmvfc_get_reserved_event (&vhost->crq); +struct ibmvfc_event *evt =ibmvfc_get_disc_event (&vhost->scsi_scrqs);int level =IBMVFC_DEFAULT_LOG_LEVEL ;if (!evt) {
@@ -5040,16 +5065,6 @@ static void ibmvfc_discover_targets(struct ibmvfc_host *vhost)return ; } -ibmvfc_init_event (evt, ibmvfc_discover_targets_done,IBMVFC_MAD_FORMAT ); - mad = &evt->iu.discover_targets; -memset (mad,0 ,sizeof (*mad)); - mad->common.version =cpu_to_be32 (1 ); - mad->common.opcode =cpu_to_be32 (IBMVFC_DISC_TARGETS ); - mad->common.length =cpu_to_be16 (sizeof (*mad)); - mad->bufflen =cpu_to_be32 (vhost->scsi_scrqs.disc_buf_sz); - mad->buffer.va =cpu_to_be64 (vhost->scsi_scrqs.disc_buf_dma); - mad->buffer.len =cpu_to_be32 (vhost->scsi_scrqs.disc_buf_sz); - mad->flags =cpu_to_be32 (IBMVFC_DISC_TGT_PORT_ID_WWPN_LIST );ibmvfc_set_host_action (vhost,IBMVFC_HOST_ACTION_INIT_WAIT );if (!ibmvfc_send_event (evt, vhost, default_timeout))
Signature · 1 line
2.54.0
Tyrel Datwyler
maintainerPATCH 9/29 ibmvfc: add helper to check NVMe/FC support with active channels
In reply to Tyrel DatwylerIt can be the case that NVMeoF is enabled on both the client and VIOS, but no queues are configured making the need to do NVMe target discovery pointless. Add a helper to short hand the capabilities check and active NVMe queue check.
--- drivers/scsi/ibmvscsi/ibmvfc-core.c | 6 ++++++ 1 file changed, 6 insertions(+)
drivers/scsi/ibmvscsi/ibmvfc-core.c +6 -0
@@ -207,6 +207,12 @@ static int ibmvfc_check_caps(struct ibmvfc_host *vhost, unsigned long cap_flags)return (host_caps & cap_flags) ?1 :0 ; } +static int ibmvfc_nvme_active (struct ibmvfc_host *vhost) +{ +return (ibmvfc_check_caps (vhost,IBMVFC_SUPPORT_NVMEOF ) && + vhost->nvme_scrqs.active_queues); +} +static struct ibmvfc_fcp_cmd_iu *ibmvfc_get_fcp_iu (struct ibmvfc_host *vhost,struct ibmvfc_cmd *vfc_cmd) {
Signature · 1 line
2.54.0
Tyrel Datwyler
maintainerPATCH 10/29 ibmvfc: allocate and free NVMe channel group discover buffer
In reply to Tyrel DatwylerAllocate a discovery buffer for the NVMe channel group and free it on all teardown and error paths.
The existing discovery-buffer allocation only covered the SCSI channel group. This patch is prepratory for sending NVMe/FC target discovery MAD.
--- drivers/scsi/ibmvscsi/ibmvfc-core.c | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-)
drivers/scsi/ibmvscsi/ibmvfc-core.c +7 -1
@@ -6365,6 +6365,7 @@ static void ibmvfc_free_mem(struct ibmvfc_host *vhost)mempool_destroy (vhost->tgt_pool);kfree (vhost->trace);ibmvfc_free_disc_buf (vhost->dev, &vhost->scsi_scrqs); +ibmvfc_free_disc_buf (vhost->dev, &vhost->nvme_scrqs);dma_free_coherent (vhost->dev,sizeof (*vhost->login_buf), vhost->login_buf, vhost->login_buf_dma); dma_free_coherent(vhost->dev, sizeof(*vhost->channel_setup_buf),
@@ -6427,12 +6428,15 @@ static int ibmvfc_alloc_mem(struct ibmvfc_host *vhost)if (ibmvfc_alloc_disc_buf (dev, &vhost->scsi_scrqs)) goto free_login_buffer; +if (ibmvfc_alloc_disc_buf (dev, &vhost->nvme_scrqs)) + goto free_scsi_disc_buffer; + vhost->trace = kzalloc_objs(struct ibmvfc_trace_entry,IBMVFC_NUM_TRACE_ENTRIES );atomic_set (&vhost->trace_index,-1 );if (!vhost->trace) - goto free_scsi_disc_buffer; + goto free_nvme_disc_buffer; vhost->tgt_pool =mempool_create_kmalloc_pool (IBMVFC_TGT_MEMPOOL_SZ ,sizeof (struct ibmvfc_target ));
@@ -6458,6 +6462,8 @@ static int ibmvfc_alloc_mem(struct ibmvfc_host *vhost)mempool_destroy (vhost->tgt_pool); free_trace:kfree (vhost->trace); +free_nvme_disc_buffer: +ibmvfc_free_disc_buf (dev, &vhost->nvme_scrqs); free_scsi_disc_buffer:ibmvfc_free_disc_buf (dev, &vhost->scsi_scrqs); free_login_buffer:
Signature · 1 line
2.54.0
Tyrel Datwyler
maintainerPATCH 11/29 ibmvfc: send NVMe target discovery MAD
In reply to Tyrel DatwylerExtend target discovery to send protocol-specific discover-target MADs for NVMe/FC.
Use the protocol-aware discovery helper to build an NVMe discover-target request, submit it when NVMe/FC support is active, and process the returned target count using the NVMe channel group's discovery buffer.
This allows the driver to discover NVMe/FC targets in parallel with the existing SCSI discovery flow while keeping protocol-specific target data separate.
--- drivers/scsi/ibmvscsi/ibmvfc-core.c | 50 +++++++++++++++++++++++++---- 1 file changed, 43 insertions(+), 7 deletions(-)
drivers/scsi/ibmvscsi/ibmvfc-core.c +43 -7
@@ -173,6 +173,11 @@ static const struct { { IBMVFC_FC_SCSI_ERROR, IBMVFC_COMMAND_FAILED, DID_ERROR, 0, 1, "PRLI to device failed." }, }; +const char *proto_type[] = { +"SCSI" , +"NVMe" , +}; +static void ibmvfc_npiv_login (struct ibmvfc_host *);static void ibmvfc_tgt_send_prli (struct ibmvfc_target *);static void ibmvfc_tgt_send_plogi (struct ibmvfc_target *);
@@ -5001,19 +5006,30 @@ static void ibmvfc_discover_targets_done(struct ibmvfc_event *evt) {struct ibmvfc_host *vhost = evt->vhost;struct ibmvfc_discover_targets *rsp = &evt->xfer_iu->discover_targets; +struct ibmvfc_channels *channels;u32 mad_status =be16_to_cpu (rsp->common.status); +u32 opcode =be32_to_cpu (rsp->common.opcode);int level =IBMVFC_DEFAULT_LOG_LEVEL ; +if (opcode ==IBMVFC_DISC_TARGETS ) + channels = &vhost->scsi_scrqs; +else + channels = &vhost->nvme_scrqs; +switch (mad_status) {case IBMVFC_MAD_SUCCESS : -ibmvfc_dbg (vhost,"Discover Targets succeeded\n" ); - vhost->scsi_scrqs.num_targets =min_t (u32,be32_to_cpu (rsp->num_written), - max_targets); +ibmvfc_dbg (vhost,"Discover %s Targets succeeded\n" , + proto_type[channels->protocol]); + channels->num_targets =min_t (u32,be32_to_cpu (rsp->num_written), + max_targets); +ibmvfc_dbg (vhost,"%d %s targets found\n" , channels->num_targets, + proto_type[channels->protocol]);ibmvfc_set_host_action (vhost,IBMVFC_HOST_ACTION_ALLOC_TGTS );break ;case IBMVFC_MAD_FAILED : level +=ibmvfc_retry_host_init (vhost); -ibmvfc_log (vhost, level,"Discover Targets failed: %s (%x:%x)\n" , +ibmvfc_log (vhost, level,"Discover %s Targets failed: %s (%x:%x)\n" , + proto_type[channels->protocol],ibmvfc_get_cmd_error (be16_to_cpu (rsp->status),be16_to_cpu (rsp->error)),be16_to_cpu (rsp->status),be16_to_cpu (rsp->error));break ;
@@ -5066,7 +5082,7 @@ static void ibmvfc_discover_targets(struct ibmvfc_host *vhost)int level =IBMVFC_DEFAULT_LOG_LEVEL ;if (!evt) { -ibmvfc_log (vhost, level,"Discover Targets failed: no available events\n" ); +ibmvfc_log (vhost, level,"Discover SCSI Targets failed: no available events\n" );ibmvfc_hard_reset_host (vhost);return ; }
@@ -5074,9 +5090,29 @@ static void ibmvfc_discover_targets(struct ibmvfc_host *vhost)ibmvfc_set_host_action (vhost,IBMVFC_HOST_ACTION_INIT_WAIT );if (!ibmvfc_send_event (evt, vhost, default_timeout)) -ibmvfc_dbg (vhost,"Sent discover targets\n" ); +ibmvfc_dbg (vhost,"Sent discover SCSI targets\n" );else -ibmvfc_link_down (vhost,IBMVFC_LINK_DEAD ); + goto link_down; + +if (!ibmvfc_nvme_active (vhost)) +return ; + + evt =ibmvfc_get_disc_event (&vhost->nvme_scrqs); +if (!evt) { +ibmvfc_log (vhost, level,"Discover NVMe Targets failed: no available events\n" ); +ibmvfc_hard_reset_host (vhost); +return ; + } + +if (!ibmvfc_send_event (evt, vhost, default_timeout)) +ibmvfc_dbg (vhost,"Sent discover NVMe targets\n" ); +else + goto link_down; + +return ; + +link_down: +ibmvfc_link_down (vhost,IBMVFC_LINK_DEAD ); }static void ibmvfc_fabric_login_nvme_done (struct ibmvfc_event *evt)
Signature · 1 line
2.54.0
[email protected]
AI reviewRe: [PATCH 11/29] ibmvfc: send NVMe target discovery MAD
Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider: - [High] Parallel SCSI and NVMe discover targets MADs race to advance the host initialization state…
Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
- [High] Parallel SCSI and NVMe discover targets MADs race to advance the host initialization state machine, causing the slower protocol's targets to be permanently ignored and breaking retry handling.
- [Low] The newly added proto_type array is missing the static keyword, making it a globally visible symbol.
--
commit e89ce7e682793b58e5019c746ffde18602fe635b
Author: Tyrel Datwyler <[email protected]>
ibmvfc: send NVMe target discovery MAD
This commit extends target discovery to send protocol-specific discover-target MADs for NVMe/FC. It introduces a parallel discovery flow for NVMe alongside SCSI targets.
drivers/scsi/ibmvscsi/ibmvfc-core.c static const struct { · line 179
Quoted 12 lines
drivers/scsi/ibmvscsi/ibmvfc-core.c +4
-0@@ -173,6 +173,11 @@ static const struct { { IBMVFC_FC_SCSI_ERROR, IBMVFC_COMMAND_FAILED, DID_ERROR, 0, 1, "PRLI to device failed." }, }; +const char *proto_type[] = { +"SCSI" , +"NVMe" , +};
[Severity: Low]
This isn't a bug, but can this array be declared as static? It is currently exposed to the global kernel namespace which could lead to linker collisions.
Quoted 4 lines
drivers/scsi/ibmvscsi/ibmvfc-core.c +1
-0+static void ibmvfc_npiv_login (struct ibmvfc_host *);static void ibmvfc_tgt_send_prli (struct ibmvfc_target *);static void ibmvfc_tgt_send_plogi (struct ibmvfc_target *);
[ ... ]
Quoted 16 lines
drivers/scsi/ibmvscsi/ibmvfc-core.c +7
-0@@ -5001,19 +5006,30 @@ static void ibmvfc_discover_targets_done(struct ibmvfc_event *evt) {struct ibmvfc_host *vhost = evt->vhost;struct ibmvfc_discover_targets *rsp = &evt->xfer_iu->discover_targets; +struct ibmvfc_channels *channels;u32 mad_status =be16_to_cpu (rsp->common.status); +u32 opcode =be32_to_cpu (rsp->common.opcode);int level =IBMVFC_DEFAULT_LOG_LEVEL ; +if (opcode ==IBMVFC_DISC_TARGETS ) + channels = &vhost->scsi_scrqs; +else + channels = &vhost->nvme_scrqs; +switch (mad_status) {case IBMVFC_MAD_SUCCESS :
drivers/scsi/ibmvscsi/ibmvfc-core.c static void ibmvfc_discover_targets_done(struct ibmvfc_event *evt) · line 5027
Quoted 10 lines
drivers/scsi/ibmvscsi/ibmvfc-core.c +6
-3-ibmvfc_dbg (vhost,"Discover Targets succeeded\n" ); - vhost->scsi_scrqs.num_targets =min_t (u32,be32_to_cpu (rsp->num_written), - max_targets); +ibmvfc_dbg (vhost,"Discover %s Targets succeeded\n" , + proto_type[channels->protocol]); + channels->num_targets =min_t (u32,be32_to_cpu (rsp->num_written), + max_targets); +ibmvfc_dbg (vhost,"%d %s targets found\n" , channels->num_targets, + proto_type[channels->protocol]);ibmvfc_set_host_action (vhost,IBMVFC_HOST_ACTION_ALLOC_TGTS );
[Severity: High]
Does this code create a race condition between the SCSI and NVMe discovery MADs?
If both SCSI and NVMe discoveries are running asynchronously, whichever MAD completes first will call ibmvfc_set_host_action() and advance the host state from IBMVFC_HOST_ACTION_INIT_WAIT to IBMVFC_HOST_ACTION_ALLOC_TGTS.
When the slower MAD subsequently completes, ibmvfc_set_host_action() will ignore the transition to IBMVFC_HOST_ACTION_ALLOC_TGTS because the state is no longer IBMVFC_HOST_ACTION_INIT_WAIT. This causes the targets from the slower protocol to be permanently ignored.
Additionally, if the first MAD fails, could the state machine revert to IBMVFC_HOST_ACTION_INIT and issue a new batch of discovery commands while the slower MAD is still in-flight?
Quoted 3 lines
drivers/scsi/ibmvscsi/ibmvfc-core.c +0
-0break ;case IBMVFC_MAD_FAILED : level +=ibmvfc_retry_host_init (vhost);
[ ... ]
Quoted 26 lines
drivers/scsi/ibmvscsi/ibmvfc-core.c +19
-2@@ -5074,9 +5090,29 @@ static void ibmvfc_discover_targets(struct ibmvfc_host *vhost)ibmvfc_set_host_action (vhost,IBMVFC_HOST_ACTION_INIT_WAIT );if (!ibmvfc_send_event (evt, vhost, default_timeout)) -ibmvfc_dbg (vhost,"Sent discover targets\n" ); +ibmvfc_dbg (vhost,"Sent discover SCSI targets\n" );else -ibmvfc_link_down (vhost,IBMVFC_LINK_DEAD ); + goto link_down; + +if (!ibmvfc_nvme_active (vhost)) +return ; + + evt =ibmvfc_get_disc_event (&vhost->nvme_scrqs); +if (!evt) { +ibmvfc_log (vhost, level,"Discover NVMe Targets failed: no available events\n" ); +ibmvfc_hard_reset_host (vhost); +return ; + } + +if (!ibmvfc_send_event (evt, vhost, default_timeout)) +ibmvfc_dbg (vhost,"Sent discover NVMe targets\n" ); +else + goto link_down; + +return ;
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=11
Tyrel Datwyler
maintainerPATCH 12/29 ibmvfc: add NVMe/FC Implicit Logout and Move Login support
In reply to Tyrel DatwylerAdd protocol-specific handling for implicit logout and move-login operations on NVMe/FC targets.
Select the NVMe/FC-specific implicit logout opcode when operating on an NVMe target and update the associated logging so protocol-specific operations are visible in debug output. This extends the existing target relogin and migration-related flows to work with NVMe targets as well as SCSI targets.
These changes are needed so target reauthentication and target movement continue to work once NVMe/FC targets are added to the driver's state machine.
--- drivers/scsi/ibmvscsi/ibmvfc-core.c | 33 +++++++++++++++++++---------- 1 file changed, 22 insertions(+), 11 deletions(-)
drivers/scsi/ibmvscsi/ibmvfc-core.c +22 -11
@@ -4316,7 +4316,7 @@ static void ibmvfc_tgt_implicit_logout_done(struct ibmvfc_event *evt)switch (status) {case IBMVFC_MAD_SUCCESS : -tgt_dbg (tgt,"Implicit Logout succeeded\n" ); +tgt_dbg (tgt,"%s Implicit Logout succeeded\n" , proto_type[tgt->protocol]);break ;case IBMVFC_MAD_DRIVER_FAILED :kref_put (&tgt->kref, ibmvfc_release_tgt);
@@ -4324,7 +4324,8 @@ static void ibmvfc_tgt_implicit_logout_done(struct ibmvfc_event *evt)return ;case IBMVFC_MAD_FAILED :default : -tgt_err (tgt,"Implicit Logout failed: rc=0x%02X\n" , status); +tgt_err (tgt,"%s Implicit Logout failed: rc=0x%02X\n" , + proto_type[tgt->protocol], status);break ; }
@@ -4357,7 +4358,10 @@ static struct ibmvfc_event *__ibmvfc_tgt_get_implicit_logout_evt(struct ibmvfc_t mad = &evt->iu.implicit_logout;memset (mad,0 ,sizeof (*mad)); mad->common.version =cpu_to_be32 (1 ); - mad->common.opcode =cpu_to_be32 (IBMVFC_IMPLICIT_LOGOUT ); +if (tgt->protocol ==IBMVFC_PROTO_SCSI ) + mad->common.opcode =cpu_to_be32 (IBMVFC_IMPLICIT_LOGOUT ); +else + mad->common.opcode =cpu_to_be32 (IBMVFC_NVMF_IMPLICIT_LOGOUT ); mad->common.length =cpu_to_be16 (sizeof (*mad)); mad->old_scsi_id =cpu_to_be64 (tgt->scsi_id);return evt;
@@ -4393,7 +4397,7 @@ static void ibmvfc_tgt_implicit_logout(struct ibmvfc_target *tgt)ibmvfc_set_tgt_action (tgt,IBMVFC_TGT_ACTION_NONE );kref_put (&tgt->kref, ibmvfc_release_tgt); }else -tgt_dbg (tgt,"Sent Implicit Logout\n" ); +tgt_dbg (tgt,"%s Sent Implicit Logout\n" , proto_type[tgt->protocol]); }/**
@@ -4423,7 +4427,8 @@ static void ibmvfc_tgt_implicit_logout_and_del_done(struct ibmvfc_event *evt)else ibmvfc_set_tgt_action (tgt,IBMVFC_TGT_ACTION_DEL_AND_LOGOUT_RPORT ); -tgt_dbg (tgt,"Implicit Logout %s\n" , (status ==IBMVFC_MAD_SUCCESS ) ?"succeeded" :"failed" ); +tgt_dbg (tgt,"%s Implicit Logout %s\n" , proto_type[tgt->protocol], + (status ==IBMVFC_MAD_SUCCESS ) ?"succeeded" :"failed" );kref_put (&tgt->kref, ibmvfc_release_tgt);wake_up (&vhost->work_wait_q); }
@@ -4456,7 +4461,7 @@ static void ibmvfc_tgt_implicit_logout_and_del(struct ibmvfc_target *tgt)ibmvfc_set_tgt_action (tgt,IBMVFC_TGT_ACTION_DEL_RPORT );kref_put (&tgt->kref, ibmvfc_release_tgt); }else -tgt_dbg (tgt,"Sent Implicit Logout\n" ); +tgt_dbg (tgt,"%s Sent Implicit Logout\n" , proto_type[tgt->protocol]); }/**
@@ -4476,7 +4481,8 @@ static void ibmvfc_tgt_move_login_done(struct ibmvfc_event *evt)ibmvfc_set_tgt_action (tgt,IBMVFC_TGT_ACTION_NONE );switch (status) {case IBMVFC_MAD_SUCCESS : -tgt_dbg (tgt,"Move Login succeeded for new scsi_id: %llX\n" , tgt->new_scsi_id); +tgt_dbg (tgt,"%s Move Login succeeded for new scsi_id: %llX\n" , + proto_type[tgt->protocol], tgt->new_scsi_id); tgt->ids.node_name =wwn_to_u64 (rsp->service_parms.node_name); tgt->ids.port_name =wwn_to_u64 (rsp->service_parms.port_name); tgt->scsi_id = tgt->new_scsi_id;
@@ -4497,8 +4503,9 @@ static void ibmvfc_tgt_move_login_done(struct ibmvfc_event *evt) level +=ibmvfc_retry_tgt_init (tgt, ibmvfc_tgt_move_login);tgt_log (tgt, level, -"Move Login failed: new scsi_id: %llX, flags:%x, vios_flags:%x, rc=0x%02X\n" , - tgt->new_scsi_id,be32_to_cpu (rsp->flags),be16_to_cpu (rsp->vios_flags), +"%s Move Login failed: new scsi_id: %llX, flags:%x, vios_flags:%x, rc=0x%02X\n" , + proto_type[tgt->protocol], tgt->new_scsi_id, +be32_to_cpu (rsp->flags),be16_to_cpu (rsp->vios_flags), status);break ; }
@@ -4538,7 +4545,10 @@ static void ibmvfc_tgt_move_login(struct ibmvfc_target *tgt) move = &evt->iu.move_login;memset (move,0 ,sizeof (*move)); move->common.version =cpu_to_be32 (1 ); - move->common.opcode =cpu_to_be32 (IBMVFC_MOVE_LOGIN ); +if (tgt->protocol ==IBMVFC_PROTO_SCSI ) + move->common.opcode =cpu_to_be32 (IBMVFC_MOVE_LOGIN ); +else + move->common.opcode =cpu_to_be32 (IBMVFC_NVMF_MOVE_LOGIN ); move->common.length =cpu_to_be16 (sizeof (*move)); move->old_scsi_id =cpu_to_be64 (tgt->scsi_id);
@@ -4551,7 +4561,8 @@ static void ibmvfc_tgt_move_login(struct ibmvfc_target *tgt)ibmvfc_set_tgt_action (tgt,IBMVFC_TGT_ACTION_DEL_RPORT );kref_put (&tgt->kref, ibmvfc_release_tgt); }else -tgt_dbg (tgt,"Sent Move Login for new scsi_id: %llX\n" , tgt->new_scsi_id); +tgt_dbg (tgt,"Sent %s Move Login for new scsi_id: %llX\n" , + proto_type[tgt->protocol], tgt->new_scsi_id); }/**
Signature · 1 line
2.54.0
[email protected]
AI reviewRe: [PATCH 12/29] ibmvfc: add NVMe/FC Implicit Logout and Move Login support
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
Pre-existing issues:
- [High] Missing NULL pointer check on the event allocated for implicit logout in ibmvfc_tgt_implicit_logout_and_del(), leading to a crash in ibmvfc_send_event(). --
commit d7f6f97d0c832e12c02c910b6675c0a3be2fb197
Author: Tyrel Datwyler <[email protected]>
ibmvfc: add NVMe/FC Implicit Logout and Move Login support
This patch adds protocol-specific handling for implicit logout and move-login operations on NVMe/FC targets. It extends the existing target relogin and migration flows by selecting NVMe/FC-specific opcodes and updating the associated logging.
Quoted 4 lines
drivers/scsi/ibmvscsi/ibmvfc-core.c +0
-0
[ ... ]
drivers/scsi/ibmvscsi/ibmvfc-core.c static void ibmvfc_tgt_implicit_logout_and_del(struct ibmvfc_target *tgt) · line 4465
Quoted 7 lines
drivers/scsi/ibmvscsi/ibmvfc-core.c +1
-1@@ -4456,7 +4461,7 @@ static void ibmvfc_tgt_implicit_logout_and_del(struct ibmvfc_target *tgt)ibmvfc_set_tgt_action (tgt,IBMVFC_TGT_ACTION_DEL_RPORT );kref_put (&tgt->kref, ibmvfc_release_tgt); }else -tgt_dbg (tgt,"Sent Implicit Logout\n" ); +tgt_dbg (tgt,"%s Sent Implicit Logout\n" , proto_type[tgt->protocol]); }
[Severity: High]
This is a pre-existing issue, but does ibmvfc_tgt_implicit_logout_and_del() need to check if evt is NULL before calling ibmvfc_send_event()?
If the reserved event pool is exhausted, __ibmvfc_tgt_get_implicit_logout_evt() can return NULL:
ibmvfc-core.c:__ibmvfc_tgt_get_implicit_logout_evt() {
...
evt = ibmvfc_get_reserved_event(&vhost->crq);
if (!evt)
return NULL;
...
}Unlike other callers such as ibmvfc_tgt_implicit_logout(), this function lacks a NULL check and passes the evt pointer directly to ibmvfc_send_event(), which unconditionally dereferences it:
ibmvfc-core.c:ibmvfc_send_event() {
__be64 *crq_as_u64 = (__be64 *) &evt->crq;
...
}Could this lead to a NULL pointer dereference under heavy queue pressure or concurrent target teardowns?
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=12
Tyrel Datwyler
maintainerPATCH 13/29 ibmvfc: add NVMe/FC Port Login support
In reply to Tyrel DatwylerExpand the target login path to issue the NVMe/FC-specific port login MAD for NVMe targets.
Select the correct PLOGI MAD opcode based on the target protocol and include the protocol name in success and failure logging. The rest of the target login flow remains shared with the existing SCSI implementation.
--- drivers/scsi/ibmvscsi/ibmvfc-core.c | 15 +++++++++------ 1 file changed, 9 insertions(+), 6 deletions(-)
drivers/scsi/ibmvscsi/ibmvfc-core.c +9 -6
@@ -4210,7 +4210,7 @@ static void ibmvfc_tgt_plogi_done(struct ibmvfc_event *evt)ibmvfc_set_tgt_action (tgt,IBMVFC_TGT_ACTION_NONE );switch (status) {case IBMVFC_MAD_SUCCESS : -tgt_dbg (tgt,"Port Login succeeded\n" ); +tgt_dbg (tgt,"%s Port Login succeeded\n" , proto_type[tgt->protocol]);if (tgt->ids.port_name && tgt->ids.port_name !=wwn_to_u64 (rsp->service_parms.port_name)) { vhost->reinit =1 ;
@@ -4238,9 +4238,9 @@ static void ibmvfc_tgt_plogi_done(struct ibmvfc_event *evt)else ibmvfc_del_tgt (tgt); -tgt_log (tgt, level,"Port Login failed: %s (%x:%x) %s (%x) %s (%x) rc=0x%02X\n" , -ibmvfc_get_cmd_error (be16_to_cpu (rsp->status),be16_to_cpu (rsp->error)), -be16_to_cpu (rsp->status),be16_to_cpu (rsp->error), +tgt_log (tgt, level,"%s Port Login failed: %s (%x:%x) %s (%x) %s (%x) rc=0x%02X\n" , + proto_type[tgt->protocol],ibmvfc_get_cmd_error (be16_to_cpu (rsp->status), +be16_to_cpu (rsp->error)),be16_to_cpu (rsp->status),be16_to_cpu (rsp->error),ibmvfc_get_fc_type (be16_to_cpu (rsp->fc_type)),be16_to_cpu (rsp->fc_type),ibmvfc_get_ls_explain (be16_to_cpu (rsp->fc_explain)),be16_to_cpu (rsp->fc_explain), status);break ;
@@ -4286,7 +4286,10 @@ static void ibmvfc_tgt_send_plogi(struct ibmvfc_target *tgt) }else { plogi->common.version =cpu_to_be32 (1 ); } - plogi->common.opcode =cpu_to_be32 (IBMVFC_PORT_LOGIN ); +if (tgt->protocol ==IBMVFC_PROTO_SCSI ) + plogi->common.opcode =cpu_to_be32 (IBMVFC_PORT_LOGIN ); +else + plogi->common.opcode =cpu_to_be32 (IBMVFC_NVMF_PORT_LOGIN ); plogi->common.length =cpu_to_be16 (sizeof (*plogi)); plogi->scsi_id =cpu_to_be64 (tgt->scsi_id);
@@ -4295,7 +4298,7 @@ static void ibmvfc_tgt_send_plogi(struct ibmvfc_target *tgt)ibmvfc_set_tgt_action (tgt,IBMVFC_TGT_ACTION_NONE );kref_put (&tgt->kref, ibmvfc_release_tgt); }else -tgt_dbg (tgt,"Sent port login\n" ); +tgt_dbg (tgt,"Sent %s port login\n" , proto_type[tgt->protocol]); }/**
Signature · 1 line
2.54.0
[email protected]
AI reviewRe: [PATCH 13/29] ibmvfc: add NVMe/FC Port Login support
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
Pre-existing issues:
- [High] Targets allocated from a mempool are incorrectly freed using kfree() directly, permanently depleting the mempool's reserved elements. --
commit 6a208595368e371e1b9ab8d0fccb04902f87daf8
Author: Tyrel Datwyler <[email protected]>
ibmvfc: add NVMe/FC Port Login support
Expand the target login path to issue the NVMe/FC-specific port login MAD for NVMe targets. Select the correct PLOGI MAD opcode based on the target protocol and include the protocol name in success and failure logging.
Quoted 4 lines
drivers/scsi/ibmvscsi/ibmvfc-core.c +0
-0
[ ... ]
Quoted 13 lines
drivers/scsi/ibmvscsi/ibmvfc-core.c +4
-1@@ -4286,7 +4286,10 @@ static void ibmvfc_tgt_send_plogi(struct ibmvfc_target *tgt) }else { plogi->common.version =cpu_to_be32 (1 ); } - plogi->common.opcode =cpu_to_be32 (IBMVFC_PORT_LOGIN ); +if (tgt->protocol ==IBMVFC_PROTO_SCSI ) + plogi->common.opcode =cpu_to_be32 (IBMVFC_PORT_LOGIN ); +else + plogi->common.opcode =cpu_to_be32 (IBMVFC_NVMF_PORT_LOGIN ); plogi->common.length =cpu_to_be16 (sizeof (*plogi)); plogi->scsi_id =cpu_to_be64 (tgt->scsi_id);
drivers/scsi/ibmvscsi/ibmvfc-core.c static void ibmvfc_tgt_send_plogi(struct ibmvfc_target *tgt) · line 4299
Quoted 2 lines
drivers/scsi/ibmvscsi/ibmvfc-core.c +0
-0@@ -4295,7 +4298,7 @@ static void ibmvfc_tgt_send_plogi(struct ibmvfc_target *tgt)ibmvfc_set_tgt_action (tgt,IBMVFC_TGT_ACTION_NONE );kref_put (&tgt->kref, ibmvfc_release_tgt);
[Severity: High]
This isn't a bug introduced by this patch, but does the target release function deplete the mempool's reserved elements during memory pressure?
The target structure is allocated from the driver's mempool:
drivers/scsi/ibmvscsi/ibmvfc-core.c:ibmvfc_alloc_target() {
...
tgt = mempool_alloc(vhost->tgt_pool, GFP_NOIO);
...
}However, it appears ibmvfc_release_tgt() uses kfree() instead of mempool_free():
drivers/scsi/ibmvscsi/ibmvfc-core.c:ibmvfc_release_tgt() {
struct ibmvfc_target *tgt = container_of(kref, struct ibmvfc_target, kref);
kfree(tgt);
}When the system is under memory pressure, mempool_alloc() falls back to using its pre-allocated reserve elements.
If these elements are returned to the general slab allocator via kfree() instead of mempool_free(), wouldn't this permanently deplete the mempool's reserve and cause subsequent allocations under memory pressure to hang indefinitely?
Quoted 4 lines
drivers/scsi/ibmvscsi/ibmvfc-core.c +1
-1}else -tgt_dbg (tgt,"Sent port login\n" ); +tgt_dbg (tgt,"Sent %s port login\n" , proto_type[tgt->protocol]); }
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=13
Tyrel Datwyler
maintainerPATCH 14/29 ibmvfc: add NVMe/FC Process Login support
In reply to Tyrel DatwylerExtend PRLI handling code to support NVMe/FC targets.
When the target protocol is NVMe/FC, issue the NVMe process login MAD, set the NVMe FC-4 type, and populate NVMe-specific service parameters. On completion, decode the returned PRLI service parameters and derive the appropriate remote-port roles for NVMe initiator, target, and discovery ports.
Keep the existing SCSI PRLI flow unchanged while allowing the common target state machine to complete login for NVMe/FC targets.
--- drivers/scsi/ibmvscsi/ibmvfc-core.c | 55 ++++++++++++++++++++--------- drivers/scsi/ibmvscsi/ibmvfc.h | 6 ++++ 2 files changed, 45 insertions(+), 16 deletions(-)
drivers/scsi/ibmvscsi/ibmvfc-core.c +39 -16
@@ -4088,11 +4088,12 @@ static void ibmvfc_tgt_prli_done(struct ibmvfc_event *evt)ibmvfc_set_tgt_action (tgt,IBMVFC_TGT_ACTION_NONE );switch (status) {case IBMVFC_MAD_SUCCESS : -tgt_dbg (tgt,"Process Login succeeded: %X %02X %04X\n" , - parms->type, parms->flags, parms->service_parms); +tgt_dbg (tgt,"%s Process Login succeeded: %X %02X %04X\n" , + proto_type[tgt->protocol], parms->type, parms->flags, + parms->service_parms); + index =ibmvfc_get_prli_rsp (be16_to_cpu (parms->flags));if (parms->type ==IBMVFC_SCSI_FCP_TYPE ) { - index =ibmvfc_get_prli_rsp (be16_to_cpu (parms->flags));if (prli_rsp[index].logged_in) {if (be16_to_cpu (parms->flags) &IBMVFC_PRLI_EST_IMG_PAIR ) { tgt->need_login =0 ;
@@ -4108,6 +4109,22 @@ static void ibmvfc_tgt_prli_done(struct ibmvfc_event *evt)ibmvfc_retry_tgt_init (tgt, ibmvfc_tgt_send_prli); elseibmvfc_del_tgt (tgt ); + }else if (parms->type ==IBMVFC_NVME_FCP_TYPE ) { +if (prli_rsp[index].logged_in) { +/* For FC-NVMe PRLI the Image Pair field is always set to zero (see 6.3.3) */ + tgt->need_login =0 ; + tgt->ids.roles =0 ; +if (be32_to_cpu (parms->service_parms) &IBMVFC_PRLI_NVME_TARGET ) + tgt->ids.roles |=FC_PORT_ROLE_NVME_TARGET ; +if (be32_to_cpu (parms->service_parms) &IBMVFC_PRLI_NVME_INITIATOR ) + tgt->ids.roles |=FC_PORT_ROLE_NVME_INITIATOR ; +if (be32_to_cpu (parms->service_parms) &IBMVFC_PRLI_NVME_DISCOVERY ) + tgt->ids.roles |=FC_PORT_ROLE_NVME_DISCOVERY ; + tgt->add_rport =1 ; + }else if (prli_rsp[index].retry) +ibmvfc_retry_tgt_init (tgt, ibmvfc_tgt_send_prli); +else +ibmvfc_del_tgt (tgt); }else ibmvfc_del_tgt (tgt);break ;
@@ -4128,9 +4145,10 @@ static void ibmvfc_tgt_prli_done(struct ibmvfc_event *evt)else ibmvfc_del_tgt (tgt); -tgt_log (tgt, level,"Process Login failed: %s (%x:%x) rc=0x%02X\n" , -ibmvfc_get_cmd_error (be16_to_cpu (rsp->status),be16_to_cpu (rsp->error)), -be16_to_cpu (rsp->status),be16_to_cpu (rsp->error), status); +tgt_log (tgt, level,"%s Process Login failed: %s (%x:%x) rc=0x%02X\n" , + proto_type[tgt->protocol],ibmvfc_get_cmd_error (be16_to_cpu (rsp->status), +be16_to_cpu (rsp->error)),be16_to_cpu (rsp->status), +be16_to_cpu (rsp->error), status);break ; }
@@ -4172,25 +4190,30 @@ static void ibmvfc_tgt_send_prli(struct ibmvfc_target *tgt) }else { prli->common.version =cpu_to_be32 (1 ); } - prli->common.opcode =cpu_to_be32 (IBMVFC_PROCESS_LOGIN ); +if (tgt->protocol ==IBMVFC_PROTO_SCSI ) { + prli->common.opcode =cpu_to_be32 (IBMVFC_PROCESS_LOGIN ); + prli->parms.type =IBMVFC_SCSI_FCP_TYPE ; + prli->parms.flags =cpu_to_be16 (IBMVFC_PRLI_EST_IMG_PAIR ); + prli->parms.service_parms =cpu_to_be32 (IBMVFC_PRLI_INITIATOR_FUNC ); + prli->parms.service_parms |=cpu_to_be32 (IBMVFC_PRLI_READ_FCP_XFER_RDY_DISABLED ); + +if (cls3_error) + prli->parms.service_parms |=cpu_to_be32 (IBMVFC_PRLI_RETRY ); + }else { + prli->common.opcode =cpu_to_be32 (IBMVFC_NVMF_PROCESS_LOGIN ); + prli->parms.type =IBMVFC_NVME_FCP_TYPE ; + prli->parms.service_parms =cpu_to_be32 (IBMVFC_PRLI_NVME_INITIATOR ); + } prli->common.length =cpu_to_be16 (sizeof (*prli)); prli->scsi_id =cpu_to_be64 (tgt->scsi_id); - prli->parms.type =IBMVFC_SCSI_FCP_TYPE ; - prli->parms.flags =cpu_to_be16 (IBMVFC_PRLI_EST_IMG_PAIR ); - prli->parms.service_parms =cpu_to_be32 (IBMVFC_PRLI_INITIATOR_FUNC ); - prli->parms.service_parms |=cpu_to_be32 (IBMVFC_PRLI_READ_FCP_XFER_RDY_DISABLED ); - -if (cls3_error) - prli->parms.service_parms |=cpu_to_be32 (IBMVFC_PRLI_RETRY ); -ibmvfc_set_tgt_action (tgt,IBMVFC_TGT_ACTION_INIT_WAIT );if (ibmvfc_send_event (evt, vhost, default_timeout)) { vhost->discovery_threads--;ibmvfc_set_tgt_action (tgt,IBMVFC_TGT_ACTION_NONE );kref_put (&tgt->kref, ibmvfc_release_tgt); }else -tgt_dbg (tgt,"Sent process login\n" ); +tgt_dbg (tgt,"%s Sent process login\n" , proto_type[tgt->protocol]); }/**
drivers/scsi/ibmvscsi/ibmvfc.h +6 -0
@@ -381,6 +381,7 @@ struct ibmvfc_move_login {struct ibmvfc_prli_svc_parms {u8 type;#define IBMVFC_SCSI_FCP_TYPE 0x08 +#define IBMVFC_NVME_FCP_TYPE 0x28u8 type_ext;__be16 flags;#define IBMVFC_PRLI_ORIG_PA_VALID 0x8000
@@ -396,6 +397,11 @@ struct ibmvfc_prli_svc_parms {#define IBMVFC_PRLI_TARGET_FUNC 0x00000010#define IBMVFC_PRLI_READ_FCP_XFER_RDY_DISABLED 0x00000002#define IBMVFC_PRLI_WR_FCP_XFER_RDY_DISABLED 0x00000001 +#define IBMVFC_PRLI_NVME_PI_CTRL 0x00000200 +#define IBMVFC_PRLI_NVME_SLER 0x00000100 +#define IBMVFC_PRLI_NVME_INITIATOR 0x00000020 +#define IBMVFC_PRLI_NVME_TARGET 0x00000010 +#define IBMVFC_PRLI_NVME_DISCOVERY 0x00000008 } __packed__aligned(4) ;struct ibmvfc_process_login {
Signature · 1 line
2.54.0
[email protected]
AI reviewRe: [PATCH 14/29] ibmvfc: add NVMe/FC Process Login support
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
Pre-existing issues:
- [High] Self-deadlock on host_lock if a management CRQ event fails to send. --
commit ee31c1669f64561bd4a4e52947529fea4cf6545e
Author: Tyrel Datwyler <[email protected]>
ibmvfc: add NVMe/FC Process Login support
This commit extends the PRLI handling code to support NVMe/FC targets. It adds NVMe-specific service parameters and sets the remote-port roles upon successful process login. The existing SCSI PRLI flow is preserved.
Quoted 4 lines
drivers/scsi/ibmvscsi/ibmvfc-core.c +0
-0
[ ... ]
Quoted 1 line
drivers/scsi/ibmvscsi/ibmvfc-core.c +0
-0@@ -4172,25 +4190,30 @@ static void ibmvfc_tgt_send_prli(struct ibmvfc_target *tgt)
[ ... ]
drivers/scsi/ibmvscsi/ibmvfc-core.c static void ibmvfc_tgt_send_prli(struct ibmvfc_target *tgt) · line 4211
Quoted 2 lines
drivers/scsi/ibmvscsi/ibmvfc-core.c +0
-0ibmvfc_set_tgt_action (tgt,IBMVFC_TGT_ACTION_INIT_WAIT );if (ibmvfc_send_event (evt, vhost, default_timeout)) {
[Severity: High]
This is a pre-existing issue, but can this call path trigger a deadlock?
When ibmvfc_tgt_send_prli() is called by the target's job step from ibmvfc_do_work(), the host_lock is already held.
If ibmvfc_send_event() fails (e.g. due to a hypercall failure), it synchronously invokes the completion callback evt->done(evt).
Because the event was initialized with IBMVFC_MAD_FORMAT via ibmvfc_init_event(), evt->done is set to ibmvfc_locked_done(), which attempts to acquire the host_lock again:
spin_lock_irqsave(evt->vhost->host->host_lock, flags);
Would this cause a hard deadlock on the CPU if the underlying hardware encounters an error and fails to send the process login?
Quoted 7 lines
drivers/scsi/ibmvscsi/ibmvfc-core.c +1
-1vhost->discovery_threads--;ibmvfc_set_tgt_action (tgt,IBMVFC_TGT_ACTION_NONE );kref_put (&tgt->kref, ibmvfc_release_tgt); }else -tgt_dbg (tgt,"Sent process login\n" ); +tgt_dbg (tgt,"%s Sent process login\n" , proto_type[tgt->protocol]); }
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=14
Tyrel Datwyler
maintainerPATCH 15/29 ibmvfc: add NVMe/FC Query Target support
In reply to Tyrel DatwylerAdd protocol-specific query-target support for NVMe/FC targets.
Use the NVMe query-target specific MAD when querying an NVMe target and update the associated debug and error logging to include the target protocol.
--- drivers/scsi/ibmvscsi/ibmvfc-core.c | 15 +++++++++------ 1 file changed, 9 insertions(+), 6 deletions(-)
drivers/scsi/ibmvscsi/ibmvfc-core.c +9 -6
@@ -4849,7 +4849,7 @@ static void ibmvfc_tgt_query_target_done(struct ibmvfc_event *evt)ibmvfc_set_tgt_action (tgt,IBMVFC_TGT_ACTION_NONE );switch (status) {case IBMVFC_MAD_SUCCESS : -tgt_dbg (tgt,"Query Target succeeded\n" ); +tgt_dbg (tgt,"%s Query Target succeeded\n" , proto_type[tgt->protocol]);if (be64_to_cpu (rsp->scsi_id) != tgt->scsi_id)ibmvfc_del_tgt (tgt);else
@@ -4871,9 +4871,9 @@ static void ibmvfc_tgt_query_target_done(struct ibmvfc_event *evt)else ibmvfc_del_tgt (tgt); -tgt_log (tgt, level,"Query Target failed: %s (%x:%x) %s (%x) %s (%x) rc=0x%02X\n" , -ibmvfc_get_cmd_error (be16_to_cpu (rsp->status),be16_to_cpu (rsp->error)), -be16_to_cpu (rsp->status),be16_to_cpu (rsp->error), +tgt_log (tgt, level,"%s Query Target failed: %s (%x:%x) %s (%x) %s (%x) rc=0x%02X\n" , + proto_type[tgt->protocol],ibmvfc_get_cmd_error (be16_to_cpu (rsp->status), +be16_to_cpu (rsp->error)),be16_to_cpu (rsp->status),be16_to_cpu (rsp->error),ibmvfc_get_fc_type (be16_to_cpu (rsp->fc_type)),be16_to_cpu (rsp->fc_type),ibmvfc_get_gs_explain (be16_to_cpu (rsp->fc_explain)),be16_to_cpu (rsp->fc_explain), status);
@@ -4913,7 +4913,10 @@ static void ibmvfc_tgt_query_target(struct ibmvfc_target *tgt) query_tgt = &evt->iu.query_tgt;memset (query_tgt,0 ,sizeof (*query_tgt)); query_tgt->common.version =cpu_to_be32 (1 ); - query_tgt->common.opcode =cpu_to_be32 (IBMVFC_QUERY_TARGET ); +if (tgt->protocol ==IBMVFC_PROTO_SCSI ) + query_tgt->common.opcode =cpu_to_be32 (IBMVFC_QUERY_TARGET ); +else + query_tgt->common.opcode =cpu_to_be32 (IBMVFC_NVMF_QUERY_TARGET ); query_tgt->common.length =cpu_to_be16 (sizeof (*query_tgt)); query_tgt->wwpn =cpu_to_be64 (tgt->ids.port_name);
@@ -4923,7 +4926,7 @@ static void ibmvfc_tgt_query_target(struct ibmvfc_target *tgt)ibmvfc_set_tgt_action (tgt,IBMVFC_TGT_ACTION_NONE );kref_put (&tgt->kref, ibmvfc_release_tgt); }else -tgt_dbg (tgt,"Sent Query Target\n" ); +tgt_dbg (tgt,"Sent %s Query Target\n" , proto_type[tgt->protocol]); }/**
Signature · 1 line
2.54.0
[email protected]
AI reviewRe: [PATCH 15/29] ibmvfc: add NVMe/FC Query Target support
Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
New issues:
- [Low] The global array proto_type is missing the static keyword, causing namespace pollution.
Pre-existing issues:
- [High] Memory allocated from a mempool is freed directly via kfree(), permanently depleting the mempool reserve under memory pressure. --
commit edf90dca8246218c80f44b69406be60422857d3c
Author: Tyrel Datwyler <[email protected]>
ibmvfc: add NVMe/FC Query Target support
Add protocol-specific query-target support for NVMe/FC targets. It uses the NVMe query-target specific MAD when querying an NVMe target and updates the associated debug and error logging to include the target protocol.
drivers/scsi/ibmvscsi/ibmvfc-core.c static void ibmvfc_tgt_query_target_done(struct ibmvfc_event *evt) · line 4852
Quoted 10 lines
drivers/scsi/ibmvscsi/ibmvfc-core.c +1
-1@@ -4849,7 +4849,7 @@ static void ibmvfc_tgt_query_target_done(struct ibmvfc_event *evt)ibmvfc_set_tgt_action (tgt,IBMVFC_TGT_ACTION_NONE );switch (status) {case IBMVFC_MAD_SUCCESS : -tgt_dbg (tgt,"Query Target succeeded\n" ); +tgt_dbg (tgt,"%s Query Target succeeded\n" , proto_type[tgt->protocol]);
[Severity: Low]
Does proto_type need the static keyword? Since it is a generic identifier defined at file scope without being in a header:
drivers/scsi/ibmvscsi/ibmvfc-core.c:
const char *proto_type[] = {
"SCSI",
"NVMe",
};Could this cause symbol collisions in the global kernel namespace during linking?
Quoted 3 lines
drivers/scsi/ibmvscsi/ibmvfc-core.c +0
-0if (be64_to_cpu (rsp->scsi_id) != tgt->scsi_id)ibmvfc_del_tgt (tgt);else
[ ... ]
Quoted 13 lines
drivers/scsi/ibmvscsi/ibmvfc-core.c +4
-1@@ -4913,7 +4913,10 @@ static void ibmvfc_tgt_query_target(struct ibmvfc_target *tgt) query_tgt = &evt->iu.query_tgt;memset (query_tgt,0 ,sizeof (*query_tgt)); query_tgt->common.version =cpu_to_be32 (1 ); - query_tgt->common.opcode =cpu_to_be32 (IBMVFC_QUERY_TARGET ); +if (tgt->protocol ==IBMVFC_PROTO_SCSI ) + query_tgt->common.opcode =cpu_to_be32 (IBMVFC_QUERY_TARGET ); +else + query_tgt->common.opcode =cpu_to_be32 (IBMVFC_NVMF_QUERY_TARGET ); query_tgt->common.length =cpu_to_be16 (sizeof (*query_tgt)); query_tgt->wwpn =cpu_to_be64 (tgt->ids.port_name);
drivers/scsi/ibmvscsi/ibmvfc-core.c static void ibmvfc_tgt_query_target(struct ibmvfc_target *tgt) · line 4927
Quoted 2 lines
drivers/scsi/ibmvscsi/ibmvfc-core.c +0
-0@@ -4923,7 +4926,7 @@ static void ibmvfc_tgt_query_target(struct ibmvfc_target *tgt)ibmvfc_set_tgt_action (tgt,IBMVFC_TGT_ACTION_NONE );kref_put (&tgt->kref, ibmvfc_release_tgt);
[Severity: High]
This is a pre-existing issue, but does ibmvfc_release_tgt() incorrectly free the target?
The target is originally allocated using mempool_alloc():
drivers/scsi/ibmvscsi/ibmvfc-core.c:ibmvfc_alloc_target() {
...
tgt = mempool_alloc(vhost->tgt_pool, GFP_NOIO);
...
}However, it is freed directly via kfree() instead of mempool_free():
drivers/scsi/ibmvscsi/ibmvfc-core.c:ibmvfc_release_tgt() {
struct ibmvfc_target *tgt = container_of(kref, struct ibmvfc_target, kref);
kfree(tgt);
}Under memory pressure, mempool_alloc() falls back to its preallocated reserve. Will bypassing mempool_free() permanently leak the reserved elements, eventually causing deadlocks when the pool is depleted and mempool_alloc() blocks waiting for elements to be returned?
Quoted 4 lines
drivers/scsi/ibmvscsi/ibmvfc-core.c +1
-1}else -tgt_dbg (tgt,"Sent Query Target\n" ); +tgt_dbg (tgt,"Sent %s Query Target\n" , proto_type[tgt->protocol]); }
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=15
Tyrel Datwyler
maintainerPATCH 16/29 ibmvfc: allocate targets based on protocol
In reply to Tyrel DatwylerAllocate discovered targets onto the channel-group list that matches their protocol.
When a target is created, use the discovered protocol type to decide which list it belongs on. This keeps protocol-specific discovery results isolated and allows later state-machine and remote-port code to walk the correct target set.
--- drivers/scsi/ibmvscsi/ibmvfc-core.c | 22 +++++++++++++++++----- 1 file changed, 17 insertions(+), 5 deletions(-)
drivers/scsi/ibmvscsi/ibmvfc-core.c +17 -5
@@ -4938,25 +4938,32 @@ static void ibmvfc_tgt_query_target(struct ibmvfc_target *tgt)* 0 on success / other on failure **/ static int ibmvfc_alloc_target (struct ibmvfc_host *vhost, -struct ibmvfc_discover_targets_entry *target) +struct ibmvfc_discover_targets_entry *target, +enum ibmvfc_protocol protocol) {struct ibmvfc_target *stgt =NULL ;struct ibmvfc_target *wtgt =NULL ;struct ibmvfc_target *tgt; +struct ibmvfc_channels *channels;unsigned long flags;u64 scsi_id =be32_to_cpu (target->scsi_id) &IBMVFC_DISC_TGT_SCSI_ID_MASK ;u64 wwpn =be64_to_cpu (target->wwpn); +if (protocol ==IBMVFC_PROTO_SCSI ) + channels = &vhost->scsi_scrqs; +else + channels = &vhost->nvme_scrqs; +/* Look to see if we already have a target allocated for this SCSI ID or WWPN */ spin_lock_irqsave (vhost->host->host_lock, flags); -list_for_each_entry (tgt, &vhost->scsi_scrqs.targets, queue) { +list_for_each_entry (tgt, &channels->targets, queue) {if (tgt->wwpn == wwpn) { wtgt = tgt;break ; } } -list_for_each_entry (tgt, &vhost->scsi_scrqs.targets, queue) { +list_for_each_entry (tgt, &channels->targets, queue) {if (tgt->scsi_id == scsi_id) { stgt = tgt;break ;
@@ -5004,6 +5011,7 @@ static int ibmvfc_alloc_target(struct ibmvfc_host *vhost, tgt =mempool_alloc (vhost->tgt_pool,GFP_NOIO );memset (tgt,0 ,sizeof (*tgt)); + tgt->protocol = protocol; tgt->scsi_id = scsi_id; tgt->wwpn = wwpn; tgt->vhost = vhost;
@@ -5013,7 +5021,7 @@ static int ibmvfc_alloc_target(struct ibmvfc_host *vhost,ibmvfc_init_tgt (tgt, ibmvfc_tgt_implicit_logout);spin_lock_irqsave (vhost->host->host_lock, flags); tgt->cancel_key = vhost->task_set++; -list_add_tail (&tgt->queue, &vhost->scsi_scrqs.targets); +list_add_tail (&tgt->queue, &channels->targets); unlock_out:spin_unlock_irqrestore (vhost->host->host_lock, flags);
@@ -5032,7 +5040,11 @@ static int ibmvfc_alloc_targets(struct ibmvfc_host *vhost)int i, rc;for (i =0 , rc =0 ; !rc && i < vhost->scsi_scrqs.num_targets; i++) - rc =ibmvfc_alloc_target (vhost, &vhost->scsi_scrqs.disc_buf[i]); + rc =ibmvfc_alloc_target (vhost, &vhost->scsi_scrqs.disc_buf[i], + vhost->scsi_scrqs.protocol); +for (i =0 ; !rc && i < vhost->nvme_scrqs.num_targets; i++) + rc =ibmvfc_alloc_target (vhost, &vhost->nvme_scrqs.disc_buf[i], + vhost->nvme_scrqs.protocol);return rc; }
Signature · 1 line
2.54.0
[email protected]
AI reviewRe: [PATCH 16/29] ibmvfc: allocate targets based on protocol
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
Pre-existing issues:
- [High] Target structures allocated from a mempool are incorrectly freed using kfree() instead of mempool_free(), permanently depleting the mempool reserve. --
commit 2827414150eab249a5dd4c4a093ced5d74ae8737
Author: Tyrel Datwyler <[email protected]>
ibmvfc: allocate targets based on protocol
Allocate discovered targets onto the channel-group list that matches their protocol.
Quoted 4 lines
drivers/scsi/ibmvscsi/ibmvfc-core.c +0
-0
[ ... ]
drivers/scsi/ibmvscsi/ibmvfc-core.c static int ibmvfc_alloc_target(struct ibmvfc_host *vhost, · line 5014
Quoted 5 lines
drivers/scsi/ibmvscsi/ibmvfc-core.c +1
-0@@ -5004,6 +5011,7 @@ static int ibmvfc_alloc_target(struct ibmvfc_host *vhost, tgt =mempool_alloc (vhost->tgt_pool,GFP_NOIO );memset (tgt,0 ,sizeof (*tgt)); + tgt->protocol = protocol;
[Severity: High]
This is a pre-existing issue and was not introduced by the patch under review, but since we are looking at this allocation path, is the target structure freed correctly when released?
This function allocates tgt from a mempool, but looking at the reference count release routine ibmvfc_release_tgt(), the object is freed directly to the slab allocator:
drivers/scsi/ibmvscsi/ibmvfc-core.c:ibmvfc_release_tgt() {
struct ibmvfc_target *tgt = container_of(kref, struct ibmvfc_target, kref);
kfree(tgt);
}Because it is freed via kfree() instead of mempool_free(), the mempool internal reserve count is never replenished and waiting threads are not woken up. Under memory pressure, subsequent mempool_alloc() calls with GFP_NOIO could block indefinitely on the mempool waitqueue once the reserve is depleted.
Quoted 3 lines
drivers/scsi/ibmvscsi/ibmvfc-core.c +0
-0tgt->scsi_id = scsi_id; tgt->wwpn = wwpn; tgt->vhost = vhost;
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=16
Tyrel Datwyler
maintainerPATCH 17/29 ibmvfc: delete NVMe/FC targets as well as SCSI
In reply to Tyrel DatwylerExtend target deletion paths to process NVMe targets as well as SCSI targets.
Update link-down, and host reinitialization flows to walk both the SCSI and NVMe target lists when marking targets for deletion. This ensures that protocol-specific target state stays consistent across adapter resets and fabric events.
Rename ibmvfc_relogin to ibmvfc_scsi_relogin as it acts on a scsi command.
--- drivers/scsi/ibmvscsi/ibmvfc-core.c | 12 ++++++++++-- 1 file changed, 10 insertions(+), 2 deletions(-)
drivers/scsi/ibmvscsi/ibmvfc-core.c +10 -2
@@ -714,6 +714,8 @@ static void ibmvfc_link_down(struct ibmvfc_host *vhost,scsi_block_requests (vhost->host);list_for_each_entry (tgt, &vhost->scsi_scrqs.targets, queue)ibmvfc_del_tgt (tgt); +list_for_each_entry (tgt, &vhost->nvme_scrqs.targets, queue) +ibmvfc_del_tgt (tgt);ibmvfc_set_host_state (vhost, state);ibmvfc_set_host_action (vhost,IBMVFC_HOST_ACTION_TGT_DEL ); vhost->events_to_log |=IBMVFC_AE_LINKDOWN ;
@@ -751,6 +753,12 @@ static void ibmvfc_init_host(struct ibmvfc_host *vhost)else ibmvfc_del_tgt (tgt); } +list_for_each_entry (tgt, &vhost->nvme_scrqs.targets, queue) { +if (vhost->client_migrated) + tgt->need_login =1 ; +else +ibmvfc_del_tgt (tgt); + }scsi_block_requests (vhost->host);ibmvfc_set_host_action (vhost,IBMVFC_HOST_ACTION_INIT );
@@ -1868,7 +1876,7 @@ static void ibmvfc_log_error(struct ibmvfc_event *evt)* @sdev: scsi device struct * **/ -static void ibmvfc_relogin (struct scsi_device *sdev) +static void ibmvfc_scsi_relogin (struct scsi_device *sdev) {struct ibmvfc_host *vhost =shost_priv (sdev->host);struct fc_rport *rport =starget_to_rport (scsi_target (sdev));
@@ -1920,7 +1928,7 @@ static void ibmvfc_scsi_done(struct ibmvfc_event *evt)memcpy (cmnd->sense_buffer, rsp->data.sense + rsp_len, sense_len);if ((be16_to_cpu (vfc_cmd->status) &IBMVFC_VIOS_FAILURE ) && (be16_to_cpu (vfc_cmd->error) ==IBMVFC_PLOGI_REQUIRED )) -ibmvfc_relogin (cmnd->device); +ibmvfc_scsi_relogin (cmnd->device);if (!cmnd->result && (!scsi_get_resid (cmnd) || (rsp->flags &FCP_RESID_OVER ))) cmnd->result = (DID_ERROR <<16 );
Signature · 1 line
2.54.0
[email protected]
AI reviewRe: [PATCH 17/29] ibmvfc: delete NVMe/FC targets as well as SCSI
Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
New issues:
- [High] Fabric ELS events (LOGO, PRLO, PLOGI) fail to process NVMe targets, violating the commit's claim regarding consistent fabric event handling.
Pre-existing issues:
- [High] An integer overflow in the calculation of sense_len + rsp_len bypasses bounds checking, leading to an out-of-bounds read and write in memcpy(). --
commit 56df0a05e56764e1456bdc6176dcf26a3a1bc6b0
Author: Tyrel Datwyler <[email protected]>
ibmvfc: delete NVMe/FC targets as well as SCSI
This commit extends target deletion paths to process NVMe targets alongside SCSI targets. It updates link-down and host reinitialization flows to walk both lists to maintain consistent target state. It also renames ibmvfc_relogin to ibmvfc_scsi_relogin.
Quoted 4 lines
Update link-down, and host reinitialization flows to walk both the SCSI and NVMe target lists when marking targets for deletion. This ensures that protocol-specific target state stays consistent across adapter resets and fabric events.
[Severity: High]
Does the current patch fully handle fabric events for NVMe targets?
The commit message mentions maintaining consistent target state across fabric events, but it appears ibmvfc_handle_async in drivers/scsi/ibmvscsi/ibmvfc-core.c was not updated to process the nvme_scrqs.targets list.
When an NVMe target logs out of the fabric and the adapter sends an async IBMVFC_AE_ELS_LOGO event, the ELS event handler loop only searches scsi_scrqs.targets. Can this cause stale NVMe targets to remain in memory indefinitely and break the target state machine?
Quoted 4 lines
drivers/scsi/ibmvscsi/ibmvfc-core.c +0
-0
[ ... ]
drivers/scsi/ibmvscsi/ibmvfc-core.c static void ibmvfc_scsi_done(struct ibmvfc_event *evt) · line 1928
Quoted 2 lines
drivers/scsi/ibmvscsi/ibmvfc-core.c +0
-0@@ -1920,7 +1928,7 @@ static void ibmvfc_scsi_done(struct ibmvfc_event *evt)memcpy (cmnd->sense_buffer, rsp->data.sense + rsp_len, sense_len);
[Severity: High]
This is a pre-existing issue, but can an integer overflow occur here when calculating sense_len + rsp_len in ibmvfc_scsi_done?
If a buggy or compromised virtual I/O server returns a large fcp_sense_len like 0xFFFFFFFE and a small fcp_rsp_len like 4, their sum could wrap around to 2.
This wrapped value would bypass the greater than SCSI_SENSE_BUFFERSIZE bounds check. Since rsp_len <= 8 would still be true, could the memcpy call attempt to copy nearly 4GB of data and cause memory corruption?
Quoted 7 lines
drivers/scsi/ibmvscsi/ibmvfc-core.c +1
-1if ((be16_to_cpu (vfc_cmd->status) &IBMVFC_VIOS_FAILURE ) && (be16_to_cpu (vfc_cmd->error) ==IBMVFC_PLOGI_REQUIRED )) -ibmvfc_relogin (cmnd->device); +ibmvfc_scsi_relogin (cmnd->device);if (!cmnd->result && (!scsi_get_resid (cmnd) || (rsp->flags &FCP_RESID_OVER ))) cmnd->result = (DID_ERROR <<16 );
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=17
Tyrel Datwyler
maintainerPATCH 18/29 ibmvfc: update state machine to process NVMe/FC targets
In reply to Tyrel DatwylerUpdate the host work loop and target state-machine helpers to process NVMe targets in addition to SCSI targets.
Check both protocol-specific target lists when determining whether there is initialization or logout work pending, and extend the query, target init, and target delete phases to dispatch work for NVMe targets using the same common state-machine callbacks.
This allows the existing discovery and login state machine to drive NVMe/FC targets through query, login, and deletion without duplicating the control flow.
--- drivers/scsi/ibmvscsi/ibmvfc-core.c | 42 +++++++++++++++++++++++++++++ 1 file changed, 42 insertions(+)
drivers/scsi/ibmvscsi/ibmvfc-core.c +42 -0
@@ -5692,6 +5692,11 @@ static int ibmvfc_dev_init_to_do(struct ibmvfc_host *vhost) tgt->action == IBMVFC_TGT_ACTION_INIT_WAIT)return 1 ; } +list_for_each_entry (tgt, &vhost->nvme_scrqs.targets, queue) { +if (tgt->action ==IBMVFC_TGT_ACTION_INIT || + tgt->action ==IBMVFC_TGT_ACTION_INIT_WAIT ) +return 1 ; + }return 0 ; }
@@ -5712,6 +5717,11 @@ static int ibmvfc_dev_logo_to_do(struct ibmvfc_host *vhost) tgt->action == IBMVFC_TGT_ACTION_LOGOUT_RPORT_WAIT)return 1 ; } +list_for_each_entry (tgt, &vhost->nvme_scrqs.targets, queue) { +if (tgt->action ==IBMVFC_TGT_ACTION_LOGOUT_RPORT || + tgt->action ==IBMVFC_TGT_ACTION_LOGOUT_RPORT_WAIT ) +return 1 ; + }return 0 ; }
@@ -5740,9 +5750,15 @@ static int __ibmvfc_work_to_do(struct ibmvfc_host *vhost)list_for_each_entry (tgt, &vhost->scsi_scrqs.targets, queue)if (tgt->action ==IBMVFC_TGT_ACTION_INIT )return 1 ; +list_for_each_entry (tgt, &vhost->nvme_scrqs.targets, queue) +if (tgt->action ==IBMVFC_TGT_ACTION_INIT ) +return 1 ;list_for_each_entry (tgt, &vhost->scsi_scrqs.targets, queue)if (tgt->action ==IBMVFC_TGT_ACTION_INIT_WAIT )return 0 ; +list_for_each_entry (tgt, &vhost->nvme_scrqs.targets, queue) +if (tgt->action ==IBMVFC_TGT_ACTION_INIT_WAIT ) +return 0 ;return 1 ;case IBMVFC_HOST_ACTION_TGT_DEL :case IBMVFC_HOST_ACTION_TGT_DEL_FAILED :
@@ -5751,9 +5767,15 @@ static int __ibmvfc_work_to_do(struct ibmvfc_host *vhost)list_for_each_entry (tgt, &vhost->scsi_scrqs.targets, queue)if (tgt->action ==IBMVFC_TGT_ACTION_LOGOUT_RPORT )return 1 ; +list_for_each_entry (tgt, &vhost->nvme_scrqs.targets, queue) +if (tgt->action ==IBMVFC_TGT_ACTION_LOGOUT_RPORT ) +return 1 ;list_for_each_entry (tgt, &vhost->scsi_scrqs.targets, queue)if (tgt->action ==IBMVFC_TGT_ACTION_LOGOUT_RPORT_WAIT )return 0 ; +list_for_each_entry (tgt, &vhost->nvme_scrqs.targets, queue) +if (tgt->action ==IBMVFC_TGT_ACTION_LOGOUT_RPORT_WAIT ) +return 0 ;return 1 ;case IBMVFC_HOST_ACTION_LOGO :case IBMVFC_HOST_ACTION_INIT :
@@ -5941,6 +5963,8 @@ static void ibmvfc_do_work(struct ibmvfc_host *vhost)case IBMVFC_HOST_ACTION_QUERY :list_for_each_entry (tgt, &vhost->scsi_scrqs.targets, queue)ibmvfc_init_tgt (tgt, ibmvfc_tgt_query_target); +list_for_each_entry (tgt, &vhost->nvme_scrqs.targets, queue) +ibmvfc_init_tgt (tgt, ibmvfc_tgt_query_target);ibmvfc_set_host_action (vhost,IBMVFC_HOST_ACTION_QUERY_TGTS );break ;case IBMVFC_HOST_ACTION_QUERY_TGTS :
@@ -5950,6 +5974,12 @@ static void ibmvfc_do_work(struct ibmvfc_host *vhost)break ; } } +list_for_each_entry (tgt, &vhost->nvme_scrqs.targets, queue) { +if (tgt->action ==IBMVFC_TGT_ACTION_INIT ) { + tgt->job_step (tgt); +break ; + } + }if (!ibmvfc_dev_init_to_do (vhost))ibmvfc_set_host_action (vhost,IBMVFC_HOST_ACTION_TGT_DEL );
@@ -5962,6 +5992,12 @@ static void ibmvfc_do_work(struct ibmvfc_host *vhost)break ; } } +list_for_each_entry (tgt, &vhost->nvme_scrqs.targets, queue) { +if (tgt->action ==IBMVFC_TGT_ACTION_LOGOUT_RPORT ) { + tgt->job_step (tgt); +break ; + } + }if (ibmvfc_dev_logo_to_do (vhost)) {spin_unlock_irqrestore (vhost->host->host_lock, flags);
@@ -6049,6 +6085,12 @@ static void ibmvfc_do_work(struct ibmvfc_host *vhost)break ; } } +list_for_each_entry (tgt, &vhost->nvme_scrqs.targets, queue) { +if (tgt->action ==IBMVFC_TGT_ACTION_INIT ) { + tgt->job_step (tgt); +break ; + } + }if (!ibmvfc_dev_init_to_do (vhost))ibmvfc_set_host_action (vhost,IBMVFC_HOST_ACTION_TGT_DEL_FAILED );
Signature · 1 line
2.54.0
[email protected]
AI reviewRe: [PATCH 18/29] ibmvfc: update state machine to process NVMe/FC targets
Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider: - [High] NVMe targets leak memory and become permanently stranded because their cleanup logic is omitted…
Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
- [High] NVMe targets leak memory and become permanently stranded because their cleanup logic is omitted from the target deletion phase in ibmvfc_do_work().
- [High] Asynchronous ELS events (LOGO, PRLO, PLOGI) from NVMe targets are ignored because the event handler only loops over the SCSI target list. --
commit 2221ef8b59f01b903ca3e2cc56b9bd27be8a9dc1
Author: Tyrel Datwyler <[email protected]>
ibmvfc: update state machine to process NVMe/FC targets
Update the host work loop and target state-machine helpers to process NVMe targets in addition to SCSI targets.
Quoted 4 lines
drivers/scsi/ibmvscsi/ibmvfc-core.c +0
-0
[ ... ]
drivers/scsi/ibmvscsi/ibmvfc-core.c static void ibmvfc_do_work(struct ibmvfc_host *vhost) · line 6003
Quoted 13 lines
drivers/scsi/ibmvscsi/ibmvfc-core.c +6
-0@@ -5962,6 +5992,12 @@ static void ibmvfc_do_work(struct ibmvfc_host *vhost)break ; } } +list_for_each_entry (tgt, &vhost->nvme_scrqs.targets, queue) { +if (tgt->action ==IBMVFC_TGT_ACTION_LOGOUT_RPORT ) { + tgt->job_step (tgt); +break ; + } + }if (ibmvfc_dev_logo_to_do (vhost)) {spin_unlock_irqrestore (vhost->host->host_lock, flags);
[Severity: High]
Does this code also need to loop over nvme_scrqs.targets for final target deletion?
Looking further down in ibmvfc_do_work(), there is a loop handling IBMVFC_TGT_ACTION_DEL_RPORT and IBMVFC_TGT_ACTION_DEL_AND_LOGOUT_RPORT:
list_for_each_entry(tgt, &vhost->scsi_scrqs.targets, queue) {
if (tgt->action == IBMVFC_TGT_ACTION_DEL_RPORT) {
tgt_dbg(tgt, "Deleting rport\n");
rport = tgt->rport;
tgt->rport = NULL;
list_del(&tgt->queue);
ibmvfc_set_tgt_action(tgt, IBMVFC_TGT_ACTION_DELETED_RPORT);
...Since this loop only processes scsi_scrqs.targets, wouldn't NVMe targets in the DEL_RPORT state be leaked and stranded?
[ ... ]
[Severity: High]
Are asynchronous ELS events from NVMe targets being ignored?
Looking at ibmvfc_handle_async(), which processes events like LOGO, PRLO, and PLOGI, the loop only iterates over the SCSI targets:
case IBMVFC_AE_ELS_LOGO:
case IBMVFC_AE_ELS_PRLO:
case IBMVFC_AE_ELS_PLOGI:
list_for_each_entry(tgt, &vhost->scsi_scrqs.targets, queue) {Since the NVMe targets are now in a separate nvme_scrqs.targets list, wouldn't an explicit logout or state change from an NVMe target be dropped, leaving the driver with stale state?
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=18
Tyrel Datwyler
maintainerPATCH 19/29 ibmvfc: implement NVMe/FC stubs for local/remote port registration
In reply to Tyrel DatwylerImplement the initial NVMe/FC local-port and remote-port registration functions that notify the NVMe-FC midlayr of port discovery and loss.
Register the local port with the NVMe-FC transport, register discovered remote ports against that local port, and add matching unregister paths that wait for the NVMe-FC core to complete asynchronous deletion before dropping driver references. Also store driver-private host and target pointers in the registered NVMe-FC port objects.
--- drivers/scsi/ibmvscsi/ibmvfc-nvme.c | 79 ++++++++++++++++++++++++++++- drivers/scsi/ibmvscsi/ibmvfc-nvme.h | 4 +- drivers/scsi/ibmvscsi/ibmvfc.h | 4 ++ 3 files changed, 83 insertions(+), 4 deletions(-)
drivers/scsi/ibmvscsi/ibmvfc-nvme.c +77 -2
@@ -14,10 +14,18 @@static void ibmvfc_nvme_localport_delete (struct nvme_fc_local_port *lport) { +struct ibmvfc_host *vhost = lport->private; + + vhost->nvme_local_port =NULL ; +complete (&vhost->nvme_delete_done); }static void ibmvfc_nvme_remoteport_delete (struct nvme_fc_remote_port *rport) { +struct ibmvfc_target *tgt = rport->private; + + tgt->nvme_remote_port =NULL ; +complete (&tgt->nvme_delete_done); } static int ibmvfc_nvme_ls_req(struct nvme_fc_local_port *lport,
@@ -70,18 +78,85 @@ static struct nvme_fc_port_template ibmvfc_nvme_fc_transport = {int ibmvfc_nvme_register_remoteport (struct ibmvfc_target *tgt) { -return 0 ; +struct ibmvfc_host *vhost = tgt->vhost; +struct nvme_fc_port_info pinfo; +int rc; + +if (!vhost->nvme_local_port) { +dev_err (vhost->dev,"Attempt to register NVMe fc remoteport without valid localport\n" ); +return -EINVAL ; + } + +memset (&pinfo,0 ,sizeof (struct nvme_fc_port_info )); + pinfo.node_name = tgt->ids.node_name; + pinfo.port_name = tgt->ids.port_name; + pinfo.port_id = tgt->ids.port_id; + pinfo.port_role =FC_PORT_ROLE_NVME_TARGET ; + + rc =nvme_fc_register_remoteport (vhost->nvme_local_port, &pinfo, + &tgt->nvme_remote_port); + +if (!rc) { +ibmvfc_log (vhost,2 ,"register_remoteport: traddr=nn-0x%llx:pn-0x%llx PortID:%x\n" , + pinfo.node_name, pinfo.port_name, pinfo.port_id); + tgt->nvme_remote_port->private = tgt; + } + +return rc; }void ibmvfc_nvme_unregister_remoteport (struct ibmvfc_target *tgt) { +struct ibmvfc_host *vhost = tgt->vhost; +struct nvme_fc_remote_port *rport = tgt->nvme_remote_port; +int rc; + +if (!tgt->nvme_remote_port) +return ; + +ibmvfc_log (vhost,2 ,"unregister_remoteport: traddr=nn-0x%llx:pn-0x%llx PortID:%x\n" , + rport->node_name, rport->port_name, rport->port_id); +init_completion (&tgt->nvme_delete_done); + rc =nvme_fc_unregister_remoteport (tgt->nvme_remote_port); + +if (!rc) { +wait_for_completion (&tgt->nvme_delete_done); + tgt->nvme_remote_port->private =NULL ; + } }int ibmvfc_nvme_register (struct ibmvfc_host *vhost) { -return 0 ; +struct nvme_fc_port_info pinfo; +int rc; + + pinfo.node_name =fc_host_node_name (vhost->host); + pinfo.port_name =fc_host_port_name (vhost->host); + pinfo.port_id =fc_host_port_id (vhost->host); + pinfo.port_role =FC_PORT_ROLE_NVME_INITIATOR ; + pinfo.dev_loss_tmo =0 ; + + rc =nvme_fc_register_localport (&pinfo, &ibmvfc_nvme_fc_transport, + vhost->dev, &vhost->nvme_local_port); + +if (!rc) { +ibmvfc_log (vhost,2 ,"register_localport: host-traddr=nn-0x%llx:pn-0x%llx on portID:%x\n" , + pinfo.node_name, pinfo.port_name, pinfo.port_id); + vhost->nvme_local_port->private = vhost; + }else +dev_err (vhost->dev,"Failed to register NVMe fc localport (%d)\n" , rc); + +return rc; }void ibmvfc_nvme_unregister (struct ibmvfc_host *vhost) { +int rc; + +if (vhost->nvme_local_port) { +init_completion (&vhost->nvme_delete_done); + rc =nvme_fc_unregister_localport (vhost->nvme_local_port); +if (!rc) +wait_for_completion (&vhost->nvme_delete_done); + } }
drivers/scsi/ibmvscsi/ibmvfc-nvme.h +2 -2
@@ -11,8 +11,8 @@#ifndef _IBMVFC_NVME_H#define _IBMVFC_NVME_H -#include <uapi/scsi/fc/fc_fs.h> -#include <uapi/scsi/fc/fc_els.h> +#include <scsi/fc/fc_fs.h> +#include <scsi/fc/fc_els.h> #include <linux/nvme-fc-driver.h> #include "ibmvfc.h"
drivers/scsi/ibmvscsi/ibmvfc.h +4 -0
@@ -839,6 +839,8 @@ struct ibmvfc_target {void (*job_step) (struct ibmvfc_target *);struct timer_list timer;struct kref kref; +struct completion nvme_delete_done; +struct nvme_fc_remote_port *nvme_remote_port; };/* a unit of work for the hosting partition */
@@ -1015,6 +1017,8 @@ struct ibmvfc_host {struct work_struct rport_add_work_q;wait_queue_head_t init_wait_q;wait_queue_head_t work_wait_q; +struct nvme_fc_local_port *nvme_local_port; +struct completion nvme_delete_done; };static inline struct ibmvfc_host *ibmvfc_channels_to_vhost (struct ibmvfc_channels *channels)
Signature · 1 line
2.54.0
[email protected]
AI reviewRe: [PATCH 19/29] ibmvfc: implement NVMe/FC stubs for local/remote port registration
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [High] Unconditional NULL pointer dereference in NVMe-FC local and remote port unregistration paths. --
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [High] Unconditional NULL pointer dereference in NVMe-FC local and remote port unregistration paths. --
commit 70b1c2d3d0ca257b156db2ba118e3a6f944e519d
Author: Tyrel Datwyler <[email protected]>
ibmvfc: implement NVMe/FC stubs for local/remote port registration
Implement the initial NVMe/FC local-port and remote-port registration functions that notify the NVMe-FC midlayr of port discovery and loss.
Quoted 21 lines
drivers/scsi/ibmvscsi/ibmvfc-nvme.c +8
-0@@ -14,10 +14,18 @@static void ibmvfc_nvme_localport_delete (struct nvme_fc_local_port *lport) { +struct ibmvfc_host *vhost = lport->private; + + vhost->nvme_local_port =NULL ; +complete (&vhost->nvme_delete_done); }static void ibmvfc_nvme_remoteport_delete (struct nvme_fc_remote_port *rport) { +struct ibmvfc_target *tgt = rport->private; + + tgt->nvme_remote_port =NULL ; +complete (&tgt->nvme_delete_done); }
[ ... ]
Quoted 4 lines
drivers/scsi/ibmvscsi/ibmvfc-nvme.c +0
-0@@ -70,18 +78,85 @@ static struct nvme_fc_port_template ibmvfc_nvme_fc_transport = {int ibmvfc_nvme_register_remoteport (struct ibmvfc_target *tgt) {
[ ... ]
Quoted 7 lines
void
ibmvfc_nvme_unregister_remoteport(struct ibmvfc_target *tgt){drivers/scsi/ibmvscsi/ibmvfc-nvme.c +5
-0+struct ibmvfc_host *vhost = tgt->vhost; +struct nvme_fc_remote_port *rport = tgt->nvme_remote_port; +int rc; + +if (!tgt->nvme_remote_port)
drivers/scsi/ibmvscsi/ibmvfc-nvme.c static struct nvme_fc_port_template ibmvfc_nvme_fc_transport = { · line 124
Quoted 10 lines
drivers/scsi/ibmvscsi/ibmvfc-nvme.c +10
-0+return ; + +ibmvfc_log (vhost,2 ,"unregister_remoteport: traddr=nn-0x%llx:pn-0x%llx PortID:%x\n" , + rport->node_name, rport->port_name, rport->port_id); +init_completion (&tgt->nvme_delete_done); + rc =nvme_fc_unregister_remoteport (tgt->nvme_remote_port); + +if (!rc) { +wait_for_completion (&tgt->nvme_delete_done); + tgt->nvme_remote_port->private =NULL ;
[Severity: High]
Does this code unconditionally dereference a NULL pointer?
When nvme_fc_unregister_remoteport() finishes, the asynchronous delete callback ibmvfc_nvme_remoteport_delete() is called. This callback executes:
tgt->nvme_remote_port = NULL;
complete(&tgt->nvme_delete_done);After waking up from wait_for_completion(), tgt->nvme_remote_port will always be NULL, meaning the access to tgt->nvme_remote_port->private will cause a kernel panic.
Quoted 2 lines
drivers/scsi/ibmvscsi/ibmvfc-nvme.c +1
-0+ } }
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=19
Tyrel Datwyler
maintainerPATCH 20/29 ibmvfc: register local nvme fc port after fabric login
In reply to Tyrel DatwylerRegister the local NVMe/FC port only after fabric login has completed.
The VIOS returns the client port ID in the fabric login response, and that port ID is required to populate the local-port information passed to the NVMe-FC midlayer. Delay local-port registration until that data is available and update the registration helper accordingly.
--- drivers/scsi/ibmvscsi/ibmvfc-core.c | 3 +++ drivers/scsi/ibmvscsi/ibmvfc-nvme.c | 18 +++++++++++++++--- 2 files changed, 18 insertions(+), 3 deletions(-)
drivers/scsi/ibmvscsi/ibmvfc-core.c +3 -0
@@ -5184,6 +5184,8 @@ static void ibmvfc_fabric_login_nvme_done(struct ibmvfc_event *evt)switch (mad_status) {case IBMVFC_MAD_SUCCESS : +fc_host_port_id (vhost->host) =be64_to_cpu (rsp->nport_id); +ibmvfc_nvme_register (vhost);ibmvfc_dbg (vhost,"NVMe fabric login succeeded\n" );break ;case IBMVFC_MAD_FAILED :
@@ -5245,6 +5247,7 @@ static void ibmvfc_fabric_login_scsi_done(struct ibmvfc_event *evt)switch (mad_status) {case IBMVFC_MAD_SUCCESS : +fc_host_port_id (vhost->host) =be64_to_cpu (rsp->nport_id);ibmvfc_dbg (vhost,"SCSI fabric login succeeded\n" );break ;case IBMVFC_MAD_FAILED :
drivers/scsi/ibmvscsi/ibmvfc-nvme.c +15 -3
@@ -137,14 +137,17 @@ int ibmvfc_nvme_register(struct ibmvfc_host *vhost) pinfo.dev_loss_tmo =0 ; rc =nvme_fc_register_localport (&pinfo, &ibmvfc_nvme_fc_transport, - vhost->dev, &vhost->nvme_local_port); +get_device (vhost->dev), + &vhost->nvme_local_port);if (!rc) {ibmvfc_log (vhost,2 ,"register_localport: host-traddr=nn-0x%llx:pn-0x%llx on portID:%x\n" , pinfo.node_name, pinfo.port_name, pinfo.port_id); vhost->nvme_local_port->private = vhost; - }else + }else {dev_err (vhost->dev,"Failed to register NVMe fc localport (%d)\n" , rc); +put_device (vhost->dev); + }return rc; }
@@ -154,9 +157,18 @@ void ibmvfc_nvme_unregister(struct ibmvfc_host *vhost)int rc;if (vhost->nvme_local_port) { +ibmvfc_log (vhost,2 ,"unregister_localport: host-traddr=nn-0x%llx:pn-0x%llx on portID:%x\n" , + vhost->nvme_local_port->node_name, + vhost->nvme_local_port->port_name, + vhost->nvme_local_port->port_id);init_completion (&vhost->nvme_delete_done); rc =nvme_fc_unregister_localport (vhost->nvme_local_port); -if (!rc) +if (!rc) {wait_for_completion (&vhost->nvme_delete_done); + vhost->nvme_local_port->private =NULL ; + }else +dev_err (vhost->dev,"Failed to unregister NVMe fc localport (%d)\n" , rc); + +put_device (vhost->dev); } }
Signature · 1 line
2.54.0
[email protected]
AI reviewRe: [PATCH 20/29] ibmvfc: register local nvme fc port after fabric login
Thank you for your contribution! Sashiko AI review found 4 potential issue(s) to consider:
Thank you for your contribution! Sashiko AI review found 4 potential issue(s) to consider:
New issues:
- [High] Guaranteed NULL pointer dereference in ibmvfc_nvme_unregister() upon successful local port unregistration.
- [High] Permanent memory, device reference, and state leak due to unconditional NVMe local port registration without teardown.
- [High] Sleeping in atomic context bug by calling nvme_fc_register_localport() from a CRQ response handler.
Pre-existing issues:
- [High] Guaranteed NULL pointer dereference in ibmvfc_nvme_unregister_remoteport(). --
commit 84b5a798e09cfb51baf820040579b6001b3aae40
Author: Tyrel Datwyler <[email protected]> Date: Tue Jun 23 01:30:26 2026 +0000
ibmvfc: register local nvme fc port after fabric login
Register the local NVMe/FC port only after fabric login has completed. The VIOS returns the client port ID in the fabric login response, and that port ID is required to populate the local-port information passed to the NVMe-FC midlayer.
drivers/scsi/ibmvscsi/ibmvfc-core.c static void ibmvfc_fabric_login_nvme_done(struct ibmvfc_event *evt) · line 5188
Quoted 10 lines
drivers/scsi/ibmvscsi/ibmvfc-core.c +2
-0@@ -5184,6 +5184,8 @@ static void ibmvfc_fabric_login_nvme_done(struct ibmvfc_event *evt)switch (mad_status) {case IBMVFC_MAD_SUCCESS : +fc_host_port_id (vhost->host) =be64_to_cpu (rsp->nport_id); +ibmvfc_nvme_register (vhost);
[Severity: High]
Does this code sleep in atomic context?
It looks like ibmvfc_fabric_login_nvme_done() is executed as a callback from ibmvfc_locked_done(), which holds the host_lock spinlock with interrupts disabled:
ibmvfc_locked_done() {
spin_lock_irqsave(evt->vhost->host->host_lock, flags);
evt->_done(evt);
spin_unlock_irqrestore(evt->vhost->host->host_lock, flags);
}The callback invokes ibmvfc_nvme_register(), which calls the NVMe FC midlayer function nvme_fc_register_localport(). This midlayer function makes GFP_KERNEL allocations, which could lead to deadlocks or panics when atomic sleep checks are enabled.
Additionally, does this introduce a permanent resource leak?
This function unconditionally registers a new NVMe local port upon every successful fabric login. These logins can happen repeatedly during probe, adapter resets, or network link flaps. Since it never unregisters the previously registered port before grabbing a new device reference, it appears the memory, device references, and NVMe transport state will leak.
Quoted 3 lines
drivers/scsi/ibmvscsi/ibmvfc-core.c +0
-0ibmvfc_dbg (vhost,"NVMe fabric login succeeded\n" );break ;case IBMVFC_MAD_FAILED :
[ ... ]
Quoted 4 lines
drivers/scsi/ibmvscsi/ibmvfc-nvme.c +0
-0
[ ... ]
Quoted 4 lines
drivers/scsi/ibmvscsi/ibmvfc-nvme.c +0
-0@@ -154,9 +157,18 @@ void ibmvfc_nvme_unregister(struct ibmvfc_host *vhost)int rc;if (vhost->nvme_local_port) {
drivers/scsi/ibmvscsi/ibmvfc-nvme.c void ibmvfc_nvme_unregister(struct ibmvfc_host *vhost) · line 168
Quoted 10 lines
drivers/scsi/ibmvscsi/ibmvfc-nvme.c +6
-1+ibmvfc_log (vhost,2 ,"unregister_localport: host-traddr=nn-0x%llx:pn-0x%llx on portID:%x\n" , + vhost->nvme_local_port->node_name, + vhost->nvme_local_port->port_name, + vhost->nvme_local_port->port_id);init_completion (&vhost->nvme_delete_done); rc =nvme_fc_unregister_localport (vhost->nvme_local_port); -if (!rc) +if (!rc) {wait_for_completion (&vhost->nvme_delete_done); + vhost->nvme_local_port->private =NULL ;
[Severity: High]
Can this dereference a NULL pointer?
The completion for port deletion is signaled by the midlayer callback ibmvfc_nvme_localport_delete(), which explicitly clears the local port pointer:
ibmvfc_nvme_localport_delete() {
vhost->nvme_local_port = NULL;
complete(&vhost->nvme_delete_done);
}When wait_for_completion() returns here, vhost->nvme_local_port is guaranteed to be NULL, meaning the assignment to private will unconditionally cause a crash.
Quoted 6 lines
drivers/scsi/ibmvscsi/ibmvfc-nvme.c +4
-0+ }else +dev_err (vhost->dev,"Failed to unregister NVMe fc localport (%d)\n" , rc); + +put_device (vhost->dev); } }
[ ... ]
Quoted 7 lines
static void
ibmvfc_nvme_unregister_remoteport(struct ibmvfc_target *tgt){ if (tgt->nvme_remote_port) { init_completion(&tgt->nvme_delete_done); nvme_fc_unregister_remoteport(tgt->nvme_remote_port); wait_for_completion(&tgt->nvme_delete_done); tgt->nvme_remote_port->private = NULL;
[Severity: High]
This is a pre-existing issue, but does this suffer from the same NULL pointer dereference issue as the local port unregister function?
Similar to the local port, the completion is signaled by ibmvfc_nvme_remoteport_delete(), which sets tgt->nvme_remote_port = NULL before completing. This means tgt->nvme_remote_port will be NULL when the wait finishes, leading to an unconditional crash here during target teardown.
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=20
Tyrel Datwyler
maintainerPATCH 21/29 ibmvfc: process NVMe/FC rports in work thread
In reply to Tyrel DatwylerAdd an NVMe-specific remote-port add helper and update the rport worker thread to walk the NVMe target list, register new NVMe remote ports, and rescan existing ones through the NVMe-FC midlayer. Also handle delete and delete-with-logout transitions for NVMe remote ports in the same worker context used for SCSI rports.
This keeps remote-port registration serialized in the existing worker model while allowing NVMe targets to participate in the common target state machine.
--- drivers/scsi/ibmvscsi/ibmvfc-core.c | 68 ++++++++++++++++++++++++++++- 1 file changed, 67 insertions(+), 1 deletion(-)
drivers/scsi/ibmvscsi/ibmvfc-core.c +67 -1
@@ -5829,6 +5829,51 @@ static void ibmvfc_log_ae(struct ibmvfc_host *vhost, int events)fc_host_post_event (vhost->host,fc_get_event_number (),FCH_EVT_LINKUP ,0 ); } +/** +* ibmvfc_tgt_add_nvme_rport - Tell the FC transport about a new remote port +* @tgt: ibmvfc target struct +* +**/ +static void ibmvfc_tgt_add_nvme_rport (struct ibmvfc_target *tgt) +{ +struct ibmvfc_host *vhost = tgt->vhost; +struct nvme_fc_remote_port *rport; +unsigned long flags; + +tgt_dbg (tgt,"Adding NVMe rport\n" ); +ibmvfc_nvme_register_remoteport (tgt); +spin_lock_irqsave (vhost->host->host_lock, flags); + rport = tgt->nvme_remote_port; + +if (rport && tgt->action ==IBMVFC_TGT_ACTION_DEL_RPORT ) { +tgt_dbg (tgt,"Deleting NVMe rport\n" ); +list_del (&tgt->queue); +ibmvfc_set_tgt_action (tgt,IBMVFC_TGT_ACTION_DELETED_RPORT ); +spin_unlock_irqrestore (vhost->host->host_lock, flags); +ibmvfc_nvme_unregister_remoteport (tgt); +timer_delete_sync (&tgt->timer); +kref_put (&tgt->kref, ibmvfc_release_tgt); +return ; + }else if (rport && tgt->action ==IBMVFC_TGT_ACTION_DEL_AND_LOGOUT_RPORT ) { +tgt_dbg (tgt,"Deleting NVMe rport with outstanding I/O\n" ); +ibmvfc_set_tgt_action (tgt,IBMVFC_TGT_ACTION_LOGOUT_DELETED_RPORT ); + tgt->init_retries =0 ; +spin_unlock_irqrestore (vhost->host->host_lock, flags); +ibmvfc_nvme_unregister_remoteport (tgt); +return ; + }else if (rport && tgt->action ==IBMVFC_TGT_ACTION_DELETED_RPORT ) { +spin_unlock_irqrestore (vhost->host->host_lock, flags); +return ; + } + +if (rport) { +tgt_dbg (tgt,"NVMe rport add succeeded\n" ); + tgt->target_id = tgt->nvme_remote_port->port_id; + }else +tgt_dbg (tgt,"NVMe rport add failed\n" ); +spin_unlock_irqrestore (vhost->host->host_lock, flags); +} +/** * ibmvfc_tgt_add_rport - Tell the FC transport about a new remote port * @tgt: ibmvfc target struct
@@ -6630,6 +6675,7 @@ static void ibmvfc_rport_add_thread(struct work_struct *work) rport_add_work_q);struct ibmvfc_target *tgt;struct fc_rport *rport; +struct nvme_fc_remote_port *nvme_rport;unsigned long flags;int did_work;
@@ -6663,7 +6709,27 @@ static void ibmvfc_rport_add_thread(struct work_struct *work)break ; } } - }while (did_work); + +list_for_each_entry (tgt, &vhost->nvme_scrqs.targets, queue) { +if (tgt->add_rport) { + did_work =1 ; + tgt->add_rport =0 ; +kref_get (&tgt->kref); + nvme_rport = tgt->nvme_remote_port; +if (!nvme_rport) { +spin_unlock_irqrestore (vhost->host->host_lock, flags); +ibmvfc_tgt_add_nvme_rport (tgt); + }else { +spin_unlock_irqrestore (vhost->host->host_lock, flags); +nvme_fc_rescan_remoteport (nvme_rport); + } + +kref_put (&tgt->kref, ibmvfc_release_tgt); +spin_lock_irqsave (vhost->host->host_lock, flags); +break ; + } + } + }while (did_work);if (vhost->state ==IBMVFC_ACTIVE ) vhost->scan_complete =1 ;
Signature · 1 line
2.54.0
[email protected]
AI reviewRe: [PATCH 21/29] ibmvfc: process NVMe/FC rports in work thread
Thank you for your contribution! Sashiko AI review found 5 potential issue(s) to consider:
Thank you for your contribution! Sashiko AI review found 5 potential issue(s) to consider:
New issues:
- [Critical] Deterministic NULL pointer dereference during NVMe remote port and local port unregistration.
- [High] NVMe targets marked for deletion are completely ignored by the main state machine, resulting in permanent resource leaks and stuck target states.
- [High] ibmvfc_nvme_unregister is dead code, permanently leaking the NVMe local port on module unload.
Pre-existing issues:
- [Critical] Concurrent target deletion during port addition completely leaks the newly registered remote port, leading to a Use-After-Free of the private data pointer for NVMe targets.
- [High] Target structures allocated from a mempool are incorrectly freed using kfree(), draining the mempool reserves and leading to hangs under memory pressure. --
commit f73360d74019cb783c285fb8eef798a237334c34
Author: Tyrel Datwyler <[email protected]>
ibmvfc: process NVMe/FC rports in work thread
This commit adds an NVMe-specific remote-port add helper and updates the rport worker thread to walk the NVMe target list. It handles registration, rescan, and deletion transitions for NVMe remote ports in the same worker context used for SCSI rports.
Quoted 3 lines
drivers/scsi/ibmvscsi/ibmvfc-core.c +0
-0
[ ... ]
Quoted 3 lines
drivers/scsi/ibmvscsi/ibmvfc-core.c +3
-0+static void ibmvfc_tgt_add_nvme_rport (struct ibmvfc_target *tgt) +{ +struct ibmvfc_host *vhost = tgt->vhost;
[ ... ]
drivers/scsi/ibmvscsi/ibmvfc-core.c static void ibmvfc_log_ae(struct ibmvfc_host *vhost, int events) · line 5853
Quoted 6 lines
drivers/scsi/ibmvscsi/ibmvfc-core.c +6
-0+if (rport && tgt->action ==IBMVFC_TGT_ACTION_DEL_RPORT ) { +tgt_dbg (tgt,"Deleting NVMe rport\n" ); +list_del (&tgt->queue); +ibmvfc_set_tgt_action (tgt,IBMVFC_TGT_ACTION_DELETED_RPORT ); +spin_unlock_irqrestore (vhost->host->host_lock, flags); +ibmvfc_nvme_unregister_remoteport (tgt);
[Severity: Critical]
Does ibmvfc_nvme_unregister_remoteport() contain a null pointer dereference?
When unregistering the remote port, the completion callback appears to explicitly set the tgt->nvme_remote_port pointer to NULL:
drivers/scsi/ibmvscsi/ibmvfc-nvme.c:ibmvfc_nvme_unregister_remoteport() {
...
rc = nvme_fc_unregister_remoteport(tgt->nvme_remote_port);
if (!rc) {
wait_for_completion(&tgt->nvme_delete_done);
tgt->nvme_remote_port->private = NULL;
}
}Will this attempt to dereference tgt->nvme_remote_port right after it was set to NULL by ibmvfc_nvme_remoteport_delete before the completion signaled? The same pattern seems to exist in ibmvfc_nvme_unregister() for the local port.
drivers/scsi/ibmvscsi/ibmvfc-core.c static void ibmvfc_log_ae(struct ibmvfc_host *vhost, int events) · line 5855
Quoted 2 lines
drivers/scsi/ibmvscsi/ibmvfc-core.c +2
-0+timer_delete_sync (&tgt->timer); +kref_put (&tgt->kref, ibmvfc_release_tgt);
[Severity: High]
This is a pre-existing issue, but does ibmvfc_release_tgt() incorrectly free mempool-allocated targets?
The targets are allocated via mempool_alloc(vhost->tgt_pool, GFP_NOIO) in ibmvfc_alloc_targets(), but the kref release function calls kfree(tgt):
drivers/scsi/ibmvscsi/ibmvfc-core.c:ibmvfc_release_tgt() {
struct ibmvfc_target *tgt = container_of(kref, struct ibmvfc_target, kref);
kfree(tgt);
}Does this bypass the mempool reserves and eventually cause mempool_alloc() to hang under memory pressure?
Quoted 2 lines
drivers/scsi/ibmvscsi/ibmvfc-core.c +2
-0+return ; + }else if (rport && tgt->action ==IBMVFC_TGT_ACTION_DEL_AND_LOGOUT_RPORT ) {
[ ... ]
drivers/scsi/ibmvscsi/ibmvfc-core.c static void ibmvfc_log_ae(struct ibmvfc_host *vhost, int events) · line 5867
Quoted 4 lines
drivers/scsi/ibmvscsi/ibmvfc-core.c +4
-0+ }else if (rport && tgt->action ==IBMVFC_TGT_ACTION_DELETED_RPORT ) { +spin_unlock_irqrestore (vhost->host->host_lock, flags); +return ; + }
[Severity: Critical]
This is a pre-existing issue, but can this early return leak the newly registered remote port?
If tgt->action is transitioned to IBMVFC_TGT_ACTION_DELETED_RPORT by the state machine while the worker thread drops the lock to register the port, we return early here without unregistering the newly created port.
For NVMe targets, tgt->nvme_remote_port->private points to the tgt structure. If the tgt structure is subsequently freed via kref_put(), could this cause a use-after-free in the NVMe midlayer when it uses the dangling private data pointer?
[ ... ]
Quoted 1 line
drivers/scsi/ibmvscsi/ibmvfc-core.c +0
-0@@ -6630,6 +6675,7 @@ static void ibmvfc_rport_add_thread(struct work_struct *work)
[ ... ]
drivers/scsi/ibmvscsi/ibmvfc-core.c static void ibmvfc_rport_add_thread(struct work_struct *work) · line 6714
Quoted 4 lines
drivers/scsi/ibmvscsi/ibmvfc-core.c +3
-1- }while (did_work); + +list_for_each_entry (tgt, &vhost->nvme_scrqs.targets, queue) { +if (tgt->add_rport) {
[Severity: High]
Are NVMe targets marked for deletion completely ignored by the main state machine?
The normal deletion path clears tgt->add_rport, which causes this worker thread to ignore them. However, looking at the main state machine in ibmvfc_do_work(), the deletion loop exists only for SCSI targets:
drivers/scsi/ibmvscsi/ibmvfc-core.c:ibmvfc_do_work() {
...
list_for_each_entry(tgt, &vhost->scsi_scrqs.targets, queue) {
if (tgt->action == IBMVFC_TGT_ACTION_DEL_RPORT) {
...
}Since the equivalent loop is omitted for nvme_scrqs.targets, do targets marked for deletion remain unlinked and permanently leaked?
[Severity: High]
On a related note, does the driver fail to unregister the NVMe local port on module unload?
The teardown path in ibmvfc_remove() removes the SCSI host via scsi_remove_host() but does not appear to invoke the NVMe equivalent logic. There is an ibmvfc_nvme_unregister() function provided in the code, but it seems to be completely dead code that is never called.
Could this lead to a permanent resource leak and potential use-after-free in the NVMe midlayer if it interacts with the removed host?
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=21
Tyrel Datwyler
maintainerPATCH 22/29 ibmvfc: extend ibmvfc_debug visibility to ibmvfc-nvme.h
In reply to Tyrel DatwylerExport ibmvfc_debug so the NVMe support code can use the existing ibmvfc_dbg logging macro.
The debug control variable is currently file-local to the core driver, which prevents protocol-specific code in ibmvfc-nvme.c from using the shared debug infrastructure. Make the variable global within the module and declare it in ibmvfc-nvme.h.
--- drivers/scsi/ibmvscsi/ibmvfc-core.c | 3 ++- drivers/scsi/ibmvscsi/ibmvfc-nvme.h | 2 ++ 2 files changed, 4 insertions(+), 1 deletion(-)
drivers/scsi/ibmvscsi/ibmvfc-core.c +2 -1
@@ -41,7 +41,6 @@ static unsigned int max_requests = IBMVFC_MAX_REQUESTS_DEFAULT;static u16 max_sectors =IBMVFC_MAX_SECTORS ;static u16 scsi_qdepth =IBMVFC_SCSI_QDEPTH ;static unsigned int disc_threads =IBMVFC_MAX_DISC_THREADS ; -static unsigned int ibmvfc_debug =IBMVFC_DEBUG ;static unsigned int log_level =IBMVFC_DEFAULT_LOG_LEVEL ;static unsigned int cls3_error =IBMVFC_CLS3_ERROR ;static unsigned int mq_enabled =IBMVFC_MQ ;
@@ -53,6 +52,8 @@ static unsigned int nr_nvme_channels = IBMVFC_NVME_CHANNELS;static unsigned int mig_channels_only =IBMVFC_MIG_NO_SUB_TO_CRQ ;static unsigned int mig_no_less_channels =IBMVFC_MIG_NO_N_TO_M ; +unsigned int ibmvfc_debug =IBMVFC_DEBUG ; +static LIST_HEAD (ibmvfc_head);static DEFINE_SPINLOCK (ibmvfc_driver_lock);static struct scsi_transport_template *ibmvfc_transport_template;
drivers/scsi/ibmvscsi/ibmvfc-nvme.h +2 -0
@@ -22,6 +22,8 @@#define IBMVFC_MAX_NVME_QUEUES 16#define IBMVFC_NVME_CHANNELS 8 +extern unsigned int ibmvfc_debug; +struct ibmvfc_host ;struct ibmvfc_target ;
Signature · 1 line
2.54.0
[email protected]
AI reviewRe: [PATCH 22/29] ibmvfc: extend ibmvfc_debug visibility to ibmvfc-nvme.h
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [Low] Core driver debug variable declared in protocol-specific header --
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Low] Core driver debug variable declared in protocol-specific header
--
commit 719e4b33f0990bbcd15f1244a436cf69d36807ef
Author: Tyrel Datwyler <[email protected]>
ibmvfc: extend ibmvfc_debug visibility to ibmvfc-nvme.h
Export ibmvfc_debug so the NVMe support code can use the existing ibmvfc_dbg logging macro.
drivers/scsi/ibmvscsi/ibmvfc-nvme.h line 26
Quoted 10 lines
drivers/scsi/ibmvscsi/ibmvfc-nvme.h +2
-0@@ -22,6 +22,8 @@#define IBMVFC_MAX_NVME_QUEUES 16#define IBMVFC_NVME_CHANNELS 8 +extern unsigned int ibmvfc_debug; +
[Severity: Low]
This isn't a bug, but should this core driver debug variable be declared in the core ibmvfc.h header rather than this protocol-specific header?
Declaring it here creates a confusing dependency where the core header relies on a protocol header for its macro dependencies.
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=22
Tyrel Datwyler
maintainerPATCH 23/29 ibmvfc: declare global function definitions
In reply to Tyrel DatwylerSome common functions will require visibility by both SCSI and NVMe protocols. Make common ibmvfc helper routines available to the NVMe support code.
Remove static from the core event allocation, event initialization, event free, target release, command error, and event send helpers, and declare them in ibmvfc.h. This allows ibmvfc-nvme.c to reuse the existing event and target infrastructure.
No functional change is intended.
--- drivers/scsi/ibmvscsi/ibmvfc-core.c | 22 ++++++---------------- drivers/scsi/ibmvscsi/ibmvfc.h | 17 +++++++++++++++++ 2 files changed, 23 insertions(+), 16 deletions(-)
drivers/scsi/ibmvscsi/ibmvfc-core.c +6 -16
@@ -206,13 +206,6 @@ static long h_reg_sub_crq(unsigned long unit_address, unsigned long ioba,return rc; } -static int ibmvfc_check_caps (struct ibmvfc_host *vhost,unsigned long cap_flags) -{ -u64 host_caps =be64_to_cpu (vhost->login_buf->resp.capabilities); - -return (host_caps & cap_flags) ?1 :0 ; -} -static int ibmvfc_nvme_active (struct ibmvfc_host *vhost) { return (ibmvfc_check_caps(vhost, IBMVFC_SUPPORT_NVMEOF) &&
@@ -354,7 +347,7 @@ static int ibmvfc_get_err_index(u16 status, u16 error)* Return value: * error description string **/ -static const char *ibmvfc_get_cmd_error (u16 status,u16 error) +const char *ibmvfc_get_cmd_error (u16 status,u16 error) {int rc =ibmvfc_get_err_index (status, error);if (rc >=0 )
@@ -1076,7 +1069,7 @@ static int ibmvfc_valid_event(struct ibmvfc_event_pool *pool,* @evt: ibmvfc_event to be freed * **/ -static void ibmvfc_free_event (struct ibmvfc_event *evt) +void ibmvfc_free_event (struct ibmvfc_event *evt) {struct ibmvfc_event_pool *pool = &evt->queue->evt_pool;unsigned long flags;
@@ -1410,7 +1403,7 @@ static void ibmvfc_set_rport_dev_loss_tmo(struct fc_rport *rport, u32 timeout)* @kref: kref struct * **/ -static void ibmvfc_release_tgt (struct kref *kref) +void ibmvfc_release_tgt (struct kref *kref) { struct ibmvfc_target *tgt = container_of(kref, struct ibmvfc_target, kref);kfree (tgt);
@@ -1588,7 +1581,7 @@ static void ibmvfc_set_login_info(struct ibmvfc_host *vhost)* * Returns a free event from the pool. **/ -static struct ibmvfc_event *__ibmvfc_get_event (struct ibmvfc_queue *queue,int reserved) +struct ibmvfc_event *__ibmvfc_get_event (struct ibmvfc_queue *queue,int reserved) {struct ibmvfc_event *evt =NULL ;unsigned long flags;
@@ -1612,9 +1605,6 @@ static struct ibmvfc_event *__ibmvfc_get_event(struct ibmvfc_queue *queue, int rreturn evt; } -#define ibmvfc_get_event (queue) __ibmvfc_get_event(queue, 0) -#define ibmvfc_get_reserved_event (queue) __ibmvfc_get_event(queue, 1) -/** * ibmvfc_locked_done - Calls evt completion with host_lock held * @evt: ibmvfc evt to complete
@@ -1639,7 +1629,7 @@ static void ibmvfc_locked_done(struct ibmvfc_event *evt)* @done: Routine to call when the event is responded to * @format: SRP or MAD format **/ -static void ibmvfc_init_event (struct ibmvfc_event *evt, +void ibmvfc_init_event (struct ibmvfc_event *evt,void (*done) (struct ibmvfc_event *),u8 format) { evt->cmnd =NULL ;
@@ -1764,7 +1754,7 @@ static void ibmvfc_timeout(struct timer_list *t)* * Returns the value returned from ibmvfc_send_crq(). (Zero for success) **/ -static int ibmvfc_send_event (struct ibmvfc_event *evt, +int ibmvfc_send_event (struct ibmvfc_event *evt,struct ibmvfc_host *vhost,unsigned long timeout) {__be64 *crq_as_u64 = (__be64 *) &evt->crq;
drivers/scsi/ibmvscsi/ibmvfc.h +17 -0
@@ -1021,6 +1021,23 @@ struct ibmvfc_host {struct completion nvme_delete_done; }; +struct ibmvfc_event *__ibmvfc_get_event (struct ibmvfc_queue *queue,int reserved); +#define ibmvfc_get_event (queue) __ibmvfc_get_event(queue, 0) +#define ibmvfc_get_reserved_event (queue) __ibmvfc_get_event(queue, 1) + +void ibmvfc_init_event (struct ibmvfc_event *evt,void (*done) (struct ibmvfc_event *),u8 format); +void ibmvfc_free_event (struct ibmvfc_event *evt); +void ibmvfc_release_tgt (struct kref *kref); +int ibmvfc_send_event (struct ibmvfc_event *evt,struct ibmvfc_host *vhost,unsigned long timeout); +const char *ibmvfc_get_cmd_error (u16 status,u16 error); + +static inline int ibmvfc_check_caps (struct ibmvfc_host *vhost,unsigned long cap_flags) +{ +u64 host_caps =be64_to_cpu (vhost->login_buf->resp.capabilities); + +return (host_caps & cap_flags) ?1 :0 ; +} +static inline struct ibmvfc_host *ibmvfc_channels_to_vhost (struct ibmvfc_channels *channels) {if (channels->protocol ==IBMVFC_PROTO_SCSI )
Signature · 1 line
2.54.0
[email protected]
AI reviewRe: [PATCH 23/29] ibmvfc: declare global function definitions
Thank you for your contribution! Sashiko AI review found 3 potential issue(s) to consider:
Thank you for your contribution! Sashiko AI review found 3 potential issue(s) to consider:
Pre-existing issues:
- [High] Objects allocated via mempool_alloc are incorrectly freed using kfree, breaking mempool semantics and causing deadlock under memory pressure.
- [High] Non-synchronous timer cancellation during host removal leads to Use-After-Free on vhost in the timer callback.
- [High] Recursive deadlock in ibmvfc_send_event() error path when sending management (MAD format) events.
--
commit 433fbb97fc83e37f1339b6c6598956ee0f1ae0a5
Author: Tyrel Datwyler <[email protected]>
ibmvfc: declare global function definitions
Some common functions will require visibility by both SCSI and NVMe protocols. Make common ibmvfc helper routines available to the NVMe support code.
Quoted 4 lines
drivers/scsi/ibmvscsi/ibmvfc-core.c +0
-0
[ ... ]
Quoted 10 lines
drivers/scsi/ibmvscsi/ibmvfc-core.c +1
-1@@ -1410,7 +1403,7 @@ static void ibmvfc_set_rport_dev_loss_tmo(struct fc_rport *rport, u32 timeout)* @kref: kref struct * **/ -static void ibmvfc_release_tgt (struct kref *kref) +void ibmvfc_release_tgt (struct kref *kref) { struct ibmvfc_target *tgt = container_of(kref, struct ibmvfc_target, kref);kfree (tgt);}
[Severity: High]
This isn't a bug introduced by this patch, but does this leak reserved objects to the slab allocator and risk deadlock?
Since the target is allocated from tgt_pool (a mempool), shouldn't it be freed with mempool_free() rather than kfree()?
Under memory pressure, if mempool allocations fail, mempool_alloc() sleeps waiting for objects to return. Freeing directly to the slab bypasses the mempool and its waitqueue is never woken.
[ ... ]
drivers/scsi/ibmvscsi/ibmvfc-core.c static void ibmvfc_locked_done(struct ibmvfc_event *evt)
Quoted 1 line
drivers/scsi/ibmvscsi/ibmvfc-core.c +0
-0@@ -1639,7 +1629,7 @@ static void ibmvfc_locked_done(struct ibmvfc_event *evt)
[Severity: High]
This isn't a bug introduced by this patch, but does this code path contain a recursive deadlock when sending management (MAD format) events?
Management commands are often issued from a context that already holds vhost->host->host_lock. If ibmvfc_send_event() encounters a local error, it synchronously calls evt->done(evt):
ibmvfc_send_event() {
...
if (rc) {
...
evt->done(evt);
}
}For MAD format events, evt->done is often assigned to ibmvfc_locked_done(), which unconditionally attempts to lock the same spinlock again:
ibmvfc_locked_done() {
...
spin_lock_irqsave(evt->vhost->host->host_lock, flags);
...
}Will this cause a fatal recursive deadlock?
Quoted 5 lines
drivers/scsi/ibmvscsi/ibmvfc-core.c +1
-1* @done: Routine to call when the event is responded to * @format: SRP or MAD format **/ -static void ibmvfc_init_event(struct ibmvfc_event *evt, +void ibmvfc_init_event(struct ibmvfc_event *evt,
[ ... ]
drivers/scsi/ibmvscsi/ibmvfc-core.c static void ibmvfc_timeout(struct timer_list *t)
Quoted 1 line
drivers/scsi/ibmvscsi/ibmvfc-core.c +0
-0@@ -1764,7 +1754,7 @@ static void ibmvfc_timeout(struct timer_list *t)
[Severity: High]
This isn't a bug introduced by this patch, but can ibmvfc_timeout() execute concurrently and cause a use-after-free on vhost?
During teardown, ibmvfc_purge_requests() cancels each event's timer using the non-synchronous timer_delete() while holding host_lock.
If the timer callback ibmvfc_timeout() is concurrently firing on another CPU, it blocks waiting for that lock:
ibmvfc_timeout() {
...
ibmvfc_reset_host(vhost);
}ibmvfc_reset_host() then calls spin_lock_irqsave(vhost->host->host_lock, flags).
Once the teardown drops the lock and frees vhost, the timer callback acquires the lock and operates on the freed vhost object. Could this lead to a use-after-free?
Quoted 6 lines
drivers/scsi/ibmvscsi/ibmvfc-core.c +1
-1* * Returns the value returned from ibmvfc_send_crq(). (Zero for success) **/ -static int ibmvfc_send_event (struct ibmvfc_event *evt, +int ibmvfc_send_event (struct ibmvfc_event *evt,struct ibmvfc_host *vhost,unsigned long timeout)
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=23
Tyrel Datwyler
maintainerPATCH 24/29 ibmvfc: implement LLDD callbacks for mapping nvme-fc queues
In reply to Tyrel DatwylerImplement the NVMe-FC queue create and delete callbacks and map NVMe controller queues onto ibmvfc hardware queues.
Use qidx of NVMe controller queue to map onto a ibmvfc_queue channel. The Admin queue is always qidx 0 and general practice among other drivers is to map both the Admin queue and first IO queue to the same HW queue. Add a new ibmvfc_nvme_qhandle struct that will be used as the opaque queue handle by the NVMe-FC layer when issuing fcp IO.
--- drivers/scsi/ibmvscsi/ibmvfc-nvme.c | 38 +++++++++++++++++++++++++++-- drivers/scsi/ibmvscsi/ibmvfc-nvme.h | 7 ++++++ 2 files changed, 43 insertions(+), 2 deletions(-)
drivers/scsi/ibmvscsi/ibmvfc-nvme.c +36 -2
@@ -28,6 +28,40 @@ static void ibmvfc_nvme_remoteport_delete(struct nvme_fc_remote_port *rport)complete (&tgt->nvme_delete_done); } +static int ibmvfc_nvme_create_queue (struct nvme_fc_local_port *lport,unsigned int qidx, +u16 qsize,void **handle) +{ +struct ibmvfc_host *vhost = lport->private; +struct ibmvfc_nvme_qhandle *qhandle; + +if (!vhost->nvme_scrqs.active_queues) +return -ENODEV ; + + qhandle = kzalloc_obj(struct ibmvfc_nvme_qhandle); +if (!qhandle) +return -ENOMEM ; + + qhandle->cpu_id =raw_smp_processor_id (); + qhandle->qidx = qidx; + +/* Admin and first IO queue are both mapped to index 0 */ +if (qidx) + qhandle->index = (qidx -1 ) % vhost->nvme_scrqs.active_queues; +else + qhandle->index = qidx; + + qhandle->queue = &vhost->nvme_scrqs.scrqs[qhandle->index]; + + *handle = qhandle; +return 0 ; +} + +static void ibmvfc_nvme_delete_queue (struct nvme_fc_local_port *lport,unsigned int qidx, +void *handle) +{ +kfree (handle); +} +static int ibmvfc_nvme_ls_req (struct nvme_fc_local_port *lport,struct nvme_fc_remote_port *rport,struct nvmefc_ls_req *ls_req)
@@ -59,8 +93,8 @@ static void ibmvfc_nvme_fcp_abort(struct nvme_fc_local_port *lport,static struct nvme_fc_port_template ibmvfc_nvme_fc_transport = { .localport_delete = ibmvfc_nvme_localport_delete, .remoteport_delete = ibmvfc_nvme_remoteport_delete, - .create_queue =NULL , - .delete_queue =NULL , + .create_queue = ibmvfc_nvme_create_queue, + .delete_queue = ibmvfc_nvme_delete_queue, .ls_req = ibmvfc_nvme_ls_req, .ls_abort = ibmvfc_nvme_ls_abort, .fcp_io = ibmvfc_nvme_fcp_io,
drivers/scsi/ibmvscsi/ibmvfc-nvme.h +7 -0
@@ -27,6 +27,13 @@ extern unsigned int ibmvfc_debug;struct ibmvfc_host ;struct ibmvfc_target ; +struct ibmvfc_nvme_qhandle { +unsigned int qidx; +u16 cpu_id; +unsigned long index; +struct ibmvfc_queue *queue; +}; +int ibmvfc_nvme_register_remoteport (struct ibmvfc_target *tgt);void ibmvfc_nvme_unregister_remoteport (struct ibmvfc_target *tgt);int ibmvfc_nvme_register (struct ibmvfc_host *vhost);
Signature · 1 line
2.54.0
[email protected]
AI reviewRe: [PATCH 24/29] ibmvfc: implement LLDD callbacks for mapping nvme-fc queues
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [High] Lockless double-read of `vhost->nvme_scrqs.active_queues` can cause a division-by-zero kernel…
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [High] Lockless double-read of vhost->nvme_scrqs.active_queues can cause a division-by-zero kernel panic. --
commit fd68a3e765846c041d31997c784f4e4a733bb8aa
Author: Tyrel Datwyler <[email protected]>
ibmvfc: implement LLDD callbacks for mapping nvme-fc queues
This commit implements the NVMe-FC queue create and delete callbacks. It maps NVMe controller queues onto ibmvfc hardware queues using the qidx.
Quoted 4 lines
drivers/scsi/ibmvscsi/ibmvfc-nvme.c +0
-0
[ ... ]
Quoted 13 lines
drivers/scsi/ibmvscsi/ibmvfc-nvme.c +9
-0@@ -28,6 +28,40 @@ static void ibmvfc_nvme_remoteport_delete(struct nvme_fc_remote_port *rport)complete (&tgt->nvme_delete_done); } +static int ibmvfc_nvme_create_queue (struct nvme_fc_local_port *lport,unsigned int qidx, +u16 qsize,void **handle) +{ +struct ibmvfc_host *vhost = lport->private; +struct ibmvfc_nvme_qhandle *qhandle; + +if (!vhost->nvme_scrqs.active_queues) +return -ENODEV ; +
drivers/scsi/ibmvscsi/ibmvfc-nvme.c static void ibmvfc_nvme_remoteport_delete(struct nvme_fc_remote_port *rport) · line 49
Quoted 10 lines
drivers/scsi/ibmvscsi/ibmvfc-nvme.c +10
-0+ qhandle = kzalloc_obj(struct ibmvfc_nvme_qhandle); +if (!qhandle) +return -ENOMEM ; + + qhandle->cpu_id =raw_smp_processor_id (); + qhandle->qidx = qidx; + +/* Admin and first IO queue are both mapped to index 0 */ +if (qidx) + qhandle->index = (qidx -1 ) % vhost->nvme_scrqs.active_queues;
[Severity: High]
Could a lockless double-read of vhost->nvme_scrqs.active_queues lead to a division by zero?
If a concurrent host reset or channel setup failure sets active_queues to 0, it seems the compiler could reload the value for the modulo operation since the memory allocation acts as a compiler barrier.
Would reading active_queues once into a local variable prevent a potential kernel panic here?
Quoted 8 lines
drivers/scsi/ibmvscsi/ibmvfc-nvme.c +8
-0+else + qhandle->index = qidx; + + qhandle->queue = &vhost->nvme_scrqs.scrqs[qhandle->index]; + + *handle = qhandle; +return 0 ; +}
[ ... ]
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=24
Tyrel Datwyler
maintainerPATCH 25/29 ibmvfc: implement nvme-fc LS submission transport callback
In reply to Tyrel DatwylerNVMe FC Link Service commands are required to use the ibmvfc_passthru MAD. Initialize a pssthru mad for the target port including the DMA addresses for the FC4_LS request and response as well as the max length of each IU as provided in the nvmefc_ls_req struct. FC4_LS commands are sent via the primary CRQ. Further, store the assoc_id during a create association request as this is a required field in our vfc_cmd struct for nvme_fcp_io commands.
--- drivers/scsi/ibmvscsi/ibmvfc-nvme.c | 89 +++++++++++++++++++++++++++++ drivers/scsi/ibmvscsi/ibmvfc-nvme.h | 8 +++ drivers/scsi/ibmvscsi/ibmvfc.h | 2 + 3 files changed, 99 insertions(+)
drivers/scsi/ibmvscsi/ibmvfc-nvme.c +89 -0
@@ -62,10 +62,99 @@ static void ibmvfc_nvme_delete_queue(struct nvme_fc_local_port *lport, unsignedkfree (handle); } +static void ibmvfc_ls_req_done (struct ibmvfc_event *evt) +{ +struct ibmvfc_target *tgt = evt->tgt; +struct ibmvfc_passthru_mad *mad = &evt->xfer_iu->passthru; +struct fcnvme_ls_rqst_w0 *ls_rqst; +struct fcnvme_ls_cr_assoc_acc *ls_resp; +u32 status =be16_to_cpu (mad->common.status); +int rc =0 ; + + ls_rqst = (struct fcnvme_ls_rqst_w0 *)evt->ls_req->rqstaddr; + ls_resp = (struct fcnvme_ls_cr_assoc_acc *)evt->ls_req->rspaddr; + +switch (status) { +case IBMVFC_MAD_SUCCESS : +tgt_dbg (tgt,"ls_req succeeded\n" ); +if ((ls_rqst->ls_cmd ==FCNVME_LS_CREATE_ASSOCIATION ) && + (ls_resp->hdr.w0.ls_cmd ==FCNVME_LS_ACC )) { + tgt->assoc_id =be64_to_cpu (ls_resp->associd.association_id); +tgt_dbg (tgt,"assoc_id 0x%llx\n" , tgt->assoc_id); + } +break ; +case IBMVFC_MAD_DRIVER_FAILED : +break ; +case IBMVFC_MAD_FAILED : +default : +tgt_info (tgt,"ls_req failed: %s (%x:%x) rc=0x%02X\n" , +ibmvfc_get_cmd_error (be16_to_cpu (mad->iu.status),be16_to_cpu (mad->iu.error)), +be16_to_cpu (mad->iu.status),be16_to_cpu (mad->iu.error), status); +break ; + } + +if (status) + rc = -EIO ; + + evt->ls_req->done (evt->ls_req, rc); + +kref_put (&tgt->kref, ibmvfc_release_tgt); +ibmvfc_free_event (evt); +} + +static void ibmvfc_init_ls_req (struct ibmvfc_event *evt,struct nvmefc_ls_req *ls_req) +{ +struct ibmvfc_passthru_mad *mad = &evt->iu.passthru; + +memset (mad,0 ,sizeof (*mad)); + mad->common.version =cpu_to_be32 (2 ); + mad->common.opcode =cpu_to_be32 (IBMVFC_NVMF_PASSTHRU ); + mad->common.length =cpu_to_be16 (sizeof (*mad) -sizeof (mad->fc_iu) -sizeof (mad->iu)); + mad->cmd_ioba.va =cpu_to_be64 ((u64 )be64_to_cpu (evt->crq.ioba) + + offsetof(struct ibmvfc_passthru_mad , iu)); + mad->cmd_ioba.len =cpu_to_be32 (sizeof (mad->iu)); + mad->iu.cmd_len =cpu_to_be32 (ls_req->rqstlen); + mad->iu.rsp_len =cpu_to_be32 (ls_req->rsplen); + mad->iu.cmd.va =cpu_to_be64 (ls_req->rqstdma); + mad->iu.cmd.len =cpu_to_be32 (ls_req->rqstlen); + mad->iu.rsp.va =cpu_to_be64 (ls_req->rspdma); + mad->iu.rsp.len =cpu_to_be32 (ls_req->rsplen); +} +static int ibmvfc_nvme_ls_req (struct nvme_fc_local_port *lport,struct nvme_fc_remote_port *rport,struct nvmefc_ls_req *ls_req) { +struct ibmvfc_host *vhost = lport->private; +struct ibmvfc_target *tgt = rport->private; +struct ibmvfc_passthru_mad *mad; +struct ibmvfc_event *evt; + +kref_get (&tgt->kref); + evt =ibmvfc_get_event (&vhost->crq); +if (!evt) { +kref_put (&tgt->kref, ibmvfc_release_tgt); +return -EBUSY ; + } + +ibmvfc_init_event (evt, ibmvfc_ls_req_done,IBMVFC_MAD_FORMAT ); + evt->tgt = tgt; + evt->ls_req = ls_req; + ls_req->private = evt; + +ibmvfc_init_ls_req (evt, ls_req); + mad = &evt->iu.passthru; + mad->iu.flags =cpu_to_be32 (IBMVFC_FC4_LS_DSC_CTRL ); + mad->iu.scsi_id =cpu_to_be64 (tgt->scsi_id); + mad->iu.cancel_key =cpu_to_be32 ((u64 )evt); + mad->iu.target_wwpn =cpu_to_be64 (tgt->wwpn); + +ibmvfc_dbg (vhost,"nvme_ls_req\n" ); +if (ibmvfc_send_event (evt, vhost,IBMVFC_FC4_LS_PLUS_CANCEL_TIMEOUT )) { +kref_put (&tgt->kref, ibmvfc_release_tgt); +return -ENXIO ; + } +return 0 ; }
drivers/scsi/ibmvscsi/ibmvfc-nvme.h +8 -0
@@ -14,6 +14,8 @@#include <scsi/fc/fc_fs.h> #include <scsi/fc/fc_els.h> #include <linux/nvme-fc-driver.h> +#include <linux/nvme.h> +#include <linux/nvme-fc.h> #include "ibmvfc.h"
@@ -22,10 +24,16 @@#define IBMVFC_MAX_NVME_QUEUES 16#define IBMVFC_NVME_CHANNELS 8 +#define IBMVFC_FC4_LS_TIMEOUT 15 +#define IBMVFC_FC4_LS_CANCEL_TIMEOUT 45 +#define IBMVFC_FC4_LS_PLUS_CANCEL_TIMEOUT \ + (IBMVFC_FC4_LS_TIMEOUT + IBMVFC_FC4_LS_CANCEL_TIMEOUT) +extern unsigned int ibmvfc_debug;struct ibmvfc_host ;struct ibmvfc_target ; +struct ibmvfc_queue ;struct ibmvfc_nvme_qhandle {unsigned int qidx;
drivers/scsi/ibmvscsi/ibmvfc.h +2 -0
@@ -824,6 +824,7 @@ struct ibmvfc_target {u64 scsi_id;u64 wwpn;u64 new_scsi_id; +u64 assoc_id;struct fc_rport *rport;int target_id;enum ibmvfc_target_action action;
@@ -851,6 +852,7 @@ struct ibmvfc_event {struct ibmvfc_queue *queue;struct ibmvfc_target *tgt;struct scsi_cmnd *cmnd; +struct nvmefc_ls_req *ls_req;atomic_t free;atomic_t active;union ibmvfc_iu *xfer_iu;
Signature · 1 line
2.54.0
[email protected]
AI reviewRe: [PATCH 25/29] ibmvfc: implement nvme-fc LS submission transport callback
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [High] The driver accesses NVMe-FC response payloads (`evt->ls_req->rspaddr` and `fcp_req->rspaddr`) from…
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [High] The driver accesses NVMe-FC response payloads (evt->ls_req->rspaddr and fcp_req->rspaddr) from CPU memory while they are still mapped for DMA, before the NVMe-FC core unmaps or syncs them. On architectures using SWIOTLB bounce buffers (like PowerPC Secure Execution), this causes the driver to read stale data and its manual payload writes to be clobbered by the subsequent DMA sync. --
commit 1915a13751880ed8af4917b3092d7cf5b63c79eb
Author: Tyrel Datwyler <[email protected]>
ibmvfc: implement nvme-fc LS submission transport callback
NVMe FC Link Service commands are required to use the ibmvfc_passthru MAD. Initialize a pssthru mad for the target port including the DMA addresses for the FC4_LS request and response as well as the max length of each IU as provided in the nvmefc_ls_req struct. FC4_LS commands are sent via the primary CRQ. Further, store the assoc_id during a create association request as this is a required field in our vfc_cmd struct for nvme_fcp_io commands.
Quoted 4 lines
drivers/scsi/ibmvscsi/ibmvfc-nvme.c +0
-0
[ ... ]
Quoted 12 lines
drivers/scsi/ibmvscsi/ibmvfc-nvme.c +8
-0@@ -62,10 +62,99 @@ static void ibmvfc_nvme_delete_queue(struct nvme_fc_local_port *lport, unsignedkfree (handle); } +static void ibmvfc_ls_req_done (struct ibmvfc_event *evt) +{ +struct ibmvfc_target *tgt = evt->tgt; +struct ibmvfc_passthru_mad *mad = &evt->xfer_iu->passthru; +struct fcnvme_ls_rqst_w0 *ls_rqst; +struct fcnvme_ls_cr_assoc_acc *ls_resp; +u32 status =be16_to_cpu (mad->common.status); +int rc =0 ;
drivers/scsi/ibmvscsi/ibmvfc-nvme.c static void ibmvfc_nvme_delete_queue(struct nvme_fc_local_port *lport, unsigned · line 82
Quoted 10 lines
drivers/scsi/ibmvscsi/ibmvfc-nvme.c +10
-0+ + ls_rqst = (struct fcnvme_ls_rqst_w0 *)evt->ls_req->rqstaddr; + ls_resp = (struct fcnvme_ls_cr_assoc_acc *)evt->ls_req->rspaddr; + +switch (status) { +case IBMVFC_MAD_SUCCESS : +tgt_dbg (tgt,"ls_req succeeded\n" ); +if ((ls_rqst->ls_cmd ==FCNVME_LS_CREATE_ASSOCIATION ) && + (ls_resp->hdr.w0.ls_cmd ==FCNVME_LS_ACC )) { + tgt->assoc_id =be64_to_cpu (ls_resp->associd.association_id);
[Severity: High]
Does this code read stale data from the response buffer when bounce buffers are active?
Since the response payload at evt->ls_req->rspaddr is mapped for DMA, and the NVMe-FC core hasn't unmapped or synced it yet, reading the association_id here without a prior dma_sync_single_for_cpu() could result in reading stale memory on architectures using SWIOTLB bounce buffers (like PowerPC Secure Execution).
Additionally, does a similar issue occur in ibmvfc_nvme_done()? If the driver reads ersp->xfrd_len from rspaddr and writes to cqe->command_id before calling fcp_req->done(), the subsequent DMA sync inside fcp_req->done() could overwrite the manually written command_id with unmodified bounce buffer memory, causing the I/O to fail.
Quoted 5 lines
drivers/scsi/ibmvscsi/ibmvfc-nvme.c +5
-0+tgt_dbg (tgt,"assoc_id 0x%llx\n" , tgt->assoc_id); + } +break ; +case IBMVFC_MAD_DRIVER_FAILED : +break ;
[ ... ]
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=25
Tyrel Datwyler
maintainerPATCH 26/29 ibmvfc: implement nvme-fc IO command submission callback
In reply to Tyrel DatwylerAdd helpers to initialize an ibmvfc command from an nvmefc_fcp_req, map request scatterlists into either an inline descriptor or an external DMA pool list, and submit the request on the selected NVMe hardware queue. On completion, translate ibmvfc status into the NVMe-FC response format, including transferred length and CQE handling for no-DMA responses.
Also store the NVMe request pointer in struct ibmvfc_event so the completion path can finish the original request.
--- drivers/scsi/ibmvscsi/ibmvfc-core.c | 2 +- drivers/scsi/ibmvscsi/ibmvfc-nvme.c | 140 +++++++++++++++++++++++++++- drivers/scsi/ibmvscsi/ibmvfc.h | 1 + 3 files changed, 141 insertions(+), 2 deletions(-)
drivers/scsi/ibmvscsi/ibmvfc-core.c +1 -1
@@ -1539,7 +1539,7 @@ static void ibmvfc_set_login_info(struct ibmvfc_host *vhost) login_info->ostype =cpu_to_be32 (IBMVFC_OS_LINUX ); login_info->max_dma_len =cpu_to_be64 (max_sectors <<9 ); - login_info->max_payload =cpu_to_be32 (sizeof (struct ibmvfc_fcp_cmd_iu )); + login_info->max_payload =cpu_to_be32 (sizeof (struct nvme_fc_cmd_iu )); login_info->max_response =cpu_to_be32 (sizeof (struct ibmvfc_fcp_rsp )); login_info->partition_num =cpu_to_be32 (vhost->partition_number); login_info->vfc_frame_version =cpu_to_be32 (1 );
drivers/scsi/ibmvscsi/ibmvfc-nvme.c +139 -1
157 lines not on this page · raw message
drivers/scsi/ibmvscsi/ibmvfc.h +1 -0
@@ -853,6 +853,7 @@ struct ibmvfc_event {struct ibmvfc_target *tgt;struct scsi_cmnd *cmnd;struct nvmefc_ls_req *ls_req; +struct nvmefc_fcp_req *fcp_req;atomic_t free;atomic_t active;union ibmvfc_iu *xfer_iu;
Signature · 1 line
2.54.0
[email protected]
AI reviewRe: [PATCH 26/29] ibmvfc: implement nvme-fc IO command submission callback
Thank you for your contribution! Sashiko AI review found 4 potential issue(s) to consider: - [High] Buffer overflow in NVMe scatterlist mapping when sg_cnt exceeds SG_ALL. - [High] Returning…
Thank you for your contribution! Sashiko AI review found 4 potential issue(s) to consider:
- [High] Buffer overflow in NVMe scatterlist mapping when sg_cnt exceeds SG_ALL.
- [High] Returning SCSI-specific or bare memory allocation errors from fcp_io causes the NVMe-FC core to permanently fail retriable commands.
- [High] Overwriting fcp_req->private causes an invalid kfree of an interior pool pointer and a memory leak.
- [High] Stale data is read from the ERSP buffer when the target responds with IBMVFC_NVMS_VALID_NODMA_CQE.
--
commit 6b37a58cc483eca7d86fda91190fd35dfdfe359a
Author: Tyrel Datwyler <[email protected]>
ibmvfc: implement nvme-fc IO command submission callback
This commit adds helpers to initialize an ibmvfc command from an nvmefc_fcp_req. It handles scatterlist mapping, submits the request on the NVMe hardware queue, and handles command completion translation.
Quoted 4 lines
drivers/scsi/ibmvscsi/ibmvfc-nvme.c +0
-0
[ ... ]
Quoted 16 lines
drivers/scsi/ibmvscsi/ibmvfc-nvme.c +12
-0@@ -164,12 +165,149 @@ static void ibmvfc_nvme_ls_abort(struct nvme_fc_local_port *lport, { } +static void ibmvfc_nvme_done (struct ibmvfc_event *evt) +{ +struct ibmvfc_cmd *vfc_cmd = &evt->xfer_iu->cmd; +struct nvmefc_fcp_req *fcp_req = evt->fcp_req; +struct nvme_fc_ersp_iu *ersp = (struct nvme_fc_ersp_iu *)fcp_req->rspaddr; +struct nvme_completion *cqe = &ersp->cqe; +struct nvme_command *sqe = &((struct nvme_fc_cmd_iu *)fcp_req->cmdaddr)->sqe; + +ibmvfc_dbg (evt->vhost,"fc_done: (%x:%x)\n" ,be16_to_cpu (vfc_cmd->status), +be16_to_cpu (vfc_cmd->error)); +ibmvfc_dbg (evt->vhost,"fc_done: cmdlen: %d, rsplen %d, payload_len %d\n" , + fcp_req->cmdlen, fcp_req->rsplen, fcp_req->payload_length);
drivers/scsi/ibmvscsi/ibmvfc-nvme.c static void ibmvfc_nvme_ls_abort(struct nvme_fc_local_port *lport, · line 189
Quoted 10 lines
drivers/scsi/ibmvscsi/ibmvfc-nvme.c +10
-0+ + fcp_req->status =0 ; +if (!vfc_cmd->status) { + fcp_req->rcv_rsplen =NVME_FC_SIZEOF_ZEROS_RSP ; + fcp_req->transferred_length = fcp_req->payload_length; + }else if (be16_to_cpu (vfc_cmd->status) &IBMVFC_FC_NVME_STATUS ) { + fcp_req->rcv_rsplen =sizeof (struct nvme_fc_ersp_iu ); + fcp_req->transferred_length =be32_to_cpu (ersp->xfrd_len); +if (be16_to_cpu (vfc_cmd->error) &IBMVFC_NVMS_VALID_NODMA_CQE ) + cqe->command_id = sqe->common.command_id;
[Severity: High]
Does this code read stale data from the ERSP buffer when the target responds with IBMVFC_NVMS_VALID_NODMA_CQE?
If the completion queue entry was not DMA'd into the ersp buffer, ersp->xfrd_len, cqe->status, and cqe->result are not initialized, but this path proceeds to use be32_to_cpu(ersp->xfrd_len) to set the transferred length.
Since the NVMe-FC core reuses the request structure without zeroing the rspaddr buffer between commands, this can cause the driver to read stale data from previous commands, resulting in data corruption and validation failures.
Quoted 9 lines
drivers/scsi/ibmvscsi/ibmvfc-nvme.c +9
-0+ }else { + fcp_req->rcv_rsplen =0 ; + fcp_req->transferred_length =0 ; + fcp_req->status =NVME_SC_INTERNAL ; + } + + fcp_req->done (fcp_req); +ibmvfc_free_event (evt); +}
[ ... ]
Quoted 23 lines
drivers/scsi/ibmvscsi/ibmvfc-nvme.c +23
-0+static int ibmvfc_nvme_map_sg_data (struct nvmefc_fcp_req *fcp_req, +struct ibmvfc_event *evt, +struct ibmvfc_cmd *vfc_cmd) +{ +struct srp_direct_buf *data = &vfc_cmd->ioba; +struct ibmvfc_host *vhost = evt->vhost; + +if (!fcp_req->sg_cnt) { + vfc_cmd->flags |=cpu_to_be16 (IBMVFC_NO_MEM_DESC ); +return 0 ; + } + +if (fcp_req->io_dir ==NVMEFC_FCP_WRITE ) + vfc_cmd->flags |=cpu_to_be16 (IBMVFC_WRITE ); +else + vfc_cmd->flags |=cpu_to_be16 (IBMVFC_READ ); + +if (fcp_req->sg_cnt ==1 ) { +ibmvfc_nvme_map_sg_list (fcp_req, data); +return 0 ; + } + + vfc_cmd->flags |=cpu_to_be16 (IBMVFC_SCATTERLIST );
drivers/scsi/ibmvscsi/ibmvfc-nvme.c static void ibmvfc_nvme_ls_abort(struct nvme_fc_local_port *lport, · line 270
Quoted 10 lines
drivers/scsi/ibmvscsi/ibmvfc-nvme.c +10
-0+ +if (!evt->ext_list) { + evt->ext_list =dma_pool_alloc (vhost->sg_pool,GFP_ATOMIC , + &evt->ext_list_token); + +if (!evt->ext_list) +return -ENOMEM ; + } + +ibmvfc_nvme_map_sg_list (fcp_req, evt->ext_list);
[Severity: High]
Can this cause a buffer overflow when fcp_req->sg_cnt exceeds SG_ALL?
The NVMe-FC transport template sets max_sgl_segments to 1024, allowing requests with up to 1024 scatterlist elements.
However, the external list is allocated from vhost->sg_pool, which is sized at SG_ALL (128) elements. If fcp_req->sg_cnt is greater than 128, ibmvfc_nvme_map_sg_list() will iterate up to sg_cnt and write out of bounds, corrupting adjacent DMA pool memory.
Quoted 20 lines
drivers/scsi/ibmvscsi/ibmvfc-nvme.c +14
-1+ + data->va =cpu_to_be64 (evt->ext_list_token); + data->len =cpu_to_be32 (fcp_req->sg_cnt *sizeof (struct srp_direct_buf )); + data->key =0 ; +return 0 ; +} +static int ibmvfc_nvme_fcp_io (struct nvme_fc_local_port *lport,struct nvme_fc_remote_port *rport,void *hw_queue_handle,struct nvmefc_fcp_req *fcp_req) { -return 0 ; +struct ibmvfc_host *vhost = lport->private; +struct ibmvfc_nvme_qhandle *qhandle = hw_queue_handle; +struct ibmvfc_cmd *vfc_cmd; +struct ibmvfc_event *evt; +int rc; + +ibmvfc_dbg (vhost,"nvme_fcp_io\n" );
drivers/scsi/ibmvscsi/ibmvfc-nvme.c static void ibmvfc_nvme_ls_abort(struct nvme_fc_local_port *lport, · line 299
Quoted 10 lines
drivers/scsi/ibmvscsi/ibmvfc-nvme.c +10
-0+ evt =ibmvfc_get_event (qhandle->queue); +if (!evt) +return -EBUSY ; + + evt->hwq = qhandle->index; +ibmvfc_dbg (vhost,"vfc-nvme-mq-%d\n" , evt->hwq); + +ibmvfc_init_event (evt, ibmvfc_nvme_done,IBMVFC_CMD_FORMAT ); + evt->fcp_req = fcp_req; + fcp_req->private = evt;
[Severity: High]
Could overwriting fcp_req->private cause an invalid kfree and memory leak?
The NVMe-FC core allocates the private area for the driver (specified by fcprqst_priv_sz) and stores the pointer in fcp_req->private. For Asynchronous Event Notifications (AENs), this memory is allocated via kzalloc.
By overwriting the private pointer with evt here, during controller resets or teardown, the NVMe-FC core will call kfree(aen_op->fcp_req.private). Because the pointer now points to evt (which resides within the contiguous pool->events array), this results in freeing an interior pointer, triggering a kernel bug.
drivers/scsi/ibmvscsi/ibmvfc-nvme.c static void ibmvfc_nvme_ls_abort(struct nvme_fc_local_port *lport, · line 310
Quoted 11 lines
drivers/scsi/ibmvscsi/ibmvfc-nvme.c +11
-0+ + vfc_cmd =ibmvfc_nvme_init_vfc_cmd (evt, rport, fcp_req); + + vfc_cmd->correlation =cpu_to_be64 ((u64 )evt); + +if (likely (!(rc =ibmvfc_nvme_map_sg_data (fcp_req, evt, vfc_cmd)))) +return ibmvfc_send_event (evt, vhost,0 ); + +ibmvfc_free_event (evt); + +return rc;
[Severity: High]
Does returning rc directly bypass the expected -EBUSY retry mechanism?
The ibmvfc_nvme_fcp_io callback can return -ENOMEM from map_sg_data or SCSI_MLQUEUE_HOST_BUSY (0x1055) from ibmvfc_send_event on temporary allocation or queue closure failures.
The NVMe-FC core expects fcp_io to return exactly -EBUSY to trigger a requeue (BLK_STS_RESOURCE). For any other non-zero return value, nvme_fc_start_fcp_op returns BLK_STS_IOERR, translating transient hardware constraints into permanent I/O errors and data loss for the application.
Quoted 1 line
}
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=26
Tyrel Datwyler
maintainerPATCH 27/29 ibmvfc: implement nvme-fc LS abort handling callback
In reply to Tyrel DatwylerImplement the NVMe FC-LS abort callback by issuing an ibmvfc cancel MAD to the VIOS for the outstanding link-service request.
Use the saved event pointer from the original FC-LS request to identify the command to cancel, submit the cancel operation, and complete the abort request based on the returned status.
--- drivers/scsi/ibmvscsi/ibmvfc-nvme.c | 64 +++++++++++++++++++++++++++++ 1 file changed, 64 insertions(+)
drivers/scsi/ibmvscsi/ibmvfc-nvme.c +64 -0
@@ -13,6 +13,8 @@#include "ibmvfc-nvme.h" +static unsigned int default_timeout =IBMVFC_DEFAULT_TIMEOUT ; +static void ibmvfc_nvme_localport_delete (struct nvme_fc_local_port *lport) {struct ibmvfc_host *vhost = lport->private;
@@ -159,10 +161,72 @@ static int ibmvfc_nvme_ls_req(struct nvme_fc_local_port *lport,return 0 ; } +static void ibmvfc_sync_nvme_completion (struct ibmvfc_event *evt) +{ +/* copy the response back */ +if (evt->sync_iu) + *evt->sync_iu = *evt->xfer_iu; + +complete (&evt->comp); +} + +static void ibmvfc_init_ls_abort (struct ibmvfc_event *evt,struct nvmefc_ls_req *ls_abort) +{ +struct ibmvfc_tmf *tmf; +struct ibmvfc_event *abt_evt = ls_abort->private; +struct ibmvfc_target *tgt = abt_evt->tgt; +struct ibmvfc_host *vhost = evt->vhost; + + tmf = &evt->iu.tmf; +memset (tmf,0 ,sizeof (*tmf)); + tmf->common.version =cpu_to_be32 (2 ); + tmf->target_wwpn =cpu_to_be64 (tgt->wwpn); + tmf->common.opcode =cpu_to_be32 (IBMVFC_NVMF_TMF_MAD ); + tmf->common.length =cpu_to_be16 (sizeof (*tmf)); +if (vhost->state !=IBMVFC_ACTIVE ) +if (!ibmvfc_check_caps (vhost,IBMVFC_CAN_SUPPRESS_ABTS )) + tmf->flags =cpu_to_be32 (IBMVFC_TMF_SUPPRESS_ABTS ); + tmf->cancel_key =cpu_to_be32 ((u64 )abt_evt); + tmf->my_cancel_key =cpu_to_be32 ((u64 )evt); + tmf->assoc_id =cpu_to_be64 (tgt->assoc_id); + +init_completion (&evt->comp); +} +static void ibmvfc_nvme_ls_abort (struct nvme_fc_local_port *lport,struct nvme_fc_remote_port *rport,struct nvmefc_ls_req *ls_abort) { +struct ibmvfc_host *vhost = lport->private; +struct ibmvfc_target *tgt = rport->private; +struct ibmvfc_event *evt; +union ibmvfc_iu rsp; +unsigned long flags; +u16 status; + + evt =ibmvfc_get_event (&vhost->crq); +if (!vhost->logged_in || !evt) +return ; + +spin_lock_irqsave (vhost->host->host_lock, flags); +kref_get (&tgt->kref); +ibmvfc_init_event (evt, ibmvfc_sync_nvme_completion,IBMVFC_MAD_FORMAT ); +ibmvfc_init_ls_abort (evt, ls_abort); + evt->sync_iu = &rsp; + +if (ibmvfc_send_event (evt, vhost, default_timeout)) + goto out; + +spin_unlock_irqrestore (vhost->host->host_lock, flags); + +wait_for_completion (&evt->comp); + status =be16_to_cpu (rsp.mad_common.status); +spin_lock_irqsave (vhost->host->host_lock, flags); +ibmvfc_free_event (evt); +out: +spin_unlock_irqrestore (vhost->host->host_lock, flags); +ibmvfc_dbg (vhost,"ls_abort: cancel failed with rc=%x\n" , status); +kref_put (&tgt->kref, ibmvfc_release_tgt); }static void ibmvfc_nvme_done (struct ibmvfc_event *evt)
Signature · 1 line
2.54.0
[email protected]
AI reviewRe: [PATCH 27/29] ibmvfc: implement nvme-fc LS abort handling callback
Thank you for your contribution! Sashiko AI review found 7 potential issue(s) to consider: - [High] The allocated ibmvfc_event structure is leaked on early return if the virtual host is not logged…
Thank you for your contribution! Sashiko AI review found 7 potential issue(s) to consider:
- [High] The allocated ibmvfc_event structure is leaked on early return if the virtual host is not logged in.
- [High] Recursive spinlock self-deadlock or Use-After-Free on ibmvfc_send_event() failure.
- [High] Unsynchronized access to a potentially freed event structure (abt_evt) via ls_abort->private leads to a Use-After-Free.
- [High] Event leak when ibmvfc_send_event() fails, bypassing the freeing logic.
- [Medium] Logic inversion defeats the IBMVFC_CAN_SUPPRESS_ABTS capability check.
- [Low] Uninitialized stack variable status is read and printed on the error path.
- [Low] Returning a SCSI midlayer error code (SCSI_MLQUEUE_HOST_BUSY) to the NVMe FC transport causes unintended API violations. --
commit e8f5bb9c65a6243c394bbc91b4a31f61c71ee322
Author: Tyrel Datwyler <[email protected]>
ibmvfc: implement nvme-fc LS abort handling callback
Implement the NVMe FC-LS abort callback by issuing an ibmvfc cancel MAD to the VIOS for the outstanding link-service request.
Use the saved event pointer from the original FC-LS request to identify the command to cancel, submit the cancel operation, and complete the abort request based on the returned status.
Quoted 4 lines
drivers/scsi/ibmvscsi/ibmvfc-nvme.c +0
-0
[ ... ]
Quoted 1 line
drivers/scsi/ibmvscsi/ibmvfc-nvme.c +0
-0@@ -159,10 +161,72 @@ static int ibmvfc_nvme_ls_req(struct nvme_fc_local_port *lport,
[ ... ]
drivers/scsi/ibmvscsi/ibmvfc-nvme.c static int ibmvfc_nvme_ls_req(struct nvme_fc_local_port *lport, · line 177
Quoted 5 lines
drivers/scsi/ibmvscsi/ibmvfc-nvme.c +5
-0+static void ibmvfc_init_ls_abort (struct ibmvfc_event *evt,struct nvmefc_ls_req *ls_abort) +{ +struct ibmvfc_tmf *tmf; +struct ibmvfc_event *abt_evt = ls_abort->private; +struct ibmvfc_target *tgt = abt_evt->tgt;
[Severity: High]
Can this sequence actually occur? If the NVMe FC transport invokes the abort callback concurrently with the normal completion of the LS request, could ibmvfc_ls_req_done() complete the request and free abt_evt to the pool before we access abt_evt->tgt here?
drivers/scsi/ibmvscsi/ibmvfc-nvme.c static int ibmvfc_nvme_ls_req(struct nvme_fc_local_port *lport, · line 188
Quoted 11 lines
drivers/scsi/ibmvscsi/ibmvfc-nvme.c +11
-0+struct ibmvfc_host *vhost = evt->vhost; + + tmf = &evt->iu.tmf; +memset (tmf,0 ,sizeof (*tmf)); + tmf->common.version =cpu_to_be32 (2 ); + tmf->target_wwpn =cpu_to_be64 (tgt->wwpn); + tmf->common.opcode =cpu_to_be32 (IBMVFC_NVMF_TMF_MAD ); + tmf->common.length =cpu_to_be16 (sizeof (*tmf)); +if (vhost->state !=IBMVFC_ACTIVE ) +if (!ibmvfc_check_caps (vhost,IBMVFC_CAN_SUPPRESS_ABTS )) + tmf->flags =cpu_to_be32 (IBMVFC_TMF_SUPPRESS_ABTS );
[Severity: Medium]
Does this logic correctly evaluate the capability? The check evaluates to true only when the adapter does not support suppressing ABTS, which appears to set IBMVFC_TMF_SUPPRESS_ABTS on adapters that do not support it, and omits it on adapters that do.
Quoted 11 lines
drivers/scsi/ibmvscsi/ibmvfc-nvme.c +7
-0+ tmf->cancel_key =cpu_to_be32 ((u64 )abt_evt); + tmf->my_cancel_key =cpu_to_be32 ((u64 )evt); + tmf->assoc_id =cpu_to_be64 (tgt->assoc_id); + +init_completion (&evt->comp); +} +static void ibmvfc_nvme_ls_abort (struct nvme_fc_local_port *lport,struct nvme_fc_remote_port *rport,struct nvmefc_ls_req *ls_abort) {
drivers/scsi/ibmvscsi/ibmvfc-nvme.c static int ibmvfc_nvme_ls_req(struct nvme_fc_local_port *lport, · line 209
Quoted 10 lines
drivers/scsi/ibmvscsi/ibmvfc-nvme.c +10
-0+struct ibmvfc_host *vhost = lport->private; +struct ibmvfc_target *tgt = rport->private; +struct ibmvfc_event *evt; +union ibmvfc_iu rsp; +unsigned long flags; +u16 status; + + evt =ibmvfc_get_event (&vhost->crq); +if (!vhost->logged_in || !evt) +return ;
[Severity: High]
Does this code leak the event structure? If vhost->logged_in is false, ibmvfc_get_event() might successfully allocate an event from the vhost->crq pool, but returning immediately here leaves the allocated event unreleased. Repeated abort attempts could permanently exhaust the event pool.
drivers/scsi/ibmvscsi/ibmvfc-nvme.c static int ibmvfc_nvme_ls_req(struct nvme_fc_local_port *lport, · line 217
Quoted 8 lines
drivers/scsi/ibmvscsi/ibmvfc-nvme.c +8
-0+ +spin_lock_irqsave (vhost->host->host_lock, flags); +kref_get (&tgt->kref); +ibmvfc_init_event (evt, ibmvfc_sync_nvme_completion,IBMVFC_MAD_FORMAT ); +ibmvfc_init_ls_abort (evt, ls_abort); + evt->sync_iu = &rsp; + +if (ibmvfc_send_event (evt, vhost, default_timeout))
[Severity: High]
Can this lead to a recursive self-deadlock or a use-after-free?
If ibmvfc_send_event() fails, it invokes evt->done(evt) synchronously:
ibmvfc_send_event()
if (rc)
evt->done(evt);If the event is fresh, evt->done remains ibmvfc_locked_done(), which attempts to acquire vhost->host->host_lock. Since this lock is already held here before calling ibmvfc_send_event(), will this self-deadlock?
Additionally, because ibmvfc_init_event() doesn't zero evt->ls_req or evt->fcp_req, could recycled events contain stale pointers that cause a use-after-free when ibmvfc_send_event() evaluates those fields and replaces evt->done with ibmvfc_vfc_eh_done() on failure?
[Severity: Low]
On a related note regarding ibmvfc_send_event(), when the receive queue is closed it returns SCSI_MLQUEUE_HOST_BUSY. In places like ibmvfc_nvme_fcp_io() that directly return the output of ibmvfc_send_event(), does returning a positive SCSI midlayer error code to the NVMe FC transport cause unintended API violations? The NVMe FC transport typically expects 0 for success or standard negative error codes.
drivers/scsi/ibmvscsi/ibmvfc-nvme.c static int ibmvfc_nvme_ls_req(struct nvme_fc_local_port *lport, · line 218
Quoted 1 line
drivers/scsi/ibmvscsi/ibmvfc-nvme.c +1
-0+ goto out;
[Severity: High]
Does this goto skip the event cleanup? Although ibmvfc_send_event()'s error path executes evt->done, ibmvfc_sync_nvme_completion() only calls complete() on the event wait structure, meaning the event is never explicitly freed and could be permanently leaked.
drivers/scsi/ibmvscsi/ibmvfc-nvme.c static int ibmvfc_nvme_ls_req(struct nvme_fc_local_port *lport, · line 228
Quoted 10 lines
drivers/scsi/ibmvscsi/ibmvfc-nvme.c +10
-0+ +spin_unlock_irqrestore (vhost->host->host_lock, flags); + +wait_for_completion (&evt->comp); + status =be16_to_cpu (rsp.mad_common.status); +spin_lock_irqsave (vhost->host->host_lock, flags); +ibmvfc_free_event (evt); +out: +spin_unlock_irqrestore (vhost->host->host_lock, flags); +ibmvfc_dbg (vhost,"ls_abort: cancel failed with rc=%x\n" , status);
[Severity: Low]
Is status uninitialized when we reach the out label? If ibmvfc_send_event() fails and triggers the goto, status is never assigned and might leak uninitialized stack memory into the logs.
Quoted 2 lines
drivers/scsi/ibmvscsi/ibmvfc-nvme.c +1
-0+kref_put (&tgt->kref, ibmvfc_release_tgt); }
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=27
Tyrel Datwyler
maintainerPATCH 28/29 ibmvfc: implement nvme-fc FCP abort callback
In reply to Tyrel DatwylerImplement the NVMe-FC FCP abort callback by issuing an NVMF cancel MAD on the same submission queue used by the original FCP request.
Use the original request event stored in abort_req->private to recover the associated ibmvfc queue, then allocate a new event from that queue so the cancel is sent on the matching nvme_scrq. Factor the TMF setup into a dedicated helper, mirroring the LS abort path, and populate the cancel key, task tag, target WWPN, and association ID needed for the VIOS NVMF abort request.
The abort path sends the cancel synchronously, waits for completion, frees the temporary event, and logs non-zero MAD status values for debugging.
--- drivers/scsi/ibmvscsi/ibmvfc-nvme.c | 64 +++++++++++++++++++++++++++++ 1 file changed, 64 insertions(+)
drivers/scsi/ibmvscsi/ibmvfc-nvme.c +64 -0
@@ -374,11 +374,75 @@ static int ibmvfc_nvme_fcp_io(struct nvme_fc_local_port *lport,return rc; } +static void ibmvfc_init_fcp_abort (struct ibmvfc_event *evt, +struct nvmefc_fcp_req *abort_req) +{ +struct ibmvfc_tmf *tmf; +struct ibmvfc_event *abt_evt = abort_req->private; +struct ibmvfc_target *tgt = abt_evt->tgt; + + tmf = &evt->iu.tmf; +memset (tmf,0 ,sizeof (*tmf)); + tmf->common.version =cpu_to_be32 (2 ); + tmf->common.opcode =cpu_to_be32 (IBMVFC_NVMF_TMF_MAD ); + tmf->common.length =cpu_to_be16 (sizeof (*tmf)); + tmf->flags =cpu_to_be32 (IBMVFC_TMF_ABORT_TASK |IBMVFC_TMF_NVMF_ASSOC ); + tmf->cancel_key =cpu_to_be32 ((u64 )abt_evt); + tmf->my_cancel_key =cpu_to_be32 ((u64 )evt); + tmf->target_wwpn =cpu_to_be64 (tgt->wwpn); + tmf->assoc_id =cpu_to_be64 (tgt->assoc_id); + tmf->task_tag =cpu_to_be64 ((u64 )abt_evt); + +init_completion (&evt->comp); +} +static void ibmvfc_nvme_fcp_abort (struct nvme_fc_local_port *lport,struct nvme_fc_remote_port *rport,void *hw_queue_handle,struct nvmefc_fcp_req *abort_req) { +struct ibmvfc_host *vhost = lport->private; +struct ibmvfc_target *tgt = rport->private; +struct ibmvfc_event *evt, *abt_evt = abort_req->private; +struct ibmvfc_queue *queue; +union ibmvfc_iu rsp; +unsigned long flags; +u16 status =0 ; + +if (!abt_evt) +return ; + + queue = abt_evt->queue; +if (!vhost->logged_in || !queue) +return ; + + evt =ibmvfc_get_event (queue); +if (!evt) +return ; + +spin_lock_irqsave (queue->q_lock, flags); +kref_get (&tgt->kref); +ibmvfc_init_event (evt, ibmvfc_sync_nvme_completion,IBMVFC_MAD_FORMAT ); +ibmvfc_init_fcp_abort (evt, abort_req); + evt->sync_iu = &rsp; + +if (ibmvfc_send_event (evt, vhost, default_timeout)) + goto out; + +spin_unlock_irqrestore (queue->q_lock, flags); + +wait_for_completion (&evt->comp); + status =be16_to_cpu (rsp.mad_common.status); + +spin_lock_irqsave (queue->q_lock, flags); +ibmvfc_free_event (evt); +out: +spin_unlock_irqrestore (queue->q_lock, flags); + +if (status) +ibmvfc_dbg (vhost,"fcp_abort: cancel failed with rc=%x\n" , status); + +kref_put (&tgt->kref, ibmvfc_release_tgt); }static struct nvme_fc_port_template ibmvfc_nvme_fc_transport = {
Signature · 1 line
2.54.0
[email protected]
AI reviewRe: [PATCH 28/29] ibmvfc: implement nvme-fc FCP abort callback
Thank you for your contribution! Sashiko AI review found 6 potential issue(s) to consider: - [High] Synchronous wait in `fcp_abort` violates the NVMe-FC transport API and blocks the block layer…
Thank you for your contribution! Sashiko AI review found 6 potential issue(s) to consider:
- [High] Synchronous wait in fcp_abort violates the NVMe-FC transport API and blocks the block layer timeout handler.
- [High] Lack of synchronization against command completion allows fcp_abort to abort the wrong I/O request and causes a Use-After-Free of ibmvfc_event.
- [High] Indefinite hang in wait_for_completion when an NVMe MAD command times out, due to missing NVMe queue cleanup in the host reset path.
- [High] Lock inversion (AB-BA) deadlock between the hardware queue lock and the SCSI host lock during the failure path of ibmvfc_send_event.
- [High] Uninitialized evt->tgt pointer in FCP IO requests causes a NULL pointer dereference or Use-After-Free in the abort path.
- [Medium] Event pool exhaustion prevents aborts from issuing during hardware queue saturation, causing the transport to hang.
--
commit 0e079891124178111c0e0055ee9e67515aff453b
Author: Tyrel Datwyler <[email protected]>
ibmvfc: implement nvme-fc FCP abort callback
This commit implements the NVMe-FC FCP abort callback by issuing an NVMF cancel MAD on the same submission queue used by the original FCP request. It factors the TMF setup into a dedicated helper and sends the cancel synchronously, waiting for completion.
Quoted 5 lines
drivers/scsi/ibmvscsi/ibmvfc-nvme.c +0
-0@@ -374,11 +374,75 @@ static int ibmvfc_nvme_fcp_io(struct nvme_fc_local_port *lport,
[ ... ]
Quoted 6 lines
drivers/scsi/ibmvscsi/ibmvfc-nvme.c +6
-0+static void ibmvfc_init_fcp_abort (struct ibmvfc_event *evt, +struct nvmefc_fcp_req *abort_req) +{ +struct ibmvfc_tmf *tmf; +struct ibmvfc_event *abt_evt = abort_req->private; +struct ibmvfc_target *tgt = abt_evt->tgt;
drivers/scsi/ibmvscsi/ibmvfc-nvme.c static int ibmvfc_nvme_fcp_io(struct nvme_fc_local_port *lport, · line 392
Quoted 10 lines
drivers/scsi/ibmvscsi/ibmvfc-nvme.c +10
-0+ + tmf = &evt->iu.tmf; +memset (tmf,0 ,sizeof (*tmf)); + tmf->common.version =cpu_to_be32 (2 ); + tmf->common.opcode =cpu_to_be32 (IBMVFC_NVMF_TMF_MAD ); + tmf->common.length =cpu_to_be16 (sizeof (*tmf)); + tmf->flags =cpu_to_be32 (IBMVFC_TMF_ABORT_TASK |IBMVFC_TMF_NVMF_ASSOC ); + tmf->cancel_key =cpu_to_be32 ((u64 )abt_evt); + tmf->my_cancel_key =cpu_to_be32 ((u64 )evt); + tmf->target_wwpn =cpu_to_be64 (tgt->wwpn);
[Severity: High]
Does this dereference a NULL or stale pointer?
It appears that when an NVMe FCP request is submitted via ibmvfc_nvme_fcp_io(), an ibmvfc_event is allocated but its evt->tgt field is never initialized. Because evt->tgt was never set during submission, could it contain NULL (from pool zeroing) or a stale pointer from a previous LS request when we extract it here?
Quoted 13 lines
drivers/scsi/ibmvscsi/ibmvfc-nvme.c +13
-0+ tmf->assoc_id =cpu_to_be64 (tgt->assoc_id); + tmf->task_tag =cpu_to_be64 ((u64 )abt_evt); + +init_completion (&evt->comp); +} + +static void ibmvfc_nvme_fcp_abort (struct nvme_fc_local_port *lport, +struct nvme_fc_remote_port *rport, +void *hw_queue_handle, +struct nvmefc_fcp_req *abort_req) +{ +struct ibmvfc_host *vhost = lport->private; +struct ibmvfc_target *tgt = rport->private;
drivers/scsi/ibmvscsi/ibmvfc-nvme.c static int ibmvfc_nvme_fcp_io(struct nvme_fc_local_port *lport, · line 415
Quoted 10 lines
drivers/scsi/ibmvscsi/ibmvfc-nvme.c +10
-0+struct ibmvfc_event *evt, *abt_evt = abort_req->private; +struct ibmvfc_queue *queue; +union ibmvfc_iu rsp; +unsigned long flags; +u16 status =0 ; + +if (!abt_evt) +return ; + + queue = abt_evt->queue;
[Severity: High]
Is it possible for the original command to complete concurrently at the hardware level before this code accesses abt_evt?
If the interrupt handler returns abt_evt to the free pool, could this cause a use-after-free? Furthermore, if the event is reallocated to a new I/O request on another CPU, could the TMF payload be populated with the new event's tag, causing the VIOS to abort the wrong I/O request?
drivers/scsi/ibmvscsi/ibmvfc-nvme.c static int ibmvfc_nvme_fcp_io(struct nvme_fc_local_port *lport, · line 421
Quoted 6 lines
drivers/scsi/ibmvscsi/ibmvfc-nvme.c +6
-0+if (!vhost->logged_in || !queue) +return ; + + evt =ibmvfc_get_event (queue); +if (!evt) +return ;
[Severity: Medium]
Will this silently drop the abort request if the standard event pool is exhausted?
Since aborts are typically invoked when commands are timing out (often due to submission queue saturation), should this use a reserved event? If this returns NULL without sending the abort, could the command hang indefinitely since the NVMe FC transport assumes the abort was initiated?
drivers/scsi/ibmvscsi/ibmvfc-nvme.c static int ibmvfc_nvme_fcp_io(struct nvme_fc_local_port *lport, · line 429
Quoted 8 lines
drivers/scsi/ibmvscsi/ibmvfc-nvme.c +8
-0+ +spin_lock_irqsave (queue->q_lock, flags); +kref_get (&tgt->kref); +ibmvfc_init_event (evt, ibmvfc_sync_nvme_completion,IBMVFC_MAD_FORMAT ); +ibmvfc_init_fcp_abort (evt, abort_req); + evt->sync_iu = &rsp; + +if (ibmvfc_send_event (evt, vhost, default_timeout))
[Severity: High]
Can this cause a lock inversion (AB-BA) deadlock?
We acquire queue->q_lock before calling ibmvfc_send_event(). If ibmvfc_send_event() encounters an error, it calls evt->done(evt). Because the event is initialized with IBMVFC_MAD_FORMAT, the callback is ibmvfc_locked_done(), which unconditionally acquires host_lock.
This creates an acquisition order of queue->q_lock -> host_lock. However, the driver's reset path in ibmvfc_reset_host() acquires host_lock first, then iterates over the hardware queues to acquire queue->q_lock inside ibmvfc_purge_requests().
drivers/scsi/ibmvscsi/ibmvfc-nvme.c static int ibmvfc_nvme_fcp_io(struct nvme_fc_local_port *lport, · line 434
Quoted 5 lines
drivers/scsi/ibmvscsi/ibmvfc-nvme.c +5
-0+ goto out; + +spin_unlock_irqrestore (queue->q_lock, flags); + +wait_for_completion (&evt->comp);
[Severity: High]
Does this synchronous wait violate the NVMe-FC transport API?
The NVMe FC transport requires fcp_abort to be an asynchronous operation. Because fcp_abort is invoked by the block layer's timeout worker, waiting synchronously for the abort MAD to complete could stall the timeout processing thread for the system until the VIOS responds.
[Severity: High]
Can this wait_for_completion() hang indefinitely if the NVMe MAD command times out?
The fcp_abort TMF event is allocated on the NVMe queue (nvme_scrqs). If the VIOS fails to respond to the cancel MAD, ibmvfc_send_event() times out and triggers a host reset, invoking ibmvfc_purge_requests().
However, ibmvfc_purge_requests() iterates only over the legacy CRQ and vhost->scsi_scrqs, entirely ignoring vhost->nvme_scrqs. Because the event is never failed by the purge logic, could this wait forever?
Quoted 2 lines
drivers/scsi/ibmvscsi/ibmvfc-nvme.c +2
-0+ status =be16_to_cpu (rsp.mad_common.status); +
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=28
Tyrel Datwyler
maintainerPATCH 29/29 ibmvfc: fail nvme-fc fcp-io and ls requests during transport reset
In reply to Tyrel DatwylerThe current purge code for flushing outstanding commands during a transport reset only deals with SCSI commands. Rename the ibmvfc_scsi_eh_done completion handler to ibmvfc_vfc_eh_done and wire it to correctly complete nvme fcp and ls commands when flushing the inflight command list during a reset.
--- drivers/scsi/ibmvscsi/ibmvfc-core.c | 30 ++++++++++++++++++++--------- 1 file changed, 21 insertions(+), 9 deletions(-)
drivers/scsi/ibmvscsi/ibmvfc-core.c +21 -9
@@ -1092,20 +1092,28 @@ void ibmvfc_free_event(struct ibmvfc_event *evt) }/** -* ibmvfc_scsi_eh_done - EH done function for queuecommand commands +* ibmvfc_vfc_eh_done - EH done function for queued IO * @evt: ibmvfc event struct * -* This function does not setup any error status, that must be done -* before this function gets called. +* This function does not setup any error status for scsi commands, that must be +* done before this function gets called. **/ -static void ibmvfc_scsi_eh_done (struct ibmvfc_event *evt) +static void ibmvfc_vfc_eh_done (struct ibmvfc_event *evt) {struct scsi_cmnd *cmnd = evt->cmnd; +struct nvmefc_ls_req *ls_req = evt->ls_req; +struct nvmefc_fcp_req *fcp_req = evt->fcp_req;if (cmnd) {scsi_dma_unmap (cmnd);scsi_done (cmnd); - } + }else if (fcp_req) { + fcp_req->rcv_rsplen =0 ; + fcp_req->transferred_length =0 ; + fcp_req->status =NVME_SC_INTERNAL ; + fcp_req->done (fcp_req); + }else if (ls_req) + ls_req->done (ls_req, -EIO );ibmvfc_free_event (evt); }
@@ -1146,8 +1154,10 @@ static void ibmvfc_fail_request(struct ibmvfc_event *evt, int error_code)BUG_ON (!atomic_dec_and_test (&evt->active));if (evt->cmnd) { evt->cmnd->result = (error_code <<16 ); - evt->done = ibmvfc_scsi_eh_done; - }else + evt->done = ibmvfc_vfc_eh_done; + }else if (evt->fcp_req || evt->ls_req) + evt->done = ibmvfc_vfc_eh_done; +else evt->xfer_iu->mad_common.status =cpu_to_be16 (IBMVFC_MAD_DRIVER_FAILED );timer_delete (&evt->timer);
@@ -1816,8 +1826,10 @@ int ibmvfc_send_event(struct ibmvfc_event *evt,dev_err (vhost->dev,"Send error (rc=%d)\n" , rc);if (evt->cmnd) { evt->cmnd->result =DID_ERROR <<16 ; - evt->done = ibmvfc_scsi_eh_done; - }else + evt->done = ibmvfc_vfc_eh_done; + }else if (evt->fcp_req || evt->ls_req) + evt->done = ibmvfc_vfc_eh_done; +else evt->xfer_iu->mad_common.status =cpu_to_be16 (IBMVFC_MAD_CRQ_ERROR ); evt->done (evt);
Signature · 1 line
2.54.0
[email protected]
AI reviewRe: [PATCH 29/29] ibmvfc: fail nvme-fc fcp-io and ls requests during transport reset
Thank you for your contribution! Sashiko AI review found 4 potential issue(s) to consider:
Thank you for your contribution! Sashiko AI review found 4 potential issue(s) to consider:
New issues:
- [Critical] Pooled event structures retain stale NVMe request pointers, leading to exploitable Use-After-Free and double completions when non-NVMe commands fail.
- [High] Target kref is permanently leaked when an NVMe LS request fails or is purged.
Pre-existing issues:
- [High] Returning SCSI_MLQUEUE_HOST_BUSY for NVMe requests on transport closure causes permanent IO failures instead of requeuing.
- [Medium] NVMe FCP requests are not purged during a transport reset, rendering the patch's FCP functionality dead code. --
commit 88ce7ce7f11c11cfca700c6919820cf4d9865c8a
Author: Tyrel Datwyler <[email protected]>
ibmvfc: fail nvme-fc fcp-io and ls requests during transport reset
The current purge code for flushing outstanding commands during a transport reset only deals with SCSI commands. Rename the ibmvfc_scsi_eh_done completion handler to ibmvfc_vfc_eh_done and wire it to correctly complete nvme fcp and ls commands when flushing the inflight command list during a reset.
Quoted 5 lines
The current purge code for flushing outstanding commands during a transport reset only deals with SCSI commands. Rename the
ibmvfc_scsi_eh_donecompletion handler toibmvfc_vfc_eh_doneand wire it to correctly complete nvme fcp and ls commands when flushing the inflight command list during a reset.
[Severity: Medium]
This isn't a bug introduced by this patch, but does the purge code actually evaluate inflight NVMe FCP commands?
Looking at ibmvfc_purge_requests(), the loop iterates over vhost->crq and vhost->scsi_scrqs.scrqs:
drivers/scsi/ibmvscsi/ibmvfc-core.c:ibmvfc_purge_requests() {
...
for (i = 0; i < hwqs; i++) {
spin_lock_irqsave(queues[i].q_lock, flags);
...
}It seems to entirely miss vhost->nvme_scrqs.scrqs where FCP requests are queued. If so, inflight NVMe FCP commands might be orphaned during a reset, making the new FCP completion handling unreachable.
Quoted 4 lines
drivers/scsi/ibmvscsi/ibmvfc-core.c +0
-0
[ ... ]
Quoted 21 lines
drivers/scsi/ibmvscsi/ibmvfc-core.c +6
-4@@ -1092,20 +1092,28 @@ void ibmvfc_free_event(struct ibmvfc_event *evt) }/** -* ibmvfc_scsi_eh_done - EH done function for queuecommand commands +* ibmvfc_vfc_eh_done - EH done function for queued IO * @evt: ibmvfc event struct * -* This function does not setup any error status, that must be done -* before this function gets called. +* This function does not setup any error status for scsi commands, that must be +* done before this function gets called. **/ -static void ibmvfc_scsi_eh_done (struct ibmvfc_event *evt) +static void ibmvfc_vfc_eh_done (struct ibmvfc_event *evt) {struct scsi_cmnd *cmnd = evt->cmnd; +struct nvmefc_ls_req *ls_req = evt->ls_req; +struct nvmefc_fcp_req *fcp_req = evt->fcp_req;if (cmnd) {
drivers/scsi/ibmvscsi/ibmvfc-core.c void ibmvfc_free_event(struct ibmvfc_event *evt) · line 1116
Quoted 10 lines
drivers/scsi/ibmvscsi/ibmvfc-core.c +7
-1scsi_dma_unmap (cmnd);scsi_done (cmnd); - } + }else if (fcp_req) { + fcp_req->rcv_rsplen =0 ; + fcp_req->transferred_length =0 ; + fcp_req->status =NVME_SC_INTERNAL ; + fcp_req->done (fcp_req); + }else if (ls_req) + ls_req->done (ls_req, -EIO );
[Severity: High]
Does this error handler leak the target kref for LS requests?
In ibmvfc_nvme_ls_req(), a reference is taken via kref_get(&tgt->kref) which is typically dropped in the success path by ibmvfc_ls_req_done().
Since this patch replaces the completion handler with ibmvfc_vfc_eh_done(), the transport request is completed here but kref_put(&evt->tgt->kref, ...) is never called, potentially leaving the target structure leaked.
drivers/scsi/ibmvscsi/ibmvfc-core.c static void ibmvfc_fail_request(struct ibmvfc_event *evt, int error_code) · line 1160
Quoted 13 lines
drivers/scsi/ibmvscsi/ibmvfc-core.c +4
-2ibmvfc_free_event (evt); }@@ -1146,8 +1154,10 @@ static void ibmvfc_fail_request(struct ibmvfc_event *evt, int error_code)BUG_ON (!atomic_dec_and_test (&evt->active));if (evt->cmnd) { evt->cmnd->result = (error_code <<16 ); - evt->done = ibmvfc_scsi_eh_done; - }else + evt->done = ibmvfc_vfc_eh_done; + }else if (evt->fcp_req || evt->ls_req) + evt->done = ibmvfc_vfc_eh_done; +else
[Severity: Critical]
Can these checks be triggered by stale NVMe pointers on non-NVMe commands?
When ibmvfc_get_event() reallocates an event structure from the pool, ibmvfc_init_event() doesn't clear evt->fcp_req or evt->ls_req to NULL:
drivers/scsi/ibmvscsi/ibmvfc-core.c:ibmvfc_init_event() {
...
evt->cmnd = NULL;
evt->sync_iu = NULL;
evt->eh_comp = NULL;
...
}If a non-NVMe command (e.g., a MAD command) is allocated an event with stale NVMe pointers and fails here, will it incorrectly replace the completion handler with ibmvfc_vfc_eh_done() and later use-after-free the stale NVMe pointers?
Quoted 3 lines
drivers/scsi/ibmvscsi/ibmvfc-core.c +0
-0evt->xfer_iu->mad_common.status =cpu_to_be16 (IBMVFC_MAD_DRIVER_FAILED );timer_delete (&evt->timer);
[ ... ]
drivers/scsi/ibmvscsi/ibmvfc-core.c int ibmvfc_send_event(struct ibmvfc_event *evt,
Quoted 1 line
drivers/scsi/ibmvscsi/ibmvfc-core.c +0
-0@@ -1816,8 +1826,10 @@ int ibmvfc_send_event(struct ibmvfc_event *evt,
[Severity: High]
This isn't a bug introduced by this patch, but earlier in ibmvfc_send_event(), it hardcodes a return of SCSI_MLQUEUE_HOST_BUSY on H_CLOSED without distinguishing between SCSI and NVMe commands:
drivers/scsi/ibmvscsi/ibmvfc-core.c:ibmvfc_send_event() {
...
if (rc == H_CLOSED) {
...
return SCSI_MLQUEUE_HOST_BUSY;
}
...
}This value (0x1055) is propagated back up through ibmvfc_nvme_fcp_io() directly to the NVMe FC transport layer. Since 0x1055 is neither 0 nor -EBUSY, will the NVMe FC core treat it as a hard failure and abort the request with BLK_STS_IOERR instead of handling the queue pressure appropriately?
Quoted 12 lines
drivers/scsi/ibmvscsi/ibmvfc-core.c +4
-2dev_err (vhost->dev,"Send error (rc=%d)\n" , rc);if (evt->cmnd) { evt->cmnd->result =DID_ERROR <<16 ; - evt->done = ibmvfc_scsi_eh_done; - }else + evt->done = ibmvfc_vfc_eh_done; + }else if (evt->fcp_req || evt->ls_req) + evt->done = ibmvfc_vfc_eh_done; +else evt->xfer_iu->mad_common.status =cpu_to_be16 (IBMVFC_MAD_CRQ_ERROR ); evt->done (evt);
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=29
Martin K. Petersen
AppliedTyrel,
Quoted 3 lines
This patch series adds NVMe-FC protocol support to the ibmvfc driver, enabling IBM POWER virtual Fibre Channel adapters to handle both SCSI and NVMe storage traffic through a unified driver architecture.
Applied to 7.3/scsi-staging, thanks!
Signature · 1 line
Martin K. Petersen
Tyrel Datwyler
maintainerQuoted 9 lines · Martin K. Petersen
Tyrel,
Quoted 3 lines
This patch series adds NVMe-FC protocol support to the ibmvfc driver, enabling IBM POWER virtual Fibre Channel adapters to handle both SCSI and NVMe storage traffic through a unified driver architecture.
Applied to 7.3/scsi-staging, thanks!
Martin,
Can you drop this series from the staging branch? I've got a v2 respin that I will get sent out today or tomorrow that addresses issues raised by the Sashiko review. In particular I clearly had a bad rebase somewhere where I dropped/left out some of the remote port processing code. There are also several fixes for long standing edge case bugs that Sashiko points out with both the NVMf work and Dave's FPIN series.
Thanks,
-Tyrel