nvme-fc: FPIN link integrity handling
v6 cover nvme-fc: FPIN link integrity handling
FPIN LI (link integrity) messages are received when the attached fabric detects hardware errors. In response to these messages I/O should be directed away from the affected ports, and only used if the 'optimized' paths are unavailable.
Upon port reset the paths should be put back in service as the affected hardware might have been replaced.
This patch adds a new controller flag 'NVME_CTRL_MARGINAL' which will be checked during multipath path selection, causing the path to be skipped when checking for 'optimized' paths. If no optimized paths are available the 'marginal' paths are considered for path selection alongside the 'non-optimized' paths. It also introduces a new nvme-fc callback 'nvme_fc_fpin_rcv()' to evaluate the FPIN LI TLV payload and set the 'marginal' state on all affected rports.
The testing for this patch set was performed by Bryan Gurney, using the process outlined by John Meneghini's presentation at LSFMM 2024, where the fibre channel switch sends an FPIN notification on a specific switch port, and the following is checked on the initiator:
1. The controllers corresponding to the paths on the port that has received the notification are showing a set NVME_CTRL_MARGINAL flag.
\
+- nvme4 fc traddr=c,host_traddr=e live optimized
+- nvme5 fc traddr=8,host_traddr=e live non-optimized
+- nvme8 fc traddr=e,host_traddr=f marginal optimized
+- nvme9 fc traddr=a,host_traddr=f marginal non-optimized2. The I/O statistics of the test namespace show no I/O activity on the controllers with NVME_CTRL_MARGINAL set.
Device tps MB_read/s MB_wrtn/s MB_dscd/s
nvme4c4n1 0.00 0.00 0.00 0.00 nvme4c5n1 25001.00 0.00 97.66 0.00 nvme4c9n1 25000.00 0.00 97.66 0.00 nvme4n1 50011.00 0.00 195.36 0.00
Device tps MB_read/s MB_wrtn/s MB_dscd/s
nvme4c4n1 0.00 0.00 0.00 0.00 nvme4c5n1 48360.00 0.00 188.91 0.00 nvme4c9n1 1642.00 0.00 6.41 0.00 nvme4n1 49981.00 0.00 195.24 0.00
Device tps MB_read/s MB_wrtn/s MB_dscd/s
nvme4c4n1 0.00 0.00 0.00 0.00 nvme4c5n1 50001.00 0.00 195.32 0.00 nvme4c9n1 0.00 0.00 0.00 0.00 nvme4n1 50016.00 0.00 195.38 0.00
Changes to the original submission:
- Changed flag name to 'marginal'
- Do not block marginal path; influence path selection instead to de-prioritize marginal paths
Changes to v2:
- Split off driver-specific modifications
- Introduce 'union fc_tlv_desc' to avoid casts
Changes to v3:
- Include reviews from Justin Tee
- Split marginal path handling patch
Changes to v4:
- Change 'u8' to '__u8' on fc_tlv_desc to fix a failure to build
- Print 'marginal' instead of 'live' in the state of controllers when they are marginal
Changes to v5:
- Minor spelling corrections to patch descriptions
Hannes Reinecke (5):
fc_els: use 'union fc_tlv_desc'
nvme-fc: marginal path handling
nvme-fc: nvme_fc_fpin_rcv() callback
lpfc: enable FPIN notification for NVMe
qla2xxx: enable FPIN notification for NVMe
Bryan Gurney (1):
nvme: sysfs: emit the marginal path state in show_state()
drivers/nvme/host/core.c | 1 + drivers/nvme/host/fc.c | 99 +++++++++++++++++++ drivers/nvme/host/multipath.c | 17 ++-- drivers/nvme/host/nvme.h | 6 ++ drivers/nvme/host/sysfs.c | 4 +- drivers/scsi/lpfc/lpfc_els.c | 84 ++++++++-------- drivers/scsi/qla2xxx/qla_isr.c | 3 + drivers/scsi/scsi_transport_fc.c | 27 +++-- include/linux/nvme-fc-driver.h | 3 + include/uapi/scsi/fc/fc_els.h | 165 +++++++++++++++++-------------- 10 files changed, 269 insertions(+), 140 deletions(-)
Signature · 1 line
2.49.0