connect reject event helpers
Subsystems (7)
v5 cover connect reject event helpers
While reviewing:
http://lists.infradead.org/pipermail/linux-nvme/2016-October/006681.html
I decided to propose transport-agnostic helper functions to better handle connection reject event information. Included are patches for nvme/fabrics, iser, and rds to utilize the new helpers.
Doug, this series applies on top of v4.9-rc2 as well as sagi's nvmf-4.9-rc branch as of today. The series should be targeted for 4.10.
Changes since v4:
- fixed typos in the review-by tags for Sagi
- make rej_data a const pointer in nvme_rdma_conn_rejected()
- balanced braces for if/then/else in nvme_rdma_conn_rejected()
Changes since v3:
- added reviewed-by/acked-by tags
- positive logic in reject_msg helpers
- cleaned up reject message strings
- added patches to use helpers in isert and nvmet_rdma
- fixed zero-day build warning
Changes since v2:
- reworked ibcm/iwcm_reject_msg() as per Christoph's recommendation
- use ibcm_ and iwcm_ prefix instead of ib_ and iw_ for reject_msg funcs
- change rdma_consumer_reject() to rdma_is_consumer_reject()
- add rdma_consumer_reject_data() helper function to return private data/len
- use new helpers in nvme_rdma, ib_iser, and rdma_rds
- in nvme_rdma, add strings for nvme_rdma_cm_status values
---
Steve Wise (8):
rdma_cm: add rdma_reject_msg() helper function
rdma_cm: add rdma_is_consumer_reject() helper function
rdma_cm: add rdma_consumer_reject_data helper function
nvme-rdma: use rdma connection reject helper functions
ib_iser: log the connection reject message
rds_rdma: log the connection reject message
ib_isert: log the connection reject message
nvmet_rdma: log the connection reject message
drivers/infiniband/core/cm.c | 48 ++++++++++++++++++++++++++++++++ drivers/infiniband/core/cma.c | 43 ++++++++++++++++++++++++++++ drivers/infiniband/core/iwcm.c | 21 ++++++++++++++ drivers/infiniband/ulp/iser/iser_verbs.c | 5 +++- drivers/infiniband/ulp/isert/ib_isert.c | 2 ++ drivers/nvme/host/rdma.c | 42 ++++++++++++++++++++++++---- drivers/nvme/target/rdma.c | 3 ++ include/rdma/ib_cm.h | 6 ++++ include/rdma/iw_cm.h | 6 ++++ include/rdma/rdma_cm.h | 25 +++++++++++++++++ net/rds/rdma_transport.c | 5 +++- 11 files changed, 198 insertions(+), 8 deletions(-)
Signature · 1 line
2.7.0
PATCH v5 1/8 rdma_cm: add rdma_reject_msg() helper function
rdma_reject_msg() returns a pointer to a string message associated with the transport reject reason codes.
--- drivers/infiniband/core/cm.c | 48 ++++++++++++++++++++++++++++++++++++++++++ drivers/infiniband/core/cma.c | 14 ++++++++++++ drivers/infiniband/core/iwcm.c | 21 ++++++++++++++++++ include/rdma/ib_cm.h | 6 ++++++ include/rdma/iw_cm.h | 6 ++++++ include/rdma/rdma_cm.h | 8 +++++++ 6 files changed, 103 insertions(+)
drivers/infiniband/core/cm.c +48 -0
@@ -57,6 +57,54 @@ MODULE_AUTHOR("Sean Hefty");MODULE_DESCRIPTION ("InfiniBand CM" );MODULE_LICENSE ("Dual BSD/GPL" ); +static const char *const ibcm_rej_reason_strs[] = { + [IB_CM_REJ_NO_QP ] ="no QP" , + [IB_CM_REJ_NO_EEC ] ="no EEC" , + [IB_CM_REJ_NO_RESOURCES ] ="no resources" , + [IB_CM_REJ_TIMEOUT ] ="timeout" , + [IB_CM_REJ_UNSUPPORTED ] ="unsupported" , + [IB_CM_REJ_INVALID_COMM_ID ] ="invalid comm ID" , + [IB_CM_REJ_INVALID_COMM_INSTANCE ] ="invalid comm instance" , + [IB_CM_REJ_INVALID_SERVICE_ID ] ="invalid service ID" , + [IB_CM_REJ_INVALID_TRANSPORT_TYPE ] ="invalid transport type" , + [IB_CM_REJ_STALE_CONN ] ="stale conn" , + [IB_CM_REJ_RDC_NOT_EXIST ] ="RDC not exist" , + [IB_CM_REJ_INVALID_GID ] ="invalid GID" , + [IB_CM_REJ_INVALID_LID ] ="invalid LID" , + [IB_CM_REJ_INVALID_SL ] ="invalid SL" , + [IB_CM_REJ_INVALID_TRAFFIC_CLASS ] ="invalid traffic class" , + [IB_CM_REJ_INVALID_HOP_LIMIT ] ="invalid hop limit" , + [IB_CM_REJ_INVALID_PACKET_RATE ] ="invalid packet rate" , + [IB_CM_REJ_INVALID_ALT_GID ] ="invalid alt GID" , + [IB_CM_REJ_INVALID_ALT_LID ] ="invalid alt LID" , + [IB_CM_REJ_INVALID_ALT_SL ] ="invalid alt SL" , + [IB_CM_REJ_INVALID_ALT_TRAFFIC_CLASS ] ="invalid alt traffic class" , + [IB_CM_REJ_INVALID_ALT_HOP_LIMIT ] ="invalid alt hop limit" , + [IB_CM_REJ_INVALID_ALT_PACKET_RATE ] ="invalid alt packet rate" , + [IB_CM_REJ_PORT_CM_REDIRECT ] ="port CM redirect" , + [IB_CM_REJ_PORT_REDIRECT ] ="port redirect" , + [IB_CM_REJ_INVALID_MTU ] ="invalid MTU" , + [IB_CM_REJ_INSUFFICIENT_RESP_RESOURCES ] ="insufficient resp resources" , + [IB_CM_REJ_CONSUMER_DEFINED ] ="consumer defined" , + [IB_CM_REJ_INVALID_RNR_RETRY ] ="invalid RNR retry" , + [IB_CM_REJ_DUPLICATE_LOCAL_COMM_ID ] ="duplicate local comm ID" , + [IB_CM_REJ_INVALID_CLASS_VERSION ] ="invalid class version" , + [IB_CM_REJ_INVALID_FLOW_LABEL ] ="invalid flow label" , + [IB_CM_REJ_INVALID_ALT_FLOW_LABEL ] ="invalid alt flow label" , +}; + +const char *__attribute_const__ ibcm_reject_msg(int reason) +{ +size_t index = reason; + +if (index <ARRAY_SIZE (ibcm_rej_reason_strs) && + ibcm_rej_reason_strs[index]) +return ibcm_rej_reason_strs[index]; +else +return "unrecognized reason" ; +} +EXPORT_SYMBOL (ibcm_reject_msg); +static void cm_add_one (struct ib_device *device);static void cm_remove_one (struct ib_device *device,void *client_data);
drivers/infiniband/core/cma.c +14 -0
@@ -101,6 +101,20 @@ const char *__attribute_const__ rdma_event_msg(enum rdma_cm_event_type event) }EXPORT_SYMBOL (rdma_event_msg); +const char *__attribute_const__ rdma_reject_msg(struct rdma_cm_id *id, +int reason) +{ +if (rdma_ib_or_roce (id->device, id->port_num)) +return ibcm_reject_msg (reason); + +if (rdma_protocol_iwarp (id->device, id->port_num)) +return iwcm_reject_msg (reason); + +WARN_ON_ONCE (1 ); +return "unrecognized transport" ; +} +EXPORT_SYMBOL (rdma_reject_msg); +static void cma_add_one (struct ib_device *device);static void cma_remove_one (struct ib_device *device,void *client_data);
drivers/infiniband/core/iwcm.c +21 -0
@@ -59,6 +59,27 @@ MODULE_AUTHOR("Tom Tucker");MODULE_DESCRIPTION ("iWARP CM" );MODULE_LICENSE ("Dual BSD/GPL" ); +static const char *const iwcm_rej_reason_strs[] = { + [ECONNRESET ] ="reset by remote host" , + [ECONNREFUSED ] ="refused by remote application" , + [ETIMEDOUT ] ="setup timeout" , +}; + +const char *__attribute_const__ iwcm_reject_msg(int reason) +{ +size_t index; + +/* iWARP uses negative errnos */ + index = -reason; + +if (index <ARRAY_SIZE (iwcm_rej_reason_strs) && + iwcm_rej_reason_strs[index]) +return iwcm_rej_reason_strs[index]; +else +return "unrecognized reason" ; +} +EXPORT_SYMBOL (iwcm_reject_msg); +static struct ibnl_client_cbs iwcm_nl_cb_table[] = { [RDMA_NL_IWPM_REG_PID ] = {.dump = iwpm_register_pid_cb}, [RDMA_NL_IWPM_ADD_MAPPING ] = {.dump = iwpm_add_mapping_cb},
include/rdma/ib_cm.h +6 -0
@@ -603,4 +603,10 @@ struct ib_cm_sidr_rep_param {int ib_send_cm_sidr_rep (struct ib_cm_id *cm_id,struct ib_cm_sidr_rep_param *param); +/** +* ibcm_reject_msg - return a pointer to a reject message string. +* @reason: Value returned in the REJECT event status field. +*/ +const char *__attribute_const__ ibcm_reject_msg(int reason); +#endif /* IB_CM_H */
include/rdma/iw_cm.h +6 -0
@@ -253,4 +253,10 @@ int iw_cm_disconnect(struct iw_cm_id *cm_id, int abrupt);int iw_cm_init_qp_attr (struct iw_cm_id *cm_id,struct ib_qp_attr *qp_attr,int *qp_attr_mask); +/** +* iwcm_reject_msg - return a pointer to a reject message string. +* @reason: Value returned in the REJECT event status field. +*/ +const char *__attribute_const__ iwcm_reject_msg(int reason); +#endif /* IW_CM_H */
include/rdma/rdma_cm.h +8 -0
@@ -388,4 +388,12 @@ int rdma_set_afonly(struct rdma_cm_id *id, int afonly);*/ __be64 rdma_get_service_id (struct rdma_cm_id *id,struct sockaddr *addr); +/** +* rdma_reject_msg - return a pointer to a reject message string. +* @id: Communication identifier that received the REJECT event. +* @reason: Value returned in the REJECT event status field. +*/ +const char *__attribute_const__ rdma_reject_msg(struct rdma_cm_id *id, +int reason); +#endif /* RDMA_CM_H */
Signature · 1 line
2.7.0
PATCH v5 1/8 rdma_cm: add rdma_reject_msg() helper function
Quoted 2 lines · Steve Wise
rdma_reject_msg()returns a pointer to a string message associated with the transport reject reason codes.
PATCH v5 2/8 rdma_cm: add rdma_is_consumer_reject() helper function
In reply to Steve WiseReturn true if the peer consumer application rejected the connection attempt.
--- drivers/infiniband/core/cma.c | 13 +++++++++++++ include/rdma/rdma_cm.h | 7 +++++++ 2 files changed, 20 insertions(+)
drivers/infiniband/core/cma.c +13 -0
@@ -115,6 +115,19 @@ const char *__attribute_const__ rdma_reject_msg(struct rdma_cm_id *id, }EXPORT_SYMBOL (rdma_reject_msg); +bool rdma_is_consumer_reject (struct rdma_cm_id *id,int reason) +{ +if (rdma_ib_or_roce (id->device, id->port_num)) +return reason ==IB_CM_REJ_CONSUMER_DEFINED ; + +if (rdma_protocol_iwarp (id->device, id->port_num)) +return reason == -ECONNREFUSED ; + +WARN_ON_ONCE (1 ); +return false; +} +EXPORT_SYMBOL (rdma_is_consumer_reject); +static void cma_add_one (struct ib_device *device);static void cma_remove_one (struct ib_device *device,void *client_data);
include/rdma/rdma_cm.h +7 -0
@@ -395,5 +395,12 @@ __be64 rdma_get_service_id(struct rdma_cm_id *id, struct sockaddr *addr);*/ const char *__attribute_const__ rdma_reject_msg(struct rdma_cm_id *id,int reason); +/** +* rdma_is_consumer_reject - return true if the consumer rejected the connect +* request. +* @id: Communication identifier that received the REJECT event. +* @reason: Value returned in the REJECT event status field. +*/ +bool rdma_is_consumer_reject (struct rdma_cm_id *id,int reason);#endif /* RDMA_CM_H */
Signature · 1 line
2.7.0
PATCH v5 2/8 rdma_cm: add rdma_is_consumer_reject() helper function
Quoted 2 lines · Steve Wise
Return true if the peer consumer application rejected the connection attempt.
PATCH v5 3/8 rdma_cm: add rdma_consumer_reject_data helper function
In reply to Steve Wiserdma_consumer_reject_data() will return the private data pointer and length if any is available.
--- drivers/infiniband/core/cma.c | 16 ++++++++++++++++ include/rdma/rdma_cm.h | 10 ++++++++++ 2 files changed, 26 insertions(+)
drivers/infiniband/core/cma.c +16 -0
@@ -128,6 +128,22 @@ bool rdma_is_consumer_reject(struct rdma_cm_id *id, int reason) }EXPORT_SYMBOL (rdma_is_consumer_reject); +const void *rdma_consumer_reject_data (struct rdma_cm_id *id, +struct rdma_cm_event *ev,u8 *data_len) +{ +const void *p; + +if (rdma_is_consumer_reject (id, ev->status)) { + *data_len = ev->param.conn.private_data_len; + p = ev->param.conn.private_data; + }else { + *data_len =0 ; + p =NULL ; + } +return p; +} +EXPORT_SYMBOL (rdma_consumer_reject_data); +static void cma_add_one (struct ib_device *device);static void cma_remove_one (struct ib_device *device,void *client_data);
include/rdma/rdma_cm.h +10 -0
@@ -403,4 +403,14 @@ const char *__attribute_const__ rdma_reject_msg(struct rdma_cm_id *id,*/ bool rdma_is_consumer_reject (struct rdma_cm_id *id,int reason); +/** +* rdma_consumer_reject_data - return the consumer reject private data and +* length, if any. +* @id: Communication identifier that received the REJECT event. +* @ev: RDMA CM reject event. +* @data_len: Pointer to the resulting length of the consumer data. +*/ +const void *rdma_consumer_reject_data (struct rdma_cm_id *id, +struct rdma_cm_event *ev,u8 *data_len); +#endif /* RDMA_CM_H */
Signature · 1 line
2.7.0
PATCH v5 3/8 rdma_cm: add rdma_consumer_reject_data helper function
Quoted 2 lines · Steve Wise
rdma_consumer_reject_data()will return the private data pointer and length if any is available.
PATCH v5 4/8 nvme-rdma: use rdma connection reject helper functions
In reply to Steve WiseAlso add nvme cm status strings and use them.
--- drivers/nvme/host/rdma.c | 42 ++++++++++++++++++++++++++++++++++++------ 1 file changed, 36 insertions(+), 6 deletions(-)
drivers/nvme/host/rdma.c +36 -6
@@ -43,6 +43,28 @@#define NVME_RDMA_MAX_INLINE_SEGMENTS 1 +static const char *const nvme_rdma_cm_status_strs[] = { + [NVME_RDMA_CM_INVALID_LEN ] ="invalid length" , + [NVME_RDMA_CM_INVALID_RECFMT ] ="invalid record format" , + [NVME_RDMA_CM_INVALID_QID ] ="invalid queue ID" , + [NVME_RDMA_CM_INVALID_HSQSIZE ] ="invalid host SQ size" , + [NVME_RDMA_CM_INVALID_HRQSIZE ] ="invalid host RQ size" , + [NVME_RDMA_CM_NO_RSC ] ="resource not found" , + [NVME_RDMA_CM_INVALID_IRD ] ="invalid IRD" , + [NVME_RDMA_CM_INVALID_ORD ] ="Invalid ORD" , +}; + +static const char *nvme_rdma_cm_msg (enum nvme_rdma_cm_status status) +{ +size_t index = status; + +if (index <ARRAY_SIZE (nvme_rdma_cm_status_strs) && + nvme_rdma_cm_status_strs[index]) +return nvme_rdma_cm_status_strs[index]; +else +return "unrecognized reason" ; +}; +/* * We handle AEN commands ourselves and don't even let the * block layer know about them.
@@ -1222,16 +1244,24 @@ out_destroy_queue_ib:static int nvme_rdma_conn_rejected (struct nvme_rdma_queue *queue,struct rdma_cm_event *ev) { -if (ev->param.conn.private_data_len) { -struct nvme_rdma_cm_rej *rej = - (struct nvme_rdma_cm_rej *)ev->param.conn.private_data; +struct rdma_cm_id *cm_id = queue->cm_id; +int status = ev->status; +const char *rej_msg; +const struct nvme_rdma_cm_rej *rej_data; +u8 rej_data_len; + + rej_msg =rdma_reject_msg (cm_id, status); + rej_data =rdma_consumer_reject_data (cm_id, ev, &rej_data_len); + +if (rej_data && rej_data_len >=sizeof (u16)) { +u16 sts =le16_to_cpu (rej_data->sts);dev_err (queue->ctrl->ctrl.device, -"Connect rejected, status %d." ,le16_to_cpu (rej->sts)); -/* XXX: Think of something clever to do here... */ +"Connect rejected: status %d (%s) nvme status %d (%s).\n" , + status, rej_msg, sts,nvme_rdma_cm_msg (sts)); }else {dev_err (queue->ctrl->ctrl.device, -"Connect rejected, no private data.\n" ); +"Connect rejected: status %d (%s).\n" , status, rej_msg); }return -ECONNRESET ;
Signature · 1 line
2.7.0
PATCH v5 4/8 nvme-rdma: use rdma connection reject helper functions
Quoted 1 line · Steve Wise
Also add nvme cm status strings and use them.
PATCH v5 5/8 ib_iser: log the connection reject message
In reply to Steve Wise--- drivers/infiniband/ulp/iser/iser_verbs.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-)
drivers/infiniband/ulp/iser/iser_verbs.c +4 -1
@@ -906,11 +906,14 @@ static int iser_cma_handler(struct rdma_cm_id *cma_id, struct rdma_cm_event *evecase RDMA_CM_EVENT_ESTABLISHED :iser_connected_handler (cma_id, event->param.conn.private_data);break ; +case RDMA_CM_EVENT_REJECTED : +iser_info ("Connection rejected: %s\n" , +rdma_reject_msg (cma_id, event->status)); +/* FALLTHROUGH */ case RDMA_CM_EVENT_ADDR_ERROR :case RDMA_CM_EVENT_ROUTE_ERROR :case RDMA_CM_EVENT_CONNECT_ERROR :case RDMA_CM_EVENT_UNREACHABLE : -case RDMA_CM_EVENT_REJECTED :iser_connect_error (cma_id);break ;case RDMA_CM_EVENT_DISCONNECTED :
Signature · 1 line
2.7.0
Christoph Hellwig
maintainerPATCH v5 5/8 ib_iser: log the connection reject message
Looks fine,
PATCH v5 6/8 rds_rdma: log the connection reject message
In reply to Steve Wise--- net/rds/rdma_transport.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-)
net/rds/rdma_transport.c +4 -1
@@ -100,11 +100,14 @@ int rds_rdma_cm_event_handler(struct rdma_cm_id *cm_id, trans->cm_connect_complete (conn, event);break ; +case RDMA_CM_EVENT_REJECTED : +rdsdebug ("Connection rejected: %s\n" , +rdma_reject_msg (cm_id, event->status)); +/* FALLTHROUGH */ case RDMA_CM_EVENT_ADDR_ERROR :case RDMA_CM_EVENT_ROUTE_ERROR :case RDMA_CM_EVENT_CONNECT_ERROR :case RDMA_CM_EVENT_UNREACHABLE : -case RDMA_CM_EVENT_REJECTED :case RDMA_CM_EVENT_DEVICE_REMOVAL :case RDMA_CM_EVENT_ADDR_CHANGE :if (conn)
Signature · 1 line
2.7.0
PATCH v5 7/8 ib_isert: log the connection reject message
In reply to Steve Wise--- drivers/infiniband/ulp/isert/ib_isert.c | 2 ++ 1 file changed, 2 insertions(+)
drivers/infiniband/ulp/isert/ib_isert.c +2 -0
@@ -795,6 +795,8 @@ isert_cma_handler(struct rdma_cm_id *cma_id, struct rdma_cm_event *event)*/ return 1 ;case RDMA_CM_EVENT_REJECTED :/* FALLTHRU */ +isert_info ("Connection rejected: %s\n" , +rdma_reject_msg (cma_id, event->status));case RDMA_CM_EVENT_UNREACHABLE :/* FALLTHRU */ case RDMA_CM_EVENT_CONNECT_ERROR : ret =isert_connect_error (cma_id);
Signature · 1 line
2.7.0
Christoph Hellwig
maintainerPATCH v5 7/8 ib_isert: log the connection reject message
Looks fine,
PATCH v5 8/8 nvmet_rdma: log the connection reject message
In reply to Steve Wise--- drivers/nvme/target/rdma.c | 3 +++ 1 file changed, 3 insertions(+)
drivers/nvme/target/rdma.c +3 -0
@@ -1358,6 +1358,9 @@ static int nvmet_rdma_cm_handler(struct rdma_cm_id *cm_id, ret =nvmet_rdma_device_removal (cm_id, queue);break ;case RDMA_CM_EVENT_REJECTED : +pr_debug ("Connection rejected: %s\n" , +rdma_reject_msg (cm_id, event->status)); +/* FALLTHROUGH */ case RDMA_CM_EVENT_UNREACHABLE :case RDMA_CM_EVENT_CONNECT_ERROR :nvmet_rdma_queue_connect_fail (cm_id, queue);
Signature · 1 line
2.7.0
Christoph Hellwig
maintainerPATCH v5 8/8 nvmet_rdma: log the connection reject message
Looks fine,
v5 cover connect reject event helpers
In reply to Steve WiseHey Doug,
Are you planning to merge this series for 4.10? They're still marked 'new' on patchworks.
Thanks,
Steve.
v5 cover connect reject event helpers
Quoted 9 lines · Steve Wise
Hey Doug,
Are you planning to merge this series for 4.10? They're still marked 'new' on patchworks.
Thanks,
Steve.
Yes, series applied.
Signature · 10 lines
Doug Ledford <dledford at redhat.com>
GPG Key ID: 0E572FDD
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 884 bytes
Desc: OpenPGP digital signature
URL: <http://lists.infradead.org/pipermail/linux-nvme/attachments/20161214/b925e938/attachment.sig>