IB/Verbs: IB Management Helpers
Subsystems (6)
cover IB/Verbs: IB Management Helpers
There are plenty of lengthy code to check the transport type of IB device, or the link layer type of it's port, but actually we are just speculating whether a particular management is supported by the device/port.
Thus instead of inferring, we should have our own mechanism for IB management capability/protocol checking, several proposals below.
This patch set is the pioneer which try to collecting all the cases where a management helper applicable, and put all the speculating logical together, in order to be easily integrated to the new querying mechanism in future.
TODO:
The helper in patch 10 and 11 are still to be classified, that is cap_ib() and cap_eth().
The name/description of each helper may not correct enough, those folks who are familiar with these particular parts, your suggestion would be invaluable ;-)
Proposals:
Sean:
https://www.mail-archive.com/[email protected]/msg23339.html
Doug:
https://www.mail-archive.com/[email protected]/msg23418.html
Jason:
https://www.mail-archive.com/[email protected]/msg23425.htmlMichael Wang (11):
[PATCH 01/11] IB/Verbs: Use helpers to check transport and link layer
[RFC PATCH 02/11] IB/Verbs: Use management helper tech_iboe() for iboe-check [RFC PATCH 03/11] IB/Verbs: Use management helper has_mad() for mad-check [RFC PATCH 04/11] IB/Verbs: Use management helper cap_smi() for smi-check [RFC PATCH 05/11] IB/Verbs: Use management helper has_cm() for cm-check
[RFC PATCH 06/11] IB/Verbs: Use management helper has_sa() and cap_sa() for sa-check [RFC PATCH 07/11] IB/Verbs: Use management helper has_mcast() and cap_mcast() for mcast-check [RFC PATCH 08/11] IB/Verbs: Use management helper has_iwarp() for iwarp-check [RFC PATCH 09/11] IB/Verbs: Use management helper has_ipoib() and cap_ipoib() for ipoib-check [RFC PATCH 10/11] IB/Verbs: Use management helper cap_ib() for ib-check
[RFC PATCH 11/11] IB/Verbs: Use management helper cap_eth() for eth-check
--- drivers/infiniband/core/agent.c | 6 drivers/infiniband/core/cm.c | 6 drivers/infiniband/core/cma.c | 79 ++++------ drivers/infiniband/core/mad.c | 18 +- drivers/infiniband/core/multicast.c | 30 +--- drivers/infiniband/core/sa_query.c | 42 ++--- drivers/infiniband/core/ucm.c | 8 - drivers/infiniband/core/user_mad.c | 6 drivers/infiniband/core/verbs.c | 14 - drivers/infiniband/hw/mlx4/ah.c | 6 drivers/infiniband/hw/mlx4/cq.c | 10 - drivers/infiniband/hw/mlx4/mad.c | 36 +--- drivers/infiniband/hw/mlx4/main.c | 23 +-- drivers/infiniband/hw/mlx4/mlx4_ib.h | 6 drivers/infiniband/hw/mlx4/qp.c | 57 ++----- drivers/infiniband/hw/mlx4/sysfs.c | 16 -- drivers/infiniband/ulp/ipoib/ipoib_main.c | 18 +- include/rdma/ib_verbs.h | 218 ++++++++++++++++++++++++++++++ net/sunrpc/xprtrdma/svc_rdma_recvfrom.c | 8 - 19 files changed, 381 insertions(+), 226 deletions(-)
--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
PATCH 1/11 IB/Verbs: Use helpers to check transport and link layer
We have so much places to check transport type and link layer type, it's now make sense to introduce some helpers in order to refine the lengthy code.
This patch will introduce helpers:
rdma_transport_is_ib()
rdma_transport_is_iwarp()
rdma_port_ll_is_ib()
rdma_port_ll_is_eth()and use them to save some code for us.
--- drivers/infiniband/core/agent.c | 2 +- drivers/infiniband/core/cm.c | 2 +- drivers/infiniband/core/cma.c | 27 ++++++++++++--------------- drivers/infiniband/core/mad.c | 6 +++--- drivers/infiniband/core/multicast.c | 11 ++++------- drivers/infiniband/core/sa_query.c | 14 +++++++------- drivers/infiniband/core/ucm.c | 3 +-- drivers/infiniband/core/user_mad.c | 2 +- drivers/infiniband/core/verbs.c | 5 ++--- drivers/infiniband/hw/mlx4/ah.c | 2 +- drivers/infiniband/hw/mlx4/cq.c | 4 +--- drivers/infiniband/hw/mlx4/mad.c | 14 ++++---------- drivers/infiniband/hw/mlx4/main.c | 8 +++----- drivers/infiniband/hw/mlx4/mlx4_ib.h | 2 +- drivers/infiniband/hw/mlx4/qp.c | 21 +++++++-------------- drivers/infiniband/hw/mlx4/sysfs.c | 6 ++---- drivers/infiniband/ulp/ipoib/ipoib_main.c | 6 +++--- include/rdma/ib_verbs.h | 24 ++++++++++++++++++++++++ net/sunrpc/xprtrdma/svc_rdma_recvfrom.c | 3 +-- 19 files changed, 79 insertions(+), 83 deletions(-)
drivers/infiniband/core/agent.c +1 -1
@@ -156,7 +156,7 @@ int ib_agent_port_open(struct ib_device *device, int port_num) goto error1; } -if (rdma_port_get_link_layer (device, port_num) ==IB_LINK_LAYER_INFINIBAND ) { +if (rdma_port_ll_is_ib (device, port_num)) {/* Obtain send only MAD agent for SMI QP */ port_priv->agent[0] = ib_register_mad_agent(device, port_num, IB_QPT_SMI, NULL, 0,
drivers/infiniband/core/cm.c +1 -1
@@ -3762,7 +3762,7 @@ static void cm_add_one(struct ib_device *ib_device)int ret;u8 i; -if (rdma_node_get_transport (ib_device->node_type) !=RDMA_TRANSPORT_IB ) +if (!rdma_transport_is_ib (ib_device))return ; cm_dev = kzalloc(sizeof(*cm_dev) + sizeof(*port) *
drivers/infiniband/core/cma.c +12 -15
@@ -375,8 +375,8 @@ static int cma_acquire_dev(struct rdma_id_private *id_priv, listen_id_priv->id.port_num) == dev_ll) { cma_dev = listen_id_priv->cma_dev; port = listen_id_priv->id.port_num; -if (rdma_node_get_transport (cma_dev->device->node_type) ==RDMA_TRANSPORT_IB && -rdma_port_get_link_layer (cma_dev->device, port) ==IB_LINK_LAYER_ETHERNET ) +if (rdma_transport_is_ib (cma_dev->device) && +rdma_port_ll_is_eth (cma_dev->device, port)) ret =ib_find_cached_gid (cma_dev->device, &iboe_gid, &found_port,NULL );else
@@ -395,8 +395,8 @@ static int cma_acquire_dev(struct rdma_id_private *id_priv, listen_id_priv->id.port_num == port)continue ;if (rdma_port_get_link_layer (cma_dev->device, port) == dev_ll) { -if (rdma_node_get_transport (cma_dev->device->node_type) ==RDMA_TRANSPORT_IB && -rdma_port_get_link_layer (cma_dev->device, port) ==IB_LINK_LAYER_ETHERNET ) +if (rdma_transport_is_ib (cma_dev->device) && +rdma_port_ll_is_eth (cma_dev->device, port)) ret =ib_find_cached_gid (cma_dev->device, &iboe_gid, &found_port,NULL );else ret =ib_find_cached_gid (cma_dev->device, &gid, &found_port,NULL );
@@ -435,7 +435,7 @@ static int cma_resolve_ib_dev(struct rdma_id_private *id_priv) pkey =ntohs (addr->sib_pkey);list_for_each_entry (cur_dev, &dev_list, list) { -if (rdma_node_get_transport (cur_dev->device->node_type) !=RDMA_TRANSPORT_IB ) +if (!rdma_transport_is_ib (cur_dev->device))continue ;for (p =1 ; p <= cur_dev->device->phys_port_cnt; ++p) {
@@ -633,10 +633,8 @@ static int cma_modify_qp_rtr(struct rdma_id_private *id_priv,if (ret) goto out; -if (rdma_node_get_transport (id_priv->cma_dev->device->node_type) - ==RDMA_TRANSPORT_IB && -rdma_port_get_link_layer (id_priv->id.device, id_priv->id.port_num) - ==IB_LINK_LAYER_ETHERNET ) { +if (rdma_transport_is_ib (id_priv->cma_dev->device) && +rdma_port_ll_is_eth (id_priv->id.device, id_priv->id.port_num)) { ret =rdma_addr_find_smac_by_sgid (&sgid, qp_attr.smac,NULL );if (ret)
@@ -700,8 +698,7 @@ static int cma_ib_init_qp_attr(struct rdma_id_private *id_priv,int ret;u16 pkey; -if (rdma_port_get_link_layer (id_priv->id.device, id_priv->id.port_num) == -IB_LINK_LAYER_INFINIBAND ) +if (rdma_port_ll_is_ib (id_priv->id.device, id_priv->id.port_num)) pkey =ib_addr_get_pkey (dev_addr);else pkey =0xffff ;
@@ -1626,7 +1623,7 @@ static void cma_listen_on_dev(struct rdma_id_private *id_priv,int ret;if (cma_family (id_priv) ==AF_IB && -rdma_node_get_transport (cma_dev->device->node_type) !=RDMA_TRANSPORT_IB ) + !rdma_transport_is_ib (cma_dev->device))return ; id = rdma_create_id(cma_listen_handler, id_priv, id_priv->id.ps,
@@ -2028,7 +2025,7 @@ static int cma_bind_loopback(struct rdma_id_private *id_priv)mutex_lock (&lock);list_for_each_entry (cur_dev, &dev_list, list) {if (cma_family (id_priv) ==AF_IB && -rdma_node_get_transport (cur_dev->device->node_type) !=RDMA_TRANSPORT_IB ) + !rdma_transport_is_ib (cur_dev->device))continue ;if (!cma_dev)
@@ -2060,7 +2057,7 @@ port_found: goto out; id_priv->id.route.addr.dev_addr.dev_type = - (rdma_port_get_link_layer (cma_dev->device, p) ==IB_LINK_LAYER_INFINIBAND ) ? + (rdma_port_ll_is_ib (cma_dev->device, p)) ?ARPHRD_INFINIBAND :ARPHRD_ETHER ;rdma_addr_set_sgid (&id_priv->id.route.addr.dev_addr, &gid);
@@ -3405,7 +3402,7 @@ void rdma_leave_multicast(struct rdma_cm_id *id, struct sockaddr *addr)ib_detach_mcast (id->qp, &mc->multicast.ib->rec.mgid,be16_to_cpu (mc->multicast.ib->rec.mlid)); -if (rdma_node_get_transport (id_priv->cma_dev->device->node_type) ==RDMA_TRANSPORT_IB ) { +if (rdma_transport_is_ib (id_priv->cma_dev->device)) {switch (rdma_port_get_link_layer (id->device, id->port_num)) {case IB_LINK_LAYER_INFINIBAND :ib_sa_free_multicast (mc->multicast.ib);
drivers/infiniband/core/mad.c +3 -3
@@ -2938,7 +2938,7 @@ static int ib_mad_port_open(struct ib_device *device,init_mad_qp (port_priv, &port_priv->qp_info[1 ]); cq_size = mad_sendq_size + mad_recvq_size; - has_smi =rdma_port_get_link_layer (device, port_num) ==IB_LINK_LAYER_INFINIBAND ; + has_smi =rdma_port_ll_is_ib (device, port_num);if (has_smi) cq_size *=2 ;
@@ -3057,7 +3057,7 @@ static void ib_mad_init_device(struct ib_device *device) {int start, end, i; -if (rdma_node_get_transport (device->node_type) !=RDMA_TRANSPORT_IB ) +if (!rdma_transport_is_ib (device))return ;if (device->node_type ==RDMA_NODE_IB_SWITCH ) {
@@ -3102,7 +3102,7 @@ static void ib_mad_remove_device(struct ib_device *device) {int i, num_ports, cur_port; -if (rdma_node_get_transport (device->node_type) !=RDMA_TRANSPORT_IB ) +if (!rdma_transport_is_ib (device))return ;if (device->node_type ==RDMA_NODE_IB_SWITCH ) {
drivers/infiniband/core/multicast.c +4 -7
@@ -780,8 +780,7 @@ static void mcast_event_handler(struct ib_event_handler *handler,int index; dev = container_of(handler, struct mcast_device, event_handler); -if (rdma_port_get_link_layer (dev->device, event->element.port_num) != -IB_LINK_LAYER_INFINIBAND ) +if (!rdma_port_ll_is_ib (dev->device, event->element.port_num))return ; index = event->element.port_num - dev->start_port;
@@ -808,7 +807,7 @@ static void mcast_add_one(struct ib_device *device)int i;int count =0 ; -if (rdma_node_get_transport (device->node_type) !=RDMA_TRANSPORT_IB ) +if (!rdma_transport_is_ib (device))return ; dev = kmalloc(sizeof *dev + device->phys_port_cnt * sizeof *port,
@@ -824,8 +823,7 @@ static void mcast_add_one(struct ib_device *device) }for (i =0 ; i <= dev->end_port - dev->start_port; i++) { -if (rdma_port_get_link_layer (device, dev->start_port + i) != -IB_LINK_LAYER_INFINIBAND ) +if (!rdma_port_ll_is_ib (device, dev->start_port + i))continue ; port = &dev->port[i]; port->dev = dev;
@@ -863,8 +861,7 @@ static void mcast_remove_one(struct ib_device *device)flush_workqueue (mcast_wq);for (i =0 ; i <= dev->end_port - dev->start_port; i++) { -if (rdma_port_get_link_layer (device, dev->start_port + i) == -IB_LINK_LAYER_INFINIBAND ) { +if (rdma_port_ll_is_ib (device, dev->start_port + i)) { port = &dev->port[i];deref_port (port);wait_for_completion (&port->comp);
drivers/infiniband/core/sa_query.c +7 -7
@@ -450,7 +450,7 @@ static void ib_sa_event(struct ib_event_handler *handler, struct ib_event *eventstruct ib_sa_port *port = &sa_dev->port[event->element.port_num - sa_dev->start_port]; -if (rdma_port_get_link_layer (handler->device, port->port_num) !=IB_LINK_LAYER_INFINIBAND ) +if (!rdma_port_ll_is_ib (handler->device, port->port_num))return ;spin_lock_irqsave (&port->ah_lock, flags);
@@ -540,7 +540,7 @@ int ib_init_ah_from_path(struct ib_device *device, u8 port_num, ah_attr->port_num = port_num; ah_attr->static_rate = rec->rate; - force_grh =rdma_port_get_link_layer (device, port_num) ==IB_LINK_LAYER_ETHERNET ; + force_grh =rdma_port_ll_is_eth (device, port_num);if (rec->hop_limit >1 || force_grh) { ah_attr->ah_flags =IB_AH_GRH ;
@@ -1154,7 +1154,7 @@ static void ib_sa_add_one(struct ib_device *device)struct ib_sa_device *sa_dev;int s, e, i; -if (rdma_node_get_transport (device->node_type) !=RDMA_TRANSPORT_IB ) +if (!rdma_transport_is_ib (device))return ;if (device->node_type ==RDMA_NODE_IB_SWITCH )
@@ -1175,7 +1175,7 @@ static void ib_sa_add_one(struct ib_device *device)for (i =0 ; i <= e - s; ++i) {spin_lock_init (&sa_dev->port[i].ah_lock); -if (rdma_port_get_link_layer (device, i +1 ) !=IB_LINK_LAYER_INFINIBAND ) +if (!rdma_port_ll_is_ib (device, i +1 ))continue ; sa_dev->port[i].sm_ah =NULL ;
@@ -1205,14 +1205,14 @@ static void ib_sa_add_one(struct ib_device *device) goto err;for (i =0 ; i <= e - s; ++i) -if (rdma_port_get_link_layer (device, i +1 ) ==IB_LINK_LAYER_INFINIBAND ) +if (rdma_port_ll_is_ib (device, i +1 ))update_sm_ah (&sa_dev->port[i].update_task);return ; err:while (--i >=0 ) -if (rdma_port_get_link_layer (device, i +1 ) ==IB_LINK_LAYER_INFINIBAND ) +if (rdma_port_ll_is_ib (device, i +1 ))ib_unregister_mad_agent (sa_dev->port[i].agent);kfree (sa_dev);
@@ -1233,7 +1233,7 @@ static void ib_sa_remove_one(struct ib_device *device)flush_workqueue (ib_wq);for (i =0 ; i <= sa_dev->end_port - sa_dev->start_port; ++i) { -if (rdma_port_get_link_layer (device, i +1 ) ==IB_LINK_LAYER_INFINIBAND ) { +if (rdma_port_ll_is_ib (device, i +1 )) {ib_unregister_mad_agent (sa_dev->port[i].agent);if (sa_dev->port[i].sm_ah)kref_put (&sa_dev->port[i].sm_ah->ref, free_sm_ah);
drivers/infiniband/core/ucm.c +1 -2
@@ -1253,8 +1253,7 @@ static void ib_ucm_add_one(struct ib_device *device)dev_t base;struct ib_ucm_device *ucm_dev; -if (!device->alloc_ucontext || -rdma_node_get_transport (device->node_type) !=RDMA_TRANSPORT_IB ) +if (!device->alloc_ucontext || !rdma_transport_is_ib (device))return ; ucm_dev =kzalloc (sizeof *ucm_dev,GFP_KERNEL );
drivers/infiniband/core/user_mad.c +1 -1
@@ -1274,7 +1274,7 @@ static void ib_umad_add_one(struct ib_device *device)struct ib_umad_device *umad_dev;int s, e, i; -if (rdma_node_get_transport (device->node_type) !=RDMA_TRANSPORT_IB ) +if (!rdma_transport_is_ib (device))return ;if (device->node_type ==RDMA_NODE_IB_SWITCH )
drivers/infiniband/core/verbs.c +2 -3
@@ -198,8 +198,7 @@ int ib_init_ah_from_wc(struct ib_device *device, u8 port_num, struct ib_wc *wc,u32 flow_class;u16 gid_index;int ret; -int is_eth = (rdma_port_get_link_layer (device, port_num) == -IB_LINK_LAYER_ETHERNET ); +int is_eth = (rdma_port_ll_is_eth (device, port_num));memset (ah_attr,0 ,sizeof *ah_attr);if (is_eth) {
@@ -871,7 +870,7 @@ int ib_resolve_eth_l2_attrs(struct ib_qp *qp,union ib_gid sgid;if ((*qp_attr_mask &IB_QP_AV ) && - (rdma_port_get_link_layer (qp->device, qp_attr->ah_attr.port_num) ==IB_LINK_LAYER_ETHERNET )) { + (rdma_port_ll_is_eth (qp->device, qp_attr->ah_attr.port_num))) { ret =ib_query_gid (qp->device, qp_attr->ah_attr.port_num, qp_attr->ah_attr.grh.sgid_index, &sgid);if (ret)
drivers/infiniband/hw/mlx4/ah.c +1 -1
@@ -118,7 +118,7 @@ struct ib_ah *mlx4_ib_create_ah(struct ib_pd *pd, struct ib_ah_attr *ah_attr)if (!ah)return ERR_PTR (-ENOMEM ); -if (rdma_port_get_link_layer (pd->device, ah_attr->port_num) ==IB_LINK_LAYER_ETHERNET ) { +if (rdma_port_ll_is_eth (pd->device, ah_attr->port_num)) {if (!(ah_attr->ah_flags &IB_AH_GRH )) { ret =ERR_PTR (-EINVAL ); }else {
drivers/infiniband/hw/mlx4/cq.c +1 -3
@@ -789,9 +789,7 @@ repoll:break ; } - is_eth = (rdma_port_get_link_layer (wc->qp->device, - (*cur_qp)->port) == -IB_LINK_LAYER_ETHERNET ); + is_eth = (rdma_port_ll_is_eth (wc->qp->device, (*cur_qp)->port));if (mlx4_is_mfunc (to_mdev (cq->ibcq.device)->dev)) { if ((*cur_qp)->mlx4_ib_qp_type & (MLX4_IB_QPT_PROXY_SMI_OWNER |
drivers/infiniband/hw/mlx4/mad.c +4 -10
@@ -606,12 +606,7 @@ static int mlx4_ib_demux_mad(struct ib_device *ibdev, u8 port,int err;int slave;u8 *slave_id; -int is_eth =0 ; - -if (rdma_port_get_link_layer (ibdev, port) ==IB_LINK_LAYER_INFINIBAND ) - is_eth =0 ; -else - is_eth =1 ; +int is_eth =rdma_port_ll_is_eth (ibdev, port);if (is_eth) {if (!(wc->wc_flags &IB_WC_GRH )) {
@@ -1252,7 +1247,7 @@ out:static int get_slave_base_gid_ix (struct mlx4_ib_dev *dev,int slave,int port) { -if (rdma_port_get_link_layer (&dev->ib_dev, port) ==IB_LINK_LAYER_INFINIBAND ) +if (rdma_port_ll_is_ib (&dev->ib_dev, port))return slave;return mlx4_get_base_gid_ix (dev->dev, slave, port); }
@@ -1260,7 +1255,7 @@ static int get_slave_base_gid_ix(struct mlx4_ib_dev *dev, int slave, int port)static void fill_in_real_sgid_index (struct mlx4_ib_dev *dev,int slave,int port,struct ib_ah_attr *ah_attr) { -if (rdma_port_get_link_layer (&dev->ib_dev, port) ==IB_LINK_LAYER_INFINIBAND ) +if (rdma_port_ll_is_ib (&dev->ib_dev, port)) ah_attr->grh.sgid_index = slave;else ah_attr->grh.sgid_index +=get_slave_base_gid_ix (dev, slave, port);
@@ -1758,8 +1753,7 @@ static int create_pv_resources(struct ib_device *ibdev, int slave, int port, ctx->state =DEMUX_PV_STATE_STARTING ;/* have QP0 only if link layer is IB */ -if (rdma_port_get_link_layer (ibdev, ctx->port) == -IB_LINK_LAYER_INFINIBAND ) +if (rdma_port_ll_is_ib (ibdev, ctx->port)) ctx->has_smi =1 ;if (ctx->has_smi) {
drivers/infiniband/hw/mlx4/main.c +3 -5
@@ -482,7 +482,7 @@ static int iboe_query_gid(struct ib_device *ibdev, u8 port, int index,static int mlx4_ib_query_gid (struct ib_device *ibdev,u8 port,int index,union ib_gid *gid) { -if (rdma_port_get_link_layer (ibdev, port) ==IB_LINK_LAYER_INFINIBAND ) +if (rdma_port_ll_is_ib (ibdev, port))return __mlx4_ib_query_gid (ibdev, port, index, gid,0 );else return iboe_query_gid (ibdev, port, index, gid);
@@ -1801,8 +1801,7 @@ static int mlx4_ib_init_gid_table(struct mlx4_ib_dev *ibdev)int err =0 ;for (i =1 ; i <= ibdev->num_ports; ++i) { -if (rdma_port_get_link_layer (&ibdev->ib_dev, i) == -IB_LINK_LAYER_ETHERNET ) { +if (rdma_port_ll_is_eth (&ibdev->ib_dev, i)) { err =reset_gid_table (ibdev, i);if (err) goto out;
@@ -2554,8 +2553,7 @@ static void mlx4_ib_event(struct mlx4_dev *dev, void *ibdev_ptr,if (p > ibdev->num_ports)return ;if (mlx4_is_master (dev) && -rdma_port_get_link_layer (&ibdev->ib_dev, p) == -IB_LINK_LAYER_INFINIBAND ) { +rdma_port_ll_is_ib (&ibdev->ib_dev, p)) {mlx4_ib_invalidate_all_guid_record (ibdev, p); } ibev.event =IB_EVENT_PORT_ACTIVE ;
drivers/infiniband/hw/mlx4/mlx4_ib.h +1 -1
@@ -712,7 +712,7 @@ static inline bool mlx4_ib_ah_grh_present(struct mlx4_ib_ah *ah) {u8 port =be32_to_cpu (ah->av.ib.port_pd) >>24 &3 ; -if (rdma_port_get_link_layer (ah->ibah.device, port) ==IB_LINK_LAYER_ETHERNET ) +if (rdma_port_ll_is_eth (ah->ibah.device, port))return true;return !!(ah->av.ib.g_slid &0x80 );
drivers/infiniband/hw/mlx4/qp.c +7 -14
@@ -1248,8 +1248,7 @@ static int _mlx4_set_path(struct mlx4_ib_dev *dev, const struct ib_ah_attr *ah, u64 smac, u16 vlan_tag, struct mlx4_qp_path *path, struct mlx4_roce_smac_vlan_info *smac_info, u8 port) { - int is_eth = rdma_port_get_link_layer(&dev->ib_dev, port) == -IB_LINK_LAYER_ETHERNET ; + int is_eth = rdma_port_ll_is_eth(&dev->ib_dev, port);int vidx;int smac_index;int err;
@@ -1433,8 +1432,7 @@ static int __mlx4_ib_modify_qp(struct ib_qp *ibqp,/* APM is not supported under RoCE */ if (attr_mask &IB_QP_ALT_PATH && -rdma_port_get_link_layer (&dev->ib_dev, qp->port) == -IB_LINK_LAYER_ETHERNET ) +rdma_port_ll_is_eth (&dev->ib_dev, qp->port))return -ENOTSUPP ; context =kzalloc (sizeof *context,GFP_KERNEL );
@@ -1664,8 +1662,7 @@ static int __mlx4_ib_modify_qp(struct ib_qp *ibqp, context->pri_path.fl =0x80 ; context->pri_path.sched_queue |=MLX4_IB_DEFAULT_SCHED_QUEUE ; } -if (rdma_port_get_link_layer (&dev->ib_dev, qp->port) == -IB_LINK_LAYER_ETHERNET ) { +if (rdma_port_ll_is_eth (&dev->ib_dev, qp->port)) {if (qp->mlx4_ib_qp_type ==MLX4_IB_QPT_TUN_GSI || qp->mlx4_ib_qp_type ==MLX4_IB_QPT_GSI ) context->pri_path.feup =1 <<7 ;/* don't fsm */
@@ -1695,9 +1692,7 @@ static int __mlx4_ib_modify_qp(struct ib_qp *ibqp, }if (ibqp->qp_type ==IB_QPT_UD && (new_state ==IB_QPS_RTR )) { -int is_eth =rdma_port_get_link_layer ( - &dev->ib_dev, qp->port) == -IB_LINK_LAYER_ETHERNET ; +int is_eth =rdma_port_ll_is_eth (&dev->ib_dev, qp->port);if (is_eth) { context->pri_path.ackto =MLX4_IB_LINK_TYPE_ETH ; optpar |=MLX4_QP_OPTPAR_PRIMARY_ADDR_PATH ;
@@ -1927,8 +1922,7 @@ int mlx4_ib_modify_qp(struct ib_qp *ibqp, struct ib_qp_attr *attr, }if ((attr_mask &IB_QP_PORT ) && (ibqp->qp_type ==IB_QPT_RAW_PACKET ) && - (rdma_port_get_link_layer (&dev->ib_dev, attr->port_num) != -IB_LINK_LAYER_ETHERNET )) + !rdma_port_ll_is_eth (&dev->ib_dev, attr->port_num)) goto out;if (attr_mask &IB_QP_PKEY_INDEX ) {
@@ -2132,7 +2126,7 @@ static int build_mlx_header(struct mlx4_ib_sqp *sqp, struct ib_send_wr *wr,for (i =0 ; i < wr->num_sge; ++i) send_size += wr->sg_list[i].length; - is_eth =rdma_port_get_link_layer (sqp->qp.ibqp.device, sqp->qp.port) ==IB_LINK_LAYER_ETHERNET ; + is_eth =rdma_port_ll_is_eth (sqp->qp.ibqp.device, sqp->qp.port); is_grh =mlx4_ib_ah_grh_present (ah);if (is_eth) {if (mlx4_is_mfunc (to_mdev (ib_dev)->dev)) {
@@ -3029,8 +3023,7 @@ static void to_ib_ah_attr(struct mlx4_ib_dev *ibdev, struct ib_ah_attr *ib_ah_atif (ib_ah_attr->port_num ==0 || ib_ah_attr->port_num > dev->caps.num_ports)return ; - is_eth =rdma_port_get_link_layer (&ibdev->ib_dev, ib_ah_attr->port_num) == -IB_LINK_LAYER_ETHERNET ; + is_eth =rdma_port_ll_is_eth (&ibdev->ib_dev, ib_ah_attr->port_num);if (is_eth) ib_ah_attr->sl = ((path->sched_queue >>3 ) &0x7 ) | ((path->sched_queue &4 ) <<1 );
drivers/infiniband/hw/mlx4/sysfs.c +2 -4
@@ -610,8 +610,7 @@ static ssize_t sysfs_store_enable_smi_admin(struct device *dev,static int add_vf_smi_entries (struct mlx4_port *p) { -int is_eth =rdma_port_get_link_layer (&p->dev->ib_dev, p->port_num) == -IB_LINK_LAYER_ETHERNET ; +int is_eth =rdma_port_ll_is_eth (&p->dev->ib_dev, p->port_num);int ret;/* do not display entries if eth transport, or if master */
@@ -645,8 +644,7 @@ static int add_vf_smi_entries(struct mlx4_port *p)static void remove_vf_smi_entries (struct mlx4_port *p) { -int is_eth =rdma_port_get_link_layer (&p->dev->ib_dev, p->port_num) == -IB_LINK_LAYER_ETHERNET ; +int is_eth =rdma_port_ll_is_eth (&p->dev->ib_dev, p->port_num);if (is_eth || p->slave ==mlx4_master_func_num (p->dev->dev))return ;
drivers/infiniband/ulp/ipoib/ipoib_main.c +3 -3
@@ -1655,7 +1655,7 @@ static void ipoib_add_one(struct ib_device *device)struct ipoib_dev_priv *priv;int s, e, p; -if (rdma_node_get_transport (device->node_type) !=RDMA_TRANSPORT_IB ) +if (!rdma_transport_is_ib (device))return ; dev_list =kmalloc (sizeof *dev_list,GFP_KERNEL );
@@ -1673,7 +1673,7 @@ static void ipoib_add_one(struct ib_device *device) }for (p = s; p <= e; ++p) { -if (rdma_port_get_link_layer (device, p) !=IB_LINK_LAYER_INFINIBAND ) +if (!rdma_port_ll_is_ib (device, p))continue ; dev =ipoib_add_port ("ib%d" , device, p);if (!IS_ERR (dev)) {
@@ -1690,7 +1690,7 @@ static void ipoib_remove_one(struct ib_device *device)struct ipoib_dev_priv *priv, *tmp;struct list_head *dev_list; -if (rdma_node_get_transport (device->node_type) !=RDMA_TRANSPORT_IB ) +if (!rdma_transport_is_ib (device))return ; dev_list =ib_get_client_data (device, &ipoib_client);
include/rdma/ib_verbs.h +24 -0
@@ -1743,6 +1743,30 @@ int ib_query_port(struct ib_device *device,enum rdma_link_layer rdma_port_get_link_layer (struct ib_device *device,u8 port_num); +static inline int rdma_transport_is_ib (struct ib_device *device) +{ +return rdma_node_get_transport (device->node_type) + ==RDMA_TRANSPORT_IB ; +} + +static inline int rdma_transport_is_iwarp (struct ib_device *device) +{ +return rdma_node_get_transport (device->node_type) + ==RDMA_TRANSPORT_IWARP ; +} + +static inline int rdma_port_ll_is_ib (struct ib_device *device,u8 port_num) +{ +return rdma_port_get_link_layer (device, port_num) + ==IB_LINK_LAYER_INFINIBAND ; +} + +static inline int rdma_port_ll_is_eth (struct ib_device *device,u8 port_num) +{ +return rdma_port_get_link_layer (device, port_num) + ==IB_LINK_LAYER_ETHERNET ; +} +int ib_query_gid (struct ib_device *device,u8 port_num,int index,union ib_gid *gid);
net/sunrpc/xprtrdma/svc_rdma_recvfrom.c +1 -2
@@ -118,8 +118,7 @@ static void rdma_build_arg_xdr(struct svc_rqst *rqstp,static int rdma_read_max_sge (struct svcxprt_rdma *xprt,int sge_count) { -if (rdma_node_get_transport (xprt->sc_cm_id->device->node_type) == -RDMA_TRANSPORT_IWARP ) +if (rdma_transport_is_iwarp (xprt->sc_cm_id->device))return 1 ;else return min_t (int, sge_count, xprt->sc_max_sge);
Signature · 6 lines
2.1.0 -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 01/11] IB/Verbs: Use helpers to check transport and link layer
Quoted 9 lines · Michael Wang
We have so much places to check transport type and link layer type, it's now make sense to introduce some helpers in order to refine the lengthy code.
This patch will introduce helpers:
rdma_transport_is_ib() rdma_transport_is_iwarp() rdma_port_ll_is_ib() rdma_port_ll_is_eth()and use them to save some code for us.
If the end result is to do something like I proposed, then why take this intermediate step that just has to be backed out later?
In other words, if our end goal is to have
rdma_transport_is_ib()rdma_transport_is_iwarp()rdma_transport_is_roce()rdma_transport_is_opa()
Then we should skip doing rdma_port_ll_is_*() as the answers to these items would be implied by rdma_transport_is_roce() and such.
Quoted 635 lines
Cc Jason Gunthorpe <[email protected]>Cc Doug Ledford <[email protected]>Cc Ira Weiny <[email protected]>Cc Sean Hefty <[email protected]>Signed-off-by Michael Wang <[email protected]>--- drivers/infiniband/core/agent.c | 2 +- drivers/infiniband/core/cm.c | 2 +- drivers/infiniband/core/cma.c | 27 ++++++++++++--------------- drivers/infiniband/core/mad.c | 6 +++--- drivers/infiniband/core/multicast.c | 11 ++++------- drivers/infiniband/core/sa_query.c | 14 +++++++------- drivers/infiniband/core/ucm.c | 3 +-- drivers/infiniband/core/user_mad.c | 2 +- drivers/infiniband/core/verbs.c | 5 ++--- drivers/infiniband/hw/mlx4/ah.c | 2 +- drivers/infiniband/hw/mlx4/cq.c | 4 +--- drivers/infiniband/hw/mlx4/mad.c | 14 ++++---------- drivers/infiniband/hw/mlx4/main.c | 8 +++----- drivers/infiniband/hw/mlx4/mlx4_ib.h | 2 +- drivers/infiniband/hw/mlx4/qp.c | 21 +++++++-------------- drivers/infiniband/hw/mlx4/sysfs.c | 6 ++---- drivers/infiniband/ulp/ipoib/ipoib_main.c | 6 +++--- include/rdma/ib_verbs.h | 24 ++++++++++++++++++++++++ net/sunrpc/xprtrdma/svc_rdma_recvfrom.c | 3 +-- 19 files changed, 79 insertions(+), 83 deletions(-)drivers/infiniband/core/agent.c +1
-1@@ -156,7 +156,7 @@ int ib_agent_port_open(struct ib_device *device, int port_num) goto error1; } -if (rdma_port_get_link_layer (device, port_num) ==IB_LINK_LAYER_INFINIBAND ) { +if (rdma_port_ll_is_ib (device, port_num)) {/* Obtain send only MAD agent for SMI QP */ port_priv->agent[0] = ib_register_mad_agent(device, port_num, IB_QPT_SMI, NULL, 0,drivers/infiniband/core/cm.c +1
-1@@ -3762,7 +3762,7 @@ static void cm_add_one(struct ib_device *ib_device)int ret;u8 i; -if (rdma_node_get_transport (ib_device->node_type) !=RDMA_TRANSPORT_IB ) +if (!rdma_transport_is_ib (ib_device))return ; cm_dev = kzalloc(sizeof(*cm_dev) + sizeof(*port) *drivers/infiniband/core/cma.c +12
-15@@ -375,8 +375,8 @@ static int cma_acquire_dev(struct rdma_id_private *id_priv, listen_id_priv->id.port_num) == dev_ll) { cma_dev = listen_id_priv->cma_dev; port = listen_id_priv->id.port_num; -if (rdma_node_get_transport (cma_dev->device->node_type) ==RDMA_TRANSPORT_IB && -rdma_port_get_link_layer (cma_dev->device, port) ==IB_LINK_LAYER_ETHERNET ) +if (rdma_transport_is_ib (cma_dev->device) && +rdma_port_ll_is_eth (cma_dev->device, port)) ret =ib_find_cached_gid (cma_dev->device, &iboe_gid, &found_port,NULL );else @@ -395,8 +395,8 @@ static int cma_acquire_dev(struct rdma_id_private *id_priv, listen_id_priv->id.port_num == port)continue ;if (rdma_port_get_link_layer (cma_dev->device, port) == dev_ll) { -if (rdma_node_get_transport (cma_dev->device->node_type) ==RDMA_TRANSPORT_IB && -rdma_port_get_link_layer (cma_dev->device, port) ==IB_LINK_LAYER_ETHERNET ) +if (rdma_transport_is_ib (cma_dev->device) && +rdma_port_ll_is_eth (cma_dev->device, port)) ret =ib_find_cached_gid (cma_dev->device, &iboe_gid, &found_port,NULL );else ret =ib_find_cached_gid (cma_dev->device, &gid, &found_port,NULL );@@ -435,7 +435,7 @@ static int cma_resolve_ib_dev(struct rdma_id_private *id_priv) pkey =ntohs (addr->sib_pkey);list_for_each_entry (cur_dev, &dev_list, list) { -if (rdma_node_get_transport (cur_dev->device->node_type) !=RDMA_TRANSPORT_IB ) +if (!rdma_transport_is_ib (cur_dev->device))continue ;for (p =1 ; p <= cur_dev->device->phys_port_cnt; ++p) {@@ -633,10 +633,8 @@ static int cma_modify_qp_rtr(struct rdma_id_private *id_priv,if (ret) goto out; -if (rdma_node_get_transport (id_priv->cma_dev->device->node_type) - ==RDMA_TRANSPORT_IB && -rdma_port_get_link_layer (id_priv->id.device, id_priv->id.port_num) - ==IB_LINK_LAYER_ETHERNET ) { +if (rdma_transport_is_ib (id_priv->cma_dev->device) && +rdma_port_ll_is_eth (id_priv->id.device, id_priv->id.port_num)) { ret =rdma_addr_find_smac_by_sgid (&sgid, qp_attr.smac,NULL );if (ret)@@ -700,8 +698,7 @@ static int cma_ib_init_qp_attr(struct rdma_id_private *id_priv,int ret;u16 pkey; -if (rdma_port_get_link_layer (id_priv->id.device, id_priv->id.port_num) == -IB_LINK_LAYER_INFINIBAND ) +if (rdma_port_ll_is_ib (id_priv->id.device, id_priv->id.port_num)) pkey =ib_addr_get_pkey (dev_addr);else pkey =0xffff ;@@ -1626,7 +1623,7 @@ static void cma_listen_on_dev(struct rdma_id_private *id_priv,int ret;if (cma_family (id_priv) ==AF_IB && -rdma_node_get_transport (cma_dev->device->node_type) !=RDMA_TRANSPORT_IB ) + !rdma_transport_is_ib (cma_dev->device))return ; id = rdma_create_id(cma_listen_handler, id_priv, id_priv->id.ps,@@ -2028,7 +2025,7 @@ static int cma_bind_loopback(struct rdma_id_private *id_priv)mutex_lock (&lock);list_for_each_entry (cur_dev, &dev_list, list) {if (cma_family (id_priv) ==AF_IB && -rdma_node_get_transport (cur_dev->device->node_type) !=RDMA_TRANSPORT_IB ) + !rdma_transport_is_ib (cur_dev->device))continue ;if (!cma_dev)@@ -2060,7 +2057,7 @@ port_found: goto out; id_priv->id.route.addr.dev_addr.dev_type = - (rdma_port_get_link_layer (cma_dev->device, p) ==IB_LINK_LAYER_INFINIBAND ) ? + (rdma_port_ll_is_ib (cma_dev->device, p)) ?ARPHRD_INFINIBAND :ARPHRD_ETHER ;rdma_addr_set_sgid (&id_priv->id.route.addr.dev_addr, &gid);@@ -3405,7 +3402,7 @@ void rdma_leave_multicast(struct rdma_cm_id *id, struct sockaddr *addr)ib_detach_mcast (id->qp, &mc->multicast.ib->rec.mgid,be16_to_cpu (mc->multicast.ib->rec.mlid)); -if (rdma_node_get_transport (id_priv->cma_dev->device->node_type) ==RDMA_TRANSPORT_IB ) { +if (rdma_transport_is_ib (id_priv->cma_dev->device)) {switch (rdma_port_get_link_layer (id->device, id->port_num)) {case IB_LINK_LAYER_INFINIBAND :ib_sa_free_multicast (mc->multicast.ib);drivers/infiniband/core/mad.c +3
-3@@ -2938,7 +2938,7 @@ static int ib_mad_port_open(struct ib_device *device,init_mad_qp (port_priv, &port_priv->qp_info[1 ]); cq_size = mad_sendq_size + mad_recvq_size; - has_smi =rdma_port_get_link_layer (device, port_num) ==IB_LINK_LAYER_INFINIBAND ; + has_smi =rdma_port_ll_is_ib (device, port_num);if (has_smi) cq_size *=2 ;@@ -3057,7 +3057,7 @@ static void ib_mad_init_device(struct ib_device *device) {int start, end, i; -if (rdma_node_get_transport (device->node_type) !=RDMA_TRANSPORT_IB ) +if (!rdma_transport_is_ib (device))return ;if (device->node_type ==RDMA_NODE_IB_SWITCH ) {@@ -3102,7 +3102,7 @@ static void ib_mad_remove_device(struct ib_device *device) {int i, num_ports, cur_port; -if (rdma_node_get_transport (device->node_type) !=RDMA_TRANSPORT_IB ) +if (!rdma_transport_is_ib (device))return ;if (device->node_type ==RDMA_NODE_IB_SWITCH ) {drivers/infiniband/core/multicast.c +4
-7@@ -780,8 +780,7 @@ static void mcast_event_handler(struct ib_event_handler *handler,int index; dev = container_of(handler, struct mcast_device, event_handler); -if (rdma_port_get_link_layer (dev->device, event->element.port_num) != -IB_LINK_LAYER_INFINIBAND ) +if (!rdma_port_ll_is_ib (dev->device, event->element.port_num))return ; index = event->element.port_num - dev->start_port;@@ -808,7 +807,7 @@ static void mcast_add_one(struct ib_device *device)int i;int count =0 ; -if (rdma_node_get_transport (device->node_type) !=RDMA_TRANSPORT_IB ) +if (!rdma_transport_is_ib (device))return ; dev = kmalloc(sizeof *dev + device->phys_port_cnt * sizeof *port,@@ -824,8 +823,7 @@ static void mcast_add_one(struct ib_device *device) }for (i =0 ; i <= dev->end_port - dev->start_port; i++) { -if (rdma_port_get_link_layer (device, dev->start_port + i) != -IB_LINK_LAYER_INFINIBAND ) +if (!rdma_port_ll_is_ib (device, dev->start_port + i))continue ; port = &dev->port[i]; port->dev = dev;@@ -863,8 +861,7 @@ static void mcast_remove_one(struct ib_device *device)flush_workqueue (mcast_wq);for (i =0 ; i <= dev->end_port - dev->start_port; i++) { -if (rdma_port_get_link_layer (device, dev->start_port + i) == -IB_LINK_LAYER_INFINIBAND ) { +if (rdma_port_ll_is_ib (device, dev->start_port + i)) { port = &dev->port[i];deref_port (port);wait_for_completion (&port->comp);drivers/infiniband/core/sa_query.c +7
-7@@ -450,7 +450,7 @@ static void ib_sa_event(struct ib_event_handler *handler, struct ib_event *eventstruct ib_sa_port *port = &sa_dev->port[event->element.port_num - sa_dev->start_port]; -if (rdma_port_get_link_layer (handler->device, port->port_num) !=IB_LINK_LAYER_INFINIBAND ) +if (!rdma_port_ll_is_ib (handler->device, port->port_num))return ;spin_lock_irqsave (&port->ah_lock, flags);@@ -540,7 +540,7 @@ int ib_init_ah_from_path(struct ib_device *device, u8 port_num, ah_attr->port_num = port_num; ah_attr->static_rate = rec->rate; - force_grh =rdma_port_get_link_layer (device, port_num) ==IB_LINK_LAYER_ETHERNET ; + force_grh =rdma_port_ll_is_eth (device, port_num);if (rec->hop_limit >1 || force_grh) { ah_attr->ah_flags =IB_AH_GRH ;@@ -1154,7 +1154,7 @@ static void ib_sa_add_one(struct ib_device *device)struct ib_sa_device *sa_dev;int s, e, i; -if (rdma_node_get_transport (device->node_type) !=RDMA_TRANSPORT_IB ) +if (!rdma_transport_is_ib (device))return ;if (device->node_type ==RDMA_NODE_IB_SWITCH )@@ -1175,7 +1175,7 @@ static void ib_sa_add_one(struct ib_device *device)for (i =0 ; i <= e - s; ++i) {spin_lock_init (&sa_dev->port[i].ah_lock); -if (rdma_port_get_link_layer (device, i +1 ) !=IB_LINK_LAYER_INFINIBAND ) +if (!rdma_port_ll_is_ib (device, i +1 ))continue ; sa_dev->port[i].sm_ah =NULL ;@@ -1205,14 +1205,14 @@ static void ib_sa_add_one(struct ib_device *device) goto err;for (i =0 ; i <= e - s; ++i) -if (rdma_port_get_link_layer (device, i +1 ) ==IB_LINK_LAYER_INFINIBAND ) +if (rdma_port_ll_is_ib (device, i +1 ))update_sm_ah (&sa_dev->port[i].update_task);return ; err:while (--i >=0 ) -if (rdma_port_get_link_layer (device, i +1 ) ==IB_LINK_LAYER_INFINIBAND ) +if (rdma_port_ll_is_ib (device, i +1 ))ib_unregister_mad_agent (sa_dev->port[i].agent);kfree (sa_dev);@@ -1233,7 +1233,7 @@ static void ib_sa_remove_one(struct ib_device *device)flush_workqueue (ib_wq);for (i =0 ; i <= sa_dev->end_port - sa_dev->start_port; ++i) { -if (rdma_port_get_link_layer (device, i +1 ) ==IB_LINK_LAYER_INFINIBAND ) { +if (rdma_port_ll_is_ib (device, i +1 )) {ib_unregister_mad_agent (sa_dev->port[i].agent);if (sa_dev->port[i].sm_ah)kref_put (&sa_dev->port[i].sm_ah->ref, free_sm_ah);drivers/infiniband/core/ucm.c +1
-2@@ -1253,8 +1253,7 @@ static void ib_ucm_add_one(struct ib_device *device)dev_t base;struct ib_ucm_device *ucm_dev; -if (!device->alloc_ucontext || -rdma_node_get_transport (device->node_type) !=RDMA_TRANSPORT_IB ) +if (!device->alloc_ucontext || !rdma_transport_is_ib (device))return ; ucm_dev =kzalloc (sizeof *ucm_dev,GFP_KERNEL );drivers/infiniband/core/user_mad.c +1
-1@@ -1274,7 +1274,7 @@ static void ib_umad_add_one(struct ib_device *device)struct ib_umad_device *umad_dev;int s, e, i; -if (rdma_node_get_transport (device->node_type) !=RDMA_TRANSPORT_IB ) +if (!rdma_transport_is_ib (device))return ;if (device->node_type ==RDMA_NODE_IB_SWITCH )drivers/infiniband/core/verbs.c +2
-3@@ -198,8 +198,7 @@ int ib_init_ah_from_wc(struct ib_device *device, u8 port_num, struct ib_wc *wc,u32 flow_class;u16 gid_index;int ret; -int is_eth = (rdma_port_get_link_layer (device, port_num) == -IB_LINK_LAYER_ETHERNET ); +int is_eth = (rdma_port_ll_is_eth (device, port_num));memset (ah_attr,0 ,sizeof *ah_attr);if (is_eth) {@@ -871,7 +870,7 @@ int ib_resolve_eth_l2_attrs(struct ib_qp *qp,union ib_gid sgid;if ((*qp_attr_mask &IB_QP_AV ) && - (rdma_port_get_link_layer (qp->device, qp_attr->ah_attr.port_num) ==IB_LINK_LAYER_ETHERNET )) { + (rdma_port_ll_is_eth (qp->device, qp_attr->ah_attr.port_num))) { ret =ib_query_gid (qp->device, qp_attr->ah_attr.port_num, qp_attr->ah_attr.grh.sgid_index, &sgid);if (ret)drivers/infiniband/hw/mlx4/ah.c +1
-1@@ -118,7 +118,7 @@ struct ib_ah *mlx4_ib_create_ah(struct ib_pd *pd, struct ib_ah_attr *ah_attr)if (!ah)return ERR_PTR (-ENOMEM ); -if (rdma_port_get_link_layer (pd->device, ah_attr->port_num) ==IB_LINK_LAYER_ETHERNET ) { +if (rdma_port_ll_is_eth (pd->device, ah_attr->port_num)) {if (!(ah_attr->ah_flags &IB_AH_GRH )) { ret =ERR_PTR (-EINVAL ); }else {drivers/infiniband/hw/mlx4/cq.c +1
-3@@ -789,9 +789,7 @@ repoll:break ; } - is_eth = (rdma_port_get_link_layer (wc->qp->device, - (*cur_qp)->port) == -IB_LINK_LAYER_ETHERNET ); + is_eth = (rdma_port_ll_is_eth (wc->qp->device, (*cur_qp)->port));if (mlx4_is_mfunc (to_mdev (cq->ibcq.device)->dev)) { if ((*cur_qp)->mlx4_ib_qp_type & (MLX4_IB_QPT_PROXY_SMI_OWNER |drivers/infiniband/hw/mlx4/mad.c +4
-10@@ -606,12 +606,7 @@ static int mlx4_ib_demux_mad(struct ib_device *ibdev, u8 port,int err;int slave;u8 *slave_id; -int is_eth =0 ; - -if (rdma_port_get_link_layer (ibdev, port) ==IB_LINK_LAYER_INFINIBAND ) - is_eth =0 ; -else - is_eth =1 ; +int is_eth =rdma_port_ll_is_eth (ibdev, port);if (is_eth) {if (!(wc->wc_flags &IB_WC_GRH )) {@@ -1252,7 +1247,7 @@ out:static int get_slave_base_gid_ix (struct mlx4_ib_dev *dev,int slave,int port) { -if (rdma_port_get_link_layer (&dev->ib_dev, port) ==IB_LINK_LAYER_INFINIBAND ) +if (rdma_port_ll_is_ib (&dev->ib_dev, port))return slave;return mlx4_get_base_gid_ix (dev->dev, slave, port); }@@ -1260,7 +1255,7 @@ static int get_slave_base_gid_ix(struct mlx4_ib_dev *dev, int slave, int port)static void fill_in_real_sgid_index (struct mlx4_ib_dev *dev,int slave,int port,struct ib_ah_attr *ah_attr) { -if (rdma_port_get_link_layer (&dev->ib_dev, port) ==IB_LINK_LAYER_INFINIBAND ) +if (rdma_port_ll_is_ib (&dev->ib_dev, port)) ah_attr->grh.sgid_index = slave;else ah_attr->grh.sgid_index +=get_slave_base_gid_ix (dev, slave, port);@@ -1758,8 +1753,7 @@ static int create_pv_resources(struct ib_device *ibdev, int slave, int port, ctx->state =DEMUX_PV_STATE_STARTING ;/* have QP0 only if link layer is IB */ -if (rdma_port_get_link_layer (ibdev, ctx->port) == -IB_LINK_LAYER_INFINIBAND ) +if (rdma_port_ll_is_ib (ibdev, ctx->port)) ctx->has_smi =1 ;if (ctx->has_smi) {drivers/infiniband/hw/mlx4/main.c +3
-5@@ -482,7 +482,7 @@ static int iboe_query_gid(struct ib_device *ibdev, u8 port, int index,static int mlx4_ib_query_gid (struct ib_device *ibdev,u8 port,int index,union ib_gid *gid) { -if (rdma_port_get_link_layer (ibdev, port) ==IB_LINK_LAYER_INFINIBAND ) +if (rdma_port_ll_is_ib (ibdev, port))return __mlx4_ib_query_gid (ibdev, port, index, gid,0 );else return iboe_query_gid (ibdev, port, index, gid);@@ -1801,8 +1801,7 @@ static int mlx4_ib_init_gid_table(struct mlx4_ib_dev *ibdev)int err =0 ;for (i =1 ; i <= ibdev->num_ports; ++i) { -if (rdma_port_get_link_layer (&ibdev->ib_dev, i) == -IB_LINK_LAYER_ETHERNET ) { +if (rdma_port_ll_is_eth (&ibdev->ib_dev, i)) { err =reset_gid_table (ibdev, i);if (err) goto out;@@ -2554,8 +2553,7 @@ static void mlx4_ib_event(struct mlx4_dev *dev, void *ibdev_ptr,if (p > ibdev->num_ports)return ;if (mlx4_is_master (dev) && -rdma_port_get_link_layer (&ibdev->ib_dev, p) == -IB_LINK_LAYER_INFINIBAND ) { +rdma_port_ll_is_ib (&ibdev->ib_dev, p)) {mlx4_ib_invalidate_all_guid_record (ibdev, p); } ibev.event =IB_EVENT_PORT_ACTIVE ;drivers/infiniband/hw/mlx4/mlx4_ib.h +1
-1@@ -712,7 +712,7 @@ static inline bool mlx4_ib_ah_grh_present(struct mlx4_ib_ah *ah) {u8 port =be32_to_cpu (ah->av.ib.port_pd) >>24 &3 ; -if (rdma_port_get_link_layer (ah->ibah.device, port) ==IB_LINK_LAYER_ETHERNET ) +if (rdma_port_ll_is_eth (ah->ibah.device, port))return true;return !!(ah->av.ib.g_slid &0x80 );drivers/infiniband/hw/mlx4/qp.c +7
-14@@ -1248,8 +1248,7 @@ static int _mlx4_set_path(struct mlx4_ib_dev *dev, const struct ib_ah_attr *ah, u64 smac, u16 vlan_tag, struct mlx4_qp_path *path, struct mlx4_roce_smac_vlan_info *smac_info, u8 port) { - int is_eth = rdma_port_get_link_layer(&dev->ib_dev, port) == -IB_LINK_LAYER_ETHERNET ; + int is_eth = rdma_port_ll_is_eth(&dev->ib_dev, port);int vidx;int smac_index;int err;@@ -1433,8 +1432,7 @@ static int __mlx4_ib_modify_qp(struct ib_qp *ibqp,/* APM is not supported under RoCE */ if (attr_mask &IB_QP_ALT_PATH && -rdma_port_get_link_layer (&dev->ib_dev, qp->port) == -IB_LINK_LAYER_ETHERNET ) +rdma_port_ll_is_eth (&dev->ib_dev, qp->port))return -ENOTSUPP ; context =kzalloc (sizeof *context,GFP_KERNEL );@@ -1664,8 +1662,7 @@ static int __mlx4_ib_modify_qp(struct ib_qp *ibqp, context->pri_path.fl =0x80 ; context->pri_path.sched_queue |=MLX4_IB_DEFAULT_SCHED_QUEUE ; } -if (rdma_port_get_link_layer (&dev->ib_dev, qp->port) == -IB_LINK_LAYER_ETHERNET ) { +if (rdma_port_ll_is_eth (&dev->ib_dev, qp->port)) {if (qp->mlx4_ib_qp_type ==MLX4_IB_QPT_TUN_GSI || qp->mlx4_ib_qp_type ==MLX4_IB_QPT_GSI ) context->pri_path.feup =1 <<7 ;/* don't fsm */ @@ -1695,9 +1692,7 @@ static int __mlx4_ib_modify_qp(struct ib_qp *ibqp, }if (ibqp->qp_type ==IB_QPT_UD && (new_state ==IB_QPS_RTR )) { -int is_eth =rdma_port_get_link_layer ( - &dev->ib_dev, qp->port) == -IB_LINK_LAYER_ETHERNET ; +int is_eth =rdma_port_ll_is_eth (&dev->ib_dev, qp->port);if (is_eth) { context->pri_path.ackto =MLX4_IB_LINK_TYPE_ETH ; optpar |=MLX4_QP_OPTPAR_PRIMARY_ADDR_PATH ;@@ -1927,8 +1922,7 @@ int mlx4_ib_modify_qp(struct ib_qp *ibqp, struct ib_qp_attr *attr, }if ((attr_mask &IB_QP_PORT ) && (ibqp->qp_type ==IB_QPT_RAW_PACKET ) && - (rdma_port_get_link_layer (&dev->ib_dev, attr->port_num) != -IB_LINK_LAYER_ETHERNET )) + !rdma_port_ll_is_eth (&dev->ib_dev, attr->port_num)) goto out;if (attr_mask &IB_QP_PKEY_INDEX ) {@@ -2132,7 +2126,7 @@ static int build_mlx_header(struct mlx4_ib_sqp *sqp, struct ib_send_wr *wr,for (i =0 ; i < wr->num_sge; ++i) send_size += wr->sg_list[i].length; - is_eth =rdma_port_get_link_layer (sqp->qp.ibqp.device, sqp->qp.port) ==IB_LINK_LAYER_ETHERNET ; + is_eth =rdma_port_ll_is_eth (sqp->qp.ibqp.device, sqp->qp.port); is_grh =mlx4_ib_ah_grh_present (ah);if (is_eth) {if (mlx4_is_mfunc (to_mdev (ib_dev)->dev)) {@@ -3029,8 +3023,7 @@ static void to_ib_ah_attr(struct mlx4_ib_dev *ibdev, struct ib_ah_attr *ib_ah_atif (ib_ah_attr->port_num ==0 || ib_ah_attr->port_num > dev->caps.num_ports)return ; - is_eth =rdma_port_get_link_layer (&ibdev->ib_dev, ib_ah_attr->port_num) == -IB_LINK_LAYER_ETHERNET ; + is_eth =rdma_port_ll_is_eth (&ibdev->ib_dev, ib_ah_attr->port_num);if (is_eth) ib_ah_attr->sl = ((path->sched_queue >>3 ) &0x7 ) | ((path->sched_queue &4 ) <<1 );drivers/infiniband/hw/mlx4/sysfs.c +2
-4@@ -610,8 +610,7 @@ static ssize_t sysfs_store_enable_smi_admin(struct device *dev,static int add_vf_smi_entries (struct mlx4_port *p) { -int is_eth =rdma_port_get_link_layer (&p->dev->ib_dev, p->port_num) == -IB_LINK_LAYER_ETHERNET ; +int is_eth =rdma_port_ll_is_eth (&p->dev->ib_dev, p->port_num);int ret;/* do not display entries if eth transport, or if master */ @@ -645,8 +644,7 @@ static int add_vf_smi_entries(struct mlx4_port *p)static void remove_vf_smi_entries (struct mlx4_port *p) { -int is_eth =rdma_port_get_link_layer (&p->dev->ib_dev, p->port_num) == -IB_LINK_LAYER_ETHERNET ; +int is_eth =rdma_port_ll_is_eth (&p->dev->ib_dev, p->port_num);if (is_eth || p->slave ==mlx4_master_func_num (p->dev->dev))return ;drivers/infiniband/ulp/ipoib/ipoib_main.c +3
-3@@ -1655,7 +1655,7 @@ static void ipoib_add_one(struct ib_device *device)struct ipoib_dev_priv *priv;int s, e, p; -if (rdma_node_get_transport (device->node_type) !=RDMA_TRANSPORT_IB ) +if (!rdma_transport_is_ib (device))return ; dev_list =kmalloc (sizeof *dev_list,GFP_KERNEL );@@ -1673,7 +1673,7 @@ static void ipoib_add_one(struct ib_device *device) }for (p = s; p <= e; ++p) { -if (rdma_port_get_link_layer (device, p) !=IB_LINK_LAYER_INFINIBAND ) +if (!rdma_port_ll_is_ib (device, p))continue ; dev =ipoib_add_port ("ib%d" , device, p);if (!IS_ERR (dev)) {@@ -1690,7 +1690,7 @@ static void ipoib_remove_one(struct ib_device *device)struct ipoib_dev_priv *priv, *tmp;struct list_head *dev_list; -if (rdma_node_get_transport (device->node_type) !=RDMA_TRANSPORT_IB ) +if (!rdma_transport_is_ib (device))return ; dev_list =ib_get_client_data (device, &ipoib_client);include/rdma/ib_verbs.h +24
-0@@ -1743,6 +1743,30 @@ int ib_query_port(struct ib_device *device,enum rdma_link_layer rdma_port_get_link_layer (struct ib_device *device,u8 port_num); +static inline int rdma_transport_is_ib (struct ib_device *device) +{ +return rdma_node_get_transport (device->node_type) + ==RDMA_TRANSPORT_IB ; +} + +static inline int rdma_transport_is_iwarp (struct ib_device *device) +{ +return rdma_node_get_transport (device->node_type) + ==RDMA_TRANSPORT_IWARP ; +} + +static inline int rdma_port_ll_is_ib (struct ib_device *device,u8 port_num) +{ +return rdma_port_get_link_layer (device, port_num) + ==IB_LINK_LAYER_INFINIBAND ; +} + +static inline int rdma_port_ll_is_eth (struct ib_device *device,u8 port_num) +{ +return rdma_port_get_link_layer (device, port_num) + ==IB_LINK_LAYER_ETHERNET ; +} +int ib_query_gid (struct ib_device *device,u8 port_num,int index,union ib_gid *gid);net/sunrpc/xprtrdma/svc_rdma_recvfrom.c +1
-2@@ -118,8 +118,7 @@ static void rdma_build_arg_xdr(struct svc_rqst *rqstp,static int rdma_read_max_sge (struct svcxprt_rdma *xprt,int sge_count) { -if (rdma_node_get_transport (xprt->sc_cm_id->device->node_type) == -RDMA_TRANSPORT_IWARP ) +if (rdma_transport_is_iwarp (xprt->sc_cm_id->device))return 1 ;else return min_t (int, sge_count, xprt->sc_max_sge);
Signature · 2 lines
Doug Ledford <[email protected]> GPG KeyID: 0E572FDD
Re: [PATCH 01/11] IB/Verbs: Use helpers to check transport and link layer
Hi, Doug
Thanks for the comments :-)
Quoted 4 lines · Doug Ledford
Quoted 3 lines · Michael Wang
We have so much places to check transport type and link layer type, it's now make sense to introduce some helpers in order to refine the lengthy code.
Quoted 8 lines
Quoted 6 lines
This patch will introduce helpers:
rdma_transport_is_ib() rdma_transport_is_iwarp() rdma_port_ll_is_ib() rdma_port_ll_is_eth()and use them to save some code for us.
If the end result is to do something like I proposed, then why take this intermediate step that just has to be backed out later?
The problem is that I found there are still many places our new mechanism may could not take care, especially inside device driver, this is just try to collect the issues together as a basement so we can gradually eliminate them.
Sure if finally we do capture all the cases, we can just get rid of this one, but I guess it won't be that easy to directly jump into next stage :-P
As I could imaging, after this reform, next stage could be introducing the new mechanism without changing device driver, and the last stage is to asking vendor adapt their code into the new mechanism.
Quoted 10 lines
In other words, if our end goal is to have
rdma_transport_is_ib()rdma_transport_is_iwarp()rdma_transport_is_roce()rdma_transport_is_opa()Then we should skip doing rdma_port_ll_is_*() as the answers to these items would be implied by
rdma_transport_is_roce()and such.
Great if we achieved that ;-) but currently I just wondering maybe these helpers can only cover part of the cases where we check transport and link layer, there are still some cases we'll need the very rough helper to save some code and make things clean~
Regards,
Michael Wang
Quoted 628 lines
Quoted 628 lines
Cc Jason Gunthorpe <jgunthorpe-ePGOBjL8dl3ta4EC/59zMFaTQe2KTcn/@public.gmane.org>Cc Doug Ledford <[email protected]>Cc Ira Weiny <[email protected]>Cc Sean Hefty <[email protected]>Signed-off-by Michael Wang <[email protected]>--- drivers/infiniband/core/agent.c | 2 +- drivers/infiniband/core/cm.c | 2 +- drivers/infiniband/core/cma.c | 27 ++++++++++++--------------- drivers/infiniband/core/mad.c | 6 +++--- drivers/infiniband/core/multicast.c | 11 ++++------- drivers/infiniband/core/sa_query.c | 14 +++++++------- drivers/infiniband/core/ucm.c | 3 +-- drivers/infiniband/core/user_mad.c | 2 +- drivers/infiniband/core/verbs.c | 5 ++--- drivers/infiniband/hw/mlx4/ah.c | 2 +- drivers/infiniband/hw/mlx4/cq.c | 4 +--- drivers/infiniband/hw/mlx4/mad.c | 14 ++++---------- drivers/infiniband/hw/mlx4/main.c | 8 +++----- drivers/infiniband/hw/mlx4/mlx4_ib.h | 2 +- drivers/infiniband/hw/mlx4/qp.c | 21 +++++++-------------- drivers/infiniband/hw/mlx4/sysfs.c | 6 ++---- drivers/infiniband/ulp/ipoib/ipoib_main.c | 6 +++--- include/rdma/ib_verbs.h | 24 ++++++++++++++++++++++++ net/sunrpc/xprtrdma/svc_rdma_recvfrom.c | 3 +-- 19 files changed, 79 insertions(+), 83 deletions(-)drivers/infiniband/core/agent.c +1
-1@@ -156,7 +156,7 @@ int ib_agent_port_open(struct ib_device *device, int port_num) goto error1; } -if (rdma_port_get_link_layer (device, port_num) ==IB_LINK_LAYER_INFINIBAND ) { +if (rdma_port_ll_is_ib (device, port_num)) {/* Obtain send only MAD agent for SMI QP */ port_priv->agent[0] = ib_register_mad_agent(device, port_num, IB_QPT_SMI, NULL, 0,drivers/infiniband/core/cm.c +1
-1@@ -3762,7 +3762,7 @@ static void cm_add_one(struct ib_device *ib_device)int ret;u8 i; -if (rdma_node_get_transport (ib_device->node_type) !=RDMA_TRANSPORT_IB ) +if (!rdma_transport_is_ib (ib_device))return ; cm_dev = kzalloc(sizeof(*cm_dev) + sizeof(*port) *drivers/infiniband/core/cma.c +12
-15@@ -375,8 +375,8 @@ static int cma_acquire_dev(struct rdma_id_private *id_priv, listen_id_priv->id.port_num) == dev_ll) { cma_dev = listen_id_priv->cma_dev; port = listen_id_priv->id.port_num; -if (rdma_node_get_transport (cma_dev->device->node_type) ==RDMA_TRANSPORT_IB && -rdma_port_get_link_layer (cma_dev->device, port) ==IB_LINK_LAYER_ETHERNET ) +if (rdma_transport_is_ib (cma_dev->device) && +rdma_port_ll_is_eth (cma_dev->device, port)) ret =ib_find_cached_gid (cma_dev->device, &iboe_gid, &found_port,NULL );else @@ -395,8 +395,8 @@ static int cma_acquire_dev(struct rdma_id_private *id_priv, listen_id_priv->id.port_num == port)continue ;if (rdma_port_get_link_layer (cma_dev->device, port) == dev_ll) { -if (rdma_node_get_transport (cma_dev->device->node_type) ==RDMA_TRANSPORT_IB && -rdma_port_get_link_layer (cma_dev->device, port) ==IB_LINK_LAYER_ETHERNET ) +if (rdma_transport_is_ib (cma_dev->device) && +rdma_port_ll_is_eth (cma_dev->device, port)) ret =ib_find_cached_gid (cma_dev->device, &iboe_gid, &found_port,NULL );else ret =ib_find_cached_gid (cma_dev->device, &gid, &found_port,NULL );@@ -435,7 +435,7 @@ static int cma_resolve_ib_dev(struct rdma_id_private *id_priv) pkey =ntohs (addr->sib_pkey);list_for_each_entry (cur_dev, &dev_list, list) { -if (rdma_node_get_transport (cur_dev->device->node_type) !=RDMA_TRANSPORT_IB ) +if (!rdma_transport_is_ib (cur_dev->device))continue ;for (p =1 ; p <= cur_dev->device->phys_port_cnt; ++p) {@@ -633,10 +633,8 @@ static int cma_modify_qp_rtr(struct rdma_id_private *id_priv,if (ret) goto out; -if (rdma_node_get_transport (id_priv->cma_dev->device->node_type) - ==RDMA_TRANSPORT_IB && -rdma_port_get_link_layer (id_priv->id.device, id_priv->id.port_num) - ==IB_LINK_LAYER_ETHERNET ) { +if (rdma_transport_is_ib (id_priv->cma_dev->device) && +rdma_port_ll_is_eth (id_priv->id.device, id_priv->id.port_num)) { ret =rdma_addr_find_smac_by_sgid (&sgid, qp_attr.smac,NULL );if (ret)@@ -700,8 +698,7 @@ static int cma_ib_init_qp_attr(struct rdma_id_private *id_priv,int ret;u16 pkey; -if (rdma_port_get_link_layer (id_priv->id.device, id_priv->id.port_num) == -IB_LINK_LAYER_INFINIBAND ) +if (rdma_port_ll_is_ib (id_priv->id.device, id_priv->id.port_num)) pkey =ib_addr_get_pkey (dev_addr);else pkey =0xffff ;@@ -1626,7 +1623,7 @@ static void cma_listen_on_dev(struct rdma_id_private *id_priv,int ret;if (cma_family (id_priv) ==AF_IB && -rdma_node_get_transport (cma_dev->device->node_type) !=RDMA_TRANSPORT_IB ) + !rdma_transport_is_ib (cma_dev->device))return ; id = rdma_create_id(cma_listen_handler, id_priv, id_priv->id.ps,@@ -2028,7 +2025,7 @@ static int cma_bind_loopback(struct rdma_id_private *id_priv)mutex_lock (&lock);list_for_each_entry (cur_dev, &dev_list, list) {if (cma_family (id_priv) ==AF_IB && -rdma_node_get_transport (cur_dev->device->node_type) !=RDMA_TRANSPORT_IB ) + !rdma_transport_is_ib (cur_dev->device))continue ;if (!cma_dev)@@ -2060,7 +2057,7 @@ port_found: goto out; id_priv->id.route.addr.dev_addr.dev_type = - (rdma_port_get_link_layer (cma_dev->device, p) ==IB_LINK_LAYER_INFINIBAND ) ? + (rdma_port_ll_is_ib (cma_dev->device, p)) ?ARPHRD_INFINIBAND :ARPHRD_ETHER ;rdma_addr_set_sgid (&id_priv->id.route.addr.dev_addr, &gid);@@ -3405,7 +3402,7 @@ void rdma_leave_multicast(struct rdma_cm_id *id, struct sockaddr *addr)ib_detach_mcast (id->qp, &mc->multicast.ib->rec.mgid,be16_to_cpu (mc->multicast.ib->rec.mlid)); -if (rdma_node_get_transport (id_priv->cma_dev->device->node_type) ==RDMA_TRANSPORT_IB ) { +if (rdma_transport_is_ib (id_priv->cma_dev->device)) {switch (rdma_port_get_link_layer (id->device, id->port_num)) {case IB_LINK_LAYER_INFINIBAND :ib_sa_free_multicast (mc->multicast.ib);drivers/infiniband/core/mad.c +3
-3@@ -2938,7 +2938,7 @@ static int ib_mad_port_open(struct ib_device *device,init_mad_qp (port_priv, &port_priv->qp_info[1 ]); cq_size = mad_sendq_size + mad_recvq_size; - has_smi =rdma_port_get_link_layer (device, port_num) ==IB_LINK_LAYER_INFINIBAND ; + has_smi =rdma_port_ll_is_ib (device, port_num);if (has_smi) cq_size *=2 ;@@ -3057,7 +3057,7 @@ static void ib_mad_init_device(struct ib_device *device) {int start, end, i; -if (rdma_node_get_transport (device->node_type) !=RDMA_TRANSPORT_IB ) +if (!rdma_transport_is_ib (device))return ;if (device->node_type ==RDMA_NODE_IB_SWITCH ) {@@ -3102,7 +3102,7 @@ static void ib_mad_remove_device(struct ib_device *device) {int i, num_ports, cur_port; -if (rdma_node_get_transport (device->node_type) !=RDMA_TRANSPORT_IB ) +if (!rdma_transport_is_ib (device))return ;if (device->node_type ==RDMA_NODE_IB_SWITCH ) {drivers/infiniband/core/multicast.c +4
-7@@ -780,8 +780,7 @@ static void mcast_event_handler(struct ib_event_handler *handler,int index; dev = container_of(handler, struct mcast_device, event_handler); -if (rdma_port_get_link_layer (dev->device, event->element.port_num) != -IB_LINK_LAYER_INFINIBAND ) +if (!rdma_port_ll_is_ib (dev->device, event->element.port_num))return ; index = event->element.port_num - dev->start_port;@@ -808,7 +807,7 @@ static void mcast_add_one(struct ib_device *device)int i;int count =0 ; -if (rdma_node_get_transport (device->node_type) !=RDMA_TRANSPORT_IB ) +if (!rdma_transport_is_ib (device))return ; dev = kmalloc(sizeof *dev + device->phys_port_cnt * sizeof *port,@@ -824,8 +823,7 @@ static void mcast_add_one(struct ib_device *device) }for (i =0 ; i <= dev->end_port - dev->start_port; i++) { -if (rdma_port_get_link_layer (device, dev->start_port + i) != -IB_LINK_LAYER_INFINIBAND ) +if (!rdma_port_ll_is_ib (device, dev->start_port + i))continue ; port = &dev->port[i]; port->dev = dev;@@ -863,8 +861,7 @@ static void mcast_remove_one(struct ib_device *device)flush_workqueue (mcast_wq);for (i =0 ; i <= dev->end_port - dev->start_port; i++) { -if (rdma_port_get_link_layer (device, dev->start_port + i) == -IB_LINK_LAYER_INFINIBAND ) { +if (rdma_port_ll_is_ib (device, dev->start_port + i)) { port = &dev->port[i];deref_port (port);wait_for_completion (&port->comp);drivers/infiniband/core/sa_query.c +7
-7@@ -450,7 +450,7 @@ static void ib_sa_event(struct ib_event_handler *handler, struct ib_event *eventstruct ib_sa_port *port = &sa_dev->port[event->element.port_num - sa_dev->start_port]; -if (rdma_port_get_link_layer (handler->device, port->port_num) !=IB_LINK_LAYER_INFINIBAND ) +if (!rdma_port_ll_is_ib (handler->device, port->port_num))return ;spin_lock_irqsave (&port->ah_lock, flags);@@ -540,7 +540,7 @@ int ib_init_ah_from_path(struct ib_device *device, u8 port_num, ah_attr->port_num = port_num; ah_attr->static_rate = rec->rate; - force_grh =rdma_port_get_link_layer (device, port_num) ==IB_LINK_LAYER_ETHERNET ; + force_grh =rdma_port_ll_is_eth (device, port_num);if (rec->hop_limit >1 || force_grh) { ah_attr->ah_flags =IB_AH_GRH ;@@ -1154,7 +1154,7 @@ static void ib_sa_add_one(struct ib_device *device)struct ib_sa_device *sa_dev;int s, e, i; -if (rdma_node_get_transport (device->node_type) !=RDMA_TRANSPORT_IB ) +if (!rdma_transport_is_ib (device))return ;if (device->node_type ==RDMA_NODE_IB_SWITCH )@@ -1175,7 +1175,7 @@ static void ib_sa_add_one(struct ib_device *device)for (i =0 ; i <= e - s; ++i) {spin_lock_init (&sa_dev->port[i].ah_lock); -if (rdma_port_get_link_layer (device, i +1 ) !=IB_LINK_LAYER_INFINIBAND ) +if (!rdma_port_ll_is_ib (device, i +1 ))continue ; sa_dev->port[i].sm_ah =NULL ;@@ -1205,14 +1205,14 @@ static void ib_sa_add_one(struct ib_device *device) goto err;for (i =0 ; i <= e - s; ++i) -if (rdma_port_get_link_layer (device, i +1 ) ==IB_LINK_LAYER_INFINIBAND ) +if (rdma_port_ll_is_ib (device, i +1 ))update_sm_ah (&sa_dev->port[i].update_task);return ; err:while (--i >=0 ) -if (rdma_port_get_link_layer (device, i +1 ) ==IB_LINK_LAYER_INFINIBAND ) +if (rdma_port_ll_is_ib (device, i +1 ))ib_unregister_mad_agent (sa_dev->port[i].agent);kfree (sa_dev);@@ -1233,7 +1233,7 @@ static void ib_sa_remove_one(struct ib_device *device)flush_workqueue (ib_wq);for (i =0 ; i <= sa_dev->end_port - sa_dev->start_port; ++i) { -if (rdma_port_get_link_layer (device, i +1 ) ==IB_LINK_LAYER_INFINIBAND ) { +if (rdma_port_ll_is_ib (device, i +1 )) {ib_unregister_mad_agent (sa_dev->port[i].agent);if (sa_dev->port[i].sm_ah)kref_put (&sa_dev->port[i].sm_ah->ref, free_sm_ah);drivers/infiniband/core/ucm.c +1
-2@@ -1253,8 +1253,7 @@ static void ib_ucm_add_one(struct ib_device *device)dev_t base;struct ib_ucm_device *ucm_dev; -if (!device->alloc_ucontext || -rdma_node_get_transport (device->node_type) !=RDMA_TRANSPORT_IB ) +if (!device->alloc_ucontext || !rdma_transport_is_ib (device))return ; ucm_dev =kzalloc (sizeof *ucm_dev,GFP_KERNEL );drivers/infiniband/core/user_mad.c +1
-1@@ -1274,7 +1274,7 @@ static void ib_umad_add_one(struct ib_device *device)struct ib_umad_device *umad_dev;int s, e, i; -if (rdma_node_get_transport (device->node_type) !=RDMA_TRANSPORT_IB ) +if (!rdma_transport_is_ib (device))return ;if (device->node_type ==RDMA_NODE_IB_SWITCH )drivers/infiniband/core/verbs.c +2
-3@@ -198,8 +198,7 @@ int ib_init_ah_from_wc(struct ib_device *device, u8 port_num, struct ib_wc *wc,u32 flow_class;u16 gid_index;int ret; -int is_eth = (rdma_port_get_link_layer (device, port_num) == -IB_LINK_LAYER_ETHERNET ); +int is_eth = (rdma_port_ll_is_eth (device, port_num));memset (ah_attr,0 ,sizeof *ah_attr);if (is_eth) {@@ -871,7 +870,7 @@ int ib_resolve_eth_l2_attrs(struct ib_qp *qp,union ib_gid sgid;if ((*qp_attr_mask &IB_QP_AV ) && - (rdma_port_get_link_layer (qp->device, qp_attr->ah_attr.port_num) ==IB_LINK_LAYER_ETHERNET )) { + (rdma_port_ll_is_eth (qp->device, qp_attr->ah_attr.port_num))) { ret =ib_query_gid (qp->device, qp_attr->ah_attr.port_num, qp_attr->ah_attr.grh.sgid_index, &sgid);if (ret)drivers/infiniband/hw/mlx4/ah.c +1
-1@@ -118,7 +118,7 @@ struct ib_ah *mlx4_ib_create_ah(struct ib_pd *pd, struct ib_ah_attr *ah_attr)if (!ah)return ERR_PTR (-ENOMEM ); -if (rdma_port_get_link_layer (pd->device, ah_attr->port_num) ==IB_LINK_LAYER_ETHERNET ) { +if (rdma_port_ll_is_eth (pd->device, ah_attr->port_num)) {if (!(ah_attr->ah_flags &IB_AH_GRH )) { ret =ERR_PTR (-EINVAL ); }else {drivers/infiniband/hw/mlx4/cq.c +1
-3@@ -789,9 +789,7 @@ repoll:break ; } - is_eth = (rdma_port_get_link_layer (wc->qp->device, - (*cur_qp)->port) == -IB_LINK_LAYER_ETHERNET ); + is_eth = (rdma_port_ll_is_eth (wc->qp->device, (*cur_qp)->port));if (mlx4_is_mfunc (to_mdev (cq->ibcq.device)->dev)) { if ((*cur_qp)->mlx4_ib_qp_type & (MLX4_IB_QPT_PROXY_SMI_OWNER |drivers/infiniband/hw/mlx4/mad.c +4
-10@@ -606,12 +606,7 @@ static int mlx4_ib_demux_mad(struct ib_device *ibdev, u8 port,int err;int slave;u8 *slave_id; -int is_eth =0 ; - -if (rdma_port_get_link_layer (ibdev, port) ==IB_LINK_LAYER_INFINIBAND ) - is_eth =0 ; -else - is_eth =1 ; +int is_eth =rdma_port_ll_is_eth (ibdev, port);if (is_eth) {if (!(wc->wc_flags &IB_WC_GRH )) {@@ -1252,7 +1247,7 @@ out:static int get_slave_base_gid_ix (struct mlx4_ib_dev *dev,int slave,int port) { -if (rdma_port_get_link_layer (&dev->ib_dev, port) ==IB_LINK_LAYER_INFINIBAND ) +if (rdma_port_ll_is_ib (&dev->ib_dev, port))return slave;return mlx4_get_base_gid_ix (dev->dev, slave, port); }@@ -1260,7 +1255,7 @@ static int get_slave_base_gid_ix(struct mlx4_ib_dev *dev, int slave, int port)static void fill_in_real_sgid_index (struct mlx4_ib_dev *dev,int slave,int port,struct ib_ah_attr *ah_attr) { -if (rdma_port_get_link_layer (&dev->ib_dev, port) ==IB_LINK_LAYER_INFINIBAND ) +if (rdma_port_ll_is_ib (&dev->ib_dev, port)) ah_attr->grh.sgid_index = slave;else ah_attr->grh.sgid_index +=get_slave_base_gid_ix (dev, slave, port);@@ -1758,8 +1753,7 @@ static int create_pv_resources(struct ib_device *ibdev, int slave, int port, ctx->state =DEMUX_PV_STATE_STARTING ;/* have QP0 only if link layer is IB */ -if (rdma_port_get_link_layer (ibdev, ctx->port) == -IB_LINK_LAYER_INFINIBAND ) +if (rdma_port_ll_is_ib (ibdev, ctx->port)) ctx->has_smi =1 ;if (ctx->has_smi) {drivers/infiniband/hw/mlx4/main.c +3
-5@@ -482,7 +482,7 @@ static int iboe_query_gid(struct ib_device *ibdev, u8 port, int index,static int mlx4_ib_query_gid (struct ib_device *ibdev,u8 port,int index,union ib_gid *gid) { -if (rdma_port_get_link_layer (ibdev, port) ==IB_LINK_LAYER_INFINIBAND ) +if (rdma_port_ll_is_ib (ibdev, port))return __mlx4_ib_query_gid (ibdev, port, index, gid,0 );else return iboe_query_gid (ibdev, port, index, gid);@@ -1801,8 +1801,7 @@ static int mlx4_ib_init_gid_table(struct mlx4_ib_dev *ibdev)int err =0 ;for (i =1 ; i <= ibdev->num_ports; ++i) { -if (rdma_port_get_link_layer (&ibdev->ib_dev, i) == -IB_LINK_LAYER_ETHERNET ) { +if (rdma_port_ll_is_eth (&ibdev->ib_dev, i)) { err =reset_gid_table (ibdev, i);if (err) goto out;@@ -2554,8 +2553,7 @@ static void mlx4_ib_event(struct mlx4_dev *dev, void *ibdev_ptr,if (p > ibdev->num_ports)return ;if (mlx4_is_master (dev) && -rdma_port_get_link_layer (&ibdev->ib_dev, p) == -IB_LINK_LAYER_INFINIBAND ) { +rdma_port_ll_is_ib (&ibdev->ib_dev, p)) {mlx4_ib_invalidate_all_guid_record (ibdev, p); } ibev.event =IB_EVENT_PORT_ACTIVE ;drivers/infiniband/hw/mlx4/mlx4_ib.h +1
-1@@ -712,7 +712,7 @@ static inline bool mlx4_ib_ah_grh_present(struct mlx4_ib_ah *ah) {u8 port =be32_to_cpu (ah->av.ib.port_pd) >>24 &3 ; -if (rdma_port_get_link_layer (ah->ibah.device, port) ==IB_LINK_LAYER_ETHERNET ) +if (rdma_port_ll_is_eth (ah->ibah.device, port))return true;return !!(ah->av.ib.g_slid &0x80 );drivers/infiniband/hw/mlx4/qp.c +7
-14@@ -1248,8 +1248,7 @@ static int _mlx4_set_path(struct mlx4_ib_dev *dev, const struct ib_ah_attr *ah, u64 smac, u16 vlan_tag, struct mlx4_qp_path *path, struct mlx4_roce_smac_vlan_info *smac_info, u8 port) { - int is_eth = rdma_port_get_link_layer(&dev->ib_dev, port) == -IB_LINK_LAYER_ETHERNET ; + int is_eth = rdma_port_ll_is_eth(&dev->ib_dev, port);int vidx;int smac_index;int err;@@ -1433,8 +1432,7 @@ static int __mlx4_ib_modify_qp(struct ib_qp *ibqp,/* APM is not supported under RoCE */ if (attr_mask &IB_QP_ALT_PATH && -rdma_port_get_link_layer (&dev->ib_dev, qp->port) == -IB_LINK_LAYER_ETHERNET ) +rdma_port_ll_is_eth (&dev->ib_dev, qp->port))return -ENOTSUPP ; context =kzalloc (sizeof *context,GFP_KERNEL );@@ -1664,8 +1662,7 @@ static int __mlx4_ib_modify_qp(struct ib_qp *ibqp, context->pri_path.fl =0x80 ; context->pri_path.sched_queue |=MLX4_IB_DEFAULT_SCHED_QUEUE ; } -if (rdma_port_get_link_layer (&dev->ib_dev, qp->port) == -IB_LINK_LAYER_ETHERNET ) { +if (rdma_port_ll_is_eth (&dev->ib_dev, qp->port)) {if (qp->mlx4_ib_qp_type ==MLX4_IB_QPT_TUN_GSI || qp->mlx4_ib_qp_type ==MLX4_IB_QPT_GSI ) context->pri_path.feup =1 <<7 ;/* don't fsm */ @@ -1695,9 +1692,7 @@ static int __mlx4_ib_modify_qp(struct ib_qp *ibqp, }if (ibqp->qp_type ==IB_QPT_UD && (new_state ==IB_QPS_RTR )) { -int is_eth =rdma_port_get_link_layer ( - &dev->ib_dev, qp->port) == -IB_LINK_LAYER_ETHERNET ; +int is_eth =rdma_port_ll_is_eth (&dev->ib_dev, qp->port);if (is_eth) { context->pri_path.ackto =MLX4_IB_LINK_TYPE_ETH ; optpar |=MLX4_QP_OPTPAR_PRIMARY_ADDR_PATH ;@@ -1927,8 +1922,7 @@ int mlx4_ib_modify_qp(struct ib_qp *ibqp, struct ib_qp_attr *attr, }if ((attr_mask &IB_QP_PORT ) && (ibqp->qp_type ==IB_QPT_RAW_PACKET ) && - (rdma_port_get_link_layer (&dev->ib_dev, attr->port_num) != -IB_LINK_LAYER_ETHERNET )) + !rdma_port_ll_is_eth (&dev->ib_dev, attr->port_num)) goto out;if (attr_mask &IB_QP_PKEY_INDEX ) {@@ -2132,7 +2126,7 @@ static int build_mlx_header(struct mlx4_ib_sqp *sqp, struct ib_send_wr *wr,for (i =0 ; i < wr->num_sge; ++i) send_size += wr->sg_list[i].length; - is_eth =rdma_port_get_link_layer (sqp->qp.ibqp.device, sqp->qp.port) ==IB_LINK_LAYER_ETHERNET ; + is_eth =rdma_port_ll_is_eth (sqp->qp.ibqp.device, sqp->qp.port); is_grh =mlx4_ib_ah_grh_present (ah);if (is_eth) {if (mlx4_is_mfunc (to_mdev (ib_dev)->dev)) {@@ -3029,8 +3023,7 @@ static void to_ib_ah_attr(struct mlx4_ib_dev *ibdev, struct ib_ah_attr *ib_ah_atif (ib_ah_attr->port_num ==0 || ib_ah_attr->port_num > dev->caps.num_ports)return ; - is_eth =rdma_port_get_link_layer (&ibdev->ib_dev, ib_ah_attr->port_num) == -IB_LINK_LAYER_ETHERNET ; + is_eth =rdma_port_ll_is_eth (&ibdev->ib_dev, ib_ah_attr->port_num);if (is_eth) ib_ah_attr->sl = ((path->sched_queue >>3 ) &0x7 ) | ((path->sched_queue &4 ) <<1 );drivers/infiniband/hw/mlx4/sysfs.c +2
-4@@ -610,8 +610,7 @@ static ssize_t sysfs_store_enable_smi_admin(struct device *dev,static int add_vf_smi_entries (struct mlx4_port *p) { -int is_eth =rdma_port_get_link_layer (&p->dev->ib_dev, p->port_num) == -IB_LINK_LAYER_ETHERNET ; +int is_eth =rdma_port_ll_is_eth (&p->dev->ib_dev, p->port_num);int ret;/* do not display entries if eth transport, or if master */ @@ -645,8 +644,7 @@ static int add_vf_smi_entries(struct mlx4_port *p)static void remove_vf_smi_entries (struct mlx4_port *p) { -int is_eth =rdma_port_get_link_layer (&p->dev->ib_dev, p->port_num) == -IB_LINK_LAYER_ETHERNET ; +int is_eth =rdma_port_ll_is_eth (&p->dev->ib_dev, p->port_num);if (is_eth || p->slave ==mlx4_master_func_num (p->dev->dev))return ;drivers/infiniband/ulp/ipoib/ipoib_main.c +3
-3@@ -1655,7 +1655,7 @@ static void ipoib_add_one(struct ib_device *device)struct ipoib_dev_priv *priv;int s, e, p; -if (rdma_node_get_transport (device->node_type) !=RDMA_TRANSPORT_IB ) +if (!rdma_transport_is_ib (device))return ; dev_list =kmalloc (sizeof *dev_list,GFP_KERNEL );@@ -1673,7 +1673,7 @@ static void ipoib_add_one(struct ib_device *device) }for (p = s; p <= e; ++p) { -if (rdma_port_get_link_layer (device, p) !=IB_LINK_LAYER_INFINIBAND ) +if (!rdma_port_ll_is_ib (device, p))continue ; dev =ipoib_add_port ("ib%d" , device, p);if (!IS_ERR (dev)) {@@ -1690,7 +1690,7 @@ static void ipoib_remove_one(struct ib_device *device)struct ipoib_dev_priv *priv, *tmp;struct list_head *dev_list; -if (rdma_node_get_transport (device->node_type) !=RDMA_TRANSPORT_IB ) +if (!rdma_transport_is_ib (device))return ; dev_list =ib_get_client_data (device, &ipoib_client);include/rdma/ib_verbs.h +24
-0@@ -1743,6 +1743,30 @@ int ib_query_port(struct ib_device *device,enum rdma_link_layer rdma_port_get_link_layer (struct ib_device *device,u8 port_num); +static inline int rdma_transport_is_ib (struct ib_device *device) +{ +return rdma_node_get_transport (device->node_type) + ==RDMA_TRANSPORT_IB ; +} + +static inline int rdma_transport_is_iwarp (struct ib_device *device) +{ +return rdma_node_get_transport (device->node_type) + ==RDMA_TRANSPORT_IWARP ; +} + +static inline int rdma_port_ll_is_ib (struct ib_device *device,u8 port_num) +{ +return rdma_port_get_link_layer (device, port_num) + ==IB_LINK_LAYER_INFINIBAND ; +} + +static inline int rdma_port_ll_is_eth (struct ib_device *device,u8 port_num) +{ +return rdma_port_get_link_layer (device, port_num) + ==IB_LINK_LAYER_ETHERNET ; +} +int ib_query_gid (struct ib_device *device,u8 port_num,int index,union ib_gid *gid);net/sunrpc/xprtrdma/svc_rdma_recvfrom.c +0
-0@@ -118,8 +118,7 @@ static void rdma_build_arg_xdr(struct svc_rqst *rqstp,
Quoted 9 lines
net/sunrpc/xprtrdma/svc_rdma_recvfrom.c static void rdma_build_arg_xdr(struct svc_rqst *rqstp, · line 124
Quoted 8 lines
net/sunrpc/xprtrdma/svc_rdma_recvfrom.c +1
-2static int rdma_read_max_sge (struct svcxprt_rdma *xprt,int sge_count) { -if (rdma_node_get_transport (xprt->sc_cm_id->device->node_type) == -RDMA_TRANSPORT_IWARP ) +if (rdma_transport_is_iwarp (xprt->sc_cm_id->device))return 1 ;else return min_t (int, sge_count, xprt->sc_max_sge);
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 01/11] IB/Verbs: Use helpers to check transport and link layer
Quoted 15 lines · Michael Wang
Hi, Doug
Thanks for the comments :-)
Quoted 10 lines · Doug Ledford
Quoted 9 lines · Michael Wang
We have so much places to check transport type and link layer type, it's now make sense to introduce some helpers in order to refine the lengthy code.
This patch will introduce helpers:
rdma_transport_is_ib() rdma_transport_is_iwarp() rdma_port_ll_is_ib() rdma_port_ll_is_eth()and use them to save some code for us.
Quoted 7 lines
Quoted 2 lines
If the end result is to do something like I proposed, then why take this intermediate step that just has to be backed out later?
The problem is that I found there are still many places our new mechanism may could not take care, especially inside device driver, this is just try to collect the issues together as a basement so we can gradually eliminate them.
There is no "gradually eliminate them" to the suggestion I made. Remember, my suggestion was to remove the transport and link_layer items from the port settings and replace it with just one transport item that is a bitmask of the possible transport types. This can not be done gradually, it must be a complete change all at once as the two methods of setting things are incompatible. As there is only one out of tree driver that I know of, lustre, we can give them the information they need to make their driver work both before and after the change.
Quoted 670 lines
Sure if finally we do capture all the cases, we can just get rid of this one, but I guess it won't be that easy to directly jump into next stage :-P
As I could imaging, after this reform, next stage could be introducing the new mechanism without changing device driver, and the last stage is to asking vendor adapt their code into the new mechanism.
Quoted 10 lines
In other words, if our end goal is to have
rdma_transport_is_ib()rdma_transport_is_iwarp()rdma_transport_is_roce()rdma_transport_is_opa()Then we should skip doing rdma_port_ll_is_*() as the answers to these items would be implied by
rdma_transport_is_roce()and such.Great if we achieved that ;-) but currently I just wondering maybe these helpers can only cover part of the cases where we check transport and link layer, there are still some cases we'll need the very rough helper to save some code and make things clean~
Regards,
Michael WangQuoted 628 lines
Quoted 628 lines
Cc Jason Gunthorpe <[email protected]>Cc Doug Ledford <[email protected]>Cc Ira Weiny <[email protected]>Cc Sean Hefty <[email protected]>Signed-off-by Michael Wang <[email protected]>--- drivers/infiniband/core/agent.c | 2 +- drivers/infiniband/core/cm.c | 2 +- drivers/infiniband/core/cma.c | 27 ++++++++++++--------------- drivers/infiniband/core/mad.c | 6 +++--- drivers/infiniband/core/multicast.c | 11 ++++------- drivers/infiniband/core/sa_query.c | 14 +++++++------- drivers/infiniband/core/ucm.c | 3 +-- drivers/infiniband/core/user_mad.c | 2 +- drivers/infiniband/core/verbs.c | 5 ++--- drivers/infiniband/hw/mlx4/ah.c | 2 +- drivers/infiniband/hw/mlx4/cq.c | 4 +--- drivers/infiniband/hw/mlx4/mad.c | 14 ++++---------- drivers/infiniband/hw/mlx4/main.c | 8 +++----- drivers/infiniband/hw/mlx4/mlx4_ib.h | 2 +- drivers/infiniband/hw/mlx4/qp.c | 21 +++++++-------------- drivers/infiniband/hw/mlx4/sysfs.c | 6 ++---- drivers/infiniband/ulp/ipoib/ipoib_main.c | 6 +++--- include/rdma/ib_verbs.h | 24 ++++++++++++++++++++++++ net/sunrpc/xprtrdma/svc_rdma_recvfrom.c | 3 +-- 19 files changed, 79 insertions(+), 83 deletions(-)drivers/infiniband/core/agent.c +1
-1@@ -156,7 +156,7 @@ int ib_agent_port_open(struct ib_device *device, int port_num) goto error1; } -if (rdma_port_get_link_layer (device, port_num) ==IB_LINK_LAYER_INFINIBAND ) { +if (rdma_port_ll_is_ib (device, port_num)) {/* Obtain send only MAD agent for SMI QP */ port_priv->agent[0] = ib_register_mad_agent(device, port_num, IB_QPT_SMI, NULL, 0,drivers/infiniband/core/cm.c +1
-1@@ -3762,7 +3762,7 @@ static void cm_add_one(struct ib_device *ib_device)int ret;u8 i; -if (rdma_node_get_transport (ib_device->node_type) !=RDMA_TRANSPORT_IB ) +if (!rdma_transport_is_ib (ib_device))return ; cm_dev = kzalloc(sizeof(*cm_dev) + sizeof(*port) *drivers/infiniband/core/cma.c +12
-15@@ -375,8 +375,8 @@ static int cma_acquire_dev(struct rdma_id_private *id_priv, listen_id_priv->id.port_num) == dev_ll) { cma_dev = listen_id_priv->cma_dev; port = listen_id_priv->id.port_num; -if (rdma_node_get_transport (cma_dev->device->node_type) ==RDMA_TRANSPORT_IB && -rdma_port_get_link_layer (cma_dev->device, port) ==IB_LINK_LAYER_ETHERNET ) +if (rdma_transport_is_ib (cma_dev->device) && +rdma_port_ll_is_eth (cma_dev->device, port)) ret =ib_find_cached_gid (cma_dev->device, &iboe_gid, &found_port,NULL );else @@ -395,8 +395,8 @@ static int cma_acquire_dev(struct rdma_id_private *id_priv, listen_id_priv->id.port_num == port)continue ;if (rdma_port_get_link_layer (cma_dev->device, port) == dev_ll) { -if (rdma_node_get_transport (cma_dev->device->node_type) ==RDMA_TRANSPORT_IB && -rdma_port_get_link_layer (cma_dev->device, port) ==IB_LINK_LAYER_ETHERNET ) +if (rdma_transport_is_ib (cma_dev->device) && +rdma_port_ll_is_eth (cma_dev->device, port)) ret =ib_find_cached_gid (cma_dev->device, &iboe_gid, &found_port,NULL );else ret =ib_find_cached_gid (cma_dev->device, &gid, &found_port,NULL );@@ -435,7 +435,7 @@ static int cma_resolve_ib_dev(struct rdma_id_private *id_priv) pkey =ntohs (addr->sib_pkey);list_for_each_entry (cur_dev, &dev_list, list) { -if (rdma_node_get_transport (cur_dev->device->node_type) !=RDMA_TRANSPORT_IB ) +if (!rdma_transport_is_ib (cur_dev->device))continue ;for (p =1 ; p <= cur_dev->device->phys_port_cnt; ++p) {@@ -633,10 +633,8 @@ static int cma_modify_qp_rtr(struct rdma_id_private *id_priv,if (ret) goto out; -if (rdma_node_get_transport (id_priv->cma_dev->device->node_type) - ==RDMA_TRANSPORT_IB && -rdma_port_get_link_layer (id_priv->id.device, id_priv->id.port_num) - ==IB_LINK_LAYER_ETHERNET ) { +if (rdma_transport_is_ib (id_priv->cma_dev->device) && +rdma_port_ll_is_eth (id_priv->id.device, id_priv->id.port_num)) { ret =rdma_addr_find_smac_by_sgid (&sgid, qp_attr.smac,NULL );if (ret)@@ -700,8 +698,7 @@ static int cma_ib_init_qp_attr(struct rdma_id_private *id_priv,int ret;u16 pkey; -if (rdma_port_get_link_layer (id_priv->id.device, id_priv->id.port_num) == -IB_LINK_LAYER_INFINIBAND ) +if (rdma_port_ll_is_ib (id_priv->id.device, id_priv->id.port_num)) pkey =ib_addr_get_pkey (dev_addr);else pkey =0xffff ;@@ -1626,7 +1623,7 @@ static void cma_listen_on_dev(struct rdma_id_private *id_priv,int ret;if (cma_family (id_priv) ==AF_IB && -rdma_node_get_transport (cma_dev->device->node_type) !=RDMA_TRANSPORT_IB ) + !rdma_transport_is_ib (cma_dev->device))return ; id = rdma_create_id(cma_listen_handler, id_priv, id_priv->id.ps,@@ -2028,7 +2025,7 @@ static int cma_bind_loopback(struct rdma_id_private *id_priv)mutex_lock (&lock);list_for_each_entry (cur_dev, &dev_list, list) {if (cma_family (id_priv) ==AF_IB && -rdma_node_get_transport (cur_dev->device->node_type) !=RDMA_TRANSPORT_IB ) + !rdma_transport_is_ib (cur_dev->device))continue ;if (!cma_dev)@@ -2060,7 +2057,7 @@ port_found: goto out; id_priv->id.route.addr.dev_addr.dev_type = - (rdma_port_get_link_layer (cma_dev->device, p) ==IB_LINK_LAYER_INFINIBAND ) ? + (rdma_port_ll_is_ib (cma_dev->device, p)) ?ARPHRD_INFINIBAND :ARPHRD_ETHER ;rdma_addr_set_sgid (&id_priv->id.route.addr.dev_addr, &gid);@@ -3405,7 +3402,7 @@ void rdma_leave_multicast(struct rdma_cm_id *id, struct sockaddr *addr)ib_detach_mcast (id->qp, &mc->multicast.ib->rec.mgid,be16_to_cpu (mc->multicast.ib->rec.mlid)); -if (rdma_node_get_transport (id_priv->cma_dev->device->node_type) ==RDMA_TRANSPORT_IB ) { +if (rdma_transport_is_ib (id_priv->cma_dev->device)) {switch (rdma_port_get_link_layer (id->device, id->port_num)) {case IB_LINK_LAYER_INFINIBAND :ib_sa_free_multicast (mc->multicast.ib);drivers/infiniband/core/mad.c +3
-3@@ -2938,7 +2938,7 @@ static int ib_mad_port_open(struct ib_device *device,init_mad_qp (port_priv, &port_priv->qp_info[1 ]); cq_size = mad_sendq_size + mad_recvq_size; - has_smi =rdma_port_get_link_layer (device, port_num) ==IB_LINK_LAYER_INFINIBAND ; + has_smi =rdma_port_ll_is_ib (device, port_num);if (has_smi) cq_size *=2 ;@@ -3057,7 +3057,7 @@ static void ib_mad_init_device(struct ib_device *device) {int start, end, i; -if (rdma_node_get_transport (device->node_type) !=RDMA_TRANSPORT_IB ) +if (!rdma_transport_is_ib (device))return ;if (device->node_type ==RDMA_NODE_IB_SWITCH ) {@@ -3102,7 +3102,7 @@ static void ib_mad_remove_device(struct ib_device *device) {int i, num_ports, cur_port; -if (rdma_node_get_transport (device->node_type) !=RDMA_TRANSPORT_IB ) +if (!rdma_transport_is_ib (device))return ;if (device->node_type ==RDMA_NODE_IB_SWITCH ) {drivers/infiniband/core/multicast.c +4
-7@@ -780,8 +780,7 @@ static void mcast_event_handler(struct ib_event_handler *handler,int index; dev = container_of(handler, struct mcast_device, event_handler); -if (rdma_port_get_link_layer (dev->device, event->element.port_num) != -IB_LINK_LAYER_INFINIBAND ) +if (!rdma_port_ll_is_ib (dev->device, event->element.port_num))return ; index = event->element.port_num - dev->start_port;@@ -808,7 +807,7 @@ static void mcast_add_one(struct ib_device *device)int i;int count =0 ; -if (rdma_node_get_transport (device->node_type) !=RDMA_TRANSPORT_IB ) +if (!rdma_transport_is_ib (device))return ; dev = kmalloc(sizeof *dev + device->phys_port_cnt * sizeof *port,@@ -824,8 +823,7 @@ static void mcast_add_one(struct ib_device *device) }for (i =0 ; i <= dev->end_port - dev->start_port; i++) { -if (rdma_port_get_link_layer (device, dev->start_port + i) != -IB_LINK_LAYER_INFINIBAND ) +if (!rdma_port_ll_is_ib (device, dev->start_port + i))continue ; port = &dev->port[i]; port->dev = dev;@@ -863,8 +861,7 @@ static void mcast_remove_one(struct ib_device *device)flush_workqueue (mcast_wq);for (i =0 ; i <= dev->end_port - dev->start_port; i++) { -if (rdma_port_get_link_layer (device, dev->start_port + i) == -IB_LINK_LAYER_INFINIBAND ) { +if (rdma_port_ll_is_ib (device, dev->start_port + i)) { port = &dev->port[i];deref_port (port);wait_for_completion (&port->comp);drivers/infiniband/core/sa_query.c +7
-7@@ -450,7 +450,7 @@ static void ib_sa_event(struct ib_event_handler *handler, struct ib_event *eventstruct ib_sa_port *port = &sa_dev->port[event->element.port_num - sa_dev->start_port]; -if (rdma_port_get_link_layer (handler->device, port->port_num) !=IB_LINK_LAYER_INFINIBAND ) +if (!rdma_port_ll_is_ib (handler->device, port->port_num))return ;spin_lock_irqsave (&port->ah_lock, flags);@@ -540,7 +540,7 @@ int ib_init_ah_from_path(struct ib_device *device, u8 port_num, ah_attr->port_num = port_num; ah_attr->static_rate = rec->rate; - force_grh =rdma_port_get_link_layer (device, port_num) ==IB_LINK_LAYER_ETHERNET ; + force_grh =rdma_port_ll_is_eth (device, port_num);if (rec->hop_limit >1 || force_grh) { ah_attr->ah_flags =IB_AH_GRH ;@@ -1154,7 +1154,7 @@ static void ib_sa_add_one(struct ib_device *device)struct ib_sa_device *sa_dev;int s, e, i; -if (rdma_node_get_transport (device->node_type) !=RDMA_TRANSPORT_IB ) +if (!rdma_transport_is_ib (device))return ;if (device->node_type ==RDMA_NODE_IB_SWITCH )@@ -1175,7 +1175,7 @@ static void ib_sa_add_one(struct ib_device *device)for (i =0 ; i <= e - s; ++i) {spin_lock_init (&sa_dev->port[i].ah_lock); -if (rdma_port_get_link_layer (device, i +1 ) !=IB_LINK_LAYER_INFINIBAND ) +if (!rdma_port_ll_is_ib (device, i +1 ))continue ; sa_dev->port[i].sm_ah =NULL ;@@ -1205,14 +1205,14 @@ static void ib_sa_add_one(struct ib_device *device) goto err;for (i =0 ; i <= e - s; ++i) -if (rdma_port_get_link_layer (device, i +1 ) ==IB_LINK_LAYER_INFINIBAND ) +if (rdma_port_ll_is_ib (device, i +1 ))update_sm_ah (&sa_dev->port[i].update_task);return ; err:while (--i >=0 ) -if (rdma_port_get_link_layer (device, i +1 ) ==IB_LINK_LAYER_INFINIBAND ) +if (rdma_port_ll_is_ib (device, i +1 ))ib_unregister_mad_agent (sa_dev->port[i].agent);kfree (sa_dev);@@ -1233,7 +1233,7 @@ static void ib_sa_remove_one(struct ib_device *device)flush_workqueue (ib_wq);for (i =0 ; i <= sa_dev->end_port - sa_dev->start_port; ++i) { -if (rdma_port_get_link_layer (device, i +1 ) ==IB_LINK_LAYER_INFINIBAND ) { +if (rdma_port_ll_is_ib (device, i +1 )) {ib_unregister_mad_agent (sa_dev->port[i].agent);if (sa_dev->port[i].sm_ah)kref_put (&sa_dev->port[i].sm_ah->ref, free_sm_ah);drivers/infiniband/core/ucm.c +1
-2@@ -1253,8 +1253,7 @@ static void ib_ucm_add_one(struct ib_device *device)dev_t base;struct ib_ucm_device *ucm_dev; -if (!device->alloc_ucontext || -rdma_node_get_transport (device->node_type) !=RDMA_TRANSPORT_IB ) +if (!device->alloc_ucontext || !rdma_transport_is_ib (device))return ; ucm_dev =kzalloc (sizeof *ucm_dev,GFP_KERNEL );drivers/infiniband/core/user_mad.c +1
-1@@ -1274,7 +1274,7 @@ static void ib_umad_add_one(struct ib_device *device)struct ib_umad_device *umad_dev;int s, e, i; -if (rdma_node_get_transport (device->node_type) !=RDMA_TRANSPORT_IB ) +if (!rdma_transport_is_ib (device))return ;if (device->node_type ==RDMA_NODE_IB_SWITCH )drivers/infiniband/core/verbs.c +2
-3@@ -198,8 +198,7 @@ int ib_init_ah_from_wc(struct ib_device *device, u8 port_num, struct ib_wc *wc,u32 flow_class;u16 gid_index;int ret; -int is_eth = (rdma_port_get_link_layer (device, port_num) == -IB_LINK_LAYER_ETHERNET ); +int is_eth = (rdma_port_ll_is_eth (device, port_num));memset (ah_attr,0 ,sizeof *ah_attr);if (is_eth) {@@ -871,7 +870,7 @@ int ib_resolve_eth_l2_attrs(struct ib_qp *qp,union ib_gid sgid;if ((*qp_attr_mask &IB_QP_AV ) && - (rdma_port_get_link_layer (qp->device, qp_attr->ah_attr.port_num) ==IB_LINK_LAYER_ETHERNET )) { + (rdma_port_ll_is_eth (qp->device, qp_attr->ah_attr.port_num))) { ret =ib_query_gid (qp->device, qp_attr->ah_attr.port_num, qp_attr->ah_attr.grh.sgid_index, &sgid);if (ret)drivers/infiniband/hw/mlx4/ah.c +1
-1@@ -118,7 +118,7 @@ struct ib_ah *mlx4_ib_create_ah(struct ib_pd *pd, struct ib_ah_attr *ah_attr)if (!ah)return ERR_PTR (-ENOMEM ); -if (rdma_port_get_link_layer (pd->device, ah_attr->port_num) ==IB_LINK_LAYER_ETHERNET ) { +if (rdma_port_ll_is_eth (pd->device, ah_attr->port_num)) {if (!(ah_attr->ah_flags &IB_AH_GRH )) { ret =ERR_PTR (-EINVAL ); }else {drivers/infiniband/hw/mlx4/cq.c +1
-3@@ -789,9 +789,7 @@ repoll:break ; } - is_eth = (rdma_port_get_link_layer (wc->qp->device, - (*cur_qp)->port) == -IB_LINK_LAYER_ETHERNET ); + is_eth = (rdma_port_ll_is_eth (wc->qp->device, (*cur_qp)->port));if (mlx4_is_mfunc (to_mdev (cq->ibcq.device)->dev)) { if ((*cur_qp)->mlx4_ib_qp_type & (MLX4_IB_QPT_PROXY_SMI_OWNER |drivers/infiniband/hw/mlx4/mad.c +4
-10@@ -606,12 +606,7 @@ static int mlx4_ib_demux_mad(struct ib_device *ibdev, u8 port,int err;int slave;u8 *slave_id; -int is_eth =0 ; - -if (rdma_port_get_link_layer (ibdev, port) ==IB_LINK_LAYER_INFINIBAND ) - is_eth =0 ; -else - is_eth =1 ; +int is_eth =rdma_port_ll_is_eth (ibdev, port);if (is_eth) {if (!(wc->wc_flags &IB_WC_GRH )) {@@ -1252,7 +1247,7 @@ out:static int get_slave_base_gid_ix (struct mlx4_ib_dev *dev,int slave,int port) { -if (rdma_port_get_link_layer (&dev->ib_dev, port) ==IB_LINK_LAYER_INFINIBAND ) +if (rdma_port_ll_is_ib (&dev->ib_dev, port))return slave;return mlx4_get_base_gid_ix (dev->dev, slave, port); }@@ -1260,7 +1255,7 @@ static int get_slave_base_gid_ix(struct mlx4_ib_dev *dev, int slave, int port)static void fill_in_real_sgid_index (struct mlx4_ib_dev *dev,int slave,int port,struct ib_ah_attr *ah_attr) { -if (rdma_port_get_link_layer (&dev->ib_dev, port) ==IB_LINK_LAYER_INFINIBAND ) +if (rdma_port_ll_is_ib (&dev->ib_dev, port)) ah_attr->grh.sgid_index = slave;else ah_attr->grh.sgid_index +=get_slave_base_gid_ix (dev, slave, port);@@ -1758,8 +1753,7 @@ static int create_pv_resources(struct ib_device *ibdev, int slave, int port, ctx->state =DEMUX_PV_STATE_STARTING ;/* have QP0 only if link layer is IB */ -if (rdma_port_get_link_layer (ibdev, ctx->port) == -IB_LINK_LAYER_INFINIBAND ) +if (rdma_port_ll_is_ib (ibdev, ctx->port)) ctx->has_smi =1 ;if (ctx->has_smi) {drivers/infiniband/hw/mlx4/main.c +3
-5@@ -482,7 +482,7 @@ static int iboe_query_gid(struct ib_device *ibdev, u8 port, int index,static int mlx4_ib_query_gid (struct ib_device *ibdev,u8 port,int index,union ib_gid *gid) { -if (rdma_port_get_link_layer (ibdev, port) ==IB_LINK_LAYER_INFINIBAND ) +if (rdma_port_ll_is_ib (ibdev, port))return __mlx4_ib_query_gid (ibdev, port, index, gid,0 );else return iboe_query_gid (ibdev, port, index, gid);@@ -1801,8 +1801,7 @@ static int mlx4_ib_init_gid_table(struct mlx4_ib_dev *ibdev)int err =0 ;for (i =1 ; i <= ibdev->num_ports; ++i) { -if (rdma_port_get_link_layer (&ibdev->ib_dev, i) == -IB_LINK_LAYER_ETHERNET ) { +if (rdma_port_ll_is_eth (&ibdev->ib_dev, i)) { err =reset_gid_table (ibdev, i);if (err) goto out;@@ -2554,8 +2553,7 @@ static void mlx4_ib_event(struct mlx4_dev *dev, void *ibdev_ptr,if (p > ibdev->num_ports)return ;if (mlx4_is_master (dev) && -rdma_port_get_link_layer (&ibdev->ib_dev, p) == -IB_LINK_LAYER_INFINIBAND ) { +rdma_port_ll_is_ib (&ibdev->ib_dev, p)) {mlx4_ib_invalidate_all_guid_record (ibdev, p); } ibev.event =IB_EVENT_PORT_ACTIVE ;drivers/infiniband/hw/mlx4/mlx4_ib.h +1
-1@@ -712,7 +712,7 @@ static inline bool mlx4_ib_ah_grh_present(struct mlx4_ib_ah *ah) {u8 port =be32_to_cpu (ah->av.ib.port_pd) >>24 &3 ; -if (rdma_port_get_link_layer (ah->ibah.device, port) ==IB_LINK_LAYER_ETHERNET ) +if (rdma_port_ll_is_eth (ah->ibah.device, port))return true;return !!(ah->av.ib.g_slid &0x80 );drivers/infiniband/hw/mlx4/qp.c +7
-14@@ -1248,8 +1248,7 @@ static int _mlx4_set_path(struct mlx4_ib_dev *dev, const struct ib_ah_attr *ah, u64 smac, u16 vlan_tag, struct mlx4_qp_path *path, struct mlx4_roce_smac_vlan_info *smac_info, u8 port) { - int is_eth = rdma_port_get_link_layer(&dev->ib_dev, port) == -IB_LINK_LAYER_ETHERNET ; + int is_eth = rdma_port_ll_is_eth(&dev->ib_dev, port);int vidx;int smac_index;int err;@@ -1433,8 +1432,7 @@ static int __mlx4_ib_modify_qp(struct ib_qp *ibqp,/* APM is not supported under RoCE */ if (attr_mask &IB_QP_ALT_PATH && -rdma_port_get_link_layer (&dev->ib_dev, qp->port) == -IB_LINK_LAYER_ETHERNET ) +rdma_port_ll_is_eth (&dev->ib_dev, qp->port))return -ENOTSUPP ; context =kzalloc (sizeof *context,GFP_KERNEL );@@ -1664,8 +1662,7 @@ static int __mlx4_ib_modify_qp(struct ib_qp *ibqp, context->pri_path.fl =0x80 ; context->pri_path.sched_queue |=MLX4_IB_DEFAULT_SCHED_QUEUE ; } -if (rdma_port_get_link_layer (&dev->ib_dev, qp->port) == -IB_LINK_LAYER_ETHERNET ) { +if (rdma_port_ll_is_eth (&dev->ib_dev, qp->port)) {if (qp->mlx4_ib_qp_type ==MLX4_IB_QPT_TUN_GSI || qp->mlx4_ib_qp_type ==MLX4_IB_QPT_GSI ) context->pri_path.feup =1 <<7 ;/* don't fsm */ @@ -1695,9 +1692,7 @@ static int __mlx4_ib_modify_qp(struct ib_qp *ibqp, }if (ibqp->qp_type ==IB_QPT_UD && (new_state ==IB_QPS_RTR )) { -int is_eth =rdma_port_get_link_layer ( - &dev->ib_dev, qp->port) == -IB_LINK_LAYER_ETHERNET ; +int is_eth =rdma_port_ll_is_eth (&dev->ib_dev, qp->port);if (is_eth) { context->pri_path.ackto =MLX4_IB_LINK_TYPE_ETH ; optpar |=MLX4_QP_OPTPAR_PRIMARY_ADDR_PATH ;@@ -1927,8 +1922,7 @@ int mlx4_ib_modify_qp(struct ib_qp *ibqp, struct ib_qp_attr *attr, }if ((attr_mask &IB_QP_PORT ) && (ibqp->qp_type ==IB_QPT_RAW_PACKET ) && - (rdma_port_get_link_layer (&dev->ib_dev, attr->port_num) != -IB_LINK_LAYER_ETHERNET )) + !rdma_port_ll_is_eth (&dev->ib_dev, attr->port_num)) goto out;if (attr_mask &IB_QP_PKEY_INDEX ) {@@ -2132,7 +2126,7 @@ static int build_mlx_header(struct mlx4_ib_sqp *sqp, struct ib_send_wr *wr,for (i =0 ; i < wr->num_sge; ++i) send_size += wr->sg_list[i].length; - is_eth =rdma_port_get_link_layer (sqp->qp.ibqp.device, sqp->qp.port) ==IB_LINK_LAYER_ETHERNET ; + is_eth =rdma_port_ll_is_eth (sqp->qp.ibqp.device, sqp->qp.port); is_grh =mlx4_ib_ah_grh_present (ah);if (is_eth) {if (mlx4_is_mfunc (to_mdev (ib_dev)->dev)) {@@ -3029,8 +3023,7 @@ static void to_ib_ah_attr(struct mlx4_ib_dev *ibdev, struct ib_ah_attr *ib_ah_atif (ib_ah_attr->port_num ==0 || ib_ah_attr->port_num > dev->caps.num_ports)return ; - is_eth =rdma_port_get_link_layer (&ibdev->ib_dev, ib_ah_attr->port_num) == -IB_LINK_LAYER_ETHERNET ; + is_eth =rdma_port_ll_is_eth (&ibdev->ib_dev, ib_ah_attr->port_num);if (is_eth) ib_ah_attr->sl = ((path->sched_queue >>3 ) &0x7 ) | ((path->sched_queue &4 ) <<1 );drivers/infiniband/hw/mlx4/sysfs.c +2
-4@@ -610,8 +610,7 @@ static ssize_t sysfs_store_enable_smi_admin(struct device *dev,static int add_vf_smi_entries (struct mlx4_port *p) { -int is_eth =rdma_port_get_link_layer (&p->dev->ib_dev, p->port_num) == -IB_LINK_LAYER_ETHERNET ; +int is_eth =rdma_port_ll_is_eth (&p->dev->ib_dev, p->port_num);int ret;/* do not display entries if eth transport, or if master */ @@ -645,8 +644,7 @@ static int add_vf_smi_entries(struct mlx4_port *p)static void remove_vf_smi_entries (struct mlx4_port *p) { -int is_eth =rdma_port_get_link_layer (&p->dev->ib_dev, p->port_num) == -IB_LINK_LAYER_ETHERNET ; +int is_eth =rdma_port_ll_is_eth (&p->dev->ib_dev, p->port_num);if (is_eth || p->slave ==mlx4_master_func_num (p->dev->dev))return ;drivers/infiniband/ulp/ipoib/ipoib_main.c +3
-3@@ -1655,7 +1655,7 @@ static void ipoib_add_one(struct ib_device *device)struct ipoib_dev_priv *priv;int s, e, p; -if (rdma_node_get_transport (device->node_type) !=RDMA_TRANSPORT_IB ) +if (!rdma_transport_is_ib (device))return ; dev_list =kmalloc (sizeof *dev_list,GFP_KERNEL );@@ -1673,7 +1673,7 @@ static void ipoib_add_one(struct ib_device *device) }for (p = s; p <= e; ++p) { -if (rdma_port_get_link_layer (device, p) !=IB_LINK_LAYER_INFINIBAND ) +if (!rdma_port_ll_is_ib (device, p))continue ; dev =ipoib_add_port ("ib%d" , device, p);if (!IS_ERR (dev)) {@@ -1690,7 +1690,7 @@ static void ipoib_remove_one(struct ib_device *device)struct ipoib_dev_priv *priv, *tmp;struct list_head *dev_list; -if (rdma_node_get_transport (device->node_type) !=RDMA_TRANSPORT_IB ) +if (!rdma_transport_is_ib (device))return ; dev_list =ib_get_client_data (device, &ipoib_client);include/rdma/ib_verbs.h +24
-0@@ -1743,6 +1743,30 @@ int ib_query_port(struct ib_device *device,enum rdma_link_layer rdma_port_get_link_layer (struct ib_device *device,u8 port_num); +static inline int rdma_transport_is_ib (struct ib_device *device) +{ +return rdma_node_get_transport (device->node_type) + ==RDMA_TRANSPORT_IB ; +} + +static inline int rdma_transport_is_iwarp (struct ib_device *device) +{ +return rdma_node_get_transport (device->node_type) + ==RDMA_TRANSPORT_IWARP ; +} + +static inline int rdma_port_ll_is_ib (struct ib_device *device,u8 port_num) +{ +return rdma_port_get_link_layer (device, port_num) + ==IB_LINK_LAYER_INFINIBAND ; +} + +static inline int rdma_port_ll_is_eth (struct ib_device *device,u8 port_num) +{ +return rdma_port_get_link_layer (device, port_num) + ==IB_LINK_LAYER_ETHERNET ; +} +int ib_query_gid (struct ib_device *device,u8 port_num,int index,union ib_gid *gid);net/sunrpc/xprtrdma/svc_rdma_recvfrom.c +0
-0@@ -118,8 +118,7 @@ static void rdma_build_arg_xdr(struct svc_rqst *rqstp,Quoted 9 lines
net/sunrpc/xprtrdma/svc_rdma_recvfrom.c static void rdma_build_arg_xdr(struct svc_rqst *rqstp, · line 124
Quoted 8 lines
net/sunrpc/xprtrdma/svc_rdma_recvfrom.c +1
-2static int rdma_read_max_sge (struct svcxprt_rdma *xprt,int sge_count) { -if (rdma_node_get_transport (xprt->sc_cm_id->device->node_type) == -RDMA_TRANSPORT_IWARP ) +if (rdma_transport_is_iwarp (xprt->sc_cm_id->device))return 1 ;else return min_t (int, sge_count, xprt->sc_max_sge);--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Signature · 2 lines
Doug Ledford <[email protected]> GPG KeyID: 0E572FDD
Re: [PATCH 01/11] IB/Verbs: Use helpers to check transport and link layer
Quoted 10 lines · Doug Ledford
Quoted 1 line · Michael Wang
[snip]
There is no "gradually eliminate them" to the suggestion I made. Remember, my suggestion was to remove the transport and
link_layeritems from the port settings and replace it with just one transport item that is a bitmask of the possible transport types. This can not be done gradually, it must be a complete change all at once as the two methods of setting things are incompatible. As there is only one out of tree driver that I know of, lustre, we can give them the information they need to make their driver work both before and after the change.
Actually there is something confused me on transport and link layer here, basically we have defined:
transport type
RDMA_TRANSPORT_IB,
RDMA_TRANSPORT_IWARP,
RDMA_TRANSPORT_USNIC,
RDMA_TRANSPORT_USNIC_UDP
link layer
IB_LINK_LAYER_INFINIBAND,
IB_LINK_LAYER_ETHERNET,
So we could have a table:
LL_INFINIBAND LL_ETHERNET UNCARETRANSPORT_IB 1 2 3TRANSPORT_IWARP, 4
UNCARE 5 6
In current implementation I've found all these combination in core or driver, and I could see:
rdma_transport_is_ib() 1rdma_transport_is_iwarp() 4rdma_transport_is_roce() 2
Just confusing how to take care the combination 3,5,6?
Regards,
Michael Wang
Quoted 654 lines
Quoted 654 lines
Sure if finally we do capture all the cases, we can just get rid of this one, but I guess it won't be that easy to directly jump into next stage :-P
As I could imaging, after this reform, next stage could be introducing the new mechanism without changing device driver, and the last stage is to asking vendor adapt their code into the new mechanism.
Quoted 9 lines
In other words, if our end goal is to have
rdma_transport_is_ib()rdma_transport_is_iwarp()rdma_transport_is_roce()rdma_transport_is_opa()Then we should skip doing rdma_port_ll_is_*() as the answers to these items would be implied by
rdma_transport_is_roce()and such.Great if we achieved that ;-) but currently I just wondering maybe these helpers can only cover part of the cases where we check transport and link layer, there are still some cases we'll need the very rough helper to save some code and make things clean~
Regards,
Michael WangQuoted 627 lines
Quoted 627 lines
Cc Jason Gunthorpe <jgunthorpe-ePGOBjL8dl3ta4EC/59zMFaTQe2KTcn/@public.gmane.org>Cc Doug Ledford <[email protected]>Cc Ira Weiny <[email protected]>Cc Sean Hefty <[email protected]>Signed-off-by Michael Wang <[email protected]>--- drivers/infiniband/core/agent.c | 2 +- drivers/infiniband/core/cm.c | 2 +- drivers/infiniband/core/cma.c | 27 ++++++++++++--------------- drivers/infiniband/core/mad.c | 6 +++--- drivers/infiniband/core/multicast.c | 11 ++++------- drivers/infiniband/core/sa_query.c | 14 +++++++------- drivers/infiniband/core/ucm.c | 3 +-- drivers/infiniband/core/user_mad.c | 2 +- drivers/infiniband/core/verbs.c | 5 ++--- drivers/infiniband/hw/mlx4/ah.c | 2 +- drivers/infiniband/hw/mlx4/cq.c | 4 +--- drivers/infiniband/hw/mlx4/mad.c | 14 ++++---------- drivers/infiniband/hw/mlx4/main.c | 8 +++----- drivers/infiniband/hw/mlx4/mlx4_ib.h | 2 +- drivers/infiniband/hw/mlx4/qp.c | 21 +++++++-------------- drivers/infiniband/hw/mlx4/sysfs.c | 6 ++---- drivers/infiniband/ulp/ipoib/ipoib_main.c | 6 +++--- include/rdma/ib_verbs.h | 24 ++++++++++++++++++++++++ net/sunrpc/xprtrdma/svc_rdma_recvfrom.c | 3 +-- 19 files changed, 79 insertions(+), 83 deletions(-)drivers/infiniband/core/agent.c +1
-1@@ -156,7 +156,7 @@ int ib_agent_port_open(struct ib_device *device, int port_num) goto error1; } -if (rdma_port_get_link_layer (device, port_num) ==IB_LINK_LAYER_INFINIBAND ) { +if (rdma_port_ll_is_ib (device, port_num)) {/* Obtain send only MAD agent for SMI QP */ port_priv->agent[0] = ib_register_mad_agent(device, port_num, IB_QPT_SMI, NULL, 0,drivers/infiniband/core/cm.c +1
-1@@ -3762,7 +3762,7 @@ static void cm_add_one(struct ib_device *ib_device)int ret;u8 i; -if (rdma_node_get_transport (ib_device->node_type) !=RDMA_TRANSPORT_IB ) +if (!rdma_transport_is_ib (ib_device))return ; cm_dev = kzalloc(sizeof(*cm_dev) + sizeof(*port) *drivers/infiniband/core/cma.c +12
-15@@ -375,8 +375,8 @@ static int cma_acquire_dev(struct rdma_id_private *id_priv, listen_id_priv->id.port_num) == dev_ll) { cma_dev = listen_id_priv->cma_dev; port = listen_id_priv->id.port_num; -if (rdma_node_get_transport (cma_dev->device->node_type) ==RDMA_TRANSPORT_IB && -rdma_port_get_link_layer (cma_dev->device, port) ==IB_LINK_LAYER_ETHERNET ) +if (rdma_transport_is_ib (cma_dev->device) && +rdma_port_ll_is_eth (cma_dev->device, port)) ret =ib_find_cached_gid (cma_dev->device, &iboe_gid, &found_port,NULL );else @@ -395,8 +395,8 @@ static int cma_acquire_dev(struct rdma_id_private *id_priv, listen_id_priv->id.port_num == port)continue ;if (rdma_port_get_link_layer (cma_dev->device, port) == dev_ll) { -if (rdma_node_get_transport (cma_dev->device->node_type) ==RDMA_TRANSPORT_IB && -rdma_port_get_link_layer (cma_dev->device, port) ==IB_LINK_LAYER_ETHERNET ) +if (rdma_transport_is_ib (cma_dev->device) && +rdma_port_ll_is_eth (cma_dev->device, port)) ret =ib_find_cached_gid (cma_dev->device, &iboe_gid, &found_port,NULL );else ret =ib_find_cached_gid (cma_dev->device, &gid, &found_port,NULL );@@ -435,7 +435,7 @@ static int cma_resolve_ib_dev(struct rdma_id_private *id_priv) pkey =ntohs (addr->sib_pkey);list_for_each_entry (cur_dev, &dev_list, list) { -if (rdma_node_get_transport (cur_dev->device->node_type) !=RDMA_TRANSPORT_IB ) +if (!rdma_transport_is_ib (cur_dev->device))continue ;for (p =1 ; p <= cur_dev->device->phys_port_cnt; ++p) {@@ -633,10 +633,8 @@ static int cma_modify_qp_rtr(struct rdma_id_private *id_priv,if (ret) goto out; -if (rdma_node_get_transport (id_priv->cma_dev->device->node_type) - ==RDMA_TRANSPORT_IB && -rdma_port_get_link_layer (id_priv->id.device, id_priv->id.port_num) - ==IB_LINK_LAYER_ETHERNET ) { +if (rdma_transport_is_ib (id_priv->cma_dev->device) && +rdma_port_ll_is_eth (id_priv->id.device, id_priv->id.port_num)) { ret =rdma_addr_find_smac_by_sgid (&sgid, qp_attr.smac,NULL );if (ret)@@ -700,8 +698,7 @@ static int cma_ib_init_qp_attr(struct rdma_id_private *id_priv,int ret;u16 pkey; -if (rdma_port_get_link_layer (id_priv->id.device, id_priv->id.port_num) == -IB_LINK_LAYER_INFINIBAND ) +if (rdma_port_ll_is_ib (id_priv->id.device, id_priv->id.port_num)) pkey =ib_addr_get_pkey (dev_addr);else pkey =0xffff ;@@ -1626,7 +1623,7 @@ static void cma_listen_on_dev(struct rdma_id_private *id_priv,int ret;if (cma_family (id_priv) ==AF_IB && -rdma_node_get_transport (cma_dev->device->node_type) !=RDMA_TRANSPORT_IB ) + !rdma_transport_is_ib (cma_dev->device))return ; id = rdma_create_id(cma_listen_handler, id_priv, id_priv->id.ps,@@ -2028,7 +2025,7 @@ static int cma_bind_loopback(struct rdma_id_private *id_priv)mutex_lock (&lock);list_for_each_entry (cur_dev, &dev_list, list) {if (cma_family (id_priv) ==AF_IB && -rdma_node_get_transport (cur_dev->device->node_type) !=RDMA_TRANSPORT_IB ) + !rdma_transport_is_ib (cur_dev->device))continue ;if (!cma_dev)@@ -2060,7 +2057,7 @@ port_found: goto out; id_priv->id.route.addr.dev_addr.dev_type = - (rdma_port_get_link_layer (cma_dev->device, p) ==IB_LINK_LAYER_INFINIBAND ) ? + (rdma_port_ll_is_ib (cma_dev->device, p)) ?ARPHRD_INFINIBAND :ARPHRD_ETHER ;rdma_addr_set_sgid (&id_priv->id.route.addr.dev_addr, &gid);@@ -3405,7 +3402,7 @@ void rdma_leave_multicast(struct rdma_cm_id *id, struct sockaddr *addr)ib_detach_mcast (id->qp, &mc->multicast.ib->rec.mgid,be16_to_cpu (mc->multicast.ib->rec.mlid)); -if (rdma_node_get_transport (id_priv->cma_dev->device->node_type) ==RDMA_TRANSPORT_IB ) { +if (rdma_transport_is_ib (id_priv->cma_dev->device)) {switch (rdma_port_get_link_layer (id->device, id->port_num)) {case IB_LINK_LAYER_INFINIBAND :ib_sa_free_multicast (mc->multicast.ib);drivers/infiniband/core/mad.c +3
-3@@ -2938,7 +2938,7 @@ static int ib_mad_port_open(struct ib_device *device,init_mad_qp (port_priv, &port_priv->qp_info[1 ]); cq_size = mad_sendq_size + mad_recvq_size; - has_smi =rdma_port_get_link_layer (device, port_num) ==IB_LINK_LAYER_INFINIBAND ; + has_smi =rdma_port_ll_is_ib (device, port_num);if (has_smi) cq_size *=2 ;@@ -3057,7 +3057,7 @@ static void ib_mad_init_device(struct ib_device *device) {int start, end, i; -if (rdma_node_get_transport (device->node_type) !=RDMA_TRANSPORT_IB ) +if (!rdma_transport_is_ib (device))return ;if (device->node_type ==RDMA_NODE_IB_SWITCH ) {@@ -3102,7 +3102,7 @@ static void ib_mad_remove_device(struct ib_device *device) {int i, num_ports, cur_port; -if (rdma_node_get_transport (device->node_type) !=RDMA_TRANSPORT_IB ) +if (!rdma_transport_is_ib (device))return ;if (device->node_type ==RDMA_NODE_IB_SWITCH ) {drivers/infiniband/core/multicast.c +4
-7@@ -780,8 +780,7 @@ static void mcast_event_handler(struct ib_event_handler *handler,int index; dev = container_of(handler, struct mcast_device, event_handler); -if (rdma_port_get_link_layer (dev->device, event->element.port_num) != -IB_LINK_LAYER_INFINIBAND ) +if (!rdma_port_ll_is_ib (dev->device, event->element.port_num))return ; index = event->element.port_num - dev->start_port;@@ -808,7 +807,7 @@ static void mcast_add_one(struct ib_device *device)int i;int count =0 ; -if (rdma_node_get_transport (device->node_type) !=RDMA_TRANSPORT_IB ) +if (!rdma_transport_is_ib (device))return ; dev = kmalloc(sizeof *dev + device->phys_port_cnt * sizeof *port,@@ -824,8 +823,7 @@ static void mcast_add_one(struct ib_device *device) }for (i =0 ; i <= dev->end_port - dev->start_port; i++) { -if (rdma_port_get_link_layer (device, dev->start_port + i) != -IB_LINK_LAYER_INFINIBAND ) +if (!rdma_port_ll_is_ib (device, dev->start_port + i))continue ; port = &dev->port[i]; port->dev = dev;@@ -863,8 +861,7 @@ static void mcast_remove_one(struct ib_device *device)flush_workqueue (mcast_wq);for (i =0 ; i <= dev->end_port - dev->start_port; i++) { -if (rdma_port_get_link_layer (device, dev->start_port + i) == -IB_LINK_LAYER_INFINIBAND ) { +if (rdma_port_ll_is_ib (device, dev->start_port + i)) { port = &dev->port[i];deref_port (port);wait_for_completion (&port->comp);drivers/infiniband/core/sa_query.c +7
-7@@ -450,7 +450,7 @@ static void ib_sa_event(struct ib_event_handler *handler, struct ib_event *eventstruct ib_sa_port *port = &sa_dev->port[event->element.port_num - sa_dev->start_port]; -if (rdma_port_get_link_layer (handler->device, port->port_num) !=IB_LINK_LAYER_INFINIBAND ) +if (!rdma_port_ll_is_ib (handler->device, port->port_num))return ;spin_lock_irqsave (&port->ah_lock, flags);@@ -540,7 +540,7 @@ int ib_init_ah_from_path(struct ib_device *device, u8 port_num, ah_attr->port_num = port_num; ah_attr->static_rate = rec->rate; - force_grh =rdma_port_get_link_layer (device, port_num) ==IB_LINK_LAYER_ETHERNET ; + force_grh =rdma_port_ll_is_eth (device, port_num);if (rec->hop_limit >1 || force_grh) { ah_attr->ah_flags =IB_AH_GRH ;@@ -1154,7 +1154,7 @@ static void ib_sa_add_one(struct ib_device *device)struct ib_sa_device *sa_dev;int s, e, i; -if (rdma_node_get_transport (device->node_type) !=RDMA_TRANSPORT_IB ) +if (!rdma_transport_is_ib (device))return ;if (device->node_type ==RDMA_NODE_IB_SWITCH )@@ -1175,7 +1175,7 @@ static void ib_sa_add_one(struct ib_device *device)for (i =0 ; i <= e - s; ++i) {spin_lock_init (&sa_dev->port[i].ah_lock); -if (rdma_port_get_link_layer (device, i +1 ) !=IB_LINK_LAYER_INFINIBAND ) +if (!rdma_port_ll_is_ib (device, i +1 ))continue ; sa_dev->port[i].sm_ah =NULL ;@@ -1205,14 +1205,14 @@ static void ib_sa_add_one(struct ib_device *device) goto err;for (i =0 ; i <= e - s; ++i) -if (rdma_port_get_link_layer (device, i +1 ) ==IB_LINK_LAYER_INFINIBAND ) +if (rdma_port_ll_is_ib (device, i +1 ))update_sm_ah (&sa_dev->port[i].update_task);return ; err:while (--i >=0 ) -if (rdma_port_get_link_layer (device, i +1 ) ==IB_LINK_LAYER_INFINIBAND ) +if (rdma_port_ll_is_ib (device, i +1 ))ib_unregister_mad_agent (sa_dev->port[i].agent);kfree (sa_dev);@@ -1233,7 +1233,7 @@ static void ib_sa_remove_one(struct ib_device *device)flush_workqueue (ib_wq);for (i =0 ; i <= sa_dev->end_port - sa_dev->start_port; ++i) { -if (rdma_port_get_link_layer (device, i +1 ) ==IB_LINK_LAYER_INFINIBAND ) { +if (rdma_port_ll_is_ib (device, i +1 )) {ib_unregister_mad_agent (sa_dev->port[i].agent);if (sa_dev->port[i].sm_ah)kref_put (&sa_dev->port[i].sm_ah->ref, free_sm_ah);drivers/infiniband/core/ucm.c +1
-2@@ -1253,8 +1253,7 @@ static void ib_ucm_add_one(struct ib_device *device)dev_t base;struct ib_ucm_device *ucm_dev; -if (!device->alloc_ucontext || -rdma_node_get_transport (device->node_type) !=RDMA_TRANSPORT_IB ) +if (!device->alloc_ucontext || !rdma_transport_is_ib (device))return ; ucm_dev =kzalloc (sizeof *ucm_dev,GFP_KERNEL );drivers/infiniband/core/user_mad.c +1
-1@@ -1274,7 +1274,7 @@ static void ib_umad_add_one(struct ib_device *device)struct ib_umad_device *umad_dev;int s, e, i; -if (rdma_node_get_transport (device->node_type) !=RDMA_TRANSPORT_IB ) +if (!rdma_transport_is_ib (device))return ;if (device->node_type ==RDMA_NODE_IB_SWITCH )drivers/infiniband/core/verbs.c +2
-3@@ -198,8 +198,7 @@ int ib_init_ah_from_wc(struct ib_device *device, u8 port_num, struct ib_wc *wc,u32 flow_class;u16 gid_index;int ret; -int is_eth = (rdma_port_get_link_layer (device, port_num) == -IB_LINK_LAYER_ETHERNET ); +int is_eth = (rdma_port_ll_is_eth (device, port_num));memset (ah_attr,0 ,sizeof *ah_attr);if (is_eth) {@@ -871,7 +870,7 @@ int ib_resolve_eth_l2_attrs(struct ib_qp *qp,union ib_gid sgid;if ((*qp_attr_mask &IB_QP_AV ) && - (rdma_port_get_link_layer (qp->device, qp_attr->ah_attr.port_num) ==IB_LINK_LAYER_ETHERNET )) { + (rdma_port_ll_is_eth (qp->device, qp_attr->ah_attr.port_num))) { ret =ib_query_gid (qp->device, qp_attr->ah_attr.port_num, qp_attr->ah_attr.grh.sgid_index, &sgid);if (ret)drivers/infiniband/hw/mlx4/ah.c +1
-1@@ -118,7 +118,7 @@ struct ib_ah *mlx4_ib_create_ah(struct ib_pd *pd, struct ib_ah_attr *ah_attr)if (!ah)return ERR_PTR (-ENOMEM ); -if (rdma_port_get_link_layer (pd->device, ah_attr->port_num) ==IB_LINK_LAYER_ETHERNET ) { +if (rdma_port_ll_is_eth (pd->device, ah_attr->port_num)) {if (!(ah_attr->ah_flags &IB_AH_GRH )) { ret =ERR_PTR (-EINVAL ); }else {drivers/infiniband/hw/mlx4/cq.c +1
-3@@ -789,9 +789,7 @@ repoll:break ; } - is_eth = (rdma_port_get_link_layer (wc->qp->device, - (*cur_qp)->port) == -IB_LINK_LAYER_ETHERNET ); + is_eth = (rdma_port_ll_is_eth (wc->qp->device, (*cur_qp)->port));if (mlx4_is_mfunc (to_mdev (cq->ibcq.device)->dev)) { if ((*cur_qp)->mlx4_ib_qp_type & (MLX4_IB_QPT_PROXY_SMI_OWNER |drivers/infiniband/hw/mlx4/mad.c +4
-10@@ -606,12 +606,7 @@ static int mlx4_ib_demux_mad(struct ib_device *ibdev, u8 port,int err;int slave;u8 *slave_id; -int is_eth =0 ; - -if (rdma_port_get_link_layer (ibdev, port) ==IB_LINK_LAYER_INFINIBAND ) - is_eth =0 ; -else - is_eth =1 ; +int is_eth =rdma_port_ll_is_eth (ibdev, port);if (is_eth) {if (!(wc->wc_flags &IB_WC_GRH )) {@@ -1252,7 +1247,7 @@ out:static int get_slave_base_gid_ix (struct mlx4_ib_dev *dev,int slave,int port) { -if (rdma_port_get_link_layer (&dev->ib_dev, port) ==IB_LINK_LAYER_INFINIBAND ) +if (rdma_port_ll_is_ib (&dev->ib_dev, port))return slave;return mlx4_get_base_gid_ix (dev->dev, slave, port); }@@ -1260,7 +1255,7 @@ static int get_slave_base_gid_ix(struct mlx4_ib_dev *dev, int slave, int port)static void fill_in_real_sgid_index (struct mlx4_ib_dev *dev,int slave,int port,struct ib_ah_attr *ah_attr) { -if (rdma_port_get_link_layer (&dev->ib_dev, port) ==IB_LINK_LAYER_INFINIBAND ) +if (rdma_port_ll_is_ib (&dev->ib_dev, port)) ah_attr->grh.sgid_index = slave;else ah_attr->grh.sgid_index +=get_slave_base_gid_ix (dev, slave, port);@@ -1758,8 +1753,7 @@ static int create_pv_resources(struct ib_device *ibdev, int slave, int port, ctx->state =DEMUX_PV_STATE_STARTING ;/* have QP0 only if link layer is IB */ -if (rdma_port_get_link_layer (ibdev, ctx->port) == -IB_LINK_LAYER_INFINIBAND ) +if (rdma_port_ll_is_ib (ibdev, ctx->port)) ctx->has_smi =1 ;if (ctx->has_smi) {drivers/infiniband/hw/mlx4/main.c +3
-5@@ -482,7 +482,7 @@ static int iboe_query_gid(struct ib_device *ibdev, u8 port, int index,static int mlx4_ib_query_gid (struct ib_device *ibdev,u8 port,int index,union ib_gid *gid) { -if (rdma_port_get_link_layer (ibdev, port) ==IB_LINK_LAYER_INFINIBAND ) +if (rdma_port_ll_is_ib (ibdev, port))return __mlx4_ib_query_gid (ibdev, port, index, gid,0 );else return iboe_query_gid (ibdev, port, index, gid);@@ -1801,8 +1801,7 @@ static int mlx4_ib_init_gid_table(struct mlx4_ib_dev *ibdev)int err =0 ;for (i =1 ; i <= ibdev->num_ports; ++i) { -if (rdma_port_get_link_layer (&ibdev->ib_dev, i) == -IB_LINK_LAYER_ETHERNET ) { +if (rdma_port_ll_is_eth (&ibdev->ib_dev, i)) { err =reset_gid_table (ibdev, i);if (err) goto out;@@ -2554,8 +2553,7 @@ static void mlx4_ib_event(struct mlx4_dev *dev, void *ibdev_ptr,if (p > ibdev->num_ports)return ;if (mlx4_is_master (dev) && -rdma_port_get_link_layer (&ibdev->ib_dev, p) == -IB_LINK_LAYER_INFINIBAND ) { +rdma_port_ll_is_ib (&ibdev->ib_dev, p)) {mlx4_ib_invalidate_all_guid_record (ibdev, p); } ibev.event =IB_EVENT_PORT_ACTIVE ;drivers/infiniband/hw/mlx4/mlx4_ib.h +1
-1@@ -712,7 +712,7 @@ static inline bool mlx4_ib_ah_grh_present(struct mlx4_ib_ah *ah) {u8 port =be32_to_cpu (ah->av.ib.port_pd) >>24 &3 ; -if (rdma_port_get_link_layer (ah->ibah.device, port) ==IB_LINK_LAYER_ETHERNET ) +if (rdma_port_ll_is_eth (ah->ibah.device, port))return true;return !!(ah->av.ib.g_slid &0x80 );drivers/infiniband/hw/mlx4/qp.c +7
-14@@ -1248,8 +1248,7 @@ static int _mlx4_set_path(struct mlx4_ib_dev *dev, const struct ib_ah_attr *ah, u64 smac, u16 vlan_tag, struct mlx4_qp_path *path, struct mlx4_roce_smac_vlan_info *smac_info, u8 port) { - int is_eth = rdma_port_get_link_layer(&dev->ib_dev, port) == -IB_LINK_LAYER_ETHERNET ; + int is_eth = rdma_port_ll_is_eth(&dev->ib_dev, port);int vidx;int smac_index;int err;@@ -1433,8 +1432,7 @@ static int __mlx4_ib_modify_qp(struct ib_qp *ibqp,/* APM is not supported under RoCE */ if (attr_mask &IB_QP_ALT_PATH && -rdma_port_get_link_layer (&dev->ib_dev, qp->port) == -IB_LINK_LAYER_ETHERNET ) +rdma_port_ll_is_eth (&dev->ib_dev, qp->port))return -ENOTSUPP ; context =kzalloc (sizeof *context,GFP_KERNEL );@@ -1664,8 +1662,7 @@ static int __mlx4_ib_modify_qp(struct ib_qp *ibqp, context->pri_path.fl =0x80 ; context->pri_path.sched_queue |=MLX4_IB_DEFAULT_SCHED_QUEUE ; } -if (rdma_port_get_link_layer (&dev->ib_dev, qp->port) == -IB_LINK_LAYER_ETHERNET ) { +if (rdma_port_ll_is_eth (&dev->ib_dev, qp->port)) {if (qp->mlx4_ib_qp_type ==MLX4_IB_QPT_TUN_GSI || qp->mlx4_ib_qp_type ==MLX4_IB_QPT_GSI ) context->pri_path.feup =1 <<7 ;/* don't fsm */ @@ -1695,9 +1692,7 @@ static int __mlx4_ib_modify_qp(struct ib_qp *ibqp, }if (ibqp->qp_type ==IB_QPT_UD && (new_state ==IB_QPS_RTR )) { -int is_eth =rdma_port_get_link_layer ( - &dev->ib_dev, qp->port) == -IB_LINK_LAYER_ETHERNET ; +int is_eth =rdma_port_ll_is_eth (&dev->ib_dev, qp->port);if (is_eth) { context->pri_path.ackto =MLX4_IB_LINK_TYPE_ETH ; optpar |=MLX4_QP_OPTPAR_PRIMARY_ADDR_PATH ;@@ -1927,8 +1922,7 @@ int mlx4_ib_modify_qp(struct ib_qp *ibqp, struct ib_qp_attr *attr, }if ((attr_mask &IB_QP_PORT ) && (ibqp->qp_type ==IB_QPT_RAW_PACKET ) && - (rdma_port_get_link_layer (&dev->ib_dev, attr->port_num) != -IB_LINK_LAYER_ETHERNET )) + !rdma_port_ll_is_eth (&dev->ib_dev, attr->port_num)) goto out;if (attr_mask &IB_QP_PKEY_INDEX ) {@@ -2132,7 +2126,7 @@ static int build_mlx_header(struct mlx4_ib_sqp *sqp, struct ib_send_wr *wr,for (i =0 ; i < wr->num_sge; ++i) send_size += wr->sg_list[i].length; - is_eth =rdma_port_get_link_layer (sqp->qp.ibqp.device, sqp->qp.port) ==IB_LINK_LAYER_ETHERNET ; + is_eth =rdma_port_ll_is_eth (sqp->qp.ibqp.device, sqp->qp.port); is_grh =mlx4_ib_ah_grh_present (ah);if (is_eth) {if (mlx4_is_mfunc (to_mdev (ib_dev)->dev)) {@@ -3029,8 +3023,7 @@ static void to_ib_ah_attr(struct mlx4_ib_dev *ibdev, struct ib_ah_attr *ib_ah_atif (ib_ah_attr->port_num ==0 || ib_ah_attr->port_num > dev->caps.num_ports)return ; - is_eth =rdma_port_get_link_layer (&ibdev->ib_dev, ib_ah_attr->port_num) == -IB_LINK_LAYER_ETHERNET ; + is_eth =rdma_port_ll_is_eth (&ibdev->ib_dev, ib_ah_attr->port_num);if (is_eth) ib_ah_attr->sl = ((path->sched_queue >>3 ) &0x7 ) | ((path->sched_queue &4 ) <<1 );drivers/infiniband/hw/mlx4/sysfs.c +2
-4@@ -610,8 +610,7 @@ static ssize_t sysfs_store_enable_smi_admin(struct device *dev,static int add_vf_smi_entries (struct mlx4_port *p) { -int is_eth =rdma_port_get_link_layer (&p->dev->ib_dev, p->port_num) == -IB_LINK_LAYER_ETHERNET ; +int is_eth =rdma_port_ll_is_eth (&p->dev->ib_dev, p->port_num);int ret;/* do not display entries if eth transport, or if master */ @@ -645,8 +644,7 @@ static int add_vf_smi_entries(struct mlx4_port *p)static void remove_vf_smi_entries (struct mlx4_port *p) { -int is_eth =rdma_port_get_link_layer (&p->dev->ib_dev, p->port_num) == -IB_LINK_LAYER_ETHERNET ; +int is_eth =rdma_port_ll_is_eth (&p->dev->ib_dev, p->port_num);if (is_eth || p->slave ==mlx4_master_func_num (p->dev->dev))return ;drivers/infiniband/ulp/ipoib/ipoib_main.c +3
-3@@ -1655,7 +1655,7 @@ static void ipoib_add_one(struct ib_device *device)struct ipoib_dev_priv *priv;int s, e, p; -if (rdma_node_get_transport (device->node_type) !=RDMA_TRANSPORT_IB ) +if (!rdma_transport_is_ib (device))return ; dev_list =kmalloc (sizeof *dev_list,GFP_KERNEL );@@ -1673,7 +1673,7 @@ static void ipoib_add_one(struct ib_device *device) }for (p = s; p <= e; ++p) { -if (rdma_port_get_link_layer (device, p) !=IB_LINK_LAYER_INFINIBAND ) +if (!rdma_port_ll_is_ib (device, p))continue ; dev =ipoib_add_port ("ib%d" , device, p);if (!IS_ERR (dev)) {@@ -1690,7 +1690,7 @@ static void ipoib_remove_one(struct ib_device *device)struct ipoib_dev_priv *priv, *tmp;struct list_head *dev_list; -if (rdma_node_get_transport (device->node_type) !=RDMA_TRANSPORT_IB ) +if (!rdma_transport_is_ib (device))return ; dev_list =ib_get_client_data (device, &ipoib_client);include/rdma/ib_verbs.h +24
-0@@ -1743,6 +1743,30 @@ int ib_query_port(struct ib_device *device,enum rdma_link_layer rdma_port_get_link_layer (struct ib_device *device,u8 port_num); +static inline int rdma_transport_is_ib (struct ib_device *device) +{ +return rdma_node_get_transport (device->node_type) + ==RDMA_TRANSPORT_IB ; +} + +static inline int rdma_transport_is_iwarp (struct ib_device *device) +{ +return rdma_node_get_transport (device->node_type) + ==RDMA_TRANSPORT_IWARP ; +} + +static inline int rdma_port_ll_is_ib (struct ib_device *device,u8 port_num) +{ +return rdma_port_get_link_layer (device, port_num) + ==IB_LINK_LAYER_INFINIBAND ; +} + +static inline int rdma_port_ll_is_eth (struct ib_device *device,u8 port_num) +{ +return rdma_port_get_link_layer (device, port_num) + ==IB_LINK_LAYER_ETHERNET ; +} +int ib_query_gid (struct ib_device *device,u8 port_num,int index,union ib_gid *gid);net/sunrpc/xprtrdma/svc_rdma_recvfrom.c +0
-0@@ -118,8 +118,7 @@ static void rdma_build_arg_xdr(struct svc_rqst *rqstp,
Quoted 13 lines
Quoted 8 lines
Quoted 8 lines
net/sunrpc/xprtrdma/svc_rdma_recvfrom.c static void rdma_build_arg_xdr(struct svc_rqst *rqstp, · line 124
Quoted 8 lines
net/sunrpc/xprtrdma/svc_rdma_recvfrom.c +1
-2static int rdma_read_max_sge (struct svcxprt_rdma *xprt,int sge_count) { -if (rdma_node_get_transport (xprt->sc_cm_id->device->node_type) == -RDMA_TRANSPORT_IWARP ) +if (rdma_transport_is_iwarp (xprt->sc_cm_id->device))return 1 ;else return min_t (int, sge_count, xprt->sc_max_sge);Quoted 4 lines
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
RFC PATCH 2/11 IB/Verbs: Use management helper tech_iboe() for iboe-check
In reply to Michael WangIntroduce helper tech_iboe() to help us check if the port of an IB device is using RoCE/IBoE technology.
--- drivers/infiniband/core/cma.c | 6 ++---- include/rdma/ib_verbs.h | 16 ++++++++++++++++ 2 files changed, 18 insertions(+), 4 deletions(-)
drivers/infiniband/core/cma.c +2 -4
@@ -375,8 +375,7 @@ static int cma_acquire_dev(struct rdma_id_private *id_priv, listen_id_priv->id.port_num) == dev_ll) { cma_dev = listen_id_priv->cma_dev; port = listen_id_priv->id.port_num; -if (rdma_transport_is_ib (cma_dev->device) && -rdma_port_ll_is_eth (cma_dev->device, port)) +if (tech_iboe (cma_dev->device, port)) ret =ib_find_cached_gid (cma_dev->device, &iboe_gid, &found_port,NULL );else
@@ -395,8 +394,7 @@ static int cma_acquire_dev(struct rdma_id_private *id_priv, listen_id_priv->id.port_num == port)continue ;if (rdma_port_get_link_layer (cma_dev->device, port) == dev_ll) { -if (rdma_transport_is_ib (cma_dev->device) && -rdma_port_ll_is_eth (cma_dev->device, port)) +if (tech_iboe (cma_dev->device, port)) ret =ib_find_cached_gid (cma_dev->device, &iboe_gid, &found_port,NULL );else ret =ib_find_cached_gid (cma_dev->device, &gid, &found_port,NULL );
include/rdma/ib_verbs.h +16 -0
@@ -1767,6 +1767,22 @@ static inline int rdma_port_ll_is_eth(struct ib_device *device, u8 port_num) == IB_LINK_LAYER_ETHERNET; } +/** +* tech_iboe - Check if the port of device using technology +* RoCE/IBoE. +* +* @device: Device to be checked +* @port_num: Port number of the device +* +* Return 0 when port of the device is not using technology +* RoCE/IBoE. +*/ +static inline int tech_iboe (struct ib_device *device,u8 port_num) +{ +return rdma_transport_is_ib (device) && +rdma_port_ll_is_eth (device, port_num); +} +int ib_query_gid (struct ib_device *device,u8 port_num,int index,union ib_gid *gid);
Signature · 1 line
2.1.0
Re: [RFC PATCH 02/11] IB/Verbs: Use management helper tech_iboe() for iboe-check
Quoted 2 lines · Michael Wang
Introduce helper
tech_iboe()to help us check if the port of an IB device is using RoCE/IBoE technology.
Just use rdma_transport_is_roce() instead.
Quoted 61 lines
Cc Jason Gunthorpe <[email protected]>Cc Doug Ledford <[email protected]>Cc Ira Weiny <[email protected]>Cc Sean Hefty <[email protected]>Signed-off-by Michael Wang <[email protected]>--- drivers/infiniband/core/cma.c | 6 ++---- include/rdma/ib_verbs.h | 16 ++++++++++++++++ 2 files changed, 18 insertions(+), 4 deletions(-)drivers/infiniband/core/cma.c +2
-4@@ -375,8 +375,7 @@ static int cma_acquire_dev(struct rdma_id_private *id_priv, listen_id_priv->id.port_num) == dev_ll) { cma_dev = listen_id_priv->cma_dev; port = listen_id_priv->id.port_num; -if (rdma_transport_is_ib (cma_dev->device) && -rdma_port_ll_is_eth (cma_dev->device, port)) +if (tech_iboe (cma_dev->device, port)) ret =ib_find_cached_gid (cma_dev->device, &iboe_gid, &found_port,NULL );else @@ -395,8 +394,7 @@ static int cma_acquire_dev(struct rdma_id_private *id_priv, listen_id_priv->id.port_num == port)continue ;if (rdma_port_get_link_layer (cma_dev->device, port) == dev_ll) { -if (rdma_transport_is_ib (cma_dev->device) && -rdma_port_ll_is_eth (cma_dev->device, port)) +if (tech_iboe (cma_dev->device, port)) ret =ib_find_cached_gid (cma_dev->device, &iboe_gid, &found_port,NULL );else ret =ib_find_cached_gid (cma_dev->device, &gid, &found_port,NULL );include/rdma/ib_verbs.h +16
-0@@ -1767,6 +1767,22 @@ static inline int rdma_port_ll_is_eth(struct ib_device *device, u8 port_num) == IB_LINK_LAYER_ETHERNET; } +/** +* tech_iboe - Check if the port of device using technology +* RoCE/IBoE. +* +* @device: Device to be checked +* @port_num: Port number of the device +* +* Return 0 when port of the device is not using technology +* RoCE/IBoE. +*/ +static inline int tech_iboe (struct ib_device *device,u8 port_num) +{ +return rdma_transport_is_ib (device) && +rdma_port_ll_is_eth (device, port_num); +} +int ib_query_gid (struct ib_device *device,u8 port_num,int index,union ib_gid *gid);
Signature · 2 lines
Doug Ledford <[email protected]> GPG KeyID: 0E572FDD
Re: [RFC PATCH 02/11] IB/Verbs: Use management helper tech_iboe() for iboe-check
Quoted 4 lines · Doug Ledford
Quoted 2 lines · Michael Wang
Introduce helper
tech_iboe()to help us check if the port of an IB device is using RoCE/IBoE technology.Just use
rdma_transport_is_roce()instead.
Sounds good :-) will be in next version.
Regards,
Michael Wang
Quoted 54 lines
Quoted 54 lines
Cc Jason Gunthorpe <jgunthorpe-ePGOBjL8dl3ta4EC/59zMFaTQe2KTcn/@public.gmane.org>Cc Doug Ledford <[email protected]>Cc Ira Weiny <[email protected]>Cc Sean Hefty <[email protected]>Signed-off-by Michael Wang <[email protected]>--- drivers/infiniband/core/cma.c | 6 ++---- include/rdma/ib_verbs.h | 16 ++++++++++++++++ 2 files changed, 18 insertions(+), 4 deletions(-)drivers/infiniband/core/cma.c +2
-4@@ -375,8 +375,7 @@ static int cma_acquire_dev(struct rdma_id_private *id_priv, listen_id_priv->id.port_num) == dev_ll) { cma_dev = listen_id_priv->cma_dev; port = listen_id_priv->id.port_num; -if (rdma_transport_is_ib (cma_dev->device) && -rdma_port_ll_is_eth (cma_dev->device, port)) +if (tech_iboe (cma_dev->device, port)) ret =ib_find_cached_gid (cma_dev->device, &iboe_gid, &found_port,NULL );else @@ -395,8 +394,7 @@ static int cma_acquire_dev(struct rdma_id_private *id_priv, listen_id_priv->id.port_num == port)continue ;if (rdma_port_get_link_layer (cma_dev->device, port) == dev_ll) { -if (rdma_transport_is_ib (cma_dev->device) && -rdma_port_ll_is_eth (cma_dev->device, port)) +if (tech_iboe (cma_dev->device, port)) ret =ib_find_cached_gid (cma_dev->device, &iboe_gid, &found_port,NULL );else ret =ib_find_cached_gid (cma_dev->device, &gid, &found_port,NULL );include/rdma/ib_verbs.h +11
-0@@ -1767,6 +1767,22 @@ static inline int rdma_port_ll_is_eth(struct ib_device *device, u8 port_num) == IB_LINK_LAYER_ETHERNET; } +/** +* tech_iboe - Check if the port of device using technology +* RoCE/IBoE. +* +* @device: Device to be checked +* @port_num: Port number of the device +* +* Return 0 when port of the device is not using technology +* RoCE/IBoE. +*/ +static inline int tech_iboe (struct ib_device *device,u8 port_num)
Quoted 9 lines
include/rdma/ib_verbs.h static inline int rdma_port_ll_is_eth(struct ib_device *device, u8 port_num) · line 1788
Quoted 8 lines
include/rdma/ib_verbs.h +5
-0+{ +return rdma_transport_is_ib (device) && +rdma_port_ll_is_eth (device, port_num); +} +int ib_query_gid (struct ib_device *device,u8 port_num,int index,union ib_gid *gid);
--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
RFC PATCH 3/11 IB/Verbs: Use management helper has_mad() for mad-check
In reply to Michael WangIntroduce helper has_mad() to help us check if an IB device support Management Datagrams.
--- drivers/infiniband/core/mad.c | 4 ++-- drivers/infiniband/core/user_mad.c | 2 +- include/rdma/ib_verbs.h | 13 +++++++++++++ 3 files changed, 16 insertions(+), 3 deletions(-)
drivers/infiniband/core/mad.c +2 -2
@@ -3057,7 +3057,7 @@ static void ib_mad_init_device(struct ib_device *device) {int start, end, i; -if (!rdma_transport_is_ib (device)) +if (!has_mad (device))return ;if (device->node_type ==RDMA_NODE_IB_SWITCH ) {
@@ -3102,7 +3102,7 @@ static void ib_mad_remove_device(struct ib_device *device) {int i, num_ports, cur_port; -if (!rdma_transport_is_ib (device)) +if (!has_mad (device))return ;if (device->node_type ==RDMA_NODE_IB_SWITCH ) {
drivers/infiniband/core/user_mad.c +1 -1
@@ -1274,7 +1274,7 @@ static void ib_umad_add_one(struct ib_device *device)struct ib_umad_device *umad_dev;int s, e, i; -if (!rdma_transport_is_ib (device)) +if (!has_mad (device))return ;if (device->node_type ==RDMA_NODE_IB_SWITCH )
include/rdma/ib_verbs.h +13 -0
@@ -1783,6 +1783,19 @@ static inline int tech_iboe(struct ib_device *device, u8 port_num)rdma_port_ll_is_eth (device, port_num); } +/** +* has_mad - Check if a device support Management Datagrams. +* +* @device: Device to be checked +* +* Return 0 when a device has none port to support +* Management Datagrams. +*/ +static inline int has_mad (struct ib_device *device) +{ +return rdma_transport_is_ib (device); +} +int ib_query_gid (struct ib_device *device,u8 port_num,int index,union ib_gid *gid);
Signature · 1 line
2.1.0
RFC PATCH 4/11 IB/Verbs: Use management helper cap_smi() for smi-check
In reply to Michael WangIntroduce helper cap_smi() to help us check if the port of an IB device support Subnet Management Interface.
--- drivers/infiniband/core/agent.c | 2 +- drivers/infiniband/core/mad.c | 2 +- drivers/infiniband/hw/mlx4/mad.c | 2 +- include/rdma/ib_verbs.h | 15 +++++++++++++++ 4 files changed, 18 insertions(+), 3 deletions(-)
drivers/infiniband/core/agent.c +1 -1
@@ -156,7 +156,7 @@ int ib_agent_port_open(struct ib_device *device, int port_num) goto error1; } -if (rdma_port_ll_is_ib (device, port_num)) { +if (cap_smi (device, port_num)) {/* Obtain send only MAD agent for SMI QP */ port_priv->agent[0] = ib_register_mad_agent(device, port_num, IB_QPT_SMI, NULL, 0,
drivers/infiniband/core/mad.c +1 -1
@@ -2938,7 +2938,7 @@ static int ib_mad_port_open(struct ib_device *device,init_mad_qp (port_priv, &port_priv->qp_info[1 ]); cq_size = mad_sendq_size + mad_recvq_size; - has_smi =rdma_port_ll_is_ib (device, port_num); + has_smi =cap_smi (device, port_num);if (has_smi) cq_size *=2 ;
drivers/infiniband/hw/mlx4/mad.c +1 -1
@@ -1753,7 +1753,7 @@ static int create_pv_resources(struct ib_device *ibdev, int slave, int port, ctx->state =DEMUX_PV_STATE_STARTING ;/* have QP0 only if link layer is IB */ -if (rdma_port_ll_is_ib (ibdev, ctx->port)) +if (cap_smi (ibdev, ctx->port)) ctx->has_smi =1 ;if (ctx->has_smi) {
include/rdma/ib_verbs.h +15 -0
@@ -1796,6 +1796,21 @@ static inline int has_mad(struct ib_device *device)return rdma_transport_is_ib (device); } +/** +* cap_smi - Check if the port of device has the capability +* Subnet Management Interface. +* +* @device: Device to be checked +* @port_num: Port number of the device +* +* Return 0 when port of the device don't support +* Subnet Management Interface. +*/ +static inline int cap_smi (struct ib_device *device,u8 port_num) +{ +return rdma_port_ll_is_ib (device, port_num); +} +int ib_query_gid (struct ib_device *device,u8 port_num,int index,union ib_gid *gid);
Signature · 1 line
2.1.0
Re: [RFC PATCH 04/11] IB/Verbs: Use management helper cap_smi() for smi-check
Quoted 3 lines · Michael Wang
Introduce helper
cap_smi()to help us check if the port of an IB device support Subnet Management Interface.
This and most of the others should be qualified with 'cap_ib_smi' since we already know we will need 'cap_opa_smi' someday
Jason
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [RFC PATCH 04/11] IB/Verbs: Use management helper cap_smi() for smi-check
Make sense to me if opa smi will be handled differently, will be changed in next version :-)
Regards,
Michael Wang
Quoted 9 lines · Jason Gunthorpe
Quoted 3 lines · Michael Wang
Introduce helper
cap_smi()to help us check if the port of an IB device support Subnet Management Interface.This and most of the others should be qualified with '
cap_ib_smi' since we already know we will need 'cap_opa_smi' somedayJason
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
RFC PATCH 5/11 IB/Verbs: Use management helper has_cm() for cm-check
In reply to Michael WangIntroduce helper has_cm() to help us check if an IB device support Communication Manager.
--- drivers/infiniband/core/cm.c | 2 +- drivers/infiniband/core/cma.c | 10 ++++------ drivers/infiniband/core/ucm.c | 2 +- include/rdma/ib_verbs.h | 13 +++++++++++++ 4 files changed, 19 insertions(+), 8 deletions(-)
drivers/infiniband/core/cm.c +1 -1
@@ -3762,7 +3762,7 @@ static void cm_add_one(struct ib_device *ib_device)int ret;u8 i; -if (!rdma_transport_is_ib (ib_device)) +if (!has_cm (ib_device))return ; cm_dev = kzalloc(sizeof(*cm_dev) + sizeof(*port) *
drivers/infiniband/core/cma.c +4 -6
@@ -433,7 +433,7 @@ static int cma_resolve_ib_dev(struct rdma_id_private *id_priv) pkey =ntohs (addr->sib_pkey);list_for_each_entry (cur_dev, &dev_list, list) { -if (!rdma_transport_is_ib (cur_dev->device)) +if (!has_cm (cur_dev->device))continue ;for (p =1 ; p <= cur_dev->device->phys_port_cnt; ++p) {
@@ -631,7 +631,7 @@ static int cma_modify_qp_rtr(struct rdma_id_private *id_priv,if (ret) goto out; -if (rdma_transport_is_ib (id_priv->cma_dev->device) && +if (has_cm (id_priv->cma_dev->device) &&rdma_port_ll_is_eth (id_priv->id.device, id_priv->id.port_num)) { ret =rdma_addr_find_smac_by_sgid (&sgid, qp_attr.smac,NULL );
@@ -1620,8 +1620,7 @@ static void cma_listen_on_dev(struct rdma_id_private *id_priv,struct rdma_cm_id *id;int ret; -if (cma_family (id_priv) ==AF_IB && - !rdma_transport_is_ib (cma_dev->device)) +if (cma_family (id_priv) ==AF_IB && !has_cm (cma_dev->device))return ; id = rdma_create_id(cma_listen_handler, id_priv, id_priv->id.ps,
@@ -2022,8 +2021,7 @@ static int cma_bind_loopback(struct rdma_id_private *id_priv) cma_dev =NULL ;mutex_lock (&lock);list_for_each_entry (cur_dev, &dev_list, list) { -if (cma_family (id_priv) ==AF_IB && - !rdma_transport_is_ib (cur_dev->device)) +if (cma_family (id_priv) ==AF_IB && !has_cm (cur_dev->device))continue ;if (!cma_dev)
drivers/infiniband/core/ucm.c +1 -1
@@ -1253,7 +1253,7 @@ static void ib_ucm_add_one(struct ib_device *device)dev_t base;struct ib_ucm_device *ucm_dev; -if (!device->alloc_ucontext || !rdma_transport_is_ib (device)) +if (!device->alloc_ucontext || !has_cm (device))return ; ucm_dev =kzalloc (sizeof *ucm_dev,GFP_KERNEL );
include/rdma/ib_verbs.h +13 -0
@@ -1797,6 +1797,19 @@ static inline int has_mad(struct ib_device *device) }/** +* has_cm - Check if a device support Communication Manager. +* +* @device: Device to be checked +* +* Return 0 when a device has none port to support Communication +* Manager. +*/ +static inline int has_cm (struct ib_device *device) +{ +return rdma_transport_is_ib (device); +} + +/** * cap_smi - Check if the port of device has the capability * Subnet Management Interface. *
Signature · 1 line
2.1.0
RFC PATCH 6/11 IB/Verbs: Use management helper has_sa() and cap_sa(), for sa-check
In reply to Michael WangIntroduce helper has_sa() and cap_sa() to help us check if an IB device or it's port support Subnet Administrator.
--- drivers/infiniband/core/sa_query.c | 12 ++++++------ include/rdma/ib_verbs.h | 28 ++++++++++++++++++++++++++++ 2 files changed, 34 insertions(+), 6 deletions(-)
drivers/infiniband/core/sa_query.c +6 -6
@@ -450,7 +450,7 @@ static void ib_sa_event(struct ib_event_handler *handler, struct ib_event *eventstruct ib_sa_port *port = &sa_dev->port[event->element.port_num - sa_dev->start_port]; -if (!rdma_port_ll_is_ib (handler->device, port->port_num)) +if (!cap_sa (handler->device, port->port_num))return ;spin_lock_irqsave (&port->ah_lock, flags);
@@ -1154,7 +1154,7 @@ static void ib_sa_add_one(struct ib_device *device)struct ib_sa_device *sa_dev;int s, e, i; -if (!rdma_transport_is_ib (device)) +if (!has_sa (device))return ;if (device->node_type ==RDMA_NODE_IB_SWITCH )
@@ -1175,7 +1175,7 @@ static void ib_sa_add_one(struct ib_device *device)for (i =0 ; i <= e - s; ++i) {spin_lock_init (&sa_dev->port[i].ah_lock); -if (!rdma_port_ll_is_ib (device, i +1 )) +if (!cap_sa (device, i +1 ))continue ; sa_dev->port[i].sm_ah =NULL ;
@@ -1205,14 +1205,14 @@ static void ib_sa_add_one(struct ib_device *device) goto err;for (i =0 ; i <= e - s; ++i) -if (rdma_port_ll_is_ib (device, i +1 )) +if (cap_sa (device, i +1 ))update_sm_ah (&sa_dev->port[i].update_task);return ; err:while (--i >=0 ) -if (rdma_port_ll_is_ib (device, i +1 )) +if (cap_sa (device, i +1 ))ib_unregister_mad_agent (sa_dev->port[i].agent);kfree (sa_dev);
@@ -1233,7 +1233,7 @@ static void ib_sa_remove_one(struct ib_device *device)flush_workqueue (ib_wq);for (i =0 ; i <= sa_dev->end_port - sa_dev->start_port; ++i) { -if (rdma_port_ll_is_ib (device, i +1 )) { +if (cap_sa (device, i +1 )) {ib_unregister_mad_agent (sa_dev->port[i].agent);if (sa_dev->port[i].sm_ah)kref_put (&sa_dev->port[i].sm_ah->ref, free_sm_ah);
include/rdma/ib_verbs.h +28 -0
@@ -1810,6 +1810,19 @@ static inline int has_cm(struct ib_device *device) }/** +* has_sa - Check if a device support Subnet Administrator. +* +* @device: Device to be checked +* +* Return 0 when a device has none port to support +* Subnet Administrator. +*/ +static inline int has_sa (struct ib_device *device) +{ +return rdma_transport_is_ib (device); +} + +/** * cap_smi - Check if the port of device has the capability * Subnet Management Interface. *
@@ -1824,6 +1837,21 @@ static inline int cap_smi(struct ib_device *device, u8 port_num)return rdma_port_ll_is_ib (device, port_num); } +/** +* cap_sa - Check if the port of device has the capability +* Subnet Administrator. +* +* @device: Device to be checked +* @port_num: Port number of the device +* +* Return 0 when port of the device don't support +* Subnet Administrator. +*/ +static inline int cap_sa (struct ib_device *device,u8 port_num) +{ +return rdma_port_ll_is_ib (device, port_num); +} +int ib_query_gid (struct ib_device *device,u8 port_num,int index,union ib_gid *gid);
Signature · 6 lines
2.1.0 -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [RFC PATCH 06/11] IB/Verbs: Use management helper has_sa() and cap_sa(), for sa-check
Quoted 3 lines · Michael Wang
Introduce helper
has_sa()andcap_sa()to help us check if an IB device or it's port support Subnet Administrator.
I think these 2 should be combined. The question is if a port requires the use of the SA depending on the network it is connected to.
Aren't some dual port Mellanox cards capable of doing IB on 1 port and Eth on the other? Do they show up as 2 devices?
Regardless I think we should define the SA access on a per port basis.
Quoted 25 lines
Cc Jason Gunthorpe <[email protected]>Cc Doug Ledford <[email protected]>Cc Ira Weiny <[email protected]>Cc Sean Hefty <[email protected]>Signed-off-by Michael Wang <[email protected]>--- drivers/infiniband/core/sa_query.c | 12 ++++++------ include/rdma/ib_verbs.h | 28 ++++++++++++++++++++++++++++ 2 files changed, 34 insertions(+), 6 deletions(-)drivers/infiniband/core/sa_query.c +1
-1@@ -450,7 +450,7 @@ static void ib_sa_event(struct ib_event_handler *handler, struct ib_event *eventstruct ib_sa_port *port = &sa_dev->port[event->element.port_num - sa_dev->start_port]; -if (!rdma_port_ll_is_ib (handler->device, port->port_num)) +if (!cap_sa (handler->device, port->port_num))return ;spin_lock_irqsave (&port->ah_lock, flags);
drivers/infiniband/core/sa_query.c static void ib_sa_add_one(struct ib_device *device) · line 1157
Quoted 5 lines
drivers/infiniband/core/sa_query.c +1
-1@@ -1154,7 +1154,7 @@ static void ib_sa_add_one(struct ib_device *device)struct ib_sa_device *sa_dev;int s, e, i; -if (!rdma_transport_is_ib (device)) +if (!has_sa (device))
The logic here should be:
if (no ports of this device need sa access)
return;
So why not eliminate this check and allow the cap_sa(s) to handle the support?
-- Ira
Quoted 87 lines
drivers/infiniband/core/sa_query.c +4
-4return ;if (device->node_type ==RDMA_NODE_IB_SWITCH )@@ -1175,7 +1175,7 @@ static void ib_sa_add_one(struct ib_device *device)for (i =0 ; i <= e - s; ++i) {spin_lock_init (&sa_dev->port[i].ah_lock); -if (!rdma_port_ll_is_ib (device, i +1 )) +if (!cap_sa (device, i +1 ))continue ; sa_dev->port[i].sm_ah =NULL ;@@ -1205,14 +1205,14 @@ static void ib_sa_add_one(struct ib_device *device) goto err;for (i =0 ; i <= e - s; ++i) -if (rdma_port_ll_is_ib (device, i +1 )) +if (cap_sa (device, i +1 ))update_sm_ah (&sa_dev->port[i].update_task);return ; err:while (--i >=0 ) -if (rdma_port_ll_is_ib (device, i +1 )) +if (cap_sa (device, i +1 ))ib_unregister_mad_agent (sa_dev->port[i].agent);kfree (sa_dev);@@ -1233,7 +1233,7 @@ static void ib_sa_remove_one(struct ib_device *device)flush_workqueue (ib_wq);for (i =0 ; i <= sa_dev->end_port - sa_dev->start_port; ++i) { -if (rdma_port_ll_is_ib (device, i +1 )) { +if (cap_sa (device, i +1 )) {ib_unregister_mad_agent (sa_dev->port[i].agent);if (sa_dev->port[i].sm_ah)kref_put (&sa_dev->port[i].sm_ah->ref, free_sm_ah);include/rdma/ib_verbs.h +28
-0@@ -1810,6 +1810,19 @@ static inline int has_cm(struct ib_device *device) }/** +* has_sa - Check if a device support Subnet Administrator. +* +* @device: Device to be checked +* +* Return 0 when a device has none port to support +* Subnet Administrator. +*/ +static inline int has_sa (struct ib_device *device) +{ +return rdma_transport_is_ib (device); +} + +/** * cap_smi - Check if the port of device has the capability * Subnet Management Interface. * @@ -1824,6 +1837,21 @@ static inline int cap_smi(struct ib_device *device, u8 port_num)return rdma_port_ll_is_ib (device, port_num); } +/** +* cap_sa - Check if the port of device has the capability +* Subnet Administrator. +* +* @device: Device to be checked +* @port_num: Port number of the device +* +* Return 0 when port of the device don't support +* Subnet Administrator. +*/ +static inline int cap_sa (struct ib_device *device,u8 port_num) +{ +return rdma_port_ll_is_ib (device, port_num); +} +int ib_query_gid (struct ib_device *device,u8 port_num,int index,union ib_gid *gid);Signature · 1 line
2.1.0
Re: [RFC PATCH 06/11] IB/Verbs: Use management helper has_sa() and cap_sa(), for sa-check
Quoted 10 lines · ira.weiny
Quoted 3 lines · Michael Wang
Introduce helper
has_sa()andcap_sa()to help us check if an IB device or it's port support Subnet Administrator.I think these 2 should be combined. The question is if a port requires the use of the SA depending on the network it is connected to.
Aren't some dual port Mellanox cards capable of doing IB on 1 port and Eth on the other? Do they show up as 2 devices?
I'm not sure about this... but sounds like your opinion is same to Jason on ipoib stuff, I'll do more investigation on that part, if it's just for optimization, then we should be able to eliminate has_xx() stuff :-) after all, init/exit is not a hot path.
Quoted 36 lines
Regardless I think we should define the SA access on a per port basis.
Quoted 25 lines
Cc Jason Gunthorpe <[email protected]>Cc Doug Ledford <[email protected]>Cc Ira Weiny <[email protected]>Cc Sean Hefty <[email protected]>Signed-off-by Michael Wang <[email protected]>--- drivers/infiniband/core/sa_query.c | 12 ++++++------ include/rdma/ib_verbs.h | 28 ++++++++++++++++++++++++++++ 2 files changed, 34 insertions(+), 6 deletions(-)drivers/infiniband/core/sa_query.c +1
-1@@ -450,7 +450,7 @@ static void ib_sa_event(struct ib_event_handler *handler, struct ib_event *eventstruct ib_sa_port *port = &sa_dev->port[event->element.port_num - sa_dev->start_port]; -if (!rdma_port_ll_is_ib (handler->device, port->port_num)) +if (!cap_sa (handler->device, port->port_num))return ;spin_lock_irqsave (&port->ah_lock, flags);drivers/infiniband/core/sa_query.c static void ib_sa_add_one(struct ib_device *device) · line 1157
Quoted 5 lines
drivers/infiniband/core/sa_query.c +1
-1@@ -1154,7 +1154,7 @@ static void ib_sa_add_one(struct ib_device *device)struct ib_sa_device *sa_dev;int s, e, i; -if (!rdma_transport_is_ib (device)) +if (!has_sa (device))The logic here should be:
Quoted 4 lines
if (no ports of this device need sa access)
return;So why not eliminate this check and allow the
cap_sa(s)to handle the support?
I'll check if we could merge this part to the following iteration :-)
Regards,
Michael Wang
Quoted 90 lines
-- Ira
Quoted 87 lines
drivers/infiniband/core/sa_query.c +4
-4return ;if (device->node_type ==RDMA_NODE_IB_SWITCH )@@ -1175,7 +1175,7 @@ static void ib_sa_add_one(struct ib_device *device)for (i =0 ; i <= e - s; ++i) {spin_lock_init (&sa_dev->port[i].ah_lock); -if (!rdma_port_ll_is_ib (device, i +1 )) +if (!cap_sa (device, i +1 ))continue ; sa_dev->port[i].sm_ah =NULL ;@@ -1205,14 +1205,14 @@ static void ib_sa_add_one(struct ib_device *device) goto err;for (i =0 ; i <= e - s; ++i) -if (rdma_port_ll_is_ib (device, i +1 )) +if (cap_sa (device, i +1 ))update_sm_ah (&sa_dev->port[i].update_task);return ; err:while (--i >=0 ) -if (rdma_port_ll_is_ib (device, i +1 )) +if (cap_sa (device, i +1 ))ib_unregister_mad_agent (sa_dev->port[i].agent);kfree (sa_dev);@@ -1233,7 +1233,7 @@ static void ib_sa_remove_one(struct ib_device *device)flush_workqueue (ib_wq);for (i =0 ; i <= sa_dev->end_port - sa_dev->start_port; ++i) { -if (rdma_port_ll_is_ib (device, i +1 )) { +if (cap_sa (device, i +1 )) {ib_unregister_mad_agent (sa_dev->port[i].agent);if (sa_dev->port[i].sm_ah)kref_put (&sa_dev->port[i].sm_ah->ref, free_sm_ah);include/rdma/ib_verbs.h +28
-1@@ -1810,6 +1810,19 @@ static inline int has_cm(struct ib_device *device) }/** +* has_sa - Check if a device support Subnet Administrator. +* +* @device: Device to be checked +* +* Return 0 when a device has none port to support +* Subnet Administrator. +*/ +static inline int has_sa (struct ib_device *device) +{ +return rdma_transport_is_ib (device); +} + +/** * cap_smi - Check if the port of device has the capability * Subnet Management Interface. * @@ -1824,6 +1837,21 @@ static inline int cap_smi(struct ib_device *device, u8 port_num)return rdma_port_ll_is_ib (device, port_num); } +/** +* cap_sa - Check if the port of device has the capability +* Subnet Administrator. +* +* @device: Device to be checked +* @port_num: Port number of the device +* +* Return 0 when port of the device don't support +* Subnet Administrator. +*/ +static inline int cap_sa (struct ib_device *device,u8 port_num) +{ +return rdma_port_ll_is_ib (device, port_num); +} +int ib_query_gid (struct ib_device *device,u8 port_num,int index,union ib_gid *gid); --2.1.0
Re: [RFC PATCH 06/11] IB/Verbs: Use management helper has_sa() and cap_sa(), for sa-check
In reply to ira.weinyQuoted 10 lines · ira.weiny
Quoted 3 lines · Michael Wang
Introduce helper
has_sa()andcap_sa()to help us check if an IB device or it's port support Subnet Administrator.I think these 2 should be combined. The question is if a port requires the use of the SA depending on the network it is connected to.
Aren't some dual port Mellanox cards capable of doing IB on 1 port and Eth on the other?
Yes.
Quoted 1 line
Do they show up as 2 devices?
No. They are two ports of the same device with different link layers. See here:
hca_id: mlx4_1
transport: InfiniBand (0) fw_ver: 2.31.5050 node_guid: f452:1403:007b:cba0 sys_image_guid: f452:1403:007b:cba3 vendor_id: 0x02c9 vendor_part_id: 4099 hw_ver: 0x0 board_id: MT_1090120019 phys_port_cnt: 2 port: 1 state: PORT_ACTIVE (4) max_mtu: 2048 (4) active_mtu: 2048 (4) sm_lid: 2 port_lid: 2 port_lmc: 0x01 link_layer: InfiniBand port: 2 state: PORT_ACTIVE (4) max_mtu: 4096 (5) active_mtu: 4096 (5) sm_lid: 0 port_lid: 0 port_lmc: 0x00 link_layer: Ethernet
Quoted 1 line
Regardless I think we should define the SA access on a per port basis.
Yes.
Quoted 127 lines
Quoted 25 lines
Cc Jason Gunthorpe <[email protected]>Cc Doug Ledford <[email protected]>Cc Ira Weiny <[email protected]>Cc Sean Hefty <[email protected]>Signed-off-by Michael Wang <[email protected]>--- drivers/infiniband/core/sa_query.c | 12 ++++++------ include/rdma/ib_verbs.h | 28 ++++++++++++++++++++++++++++ 2 files changed, 34 insertions(+), 6 deletions(-)drivers/infiniband/core/sa_query.c +1
-1@@ -450,7 +450,7 @@ static void ib_sa_event(struct ib_event_handler *handler, struct ib_event *eventstruct ib_sa_port *port = &sa_dev->port[event->element.port_num - sa_dev->start_port]; -if (!rdma_port_ll_is_ib (handler->device, port->port_num)) +if (!cap_sa (handler->device, port->port_num))return ;spin_lock_irqsave (&port->ah_lock, flags);drivers/infiniband/core/sa_query.c static void ib_sa_add_one(struct ib_device *device) · line 1157
Quoted 5 lines
drivers/infiniband/core/sa_query.c +1
-1@@ -1154,7 +1154,7 @@ static void ib_sa_add_one(struct ib_device *device)struct ib_sa_device *sa_dev;int s, e, i; -if (!rdma_transport_is_ib (device)) +if (!has_sa (device))The logic here should be:
if (no ports of this device need sa access)
return;So why not eliminate this check and allow the
cap_sa(s)to handle the support?-- Ira
Quoted 87 lines
drivers/infiniband/core/sa_query.c +4
-4return ;if (device->node_type ==RDMA_NODE_IB_SWITCH )@@ -1175,7 +1175,7 @@ static void ib_sa_add_one(struct ib_device *device)for (i =0 ; i <= e - s; ++i) {spin_lock_init (&sa_dev->port[i].ah_lock); -if (!rdma_port_ll_is_ib (device, i +1 )) +if (!cap_sa (device, i +1 ))continue ; sa_dev->port[i].sm_ah =NULL ;@@ -1205,14 +1205,14 @@ static void ib_sa_add_one(struct ib_device *device) goto err;for (i =0 ; i <= e - s; ++i) -if (rdma_port_ll_is_ib (device, i +1 )) +if (cap_sa (device, i +1 ))update_sm_ah (&sa_dev->port[i].update_task);return ; err:while (--i >=0 ) -if (rdma_port_ll_is_ib (device, i +1 )) +if (cap_sa (device, i +1 ))ib_unregister_mad_agent (sa_dev->port[i].agent);kfree (sa_dev);@@ -1233,7 +1233,7 @@ static void ib_sa_remove_one(struct ib_device *device)flush_workqueue (ib_wq);for (i =0 ; i <= sa_dev->end_port - sa_dev->start_port; ++i) { -if (rdma_port_ll_is_ib (device, i +1 )) { +if (cap_sa (device, i +1 )) {ib_unregister_mad_agent (sa_dev->port[i].agent);if (sa_dev->port[i].sm_ah)kref_put (&sa_dev->port[i].sm_ah->ref, free_sm_ah);include/rdma/ib_verbs.h +28
-0@@ -1810,6 +1810,19 @@ static inline int has_cm(struct ib_device *device) }/** +* has_sa - Check if a device support Subnet Administrator. +* +* @device: Device to be checked +* +* Return 0 when a device has none port to support +* Subnet Administrator. +*/ +static inline int has_sa (struct ib_device *device) +{ +return rdma_transport_is_ib (device); +} + +/** * cap_smi - Check if the port of device has the capability * Subnet Management Interface. * @@ -1824,6 +1837,21 @@ static inline int cap_smi(struct ib_device *device, u8 port_num)return rdma_port_ll_is_ib (device, port_num); } +/** +* cap_sa - Check if the port of device has the capability +* Subnet Administrator. +* +* @device: Device to be checked +* @port_num: Port number of the device +* +* Return 0 when port of the device don't support +* Subnet Administrator. +*/ +static inline int cap_sa (struct ib_device *device,u8 port_num) +{ +return rdma_port_ll_is_ib (device, port_num); +} +int ib_query_gid (struct ib_device *device,u8 port_num,int index,union ib_gid *gid);Signature · 1 line
2.1.0
Signature · 2 lines
Doug Ledford <[email protected]> GPG KeyID: 0E572FDD
Re: [RFC PATCH 06/11] IB/Verbs: Use management helper has_sa() and cap_sa(), for sa-check
In reply to Michael WangQuoted 2 lines · Michael Wang
Introduce helper
has_sa()andcap_sa()to help us check if an IB device or it's port support Subnet Administrator.
There's no functional reason to have both rdma_transport_is_ib and rdma_port_ll_is_ib, just use one. Then there is also no reason for both has_sa and cap_sa. Just use one.
Quoted 113 lines
Cc Jason Gunthorpe <jgunthorpe-ePGOBjL8dl3ta4EC/59zMFaTQe2KTcn/@public.gmane.org>Cc Doug Ledford <[email protected]>Cc Ira Weiny <[email protected]>Cc Sean Hefty <[email protected]>Signed-off-by Michael Wang <[email protected]>--- drivers/infiniband/core/sa_query.c | 12 ++++++------ include/rdma/ib_verbs.h | 28 ++++++++++++++++++++++++++++ 2 files changed, 34 insertions(+), 6 deletions(-)drivers/infiniband/core/sa_query.c +6
-6@@ -450,7 +450,7 @@ static void ib_sa_event(struct ib_event_handler *handler, struct ib_event *eventstruct ib_sa_port *port = &sa_dev->port[event->element.port_num - sa_dev->start_port]; -if (!rdma_port_ll_is_ib (handler->device, port->port_num)) +if (!cap_sa (handler->device, port->port_num))return ;spin_lock_irqsave (&port->ah_lock, flags);@@ -1154,7 +1154,7 @@ static void ib_sa_add_one(struct ib_device *device)struct ib_sa_device *sa_dev;int s, e, i; -if (!rdma_transport_is_ib (device)) +if (!has_sa (device))return ;if (device->node_type ==RDMA_NODE_IB_SWITCH )@@ -1175,7 +1175,7 @@ static void ib_sa_add_one(struct ib_device *device)for (i =0 ; i <= e - s; ++i) {spin_lock_init (&sa_dev->port[i].ah_lock); -if (!rdma_port_ll_is_ib (device, i +1 )) +if (!cap_sa (device, i +1 ))continue ; sa_dev->port[i].sm_ah =NULL ;@@ -1205,14 +1205,14 @@ static void ib_sa_add_one(struct ib_device *device) goto err;for (i =0 ; i <= e - s; ++i) -if (rdma_port_ll_is_ib (device, i +1 )) +if (cap_sa (device, i +1 ))update_sm_ah (&sa_dev->port[i].update_task);return ; err:while (--i >=0 ) -if (rdma_port_ll_is_ib (device, i +1 )) +if (cap_sa (device, i +1 ))ib_unregister_mad_agent (sa_dev->port[i].agent);kfree (sa_dev);@@ -1233,7 +1233,7 @@ static void ib_sa_remove_one(struct ib_device *device)flush_workqueue (ib_wq);for (i =0 ; i <= sa_dev->end_port - sa_dev->start_port; ++i) { -if (rdma_port_ll_is_ib (device, i +1 )) { +if (cap_sa (device, i +1 )) {ib_unregister_mad_agent (sa_dev->port[i].agent);if (sa_dev->port[i].sm_ah)kref_put (&sa_dev->port[i].sm_ah->ref, free_sm_ah);include/rdma/ib_verbs.h +28
-0@@ -1810,6 +1810,19 @@ static inline int has_cm(struct ib_device *device) }/** +* has_sa - Check if a device support Subnet Administrator. +* +* @device: Device to be checked +* +* Return 0 when a device has none port to support +* Subnet Administrator. +*/ +static inline int has_sa (struct ib_device *device) +{ +return rdma_transport_is_ib (device); +} + +/** * cap_smi - Check if the port of device has the capability * Subnet Management Interface. * @@ -1824,6 +1837,21 @@ static inline int cap_smi(struct ib_device *device, u8 port_num)return rdma_port_ll_is_ib (device, port_num); } +/** +* cap_sa - Check if the port of device has the capability +* Subnet Administrator. +* +* @device: Device to be checked +* @port_num: Port number of the device +* +* Return 0 when port of the device don't support +* Subnet Administrator. +*/ +static inline int cap_sa (struct ib_device *device,u8 port_num) +{ +return rdma_port_ll_is_ib (device, port_num); +} +int ib_query_gid (struct ib_device *device,u8 port_num,int index,union ib_gid *gid);
Signature · 2 lines
Doug Ledford <[email protected]> GPG KeyID: 0E572FDD
Re: [RFC PATCH 06/11] IB/Verbs: Use management helper has_sa() and cap_sa(), for sa-check
Quoted 6 lines · Doug Ledford
Quoted 2 lines · Michael Wang
Introduce helper
has_sa()andcap_sa()to help us check if an IB device or it's port support Subnet Administrator.There's no functional reason to have both
rdma_transport_is_ibandrdma_port_ll_is_ib, just use one. Then there is also no reason for bothhas_saandcap_sa. Just use one.
The has_sa() will be eliminated :-)
rdma_transport_is_ib and rdma_port_ll_is_ib is actually just rough helper to save some code, we can get rid of them when we no longer need them, but currently device driver still using them a lot, I'm not sure if the new mechanism could take cover all these cases...
Regards,
Michael Wang
Quoted 114 lines
Quoted 113 lines
Cc Jason Gunthorpe <[email protected]>Cc Doug Ledford <[email protected]>Cc Ira Weiny <[email protected]>Cc Sean Hefty <[email protected]>Signed-off-by Michael Wang <[email protected]>--- drivers/infiniband/core/sa_query.c | 12 ++++++------ include/rdma/ib_verbs.h | 28 ++++++++++++++++++++++++++++ 2 files changed, 34 insertions(+), 6 deletions(-)drivers/infiniband/core/sa_query.c +6
-6@@ -450,7 +450,7 @@ static void ib_sa_event(struct ib_event_handler *handler, struct ib_event *eventstruct ib_sa_port *port = &sa_dev->port[event->element.port_num - sa_dev->start_port]; -if (!rdma_port_ll_is_ib (handler->device, port->port_num)) +if (!cap_sa (handler->device, port->port_num))return ;spin_lock_irqsave (&port->ah_lock, flags);@@ -1154,7 +1154,7 @@ static void ib_sa_add_one(struct ib_device *device)struct ib_sa_device *sa_dev;int s, e, i; -if (!rdma_transport_is_ib (device)) +if (!has_sa (device))return ;if (device->node_type ==RDMA_NODE_IB_SWITCH )@@ -1175,7 +1175,7 @@ static void ib_sa_add_one(struct ib_device *device)for (i =0 ; i <= e - s; ++i) {spin_lock_init (&sa_dev->port[i].ah_lock); -if (!rdma_port_ll_is_ib (device, i +1 )) +if (!cap_sa (device, i +1 ))continue ; sa_dev->port[i].sm_ah =NULL ;@@ -1205,14 +1205,14 @@ static void ib_sa_add_one(struct ib_device *device) goto err;for (i =0 ; i <= e - s; ++i) -if (rdma_port_ll_is_ib (device, i +1 )) +if (cap_sa (device, i +1 ))update_sm_ah (&sa_dev->port[i].update_task);return ; err:while (--i >=0 ) -if (rdma_port_ll_is_ib (device, i +1 )) +if (cap_sa (device, i +1 ))ib_unregister_mad_agent (sa_dev->port[i].agent);kfree (sa_dev);@@ -1233,7 +1233,7 @@ static void ib_sa_remove_one(struct ib_device *device)flush_workqueue (ib_wq);for (i =0 ; i <= sa_dev->end_port - sa_dev->start_port; ++i) { -if (rdma_port_ll_is_ib (device, i +1 )) { +if (cap_sa (device, i +1 )) {ib_unregister_mad_agent (sa_dev->port[i].agent);if (sa_dev->port[i].sm_ah)kref_put (&sa_dev->port[i].sm_ah->ref, free_sm_ah);include/rdma/ib_verbs.h +28
-0@@ -1810,6 +1810,19 @@ static inline int has_cm(struct ib_device *device) }/** +* has_sa - Check if a device support Subnet Administrator. +* +* @device: Device to be checked +* +* Return 0 when a device has none port to support +* Subnet Administrator. +*/ +static inline int has_sa (struct ib_device *device) +{ +return rdma_transport_is_ib (device); +} + +/** * cap_smi - Check if the port of device has the capability * Subnet Management Interface. * @@ -1824,6 +1837,21 @@ static inline int cap_smi(struct ib_device *device, u8 port_num)return rdma_port_ll_is_ib (device, port_num); } +/** +* cap_sa - Check if the port of device has the capability +* Subnet Administrator. +* +* @device: Device to be checked +* @port_num: Port number of the device +* +* Return 0 when port of the device don't support +* Subnet Administrator. +*/ +static inline int cap_sa (struct ib_device *device,u8 port_num) +{ +return rdma_port_ll_is_ib (device, port_num); +} +int ib_query_gid (struct ib_device *device,u8 port_num,int index,union ib_gid *gid);
Re: [RFC PATCH 06/11] IB/Verbs: Use management helper has_sa() and cap_sa(), for sa-check
Quoted 7 lines · Michael Wang
Quoted 6 lines · Doug Ledford
Quoted 2 lines · Michael Wang
Introduce helper
has_sa()andcap_sa()to help us check if an IB device or it's port support Subnet Administrator.There's no functional reason to have both
rdma_transport_is_ibandrdma_port_ll_is_ib, just use one. Then there is also no reason for bothhas_saandcap_sa. Just use one.
Quoted 6 lines
The
has_sa()will be eliminated :-)
rdma_transport_is_ibandrdma_port_ll_is_ibis actually just rough helper to save some code, we can get rid of them when we no longer need them, but currently device driver still using them a lot, I'm not sure if the new mechanism could take cover all these cases...
Sure it would. This is what I had suggested (well, close to this, I rearranged the order this time around):
enum rdma_transport {
RDMA_TRANSPORT_IB = 0x01,
RDMA_TRANSPORT_OPA = 0x02,
RDMA_TRANSPORT_IWARP = 0x04,
RDMA_TRANSPORT_ROCE_V1 = 0x08,
RDMA_TRANSPORT_ROCE_V2 = 0x10,
};struct ib_port {
...
enum rdma_transport;
...
};static inline bool rdma_transport_is_ib(struct ib_port *port)
{
return port->transport & (RDMA_TRANSPORT_IB | RDMA_TRANSPORT_OPA);
}static inline bool rdma_transport_is_opa(struct ib_port *port)
{
return port->transport & RDMA_TRANSPORT_OPA;
}static inline bool rdma_transport_is_iwarp(struct ib_port *port)
{
return port->transport & RDMA_TRANSPORT_IWARP;
}static inline bool rdma_transport_is_roce(struct ib_port *port)
{
return port->transport & (RDMA_TRANSPORT_ROCE_V1 | RDMA_TRANSPORT_ROCE_V2);
}static inline bool rdma_ib_mgmt(struct ib_port *port)
{
return port->transport & (RDMA_TRANSPORT_IB | RDMA_TRANSPORT_OPA);
}static inline bool rdma_opa_mgmt(struct ib_port *port)
{
return port->transport & RDMA_TRANSPORT_OPA;
}If we use something like this, then the above is all you need. Then every place in the code that checks for something like has_sa or cap_sa can be replaced with rdma_ib_mgmt. When Ira updates his patches for this, he can check for rdma_opa_mgmt to enable jumbo MAD packets and whatever else he needs. Every place that does transport == IB and ll == Ethernet can become rdma_transport_is_roce. Every place that does transport == IB and ll == INFINIBAND becomes rdma_transport_is_ib. The code in multicast.c just needs to check rdma_ib_mgmt() (which happens to make perfect sense anyway as the code in multicast.c that is checking that we are on an IB interface is doing so because IB requires extra management of the multicast group joins/leaves). But, like I said, this is an all or nothing change, it isn't something we can ease into.
Signature · 2 lines
Doug Ledford <[email protected]> GPG KeyID: 0E572FDD
Re: [RFC PATCH 06/11] IB/Verbs: Use management helper has_sa() and cap_sa(), for sa-check
Quoted 9 lines · Doug Ledford
Quoted 6 lines · Michael Wang
[snip]
rdma_transport_is_ibandrdma_port_ll_is_ibis actually just rough helper to save some code, we can get rid of them when we no longer need them, but currently device driver still using them a lot, I'm not sure if the new mechanism could take cover all these cases...[snip]
Quoted 7 lines
If we use something like this, then the above is all you need. Then every place in the code that checks for something like
has_saorcap_sacan be replaced withrdma_ib_mgmt. When Ira updates his patches for this, he can check forrdma_opa_mgmtto enable jumbo MAD packets and whatever else he needs. Every place that does transport == IB and ll == Ethernet can becomerdma_transport_is_roce. Every place that does transport == IB and ll == INFINIBAND becomesrdma_transport_is_ib.
Get your point :-) I need to investigate all those cases see if it works, in case if things unclear, I'll reform it according to my understanding to adapt to this new mechanism, if there are any misunderstanding, we can address them case by case during the review.
Quoted 5 lines
The
code inmulticast.cjust needs to checkrdma_ib_mgmt()(which happens to make perfect sense anyway as the code inmulticast.cthat is checking that we are on an IB interface is doing so because IB requires extra management of the multicast group joins/leaves).
I'll adopt those helpers already been discussed, and they will be implemented with the helpers from new mechanism
(eg, rdma_ib_mgmt) in next version.
Quoted 2 lines
But, like I said, this
is an all or nothing change, it isn't something we can ease into.
It's will be great if we can make it, let's see ;-)
Regards,
Michael Wang
Quoted 1 line
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [RFC PATCH 06/11] IB/Verbs: Use management helper has_sa() and cap_sa(), for sa-check
In reply to Doug LedfordQuoted 3 lines · Doug Ledford
If we use something like this, then the above is all you need. Then every place in the code that checks for something like
has_saorcap_sacan be replaced withrdma_ib_mgmt.
I don't want to see this slide back into some ill defined feature test.
We need to clearly define exactly what these tests are checking, and I think, since we have so much code sharing, the checks should be narrow in scope, not just an entire standard.
I think the basic family of checks Michael identified seemed like a reasonable start.
Going forward we want to NAK stuff like this:
if (rdma_ib_mgmt() || rdma_opa_mgmt()) if (has_sa() || has_opa_foobar())
That indicates we need a new micro feature test.
A big problem here is people working on the core may not know the intricate details of all the families. This will only get worse when proprietary tech like OPA gets added. Documenting requirements via a narrow feature test gives a much higher chance that core stuff will be right via review.
Quoted 2 lines
But, like I said, this is an all or nothing change, it isn't something we can ease into.
Well, we ease into it by introducing the micro tests and then wiping the legacy ones, followed by changing how the driver/core communicates the port and device feature set, ideally to a bitset like you've described.
Michael has tackled the core code, another series could work on the drivers..
Jason
Re: [RFC PATCH 06/11] IB/Verbs: Use management helper has_sa() and cap_sa(), for sa-check
Quoted 6 lines · Jason Gunthorpe
Quoted 3 lines · Doug Ledford
If we use something like this, then the above is all you need. Then every place in the code that checks for something like
has_saorcap_sacan be replaced withrdma_ib_mgmt.
Quoted 6 lines
I don't want to see this slide back into some ill defined feature test.
We need to clearly define exactly what these tests are checking, and I think, since we have so much code sharing, the checks should be narrow in scope, not just an entire standard.
Somewhat agree.
Quoted 8 lines
I think the basic family of checks Michael identified seemed like a reasonable start.
Going forward we want to NAK stuff like this:
if (rdma_ib_mgmt() || rdma_opa_mgmt()) if (has_sa() || has_opa_foobar())
I'm confused.
Is the idea that you would NAK has_sa but be in favor of has_ib_sa?
I think cap_opa_mad is reasonable. And this confuses me why has_opa_foobar would be NAKed.
I believe it is reasonable to flag OPA MAD support as a feature set through a single bit. From this the MAD stack can know if OPA MAD base/class versions are allowed (or treated differently from future IB versions) and if it should processing OPA SM Class DR MADs differently. I don't want devices to be required to supply a list of MAD Base/Class Versions they support.
Quoted 2 lines
That indicates we need a new micro feature test.
The million dollar question is how micro we should go.
More specifically which feature sets can be appropriately communicated with a single bit vs not.
For example, the MAD size is more generally (and perhaps better) communicated via an actual mad_size attribute rather than as part of the OPA MAD feature set.
Another example is the question of is it appropriate to wrap up the single READ SGL entry support within the "is iwarp" flag?
https://www.mail-archive.com/[email protected]/msg23611.html
Right now there is implicit information being gleaned from the "is iwarp" flag. That is that an iWarp device is limited to only 1 READ SGL entry. As this is the only device type which has this limitation _and_ that number is "well known" it works. Even if not the best solution.
The task at hand is to clean up implicit information passing like this while not getting mired down in our own refactoring.
Quoted 6 lines
A big problem here is people working on the core may not know the intricate details of all the families. This will only get worse when proprietary tech like OPA gets added. Documenting requirements via a narrow feature test gives a much higher chance that core stuff will be right via review.
Agreed as long as we get the width right.
Quoted 8 lines
Quoted 3 lines
But, like I said, this is an all or nothing change, it isn't something we can ease into.
Well, we ease into it by introducing the micro tests and then wiping the legacy ones, followed by changing how the driver/core communicates the port and device feature set, ideally to a bitset like you've described.
I think this does need to be eased into. We seem to agree that the feature flags we have now are not granular and/or explicit enough. So lets start cleaning up some of these tests and see how it goes.
Ira
Quoted 5 lines
Michael has tackled the core code, another series could work on the drivers..
Jason
--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [RFC PATCH 06/11] IB/Verbs: Use management helper has_sa() and cap_sa(), for sa-check
Quoted 8 lines · ira.weiny
Quoted 4 lines
Going forward we want to NAK stuff like this:
if (rdma_ib_mgmt() || rdma_opa_mgmt()) if (has_sa() || has_opa_foobar())I'm confused.
Is the idea that you would NAK
has_sabut be in favor ofhas_ib_sa?
It is the || I don't want to see. Feature tests should be one thing.
Quoted 5 lines
I believe it is reasonable to flag OPA MAD support as a feature set through a single bit. From this the MAD stack can know if OPA MAD base/class versions are allowed (or treated differently from future IB versions) and if it should processing OPA SM Class DR MADs differently. I don't want devices to be required to supply a list of MAD Base/Class Versions they support.
That seems reasonable, but we'd gain a is_ib_smp and is_opa_smp test to do that.
Quoted 3 lines
For example, the MAD size is more generally (and perhaps better) communicated via an actual
mad_sizeattribute rather than as part of the OPA MAD feature set.
Start with a bit and change when that makes no sense, MTUs as ints make sense.
Quoted 2 lines
Another example is the question of is it appropriate to wrap up the single READ SGL entry support within the "is iwarp" flag?
Again, stick with a single bit test until a HW vendors needs more, then someone can clean it. At least they now know what to clean and why.
Remember, this is in the kernel, we can change it when it outlives its life.
Jason
--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
RFC PATCH 7/11 IB/Verbs: Use management helper has_mcast() and, cap_mcast() for mcast-check
In reply to Michael WangIntroduce helper has_mcast() and cap_mcast() to help us check if an IB device or it's port support Multicast.
--- drivers/infiniband/core/cma.c | 2 +- drivers/infiniband/core/multicast.c | 8 ++++---- include/rdma/ib_verbs.h | 28 ++++++++++++++++++++++++++++ 3 files changed, 33 insertions(+), 5 deletions(-)
drivers/infiniband/core/cma.c +1 -1
@@ -3398,7 +3398,7 @@ void rdma_leave_multicast(struct rdma_cm_id *id, struct sockaddr *addr)ib_detach_mcast (id->qp, &mc->multicast.ib->rec.mgid,be16_to_cpu (mc->multicast.ib->rec.mlid)); -if (rdma_transport_is_ib (id_priv->cma_dev->device)) { +if (has_mcast (id_priv->cma_dev->device)) {switch (rdma_port_get_link_layer (id->device, id->port_num)) {case IB_LINK_LAYER_INFINIBAND :ib_sa_free_multicast (mc->multicast.ib);
drivers/infiniband/core/multicast.c +4 -4
@@ -780,7 +780,7 @@ static void mcast_event_handler(struct ib_event_handler *handler,int index; dev = container_of(handler, struct mcast_device, event_handler); -if (!rdma_port_ll_is_ib (dev->device, event->element.port_num)) +if (!cap_mcast (dev->device, event->element.port_num))return ; index = event->element.port_num - dev->start_port;
@@ -807,7 +807,7 @@ static void mcast_add_one(struct ib_device *device)int i;int count =0 ; -if (!rdma_transport_is_ib (device)) +if (!has_mcast (device))return ; dev = kmalloc(sizeof *dev + device->phys_port_cnt * sizeof *port,
@@ -823,7 +823,7 @@ static void mcast_add_one(struct ib_device *device) }for (i =0 ; i <= dev->end_port - dev->start_port; i++) { -if (!rdma_port_ll_is_ib (device, dev->start_port + i)) +if (!cap_mcast (device, dev->start_port + i))continue ; port = &dev->port[i]; port->dev = dev;
@@ -861,7 +861,7 @@ static void mcast_remove_one(struct ib_device *device)flush_workqueue (mcast_wq);for (i =0 ; i <= dev->end_port - dev->start_port; i++) { -if (rdma_port_ll_is_ib (device, dev->start_port + i)) { +if (cap_mcast (device, dev->start_port + i)) { port = &dev->port[i];deref_port (port);wait_for_completion (&port->comp);
include/rdma/ib_verbs.h +28 -0
@@ -1823,6 +1823,19 @@ static inline int has_sa(struct ib_device *device) }/** +* has_mcast - Check if a device support Multicast. +* +* @device: Device to be checked +* +* Return 0 when a device has none port to support +* Multicast. +*/ +static inline int has_mcast (struct ib_device *device) +{ +return rdma_transport_is_ib (device); +} + +/** * cap_smi - Check if the port of device has the capability * Subnet Management Interface. *
@@ -1852,6 +1865,21 @@ static inline int cap_sa(struct ib_device *device, u8 port_num)return rdma_port_ll_is_ib (device, port_num); } +/** +* cap_mcast - Check if the port of device has the capability +* Multicast. +* +* @device: Device to be checked +* @port_num: Port number of the device +* +* Return 0 when port of the device don't support +* Multicast. +*/ +static inline int cap_mcast (struct ib_device *device,u8 port_num) +{ +return rdma_port_ll_is_ib (device, port_num); +} +int ib_query_gid (struct ib_device *device,u8 port_num,int index,union ib_gid *gid);
Signature · 6 lines
2.1.0 -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [RFC PATCH 07/11] IB/Verbs: Use management helper has_mcast() and, cap_mcast() for mcast-check
Quoted 18 lines · Michael Wang
Introduce helper
has_mcast()andcap_mcast()to help us check if an IB device or it's port support Multicast.Cc Jason Gunthorpe <jgunthorpe-ePGOBjL8dl3ta4EC/59zMFaTQe2KTcn/@public.gmane.org>Cc Doug Ledford <[email protected]>Cc Ira Weiny <[email protected]>Cc Sean Hefty <[email protected]>Signed-off-by Michael Wang <[email protected]>drivers/infiniband/core/cma.c | 2 +- drivers/infiniband/core/multicast.c | 8 ++++---- include/rdma/ib_verbs.h | 28 ++++++++++++++++++++++++++++ 3 files changed, 33 insertions(+), 5 deletions(-)drivers/infiniband/core/cma.c +1
-0+++ b/drivers/infiniband/core/cma.c
drivers/infiniband/core/cma.c void rdma_leave_multicast(struct rdma_cm_id *id, struct sockaddr *addr) · line 3401
Quoted 5 lines
drivers/infiniband/core/cma.c +1
-1@@ -3398,7 +3398,7 @@ void rdma_leave_multicast(struct rdma_cm_id *id, struct sockaddr *addr)ib_detach_mcast (id->qp, &mc->multicast.ib->rec.mgid,be16_to_cpu (mc->multicast.ib->rec.mlid)); -if (rdma_transport_is_ib (id_priv->cma_dev->device)) { +if (has_mcast (id_priv->cma_dev->device)) {
This might make more sense as cap_ib_multicast / cap_ip_multicast
Quoted 8 lines
drivers/infiniband/core/cma.c +0
-0switch (rdma_port_get_link_layer (id->device, id->port_num)) {case IB_LINK_LAYER_INFINIBAND :ib_sa_free_multicast (mc->multicast.ib);drivers/infiniband/core/multicast.c +1
-0+++ b/drivers/infiniband/core/multicast.c
drivers/infiniband/core/multicast.c static void mcast_event_handler(struct ib_event_handler *handler, · line 784
Quoted 6 lines
drivers/infiniband/core/multicast.c +1
-1@@ -780,7 +780,7 @@ static void mcast_event_handler(struct ib_event_handler *handler,int index; dev = container_of(handler, struct mcast_device, event_handler); -if (!rdma_port_ll_is_ib (dev->device, event->element.port_num)) +if (!cap_mcast (dev->device, event->element.port_num))return ;
These should probably be cap_ib_sa - that is what they are guarding against.
But it seems redudent, since mcast_add_one will already not add a port that is not IB, so mcast_event_handler is not callable. Something to do with rocee/ib switching?
drivers/infiniband/core/multicast.c static void mcast_add_one(struct ib_device *device) · line 811
Quoted 8 lines
drivers/infiniband/core/multicast.c +1
-1index = event->element.port_num - dev->start_port;@@ -807,7 +807,7 @@ static void mcast_add_one(struct ib_device *device)int i;int count =0 ; -if (!rdma_transport_is_ib (device)) +if (!has_mcast (device))return ;
Again, this seems redundant, every port is tested directly below, why is this check needed?
Looking at this, I do wonder how a port can dynamically change between rocee and IB.. If the link value changes then mcast_remove_one will not be a perfect reversal of mcast_add_one. Bug?
It feels necessary to understand what happens when a port dynamically switches to ethernet on mlx hardware to validate these patches :(
Jason
--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [RFC PATCH 07/11] IB/Verbs: Use management helper has_mcast() and, cap_mcast() for mcast-check
Quoted 15 lines · Jason Gunthorpe
Quoted 14 lines · Michael Wang
Introduce helper
has_mcast()andcap_mcast()to help us check if an IB device or it's port support Multicast.Cc Jason Gunthorpe <[email protected]>Cc Doug Ledford <[email protected]>Cc Ira Weiny <[email protected]>Cc Sean Hefty <[email protected]>Signed-off-by Michael Wang <[email protected]>drivers/infiniband/core/cma.c | 2 +- drivers/infiniband/core/multicast.c | 8 ++++---- include/rdma/ib_verbs.h | 28 ++++++++++++++++++++++++++++ 3 files changed, 33 insertions(+), 5 deletions(-)
Quoted 11 lines
drivers/infiniband/core/cma.c void rdma_leave_multicast(struct rdma_cm_id *id, struct sockaddr *addr) · line 3401
Quoted 9 lines
drivers/infiniband/core/cma.c +2
-1+++ b/drivers/infiniband/core/cma.c@@ -3398,7 +3398,7 @@ void rdma_leave_multicast(struct rdma_cm_id *id, struct sockaddr *addr)ib_detach_mcast (id->qp, &mc->multicast.ib->rec.mgid,be16_to_cpu (mc->multicast.ib->rec.mlid)); -if (rdma_transport_is_ib (id_priv->cma_dev->device)) { +if (has_mcast (id_priv->cma_dev->device)) {This might make more sense as
cap_ib_multicast/cap_ip_multicast
Agreed.
Quoted 19 lines
Quoted 9 lines
drivers/infiniband/core/cma.c +0
-0switch (rdma_port_get_link_layer (id->device, id->port_num)) {case IB_LINK_LAYER_INFINIBAND :ib_sa_free_multicast (mc->multicast.ib);drivers/infiniband/core/multicast.c +1
-0+++ b/drivers/infiniband/core/multicast.cdrivers/infiniband/core/multicast.c static void mcast_event_handler(struct ib_event_handler *handler, · line 784
Quoted 6 lines
drivers/infiniband/core/multicast.c +1
-1@@ -780,7 +780,7 @@ static void mcast_event_handler(struct ib_event_handler *handler,int index; dev = container_of(handler, struct mcast_device, event_handler); -if (!rdma_port_ll_is_ib (dev->device, event->element.port_num)) +if (!cap_mcast (dev->device, event->element.port_num))return ;These should probably be
cap_ib_sa- that is what they are guarding against.
Quoted 3 lines
But it seems redudent, since
mcast_add_onewill already not add a port that is not IB, somcast_event_handleris not callable. Something to do with rocee/ib switching?
I'm not sure about this either. This check seems to be necessary only on a per-port level. It does seem apparent that one can't go from Eth to IB. What happens if you go from IB to Eth on the port?
Quoted 6 lines
Quoted 6 lines
drivers/infiniband/core/multicast.c +0
-0index = event->element.port_num - dev->start_port;@@ -807,7 +807,7 @@ static void mcast_add_one(struct ib_device *device)int i;int count =0 ;
Quoted 6 lines
drivers/infiniband/core/multicast.c static void mcast_add_one(struct ib_device *device) · line 811
Quoted 3 lines
drivers/infiniband/core/multicast.c +1
-1-if (!rdma_transport_is_ib (device)) +if (!has_mcast (device))return ;Again, this seems redundant, every port is tested directly below, why is this check needed?
Agreed. Same as my comments about the SA support. This is really only needed on ports which need to register with the SA (or perhaps some future entity) for Mcast support.
Also this is part of the ib_sa module and exports the function ib_sa_join_multicast. So that this point it is covered under the cap_sa(device, port) call.
So the implementation of cap_mcast at this point is:
cap_mcast(device, port)
{
return cap_sa(device,port);
}Quoted 7 lines
Looking at this, I do wonder how a port can dynamically change between rocee and IB.. If the link value changes then
mcast_remove_onewill not be a perfect reversal ofmcast_add_one. Bug?It feels necessary to understand what happens when a port dynamically switches to ethernet on mlx hardware to validate these patches :(
Agreed.
:-(
-- Ira
Quoted 2 lines
Jason
Re: [RFC PATCH 07/11] IB/Verbs: Use management helper has_mcast() and, cap_mcast() for mcast-check
Quoted 3 lines · ira.weiny
Quoted 2 lines · Jason Gunthorpe
Quoted 1 line · Michael Wang
[snip]
Quoted 7 lines
Quoted 4 lines
drivers/infiniband/core/cma.c void rdma_leave_multicast(struct rdma_cm_id *id, struct sockaddr *addr) · line 3401
Quoted 2 lines
drivers/infiniband/core/cma.c +1
-1-if (rdma_transport_is_ib (id_priv->cma_dev->device)) { +if (has_mcast (id_priv->cma_dev->device)) {This might make more sense as
cap_ib_multicast/cap_ip_multicastAgreed.
Will be changed in next version :-)
Quoted 19 lines
Quoted 19 lines
Quoted 9 lines
switch (rdma_port_get_link_layer(id->device, id->port_num)) { case IB_LINK_LAYER_INFINIBAND: ib_sa_free_multicast(mc->multicast.ib);drivers/infiniband/core/multicast.c +1
-0+++ b/drivers/infiniband/core/multicast.cdrivers/infiniband/core/multicast.c static void mcast_event_handler(struct ib_event_handler *handler, · line 784
Quoted 6 lines
drivers/infiniband/core/multicast.c +1
-1@@ -780,7 +780,7 @@ static void mcast_event_handler(struct ib_event_handler *handler,int index; dev = container_of(handler, struct mcast_device, event_handler); -if (!rdma_port_ll_is_ib (dev->device, event->element.port_num)) +if (!cap_mcast (dev->device, event->element.port_num))return ;These should probably be
cap_ib_sa- that is what they are guarding against.
Quoted 7 lines
Quoted 3 lines
But it seems redudent, since
mcast_add_onewill already not add a port that is not IB, somcast_event_handleris not callable. Something to do with rocee/ib switching?I'm not sure about this either. This check seems to be necessary only on a per-port level. It does seem apparent that one can't go from Eth to IB. What happens if you go from IB to Eth on the port?
I also feel it's redundant at first glance, but just not sure if it could be removed, lack of some knowledge :-P
Quoted 14 lines
Quoted 7 lines
Quoted 7 lines
drivers/infiniband/core/multicast.c +0
-0index = event->element.port_num - dev->start_port;@@ -807,7 +807,7 @@ static void mcast_add_one(struct ib_device *device)int i;int count =0 ;Quoted 6 lines
drivers/infiniband/core/multicast.c static void mcast_add_one(struct ib_device *device) · line 811
Quoted 3 lines
drivers/infiniband/core/multicast.c +1
-1-if (!rdma_transport_is_ib (device)) +if (!has_mcast (device))return ;Again, this seems redundant, every port is tested directly below, why is this check needed?
Quoted 3 lines
Agreed. Same as my comments about the SA support. This is really only needed on ports which need to register with the SA (or perhaps some future entity) for Mcast support.
I will recheck all the logical around has_XX() see if we can get rid of them ;-)
Quoted 5 lines
Also this is part of the
ib_samodule and exports the functionib_sa_join_multicast. So that this point it is covered under thecap_sa(device, port)call.
Quoted 7 lines
So the implementation of
cap_mcastat this point is:
cap_mcast(device, port){ return cap_sa(device,port); }
Sounds good :-) will be in next version.
Quoted 9 lines
Quoted 7 lines
Looking at this, I do wonder how a port can dynamically change between rocee and IB.. If the link value changes then
mcast_remove_onewill not be a perfect reversal ofmcast_add_one. Bug?It feels necessary to understand what happens when a port dynamically switches to ethernet on mlx hardware to validate these patches :(
Agreed.
Maybe we can temporarily reserve the old logical, and gradually solve these problems?
Regards,
Michael Wang
Quoted 7 lines
:-(
-- Ira
Quoted 2 lines
Jason
--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [RFC PATCH 07/11] IB/Verbs: Use management helper has_mcast() and, cap_mcast() for mcast-check
Quoted 2 lines · Yun Wang
Maybe we can temporarily reserve the old logical, and gradually solve these problems?
It is best to make behavioral changes in small patches, yes.
I think it is best to address these sorts of problems before trying to tackle the driver interface side - that will avoid complexity.
ie how does a driver set has_ipoib? It isn't even a sensible question when it is a per port capability.
Jason
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [RFC PATCH 07/11] IB/Verbs: Use management helper has_mcast() and, cap_mcast() for mcast-check
Quoted 5 lines · Jason Gunthorpe
Quoted 2 lines · Yun Wang
Maybe we can temporarily reserve the old logical, and gradually solve these problems?
Quoted 7 lines
It is best to make behavioral changes in small patches, yes.
I think it is best to address these sorts of problems before trying to tackle the driver interface side - that will avoid complexity.
ie how does a driver set
has_ipoib? It isn't even a sensible question when it is a per port capability.
I used to imaging it would like:
init_device_mgmt_attributs(device)
{
for_each_port(device)
if (port support XX) {
port.mgmt_attribute |= CAP_XX
if !(device.mgmt_attribute & HAS_XX)
device.mgmt_attribute |= HAS_XX
}
}That is incase if a device got one port have some capability, the device have it too, so has_XX() will check on device level that if it has any port support XX, and cap_XX() to check on port level.
But if has_XX() is only for optimizing the init/exit path, then it doesn't make sense to me any more...
Regards,
Michael Wang
Quoted 2 lines
Jason
Re: [RFC PATCH 07/11] IB/Verbs: Use management helper has_mcast() and, cap_mcast() for mcast-check
In reply to ira.weinyQuoted 7 lines · ira.weiny
Quoted 3 lines
But it seems redudent, since
mcast_add_onewill already not add a port that is not IB, somcast_event_handleris not callable. Something to do with rocee/ib switching?I'm not sure about this either. This check seems to be necessary only on a per-port level. It does seem apparent that one can't go from Eth to IB. What happens if you go from IB to Eth on the port?
Hmm... I see a mlx4_change_port_types which ultimately calls ib_unregister_device, which suggests the port type doesn't change at runtime (yay)
So maybe these checks really are redundant?
Jason
--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [RFC PATCH 07/11] IB/Verbs: Use management helper has_mcast() and, cap_mcast() for mcast-check
Quoted 11 lines · Jason Gunthorpe
Quoted 6 lines · ira.weiny
Quoted 3 lines
But it seems redudent, since
mcast_add_onewill already not add a port that is not IB, somcast_event_handleris not callable. Something to do with rocee/ib switching?I'm not sure about this either. This check seems to be necessary only on a per-port level. It does seem apparent that one can't go from Eth to IB. What happens if you go from IB to Eth on the port?
Hmm... I see a
mlx4_change_port_typeswhich ultimately callsib_unregister_device, which suggests the port type doesn't change at runtime (yay)
Yeah, seems like mlx4 will reinitialize the device when port link layer changed.
I've take a look at other HW, they directly return a static type or infer from transport type (I suppose this won't change dynamically).
Thus I also agreed check inside mcast_event_handler() is unnecessary, maybe we can change that logical to WARN_ON(!cap_mcast()) ?
Regards,
Michael Wang
Quoted 4 lines
So maybe these checks really are redundant?
Jason
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [RFC PATCH 07/11] IB/Verbs: Use management helper has_mcast() and, cap_mcast() for mcast-check
Quoted 2 lines · Michael Wang
Thus I also agreed check inside
mcast_event_handler()is unnecessary, maybe we can change that logical to WARN_ON(!cap_mcast()) ?
Seems reasonable to me.
Jason
--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [RFC PATCH 07/11] IB/Verbs: Use management helper has_mcast() and, cap_mcast() for mcast-check
In reply to Michael WangQuoted 2 lines · Michael Wang
Introduce helper
has_mcast()andcap_mcast()to help us check if an IB device or it's port support Multicast.
This probably needs reworded or rethought. In truth, *all* rdma devices are multicast capable. *BUT*, IB/OPA devices require multicast registration done the IB way (including for sendonly multicast sends), while Ethernet devices do multicast the Ethernet way. These tests are really just for IB specific multicast registration and deregistration. Call it has_mcast() and cap_mcast() is incorrect.
Quoted 110 lines
Cc Jason Gunthorpe <jgunthorpe-ePGOBjL8dl3ta4EC/59zMFaTQe2KTcn/@public.gmane.org>Cc Doug Ledford <[email protected]>Cc Ira Weiny <[email protected]>Cc Sean Hefty <[email protected]>Signed-off-by Michael Wang <[email protected]>--- drivers/infiniband/core/cma.c | 2 +- drivers/infiniband/core/multicast.c | 8 ++++---- include/rdma/ib_verbs.h | 28 ++++++++++++++++++++++++++++ 3 files changed, 33 insertions(+), 5 deletions(-)drivers/infiniband/core/cma.c +1
-1@@ -3398,7 +3398,7 @@ void rdma_leave_multicast(struct rdma_cm_id *id, struct sockaddr *addr)ib_detach_mcast (id->qp, &mc->multicast.ib->rec.mgid,be16_to_cpu (mc->multicast.ib->rec.mlid)); -if (rdma_transport_is_ib (id_priv->cma_dev->device)) { +if (has_mcast (id_priv->cma_dev->device)) {switch (rdma_port_get_link_layer (id->device, id->port_num)) {case IB_LINK_LAYER_INFINIBAND :ib_sa_free_multicast (mc->multicast.ib);drivers/infiniband/core/multicast.c +4
-4@@ -780,7 +780,7 @@ static void mcast_event_handler(struct ib_event_handler *handler,int index; dev = container_of(handler, struct mcast_device, event_handler); -if (!rdma_port_ll_is_ib (dev->device, event->element.port_num)) +if (!cap_mcast (dev->device, event->element.port_num))return ; index = event->element.port_num - dev->start_port;@@ -807,7 +807,7 @@ static void mcast_add_one(struct ib_device *device)int i;int count =0 ; -if (!rdma_transport_is_ib (device)) +if (!has_mcast (device))return ; dev = kmalloc(sizeof *dev + device->phys_port_cnt * sizeof *port,@@ -823,7 +823,7 @@ static void mcast_add_one(struct ib_device *device) }for (i =0 ; i <= dev->end_port - dev->start_port; i++) { -if (!rdma_port_ll_is_ib (device, dev->start_port + i)) +if (!cap_mcast (device, dev->start_port + i))continue ; port = &dev->port[i]; port->dev = dev;@@ -861,7 +861,7 @@ static void mcast_remove_one(struct ib_device *device)flush_workqueue (mcast_wq);for (i =0 ; i <= dev->end_port - dev->start_port; i++) { -if (rdma_port_ll_is_ib (device, dev->start_port + i)) { +if (cap_mcast (device, dev->start_port + i)) { port = &dev->port[i];deref_port (port);wait_for_completion (&port->comp);include/rdma/ib_verbs.h +28
-0@@ -1823,6 +1823,19 @@ static inline int has_sa(struct ib_device *device) }/** +* has_mcast - Check if a device support Multicast. +* +* @device: Device to be checked +* +* Return 0 when a device has none port to support +* Multicast. +*/ +static inline int has_mcast (struct ib_device *device) +{ +return rdma_transport_is_ib (device); +} + +/** * cap_smi - Check if the port of device has the capability * Subnet Management Interface. * @@ -1852,6 +1865,21 @@ static inline int cap_sa(struct ib_device *device, u8 port_num)return rdma_port_ll_is_ib (device, port_num); } +/** +* cap_mcast - Check if the port of device has the capability +* Multicast. +* +* @device: Device to be checked +* @port_num: Port number of the device +* +* Return 0 when port of the device don't support +* Multicast. +*/ +static inline int cap_mcast (struct ib_device *device,u8 port_num) +{ +return rdma_port_ll_is_ib (device, port_num); +} +int ib_query_gid (struct ib_device *device,u8 port_num,int index,union ib_gid *gid);
Signature · 2 lines
Doug Ledford <[email protected]> GPG KeyID: 0E572FDD
Re: [RFC PATCH 07/11] IB/Verbs: Use management helper has_mcast() and, cap_mcast() for mcast-check
Quoted 9 lines · Doug Ledford
Quoted 2 lines · Michael Wang
Introduce helper
has_mcast()andcap_mcast()to help us check if an IB device or it's port support Multicast.This probably needs reworded or rethought. In truth, *all* rdma devices are multicast capable. *BUT*, IB/OPA devices require multicast registration done the IB way (including for sendonly multicast sends), while Ethernet devices do multicast the Ethernet way. These tests are really just for IB specific multicast registration and deregistration. Call it
has_mcast()andcap_mcast()is incorrect.
Thanks for the explanation :-)
Jason also mentioned we should use cap_ib_XX() instead, I'll use that name then we can distinguish the management between Eth and IB/OPA.
Regards,
Michael Wang
Quoted 112 lines
Quoted 111 lines
Cc Jason Gunthorpe <[email protected]>Cc Doug Ledford <[email protected]>Cc Ira Weiny <[email protected]>Cc Sean Hefty <[email protected]>Signed-off-by Michael Wang <[email protected]>--- drivers/infiniband/core/cma.c | 2 +- drivers/infiniband/core/multicast.c | 8 ++++---- include/rdma/ib_verbs.h | 28 ++++++++++++++++++++++++++++ 3 files changed, 33 insertions(+), 5 deletions(-)drivers/infiniband/core/cma.c +1
-1@@ -3398,7 +3398,7 @@ void rdma_leave_multicast(struct rdma_cm_id *id, struct sockaddr *addr)ib_detach_mcast (id->qp, &mc->multicast.ib->rec.mgid,be16_to_cpu (mc->multicast.ib->rec.mlid)); -if (rdma_transport_is_ib (id_priv->cma_dev->device)) { +if (has_mcast (id_priv->cma_dev->device)) {switch (rdma_port_get_link_layer (id->device, id->port_num)) {case IB_LINK_LAYER_INFINIBAND :ib_sa_free_multicast (mc->multicast.ib);drivers/infiniband/core/multicast.c +4
-4@@ -780,7 +780,7 @@ static void mcast_event_handler(struct ib_event_handler *handler,int index; dev = container_of(handler, struct mcast_device, event_handler); -if (!rdma_port_ll_is_ib (dev->device, event->element.port_num)) +if (!cap_mcast (dev->device, event->element.port_num))return ; index = event->element.port_num - dev->start_port;@@ -807,7 +807,7 @@ static void mcast_add_one(struct ib_device *device)int i;int count =0 ; -if (!rdma_transport_is_ib (device)) +if (!has_mcast (device))return ; dev = kmalloc(sizeof *dev + device->phys_port_cnt * sizeof *port,@@ -823,7 +823,7 @@ static void mcast_add_one(struct ib_device *device) }for (i =0 ; i <= dev->end_port - dev->start_port; i++) { -if (!rdma_port_ll_is_ib (device, dev->start_port + i)) +if (!cap_mcast (device, dev->start_port + i))continue ; port = &dev->port[i]; port->dev = dev;@@ -861,7 +861,7 @@ static void mcast_remove_one(struct ib_device *device)flush_workqueue (mcast_wq);for (i =0 ; i <= dev->end_port - dev->start_port; i++) { -if (rdma_port_ll_is_ib (device, dev->start_port + i)) { +if (cap_mcast (device, dev->start_port + i)) { port = &dev->port[i];deref_port (port);wait_for_completion (&port->comp);include/rdma/ib_verbs.h +28
-0@@ -1823,6 +1823,19 @@ static inline int has_sa(struct ib_device *device) }/** +* has_mcast - Check if a device support Multicast. +* +* @device: Device to be checked +* +* Return 0 when a device has none port to support +* Multicast. +*/ +static inline int has_mcast (struct ib_device *device) +{ +return rdma_transport_is_ib (device); +} + +/** * cap_smi - Check if the port of device has the capability * Subnet Management Interface. * @@ -1852,6 +1865,21 @@ static inline int cap_sa(struct ib_device *device, u8 port_num)return rdma_port_ll_is_ib (device, port_num); } +/** +* cap_mcast - Check if the port of device has the capability +* Multicast. +* +* @device: Device to be checked +* @port_num: Port number of the device +* +* Return 0 when port of the device don't support +* Multicast. +*/ +static inline int cap_mcast (struct ib_device *device,u8 port_num) +{ +return rdma_port_ll_is_ib (device, port_num); +} +int ib_query_gid (struct ib_device *device,u8 port_num,int index,union ib_gid *gid);
Re: [RFC PATCH 07/11] IB/Verbs: Use management helper has_mcast() and, cap_mcast() for mcast-check
Quoted 33 lines · Michael Wang
Quoted 9 lines · Doug Ledford
Quoted 2 lines · Michael Wang
Introduce helper
has_mcast()andcap_mcast()to help us check if an IB device or it's port support Multicast.This probably needs reworded or rethought. In truth, *all* rdma devices are multicast capable. *BUT*, IB/OPA devices require multicast registration done the IB way (including for sendonly multicast sends), while Ethernet devices do multicast the Ethernet way. These tests are really just for IB specific multicast registration and deregistration. Call it
has_mcast()andcap_mcast()is incorrect.Thanks for the explanation :-)
Jason also mentioned we should use
cap_ib_XX()instead, I'll use that name then we can distinguish the management between Eth and IB/OPA.Regards,
Michael WangQuoted 12 lines
Quoted 12 lines
Cc Jason Gunthorpe <[email protected]>Cc Doug Ledford <[email protected]>Cc Ira Weiny <[email protected]>Cc Sean Hefty <[email protected]>Signed-off-by Michael Wang <[email protected]>--- drivers/infiniband/core/cma.c | 2 +- drivers/infiniband/core/multicast.c | 8 ++++---- include/rdma/ib_verbs.h | 28 ++++++++++++++++++++++++++++ 3 files changed, 33 insertions(+), 5 deletions(-)
Quoted 10 lines
Quoted 10 lines
drivers/infiniband/core/cma.c void rdma_leave_multicast(struct rdma_cm_id *id, struct sockaddr *addr) · line 3401
Quoted 10 lines
drivers/infiniband/core/cma.c +1
-1@@ -3398,7 +3398,7 @@ void rdma_leave_multicast(struct rdma_cm_id *id, struct sockaddr *addr)ib_detach_mcast (id->qp, &mc->multicast.ib->rec.mgid,be16_to_cpu (mc->multicast.ib->rec.mlid)); -if (rdma_transport_is_ib (id_priv->cma_dev->device)) { +if (has_mcast (id_priv->cma_dev->device)) {
You need a similar check in rdma_join_multicast.
Ira
Quoted 95 lines
Quoted 81 lines
Quoted 81 lines
drivers/infiniband/core/cma.c +0
-0switch (rdma_port_get_link_layer (id->device, id->port_num)) {case IB_LINK_LAYER_INFINIBAND :ib_sa_free_multicast (mc->multicast.ib);drivers/infiniband/core/multicast.c +4
-4@@ -780,7 +780,7 @@ static void mcast_event_handler(struct ib_event_handler *handler,int index; dev = container_of(handler, struct mcast_device, event_handler); -if (!rdma_port_ll_is_ib (dev->device, event->element.port_num)) +if (!cap_mcast (dev->device, event->element.port_num))return ; index = event->element.port_num - dev->start_port;@@ -807,7 +807,7 @@ static void mcast_add_one(struct ib_device *device)int i;int count =0 ; -if (!rdma_transport_is_ib (device)) +if (!has_mcast (device))return ; dev = kmalloc(sizeof *dev + device->phys_port_cnt * sizeof *port,@@ -823,7 +823,7 @@ static void mcast_add_one(struct ib_device *device) }for (i =0 ; i <= dev->end_port - dev->start_port; i++) { -if (!rdma_port_ll_is_ib (device, dev->start_port + i)) +if (!cap_mcast (device, dev->start_port + i))continue ; port = &dev->port[i]; port->dev = dev;@@ -861,7 +861,7 @@ static void mcast_remove_one(struct ib_device *device)flush_workqueue (mcast_wq);for (i =0 ; i <= dev->end_port - dev->start_port; i++) { -if (rdma_port_ll_is_ib (device, dev->start_port + i)) { +if (cap_mcast (device, dev->start_port + i)) { port = &dev->port[i];deref_port (port);wait_for_completion (&port->comp);include/rdma/ib_verbs.h +23
-0@@ -1823,6 +1823,19 @@ static inline int has_sa(struct ib_device *device) }/** +* has_mcast - Check if a device support Multicast. +* +* @device: Device to be checked +* +* Return 0 when a device has none port to support +* Multicast. +*/ +static inline int has_mcast (struct ib_device *device) +{ +return rdma_transport_is_ib (device); +} + +/** * cap_smi - Check if the port of device has the capability * Subnet Management Interface. * @@ -1852,6 +1865,21 @@ static inline int cap_sa(struct ib_device *device, u8 port_num)return rdma_port_ll_is_ib (device, port_num); } +/** +* cap_mcast - Check if the port of device has the capability +* Multicast. +* +* @device: Device to be checked +* @port_num: Port number of the device +* +* Return 0 when port of the device don't support +* Multicast. +*/ Quoted 9 lines
include/rdma/ib_verbs.h static inline int cap_sa(struct ib_device *device, u8 port_num) · line 1885
Quoted 8 lines
include/rdma/ib_verbs.h +5
-0+static inline int cap_mcast (struct ib_device *device,u8 port_num) +{ +return rdma_port_ll_is_ib (device, port_num); +} +int ib_query_gid (struct ib_device *device,u8 port_num,int index,union ib_gid *gid);--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [RFC PATCH 07/11] IB/Verbs: Use management helper has_mcast() and, cap_mcast() for mcast-check
Quoted 13 lines · ira.weiny
Quoted 12 lines · Michael Wang
[snip]
Quoted 11 lines
Quoted 11 lines
Cc Jason Gunthorpe <jgunthorpe-ePGOBjL8dl3ta4EC/59zMFaTQe2KTcn/@public.gmane.org>Cc Doug Ledford <[email protected]>Cc Ira Weiny <[email protected]>Cc Sean Hefty <[email protected]>Signed-off-by Michael Wang <[email protected]>--- drivers/infiniband/core/cma.c | 2 +- drivers/infiniband/core/multicast.c | 8 ++++---- include/rdma/ib_verbs.h | 28 ++++++++++++++++++++++++++++ 3 files changed, 33 insertions(+), 5 deletions(-)
Quoted 12 lines
Quoted 10 lines
Quoted 10 lines
drivers/infiniband/core/cma.c void rdma_leave_multicast(struct rdma_cm_id *id, struct sockaddr *addr) · line 3401
Quoted 10 lines
drivers/infiniband/core/cma.c +1
-1@@ -3398,7 +3398,7 @@ void rdma_leave_multicast(struct rdma_cm_id *id, struct sockaddr *addr)ib_detach_mcast (id->qp, &mc->multicast.ib->rec.mgid,be16_to_cpu (mc->multicast.ib->rec.mlid)); -if (rdma_transport_is_ib (id_priv->cma_dev->device)) { +if (has_mcast (id_priv->cma_dev->device)) {You need a similar check in
rdma_join_multicast.
Thanks for the remind, I'll give it a reform :-)
Regards,
Michael Wang
Quoted 97 lines
Ira
Quoted 93 lines
Quoted 81 lines
Quoted 81 lines
drivers/infiniband/core/cma.c +0
-0switch (rdma_port_get_link_layer (id->device, id->port_num)) {case IB_LINK_LAYER_INFINIBAND :ib_sa_free_multicast (mc->multicast.ib);drivers/infiniband/core/multicast.c +4
-4@@ -780,7 +780,7 @@ static void mcast_event_handler(struct ib_event_handler *handler,int index; dev = container_of(handler, struct mcast_device, event_handler); -if (!rdma_port_ll_is_ib (dev->device, event->element.port_num)) +if (!cap_mcast (dev->device, event->element.port_num))return ; index = event->element.port_num - dev->start_port;@@ -807,7 +807,7 @@ static void mcast_add_one(struct ib_device *device)int i;int count =0 ; -if (!rdma_transport_is_ib (device)) +if (!has_mcast (device))return ; dev = kmalloc(sizeof *dev + device->phys_port_cnt * sizeof *port,@@ -823,7 +823,7 @@ static void mcast_add_one(struct ib_device *device) }for (i =0 ; i <= dev->end_port - dev->start_port; i++) { -if (!rdma_port_ll_is_ib (device, dev->start_port + i)) +if (!cap_mcast (device, dev->start_port + i))continue ; port = &dev->port[i]; port->dev = dev;@@ -861,7 +861,7 @@ static void mcast_remove_one(struct ib_device *device)flush_workqueue (mcast_wq);for (i =0 ; i <= dev->end_port - dev->start_port; i++) { -if (rdma_port_ll_is_ib (device, dev->start_port + i)) { +if (cap_mcast (device, dev->start_port + i)) { port = &dev->port[i];deref_port (port);wait_for_completion (&port->comp);include/rdma/ib_verbs.h +23
-0@@ -1823,6 +1823,19 @@ static inline int has_sa(struct ib_device *device) }/** +* has_mcast - Check if a device support Multicast. +* +* @device: Device to be checked +* +* Return 0 when a device has none port to support +* Multicast. +*/ +static inline int has_mcast (struct ib_device *device) +{ +return rdma_transport_is_ib (device); +} + +/** * cap_smi - Check if the port of device has the capability * Subnet Management Interface. * @@ -1852,6 +1865,21 @@ static inline int cap_sa(struct ib_device *device, u8 port_num)return rdma_port_ll_is_ib (device, port_num); } +/** +* cap_mcast - Check if the port of device has the capability +* Multicast. +* +* @device: Device to be checked +* @port_num: Port number of the device +* +* Return 0 when port of the device don't support +* Multicast. +*/ Quoted 8 lines
include/rdma/ib_verbs.h static inline int cap_sa(struct ib_device *device, u8 port_num) · line 1885
Quoted 8 lines
include/rdma/ib_verbs.h +5
-0+static inline int cap_mcast (struct ib_device *device,u8 port_num) +{ +return rdma_port_ll_is_ib (device, port_num); +} +int ib_query_gid (struct ib_device *device,u8 port_num,int index,union ib_gid *gid);--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
RFC PATCH 8/11 IB/Verbs: Use management helper has_iwarp() for, iwarp-check
In reply to Michael WangIntroduce helper has_iwarp() to help us check if an IB device support IWARP protocol.
--- include/rdma/ib_verbs.h | 13 +++++++++++++ net/sunrpc/xprtrdma/svc_rdma_recvfrom.c | 2 +- 2 files changed, 14 insertions(+), 1 deletion(-)
include/rdma/ib_verbs.h +13 -0
@@ -1836,6 +1836,19 @@ static inline int has_mcast(struct ib_device *device) }/** +* has_iwarp - Check if a device support IWARP protocol. +* +* @device: Device to be checked +* +* Return 0 when a device has none port to support +* IWARP protocol. +*/ +static inline int has_iwarp (struct ib_device *device) +{ +return rdma_transport_is_iwarp (device); +} + +/** * cap_smi - Check if the port of device has the capability * Subnet Management Interface. *
net/sunrpc/xprtrdma/svc_rdma_recvfrom.c +1 -1
@@ -118,7 +118,7 @@ static void rdma_build_arg_xdr(struct svc_rqst *rqstp,static int rdma_read_max_sge (struct svcxprt_rdma *xprt,int sge_count) { -if (rdma_transport_is_iwarp (xprt->sc_cm_id->device)) +if (has_iwarp (xprt->sc_cm_id->device))return 1 ;else return min_t (int, sge_count, xprt->sc_max_sge);
Signature · 1 line
2.1.0
Re: [RFC PATCH 08/11] IB/Verbs: Use management helper has_iwarp() for, iwarp-check
Quoted 3 lines · Michael Wang
Introduce helper
has_iwarp()to help us check if an IB device support IWARP protocol.
Should probably be !has_rdma_read_sges()
True if the device can handle more than one SGE entry on a RDMA READ work request.
Jason
Re: [RFC PATCH 08/11] IB/Verbs: Use management helper has_iwarp() for, iwarp-check
Quoted 7 lines · Jason Gunthorpe
Quoted 2 lines · Michael Wang
Introduce helper
has_iwarp()to help us check if an IB device support IWARP protocol.Should probably be !has_rdma_read_sges()
True if the device can handle more than one SGE entry on a RDMA READ work request.
Sounds better :-) will change it in next version.
Regards,
Michael Wang
Quoted 2 lines
Jason
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [RFC PATCH 08/11] IB/Verbs: Use management helper has_iwarp() for, iwarp-check
In reply to Jason GunthorpeQuoted 9 lines · Jason Gunthorpe
Quoted 3 lines · Michael Wang
Introduce helper
has_iwarp()to help us check if an IB device support IWARP protocol.Should probably be !has_rdma_read_sges()
True if the device can handle more than one SGE entry on a RDMA READ work request.
Isn't this value already provided by the query_device verb?
The verbs spec states the Query HCA contains the:
"Maximum number of scatter/gather entries per Work Request supported by the HCA."
-- Ira
Quoted 2 lines
Jason
Re: [RFC PATCH 08/11] IB/Verbs: Use management helper has_iwarp() for, iwarp-check
Quoted 11 lines · ira.weiny
Quoted 9 lines · Jason Gunthorpe
Quoted 3 lines · Michael Wang
Introduce helper
has_iwarp()to help us check if an IB device support IWARP protocol.Should probably be !has_rdma_read_sges()
True if the device can handle more than one SGE entry on a RDMA READ work request.
Quoted 6 lines
Isn't this value already provided by the
query_deviceverb?The verbs spec states the Query HCA contains the:
"Maximum number of scatter/gather entries per Work Request supported by the HCA."
http://www.spinics.net/lists/linux-rdma/msg22565.html
''Unlike IB, the iWARP protocol only allows 1 target/sink SGE in an rdma read''
It is one of those annoying verbs is different on iWarp things.
So the max sge in the query_verbs must only apply to send/rdma write on iWarp?
Jason
Re: [RFC PATCH 08/11] IB/Verbs: Use management helper has_iwarp() for, iwarp-check
Quoted 11 lines · Jason Gunthorpe
Quoted 1 line · ira.weiny
[snip]
http://www.spinics.net/lists/linux-rdma/msg22565.html
''Unlike IB, the iWARP protocol only allows 1 target/sink SGE in an rdma read''
It is one of those annoying verbs is different on iWarp things.
So the max sge in the
query_verbsmust only apply to send/rdma write on iWarp?
I found that actually we don't have to touch this one which only used by HW driver currently.
I think we can leave these scenes there in device driver, since vendor could have different way to classify the usage of transfer and link layer.
Our purpose is to introduce IB core management approach, which may not be applicable on device level, maybe we can just pass them :-)
Regards,
Michael Wang
Quoted 2 lines
Jason
--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [RFC PATCH 08/11] IB/Verbs: Use management helper has_iwarp() for, iwarp-check
Quoted 2 lines · Michael Wang
I found that actually we don't have to touch this one which only used by HW driver currently.
I'm having a hard time understanding this, the code in question was in
net/sunrpc/xprtrdma/svc_rdma_recvfrom.c
Which is the NFS ULP, not a device driver.
Regards,
Jason
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [RFC PATCH 08/11] IB/Verbs: Use management helper has_iwarp() for, iwarp-check
Quoted 8 lines · Jason Gunthorpe
Quoted 2 lines · Michael Wang
I found that actually we don't have to touch this one which only used by HW driver currently.
I'm having a hard time understanding this, the code in question was in
net/sunrpc/xprtrdma/svc_rdma_recvfrom.cWhich is the NFS ULP, not a device driver.
I'm not familiar with this part too :-P but yes, it looks like an ulp to support NFS.
Actually I'm thinking about Doug's idea to use rdma_transport_is_XX() instead of the current basic helper, thus may be use rdma_transport_is_iwarp() in here could be better, since it's actually a feature of iwarp tech that RDMA Read only support one scatter-gather entry.
But I need more investigation on that idea, there are some part especially inside device driver I'm not very clear, things could be more easier if the semantic there is compatible with Doug's proposal ;-)
Regards,
Michael Wang
Quoted 3 lines
Regards,
Jason
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
Tom Talpey
reviewerRe: [RFC PATCH 08/11] IB/Verbs: Use management helper has_iwarp() for, iwarp-check
Quoted 4 lines · Michael Wang
Quoted 3 lines · Jason Gunthorpe
Quoted 2 lines · Michael Wang
I found that actually we don't have to touch this one which only used by HW driver currently.
Quoted 8 lines
Quoted 5 lines
I'm having a hard time understanding this, the code in question was in
net/sunrpc/xprtrdma/svc_rdma_recvfrom.cWhich is the NFS ULP, not a device driver.
I'm not familiar with this part too :-P but yes, it looks like an ulp to support NFS.
It's the NFS server itself.
Quoted 5 lines
Actually I'm thinking about Doug's idea to use
rdma_transport_is_XX()instead of the current basic helper, thus may be userdma_transport_is_iwarp()in here could be better, since it's actually a feature of iwarp tech that RDMA Read only support one scatter-gather entry.
No, you should expose an attribute to surface the maximum length of the remote gather list, which varies by adapter as well as protocol. The fact that iWARP is different from IB is not relevant, and conflates unrelated properties.
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [RFC PATCH 08/11] IB/Verbs: Use management helper has_iwarp() for, iwarp-check
Hi, Tom
Thanks for the comments :-)
Quoted 8 lines · Tom Talpey
[snip]
Quoted 5 lines
Actually I'm thinking about Doug's idea to use
rdma_transport_is_XX()instead of the current basic helper, thus may be userdma_transport_is_iwarp()in here could be better, since it's actually a feature of iwarp tech that RDMA Read only support one scatter-gather entry.
Quoted 4 lines
No, you should expose an attribute to surface the maximum length of the remote gather list, which varies by adapter as well as protocol. The fact that iWARP is different from IB is not relevant, and conflates unrelated properties.
To be confirmed, so your point is that the max-read-sges will be different even the transport is the same IWRAP, and that depends on the capability of adapter, correct?
I currently only find this one place where infer max-read-sges from transport type, it looks more like a special case to me rather than a generic method we could exposed... and also not very related with IB management helper concept IMHO.
Regards,
Michael Wang
Tom Talpey
reviewerRe: [RFC PATCH 08/11] IB/Verbs: Use management helper has_iwarp() for, iwarp-check
Quoted 5 lines · Michael Wang
Hi, Tom
Thanks for the comments :-)
On 03/31/2015 01:19 PM, Tom Talpey wrote:
[oops - repeating my reply with full cc's]
Quoted 8 lines
Quoted 8 lines
[snip]
Quoted 5 lines
Actually I'm thinking about Doug's idea to use
rdma_transport_is_XX()instead of the current basic helper, thus may be userdma_transport_is_iwarp()in here could be better, since it's actually a feature of iwarp tech that RDMA Read only support one scatter-gather entry.
Quoted 8 lines
Quoted 4 lines
No, you should expose an attribute to surface the maximum length of the remote gather list, which varies by adapter as well as protocol. The fact that iWARP is different from IB is not relevant, and conflates unrelated properties.
To be confirmed, so your point is that the max-read-sges will be different even the transport is the same IWRAP, and that depends on the capability of adapter, correct?
Yes, in fact the iWARP protocol does not preclude multiple read SGEs, even though most iWARP implementations have chosen to support just one.
Even for multi-SGE-capable adapters, there is a limit of SGL size, based on the adapter's work request format and other factors. So my argument is that upper layers can and should query that, not make a blanket decision based on protocol type.
Quoted 5 lines
I currently only find this one place where infer max-read-sges from transport type, it looks more like a special case to me rather than a generic method we could exposed... and also not very related with IB management helper concept IMHO.
It is most certainly not a special case, but you could decide to introduce it in many ways. I'm not commenting on that.
My main concern is that you do not introduce a new and clumsy "is iWARP" rule as an adapter-specific API requirement to expose the RDMA Read SGE behavior. That's what your initial message seemed to imply?
--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [RFC PATCH 08/11] IB/Verbs: Use management helper has_iwarp() for, iwarp-check
Quoted 9 lines · Tom Talpey
Quoted 1 line · Michael Wang
[snip]
Yes, in fact the iWARP protocol does not preclude multiple read SGEs, even though most iWARP implementations have chosen to support just one.
Even for multi-SGE-capable adapters, there is a limit of SGL size, based on the adapter's work request format and other factors. So my argument is that upper layers can and should query that, not make a blanket decision based on protocol type.
Thanks for the explanation Sounds like some new callback on device level like query_device() to acquire the right info.
Quoted 10 lines
Quoted 4 lines
I currently only find this one place where infer max-read-sges from transport type, it looks more like a special case to me rather than a generic method we could exposed... and also not very related with IB management helper concept IMHO.
It is most certainly not a special case, but you could decide to introduce it in many ways. I'm not commenting on that.
My main concern is that you do not introduce a new and clumsy "is iWARP" rule as an adapter-specific API requirement to expose the RDMA Read SGE behavior. That's what your initial message seemed to imply?
Yeah I planed to just use rdma_transport_iwarp() to replace the check, it's actually meaningless but just refine, frankly speaking I would prefer some driver developer to work on that part, at this point I prefer to focus on introducing the management helpers firstly
Maybe we could mark it as a TODO temporarily, if later we found more scenes using similar logical, we can collect them and do some reform
Regards,
Michael Wang
Re: [RFC PATCH 08/11] IB/Verbs: Use management helper has_iwarp() for, iwarp-check
In reply to ira.weinyQuoted 11 lines · ira.weiny
Quoted 9 lines · Jason Gunthorpe
Quoted 3 lines · Michael Wang
Introduce helper
has_iwarp()to help us check if an IB device support IWARP protocol.Should probably be !has_rdma_read_sges()
True if the device can handle more than one SGE entry on a RDMA READ work request.
Quoted 6 lines
Isn't this value already provided by the
query_deviceverb?The verbs spec states the Query HCA contains the:
"Maximum number of scatter/gather entries per Work Request supported by the HCA."
I'm not sure but may be query_device() is just too expensive for this path? I need some investigation here too.
Regards,
Michael Wang
Quoted 5 lines
-- Ira
Quoted 2 lines
Jason
Re: [RFC PATCH 08/11] IB/Verbs: Use management helper has_iwarp() for, iwarp-check
In reply to Michael WangQuoted 2 lines · Michael Wang
Introduce helper
has_iwarp()to help us check if an IB device support IWARP protocol.
This is a needless redirection. Just stick with the original rdma_transport_is_iwarp().
Quoted 47 lines
Cc Jason Gunthorpe <[email protected]>Cc Doug Ledford <[email protected]>Cc Ira Weiny <[email protected]>Cc Sean Hefty <[email protected]>Signed-off-by Michael Wang <[email protected]>--- include/rdma/ib_verbs.h | 13 +++++++++++++ net/sunrpc/xprtrdma/svc_rdma_recvfrom.c | 2 +- 2 files changed, 14 insertions(+), 1 deletion(-)include/rdma/ib_verbs.h +13
-0@@ -1836,6 +1836,19 @@ static inline int has_mcast(struct ib_device *device) }/** +* has_iwarp - Check if a device support IWARP protocol. +* +* @device: Device to be checked +* +* Return 0 when a device has none port to support +* IWARP protocol. +*/ +static inline int has_iwarp (struct ib_device *device) +{ +return rdma_transport_is_iwarp (device); +} + +/** * cap_smi - Check if the port of device has the capability * Subnet Management Interface. * net/sunrpc/xprtrdma/svc_rdma_recvfrom.c +1
-1@@ -118,7 +118,7 @@ static void rdma_build_arg_xdr(struct svc_rqst *rqstp,static int rdma_read_max_sge (struct svcxprt_rdma *xprt,int sge_count) { -if (rdma_transport_is_iwarp (xprt->sc_cm_id->device)) +if (has_iwarp (xprt->sc_cm_id->device))return 1 ;else return min_t (int, sge_count, xprt->sc_max_sge);
Signature · 2 lines
Doug Ledford <[email protected]> GPG KeyID: 0E572FDD
Re: [RFC PATCH 08/11] IB/Verbs: Use management helper has_iwarp() for, iwarp-check
Quoted 5 lines · Doug Ledford
Quoted 2 lines · Michael Wang
Introduce helper
has_iwarp()to help us check if an IB device support IWARP protocol.This is a needless redirection. Just stick with the original
rdma_transport_is_iwarp().
Agree, will leave it there :-)
Regards,
Michael Wang
Quoted 41 lines
Quoted 41 lines
Cc Jason Gunthorpe <jgunthorpe-ePGOBjL8dl3ta4EC/59zMFaTQe2KTcn/@public.gmane.org>Cc Doug Ledford <[email protected]>Cc Ira Weiny <[email protected]>Cc Sean Hefty <[email protected]>Signed-off-by Michael Wang <[email protected]>--- include/rdma/ib_verbs.h | 13 +++++++++++++ net/sunrpc/xprtrdma/svc_rdma_recvfrom.c | 2 +- 2 files changed, 14 insertions(+), 1 deletion(-)include/rdma/ib_verbs.h +13
-0@@ -1836,6 +1836,19 @@ static inline int has_mcast(struct ib_device *device) }/** +* has_iwarp - Check if a device support IWARP protocol. +* +* @device: Device to be checked +* +* Return 0 when a device has none port to support +* IWARP protocol. +*/ +static inline int has_iwarp (struct ib_device *device) +{ +return rdma_transport_is_iwarp (device); +} + +/** * cap_smi - Check if the port of device has the capability * Subnet Management Interface. * net/sunrpc/xprtrdma/svc_rdma_recvfrom.c +0
-0@@ -118,7 +118,7 @@ static void rdma_build_arg_xdr(struct svc_rqst *rqstp,
Quoted 8 lines
net/sunrpc/xprtrdma/svc_rdma_recvfrom.c static void rdma_build_arg_xdr(struct svc_rqst *rqstp, · line 124
Quoted 7 lines
net/sunrpc/xprtrdma/svc_rdma_recvfrom.c +1
-1static int rdma_read_max_sge (struct svcxprt_rdma *xprt,int sge_count) { -if (rdma_transport_is_iwarp (xprt->sc_cm_id->device)) +if (has_iwarp (xprt->sc_cm_id->device))return 1 ;else return min_t (int, sge_count, xprt->sc_max_sge);
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [RFC PATCH 08/11] IB/Verbs: Use management helper has_iwarp() for, iwarp-check
In reply to Doug LedfordQuoted 6 lines · Doug Ledford
Quoted 2 lines · Michael Wang
Introduce helper
has_iwarp()to help us check if an IB device support IWARP protocol.This is a needless redirection. Just stick with the original
rdma_transport_is_iwarp().
Sticking with the original isn't really the point.
The point here, is to document what Tom was talking about - some ports can only support one RDMA READ SGL entry, even though they support multiple RDMA WRITE SGL entries. Today the query API assumes READ/WRITE/SEND are symmetrical.
has_rdma_read_sgl() is a good way to document that for now, and is a big flashing marker where something might need to be fixed in the future.
This tells everyone reading the code and the API that when working with RDMA READ they need to be aware of this limitation.
Jason
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
RFC PATCH 9/11 IB/Verbs: Use management helper has_ipoib() and, cap_ipoib() for ipoib-check
In reply to Michael WangIntroduce helper has_ipoib() and cap_ipoib() to help us check if an IB device or it's port support IP over Infiniband.
--- drivers/infiniband/ulp/ipoib/ipoib_main.c | 6 +++--- include/rdma/ib_verbs.h | 28 ++++++++++++++++++++++++++++ 2 files changed, 31 insertions(+), 3 deletions(-)
drivers/infiniband/ulp/ipoib/ipoib_main.c +3 -3
@@ -1655,7 +1655,7 @@ static void ipoib_add_one(struct ib_device *device)struct ipoib_dev_priv *priv;int s, e, p; -if (!rdma_transport_is_ib (device)) +if (!has_ipoib (device))return ; dev_list =kmalloc (sizeof *dev_list,GFP_KERNEL );
@@ -1673,7 +1673,7 @@ static void ipoib_add_one(struct ib_device *device) }for (p = s; p <= e; ++p) { -if (!rdma_port_ll_is_ib (device, p)) +if (!cap_ipoib (device, p))continue ; dev =ipoib_add_port ("ib%d" , device, p);if (!IS_ERR (dev)) {
@@ -1690,7 +1690,7 @@ static void ipoib_remove_one(struct ib_device *device)struct ipoib_dev_priv *priv, *tmp;struct list_head *dev_list; -if (!rdma_transport_is_ib (device)) +if (!has_ipoib (device))return ; dev_list =ib_get_client_data (device, &ipoib_client);
include/rdma/ib_verbs.h +28 -0
@@ -1849,6 +1849,19 @@ static inline int has_iwarp(struct ib_device *device) }/** +* has_ipoib - Check if a device support IP over Infiniband. +* +* @device: Device to be checked +* +* Return 0 when a device has none port to support +* IP over Infiniband. +*/ +static inline int has_ipoib (struct ib_device *device) +{ +return rdma_transport_is_ib (device); +} + +/** * cap_smi - Check if the port of device has the capability * Subnet Management Interface. *
@@ -1893,6 +1906,21 @@ static inline int cap_mcast(struct ib_device *device, u8 port_num)return rdma_port_ll_is_ib (device, port_num); } +/** +* cap_ipoib - Check if the port of device has the capability +* IP over Infiniband. +* +* @device: Device to be checked +* @port_num: Port number of the device +* +* Return 0 when port of the device don't support +* IP over Infiniband. +*/ +static inline int cap_ipoib (struct ib_device *device,u8 port_num) +{ +return rdma_port_ll_is_ib (device, port_num); +} +int ib_query_gid (struct ib_device *device,u8 port_num,int index,union ib_gid *gid);
Signature · 1 line
2.1.0
Re: [RFC PATCH 09/11] IB/Verbs: Use management helper has_ipoib() and, cap_ipoib() for ipoib-check
drivers/infiniband/ulp/ipoib/ipoib_main.c static void ipoib_add_one(struct ib_device *device) · line 1659
Quoted 10 lines · Michael Wang
drivers/infiniband/ulp/ipoib/ipoib_main.c +2
-1+++ b/drivers/infiniband/ulp/ipoib/ipoib_main.c@@ -1655,7 +1655,7 @@ static void ipoib_add_one(struct ib_device *device)struct ipoib_dev_priv *priv;int s, e, p; -if (!rdma_transport_is_ib (device)) +if (!has_ipoib (device))return ;
This is a good example of a test that doesn't really make sense, IPoIB is certainly a port specific attribute.
drivers/infiniband/ulp/ipoib/ipoib_main.c static void ipoib_add_one(struct ib_device *device) · line 1677
Quoted 8 lines
drivers/infiniband/ulp/ipoib/ipoib_main.c +1
-1dev_list =kmalloc (sizeof *dev_list,GFP_KERNEL );@@ -1673,7 +1673,7 @@ static void ipoib_add_one(struct ib_device *device) }for (p = s; p <= e; ++p) { -if (!rdma_port_ll_is_ib (device, p)) +if (!cap_ipoib (device, p))continue ;
And down here we test every port.
The routine should just test every port and do nothing if no ports need IPoIB.
drivers/infiniband/ulp/ipoib/ipoib_main.c static void ipoib_remove_one(struct ib_device *device) · line 1694
Quoted 9 lines
drivers/infiniband/ulp/ipoib/ipoib_main.c +1
-1dev =ipoib_add_port ("ib%d" , device, p);if (!IS_ERR (dev)) {@@ -1690,7 +1690,7 @@ static void ipoib_remove_one(struct ib_device *device)struct ipoib_dev_priv *priv, *tmp;struct list_head *dev_list; -if (!rdma_transport_is_ib (device)) +if (!has_ipoib (device))return ;
This test should be made redundant by having ib_get_client_data return null if ipoib_add_one didn't do anything. Maybe that already happens?
Jason
Re: [RFC PATCH 09/11] IB/Verbs: Use management helper has_ipoib() and, cap_ipoib() for ipoib-check
Quoted 9 lines · Jason Gunthorpe
Quoted 7 lines · Michael Wang
drivers/infiniband/ulp/ipoib/ipoib_main.c +1
-0+++ b/drivers/infiniband/ulp/ipoib/ipoib_main.c@@ -1655,7 +1655,7 @@ static void ipoib_add_one(struct ib_device *device)struct ipoib_dev_priv *priv;int s, e, p;
Quoted 5 lines
drivers/infiniband/ulp/ipoib/ipoib_main.c static void ipoib_add_one(struct ib_device *device) · line 1659
Quoted 3 lines
drivers/infiniband/ulp/ipoib/ipoib_main.c +1
-1-if (!rdma_transport_is_ib (device)) +if (!has_ipoib (device))return ;This is a good example of a test that doesn't really make sense, IPoIB is certainly a port specific attribute.
According to my understanding, seems like the logical is:
if 'device have no port need ipoib initialization'
return
Quoted 5 lines
Quoted 5 lines
drivers/infiniband/ulp/ipoib/ipoib_main.c +0
-0dev_list =kmalloc (sizeof *dev_list,GFP_KERNEL );@@ -1673,7 +1673,7 @@ static void ipoib_add_one(struct ib_device *device) }
Quoted 8 lines
drivers/infiniband/ulp/ipoib/ipoib_main.c static void ipoib_add_one(struct ib_device *device) · line 1677
Quoted 4 lines
drivers/infiniband/ulp/ipoib/ipoib_main.c +1
-1for (p = s; p <= e; ++p) { -if (!rdma_port_ll_is_ib (device, p)) +if (!cap_ipoib (device, p))continue ;And down here we test every port.
The routine should just test every port and do nothing if no ports need IPoIB.
And here since there are some initial work to do, it iterate all the port to find out which one need to be initialized.
Quoted 7 lines
Quoted 7 lines
drivers/infiniband/ulp/ipoib/ipoib_main.c +0
-0dev =ipoib_add_port ("ib%d" , device, p);if (!IS_ERR (dev)) {@@ -1690,7 +1690,7 @@ static void ipoib_remove_one(struct ib_device *device)struct ipoib_dev_priv *priv, *tmp;struct list_head *dev_list;
Quoted 5 lines
drivers/infiniband/ulp/ipoib/ipoib_main.c static void ipoib_add_one(struct ib_device *device) · line 1659
Quoted 3 lines
drivers/infiniband/ulp/ipoib/ipoib_main.c +1
-1-if (!rdma_transport_is_ib (device)) +if (!has_ipoib (device))return ;This test should be made redundant by having
ib_get_client_datareturn null ifipoib_add_onedidn't do anything. Maybe that already happens?
Here if we have done nothing when add a device, then nothing need to be cleanup when remove it.
I'm not sure if it's a good idea, but seems like the idea is use twice check on different level to save some cycles?
Regards,
Michael Wang
Quoted 2 lines
Jason
--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [RFC PATCH 09/11] IB/Verbs: Use management helper has_ipoib() and, cap_ipoib() for ipoib-check
Quoted 2 lines · Michael Wang
I'm not sure if it's a good idea, but seems like the idea is use twice check on different level to save some cycles?
That is what it looks like to me, right now. Which is why I'd drop it. This isn't performance sensitive, make it simple.
Jason
RFC PATCH 10/11 IB/Verbs: Use management helper cap_ib() for ib-check
In reply to Michael WangTODO:
We need to classify these check into the management helper.
--- drivers/infiniband/core/cma.c | 5 ++--- drivers/infiniband/hw/mlx4/mad.c | 4 ++-- drivers/infiniband/hw/mlx4/main.c | 5 ++--- include/rdma/ib_verbs.h | 8 ++++++++ 4 files changed, 14 insertions(+), 8 deletions(-)
drivers/infiniband/core/cma.c +2 -3
@@ -696,7 +696,7 @@ static int cma_ib_init_qp_attr(struct rdma_id_private *id_priv,int ret;u16 pkey; -if (rdma_port_ll_is_ib (id_priv->id.device, id_priv->id.port_num)) +if (cap_ib (id_priv->id.device, id_priv->id.port_num)) pkey =ib_addr_get_pkey (dev_addr);else pkey =0xffff ;
@@ -2053,8 +2053,7 @@ port_found: goto out; id_priv->id.route.addr.dev_addr.dev_type = - (rdma_port_ll_is_ib (cma_dev->device, p)) ? -ARPHRD_INFINIBAND :ARPHRD_ETHER ; + (cap_ib (cma_dev->device, p)) ?ARPHRD_INFINIBAND :ARPHRD_ETHER ;rdma_addr_set_sgid (&id_priv->id.route.addr.dev_addr, &gid);ib_addr_set_pkey (&id_priv->id.route.addr.dev_addr, pkey);
drivers/infiniband/hw/mlx4/mad.c +2 -2
@@ -1247,7 +1247,7 @@ out:static int get_slave_base_gid_ix (struct mlx4_ib_dev *dev,int slave,int port) { -if (rdma_port_ll_is_ib (&dev->ib_dev, port)) +if (cap_ib (&dev->ib_dev, port))return slave;return mlx4_get_base_gid_ix (dev->dev, slave, port); }
@@ -1255,7 +1255,7 @@ static int get_slave_base_gid_ix(struct mlx4_ib_dev *dev, int slave, int port)static void fill_in_real_sgid_index (struct mlx4_ib_dev *dev,int slave,int port,struct ib_ah_attr *ah_attr) { -if (rdma_port_ll_is_ib (&dev->ib_dev, port)) +if (cap_ib (&dev->ib_dev, port)) ah_attr->grh.sgid_index = slave;else ah_attr->grh.sgid_index +=get_slave_base_gid_ix (dev, slave, port);
drivers/infiniband/hw/mlx4/main.c +2 -3
@@ -482,7 +482,7 @@ static int iboe_query_gid(struct ib_device *ibdev, u8 port, int index,static int mlx4_ib_query_gid (struct ib_device *ibdev,u8 port,int index,union ib_gid *gid) { -if (rdma_port_ll_is_ib (ibdev, port)) +if (cap_ib (ibdev, port))return __mlx4_ib_query_gid (ibdev, port, index, gid,0 );else return iboe_query_gid (ibdev, port, index, gid);
@@ -2552,8 +2552,7 @@ static void mlx4_ib_event(struct mlx4_dev *dev, void *ibdev_ptr,case MLX4_DEV_EVENT_PORT_UP :if (p > ibdev->num_ports)return ; -if (mlx4_is_master (dev) && -rdma_port_ll_is_ib (&ibdev->ib_dev, p)) { +if (mlx4_is_master (dev) &&cap_ib (&ibdev->ib_dev, p)) {mlx4_ib_invalidate_all_guid_record (ibdev, p); } ibev.event =IB_EVENT_PORT_ACTIVE ;
include/rdma/ib_verbs.h +8 -0
@@ -1921,6 +1921,14 @@ static inline int cap_ipoib(struct ib_device *device, u8 port_num)return rdma_port_ll_is_ib (device, port_num); } +/** +* TO BE CLASSIFIED +*/ +static inline int cap_ib (struct ib_device *device,u8 port_num) +{ +return rdma_port_ll_is_ib (device, port_num); +} +int ib_query_gid (struct ib_device *device,u8 port_num,int index,union ib_gid *gid);
Signature · 1 line
2.1.0
RFC PATCH 11/11 IB/Verbs: Use management helper cap_eth() for eth-check
In reply to Michael WangTODO:
We need to classify these check into the management helper.
--- drivers/infiniband/core/cma.c | 2 +- drivers/infiniband/core/sa_query.c | 2 +- drivers/infiniband/core/verbs.c | 4 ++-- drivers/infiniband/hw/mlx4/ah.c | 2 +- drivers/infiniband/hw/mlx4/cq.c | 2 +- drivers/infiniband/hw/mlx4/mad.c | 2 +- drivers/infiniband/hw/mlx4/main.c | 2 +- drivers/infiniband/hw/mlx4/mlx4_ib.h | 2 +- drivers/infiniband/hw/mlx4/qp.c | 15 +++++++-------- drivers/infiniband/hw/mlx4/sysfs.c | 4 ++-- include/rdma/ib_verbs.h | 8 ++++++++ 11 files changed, 26 insertions(+), 19 deletions(-)
drivers/infiniband/core/cma.c +1 -1
@@ -632,7 +632,7 @@ static int cma_modify_qp_rtr(struct rdma_id_private *id_priv, goto out;if (has_cm (id_priv->cma_dev->device) && -rdma_port_ll_is_eth (id_priv->id.device, id_priv->id.port_num)) { +cap_eth (id_priv->id.device, id_priv->id.port_num)) { ret =rdma_addr_find_smac_by_sgid (&sgid, qp_attr.smac,NULL );if (ret)
drivers/infiniband/core/sa_query.c +1 -1
@@ -540,7 +540,7 @@ int ib_init_ah_from_path(struct ib_device *device, u8 port_num, ah_attr->port_num = port_num; ah_attr->static_rate = rec->rate; - force_grh =rdma_port_ll_is_eth (device, port_num); + force_grh =cap_eth (device, port_num);if (rec->hop_limit >1 || force_grh) { ah_attr->ah_flags =IB_AH_GRH ;
drivers/infiniband/core/verbs.c +2 -2
@@ -198,7 +198,7 @@ int ib_init_ah_from_wc(struct ib_device *device, u8 port_num, struct ib_wc *wc,u32 flow_class;u16 gid_index;int ret; -int is_eth = (rdma_port_ll_is_eth (device, port_num)); +int is_eth =cap_eth (device, port_num);memset (ah_attr,0 ,sizeof *ah_attr);if (is_eth) {
@@ -870,7 +870,7 @@ int ib_resolve_eth_l2_attrs(struct ib_qp *qp,union ib_gid sgid;if ((*qp_attr_mask &IB_QP_AV ) && - (rdma_port_ll_is_eth (qp->device, qp_attr->ah_attr.port_num))) { + (cap_eth (qp->device, qp_attr->ah_attr.port_num))) { ret =ib_query_gid (qp->device, qp_attr->ah_attr.port_num, qp_attr->ah_attr.grh.sgid_index, &sgid);if (ret)
drivers/infiniband/hw/mlx4/ah.c +1 -1
@@ -118,7 +118,7 @@ struct ib_ah *mlx4_ib_create_ah(struct ib_pd *pd, struct ib_ah_attr *ah_attr)if (!ah)return ERR_PTR (-ENOMEM ); -if (rdma_port_ll_is_eth (pd->device, ah_attr->port_num)) { +if (cap_eth (pd->device, ah_attr->port_num)) {if (!(ah_attr->ah_flags &IB_AH_GRH )) { ret =ERR_PTR (-EINVAL ); }else {
drivers/infiniband/hw/mlx4/cq.c +1 -1
@@ -789,7 +789,7 @@ repoll:break ; } - is_eth = (rdma_port_ll_is_eth (wc->qp->device, (*cur_qp)->port)); + is_eth =cap_eth (wc->qp->device, (*cur_qp)->port);if (mlx4_is_mfunc (to_mdev (cq->ibcq.device)->dev)) { if ((*cur_qp)->mlx4_ib_qp_type & (MLX4_IB_QPT_PROXY_SMI_OWNER |
drivers/infiniband/hw/mlx4/mad.c +1 -1
@@ -606,7 +606,7 @@ static int mlx4_ib_demux_mad(struct ib_device *ibdev, u8 port,int err;int slave;u8 *slave_id; -int is_eth =rdma_port_ll_is_eth (ibdev, port); +int is_eth =cap_eth (ibdev, port);if (is_eth) {if (!(wc->wc_flags &IB_WC_GRH )) {
drivers/infiniband/hw/mlx4/main.c +1 -1
@@ -1801,7 +1801,7 @@ static int mlx4_ib_init_gid_table(struct mlx4_ib_dev *ibdev)int err =0 ;for (i =1 ; i <= ibdev->num_ports; ++i) { -if (rdma_port_ll_is_eth (&ibdev->ib_dev, i)) { +if (cap_eth (&ibdev->ib_dev, i)) { err =reset_gid_table (ibdev, i);if (err) goto out;
drivers/infiniband/hw/mlx4/mlx4_ib.h +1 -1
@@ -712,7 +712,7 @@ static inline bool mlx4_ib_ah_grh_present(struct mlx4_ib_ah *ah) {u8 port =be32_to_cpu (ah->av.ib.port_pd) >>24 &3 ; -if (rdma_port_ll_is_eth (ah->ibah.device, port)) +if (cap_eth (ah->ibah.device, port))return true;return !!(ah->av.ib.g_slid &0x80 );
drivers/infiniband/hw/mlx4/qp.c +7 -8
@@ -1248,7 +1248,7 @@ static int _mlx4_set_path(struct mlx4_ib_dev *dev, const struct ib_ah_attr *ah, u64 smac, u16 vlan_tag, struct mlx4_qp_path *path, struct mlx4_roce_smac_vlan_info *smac_info, u8 port) { - int is_eth = rdma_port_ll_is_eth(&dev->ib_dev, port); + int is_eth = cap_eth(&dev->ib_dev, port);int vidx;int smac_index;int err;
@@ -1431,8 +1431,7 @@ static int __mlx4_ib_modify_qp(struct ib_qp *ibqp,int err = -EINVAL ;/* APM is not supported under RoCE */ -if (attr_mask &IB_QP_ALT_PATH && -rdma_port_ll_is_eth (&dev->ib_dev, qp->port)) +if (attr_mask &IB_QP_ALT_PATH &&cap_eth (&dev->ib_dev, qp->port))return -ENOTSUPP ; context =kzalloc (sizeof *context,GFP_KERNEL );
@@ -1662,7 +1661,7 @@ static int __mlx4_ib_modify_qp(struct ib_qp *ibqp, context->pri_path.fl =0x80 ; context->pri_path.sched_queue |=MLX4_IB_DEFAULT_SCHED_QUEUE ; } -if (rdma_port_ll_is_eth (&dev->ib_dev, qp->port)) { +if (cap_eth (&dev->ib_dev, qp->port)) {if (qp->mlx4_ib_qp_type ==MLX4_IB_QPT_TUN_GSI || qp->mlx4_ib_qp_type ==MLX4_IB_QPT_GSI ) context->pri_path.feup =1 <<7 ;/* don't fsm */
@@ -1692,7 +1691,7 @@ static int __mlx4_ib_modify_qp(struct ib_qp *ibqp, }if (ibqp->qp_type ==IB_QPT_UD && (new_state ==IB_QPS_RTR )) { -int is_eth =rdma_port_ll_is_eth (&dev->ib_dev, qp->port); +int is_eth =cap_eth (&dev->ib_dev, qp->port);if (is_eth) { context->pri_path.ackto =MLX4_IB_LINK_TYPE_ETH ; optpar |=MLX4_QP_OPTPAR_PRIMARY_ADDR_PATH ;
@@ -1922,7 +1921,7 @@ int mlx4_ib_modify_qp(struct ib_qp *ibqp, struct ib_qp_attr *attr, }if ((attr_mask &IB_QP_PORT ) && (ibqp->qp_type ==IB_QPT_RAW_PACKET ) && - !rdma_port_ll_is_eth (&dev->ib_dev, attr->port_num)) + !cap_eth (&dev->ib_dev, attr->port_num)) goto out;if (attr_mask &IB_QP_PKEY_INDEX ) {
@@ -2126,7 +2125,7 @@ static int build_mlx_header(struct mlx4_ib_sqp *sqp, struct ib_send_wr *wr,for (i =0 ; i < wr->num_sge; ++i) send_size += wr->sg_list[i].length; - is_eth =rdma_port_ll_is_eth (sqp->qp.ibqp.device, sqp->qp.port); + is_eth =cap_eth (sqp->qp.ibqp.device, sqp->qp.port); is_grh =mlx4_ib_ah_grh_present (ah);if (is_eth) {if (mlx4_is_mfunc (to_mdev (ib_dev)->dev)) {
@@ -3023,7 +3022,7 @@ static void to_ib_ah_attr(struct mlx4_ib_dev *ibdev, struct ib_ah_attr *ib_ah_atif (ib_ah_attr->port_num ==0 || ib_ah_attr->port_num > dev->caps.num_ports)return ; - is_eth =rdma_port_ll_is_eth (&ibdev->ib_dev, ib_ah_attr->port_num); + is_eth =cap_eth (&ibdev->ib_dev, ib_ah_attr->port_num);if (is_eth) ib_ah_attr->sl = ((path->sched_queue >>3 ) &0x7 ) | ((path->sched_queue &4 ) <<1 );
drivers/infiniband/hw/mlx4/sysfs.c +2 -2
@@ -610,7 +610,7 @@ static ssize_t sysfs_store_enable_smi_admin(struct device *dev,static int add_vf_smi_entries (struct mlx4_port *p) { -int is_eth =rdma_port_ll_is_eth (&p->dev->ib_dev, p->port_num); +int is_eth =cap_eth (&p->dev->ib_dev, p->port_num);int ret;/* do not display entries if eth transport, or if master */
@@ -644,7 +644,7 @@ static int add_vf_smi_entries(struct mlx4_port *p)static void remove_vf_smi_entries (struct mlx4_port *p) { -int is_eth =rdma_port_ll_is_eth (&p->dev->ib_dev, p->port_num); +int is_eth =cap_eth (&p->dev->ib_dev, p->port_num);if (is_eth || p->slave ==mlx4_master_func_num (p->dev->dev))return ;
include/rdma/ib_verbs.h +8 -0
@@ -1929,6 +1929,14 @@ static inline int cap_ib(struct ib_device *device, u8 port_num)return rdma_port_ll_is_ib (device, port_num); } +/** +* TO BE CLASSIFIED +*/ +static inline int cap_eth (struct ib_device *device,u8 port_num) +{ +return rdma_port_ll_is_eth (device, port_num); +} +int ib_query_gid (struct ib_device *device,u8 port_num,int index,union ib_gid *gid);
Signature · 1 line
2.1.0
Quoted 4 lines · Michael Wang
There are plenty of lengthy code to check the transport type of IB device, or the link layer type of it's port, but actually we are just speculating whether a particular management is supported by the device/port.
Michael,
There was a lot of input on this patch series which I believe indicates we would like to move forward. Were you working through another version of the series with the suggested changes?
Thanks,
Ira
Quoted 59 lines
Thus instead of inferring, we should have our own mechanism for IB management capability/protocol checking, several proposals below.
This patch set is the pioneer which try to collecting all the cases where a management helper applicable, and put all the speculating logical together, in order to be easily integrated to the new querying mechanism in future.
TODO:
The helper in patch 10 and 11 are still to be classified, that iscap_ib()andcap_eth().The name/description of each helper may not correct enough, those folks who are familiar with these particular parts, your suggestion would be invaluable ;-)
Proposals:
Sean: https://www.mail-archive.com/[email protected]/msg23339.html Doug: https://www.mail-archive.com/[email protected]/msg23418.html Jason: https://www.mail-archive.com/[email protected]/msg23425.htmlMichael Wang (11):
[PATCH 01/11] IB/Verbs: Use helpers to check transport and link layer
[RFC PATCH 02/11] IB/Verbs: Use management helpertech_iboe()for iboe-check [RFC PATCH 03/11] IB/Verbs: Use management helperhas_mad()for mad-check [RFC PATCH 04/11] IB/Verbs: Use management helpercap_smi()for smi-check [RFC PATCH 05/11] IB/Verbs: Use management helperhas_cm()for cm-check
[RFC PATCH 06/11] IB/Verbs: Use management helperhas_sa()andcap_sa()for sa-check [RFC PATCH 07/11] IB/Verbs: Use management helperhas_mcast()andcap_mcast()for mcast-check [RFC PATCH 08/11] IB/Verbs: Use management helperhas_iwarp()for iwarp-check [RFC PATCH 09/11] IB/Verbs: Use management helperhas_ipoib()andcap_ipoib()for ipoib-check [RFC PATCH 10/11] IB/Verbs: Use management helpercap_ib()for ib-check
[RFC PATCH 11/11] IB/Verbs: Use management helpercap_eth()for eth-check--- drivers/infiniband/core/agent.c | 6 drivers/infiniband/core/cm.c | 6 drivers/infiniband/core/cma.c | 79 ++++------ drivers/infiniband/core/mad.c | 18 +- drivers/infiniband/core/multicast.c | 30 +--- drivers/infiniband/core/sa_query.c | 42 ++--- drivers/infiniband/core/ucm.c | 8 - drivers/infiniband/core/user_mad.c | 6 drivers/infiniband/core/verbs.c | 14 - drivers/infiniband/hw/mlx4/ah.c | 6 drivers/infiniband/hw/mlx4/cq.c | 10 - drivers/infiniband/hw/mlx4/mad.c | 36 +--- drivers/infiniband/hw/mlx4/main.c | 23 +-- drivers/infiniband/hw/mlx4/mlx4_ib.h | 6 drivers/infiniband/hw/mlx4/qp.c | 57 ++----- drivers/infiniband/hw/mlx4/sysfs.c | 16 -- drivers/infiniband/ulp/ipoib/ipoib_main.c | 18 +- include/rdma/ib_verbs.h | 218 ++++++++++++++++++++++++++++++ net/sunrpc/xprtrdma/svc_rdma_recvfrom.c | 8 - 19 files changed, 381 insertions(+), 226 deletions(-)
Quoted 9 lines · ira.weiny
Quoted 3 lines · Michael Wang
There are plenty of lengthy code to check the transport type of IB device, or the link layer type of it's port, but actually we are just speculating whether a particular management is supported by the device/port.
Michael,
There was a lot of input on this patch series which I believe indicates we would like to move forward. Were you working through another version of the series with the suggested changes?
Thanks for the remind, Ira :-)
I'm working on next version these days, I've add a new callback query_transport() and implement it for all the HW driver, there are some issue during reform the cma stuff previously, but now I've finished most of the part.
I'll separate the work to small patch so review could be easier, and give some description as well as I can, it will be send out in this week ;-)
Regards,
Michael Wang
Quoted 63 lines
Thanks,
IraQuoted 58 lines
Thus instead of inferring, we should have our own mechanism for IB management capability/protocol checking, several proposals below.
This patch set is the pioneer which try to collecting all the cases where a management helper applicable, and put all the speculating logical together, in order to be easily integrated to the new querying mechanism in future.
TODO:
The helper in patch 10 and 11 are still to be classified, that iscap_ib()andcap_eth().The name/description of each helper may not correct enough, those folks who are familiar with these particular parts, your suggestion would be invaluable ;-)
Proposals:
Sean: https://www.mail-archive.com/[email protected]/msg23339.html Doug: https://www.mail-archive.com/[email protected]/msg23418.html Jason: https://www.mail-archive.com/[email protected]/msg23425.htmlMichael Wang (11):
[PATCH 01/11] IB/Verbs: Use helpers to check transport and link layer
[RFC PATCH 02/11] IB/Verbs: Use management helpertech_iboe()for iboe-check [RFC PATCH 03/11] IB/Verbs: Use management helperhas_mad()for mad-check [RFC PATCH 04/11] IB/Verbs: Use management helpercap_smi()for smi-check [RFC PATCH 05/11] IB/Verbs: Use management helperhas_cm()for cm-check
[RFC PATCH 06/11] IB/Verbs: Use management helperhas_sa()andcap_sa()for sa-check [RFC PATCH 07/11] IB/Verbs: Use management helperhas_mcast()andcap_mcast()for mcast-check [RFC PATCH 08/11] IB/Verbs: Use management helperhas_iwarp()for iwarp-check [RFC PATCH 09/11] IB/Verbs: Use management helperhas_ipoib()andcap_ipoib()for ipoib-check [RFC PATCH 10/11] IB/Verbs: Use management helpercap_ib()for ib-check
[RFC PATCH 11/11] IB/Verbs: Use management helpercap_eth()for eth-check--- drivers/infiniband/core/agent.c | 6 drivers/infiniband/core/cm.c | 6 drivers/infiniband/core/cma.c | 79 ++++------ drivers/infiniband/core/mad.c | 18 +- drivers/infiniband/core/multicast.c | 30 +--- drivers/infiniband/core/sa_query.c | 42 ++--- drivers/infiniband/core/ucm.c | 8 - drivers/infiniband/core/user_mad.c | 6 drivers/infiniband/core/verbs.c | 14 - drivers/infiniband/hw/mlx4/ah.c | 6 drivers/infiniband/hw/mlx4/cq.c | 10 - drivers/infiniband/hw/mlx4/mad.c | 36 +--- drivers/infiniband/hw/mlx4/main.c | 23 +-- drivers/infiniband/hw/mlx4/mlx4_ib.h | 6 drivers/infiniband/hw/mlx4/qp.c | 57 ++----- drivers/infiniband/hw/mlx4/sysfs.c | 16 -- drivers/infiniband/ulp/ipoib/ipoib_main.c | 18 +- include/rdma/ib_verbs.h | 218 ++++++++++++++++++++++++++++++ net/sunrpc/xprtrdma/svc_rdma_recvfrom.c | 8 - 19 files changed, 381 insertions(+), 226 deletions(-)
Quoted 11 lines · ira.weiny
Quoted 4 lines · Michael Wang
There are plenty of lengthy code to check the transport type of IB device, or the link layer type of it's port, but actually we are just speculating whether a particular management is supported by the device/port.
Michael,
There was a lot of input on this patch series which I believe indicates we would like to move forward. Were you working through another version of the series with the suggested changes?
I've sent the v2 which is more complete as a solution IMHO, let's see if it match the requirements :-)
Regards,
Michael Wang
Quoted 64 lines
Thanks,
IraQuoted 59 lines
Thus instead of inferring, we should have our own mechanism for IB management capability/protocol checking, several proposals below.
This patch set is the pioneer which try to collecting all the cases where a management helper applicable, and put all the speculating logical together, in order to be easily integrated to the new querying mechanism in future.
TODO:
The helper in patch 10 and 11 are still to be classified, that iscap_ib()andcap_eth().The name/description of each helper may not correct enough, those folks who are familiar with these particular parts, your suggestion would be invaluable ;-)
Proposals:
Sean: https://www.mail-archive.com/[email protected]/msg23339.html Doug: https://www.mail-archive.com/[email protected]/msg23418.html Jason: https://www.mail-archive.com/[email protected]/msg23425.htmlMichael Wang (11):
[PATCH 01/11] IB/Verbs: Use helpers to check transport and link layer
[RFC PATCH 02/11] IB/Verbs: Use management helpertech_iboe()for iboe-check [RFC PATCH 03/11] IB/Verbs: Use management helperhas_mad()for mad-check [RFC PATCH 04/11] IB/Verbs: Use management helpercap_smi()for smi-check [RFC PATCH 05/11] IB/Verbs: Use management helperhas_cm()for cm-check
[RFC PATCH 06/11] IB/Verbs: Use management helperhas_sa()andcap_sa()for sa-check [RFC PATCH 07/11] IB/Verbs: Use management helperhas_mcast()andcap_mcast()for mcast-check [RFC PATCH 08/11] IB/Verbs: Use management helperhas_iwarp()for iwarp-check [RFC PATCH 09/11] IB/Verbs: Use management helperhas_ipoib()andcap_ipoib()for ipoib-check [RFC PATCH 10/11] IB/Verbs: Use management helpercap_ib()for ib-check
[RFC PATCH 11/11] IB/Verbs: Use management helpercap_eth()for eth-check--- drivers/infiniband/core/agent.c | 6 drivers/infiniband/core/cm.c | 6 drivers/infiniband/core/cma.c | 79 ++++------ drivers/infiniband/core/mad.c | 18 +- drivers/infiniband/core/multicast.c | 30 +--- drivers/infiniband/core/sa_query.c | 42 ++--- drivers/infiniband/core/ucm.c | 8 - drivers/infiniband/core/user_mad.c | 6 drivers/infiniband/core/verbs.c | 14 - drivers/infiniband/hw/mlx4/ah.c | 6 drivers/infiniband/hw/mlx4/cq.c | 10 - drivers/infiniband/hw/mlx4/mad.c | 36 +--- drivers/infiniband/hw/mlx4/main.c | 23 +-- drivers/infiniband/hw/mlx4/mlx4_ib.h | 6 drivers/infiniband/hw/mlx4/qp.c | 57 ++----- drivers/infiniband/hw/mlx4/sysfs.c | 16 -- drivers/infiniband/ulp/ipoib/ipoib_main.c | 18 +- include/rdma/ib_verbs.h | 218 ++++++++++++++++++++++++++++++ net/sunrpc/xprtrdma/svc_rdma_recvfrom.c | 8 - 19 files changed, 381 insertions(+), 226 deletions(-)