IB/Verbs: IB Management Helpers
Subsystems (9)
v4 cover IB/Verbs: IB Management Helpers
Since v3:
* Apply suggestions from Sean, Ira, Jason, thanks for the comments :-) and please remind me if I missed anything :-P
* No more cap_XX_dev(), all the check are per-port now, use port 1 for cases only care about the cm type, can't support mixing cm type device in future.
There are plenty of lengthy code to check the transport type of IB device, or the link layer type of it's port, but actually we are just speculating whether a particular management/feature is supported by the device/port.
Thus instead of inferring, we should have our own mechanism for IB management capability/protocol/feature checking, several proposals below.
This patch set will reform the method of getting transport type, we will now using query_transport() instead of inferring from transport and link layer respectively, also we defined the new transport type to make the concept more reasonable.
Mapping List:
node-type link-layer old-transport new-transport
nes RNIC ETH IWARP IWARP
amso1100 RNIC ETH IWARP IWARP
cxgb3 RNIC ETH IWARP IWARP cxgb4 RNIC ETH IWARP IWARP usnic USNIC_UDP ETH USNIC_UDP USNIC_UDP ocrdma IB_CA ETH IB IBOE mlx4 IB_CA IB/ETH IB IB/IBOE mlx5 IB_CA IB IB IB ehca IB_CA IB IB IB ipath IB_CA IB IB IB mthca IB_CA IB IB IB qib IB_CA IB IB IB
For example:
if (transport == IB) && (link-layer == ETH) will now become: if (query_transport() == IBOE)
Thus we will be able to get rid of the respective transport and link-layer checking, and it will help us to add new protocol/Technology (like OPA) more easier, also with the introduced management helpers, IB management logical will be more clear and easier for extending.
Highlights:
The patch set covered a wide range of IB stuff, thus for those who are familiar with the particular part, your suggestion would be invaluable ;-)
Patches haven't been tested yet, we appreciate if any one who have these HW willing to provide his Tested-by :-)
Doug suggested the bitmask mechanism:
https://www.mail-archive.com/[email protected]/msg23765.html
which could be the plan for future reforming, we prefer that to be another series which focus on semantic and performance.
This patch-set is somewhat 'bloated' now and it may be a good timing for staging, I'd like to suggest we focus on improving existed helpers and push all the further reforms into next series ;-)
Proposals:
Sean: https://www.mail-archive.com/[email protected]/msg23339.html Doug: https://www.mail-archive.com/[email protected]/msg23418.html https://www.mail-archive.com/[email protected]/msg23765.html Jason: https://www.mail-archive.com/[email protected]/msg23425.html
Michael Wang (27):
IB/Verbs: Implement new callback query_transport() IB/Verbs: Implement raw management helpers IB/Verbs: Reform IB-core mad/agent/user_mad IB/Verbs: Reform IB-core cm
IB/Verbs: Reform IB-core sa_query
IB/Verbs: Reform IB-core multicast
IB/Verbs: Reform IB-ulp ipoib
IB/Verbs: Reform IB-ulp xprtrdma
IB/Verbs: Reform IB-core verbs/uverbs_cmd/sysfs IB/Verbs: Reform cm related part in IB-core cma/ucm IB/Verbs: Reform route related part in IB-core cma IB/Verbs: Reform mcast related part in IB-core cma IB/Verbs: Reserve legacy transport type in 'dev_addr' IB/Verbs: Reform cma_acquire_dev()
IB/Verbs: Reform rest part in IB-core cma IB/Verbs: Use management helper cap_ib_mad() IB/Verbs: Use management helper cap_ib_smi() IB/Verbs: Use management helper cap_ib_cm() IB/Verbs: Use management helper cap_iw_cm() IB/Verbs: Use management helper cap_ib_sa() IB/Verbs: Use management helper cap_ib_mcast() IB/Verbs: Use management helper cap_ipoib() IB/Verbs: Use management helper cap_read_multi_sge() IB/Verbs: Use management helper cap_af_ib() IB/Verbs: Use management helper cap_eth_ah() IB/Verbs: Clean up rdma_ib_or_iboe() IB/Verbs: Cleanup rdma_node_get_transport()
--- drivers/infiniband/core/agent.c | 4 drivers/infiniband/core/cm.c | 28 +- drivers/infiniband/core/cma.c | 330 ++++++++++++--------------- drivers/infiniband/core/device.c | 1 drivers/infiniband/core/mad.c | 28 +- drivers/infiniband/core/multicast.c | 18 - drivers/infiniband/core/sa_query.c | 41 +-- drivers/infiniband/core/sysfs.c | 8 drivers/infiniband/core/ucm.c | 5 drivers/infiniband/core/ucma.c | 27 -- drivers/infiniband/core/user_mad.c | 32 +- drivers/infiniband/core/uverbs_cmd.c | 6 drivers/infiniband/core/verbs.c | 33 -- drivers/infiniband/hw/amso1100/c2_provider.c | 7 drivers/infiniband/hw/cxgb3/iwch_provider.c | 7 drivers/infiniband/hw/cxgb4/provider.c | 7 drivers/infiniband/hw/ehca/ehca_hca.c | 6 drivers/infiniband/hw/ehca/ehca_iverbs.h | 3 drivers/infiniband/hw/ehca/ehca_main.c | 1 drivers/infiniband/hw/ipath/ipath_verbs.c | 7 drivers/infiniband/hw/mlx4/main.c | 10 drivers/infiniband/hw/mlx5/main.c | 7 drivers/infiniband/hw/mthca/mthca_provider.c | 7 drivers/infiniband/hw/nes/nes_verbs.c | 6 drivers/infiniband/hw/ocrdma/ocrdma_main.c | 1 drivers/infiniband/hw/ocrdma/ocrdma_verbs.c | 6 drivers/infiniband/hw/ocrdma/ocrdma_verbs.h | 3 drivers/infiniband/hw/qib/qib_verbs.c | 7 drivers/infiniband/hw/usnic/usnic_ib_main.c | 1 drivers/infiniband/hw/usnic/usnic_ib_verbs.c | 6 drivers/infiniband/hw/usnic/usnic_ib_verbs.h | 2 drivers/infiniband/ulp/ipoib/ipoib_main.c | 17 - include/rdma/ib_verbs.h | 204 +++++++++++++++- net/sunrpc/xprtrdma/svc_rdma_recvfrom.c | 6 net/sunrpc/xprtrdma/svc_rdma_transport.c | 51 +--- 35 files changed, 579 insertions(+), 354 deletions(-)
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
PATCH v4 1/27 IB/Verbs: Implement new callback query_transport()
Add new callback query_transport() and implement for each HW.
Mapping List:
node-type link-layer old-transport new-transport
nes RNIC ETH IWARP IWARP
amso1100 RNIC ETH IWARP IWARP
cxgb3 RNIC ETH IWARP IWARP cxgb4 RNIC ETH IWARP IWARP usnic USNIC_UDP ETH USNIC_UDP USNIC_UDP ocrdma IB_CA ETH IB IBOE mlx4 IB_CA IB/ETH IB IB/IBOE mlx5 IB_CA IB IB IB ehca IB_CA IB IB IB ipath IB_CA IB IB IB mthca IB_CA IB IB IB qib IB_CA IB IB IB
--- drivers/infiniband/core/device.c | 1 + drivers/infiniband/core/verbs.c | 4 +++- drivers/infiniband/hw/amso1100/c2_provider.c | 7 +++++++ drivers/infiniband/hw/cxgb3/iwch_provider.c | 7 +++++++ drivers/infiniband/hw/cxgb4/provider.c | 7 +++++++ drivers/infiniband/hw/ehca/ehca_hca.c | 6 ++++++ drivers/infiniband/hw/ehca/ehca_iverbs.h | 3 +++ drivers/infiniband/hw/ehca/ehca_main.c | 1 + drivers/infiniband/hw/ipath/ipath_verbs.c | 7 +++++++ drivers/infiniband/hw/mlx4/main.c | 10 ++++++++++ drivers/infiniband/hw/mlx5/main.c | 7 +++++++ drivers/infiniband/hw/mthca/mthca_provider.c | 7 +++++++ drivers/infiniband/hw/nes/nes_verbs.c | 6 ++++++ drivers/infiniband/hw/ocrdma/ocrdma_main.c | 1 + drivers/infiniband/hw/ocrdma/ocrdma_verbs.c | 6 ++++++ drivers/infiniband/hw/ocrdma/ocrdma_verbs.h | 3 +++ drivers/infiniband/hw/qib/qib_verbs.c | 7 +++++++ drivers/infiniband/hw/usnic/usnic_ib_main.c | 1 + drivers/infiniband/hw/usnic/usnic_ib_verbs.c | 6 ++++++ drivers/infiniband/hw/usnic/usnic_ib_verbs.h | 2 ++ include/rdma/ib_verbs.h | 7 ++++++- 21 files changed, 104 insertions(+), 2 deletions(-)
drivers/infiniband/core/device.c +1 -0
@@ -76,6 +76,7 @@ static int ib_device_check_mandatory(struct ib_device *device) } mandatory_table[] = {IB_MANDATORY_FUNC (query_device),IB_MANDATORY_FUNC (query_port), +IB_MANDATORY_FUNC (query_transport),IB_MANDATORY_FUNC (query_pkey),IB_MANDATORY_FUNC (query_gid), IB_MANDATORY_FUNC(alloc_pd),
drivers/infiniband/core/verbs.c +3 -1
@@ -133,14 +133,16 @@ enum rdma_link_layer rdma_port_get_link_layer(struct ib_device *device, u8 port_if (device->get_link_layer)return device->get_link_layer (device, port_num); -switch (rdma_node_get_transport (device->node_type)) { +switch (device->query_transport (device, port_num)) {case RDMA_TRANSPORT_IB :return IB_LINK_LAYER_INFINIBAND ; +case RDMA_TRANSPORT_IBOE :case RDMA_TRANSPORT_IWARP :case RDMA_TRANSPORT_USNIC :case RDMA_TRANSPORT_USNIC_UDP :return IB_LINK_LAYER_ETHERNET ;default : +BUG ();return IB_LINK_LAYER_UNSPECIFIED ; } }
drivers/infiniband/hw/amso1100/c2_provider.c +7 -0
@@ -99,6 +99,12 @@ static int c2_query_port(struct ib_device *ibdev,return 0 ; } +static enum rdma_transport_type +c2_query_transport (struct ib_device *device,u8 port_num) +{ +return RDMA_TRANSPORT_IWARP ; +} +static int c2_query_pkey (struct ib_device *ibdev,u8 port,u16 index,u16 * pkey) {
@@ -801,6 +807,7 @@ int c2_register_device(struct c2_dev *dev) dev->ibdev.dma_device = &dev->pcidev->dev; dev->ibdev.query_device = c2_query_device; dev->ibdev.query_port = c2_query_port; + dev->ibdev.query_transport = c2_query_transport; dev->ibdev.query_pkey = c2_query_pkey; dev->ibdev.query_gid = c2_query_gid; dev->ibdev.alloc_ucontext = c2_alloc_ucontext;
drivers/infiniband/hw/cxgb3/iwch_provider.c +7 -0
@@ -1232,6 +1232,12 @@ static int iwch_query_port(struct ib_device *ibdev,return 0 ; } +static enum rdma_transport_type +iwch_query_transport (struct ib_device *device,u8 port_num) +{ +return RDMA_TRANSPORT_IWARP ; +} +static ssize_t show_rev (struct device *dev,struct device_attribute *attr,char *buf) {
@@ -1385,6 +1391,7 @@ int iwch_register_device(struct iwch_dev *dev) dev->ibdev.dma_device = &(dev->rdev.rnic_info.pdev->dev); dev->ibdev.query_device = iwch_query_device; dev->ibdev.query_port = iwch_query_port; + dev->ibdev.query_transport = iwch_query_transport; dev->ibdev.query_pkey = iwch_query_pkey; dev->ibdev.query_gid = iwch_query_gid; dev->ibdev.alloc_ucontext = iwch_alloc_ucontext;
drivers/infiniband/hw/cxgb4/provider.c +7 -0
@@ -390,6 +390,12 @@ static int c4iw_query_port(struct ib_device *ibdev, u8 port,return 0 ; } +static enum rdma_transport_type +c4iw_query_transport (struct ib_device *device,u8 port_num) +{ +return RDMA_TRANSPORT_IWARP ; +} +static ssize_t show_rev (struct device *dev,struct device_attribute *attr,char *buf) {
@@ -506,6 +512,7 @@ int c4iw_register_device(struct c4iw_dev *dev) dev->ibdev.dma_device = &(dev->rdev.lldi.pdev->dev); dev->ibdev.query_device = c4iw_query_device; dev->ibdev.query_port = c4iw_query_port; + dev->ibdev.query_transport = c4iw_query_transport; dev->ibdev.query_pkey = c4iw_query_pkey; dev->ibdev.query_gid = c4iw_query_gid; dev->ibdev.alloc_ucontext = c4iw_alloc_ucontext;
drivers/infiniband/hw/ehca/ehca_hca.c +6 -0
@@ -242,6 +242,12 @@ query_port1:return ret; } +enum rdma_transport_type +ehca_query_transport (struct ib_device *device,u8 port_num) +{ +return RDMA_TRANSPORT_IB ; +} +int ehca_query_sma_attr (struct ehca_shca *shca,u8 port,struct ehca_sma_attr *attr) {
drivers/infiniband/hw/ehca/ehca_iverbs.h +3 -0
@@ -49,6 +49,9 @@ int ehca_query_device(struct ib_device *ibdev, struct ib_device_attr *props);int ehca_query_port (struct ib_device *ibdev,u8 port,struct ib_port_attr *props); +enum rdma_transport_type +ehca_query_transport (struct ib_device *device,u8 port_num); +int ehca_query_sma_attr (struct ehca_shca *shca,u8 port,struct ehca_sma_attr *attr);
drivers/infiniband/hw/ehca/ehca_main.c +1 -0
@@ -467,6 +467,7 @@ static int ehca_init_device(struct ehca_shca *shca) shca->ib_device.dma_device = &shca->ofdev->dev; shca->ib_device.query_device = ehca_query_device; shca->ib_device.query_port = ehca_query_port; + shca->ib_device.query_transport = ehca_query_transport; shca->ib_device.query_gid = ehca_query_gid; shca->ib_device.query_pkey = ehca_query_pkey;/* shca->in_device.modify_device = ehca_modify_device */
drivers/infiniband/hw/ipath/ipath_verbs.c +7 -0
@@ -1638,6 +1638,12 @@ static int ipath_query_port(struct ib_device *ibdev,return 0 ; } +static enum rdma_transport_type +ipath_query_transport (struct ib_device *device,u8 port_num) +{ +return RDMA_TRANSPORT_IB ; +} +static int ipath_modify_device (struct ib_device *device,int device_modify_mask,struct ib_device_modify *device_modify)
@@ -2140,6 +2146,7 @@ int ipath_register_ib_device(struct ipath_devdata *dd) dev->query_device = ipath_query_device; dev->modify_device = ipath_modify_device; dev->query_port = ipath_query_port; + dev->query_transport = ipath_query_transport; dev->modify_port = ipath_modify_port; dev->query_pkey = ipath_query_pkey; dev->query_gid = ipath_query_gid;
drivers/infiniband/hw/mlx4/main.c +10 -0
@@ -420,6 +420,15 @@ static int mlx4_ib_query_port(struct ib_device *ibdev, u8 port,return __mlx4_ib_query_port (ibdev, port, props,0 ); } +static enum rdma_transport_type +mlx4_ib_query_transport (struct ib_device *device,u8 port_num) +{ +struct mlx4_dev *dev =to_mdev (device)->dev; + +return dev->caps.port_mask[port_num] ==MLX4_PORT_TYPE_IB ? +RDMA_TRANSPORT_IB :RDMA_TRANSPORT_IBOE ; +} +int __mlx4_ib_query_gid (struct ib_device *ibdev,u8 port,int index,union ib_gid *gid,int netw_view) {
@@ -2201,6 +2210,7 @@ static void *mlx4_ib_add(struct mlx4_dev *dev) ibdev->ib_dev.query_device = mlx4_ib_query_device; ibdev->ib_dev.query_port = mlx4_ib_query_port; + ibdev->ib_dev.query_transport = mlx4_ib_query_transport; ibdev->ib_dev.get_link_layer = mlx4_ib_port_link_layer; ibdev->ib_dev.query_gid = mlx4_ib_query_gid; ibdev->ib_dev.query_pkey = mlx4_ib_query_pkey;
drivers/infiniband/hw/mlx5/main.c +7 -0
@@ -351,6 +351,12 @@ out:return err; } +static enum rdma_transport_type +mlx5_ib_query_transport (struct ib_device *device,u8 port_num) +{ +return RDMA_TRANSPORT_IB ; +} +static int mlx5_ib_query_gid (struct ib_device *ibdev,u8 port,int index,union ib_gid *gid) {
@@ -1336,6 +1342,7 @@ static void *mlx5_ib_add(struct mlx5_core_dev *mdev) dev->ib_dev.query_device = mlx5_ib_query_device; dev->ib_dev.query_port = mlx5_ib_query_port; + dev->ib_dev.query_transport = mlx5_ib_query_transport; dev->ib_dev.query_gid = mlx5_ib_query_gid; dev->ib_dev.query_pkey = mlx5_ib_query_pkey; dev->ib_dev.modify_device = mlx5_ib_modify_device;
drivers/infiniband/hw/mthca/mthca_provider.c +7 -0
@@ -179,6 +179,12 @@ static int mthca_query_port(struct ib_device *ibdev,return err; } +static enum rdma_transport_type +mthca_query_transport (struct ib_device *device,u8 port_num) +{ +return RDMA_TRANSPORT_IB ; +} +static int mthca_modify_device (struct ib_device *ibdev,int mask,struct ib_device_modify *props)
@@ -1281,6 +1287,7 @@ int mthca_register_device(struct mthca_dev *dev) dev->ib_dev.dma_device = &dev->pdev->dev; dev->ib_dev.query_device = mthca_query_device; dev->ib_dev.query_port = mthca_query_port; + dev->ib_dev.query_transport = mthca_query_transport; dev->ib_dev.modify_device = mthca_modify_device; dev->ib_dev.modify_port = mthca_modify_port; dev->ib_dev.query_pkey = mthca_query_pkey;
drivers/infiniband/hw/nes/nes_verbs.c +6 -0
@@ -606,6 +606,11 @@ static int nes_query_port(struct ib_device *ibdev, u8 port, struct ib_port_attrreturn 0 ; } +static enum rdma_transport_type +nes_query_transport (struct ib_device *device,u8 port_num) +{ +return RDMA_TRANSPORT_IWARP ; +}/** * nes_query_pkey
@@ -3879,6 +3884,7 @@ struct nes_ib_device *nes_init_ofa_device(struct net_device *netdev) nesibdev->ibdev.dev.parent = &nesdev->pcidev->dev; nesibdev->ibdev.query_device = nes_query_device; nesibdev->ibdev.query_port = nes_query_port; + nesibdev->ibdev.query_transport = nes_query_transport; nesibdev->ibdev.query_pkey = nes_query_pkey; nesibdev->ibdev.query_gid = nes_query_gid; nesibdev->ibdev.alloc_ucontext = nes_alloc_ucontext;
drivers/infiniband/hw/ocrdma/ocrdma_main.c +1 -0
@@ -244,6 +244,7 @@ static int ocrdma_register_device(struct ocrdma_dev *dev)/* mandatory verbs. */ dev->ibdev.query_device = ocrdma_query_device; dev->ibdev.query_port = ocrdma_query_port; + dev->ibdev.query_transport = ocrdma_query_transport; dev->ibdev.modify_port = ocrdma_modify_port; dev->ibdev.query_gid = ocrdma_query_gid; dev->ibdev.get_link_layer = ocrdma_link_layer;
drivers/infiniband/hw/ocrdma/ocrdma_verbs.c +6 -0
@@ -187,6 +187,12 @@ int ocrdma_query_port(struct ib_device *ibdev,return 0 ; } +enum rdma_transport_type +ocrdma_query_transport (struct ib_device *device,u8 port_num) +{ +return RDMA_TRANSPORT_IBOE ; +} +int ocrdma_modify_port (struct ib_device *ibdev,u8 port,int mask,struct ib_port_modify *props) {
drivers/infiniband/hw/ocrdma/ocrdma_verbs.h +3 -0
@@ -41,6 +41,9 @@ int ocrdma_query_port(struct ib_device *, u8 port, struct ib_port_attr *props);int ocrdma_modify_port (struct ib_device *,u8 port,int mask,struct ib_port_modify *props); +enum rdma_transport_type +ocrdma_query_transport (struct ib_device *device,u8 port_num); +void ocrdma_get_guid (struct ocrdma_dev *,u8 *guid);int ocrdma_query_gid (struct ib_device *,u8 port,int index,union ib_gid *gid);
drivers/infiniband/hw/qib/qib_verbs.c +7 -0
@@ -1650,6 +1650,12 @@ static int qib_query_port(struct ib_device *ibdev, u8 port,return 0 ; } +static enum rdma_transport_type +qib_query_transport (struct ib_device *device,u8 port_num) +{ +return RDMA_TRANSPORT_IB ; +} +static int qib_modify_device (struct ib_device *device,int device_modify_mask,struct ib_device_modify *device_modify)
@@ -2184,6 +2190,7 @@ int qib_register_ib_device(struct qib_devdata *dd) ibdev->query_device = qib_query_device; ibdev->modify_device = qib_modify_device; ibdev->query_port = qib_query_port; + ibdev->query_transport = qib_query_transport; ibdev->modify_port = qib_modify_port; ibdev->query_pkey = qib_query_pkey; ibdev->query_gid = qib_query_gid;
drivers/infiniband/hw/usnic/usnic_ib_main.c +1 -0
@@ -360,6 +360,7 @@ static void *usnic_ib_device_add(struct pci_dev *dev) us_ibdev->ib_dev.query_device = usnic_ib_query_device; us_ibdev->ib_dev.query_port = usnic_ib_query_port; + us_ibdev->ib_dev.query_transport = usnic_ib_query_transport; us_ibdev->ib_dev.query_pkey = usnic_ib_query_pkey; us_ibdev->ib_dev.query_gid = usnic_ib_query_gid; us_ibdev->ib_dev.get_link_layer = usnic_ib_port_link_layer;
drivers/infiniband/hw/usnic/usnic_ib_verbs.c +6 -0
@@ -348,6 +348,12 @@ int usnic_ib_query_port(struct ib_device *ibdev, u8 port,return 0 ; } +enum rdma_transport_type +usnic_ib_query_transport (struct ib_device *device,u8 port_num) +{ +return RDMA_TRANSPORT_USNIC_UDP ; +} +int usnic_ib_query_qp (struct ib_qp *qp,struct ib_qp_attr *qp_attr,int qp_attr_mask,struct ib_qp_init_attr *qp_init_attr)
drivers/infiniband/hw/usnic/usnic_ib_verbs.h +2 -0
@@ -27,6 +27,8 @@ int usnic_ib_query_device(struct ib_device *ibdev, struct ib_device_attr *props);int usnic_ib_query_port (struct ib_device *ibdev,u8 port,struct ib_port_attr *props); +enum rdma_transport_type +usnic_ib_query_transport (struct ib_device *device,u8 port_num);int usnic_ib_query_qp (struct ib_qp *qp,struct ib_qp_attr *qp_attr,int qp_attr_mask,struct ib_qp_init_attr *qp_init_attr);
include/rdma/ib_verbs.h +6 -1
@@ -75,10 +75,13 @@ enum rdma_node_type { };enum rdma_transport_type { +/* legacy for users */ RDMA_TRANSPORT_IB ,RDMA_TRANSPORT_IWARP ,RDMA_TRANSPORT_USNIC , -RDMA_TRANSPORT_USNIC_UDP +RDMA_TRANSPORT_USNIC_UDP , +/* new transport */ +RDMA_TRANSPORT_IBOE , }; __attribute_const__ enum rdma_transport_type
@@ -1501,6 +1504,8 @@ struct ib_device {int (*query_port)(struct ib_device *device,u8 port_num,struct ib_port_attr *port_attr); +enum rdma_transport_type (*query_transport)(struct ib_device *device, +u8 port_num);enum rdma_link_layer (*get_link_layer)(struct ib_device *device,u8 port_num); int (*query_gid)(struct ib_device *device,
Signature · 1 line
2.1.0
PATCH v4 2/27 IB/Verbs: Implement raw management helpers
In reply to Michael WangAdd raw helpers:
rdma_tech_ib
rdma_tech_iboe
rdma_tech_iwarp
rdma_ib_or_iboe (transition, clean up later)
To help us detect which technology the port supported.
--- include/rdma/ib_verbs.h | 25 +++++++++++++++++++++++++ 1 file changed, 25 insertions(+)
include/rdma/ib_verbs.h +25 -0
@@ -1748,6 +1748,31 @@ int ib_query_port(struct ib_device *device,enum rdma_link_layer rdma_port_get_link_layer (struct ib_device *device,u8 port_num); +static inline int rdma_tech_ib (struct ib_device *device,u8 port_num) +{ +return device->query_transport (device, port_num) + ==RDMA_TRANSPORT_IB ; +} + +static inline int rdma_tech_iboe (struct ib_device *device,u8 port_num) +{ +return device->query_transport (device, port_num) + ==RDMA_TRANSPORT_IBOE ; +} + +static inline int rdma_tech_iwarp (struct ib_device *device,u8 port_num) +{ +return device->query_transport (device, port_num) + ==RDMA_TRANSPORT_IWARP ; +} + +static inline int rdma_ib_or_iboe (struct ib_device *device,u8 port_num) +{ +enum rdma_transport_type tp = device->query_transport (device, port_num); + +return (tp ==RDMA_TRANSPORT_IB || tp ==RDMA_TRANSPORT_IBOE ); +} +int ib_query_gid (struct ib_device *device,u8 port_num,int index,union ib_gid *gid);
Signature · 5 lines
2.1.0 -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
PATCH v4 3/27 IB/Verbs: Reform IB-core mad/agent/user_mad
In reply to Michael WangUse raw management helpers to reform IB-core mad/agent/user_mad.
--- drivers/infiniband/core/agent.c | 2 +- drivers/infiniband/core/mad.c | 20 ++++++++++---------- drivers/infiniband/core/user_mad.c | 26 ++++++++++++++++++++------ 3 files changed, 31 insertions(+), 17 deletions(-)
drivers/infiniband/core/agent.c +1 -1
@@ -156,7 +156,7 @@ int ib_agent_port_open(struct ib_device *device, int port_num) goto error1; } -if (rdma_port_get_link_layer (device, port_num) ==IB_LINK_LAYER_INFINIBAND ) { +if (rdma_tech_ib (device, port_num)) {/* Obtain send only MAD agent for SMI QP */ port_priv->agent[0] = ib_register_mad_agent(device, port_num, IB_QPT_SMI, NULL, 0,
drivers/infiniband/core/mad.c +10 -10
@@ -2938,7 +2938,7 @@ static int ib_mad_port_open(struct ib_device *device,init_mad_qp (port_priv, &port_priv->qp_info[1 ]); cq_size = mad_sendq_size + mad_recvq_size; - has_smi =rdma_port_get_link_layer (device, port_num) ==IB_LINK_LAYER_INFINIBAND ; + has_smi =rdma_tech_ib (device, port_num);if (has_smi) cq_size *=2 ;
@@ -3057,9 +3057,6 @@ static void ib_mad_init_device(struct ib_device *device) {int start, end, i; -if (rdma_node_get_transport (device->node_type) !=RDMA_TRANSPORT_IB ) -return ; -if (device->node_type ==RDMA_NODE_IB_SWITCH ) { start =0 ; end =0 ;
@@ -3069,6 +3066,9 @@ static void ib_mad_init_device(struct ib_device *device) }for (i = start; i <= end; i++) { +if (!rdma_ib_or_iboe (device, i)) +continue ; +if (ib_mad_port_open (device, i)) {dev_err (&device->dev,"Couldn't open port %d\n" , i); goto error;
@@ -3086,15 +3086,15 @@ error_agent:dev_err (&device->dev,"Couldn't close port %d\n" , i); error: - i--; +while (--i >= start) { +if (!rdma_ib_or_iboe (device, i)) +continue ; -while (i >= start) {if (ib_agent_port_close (device, i))dev_err (&device->dev,"Couldn't close port %d for agents\n" , i);if (ib_mad_port_close (device, i))dev_err (&device->dev,"Couldn't close port %d\n" , i); - i--; } }
@@ -3102,9 +3102,6 @@ static void ib_mad_remove_device(struct ib_device *device) {int i, num_ports, cur_port; -if (rdma_node_get_transport (device->node_type) !=RDMA_TRANSPORT_IB ) -return ; -if (device->node_type ==RDMA_NODE_IB_SWITCH ) { num_ports =1 ; cur_port =0 ;
@@ -3113,6 +3110,9 @@ static void ib_mad_remove_device(struct ib_device *device) cur_port =1 ; }for (i =0 ; i < num_ports; i++, cur_port++) { +if (!rdma_ib_or_iboe (device, i)) +continue ; +if (ib_agent_port_close (device, cur_port)) dev_err(&device->dev, "Couldn't close port %d for agents\n",
drivers/infiniband/core/user_mad.c +20 -6
@@ -1273,9 +1273,7 @@ static void ib_umad_add_one(struct ib_device *device) {struct ib_umad_device *umad_dev;int s, e, i; - -if (rdma_node_get_transport (device->node_type) !=RDMA_TRANSPORT_IB ) -return ; +int count =0 ;if (device->node_type ==RDMA_NODE_IB_SWITCH ) s = e =0 ;
@@ -1296,11 +1294,21 @@ static void ib_umad_add_one(struct ib_device *device) umad_dev->end_port = e;for (i = s; i <= e; ++i) { +if (!rdma_ib_or_iboe (device, i)) +continue ; + umad_dev->port[i - s].umad_dev = umad_dev;if (ib_umad_init_port (device, i, umad_dev, &umad_dev->port[i - s])) goto err; + + count++; + } + +if (!count) { +kobject_put (&umad_dev->kobj); +return ; }ib_set_client_data (device, &umad_client, umad_dev);
@@ -1308,8 +1316,12 @@ static void ib_umad_add_one(struct ib_device *device)return ; err: -while (--i >= s) +while (--i >= s) { +if (!rdma_ib_or_iboe (device, i)) +continue ; +ib_umad_kill_port (&umad_dev->port[i - s]); + }kobject_put (&umad_dev->kobj); }
@@ -1322,8 +1334,10 @@ static void ib_umad_remove_one(struct ib_device *device)if (!umad_dev)return ; -for (i =0 ; i <= umad_dev->end_port - umad_dev->start_port; ++i) -ib_umad_kill_port (&umad_dev->port[i]); +for (i =0 ; i <= umad_dev->end_port - umad_dev->start_port; ++i) { +if (rdma_ib_or_iboe (device, i)) +ib_umad_kill_port (&umad_dev->port[i]); + }kobject_put (&umad_dev->kobj); }
Signature · 5 lines
2.1.0 -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
PATCH v4 4/27 IB/Verbs: Reform IB-core cm
In reply to Michael WangUse raw management helpers to reform IB-core cm.
--- drivers/infiniband/core/cm.c | 22 +++++++++++++++++++--- 1 file changed, 19 insertions(+), 3 deletions(-)
drivers/infiniband/core/cm.c +19 -3
@@ -3761,9 +3761,7 @@ static void cm_add_one(struct ib_device *ib_device)unsigned long flags;int ret;u8 i; - -if (rdma_node_get_transport (ib_device->node_type) !=RDMA_TRANSPORT_IB ) -return ; +int count =0 ; cm_dev =kzalloc (sizeof (*cm_dev) +sizeof (*port) * ib_device->phys_port_cnt,GFP_KERNEL );
@@ -3783,6 +3781,9 @@ static void cm_add_one(struct ib_device *ib_device)set_bit (IB_MGMT_METHOD_SEND , reg_req.method_mask);for (i =1 ; i <= ib_device->phys_port_cnt; i++) { +if (!rdma_ib_or_iboe (ib_device, i)) +continue ; + port =kzalloc (sizeof *port,GFP_KERNEL );if (!port) goto error1;
@@ -3809,7 +3810,16 @@ static void cm_add_one(struct ib_device *ib_device) ret =ib_modify_port (ib_device, i,0 , &port_modify);if (ret) goto error3; + + count++; } + +if (!count) { +device_unregister (cm_dev->device); +kfree (cm_dev); +return ; + } +ib_set_client_data (ib_device, &cm_client, cm_dev);write_lock_irqsave (&cm.device_lock, flags);
@@ -3825,6 +3835,9 @@ error1: port_modify.set_port_cap_mask =0 ; port_modify.clr_port_cap_mask =IB_PORT_CM_SUP ;while (--i) { +if (!rdma_ib_or_iboe (ib_device, i)) +continue ; + port = cm_dev->port[i-1 ];ib_modify_port (ib_device, port->port_num,0 , &port_modify);ib_unregister_mad_agent (port->mad_agent);
@@ -3853,6 +3866,9 @@ static void cm_remove_one(struct ib_device *ib_device)write_unlock_irqrestore (&cm.device_lock, flags);for (i =1 ; i <= ib_device->phys_port_cnt; i++) { +if (!rdma_ib_or_iboe (ib_device, i)) +continue ; + port = cm_dev->port[i-1 ];ib_modify_port (ib_device, port->port_num,0 , &port_modify);ib_unregister_mad_agent (port->mad_agent);
Signature · 5 lines
2.1.0 -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
PATCH v4 5/27 IB/Verbs: Reform IB-core sa_query
In reply to Michael WangUse raw management helpers to reform IB-core sa_query.
--- drivers/infiniband/core/sa_query.c | 29 ++++++++++++++++++----------- 1 file changed, 18 insertions(+), 11 deletions(-)
drivers/infiniband/core/sa_query.c +18 -11
@@ -450,7 +450,7 @@ static void ib_sa_event(struct ib_event_handler *handler, struct ib_event *eventstruct ib_sa_port *port = &sa_dev->port[event->element.port_num - sa_dev->start_port]; -if (rdma_port_get_link_layer (handler->device, port->port_num) !=IB_LINK_LAYER_INFINIBAND ) +if (WARN_ON (!rdma_tech_ib (handler->device, port->port_num)))return ;spin_lock_irqsave (&port->ah_lock, flags);
@@ -540,7 +540,7 @@ int ib_init_ah_from_path(struct ib_device *device, u8 port_num, ah_attr->port_num = port_num; ah_attr->static_rate = rec->rate; - force_grh =rdma_port_get_link_layer (device, port_num) ==IB_LINK_LAYER_ETHERNET ; + force_grh =rdma_tech_iboe (device, port_num);if (rec->hop_limit >1 || force_grh) { ah_attr->ah_flags =IB_AH_GRH ;
@@ -1153,9 +1153,7 @@ static void ib_sa_add_one(struct ib_device *device) {struct ib_sa_device *sa_dev;int s, e, i; - -if (rdma_node_get_transport (device->node_type) !=RDMA_TRANSPORT_IB ) -return ; +int count =0 ;if (device->node_type ==RDMA_NODE_IB_SWITCH ) s = e =0 ;
@@ -1175,7 +1173,7 @@ static void ib_sa_add_one(struct ib_device *device)for (i =0 ; i <= e - s; ++i) {spin_lock_init (&sa_dev->port[i].ah_lock); -if (rdma_port_get_link_layer (device, i +1 ) !=IB_LINK_LAYER_INFINIBAND ) +if (!rdma_tech_ib (device, i +1 ))continue ; sa_dev->port[i].sm_ah =NULL ;
@@ -1189,6 +1187,13 @@ static void ib_sa_add_one(struct ib_device *device) goto err;INIT_WORK (&sa_dev->port[i].update_task, update_sm_ah); + + count++; + } + +if (!count) { +kfree (sa_dev); +return ; }ib_set_client_data (device, &sa_client, sa_dev);
@@ -1204,16 +1209,18 @@ static void ib_sa_add_one(struct ib_device *device)if (ib_register_event_handler (&sa_dev->event_handler)) goto err; -for (i =0 ; i <= e - s; ++i) -if (rdma_port_get_link_layer (device, i +1 ) ==IB_LINK_LAYER_INFINIBAND ) +for (i =0 ; i <= e - s; ++i) { +if (rdma_tech_ib (device, i +1 ))update_sm_ah (&sa_dev->port[i].update_task); + }return ; err: -while (--i >=0 ) -if (rdma_port_get_link_layer (device, i +1 ) ==IB_LINK_LAYER_INFINIBAND ) +while (--i >=0 ) { +if (rdma_tech_ib (device, i +1 ))ib_unregister_mad_agent (sa_dev->port[i].agent); + }kfree (sa_dev);
@@ -1233,7 +1240,7 @@ static void ib_sa_remove_one(struct ib_device *device)flush_workqueue (ib_wq);for (i =0 ; i <= sa_dev->end_port - sa_dev->start_port; ++i) { -if (rdma_port_get_link_layer (device, i +1 ) ==IB_LINK_LAYER_INFINIBAND ) { +if (rdma_tech_ib (device, i +1 )) {ib_unregister_mad_agent (sa_dev->port[i].agent);if (sa_dev->port[i].sm_ah)kref_put (&sa_dev->port[i].sm_ah->ref, free_sm_ah);
Signature · 5 lines
2.1.0 -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
PATCH v4 6/27 IB/Verbs: Reform IB-core multicast
In reply to Michael WangUse raw management helpers to reform IB-core multicast.
--- drivers/infiniband/core/multicast.c | 12 +++--------- 1 file changed, 3 insertions(+), 9 deletions(-)
drivers/infiniband/core/multicast.c +3 -9
@@ -780,8 +780,7 @@ static void mcast_event_handler(struct ib_event_handler *handler,int index; dev = container_of(handler, struct mcast_device, event_handler); -if (rdma_port_get_link_layer (dev->device, event->element.port_num) != -IB_LINK_LAYER_INFINIBAND ) +if (WARN_ON (!rdma_tech_ib (dev->device, event->element.port_num)))return ; index = event->element.port_num - dev->start_port;
@@ -808,9 +807,6 @@ static void mcast_add_one(struct ib_device *device)int i;int count =0 ; -if (rdma_node_get_transport (device->node_type) !=RDMA_TRANSPORT_IB ) -return ; - dev =kmalloc (sizeof *dev + device->phys_port_cnt *sizeof *port,GFP_KERNEL );if (!dev)
@@ -824,8 +820,7 @@ static void mcast_add_one(struct ib_device *device) }for (i =0 ; i <= dev->end_port - dev->start_port; i++) { -if (rdma_port_get_link_layer (device, dev->start_port + i) != -IB_LINK_LAYER_INFINIBAND ) +if (!rdma_tech_ib (device, dev->start_port + i))continue ; port = &dev->port[i]; port->dev = dev;
@@ -863,8 +858,7 @@ static void mcast_remove_one(struct ib_device *device)flush_workqueue (mcast_wq);for (i =0 ; i <= dev->end_port - dev->start_port; i++) { -if (rdma_port_get_link_layer (device, dev->start_port + i) == -IB_LINK_LAYER_INFINIBAND ) { +if (rdma_tech_ib (device, dev->start_port + i)) { port = &dev->port[i];deref_port (port);wait_for_completion (&port->comp);
Signature · 5 lines
2.1.0 -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
PATCH v4 7/27 IB/Verbs: Reform IB-ulp ipoib
In reply to Michael WangUse raw management helpers to reform IB-ulp ipoib.
--- drivers/infiniband/ulp/ipoib/ipoib_main.c | 15 ++++++++------- 1 file changed, 8 insertions(+), 7 deletions(-)
drivers/infiniband/ulp/ipoib/ipoib_main.c +8 -7
@@ -1654,9 +1654,7 @@ static void ipoib_add_one(struct ib_device *device)struct net_device *dev;struct ipoib_dev_priv *priv;int s, e, p; - -if (rdma_node_get_transport (device->node_type) !=RDMA_TRANSPORT_IB ) -return ; +int count =0 ; dev_list =kmalloc (sizeof *dev_list,GFP_KERNEL );if (!dev_list)
@@ -1673,15 +1671,21 @@ static void ipoib_add_one(struct ib_device *device) }for (p = s; p <= e; ++p) { -if (rdma_port_get_link_layer (device, p) !=IB_LINK_LAYER_INFINIBAND ) +if (!rdma_tech_ib (device, p))continue ; dev =ipoib_add_port ("ib%d" , device, p);if (!IS_ERR (dev)) { priv =netdev_priv (dev);list_add_tail (&priv->list, dev_list); + count++; } } +if (!count) { +kfree (dev_list); +return ; + } +ib_set_client_data (device, &ipoib_client, dev_list); }
@@ -1690,9 +1694,6 @@ static void ipoib_remove_one(struct ib_device *device)struct ipoib_dev_priv *priv, *tmp;struct list_head *dev_list; -if (rdma_node_get_transport (device->node_type) !=RDMA_TRANSPORT_IB ) -return ; - dev_list =ib_get_client_data (device, &ipoib_client);if (!dev_list)return ;
Signature · 5 lines
2.1.0 -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
PATCH v4 8/27 IB/Verbs: Reform IB-ulp xprtrdma
In reply to Michael WangUse raw management helpers to reform IB-ulp xprtrdma.
--- net/sunrpc/xprtrdma/svc_rdma_recvfrom.c | 3 +-- net/sunrpc/xprtrdma/svc_rdma_transport.c | 45 +++++++++++++------------------- 2 files changed, 19 insertions(+), 29 deletions(-)
net/sunrpc/xprtrdma/svc_rdma_recvfrom.c +1 -2
@@ -117,8 +117,7 @@ static void rdma_build_arg_xdr(struct svc_rqst *rqstp,static int rdma_read_max_sge (struct svcxprt_rdma *xprt,int sge_count) { -if (rdma_node_get_transport (xprt->sc_cm_id->device->node_type) == -RDMA_TRANSPORT_IWARP ) +if (rdma_tech_iwarp (xprt->sc_cm_id->device, xprt->sc_cm_id->port_num))return 1 ;else return min_t (int, sge_count, xprt->sc_max_sge);
net/sunrpc/xprtrdma/svc_rdma_transport.c +18 -27
60 lines not on this page · raw message
Signature · 1 line
2.1.0
PATCH v4 9/27 IB/Verbs: Reform IB-core verbs/uverbs_cmd/sysfs
In reply to Michael WangUse raw management helpers to reform IB-core verbs/uverbs_cmd/sysfs.
--- drivers/infiniband/core/sysfs.c | 8 ++------ drivers/infiniband/core/uverbs_cmd.c | 6 ++++-- drivers/infiniband/core/verbs.c | 6 ++---- 3 files changed, 8 insertions(+), 12 deletions(-)
drivers/infiniband/core/sysfs.c +2 -6
@@ -248,14 +248,10 @@ static ssize_t phys_state_show(struct ib_port *p, struct port_attribute *unused,static ssize_t link_layer_show (struct ib_port *p,struct port_attribute *unused,char *buf) { -switch (rdma_port_get_link_layer (p->ibdev, p->port_num)) { -case IB_LINK_LAYER_INFINIBAND : +if (rdma_tech_ib (p->ibdev, p->port_num))return sprintf (buf,"%s\n" ,"InfiniBand" ); -case IB_LINK_LAYER_ETHERNET : +else return sprintf (buf,"%s\n" ,"Ethernet" ); -default : -return sprintf (buf,"%s\n" ,"Unknown" ); - } }static PORT_ATTR_RO (state);
drivers/infiniband/core/uverbs_cmd.c +4 -2
@@ -515,8 +515,10 @@ ssize_t ib_uverbs_query_port(struct ib_uverbs_file *file, resp.active_width = attr.active_width; resp.active_speed = attr.active_speed; resp.phys_state = attr.phys_state; - resp.link_layer =rdma_port_get_link_layer (file->device->ib_dev, - cmd.port_num); + resp.link_layer =rdma_tech_ib (file->device->ib_dev, + cmd.port_num) ? +IB_LINK_LAYER_INFINIBAND : +IB_LINK_LAYER_ETHERNET ;if (copy_to_user ((void __user *) (unsigned long ) cmd.response, &resp,sizeof resp))
drivers/infiniband/core/verbs.c +2 -4
@@ -200,11 +200,9 @@ int ib_init_ah_from_wc(struct ib_device *device, u8 port_num, struct ib_wc *wc,u32 flow_class;u16 gid_index;int ret; -int is_eth = (rdma_port_get_link_layer (device, port_num) == -IB_LINK_LAYER_ETHERNET );memset (ah_attr,0 ,sizeof *ah_attr); -if (is_eth) { +if (rdma_tech_iboe (device, port_num)) {if (!(wc->wc_flags &IB_WC_GRH ))return -EPROTOTYPE ;
@@ -873,7 +871,7 @@ int ib_resolve_eth_l2_attrs(struct ib_qp *qp,union ib_gid sgid;if ((*qp_attr_mask &IB_QP_AV ) && - (rdma_port_get_link_layer (qp->device, qp_attr->ah_attr.port_num) ==IB_LINK_LAYER_ETHERNET )) { + (rdma_tech_iboe (qp->device, qp_attr->ah_attr.port_num))) { ret =ib_query_gid (qp->device, qp_attr->ah_attr.port_num, qp_attr->ah_attr.grh.sgid_index, &sgid);if (ret)
Signature · 1 line
2.1.0
PATCH v4 10/27 IB/Verbs: Reform cm related part in IB-core cma/ucm
In reply to Michael WangUse raw management helpers to reform cm related part in IB-core cma/ucm.
These checks focus on the device cm type rather than the port capability, directly pass port 1 works currently, but can't support mixing cm type device in future.
--- drivers/infiniband/core/cma.c | 81 +++++++++++++------------------------------ drivers/infiniband/core/ucm.c | 3 +- 2 files changed, 26 insertions(+), 58 deletions(-)
drivers/infiniband/core/cma.c +25 -56
@@ -735,8 +735,7 @@ int rdma_init_qp_attr(struct rdma_cm_id *id, struct ib_qp_attr *qp_attr,int ret =0 ; id_priv = container_of(id, struct rdma_id_private, id); -switch (rdma_node_get_transport (id_priv->id.device->node_type)) { -case RDMA_TRANSPORT_IB : +if (rdma_ib_or_iboe (id_priv->id.device,1 )) {if (!id_priv->cm_id.ib || (id_priv->id.qp_type ==IB_QPT_UD )) ret =cma_ib_init_qp_attr (id_priv, qp_attr, qp_attr_mask);else
@@ -745,19 +744,15 @@ int rdma_init_qp_attr(struct rdma_cm_id *id, struct ib_qp_attr *qp_attr,if (qp_attr->qp_state ==IB_QPS_RTR ) qp_attr->rq_psn = id_priv->seq_num; -break ; -case RDMA_TRANSPORT_IWARP : + }else if (rdma_tech_iwarp (id_priv->id.device,1 )) {if (!id_priv->cm_id.iw) { qp_attr->qp_access_flags =0 ; *qp_attr_mask =IB_QP_STATE |IB_QP_ACCESS_FLAGS ; }else ret =iw_cm_init_qp_attr (id_priv->cm_id.iw, qp_attr, qp_attr_mask); -break ; -default : + }else ret = -ENOSYS ; -break ; - }return ret; }
@@ -1037,17 +1032,12 @@ void rdma_destroy_id(struct rdma_cm_id *id)mutex_unlock (&id_priv->handler_mutex);if (id_priv->cma_dev) { -switch (rdma_node_get_transport (id_priv->id.device->node_type)) { -case RDMA_TRANSPORT_IB : +if (rdma_ib_or_iboe (id_priv->id.device,1 )) {if (id_priv->cm_id.ib)ib_destroy_cm_id (id_priv->cm_id.ib); -break ; -case RDMA_TRANSPORT_IWARP : + }else if (rdma_tech_iwarp (id_priv->id.device,1 )) {if (id_priv->cm_id.iw)iw_destroy_cm_id (id_priv->cm_id.iw); -break ; -default : -break ; }cma_leave_mc_groups (id_priv);cma_release_dev (id_priv);
@@ -1626,7 +1616,7 @@ static void cma_listen_on_dev(struct rdma_id_private *id_priv,int ret;if (cma_family (id_priv) ==AF_IB && -rdma_node_get_transport (cma_dev->device->node_type) !=RDMA_TRANSPORT_IB ) + !rdma_ib_or_iboe (cma_dev->device,1 ))return ; id = rdma_create_id(cma_listen_handler, id_priv, id_priv->id.ps,
@@ -2028,7 +2018,7 @@ static int cma_bind_loopback(struct rdma_id_private *id_priv)mutex_lock (&lock);list_for_each_entry (cur_dev, &dev_list, list) {if (cma_family (id_priv) ==AF_IB && -rdma_node_get_transport (cur_dev->device->node_type) !=RDMA_TRANSPORT_IB ) + !rdma_ib_or_iboe (cur_dev->device,1 ))continue ;if (!cma_dev)
@@ -2060,7 +2050,7 @@ port_found: goto out; id_priv->id.route.addr.dev_addr.dev_type = - (rdma_port_get_link_layer (cma_dev->device, p) ==IB_LINK_LAYER_INFINIBAND ) ? + (rdma_tech_ib (cma_dev->device, p)) ?ARPHRD_INFINIBAND :ARPHRD_ETHER ;rdma_addr_set_sgid (&id_priv->id.route.addr.dev_addr, &gid);
@@ -2537,18 +2527,15 @@ int rdma_listen(struct rdma_cm_id *id, int backlog) id_priv->backlog = backlog;if (id->device) { -switch (rdma_node_get_transport (id->device->node_type)) { -case RDMA_TRANSPORT_IB : +if (rdma_ib_or_iboe (id->device,1 )) { ret =cma_ib_listen (id_priv);if (ret) goto err; -break ; -case RDMA_TRANSPORT_IWARP : + }else if (rdma_tech_iwarp (id->device,1 )) { ret =cma_iw_listen (id_priv, backlog);if (ret) goto err; -break ; -default : + }else { ret = -ENOSYS ; goto err; }
@@ -2884,20 +2871,15 @@ int rdma_connect(struct rdma_cm_id *id, struct rdma_conn_param *conn_param) id_priv->srq = conn_param->srq; } -switch (rdma_node_get_transport (id->device->node_type)) { -case RDMA_TRANSPORT_IB : +if (rdma_ib_or_iboe (id->device,1 )) {if (id->qp_type ==IB_QPT_UD ) ret =cma_resolve_ib_udp (id_priv, conn_param);else ret =cma_connect_ib (id_priv, conn_param); -break ; -case RDMA_TRANSPORT_IWARP : + }else if (rdma_tech_iwarp (id->device,1 )) ret =cma_connect_iw (id_priv, conn_param); -break ; -default : +else ret = -ENOSYS ; -break ; - }if (ret) goto err;
@@ -3000,8 +2982,7 @@ int rdma_accept(struct rdma_cm_id *id, struct rdma_conn_param *conn_param) id_priv->srq = conn_param->srq; } -switch (rdma_node_get_transport (id->device->node_type)) { -case RDMA_TRANSPORT_IB : +if (rdma_ib_or_iboe (id->device,1 )) {if (id->qp_type ==IB_QPT_UD ) {if (conn_param) ret = cma_send_sidr_rep(id_priv, IB_SIDR_SUCCESS,
@@ -3017,14 +2998,10 @@ int rdma_accept(struct rdma_cm_id *id, struct rdma_conn_param *conn_param)else ret =cma_rep_recv (id_priv); } -break ; -case RDMA_TRANSPORT_IWARP : + }else if (rdma_tech_iwarp (id->device,1 )) ret =cma_accept_iw (id_priv, conn_param); -break ; -default : +else ret = -ENOSYS ; -break ; - }if (ret) goto reject;
@@ -3068,8 +3045,7 @@ int rdma_reject(struct rdma_cm_id *id, const void *private_data,if (!id_priv->cm_id.ib)return -EINVAL ; -switch (rdma_node_get_transport (id->device->node_type)) { -case RDMA_TRANSPORT_IB : +if (rdma_ib_or_iboe (id->device,1 )) {if (id->qp_type ==IB_QPT_UD ) ret =cma_send_sidr_rep (id_priv,IB_SIDR_REJECT ,0 , private_data, private_data_len);
@@ -3077,15 +3053,12 @@ int rdma_reject(struct rdma_cm_id *id, const void *private_data, ret =ib_send_cm_rej (id_priv->cm_id.ib,IB_CM_REJ_CONSUMER_DEFINED ,NULL ,0 , private_data, private_data_len); -break ; -case RDMA_TRANSPORT_IWARP : + }else if (rdma_tech_iwarp (id->device,1 )) { ret =iw_cm_reject (id_priv->cm_id.iw, private_data, private_data_len); -break ; -default : + }else ret = -ENOSYS ; -break ; - } +return ret; }EXPORT_SYMBOL (rdma_reject);
@@ -3099,22 +3072,18 @@ int rdma_disconnect(struct rdma_cm_id *id)if (!id_priv->cm_id.ib)return -EINVAL ; -switch (rdma_node_get_transport (id->device->node_type)) { -case RDMA_TRANSPORT_IB : +if (rdma_ib_or_iboe (id->device,1 )) { ret =cma_modify_qp_err (id_priv);if (ret) goto out;/* Initiate or respond to a disconnect. */ if (ib_send_cm_dreq (id_priv->cm_id.ib,NULL ,0 ))ib_send_cm_drep (id_priv->cm_id.ib,NULL ,0 ); -break ; -case RDMA_TRANSPORT_IWARP : + }else if (rdma_tech_iwarp (id->device,1 )) { ret =iw_cm_disconnect (id_priv->cm_id.iw,0 ); -break ; -default : + }else ret = -EINVAL ; -break ; - } + out:return ret; }
drivers/infiniband/core/ucm.c +1 -2
@@ -1253,8 +1253,7 @@ static void ib_ucm_add_one(struct ib_device *device)dev_t base;struct ib_ucm_device *ucm_dev; -if (!device->alloc_ucontext || -rdma_node_get_transport (device->node_type) !=RDMA_TRANSPORT_IB ) +if (!device->alloc_ucontext || !rdma_ib_or_iboe (device,1 ))return ; ucm_dev =kzalloc (sizeof *ucm_dev,GFP_KERNEL );
Signature · 5 lines
2.1.0 -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
PATCH v4 11/27 IB/Verbs: Reform route related part in IB-core cma
In reply to Michael WangUse raw management helpers to reform route related part in IB-core cma.
--- drivers/infiniband/core/cma.c | 31 ++++++++----------------------- drivers/infiniband/core/ucma.c | 25 ++++++------------------- 2 files changed, 14 insertions(+), 42 deletions(-)
drivers/infiniband/core/cma.c +8 -23
@@ -923,13 +923,9 @@ static inline int cma_user_data_offset(struct rdma_id_private *id_priv)static void cma_cancel_route (struct rdma_id_private *id_priv) { -switch (rdma_port_get_link_layer (id_priv->id.device, id_priv->id.port_num)) { -case IB_LINK_LAYER_INFINIBAND : +if (rdma_tech_ib (id_priv->id.device, id_priv->id.port_num)) {if (id_priv->query)ib_sa_cancel_query (id_priv->query_id, id_priv->query); -break ; -default : -break ; } }
@@ -1957,26 +1953,15 @@ int rdma_resolve_route(struct rdma_cm_id *id, int timeout_ms)return -EINVAL ;atomic_inc (&id_priv->refcount); -switch (rdma_node_get_transport (id->device->node_type)) { -case RDMA_TRANSPORT_IB : -switch (rdma_port_get_link_layer (id->device, id->port_num)) { -case IB_LINK_LAYER_INFINIBAND : - ret =cma_resolve_ib_route (id_priv, timeout_ms); -break ; -case IB_LINK_LAYER_ETHERNET : - ret =cma_resolve_iboe_route (id_priv); -break ; -default : - ret = -ENOSYS ; - } -break ; -case RDMA_TRANSPORT_IWARP : +if (rdma_tech_ib (id->device, id->port_num)) + ret =cma_resolve_ib_route (id_priv, timeout_ms); +else if (rdma_tech_iboe (id->device, id->port_num)) + ret =cma_resolve_iboe_route (id_priv); +else if (rdma_tech_iwarp (id->device, id->port_num)) ret =cma_resolve_iw_route (id_priv, timeout_ms); -break ; -default : +else ret = -ENOSYS ; -break ; - } +if (ret) goto err;
drivers/infiniband/core/ucma.c +6 -19
@@ -722,26 +722,13 @@ static ssize_t ucma_query_route(struct ucma_file *file, resp.node_guid = (__force __u64 ) ctx->cm_id->device->node_guid; resp.port_num = ctx->cm_id->port_num; -switch (rdma_node_get_transport (ctx->cm_id->device->node_type)) { -case RDMA_TRANSPORT_IB : -switch (rdma_port_get_link_layer (ctx->cm_id->device, - ctx->cm_id->port_num)) { -case IB_LINK_LAYER_INFINIBAND : -ucma_copy_ib_route (&resp, &ctx->cm_id->route); -break ; -case IB_LINK_LAYER_ETHERNET : -ucma_copy_iboe_route (&resp, &ctx->cm_id->route); -break ; -default : -break ; - } -break ; -case RDMA_TRANSPORT_IWARP : + +if (rdma_tech_ib (ctx->cm_id->device, ctx->cm_id->port_num)) +ucma_copy_ib_route (&resp, &ctx->cm_id->route); +else if (rdma_tech_iboe (ctx->cm_id->device, ctx->cm_id->port_num)) +ucma_copy_iboe_route (&resp, &ctx->cm_id->route); +else if (rdma_tech_iwarp (ctx->cm_id->device, ctx->cm_id->port_num))ucma_copy_iw_route (&resp, &ctx->cm_id->route); -break ; -default : -break ; - } out: if (copy_to_user((void __user *)(unsigned long)cmd.response,
Signature · 1 line
2.1.0
PATCH v4 12/27 IB/Verbs: Reform mcast related part in IB-core cma
In reply to Michael WangUse raw management helpers to reform mcast related part in IB-core cma.
--- drivers/infiniband/core/cma.c | 57 +++++++++++++++---------------------------- 1 file changed, 19 insertions(+), 38 deletions(-)
drivers/infiniband/core/cma.c +19 -38
@@ -997,17 +997,12 @@ static void cma_leave_mc_groups(struct rdma_id_private *id_priv) mc =container_of (id_priv->mc_list.next, struct cma_multicast, list);list_del (&mc->list); -switch (rdma_port_get_link_layer (id_priv->cma_dev->device, id_priv->id.port_num)) { -case IB_LINK_LAYER_INFINIBAND : +if (rdma_tech_ib (id_priv->cma_dev->device, + id_priv->id.port_num)) {ib_sa_free_multicast (mc->multicast.ib);kfree (mc); -break ; -case IB_LINK_LAYER_ETHERNET : + }else kref_put (&mc->mcref, release_mc); -break ; -default : -break ; - } } }
@@ -3314,24 +3309,13 @@ int rdma_join_multicast(struct rdma_cm_id *id, struct sockaddr *addr,list_add (&mc->list, &id_priv->mc_list);spin_unlock (&id_priv->lock); -switch (rdma_node_get_transport (id->device->node_type)) { -case RDMA_TRANSPORT_IB : -switch (rdma_port_get_link_layer (id->device, id->port_num)) { -case IB_LINK_LAYER_INFINIBAND : - ret =cma_join_ib_multicast (id_priv, mc); -break ; -case IB_LINK_LAYER_ETHERNET : -kref_init (&mc->mcref); - ret =cma_iboe_join_multicast (id_priv, mc); -break ; -default : - ret = -EINVAL ; - } -break ; -default : +if (rdma_tech_iboe (id->device, id->port_num)) { +kref_init (&mc->mcref); + ret =cma_iboe_join_multicast (id_priv, mc); + }else if (rdma_tech_ib (id->device, id->port_num)) + ret =cma_join_ib_multicast (id_priv, mc); +else ret = -ENOSYS ; -break ; - }if (ret) {spin_lock_irq (&id_priv->lock);
@@ -3359,19 +3343,16 @@ void rdma_leave_multicast(struct rdma_cm_id *id, struct sockaddr *addr)ib_detach_mcast (id->qp, &mc->multicast.ib->rec.mgid,be16_to_cpu (mc->multicast.ib->rec.mlid)); -if (rdma_node_get_transport (id_priv->cma_dev->device->node_type) ==RDMA_TRANSPORT_IB ) { -switch (rdma_port_get_link_layer (id->device, id->port_num)) { -case IB_LINK_LAYER_INFINIBAND : -ib_sa_free_multicast (mc->multicast.ib); -kfree (mc); -break ; -case IB_LINK_LAYER_ETHERNET : -kref_put (&mc->mcref, release_mc); -break ; -default : -break ; - } - } + +BUG_ON (id_priv->cma_dev->device != id->device); + +if (rdma_tech_ib (id->device, id->port_num)) { +ib_sa_free_multicast (mc->multicast.ib); +kfree (mc); + }else if (rdma_tech_iboe (id->device, + id->port_num)) +kref_put (&mc->mcref, release_mc); +return ; } }
Signature · 5 lines
2.1.0 -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
PATCH v4 13/27 IB/Verbs: Reserve legacy transport type in 'dev_addr'
In reply to Michael WangReserve the legacy transport type for the 'transport' member of 'struct rdma_dev_addr' until we make sure this is no longer needed.
--- drivers/infiniband/core/cma.c | 25 +++++++++++++++++++++++-- 1 file changed, 23 insertions(+), 2 deletions(-)
drivers/infiniband/core/cma.c +23 -2
@@ -244,14 +244,35 @@ static inline void cma_set_ip_ver(struct cma_hdr *hdr, u8 ip_ver) hdr->ip_version = (ip_ver <<4 ) | (hdr->ip_version &0xF ); } +static inline void cma_set_legacy_transport (struct rdma_cm_id *id) +{ +switch (id->device->node_type) { +case RDMA_NODE_IB_CA : +case RDMA_NODE_IB_SWITCH : +case RDMA_NODE_IB_ROUTER : + id->route.addr.dev_addr.transport =RDMA_TRANSPORT_IB ; +break ; +case RDMA_NODE_RNIC : + id->route.addr.dev_addr.transport =RDMA_TRANSPORT_IWARP ; +break ; +case RDMA_NODE_USNIC : + id->route.addr.dev_addr.transport =RDMA_TRANSPORT_USNIC ; +break ; +case RDMA_NODE_USNIC_UDP : + id->route.addr.dev_addr.transport =RDMA_TRANSPORT_USNIC_UDP ; +break ; +default : +BUG (); + } +} +static void cma_attach_to_dev (struct rdma_id_private *id_priv,struct cma_device *cma_dev) {atomic_inc (&cma_dev->refcount); id_priv->cma_dev = cma_dev; id_priv->id.device = cma_dev->device; - id_priv->id.route.addr.dev_addr.transport = -rdma_node_get_transport (cma_dev->device->node_type); +cma_set_legacy_transport (&id_priv->id);list_add_tail (&id_priv->list, &cma_dev->id_list); }
Signature · 1 line
2.1.0
PATCH v4 14/27 IB/Verbs: Reform cma_acquire_dev()
In reply to Michael WangReform cma_acquire_dev() with management helpers, introduce cma_validate_port() to make the code more clean.
--- drivers/infiniband/core/cma.c | 69 +++++++++++++++++++++++++------------------ 1 file changed, 41 insertions(+), 28 deletions(-)
drivers/infiniband/core/cma.c +41 -28
@@ -370,18 +370,36 @@ static int cma_translate_addr(struct sockaddr *addr, struct rdma_dev_addr *dev_areturn ret; } +static inline int cma_validate_port (struct ib_device *device,u8 port, +union ib_gid *gid,int dev_type) +{ +u8 found_port; +int ret = -ENODEV ; + +if ((dev_type ==ARPHRD_INFINIBAND ) && !rdma_tech_ib (device, port)) +return ret; + +if ((dev_type !=ARPHRD_INFINIBAND ) &&rdma_tech_ib (device, port)) +return ret; + + ret =ib_find_cached_gid (device, gid, &found_port,NULL ); + +if (!ret && (port == found_port)) +return 0 ; + +return ret; +} +static int cma_acquire_dev (struct rdma_id_private *id_priv,struct rdma_id_private *listen_id_priv) {struct rdma_dev_addr *dev_addr = &id_priv->id.route.addr.dev_addr;struct cma_device *cma_dev; -union ib_gid gid, iboe_gid; +union ib_gid gid, iboe_gid, *gidp;int ret = -ENODEV ; -u8 port, found_port; -enum rdma_link_layer dev_ll = dev_addr->dev_type ==ARPHRD_INFINIBAND ? -IB_LINK_LAYER_INFINIBAND :IB_LINK_LAYER_ETHERNET ; +u8 port; -if (dev_ll !=IB_LINK_LAYER_INFINIBAND && +if (dev_addr->dev_type !=ARPHRD_INFINIBAND && id_priv->id.ps ==RDMA_PS_IPOIB )return -EINVAL ;
@@ -391,41 +409,36 @@ static int cma_acquire_dev(struct rdma_id_private *id_priv,memcpy (&gid, dev_addr->src_dev_addr +rdma_addr_gid_offset (dev_addr),sizeof gid); -if (listen_id_priv && -rdma_port_get_link_layer (listen_id_priv->id.device, - listen_id_priv->id.port_num) == dev_ll) { + +if (listen_id_priv) { cma_dev = listen_id_priv->cma_dev; port = listen_id_priv->id.port_num; -if (rdma_node_get_transport (cma_dev->device->node_type) ==RDMA_TRANSPORT_IB && -rdma_port_get_link_layer (cma_dev->device, port) ==IB_LINK_LAYER_ETHERNET ) - ret =ib_find_cached_gid (cma_dev->device, &iboe_gid, - &found_port,NULL ); -else - ret =ib_find_cached_gid (cma_dev->device, &gid, - &found_port,NULL ); + gidp =rdma_tech_iboe (cma_dev->device, port) ? + &iboe_gid : &gid; -if (!ret && (port == found_port)) { - id_priv->id.port_num = found_port; + ret =cma_validate_port (cma_dev->device, port, gidp, + dev_addr->dev_type); +if (!ret) { + id_priv->id.port_num = port; goto out; } } +list_for_each_entry (cma_dev, &dev_list, list) {for (port =1 ; port <= cma_dev->device->phys_port_cnt; ++port) {if (listen_id_priv && listen_id_priv->cma_dev == cma_dev && listen_id_priv->id.port_num == port)continue ; -if (rdma_port_get_link_layer (cma_dev->device, port) == dev_ll) { -if (rdma_node_get_transport (cma_dev->device->node_type) ==RDMA_TRANSPORT_IB && -rdma_port_get_link_layer (cma_dev->device, port) ==IB_LINK_LAYER_ETHERNET ) - ret =ib_find_cached_gid (cma_dev->device, &iboe_gid, &found_port,NULL ); -else - ret =ib_find_cached_gid (cma_dev->device, &gid, &found_port,NULL ); - -if (!ret && (port == found_port)) { - id_priv->id.port_num = found_port; - goto out; - } + + gidp =rdma_tech_iboe (cma_dev->device, port) ? + &iboe_gid : &gid; + + ret =cma_validate_port (cma_dev->device, port, gidp, + dev_addr->dev_type); +if (!ret) { + id_priv->id.port_num = port; + goto out; } } }
Signature · 1 line
2.1.0
PATCH v4 15/27 IB/Verbs: Reform rest part in IB-core cma
In reply to Michael WangUse raw management helpers to reform rest part in IB-core cma.
--- drivers/infiniband/core/cma.c | 16 +++++++--------- 1 file changed, 7 insertions(+), 9 deletions(-)
drivers/infiniband/core/cma.c +7 -9
@@ -469,10 +469,10 @@ static int cma_resolve_ib_dev(struct rdma_id_private *id_priv) pkey =ntohs (addr->sib_pkey);list_for_each_entry (cur_dev, &dev_list, list) { -if (rdma_node_get_transport (cur_dev->device->node_type) !=RDMA_TRANSPORT_IB ) -continue ; -for (p =1 ; p <= cur_dev->device->phys_port_cnt; ++p) { +if (!rdma_ib_or_iboe (cur_dev->device, p)) +continue ; +if (ib_find_cached_pkey (cur_dev->device, p, pkey, &index))continue ;
@@ -667,10 +667,9 @@ static int cma_modify_qp_rtr(struct rdma_id_private *id_priv,if (ret) goto out; -if (rdma_node_get_transport (id_priv->cma_dev->device->node_type) - ==RDMA_TRANSPORT_IB && -rdma_port_get_link_layer (id_priv->id.device, id_priv->id.port_num) - ==IB_LINK_LAYER_ETHERNET ) { +BUG_ON (id_priv->cma_dev->device != id_priv->id.device); + +if (rdma_tech_iboe (id_priv->id.device, id_priv->id.port_num)) { ret =rdma_addr_find_smac_by_sgid (&sgid, qp_attr.smac,NULL );if (ret)
@@ -734,8 +733,7 @@ static int cma_ib_init_qp_attr(struct rdma_id_private *id_priv,int ret;u16 pkey; -if (rdma_port_get_link_layer (id_priv->id.device, id_priv->id.port_num) == -IB_LINK_LAYER_INFINIBAND ) +if (rdma_tech_ib (id_priv->id.device, id_priv->id.port_num)) pkey =ib_addr_get_pkey (dev_addr);else pkey =0xffff ;
Signature · 1 line
2.1.0
PATCH v4 16/27 IB/Verbs: Use management helper cap_ib_mad()
In reply to Michael WangIntroduce helper cap_ib_mad() to help us check if the port of an IB device support Infiniband Management Datagrams.
--- drivers/infiniband/core/mad.c | 6 +++--- drivers/infiniband/core/user_mad.c | 6 +++--- include/rdma/ib_verbs.h | 15 +++++++++++++++ 3 files changed, 21 insertions(+), 6 deletions(-)
drivers/infiniband/core/mad.c +3 -3
@@ -3066,7 +3066,7 @@ static void ib_mad_init_device(struct ib_device *device) }for (i = start; i <= end; i++) { -if (!rdma_ib_or_iboe (device, i)) +if (!cap_ib_mad (device, i))continue ;if (ib_mad_port_open (device, i)) {
@@ -3087,7 +3087,7 @@ error_agent: error:while (--i >= start) { -if (!rdma_ib_or_iboe (device, i)) +if (!cap_ib_mad (device, i))continue ;if (ib_agent_port_close (device, i))
@@ -3110,7 +3110,7 @@ static void ib_mad_remove_device(struct ib_device *device) cur_port =1 ; }for (i =0 ; i < num_ports; i++, cur_port++) { -if (!rdma_ib_or_iboe (device, i)) +if (!cap_ib_mad (device, i))continue ;if (ib_agent_port_close (device, cur_port))
drivers/infiniband/core/user_mad.c +3 -3
@@ -1294,7 +1294,7 @@ static void ib_umad_add_one(struct ib_device *device) umad_dev->end_port = e;for (i = s; i <= e; ++i) { -if (!rdma_ib_or_iboe (device, i)) +if (!cap_ib_mad (device, i))continue ; umad_dev->port[i - s].umad_dev = umad_dev;
@@ -1317,7 +1317,7 @@ static void ib_umad_add_one(struct ib_device *device) err:while (--i >= s) { -if (!rdma_ib_or_iboe (device, i)) +if (!cap_ib_mad (device, i))continue ;ib_umad_kill_port (&umad_dev->port[i - s]);
@@ -1335,7 +1335,7 @@ static void ib_umad_remove_one(struct ib_device *device)return ;for (i =0 ; i <= umad_dev->end_port - umad_dev->start_port; ++i) { -if (rdma_ib_or_iboe (device, i)) +if (cap_ib_mad (device, i))ib_umad_kill_port (&umad_dev->port[i]); }
include/rdma/ib_verbs.h +15 -0
@@ -1773,6 +1773,21 @@ static inline int rdma_ib_or_iboe(struct ib_device *device, u8 port_num)return (tp ==RDMA_TRANSPORT_IB || tp ==RDMA_TRANSPORT_IBOE ); } +/** +* cap_ib_mad - Check if the port of device has the capability Infiniband +* Management Datagrams. +* +* @device: Device to be checked +* @port_num: Port number of the device +* +* Return 0 when port of the device don't support Infiniband +* Management Datagrams. +*/ +static inline int cap_ib_mad (struct ib_device *device,u8 port_num) +{ +return rdma_ib_or_iboe (device, port_num); +} +int ib_query_gid (struct ib_device *device,u8 port_num,int index,union ib_gid *gid);
Signature · 5 lines
2.1.0 -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
PATCH v4 17/27 IB/Verbs: Use management helper cap_ib_smi()
In reply to Michael WangIntroduce helper cap_ib_smi() to help us check if the port of an IB device support Infiniband Subnet Management Interface.
--- drivers/infiniband/core/agent.c | 2 +- drivers/infiniband/core/mad.c | 2 +- include/rdma/ib_verbs.h | 15 +++++++++++++++ 3 files changed, 17 insertions(+), 2 deletions(-)
drivers/infiniband/core/agent.c +1 -1
@@ -156,7 +156,7 @@ int ib_agent_port_open(struct ib_device *device, int port_num) goto error1; } -if (rdma_tech_ib (device, port_num)) { +if (cap_ib_smi (device, port_num)) {/* Obtain send only MAD agent for SMI QP */ port_priv->agent[0] = ib_register_mad_agent(device, port_num, IB_QPT_SMI, NULL, 0,
drivers/infiniband/core/mad.c +1 -1
@@ -2938,7 +2938,7 @@ static int ib_mad_port_open(struct ib_device *device,init_mad_qp (port_priv, &port_priv->qp_info[1 ]); cq_size = mad_sendq_size + mad_recvq_size; - has_smi =rdma_tech_ib (device, port_num); + has_smi =cap_ib_smi (device, port_num);if (has_smi) cq_size *=2 ;
include/rdma/ib_verbs.h +15 -0
@@ -1788,6 +1788,21 @@ static inline int cap_ib_mad(struct ib_device *device, u8 port_num)return rdma_ib_or_iboe (device, port_num); } +/** +* cap_ib_smi - Check if the port of device has the capability Infiniband +* Subnet Management Interface. +* +* @device: Device to be checked +* @port_num: Port number of the device +* +* Return 0 when port of the device don't support Infiniband +* Subnet Management Interface. +*/ +static inline int cap_ib_smi (struct ib_device *device,u8 port_num) +{ +return rdma_tech_ib (device, port_num); +} +int ib_query_gid (struct ib_device *device,u8 port_num,int index,union ib_gid *gid);
Signature · 5 lines
2.1.0 -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
PATCH v4 18/27 IB/Verbs: Use management helper cap_ib_cm()
In reply to Michael WangIntroduce helper cap_ib_cm() to help us check if the port of an IB device support Infiniband Communication Manager.
--- drivers/infiniband/core/cm.c | 6 +++--- drivers/infiniband/core/cma.c | 19 +++++++++---------- drivers/infiniband/core/ucm.c | 2 +- include/rdma/ib_verbs.h | 15 +++++++++++++++ 4 files changed, 28 insertions(+), 14 deletions(-)
drivers/infiniband/core/cm.c +3 -3
@@ -3781,7 +3781,7 @@ static void cm_add_one(struct ib_device *ib_device)set_bit (IB_MGMT_METHOD_SEND , reg_req.method_mask);for (i =1 ; i <= ib_device->phys_port_cnt; i++) { -if (!rdma_ib_or_iboe (ib_device, i)) +if (!cap_ib_cm (ib_device, i))continue ; port =kzalloc (sizeof *port,GFP_KERNEL );
@@ -3835,7 +3835,7 @@ error1: port_modify.set_port_cap_mask =0 ; port_modify.clr_port_cap_mask =IB_PORT_CM_SUP ;while (--i) { -if (!rdma_ib_or_iboe (ib_device, i)) +if (!cap_ib_cm (ib_device, i))continue ; port = cm_dev->port[i-1 ];
@@ -3866,7 +3866,7 @@ static void cm_remove_one(struct ib_device *ib_device)write_unlock_irqrestore (&cm.device_lock, flags);for (i =1 ; i <= ib_device->phys_port_cnt; i++) { -if (!rdma_ib_or_iboe (ib_device, i)) +if (!cap_ib_cm (ib_device, i))continue ; port = cm_dev->port[i-1 ];
drivers/infiniband/core/cma.c +9 -10
@@ -767,7 +767,7 @@ int rdma_init_qp_attr(struct rdma_cm_id *id, struct ib_qp_attr *qp_attr,int ret =0 ; id_priv = container_of(id, struct rdma_id_private, id); -if (rdma_ib_or_iboe (id_priv->id.device,1 )) { +if (cap_ib_cm (id_priv->id.device,1 )) {if (!id_priv->cm_id.ib || (id_priv->id.qp_type ==IB_QPT_UD )) ret =cma_ib_init_qp_attr (id_priv, qp_attr, qp_attr_mask);else
@@ -1055,7 +1055,7 @@ void rdma_destroy_id(struct rdma_cm_id *id)mutex_unlock (&id_priv->handler_mutex);if (id_priv->cma_dev) { -if (rdma_ib_or_iboe (id_priv->id.device,1 )) { +if (cap_ib_cm (id_priv->id.device,1 )) {if (id_priv->cm_id.ib)ib_destroy_cm_id (id_priv->cm_id.ib); }else if (rdma_tech_iwarp (id_priv->id.device,1 )) {
@@ -1638,8 +1638,7 @@ static void cma_listen_on_dev(struct rdma_id_private *id_priv,struct rdma_cm_id *id;int ret; -if (cma_family (id_priv) ==AF_IB && - !rdma_ib_or_iboe (cma_dev->device,1 )) +if (cma_family (id_priv) ==AF_IB && !cap_ib_cm (cma_dev->device,1 ))return ; id = rdma_create_id(cma_listen_handler, id_priv, id_priv->id.ps,
@@ -2030,7 +2029,7 @@ static int cma_bind_loopback(struct rdma_id_private *id_priv)mutex_lock (&lock);list_for_each_entry (cur_dev, &dev_list, list) {if (cma_family (id_priv) ==AF_IB && - !rdma_ib_or_iboe (cur_dev->device,1 )) + !cap_ib_cm (cur_dev->device,1 ))continue ;if (!cma_dev)
@@ -2539,7 +2538,7 @@ int rdma_listen(struct rdma_cm_id *id, int backlog) id_priv->backlog = backlog;if (id->device) { -if (rdma_ib_or_iboe (id->device,1 )) { +if (cap_ib_cm (id->device,1 )) { ret =cma_ib_listen (id_priv);if (ret) goto err;
@@ -2883,7 +2882,7 @@ int rdma_connect(struct rdma_cm_id *id, struct rdma_conn_param *conn_param) id_priv->srq = conn_param->srq; } -if (rdma_ib_or_iboe (id->device,1 )) { +if (cap_ib_cm (id->device,1 )) {if (id->qp_type ==IB_QPT_UD ) ret =cma_resolve_ib_udp (id_priv, conn_param);else
@@ -2994,7 +2993,7 @@ int rdma_accept(struct rdma_cm_id *id, struct rdma_conn_param *conn_param) id_priv->srq = conn_param->srq; } -if (rdma_ib_or_iboe (id->device,1 )) { +if (cap_ib_cm (id->device,1 )) {if (id->qp_type ==IB_QPT_UD ) {if (conn_param) ret = cma_send_sidr_rep(id_priv, IB_SIDR_SUCCESS,
@@ -3057,7 +3056,7 @@ int rdma_reject(struct rdma_cm_id *id, const void *private_data,if (!id_priv->cm_id.ib)return -EINVAL ; -if (rdma_ib_or_iboe (id->device,1 )) { +if (cap_ib_cm (id->device,1 )) {if (id->qp_type ==IB_QPT_UD ) ret =cma_send_sidr_rep (id_priv,IB_SIDR_REJECT ,0 , private_data, private_data_len);
@@ -3084,7 +3083,7 @@ int rdma_disconnect(struct rdma_cm_id *id)if (!id_priv->cm_id.ib)return -EINVAL ; -if (rdma_ib_or_iboe (id->device,1 )) { +if (cap_ib_cm (id->device,1 )) { ret =cma_modify_qp_err (id_priv);if (ret) goto out;
drivers/infiniband/core/ucm.c +1 -1
@@ -1253,7 +1253,7 @@ static void ib_ucm_add_one(struct ib_device *device)dev_t base;struct ib_ucm_device *ucm_dev; -if (!device->alloc_ucontext || !rdma_ib_or_iboe (device,1 )) +if (!device->alloc_ucontext || !cap_ib_cm (device,1 ))return ; ucm_dev =kzalloc (sizeof *ucm_dev,GFP_KERNEL );
include/rdma/ib_verbs.h +15 -0
@@ -1803,6 +1803,21 @@ static inline int cap_ib_smi(struct ib_device *device, u8 port_num)return rdma_tech_ib (device, port_num); } +/** +* cap_ib_cm - Check if the port of device has the capability Infiniband +* Communication Manager. +* +* @device: Device to be checked +* @port_num: Port number of the device +* +* Return 0 when port of the device don't support Infiniband +* Communication Manager. +*/ +static inline int cap_ib_cm (struct ib_device *device,u8 port_num) +{ +return rdma_ib_or_iboe (device, port_num); +} +int ib_query_gid (struct ib_device *device,u8 port_num,int index,union ib_gid *gid);
Signature · 1 line
2.1.0
PATCH v4 19/27 IB/Verbs: Use management helper cap_iw_cm()
In reply to Michael WangIntroduce helper cap_iw_cm() to help us check if the port of an IB device support IWARP Communication Manager.
--- drivers/infiniband/core/cma.c | 14 +++++++------- include/rdma/ib_verbs.h | 15 +++++++++++++++ 2 files changed, 22 insertions(+), 7 deletions(-)
drivers/infiniband/core/cma.c +7 -7
@@ -776,7 +776,7 @@ int rdma_init_qp_attr(struct rdma_cm_id *id, struct ib_qp_attr *qp_attr,if (qp_attr->qp_state ==IB_QPS_RTR ) qp_attr->rq_psn = id_priv->seq_num; - }else if (rdma_tech_iwarp (id_priv->id.device,1 )) { + }else if (cap_iw_cm (id_priv->id.device,1 )) {if (!id_priv->cm_id.iw) { qp_attr->qp_access_flags =0 ; *qp_attr_mask =IB_QP_STATE |IB_QP_ACCESS_FLAGS ;
@@ -1058,7 +1058,7 @@ void rdma_destroy_id(struct rdma_cm_id *id)if (cap_ib_cm (id_priv->id.device,1 )) {if (id_priv->cm_id.ib)ib_destroy_cm_id (id_priv->cm_id.ib); - }else if (rdma_tech_iwarp (id_priv->id.device,1 )) { + }else if (cap_iw_cm (id_priv->id.device,1 )) {if (id_priv->cm_id.iw)iw_destroy_cm_id (id_priv->cm_id.iw); }
@@ -2542,7 +2542,7 @@ int rdma_listen(struct rdma_cm_id *id, int backlog) ret =cma_ib_listen (id_priv);if (ret) goto err; - }else if (rdma_tech_iwarp (id->device,1 )) { + }else if (cap_iw_cm (id->device,1 )) { ret =cma_iw_listen (id_priv, backlog);if (ret) goto err;
@@ -2887,7 +2887,7 @@ int rdma_connect(struct rdma_cm_id *id, struct rdma_conn_param *conn_param) ret =cma_resolve_ib_udp (id_priv, conn_param); else ret =cma_connect_ib (id_priv, conn_param); - }else if (rdma_tech_iwarp (id->device,1 )) + }else if (cap_iw_cm (id->device,1 )) ret =cma_connect_iw (id_priv, conn_param);else ret = -ENOSYS ;
@@ -3009,7 +3009,7 @@ int rdma_accept(struct rdma_cm_id *id, struct rdma_conn_param *conn_param)else ret =cma_rep_recv (id_priv); } - }else if (rdma_tech_iwarp (id->device,1 )) + }else if (cap_iw_cm (id->device,1 )) ret =cma_accept_iw (id_priv, conn_param);else ret = -ENOSYS ;
@@ -3064,7 +3064,7 @@ int rdma_reject(struct rdma_cm_id *id, const void *private_data, ret =ib_send_cm_rej (id_priv->cm_id.ib,IB_CM_REJ_CONSUMER_DEFINED ,NULL ,0 , private_data, private_data_len); - }else if (rdma_tech_iwarp (id->device,1 )) { + }else if (cap_iw_cm (id->device,1 )) { ret =iw_cm_reject (id_priv->cm_id.iw, private_data, private_data_len); }else
@@ -3090,7 +3090,7 @@ int rdma_disconnect(struct rdma_cm_id *id)/* Initiate or respond to a disconnect. */ if (ib_send_cm_dreq (id_priv->cm_id.ib,NULL ,0 ))ib_send_cm_drep (id_priv->cm_id.ib,NULL ,0 ); - }else if (rdma_tech_iwarp (id->device,1 )) { + }else if (cap_iw_cm (id->device,1 )) { ret =iw_cm_disconnect (id_priv->cm_id.iw,0 ); }else ret = -EINVAL ;
include/rdma/ib_verbs.h +15 -0
@@ -1818,6 +1818,21 @@ static inline int cap_ib_cm(struct ib_device *device, u8 port_num)return rdma_ib_or_iboe (device, port_num); } +/** +* cap_iw_cm - Check if the port of device has the capability IWARP +* Communication Manager. +* +* @device: Device to be checked +* @port_num: Port number of the device +* +* Return 0 when port of the device don't support IWARP +* Communication Manager. +*/ +static inline int cap_iw_cm (struct ib_device *device,u8 port_num) +{ +return rdma_tech_iwarp (device, port_num); +} +int ib_query_gid (struct ib_device *device,u8 port_num,int index,union ib_gid *gid);
Signature · 5 lines
2.1.0 -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
PATCH v4 20/27 IB/Verbs: Use management helper cap_ib_sa()
In reply to Michael WangIntroduce helper cap_ib_sa() to help us check if the port of an IB device support Infiniband Subnet Administrator.
--- drivers/infiniband/core/cma.c | 4 ++-- drivers/infiniband/core/sa_query.c | 10 +++++----- drivers/infiniband/core/ucma.c | 2 +- include/rdma/ib_verbs.h | 15 +++++++++++++++ 4 files changed, 23 insertions(+), 8 deletions(-)
drivers/infiniband/core/cma.c +2 -2
@@ -955,7 +955,7 @@ static inline int cma_user_data_offset(struct rdma_id_private *id_priv)static void cma_cancel_route (struct rdma_id_private *id_priv) { -if (rdma_tech_ib (id_priv->id.device, id_priv->id.port_num)) { +if (cap_ib_sa (id_priv->id.device, id_priv->id.port_num)) {if (id_priv->query)ib_sa_cancel_query (id_priv->query_id, id_priv->query); }
@@ -1979,7 +1979,7 @@ int rdma_resolve_route(struct rdma_cm_id *id, int timeout_ms)return -EINVAL ;atomic_inc (&id_priv->refcount); -if (rdma_tech_ib (id->device, id->port_num)) +if (cap_ib_sa (id->device, id->port_num)) ret =cma_resolve_ib_route (id_priv, timeout_ms);else if (rdma_tech_iboe (id->device, id->port_num)) ret =cma_resolve_iboe_route (id_priv);
drivers/infiniband/core/sa_query.c +5 -5
@@ -450,7 +450,7 @@ static void ib_sa_event(struct ib_event_handler *handler, struct ib_event *eventstruct ib_sa_port *port = &sa_dev->port[event->element.port_num - sa_dev->start_port]; -if (WARN_ON (!rdma_tech_ib (handler->device, port->port_num))) +if (WARN_ON (!cap_ib_sa (handler->device, port->port_num)))return ;spin_lock_irqsave (&port->ah_lock, flags);
@@ -1173,7 +1173,7 @@ static void ib_sa_add_one(struct ib_device *device)for (i =0 ; i <= e - s; ++i) {spin_lock_init (&sa_dev->port[i].ah_lock); -if (!rdma_tech_ib (device, i +1 )) +if (!cap_ib_sa (device, i +1 ))continue ; sa_dev->port[i].sm_ah =NULL ;
@@ -1210,7 +1210,7 @@ static void ib_sa_add_one(struct ib_device *device) goto err;for (i =0 ; i <= e - s; ++i) { -if (rdma_tech_ib (device, i +1 )) +if (cap_ib_sa (device, i +1 ))update_sm_ah (&sa_dev->port[i].update_task); }
@@ -1218,7 +1218,7 @@ static void ib_sa_add_one(struct ib_device *device) err:while (--i >=0 ) { -if (rdma_tech_ib (device, i +1 )) +if (cap_ib_sa (device, i +1 ))ib_unregister_mad_agent (sa_dev->port[i].agent); }
@@ -1240,7 +1240,7 @@ static void ib_sa_remove_one(struct ib_device *device)flush_workqueue (ib_wq);for (i =0 ; i <= sa_dev->end_port - sa_dev->start_port; ++i) { -if (rdma_tech_ib (device, i +1 )) { +if (cap_ib_sa (device, i +1 )) {ib_unregister_mad_agent (sa_dev->port[i].agent);if (sa_dev->port[i].sm_ah)kref_put (&sa_dev->port[i].sm_ah->ref, free_sm_ah);
drivers/infiniband/core/ucma.c +1 -1
@@ -723,7 +723,7 @@ static ssize_t ucma_query_route(struct ucma_file *file, resp.node_guid = (__force __u64 ) ctx->cm_id->device->node_guid; resp.port_num = ctx->cm_id->port_num; -if (rdma_tech_ib (ctx->cm_id->device, ctx->cm_id->port_num)) +if (cap_ib_sa (ctx->cm_id->device, ctx->cm_id->port_num))ucma_copy_ib_route (&resp, &ctx->cm_id->route);else if (rdma_tech_iboe (ctx->cm_id->device, ctx->cm_id->port_num))ucma_copy_iboe_route (&resp, &ctx->cm_id->route);
include/rdma/ib_verbs.h +15 -0
@@ -1833,6 +1833,21 @@ static inline int cap_iw_cm(struct ib_device *device, u8 port_num)return rdma_tech_iwarp (device, port_num); } +/** +* cap_ib_sa - Check if the port of device has the capability Infiniband +* Subnet Administrator. +* +* @device: Device to be checked +* @port_num: Port number of the device +* +* Return 0 when port of the device don't support Infiniband +* Subnet Administrator. +*/ +static inline int cap_ib_sa (struct ib_device *device,u8 port_num) +{ +return rdma_tech_ib (device, port_num); +} +int ib_query_gid (struct ib_device *device,u8 port_num,int index,union ib_gid *gid);
Signature · 1 line
2.1.0
PATCH v4 21/27 IB/Verbs: Use management helper cap_ib_mcast()
In reply to Michael WangIntroduce helper cap_ib_mcast() to help us check if the port of an IB device support Infiniband Multicast.
--- drivers/infiniband/core/cma.c | 6 +++--- drivers/infiniband/core/multicast.c | 6 +++--- include/rdma/ib_verbs.h | 15 +++++++++++++++ 3 files changed, 21 insertions(+), 6 deletions(-)
drivers/infiniband/core/cma.c +3 -3
@@ -1029,7 +1029,7 @@ static void cma_leave_mc_groups(struct rdma_id_private *id_priv) mc =container_of (id_priv->mc_list.next, struct cma_multicast, list);list_del (&mc->list); -if (rdma_tech_ib (id_priv->cma_dev->device, +if (cap_ib_mcast (id_priv->cma_dev->device, id_priv->id.port_num)) {ib_sa_free_multicast (mc->multicast.ib);kfree (mc);
@@ -3343,7 +3343,7 @@ int rdma_join_multicast(struct rdma_cm_id *id, struct sockaddr *addr,if (rdma_tech_iboe (id->device, id->port_num)) {kref_init (&mc->mcref); ret =cma_iboe_join_multicast (id_priv, mc); - }else if (rdma_tech_ib (id->device, id->port_num)) + }else if (cap_ib_mcast (id->device, id->port_num)) ret =cma_join_ib_multicast (id_priv, mc);else ret = -ENOSYS ;
@@ -3377,7 +3377,7 @@ void rdma_leave_multicast(struct rdma_cm_id *id, struct sockaddr *addr)BUG_ON (id_priv->cma_dev->device != id->device); -if (rdma_tech_ib (id->device, id->port_num)) { +if (cap_ib_mcast (id->device, id->port_num)) {ib_sa_free_multicast (mc->multicast.ib);kfree (mc); } else if (rdma_tech_iboe(id->device,
drivers/infiniband/core/multicast.c +3 -3
@@ -780,7 +780,7 @@ static void mcast_event_handler(struct ib_event_handler *handler,int index; dev = container_of(handler, struct mcast_device, event_handler); -if (WARN_ON (!rdma_tech_ib (dev->device, event->element.port_num))) +if (WARN_ON (!cap_ib_mcast (dev->device, event->element.port_num)))return ; index = event->element.port_num - dev->start_port;
@@ -820,7 +820,7 @@ static void mcast_add_one(struct ib_device *device) }for (i =0 ; i <= dev->end_port - dev->start_port; i++) { -if (!rdma_tech_ib (device, dev->start_port + i)) +if (!cap_ib_mcast (device, dev->start_port + i))continue ; port = &dev->port[i]; port->dev = dev;
@@ -858,7 +858,7 @@ static void mcast_remove_one(struct ib_device *device)flush_workqueue (mcast_wq);for (i =0 ; i <= dev->end_port - dev->start_port; i++) { -if (rdma_tech_ib (device, dev->start_port + i)) { +if (cap_ib_mcast (device, dev->start_port + i)) { port = &dev->port[i];deref_port (port);wait_for_completion (&port->comp);
include/rdma/ib_verbs.h +15 -0
@@ -1848,6 +1848,21 @@ static inline int cap_ib_sa(struct ib_device *device, u8 port_num)return rdma_tech_ib (device, port_num); } +/** +* cap_ib_mcast - Check if the port of device has the capability Infiniband +* Multicast. +* +* @device: Device to be checked +* @port_num: Port number of the device +* +* Return 0 when port of the device don't support Infiniband +* Multicast. +*/ +static inline int cap_ib_mcast (struct ib_device *device,u8 port_num) +{ +return cap_ib_sa (device, port_num); +} +int ib_query_gid (struct ib_device *device,u8 port_num,int index,union ib_gid *gid);
Signature · 1 line
2.1.0
PATCH v4 22/27 IB/Verbs: Use management helper cap_ipoib()
In reply to Michael WangIntroduce helper cap_ipoib() to help us check if the port of an IB device support IP over Infiniband.
--- drivers/infiniband/ulp/ipoib/ipoib_main.c | 2 +- include/rdma/ib_verbs.h | 15 +++++++++++++++ 2 files changed, 16 insertions(+), 1 deletion(-)
drivers/infiniband/ulp/ipoib/ipoib_main.c +1 -1
@@ -1671,7 +1671,7 @@ static void ipoib_add_one(struct ib_device *device) }for (p = s; p <= e; ++p) { -if (!rdma_tech_ib (device, p)) +if (!cap_ipoib (device, p))continue ; dev =ipoib_add_port ("ib%d" , device, p);if (!IS_ERR (dev)) {
include/rdma/ib_verbs.h +15 -0
@@ -1863,6 +1863,21 @@ static inline int cap_ib_mcast(struct ib_device *device, u8 port_num)return cap_ib_sa (device, port_num); } +/** +* cap_ipoib - Check if the port of device has the capability +* IP over Infiniband. +* +* @device: Device to be checked +* @port_num: Port number of the device +* +* Return 0 when port of the device don't support +* IP over Infiniband. +*/ +static inline int cap_ipoib (struct ib_device *device,u8 port_num) +{ +return rdma_tech_ib (device, port_num); +} +int ib_query_gid (struct ib_device *device,u8 port_num,int index,union ib_gid *gid);
Signature · 5 lines
2.1.0 -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
PATCH v4 23/27 IB/Verbs: Use management helper cap_read_multi_sge()
In reply to Michael WangIntroduce helper cap_read_multi_sge() to help us check if the port of an IB device support RDMA Read Multiple Scatter-Gather Entries.
--- include/rdma/ib_verbs.h | 15 +++++++++++++++ net/sunrpc/xprtrdma/svc_rdma_recvfrom.c | 3 ++- 2 files changed, 17 insertions(+), 1 deletion(-)
include/rdma/ib_verbs.h +15 -0
@@ -1878,6 +1878,21 @@ static inline int cap_ipoib(struct ib_device *device, u8 port_num)return rdma_tech_ib (device, port_num); } +/** +* cap_read_multi_sge - Check if the port of device has the capability +* RDMA Read Multiple Scatter-Gather Entries. +* +* @device: Device to be checked +* @port_num: Port number of the device +* +* Return 0 when port of the device don't support +* RDMA Read Multiple Scatter-Gather Entries. +*/ +static inline int cap_read_multi_sge (struct ib_device *device,u8 port_num) +{ +return !rdma_tech_iwarp (device, port_num); +} +int ib_query_gid (struct ib_device *device,u8 port_num,int index,union ib_gid *gid);
net/sunrpc/xprtrdma/svc_rdma_recvfrom.c +2 -1
@@ -117,7 +117,8 @@ static void rdma_build_arg_xdr(struct svc_rqst *rqstp,static int rdma_read_max_sge (struct svcxprt_rdma *xprt,int sge_count) { -if (rdma_tech_iwarp (xprt->sc_cm_id->device, xprt->sc_cm_id->port_num)) +if (!cap_read_multi_sge (xprt->sc_cm_id->device, + xprt->sc_cm_id->port_num))return 1 ;else return min_t (int, sge_count, xprt->sc_max_sge);
Signature · 1 line
2.1.0
PATCH v4 24/27 IB/Verbs: Use management helper cap_af_ib()
In reply to Michael WangIntroduce helper cap_af_ib() to help us check if the port of an IB device support Native Infiniband Address.
--- drivers/infiniband/core/cma.c | 2 +- include/rdma/ib_verbs.h | 15 +++++++++++++++ 2 files changed, 16 insertions(+), 1 deletion(-)
drivers/infiniband/core/cma.c +1 -1
@@ -470,7 +470,7 @@ static int cma_resolve_ib_dev(struct rdma_id_private *id_priv)list_for_each_entry (cur_dev, &dev_list, list) {for (p =1 ; p <= cur_dev->device->phys_port_cnt; ++p) { -if (!rdma_ib_or_iboe (cur_dev->device, p)) +if (!cap_af_ib (cur_dev->device, p))continue ;if (ib_find_cached_pkey (cur_dev->device, p, pkey, &index))
include/rdma/ib_verbs.h +15 -0
@@ -1879,6 +1879,21 @@ static inline int cap_ipoib(struct ib_device *device, u8 port_num) }/** +* cap_af_ib - Check if the port of device has the capability +* Native Infiniband Address. +* +* @device: Device to be checked +* @port_num: Port number of the device +* +* Return 0 when port of the device don't support +* Native Infiniband Address. +*/ +static inline int cap_af_ib (struct ib_device *device,u8 port_num) +{ +return rdma_ib_or_iboe (device, port_num); +} + +/** * cap_read_multi_sge - Check if the port of device has the capability * RDMA Read Multiple Scatter-Gather Entries. *
Signature · 5 lines
2.1.0 -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
PATCH v4 25/27 IB/Verbs: Use management helper cap_eth_ah()
In reply to Michael WangIntroduce helper cap_eth_ah() to help us check if the port of an IB device support Ethernet Address Handler.
--- drivers/infiniband/core/cma.c | 6 +++--- drivers/infiniband/core/sa_query.c | 2 +- drivers/infiniband/core/verbs.c | 2 +- include/rdma/ib_verbs.h | 15 +++++++++++++++ 4 files changed, 20 insertions(+), 5 deletions(-)
drivers/infiniband/core/cma.c +3 -3
@@ -733,10 +733,10 @@ static int cma_ib_init_qp_attr(struct rdma_id_private *id_priv,int ret;u16 pkey; -if (rdma_tech_ib (id_priv->id.device, id_priv->id.port_num)) - pkey =ib_addr_get_pkey (dev_addr); -else +if (cap_eth_ah (id_priv->id.device, id_priv->id.port_num)) pkey =0xffff ; +else + pkey =ib_addr_get_pkey (dev_addr); ret =ib_find_cached_pkey (id_priv->id.device, id_priv->id.port_num, pkey, &qp_attr->pkey_index);
drivers/infiniband/core/sa_query.c +1 -1
@@ -540,7 +540,7 @@ int ib_init_ah_from_path(struct ib_device *device, u8 port_num, ah_attr->port_num = port_num; ah_attr->static_rate = rec->rate; - force_grh =rdma_tech_iboe (device, port_num); + force_grh =cap_eth_ah (device, port_num);if (rec->hop_limit >1 || force_grh) { ah_attr->ah_flags =IB_AH_GRH ;
drivers/infiniband/core/verbs.c +1 -1
@@ -202,7 +202,7 @@ int ib_init_ah_from_wc(struct ib_device *device, u8 port_num, struct ib_wc *wc,int ret;memset (ah_attr,0 ,sizeof *ah_attr); -if (rdma_tech_iboe (device, port_num)) { +if (cap_eth_ah (device, port_num)) {if (!(wc->wc_flags &IB_WC_GRH ))return -EPROTOTYPE ;
include/rdma/ib_verbs.h +15 -0
@@ -1894,6 +1894,21 @@ static inline int cap_af_ib(struct ib_device *device, u8 port_num) }/** +* cap_eth_ah - Check if the port of device has the capability +* Ethernet Address Handler. +* +* @device: Device to be checked +* @port_num: Port number of the device +* +* Return 0 when port of the device don't support +* Ethernet Address Handler. +*/ +static inline int cap_eth_ah (struct ib_device *device,u8 port_num) +{ +return rdma_tech_iboe (device, port_num); +} + +/** * cap_read_multi_sge - Check if the port of device has the capability * RDMA Read Multiple Scatter-Gather Entries. *
Signature · 1 line
2.1.0
PATCH v4 26/27 IB/Verbs: Clean up rdma_ib_or_iboe()
In reply to Michael WangWe have finished introducing the cap_XX(), and raw helper rdma_ib_or_iboe() is no longer necessary, thus clean it up.
--- include/rdma/ib_verbs.h | 19 +++++++++---------- net/sunrpc/xprtrdma/svc_rdma_transport.c | 6 ++++-- 2 files changed, 13 insertions(+), 12 deletions(-)
include/rdma/ib_verbs.h +9 -10
@@ -1766,13 +1766,6 @@ static inline int rdma_tech_iwarp(struct ib_device *device, u8 port_num) == RDMA_TRANSPORT_IWARP; } -static inline int rdma_ib_or_iboe (struct ib_device *device,u8 port_num) -{ -enum rdma_transport_type tp = device->query_transport (device, port_num); - -return (tp ==RDMA_TRANSPORT_IB || tp ==RDMA_TRANSPORT_IBOE ); -} -/** * cap_ib_mad - Check if the port of device has the capability Infiniband * Management Datagrams.
@@ -1785,7 +1778,9 @@ static inline int rdma_ib_or_iboe(struct ib_device *device, u8 port_num)*/ static inline int cap_ib_mad (struct ib_device *device,u8 port_num) { -return rdma_ib_or_iboe (device, port_num); +enum rdma_transport_type tp = device->query_transport (device, port_num); + +return (tp ==RDMA_TRANSPORT_IB || tp ==RDMA_TRANSPORT_IBOE ); }/**
@@ -1815,7 +1810,9 @@ static inline int cap_ib_smi(struct ib_device *device, u8 port_num)*/ static inline int cap_ib_cm (struct ib_device *device,u8 port_num) { -return rdma_ib_or_iboe (device, port_num); +enum rdma_transport_type tp = device->query_transport (device, port_num); + +return (tp ==RDMA_TRANSPORT_IB || tp ==RDMA_TRANSPORT_IBOE ); }/**
@@ -1890,7 +1887,9 @@ static inline int cap_ipoib(struct ib_device *device, u8 port_num)*/ static inline int cap_af_ib (struct ib_device *device,u8 port_num) { -return rdma_ib_or_iboe (device, port_num); +enum rdma_transport_type tp = device->query_transport (device, port_num); + +return (tp ==RDMA_TRANSPORT_IB || tp ==RDMA_TRANSPORT_IBOE ); }/**
net/sunrpc/xprtrdma/svc_rdma_transport.c +4 -2
@@ -987,8 +987,10 @@ static struct svc_xprt *svc_rdma_accept(struct svc_xprt *xprt)*/ if (!rdma_tech_iwarp (newxprt->sc_cm_id->device, newxprt->sc_cm_id->port_num) && - !rdma_ib_or_iboe (newxprt->sc_cm_id->device, - newxprt->sc_cm_id->port_num)) + !rdma_tech_ib (newxprt->sc_cm_id->device, + newxprt->sc_cm_id->port_num) && + !rdma_tech_iboe (newxprt->sc_cm_id->device, + newxprt->sc_cm_id->port_num)) goto errout; if (!(newxprt->sc_dev_caps & SVCRDMA_DEVCAP_FAST_REG) ||
Signature · 5 lines
2.1.0 -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
PATCH v4 27/27 IB/Verbs: Cleanup rdma_node_get_transport()
In reply to Michael WangWe have get rid of all the scene using legacy rdma_node_get_transport(), now clean it up.
--- drivers/infiniband/core/verbs.c | 21 --------------------- include/rdma/ib_verbs.h | 3 --- 2 files changed, 24 deletions(-)
drivers/infiniband/core/verbs.c +0 -21
@@ -107,27 +107,6 @@ __attribute_const__ int ib_rate_to_mbps(enum ib_rate rate) }EXPORT_SYMBOL (ib_rate_to_mbps); -__attribute_const__ enum rdma_transport_type -rdma_node_get_transport (enum rdma_node_type node_type) -{ -switch (node_type) { -case RDMA_NODE_IB_CA : -case RDMA_NODE_IB_SWITCH : -case RDMA_NODE_IB_ROUTER : -return RDMA_TRANSPORT_IB ; -case RDMA_NODE_RNIC : -return RDMA_TRANSPORT_IWARP ; -case RDMA_NODE_USNIC : -return RDMA_TRANSPORT_USNIC ; -case RDMA_NODE_USNIC_UDP : -return RDMA_TRANSPORT_USNIC_UDP ; -default : -BUG (); -return 0 ; - } -} -EXPORT_SYMBOL (rdma_node_get_transport); -enum rdma_link_layer rdma_port_get_link_layer (struct ib_device *device,u8 port_num) {if (device->get_link_layer)
include/rdma/ib_verbs.h +0 -3
@@ -84,9 +84,6 @@ enum rdma_transport_type { RDMA_TRANSPORT_IBOE, }; -__attribute_const__ enum rdma_transport_type -rdma_node_get_transport (enum rdma_node_type node_type); -enum rdma_link_layer {IB_LINK_LAYER_UNSPECIFIED ,IB_LINK_LAYER_INFINIBAND ,
Signature · 5 lines
2.1.0 -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v4 03/27] IB/Verbs: Reform IB-core mad/agent/user_mad
In reply to Michael WangQuoted 92 lines · Michael Wang
Use raw management helpers to reform IB-core mad/agent/user_mad.
Cc Steve Wise <[email protected]>Cc Tom Talpey <[email protected]>Cc Jason Gunthorpe <[email protected]>Cc Doug Ledford <[email protected]>Cc Ira Weiny <[email protected]>Cc Sean Hefty <[email protected]>Signed-off-by Michael Wang <[email protected]>--- drivers/infiniband/core/agent.c | 2 +- drivers/infiniband/core/mad.c | 20 ++++++++++---------- drivers/infiniband/core/user_mad.c | 26 ++++++++++++++++++++------ 3 files changed, 31 insertions(+), 17 deletions(-)drivers/infiniband/core/agent.c +1
-1@@ -156,7 +156,7 @@ int ib_agent_port_open(struct ib_device *device, int port_num) goto error1; } -if (rdma_port_get_link_layer (device, port_num) ==IB_LINK_LAYER_INFINIBAND ) { +if (rdma_tech_ib (device, port_num)) {/* Obtain send only MAD agent for SMI QP */ port_priv->agent[0] = ib_register_mad_agent(device, port_num, IB_QPT_SMI, NULL, 0,drivers/infiniband/core/mad.c +7
-10@@ -2938,7 +2938,7 @@ static int ib_mad_port_open(struct ib_device *device,init_mad_qp (port_priv, &port_priv->qp_info[1 ]); cq_size = mad_sendq_size + mad_recvq_size; - has_smi =rdma_port_get_link_layer (device, port_num) ==IB_LINK_LAYER_INFINIBAND ; + has_smi =rdma_tech_ib (device, port_num);if (has_smi) cq_size *=2 ;@@ -3057,9 +3057,6 @@ static void ib_mad_init_device(struct ib_device *device) {int start, end, i; -if (rdma_node_get_transport (device->node_type) !=RDMA_TRANSPORT_IB ) -return ; -if (device->node_type ==RDMA_NODE_IB_SWITCH ) { start =0 ; end =0 ;@@ -3069,6 +3066,9 @@ static void ib_mad_init_device(struct ib_device *device) }for (i = start; i <= end; i++) { +if (!rdma_ib_or_iboe (device, i)) +continue ; +if (ib_mad_port_open (device, i)) {dev_err (&device->dev,"Couldn't open port %d\n" , i); goto error;@@ -3086,15 +3086,15 @@ error_agent:dev_err (&device->dev,"Couldn't close port %d\n" , i); error: - i--; +while (--i >= start) { +if (!rdma_ib_or_iboe (device, i)) +continue ; -while (i >= start) {if (ib_agent_port_close (device, i))dev_err (&device->dev,"Couldn't close port %d for agents\n" , i);if (ib_mad_port_close (device, i))dev_err (&device->dev,"Couldn't close port %d\n" , i); - i--; } }@@ -3102,9 +3102,6 @@ static void ib_mad_remove_device(struct ib_device *device) {int i, num_ports, cur_port; -if (rdma_node_get_transport (device->node_type) !=RDMA_TRANSPORT_IB ) -return ; -if (device->node_type ==RDMA_NODE_IB_SWITCH ) { num_ports =1 ; cur_port =0 ;
drivers/infiniband/core/mad.c static void ib_mad_remove_device(struct ib_device *device) · line 3113
Quoted 4 lines
drivers/infiniband/core/mad.c +1
-0@@ -3113,6 +3110,9 @@ static void ib_mad_remove_device(struct ib_device *device) cur_port =1 ; }for (i =0 ; i < num_ports; i++, cur_port++) { +if (!rdma_ib_or_iboe (device, i))
Shouldn't this be:
if (!rdma_ib_or_iboe(device, cur_port))
?
-- Hal
Quoted 69 lines
drivers/infiniband/core/mad.c +2
-0+continue ; +if (ib_agent_port_close (device, cur_port)) dev_err(&device->dev, "Couldn't close port %d for agents\n",drivers/infiniband/core/user_mad.c +20
-6@@ -1273,9 +1273,7 @@ static void ib_umad_add_one(struct ib_device *device) {struct ib_umad_device *umad_dev;int s, e, i; - -if (rdma_node_get_transport (device->node_type) !=RDMA_TRANSPORT_IB ) -return ; +int count =0 ;if (device->node_type ==RDMA_NODE_IB_SWITCH ) s = e =0 ;@@ -1296,11 +1294,21 @@ static void ib_umad_add_one(struct ib_device *device) umad_dev->end_port = e;for (i = s; i <= e; ++i) { +if (!rdma_ib_or_iboe (device, i)) +continue ; + umad_dev->port[i - s].umad_dev = umad_dev;if (ib_umad_init_port (device, i, umad_dev, &umad_dev->port[i - s])) goto err; + + count++; + } + +if (!count) { +kobject_put (&umad_dev->kobj); +return ; }ib_set_client_data (device, &umad_client, umad_dev);@@ -1308,8 +1316,12 @@ static void ib_umad_add_one(struct ib_device *device)return ; err: -while (--i >= s) +while (--i >= s) { +if (!rdma_ib_or_iboe (device, i)) +continue ; +ib_umad_kill_port (&umad_dev->port[i - s]); + }kobject_put (&umad_dev->kobj); }@@ -1322,8 +1334,10 @@ static void ib_umad_remove_one(struct ib_device *device)if (!umad_dev)return ; -for (i =0 ; i <= umad_dev->end_port - umad_dev->start_port; ++i) -ib_umad_kill_port (&umad_dev->port[i]); +for (i =0 ; i <= umad_dev->end_port - umad_dev->start_port; ++i) { +if (rdma_ib_or_iboe (device, i)) +ib_umad_kill_port (&umad_dev->port[i]); + }kobject_put (&umad_dev->kobj); }
Re: [PATCH v4 04/27] IB/Verbs: Reform IB-core cm
In reply to Michael WangQuoted 41 lines · Michael Wang
Use raw management helpers to reform IB-core cm.
Cc Steve Wise <[email protected]>Cc Tom Talpey <[email protected]>Cc Jason Gunthorpe <jgunthorpe-ePGOBjL8dl3ta4EC/59zMFaTQe2KTcn/@public.gmane.org>Cc Doug Ledford <[email protected]>Cc Ira Weiny <[email protected]>Cc Sean Hefty <[email protected]>Signed-off-by Michael Wang <[email protected]>--- drivers/infiniband/core/cm.c | 22 +++++++++++++++++++--- 1 file changed, 19 insertions(+), 3 deletions(-)drivers/infiniband/core/cm.c +4
-3@@ -3761,9 +3761,7 @@ static void cm_add_one(struct ib_device *ib_device)unsigned long flags;int ret;u8 i; - -if (rdma_node_get_transport (ib_device->node_type) !=RDMA_TRANSPORT_IB ) -return ; +int count =0 ; cm_dev =kzalloc (sizeof (*cm_dev) +sizeof (*port) * ib_device->phys_port_cnt,GFP_KERNEL );@@ -3783,6 +3781,9 @@ static void cm_add_one(struct ib_device *ib_device)set_bit (IB_MGMT_METHOD_SEND , reg_req.method_mask);for (i =1 ; i <= ib_device->phys_port_cnt; i++) { +if (!rdma_ib_or_iboe (ib_device, i)) +continue ; + port =kzalloc (sizeof *port,GFP_KERNEL );if (!port) goto error1;@@ -3809,7 +3810,16 @@ static void cm_add_one(struct ib_device *ib_device) ret =ib_modify_port (ib_device, i,0 , &port_modify);
drivers/infiniband/core/cm.c static void cm_add_one(struct ib_device *ib_device) · line 3820
Quoted 10 lines
drivers/infiniband/core/cm.c +7
-0if (ret) goto error3; + + count++; } + +if (!count) { +device_unregister (cm_dev->device); +kfree (cm_dev); +return ;
Nit: alternatively, this could be goto at end of this routine where there is same 2 calls. I think Jason made this comment on earlier set of these patches.
-- Hal
Quoted 16 lines
drivers/infiniband/core/cm.c +5
-0+ } +ib_set_client_data (ib_device, &cm_client, cm_dev);write_lock_irqsave (&cm.device_lock, flags);@@ -3825,6 +3835,9 @@ error1: port_modify.set_port_cap_mask =0 ; port_modify.clr_port_cap_mask =IB_PORT_CM_SUP ;while (--i) { +if (!rdma_ib_or_iboe (ib_device, i)) +continue ; + port = cm_dev->port[i-1 ];ib_modify_port (ib_device, port->port_num,0 , &port_modify);ib_unregister_mad_agent (port->mad_agent);
drivers/infiniband/core/cm.c static void cm_remove_one(struct ib_device *ib_device) · line 3874
Quoted 9 lines
drivers/infiniband/core/cm.c +3
-0@@ -3853,6 +3866,9 @@ static void cm_remove_one(struct ib_device *ib_device)write_unlock_irqrestore (&cm.device_lock, flags);for (i =1 ; i <= ib_device->phys_port_cnt; i++) { +if (!rdma_ib_or_iboe (ib_device, i)) +continue ; + port = cm_dev->port[i-1 ];ib_modify_port (ib_device, port->port_num,0 , &port_modify);ib_unregister_mad_agent (port->mad_agent);
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v4 03/27] IB/Verbs: Reform IB-core mad/agent/user_mad
In reply to Hal RosenstockHi,Hal
Thanks for the comments :-)
On 04/16/2015 02:43 PM, Hal Rosenstock wrote:
[snip]
Quoted 17 lines
Quoted 11 lines
drivers/infiniband/core/mad.c +0
-3@@ -3102,9 +3102,6 @@ static void ib_mad_remove_device(struct ib_device *device) {int i, num_ports, cur_port; -if (rdma_node_get_transport (device->node_type) !=RDMA_TRANSPORT_IB ) -return ; -if (device->node_type ==RDMA_NODE_IB_SWITCH ) { num_ports =1 ; cur_port =0 ;drivers/infiniband/core/mad.c static void ib_mad_remove_device(struct ib_device *device) · line 3113
Quoted 4 lines
drivers/infiniband/core/mad.c +1
-0@@ -3113,6 +3110,9 @@ static void ib_mad_remove_device(struct ib_device *device) cur_port =1 ; }for (i =0 ; i < num_ports; i++, cur_port++) { +if (!rdma_ib_or_iboe (device, i))Shouldn't this be:
Quoted 2 lines
if (!rdma_ib_or_iboe(device, cur_port))?
Agree, actually it's unnecessary to use both i and cur_port, we can just use the same way inside ib_mad_init_device() ;-)
Regards,
Michael Wang
Quoted 70 lines
-- Hal
Quoted 67 lines
drivers/infiniband/core/mad.c +2
-0+continue ; +if (ib_agent_port_close (device, cur_port)) dev_err(&device->dev, "Couldn't close port %d for agents\n",drivers/infiniband/core/user_mad.c +20
-6@@ -1273,9 +1273,7 @@ static void ib_umad_add_one(struct ib_device *device) {struct ib_umad_device *umad_dev;int s, e, i; - -if (rdma_node_get_transport (device->node_type) !=RDMA_TRANSPORT_IB ) -return ; +int count =0 ;if (device->node_type ==RDMA_NODE_IB_SWITCH ) s = e =0 ;@@ -1296,11 +1294,21 @@ static void ib_umad_add_one(struct ib_device *device) umad_dev->end_port = e;for (i = s; i <= e; ++i) { +if (!rdma_ib_or_iboe (device, i)) +continue ; + umad_dev->port[i - s].umad_dev = umad_dev;if (ib_umad_init_port (device, i, umad_dev, &umad_dev->port[i - s])) goto err; + + count++; + } + +if (!count) { +kobject_put (&umad_dev->kobj); +return ; }ib_set_client_data (device, &umad_client, umad_dev);@@ -1308,8 +1316,12 @@ static void ib_umad_add_one(struct ib_device *device)return ; err: -while (--i >= s) +while (--i >= s) { +if (!rdma_ib_or_iboe (device, i)) +continue ; +ib_umad_kill_port (&umad_dev->port[i - s]); + }kobject_put (&umad_dev->kobj); }@@ -1322,8 +1334,10 @@ static void ib_umad_remove_one(struct ib_device *device)if (!umad_dev)return ; -for (i =0 ; i <= umad_dev->end_port - umad_dev->start_port; ++i) -ib_umad_kill_port (&umad_dev->port[i]); +for (i =0 ; i <= umad_dev->end_port - umad_dev->start_port; ++i) { +if (rdma_ib_or_iboe (device, i)) +ib_umad_kill_port (&umad_dev->port[i]); + }
Quoted 3 lines
drivers/infiniband/core/user_mad.c static void ib_umad_remove_one(struct ib_device *device) · line 1343
Quoted 2 lines
drivers/infiniband/core/user_mad.c +0
-0kobject_put (&umad_dev->kobj); }
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v4 04/27] IB/Verbs: Reform IB-core cm
In reply to Hal RosenstockOn 04/16/2015 02:43 PM, Hal Rosenstock wrote:
[snip]
Quoted 4 lines
Quoted 4 lines
drivers/infiniband/core/cm.c +3
-0+ + count++; } +
Quoted 8 lines
drivers/infiniband/core/cm.c static void cm_add_one(struct ib_device *ib_device) · line 3820
Quoted 4 lines
drivers/infiniband/core/cm.c +4
-0+if (!count) { +device_unregister (cm_dev->device); +kfree (cm_dev); +return ;Nit: alternatively, this could be goto at end of this routine where there is same 2 calls. I think Jason made this comment on earlier set of these patches.
Thanks for the notify, I guess I missed it, my bad :-P
Will be "goto free" in next version.
Regards,
Michael Wang
Quoted 21 lines
-- Hal
Quoted 18 lines
drivers/infiniband/core/cm.c +5
-0+ } +ib_set_client_data (ib_device, &cm_client, cm_dev);write_lock_irqsave (&cm.device_lock, flags);@@ -3825,6 +3835,9 @@ error1: port_modify.set_port_cap_mask =0 ; port_modify.clr_port_cap_mask =IB_PORT_CM_SUP ;while (--i) { +if (!rdma_ib_or_iboe (ib_device, i)) +continue ; + port = cm_dev->port[i-1 ];ib_modify_port (ib_device, port->port_num,0 , &port_modify);ib_unregister_mad_agent (port->mad_agent);@@ -3853,6 +3866,9 @@ static void cm_remove_one(struct ib_device *ib_device)write_unlock_irqrestore (&cm.device_lock, flags);
Quoted 8 lines
drivers/infiniband/core/cm.c static void cm_remove_one(struct ib_device *ib_device) · line 3874
Quoted 7 lines
drivers/infiniband/core/cm.c +3
-0for (i =1 ; i <= ib_device->phys_port_cnt; i++) { +if (!rdma_ib_or_iboe (ib_device, i)) +continue ; + port = cm_dev->port[i-1 ];ib_modify_port (ib_device, port->port_num,0 , &port_modify);ib_unregister_mad_agent (port->mad_agent);
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v4 10/27] IB/Verbs: Reform cm related part in IB-core cma/ucm
In reply to Michael WangQuoted 6 lines · Michael Wang
Use raw management helpers to reform cm related part in IB-core cma/ucm.
These checks focus on the device cm type rather than the port capability, directly pass port 1 works currently, but can't support mixing cm type device in future.
This is equivalent to today where the checks are per node rather than per port.
Should all checks here be port 1 based or only certain ones like listen ? For example, in connect/reject/disconnect, don't we already have port ? Guess this can be dealt with later as this is not a regression from the current implementation.
-- Hal
Quoted 230 lines
Cc Steve Wise <[email protected]>Cc Tom Talpey <[email protected]>Cc Jason Gunthorpe <jgunthorpe-ePGOBjL8dl3ta4EC/59zMFaTQe2KTcn/@public.gmane.org>Cc Doug Ledford <[email protected]>Cc Ira Weiny <[email protected]>Cc Sean Hefty <[email protected]>Signed-off-by Michael Wang <[email protected]>--- drivers/infiniband/core/cma.c | 81 +++++++++++++------------------------------ drivers/infiniband/core/ucm.c | 3 +- 2 files changed, 26 insertions(+), 58 deletions(-)drivers/infiniband/core/cma.c +25
-56@@ -735,8 +735,7 @@ int rdma_init_qp_attr(struct rdma_cm_id *id, struct ib_qp_attr *qp_attr,int ret =0 ; id_priv = container_of(id, struct rdma_id_private, id); -switch (rdma_node_get_transport (id_priv->id.device->node_type)) { -case RDMA_TRANSPORT_IB : +if (rdma_ib_or_iboe (id_priv->id.device,1 )) {if (!id_priv->cm_id.ib || (id_priv->id.qp_type ==IB_QPT_UD )) ret =cma_ib_init_qp_attr (id_priv, qp_attr, qp_attr_mask);else @@ -745,19 +744,15 @@ int rdma_init_qp_attr(struct rdma_cm_id *id, struct ib_qp_attr *qp_attr,if (qp_attr->qp_state ==IB_QPS_RTR ) qp_attr->rq_psn = id_priv->seq_num; -break ; -case RDMA_TRANSPORT_IWARP : + }else if (rdma_tech_iwarp (id_priv->id.device,1 )) {if (!id_priv->cm_id.iw) { qp_attr->qp_access_flags =0 ; *qp_attr_mask =IB_QP_STATE |IB_QP_ACCESS_FLAGS ; }else ret =iw_cm_init_qp_attr (id_priv->cm_id.iw, qp_attr, qp_attr_mask); -break ; -default : + }else ret = -ENOSYS ; -break ; - }return ret; }@@ -1037,17 +1032,12 @@ void rdma_destroy_id(struct rdma_cm_id *id)mutex_unlock (&id_priv->handler_mutex);if (id_priv->cma_dev) { -switch (rdma_node_get_transport (id_priv->id.device->node_type)) { -case RDMA_TRANSPORT_IB : +if (rdma_ib_or_iboe (id_priv->id.device,1 )) {if (id_priv->cm_id.ib)ib_destroy_cm_id (id_priv->cm_id.ib); -break ; -case RDMA_TRANSPORT_IWARP : + }else if (rdma_tech_iwarp (id_priv->id.device,1 )) {if (id_priv->cm_id.iw)iw_destroy_cm_id (id_priv->cm_id.iw); -break ; -default : -break ; }cma_leave_mc_groups (id_priv);cma_release_dev (id_priv);@@ -1626,7 +1616,7 @@ static void cma_listen_on_dev(struct rdma_id_private *id_priv,int ret;if (cma_family (id_priv) ==AF_IB && -rdma_node_get_transport (cma_dev->device->node_type) !=RDMA_TRANSPORT_IB ) + !rdma_ib_or_iboe (cma_dev->device,1 ))return ; id = rdma_create_id(cma_listen_handler, id_priv, id_priv->id.ps,@@ -2028,7 +2018,7 @@ static int cma_bind_loopback(struct rdma_id_private *id_priv)mutex_lock (&lock);list_for_each_entry (cur_dev, &dev_list, list) {if (cma_family (id_priv) ==AF_IB && -rdma_node_get_transport (cur_dev->device->node_type) !=RDMA_TRANSPORT_IB ) + !rdma_ib_or_iboe (cur_dev->device,1 ))continue ;if (!cma_dev)@@ -2060,7 +2050,7 @@ port_found: goto out; id_priv->id.route.addr.dev_addr.dev_type = - (rdma_port_get_link_layer (cma_dev->device, p) ==IB_LINK_LAYER_INFINIBAND ) ? + (rdma_tech_ib (cma_dev->device, p)) ?ARPHRD_INFINIBAND :ARPHRD_ETHER ;rdma_addr_set_sgid (&id_priv->id.route.addr.dev_addr, &gid);@@ -2537,18 +2527,15 @@ int rdma_listen(struct rdma_cm_id *id, int backlog) id_priv->backlog = backlog;if (id->device) { -switch (rdma_node_get_transport (id->device->node_type)) { -case RDMA_TRANSPORT_IB : +if (rdma_ib_or_iboe (id->device,1 )) { ret =cma_ib_listen (id_priv);if (ret) goto err; -break ; -case RDMA_TRANSPORT_IWARP : + }else if (rdma_tech_iwarp (id->device,1 )) { ret =cma_iw_listen (id_priv, backlog);if (ret) goto err; -break ; -default : + }else { ret = -ENOSYS ; goto err; }@@ -2884,20 +2871,15 @@ int rdma_connect(struct rdma_cm_id *id, struct rdma_conn_param *conn_param) id_priv->srq = conn_param->srq; } -switch (rdma_node_get_transport (id->device->node_type)) { -case RDMA_TRANSPORT_IB : +if (rdma_ib_or_iboe (id->device,1 )) {if (id->qp_type ==IB_QPT_UD ) ret =cma_resolve_ib_udp (id_priv, conn_param);else ret =cma_connect_ib (id_priv, conn_param); -break ; -case RDMA_TRANSPORT_IWARP : + }else if (rdma_tech_iwarp (id->device,1 )) ret =cma_connect_iw (id_priv, conn_param); -break ; -default : +else ret = -ENOSYS ; -break ; - }if (ret) goto err;@@ -3000,8 +2982,7 @@ int rdma_accept(struct rdma_cm_id *id, struct rdma_conn_param *conn_param) id_priv->srq = conn_param->srq; } -switch (rdma_node_get_transport (id->device->node_type)) { -case RDMA_TRANSPORT_IB : +if (rdma_ib_or_iboe (id->device,1 )) {if (id->qp_type ==IB_QPT_UD ) {if (conn_param) ret = cma_send_sidr_rep(id_priv, IB_SIDR_SUCCESS,@@ -3017,14 +2998,10 @@ int rdma_accept(struct rdma_cm_id *id, struct rdma_conn_param *conn_param)else ret =cma_rep_recv (id_priv); } -break ; -case RDMA_TRANSPORT_IWARP : + }else if (rdma_tech_iwarp (id->device,1 )) ret =cma_accept_iw (id_priv, conn_param); -break ; -default : +else ret = -ENOSYS ; -break ; - }if (ret) goto reject;@@ -3068,8 +3045,7 @@ int rdma_reject(struct rdma_cm_id *id, const void *private_data,if (!id_priv->cm_id.ib)return -EINVAL ; -switch (rdma_node_get_transport (id->device->node_type)) { -case RDMA_TRANSPORT_IB : +if (rdma_ib_or_iboe (id->device,1 )) {if (id->qp_type ==IB_QPT_UD ) ret =cma_send_sidr_rep (id_priv,IB_SIDR_REJECT ,0 , private_data, private_data_len);@@ -3077,15 +3053,12 @@ int rdma_reject(struct rdma_cm_id *id, const void *private_data, ret =ib_send_cm_rej (id_priv->cm_id.ib,IB_CM_REJ_CONSUMER_DEFINED ,NULL ,0 , private_data, private_data_len); -break ; -case RDMA_TRANSPORT_IWARP : + }else if (rdma_tech_iwarp (id->device,1 )) { ret =iw_cm_reject (id_priv->cm_id.iw, private_data, private_data_len); -break ; -default : + }else ret = -ENOSYS ; -break ; - } +return ret; }EXPORT_SYMBOL (rdma_reject);@@ -3099,22 +3072,18 @@ int rdma_disconnect(struct rdma_cm_id *id)if (!id_priv->cm_id.ib)return -EINVAL ; -switch (rdma_node_get_transport (id->device->node_type)) { -case RDMA_TRANSPORT_IB : +if (rdma_ib_or_iboe (id->device,1 )) { ret =cma_modify_qp_err (id_priv);if (ret) goto out;/* Initiate or respond to a disconnect. */ if (ib_send_cm_dreq (id_priv->cm_id.ib,NULL ,0 ))ib_send_cm_drep (id_priv->cm_id.ib,NULL ,0 ); -break ; -case RDMA_TRANSPORT_IWARP : + }else if (rdma_tech_iwarp (id->device,1 )) { ret =iw_cm_disconnect (id_priv->cm_id.iw,0 ); -break ; -default : + }else ret = -EINVAL ; -break ; - } + out:return ret; }drivers/infiniband/core/ucm.c +0
-0
drivers/infiniband/core/ucm.c static void ib_ucm_add_one(struct ib_device *device) · line 1259
Quoted 9 lines
drivers/infiniband/core/ucm.c +1
-2@@ -1253,8 +1253,7 @@ static void ib_ucm_add_one(struct ib_device *device)dev_t base;struct ib_ucm_device *ucm_dev; -if (!device->alloc_ucontext || -rdma_node_get_transport (device->node_type) !=RDMA_TRANSPORT_IB ) +if (!device->alloc_ucontext || !rdma_ib_or_iboe (device,1 ))return ; ucm_dev =kzalloc (sizeof *ucm_dev,GFP_KERNEL );
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v4 14/27] IB/Verbs: Reform cma_acquire_dev()
In reply to Michael WangQuoted 31 lines · Michael Wang
Reform
cma_acquire_dev()with management helpers, introducecma_validate_port()to make the code more clean.Cc Steve Wise <[email protected]>Cc Tom Talpey <[email protected]>Cc Jason Gunthorpe <jgunthorpe-ePGOBjL8dl3ta4EC/59zMFaTQe2KTcn/@public.gmane.org>Cc Doug Ledford <[email protected]>Cc Ira Weiny <[email protected]>Cc Sean Hefty <[email protected]>Signed-off-by Michael Wang <[email protected]>--- drivers/infiniband/core/cma.c | 69 +++++++++++++++++++++++++------------------ 1 file changed, 41 insertions(+), 28 deletions(-)drivers/infiniband/core/cma.c +8
-0@@ -370,18 +370,36 @@ static int cma_translate_addr(struct sockaddr *addr, struct rdma_dev_addr *dev_areturn ret; } +static inline int cma_validate_port (struct ib_device *device,u8 port, +union ib_gid *gid,int dev_type) +{ +u8 found_port; +int ret = -ENODEV ; + +if ((dev_type ==ARPHRD_INFINIBAND ) && !rdma_tech_ib (device, port)) +return ret;
drivers/infiniband/core/cma.c static int cma_translate_addr(struct sockaddr *addr, struct rdma_dev_addr *dev_a · line 390
Quoted 10 lines
drivers/infiniband/core/cma.c +10
-0+ +if ((dev_type !=ARPHRD_INFINIBAND ) &&rdma_tech_ib (device, port)) +return ret; + + ret =ib_find_cached_gid (device, gid, &found_port,NULL ); + +if (!ret && (port == found_port)) +return 0 ; + +return ret;
Should the case where ret = 0 and port != found_port need to be handled the same as currently ? It looks different to me since in this case the port will be saved into id_priv->id.port_num whereas currently it isn't.
-- Hal
Quoted 70 lines
drivers/infiniband/core/cma.c +16
-28+} +static int cma_acquire_dev (struct rdma_id_private *id_priv,struct rdma_id_private *listen_id_priv) {struct rdma_dev_addr *dev_addr = &id_priv->id.route.addr.dev_addr;struct cma_device *cma_dev; -union ib_gid gid, iboe_gid; +union ib_gid gid, iboe_gid, *gidp;int ret = -ENODEV ; -u8 port, found_port; -enum rdma_link_layer dev_ll = dev_addr->dev_type ==ARPHRD_INFINIBAND ? -IB_LINK_LAYER_INFINIBAND :IB_LINK_LAYER_ETHERNET ; +u8 port; -if (dev_ll !=IB_LINK_LAYER_INFINIBAND && +if (dev_addr->dev_type !=ARPHRD_INFINIBAND && id_priv->id.ps ==RDMA_PS_IPOIB )return -EINVAL ;@@ -391,41 +409,36 @@ static int cma_acquire_dev(struct rdma_id_private *id_priv,memcpy (&gid, dev_addr->src_dev_addr +rdma_addr_gid_offset (dev_addr),sizeof gid); -if (listen_id_priv && -rdma_port_get_link_layer (listen_id_priv->id.device, - listen_id_priv->id.port_num) == dev_ll) { + +if (listen_id_priv) { cma_dev = listen_id_priv->cma_dev; port = listen_id_priv->id.port_num; -if (rdma_node_get_transport (cma_dev->device->node_type) ==RDMA_TRANSPORT_IB && -rdma_port_get_link_layer (cma_dev->device, port) ==IB_LINK_LAYER_ETHERNET ) - ret =ib_find_cached_gid (cma_dev->device, &iboe_gid, - &found_port,NULL ); -else - ret =ib_find_cached_gid (cma_dev->device, &gid, - &found_port,NULL ); + gidp =rdma_tech_iboe (cma_dev->device, port) ? + &iboe_gid : &gid; -if (!ret && (port == found_port)) { - id_priv->id.port_num = found_port; + ret =cma_validate_port (cma_dev->device, port, gidp, + dev_addr->dev_type); +if (!ret) { + id_priv->id.port_num = port; goto out; } } +list_for_each_entry (cma_dev, &dev_list, list) {for (port =1 ; port <= cma_dev->device->phys_port_cnt; ++port) {if (listen_id_priv && listen_id_priv->cma_dev == cma_dev && listen_id_priv->id.port_num == port)continue ; -if (rdma_port_get_link_layer (cma_dev->device, port) == dev_ll) { -if (rdma_node_get_transport (cma_dev->device->node_type) ==RDMA_TRANSPORT_IB && -rdma_port_get_link_layer (cma_dev->device, port) ==IB_LINK_LAYER_ETHERNET ) - ret =ib_find_cached_gid (cma_dev->device, &iboe_gid, &found_port,NULL ); -else - ret =ib_find_cached_gid (cma_dev->device, &gid, &found_port,NULL ); - -if (!ret && (port == found_port)) { - id_priv->id.port_num = found_port; - goto out; - } + + gidp = rdma_tech_iboe(cma_dev->device, port) ?
drivers/infiniband/core/cma.c static int cma_acquire_dev(struct rdma_id_private *id_priv, · line 444
Quoted 10 lines
drivers/infiniband/core/cma.c +7
-0+ &iboe_gid : &gid; + + ret =cma_validate_port (cma_dev->device, port, gidp, + dev_addr->dev_type); +if (!ret) { + id_priv->id.port_num = port; + goto out; } } }
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v4 10/27] IB/Verbs: Reform cm related part in IB-core cma/ucm
In reply to Hal RosenstockQuoted 8 lines · Hal Rosenstock
Quoted 6 lines · Michael Wang
Use raw management helpers to reform cm related part in IB-core cma/ucm.
These checks focus on the device cm type rather than the port capability, directly pass port 1 works currently, but can't support mixing cm type device in future.
Quoted 7 lines
This is equivalent to today where the checks are per node rather than per port.
Should all checks here be port 1 based or only certain ones like listen ? For example, in connect/reject/disconnect, don't we already have port ? Guess this can be dealt with later as this is not a regression from the current implementation.
Yeah, these parts of cma may need more carve in future, like some new callback for different CM type as Sean suggested.
Maybe directly using 1 could help to highlight the problem ;-)
Regards,
Michael Wanga
Quoted 236 lines
-- Hal
Quoted 233 lines
Cc Steve Wise <[email protected]>Cc Tom Talpey <[email protected]>Cc Jason Gunthorpe <jgunthorpe-ePGOBjL8dl3ta4EC/59zMFaTQe2KTcn/@public.gmane.org>Cc Doug Ledford <[email protected]>Cc Ira Weiny <[email protected]>Cc Sean Hefty <[email protected]>Signed-off-by Michael Wang <[email protected]>--- drivers/infiniband/core/cma.c | 81 +++++++++++++------------------------------ drivers/infiniband/core/ucm.c | 3 +- 2 files changed, 26 insertions(+), 58 deletions(-)drivers/infiniband/core/cma.c +25
-56@@ -735,8 +735,7 @@ int rdma_init_qp_attr(struct rdma_cm_id *id, struct ib_qp_attr *qp_attr,int ret =0 ; id_priv = container_of(id, struct rdma_id_private, id); -switch (rdma_node_get_transport (id_priv->id.device->node_type)) { -case RDMA_TRANSPORT_IB : +if (rdma_ib_or_iboe (id_priv->id.device,1 )) {if (!id_priv->cm_id.ib || (id_priv->id.qp_type ==IB_QPT_UD )) ret =cma_ib_init_qp_attr (id_priv, qp_attr, qp_attr_mask);else @@ -745,19 +744,15 @@ int rdma_init_qp_attr(struct rdma_cm_id *id, struct ib_qp_attr *qp_attr,if (qp_attr->qp_state ==IB_QPS_RTR ) qp_attr->rq_psn = id_priv->seq_num; -break ; -case RDMA_TRANSPORT_IWARP : + }else if (rdma_tech_iwarp (id_priv->id.device,1 )) {if (!id_priv->cm_id.iw) { qp_attr->qp_access_flags =0 ; *qp_attr_mask =IB_QP_STATE |IB_QP_ACCESS_FLAGS ; }else ret =iw_cm_init_qp_attr (id_priv->cm_id.iw, qp_attr, qp_attr_mask); -break ; -default : + }else ret = -ENOSYS ; -break ; - }return ret; }@@ -1037,17 +1032,12 @@ void rdma_destroy_id(struct rdma_cm_id *id)mutex_unlock (&id_priv->handler_mutex);if (id_priv->cma_dev) { -switch (rdma_node_get_transport (id_priv->id.device->node_type)) { -case RDMA_TRANSPORT_IB : +if (rdma_ib_or_iboe (id_priv->id.device,1 )) {if (id_priv->cm_id.ib)ib_destroy_cm_id (id_priv->cm_id.ib); -break ; -case RDMA_TRANSPORT_IWARP : + }else if (rdma_tech_iwarp (id_priv->id.device,1 )) {if (id_priv->cm_id.iw)iw_destroy_cm_id (id_priv->cm_id.iw); -break ; -default : -break ; }cma_leave_mc_groups (id_priv);cma_release_dev (id_priv);@@ -1626,7 +1616,7 @@ static void cma_listen_on_dev(struct rdma_id_private *id_priv,int ret;if (cma_family (id_priv) ==AF_IB && -rdma_node_get_transport (cma_dev->device->node_type) !=RDMA_TRANSPORT_IB ) + !rdma_ib_or_iboe (cma_dev->device,1 ))return ; id = rdma_create_id(cma_listen_handler, id_priv, id_priv->id.ps,@@ -2028,7 +2018,7 @@ static int cma_bind_loopback(struct rdma_id_private *id_priv)mutex_lock (&lock);list_for_each_entry (cur_dev, &dev_list, list) {if (cma_family (id_priv) ==AF_IB && -rdma_node_get_transport (cur_dev->device->node_type) !=RDMA_TRANSPORT_IB ) + !rdma_ib_or_iboe (cur_dev->device,1 ))continue ;if (!cma_dev)@@ -2060,7 +2050,7 @@ port_found: goto out; id_priv->id.route.addr.dev_addr.dev_type = - (rdma_port_get_link_layer (cma_dev->device, p) ==IB_LINK_LAYER_INFINIBAND ) ? + (rdma_tech_ib (cma_dev->device, p)) ?ARPHRD_INFINIBAND :ARPHRD_ETHER ;rdma_addr_set_sgid (&id_priv->id.route.addr.dev_addr, &gid);@@ -2537,18 +2527,15 @@ int rdma_listen(struct rdma_cm_id *id, int backlog) id_priv->backlog = backlog;if (id->device) { -switch (rdma_node_get_transport (id->device->node_type)) { -case RDMA_TRANSPORT_IB : +if (rdma_ib_or_iboe (id->device,1 )) { ret =cma_ib_listen (id_priv);if (ret) goto err; -break ; -case RDMA_TRANSPORT_IWARP : + }else if (rdma_tech_iwarp (id->device,1 )) { ret =cma_iw_listen (id_priv, backlog);if (ret) goto err; -break ; -default : + }else { ret = -ENOSYS ; goto err; }@@ -2884,20 +2871,15 @@ int rdma_connect(struct rdma_cm_id *id, struct rdma_conn_param *conn_param) id_priv->srq = conn_param->srq; } -switch (rdma_node_get_transport (id->device->node_type)) { -case RDMA_TRANSPORT_IB : +if (rdma_ib_or_iboe (id->device,1 )) {if (id->qp_type ==IB_QPT_UD ) ret =cma_resolve_ib_udp (id_priv, conn_param);else ret =cma_connect_ib (id_priv, conn_param); -break ; -case RDMA_TRANSPORT_IWARP : + }else if (rdma_tech_iwarp (id->device,1 )) ret =cma_connect_iw (id_priv, conn_param); -break ; -default : +else ret = -ENOSYS ; -break ; - }if (ret) goto err;@@ -3000,8 +2982,7 @@ int rdma_accept(struct rdma_cm_id *id, struct rdma_conn_param *conn_param) id_priv->srq = conn_param->srq; } -switch (rdma_node_get_transport (id->device->node_type)) { -case RDMA_TRANSPORT_IB : +if (rdma_ib_or_iboe (id->device,1 )) {if (id->qp_type ==IB_QPT_UD ) {if (conn_param) ret = cma_send_sidr_rep(id_priv, IB_SIDR_SUCCESS,@@ -3017,14 +2998,10 @@ int rdma_accept(struct rdma_cm_id *id, struct rdma_conn_param *conn_param)else ret =cma_rep_recv (id_priv); } -break ; -case RDMA_TRANSPORT_IWARP : + }else if (rdma_tech_iwarp (id->device,1 )) ret =cma_accept_iw (id_priv, conn_param); -break ; -default : +else ret = -ENOSYS ; -break ; - }if (ret) goto reject;@@ -3068,8 +3045,7 @@ int rdma_reject(struct rdma_cm_id *id, const void *private_data,if (!id_priv->cm_id.ib)return -EINVAL ; -switch (rdma_node_get_transport (id->device->node_type)) { -case RDMA_TRANSPORT_IB : +if (rdma_ib_or_iboe (id->device,1 )) {if (id->qp_type ==IB_QPT_UD ) ret =cma_send_sidr_rep (id_priv,IB_SIDR_REJECT ,0 , private_data, private_data_len);@@ -3077,15 +3053,12 @@ int rdma_reject(struct rdma_cm_id *id, const void *private_data, ret =ib_send_cm_rej (id_priv->cm_id.ib,IB_CM_REJ_CONSUMER_DEFINED ,NULL ,0 , private_data, private_data_len); -break ; -case RDMA_TRANSPORT_IWARP : + }else if (rdma_tech_iwarp (id->device,1 )) { ret =iw_cm_reject (id_priv->cm_id.iw, private_data, private_data_len); -break ; -default : + }else ret = -ENOSYS ; -break ; - } +return ret; }EXPORT_SYMBOL (rdma_reject);@@ -3099,22 +3072,18 @@ int rdma_disconnect(struct rdma_cm_id *id)if (!id_priv->cm_id.ib)return -EINVAL ; -switch (rdma_node_get_transport (id->device->node_type)) { -case RDMA_TRANSPORT_IB : +if (rdma_ib_or_iboe (id->device,1 )) { ret =cma_modify_qp_err (id_priv);if (ret) goto out;/* Initiate or respond to a disconnect. */ if (ib_send_cm_dreq (id_priv->cm_id.ib,NULL ,0 ))ib_send_cm_drep (id_priv->cm_id.ib,NULL ,0 ); -break ; -case RDMA_TRANSPORT_IWARP : + }else if (rdma_tech_iwarp (id->device,1 )) { ret =iw_cm_disconnect (id_priv->cm_id.iw,0 ); -break ; -default : + }else ret = -EINVAL ; -break ; - } + out:return ret; }drivers/infiniband/core/ucm.c +0
-0@@ -1253,8 +1253,7 @@ static void ib_ucm_add_one(struct ib_device *device)dev_t base;struct ib_ucm_device *ucm_dev;
Quoted 7 lines
drivers/infiniband/core/ucm.c static void ib_ucm_add_one(struct ib_device *device) · line 1259
Quoted 6 lines
drivers/infiniband/core/ucm.c +1
-2-if (!device->alloc_ucontext || -rdma_node_get_transport (device->node_type) !=RDMA_TRANSPORT_IB ) +if (!device->alloc_ucontext || !rdma_ib_or_iboe (device,1 ))return ; ucm_dev =kzalloc (sizeof *ucm_dev,GFP_KERNEL );
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v4 14/27] IB/Verbs: Reform cma_acquire_dev()
In reply to Hal RosenstockOn 04/16/2015 03:19 PM, Hal Rosenstock wrote:
[snip]
Quoted 15 lines
Quoted 15 lines
drivers/infiniband/core/cma.c +14
-0+static inline int cma_validate_port (struct ib_device *device,u8 port, +union ib_gid *gid,int dev_type) +{ +u8 found_port; +int ret = -ENODEV ; + +if ((dev_type ==ARPHRD_INFINIBAND ) && !rdma_tech_ib (device, port)) +return ret; + +if ((dev_type !=ARPHRD_INFINIBAND ) &&rdma_tech_ib (device, port)) +return ret; + + ret =ib_find_cached_gid (device, gid, &found_port,NULL ); +
Quoted 8 lines
drivers/infiniband/core/cma.c static int cma_translate_addr(struct sockaddr *addr, struct rdma_dev_addr *dev_a · line 390
Quoted 4 lines
drivers/infiniband/core/cma.c +4
-0+if (!ret && (port == found_port)) +return 0 ; + +return ret;Should the case where ret = 0 and port !=
found_portneed to be handled the same as currently ? It looks different to me since in this case the port will be saved intoid_priv->id.port_numwhereas currently it isn't.
I get your point :-) what about:
ret = ib_find_cached_gid(device, gid, &found_port, NULL); if (port != found_port) return -ENODEV; return ret
Regards,
Michael Wang
Quoted 75 lines
-- Hal
Quoted 72 lines
drivers/infiniband/core/cma.c +18
-28+} +static int cma_acquire_dev (struct rdma_id_private *id_priv,struct rdma_id_private *listen_id_priv) {struct rdma_dev_addr *dev_addr = &id_priv->id.route.addr.dev_addr;struct cma_device *cma_dev; -union ib_gid gid, iboe_gid; +union ib_gid gid, iboe_gid, *gidp;int ret = -ENODEV ; -u8 port, found_port; -enum rdma_link_layer dev_ll = dev_addr->dev_type ==ARPHRD_INFINIBAND ? -IB_LINK_LAYER_INFINIBAND :IB_LINK_LAYER_ETHERNET ; +u8 port; -if (dev_ll !=IB_LINK_LAYER_INFINIBAND && +if (dev_addr->dev_type !=ARPHRD_INFINIBAND && id_priv->id.ps ==RDMA_PS_IPOIB )return -EINVAL ;@@ -391,41 +409,36 @@ static int cma_acquire_dev(struct rdma_id_private *id_priv,memcpy (&gid, dev_addr->src_dev_addr +rdma_addr_gid_offset (dev_addr),sizeof gid); -if (listen_id_priv && -rdma_port_get_link_layer (listen_id_priv->id.device, - listen_id_priv->id.port_num) == dev_ll) { + +if (listen_id_priv) { cma_dev = listen_id_priv->cma_dev; port = listen_id_priv->id.port_num; -if (rdma_node_get_transport (cma_dev->device->node_type) ==RDMA_TRANSPORT_IB && -rdma_port_get_link_layer (cma_dev->device, port) ==IB_LINK_LAYER_ETHERNET ) - ret =ib_find_cached_gid (cma_dev->device, &iboe_gid, - &found_port,NULL ); -else - ret =ib_find_cached_gid (cma_dev->device, &gid, - &found_port,NULL ); + gidp =rdma_tech_iboe (cma_dev->device, port) ? + &iboe_gid : &gid; -if (!ret && (port == found_port)) { - id_priv->id.port_num = found_port; + ret =cma_validate_port (cma_dev->device, port, gidp, + dev_addr->dev_type); +if (!ret) { + id_priv->id.port_num = port; goto out; } } +list_for_each_entry (cma_dev, &dev_list, list) {for (port =1 ; port <= cma_dev->device->phys_port_cnt; ++port) {if (listen_id_priv && listen_id_priv->cma_dev == cma_dev && listen_id_priv->id.port_num == port)continue ; -if (rdma_port_get_link_layer (cma_dev->device, port) == dev_ll) { -if (rdma_node_get_transport (cma_dev->device->node_type) ==RDMA_TRANSPORT_IB && -rdma_port_get_link_layer (cma_dev->device, port) ==IB_LINK_LAYER_ETHERNET ) - ret =ib_find_cached_gid (cma_dev->device, &iboe_gid, &found_port,NULL ); -else - ret =ib_find_cached_gid (cma_dev->device, &gid, &found_port,NULL ); - -if (!ret && (port == found_port)) { - id_priv->id.port_num = found_port; - goto out; - } + + gidp =rdma_tech_iboe (cma_dev->device, port) ? + &iboe_gid : &gid; +
Quoted 9 lines
drivers/infiniband/core/cma.c static int cma_acquire_dev(struct rdma_id_private *id_priv, · line 444
Quoted 8 lines
drivers/infiniband/core/cma.c +5
-0+ ret =cma_validate_port (cma_dev->device, port, gidp, + dev_addr->dev_type); +if (!ret) { + id_priv->id.port_num = port; + goto out; } } }
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v4 20/27] IB/Verbs: Use management helper cap_ib_sa()
In reply to Michael WangQuoted 3 lines · Michael Wang
Introduce helper
cap_ib_sa()to help us check if the port of an IB device support Infiniband Subnet Administrator.
Nit: Administrator -> Administration
Quoted 104 lines
Cc Steve Wise <[email protected]>Cc Tom Talpey <[email protected]>Cc Jason Gunthorpe <jgunthorpe-ePGOBjL8dl3ta4EC/59zMFaTQe2KTcn/@public.gmane.org>Cc Doug Ledford <[email protected]>Cc Ira Weiny <[email protected]>Cc Sean Hefty <[email protected]>Signed-off-by Michael Wang <[email protected]>--- drivers/infiniband/core/cma.c | 4 ++-- drivers/infiniband/core/sa_query.c | 10 +++++----- drivers/infiniband/core/ucma.c | 2 +- include/rdma/ib_verbs.h | 15 +++++++++++++++ 4 files changed, 23 insertions(+), 8 deletions(-)drivers/infiniband/core/cma.c +2
-2@@ -955,7 +955,7 @@ static inline int cma_user_data_offset(struct rdma_id_private *id_priv)static void cma_cancel_route (struct rdma_id_private *id_priv) { -if (rdma_tech_ib (id_priv->id.device, id_priv->id.port_num)) { +if (cap_ib_sa (id_priv->id.device, id_priv->id.port_num)) {if (id_priv->query)ib_sa_cancel_query (id_priv->query_id, id_priv->query); }@@ -1979,7 +1979,7 @@ int rdma_resolve_route(struct rdma_cm_id *id, int timeout_ms)return -EINVAL ;atomic_inc (&id_priv->refcount); -if (rdma_tech_ib (id->device, id->port_num)) +if (cap_ib_sa (id->device, id->port_num)) ret =cma_resolve_ib_route (id_priv, timeout_ms);else if (rdma_tech_iboe (id->device, id->port_num)) ret =cma_resolve_iboe_route (id_priv);drivers/infiniband/core/sa_query.c +5
-5@@ -450,7 +450,7 @@ static void ib_sa_event(struct ib_event_handler *handler, struct ib_event *eventstruct ib_sa_port *port = &sa_dev->port[event->element.port_num - sa_dev->start_port]; -if (WARN_ON (!rdma_tech_ib (handler->device, port->port_num))) +if (WARN_ON (!cap_ib_sa (handler->device, port->port_num)))return ;spin_lock_irqsave (&port->ah_lock, flags);@@ -1173,7 +1173,7 @@ static void ib_sa_add_one(struct ib_device *device)for (i =0 ; i <= e - s; ++i) {spin_lock_init (&sa_dev->port[i].ah_lock); -if (!rdma_tech_ib (device, i +1 )) +if (!cap_ib_sa (device, i +1 ))continue ; sa_dev->port[i].sm_ah =NULL ;@@ -1210,7 +1210,7 @@ static void ib_sa_add_one(struct ib_device *device) goto err;for (i =0 ; i <= e - s; ++i) { -if (rdma_tech_ib (device, i +1 )) +if (cap_ib_sa (device, i +1 ))update_sm_ah (&sa_dev->port[i].update_task); }@@ -1218,7 +1218,7 @@ static void ib_sa_add_one(struct ib_device *device) err:while (--i >=0 ) { -if (rdma_tech_ib (device, i +1 )) +if (cap_ib_sa (device, i +1 ))ib_unregister_mad_agent (sa_dev->port[i].agent); }@@ -1240,7 +1240,7 @@ static void ib_sa_remove_one(struct ib_device *device)flush_workqueue (ib_wq);for (i =0 ; i <= sa_dev->end_port - sa_dev->start_port; ++i) { -if (rdma_tech_ib (device, i +1 )) { +if (cap_ib_sa (device, i +1 )) {ib_unregister_mad_agent (sa_dev->port[i].agent);if (sa_dev->port[i].sm_ah)kref_put (&sa_dev->port[i].sm_ah->ref, free_sm_ah);drivers/infiniband/core/ucma.c +1
-1@@ -723,7 +723,7 @@ static ssize_t ucma_query_route(struct ucma_file *file, resp.node_guid = (__force __u64 ) ctx->cm_id->device->node_guid; resp.port_num = ctx->cm_id->port_num; -if (rdma_tech_ib (ctx->cm_id->device, ctx->cm_id->port_num)) +if (cap_ib_sa (ctx->cm_id->device, ctx->cm_id->port_num))ucma_copy_ib_route (&resp, &ctx->cm_id->route);else if (rdma_tech_iboe (ctx->cm_id->device, ctx->cm_id->port_num))ucma_copy_iboe_route (&resp, &ctx->cm_id->route);include/rdma/ib_verbs.h +0
-0
include/rdma/ib_verbs.h static inline int cap_iw_cm(struct ib_device *device, u8 port_num) · line 1838
Quoted 6 lines
include/rdma/ib_verbs.h +3
-0@@ -1833,6 +1833,21 @@ static inline int cap_iw_cm(struct ib_device *device, u8 port_num)return rdma_tech_iwarp (device, port_num); } +/** +* cap_ib_sa - Check if the port of device has the capability Infiniband +* Subnet Administrator.
Nit: Administrator -> Administration
include/rdma/ib_verbs.h static inline int cap_iw_cm(struct ib_device *device, u8 port_num) · line 1844
Quoted 6 lines
include/rdma/ib_verbs.h +6
-0+* +* @device: Device to be checked +* @port_num: Port number of the device +* +* Return 0 when port of the device don't support Infiniband +* Subnet Administrator.
Nit: Administrator -> Administration
include/rdma/ib_verbs.h static inline int cap_iw_cm(struct ib_device *device, u8 port_num) · line 1853
Quoted 9 lines
include/rdma/ib_verbs.h +6
-0+*/ +static inline int cap_ib_sa (struct ib_device *device,u8 port_num) +{ +return rdma_tech_ib (device, port_num); +} +int ib_query_gid (struct ib_device *device,u8 port_num,int index,union ib_gid *gid);
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v4 27/27] IB/Verbs: Cleanup rdma_node_get_transport()
In reply to Michael WangQuoted 53 lines · Michael Wang
We have get rid of all the scene using legacy
rdma_node_get_transport(), now clean it up.Cc Steve Wise <[email protected]>Cc Tom Talpey <[email protected]>Cc Jason Gunthorpe <jgunthorpe-ePGOBjL8dl3ta4EC/59zMFaTQe2KTcn/@public.gmane.org>Cc Doug Ledford <[email protected]>Cc Ira Weiny <[email protected]>Cc Sean Hefty <[email protected]>Signed-off-by Michael Wang <[email protected]>--- drivers/infiniband/core/verbs.c | 21 --------------------- include/rdma/ib_verbs.h | 3 --- 2 files changed, 24 deletions(-)drivers/infiniband/core/verbs.c +0
-21@@ -107,27 +107,6 @@ __attribute_const__ int ib_rate_to_mbps(enum ib_rate rate) }EXPORT_SYMBOL (ib_rate_to_mbps); -__attribute_const__ enum rdma_transport_type -rdma_node_get_transport (enum rdma_node_type node_type) -{ -switch (node_type) { -case RDMA_NODE_IB_CA : -case RDMA_NODE_IB_SWITCH : -case RDMA_NODE_IB_ROUTER : -return RDMA_TRANSPORT_IB ; -case RDMA_NODE_RNIC : -return RDMA_TRANSPORT_IWARP ; -case RDMA_NODE_USNIC : -return RDMA_TRANSPORT_USNIC ; -case RDMA_NODE_USNIC_UDP : -return RDMA_TRANSPORT_USNIC_UDP ; -default : -BUG (); -return 0 ; - } -} -EXPORT_SYMBOL (rdma_node_get_transport); -enum rdma_link_layer rdma_port_get_link_layer (struct ib_device *device,u8 port_num) {if (device->get_link_layer)include/rdma/ib_verbs.h +0
-0
include/rdma/ib_verbs.h enum rdma_transport_type { · line 88
Quoted 8 lines
include/rdma/ib_verbs.h +0
-3@@ -84,9 +84,6 @@ enum rdma_transport_type { RDMA_TRANSPORT_IBOE, }; -__attribute_const__ enum rdma_transport_type -rdma_node_get_transport (enum rdma_node_type node_type); -enum rdma_link_layer {IB_LINK_LAYER_UNSPECIFIED ,
Is IB_LINK_LAYER_UNSPECIFIED still possible ?
include/rdma/ib_verbs.h enum rdma_transport_type { · line 89
Quoted 1 line
include/rdma/ib_verbs.h +0
-0IB_LINK_LAYER_INFINIBAND,
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v4 20/27] IB/Verbs: Use management helper cap_ib_sa()
In reply to Hal RosenstockQuoted 6 lines · Hal Rosenstock
Quoted 3 lines · Michael Wang
Introduce helper
cap_ib_sa()to help us check if the port of an IB device support Infiniband Subnet Administrator.Nit: Administrator -> Administration
Will be in next version :-)
Regards,
Michael Wang
Quoted 123 lines
Quoted 105 lines
Cc Steve Wise <[email protected]>Cc Tom Talpey <[email protected]>Cc Jason Gunthorpe <jgunthorpe-ePGOBjL8dl3ta4EC/59zMFaTQe2KTcn/@public.gmane.org>Cc Doug Ledford <[email protected]>Cc Ira Weiny <[email protected]>Cc Sean Hefty <[email protected]>Signed-off-by Michael Wang <[email protected]>--- drivers/infiniband/core/cma.c | 4 ++-- drivers/infiniband/core/sa_query.c | 10 +++++----- drivers/infiniband/core/ucma.c | 2 +- include/rdma/ib_verbs.h | 15 +++++++++++++++ 4 files changed, 23 insertions(+), 8 deletions(-)drivers/infiniband/core/cma.c +2
-2@@ -955,7 +955,7 @@ static inline int cma_user_data_offset(struct rdma_id_private *id_priv)static void cma_cancel_route (struct rdma_id_private *id_priv) { -if (rdma_tech_ib (id_priv->id.device, id_priv->id.port_num)) { +if (cap_ib_sa (id_priv->id.device, id_priv->id.port_num)) {if (id_priv->query)ib_sa_cancel_query (id_priv->query_id, id_priv->query); }@@ -1979,7 +1979,7 @@ int rdma_resolve_route(struct rdma_cm_id *id, int timeout_ms)return -EINVAL ;atomic_inc (&id_priv->refcount); -if (rdma_tech_ib (id->device, id->port_num)) +if (cap_ib_sa (id->device, id->port_num)) ret =cma_resolve_ib_route (id_priv, timeout_ms);else if (rdma_tech_iboe (id->device, id->port_num)) ret =cma_resolve_iboe_route (id_priv);drivers/infiniband/core/sa_query.c +5
-5@@ -450,7 +450,7 @@ static void ib_sa_event(struct ib_event_handler *handler, struct ib_event *eventstruct ib_sa_port *port = &sa_dev->port[event->element.port_num - sa_dev->start_port]; -if (WARN_ON (!rdma_tech_ib (handler->device, port->port_num))) +if (WARN_ON (!cap_ib_sa (handler->device, port->port_num)))return ;spin_lock_irqsave (&port->ah_lock, flags);@@ -1173,7 +1173,7 @@ static void ib_sa_add_one(struct ib_device *device)for (i =0 ; i <= e - s; ++i) {spin_lock_init (&sa_dev->port[i].ah_lock); -if (!rdma_tech_ib (device, i +1 )) +if (!cap_ib_sa (device, i +1 ))continue ; sa_dev->port[i].sm_ah =NULL ;@@ -1210,7 +1210,7 @@ static void ib_sa_add_one(struct ib_device *device) goto err;for (i =0 ; i <= e - s; ++i) { -if (rdma_tech_ib (device, i +1 )) +if (cap_ib_sa (device, i +1 ))update_sm_ah (&sa_dev->port[i].update_task); }@@ -1218,7 +1218,7 @@ static void ib_sa_add_one(struct ib_device *device) err:while (--i >=0 ) { -if (rdma_tech_ib (device, i +1 )) +if (cap_ib_sa (device, i +1 ))ib_unregister_mad_agent (sa_dev->port[i].agent); }@@ -1240,7 +1240,7 @@ static void ib_sa_remove_one(struct ib_device *device)flush_workqueue (ib_wq);for (i =0 ; i <= sa_dev->end_port - sa_dev->start_port; ++i) { -if (rdma_tech_ib (device, i +1 )) { +if (cap_ib_sa (device, i +1 )) {ib_unregister_mad_agent (sa_dev->port[i].agent);if (sa_dev->port[i].sm_ah)kref_put (&sa_dev->port[i].sm_ah->ref, free_sm_ah);drivers/infiniband/core/ucma.c +1
-1@@ -723,7 +723,7 @@ static ssize_t ucma_query_route(struct ucma_file *file, resp.node_guid = (__force __u64 ) ctx->cm_id->device->node_guid; resp.port_num = ctx->cm_id->port_num; -if (rdma_tech_ib (ctx->cm_id->device, ctx->cm_id->port_num)) +if (cap_ib_sa (ctx->cm_id->device, ctx->cm_id->port_num))ucma_copy_ib_route (&resp, &ctx->cm_id->route);else if (rdma_tech_iboe (ctx->cm_id->device, ctx->cm_id->port_num))ucma_copy_iboe_route (&resp, &ctx->cm_id->route);include/rdma/ib_verbs.h +0
-0include/rdma/ib_verbs.h static inline int cap_iw_cm(struct ib_device *device, u8 port_num) · line 1838
Quoted 6 lines
include/rdma/ib_verbs.h +3
-0@@ -1833,6 +1833,21 @@ static inline int cap_iw_cm(struct ib_device *device, u8 port_num)return rdma_tech_iwarp (device, port_num); } +/** +* cap_ib_sa - Check if the port of device has the capability Infiniband +* Subnet Administrator. Nit: Administrator -> Administration
include/rdma/ib_verbs.h static inline int cap_iw_cm(struct ib_device *device, u8 port_num) · line 1844
Quoted 6 lines
include/rdma/ib_verbs.h +6
-0+* +* @device: Device to be checked +* @port_num: Port number of the device +* +* Return 0 when port of the device don't support Infiniband +* Subnet Administrator. Nit: Administrator -> Administration
Quoted 10 lines
include/rdma/ib_verbs.h static inline int cap_iw_cm(struct ib_device *device, u8 port_num) · line 1853
Quoted 9 lines
include/rdma/ib_verbs.h +6
-0+*/ +static inline int cap_ib_sa (struct ib_device *device,u8 port_num) +{ +return rdma_tech_ib (device, port_num); +} +int ib_query_gid (struct ib_device *device,u8 port_num,int index,union ib_gid *gid);
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v4 27/27] IB/Verbs: Cleanup rdma_node_get_transport()
In reply to Hal RosenstockOn 04/16/2015 03:36 PM, Hal Rosenstock wrote:
[snip]
Quoted 13 lines
Quoted 13 lines
drivers/infiniband/core/verbs.c +0
-2-EXPORT_SYMBOL (rdma_node_get_transport); -enum rdma_link_layer rdma_port_get_link_layer (struct ib_device *device,u8 port_num) {if (device->get_link_layer)include/rdma/ib_verbs.h +0
-0@@ -84,9 +84,6 @@ enum rdma_transport_type { RDMA_TRANSPORT_IBOE, };
Quoted 7 lines
include/rdma/ib_verbs.h enum rdma_transport_type { · line 88
Quoted 5 lines
include/rdma/ib_verbs.h +0
-3-__attribute_const__ enum rdma_transport_type -rdma_node_get_transport (enum rdma_node_type node_type); -enum rdma_link_layer {IB_LINK_LAYER_UNSPECIFIED ,Is
IB_LINK_LAYER_UNSPECIFIEDstill possible ?
Actually it's impossible in kernel at first, all those who implemented the callback won't return UNSPECIFIED, others all have the correct transport type (otherwise BUG()) and won't result UNSPECIFIED :-)
Regards,
Michael Wang
Quoted 2 lines
include/rdma/ib_verbs.h enum rdma_transport_type { · line 89
Quoted 2 lines
include/rdma/ib_verbs.h +0
-0IB_LINK_LAYER_INFINIBAND,
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v4 14/27] IB/Verbs: Reform cma_acquire_dev()
In reply to Michael WangQuoted 28 lines · Michael Wang
On 04/16/2015 03:19 PM, Hal Rosenstock wrote:
[snip]Quoted 15 lines
Quoted 15 lines
drivers/infiniband/core/cma.c +14
-0+static inline int cma_validate_port (struct ib_device *device,u8 port, +union ib_gid *gid,int dev_type) +{ +u8 found_port; +int ret = -ENODEV ; + +if ((dev_type ==ARPHRD_INFINIBAND ) && !rdma_tech_ib (device, port)) +return ret; + +if ((dev_type !=ARPHRD_INFINIBAND ) &&rdma_tech_ib (device, port)) +return ret; + + ret =ib_find_cached_gid (device, gid, &found_port,NULL ); +Quoted 8 lines
drivers/infiniband/core/cma.c static int cma_translate_addr(struct sockaddr *addr, struct rdma_dev_addr *dev_a · line 390
Quoted 4 lines
drivers/infiniband/core/cma.c +4
-0+if (!ret && (port == found_port)) +return 0 ; + +return ret;Should the case where ret = 0 and port !=
found_portneed to be handled the same as currently ? It looks different to me since in this case the port will be saved intoid_priv->id.port_numwhereas currently it isn't.
Quoted 7 lines
I get your point :-) what about:
ret = ib_find_cached_gid(device, gid, &found_port, NULL); if (port != found_port) return -ENODEV; return ret;
Yes, that looks to me to be consistent with the current implementation.
-- Hal
Quoted 88 lines
Regards,
Michael WangQuoted 84 lines
-- Hal
Quoted 80 lines
drivers/infiniband/core/cma.c +23
-28+} +static int cma_acquire_dev (struct rdma_id_private *id_priv,struct rdma_id_private *listen_id_priv) {struct rdma_dev_addr *dev_addr = &id_priv->id.route.addr.dev_addr;struct cma_device *cma_dev; -union ib_gid gid, iboe_gid; +union ib_gid gid, iboe_gid, *gidp;int ret = -ENODEV ; -u8 port, found_port; -enum rdma_link_layer dev_ll = dev_addr->dev_type ==ARPHRD_INFINIBAND ? -IB_LINK_LAYER_INFINIBAND :IB_LINK_LAYER_ETHERNET ; +u8 port; -if (dev_ll !=IB_LINK_LAYER_INFINIBAND && +if (dev_addr->dev_type !=ARPHRD_INFINIBAND && id_priv->id.ps ==RDMA_PS_IPOIB )return -EINVAL ;@@ -391,41 +409,36 @@ static int cma_acquire_dev(struct rdma_id_private *id_priv,memcpy (&gid, dev_addr->src_dev_addr +rdma_addr_gid_offset (dev_addr),sizeof gid); -if (listen_id_priv && -rdma_port_get_link_layer (listen_id_priv->id.device, - listen_id_priv->id.port_num) == dev_ll) { + +if (listen_id_priv) { cma_dev = listen_id_priv->cma_dev; port = listen_id_priv->id.port_num; -if (rdma_node_get_transport (cma_dev->device->node_type) ==RDMA_TRANSPORT_IB && -rdma_port_get_link_layer (cma_dev->device, port) ==IB_LINK_LAYER_ETHERNET ) - ret =ib_find_cached_gid (cma_dev->device, &iboe_gid, - &found_port,NULL ); -else - ret =ib_find_cached_gid (cma_dev->device, &gid, - &found_port,NULL ); + gidp =rdma_tech_iboe (cma_dev->device, port) ? + &iboe_gid : &gid; -if (!ret && (port == found_port)) { - id_priv->id.port_num = found_port; + ret =cma_validate_port (cma_dev->device, port, gidp, + dev_addr->dev_type); +if (!ret) { + id_priv->id.port_num = port; goto out; } } +list_for_each_entry (cma_dev, &dev_list, list) {for (port =1 ; port <= cma_dev->device->phys_port_cnt; ++port) {if (listen_id_priv && listen_id_priv->cma_dev == cma_dev && listen_id_priv->id.port_num == port)continue ; -if (rdma_port_get_link_layer (cma_dev->device, port) == dev_ll) { -if (rdma_node_get_transport (cma_dev->device->node_type) ==RDMA_TRANSPORT_IB && -rdma_port_get_link_layer (cma_dev->device, port) ==IB_LINK_LAYER_ETHERNET ) - ret =ib_find_cached_gid (cma_dev->device, &iboe_gid, &found_port,NULL ); -else - ret =ib_find_cached_gid (cma_dev->device, &gid, &found_port,NULL ); - -if (!ret && (port == found_port)) { - id_priv->id.port_num = found_port; - goto out; - } + + gidp =rdma_tech_iboe (cma_dev->device, port) ? + &iboe_gid : &gid; + + ret =cma_validate_port (cma_dev->device, port, gidp, + dev_addr->dev_type); +if (!ret) { + id_priv->id.port_num = port; + goto out; } } }
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v4 27/27] IB/Verbs: Cleanup rdma_node_get_transport()
In reply to Michael WangQuoted 25 lines · Michael Wang
On 04/16/2015 03:36 PM, Hal Rosenstock wrote:
[snip]Quoted 13 lines
Quoted 13 lines
drivers/infiniband/core/verbs.c +0
-2-EXPORT_SYMBOL (rdma_node_get_transport); -enum rdma_link_layer rdma_port_get_link_layer (struct ib_device *device,u8 port_num) {if (device->get_link_layer)include/rdma/ib_verbs.h +0
-0@@ -84,9 +84,6 @@ enum rdma_transport_type { RDMA_TRANSPORT_IBOE, };Quoted 7 lines
include/rdma/ib_verbs.h enum rdma_transport_type { · line 88
Quoted 5 lines
include/rdma/ib_verbs.h +0
-3-__attribute_const__ enum rdma_transport_type -rdma_node_get_transport (enum rdma_node_type node_type); -enum rdma_link_layer {IB_LINK_LAYER_UNSPECIFIED ,Is
IB_LINK_LAYER_UNSPECIFIEDstill possible ?
Quoted 3 lines
Actually it's impossible in kernel at first, all those who implemented the callback won't return UNSPECIFIED, others all have the correct transport type (otherwise
BUG()) and won't result UNSPECIFIED :-)
Should it be removed from this enum somewhere in this patch series (perhaps early on) ?
-- Hal
Quoted 6 lines
Regards,
Michael WangQuoted 2 lines
Quoted 2 lines
include/rdma/ib_verbs.h +0
-0IB_LINK_LAYER_INFINIBAND,
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v4 27/27] IB/Verbs: Cleanup rdma_node_get_transport()
Quoted 26 lines · Hal Rosenstock
Quoted 25 lines · Michael Wang
On 04/16/2015 03:36 PM, Hal Rosenstock wrote:
[snip]Quoted 20 lines
Quoted 10 lines
drivers/infiniband/core/verbs.c +0
-2-EXPORT_SYMBOL (rdma_node_get_transport); -enum rdma_link_layer rdma_port_get_link_layer (struct ib_device *device,u8 port_num) {if (device->get_link_layer)include/rdma/ib_verbs.h +0
-0include/rdma/ib_verbs.h enum rdma_transport_type { · line 88
Quoted 8 lines
include/rdma/ib_verbs.h +0
-3@@ -84,9 +84,6 @@ enum rdma_transport_type { RDMA_TRANSPORT_IBOE, }; -__attribute_const__ enum rdma_transport_type -rdma_node_get_transport (enum rdma_node_type node_type); -enum rdma_link_layer {IB_LINK_LAYER_UNSPECIFIED ,Is
IB_LINK_LAYER_UNSPECIFIEDstill possible ?
Quoted 6 lines
Quoted 3 lines
Actually it's impossible in kernel at first, all those who implemented the callback won't return UNSPECIFIED, others all have the correct transport type (otherwise
BUG()) and won't result UNSPECIFIED :-)Should it be removed from this enum somewhere in this patch series (perhaps early on) ?
It was still directly used by helper like ib_modify_qp_is_ok() as indicator, may be better in another following patch to reform those part :-)
Regards,
Michael Wang
Quoted 10 lines
-- Hal
Quoted 6 lines
Regards,
Michael WangQuoted 2 lines
Quoted 2 lines
include/rdma/ib_verbs.h +0
-0IB_LINK_LAYER_INFINIBAND,
Re: [PATCH v4 27/27] IB/Verbs: Cleanup rdma_node_get_transport()
Quoted 25 lines · Michael Wang
Quoted 23 lines · Hal Rosenstock
Quoted 22 lines · Michael Wang
On 04/16/2015 03:36 PM, Hal Rosenstock wrote:
[snip]Quoted 19 lines
Quoted 10 lines
drivers/infiniband/core/verbs.c +0
-2-EXPORT_SYMBOL (rdma_node_get_transport); -enum rdma_link_layer rdma_port_get_link_layer (struct ib_device *device,u8 port_num) {if (device->get_link_layer)include/rdma/ib_verbs.h +0
-0include/rdma/ib_verbs.h enum rdma_transport_type { · line 88
Quoted 8 lines
include/rdma/ib_verbs.h +0
-3@@ -84,9 +84,6 @@ enum rdma_transport_type { RDMA_TRANSPORT_IBOE, }; -__attribute_const__ enum rdma_transport_type -rdma_node_get_transport (enum rdma_node_type node_type); -enum rdma_link_layer {IB_LINK_LAYER_UNSPECIFIED ,Is
IB_LINK_LAYER_UNSPECIFIEDstill possible ?
Quoted 5 lines
Quoted 5 lines
Quoted 3 lines
Actually it's impossible in kernel at first, all those who implemented the callback won't return UNSPECIFIED, others all have the correct transport type (otherwise
BUG()) and won't result UNSPECIFIED :-)Should it be removed from this enum somewhere in this patch series (perhaps early on) ?
I don't think it's ever been 'possible.' It's purpose is to catch initialized errors where the transport fails to initialize it's transport type. So for example,
provider = calloc(1, sizeof *provider)
If 0 is a valid link layer type, then you wouldn't catch these kinds of errors.
Tom
Quoted 12 lines
It was still directly used by helper like
ib_modify_qp_is_ok()as indicator, may be better in another following patch to reform those part :-)Regards,
Michael WangQuoted 6 lines
-- Hal
Quoted 4 lines
Regards,
Michael WangQuoted 1 line
Quoted 1 line
include/rdma/ib_verbs.h +0
-0IB_LINK_LAYER_INFINIBAND,
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
RE: [PATCH v4 10/27] IB/Verbs: Reform cm related part in IB-core cma/ucm
In reply to Michael WangPiA+IFRoaXMgaXMgZXF1aXZhbGVudCB0byB0b2RheSB3aGVyZSB0aGUgY2hlY2tzIGFyZSBwZXIg bm9kZSByYXRoZXIgdGhhbg0KPiA+IHBlciBwb3J0Lg0KPiA+DQo+ID4gU2hvdWxkIGFsbCBjaGVj a3MgaGVyZSBiZSBwb3J0IDEgYmFzZWQgb3Igb25seSBjZXJ0YWluIG9uZXMgbGlrZSBsaXN0ZW4N Cj4gPiA/IEZvciBleGFtcGxlLCBpbiBjb25uZWN0L3JlamVjdC9kaXNjb25uZWN0LCBkb24ndCB3 ZSBhbHJlYWR5IGhhdmUgcG9ydA0KPiA+ID8gR3Vlc3MgdGhpcyBjYW4gYmUgZGVhbHQgd2l0aCBs YXRlciBhcyB0aGlzIGlzIG5vdCBhIHJlZ3Jlc3Npb24gZnJvbQ0KPiA+IHRoZSBjdXJyZW50IGlt cGxlbWVudGF0aW9uLg0KPiANCj4gWWVhaCwgdGhlc2UgcGFydHMgb2YgY21hIG1heSBuZWVkIG1v cmUgY2FydmUgaW4gZnV0dXJlLCBsaWtlIHNvbWUgbmV3DQo+IGNhbGxiYWNrDQo+IGZvciBkaWZm ZXJlbnQgQ00gdHlwZSBhcyBTZWFuIHN1Z2dlc3RlZC4NCj4gDQo+IE1heWJlIGRpcmVjdGx5IHVz aW5nIDEgY291bGQgaGVscCB0byBoaWdobGlnaHQgdGhlIHByb2JsZW0gOy0pDQoNCk9ubHkgYSBm ZXcgY2hlY2tzIG5lZWQgdG8gYmUgcGVyIGRldmljZS4gIEkgdGhpbmsgSSBwb2ludGVkIHRob3Nl IG91dCBwcmV2aW91c2x5LiAgVGVzdGluZyBzaG91bGQgc2hvdyBhbnl3aGVyZSB0aGF0IHdlIG1p c3MgZmFpcmx5IHF1aWNrbHksIHNpbmNlIHBvcnQgd291bGQgc3RpbGwgYmUgMC4gIEZvciB0aGUg Y2hlY2tzIHRoYXQgY2FuIGJlIHVwZGF0ZWQgdG8gYmUgcGVyIHBvcnQsIEkgd291bGQgcmF0aGVy IGdvIGFoZWFkIGFuZCBjb252ZXJ0IHRoZW0uDQoNCi0gU2Vhbg0K
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v4 10/27] IB/Verbs: Reform cm related part in IB-core cma/ucm
Quoted 8 lines · Sean
Quoted 8 lines
Quoted 7 lines
This is equivalent to today where the checks are per node rather than per port.
Should all checks here be port 1 based or only certain ones like listen ? For example, in connect/reject/disconnect, don't we already have port ? Guess this can be dealt with later as this is not a regression from the current implementation.
Quoted 7 lines
Quoted 5 lines
Yeah, these parts of cma may need more carve in future, like some new callback
for different CM type as Sean suggested.Maybe directly using 1 could help to highlight the problem ;-)
Only a few checks need to be per device. I think I pointed those out previously. Testing should show anywhere that we miss fairly quickly, since port would still be 0. For the checks that can be updated to be per port, I would rather go ahead and convert them.
Got it, will be changed in next version :-)
To be confirmed:
PORT ASSIGNED rdma_init_qp_attr Y rdma_destroy_id unknown cma_listen_on_dev N cma_bind_loopback N rdma_listen N rdma_connect Y rdma_accept Y rdma_reject Y rdma_disconnect Y ib_ucm_add_one N
Is this list correct?
Regards,
Michael Wang
Quoted 3 lines
- Sean
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v4 10/27] IB/Verbs: Reform cm related part in IB-core cma/ucm
In reply to Michael WangQuoted 6 lines · Michael Wang
Use raw management helpers to reform cm related part in IB-core cma/ucm.
These checks focus on the device cm type rather than the port capability, directly pass port 1 works currently, but can't support mixing cm type device in future.
After the discussion settled, I ended up thinking that implementing explicit device checks, for use by CM, and the BUG_ON at register to require all ports have the same value was the best option.
It also looks like hardwired 1 won't work on switch ports, so it is no-go.
Jason
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
RE: [PATCH v4 10/27] IB/Verbs: Reform cm related part in IB-core cma/ucm
Quoted 3 lines
After the discussion settled, I ended up thinking that implementing explicit device checks, for use by CM, and the
BUG_ONat register to require all ports have the same value was the best option.
Sure, but why not update the other areas anyway? This way when listens become per port, rather than per device, we only need to update that portion of the code.
Re: [PATCH v4 10/27] IB/Verbs: Reform cm related part in IB-core cma/ucm
Quoted 7 lines · Sean
Quoted 3 lines
After the discussion settled, I ended up thinking that implementing explicit device checks, for use by CM, and the
BUG_ONat register to require all ports have the same value was the best option.Sure, but why not update the other areas anyway? This way when listens become per port, rather than per device, we only need to update that portion of the code.
I wasn't clear, I agree: Yes, update what is possible in the CM, but use an explicit device test for the areas we can't fix.
Jason
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
Tom Talpey
reviewerRe: [PATCH v4 10/27] IB/Verbs: Reform cm related part in IB-core cma/ucm
In reply to Michael WangQuoted 21 lines · Michael Wang
Quoted 8 lines · Sean
Quoted 8 lines
Quoted 7 lines
This is equivalent to today where the checks are per node rather than per port.
Should all checks here be port 1 based or only certain ones like listen ? For example, in connect/reject/disconnect, don't we already have port ? Guess this can be dealt with later as this is not a regression from the current implementation.
Quoted 7 lines
Quoted 5 lines
Yeah, these parts of cma may need more carve in future, like some new callback
for different CM type as Sean suggested.Maybe directly using 1 could help to highlight the problem ;-)
Only a few checks need to be per device. I think I pointed those out previously. Testing should show anywhere that we miss fairly quickly, since port would still be 0. For the checks that can be updated to be per port, I would rather go ahead and convert them.
Got it, will be changed in next version :-)
Quoted 7 lines
To be confirmed:
PORT ASSIGNED
rdma_init_qp_attrY
rdma_destroy_idunknown
cma_listen_on_devN
cma_bind_loopbackN
rdma_listenN
Why "N"? rdma_listen() can be constrained to a single port, right? And even if wildcarded, it needs to act on multiple ports, which is to say, it will fail only if no ports are eligible.
Tom.
Quoted 6 lines
rdma_connect Y rdma_accept Y rdma_reject Y rdma_disconnect Y ib_ucm_add_one N
Quoted 10 lines
Is this list correct?
Regards,
Michael WangQuoted 3 lines
- Sean
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
RE: [PATCH v4 10/27] IB/Verbs: Reform cm related part in IB-core cma/ucm
PiA+IFRvIGJlIGNvbmZpcm1lZDoNCj4gPiAJCQkJUE9SVCBBU1NJR05FRA0KPiA+IAlyZG1hX2lu aXRfcXBfYXR0cglZDQo+ID4gCXJkbWFfZGVzdHJveV9pZAkJdW5rbm93bg0KPiA+IAljbWFfbGlz dGVuX29uX2RldglODQo+ID4gCWNtYV9iaW5kX2xvb3BiYWNrCU4NCg0KQmluZCBsb29wYmFjayB3 aWxsIGF0dGFjaCB0byBhIHBvcnQsIGJ1dCB0aGUgaWQgZG9lcyBub3QgaGF2ZSBvbiBlbnRyeS4N Cg0KPiA+IAlyZG1hX2xpc3RlbgkJTg0KPiANCj4gV2h5ICJOIj8gcmRtYV9saXN0ZW4oKSBjYW4g YmUgY29uc3RyYWluZWQgdG8gYSBzaW5nbGUgcG9ydCwgcmlnaHQ/DQo+IEFuZCBldmVuIGlmIHdp bGRjYXJkZWQsIGl0IG5lZWRzIHRvIGFjdCBvbiBtdWx0aXBsZSBwb3J0cywgd2hpY2ggaXMNCj4g dG8gc2F5LCBpdCB3aWxsIGZhaWwgb25seSBpZiBubyBwb3J0cyBhcmUgZWxpZ2libGUuDQoNClJk bWEgbGlzdGVuIHNob3VsZCBiZSB1bmtub3duLiAgVGhlIGlkIG1heSBiZSBhc3NpZ25lZCB0byBh IHBvcnQuICBJdCBkZXBlbmRzIG9uIHRoZSBzb3VyY2UgYWRkcmVzcy4NCg0KPiA+IAlyZG1hX2Nv bm5lY3QJCVkNCj4gPiAJcmRtYV9hY2NlcHQJCVkNCj4gPiAJcmRtYV9yZWplY3QJCVkNCj4gPiAJ cmRtYV9kaXNjb25uZWN0CQlZDQo+ID4gCWliX3VjbV9hZGRfb25lCQlODQoNCk90aGVycyBsb29r IGNvcnJlY3QuDQoNCkJ0dywgdGhhbmtzIGZvciB5b3VyIHdvcmsgb24gdGhpcy4gIEkga25vdyB0 aGlzIGlzIGJlY29taW5nIGEgbXVjaCBiaWdnZXIgY2hhbmdlIHRoYW4geW91IG9yaWdpbmFsbHkg aW50ZW5kZWQuICA6KQ0KDQotIFNlYW4NCg==
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v4 10/27] IB/Verbs: Reform cm related part in IB-core cma/ucm
In reply to Jason GunthorpeQuoted 1 line · Jason Gunthorpe
It also looks like hardwired 1 won't work on switch ports, so it is no-go.
AFAIK enhanced switch port 0 is not supported by CM/RDMA CM in the current code. There is no need for CM/RDMA CM on base switch port 0. --
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
RE: [PATCH v4 10/27] IB/Verbs: Reform cm related part in IB-core cma/ucm
Quoted 6 lines · Jason Gunthorpe
Quoted 1 line
It also looks like hardwired 1 won't work on switch ports, so it is no-go.
AFAIK enhanced switch port 0 is not supported by CM/RDMA CM in the current code. There is no need for CM/RDMA CM on base switch port 0.
I concur and I thought I mentioned that before.
Ira
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v4 10/27] IB/Verbs: Reform cm related part in IB-core cma/ucm
In reply to Hal RosenstockQuoted 5 lines · Hal Rosenstock
Quoted 1 line · Jason Gunthorpe
It also looks like hardwired 1 won't work on switch ports, so it is no-go.
AFAIK enhanced switch port 0 is not supported by CM/RDMA CM in the current code. There is no need for CM/RDMA CM on base switch port 0.
Hurm, yet another reason to have a proper device level test to capture all this weirdness. I am surprised to hear this, actually.
Jason
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v4 10/27] IB/Verbs: Reform cm related part in IB-core cma/ucm
In reply to Tom TalpeyQuoted 30 lines · Tom Talpey
Quoted 21 lines · Michael Wang
Quoted 8 lines · Sean
Quoted 8 lines
Quoted 7 lines
This is equivalent to today where the checks are per node rather than per port.
Should all checks here be port 1 based or only certain ones like listen ? For example, in connect/reject/disconnect, don't we already have port ? Guess this can be dealt with later as this is not a regression from the current implementation.
Quoted 7 lines
Quoted 5 lines
Yeah, these parts of cma may need more carve in future, like some new callback
for different CM type as Sean suggested.Maybe directly using 1 could help to highlight the problem ;-)
Only a few checks need to be per device. I think I pointed those out previously. Testing should show anywhere that we miss fairly quickly, since port would still be 0. For the checks that can be updated to be per port, I would rather go ahead and convert them.
Got it, will be changed in next version :-)
Quoted 7 lines
To be confirmed:
PORT ASSIGNED
rdma_init_qp_attrY
rdma_destroy_idunknown
cma_listen_on_devN
cma_bind_loopbackN
rdma_listenN
Quoted 3 lines
Why "N"?
rdma_listen()can be constrained to a single port, right? And even if wildcarded, it needs to act on multiple ports, which is to say, it will fail only if no ports are eligible.
Yeah, it can or can't, maybe 'unknown' is better :-)
Regards,
Michael Wang
Quoted 21 lines
Tom.
Quoted 16 lines
rdma_connectY
rdma_acceptY
rdma_rejectY
rdma_disconnectY
ib_ucm_add_oneNIs this list correct?
Regards,
Michael WangQuoted 3 lines
- Sean
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v4 10/27] IB/Verbs: Reform cm related part in IB-core cma/ucm
In reply to Hefty, SeanQuoted 9 lines · Sean
Quoted 6 lines
Quoted 6 lines
To be confirmed:
PORT ASSIGNED
rdma_init_qp_attrY
rdma_destroy_idunknown
cma_listen_on_devN
cma_bind_loopbackNBind loopback will attach to a port, but the id does not have on entry.
Quoted 7 lines
Quoted 5 lines
Quoted 1 line
rdma_listenNWhy "N"?
rdma_listen()can be constrained to a single port, right? And even if wildcarded, it needs to act on multiple ports, which is to say, it will fail only if no ports are eligible.Rdma listen should be unknown. The id may be assigned to a port. It depends on the source address.
Agree, so for those 'N' or 'unknow', let's use port 1 directly.
Quoted 10 lines
Quoted 6 lines
Quoted 6 lines
rdma_connect Y rdma_accept Y rdma_reject Y rdma_disconnect Y ib_ucm_add_one NOthers look correct.
Btw, thanks for your work on this. I know this is becoming a much bigger change than you originally intended. :)
My pleasure :-) It do exceeded my expectations, fortunately I have you guys on my side, without your comments, it's impossible to make it so far :-)
Regards,
Michael Wang
Quoted 3 lines
- Sean
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v4 10/27] IB/Verbs: Reform cm related part in IB-core cma/ucm
In reply to Jason GunthorpeQuoted 8 lines · Jason Gunthorpe
Quoted 6 lines · Michael Wang
Use raw management helpers to reform cm related part in IB-core cma/ucm.
These checks focus on the device cm type rather than the port capability, directly pass port 1 works currently, but can't support mixing cm type device in future.
Quoted 5 lines
After the discussion settled, I ended up thinking that implementing explicit device checks, for use by CM, and the
BUG_ONat register to require all ports have the same value was the best option.It also looks like hardwired 1 won't work on switch ports, so it is no-go.
AFAIK, the current HW won't trigger such Bug, actually only mlx4 using port_num to check the link-layer (but still ib cm anyway), others are just static whatever the port_num is.
Thus as long as the check is still count on transport type and link-layer, the BUG may never be triggered...
Regards,
Michael Wang
Quoted 3 lines
Jason
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html