Enhanced mode for IPoIB driver
cover Enhanced mode for IPoIB driver
Hi Doug,
This patchset mostly comes from Erez with one exception in the first patch.
That patch origins from two different commits, first from Niranjana who added RDMA netdev interface and second from Erez who added IPoIB support.
During the preparation to submission, I squashed their commits into one and refactored code to allow submission as a standalone topic without creating dependecies between different submissions. This caused to change in author line. I hope that it doesn't really matter and it won't stop you from merging it.
Per-your request, I based this patch set on v4.11-rc3.
Changes from V0:
----------------------
1. ipoib_priv function returns struct ipoib_dev_priv * instead of void
2. Move all the refactoring of netdev_priv to be in single patch
3. Change the name of the Formation commit.
4. Remove unneeded blank lines
5. Set rn->hca on creation.
6. Return -EOPNOTSUPP instead of ENOTSUPP
7. Remove hunk with DMA from ib_device struct.
Thanks,
Leon
---
The rest comes from Erez:
The IPoIB protocol encapsulates IP packets over Infiniband datagrams. As a direct RDMA Upper Layer Protocol (ULP), IPoIB cannot support HW features that are specific to the IP protocol stack.
Nevertheless, RDMA interfaces have been extended to support some of the prominent IP offload features, such as TCP/UDP checksum and TSO. This provided reasonable performance gain for IPoIB but is still insufficient to cope with the increasing network bandwidth demand.
However, New features are exisiting in common network interfaces that are very hard to implement in IPoIB interfaces while it uses the RDMA layer, examples include TSS and RSS, tunneling offloads, and XDP. Rather than continuously porting IP network interface developments into the RDMA stack, we propose adding an abstract network data-path interfaces to RDMA devices.
In order to present a consistent interface to users, the IPoIB ULP continues to represent the network device to the IP stack. The common code also manages the IPoIB control plane, such as resolving path queries and registering to multicast groups.
Data path operations are forwarded to devices that implement the new API, or fallback to the standard implementation otherwise. Using the forgoing approach, we show how IPoIB closes the performance gap compared to state-of-the-art Ethernet network interfaces.
The implementation idea is to use the api of alloc_rdma_netdev/free_rdma_netdev to expose a struct that has data members and set of functions that are used for IB network interfaces, like attach/detach multicast to qp, and send IB packet.
The functions are specific for IB operations and are not part of the common api the the netdev struct exposes via the ndo functions.
1. multicast handling - attach/detach
2. send operation - the ndo start_xmit has only 2 parameters and the IB send needs the destination qp and the ah object, there were few options to handle it via the netdev ndo, but they don't make more sense than using a specific send function (we are rdma_netdev after all)
The IPoIB code will be adapted to enable the option of accelerating the network interface, but the code will work as before if the HW below doesn't support the acceleration.
That means that in the default mode of ipoib I tried to keep it as much as it was before, not to force it to adopt the new api, where there is no code sharing between the ipoib and the vnic/hfi.
The default code uses the controll and the data, the accelerator uses only the control flows andstructors.
The changes of the default ipoib can be made on top of that series.
Each HW vendor can supply the acceleration for the IPoIB or to leave IPoIB to work as before.
Thanks,
Erez
Erez Shitrit (5):
IB/IPoIB: Separate control and data related initializations IB/IPoIB: Separate control from HW operation on ipoib_open/stop ndo IB/IPoIB: Rename qpn to be dqpn in ipoib_send and post_send functions IB/IPoIB: Use defined function for netdev_priv function IB/IPoIB: Support acceleration options callbacks
Leon Romanovsky (1):
IB/IPoIB: Introduce RDMA netdev interface and IPoIB structs
drivers/infiniband/ulp/ipoib/ipoib.h | 39 ++- drivers/infiniband/ulp/ipoib/ipoib_cm.c | 66 ++--- drivers/infiniband/ulp/ipoib/ipoib_ethtool.c | 6 +- drivers/infiniband/ulp/ipoib/ipoib_fs.c | 4 +- drivers/infiniband/ulp/ipoib/ipoib_ib.c | 340 +++++++++++++------------ drivers/infiniband/ulp/ipoib/ipoib_main.c | 306 ++++++++++++++++------ drivers/infiniband/ulp/ipoib/ipoib_multicast.c | 39 +-- drivers/infiniband/ulp/ipoib/ipoib_netlink.c | 12 +- drivers/infiniband/ulp/ipoib/ipoib_verbs.c | 64 ++--- drivers/infiniband/ulp/ipoib/ipoib_vlan.c | 9 +- include/rdma/ib_verbs.h | 41 +++ 11 files changed, 568 insertions(+), 358 deletions(-)
--
2.12.0
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
A message that is not in the archive (sent off-list or not yet received): 20170405140913.2551-1-leon-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org
PATCH 1/6 IB/IPoIB: Introduce RDMA netdev interface and IPoIB structs
Add RDMA netdev interface to ib device structure allowing RDMA netdev devices to be allocated by ib clients.
The idea is to allow to providers to optimize IPoIB data path. New struct that includes functions and data member is exposed. It exposes set of callback functions for handling data path flows in IPoIB driver.
Each provider can support these set of functions in order to optimize its specific data path, and let IPoIB to leverage its data path.
There is an assumption, that providers should give the full set of functions and not only part of them, in order to work properly.
--- include/rdma/ib_verbs.h | 41 +++++++++++++++++++++++++++++++++++++++++ 1 file changed, 41 insertions(+)
include/rdma/ib_verbs.h +41 -0
@@ -55,6 +55,7 @@#include <net/ip.h> #include <linux/string.h> #include <linux/slab.h> +#include <linux/netdevice.h> #include <linux/if_link.h> #include <linux/atomic.h>
@@ -1862,7 +1863,38 @@ struct ib_port_immutable {u32 max_mad_size; }; +/* rdma netdev type - specifies protocol type */ +enum rdma_netdev_t { +RDMA_NETDEV_IPOIB , +}; + +/* +* struct rdma_netdev - rdma netdev +* For cases where netstack interfacing is required. +*/ +struct rdma_netdev { +void *clnt_priv; + +/* control functions */ +void (*set_id)(struct net_device *netdev,int id); +/* send packet */ +int (*send)(struct net_device *dev,struct sk_buff *skb, +struct ib_ah *address,u32 dqpn); +/* multicast */ +int (*attach_mcast)(struct net_device *dev,struct ib_device *hca, +union ib_gid *gid,u16 mlid, +int set_qkey,u32 qkey); +int (*detach_mcast)(struct net_device *dev,struct ib_device *hca, +union ib_gid *gid,u16 mlid); + +/* rdma-netdev private members */ +struct ib_device *hca; +}; +struct ib_device { +/* Do not access @dma_device directly from ULP nor from HW drivers. */ +struct device *dma_device; +char name[IB_DEVICE_NAME_MAX ];struct list_head event_handler_list;
@@ -2112,6 +2144,15 @@ struct ib_device {struct ib_rwq_ind_table_init_attr *init_attr, struct ib_udata *udata);int (*destroy_rwq_ind_table)(struct ib_rwq_ind_table *wq_ind_table); +/* rdma netdev operations */ +struct net_device *(*alloc_rdma_netdev)( +struct ib_device *device, +u8 port_num, +enum rdma_netdev_t type, +const char *name, +unsigned char name_assign_type, +void (*setup)(struct net_device *)); +void (*free_rdma_netdev)(struct net_device *netdev);struct module *owner;struct device dev;
Signature · 6 lines
2.12.0 -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
PATCH 2/6 IB/IPoIB: Separate control and data related initializations
In reply to a message not in the archiveFrom: Erez Shitrit <erezsh-VPRAkNaXOzVWk0Htik3J/[email protected]>
This patch prepares init and teardown flows so we can call them through ipoib_options function pointers.
It arranges that area of code as the following:
* All operations which deal with the resource allocation/deletion are performed in one place.
* All operations that are control oriented, meaning that they are not connected to a specific hardware, are performed in a separate place.
The operations for allocation of hardware resources are now in the function ipoib_dev_init_default, and the deletion of all the resources are in ipoib_dev_uninit_default
The only exception is the creation of the PD object, which is used both for resource allocation (create QP etc.) and for control flows like creating AH.
It also does:
* Move creation of rx_ring and tx_ring to be in the resources allocation area.
* Move the function ipoib_ib_dev_open that does the open device to the control area instead of the dev_init which creates resources.
--- drivers/infiniband/ulp/ipoib/ipoib.h | 3 +- drivers/infiniband/ulp/ipoib/ipoib_ib.c | 38 +++------- drivers/infiniband/ulp/ipoib/ipoib_main.c | 116 ++++++++++++++++++++++++----- drivers/infiniband/ulp/ipoib/ipoib_verbs.c | 40 +--------- 4 files changed, 109 insertions(+), 88 deletions(-)
drivers/infiniband/ulp/ipoib/ipoib.h +2 -1
@@ -492,13 +492,14 @@ void ipoib_flush_paths(struct net_device *dev);int ipoib_check_sm_sendonly_fullmember_support (struct ipoib_dev_priv *priv);struct ipoib_dev_priv *ipoib_intf_alloc (const char *format); -int ipoib_ib_dev_init (struct net_device *dev,struct ib_device *ca,int port); +void ipoib_ib_tx_timer_func (unsigned long ctx);void ipoib_ib_dev_flush_light (struct work_struct *work);void ipoib_ib_dev_flush_normal (struct work_struct *work);void ipoib_ib_dev_flush_heavy (struct work_struct *work);void ipoib_pkey_event (struct work_struct *work);void ipoib_ib_dev_cleanup (struct net_device *dev); +void ipoib_dev_uninit_default (struct net_device *dev);int ipoib_ib_dev_open (struct net_device *dev);void ipoib_ib_dev_up (struct net_device *dev);void ipoib_ib_dev_down (struct net_device *dev);
drivers/infiniband/ulp/ipoib/ipoib_ib.c +9 -29
@@ -692,7 +692,7 @@ static void ipoib_stop_ah(struct net_device *dev)ipoib_flush_ah (dev); } -static void ipoib_ib_tx_timer_func (unsigned long ctx) +void ipoib_ib_tx_timer_func (unsigned long ctx) {drain_tx_cq ((struct net_device *)ctx); }
@@ -913,32 +913,6 @@ void ipoib_ib_dev_stop(struct net_device *dev)ib_req_notify_cq (priv->recv_cq,IB_CQ_NEXT_COMP ); } -int ipoib_ib_dev_init (struct net_device *dev,struct ib_device *ca,int port) -{ -struct ipoib_dev_priv *priv =netdev_priv (dev); - - priv->ca = ca; - priv->port = port; - priv->qp =NULL ; - -if (ipoib_transport_dev_init (dev, ca)) { -printk (KERN_WARNING "%s: ipoib_transport_dev_init failed\n" , ca->name); -return -ENODEV ; - } - -setup_timer (&priv->poll_timer, ipoib_ib_tx_timer_func, - (unsigned long ) dev); - -if (dev->flags &IFF_UP ) { -if (ipoib_ib_dev_open (dev)) { -ipoib_transport_dev_cleanup (dev); -return -ENODEV ; - } - } - -return 0 ; -} -/* * Takes whatever value which is in pkey index 0 and updates priv->pkey * returns 0 if the pkey value was changed.
@@ -1236,7 +1210,13 @@ void ipoib_ib_dev_cleanup(struct net_device *dev)*/ ipoib_stop_ah (dev); -ipoib_transport_dev_cleanup (dev); -} +clear_bit (IPOIB_PKEY_ASSIGNED , &priv->flags); + +ipoib_dev_uninit_default (dev); +if (priv->pd) { +ib_dealloc_pd (priv->pd); + priv->pd =NULL ; + } +}
drivers/infiniband/ulp/ipoib/ipoib_main.c +97 -19
153 lines not on this page · raw message
drivers/infiniband/ulp/ipoib/ipoib_verbs.c +1 -39
@@ -147,22 +147,6 @@ int ipoib_transport_dev_init(struct net_device *dev, struct ib_device *ca)int ret, size;int i; - priv->pd =ib_alloc_pd (priv->ca,0 ); -if (IS_ERR (priv->pd)) { -printk (KERN_WARNING "%s: failed to allocate PD\n" , ca->name); -return -ENODEV ; - } - -/* -* the various IPoIB tasks assume they will never race against -* themselves, so always use a single thread workqueue -*/ - priv->wq =alloc_ordered_workqueue ("ipoib_wq" ,WQ_MEM_RECLAIM ); -if (!priv->wq) { -printk (KERN_WARNING "ipoib: failed to allocate device WQ\n" ); - goto out_free_pd; - } - size = ipoib_recvq_size +1 ; ret =ipoib_cm_dev_init (dev);if (!ret) {
@@ -173,7 +157,7 @@ int ipoib_transport_dev_init(struct net_device *dev, struct ib_device *ca) size += ipoib_recvq_size * ipoib_max_conn_qp; }else if (ret != -ENOSYS ) - goto out_free_wq; +return -ENODEV ; cq_attr.cqe = size; priv->recv_cq = ib_create_cq(priv->ca, ipoib_ib_completion, NULL,
@@ -212,10 +196,6 @@ int ipoib_transport_dev_init(struct net_device *dev, struct ib_device *ca) goto out_free_send_cq; } - priv->dev->dev_addr[1 ] = (priv->qp->qp_num >>16 ) &0xff ; - priv->dev->dev_addr[2 ] = (priv->qp->qp_num >>8 ) &0xff ; - priv->dev->dev_addr[3 ] = (priv->qp->qp_num ) &0xff ; -for (i =0 ; i <MAX_SKB_FRAGS +1 ; ++i) priv->tx_sge[i].lkey = priv->pd->local_dma_lkey;
@@ -247,13 +227,6 @@ int ipoib_transport_dev_init(struct net_device *dev, struct ib_device *ca) out_cm_dev_cleanup:ipoib_cm_dev_cleanup (dev); -out_free_wq: -destroy_workqueue (priv->wq); - priv->wq =NULL ; - -out_free_pd: -ib_dealloc_pd (priv->pd); -return -ENODEV ; }
@@ -266,7 +239,6 @@ void ipoib_transport_dev_cleanup(struct net_device *dev)ipoib_warn (priv,"ib_qp_destroy failed\n" ); priv->qp =NULL ; -clear_bit (IPOIB_PKEY_ASSIGNED , &priv->flags); }if (ib_destroy_cq (priv->send_cq))
@@ -274,16 +246,6 @@ void ipoib_transport_dev_cleanup(struct net_device *dev)if (ib_destroy_cq (priv->recv_cq))ipoib_warn (priv,"ib_cq_destroy (recv) failed\n" ); - -ipoib_cm_dev_cleanup (dev); - -if (priv->wq) { -flush_workqueue (priv->wq); -destroy_workqueue (priv->wq); - priv->wq =NULL ; - } - -ib_dealloc_pd (priv->pd); } void ipoib_event(struct ib_event_handler *handler,
Signature · 6 lines
2.12.0 -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
PATCH 3/6 IB/IPoIB: Separate control from HW operation on ipoib_open/stop ndo
In reply to a message not in the archiveFrom: Erez Shitrit <erezsh-VPRAkNaXOzVWk0Htik3J/[email protected]>
This patch is preparing the netdev part at the IPoIB driver to be able to use the ipoib_options.
It deals with the two flows from the .ndo: ipoib_open and ipoib_stop.
The code is rearranged as follows:
* All operations which deal with the hardware resources, (for example change QP state, post-receive etc.) are performed in one place.
* All operations that are control oriented (like restart multicast task, start the reap_ah etc.) are performed in separate place.
The functions that deal with the hardware resources now located at __ipoib_ib_dev_open for the ipoib_open flow and __ipoib_ib_dev_stop for ipoib_stop.
--- drivers/infiniband/ulp/ipoib/ipoib.h | 2 +- drivers/infiniband/ulp/ipoib/ipoib_ib.c | 228 +++++++++++++++++------------- drivers/infiniband/ulp/ipoib/ipoib_main.c | 2 +- 3 files changed, 129 insertions(+), 103 deletions(-)
drivers/infiniband/ulp/ipoib/ipoib.h +1 -1
@@ -503,7 +503,7 @@ void ipoib_dev_uninit_default(struct net_device *dev);int ipoib_ib_dev_open (struct net_device *dev);void ipoib_ib_dev_up (struct net_device *dev);void ipoib_ib_dev_down (struct net_device *dev); -void ipoib_ib_dev_stop (struct net_device *dev); +int ipoib_ib_dev_stop_default (struct net_device *dev);void ipoib_pkey_dev_check_presence (struct net_device *dev);int ipoib_dev_init (struct net_device *dev,struct ib_device *ca,int port);
drivers/infiniband/ulp/ipoib/ipoib_ib.c +127 -101
267 lines not on this page · raw message
drivers/infiniband/ulp/ipoib/ipoib_main.c +1 -1
@@ -166,7 +166,7 @@ static int ipoib_stop(struct net_device *dev)netif_stop_queue (dev);ipoib_ib_dev_down (dev); -ipoib_ib_dev_stop (dev); +ipoib_ib_dev_stop_default (dev);if (!test_bit (IPOIB_FLAG_SUBINTERFACE , &priv->flags)) {struct ipoib_dev_priv *cpriv;
Signature · 6 lines
2.12.0 -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
PATCH 4/6 IB/IPoIB: Rename qpn to be dqpn in ipoib_send and post_send functions
In reply to a message not in the archiveFrom: Erez Shitrit <erezsh-VPRAkNaXOzVWk0Htik3J/[email protected]>
Change of function parameter name from qpn to be dqpn.
--- drivers/infiniband/ulp/ipoib/ipoib.h | 2 +- drivers/infiniband/ulp/ipoib/ipoib_ib.c | 10 +++++----- 2 files changed, 6 insertions(+), 6 deletions(-)
drivers/infiniband/ulp/ipoib/ipoib.h +1 -1
@@ -483,7 +483,7 @@ int ipoib_add_pkey_attr(struct net_device *dev);int ipoib_add_umcast_attr (struct net_device *dev);void ipoib_send (struct net_device *dev,struct sk_buff *skb, -struct ipoib_ah *address,u32 qpn); +struct ipoib_ah *address,u32 dqpn);void ipoib_reap_ah (struct work_struct *work);struct ipoib_path *__path_find (struct net_device *dev,void *gid);
drivers/infiniband/ulp/ipoib/ipoib_ib.c +5 -5
@@ -513,7 +513,7 @@ void ipoib_send_comp_handler(struct ib_cq *cq, void *dev_ptr)static inline int post_send (struct ipoib_dev_priv *priv,unsigned int wr_id, -struct ib_ah *address,u32 qpn, +struct ib_ah *address,u32 dqpn,struct ipoib_tx_buf *tx_req,void *head,int hlen) {
@@ -523,7 +523,7 @@ static inline int post_send(struct ipoib_dev_priv *priv,ipoib_build_sge (priv, tx_req); priv->tx_wr.wr.wr_id = wr_id; - priv->tx_wr.remote_qpn = qpn; + priv->tx_wr.remote_qpn = dqpn; priv->tx_wr.ah = address;if (head) {
@@ -538,7 +538,7 @@ static inline int post_send(struct ipoib_dev_priv *priv, }void ipoib_send (struct net_device *dev,struct sk_buff *skb, -struct ipoib_ah *address,u32 qpn) +struct ipoib_ah *address,u32 dqpn) {struct ipoib_dev_priv *priv =netdev_priv (dev);struct ipoib_tx_buf *tx_req;
@@ -587,7 +587,7 @@ void ipoib_send(struct net_device *dev, struct sk_buff *skb, }ipoib_dbg_data (priv,"sending packet, length=%d address=%p qpn=0x%06x\n" , - skb->len, address, qpn); + skb->len, address, dqpn);/* * We put the skb into the tx_ring _before_ we call post_send()
@@ -620,7 +620,7 @@ void ipoib_send(struct net_device *dev, struct sk_buff *skb,skb_dst_drop (skb); rc =post_send (priv, priv->tx_head & (ipoib_sendq_size -1 ), - address->ah, qpn, tx_req, phead, hlen); + address->ah, dqpn, tx_req, phead, hlen);if (unlikely (rc)) {ipoib_warn (priv,"post_send failed, error %d\n" , rc); ++dev->stats.tx_errors;
Signature · 6 lines
2.12.0 -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
PATCH 5/6 IB/IPoIB: Use defined function for netdev_priv function
In reply to a message not in the archiveFrom: Erez Shitrit <erezsh-VPRAkNaXOzVWk0Htik3J/[email protected]>
Make ipoib_priv point to netdev_priv where the code calls netdev_priv.
--- drivers/infiniband/ulp/ipoib/ipoib.h | 11 ++-- drivers/infiniband/ulp/ipoib/ipoib_cm.c | 66 ++++++++++----------- drivers/infiniband/ulp/ipoib/ipoib_ethtool.c | 6 +- drivers/infiniband/ulp/ipoib/ipoib_fs.c | 4 +- drivers/infiniband/ulp/ipoib/ipoib_ib.c | 46 +++++++-------- drivers/infiniband/ulp/ipoib/ipoib_main.c | 80 +++++++++++++------------- drivers/infiniband/ulp/ipoib/ipoib_multicast.c | 24 ++++---- drivers/infiniband/ulp/ipoib/ipoib_netlink.c | 10 ++-- drivers/infiniband/ulp/ipoib/ipoib_verbs.c | 8 +-- drivers/infiniband/ulp/ipoib/ipoib_vlan.c | 4 +- 10 files changed, 131 insertions(+), 128 deletions(-)
drivers/infiniband/ulp/ipoib/ipoib.h +7 -4
@@ -153,6 +153,9 @@ static inline void skb_add_pseudo_hdr(struct sk_buff *skb)skb_pull (skb,IPOIB_HARD_LEN ); } +/* Keep the refactoring compile able */ +#define ipoib_priv netdev_priv +/* Used for all multicast joins (broadcast, IPv4 mcast and IPv6 mcast) */ struct ipoib_mcast {struct ib_sa_mcmember_rec mcmember;
@@ -608,14 +611,14 @@ extern int ipoib_max_conn_qp;static inline int ipoib_cm_admin_enabled (struct net_device *dev) { -struct ipoib_dev_priv *priv =netdev_priv (dev); +struct ipoib_dev_priv *priv =ipoib_priv (dev);return IPOIB_CM_SUPPORTED (dev->dev_addr) &&test_bit (IPOIB_FLAG_ADMIN_CM , &priv->flags); }static inline int ipoib_cm_enabled (struct net_device *dev,u8 *hwaddr) { -struct ipoib_dev_priv *priv =netdev_priv (dev); +struct ipoib_dev_priv *priv =ipoib_priv (dev);return IPOIB_CM_SUPPORTED (hwaddr) &&test_bit (IPOIB_FLAG_ADMIN_CM , &priv->flags); }
@@ -638,13 +641,13 @@ static inline void ipoib_cm_set(struct ipoib_neigh *neigh, struct ipoib_cm_tx *tstatic inline int ipoib_cm_has_srq (struct net_device *dev) { -struct ipoib_dev_priv *priv =netdev_priv (dev); +struct ipoib_dev_priv *priv =ipoib_priv (dev);return !!priv->cm.srq; }static inline unsigned int ipoib_cm_max_mtu (struct net_device *dev) { -struct ipoib_dev_priv *priv =netdev_priv (dev); +struct ipoib_dev_priv *priv =ipoib_priv (dev);return priv->cm.max_cm_mtu; }
drivers/infiniband/ulp/ipoib/ipoib_cm.c +33 -33
264 lines not on this page · raw message
drivers/infiniband/ulp/ipoib/ipoib_ethtool.c +3 -3
@@ -60,7 +60,7 @@ static const struct ipoib_stats ipoib_gstrings_stats[] = {static void ipoib_get_drvinfo (struct net_device *netdev,struct ethtool_drvinfo *drvinfo) { -struct ipoib_dev_priv *priv =netdev_priv (netdev); +struct ipoib_dev_priv *priv =ipoib_priv (netdev);ib_get_device_fw_str (priv->ca, drvinfo->fw_version,sizeof (drvinfo->fw_version));
@@ -77,7 +77,7 @@ static void ipoib_get_drvinfo(struct net_device *netdev,static int ipoib_get_coalesce (struct net_device *dev,struct ethtool_coalesce *coal) { -struct ipoib_dev_priv *priv =netdev_priv (dev); +struct ipoib_dev_priv *priv =ipoib_priv (dev); coal->rx_coalesce_usecs = priv->ethtool.coalesce_usecs; coal->rx_max_coalesced_frames = priv->ethtool.max_coalesced_frames;
@@ -88,7 +88,7 @@ static int ipoib_get_coalesce(struct net_device *dev,static int ipoib_set_coalesce (struct net_device *dev,struct ethtool_coalesce *coal) { -struct ipoib_dev_priv *priv =netdev_priv (dev); +struct ipoib_dev_priv *priv =ipoib_priv (dev);int ret;/*
drivers/infiniband/ulp/ipoib/ipoib_fs.c +2 -2
@@ -261,7 +261,7 @@ static const struct file_operations ipoib_path_fops = {void ipoib_create_debug_files (struct net_device *dev) { -struct ipoib_dev_priv *priv =netdev_priv (dev); +struct ipoib_dev_priv *priv =ipoib_priv (dev);char name[IFNAMSIZ +sizeof "_path" ];snprintf (name,sizeof name,"%s_mcg" , dev->name);
@@ -279,7 +279,7 @@ void ipoib_create_debug_files(struct net_device *dev)void ipoib_delete_debug_files (struct net_device *dev) { -struct ipoib_dev_priv *priv =netdev_priv (dev); +struct ipoib_dev_priv *priv =ipoib_priv (dev);debugfs_remove (priv->mcg_dentry);debugfs_remove (priv->path_dentry);
drivers/infiniband/ulp/ipoib/ipoib_ib.c +23 -23
184 lines not on this page · raw message
drivers/infiniband/ulp/ipoib/ipoib_main.c +40 -40
320 lines not on this page · raw message
drivers/infiniband/ulp/ipoib/ipoib_multicast.c +12 -12
@@ -114,7 +114,7 @@ static void ipoib_mcast_free(struct ipoib_mcast *mcast)struct net_device *dev = mcast->dev;int tx_dropped =0 ; -ipoib_dbg_mcast (netdev_priv (dev),"deleting multicast group %pI6\n" , +ipoib_dbg_mcast (ipoib_priv (dev),"deleting multicast group %pI6\n" , mcast->mcmember.mgid.raw);/* remove all neigh connected to this mcast */
@@ -158,7 +158,7 @@ static struct ipoib_mcast *ipoib_mcast_alloc(struct net_device *dev,static struct ipoib_mcast *__ipoib_mcast_find (struct net_device *dev,void *mgid) { -struct ipoib_dev_priv *priv =netdev_priv (dev); +struct ipoib_dev_priv *priv =ipoib_priv (dev);struct rb_node *n = priv->multicast_tree.rb_node;while (n) {
@@ -182,7 +182,7 @@ static struct ipoib_mcast *__ipoib_mcast_find(struct net_device *dev, void *mgidstatic int __ipoib_mcast_add (struct net_device *dev,struct ipoib_mcast *mcast) { -struct ipoib_dev_priv *priv =netdev_priv (dev); +struct ipoib_dev_priv *priv =ipoib_priv (dev);struct rb_node **n = &priv->multicast_tree.rb_node, *pn =NULL ;while (*n) {
@@ -212,7 +212,7 @@ static int ipoib_mcast_join_finish(struct ipoib_mcast *mcast, struct ib_sa_mcmember_rec *mcmember) {struct net_device *dev = mcast->dev; -struct ipoib_dev_priv *priv =netdev_priv (dev); +struct ipoib_dev_priv *priv =ipoib_priv (dev);struct ipoib_ah *ah;int ret;int set_qkey =0 ;
@@ -375,7 +375,7 @@ static int ipoib_mcast_join_complete(int status, {struct ipoib_mcast *mcast = multicast->context;struct net_device *dev = mcast->dev; -struct ipoib_dev_priv *priv =netdev_priv (dev); +struct ipoib_dev_priv *priv =ipoib_priv (dev); ipoib_dbg_mcast(priv, "%sjoin completion for %pI6 (status %d)\n", test_bit(IPOIB_MCAST_FLAG_SENDONLY, &mcast->flags) ?
@@ -477,7 +477,7 @@ static int ipoib_mcast_join_complete(int status,*/ static int ipoib_mcast_join (struct net_device *dev,struct ipoib_mcast *mcast) { -struct ipoib_dev_priv *priv =netdev_priv (dev); +struct ipoib_dev_priv *priv =ipoib_priv (dev);struct ib_sa_multicast *multicast;struct ib_sa_mcmember_rec rec = { .join_state =1
@@ -678,7 +678,7 @@ void ipoib_mcast_join_task(struct work_struct *work)void ipoib_mcast_start_thread (struct net_device *dev) { -struct ipoib_dev_priv *priv =netdev_priv (dev); +struct ipoib_dev_priv *priv =ipoib_priv (dev);unsigned long flags;ipoib_dbg_mcast (priv,"starting multicast thread\n" );
@@ -690,7 +690,7 @@ void ipoib_mcast_start_thread(struct net_device *dev)int ipoib_mcast_stop_thread (struct net_device *dev) { -struct ipoib_dev_priv *priv =netdev_priv (dev); +struct ipoib_dev_priv *priv =ipoib_priv (dev);unsigned long flags;ipoib_dbg_mcast (priv,"stopping multicast thread\n" );
@@ -706,7 +706,7 @@ int ipoib_mcast_stop_thread(struct net_device *dev)static int ipoib_mcast_leave (struct net_device *dev,struct ipoib_mcast *mcast) { -struct ipoib_dev_priv *priv =netdev_priv (dev); +struct ipoib_dev_priv *priv =ipoib_priv (dev);int ret =0 ;if (test_and_clear_bit (IPOIB_MCAST_FLAG_BUSY , &mcast->flags))
@@ -762,7 +762,7 @@ void ipoib_mcast_remove_list(struct list_head *remove_list)void ipoib_mcast_send (struct net_device *dev,u8 *daddr,struct sk_buff *skb) { -struct ipoib_dev_priv *priv =netdev_priv (dev); +struct ipoib_dev_priv *priv =ipoib_priv (dev);struct ipoib_mcast *mcast;unsigned long flags;void *mgid = daddr +4 ;
@@ -837,7 +837,7 @@ void ipoib_mcast_send(struct net_device *dev, u8 *daddr, struct sk_buff *skb)void ipoib_mcast_dev_flush (struct net_device *dev) { -struct ipoib_dev_priv *priv =netdev_priv (dev); +struct ipoib_dev_priv *priv =ipoib_priv (dev);LIST_HEAD (remove_list);struct ipoib_mcast *mcast, *tmcast;unsigned long flags;
@@ -1029,7 +1029,7 @@ struct ipoib_mcast_iter *ipoib_mcast_iter_init(struct net_device *dev)int ipoib_mcast_iter_next (struct ipoib_mcast_iter *iter) { -struct ipoib_dev_priv *priv =netdev_priv (iter->dev); +struct ipoib_dev_priv *priv =ipoib_priv (iter->dev);struct rb_node *n;struct ipoib_mcast *mcast;int ret =1 ;
drivers/infiniband/ulp/ipoib/ipoib_netlink.c +5 -5
@@ -44,7 +44,7 @@ static const struct nla_policy ipoib_policy[IFLA_IPOIB_MAX + 1] = {static int ipoib_fill_info (struct sk_buff *skb,const struct net_device *dev) { -struct ipoib_dev_priv *priv =netdev_priv (dev); +struct ipoib_dev_priv *priv =ipoib_priv (dev);u16 val;if (nla_put_u16 (skb,IFLA_IPOIB_PKEY , priv->pkey))
@@ -107,7 +107,7 @@ static int ipoib_new_child_link(struct net *src_net, struct net_device *dev,if (!pdev || pdev->type !=ARPHRD_INFINIBAND )return -ENODEV ; - ppriv =netdev_priv (pdev); + ppriv =ipoib_priv (pdev);if (test_bit (IPOIB_FLAG_SUBINTERFACE , &ppriv->flags)) {ipoib_warn (ppriv,"child creation disallowed for child devices\n" );
@@ -129,7 +129,7 @@ static int ipoib_new_child_link(struct net *src_net, struct net_device *dev,*/ child_pkey |=0x8000 ; - err =__ipoib_vlan_add (ppriv,netdev_priv (dev), child_pkey,IPOIB_RTNL_CHILD ); + err =__ipoib_vlan_add (ppriv,ipoib_priv (dev), child_pkey,IPOIB_RTNL_CHILD );if (!err && data) err =ipoib_changelink (dev, tb, data);
@@ -140,8 +140,8 @@ static void ipoib_unregister_child_dev(struct net_device *dev, struct list_head {struct ipoib_dev_priv *priv, *ppriv; - priv =netdev_priv (dev); - ppriv =netdev_priv (priv->parent); + priv =ipoib_priv (dev); + ppriv =ipoib_priv (priv->parent);down_write (&ppriv->vlan_rwsem);unregister_netdevice_queue (dev, head);
drivers/infiniband/ulp/ipoib/ipoib_verbs.c +4 -4
@@ -37,7 +37,7 @@int ipoib_mcast_attach (struct net_device *dev,u16 mlid,union ib_gid *mgid,int set_qkey) { -struct ipoib_dev_priv *priv =netdev_priv (dev); +struct ipoib_dev_priv *priv =ipoib_priv (dev);struct ib_qp_attr *qp_attr =NULL ;int ret;u16 pkey_index;
@@ -76,7 +76,7 @@ int ipoib_mcast_attach(struct net_device *dev, u16 mlid, union ib_gid *mgid, intint ipoib_init_qp (struct net_device *dev) { -struct ipoib_dev_priv *priv =netdev_priv (dev); +struct ipoib_dev_priv *priv =ipoib_priv (dev);int ret;struct ib_qp_attr qp_attr;int attr_mask;
@@ -130,7 +130,7 @@ int ipoib_init_qp(struct net_device *dev)int ipoib_transport_dev_init (struct net_device *dev,struct ib_device *ca) { -struct ipoib_dev_priv *priv =netdev_priv (dev); +struct ipoib_dev_priv *priv =ipoib_priv (dev);struct ib_qp_init_attr init_attr = { .cap = { .max_send_wr = ipoib_sendq_size,
@@ -232,7 +232,7 @@ int ipoib_transport_dev_init(struct net_device *dev, struct ib_device *ca)void ipoib_transport_dev_cleanup (struct net_device *dev) { -struct ipoib_dev_priv *priv =netdev_priv (dev); +struct ipoib_dev_priv *priv =ipoib_priv (dev);if (priv->qp) {if (ib_destroy_qp (priv->qp))
drivers/infiniband/ulp/ipoib/ipoib_vlan.c +2 -2
@@ -44,7 +44,7 @@ static ssize_t show_parent(struct device *d, struct device_attribute *attr, char *buf) {struct net_device *dev =to_net_dev (d); -struct ipoib_dev_priv *priv =netdev_priv (dev); +struct ipoib_dev_priv *priv =ipoib_priv (dev);return sprintf (buf,"%s\n" , priv->parent->name); }
@@ -183,7 +183,7 @@ int ipoib_vlan_delete(struct net_device *pdev, unsigned short pkey)if (!capable (CAP_NET_ADMIN ))return -EPERM ; - ppriv =netdev_priv (pdev); + ppriv =ipoib_priv (pdev);if (test_bit (IPOIB_FLAG_GOING_DOWN , &ppriv->flags))return -EPERM ;
Signature · 6 lines
2.12.0 -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
PATCH 6/6 IB/IPoIB: Support acceleration options callbacks
In reply to a message not in the archiveFrom: Erez Shitrit <erezsh-VPRAkNaXOzVWk0Htik3J/[email protected]>
IPoIB driver now uses the new set of callback functions.
If the hardware provider supports the new ipoib_options implementation, the driver uses the callbacks in its data path flows, otherwise it uses the driver default implementation for all data flows in its code.
The default implementation wasn't change and it is exactly as it was before introduction of acceleration support.
--- drivers/infiniband/ulp/ipoib/ipoib.h | 29 ++++-- drivers/infiniband/ulp/ipoib/ipoib_ib.c | 52 ++++++---- drivers/infiniband/ulp/ipoib/ipoib_main.c | 136 ++++++++++++++++++++----- drivers/infiniband/ulp/ipoib/ipoib_multicast.c | 15 ++- drivers/infiniband/ulp/ipoib/ipoib_netlink.c | 2 +- drivers/infiniband/ulp/ipoib/ipoib_verbs.c | 16 ++- drivers/infiniband/ulp/ipoib/ipoib_vlan.c | 5 +- 7 files changed, 187 insertions(+), 68 deletions(-)
drivers/infiniband/ulp/ipoib/ipoib.h +18 -11
@@ -52,7 +52,6 @@#include <rdma/ib_pack.h> #include <rdma/ib_sa.h> #include <linux/sched.h> -/* constants */ enum ipoib_flush_level {
@@ -153,8 +152,12 @@ static inline void skb_add_pseudo_hdr(struct sk_buff *skb)skb_pull (skb,IPOIB_HARD_LEN ); } -/* Keep the refactoring compile able */ -#define ipoib_priv netdev_priv +static inline struct ipoib_dev_priv *ipoib_priv (const struct net_device *dev) +{ +struct rdma_netdev *rn =netdev_priv (dev); + +return rn->clnt_priv; +}/* Used for all multicast joins (broadcast, IPv4 mcast and IPv6 mcast) */ struct ipoib_mcast {
@@ -407,6 +410,7 @@ struct ipoib_dev_priv {struct timer_list poll_timer;unsigned max_send_sge;bool sm_fullmember_sendonly_support; +const struct net_device_ops *rn_ops; };struct ipoib_ah {
@@ -485,16 +489,16 @@ int ipoib_open(struct net_device *dev);int ipoib_add_pkey_attr (struct net_device *dev);int ipoib_add_umcast_attr (struct net_device *dev); -void ipoib_send (struct net_device *dev,struct sk_buff *skb, -struct ipoib_ah *address,u32 dqpn); +int ipoib_send (struct net_device *dev,struct sk_buff *skb, +struct ib_ah *address,u32 dqpn);void ipoib_reap_ah (struct work_struct *work);struct ipoib_path *__path_find (struct net_device *dev,void *gid);void ipoib_mark_paths_invalid (struct net_device *dev);void ipoib_flush_paths (struct net_device *dev);int ipoib_check_sm_sendonly_fullmember_support (struct ipoib_dev_priv *priv); -struct ipoib_dev_priv *ipoib_intf_alloc (const char *format); - +struct ipoib_dev_priv *ipoib_intf_alloc (struct ib_device *hca,u8 port, +const char *format);void ipoib_ib_tx_timer_func (unsigned long ctx);void ipoib_ib_dev_flush_light (struct work_struct *work);void ipoib_ib_dev_flush_normal (struct work_struct *work);
@@ -502,8 +506,9 @@ void ipoib_ib_dev_flush_heavy(struct work_struct *work);void ipoib_pkey_event (struct work_struct *work);void ipoib_ib_dev_cleanup (struct net_device *dev); -void ipoib_dev_uninit_default (struct net_device *dev); +int ipoib_ib_dev_open_default (struct net_device *dev);int ipoib_ib_dev_open (struct net_device *dev); +int ipoib_ib_dev_stop (struct net_device *dev);void ipoib_ib_dev_up (struct net_device *dev);void ipoib_ib_dev_down (struct net_device *dev);int ipoib_ib_dev_stop_default (struct net_device *dev);
@@ -566,8 +571,10 @@ void ipoib_path_iter_read(struct ipoib_path_iter *iter, struct ipoib_path *path);#endif -int ipoib_mcast_attach (struct net_device *dev,u16 mlid, -union ib_gid *mgid,int set_qkey); +int ipoib_mcast_attach (struct net_device *dev,struct ib_device *hca, +union ib_gid *mgid,u16 mlid,int set_qkey,u32 qkey); +int ipoib_mcast_detach (struct net_device *dev,struct ib_device *hca, +union ib_gid *mgid,u16 mlid);void ipoib_mcast_remove_list (struct list_head *remove_list);void ipoib_check_and_add_mcast_sendonly (struct ipoib_dev_priv *priv,u8 *mgid,struct list_head *remove_list);
@@ -591,7 +598,7 @@ void __exit ipoib_netlink_fini(void);void ipoib_set_umcast (struct net_device *ndev,int umcast_val);int ipoib_set_mode (struct net_device *dev,const char *buf); -void ipoib_setup (struct net_device *dev); +void ipoib_setup_common (struct net_device *dev);void ipoib_pkey_open (struct ipoib_dev_priv *priv);void ipoib_drain_cq (struct net_device *dev);
drivers/infiniband/ulp/ipoib/ipoib_ib.c +30 -22
157 lines not on this page · raw message
drivers/infiniband/ulp/ipoib/ipoib_main.c +111 -25
262 lines not on this page · raw message
drivers/infiniband/ulp/ipoib/ipoib_multicast.c +10 -5
@@ -213,6 +213,7 @@ static int ipoib_mcast_join_finish(struct ipoib_mcast *mcast, {struct net_device *dev = mcast->dev;struct ipoib_dev_priv *priv =ipoib_priv (dev); +struct rdma_netdev *rn =netdev_priv (dev);struct ipoib_ah *ah;int ret;int set_qkey =0 ;
@@ -260,8 +261,9 @@ static int ipoib_mcast_join_finish(struct ipoib_mcast *mcast,return 0 ; } - ret =ipoib_mcast_attach (dev,be16_to_cpu (mcast->mcmember.mlid), - &mcast->mcmember.mgid, set_qkey); + ret = rn->attach_mcast (dev, priv->ca, &mcast->mcmember.mgid, +be16_to_cpu (mcast->mcmember.mlid), + set_qkey, priv->qkey);if (ret <0 ) {ipoib_warn (priv,"couldn't attach QP to multicast group %pI6\n" , mcast->mcmember.mgid.raw);
@@ -707,6 +709,7 @@ int ipoib_mcast_stop_thread(struct net_device *dev)static int ipoib_mcast_leave (struct net_device *dev,struct ipoib_mcast *mcast) {struct ipoib_dev_priv *priv =ipoib_priv (dev); +struct rdma_netdev *rn =netdev_priv (dev);int ret =0 ;if (test_and_clear_bit (IPOIB_MCAST_FLAG_BUSY , &mcast->flags))
@@ -720,8 +723,8 @@ static int ipoib_mcast_leave(struct net_device *dev, struct ipoib_mcast *mcast) mcast->mcmember.mgid.raw);/* Remove ourselves from the multicast group */ - ret =ib_detach_mcast (priv->qp, &mcast->mcmember.mgid, -be16_to_cpu (mcast->mcmember.mlid)); + ret = rn->detach_mcast (dev, priv->ca, &mcast->mcmember.mgid, +be16_to_cpu (mcast->mcmember.mlid));if (ret)ipoib_warn (priv,"ib_detach_mcast failed (result = %d)\n" , ret); }else if (!test_bit (IPOIB_MCAST_FLAG_SENDONLY , &mcast->flags))
@@ -763,6 +766,7 @@ void ipoib_mcast_remove_list(struct list_head *remove_list)void ipoib_mcast_send (struct net_device *dev,u8 *daddr,struct sk_buff *skb) {struct ipoib_dev_priv *priv =ipoib_priv (dev); +struct rdma_netdev *rn =netdev_priv (dev);struct ipoib_mcast *mcast;unsigned long flags;void *mgid = daddr +4 ;
@@ -825,7 +829,8 @@ void ipoib_mcast_send(struct net_device *dev, u8 *daddr, struct sk_buff *skb) } }spin_unlock_irqrestore (&priv->lock, flags); -ipoib_send (dev, skb, mcast->ah,IB_MULTICAST_QPN ); + mcast->ah->last_send = rn->send (dev, skb, mcast->ah->ah, +IB_MULTICAST_QPN );if (neigh)ipoib_neigh_put (neigh);return ;
drivers/infiniband/ulp/ipoib/ipoib_netlink.c +1 -1
@@ -161,7 +161,7 @@ static struct rtnl_link_ops ipoib_link_ops __read_mostly = { .maxtype = IFLA_IPOIB_MAX, .policy = ipoib_policy, .priv_size = sizeof(struct ipoib_dev_priv), - .setup = ipoib_setup, + .setup = ipoib_setup_common, .newlink = ipoib_new_child_link, .changelink = ipoib_changelink, .dellink = ipoib_unregister_child_dev,
drivers/infiniband/ulp/ipoib/ipoib_verbs.c +14 -2
@@ -35,7 +35,8 @@#include "ipoib.h" -int ipoib_mcast_attach (struct net_device *dev,u16 mlid,union ib_gid *mgid,int set_qkey) +int ipoib_mcast_attach (struct net_device *dev,struct ib_device *hca, +union ib_gid *mgid,u16 mlid,int set_qkey,u32 qkey) {struct ipoib_dev_priv *priv =ipoib_priv (dev);struct ib_qp_attr *qp_attr =NULL ;
@@ -56,7 +57,7 @@ int ipoib_mcast_attach(struct net_device *dev, u16 mlid, union ib_gid *mgid, int goto out;/* set correct QKey for QP */ - qp_attr->qkey = priv->qkey; + qp_attr->qkey = qkey; ret =ib_modify_qp (priv->qp, qp_attr,IB_QP_QKEY );if (ret) {ipoib_warn (priv,"failed to modify QP, ret = %d\n" , ret);
@@ -74,6 +75,17 @@ int ipoib_mcast_attach(struct net_device *dev, u16 mlid, union ib_gid *mgid, intreturn ret; } +int ipoib_mcast_detach (struct net_device *dev,struct ib_device *hca, +union ib_gid *mgid,u16 mlid) +{ +struct ipoib_dev_priv *priv =ipoib_priv (dev); +int ret; + + ret =ib_detach_mcast (priv->qp, mgid, mlid); + +return ret; +} +int ipoib_init_qp (struct net_device *dev) {struct ipoib_dev_priv *priv =ipoib_priv (dev);
drivers/infiniband/ulp/ipoib/ipoib_vlan.c +3 -2
@@ -128,14 +128,15 @@ int ipoib_vlan_add(struct net_device *pdev, unsigned short pkey)if (!capable (CAP_NET_ADMIN ))return -EPERM ; - ppriv =netdev_priv (pdev); + ppriv =ipoib_priv (pdev);if (test_bit (IPOIB_FLAG_GOING_DOWN , &ppriv->flags))return -EPERM ;snprintf (intf_name,sizeof intf_name,"%s.%04x" , ppriv->dev->name, pkey); - priv =ipoib_intf_alloc (intf_name); + + priv =ipoib_intf_alloc (ppriv->ca, ppriv->port, intf_name);if (!priv)return -ENOMEM ;
Signature · 6 lines
2.12.0 -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
Quoted 31 lines · Leon Romanovsky
Hi Doug,
This patchset mostly comes from Erez with one exception in the first patch.
That patch origins from two different commits, first from Niranjana who added
RDMA netdev interface and second from Erez who added IPoIB support.During the preparation to submission, I squashed their commits into one
and refactored code to allow submission as a standalone topic without creating
dependecies between different submissions. This caused to change in author line.
I hope that it doesn't really matter and it won't stop you from merging it.Per-your request, I based this patch set on v4.11-rc3.
Changes from V0:
----------------------
1.ipoib_privfunction returns structipoib_dev_priv* instead of void
2. Move all the refactoring ofnetdev_privto be in single patch
3. Change the name of the Formation commit.
4. Remove unneeded blank lines
5. Setrn->hcaon creation.
6. Return-EOPNOTSUPPinstead of ENOTSUPP
7. Remove hunk with DMA fromib_devicestruct.
Quoted 5 lines
Thanks,
LeonCC Niranjana Vishwanathapura <niranjana.vishwanathapura-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>CC Alex Vesker <valex-VPRAkNaXOzVWk0Htik3J/[email protected]>
I pulled the mlx5 changes for this from DaveM's net-next tree, then applied the Intel VNIC patches, then applied these patches (which results in conflicts between your first patch and Intel's patches). I hand fixed those conflicts, so I would double check that you are happy with how I laid out the struct elements and what not. Otherwise, thanks, series applied.
Signature · 9 lines
Doug Ledford <[email protected]> GPG KeyID: B826A3330E572FDD Key fingerprint = AE6B 1BDA 122B 23B4 265B 1274 B826 A333 0E57 2FDD -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html