RDMA/rxe: Misc fixes and cleanups
Subsystems (2)
cover RDMA/rxe: Misc fixes and cleanups
This patch set includes several miscellaneous fixes and cleanups developed while getting the rxe driver to pass a 24 hour simulated cable pull fail-over fail-back stress test using Lustre on a 256 node system. These patches apply over for-next with three recently submitted patches as prerequisites:
RDMA/rxe: Fix incomplete state save in rxe_requester RDMA/core: Support drivers use of rcu locking RDMA/rxe: Enable rcu locking of indexed objects
Bob Pearson (9):
RDMA/rxe: Fix handling sleepable in rxe_pool.c
RDMA/rxe: Fix xarray locking in rxe_pool.c
RDMA/rxe: Fix freeing busy objects
RDMA/rxe: Fix delayed send packet handling
RDMA/rxe: Optimize rxe_init_packet in rxe_net.c
RDMA/rxe: Delete unused field elem->list
RDMA/rxe: Add elem->valid field
RDMA/rxe: Report leaked objects
RDMA/rxe: Protect pending send packets
drivers/infiniband/sw/rxe/rxe.c | 26 ++++++ drivers/infiniband/sw/rxe/rxe.h | 3 + drivers/infiniband/sw/rxe/rxe_net.c | 119 +++++++++++++++++--------- drivers/infiniband/sw/rxe/rxe_pool.c | 85 ++++++++++-------- drivers/infiniband/sw/rxe/rxe_pool.h | 9 +- drivers/infiniband/sw/rxe/rxe_qp.c | 1 - drivers/infiniband/sw/rxe/rxe_verbs.c | 86 ++++++------------- 7 files changed, 185 insertions(+), 144 deletions(-)
base-commit: b3d2b014b259ba758d72d7026685091bde1cf2d6
prerequisite-patch-id: c3994e7a93e37e0ce4f50e0c768f3c1a0059a02f
prerequisite-patch-id: 48e13f6ccb560fdeacbd20aaf6696782c23d1190
prerequisite-patch-id: da75fb8eaa863df840e7b392b5048fcc72b0bef3
Signature · 1 line
2.39.2
PATCH 1/9 RDMA/rxe: Fix handling sleepable in rxe_pool.c
Since the AH creation and destroy verbs APIs can be called in interrupt context it is necessary to not sleep in these cases. This was partially dealt with in previous fixes but some cases remain where the code could still potentially sleep.
This patch fixes this by extending the __rxe_finalize() call to have a sleepable parameter and using this in the AH verbs calls. It also fixes one call to rxe_cleanup which did not use the sleepable API.
--- drivers/infiniband/sw/rxe/rxe_pool.c | 5 +++-- drivers/infiniband/sw/rxe/rxe_pool.h | 5 +++-- drivers/infiniband/sw/rxe/rxe_verbs.c | 11 ++++++----- 3 files changed, 12 insertions(+), 9 deletions(-)
drivers/infiniband/sw/rxe/rxe_pool.c +3 -2
@@ -247,10 +247,11 @@ int __rxe_put(struct rxe_pool_elem *elem)return kref_put (&elem->ref_cnt, rxe_elem_release); } -void __rxe_finalize (struct rxe_pool_elem *elem) +void __rxe_finalize (struct rxe_pool_elem *elem,bool sleepable) {void *xa_ret; +gfp_t gfp_flags = sleepable ?GFP_KERNEL :GFP_ATOMIC ; - xa_ret =xa_store (&elem->pool->xa, elem->index, elem,GFP_KERNEL ); + xa_ret =xa_store (&elem->pool->xa, elem->index, elem, gfp_flags);WARN_ON (xa_err (xa_ret)); }
drivers/infiniband/sw/rxe/rxe_pool.h +3 -2
@@ -77,7 +77,8 @@ int __rxe_cleanup(struct rxe_pool_elem *elem, bool sleepable);#define rxe_read (obj) kref_read(&(obj)->elem.ref_cnt) -void __rxe_finalize (struct rxe_pool_elem *elem); -#define rxe_finalize (obj) __rxe_finalize(&(obj)->elem) +void __rxe_finalize (struct rxe_pool_elem *elem,bool sleepable); +#define rxe_finalize (obj) __rxe_finalize(&(obj)->elem, true) +#define rxe_finalize_ah (obj, sleepable) __rxe_finalize(&(obj)->elem, sleepable)#endif /* RXE_POOL_H */
drivers/infiniband/sw/rxe/rxe_verbs.c +6 -5
@@ -265,6 +265,7 @@ static int rxe_create_ah(struct ib_ah *ibah,struct rxe_dev *rxe =to_rdev (ibah->device);struct rxe_ah *ah =to_rah (ibah);struct rxe_create_ah_resp __user *uresp =NULL ; +bool sleepable = init_attr->flags &RDMA_CREATE_AH_SLEEPABLE ;int err, cleanup_err;if (udata) {
@@ -276,8 +277,7 @@ static int rxe_create_ah(struct ib_ah *ibah, ah->is_user = false; } - err =rxe_add_to_pool_ah (&rxe->ah_pool, ah, - init_attr->flags &RDMA_CREATE_AH_SLEEPABLE ); + err =rxe_add_to_pool_ah (&rxe->ah_pool, ah, sleepable);if (err) {rxe_dbg_dev (rxe,"unable to create ah" ); goto err_out;
@@ -307,12 +307,12 @@ static int rxe_create_ah(struct ib_ah *ibah, }rxe_init_av (init_attr->ah_attr, &ah->av); -rxe_finalize (ah); +rxe_finalize_ah (ah, sleepable);return 0 ; err_cleanup: - cleanup_err =rxe_cleanup (ah); + cleanup_err =rxe_cleanup_ah (ah, sleepable);if (cleanup_err)rxe_err_ah (ah,"cleanup failed, err = %d" , cleanup_err); err_out:
@@ -354,9 +354,10 @@ static int rxe_query_ah(struct ib_ah *ibah, struct rdma_ah_attr *attr)static int rxe_destroy_ah (struct ib_ah *ibah,u32 flags) {struct rxe_ah *ah =to_rah (ibah); +bool sleepable = flags &RDMA_DESTROY_AH_SLEEPABLE ;int err; - err =rxe_cleanup_ah (ah, flags &RDMA_DESTROY_AH_SLEEPABLE ); + err =rxe_cleanup_ah (ah, sleepable);if (err)rxe_err_ah (ah,"cleanup failed, err = %d" , err);
Signature · 1 line
2.39.2
Jason Gunthorpe
maintainerRe: [PATCH for-next 1/9] RDMA/rxe: Fix handling sleepable in rxe_pool.c
Quoted 26 lines · Bob Pearson
Since the AH creation and destroy verbs APIs can be called in interrupt context it is necessary to not sleep in these cases. This was partially dealt with in previous fixes but some cases remain where the code could still potentially sleep.
This patch fixes this by extending the
__rxe_finalize()call to have a sleepable parameter and using this in the AH verbs calls. It also fixes one call torxe_cleanupwhich did not use the sleepable API.Fixes 215d0a755e1b ("RDMA/rxe: Stop lookup of partially built objects")Signed-off-by Bob Pearson <[email protected]>--- drivers/infiniband/sw/rxe/rxe_pool.c | 5 +++-- drivers/infiniband/sw/rxe/rxe_pool.h | 5 +++-- drivers/infiniband/sw/rxe/rxe_verbs.c | 11 ++++++----- 3 files changed, 12 insertions(+), 9 deletions(-)drivers/infiniband/sw/rxe/rxe_pool.c +0
-0@@ -247,10 +247,11 @@ int __rxe_put(struct rxe_pool_elem *elem)return kref_put (&elem->ref_cnt, rxe_elem_release); }
drivers/infiniband/sw/rxe/rxe_pool.c int __rxe_put(struct rxe_pool_elem *elem) · line 257
Quoted 10 lines
drivers/infiniband/sw/rxe/rxe_pool.c +3
-2-void __rxe_finalize (struct rxe_pool_elem *elem) +void __rxe_finalize (struct rxe_pool_elem *elem,bool sleepable) {void *xa_ret; +gfp_t gfp_flags = sleepable ?GFP_KERNEL :GFP_ATOMIC ; - xa_ret =xa_store (&elem->pool->xa, elem->index, elem,GFP_KERNEL ); + xa_ret =xa_store (&elem->pool->xa, elem->index, elem, gfp_flags);WARN_ON (xa_err (xa_ret)); }
You don't need to do this, there is no allocation here, when you call xa_alloc_cyclic() it reserved all the memory necessary for this.
Just add a comment and mark it GFP_ATOMIC.
Jason
PATCH 2/9 RDMA/rxe: Fix xarray locking in rxe_pool.c
In reply to Bob PearsonThe kernel rdma verbs API calls are not documented to require that the caller be in process context and it is well known that the address handle calls sometimes are in interrupt context while the other ones are in process context. If we replace the xarray spin_lock in the rxe_pool APIs by irqsave spin_locks we make most of the verbs API calls safe in any context.
This patch replaces the xa locks with xa_lock_irqsave.
--- drivers/infiniband/sw/rxe/rxe_pool.c | 21 ++++++++++++++++----- 1 file changed, 16 insertions(+), 5 deletions(-)
drivers/infiniband/sw/rxe/rxe_pool.c +16 -5
@@ -119,8 +119,10 @@ void rxe_pool_cleanup(struct rxe_pool *pool)int __rxe_add_to_pool (struct rxe_pool *pool,struct rxe_pool_elem *elem,bool sleepable) { -int err; +struct xarray *xa = &pool->xa; +unsigned long flags;gfp_t gfp_flags; +int err;if (atomic_inc_return (&pool->num_elem) > pool->max_elem) goto err_cnt;
@@ -138,8 +140,10 @@ int __rxe_add_to_pool(struct rxe_pool *pool, struct rxe_pool_elem *elem,if (sleepable)might_sleep (); - err =xa_alloc_cyclic (&pool->xa, &elem->index,NULL , pool->limit, +xa_lock_irqsave (xa, flags); + err =__xa_alloc_cyclic (xa, &elem->index,NULL , pool->limit, &pool->next, gfp_flags); +xa_unlock_irqrestore (xa, flags);if (err <0 ) goto err_cnt;
@@ -179,6 +183,7 @@ int __rxe_cleanup(struct rxe_pool_elem *elem, bool sleepable)struct rxe_pool *pool = elem->pool;struct xarray *xa = &pool->xa;static int timeout =RXE_POOL_TIMEOUT ; +unsigned long flags;int ret, err =0 ;void *xa_ret;
@@ -188,7 +193,9 @@ int __rxe_cleanup(struct rxe_pool_elem *elem, bool sleepable)/* erase xarray entry to prevent looking up * the pool elem from its index */ - xa_ret =xa_erase (xa, elem->index); +xa_lock_irqsave (xa, flags); + xa_ret =__xa_erase (xa, elem->index); +xa_unlock_irqrestore (xa, flags);WARN_ON (xa_err (xa_ret));/* if this is the last call to rxe_put complete the
@@ -249,9 +256,13 @@ int __rxe_put(struct rxe_pool_elem *elem)void __rxe_finalize (struct rxe_pool_elem *elem,bool sleepable) { -void *xa_ret;gfp_t gfp_flags = sleepable ?GFP_KERNEL :GFP_ATOMIC ; +struct xarray *xa = &elem->pool->xa; +unsigned long flags; +void *xa_ret; - xa_ret =xa_store (&elem->pool->xa, elem->index, elem, gfp_flags); +xa_lock_irqsave (xa, flags); + xa_ret =__xa_store (xa, elem->index, elem, gfp_flags); +xa_unlock_irqrestore (xa, flags);WARN_ON (xa_err (xa_ret)); }
Signature · 1 line
2.39.2
PATCH 3/9 RDMA/rxe: Fix freeing busy objects
In reply to Bob PearsonCurrently the rxe driver calls wait_for_completion_timeout() in rxe_complete() to wait for the references to the object to be freed before final cleanup and returning to the rdma-core destroy verb. If this does not happen within the timeout interval it prints a WARN_ON and returns to the 'destroy' verb caller without any indication that there was an error. This is incorrect.
A heavily loaded system can take an arbitrarily long time to complete the work needed before freeing all the references with no guarantees of performance within a specific time.
Another frequent cause of these timeouts is due to ref counting bugs introduced by changes in the driver so it is helpful to report the timeouts if they occur.
This patch changes the type of the rxe_cleanup() subroutine to void and fixes calls to reflect this API change in rxe_verbs.c. This is better aligned with the code in rdma-core which sometimes fails to check the return value of destroy verb calls assuming they will always succeed. Specifically this is the case for kernel qp's.
Not able to return an error, this patch puts the completion timeout or busy wait code into a subroutine called wait_until_done(). And places the call in a loop with a 10 second timeout that issues a WARN_ON each pass through the loop. This is slow enough to not overload the logs.
--- drivers/infiniband/sw/rxe/rxe_pool.c | 39 ++++++------- drivers/infiniband/sw/rxe/rxe_pool.h | 2 +- drivers/infiniband/sw/rxe/rxe_verbs.c | 79 +++++++-------------------- 3 files changed, 38 insertions(+), 82 deletions(-)
drivers/infiniband/sw/rxe/rxe_pool.c +16 -23
@@ -6,7 +6,7 @@#include "rxe.h" -#define RXE_POOL_TIMEOUT (200) +#define RXE_POOL_TIMEOUT (10000)/* 10 seconds */ #define RXE_POOL_ALIGN (16) static const struct rxe_type_info {
@@ -175,16 +175,17 @@ static void rxe_elem_release(struct kref *kref) {struct rxe_pool_elem *elem =container_of (kref,typeof (*elem), ref_cnt); -complete (&elem->complete); +complete_all (&elem->complete); } -int __rxe_cleanup (struct rxe_pool_elem *elem,bool sleepable) +void __rxe_cleanup (struct rxe_pool_elem *elem,bool sleepable) {struct rxe_pool *pool = elem->pool;struct xarray *xa = &pool->xa; -static int timeout =RXE_POOL_TIMEOUT ; +int timeout =RXE_POOL_TIMEOUT ; +unsigned long until;unsigned long flags; -int ret, err =0 ; +int ret;void *xa_ret;if (sleepable)
@@ -209,39 +210,31 @@ int __rxe_cleanup(struct rxe_pool_elem *elem, bool sleepable)* return to rdma-core */ if (sleepable) { -if (!completion_done (&elem->complete) && timeout) { +while (!completion_done (&elem->complete) && timeout) { ret =wait_for_completion_timeout (&elem->complete, timeout); - -/* Shouldn't happen. There are still references to -* the object but, rather than deadlock, free the -* object or pass back to rdma-core. -*/ -if (WARN_ON (!ret)) - err = -EINVAL ; +WARN_ON (!ret); } }else { -unsigned long until = jiffies + timeout; -/* AH objects are unique in that the destroy_ah verb * can be called in atomic context. This delay * replaces the wait_for_completion call above * when the destroy_ah call is not sleepable */ -while (!completion_done (&elem->complete) && -time_before (jiffies, until)) -mdelay (1 ); - -if (WARN_ON (!completion_done (&elem->complete))) - err = -EINVAL ; +while (!completion_done (&elem->complete) && timeout) { + until = jiffies + timeout; +while (!completion_done (&elem->complete) && +time_before (jiffies, until)) { +mdelay (10 ); + } +WARN_ON (!completion_done (&elem->complete)); + } }if (pool->cleanup) pool->cleanup (elem);atomic_dec (&pool->num_elem); - -return err; }int __rxe_get (struct rxe_pool_elem *elem)
drivers/infiniband/sw/rxe/rxe_pool.h +1 -1
@@ -71,7 +71,7 @@ int __rxe_get(struct rxe_pool_elem *elem);int __rxe_put (struct rxe_pool_elem *elem);#define rxe_put (obj) __rxe_put(&(obj)->elem) -int __rxe_cleanup (struct rxe_pool_elem *elem,bool sleepable); +void __rxe_cleanup (struct rxe_pool_elem *elem,bool sleepable);#define rxe_cleanup (obj) __rxe_cleanup(&(obj)->elem, true)#define rxe_cleanup_ah (obj, sleepable) __rxe_cleanup(&(obj)->elem, sleepable)
drivers/infiniband/sw/rxe/rxe_verbs.c +21 -58
204 lines not on this page · raw message
Signature · 1 line
2.39.2
Jason Gunthorpe
maintainerRe: [PATCH for-next 3/9] RDMA/rxe: Fix freeing busy objects
drivers/infiniband/sw/rxe/rxe_pool.c static void rxe_elem_release(struct kref *kref) · line 182
Quoted 11 lines · Bob Pearson
drivers/infiniband/sw/rxe/rxe_pool.c +2
-2@@ -175,16 +175,17 @@ static void rxe_elem_release(struct kref *kref) {struct rxe_pool_elem *elem =container_of (kref,typeof (*elem), ref_cnt); -complete (&elem->complete); +complete_all (&elem->complete); } -int __rxe_cleanup (struct rxe_pool_elem *elem,bool sleepable) +void __rxe_cleanup (struct rxe_pool_elem *elem,bool sleepable) {
You should definately put this one change in its own patch doing just that.
Jason
Re: [PATCH for-next 3/9] RDMA/rxe: Fix freeing busy objects
Quoted 9 lines · Jason Gunthorpe
Quoted 8 lines · Bob Pearson
drivers/infiniband/sw/rxe/rxe_pool.c +1
-1@@ -175,16 +175,17 @@ static void rxe_elem_release(struct kref *kref) {struct rxe_pool_elem *elem =container_of (kref,typeof (*elem), ref_cnt); -complete (&elem->complete); +complete_all (&elem->complete); }
Quoted 8 lines
drivers/infiniband/sw/rxe/rxe_pool.c static void rxe_elem_release(struct kref *kref) · line 182
Quoted 3 lines
drivers/infiniband/sw/rxe/rxe_pool.c +1
-1-int __rxe_cleanup (struct rxe_pool_elem *elem,bool sleepable) +void __rxe_cleanup (struct rxe_pool_elem *elem,bool sleepable) {You should definately put this one change in its own patch doing just that.
Jason
Please explain a little more. I only found this change because the change in rxe_complete to repeat the wait_for_completion call didn't work. This seemed to fix it but the man page and comments didn't explain complete_all very well.
Bob
Jason Gunthorpe
maintainerRe: [PATCH for-next 3/9] RDMA/rxe: Fix freeing busy objects
Quoted 17 lines · Bob Pearson
Quoted 16 lines · Jason Gunthorpe
drivers/infiniband/sw/rxe/rxe_pool.c static void rxe_elem_release(struct kref *kref) · line 182
Quoted 11 lines · Bob Pearson
drivers/infiniband/sw/rxe/rxe_pool.c +2
-2@@ -175,16 +175,17 @@ static void rxe_elem_release(struct kref *kref) {struct rxe_pool_elem *elem =container_of (kref,typeof (*elem), ref_cnt); -complete (&elem->complete); +complete_all (&elem->complete); } -int __rxe_cleanup (struct rxe_pool_elem *elem,bool sleepable) +void __rxe_cleanup (struct rxe_pool_elem *elem,bool sleepable) {You should definately put this one change in its own patch doing just that.
Quoted 3 lines
Quoted 1 line
Jason
Please explain a little more.
I mean just remove the function return in one patch, these functions are not supposed to fail anyhow.
Quoted 4 lines
I only found this change because the change in
rxe_completeto repeat thewait_for_completioncall didn't work. This seemed to fix it but the man page and comments didn't explaincomplete_allvery well.
I didn't mean the complete_all hunk, I ment the __rxe_cleanup hunk :)
Jason
PATCH 4/9 RDMA/rxe: Fix delayed send packet handling
In reply to Bob PearsonIn cable pull testing some NICs can hold a send packet long enough to allow ulp protocol stacks to destroy the qp and the cleanup routines to timeout waiting for all qp references to be released. When the NIC driver finally frees the SKB the qp pointer is no longer valid and causes a seg fault in rxe_skb_tx_dtor().
This patch passes the qp index instead of the qp to the skb destructor callback function. The call back is required to lookup the qp from the index and if it has been destroyed the lookup will return NULL and the qp will not be referenced avoiding the seg fault.
--- drivers/infiniband/sw/rxe/rxe_net.c | 87 ++++++++++++++++++++++------- drivers/infiniband/sw/rxe/rxe_qp.c | 1 - 2 files changed, 67 insertions(+), 21 deletions(-)
drivers/infiniband/sw/rxe/rxe_net.c +67 -20
@@ -131,19 +131,28 @@ static struct dst_entry *rxe_find_route(struct net_device *ndev,return dst; } +static struct rxe_dev *get_rxe_from_skb (struct sk_buff *skb) +{ +struct rxe_dev *rxe; +struct net_device *ndev = skb->dev; + + rxe =rxe_get_dev_from_net (ndev); +if (!rxe &&is_vlan_dev (ndev)) + rxe =rxe_get_dev_from_net (vlan_dev_real_dev (ndev)); + +return rxe; +} +static int rxe_udp_encap_recv (struct sock *sk,struct sk_buff *skb) {struct udphdr *udph;struct rxe_dev *rxe; -struct net_device *ndev = skb->dev;struct rxe_pkt_info *pkt =SKB_TO_PKT (skb);/* takes a reference on rxe->ib_dev * drop when skb is freed */ - rxe =rxe_get_dev_from_net (ndev); -if (!rxe &&is_vlan_dev (ndev)) - rxe =rxe_get_dev_from_net (vlan_dev_real_dev (ndev)); + rxe =get_rxe_from_skb (skb);if (!rxe) goto drop;
@@ -345,46 +354,84 @@ int rxe_prepare(struct rxe_av *av, struct rxe_pkt_info *pkt,static void rxe_skb_tx_dtor (struct sk_buff *skb) { -struct sock *sk = skb->sk; -struct rxe_qp *qp = sk->sk_user_data; -int skb_out =atomic_dec_return (&qp->skb_out); +struct rxe_dev *rxe; +unsigned int index; +struct rxe_qp *qp; +int skb_out; + +/* takes a ref on ib device if success */ + rxe =get_rxe_from_skb (skb); +if (!rxe) + goto out; + +/* recover source qp index from sk->sk_user_data +* free the reference taken in rxe_send +*/ + index = (int )(uintptr_t )skb->sk->sk_user_data; +sock_put (skb->sk); +/* lookup qp from index, takes a ref on success */ + qp =rxe_pool_get_index (&rxe->qp_pool, index); +if (!qp) + goto out_put_ibdev; + + skb_out =atomic_dec_return (&qp->skb_out);if (unlikely (qp->need_req_skb && skb_out <RXE_INFLIGHT_SKBS_PER_QP_LOW ))rxe_sched_task (&qp->req.task);rxe_put (qp); +out_put_ibdev: +ib_device_put (&rxe->ib_dev); +out: +return ; }static int rxe_send (struct sk_buff *skb,struct rxe_pkt_info *pkt) { +struct rxe_qp *qp = pkt->qp; +struct sock *sk;int err; - skb->destructor = rxe_skb_tx_dtor; - skb->sk = pkt->qp->sk->sk; +/* qp can be destroyed while this packet is waiting on +* the tx queue. So need to protect sk. +*/ + sk = qp->sk->sk; +sock_hold (sk); + skb->sk = sk; -rxe_get (pkt->qp);atomic_inc (&pkt->qp->skb_out); + sk->sk_user_data = (void *)(long )qp->elem.index; + skb->destructor = rxe_skb_tx_dtor; +if (skb->protocol ==htons (ETH_P_IP )) { - err =ip_local_out (dev_net (skb_dst (skb)->dev), skb->sk, skb); + err =ip_local_out (dev_net (skb_dst (skb)->dev), sk, skb); }else if (skb->protocol ==htons (ETH_P_IPV6 )) { - err =ip6_local_out (dev_net (skb_dst (skb)->dev), skb->sk, skb); + err =ip6_local_out (dev_net (skb_dst (skb)->dev), sk, skb); }else { -rxe_dbg_qp (pkt->qp,"Unknown layer 3 protocol: %d\n" , - skb->protocol); -atomic_dec (&pkt->qp->skb_out); -rxe_put (pkt->qp); -kfree_skb (skb); -return -EINVAL ; +rxe_dbg_qp (qp,"Unknown layer 3 protocol: %d" , + skb->protocol); + err = -EINVAL ; + goto err_not_sent; } +/* IP consumed the packet, the destructor will handle cleanup */ if (unlikely (net_xmit_eval (err))) { -rxe_dbg_qp (pkt->qp,"error sending packet: %d\n" , err); -return -EAGAIN ; +rxe_dbg_qp (qp,"Error sending packet: %d" , err); + err = -EAGAIN ; + goto err_out; }return 0 ; + +err_not_sent: + skb->destructor =NULL ; +atomic_dec (&pkt->qp->skb_out); +kfree_skb (skb); +sock_put (sk); +err_out: +return err; }/* fix up a send packet to match the packets
drivers/infiniband/sw/rxe/rxe_qp.c +0 -1
@@ -194,7 +194,6 @@ static int rxe_qp_init_req(struct rxe_dev *rxe, struct rxe_qp *qp, err =sock_create_kern (&init_net,AF_INET ,SOCK_DGRAM ,0 , &qp->sk);if (err <0 )return err; - qp->sk->sk->sk_user_data = qp;/* pick a source UDP port number for this QP based on * the source QPN. this spreads traffic for different QPs
Signature · 1 line
2.39.2
Zhu Yanjun
maintainerRe: [PATCH for-next 4/9] RDMA/rxe: Fix delayed send packet handling
Quoted 6 lines · Bob Pearson
In cable pull testing some NICs can hold a send packet long enough to allow ulp protocol stacks to destroy the qp and the cleanup routines to timeout waiting for all qp references to be released. When the NIC driver finally frees the SKB the qp pointer is no longer valid and causes a seg fault in
rxe_skb_tx_dtor().
If a packet is held in some NICs, the later packets of this packet will also be held by this NIC. So this will cause QP not to work well. This is a serious problem. And the problem should be fixed in this kind of NICs. We should not fix it in RXE.
And a QP exists for at least several seconds. A packet can be held in NIC for such a long time? This problem does exist in the real world, or you imagine this scenario?
I can not imagine this kind of scenario. Please Jason or Leon also check this.
Thanks.
Zhu Yanjun
Quoted 166 lines
This patch passes the qp index instead of the qp to the skb destructor callback function. The call back is required to lookup the qp from the index and if it has been destroyed the lookup will return NULL and the qp will not be referenced avoiding the seg fault.
Fixes 8700e3e7c485 ("Soft RoCE driver")Signed-off-by Bob Pearson <[email protected]>--- drivers/infiniband/sw/rxe/rxe_net.c | 87 ++++++++++++++++++++++------- drivers/infiniband/sw/rxe/rxe_qp.c | 1 - 2 files changed, 67 insertions(+), 21 deletions(-)drivers/infiniband/sw/rxe/rxe_net.c +67
-20@@ -131,19 +131,28 @@ static struct dst_entry *rxe_find_route(struct net_device *ndev,return dst; } +static struct rxe_dev *get_rxe_from_skb (struct sk_buff *skb) +{ +struct rxe_dev *rxe; +struct net_device *ndev = skb->dev; + + rxe =rxe_get_dev_from_net (ndev); +if (!rxe &&is_vlan_dev (ndev)) + rxe =rxe_get_dev_from_net (vlan_dev_real_dev (ndev)); + +return rxe; +} +static int rxe_udp_encap_recv (struct sock *sk,struct sk_buff *skb) {struct udphdr *udph;struct rxe_dev *rxe; -struct net_device *ndev = skb->dev;struct rxe_pkt_info *pkt =SKB_TO_PKT (skb);/* takes a reference on rxe->ib_dev * drop when skb is freed */ - rxe =rxe_get_dev_from_net (ndev); -if (!rxe &&is_vlan_dev (ndev)) - rxe =rxe_get_dev_from_net (vlan_dev_real_dev (ndev)); + rxe =get_rxe_from_skb (skb);if (!rxe) goto drop;@@ -345,46 +354,84 @@ int rxe_prepare(struct rxe_av *av, struct rxe_pkt_info *pkt,static void rxe_skb_tx_dtor (struct sk_buff *skb) { -struct sock *sk = skb->sk; -struct rxe_qp *qp = sk->sk_user_data; -int skb_out =atomic_dec_return (&qp->skb_out); +struct rxe_dev *rxe; +unsigned int index; +struct rxe_qp *qp; +int skb_out; + +/* takes a ref on ib device if success */ + rxe =get_rxe_from_skb (skb); +if (!rxe) + goto out; + +/* recover source qp index from sk->sk_user_data +* free the reference taken in rxe_send +*/ + index = (int )(uintptr_t )skb->sk->sk_user_data; +sock_put (skb->sk); +/* lookup qp from index, takes a ref on success */ + qp =rxe_pool_get_index (&rxe->qp_pool, index); +if (!qp) + goto out_put_ibdev; + + skb_out =atomic_dec_return (&qp->skb_out);if (unlikely (qp->need_req_skb && skb_out <RXE_INFLIGHT_SKBS_PER_QP_LOW ))rxe_sched_task (&qp->req.task);rxe_put (qp); +out_put_ibdev: +ib_device_put (&rxe->ib_dev); +out: +return ; }static int rxe_send (struct sk_buff *skb,struct rxe_pkt_info *pkt) { +struct rxe_qp *qp = pkt->qp; +struct sock *sk;int err; - skb->destructor = rxe_skb_tx_dtor; - skb->sk = pkt->qp->sk->sk; +/* qp can be destroyed while this packet is waiting on +* the tx queue. So need to protect sk. +*/ + sk = qp->sk->sk; +sock_hold (sk); + skb->sk = sk; -rxe_get (pkt->qp);atomic_inc (&pkt->qp->skb_out); + sk->sk_user_data = (void *)(long )qp->elem.index; + skb->destructor = rxe_skb_tx_dtor; +if (skb->protocol ==htons (ETH_P_IP )) { - err =ip_local_out (dev_net (skb_dst (skb)->dev), skb->sk, skb); + err =ip_local_out (dev_net (skb_dst (skb)->dev), sk, skb); }else if (skb->protocol ==htons (ETH_P_IPV6 )) { - err =ip6_local_out (dev_net (skb_dst (skb)->dev), skb->sk, skb); + err =ip6_local_out (dev_net (skb_dst (skb)->dev), sk, skb); }else { -rxe_dbg_qp (pkt->qp,"Unknown layer 3 protocol: %d\n" , - skb->protocol); -atomic_dec (&pkt->qp->skb_out); -rxe_put (pkt->qp); -kfree_skb (skb); -return -EINVAL ; +rxe_dbg_qp (qp,"Unknown layer 3 protocol: %d" , + skb->protocol); + err = -EINVAL ; + goto err_not_sent; } +/* IP consumed the packet, the destructor will handle cleanup */ if (unlikely (net_xmit_eval (err))) { -rxe_dbg_qp (pkt->qp,"error sending packet: %d\n" , err); -return -EAGAIN ; +rxe_dbg_qp (qp,"Error sending packet: %d" , err); + err = -EAGAIN ; + goto err_out; }return 0 ; + +err_not_sent: + skb->destructor =NULL ; +atomic_dec (&pkt->qp->skb_out); +kfree_skb (skb); +sock_put (sk); +err_out: +return err; }/* fix up a send packet to match the packets drivers/infiniband/sw/rxe/rxe_qp.c +0
-2@@ -194,7 +194,6 @@ static int rxe_qp_init_req(struct rxe_dev *rxe, struct rxe_qp *qp, err =sock_create_kern (&init_net,AF_INET ,SOCK_DGRAM ,0 , &qp->sk);if (err <0 )return err; - qp->sk->sk->sk_user_data = qp;/* pick a source UDP port number for this QP based on * the source QPN. this spreads traffic for different QPs -- 2.39.2
Re: [PATCH for-next 4/9] RDMA/rxe: Fix delayed send packet handling
Quoted 8 lines · Zhu Yanjun
Quoted 6 lines · Bob Pearson
In cable pull testing some NICs can hold a send packet long enough to allow ulp protocol stacks to destroy the qp and the cleanup routines to timeout waiting for all qp references to be released. When the NIC driver finally frees the SKB the qp pointer is no longer valid and causes a seg fault in
rxe_skb_tx_dtor().
Quoted 8 lines
If a packet is held in some NICs, the later packets of this packet will also be held by this NIC. So this will cause QP not to work well. This is a serious problem. And the problem should be fixed in this kind of NICs. We should not fix it in RXE.
And a QP exists for at least several seconds. A packet can be held in NIC for such a long time? This problem does exist in the real world, or you imagine this scenario?
This was a real bug observed in a 24 hour stress test where we were running IOR traffic over Lustre on a 256 node cluster. Then we ran a test that randomly disabled links for 10-20 second and then reenbled them. Lustre will automatically detect the down links and reroute the storage traffic to a new RC connection and then when the link recovers it will reestablish RC connections on the link. The failover is triggered by Lustre timing out on an IO operation and it then tears down the QPs. When the link is back up the tx queue is finally processed by the NIC and the skbs are freed in the sender calling the rxe skb destructor function which attempts to update a counter in the QP state. During the test we would observe 2-3 kernel panics from referencing the destroyed qp pointer. We made this change to avoid delaying the qp destroy call for an arbitrarily long time and instead took a reference on the struct sock and used the qpn instead of the qp pointer. There is a later patch in this series that fixes a related but different problem which can occur at the end of the test if someone attempts to rmmod the rxe driver while a packet is still pending. In this case it the code in the destructor function which is gone not the QP. But still need to protect with ref counts.
Bob
Quoted 171 lines
I can not imagine this kind of scenario. Please Jason or Leon also check this.
Thanks.
Zhu YanjunQuoted 166 lines
This patch passes the qp index instead of the qp to the skb destructor callback function. The call back is required to lookup the qp from the index and if it has been destroyed the lookup will return NULL and the qp will not be referenced avoiding the seg fault.
Fixes 8700e3e7c485 ("Soft RoCE driver")Signed-off-by Bob Pearson <[email protected]>--- drivers/infiniband/sw/rxe/rxe_net.c | 87 ++++++++++++++++++++++------- drivers/infiniband/sw/rxe/rxe_qp.c | 1 - 2 files changed, 67 insertions(+), 21 deletions(-)drivers/infiniband/sw/rxe/rxe_net.c +67
-20@@ -131,19 +131,28 @@ static struct dst_entry *rxe_find_route(struct net_device *ndev,return dst; } +static struct rxe_dev *get_rxe_from_skb (struct sk_buff *skb) +{ +struct rxe_dev *rxe; +struct net_device *ndev = skb->dev; + + rxe =rxe_get_dev_from_net (ndev); +if (!rxe &&is_vlan_dev (ndev)) + rxe =rxe_get_dev_from_net (vlan_dev_real_dev (ndev)); + +return rxe; +} +static int rxe_udp_encap_recv (struct sock *sk,struct sk_buff *skb) {struct udphdr *udph;struct rxe_dev *rxe; -struct net_device *ndev = skb->dev;struct rxe_pkt_info *pkt =SKB_TO_PKT (skb);/* takes a reference on rxe->ib_dev * drop when skb is freed */ - rxe =rxe_get_dev_from_net (ndev); -if (!rxe &&is_vlan_dev (ndev)) - rxe =rxe_get_dev_from_net (vlan_dev_real_dev (ndev)); + rxe =get_rxe_from_skb (skb);if (!rxe) goto drop;@@ -345,46 +354,84 @@ int rxe_prepare(struct rxe_av *av, struct rxe_pkt_info *pkt,static void rxe_skb_tx_dtor (struct sk_buff *skb) { -struct sock *sk = skb->sk; -struct rxe_qp *qp = sk->sk_user_data; -int skb_out =atomic_dec_return (&qp->skb_out); +struct rxe_dev *rxe; +unsigned int index; +struct rxe_qp *qp; +int skb_out; + +/* takes a ref on ib device if success */ + rxe =get_rxe_from_skb (skb); +if (!rxe) + goto out; + +/* recover source qp index from sk->sk_user_data +* free the reference taken in rxe_send +*/ + index = (int )(uintptr_t )skb->sk->sk_user_data; +sock_put (skb->sk); +/* lookup qp from index, takes a ref on success */ + qp =rxe_pool_get_index (&rxe->qp_pool, index); +if (!qp) + goto out_put_ibdev; + + skb_out =atomic_dec_return (&qp->skb_out);if (unlikely (qp->need_req_skb && skb_out <RXE_INFLIGHT_SKBS_PER_QP_LOW ))rxe_sched_task (&qp->req.task);rxe_put (qp); +out_put_ibdev: +ib_device_put (&rxe->ib_dev); +out: +return ; }static int rxe_send (struct sk_buff *skb,struct rxe_pkt_info *pkt) { +struct rxe_qp *qp = pkt->qp; +struct sock *sk;int err; - skb->destructor = rxe_skb_tx_dtor; - skb->sk = pkt->qp->sk->sk; +/* qp can be destroyed while this packet is waiting on +* the tx queue. So need to protect sk. +*/ + sk = qp->sk->sk; +sock_hold (sk); + skb->sk = sk; -rxe_get (pkt->qp);atomic_inc (&pkt->qp->skb_out); + sk->sk_user_data = (void *)(long )qp->elem.index; + skb->destructor = rxe_skb_tx_dtor; +if (skb->protocol ==htons (ETH_P_IP )) { - err =ip_local_out (dev_net (skb_dst (skb)->dev), skb->sk, skb); + err =ip_local_out (dev_net (skb_dst (skb)->dev), sk, skb); }else if (skb->protocol ==htons (ETH_P_IPV6 )) { - err =ip6_local_out (dev_net (skb_dst (skb)->dev), skb->sk, skb); + err =ip6_local_out (dev_net (skb_dst (skb)->dev), sk, skb); }else { -rxe_dbg_qp (pkt->qp,"Unknown layer 3 protocol: %d\n" , - skb->protocol); -atomic_dec (&pkt->qp->skb_out); -rxe_put (pkt->qp); -kfree_skb (skb); -return -EINVAL ; +rxe_dbg_qp (qp,"Unknown layer 3 protocol: %d" , + skb->protocol); + err = -EINVAL ; + goto err_not_sent; } +/* IP consumed the packet, the destructor will handle cleanup */ if (unlikely (net_xmit_eval (err))) { -rxe_dbg_qp (pkt->qp,"error sending packet: %d\n" , err); -return -EAGAIN ; +rxe_dbg_qp (qp,"Error sending packet: %d" , err); + err = -EAGAIN ; + goto err_out; }return 0 ; + +err_not_sent: + skb->destructor =NULL ; +atomic_dec (&pkt->qp->skb_out); +kfree_skb (skb); +sock_put (sk); +err_out: +return err; }/* fix up a send packet to match the packets drivers/infiniband/sw/rxe/rxe_qp.c +0
-2@@ -194,7 +194,6 @@ static int rxe_qp_init_req(struct rxe_dev *rxe, struct rxe_qp *qp, err =sock_create_kern (&init_net,AF_INET ,SOCK_DGRAM ,0 , &qp->sk);if (err <0 )return err; - qp->sk->sk->sk_user_data = qp;/* pick a source UDP port number for this QP based on * the source QPN. this spreads traffic for different QPs -- 2.39.2
Re: [PATCH for-next 4/9] RDMA/rxe: Fix delayed send packet handling
In reply to Zhu YanjunQuoted 13 lines · Zhu Yanjun
Quoted 6 lines · Bob Pearson
In cable pull testing some NICs can hold a send packet long enough to allow ulp protocol stacks to destroy the qp and the cleanup routines to timeout waiting for all qp references to be released. When the NIC driver finally frees the SKB the qp pointer is no longer valid and causes a seg fault in
rxe_skb_tx_dtor().If a packet is held in some NICs, the later packets of this packet will also be held by this NIC. So this will cause QP not to work well. This is a serious problem. And the problem should be fixed in this kind of NICs. We should not fix it in RXE.
Quoted 5 lines
And a QP exists for at least several seconds. A packet can be held in NIC for such a long time? This problem does exist in the real world, or you imagine this scenario?
I can not imagine this kind of scenario. Please Jason or Leon also check this.
I stopped to follow RXE changes for a long time. They don't make any sense to me. Even this patch, which moves existing IB reference counter from one place to another and does it for every SKB is beyond my understanding.
Thanks
Quoted 169 lines
Thanks.
Zhu YanjunQuoted 166 lines
This patch passes the qp index instead of the qp to the skb destructor callback function. The call back is required to lookup the qp from the index and if it has been destroyed the lookup will return NULL and the qp will not be referenced avoiding the seg fault.
Fixes 8700e3e7c485 ("Soft RoCE driver")Signed-off-by Bob Pearson <[email protected]>--- drivers/infiniband/sw/rxe/rxe_net.c | 87 ++++++++++++++++++++++------- drivers/infiniband/sw/rxe/rxe_qp.c | 1 - 2 files changed, 67 insertions(+), 21 deletions(-)drivers/infiniband/sw/rxe/rxe_net.c +67
-20@@ -131,19 +131,28 @@ static struct dst_entry *rxe_find_route(struct net_device *ndev,return dst; } +static struct rxe_dev *get_rxe_from_skb (struct sk_buff *skb) +{ +struct rxe_dev *rxe; +struct net_device *ndev = skb->dev; + + rxe =rxe_get_dev_from_net (ndev); +if (!rxe &&is_vlan_dev (ndev)) + rxe =rxe_get_dev_from_net (vlan_dev_real_dev (ndev)); + +return rxe; +} +static int rxe_udp_encap_recv (struct sock *sk,struct sk_buff *skb) {struct udphdr *udph;struct rxe_dev *rxe; -struct net_device *ndev = skb->dev;struct rxe_pkt_info *pkt =SKB_TO_PKT (skb);/* takes a reference on rxe->ib_dev * drop when skb is freed */ - rxe =rxe_get_dev_from_net (ndev); -if (!rxe &&is_vlan_dev (ndev)) - rxe =rxe_get_dev_from_net (vlan_dev_real_dev (ndev)); + rxe =get_rxe_from_skb (skb);if (!rxe) goto drop;@@ -345,46 +354,84 @@ int rxe_prepare(struct rxe_av *av, struct rxe_pkt_info *pkt,static void rxe_skb_tx_dtor (struct sk_buff *skb) { -struct sock *sk = skb->sk; -struct rxe_qp *qp = sk->sk_user_data; -int skb_out =atomic_dec_return (&qp->skb_out); +struct rxe_dev *rxe; +unsigned int index; +struct rxe_qp *qp; +int skb_out; + +/* takes a ref on ib device if success */ + rxe =get_rxe_from_skb (skb); +if (!rxe) + goto out; + +/* recover source qp index from sk->sk_user_data +* free the reference taken in rxe_send +*/ + index = (int )(uintptr_t )skb->sk->sk_user_data; +sock_put (skb->sk); +/* lookup qp from index, takes a ref on success */ + qp =rxe_pool_get_index (&rxe->qp_pool, index); +if (!qp) + goto out_put_ibdev; + + skb_out =atomic_dec_return (&qp->skb_out);if (unlikely (qp->need_req_skb && skb_out <RXE_INFLIGHT_SKBS_PER_QP_LOW ))rxe_sched_task (&qp->req.task);rxe_put (qp); +out_put_ibdev: +ib_device_put (&rxe->ib_dev); +out: +return ; }static int rxe_send (struct sk_buff *skb,struct rxe_pkt_info *pkt) { +struct rxe_qp *qp = pkt->qp; +struct sock *sk;int err; - skb->destructor = rxe_skb_tx_dtor; - skb->sk = pkt->qp->sk->sk; +/* qp can be destroyed while this packet is waiting on +* the tx queue. So need to protect sk. +*/ + sk = qp->sk->sk; +sock_hold (sk); + skb->sk = sk; -rxe_get (pkt->qp);atomic_inc (&pkt->qp->skb_out); + sk->sk_user_data = (void *)(long )qp->elem.index; + skb->destructor = rxe_skb_tx_dtor; +if (skb->protocol ==htons (ETH_P_IP )) { - err =ip_local_out (dev_net (skb_dst (skb)->dev), skb->sk, skb); + err =ip_local_out (dev_net (skb_dst (skb)->dev), sk, skb); }else if (skb->protocol ==htons (ETH_P_IPV6 )) { - err =ip6_local_out (dev_net (skb_dst (skb)->dev), skb->sk, skb); + err =ip6_local_out (dev_net (skb_dst (skb)->dev), sk, skb); }else { -rxe_dbg_qp (pkt->qp,"Unknown layer 3 protocol: %d\n" , - skb->protocol); -atomic_dec (&pkt->qp->skb_out); -rxe_put (pkt->qp); -kfree_skb (skb); -return -EINVAL ; +rxe_dbg_qp (qp,"Unknown layer 3 protocol: %d" , + skb->protocol); + err = -EINVAL ; + goto err_not_sent; } +/* IP consumed the packet, the destructor will handle cleanup */ if (unlikely (net_xmit_eval (err))) { -rxe_dbg_qp (pkt->qp,"error sending packet: %d\n" , err); -return -EAGAIN ; +rxe_dbg_qp (qp,"Error sending packet: %d" , err); + err = -EAGAIN ; + goto err_out; }return 0 ; + +err_not_sent: + skb->destructor =NULL ; +atomic_dec (&pkt->qp->skb_out); +kfree_skb (skb); +sock_put (sk); +err_out: +return err; }/* fix up a send packet to match the packets drivers/infiniband/sw/rxe/rxe_qp.c +0
-2@@ -194,7 +194,6 @@ static int rxe_qp_init_req(struct rxe_dev *rxe, struct rxe_qp *qp, err =sock_create_kern (&init_net,AF_INET ,SOCK_DGRAM ,0 , &qp->sk);if (err <0 )return err; - qp->sk->sk->sk_user_data = qp;/* pick a source UDP port number for this QP based on * the source QPN. this spreads traffic for different QPs -- 2.39.2
Re: [PATCH for-next 4/9] RDMA/rxe: Fix delayed send packet handling
Quoted 18 lines · Leon Romanovsky
Quoted 17 lines · Zhu Yanjun
Quoted 6 lines · Bob Pearson
In cable pull testing some NICs can hold a send packet long enough to allow ulp protocol stacks to destroy the qp and the cleanup routines to timeout waiting for all qp references to be released. When the NIC driver finally frees the SKB the qp pointer is no longer valid and causes a seg fault in
rxe_skb_tx_dtor().If a packet is held in some NICs, the later packets of this packet will also be held by this NIC. So this will cause QP not to work well. This is a serious problem. And the problem should be fixed in this kind of NICs. We should not fix it in RXE.
And a QP exists for at least several seconds. A packet can be held in NIC for such a long time? This problem does exist in the real world, or you imagine this scenario?
Quoted 5 lines
Quoted 1 line
I can not imagine this kind of scenario. Please Jason or Leon also check this.
I stopped to follow RXE changes for a long time. They don't make any sense to me. Even this patch, which moves existing IB reference counter from one place to another and does it for every SKB is beyond my understanding.
At the end of the day, we used to take a reference on the QP for each send packet and drop it in the destructor function. (why? we are keeping a count of pending send packets to keep from overloading the send queue.) But with link failures which can take many seconds to recover this gets in the way of lustre cleaning up a bad connection when it times out first. So instead we take a reference on the sock struct since if you destroy the QP you also destroy the QP socket struct which drops its reference on the sock struct which gets it freed before the packet is finally sent and the destructor gets called. That reference is dropped in the destructor. We still need to get to the qp pointer in the destructor so we pass the qp# in the sock->user_data instead of the qp pointer. Now if the qp has been destroyed the qp lookup from the qp# fails and you just don't touch the counter. It really all just does work fine.
Bob
Quoted 172 lines
Thanks
Quoted 169 lines
Thanks.
Zhu YanjunQuoted 166 lines
This patch passes the qp index instead of the qp to the skb destructor callback function. The call back is required to lookup the qp from the index and if it has been destroyed the lookup will return NULL and the qp will not be referenced avoiding the seg fault.
Fixes 8700e3e7c485 ("Soft RoCE driver")Signed-off-by Bob Pearson <[email protected]>--- drivers/infiniband/sw/rxe/rxe_net.c | 87 ++++++++++++++++++++++------- drivers/infiniband/sw/rxe/rxe_qp.c | 1 - 2 files changed, 67 insertions(+), 21 deletions(-)drivers/infiniband/sw/rxe/rxe_net.c +67
-20@@ -131,19 +131,28 @@ static struct dst_entry *rxe_find_route(struct net_device *ndev,return dst; } +static struct rxe_dev *get_rxe_from_skb (struct sk_buff *skb) +{ +struct rxe_dev *rxe; +struct net_device *ndev = skb->dev; + + rxe =rxe_get_dev_from_net (ndev); +if (!rxe &&is_vlan_dev (ndev)) + rxe =rxe_get_dev_from_net (vlan_dev_real_dev (ndev)); + +return rxe; +} +static int rxe_udp_encap_recv (struct sock *sk,struct sk_buff *skb) {struct udphdr *udph;struct rxe_dev *rxe; -struct net_device *ndev = skb->dev;struct rxe_pkt_info *pkt =SKB_TO_PKT (skb);/* takes a reference on rxe->ib_dev * drop when skb is freed */ - rxe =rxe_get_dev_from_net (ndev); -if (!rxe &&is_vlan_dev (ndev)) - rxe =rxe_get_dev_from_net (vlan_dev_real_dev (ndev)); + rxe =get_rxe_from_skb (skb);if (!rxe) goto drop;@@ -345,46 +354,84 @@ int rxe_prepare(struct rxe_av *av, struct rxe_pkt_info *pkt,static void rxe_skb_tx_dtor (struct sk_buff *skb) { -struct sock *sk = skb->sk; -struct rxe_qp *qp = sk->sk_user_data; -int skb_out =atomic_dec_return (&qp->skb_out); +struct rxe_dev *rxe; +unsigned int index; +struct rxe_qp *qp; +int skb_out; + +/* takes a ref on ib device if success */ + rxe =get_rxe_from_skb (skb); +if (!rxe) + goto out; + +/* recover source qp index from sk->sk_user_data +* free the reference taken in rxe_send +*/ + index = (int )(uintptr_t )skb->sk->sk_user_data; +sock_put (skb->sk); +/* lookup qp from index, takes a ref on success */ + qp =rxe_pool_get_index (&rxe->qp_pool, index); +if (!qp) + goto out_put_ibdev; + + skb_out =atomic_dec_return (&qp->skb_out);if (unlikely (qp->need_req_skb && skb_out <RXE_INFLIGHT_SKBS_PER_QP_LOW ))rxe_sched_task (&qp->req.task);rxe_put (qp); +out_put_ibdev: +ib_device_put (&rxe->ib_dev); +out: +return ; }static int rxe_send (struct sk_buff *skb,struct rxe_pkt_info *pkt) { +struct rxe_qp *qp = pkt->qp; +struct sock *sk;int err; - skb->destructor = rxe_skb_tx_dtor; - skb->sk = pkt->qp->sk->sk; +/* qp can be destroyed while this packet is waiting on +* the tx queue. So need to protect sk. +*/ + sk = qp->sk->sk; +sock_hold (sk); + skb->sk = sk; -rxe_get (pkt->qp);atomic_inc (&pkt->qp->skb_out); + sk->sk_user_data = (void *)(long )qp->elem.index; + skb->destructor = rxe_skb_tx_dtor; +if (skb->protocol ==htons (ETH_P_IP )) { - err =ip_local_out (dev_net (skb_dst (skb)->dev), skb->sk, skb); + err =ip_local_out (dev_net (skb_dst (skb)->dev), sk, skb); }else if (skb->protocol ==htons (ETH_P_IPV6 )) { - err =ip6_local_out (dev_net (skb_dst (skb)->dev), skb->sk, skb); + err =ip6_local_out (dev_net (skb_dst (skb)->dev), sk, skb); }else { -rxe_dbg_qp (pkt->qp,"Unknown layer 3 protocol: %d\n" , - skb->protocol); -atomic_dec (&pkt->qp->skb_out); -rxe_put (pkt->qp); -kfree_skb (skb); -return -EINVAL ; +rxe_dbg_qp (qp,"Unknown layer 3 protocol: %d" , + skb->protocol); + err = -EINVAL ; + goto err_not_sent; } +/* IP consumed the packet, the destructor will handle cleanup */ if (unlikely (net_xmit_eval (err))) { -rxe_dbg_qp (pkt->qp,"error sending packet: %d\n" , err); -return -EAGAIN ; +rxe_dbg_qp (qp,"Error sending packet: %d" , err); + err = -EAGAIN ; + goto err_out; }return 0 ; + +err_not_sent: + skb->destructor =NULL ; +atomic_dec (&pkt->qp->skb_out); +kfree_skb (skb); +sock_put (sk); +err_out: +return err; }/* fix up a send packet to match the packets drivers/infiniband/sw/rxe/rxe_qp.c +0
-2@@ -194,7 +194,6 @@ static int rxe_qp_init_req(struct rxe_dev *rxe, struct rxe_qp *qp, err =sock_create_kern (&init_net,AF_INET ,SOCK_DGRAM ,0 , &qp->sk);if (err <0 )return err; - qp->sk->sk->sk_user_data = qp;/* pick a source UDP port number for this QP based on * the source QPN. this spreads traffic for different QPs -- 2.39.2
Jason Gunthorpe
maintainerRe: [PATCH for-next 4/9] RDMA/rxe: Fix delayed send packet handling
In reply to Bob PearsonQuoted 10 lines · Bob Pearson
In cable pull testing some NICs can hold a send packet long enough to allow ulp protocol stacks to destroy the qp and the cleanup routines to timeout waiting for all qp references to be released. When the NIC driver finally frees the SKB the qp pointer is no longer valid and causes a seg fault in
rxe_skb_tx_dtor().This patch passes the qp index instead of the qp to the skb destructor callback function. The call back is required to lookup the qp from the index and if it has been destroyed the lookup will return NULL and the qp will not be referenced avoiding the seg fault.
And what if it is a different QP returned?
Jason
Re: [PATCH for-next 4/9] RDMA/rxe: Fix delayed send packet handling
Quoted 7 lines · Jason Gunthorpe
Quoted 6 lines · Bob Pearson
In cable pull testing some NICs can hold a send packet long enough to allow ulp protocol stacks to destroy the qp and the cleanup routines to timeout waiting for all qp references to be released. When the NIC driver finally frees the SKB the qp pointer is no longer valid and causes a seg fault in
rxe_skb_tx_dtor().
Quoted 8 lines
Quoted 4 lines
This patch passes the qp index instead of the qp to the skb destructor callback function. The call back is required to lookup the qp from the index and if it has been destroyed the lookup will return NULL and the qp will not be referenced avoiding the seg fault.
And what if it is a different QP returned?
Jason
Since we are using xarray cyclic alloc you would have to create 16M QPs before the index was reused. This is as good as it gets I think.
Bob
Jason Gunthorpe
maintainerRe: [PATCH for-next 4/9] RDMA/rxe: Fix delayed send packet handling
Quoted 13 lines · Bob Pearson
Quoted 12 lines · Jason Gunthorpe
Quoted 10 lines · Bob Pearson
In cable pull testing some NICs can hold a send packet long enough to allow ulp protocol stacks to destroy the qp and the cleanup routines to timeout waiting for all qp references to be released. When the NIC driver finally frees the SKB the qp pointer is no longer valid and causes a seg fault in
rxe_skb_tx_dtor().This patch passes the qp index instead of the qp to the skb destructor callback function. The call back is required to lookup the qp from the index and if it has been destroyed the lookup will return NULL and the qp will not be referenced avoiding the seg fault.
Quoted 6 lines
Quoted 3 lines
And what if it is a different QP returned?
Jason
Since we are using xarray cyclic alloc you would have to create 16M QPs before the index was reused. This is as good as it gets I think.
Sounds terrible, why can't you store the QP pointer instead and hold a refcount on it?
Jason
Re: [PATCH for-next 4/9] RDMA/rxe: Fix delayed send packet handling
Quoted 16 lines · Jason Gunthorpe
Quoted 15 lines · Bob Pearson
Quoted 14 lines · Jason Gunthorpe
Quoted 10 lines · Bob Pearson
In cable pull testing some NICs can hold a send packet long enough to allow ulp protocol stacks to destroy the qp and the cleanup routines to timeout waiting for all qp references to be released. When the NIC driver finally frees the SKB the qp pointer is no longer valid and causes a seg fault in
rxe_skb_tx_dtor().This patch passes the qp index instead of the qp to the skb destructor callback function. The call back is required to lookup the qp from the index and if it has been destroyed the lookup will return NULL and the qp will not be referenced avoiding the seg fault.
And what if it is a different QP returned?
Quoted 7 lines
Quoted 4 lines
Quoted 1 line
Jason
Since we are using xarray cyclic alloc you would have to create 16M QPs before the index was reused. This is as good as it gets I think.
Sounds terrible, why can't you store the QP pointer instead and hold a refcount on it?
The goal here was to make packet send semantics to be 'fire and forget' i.e. once we send the packet not have any dependencies hanging around. But we still wanted to count the packets pending to avoid overrunning the send queue.
This allows lustre to do its normal error recovery and destroy the qp and try to create a new one when it times out.
Bob
Quoted 2 lines
Jason
Jason Gunthorpe
maintainerRe: [PATCH for-next 4/9] RDMA/rxe: Fix delayed send packet handling
Quoted 22 lines · Bob Pearson
Quoted 21 lines · Jason Gunthorpe
Quoted 19 lines · Bob Pearson
Quoted 7 lines · Jason Gunthorpe
Quoted 6 lines · Bob Pearson
In cable pull testing some NICs can hold a send packet long enough to allow ulp protocol stacks to destroy the qp and the cleanup routines to timeout waiting for all qp references to be released. When the NIC driver finally frees the SKB the qp pointer is no longer valid and causes a seg fault in
rxe_skb_tx_dtor().Quoted 8 lines
Quoted 4 lines
This patch passes the qp index instead of the qp to the skb destructor callback function. The call back is required to lookup the qp from the index and if it has been destroyed the lookup will return NULL and the qp will not be referenced avoiding the seg fault.
And what if it is a different QP returned?
Jason
Since we are using xarray cyclic alloc you would have to create 16M QPs before the index was reused. This is as good as it gets I think.
Quoted 6 lines
Quoted 2 lines
Sounds terrible, why can't you store the QP pointer instead and hold a refcount on it?
The goal here was to make packet send semantics to be 'fire and forget' i.e. once we send the packet not have any dependencies hanging around. But we still wanted to count the packets pending to avoid overrunning the send queue.
Well, you can't have it both ways really.
Maybe you need another bit of memory to track the packet counters that can be refcounted independently of the qp.
And wait for those refcounts to zero out before allowing the driver to unprobe.
Jason
PATCH 5/9 RDMA/rxe: Optimize rxe_init_packet in rxe_net.c
In reply to Bob PearsonThis patch replaces code that looks up the ndev from the ib_device and port number of a rxe device with the saved value of the ndev used when the rxe device was created. Since the rxe driver does not support multi port operation or path migration, the ndev is constant for the life of the rxe device. The ndev lookup code in rxe_init_packet has a significant performance impact under heavy load and can consume a noticeable fraction of the overall cpu time.
--- drivers/infiniband/sw/rxe/rxe_net.c | 25 ++++--------------------- 1 file changed, 4 insertions(+), 21 deletions(-)
drivers/infiniband/sw/rxe/rxe_net.c +4 -21
@@ -508,14 +508,9 @@ struct sk_buff *rxe_init_packet(struct rxe_dev *rxe, struct rxe_av *av, {unsigned int hdr_len;struct sk_buff *skb =NULL ; -struct net_device *ndev; -const struct ib_gid_attr *attr; +struct net_device *ndev = rxe->ndev;const int port_num =1 ; - attr =rdma_get_gid_attr (&rxe->ib_dev, port_num, av->grh.sgid_index); -if (IS_ERR (attr)) -return NULL ; -if (av->network_type ==RXE_NETWORK_TYPE_IPV4 ) hdr_len =ETH_HLEN +sizeof (struct udphdr ) +sizeof (struct iphdr );
@@ -523,25 +518,14 @@ struct sk_buff *rxe_init_packet(struct rxe_dev *rxe, struct rxe_av *av, hdr_len =ETH_HLEN +sizeof (struct udphdr ) +sizeof (struct ipv6hdr ); -rcu_read_lock (); - ndev =rdma_read_gid_attr_ndev_rcu (attr); -if (IS_ERR (ndev)) { -rcu_read_unlock (); - goto out; - } - skb =alloc_skb (paylen + hdr_len +LL_RESERVED_SPACE (ndev), -GFP_ATOMIC ); - -if (unlikely (!skb)) { -rcu_read_unlock (); + skb =alloc_skb (paylen + hdr_len +LL_RESERVED_SPACE (ndev),GFP_ATOMIC ); +if (unlikely (!skb)) goto out; - }skb_reserve (skb, hdr_len +LL_RESERVED_SPACE (ndev));/* FIXME: hold reference to this netdev until life of this skb. */ - skb->dev = ndev; -rcu_read_unlock (); + skb->dev = ndev;if (av->network_type ==RXE_NETWORK_TYPE_IPV4 ) skb->protocol =htons (ETH_P_IP );
@@ -554,7 +538,6 @@ struct sk_buff *rxe_init_packet(struct rxe_dev *rxe, struct rxe_av *av, pkt->mask |=RXE_GRH_MASK ; out: -rdma_put_gid_attr (attr);return skb; }
Signature · 1 line
2.39.2
PATCH 6/9 RDMA/rxe: Delete unused field elem->list
In reply to Bob Pearsonstruct rxe_pool_elem field list is not used. This patch removes it.
--- drivers/infiniband/sw/rxe/rxe_pool.h | 1 - 1 file changed, 1 deletion(-)
drivers/infiniband/sw/rxe/rxe_pool.h +0 -1
@@ -23,7 +23,6 @@ struct rxe_pool_elem {struct rxe_pool *pool;void *obj;struct kref ref_cnt; -struct list_head list;struct completion complete;struct rcu_head rcu;u32 index;
Signature · 1 line
2.39.2
PATCH 7/9 RDMA/rxe: Add elem->valid field
In reply to Bob PearsonCurrently the rxe driver stores NULL in the pool xarray to indicate that the pool element should not be looked up from its index. This prevents looping over pool elements during driver cleanup. This patch adds a new valid field to struct rxe_pool_elem as an alternative way to accomplish the same thing.
--- drivers/infiniband/sw/rxe/rxe_pool.c | 32 ++++++++++++---------------- drivers/infiniband/sw/rxe/rxe_pool.h | 1 + 2 files changed, 15 insertions(+), 18 deletions(-)
drivers/infiniband/sw/rxe/rxe_pool.c +14 -18
@@ -141,7 +141,7 @@ int __rxe_add_to_pool(struct rxe_pool *pool, struct rxe_pool_elem *elem,if (sleepable)might_sleep ();xa_lock_irqsave (xa, flags); - err =__xa_alloc_cyclic (xa, &elem->index,NULL , pool->limit, + err =__xa_alloc_cyclic (xa, &elem->index, elem, pool->limit, &pool->next, gfp_flags);xa_unlock_irqrestore (xa, flags);if (err <0 )
@@ -158,11 +158,14 @@ void *rxe_pool_get_index(struct rxe_pool *pool, u32 index) {struct rxe_pool_elem *elem;struct xarray *xa = &pool->xa; +int valid;void *obj;rcu_read_lock (); elem =xa_load (xa, index); -if (elem &&kref_get_unless_zero (&elem->ref_cnt)) +/* get current value of elem->valid */ + valid = elem ?smp_load_acquire (&elem->valid) :0 ; +if (valid &&kref_get_unless_zero (&elem->ref_cnt)) obj = elem->obj;else obj =NULL ;
@@ -191,13 +194,8 @@ void __rxe_cleanup(struct rxe_pool_elem *elem, bool sleepable)if (sleepable)might_sleep (); -/* erase xarray entry to prevent looking up -* the pool elem from its index -*/ -xa_lock_irqsave (xa, flags); - xa_ret =__xa_erase (xa, elem->index); -xa_unlock_irqrestore (xa, flags); -WARN_ON (xa_err (xa_ret)); +/* prevent looking up element from index */ +smp_store_release (&elem->valid,0 );/* if this is the last call to rxe_put complete the * object. It is safe to touch obj->elem after this since
@@ -231,6 +229,11 @@ void __rxe_cleanup(struct rxe_pool_elem *elem, bool sleepable) } } +xa_lock_irqsave (xa, flags); + xa_ret =__xa_erase (xa, elem->index); +xa_unlock_irqrestore (xa, flags); +WARN_ON (xa_err (xa_ret)); +if (pool->cleanup) pool->cleanup (elem);
@@ -249,13 +252,6 @@ int __rxe_put(struct rxe_pool_elem *elem)void __rxe_finalize (struct rxe_pool_elem *elem,bool sleepable) { -gfp_t gfp_flags = sleepable ?GFP_KERNEL :GFP_ATOMIC ; -struct xarray *xa = &elem->pool->xa; -unsigned long flags; -void *xa_ret; - -xa_lock_irqsave (xa, flags); - xa_ret =__xa_store (xa, elem->index, elem, gfp_flags); -xa_unlock_irqrestore (xa, flags); -WARN_ON (xa_err (xa_ret)); +/* allow looking up element from index */ +smp_store_release (&elem->valid,1 ); }
drivers/infiniband/sw/rxe/rxe_pool.h +1 -0
@@ -26,6 +26,7 @@ struct rxe_pool_elem {struct completion complete;struct rcu_head rcu;u32 index; +int valid; };struct rxe_pool {
Signature · 1 line
2.39.2
Jason Gunthorpe
maintainerRe: [PATCH for-next 7/9] RDMA/rxe: Add elem->valid field
Quoted 5 lines · Bob Pearson
Currently the rxe driver stores NULL in the pool xarray to indicate that the pool element should not be looked up from its index. This prevents looping over pool elements during driver cleanup. This patch adds a new valid field to struct
rxe_pool_elemas an alternative way to accomplish the same thing.
Wah? Why? Yuk.
Quoted 6 lines
drivers/infiniband/sw/rxe/rxe_pool.c +0
-2@@ -249,13 +252,6 @@ int __rxe_put(struct rxe_pool_elem *elem)void __rxe_finalize (struct rxe_pool_elem *elem,bool sleepable) { -gfp_t gfp_flags = sleepable ?GFP_KERNEL :GFP_ATOMIC ; -struct xarray *xa = &elem->pool->xa;
drivers/infiniband/sw/rxe/rxe_pool.c int __rxe_put(struct rxe_pool_elem *elem) · line 257
Quoted 10 lines
drivers/infiniband/sw/rxe/rxe_pool.c +2
-7-unsigned long flags; -void *xa_ret; - -xa_lock_irqsave (xa, flags); - xa_ret =__xa_store (xa, elem->index, elem, gfp_flags); -xa_unlock_irqrestore (xa, flags); -WARN_ON (xa_err (xa_ret)); +/* allow looking up element from index */ +smp_store_release (&elem->valid,1 ); }
That is horrible, why?
Jason
PATCH 8/9 RDMA/rxe: Report leaked objects
In reply to Bob PearsonThis patch gives a more detailed list of objects that are not freed in case of error before the module exits.
--- drivers/infiniband/sw/rxe/rxe_pool.c | 12 +++++++++++- 1 file changed, 11 insertions(+), 1 deletion(-)
drivers/infiniband/sw/rxe/rxe_pool.c +11 -1
@@ -113,7 +113,17 @@ void rxe_pool_init(struct rxe_dev *rxe, struct rxe_pool *pool,void rxe_pool_cleanup (struct rxe_pool *pool) { -WARN_ON (!xa_empty (&pool->xa)); +unsigned long index; +struct rxe_pool_elem *elem; + +xa_lock (&pool->xa); +xa_for_each (&pool->xa, index, elem) { +rxe_err_dev (pool->rxe,"%s#%d: Leaked" , pool->name, + elem->index); + } +xa_unlock (&pool->xa); + +xa_destroy (&pool->xa); } int __rxe_add_to_pool(struct rxe_pool *pool, struct rxe_pool_elem *elem,
Signature · 1 line
2.39.2
Jason Gunthorpe
maintainerRe: [PATCH for-next 8/9] RDMA/rxe: Report leaked objects
Quoted 19 lines · Bob Pearson
This patch gives a more detailed list of objects that are not freed in case of error before the module exits.
Signed-off-by Bob Pearson <[email protected]>--- drivers/infiniband/sw/rxe/rxe_pool.c | 12 +++++++++++- 1 file changed, 11 insertions(+), 1 deletion(-)drivers/infiniband/sw/rxe/rxe_pool.c +2
-1@@ -113,7 +113,17 @@ void rxe_pool_init(struct rxe_dev *rxe, struct rxe_pool *pool,void rxe_pool_cleanup (struct rxe_pool *pool) { -WARN_ON (!xa_empty (&pool->xa)); +unsigned long index; +struct rxe_pool_elem *elem;
drivers/infiniband/sw/rxe/rxe_pool.c void rxe_pool_init(struct rxe_dev *rxe, struct rxe_pool *pool, · line 127
Quoted 10 lines
drivers/infiniband/sw/rxe/rxe_pool.c +9
-0+ +xa_lock (&pool->xa); +xa_for_each (&pool->xa, index, elem) { +rxe_err_dev (pool->rxe,"%s#%d: Leaked" , pool->name, + elem->index); + } +xa_unlock (&pool->xa); + +xa_destroy (&pool->xa); }
Is this why? Just count the number of unfinalized objects and report if there is difference, don't mess up the xarray.
Jason
Re: [PATCH for-next 8/9] RDMA/rxe: Report leaked objects
Quoted 34 lines · Jason Gunthorpe
Quoted 19 lines · Bob Pearson
This patch gives a more detailed list of objects that are not freed in case of error before the module exits.
Signed-off-by Bob Pearson <[email protected]>--- drivers/infiniband/sw/rxe/rxe_pool.c | 12 +++++++++++- 1 file changed, 11 insertions(+), 1 deletion(-)drivers/infiniband/sw/rxe/rxe_pool.c +2
-1@@ -113,7 +113,17 @@ void rxe_pool_init(struct rxe_dev *rxe, struct rxe_pool *pool,void rxe_pool_cleanup (struct rxe_pool *pool) { -WARN_ON (!xa_empty (&pool->xa)); +unsigned long index; +struct rxe_pool_elem *elem;drivers/infiniband/sw/rxe/rxe_pool.c void rxe_pool_init(struct rxe_dev *rxe, struct rxe_pool *pool, · line 127
Quoted 10 lines
drivers/infiniband/sw/rxe/rxe_pool.c +9
-0+ +xa_lock (&pool->xa); +xa_for_each (&pool->xa, index, elem) { +rxe_err_dev (pool->rxe,"%s#%d: Leaked" , pool->name, + elem->index); + } +xa_unlock (&pool->xa); + +xa_destroy (&pool->xa); }Is this why? Just count the number of unfinalized objects and report if there is difference, don't mess up the xarray.
Quoted 1 line
Jason
This is why I made the last change but I really didn't like that there was no way to lookup the qp from its index since we were using a NULL xarray entry to indicate the state of the qp. Making it explicit, i.e. a variable in the struct seems much more straight forward. Not sure why you hated the last change.
Bob
Jason Gunthorpe
maintainerRe: [PATCH for-next 8/9] RDMA/rxe: Report leaked objects
Quoted 35 lines · Bob Pearson
Quoted 34 lines · Jason Gunthorpe
Quoted 19 lines · Bob Pearson
This patch gives a more detailed list of objects that are not freed in case of error before the module exits.
Signed-off-by Bob Pearson <[email protected]>--- drivers/infiniband/sw/rxe/rxe_pool.c | 12 +++++++++++- 1 file changed, 11 insertions(+), 1 deletion(-)drivers/infiniband/sw/rxe/rxe_pool.c +2
-1@@ -113,7 +113,17 @@ void rxe_pool_init(struct rxe_dev *rxe, struct rxe_pool *pool,void rxe_pool_cleanup (struct rxe_pool *pool) { -WARN_ON (!xa_empty (&pool->xa)); +unsigned long index; +struct rxe_pool_elem *elem;drivers/infiniband/sw/rxe/rxe_pool.c void rxe_pool_init(struct rxe_dev *rxe, struct rxe_pool *pool, · line 127
Quoted 10 lines
drivers/infiniband/sw/rxe/rxe_pool.c +9
-0+ +xa_lock (&pool->xa); +xa_for_each (&pool->xa, index, elem) { +rxe_err_dev (pool->rxe,"%s#%d: Leaked" , pool->name, + elem->index); + } +xa_unlock (&pool->xa); + +xa_destroy (&pool->xa); }Is this why? Just count the number of unfinalized objects and report if there is difference, don't mess up the xarray.
Quoted 6 lines
Quoted 1 line
Jason
This is why I made the last change but I really didn't like that there was no way to lookup the qp from its index since we were using a NULL xarray entry to indicate the state of the qp. Making it explicit, i.e. a variable in the struct seems much more straight forward. Not sure why you hated the last change.
Because if you don't call finalize abort has to be deterministic, and abort can't be that if it someone else can access the poitner and, eg, take a reference.
It breaks the entire model of how the memory ownership works during creation.
Jason
Re: [PATCH for-next 8/9] RDMA/rxe: Report leaked objects
Quoted 43 lines · Jason Gunthorpe
Quoted 35 lines · Bob Pearson
Quoted 34 lines · Jason Gunthorpe
Quoted 19 lines · Bob Pearson
This patch gives a more detailed list of objects that are not freed in case of error before the module exits.
Signed-off-by Bob Pearson <[email protected]>--- drivers/infiniband/sw/rxe/rxe_pool.c | 12 +++++++++++- 1 file changed, 11 insertions(+), 1 deletion(-)drivers/infiniband/sw/rxe/rxe_pool.c +2
-1@@ -113,7 +113,17 @@ void rxe_pool_init(struct rxe_dev *rxe, struct rxe_pool *pool,void rxe_pool_cleanup (struct rxe_pool *pool) { -WARN_ON (!xa_empty (&pool->xa)); +unsigned long index; +struct rxe_pool_elem *elem;drivers/infiniband/sw/rxe/rxe_pool.c void rxe_pool_init(struct rxe_dev *rxe, struct rxe_pool *pool, · line 127
Quoted 10 lines
drivers/infiniband/sw/rxe/rxe_pool.c +9
-0+ +xa_lock (&pool->xa); +xa_for_each (&pool->xa, index, elem) { +rxe_err_dev (pool->rxe,"%s#%d: Leaked" , pool->name, + elem->index); + } +xa_unlock (&pool->xa); + +xa_destroy (&pool->xa); }Is this why? Just count the number of unfinalized objects and report if there is difference, don't mess up the xarray.
Quoted 6 lines
Quoted 1 line
Jason
This is why I made the last change but I really didn't like that there was no way to lookup the qp from its index since we were using a NULL xarray entry to indicate the state of the qp. Making it explicit, i.e. a variable in the struct seems much more straight forward. Not sure why you hated the last change.
Quoted 3 lines
Because if you don't call finalize abort has to be deterministic, and abort can't be that if it someone else can access the poitner and, eg, take a reference.
rxe_pool_get_index() is the only 'correct' way to look up the pointer and it checks the valid state (now). Scanning the xarray or just looking up the qp is something outside the scope of the normal flows. Like listing orphan objects on module exit.
Memory ownership didn't change. It is still the same. The only change is how we mark whether the object is valid for lookup.
Bob
Quoted 4 lines
It breaks the entire model of how the memory ownership works during creation.
Jason
Jason Gunthorpe
maintainerRe: [PATCH for-next 8/9] RDMA/rxe: Report leaked objects
Quoted 44 lines · Bob Pearson
Quoted 43 lines · Jason Gunthorpe
Quoted 41 lines · Bob Pearson
Quoted 34 lines · Jason Gunthorpe
Quoted 19 lines · Bob Pearson
This patch gives a more detailed list of objects that are not freed in case of error before the module exits.
Signed-off-by Bob Pearson <[email protected]>--- drivers/infiniband/sw/rxe/rxe_pool.c | 12 +++++++++++- 1 file changed, 11 insertions(+), 1 deletion(-)drivers/infiniband/sw/rxe/rxe_pool.c +2
-1@@ -113,7 +113,17 @@ void rxe_pool_init(struct rxe_dev *rxe, struct rxe_pool *pool,void rxe_pool_cleanup (struct rxe_pool *pool) { -WARN_ON (!xa_empty (&pool->xa)); +unsigned long index; +struct rxe_pool_elem *elem;drivers/infiniband/sw/rxe/rxe_pool.c void rxe_pool_init(struct rxe_dev *rxe, struct rxe_pool *pool, · line 127
Quoted 10 lines
drivers/infiniband/sw/rxe/rxe_pool.c +9
-0+ +xa_lock (&pool->xa); +xa_for_each (&pool->xa, index, elem) { +rxe_err_dev (pool->rxe,"%s#%d: Leaked" , pool->name, + elem->index); + } +xa_unlock (&pool->xa); + +xa_destroy (&pool->xa); }Is this why? Just count the number of unfinalized objects and report if there is difference, don't mess up the xarray.
Quoted 1 line
Jason
This is why I made the last change but I really didn't like that there was no way to lookup the qp from its index since we were using a NULL xarray entry to indicate the state of the qp. Making it explicit, i.e. a variable in the struct seems much more straight forward. Not sure why you hated the last change.
Quoted 8 lines
Quoted 3 lines
Because if you don't call finalize abort has to be deterministic, and abort can't be that if it someone else can access the poitner and, eg, take a reference.
rxe_pool_get_index()is the only 'correct' way to look up the pointer and it checks the valid state (now). Scanning the xarray or just looking up the qp is something outside the scope of the normal flows. Like listing orphan objects on module exit.
Maybe at this instance, but people keep changing this and it is fundamentally wrong to store a pointer to an incompletely initialized memory for other threads to see.
Especially for some minor debugging feature.
Jason
Re: [PATCH for-next 8/9] RDMA/rxe: Report leaked objects
Quoted 54 lines · Jason Gunthorpe
Quoted 44 lines · Bob Pearson
Quoted 43 lines · Jason Gunthorpe
Quoted 41 lines · Bob Pearson
Quoted 34 lines · Jason Gunthorpe
Quoted 19 lines · Bob Pearson
This patch gives a more detailed list of objects that are not freed in case of error before the module exits.
Signed-off-by Bob Pearson <[email protected]>--- drivers/infiniband/sw/rxe/rxe_pool.c | 12 +++++++++++- 1 file changed, 11 insertions(+), 1 deletion(-)drivers/infiniband/sw/rxe/rxe_pool.c +2
-1@@ -113,7 +113,17 @@ void rxe_pool_init(struct rxe_dev *rxe, struct rxe_pool *pool,void rxe_pool_cleanup (struct rxe_pool *pool) { -WARN_ON (!xa_empty (&pool->xa)); +unsigned long index; +struct rxe_pool_elem *elem;drivers/infiniband/sw/rxe/rxe_pool.c void rxe_pool_init(struct rxe_dev *rxe, struct rxe_pool *pool, · line 127
Quoted 10 lines
drivers/infiniband/sw/rxe/rxe_pool.c +9
-0+ +xa_lock (&pool->xa); +xa_for_each (&pool->xa, index, elem) { +rxe_err_dev (pool->rxe,"%s#%d: Leaked" , pool->name, + elem->index); + } +xa_unlock (&pool->xa); + +xa_destroy (&pool->xa); }Is this why? Just count the number of unfinalized objects and report if there is difference, don't mess up the xarray.
Quoted 1 line
Jason
This is why I made the last change but I really didn't like that there was no way to lookup the qp from its index since we were using a NULL xarray entry to indicate the state of the qp. Making it explicit, i.e. a variable in the struct seems much more straight forward. Not sure why you hated the last change.
Quoted 8 lines
Quoted 3 lines
Because if you don't call finalize abort has to be deterministic, and abort can't be that if it someone else can access the poitner and, eg, take a reference.
rxe_pool_get_index()is the only 'correct' way to look up the pointer and it checks the valid state (now). Scanning the xarray or just looking up the qp is something outside the scope of the normal flows. Like listing orphan objects on module exit.
Quoted 5 lines
Maybe at this instance, but people keep changing this and it is fundamentally wrong to store a pointer to an incompletely initialized memory for other threads to see.
Especially for some minor debugging feature.
Maybe warning comments would help. There are lots of ways to break the code, sigh. The typical one is someone makes a change that breaks reference counting.
Quoted 2 lines
Jason
Jason Gunthorpe
maintainerRe: [PATCH for-next 8/9] RDMA/rxe: Report leaked objects
Quoted 59 lines · Bob Pearson
Quoted 58 lines · Jason Gunthorpe
Quoted 44 lines · Bob Pearson
Quoted 43 lines · Jason Gunthorpe
Quoted 41 lines · Bob Pearson
Quoted 34 lines · Jason Gunthorpe
Quoted 19 lines · Bob Pearson
This patch gives a more detailed list of objects that are not freed in case of error before the module exits.
Signed-off-by Bob Pearson <[email protected]>--- drivers/infiniband/sw/rxe/rxe_pool.c | 12 +++++++++++- 1 file changed, 11 insertions(+), 1 deletion(-)drivers/infiniband/sw/rxe/rxe_pool.c +2
-1@@ -113,7 +113,17 @@ void rxe_pool_init(struct rxe_dev *rxe, struct rxe_pool *pool,void rxe_pool_cleanup (struct rxe_pool *pool) { -WARN_ON (!xa_empty (&pool->xa)); +unsigned long index; +struct rxe_pool_elem *elem;drivers/infiniband/sw/rxe/rxe_pool.c void rxe_pool_init(struct rxe_dev *rxe, struct rxe_pool *pool, · line 127
Quoted 10 lines
drivers/infiniband/sw/rxe/rxe_pool.c +9
-0+ +xa_lock (&pool->xa); +xa_for_each (&pool->xa, index, elem) { +rxe_err_dev (pool->rxe,"%s#%d: Leaked" , pool->name, + elem->index); + } +xa_unlock (&pool->xa); + +xa_destroy (&pool->xa); }Is this why? Just count the number of unfinalized objects and report if there is difference, don't mess up the xarray.
Quoted 1 line
Jason
This is why I made the last change but I really didn't like that there was no way to lookup the qp from its index since we were using a NULL xarray entry to indicate the state of the qp. Making it explicit, i.e. a variable in the struct seems much more straight forward. Not sure why you hated the last change.
Quoted 8 lines
Quoted 3 lines
Because if you don't call finalize abort has to be deterministic, and abort can't be that if it someone else can access the poitner and, eg, take a reference.
rxe_pool_get_index()is the only 'correct' way to look up the pointer and it checks the valid state (now). Scanning the xarray or just looking up the qp is something outside the scope of the normal flows. Like listing orphan objects on module exit.Maybe at this instance, but people keep changing this and it is fundamentally wrong to store a pointer to an incompletely initialized memory for other threads to see.
Quoted 5 lines
Quoted 1 line
Especially for some minor debugging feature.
Maybe warning comments would help. There are lots of ways to break the code, sigh. The typical one is someone makes a change that breaks reference counting.
Don't do it wrong in the first place is the most important thing.
Don't put incompletely intialized objects in global memory. It is a very simple rule.
Jason
PATCH 9/9 RDMA/rxe: Protect pending send packets
In reply to Bob PearsonNetwork interruptions may cause long delays in the processing of send packets during which time the rxe driver may be unloaded. This will cause seg faults when the packet is ultimately freed as it calls the destructor function in the rxe driver. This has been observed in cable pull fail over fail back testing.
This patch takes a reference on the driver to protect the packet from this happening.
--- drivers/infiniband/sw/rxe/rxe.c | 26 ++++++++++++++++++++++++++ drivers/infiniband/sw/rxe/rxe.h | 3 +++ drivers/infiniband/sw/rxe/rxe_net.c | 7 +++++++ 3 files changed, 36 insertions(+)
drivers/infiniband/sw/rxe/rxe.c +26 -0
@@ -208,10 +208,33 @@ static struct rdma_link_ops rxe_link_ops = { .newlink = rxe_newlink, }; +static struct rxe_module { +struct kref ref_cnt; +struct completion complete; +} rxe_module; + +static void rxe_module_release (struct kref *kref) +{ +complete (&rxe_module.complete); +} + +int rxe_module_get (void ) +{ +return kref_get_unless_zero (&rxe_module.ref_cnt); +} + +int rxe_module_put (void ) +{ +return kref_put (&rxe_module.ref_cnt, rxe_module_release); +} +static int __init rxe_module_init (void ) {int err; +kref_init (&rxe_module.ref_cnt); +init_completion (&rxe_module.complete); + err =rxe_alloc_wq ();if (err)return err;
@@ -229,6 +252,9 @@ static int __init rxe_module_init(void) static void __exit rxe_module_exit(void) { +rxe_module_put (); +wait_for_completion (&rxe_module.complete); +rdma_link_unregister (&rxe_link_ops);ib_unregister_driver (RDMA_DRIVER_RXE );rxe_net_exit ();
drivers/infiniband/sw/rxe/rxe.h +3 -0
@@ -158,4 +158,7 @@ void rxe_port_up(struct rxe_dev *rxe);void rxe_port_down (struct rxe_dev *rxe);void rxe_set_port_state (struct rxe_dev *rxe); +int rxe_module_get (void ); +int rxe_module_put (void ); +#endif /* RXE_H */
drivers/infiniband/sw/rxe/rxe_net.c +7 -0
@@ -384,6 +384,7 @@ static void rxe_skb_tx_dtor(struct sk_buff *skb) out_put_ibdev:ib_device_put (&rxe->ib_dev); out: +rxe_module_put ();return ; }
@@ -400,6 +401,12 @@ static int rxe_send(struct sk_buff *skb, struct rxe_pkt_info *pkt)sock_hold (sk); skb->sk = sk; +/* the module may be potentially removed while this packet +* is waiting on the tx queue causing seg faults. So need +* to protect the module. +*/ +rxe_module_get (); +atomic_inc (&pkt->qp->skb_out); sk->sk_user_data = (void *)(long )qp->elem.index;
Signature · 1 line
2.39.2
Jason Gunthorpe
maintainerRe: [PATCH for-next 9/9] RDMA/rxe: Protect pending send packets
Quoted 5 lines · Bob Pearson
Network interruptions may cause long delays in the processing of send packets during which time the rxe driver may be unloaded. This will cause seg faults when the packet is ultimately freed as it calls the destructor function in the rxe driver. This has been observed in cable pull fail over fail back testing.
No, module reference counts are only for code that is touching function pointers.
If your driver is becoming removed and that messes it up then you need to prevent the driver from unloading by adding something to the remove function (dellink, I guess in this case)
Jason
Re: [PATCH for-next 9/9] RDMA/rxe: Protect pending send packets
Quoted 9 lines · Jason Gunthorpe
Quoted 5 lines · Bob Pearson
Network interruptions may cause long delays in the processing of send packets during which time the rxe driver may be unloaded. This will cause seg faults when the packet is ultimately freed as it calls the destructor function in the rxe driver. This has been observed in cable pull fail over fail back testing.
No, module reference counts are only for code that is touching function pointers.
this is exactly the case here. it is the skb destructor function that is carried by the skb.
Quoted 6 lines
If your driver is becoming removed and that messes it up then you need to prevent the driver from unloading by adding something to the remove function (dellink, I guess in this case)
Jason
Jason Gunthorpe
maintainerRe: [PATCH for-next 9/9] RDMA/rxe: Protect pending send packets
Quoted 8 lines · Bob Pearson
Quoted 7 lines · Jason Gunthorpe
Quoted 5 lines · Bob Pearson
Network interruptions may cause long delays in the processing of send packets during which time the rxe driver may be unloaded. This will cause seg faults when the packet is ultimately freed as it calls the destructor function in the rxe driver. This has been observed in cable pull fail over fail back testing.
Quoted 5 lines
Quoted 2 lines
No, module reference counts are only for code that is touching function pointers.
this is exactly the case here. it is the skb destructor function that is carried by the skb.
It can't possibly call it correctly without also having the rxe ib_device reference too though??
Jason
Re: [PATCH for-next 9/9] RDMA/rxe: Protect pending send packets
Quoted 9 lines · Jason Gunthorpe
Quoted 8 lines · Bob Pearson
Quoted 7 lines · Jason Gunthorpe
Quoted 5 lines · Bob Pearson
Network interruptions may cause long delays in the processing of send packets during which time the rxe driver may be unloaded. This will cause seg faults when the packet is ultimately freed as it calls the destructor function in the rxe driver. This has been observed in cable pull fail over fail back testing.
Quoted 8 lines
Quoted 5 lines
Quoted 2 lines
No, module reference counts are only for code that is touching function pointers.
this is exactly the case here. it is the skb destructor function that is carried by the skb.
It can't possibly call it correctly without also having the rxe
ib_devicereference too though??
Nope. This was causing seg faults in testing when there was a long network hang and the admin tried to reload the rxe driver. The skb code doesn't care about the ib device at all.
Bob
Quoted 2 lines
Jason
Jason Gunthorpe
maintainerRe: [PATCH for-next 9/9] RDMA/rxe: Protect pending send packets
Quoted 16 lines · Bob Pearson
Quoted 15 lines · Jason Gunthorpe
Quoted 13 lines · Bob Pearson
Quoted 9 lines · Jason Gunthorpe
Quoted 5 lines · Bob Pearson
Network interruptions may cause long delays in the processing of send packets during which time the rxe driver may be unloaded. This will cause seg faults when the packet is ultimately freed as it calls the destructor function in the rxe driver. This has been observed in cable pull fail over fail back testing.
No, module reference counts are only for code that is touching function pointers.
this is exactly the case here. it is the skb destructor function that is carried by the skb.
Quoted 6 lines
Quoted 2 lines
It can't possibly call it correctly without also having the rxe
ib_devicereference too though??Nope. This was causing seg faults in testing when there was a long network hang and the admin tried to reload the rxe driver. The skb code doesn't care about the ib device at all.
I don't get it, there aren't globals in rxe, so WTF is it doing if it isn't somehow tracing back to memory that is under the ib_device lifetime?
Jason
Re: [PATCH for-next 9/9] RDMA/rxe: Protect pending send packets
Quoted 24 lines · Jason Gunthorpe
Quoted 16 lines · Bob Pearson
Quoted 15 lines · Jason Gunthorpe
Quoted 13 lines · Bob Pearson
Quoted 9 lines · Jason Gunthorpe
Quoted 5 lines · Bob Pearson
Network interruptions may cause long delays in the processing of send packets during which time the rxe driver may be unloaded. This will cause seg faults when the packet is ultimately freed as it calls the destructor function in the rxe driver. This has been observed in cable pull fail over fail back testing.
No, module reference counts are only for code that is touching function pointers.
this is exactly the case here. it is the skb destructor function that is carried by the skb.
Quoted 6 lines
Quoted 2 lines
It can't possibly call it correctly without also having the rxe
ib_devicereference too though??Nope. This was causing seg faults in testing when there was a long network hang and the admin tried to reload the rxe driver. The skb code doesn't care about the ib device at all.
Quoted 5 lines
I don't get it, there aren't globals in rxe, so WTF is it doing if it isn't somehow tracing back to memory that is under the
ib_devicelifetime?Jason
When the rxe driver builds a send packet it puts the address of its destructor subroutine in the skb before calling ip_local_out and sending it. The address of driver software is now hanging around. If you don't delay the module exit routine until all the skb's are freed you can cause seg faults. The only way to cause this to happen is to call rmmod on the driver too early but people have done this occasionally and report it as a bug.
Bob
Jason Gunthorpe
maintainerRe: [PATCH for-next 9/9] RDMA/rxe: Protect pending send packets
Quoted 29 lines · Bob Pearson
Quoted 28 lines · Jason Gunthorpe
Quoted 16 lines · Bob Pearson
Quoted 15 lines · Jason Gunthorpe
Quoted 13 lines · Bob Pearson
Quoted 9 lines · Jason Gunthorpe
Quoted 5 lines · Bob Pearson
Network interruptions may cause long delays in the processing of send packets during which time the rxe driver may be unloaded. This will cause seg faults when the packet is ultimately freed as it calls the destructor function in the rxe driver. This has been observed in cable pull fail over fail back testing.
No, module reference counts are only for code that is touching function pointers.
this is exactly the case here. it is the skb destructor function that is carried by the skb.
Quoted 6 lines
Quoted 2 lines
It can't possibly call it correctly without also having the rxe
ib_devicereference too though??Nope. This was causing seg faults in testing when there was a long network hang and the admin tried to reload the rxe driver. The skb code doesn't care about the ib device at all.
I don't get it, there aren't globals in rxe, so WTF is it doing if it isn't somehow tracing back to memory that is under the
ib_devicelifetime?
Quoted 8 lines
Quoted 1 line
Jason
When the rxe driver builds a send packet it puts the address of its destructor subroutine in the skb before calling
ip_local_outand sending it. The address of driver software is now hanging around. If you don't delay the module exit routine until all the skb's are freed you can cause seg faults. The only way to cause this to happen is to call rmmod on the driver too early but people have done this occasionally and report it as a bug.
You are missing the point, the destructor currently does this:
static void rxe_skb_tx_dtor(struct sk_buff *skb)
{
struct sock *sk = skb->sk;
struct rxe_qp *qp = sk->sk_user_data;So you've already UAF'd because rxe_qp is freed memory well before you get to unloading the module.
This series changed it to do this:
static void rxe_skb_tx_dtor(struct sk_buff *skb)
{
struct rxe_dev *rxe;
unsigned int index;
struct rxe_qp *qp;
int skb_out;
/* takes a ref on ib device if success */
rxe = get_rxe_from_skb(skb);
if (!rxe)
goto out;static struct rxe_dev *get_rxe_from_skb(struct sk_buff *skb)
{
struct rxe_dev *rxe;
struct net_device *ndev = skb->dev;
rxe = rxe_get_dev_from_net(ndev);
if (!rxe && is_vlan_dev(ndev))
rxe = rxe_get_dev_from_net(vlan_dev_real_dev(ndev));Which seems totally nutz, you are now relying on the global hash table in ib_core to resolve the ib device.
Again, why can't this code do something sane like refcount the qp or ib_device so the destruction doesn't progress until all the SKBs are flushed out?
Jason