xdp: obtain the mem_id mutex before trying to remove an entry.
Subsystems (3)
PATCH 1/1 xdp: obtain the mem_id mutex before trying to remove an entry.
A lockdep splat was observed when trying to remove an xdp memory model from the table since the mutex was obtained when trying to remove the entry, but not before the table walk started:
Fix the splat by obtaining the lock before starting the table walk.
--- net/core/xdp.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-)
net/core/xdp.c +4 -4
@@ -80,12 +80,8 @@ static void mem_xa_remove(struct xdp_mem_allocator *xa) {trace_mem_disconnect (xa); -mutex_lock (&mem_id_lock); -if (!rhashtable_remove_fast (mem_id_ht, &xa->node, mem_id_rht_params))call_rcu (&xa->rcu, __xdp_mem_allocator_rcu_free); - -mutex_unlock (&mem_id_lock); }static void mem_allocator_disconnect (void *allocator)
@@ -93,6 +89,8 @@ static void mem_allocator_disconnect(void *allocator)struct xdp_mem_allocator *xa;struct rhashtable_iter iter; +mutex_lock (&mem_id_lock); +rhashtable_walk_enter (mem_id_ht, &iter); do { rhashtable_walk_start(&iter);
@@ -106,6 +104,8 @@ static void mem_allocator_disconnect(void *allocator) }while (xa ==ERR_PTR (-EAGAIN ));rhashtable_walk_exit (&iter); + +mutex_unlock (&mem_id_lock); }static void mem_id_disconnect (int id)
Signature · 1 line
2.17.1
Quoted 9 lines · On Tue, 3 Dec 2019 14:01:14 -0800 Jonathan Lemon <[email protected]>…
A lockdep splat was observed when trying to remove an xdp memory model from the table since the mutex was obtained when trying to remove the entry, but not before the table walk started:
Fix the splat by obtaining the lock before starting the table walk.
Fixes c3f812cea0d7 ("page_pool: do not release pool until inflight == 0.")Reported-by Grygorii Strashko <[email protected]>Signed-off-by Jonathan Lemon <[email protected]>
Have you tested if this patch fix the problem reported by Grygorii?
Grygorii can you test this?
Quoted 31 lines
--- net/core/xdp.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-)net/core/xdp.c +2
-4@@ -80,12 +80,8 @@ static void mem_xa_remove(struct xdp_mem_allocator *xa) {trace_mem_disconnect (xa); -mutex_lock (&mem_id_lock); -if (!rhashtable_remove_fast (mem_id_ht, &xa->node, mem_id_rht_params))call_rcu (&xa->rcu, __xdp_mem_allocator_rcu_free); - -mutex_unlock (&mem_id_lock); }static void mem_allocator_disconnect (void *allocator)@@ -93,6 +89,8 @@ static void mem_allocator_disconnect(void *allocator)struct xdp_mem_allocator *xa;struct rhashtable_iter iter; +mutex_lock (&mem_id_lock); +rhashtable_walk_enter (mem_id_ht, &iter); do { rhashtable_walk_start(&iter);
net/core/xdp.c static void mem_allocator_disconnect(void *allocator) · line 111
Quoted 8 lines
net/core/xdp.c +2
-0@@ -106,6 +104,8 @@ static void mem_allocator_disconnect(void *allocator) }while (xa ==ERR_PTR (-EAGAIN ));rhashtable_walk_exit (&iter); + +mutex_unlock (&mem_id_lock); }static void mem_id_disconnect (int id)
Moving the mutex-lock might be a good idea, but I'm not sure that fixes the issue. I'm more suspect about the usage of rcu_read_lock() in xdp_rxq_info_unreg_mem_model(), listed below.
void xdp_rxq_info_unreg_mem_model(struct xdp_rxq_info *xdp_rxq)
{
struct xdp_mem_allocator *xa;
int id = xdp_rxq->mem.id;
if (xdp_rxq->reg_state != REG_STATE_REGISTERED) {
WARN(1, "Missing register, driver bug");
return;
}
if (id == 0)
return;
if (xdp_rxq->mem.type == MEM_TYPE_ZERO_COPY)
return mem_id_disconnect(id);
if (xdp_rxq->mem.type == MEM_TYPE_PAGE_POOL) {
rcu_read_lock();
xa = rhashtable_lookup(mem_id_ht, &id, mem_id_rht_params);
page_pool_destroy(xa->page_pool);
rcu_read_unlock();
}
}EXPORT_SYMBOL_GPL(xdp_rxq_info_unreg_mem_model);
Signature · 4 lines
Best regards, Jesper Dangaard Brouer MSc.CS, Principal Kernel Engineer at Red Hat LinkedIn: http://www.linkedin.com/in/brouer
Grygorii Strashko
Tested-byQuoted 13 lines · Jesper Dangaard Brouer
Quoted 9 lines · On Tue, 3 Dec 2019 14:01:14 -0800 Jonathan Lemon <[email protected]>…
A lockdep splat was observed when trying to remove an xdp memory model from the table since the mutex was obtained when trying to remove the entry, but not before the table walk started:
Fix the splat by obtaining the lock before starting the table walk.
Fixes c3f812cea0d7 ("page_pool: do not release pool until inflight == 0.")Reported-by Grygorii Strashko <[email protected]>Signed-off-by Jonathan Lemon <[email protected]>
Quoted 5 lines
Have you tested if this patch fix the problem reported by Grygorii?
Grygorii can you test this?
Thanks.
I do not see this trace any more and networking is working after if down/up
Signature · 2 lines
Best regards, grygorii
Jesper Dangaard Brouer
Acked-byQuoted 16 lines · On Wed, 4 Dec 2019 12:07:22 +0200 Grygorii Strashko <[email protected]>…
Quoted 15 lines · Jesper Dangaard Brouer
Quoted 9 lines · On Tue, 3 Dec 2019 14:01:14 -0800 Jonathan Lemon <[email protected]>…
A lockdep splat was observed when trying to remove an xdp memory model from the table since the mutex was obtained when trying to remove the entry, but not before the table walk started:
Fix the splat by obtaining the lock before starting the table walk.
Fixes c3f812cea0d7 ("page_pool: do not release pool until inflight == 0.")Reported-by Grygorii Strashko <[email protected]>Signed-off-by Jonathan Lemon <[email protected]>Have you tested if this patch fix the problem reported by Grygorii?
Quoted 9 lines
Quoted 3 lines
Grygorii can you test this?
Thanks.
I do not see this trace any more and networking is working after if down/upTested-by Grygorii Strashko <[email protected]>
Well if it fixes you issue, then I guess its okay.
I just though it was related to the rcu_read_lock() around the page_pool_destroy() call. Guess, I was wrong.
Signature · 4 lines
Best regards, Jesper Dangaard Brouer MSc.CS, Principal Kernel Engineer at Red Hat LinkedIn: http://www.linkedin.com/in/brouer
Quoted 11 lines · Jesper Dangaard Brouer
Quoted 8 lines · On Wed, 4 Dec 2019 12:07:22 +0200 Grygorii Strashko <[email protected]>…
Quoted 7 lines · Jesper Dangaard Brouer
Quoted 4 lines · On Tue, 3 Dec 2019 14:01:14 -0800 Jonathan Lemon <[email protected]>…
A lockdep splat was observed when trying to remove an xdp memory model from the table since the mutex was obtained when trying to remove the entry, but not before the table walk started:
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 5 lines
Fix the splat by obtaining the lock before starting the table walk.
Fixes c3f812cea0d7 ("page_pool: do not release pool until inflight == 0.")Reported-by Grygorii Strashko <[email protected]>Signed-off-by Jonathan Lemon <[email protected]>Have you tested if this patch fix the problem reported by Grygorii?
Yes, I reproduced the problem locally, and confirmed that the patch resolves the issue.
--
Jonathan
Quoted 23 lines
Quoted 10 lines
Quoted 4 lines
Grygorii can you test this?
Thanks.
I do not see this trace any more and networking is working after if down/upTested-by Grygorii Strashko <[email protected]>Well if it fixes you issue, then I guess its okay.
I just though it was related to the
rcu_read_lock()around thepage_pool_destroy()call. Guess, I was wrong.Signature · 4 lines
Best regards, Jesper Dangaard Brouer MSc.CS, Principal Kernel Engineer at Red Hat LinkedIn: http://www.linkedin.com/in/brouer
Ilias Apalodimas
Acked-byQuoted 18 lines · Grygorii Strashko
Quoted 15 lines · Jesper Dangaard Brouer
Quoted 9 lines · On Tue, 3 Dec 2019 14:01:14 -0800 Jonathan Lemon <[email protected]>…
A lockdep splat was observed when trying to remove an xdp memory model from the table since the mutex was obtained when trying to remove the entry, but not before the table walk started:
Fix the splat by obtaining the lock before starting the table walk.
Fixes c3f812cea0d7 ("page_pool: do not release pool until inflight == 0.")Reported-by Grygorii Strashko <[email protected]>Signed-off-by Jonathan Lemon <[email protected]>Have you tested if this patch fix the problem reported by Grygorii?
Quoted 13 lines
Quoted 3 lines
Grygorii can you test this?
Thanks.
I do not see this trace any more and networking is working after if down/upTested-by Grygorii Strashko <[email protected]>Signature · 2 lines
Best regards, grygorii
From: Jonathan Lemon <[email protected]>
Date: Tue, 3 Dec 2019 14:01:14 -0800
Quoted 9 lines
A lockdep splat was observed when trying to remove an xdp memory model from the table since the mutex was obtained when trying to remove the entry, but not before the table walk started:
Fix the splat by obtaining the lock before starting the table walk.
Fixes c3f812cea0d7 ("page_pool: do not release pool until inflight == 0.")Reported-by Grygorii Strashko <[email protected]>Signed-off-by Jonathan Lemon <[email protected]>
Applied, and since this fixes a bug fix that went back to v5.3 in -stable I will queue it there as well.
Thanks.