Skip to contentSkip to conversation

RDMA/rxe: Fix "Replace red-black trees by xarrays"

PATCH In review Bob Pearson 6 participants 10 messages Apr 10, 2022 – Apr 12, 2022

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

PATCH 1/1 RDMA/rxe: Fix "Replace red-black trees by xarrays"

The referenced commit causes lockdep warnings by using the default spin_lock in xa_alloc_cyclic and xa_erase which include calls to xa_lock()/xa_unlock() while at the same time explicitly calling xa_lock_irqsave() in rxe_pool_get_index().

The latter is required to handle some object lookups correctly. The immediate fix is to explicitly use xa_lock_irqsave() everywhere.

This commit replaces xa_alloc_cyclic() by __xa_alloc_cyclic() and xa_erase() by __xa_erase() and explicitly lock these calls with xa_lock_irqsave().

This commit will be reverted later when the read side operations in rxe_pool.c will be converted to rcu_read_locks which will not require locking the write side operations with irqsave locks.

This commit fixes the "WARNING: Inconsistent lock state" bug in blktests. The recent revert patch from Bart fixes the other bug in blktests with very long delays.

Fixes 3225717f6dfa ("RDMA/rxe: Replace red-black trees by carrays")
---
 drivers/infiniband/sw/rxe/rxe_pool.c | 18 +++++++++++++++---
 1 file changed, 15 insertions(+), 3 deletions(-)
drivers/infiniband/sw/rxe/rxe_pool.c +15 -3
index 87066d04ed18..440f96af213b 100644
@@ -118,7 +118,9 @@ void rxe_pool_cleanup(struct rxe_pool *pool)
 
 void *rxe_alloc(struct rxe_pool *pool)
 {
+	struct xarray *xa = &pool->xa;
 	struct rxe_pool_elem *elem;
+	unsigned long flags;
 	void *obj;
 	int err;
 
@@ -138,8 +140,10 @@ void *rxe_alloc(struct rxe_pool *pool)
 	elem->obj = obj;
 	kref_init(&elem->ref_cnt);
 
-	err = xa_alloc_cyclic(&pool->xa, &elem->index, elem, pool->limit,
+	xa_lock_irqsave(xa, flags);
+	err = __xa_alloc_cyclic(&pool->xa, &elem->index, elem, pool->limit,
 			      &pool->next, GFP_KERNEL);
+	xa_unlock_irqrestore(xa, flags);
 	if (err)
 		goto err_free;
 
@@ -154,6 +158,8 @@ void *rxe_alloc(struct rxe_pool *pool)
 
 int __rxe_add_to_pool(struct rxe_pool *pool, struct rxe_pool_elem *elem)
 {
+	struct xarray *xa = &pool->xa;
+	unsigned long flags;
 	int err;
 
 	if (WARN_ON(pool->flags & RXE_POOL_ALLOC))
@@ -166,8 +172,10 @@ int __rxe_add_to_pool(struct rxe_pool *pool, struct rxe_pool_elem *elem)
 	elem->obj = (u8 *)elem - pool->elem_offset;
 	kref_init(&elem->ref_cnt);
 
-	err = xa_alloc_cyclic(&pool->xa, &elem->index, elem, pool->limit,
+	xa_lock_irqsave(xa, flags);
+	err = __xa_alloc_cyclic(&pool->xa, &elem->index, elem, pool->limit,
 			      &pool->next, GFP_KERNEL);
+	xa_unlock_irqrestore(xa, flags);
 	if (err)
 		goto err_cnt;
 
@@ -200,8 +208,12 @@ static void rxe_elem_release(struct kref *kref)
 {
 	struct rxe_pool_elem *elem = container_of(kref, typeof(*elem), ref_cnt);
 	struct rxe_pool *pool = elem->pool;
+	struct xarray *xa = &pool->xa;
+	unsigned long flags;
 
-	xa_erase(&pool->xa, elem->index);
+	xa_lock_irqsave(xa, flags);
+	__xa_erase(&pool->xa, elem->index);
+	xa_unlock_irqrestore(xa, flags);
 
 	if (pool->cleanup)
 		pool->cleanup(elem);
Signature · 1 line
2.32.0
#1 lore raw
Quoted 1 line · Bob Pearson
Fixes 3225717f6dfa ("RDMA/rxe: Replace red-black trees by carrays")
                                                             ^^^^^^^
                                                             xarrays?

drivers/infiniband/sw/rxe/rxe_pool.c void *rxe_alloc(struct rxe_pool *pool) · line 146

Quoted 9 lines
drivers/infiniband/sw/rxe/rxe_pool.c +3 -1
@@ -138,8 +140,10 @@ void *rxe_alloc(struct rxe_pool *pool)
  	elem->obj = obj;
  	kref_init(&elem->ref_cnt);
 
-	err = xa_alloc_cyclic(&pool->xa, &elem->index, elem, pool->limit,
+	xa_lock_irqsave(xa, flags);
+	err = __xa_alloc_cyclic(&pool->xa, &elem->index, elem, pool->limit,
  			      &pool->next, GFP_KERNEL);
+	xa_unlock_irqrestore(xa, flags);

Please take a look at the xas_unlock_type() and xas_lock_type() calls in __xas_nomem(). I think that the above change will trigger a GFP_KERNEL allocation with interrupts disabled. My understanding is that GFP_KERNEL allocations may sleep and hence that the above code may cause __xas_nomem() to sleep with interrupts disabled. I don't think that is allowed.

Thanks,

Bart.

#2 lore raw
Quoted 16 lines · Bart Van Assche
Quoted 1 line · Bob Pearson
Fixes 3225717f6dfa ("RDMA/rxe: Replace red-black trees by carrays")

                                                             ^^^^^^^ xarrays?

drivers/infiniband/sw/rxe/rxe_pool.c void *rxe_alloc(struct rxe_pool *pool) · line 146

Quoted 8 lines
drivers/infiniband/sw/rxe/rxe_pool.c +3 -0
@@ -138,8 +140,10 @@ void *rxe_alloc(struct rxe_pool *pool)
       elem->obj = obj;
       kref_init(&elem->ref_cnt);
   -    err = xa_alloc_cyclic(&pool->xa, &elem->index, elem, pool->limit,
+    xa_lock_irqsave(xa, flags);
+    err = __xa_alloc_cyclic(&pool->xa, &elem->index, elem, pool->limit,
                     &pool->next, GFP_KERNEL);
+    xa_unlock_irqrestore(xa, flags);

Please take a look at the xas_unlock_type() and xas_lock_type() calls in __xas_nomem(). I think that the above change will trigger a GFP_KERNEL allocation with interrupts disabled. My understanding is that GFP_KERNEL allocations may sleep and hence that the above code may cause __xas_nomem() to sleep with interrupts disabled. I don't think that is allowed.

Quoted 3 lines

Thanks,

Bart.

You're right. I missed that. Zhu wants to write the patch so hopefully he's on top of that. For now we could use GFP_ATOMIC.

Bob

maintainer
#3 lore raw
Quoted 17 lines · Bob Pearson
Quoted 16 lines · Bart Van Assche
Quoted 1 line · Bob Pearson
Fixes 3225717f6dfa ("RDMA/rxe: Replace red-black trees by carrays")

                                                             ^^^^^^^ xarrays?

drivers/infiniband/sw/rxe/rxe_pool.c void *rxe_alloc(struct rxe_pool *pool) · line 146

Quoted 8 lines
drivers/infiniband/sw/rxe/rxe_pool.c +3 -0
@@ -138,8 +140,10 @@ void *rxe_alloc(struct rxe_pool *pool)
       elem->obj = obj;
       kref_init(&elem->ref_cnt);
   -    err = xa_alloc_cyclic(&pool->xa, &elem->index, elem, pool->limit,
+    xa_lock_irqsave(xa, flags);
+    err = __xa_alloc_cyclic(&pool->xa, &elem->index, elem, pool->limit,
                     &pool->next, GFP_KERNEL);
+    xa_unlock_irqrestore(xa, flags);

Please take a look at the xas_unlock_type() and xas_lock_type() calls in __xas_nomem(). I think that the above change will trigger a GFP_KERNEL allocation with interrupts disabled. My understanding is that GFP_KERNEL allocations may sleep and hence that the above code may cause __xas_nomem() to sleep with interrupts disabled. I don't think that is allowed.

Quoted 6 lines
Quoted 3 lines

Thanks,

Bart.

You're right. I missed that. Zhu wants to write the patch so hopefully he's on top of that. For now we could use GFP_ATOMIC.

Yes, you cannot use irq_save varients here. You have to know your calling context is non-atomic already and use the irq wrapper.

Jason

#4 lore raw
Quoted 2 lines

Yes, you cannot use irq_save variants here. You have to know your calling context is non-atomic already and use the irq wrapper.

Not sure why. If you call irqsave in a non-atomic context doesn't it behave the same as irq? I.e. flags = 0. xarray provides xa_lock_xxx() for people to use. Are you saying I have to call xa_alloc_cyclic_xxx() instead which is the same thing?

The problem is I've gotten gun shy about people calling into the verbs APIs in strange contexts. The rules don't seem to be written down. If I could be certain that no one ever is allowed to call a verbs API in an interrupt then this is correct but how do I know that?

Bob

maintainer
#5 lore raw
Quoted 7 lines · Robert B
Quoted 2 lines

Yes, you cannot use irq_save variants here. You have to know your calling context is non-atomic already and use the irq wrapper.

Not sure why. If you call irqsave in a non-atomic context doesn't it behave the same as irq? I.e. flags = 0. xarray provides xa_lock_xxx() for people to use. Are you saying I have to call xa_alloc_cyclic_xxx() instead which is the same thing?

The xa_lock is a magic thing, when you call a __xa_*(.., GFP_KERNEL) type function then it will unlock and relock the xa_lock internally.

Thus you cannot wrapper an irqsave lock across the GFP_KERNEL call sites because XA doesn't know the 'flags' to be able to properly unlock it.

Quoted 5 lines

The problem is I've gotten gun shy about people calling into the verbs APIs in strange contexts. The rules don't seem to be written down. If I could be certain that no one ever is allowed to call a verbs API in an interrupt then this is correct but how do I know that?

rxe used GFP_KERNEL so it already has assumed it is a process context.

Jason

#6 lore raw
Quoted 16 lines · Jason Gunthorpe
Quoted 7 lines · Robert B
Quoted 2 lines

Yes, you cannot use irq_save variants here. You have to know your calling context is non-atomic already and use the irq wrapper.

Not sure why. If you call irqsave in a non-atomic context doesn't it behave the same as irq? I.e. flags = 0. xarray provides xa_lock_xxx() for people to use. Are you saying I have to call xa_alloc_cyclic_xxx() instead which is the same thing?

The xa_lock is a magic thing, when you call a __xa_*(.., GFP_KERNEL) type function then it will unlock and relock the xa_lock internally.

Thus you cannot wrapper an irqsave lock across the GFP_KERNEL call sites because XA doesn't know the 'flags' to be able to properly unlock it.

Quoted 7 lines
Quoted 5 lines

The problem is I've gotten gun shy about people calling into the verbs APIs in strange contexts. The rules don't seem to be written down. If I could be certain that no one ever is allowed to call a verbs API in an interrupt then this is correct but how do I know that?

rxe used GFP_KERNEL so it already has assumed it is a process context.

Got it.

__xa_alloc_cyclic(..., GFP_NOWAIT) will unlock xa lock internally. But it does not handle flags. So the irq is still disabled. Because GFP_NOWAIT is used in __xa_alloc_cyclic, it will not sleep.

__xa_alloc_cyclic will lock xa lock and return to xa_unlock_irqrestore.

So the following code should be OK?

        xa_lock_irqsave(&pool->xa, flags);
        err = __xa_alloc_cyclic(&pool->xa, &elem->index, elem, pool->limit,
                                &pool->next, GFP_NOWAIT);
        xa_unlock_irqrestore(&pool->xa, flags);

Zhu Yanjun

Quoted 2 lines

Jason

maintainer
#7 lore raw
Quoted 30 lines · Yanjun Zhu
Quoted 16 lines · Jason Gunthorpe
Quoted 7 lines · Robert B
Quoted 2 lines

Yes, you cannot use irq_save variants here. You have to know your calling context is non-atomic already and use the irq wrapper.

Not sure why. If you call irqsave in a non-atomic context doesn't it behave the same as irq? I.e. flags = 0. xarray provides xa_lock_xxx() for people to use. Are you saying I have to call xa_alloc_cyclic_xxx() instead which is the same thing?

The xa_lock is a magic thing, when you call a __xa_*(.., GFP_KERNEL) type function then it will unlock and relock the xa_lock internally.

Thus you cannot wrapper an irqsave lock across the GFP_KERNEL call sites because XA doesn't know the 'flags' to be able to properly unlock it.

Quoted 7 lines
Quoted 5 lines

The problem is I've gotten gun shy about people calling into the verbs APIs in strange contexts. The rules don't seem to be written down. If I could be certain that no one ever is allowed to call a verbs API in an interrupt then this is correct but how do I know that?

rxe used GFP_KERNEL so it already has assumed it is a process context.

Got it.

__xa_alloc_cyclic(..., GFP_NOWAIT) will unlock xa lock internally. But it does not handle flags. So the irq is still disabled. Because GFP_NOWAIT is used in __xa_alloc_cyclic, it will not sleep.

Quoted 8 lines

__xa_alloc_cyclic will lock xa lock and return to xa_unlock_irqrestore.

So the following code should be OK?

        xa_lock_irqsave(&pool->xa, flags);
        err = __xa_alloc_cyclic(&pool->xa, &elem->index, elem, pool->limit,
                                &pool->next, GFP_NOWAIT);
        xa_unlock_irqrestore(&pool->xa, flags);

No, use GFP_KERNEL an use the proper irq or bh version of xa_alloc_cyclic.

save is only required if you don't know the contex you are in, and here we know we are in a process context.

Jason

#8 lore raw
Quoted 35 lines · Jason Gunthorpe
Quoted 26 lines · Yanjun Zhu
Quoted 13 lines · Jason Gunthorpe
Quoted 5 lines · Robert B
Quoted 1 line

Yes, you cannot use irq_save variants here. You have to know your calling context is non-atomic already and use the irq wrapper.

Not sure why. If you call irqsave in a non-atomic context doesn't it behave the same as irq? I.e. flags = 0. xarray provides xa_lock_xxx() for people to use. Are you saying I have to call xa_alloc_cyclic_xxx() instead which is the same thing?

The xa_lock is a magic thing, when you call a __xa_*(.., GFP_KERNEL) type function then it will unlock and relock the xa_lock internally.

Thus you cannot wrapper an irqsave lock across the GFP_KERNEL call sites because XA doesn't know the 'flags' to be able to properly unlock it.

Quoted 6 lines
Quoted 5 lines

The problem is I've gotten gun shy about people calling into the verbs APIs in strange contexts. The rules don't seem to be written down. If I could be certain that no one ever is allowed to call a verbs API in an interrupt then this is correct but how do I know that?

rxe used GFP_KERNEL so it already has assumed it is a process context.

Got it.

__xa_alloc_cyclic(..., GFP_NOWAIT) will unlock xa lock internally. But it does not handle flags. So the irq is still disabled. Because GFP_NOWAIT is used in __xa_alloc_cyclic, it will not sleep.

Quoted 8 lines

__xa_alloc_cyclic will lock xa lock and return to xa_unlock_irqrestore.

So the following code should be OK?

         xa_lock_irqsave(&pool->xa, flags);
         err = __xa_alloc_cyclic(&pool->xa, &elem->index, elem, pool->limit,
                                 &pool->next, GFP_NOWAIT);
         xa_unlock_irqrestore(&pool->xa, flags);
Quoted 5 lines

No, use GFP_KERNEL an use the proper irq or bh version of xa_alloc_cyclic.

save is only required if you don't know the contex you are in, and here we know we are in a process context.

if we use xa_lock_irq/xa_unlock_irq or xa_lock_bh/xa_unlock_bh instead of xa_unlock_irqrestore,

the warning as below will appear. This means that __rxe_add_to_pool disables softirq, but fpu_clone enables softirq.

"

Apr 12 16:24:53 kernel: softirqs last  enabled at (13086): [<ffffffff91830d26>] fpu_clone+0xf6/0x570
Apr 12 16:24:53 kernel: softirqs last disabled at (13129): [<ffffffffc077f319>] __rxe_add_to_pool+0x49/0xa0 [rdma_rxe]

"

As such, it is better to use xa_unlock_irqrestore + __xa_alloc(...,GFP_ATOMIC/GFP_NOWAIT).

Zhu Yanjun

Quoted 2 lines

Jason

maintainer
#9 lore raw
In reply to Bart Van Assche
Quoted 10 lines · Bart Van Assche
Quoted 1 line · Bob Pearson
Fixes 3225717f6dfa ("RDMA/rxe: Replace red-black trees by carrays")
                                                              ^^^^^^^
                                                              xarrays?
Quoted 4 lines
drivers/infiniband/sw/rxe/rxe_pool.c +0 -0
@@ -138,8 +140,10 @@ void *rxe_alloc(struct rxe_pool *pool)
      elem->obj = obj;
      kref_init(&elem->ref_cnt);
 
Quoted 12 lines

drivers/infiniband/sw/rxe/rxe_pool.c void *rxe_alloc(struct rxe_pool *pool) · line 146

Quoted 5 lines
drivers/infiniband/sw/rxe/rxe_pool.c +3 -1
-     err = xa_alloc_cyclic(&pool->xa, &elem->index, elem, pool->limit,
+     xa_lock_irqsave(xa, flags);
+     err = __xa_alloc_cyclic(&pool->xa, &elem->index, elem, pool->limit,
                            &pool->next, GFP_KERNEL);
+     xa_unlock_irqrestore(xa, flags);

Please take a look at the xas_unlock_type() and xas_lock_type() calls in __xas_nomem(). I think that the above change will trigger a GFP_KERNEL allocation with interrupts disabled. My understanding is that GFP_KERNEL allocations may sleep and hence that the above code may cause __xas_nomem() to sleep with interrupts disabled. I don't think that is allowed.

Thanks. I will send V2.

Zhu Yanjun

Quoted 4 lines

Thanks,

Bart.