Skip to contentSkip to conversation

Possible bug in test_mr.py

DISCUSSION Bob Pearson 2 participants 3 messages Mar 29, 2021 1 message not in the archive InfiniBand

lore
Share & tools
lore

A message that is not in the archive (sent off-list or not yet received): [email protected]

#1 lore raw

-------- Forwarded Message --------
Subject: Possible bug in test_mr.py
Date: Sun, 28 Mar 2021 22:52:01 -0500
From: Bob Pearson <[email protected]>
To: Jason Gunthorpe <[email protected]>, Zhu Yanjun <[email protected]>, [email protected]

Testing ibv_rereg_mr() I noticed that the test uses the rkey originally assigned to the MR by ibv_reg_mr() and not the rkey subsequently assigned after calling ibv_rereg_mr(). This matters when the original MR did not have remote memory access and rkey was set to zero. If the rereg changes access to allow remote memory access then the rkey is set when the verb returns. But the test code never looks again after setting up the original MRs.

In rxe setting rkey = lkey always gets the first rereg test case to pass.

bob

#2 lore raw

Hi Bob,

You're referring to the "test_mr_rereg_access" of test_mr.py, aren't you?
We need to re-sync the rkeys between the two sides, for example by calling the sync_remote_attr() after reregistering the MRs. And if that's the case, and the MR keys could change, maybe it's better to re-sync the keys between the two sides after each rereg call.

I'll push a fix - in case I misunderstood you or that's not the point you tried to make, please let me know.

Thanks,
Edward.

Quoted 16 lines · Bob Pearson

-------- Forwarded Message --------
Subject: Possible bug in test_mr.py
Date: Sun, 28 Mar 2021 22:52:01 -0500
From: Bob Pearson <[email protected]>
To: Jason Gunthorpe <[email protected]>, Zhu Yanjun <[email protected]>, [email protected]

Testing ibv_rereg_mr() I noticed that the test uses the rkey originally assigned to the MR by ibv_reg_mr() and not the rkey subsequently assigned after calling ibv_rereg_mr(). This matters when the original MR did not have remote memory access and rkey was set to zero. If the rereg changes access to allow remote memory access then the rkey is set when the verb returns. But the test code never looks again after setting up the original MRs.

In rxe setting rkey = lkey always gets the first rereg test case to pass.

bob

#3 lore raw

Possible bug in test_mr_rereg_pd

Edward,

In a later test (test_mr_rereg_pd) which is also failing, I get the following

======================================================================
ERROR: test_mr_rereg_pd (tests.test_mr.MRTest)
Test that cover rereg MR's PD with this flow:
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/rpearson/src/rdma-core/tests/test_mr.py", line 160, in test_mr_rereg_pd
u.traffic(**self.traffic_args)
File "/home/rpearson/src/rdma-core/tests/utils.py", line 653, in traffic
poll(client.cq)
File "/home/rpearson/src/rdma-core/tests/utils.py", line 524, in poll_cq
raise PyverbsRDMAError('Completion status is {s}'.
pyverbs.pyverbs_error.PyverbsRDMAError: Completion status is WR flush error. Errno: 5, Input/output error

But, adding tracing to the kernel driver I see that that part of the test actually succeeded after resetting the two QPs and reregistering the MR back to original PD. However, there were some WQEs that got flushed when the QPs were reset but I don't see anything in the test to drain the RCQs after the reset. So it is not surprising that the test sees the flush errors when it polls the RCQ.

Since CQs are independent of QPs I thought it was correct to show these completions as flushed even if the QP is reset.

Bob