Skip to contentSkip to conversation

10 more python test cases for rxe

PATCH Bob Pearson 5 participants 6 messages Oct 13, 2021 – Oct 20, 2021 InfiniBand

lore
Share & tools
lore
#0 lore raw

Zhu,

There are about 10 test cases in the python suite that do not run for rxe because

... skipped "Device rxe0 doesn't have net interface"

Clearly this is wrong and I don't know how to address the root cause yet but the following hack where enp0s3 is the actual net device that rxe0 is based on in my case enables these test cases to run and it appears they all do.

tests/base.py +0 -0

index 3460c546..d6fd29b8 100644

--- a/tests/base.py

+++ b/tests/base.py

@@ -240,10 +240,11 @@ class RDMATestCase(unittest.TestCase):

             if self.gid_type is not None and ctx.query_gid_type(port, idx) != \

                     self.gid_type:

                 continue

- if not os.path.exists('/sys/class/infiniband/{}/device/net/'.format(dev)):

- self.args.append([dev, port, idx, None, None])

- continue

- net_name = self.get_net_name(dev)

+ #if not os.path.exists('/sys/class/infiniband/{}/device/net/'.format(dev)):

+ #self.args.append([dev, port, idx, None, None])

+ #continue

+ #net_name = self.get_net_name(dev)

+ net_name = "enp0s3"

try:

ip_addr, mac_addr = self.get_ip_mac_address(net_name)

except (KeyError, IndexError):

Apparently base.py expects there to be a directory /sys/class/infiniband/rxe0/device/net that contains another directory with the same name as enp0s3. When rdma-core builds the kobj tree 'device/net' is not added to rxe0. There is a 'parent' plain file in rxe0 with the contents enp0s3 so there should be an easy work around in base.py but I don't know if there are other places where the 'device/net' branch is expected.

If you know the best approach to fixing this please let me know. In the mean time this gives you a way to run the rdmacm test cases.

Bob

maintainer
#1 lore raw
Quoted 24 lines · Bob Pearson

Zhu,

There are about 10 test cases in the python suite that do not run for rxe because

... skipped "Device rxe0 doesn't have net interface"

Clearly this is wrong and I don't know how to address the root cause yet but the following hack where enp0s3 is the actual net device that rxe0 is based on in my case enables these test cases to run and it appears they all do.

tests/base.py +0 -0

index 3460c546..d6fd29b8 100644

+++ b/tests/base.py

tests/base.py +0 -0
@@ -240,10 +240,11 @@ class RDMATestCase(unittest.TestCase):
 
             if self.gid_type is not None and ctx.query_gid_type(port, idx) != \
 
                     self.gid_type:
 
                 continue

tests/base.py class RDMATestCase(unittest.TestCase):

Quoted 10 lines
tests/base.py +1 -4
 
-            if not os.path.exists('/sys/class/infiniband/{}/device/net/'.format(dev)):
 
-                self.args.append([dev, port, idx, None, None])
 
-                continue
 
-            net_name = self.get_net_name(dev)
 
+            #if not os.path.exists('/sys/class/infiniband/{}/device/net/'.format(dev)):

The pytests code is wrong - it should be querying the netdev through the verbs APIs, not hacking in sysfs like this.

Jason

#2 lore raw
Quoted 31 lines · Jason Gunthorpe
Quoted 29 lines · Bob Pearson

Zhu,

There are about 10 test cases in the python suite that do not run for rxe because

... skipped "Device rxe0 doesn't have net interface"

Clearly this is wrong and I don't know how to address the root cause yet but the following hack where enp0s3 is the actual net device that rxe0 is based on in my case enables these test cases to run and it appears they all do.

tests/base.py +0 -0

index 3460c546..d6fd29b8 100644

+++ b/tests/base.py

tests/base.py +0 -2
@@ -240,10 +240,11 @@ class RDMATestCase(unittest.TestCase):
 
             if self.gid_type is not None and ctx.query_gid_type(port, idx) != \
 
                     self.gid_type:
 
                 continue
 
-            if not os.path.exists('/sys/class/infiniband/{}/device/net/'.format(dev)):
 
-                self.args.append([dev, port, idx, None, None])
Quoted 8 lines

tests/base.py class RDMATestCase(unittest.TestCase):

Quoted 5 lines
tests/base.py +1 -2
-                continue
 
-            net_name = self.get_net_name(dev)
 
+            #if not os.path.exists('/sys/class/infiniband/{}/device/net/'.format(dev)):

The pytests code is wrong - it should be querying the netdev through the verbs APIs, not hacking in sysfs like this.

Got it. Thanks

Zhu Yanjun

Quoted 2 lines

Jason

#3 lore raw
Quoted 41 lines · Jason Gunthorpe
Quoted 12 lines · Bob Pearson
Quoted 10 lines

Zhu,

There are about 10 test cases in the python suite that do not run for rxe because

... skipped "Device rxe0 doesn't have net interface"

Clearly this is wrong and I don't know how to address the root cause yet but the following hack where enp0s3 is the actual net device that rxe0 is based on in my case enables these test cases to run and it

appears they all do.

Quoted 10 lines
Quoted 10 lines
tests/base.py +0 -0

index 3460c546..d6fd29b8 100644

+++ b/tests/base.py

tests/base.py +0 -0
@@ -240,10 +240,11 @@ class RDMATestCase(unittest.TestCase):
 
Quoted 8 lines

tests/base.py class RDMATestCase(unittest.TestCase): · line 248

Quoted 8 lines
tests/base.py +0 -1
             if self.gid_type is not None and
 ctx.query_gid_type(port, idx) != \
 
                     self.gid_type:
 
                 continue
 
-            if not

os.path.exists('/sys/class/infiniband/{}/device/net/'.format(dev)):

Quoted 8 lines

tests/base.py class RDMATestCase(unittest.TestCase):

Quoted 8 lines
tests/base.py +1 -3
-                self.args.append([dev, port, idx, None, None])
 
-                continue
 
-            net_name = self.get_net_name(dev)
 
+            #if not
Quoted 11 lines

os.path.exists('/sys/class/infiniband/{}/device/net/'.format(dev)):

Quoted 3 lines

The pytests code is wrong - it should be querying the netdev through the verbs APIs, not hacking in sysfs like this.

Got it. Thanks

Zhu Yanjun

Quoted 2 lines

Jason

I will modify the base test file to use verbs API instead of accessing the sysfs directly.
I wanted to do that using ibv_query_gid_ex to get the netdev ifindex, but in case of IB (IPoIB) the netdex is just 0 and not updated accordingly. Not sure if it's a bug or by design (looks like a bug for me).
I'll check that and update accordingly.

Thanks,
Edward.

maintainer
#4 lore raw
Quoted 4 lines · Edward Srouji

I will modify the base test file to use verbs API instead of accessing the sysfs directly.
I wanted to do that using ibv_query_gid_ex to get the netdev ifindex, but in case of IB (IPoIB) the netdex is just 0 and not updated accordingly. Not sure if it's a bug or by design (looks like a bug for me).
I'll check that and update accordingly.

IB does not have IP based GIDs, so it is not a bug.

To find the ipoib interface you need to scan all the ipoib devices and match them to their IB devices, there can be many due to pkeys.

Jason

#5 lore raw
In reply to Edward Srouji
Quoted 51 lines · Edward Srouji
Quoted 51 lines · Jason Gunthorpe
Quoted 12 lines · Bob Pearson
Quoted 10 lines

Zhu,

There are about 10 test cases in the python suite that do not run for rxe because

... skipped "Device rxe0 doesn't have net interface"

Clearly this is wrong and I don't know how to address the root cause yet but the following hack where enp0s3 is the actual net device that rxe0 is based on in my case enables these test cases to run and it

appears they all do.

Quoted 10 lines
Quoted 10 lines
tests/base.py +0 -0

index 3460c546..d6fd29b8 100644

+++ b/tests/base.py

tests/base.py +0 -0
@@ -240,10 +240,11 @@ class RDMATestCase(unittest.TestCase):
 
Quoted 8 lines

tests/base.py class RDMATestCase(unittest.TestCase): · line 248

Quoted 8 lines
tests/base.py +0 -1
              if self.gid_type is not None and
 ctx.query_gid_type(port, idx) != \
 
                      self.gid_type:
 
                  continue
 
-            if not

os.path.exists('/sys/class/infiniband/{}/device/net/'.format(dev)):

Quoted 8 lines

tests/base.py class RDMATestCase(unittest.TestCase):

Quoted 8 lines
tests/base.py +1 -3
-                self.args.append([dev, port, idx, None, None])
 
-                continue
 
-            net_name = self.get_net_name(dev)
 
+            #if not

os.path.exists('/sys/class/infiniband/{}/device/net/'.format(dev)):

Quoted 3 lines

The pytests code is wrong - it should be querying the netdev through the verbs APIs, not hacking in sysfs like this.

Got it. Thanks

Zhu Yanjun

Quoted 1 line
Quoted 6 lines
Quoted 1 line
Quoted 1 line

Jason

I will modify the base test file to use verbs API instead of accessing the sysfs directly.
I wanted to do that using ibv_query_gid_ex to get the netdev ifindex, but in case of IB (IPoIB) the netdex is just 0 and not updated accordingly. Not sure if it's a bug or by design (looks like a bug for me).
I'll check that and update accordingly.

Hope all the test cases are ready.

Zhu Yanjun

Quoted 4 lines

Thanks,
Edward.