Skip to contentSkip to conversation

Extension of migration tests

PATCH Superseded Oksana Vohchana 3 participants 10 messages Feb 14, 2020 – Feb 24, 2020

v1 of 5 Superseded Latest is v5 →

lore
Share & tools
lore
#0 lore raw

cover Extension of migration tests

This series adds a new migration test through RDMA and provides new functions to it.
The last update by mistake was not provided a full scenario to the EXEC migration test. One of patch fixed it.

Oksana Vohchana (4):
Acceptance test: add address as param
Acceptance test: EXEC migration
Acceptance test: provides new functions
Acceptance test: provides to use RDMA transport for migration

 tests/acceptance/migration.py | 31 +++++++++++++++++++++++++++++--
 1 file changed, 29 insertions(+), 2 deletions(-)
Signature · 1 line
2.21.1
#1 lore raw

PATCH 1/4 Acceptance test: add address as param

Provides param address in _get_free_port()
because by default it takes free port only on the localhost

---
 tests/acceptance/migration.py | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
tests/acceptance/migration.py +2 -2
index a8367ca023..e4c39b85a1 100644
@@ -52,8 +52,8 @@ class Migration(Test):
         source_vm.qmp('migrate', uri=src_uri)
         self.assert_migration(source_vm, dest_vm)
 
-    def _get_free_port(self):
-        port = network.find_free_port()
+    def _get_free_port(self, address='localhost'):
+        port = network.find_free_port(address=address)
         if port is None:
             self.cancel('Failed to find a free port')
         return port
Signature · 1 line
2.21.1
Reviewed-by on v1 1/4
#2 lore raw

Re: [PATCH v1 1/4] Acceptance test: add address as param

Hi Oksana,

Quoted 13 lines · Oksana Vohchana

Provides param address in _get_free_port()
because by default it takes free port only on the localhost

---
  tests/acceptance/migration.py | 4 ++--
  1 file changed, 2 insertions(+), 2 deletions(-)
tests/acceptance/migration.py +0 -0
index a8367ca023..e4c39b85a1 100644

tests/acceptance/migration.py class Migration(Test): · line 56

Quoted 7 lines
tests/acceptance/migration.py +2 -2
@@ -52,8 +52,8 @@ class Migration(Test):
          source_vm.qmp('migrate', uri=src_uri)
          self.assert_migration(source_vm, dest_vm)
 
-    def _get_free_port(self):
-        port = network.find_free_port()
+    def _get_free_port(self, address='localhost'):
+        port = network.find_free_port(address=address)

Looks good to me, but:

- This patch is out of order, it should be followed by patches 03 and 04.

- You could have described in the cover letter (or in the commit) your intention with this change.

Thanks!

- Wainer

Quoted 3 lines
tests/acceptance/migration.py +0 -0
          if port is None:
              self.cancel('Failed to find a free port')
          return port
#3 lore raw

PATCH 2/4 Acceptance test: EXEC migration

In reply to Oksana Vohchana

Improves EXEC migration to run whole test stage

---
 tests/acceptance/migration.py | 2 ++
 1 file changed, 2 insertions(+)
tests/acceptance/migration.py +2 -0
index e4c39b85a1..8209dcf71d 100644
@@ -75,3 +75,5 @@ class Migration(Test):
         """
         free_port = self._get_free_port()
         dest_uri = 'exec:nc -l localhost %u' % free_port
+        src_uri = 'exec:nc localhost %u' % free_port
+        self.do_migrate(dest_uri, src_uri)
Signature · 1 line
2.21.1
Reviewed-by on v1 2/4
#4 lore raw

Re: [PATCH v1 2/4] Acceptance test: EXEC migration

Quoted 3 lines · Oksana Vohchana

Improves EXEC migration to run whole test stage

Fixes 2e768cb682bf
Quoted 10 lines
---
  tests/acceptance/migration.py | 2 ++
  1 file changed, 2 insertions(+)
tests/acceptance/migration.py +0 -0
index e4c39b85a1..8209dcf71d 100644

tests/acceptance/migration.py class Migration(Test): · line 79

Quoted 5 lines
tests/acceptance/migration.py +2 -0
@@ -75,3 +75,5 @@ class Migration(Test):
          """
          free_port = self._get_free_port()
          dest_uri = 'exec:nc -l localhost %u' % free_port
+        src_uri = 'exec:nc localhost %u' % free_port
+        self.do_migrate(dest_uri, src_uri)

I was very surprised because commit 2e768cb682bf is not the content I reviewed. Apparently what happened here is you sent 2 different series with the same subject... I got confused while applying I selected the incorrect id:

$ pwclient list 'Acceptance test: provides to use different transport for migration'
ID      State        Name
--      -----        ----
11348377 New          Acceptance test: provides to use different transport for migration
11348719 New          Acceptance test: provides to use different transport for migration
11355797 New          [v2] Acceptance test: provides to use different transport for migration
11358447 New          [v2,REPOST] Acceptance test: provides to use different transport for migration
11362397 New          [v3,1/2] Acceptance test: provides to use different transport for migration
11362399 New          [v3,2/2] Acceptance test: provides to use different transport for migration
11362479 New          [v3,1/2] Acceptance test: provides to use different transport for migration
11362485 New          [v3,2/2] Acceptance test: provides to use different transport for migration

I am sorry I missed that.

Tested-by on v1 2/4
#5 lore raw

Re: [PATCH v1 2/4] Acceptance test: EXEC migration

In reply to Oksana Vohchana
Quoted 6 lines · Oksana Vohchana

Improves EXEC migration to run whole test stage

---
  tests/acceptance/migration.py | 2 ++
  1 file changed, 2 insertions(+)

Indeed, with this changes the migration is triggered.

Quoted 11 lines
tests/acceptance/migration.py +2 -0
index e4c39b85a1..8209dcf71d 100644
@@ -75,3 +75,5 @@ class Migration(Test):
          """
          free_port = self._get_free_port()
          dest_uri = 'exec:nc -l localhost %u' % free_port
+        src_uri = 'exec:nc localhost %u' % free_port
+        self.do_migrate(dest_uri, src_uri)
#6 lore raw

PATCH 3/4 Acceptance test: provides new functions

In reply to Oksana Vohchana

Adds functions to check if service RDMA is enabled and gets the interface where it was configured

---
 tests/acceptance/migration.py | 17 +++++++++++++++++
 1 file changed, 17 insertions(+)
tests/acceptance/migration.py +17 -0
index 8209dcf71d..bbd88f8dda 100644
@@ -11,12 +11,16 @@
 
 
 import tempfile
+import re
+import netifaces
 from avocado_qemu import Test
 from avocado import skipUnless
 
 from avocado.utils import network
 from avocado.utils import wait
 from avocado.utils.path import find_command
+from avocado.utils import service
+from avocado.utils import process
 
 
 class Migration(Test):
@@ -58,6 +62,19 @@ class Migration(Test):
             self.cancel('Failed to find a free port')
         return port
 
+    def _if_rdma_enable(self):
+        rdma_stat = service.ServiceManager()
+        rdma = rdma_stat.status('rdma')
+        return rdma
+
+    def _get_ip_rdma(self):
+        get_ip_rdma = process.run('rdma link show').stdout.decode()
+        for line in get_ip_rdma.split('\n'):
+            if re.search(r"ACTIVE", line):
+                interface = line.split(" ")[-2]
+                ip = netifaces.ifaddresses(interface)[netifaces.AF_INET][0]['addr']
+                return ip
+
 
     def test_migration_with_tcp_localhost(self):
         dest_uri = 'tcp:localhost:%u' % self._get_free_port()
Signature · 1 line
2.21.1
#7 lore raw

Re: [PATCH v1 3/4] Acceptance test: provides new functions

Hi Oksana,

Quoted 13 lines · Oksana Vohchana

Adds functions to check if service RDMA is enabled and gets the interface where it was configured

---
  tests/acceptance/migration.py | 17 +++++++++++++++++
  1 file changed, 17 insertions(+)
tests/acceptance/migration.py +0 -0
index 8209dcf71d..bbd88f8dda 100644

tests/acceptance/migration.py line 15

Quoted 5 lines
tests/acceptance/migration.py +2 -0
@@ -11,12 +11,16 @@
 
 
  import tempfile
+import re
+import netifaces

Since netifaces isn't a standard Python library that import might fail.

The tests dependencies are listed in tests/requirements.txt, and installed in the environment created by make check-acceptance. If you want to ensure the test behaves well even when executed manually (i.e. not via make check-acceptance), you can add runtime checks as can be seen in tests/acceptance/machine_m68k_nextcube.py

Quoted 12 lines
tests/acceptance/migration.py +2 -0
  from avocado_qemu import Test
  from avocado import skipUnless
 
  from avocado.utils import network
  from avocado.utils import wait
  from avocado.utils.path import find_command
+from avocado.utils import service
+from avocado.utils import process
 
 
  class Migration(Test):

tests/acceptance/migration.py class Migration(Test): · line 68

Quoted 7 lines
tests/acceptance/migration.py +4 -0
@@ -58,6 +62,19 @@ class Migration(Test):
              self.cancel('Failed to find a free port')
          return port
 
+    def _if_rdma_enable(self):
+        rdma_stat = service.ServiceManager()
+        rdma = rdma_stat.status('rdma')
+        return rdma

Above function is used on patch04, but actually I don't think it needs to check this service for RoCE. It would be needed if it was using the rxe_cfg to configure the rdma link. Or am I missing something?

tests/acceptance/migration.py class Migration(Test): · line 77

Quoted 9 lines
tests/acceptance/migration.py +9 -0
+
+    def _get_ip_rdma(self):
+        get_ip_rdma = process.run('rdma link show').stdout.decode()
+        for line in get_ip_rdma.split('\n'):
+            if re.search(r"ACTIVE", line):
+                interface = line.split(" ")[-2]
+                ip = netifaces.ifaddresses(interface)[netifaces.AF_INET][0]['addr']
+                return ip
+

I suggest that it explicitly returns None if none is found.

Thanks!

- Wainer

Quoted 3 lines
tests/acceptance/migration.py +0 -0
      def test_migration_with_tcp_localhost(self):
          dest_uri = 'tcp:localhost:%u' % self._get_free_port()
#8 lore raw

Re: [PATCH v1 3/4] Acceptance test: provides new functions

Hi Wainer,
Thanks for review

Quoted 25 lines · Wainer dos Santos Moschetta

Hi Oksana,

Review

Quoted 9 lines · Oksana Vohchana

Adds functions to check if service RDMA is enabled and gets the interface where it was configured

---
  tests/acceptance/migration.py | 17 +++++++++++++++++
  1 file changed, 17 insertions(+)
(quoted diff) +0 -0

b/tests/acceptance/migration.py

tests/acceptance/migration.py line 15

Quoted 9 lines

index 8209dcf71d..bbd88f8dda 100644

tests/acceptance/migration.py +2 -0
@@ -11,12 +11,16 @@
 
 
  import tempfile
+import re
+import netifaces

Since netifaces isn't a standard Python library that import might fail.

Quoted 6 lines

The tests dependencies are listed in tests/requirements.txt, and installed in the environment created by make check-acceptance. If you want to ensure the test behaves well even when executed manually (i.e. not via make check-acceptance), you can add runtime checks as can be seen in tests/acceptance/machine_m68k_nextcube.py

Thanks, a "runtime checks" is a good approach
I'll improve it

Quoted 26 lines
Quoted 13 lines
tests/acceptance/migration.py +2 -0
  from avocado_qemu import Test
  from avocado import skipUnless
 
  from avocado.utils import network
  from avocado.utils import wait
  from avocado.utils.path import find_command
+from avocado.utils import service
+from avocado.utils import process
 
 
  class Migration(Test):

tests/acceptance/migration.py class Migration(Test): · line 68

Quoted 7 lines
tests/acceptance/migration.py +4 -0
@@ -58,6 +62,19 @@ class Migration(Test):
              self.cancel('Failed to find a free port')
          return port
 
+    def _if_rdma_enable(self):
+        rdma_stat = service.ServiceManager()
+        rdma = rdma_stat.status('rdma')
+        return rdma

Above function is used on patch04, but actually I don't think it needs to check this service for RoCE. It would be needed if it was using the rxe_cfg to configure the rdma link. Or am I missing something?

Quoted 1 line

The function _if_rdma_enable() checks if RDMA service enabled in the

system it does not depend on wich utils we will use to check if some network configuration present (rdma or rxe_cfg)

Quoted 7 lines

tests/acceptance/migration.py class Migration(Test):

Quoted 7 lines
tests/acceptance/migration.py +7 -0
+
+    def _get_ip_rdma(self):
+        get_ip_rdma = process.run('rdma link show').stdout.decode()
+        for line in get_ip_rdma.split('\n'):
+            if re.search(r"ACTIVE", line):
+                interface = line.split(" ")[-2]
+                ip =
Quoted 6 lines

netifaces.ifaddresses(interface)[netifaces.AF_INET][0]['addr']

tests/acceptance/migration.py class Migration(Test): · line 77

Quoted 2 lines
tests/acceptance/migration.py +2 -0
+                return ip
+

I suggest that it explicitly returns None if none is found.

Ok, I agree
I'll improve it

Quoted 9 lines

Thanks!

- Wainer

Quoted 3 lines
tests/acceptance/migration.py +0 -0
      def test_migration_with_tcp_localhost(self):
          dest_uri = 'tcp:localhost:%u' % self._get_free_port()

Thanks!

#9 lore raw

PATCH 4/4 Acceptance test: provides to use RDMA transport for migration

In reply to Oksana Vohchana

Adds test for RDMA migration check

---
 tests/acceptance/migration.py | 8 ++++++++
 1 file changed, 8 insertions(+)
tests/acceptance/migration.py +8 -0
index bbd88f8dda..c0a3031e67 100644
@@ -94,3 +94,11 @@ class Migration(Test):
         dest_uri = 'exec:nc -l localhost %u' % free_port
         src_uri = 'exec:nc localhost %u' % free_port
         self.do_migrate(dest_uri, src_uri)
+
+    @skipUnless(_if_rdma_enable(None), "Unit rdma.service could not be found")
+    @skipUnless(_get_ip_rdma(None), 'RoCE(RDMA) service or interface not configured')
+    def test_migration_with_rdma_localhost(self):
+        ip = self._get_ip_rdma()
+        free_port = self._get_free_port(address=ip)
+        dest_uri = 'rdma:%s:%u' % (ip, free_port)
+        self.do_migrate(dest_uri)
Signature · 1 line
2.21.1