Skip to contentSkip to conversation

Improve code coverage for ppc64

PATCH Superseded Kautuk Consul 4 participants 12 messages Apr 21, 2023 – Apr 24, 2023

v4 of 5 Superseded Latest is v5 →

lore
Share & tools
lore
#0 lore raw

v4 cover Improve code coverage for ppc64

Commit c0c8687ef0fd990db8db1655a8a6c5a5e35dd4bb disabled the boot_linux.py test-case due to which the code coverage for ppc decreased by around 2%. As per the discussion on
https://lore.kernel.org/qemu-devel/[email protected]/ it was mentioned that the baseline test for ppc64 could be modified to make up this 2% code coverage. This patchset attempts to achieve this 2% code coverage by adding various device command line arguments (to ./qemu-system-ppc64) in the tuxrun_baselines.py test-case.

Changes since v3:
- Create a common ppc64_common_tuxrun routine in tuxrun_baselines.py and call that from the ppc64 and ppc64le test case routines.

Kautuk Consul (2):
avocado_qemu/__init__.py: factor out the qemu-img finding tests/avocado/tuxrun_baselines.py: improve code coverage for ppc64

 tests/avocado/avocado_qemu/__init__.py | 27 +++++-----
 tests/avocado/tuxrun_baselines.py      | 68 ++++++++++++++++++++++++--
 2 files changed, 80 insertions(+), 15 deletions(-)
Signature · 1 line
2.25.1
#1 lore raw

PATCH v4 1/2 avocado_qemu/__init__.py: factor out the qemu-img finding

Factor out the code that finds the qemu-img binary in the QemuSystemTest class and create a new get_qemu_img() function with it. This function will get called also from the new code in tuxrun_baselines.py avocado test-case.

---
 tests/avocado/avocado_qemu/__init__.py | 27 +++++++++++++++-----------
 1 file changed, 16 insertions(+), 11 deletions(-)
tests/avocado/avocado_qemu/__init__.py +16 -11
index cb71f50db9..d8f390e27a 100644
@@ -330,6 +330,19 @@ def _new_vm(self, name, *args):
             vm.add_args(*args)
         return vm
 
+    def get_qemu_img(self):
+        self.log.debug('Looking for and selecting a qemu-img binary')
+
+        # If qemu-img has been built, use it, otherwise the system wide one
+        # will be used.
+        qemu_img = os.path.join(BUILD_DIR, 'qemu-img')
+        if not os.path.exists(qemu_img):
+            qemu_img = find_command('qemu-img', False)
+        if qemu_img is False:
+            self.cancel('Could not find "qemu-img"')
+
+        return qemu_img
+
     @property
     def vm(self):
         return self.get_vm(name='default')
@@ -594,17 +607,9 @@ def set_up_existing_ssh_keys(self):
         return (ssh_public_key, ssh_private_key)
 
     def download_boot(self):
-        self.log.debug('Looking for and selecting a qemu-img binary to be '
-                       'used to create the bootable snapshot image')
-        # If qemu-img has been built, use it, otherwise the system wide one
-        # will be used.  If none is available, the test will cancel.
-        qemu_img = os.path.join(BUILD_DIR, 'qemu-img')
-        if not os.path.exists(qemu_img):
-            qemu_img = find_command('qemu-img', False)
-        if qemu_img is False:
-            self.cancel('Could not find "qemu-img", which is required to '
-                        'create the bootable image')
-        vmimage.QEMU_IMG = qemu_img
+        # Set the qemu-img binary.
+        # If none is available, the test will cancel.
+        vmimage.QEMU_IMG = super().get_qemu_img()
 
         self.log.info('Downloading/preparing boot image')
         # Fedora 31 only provides ppc64le images
Signature · 1 line
2.25.1
Reviewed-by on v4 1/2
#2 lore raw

Re: [PATCH v4 1/2] avocado_qemu/__init__.py: factor out the qemu-img finding

Quoted 9 lines · Kautuk Consul

Factor out the code that finds the qemu-img binary in the QemuSystemTest class and create a new get_qemu_img() function with it. This function will get called also from the new code in tuxrun_baselines.py avocado test-case.

---
  tests/avocado/avocado_qemu/__init__.py | 27 +++++++++++++++-----------
  1 file changed, 16 insertions(+), 11 deletions(-)
#3 lore raw

Re: [PATCH v4 1/2] avocado_qemu/__init__.py: factor out the qemu-img finding

In reply to Kautuk Consul

Adding Harsh Prateel Bora.

Quoted 59 lines · Kautuk Consul

Factor out the code that finds the qemu-img binary in the QemuSystemTest class and create a new get_qemu_img() function with it. This function will get called also from the new code in tuxrun_baselines.py avocado test-case.

---
 tests/avocado/avocado_qemu/__init__.py | 27 +++++++++++++++-----------
 1 file changed, 16 insertions(+), 11 deletions(-)
tests/avocado/avocado_qemu/__init__.py +16 -11
index cb71f50db9..d8f390e27a 100644
@@ -330,6 +330,19 @@ def _new_vm(self, name, *args):
             vm.add_args(*args)
         return vm
 
+    def get_qemu_img(self):
+        self.log.debug('Looking for and selecting a qemu-img binary')
+
+        # If qemu-img has been built, use it, otherwise the system wide one
+        # will be used.
+        qemu_img = os.path.join(BUILD_DIR, 'qemu-img')
+        if not os.path.exists(qemu_img):
+            qemu_img = find_command('qemu-img', False)
+        if qemu_img is False:
+            self.cancel('Could not find "qemu-img"')
+
+        return qemu_img
+
     @property
     def vm(self):
         return self.get_vm(name='default')
@@ -594,17 +607,9 @@ def set_up_existing_ssh_keys(self):
         return (ssh_public_key, ssh_private_key)
 
     def download_boot(self):
-        self.log.debug('Looking for and selecting a qemu-img binary to be '
-                       'used to create the bootable snapshot image')
-        # If qemu-img has been built, use it, otherwise the system wide one
-        # will be used.  If none is available, the test will cancel.
-        qemu_img = os.path.join(BUILD_DIR, 'qemu-img')
-        if not os.path.exists(qemu_img):
-            qemu_img = find_command('qemu-img', False)
-        if qemu_img is False:
-            self.cancel('Could not find "qemu-img", which is required to '
-                        'create the bootable image')
-        vmimage.QEMU_IMG = qemu_img
+        # Set the qemu-img binary.
+        # If none is available, the test will cancel.
+        vmimage.QEMU_IMG = super().get_qemu_img()
 
         self.log.info('Downloading/preparing boot image')
         # Fedora 31 only provides ppc64le images
Signature · 1 line
2.25.1
Reviewed-by
#4 lore raw

Re: [PATCH v4 1/2] avocado_qemu/__init__.py: factor out the qemu-img finding

Quoted 40 lines · Kautuk Consul

Adding Harsh Prateek Bora.

Quoted 37 lines · Kautuk Consul

Factor out the code that finds the qemu-img binary in the QemuSystemTest class and create a new get_qemu_img() function with it. This function will get called also from the new code in tuxrun_baselines.py avocado test-case.

---
 tests/avocado/avocado_qemu/__init__.py | 27 +++++++++++++++-----------
 1 file changed, 16 insertions(+), 11 deletions(-)
tests/avocado/avocado_qemu/__init__.py +13 -0
index cb71f50db9..d8f390e27a 100644
@@ -330,6 +330,19 @@ def _new_vm(self, name, *args):
             vm.add_args(*args)
         return vm
 
+    def get_qemu_img(self):
+        self.log.debug('Looking for and selecting a qemu-img binary')
+
+        # If qemu-img has been built, use it, otherwise the system wide one
+        # will be used.
+        qemu_img = os.path.join(BUILD_DIR, 'qemu-img')
+        if not os.path.exists(qemu_img):
+            qemu_img = find_command('qemu-img', False)
+        if qemu_img is False:
+            self.cancel('Could not find "qemu-img"')
+
+        return qemu_img
+
     @property
     def vm(self):
         return self.get_vm(name='default')
@@ -594,17 +607,9 @@ def set_up_existing_ssh_keys(self):
         return (ssh_public_key, ssh_private_key)
 
Quoted 16 lines
Quoted 6 lines
tests/avocado/avocado_qemu/__init__.py +0 -5
     def download_boot(self):
-        self.log.debug('Looking for and selecting a qemu-img binary to be '
-                       'used to create the bootable snapshot image')
-        # If qemu-img has been built, use it, otherwise the system wide one
-        # will be used.  If none is available, the test will cancel.
-        qemu_img = os.path.join(BUILD_DIR, 'qemu-img')

tests/avocado/avocado_qemu/__init__.py def set_up_existing_ssh_keys(self): · line 613

Quoted 10 lines
tests/avocado/avocado_qemu/__init__.py +3 -6
-        if not os.path.exists(qemu_img):
-            qemu_img = find_command('qemu-img', False)
-        if qemu_img is False:
-            self.cancel('Could not find "qemu-img", which is required to '
-                        'create the bootable image')
-        vmimage.QEMU_IMG = qemu_img
+        # Set the qemu-img binary.
+        # If none is available, the test will cancel.
+        vmimage.QEMU_IMG = super().get_qemu_img()
 
Quoted 7 lines
Quoted 6 lines
tests/avocado/avocado_qemu/__init__.py +0 -0
         self.log.info('Downloading/preparing boot image')
         # Fedora 31 only provides ppc64le images
Signature · 1 line
2.25.1
#5 lore raw

PATCH v4 2/2 tests/avocado/tuxrun_baselines.py: improve code coverage for ppc64

In reply to Kautuk Consul

Commit c0c8687ef0fd990db8db1655a8a6c5a5e35dd4bb disabled the boot_linux.py test-case due to which the code coverage for ppc decreased by around 2%. As per the discussion on
https://lore.kernel.org/qemu-devel/[email protected]/ it was mentioned that the baseline test for ppc64 could be modified to make up this 2% code coverage. This patch attempts to achieve this 2% code coverage by adding various device command line arguments (to ./qemu-system-ppc64) in the tuxrun_baselines.py test-case.

The code coverage report with boot_linux.py, without it and finally with these tuxrun_baselines.py changes is as follows:

With boot_linux.py
------------------
lines......: 13.8% (58006 of 420997 lines) functions..: 20.7% (7675 of 36993 functions) branches...: 9.2% (22146 of 240611 branches)
Without boot_linux.py (without this patch changes)
--------------------------------------------------
lines......: 11.9% (50174 of 420997 lines) functions..: 18.8% (6947 of 36993 functions) branches...: 7.4% (17580 of 239017 branches)
Without boot_linux.py (with this patch changes)
-----------------------------------------------
lines......: 13.8% (58287 of 420997 lines) functions..: 20.7% (7640 of 36993 functions) branches...: 8.4% (20223 of 240611 branches)

---
 tests/avocado/tuxrun_baselines.py | 68 +++++++++++++++++++++++++++++--
 1 file changed, 64 insertions(+), 4 deletions(-)
tests/avocado/tuxrun_baselines.py +64 -4
index d343376faa..546abd9396 100644
@@ -11,6 +11,7 @@
 
 import os
 import time
+import tempfile
 
 from avocado import skip, skipIf
 from avocado_qemu import QemuSystemTest
@@ -72,6 +73,8 @@ def setUp(self):
         # Occasionally we need extra devices to hook things up
         self.extradev = self.get_tag('extradev')
 
+        self.qemu_img = super().get_qemu_img()
+
     def wait_for_console_pattern(self, success_message, vm=None):
         wait_for_console_pattern(self, success_message,
                                  failure_message='Kernel panic - not syncing',
@@ -164,6 +167,63 @@ def common_tuxrun(self, dt=None,
         self.vm.launch()
         self.run_tuxtest_tests(haltmsg)
 
+    def ppc64_common_tuxrun(self, prefix):
+        # add device args to command line.
+        self.vm.add_args('-netdev', 'user,id=vnet,hostfwd=:127.0.0.1:0-:22',
+                         '-device', 'virtio-net,netdev=vnet')
+        self.vm.add_args('-netdev', '{"type":"user","id":"hostnet0"}',
+                         '-device', '{"driver":"virtio-net-pci","netdev":'
+                         '"hostnet0","id":"net0","mac":"52:54:00:4c:e3:86",'
+                         '"bus":"pci.0","addr":"0x9"}')
+        self.vm.add_args('-device', '{"driver":"qemu-xhci","p2":15,"p3":15,'
+                         '"id":"usb","bus":"pci.0","addr":"0x2"}')
+        self.vm.add_args('-device', '{"driver":"virtio-scsi-pci","id":"scsi0"'
+                         ',"bus":"pci.0","addr":"0x3"}')
+        self.vm.add_args('-device', '{"driver":"virtio-serial-pci","id":'
+                         '"virtio-serial0","bus":"pci.0","addr":"0x4"}')
+        self.vm.add_args('-device', '{"driver":"scsi-cd","bus":"scsi0.0"'
+                         ',"channel":0,"scsi-id":0,"lun":0,"device_id":'
+                         '"drive-scsi0-0-0-0","id":"scsi0-0-0-0"}')
+        self.vm.add_args('-device', '{"driver":"virtio-balloon-pci",'
+                         '"id":"balloon0","bus":"pci.0","addr":"0x6"}')
+        self.vm.add_args('-audiodev', '{"id":"audio1","driver":"none"}')
+        self.vm.add_args('-device', '{"driver":"usb-tablet","id":"input0"'
+                         ',"bus":"usb.0","port":"1"}')
+        self.vm.add_args('-device', '{"driver":"usb-kbd","id":"input1"'
+                         ',"bus":"usb.0","port":"2"}')
+        self.vm.add_args('-device', '{"driver":"VGA","id":"video0",'
+                         '"vgamem_mb":16,"bus":"pci.0","addr":"0x7"}')
+        self.vm.add_args('-object', '{"qom-type":"rng-random","id":"objrng0"'
+                         ',"filename":"/dev/urandom"}',
+                         '-device', '{"driver":"virtio-rng-pci","rng":"objrng0"'
+                         ',"id":"rng0","bus":"pci.0","addr":"0x8"}')
+        self.vm.add_args('-object', '{"qom-type":"cryptodev-backend-builtin",'
+                         '"id":"objcrypto0","queues":1}',
+                         '-device', '{"driver":"virtio-crypto-pci",'
+                         '"cryptodev":"objcrypto0","id":"crypto0","bus"'
+                         ':"pci.0","addr":"0xa"}')
+        self.vm.add_args('-device', '{"driver":"spapr-pci-host-bridge"'
+                         ',"index":1,"id":"pci.1"}')
+        self.vm.add_args('-device', '{"driver":"spapr-vscsi","id":"scsi1"'
+                         ',"reg":12288}')
+        self.vm.add_args('-m', '2G,slots=32,maxmem=4G',
+                         '-object', 'memory-backend-ram,id=ram1,size=1G',
+                         '-device', 'pc-dimm,id=dimm1,memdev=ram1')
+
+        # Create a temporary qcow2 and launch the test-case
+        with tempfile.NamedTemporaryFile(prefix=prefix,
+                                         suffix='.qcow2') as qcow2:
+            process.run(self.qemu_img + ' create -f qcow2 ' +
+                        qcow2.name + ' 1G')
+
+            self.vm.add_args('-drive', 'file=' + qcow2.name +
+                         ',format=qcow2,if=none,id='
+                         'drive-virtio-disk1',
+                         '-device', 'virtio-blk-pci,scsi=off,bus=pci.0,'
+                         'addr=0xb,drive=drive-virtio-disk1,id=virtio-disk1'
+                         ',bootindex=2')
+            self.common_tuxrun(drive="scsi-hd")
+
     #
     # The tests themselves. The configuration is derived from how
     # tuxrun invokes qemu (with minor tweaks like using -blockdev
@@ -308,7 +368,7 @@ def test_ppc64(self):
         """
         :avocado: tags=arch:ppc64
         :avocado: tags=machine:pseries
-        :avocado: tags=cpu:POWER8
+        :avocado: tags=cpu:POWER10
         :avocado: tags=endian:big
         :avocado: tags=console:hvc0
         :avocado: tags=tuxboot:ppc64
@@ -316,20 +376,20 @@ def test_ppc64(self):
         :avocado: tags=extradev:driver=spapr-vscsi
         :avocado: tags=root:sda
         """
-        self.common_tuxrun(drive="scsi-hd")
+        self.ppc64_common_tuxrun(prefix='tuxrun_ppc64_')
 
     def test_ppc64le(self):
         """
         :avocado: tags=arch:ppc64
         :avocado: tags=machine:pseries
-        :avocado: tags=cpu:POWER8
+        :avocado: tags=cpu:POWER10
         :avocado: tags=console:hvc0
         :avocado: tags=tuxboot:ppc64le
         :avocado: tags=image:vmlinux
         :avocado: tags=extradev:driver=spapr-vscsi
         :avocado: tags=root:sda
         """
-        self.common_tuxrun(drive="scsi-hd")
+        self.ppc64_common_tuxrun(prefix='tuxrun_ppc64le_')
 
     def test_riscv32(self):
         """
Signature · 1 line
2.25.1
Reviewed-by on v4 2/2
#6 lore raw

Re: [PATCH v4 2/2] tests/avocado/tuxrun_baselines.py: improve code coverage for ppc64

Quoted 29 lines · Kautuk Consul

Commit c0c8687ef0fd990db8db1655a8a6c5a5e35dd4bb disabled the boot_linux.py test-case due to which the code coverage for ppc decreased by around 2%. As per the discussion on
https://lore.kernel.org/qemu-devel/[email protected]/ it was mentioned that the baseline test for ppc64 could be modified to make up this 2% code coverage. This patch attempts to achieve this 2% code coverage by adding various device command line arguments (to ./qemu-system-ppc64) in the tuxrun_baselines.py test-case.

The code coverage report with boot_linux.py, without it and finally with these tuxrun_baselines.py changes is as follows:

With boot_linux.py
------------------
lines......: 13.8% (58006 of 420997 lines) functions..: 20.7% (7675 of 36993 functions) branches...: 9.2% (22146 of 240611 branches)
Without boot_linux.py (without this patch changes)
--------------------------------------------------
lines......: 11.9% (50174 of 420997 lines) functions..: 18.8% (6947 of 36993 functions) branches...: 7.4% (17580 of 239017 branches)
Without boot_linux.py (with this patch changes)
-----------------------------------------------
lines......: 13.8% (58287 of 420997 lines) functions..: 20.7% (7640 of 36993 functions) branches...: 8.4% (20223 of 240611 branches)

Quoted 5 lines
---
  tests/avocado/tuxrun_baselines.py | 68 +++++++++++++++++++++++++++++--
  1 file changed, 64 insertions(+), 4 deletions(-)
#7 lore raw

Re: [PATCH v4 2/2] tests/avocado/tuxrun_baselines.py: improve code coverage for ppc64

In reply to Kautuk Consul

Adding Harsh Prateek Bora <[email protected]>.

Quoted 157 lines · Kautuk Consul

Commit c0c8687ef0fd990db8db1655a8a6c5a5e35dd4bb disabled the boot_linux.py test-case due to which the code coverage for ppc decreased by around 2%. As per the discussion on
https://lore.kernel.org/qemu-devel/[email protected]/ it was mentioned that the baseline test for ppc64 could be modified to make up this 2% code coverage. This patch attempts to achieve this 2% code coverage by adding various device command line arguments (to ./qemu-system-ppc64) in the tuxrun_baselines.py test-case.

The code coverage report with boot_linux.py, without it and finally with these tuxrun_baselines.py changes is as follows:

With boot_linux.py
------------------
lines......: 13.8% (58006 of 420997 lines) functions..: 20.7% (7675 of 36993 functions) branches...: 9.2% (22146 of 240611 branches)
Without boot_linux.py (without this patch changes)
--------------------------------------------------
lines......: 11.9% (50174 of 420997 lines) functions..: 18.8% (6947 of 36993 functions) branches...: 7.4% (17580 of 239017 branches)
Without boot_linux.py (with this patch changes)
-----------------------------------------------
lines......: 13.8% (58287 of 420997 lines) functions..: 20.7% (7640 of 36993 functions) branches...: 8.4% (20223 of 240611 branches)

---
 tests/avocado/tuxrun_baselines.py | 68 +++++++++++++++++++++++++++++--
 1 file changed, 64 insertions(+), 4 deletions(-)
tests/avocado/tuxrun_baselines.py +64 -4
index d343376faa..546abd9396 100644
@@ -11,6 +11,7 @@
 
 import os
 import time
+import tempfile
 
 from avocado import skip, skipIf
 from avocado_qemu import QemuSystemTest
@@ -72,6 +73,8 @@ def setUp(self):
         # Occasionally we need extra devices to hook things up
         self.extradev = self.get_tag('extradev')
 
+        self.qemu_img = super().get_qemu_img()
+
     def wait_for_console_pattern(self, success_message, vm=None):
         wait_for_console_pattern(self, success_message,
                                  failure_message='Kernel panic - not syncing',
@@ -164,6 +167,63 @@ def common_tuxrun(self, dt=None,
         self.vm.launch()
         self.run_tuxtest_tests(haltmsg)
 
+    def ppc64_common_tuxrun(self, prefix):
+        # add device args to command line.
+        self.vm.add_args('-netdev', 'user,id=vnet,hostfwd=:127.0.0.1:0-:22',
+                         '-device', 'virtio-net,netdev=vnet')
+        self.vm.add_args('-netdev', '{"type":"user","id":"hostnet0"}',
+                         '-device', '{"driver":"virtio-net-pci","netdev":'
+                         '"hostnet0","id":"net0","mac":"52:54:00:4c:e3:86",'
+                         '"bus":"pci.0","addr":"0x9"}')
+        self.vm.add_args('-device', '{"driver":"qemu-xhci","p2":15,"p3":15,'
+                         '"id":"usb","bus":"pci.0","addr":"0x2"}')
+        self.vm.add_args('-device', '{"driver":"virtio-scsi-pci","id":"scsi0"'
+                         ',"bus":"pci.0","addr":"0x3"}')
+        self.vm.add_args('-device', '{"driver":"virtio-serial-pci","id":'
+                         '"virtio-serial0","bus":"pci.0","addr":"0x4"}')
+        self.vm.add_args('-device', '{"driver":"scsi-cd","bus":"scsi0.0"'
+                         ',"channel":0,"scsi-id":0,"lun":0,"device_id":'
+                         '"drive-scsi0-0-0-0","id":"scsi0-0-0-0"}')
+        self.vm.add_args('-device', '{"driver":"virtio-balloon-pci",'
+                         '"id":"balloon0","bus":"pci.0","addr":"0x6"}')
+        self.vm.add_args('-audiodev', '{"id":"audio1","driver":"none"}')
+        self.vm.add_args('-device', '{"driver":"usb-tablet","id":"input0"'
+                         ',"bus":"usb.0","port":"1"}')
+        self.vm.add_args('-device', '{"driver":"usb-kbd","id":"input1"'
+                         ',"bus":"usb.0","port":"2"}')
+        self.vm.add_args('-device', '{"driver":"VGA","id":"video0",'
+                         '"vgamem_mb":16,"bus":"pci.0","addr":"0x7"}')
+        self.vm.add_args('-object', '{"qom-type":"rng-random","id":"objrng0"'
+                         ',"filename":"/dev/urandom"}',
+                         '-device', '{"driver":"virtio-rng-pci","rng":"objrng0"'
+                         ',"id":"rng0","bus":"pci.0","addr":"0x8"}')
+        self.vm.add_args('-object', '{"qom-type":"cryptodev-backend-builtin",'
+                         '"id":"objcrypto0","queues":1}',
+                         '-device', '{"driver":"virtio-crypto-pci",'
+                         '"cryptodev":"objcrypto0","id":"crypto0","bus"'
+                         ':"pci.0","addr":"0xa"}')
+        self.vm.add_args('-device', '{"driver":"spapr-pci-host-bridge"'
+                         ',"index":1,"id":"pci.1"}')
+        self.vm.add_args('-device', '{"driver":"spapr-vscsi","id":"scsi1"'
+                         ',"reg":12288}')
+        self.vm.add_args('-m', '2G,slots=32,maxmem=4G',
+                         '-object', 'memory-backend-ram,id=ram1,size=1G',
+                         '-device', 'pc-dimm,id=dimm1,memdev=ram1')
+
+        # Create a temporary qcow2 and launch the test-case
+        with tempfile.NamedTemporaryFile(prefix=prefix,
+                                         suffix='.qcow2') as qcow2:
+            process.run(self.qemu_img + ' create -f qcow2 ' +
+                        qcow2.name + ' 1G')
+
+            self.vm.add_args('-drive', 'file=' + qcow2.name +
+                         ',format=qcow2,if=none,id='
+                         'drive-virtio-disk1',
+                         '-device', 'virtio-blk-pci,scsi=off,bus=pci.0,'
+                         'addr=0xb,drive=drive-virtio-disk1,id=virtio-disk1'
+                         ',bootindex=2')
+            self.common_tuxrun(drive="scsi-hd")
+
     #
     # The tests themselves. The configuration is derived from how
     # tuxrun invokes qemu (with minor tweaks like using -blockdev
@@ -308,7 +368,7 @@ def test_ppc64(self):
         """
         :avocado: tags=arch:ppc64
         :avocado: tags=machine:pseries
-        :avocado: tags=cpu:POWER8
+        :avocado: tags=cpu:POWER10
         :avocado: tags=endian:big
         :avocado: tags=console:hvc0
         :avocado: tags=tuxboot:ppc64
@@ -316,20 +376,20 @@ def test_ppc64(self):
         :avocado: tags=extradev:driver=spapr-vscsi
         :avocado: tags=root:sda
         """
-        self.common_tuxrun(drive="scsi-hd")
+        self.ppc64_common_tuxrun(prefix='tuxrun_ppc64_')
 
     def test_ppc64le(self):
         """
         :avocado: tags=arch:ppc64
         :avocado: tags=machine:pseries
-        :avocado: tags=cpu:POWER8
+        :avocado: tags=cpu:POWER10
         :avocado: tags=console:hvc0
         :avocado: tags=tuxboot:ppc64le
         :avocado: tags=image:vmlinux
         :avocado: tags=extradev:driver=spapr-vscsi
         :avocado: tags=root:sda
         """
-        self.common_tuxrun(drive="scsi-hd")
+        self.ppc64_common_tuxrun(prefix='tuxrun_ppc64le_')
 
     def test_riscv32(self):
         """
Signature · 1 line
2.25.1
Reviewed-by
#8 lore raw

Re: [PATCH v4 2/2] tests/avocado/tuxrun_baselines.py: improve code coverage for ppc64

Quoted 139 lines · Kautuk Consul

Adding Harsh Prateek Bora <[email protected]>.

Quoted 136 lines · Kautuk Consul

Commit c0c8687ef0fd990db8db1655a8a6c5a5e35dd4bb disabled the boot_linux.py test-case due to which the code coverage for ppc decreased by around 2%. As per the discussion on
https://lore.kernel.org/qemu-devel/[email protected]/ it was mentioned that the baseline test for ppc64 could be modified to make up this 2% code coverage. This patch attempts to achieve this 2% code coverage by adding various device command line arguments (to ./qemu-system-ppc64) in the tuxrun_baselines.py test-case.

The code coverage report with boot_linux.py, without it and finally with these tuxrun_baselines.py changes is as follows:

With boot_linux.py
------------------
lines......: 13.8% (58006 of 420997 lines) functions..: 20.7% (7675 of 36993 functions) branches...: 9.2% (22146 of 240611 branches)
Without boot_linux.py (without this patch changes)
--------------------------------------------------
lines......: 11.9% (50174 of 420997 lines) functions..: 18.8% (6947 of 36993 functions) branches...: 7.4% (17580 of 239017 branches)
Without boot_linux.py (with this patch changes)
-----------------------------------------------
lines......: 13.8% (58287 of 420997 lines) functions..: 20.7% (7640 of 36993 functions) branches...: 8.4% (20223 of 240611 branches)

---
  tests/avocado/tuxrun_baselines.py | 68 +++++++++++++++++++++++++++++--
  1 file changed, 64 insertions(+), 4 deletions(-)
tests/avocado/tuxrun_baselines.py +62 -2
index d343376faa..546abd9396 100644
@@ -11,6 +11,7 @@
 
  import os
  import time
+import tempfile
 
  from avocado import skip, skipIf
  from avocado_qemu import QemuSystemTest
@@ -72,6 +73,8 @@ def setUp(self):
          # Occasionally we need extra devices to hook things up
          self.extradev = self.get_tag('extradev')
 
+        self.qemu_img = super().get_qemu_img()
+
      def wait_for_console_pattern(self, success_message, vm=None):
          wait_for_console_pattern(self, success_message,
                                   failure_message='Kernel panic - not syncing',
@@ -164,6 +167,63 @@ def common_tuxrun(self, dt=None,
          self.vm.launch()
          self.run_tuxtest_tests(haltmsg)
 
+    def ppc64_common_tuxrun(self, prefix):
+        # add device args to command line.
+        self.vm.add_args('-netdev', 'user,id=vnet,hostfwd=:127.0.0.1:0-:22',
+                         '-device', 'virtio-net,netdev=vnet')
+        self.vm.add_args('-netdev', '{"type":"user","id":"hostnet0"}',
+                         '-device', '{"driver":"virtio-net-pci","netdev":'
+                         '"hostnet0","id":"net0","mac":"52:54:00:4c:e3:86",'
+                         '"bus":"pci.0","addr":"0x9"}')
+        self.vm.add_args('-device', '{"driver":"qemu-xhci","p2":15,"p3":15,'
+                         '"id":"usb","bus":"pci.0","addr":"0x2"}')
+        self.vm.add_args('-device', '{"driver":"virtio-scsi-pci","id":"scsi0"'
+                         ',"bus":"pci.0","addr":"0x3"}')
+        self.vm.add_args('-device', '{"driver":"virtio-serial-pci","id":'
+                         '"virtio-serial0","bus":"pci.0","addr":"0x4"}')
+        self.vm.add_args('-device', '{"driver":"scsi-cd","bus":"scsi0.0"'
+                         ',"channel":0,"scsi-id":0,"lun":0,"device_id":'
+                         '"drive-scsi0-0-0-0","id":"scsi0-0-0-0"}')
+        self.vm.add_args('-device', '{"driver":"virtio-balloon-pci",'
+                         '"id":"balloon0","bus":"pci.0","addr":"0x6"}')
+        self.vm.add_args('-audiodev', '{"id":"audio1","driver":"none"}')
+        self.vm.add_args('-device', '{"driver":"usb-tablet","id":"input0"'
+                         ',"bus":"usb.0","port":"1"}')
+        self.vm.add_args('-device', '{"driver":"usb-kbd","id":"input1"'
+                         ',"bus":"usb.0","port":"2"}')
+        self.vm.add_args('-device', '{"driver":"VGA","id":"video0",'
+                         '"vgamem_mb":16,"bus":"pci.0","addr":"0x7"}')
+        self.vm.add_args('-object', '{"qom-type":"rng-random","id":"objrng0"'
+                         ',"filename":"/dev/urandom"}',
+                         '-device', '{"driver":"virtio-rng-pci","rng":"objrng0"'
+                         ',"id":"rng0","bus":"pci.0","addr":"0x8"}')
+        self.vm.add_args('-object', '{"qom-type":"cryptodev-backend-builtin",'
+                         '"id":"objcrypto0","queues":1}',
+                         '-device', '{"driver":"virtio-crypto-pci",'
+                         '"cryptodev":"objcrypto0","id":"crypto0","bus"'
+                         ':"pci.0","addr":"0xa"}')
+        self.vm.add_args('-device', '{"driver":"spapr-pci-host-bridge"'
+                         ',"index":1,"id":"pci.1"}')
+        self.vm.add_args('-device', '{"driver":"spapr-vscsi","id":"scsi1"'
+                         ',"reg":12288}')
+        self.vm.add_args('-m', '2G,slots=32,maxmem=4G',
+                         '-object', 'memory-backend-ram,id=ram1,size=1G',
+                         '-device', 'pc-dimm,id=dimm1,memdev=ram1')
+
+        # Create a temporary qcow2 and launch the test-case
+        with tempfile.NamedTemporaryFile(prefix=prefix,
+                                         suffix='.qcow2') as qcow2:
+            process.run(self.qemu_img + ' create -f qcow2 ' +
+                        qcow2.name + ' 1G')
+
+            self.vm.add_args('-drive', 'file=' + qcow2.name +
+                         ',format=qcow2,if=none,id='
+                         'drive-virtio-disk1',
+                         '-device', 'virtio-blk-pci,scsi=off,bus=pci.0,'
+                         'addr=0xb,drive=drive-virtio-disk1,id=virtio-disk1'
+                         ',bootindex=2')
+            self.common_tuxrun(drive="scsi-hd")
+
      #
      # The tests themselves. The configuration is derived from how
      # tuxrun invokes qemu (with minor tweaks like using -blockdev
@@ -308,7 +368,7 @@ def test_ppc64(self):
          """
          :avocado: tags=arch:ppc64
          :avocado: tags=machine:pseries
-        :avocado: tags=cpu:POWER8
+        :avocado: tags=cpu:POWER10
          :avocado: tags=endian:big
          :avocado: tags=console:hvc0
          :avocado: tags=tuxboot:ppc64
@@ -316,20 +376,20 @@ def test_ppc64(self):
          :avocado: tags=extradev:driver=spapr-vscsi
          :avocado: tags=root:sda
          """
-        self.common_tuxrun(drive="scsi-hd")
+        self.ppc64_common_tuxrun(prefix='tuxrun_ppc64_')
 
Quoted 15 lines
Quoted 5 lines
tests/avocado/tuxrun_baselines.py +0 -1
      def test_ppc64le(self):
          """
          :avocado: tags=arch:ppc64
          :avocado: tags=machine:pseries
-        :avocado: tags=cpu:POWER8

tests/avocado/tuxrun_baselines.py def test_ppc64(self): · line 393

Quoted 10 lines
tests/avocado/tuxrun_baselines.py +2 -1
+        :avocado: tags=cpu:POWER10
          :avocado: tags=console:hvc0
          :avocado: tags=tuxboot:ppc64le
          :avocado: tags=image:vmlinux
          :avocado: tags=extradev:driver=spapr-vscsi
          :avocado: tags=root:sda
          """
-        self.common_tuxrun(drive="scsi-hd")
+        self.ppc64_common_tuxrun(prefix='tuxrun_ppc64le_')
 

Looks neat, thanks for addressing review comments!

Quoted 6 lines
Quoted 6 lines
tests/avocado/tuxrun_baselines.py +0 -0
      def test_riscv32(self):
          """
Signature · 1 line
2.25.1
#9 lore raw
In reply to Kautuk Consul

Adding Harsh Prateek Bora.

Quoted 26 lines · Kautuk Consul

Commit c0c8687ef0fd990db8db1655a8a6c5a5e35dd4bb disabled the boot_linux.py test-case due to which the code coverage for ppc decreased by around 2%. As per the discussion on
https://lore.kernel.org/qemu-devel/[email protected]/ it was mentioned that the baseline test for ppc64 could be modified to make up this 2% code coverage. This patchset attempts to achieve this 2% code coverage by adding various device command line arguments (to ./qemu-system-ppc64) in the tuxrun_baselines.py test-case.

Changes since v3:
- Create a common ppc64_common_tuxrun routine in tuxrun_baselines.py and call that from the ppc64 and ppc64le test case routines.

Kautuk Consul (2):
avocado_qemu/__init__.py: factor out the qemu-img finding tests/avocado/tuxrun_baselines.py: improve code coverage for ppc64

 tests/avocado/avocado_qemu/__init__.py | 27 +++++-----
 tests/avocado/tuxrun_baselines.py      | 68 ++++++++++++++++++++++++--
 2 files changed, 80 insertions(+), 15 deletions(-)
Signature · 1 line
2.25.1
#10 lore raw
In reply to Kautuk Consul
Quoted 9 lines · Kautuk Consul

Commit c0c8687ef0fd990db8db1655a8a6c5a5e35dd4bb disabled the boot_linux.py test-case due to which the code coverage for ppc decreased by around 2%. As per the discussion on
https://lore.kernel.org/qemu-devel/[email protected]/ it was mentioned that the baseline test for ppc64 could be modified to make up this 2% code coverage. This patchset attempts to achieve this 2% code coverage by adding various device command line arguments (to ./qemu-system-ppc64) in the tuxrun_baselines.py test-case.

I've pulled the first patch into my testing/next, but the second conflicts with the inflight patch which adds checksums:

Message-Id: <[email protected]>
Date: Mon, 17 Apr 2023 14:43:17 +0100
Subject: [PATCH v4 2/6] tests/avocado: use the new snapshots for testing
From: =?UTF-8?q?Alex=20Benn=C3=A9e?= <[email protected]>

You can either wait and re-base once the PR goes in (I'll send a pre-PR Monday) or do you can re-base directly off my branch at:

https://gitlab.com/stsquad/qemu/-/tree/testing/next

and send the tested patch mentioning its based off my testing/next and I'll include it in the pre-PR.

Quoted 12 lines

Changes since v3:
- Create a common ppc64_common_tuxrun routine in tuxrun_baselines.py and call that from the ppc64 and ppc64le test case routines.

Kautuk Consul (2):
avocado_qemu/__init__.py: factor out the qemu-img finding tests/avocado/tuxrun_baselines.py: improve code coverage for ppc64

 tests/avocado/avocado_qemu/__init__.py | 27 +++++-----
 tests/avocado/tuxrun_baselines.py      | 68 ++++++++++++++++++++++++--
 2 files changed, 80 insertions(+), 15 deletions(-)
Signature · 2 lines
Alex Bennée
Virtualisation Tech Lead @ Linaro
#11 lore raw
Quoted 21 lines · Alex Bennée
Quoted 9 lines · Kautuk Consul

Commit c0c8687ef0fd990db8db1655a8a6c5a5e35dd4bb disabled the boot_linux.py test-case due to which the code coverage for ppc decreased by around 2%. As per the discussion on
https://lore.kernel.org/qemu-devel/[email protected]/ it was mentioned that the baseline test for ppc64 could be modified to make up this 2% code coverage. This patchset attempts to achieve this 2% code coverage by adding various device command line arguments (to ./qemu-system-ppc64) in the tuxrun_baselines.py test-case.

I've pulled the first patch into my testing/next, but the second conflicts with the inflight patch which adds checksums:

Message-Id: <[email protected]>
Date: Mon, 17 Apr 2023 14:43:17 +0100
Subject: [PATCH v4 2/6] tests/avocado: use the new snapshots for testing
From: =?UTF-8?q?Alex=20Benn=C3=A9e?= <[email protected]>

Quoted 7 lines

You can either wait and re-base once the PR goes in (I'll send a pre-PR Monday) or do you can re-base directly off my branch at:

https://gitlab.com/stsquad/qemu/-/tree/testing/next

and send the tested patch mentioning its based off my testing/next and I'll include it in the pre-PR.

Sent a v5 for this rebased off
https://gitlab.com/stsquad/qemu/-/tree/testing/next.

Thanks.

Quoted 19 lines
Quoted 13 lines

Changes since v3:
- Create a common ppc64_common_tuxrun routine in tuxrun_baselines.py and call that from the ppc64 and ppc64le test case routines.

Kautuk Consul (2):
avocado_qemu/__init__.py: factor out the qemu-img finding tests/avocado/tuxrun_baselines.py: improve code coverage for ppc64

 tests/avocado/avocado_qemu/__init__.py | 27 +++++-----
 tests/avocado/tuxrun_baselines.py      | 68 ++++++++++++++++++++++++--
 2 files changed, 80 insertions(+), 15 deletions(-)
Signature · 2 lines
Alex Bennée
Virtualisation Tech Lead @ Linaro