Skip to contentSkip to conversation

Improve avocado code coverage for ppc64

PATCH Superseded Kautuk Consul 5 participants 10 messages Apr 20, 2023 – Apr 21, 2023

v3 of 5 Superseded Latest is v5 →

lore
Share & tools
lore
#0 lore raw

v3 cover Improve avocado code coverage for ppc64

Commit c0c8687ef0fd990db8db1655a8a6c5a5e35dd4bb disabled the boot_linux.py test-case due to which the code coverage for powerpc decreased by 2%. This patchset aims to make up this 2% code coverage by modifying the tuxrun_baselines.py test-case.

Changes since v1 and v2:
- Modified the way and code that creates the temporary qcow2 image that the tuxrun_baselines.py is now using.
- Factored out code in avocado_qemu/__init__.py to create the get_qemu_img() function that will now get called from tuxrun_baselines.py.

Kautuk Consul (2):
avocado_qemu/__init__.py: factor out the qemu-img finding tests/avocado/tuxrun_baselines.py: improve code coverage for ppc64

 tests/avocado/avocado_qemu/__init__.py |  27 +++---
 tests/avocado/tuxrun_baselines.py      | 120 ++++++++++++++++++++++++-
 2 files changed, 132 insertions(+), 15 deletions(-)
Signature · 1 line
2.25.1
#1 lore raw

PATCH v3 1/2 avocado_qemu/__init__.py: factor out the qemu-img finding

Factor out the code that finds the qemu-img binary in the QemuSystemTest class and create a new get_qemu_img() function with it. This function will get called also from the new code in tuxrun_baselines.py avocado test-case.

---
 tests/avocado/avocado_qemu/__init__.py | 27 +++++++++++++++-----------
 1 file changed, 16 insertions(+), 11 deletions(-)
tests/avocado/avocado_qemu/__init__.py +16 -11
index cb71f50db9..d8f390e27a 100644
@@ -330,6 +330,19 @@ def _new_vm(self, name, *args):
             vm.add_args(*args)
         return vm
 
+    def get_qemu_img(self):
+        self.log.debug('Looking for and selecting a qemu-img binary')
+
+        # If qemu-img has been built, use it, otherwise the system wide one
+        # will be used.
+        qemu_img = os.path.join(BUILD_DIR, 'qemu-img')
+        if not os.path.exists(qemu_img):
+            qemu_img = find_command('qemu-img', False)
+        if qemu_img is False:
+            self.cancel('Could not find "qemu-img"')
+
+        return qemu_img
+
     @property
     def vm(self):
         return self.get_vm(name='default')
@@ -594,17 +607,9 @@ def set_up_existing_ssh_keys(self):
         return (ssh_public_key, ssh_private_key)
 
     def download_boot(self):
-        self.log.debug('Looking for and selecting a qemu-img binary to be '
-                       'used to create the bootable snapshot image')
-        # If qemu-img has been built, use it, otherwise the system wide one
-        # will be used.  If none is available, the test will cancel.
-        qemu_img = os.path.join(BUILD_DIR, 'qemu-img')
-        if not os.path.exists(qemu_img):
-            qemu_img = find_command('qemu-img', False)
-        if qemu_img is False:
-            self.cancel('Could not find "qemu-img", which is required to '
-                        'create the bootable image')
-        vmimage.QEMU_IMG = qemu_img
+        # Set the qemu-img binary.
+        # If none is available, the test will cancel.
+        vmimage.QEMU_IMG = super().get_qemu_img()
 
         self.log.info('Downloading/preparing boot image')
         # Fedora 31 only provides ppc64le images
Signature · 1 line
2.25.1
#2 lore raw

PATCH v3 2/2 tests/avocado/tuxrun_baselines.py: improve code coverage for ppc64

In reply to Kautuk Consul

Commit c0c8687ef0fd990db8db1655a8a6c5a5e35dd4bb disabled the boot_linux.py test-case due to which the code coverage for ppc decreased by around 2%. As per the discussion on
https://lore.kernel.org/qemu-devel/[email protected]/ it was mentioned that the baseline test for ppc64 could be modified to make up this 2% code coverage. This patch attempts to achieve this 2% code coverage by adding various device command line arguments (to ./qemu-system-ppc64) in the tuxrun_baselines.py test-case.

The code coverage report with boot_linux.py, without it and finally with these tuxrun_baselines.py changes is as follows:

With boot_linux.py
------------------
lines......: 13.8% (58006 of 420997 lines) functions..: 20.7% (7675 of 36993 functions) branches...: 9.2% (22146 of 240611 branches)
Without boot_linux.py (without this patch changes)
--------------------------------------------------
lines......: 11.9% (50174 of 420997 lines) functions..: 18.8% (6947 of 36993 functions) branches...: 7.4% (17580 of 239017 branches)
Without boot_linux.py (with this patch changes)
-----------------------------------------------
lines......: 13.8% (58287 of 420997 lines) functions..: 20.7% (7640 of 36993 functions) branches...: 8.4% (20223 of 240611 branches)

---
 tests/avocado/tuxrun_baselines.py | 120 +++++++++++++++++++++++++++++-
 1 file changed, 116 insertions(+), 4 deletions(-)
tests/avocado/tuxrun_baselines.py +116 -4
index d343376faa..ae082ac028 100644
@@ -11,6 +11,7 @@
 
 import os
 import time
+import tempfile
 
 from avocado import skip, skipIf
 from avocado_qemu import QemuSystemTest
@@ -72,6 +73,8 @@ def setUp(self):
         # Occasionally we need extra devices to hook things up
         self.extradev = self.get_tag('extradev')
 
+        self.qemu_img = super().get_qemu_img()
+
     def wait_for_console_pattern(self, success_message, vm=None):
         wait_for_console_pattern(self, success_message,
                                  failure_message='Kernel panic - not syncing',
@@ -308,7 +311,7 @@ def test_ppc64(self):
         """
         :avocado: tags=arch:ppc64
         :avocado: tags=machine:pseries
-        :avocado: tags=cpu:POWER8
+        :avocado: tags=cpu:POWER10
         :avocado: tags=endian:big
         :avocado: tags=console:hvc0
         :avocado: tags=tuxboot:ppc64
@@ -316,20 +319,129 @@ def test_ppc64(self):
         :avocado: tags=extradev:driver=spapr-vscsi
         :avocado: tags=root:sda
         """
-        self.common_tuxrun(drive="scsi-hd")
+
+        # add device args to command line.
+        self.vm.add_args('-netdev', 'user,id=vnet,hostfwd=:127.0.0.1:0-:22',
+                         '-device', 'virtio-net,netdev=vnet')
+        self.vm.add_args('-netdev', '{"type":"user","id":"hostnet0"}',
+                         '-device', '{"driver":"virtio-net-pci","netdev":'
+                         '"hostnet0","id":"net0","mac":"52:54:00:4c:e3:86",'
+                         '"bus":"pci.0","addr":"0x9"}')
+        self.vm.add_args('-device', '{"driver":"qemu-xhci","p2":15,"p3":15,'
+                         '"id":"usb","bus":"pci.0","addr":"0x2"}')
+        self.vm.add_args('-device', '{"driver":"virtio-scsi-pci","id":"scsi0"'
+                         ',"bus":"pci.0","addr":"0x3"}')
+        self.vm.add_args('-device', '{"driver":"virtio-serial-pci","id":'
+                         '"virtio-serial0","bus":"pci.0","addr":"0x4"}')
+        self.vm.add_args('-device', '{"driver":"scsi-cd","bus":"scsi0.0"'
+                         ',"channel":0,"scsi-id":0,"lun":0,"device_id":'
+                         '"drive-scsi0-0-0-0","id":"scsi0-0-0-0"}')
+        self.vm.add_args('-device', '{"driver":"virtio-balloon-pci",'
+                         '"id":"balloon0","bus":"pci.0","addr":"0x6"}')
+        self.vm.add_args('-audiodev', '{"id":"audio1","driver":"none"}')
+        self.vm.add_args('-device', '{"driver":"usb-tablet","id":"input0"'
+                         ',"bus":"usb.0","port":"1"}')
+        self.vm.add_args('-device', '{"driver":"usb-kbd","id":"input1"'
+                         ',"bus":"usb.0","port":"2"}')
+        self.vm.add_args('-device', '{"driver":"VGA","id":"video0",'
+                         '"vgamem_mb":16,"bus":"pci.0","addr":"0x7"}')
+        self.vm.add_args('-object', '{"qom-type":"rng-random","id":"objrng0"'
+                         ',"filename":"/dev/urandom"}',
+                         '-device', '{"driver":"virtio-rng-pci","rng":"objrng0"'
+                         ',"id":"rng0","bus":"pci.0","addr":"0x8"}')
+        self.vm.add_args('-object', '{"qom-type":"cryptodev-backend-builtin",'
+                         '"id":"objcrypto0","queues":1}',
+                         '-device', '{"driver":"virtio-crypto-pci",'
+                         '"cryptodev":"objcrypto0","id":"crypto0","bus"'
+                         ':"pci.0","addr":"0xa"}')
+        self.vm.add_args('-device', '{"driver":"spapr-pci-host-bridge"'
+                         ',"index":1,"id":"pci.1"}')
+        self.vm.add_args('-device', '{"driver":"spapr-vscsi","id":"scsi1"'
+                         ',"reg":12288}')
+        self.vm.add_args('-m', '2G,slots=32,maxmem=4G',
+                         '-object', 'memory-backend-ram,id=ram1,size=1G',
+                         '-device', 'pc-dimm,id=dimm1,memdev=ram1')
+
+        # Create a temporary qcow2 and launch the test-case
+        with tempfile.NamedTemporaryFile(prefix='tuxrun_ppc64_',
+                                         suffix='.qcow2') as qcow2:
+            process.run(self.qemu_img + ' create -f qcow2 ' +
+                        qcow2.name + ' 1G')
+
+            self.vm.add_args('-drive', 'file=' + qcow2.name +
+                         ',format=qcow2,if=none,id='
+                         'drive-virtio-disk1',
+                         '-device', 'virtio-blk-pci,scsi=off,bus=pci.0,'
+                         'addr=0xb,drive=drive-virtio-disk1,id=virtio-disk1'
+                         ',bootindex=2')
+            self.common_tuxrun(drive="scsi-hd")
 
     def test_ppc64le(self):
         """
         :avocado: tags=arch:ppc64
         :avocado: tags=machine:pseries
-        :avocado: tags=cpu:POWER8
+        :avocado: tags=cpu:POWER10
         :avocado: tags=console:hvc0
         :avocado: tags=tuxboot:ppc64le
         :avocado: tags=image:vmlinux
         :avocado: tags=extradev:driver=spapr-vscsi
         :avocado: tags=root:sda
         """
-        self.common_tuxrun(drive="scsi-hd")
+        # add device args to command line.
+        self.vm.add_args('-netdev', 'user,id=vnet,hostfwd=:127.0.0.1:0-:22',
+                         '-device', 'virtio-net,netdev=vnet')
+        self.vm.add_args('-netdev', '{"type":"user","id":"hostnet0"}',
+                         '-device', '{"driver":"virtio-net-pci","netdev":'
+                         '"hostnet0","id":"net0","mac":"52:54:00:4c:e3:86",'
+                         '"bus":"pci.0","addr":"0x9"}')
+        self.vm.add_args('-device', '{"driver":"qemu-xhci","p2":15,"p3":15,'
+                         '"id":"usb","bus":"pci.0","addr":"0x2"}')
+        self.vm.add_args('-device', '{"driver":"virtio-scsi-pci","id":"scsi0"'
+                         ',"bus":"pci.0","addr":"0x3"}')
+        self.vm.add_args('-device', '{"driver":"virtio-serial-pci","id":'
+                         '"virtio-serial0","bus":"pci.0","addr":"0x4"}')
+        self.vm.add_args('-device', '{"driver":"scsi-cd","bus":"scsi0.0"'
+                         ',"channel":0,"scsi-id":0,"lun":0,"device_id":'
+                         '"drive-scsi0-0-0-0","id":"scsi0-0-0-0"}')
+        self.vm.add_args('-device', '{"driver":"virtio-balloon-pci",'
+                         '"id":"balloon0","bus":"pci.0","addr":"0x6"}')
+        self.vm.add_args('-audiodev', '{"id":"audio1","driver":"none"}')
+        self.vm.add_args('-device', '{"driver":"usb-tablet","id":"input0"'
+                         ',"bus":"usb.0","port":"1"}')
+        self.vm.add_args('-device', '{"driver":"usb-kbd","id":"input1"'
+                         ',"bus":"usb.0","port":"2"}')
+        self.vm.add_args('-device', '{"driver":"VGA","id":"video0",'
+                         '"vgamem_mb":16,"bus":"pci.0","addr":"0x7"}')
+        self.vm.add_args('-object', '{"qom-type":"rng-random","id":"objrng0"'
+                         ',"filename":"/dev/urandom"}',
+                         '-device', '{"driver":"virtio-rng-pci","rng":"objrng0"'
+                         ',"id":"rng0","bus":"pci.0","addr":"0x8"}')
+        self.vm.add_args('-object', '{"qom-type":"cryptodev-backend-builtin",'
+                         '"id":"objcrypto0","queues":1}',
+                         '-device', '{"driver":"virtio-crypto-pci",'
+                         '"cryptodev":"objcrypto0","id":"crypto0","bus"'
+                         ':"pci.0","addr":"0xa"}')
+        self.vm.add_args('-device', '{"driver":"spapr-pci-host-bridge"'
+                         ',"index":1,"id":"pci.1"}')
+        self.vm.add_args('-device', '{"driver":"spapr-vscsi","id":"scsi1"'
+                         ',"reg":12288}')
+        self.vm.add_args('-m', '2G,slots=32,maxmem=4G',
+                         '-object', 'memory-backend-ram,id=ram1,size=1G',
+                         '-device', 'pc-dimm,id=dimm1,memdev=ram1')
+
+        # Create a temporary qcow2 and launch the test-case
+        with tempfile.NamedTemporaryFile(prefix='tuxrun_ppc64le_',
+                                         suffix='.qcow2') as qcow2:
+            process.run(self.qemu_img + ' create -f qcow2 ' +
+                        qcow2.name + ' 1G')
+
+            self.vm.add_args('-drive', 'file=' + qcow2.name +
+                         ',format=qcow2,if=none,id='
+                         'drive-virtio-disk1',
+                         '-device', 'virtio-blk-pci,scsi=off,bus=pci.0,'
+                         'addr=0xb,drive=drive-virtio-disk1,id=virtio-disk1'
+                         ',bootindex=2')
+            self.common_tuxrun(drive="scsi-hd")
 
     def test_riscv32(self):
         """
Signature · 1 line
2.25.1
#3 lore raw

Re: [PATCH v3 2/2] tests/avocado/tuxrun_baselines.py: improve code coverage for ppc64

Since we are optimising code a lot, one suggestion below:

Quoted 90 lines · Kautuk Consul

Commit c0c8687ef0fd990db8db1655a8a6c5a5e35dd4bb disabled the boot_linux.py test-case due to which the code coverage for ppc decreased by around 2%. As per the discussion on
https://lore.kernel.org/qemu-devel/[email protected]/ it was mentioned that the baseline test for ppc64 could be modified to make up this 2% code coverage. This patch attempts to achieve this 2% code coverage by adding various device command line arguments (to ./qemu-system-ppc64) in the tuxrun_baselines.py test-case.

The code coverage report with boot_linux.py, without it and finally with these tuxrun_baselines.py changes is as follows:

With boot_linux.py
------------------
lines......: 13.8% (58006 of 420997 lines) functions..: 20.7% (7675 of 36993 functions) branches...: 9.2% (22146 of 240611 branches)
Without boot_linux.py (without this patch changes)
--------------------------------------------------
lines......: 11.9% (50174 of 420997 lines) functions..: 18.8% (6947 of 36993 functions) branches...: 7.4% (17580 of 239017 branches)
Without boot_linux.py (with this patch changes)
-----------------------------------------------
lines......: 13.8% (58287 of 420997 lines) functions..: 20.7% (7640 of 36993 functions) branches...: 8.4% (20223 of 240611 branches)

---
 tests/avocado/tuxrun_baselines.py | 120 +++++++++++++++++++++++++++++-
 1 file changed, 116 insertions(+), 4 deletions(-)
tests/avocado/tuxrun_baselines.py +10 -2
index d343376faa..ae082ac028 100644
@@ -11,6 +11,7 @@
 
 import os
 import time
+import tempfile
 
<snip>
         """
-        self.common_tuxrun(drive="scsi-hd")
+
+        # add device args to command line.
+        self.vm.add_args('-netdev', 'user,id=vnet,hostfwd=:127.0.0.1:0
-:22',
+                         '-device', 'virtio-net,netdev=vnet')
+        self.vm.add_args('-netdev', '{"type":"user","id":"hostnet0"}',
+                         '-device', '{"driver":"virtio-net-pci","netdev":'
+
 '"hostnet0","id":"net0","mac":"52:54:00:4c:e3:86",'
+                         '"bus":"pci.0","addr":"0x9"}')
+        self.vm.add_args('-device',

'{"driver":"qemu-xhci","p2":15,"p3":15,'

tests/avocado/tuxrun_baselines.py +2 -0
+                         '"id":"usb","bus":"pci.0","addr":"0x2"}')
+        self.vm.add_args('-device',

'{"driver":"virtio-scsi-pci","id":"scsi0"'

tests/avocado/tuxrun_baselines.py +16 -0
+                         ',"bus":"pci.0","addr":"0x3"}')
+        self.vm.add_args('-device', '{"driver":"virtio-serial-pci","id":'
+                         '"virtio-serial0","bus":"pci.0","addr":"0x4"}')
+        self.vm.add_args('-device', '{"driver":"scsi-cd","bus":"scsi0.0"'
+                         ',"channel":0,"scsi-id":0,"lun":0,"device_id":'
+                         '"drive-scsi0-0-0-0","id":"scsi0-0-0-0"}')
+        self.vm.add_args('-device', '{"driver":"virtio-balloon-pci",'
+                         '"id":"balloon0","bus":"pci.0","addr":"0x6"}')
+        self.vm.add_args('-audiodev', '{"id":"audio1","driver":"none"}')
+        self.vm.add_args('-device', '{"driver":"usb-tablet","id":"input0"'
+                         ',"bus":"usb.0","port":"1"}')
+        self.vm.add_args('-device', '{"driver":"usb-kbd","id":"input1"'
+                         ',"bus":"usb.0","port":"2"}')
+        self.vm.add_args('-device', '{"driver":"VGA","id":"video0",'
+                         '"vgamem_mb":16,"bus":"pci.0","addr":"0x7"}')
+        self.vm.add_args('-object',

'{"qom-type":"rng-random","id":"objrng0"'

tests/avocado/tuxrun_baselines.py +2 -0
+                         ',"filename":"/dev/urandom"}',
+                         '-device',

'{"driver":"virtio-rng-pci","rng":"objrng0"'

tests/avocado/tuxrun_baselines.py +2 -0
+                         ',"id":"rng0","bus":"pci.0","addr":"0x8"}')
+        self.vm.add_args('-object',

'{"qom-type":"cryptodev-backend-builtin",'

tests/avocado/tuxrun_baselines.py +2 -0
+                         '"id":"objcrypto0","queues":1}',
+                         '-device', '{"driver":"virtio-crypto-pci",'

tests/avocado/tuxrun_baselines.py def test_ppc64(self): · line 364

Quoted 10 lines
tests/avocado/tuxrun_baselines.py +10 -0
+                         '"cryptodev":"objcrypto0","id":"crypto0","bus"'
+                         ':"pci.0","addr":"0xa"}')
+        self.vm.add_args('-device', '{"driver":"spapr-pci-host-bridge"'
+                         ',"index":1,"id":"pci.1"}')
+        self.vm.add_args('-device', '{"driver":"spapr-vscsi","id":"scsi1"'
+                         ',"reg":12288}')
+        self.vm.add_args('-m', '2G,slots=32,maxmem=4G',
+                         '-object', 'memory-backend-ram,id=ram1,size=1G',
+                         '-device', 'pc-dimm,id=dimm1,memdev=ram1')
+

Most of the above code for adding args is getting duplicated in the test for ppc64le below as well.
It could be contained in a helper routine which could be called from both, and would be easier to maintain.

regards,
Harsh

+ # Create a temporary qcow2 and launch the test-case

Quoted 98 lines
tests/avocado/tuxrun_baselines.py +21 -3
+        with tempfile.NamedTemporaryFile(prefix='tuxrun_ppc64_',
+                                         suffix='.qcow2') as qcow2:
+            process.run(self.qemu_img + ' create -f qcow2 ' +
+                        qcow2.name + ' 1G')
+
+            self.vm.add_args('-drive', 'file=' + qcow2.name +
+                         ',format=qcow2,if=none,id='
+                         'drive-virtio-disk1',
+                         '-device', 'virtio-blk-pci,scsi=off,bus=pci.0,'
+
 'addr=0xb,drive=drive-virtio-disk1,id=virtio-disk1'
+                         ',bootindex=2')
+            self.common_tuxrun(drive="scsi-hd")
 
     def test_ppc64le(self):
         """
         :avocado: tags=arch:ppc64
         :avocado: tags=machine:pseries
-        :avocado: tags=cpu:POWER8
+        :avocado: tags=cpu:POWER10
         :avocado: tags=console:hvc0
         :avocado: tags=tuxboot:ppc64le
         :avocado: tags=image:vmlinux
         :avocado: tags=extradev:driver=spapr-vscsi
         :avocado: tags=root:sda
         """
-        self.common_tuxrun(drive="scsi-hd")
+        # add device args to command line.
+        self.vm.add_args('-netdev', 'user,id=vnet,hostfwd=:127.0.0.1:0
-:22',
+                         '-device', 'virtio-net,netdev=vnet')
+        self.vm.add_args('-netdev', '{"type":"user","id":"hostnet0"}',
+                         '-device', '{"driver":"virtio-net-pci","netdev":'
+
 '"hostnet0","id":"net0","mac":"52:54:00:4c:e3:86",'
+                         '"bus":"pci.0","addr":"0x9"}')
+        self.vm.add_args('-device',

'{"driver":"qemu-xhci","p2":15,"p3":15,'

tests/avocado/tuxrun_baselines.py +2 -0
+                         '"id":"usb","bus":"pci.0","addr":"0x2"}')
+        self.vm.add_args('-device',

'{"driver":"virtio-scsi-pci","id":"scsi0"'

tests/avocado/tuxrun_baselines.py +16 -0
+                         ',"bus":"pci.0","addr":"0x3"}')
+        self.vm.add_args('-device', '{"driver":"virtio-serial-pci","id":'
+                         '"virtio-serial0","bus":"pci.0","addr":"0x4"}')
+        self.vm.add_args('-device', '{"driver":"scsi-cd","bus":"scsi0.0"'
+                         ',"channel":0,"scsi-id":0,"lun":0,"device_id":'
+                         '"drive-scsi0-0-0-0","id":"scsi0-0-0-0"}')
+        self.vm.add_args('-device', '{"driver":"virtio-balloon-pci",'
+                         '"id":"balloon0","bus":"pci.0","addr":"0x6"}')
+        self.vm.add_args('-audiodev', '{"id":"audio1","driver":"none"}')
+        self.vm.add_args('-device', '{"driver":"usb-tablet","id":"input0"'
+                         ',"bus":"usb.0","port":"1"}')
+        self.vm.add_args('-device', '{"driver":"usb-kbd","id":"input1"'
+                         ',"bus":"usb.0","port":"2"}')
+        self.vm.add_args('-device', '{"driver":"VGA","id":"video0",'
+                         '"vgamem_mb":16,"bus":"pci.0","addr":"0x7"}')
+        self.vm.add_args('-object',

'{"qom-type":"rng-random","id":"objrng0"'

tests/avocado/tuxrun_baselines.py +2 -0
+                         ',"filename":"/dev/urandom"}',
+                         '-device',

'{"driver":"virtio-rng-pci","rng":"objrng0"'

tests/avocado/tuxrun_baselines.py +2 -0
+                         ',"id":"rng0","bus":"pci.0","addr":"0x8"}')
+        self.vm.add_args('-object',

'{"qom-type":"cryptodev-backend-builtin",'

tests/avocado/tuxrun_baselines.py +25 -1
+                         '"id":"objcrypto0","queues":1}',
+                         '-device', '{"driver":"virtio-crypto-pci",'
+                         '"cryptodev":"objcrypto0","id":"crypto0","bus"'
+                         ':"pci.0","addr":"0xa"}')
+        self.vm.add_args('-device', '{"driver":"spapr-pci-host-bridge"'
+                         ',"index":1,"id":"pci.1"}')
+        self.vm.add_args('-device', '{"driver":"spapr-vscsi","id":"scsi1"'
+                         ',"reg":12288}')
+        self.vm.add_args('-m', '2G,slots=32,maxmem=4G',
+                         '-object', 'memory-backend-ram,id=ram1,size=1G',
+                         '-device', 'pc-dimm,id=dimm1,memdev=ram1')
+
+        # Create a temporary qcow2 and launch the test-case
+        with tempfile.NamedTemporaryFile(prefix='tuxrun_ppc64le_',
+                                         suffix='.qcow2') as qcow2:
+            process.run(self.qemu_img + ' create -f qcow2 ' +
+                        qcow2.name + ' 1G')
+
+            self.vm.add_args('-drive', 'file=' + qcow2.name +
+                         ',format=qcow2,if=none,id='
+                         'drive-virtio-disk1',
+                         '-device', 'virtio-blk-pci,scsi=off,bus=pci.0,'
+
 'addr=0xb,drive=drive-virtio-disk1,id=virtio-disk1'
+                         ',bootindex=2')
+            self.common_tuxrun(drive="scsi-hd")
 
     def test_riscv32(self):
         """
--

2.25.1

#4 lore raw

Re: [PATCH v3 2/2] tests/avocado/tuxrun_baselines.py: improve code coverage for ppc64

Hi,

Quoted 104 lines · Harsh Prateek Bora

Since we are optimising code a lot, one suggestion below:

Quoted 98 lines · Kautuk Consul

Commit c0c8687ef0fd990db8db1655a8a6c5a5e35dd4bb disabled the boot_linux.py test-case due to which the code coverage for ppc decreased by around 2%. As per the discussion on
https://lore.kernel.org/qemu-devel/[email protected]/ it was mentioned that the baseline test for ppc64 could be modified to make up this 2% code coverage. This patch attempts to achieve this 2% code coverage by adding various device command line arguments (to ./qemu-system-ppc64) in the tuxrun_baselines.py test-case.

The code coverage report with boot_linux.py, without it and finally with these tuxrun_baselines.py changes is as follows:

With boot_linux.py
------------------
lines......: 13.8% (58006 of 420997 lines) functions..: 20.7% (7675 of 36993 functions) branches...: 9.2% (22146 of 240611 branches)
Without boot_linux.py (without this patch changes)
--------------------------------------------------
lines......: 11.9% (50174 of 420997 lines) functions..: 18.8% (6947 of 36993 functions) branches...: 7.4% (17580 of 239017 branches)
Without boot_linux.py (with this patch changes)
-----------------------------------------------
lines......: 13.8% (58287 of 420997 lines) functions..: 20.7% (7640 of 36993 functions) branches...: 8.4% (20223 of 240611 branches)

---
 tests/avocado/tuxrun_baselines.py | 120 +++++++++++++++++++++++++++++-
 1 file changed, 116 insertions(+), 4 deletions(-)
tests/avocado/tuxrun_baselines.py +10 -2
index d343376faa..ae082ac028 100644
@@ -11,6 +11,7 @@
 
 import os
 import time
+import tempfile
 
<snip>
         """
-        self.common_tuxrun(drive="scsi-hd")
+
+        # add device args to command line.
+        self.vm.add_args('-netdev', 'user,id=vnet,hostfwd=:127.0.0.1:0
-:22',
+                         '-device', 'virtio-net,netdev=vnet')
+        self.vm.add_args('-netdev', '{"type":"user","id":"hostnet0"}',
+                         '-device', '{"driver":"virtio-net-pci","netdev":'
+
 '"hostnet0","id":"net0","mac":"52:54:00:4c:e3:86",'
+                         '"bus":"pci.0","addr":"0x9"}')
+        self.vm.add_args('-device',

'{"driver":"qemu-xhci","p2":15,"p3":15,'

tests/avocado/tuxrun_baselines.py +2 -0
+                         '"id":"usb","bus":"pci.0","addr":"0x2"}')
+        self.vm.add_args('-device',

'{"driver":"virtio-scsi-pci","id":"scsi0"'

tests/avocado/tuxrun_baselines.py +16 -0
+                         ',"bus":"pci.0","addr":"0x3"}')
+        self.vm.add_args('-device', '{"driver":"virtio-serial-pci","id":'
+                         '"virtio-serial0","bus":"pci.0","addr":"0x4"}')
+        self.vm.add_args('-device', '{"driver":"scsi-cd","bus":"scsi0.0"'
+                         ',"channel":0,"scsi-id":0,"lun":0,"device_id":'
+                         '"drive-scsi0-0-0-0","id":"scsi0-0-0-0"}')
+        self.vm.add_args('-device', '{"driver":"virtio-balloon-pci",'
+                         '"id":"balloon0","bus":"pci.0","addr":"0x6"}')
+        self.vm.add_args('-audiodev', '{"id":"audio1","driver":"none"}')
+        self.vm.add_args('-device', '{"driver":"usb-tablet","id":"input0"'
+                         ',"bus":"usb.0","port":"1"}')
+        self.vm.add_args('-device', '{"driver":"usb-kbd","id":"input1"'
+                         ',"bus":"usb.0","port":"2"}')
+        self.vm.add_args('-device', '{"driver":"VGA","id":"video0",'
+                         '"vgamem_mb":16,"bus":"pci.0","addr":"0x7"}')
+        self.vm.add_args('-object',

'{"qom-type":"rng-random","id":"objrng0"'

tests/avocado/tuxrun_baselines.py +2 -0
+                         ',"filename":"/dev/urandom"}',
+                         '-device',

'{"driver":"virtio-rng-pci","rng":"objrng0"'

tests/avocado/tuxrun_baselines.py +2 -0
+                         ',"id":"rng0","bus":"pci.0","addr":"0x8"}')
+        self.vm.add_args('-object',

'{"qom-type":"cryptodev-backend-builtin",'

tests/avocado/tuxrun_baselines.py +11 -0
+                         '"id":"objcrypto0","queues":1}',
+                         '-device', '{"driver":"virtio-crypto-pci",'
+                         '"cryptodev":"objcrypto0","id":"crypto0","bus"'
+                         ':"pci.0","addr":"0xa"}')
+        self.vm.add_args('-device', '{"driver":"spapr-pci-host-bridge"'
+                         ',"index":1,"id":"pci.1"}')
+        self.vm.add_args('-device', '{"driver":"spapr-vscsi","id":"scsi1"'
+                         ',"reg":12288}')
+        self.vm.add_args('-m', '2G,slots=32,maxmem=4G',
+                         '-object', 'memory-backend-ram,id=ram1,size=1G',
+                         '-device', 'pc-dimm,id=dimm1,memdev=ram1')
Quoted 8 lines
Quoted 2 lines

+

Most of the above code for adding args is getting duplicated in the test for ppc64le below as well.
It could be contained in a helper routine which could be called from both, and would be easier to maintain.

I kept it separate because there may be some future reason to make the devices different for ppc and ppc64le. I know that probably won't happen but thought to keep it separate to allow for that possibility. Or if you feel that they should be exactly the same maybe I could send another patch after this to refactor the code into a single routine ?

Quoted 103 lines

regards,
Harsh

+ # Create a temporary qcow2 and launch the test-case

Quoted 98 lines
tests/avocado/tuxrun_baselines.py +21 -3
+        with tempfile.NamedTemporaryFile(prefix='tuxrun_ppc64_',
+                                         suffix='.qcow2') as qcow2:
+            process.run(self.qemu_img + ' create -f qcow2 ' +
+                        qcow2.name + ' 1G')
+
+            self.vm.add_args('-drive', 'file=' + qcow2.name +
+                         ',format=qcow2,if=none,id='
+                         'drive-virtio-disk1',
+                         '-device', 'virtio-blk-pci,scsi=off,bus=pci.0,'
+
 'addr=0xb,drive=drive-virtio-disk1,id=virtio-disk1'
+                         ',bootindex=2')
+            self.common_tuxrun(drive="scsi-hd")
 
     def test_ppc64le(self):
         """
         :avocado: tags=arch:ppc64
         :avocado: tags=machine:pseries
-        :avocado: tags=cpu:POWER8
+        :avocado: tags=cpu:POWER10
         :avocado: tags=console:hvc0
         :avocado: tags=tuxboot:ppc64le
         :avocado: tags=image:vmlinux
         :avocado: tags=extradev:driver=spapr-vscsi
         :avocado: tags=root:sda
         """
-        self.common_tuxrun(drive="scsi-hd")
+        # add device args to command line.
+        self.vm.add_args('-netdev', 'user,id=vnet,hostfwd=:127.0.0.1:0
-:22',
+                         '-device', 'virtio-net,netdev=vnet')
+        self.vm.add_args('-netdev', '{"type":"user","id":"hostnet0"}',
+                         '-device', '{"driver":"virtio-net-pci","netdev":'
+
 '"hostnet0","id":"net0","mac":"52:54:00:4c:e3:86",'
+                         '"bus":"pci.0","addr":"0x9"}')
+        self.vm.add_args('-device',

'{"driver":"qemu-xhci","p2":15,"p3":15,'

tests/avocado/tuxrun_baselines.py +2 -0
+                         '"id":"usb","bus":"pci.0","addr":"0x2"}')
+        self.vm.add_args('-device',

'{"driver":"virtio-scsi-pci","id":"scsi0"'

tests/avocado/tuxrun_baselines.py +16 -0
+                         ',"bus":"pci.0","addr":"0x3"}')
+        self.vm.add_args('-device', '{"driver":"virtio-serial-pci","id":'
+                         '"virtio-serial0","bus":"pci.0","addr":"0x4"}')
+        self.vm.add_args('-device', '{"driver":"scsi-cd","bus":"scsi0.0"'
+                         ',"channel":0,"scsi-id":0,"lun":0,"device_id":'
+                         '"drive-scsi0-0-0-0","id":"scsi0-0-0-0"}')
+        self.vm.add_args('-device', '{"driver":"virtio-balloon-pci",'
+                         '"id":"balloon0","bus":"pci.0","addr":"0x6"}')
+        self.vm.add_args('-audiodev', '{"id":"audio1","driver":"none"}')
+        self.vm.add_args('-device', '{"driver":"usb-tablet","id":"input0"'
+                         ',"bus":"usb.0","port":"1"}')
+        self.vm.add_args('-device', '{"driver":"usb-kbd","id":"input1"'
+                         ',"bus":"usb.0","port":"2"}')
+        self.vm.add_args('-device', '{"driver":"VGA","id":"video0",'
+                         '"vgamem_mb":16,"bus":"pci.0","addr":"0x7"}')
+        self.vm.add_args('-object',

'{"qom-type":"rng-random","id":"objrng0"'

tests/avocado/tuxrun_baselines.py +2 -0
+                         ',"filename":"/dev/urandom"}',
+                         '-device',

'{"driver":"virtio-rng-pci","rng":"objrng0"'

tests/avocado/tuxrun_baselines.py +2 -0
+                         ',"id":"rng0","bus":"pci.0","addr":"0x8"}')
+        self.vm.add_args('-object',

'{"qom-type":"cryptodev-backend-builtin",'

tests/avocado/tuxrun_baselines.py +25 -1
+                         '"id":"objcrypto0","queues":1}',
+                         '-device', '{"driver":"virtio-crypto-pci",'
+                         '"cryptodev":"objcrypto0","id":"crypto0","bus"'
+                         ':"pci.0","addr":"0xa"}')
+        self.vm.add_args('-device', '{"driver":"spapr-pci-host-bridge"'
+                         ',"index":1,"id":"pci.1"}')
+        self.vm.add_args('-device', '{"driver":"spapr-vscsi","id":"scsi1"'
+                         ',"reg":12288}')
+        self.vm.add_args('-m', '2G,slots=32,maxmem=4G',
+                         '-object', 'memory-backend-ram,id=ram1,size=1G',
+                         '-device', 'pc-dimm,id=dimm1,memdev=ram1')
+
+        # Create a temporary qcow2 and launch the test-case
+        with tempfile.NamedTemporaryFile(prefix='tuxrun_ppc64le_',
+                                         suffix='.qcow2') as qcow2:
+            process.run(self.qemu_img + ' create -f qcow2 ' +
+                        qcow2.name + ' 1G')
+
+            self.vm.add_args('-drive', 'file=' + qcow2.name +
+                         ',format=qcow2,if=none,id='
+                         'drive-virtio-disk1',
+                         '-device', 'virtio-blk-pci,scsi=off,bus=pci.0,'
+
 'addr=0xb,drive=drive-virtio-disk1,id=virtio-disk1'
+                         ',bootindex=2')
+            self.common_tuxrun(drive="scsi-hd")
 
     def test_riscv32(self):
         """
--

2.25.1

#5 lore raw

Re: [PATCH v3 2/2] tests/avocado/tuxrun_baselines.py: improve code coverage for ppc64

Quoted 115 lines · Kautuk Consul

Hi,

Quoted 104 lines · Harsh Prateek Bora

Since we are optimising code a lot, one suggestion below:

Quoted 98 lines · Kautuk Consul

Commit c0c8687ef0fd990db8db1655a8a6c5a5e35dd4bb disabled the boot_linux.py test-case due to which the code coverage for ppc decreased by around 2%. As per the discussion on
https://lore.kernel.org/qemu-devel/[email protected]/ it was mentioned that the baseline test for ppc64 could be modified to make up this 2% code coverage. This patch attempts to achieve this 2% code coverage by adding various device command line arguments (to ./qemu-system-ppc64) in the tuxrun_baselines.py test-case.

The code coverage report with boot_linux.py, without it and finally with these tuxrun_baselines.py changes is as follows:

With boot_linux.py
------------------
lines......: 13.8% (58006 of 420997 lines) functions..: 20.7% (7675 of 36993 functions) branches...: 9.2% (22146 of 240611 branches)
Without boot_linux.py (without this patch changes)
--------------------------------------------------
lines......: 11.9% (50174 of 420997 lines) functions..: 18.8% (6947 of 36993 functions) branches...: 7.4% (17580 of 239017 branches)
Without boot_linux.py (with this patch changes)
-----------------------------------------------
lines......: 13.8% (58287 of 420997 lines) functions..: 20.7% (7640 of 36993 functions) branches...: 8.4% (20223 of 240611 branches)

---
  tests/avocado/tuxrun_baselines.py | 120 +++++++++++++++++++++++++++++-
  1 file changed, 116 insertions(+), 4 deletions(-)
tests/avocado/tuxrun_baselines.py +10 -2
index d343376faa..ae082ac028 100644
@@ -11,6 +11,7 @@
 
  import os
  import time
+import tempfile
 
<snip>
          """
-        self.common_tuxrun(drive="scsi-hd")
+
+        # add device args to command line.
+        self.vm.add_args('-netdev', 'user,id=vnet,hostfwd=:127.0.0.1:0
-:22',
+                         '-device', 'virtio-net,netdev=vnet')
+        self.vm.add_args('-netdev', '{"type":"user","id":"hostnet0"}',
+                         '-device', '{"driver":"virtio-net-pci","netdev":'
+
  '"hostnet0","id":"net0","mac":"52:54:00:4c:e3:86",'
+                         '"bus":"pci.0","addr":"0x9"}')
+        self.vm.add_args('-device',

'{"driver":"qemu-xhci","p2":15,"p3":15,'

tests/avocado/tuxrun_baselines.py +2 -0
+                         '"id":"usb","bus":"pci.0","addr":"0x2"}')
+        self.vm.add_args('-device',

'{"driver":"virtio-scsi-pci","id":"scsi0"'

tests/avocado/tuxrun_baselines.py +16 -0
+                         ',"bus":"pci.0","addr":"0x3"}')
+        self.vm.add_args('-device', '{"driver":"virtio-serial-pci","id":'
+                         '"virtio-serial0","bus":"pci.0","addr":"0x4"}')
+        self.vm.add_args('-device', '{"driver":"scsi-cd","bus":"scsi0.0"'
+                         ',"channel":0,"scsi-id":0,"lun":0,"device_id":'
+                         '"drive-scsi0-0-0-0","id":"scsi0-0-0-0"}')
+        self.vm.add_args('-device', '{"driver":"virtio-balloon-pci",'
+                         '"id":"balloon0","bus":"pci.0","addr":"0x6"}')
+        self.vm.add_args('-audiodev', '{"id":"audio1","driver":"none"}')
+        self.vm.add_args('-device', '{"driver":"usb-tablet","id":"input0"'
+                         ',"bus":"usb.0","port":"1"}')
+        self.vm.add_args('-device', '{"driver":"usb-kbd","id":"input1"'
+                         ',"bus":"usb.0","port":"2"}')
+        self.vm.add_args('-device', '{"driver":"VGA","id":"video0",'
+                         '"vgamem_mb":16,"bus":"pci.0","addr":"0x7"}')
+        self.vm.add_args('-object',

'{"qom-type":"rng-random","id":"objrng0"'

tests/avocado/tuxrun_baselines.py +2 -0
+                         ',"filename":"/dev/urandom"}',
+                         '-device',

'{"driver":"virtio-rng-pci","rng":"objrng0"'

tests/avocado/tuxrun_baselines.py +2 -0
+                         ',"id":"rng0","bus":"pci.0","addr":"0x8"}')
+        self.vm.add_args('-object',

'{"qom-type":"cryptodev-backend-builtin",'

tests/avocado/tuxrun_baselines.py +11 -0
+                         '"id":"objcrypto0","queues":1}',
+                         '-device', '{"driver":"virtio-crypto-pci",'
+                         '"cryptodev":"objcrypto0","id":"crypto0","bus"'
+                         ':"pci.0","addr":"0xa"}')
+        self.vm.add_args('-device', '{"driver":"spapr-pci-host-bridge"'
+                         ',"index":1,"id":"pci.1"}')
+        self.vm.add_args('-device', '{"driver":"spapr-vscsi","id":"scsi1"'
+                         ',"reg":12288}')
+        self.vm.add_args('-m', '2G,slots=32,maxmem=4G',
+                         '-object', 'memory-backend-ram,id=ram1,size=1G',
+                         '-device', 'pc-dimm,id=dimm1,memdev=ram1')
Quoted 8 lines
Quoted 2 lines

+

Most of the above code for adding args is getting duplicated in the test for ppc64le below as well.
It could be contained in a helper routine which could be called from both, and would be easier to maintain.

Quoted 5 lines

I kept it separate because there may be some future reason to make the devices different for ppc and ppc64le. I know that probably won't happen but thought to keep it separate to allow for that possibility. Or if you feel that they should be exactly the same maybe I could send another patch after this to refactor the code into a single routine ?

We can keep a single routine if all the devices are exactly same for now. Whenever (or if now) there is a change in device needed, that can be conveyed to that routine via a flag for be/le and the routine can add additional devices based on the flag passed.

Quoted 103 lines
Quoted 103 lines

regards,
Harsh

+ # Create a temporary qcow2 and launch the test-case

Quoted 98 lines
tests/avocado/tuxrun_baselines.py +21 -3
+        with tempfile.NamedTemporaryFile(prefix='tuxrun_ppc64_',
+                                         suffix='.qcow2') as qcow2:
+            process.run(self.qemu_img + ' create -f qcow2 ' +
+                        qcow2.name + ' 1G')
+
+            self.vm.add_args('-drive', 'file=' + qcow2.name +
+                         ',format=qcow2,if=none,id='
+                         'drive-virtio-disk1',
+                         '-device', 'virtio-blk-pci,scsi=off,bus=pci.0,'
+
  'addr=0xb,drive=drive-virtio-disk1,id=virtio-disk1'
+                         ',bootindex=2')
+            self.common_tuxrun(drive="scsi-hd")
 
      def test_ppc64le(self):
          """
          :avocado: tags=arch:ppc64
          :avocado: tags=machine:pseries
-        :avocado: tags=cpu:POWER8
+        :avocado: tags=cpu:POWER10
          :avocado: tags=console:hvc0
          :avocado: tags=tuxboot:ppc64le
          :avocado: tags=image:vmlinux
          :avocado: tags=extradev:driver=spapr-vscsi
          :avocado: tags=root:sda
          """
-        self.common_tuxrun(drive="scsi-hd")
+        # add device args to command line.
+        self.vm.add_args('-netdev', 'user,id=vnet,hostfwd=:127.0.0.1:0
-:22',
+                         '-device', 'virtio-net,netdev=vnet')
+        self.vm.add_args('-netdev', '{"type":"user","id":"hostnet0"}',
+                         '-device', '{"driver":"virtio-net-pci","netdev":'
+
  '"hostnet0","id":"net0","mac":"52:54:00:4c:e3:86",'
+                         '"bus":"pci.0","addr":"0x9"}')
+        self.vm.add_args('-device',

'{"driver":"qemu-xhci","p2":15,"p3":15,'

tests/avocado/tuxrun_baselines.py +2 -0
+                         '"id":"usb","bus":"pci.0","addr":"0x2"}')
+        self.vm.add_args('-device',

'{"driver":"virtio-scsi-pci","id":"scsi0"'

tests/avocado/tuxrun_baselines.py +16 -0
+                         ',"bus":"pci.0","addr":"0x3"}')
+        self.vm.add_args('-device', '{"driver":"virtio-serial-pci","id":'
+                         '"virtio-serial0","bus":"pci.0","addr":"0x4"}')
+        self.vm.add_args('-device', '{"driver":"scsi-cd","bus":"scsi0.0"'
+                         ',"channel":0,"scsi-id":0,"lun":0,"device_id":'
+                         '"drive-scsi0-0-0-0","id":"scsi0-0-0-0"}')
+        self.vm.add_args('-device', '{"driver":"virtio-balloon-pci",'
+                         '"id":"balloon0","bus":"pci.0","addr":"0x6"}')
+        self.vm.add_args('-audiodev', '{"id":"audio1","driver":"none"}')
+        self.vm.add_args('-device', '{"driver":"usb-tablet","id":"input0"'
+                         ',"bus":"usb.0","port":"1"}')
+        self.vm.add_args('-device', '{"driver":"usb-kbd","id":"input1"'
+                         ',"bus":"usb.0","port":"2"}')
+        self.vm.add_args('-device', '{"driver":"VGA","id":"video0",'
+                         '"vgamem_mb":16,"bus":"pci.0","addr":"0x7"}')
+        self.vm.add_args('-object',

'{"qom-type":"rng-random","id":"objrng0"'

tests/avocado/tuxrun_baselines.py +2 -0
+                         ',"filename":"/dev/urandom"}',
+                         '-device',

'{"driver":"virtio-rng-pci","rng":"objrng0"'

tests/avocado/tuxrun_baselines.py +2 -0
+                         ',"id":"rng0","bus":"pci.0","addr":"0x8"}')
+        self.vm.add_args('-object',

'{"qom-type":"cryptodev-backend-builtin",'

tests/avocado/tuxrun_baselines.py +25 -1
+                         '"id":"objcrypto0","queues":1}',
+                         '-device', '{"driver":"virtio-crypto-pci",'
+                         '"cryptodev":"objcrypto0","id":"crypto0","bus"'
+                         ':"pci.0","addr":"0xa"}')
+        self.vm.add_args('-device', '{"driver":"spapr-pci-host-bridge"'
+                         ',"index":1,"id":"pci.1"}')
+        self.vm.add_args('-device', '{"driver":"spapr-vscsi","id":"scsi1"'
+                         ',"reg":12288}')
+        self.vm.add_args('-m', '2G,slots=32,maxmem=4G',
+                         '-object', 'memory-backend-ram,id=ram1,size=1G',
+                         '-device', 'pc-dimm,id=dimm1,memdev=ram1')
+
+        # Create a temporary qcow2 and launch the test-case
+        with tempfile.NamedTemporaryFile(prefix='tuxrun_ppc64le_',
+                                         suffix='.qcow2') as qcow2:
+            process.run(self.qemu_img + ' create -f qcow2 ' +
+                        qcow2.name + ' 1G')
+
+            self.vm.add_args('-drive', 'file=' + qcow2.name +
+                         ',format=qcow2,if=none,id='
+                         'drive-virtio-disk1',
+                         '-device', 'virtio-blk-pci,scsi=off,bus=pci.0,'
+
  'addr=0xb,drive=drive-virtio-disk1,id=virtio-disk1'
+                         ',bootindex=2')
+            self.common_tuxrun(drive="scsi-hd")
 
      def test_riscv32(self):
          """
--

2.25.1

#6 lore raw

Re: [PATCH v3 2/2] tests/avocado/tuxrun_baselines.py: improve code coverage for ppc64

Quoted 124 lines · Harsh Prateek Bora
Quoted 115 lines · Kautuk Consul

Hi,

Quoted 104 lines · Harsh Prateek Bora

Since we are optimising code a lot, one suggestion below:

Quoted 98 lines · Kautuk Consul

Commit c0c8687ef0fd990db8db1655a8a6c5a5e35dd4bb disabled the boot_linux.py test-case due to which the code coverage for ppc decreased by around 2%. As per the discussion on
https://lore.kernel.org/qemu-devel/[email protected]/ it was mentioned that the baseline test for ppc64 could be modified to make up this 2% code coverage. This patch attempts to achieve this 2% code coverage by adding various device command line arguments (to ./qemu-system-ppc64) in the tuxrun_baselines.py test-case.

The code coverage report with boot_linux.py, without it and finally with these tuxrun_baselines.py changes is as follows:

With boot_linux.py
------------------
lines......: 13.8% (58006 of 420997 lines) functions..: 20.7% (7675 of 36993 functions) branches...: 9.2% (22146 of 240611 branches)
Without boot_linux.py (without this patch changes)
--------------------------------------------------
lines......: 11.9% (50174 of 420997 lines) functions..: 18.8% (6947 of 36993 functions) branches...: 7.4% (17580 of 239017 branches)
Without boot_linux.py (with this patch changes)
-----------------------------------------------
lines......: 13.8% (58287 of 420997 lines) functions..: 20.7% (7640 of 36993 functions) branches...: 8.4% (20223 of 240611 branches)

---
  tests/avocado/tuxrun_baselines.py | 120 +++++++++++++++++++++++++++++-
  1 file changed, 116 insertions(+), 4 deletions(-)
tests/avocado/tuxrun_baselines.py +10 -2
index d343376faa..ae082ac028 100644
@@ -11,6 +11,7 @@
 
  import os
  import time
+import tempfile
 
<snip>
          """
-        self.common_tuxrun(drive="scsi-hd")
+
+        # add device args to command line.
+        self.vm.add_args('-netdev', 'user,id=vnet,hostfwd=:127.0.0.1:0
-:22',
+                         '-device', 'virtio-net,netdev=vnet')
+        self.vm.add_args('-netdev', '{"type":"user","id":"hostnet0"}',
+                         '-device', '{"driver":"virtio-net-pci","netdev":'
+
  '"hostnet0","id":"net0","mac":"52:54:00:4c:e3:86",'
+                         '"bus":"pci.0","addr":"0x9"}')
+        self.vm.add_args('-device',

'{"driver":"qemu-xhci","p2":15,"p3":15,'

tests/avocado/tuxrun_baselines.py +2 -0
+                         '"id":"usb","bus":"pci.0","addr":"0x2"}')
+        self.vm.add_args('-device',

'{"driver":"virtio-scsi-pci","id":"scsi0"'

tests/avocado/tuxrun_baselines.py +16 -0
+                         ',"bus":"pci.0","addr":"0x3"}')
+        self.vm.add_args('-device', '{"driver":"virtio-serial-pci","id":'
+                         '"virtio-serial0","bus":"pci.0","addr":"0x4"}')
+        self.vm.add_args('-device', '{"driver":"scsi-cd","bus":"scsi0.0"'
+                         ',"channel":0,"scsi-id":0,"lun":0,"device_id":'
+                         '"drive-scsi0-0-0-0","id":"scsi0-0-0-0"}')
+        self.vm.add_args('-device', '{"driver":"virtio-balloon-pci",'
+                         '"id":"balloon0","bus":"pci.0","addr":"0x6"}')
+        self.vm.add_args('-audiodev', '{"id":"audio1","driver":"none"}')
+        self.vm.add_args('-device', '{"driver":"usb-tablet","id":"input0"'
+                         ',"bus":"usb.0","port":"1"}')
+        self.vm.add_args('-device', '{"driver":"usb-kbd","id":"input1"'
+                         ',"bus":"usb.0","port":"2"}')
+        self.vm.add_args('-device', '{"driver":"VGA","id":"video0",'
+                         '"vgamem_mb":16,"bus":"pci.0","addr":"0x7"}')
+        self.vm.add_args('-object',

'{"qom-type":"rng-random","id":"objrng0"'

tests/avocado/tuxrun_baselines.py +2 -0
+                         ',"filename":"/dev/urandom"}',
+                         '-device',

'{"driver":"virtio-rng-pci","rng":"objrng0"'

tests/avocado/tuxrun_baselines.py +2 -0
+                         ',"id":"rng0","bus":"pci.0","addr":"0x8"}')
+        self.vm.add_args('-object',

'{"qom-type":"cryptodev-backend-builtin",'

tests/avocado/tuxrun_baselines.py +11 -0
+                         '"id":"objcrypto0","queues":1}',
+                         '-device', '{"driver":"virtio-crypto-pci",'
+                         '"cryptodev":"objcrypto0","id":"crypto0","bus"'
+                         ':"pci.0","addr":"0xa"}')
+        self.vm.add_args('-device', '{"driver":"spapr-pci-host-bridge"'
+                         ',"index":1,"id":"pci.1"}')
+        self.vm.add_args('-device', '{"driver":"spapr-vscsi","id":"scsi1"'
+                         ',"reg":12288}')
+        self.vm.add_args('-m', '2G,slots=32,maxmem=4G',
+                         '-object', 'memory-backend-ram,id=ram1,size=1G',
+                         '-device', 'pc-dimm,id=dimm1,memdev=ram1')
Quoted 8 lines
Quoted 2 lines

+

Most of the above code for adding args is getting duplicated in the test for ppc64le below as well.
It could be contained in a helper routine which could be called from both, and would be easier to maintain.

Quoted 5 lines

I kept it separate because there may be some future reason to make the devices different for ppc and ppc64le. I know that probably won't happen but thought to keep it separate to allow for that possibility. Or if you feel that they should be exactly the same maybe I could send another patch after this to refactor the code into a single routine ?

Quoted 4 lines

We can keep a single routine if all the devices are exactly same for now. Whenever (or if now) there is a change in device needed, that can be conveyed to that routine via a flag for be/le and the routine can add additional devices based on the flag passed.

Sure. Made this change and sent a v4.

Quoted 105 lines
Quoted 104 lines
Quoted 104 lines

regards,
Harsh

+ # Create a temporary qcow2 and launch the test-case

Quoted 98 lines
tests/avocado/tuxrun_baselines.py +21 -3
+        with tempfile.NamedTemporaryFile(prefix='tuxrun_ppc64_',
+                                         suffix='.qcow2') as qcow2:
+            process.run(self.qemu_img + ' create -f qcow2 ' +
+                        qcow2.name + ' 1G')
+
+            self.vm.add_args('-drive', 'file=' + qcow2.name +
+                         ',format=qcow2,if=none,id='
+                         'drive-virtio-disk1',
+                         '-device', 'virtio-blk-pci,scsi=off,bus=pci.0,'
+
  'addr=0xb,drive=drive-virtio-disk1,id=virtio-disk1'
+                         ',bootindex=2')
+            self.common_tuxrun(drive="scsi-hd")
 
      def test_ppc64le(self):
          """
          :avocado: tags=arch:ppc64
          :avocado: tags=machine:pseries
-        :avocado: tags=cpu:POWER8
+        :avocado: tags=cpu:POWER10
          :avocado: tags=console:hvc0
          :avocado: tags=tuxboot:ppc64le
          :avocado: tags=image:vmlinux
          :avocado: tags=extradev:driver=spapr-vscsi
          :avocado: tags=root:sda
          """
-        self.common_tuxrun(drive="scsi-hd")
+        # add device args to command line.
+        self.vm.add_args('-netdev', 'user,id=vnet,hostfwd=:127.0.0.1:0
-:22',
+                         '-device', 'virtio-net,netdev=vnet')
+        self.vm.add_args('-netdev', '{"type":"user","id":"hostnet0"}',
+                         '-device', '{"driver":"virtio-net-pci","netdev":'
+
  '"hostnet0","id":"net0","mac":"52:54:00:4c:e3:86",'
+                         '"bus":"pci.0","addr":"0x9"}')
+        self.vm.add_args('-device',

'{"driver":"qemu-xhci","p2":15,"p3":15,'

tests/avocado/tuxrun_baselines.py +2 -0
+                         '"id":"usb","bus":"pci.0","addr":"0x2"}')
+        self.vm.add_args('-device',

'{"driver":"virtio-scsi-pci","id":"scsi0"'

tests/avocado/tuxrun_baselines.py +16 -0
+                         ',"bus":"pci.0","addr":"0x3"}')
+        self.vm.add_args('-device', '{"driver":"virtio-serial-pci","id":'
+                         '"virtio-serial0","bus":"pci.0","addr":"0x4"}')
+        self.vm.add_args('-device', '{"driver":"scsi-cd","bus":"scsi0.0"'
+                         ',"channel":0,"scsi-id":0,"lun":0,"device_id":'
+                         '"drive-scsi0-0-0-0","id":"scsi0-0-0-0"}')
+        self.vm.add_args('-device', '{"driver":"virtio-balloon-pci",'
+                         '"id":"balloon0","bus":"pci.0","addr":"0x6"}')
+        self.vm.add_args('-audiodev', '{"id":"audio1","driver":"none"}')
+        self.vm.add_args('-device', '{"driver":"usb-tablet","id":"input0"'
+                         ',"bus":"usb.0","port":"1"}')
+        self.vm.add_args('-device', '{"driver":"usb-kbd","id":"input1"'
+                         ',"bus":"usb.0","port":"2"}')
+        self.vm.add_args('-device', '{"driver":"VGA","id":"video0",'
+                         '"vgamem_mb":16,"bus":"pci.0","addr":"0x7"}')
+        self.vm.add_args('-object',

'{"qom-type":"rng-random","id":"objrng0"'

tests/avocado/tuxrun_baselines.py +2 -0
+                         ',"filename":"/dev/urandom"}',
+                         '-device',

'{"driver":"virtio-rng-pci","rng":"objrng0"'

tests/avocado/tuxrun_baselines.py +2 -0
+                         ',"id":"rng0","bus":"pci.0","addr":"0x8"}')
+        self.vm.add_args('-object',

'{"qom-type":"cryptodev-backend-builtin",'

tests/avocado/tuxrun_baselines.py +25 -1
+                         '"id":"objcrypto0","queues":1}',
+                         '-device', '{"driver":"virtio-crypto-pci",'
+                         '"cryptodev":"objcrypto0","id":"crypto0","bus"'
+                         ':"pci.0","addr":"0xa"}')
+        self.vm.add_args('-device', '{"driver":"spapr-pci-host-bridge"'
+                         ',"index":1,"id":"pci.1"}')
+        self.vm.add_args('-device', '{"driver":"spapr-vscsi","id":"scsi1"'
+                         ',"reg":12288}')
+        self.vm.add_args('-m', '2G,slots=32,maxmem=4G',
+                         '-object', 'memory-backend-ram,id=ram1,size=1G',
+                         '-device', 'pc-dimm,id=dimm1,memdev=ram1')
+
+        # Create a temporary qcow2 and launch the test-case
+        with tempfile.NamedTemporaryFile(prefix='tuxrun_ppc64le_',
+                                         suffix='.qcow2') as qcow2:
+            process.run(self.qemu_img + ' create -f qcow2 ' +
+                        qcow2.name + ' 1G')
+
+            self.vm.add_args('-drive', 'file=' + qcow2.name +
+                         ',format=qcow2,if=none,id='
+                         'drive-virtio-disk1',
+                         '-device', 'virtio-blk-pci,scsi=off,bus=pci.0,'
+
  'addr=0xb,drive=drive-virtio-disk1,id=virtio-disk1'
+                         ',bootindex=2')
+            self.common_tuxrun(drive="scsi-hd")
 
      def test_riscv32(self):
          """
--

2.25.1

Reviewed-by on v3 2/2
#7 lore raw

Re: [PATCH v3 2/2] tests/avocado/tuxrun_baselines.py: improve code coverage for ppc64

In reply to Kautuk Consul
Quoted 39 lines · Kautuk Consul

Commit c0c8687ef0fd990db8db1655a8a6c5a5e35dd4bb disabled the boot_linux.py test-case due to which the code coverage for ppc decreased by around 2%. As per the discussion on
https://lore.kernel.org/qemu-devel/[email protected]/ it was mentioned that the baseline test for ppc64 could be modified to make up this 2% code coverage. This patch attempts to achieve this 2% code coverage by adding various device command line arguments (to ./qemu-system-ppc64) in the tuxrun_baselines.py test-case.

The code coverage report with boot_linux.py, without it and finally with these tuxrun_baselines.py changes is as follows:

With boot_linux.py
------------------
lines......: 13.8% (58006 of 420997 lines) functions..: 20.7% (7675 of 36993 functions) branches...: 9.2% (22146 of 240611 branches)
Without boot_linux.py (without this patch changes)
--------------------------------------------------
lines......: 11.9% (50174 of 420997 lines) functions..: 18.8% (6947 of 36993 functions) branches...: 7.4% (17580 of 239017 branches)
Without boot_linux.py (with this patch changes)
-----------------------------------------------
lines......: 13.8% (58287 of 420997 lines) functions..: 20.7% (7640 of 36993 functions) branches...: 8.4% (20223 of 240611 branches)

---
  tests/avocado/tuxrun_baselines.py | 120 +++++++++++++++++++++++++++++-
  1 file changed, 116 insertions(+), 4 deletions(-)
tests/avocado/tuxrun_baselines.py +3 -0
+        # Create a temporary qcow2 and launch the test-case
+        with tempfile.NamedTemporaryFile(prefix='tuxrun_ppc64le_',
+                                         suffix='.qcow2') as qcow2:

tests/avocado/tuxrun_baselines.py def test_ppc64(self): · line 444

Quoted 10 lines
tests/avocado/tuxrun_baselines.py +10 -0
+            process.run(self.qemu_img + ' create -f qcow2 ' +
+                        qcow2.name + ' 1G')
+
+            self.vm.add_args('-drive', 'file=' + qcow2.name +
+                         ',format=qcow2,if=none,id='
+                         'drive-virtio-disk1',
+                         '-device', 'virtio-blk-pci,scsi=off,bus=pci.0,'
+                         'addr=0xb,drive=drive-virtio-disk1,id=virtio-disk1'
+                         ',bootindex=2')
+            self.common_tuxrun(drive="scsi-hd")

Nicer, thanks!

#8 lore raw
In reply to Kautuk Consul
Quoted 4 lines · Kautuk Consul

Commit c0c8687ef0fd990db8db1655a8a6c5a5e35dd4bb disabled the boot_linux.py test-case due to which the code coverage for powerpc decreased by 2%. This patchset aims to make up this 2% code coverage by modifying the tuxrun_baselines.py test-case.

BTW did you see we have a basic tool to compare runs now in scripts/coverage/compare_gcov_json.py if you want to see exactly what the differences between a boot_linux and tuxrun_baseline test is.

Currently its only file granularity but it could be fixed up to do a more detailed line by line breakdown.

Quoted 15 lines

Changes since v1 and v2:
- Modified the way and code that creates the temporary qcow2 image that the tuxrun_baselines.py is now using.
- Factored out code in avocado_qemu/__init__.py to create the get_qemu_img() function that will now get called from tuxrun_baselines.py.

Kautuk Consul (2):
avocado_qemu/__init__.py: factor out the qemu-img finding tests/avocado/tuxrun_baselines.py: improve code coverage for ppc64

 tests/avocado/avocado_qemu/__init__.py |  27 +++---
 tests/avocado/tuxrun_baselines.py      | 120 ++++++++++++++++++++++++-
 2 files changed, 132 insertions(+), 15 deletions(-)
Signature · 2 lines
Alex Bennée
Virtualisation Tech Lead @ Linaro
#9 lore raw
Quoted 11 lines · Alex Bennée
Quoted 4 lines · Kautuk Consul

Commit c0c8687ef0fd990db8db1655a8a6c5a5e35dd4bb disabled the boot_linux.py test-case due to which the code coverage for powerpc decreased by 2%. This patchset aims to make up this 2% code coverage by modifying the tuxrun_baselines.py test-case.

BTW did you see we have a basic tool to compare runs now in scripts/coverage/compare_gcov_json.py if you want to see exactly what the differences between a boot_linux and tuxrun_baseline test is.

I focused on just getting the lines' and functions' coverage equal to the boot_linux.py test-case. I just added devices to meet the coverage percentage and didn't really focus on imitating the code coverage that boot_linux.py does.

Quoted 3 lines

Currently its only file granularity but it could be fixed up to do a more detailed line by line breakdown.

I am currently getting the following error while trying to execute this script:

root@melvil:/home/kautuk/qemu/build_without# python3 ../scripts/coverage/compare_gcov_json.py 
Traceback (most recent call last):
  File "../scripts/coverage/compare_gcov_json.py", line 39, in <module>
    def load_json(json_file_path: Path, verbose = False) -> dict[str, set[int]]:
TypeError: 'type' object is not subscriptable
root@melvil:/home/kautuk/qemu/build_without#
Quoted 21 lines
Quoted 16 lines

Changes since v1 and v2:
- Modified the way and code that creates the temporary qcow2 image that the tuxrun_baselines.py is now using.
- Factored out code in avocado_qemu/__init__.py to create the get_qemu_img() function that will now get called from tuxrun_baselines.py.

Kautuk Consul (2):
avocado_qemu/__init__.py: factor out the qemu-img finding tests/avocado/tuxrun_baselines.py: improve code coverage for ppc64

 tests/avocado/avocado_qemu/__init__.py |  27 +++---
 tests/avocado/tuxrun_baselines.py      | 120 ++++++++++++++++++++++++-
 2 files changed, 132 insertions(+), 15 deletions(-)
Signature · 2 lines
Alex Bennée
Virtualisation Tech Lead @ Linaro