Improve avocado code coverage for ppc64
v3 cover Improve avocado code coverage for ppc64
Commit c0c8687ef0fd990db8db1655a8a6c5a5e35dd4bb disabled the boot_linux.py test-case due to which the code coverage for powerpc decreased by 2%. This patchset aims to make up this 2% code coverage by modifying the tuxrun_baselines.py test-case.
Changes since v1 and v2:
- Modified the way and code that creates the temporary qcow2 image that the tuxrun_baselines.py is now using.
- Factored out code in avocado_qemu/__init__.py to create the get_qemu_img() function that will now get called from tuxrun_baselines.py.
Kautuk Consul (2):
avocado_qemu/__init__.py: factor out the qemu-img finding tests/avocado/tuxrun_baselines.py: improve code coverage for ppc64
tests/avocado/avocado_qemu/__init__.py | 27 +++--- tests/avocado/tuxrun_baselines.py | 120 ++++++++++++++++++++++++- 2 files changed, 132 insertions(+), 15 deletions(-)
Signature · 1 line
2.25.1
PATCH v3 1/2 avocado_qemu/__init__.py: factor out the qemu-img finding
Factor out the code that finds the qemu-img binary in the QemuSystemTest class and create a new get_qemu_img() function with it. This function will get called also from the new code in tuxrun_baselines.py avocado test-case.
--- tests/avocado/avocado_qemu/__init__.py | 27 +++++++++++++++----------- 1 file changed, 16 insertions(+), 11 deletions(-)
tests/avocado/avocado_qemu/__init__.py +16 -11
@@ -330,6 +330,19 @@ def _new_vm(self, name, *args): vm.add_args(*args) return vm + def get_qemu_img(self): + self.log.debug('Looking for and selecting a qemu-img binary') + + # If qemu-img has been built, use it, otherwise the system wide one + # will be used. + qemu_img = os.path.join(BUILD_DIR, 'qemu-img') + if not os.path.exists(qemu_img): + qemu_img = find_command('qemu-img', False) + if qemu_img is False: + self.cancel('Could not find "qemu-img"') + + return qemu_img + @property def vm(self): return self.get_vm(name='default')
@@ -594,17 +607,9 @@ def set_up_existing_ssh_keys(self): return (ssh_public_key, ssh_private_key) def download_boot(self): - self.log.debug('Looking for and selecting a qemu-img binary to be ' - 'used to create the bootable snapshot image') - # If qemu-img has been built, use it, otherwise the system wide one - # will be used. If none is available, the test will cancel. - qemu_img = os.path.join(BUILD_DIR, 'qemu-img') - if not os.path.exists(qemu_img): - qemu_img = find_command('qemu-img', False) - if qemu_img is False: - self.cancel('Could not find "qemu-img", which is required to ' - 'create the bootable image') - vmimage.QEMU_IMG = qemu_img + # Set the qemu-img binary. + # If none is available, the test will cancel. + vmimage.QEMU_IMG = super().get_qemu_img() self.log.info('Downloading/preparing boot image') # Fedora 31 only provides ppc64le images
Signature · 1 line
2.25.1
PATCH v3 2/2 tests/avocado/tuxrun_baselines.py: improve code coverage for ppc64
In reply to Kautuk ConsulCommit c0c8687ef0fd990db8db1655a8a6c5a5e35dd4bb disabled the boot_linux.py test-case due to which the code coverage for ppc decreased by around 2%. As per the discussion on
https://lore.kernel.org/qemu-devel/[email protected]/ it was mentioned that the baseline test for ppc64 could be modified to make up this 2% code coverage. This patch attempts to achieve this 2% code coverage by adding various device command line arguments (to ./qemu-system-ppc64) in the tuxrun_baselines.py test-case.
The code coverage report with boot_linux.py, without it and finally with these tuxrun_baselines.py changes is as follows:
With boot_linux.py
------------------
lines......: 13.8% (58006 of 420997 lines) functions..: 20.7% (7675 of 36993 functions) branches...: 9.2% (22146 of 240611 branches)
Without boot_linux.py (without this patch changes)
--------------------------------------------------
lines......: 11.9% (50174 of 420997 lines) functions..: 18.8% (6947 of 36993 functions) branches...: 7.4% (17580 of 239017 branches)
Without boot_linux.py (with this patch changes)
-----------------------------------------------
lines......: 13.8% (58287 of 420997 lines) functions..: 20.7% (7640 of 36993 functions) branches...: 8.4% (20223 of 240611 branches)
--- tests/avocado/tuxrun_baselines.py | 120 +++++++++++++++++++++++++++++- 1 file changed, 116 insertions(+), 4 deletions(-)
tests/avocado/tuxrun_baselines.py +116 -4
@@ -11,6 +11,7 @@ import os import time +import tempfile from avocado import skip, skipIf from avocado_qemu import QemuSystemTest
@@ -72,6 +73,8 @@ def setUp(self): # Occasionally we need extra devices to hook things up self.extradev = self.get_tag('extradev') + self.qemu_img = super().get_qemu_img() + def wait_for_console_pattern(self, success_message, vm=None): wait_for_console_pattern(self, success_message, failure_message='Kernel panic - not syncing',
@@ -308,7 +311,7 @@ def test_ppc64(self): """ :avocado: tags=arch:ppc64 :avocado: tags=machine:pseries - :avocado: tags=cpu:POWER8 + :avocado: tags=cpu:POWER10 :avocado: tags=endian:big :avocado: tags=console:hvc0 :avocado: tags=tuxboot:ppc64
@@ -316,20 +319,129 @@ def test_ppc64(self): :avocado: tags=extradev:driver=spapr-vscsi :avocado: tags=root:sda """ - self.common_tuxrun(drive="scsi-hd") + + # add device args to command line. + self.vm.add_args('-netdev', 'user,id=vnet,hostfwd=:127.0.0.1:0-:22', + '-device', 'virtio-net,netdev=vnet') + self.vm.add_args('-netdev', '{"type":"user","id":"hostnet0"}', + '-device', '{"driver":"virtio-net-pci","netdev":' + '"hostnet0","id":"net0","mac":"52:54:00:4c:e3:86",' + '"bus":"pci.0","addr":"0x9"}') + self.vm.add_args('-device', '{"driver":"qemu-xhci","p2":15,"p3":15,' + '"id":"usb","bus":"pci.0","addr":"0x2"}') + self.vm.add_args('-device', '{"driver":"virtio-scsi-pci","id":"scsi0"' + ',"bus":"pci.0","addr":"0x3"}') + self.vm.add_args('-device', '{"driver":"virtio-serial-pci","id":' + '"virtio-serial0","bus":"pci.0","addr":"0x4"}') + self.vm.add_args('-device', '{"driver":"scsi-cd","bus":"scsi0.0"' + ',"channel":0,"scsi-id":0,"lun":0,"device_id":' + '"drive-scsi0-0-0-0","id":"scsi0-0-0-0"}') + self.vm.add_args('-device', '{"driver":"virtio-balloon-pci",' + '"id":"balloon0","bus":"pci.0","addr":"0x6"}') + self.vm.add_args('-audiodev', '{"id":"audio1","driver":"none"}') + self.vm.add_args('-device', '{"driver":"usb-tablet","id":"input0"' + ',"bus":"usb.0","port":"1"}') + self.vm.add_args('-device', '{"driver":"usb-kbd","id":"input1"' + ',"bus":"usb.0","port":"2"}') + self.vm.add_args('-device', '{"driver":"VGA","id":"video0",' + '"vgamem_mb":16,"bus":"pci.0","addr":"0x7"}') + self.vm.add_args('-object', '{"qom-type":"rng-random","id":"objrng0"' + ',"filename":"/dev/urandom"}', + '-device', '{"driver":"virtio-rng-pci","rng":"objrng0"' + ',"id":"rng0","bus":"pci.0","addr":"0x8"}') + self.vm.add_args('-object', '{"qom-type":"cryptodev-backend-builtin",' + '"id":"objcrypto0","queues":1}', + '-device', '{"driver":"virtio-crypto-pci",' + '"cryptodev":"objcrypto0","id":"crypto0","bus"' + ':"pci.0","addr":"0xa"}') + self.vm.add_args('-device', '{"driver":"spapr-pci-host-bridge"' + ',"index":1,"id":"pci.1"}') + self.vm.add_args('-device', '{"driver":"spapr-vscsi","id":"scsi1"' + ',"reg":12288}') + self.vm.add_args('-m', '2G,slots=32,maxmem=4G', + '-object', 'memory-backend-ram,id=ram1,size=1G', + '-device', 'pc-dimm,id=dimm1,memdev=ram1') + + # Create a temporary qcow2 and launch the test-case + with tempfile.NamedTemporaryFile(prefix='tuxrun_ppc64_', + suffix='.qcow2') as qcow2: + process.run(self.qemu_img + ' create -f qcow2 ' + + qcow2.name + ' 1G') + + self.vm.add_args('-drive', 'file=' + qcow2.name + + ',format=qcow2,if=none,id=' + 'drive-virtio-disk1', + '-device', 'virtio-blk-pci,scsi=off,bus=pci.0,' + 'addr=0xb,drive=drive-virtio-disk1,id=virtio-disk1' + ',bootindex=2') + self.common_tuxrun(drive="scsi-hd") def test_ppc64le(self): """ :avocado: tags=arch:ppc64 :avocado: tags=machine:pseries - :avocado: tags=cpu:POWER8 + :avocado: tags=cpu:POWER10 :avocado: tags=console:hvc0 :avocado: tags=tuxboot:ppc64le :avocado: tags=image:vmlinux :avocado: tags=extradev:driver=spapr-vscsi :avocado: tags=root:sda """ - self.common_tuxrun(drive="scsi-hd") + # add device args to command line. + self.vm.add_args('-netdev', 'user,id=vnet,hostfwd=:127.0.0.1:0-:22', + '-device', 'virtio-net,netdev=vnet') + self.vm.add_args('-netdev', '{"type":"user","id":"hostnet0"}', + '-device', '{"driver":"virtio-net-pci","netdev":' + '"hostnet0","id":"net0","mac":"52:54:00:4c:e3:86",' + '"bus":"pci.0","addr":"0x9"}') + self.vm.add_args('-device', '{"driver":"qemu-xhci","p2":15,"p3":15,' + '"id":"usb","bus":"pci.0","addr":"0x2"}') + self.vm.add_args('-device', '{"driver":"virtio-scsi-pci","id":"scsi0"' + ',"bus":"pci.0","addr":"0x3"}') + self.vm.add_args('-device', '{"driver":"virtio-serial-pci","id":' + '"virtio-serial0","bus":"pci.0","addr":"0x4"}') + self.vm.add_args('-device', '{"driver":"scsi-cd","bus":"scsi0.0"' + ',"channel":0,"scsi-id":0,"lun":0,"device_id":' + '"drive-scsi0-0-0-0","id":"scsi0-0-0-0"}') + self.vm.add_args('-device', '{"driver":"virtio-balloon-pci",' + '"id":"balloon0","bus":"pci.0","addr":"0x6"}') + self.vm.add_args('-audiodev', '{"id":"audio1","driver":"none"}') + self.vm.add_args('-device', '{"driver":"usb-tablet","id":"input0"' + ',"bus":"usb.0","port":"1"}') + self.vm.add_args('-device', '{"driver":"usb-kbd","id":"input1"' + ',"bus":"usb.0","port":"2"}') + self.vm.add_args('-device', '{"driver":"VGA","id":"video0",' + '"vgamem_mb":16,"bus":"pci.0","addr":"0x7"}') + self.vm.add_args('-object', '{"qom-type":"rng-random","id":"objrng0"' + ',"filename":"/dev/urandom"}', + '-device', '{"driver":"virtio-rng-pci","rng":"objrng0"' + ',"id":"rng0","bus":"pci.0","addr":"0x8"}') + self.vm.add_args('-object', '{"qom-type":"cryptodev-backend-builtin",' + '"id":"objcrypto0","queues":1}', + '-device', '{"driver":"virtio-crypto-pci",' + '"cryptodev":"objcrypto0","id":"crypto0","bus"' + ':"pci.0","addr":"0xa"}') + self.vm.add_args('-device', '{"driver":"spapr-pci-host-bridge"' + ',"index":1,"id":"pci.1"}') + self.vm.add_args('-device', '{"driver":"spapr-vscsi","id":"scsi1"' + ',"reg":12288}') + self.vm.add_args('-m', '2G,slots=32,maxmem=4G', + '-object', 'memory-backend-ram,id=ram1,size=1G', + '-device', 'pc-dimm,id=dimm1,memdev=ram1') + + # Create a temporary qcow2 and launch the test-case + with tempfile.NamedTemporaryFile(prefix='tuxrun_ppc64le_', + suffix='.qcow2') as qcow2: + process.run(self.qemu_img + ' create -f qcow2 ' + + qcow2.name + ' 1G') + + self.vm.add_args('-drive', 'file=' + qcow2.name + + ',format=qcow2,if=none,id=' + 'drive-virtio-disk1', + '-device', 'virtio-blk-pci,scsi=off,bus=pci.0,' + 'addr=0xb,drive=drive-virtio-disk1,id=virtio-disk1' + ',bootindex=2') + self.common_tuxrun(drive="scsi-hd") def test_riscv32(self): """
Signature · 1 line
2.25.1
Re: [PATCH v3 2/2] tests/avocado/tuxrun_baselines.py: improve code coverage for ppc64
Since we are optimising code a lot, one suggestion below:
Quoted 90 lines · Kautuk Consul
Commit
c0c8687ef0fd990db8db1655a8a6c5a5e35dd4bbdisabled theboot_linux.pytest-case due to which the code coverage for ppc decreased by around 2%. As per the discussion on
https://lore.kernel.org/qemu-devel/[email protected]/ it was mentioned that the baseline test for ppc64 could be modified to make up this 2% code coverage. This patch attempts to achieve this 2% code coverage by adding various device command line arguments (to ./qemu-system-ppc64) in thetuxrun_baselines.pytest-case.The code coverage report with
boot_linux.py, without it and finally with thesetuxrun_baselines.pychanges is as follows:With
boot_linux.py
------------------
lines......: 13.8% (58006 of 420997 lines) functions..: 20.7% (7675 of 36993 functions) branches...: 9.2% (22146 of 240611 branches)
Withoutboot_linux.py(without this patch changes)
--------------------------------------------------
lines......: 11.9% (50174 of 420997 lines) functions..: 18.8% (6947 of 36993 functions) branches...: 7.4% (17580 of 239017 branches)
Withoutboot_linux.py(with this patch changes)
-----------------------------------------------
lines......: 13.8% (58287 of 420997 lines) functions..: 20.7% (7640 of 36993 functions) branches...: 8.4% (20223 of 240611 branches)Signed-off-by Kautuk Consul <[email protected]>Reported-by Alex Bennée <[email protected]>--- tests/avocado/tuxrun_baselines.py | 120 +++++++++++++++++++++++++++++- 1 file changed, 116 insertions(+), 4 deletions(-)tests/avocado/tuxrun_baselines.py +10
-2@@ -11,6 +11,7 @@ import os import time +import tempfile """ - self.common_tuxrun(drive="scsi-hd") + + # add device args to command line. + self.vm.add_args('-netdev', 'user,id=vnet,hostfwd=:127.0.0.1:0 -:22', + '-device', 'virtio-net,netdev=vnet') + self.vm.add_args('-netdev', '{"type":"user","id":"hostnet0"}', + '-device', '{"driver":"virtio-net-pci","netdev":' + '"hostnet0","id":"net0","mac":"52:54:00:4c:e3:86",' + '"bus":"pci.0","addr":"0x9"}') + self.vm.add_args('-device','{"driver":"qemu-xhci","p2":15,"p3":15,'
tests/avocado/tuxrun_baselines.py +2
-0+ '"id":"usb","bus":"pci.0","addr":"0x2"}') + self.vm.add_args('-device','{"driver":"virtio-scsi-pci","id":"scsi0"'
tests/avocado/tuxrun_baselines.py +16
-0+ ',"bus":"pci.0","addr":"0x3"}') + self.vm.add_args('-device', '{"driver":"virtio-serial-pci","id":' + '"virtio-serial0","bus":"pci.0","addr":"0x4"}') + self.vm.add_args('-device', '{"driver":"scsi-cd","bus":"scsi0.0"' + ',"channel":0,"scsi-id":0,"lun":0,"device_id":' + '"drive-scsi0-0-0-0","id":"scsi0-0-0-0"}') + self.vm.add_args('-device', '{"driver":"virtio-balloon-pci",' + '"id":"balloon0","bus":"pci.0","addr":"0x6"}') + self.vm.add_args('-audiodev', '{"id":"audio1","driver":"none"}') + self.vm.add_args('-device', '{"driver":"usb-tablet","id":"input0"' + ',"bus":"usb.0","port":"1"}') + self.vm.add_args('-device', '{"driver":"usb-kbd","id":"input1"' + ',"bus":"usb.0","port":"2"}') + self.vm.add_args('-device', '{"driver":"VGA","id":"video0",' + '"vgamem_mb":16,"bus":"pci.0","addr":"0x7"}') + self.vm.add_args('-object','{"qom-type":"rng-random","id":"objrng0"'
tests/avocado/tuxrun_baselines.py +2
-0+ ',"filename":"/dev/urandom"}', + '-device','{"driver":"virtio-rng-pci","rng":"objrng0"'
tests/avocado/tuxrun_baselines.py +2
-0+ ',"id":"rng0","bus":"pci.0","addr":"0x8"}') + self.vm.add_args('-object','{"qom-type":"cryptodev-backend-builtin",'
tests/avocado/tuxrun_baselines.py +2
-0+ '"id":"objcrypto0","queues":1}', + '-device', '{"driver":"virtio-crypto-pci",'
tests/avocado/tuxrun_baselines.py def test_ppc64(self): · line 364
Quoted 10 lines
tests/avocado/tuxrun_baselines.py +10
-0+ '"cryptodev":"objcrypto0","id":"crypto0","bus"' + ':"pci.0","addr":"0xa"}') + self.vm.add_args('-device', '{"driver":"spapr-pci-host-bridge"' + ',"index":1,"id":"pci.1"}') + self.vm.add_args('-device', '{"driver":"spapr-vscsi","id":"scsi1"' + ',"reg":12288}') + self.vm.add_args('-m', '2G,slots=32,maxmem=4G', + '-object', 'memory-backend-ram,id=ram1,size=1G', + '-device', 'pc-dimm,id=dimm1,memdev=ram1') +
Most of the above code for adding args is getting duplicated in the test for ppc64le below as well.
It could be contained in a helper routine which could be called from both, and would be easier to maintain.
regards,
Harsh
+ # Create a temporary qcow2 and launch the test-case
Quoted 98 lines
tests/avocado/tuxrun_baselines.py +21
-3+ with tempfile.NamedTemporaryFile(prefix='tuxrun_ppc64_', + suffix='.qcow2') as qcow2: + process.run(self.qemu_img + ' create -f qcow2 ' + + qcow2.name + ' 1G') + + self.vm.add_args('-drive', 'file=' + qcow2.name + + ',format=qcow2,if=none,id=' + 'drive-virtio-disk1', + '-device', 'virtio-blk-pci,scsi=off,bus=pci.0,' + 'addr=0xb,drive=drive-virtio-disk1,id=virtio-disk1' + ',bootindex=2') + self.common_tuxrun(drive="scsi-hd") def test_ppc64le(self): """ :avocado: tags=arch:ppc64 :avocado: tags=machine:pseries - :avocado: tags=cpu:POWER8 + :avocado: tags=cpu:POWER10 :avocado: tags=console:hvc0 :avocado: tags=tuxboot:ppc64le :avocado: tags=image:vmlinux :avocado: tags=extradev:driver=spapr-vscsi :avocado: tags=root:sda """ - self.common_tuxrun(drive="scsi-hd") + # add device args to command line. + self.vm.add_args('-netdev', 'user,id=vnet,hostfwd=:127.0.0.1:0 -:22', + '-device', 'virtio-net,netdev=vnet') + self.vm.add_args('-netdev', '{"type":"user","id":"hostnet0"}', + '-device', '{"driver":"virtio-net-pci","netdev":' + '"hostnet0","id":"net0","mac":"52:54:00:4c:e3:86",' + '"bus":"pci.0","addr":"0x9"}') + self.vm.add_args('-device','{"driver":"qemu-xhci","p2":15,"p3":15,'
tests/avocado/tuxrun_baselines.py +2
-0+ '"id":"usb","bus":"pci.0","addr":"0x2"}') + self.vm.add_args('-device','{"driver":"virtio-scsi-pci","id":"scsi0"'
tests/avocado/tuxrun_baselines.py +16
-0+ ',"bus":"pci.0","addr":"0x3"}') + self.vm.add_args('-device', '{"driver":"virtio-serial-pci","id":' + '"virtio-serial0","bus":"pci.0","addr":"0x4"}') + self.vm.add_args('-device', '{"driver":"scsi-cd","bus":"scsi0.0"' + ',"channel":0,"scsi-id":0,"lun":0,"device_id":' + '"drive-scsi0-0-0-0","id":"scsi0-0-0-0"}') + self.vm.add_args('-device', '{"driver":"virtio-balloon-pci",' + '"id":"balloon0","bus":"pci.0","addr":"0x6"}') + self.vm.add_args('-audiodev', '{"id":"audio1","driver":"none"}') + self.vm.add_args('-device', '{"driver":"usb-tablet","id":"input0"' + ',"bus":"usb.0","port":"1"}') + self.vm.add_args('-device', '{"driver":"usb-kbd","id":"input1"' + ',"bus":"usb.0","port":"2"}') + self.vm.add_args('-device', '{"driver":"VGA","id":"video0",' + '"vgamem_mb":16,"bus":"pci.0","addr":"0x7"}') + self.vm.add_args('-object','{"qom-type":"rng-random","id":"objrng0"'
tests/avocado/tuxrun_baselines.py +2
-0+ ',"filename":"/dev/urandom"}', + '-device','{"driver":"virtio-rng-pci","rng":"objrng0"'
tests/avocado/tuxrun_baselines.py +2
-0+ ',"id":"rng0","bus":"pci.0","addr":"0x8"}') + self.vm.add_args('-object','{"qom-type":"cryptodev-backend-builtin",'
tests/avocado/tuxrun_baselines.py +25
-1+ '"id":"objcrypto0","queues":1}', + '-device', '{"driver":"virtio-crypto-pci",' + '"cryptodev":"objcrypto0","id":"crypto0","bus"' + ':"pci.0","addr":"0xa"}') + self.vm.add_args('-device', '{"driver":"spapr-pci-host-bridge"' + ',"index":1,"id":"pci.1"}') + self.vm.add_args('-device', '{"driver":"spapr-vscsi","id":"scsi1"' + ',"reg":12288}') + self.vm.add_args('-m', '2G,slots=32,maxmem=4G', + '-object', 'memory-backend-ram,id=ram1,size=1G', + '-device', 'pc-dimm,id=dimm1,memdev=ram1') + + # Create a temporary qcow2 and launch the test-case + with tempfile.NamedTemporaryFile(prefix='tuxrun_ppc64le_', + suffix='.qcow2') as qcow2: + process.run(self.qemu_img + ' create -f qcow2 ' + + qcow2.name + ' 1G') + + self.vm.add_args('-drive', 'file=' + qcow2.name + + ',format=qcow2,if=none,id=' + 'drive-virtio-disk1', + '-device', 'virtio-blk-pci,scsi=off,bus=pci.0,' + 'addr=0xb,drive=drive-virtio-disk1,id=virtio-disk1' + ',bootindex=2') + self.common_tuxrun(drive="scsi-hd") def test_riscv32(self): """ --2.25.1
Re: [PATCH v3 2/2] tests/avocado/tuxrun_baselines.py: improve code coverage for ppc64
Hi,
Quoted 104 lines · Harsh Prateek Bora
Since we are optimising code a lot, one suggestion below:
Quoted 98 lines · Kautuk Consul
Commit
c0c8687ef0fd990db8db1655a8a6c5a5e35dd4bbdisabled theboot_linux.pytest-case due to which the code coverage for ppc decreased by around 2%. As per the discussion on
https://lore.kernel.org/qemu-devel/[email protected]/ it was mentioned that the baseline test for ppc64 could be modified to make up this 2% code coverage. This patch attempts to achieve this 2% code coverage by adding various device command line arguments (to ./qemu-system-ppc64) in thetuxrun_baselines.pytest-case.The code coverage report with
boot_linux.py, without it and finally with thesetuxrun_baselines.pychanges is as follows:With
boot_linux.py
------------------
lines......: 13.8% (58006 of 420997 lines) functions..: 20.7% (7675 of 36993 functions) branches...: 9.2% (22146 of 240611 branches)
Withoutboot_linux.py(without this patch changes)
--------------------------------------------------
lines......: 11.9% (50174 of 420997 lines) functions..: 18.8% (6947 of 36993 functions) branches...: 7.4% (17580 of 239017 branches)
Withoutboot_linux.py(with this patch changes)
-----------------------------------------------
lines......: 13.8% (58287 of 420997 lines) functions..: 20.7% (7640 of 36993 functions) branches...: 8.4% (20223 of 240611 branches)Signed-off-by Kautuk Consul <[email protected]>Reported-by Alex Bennée <[email protected]>--- tests/avocado/tuxrun_baselines.py | 120 +++++++++++++++++++++++++++++- 1 file changed, 116 insertions(+), 4 deletions(-)tests/avocado/tuxrun_baselines.py +10
-2@@ -11,6 +11,7 @@ import os import time +import tempfile """ - self.common_tuxrun(drive="scsi-hd") + + # add device args to command line. + self.vm.add_args('-netdev', 'user,id=vnet,hostfwd=:127.0.0.1:0 -:22', + '-device', 'virtio-net,netdev=vnet') + self.vm.add_args('-netdev', '{"type":"user","id":"hostnet0"}', + '-device', '{"driver":"virtio-net-pci","netdev":' + '"hostnet0","id":"net0","mac":"52:54:00:4c:e3:86",' + '"bus":"pci.0","addr":"0x9"}') + self.vm.add_args('-device','{"driver":"qemu-xhci","p2":15,"p3":15,'
tests/avocado/tuxrun_baselines.py +2
-0+ '"id":"usb","bus":"pci.0","addr":"0x2"}') + self.vm.add_args('-device','{"driver":"virtio-scsi-pci","id":"scsi0"'
tests/avocado/tuxrun_baselines.py +16
-0+ ',"bus":"pci.0","addr":"0x3"}') + self.vm.add_args('-device', '{"driver":"virtio-serial-pci","id":' + '"virtio-serial0","bus":"pci.0","addr":"0x4"}') + self.vm.add_args('-device', '{"driver":"scsi-cd","bus":"scsi0.0"' + ',"channel":0,"scsi-id":0,"lun":0,"device_id":' + '"drive-scsi0-0-0-0","id":"scsi0-0-0-0"}') + self.vm.add_args('-device', '{"driver":"virtio-balloon-pci",' + '"id":"balloon0","bus":"pci.0","addr":"0x6"}') + self.vm.add_args('-audiodev', '{"id":"audio1","driver":"none"}') + self.vm.add_args('-device', '{"driver":"usb-tablet","id":"input0"' + ',"bus":"usb.0","port":"1"}') + self.vm.add_args('-device', '{"driver":"usb-kbd","id":"input1"' + ',"bus":"usb.0","port":"2"}') + self.vm.add_args('-device', '{"driver":"VGA","id":"video0",' + '"vgamem_mb":16,"bus":"pci.0","addr":"0x7"}') + self.vm.add_args('-object','{"qom-type":"rng-random","id":"objrng0"'
tests/avocado/tuxrun_baselines.py +2
-0+ ',"filename":"/dev/urandom"}', + '-device','{"driver":"virtio-rng-pci","rng":"objrng0"'
tests/avocado/tuxrun_baselines.py +2
-0+ ',"id":"rng0","bus":"pci.0","addr":"0x8"}') + self.vm.add_args('-object','{"qom-type":"cryptodev-backend-builtin",'
tests/avocado/tuxrun_baselines.py +11
-0+ '"id":"objcrypto0","queues":1}', + '-device', '{"driver":"virtio-crypto-pci",' + '"cryptodev":"objcrypto0","id":"crypto0","bus"' + ':"pci.0","addr":"0xa"}') + self.vm.add_args('-device', '{"driver":"spapr-pci-host-bridge"' + ',"index":1,"id":"pci.1"}') + self.vm.add_args('-device', '{"driver":"spapr-vscsi","id":"scsi1"' + ',"reg":12288}') + self.vm.add_args('-m', '2G,slots=32,maxmem=4G', + '-object', 'memory-backend-ram,id=ram1,size=1G', + '-device', 'pc-dimm,id=dimm1,memdev=ram1')
Quoted 8 lines
Quoted 2 lines
+
Most of the above code for adding args is getting duplicated in the test for ppc64le below as well.
It could be contained in a helper routine which could be called from both, and would be easier to maintain.
I kept it separate because there may be some future reason to make the devices different for ppc and ppc64le. I know that probably won't happen but thought to keep it separate to allow for that possibility. Or if you feel that they should be exactly the same maybe I could send another patch after this to refactor the code into a single routine ?
Quoted 103 lines
regards,
Harsh+ # Create a temporary qcow2 and launch the test-case
Quoted 98 lines
tests/avocado/tuxrun_baselines.py +21
-3+ with tempfile.NamedTemporaryFile(prefix='tuxrun_ppc64_', + suffix='.qcow2') as qcow2: + process.run(self.qemu_img + ' create -f qcow2 ' + + qcow2.name + ' 1G') + + self.vm.add_args('-drive', 'file=' + qcow2.name + + ',format=qcow2,if=none,id=' + 'drive-virtio-disk1', + '-device', 'virtio-blk-pci,scsi=off,bus=pci.0,' + 'addr=0xb,drive=drive-virtio-disk1,id=virtio-disk1' + ',bootindex=2') + self.common_tuxrun(drive="scsi-hd") def test_ppc64le(self): """ :avocado: tags=arch:ppc64 :avocado: tags=machine:pseries - :avocado: tags=cpu:POWER8 + :avocado: tags=cpu:POWER10 :avocado: tags=console:hvc0 :avocado: tags=tuxboot:ppc64le :avocado: tags=image:vmlinux :avocado: tags=extradev:driver=spapr-vscsi :avocado: tags=root:sda """ - self.common_tuxrun(drive="scsi-hd") + # add device args to command line. + self.vm.add_args('-netdev', 'user,id=vnet,hostfwd=:127.0.0.1:0 -:22', + '-device', 'virtio-net,netdev=vnet') + self.vm.add_args('-netdev', '{"type":"user","id":"hostnet0"}', + '-device', '{"driver":"virtio-net-pci","netdev":' + '"hostnet0","id":"net0","mac":"52:54:00:4c:e3:86",' + '"bus":"pci.0","addr":"0x9"}') + self.vm.add_args('-device','{"driver":"qemu-xhci","p2":15,"p3":15,'
tests/avocado/tuxrun_baselines.py +2
-0+ '"id":"usb","bus":"pci.0","addr":"0x2"}') + self.vm.add_args('-device','{"driver":"virtio-scsi-pci","id":"scsi0"'
tests/avocado/tuxrun_baselines.py +16
-0+ ',"bus":"pci.0","addr":"0x3"}') + self.vm.add_args('-device', '{"driver":"virtio-serial-pci","id":' + '"virtio-serial0","bus":"pci.0","addr":"0x4"}') + self.vm.add_args('-device', '{"driver":"scsi-cd","bus":"scsi0.0"' + ',"channel":0,"scsi-id":0,"lun":0,"device_id":' + '"drive-scsi0-0-0-0","id":"scsi0-0-0-0"}') + self.vm.add_args('-device', '{"driver":"virtio-balloon-pci",' + '"id":"balloon0","bus":"pci.0","addr":"0x6"}') + self.vm.add_args('-audiodev', '{"id":"audio1","driver":"none"}') + self.vm.add_args('-device', '{"driver":"usb-tablet","id":"input0"' + ',"bus":"usb.0","port":"1"}') + self.vm.add_args('-device', '{"driver":"usb-kbd","id":"input1"' + ',"bus":"usb.0","port":"2"}') + self.vm.add_args('-device', '{"driver":"VGA","id":"video0",' + '"vgamem_mb":16,"bus":"pci.0","addr":"0x7"}') + self.vm.add_args('-object','{"qom-type":"rng-random","id":"objrng0"'
tests/avocado/tuxrun_baselines.py +2
-0+ ',"filename":"/dev/urandom"}', + '-device','{"driver":"virtio-rng-pci","rng":"objrng0"'
tests/avocado/tuxrun_baselines.py +2
-0+ ',"id":"rng0","bus":"pci.0","addr":"0x8"}') + self.vm.add_args('-object','{"qom-type":"cryptodev-backend-builtin",'
tests/avocado/tuxrun_baselines.py +25
-1+ '"id":"objcrypto0","queues":1}', + '-device', '{"driver":"virtio-crypto-pci",' + '"cryptodev":"objcrypto0","id":"crypto0","bus"' + ':"pci.0","addr":"0xa"}') + self.vm.add_args('-device', '{"driver":"spapr-pci-host-bridge"' + ',"index":1,"id":"pci.1"}') + self.vm.add_args('-device', '{"driver":"spapr-vscsi","id":"scsi1"' + ',"reg":12288}') + self.vm.add_args('-m', '2G,slots=32,maxmem=4G', + '-object', 'memory-backend-ram,id=ram1,size=1G', + '-device', 'pc-dimm,id=dimm1,memdev=ram1') + + # Create a temporary qcow2 and launch the test-case + with tempfile.NamedTemporaryFile(prefix='tuxrun_ppc64le_', + suffix='.qcow2') as qcow2: + process.run(self.qemu_img + ' create -f qcow2 ' + + qcow2.name + ' 1G') + + self.vm.add_args('-drive', 'file=' + qcow2.name + + ',format=qcow2,if=none,id=' + 'drive-virtio-disk1', + '-device', 'virtio-blk-pci,scsi=off,bus=pci.0,' + 'addr=0xb,drive=drive-virtio-disk1,id=virtio-disk1' + ',bootindex=2') + self.common_tuxrun(drive="scsi-hd") def test_riscv32(self): """ --2.25.1
Re: [PATCH v3 2/2] tests/avocado/tuxrun_baselines.py: improve code coverage for ppc64
Quoted 115 lines · Kautuk Consul
Hi,
Quoted 104 lines · Harsh Prateek Bora
Since we are optimising code a lot, one suggestion below:
Quoted 98 lines · Kautuk Consul
Commit
c0c8687ef0fd990db8db1655a8a6c5a5e35dd4bbdisabled theboot_linux.pytest-case due to which the code coverage for ppc decreased by around 2%. As per the discussion on
https://lore.kernel.org/qemu-devel/[email protected]/ it was mentioned that the baseline test for ppc64 could be modified to make up this 2% code coverage. This patch attempts to achieve this 2% code coverage by adding various device command line arguments (to ./qemu-system-ppc64) in thetuxrun_baselines.pytest-case.The code coverage report with
boot_linux.py, without it and finally with thesetuxrun_baselines.pychanges is as follows:With
boot_linux.py
------------------
lines......: 13.8% (58006 of 420997 lines) functions..: 20.7% (7675 of 36993 functions) branches...: 9.2% (22146 of 240611 branches)
Withoutboot_linux.py(without this patch changes)
--------------------------------------------------
lines......: 11.9% (50174 of 420997 lines) functions..: 18.8% (6947 of 36993 functions) branches...: 7.4% (17580 of 239017 branches)
Withoutboot_linux.py(with this patch changes)
-----------------------------------------------
lines......: 13.8% (58287 of 420997 lines) functions..: 20.7% (7640 of 36993 functions) branches...: 8.4% (20223 of 240611 branches)Signed-off-by Kautuk Consul <[email protected]>Reported-by Alex Bennée <[email protected]>--- tests/avocado/tuxrun_baselines.py | 120 +++++++++++++++++++++++++++++- 1 file changed, 116 insertions(+), 4 deletions(-)tests/avocado/tuxrun_baselines.py +10
-2@@ -11,6 +11,7 @@ import os import time +import tempfile """ - self.common_tuxrun(drive="scsi-hd") + + # add device args to command line. + self.vm.add_args('-netdev', 'user,id=vnet,hostfwd=:127.0.0.1:0 -:22', + '-device', 'virtio-net,netdev=vnet') + self.vm.add_args('-netdev', '{"type":"user","id":"hostnet0"}', + '-device', '{"driver":"virtio-net-pci","netdev":' + '"hostnet0","id":"net0","mac":"52:54:00:4c:e3:86",' + '"bus":"pci.0","addr":"0x9"}') + self.vm.add_args('-device','{"driver":"qemu-xhci","p2":15,"p3":15,'
tests/avocado/tuxrun_baselines.py +2
-0+ '"id":"usb","bus":"pci.0","addr":"0x2"}') + self.vm.add_args('-device','{"driver":"virtio-scsi-pci","id":"scsi0"'
tests/avocado/tuxrun_baselines.py +16
-0+ ',"bus":"pci.0","addr":"0x3"}') + self.vm.add_args('-device', '{"driver":"virtio-serial-pci","id":' + '"virtio-serial0","bus":"pci.0","addr":"0x4"}') + self.vm.add_args('-device', '{"driver":"scsi-cd","bus":"scsi0.0"' + ',"channel":0,"scsi-id":0,"lun":0,"device_id":' + '"drive-scsi0-0-0-0","id":"scsi0-0-0-0"}') + self.vm.add_args('-device', '{"driver":"virtio-balloon-pci",' + '"id":"balloon0","bus":"pci.0","addr":"0x6"}') + self.vm.add_args('-audiodev', '{"id":"audio1","driver":"none"}') + self.vm.add_args('-device', '{"driver":"usb-tablet","id":"input0"' + ',"bus":"usb.0","port":"1"}') + self.vm.add_args('-device', '{"driver":"usb-kbd","id":"input1"' + ',"bus":"usb.0","port":"2"}') + self.vm.add_args('-device', '{"driver":"VGA","id":"video0",' + '"vgamem_mb":16,"bus":"pci.0","addr":"0x7"}') + self.vm.add_args('-object','{"qom-type":"rng-random","id":"objrng0"'
tests/avocado/tuxrun_baselines.py +2
-0+ ',"filename":"/dev/urandom"}', + '-device','{"driver":"virtio-rng-pci","rng":"objrng0"'
tests/avocado/tuxrun_baselines.py +2
-0+ ',"id":"rng0","bus":"pci.0","addr":"0x8"}') + self.vm.add_args('-object','{"qom-type":"cryptodev-backend-builtin",'
tests/avocado/tuxrun_baselines.py +11
-0+ '"id":"objcrypto0","queues":1}', + '-device', '{"driver":"virtio-crypto-pci",' + '"cryptodev":"objcrypto0","id":"crypto0","bus"' + ':"pci.0","addr":"0xa"}') + self.vm.add_args('-device', '{"driver":"spapr-pci-host-bridge"' + ',"index":1,"id":"pci.1"}') + self.vm.add_args('-device', '{"driver":"spapr-vscsi","id":"scsi1"' + ',"reg":12288}') + self.vm.add_args('-m', '2G,slots=32,maxmem=4G', + '-object', 'memory-backend-ram,id=ram1,size=1G', + '-device', 'pc-dimm,id=dimm1,memdev=ram1')Quoted 8 lines
Quoted 2 lines
+
Most of the above code for adding args is getting duplicated in the test for ppc64le below as well.
It could be contained in a helper routine which could be called from both, and would be easier to maintain.
Quoted 5 lines
I kept it separate because there may be some future reason to make the devices different for ppc and ppc64le. I know that probably won't happen but thought to keep it separate to allow for that possibility. Or if you feel that they should be exactly the same maybe I could send another patch after this to refactor the code into a single routine ?
We can keep a single routine if all the devices are exactly same for now. Whenever (or if now) there is a change in device needed, that can be conveyed to that routine via a flag for be/le and the routine can add additional devices based on the flag passed.
Quoted 103 lines
Quoted 103 lines
regards,
Harsh+ # Create a temporary qcow2 and launch the test-case
Quoted 98 lines
tests/avocado/tuxrun_baselines.py +21
-3+ with tempfile.NamedTemporaryFile(prefix='tuxrun_ppc64_', + suffix='.qcow2') as qcow2: + process.run(self.qemu_img + ' create -f qcow2 ' + + qcow2.name + ' 1G') + + self.vm.add_args('-drive', 'file=' + qcow2.name + + ',format=qcow2,if=none,id=' + 'drive-virtio-disk1', + '-device', 'virtio-blk-pci,scsi=off,bus=pci.0,' + 'addr=0xb,drive=drive-virtio-disk1,id=virtio-disk1' + ',bootindex=2') + self.common_tuxrun(drive="scsi-hd") def test_ppc64le(self): """ :avocado: tags=arch:ppc64 :avocado: tags=machine:pseries - :avocado: tags=cpu:POWER8 + :avocado: tags=cpu:POWER10 :avocado: tags=console:hvc0 :avocado: tags=tuxboot:ppc64le :avocado: tags=image:vmlinux :avocado: tags=extradev:driver=spapr-vscsi :avocado: tags=root:sda """ - self.common_tuxrun(drive="scsi-hd") + # add device args to command line. + self.vm.add_args('-netdev', 'user,id=vnet,hostfwd=:127.0.0.1:0 -:22', + '-device', 'virtio-net,netdev=vnet') + self.vm.add_args('-netdev', '{"type":"user","id":"hostnet0"}', + '-device', '{"driver":"virtio-net-pci","netdev":' + '"hostnet0","id":"net0","mac":"52:54:00:4c:e3:86",' + '"bus":"pci.0","addr":"0x9"}') + self.vm.add_args('-device','{"driver":"qemu-xhci","p2":15,"p3":15,'
tests/avocado/tuxrun_baselines.py +2
-0+ '"id":"usb","bus":"pci.0","addr":"0x2"}') + self.vm.add_args('-device','{"driver":"virtio-scsi-pci","id":"scsi0"'
tests/avocado/tuxrun_baselines.py +16
-0+ ',"bus":"pci.0","addr":"0x3"}') + self.vm.add_args('-device', '{"driver":"virtio-serial-pci","id":' + '"virtio-serial0","bus":"pci.0","addr":"0x4"}') + self.vm.add_args('-device', '{"driver":"scsi-cd","bus":"scsi0.0"' + ',"channel":0,"scsi-id":0,"lun":0,"device_id":' + '"drive-scsi0-0-0-0","id":"scsi0-0-0-0"}') + self.vm.add_args('-device', '{"driver":"virtio-balloon-pci",' + '"id":"balloon0","bus":"pci.0","addr":"0x6"}') + self.vm.add_args('-audiodev', '{"id":"audio1","driver":"none"}') + self.vm.add_args('-device', '{"driver":"usb-tablet","id":"input0"' + ',"bus":"usb.0","port":"1"}') + self.vm.add_args('-device', '{"driver":"usb-kbd","id":"input1"' + ',"bus":"usb.0","port":"2"}') + self.vm.add_args('-device', '{"driver":"VGA","id":"video0",' + '"vgamem_mb":16,"bus":"pci.0","addr":"0x7"}') + self.vm.add_args('-object','{"qom-type":"rng-random","id":"objrng0"'
tests/avocado/tuxrun_baselines.py +2
-0+ ',"filename":"/dev/urandom"}', + '-device','{"driver":"virtio-rng-pci","rng":"objrng0"'
tests/avocado/tuxrun_baselines.py +2
-0+ ',"id":"rng0","bus":"pci.0","addr":"0x8"}') + self.vm.add_args('-object','{"qom-type":"cryptodev-backend-builtin",'
tests/avocado/tuxrun_baselines.py +25
-1+ '"id":"objcrypto0","queues":1}', + '-device', '{"driver":"virtio-crypto-pci",' + '"cryptodev":"objcrypto0","id":"crypto0","bus"' + ':"pci.0","addr":"0xa"}') + self.vm.add_args('-device', '{"driver":"spapr-pci-host-bridge"' + ',"index":1,"id":"pci.1"}') + self.vm.add_args('-device', '{"driver":"spapr-vscsi","id":"scsi1"' + ',"reg":12288}') + self.vm.add_args('-m', '2G,slots=32,maxmem=4G', + '-object', 'memory-backend-ram,id=ram1,size=1G', + '-device', 'pc-dimm,id=dimm1,memdev=ram1') + + # Create a temporary qcow2 and launch the test-case + with tempfile.NamedTemporaryFile(prefix='tuxrun_ppc64le_', + suffix='.qcow2') as qcow2: + process.run(self.qemu_img + ' create -f qcow2 ' + + qcow2.name + ' 1G') + + self.vm.add_args('-drive', 'file=' + qcow2.name + + ',format=qcow2,if=none,id=' + 'drive-virtio-disk1', + '-device', 'virtio-blk-pci,scsi=off,bus=pci.0,' + 'addr=0xb,drive=drive-virtio-disk1,id=virtio-disk1' + ',bootindex=2') + self.common_tuxrun(drive="scsi-hd") def test_riscv32(self): """ --2.25.1
Re: [PATCH v3 2/2] tests/avocado/tuxrun_baselines.py: improve code coverage for ppc64
Quoted 124 lines · Harsh Prateek Bora
Quoted 115 lines · Kautuk Consul
Hi,
Quoted 104 lines · Harsh Prateek Bora
Since we are optimising code a lot, one suggestion below:
Quoted 98 lines · Kautuk Consul
Commit
c0c8687ef0fd990db8db1655a8a6c5a5e35dd4bbdisabled theboot_linux.pytest-case due to which the code coverage for ppc decreased by around 2%. As per the discussion on
https://lore.kernel.org/qemu-devel/[email protected]/ it was mentioned that the baseline test for ppc64 could be modified to make up this 2% code coverage. This patch attempts to achieve this 2% code coverage by adding various device command line arguments (to ./qemu-system-ppc64) in thetuxrun_baselines.pytest-case.The code coverage report with
boot_linux.py, without it and finally with thesetuxrun_baselines.pychanges is as follows:With
boot_linux.py
------------------
lines......: 13.8% (58006 of 420997 lines) functions..: 20.7% (7675 of 36993 functions) branches...: 9.2% (22146 of 240611 branches)
Withoutboot_linux.py(without this patch changes)
--------------------------------------------------
lines......: 11.9% (50174 of 420997 lines) functions..: 18.8% (6947 of 36993 functions) branches...: 7.4% (17580 of 239017 branches)
Withoutboot_linux.py(with this patch changes)
-----------------------------------------------
lines......: 13.8% (58287 of 420997 lines) functions..: 20.7% (7640 of 36993 functions) branches...: 8.4% (20223 of 240611 branches)Signed-off-by Kautuk Consul <[email protected]>Reported-by Alex Bennée <[email protected]>--- tests/avocado/tuxrun_baselines.py | 120 +++++++++++++++++++++++++++++- 1 file changed, 116 insertions(+), 4 deletions(-)tests/avocado/tuxrun_baselines.py +10
-2@@ -11,6 +11,7 @@ import os import time +import tempfile """ - self.common_tuxrun(drive="scsi-hd") + + # add device args to command line. + self.vm.add_args('-netdev', 'user,id=vnet,hostfwd=:127.0.0.1:0 -:22', + '-device', 'virtio-net,netdev=vnet') + self.vm.add_args('-netdev', '{"type":"user","id":"hostnet0"}', + '-device', '{"driver":"virtio-net-pci","netdev":' + '"hostnet0","id":"net0","mac":"52:54:00:4c:e3:86",' + '"bus":"pci.0","addr":"0x9"}') + self.vm.add_args('-device','{"driver":"qemu-xhci","p2":15,"p3":15,'
tests/avocado/tuxrun_baselines.py +2
-0+ '"id":"usb","bus":"pci.0","addr":"0x2"}') + self.vm.add_args('-device','{"driver":"virtio-scsi-pci","id":"scsi0"'
tests/avocado/tuxrun_baselines.py +16
-0+ ',"bus":"pci.0","addr":"0x3"}') + self.vm.add_args('-device', '{"driver":"virtio-serial-pci","id":' + '"virtio-serial0","bus":"pci.0","addr":"0x4"}') + self.vm.add_args('-device', '{"driver":"scsi-cd","bus":"scsi0.0"' + ',"channel":0,"scsi-id":0,"lun":0,"device_id":' + '"drive-scsi0-0-0-0","id":"scsi0-0-0-0"}') + self.vm.add_args('-device', '{"driver":"virtio-balloon-pci",' + '"id":"balloon0","bus":"pci.0","addr":"0x6"}') + self.vm.add_args('-audiodev', '{"id":"audio1","driver":"none"}') + self.vm.add_args('-device', '{"driver":"usb-tablet","id":"input0"' + ',"bus":"usb.0","port":"1"}') + self.vm.add_args('-device', '{"driver":"usb-kbd","id":"input1"' + ',"bus":"usb.0","port":"2"}') + self.vm.add_args('-device', '{"driver":"VGA","id":"video0",' + '"vgamem_mb":16,"bus":"pci.0","addr":"0x7"}') + self.vm.add_args('-object','{"qom-type":"rng-random","id":"objrng0"'
tests/avocado/tuxrun_baselines.py +2
-0+ ',"filename":"/dev/urandom"}', + '-device','{"driver":"virtio-rng-pci","rng":"objrng0"'
tests/avocado/tuxrun_baselines.py +2
-0+ ',"id":"rng0","bus":"pci.0","addr":"0x8"}') + self.vm.add_args('-object','{"qom-type":"cryptodev-backend-builtin",'
tests/avocado/tuxrun_baselines.py +11
-0+ '"id":"objcrypto0","queues":1}', + '-device', '{"driver":"virtio-crypto-pci",' + '"cryptodev":"objcrypto0","id":"crypto0","bus"' + ':"pci.0","addr":"0xa"}') + self.vm.add_args('-device', '{"driver":"spapr-pci-host-bridge"' + ',"index":1,"id":"pci.1"}') + self.vm.add_args('-device', '{"driver":"spapr-vscsi","id":"scsi1"' + ',"reg":12288}') + self.vm.add_args('-m', '2G,slots=32,maxmem=4G', + '-object', 'memory-backend-ram,id=ram1,size=1G', + '-device', 'pc-dimm,id=dimm1,memdev=ram1')Quoted 8 lines
Quoted 2 lines
+
Most of the above code for adding args is getting duplicated in the test for ppc64le below as well.
It could be contained in a helper routine which could be called from both, and would be easier to maintain.Quoted 5 lines
I kept it separate because there may be some future reason to make the devices different for ppc and ppc64le. I know that probably won't happen but thought to keep it separate to allow for that possibility. Or if you feel that they should be exactly the same maybe I could send another patch after this to refactor the code into a single routine ?
Quoted 4 lines
We can keep a single routine if all the devices are exactly same for now. Whenever (or if now) there is a change in device needed, that can be conveyed to that routine via a flag for be/le and the routine can add additional devices based on the flag passed.
Sure. Made this change and sent a v4.
Quoted 105 lines
Quoted 104 lines
Quoted 104 lines
regards,
Harsh+ # Create a temporary qcow2 and launch the test-case
Quoted 98 lines
tests/avocado/tuxrun_baselines.py +21
-3+ with tempfile.NamedTemporaryFile(prefix='tuxrun_ppc64_', + suffix='.qcow2') as qcow2: + process.run(self.qemu_img + ' create -f qcow2 ' + + qcow2.name + ' 1G') + + self.vm.add_args('-drive', 'file=' + qcow2.name + + ',format=qcow2,if=none,id=' + 'drive-virtio-disk1', + '-device', 'virtio-blk-pci,scsi=off,bus=pci.0,' + 'addr=0xb,drive=drive-virtio-disk1,id=virtio-disk1' + ',bootindex=2') + self.common_tuxrun(drive="scsi-hd") def test_ppc64le(self): """ :avocado: tags=arch:ppc64 :avocado: tags=machine:pseries - :avocado: tags=cpu:POWER8 + :avocado: tags=cpu:POWER10 :avocado: tags=console:hvc0 :avocado: tags=tuxboot:ppc64le :avocado: tags=image:vmlinux :avocado: tags=extradev:driver=spapr-vscsi :avocado: tags=root:sda """ - self.common_tuxrun(drive="scsi-hd") + # add device args to command line. + self.vm.add_args('-netdev', 'user,id=vnet,hostfwd=:127.0.0.1:0 -:22', + '-device', 'virtio-net,netdev=vnet') + self.vm.add_args('-netdev', '{"type":"user","id":"hostnet0"}', + '-device', '{"driver":"virtio-net-pci","netdev":' + '"hostnet0","id":"net0","mac":"52:54:00:4c:e3:86",' + '"bus":"pci.0","addr":"0x9"}') + self.vm.add_args('-device','{"driver":"qemu-xhci","p2":15,"p3":15,'
tests/avocado/tuxrun_baselines.py +2
-0+ '"id":"usb","bus":"pci.0","addr":"0x2"}') + self.vm.add_args('-device','{"driver":"virtio-scsi-pci","id":"scsi0"'
tests/avocado/tuxrun_baselines.py +16
-0+ ',"bus":"pci.0","addr":"0x3"}') + self.vm.add_args('-device', '{"driver":"virtio-serial-pci","id":' + '"virtio-serial0","bus":"pci.0","addr":"0x4"}') + self.vm.add_args('-device', '{"driver":"scsi-cd","bus":"scsi0.0"' + ',"channel":0,"scsi-id":0,"lun":0,"device_id":' + '"drive-scsi0-0-0-0","id":"scsi0-0-0-0"}') + self.vm.add_args('-device', '{"driver":"virtio-balloon-pci",' + '"id":"balloon0","bus":"pci.0","addr":"0x6"}') + self.vm.add_args('-audiodev', '{"id":"audio1","driver":"none"}') + self.vm.add_args('-device', '{"driver":"usb-tablet","id":"input0"' + ',"bus":"usb.0","port":"1"}') + self.vm.add_args('-device', '{"driver":"usb-kbd","id":"input1"' + ',"bus":"usb.0","port":"2"}') + self.vm.add_args('-device', '{"driver":"VGA","id":"video0",' + '"vgamem_mb":16,"bus":"pci.0","addr":"0x7"}') + self.vm.add_args('-object','{"qom-type":"rng-random","id":"objrng0"'
tests/avocado/tuxrun_baselines.py +2
-0+ ',"filename":"/dev/urandom"}', + '-device','{"driver":"virtio-rng-pci","rng":"objrng0"'
tests/avocado/tuxrun_baselines.py +2
-0+ ',"id":"rng0","bus":"pci.0","addr":"0x8"}') + self.vm.add_args('-object','{"qom-type":"cryptodev-backend-builtin",'
tests/avocado/tuxrun_baselines.py +25
-1+ '"id":"objcrypto0","queues":1}', + '-device', '{"driver":"virtio-crypto-pci",' + '"cryptodev":"objcrypto0","id":"crypto0","bus"' + ':"pci.0","addr":"0xa"}') + self.vm.add_args('-device', '{"driver":"spapr-pci-host-bridge"' + ',"index":1,"id":"pci.1"}') + self.vm.add_args('-device', '{"driver":"spapr-vscsi","id":"scsi1"' + ',"reg":12288}') + self.vm.add_args('-m', '2G,slots=32,maxmem=4G', + '-object', 'memory-backend-ram,id=ram1,size=1G', + '-device', 'pc-dimm,id=dimm1,memdev=ram1') + + # Create a temporary qcow2 and launch the test-case + with tempfile.NamedTemporaryFile(prefix='tuxrun_ppc64le_', + suffix='.qcow2') as qcow2: + process.run(self.qemu_img + ' create -f qcow2 ' + + qcow2.name + ' 1G') + + self.vm.add_args('-drive', 'file=' + qcow2.name + + ',format=qcow2,if=none,id=' + 'drive-virtio-disk1', + '-device', 'virtio-blk-pci,scsi=off,bus=pci.0,' + 'addr=0xb,drive=drive-virtio-disk1,id=virtio-disk1' + ',bootindex=2') + self.common_tuxrun(drive="scsi-hd") def test_riscv32(self): """ --2.25.1
Re: [PATCH v3 2/2] tests/avocado/tuxrun_baselines.py: improve code coverage for ppc64
In reply to Kautuk ConsulQuoted 39 lines · Kautuk Consul
Commit
c0c8687ef0fd990db8db1655a8a6c5a5e35dd4bbdisabled theboot_linux.pytest-case due to which the code coverage for ppc decreased by around 2%. As per the discussion on
https://lore.kernel.org/qemu-devel/[email protected]/ it was mentioned that the baseline test for ppc64 could be modified to make up this 2% code coverage. This patch attempts to achieve this 2% code coverage by adding various device command line arguments (to ./qemu-system-ppc64) in thetuxrun_baselines.pytest-case.The code coverage report with
boot_linux.py, without it and finally with thesetuxrun_baselines.pychanges is as follows:With
boot_linux.py
------------------
lines......: 13.8% (58006 of 420997 lines) functions..: 20.7% (7675 of 36993 functions) branches...: 9.2% (22146 of 240611 branches)
Withoutboot_linux.py(without this patch changes)
--------------------------------------------------
lines......: 11.9% (50174 of 420997 lines) functions..: 18.8% (6947 of 36993 functions) branches...: 7.4% (17580 of 239017 branches)
Withoutboot_linux.py(with this patch changes)
-----------------------------------------------
lines......: 13.8% (58287 of 420997 lines) functions..: 20.7% (7640 of 36993 functions) branches...: 8.4% (20223 of 240611 branches)Signed-off-by Kautuk Consul <[email protected]>Reported-by Alex Bennée <[email protected]>--- tests/avocado/tuxrun_baselines.py | 120 +++++++++++++++++++++++++++++- 1 file changed, 116 insertions(+), 4 deletions(-)tests/avocado/tuxrun_baselines.py +3
-0+ # Create a temporary qcow2 and launch the test-case + with tempfile.NamedTemporaryFile(prefix='tuxrun_ppc64le_', + suffix='.qcow2') as qcow2:
tests/avocado/tuxrun_baselines.py def test_ppc64(self): · line 444
Quoted 10 lines
tests/avocado/tuxrun_baselines.py +10
-0+ process.run(self.qemu_img + ' create -f qcow2 ' + + qcow2.name + ' 1G') + + self.vm.add_args('-drive', 'file=' + qcow2.name + + ',format=qcow2,if=none,id=' + 'drive-virtio-disk1', + '-device', 'virtio-blk-pci,scsi=off,bus=pci.0,' + 'addr=0xb,drive=drive-virtio-disk1,id=virtio-disk1' + ',bootindex=2') + self.common_tuxrun(drive="scsi-hd")
Nicer, thanks!
Quoted 4 lines · Kautuk Consul
Commit
c0c8687ef0fd990db8db1655a8a6c5a5e35dd4bbdisabled theboot_linux.pytest-case due to which the code coverage for powerpc decreased by 2%. This patchset aims to make up this 2% code coverage by modifying thetuxrun_baselines.pytest-case.
BTW did you see we have a basic tool to compare runs now in scripts/coverage/compare_gcov_json.py if you want to see exactly what the differences between a boot_linux and tuxrun_baseline test is.
Currently its only file granularity but it could be fixed up to do a more detailed line by line breakdown.
Quoted 15 lines
Changes since v1 and v2:
- Modified the way and code that creates the temporary qcow2 image that thetuxrun_baselines.pyis now using.
- Factored out code inavocado_qemu/__init__.pyto create theget_qemu_img()function that will now get called fromtuxrun_baselines.py.Kautuk Consul (2):
avocado_qemu/__init__.py: factor out the qemu-img findingtests/avocado/tuxrun_baselines.py: improve code coverage for ppc64tests/avocado/avocado_qemu/__init__.py | 27 +++--- tests/avocado/tuxrun_baselines.py | 120 ++++++++++++++++++++++++- 2 files changed, 132 insertions(+), 15 deletions(-)
Signature · 2 lines
Alex Bennée Virtualisation Tech Lead @ Linaro
Quoted 11 lines · Alex Bennée
Quoted 4 lines · Kautuk Consul
Commit
c0c8687ef0fd990db8db1655a8a6c5a5e35dd4bbdisabled theboot_linux.pytest-case due to which the code coverage for powerpc decreased by 2%. This patchset aims to make up this 2% code coverage by modifying thetuxrun_baselines.pytest-case.BTW did you see we have a basic tool to compare runs now in
scripts/coverage/compare_gcov_json.pyif you want to see exactly what the differences between aboot_linuxandtuxrun_baselinetest is.
I focused on just getting the lines' and functions' coverage equal to the boot_linux.py test-case. I just added devices to meet the coverage percentage and didn't really focus on imitating the code coverage that boot_linux.py does.
Quoted 3 lines
Currently its only file granularity but it could be fixed up to do a more detailed line by line breakdown.
I am currently getting the following error while trying to execute this script:
root@melvil:/home/kautuk/qemu/build_without# python3 ../scripts/coverage/compare_gcov_json.py
Traceback (most recent call last):
File "../scripts/coverage/compare_gcov_json.py", line 39, in <module>
def load_json(json_file_path: Path, verbose = False) -> dict[str, set[int]]:
TypeError: 'type' object is not subscriptable
root@melvil:/home/kautuk/qemu/build_without#Quoted 21 lines
Quoted 16 lines
Changes since v1 and v2:
- Modified the way and code that creates the temporary qcow2 image that thetuxrun_baselines.pyis now using.
- Factored out code inavocado_qemu/__init__.pyto create theget_qemu_img()function that will now get called fromtuxrun_baselines.py.Kautuk Consul (2):
avocado_qemu/__init__.py: factor out the qemu-img findingtests/avocado/tuxrun_baselines.py: improve code coverage for ppc64tests/avocado/avocado_qemu/__init__.py | 27 +++--- tests/avocado/tuxrun_baselines.py | 120 ++++++++++++++++++++++++- 2 files changed, 132 insertions(+), 15 deletions(-)Signature · 2 lines
Alex Bennée Virtualisation Tech Lead @ Linaro