Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 2B488C77B75 for ; Fri, 21 Apr 2023 03:54:35 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pphqb-0004sS-MD; Thu, 20 Apr 2023 23:54:10 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pphqa-0004s5-ES for qemu-devel@nongnu.org; Thu, 20 Apr 2023 23:54:08 -0400 Received: from mx0b-001b2d01.pphosted.com ([148.163.158.5]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pphqX-0001y0-TJ for qemu-devel@nongnu.org; Thu, 20 Apr 2023 23:54:08 -0400 Received: from pps.filterd (m0353725.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 33L3hMWS011972; Fri, 21 Apr 2023 03:54:04 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=message-id : date : subject : to : cc : references : from : in-reply-to : content-type : content-transfer-encoding : mime-version; s=pp1; bh=JrcuJ1iN6yFbi0h9t8QRMSY8nriCHO0q/FdlMNtZTQU=; b=Bi1bLsBkZn6alPlN1yR1IMofU0y1JSXHkVytLB9lIZvoYJCUd2C0yuY9LrRME/E2Nqkv gfDYCEovqdorncdmhICwnH+KZOETrdWwxjV65i7lT9tjqqFfAp5vzgIG4Vwzll2inJxA gpsrP+URlgQVVRhgGJKIYBK4fi2VQNrLD29XHovkAoKh+mpR3oK7ndpDqa6leiAWjFSg BpYkyI9altfTm2w3qCNa8f4Qnf2NYB/QSSZf27hbYyOTH45aqtKqBd+4OzJW6HbUJwUQ YnfDNJRdNa0254n/FMv7KjOHCWGCVMdmmpMm1GSO3HBz1oiVVkYE9Hn96g57/Cw42xdr bg== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 3q3ju1gbca-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 21 Apr 2023 03:54:03 +0000 Received: from m0353725.ppops.net (m0353725.ppops.net [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 33L3pFqP011943; Fri, 21 Apr 2023 03:54:03 GMT Received: from ppma01dal.us.ibm.com (83.d6.3fa9.ip4.static.sl-reverse.com [169.63.214.131]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 3q3ju1gbbs-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 21 Apr 2023 03:54:03 +0000 Received: from pps.filterd (ppma01dal.us.ibm.com [127.0.0.1]) by ppma01dal.us.ibm.com (8.17.1.19/8.17.1.19) with ESMTP id 33L1FIRM003725; Fri, 21 Apr 2023 03:54:02 GMT Received: from smtprelay03.dal12v.mail.ibm.com ([9.208.130.98]) by ppma01dal.us.ibm.com (PPS) with ESMTPS id 3pykj8fb0u-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 21 Apr 2023 03:54:02 +0000 Received: from smtpav06.wdc07v.mail.ibm.com (smtpav06.wdc07v.mail.ibm.com [10.39.53.233]) by smtprelay03.dal12v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 33L3s0gX10027570 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 21 Apr 2023 03:54:01 GMT Received: from smtpav06.wdc07v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id C25C458054; Fri, 21 Apr 2023 03:54:00 +0000 (GMT) Received: from smtpav06.wdc07v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id C40D55804E; Fri, 21 Apr 2023 03:53:54 +0000 (GMT) Received: from [9.43.106.147] (unknown [9.43.106.147]) by smtpav06.wdc07v.mail.ibm.com (Postfix) with ESMTP; Fri, 21 Apr 2023 03:53:54 +0000 (GMT) Message-ID: <1be5d2af-23cd-5fda-7b41-cd0c8374d866@linux.ibm.com> Date: Fri, 21 Apr 2023 09:23:52 +0530 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.9.0 Subject: Re: [PATCH v3 2/2] tests/avocado/tuxrun_baselines.py: improve code coverage for ppc64 Content-Language: en-US To: Kautuk Consul , Harsh Prateek Bora Cc: =?UTF-8?Q?Alex_Benn=c3=a9e?= , =?UTF-8?Q?Philippe_Mathieu-Daud=c3=a9?= , Thomas Huth , Wainer dos Santos Moschetta , Beraldo Leal , Cleber Rosa , qemu-devel@nongnu.org References: <20230420125217.620928-1-kconsul@linux.vnet.ibm.com> <20230420125217.620928-3-kconsul@linux.vnet.ibm.com> From: Harsh Prateek Bora In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed X-TM-AS-GCONF: 00 X-Proofpoint-GUID: c4EZruyTbVdDB38zFOVsLmsRCyEOyoy9 X-Proofpoint-ORIG-GUID: 2zF1e4PA2AA4e88QCvLq6YuAUQz3d89v Content-Transfer-Encoding: 8bit X-Proofpoint-UnRewURL: 0 URL was un-rewritten MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.254,Aquarius:18.0.942,Hydra:6.0.573,FMLib:17.11.170.22 definitions=2023-04-20_17,2023-04-20_01,2023-02-09_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 adultscore=0 bulkscore=0 priorityscore=1501 suspectscore=0 phishscore=0 lowpriorityscore=0 malwarescore=0 mlxscore=0 clxscore=1015 mlxlogscore=999 spamscore=0 impostorscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2303200000 definitions=main-2304210029 Received-SPF: pass client-ip=148.163.158.5; envelope-from=harshpb@linux.ibm.com; helo=mx0b-001b2d01.pphosted.com X-Spam_score_int: -36 X-Spam_score: -3.7 X-Spam_bar: --- X-Spam_report: (-3.7 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_EF=-0.1, NICE_REPLY_A=-1.669, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org On 4/21/23 09:12, Kautuk Consul wrote: > Hi, > > On 2023-04-20 19:20:40, Harsh Prateek Bora wrote: >> Since we are optimising code a lot, one suggestion below: >> >> >> On Thu, Apr 20, 2023 at 6:23 PM Kautuk Consul >> wrote: >> >>> Commit c0c8687ef0fd990db8db1655a8a6c5a5e35dd4bb disabled the >>> boot_linux.py test-case due to which the code coverage for ppc >>> decreased by around 2%. As per the discussion on >>> https://lore.kernel.org/qemu-devel/87sfdpqcy4.fsf@linaro.org/ it >>> was mentioned that the baseline test for ppc64 could be modified >>> to make up this 2% code coverage. This patch attempts to achieve >>> this 2% code coverage by adding various device command line >>> arguments (to ./qemu-system-ppc64) in the tuxrun_baselines.py >>> test-case. >>> >>> The code coverage report with boot_linux.py, without it and finally >>> with these tuxrun_baselines.py changes is as follows: >>> >>> With boot_linux.py >>> ------------------ >>> lines......: 13.8% (58006 of 420997 lines) >>> functions..: 20.7% (7675 of 36993 functions) >>> branches...: 9.2% (22146 of 240611 branches) >>> Without boot_linux.py (without this patch changes) >>> -------------------------------------------------- >>> lines......: 11.9% (50174 of 420997 lines) >>> functions..: 18.8% (6947 of 36993 functions) >>> branches...: 7.4% (17580 of 239017 branches) >>> Without boot_linux.py (with this patch changes) >>> ----------------------------------------------- >>> lines......: 13.8% (58287 of 420997 lines) >>> functions..: 20.7% (7640 of 36993 functions) >>> branches...: 8.4% (20223 of 240611 branches) >>> >>> Signed-off-by: Kautuk Consul >>> Reported-by: Alex Bennée >>> --- >>> tests/avocado/tuxrun_baselines.py | 120 +++++++++++++++++++++++++++++- >>> 1 file changed, 116 insertions(+), 4 deletions(-) >>> >>> diff --git a/tests/avocado/tuxrun_baselines.py >>> b/tests/avocado/tuxrun_baselines.py >>> index d343376faa..ae082ac028 100644 >>> --- a/tests/avocado/tuxrun_baselines.py >>> +++ b/tests/avocado/tuxrun_baselines.py >>> @@ -11,6 +11,7 @@ >>> >>> import os >>> import time >>> +import tempfile >>> >>> >>> """ >>> - self.common_tuxrun(drive="scsi-hd") >>> + >>> + # add device args to command line. >>> + self.vm.add_args('-netdev', 'user,id=vnet,hostfwd=:127.0.0.1:0 >>> -:22', >>> + '-device', 'virtio-net,netdev=vnet') >>> + self.vm.add_args('-netdev', '{"type":"user","id":"hostnet0"}', >>> + '-device', '{"driver":"virtio-net-pci","netdev":' >>> + >>> '"hostnet0","id":"net0","mac":"52:54:00:4c:e3:86",' >>> + '"bus":"pci.0","addr":"0x9"}') >>> + self.vm.add_args('-device', >>> '{"driver":"qemu-xhci","p2":15,"p3":15,' >>> + '"id":"usb","bus":"pci.0","addr":"0x2"}') >>> + self.vm.add_args('-device', >>> '{"driver":"virtio-scsi-pci","id":"scsi0"' >>> + ',"bus":"pci.0","addr":"0x3"}') >>> + self.vm.add_args('-device', '{"driver":"virtio-serial-pci","id":' >>> + '"virtio-serial0","bus":"pci.0","addr":"0x4"}') >>> + self.vm.add_args('-device', '{"driver":"scsi-cd","bus":"scsi0.0"' >>> + ',"channel":0,"scsi-id":0,"lun":0,"device_id":' >>> + '"drive-scsi0-0-0-0","id":"scsi0-0-0-0"}') >>> + self.vm.add_args('-device', '{"driver":"virtio-balloon-pci",' >>> + '"id":"balloon0","bus":"pci.0","addr":"0x6"}') >>> + self.vm.add_args('-audiodev', '{"id":"audio1","driver":"none"}') >>> + self.vm.add_args('-device', '{"driver":"usb-tablet","id":"input0"' >>> + ',"bus":"usb.0","port":"1"}') >>> + self.vm.add_args('-device', '{"driver":"usb-kbd","id":"input1"' >>> + ',"bus":"usb.0","port":"2"}') >>> + self.vm.add_args('-device', '{"driver":"VGA","id":"video0",' >>> + '"vgamem_mb":16,"bus":"pci.0","addr":"0x7"}') >>> + self.vm.add_args('-object', >>> '{"qom-type":"rng-random","id":"objrng0"' >>> + ',"filename":"/dev/urandom"}', >>> + '-device', >>> '{"driver":"virtio-rng-pci","rng":"objrng0"' >>> + ',"id":"rng0","bus":"pci.0","addr":"0x8"}') >>> + self.vm.add_args('-object', >>> '{"qom-type":"cryptodev-backend-builtin",' >>> + '"id":"objcrypto0","queues":1}', >>> + '-device', '{"driver":"virtio-crypto-pci",' >>> + '"cryptodev":"objcrypto0","id":"crypto0","bus"' >>> + ':"pci.0","addr":"0xa"}') >>> + self.vm.add_args('-device', '{"driver":"spapr-pci-host-bridge"' >>> + ',"index":1,"id":"pci.1"}') >>> + self.vm.add_args('-device', '{"driver":"spapr-vscsi","id":"scsi1"' >>> + ',"reg":12288}') >>> + self.vm.add_args('-m', '2G,slots=32,maxmem=4G', >>> + '-object', 'memory-backend-ram,id=ram1,size=1G', >>> + '-device', 'pc-dimm,id=dimm1,memdev=ram1') >>> + >>> >> >> >> Most of the above code for adding args is getting duplicated in the test >> for ppc64le below as well. >> It could be contained in a helper routine which could be called from both, >> and would be easier to maintain. > I kept it separate because there may be some future reason to make the > devices different for ppc and ppc64le. I know that probably won't happen > but thought to keep it separate to allow for that possibility. > Or if you feel that they should be exactly the same maybe I could send > another patch after this to refactor the code into a single routine ? We can keep a single routine if all the devices are exactly same for now. Whenever (or if now) there is a change in device needed, that can be conveyed to that routine via a flag for be/le and the routine can add additional devices based on the flag passed. >> >> regards, >> Harsh >> >> + # Create a temporary qcow2 and launch the test-case >>> + with tempfile.NamedTemporaryFile(prefix='tuxrun_ppc64_', >>> + suffix='.qcow2') as qcow2: >>> + process.run(self.qemu_img + ' create -f qcow2 ' + >>> + qcow2.name + ' 1G') >>> + >>> + self.vm.add_args('-drive', 'file=' + qcow2.name + >>> + ',format=qcow2,if=none,id=' >>> + 'drive-virtio-disk1', >>> + '-device', 'virtio-blk-pci,scsi=off,bus=pci.0,' >>> + >>> 'addr=0xb,drive=drive-virtio-disk1,id=virtio-disk1' >>> + ',bootindex=2') >>> + self.common_tuxrun(drive="scsi-hd") >>> >>> def test_ppc64le(self): >>> """ >>> :avocado: tags=arch:ppc64 >>> :avocado: tags=machine:pseries >>> - :avocado: tags=cpu:POWER8 >>> + :avocado: tags=cpu:POWER10 >>> :avocado: tags=console:hvc0 >>> :avocado: tags=tuxboot:ppc64le >>> :avocado: tags=image:vmlinux >>> :avocado: tags=extradev:driver=spapr-vscsi >>> :avocado: tags=root:sda >>> """ >>> - self.common_tuxrun(drive="scsi-hd") >>> + # add device args to command line. >>> + self.vm.add_args('-netdev', 'user,id=vnet,hostfwd=:127.0.0.1:0 >>> -:22', >>> + '-device', 'virtio-net,netdev=vnet') >>> + self.vm.add_args('-netdev', '{"type":"user","id":"hostnet0"}', >>> + '-device', '{"driver":"virtio-net-pci","netdev":' >>> + >>> '"hostnet0","id":"net0","mac":"52:54:00:4c:e3:86",' >>> + '"bus":"pci.0","addr":"0x9"}') >>> + self.vm.add_args('-device', >>> '{"driver":"qemu-xhci","p2":15,"p3":15,' >>> + '"id":"usb","bus":"pci.0","addr":"0x2"}') >>> + self.vm.add_args('-device', >>> '{"driver":"virtio-scsi-pci","id":"scsi0"' >>> + ',"bus":"pci.0","addr":"0x3"}') >>> + self.vm.add_args('-device', '{"driver":"virtio-serial-pci","id":' >>> + '"virtio-serial0","bus":"pci.0","addr":"0x4"}') >>> + self.vm.add_args('-device', '{"driver":"scsi-cd","bus":"scsi0.0"' >>> + ',"channel":0,"scsi-id":0,"lun":0,"device_id":' >>> + '"drive-scsi0-0-0-0","id":"scsi0-0-0-0"}') >>> + self.vm.add_args('-device', '{"driver":"virtio-balloon-pci",' >>> + '"id":"balloon0","bus":"pci.0","addr":"0x6"}') >>> + self.vm.add_args('-audiodev', '{"id":"audio1","driver":"none"}') >>> + self.vm.add_args('-device', '{"driver":"usb-tablet","id":"input0"' >>> + ',"bus":"usb.0","port":"1"}') >>> + self.vm.add_args('-device', '{"driver":"usb-kbd","id":"input1"' >>> + ',"bus":"usb.0","port":"2"}') >>> + self.vm.add_args('-device', '{"driver":"VGA","id":"video0",' >>> + '"vgamem_mb":16,"bus":"pci.0","addr":"0x7"}') >>> + self.vm.add_args('-object', >>> '{"qom-type":"rng-random","id":"objrng0"' >>> + ',"filename":"/dev/urandom"}', >>> + '-device', >>> '{"driver":"virtio-rng-pci","rng":"objrng0"' >>> + ',"id":"rng0","bus":"pci.0","addr":"0x8"}') >>> + self.vm.add_args('-object', >>> '{"qom-type":"cryptodev-backend-builtin",' >>> + '"id":"objcrypto0","queues":1}', >>> + '-device', '{"driver":"virtio-crypto-pci",' >>> + '"cryptodev":"objcrypto0","id":"crypto0","bus"' >>> + ':"pci.0","addr":"0xa"}') >>> + self.vm.add_args('-device', '{"driver":"spapr-pci-host-bridge"' >>> + ',"index":1,"id":"pci.1"}') >>> + self.vm.add_args('-device', '{"driver":"spapr-vscsi","id":"scsi1"' >>> + ',"reg":12288}') >>> + self.vm.add_args('-m', '2G,slots=32,maxmem=4G', >>> + '-object', 'memory-backend-ram,id=ram1,size=1G', >>> + '-device', 'pc-dimm,id=dimm1,memdev=ram1') >>> + >>> + # Create a temporary qcow2 and launch the test-case >>> + with tempfile.NamedTemporaryFile(prefix='tuxrun_ppc64le_', >>> + suffix='.qcow2') as qcow2: >>> + process.run(self.qemu_img + ' create -f qcow2 ' + >>> + qcow2.name + ' 1G') >>> + >>> + self.vm.add_args('-drive', 'file=' + qcow2.name + >>> + ',format=qcow2,if=none,id=' >>> + 'drive-virtio-disk1', >>> + '-device', 'virtio-blk-pci,scsi=off,bus=pci.0,' >>> + >>> 'addr=0xb,drive=drive-virtio-disk1,id=virtio-disk1' >>> + ',bootindex=2') >>> + self.common_tuxrun(drive="scsi-hd") >>> >>> def test_riscv32(self): >>> """ >>> -- >>> 2.25.1 >>> >>> >>>