Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.3 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_SANE_1 autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 64EE7C07E95 for ; Fri, 2 Jul 2021 20:01:17 +0000 (UTC) Received: from phobos.denx.de (phobos.denx.de [85.214.62.61]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id B3F266108B for ; Fri, 2 Jul 2021 20:01:16 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org B3F266108B Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=konsulko.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=u-boot-bounces@lists.denx.de Received: from h2850616.stratoserver.net (localhost [IPv6:::1]) by phobos.denx.de (Postfix) with ESMTP id 3E254828A4; Fri, 2 Jul 2021 22:01:14 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=none (p=none dis=none) header.from=konsulko.com Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=u-boot-bounces@lists.denx.de Authentication-Results: phobos.denx.de; dkim=pass (1024-bit key; unprotected) header.d=konsulko.com header.i=@konsulko.com header.b="Lfd0BDVl"; dkim-atps=neutral Received: by phobos.denx.de (Postfix, from userid 109) id 32F4B82903; Fri, 2 Jul 2021 22:01:13 +0200 (CEST) Received: from mail-qt1-x82f.google.com (mail-qt1-x82f.google.com [IPv6:2607:f8b0:4864:20::82f]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits)) (No client certificate requested) by phobos.denx.de (Postfix) with ESMTPS id AE61681D48 for ; Fri, 2 Jul 2021 22:01:09 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=none (p=none dis=none) header.from=konsulko.com Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=trini@konsulko.com Received: by mail-qt1-x82f.google.com with SMTP id h11so4570148qtp.5 for ; Fri, 02 Jul 2021 13:01:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=konsulko.com; s=google; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to:user-agent; bh=Dd+Oe28wc/9BdGed2+yrEtpDvS0Da7mgUMLADLD0mQk=; b=Lfd0BDVlPJDB+2xrnxtkb5ZPMuYm9BPWqMkqAODLBV7GxwhuckQRw3zdN99sSGyoDz Y/yveDVZfwHNF9wDPvfslVWzbxlFeBU8ZtJpQ5Q/6PMM4Qz7CmaSfkyi/xkxQ0BPn1BA WcOj5XvKVtX8HX9OkXzM15yW2QU0pvd8wn3E8= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to:user-agent; bh=Dd+Oe28wc/9BdGed2+yrEtpDvS0Da7mgUMLADLD0mQk=; b=FWkezblC5JjOKod/kemFlbRgR3GoYFzpi24JyW3ap1kPFlBWaH8uZOPPmlknkqUDQQ y2dqbQ9Rx3G5y3YrNryCVSVkJ/oabUlJ5y+vSkztSV/C7CVpvaeEeHGQRXFG1svD0rG1 x7IEBN1qz1t6/EQ3IvJpg6l848pHZvybVHtQp+sJv0W83wqliwjZ9zoA+MzJtCbAq+2D FF3Q7zNE5fo295P41F7Ar9d0dJb+LJB2BANmYuo2sb/LrCI3yLOHx8ehMExWPoqk3gUc hh7qyPwELl7dJCY0NFJrDIDF6hGEMSYPcp81+ZqxHzyuhQkQhNN12y7/XgmYVSgv9Ghw SXjg== X-Gm-Message-State: AOAM533ysxUdODeHcAQUnncwffvizmRlOno5SEzneBHqwUSI+cc7yj6/ doP7J0D1LKqLHg+kAmgRLENtbw== X-Google-Smtp-Source: ABdhPJwulsxfdVWau9M/iMGeIeldaaMd44IXA1tdoFLrt2ZEL5bGQGiwX0F7p8VWvp+KOalMyB1BBA== X-Received: by 2002:a05:622a:13d1:: with SMTP id p17mr1415056qtk.215.1625256068460; Fri, 02 Jul 2021 13:01:08 -0700 (PDT) Received: from bill-the-cat (2603-6081-7b01-cbda-447a-194c-2139-e81a.res6.spectrum.com. [2603:6081:7b01:cbda:447a:194c:2139:e81a]) by smtp.gmail.com with ESMTPSA id x5sm1792463qke.92.2021.07.02.13.01.07 (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 02 Jul 2021 13:01:07 -0700 (PDT) Date: Fri, 2 Jul 2021 16:01:05 -0400 From: Tom Rini To: Simon Glass Cc: U-Boot Mailing List , Andy Shevchenko , Alper Nebi Yasak , Heinrich Schuchardt , Akashi Takahiro Subject: Re: [RFC] Start using guestfish for U-Boot fs tests Message-ID: <20210702200105.GC9516@bill-the-cat> References: <20210702190107.GA9516@bill-the-cat> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="BdKFqHPPyZPjKVxo" Content-Disposition: inline In-Reply-To: X-Clacks-Overhead: GNU Terry Pratchett User-Agent: Mutt/1.9.4 (2018-02-28) X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.34 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" X-Virus-Scanned: clamav-milter 0.103.2 at phobos.denx.de X-Virus-Status: Clean --BdKFqHPPyZPjKVxo Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Fri, Jul 02, 2021 at 01:06:23PM -0600, Simon Glass wrote: > Hi Tom, >=20 > On Fri, 2 Jul 2021 at 13:01, Tom Rini wrote: > > > > Hey all, > > > > I started taking a look at moving to guestfish to see if this resolves > > the latest problem I've run in to: > > https://source.denx.de/u-boot/u-boot/-/jobs/284763#L307 > > which I think is due to guestmount not being done in time for the test. > > So I started converting things to use guestfish directly: > > > > diff --git a/test/py/tests/test_fs/conftest.py b/test/py/tests/test_fs/= conftest.py > > index 7325486cdb1a..e8899cfdd118 100644 > > --- a/test/py/tests/test_fs/conftest.py > > +++ b/test/py/tests/test_fs/conftest.py > > @@ -265,10 +265,10 @@ def fs_obj_basic(request, u_boot_config): > > fs_ubtype =3D fstype_to_ubname(fs_type) > > check_ubconfig(u_boot_config, fs_ubtype) > > > > - mount_dir =3D u_boot_config.persistent_data_dir + '/mnt' > > + data_dir =3D u_boot_config.persistent_data_dir + '/data' > > > > - small_file =3D mount_dir + '/' + SMALL_FILE > > - big_file =3D mount_dir + '/' + BIG_FILE > > + small_file =3D data_dir + '/' + SMALL_FILE > > + big_file =3D data_dir + '/' + BIG_FILE > > > > try: > > > > @@ -279,26 +279,14 @@ def fs_obj_basic(request, u_boot_config): > > return > > > > try: > > - check_call('mkdir -p %s' % mount_dir, shell=3DTrue) > > + check_call('mkdir -p %s' % data_dir, shell=3DTrue) > > except CalledProcessError as err: > > pytest.skip('Preparing mount folder failed for filesystem: ' += fs_type + '. {}'.format(err)) > > call('rm -f %s' % fs_img, shell=3DTrue) > > return > > > > try: > > - # Mount the image so we can populate it. > > - mount_fs(fs_type, fs_img, mount_dir) > > - except CalledProcessError as err: > > - pytest.skip('Mounting to folder failed for filesystem: ' + fs_= type + '. {}'.format(err)) > > - call('rmdir %s' % mount_dir, shell=3DTrue) > > - call('rm -f %s' % fs_img, shell=3DTrue) > > - return > > - > > - try: > > - # Create a subdirectory. > > - check_call('mkdir %s/SUBDIR' % mount_dir, shell=3DTrue) > > - > > - # Create big file in this image. > > + # Create big file to copy in to the image. > > # Note that we work only on the start 1MB, couple MBs in the 2= GB range > > # and the last 1 MB of the huge 2.5GB file. > > # So, just put random values only in those areas. > > @@ -309,10 +297,14 @@ def fs_obj_basic(request, u_boot_config): > > check_call('dd if=3D/dev/urandom of=3D%s bs=3D1M count=3D1 see= k=3D2499' > > % big_file, shell=3DTrue) > > > > - # Create a small file in this image. > > + # Create a small file to copy in to the image. > > check_call('dd if=3D/dev/urandom of=3D%s bs=3D1M count=3D1' > > % small_file, shell=3DTrue) > > > > + # Copy the files in to the image and add a subdirectory. > > + # Create a subdirectory. > > + check_call('guestfish add %s : run : mount /dev/sda / : mkdir = /SUBDIR : copy-in %s %s /' > > + % (fs_img, big_file, small_file), shell=3DTrue) > > # Delete the small file copies which possibly are written as p= art of a > > # previous test. > > # check_call('rm -f "%s.w"' % MB1, shell=3DTrue) > > @@ -357,13 +349,11 @@ def fs_obj_basic(request, u_boot_config): > > > > except CalledProcessError as err: > > pytest.skip('Setup failed for filesystem: ' + fs_type + '. {}'= =2Eformat(err)) > > - umount_fs(mount_dir) > > return > > else: > > - umount_fs(mount_dir) > > yield [fs_ubtype, fs_img, md5val] > > finally: > > - call('rmdir %s' % mount_dir, shell=3DTrue) > > + call('rmdir %s' % data_dir, shell=3DTrue) > > call('rm -f %s' % fs_img, shell=3DTrue) > > > > # > > > > The problem here is that a test run went from taking about 5 minutes to > > taking about 17 minutes. I can reduce this to closer to 15 minutes with > > LIBGUESTFS_BACKEND=3Ddirect and using libguestfs-make-fixed-appliance to > > make an appliance we reuse. But that's still too long to be usable. > > I'm hoping someone has some ideas here on how to improve things. >=20 > Well even 5 minutes is too long. Could we make the filesystems and > files very small? It does not seem useful to make 2GB things. Adding "not fs" brings me down to 2 minutes. Doing "not fs and not efi" (because I can see the efi secboot stuff taking visible time) brings me to 1 minute. Adding in "and not vboot" is 36 seconds. That said, there's only a subset of the tests that we could turn in to "use these static images" as we've also got tests to generate new signed content and verify the signatures, and that's got to test both the sign and verify paths. > Also we could try harder to get the tests running in parallel. It is > not that far away. This, along with easier / clearer documentation on how to run tests directly so that you can pass "not fs and not ..." might help too. I yanked this out of .gitlab-ci.yml today to make iterating on this under docker easier, and have a slightly more robust one in my bin dir: export TEST_PY_BD=3Dsandbox;export UBOOT_TRAVIS_BUILD_DIR=3D/tmp/${TEST_PY_= BD} tools/buildman/buildman -o ${UBOOT_TRAVIS_BUILD_DIR} -w -E -W -e --board ${= TEST_PY_BD} virtualenv -p /usr/bin/python3 /tmp/venv =2E /tmp/venv/bin/activate pip install -r test/py/requirements.txt export PATH=3D/opt/qemu/bin:/tmp/uboot-test-hooks/bin:${PATH} export PYTHONPATH=3D/tmp/uboot-test-hooks/py/travis-ci time ./test/py/test.py -ra --bd ${TEST_PY_BD} --build-dir "$UBOOT_TRAVIS_BU= ILD_DIR" --=20 Tom --BdKFqHPPyZPjKVxo Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQGzBAABCgAdFiEEGjx/cOCPqxcHgJu/FHw5/5Y0tywFAmDfcH4ACgkQFHw5/5Y0 tyzsmwv/cEf8v16RaXMsibKueJh4ar8cRkidBzweqBSpD5azrwkK22mabtcWPi+i KLB3Lg0aX2rLN33mzRzoFpFubXiElnKIOi6uqPp5vuGAkIIk+qMiW4oETlq5w7tW uSDFYyhUzFK3Qbx7qUpn1wU/0lwyShqtxItGImPWVFNTM+jviOSoOGWjGzVanE4e 6WlP/v3CzqcEyZ3YayfmNwd77BLRac6Cpp3RqT6q6VUw4HWw5e2BcfjOjECCXgR2 vu0PwXWB+IqxhEzflKz+9L+6j5yqMMQQ2DJCfi7I2+mO5S31GxRK2VUj8kcdJ8ME +VVv5BL7fhWmzJttE7yYjrR3CJLDgIdE6yxUQZi2ctwo33L5Bn+qxEf8pGQmCI6w R+vv92tvKApJY4VD4LfkNLeOkOmrF3H1cLMafVoG101Da8g6SJ8/z4f4X5wA4etD QbFUj+Cpxlf9iZYDnsbybh5ZPQ/rColRE9S3tXjPZ5vU9tcpR91FCyCmkFII2Kb3 ZloDXlfg =vp8U -----END PGP SIGNATURE----- --BdKFqHPPyZPjKVxo--