Skip to contentSkip to conversation

test: use virt-make-fs to build image

PATCH New Heinrich Schuchardt 3 participants 6 messages Jul 13, 2020 – Jul 16, 2020

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 test: use virt-make-fs to build image

Avoid sudo for test/py/tests/test_efi_secboot by using virt-make-fs.

---
 test/py/tests/test_efi_secboot/conftest.py | 27 ++++------------------
 test/py/tests/test_efi_secboot/defs.py     |  7 ------
 2 files changed, 4 insertions(+), 30 deletions(-)
test/py/tests/test_efi_secboot/conftest.py +4 -23
index 71ef723e59..c6709700a8 100644
@@ -38,34 +38,15 @@ def efi_boot_env(request, u_boot_config):
 
     image_path = u_boot_config.persistent_data_dir
     image_path = image_path + '/' + EFI_SECBOOT_IMAGE_NAME
-    image_size = EFI_SECBOOT_IMAGE_SIZE
-    part_size = EFI_SECBOOT_PART_SIZE
-    fs_type = EFI_SECBOOT_FS_TYPE
 
     if HELLO_PATH == '':
         HELLO_PATH = u_boot_config.build_dir + '/lib/efi_loader/helloworld.efi'
 
     try:
-        mnt_point = u_boot_config.persistent_data_dir + '/mnt_efisecure'
+        mnt_point = u_boot_config.build_dir + '/mnt_efisecure'
+        check_call('rm -rf {}'.format(mnt_point), shell=True)
         check_call('mkdir -p {}'.format(mnt_point), shell=True)
 
-        # create a disk/partition
-        check_call('dd if=/dev/zero of=%s bs=1MiB count=%d'
-                   % (image_path, image_size), shell=True)
-        check_call('sgdisk %s -n 1:0:+%dMiB'
-                   % (image_path, part_size), shell=True)
-        # create a file system
-        check_call('dd if=/dev/zero of=%s.tmp bs=1MiB count=%d'
-                   % (image_path, part_size), shell=True)
-        check_call('mkfs -t %s %s.tmp' % (fs_type, image_path), shell=True)
-        check_call('dd if=%s.tmp of=%s bs=1MiB seek=1 count=%d conv=notrunc'
-                   % (image_path, image_path, 1), shell=True)
-        check_call('rm %s.tmp' % image_path, shell=True)
-        loop_dev = check_output('sudo losetup -o 1MiB --sizelimit %dMiB --show -f %s | tr -d "\n"'
-                                % (part_size, image_path), shell=True).decode()
-        check_output('sudo mount -t %s -o umask=000 %s %s'
-                     % (fs_type, loop_dev, mnt_point), shell=True)
-
         # suffix
         # *.key: RSA private key in PEM
         # *.crt: X509 certificate (self-signed) in PEM
@@ -145,8 +126,8 @@ def efi_boot_env(request, u_boot_config):
                    % (mnt_point, EFITOOLS_PATH),
                    shell=True)
 
-        check_call('sudo umount %s' % loop_dev, shell=True)
-        check_call('sudo losetup -d %s' % loop_dev, shell=True)
+        check_call('virt-make-fs --partition=gpt --size=+1M --type=vfat {} {}'.format(mnt_point, image_path), shell=True)
+        check_call('rm -rf {}'.format(mnt_point), shell=True)
 
     except CalledProcessError as exception:
         pytest.skip('Setup failed: %s' % exception.cmd)
test/py/tests/test_efi_secboot/defs.py +0 -7
index 099f453979..ba6b9f391e 100644
@@ -3,13 +3,6 @@
 # Disk image name
 EFI_SECBOOT_IMAGE_NAME = 'test_efi_secboot.img'
 
-# Size in MiB
-EFI_SECBOOT_IMAGE_SIZE = 16
-EFI_SECBOOT_PART_SIZE = 8
-
-# Partition file system type
-EFI_SECBOOT_FS_TYPE = 'vfat'
-
 # Owner guid
 GUID = '11111111-2222-3333-4444-123456789abc'
 

--
2.27.0

#1 lore raw

PATCH 1/1 test: use virt-make-fs to build image

Heinrich,

Quoted 1 line · Heinrich Schuchardt

Avoid sudo for test/py/tests/test_efi_secboot by using virt-make-fs.

Have you read this?
https://lists.denx.de/pipermail/u-boot/2020-July/419345.html

-Takahiro Akashi

Quoted 79 lines
---
 test/py/tests/test_efi_secboot/conftest.py | 27 ++++------------------
 test/py/tests/test_efi_secboot/defs.py     |  7 ------
 2 files changed, 4 insertions(+), 30 deletions(-)
test/py/tests/test_efi_secboot/conftest.py +4 -23
index 71ef723e59..c6709700a8 100644
@@ -38,34 +38,15 @@ def efi_boot_env(request, u_boot_config):
 
     image_path = u_boot_config.persistent_data_dir
     image_path = image_path + '/' + EFI_SECBOOT_IMAGE_NAME
-    image_size = EFI_SECBOOT_IMAGE_SIZE
-    part_size = EFI_SECBOOT_PART_SIZE
-    fs_type = EFI_SECBOOT_FS_TYPE
 
     if HELLO_PATH == '':
         HELLO_PATH = u_boot_config.build_dir + '/lib/efi_loader/helloworld.efi'
 
     try:
-        mnt_point = u_boot_config.persistent_data_dir + '/mnt_efisecure'
+        mnt_point = u_boot_config.build_dir + '/mnt_efisecure'
+        check_call('rm -rf {}'.format(mnt_point), shell=True)
         check_call('mkdir -p {}'.format(mnt_point), shell=True)
 
-        # create a disk/partition
-        check_call('dd if=/dev/zero of=%s bs=1MiB count=%d'
-                   % (image_path, image_size), shell=True)
-        check_call('sgdisk %s -n 1:0:+%dMiB'
-                   % (image_path, part_size), shell=True)
-        # create a file system
-        check_call('dd if=/dev/zero of=%s.tmp bs=1MiB count=%d'
-                   % (image_path, part_size), shell=True)
-        check_call('mkfs -t %s %s.tmp' % (fs_type, image_path), shell=True)
-        check_call('dd if=%s.tmp of=%s bs=1MiB seek=1 count=%d conv=notrunc'
-                   % (image_path, image_path, 1), shell=True)
-        check_call('rm %s.tmp' % image_path, shell=True)
-        loop_dev = check_output('sudo losetup -o 1MiB --sizelimit %dMiB --show -f %s | tr -d "\n"'
-                                % (part_size, image_path), shell=True).decode()
-        check_output('sudo mount -t %s -o umask=000 %s %s'
-                     % (fs_type, loop_dev, mnt_point), shell=True)
-
         # suffix
         # *.key: RSA private key in PEM
         # *.crt: X509 certificate (self-signed) in PEM
@@ -145,8 +126,8 @@ def efi_boot_env(request, u_boot_config):
                    % (mnt_point, EFITOOLS_PATH),
                    shell=True)
 
-        check_call('sudo umount %s' % loop_dev, shell=True)
-        check_call('sudo losetup -d %s' % loop_dev, shell=True)
+        check_call('virt-make-fs --partition=gpt --size=+1M --type=vfat {} {}'.format(mnt_point, image_path), shell=True)
+        check_call('rm -rf {}'.format(mnt_point), shell=True)
 
     except CalledProcessError as exception:
         pytest.skip('Setup failed: %s' % exception.cmd)
test/py/tests/test_efi_secboot/defs.py +0 -8
index 099f453979..ba6b9f391e 100644
@@ -3,13 +3,6 @@
 # Disk image name
 EFI_SECBOOT_IMAGE_NAME = 'test_efi_secboot.img'
 
-# Size in MiB
-EFI_SECBOOT_IMAGE_SIZE = 16
-EFI_SECBOOT_PART_SIZE = 8
-
-# Partition file system type
-EFI_SECBOOT_FS_TYPE = 'vfat'
-
 # Owner guid
 GUID = '11111111-2222-3333-4444-123456789abc'
 
--

2.27.0

#2 lore raw

PATCH 1/1 test: use virt-make-fs to build image

Am 14. Juli 2020 01:02:07 MESZ schrieb AKASHI Takahiro <[email protected]>:

Quoted 9 lines

Heinrich,

Quoted 1 line · Heinrich Schuchardt

Avoid sudo for test/py/tests/test_efi_secboot by using virt-make-fs.

Have you read this?
https://lists.denx.de/pipermail/u-boot/2020-July/419345.html

-Takahiro Akashi

Just put chmod 644 /boot/vmlinu* into your update-initramfs hook directory /etc/initramfs-tools and complain to Ubuntu maintainers about them breaking their own packages.

Hook scripts are described here: http://manpages.ubuntu.com/manpages/xenial/man8/initramfs-tools.8.html

Quoted 90 lines
Quoted 4 lines
---
 test/py/tests/test_efi_secboot/conftest.py | 27

++++------------------

Review

Quoted 4 lines
 test/py/tests/test_efi_secboot/defs.py     |  7 ------
 2 files changed, 4 insertions(+), 30 deletions(-)
(quoted diff) +0 -0

b/test/py/tests/test_efi_secboot/conftest.py

test/py/tests/test_efi_secboot/conftest.py def efi_boot_env(request, u_boot_config): · line 43

Quoted 13 lines

index 71ef723e59..c6709700a8 100644

test/py/tests/test_efi_secboot/conftest.py +0 -3
@@ -38,34 +38,15 @@ def efi_boot_env(request, u_boot_config):
 
     image_path = u_boot_config.persistent_data_dir
     image_path = image_path + '/' + EFI_SECBOOT_IMAGE_NAME
-    image_size = EFI_SECBOOT_IMAGE_SIZE
-    part_size = EFI_SECBOOT_PART_SIZE
-    fs_type = EFI_SECBOOT_FS_TYPE
 
     if HELLO_PATH == '':
         HELLO_PATH = u_boot_config.build_dir +

'/lib/efi_loader/helloworld.efi'

test/py/tests/test_efi_secboot/conftest.py def efi_boot_env(request, u_boot_config): · line 48

Quoted 3 lines
test/py/tests/test_efi_secboot/conftest.py +0 -1
     try:
-        mnt_point = u_boot_config.persistent_data_dir +

'/mnt_efisecure'

test/py/tests/test_efi_secboot/conftest.py def efi_boot_env(request, u_boot_config):

Quoted 13 lines
test/py/tests/test_efi_secboot/conftest.py +2 -9
+        mnt_point = u_boot_config.build_dir + '/mnt_efisecure'
+        check_call('rm -rf {}'.format(mnt_point), shell=True)
         check_call('mkdir -p {}'.format(mnt_point), shell=True)
 
-        # create a disk/partition
-        check_call('dd if=/dev/zero of=%s bs=1MiB count=%d'
-                   % (image_path, image_size), shell=True)
-        check_call('sgdisk %s -n 1:0:+%dMiB'
-                   % (image_path, part_size), shell=True)
-        # create a file system
-        check_call('dd if=/dev/zero of=%s.tmp bs=1MiB count=%d'
-                   % (image_path, part_size), shell=True)
-        check_call('mkfs -t %s %s.tmp' % (fs_type, image_path),

shell=True)

test/py/tests/test_efi_secboot/conftest.py def efi_boot_env(request, u_boot_config): · line 60

Quoted 1 line
test/py/tests/test_efi_secboot/conftest.py +0 -1
-        check_call('dd if=%s.tmp of=%s bs=1MiB seek=1 count=%d

conv=notrunc'

test/py/tests/test_efi_secboot/conftest.py def efi_boot_env(request, u_boot_config):

Quoted 3 lines
test/py/tests/test_efi_secboot/conftest.py +0 -3
-                   % (image_path, image_path, 1), shell=True)
-        check_call('rm %s.tmp' % image_path, shell=True)
-        loop_dev = check_output('sudo losetup -o 1MiB --sizelimit

%dMiB --show -f %s | tr -d "\n"'

test/py/tests/test_efi_secboot/conftest.py def efi_boot_env(request, u_boot_config): · line 64

Quoted 1 line
test/py/tests/test_efi_secboot/conftest.py +0 -1
-                                % (part_size, image_path),

shell=True).decode()

test/py/tests/test_efi_secboot/conftest.py def efi_boot_env(request, u_boot_config): · line 129

Quoted 13 lines
test/py/tests/test_efi_secboot/conftest.py +1 -5
-        check_output('sudo mount -t %s -o umask=000 %s %s'
-                     % (fs_type, loop_dev, mnt_point), shell=True)
-
         # suffix
         # *.key: RSA private key in PEM
         # *.crt: X509 certificate (self-signed) in PEM
@@ -145,8 +126,8 @@ def efi_boot_env(request, u_boot_config):
                    % (mnt_point, EFITOOLS_PATH),
                    shell=True)
 
-        check_call('sudo umount %s' % loop_dev, shell=True)
-        check_call('sudo losetup -d %s' % loop_dev, shell=True)
+        check_call('virt-make-fs --partition=gpt --size=+1M

--type=vfat {} {}'.format(mnt_point, image_path), shell=True)

Review

Quoted 5 lines
test/py/tests/test_efi_secboot/conftest.py +1 -0
+        check_call('rm -rf {}'.format(mnt_point), shell=True)
 
     except CalledProcessError as exception:
         pytest.skip('Setup failed: %s' % exception.cmd)
(quoted diff) +0 -0

b/test/py/tests/test_efi_secboot/defs.py

Quoted 20 lines

index 099f453979..ba6b9f391e 100644

test/py/tests/test_efi_secboot/defs.py +0 -8
@@ -3,13 +3,6 @@
 # Disk image name
 EFI_SECBOOT_IMAGE_NAME = 'test_efi_secboot.img'
 
-# Size in MiB
-EFI_SECBOOT_IMAGE_SIZE = 16
-EFI_SECBOOT_PART_SIZE = 8
-
-# Partition file system type
-EFI_SECBOOT_FS_TYPE = 'vfat'
-
 # Owner guid
 GUID = '11111111-2222-3333-4444-123456789abc'
 
--

2.27.0

#3 lore raw

PATCH 1/1 test: use virt-make-fs to build image

Heinrich,

Quoted 4 lines · Heinrich Schuchardt

Am 14. Juli 2020 01:02:07 MESZ schrieb AKASHI Takahiro <[email protected]>:

Quoted 3 lines

Heinrich,

On Tue, Jul 14, 2020 at 12:15:34AM +0200, Heinrich Schuchardt wrote:

Quoted 8 lines
Quoted 6 lines
Quoted 1 line

Avoid sudo for test/py/tests/test_efi_secboot by using virt-make-fs.

Have you read this?
https://lists.denx.de/pipermail/u-boot/2020-July/419345.html

-Takahiro Akashi

Just put chmod 644 /boot/vmlinu* into your update-initramfs hook directory /etc/initramfs-tools and complain to Ubuntu maintainers about them breaking their own packages.

If this solves all the issues, that's fine.
But if you require this extra step to run the test, you should describe it explicitly somewhere. Otherwise, people can get confused.

I believe that it would be still useful to have a fall-back method of "sudo" version.

Quoted 31 lines

Hook scripts are described here: http://manpages.ubuntu.com/manpages/xenial/man8/initramfs-tools.8.html

Quoted 29 lines
Quoted 4 lines
---
 test/py/tests/test_efi_secboot/conftest.py | 27

++++------------------

Review

Quoted 4 lines
 test/py/tests/test_efi_secboot/defs.py     |  7 ------
 2 files changed, 4 insertions(+), 30 deletions(-)
(quoted diff) +0 -0

b/test/py/tests/test_efi_secboot/conftest.py

test/py/tests/test_efi_secboot/conftest.py def efi_boot_env(request, u_boot_config): · line 43

Quoted 13 lines

index 71ef723e59..c6709700a8 100644

test/py/tests/test_efi_secboot/conftest.py +0 -3
@@ -38,34 +38,15 @@ def efi_boot_env(request, u_boot_config):
 
     image_path = u_boot_config.persistent_data_dir
     image_path = image_path + '/' + EFI_SECBOOT_IMAGE_NAME
-    image_size = EFI_SECBOOT_IMAGE_SIZE
-    part_size = EFI_SECBOOT_PART_SIZE
-    fs_type = EFI_SECBOOT_FS_TYPE
 
     if HELLO_PATH == '':
         HELLO_PATH = u_boot_config.build_dir +

'/lib/efi_loader/helloworld.efi'

test/py/tests/test_efi_secboot/conftest.py def efi_boot_env(request, u_boot_config): · line 48

Quoted 3 lines
test/py/tests/test_efi_secboot/conftest.py +0 -1
     try:
-        mnt_point = u_boot_config.persistent_data_dir +

'/mnt_efisecure'

Quoted 1 line
test/py/tests/test_efi_secboot/conftest.py +1 -0
+        mnt_point = u_boot_config.build_dir + '/mnt_efisecure'

Please use persistent_data_dir here as a work directory. 'build_dir' is for U-Boot build.

-Takahiro Akashi

Quoted 62 lines
Quoted 61 lines

test/py/tests/test_efi_secboot/conftest.py def efi_boot_env(request, u_boot_config):

Quoted 12 lines
test/py/tests/test_efi_secboot/conftest.py +1 -9
+        check_call('rm -rf {}'.format(mnt_point), shell=True)
         check_call('mkdir -p {}'.format(mnt_point), shell=True)
 
-        # create a disk/partition
-        check_call('dd if=/dev/zero of=%s bs=1MiB count=%d'
-                   % (image_path, image_size), shell=True)
-        check_call('sgdisk %s -n 1:0:+%dMiB'
-                   % (image_path, part_size), shell=True)
-        # create a file system
-        check_call('dd if=/dev/zero of=%s.tmp bs=1MiB count=%d'
-                   % (image_path, part_size), shell=True)
-        check_call('mkfs -t %s %s.tmp' % (fs_type, image_path),

shell=True)

test/py/tests/test_efi_secboot/conftest.py def efi_boot_env(request, u_boot_config): · line 60

Quoted 1 line
test/py/tests/test_efi_secboot/conftest.py +0 -1
-        check_call('dd if=%s.tmp of=%s bs=1MiB seek=1 count=%d

conv=notrunc'

test/py/tests/test_efi_secboot/conftest.py def efi_boot_env(request, u_boot_config):

Quoted 3 lines
test/py/tests/test_efi_secboot/conftest.py +0 -3
-                   % (image_path, image_path, 1), shell=True)
-        check_call('rm %s.tmp' % image_path, shell=True)
-        loop_dev = check_output('sudo losetup -o 1MiB --sizelimit

%dMiB --show -f %s | tr -d "\n"'

test/py/tests/test_efi_secboot/conftest.py def efi_boot_env(request, u_boot_config): · line 64

Quoted 1 line
test/py/tests/test_efi_secboot/conftest.py +0 -1
-                                % (part_size, image_path),

shell=True).decode()

test/py/tests/test_efi_secboot/conftest.py def efi_boot_env(request, u_boot_config): · line 129

Quoted 13 lines
test/py/tests/test_efi_secboot/conftest.py +1 -5
-        check_output('sudo mount -t %s -o umask=000 %s %s'
-                     % (fs_type, loop_dev, mnt_point), shell=True)
-
         # suffix
         # *.key: RSA private key in PEM
         # *.crt: X509 certificate (self-signed) in PEM
@@ -145,8 +126,8 @@ def efi_boot_env(request, u_boot_config):
                    % (mnt_point, EFITOOLS_PATH),
                    shell=True)
 
-        check_call('sudo umount %s' % loop_dev, shell=True)
-        check_call('sudo losetup -d %s' % loop_dev, shell=True)
+        check_call('virt-make-fs --partition=gpt --size=+1M

--type=vfat {} {}'.format(mnt_point, image_path), shell=True)

Review

Quoted 5 lines
test/py/tests/test_efi_secboot/conftest.py +1 -0
+        check_call('rm -rf {}'.format(mnt_point), shell=True)
 
     except CalledProcessError as exception:
         pytest.skip('Setup failed: %s' % exception.cmd)
(quoted diff) +0 -0

b/test/py/tests/test_efi_secboot/defs.py

Quoted 20 lines

index 099f453979..ba6b9f391e 100644

test/py/tests/test_efi_secboot/defs.py +0 -8
@@ -3,13 +3,6 @@
 # Disk image name
 EFI_SECBOOT_IMAGE_NAME = 'test_efi_secboot.img'
 
-# Size in MiB
-EFI_SECBOOT_IMAGE_SIZE = 16
-EFI_SECBOOT_PART_SIZE = 8
-
-# Partition file system type
-EFI_SECBOOT_FS_TYPE = 'vfat'
-
 # Owner guid
 GUID = '11111111-2222-3333-4444-123456789abc'
 
--

2.27.0

#4 lore raw

PATCH 1/1 test: use virt-make-fs to build image

Quoted 14 lines · AKASHI Takahiro

Heinrich,

Quoted 11 lines · Heinrich Schuchardt

Am 14. Juli 2020 01:02:07 MESZ schrieb AKASHI Takahiro <[email protected]>:

Quoted 9 lines

Heinrich,

Quoted 1 line · Heinrich Schuchardt

Avoid sudo for test/py/tests/test_efi_secboot by using virt-make-fs.

Have you read this?
https://lists.denx.de/pipermail/u-boot/2020-July/419345.html

-Takahiro Akashi

Quoted 8 lines
Quoted 1 line

Just put chmod 644 /boot/vmlinu* into your update-initramfs hook directory /etc/initramfs-tools and complain to Ubuntu maintainers about them breaking their own packages.

If this solves all the issues, that's fine.
But if you require this extra step to run the test, you should describe it explicitly somewhere. Otherwise, people can get confused.

I believe that it would be still useful to have a fall-back method of "sudo" version.

We're not dropping sudo until CI runs and passes the tests that use guestfstools, etc, instead of sudo. To be clear, once we fix the current broken check that doesn't try and run sudo when guestfstools exists but exits badly, the tests pass. We aren't there yet with just using guestfstools only.

Signature · 8 lines
Tom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 659 bytes
Desc: not available
URL: <https://lists.denx.de/pipermail/u-boot/attachments/20200715/bad38c16/attachment.sig>
#5 lore raw

PATCH 1/1 test: use virt-make-fs to build image

Heinrich,

Quoted 21 lines · Tom Rini
Quoted 20 lines · AKASHI Takahiro

Heinrich,

Quoted 4 lines · Heinrich Schuchardt

Am 14. Juli 2020 01:02:07 MESZ schrieb AKASHI Takahiro <[email protected]>:

Quoted 3 lines

Heinrich,

On Tue, Jul 14, 2020 at 12:15:34AM +0200, Heinrich Schuchardt wrote:

Quoted 8 lines
Quoted 6 lines
Quoted 1 line

Avoid sudo for test/py/tests/test_efi_secboot by using virt-make-fs.

Have you read this?
https://lists.denx.de/pipermail/u-boot/2020-July/419345.html

-Takahiro Akashi

Just put chmod 644 /boot/vmlinu* into your update-initramfs hook directory /etc/initramfs-tools and complain to Ubuntu maintainers about them breaking their own packages.

If this solves all the issues, that's fine.
But if you require this extra step to run the test, you should describe it explicitly somewhere. Otherwise, people can get confused.

Quoted 8 lines
Quoted 2 lines

I believe that it would be still useful to have a fall-back method of "sudo" version.

We're not dropping sudo until CI runs and passes the tests that use guestfstools, etc, instead of sudo. To be clear, once we fix the current broken check that doesn't try and run sudo when guestfstools exists but exits badly, the tests pass. We aren't there yet with just using guestfstools only.

So are you going to work on this?

-Takahiro Akashi

Quoted 2 lines
Signature · 1 line
Tom