support/testing/tests/fs/test_ext: specify 16 MB as ext filesystem size
PATCH 1/4 support/testing/tests/fs/test_ext: specify 16 MB as ext filesystem size
Since Qemu 5.1, the SD card size must be a power of two, so the default size for ext2/3/4 filesystem images of 60 MB is not suitable. Since 16 MB is used for the Ext4 test, let's use the same value for the other tests as well (ext2, ext2r1, ext3). Without this change, the ext2, ext2r1 and ext3 simply fail to run under Qemu >= 5.1.
--- support/testing/tests/fs/test_ext.py | 3 +++ 1 file changed, 3 insertions(+)
support/testing/tests/fs/test_ext.py +3 -0
@@ -43,6 +43,7 @@ class TestExt2(infra.basetest.BRTest): BR2_TARGET_ROOTFS_EXT2=y BR2_TARGET_ROOTFS_EXT2_2r0=y BR2_TARGET_ROOTFS_EXT2_LABEL="foobaz" + BR2_TARGET_ROOTFS_EXT2_SIZE="16384" # BR2_TARGET_ROOTFS_TAR is not set """
@@ -62,6 +63,7 @@ class TestExt2r1(infra.basetest.BRTest): BR2_TARGET_ROOTFS_EXT2=y BR2_TARGET_ROOTFS_EXT2_2r1=y BR2_TARGET_ROOTFS_EXT2_LABEL="foobar" + BR2_TARGET_ROOTFS_EXT2_SIZE="16384" # BR2_TARGET_ROOTFS_TAR is not set """
@@ -81,6 +83,7 @@ class TestExt3(infra.basetest.BRTest): """ BR2_TARGET_ROOTFS_EXT2=y BR2_TARGET_ROOTFS_EXT2_3=y + BR2_TARGET_ROOTFS_EXT2_SIZE="16384" # BR2_TARGET_ROOTFS_TAR is not set """
Signature · 1 line
2.31.1
PATCH 2/4 support/testing/infra: add img_round_power2() function
Since Qemu 5.1, SD card images must have a size that are a power of two. While some filesystem (such as ext2/3/4) allow to specify the expected size of the filesystem, others such as SquashFS do not have this capability.
We were already extending the size of such images to the next 1 MB boundary using "truncate -s %1M", but that is no longer sufficient. So instead, we introduce a helper function that extends the size of an image to the next power of two.
--- support/testing/infra/__init__.py | 11 +++++++++++ 1 file changed, 11 insertions(+)
support/testing/infra/__init__.py +11 -0
@@ -112,3 +112,14 @@ def get_elf_prog_interpreter(builddir, prefix, fpath): continue return m.group(1) return None + + +def img_round_power2(img): + """ + Rounds up the size of an image file to the next power of 2 + """ + sz = os.stat(img).st_size + pow2 = 1 + while pow2 < sz: + pow2 = pow2 << 1 + subprocess.call(["truncate", "-s", str(pow2), img])
Signature · 1 line
2.31.1
Yann E. MORIN
AppliedPATCH 2/4 support/testing/infra: add img_round_power2() function
Thomas, All,
On 2021-06-21 22:25 +0200, Thomas Petazzoni spake thusly:
Quoted 24 lines
Since Qemu 5.1, SD card images must have a size that are a power of two. While some filesystem (such as ext2/3/4) allow to specify the expected size of the filesystem, others such as SquashFS do not have this capability.
We were already extending the size of such images to the next 1 MB boundary using "truncate -s %1M", but that is no longer sufficient. So instead, we introduce a helper function that extends the size of an image to the next power of two.
Signed-off-by Thomas Petazzoni <[email protected]>--- support/testing/infra/__init__.py | 11 +++++++++++ 1 file changed, 11 insertions(+)support/testing/infra/__init__.py +1
-0@@ -112,3 +112,14 @@ def get_elf_prog_interpreter(builddir, prefix, fpath): continue return m.group(1) return None +
support/testing/infra/__init__.py def get_elf_prog_interpreter(builddir, prefix, fpath): · line 125
Quoted 10 lines
support/testing/infra/__init__.py +10
-0+ +def img_round_power2(img): + """ + Rounds up the size of an image file to the next power of 2 + """ + sz = os.stat(img).st_size + pow2 = 1 + while pow2 < sz: + pow2 = pow2 << 1 + subprocess.call(["truncate", "-s", str(pow2), img])
This subprocess call is not very pythonic, so I've changed that to:
with open(img, 'ab') as f:
f.truncate(pow2)
Applied to master, thanks.
Regards,
Yann E. MORIN.
Quoted 7 lines
Signature · 6 lines
2.31.1 _______________________________________________ buildroot mailing list buildroot at busybox.net http://lists.busybox.net/mailman/listinfo/buildroot
Signature · 6 lines
.-----------------.--------------------.------------------.--------------------. | Yann E. MORIN | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: | | +33 662 376 056 | Software Designer | \ / CAMPAIGN | ___ | | +33 561 099 427 `------------.-------: X AGAINST | \e/ There is no | | http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL | v conspiracy. | '------------------------------^-------^------------------^--------------------'
PATCH 3/4 support/testing/tests: fix tests to use infra.img_round_power2()
In reply to Thomas PetazzoniAll the tests that are using if=sd as a Qemu options are changed to use infra.img_round_power2() instead of simply extending the size of the image to the next MB boundary, which is not longer sufficient with Qemu >= 5.1.
--- support/testing/tests/core/test_file_capabilities.py | 2 +- support/testing/tests/fs/test_squashfs.py | 2 +- support/testing/tests/init/base.py | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-)
support/testing/tests/core/test_file_capabilities.py +1 -1
@@ -28,7 +28,7 @@ class TestFileCapabilities(infra.basetest.BRTest): def test_run(self): img = os.path.join(self.builddir, "images", "rootfs.squashfs") - subprocess.call(["truncate", "-s", "%1M", img]) + infra.img_round_power2(img) self.emulator.boot(arch="armv7", kernel=os.path.join(self.builddir, "images", "zImage"),
support/testing/tests/fs/test_squashfs.py +1 -1
@@ -22,7 +22,7 @@ class TestSquashfs(infra.basetest.BRTest): self.assertEqual(out[3], "Compression lz4") img = os.path.join(self.builddir, "images", "rootfs.squashfs") - subprocess.call(["truncate", "-s", "%1M", img]) + infra.img_round_power2(img) self.emulator.boot(arch="armv7", kernel="builtin",
support/testing/tests/init/base.py +1 -1
@@ -7,7 +7,7 @@ class InitSystemBase(infra.basetest.BRTest): def start_emulator(self, fs_type, kernel=None, dtb=None, init=None): img = os.path.join(self.builddir, "images", "rootfs.{}".format(fs_type)) - subprocess.call(["truncate", "-s", "%1M", img]) + infra.img_round_power2(img) options = ["-drive", "file={},if=sd,format=raw".format(img),
Signature · 1 line
2.31.1
Yann E. MORIN
AppliedPATCH 3/4 support/testing/tests: fix tests to use infra.img_round_power2()
Thomas, All,
On 2021-06-21 22:25 +0200, Thomas Petazzoni spake thusly:
Quoted 30 lines
All the tests that are using if=sd as a Qemu options are changed to use
infra.img_round_power2()instead of simply extending the size of the image to the next MB boundary, which is not longer sufficient with Qemu >= 5.1.Signed-off-by Thomas Petazzoni <[email protected]>--- support/testing/tests/core/test_file_capabilities.py | 2 +- support/testing/tests/fs/test_squashfs.py | 2 +- support/testing/tests/init/base.py | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-)support/testing/tests/core/test_file_capabilities.py +1
-1@@ -28,7 +28,7 @@ class TestFileCapabilities(infra.basetest.BRTest): def test_run(self): img = os.path.join(self.builddir, "images", "rootfs.squashfs") - subprocess.call(["truncate", "-s", "%1M", img]) + infra.img_round_power2(img) self.emulator.boot(arch="armv7", kernel=os.path.join(self.builddir, "images", "zImage"),support/testing/tests/fs/test_squashfs.py +0
-0
support/testing/tests/fs/test_squashfs.py class TestSquashfs(infra.basetest.BRTest): · line 25
Quoted 5 lines
support/testing/tests/fs/test_squashfs.py +1
-1@@ -22,7 +22,7 @@ class TestSquashfs(infra.basetest.BRTest): self.assertEqual(out[3], "Compression lz4") img = os.path.join(self.builddir, "images", "rootfs.squashfs") - subprocess.call(["truncate", "-s", "%1M", img]) + infra.img_round_power2(img)
Not the fault of this patch, but still your fault: this squashfs test no longer works after the kernel update in 3cf2782906d5...
I temporarily switched it back to using gxip to test it (but did not commit that change).
Applied to master, thanks.
Regards,
Yann E. MORIN.
Quoted 22 lines
support/testing/tests/fs/test_squashfs.py +0
-0self.emulator.boot(arch="armv7", kernel="builtin",support/testing/tests/init/base.py +1
-1@@ -7,7 +7,7 @@ class InitSystemBase(infra.basetest.BRTest): def start_emulator(self, fs_type, kernel=None, dtb=None, init=None): img = os.path.join(self.builddir, "images", "rootfs.{}".format(fs_type)) - subprocess.call(["truncate", "-s", "%1M", img]) + infra.img_round_power2(img) options = ["-drive", "file={},if=sd,format=raw".format(img),Signature · 6 lines
2.31.1 _______________________________________________ buildroot mailing list buildroot at busybox.net http://lists.busybox.net/mailman/listinfo/buildroot
Signature · 6 lines
.-----------------.--------------------.------------------.--------------------. | Yann E. MORIN | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: | | +33 662 376 056 | Software Designer | \ / CAMPAIGN | ___ | | +33 561 099 427 `------------.-------: X AGAINST | \e/ There is no | | http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL | v conspiracy. | '------------------------------^-------^------------------^--------------------'
PATCH 4/4 support/testing/tests/fs/test_ext: add missing "format" option for Qemu
In reply to Thomas PetazzoniWill avoid the following warning:
WARNING: Image format was not specified for '/home/thomas/projets/outputs/TestExt3/images/rootfs.ext3' and probing guessed raw. Automatically detecting the format is dangerous for raw images, write operations on block 0 will be restricted. Specify the 'raw' format explicitly to remove the restrictions.
--- support/testing/tests/fs/test_ext.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
support/testing/tests/fs/test_ext.py +1 -1
@@ -31,7 +31,7 @@ def boot_img_and_check_fs_type(emulator, builddir, fs_type): kernel="builtin", kernel_cmdline=["root=/dev/mmcblk0", "rootfstype={}".format(fs_type)], - options=["-drive", "file={},if=sd".format(img)]) + options=["-drive", "file={},if=sd,format=raw".format(img)]) emulator.login() _, exit_code = emulator.run(CHECK_FS_TYPE_CMD.format(fs_type)) return exit_code
Signature · 1 line
2.31.1
Yann E. MORIN
AppliedPATCH 4/4 support/testing/tests/fs/test_ext: add missing "format" option for Qemu
Thomas, All,
On 2021-06-21 22:25 +0200, Thomas Petazzoni spake thusly:
Quoted 10 lines
Will avoid the following warning:
WARNING: Image format was not specified for '
/home/thomas/projets/outputs/TestExt3/images/rootfs.ext3' and probing guessed raw. Automatically detecting the format is dangerous for raw images, write operations on block 0 will be restricted. Specify the 'raw' format explicitly to remove the restrictions.Signed-off-by Thomas Petazzoni <[email protected]>
Applied to master, thanks.
Regards,
Yann E. MORIN.
Quoted 24 lines
--- support/testing/tests/fs/test_ext.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)support/testing/tests/fs/test_ext.py +1
-1@@ -31,7 +31,7 @@ def boot_img_and_check_fs_type(emulator, builddir, fs_type): kernel="builtin", kernel_cmdline=["root=/dev/mmcblk0", "rootfstype={}".format(fs_type)], - options=["-drive", "file={},if=sd".format(img)]) + options=["-drive", "file={},if=sd,format=raw".format(img)]) emulator.login() _, exit_code = emulator.run(CHECK_FS_TYPE_CMD.format(fs_type)) return exit_codeSignature · 6 lines
2.31.1 _______________________________________________ buildroot mailing list buildroot at busybox.net http://lists.busybox.net/mailman/listinfo/buildroot
Signature · 6 lines
.-----------------.--------------------.------------------.--------------------. | Yann E. MORIN | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: | | +33 662 376 056 | Software Designer | \ / CAMPAIGN | ___ | | +33 561 099 427 `------------.-------: X AGAINST | \e/ There is no | | http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL | v conspiracy. | '------------------------------^-------^------------------^--------------------'
Yann E. MORIN
AppliedPATCH 1/4 support/testing/tests/fs/test_ext: specify 16 MB as ext filesystem size
In reply to Thomas PetazzoniThomas, All,
On 2021-06-21 22:25 +0200, Thomas Petazzoni spake thusly:
Quoted 8 lines
Since Qemu 5.1, the SD card size must be a power of two, so the default size for ext2/3/4 filesystem images of 60 MB is not suitable. Since 16 MB is used for the Ext4 test, let's use the same value for the other tests as well (ext2, ext2r1, ext3). Without this change, the ext2, ext2r1 and ext3 simply fail to run under Qemu >= 5.1.
Signed-off-by Thomas Petazzoni <[email protected]>
Applied to master, thanks.
Regards,
Yann E. MORIN.
Quoted 39 lines
--- support/testing/tests/fs/test_ext.py | 3 +++ 1 file changed, 3 insertions(+)support/testing/tests/fs/test_ext.py +3
-0@@ -43,6 +43,7 @@ class TestExt2(infra.basetest.BRTest): BR2_TARGET_ROOTFS_EXT2=y BR2_TARGET_ROOTFS_EXT2_2r0=y BR2_TARGET_ROOTFS_EXT2_LABEL="foobaz" + BR2_TARGET_ROOTFS_EXT2_SIZE="16384" # BR2_TARGET_ROOTFS_TAR is not set """@@ -62,6 +63,7 @@ class TestExt2r1(infra.basetest.BRTest): BR2_TARGET_ROOTFS_EXT2=y BR2_TARGET_ROOTFS_EXT2_2r1=y BR2_TARGET_ROOTFS_EXT2_LABEL="foobar" + BR2_TARGET_ROOTFS_EXT2_SIZE="16384" # BR2_TARGET_ROOTFS_TAR is not set """@@ -81,6 +83,7 @@ class TestExt3(infra.basetest.BRTest): """ BR2_TARGET_ROOTFS_EXT2=y BR2_TARGET_ROOTFS_EXT2_3=y + BR2_TARGET_ROOTFS_EXT2_SIZE="16384" # BR2_TARGET_ROOTFS_TAR is not set """Signature · 6 lines
2.31.1 _______________________________________________ buildroot mailing list buildroot at busybox.net http://lists.busybox.net/mailman/listinfo/buildroot
Signature · 6 lines
.-----------------.--------------------.------------------.--------------------. | Yann E. MORIN | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: | | +33 662 376 056 | Software Designer | \ / CAMPAIGN | ___ | | +33 561 099 427 `------------.-------: X AGAINST | \e/ There is no | | http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL | v conspiracy. | '------------------------------^-------^------------------^--------------------'