Skip to contentSkip to conversation

user-selectable wic format

PATCH Superseded Trevor Woerner 2 participants 5 messages Jun 11, 2024 Arm Rockchip

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 user-selectable wic format

Allow the user to choose their preferred wic image format.

---
 conf/machine/include/rockchip-wic.inc | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
conf/machine/include/rockchip-wic.inc +2 -1
index dab61d83ed2c..eb895cd0b4ad 100644
@@ -5,7 +5,8 @@ require conf/machine/include/rockchip-rk-u-boot-env.inc
 
 SPL_BINARY ?= "idbloader.img"
 
-IMAGE_FSTYPES += "wic wic.bmap"
+WIC_FSTYPE ?= "wic"
+IMAGE_FSTYPES += "${WIC_FSTYPE} wic.bmap"
 WKS_FILE ?= "rockchip.wks"
 WKS_FILE_DEPENDS ?= " \
 	e2fsprogs-native \
Signature · 1 line
2.44.0.478.g7774cfed6261
#1 lore raw

Hi Trevor,

Quoted 2 lines · Trevor Woerner via lists.yoctoproject.org

Allow the user to choose their preferred wic image format.

Can you provide some use case for this?

Quoted 10 lines
---
  conf/machine/include/rockchip-wic.inc | 3 ++-
  1 file changed, 2 insertions(+), 1 deletion(-)
conf/machine/include/rockchip-wic.inc +0 -0
index dab61d83ed2c..eb895cd0b4ad 100644

conf/machine/include/rockchip-wic.inc require conf/machine/include/rockchip-rk-u-boot-env.inc · line 9

Quoted 6 lines
conf/machine/include/rockchip-wic.inc +2 -1
@@ -5,7 +5,8 @@ require conf/machine/include/rockchip-rk-u-boot-env.inc
 
  SPL_BINARY ?= "idbloader.img"
 
-IMAGE_FSTYPES += "wic wic.bmap"
+WIC_FSTYPE ?= "wic"
+IMAGE_FSTYPES += "${WIC_FSTYPE} wic.bmap"

Would this be a way to NOT have wic in IMAGE_FSTYPES? What are we trying to achieve here?

If so, shouldn't we also not build wic.bmap there?

Cheers,
Quentin

#2 lore raw
Quoted 7 lines · Quentin Schulz via lists.yoctoproject.org

Hi Trevor,

Quoted 2 lines · Trevor Woerner via lists.yoctoproject.org

Allow the user to choose their preferred wic image format.

Can you provide some use case for this?

In order to build a wic.xz image the build has to first create a wic image, then compress it as an extra step. On a slow machine this extra step takes a noticeable amount of time.

On my local, slow build machine I prefer wic images since it saves build time not having to do the compression and since everything is local, there's no over-the-internet xfer time to consider.

When I build using some remote, fast build machine, the extra time spent doing the extra step of compressing is recuperated by the savings in transfer time retrieving the image from the remote. Therefore I prefer to make wic.xz images. Building wic images on remote machines would lead to very long xfer times to download the image artifact.

If I simply do:

	IMAGE_FSTYPES += "wic.xz"

in my conf/local.conf then MACHINEs that don't normally build wic images will fail. So the only way to build wic.xz images on remote builds is to tweak the meta-rockchip:conf/machine/include/rockchip-wic.inc file. So I'm always carrying this tweak for every remote build that I do.

So I could simply modify conf/machine/includes/rockchip-wic.inc to set it to wic.xz but that wouldn't suit my use-case since I could then have to tweak it again for any local builds. Besides, I could never guess what preferred wic version others would prefer and I don't want to be changing it every other week.

So this patch maintains the existing behaviour completely, and when I do a build on a remote machine I can simply set

WIC_FSTYPE = "wic.xz"

in my conf/local.conf without having to tweak the layer.

And if others want compression but would prefer bz2 or gzip (or whatever) then then can specify it however they wish.

Quoted 18 lines
Quoted 10 lines
---
  conf/machine/include/rockchip-wic.inc | 3 ++-
  1 file changed, 2 insertions(+), 1 deletion(-)
conf/machine/include/rockchip-wic.inc +0 -0
index dab61d83ed2c..eb895cd0b4ad 100644

conf/machine/include/rockchip-wic.inc require conf/machine/include/rockchip-rk-u-boot-env.inc · line 7

Quoted 4 lines
conf/machine/include/rockchip-wic.inc +2 -1
@@ -5,7 +5,8 @@ require conf/machine/include/rockchip-rk-u-boot-env.inc
  SPL_BINARY ?= "idbloader.img"
-IMAGE_FSTYPES += "wic wic.bmap"
+WIC_FSTYPE ?= "wic"
+IMAGE_FSTYPES += "${WIC_FSTYPE} wic.bmap"

Would this be a way to NOT have wic in IMAGE_FSTYPES? What are we trying to achieve here?

Quoted 1 line

If so, shouldn't we also not build wic.bmap there?

The goal isn't to not build a wic image, the point is to allow the user to specify which type of wic image (i.e. with or without compression and if with compression, which one?).

I guess I could do:

	WIC_COMPRESSION ?= ""
	IMAGE_FSTYPES += "wic${WIC_COMPRESSION} wic.bmap"

And then specify:

WIC_COMPRESSION = ".xz"

in my conf/local.conf?

#3 lore raw

Hi Trevor,

Quoted 74 lines · Trevor Woerner via lists.yoctoproject.org
Quoted 7 lines · Quentin Schulz via lists.yoctoproject.org

Hi Trevor,

Quoted 2 lines · Trevor Woerner via lists.yoctoproject.org

Allow the user to choose their preferred wic image format.

Can you provide some use case for this?

In order to build a wic.xz image the build has to first create a wic image, then compress it as an extra step. On a slow machine this extra step takes a noticeable amount of time.

On my local, slow build machine I prefer wic images since it saves build time not having to do the compression and since everything is local, there's no over-the-internet xfer time to consider.

When I build using some remote, fast build machine, the extra time spent doing the extra step of compressing is recuperated by the savings in transfer time retrieving the image from the remote. Therefore I prefer to make wic.xz images. Building wic images on remote machines would lead to very long xfer times to download the image artifact.

If I simply do:

	IMAGE_FSTYPES += "wic.xz"

in my conf/local.conf then MACHINEs that don't normally build wic images will fail. So the only way to build wic.xz images on remote builds is to tweak the meta-rockchip:conf/machine/include/rockchip-wic.inc file. So I'm always carrying this tweak for every remote build that I do.

So I could simply modify conf/machine/includes/rockchip-wic.inc to set it to wic.xz but that wouldn't suit my use-case since I could then have to tweak it again for any local builds. Besides, I could never guess what preferred wic version others would prefer and I don't want to be changing it every other week.

So this patch maintains the existing behaviour completely, and when I do a build on a remote machine I can simply set

WIC_FSTYPE = "wic.xz"

in my conf/local.conf without having to tweak the layer.

And if others want compression but would prefer bz2 or gzip (or whatever) then then can specify it however they wish.

Quoted 18 lines
Quoted 10 lines
---
   conf/machine/include/rockchip-wic.inc | 3 ++-
   1 file changed, 2 insertions(+), 1 deletion(-)
conf/machine/include/rockchip-wic.inc +0 -0
index dab61d83ed2c..eb895cd0b4ad 100644

conf/machine/include/rockchip-wic.inc require conf/machine/include/rockchip-rk-u-boot-env.inc · line 7

Quoted 4 lines
conf/machine/include/rockchip-wic.inc +2 -1
@@ -5,7 +5,8 @@ require conf/machine/include/rockchip-rk-u-boot-env.inc
   SPL_BINARY ?= "idbloader.img"
-IMAGE_FSTYPES += "wic wic.bmap"
+WIC_FSTYPE ?= "wic"
+IMAGE_FSTYPES += "${WIC_FSTYPE} wic.bmap"

Would this be a way to NOT have wic in IMAGE_FSTYPES? What are we trying to achieve here?

Quoted 1 line

If so, shouldn't we also not build wic.bmap there?

The goal isn't to not build a wic image, the point is to allow the user to specify which type of wic image (i.e. with or without compression and if with compression, which one?).

I guess I could do:

Quoted 9 lines
	WIC_COMPRESSION ?= ""
	IMAGE_FSTYPES += "wic${WIC_COMPRESSION} wic.bmap"

And then specify:

WIC_COMPRESSION = ".xz"

in my conf/local.conf?

Thanks for taking the time to explain your use case :)

I was about to suggest this :)

Should we allow multiple values in WIC_COMPRESSION?

Also maybe it should be named "EXTENSION" instead since we have the dot in there?

Additionally, I don't see wic.xz in IMAGE_TYPES in https://git.openembedded.org/openembedded-core/tree/meta/classes-recipe/image_types.bbclass, is xz really working? If so, why do we need to specify wic.gz and all the others in IMAGE_TYPES? Maybe we do not?

Cheers,
Quentin

#4 lore raw
Quoted 87 lines · Quentin Schulz via lists.yoctoproject.org

Hi Trevor,

Quoted 74 lines · Trevor Woerner via lists.yoctoproject.org
Quoted 7 lines · Quentin Schulz via lists.yoctoproject.org

Hi Trevor,

Quoted 2 lines · Trevor Woerner via lists.yoctoproject.org

Allow the user to choose their preferred wic image format.

Can you provide some use case for this?

In order to build a wic.xz image the build has to first create a wic image, then compress it as an extra step. On a slow machine this extra step takes a noticeable amount of time.

On my local, slow build machine I prefer wic images since it saves build time not having to do the compression and since everything is local, there's no over-the-internet xfer time to consider.

When I build using some remote, fast build machine, the extra time spent doing the extra step of compressing is recuperated by the savings in transfer time retrieving the image from the remote. Therefore I prefer to make wic.xz images. Building wic images on remote machines would lead to very long xfer times to download the image artifact.

If I simply do:

	IMAGE_FSTYPES += "wic.xz"

in my conf/local.conf then MACHINEs that don't normally build wic images will fail. So the only way to build wic.xz images on remote builds is to tweak the meta-rockchip:conf/machine/include/rockchip-wic.inc file. So I'm always carrying this tweak for every remote build that I do.

So I could simply modify conf/machine/includes/rockchip-wic.inc to set it to wic.xz but that wouldn't suit my use-case since I could then have to tweak it again for any local builds. Besides, I could never guess what preferred wic version others would prefer and I don't want to be changing it every other week.

So this patch maintains the existing behaviour completely, and when I do a build on a remote machine I can simply set

WIC_FSTYPE = "wic.xz"

in my conf/local.conf without having to tweak the layer.

And if others want compression but would prefer bz2 or gzip (or whatever) then then can specify it however they wish.

Quoted 18 lines
Quoted 10 lines
---
   conf/machine/include/rockchip-wic.inc | 3 ++-
   1 file changed, 2 insertions(+), 1 deletion(-)
conf/machine/include/rockchip-wic.inc +0 -0
index dab61d83ed2c..eb895cd0b4ad 100644

conf/machine/include/rockchip-wic.inc require conf/machine/include/rockchip-rk-u-boot-env.inc · line 7

Quoted 4 lines
conf/machine/include/rockchip-wic.inc +2 -1
@@ -5,7 +5,8 @@ require conf/machine/include/rockchip-rk-u-boot-env.inc
   SPL_BINARY ?= "idbloader.img"
-IMAGE_FSTYPES += "wic wic.bmap"
+WIC_FSTYPE ?= "wic"
+IMAGE_FSTYPES += "${WIC_FSTYPE} wic.bmap"

Would this be a way to NOT have wic in IMAGE_FSTYPES? What are we trying to achieve here?

Quoted 1 line

If so, shouldn't we also not build wic.bmap there?

The goal isn't to not build a wic image, the point is to allow the user to specify which type of wic image (i.e. with or without compression and if with compression, which one?).

I guess I could do:

Quoted 9 lines
	WIC_COMPRESSION ?= ""
	IMAGE_FSTYPES += "wic${WIC_COMPRESSION} wic.bmap"

And then specify:

WIC_COMPRESSION = ".xz"

in my conf/local.conf?

Quoted 8 lines

Thanks for taking the time to explain your use case :)

I was about to suggest this :)

Should we allow multiple values in WIC_COMPRESSION?

Also maybe it should be named "EXTENSION" instead since we have the dot in there?

Sounds good.

Quoted 2 lines

Additionally, I don't see wic.xz in IMAGE_TYPES in https://git.openembedded.org/openembedded-core/tree/meta/classes-recipe/image_types.bbclass,
is xz really working?

I have been using "wic.xz" in IMAGE_FSTYPES for years. Works with bmaptool too.

Quoted 2 lines

If so, why do we need to specify wic.gz and all the others in IMAGE_TYPES? Maybe we do not?

I guess that's for others to answer/investigate.