IMAGE_ROOTFS_SIZE Cleanup

RFC In review Saul Wold 6 participants 20 messages May 24, 2011 – Jun 2, 2011

lore

Share & tools

lore

Saul Wold

author

2011-05-24 06:42 UTC #0 lore raw

cover IMAGE_ROOTFS_SIZE Cleanup

This RFC is still in the works, I would like to have folks look at it.

Previously, IMAGE_ROOTFS_SIZE was being set with _ext2 or _ext3 in various places, this caused problem, because it was not easily overriden. By removing all the overrides, it allows the default mechanism to work correctly.

Currently that default mechanism takes a du of the rootfs image and then adds in an overhead factor (IMAGE_OVERHEAD_FACTOR) or 1.3x of the du size, in most cases this should be sufficent. I am still testing this, and I heard that there might be a post-install issue with RPM DB being larger.

This additional addresses a couple of other issues, by clearing all the bad overrides, it allows Image or Layer overrides to work correctly either by setting IMAGE_ROOTFS_SIZE to some arbitray number or by allowing some extra space be added above the existing overhead via the IMAGE_ROOTFS_EXTRA_SPACE option.

The base default is now 64M (maybe I need to only set the one in bitbake.conf, comments abou tthat?) with the minimal images base as 8M (currently qemux86 sizes up to 9.9M used on an 11M rootfs).

There will be a corresponding patch for meta-yocto to remove the IMAGE_ROOTFS_SIZE defaults in atom-pc.conf and routerstationpro.conf

I welcome comments.

Thanks
Sau!

The following changes since commit ad72ffadca975166cd8a416e4bf7b7fe33616d39:

Merge branch 'master', remote-tracking branch 'origin/master' (2011-05-23 23:00:20 +0100)

are available in the git repository at:

git://git.openembedded.org/openembedded-core-contrib.git sgw/rootfs_size
http://cgit.openembedded.org/cgit.cgi/openembedded-core-contrib/log/?h=sgw/rootfs_size

Saul Wold (2):
IMAGE_ROOTFS_SIZE Cleanup
image_types: add IMAGE_ROOTFS_EXTRA_SPACE

 meta/classes/image_types.bbclass                   |    9 ++++++---
 meta/conf/bitbake.conf                             |    4 ++++
 meta/conf/distro/include/default-distrovars.inc    |    2 --
 meta/conf/machine/include/qemu.inc                 |    2 --
 .../images/core-image-minimal-initramfs.bb         |    2 ++
 meta/recipes-core/images/core-image-minimal.bb     |    2 ++
 6 files changed, 14 insertions(+), 7 deletions(-)

Signature · 1 line

1.7.3.4

Saul Wold

author

2011-05-24 06:42 UTC #1 lore raw

RFC 1/2 IMAGE_ROOTFS_SIZE Cleanup

This basic cleanup removes the _ext2/3 overrides from places they no longer belong since they did not allow further overrides. In doing this the core-image-minimal* recipes can now set a reasonably small rootfs so that it's a realistic size for minimal.

The new default for minimal is 8M and will be adujsted upward by the IMAGE_OVERHEAD_FACTOR (of 1.2).

This fixes the ROOTFS_SIZE usage in the IMAGE_CMD_<fstype> code

Signed-off-by Saul Wold <[email protected]>

---
 meta/classes/image_types.bbclass                   |    7 +++++--
 meta/conf/distro/include/default-distrovars.inc    |    2 --
 meta/conf/machine/include/qemu.inc                 |    2 --
 .../images/core-image-minimal-initramfs.bb         |    2 ++
 meta/recipes-core/images/core-image-minimal.bb     |    2 ++
 5 files changed, 9 insertions(+), 6 deletions(-)

meta/classes/image_types.bbclass +5 -2

index ec0cafd..69f859e 100644
@@ -21,22 +21,25 @@ runimagecmd () {
 }
 
 IMAGE_CMD_jffs2 = "mkfs.jffs2 --root=${IMAGE_ROOTFS} --faketime --output=${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.jffs2 ${EXTRA_IMAGECMD}"
+
 IMAGE_CMD_cramfs = "mkcramfs ${IMAGE_ROOTFS} ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.cramfs ${EXTRA_IMAGECMD}"
+
 IMAGE_CMD_ext2 = "genext2fs -b $ROOTFS_SIZE -d ${IMAGE_ROOTFS} ${EXTRA_IMAGECMD} ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.ext2"
 IMAGE_CMD_ext2.gz () {
 	rm -rf ${DEPLOY_DIR_IMAGE}/tmp.gz && mkdir ${DEPLOY_DIR_IMAGE}/tmp.gz
-	genext2fs -b ${IMAGE_ROOTFS_SIZE} -d ${IMAGE_ROOTFS} ${EXTRA_IMAGECMD} ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext2
+	genext2fs -b $ROOTFS_SIZE -d ${IMAGE_ROOTFS} ${EXTRA_IMAGECMD} ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext2
 	gzip -f -9 ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext2
 	mv ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext2.gz ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.ext2.gz
 	rmdir ${DEPLOY_DIR_IMAGE}/tmp.gz
 }
+
 IMAGE_CMD_ext3 () {
 	genext2fs -b $ROOTFS_SIZE -d ${IMAGE_ROOTFS} ${EXTRA_IMAGECMD} ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.ext3
 	tune2fs -j ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.ext3
 }
 IMAGE_CMD_ext3.gz () {
 	rm -rf ${DEPLOY_DIR_IMAGE}/tmp.gz && mkdir ${DEPLOY_DIR_IMAGE}/tmp.gz
-	genext2fs -b ${IMAGE_ROOTFS_SIZE} -d ${IMAGE_ROOTFS} ${EXTRA_IMAGECMD} ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext3
+	genext2fs -b $ROOTFS_SIZE -d ${IMAGE_ROOTFS} ${EXTRA_IMAGECMD} ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext3
 	tune2fs -j ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext3
 	gzip -f -9 ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext3
 	mv ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext3.gz ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.ext3.gz

meta/conf/distro/include/default-distrovars.inc +0 -2

index 1aa45c8..4b68a0a 100644
@@ -1,7 +1,5 @@
 QA_LOGFILE = "${TMPDIR}/qa.log"
 
-IMAGE_ROOTFS_SIZE_ext2 ?= "131072"
-
 OEINCLUDELOGS ?= "yes"
 KERNEL_CONSOLE ?= "ttyS0"

meta/conf/machine/include/qemu.inc +0 -2

index 4122a88..9ef242f 100644
@@ -6,8 +6,6 @@ MACHINE_FEATURES = "kernel26 apm alsa pcmcia bluetooth irda usbgadget screen"
 IMAGE_FSTYPES ?= "tar.bz2 ext3"
 
 ROOT_FLASH_SIZE = "280"
-IMAGE_ROOTFS_SIZE_ext2 ?= "280000"
-IMAGE_ROOTFS_SIZE_ext3 ?= "280000"
 
 # Don't include kernels in standard images
 RDEPENDS_kernel-base = ""

meta/recipes-core/images/core-image-minimal-initramfs.bb +2 -0

index 21aaa6c..3246d5c 100644
@@ -8,3 +8,5 @@ IMAGE_LINGUAS = ""
 LICENSE = "MIT"
 
 inherit core-image
+
+IMAGE_ROOTFS_SIZE = "8192"

meta/recipes-core/images/core-image-minimal.bb +2 -0

index aa00e67..743e121 100644
@@ -9,5 +9,7 @@ LICENSE = "MIT"
 
 inherit core-image
 
+IMAGE_ROOTFS_SIZE = "8192"
+
 # remove not needed ipkg informations
 ROOTFS_POSTPROCESS_COMMAND += "remove_packaging_data_files ; "

Signature · 1 line

1.7.3.4

Saul Wold

author

2011-05-24 06:42 UTC #16 lore raw

RFC 2/2 image_types: add IMAGE_ROOTFS_EXTRA_SPACE

In reply to Saul Wold

[YOCTO #886]

This address 2 needs after the IMAGE_ROOTFS_SIZE is cleaned up, by removing some of the _ext2/3 overrides it allows for a cleaner override using IMAGE_ROOTFS_SIZE to create a large rootfs, or by setting the IMAGE_ROOTFS_EXTRA_SPACE, will allow for extra space allocated in Kilobytes above the base size (determined by du of the rootfs * IMAGE_OVERHEAD_FACTOR, default of 1.3).

Signed-off-by Saul Wold <[email protected]>

---
 meta/classes/image_types.bbclass |    2 +-
 meta/conf/bitbake.conf           |    4 ++++
 2 files changed, 5 insertions(+), 1 deletions(-)

meta/classes/image_types.bbclass +1 -1

index 69f859e..601a5fb 100644
@@ -13,7 +13,7 @@ def get_imagecmds(d):
 
 runimagecmd () {
 	# Image generation code for image type ${type}
-	ROOTFS_SIZE=`du -ks ${IMAGE_ROOTFS}|awk '{size = $1 * ${IMAGE_OVERHEAD_FACTOR}; print (size > ${IMAGE_ROOTFS_SIZE} ? size : ${IMAGE_ROOTFS_SIZE}) }'`
+	ROOTFS_SIZE=`du -ks ${IMAGE_ROOTFS}|awk '{size = $1 * ${IMAGE_OVERHEAD_FACTOR} + ${IMAGE_ROOTFS_EXTRA_SPACE}; print (size > ${IMAGE_ROOTFS_SIZE} ? size : ${IMAGE_ROOTFS_SIZE}) }'`
 	${cmd}
 	cd ${DEPLOY_DIR_IMAGE}/
 	rm -f ${DEPLOY_DIR_IMAGE}/${IMAGE_LINK_NAME}.${type}

meta/conf/bitbake.conf +4 -0

index a0af672..fa3618c 100644
@@ -350,6 +350,10 @@ IMAGE_LINK_NAME = "${IMAGE_BASENAME}-${MACHINE}"
 # This option allows for a precentage overage of the actaul image size rather than a
 # fixed extra space
 IMAGE_OVERHEAD_FACTOR ?= 1.3
+# This option allow for adding addition space in K above and beyond what the 
+# IMAGE_OVERHEAD_FACTOR might add. To set a fixed size then overrideing IMAGE_ROOTFS_SIZE
+# with the max size one wants should do the trick
+IMAGE_OVERHEAD_EXTRA_SPACE = "0"
 
 EXTRA_IMAGEDEPENDS = ""

Signature · 1 line

1.7.3.4

Joshua Lock

2011-05-26 18:04 UTC #2 lore raw

In reply to Saul Wold

Quoted 4 lines · Saul Wold

This basic cleanup removes the _ext2/3 overrides from places they no longer belong since they did not allow further overrides. In doing this the core-image-minimal* recipes can now set a reasonably small rootfs so that it's a realistic size for minimal.

Awesome. Thanks for tackling this!

Few questions below.

Quoted 77 lines

The new default for minimal is 8M and will be adujsted upward by the IMAGE_OVERHEAD_FACTOR (of 1.2).

This fixes the ROOTFS_SIZE usage in the IMAGE_CMD_<fstype> code

Signed-off-by Saul Wold <[email protected]>

---
 meta/classes/image_types.bbclass                   |    7 +++++--
 meta/conf/distro/include/default-distrovars.inc    |    2 --
 meta/conf/machine/include/qemu.inc                 |    2 --
 .../images/core-image-minimal-initramfs.bb         |    2 ++
 meta/recipes-core/images/core-image-minimal.bb     |    2 ++
 5 files changed, 9 insertions(+), 6 deletions(-)

meta/classes/image_types.bbclass +5 -2

index ec0cafd..69f859e 100644
@@ -21,22 +21,25 @@ runimagecmd () {
 }
 
 IMAGE_CMD_jffs2 = "mkfs.jffs2 --root=${IMAGE_ROOTFS} --faketime --output=${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.jffs2 ${EXTRA_IMAGECMD}"
+
 IMAGE_CMD_cramfs = "mkcramfs ${IMAGE_ROOTFS} ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.cramfs ${EXTRA_IMAGECMD}"
+
 IMAGE_CMD_ext2 = "genext2fs -b $ROOTFS_SIZE -d ${IMAGE_ROOTFS} ${EXTRA_IMAGECMD} ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.ext2"
 IMAGE_CMD_ext2.gz () {
 	rm -rf ${DEPLOY_DIR_IMAGE}/tmp.gz && mkdir ${DEPLOY_DIR_IMAGE}/tmp.gz
-	genext2fs -b ${IMAGE_ROOTFS_SIZE} -d ${IMAGE_ROOTFS} ${EXTRA_IMAGECMD} ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext2
+	genext2fs -b $ROOTFS_SIZE -d ${IMAGE_ROOTFS} ${EXTRA_IMAGECMD} ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext2
 	gzip -f -9 ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext2
 	mv ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext2.gz ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.ext2.gz
 	rmdir ${DEPLOY_DIR_IMAGE}/tmp.gz
 }
+
 IMAGE_CMD_ext3 () {
 	genext2fs -b $ROOTFS_SIZE -d ${IMAGE_ROOTFS} ${EXTRA_IMAGECMD} ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.ext3
 	tune2fs -j ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.ext3
 }
 IMAGE_CMD_ext3.gz () {
 	rm -rf ${DEPLOY_DIR_IMAGE}/tmp.gz && mkdir ${DEPLOY_DIR_IMAGE}/tmp.gz
-	genext2fs -b ${IMAGE_ROOTFS_SIZE} -d ${IMAGE_ROOTFS} ${EXTRA_IMAGECMD} ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext3
+	genext2fs -b $ROOTFS_SIZE -d ${IMAGE_ROOTFS} ${EXTRA_IMAGECMD} ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext3
 	tune2fs -j ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext3
 	gzip -f -9 ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext3
 	mv ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext3.gz ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.ext3.gz

meta/conf/distro/include/default-distrovars.inc +0 -2

index 1aa45c8..4b68a0a 100644
@@ -1,7 +1,5 @@
 QA_LOGFILE = "${TMPDIR}/qa.log"
 
-IMAGE_ROOTFS_SIZE_ext2 ?= "131072"
-
 OEINCLUDELOGS ?= "yes"
 KERNEL_CONSOLE ?= "ttyS0"

meta/conf/machine/include/qemu.inc +0 -2

index 4122a88..9ef242f 100644
@@ -6,8 +6,6 @@ MACHINE_FEATURES = "kernel26 apm alsa pcmcia bluetooth irda usbgadget screen"
 IMAGE_FSTYPES ?= "tar.bz2 ext3"
 
 ROOT_FLASH_SIZE = "280"
-IMAGE_ROOTFS_SIZE_ext2 ?= "280000"
-IMAGE_ROOTFS_SIZE_ext3 ?= "280000"
 
 # Don't include kernels in standard images
 RDEPENDS_kernel-base = ""

meta/recipes-core/images/core-image-minimal-initramfs.bb +0 -0

index 21aaa6c..3246d5c 100644

meta/recipes-core/images/core-image-minimal-initramfs.bb IMAGE_LINGUAS = "" · line 12

Quoted 5 lines

meta/recipes-core/images/core-image-minimal-initramfs.bb +2 -0
@@ -8,3 +8,5 @@ IMAGE_LINGUAS = ""
 LICENSE = "MIT"
 
 inherit core-image
+
+IMAGE_ROOTFS_SIZE = "8192"

I'm not really sure about this, an initramfs that's the same size as a more fully featured rootfs?

meta/recipes-core/images/core-image-minimal.bb LICENSE = "MIT" · line 13

Quoted 10 lines

meta/recipes-core/images/core-image-minimal.bb +2 -0

index aa00e67..743e121 100644
@@ -9,5 +9,7 @@ LICENSE = "MIT"
 
 inherit core-image
 
+IMAGE_ROOTFS_SIZE = "8192"
+

In your cover letter you stated that the minimal image is currently 9.9M, which means we *need* the overhead to contain the entire image contents. Correct? That seems a little unwise.

Quoted 2 lines

meta/recipes-core/images/core-image-minimal.bb +0 -0
 # remove not needed ipkg informations
 ROOTFS_POSTPROCESS_COMMAND += "remove_packaging_data_files ; "

Signature · 3 lines

Joshua Lock
        Yocto Project Build Monkey
        Intel Open Source Technology Centre

Joshua Lock

2011-05-26 18:04 UTC #17 lore raw

Re: [RFC 2/2] image_types: add IMAGE_ROOTFS_EXTRA_SPACE

In reply to Saul Wold

Quoted 33 lines · Saul Wold

[YOCTO #886]
This address 2 needs after the IMAGE_ROOTFS_SIZE is cleaned up, by removing some of the _ext2/3 overrides it allows for a cleaner override using IMAGE_ROOTFS_SIZE to create a large rootfs, or by setting the IMAGE_ROOTFS_EXTRA_SPACE, will allow for extra space allocated in Kilobytes above the base size (determined by du of the rootfs * IMAGE_OVERHEAD_FACTOR, default of 1.3).
Signed-off-by Saul Wold <[email protected]>
---
 meta/classes/image_types.bbclass |    2 +-
 meta/conf/bitbake.conf           |    4 ++++
 2 files changed, 5 insertions(+), 1 deletions(-)
meta/classes/image_types.bbclass +1 -1
index 69f859e..601a5fb 100644
@@ -13,7 +13,7 @@ def get_imagecmds(d):
 
 runimagecmd () {
 	# Image generation code for image type ${type}
-	ROOTFS_SIZE=`du -ks ${IMAGE_ROOTFS}|awk '{size = $1 * ${IMAGE_OVERHEAD_FACTOR}; print (size > ${IMAGE_ROOTFS_SIZE} ? size : ${IMAGE_ROOTFS_SIZE}) }'`
+	ROOTFS_SIZE=`du -ks ${IMAGE_ROOTFS}|awk '{size = $1 * ${IMAGE_OVERHEAD_FACTOR} + ${IMAGE_ROOTFS_EXTRA_SPACE}; print (size > ${IMAGE_ROOTFS_SIZE} ? size : ${IMAGE_ROOTFS_SIZE}) }'`
 	${cmd}
 	cd ${DEPLOY_DIR_IMAGE}/
 	rm -f ${DEPLOY_DIR_IMAGE}/${IMAGE_LINK_NAME}.${type}
meta/conf/bitbake.conf +0 -0
index a0af672..fa3618c 100644

meta/conf/bitbake.conf IMAGE_LINK_NAME = "${IMAGE_BASENAME}-${MACHINE}" · line 356

Quoted 7 lines

meta/conf/bitbake.conf +4 -0

@@ -350,6 +350,10 @@ IMAGE_LINK_NAME = "${IMAGE_BASENAME}-${MACHINE}"
 # This option allows for a precentage overage of the actaul image size rather than a
 # fixed extra space
 IMAGE_OVERHEAD_FACTOR ?= 1.3
+# This option allow for adding addition space in K above and beyond what the 
+# IMAGE_OVERHEAD_FACTOR might add. To set a fixed size then overrideing IMAGE_ROOTFS_SIZE
+# with the max size one wants should do the trick
+IMAGE_OVERHEAD_EXTRA_SPACE = "0"

This appears to be a different variable to the IMAGE_ROOTFS_EXTRA_SPACE used in the command above? Also, you've done a hard assignment here. Shouldn't it be a soft (?=) assignment?

Cheers,
Joshua

Signature · 3 lines

Joshua Lock
        Yocto Project Build Monkey
        Intel Open Source Technology Centre

Saul Wold

2011-05-26 18:25 UTC #18 lore raw

Re: [RFC 2/2] image_types: add IMAGE_ROOTFS_EXTRA_SPACE

Quoted 37 lines · Joshua Lock

Quoted 36 lines · Saul Wold
[YOCTO #886]
This address 2 needs after the IMAGE_ROOTFS_SIZE is cleaned up, by removing some of the _ext2/3 overrides it allows for a cleaner override using IMAGE_ROOTFS_SIZE to create a large rootfs, or by setting the IMAGE_ROOTFS_EXTRA_SPACE, will allow for extra space allocated in Kilobytes above the base size (determined by du of the rootfs * IMAGE_OVERHEAD_FACTOR, default of 1.3).
Signed-off-by Saul Wold<[email protected]>
---
  meta/classes/image_types.bbclass |    2 +-
  meta/conf/bitbake.conf           |    4 ++++
  2 files changed, 5 insertions(+), 1 deletions(-)
meta/classes/image_types.bbclass +1 -1
index 69f859e..601a5fb 100644
@@ -13,7 +13,7 @@ def get_imagecmds(d):
 
  runimagecmd () {
  	# Image generation code for image type ${type}
-	ROOTFS_SIZE=`du -ks ${IMAGE_ROOTFS}|awk '{size = $1 * ${IMAGE_OVERHEAD_FACTOR}; print (size>  ${IMAGE_ROOTFS_SIZE} ? size : ${IMAGE_ROOTFS_SIZE}) }'`
+	ROOTFS_SIZE=`du -ks ${IMAGE_ROOTFS}|awk '{size = $1 * ${IMAGE_OVERHEAD_FACTOR} + ${IMAGE_ROOTFS_EXTRA_SPACE}; print (size>  ${IMAGE_ROOTFS_SIZE} ? size : ${IMAGE_ROOTFS_SIZE}) }'`
  	${cmd}
  	cd ${DEPLOY_DIR_IMAGE}/
  	rm -f ${DEPLOY_DIR_IMAGE}/${IMAGE_LINK_NAME}.${type}
meta/conf/bitbake.conf +0 -0
index a0af672..fa3618c 100644
@@ -350,6 +350,10 @@ IMAGE_LINK_NAME = "${IMAGE_BASENAME}-${MACHINE}"
  # This option allows for a precentage overage of the actaul image size rather than a
  # fixed extra space
  IMAGE_OVERHEAD_FACTOR ?= 1.3

Quoted 9 lines

meta/conf/bitbake.conf IMAGE_LINK_NAME = "${IMAGE_BASENAME}-${MACHINE}" · line 356
Quoted 4 lines
meta/conf/bitbake.conf +4 -0
+# This option allow for adding addition space in K above and beyond what the
+# IMAGE_OVERHEAD_FACTOR might add. To set a fixed size then overrideing IMAGE_ROOTFS_SIZE
+# with the max size one wants should do the trick
+IMAGE_OVERHEAD_EXTRA_SPACE = "0"
This appears to be a different variable to the IMAGE_ROOTFS_EXTRA_SPACE used in the command above? Also, you've done a hard assignment here. Shouldn't it be a soft (?=) assignment?

Yes, I fixed the OVERHEAD vs ROOTFS already, noticed that during post RFC testing. I did not see the soft (?=) problem, but agree it needs to be fixed.

Sau!

Quoted 2 lines

Cheers,
Joshua

Saul Wold

author

2011-05-26 18:31 UTC #3 lore raw

In reply to Joshua Lock

Quoted 89 lines · Joshua Lock

Quoted 4 lines · Saul Wold

This basic cleanup removes the _ext2/3 overrides from places they no longer belong since they did not allow further overrides. In doing this the core-image-minimal* recipes can now set a reasonably small rootfs so that it's a realistic size for minimal.

Awesome. Thanks for tackling this!

Few questions below.

Quoted 79 lines

The new default for minimal is 8M and will be adujsted upward by the IMAGE_OVERHEAD_FACTOR (of 1.2).

This fixes the ROOTFS_SIZE usage in the IMAGE_CMD_<fstype> code

Signed-off-by Saul Wold<[email protected]>

---
  meta/classes/image_types.bbclass                   |    7 +++++--
  meta/conf/distro/include/default-distrovars.inc    |    2 --
  meta/conf/machine/include/qemu.inc                 |    2 --
  .../images/core-image-minimal-initramfs.bb         |    2 ++
  meta/recipes-core/images/core-image-minimal.bb     |    2 ++
  5 files changed, 9 insertions(+), 6 deletions(-)

meta/classes/image_types.bbclass +5 -2

index ec0cafd..69f859e 100644
@@ -21,22 +21,25 @@ runimagecmd () {
  }
 
  IMAGE_CMD_jffs2 = "mkfs.jffs2 --root=${IMAGE_ROOTFS} --faketime --output=${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.jffs2 ${EXTRA_IMAGECMD}"
+
  IMAGE_CMD_cramfs = "mkcramfs ${IMAGE_ROOTFS} ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.cramfs ${EXTRA_IMAGECMD}"
+
  IMAGE_CMD_ext2 = "genext2fs -b $ROOTFS_SIZE -d ${IMAGE_ROOTFS} ${EXTRA_IMAGECMD} ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.ext2"
  IMAGE_CMD_ext2.gz () {
  	rm -rf ${DEPLOY_DIR_IMAGE}/tmp.gz&&  mkdir ${DEPLOY_DIR_IMAGE}/tmp.gz
-	genext2fs -b ${IMAGE_ROOTFS_SIZE} -d ${IMAGE_ROOTFS} ${EXTRA_IMAGECMD} ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext2
+	genext2fs -b $ROOTFS_SIZE -d ${IMAGE_ROOTFS} ${EXTRA_IMAGECMD} ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext2
  	gzip -f -9 ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext2
  	mv ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext2.gz ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.ext2.gz
  	rmdir ${DEPLOY_DIR_IMAGE}/tmp.gz
  }
+
  IMAGE_CMD_ext3 () {
  	genext2fs -b $ROOTFS_SIZE -d ${IMAGE_ROOTFS} ${EXTRA_IMAGECMD} ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.ext3
  	tune2fs -j ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.ext3
  }
  IMAGE_CMD_ext3.gz () {
  	rm -rf ${DEPLOY_DIR_IMAGE}/tmp.gz&&  mkdir ${DEPLOY_DIR_IMAGE}/tmp.gz
-	genext2fs -b ${IMAGE_ROOTFS_SIZE} -d ${IMAGE_ROOTFS} ${EXTRA_IMAGECMD} ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext3
+	genext2fs -b $ROOTFS_SIZE -d ${IMAGE_ROOTFS} ${EXTRA_IMAGECMD} ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext3
  	tune2fs -j ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext3
  	gzip -f -9 ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext3
  	mv ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext3.gz ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.ext3.gz

meta/conf/distro/include/default-distrovars.inc +0 -2

index 1aa45c8..4b68a0a 100644
@@ -1,7 +1,5 @@
  QA_LOGFILE = "${TMPDIR}/qa.log"
 
-IMAGE_ROOTFS_SIZE_ext2 ?= "131072"
-
  OEINCLUDELOGS ?= "yes"
  KERNEL_CONSOLE ?= "ttyS0"

meta/conf/machine/include/qemu.inc +0 -2

index 4122a88..9ef242f 100644
@@ -6,8 +6,6 @@ MACHINE_FEATURES = "kernel26 apm alsa pcmcia bluetooth irda usbgadget screen"
  IMAGE_FSTYPES ?= "tar.bz2 ext3"
 
  ROOT_FLASH_SIZE = "280"
-IMAGE_ROOTFS_SIZE_ext2 ?= "280000"
-IMAGE_ROOTFS_SIZE_ext3 ?= "280000"
 
  # Don't include kernels in standard images
  RDEPENDS_kernel-base = ""

meta/recipes-core/images/core-image-minimal-initramfs.bb +0 -0

index 21aaa6c..3246d5c 100644
@@ -8,3 +8,5 @@ IMAGE_LINGUAS = ""
  LICENSE = "MIT"

Quoted 7 lines

meta/recipes-core/images/core-image-minimal-initramfs.bb IMAGE_LINGUAS = "" · line 12
Quoted 3 lines
meta/recipes-core/images/core-image-minimal-initramfs.bb +2 -0
  inherit core-image
+
+IMAGE_ROOTFS_SIZE = "8192"
I'm not really sure about this, an initramfs that's the same size as a more fully featured rootfs?

That may be, then we need to increase the size slightly, but this will trigger the correct behavior of actual size * IMAGE_OVERHEAD_FACTOR, rather than the 64M which would be the default, this was to ensure it could get smaller. I don't have a current initramfs size, I will build and verify.

Quoted 6 lines

Quoted 6 lines
meta/recipes-core/images/core-image-minimal.bb +0 -0
index aa00e67..743e121 100644
@@ -9,5 +9,7 @@ LICENSE = "MIT"
 

Quoted 9 lines

meta/recipes-core/images/core-image-minimal.bb LICENSE = "MIT" · line 13
Quoted 4 lines
meta/recipes-core/images/core-image-minimal.bb +2 -0
  inherit core-image
 
+IMAGE_ROOTFS_SIZE = "8192"
+
In your cover letter you stated that the minimal image is currently 9.9M, which means we *need* the overhead to contain the entire image contents. Correct? That seems a little unwise.

Right, that's currently, we want to see the image get smaller even, so the 8M is an appropriate setting. Currently the actual ext3 image is 13M with 10M of contents.

So are you suggested that the 8M size is unwise or the overhead, not sure I am catching your meaning here.

Quoted 3 lines

Quoted 2 lines
meta/recipes-core/images/core-image-minimal.bb +0 -0
  # remove not needed ipkg informations
  ROOTFS_POSTPROCESS_COMMAND += "remove_packaging_data_files ; "

Joshua Lock

2011-05-26 19:54 UTC #4 lore raw

Quoted 119 lines · Saul Wold

Quoted 89 lines · Joshua Lock
Quoted 4 lines · Saul Wold
This basic cleanup removes the _ext2/3 overrides from places they no longer belong since they did not allow further overrides. In doing this the core-image-minimal* recipes can now set a reasonably small rootfs so that it's a realistic size for minimal.
Awesome. Thanks for tackling this!
Few questions below.
Quoted 79 lines
The new default for minimal is 8M and will be adujsted upward by the IMAGE_OVERHEAD_FACTOR (of 1.2).
This fixes the ROOTFS_SIZE usage in the IMAGE_CMD_<fstype> code
Signed-off-by Saul Wold<[email protected]>
---
  meta/classes/image_types.bbclass                   |    7 +++++--
  meta/conf/distro/include/default-distrovars.inc    |    2 --
  meta/conf/machine/include/qemu.inc                 |    2 --
  .../images/core-image-minimal-initramfs.bb         |    2 ++
  meta/recipes-core/images/core-image-minimal.bb     |    2 ++
  5 files changed, 9 insertions(+), 6 deletions(-)
meta/classes/image_types.bbclass +5 -2
index ec0cafd..69f859e 100644
@@ -21,22 +21,25 @@ runimagecmd () {
  }
 
  IMAGE_CMD_jffs2 = "mkfs.jffs2 --root=${IMAGE_ROOTFS} --faketime --output=${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.jffs2 ${EXTRA_IMAGECMD}"
+
  IMAGE_CMD_cramfs = "mkcramfs ${IMAGE_ROOTFS} ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.cramfs ${EXTRA_IMAGECMD}"
+
  IMAGE_CMD_ext2 = "genext2fs -b $ROOTFS_SIZE -d ${IMAGE_ROOTFS} ${EXTRA_IMAGECMD} ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.ext2"
  IMAGE_CMD_ext2.gz () {
  	rm -rf ${DEPLOY_DIR_IMAGE}/tmp.gz&&  mkdir ${DEPLOY_DIR_IMAGE}/tmp.gz
-	genext2fs -b ${IMAGE_ROOTFS_SIZE} -d ${IMAGE_ROOTFS} ${EXTRA_IMAGECMD} ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext2
+	genext2fs -b $ROOTFS_SIZE -d ${IMAGE_ROOTFS} ${EXTRA_IMAGECMD} ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext2
  	gzip -f -9 ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext2
  	mv ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext2.gz ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.ext2.gz
  	rmdir ${DEPLOY_DIR_IMAGE}/tmp.gz
  }
+
  IMAGE_CMD_ext3 () {
  	genext2fs -b $ROOTFS_SIZE -d ${IMAGE_ROOTFS} ${EXTRA_IMAGECMD} ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.ext3
  	tune2fs -j ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.ext3
  }
  IMAGE_CMD_ext3.gz () {
  	rm -rf ${DEPLOY_DIR_IMAGE}/tmp.gz&&  mkdir ${DEPLOY_DIR_IMAGE}/tmp.gz
-	genext2fs -b ${IMAGE_ROOTFS_SIZE} -d ${IMAGE_ROOTFS} ${EXTRA_IMAGECMD} ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext3
+	genext2fs -b $ROOTFS_SIZE -d ${IMAGE_ROOTFS} ${EXTRA_IMAGECMD} ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext3
  	tune2fs -j ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext3
  	gzip -f -9 ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext3
  	mv ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext3.gz ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.ext3.gz
meta/conf/distro/include/default-distrovars.inc +0 -2
index 1aa45c8..4b68a0a 100644
@@ -1,7 +1,5 @@
  QA_LOGFILE = "${TMPDIR}/qa.log"
 
-IMAGE_ROOTFS_SIZE_ext2 ?= "131072"
-
  OEINCLUDELOGS ?= "yes"
  KERNEL_CONSOLE ?= "ttyS0"
 
meta/conf/machine/include/qemu.inc +0 -2
index 4122a88..9ef242f 100644
@@ -6,8 +6,6 @@ MACHINE_FEATURES = "kernel26 apm alsa pcmcia bluetooth irda usbgadget screen"
  IMAGE_FSTYPES ?= "tar.bz2 ext3"
 
  ROOT_FLASH_SIZE = "280"
-IMAGE_ROOTFS_SIZE_ext2 ?= "280000"
-IMAGE_ROOTFS_SIZE_ext3 ?= "280000"
 
  # Don't include kernels in standard images
  RDEPENDS_kernel-base = ""
meta/recipes-core/images/core-image-minimal-initramfs.bb +0 -0
index 21aaa6c..3246d5c 100644
@@ -8,3 +8,5 @@ IMAGE_LINGUAS = ""
  LICENSE = "MIT"
 
Quoted 7 lines
meta/recipes-core/images/core-image-minimal-initramfs.bb IMAGE_LINGUAS = "" · line 12
Quoted 3 lines
meta/recipes-core/images/core-image-minimal-initramfs.bb +2 -0
  inherit core-image
+
+IMAGE_ROOTFS_SIZE = "8192"
I'm not really sure about this, an initramfs that's the same size as a more fully featured rootfs?
That may be, then we need to increase the size slightly, but this will trigger the correct behavior of actual size * IMAGE_OVERHEAD_FACTOR, rather than the 64M which would be the default, this was to ensure it could get smaller. I don't have a current initramfs size, I will build and verify.
Quoted 6 lines
Quoted 6 lines
meta/recipes-core/images/core-image-minimal.bb +0 -0
index aa00e67..743e121 100644
@@ -9,5 +9,7 @@ LICENSE = "MIT"
 
Quoted 9 lines
meta/recipes-core/images/core-image-minimal.bb LICENSE = "MIT" · line 13
Quoted 4 lines
meta/recipes-core/images/core-image-minimal.bb +2 -0
  inherit core-image
 
+IMAGE_ROOTFS_SIZE = "8192"
+
In your cover letter you stated that the minimal image is currently 9.9M, which means we *need* the overhead to contain the entire image contents. Correct? That seems a little unwise.

Quoted 6 lines

Right, that's currently, we want to see the image get smaller even, so the 8M is an appropriate setting. Currently the actual ext3 image is 13M with 10M of contents.
So are you suggested that the 8M size is unwise or the overhead, not sure I am catching your meaning here.

I'm just nervous about relying on the overhead, what if someone sets it lower and then the image doesn't fit?

Having the goal of a smaller minimal image is good but in my opinion we should adjust the rootfs size when the image is smaller, not before.

Cheers,
Joshua

Signature · 3 lines

Joshua Lock
        Yocto Project Build Monkey
        Intel Open Source Technology Centre

Darren Hart

2011-05-26 20:58 UTC #5 lore raw

Quoted 124 lines · Joshua Lock

Quoted 123 lines · Saul Wold
Quoted 89 lines · Joshua Lock
Quoted 4 lines · Saul Wold
This basic cleanup removes the _ext2/3 overrides from places they no longer belong since they did not allow further overrides. In doing this the core-image-minimal* recipes can now set a reasonably small rootfs so that it's a realistic size for minimal.
Awesome. Thanks for tackling this!
Few questions below.
Quoted 79 lines
The new default for minimal is 8M and will be adujsted upward by the IMAGE_OVERHEAD_FACTOR (of 1.2).
This fixes the ROOTFS_SIZE usage in the IMAGE_CMD_<fstype> code
Signed-off-by Saul Wold<[email protected]>
---
  meta/classes/image_types.bbclass                   |    7 +++++--
  meta/conf/distro/include/default-distrovars.inc    |    2 --
  meta/conf/machine/include/qemu.inc                 |    2 --
  .../images/core-image-minimal-initramfs.bb         |    2 ++
  meta/recipes-core/images/core-image-minimal.bb     |    2 ++
  5 files changed, 9 insertions(+), 6 deletions(-)
meta/classes/image_types.bbclass +5 -2
index ec0cafd..69f859e 100644
@@ -21,22 +21,25 @@ runimagecmd () {
  }
 
  IMAGE_CMD_jffs2 = "mkfs.jffs2 --root=${IMAGE_ROOTFS} --faketime --output=${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.jffs2 ${EXTRA_IMAGECMD}"
+
  IMAGE_CMD_cramfs = "mkcramfs ${IMAGE_ROOTFS} ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.cramfs ${EXTRA_IMAGECMD}"
+
  IMAGE_CMD_ext2 = "genext2fs -b $ROOTFS_SIZE -d ${IMAGE_ROOTFS} ${EXTRA_IMAGECMD} ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.ext2"
  IMAGE_CMD_ext2.gz () {
  	rm -rf ${DEPLOY_DIR_IMAGE}/tmp.gz&&  mkdir ${DEPLOY_DIR_IMAGE}/tmp.gz
-	genext2fs -b ${IMAGE_ROOTFS_SIZE} -d ${IMAGE_ROOTFS} ${EXTRA_IMAGECMD} ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext2
+	genext2fs -b $ROOTFS_SIZE -d ${IMAGE_ROOTFS} ${EXTRA_IMAGECMD} ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext2
  	gzip -f -9 ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext2
  	mv ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext2.gz ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.ext2.gz
  	rmdir ${DEPLOY_DIR_IMAGE}/tmp.gz
  }
+
  IMAGE_CMD_ext3 () {
  	genext2fs -b $ROOTFS_SIZE -d ${IMAGE_ROOTFS} ${EXTRA_IMAGECMD} ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.ext3
  	tune2fs -j ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.ext3
  }
  IMAGE_CMD_ext3.gz () {
  	rm -rf ${DEPLOY_DIR_IMAGE}/tmp.gz&&  mkdir ${DEPLOY_DIR_IMAGE}/tmp.gz
-	genext2fs -b ${IMAGE_ROOTFS_SIZE} -d ${IMAGE_ROOTFS} ${EXTRA_IMAGECMD} ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext3
+	genext2fs -b $ROOTFS_SIZE -d ${IMAGE_ROOTFS} ${EXTRA_IMAGECMD} ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext3
  	tune2fs -j ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext3
  	gzip -f -9 ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext3
  	mv ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext3.gz ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.ext3.gz
meta/conf/distro/include/default-distrovars.inc +0 -2
index 1aa45c8..4b68a0a 100644
@@ -1,7 +1,5 @@
  QA_LOGFILE = "${TMPDIR}/qa.log"
 
-IMAGE_ROOTFS_SIZE_ext2 ?= "131072"
-
  OEINCLUDELOGS ?= "yes"
  KERNEL_CONSOLE ?= "ttyS0"
 
meta/conf/machine/include/qemu.inc +0 -2
index 4122a88..9ef242f 100644
@@ -6,8 +6,6 @@ MACHINE_FEATURES = "kernel26 apm alsa pcmcia bluetooth irda usbgadget screen"
  IMAGE_FSTYPES ?= "tar.bz2 ext3"
 
  ROOT_FLASH_SIZE = "280"
-IMAGE_ROOTFS_SIZE_ext2 ?= "280000"
-IMAGE_ROOTFS_SIZE_ext3 ?= "280000"
 
  # Don't include kernels in standard images
  RDEPENDS_kernel-base = ""
meta/recipes-core/images/core-image-minimal-initramfs.bb +0 -0
index 21aaa6c..3246d5c 100644
@@ -8,3 +8,5 @@ IMAGE_LINGUAS = ""
  LICENSE = "MIT"
 
Quoted 7 lines
meta/recipes-core/images/core-image-minimal-initramfs.bb IMAGE_LINGUAS = "" · line 12
Quoted 3 lines
meta/recipes-core/images/core-image-minimal-initramfs.bb +2 -0
  inherit core-image
+
+IMAGE_ROOTFS_SIZE = "8192"
I'm not really sure about this, an initramfs that's the same size as a more fully featured rootfs?
That may be, then we need to increase the size slightly, but this will trigger the correct behavior of actual size * IMAGE_OVERHEAD_FACTOR, rather than the 64M which would be the default, this was to ensure it could get smaller. I don't have a current initramfs size, I will build and verify.
Quoted 6 lines
Quoted 6 lines
meta/recipes-core/images/core-image-minimal.bb +0 -0
index aa00e67..743e121 100644
@@ -9,5 +9,7 @@ LICENSE = "MIT"
 
Quoted 9 lines
meta/recipes-core/images/core-image-minimal.bb LICENSE = "MIT" · line 13
Quoted 4 lines
meta/recipes-core/images/core-image-minimal.bb +2 -0
  inherit core-image
 
+IMAGE_ROOTFS_SIZE = "8192"
+
In your cover letter you stated that the minimal image is currently 9.9M, which means we *need* the overhead to contain the entire image contents. Correct? That seems a little unwise.
Right, that's currently, we want to see the image get smaller even, so the 8M is an appropriate setting. Currently the actual ext3 image is 13M with 10M of contents.

Quoted 8 lines

Quoted 2 lines
So are you suggested that the 8M size is unwise or the overhead, not sure I am catching your meaning here.
I'm just nervous about relying on the overhead, what if someone sets it lower and then the image doesn't fit?
Having the goal of a smaller minimal image is good but in my opinion we should adjust the rootfs size when the image is smaller, not before.

Every argument I've heard in favor of the IMAGE_OVERHEAD was centered around ensuring there is enough space for the user to actually use the system after it was installed and booted. Using this single metric to try and both make up for oversized images and provide space for user data seems like to cause maintenance issues going forward.

While our goal should be to reduce the image size, the default image size in the recipes should default to the largest of the images built for the supported machines. The overhead should be used strictly for user data.

Signature · 3 lines

Darren Hart
Intel Open Source Technology Center
Yocto Project - Linux Kernel

Joshua Lock

2011-05-26 21:01 UTC #6 lore raw

Quoted 141 lines · Darren Hart

Quoted 124 lines · Joshua Lock
Quoted 123 lines · Saul Wold
Quoted 89 lines · Joshua Lock
Quoted 4 lines · Saul Wold
This basic cleanup removes the _ext2/3 overrides from places they no longer belong since they did not allow further overrides. In doing this the core-image-minimal* recipes can now set a reasonably small rootfs so that it's a realistic size for minimal.
Awesome. Thanks for tackling this!
Few questions below.
Quoted 79 lines
The new default for minimal is 8M and will be adujsted upward by the IMAGE_OVERHEAD_FACTOR (of 1.2).
This fixes the ROOTFS_SIZE usage in the IMAGE_CMD_<fstype> code
Signed-off-by Saul Wold<[email protected]>
---
  meta/classes/image_types.bbclass                   |    7 +++++--
  meta/conf/distro/include/default-distrovars.inc    |    2 --
  meta/conf/machine/include/qemu.inc                 |    2 --
  .../images/core-image-minimal-initramfs.bb         |    2 ++
  meta/recipes-core/images/core-image-minimal.bb     |    2 ++
  5 files changed, 9 insertions(+), 6 deletions(-)
meta/classes/image_types.bbclass +5 -2
index ec0cafd..69f859e 100644
@@ -21,22 +21,25 @@ runimagecmd () {
  }
 
  IMAGE_CMD_jffs2 = "mkfs.jffs2 --root=${IMAGE_ROOTFS} --faketime --output=${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.jffs2 ${EXTRA_IMAGECMD}"
+
  IMAGE_CMD_cramfs = "mkcramfs ${IMAGE_ROOTFS} ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.cramfs ${EXTRA_IMAGECMD}"
+
  IMAGE_CMD_ext2 = "genext2fs -b $ROOTFS_SIZE -d ${IMAGE_ROOTFS} ${EXTRA_IMAGECMD} ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.ext2"
  IMAGE_CMD_ext2.gz () {
  	rm -rf ${DEPLOY_DIR_IMAGE}/tmp.gz&&  mkdir ${DEPLOY_DIR_IMAGE}/tmp.gz
-	genext2fs -b ${IMAGE_ROOTFS_SIZE} -d ${IMAGE_ROOTFS} ${EXTRA_IMAGECMD} ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext2
+	genext2fs -b $ROOTFS_SIZE -d ${IMAGE_ROOTFS} ${EXTRA_IMAGECMD} ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext2
  	gzip -f -9 ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext2
  	mv ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext2.gz ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.ext2.gz
  	rmdir ${DEPLOY_DIR_IMAGE}/tmp.gz
  }
+
  IMAGE_CMD_ext3 () {
  	genext2fs -b $ROOTFS_SIZE -d ${IMAGE_ROOTFS} ${EXTRA_IMAGECMD} ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.ext3
  	tune2fs -j ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.ext3
  }
  IMAGE_CMD_ext3.gz () {
  	rm -rf ${DEPLOY_DIR_IMAGE}/tmp.gz&&  mkdir ${DEPLOY_DIR_IMAGE}/tmp.gz
-	genext2fs -b ${IMAGE_ROOTFS_SIZE} -d ${IMAGE_ROOTFS} ${EXTRA_IMAGECMD} ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext3
+	genext2fs -b $ROOTFS_SIZE -d ${IMAGE_ROOTFS} ${EXTRA_IMAGECMD} ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext3
  	tune2fs -j ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext3
  	gzip -f -9 ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext3
  	mv ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext3.gz ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.ext3.gz
meta/conf/distro/include/default-distrovars.inc +0 -2
index 1aa45c8..4b68a0a 100644
@@ -1,7 +1,5 @@
  QA_LOGFILE = "${TMPDIR}/qa.log"
 
-IMAGE_ROOTFS_SIZE_ext2 ?= "131072"
-
  OEINCLUDELOGS ?= "yes"
  KERNEL_CONSOLE ?= "ttyS0"
 
meta/conf/machine/include/qemu.inc +0 -2
index 4122a88..9ef242f 100644
@@ -6,8 +6,6 @@ MACHINE_FEATURES = "kernel26 apm alsa pcmcia bluetooth irda usbgadget screen"
  IMAGE_FSTYPES ?= "tar.bz2 ext3"
 
  ROOT_FLASH_SIZE = "280"
-IMAGE_ROOTFS_SIZE_ext2 ?= "280000"
-IMAGE_ROOTFS_SIZE_ext3 ?= "280000"
 
  # Don't include kernels in standard images
  RDEPENDS_kernel-base = ""
meta/recipes-core/images/core-image-minimal-initramfs.bb +0 -0
index 21aaa6c..3246d5c 100644
@@ -8,3 +8,5 @@ IMAGE_LINGUAS = ""
  LICENSE = "MIT"
 
Quoted 7 lines
meta/recipes-core/images/core-image-minimal-initramfs.bb IMAGE_LINGUAS = "" · line 12
Quoted 3 lines
meta/recipes-core/images/core-image-minimal-initramfs.bb +2 -0
  inherit core-image
+
+IMAGE_ROOTFS_SIZE = "8192"
I'm not really sure about this, an initramfs that's the same size as a more fully featured rootfs?
That may be, then we need to increase the size slightly, but this will trigger the correct behavior of actual size * IMAGE_OVERHEAD_FACTOR, rather than the 64M which would be the default, this was to ensure it could get smaller. I don't have a current initramfs size, I will build and verify.
Quoted 6 lines
Quoted 6 lines
meta/recipes-core/images/core-image-minimal.bb +0 -0
index aa00e67..743e121 100644
@@ -9,5 +9,7 @@ LICENSE = "MIT"
 
Quoted 9 lines
meta/recipes-core/images/core-image-minimal.bb LICENSE = "MIT" · line 13
Quoted 4 lines
meta/recipes-core/images/core-image-minimal.bb +2 -0
  inherit core-image
 
+IMAGE_ROOTFS_SIZE = "8192"
+
In your cover letter you stated that the minimal image is currently 9.9M, which means we *need* the overhead to contain the entire image contents. Correct? That seems a little unwise.
Right, that's currently, we want to see the image get smaller even, so the 8M is an appropriate setting. Currently the actual ext3 image is 13M with 10M of contents.
Quoted 8 lines
Quoted 2 lines
So are you suggested that the 8M size is unwise or the overhead, not sure I am catching your meaning here.
I'm just nervous about relying on the overhead, what if someone sets it lower and then the image doesn't fit?
Having the goal of a smaller minimal image is good but in my opinion we should adjust the rootfs size when the image is smaller, not before.
Every argument I've heard in favor of the IMAGE_OVERHEAD was centered around ensuring there is enough space for the user to actually use the system after it was installed and booted. Using this single metric to try and both make up for oversized images and provide space for user data seems like to cause maintenance issues going forward.

Quoted 5 lines

While our goal should be to reduce the image size, the default image size in the recipes should default to the largest of the images built for the supported machines. The overhead should be used strictly for user data.

So it turns out that if IMAGE_ROOTFS is smaller than the contents the size of the contents is used. Once this if test has been performed the overhead factor is applied.

I found this to be a tad confusing (hence all the back and forth here) so have filed a documentation bug about it:
http://bugzilla.pokylinux.org/show_bug.cgi?id=1110

Maybe we can have some comments in the code near where those variables are set too?

Signature · 3 lines

Joshua Lock
        Yocto Project Build Monkey
        Intel Open Source Technology Centre

Darren Hart

2011-05-26 21:01 UTC #19 lore raw

Re: [RFC 2/2] image_types: add IMAGE_ROOTFS_EXTRA_SPACE

In reply to Saul Wold

Quoted 23 lines · Saul Wold

Quoted 22 lines · Joshua Lock
Quoted 21 lines · Saul Wold
[YOCTO #886]
This address 2 needs after the IMAGE_ROOTFS_SIZE is cleaned up, by removing some of the _ext2/3 overrides it allows for a cleaner override using IMAGE_ROOTFS_SIZE to create a large rootfs, or by setting the IMAGE_ROOTFS_EXTRA_SPACE, will allow for extra space allocated in Kilobytes above the base size (determined by du of the rootfs * IMAGE_OVERHEAD_FACTOR, default of 1.3).
Signed-off-by Saul Wold<[email protected]>
---
  meta/classes/image_types.bbclass |    2 +-
  meta/conf/bitbake.conf           |    4 ++++
  2 files changed, 5 insertions(+), 1 deletions(-)
meta/classes/image_types.bbclass +0 -0
index 69f859e..601a5fb 100644
@@ -13,7 +13,7 @@ def get_imagecmds(d):
 

Quoted 13 lines

Quoted 5 lines

meta/classes/image_types.bbclass +1 -1

  runimagecmd () {
  	# Image generation code for image type ${type}
-	ROOTFS_SIZE=`du -ks ${IMAGE_ROOTFS}|awk '{size = $1 * ${IMAGE_OVERHEAD_FACTOR}; print (size>  ${IMAGE_ROOTFS_SIZE} ? size : ${IMAGE_ROOTFS_SIZE}) }'`
+	ROOTFS_SIZE=`du -ks ${IMAGE_ROOTFS}|awk '{size = $1 * ${IMAGE_OVERHEAD_FACTOR} + ${IMAGE_ROOTFS_EXTRA_SPACE}; print (size>  ${IMAGE_ROOTFS_SIZE} ? size : ${IMAGE_ROOTFS_SIZE}) }'`
  	${cmd}

Quoted 8 lines

meta/conf/bitbake.conf IMAGE_LINK_NAME = "${IMAGE_BASENAME}-${MACHINE}" · line 350
Quoted 8 lines
meta/classes/image_types.bbclass +0 -0
  	cd ${DEPLOY_DIR_IMAGE}/
  	rm -f ${DEPLOY_DIR_IMAGE}/${IMAGE_LINK_NAME}.${type}
meta/conf/bitbake.conf +0 -0
index a0af672..fa3618c 100644
@@ -350,6 +350,10 @@ IMAGE_LINK_NAME = "${IMAGE_BASENAME}-${MACHINE}"
  # This option allows for a precentage overage of the actaul image size rather than a

contextual fix while you're at it:
s/precentage/percentage/
s/actaul/actual/

Quoted 11 lines

Quoted 11 lines
meta/conf/bitbake.conf IMAGE_LINK_NAME = "${IMAGE_BASENAME}-${MACHINE}" · line 356
Quoted 6 lines
meta/conf/bitbake.conf +4 -0
  # fixed extra space
  IMAGE_OVERHEAD_FACTOR ?= 1.3
+# This option allow for adding addition space in K above and beyond what the
+# IMAGE_OVERHEAD_FACTOR might add. To set a fixed size then overrideing IMAGE_ROOTFS_SIZE
+# with the max size one wants should do the trick
+IMAGE_OVERHEAD_EXTRA_SPACE = "0"
This appears to be a different variable to the IMAGE_ROOTFS_EXTRA_SPACE used in the command above? Also, you've done a hard assignment here. Shouldn't it be a soft (?=) assignment?

Quoted 4 lines

Yes, I fixed the OVERHEAD vs ROOTFS already, noticed that during post RFC testing. I did not see the soft (?=) problem, but agree it needs to be fixed.

Hrm, I didn't realize there were two variables. If IMAGE_ROOTFS_EXTRA_SPACE is used to ensure the user has some space for their data, what then is the idea driving IMAGE_OVERHEAD_FACTOR?

Signature · 3 lines

Darren Hart
Intel Open Source Technology Center
Yocto Project - Linux Kernel

Darren Hart

2011-05-26 21:05 UTC #7 lore raw

In reply to Joshua Lock

Quoted 148 lines · Joshua Lock

Quoted 141 lines · Darren Hart
Quoted 124 lines · Joshua Lock
Quoted 123 lines · Saul Wold
Quoted 89 lines · Joshua Lock
Quoted 4 lines · Saul Wold
This basic cleanup removes the _ext2/3 overrides from places they no longer belong since they did not allow further overrides. In doing this the core-image-minimal* recipes can now set a reasonably small rootfs so that it's a realistic size for minimal.
Awesome. Thanks for tackling this!
Few questions below.
Quoted 79 lines
The new default for minimal is 8M and will be adujsted upward by the IMAGE_OVERHEAD_FACTOR (of 1.2).
This fixes the ROOTFS_SIZE usage in the IMAGE_CMD_<fstype> code
Signed-off-by Saul Wold<[email protected]>
---
  meta/classes/image_types.bbclass                   |    7 +++++--
  meta/conf/distro/include/default-distrovars.inc    |    2 --
  meta/conf/machine/include/qemu.inc                 |    2 --
  .../images/core-image-minimal-initramfs.bb         |    2 ++
  meta/recipes-core/images/core-image-minimal.bb     |    2 ++
  5 files changed, 9 insertions(+), 6 deletions(-)
meta/classes/image_types.bbclass +5 -2
index ec0cafd..69f859e 100644
@@ -21,22 +21,25 @@ runimagecmd () {
  }
 
  IMAGE_CMD_jffs2 = "mkfs.jffs2 --root=${IMAGE_ROOTFS} --faketime --output=${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.jffs2 ${EXTRA_IMAGECMD}"
+
  IMAGE_CMD_cramfs = "mkcramfs ${IMAGE_ROOTFS} ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.cramfs ${EXTRA_IMAGECMD}"
+
  IMAGE_CMD_ext2 = "genext2fs -b $ROOTFS_SIZE -d ${IMAGE_ROOTFS} ${EXTRA_IMAGECMD} ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.ext2"
  IMAGE_CMD_ext2.gz () {
  	rm -rf ${DEPLOY_DIR_IMAGE}/tmp.gz&&  mkdir ${DEPLOY_DIR_IMAGE}/tmp.gz
-	genext2fs -b ${IMAGE_ROOTFS_SIZE} -d ${IMAGE_ROOTFS} ${EXTRA_IMAGECMD} ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext2
+	genext2fs -b $ROOTFS_SIZE -d ${IMAGE_ROOTFS} ${EXTRA_IMAGECMD} ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext2
  	gzip -f -9 ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext2
  	mv ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext2.gz ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.ext2.gz
  	rmdir ${DEPLOY_DIR_IMAGE}/tmp.gz
  }
+
  IMAGE_CMD_ext3 () {
  	genext2fs -b $ROOTFS_SIZE -d ${IMAGE_ROOTFS} ${EXTRA_IMAGECMD} ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.ext3
  	tune2fs -j ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.ext3
  }
  IMAGE_CMD_ext3.gz () {
  	rm -rf ${DEPLOY_DIR_IMAGE}/tmp.gz&&  mkdir ${DEPLOY_DIR_IMAGE}/tmp.gz
-	genext2fs -b ${IMAGE_ROOTFS_SIZE} -d ${IMAGE_ROOTFS} ${EXTRA_IMAGECMD} ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext3
+	genext2fs -b $ROOTFS_SIZE -d ${IMAGE_ROOTFS} ${EXTRA_IMAGECMD} ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext3
  	tune2fs -j ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext3
  	gzip -f -9 ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext3
  	mv ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext3.gz ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.ext3.gz
meta/conf/distro/include/default-distrovars.inc +0 -2
index 1aa45c8..4b68a0a 100644
@@ -1,7 +1,5 @@
  QA_LOGFILE = "${TMPDIR}/qa.log"
 
-IMAGE_ROOTFS_SIZE_ext2 ?= "131072"
-
  OEINCLUDELOGS ?= "yes"
  KERNEL_CONSOLE ?= "ttyS0"
 
meta/conf/machine/include/qemu.inc +0 -2
index 4122a88..9ef242f 100644
@@ -6,8 +6,6 @@ MACHINE_FEATURES = "kernel26 apm alsa pcmcia bluetooth irda usbgadget screen"
  IMAGE_FSTYPES ?= "tar.bz2 ext3"
 
  ROOT_FLASH_SIZE = "280"
-IMAGE_ROOTFS_SIZE_ext2 ?= "280000"
-IMAGE_ROOTFS_SIZE_ext3 ?= "280000"
 
  # Don't include kernels in standard images
  RDEPENDS_kernel-base = ""
meta/recipes-core/images/core-image-minimal-initramfs.bb +0 -0
index 21aaa6c..3246d5c 100644
@@ -8,3 +8,5 @@ IMAGE_LINGUAS = ""
  LICENSE = "MIT"
 
Quoted 7 lines
meta/recipes-core/images/core-image-minimal-initramfs.bb IMAGE_LINGUAS = "" · line 12
Quoted 3 lines
meta/recipes-core/images/core-image-minimal-initramfs.bb +2 -0
  inherit core-image
+
+IMAGE_ROOTFS_SIZE = "8192"
I'm not really sure about this, an initramfs that's the same size as a more fully featured rootfs?
That may be, then we need to increase the size slightly, but this will trigger the correct behavior of actual size * IMAGE_OVERHEAD_FACTOR, rather than the 64M which would be the default, this was to ensure it could get smaller. I don't have a current initramfs size, I will build and verify.
Quoted 6 lines
Quoted 6 lines
meta/recipes-core/images/core-image-minimal.bb +0 -0
index aa00e67..743e121 100644
@@ -9,5 +9,7 @@ LICENSE = "MIT"
 
Quoted 9 lines
meta/recipes-core/images/core-image-minimal.bb LICENSE = "MIT" · line 13
Quoted 4 lines
meta/recipes-core/images/core-image-minimal.bb +2 -0
  inherit core-image
 
+IMAGE_ROOTFS_SIZE = "8192"
+
In your cover letter you stated that the minimal image is currently 9.9M, which means we *need* the overhead to contain the entire image contents. Correct? That seems a little unwise.
Right, that's currently, we want to see the image get smaller even, so the 8M is an appropriate setting. Currently the actual ext3 image is 13M with 10M of contents.
Quoted 8 lines
Quoted 2 lines
So are you suggested that the 8M size is unwise or the overhead, not sure I am catching your meaning here.
I'm just nervous about relying on the overhead, what if someone sets it lower and then the image doesn't fit?
Having the goal of a smaller minimal image is good but in my opinion we should adjust the rootfs size when the image is smaller, not before.
Every argument I've heard in favor of the IMAGE_OVERHEAD was centered around ensuring there is enough space for the user to actually use the system after it was installed and booted. Using this single metric to try and both make up for oversized images and provide space for user data seems like to cause maintenance issues going forward.
Quoted 5 lines
While our goal should be to reduce the image size, the default image size in the recipes should default to the largest of the images built for the supported machines. The overhead should be used strictly for user data.

Quoted 3 lines

So it turns out that if IMAGE_ROOTFS is smaller than the contents the size of the contents is used. Once this if test has been performed the overhead factor is applied.

Ah Good!

Quoted 8 lines

I found this to be a tad confusing (hence all the back and forth here) so have filed a documentation bug about it:
http://bugzilla.pokylinux.org/show_bug.cgi?id=1110
Maybe we can have some comments in the code near where those variables are set too?

Yeah, this wasn't clear to me either. And my question in 2/2 still stands - what is the goal of the overhead factor?

Signature · 3 lines

Darren Hart
Intel Open Source Technology Center
Yocto Project - Linux Kernel

Joshua Lock

2011-05-26 21:13 UTC #8 lore raw

Quoted 161 lines · Darren Hart

Quoted 148 lines · Joshua Lock
Quoted 141 lines · Darren Hart
Quoted 124 lines · Joshua Lock
Quoted 123 lines · Saul Wold
Quoted 89 lines · Joshua Lock
Quoted 4 lines · Saul Wold
This basic cleanup removes the _ext2/3 overrides from places they no longer belong since they did not allow further overrides. In doing this the core-image-minimal* recipes can now set a reasonably small rootfs so that it's a realistic size for minimal.
Awesome. Thanks for tackling this!
Few questions below.
Quoted 79 lines
The new default for minimal is 8M and will be adujsted upward by the IMAGE_OVERHEAD_FACTOR (of 1.2).
This fixes the ROOTFS_SIZE usage in the IMAGE_CMD_<fstype> code
Signed-off-by Saul Wold<[email protected]>
---
  meta/classes/image_types.bbclass                   |    7 +++++--
  meta/conf/distro/include/default-distrovars.inc    |    2 --
  meta/conf/machine/include/qemu.inc                 |    2 --
  .../images/core-image-minimal-initramfs.bb         |    2 ++
  meta/recipes-core/images/core-image-minimal.bb     |    2 ++
  5 files changed, 9 insertions(+), 6 deletions(-)
meta/classes/image_types.bbclass +5 -2
index ec0cafd..69f859e 100644
@@ -21,22 +21,25 @@ runimagecmd () {
  }
 
  IMAGE_CMD_jffs2 = "mkfs.jffs2 --root=${IMAGE_ROOTFS} --faketime --output=${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.jffs2 ${EXTRA_IMAGECMD}"
+
  IMAGE_CMD_cramfs = "mkcramfs ${IMAGE_ROOTFS} ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.cramfs ${EXTRA_IMAGECMD}"
+
  IMAGE_CMD_ext2 = "genext2fs -b $ROOTFS_SIZE -d ${IMAGE_ROOTFS} ${EXTRA_IMAGECMD} ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.ext2"
  IMAGE_CMD_ext2.gz () {
  	rm -rf ${DEPLOY_DIR_IMAGE}/tmp.gz&&  mkdir ${DEPLOY_DIR_IMAGE}/tmp.gz
-	genext2fs -b ${IMAGE_ROOTFS_SIZE} -d ${IMAGE_ROOTFS} ${EXTRA_IMAGECMD} ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext2
+	genext2fs -b $ROOTFS_SIZE -d ${IMAGE_ROOTFS} ${EXTRA_IMAGECMD} ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext2
  	gzip -f -9 ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext2
  	mv ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext2.gz ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.ext2.gz
  	rmdir ${DEPLOY_DIR_IMAGE}/tmp.gz
  }
+
  IMAGE_CMD_ext3 () {
  	genext2fs -b $ROOTFS_SIZE -d ${IMAGE_ROOTFS} ${EXTRA_IMAGECMD} ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.ext3
  	tune2fs -j ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.ext3
  }
  IMAGE_CMD_ext3.gz () {
  	rm -rf ${DEPLOY_DIR_IMAGE}/tmp.gz&&  mkdir ${DEPLOY_DIR_IMAGE}/tmp.gz
-	genext2fs -b ${IMAGE_ROOTFS_SIZE} -d ${IMAGE_ROOTFS} ${EXTRA_IMAGECMD} ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext3
+	genext2fs -b $ROOTFS_SIZE -d ${IMAGE_ROOTFS} ${EXTRA_IMAGECMD} ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext3
  	tune2fs -j ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext3
  	gzip -f -9 ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext3
  	mv ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext3.gz ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.ext3.gz
meta/conf/distro/include/default-distrovars.inc +0 -2
index 1aa45c8..4b68a0a 100644
@@ -1,7 +1,5 @@
  QA_LOGFILE = "${TMPDIR}/qa.log"
 
-IMAGE_ROOTFS_SIZE_ext2 ?= "131072"
-
  OEINCLUDELOGS ?= "yes"
  KERNEL_CONSOLE ?= "ttyS0"
 
meta/conf/machine/include/qemu.inc +0 -2
index 4122a88..9ef242f 100644
@@ -6,8 +6,6 @@ MACHINE_FEATURES = "kernel26 apm alsa pcmcia bluetooth irda usbgadget screen"
  IMAGE_FSTYPES ?= "tar.bz2 ext3"
 
  ROOT_FLASH_SIZE = "280"
-IMAGE_ROOTFS_SIZE_ext2 ?= "280000"
-IMAGE_ROOTFS_SIZE_ext3 ?= "280000"
 
  # Don't include kernels in standard images
  RDEPENDS_kernel-base = ""
meta/recipes-core/images/core-image-minimal-initramfs.bb +0 -0
index 21aaa6c..3246d5c 100644
@@ -8,3 +8,5 @@ IMAGE_LINGUAS = ""
  LICENSE = "MIT"
 
Quoted 7 lines
meta/recipes-core/images/core-image-minimal-initramfs.bb IMAGE_LINGUAS = "" · line 12
Quoted 3 lines
meta/recipes-core/images/core-image-minimal-initramfs.bb +2 -0
  inherit core-image
+
+IMAGE_ROOTFS_SIZE = "8192"
I'm not really sure about this, an initramfs that's the same size as a more fully featured rootfs?
That may be, then we need to increase the size slightly, but this will trigger the correct behavior of actual size * IMAGE_OVERHEAD_FACTOR, rather than the 64M which would be the default, this was to ensure it could get smaller. I don't have a current initramfs size, I will build and verify.
Quoted 6 lines
Quoted 6 lines
meta/recipes-core/images/core-image-minimal.bb +0 -0
index aa00e67..743e121 100644
@@ -9,5 +9,7 @@ LICENSE = "MIT"
 
Quoted 9 lines
meta/recipes-core/images/core-image-minimal.bb LICENSE = "MIT" · line 13
Quoted 4 lines
meta/recipes-core/images/core-image-minimal.bb +2 -0
  inherit core-image
 
+IMAGE_ROOTFS_SIZE = "8192"
+
In your cover letter you stated that the minimal image is currently 9.9M, which means we *need* the overhead to contain the entire image contents. Correct? That seems a little unwise.
Right, that's currently, we want to see the image get smaller even, so the 8M is an appropriate setting. Currently the actual ext3 image is 13M with 10M of contents.
Quoted 8 lines
Quoted 2 lines
So are you suggested that the 8M size is unwise or the overhead, not sure I am catching your meaning here.
I'm just nervous about relying on the overhead, what if someone sets it lower and then the image doesn't fit?
Having the goal of a smaller minimal image is good but in my opinion we should adjust the rootfs size when the image is smaller, not before.
Every argument I've heard in favor of the IMAGE_OVERHEAD was centered around ensuring there is enough space for the user to actually use the system after it was installed and booted. Using this single metric to try and both make up for oversized images and provide space for user data seems like to cause maintenance issues going forward.
Quoted 5 lines
While our goal should be to reduce the image size, the default image size in the recipes should default to the largest of the images built for the supported machines. The overhead should be used strictly for user data.
Quoted 3 lines
So it turns out that if IMAGE_ROOTFS is smaller than the contents the size of the contents is used. Once this if test has been performed the overhead factor is applied.
Ah Good!
Quoted 5 lines
I found this to be a tad confusing (hence all the back and forth here) so have filed a documentation bug about it:
http://bugzilla.pokylinux.org/show_bug.cgi?id=1110

Quoted 7 lines

Quoted 3 lines
Maybe we can have some comments in the code near where those variables are set too?
Yeah, this wasn't clear to me either. And my question in 2/2 still stands - what is the goal of the overhead factor?

My understanding is that it's for system overhead such as postinstalls whereas ROOTFS_IMAGE_EXTRA_SPACE is for user and data space that you want above the standard image size.

So if you want to ensure all of your images have 2GB space for data you'd set the ROOTFS_IMAGE_EXTRA_SPACE to 2GB.

Cheers,
Joshua

Signature · 3 lines

Joshua Lock
        Yocto Project Build Monkey
        Intel Open Source Technology Centre

Paul Eggleton

2011-05-26 21:17 UTC #11 lore raw

In reply to Darren Hart

Quoted 2 lines · On Thursday 26 May 2011 22:03:02 Darren Hart wrote:

Yeah, this wasn't clear to me either. And my question in 2/2 still stands - what is the goal of the overhead factor?

I think the thinking was that the more software you have installed to begin with the more user data you're likely to need to store. Personally I think it would be simpler if we just set some additional overhead as an absolute value rather than a factor - if it needs to be different per image type then it easily can be.

Cheers,
Paul

Paul Eggleton
Intel Open Source Technology Centre

Saul Wold

author

2011-05-26 21:18 UTC #9 lore raw

In reply to Joshua Lock

Quoted 166 lines · Joshua Lock

Quoted 165 lines · Darren Hart
Quoted 148 lines · Joshua Lock
Quoted 141 lines · Darren Hart
Quoted 124 lines · Joshua Lock
Quoted 123 lines · Saul Wold
Quoted 89 lines · Joshua Lock
Quoted 4 lines · Saul Wold
This basic cleanup removes the _ext2/3 overrides from places they no longer belong since they did not allow further overrides. In doing this the core-image-minimal* recipes can now set a reasonably small rootfs so that it's a realistic size for minimal.
Awesome. Thanks for tackling this!
Few questions below.
Quoted 79 lines
The new default for minimal is 8M and will be adujsted upward by the IMAGE_OVERHEAD_FACTOR (of 1.2).
This fixes the ROOTFS_SIZE usage in the IMAGE_CMD_<fstype> code
Signed-off-by Saul Wold<[email protected]>
---
   meta/classes/image_types.bbclass                   |    7 +++++--
   meta/conf/distro/include/default-distrovars.inc    |    2 --
   meta/conf/machine/include/qemu.inc                 |    2 --
   .../images/core-image-minimal-initramfs.bb         |    2 ++
   meta/recipes-core/images/core-image-minimal.bb     |    2 ++
   5 files changed, 9 insertions(+), 6 deletions(-)
meta/classes/image_types.bbclass +5 -2
index ec0cafd..69f859e 100644
@@ -21,22 +21,25 @@ runimagecmd () {
   }
 
   IMAGE_CMD_jffs2 = "mkfs.jffs2 --root=${IMAGE_ROOTFS} --faketime --output=${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.jffs2 ${EXTRA_IMAGECMD}"
+
   IMAGE_CMD_cramfs = "mkcramfs ${IMAGE_ROOTFS} ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.cramfs ${EXTRA_IMAGECMD}"
+
   IMAGE_CMD_ext2 = "genext2fs -b $ROOTFS_SIZE -d ${IMAGE_ROOTFS} ${EXTRA_IMAGECMD} ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.ext2"
   IMAGE_CMD_ext2.gz () {
   	rm -rf ${DEPLOY_DIR_IMAGE}/tmp.gz&&   mkdir ${DEPLOY_DIR_IMAGE}/tmp.gz
-	genext2fs -b ${IMAGE_ROOTFS_SIZE} -d ${IMAGE_ROOTFS} ${EXTRA_IMAGECMD} ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext2
+	genext2fs -b $ROOTFS_SIZE -d ${IMAGE_ROOTFS} ${EXTRA_IMAGECMD} ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext2
   	gzip -f -9 ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext2
   	mv ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext2.gz ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.ext2.gz
   	rmdir ${DEPLOY_DIR_IMAGE}/tmp.gz
   }
+
   IMAGE_CMD_ext3 () {
   	genext2fs -b $ROOTFS_SIZE -d ${IMAGE_ROOTFS} ${EXTRA_IMAGECMD} ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.ext3
   	tune2fs -j ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.ext3
   }
   IMAGE_CMD_ext3.gz () {
   	rm -rf ${DEPLOY_DIR_IMAGE}/tmp.gz&&   mkdir ${DEPLOY_DIR_IMAGE}/tmp.gz
-	genext2fs -b ${IMAGE_ROOTFS_SIZE} -d ${IMAGE_ROOTFS} ${EXTRA_IMAGECMD} ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext3
+	genext2fs -b $ROOTFS_SIZE -d ${IMAGE_ROOTFS} ${EXTRA_IMAGECMD} ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext3
   	tune2fs -j ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext3
   	gzip -f -9 ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext3
   	mv ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext3.gz ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.ext3.gz
meta/conf/distro/include/default-distrovars.inc +0 -2
index 1aa45c8..4b68a0a 100644
@@ -1,7 +1,5 @@
   QA_LOGFILE = "${TMPDIR}/qa.log"
 
-IMAGE_ROOTFS_SIZE_ext2 ?= "131072"
-
   OEINCLUDELOGS ?= "yes"
   KERNEL_CONSOLE ?= "ttyS0"
 
meta/conf/machine/include/qemu.inc +0 -2
index 4122a88..9ef242f 100644
@@ -6,8 +6,6 @@ MACHINE_FEATURES = "kernel26 apm alsa pcmcia bluetooth irda usbgadget screen"
   IMAGE_FSTYPES ?= "tar.bz2 ext3"
 
   ROOT_FLASH_SIZE = "280"
-IMAGE_ROOTFS_SIZE_ext2 ?= "280000"
-IMAGE_ROOTFS_SIZE_ext3 ?= "280000"
 
   # Don't include kernels in standard images
   RDEPENDS_kernel-base = ""
meta/recipes-core/images/core-image-minimal-initramfs.bb +0 -0
index 21aaa6c..3246d5c 100644
@@ -8,3 +8,5 @@ IMAGE_LINGUAS = ""
   LICENSE = "MIT"
 
Quoted 7 lines
meta/recipes-core/images/core-image-minimal-initramfs.bb IMAGE_LINGUAS = "" · line 12
Quoted 3 lines
meta/recipes-core/images/core-image-minimal-initramfs.bb +2 -0
   inherit core-image
+
+IMAGE_ROOTFS_SIZE = "8192"
I'm not really sure about this, an initramfs that's the same size as a more fully featured rootfs?
That may be, then we need to increase the size slightly, but this will trigger the correct behavior of actual size * IMAGE_OVERHEAD_FACTOR, rather than the 64M which would be the default, this was to ensure it could get smaller. I don't have a current initramfs size, I will build and verify.
Quoted 6 lines
Quoted 6 lines
meta/recipes-core/images/core-image-minimal.bb +0 -0
index aa00e67..743e121 100644
@@ -9,5 +9,7 @@ LICENSE = "MIT"
 
Quoted 9 lines
meta/recipes-core/images/core-image-minimal.bb LICENSE = "MIT" · line 13
Quoted 4 lines
meta/recipes-core/images/core-image-minimal.bb +2 -0
   inherit core-image
 
+IMAGE_ROOTFS_SIZE = "8192"
+
In your cover letter you stated that the minimal image is currently 9.9M, which means we *need* the overhead to contain the entire image contents. Correct? That seems a little unwise.
Right, that's currently, we want to see the image get smaller even, so the 8M is an appropriate setting. Currently the actual ext3 image is 13M with 10M of contents.
Quoted 8 lines
Quoted 2 lines
So are you suggested that the 8M size is unwise or the overhead, not sure I am catching your meaning here.
I'm just nervous about relying on the overhead, what if someone sets it lower and then the image doesn't fit?
Having the goal of a smaller minimal image is good but in my opinion we should adjust the rootfs size when the image is smaller, not before.
Every argument I've heard in favor of the IMAGE_OVERHEAD was centered around ensuring there is enough space for the user to actually use the system after it was installed and booted. Using this single metric to try and both make up for oversized images and provide space for user data seems like to cause maintenance issues going forward.
Quoted 5 lines
While our goal should be to reduce the image size, the default image size in the recipes should default to the largest of the images built for the supported machines. The overhead should be used strictly for user data.
Quoted 3 lines
So it turns out that if IMAGE_ROOTFS is smaller than the contents the size of the contents is used. Once this if test has been performed the overhead factor is applied.
Ah Good!
Quoted 8 lines
I found this to be a tad confusing (hence all the back and forth here) so have filed a documentation bug about it:
http://bugzilla.pokylinux.org/show_bug.cgi?id=1110
Maybe we can have some comments in the code near where those variables are set too?

Quoted 8 lines

Quoted 3 lines
Yeah, this wasn't clear to me either. And my question in 2/2 still stands - what is the goal of the overhead factor?
My understanding is that it's for system overhead such as postinstalls whereas ROOTFS_IMAGE_EXTRA_SPACE is for user and data space that you want above the standard image size.

Correct

Quoted 3 lines

So if you want to ensure all of your images have 2GB space for data you'd set the ROOTFS_IMAGE_EXTRA_SPACE to 2GB.

That's one way, another is if you want all your images to be 2G including the core contents set ROOTFS_IMAGE_SIZE to 2G and the images will all be 2G (remember to set in K, since we don't parse the M or G).

Sau!

Quoted 2 lines

Cheers,
Joshua

Joshua Lock

2011-05-26 21:31 UTC #10 lore raw

Quoted 175 lines · Saul Wold

Quoted 166 lines · Joshua Lock
Quoted 165 lines · Darren Hart
Quoted 148 lines · Joshua Lock
Quoted 141 lines · Darren Hart
Quoted 124 lines · Joshua Lock
Quoted 123 lines · Saul Wold
Quoted 89 lines · Joshua Lock
Quoted 4 lines · Saul Wold
This basic cleanup removes the _ext2/3 overrides from places they no longer belong since they did not allow further overrides. In doing this the core-image-minimal* recipes can now set a reasonably small rootfs so that it's a realistic size for minimal.
Awesome. Thanks for tackling this!
Few questions below.
Quoted 79 lines
The new default for minimal is 8M and will be adujsted upward by the IMAGE_OVERHEAD_FACTOR (of 1.2).
This fixes the ROOTFS_SIZE usage in the IMAGE_CMD_<fstype> code
Signed-off-by Saul Wold<[email protected]>
---
   meta/classes/image_types.bbclass                   |    7 +++++--
   meta/conf/distro/include/default-distrovars.inc    |    2 --
   meta/conf/machine/include/qemu.inc                 |    2 --
   .../images/core-image-minimal-initramfs.bb         |    2 ++
   meta/recipes-core/images/core-image-minimal.bb     |    2 ++
   5 files changed, 9 insertions(+), 6 deletions(-)
meta/classes/image_types.bbclass +5 -2
index ec0cafd..69f859e 100644
@@ -21,22 +21,25 @@ runimagecmd () {
   }
 
   IMAGE_CMD_jffs2 = "mkfs.jffs2 --root=${IMAGE_ROOTFS} --faketime --output=${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.jffs2 ${EXTRA_IMAGECMD}"
+
   IMAGE_CMD_cramfs = "mkcramfs ${IMAGE_ROOTFS} ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.cramfs ${EXTRA_IMAGECMD}"
+
   IMAGE_CMD_ext2 = "genext2fs -b $ROOTFS_SIZE -d ${IMAGE_ROOTFS} ${EXTRA_IMAGECMD} ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.ext2"
   IMAGE_CMD_ext2.gz () {
   	rm -rf ${DEPLOY_DIR_IMAGE}/tmp.gz&&   mkdir ${DEPLOY_DIR_IMAGE}/tmp.gz
-	genext2fs -b ${IMAGE_ROOTFS_SIZE} -d ${IMAGE_ROOTFS} ${EXTRA_IMAGECMD} ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext2
+	genext2fs -b $ROOTFS_SIZE -d ${IMAGE_ROOTFS} ${EXTRA_IMAGECMD} ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext2
   	gzip -f -9 ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext2
   	mv ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext2.gz ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.ext2.gz
   	rmdir ${DEPLOY_DIR_IMAGE}/tmp.gz
   }
+
   IMAGE_CMD_ext3 () {
   	genext2fs -b $ROOTFS_SIZE -d ${IMAGE_ROOTFS} ${EXTRA_IMAGECMD} ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.ext3
   	tune2fs -j ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.ext3
   }
   IMAGE_CMD_ext3.gz () {
   	rm -rf ${DEPLOY_DIR_IMAGE}/tmp.gz&&   mkdir ${DEPLOY_DIR_IMAGE}/tmp.gz
-	genext2fs -b ${IMAGE_ROOTFS_SIZE} -d ${IMAGE_ROOTFS} ${EXTRA_IMAGECMD} ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext3
+	genext2fs -b $ROOTFS_SIZE -d ${IMAGE_ROOTFS} ${EXTRA_IMAGECMD} ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext3
   	tune2fs -j ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext3
   	gzip -f -9 ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext3
   	mv ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext3.gz ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.ext3.gz
meta/conf/distro/include/default-distrovars.inc +0 -2
index 1aa45c8..4b68a0a 100644
@@ -1,7 +1,5 @@
   QA_LOGFILE = "${TMPDIR}/qa.log"
 
-IMAGE_ROOTFS_SIZE_ext2 ?= "131072"
-
   OEINCLUDELOGS ?= "yes"
   KERNEL_CONSOLE ?= "ttyS0"
 
meta/conf/machine/include/qemu.inc +0 -2
index 4122a88..9ef242f 100644
@@ -6,8 +6,6 @@ MACHINE_FEATURES = "kernel26 apm alsa pcmcia bluetooth irda usbgadget screen"
   IMAGE_FSTYPES ?= "tar.bz2 ext3"
 
   ROOT_FLASH_SIZE = "280"
-IMAGE_ROOTFS_SIZE_ext2 ?= "280000"
-IMAGE_ROOTFS_SIZE_ext3 ?= "280000"
 
   # Don't include kernels in standard images
   RDEPENDS_kernel-base = ""
meta/recipes-core/images/core-image-minimal-initramfs.bb +0 -0
index 21aaa6c..3246d5c 100644
@@ -8,3 +8,5 @@ IMAGE_LINGUAS = ""
   LICENSE = "MIT"
 
Quoted 7 lines
meta/recipes-core/images/core-image-minimal-initramfs.bb IMAGE_LINGUAS = "" · line 12
Quoted 3 lines
meta/recipes-core/images/core-image-minimal-initramfs.bb +2 -0
   inherit core-image
+
+IMAGE_ROOTFS_SIZE = "8192"
I'm not really sure about this, an initramfs that's the same size as a more fully featured rootfs?
That may be, then we need to increase the size slightly, but this will trigger the correct behavior of actual size * IMAGE_OVERHEAD_FACTOR, rather than the 64M which would be the default, this was to ensure it could get smaller. I don't have a current initramfs size, I will build and verify.
Quoted 6 lines
Quoted 6 lines
meta/recipes-core/images/core-image-minimal.bb +0 -0
index aa00e67..743e121 100644
@@ -9,5 +9,7 @@ LICENSE = "MIT"
 
Quoted 9 lines
meta/recipes-core/images/core-image-minimal.bb LICENSE = "MIT" · line 13
Quoted 4 lines
meta/recipes-core/images/core-image-minimal.bb +2 -0
   inherit core-image
 
+IMAGE_ROOTFS_SIZE = "8192"
+
In your cover letter you stated that the minimal image is currently 9.9M, which means we *need* the overhead to contain the entire image contents. Correct? That seems a little unwise.
Right, that's currently, we want to see the image get smaller even, so the 8M is an appropriate setting. Currently the actual ext3 image is 13M with 10M of contents.
Quoted 8 lines
Quoted 2 lines
So are you suggested that the 8M size is unwise or the overhead, not sure I am catching your meaning here.
I'm just nervous about relying on the overhead, what if someone sets it lower and then the image doesn't fit?
Having the goal of a smaller minimal image is good but in my opinion we should adjust the rootfs size when the image is smaller, not before.
Every argument I've heard in favor of the IMAGE_OVERHEAD was centered around ensuring there is enough space for the user to actually use the system after it was installed and booted. Using this single metric to try and both make up for oversized images and provide space for user data seems like to cause maintenance issues going forward.
Quoted 5 lines
While our goal should be to reduce the image size, the default image size in the recipes should default to the largest of the images built for the supported machines. The overhead should be used strictly for user data.
Quoted 3 lines
So it turns out that if IMAGE_ROOTFS is smaller than the contents the size of the contents is used. Once this if test has been performed the overhead factor is applied.
Ah Good!
Quoted 8 lines
I found this to be a tad confusing (hence all the back and forth here) so have filed a documentation bug about it:
http://bugzilla.pokylinux.org/show_bug.cgi?id=1110
Maybe we can have some comments in the code near where those variables are set too?
Quoted 8 lines
Quoted 3 lines
Yeah, this wasn't clear to me either. And my question in 2/2 still stands - what is the goal of the overhead factor?
My understanding is that it's for system overhead such as postinstalls whereas ROOTFS_IMAGE_EXTRA_SPACE is for user and data space that you want above the standard image size.

Quoted 8 lines

Correct
Quoted 3 lines
So if you want to ensure all of your images have 2GB space for data you'd set the ROOTFS_IMAGE_EXTRA_SPACE to 2GB.
That's one way, another is if you want all your images to be 2G including the core contents set ROOTFS_IMAGE_SIZE to 2G and the images will all be 2G (remember to set in K, since we don't parse the M or G).

We probably should though. Shall I file a feature request? :-)

Cheers,
Joshua

Signature · 3 lines

Joshua Lock
        Yocto Project Build Monkey
        Intel Open Source Technology Centre

Darren Hart

2011-05-27 04:10 UTC #12 lore raw

In reply to Paul Eggleton

Quoted 7 lines · Paul Eggleton

Quoted 2 lines · On Thursday 26 May 2011 22:03:02 Darren Hart wrote:
Yeah, this wasn't clear to me either. And my question in 2/2 still stands - what is the goal of the overhead factor?
I think the thinking was that the more software you have installed to begin with the more user data you're likely to need to store. Personally I think it would be simpler if we just set some additional overhead as an absolute value

I had the same thought, so "seconded".

Signature · 3 lines

Darren Hart
Intel Open Source Technology Center
Yocto Project - Linux Kernel

Saul Wold

author

2011-05-27 04:32 UTC #13 lore raw

Quoted 4 lines · Darren Hart

Quoted 1 line · Paul Eggleton
On Thursday 26 May 2011 22:03:02 Darren Hart wrote:

Quoted 9 lines

Quoted 6 lines
Quoted 2 lines
Yeah, this wasn't clear to me either. And my question in 2/2 still stands - what is the goal of the overhead factor?
I think the thinking was that the more software you have installed to begin with the more user data you're likely to need to store. Personally I think it would be simpler if we just set some additional overhead as an absolute value
I had the same thought, so "seconded".

It used to be that way and failed in unpredictable ways, what would work for one image would be way over kill for another or be too little, so the overhead factor works better.

If you want a known amount of space then you can use the IMAGE_ROOTFS_EXTRA_SPACE.

Sau!

Darren Hart

2011-05-27 05:25 UTC #14 lore raw

Quoted 12 lines · Saul Wold

Quoted 11 lines · Darren Hart
Quoted 7 lines · Paul Eggleton
Quoted 2 lines · On Thursday 26 May 2011 22:03:02 Darren Hart wrote:
Yeah, this wasn't clear to me either. And my question in 2/2 still stands - what is the goal of the overhead factor?
I think the thinking was that the more software you have installed to begin with the more user data you're likely to need to store. Personally I think it would be simpler if we just set some additional overhead as an absolute value

Quoted 6 lines

Quoted 2 lines
I had the same thought, so "seconded".
It used to be that way and failed in unpredictable ways, what would work for one image would be way over kill for another or be too little, so the overhead factor works better.

Seems to me this is just slightly more complicated, but every bit as likely to fail in unexpected ways in the future. Perhaps the simplest, most robust, and most explicit way to address this would be to set the IMAGE_ROOTFS_EXTRA_SPACE in each IMAGE file, just as we set the base size?

This still ensures you get at least IMAGE_ROOTFS_EXTRA_SPACE freespace since the base size will expand to be at least the size the image requires.

My final 0.02 USD on the matter.

Quoted 4 lines

If you want a known amount of space then you can use the IMAGE_ROOTFS_EXTRA_SPACE.
Sau!

Signature · 3 lines

Darren Hart
Intel Open Source Technology Center
Yocto Project - Linux Kernel

Tom Rini

2011-06-02 01:50 UTC #15 lore raw

In reply to Darren Hart

On 05/26/2011 02:03 PM, Darren Hart wrote:
[snip]

Quoted 2 lines

Yeah, this wasn't clear to me either. And my question in 2/2 still stands - what is the goal of the overhead factor?

I know I'm late to the thread, sorry, but the problem is that the 'du' estimation we use for the common case of "image is larger than our default" sucks pretty bad. Without some additional fudge factors (both the multiplier and additional room) most images won't actually end up fitting into an ext* image and genext2fs chokes. I know the rpmdb was mentioned before but really it's everything. Take any image, poke at what du says things take and what df shows it's taking and cry :(

Signature · 2 lines

Tom Rini
Mentor Graphics Corporation