Review Request for Build Appliance
cover Review Request for Build Appliance
Richard,
This is an updated set of changes for the Build Appliance to use directdisk. I am submitting this even as I am finding out that there is an issue with genext2fs building Large FS Size (5GB and greater).
The other option is to add a newer version of dosfstools, which is not fully vetted yet, but may allow us to use bootimg instead of driectdisk.
Please review and comment the builder, image-vmdk and boot-directdisk patches ready to go, the VMDK, update for directdisk is questionable until we can solve the large image issues.
Sau!
The following changes since commit 647595821870693186c9ecad1bdfa12fe4ea334d:
core-image-minimal-mtdutils.bb: Simple DESCRIPTION grammar fix. (2012-03-25 12:27:38 +0100)
are available in the git repository at:
git://git.openembedded.org/openembedded-core-contrib sgw/self
http://cgit.openembedded.org/cgit.cgi/openembedded-core-contrib/log/?h=sgw/oe-self
Dexuan Cui (1):
self-hosted-image: pre-populate the builder user with poky source
Saul Wold (4):
image-vmdk: Add symbolic link
boot-directdisk: Fix Block Calcuation
vmdk: Update for direstdisk
builder: Enable auto starting of Hob
meta/classes/boot-directdisk.bbclass | 19 ++++++--- meta/classes/image-vmdk.bbclass | 17 +++++++- meta/recipes-core/images/self-hosted-image.bb | 41 +++++++++++++++++++- meta/recipes-graphics/builder/builder_0.1.bb | 2 +- .../builder/files/builder_hob_start.sh | 10 +++- 5 files changed, 74 insertions(+), 15 deletions(-)
Signature · 1 line
1.7.7.6
RFC 1/5 self-hosted-image: pre-populate the builder user with poky source
From: Dexuan Cui <[email protected]>
This patch installs the poky source into the /home/builder/poky/ of the self-hosted-image.
This makes the user of self-hosted-image easier to start a build.
I think the recent poky master is stable enough, so I specify a commit number by SRCREV -- we may want to update this number before releasing 1.2.
This patch fixes [YOCTO #2065]
Added code for supporting target based pseudo
fixed indentation
--- meta/recipes-core/images/self-hosted-image.bb | 41 +++++++++++++++++++++++- 1 files changed, 39 insertions(+), 2 deletions(-)
meta/recipes-core/images/self-hosted-image.bb +39 -2
@@ -4,7 +4,7 @@ LICENSE = "MIT" LIC_FILES_CHKSUM = "file://${COREBASE}/LICENSE;md5=3f40d7994397109285ec7b81fdeb3b58 \ file://${COREBASE}/meta/COPYING.MIT;md5=3da9cfbcb788c80a0384361b4de20420" -PR = "r5" +PR = "r6" CORE_IMAGE_EXTRA_INSTALL = "\ task-self-hosted \
@@ -13,7 +13,10 @@ CORE_IMAGE_EXTRA_INSTALL = "\ IMAGE_FEATURES += "x11-mini package-management" # Ensure there's enough space to do a core-image-minimal build, with rm_work enabled -IMAGE_ROOTFS_EXTRA_SPACE = "2621440" +IMAGE_ROOTFS_EXTRA_SPACE = "1048576" +#IMAGE_ROOTFS_EXTRA_SPACE = "2621440" +#IMAGE_ROOTFS_EXTRA_SPACE = "20971520" +#IMAGE_ROOTFS_EXTRA_SPACE = "5242880" # Do a quiet boot with limited console messages APPEND += "quiet"
@@ -21,3 +24,37 @@ APPEND += "quiet" IMAGE_FSTYPES = "vmdk" inherit core-image + +SRCREV = "26a46938d3ea1821e7bec4fa6cc8379babad238b" +SRC_URI = "git://git.yoctoproject.org/poky;protocol=git" + +fakeroot do_populate_poky_src () { + # Because fetch2's git's unpack uses -s cloneflag, the unpacked git repo + # will become invalid in the target. + rm -rf ${WORKDIR}/git/.git + rm -f ${WORKDIR}/git/.gitignore + + cp -Rp ${WORKDIR}/git ${IMAGE_ROOTFS}/home/builder/poky + + mkdir -p ${IMAGE_ROOTFS}/home/builder/poky/build/conf + cp -Rp ${DL_DIR} ${IMAGE_ROOTFS}/home/builder/poky/build + echo "/usr/bin" > ${IMAGE_ROOTFS}/home/builder/poky/build/pseudodone + echo "BB_NO_NETWORK = \"1\"" > ${IMAGE_ROOTFS}/home/builder/poky/build/conf/auto.conf + echo "INHERIT += \"rm_work\"" >> ${IMAGE_ROOTFS}/home/builder/poky/build/conf/auto.conf + mkdir -p ${IMAGE_ROOTFS}/home/builder/pseudo + echo "export PSEUDO_PREFIX=/usr" >> ${IMAGE_ROOTFS}/home/builder/.bashrc + echo "export PSEUDO_LOCALSTATEDIR=/home/builder/pseudo" >> ${IMAGE_ROOTFS}/home/builder/.bashrc + echo "export PSEUDO_LIBDIR=/usr/lib/pseudo/lib64" >> ${IMAGE_ROOTFS}/home/builder/.bashrc + + chown builder.builder ${IMAGE_ROOTFS}/home/builder/pseudo + + chown -R builder.builder ${IMAGE_ROOTFS}/home/builder/poky +} + +IMAGE_PREPROCESS_COMMAND += "do_populate_poky_src; " + +python do_get_poky_src () { + bb.build.exec_func('base_do_fetch', d) + bb.build.exec_func('base_do_unpack', d) +} +addtask do_get_poky_src before do_rootfs
Signature · 1 line
1.7.7.6
RFC 2/5 image-vmdk: Add symbolic link
In reply to Saul Wold--- meta/classes/image-vmdk.bbclass | 2 ++ 1 files changed, 2 insertions(+), 0 deletions(-)
meta/classes/image-vmdk.bbclass +2 -0
@@ -10,6 +10,8 @@ inherit image-live create_vmdk_image () { qemu-img convert -O vmdk ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.hddimg ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.vmdk + ln -s ${IMAGE_NAME}.vmdk ${DEPLOY_DIR_IMAGE}/${IMAGE_LINK_NAME}.vmdk + } python do_vmdkimg() {
Signature · 1 line
1.7.7.6
RFC 3/5 boot-directdisk: Fix Block Calcuation
In reply to Saul WoldThis also changes the timeout to be settable
The block calcuation was not correctly rounding, see comment
Thanks to Darren Hart for fixing this.
--- meta/classes/boot-directdisk.bbclass | 19 +++++++++++++------ 1 files changed, 13 insertions(+), 6 deletions(-)
meta/classes/boot-directdisk.bbclass +13 -6
@@ -37,9 +37,8 @@ BOOTDD_EXTRA_SPACE ?= "16384" # Get the build_syslinux_cfg() function from the syslinux class AUTO_SYSLINUXCFG = "1" -LABELS = "boot" SYSLINUX_ROOT ?= "root=/dev/sda2" -SYSLINUX_TIMEOUT = "10" # 1 second +SYSLINUX_TIMEOUT ?= "10" SYSLINUXCFG = "${HDDDIR}/syslinux.cfg" SYSLINUXMENU = "${HDDDIR}/menu"
@@ -55,15 +54,23 @@ build_boot_dd() { install -m 444 ${STAGING_LIBDIR}/syslinux/ldlinux.sys ${HDDDIR}/ldlinux.sys BLOCKS=`du -bks ${HDDDIR} | cut -f 1` - SIZE=`expr $BLOCKS + ${BOOTDD_EXTRA_SPACE}` + BLOCKS=`expr $BLOCKS + ${BOOTDD_EXTRA_SPACE}` + + # Ensure total sectors is an integral number of sectors per + # track or mcopy will complain. Sectors are 512 bytes, and we + # generate images with 32 sectors per track. This calculation is + # done in blocks, thus the mod by 16 instead of 32. + BLOCKS=$(expr $BLOCKS + $(expr 16 - $(expr $BLOCKS % 16))) + + mkdosfs -n ${BOOTDD_VOLUME_ID} -S 512 -C ${HDDIMG} $BLOCKS + mcopy -i ${HDDIMG} -s ${HDDDIR}/* ::/ - mkdosfs -n ${BOOTDD_VOLUME_ID} -d ${HDDDIR} -C ${HDDIMG} $SIZE syslinux ${HDDIMG} chmod 644 ${HDDIMG} ROOTFSBLOCKS=`du -Lbks ${ROOTFS} | cut -f 1` - TOTALSIZE=`expr $SIZE + $ROOTFSBLOCKS` - END1=`expr $SIZE \* 1024` + TOTALSIZE=`expr $BLOCKS + $ROOTFSBLOCKS` + END1=`expr $BLOCKS \* 1024` END2=`expr $END1 + 512` END3=`expr \( $ROOTFSBLOCKS \* 1024 \) + $END1`
Signature · 1 line
1.7.7.6
RFC 4/5 vmdk: Update for direstdisk
In reply to Saul Wold--- meta/classes/image-vmdk.bbclass | 15 ++++++++++++--- 1 files changed, 12 insertions(+), 3 deletions(-)
meta/classes/image-vmdk.bbclass +12 -3
@@ -1,15 +1,23 @@ NOISO = "1" + +SYSLINUX_ROOT = "root=/dev/hda2 " SYSLINUX_PROMPT = "0" SYSLINUX_TIMEOUT = "1" SYSLINUX_LABELS = "boot" +LABELS_append = " ${SYSLINUX_LABELS} " + +# need to define the dependency and the ROOTFS for directdisk +do_bootdirectdisk[depends] += "${IMAGE_BASENAME}:do_rootfs" +ROOTFS ?= "${DEPLOY_DIR_IMAGE}/${IMAGE_BASENAME}-${MACHINE}.ext3" # creating VMDK relies on having a live hddimg so ensure we # inherit it here. -inherit image-live +#inherit image-live +inherit boot-directdisk create_vmdk_image () { - qemu-img convert -O vmdk ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.hddimg ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.vmdk + qemu-img convert -O vmdk ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.hdddirect ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.vmdk ln -s ${IMAGE_NAME}.vmdk ${DEPLOY_DIR_IMAGE}/${IMAGE_LINK_NAME}.vmdk }
@@ -18,7 +26,8 @@ python do_vmdkimg() { bb.build.exec_func('create_vmdk_image', d) } -addtask vmdkimg after do_bootimg before do_build +#addtask vmdkimg after do_bootimg before do_build +addtask vmdkimg after do_bootdirectdisk before do_build do_vmdkimg[nostamp] = "1" do_vmdkimg[depends] += "qemu-native:do_populate_sysroot"
Signature · 1 line
1.7.7.6
RFC 5/5 builder: Enable auto starting of Hob
In reply to Saul Wold--- meta/recipes-graphics/builder/builder_0.1.bb | 2 +- .../builder/files/builder_hob_start.sh | 10 +++++++--- 2 files changed, 8 insertions(+), 4 deletions(-)
meta/recipes-graphics/builder/builder_0.1.bb +1 -1
@@ -1,7 +1,7 @@ SUMMARY = "New user to do specific job" DESCRIPTION = "This recipe create a new user named ${PN}, who is used for specific jobs like building. The task can be auto started via mini X" SECTION = "x11" -PR = "r0" +PR = "r1" LICENSE = "MIT" LIC_FILES_CHKSUM = "file://builder_hob_start.sh;endline=5;md5=84796c3c41785d86100fdabcbdade00e"
meta/recipes-graphics/builder/files/builder_hob_start.sh +7 -3
@@ -4,8 +4,12 @@ #be put here # start hob here -#cd /intel/poky/poky -#. ./oe-init-build-env -#../scripts/hob +export PSEUDO_PREFIX=/usr +export PSEUDO_LOCALSTATEDIR=/home/builder/pseudo +export PSEUDO_LIBDIR=/usr/lib/pseudo/lib64 + +cd /home/builder/poky +. ./oe-init-build-env +hob & matchbox-terminal&
Signature · 1 line
1.7.7.6
Re: [Build App RFC 2/5] image-vmdk: Add symbolic link
In reply to Saul WoldThe idea being what? To keep a short-name symlink to the longer named actual image?
Please include a commit message.
Quoted 18 lines · Saul Wold
Signed-off-by Saul Wold <[email protected]>--- meta/classes/image-vmdk.bbclass | 2 ++ 1 files changed, 2 insertions(+), 0 deletions(-)meta/classes/image-vmdk.bbclass +2
-0@@ -10,6 +10,8 @@ inherit image-live create_vmdk_image () { qemu-img convert -O vmdk ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.hddimg ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.vmdk + ln -s ${IMAGE_NAME}.vmdk ${DEPLOY_DIR_IMAGE}/${IMAGE_LINK_NAME}.vmdk + } python do_vmdkimg() {
Signature · 3 lines
Darren Hart Intel Open Source Technology Center Yocto Project - Linux Kernel
Re: [Build App RFC 3/5] boot-directdisk: Fix Block Calcuation
In reply to Saul WoldQuoted 7 lines · Saul Wold
This also changes the timeout to be settable
The block calcuation was not correctly rounding, see comment
Thanks to Darren Hart for fixing this.
Cc Darren Hart <[email protected]>
Thanks, but be sure to use the option to get that added to actual email header :-) That's "-c" for the send-pull-request script.
--
Darren
Quoted 48 lines
Signed-off-by Saul Wold <[email protected]>--- meta/classes/boot-directdisk.bbclass | 19 +++++++++++++------ 1 files changed, 13 insertions(+), 6 deletions(-)meta/classes/boot-directdisk.bbclass +13
-6@@ -37,9 +37,8 @@ BOOTDD_EXTRA_SPACE ?= "16384" # Get the build_syslinux_cfg() function from the syslinux class AUTO_SYSLINUXCFG = "1" -LABELS = "boot" SYSLINUX_ROOT ?= "root=/dev/sda2" -SYSLINUX_TIMEOUT = "10" # 1 second +SYSLINUX_TIMEOUT ?= "10" SYSLINUXCFG = "${HDDDIR}/syslinux.cfg" SYSLINUXMENU = "${HDDDIR}/menu"@@ -55,15 +54,23 @@ build_boot_dd() { install -m 444 ${STAGING_LIBDIR}/syslinux/ldlinux.sys ${HDDDIR}/ldlinux.sys BLOCKS=`du -bks ${HDDDIR} | cut -f 1` - SIZE=`expr $BLOCKS + ${BOOTDD_EXTRA_SPACE}` + BLOCKS=`expr $BLOCKS + ${BOOTDD_EXTRA_SPACE}` + + # Ensure total sectors is an integral number of sectors per + # track or mcopy will complain. Sectors are 512 bytes, and we + # generate images with 32 sectors per track. This calculation is + # done in blocks, thus the mod by 16 instead of 32. + BLOCKS=$(expr $BLOCKS + $(expr 16 - $(expr $BLOCKS % 16))) + + mkdosfs -n ${BOOTDD_VOLUME_ID} -S 512 -C ${HDDIMG} $BLOCKS + mcopy -i ${HDDIMG} -s ${HDDDIR}/* ::/ - mkdosfs -n ${BOOTDD_VOLUME_ID} -d ${HDDDIR} -C ${HDDIMG} $SIZE syslinux ${HDDIMG} chmod 644 ${HDDIMG} ROOTFSBLOCKS=`du -Lbks ${ROOTFS} | cut -f 1` - TOTALSIZE=`expr $SIZE + $ROOTFSBLOCKS` - END1=`expr $SIZE \* 1024` + TOTALSIZE=`expr $BLOCKS + $ROOTFSBLOCKS` + END1=`expr $BLOCKS \* 1024` END2=`expr $END1 + 512` END3=`expr \( $ROOTFSBLOCKS \* 1024 \) + $END1`
Signature · 3 lines
Darren Hart Intel Open Source Technology Center Yocto Project - Linux Kernel
Re: [Build App RFC 4/5] vmdk: Update for direstdisk
In reply to Saul WoldThe conversion to directdisk is necessary due to the limitation in the mkdosfs version we carry and larger filesystems. Please note in the commit message so readers can know WHY the change was made.
Quoted 17 lines · Saul Wold
Signed-off-by Saul Wold <[email protected]>--- meta/classes/image-vmdk.bbclass | 15 ++++++++++++--- 1 files changed, 12 insertions(+), 3 deletions(-)meta/classes/image-vmdk.bbclass +2
-0@@ -1,15 +1,23 @@ NOISO = "1" + +SYSLINUX_ROOT = "root=/dev/hda2 " SYSLINUX_PROMPT = "0" SYSLINUX_TIMEOUT = "1" SYSLINUX_LABELS = "boot"
meta/classes/image-vmdk.bbclass line 16
Quoted 10 lines
meta/classes/image-vmdk.bbclass +6
-1+LABELS_append = " ${SYSLINUX_LABELS} " + +# need to define the dependency and the ROOTFS for directdisk +do_bootdirectdisk[depends] += "${IMAGE_BASENAME}:do_rootfs" +ROOTFS ?= "${DEPLOY_DIR_IMAGE}/${IMAGE_BASENAME}-${MACHINE}.ext3" # creating VMDK relies on having a live hddimg so ensure we # inherit it here. -inherit image-live +#inherit image-live
A development artifact I presume? No need to leave the old image type commented out. Should probably also remove the comment as with directdisk that is no longer the case - right?
Quoted 9 lines
meta/classes/image-vmdk.bbclass +2
-1+inherit boot-directdisk create_vmdk_image () { - qemu-img convert -O vmdk ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.hddimg ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.vmdk + qemu-img convert -O vmdk ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.hdddirect ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.vmdk ln -s ${IMAGE_NAME}.vmdk ${DEPLOY_DIR_IMAGE}/${IMAGE_LINK_NAME}.vmdk }
meta/classes/image-vmdk.bbclass python do_vmdkimg() { · line 29
Quoted 5 lines
meta/classes/image-vmdk.bbclass +1
-1@@ -18,7 +26,8 @@ python do_vmdkimg() { bb.build.exec_func('create_vmdk_image', d) } -addtask vmdkimg after do_bootimg before do_build +#addtask vmdkimg after do_bootimg before do_build
Ditto.
meta/classes/image-vmdk.bbclass python do_vmdkimg() { · line 33
Quoted 4 lines
meta/classes/image-vmdk.bbclass +1
-0+addtask vmdkimg after do_bootdirectdisk before do_build do_vmdkimg[nostamp] = "1" do_vmdkimg[depends] += "qemu-native:do_populate_sysroot"
Thanks,
Signature · 3 lines
Darren Hart Intel Open Source Technology Center Yocto Project - Linux Kernel