classes/kernel*: allow disabling symlink creation
cover classes/kernel*: allow disabling symlink creation
The following changes since commit 6e3785a3f1f3cf68f5fe101cd6bebe91db165973:
uninative: Don't use single sstate for pseudo-native (2020-11-24 15:53:04 +0000)
are available in the git repository at:
git://git.openembedded.org/openembedded-core-contrib paule/kernel-symlinks
http://cgit.openembedded.org/openembedded-core-contrib/log/?h=paule/kernel-symlinks
Paul Eggleton (1):
classes/kernel*: allow disabling symlink creation
meta/classes/kernel-devicetree.bbclass | 18 ++++++++++++------ meta/classes/kernel-fitimage.bbclass | 16 ++++++++++++---- meta/classes/kernel.bbclass | 14 ++++++++++---- 3 files changed, 34 insertions(+), 14 deletions(-)
Signature · 1 line
1.8.3.1
PATCH 1/1 classes/kernel*: allow disabling symlink creation
From: Paul Eggleton <[email protected]>
Allow setting any of the *_LINK_NAME variables to empty string in order to disable creating symlinks. This is particularly useful for situations where symlinks in DEPLOY_DIR_IMAGE will be treated externally as duplicates of the linked files (in which case you would probably set KERNEL_ARTIFACT_LINK_NAME = "" to disable them all at once).
--- meta/classes/kernel-devicetree.bbclass | 18 ++++++++++++------ meta/classes/kernel-fitimage.bbclass | 16 ++++++++++++---- meta/classes/kernel.bbclass | 14 ++++++++++---- 3 files changed, 34 insertions(+), 14 deletions(-)
meta/classes/kernel-devicetree.bbclass +12 -6
@@ -73,21 +73,27 @@ do_deploy_append() { dtb_base_name=`basename $dtb .$dtb_ext` install -d $deployDir install -m 0644 ${D}/${KERNEL_IMAGEDEST}/$dtb_base_name.$dtb_ext $deployDir/$dtb_base_name-${KERNEL_DTB_NAME}.$dtb_ext - ln -sf $dtb_base_name-${KERNEL_DTB_NAME}.$dtb_ext $deployDir/$dtb_base_name.$dtb_ext - ln -sf $dtb_base_name-${KERNEL_DTB_NAME}.$dtb_ext $deployDir/$dtb_base_name-${KERNEL_DTB_LINK_NAME}.$dtb_ext + if [ -n "${KERNEL_DTB_LINK_NAME}" ] ; then + ln -sf $dtb_base_name-${KERNEL_DTB_NAME}.$dtb_ext $deployDir/$dtb_base_name.$dtb_ext + ln -sf $dtb_base_name-${KERNEL_DTB_NAME}.$dtb_ext $deployDir/$dtb_base_name-${KERNEL_DTB_LINK_NAME}.$dtb_ext + fi for type in ${KERNEL_IMAGETYPE_FOR_MAKE}; do if [ "$type" = "zImage" ] && [ "${KERNEL_DEVICETREE_BUNDLE}" = "1" ]; then cat ${D}/${KERNEL_IMAGEDEST}/$type \ $deployDir/$dtb_base_name-${KERNEL_DTB_NAME}.$dtb_ext \ > $deployDir/$type-$dtb_base_name-${KERNEL_DTB_NAME}.$dtb_ext.bin - ln -sf $type-$dtb_base_name-${KERNEL_DTB_NAME}.$dtb_ext.bin \ - $deployDir/$type-$dtb_base_name-${KERNEL_DTB_LINK_NAME}.$dtb_ext.bin + if [ -n "${KERNEL_DTB_LINK_NAME}" ]; then + ln -sf $type-$dtb_base_name-${KERNEL_DTB_NAME}.$dtb_ext.bin \ + $deployDir/$type-$dtb_base_name-${KERNEL_DTB_LINK_NAME}.$dtb_ext.bin + fi if [ -e "${KERNEL_OUTPUT_DIR}/${type}.initramfs" ]; then cat ${KERNEL_OUTPUT_DIR}/${type}.initramfs \ $deployDir/$dtb_base_name-${KERNEL_DTB_NAME}.$dtb_ext \ > $deployDir/${type}-${INITRAMFS_NAME}-$dtb_base_name-${KERNEL_DTB_NAME}.$dtb_ext.bin - ln -sf ${type}-${INITRAMFS_NAME}-$dtb_base_name-${KERNEL_DTB_NAME}.$dtb_ext.bin \ - $deployDir/${type}-${INITRAMFS_NAME}-$dtb_base_name-${KERNEL_DTB_LINK_NAME}.$dtb_ext.bin + if [ -n "${KERNEL_DTB_LINK_NAME}" ]; then + ln -sf ${type}-${INITRAMFS_NAME}-$dtb_base_name-${KERNEL_DTB_NAME}.$dtb_ext.bin \ + $deployDir/${type}-${INITRAMFS_NAME}-$dtb_base_name-${KERNEL_DTB_LINK_NAME}.$dtb_ext.bin + fi fi fi done
meta/classes/kernel-fitimage.bbclass +12 -4
@@ -573,20 +573,28 @@ kernel_do_deploy_append() { if echo ${KERNEL_IMAGETYPES} | grep -wq "fitImage"; then echo "Copying fit-image.its source file..." install -m 0644 ${B}/fit-image.its "$deployDir/fitImage-its-${KERNEL_FIT_NAME}.its" - ln -snf fitImage-its-${KERNEL_FIT_NAME}.its "$deployDir/fitImage-its-${KERNEL_FIT_LINK_NAME}" + if [ -n "${KERNEL_FIT_LINK_NAME}" ] ; then + ln -snf fitImage-its-${KERNEL_FIT_NAME}.its "$deployDir/fitImage-its-${KERNEL_FIT_LINK_NAME}" + fi echo "Copying linux.bin file..." install -m 0644 ${B}/linux.bin $deployDir/fitImage-linux.bin-${KERNEL_FIT_NAME}.bin - ln -snf fitImage-linux.bin-${KERNEL_FIT_NAME}.bin "$deployDir/fitImage-linux.bin-${KERNEL_FIT_LINK_NAME}" + if [ -n "${KERNEL_FIT_LINK_NAME}" ] ; then + ln -snf fitImage-linux.bin-${KERNEL_FIT_NAME}.bin "$deployDir/fitImage-linux.bin-${KERNEL_FIT_LINK_NAME}" + fi if [ -n "${INITRAMFS_IMAGE}" ]; then echo "Copying fit-image-${INITRAMFS_IMAGE}.its source file..." install -m 0644 ${B}/fit-image-${INITRAMFS_IMAGE}.its "$deployDir/fitImage-its-${INITRAMFS_IMAGE_NAME}-${KERNEL_FIT_NAME}.its" - ln -snf fitImage-its-${INITRAMFS_IMAGE_NAME}-${KERNEL_FIT_NAME}.its "$deployDir/fitImage-its-${INITRAMFS_IMAGE_NAME}-${KERNEL_FIT_LINK_NAME}" + if [ -n "${KERNEL_FIT_LINK_NAME}" ] ; then + ln -snf fitImage-its-${INITRAMFS_IMAGE_NAME}-${KERNEL_FIT_NAME}.its "$deployDir/fitImage-its-${INITRAMFS_IMAGE_NAME}-${KERNEL_FIT_LINK_NAME}" + fi echo "Copying fitImage-${INITRAMFS_IMAGE} file..." install -m 0644 ${B}/arch/${ARCH}/boot/fitImage-${INITRAMFS_IMAGE} "$deployDir/fitImage-${INITRAMFS_IMAGE_NAME}-${KERNEL_FIT_NAME}.bin" - ln -snf fitImage-${INITRAMFS_IMAGE_NAME}-${KERNEL_FIT_NAME}.bin "$deployDir/fitImage-${INITRAMFS_IMAGE_NAME}-${KERNEL_FIT_LINK_NAME}" + if [ -n "${KERNEL_FIT_LINK_NAME}" ] ; then + ln -snf fitImage-${INITRAMFS_IMAGE_NAME}-${KERNEL_FIT_NAME}.bin "$deployDir/fitImage-${INITRAMFS_IMAGE_NAME}-${KERNEL_FIT_LINK_NAME}" + fi fi if [ "${UBOOT_SIGN_ENABLE}" = "1" -a -n "${UBOOT_DTB_BINARY}" ] ; then # UBOOT_DTB_IMAGE is a realfile, but we can't use
meta/classes/kernel.bbclass +10 -4
@@ -725,8 +725,10 @@ kernel_do_deploy() { for imageType in ${KERNEL_IMAGETYPES} ; do baseName=$imageType-${KERNEL_IMAGE_NAME} install -m 0644 ${KERNEL_OUTPUT_DIR}/$imageType $deployDir/$baseName.bin - ln -sf $baseName.bin $deployDir/$imageType-${KERNEL_IMAGE_LINK_NAME}.bin - ln -sf $baseName.bin $deployDir/$imageType + if [ -n "${KERNEL_IMAGE_LINK_NAME}" ] ; then + ln -sf $baseName.bin $deployDir/$imageType-${KERNEL_IMAGE_LINK_NAME}.bin + ln -sf $baseName.bin $deployDir/$imageType + fi done if [ ${MODULE_TARBALL_DEPLOY} = "1" ] && (grep -q -i -e '^CONFIG_MODULES=y$' .config); then
@@ -739,7 +741,9 @@ kernel_do_deploy() { TAR_ARGS="$TAR_ARGS --owner=0 --group=0" tar $TAR_ARGS -cv -C ${D}${root_prefix} lib | gzip -9n > $deployDir/modules-${MODULE_TARBALL_NAME}.tgz - ln -sf modules-${MODULE_TARBALL_NAME}.tgz $deployDir/modules-${MODULE_TARBALL_LINK_NAME}.tgz + if [ -n "${MODULE_TARBALL_LINK_NAME}" ] ; then + ln -sf modules-${MODULE_TARBALL_NAME}.tgz $deployDir/modules-${MODULE_TARBALL_LINK_NAME}.tgz + fi fi if [ ! -z "${INITRAMFS_IMAGE}" -a x"${INITRAMFS_IMAGE_BUNDLE}" = x1 ]; then
@@ -749,7 +753,9 @@ kernel_do_deploy() { fi initramfsBaseName=$imageType-${INITRAMFS_NAME} install -m 0644 ${KERNEL_OUTPUT_DIR}/$imageType.initramfs $deployDir/$initramfsBaseName.bin - ln -sf $initramfsBaseName.bin $deployDir/$imageType-${INITRAMFS_LINK_NAME}.bin + if [ -n "${INITRAMFS_LINK_NAME}" ] ; then + ln -sf $initramfsBaseName.bin $deployDir/$imageType-${INITRAMFS_LINK_NAME}.bin + fi done fi }
Signature · 1 line
1.8.3.1
Quoted 7 lines · On Thursday, 26 November 2020 14:01:47 NZDT Paul Eggleton wrote:
From: Paul Eggleton <[email protected]>
Allow setting any of the
*_LINK_NAMEvariables to empty string in order to disable creating symlinks. This is particularly useful for situations where symlinks inDEPLOY_DIR_IMAGEwill be treated externally as duplicates of the linked files (in which case you would probably setKERNEL_ARTIFACT_LINK_NAME= "" to disable them all at once).
I missed something - v2 incoming.
Cheers
Paul
Can we use a separate variable controlling this behavior other than *_LINK_NAME variables being set to empty?
Then we could easily extend it to stop creating the links also for image artifacts as well.
Quoted 20 lines · Paul Eggleton
Quoted 7 lines · On Thursday, 26 November 2020 14:01:47 NZDT Paul Eggleton wrote:
From: Paul Eggleton <[email protected]>
Allow setting any of the
*_LINK_NAMEvariables to empty string in order to disable creating symlinks. This is particularly useful for situations where symlinks inDEPLOY_DIR_IMAGEwill be treated externally as duplicates of the linked files (in which case you would probably setKERNEL_ARTIFACT_LINK_NAME= "" to disable them all at once).I missed something - v2 incoming.
Cheers
Paul
Hi Martin
(replying all this time)
Quoted 5 lines · On Tuesday, 1 December 2020 15:40:36 NZDT you wrote:
Can we use a separate variable controlling this behavior other than
*_LINK_NAMEvariables being set to empty?Then we could easily extend it to stop creating the links also for image artifacts as well.
Well, I didn't call it out explicitly but I was just mirroring the image artifacts behaviour - we set those to "" as well and there are checks there already to make that disable the symlink in that case.
Paul
Hi Paul,
I see, I was just worried that this case will block me pushing the last bit of [YOCTO #12937] which changes symlinks to hardlinks:
https://git.openembedded.org/openembedded-core-contrib/log/?h=jansa/artifacts
Which makes the use case you're fixing worse (as even less systems will correctly handle hardlinks), so I support the idea to disable having the links. My current work around is to just remove the unnecessary files when rsyncing to file-server.
But the main issue which I'm seeing with my current implementation (and with webos_deploy this is based on) is that we need the version-less artifacts to be available, because e.g. image recipe searching kernel artifact when there is DATETIME or SRCPV in the kernel artifact-name won't be able to find it, because the image might be built at different DATETIME and doesn't know anything about kernel SRCREV/SRCPV. To resolve this, the build is using the version-less artifacts (always the latest one).
I guess your implementation might fail in such DATETIME cases as well.
With separate variable to disable the links we can somehow work around this or at least warn user that without the version-less filenames, the VERSION_SUFFIX variable needs to be stable (as no PV/SRCPV/DATATIME in it).
Cheers,
Quoted 20 lines · Paul Eggleton
Hi Martin
(replying all this time)
Quoted 5 lines · On Tuesday, 1 December 2020 15:40:36 NZDT you wrote:
Can we use a separate variable controlling this behavior other than
*_LINK_NAMEvariables being set to empty?Then we could easily extend it to stop creating the links also for image artifacts as well.
Well, I didn't call it out explicitly but I was just mirroring the image artifacts behaviour - we set those to "" as well and there are checks there
already to make that disable the symlink in that case.Paul
Quoted 18 lines · On Wednesday, 2 December 2020 05:22:43 NZDT Martin Jansa wrote:
I see, I was just worried that this case will block me pushing the last bit of [YOCTO #12937] which changes symlinks to hardlinks:
https://git.openembedded.org/openembedded-core-contrib/log/?h=jansa/artifact
sWhich makes the use case you're fixing worse (as even less systems will correctly handle hardlinks), so I support the idea to disable having the links. My current work around is to just remove the unnecessary files when rsyncing to file-server.
But the main issue which I'm seeing with my current implementation (and with
webos_deploythis is based on) is that we need the version-less artifacts to be available, because e.g. image recipe searching kernel artifact when there is DATETIME or SRCPV in the kernel artifact-name won't be able to find it, because the image might be built at different DATETIME and doesn't know anything about kernel SRCREV/SRCPV. To resolve this, the build is using the version-less artifacts (always the latest one).
Quoted 5 lines
I guess your implementation might fail in such DATETIME cases as well.
With separate variable to disable the links we can somehow work around this or at least warn user that without the version-less filenames, the
VERSION_SUFFIXvariable needs to be stable (as no PV/SRCPV/DATATIME in it).
That is a good point and we have the same problem here - what I'm planning to do is to set a bunch of the other variables such that there are no versioned artifacts - i.e. the real files effectively appear in place of the symlinks and the versioned ones disappear. It seems to work in my tests so far. With my patchset this is what I set:
------------ snip ------------IMAGE_LINK_NAME = ""IMAGE_VERSION_SUFFIX = ""IMAGE_NAME_SUFFIX = ""IMAGE_MANIFEST_distroname = "${IMGDEPLOYDIR}/${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.manifest"
KERNEL_ARTIFACT_NAME = "${MACHINE}"KERNEL_ARTIFACT_LINK_NAME = ""
------------ snip ------------
I have found that I still need the plain type named kernel artifacts (e.g. vmlinux), and since these don't actually use the link name variable I have changed those to be written out even if the link name is blank (v2 patchset still in testing). We could introduce another variable to disable those but I hadn't planned to do that for the moment.
Cheers,
Paul