Skip to contentSkip to conversation

classes/kernel*: allow disabling symlink creation

PATCH In review Paul Eggleton 3 participants 7 messages Nov 26, 2020 – Dec 1, 2020

lore
Share & tools
lore
#0 lore raw

cover classes/kernel*: allow disabling symlink creation

The following changes since commit 6e3785a3f1f3cf68f5fe101cd6bebe91db165973:

uninative: Don't use single sstate for pseudo-native (2020-11-24 15:53:04 +0000)

are available in the git repository at:

git://git.openembedded.org/openembedded-core-contrib paule/kernel-symlinks
http://cgit.openembedded.org/openembedded-core-contrib/log/?h=paule/kernel-symlinks

Paul Eggleton (1):
classes/kernel*: allow disabling symlink creation

 meta/classes/kernel-devicetree.bbclass | 18 ++++++++++++------
 meta/classes/kernel-fitimage.bbclass   | 16 ++++++++++++----
 meta/classes/kernel.bbclass            | 14 ++++++++++----
 3 files changed, 34 insertions(+), 14 deletions(-)
Signature · 1 line
1.8.3.1
#1 lore raw

PATCH 1/1 classes/kernel*: allow disabling symlink creation

From: Paul Eggleton <[email protected]>

Allow setting any of the *_LINK_NAME variables to empty string in order to disable creating symlinks. This is particularly useful for situations where symlinks in DEPLOY_DIR_IMAGE will be treated externally as duplicates of the linked files (in which case you would probably set KERNEL_ARTIFACT_LINK_NAME = "" to disable them all at once).

---
 meta/classes/kernel-devicetree.bbclass | 18 ++++++++++++------
 meta/classes/kernel-fitimage.bbclass   | 16 ++++++++++++----
 meta/classes/kernel.bbclass            | 14 ++++++++++----
 3 files changed, 34 insertions(+), 14 deletions(-)
meta/classes/kernel-devicetree.bbclass +12 -6
index 81dda80..7cde549 100644
@@ -73,21 +73,27 @@ do_deploy_append() {
 		dtb_base_name=`basename $dtb .$dtb_ext`
 		install -d $deployDir
 		install -m 0644 ${D}/${KERNEL_IMAGEDEST}/$dtb_base_name.$dtb_ext $deployDir/$dtb_base_name-${KERNEL_DTB_NAME}.$dtb_ext
-		ln -sf $dtb_base_name-${KERNEL_DTB_NAME}.$dtb_ext $deployDir/$dtb_base_name.$dtb_ext
-		ln -sf $dtb_base_name-${KERNEL_DTB_NAME}.$dtb_ext $deployDir/$dtb_base_name-${KERNEL_DTB_LINK_NAME}.$dtb_ext
+		if [ -n "${KERNEL_DTB_LINK_NAME}" ] ; then
+			ln -sf $dtb_base_name-${KERNEL_DTB_NAME}.$dtb_ext $deployDir/$dtb_base_name.$dtb_ext
+			ln -sf $dtb_base_name-${KERNEL_DTB_NAME}.$dtb_ext $deployDir/$dtb_base_name-${KERNEL_DTB_LINK_NAME}.$dtb_ext
+		fi
 		for type in ${KERNEL_IMAGETYPE_FOR_MAKE}; do
 			if [ "$type" = "zImage" ] && [ "${KERNEL_DEVICETREE_BUNDLE}" = "1" ]; then
 				cat ${D}/${KERNEL_IMAGEDEST}/$type \
 					$deployDir/$dtb_base_name-${KERNEL_DTB_NAME}.$dtb_ext \
 					> $deployDir/$type-$dtb_base_name-${KERNEL_DTB_NAME}.$dtb_ext.bin
-				ln -sf $type-$dtb_base_name-${KERNEL_DTB_NAME}.$dtb_ext.bin \
-					$deployDir/$type-$dtb_base_name-${KERNEL_DTB_LINK_NAME}.$dtb_ext.bin
+				if [ -n "${KERNEL_DTB_LINK_NAME}" ]; then
+					ln -sf $type-$dtb_base_name-${KERNEL_DTB_NAME}.$dtb_ext.bin \
+						$deployDir/$type-$dtb_base_name-${KERNEL_DTB_LINK_NAME}.$dtb_ext.bin
+				fi
 				if [ -e "${KERNEL_OUTPUT_DIR}/${type}.initramfs" ]; then
 					cat ${KERNEL_OUTPUT_DIR}/${type}.initramfs \
 						$deployDir/$dtb_base_name-${KERNEL_DTB_NAME}.$dtb_ext \
 						>  $deployDir/${type}-${INITRAMFS_NAME}-$dtb_base_name-${KERNEL_DTB_NAME}.$dtb_ext.bin
-					ln -sf ${type}-${INITRAMFS_NAME}-$dtb_base_name-${KERNEL_DTB_NAME}.$dtb_ext.bin \
-						$deployDir/${type}-${INITRAMFS_NAME}-$dtb_base_name-${KERNEL_DTB_LINK_NAME}.$dtb_ext.bin
+					if [ -n "${KERNEL_DTB_LINK_NAME}" ]; then
+						ln -sf ${type}-${INITRAMFS_NAME}-$dtb_base_name-${KERNEL_DTB_NAME}.$dtb_ext.bin \
+							$deployDir/${type}-${INITRAMFS_NAME}-$dtb_base_name-${KERNEL_DTB_LINK_NAME}.$dtb_ext.bin
+					fi
 				fi
 			fi
 		done
meta/classes/kernel-fitimage.bbclass +12 -4
index bb2f3c4..e91c95f 100644
@@ -573,20 +573,28 @@ kernel_do_deploy_append() {
 	if echo ${KERNEL_IMAGETYPES} | grep -wq "fitImage"; then
 		echo "Copying fit-image.its source file..."
 		install -m 0644 ${B}/fit-image.its "$deployDir/fitImage-its-${KERNEL_FIT_NAME}.its"
-		ln -snf fitImage-its-${KERNEL_FIT_NAME}.its "$deployDir/fitImage-its-${KERNEL_FIT_LINK_NAME}"
+		if [ -n "${KERNEL_FIT_LINK_NAME}" ] ; then
+			ln -snf fitImage-its-${KERNEL_FIT_NAME}.its "$deployDir/fitImage-its-${KERNEL_FIT_LINK_NAME}"
+		fi
 
 		echo "Copying linux.bin file..."
 		install -m 0644 ${B}/linux.bin $deployDir/fitImage-linux.bin-${KERNEL_FIT_NAME}.bin
-		ln -snf fitImage-linux.bin-${KERNEL_FIT_NAME}.bin "$deployDir/fitImage-linux.bin-${KERNEL_FIT_LINK_NAME}"
+		if [ -n "${KERNEL_FIT_LINK_NAME}" ] ; then
+			ln -snf fitImage-linux.bin-${KERNEL_FIT_NAME}.bin "$deployDir/fitImage-linux.bin-${KERNEL_FIT_LINK_NAME}"
+		fi
 
 		if [ -n "${INITRAMFS_IMAGE}" ]; then
 			echo "Copying fit-image-${INITRAMFS_IMAGE}.its source file..."
 			install -m 0644 ${B}/fit-image-${INITRAMFS_IMAGE}.its "$deployDir/fitImage-its-${INITRAMFS_IMAGE_NAME}-${KERNEL_FIT_NAME}.its"
-			ln -snf fitImage-its-${INITRAMFS_IMAGE_NAME}-${KERNEL_FIT_NAME}.its "$deployDir/fitImage-its-${INITRAMFS_IMAGE_NAME}-${KERNEL_FIT_LINK_NAME}"
+			if [ -n "${KERNEL_FIT_LINK_NAME}" ] ; then
+				ln -snf fitImage-its-${INITRAMFS_IMAGE_NAME}-${KERNEL_FIT_NAME}.its "$deployDir/fitImage-its-${INITRAMFS_IMAGE_NAME}-${KERNEL_FIT_LINK_NAME}"
+			fi
 
 			echo "Copying fitImage-${INITRAMFS_IMAGE} file..."
 			install -m 0644 ${B}/arch/${ARCH}/boot/fitImage-${INITRAMFS_IMAGE} "$deployDir/fitImage-${INITRAMFS_IMAGE_NAME}-${KERNEL_FIT_NAME}.bin"
-			ln -snf fitImage-${INITRAMFS_IMAGE_NAME}-${KERNEL_FIT_NAME}.bin "$deployDir/fitImage-${INITRAMFS_IMAGE_NAME}-${KERNEL_FIT_LINK_NAME}"
+			if [ -n "${KERNEL_FIT_LINK_NAME}" ] ; then
+				ln -snf fitImage-${INITRAMFS_IMAGE_NAME}-${KERNEL_FIT_NAME}.bin "$deployDir/fitImage-${INITRAMFS_IMAGE_NAME}-${KERNEL_FIT_LINK_NAME}"
+			fi
 		fi
 		if [ "${UBOOT_SIGN_ENABLE}" = "1" -a -n "${UBOOT_DTB_BINARY}" ] ; then
 			# UBOOT_DTB_IMAGE is a realfile, but we can't use
meta/classes/kernel.bbclass +10 -4
index 1a444ef..299e559 100644
@@ -725,8 +725,10 @@ kernel_do_deploy() {
 	for imageType in ${KERNEL_IMAGETYPES} ; do
 		baseName=$imageType-${KERNEL_IMAGE_NAME}
 		install -m 0644 ${KERNEL_OUTPUT_DIR}/$imageType $deployDir/$baseName.bin
-		ln -sf $baseName.bin $deployDir/$imageType-${KERNEL_IMAGE_LINK_NAME}.bin
-		ln -sf $baseName.bin $deployDir/$imageType
+		if [ -n "${KERNEL_IMAGE_LINK_NAME}" ] ; then
+			ln -sf $baseName.bin $deployDir/$imageType-${KERNEL_IMAGE_LINK_NAME}.bin
+			ln -sf $baseName.bin $deployDir/$imageType
+		fi
 	done
 
 	if [ ${MODULE_TARBALL_DEPLOY} = "1" ] && (grep -q -i -e '^CONFIG_MODULES=y$' .config); then
@@ -739,7 +741,9 @@ kernel_do_deploy() {
 		TAR_ARGS="$TAR_ARGS --owner=0 --group=0"
 		tar $TAR_ARGS -cv -C ${D}${root_prefix} lib | gzip -9n > $deployDir/modules-${MODULE_TARBALL_NAME}.tgz
 
-		ln -sf modules-${MODULE_TARBALL_NAME}.tgz $deployDir/modules-${MODULE_TARBALL_LINK_NAME}.tgz
+		if [ -n "${MODULE_TARBALL_LINK_NAME}" ] ; then
+			ln -sf modules-${MODULE_TARBALL_NAME}.tgz $deployDir/modules-${MODULE_TARBALL_LINK_NAME}.tgz
+		fi
 	fi
 
 	if [ ! -z "${INITRAMFS_IMAGE}" -a x"${INITRAMFS_IMAGE_BUNDLE}" = x1 ]; then
@@ -749,7 +753,9 @@ kernel_do_deploy() {
 			fi
 			initramfsBaseName=$imageType-${INITRAMFS_NAME}
 			install -m 0644 ${KERNEL_OUTPUT_DIR}/$imageType.initramfs $deployDir/$initramfsBaseName.bin
-			ln -sf $initramfsBaseName.bin $deployDir/$imageType-${INITRAMFS_LINK_NAME}.bin
+			if [ -n "${INITRAMFS_LINK_NAME}" ] ; then
+				ln -sf $initramfsBaseName.bin $deployDir/$imageType-${INITRAMFS_LINK_NAME}.bin
+			fi
 		done
 	fi
 }
Signature · 1 line
1.8.3.1
#2 lore raw
Quoted 7 lines · On Thursday, 26 November 2020 14:01:47 NZDT Paul Eggleton wrote:

From: Paul Eggleton <[email protected]>

Allow setting any of the *_LINK_NAME variables to empty string in order to disable creating symlinks. This is particularly useful for situations where symlinks in DEPLOY_DIR_IMAGE will be treated externally as duplicates of the linked files (in which case you would probably set KERNEL_ARTIFACT_LINK_NAME = "" to disable them all at once).

I missed something - v2 incoming.

Cheers
Paul

#3 lore raw

Can we use a separate variable controlling this behavior other than *_LINK_NAME variables being set to empty?

Then we could easily extend it to stop creating the links also for image artifacts as well.

Quoted 20 lines · Paul Eggleton
Quoted 7 lines · On Thursday, 26 November 2020 14:01:47 NZDT Paul Eggleton wrote:

From: Paul Eggleton <[email protected]>

Allow setting any of the *_LINK_NAME variables to empty string in order to disable creating symlinks. This is particularly useful for situations where symlinks in DEPLOY_DIR_IMAGE will be treated externally as duplicates of the linked files (in which case you would probably set KERNEL_ARTIFACT_LINK_NAME = "" to disable them all at once).

I missed something - v2 incoming.

Cheers
Paul

#4 lore raw

Hi Martin

(replying all this time)

Quoted 5 lines · On Tuesday, 1 December 2020 15:40:36 NZDT you wrote:

Can we use a separate variable controlling this behavior other than *_LINK_NAME variables being set to empty?

Then we could easily extend it to stop creating the links also for image artifacts as well.

Well, I didn't call it out explicitly but I was just mirroring the image artifacts behaviour - we set those to "" as well and there are checks there already to make that disable the symlink in that case.

Paul

#5 lore raw

Hi Paul,

I see, I was just worried that this case will block me pushing the last bit of [YOCTO #12937] which changes symlinks to hardlinks:
https://git.openembedded.org/openembedded-core-contrib/log/?h=jansa/artifacts

Which makes the use case you're fixing worse (as even less systems will correctly handle hardlinks), so I support the idea to disable having the links. My current work around is to just remove the unnecessary files when rsyncing to file-server.

But the main issue which I'm seeing with my current implementation (and with webos_deploy this is based on) is that we need the version-less artifacts to be available, because e.g. image recipe searching kernel artifact when there is DATETIME or SRCPV in the kernel artifact-name won't be able to find it, because the image might be built at different DATETIME and doesn't know anything about kernel SRCREV/SRCPV. To resolve this, the build is using the version-less artifacts (always the latest one).

I guess your implementation might fail in such DATETIME cases as well.

With separate variable to disable the links we can somehow work around this or at least warn user that without the version-less filenames, the VERSION_SUFFIX variable needs to be stable (as no PV/SRCPV/DATATIME in it).

Cheers,

Quoted 20 lines · Paul Eggleton

Hi Martin

(replying all this time)

Quoted 5 lines · On Tuesday, 1 December 2020 15:40:36 NZDT you wrote:

Can we use a separate variable controlling this behavior other than *_LINK_NAME variables being set to empty?

Then we could easily extend it to stop creating the links also for image artifacts as well.

Well, I didn't call it out explicitly but I was just mirroring the image artifacts behaviour - we set those to "" as well and there are checks there
already to make that disable the symlink in that case.

Paul

#6 lore raw
Quoted 18 lines · On Wednesday, 2 December 2020 05:22:43 NZDT Martin Jansa wrote:

I see, I was just worried that this case will block me pushing the last bit of [YOCTO #12937] which changes symlinks to hardlinks:
https://git.openembedded.org/openembedded-core-contrib/log/?h=jansa/artifact
s

Which makes the use case you're fixing worse (as even less systems will correctly handle hardlinks), so I support the idea to disable having the links. My current work around is to just remove the unnecessary files when rsyncing to file-server.

But the main issue which I'm seeing with my current implementation (and with webos_deploy this is based on) is that we need the version-less artifacts to be available, because e.g. image recipe searching kernel artifact when there is DATETIME or SRCPV in the kernel artifact-name won't be able to find it, because the image might be built at different DATETIME and doesn't know anything about kernel SRCREV/SRCPV. To resolve this, the build is using the version-less artifacts (always the latest one).

Quoted 5 lines

I guess your implementation might fail in such DATETIME cases as well.

With separate variable to disable the links we can somehow work around this or at least warn user that without the version-less filenames, the VERSION_SUFFIX variable needs to be stable (as no PV/SRCPV/DATATIME in it).

That is a good point and we have the same problem here - what I'm planning to do is to set a bunch of the other variables such that there are no versioned artifacts - i.e. the real files effectively appear in place of the symlinks and the versioned ones disappear. It seems to work in my tests so far. With my patchset this is what I set:

------------ snip ------------
IMAGE_LINK_NAME = ""
IMAGE_VERSION_SUFFIX = ""
IMAGE_NAME_SUFFIX = ""
IMAGE_MANIFEST_distroname = "${IMGDEPLOYDIR}/${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.manifest"

KERNEL_ARTIFACT_NAME = "${MACHINE}"
KERNEL_ARTIFACT_LINK_NAME = ""
------------ snip ------------

I have found that I still need the plain type named kernel artifacts (e.g. vmlinux), and since these don't actually use the link name variable I have changed those to be written out even if the link name is blank (v2 patchset still in testing). We could introduce another variable to disable those but I hadn't planned to do that for the moment.

Cheers,
Paul