kernel: fix do_symlink_kernsrc function
PATCH 1/1 kernel: fix do_symlink_kernsrc function
According to the comment in do_symlink_kernsrc, this function exists for compatibility with old style kernel recipes:
# Old style kernels may set ${S} = ${WORKDIR}/git for exampleFor such recipes S will always be different from STAGING_KERNEL_DIR. This is fine for the first build or when unpack is rerun because new sources are in S. However the following command breaks the build:
bitbake -C do_symlink_kernsrc virtual/kernel
At this point, S is a symlink to STAGING_KERNEL_DIR, (meaning S != STAGING_KERNEL_DIR). We first remove the contents of STAGING_KERNEL_DIR without removing the folder itself causing us to lose kernel sources. Then we create a symlink from S to STAGING_KERNEL_DIR which results in the following broken symlinks:
${WORKDIR}/git -> <...>/build/tmp/work-shared/<machine>/kernel-source
kernel-source -> <...>/build/tmp/work-shared/<machine>/kernel-sourceThe build fails with the following error:
ERROR: <linux_recipe> do_kernel_checkout: FileExistsError(17, 'File exists')
ERROR: Task (<linux_recipe>:do_kernel_checkout)
failed with exit code '1'
Attempting to access the kernel-source directory results in:
ls: cannot access 'kernel-source': Too many levels of symbolic links
Fix this by checking if S is a symlink, and if so, verifying whether the symlink points to STAGING_KERNEL_DIR to avoid losing sources
--- meta/classes-recipe/kernel.bbclass | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-)
meta/classes-recipe/kernel.bbclass +6 -1
@@ -184,7 +184,12 @@ do_clean[cleandirs] += " ${S} ${STAGING_KERNEL_DIR} ${B} ${STAGING_KERNEL_BUILDD python do_symlink_kernsrc () { s = d.getVar("S") kernsrc = d.getVar("STAGING_KERNEL_DIR") - if s != kernsrc: + if os.path.islink(s): + _s = os.readlink(s) + else: + _s = s + + if _s != kernsrc: bb.utils.mkdirhier(kernsrc) bb.utils.remove(kernsrc, recurse=True) if s[-1] == '/':
Signature · 1 line
2.45.1
Julien Stephan via lists.openembedded.org <jstephan= [email protected]> escreveu (sexta, 5/07/2024 à(s) 12:31):
Quoted 61 lines
According to the comment in
do_symlink_kernsrc, this function exists for compatibility with old style kernel recipes:# Old style kernels may set ${S} = ${WORKDIR}/git for exampleFor such recipes S will always be different from
STAGING_KERNEL_DIR. This is fine for the first build or when unpack is rerun because new sources are in S. However the following command breaks the build:bitbake -C
do_symlink_kernsrcvirtual/kernelAt this point, S is a symlink to
STAGING_KERNEL_DIR, (meaning S !=STAGING_KERNEL_DIR). We first remove the contents ofSTAGING_KERNEL_DIRwithout removing the folder itself causing us to lose kernel sources. Then we create a symlink from S toSTAGING_KERNEL_DIRwhich results in the following broken symlinks:${WORKDIR}/git -> <...>/build/tmp/work-shared/<machine>/kernel-source kernel-source -> <...>/build/tmp/work-shared/<machine>/kernel-sourceThe build fails with the following error:
ERROR: <
linux_recipe>do_kernel_checkout: FileExistsError(17, 'File
exists')
ERROR: Task (<linux_recipe>:do_kernel_checkout)
failed with exit code '1'Attempting to access the kernel-source directory results in:
ls: cannot access 'kernel-source': Too many levels of symbolic links
Fix this by checking if S is a symlink, and if so, verifying whether the symlink
points toSTAGING_KERNEL_DIRto avoid losing sourcesSigned-off-by Julien Stephan <[email protected]>--- meta/classes-recipe/kernel.bbclass | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-)meta/classes-recipe/kernel.bbclass +0
-0@@ -184,7 +184,12 @@ do_clean[cleandirs] += " ${S} ${STAGING_KERNEL_DIR}${B} ${STAGING_KERNEL_BUILDD
pythondo_symlink_kernsrc() {
s =d.getVar("S")
kernsrc =d.getVar("STAGING_KERNEL_DIR")meta/classes-recipe/kernel.bbclass +6
-2- if s != kernsrc: + if os.path.islink(s): + _s = os.readlink(s) + else: + _s = s + + if _s != kernsrc: bb.utils.mkdirhier(kernsrc) bb.utils.remove(kernsrc, recurse=True) if s[-1] == '/': --
Quoted 2 lines
2.45.1
What kernel bbclass are you using to build your kernel? Please note that a very similar logic was replicated on the kernel-yocto.bbclass do_kernel_checkout function
so it will be good to test with and without the kernel-yocto.bbclass.
https://git.yoctoproject.org/poky/tree/meta/classes-recipe/kernel-yocto.bbclass#n376
I recently came across this problem in a change that I ended up having to revert.
https://github.com/Freescale/meta-freescale/pull/1843
https://github.com/Freescale/meta-freescale/pull/1857
Jose
Quoted 13 lines
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group. View/Reply Online (#201601):
https://lists.openembedded.org/g/openembedded-core/message/201601
Mute This Topic: https://lists.openembedded.org/mt/107052142/5052612
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [ [email protected]]
-=-=-=-=-=-=-=-=-=-=-=-
Signature · 3 lines
Best regards, José Quaresma
Quoted 70 lines · Jose Quaresma
Julien Stephan via lists.openembedded.org <[email protected]> escreveu (sexta, 5/07/2024 à(s) 12:31):
Quoted 58 lines
According to the comment in
do_symlink_kernsrc, this function exists for compatibility with old style kernel recipes:# Old style kernels may set ${S} = ${WORKDIR}/git for exampleFor such recipes S will always be different from
STAGING_KERNEL_DIR. This is fine for the first build or when unpack is rerun because new sources are in S. However the following command breaks the build:bitbake -C
do_symlink_kernsrcvirtual/kernelAt this point, S is a symlink to
STAGING_KERNEL_DIR, (meaning S !=STAGING_KERNEL_DIR). We first remove the contents ofSTAGING_KERNEL_DIRwithout removing the folder itself causing us to lose kernel sources. Then we create a symlink from S toSTAGING_KERNEL_DIRwhich results in the following broken symlinks:${WORKDIR}/git -> <...>/build/tmp/work-shared/<machine>/kernel-source kernel-source -> <...>/build/tmp/work-shared/<machine>/kernel-sourceThe build fails with the following error:
ERROR: <
linux_recipe>do_kernel_checkout:FileExistsError(17, 'File exists')
ERROR: Task (<linux_recipe>:do_kernel_checkout)
failed with exit code '1'Attempting to access the kernel-source directory results in:
ls: cannot access 'kernel-source': Too many levels of symbolic links
Fix this by checking if S is a symlink, and if so, verifying whether the symlink points to
STAGING_KERNEL_DIRto avoid losing sourcesSigned-off-by Julien Stephan <[email protected]>--- meta/classes-recipe/kernel.bbclass | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-)meta/classes-recipe/kernel.bbclass +6
-2@@ -184,7 +184,12 @@ do_clean[cleandirs] += " ${S} ${STAGING_KERNEL_DIR} ${B} ${STAGING_KERNEL_BUILDD python do_symlink_kernsrc () { s = d.getVar("S") kernsrc = d.getVar("STAGING_KERNEL_DIR") - if s != kernsrc: + if os.path.islink(s): + _s = os.readlink(s) + else: + _s = s + + if _s != kernsrc: bb.utils.mkdirhier(kernsrc) bb.utils.remove(kernsrc, recurse=True) if s[-1] == '/': --Quoted 1 line
2.45.1
What kernel bbclass are you using to build your kernel?
Please note that a very similar logic was replicated on the kernel-yocto.bbclassdo_kernel_checkoutfunction so it will be good to test with and without the kernel-yocto.bbclass.
https://git.yoctoproject.org/poky/tree/meta/classes-recipe/kernel-yocto.bbclass#n376
Quoted 5 lines
I recently came across this problem in a change that I ended up having to revert.
https://github.com/Freescale/meta-freescale/pull/1843
https://github.com/Freescale/meta-freescale/pull/1857Jose
Interesting, I'll check this. Thank you for reporting :)
Julien
Quoted 18 lines
Quoted 4 lines
Quoted 8 lines
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#201601): https://lists.openembedded.org/g/openembedded-core/message/201601
Mute This Topic: https://lists.openembedded.org/mt/107052142/5052612
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]]
-=-=-=-=-=-=-=-=-=-=-=---
Best regards,José Quaresma
Hi Julien,
did you see my previous patch and the follow-up discussion?
Ticket: https://bugzilla.yoctoproject.org/show_bug.cgi?id=15325
Previous discussion:
https://lists.openembedded.org/g/openembedded-core/topic/103308574#msg195238
I think Richard wanted a solution where this errors out:
Quoted 9 lines
I'm very tempted to say the "using a non default S value" just error out.
Whilst I understand how we got to this level of complexity, I think it will just end up causing us pain in the long run and I'd rather just not support it.
Cheers,
Richard
Étienne
Quoted 77 lines · Julien Stephan via lists.openembedded.org
According to the comment in
do_symlink_kernsrc, this function exists for compatibility with old style kernel recipes:# Old style kernels may set ${S} = ${WORKDIR}/git for exampleFor such recipes S will always be different from
STAGING_KERNEL_DIR. This is fine for the first build or when unpack is rerun because new sources are in S. However the following command breaks the build:bitbake -C
do_symlink_kernsrcvirtual/kernelAt this point, S is a symlink to
STAGING_KERNEL_DIR, (meaning S !=STAGING_KERNEL_DIR). We first remove the contents ofSTAGING_KERNEL_DIRwithout removing the folder itself causing us to lose kernel sources. Then we create a symlink from S toSTAGING_KERNEL_DIRwhich results in the following broken symlinks:${WORKDIR}/git -> <...>/build/tmp/work-shared/<machine>/kernel-source kernel-source -> <...>/build/tmp/work-shared/<machine>/kernel-sourceThe build fails with the following error:
ERROR: <
linux_recipe>do_kernel_checkout: FileExistsError(17, 'File
exists')
ERROR: Task (<linux_recipe>:do_kernel_checkout)
failed with exit code '1'Attempting to access the kernel-source directory results in:
ls: cannot access 'kernel-source': Too many levels of symbolic links
Fix this by checking if S is a symlink, and if so, verifying whether the symlink
points toSTAGING_KERNEL_DIRto avoid losing sourcesSigned-off-by Julien Stephan <[email protected]>--- meta/classes-recipe/kernel.bbclass | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-)meta/classes-recipe/kernel.bbclass +0
-0@@ -184,7 +184,12 @@ do_clean[cleandirs] += " ${S} ${STAGING_KERNEL_DIR}${B} ${STAGING_KERNEL_BUILDD
pythondo_symlink_kernsrc() {
s =d.getVar("S")
kernsrc =d.getVar("STAGING_KERNEL_DIR")meta/classes-recipe/kernel.bbclass +6
-2- if s != kernsrc: + if os.path.islink(s): + _s = os.readlink(s) + else: + _s = s + + if _s != kernsrc: bb.utils.mkdirhier(kernsrc) bb.utils.remove(kernsrc, recurse=True) if s[-1] == '/': --2.45.1
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group. View/Reply Online (#201601):
https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.openembedded.org_g_openembedded-2Dcore_message_201601&d=DwIFaQ&c=ncDTmphkJTvjIDPh0hpF_4vCHvabgGkICC2epckfdiw&r=AhkbNonVuMIGRfPx_Qj9TsRih1DULJTKUkSGa66m67E&m=ADx-AJwK4hclmY2pDBh97bnassJ0KI9f_OAKb9D6WpFFcYvO-chBh9k0SVobjYMK&s=N3u4aN5Ob1LBtzYAPrUgW7w1Yk_vN3KvuImWnp5g6LA&e=
Mute This Topic:
https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.openembedded.org_mt_107052142_7048771&d=DwIFaQ&c=ncDTmphkJTvjIDPh0hpF_4vCHvabgGkICC2epckfdiw&r=AhkbNonVuMIGRfPx_Qj9TsRih1DULJTKUkSGa66m67E&m=ADx-AJwK4hclmY2pDBh97bnassJ0KI9f_OAKb9D6WpFFcYvO-chBh9k0SVobjYMK&s=VbM00DcAJR0ZUsuhhPWtLEKKZ5MiBkhgS_Q4Aq3X3KM&e=
Group Owner: [email protected] Unsubscribe:
https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.openembedded.org_g_openembedded-2Dcore_unsub&d=DwIFaQ&c=ncDTmphkJTvjIDPh0hpF_4vCHvabgGkICC2epckfdiw&r=AhkbNonVuMIGRfPx_Qj9TsRih1DULJTKUkSGa66m67E&m=ADx-AJwK4hclmY2pDBh97bnassJ0KI9f_OAKb9D6WpFFcYvO-chBh9k0SVobjYMK&s=7EbQmI1Lz7r_bu1RArY3qabfEim4bh48mxJPm1oN_N4&e=
[[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-
Quoted 10 lines · Etienne Cordonnier
Hi Julien,
did you see my previous patch and the follow-up discussion?
Ticket: https://bugzilla.yoctoproject.org/show_bug.cgi?id=15325
Previous discussion: https://lists.openembedded.org/g/openembedded-core/topic/103308574#msg195238I think Richard wanted a solution where this errors out:
Quoted 3 lines
I'm very tempted to say the "using a non default S value" just error out.
Quoted 9 lines
Quoted 6 lines
Whilst I understand how we got to this level of complexity, I think it will just end up causing us pain in the long run and I'd rather just not support it.
Cheers,
RichardÉtienne
I didn't see it. Let me check this thread :)
Cheers
Julien
Quoted 70 lines · Julien Stephan via lists.openembedded.org
Quoted 69 lines
According to the comment in
do_symlink_kernsrc, this function exists for compatibility with old style kernel recipes:# Old style kernels may set ${S} = ${WORKDIR}/git for exampleFor such recipes S will always be different from
STAGING_KERNEL_DIR. This is fine for the first build or when unpack is rerun because new sources are in S. However the following command breaks the build:bitbake -C
do_symlink_kernsrcvirtual/kernelAt this point, S is a symlink to
STAGING_KERNEL_DIR, (meaning S !=STAGING_KERNEL_DIR). We first remove the contents ofSTAGING_KERNEL_DIRwithout removing the folder itself causing us to lose kernel sources. Then we create a symlink from S toSTAGING_KERNEL_DIRwhich results in the following broken symlinks:${WORKDIR}/git -> <...>/build/tmp/work-shared/<machine>/kernel-source kernel-source -> <...>/build/tmp/work-shared/<machine>/kernel-sourceThe build fails with the following error:
ERROR: <
linux_recipe>do_kernel_checkout:FileExistsError(17, 'File exists')
ERROR: Task (<linux_recipe>:do_kernel_checkout)
failed with exit code '1'Attempting to access the kernel-source directory results in:
ls: cannot access 'kernel-source': Too many levels of symbolic links
Fix this by checking if S is a symlink, and if so, verifying whether the symlink points to
STAGING_KERNEL_DIRto avoid losing sourcesSigned-off-by Julien Stephan <[email protected]>--- meta/classes-recipe/kernel.bbclass | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-)meta/classes-recipe/kernel.bbclass +6
-2@@ -184,7 +184,12 @@ do_clean[cleandirs] += " ${S} ${STAGING_KERNEL_DIR} ${B} ${STAGING_KERNEL_BUILDD python do_symlink_kernsrc () { s = d.getVar("S") kernsrc = d.getVar("STAGING_KERNEL_DIR") - if s != kernsrc: + if os.path.islink(s): + _s = os.readlink(s) + else: + _s = s + + if _s != kernsrc: bb.utils.mkdirhier(kernsrc) bb.utils.remove(kernsrc, recurse=True) if s[-1] == '/': --2.45.1
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#201601): https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.openembedded.org_g_openembedded-2Dcore_message_201601&d=DwIFaQ&c=ncDTmphkJTvjIDPh0hpF_4vCHvabgGkICC2epckfdiw&r=AhkbNonVuMIGRfPx_Qj9TsRih1DULJTKUkSGa66m67E&m=ADx-AJwK4hclmY2pDBh97bnassJ0KI9f_OAKb9D6WpFFcYvO-chBh9k0SVobjYMK&s=N3u4aN5Ob1LBtzYAPrUgW7w1Yk_vN3KvuImWnp5g6LA&e=
Mute This Topic: https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.openembedded.org_mt_107052142_7048771&d=DwIFaQ&c=ncDTmphkJTvjIDPh0hpF_4vCHvabgGkICC2epckfdiw&r=AhkbNonVuMIGRfPx_Qj9TsRih1DULJTKUkSGa66m67E&m=ADx-AJwK4hclmY2pDBh97bnassJ0KI9f_OAKb9D6WpFFcYvO-chBh9k0SVobjYMK&s=VbM00DcAJR0ZUsuhhPWtLEKKZ5MiBkhgS_Q4Aq3X3KM&e=
Group Owner: [email protected]
Unsubscribe: https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.openembedded.org_g_openembedded-2Dcore_unsub&d=DwIFaQ&c=ncDTmphkJTvjIDPh0hpF_4vCHvabgGkICC2epckfdiw&r=AhkbNonVuMIGRfPx_Qj9TsRih1DULJTKUkSGa66m67E&m=ADx-AJwK4hclmY2pDBh97bnassJ0KI9f_OAKb9D6WpFFcYvO-chBh9k0SVobjYMK&s=7EbQmI1Lz7r_bu1RArY3qabfEim4bh48mxJPm1oN_N4&e= [[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-