Skip to contentSkip to conversation

kernel.bbclass: make do_symlink_kernelsrc reentrant

PATCH Etienne Cordonnier 3 participants 6 messages Dec 19, 2023 – Dec 21, 2023 1 message not in the archive

lore
Share & tools
lore

A message that is not in the archive (sent off-list or not yet received): [email protected]

#1 lore raw

Patch v4: use idempotent copy of the files instead of simply skipping the code (see discussion under v2 of the patch).

Quoted 64 lines · Etienne Cordonnier via lists.openembedded.org

From: Etienne Cordonnier <[email protected]>

The function do_symlink_kernsrc is not reentrant in the case where S is defined
to a non-default value. This causes build-failures e.g. when building linux-yocto, then updating
poky to a commit which modifies kernel.bbclass, and then building linux-yocto again.

Bugzilla:
https://urldefense.proofpoint.com/v2/url?u=https-3A__bugzilla.yoctoproject.org_show-5Fbug.cgi-3Fid-3D15325&d=DwIDAg&c=ncDTmphkJTvjIDPh0hpF_4vCHvabgGkICC2epckfdiw&r=AhkbNonVuMIGRfPx_Qj9TsRih1DULJTKUkSGa66m67E&m=VB1zXAiE9q_CqM9DgoA0BbtloUxpQE_rQ7eqk7HGzBW9KzFu44K9hS1iahsqSuIk&s=9Cr8vwWgUwwGM5oCO07DB4sO9sQpMCdweS1u8CmZ0Qw&e=

Tested with a recipe "my-custom-linux" which unpacks sources to a custom ${S} directory
and ran symlink_kernsrc several times:

$ bitbake -f -c symlink_kernsrc my-custom-linux
---
 meta/classes-recipe/kernel.bbclass | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)
meta/classes-recipe/kernel.bbclass +0 -0
index 9ff37f5c38..42c5519f81 100644
@@ -189,11 +189,17 @@ python do_symlink_kernsrc () {
             # drop trailing slash, so that os.symlink(kernsrc, s) doesn't

use s as

             # directory name and fail
             s = s[:-1]
meta/classes-recipe/kernel.bbclass +7 -2
-        if d.getVar("EXTERNALSRC"):
+        if d.getVar("EXTERNALSRC") and not os.path.islink(s):
             # With EXTERNALSRC S will not be wiped so we can symlink to it
             os.symlink(s, kernsrc)
         else:
             import shutil
+            # perform idempotent/reentrant copy
+            s_copy = s + ".orig"
+            if not os.path.isdir(s_copy):
+                shutil.copytree(s, s_copy)
+            bb.utils.remove(s, recurse=True)
+            shutil.copytree(s_copy, s)
             shutil.move(s, kernsrc)
             os.symlink(kernsrc, s)
 }
--

2.36.1.vfs.0.0

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group. View/Reply Online (#192730):
https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.openembedded.org_g_openembedded-2Dcore_message_192730&d=DwIFaQ&c=ncDTmphkJTvjIDPh0hpF_4vCHvabgGkICC2epckfdiw&r=AhkbNonVuMIGRfPx_Qj9TsRih1DULJTKUkSGa66m67E&m=VB1zXAiE9q_CqM9DgoA0BbtloUxpQE_rQ7eqk7HGzBW9KzFu44K9hS1iahsqSuIk&s=HC8D2EMiR_G_qbq-osmSiZT2ydpAE87MF9ylA3VnOiU&e=
Mute This Topic:
https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.openembedded.org_mt_103262114_7048771&d=DwIFaQ&c=ncDTmphkJTvjIDPh0hpF_4vCHvabgGkICC2epckfdiw&r=AhkbNonVuMIGRfPx_Qj9TsRih1DULJTKUkSGa66m67E&m=VB1zXAiE9q_CqM9DgoA0BbtloUxpQE_rQ7eqk7HGzBW9KzFu44K9hS1iahsqSuIk&s=kJzafl7EdrE7kH-hS7V1tMze-xICCnm60bSF50NZ4iA&e=
Group Owner: [email protected] Unsubscribe:
https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.openembedded.org_g_openembedded-2Dcore_unsub&d=DwIFaQ&c=ncDTmphkJTvjIDPh0hpF_4vCHvabgGkICC2epckfdiw&r=AhkbNonVuMIGRfPx_Qj9TsRih1DULJTKUkSGa66m67E&m=VB1zXAiE9q_CqM9DgoA0BbtloUxpQE_rQ7eqk7HGzBW9KzFu44K9hS1iahsqSuIk&s=c066PNgPnuyQ_O7rM_bVFBC8MVWbYwd5DcSrRCmIcJ4&e=
[[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-

#2 lore raw

this fails in linux-variscrite recipe from meta-variscite-bsp

https://snips.sh/f/9klGZxlCRE

Quoted 60 lines · Etienne Cordonnier via lists.openembedded.org

Patch v4: use idempotent copy of the files instead of simply skipping the code (see discussion under v2 of the patch).

Quoted 42 lines · Etienne Cordonnier via lists.openembedded.org

From: Etienne Cordonnier <[email protected]>

The function do_symlink_kernsrc is not reentrant in the case where S is defined to a non-default value. This causes build-failures e.g. when building linux-yocto, then updating poky to a commit which modifies kernel.bbclass, and then building linux-yocto again.

Bugzilla: https://urldefense.proofpoint.com/v2/url?u=https-3A__bugzilla.yoctoproject.org_show-5Fbug.cgi-3Fid-3D15325&d=DwIDAg&c=ncDTmphkJTvjIDPh0hpF_4vCHvabgGkICC2epckfdiw&r=AhkbNonVuMIGRfPx_Qj9TsRih1DULJTKUkSGa66m67E&m=VB1zXAiE9q_CqM9DgoA0BbtloUxpQE_rQ7eqk7HGzBW9KzFu44K9hS1iahsqSuIk&s=9Cr8vwWgUwwGM5oCO07DB4sO9sQpMCdweS1u8CmZ0Qw&e=

Tested with a recipe "my-custom-linux" which unpacks sources to a custom ${S} directory and ran symlink_kernsrc several times:

$ bitbake -f -c symlink_kernsrc my-custom-linux
---
 meta/classes-recipe/kernel.bbclass | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)
meta/classes-recipe/kernel.bbclass +7 -2
index 9ff37f5c38..42c5519f81 100644
@@ -189,11 +189,17 @@ python do_symlink_kernsrc () {
             # drop trailing slash, so that os.symlink(kernsrc, s) doesn't use s as
             # directory name and fail
             s = s[:-1]
-        if d.getVar("EXTERNALSRC"):
+        if d.getVar("EXTERNALSRC") and not os.path.islink(s):
             # With EXTERNALSRC S will not be wiped so we can symlink to it
             os.symlink(s, kernsrc)
         else:
             import shutil
+            # perform idempotent/reentrant copy
+            s_copy = s + ".orig"
+            if not os.path.isdir(s_copy):
+                shutil.copytree(s, s_copy)
+            bb.utils.remove(s, recurse=True)
+            shutil.copytree(s_copy, s)
             shutil.move(s, kernsrc)
             os.symlink(kernsrc, s)
 }
--
Quoted 5 lines

2.36.1.vfs.0.0

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#192731): https://lists.openembedded.org/g/openembedded-core/message/192731
Mute This Topic: https://lists.openembedded.org/mt/103262114/1997914
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-

#3 lore raw

Is it from https://github.com/varigit/meta-variscite-bsp-ti or
https://github.com/varigit/meta-variscite-bsp-imx ? I did not find a repository called "meta-variscite-bsp".

Quoted 71 lines · Khem Raj

this fails in linux-variscrite recipe from meta-variscite-bsp

https://snips.sh/f/9klGZxlCRE

Quoted 43 lines · Etienne Cordonnier via lists.openembedded.org

Patch v4: use idempotent copy of the files instead of simply skipping the code (see discussion under v2 of the patch).

Quoted 39 lines · Etienne Cordonnier via lists.openembedded.org

From: Etienne Cordonnier <[email protected]>

The function do_symlink_kernsrc is not reentrant in the case where S is defined to a non-default value. This causes build-failures e.g. when building linux-yocto, then updating poky to a commit which modifies kernel.bbclass, and then building linux-yocto again.

Bugzilla: https://urldefense.proofpoint.com/v2/url?u=https-3A__bugzilla.yoctoproject.org_show-5Fbug.cgi-3Fid-3D15325&d=DwIDAg&c=ncDTmphkJTvjIDPh0hpF_4vCHvabgGkICC2epckfdiw&r=AhkbNonVuMIGRfPx_Qj9TsRih1DULJTKUkSGa66m67E&m=VB1zXAiE9q_CqM9DgoA0BbtloUxpQE_rQ7eqk7HGzBW9KzFu44K9hS1iahsqSuIk&s=9Cr8vwWgUwwGM5oCO07DB4sO9sQpMCdweS1u8CmZ0Qw&e=

Tested with a recipe "my-custom-linux" which unpacks sources to a custom ${S} directory and ran symlink_kernsrc several times:

$ bitbake -f -c symlink_kernsrc my-custom-linux
---
 meta/classes-recipe/kernel.bbclass | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)
meta/classes-recipe/kernel.bbclass +7 -1
index 9ff37f5c38..42c5519f81 100644
@@ -189,11 +189,17 @@ python do_symlink_kernsrc () {
             # drop trailing slash, so that os.symlink(kernsrc, s) doesn't use s as
             # directory name and fail
             s = s[:-1]
-        if d.getVar("EXTERNALSRC"):
+        if d.getVar("EXTERNALSRC") and not os.path.islink(s):
             # With EXTERNALSRC S will not be wiped so we can symlink to it
             os.symlink(s, kernsrc)
         else:
             import shutil
+            # perform idempotent/reentrant copy
+            s_copy = s + ".orig"
+            if not os.path.isdir(s_copy):
+                shutil.copytree(s, s_copy)
+            bb.utils.remove(s, recurse=True)
+            shutil.copytree(s_copy, s)
             shutil.move(s, kernsrc)
Quoted 11 lines
Quoted 8 lines
meta/classes-recipe/kernel.bbclass +0 -1
             os.symlink(kernsrc, s)
 }
--

2.36.1.vfs.0.0

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#192838): https://lists.openembedded.org/g/openembedded-core/message/192838
Mute This Topic: https://lists.openembedded.org/mt/103262114/7259375
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-

#4 lore raw

Also which release of yocto are you referring to? Both those repositories don't have a nanbield branch.

Quoted 77 lines · Etienne Cordonnier

Is it from https://github.com/varigit/meta-variscite-bsp-ti or
https://github.com/varigit/meta-variscite-bsp-imx ? I did not find a repository called "meta-variscite-bsp".

Quoted 71 lines · Khem Raj

this fails in linux-variscrite recipe from meta-variscite-bsp

https://snips.sh/f/9klGZxlCRE

Quoted 43 lines · Etienne Cordonnier via lists.openembedded.org

Patch v4: use idempotent copy of the files instead of simply skipping the code (see discussion under v2 of the patch).

Quoted 39 lines · Etienne Cordonnier via lists.openembedded.org

From: Etienne Cordonnier <[email protected]>

The function do_symlink_kernsrc is not reentrant in the case where S is defined to a non-default value. This causes build-failures e.g. when building linux-yocto, then updating poky to a commit which modifies kernel.bbclass, and then building linux-yocto again.

Bugzilla: https://urldefense.proofpoint.com/v2/url?u=https-3A__bugzilla.yoctoproject.org_show-5Fbug.cgi-3Fid-3D15325&d=DwIDAg&c=ncDTmphkJTvjIDPh0hpF_4vCHvabgGkICC2epckfdiw&r=AhkbNonVuMIGRfPx_Qj9TsRih1DULJTKUkSGa66m67E&m=VB1zXAiE9q_CqM9DgoA0BbtloUxpQE_rQ7eqk7HGzBW9KzFu44K9hS1iahsqSuIk&s=9Cr8vwWgUwwGM5oCO07DB4sO9sQpMCdweS1u8CmZ0Qw&e=

Tested with a recipe "my-custom-linux" which unpacks sources to a custom ${S} directory and ran symlink_kernsrc several times:

$ bitbake -f -c symlink_kernsrc my-custom-linux
---
 meta/classes-recipe/kernel.bbclass | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)
meta/classes-recipe/kernel.bbclass +7 -1
index 9ff37f5c38..42c5519f81 100644
@@ -189,11 +189,17 @@ python do_symlink_kernsrc () {
             # drop trailing slash, so that os.symlink(kernsrc, s) doesn't use s as
             # directory name and fail
             s = s[:-1]
-        if d.getVar("EXTERNALSRC"):
+        if d.getVar("EXTERNALSRC") and not os.path.islink(s):
             # With EXTERNALSRC S will not be wiped so we can symlink to it
             os.symlink(s, kernsrc)
         else:
             import shutil
+            # perform idempotent/reentrant copy
+            s_copy = s + ".orig"
+            if not os.path.isdir(s_copy):
+                shutil.copytree(s, s_copy)
+            bb.utils.remove(s, recurse=True)
+            shutil.copytree(s_copy, s)
             shutil.move(s, kernsrc)
Quoted 11 lines
Quoted 8 lines
meta/classes-recipe/kernel.bbclass +0 -1
             os.symlink(kernsrc, s)
 }
--

2.36.1.vfs.0.0

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#192838): https://lists.openembedded.org/g/openembedded-core/message/192838
Mute This Topic: https://lists.openembedded.org/mt/103262114/7259375
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-

#5 lore raw
Quoted 10 lines · Etienne Cordonnier

Also which release of yocto are you referring to? Both those repositories don't have a nanbield branch.

Quoted 5 lines · Etienne Cordonnier

Is it from https://github.com/varigit/meta-variscite-bsp-ti or
https://github.com/varigit/meta-variscite-bsp-imx ? I did not find a repository called "meta-variscite-bsp".

yes it has been renamed upstream
https://github.com/varigit/meta-variscite-bsp-imx/tree/kirkstone-var02/recipes-kernel/linux

use kirkstone-var02 branch

Quoted 72 lines
Quoted 72 lines · Khem Raj
Quoted 71 lines

this fails in linux-variscrite recipe from meta-variscite-bsp

https://snips.sh/f/9klGZxlCRE

Quoted 43 lines · Etienne Cordonnier via lists.openembedded.org

Patch v4: use idempotent copy of the files instead of simply skipping the code (see discussion under v2 of the patch).

Quoted 39 lines · Etienne Cordonnier via lists.openembedded.org

From: Etienne Cordonnier <[email protected]>

The function do_symlink_kernsrc is not reentrant in the case where S is defined to a non-default value. This causes build-failures e.g. when building linux-yocto, then updating poky to a commit which modifies kernel.bbclass, and then building linux-yocto again.

Bugzilla: https://urldefense.proofpoint.com/v2/url?u=https-3A__bugzilla.yoctoproject.org_show-5Fbug.cgi-3Fid-3D15325&d=DwIDAg&c=ncDTmphkJTvjIDPh0hpF_4vCHvabgGkICC2epckfdiw&r=AhkbNonVuMIGRfPx_Qj9TsRih1DULJTKUkSGa66m67E&m=VB1zXAiE9q_CqM9DgoA0BbtloUxpQE_rQ7eqk7HGzBW9KzFu44K9hS1iahsqSuIk&s=9Cr8vwWgUwwGM5oCO07DB4sO9sQpMCdweS1u8CmZ0Qw&e=

Tested with a recipe "my-custom-linux" which unpacks sources to a custom ${S} directory and ran symlink_kernsrc several times:

$ bitbake -f -c symlink_kernsrc my-custom-linux
---
  meta/classes-recipe/kernel.bbclass | 8 +++++++-
  1 file changed, 7 insertions(+), 1 deletion(-)
meta/classes-recipe/kernel.bbclass +7 -1
index 9ff37f5c38..42c5519f81 100644
@@ -189,11 +189,17 @@ python do_symlink_kernsrc () {
              # drop trailing slash, so that os.symlink(kernsrc, s) doesn't use s as
              # directory name and fail
              s = s[:-1]
-        if d.getVar("EXTERNALSRC"):
+        if d.getVar("EXTERNALSRC") and not os.path.islink(s):
              # With EXTERNALSRC S will not be wiped so we can symlink to it
              os.symlink(s, kernsrc)
          else:
              import shutil
+            # perform idempotent/reentrant copy
+            s_copy = s + ".orig"
+            if not os.path.isdir(s_copy):
+                shutil.copytree(s, s_copy)
+            bb.utils.remove(s, recurse=True)
+            shutil.copytree(s_copy, s)
              shutil.move(s, kernsrc)
Quoted 11 lines
Quoted 8 lines
meta/classes-recipe/kernel.bbclass +0 -1
              os.symlink(kernsrc, s)
  }
--

2.36.1.vfs.0.0

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#192838): https://lists.openembedded.org/g/openembedded-core/message/192838
Mute This Topic: https://lists.openembedded.org/mt/103262114/7259375
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-

#6 lore raw

I was able to reproduce. Adding ignore_dangling_symlinks=True was enough to fix the issue (the repository contains dangling symlinks:
https://github.com/varigit/linux-imx/tree/5.15-2.0.x-imx_var01/tools/testing/selftests/drivers/net/dsa
).

Quoted 93 lines · Khem Raj
Quoted 10 lines · Etienne Cordonnier

Also which release of yocto are you referring to? Both those repositories don't have a nanbield branch.

Quoted 5 lines · Etienne Cordonnier

Is it from https://github.com/varigit/meta-variscite-bsp-ti or
https://github.com/varigit/meta-variscite-bsp-imx ? I did not find a repository called "meta-variscite-bsp".

yes it has been renamed upstream
https://github.com/varigit/meta-variscite-bsp-imx/tree/kirkstone-var02/recipes-kernel/linux

use kirkstone-var02 branch

Quoted 58 lines
Quoted 58 lines · Khem Raj
Quoted 57 lines

this fails in linux-variscrite recipe from meta-variscite-bsp

https://snips.sh/f/9klGZxlCRE

Quoted 49 lines · Etienne Cordonnier via lists.openembedded.org

Patch v4: use idempotent copy of the files instead of simply skipping the code (see discussion under v2 of the patch).

Quoted 45 lines · Etienne Cordonnier via lists.openembedded.org

From: Etienne Cordonnier <[email protected]>

The function do_symlink_kernsrc is not reentrant in the case where S is defined to a non-default value. This causes build-failures e.g. when building linux-yocto, then updating poky to a commit which modifies kernel.bbclass, and then building linux-yocto again.

Bugzilla: https://urldefense.proofpoint.com/v2/url?u=https-3A__bugzilla.yoctoproject.org_show-5Fbug.cgi-3Fid-3D15325&d=DwIDAg&c=ncDTmphkJTvjIDPh0hpF_4vCHvabgGkICC2epckfdiw&r=AhkbNonVuMIGRfPx_Qj9TsRih1DULJTKUkSGa66m67E&m=VB1zXAiE9q_CqM9DgoA0BbtloUxpQE_rQ7eqk7HGzBW9KzFu44K9hS1iahsqSuIk&s=9Cr8vwWgUwwGM5oCO07DB4sO9sQpMCdweS1u8CmZ0Qw&e=

Tested with a recipe "my-custom-linux" which unpacks sources to a custom ${S} directory and ran symlink_kernsrc several times:

$ bitbake -f -c symlink_kernsrc my-custom-linux
---
  meta/classes-recipe/kernel.bbclass | 8 +++++++-
  1 file changed, 7 insertions(+), 1 deletion(-)
meta/classes-recipe/kernel.bbclass +7 -2
index 9ff37f5c38..42c5519f81 100644
@@ -189,11 +189,17 @@ python do_symlink_kernsrc () {
              # drop trailing slash, so that os.symlink(kernsrc, s) doesn't use s as
              # directory name and fail
              s = s[:-1]
-        if d.getVar("EXTERNALSRC"):
+        if d.getVar("EXTERNALSRC") and not os.path.islink(s):
              # With EXTERNALSRC S will not be wiped so we can symlink to it
              os.symlink(s, kernsrc)
          else:
              import shutil
+            # perform idempotent/reentrant copy
+            s_copy = s + ".orig"
+            if not os.path.isdir(s_copy):
+                shutil.copytree(s, s_copy)
+            bb.utils.remove(s, recurse=True)
+            shutil.copytree(s_copy, s)
              shutil.move(s, kernsrc)
              os.symlink(kernsrc, s)
  }
--

2.36.1.vfs.0.0

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 5 lines
Quoted 2 lines

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#192848): https://lists.openembedded.org/g/openembedded-core/message/192848
Mute This Topic: https://lists.openembedded.org/mt/103262114/7259375
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-