populate perl-native into its own directory

meta/classes/native.bbclass base_prefix = "${STAGING_DIR_NATIVE}" · line 73

Quoted 4 lines · Dexuan Cui

meta/classes/native.bbclass +4 -0

+PACKAGE_OWN_DIR = ""
+bindir .= "${PACKAGE_OWN_DIR}"
+libdir .= "${PACKAGE_OWN_DIR}"
+libexecdir .= "${PACKAGE_OWN_DIR}"

I think the general idea of this patch is a good one but I'm not terribly fond of the name of that variable. If this is meant to be a path suffix that you can optionally set for native packages then how about calling it something like NATIVE_PACKAGE_PATH_SUFFIX?

Quoted 10 lines · Phil Blundell

meta/classes/native.bbclass base_prefix = "${STAGING_DIR_NATIVE}" · line 73

Quoted 4 lines · Dexuan Cui

meta/classes/native.bbclass +4 -0

+PACKAGE_OWN_DIR = ""
+bindir .= "${PACKAGE_OWN_DIR}"
+libdir .= "${PACKAGE_OWN_DIR}"
+libexecdir .= "${PACKAGE_OWN_DIR}"

I think the general idea of this patch is a good one but I'm not terribly fond of the name of that variable. If this is meant to be a path suffix that you can optionally set for native packages then how about calling it something like NATIVE_PACKAGE_PATH_SUFFIX?

Hi Phil,
Thanks a lot for the suggestion.
Surely NATIVE_PACKAGE_PATH_SUFFIX is a much better naming! :-) I'll change to use it.

Signed-off-by Dexuan Cui <[email protected]>

---
 meta/recipes-devtools/git/git.inc        |    4 ++--
 meta/recipes-devtools/git/git_1.7.5.1.bb |    2 +-
 2 files changed, 3 insertions(+), 3 deletions(-)

meta/recipes-devtools/git/git.inc +0 -0

index c884f9c..3cdd1fa 100644

Quoted 12 lines · Dexuan Cui

Signed-off-by Dexuan Cui <[email protected]>

---
 meta/recipes-devtools/git/git.inc        |    4 ++--
 meta/recipes-devtools/git/git_1.7.5.1.bb |    2 +-
 2 files changed, 3 insertions(+), 3 deletions(-)

(quoted diff) +0 -0

b/meta/recipes-devtools/git/git.inc index c884f9c..3cdd1fa 100644 --- a/meta/recipes-devtools/git/git.inc +++ b/meta/recipes-devtools/git/git.inc @@ -1,14 +1,14 @@
DESCRIPTION = "The git revision control system used by the Linux kernel developers" SECTION = "console/utils" LICENSE = "GPLv2"

meta/recipes-devtools/git/git.inc line 4

Quoted 2 lines

meta/recipes-devtools/git/git.inc +1 -1

-DEPENDS = "perl-native openssl curl zlib expat"
+DEPENDS = "perl-native-runtime openssl curl zlib expat"

This change made git-native fail to build for me until I installed perl-ExtUtils-MakeMaker on my Fedora 15 workstation.

If this is expected we need to be documenting this additional host dependency.

Tracking bug: http://bugzilla.yoctoproject.org/show_bug.cgi?id=1155

Currently both perl-native (a.k.a. ${STAGING_BINDIR_NATIVE}/perl )and perl-native-runtime(a.k.a. the system perl, or /usr/bin/perl) appear in the PATH when bitbake is running.
This can cause some race conditions: many places detecting perl from PATH can't make sure which path will be used as this depends on when perl-native's populate_sysroot is finished, e.g., automake-native and autoconf-native could use perl-native-runtime while gnu-config-native could use perl-native and this inconsistent usages can cause trouble, e.g., bug 941.

And, as RP suggested, "the time to use perl-native instead of perl-native-runtime is where any perl module is being built, perl itself is being built or anything that has an explicit dependency on the perl version present".

So I made the following changes to try to address the aboves issues:
1) populate perl-native into its own directory so it won't appear in PATH by default, and add perlnative.bbclass for these recipes that really depend on perl-native;
2) check all perl-native references and correct the ones that should be perl-native-runtime;
3) fix any building issues due to the new location of perl-native, especially cpan and cpan-base .bbclass.

With the changes, bug 941 doesn't appear.

Tests I did are:
I tried "bitbale core-image-sato-sdk and meta-toolchain-gmae" in x86_32 and x86_64 Ubuntu hosts and everything seems building fine.

Please review the changes and comment on them. Thanks!

Currently both perl-native (a.k.a. ${STAGING_BINDIR_NATIVE}/perl )and perl-native-runtime(a.k.a. the system perl, or /usr/bin/perl) appear in the PATH when bitbake is running.
This can cause some race conditions: many places detecting perl from PATH can't make sure which path will be used as this depends on when perl-native's populate_sysroot is finished, e.g., automake-native and autoconf-native could use perl-native-runtime while gnu-config-native could use perl-native and this inconsistent usages can cause trouble, e.g., bug 941.

And, as RP suggested, "the time to use perl-native instead of perl-native-runtime is where any perl module is being built, perl itself is being built or anything that has an explicit dependency on the perl version present".

So I made the following changes to try to address the aboves issues:
1) populate perl-native into its own directory so it won't appear in PATH by default, and add perlnative.bbclass for these recipes that really depend on perl-native;
2) check all perl-native references and correct the ones that should be perl-native-runtime;
3) fix any building issues due to the new location of perl-native, especially cpan and cpan-base .bbclass.

With the changes, bug 941 doesn't appear.

Tests I did are:
I tried "bitbale core-image-sato-sdk and meta-toolchain-gmae" in x86_32 and x86_64 Ubuntu hosts and everything seems building fine.

Quoted 23 lines · Dexuan Cui

Currently both perl-native (a.k.a. ${STAGING_BINDIR_NATIVE}/perl )and perl-native-runtime(a.k.a. the system perl, or /usr/bin/perl) appear in the PATH when bitbake is running.
This can cause some race conditions: many places detecting perl from PATH can't make sure which path will be used as this depends on when perl-native's populate_sysroot is finished, e.g., automake-native and autoconf-native could use perl-native-runtime while gnu-config-native could use perl-native and this inconsistent usages can cause trouble, e.g., bug 941.

And, as RP suggested, "the time to use perl-native instead of perl-native-runtime is where any perl module is being built, perl itself is being built or anything that has an explicit dependency on the perl version present".

So I made the following changes to try to address the aboves issues:
1) populate perl-native into its own directory so it won't appear in PATH by default, and add perlnative.bbclass for these recipes that really depend on perl-native;
2) check all perl-native references and correct the ones that should be perl-native-runtime;
3) fix any building issues due to the new location of perl-native, especially cpan and cpan-base .bbclass.

Quoted 5 lines

With the changes, bug 941 doesn't appear.

Tests I did are:
I tried "bitbale core-image-sato-sdk and meta-toolchain-gmae" in x86_32 and x86_64 Ubuntu hosts and everything seems building fine.

How does this work (by which I mean, test some please :)) with meta-oe where we have (or will have soon) a lot of perl module recipes? My concern is that we've just moved the race around a bit and we'll hit this somewhere else where we both really need "perl-native" (since we need some cpan stuff we've built) and also mangle in the perl we found in PATH into some scripts...

Quoted 30 lines · Tom Rini

Quoted 23 lines · Dexuan Cui

Currently both perl-native (a.k.a. ${STAGING_BINDIR_NATIVE}/perl )and perl-native-runtime(a.k.a. the system perl, or /usr/bin/perl) appear in the PATH when bitbake is running.
This can cause some race conditions: many places detecting perl from PATH can't make sure which path will be used as this depends on when perl-native's populate_sysroot is finished, e.g., automake-native and autoconf-native could use perl-native-runtime while gnu-config-native could use perl-native and this inconsistent usages can cause trouble, e.g., bug 941.

And, as RP suggested, "the time to use perl-native instead of perl-native-runtime is where any perl module is being built, perl itself is being built or anything that has an explicit dependency on the perl version present".

So I made the following changes to try to address the aboves issues:
1) populate perl-native into its own directory so it won't appear in PATH by default, and add perlnative.bbclass for these recipes that really depend on perl-native;
2) check all perl-native references and correct the ones that should be perl-native-runtime;
3) fix any building issues due to the new location of perl-native, especially cpan and cpan-base .bbclass.

Quoted 5 lines

With the changes, bug 941 doesn't appear.

Tests I did are:
I tried "bitbale core-image-sato-sdk and meta-toolchain-gmae" in x86_32 and x86_64 Ubuntu hosts and everything seems building fine.

Quoted 6 lines

How does this work (by which I mean, test some please :)) with meta-oe where we have (or will have soon) a lot of perl module recipes? My concern is that we've just moved the race around a bit and we'll hit this somewhere else where we both really need "perl-native" (since we need some cpan stuff we've built) and also mangle in the perl we found in PATH into some scripts...

Anything building or using perl modules should be inheriting the perlnative class. This adds the dependency on perl-native and the appropriate PATH entry. Where is the race?

Maybe race isn't quite the right word. But recipe A depends on lib$something-perl-native, and brings in perl-native. It also checks for perl in its auto-foo and finds our perl. It now also uses our perl when it wants a host perl and all of the potential bad things happen, yes?

Quoted 4 lines · Tom Rini

Maybe race isn't quite the right word. But recipe A depends on lib$something-perl-native, and brings in perl-native. It also checks for perl in its auto-foo and finds our perl. It now also uses our perl when it wants a host perl and all of the potential bad things happen, yes?

What are the circumstances in which it would want a host perl? I don't quite understand the issue here. Surely anything that the host perl can do, perl-native plus some combination of libfoo-perl-native can also do, right?

What falls down in this case is that once
perl-native is built (and in our PATH), if it's a different version than system-wide perl, stuff starts failing on version mis-match.

Quoted 3 lines · Tom Rini

What falls down in this case is that once
perl-native is built (and in our PATH), if it's a different version than system-wide perl, stuff starts failing on version mis-match.

I think that's the bit that I'm not properly understanding. Which versions are mismatching, exactly?

Surely the local perl from the sysroot ought to be completely self-contained and shouldn't be using any bits from the host perl install at that point.

Quoted 11 lines · Phil Blundell

Quoted 3 lines · Tom Rini

What falls down in this case is that once
perl-native is built (and in our PATH), if it's a different version than system-wide perl, stuff starts failing on version mis-match.

I think that's the bit that I'm not properly understanding. Which versions are mismatching, exactly?

Surely the local perl from the sysroot ought to be completely self-contained and shouldn't be using any bits from the host perl install at that point.

So this jogs my memory a bit! It's not so much perl itself but stuff that uses perl that can get dirty and then no, you have stuff thats built for system perl and stuff that's built with perl-native clashing.

Relying even more on memory, I think help2man was one of the "easy" culprits and since we also modify the env, we do things like have help2man run with PERL5LIB and so on pointing system-wide perl at perl-native's lib directory and so forth.

Quoted 13 lines · Tom Rini

Quoted 11 lines · Phil Blundell

Quoted 3 lines · Tom Rini

What falls down in this case is that once
perl-native is built (and in our PATH), if it's a different version than system-wide perl, stuff starts failing on version mis-match.

I think that's the bit that I'm not properly understanding. Which versions are mismatching, exactly?

Surely the local perl from the sysroot ought to be completely self-contained and shouldn't be using any bits from the host perl install at that point.

Quoted 8 lines

So this jogs my memory a bit! It's not so much perl itself but stuff that uses perl that can get dirty and then no, you have stuff thats built for system perl and stuff that's built with perl-native clashing.

Relying even more on memory, I think help2man was one of the "easy" culprits and since we also modify the env, we do things like have help2man run with PERL5LIB and so on pointing system-wide perl at perl-native's lib directory and so forth.

But with the proposed patch series either:

a) help2man depends perlnative.bbclass

In this case it can depend on perl-native being there, its in path and things work as per OE.dev.

b) help2man doesn't depend in perlnative.bbclass

It only sees the system perl.

So I'm still not clear where the problem is?

But help2man is just the easy/common case. Heck, it _may_ blow up even with the host help2man instead of help2man-native, if a recipe uses system-wide help2man and perlnative.bbclass. The root problem (again, from memory) is that since we modify PERL5LIB and so on, when we do that, we've opened ourselves up for system-wide perl trying to use perl-native's stuff.

Quoted 6 lines · Tom Rini

But help2man is just the easy/common case. Heck, it _may_ blow up even with the host help2man instead of help2man-native, if a recipe uses system-wide help2man and perlnative.bbclass. The root problem (again, from memory) is that since we modify PERL5LIB and so on, when we do that, we've opened ourselves up for system-wide perl trying to use perl-native's stuff.

Ah right, I think I see what you're getting at now.

If we've got a clean separation between perl-native and host perl, though, can't we now just eliminate all of that futzing with PERL5LIB in cpan.bbclass and such like places? perl-native already knows how to look in the right places within the sysroot for its modules so there should be no need for anything else to be overriding it.

Quoted 10 lines · Phil Blundell

Quoted 6 lines · Tom Rini

But help2man is just the easy/common case. Heck, it _may_ blow up even with the host help2man instead of help2man-native, if a recipe uses system-wide help2man and perlnative.bbclass. The root problem (again, from memory) is that since we modify PERL5LIB and so on, when we do that, we've opened ourselves up for system-wide perl trying to use perl-native's stuff.

Ah right, I think I see what you're getting at now.

Quoted 5 lines

If we've got a clean separation between perl-native and host perl, though, can't we now just eliminate all of that futzing with PERL5LIB in cpan.bbclass and such like places? perl-native already knows how to look in the right places within the sysroot for its modules so there should be no need for anything else to be overriding it.

Well, my question is, does it, really?

Even if we're using the sstate
cache from /foo/oecore/tmp over in /bar/oecore/tmp (and /foo/oecore/tmp is rm -rf'd) ? Since we've got a create_wrapper around perl and perl${PV} it should be I suppose (or is easily added there), but I'd feel a lot better with some testing of the above case (and the updates to cpan*bbclass).

Quoted 17 lines · Tom Rini

Quoted 15 lines · Phil Blundell

Quoted 6 lines · Tom Rini

But help2man is just the easy/common case. Heck, it _may_ blow up even with the host help2man instead of help2man-native, if a recipe uses system-wide help2man and perlnative.bbclass. The root problem (again, from memory) is that since we modify PERL5LIB and so on, when we do that, we've opened ourselves up for system-wide perl trying to use perl-native's stuff.

Ah right, I think I see what you're getting at now.

If we've got a clean separation between perl-native and host perl, though, can't we now just eliminate all of that futzing with PERL5LIB in cpan.bbclass and such like places? perl-native already knows how to look in the right places within the sysroot for its modules so there should be no need for anything else to be overriding it.

Quoted 1 line

Well, my question is, does it, really?

If it doesn't it really does need to get fixed. I've not seen any evidence this has been the problem but it still could be.

Quoted 6 lines

Even if we're using the sstate
cache from /foo/oecore/tmp over in /bar/oecore/tmp (and /foo/oecore/tmp is rm -rf'd) ? Since we've got a create_wrapper around perl and perl${PV} it should be I suppose (or is easily added there), but I'd feel a lot better with some testing of the above case (and the updates to cpan*bbclass).

I only took the perl-native DEPENDS patch on the condition this gets fixed properly. The patches that are there look to do that, at least for OE-Core. If there are further issues we're going to have to take them as they arise as I have an objection to crippling the build dependencies because perl is broken. Really this could use some TLC from someone with experience in the area...

Quoted 8 lines · Richard Purdie

Quoted 6 lines · Tom Rini

Even if we're using the sstate
cache from /foo/oecore/tmp over in /bar/oecore/tmp (and /foo/oecore/tmp is rm -rf'd) ? Since we've got a create_wrapper around perl and perl${PV} it should be I suppose (or is easily added there), but I'd feel a lot better with some testing of the above case (and the updates to cpan*bbclass).

Quoted 6 lines

I only took the perl-native DEPENDS patch on the condition this gets fixed properly. The patches that are there look to do that, at least for OE-Core. If there are further issues we're going to have to take them as they arise as I have an objection to crippling the build dependencies because perl is broken. Really this could use some TLC from someone with experience in the area...

Well, I guess I'd boil down what I said above into a request like this for v3:
- Modify cpan.bbclass to NOT set PERL_INC / PERL_LIB / PERL_ARCHLIB / PERLHOSTLIB.

- In /scratch/oecore/tmp0 build the images that were built for v1
- In /scratch/oecore/tmp1 build perl-native's full sstate cache.
- Keep the sstate cache from tmp1 and otherwise rm -rf tmp1.
- In /scratch/oecore/tmp2 using the sstate from tmp1, build the same images that were built for tmp0.

Quoted 16 lines · Tom Rini

Quoted 8 lines · Richard Purdie

Quoted 6 lines · Tom Rini

Even if we're using the sstate
cache from /foo/oecore/tmp over in /bar/oecore/tmp (and /foo/oecore/tmp is rm -rf'd) ? Since we've got a create_wrapper around perl and perl${PV} it should be I suppose (or is easily added there), but I'd feel a lot better with some testing of the above case (and the updates to cpan*bbclass).

Quoted 6 lines

I only took the perl-native DEPENDS patch on the condition this gets fixed properly. The patches that are there look to do that, at least for OE-Core. If there are further issues we're going to have to take them as they arise as I have an objection to crippling the build dependencies because perl is broken. Really this could use some TLC from someone with experience in the area...

Quoted 4 lines

Well, I guess I'd boil down what I said above into a request like this for v3:
- Modify cpan.bbclass to NOT set PERL_INC / PERL_LIB / PERL_ARCHLIB / PERLHOSTLIB.

The first three of these are all about the *target* perl location and I think we still need them due the mess that perl's build system is. With the patch series in question they won't actually point at perl-native in the target case and they are only really used for cross compiling purposes.

PERLHOSTLIB is used by the target perl when cross compiling to find native .so files. perl-native will always be present at this point and again, it seems like a valid use case.

Summary is that I don't think perl-native is broken in any way but we do need those variables.

Quoted 5 lines

- In /scratch/oecore/tmp0 build the images that were built for v1
- In /scratch/oecore/tmp1 build perl-native's full sstate cache.
- Keep the sstate cache from tmp1 and otherwise rm -rf tmp1.
- In /scratch/oecore/tmp2 using the sstate from tmp1, build the same images that were built for tmp0.

I'm confused by this test cycle. What do you mean by "build perl-native's full sstate cache"?

I suspect you've asking for some partial sstate cache to be shared between two builds?

Put simpler, you probably want:

in tmp0 "bitbake perl-native"
in tmp1, different location to tmp0, "bitbake core-image-sato" but sharing the same sstate cache

Is that what you're thinking?

FWIW, I've been running locally with the patch series and I think I'd like to merge it. If there are issues we can address them as and when they're identified...

Cheers,

Richard

_______________________________________________
Openembedded-core mailing list
[email protected]
http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core

Quoted 48 lines · Richard Purdie

Quoted 16 lines · Tom Rini

Quoted 8 lines · Richard Purdie

Quoted 6 lines · Tom Rini

Even if we're using the sstate
cache from /foo/oecore/tmp over in /bar/oecore/tmp (and /foo/oecore/tmp is rm -rf'd) ? Since we've got a create_wrapper around perl and perl${PV} it should be I suppose (or is easily added there), but I'd feel a lot better with some testing of the above case (and the updates to cpan*bbclass).

Quoted 6 lines

I only took the perl-native DEPENDS patch on the condition this gets fixed properly. The patches that are there look to do that, at least for OE-Core. If there are further issues we're going to have to take them as they arise as I have an objection to crippling the build dependencies because perl is broken. Really this could use some TLC from someone with experience in the area...

Quoted 3 lines

Well, I guess I'd boil down what I said above into a request like this for v3: - Modify cpan.bbclass to NOT set PERL_INC / PERL_LIB / PERL_ARCHLIB / PERLHOSTLIB.

The first three of these are all about the *target* perl location and I think we still need them due the mess that perl's build system is. With the patch series in question they won't actually point at perl-native in the target case and they are only really used for cross compiling purposes.

PERLHOSTLIB is used by the target perl when cross compiling to find native .so files. perl-native will always be present at this point and again, it seems like a valid use case.

Summary is that I don't think perl-native is broken in any way but we do need those variables.

Quoted 5 lines

- In /scratch/oecore/tmp0 build the images that were built for v1
- In /scratch/oecore/tmp1 build perl-native's full sstate cache.
- Keep the sstate cache from tmp1 and otherwise rm -rf tmp1.
- In /scratch/oecore/tmp2 using the sstate from tmp1, build the same images that were built for tmp0.

I'm confused by this test cycle. What do you mean by "build perl-native's full sstate cache"?

I suspect you've asking for some partial sstate cache to be shared between two builds?

Put simpler, you probably want:

Quoted 3 lines

in tmp0 "bitbake perl-native"
in tmp1, different location to tmp0, "bitbake core-image-sato" but sharing the same sstate cache

I meant to add that tmp0 should be renamed before this second step so if there are hardcoded references in any of the sstate packages they couldn't find anything in tmp0 as it would no longer exist.

Quoted 6 lines · Richard Purdie

Quoted 5 lines · Richard Purdie

I suspect you've asking for some partial sstate cache to be shared between two builds?

Put simpler, you probably want:

Quoted 7 lines

Quoted 3 lines

in tmp0 "bitbake perl-native"
in tmp1, different location to tmp0, "bitbake core-image-sato" but sharing the same sstate cache

I meant to add that tmp0 should be renamed before this second step so if there are hardcoded references in any of the sstate packages they couldn't find anything in tmp0 as it would no longer exist.

Actually this doesn't work even without my patch series?
i.e., without my patch series,
1) I "bitbake perl-native" in /poky.git/build/;
2) mv /poky.git/build /poky.git/build.bak;
3) in /poky.git/build2/, modify the conf/local.conf to set SSTATE_MIRRORS to point to /poky.git/build.bak/sstate-cache/, and "bitbake sgmlspl-native" would fail in do_compile:
"Can't locate ExtUtils/Command.pm in @INC (@INC contains: /poky.git/build/tmp/sysroots/i686-linux/usr/lib/perl/5.12.3 /poky.git/build/tmp/sysroots/i686-linux/usr/lib/perl/5.12.3 /poky.git/build/tmp/sysroots/i686-"linux/usr/lib/perl/5.12.3" Is this an existing bug?

Quoted 15 lines · Dexuan

Quoted 6 lines · Richard Purdie

Quoted 5 lines · Richard Purdie

I suspect you've asking for some partial sstate cache to be shared between two builds?

Put simpler, you probably want:

Quoted 7 lines

Quoted 3 lines

in tmp0 "bitbake perl-native"
in tmp1, different location to tmp0, "bitbake core-image-sato" but sharing the same sstate cache

I meant to add that tmp0 should be renamed before this second step so if there are hardcoded references in any of the sstate packages they couldn't find anything in tmp0 as it would no longer exist.

Quoted 12 lines

Actually this doesn't work even without my patch series? i.e., without my patch series,
1) I "bitbake perl-native" in /poky.git/build/;
2) mv /poky.git/build /poky.git/build.bak;
3) in /poky.git/build2/, modify the conf/local.conf to set SSTATE_MIRRORS to point to /poky.git/build.bak/sstate-cache/, and "bitbake sgmlspl-native" would fail in do_compile: "Can't locate ExtUtils/Command.pm in @INC (@INC contains:
/poky.git/build/tmp/sysroots/i686-linux/usr/lib/perl/5.12.3 /poky.git/build/tmp/sysroots/i686-linux/usr/lib/perl/5.12.3 /poky.git/build/tmp/sysroots/i686-"linux/usr/lib/perl/5.12.3" Is this an existing bug?

Looks like it. This did work but it looks like its been broken somehow :(

Quoted 49 lines · Richard Purdie

Quoted 16 lines · Tom Rini

Quoted 8 lines · Richard Purdie

Quoted 6 lines · Tom Rini

Even if we're using the sstate
cache from /foo/oecore/tmp over in /bar/oecore/tmp (and /foo/oecore/tmp is rm -rf'd) ? Since we've got a create_wrapper around perl and perl${PV} it should be I suppose (or is easily added there), but I'd feel a lot better with some testing of the above case (and the updates to cpan*bbclass).

Quoted 6 lines

I only took the perl-native DEPENDS patch on the condition this gets fixed properly. The patches that are there look to do that, at least for OE-Core. If there are further issues we're going to have to take them as they arise as I have an objection to crippling the build dependencies because perl is broken. Really this could use some TLC from someone with experience in the area...

Quoted 4 lines

Well, I guess I'd boil down what I said above into a request like this for v3:
- Modify cpan.bbclass to NOT set PERL_INC / PERL_LIB / PERL_ARCHLIB / PERLHOSTLIB.

The first three of these are all about the *target* perl location and I think we still need them due the mess that perl's build system is. With the patch series in question they won't actually point at perl-native in the target case and they are only really used for cross compiling purposes.

PERLHOSTLIB is used by the target perl when cross compiling to find native .so files. perl-native will always be present at this point and again, it seems like a valid use case.

Summary is that I don't think perl-native is broken in any way but we do need those variables.

Quoted 5 lines

- In /scratch/oecore/tmp0 build the images that were built for v1
- In /scratch/oecore/tmp1 build perl-native's full sstate cache.
- Keep the sstate cache from tmp1 and otherwise rm -rf tmp1.
- In /scratch/oecore/tmp2 using the sstate from tmp1, build the same images that were built for tmp0.

I'm confused by this test cycle. What do you mean by "build perl-native's full sstate cache"?

I suspect you've asking for some partial sstate cache to be shared between two builds?

Put simpler, you probably want:

Quoted 3 lines

in tmp0 "bitbake perl-native"
in tmp1, different location to tmp0, "bitbake core-image-sato" but sharing the same sstate cache

I meant to add that tmp0 should be renamed before this second step so if there are hardcoded references in any of the sstate packages they couldn't find anything in tmp0 as it would no longer exist.

Quoted 16 lines · Tom Rini

Quoted 8 lines · Richard Purdie

Quoted 6 lines · Tom Rini

Even if we're using the sstate
cache from /foo/oecore/tmp over in /bar/oecore/tmp (and /foo/oecore/tmp is rm -rf'd) ? Since we've got a create_wrapper around perl and perl${PV} it should be I suppose (or is easily added there), but I'd feel a lot better with some testing of the above case (and the updates to cpan*bbclass).

Quoted 6 lines

I only took the perl-native DEPENDS patch on the condition this gets fixed properly. The patches that are there look to do that, at least for OE-Core. If there are further issues we're going to have to take them as they arise as I have an objection to crippling the build dependencies because perl is broken. Really this could use some TLC from someone with experience in the area...

Quoted 4 lines

Well, I guess I'd boil down what I said above into a request like this for v3:
- Modify cpan.bbclass to NOT set PERL_INC / PERL_LIB / PERL_ARCHLIB / PERLHOSTLIB.

The first three of these are all about the *target* perl location and I think we still need them due the mess that perl's build system is. With the patch series in question they won't actually point at perl-native in the target case and they are only really used for cross compiling purposes.

PERLHOSTLIB is used by the target perl when cross compiling to find native .so files. perl-native will always be present at this point and again, it seems like a valid use case.

Summary is that I don't think perl-native is broken in any way but we do need those variables.

Quoted 8 lines · Phil Blundell

Quoted 6 lines · Tom Rini

Maybe race isn't quite the right word. But recipe A depends on lib$something-perl-native, and brings in perl-native. It also checks
for perl in its auto-foo and finds our perl. It now also uses our perl when it wants a host perl and all of the potential bad things happen, yes?

Quoted 4 lines

What are the circumstances in which it would want a host perl? I don't quite understand the issue here. Surely anything that the host perl can do, perl-native plus some combination of libfoo-perl-native can also do, right?

So, here's the two things going on:
- In some cases (libfoo-perl-native) we need to install various perl modules in order to build other target apps. This is the cpan case, iirc. We don't currently (and can't?) just play whatever games perl would like to do to use system-wide perl and a local to TMPDIR cpan directory. We instead go down the path of having perl-native.
- In order to build perl for the target we need to have the same perl version available for use.

What we do in oe-core (and oe.dev, historically) is provide perl-native
for both of these cases. What falls down in this case is that once perl-native is built (and in our PATH), if it's a different version than system-wide perl, stuff starts failing on version mis-match. The patch series here fixes that by saying stuff must opt-in to having perl-native
be in PATH rather than just system-wide perl. What I'm saying is that while in oe-core nothing falls down here (since nothing that gets perl-native also tries to just run 'perl') meta-oe is or will have failures.

Unfortunately I don't have the test cases that drove me to make oe.dev like it is handy but I have a feeling that if we go down this path it won't be the last we see of the problem, but it might well pop up again.

There is another option here, which is to make perl-native be perl-cross, live in its own special directory and figure out how to make
a TMPDIR-specific cpan repository available for use by system-wide perl.