perl 5.22 and 32 bit targets

I hope this is the correct place to discuss this problem. It is all about a difference in behavior between a program built using bitbake/OE (only OE-core is needed) vs building the program on the target hardware itself.

I've been struggling with this problem since perl was upgraded to version 5.22. I'm working on Amanda (Advanced Maryland Archive tool) which is written primarily in perl and uses swig interfaces to access native C functions. This code works great when using the previous perl (5.20.x) but fails on all 32 bit targets with perl 5.22

The interesting thing is that if I build Amanda on my target directly (using SDK tools), it works perfectly even with perl 5.22, so it seems that there is some [subtle] difference between building using bitbake/OE than when built on the self-hosted target. I've compared the builds and the only thing I could find (from the output of configure) is a difference in sizeof(off_t) Sadly, when I tried to adjust this in the OE build, it didn't make any difference, but perhaps I didn't make this change correctly or completely.

Anyway, I'm looking for some help to solve this. I've put all the relevant pieces and notes about the process at:
https://github.com/GaryThomas/meta-amanda.git

--
------------------------------------------------------------
Gary Thomas | Consulting for the
MLB Associates | Embedded world
------------------------------------------------------------
--
_______________________________________________
yocto mailing list
[email protected]
https://lists.yoctoproject.org/listinfo/yocto

Quoted 12 lines · Gary Thomas

I hope this is the correct place to discuss this problem. It is all about a difference in behavior between a program built using bitbake/OE (only OE-core is needed) vs building the program on the target hardware itself.

I've been struggling with this problem since perl was upgraded to version 5.22. I'm working on Amanda (Advanced Maryland Archive tool) which is written primarily in perl and uses swig interfaces to access native C functions. This code works great when using the previous perl (5.20.x) but fails on all 32 bit targets with perl 5.22

Quoted 9 lines

The interesting thing is that if I build Amanda on my target directly (using SDK tools), it works perfectly even with perl 5.22, so it seems that there is some [subtle] difference between building using bitbake/OE than when built on the self-hosted target. I've compared the builds and the only thing I could find (from the output of configure) is a difference in sizeof(off_t) Sadly, when I tried to adjust this in the OE build, it didn't make any difference, but perhaps I didn't make this change correctly or completely.

do you have largefile support turned on ? if you do then it might be detecting it wrongly during configure since we cache it to a non-largefile case

so try to add something like

EXTRA_OECONF += "${@bb.utils.contains('DISTRO_FEATURES', 'largefile', 'ac_cv_sizeof_off_t=8', '', d)}"

while building perl or the affected program and see if that helps

Quoted 4 lines

Anyway, I'm looking for some help to solve this. I've put all the relevant pieces and notes about the process at:
https://github.com/GaryThomas/meta-amanda.git

Quoted 32 lines · Khem Raj

Quoted 12 lines · Gary Thomas

I hope this is the correct place to discuss this problem. It is all about a difference in behavior between a program built using bitbake/OE (only OE-core is needed) vs building the program on the target hardware itself.

I've been struggling with this problem since perl was upgraded to version 5.22. I'm working on Amanda (Advanced Maryland Archive tool) which is written primarily in perl and uses swig interfaces to access native C functions. This code works great when using the previous perl (5.20.x) but fails on all 32 bit targets with perl 5.22

Quoted 9 lines

The interesting thing is that if I build Amanda on my target directly (using SDK tools), it works perfectly even with perl 5.22, so it seems that there is some [subtle] difference between building using bitbake/OE than when built on the self-hosted target. I've compared the builds and the only thing I could find (from the output of configure) is a difference in sizeof(off_t) Sadly, when I tried to adjust this in the OE build, it didn't make any difference, but perhaps I didn't make this change correctly or completely.

do you have largefile support turned on ? if you do then it might be detecting it wrongly during configure since we cache it to a non-largefile case

so try to add something like

EXTRA_OECONF += "${@bb.utils.contains('DISTRO_FEATURES', 'largefile', 'ac_cv_sizeof_off_t=8', '', d)}"

Quoted 1 line

while building perl or the affected program and see if that helps

Thanks for the idea, but that didn't help. I also forced some CFLAGS to match, in particular:
-D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64
but this didn't make any difference either.

Quoted 4 lines

Quoted 4 lines

Anyway, I'm looking for some help to solve this. I've put all the relevant pieces and notes about the process at:
https://github.com/GaryThomas/meta-amanda.git

Am 23.03.2016 um 09:40 schrieb Gary Thomas <[email protected]>:

Quoted 33 lines · Gary Thomas

Quoted 32 lines · Khem Raj

Quoted 12 lines · Gary Thomas

I hope this is the correct place to discuss this problem. It is all about a difference in behavior between a program built using bitbake/OE (only OE-core is needed) vs building the program on the target hardware itself.

I've been struggling with this problem since perl was upgraded to version 5.22. I'm working on Amanda (Advanced Maryland Archive tool) which is written primarily in perl and uses swig interfaces to access native C functions. This code works great when using the previous perl (5.20.x) but fails on all 32 bit targets with perl 5.22

Quoted 9 lines

The interesting thing is that if I build Amanda on my target directly (using SDK tools), it works perfectly even with perl 5.22, so it seems that there is some [subtle] difference between building using bitbake/OE than when built on the self-hosted target. I've compared the builds and the only thing I could find (from the output of configure) is a difference in sizeof(off_t) Sadly, when I tried to adjust this in the OE build, it didn't make any difference, but perhaps I didn't make this change correctly or completely.

do you have largefile support turned on ? if you do then it might be detecting it wrongly during configure since we cache it to a non-largefile case

so try to add something like

EXTRA_OECONF += "${@bb.utils.contains('DISTRO_FEATURES', 'largefile', 'ac_cv_sizeof_off_t=8', '', d)}"

Quoted 7 lines

Quoted 1 line

while building perl or the affected program and see if that helps

Thanks for the idea, but that didn't help. I also forced some CFLAGS to match, in particular:
-D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64
but this didn't make any difference either.

On a whim I just tried a little experiment where I took the *.o files from the perl subdirectory (where all the swig shims live) from a working (self-hosted) build and moved them to my bitbake/OE build. I then touched all the *.o and *.lo files in the perl tree to force a relink. I then ran
% bitbake amanda -C compile && bitbake core-image-base
to my surprise, amanda works! So the culprit lies somewhere within the swig generated glue. I've tried comparing these files before and I didn't find anything other than cosmetic differences (mostly comments about the name of the file processed, etc). I've added this subtree to "results" in my github layer in case someone can see what might be relevant.

Any ideas what might be different and make this swig generated glue fail? Note that the swig interface files are rebuilt as part of the build process and both bitbake/OE and self-hosted are using the same swig version.

Quoted 56 lines

Am 23.03.2016 um 09:40 schrieb Gary Thomas <[email protected]>:

Quoted 33 lines · Gary Thomas

Quoted 32 lines · Khem Raj

Quoted 12 lines · Gary Thomas

I hope this is the correct place to discuss this problem. It is all about a difference in behavior between a program built using bitbake/OE (only OE-core is needed) vs building the program on the target hardware itself.

I've been struggling with this problem since perl was upgraded to version 5.22. I'm working on Amanda (Advanced Maryland Archive tool) which is written primarily in perl and uses swig interfaces to access native C functions. This code works great when using the previous perl (5.20.x) but fails on all 32 bit targets with perl 5.22

Quoted 9 lines

The interesting thing is that if I build Amanda on my target directly (using SDK tools), it works perfectly even with perl 5.22, so it seems that there is some [subtle] difference between building using bitbake/OE than when built on the self-hosted target. I've compared the builds and the only thing I could find (from the output of configure) is a difference in sizeof(off_t) Sadly, when I tried to adjust this in the OE build, it didn't make any difference, but perhaps I didn't make this change correctly or completely.

do you have largefile support turned on ? if you do then it might be detecting it wrongly during configure since we cache it to a non-largefile case

so try to add something like

EXTRA_OECONF += "${@bb.utils.contains('DISTRO_FEATURES', 'largefile', 'ac_cv_sizeof_off_t=8', '', d)}"

Quoted 7 lines

Quoted 1 line

while building perl or the affected program and see if that helps

Thanks for the idea, but that didn't help. I also forced some CFLAGS to match, in particular:
-D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64
but this didn't make any difference either.

On a whim I just tried a little experiment where I took the *.o files from the perl subdirectory (where all the swig shims live) from a working (self-hosted) build and moved them to my bitbake/OE build. I then touched all the *.o and *.lo files in the perl tree to force a relink. I then ran
% bitbake amanda -C compile && bitbake core-image-base
to my surprise, amanda works! So the culprit lies somewhere within the swig generated glue. I've tried comparing these files before and I didn't find anything other than cosmetic differences (mostly comments about the name of the file processed, etc). I've added this subtree to "results" in my github layer in case someone can see what might be relevant.

Quoted 3 lines

Any ideas what might be different and make this swig generated glue fail? Note that the swig interface files are rebuilt as part of the build process and both bitbake/OE and self-hosted are using the same swig version.

I digged a bit through your layer (while my up2date scanner over meta-cpan blocks my build chain :P) and realized that you use perl-5.20.0 as it was in poky. A "simple" downgrade would be more reasonable ... if reason applys here in general :)

When you fail on cross-build and succeed in target build, try to compare the C files and includes (even swig libraries) used.

It smells more like a "wrong source" than a "perl problem" (and even when I never would read any python thread, the same problem would likely occur there, too ^^).

Which perl headers are used in your build? To dig down, more logs would be reasonable ...

Am 23.03.2016 um 10:09 schrieb Gary Thomas <[email protected]>:

Quoted 60 lines · Jens Rehsack

Quoted 59 lines

Am 23.03.2016 um 09:40 schrieb Gary Thomas <[email protected]>:

Quoted 33 lines · Gary Thomas

Quoted 32 lines · Khem Raj

Quoted 12 lines · Gary Thomas

I hope this is the correct place to discuss this problem. It is all about a difference in behavior between a program built using bitbake/OE (only OE-core is needed) vs building the program on the target hardware itself.

I've been struggling with this problem since perl was upgraded to version 5.22. I'm working on Amanda (Advanced Maryland Archive tool) which is written primarily in perl and uses swig interfaces to access native C functions. This code works great when using the previous perl (5.20.x) but fails on all 32 bit targets with perl 5.22

Quoted 9 lines

The interesting thing is that if I build Amanda on my target directly (using SDK tools), it works perfectly even with perl 5.22, so it seems that there is some [subtle] difference between building using bitbake/OE than when built on the self-hosted target. I've compared the builds and the only thing I could find (from the output of configure) is a difference in sizeof(off_t) Sadly, when I tried to adjust this in the OE build, it didn't make any difference, but perhaps I didn't make this change correctly or completely.

do you have largefile support turned on ? if you do then it might be detecting it wrongly during configure since we cache it to a non-largefile case

so try to add something like

EXTRA_OECONF += "${@bb.utils.contains('DISTRO_FEATURES', 'largefile', 'ac_cv_sizeof_off_t=8', '', d)}"

Quoted 7 lines

Quoted 1 line

while building perl or the affected program and see if that helps

Thanks for the idea, but that didn't help. I also forced some CFLAGS to match, in particular:
-D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64
but this didn't make any difference either.

On a whim I just tried a little experiment where I took the *.o files from the perl subdirectory (where all the swig shims live) from a working (self-hosted) build and moved them to my bitbake/OE build. I then touched all the *.o and *.lo files in the perl tree to force a relink. I then ran
% bitbake amanda -C compile && bitbake core-image-base
to my surprise, amanda works! So the culprit lies somewhere within the swig generated glue. I've tried comparing these files before and I didn't find anything other than cosmetic differences (mostly comments about the name of the file processed, etc). I've added this subtree to "results" in my github layer in case someone can see what might be relevant.

Any ideas what might be different and make this swig generated glue fail? Note that the swig interface files are rebuilt as part of the build process and both bitbake/OE and self-hosted are using the same swig version.

Quoted 4 lines

I digged a bit through your layer (while my up2date scanner over meta-cpan blocks my build chain :P) and realized that you use perl-5.20.0 as it was in poky. A "simple" downgrade would be more reasonable ... if reason applys here in general :)

In practice, I am doing that. However, I want to understand why perl 5.22 breaks things and get it fixed.

Quoted 7 lines

When you fail on cross-build and succeed in target build, try to compare the C files and includes (even swig libraries) used.

It smells more like a "wrong source" than a "perl problem" (and even when I never would read any python thread, the same problem would likely occur there, too ^^).

Quoted 2 lines

Which perl headers are used in your build? To dig down, more logs would be reasonable ...

Everything comes from the same sources, same revisions, etc, as I'm using either a bitbake/OE build or the embedded (self-hosted) version from the same build plus SDK tools.

Am 23.03.2016 um 10:14 schrieb Jens Rehsack <[email protected]>:

Quoted 64 lines

Am 23.03.2016 um 10:09 schrieb Gary Thomas <[email protected]>:

Quoted 60 lines · Jens Rehsack

Quoted 59 lines

Am 23.03.2016 um 09:40 schrieb Gary Thomas <[email protected]>:

Quoted 33 lines · Gary Thomas

Quoted 32 lines · Khem Raj

Quoted 12 lines · Gary Thomas

I hope this is the correct place to discuss this problem. It is all about a difference in behavior between a program built using bitbake/OE (only OE-core is needed) vs building the program on the target hardware itself.

I've been struggling with this problem since perl was upgraded to version 5.22. I'm working on Amanda (Advanced Maryland Archive tool) which is written primarily in perl and uses swig interfaces to access native C functions. This code works great when using the previous perl (5.20.x) but fails on all 32 bit targets with perl 5.22

Quoted 9 lines

The interesting thing is that if I build Amanda on my target directly (using SDK tools), it works perfectly even with perl 5.22, so it seems that there is some [subtle] difference between building using bitbake/OE than when built on the self-hosted target. I've compared the builds and the only thing I could find (from the output of configure) is a difference in sizeof(off_t) Sadly, when I tried to adjust this in the OE build, it didn't make any difference, but perhaps I didn't make this change correctly or completely.

do you have largefile support turned on ? if you do then it might be detecting it wrongly during configure since we cache it to a non-largefile case

so try to add something like

EXTRA_OECONF += "${@bb.utils.contains('DISTRO_FEATURES', 'largefile', 'ac_cv_sizeof_off_t=8', '', d)}"

Quoted 7 lines

Quoted 1 line

while building perl or the affected program and see if that helps

Thanks for the idea, but that didn't help. I also forced some CFLAGS to match, in particular:
-D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64
but this didn't make any difference either.

On a whim I just tried a little experiment where I took the *.o files from the perl subdirectory (where all the swig shims live) from a working (self-hosted) build and moved them to my bitbake/OE build. I then touched all the *.o and *.lo files in the perl tree to force a relink. I then ran % bitbake amanda -C compile && bitbake core-image-base
to my surprise, amanda works! So the culprit lies somewhere within the swig generated glue. I've tried comparing these files before and I didn't find anything other than cosmetic differences (mostly comments about the name of the file processed, etc). I've added this subtree to "results" in my github layer in case someone can see what might be relevant.

Any ideas what might be different and make this swig generated glue fail? Note that the swig interface files are rebuilt as part of the build process and both bitbake/OE and self-hosted are using the same swig version.

Quoted 7 lines

Quoted 4 lines

I digged a bit through your layer (while my up2date scanner over meta-cpan blocks my build chain :P) and realized that you use perl-5.20.0 as it was in poky. A "simple" downgrade would be more reasonable ... if reason applys here in general :)

In practice, I am doing that. However, I want to understand why perl 5.22 breaks things and get it fixed.

I did a diff between your 5.20 and poky's 5.22 and realize some fixes applied in 5.22 regarding library path's aren't applied in your copy. Maybe swig relies on wrong library locations and when we know, we can fix.

So it's maybe not a 5.20 vs. 5.22 problem, it's maybe a weird swig setup problem.

Quoted 10 lines

Quoted 9 lines

When you fail on cross-build and succeed in target build, try to compare the C files and includes (even swig libraries) used.

It smells more like a "wrong source" than a "perl problem" (and even when I never would read any python thread, the same problem would likely occur there, too ^^).

Which perl headers are used in your build? To dig down, more logs would be reasonable ...

Quoted 3 lines

Everything comes from the same sources, same revisions, etc, as I'm using either a bitbake/OE build or the embedded (self-hosted) version from the same build plus SDK tools.

And your SDK does not include any host tools? Did you prove the intermediate amanda build files (eg. generated by SWIG) for relicts from wrong source? Did you check the logs which include directories had been used?

Quoted 91 lines

Am 23.03.2016 um 10:14 schrieb Jens Rehsack <[email protected]>:

Quoted 64 lines

Am 23.03.2016 um 10:09 schrieb Gary Thomas <[email protected]>:

Quoted 60 lines · Jens Rehsack

Quoted 59 lines

Am 23.03.2016 um 09:40 schrieb Gary Thomas <[email protected]>:

Quoted 33 lines · Gary Thomas

Quoted 32 lines · Khem Raj

Quoted 12 lines · Gary Thomas

I hope this is the correct place to discuss this problem. It is all about a difference in behavior between a program built using bitbake/OE (only OE-core is needed) vs building the program on the target hardware itself.

I've been struggling with this problem since perl was upgraded to version 5.22. I'm working on Amanda (Advanced Maryland Archive tool) which is written primarily in perl and uses swig interfaces to access native C functions. This code works great when using the previous perl (5.20.x) but fails on all 32 bit targets with perl 5.22

Quoted 9 lines

The interesting thing is that if I build Amanda on my target directly (using SDK tools), it works perfectly even with perl 5.22, so it seems that there is some [subtle] difference between building using bitbake/OE than when built on the self-hosted target. I've compared the builds and the only thing I could find (from the output of configure) is a difference in sizeof(off_t) Sadly, when I tried to adjust this in the OE build, it didn't make any difference, but perhaps I didn't make this change correctly or completely.

do you have largefile support turned on ? if you do then it might be detecting it wrongly during configure since we cache it to a non-largefile case

so try to add something like

EXTRA_OECONF += "${@bb.utils.contains('DISTRO_FEATURES', 'largefile', 'ac_cv_sizeof_off_t=8', '', d)}"

Quoted 7 lines

Quoted 1 line

while building perl or the affected program and see if that helps

Thanks for the idea, but that didn't help. I also forced some CFLAGS to match, in particular:
-D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64
but this didn't make any difference either.

On a whim I just tried a little experiment where I took the *.o files from the perl subdirectory (where all the swig shims live) from a working (self-hosted) build and moved them to my bitbake/OE build. I then touched all the *.o and *.lo files in the perl tree to force a relink. I then ran % bitbake amanda -C compile && bitbake core-image-base
to my surprise, amanda works! So the culprit lies somewhere within the swig generated glue. I've tried comparing these files before and I didn't find anything other than cosmetic differences (mostly comments about the name of the file processed, etc). I've added this subtree to "results" in my github layer in case someone can see what might be relevant.

Any ideas what might be different and make this swig generated glue fail? Note that the swig interface files are rebuilt as part of the build process and both bitbake/OE and self-hosted are using the same swig version.

Quoted 7 lines

Quoted 4 lines

I digged a bit through your layer (while my up2date scanner over meta-cpan blocks my build chain :P) and realized that you use perl-5.20.0 as it was in poky. A "simple" downgrade would be more reasonable ... if reason applys here in general :)

In practice, I am doing that. However, I want to understand why perl 5.22 breaks things and get it fixed.

I did a diff between your 5.20 and poky's 5.22 and realize some fixes applied in 5.22 regarding library path's aren't applied in your copy. Maybe swig relies on wrong library locations and when we know, we can fix.

So it's maybe not a 5.20 vs. 5.22 problem, it's maybe a weird swig setup problem.

Quoted 10 lines

Quoted 9 lines

When you fail on cross-build and succeed in target build, try to compare the C files and includes (even swig libraries) used.

It smells more like a "wrong source" than a "perl problem" (and even when I never would read any python thread, the same problem would likely occur there, too ^^).

Which perl headers are used in your build? To dig down, more logs would be reasonable ...

Quoted 7 lines

Quoted 3 lines

Everything comes from the same sources, same revisions, etc, as I'm using either a bitbake/OE build or the embedded (self-hosted) version from the same build plus SDK tools.

And your SDK does not include any host tools? Did you prove the intermediate amanda build files (eg. generated by SWIG) for relicts from wrong source? Did you check the logs which include directories had been used?

I give it a quick shot and got:

../../arm-poky-linux-gnueabi-libtool --tag=CC --mode=compile arm-poky-linux-gnueabi-gcc -march=armv7-a -marm -mthumb-interwork -mfloat-abi=hard -mfpu=neon -mtune=cortex-a9 --sysroot=/homes/sno/fsl-release-bsp/ornithologen-kann-man-mit-voegeln-eine-freude-machen/tmp/sysroots/curie -DHAVE_CONFIG_H -I. -I../../config -I../../common-src -I../../common-src -I../../xfer-src -I../../gnulib -I../../ndmp-src -I/homes/sno/fsl-release-bsp/ornithologen-kann-man-mit-voegeln-eine-freude-machen/tmp/sysroots/x86_64-linux/usr/lib/perl-native/perl/5.22.1/CORE -fno-strict-aliasing -D_GNU_SOURCE -pthread -I/homes/sno/fsl-release-bsp/ornithologen-kann-man-mit-voegeln-eine-freude-machen/tmp/sysroots/curie/usr/include/glib-2.0 -I/homes/sno/fsl-release-bsp/ornithologen-kann-man-mit-voegeln-eine-freude-machen/tmp/sysroots/curie/usr/lib/glib-2.0/include -DSWIG -O2 -pipe -g -feliminate-unused-debug-types -fipa-pta -ftree-partial-pre -ftree-loop-distribution -ftree-loop-distribute-patterns -fgcse-after-reload -fgcse-sm -fgcse-las -fno-strict-aliasing -c -o xferwrap.lo xferwrap.c

Do you see the "-I/homes/sno/fsl-release-bsp/ornithologen-kann-man-mit-voegeln-eine-freude-machen/tmp/sysroots/x86_64-linux/usr/lib/perl-native/perl/5.22.1/CORE" ?

Any further blames of perl-5.22? :)

Quoted 95 lines

Am 23.03.2016 um 10:14 schrieb Jens Rehsack <[email protected]>:

Quoted 64 lines

Am 23.03.2016 um 10:09 schrieb Gary Thomas <[email protected]>:

Quoted 60 lines · Jens Rehsack

Quoted 59 lines

Am 23.03.2016 um 09:40 schrieb Gary Thomas <[email protected]>:

Quoted 33 lines · Gary Thomas

Quoted 32 lines · Khem Raj

Quoted 12 lines · Gary Thomas

I hope this is the correct place to discuss this problem. It is all about a difference in behavior between a program built using bitbake/OE (only OE-core is needed) vs building the program on the target hardware itself.

I've been struggling with this problem since perl was upgraded to version 5.22. I'm working on Amanda (Advanced Maryland Archive tool) which is written primarily in perl and uses swig interfaces to access native C functions. This code works great when using the previous perl (5.20.x) but fails on all 32 bit targets with perl 5.22

Quoted 9 lines

The interesting thing is that if I build Amanda on my target directly (using SDK tools), it works perfectly even with perl 5.22, so it seems that there is some [subtle] difference between building using bitbake/OE than when built on the self-hosted target. I've compared the builds and the only thing I could find (from the output of configure) is a difference in sizeof(off_t) Sadly, when I tried to adjust this in the OE build, it didn't make any difference, but perhaps I didn't make this change correctly or completely.

do you have largefile support turned on ? if you do then it might be detecting it wrongly during configure since we cache it to a non-largefile case

so try to add something like

EXTRA_OECONF += "${@bb.utils.contains('DISTRO_FEATURES', 'largefile', 'ac_cv_sizeof_off_t=8', '', d)}"

Quoted 7 lines

Quoted 1 line

while building perl or the affected program and see if that helps

Thanks for the idea, but that didn't help. I also forced some CFLAGS to match, in particular:
-D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64
but this didn't make any difference either.

On a whim I just tried a little experiment where I took the *.o files from the perl subdirectory (where all the swig shims live) from a working (self-hosted) build and moved them to my bitbake/OE build. I then touched all the *.o and *.lo files in the perl tree to force a relink. I then ran % bitbake amanda -C compile && bitbake core-image-base
to my surprise, amanda works! So the culprit lies somewhere within the swig generated glue. I've tried comparing these files before and I didn't find anything other than cosmetic differences (mostly comments about the name of the file processed, etc). I've added this subtree to "results" in my github layer in case someone can see what might be relevant.

Any ideas what might be different and make this swig generated glue fail? Note that the swig interface files are rebuilt as part of the build process and both bitbake/OE and self-hosted are using the same swig version.

Quoted 7 lines

Quoted 4 lines

I digged a bit through your layer (while my up2date scanner over meta-cpan blocks my build chain :P) and realized that you use perl-5.20.0 as it was in poky. A "simple" downgrade would be more reasonable ... if reason applys here in general :)

In practice, I am doing that. However, I want to understand why perl 5.22 breaks things and get it fixed.

I did a diff between your 5.20 and poky's 5.22 and realize some fixes applied in 5.22 regarding library path's aren't applied in your copy. Maybe swig relies on wrong library locations and when we know, we can fix.

So it's maybe not a 5.20 vs. 5.22 problem, it's maybe a weird swig setup problem.

Quoted 13 lines

Quoted 9 lines

When you fail on cross-build and succeed in target build, try to compare the C files and includes (even swig libraries) used.

It smells more like a "wrong source" than a "perl problem" (and even when I never would read any python thread, the same problem would likely occur there, too ^^).

Which perl headers are used in your build? To dig down, more logs would be reasonable ...

Everything comes from the same sources, same revisions, etc, as I'm using either a bitbake/OE build or the embedded (self-hosted) version from the same build plus SDK tools.

Quoted 3 lines

And your SDK does not include any host tools? Did you prove the intermediate amanda build files (eg. generated by SWIG) for relicts from wrong source? Did you check the logs which include directories had been used?

I give it a quick shot and got:

../../arm-poky-linux-gnueabi-libtool --tag=CC --mode=compile arm-poky-linux-gnueabi-gcc -march=armv7-a -marm -mthumb-interwork -mfloat-abi=hard -mfpu=neon -mtune=cortex-a9 --sysroot=/homes/sno/fsl-release-bsp/ornithologen-kann-man-mit-voegeln-eine-freude-machen/tmp/sysroots/curie -DHAVE_CONFIG_H -I. -I../../config -I../../common-src -I../../common-src -I../../xfer-src -I../../gnulib -I../../ndmp-src -I/homes/sno/fsl-release-bsp/ornithologen-kann-man-mit-voegeln-eine-freude-machen/tmp/sysroots/x86_64-linux/usr/lib/perl-native/perl/5.22.1/CORE -fno-strict-aliasing -D_GNU_SOURCE -pthread -I/homes/sno/fsl-release-bsp/ornithologen-kann-man-mit-voegeln-eine-freude-machen/tmp/sysroots/curie/usr/include/glib-2.0 -I/homes/sno/fsl-release-bsp/ornithologen-kann-man-mit-voegeln-eine-freude-machen/tmp/sysroots/curie/usr/lib/glib-2.0/include -DSWIG -O2 -pipe -g -feliminate-unused-debug-types -fipa-pta -ftree-partial-pre -ftree-loop-distribution -ftree-loop-distribute-patterns -fgcse-afte

Do you see the "-I/homes/sno/fsl-release-bsp/ornithologen-kann-man-mit-voegeln-eine-freude-machen/tmp/sysroots/x86_64-linux/usr/lib/perl-native/perl/5.22.1/CORE" ?

Any further blames of perl-5.22? :)

Am 23.03.2016 um 12:05 schrieb Gary Thomas <[email protected]>:

Quoted 95 lines · Jens Rehsack

Quoted 95 lines

Am 23.03.2016 um 10:14 schrieb Jens Rehsack <[email protected]>:

Quoted 64 lines

Am 23.03.2016 um 10:09 schrieb Gary Thomas <[email protected]>:

Quoted 60 lines · Jens Rehsack

Quoted 59 lines

Am 23.03.2016 um 09:40 schrieb Gary Thomas <[email protected]>:

Quoted 33 lines · Gary Thomas

Quoted 32 lines · Khem Raj

Quoted 12 lines · Gary Thomas

I hope this is the correct place to discuss this problem. It is all about a difference in behavior between a program built using bitbake/OE (only OE-core is needed) vs building the program on the target hardware itself.

I've been struggling with this problem since perl was upgraded to version 5.22. I'm working on Amanda (Advanced Maryland Archive tool) which is written primarily in perl and uses swig interfaces to access native C functions. This code works great when using the previous perl (5.20.x) but fails on all 32 bit targets with perl 5.22

Quoted 9 lines

The interesting thing is that if I build Amanda on my target directly (using SDK tools), it works perfectly even with perl 5.22, so it seems that there is some [subtle] difference between building using bitbake/OE than when built on the self-hosted target. I've compared the builds and the only thing I could find (from the output of configure) is a difference in sizeof(off_t) Sadly, when I tried to adjust this in the OE build, it didn't make any difference, but perhaps I didn't make this change correctly or completely.

do you have largefile support turned on ? if you do then it might be detecting it wrongly during configure since we cache it to a non-largefile case

so try to add something like

EXTRA_OECONF += "${@bb.utils.contains('DISTRO_FEATURES', 'largefile', 'ac_cv_sizeof_off_t=8', '', d)}"

Quoted 7 lines

Quoted 1 line

while building perl or the affected program and see if that helps

Thanks for the idea, but that didn't help. I also forced some CFLAGS to match, in particular:
-D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64
but this didn't make any difference either.

On a whim I just tried a little experiment where I took the *.o files from the perl subdirectory (where all the swig shims live) from a working (self-hosted) build and moved them to my bitbake/OE build. I then touched all the *.o and *.lo files in the perl tree to force a relink. I then ran % bitbake amanda -C compile && bitbake core-image-base
to my surprise, amanda works! So the culprit lies somewhere within the swig generated glue. I've tried comparing these files before and I didn't find anything other than cosmetic differences (mostly comments about the name of the file processed, etc). I've added this subtree to "results" in my github layer in case someone can see what might be relevant.

Any ideas what might be different and make this swig generated glue fail? Note that the swig interface files are rebuilt as part of the build process and both bitbake/OE and self-hosted are using the same swig version.

Quoted 7 lines

Quoted 4 lines

I digged a bit through your layer (while my up2date scanner over meta-cpan blocks my build chain :P) and realized that you use perl-5.20.0 as it was in poky. A "simple" downgrade would be more reasonable ... if reason applys here in general :)

In practice, I am doing that. However, I want to understand why perl 5.22 breaks things and get it fixed.

I did a diff between your 5.20 and poky's 5.22 and realize some fixes applied in 5.22 regarding library path's aren't applied in your copy. Maybe swig relies on wrong library locations and when we know, we can fix.

So it's maybe not a 5.20 vs. 5.22 problem, it's maybe a weird swig setup problem.

Quoted 13 lines

Quoted 9 lines

When you fail on cross-build and succeed in target build, try to compare the C files and includes (even swig libraries) used.

It smells more like a "wrong source" than a "perl problem" (and even when I never would read any python thread, the same problem would likely occur there, too ^^).

Which perl headers are used in your build? To dig down, more logs would be reasonable ...

Everything comes from the same sources, same revisions, etc, as I'm using either a bitbake/OE build or the embedded (self-hosted) version from the same build plus SDK tools.

Quoted 7 lines

Quoted 3 lines

And your SDK does not include any host tools? Did you prove the intermediate amanda build files (eg. generated by SWIG) for relicts from wrong source? Did you check the logs which include directories had been used?

I give it a quick shot and got:

../../arm-poky-linux-gnueabi-libtool --tag=CC --mode=compile arm-poky-linux-gnueabi-gcc -march=armv7-a -marm -mthumb-interwork -mfloat-abi=hard -mfpu=neon -mtune=cortex-a9 --sysroot=/homes/sno/fsl-release-bsp/ornithologen-kann-man-mit-voegeln-eine-freude-machen/tmp/sysroots/curie -DHAVE_CONFIG_H -I. -I../../config -I../../common-src -I../../common-src -I../../xfer-src -I../../gnulib -I../../ndmp-src -I/homes/sno/fsl-release-bsp/ornithologen-kann-man-mit-voegeln-eine-freude-machen/tmp/sysroots/x86_64-linux/usr/lib/perl-native/perl/5.22.1/CORE -fno-strict-aliasing -D_GNU_SOURCE -pthread -I/homes/sno/fsl-release-bsp/ornithologen-kann-man-mit-voegeln-eine-freude-machen/tmp/sysroots/curie/usr/include/glib-2.0 -I/homes/sno/fsl-release-bsp/ornithologen-kann-man-mit-voegeln-eine-freude-machen/tmp/sysroots/curie/usr/lib/glib-2.0/include -DSWIG -O2 -pipe -g -feliminate-unused-debug-types -fipa-pta -ftree-partial-pre -ftree-loop-distribution -ftree-loop-distribute-patterns -fgcse-afte

r
-reload -fgcse-sm -fgcse-las -fno-strict-aliasing -c -o xferwrap.lo xferwrap.c

Quoted 1 line

Quoted 3 lines

Do you see the "-I/homes/sno/fsl-release-bsp/ornithologen-kann-man-mit-voegeln-eine-freude-machen/tmp/sysroots/x86_64-linux/usr/lib/perl-native/perl/5.22.1/CORE" ?

Any further blames of perl-5.22? :)

I'm not sure what you are getting at here. Indeed it is supposed to be using perl-5.22.1 for everything. The version 5.22.0 in meta-amanda layer is just for testing and should not be used at the moment.

All of my comparisons and testing (see instructions in the README) are for perl-5.22.1 only and in fact I've only been using the bits from bitbake/OE and not building that any more (I found that it wasn't necessary - one only needs to build amanda on the target)

Sorry for the confusion.

Quoted 99 lines

Am 23.03.2016 um 12:05 schrieb Gary Thomas <[email protected]>:

Quoted 95 lines · Jens Rehsack

Quoted 95 lines

Am 23.03.2016 um 10:14 schrieb Jens Rehsack <[email protected]>:

Quoted 64 lines

Am 23.03.2016 um 10:09 schrieb Gary Thomas <[email protected]>:

Quoted 60 lines · Jens Rehsack

Quoted 59 lines

Am 23.03.2016 um 09:40 schrieb Gary Thomas <[email protected]>:

Quoted 33 lines · Gary Thomas

Quoted 32 lines · Khem Raj

Quoted 12 lines · Gary Thomas

I hope this is the correct place to discuss this problem. It is all about a difference in behavior between a program built using bitbake/OE (only OE-core is needed) vs building the program on the target hardware itself.

I've been struggling with this problem since perl was upgraded to version 5.22. I'm working on Amanda (Advanced Maryland Archive tool) which is written primarily in perl and uses swig interfaces to access native C functions. This code works great when using the previous perl (5.20.x) but fails on all 32 bit targets with perl 5.22

Quoted 9 lines

The interesting thing is that if I build Amanda on my target directly (using SDK tools), it works perfectly even with perl 5.22, so it seems that there is some [subtle] difference between building using bitbake/OE than when built on the self-hosted target. I've compared the builds and the only thing I could find (from the output of configure) is a difference in sizeof(off_t) Sadly, when I tried to adjust this in the OE build, it didn't make any difference, but perhaps I didn't make this change correctly or completely.

do you have largefile support turned on ? if you do then it might be detecting it wrongly during configure since we cache it to a non-largefile case

so try to add something like

EXTRA_OECONF += "${@bb.utils.contains('DISTRO_FEATURES', 'largefile', 'ac_cv_sizeof_off_t=8', '', d)}"

Quoted 7 lines

Quoted 1 line

while building perl or the affected program and see if that helps

Thanks for the idea, but that didn't help. I also forced some CFLAGS to match, in particular:
-D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64
but this didn't make any difference either.

On a whim I just tried a little experiment where I took the *.o files from the perl subdirectory (where all the swig shims live) from a working (self-hosted) build and moved them to my bitbake/OE build. I then touched all the *.o and *.lo files in the perl tree to force a relink. I then ran % bitbake amanda -C compile && bitbake core-image-base
to my surprise, amanda works! So the culprit lies somewhere within the swig generated glue. I've tried comparing these files before and I didn't find anything other than cosmetic differences (mostly comments about the name of the file processed, etc). I've added this subtree to "results" in my github layer in case someone can see what might be relevant.

Any ideas what might be different and make this swig generated glue fail? Note that the swig interface files are rebuilt as part of the build process and both bitbake/OE and self-hosted are using the same swig version.

Quoted 7 lines

Quoted 4 lines

I digged a bit through your layer (while my up2date scanner over meta-cpan blocks my build chain :P) and realized that you use perl-5.20.0 as it was in poky. A "simple" downgrade would be more reasonable ... if reason applys here in general :)

In practice, I am doing that. However, I want to understand why perl 5.22 breaks things and get it fixed.

I did a diff between your 5.20 and poky's 5.22 and realize some fixes applied in 5.22 regarding library path's aren't applied in your copy. Maybe swig relies on wrong library locations and when we know, we can fix.

So it's maybe not a 5.20 vs. 5.22 problem, it's maybe a weird swig setup problem.

Quoted 13 lines

Quoted 9 lines

When you fail on cross-build and succeed in target build, try to compare the C files and includes (even swig libraries) used.

It smells more like a "wrong source" than a "perl problem" (and even when I never would read any python thread, the same problem would likely occur there, too ^^).

Which perl headers are used in your build? To dig down, more logs would be reasonable ...

Everything comes from the same sources, same revisions, etc, as I'm using either a bitbake/OE build or the embedded (self-hosted) version from the same build plus SDK tools.

Quoted 7 lines

Quoted 7 lines

Quoted 3 lines

And your SDK does not include any host tools? Did you prove the intermediate amanda build files (eg. generated by SWIG) for relicts from wrong source? Did you check the logs which include directories had been used?

I give it a quick shot and got:

../../arm-poky-linux-gnueabi-libtool --tag=CC --mode=compile arm-poky-linux-gnueabi-gcc -march=armv7-a -marm -mthumb-interwork -mfloat-abi=hard -mfpu=neon -mtune=cortex-a9 --sysroot=/homes/sno/fsl-release-bsp/ornithologen-kann-man-mit-voegeln-eine-freude-machen/tmp/sysroots/curie -DHAVE_CONFIG_H -I. -I../../config -I../../common-src -I../../common-src -I../../xfer-src -I../../gnulib -I../../ndmp-src -I/homes/sno/fsl-release-bsp/ornithologen-kann-man-mit-voegeln-eine-freude-machen/tmp/sysroots/x86_64-linux/usr/lib/perl-native/perl/5.22.1/CORE -fno-strict-aliasing -D_GNU_SOURCE -pthread -I/homes/sno/fsl-release-bsp/ornithologen-kann-man-mit-voegeln-eine-freude-machen/tmp/sysroots/curie/usr/include/glib-2.0 -I/homes/sno/fsl-release-bsp/ornithologen-kann-man-mit-voegeln-eine-freude-machen/tmp/sysroots/curie/usr/lib/glib-2.0/include -DSWIG -O2 -pipe -g -feliminate-unused-debug-types -fipa-pta -ftree-partial-pre -ftree-loop-distribution -ftree-loop-distribute-patterns -fgcse-af

Quoted 10 lines

r
-reload -fgcse-sm -fgcse-las -fno-strict-aliasing -c -o xferwrap.lo xferwrap.c

Quoted 4 lines

Do you see the "-I/homes/sno/fsl-release-bsp/ornithologen-kann-man-mit-voegeln-eine-freude-machen/tmp/sysroots/x86_64-linux/usr/lib/perl-native/perl/5.22.1/CORE" ?

Any further blames of perl-5.22? :)

I'm not sure what you are getting at here. Indeed it is supposed to be using perl-5.22.1 for everything. The version 5.22.0 in meta-amanda layer is just for testing and should not be used at the moment.

Please compare word wise ...

(quoted diff) +0 -2

-I/homes/sno/fsl-release-bsp/ornithologen-kann-man-mit-voegeln-eine-freude-machen/tmp/sysroots/x86_64-linux/usr/lib/perl-native/perl/5.22.1/CORE
-I/homes/sno/fsl-release-bsp/ornithologen-kann-man-mit-voegeln-eine-freude-machen/tmp/sysroots/curie/usr/include/glib-2.0

Probably this helps getting my point:

$ ls -l tmp/work
total 149
drwxrwxr-x  54 sno sno  54 Mar 22 23:49 all-poky-linux
drwxrwxr-x 269 sno sno 269 Mar 22 23:49 arm926ejse-poky-linux-gnueabi
drwxrwxr-x   3 sno sno   3 Mar 22 23:48 bohr-nand-poky-linux-gnueabi
drwxrwxr-x  19 sno sno  19 Mar 22 23:49 bohr-poky-linux-gnueabi
drwxrwxr-x   9 sno sno   9 Mar 22 21:38 cortexa9hf-vfp-neon-mx6qdl-poky-linux-gnueabi
drwxrwxr-x 326 sno sno 326 Mar 23 11:28 cortexa9hf-vfp-neon-poky-linux-gnueabi
drwxrwxr-x   3 sno sno   3 Mar 22 21:37 curie-emmc-poky-linux-gnueabi
drwxrwxr-x  29 sno sno  29 Mar 22 21:50 curie-poky-linux-gnueabi
drwxrwxr-x 196 sno sno 196 Mar 23 11:28 x86_64-linux

Quoted 6 lines

All of my comparisons and testing (see instructions in the README) are for perl-5.22.1 only and in fact I've only been using the bits from bitbake/OE and not building that any more (I found that it wasn't necessary - one only needs to build amanda on the target)

Sorry for the confusion.

Am 23.03.2016 um 13:07 schrieb Gary Thomas <[email protected]>:

Quoted 99 lines · Jens Rehsack

Quoted 99 lines

Am 23.03.2016 um 12:05 schrieb Gary Thomas <[email protected]>:

Quoted 95 lines · Jens Rehsack

Quoted 95 lines

Am 23.03.2016 um 10:14 schrieb Jens Rehsack <[email protected]>:

Quoted 64 lines

Am 23.03.2016 um 10:09 schrieb Gary Thomas <[email protected]>:

Quoted 60 lines · Jens Rehsack

Quoted 59 lines

Am 23.03.2016 um 09:40 schrieb Gary Thomas <[email protected]>:

Quoted 33 lines · Gary Thomas

Quoted 32 lines · Khem Raj

Quoted 12 lines · Gary Thomas

I hope this is the correct place to discuss this problem. It is all about a difference in behavior between a program built using bitbake/OE (only OE-core is needed) vs building the program on the target hardware itself.

I've been struggling with this problem since perl was upgraded to version 5.22. I'm working on Amanda (Advanced Maryland Archive tool) which is written primarily in perl and uses swig interfaces to access native C functions. This code works great when using the previous perl (5.20.x) but fails on all 32 bit targets with perl 5.22

Quoted 9 lines

The interesting thing is that if I build Amanda on my target directly (using SDK tools), it works perfectly even with perl 5.22, so it seems that there is some [subtle] difference between building using bitbake/OE than when built on the self-hosted target. I've compared the builds and the only thing I could find (from the output of configure) is a difference in sizeof(off_t) Sadly, when I tried to adjust this in the OE build, it didn't make any difference, but perhaps I didn't make this change correctly or completely.

do you have largefile support turned on ? if you do then it might be detecting it wrongly during configure since we cache it to a non-largefile case

so try to add something like

EXTRA_OECONF += "${@bb.utils.contains('DISTRO_FEATURES', 'largefile', 'ac_cv_sizeof_off_t=8', '', d)}"

Quoted 7 lines

Quoted 1 line

while building perl or the affected program and see if that helps

Thanks for the idea, but that didn't help. I also forced some CFLAGS to match, in particular:
-D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64
but this didn't make any difference either.

On a whim I just tried a little experiment where I took the *.o files from the perl subdirectory (where all the swig shims live) from a working (self-hosted) build and moved them to my bitbake/OE build. I then touched all the *.o and *.lo files in the perl tree to force a relink. I then ran % bitbake amanda -C compile && bitbake core-image-base
to my surprise, amanda works! So the culprit lies somewhere within the swig generated glue. I've tried comparing these files before and I didn't find anything other than cosmetic differences (mostly comments about the name of the file processed, etc). I've added this subtree to "results" in my github layer in case someone can see what might be relevant.

Any ideas what might be different and make this swig generated glue fail? Note that the swig interface files are rebuilt as part of the build process and both bitbake/OE and self-hosted are using the same swig version.

Quoted 7 lines

Quoted 4 lines

I digged a bit through your layer (while my up2date scanner over meta-cpan blocks my build chain :P) and realized that you use perl-5.20.0 as it was in poky. A "simple" downgrade would be more reasonable ... if reason applys here in general :)

In practice, I am doing that. However, I want to understand why perl 5.22 breaks things and get it fixed.

I did a diff between your 5.20 and poky's 5.22 and realize some fixes applied in 5.22 regarding library path's aren't applied in your copy. Maybe swig relies on wrong library locations and when we know, we can fix.

So it's maybe not a 5.20 vs. 5.22 problem, it's maybe a weird swig setup problem.

Quoted 13 lines

Quoted 9 lines

When you fail on cross-build and succeed in target build, try to compare the C files and includes (even swig libraries) used.

It smells more like a "wrong source" than a "perl problem" (and even when I never would read any python thread, the same problem would likely occur there, too ^^).

Which perl headers are used in your build? To dig down, more logs would be reasonable ...

Everything comes from the same sources, same revisions, etc, as I'm using either a bitbake/OE build or the embedded (self-hosted) version from the same build plus SDK tools.

Quoted 7 lines

Quoted 7 lines

Quoted 7 lines

Quoted 3 lines

And your SDK does not include any host tools? Did you prove the intermediate amanda build files (eg. generated by SWIG) for relicts from wrong source? Did you check the logs which include directories had been used?

I give it a quick shot and got:

../../arm-poky-linux-gnueabi-libtool --tag=CC --mode=compile arm-poky-linux-gnueabi-gcc -march=armv7-a -marm -mthumb-interwork -mfloat-abi=hard -mfpu=neon -mtune=cortex-a9 --sysroot=/homes/sno/fsl-release-bsp/ornithologen-kann-man-mit-voegeln-eine-freude-machen/tmp/sysroots/curie -DHAVE_CONFIG_H -I. -I../../config -I../../common-src -I../../common-src -I../../xfer-src -I../../gnulib -I../../ndmp-src -I/homes/sno/fsl-release-bsp/ornithologen-kann-man-mit-voegeln-eine-freude-machen/tmp/sysroots/x86_64-linux/usr/lib/perl-native/perl/5.22.1/CORE -fno-strict-aliasing -D_GNU_SOURCE -pthread -I/homes/sno/fsl-release-bsp/ornithologen-kann-man-mit-voegeln-eine-freude-machen/tmp/sysroots/curie/usr/include/glib-2.0 -I/homes/sno/fsl-release-bsp/ornithologen-kann-man-mit-voegeln-eine-freude-machen/tmp/sysroots/curie/usr/lib/glib-2.0/include -DSWIG -O2 -pipe -g -feliminate-unused-debug-types -fipa-pta -ftree-partial-pre -ftree-loop-distribution -ftree-loop-distribute-patterns -fgcse-af

t
e

Quoted 18 lines

Quoted 10 lines

r
-reload -fgcse-sm -fgcse-las -fno-strict-aliasing -c -o xferwrap.lo xferwrap.c

Quoted 4 lines

Do you see the "-I/homes/sno/fsl-release-bsp/ornithologen-kann-man-mit-voegeln-eine-freude-machen/tmp/sysroots/x86_64-linux/usr/lib/perl-native/perl/5.22.1/CORE" ?

Any further blames of perl-5.22? :)

I'm not sure what you are getting at here. Indeed it is supposed to be using perl-5.22.1 for everything. The version 5.22.0 in meta-amanda layer is just for testing and should not be used at the moment.

Please compare word wise ...

(quoted diff) +0 -2

-I/homes/sno/fsl-release-bsp/ornithologen-kann-man-mit-voegeln-eine-freude-machen/tmp/sysroots/x86_64-linux/usr/lib/perl-native/perl/5.22.1/CORE
-I/homes/sno/fsl-release-bsp/ornithologen-kann-man-mit-voegeln-eine-freude-machen/tmp/sysroots/curie/usr/include/glib-2.0

Probably this helps getting my point:

Quoted 19 lines

$ ls -l tmp/work
total 149
drwxrwxr-x  54 sno sno  54 Mar 22 23:49 all-poky-linux
drwxrwxr-x 269 sno sno 269 Mar 22 23:49 arm926ejse-poky-linux-gnueabi
drwxrwxr-x   3 sno sno   3 Mar 22 23:48 bohr-nand-poky-linux-gnueabi
drwxrwxr-x  19 sno sno  19 Mar 22 23:49 bohr-poky-linux-gnueabi
drwxrwxr-x   9 sno sno   9 Mar 22 21:38 cortexa9hf-vfp-neon-mx6qdl-poky-linux-gnueabi
drwxrwxr-x 326 sno sno 326 Mar 23 11:28 cortexa9hf-vfp-neon-poky-linux-gnueabi
drwxrwxr-x   3 sno sno   3 Mar 22 21:37 curie-emmc-poky-linux-gnueabi
drwxrwxr-x  29 sno sno  29 Mar 22 21:50 curie-poky-linux-gnueabi
drwxrwxr-x 196 sno sno 196 Mar 23 11:28 x86_64-linux

Quoted 6 lines

All of my comparisons and testing (see instructions in the README) are for perl-5.22.1 only and in fact I've only been using the bits from bitbake/OE and not building that any more (I found that it wasn't necessary - one only needs to build amanda on the target)

Sorry for the confusion.

Huzzah! This seems to be on the right track. I adjusted the PERL_INC value in perl/Makefile (the *.h files used by the swig interfaces) to point to the target ones and not the perl-native ones and now it works :-)

I'm not sure what's different between the two sets of include files, and more interestingly why it worked with previous versions of perl, but this does seem to be the culprit.

Now I just need to figure out how to convince the 'configure' process to do it correctly.

Thanks for the help!

Quoted 103 lines

Am 23.03.2016 um 13:07 schrieb Gary Thomas <[email protected]>:

Quoted 99 lines · Jens Rehsack

Quoted 99 lines

Am 23.03.2016 um 12:05 schrieb Gary Thomas <[email protected]>:

Quoted 95 lines · Jens Rehsack

Quoted 95 lines

Am 23.03.2016 um 10:14 schrieb Jens Rehsack <[email protected]>:

Quoted 64 lines

Am 23.03.2016 um 10:09 schrieb Gary Thomas <[email protected]>:

Quoted 60 lines · Jens Rehsack

Quoted 59 lines

Am 23.03.2016 um 09:40 schrieb Gary Thomas <[email protected]>:

Quoted 33 lines · Gary Thomas

Quoted 32 lines · Khem Raj

Quoted 12 lines · Gary Thomas

I hope this is the correct place to discuss this problem. It is all about a difference in behavior between a program built using bitbake/OE (only OE-core is needed) vs building the program on the target hardware itself.

I've been struggling with this problem since perl was upgraded to version 5.22. I'm working on Amanda (Advanced Maryland Archive tool) which is written primarily in perl and uses swig interfaces to access native C functions. This code works great when using the previous perl (5.20.x) but fails on all 32 bit targets with perl 5.22

Quoted 9 lines

The interesting thing is that if I build Amanda on my target directly (using SDK tools), it works perfectly even with perl 5.22, so it seems that there is some [subtle] difference between building using bitbake/OE than when built on the self-hosted target. I've compared the builds and the only thing I could find (from the output of configure) is a difference in sizeof(off_t) Sadly, when I tried to adjust this in the OE build, it didn't make any difference, but perhaps I didn't make this change correctly or completely.

do you have largefile support turned on ? if you do then it might be detecting it wrongly during configure since we cache it to a non-largefile case

so try to add something like

EXTRA_OECONF += "${@bb.utils.contains('DISTRO_FEATURES', 'largefile', 'ac_cv_sizeof_off_t=8', '', d)}"

Quoted 7 lines

Quoted 1 line

while building perl or the affected program and see if that helps

Thanks for the idea, but that didn't help. I also forced some CFLAGS to match, in particular:
-D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64
but this didn't make any difference either.

On a whim I just tried a little experiment where I took the *.o files from the perl subdirectory (where all the swig shims live) from a working (self-hosted) build and moved them to my bitbake/OE build. I then touched all the *.o and *.lo files in the perl tree to force a relink. I then ran % bitbake amanda -C compile && bitbake core-image-base
to my surprise, amanda works! So the culprit lies somewhere within the swig generated glue. I've tried comparing these files before and I didn't find anything other than cosmetic differences (mostly comments about the name of the file processed, etc). I've added this subtree to "results" in my github layer in case someone can see what might be relevant.

Any ideas what might be different and make this swig generated glue fail? Note that the swig interface files are rebuilt as part of the build process and both bitbake/OE and self-hosted are using the same swig version.

Quoted 7 lines

Quoted 4 lines

I digged a bit through your layer (while my up2date scanner over meta-cpan blocks my build chain :P) and realized that you use perl-5.20.0 as it was in poky. A "simple" downgrade would be more reasonable ... if reason applys here in general :)

In practice, I am doing that. However, I want to understand why perl 5.22 breaks things and get it fixed.

I did a diff between your 5.20 and poky's 5.22 and realize some fixes applied in 5.22 regarding library path's aren't applied in your copy. Maybe swig relies on wrong library locations and when we know, we can fix.

So it's maybe not a 5.20 vs. 5.22 problem, it's maybe a weird swig setup problem.

Quoted 13 lines

Quoted 9 lines

When you fail on cross-build and succeed in target build, try to compare the C files and includes (even swig libraries) used.

It smells more like a "wrong source" than a "perl problem" (and even when I never would read any python thread, the same problem would likely occur there, too ^^).

Which perl headers are used in your build? To dig down, more logs would be reasonable ...

Everything comes from the same sources, same revisions, etc, as I'm using either a bitbake/OE build or the embedded (self-hosted) version from the same build plus SDK tools.

Quoted 7 lines

Quoted 7 lines

Quoted 7 lines

Quoted 7 lines

Quoted 3 lines

And your SDK does not include any host tools? Did you prove the intermediate amanda build files (eg. generated by SWIG) for relicts from wrong source? Did you check the logs which include directories had been used?

I give it a quick shot and got:

../../arm-poky-linux-gnueabi-libtool --tag=CC --mode=compile arm-poky-linux-gnueabi-gcc -march=armv7-a -marm -mthumb-interwork -mfloat-abi=hard -mfpu=neon -mtune=cortex-a9 --sysroot=/homes/sno/fsl-release-bsp/ornithologen-kann-man-mit-voegeln-eine-freude-machen/tmp/sysroots/curie -DHAVE_CONFIG_H -I. -I../../config -I../../common-src -I../../common-src -I../../xfer-src -I../../gnulib -I../../ndmp-src -I/homes/sno/fsl-release-bsp/ornithologen-kann-man-mit-voegeln-eine-freude-machen/tmp/sysroots/x86_64-linux/usr/lib/perl-native/perl/5.22.1/CORE -fno-strict-aliasing -D_GNU_SOURCE -pthread -I/homes/sno/fsl-release-bsp/ornithologen-kann-man-mit-voegeln-eine-freude-machen/tmp/sysroots/curie/usr/include/glib-2.0 -I/homes/sno/fsl-release-bsp/ornithologen-kann-man-mit-voegeln-eine-freude-machen/tmp/sysroots/curie/usr/lib/glib-2.0/include -DSWIG -O2 -pipe -g -feliminate-unused-debug-types -fipa-pta -ftree-partial-pre -ftree-loop-distribution -ftree-loop-distribute-patterns -fgcse-

Quoted 48 lines

t
e

Quoted 18 lines

Quoted 10 lines

r
-reload -fgcse-sm -fgcse-las -fno-strict-aliasing -c -o xferwrap.lo xferwrap.c

Quoted 4 lines

Do you see the "-I/homes/sno/fsl-release-bsp/ornithologen-kann-man-mit-voegeln-eine-freude-machen/tmp/sysroots/x86_64-linux/usr/lib/perl-native/perl/5.22.1/CORE" ?

Any further blames of perl-5.22? :)

I'm not sure what you are getting at here. Indeed it is supposed to be using perl-5.22.1 for everything. The version 5.22.0 in meta-amanda layer is just for testing and should not be used at the moment.

Please compare word wise ...

(quoted diff) +0 -2

-I/homes/sno/fsl-release-bsp/ornithologen-kann-man-mit-voegeln-eine-freude-machen/tmp/sysroots/x86_64-linux/usr/lib/perl-native/perl/5.22.1/CORE
-I/homes/sno/fsl-release-bsp/ornithologen-kann-man-mit-voegeln-eine-freude-machen/tmp/sysroots/curie/usr/include/glib-2.0

Probably this helps getting my point:

Quoted 19 lines

$ ls -l tmp/work
total 149
drwxrwxr-x  54 sno sno  54 Mar 22 23:49 all-poky-linux
drwxrwxr-x 269 sno sno 269 Mar 22 23:49 arm926ejse-poky-linux-gnueabi
drwxrwxr-x   3 sno sno   3 Mar 22 23:48 bohr-nand-poky-linux-gnueabi
drwxrwxr-x  19 sno sno  19 Mar 22 23:49 bohr-poky-linux-gnueabi
drwxrwxr-x   9 sno sno   9 Mar 22 21:38 cortexa9hf-vfp-neon-mx6qdl-poky-linux-gnueabi
drwxrwxr-x 326 sno sno 326 Mar 23 11:28 cortexa9hf-vfp-neon-poky-linux-gnueabi
drwxrwxr-x   3 sno sno   3 Mar 22 21:37 curie-emmc-poky-linux-gnueabi
drwxrwxr-x  29 sno sno  29 Mar 22 21:50 curie-poky-linux-gnueabi
drwxrwxr-x 196 sno sno 196 Mar 23 11:28 x86_64-linux

Quoted 6 lines

All of my comparisons and testing (see instructions in the README) are for perl-5.22.1 only and in fact I've only been using the bits from bitbake/OE and not building that any more (I found that it wasn't necessary - one only needs to build amanda on the target)

Sorry for the confusion.

Huzzah! This seems to be on the right track. I adjusted the PERL_INC value in perl/Makefile (the *.h files used by the swig interfaces) to point to the target ones and not the perl-native ones and now it works :-)

I'm not sure what's different between the two sets of include files, and more interestingly why it worked with previous versions of perl, but this does seem to be the culprit.

Quoted 2 lines

Now I just need to figure out how to convince the 'configure' process to do it correctly.

IIRC, rrdtool is a good example to do it right. collectd had the same problem as amanda had - wrong perl used ...
Maybe vice versa - I'm getting old and forget to remember :D