core: add option to disable RPATH sanitation in target tree
RFC PATCH 1/1 core: add option to disable RPATH sanitation in target tree
---
As explained on the Config, the rationale is to speed up BR2_EXTERNAL package development build iteration.
This would make users targetting virtual machines happy.
It may be argued that embedded images are not supposed to be small, but a counter argument is that there are already in-tree packages that benefit considerably from this, e.g. ltp-testsuite, which drops the build iteration from 18s to 5s on my machine, which is a huge life improvement.
Additionaly, this will also make it easier for us to find and report the offending packages mentioned at 5f4ca518094f7690536074246ab765f36a213783 and push for a correct upstream fix, are we tracking that somewhere? Removing the rpath fix actually makes those packages not work is that it?
I'm not sure if this should be an actual config, thus the RFC. Without the config, it is also possible to use it just as:
make BR2_SANITIZE_RPATH=n
Config.in | 7 +++++++ Makefile | 2 ++ 2 files changed, 9 insertions(+)
Config.in +7 -0
@@ -678,6 +678,13 @@ config BR2_COMPILER_PARANOID_UNSAFE_PATH and external toolchain backends (through the toolchain wrapper). +config BR2_SANITIZE_RPATH + bool "sanitize rpath" + default y + help + You may want to disable this while developing a package with + BR2_EXTERNAL to make build iterations faster. + config BR2_REPRODUCIBLE bool "Make the build reproducible (experimental)" # SOURCE_DATE_EPOCH support in toolchain-wrapper requires GCC 4.4
Makefile +2 -0
@@ -743,8 +743,10 @@ endif ) > $(TARGET_DIR)/usr/lib/os-release ln -sf ../usr/lib/os-release $(TARGET_DIR)/etc +ifeq ($(BR2_SANITIZE_RPATH),y) @$(call MESSAGE,"Sanitizing RPATH in target tree") $(TOPDIR)/support/scripts/fix-rpath target +endif @$(foreach d, $(call qstrip,$(BR2_ROOTFS_OVERLAY)), \ $(call MESSAGE,"Copying overlay $(d)"); \
Signature · 1 line
2.16.1
RFC PATCH 1/1 core: add option to disable RPATH sanitation in target tree
Quoted 4 lines · Ciro Santilli
Signed-off-by Ciro Santilli <[email protected]>---
As explained on the Config, the rationale is to speed upBR2_EXTERNALpackage development build iteration.
I don't see what this has to do with package development. Anyway, when you're
developing package foo, wouldn't you run 'make foo' rather than just 'make' (and thus never even hit that code path)?
Quoted 1 line
This would make users targetting virtual machines happy.
Huh? Or you mean building in virtual machines?
Quoted 10 lines
It may be argued that embedded images are not supposed to be small, but a counter argument is that there are already in-tree packages that benefit considerably from this, e.g. ltp-testsuite, which drops the build iteration from 18s to 5s on my machine, which is a huge life improvement.
Additionaly, this will also make it easier for us to find and report the offending packages mentioned at
5f4ca518094f7690536074246ab765f36a213783and push for a correct upstream fix, are we tracking that somewhere?
No, it falls in the category "not worth the time to try to fix upstream",
especially since we're talking about things like SELinux which consists of dozens of packages that all do it wrong.
Quoted 1 line
Removing the rpath fix actually makes those packages not work is that it?
Usually not, the rpaths are mostly useless. They are typically added because
the executable is linked with a library that may not yet be installed.
Quoted 2 lines
I'm not sure if this should be an actual config, thus the RFC. Without the config, it is also possible to use it just as:
TBH, I don't think it's worth the bother either way. Just skip the entire
finalize/rootfs creation step if you're not interested.
Regards,
Arnout
Quoted 41 lines
make BR2_SANITIZE_RPATH=n
Config.in | 7 +++++++ Makefile | 2 ++ 2 files changed, 9 insertions(+)Config.in +7
-0@@ -678,6 +678,13 @@ config BR2_COMPILER_PARANOID_UNSAFE_PATH and external toolchain backends (through the toolchain wrapper). +config BR2_SANITIZE_RPATH + bool "sanitize rpath" + default y + help + You may want to disable this while developing a package with + BR2_EXTERNAL to make build iterations faster. + config BR2_REPRODUCIBLE bool "Make the build reproducible (experimental)" # SOURCE_DATE_EPOCH support in toolchain-wrapper requires GCC 4.4Makefile +2
-0@@ -743,8 +743,10 @@ endif ) > $(TARGET_DIR)/usr/lib/os-release ln -sf ../usr/lib/os-release $(TARGET_DIR)/etc +ifeq ($(BR2_SANITIZE_RPATH),y) @$(call MESSAGE,"Sanitizing RPATH in target tree") $(TOPDIR)/support/scripts/fix-rpath target +endif @$(foreach d, $(call qstrip,$(BR2_ROOTFS_OVERLAY)), \ $(call MESSAGE,"Copying overlay $(d)"); \
Signature · 6 lines
Arnout Vandecappelle arnout at mind be Senior Embedded Software Architect +32-16-286500 Essensium/Mind http://www.mind.be G.Geenslaan 9, 3001 Leuven, Belgium BE 872 984 063 RPR Leuven LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle GPG fingerprint: 7493 020B C7E3 8618 8DEC 222C 82EB F404 F9AC 0DDF
RFC PATCH 1/1 core: add option to disable RPATH sanitation in target tree
Quoted 12 lines · Arnout Vandecappelle
Quoted 4 lines · Ciro Santilli
Signed-off-by Ciro Santilli <[email protected]>---
As explained on the Config, the rationale is to speed upBR2_EXTERNALpackage development build iteration.I don't see what this has to do with package development. Anyway, when you're
developing package foo, wouldn't you run 'make foo' rather than just 'make' (and thus never even hit that code path)?
How to test the modification without rebuilding the image to run it on the target?
Quoted 4 lines
Quoted 1 line
This would make users targetting virtual machines happy.
Huh? Or you mean building in virtual machines?
No, cross compiling for VMs from host as opposed to real hardware.
Quoted 79 lines
Quoted 10 lines
It may be argued that embedded images are not supposed to be small, but a counter argument is that there are already in-tree packages that benefit considerably from this, e.g. ltp-testsuite, which drops the build iteration from 18s to 5s on my machine, which is a huge life improvement.
Additionaly, this will also make it easier for us to find and report the offending packages mentioned at
5f4ca518094f7690536074246ab765f36a213783and push for a correct upstream fix, are we tracking that somewhere?No, it falls in the category "not worth the time to try to fix upstream",
especially since we're talking about things like SELinux which consists of dozens of packages that all do it wrong.Quoted 1 line
Removing the rpath fix actually makes those packages not work is that it?
Usually not, the rpaths are mostly useless. They are typically added because
the executable is linked with a library that may not yet be installed.Quoted 2 lines
I'm not sure if this should be an actual config, thus the RFC. Without the config, it is also possible to use it just as:
TBH, I don't think it's worth the bother either way. Just skip the entire
finalize/rootfs creation step if you're not interested.Regards,
ArnoutQuoted 31 lines
make BR2_SANITIZE_RPATH=n
Config.in | 7 +++++++ Makefile | 2 ++ 2 files changed, 9 insertions(+)Config.in +7
-0@@ -678,6 +678,13 @@ config BR2_COMPILER_PARANOID_UNSAFE_PATH and external toolchain backends (through the toolchain wrapper). +config BR2_SANITIZE_RPATH + bool "sanitize rpath" + default y + help + You may want to disable this while developing a package with + BR2_EXTERNAL to make build iterations faster. + config BR2_REPRODUCIBLE bool "Make the build reproducible (experimental)" # SOURCE_DATE_EPOCH support in toolchain-wrapper requires GCC 4.4Makefile +0
-0Makefile endif · line 752
Quoted 10 lines
Makefile +2
-0@@ -743,8 +743,10 @@ endif ) > $(TARGET_DIR)/usr/lib/os-release ln -sf ../usr/lib/os-release $(TARGET_DIR)/etc +ifeq ($(BR2_SANITIZE_RPATH),y) @$(call MESSAGE,"Sanitizing RPATH in target tree") $(TOPDIR)/support/scripts/fix-rpath target +endif @$(foreach d, $(call qstrip,$(BR2_ROOTFS_OVERLAY)), \ $(call MESSAGE,"Copying overlay $(d)"); \--
Arnout Vandecappelle arnout at mind be Senior Embedded Software Architect +32-16-286500 Essensium/Mind http://www.mind.be G.Geenslaan 9, 3001 Leuven, Belgium BE 872 984 063 RPR Leuven LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle GPG fingerprint: 7493 020B C7E3 8618 8DEC 222C 82EB F404 F9AC 0DDF
RFC PATCH 1/1 core: add option to disable RPATH sanitation in target tree
Quoted 9 lines · Ciro Santilli
Quoted 8 lines · Arnout Vandecappelle
Quoted 4 lines · Ciro Santilli
Signed-off-by Ciro Santilli <[email protected]>---
As explained on the Config, the rationale is to speed upBR2_EXTERNALpackage development build iteration.
Quoted 7 lines
Quoted 4 lines
I don't see what this has to do with package development. Anyway, when you're
developing package foo, wouldn't you run 'make foo' rather than just 'make' (and thus never even hit that code path)?How to test the modification without rebuilding the image to run it on the target?
Ah, I usually just scp or rsync the (unstripped) binaries to the target system.
I'll typically write a little script that does something like 'make FOO_OVERRIDE_SRCDIR=... foo; ssh target rm -f /usr/bin/foo; scp output/target/usr/bin/foo target:/usr/bin; ssh target killall foo; ssh -f target foo"
Quoted 7 lines
Quoted 5 lines
Quoted 2 lines
This would make users targetting virtual machines happy.
Huh? Or you mean building in virtual machines?
No, cross compiling for VMs from host as opposed to real hardware.
What difference does it make whether your target is a VM or real hardware?
Regards,
Arnout
[snip]
Signature · 6 lines
Arnout Vandecappelle arnout at mind be Senior Embedded Software Architect +32-16-286500 Essensium/Mind http://www.mind.be G.Geenslaan 9, 3001 Leuven, Belgium BE 872 984 063 RPR Leuven LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle GPG fingerprint: 7493 020B C7E3 8618 8DEC 222C 82EB F404 F9AC 0DDF
RFC PATCH 1/1 core: add option to disable RPATH sanitation in target tree
Quoted 17 lines · Arnout Vandecappelle
Quoted 14 lines · Ciro Santilli
Quoted 12 lines · Arnout Vandecappelle
Quoted 4 lines · Ciro Santilli
Signed-off-by Ciro Santilli <[email protected]>---
As explained on the Config, the rationale is to speed upBR2_EXTERNALpackage development build iteration.I don't see what this has to do with package development. Anyway, when you're
developing package foo, wouldn't you run 'make foo' rather than just 'make' (and thus never even hit that code path)?
Quoted 9 lines
Quoted 2 lines
How to test the modification without rebuilding the image to run it on the target?
Ah, I usually just scp or rsync the (unstripped) binaries to the target system.
I'll typically write a little script that does something like 'make FOO_OVERRIDE_SRCDIR=... foo; ssh target rm -f/usr/bin/foo; scp output/target/usr/bin/foo target:/usr/bin; ssh target killall foo; ssh -f target foo"
That is a possibility.
But I feel it is simpler and more reliable if can just let Buildroot do exactly what it normally does, especially if a package installs many files in different directories, and since reboot is so fast.
Otherwise we force users to come up with those setups themselves for every new package they want to modify.
Even cooler would be if QEMU could Build a separate image with just a single package (automatically for all OVERRIDE_SRCIDR ?), and then if I found out how to overlay mount it on root:
https://stackoverflow.com/questions/41119656/how-can-i-overlayfs-the-root-filesystem-on-linux
But OK, let's start simple :-)
Quoted 10 lines
Quoted 7 lines
Quoted 5 lines
Quoted 2 lines
This would make users targetting virtual machines happy.
Huh? Or you mean building in virtual machines?
No, cross compiling for VMs from host as opposed to real hardware.
What difference does it make whether your target is a VM or real hardware?
Just the idea that real embedded HW has more limited disk space, and therefore RPATH fixup time will be small.
But true, now that I think about it this should be annoying for real embedded boards hardware as well, 4Gb on a RPI is more than enough.
Quoted 12 lines
Regards,
Arnout[snip]
--
Arnout Vandecappelle arnout at mind be Senior Embedded Software Architect +32-16-286500 Essensium/Mind http://www.mind.be G.Geenslaan 9, 3001 Leuven, Belgium BE 872 984 063 RPR Leuven LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle GPG fingerprint: 7493 020B C7E3 8618 8DEC 222C 82EB F404 F9AC 0DDF
RFC PATCH 1/1 core: add option to disable RPATH sanitation in target tree
In reply to Arnout VandecappelleQuoted 17 lines · Arnout Vandecappelle
Quoted 14 lines · Ciro Santilli
Quoted 12 lines · Arnout Vandecappelle
Quoted 4 lines · Ciro Santilli
Signed-off-by Ciro Santilli <[email protected]>---
As explained on the Config, the rationale is to speed upBR2_EXTERNALpackage development build iteration.I don't see what this has to do with package development. Anyway, when you're
developing package foo, wouldn't you run 'make foo' rather than just 'make' (and thus never even hit that code path)?
Quoted 9 lines
Quoted 2 lines
How to test the modification without rebuilding the image to run it on the target?
Ah, I usually just scp or rsync the (unstripped) binaries to the target system.
I'll typically write a little script that does something like 'make FOO_OVERRIDE_SRCDIR=... foo; ssh target rm -f/usr/bin/foo; scp output/target/usr/bin/foo target:/usr/bin; ssh target killall foo; ssh -f target foo"
@Thomas: do you have a better solution? How to you avoid the RPATH wait time when the image is large?
Quoted 22 lines
Quoted 7 lines
Quoted 5 lines
Quoted 2 lines
This would make users targetting virtual machines happy.
Huh? Or you mean building in virtual machines?
No, cross compiling for VMs from host as opposed to real hardware.
What difference does it make whether your target is a VM or real hardware?
Regards,
Arnout[snip]
--
Arnout Vandecappelle arnout at mind be Senior Embedded Software Architect +32-16-286500 Essensium/Mind http://www.mind.be G.Geenslaan 9, 3001 Leuven, Belgium BE 872 984 063 RPR Leuven LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle GPG fingerprint: 7493 020B C7E3 8618 8DEC 222C 82EB F404 F9AC 0DDF