core/legal-info: fixes, completness and enhancements (branch yem/legal-patch)
cover core/legal-info: fixes, completness and enhancements (branch yem/legal-patch)
Hello All!
This series introduces core changs in how we handle the legal-info for various packages.
The underlying reason for this series is that our legal-info content is far from being complete. Most notably, the patches applied to packages are not part of the legal-info output.
We do highly suggest users do redistribute Buildroot as part of their compliance delivery, so the patches we bundle would be part of that compliance deilvery.
However, patches that are downloaded or (most importantly) patches that come from a global patch directory would be missing from that compliance delivery.
The series starts with minor fixes to libecore and the legal-info infra, and with adding licensing information to the Linux headers.
Then we add a new keyword to the _REDISTRIBUTE variable, so a package can be completely ignored from the legal-info ouput (no source saved, not license file saved, not listed in the manifest). Virtual packages are forcibly set to be ignored that way.
Along the process of adding patches to the legal-info output, there was an opportunity to cleanup and enhance the gcc package wrt. legal-info. It was discovered that our legal-info for gcc was (besides the patches) incomplete, as the Xtensa overlay was not saved. So, we took the burden of cleaning, fixing and enhancing the gcc package:
- have the same source tree for initial and final steps,
- only extract sources once,
- make the gcc package pure host packages,
- add licensing information,
- save the Xtensa overlay to the legal-info output.
Still side-tracked with all those changes, we fix the perl package to also save the perl-cross sources in the legal-info putput.
Then, back to the legal-info infra, further changes allow saving the patches alongside the source archives:
- archives are saved in a subdirectory named package-version rarther than all in the same directory,
- the directory where the kicense files are stores are also named similarly, for consistency, rather than with ust the name of the package,
- patches ar eeventually saved alongside the corrsponding archive,
- eventually, legal-info is made to work in full-offline mode.
Notes:
- we considered saving the full content of _ALL_DOWNLOADS or just the content of _EXTRA_DOWNLOADS, but that does not play well with pre-configued toolchains, which use _EXTRA_DOWNLOAD to doenload additional tarballs with more binary stuff in there (sysroots), so it does not make sense to include those in the legal-info output. Since_ALL_DOWNLOADS also contains _EXTRA_DOWNLOADS, we can't use it either. Those are to be handled manually in post-legal-info hooks;
- there might be other packages for which we are not saving 'extra' stuff, but they can be fixed (with post-legal-info hooks) in followup patches.
Regards,
Yann E. MORIN.
The following changes since commit 5f2cd8f5ecd7d035f00664c604fd8bfd5f1afcb8:
lua-periphery: fix build on Sparc/Sparc64 (2015-11-16 23:19:27 +0100)
are available in the git repository at:
git://git.busybox.net/~ymorin/git/buildroot yem/legal-patch
for you to fetch changes up to 0aa743543356cbe19450b672b435f016260ee7d6:
core/pkg-infra: ensure legal-info works in off-line mode (2015-11-16 23:39:03 +0100)
----------------------------------------------------------------
Yann E. MORIN (21):
package/libecore: do not build anything at patch time core/legal-info: fix missing double-$ package/linux-headers: add licensing information core/legal-info: allow ignoring packages from the legal-info core/pkg-virtual: ignore from legal-info output package/gcc: add licensing information
package/gcc: handle licensing information
package/gcc: add a patch for arc rather than using sed package/gcc: properly extract and patch sources package/gcc: make host-gcc a pure host package package/gcc: make -initial and -final use sources from host-gcc package/gcc: include the extensa overlay in the legal-info output package/perl: include perl-cross in the legal-info output
core: add a global variable with already-unquoted global patch dir core/pkg-generic: add macro to hardlink-or-copy core/legal-info: use the macro to install source archives core/legal-info: install source archives in their own sub-dir core/legal-info: add package version to license directory core/apply-patches: store full path of applied patches core/legal-info: also save patches
core/pkg-infra: ensure legal-info works in off-line mode
Makefile | 2 + docs/manual/adding-packages-generic.txt | 7 +- package/efl/libecore/libecore.mk | 2 +- .../900-no-libgmon.a-when-inhibit_libc.patch | 43 +++++++ package/gcc/gcc-final/gcc-final.mk | 40 +++--- package/gcc/gcc-initial/gcc-initial.mk | 43 ++----- package/gcc/gcc.mk | 137 +++++++++++---------- package/linux-headers/linux-headers.mk | 2 + package/perl/perl.mk | 7 ++ package/pkg-generic.mk | 74 ++++++++--- package/pkg-virtual.mk | 2 + support/scripts/apply-patches.sh | 7 +- 12 files changed, 228 insertions(+), 138 deletions(-) create mode 100644 package/gcc/arc-2015.06/900-no-libgmon.a-when-inhibit_libc.patch
Signature · 6 lines
.-----------------.--------------------.------------------.--------------------. | Yann E. MORIN | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: | | +33 662 376 056 | Software Designer | \ / CAMPAIGN | ___ | | +33 223 225 172 `------------.-------: X AGAINST | \e/ There is no | | http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL | v conspiracy. | '------------------------------^-------^------------------^--------------------'
RFC PATCH 1/21 package/libecore: do not build anything at patch time
Trying to build anything at patch time will result in a broken legal-info, as the needed host dependencies are not yet built.
Make that hook a pre-configure hook rather than a post-extract hook.
--- package/efl/libecore/libecore.mk | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
package/efl/libecore/libecore.mk +1 -1
@@ -97,7 +97,7 @@ define LIBECORE_BUILD_MAKEKEYS_FOR_HOST $(HOST_CONFIGURE_OPTS) $(MAKE1) -C $(@D)/src/util makekeys.o makekeys endef -LIBECORE_POST_EXTRACT_HOOKS += LIBECORE_BUILD_MAKEKEYS_FOR_HOST +LIBECORE_PRE_CONFIGURE_HOOKS += LIBECORE_BUILD_MAKEKEYS_FOR_HOST else LIBECORE_CONF_OPTS += --disable-ecore-x-xcb endif
Signature · 1 line
1.9.1
RFC PATCH 1/21 package/libecore: do not build anything at patch time
Dear Yann E. MORIN,
Quoted 9 lines · Yann E. MORIN
Trying to build anything at patch time will result in a broken legal-info, as the needed host dependencies are not yet built.
Make that hook a pre-configure hook rather than a post-extract hook.
Signed-off-by "Yann E. MORIN" <[email protected]>Cc Peter Korsgaard <[email protected]>Cc Romain Naour <[email protected]>
Even though this will probably conflict with the EFL rework from Romain. However I believe this patch should be merged in master, since it actually fixes legal-info.
Thanks,
Thomas
Signature · 3 lines
Thomas Petazzoni, CTO, Free Electrons Embedded Linux, Kernel and Android engineering http://free-electrons.com
RFC PATCH 1/21 package/libecore: do not build anything at patch time
Hello Yann, Thomas,
Le 17/11/2015 12:16, Thomas Petazzoni a ?crit :
Quoted 13 lines
Dear Yann E. MORIN,
Quoted 9 lines · Yann E. MORIN
Trying to build anything at patch time will result in a broken legal-info, as the needed host dependencies are not yet built.
Make that hook a pre-configure hook rather than a post-extract hook.
Signed-off-by "Yann E. MORIN" <[email protected]>Cc Peter Korsgaard <[email protected]>Cc Romain Naour <[email protected]>
Quoted 5 lines
Reviewed-by Thomas Petazzoni <[email protected]>Even though this will probably conflict with the EFL rework from Romain. However I believe this patch should be merged in master, since it actually fixes legal-info.
Ok, it's not a big problem. The merge conflict will be easily resolved by removing the libecore package ;-)
Actually makekeys is not build by the upcoming host-efl package, I'll check if we still need it.
Otherwise:
Best regards,
Romain
Quoted 5 lines
Thanks,
Thomas
RFC PATCH 1/21 package/libecore: do not build anything at patch time
Hello, Yann, Thomas,
Le 17/11/2015 13:20, Romain Naour a ?crit :
Quoted 21 lines
Hello Yann, Thomas,
Le 17/11/2015 12:16, Thomas Petazzoni a ?crit :Quoted 13 lines
Dear Yann E. MORIN,
Quoted 9 lines · Yann E. MORIN
Trying to build anything at patch time will result in a broken legal-info, as the needed host dependencies are not yet built.
Make that hook a pre-configure hook rather than a post-extract hook.
Signed-off-by "Yann E. MORIN" <[email protected]>Cc Peter Korsgaard <[email protected]>Cc Romain Naour <[email protected]>Quoted 5 lines
Reviewed-by Thomas Petazzoni <[email protected]>Even though this will probably conflict with the EFL rework from Romain. However I believe this patch should be merged in master, since it actually fixes legal-info.
Quoted 5 lines
Ok, it's not a big problem. The merge conflict will be easily resolved by removing the libecore package ;-)
Actually makekeys is not build by the upcoming host-efl package, I'll check if we still need it.
We can just forget about makekeys tool since it's only used when X11 with XCB support is enabled.
Best regards,
Romain
RFC PATCH 1/21 package/libecore: do not build anything at patch time
In reply to Yann E. MORINQuoted 11 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
"Yann" == Yann E MORIN <[email protected]> writes:
Trying to build anything at patch time will result in a broken legal-info, as the needed host dependencies are not yet built.
Make that hook a pre-configure hook rather than a post-extract hook.
Signed-off-by "Yann E. MORIN" <[email protected]>Cc Peter Korsgaard <[email protected]>Cc Romain Naour <[email protected]>
Committed, thanks.
Signature · 1 line
Bye, Peter Korsgaard
RFC PATCH 2/21 core/legal-info: fix missing double-$
In reply to Yann E. MORIN--- package/pkg-generic.mk | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
package/pkg-generic.mk +1 -1
@@ -795,7 +795,7 @@ else ifeq ($$($(2)_REDISTRIBUTE),YES) ifneq ($$($(2)_ACTUAL_SOURCE_TARBALL),$$($(2)_SOURCE)) - $(call DOWNLOAD,$$($(2)_ACTUAL_SOURCE_SITE)/$$($(2)_ACTUAL_SOURCE_TARBALL)) + $$(call DOWNLOAD,$$($(2)_ACTUAL_SOURCE_SITE)/$$($(2)_ACTUAL_SOURCE_TARBALL)) endif # Copy the source tarball (just hardlink if possible) @cp -l $$(DL_DIR)/$$($(2)_ACTUAL_SOURCE_TARBALL) $$(REDIST_SOURCES_DIR_$$(call UPPERCASE,$(4))) 2>/dev/null || \
Signature · 1 line
1.9.1
RFC PATCH 2/21 core/legal-info: fix missing double-$
Dear Yann E. MORIN,
Quoted 5 lines · Yann E. MORIN
Signed-off-by "Yann E. MORIN" <[email protected]>Cc Luca Ceresoli <[email protected]>--- package/pkg-generic.mk | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
I tested by running "make legal-info" on a configuration that uses an external toolchain, and it properly downloaded the "actual source" for the toolchain.
This should be applied on master I believe.
Best regards,
Thomas
Signature · 3 lines
Thomas Petazzoni, CTO, Free Electrons Embedded Linux, Kernel and Android engineering http://free-electrons.com
RFC PATCH 2/21 core/legal-info: fix missing double-$
In reply to Yann E. MORINQuoted 4 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
"Yann" == Yann E MORIN <[email protected]> writes:
Signed-off-by "Yann E. MORIN" <[email protected]>Cc Luca Ceresoli <[email protected]>
Committed, thanks.
Signature · 1 line
Bye, Peter Korsgaard
RFC PATCH 3/21 package/linux-headers: add licensing information
In reply to Yann E. MORIN--- package/linux-headers/linux-headers.mk | 2 ++ 1 file changed, 2 insertions(+)
package/linux-headers/linux-headers.mk +2 -0
@@ -16,6 +16,8 @@ else ifeq ($(findstring x4.,x$(LINUX_HEADERS_VERSION)),x4.) LINUX_HEADERS_SITE = $(BR2_KERNEL_MIRROR)/linux/kernel/v4.x endif LINUX_HEADERS_SOURCE = linux-$(LINUX_HEADERS_VERSION).tar.xz +LINUX_HEADERS_LICENSE = GPLv2 +LINUX_HEADERS_LICENSE_FILES = COPYING LINUX_HEADERS_INSTALL_STAGING = YES
Signature · 1 line
1.9.1
RFC PATCH 3/21 package/linux-headers: add licensing information
Dear Yann E. MORIN,
Quoted 5 lines · Yann E. MORIN
Signed-off-by "Yann E. MORIN" <[email protected]>Cc Luca Ceresoli <[email protected]>--- package/linux-headers/linux-headers.mk | 2 ++ 1 file changed, 2 insertions(+)
Should be applied on master.
Thomas
Signature · 3 lines
Thomas Petazzoni, CTO, Free Electrons Embedded Linux, Kernel and Android engineering http://free-electrons.com
RFC PATCH 3/21 package/linux-headers: add licensing information
In reply to Yann E. MORINQuoted 4 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
"Yann" == Yann E MORIN <[email protected]> writes:
Signed-off-by "Yann E. MORIN" <[email protected]>Cc Luca Ceresoli <[email protected]>
Committed, thanks.
Signature · 1 line
Bye, Peter Korsgaard
RFC PATCH 4/21 core/legal-info: allow ignoring packages from the legal-info
In reply to Yann E. MORINIt might be necessary to not even mention a package in the output of legal-info:
- virtual package have virtually nothing to save in the legal-info
output;
- for Buildroot itself, host-gcc-initial and host-gcc-final are not real packages, they are just two different steps of the same package, gcc;
- for proprietary packages, it might not even be legal to even mention them, being under NDA or some other such restrictive conditions.
Add the new 'IGNORE' keyword to the _REDISTRIBUTE package variable, so that the legal-info infra will simply completely ignore that package.
--- docs/manual/adding-packages-generic.txt | 7 +++++-- package/pkg-generic.mk | 8 +++++++- 2 files changed, 12 insertions(+), 3 deletions(-)
docs/manual/adding-packages-generic.txt +5 -2
@@ -390,7 +390,8 @@ information is (assuming the package name is +libfoo+) : See xref:legal-info[] for more information. This variable is optional. If it is not defined, a warning will be produced to let you know, and +not saved+ will appear in the +license files+ field - of the manifest file for this package. + of the manifest file for this package (unless +LIBFOO_REDISTRIBUTE+ is + set to +IGNORE+, see below...) * +LIBFOO_ACTUAL_SOURCE_TARBALL+ only applies to packages whose +LIBFOO_SITE+ / +LIBTOO_SOURCE+ pair points to an archive that does
@@ -414,7 +415,9 @@ information is (assuming the package name is +libfoo+) : * +LIBFOO_REDISTRIBUTE+ can be set to +YES+ (default) or +NO+ to indicate if the package source code is allowed to be redistributed. Set it to +NO+ for non-opensource packages: Buildroot will not save the source code for this - package when collecting the +legal-info+. + package when collecting the +legal-info+. Alternatively, you may set it to + +IGNORE+ so that Buildroot does not even mention that package in the + +legal-info+ output. * +LIBFOO_FLAT_STACKSIZE+ defines the stack size of an application built into the FLAT binary format. The application stack size on the NOMMU architecture
package/pkg-generic.mk +7 -1
@@ -739,10 +739,12 @@ $(2)_MANIFEST_LICENSE_FILES ?= not saved # If the package declares _LICENSE_FILES, we need to extract it, # for overriden, local or normal remote packages alike, whether -# we want to redistribute it or not. +# we want to redistribute it or not, but not if we want to ignore it. +ifneq ($$($(2)_REDISTRIBUTE),IGNORE) ifneq ($$($(2)_LICENSE_FILES),) $(1)-legal-info: $(1)-patch endif +endif # We only save the sources of packages we want to redistribute, that are # non-local, and non-overriden. So only store, in the manifest, the tarball
@@ -766,6 +768,8 @@ $(2)_ACTUAL_SOURCE_SITE ?= $$(call qstrip,$$($(2)_SITE)) # legal-info: produce legally relevant info. $(1)-legal-info: +ifneq ($$($(2)_REDISTRIBUTE),IGNORE) + # Packages without a source are assumed to be part of Buildroot, skip them. $$(foreach hook,$$($(2)_PRE_LEGAL_INFO_HOOKS),$$(call $$(hook))$$(sep)) ifneq ($$(call qstrip,$$($(2)_SOURCE)),)
@@ -807,6 +811,8 @@ endif # other packages endif # ifneq ($$(call qstrip,$$($(2)_SOURCE)),) $$(foreach hook,$$($(2)_POST_LEGAL_INFO_HOOKS),$$(call $$(hook))$$(sep)) +endif # REDISTRIBUTE == IGNORE + # add package to the general list of targets if requested by the buildroot # configuration ifeq ($$($$($(2)_KCONFIG_VAR)),y)
Signature · 1 line
1.9.1
RFC PATCH 4/21 core/legal-info: allow ignoring packages from the legal-info
Yann, Luca,
Quoted 10 lines · Yann E. MORIN
It might be necessary to not even mention a package in the output of legal-info:
- virtual package have virtually nothing to save in the legal-info
output;- for Buildroot itself, host-gcc-initial and host-gcc-final are not real packages, they are just two different steps of the same package, gcc;
Quoted 8 lines
- for proprietary packages, it might not even be legal to even mention them, being under NDA or some other such restrictive conditions.
Add the new 'IGNORE' keyword to the
_REDISTRIBUTEpackage variable, so that the legal-info infra will simply completely ignore that package.Signed-off-by "Yann E. MORIN" <[email protected]>Cc Luca Ceresoli <[email protected]>
I understand the idea, but I'm not a big fan of a boolean variable that is no longer a boolean variable.
So, let me question the current handling of <pkg>_REDISTRIBUTE = NO. Does it make sense to mention such packages in the legal-info output, since their source code is not saved anyway?
Shouldn't we simply change the behavior of <pkg>_REDISTRIBUTE = NO to be that such packages are not listed at all in the legal-info output, which would match what you're looking for for with this "IGNORE" thing ?
Luca, do you see any drawback in completely omitting REDISTRIBUTE = NO packages from legal-info ?
Thanks,
Thomas
Signature · 3 lines
Thomas Petazzoni, CTO, Free Electrons Embedded Linux, Kernel and Android engineering http://free-electrons.com
RFC PATCH 4/21 core/legal-info: allow ignoring packages from the legal-info
Dear Thomas, Yann, All,
Quoted 29 lines · Thomas Petazzoni
Yann, Luca,
Quoted 10 lines · Yann E. MORIN
It might be necessary to not even mention a package in the output of legal-info:
- virtual package have virtually nothing to save in the legal-info
output;- for Buildroot itself, host-gcc-initial and host-gcc-final are not real packages, they are just two different steps of the same package, gcc;
Quoted 8 lines
- for proprietary packages, it might not even be legal to even mention them, being under NDA or some other such restrictive conditions.
Add the new 'IGNORE' keyword to the
_REDISTRIBUTEpackage variable, so that the legal-info infra will simply completely ignore that package.Signed-off-by "Yann E. MORIN" <[email protected]>Cc Luca Ceresoli <[email protected]>I understand the idea, but I'm not a big fan of a boolean variable that is no longer a boolean variable.
So, let me question the current handling of <pkg>_REDISTRIBUTE = NO. Does it make sense to mention such packages in the legal-info output, since their source code is not saved anyway?
Quoted 6 lines
Shouldn't we simply change the behavior of <pkg>_REDISTRIBUTE = NO to be that such packages are not listed at all in the legal-info output, which would match what you're looking for for with this "IGNORE" thing ?
Luca, do you see any drawback in completely omitting REDISTRIBUTE = NO packages from legal-info ?
Indeed I think we need that behaviour for some binary-only packages. imx-vpu looks like a good example. Its COPYING file contains the following clause:
Quoted 3 lines
3.4. You must reproduce any and all of Freescale's (or its third
party licensor's) copyright notices and other proprietary legends on copies of Licensed Software.
To the best of my understanding this means we must mention the package and its legal wording in legal-info.
Thus your proposed change of meaning for REDISTRIBUTE = NO is probably not doable.
How about changing the entire REDISTRIBUTE meaning from a boolean to an enumerative? Values would be:
- ALL: equals current YES
- METADATA: equals current NO, produces no tarball
- NOTHING: equals Yann's IGNORE
- YES: deprecated, for backward compatibility
- NO: deprecated, for backward compatibility
In other words, change the meaning of REDISTRIBUTE from "do you want to redistribute the tarball?" to "what do you want to redistribute?".
Signature · 1 line
Luca
RFC PATCH 4/21 core/legal-info: allow ignoring packages from the legal-info
Luca, Thomas, All,
On 2015-11-17 18:35 +0100, Luca Ceresoli spake thusly:
Quoted 52 lines
Quoted 27 lines · Thomas Petazzoni
Quoted 10 lines · Yann E. MORIN
It might be necessary to not even mention a package in the output of legal-info:
- virtual package have virtually nothing to save in the legal-info
output;- for Buildroot itself, host-gcc-initial and host-gcc-final are not real packages, they are just two different steps of the same package, gcc;
Quoted 8 lines
- for proprietary packages, it might not even be legal to even mention them, being under NDA or some other such restrictive conditions.
Add the new 'IGNORE' keyword to the
_REDISTRIBUTEpackage variable, so that the legal-info infra will simply completely ignore that package.Signed-off-by "Yann E. MORIN" <[email protected]>Cc Luca Ceresoli <[email protected]>I understand the idea, but I'm not a big fan of a boolean variable that is no longer a boolean variable.
So, let me question the current handling of <pkg>_REDISTRIBUTE = NO. Does it make sense to mention such packages in the legal-info output, since their source code is not saved anyway?
Quoted 6 lines
Shouldn't we simply change the behavior of <pkg>_REDISTRIBUTE = NO to be that such packages are not listed at all in the legal-info output, which would match what you're looking for for with this "IGNORE" thing ?
Luca, do you see any drawback in completely omitting REDISTRIBUTE = NO packages from legal-info ?
Indeed I think we need that behaviour for some binary-only packages. imx-vpu looks like a good example. Its COPYING file contains the following clause:
Quoted 3 lines
3.4. You must reproduce any and all of Freescale's (or its third
party licensor's) copyright notices and other proprietary legends on copies of Licensed Software.To the best of my understanding this means we must mention the package and its legal wording in legal-info.
Thus your proposed change of meaning for REDISTRIBUTE = NO is probably not doable.
How about changing the entire REDISTRIBUTE meaning from a boolean to an enumerative? Values would be:
Quoted 8 lines
- ALL: equals current YES
- METADATA: equals current NO, produces no tarball
- NOTHING: equals Yann's IGNORE
- YES: deprecated, for backward compatibility
- NO: deprecated, for backward compatibilityIn other words, change the meaning of REDISTRIBUTE from "do you want to redistribute the tarball?" to "what do you want to redistribute?".
Well, I don't really care whether we call it IGNORE or NOTHING or whatever makes more sense. I'm also fine with deprecating YES and NO to replace them with more meaningful values.
What I care about is having the ability to mark a package for the following conditions:
- redistribute "source" archive, list in the manifest and save licensing files,
- list in the manifest and save licensing files,
- completely omit the package from legal-info.
As Luca explained, there are packages for which we should not redistribute the "source" archive, but which we must list in the manifest. Luca talked about imx-vpu, I was thinking about nvidia-driver.
The new possibility that I would want is to completely ignore the package from the legal-info output, as there might be legal reasons that the mere existence of that package should not be disclosed.
Regards,
Yann E. MORIN.
Signature · 6 lines
.-----------------.--------------------.------------------.--------------------. | Yann E. MORIN | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: | | +33 662 376 056 | Software Designer | \ / CAMPAIGN | ___ | | +33 223 225 172 `------------.-------: X AGAINST | \e/ There is no | | http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL | v conspiracy. | '------------------------------^-------^------------------^--------------------'
RFC PATCH 4/21 core/legal-info: allow ignoring packages from the legal-info
Yann,
Quoted 7 lines · Yann E. MORIN
Well, I don't really care whether we call it IGNORE or NOTHING or whatever makes more sense. I'm also fine with deprecating YES and NO to replace them with more meaningful values.
What I care about is having the ability to mark a package for the following conditions:
Quoted 6 lines
- redistribute "source" archive, list in the manifest and save licensing files,
- list in the manifest and save licensing files,
- completely omit the package from legal-info.
Right, understood.
Quoted 4 lines
As Luca explained, there are packages for which we should not redistribute the "source" archive, but which we must list in the manifest. Luca talked about imx-vpu, I was thinking about nvidia-driver.
Seen the examples, it is more convincing now.
Quoted 3 lines
The new possibility that I would want is to completely ignore the package from the legal-info output, as there might be legal reasons that the mere existence of that package should not be disclosed.
So, I understand the need. Now, I am wondering whether we should have one single variable with "special" values as Luca/you suggest, or two clearly separated boolean variables that control each aspects (distribute the source, include in legal-info). Example:
<pkg>_GENERATE_LEGAL_INFO = YES/NO
<pkg>_REDISTRIBUTE_SOURCE = YES/NO
Of course, if <pkg>_GENERATE_LEGAL_INFO = NO, <pkg>_REDISTRIBUTE_SOURCE is ignored.
This is just a proposal, I'm totally open to better names for the variables.
Best regards,
Thomas
Signature · 3 lines
Thomas Petazzoni, CTO, Free Electrons Embedded Linux, Kernel and Android engineering http://free-electrons.com
RFC PATCH 4/21 core/legal-info: allow ignoring packages from the legal-info
Thomas, All,
On 2015-11-17 21:14 +0100, Thomas Petazzoni spake thusly:
Quoted 33 lines · Yann E. MORIN
Quoted 7 lines
Well, I don't really care whether we call it IGNORE or NOTHING or whatever makes more sense. I'm also fine with deprecating YES and NO to replace them with more meaningful values.
What I care about is having the ability to mark a package for the following conditions:
Quoted 6 lines
- redistribute "source" archive, list in the manifest and save licensing files,
- list in the manifest and save licensing files,
- completely omit the package from legal-info.
Right, understood.
Quoted 4 lines
As Luca explained, there are packages for which we should not redistribute the "source" archive, but which we must list in the manifest. Luca talked about imx-vpu, I was thinking about nvidia-driver.
Seen the examples, it is more convincing now.
Quoted 3 lines
The new possibility that I would want is to completely ignore the package from the legal-info output, as there might be legal reasons that the mere existence of that package should not be disclosed.
So, I understand the need. Now, I am wondering whether we should have one single variable with "special" values as Luca/you suggest, or two clearly separated boolean variables that control each aspects (distribute the source, include in legal-info). Example:
Quoted 8 lines
<pkg>_GENERATE_LEGAL_INFO = YES/NO
<pkg>_REDISTRIBUTE_SOURCE = YES/NOOf course, if <pkg>_GENERATE_LEGAL_INFO = NO, <pkg>_REDISTRIBUTE_SOURCE is ignored.
This is just a proposal, I'm totally open to better names for the variables.
I would prefer we keep a single variable. Especially since we can easily keep backward compatibility.
Regards,
Yann E. MORIN.
Signature · 6 lines
.-----------------.--------------------.------------------.--------------------. | Yann E. MORIN | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: | | +33 662 376 056 | Software Designer | \ / CAMPAIGN | ___ | | +33 223 225 172 `------------.-------: X AGAINST | \e/ There is no | | http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL | v conspiracy. | '------------------------------^-------^------------------^--------------------'
RFC PATCH 4/21 core/legal-info: allow ignoring packages from the legal-info
In reply to Thomas PetazzoniQuoted 25 lines · Thomas Petazzoni
Yann, Luca,
Quoted 10 lines · Yann E. MORIN
It might be necessary to not even mention a package in the output of legal-info:
- virtual package have virtually nothing to save in the legal-info
output;- for Buildroot itself, host-gcc-initial and host-gcc-final are not real packages, they are just two different steps of the same package, gcc;
Quoted 8 lines
- for proprietary packages, it might not even be legal to even mention them, being under NDA or some other such restrictive conditions.
Add the new 'IGNORE' keyword to the
_REDISTRIBUTEpackage variable, so that the legal-info infra will simply completely ignore that package.Signed-off-by "Yann E. MORIN" <[email protected]>Cc Luca Ceresoli <[email protected]>I understand the idea, but I'm not a big fan of a boolean variable that is no longer a boolean variable.
Quoted 7 lines
So, let me question the current handling of <pkg>_REDISTRIBUTE = NO. Does it make sense to mention such packages in the legal-info output, since their source code is not saved anyway?
Shouldn't we simply change the behavior of <pkg>_REDISTRIBUTE = NO to be that such packages are not listed at all in the legal-info output, which would match what you're looking for for with this "IGNORE" thing ?
Well, at least the manifest and license text files are for the legal team, not
for the end users. And I really would like them to be informed about these crazy proprietary licenses that basically state that you're not allowed to use the software.
Regards,
Arnout
Quoted 8 lines
Luca, do you see any drawback in completely omitting REDISTRIBUTE = NO packages from legal-info ?
Thanks,
Thomas
Signature · 6 lines
Arnout Vandecappelle arnout at mind be Senior Embedded Software Architect +32-16-286500 Essensium/Mind http://www.mind.be G.Geenslaan 9, 3001 Leuven, Belgium BE 872 984 063 RPR Leuven LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle GPG fingerprint: 7493 020B C7E3 8618 8DEC 222C 82EB F404 F9AC 0DDF
RFC PATCH 4/21 core/legal-info: allow ignoring packages from the legal-info
In reply to Yann E. MORINQuoted 10 lines · Yann E. MORIN
It might be necessary to not even mention a package in the output of legal-info:
- virtual package have virtually nothing to save in the legal-info
output;- for Buildroot itself, host-gcc-initial and host-gcc-final are not real packages, they are just two different steps of the same package, gcc;
Quoted 8 lines
- for proprietary packages, it might not even be legal to even mention them, being under NDA or some other such restrictive conditions.
Add the new 'IGNORE' keyword to the
_REDISTRIBUTEpackage variable, so that the legal-info infra will simply completely ignore that package.Signed-off-by "Yann E. MORIN" <[email protected]>Cc Luca Ceresoli <[email protected]>
But see below.
Quoted 20 lines
--- docs/manual/adding-packages-generic.txt | 7 +++++-- package/pkg-generic.mk | 8 +++++++- 2 files changed, 12 insertions(+), 3 deletions(-)docs/manual/adding-packages-generic.txt +2
-1@@ -390,7 +390,8 @@ information is (assuming the package name is +libfoo+) : See xref:legal-info[] for more information. This variable is optional. If it is not defined, a warning will be produced to let you know, and +not saved+ will appear in the +license files+ field - of the manifest file for this package. + of the manifest file for this package (unless +LIBFOO_REDISTRIBUTE+ is + set to +IGNORE+, see below...) * +LIBFOO_ACTUAL_SOURCE_TARBALL+ only applies to packages whose +LIBFOO_SITE+ / +LIBTOO_SOURCE+ pair points to an archive that does
docs/manual/adding-packages-generic.txt information is (assuming the package name is +libfoo+) : · line 420
Quoted 7 lines
docs/manual/adding-packages-generic.txt +3
-1@@ -414,7 +415,9 @@ information is (assuming the package name is +libfoo+) : * +LIBFOO_REDISTRIBUTE+ can be set to +YES+ (default) or +NO+ to indicate if the package source code is allowed to be redistributed. Set it to +NO+ for non-opensource packages: Buildroot will not save the source code for this - package when collecting the +legal-info+. + package when collecting the +legal-info+. Alternatively, you may set it to + +IGNORE+ so that Buildroot does not even mention that package in the + +legal-info+ output.
Perhaps add:
This is done automatically for virtual packages.
Regards,
Arnout
Quoted 40 lines
docs/manual/adding-packages-generic.txt +0
-0* +LIBFOO_FLAT_STACKSIZE+ defines the stack size of an application built into the FLAT binary format. The application stack size on the NOMMU architecturepackage/pkg-generic.mk +7
-1@@ -739,10 +739,12 @@ $(2)_MANIFEST_LICENSE_FILES ?= not saved # If the package declares _LICENSE_FILES, we need to extract it, # for overriden, local or normal remote packages alike, whether -# we want to redistribute it or not. +# we want to redistribute it or not, but not if we want to ignore it. +ifneq ($$($(2)_REDISTRIBUTE),IGNORE) ifneq ($$($(2)_LICENSE_FILES),) $(1)-legal-info: $(1)-patch endif +endif # We only save the sources of packages we want to redistribute, that are # non-local, and non-overriden. So only store, in the manifest, the tarball@@ -766,6 +768,8 @@ $(2)_ACTUAL_SOURCE_SITE ?= $$(call qstrip,$$($(2)_SITE)) # legal-info: produce legally relevant info. $(1)-legal-info: +ifneq ($$($(2)_REDISTRIBUTE),IGNORE) + # Packages without a source are assumed to be part of Buildroot, skip them. $$(foreach hook,$$($(2)_PRE_LEGAL_INFO_HOOKS),$$(call $$(hook))$$(sep)) ifneq ($$(call qstrip,$$($(2)_SOURCE)),)@@ -807,6 +811,8 @@ endif # other packages endif # ifneq ($$(call qstrip,$$($(2)_SOURCE)),) $$(foreach hook,$$($(2)_POST_LEGAL_INFO_HOOKS),$$(call $$(hook))$$(sep)) +endif # REDISTRIBUTE == IGNORE + # add package to the general list of targets if requested by the buildroot # configuration ifeq ($$($$($(2)_KCONFIG_VAR)),y)
Signature · 6 lines
Arnout Vandecappelle arnout at mind be Senior Embedded Software Architect +32-16-286500 Essensium/Mind http://www.mind.be G.Geenslaan 9, 3001 Leuven, Belgium BE 872 984 063 RPR Leuven LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle GPG fingerprint: 7493 020B C7E3 8618 8DEC 222C 82EB F404 F9AC 0DDF
RFC PATCH 5/21 core/pkg-virtual: ignore from legal-info output
In reply to Yann E. MORINVirtual packages really have nothing to save for legal-info, so we just ignore them.
Following patches will always extract-and-aptch non-ignored packages, so we would potentially see spurious "extracting" and "patching" messages on the console, which would be a bit strange. Ignoring virtual packages will get us a cleaner log.
--- package/pkg-virtual.mk | 2 ++ 1 file changed, 2 insertions(+)
package/pkg-virtual.mk +2 -0
@@ -57,6 +57,8 @@ endif # Add dependency against the provider $(2)_DEPENDENCIES += $$(call qstrip,$$(BR2_PACKAGE_PROVIDES_$(2))) +$(2)_REDISTRIBUTE = IGNORE + # Call the generic package infrastructure to generate the necessary # make targets $(call inner-generic-package,$(1),$(2),$(3),$(4))
Signature · 1 line
1.9.1
RFC PATCH 5/21 core/pkg-virtual: ignore from legal-info output
Yann,
Quoted 8 lines · Yann E. MORIN
Virtual packages really have nothing to save for legal-info, so we just ignore them.
Following patches will always extract-and-aptch non-ignored packages, so we would potentially see spurious "extracting" and "patching" messages on the console, which would be a bit strange. Ignoring virtual packages will get us a cleaner log.
aptch -> patch. And the extract-and-patch can be just "extract and patch".
It's also not really clear what you mean by "Following patches will always extract and patch non-ignored packages". Aren't we already doing that for legal-info ? (Note: I haven't looked at the following patches yet, things might get clearer when I'll reach the next patches, but it seems weird to not understand a commit log if you haven't read the next patches).
Otherwise, I'm fine with the change, as soon as we reach an agreement on PATCH 04/21.
Thanks,
Thomas
Signature · 3 lines
Thomas Petazzoni, CTO, Free Electrons Embedded Linux, Kernel and Android engineering http://free-electrons.com
RFC PATCH 5/21 core/pkg-virtual: ignore from legal-info output
On 2015-11-17 12:25 +0100, Thomas Petazzoni spake thusly:
Quoted 7 lines
Yann,
Quoted 3 lines · Yann E. MORIN
Virtual packages really have nothing to save for legal-info, so we just ignore them.
Quoted 7 lines
Quoted 4 lines
Following patches will always extract-and-aptch non-ignored packages, so we would potentially see spurious "extracting" and "patching" messages on the console, which would be a bit strange. Ignoring virtual packages will get us a cleaner log.
aptch -> patch. And the extract-and-patch can be just "extract and patch".
Damn, I missed fixing that (I used it elsewhere and then fixed those, missed that one).
Quoted 6 lines
It's also not really clear what you mean by "Following patches will always extract and patch non-ignored packages". Aren't we already doing that for legal-info ? (Note: I haven't looked at the following patches yet, things might get clearer when I'll reach the next patches, but it seems weird to not understand a commit log if you haven't read the next patches).
No, we only extract them if they define a non-empty _LICENSE_FILES.
Quoted 2 lines
Otherwise, I'm fine with the change, as soon as we reach an agreement on PATCH 04/21.
Ack. Two options, then:
- keep YES/NO, add IGNORE (or NOTHING)
- replace YES/NO with ALL, METADATA, NOTHING (and keep YES/NO for backward compatibility to ALL/METADATA).
(or a third naming, whatever...)
Regards,
Yann E. MORIN.
Signature · 6 lines
.-----------------.--------------------.------------------.--------------------. | Yann E. MORIN | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: | | +33 662 376 056 | Software Designer | \ / CAMPAIGN | ___ | | +33 223 225 172 `------------.-------: X AGAINST | \e/ There is no | | http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL | v conspiracy. | '------------------------------^-------^------------------^--------------------'
RFC PATCH 6/21 package/gcc: add licensing information
In reply to Yann E. MORINCurrently, this is not used, since there is no host-gcc (or gcc) package, and we only build host-gcc-initial and host-gcc-final.
So, this commit only really adds the licensing terms, installing them in the legal-info output will be handled in a futur commit.
--- package/gcc/gcc.mk | 9 +++++++++ 1 file changed, 9 insertions(+)
package/gcc/gcc.mk +9 -0
@@ -9,6 +9,15 @@ # GCC_VERSION = $(call qstrip,$(BR2_GCC_VERSION)) +GCC_LICENSE = \ + GPLv2+, GPLv3+, LGPLv2.1+ (w/ exception), LGPv3+ (w/ exception), \ + BSD-3c, MIT, BSL-1.0 +GCC_LICENSE_FILES = \ + COPYING COPYING3 COPYING.LIB COPYING3.LIB COPYING.RUNTIME \ + libiberty/COPYING.LIB gcc/go/gofrontend/LICENSE \ + gcc/COPYING gcc/COPYING.LIB gcc/COPYING3 gcc/COPYING3.LIB \ + libquadmath/COPYING.LIB libsanitizer/LICENSE.TXT libffi/LICENSE \ + include/COPYING include/COPYING3 zlib/contrib/dotzlib/LICENSE_1_0.txt ifeq ($(BR2_arc),y) GCC_SITE = $(call github,foss-for-synopsys-dwc-arc-processors,gcc,$(GCC_VERSION))
Signature · 1 line
1.9.1
RFC PATCH 6/21 package/gcc: add licensing information
Yann,
package/gcc/gcc.mk line 14
Quoted 4 lines · Yann E. MORIN
GCC_VERSION= $(call qstrip,$(BR2_GCC_VERSION))package/gcc/gcc.mk +3
-0+GCC_LICENSE = \ + GPLv2+, GPLv3+, LGPLv2.1+ (w/ exception), LGPv3+ (w/ exception), \ + BSD-3c, MIT, BSL-1.0
What I dislike a bit with this is that we don't clearly differentiate what goes on the target with what remains on the host. For many companies, having GPLv3 or LGPLv3 stuff on the target is (unfortunately) a no-go, so it would be good to make this clearer.
As per your implementation, such license information will only be part of the "host packages" license information, while libgcc gets actually installed in the target. To be honest, I am not entirely sure how to handle this properly with our package infrastructure.
Best regards,
Thomas
Signature · 3 lines
Thomas Petazzoni, CTO, Free Electrons Embedded Linux, Kernel and Android engineering http://free-electrons.com
RFC PATCH 6/21 package/gcc: add licensing information
Thomas, All,
On 2015-11-17 12:28 +0100, Thomas Petazzoni spake thusly:
Quoted 11 lines · Yann E. MORIN
package/gcc/gcc.mk line 14
Quoted 4 lines
GCC_VERSION= $(call qstrip,$(BR2_GCC_VERSION))package/gcc/gcc.mk +3
-0+GCC_LICENSE = \ + GPLv2+, GPLv3+, LGPLv2.1+ (w/ exception), LGPv3+ (w/ exception), \ + BSD-3c, MIT, BSL-1.0What I dislike a bit with this is that we don't clearly differentiate what goes on the target with what remains on the host. For many companies, having GPLv3 or LGPLv3 stuff on the target is (unfortunately) a no-go, so it would be good to make this clearer.
Quoted 4 lines
As per your implementation, such license information will only be part of the "host packages" license information, while libgcc gets actually installed in the target. To be honest, I am not entirely sure how to handle this properly with our package infrastructure.
Yes, the current legalese will only apply to the host variant.
We could add a target gcc-final that defines the needed legal-info stuff, but installs nothing (like I did so far with the host-gcc package). But I don't like it much either.
Regards,
Yann E. MORIN.
Signature · 6 lines
.-----------------.--------------------.------------------.--------------------. | Yann E. MORIN | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: | | +33 662 376 056 | Software Designer | \ / CAMPAIGN | ___ | | +33 223 225 172 `------------.-------: X AGAINST | \e/ There is no | | http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL | v conspiracy. | '------------------------------^-------^------------------^--------------------'
RFC PATCH 6/21 package/gcc: add licensing information
Hello,
Quoted 10 lines · Yann E. MORIN
Quoted 4 lines
As per your implementation, such license information will only be part of the "host packages" license information, while libgcc gets actually installed in the target. To be honest, I am not entirely sure how to handle this properly with our package infrastructure.
Yes, the current legalese will only apply to the host variant.
We could add a target gcc-final that defines the needed legal-info stuff, but installs nothing (like I did so far with the host-gcc package). But I don't like it much either.
It's not super nice indeed, but I don't really see a better option. We could always make this package actually do something "useful":
installing the gcc libraries to the target. I.e essentially move the following part of host-gcc-final to this package:
# Handle the installation of libraries in /usr/lib HOST_GCC_FINAL_USR_LIBS =
ifeq ($(BR2_INSTALL_LIBSTDCPP),y)HOST_GCC_FINAL_USR_LIBS += libstdc++
endif
ifeq ($(BR2_TOOLCHAIN_BUILDROOT_FORTRAN),y)HOST_GCC_FINAL_USR_LIBS += libgfortran
endif
ifeq ($(BR2_GCC_ENABLE_OPENMP),y)HOST_GCC_FINAL_USR_LIBS += libgomp
endif
ifeq ($(BR2_GCC_ENABLE_LIBMUDFLAP),y)
ifeq ($(BR2_TOOLCHAIN_HAS_THREADS),y)HOST_GCC_FINAL_USR_LIBS += libmudflapth
elseHOST_GCC_FINAL_USR_LIBS += libmudflap
endif
endif
ifneq ($(HOST_GCC_FINAL_USR_LIBS),)
define HOST_GCC_FINAL_INSTALL_STATIC_LIBS
for i in $(HOST_GCC_FINAL_USR_LIBS) ; do \
cp -dpf $(HOST_GCC_FINAL_GCC_LIB_DIR)/$${i}.a \
$(STAGING_DIR)/usr/lib/ ; \
doneendef
ifeq ($(BR2_STATIC_LIBS),)
define HOST_GCC_FINAL_INSTALL_SHARED_LIBS
for i in $(HOST_GCC_FINAL_USR_LIBS) ; do \
cp -dpf $(HOST_GCC_FINAL_GCC_LIB_DIR)/$${i}.so* \
$(STAGING_DIR)/usr/lib/ ; \
cp -dpf $(HOST_GCC_FINAL_GCC_LIB_DIR)/$${i}.so* \
$(TARGET_DIR)/usr/lib/ ; \
doneendef
endif
define HOST_GCC_FINAL_INSTALL_USR_LIBS
mkdir -p $(TARGET_DIR)/usr/lib
$(HOST_GCC_FINAL_INSTALL_STATIC_LIBS)
$(HOST_GCC_FINAL_INSTALL_SHARED_LIBS)endefHOST_GCC_FINAL_POST_INSTALL_HOOKS += HOST_GCC_FINAL_INSTALL_USR_LIBS endif
This is really making the code more complicated just for the sake of legal-info, though. Which isn't super nice. But at least we would have the gcc libraries properly mentioned in the legal-info output for the target.
Thomas
Signature · 3 lines
Thomas Petazzoni, CTO, Free Electrons Embedded Linux, Kernel and Android engineering http://free-electrons.com
RFC PATCH 6/21 package/gcc: add licensing information
Thomas, All,
On 2015-11-17 22:25 +0100, Thomas Petazzoni spake thusly:
Quoted 8 lines · Yann E. MORIN
Quoted 7 lines
Quoted 4 lines
As per your implementation, such license information will only be part of the "host packages" license information, while libgcc gets actually installed in the target. To be honest, I am not entirely sure how to handle this properly with our package infrastructure.
Yes, the current legalese will only apply to the host variant.
Quoted 8 lines
Quoted 3 lines
We could add a target gcc-final that defines the needed legal-info stuff, but installs nothing (like I did so far with the host-gcc package). But I don't like it much either.
It's not super nice indeed, but I don't really see a better option. We could always make this package actually do something "useful":
installing the gcc libraries to the target. I.e essentially move the following part of host-gcc-final to this package:
Yes, I like that idea. Even if that makes the package a bit more complex, it helps in separating the host/target/staging stuff (like what we already discussed about hte extrenal toolchain, remember?).
[--SNIP--]
Quoted 4 lines
This is really making the code more complicated just for the sake of legal-info, though. Which isn't super nice. But at least we would have the gcc libraries properly mentioned in the legal-info output for the target.
Yes, and that's a very good reason for doing it.
I'll send patches and we can discuss further with actual code.
Thanks! :-)
Regards,
Yann E. MORIN.
Signature · 6 lines
.-----------------.--------------------.------------------.--------------------. | Yann E. MORIN | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: | | +33 662 376 056 | Software Designer | \ / CAMPAIGN | ___ | | +33 223 225 172 `------------.-------: X AGAINST | \e/ There is no | | http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL | v conspiracy. | '------------------------------^-------^------------------^--------------------'
RFC PATCH 7/21 package/gcc: handle licensing information
In reply to Yann E. MORINWe currently install no licensing information for gcc. But the way we build gcc makes it tricky to add licensing information:
- we do not build host-gcc; rather, we build host-gcc-initial and host-gcc-final,
- we do not want to save licensing information for host-gcc-initial and host-gcc-final, because what really makes sense from a user's point of view is really host-gcc.
So, we make gcc a real host-generic-package for which we do not define any configure, build or install command, but for which we do have licensing information.
Then for both host-gcc-initial and host-gcc-final, we fake them to be non-redistributable, because the licensing information for them is not needed, nor is it needed to save their archives.
Finally, we make both host-gcc-initial and host-gc-final depend on host-gcc, so that host-gcc is handled by the legal-info framework.
Thus, legal-info will now save the gcc source and licensing information, and ignore host-gcc-initial and host-gcc-final.
--- package/gcc/gcc-final/gcc-final.mk | 6 ++++++ package/gcc/gcc-initial/gcc-initial.mk | 6 ++++++ package/gcc/gcc.mk | 5 +++++ 3 files changed, 17 insertions(+)
package/gcc/gcc-final/gcc-final.mk +6 -0
@@ -8,6 +8,12 @@ GCC_FINAL_VERSION = $(GCC_VERSION) GCC_FINAL_SITE = $(GCC_SITE) GCC_FINAL_SOURCE = $(GCC_SOURCE) +# We only need to redistribute the gcc sources once, and that is handled by +# the host-gcc package, not host-gcc-final. So, we fake a non-redistribuale +# package, even though it *is* redistributable, but is already redistributed +# thanks to host-gcc. +GCC_FINAL_REDISTRIBUTE = IGNORE + HOST_GCC_FINAL_DEPENDENCIES = \ $(HOST_GCC_COMMON_DEPENDENCIES) \ $(BR_LIBC)
package/gcc/gcc-initial/gcc-initial.mk +6 -0
@@ -8,6 +8,12 @@ GCC_INITIAL_VERSION = $(GCC_VERSION) GCC_INITIAL_SITE = $(GCC_SITE) GCC_INITIAL_SOURCE = $(GCC_SOURCE) +# We only need to redistribute the gcc sources once, and that is handled by +# the host-gcc package, not host-gcc-initial. So, we fake a non-redistribuale +# package, even though it *is* redistributable, but is already redistributed +# thanks to host-gcc. +GCC_INITIAL_REDISTRIBUTE = IGNORE + HOST_GCC_INITIAL_DEPENDENCIES = $(HOST_GCC_COMMON_DEPENDENCIES) HOST_GCC_INITIAL_EXCLUDES = $(HOST_GCC_EXCLUDES)
package/gcc/gcc.mk +5 -0
@@ -81,6 +81,7 @@ endef # HOST_GCC_COMMON_DEPENDENCIES = \ + host-gcc \ host-binutils \ host-gmp \ host-mpfr \
@@ -310,4 +311,8 @@ define HOST_GCC_INSTALL_WRAPPER_AND_SIMPLE_SYMLINKS endef +$(eval $(host-generic-package)) + +# We can include other files, now that we've called the package +# infrastructure, above. include $(sort $(wildcard package/gcc/*/*.mk))
Signature · 1 line
1.9.1
RFC PATCH 7/21 package/gcc: handle licensing information
Yann,
Quoted 21 lines · Yann E. MORIN
We currently install no licensing information for gcc. But the way we build gcc makes it tricky to add licensing information:
- we do not build host-gcc; rather, we build host-gcc-initial and host-gcc-final,
- we do not want to save licensing information for host-gcc-initial and host-gcc-final, because what really makes sense from a user's point of view is really host-gcc.
So, we make gcc a real host-generic-package for which we do not define any configure, build or install command, but for which we do have licensing information.
Then for both host-gcc-initial and host-gcc-final, we fake them to be non-redistributable, because the licensing information for them is not needed, nor is it needed to save their archives.
Finally, we make both host-gcc-initial and host-gc-final depend on host-gcc, so that host-gcc is handled by the legal-info framework.
Quoted 6 lines
Thus, legal-info will now save the gcc source and licensing information, and ignore host-gcc-initial and host-gcc-final.
Signed-off-by "Yann E. MORIN" <[email protected]>Cc Luca Ceresoli <[email protected]>Cc Thomas Petazzoni <[email protected]>
This I dislike very much. host-gcc is not a package, and it doesn't exist. package/gcc/gcc.mk just contains common definitions to both gcc-initial and gcc-final. Making 'gcc' a package is more confusing than anything else.
I believe a much better approach is to simply add the licensing informations to the gcc-final package, and leave gcc-initial with no licensing information. If having a package named "gcc-final" in the licensing report is really shocking for you, then we can rename gcc-final to just "gcc", and have something like:
+ package/
+ gcc/
+ gcc-common.mk
+ gcc-initial/
+ gcc-initial.mk
+ gcc/
+ gcc.mk
Best regards,
Thomas
Signature · 3 lines
Thomas Petazzoni, CTO, Free Electrons Embedded Linux, Kernel and Android engineering http://free-electrons.com
RFC PATCH 7/21 package/gcc: handle licensing information
Thomas, All,
On 2015-11-17 12:31 +0100, Thomas Petazzoni spake thusly:
Quoted 25 lines · Yann E. MORIN
Quoted 24 lines
We currently install no licensing information for gcc. But the way we build gcc makes it tricky to add licensing information:
- we do not build host-gcc; rather, we build host-gcc-initial and host-gcc-final,
- we do not want to save licensing information for host-gcc-initial and host-gcc-final, because what really makes sense from a user's point of view is really host-gcc.
So, we make gcc a real host-generic-package for which we do not define any configure, build or install command, but for which we do have licensing information.
Then for both host-gcc-initial and host-gcc-final, we fake them to be non-redistributable, because the licensing information for them is not needed, nor is it needed to save their archives.
Finally, we make both host-gcc-initial and host-gc-final depend on host-gcc, so that host-gcc is handled by the legal-info framework.
Thus, legal-info will now save the gcc source and licensing information, and ignore host-gcc-initial and host-gcc-final.
Quoted 8 lines
Quoted 3 lines
Signed-off-by "Yann E. MORIN" <[email protected]>Cc Luca Ceresoli <[email protected]>Cc Thomas Petazzoni <[email protected]>This I dislike very much. host-gcc is not a package, and it doesn't exist.
package/gcc/gcc.mkjust contains common definitions to both gcc-initial and gcc-final. Making 'gcc' a package is more confusing than anything else.
OK.
Quoted 6 lines
I believe a much better approach is to simply add the licensing informations to the gcc-final package, and leave gcc-initial with no licensing information. If having a package named "gcc-final" in the licensing report is really shocking for you, then we can rename gcc-final to just "gcc", and have something like:
Quoted 7 lines
+ package/
+ gcc/
+ gcc-common.mk
+ gcc-initial/
+ gcc-initial.mk
+ gcc/
+ gcc.mk
I have to admit this looks good to me. So, I'll be working with that new goal in mind. Thanks! :-)
Regards,
Yann E. MORIN.
Signature · 6 lines
.-----------------.--------------------.------------------.--------------------. | Yann E. MORIN | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: | | +33 662 376 056 | Software Designer | \ / CAMPAIGN | ___ | | +33 223 225 172 `------------.-------: X AGAINST | \e/ There is no | | http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL | v conspiracy. | '------------------------------^-------^------------------^--------------------'
RFC PATCH 8/21 package/gcc: add a patch for arc rather than using sed
In reply to Yann E. MORINThat way, we get the same sources for the -initial and -final passes, which will ultimately allow us not to extract gcc for the two passes and directly use the sources extracted for host-gcc.
--- .../900-no-libgmon.a-when-inhibit_libc.patch | 43 ++++++++++++++++++++++ package/gcc/gcc-initial/gcc-initial.mk | 13 ------- 2 files changed, 43 insertions(+), 13 deletions(-) create mode 100644 package/gcc/arc-2015.06/900-no-libgmon.a-when-inhibit_libc.patch
package/gcc/arc-2015.06/900-no-libgmon.a-when-inhibit_libc.patch new file +43 -0
@@ -0,0 +1,43 @@ +libgcc: on ARC, do not build libgmon.a when inhibit_libc + +gcc on ARC has a bug: in its libgcc, even when no C library is available +(--with-newlib is passed, and therefore inhibit_libc is defined), it +tries to use the C library for the libgmon library. + +Only add it to the list of extra_parts if inhibit_libc is not true. + +inhibit_libc is not available to libgcc, so we use a dirty hack to get +its value from the gcc/config.log. This is not really nice, but is not +much worse than the sed invocation we previously had to tweak that file. + +Signed-off-by: "Yann E. MORIN" <[email protected]> + +diff -durN host-gcc-arc-2015.06.orig/libgcc/config.host host-gcc-arc-2015.06/libgcc/config.host +--- host-gcc-arc-2015.06.orig/libgcc/config.host 2015-11-15 23:01:57.588638300 +0100 ++++ host-gcc-arc-2015.06/libgcc/config.host 2015-11-15 23:07:25.256450515 +0100 +@@ -75,6 +75,9 @@ + md_unwind_header=no-unwind.h + unwind_header=unwind-generic.h + ++# Hack to know whether we need to set inhibit_libc (to exclude libgmon.a) ++eval $(grep -E "^inhibit_libc='true'\$" ../../gcc/config.log 2>&1) ++ + # Set default cpu_type so it can be updated in each machine entry. + cpu_type=`echo ${host} | sed 's/-.*$//'` + case ${host} in +diff -durN host-gcc-arc-2015.06.orig/libgcc/config.host host-gcc-arc-2015.06/libgcc/config.host +--- host-gcc-arc-2015.06.orig/libgcc/config.host 2015-06-26 12:34:27.000000000 +0200 ++++ host-gcc-arc-2015.06/libgcc/config.host 2015-11-15 19:47:27.384690564 +0100 +@@ -327,7 +327,11 @@ + ;; + arc*-*-linux-uclibc*) + tmake_file="${tmake_file} t-slibgcc-libgcc t-slibgcc-nolc-override arc/t-arc700-uClibc arc/t-arc" +- extra_parts="crti.o crtn.o crtend.o crtbegin.o crtendS.o crtbeginS.o libgmon.a crtg.o crtgend.o" ++ extra_parts="crti.o crtn.o crtend.o crtbegin.o crtendS.o crtbeginS.o" ++ if [ ! "${inhibit_libc}" = "true" ]; then ++ extra_parts="${extra_parts} libgmon.a" ++ fi ++ extra_parts="${extra_parts} crtg.o crtgend.o" + ;; + arm-wrs-vxworks) + tmake_file="$tmake_file arm/t-arm arm/t-vxworks t-softfp-sfdf t-softfp-excl arm/t-softfp t-softfp"
package/gcc/gcc-initial/gcc-initial.mk +0 -13
@@ -31,19 +31,6 @@ HOST_GCC_INITIAL_SUBDIR = build HOST_GCC_INITIAL_PRE_CONFIGURE_HOOKS += HOST_GCC_CONFIGURE_SYMLINK -# gcc on ARC has a bug: in its libgcc, even when no C library is -# available (--with-newlib is passed, and therefore inhibit_libc is -# defined), it tries to use the C library for the libgmon -# library. Since it's not needed in gcc-initial, we disabled it here. -ifeq ($(BR2_GCC_VERSION_4_8_ARC),y) -define HOST_GCC_INITIAL_DISABLE_LIBGMON - $(SED) 's/crtbeginS.o libgmon.a crtg.o/crtbeginS.o crtg.o/' \ - $(@D)/libgcc/config.host -endef -HOST_GCC_INITIAL_POST_PATCH_HOOKS += HOST_GCC_INITIAL_DISABLE_LIBGMON -HOST_GCC_INITIAL_POST_RSYNC_HOOKS += HOST_GCC_INITIAL_DISABLE_LIBGMON -endif - HOST_GCC_INITIAL_CONF_OPTS = \ $(HOST_GCC_COMMON_CONF_OPTS) \ --enable-languages=c \
Signature · 1 line
1.9.1
RFC PATCH 9/21 package/gcc: properly extract and patch sources
In reply to Yann E. MORINCurrently, we're extracting the gcc sources three times, which takes a bit of place:
- for host-gcc, just for the sake of legal-info,
- for host-gcc-initial,
- for host-gcc-final.
However, gcc does not support in-tree builds, and for host-gcc-initial as well as for host-gcc-final, wiile we do not build anything for host-gcc we build in a sub-directory, which leaves the three gcc source trees intact.
Thus, we could re-use the sources from host-gcc for the two builds, but they are so far not completely patched. Most notably, the special Xtensa tarball is not extracted, and the powerpc patch is not applied when needed.
Re-organise the way the host-gcc package is defined, so that the source tree is properly prepared. We need to move blocks around, so that hooks are registered in the correct order; as a reference, we use the ordering as seen in host-gcc-initial and host-gcc-final (same order in both).
The "standard" patches are applied by the package infra, so we add a post-patch hook to apply the powerpc patch.
The list of exclusions is defined earlier, as it looks more logical to have all extract-related defintions come before the patch-related ones.
Ditto for the testsuite.
We for now keep the HOST_GCC_APPLY_PATCHES and HOST_GCC_CONFIGURE_SYMLINK macros, as host-gcc-initial and host-gcc-final have not yet been converted. Once that's been done, we can get rid of the macros.
--- package/gcc/gcc.mk | 32 +++++++++++++++----------------- 1 file changed, 15 insertions(+), 17 deletions(-)
package/gcc/gcc.mk +15 -17
@@ -24,29 +24,37 @@ GCC_SITE = $(call github,foss-for-synopsys-dwc-arc-processors,gcc,$(GCC_VERSION) GCC_SOURCE = gcc-$(GCC_VERSION).tar.gz else GCC_SITE = $(BR2_GNU_MIRROR:/=)/gcc/gcc-$(GCC_VERSION) +GCC_SOURCE = gcc-$(GCC_VERSION).tar.bz2 endif -GCC_SOURCE ?= gcc-$(GCC_VERSION).tar.bz2 +HOST_GCC_EXCLUDES = \ + libjava/* libgo/* \ + gcc/testsuite/* libstdc++-v3/testsuite/* + +define HOST_GCC_FAKE_TESTSUITE + mkdir -p $(@D)/libstdc++-v3/testsuite/ + echo "all:" > $(@D)/libstdc++-v3/testsuite/Makefile.in + echo "install:" >> $(@D)/libstdc++-v3/testsuite/Makefile.in +endef +HOST_GCC_POST_EXTRACT_HOOKS += HOST_GCC_FAKE_TESTSUITE # # Xtensa special hook # - +ifneq ($(call qstrip, $(BR2_XTENSA_CORE_NAME)),) HOST_GCC_XTENSA_OVERLAY_TAR = $(BR2_XTENSA_OVERLAY_DIR)/xtensa_$(call qstrip,$(BR2_XTENSA_CORE_NAME)).tar - define HOST_GCC_XTENSA_OVERLAY_EXTRACT tar xf $(HOST_GCC_XTENSA_OVERLAY_TAR) -C $(@D) --strip-components=1 gcc endef - -# -# Apply patches -# +HOST_GCC_POST_EXTRACT_HOOKS += HOST_GCC_XTENSA_OVERLAY_EXTRACT +endif # Xtensa ifeq ($(ARCH),powerpc) ifneq ($(BR2_SOFT_FLOAT),) define HOST_GCC_APPLY_POWERPC_PATCH $(APPLY_PATCHES) $(@D) package/gcc/$(GCC_VERSION) 1000-powerpc-link-with-math-lib.patch.conditional endef +HOST_GCC_POST_PATCH_HOOKS += HOST_GCC_APPLY_POWERPC_PATCH endif endif
@@ -57,16 +65,6 @@ define HOST_GCC_APPLY_PATCHES $(HOST_GCC_APPLY_POWERPC_PATCH) endef -HOST_GCC_EXCLUDES = \ - libjava/* libgo/* \ - gcc/testsuite/* libstdc++-v3/testsuite/* - -define HOST_GCC_FAKE_TESTSUITE - mkdir -p $(@D)/libstdc++-v3/testsuite/ - echo "all:" > $(@D)/libstdc++-v3/testsuite/Makefile.in - echo "install:" >> $(@D)/libstdc++-v3/testsuite/Makefile.in -endef - # # Create 'build' directory and configure symlink #
Signature · 1 line
1.9.1
RFC PATCH 10/21 package/gcc: make host-gcc a pure host package
In reply to Yann E. MORINCurrently, we define a mix of GCC_ and HOST_GCC_ variables, even though we only have a host variant of the gcc package.
Prefix all variables with HOST_ since gcc is a pure host package.
--- package/gcc/gcc-final/gcc-final.mk | 6 ++-- package/gcc/gcc-initial/gcc-initial.mk | 6 ++-- package/gcc/gcc.mk | 64 +++++++++++++++++----------------- 3 files changed, 38 insertions(+), 38 deletions(-)
package/gcc/gcc-final/gcc-final.mk +3 -3
@@ -4,9 +4,9 @@ # ################################################################################ -GCC_FINAL_VERSION = $(GCC_VERSION) -GCC_FINAL_SITE = $(GCC_SITE) -GCC_FINAL_SOURCE = $(GCC_SOURCE) +GCC_FINAL_VERSION = $(HOST_GCC_VERSION) +GCC_FINAL_SITE = $(HOST_GCC_SITE) +GCC_FINAL_SOURCE = $(HOST_GCC_SOURCE) # We only need to redistribute the gcc sources once, and that is handled by # the host-gcc package, not host-gcc-final. So, we fake a non-redistribuale
package/gcc/gcc-initial/gcc-initial.mk +3 -3
@@ -4,9 +4,9 @@ # ################################################################################ -GCC_INITIAL_VERSION = $(GCC_VERSION) -GCC_INITIAL_SITE = $(GCC_SITE) -GCC_INITIAL_SOURCE = $(GCC_SOURCE) +GCC_INITIAL_VERSION = $(HOST_GCC_VERSION) +GCC_INITIAL_SITE = $(HOST_GCC_SITE) +GCC_INITIAL_SOURCE = $(HOST_GCC_SOURCE) # We only need to redistribute the gcc sources once, and that is handled by # the host-gcc package, not host-gcc-initial. So, we fake a non-redistribuale
package/gcc/gcc.mk +32 -32
136 lines not on this page · raw message
Signature · 1 line
1.9.1
RFC PATCH 11/21 package/gcc: make -initial and -final use sources from host-gcc
In reply to Yann E. MORINgcc does not build in-tree. We currently catter to this situation by building gcc in a sub-dir. This virtually leaves the gcc sources intact.
So, we have three locations where we have identical gcc sources:
- host-gcc (for legal-info),
- host-gcc-initial,
- host-gcc-final.
The gcc sources are not so large, but they still take more than 350Mib (so more than 1GiB total) and it takes quite some time to extract them.
This patch:
- changes both host-gcc-initial and host-gc-final to re-use the
sources from host-gcc;
- makes host-gcc-initial and host-gcc-final pure host packages, by prefixing all variables with HOST_ (we really do not have the target variants since we do not support a compiler on the target); not defining HOST_GCC_INITIAL_SOURCE will make Buildroot try to download a taball for it, which we do not want; ditto HOST_GCC_FINAL_SOURCE;
- no longer build in a sub-directory, since we no longer have gcc
sources in the build directory;
- changes the configure symlink to a wrapper script, otherwise configure gets really confused as it only expects the source tree to be in either '.' or '..' ;
- ditches HOST_GCC_APPLY_PATCHES now that it is unused.
--- package/gcc/gcc-final/gcc-final.mk | 36 ++++++++++++---------------------- package/gcc/gcc-initial/gcc-initial.mk | 26 +++++++----------------- package/gcc/gcc.mk | 21 +++++++------------- 3 files changed, 26 insertions(+), 57 deletions(-)
package/gcc/gcc-final/gcc-final.mk +12 -24
@@ -1,37 +1,25 @@ ################################################################################ # -# gcc-final +# host-gcc-final # ################################################################################ -GCC_FINAL_VERSION = $(HOST_GCC_VERSION) -GCC_FINAL_SITE = $(HOST_GCC_SITE) -GCC_FINAL_SOURCE = $(HOST_GCC_SOURCE) +HOST_GCC_FINAL_VERSION = $(HOST_GCC_VERSION) +HOST_GCC_FINAL_SOURCE = # We only need to redistribute the gcc sources once, and that is handled by # the host-gcc package, not host-gcc-final. So, we fake a non-redistribuale # package, even though it *is* redistributable, but is already redistributed # thanks to host-gcc. -GCC_FINAL_REDISTRIBUTE = IGNORE +HOST_GCC_FINAL_REDISTRIBUTE = IGNORE HOST_GCC_FINAL_DEPENDENCIES = \ $(HOST_GCC_COMMON_DEPENDENCIES) \ $(BR_LIBC) -HOST_GCC_FINAL_EXCLUDES = $(HOST_GCC_EXCLUDES) -HOST_GCC_FINAL_POST_EXTRACT_HOOKS += HOST_GCC_FAKE_TESTSUITE - -ifneq ($(call qstrip, $(BR2_XTENSA_CORE_NAME)),) -HOST_GCC_FINAL_POST_EXTRACT_HOOKS += HOST_GCC_XTENSA_OVERLAY_EXTRACT -endif - -HOST_GCC_FINAL_POST_PATCH_HOOKS += HOST_GCC_APPLY_PATCHES - -# gcc doesn't support in-tree build, so we create a 'build' -# subdirectory in the gcc sources, and build from there. -HOST_GCC_FINAL_SUBDIR = build - -HOST_GCC_FINAL_PRE_CONFIGURE_HOOKS += HOST_GCC_CONFIGURE_SYMLINK +# gcc doesn't support in-tree build, so we re-use the sources extracted +# # for the host-gcc package and do a complete out-of-tree build +HOST_GCC_FINAL_POST_PATCH_HOOKS += HOST_GCC_CONFIGURE_WRAPPER define HOST_GCC_FINAL_CONFIGURE_CMDS (cd $(HOST_GCC_FINAL_SRCDIR) && rm -rf config.cache; \
@@ -49,14 +37,14 @@ endef # Languages supported by the cross-compiler -GCC_FINAL_CROSS_LANGUAGES-y = c -GCC_FINAL_CROSS_LANGUAGES-$(BR2_INSTALL_LIBSTDCPP) += c++ -GCC_FINAL_CROSS_LANGUAGES-$(BR2_TOOLCHAIN_BUILDROOT_FORTRAN) += fortran -GCC_FINAL_CROSS_LANGUAGES = $(subst $(space),$(comma),$(GCC_FINAL_CROSS_LANGUAGES-y)) +HOST_GCC_FINAL_CROSS_LANGUAGES-y = c +HOST_GCC_FINAL_CROSS_LANGUAGES-$(BR2_INSTALL_LIBSTDCPP) += c++ +HOST_GCC_FINAL_CROSS_LANGUAGES-$(BR2_TOOLCHAIN_BUILDROOT_FORTRAN) += fortran +HOST_GCC_FINAL_CROSS_LANGUAGES = $(subst $(space),$(comma),$(HOST_GCC_FINAL_CROSS_LANGUAGES-y)) HOST_GCC_FINAL_CONF_OPTS = \ $(HOST_GCC_COMMON_CONF_OPTS) \ - --enable-languages=$(GCC_FINAL_CROSS_LANGUAGES) \ + --enable-languages=$(HOST_GCC_FINAL_CROSS_LANGUAGES) \ --with-build-time-tools=$(HOST_DIR)/usr/$(GNU_TARGET_NAME)/bin HOST_GCC_FINAL_GCC_LIB_DIR = $(HOST_DIR)/usr/$(GNU_TARGET_NAME)/lib*
package/gcc/gcc-initial/gcc-initial.mk +7 -19
@@ -1,35 +1,23 @@ ################################################################################ # -# gcc-initial +# host-gcc-initial # ################################################################################ -GCC_INITIAL_VERSION = $(HOST_GCC_VERSION) -GCC_INITIAL_SITE = $(HOST_GCC_SITE) -GCC_INITIAL_SOURCE = $(HOST_GCC_SOURCE) +HOST_GCC_INITIAL_VERSION = $(HOST_GCC_VERSION) +HOST_GCC_INITIAL_SOURCE = # We only need to redistribute the gcc sources once, and that is handled by # the host-gcc package, not host-gcc-initial. So, we fake a non-redistribuale # package, even though it *is* redistributable, but is already redistributed # thanks to host-gcc. -GCC_INITIAL_REDISTRIBUTE = IGNORE +HOST_GCC_INITIAL_REDISTRIBUTE = IGNORE HOST_GCC_INITIAL_DEPENDENCIES = $(HOST_GCC_COMMON_DEPENDENCIES) -HOST_GCC_INITIAL_EXCLUDES = $(HOST_GCC_EXCLUDES) -HOST_GCC_INITIAL_POST_EXTRACT_HOOKS += HOST_GCC_FAKE_TESTSUITE - -ifneq ($(call qstrip, $(BR2_XTENSA_CORE_NAME)),) -HOST_GCC_INITIAL_POST_EXTRACT_HOOKS += HOST_GCC_XTENSA_OVERLAY_EXTRACT -endif - -HOST_GCC_INITIAL_POST_PATCH_HOOKS += HOST_GCC_APPLY_PATCHES - -# gcc doesn't support in-tree build, so we create a 'build' -# subdirectory in the gcc sources, and build from there. -HOST_GCC_INITIAL_SUBDIR = build - -HOST_GCC_INITIAL_PRE_CONFIGURE_HOOKS += HOST_GCC_CONFIGURE_SYMLINK +# gcc doesn't support in-tree build, so we re-use the sources extracted +# for the host-gcc package and do a complete out-of-tree build +HOST_GCC_INITIAL_POST_PATCH_HOOKS += HOST_GCC_CONFIGURE_WRAPPER HOST_GCC_INITIAL_CONF_OPTS = \ $(HOST_GCC_COMMON_CONF_OPTS) \
package/gcc/gcc.mk +7 -14
@@ -58,20 +58,13 @@ HOST_GCC_POST_PATCH_HOOKS += HOST_GCC_APPLY_POWERPC_PATCH endif endif -define HOST_GCC_APPLY_PATCHES - if test -d package/gcc/$(HOST_GCC_VERSION); then \ - $(APPLY_PATCHES) $(@D) package/gcc/$(HOST_GCC_VERSION) \*.patch ; \ - fi; - $(HOST_GCC_APPLY_POWERPC_PATCH) -endef - -# -# Create 'build' directory and configure symlink -# - -define HOST_GCC_CONFIGURE_SYMLINK - mkdir -p $(@D)/build - ln -sf ../configure $(@D)/build/configure +# Note: HOST_GCC_DIR is not yet defined, but this macro is only used in +# the host-gcc-initial and host-gcc-final packages, which are included +# only after we call the package infra, so HOST_GCC_DIR will be defined +# when it is used. +define HOST_GCC_CONFIGURE_WRAPPER + printf '#!/bin/sh\nexec "$(HOST_GCC_DIR)/configure" "$${@}"\n' >$(@D)/configure + chmod 755 $(@D)/configure endef #
Signature · 1 line
1.9.1
RFC PATCH 12/21 package/gcc: include the extensa overlay in the legal-info output
In reply to Yann E. MORINthe Xtensa overlay is combined with the gcc sources, so it must be redistributed for proper compliance.
Add a post-legal-info hook that also installs the Xtensa overlay in the legal-info output.
--- package/gcc/gcc.mk | 8 ++++++++ 1 file changed, 8 insertions(+)
package/gcc/gcc.mk +8 -0
@@ -43,10 +43,18 @@ HOST_GCC_POST_EXTRACT_HOOKS += HOST_GCC_FAKE_TESTSUITE # ifneq ($(call qstrip, $(BR2_XTENSA_CORE_NAME)),) HOST_GCC_XTENSA_OVERLAY_TAR = $(BR2_XTENSA_OVERLAY_DIR)/xtensa_$(call qstrip,$(BR2_XTENSA_CORE_NAME)).tar + define HOST_GCC_XTENSA_OVERLAY_EXTRACT tar xf $(HOST_GCC_XTENSA_OVERLAY_TAR) -C $(@D) --strip-components=1 gcc endef HOST_GCC_POST_EXTRACT_HOOKS += HOST_GCC_XTENSA_OVERLAY_EXTRACT + +define HOST_GCC_XTENSA_OVERLAY_LEGAL_INFO + $(Q)$(call hardlink-copy,$(HOST_GCC_XTENSA_OVERLAY_TAR),\ + $(HOST_GCC_REDIST_SOURCES_DIR)) +endef +HOST_GCC_POST_LEGAL_INFO_HOOKS += HOST_GCC_XTENSA_OVERLAY_LEGAL_INFO + endif # Xtensa ifeq ($(ARCH),powerpc)
Signature · 1 line
1.9.1
RFC PATCH 13/21 package/perl: include perl-cross in the legal-info output
In reply to Yann E. MORINThe perl-cross sources are combined with the perl sources, so they must be redistributed for proper compliance.
Add a post-legal-info hook that also installs them in the legal-info output.
--- package/perl/perl.mk | 7 +++++++ 1 file changed, 7 insertions(+)
package/perl/perl.mk +7 -0
@@ -32,6 +32,13 @@ define PERL_CROSS_EXTRACT endef PERL_POST_EXTRACT_HOOKS += PERL_CROSS_EXTRACT +# ... and as such should be part of the legal-info redistribution +define PERL_CROSS_LEGAL_INFO + $(Q)$(call hardlink-copy,$(DL_DIR)/$(PERL_CROSS_SOURCE),\ + $(PERL_REDIST_SOURCES_DIR)) +endef +PERL_POST_LEGAL_INFO_HOOKS += PERL_CROSS_LEGAL_INFO + define PERL_CROSS_SET_POD $(SED) s/$(PERL_CROSS_OLD_POD)/$(PERL_CROSS_NEW_POD)/g $(@D)/Makefile endef
Signature · 1 line
1.9.1
RFC PATCH 14/21 core: add a global variable with already-unquoted global patch dir
In reply to Yann E. MORIN... and use it instead of duplicating the qstrip everywhere it is needed.
--- Makefile | 2 ++ package/gcc/gcc.mk | 4 ++-- package/pkg-generic.mk | 2 +- 3 files changed, 5 insertions(+), 3 deletions(-)
Makefile +2 -0
@@ -187,6 +187,8 @@ ifneq ($(BR2_CCACHE_DIR),) BR_CACHE_DIR := $(BR2_CCACHE_DIR) endif +BR_GLOBAL_PATCH_DIR := $(call qstrip,$(BR2_GLOBAL_PATCH_DIR)) + # Need that early, before we scan packages # Avoids doing the $(or...) everytime BR_GRAPH_OUT := $(or $(BR2_GRAPH_OUT),pdf)
package/gcc/gcc.mk +2 -2
@@ -257,8 +257,8 @@ HOST_GCC_COMMON_CCACHE_HASH_FILES += $(DL_DIR)/$(HOST_GCC_SOURCE) HOST_GCC_COMMON_CCACHE_HASH_FILES += \ $(sort $(wildcard \ package/gcc/$(HOST_GCC_VERSION)/*.patch \ - $(addsuffix $((PKG)_RAWNAME)/$(HOST_GCC_VERSION)/*.patch,$(call qstrip,$(BR2_GLOBAL_PATCH_DIR))) \ - $(addsuffix $((PKG)_RAWNAME)/*.patch,$(call qstrip,$(BR2_GLOBAL_PATCH_DIR))))) + $(addsuffix $((PKG)_RAWNAME)/$(HOST_GCC_VERSION)/*.patch,$(BR_GLOBAL_PATCH_DIR)) \ + $(addsuffix $((PKG)_RAWNAME)/*.patch,$(BR_GLOBAL_PATCH_DIR)))) ifeq ($(BR2_xtensa),y) HOST_GCC_COMMON_CCACHE_HASH_FILES += $(HOST_GCC_XTENSA_OVERLAY_TAR) endif
package/pkg-generic.mk +1 -1
@@ -147,7 +147,7 @@ $(BUILD_DIR)/%/.stamp_rsynced: # For BR2_GLOBAL_PATCH_DIR, only generate if it is defined $(BUILD_DIR)/%/.stamp_patched: NAMEVER = $(RAWNAME)-$($(PKG)_VERSION) $(BUILD_DIR)/%/.stamp_patched: PATCH_BASE_DIRS = $(PKGDIR) -$(BUILD_DIR)/%/.stamp_patched: PATCH_BASE_DIRS += $(addsuffix /$(RAWNAME),$(call qstrip,$(BR2_GLOBAL_PATCH_DIR))) +$(BUILD_DIR)/%/.stamp_patched: PATCH_BASE_DIRS += $(addsuffix /$(RAWNAME),$(BR_GLOBAL_PATCH_DIR)) $(BUILD_DIR)/%/.stamp_patched: @$(call step_start,patch) @$(call MESSAGE,"Patching")
Signature · 1 line
1.9.1
RFC PATCH 15/21 core/pkg-generic: add macro to hardlink-or-copy
In reply to Yann E. MORIN--- package/pkg-generic.mk | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+)
package/pkg-generic.mk +18 -0
@@ -305,6 +305,24 @@ endif endef ################################################################################ +# hardlink-copy -- hardlink source and destination if possible, otherwise +# do a simple copy +# +# argument 1 is the source +# argument 2 is the destination *directory* +# +# example: +# $(call hard-link-copy,source,destination) +# +# Note: we make that a single command, so se can: +# - use '$(Q)' in front of it and properly silence the whole macro, +# - use '|| exit 1' after it, so we can exit on error in compound commands. +################################################################################ +define hardlink-copy + { mkdir -p $(2) && { cp -l $(1) $(2) 2>/dev/null || cp $(1) $(2); } } +endef + +################################################################################ # inner-generic-package -- generates the make targets needed to build a # generic package #
Signature · 1 line
1.9.1
RFC PATCH 15/21 core/pkg-generic: add macro to hardlink-or-copy
Dear Yann,
Quoted 21 lines · Yann E. MORIN
Signed-off-by "Yann E. MORIN" <[email protected]>Cc Luca Ceresoli <[email protected]>--- package/pkg-generic.mk | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+)package/pkg-generic.mk +7
-0@@ -305,6 +305,24 @@ endif endef ################################################################################ +# hardlink-copy -- hardlink source and destination if possible, otherwise +# do a simple copy +# +# argument 1 is the source +# argument 2 is the destination *directory* +# +# example:
package/pkg-generic.mk endif · line 324
Quoted 10 lines
package/pkg-generic.mk +10
-0+# $(call hard-link-copy,source,destination) +# +# Note: we make that a single command, so se can: +# - use '$(Q)' in front of it and properly silence the whole macro, +# - use '|| exit 1' after it, so we can exit on error in compound commands. +################################################################################ +define hardlink-copy + { mkdir -p $(2) && { cp -l $(1) $(2) 2>/dev/null || cp $(1) $(2); } } +endef +
Good idea! But wouldn't it fit better in pkg-utils.mk? Indeed it's very generic.
Signature · 1 line
Luca
RFC PATCH 16/21 core/legal-info: use the macro to install source archives
In reply to Yann E. MORIN--- package/pkg-generic.mk | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-)
package/pkg-generic.mk +4 -3
@@ -819,9 +819,10 @@ ifeq ($$($(2)_REDISTRIBUTE),YES) ifneq ($$($(2)_ACTUAL_SOURCE_TARBALL),$$($(2)_SOURCE)) $$(call DOWNLOAD,$$($(2)_ACTUAL_SOURCE_SITE)/$$($(2)_ACTUAL_SOURCE_TARBALL)) endif -# Copy the source tarball (just hardlink if possible) - @cp -l $$(DL_DIR)/$$($(2)_ACTUAL_SOURCE_TARBALL) $$(REDIST_SOURCES_DIR_$$(call UPPERCASE,$(4))) 2>/dev/null || \ - cp $$(DL_DIR)/$$($(2)_ACTUAL_SOURCE_TARBALL) $$(REDIST_SOURCES_DIR_$$(call UPPERCASE,$(4))) +# Copy the source tarball + $$(Q)$$(call hardlink-copy,\ + $$(DL_DIR)/$$($(2)_ACTUAL_SOURCE_TARBALL),\ + $$(REDIST_SOURCES_DIR_$$(call UPPERCASE,$(4)))) endif # redistribute endif # other packages
Signature · 1 line
1.9.1
RFC PATCH 17/21 core/legal-info: install source archives in their own sub-dir
In reply to Yann E. MORINCurrently, we put all source archives side-by-side in the same directory.
Since we're about to also save individual patches that were applied on those sources, we don't want to make that directory a complete mess of unassorted files.
So, we install each source archive in its own sub-directory, where we'll later store the patches too. Store that location in a variable, so it can be re-used later on (to install patches in a future commit).
--- package/pkg-generic.mk | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)
package/pkg-generic.mk +3 -1
@@ -492,6 +492,8 @@ endif $(2)_REDISTRIBUTE ?= YES +$(2)_REDIST_SOURCES_DIR = $$(REDIST_SOURCES_DIR_$$(call UPPERCASE,$(4)))/$$($(2)_RAWNAME)-$$($(2)_VERSION) + # When a target package is a toolchain dependency set this variable to # 'NO' so the 'toolchain' dependency is not added to prevent a circular # dependency
@@ -822,7 +824,7 @@ endif # Copy the source tarball $$(Q)$$(call hardlink-copy,\ $$(DL_DIR)/$$($(2)_ACTUAL_SOURCE_TARBALL),\ - $$(REDIST_SOURCES_DIR_$$(call UPPERCASE,$(4)))) + $$($(2)_REDIST_SOURCES_DIR)) endif # redistribute endif # other packages
Signature · 1 line
1.9.1
RFC PATCH 18/21 core/legal-info: add package version to license directory
In reply to Yann E. MORINNow that we save the source archives in a directory named after the package and its version, do the same for the license files, for consistenmcy.
Also save the version string in the all-licenses list.
However, do not print the version string when warning about undefined _LICENSE_FILES (it's ugly otherwise).
--- package/pkg-generic.mk | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
package/pkg-generic.mk +2 -2
@@ -801,10 +801,10 @@ ifneq ($$(call qstrip,$$($(2)_SOURCE)),) # is that the license still applies to the files distributed as part # of the rootfs, even if the sources are not themselves redistributed. ifeq ($$(call qstrip,$$($(2)_LICENSE_FILES)),) - @$$(call legal-license-nofiles,$$($(2)_RAWNAME),$$(call UPPERCASE,$(4))) + @$$(call legal-license-nofiles,$$($(2)_RAWNAME)-$$($(2)_VERSION),$$(call UPPERCASE,$(4))) @$$(call legal-warning-pkg,$$($(2)_RAWNAME),cannot save license ($(2)_LICENSE_FILES not defined)) else - @$$(foreach F,$$($(2)_LICENSE_FILES),$$(call legal-license-file,$$($(2)_RAWNAME),$$(F),$$($(2)_DIR)/$$(F),$$(call UPPERCASE,$(4)))$$(sep)) + @$$(foreach F,$$($(2)_LICENSE_FILES),$$(call legal-license-file,$$($(2)_RAWNAME)-$$($(2)_VERSION),$$(F),$$($(2)_DIR)/$$(F),$$(call UPPERCASE,$(4)))$$(sep)) endif # license files ifeq ($$($(2)_SITE_METHOD),local)
Signature · 1 line
1.9.1
RFC PATCH 19/21 core/apply-patches: store full path of applied patches
In reply to Yann E. MORINCurrently, we only store the filename of the applied patches.
However, we are soon to want to install those ptches in the legal-info directory, so we'll have to know where those patches come from.
Instead of duplicating the logic to find the patches, just store the full path to each of those patches so we can retrieve them more easily later on.
Also always create the list-file, even if empty, so that we need not test for its existence before reading it.
--- support/scripts/apply-patches.sh | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-)
support/scripts/apply-patches.sh +6 -1
@@ -64,6 +64,10 @@ find ${builddir}/ '(' -name '*.rej' -o -name '.*.rej' ')' -print0 | \ function apply_patch { path=$1 patch=$2 + case "${path}" in + /*) ;; + *) path="$(pwd)/${path}";; + esac case "$patch" in *.gz) type="gzip"; uncomp="gunzip -dc"; ;;
@@ -94,7 +98,7 @@ function apply_patch { echo "Error: missing patch file ${path}/$patch" exit 1 fi - echo $patch >> ${builddir}/.applied_patches_list + echo "${path}/${patch}" >> ${builddir}/.applied_patches_list ${uncomp} "${path}/$patch" | patch -g0 -p1 -E -d "${builddir}" -t -N $silent if [ $? != 0 ] ; then echo "Patch failed! Please fix ${patch}!"
@@ -130,6 +134,7 @@ function scan_patchdir { fi } +touch ${builddir}/.applied_patches_list scan_patchdir "$patchdir" "$patchpattern" # Check for rejects...
Signature · 1 line
1.9.1
RFC PATCH 20/21 core/legal-info: also save patches
In reply to Yann E. MORINCurrently, the legal-info infra only saves the source archive of a package. However, that's not enough as we may aaply some patches on packages sources.
We do suggest users to also redistribute the Buildroot sources as part of their compliance distribution, so the patches bundled in Buildroot would indeed be included in the compliance distribution.
However, that's not enough, since we may download some patches, or the user may use a global patch directory. Patches in there might not end up in the compliance distribution, and there are risks of non-conformity.
So, always include patches alongside the source archive.
To ensure reproducibility, we generate a series file, so patches can be re-applied in the correct order.
We get the list of patches to include from the list of patches that were applied by the package infrastructure (via the apply-patches support script). So, we need to get packages properly extracted and patched before we can save their legal-info, not only in the case they define _LICENSE_FILES (however, we need not extract and patch a package if it is set to be ignored for redistribution).
Note: this means that, when a (non-ignored) package is not patched and defines no LICENSE_FILES, we will extract and patch it for nothing. Trying to reduce the conditions under which a package has to be extracted and patched prior to running legal-info would be pretty much complex and error-prone. Also, a vast majorty of packages are either patched, or define _LICENSE_FILES, so it is best to always extract-and-patch them prior to legal-info.
--- package/pkg-generic.mk | 12 +++++++++--- 1 file changed, 9 insertions(+), 3 deletions(-)
package/pkg-generic.mk +9 -3
@@ -759,12 +759,14 @@ $(2)_MANIFEST_LICENSE_FILES ?= not saved # If the package declares _LICENSE_FILES, we need to extract it, # for overriden, local or normal remote packages alike, whether -# we want to redistribute it or not, but not if we want to ignore it. +# we want to redistribute it or not. +# To get the list of patches applied to a package, we need to +# patch it. +# If we want to ignore the package, we need not extract or patch +# it, however. ifneq ($$($(2)_REDISTRIBUTE),IGNORE) -ifneq ($$($(2)_LICENSE_FILES),) $(1)-legal-info: $(1)-patch endif -endif # We only save the sources of packages we want to redistribute, that are # non-local, and non-overriden. So only store, in the manifest, the tarball
@@ -825,6 +827,10 @@ endif $$(Q)$$(call hardlink-copy,\ $$(DL_DIR)/$$($(2)_ACTUAL_SOURCE_TARBALL),\ $$($(2)_REDIST_SOURCES_DIR)) + $$(Q)while read f; do \ + $$(call hardlink-copy,$$$${f},$$($(2)_REDIST_SOURCES_DIR)) || exit 1; \ + printf "%s\n" "$$$${f##*/}" >>$$($(2)_REDIST_SOURCES_DIR)/series || exit 1; \ + done <$$($(2)_DIR)/.applied_patches_list endif # redistribute endif # other packages
Signature · 1 line
1.9.1
RFC PATCH 21/21 core/pkg-infra: ensure legal-info works in off-line mode
In reply to Yann E. MORINAlmost all packages which are saved for legal-info have their source archives downloaded as part of 'make source', which makes an off-line build completely possible [0].
However, for the pre-configured external toolchains, the source tarball is different, as the main tarball is a binary package. And that source tarball is only downloaded during the legal-info phase, which makes it inconvenient for full off-line builds.
We fix that in two steps:
- first, we move the declaration of _ACTUAL_SOURCE_TARBALL and _SITE earlier in the pkg-generic.mk file, so we get those definitions at the time we define the -source and -all-source rules;
- second, we add a new rule, $(1)-legal-source which only $(1)-all-source depends on, so that we only download it for a top-level 'make source', not as part of the standard download mechanism (i.e. only what is really needed to build).
This way, we can do a complete [0] off-line build and are still able to generate legal-info, while at the same time we do not incur any download overhead during a simple build.
[0] Save for nodejs which invarriably wants to download stuff at build time. Sigh... :-( Fixing that is work for another time...
--- package/pkg-generic.mk | 27 ++++++++++++++++----------- 1 file changed, 16 insertions(+), 11 deletions(-)
package/pkg-generic.mk +16 -11
@@ -438,6 +438,14 @@ ifndef $(2)_SOURCE endif endif +# If FOO_ACTUAL_SOURCE_TARBALL is explicitly defined, it means FOO_SOURCE is +# indeed a binary (e.g. external toolchain) and FOO_ACTUAL_SOURCE_TARBALL/_SITE +# point to the actual sources tarball. Use the actual sources for legal-info. +# For most packages the FOO_SITE/FOO_SOURCE pair points to real source code, +# so these are the defaults for FOO_ACTUAL_*. +$(2)_ACTUAL_SOURCE_TARBALL ?= $$($(2)_SOURCE) +$(2)_ACTUAL_SOURCE_SITE ?= $$(call qstrip,$$($(2)_SITE)) + ifndef $(2)_PATCH ifdef $(3)_PATCH $(2)_PATCH = $$($(3)_PATCH)
@@ -639,6 +647,13 @@ $(1)-depends: $$($(2)_FINAL_DEPENDENCIES) $(1)-source: $$($(2)_TARGET_SOURCE) +$(1)-all-source: $(1)-legal-source + +$(1)-legal-source: $(1)-source +ifneq ($$($(2)_ACTUAL_SOURCE_TARBALL),) + $$(call DOWNLOAD,$$($(2)_ACTUAL_SOURCE_SITE)/$$($(2)_ACTUAL_SOURCE_TARBALL)) +endif + $(1)-source-check: $$(foreach p,$$($(2)_ALL_DOWNLOADS),$$(call SOURCE_CHECK,$$(p))$$(sep))
@@ -780,14 +795,6 @@ endif endif endif -# If FOO_ACTUAL_SOURCE_TARBALL is explicitly defined, it means FOO_SOURCE is -# indeed a binary (e.g. external toolchain) and FOO_ACTUAL_SOURCE_TARBALL/_SITE -# point to the actual sources tarball. Use the actual sources for legal-info. -# For most packages the FOO_SITE/FOO_SOURCE pair points to real source code, -# so these are the defaults for FOO_ACTUAL_*. -$(2)_ACTUAL_SOURCE_TARBALL ?= $$($(2)_SOURCE) -$(2)_ACTUAL_SOURCE_SITE ?= $$(call qstrip,$$($(2)_SITE)) - # legal-info: produce legally relevant info. $(1)-legal-info: ifneq ($$($(2)_REDISTRIBUTE),IGNORE)
@@ -820,9 +827,6 @@ else # Other packages ifeq ($$($(2)_REDISTRIBUTE),YES) -ifneq ($$($(2)_ACTUAL_SOURCE_TARBALL),$$($(2)_SOURCE)) - $$(call DOWNLOAD,$$($(2)_ACTUAL_SOURCE_SITE)/$$($(2)_ACTUAL_SOURCE_TARBALL)) -endif # Copy the source tarball $$(Q)$$(call hardlink-copy,\ $$(DL_DIR)/$$($(2)_ACTUAL_SOURCE_TARBALL),\
@@ -931,6 +935,7 @@ endif $(1)-show-depends \ $(1)-show-version \ $(1)-source \ + $(1)-legal-source \ $(1)-source-check ifeq ($$(patsubst %/,ERROR,$$($(2)_SITE)),ERROR)
Signature · 1 line
1.9.1