Automatically produce legal compliance info
cover Automatically produce legal compliance info
Hi,
during the latest Buildroot Developers Day in November 2011 and in this mailing list there has been some discussion about introducing in Buildroot the possibility to derive automatically legally relevant material, such as licensing info and source tarballs for open source packages.
This is a first tentative implementation of these features.
This code is not yet fully working, but I wanted to share it with you to receive some early comments. I also wanted to allow those who will participate to the next Buildroot Developer Day next Friday to have an idea of this proposal in advance, in case this were object of discussion during the meeting.
My approach is based on the creation of a per-package _LICENSE constant in eack .mk file, such as:
FOOBAR_LICENSE = GPL_V3
MYAPP_LICENSE = PROPRIETARY
This is the only effort required to the package creator. Where it is not specified it defaults to "unknown".
My overall goal is to add a new 'make legal-info' target that:
- produces a manifest file listing all packages, including closed-source ones (and, why not, prints out to stdout the same info);
- copies source code tarballs for all non-proprietary packages (although I'm saving also tarballs for BSD-licensed packages for now);
- save the complete text of all license files that must be included in the product documentation (typically COPYING and LICENSE files); this is to simplify the work of who must provide these info to documentation writers.
This patchset implements the first two points, albeit with some issues that I discuss below.
About the third point I devised a few different possible implementations, but each of them has relevant drawbacks, so I left this point out for the moment.
Here's what is currently implmented:
$ make legal-info busybox 1.19.3 GPL_V2_ONLY bzip2 1.0.5 BSD directfb 1.4.15 LGPL_V2.1 foobar 1.2.3.4 PROPRIETARY freetype 2.4.8 unknown iostat 2.2 GPL_V2 ... $ cat output/legal-info/manifest.csv package,version,license busybox,1.19.3,GPL_V2_ONLY bzip2,1.0.5,BSD directfb,1.4.15,LGPL_V2.1 foobar,1.2.3.4,PROPRIETARY freetype,2.4.8,unknown iostat,2.2,GPL_V2 ... $ ls output/legal-info/sources/ autoconf-2.65.tar.bz2 automake-1.11.1.tar.bz2 binutils-2.21.1.tar.bz2 busybox-1.19.3.tar.bz2 bzip2-1.0.5.tar.gz DirectFB-1.4.15.tar.gz fakeroot_1.9.5.tar.gz freetype-2.4.8.tar.bz2 ...
The implementation is somewhat similar to 'make external-deps'. A $(PKG)-legal-info target is created for each package which echoes the $(PKG)_LICENSE value and other info both to stdout and to the manifest file. It also copies source tarball in $(BASE_DIR)/legal-info/sources unless $(PKG)_LICENSE equals "PROPRIETARY".
A top-level 'legal-info' target collects all of these per-package targets.
The implementation takes only ~35 lines of code so it should be simple to review.
Now the long list of open issues.
The semantics of the _LICENSE variable is still non well defined. It might be a generic string (e.g. FOOBAR_LICENSE = modified 3-clause BSD), or one from a well-defined list of known licenses.
The former interpretation is probably the more useful for producing a manifest file. The latter might be useful to help an automated implementation of the last goal listed above: save the complete text of all license files to a directory.
This patchset has been tested only in a few configurations. The only one that has been really tested with reasonable depth is presented in the last patch of this series.
This might in turn grow this list of open issues as soon as different configs are tested...
Some packages do not appear in the 'make legal-info' output. It looks like they are all and only the Buildroot's internal toolchain files (gcc, mpfr, mpc etc). I couldn't find where the cause is yet, but I'm sure a more expert eye can enlighten me.
Copying the source tarball does not currently work for packages with _SITE_METHOD equal to local, as there is no tarball associated to it. Other methods, such as file and all versioning systems, are not tested, but they should work as there's a tarball in the download dir.
Non-gentargets and non-autotargets packages are not tested. I guess they need a $(PKG)-legal-info target to be defined manually.
It might be useful to remove the output/legal-info dir before populating it, to be sure there are no remnants of previous runs. It would not have a big additional cost, since the computations and copies must be done anyway.
Finally, there is no documentation yet. I will write some if the presented approach is considered good.
The patches are logically divided in these parts:
- patches 1 to 4 introduce the mechanism to extract all the legal info; this is the interesting part;
- patches 5 to 13 define the _LICENSE constant for some packages;
- patches 14 and 15 only add some testing stuff; they are not meant to be merged.
Luca
Luca Ceresoli (15):
legal-info: new target to echo basic per-package legal info
legal-info: produce a manifest file with licensing info
legal-info: save source tarballs for all packages
legal-info: do not copy sources for proprietary packages
mpc: define license
linux: define license
m4: define license
busybox: define license
bzip2: define license
directfb: define license
iostat: define license
lzop: define license
tslib: define license
foobar: create a fake proprietary package (testing only) Create a test config (testing only)
Makefile | 26 ++++++++++++++++++++++---- configs/legal_info_test_defconfig | 13 +++++++++++++ linux/linux.mk | 1 + package/Config.in | 1 + package/Makefile.package.in | 11 +++++++++++ package/busybox/busybox.mk | 1 + package/bzip2/bzip2.mk | 1 + package/directfb/directfb.mk | 1 + package/foobar/Config.in | 5 +++++ package/foobar/foobar.mk | 13 +++++++++++++ package/foobar/source/foobar.c | 7 +++++++ package/iostat/iostat.mk | 1 + package/lzop/lzop.mk | 1 + package/m4/m4.mk | 1 + package/mpc/mpc.mk | 1 + package/tslib/tslib.mk | 1 + 16 files changed, 81 insertions(+), 4 deletions(-) create mode 100644 configs/legal_info_test_defconfig create mode 100644 package/foobar/Config.in create mode 100644 package/foobar/foobar.mk create mode 100644 package/foobar/source/foobar.c
Signature · 1 line
1.7.5.4
RFC 1/15 legal-info: new target to echo basic per-package legal info
--- Makefile | 9 ++++++++- package/Makefile.package.in | 6 ++++++ 2 files changed, 14 insertions(+), 1 deletions(-)
Makefile +8 -1
@@ -354,6 +354,11 @@ HOST_DEPS = $(sort $(foreach dep,\ $($(dep)))) HOST_SOURCE += $(addsuffix -source,$(sort $(TARGETS_HOST_DEPS) $(HOST_DEPS))) +TARGETS_LEGAL_INFO:=$(patsubst %,%-legal-info,\ + $(filter-out makedevs,\ + $(patsubst host-%,%,\ + $(TARGETS) $(BASE_TARGETS) $(TARGETS_HOST_DEPS) $(HOST_DEPS)))) + # all targets depend on the crosscompiler and it's prerequisites $(TARGETS_ALL): __real_tgt_%: $(BASE_TARGETS) %
@@ -387,7 +392,7 @@ $(O)/toolchainfile.cmake: .PHONY: all world dirs clean distclean source outputmakefile \ $(BASE_TARGETS) $(TARGETS) $(TARGETS_ALL) \ - $(TARGETS_CLEAN) $(TARGETS_DIRCLEAN) $(TARGETS_SOURCE) \ + $(TARGETS_CLEAN) $(TARGETS_DIRCLEAN) $(TARGETS_SOURCE) $(TARGETS_LEGAL_INFO) \ $(DL_DIR) $(TOOLCHAIN_DIR) $(BUILD_DIR) $(STAGING_DIR) $(TARGET_DIR) \ $(HOST_DIR) $(BINARIES_DIR) $(STAMP_DIR)
@@ -497,6 +502,8 @@ source: dirs $(TARGETS_SOURCE) $(HOST_SOURCE) external-deps: @$(MAKE) -Bs DL_MODE=SHOW_EXTERNAL_DEPS $(EXTRAMAKEARGS) source | sort -u +legal-info: $(TARGETS_LEGAL_INFO) + show-targets: @echo $(TARGETS)
package/Makefile.package.in +6 -0
@@ -512,6 +512,8 @@ ifndef $(2)_SOURCE endif endif +$(2)_LICENSE ?= unknown + ifndef $(2)_PATCH ifdef $(3)_PATCH $(2)_PATCH = $($(3)_PATCH)
@@ -651,6 +653,9 @@ $(1)-rsync: $$($(2)_TARGET_RSYNC) $(1)-source: $$($(2)_TARGET_RSYNC_SOURCE) endif +$(1)-legal-info: $(1)-source + @echo -e "$(1)\t$$($(3)_VERSION)\t$$($(3)_LICENSE)" + $(1)-show-depends: @echo $$($(2)_DEPENDENCIES)
@@ -694,6 +699,7 @@ $$($(2)_TARGET_PATCH): PKG=$(2) $$($(2)_TARGET_PATCH): RAWNAME=$(patsubst host-%,%,$(1)) $$($(2)_TARGET_EXTRACT): PKG=$(2) $$($(2)_TARGET_SOURCE): PKG=$(2) +$$($(2)_TARGET_LEGAL_INFO): PKG=$(2) $$($(2)_TARGET_UNINSTALL): PKG=$(2) $$($(2)_TARGET_CLEAN): PKG=$(2) $$($(2)_TARGET_DIRCLEAN): PKG=$(2)
Signature · 1 line
1.7.5.4
RFC 1/15 legal-info: new target to echo basic per-package legal info
Quoted 1 line · On Sunday 29 January 2012 16:11:34 Luca Ceresoli wrote:
Signed-off-by Luca Ceresoli <[email protected]>
Quoted 10 lines
--- Makefile | 9 ++++++++- package/Makefile.package.in | 6 ++++++ 2 files changed, 14 insertions(+), 1 deletions(-)Makefile +0
-0
Makefile HOST_DEPS = $(sort $(foreach dep,\ · line 361
Quoted 8 lines
Makefile +5
-0@@ -354,6 +354,11 @@ HOST_DEPS = $(sort $(foreach dep,\ $($(dep)))) HOST_SOURCE += $(addsuffix -source,$(sort $(TARGETS_HOST_DEPS) $(HOST_DEPS))) +TARGETS_LEGAL_INFO:=$(patsubst %,%-legal-info,\ + $(filter-out makedevs,\ + $(patsubst host-%,%,\ + $(TARGETS) $(BASE_TARGETS) $(TARGETS_HOST_DEPS) $(HOST_DEPS)))) +
Are we creating the legal info of the target or the SDK here? If it is
for the target, the host-% are not needed since they are not distributed. I think there should be a separate command for the SDK's legal info.
[snip]
Makefile HOST_DEPS = $(sort $(foreach dep,\ · line 364
Quoted 3 lines
Makefile +3
-0+$(1)-legal-info: $(1)-source + @echo -e "$(1)\t$$($(3)_VERSION)\t$$($(3)_LICENSE)" +
Minor detail: if this patch is separate from the one that saves the source
tarball, the dependency on $(1)-source is unnecessary.
[snip]
Regards,
Arnout
Signature · 6 lines
Arnout Vandecappelle arnout at mind be Senior Embedded Software Architect +32-16-286540 Essensium/Mind http://www.mind.be G.Geenslaan 9, 3001 Leuven, Belgium BE 872 984 063 RPR Leuven LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle GPG fingerprint: 7CB5 E4CC 6C2E EFD4 6E3D A754 F963 ECAB 2450 2F1F
RFC 1/15 legal-info: new target to echo basic per-package legal info
Hi Arnout,
thanks for the review.
Quoted 17 lines · Arnout Vandecappelle
Quoted 1 line · On Sunday 29 January 2012 16:11:34 Luca Ceresoli wrote:
Signed-off-by Luca Ceresoli<[email protected]>Quoted 13 lines
--- Makefile | 9 ++++++++- package/Makefile.package.in | 6 ++++++ 2 files changed, 14 insertions(+), 1 deletions(-)Makefile +0
-0@@ -354,6 +354,11 @@ HOST_DEPS = $(sort $(foreach dep,\ $($(dep)))) HOST_SOURCE += $(addsuffix -source,$(sort $(TARGETS_HOST_DEPS) $(HOST_DEPS)))
Quoted 8 lines
Makefile HOST_DEPS = $(sort $(foreach dep,\ · line 361
Quoted 5 lines
Makefile +5
-0+TARGETS_LEGAL_INFO:=$(patsubst %,%-legal-info,\ + $(filter-out makedevs,\ + $(patsubst host-%,%,\ + $(TARGETS) $(BASE_TARGETS) $(TARGETS_HOST_DEPS) $(HOST_DEPS)))) +Are we creating the legal info of the target or the SDK here? If it is
for the target, the host-% are not needed since they are not distributed. I think there should be a separate command for the SDK's legal info.
The intent was to save also the SDK (toolchain), except it does not currently work.
But this raises a more high-level question about what we want to save
here: target packages, host packages (toolchain and others), BR sources, BR .config? I think the right answer is "all of them", but I've put it on my list of issues to be discussed in the BDD.
Quoted 7 lines
[snip]
Makefile
Quoted 3 lines
+$(1)-legal-info: $(1)-source
Makefile +2
-0+ @echo -e "$(1)\t$$($(3)_VERSION)\t$$($(3)_LICENSE)" +Minor detail: if this patch is separate from the one that saves the source
tarball, the dependency on $(1)-source is unnecessary.
Ops, this is the result of the massive rebases I did before posting. Moved to the right patch, thanks.
Luca
RFC 2/15 legal-info: produce a manifest file with licensing info
In reply to Luca Ceresoli--- Makefile | 14 +++++++++++--- package/Makefile.package.in | 4 +++- 2 files changed, 14 insertions(+), 4 deletions(-)
Makefile +11 -3
@@ -278,6 +278,9 @@ TARGET_DIR:=$(BASE_DIR)/target TOOLCHAIN_DIR=$(BASE_DIR)/toolchain TARGET_SKELETON=$(TOPDIR)/fs/skeleton +LEGAL_INFO_DIR=$(BASE_DIR)/legal-info +LEGAL_MANIFEST_CSV=$(LEGAL_INFO_DIR)/manifest.csv + ifeq ($(BR2_CCACHE),y) CCACHE:=$(HOST_DIR)/usr/bin/ccache CCACHE_CACHE_DIR=$(HOME)/.buildroot-ccache
@@ -363,7 +366,7 @@ TARGETS_LEGAL_INFO:=$(patsubst %,%-legal-info,\ $(TARGETS_ALL): __real_tgt_%: $(BASE_TARGETS) % dirs: $(DL_DIR) $(TOOLCHAIN_DIR) $(BUILD_DIR) $(STAGING_DIR) $(TARGET_DIR) \ - $(HOST_DIR) $(BINARIES_DIR) $(STAMP_DIR) + $(HOST_DIR) $(BINARIES_DIR) $(STAMP_DIR) $(LEGAL_INFO_DIR) $(BASE_TARGETS): dirs $(O)/toolchainfile.cmake
@@ -391,6 +394,7 @@ $(O)/toolchainfile.cmake: " > $@ .PHONY: all world dirs clean distclean source outputmakefile \ + legal-info-prepare-manifest \ $(BASE_TARGETS) $(TARGETS) $(TARGETS_ALL) \ $(TARGETS_CLEAN) $(TARGETS_DIRCLEAN) $(TARGETS_SOURCE) $(TARGETS_LEGAL_INFO) \ $(DL_DIR) $(TOOLCHAIN_DIR) $(BUILD_DIR) $(STAGING_DIR) $(TARGET_DIR) \
@@ -402,7 +406,7 @@ $(O)/toolchainfile.cmake: # dependencies anywhere else # ############################################################# -$(DL_DIR) $(TOOLCHAIN_DIR) $(BUILD_DIR) $(HOST_DIR) $(BINARIES_DIR) $(STAMP_DIR): +$(DL_DIR) $(TOOLCHAIN_DIR) $(BUILD_DIR) $(HOST_DIR) $(BINARIES_DIR) $(STAMP_DIR) $(LEGAL_INFO_DIR): @mkdir -p $@ $(STAGING_DIR):
@@ -502,7 +506,11 @@ source: dirs $(TARGETS_SOURCE) $(HOST_SOURCE) external-deps: @$(MAKE) -Bs DL_MODE=SHOW_EXTERNAL_DEPS $(EXTRAMAKEARGS) source | sort -u -legal-info: $(TARGETS_LEGAL_INFO) +legal-info-prepare-manifest: $(LEGAL_INFO_DIR) + @rm -f $(LEGAL_MANIFEST_CSV) + @echo "package,version,license" >$(LEGAL_MANIFEST_CSV) + +legal-info: legal-info-prepare-manifest $(TARGETS_LEGAL_INFO) show-targets: @echo $(TARGETS)
package/Makefile.package.in +3 -1
@@ -653,8 +653,10 @@ $(1)-rsync: $$($(2)_TARGET_RSYNC) $(1)-source: $$($(2)_TARGET_RSYNC_SOURCE) endif -$(1)-legal-info: $(1)-source +$(1)-legal-info: $(1)-source $(LEGAL_INFO_DIR) @echo -e "$(1)\t$$($(3)_VERSION)\t$$($(3)_LICENSE)" + @echo "$(1),$$($(3)_VERSION),$$($(3)_LICENSE)" \ + >>$(LEGAL_MANIFEST_CSV) $(1)-show-depends: @echo $$($(2)_DEPENDENCIES)
Signature · 1 line
1.7.5.4
RFC 2/15 legal-info: produce a manifest file with licensing info
Quoted 1 line · On Sunday 29 January 2012 16:11:35 Luca Ceresoli wrote:
Signed-off-by Luca Ceresoli <[email protected]>
[snip]
Makefile TARGETS_LEGAL_INFO:=$(patsubst %,%-legal-info,\ · line 369
Quoted 3 lines
dirs: $(DL_DIR) $(TOOLCHAIN_DIR) $(BUILD_DIR) $(STAGING_DIR) $(TARGET_DIR) \
Makefile +1
-1- $(HOST_DIR) $(BINARIES_DIR) $(STAMP_DIR) + $(HOST_DIR) $(BINARIES_DIR) $(STAMP_DIR) $(LEGAL_INFO_DIR)
Since legal-info-prepare-manifest already depends on LEGAL_INFO_DIR, this is redundant.
[snip]
Makefile source: dirs $(TARGETS_SOURCE) $(HOST_SOURCE) · line 512
Quoted 4 lines
Makefile +4
-0+legal-info-prepare-manifest: $(LEGAL_INFO_DIR) + @rm -f $(LEGAL_MANIFEST_CSV) + @echo "package,version,license" >$(LEGAL_MANIFEST_CSV) +
I might be wrong, but I think this won't rebuild the manifest if the
directory already exists. You'd need a dependency on FORCE (and add the phony FORCE target).
Again a minor comment: the clean target is missing here (but it's added
in the next patch). But for me, the first three patches can be rolled into one.
Quoted 9 lines
Makefile +1
-0+legal-info: legal-info-prepare-manifest $(TARGETS_LEGAL_INFO) show-targets: @echo $(TARGETS)package/Makefile.package.in +0
-0
package/Makefile.package.in $(1)-rsync: $$($(2)_TARGET_RSYNC) · line 659
Quoted 8 lines
package/Makefile.package.in +3
-1@@ -653,8 +653,10 @@ $(1)-rsync: $$($(2)_TARGET_RSYNC) $(1)-source: $$($(2)_TARGET_RSYNC_SOURCE) endif -$(1)-legal-info: $(1)-source +$(1)-legal-info: $(1)-source $(LEGAL_INFO_DIR) @echo -e "$(1)\t$$($(3)_VERSION)\t$$($(3)_LICENSE)" + @echo "$(1),$$($(3)_VERSION),$$($(3)_LICENSE)" \ + >>$(LEGAL_MANIFEST_CSV)
Same here: will the manifest be rebuilt if the directory already exists?
Regards,
Arnout
Quoted 4 lines
package/Makefile.package.in +0
-0$(1)-show-depends: @echo $$($(2)_DEPENDENCIES)
Signature · 6 lines
Arnout Vandecappelle arnout at mind be Senior Embedded Software Architect +32-16-286540 Essensium/Mind http://www.mind.be G.Geenslaan 9, 3001 Leuven, Belgium BE 872 984 063 RPR Leuven LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle GPG fingerprint: 7CB5 E4CC 6C2E EFD4 6E3D A754 F963 ECAB 2450 2F1F
RFC 2/15 legal-info: produce a manifest file with licensing info
Quoted 10 lines · Arnout Vandecappelle
Quoted 1 line · On Sunday 29 January 2012 16:11:35 Luca Ceresoli wrote:
Signed-off-by Luca Ceresoli<[email protected]>[snip]
Makefile TARGETS_LEGAL_INFO:=$(patsubst %,%-legal-info,\ · line 369
Quoted 3 lines
dirs: $(DL_DIR) $(TOOLCHAIN_DIR) $(BUILD_DIR) $(STAGING_DIR) $(TARGET_DIR) \Makefile +1
-1- $(HOST_DIR) $(BINARIES_DIR) $(STAMP_DIR) + $(HOST_DIR) $(BINARIES_DIR) $(STAMP_DIR) $(LEGAL_INFO_DIR)Since legal-info-prepare-manifest already depends on
LEGAL_INFO_DIR, this is redundant.
I used to think some Buildroot user might want to 'make dirs'. I understand this is pretty useless, so I'll remove it.
Quoted 9 lines
[snip]
Makefile source: dirs $(TARGETS_SOURCE) $(HOST_SOURCE) · line 512
Quoted 4 lines
Makefile +4
-0+legal-info-prepare-manifest: $(LEGAL_INFO_DIR) + @rm -f $(LEGAL_MANIFEST_CSV) + @echo "package,version,license">$(LEGAL_MANIFEST_CSV) +I might be wrong, but I think this won't rebuild the manifest if the
directory already exists. You'd need a dependency on FORCE (and add the phony FORCE target).
It is rebuilt because legal-info-prepare-manifest is PHONY.
Quoted 4 lines
Again a minor comment: the clean target is missing here (but it's added
in the next patch). But for me, the first three patches can be rolled into one.
I could move the clean target to the 2nd patch, but actually this maintenance burden is quite annoying, and is going to be even more since this patchset will definitely have further changes.
Anybody has something against squashing the first 3 or 4 patches as Arnout suggests?
Quoted 13 lines
Quoted 13 lines
Makefile +1
-0+legal-info: legal-info-prepare-manifest $(TARGETS_LEGAL_INFO) show-targets: @echo $(TARGETS)package/Makefile.package.in +0
-0@@ -653,8 +653,10 @@ $(1)-rsync: $$($(2)_TARGET_RSYNC) $(1)-source: $$($(2)_TARGET_RSYNC_SOURCE) endif
Quoted 6 lines
package/Makefile.package.in $(1)-rsync: $$($(2)_TARGET_RSYNC) · line 659
Quoted 5 lines
package/Makefile.package.in +3
-1-$(1)-legal-info: $(1)-source +$(1)-legal-info: $(1)-source $(LEGAL_INFO_DIR) @echo -e "$(1)\t$$($(3)_VERSION)\t$$($(3)_LICENSE)" + @echo "$(1),$$($(3)_VERSION),$$($(3)_LICENSE)" \ + >>$(LEGAL_MANIFEST_CSV)Same here: will the manifest be rebuilt if the directory already exists?
It is rebuilt as well because $(1)-info-prepare-manifest is PHONY.
Luca
RFC 2/15 legal-info: produce a manifest file with licensing info
Le Wed, 01 Feb 2012 22:29:55 +0100,
Luca Ceresoli <[email protected]> a ?crit :
Quoted 2 lines
Anybody has something against squashing the first 3 or 4 patches as Arnout suggests?
No, I think it's ok to have them squashed together, as long as the commit log in sufficiently long to explain how the implementation works.
Thomas
Signature · 4 lines
Thomas Petazzoni, Free Electrons Kernel, drivers, real-time and embedded Linux development, consulting, training and support. http://free-electrons.com
RFC 2/15 legal-info: produce a manifest file with licensing info
In reply to Luca CeresoliQuoted 15 lines · Luca Ceresoli
Quoted 14 lines · Arnout Vandecappelle
Quoted 2 lines · On Sunday 29 January 2012 16:11:35 Luca Ceresoli wrote:
Signed-off-by Luca Ceresoli<[email protected]>[snip]
Quoted 5 lines
?dirs: $(DL_DIR) $(TOOLCHAIN_DIR) $(BUILD_DIR) $(STAGING_DIR) $(TARGET_DIR) \
- ? ? ? $(HOST_DIR) $(BINARIES_DIR) $(STAMP_DIR)
+ ? ? ? $(HOST_DIR) $(BINARIES_DIR) $(STAMP_DIR) $(LEGAL_INFO_DIR)
Quoted 6 lines
Quoted 2 lines
?Since legal-info-prepare-manifest already depends on
LEGAL_INFO_DIR, this ?is redundant.I used to think some Buildroot user might want to 'make dirs'. I understand this is pretty useless, so I'll remove it.
For $(LEGAL_INFO_DIR), there isn't a real problem since you indeed put it as a dependency elsewhere, which means that the directory will be created anyway.
In other cases, like downloading packages, there is no such explicit dependency on $(DL_DIR), and there may be certain scenarios that you need to do 'make dirs' explicitly. One such scenario is when the first downloaded package is not handled by wget (which creates the DL_DIR automatically) but with e.g. git download.
Quoted 24 lines
Quoted 5 lines
[snip]
Quoted 1 line
Quoted 8 lines
Makefile source: dirs $(TARGETS_SOURCE) $(HOST_SOURCE) · line 512
Quoted 4 lines
Makefile +4
-0+legal-info-prepare-manifest: $(LEGAL_INFO_DIR) + ? ? ? @rm -f $(LEGAL_MANIFEST_CSV) + ? ? ? @echo "package,version,license">$(LEGAL_MANIFEST_CSV) +?I might be wrong, but I think this won't rebuild the manifest if the directory already exists. ?You'd need a dependency on FORCE (and add the phony FORCE target).
It is rebuilt because legal-info-prepare-manifest is PHONY.
Quoted 4 lines
?Again a minor comment: the clean target is missing here (but it's added in the next patch). ?But for me, the first three patches can be rolled into one.
Quoted 7 lines
I could move the clean target to the 2nd patch, but actually this maintenance burden is quite annoying, and is going to be even more since this patchset will definitely have further changes.
Anybody has something against squashing the first 3 or 4 patches as Arnout suggests?
No problem for me.
Quoted 32 lines
Quoted 14 lines
Quoted 14 lines
+legal-info: legal-info-prepare-manifest $(TARGETS_LEGAL_INFO)
?show-targets:
? ? ? ?@echo $(TARGETS)package/Makefile.package.in +0
-0@@ -653,8 +653,10 @@ $(1)-rsync: ? ? ? ? ? ? ? ?$$($(2)_TARGET_RSYNC) ?$(1)-source: ? ? ? ? ?$$($(2)_TARGET_RSYNC_SOURCE)?endif
Quoted 8 lines
Quoted 6 lines
-$(1)-legal-info: ? ? ? $(1)-source
+$(1)-legal-info: ? ? ? $(1)-source $(LEGAL_INFO_DIR)
? ? ? ? ? ? ? ? ? ? ? ?@echo -e "$(1)\t$$($(3)_VERSION)\t$$($(3)_LICENSE)"
+ ? ? ? ? ? ? ? ? ? ? ? @echo "$(1),$$($(3)_VERSION),$$($(3)_LICENSE)" \
+ ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? >>$(LEGAL_MANIFEST_CSV)?Same here: will the manifest be rebuilt if the directory already exists?
It is rebuilt as well because $(1)-info-prepare-manifest is PHONY.
Luca
_______________________________________________
buildroot mailing list
buildroot at busybox.net
http://lists.busybox.net/mailman/listinfo/buildroot
RFC 3/15 legal-info: save source tarballs for all packages
In reply to Luca Ceresoli--- Makefile | 11 +++++++---- package/Makefile.package.in | 3 ++- 2 files changed, 9 insertions(+), 5 deletions(-)
Makefile +7 -4
@@ -279,6 +279,7 @@ TOOLCHAIN_DIR=$(BASE_DIR)/toolchain TARGET_SKELETON=$(TOPDIR)/fs/skeleton LEGAL_INFO_DIR=$(BASE_DIR)/legal-info +REDIST_SOURCES_DIR=$(LEGAL_INFO_DIR)/sources LEGAL_MANIFEST_CSV=$(LEGAL_INFO_DIR)/manifest.csv ifeq ($(BR2_CCACHE),y)
@@ -366,7 +367,8 @@ TARGETS_LEGAL_INFO:=$(patsubst %,%-legal-info,\ $(TARGETS_ALL): __real_tgt_%: $(BASE_TARGETS) % dirs: $(DL_DIR) $(TOOLCHAIN_DIR) $(BUILD_DIR) $(STAGING_DIR) $(TARGET_DIR) \ - $(HOST_DIR) $(BINARIES_DIR) $(STAMP_DIR) $(LEGAL_INFO_DIR) + $(HOST_DIR) $(BINARIES_DIR) $(STAMP_DIR) $(LEGAL_INFO_DIR) \ + $(REDIST_SOURCES_DIR) $(BASE_TARGETS): dirs $(O)/toolchainfile.cmake
@@ -406,7 +408,7 @@ $(O)/toolchainfile.cmake: # dependencies anywhere else # ############################################################# -$(DL_DIR) $(TOOLCHAIN_DIR) $(BUILD_DIR) $(HOST_DIR) $(BINARIES_DIR) $(STAMP_DIR) $(LEGAL_INFO_DIR): +$(DL_DIR) $(TOOLCHAIN_DIR) $(BUILD_DIR) $(HOST_DIR) $(BINARIES_DIR) $(STAMP_DIR) $(LEGAL_INFO_DIR) $(REDIST_SOURCES_DIR): @mkdir -p $@ $(STAGING_DIR):
@@ -510,7 +512,7 @@ legal-info-prepare-manifest: $(LEGAL_INFO_DIR) @rm -f $(LEGAL_MANIFEST_CSV) @echo "package,version,license" >$(LEGAL_MANIFEST_CSV) -legal-info: legal-info-prepare-manifest $(TARGETS_LEGAL_INFO) +legal-info: legal-info-prepare-manifest $(REDIST_SOURCES_DIR) $(TARGETS_LEGAL_INFO) show-targets: @echo $(TARGETS)
@@ -633,7 +635,8 @@ endif clean: rm -rf $(STAGING_DIR) $(TARGET_DIR) $(BINARIES_DIR) $(HOST_DIR) \ - $(STAMP_DIR) $(BUILD_DIR) $(TOOLCHAIN_DIR) $(BASE_DIR)/staging + $(STAMP_DIR) $(BUILD_DIR) $(TOOLCHAIN_DIR) $(BASE_DIR)/staging \ + $(LEGAL_INFO_DIR) $(REDIST_SOURCES_DIR) distclean: clean ifeq ($(DL_DIR),$(TOPDIR)/dl)
package/Makefile.package.in +2 -1
@@ -653,10 +653,11 @@ $(1)-rsync: $$($(2)_TARGET_RSYNC) $(1)-source: $$($(2)_TARGET_RSYNC_SOURCE) endif -$(1)-legal-info: $(1)-source $(LEGAL_INFO_DIR) +$(1)-legal-info: $(1)-source $(LEGAL_INFO_DIR) $(REDIST_SOURCES_DIR) @echo -e "$(1)\t$$($(3)_VERSION)\t$$($(3)_LICENSE)" @echo "$(1),$$($(3)_VERSION),$$($(3)_LICENSE)" \ >>$(LEGAL_MANIFEST_CSV) + @cp $(DL_DIR)/$$($(3)_SOURCE) $(REDIST_SOURCES_DIR) $(1)-show-depends: @echo $$($(2)_DEPENDENCIES)
Signature · 1 line
1.7.5.4
RFC 3/15 legal-info: save source tarballs for all packages
Quoted 1 line · On Sunday 29 January 2012 16:11:36 Luca Ceresoli wrote:
Signed-off-by Luca Ceresoli <[email protected]>
Darn, I've been adding Reviewed-by tags even though I said I wouldn't :-)
[snip]
Makefile endif · line 639
Quoted 7 lines
Makefile +2
-1@@ -633,7 +635,8 @@ endif clean: rm -rf $(STAGING_DIR) $(TARGET_DIR) $(BINARIES_DIR) $(HOST_DIR) \ - $(STAMP_DIR) $(BUILD_DIR) $(TOOLCHAIN_DIR) $(BASE_DIR)/staging + $(STAMP_DIR) $(BUILD_DIR) $(TOOLCHAIN_DIR) $(BASE_DIR)/staging \ + $(LEGAL_INFO_DIR) $(REDIST_SOURCES_DIR)
$(REDIST_SOURCES_DIR) is a subdir of $(LEGAL_INFO_DIR), so doesn't need
to be cleaned twice.
Quoted 8 lines
Makefile +0
-0distclean: clean ifeq ($(DL_DIR),$(TOPDIR)/dl)package/Makefile.package.in +0
-0
package/Makefile.package.in $(1)-rsync: $$($(2)_TARGET_RSYNC) · line 660
Quoted 9 lines
package/Makefile.package.in +2
-1@@ -653,10 +653,11 @@ $(1)-rsync: $$($(2)_TARGET_RSYNC) $(1)-source: $$($(2)_TARGET_RSYNC_SOURCE) endif -$(1)-legal-info: $(1)-source $(LEGAL_INFO_DIR) +$(1)-legal-info: $(1)-source $(LEGAL_INFO_DIR) $(REDIST_SOURCES_DIR) @echo -e "$(1)\t$$($(3)_VERSION)\t$$($(3)_LICENSE)" @echo "$(1),$$($(3)_VERSION),$$($(3)_LICENSE)" \ >>$(LEGAL_MANIFEST_CSV) + @cp $(DL_DIR)/$$($(3)_SOURCE) $(REDIST_SOURCES_DIR)
Actually, for local and override packages (of which you said they are not
supported at the moment), $(1)-source should create a tar file.
Regards,
Arnout
Quoted 4 lines
package/Makefile.package.in +0
-0$(1)-show-depends: @echo $$($(2)_DEPENDENCIES)
Signature · 6 lines
Arnout Vandecappelle arnout at mind be Senior Embedded Software Architect +32-16-286540 Essensium/Mind http://www.mind.be G.Geenslaan 9, 3001 Leuven, Belgium BE 872 984 063 RPR Leuven LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle GPG fingerprint: 7CB5 E4CC 6C2E EFD4 6E3D A754 F963 ECAB 2450 2F1F
RFC 3/15 legal-info: save source tarballs for all packages
Quoted 7 lines · Arnout Vandecappelle
Quoted 1 line · On Sunday 29 January 2012 16:11:36 Luca Ceresoli wrote:
Signed-off-by Luca Ceresoli<[email protected]>Darn, I've been adding Reviewed-by tags even though I said I wouldn't :-)
[snip]
Quoted 2 lines
(quoted diff) +0
-0@@ -633,7 +635,8 @@ endif
Quoted 7 lines
Makefile endif · line 639
Quoted 5 lines
Makefile +2
-1clean: rm -rf $(STAGING_DIR) $(TARGET_DIR) $(BINARIES_DIR) $(HOST_DIR) \ - $(STAMP_DIR) $(BUILD_DIR) $(TOOLCHAIN_DIR) $(BASE_DIR)/staging + $(STAMP_DIR) $(BUILD_DIR) $(TOOLCHAIN_DIR) $(BASE_DIR)/staging \ + $(LEGAL_INFO_DIR) $(REDIST_SOURCES_DIR)$(REDIST_SOURCES_DIR) is a subdir of $(LEGAL_INFO_DIR), so doesn't need
to be cleaned twice.
You're right.
Again a remnant of my initial drafts, when the two subdirs ere not one inside the other...
Quoted 12 lines
Quoted 12 lines
Makefile +0
-0distclean: clean ifeq ($(DL_DIR),$(TOPDIR)/dl)package/Makefile.package.in +0
-0@@ -653,10 +653,11 @@ $(1)-rsync: $$($(2)_TARGET_RSYNC) $(1)-source: $$($(2)_TARGET_RSYNC_SOURCE) endif
Quoted 9 lines
package/Makefile.package.in $(1)-rsync: $$($(2)_TARGET_RSYNC) · line 660
Quoted 6 lines
package/Makefile.package.in +2
-1-$(1)-legal-info: $(1)-source $(LEGAL_INFO_DIR) +$(1)-legal-info: $(1)-source $(LEGAL_INFO_DIR) $(REDIST_SOURCES_DIR) @echo -e "$(1)\t$$($(3)_VERSION)\t$$($(3)_LICENSE)" @echo "$(1),$$($(3)_VERSION),$$($(3)_LICENSE)" \ >>$(LEGAL_MANIFEST_CSV) + @cp $(DL_DIR)/$$($(3)_SOURCE) $(REDIST_SOURCES_DIR)Actually, for local and override packages (of which you said they are not
supported at the moment), $(1)-source should create a tar file.
Yes, this is another point in my TODO list. I have a few implementation options to choose from.
Luca
RFC 4/15 legal-info: do not copy sources for proprietary packages
In reply to Luca Ceresoli--- package/Makefile.package.in | 2 ++ 1 files changed, 2 insertions(+), 0 deletions(-)
package/Makefile.package.in +2 -0
@@ -657,7 +657,9 @@ $(1)-legal-info: $(1)-source $(LEGAL_INFO_DIR) $(REDIST_SOURCES_DIR) @echo -e "$(1)\t$$($(3)_VERSION)\t$$($(3)_LICENSE)" @echo "$(1),$$($(3)_VERSION),$$($(3)_LICENSE)" \ >>$(LEGAL_MANIFEST_CSV) +ifneq ($$($(3)_LICENSE),PROPRIETARY) @cp $(DL_DIR)/$$($(3)_SOURCE) $(REDIST_SOURCES_DIR) +endif $(1)-show-depends: @echo $$($(2)_DEPENDENCIES)
Signature · 1 line
1.7.5.4
RFC 5/15 mpc: define license
In reply to Luca Ceresoli--- package/mpc/mpc.mk | 1 + 1 files changed, 1 insertions(+), 0 deletions(-)
package/mpc/mpc.mk +1 -0
@@ -6,6 +6,7 @@ MPC_VERSION = 0.9 MPC_SITE = http://www.multiprecision.org/mpc/download +MPC_LICENSE = LGPL_V2.1 MPC_INSTALL_STAGING = YES MPC_DEPENDENCIES = gmp mpfr MPC_AUTORECONF = YES
Signature · 1 line
1.7.5.4
RFC 6/15 linux: define license
In reply to Luca Ceresoli--- linux/linux.mk | 1 + 1 files changed, 1 insertions(+), 0 deletions(-)
linux/linux.mk +1 -0
@@ -4,6 +4,7 @@ # ############################################################################### LINUX_VERSION=$(call qstrip,$(BR2_LINUX_KERNEL_VERSION)) +LINUX_LICENSE = GPL_V2_ONLY # Compute LINUX_SOURCE and LINUX_SITE from the configuration ifeq ($(LINUX_VERSION),custom)
Signature · 1 line
1.7.5.4
RFC 7/15 m4: define license
In reply to Luca Ceresoli--- package/m4/m4.mk | 1 + 1 files changed, 1 insertions(+), 0 deletions(-)
package/m4/m4.mk +1 -0
@@ -7,6 +7,7 @@ M4_VERSION = 1.4.16 M4_SOURCE = m4-$(M4_VERSION).tar.bz2 M4_SITE = $(BR2_GNU_MIRROR)/m4 +M4_LICENSE = GPL_V3 M4_CONF_ENV = gl_cv_func_gettimeofday_clobber=no ifneq ($(BR2_USE_WCHAR),y)
Signature · 1 line
1.7.5.4
RFC 8/15 busybox: define license
In reply to Luca Ceresoli--- package/busybox/busybox.mk | 1 + 1 files changed, 1 insertions(+), 0 deletions(-)
package/busybox/busybox.mk +1 -0
@@ -12,6 +12,7 @@ BUSYBOX_VERSION = $(call qstrip,$(BR2_BUSYBOX_VERSION)) BUSYBOX_SITE = http://www.busybox.net/downloads endif BUSYBOX_SOURCE = busybox-$(BUSYBOX_VERSION).tar.bz2 +BUSYBOX_LICENSE = GPL_V2_ONLY BUSYBOX_BUILD_CONFIG = $(BUSYBOX_DIR)/.config # Allows the build system to tweak CFLAGS BUSYBOX_MAKE_ENV = $(TARGET_MAKE_ENV) CFLAGS="$(TARGET_CFLAGS) -I$(LINUX_HEADERS_DIR)/include"
Signature · 1 line
1.7.5.4
RFC 9/15 bzip2: define license
In reply to Luca Ceresoli--- package/bzip2/bzip2.mk | 1 + 1 files changed, 1 insertions(+), 0 deletions(-)
package/bzip2/bzip2.mk +1 -0
@@ -7,6 +7,7 @@ BZIP2_VERSION:=1.0.5 BZIP2_SONAME=1.0.4 BZIP2_SOURCE:=bzip2-$(BZIP2_VERSION).tar.gz BZIP2_SITE:=http://www.bzip.org/$(BZIP2_VERSION) +BZIP2_LICENSE = BSD BZIP2_INSTALL_STAGING=YES define BZIP2_FIX_MAKEFILE
Signature · 1 line
1.7.5.4
RFC 10/15 directfb: define license
In reply to Luca Ceresoli--- package/directfb/directfb.mk | 1 + 1 files changed, 1 insertions(+), 0 deletions(-)
package/directfb/directfb.mk +1 -0
@@ -7,6 +7,7 @@ DIRECTFB_VERSION_MAJOR = 1.4 DIRECTFB_VERSION = $(DIRECTFB_VERSION_MAJOR).15 DIRECTFB_SITE = http://www.directfb.org/downloads/Core/DirectFB-$(DIRECTFB_VERSION_MAJOR) DIRECTFB_SOURCE = DirectFB-$(DIRECTFB_VERSION).tar.gz +DIRECTFB_LICENSE = LGPL_V2.1 DIRECTFB_AUTORECONF = YES DIRECTFB_INSTALL_STAGING = YES DIRECTFB_CONF_OPT = \
Signature · 1 line
1.7.5.4
RFC 11/15 iostat: define license
In reply to Luca Ceresoli--- package/iostat/iostat.mk | 1 + 1 files changed, 1 insertions(+), 0 deletions(-)
package/iostat/iostat.mk +1 -0
@@ -6,6 +6,7 @@ IOSTAT_VERSION = 2.2 IOSTAT_SITE = http://www.linuxinsight.com/files +IOSTAT_LICENSE = GPL_V2 iostat-source: $(DL_DIR)/$(IOSTAT_SOURCE)
Signature · 1 line
1.7.5.4
RFC 12/15 lzop: define license
In reply to Luca Ceresoli--- package/lzop/lzop.mk | 1 + 1 files changed, 1 insertions(+), 0 deletions(-)
package/lzop/lzop.mk +1 -0
@@ -6,6 +6,7 @@ LZOP_VERSION = 1.03 LZOP_SOURCE = lzop-$(LZOP_VERSION).tar.gz LZOP_SITE = http://www.lzop.org/download/ +LZOP_LICENSE = GPL_V2 LZOP_DEPENDENCIES = lzo $(eval $(call AUTOTARGETS))
Signature · 1 line
1.7.5.4
RFC 13/15 tslib: define license
In reply to Luca Ceresoli--- package/tslib/tslib.mk | 1 + 1 files changed, 1 insertions(+), 0 deletions(-)
package/tslib/tslib.mk +1 -0
@@ -5,6 +5,7 @@ ############################################################# TSLIB_VERSION = 412d99d8b92c TSLIB_SITE = git://github.com/kergoth/tslib.git +TSLIB_LICENSE = GPL_V2 TSLIB_AUTORECONF = YES TSLIB_INSTALL_STAGING = YES TSLIB_INSTALL_STAGING_OPT = DESTDIR=$(STAGING_DIR) LDFLAGS=-L$(STAGING_DIR)/usr/lib install
Signature · 1 line
1.7.5.4
RFC 14/15 foobar: create a fake proprietary package (testing only)
In reply to Luca CeresoliThis is only for testing the proprietary package handling, not to be committed in any serious repository.
--- package/Config.in | 1 + package/foobar/Config.in | 5 +++++ package/foobar/foobar.mk | 13 +++++++++++++ package/foobar/source/foobar.c | 7 +++++++ 4 files changed, 26 insertions(+), 0 deletions(-) create mode 100644 package/foobar/Config.in create mode 100644 package/foobar/foobar.mk create mode 100644 package/foobar/source/foobar.c
package/Config.in +1 -0
@@ -409,6 +409,7 @@ endmenu menu "Miscellaneous" source "package/shared-mime-info/Config.in" +source "package/foobar/Config.in" endmenu menu "Networking applications"
package/foobar/Config.in new file +5 -0
@@ -0,0 +1,5 @@ +config BR2_PACKAGE_FOOBAR + bool "foobar" + default y + help + Fake propritary package
package/foobar/foobar.mk new file +13 -0
@@ -0,0 +1,13 @@ +############################################################# +# +# foobar - just a dummy, fake proprietary package +# +############################################################# + +FOOBAR_VERSION = 1.2.3.4 +FOOBAR_SOURCE = foobar-$(FOOBAR_VERSION).tar.bz2 +FOOBAR_SITE = package/foobar/source +FOOBAR_SITE_METHOD = local +FOOBAR_LICENSE = PROPRIETARY + +$(eval $(call GENTARGETS))
package/foobar/source/foobar.c new file +7 -0
@@ -0,0 +1,7 @@ +#include <stdlib.h> + +int main (void ) +{ +printf ("Hello, World!\n" ); +return 0 ; +}
Signature · 1 line
1.7.5.4
RFC 14/15 foobar: create a fake proprietary package (testing only)
Quoted 8 lines · Luca Ceresoli
This is only for testing the proprietary package handling, not to be committed in any serious repository.
package/foobar/source/foobar.c new file +2
-0+FOOBAR_LICENSE =PROPRIETARY +++ b/package/foobar/source/foobar.c
package/foobar/source/foobar.c line 7
Quoted 7 lines
package/foobar/source/foobar.c new file +7
-0@@ -0,0 +1,7 @@ +#include <stdlib.h> + +int main (void ) +{ + ?printf("Hello, World!\n"); + ?return 0; +}
<sarcasm>
OMG OMG... buildroot is forking into a?proprietary?applications DB... where is ./ where we need them?
</sarcasm>
/me takes a cup of water and relaxes... ;-)
RFC 14/15 foobar: create a fake proprietary package (testing only)
Quoted 7 lines · Diego Iastrubni
Quoted 6 lines · Luca Ceresoli
This is only for testing the proprietary package handling, not to be committed in any serious repository.
+FOOBAR_LICENSE = PROPRIETARY
Quoted 16 lines
package/foobar/source/foobar.c line 7
Quoted 9 lines
+++
b/package/foobar/source/foobar.cpackage/foobar/source/foobar.c new file +7
-0@@ -0,0 +1,7 @@ +#include <stdlib.h> + +int main (void ) +{ + ?printf("Hello, World!\n"); + ?return 0; +}<sarcasm>
OMG OMG... buildroot is forking into a?proprietary?applications DB... where is ./ where we need them?
</sarcasm>
Perhaps you missed the recently added support for closed source binary firmware blobs?
<sarcasm>
Your panic threshold setting may have to be tweaked up a bit. </sarcasm>
If this proposal or something like it is included in the build system, then those firmware blob licenses could also be handled with the new bit of infrastructure.
Mike
Quoted 7 lines
/me takes a cup of water and relaxes... ;-)
_______________________________________________
buildroot mailing list
buildroot at busybox.net
http://lists.busybox.net/mailman/listinfo/buildroot
RFC 14/15 foobar: create a fake proprietary package (testing only)
Michael,
I think both of us forgot to take the funny pill today.
I always get confused, is it the blue pill.. or red pill...? :)
Quoted 50 lines · Michael S. Zick
Quoted 7 lines · Diego Iastrubni
Quoted 6 lines · Luca Ceresoli
This is only for testing the proprietary package handling, not to be committed in any serious repository.
+FOOBAR_LICENSE = PROPRIETARY
Quoted 16 lines
package/foobar/source/foobar.c line 7
Quoted 9 lines
+++
b/package/foobar/source/foobar.cpackage/foobar/source/foobar.c new file +7
-0@@ -0,0 +1,7 @@ +#include <stdlib.h> + +int main (void ) +{ + ?printf("Hello, World!\n"); + ?return 0; +}<sarcasm>
OMG OMG... buildroot is forking into a?proprietary?applications DB... where is ./ where we need them?
</sarcasm>Perhaps you missed the recently added support for closed source binary firmware blobs?
<sarcasm>
Your panic threshold setting may have to be tweaked up a bit. </sarcasm>If this proposal or something like it is included in the build system, then those firmware blob licenses could also be handled with the new bit of infrastructure.
Mike
Quoted 7 lines
/me takes a cup of water and relaxes... ;-)
_______________________________________________
buildroot mailing list
buildroot at busybox.net
http://lists.busybox.net/mailman/listinfo/buildroot_______________________________________________
buildroot mailing list
buildroot at busybox.net
http://lists.busybox.net/mailman/listinfo/buildroot
RFC 14/15 foobar: create a fake proprietary package (testing only)
In reply to Michael S. ZickHi Michael,
Quoted 6 lines · Michael S. Zick
Quoted 2 lines · Diego Iastrubni
Quoted 1 line · Luca Ceresoli
If this proposal or something like it is included in the build system, then those firmware blob licenses could also be handled with the new bit of infrastructure.
Thanks for pointing out.
I had a look at the linux-firmware module, and it raises some interesting points for discussion of the compliance stuff in general, as well as the implementation.
Firstly, linux-firmware is a package composed of different parts with different licenses.
This makes it hard for BR to handle the legal stuff for this package, as it should show a different set of licenses depending on which parts are actually used.
This is not unique of linux-firmware though. binutils has as many as 9 license files in its sources.
Second, I read a few random license files in linux-firmware and most of them are similar to BSD licenses, in that they require the license text to be included in the product documentation.
This is why I did not implement the third goal of my RFC, "save the complete text of all license files that must be included in the product documentation". It would be far from trivial.
In fact, in a case such as linux-firmare, it would be relatively easy to copy all the (many) license files, but it would be of limited usefulness.
The best thing to do would be to copy only the license files for those parts of the package that are actually used in the product. I think there's no way to obtain such a feature without a manual work of the package's .mk creator. This work could probably be limited to packages with a non-obvious licensing scheme, though.
Luca
RFC 15/15 Create a test config (testing only)
In reply to Luca CeresoliThis is for testing only, not to be committed in any serious repository
--- configs/legal_info_test_defconfig | 13 +++++++++++++ 1 files changed, 13 insertions(+), 0 deletions(-) create mode 100644 configs/legal_info_test_defconfig
configs/legal_info_test_defconfig new file +13 -0
@@ -0,0 +1,13 @@ +BR2_x86_pentiumpro=y +BR2_TARGET_GENERIC_GETTY_PORT="tty1" +BR2_PACKAGE_BZIP2=y +BR2_PACKAGE_LZOP=y +BR2_PACKAGE_DIRECTFB=y +BR2_PACKAGE_IOSTAT=y +BR2_PACKAGE_KBD=y +BR2_TARGET_ROOTFS_EXT2=y +# BR2_TARGET_ROOTFS_TAR is not set +BR2_LINUX_KERNEL=y +BR2_LINUX_KERNEL_CUSTOM_VERSION=y +BR2_LINUX_KERNEL_USE_CUSTOM_CONFIG=y +BR2_LINUX_KERNEL_CUSTOM_CONFIG_FILE="board/qemu/x86/linux-3.2.config"
Signature · 1 line
1.7.5.4
cover Automatically produce legal compliance info
In reply to Luca CeresoliOn Sunday 29 January 2012 16:11:33 Luca Ceresoli wrote:
[snip]
Quoted 7 lines
$ make legal-info busybox 1.19.3 GPL_V2_ONLY bzip2 1.0.5 BSD directfb 1.4.15 LGPL_V2.1 foobar 1.2.3.4 PROPRIETARY freetype 2.4.8 unknown iostat 2.2 GPL_V2
Cool!
Quoted 10 lines
...
$ cat output/legal-info/manifest.csv package,version,license busybox,1.19.3,GPL_V2_ONLY bzip2,1.0.5,BSD directfb,1.4.15,LGPL_V2.1 foobar,1.2.3.4,PROPRIETARY freetype,2.4.8,unknown iostat,2.2,GPL_V2 ...
Personally I'd have used tabs instead of commas, so it is still
human-readable. Not sure how non-libreoffice spreadsheets deal with that, though.
Quoted 10 lines
$ ls output/legal-info/sources/ autoconf-2.65.tar.bz2 automake-1.11.1.tar.bz2 binutils-2.21.1.tar.bz2 busybox-1.19.3.tar.bz2 bzip2-1.0.5.tar.gz DirectFB-1.4.15.tar.gz fakeroot_1.9.5.tar.gz freetype-2.4.8.tar.bz2 ...
I think these should be hardlinks instead of copies of the dl
directory. You can use 'cp -l', which I believe falls back to actual copying if the target filesystem doesn't support hardlinks (although I can't think of a filesystem that supports symlinks but not hardlinks).
[snip]
Quoted 2 lines
The implementation takes only ~35 lines of code so it should be simple to review.
I'll try to do that the coming days. Without Reviewed-by tag, since this
is still RFC.
Quoted 10 lines
Now the long list of open issues.
The semantics of the
_LICENSEvariable is still non well defined. It might be a generic string (e.g.FOOBAR_LICENSE= modified 3-clause BSD), or one from a well-defined list of known licenses.
The former interpretation is probably the more useful for producing a manifest file. The latter might be useful to help an automated implementation of the last goal listed above: save the complete text of all license files to a directory.
I would pre-define a number of strings, for which a predefined license
text exists. The packager should check if the license text is identical, and attach 'dirty' or something to the name if not.
[snip]
Quoted 2 lines
Copying the source tarball does not currently work for packages with
_SITE_METHODequal to local, as there is no tarball associated to it.
That should be fixed for the final version.
Quoted 5 lines
Other methods, such as file and all versioning systems, are not tested, but they should work as there's a tarball in the download dir.
Non-gentargets and non-autotargets packages are not tested. I guess they need a $(PKG)-legal-info target to be defined manually.
Those packages should be converted anyway.
Quoted 3 lines
It might be useful to remove the output/legal-info dir before populating it, to be sure there are no remnants of previous runs. It would not have a big additional cost, since the computations and copies must be done anyway.
I agree.
[snip]
Regards,
Arnout
Signature · 6 lines
Arnout Vandecappelle arnout at mind be Senior Embedded Software Architect +32-16-286540 Essensium/Mind http://www.mind.be G.Geenslaan 9, 3001 Leuven, Belgium BE 872 984 063 RPR Leuven LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle GPG fingerprint: 7CB5 E4CC 6C2E EFD4 6E3D A754 F963 ECAB 2450 2F1F
cover Automatically produce legal compliance info
In reply to Luca CeresoliLuca, All,
Quoted 55 lines · On Sunday 29 January 2012 16:11:33 Luca Ceresoli wrote:
during the latest Buildroot Developers Day in November 2011 and in this mailing list there has been some discussion about introducing in Buildroot the possibility to derive automatically legally relevant material, such as licensing info and source tarballs for open source packages.
This is a first tentative implementation of these features.
This code is not yet fully working, but I wanted to share it with you to receive some early comments. I also wanted to allow those who will participate to the next Buildroot Developer Day next Friday to have an idea of this proposal in advance, in case this were object of discussion during the meeting.
My approach is based on the creation of a per-package
_LICENSEconstant in eack .mk file, such as:
FOOBAR_LICENSE=GPL_V3
MYAPP_LICENSE= PROPRIETARY
This is the only effort required to the package creator. Where it is not specified it defaults to "unknown".My overall goal is to add a new 'make legal-info' target that:
- produces a manifest file listing all packages, including closed-source ones (and, why not, prints out to stdout the same info);
- copies source code tarballs for all non-proprietary packages (although I'm saving also tarballs for BSD-licensed packages for now);
- save the complete text of all license files that must be included in the product documentation (typically COPYING and LICENSE files); this is to simplify the work of who must provide these info to documentation writers.This patchset implements the first two points, albeit with some issues that I discuss below.
About the third point I devised a few different possible implementations, but each of them has relevant drawbacks, so I left this point out for the moment.Here's what is currently implmented:
$ make legal-info busybox 1.19.3 GPL_V2_ONLY bzip2 1.0.5 BSD directfb 1.4.15 LGPL_V2.1 foobar 1.2.3.4 PROPRIETARY freetype 2.4.8 unknown iostat 2.2 GPL_V2 ... $ cat output/legal-info/manifest.csv package,version,license busybox,1.19.3,GPL_V2_ONLY bzip2,1.0.5,BSD directfb,1.4.15,LGPL_V2.1 foobar,1.2.3.4,PROPRIETARY freetype,2.4.8,unknown iostat,2.2,GPL_V2 ... $ ls output/legal-info/sources/ autoconf-2.65.tar.bz2
Quoted 8 lines
automake-1.11.1.tar.bz2 binutils-2.21.1.tar.bz2 busybox-1.19.3.tar.bz2 bzip2-1.0.5.tar.gz DirectFB-1.4.15.tar.gz fakeroot_1.9.5.tar.gz freetype-2.4.8.tar.bz2 ...
That's a pretty nice feature! Great!
However, I can see something missing for GPL/LGPL packages. GPL/LGPL states that you must also provide "the scripts used to control compilation and installation of the executable."
Which means that, for packages such as Linux, busybox and uClibc (maybe others as well), the associated .config file should be bundled as well.
Also, the config/build/install instructions for each GPL/LGPL package must be provided. This could probably be done by bundling the buildroot sources too in output/legal-info/sources/, or by leveraging the package infrastructure to output the executed commands for every packages.
Also, for packages that get patches applied by buildroot, you must make sure that the tarballs for those packages do contain the pathced code, or that the patches are bundled as well. From what I see, you currently only copy the downladed tarballs. Of course, if buildroot is also copied to the output/legal-info/sources/ the patches will be there.
My 2 cents...
Regards,
Yann E. MORIN.
Signature · 6 lines
.-----------------.--------------------.------------------.--------------------. | Yann E. MORIN | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: | | +33 662 376 056 | Software Designer | \ / CAMPAIGN | ___ | | +33 223 225 172 `------------.-------: X AGAINST | \e/ There is no | | http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL | v conspiracy. | '------------------------------^-------^------------------^--------------------'
cover Automatically produce legal compliance info
Hi Luca,
Quoted 81 lines · Yann E. MORIN
Luca, All,
Quoted 55 lines · On Sunday 29 January 2012 16:11:33 Luca Ceresoli wrote:
during the latest Buildroot Developers Day in November 2011 and in this mailing list there has been some discussion about introducing in Buildroot the possibility to derive automatically legally relevant material, such as licensing info and source tarballs for open source packages.
This is a first tentative implementation of these features.
This code is not yet fully working, but I wanted to share it with you to receive some early comments. I also wanted to allow those who will participate to the next Buildroot Developer Day next Friday to have an idea of this proposal in advance, in case this were object of discussion during the meeting.
My approach is based on the creation of a per-package
_LICENSEconstant in eack .mk file, such as:
?FOOBAR_LICENSE=GPL_V3
?MYAPP_LICENSE= PROPRIETARY
This is the only effort required to the package creator. Where it is not specified it defaults to "unknown".My overall goal is to add a new 'make legal-info' target that:
?- produces a manifest file listing all packages, including closed-source ones ? ?(and, why not, prints out to stdout the same info);
?- copies source code tarballs for all non-proprietary packages (although I'm ? ?saving also tarballs for BSD-licensed packages for now);
?- save the complete text of all license files that must be included in the ? ?product documentation (typically COPYING and LICENSE files); this is to ? ?simplify the work of who must provide these info to documentation writers.This patchset implements the first two points, albeit with some issues that I discuss below.
About the third point I devised a few different possible implementations, but each of them has relevant drawbacks, so I left this point out for the moment.Here's what is currently implmented:
? $ make legal-info
? busybox ?1.19.3 ? ? ?GPL_V2_ONLY
? bzip2 ? ?1.0.5 ? ? ? ?BSD
? directfb 1.4.15 ? ? ? LGPL_V2.1
? foobar ? 1.2.3.4 ? ? ?PROPRIETARY
? freetype 2.4.8 ? ? ? ?unknown
? iostat ? 2.2 ? ? ? ? ?GPL_V2
? ...
? $ cat output/legal-info/manifest.csv
? package,version,license
? busybox,1.19.3,GPL_V2_ONLY
? bzip2,1.0.5,BSD
? directfb,1.4.15,LGPL_V2.1
? foobar,1.2.3.4,PROPRIETARY
? freetype,2.4.8,unknown
? iostat,2.2,GPL_V2
? ...
? $ ls output/legal-info/sources/
? autoconf-2.65.tar.bz2Quoted 8 lines
? automake-1.11.1.tar.bz2
? binutils-2.21.1.tar.bz2
? busybox-1.19.3.tar.bz2
? bzip2-1.0.5.tar.gz
? DirectFB-1.4.15.tar.gz
? fakeroot_1.9.5.tar.gz
? freetype-2.4.8.tar.bz2
? ...That's a pretty nice feature! Great!
However, I can see something missing for GPL/LGPL packages. GPL/LGPL states that you must also provide "the scripts used to control compilation and installation of the executable."
Which means that, for packages such as Linux, busybox and uClibc (maybe others as well), the associated .config file should be bundled as well.
Also, the config/build/install instructions for each GPL/LGPL package must be provided. This could probably be done by bundling the buildroot sources too in output/legal-info/sources/, or by leveraging the package infrastructure to output the executed commands for every packages.
Quoted 5 lines
Also, for packages that get patches applied by buildroot, you must make sure that the tarballs for those packages do contain the pathced code, or that the patches are bundled as well. From what I see, you currently only copy the downladed tarballs. Of course, if buildroot is also copied to the output/legal-info/sources/ the patches will be there.
I agree with Yann: I think we should package buildroot itself as well.
In fact, I think we should:
- make distclean
- create the manifest
- download all needed sources
- run a pre-legal-package script for customization
- create a .tar file with the whole
The pre-legal-package script (whatever the name) is similar to the post-build script, and allows projects to modify certain things. For example, modify the defconfig to disable some proprietary things that are not useful or usable by the customer.
Also, I think the DL_DIR setting should be modified so that it points to the location where the source tarballs are downloaded. This way, the customer can directly use these sources from that location when re-building buildroot.
I haven't done a technical review of your patches, but I went through them and agree with the principle. Thanks for posting them before the developer day, it will certainly help the discussion!
Best regards,
Thomas
cover Automatically produce legal compliance info
Yann, Thomas,
thanks for your follow-ups.
Quoted 33 lines · Thomas De Schampheleire
Quoted 13 lines
However, I can see something missing for GPL/LGPL packages. GPL/LGPL states that you must also provide "the scripts used to control compilation and installation of the executable."
Which means that, for packages such as Linux, busybox and uClibc (maybe others as well), the associated .config file should be bundled as well.
Also, the config/build/install instructions for each GPL/LGPL package must be provided. This could probably be done by bundling the buildroot sources too in output/legal-info/sources/, or by leveraging the package infrastructure to output the executed commands for every packages.
Quoted 5 lines
Also, for packages that get patches applied by buildroot, you must make sure that the tarballs for those packages do contain the pathced code, or that the patches are bundled as well. From what I see, you currently only copy the downladed tarballs. Of course, if buildroot is also copied to the output/legal-info/sources/ the patches will be there.
I agree with Yann: I think we should package buildroot itself as well.
In fact, I think we should:
- make distclean
- create the manifest
- download all needed sources
- run a pre-legal-package script for customization
- create a .tar file with the wholeThe pre-legal-package script (whatever the name) is similar to the post-build script, and allows projects to modify certain things. For example, modify the defconfig to disable some proprietary things that are not useful or usable by the customer.
Quoted 8 lines
Also, I think the
DL_DIRsetting should be modified so that it points to the location where the source tarballs are downloaded. This way, the customer can directly use these sources from that location when re-building buildroot.I haven't done a technical review of your patches, but I went through them and agree with the principle. Thanks for posting them before the developer day, it will certainly help the discussion!
I think the best way is to just package BR itself in the legal-info subdir. I'll have to check if/how it is feasible.
I'd also copy the current .config, which IMHO is part of the "scripts used to control compilation".
OTOH I don't think a pre-legal-package script would be a good idea, as it would easily allow to trick and create a fake GPL-compliant release. In other words, what I use for building must go directly in the stuff to be released.
Luca
cover Automatically produce legal compliance info
Quoted 56 lines · Luca Ceresoli
Yann, Thomas,
thanks for your follow-ups.Quoted 36 lines · Thomas De Schampheleire
Quoted 15 lines
However, I can see something missing for GPL/LGPL packages. GPL/LGPL states
that you must also provide "the scripts used to control compilation and installation of the executable."Which means that, for packages such as Linux, busybox and uClibc (maybe others as well), the associated .config file should be bundled as well.
Also, the config/build/install instructions for each GPL/LGPL package must be provided. This could probably be done by bundling the buildroot sources too in output/legal-info/sources/, or by leveraging the package infrastructure to output the executed commands for every packages.
Quoted 5 lines
Also, for packages that get patches applied by buildroot, you must make sure that the tarballs for those packages do contain the pathced code, or that the patches are bundled as well. From what I see, you currently only copy the downladed tarballs. Of course, if buildroot is also copied to the output/legal-info/sources/ the patches will be there.
I agree with Yann: I think we should package buildroot itself as well.
In fact, I think we should:
- make distclean
- create the manifest
- download all needed sources
- run a pre-legal-package script for customization
- create a .tar file with the wholeThe pre-legal-package script (whatever the name) is similar to the post-build script, and allows projects to modify certain things. For example, modify the defconfig to disable some proprietary things that are not useful or usable by the customer.
Quoted 8 lines
Also, I think the
DL_DIRsetting should be modified so that it points to the location where the source tarballs are downloaded. This way, the customer can directly use these sources from that location when re-building buildroot.I haven't done a technical review of your patches, but I went through them and agree with the principle. Thanks for posting them before the developer day, it will certainly help the discussion!
I think the best way is to just package BR itself in the legal-info subdir. I'll have to check if/how it is feasible.
I'd also copy the current .config, which IMHO is part of the "scripts used to control compilation".
Quoted 4 lines
OTOH I don't think a pre-legal-package script would be a good idea, as it would easily allow to trick and create a fake GPL-compliant release. In other words, what I use for building must go directly in the stuff to be released.
I don't think we should attempt a guaranteed GPL-compliant release. We have no control over what someone did with its buildroot repository, whether or not they modified the LICENSE variables etc. A malicious company will be able to create a non-compliant tarball in many ways. By modifying buildroot itself, by using a pre-legal-package script, but also by post-processing the tarball. It's not something we should try to guard against IMO.
Moreover, I think there are valid use cases. For example, in our case, we build a proprietary package from within buildroot. This means we need the sources available. A customer shouldn't get these sources. But our .config or defconfig does enable the package. As a result, if the customer does 'make', there will be an attempt to find the proprietary sources, and the build will fail.
A pre-legal-package script could make such modifications to the defconfig file: disable the proprietary packages.
Another use cas is where the pre-legal-package script could copy in binary proprietary applications from elsewhere.
Or add a README file specific for the project.
Or maybe modify the login banner to indicate in the target image that this is no official release, but one built by the customer. Or ...
None of these are attempts to trick the customer or to be deviant from GPL or other license obligations.
Best regards,
Thomas
cover Automatically produce legal compliance info
Quoted 33 lines · Thomas De Schampheleire
Quoted 9 lines
I think the best way is to just package BR itself in the legal-info subdir. I'll have to check if/how it is feasible.
I'd also copy the current .config, which IMHO is part of the "scripts used to control compilation".OTOH I don't think a pre-legal-package script would be a good idea, as it would easily allow to trick and create a fake GPL-compliant release. In other words, what I use for building must go directly in the stuff to be released.
I don't think we should attempt a guaranteed GPL-compliant release. We have no control over what someone did with its buildroot repository, whether or not they modified the LICENSE variables etc. A malicious company will be able to create a non-compliant tarball in many ways. By modifying buildroot itself, by using a pre-legal-package script, but also by post-processing the tarball. It's not something we should try to guard against IMO.
Moreover, I think there are valid use cases. For example, in our case, we build a proprietary package from within buildroot. This means we need the sources available. A customer shouldn't get these sources. But our .config or defconfig does enable the package. As a result, if the customer does 'make', there will be an attempt to find the proprietary sources, and the build will fail.
A pre-legal-package script could make such modifications to the defconfig file: disable the proprietary packages.
Another use cas is where the pre-legal-package script could copy in binary proprietary applications from elsewhere.
Or add a README file specific for the project.
Or maybe modify the login banner to indicate in the target image that this is no official release, but one built by the customer. Or ...
Quoted 2 lines
None of these are attempts to trick the customer or to be deviant from GPL or other license obligations.
After having slept on it, and after your clarifications, I think you're probably right.
On one hand, it's not feasible to have Buildroot enforce a compliant release, and I would not take the risk of saying it does even if it did.
On the other hand, there are ways a pre-legal-package script may be useful, as your examples demonstrate.
By the law you must release the sources, but not necessarily make them straightforward to use. So to fulfill legal obligations you probably don't need such a script (provided that BR can package all the code that must be released).
But there are also FOSS-friendly device vendors that might want to sell a product with some proprietary software on them, while releasing the open-source part in a way that is fully usable except for the absence of the proprietary parts.
Bottom line: I think the script you suggest would be useful, but non strictly necessary. So I consider it something that might be added later, unless it's a simple addition to the current patchset. I'd prefer to keep the first patchset thin, for better review/discussion and to ease Peter's integration work.
Luca
cover Automatically produce legal compliance info
Hi Luca,
Quoted 59 lines · Luca Ceresoli
Quoted 35 lines · Thomas De Schampheleire
Quoted 10 lines
I think the best way is to just package BR itself in the legal-info subdir. I'll have to check if/how it is feasible.
I'd also copy the current .config, which IMHO is part of the "scripts used to control compilation".OTOH I don't think a pre-legal-package script would be a good idea, as it would easily allow to trick and create a fake GPL-compliant release. In other words, what I use for building must go directly in the stuff to be released.
I don't think we should attempt a guaranteed GPL-compliant release. We have no control over what someone did with its buildroot repository, whether or not they modified the LICENSE variables etc. A malicious company will be able to create a non-compliant tarball in many ways. By modifying buildroot itself, by using a pre-legal-package script, but also by post-processing the tarball. It's not something we should try to guard against IMO.
Moreover, I think there are valid use cases. For example, in our case, we build a proprietary package from within buildroot. This means we need the sources available. A customer shouldn't get these sources. But our .config or defconfig does enable the package. As a result, if the customer does 'make', there will be an attempt to find the proprietary sources, and the build will fail.
A pre-legal-package script could make such modifications to the defconfig file: disable the proprietary packages.
Another use cas is where the pre-legal-package script could copy in binary proprietary applications from elsewhere.
Or add a README file specific for the project.
Or maybe modify the login banner to indicate in the target image that this is no official release, but one built by the customer. Or ...Quoted 2 lines
None of these are attempts to trick the customer or to be deviant from GPL or other license obligations.
After having slept on it, and after your clarifications, I think you're probably right.
On one hand, it's not feasible to have Buildroot enforce a compliant release, and I would not take the risk of saying it does even if it did.
On the other hand, there are ways a pre-legal-package script may be useful, as your examples demonstrate.
By the law you must release the sources, but not necessarily make them straightforward to use. So to fulfill legal obligations you probably don't need such a script (provided that BR can package all the code that must be released).
But there are also FOSS-friendly device vendors that might want to sell a product with some proprietary software on them, while releasing the open-source part in a way that is fully usable except for the absence of the proprietary parts.
Quoted 5 lines
Bottom line: I think the script you suggest would be useful, but non strictly necessary. So I consider it something that might be added later, unless it's a simple addition to the current patchset. I'd prefer to keep the first patchset thin, for better review/discussion and to ease Peter's integration work.
Ok, I understand.
Let's keep the script out of the patchset for now. We can add it later.
Best regards,
Thomas