Skip to contentSkip to conversation

Support SBOM in CycloneDX format

RFC PATCH In review Thomas Perale via buildroot 2 participants 17 messages Apr 4, 2024 – Apr 10, 2024 Kbuild

lore
Share & tools
lore
#0 lore raw

cover Support SBOM in CycloneDX format

This RFC patch series propose to add support for the CycloneDX SBOM format.

There is a growing need to generate SBOM from buildroot configurations. Right now, there are different solutions available for buildroot users show-info, legal-info and pkg-stats. They all generate similar information (show-info showing more) but in a format that is specific to buildroot.

CycloneDX is a format already supported by tools such as
https://dependencytrack.org/ that helps track software, vulnerabilities, etc ...

A DependencyTrack instance has been deployed at https://dependencytrack-lqgs7zhwmq-ew.a.run.app/
so you can play with CycloneDX and see the possibility. Be mindful
that I used a Google Cloud instance with cold start enabled so it can
take several minutes to initialize if no one has used the service.
Login with username buildroot and password buildroot.
All the project you create will be publicly accessible and removable,
just so you know.

To generate a CycloneDX SBOM compatible with DependencyTrack I found out that one line minified JSON does not work and I had to format it. I used the tool json_reformat to do it but tools are also available online. I use the following command:

make cyclonedx | json_reformat > sbom.json

This is a first sketch and I hope to gather comments on functionality the community want me to include. I already have a todo list of feature I plan to work on:

- [ ] Find a solution to handle versioning. The "version" property should be incremented every SBOM generation.
- [ ] Add more buildroot 'property' to components
- [ ] Infrastructure (cmake, cargo etc).
- [ ] Match the column of the pkg-stat command and add the missing one that don't have a place in the current CycloneDX spec to a corresponding property
- [ ] Make it configurable like pkg-stat
- [ ] Use PACKAGES_ALL if no .config
- [ ] Fix licenses list if there are commas inside parenthesis. Example

      of wrong license list:
              {
                  "license": {
                      "name": "LGPL-2.1+ (libblkid"
                  }
              },
              {
                  "license": {
                      "name": "libfdisk"
                  }
              },
              {
                  "license": {
                      "name": "libmount)"
                  }
              },

- [ ] Introduce a variable _UPSTREAM_NAME that defaults to _RAWNAME. For most python packages for instance, buildroot python- prefix shouldn't be in the upstream name. This _UPSTREAM_NAME can also be used e.g. for the default of CPE, PURL, and maybe in other places too.
- [ ] For patches downloaded using _PATCH variable (rarely used). Modify the pedigree to specify the URL instead of writting the content.
- [ ] Include the list of upstream URL with externalReferences the type will be distribution
https://cyclonedx.org/docs/1.5/json/#components_items_externalReferences
- [ ] Add rule to the gitlab ci that check the SPDX licenses are up to date.
- [ ] Upstream changes to Dependency Track
- [ ] Support 1-line minified JSON
- [ ] Support showing patches and property

Thank you !

Thomas Perale (5):
package/pkg-generic.mk: add PURL package variable package/pkg-utils.mk: urlencode/urldecode macros support/misc/cyclonedx.mk: support CycloneDX format support/misc/cyclonedx.mk: support spdx license check
Makefile: add command to generate SBOM in CycloneDX format

 Makefile                       |  13 +
 package/pkg-generic.mk         |  12 +
 package/pkg-utils.mk           |  12 +
 support/misc/cyclonedx-spdx.mk | 617 +++++++++++++++++++++++++++++++++
 support/misc/cyclonedx.mk      | 230 ++++++++++++
 5 files changed, 884 insertions(+)
 create mode 100644 support/misc/cyclonedx-spdx.mk
 create mode 100644 support/misc/cyclonedx.mk

--
2.44.0

_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot

#1 lore raw

RFC PATCH 1/5 package/pkg-generic.mk: add PURL package variable

PURL stands for 'package URL', it's a specification that standardize how packages are identified and located.

PURL is used to reference the same package across different package manager, tracking tools, API and databases.

A purl is a URL composed of seven components:

scheme:type/namespace/name@version?qualifiers#subpath

- scheme: always 'pkg' (required)
- type: package manager used to install the package, download origin, type of package (required)
- namespace: name prefix, type specific additional information (optional)
- name: package name (required)
- version: package version (optional)
- qualifiers: extra information (optional)
- subpath: extra subpath relative to package root (optional)

A PURL for the purl-spec repository looks like this:

pkg:github/package-url/purl-spec@346589846130317464b677bc4eab30bf5040183a

It contains information like the provenance (github), organization (package-url), name (purl-spec) and version (34658984...).

This patch introduces an auto-generated PURL for non internal packages (packages with a <pkg>_SOURCE variable), with the possibility for each package to define their own PURL by defining the following variable:

<pkg>_PURL

If the variable is not defined it will be generated as follows:

  $(2)_PURL = pkg:generic/$$($(2)_RAWNAME)@$$($(2)_VERSION)

The type 'generic' is used by default but can be extended in the future to support github, gitlab, etc ...

For more information, see https://github.com/package-url/purl-spec

---
 package/pkg-generic.mk | 12 ++++++++++++
 1 file changed, 12 insertions(+)
package/pkg-generic.mk +12 -0
index 577a148c1e..0966b714ff 100644
@@ -754,6 +754,18 @@ ifeq ($$($(2)_CPE_ID_VALID),YES)
  $(2)_CPE_ID = $$($(2)_CPE_ID_PREFIX):$$($(2)_CPE_ID_VENDOR):$$($(2)_CPE_ID_PRODUCT):$$($(2)_CPE_ID_VERSION):$$($(2)_CPE_ID_UPDATE):*:*:*:*:*:*
 endif # ifeq ($$($(2)_CPE_ID_VALID),YES)
 
+# If no package url (purl) is set, a generic purl is created for non internal
+# packages.
+# see https://github.com/package-url/purl-spec
+ifndef $(2)_PURL
+ ifdef $(3)_PURL
+  $(2)_PURL = $$($(3)_PURL)
+ endif
+ ifdef $(2)_SOURCE
+   $(2)_PURL = pkg:generic/$$($(2)_RAWNAME)@$$($(2)_VERSION)
+ endif
+endif
+
 # When a target package is a toolchain dependency set this variable to
 # 'NO' so the 'toolchain' dependency is not added to prevent a circular
 # dependency.

--
2.44.0

_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot

#2 lore raw

RFC PATCH 2/5 package/pkg-utils.mk: urlencode/urldecode macros

In reply to Thomas Perale via buildroot

This patch introduces two new macros:

- urlencode
- urldecode

URL encoding consists of converting ASCII characters into a percent symbol followed by a two digit hexadecimal code. And the other way around for URL decoding.

The macros encode/decode a string passed as an argument by escaping the following characters:

- '%' is replaced by %25
- 'space' is replaced by %20

The characters covered by this patch is non exhaustive.

Because the Make language treats spaces as different entry of a list, the aim of those macros is to provide a well known encoding method to escape text containing spaces into a string that won't be treated as a list by the Make language.

---
 package/pkg-utils.mk | 12 ++++++++++++
 1 file changed, 12 insertions(+)
package/pkg-utils.mk +12 -0
index 723bbe4e24..b8cfb85fca 100644
@@ -329,3 +329,15 @@ define NXP_EXTRACT_HELPER
 	find $(@D)/$(basename $(notdir $(1))) -mindepth 1 -maxdepth 1 -exec mv {} $(@D) \;
 	rmdir $(@D)/$(basename $(notdir $(1)))
 endef
+
+# urlencode -- returns an url encoded string.
+#   - encode % into %25
+#   - encode spaces into %20
+#
+# $(1): text
+urlencode = $(subst $(space),%20,$(subst %,%25,$(1)))
+
+# urldecode -- decode an url encoded string.
+#
+# $(1): text
+urldecode = $(subst %25,%,$(subst %20,$(space),$(1)))

--
2.44.0

_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot

#3 lore raw

Re: [Buildroot] [RFC PATCH 2/5] package/pkg-utils.mk: urlencode/urldecode macros

Thomas, All,

On 2024-04-04 14:43 +0200, Thomas Perale via buildroot spake thusly:

Quoted 9 lines

This patch introduces two new macros:

- urlencode
- urldecode

URL encoding consists of converting ASCII characters into a percent symbol followed by a two digit hexadecimal code. And the other way around for URL decoding.

Quoted 7 lines

The macros encode/decode a string passed as an argument by escaping the following characters:

- '%' is replaced by %25
- 'space' is replaced by %20

The characters covered by this patch is non exhaustive.

Not sure what you meant with this sentence... Can you elaborate? Or did you mean that we can add more chars in the future?

Regards,
Yann E. MORIN.

Quoted 37 lines

Because the Make language treats spaces as different entry of a list, the aim of those macros is to provide a well known encoding method to escape text containing spaces into a string that won't be treated as a list by the Make language.

---
 package/pkg-utils.mk | 12 ++++++++++++
 1 file changed, 12 insertions(+)
package/pkg-utils.mk +12 -1
index 723bbe4e24..b8cfb85fca 100644
@@ -329,3 +329,15 @@ define NXP_EXTRACT_HELPER
 	find $(@D)/$(basename $(notdir $(1))) -mindepth 1 -maxdepth 1 -exec mv {} $(@D) \;
 	rmdir $(@D)/$(basename $(notdir $(1)))
 endef
+
+# urlencode -- returns an url encoded string.
+#   - encode % into %25
+#   - encode spaces into %20
+#
+# $(1): text
+urlencode = $(subst $(space),%20,$(subst %,%25,$(1)))
+
+# urldecode -- decode an url encoded string.
+#
+# $(1): text
+urldecode = $(subst %25,%,$(subst %20,$(space),$(1)))
--

2.44.0

_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot

Signature · 10 lines
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot
#4 lore raw

Re: [Buildroot] [RFC PATCH 2/5] package/pkg-utils.mk: urlencode/urldecode macros

Quoted 15 lines · Yann E. MORIN

Thomas, All,

On 2024-04-04 14:43 +0200, Thomas Perale via buildroot spake thusly:

Quoted 12 lines

This patch introduces two new macros:

- urlencode
- urldecode

URL encoding consists of converting ASCII characters into a percent symbol followed by a two digit hexadecimal code. And the other way around for URL decoding.

The macros encode/decode a string passed as an argument by escaping the following characters:

Quoted 6 lines
Quoted 4 lines

- '%' is replaced by %25
- 'space' is replaced by %20

The characters covered by this patch is non exhaustive.

Not sure what you meant with this sentence... Can you elaborate?

It means that not all characters that normally are urlencoded (like /, :, #)
are handled by this patch.

Quoted 1 line

Or did you mean that we can add more chars in the future?

So yes, that.

Regards,
Arnout

Quoted 37 lines

Regards,
Yann E. MORIN.

Quoted 33 lines

Because the Make language treats spaces as different entry of a list, the aim of those macros is to provide a well known encoding method to escape text containing spaces into a string that won't be treated as a list by the Make language.

---
  package/pkg-utils.mk | 12 ++++++++++++
  1 file changed, 12 insertions(+)
package/pkg-utils.mk +12 -1
index 723bbe4e24..b8cfb85fca 100644
@@ -329,3 +329,15 @@ define NXP_EXTRACT_HELPER
  	find $(@D)/$(basename $(notdir $(1))) -mindepth 1 -maxdepth 1 -exec mv {} $(@D) \;
  	rmdir $(@D)/$(basename $(notdir $(1)))
  endef
+
+# urlencode -- returns an url encoded string.
+#   - encode % into %25
+#   - encode spaces into %20
+#
+# $(1): text
+urlencode = $(subst $(space),%20,$(subst %,%25,$(1)))
+
+# urldecode -- decode an url encoded string.
+#
+# $(1): text
+urldecode = $(subst %25,%,$(subst %20,$(space),$(1)))
--

2.44.0

Quoted 5 lines
Quoted 4 lines

_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot

_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot

#5 lore raw

RFC PATCH 3/5 support/misc/cyclonedx.mk: support CycloneDX format

In reply to Thomas Perale via buildroot

CycloneDX is a software bill of materials (SBOM) specification.

There is a growing need to generate SBOM from buildroot configurations. Right now there are different solutions available for buildroot users show-info, legal-info and pkg-stats. They all generate similar information (show-info showing more) but in a format that is specific to buildroot.

This is the reason this patch introduces a new SBOM output type for
buildroot: CycloneDX. CycloneDX is a format already supported by tools such as https://dependencytrack.org/ that helps track softwares, vulnerabilities, etc ...

To match the functionality of show-info, buildroot internal packages will also be present in the SBOM with a reduced set of property. Internal packages are defined as packages without <pkg>_SOURCE defined.
In a future patch more properties can be added to cover the functionality of show-info, legal-info and pkg-stats.

The CycloneDX SBOM output as a stripped JSON formatted line as there are already macros available to work with JSON in buildroot.

For more information, see https://cyclonedx.org/
and https://cyclonedx.org/docs/1.5/json/

---
 support/misc/cyclonedx.mk | 197 ++++++++++++++++++++++++++++++++++++++
 1 file changed, 197 insertions(+)
 create mode 100644 support/misc/cyclonedx.mk
support/misc/cyclonedx.mk new file +197 -0
new file mode 100644
index 0000000000..3906a3b60a
@@ -0,0 +1,197 @@
+################################################################################
+#
+# This file contains various utility functions used to create a SBOM
+# in the JSON CycloneDX format.
+#
+# https://cyclonedx.org/docs/1.5/json/
+#
+################################################################################
+
+# Note: to avoid conflict with <pkg>_VERSION `_SPEC` is added
+CYCLONEDX_VERSION_SPEC = 1.5
+
+#
+# Licenses list helper functions
+# Since licenses in buildroot are comma separated list and the 'make' language
+# uses spaces to create list we need to replace the spaces of licenses name
+# by a character not used in any licenses name.
+#
+# _cyclonedx-licenses-as-list: create a list from the url-encoded comma
+#                              separeted license list.
+#
+# $(1): an url-encoded comma separeted list
+#
+# Turns "Public%20Domain,%20GPL-2.0" into "Public%20Domain GPL-2.0"
+_cyclonedx-licenses-as-list = $(subst $(comma)%20,$(space),$(1))
+
+# _cyclonedx-license -- create an entry of a cyclonedx component license list
+#
+# For more information on license object see
+# https://cyclonedx.org/docs/1.5/json/#components_items_licenses_oneOf_i0_items_license
+#
+# $(1): a single url-encoded license name
+define _cyclonedx-license
+	{
+		"license": {
+			"name": $(call mk-json-str,$(1))
+		}
+	},
+endef
+
+# _cyclonedx-licenses -- create a licenses list formatted for a CycloneDX
+#                        component
+#
+# $(1): a comma separated license list
+#
+# KNOWN ISSUE: Licenses name that include a parenthesis with comma inside,
+#   will result be misinterpreted as multiple licenses name:
+#   - host-util-linux: LGPL-2.1+ (libblkid, libfdisk, libmount)
+define _cyclonedx-licenses
+	$(foreach license,$(call _cyclonedx-licenses-as-list,$(call urlencode,$(1))),
+		$(call urldecode,$(call _cyclonedx-license,$(license)))
+	)
+endef
+
+# Note about patch list: this patch list might not be complete.
+# There is no variable yet that stores the patch list without applying them.
+_cyclonedx-patches-list = $(foreach patchdir,\
+	$(addsuffix /$($(1)_PKGDIR),$(CURDIR)) $(addsuffix /$($(1)_RAWNAME),$(call qstrip,$(BR2_GLOBAL_PATCH_DIR))),\
+	$(wildcard $(addsuffix /*.patch,$(patchdir)))\
+)
+
+# _cyclonedx-patch -- single entry of a patch list.
+#                     It's required to pass the type of the patch it can be
+#                     either: unofficial, monkey, backport or cherry-pick.
+#                     Since there is no information available about each
+#                     patches, we mark them as "unofficial".
+#
+# $(1): single patch path
+define _cyclonedx-patch
+	{
+		"type": "unofficial",
+		"diff": {
+			"text": {
+				"content": $(call mk-json-str,$(file < $(1)))
+			}
+		}
+	},
+endef
+
+# _cyclonedx-patches -- patch list are stored under the pedigree entry used to
+#                       document how a component is modified.
+#
+# $(1): patch path list
+define _cyclonedx-patches
+	$(intcmp $(words $(1)),0,,,
+		"pedigree": {\
+			"patches": [\
+				$(foreach patch,$(1),\
+					$(call _cyclonedx-patch,$(patch))\
+				)\
+			]\
+		}$(comma)\
+	)
+endef
+
+# _cyclonedx-component -- representation of a package for the CycloneDX format
+#  - bom-ref: is a unique identifier used to refer to this component in the
+#      'dependencies' section.
+#  - type: is a required property since we don't have enough information about
+#      the package from its definition CycloneDX spec recommend setting it
+#      to 'library'.
+#  - properties: is used to add additional information that doesn't fit the
+#      current CycloneDX specification.
+#    - BR_TYPE: {host|target}
+#
+# $(1): upper-case package name
+#
+# KNWON ISSUE: packages with a custom tarball (linux,uboot,...) will have the
+#   'version' property set to 'custom'.
+define _cyclonedx-component
+	{
+		"bom-ref": $(call mk-json-str,$($(1)_NAME)),
+		"name": $(call mk-json-str,$(if $($(1)_RAWNAME),$($(1)_RAWNAME),$($(1)_NAME))),
+		"type": "library",
+		$(if $($(1)_SOURCE),
+			"version": $(call mk-json-str,$($(1)_DL_VERSION))$(comma)
+			"licenses": [
+				$(call _cyclonedx-licenses,$($(1)_LICENSE)) \
+			]$(comma)
+		)
+		$(if $($(1)_PURL), \
+			"purl": $(call mk-json-str,$($(1)_PURL))$(comma) \
+		)
+		$(if $($(1)_CPE_ID_VALID), \
+			"cpe": $(call mk-json-str,$($(1)_CPE_ID))$(comma) \
+		)
+		$(call _cyclonedx-patches,$(call _cyclonedx-patches-list,$(1)))
+		"properties": [{
+			"name": "BR_TYPE",
+			"value": $(call mk-json-str,$($(1)_TYPE))
+		}],
+	},
+endef
+
+# _cyclonedx-dependency -- create dependency relationships between components.
+#  - ref: reference to a component bom-ref.
+#  - dependsOn: array of component bom-ref identifier to create the dependencies.
+#
+# $(1): upper-case package name
+define _cyclonedx-dependency
+	$(if $($(1)_FINAL_RECURSIVE_DEPENDENCIES),
+		{
+			"ref": $(call mk-json-str,$($(1)_NAME))$(comma)
+			"dependsOn": [
+				$(call make-comma-list,$(foreach p,\
+					$($(1)_FINAL_RECURSIVE_DEPENDENCIES), \
+					$(call mk-json-str,$(p))\
+				))
+			]
+		}$(comma)
+	)
+endef
+
+# cyclonedx-json -- return a CycloneDX SBOM formatted as a JSON dictionnary.
+#  - bomFormat: required field is always "CycloneDX"
+#  - specVersion: required field with CycloneDX spec version
+#  - version: is used by software that accept CycloneDX SBOM to differentiate
+#    the different SBOM. The bigger the number the newer the SBOM is.
+#    Here it's set to '1' and it should be incremented when the resulting
+#    SBOM is edited later.
+#    The CycloneDX spec mentions that an 'uuid' property can also be used to
+#    differentiate SBOM but is not included because there is no native command
+#    to generate an uuid in buildroot.
+#
+# $(1): packages list
+define cyclonedx-json
+	$(call clean-json,{
+		"bomFormat": "CycloneDX"$(comma)
+		"$$schema": "http://cyclonedx.org/schema/bom-$(CYCLONEDX_VERSION_SPEC).schema.json"$(comma)
+		"specVersion": $(call mk-json-str,$(CYCLONEDX_VERSION_SPEC))$(comma)
+		"version": 1$(comma)
+		"components": [ \
+			$(foreach p,$(1), \
+				$(call _cyclonedx-component,$(call UPPERCASE,$(p))) \
+			) \
+		]$(comma)
+		"dependencies": [
+			{
+				"ref": "buildroot"$(comma)\
+				"dependsOn": [$(call make-comma-list,\
+					$(foreach p,$(1),$(call mk-json-str,$(p)))\
+				)]\
+			}$(comma)
+			$(foreach p,$(1),\
+				$(call _cyclonedx-dependency,$(call UPPERCASE,$(p)),$(2)) \
+			) \
+		]$(comma)
+		"metadata": {
+			"component": {
+				"bom-ref": "buildroot"$(comma)
+				"name": "buildroot"$(comma)
+				"type": "firmware"$(comma)
+				"version": $(call mk-json-str,$(BR2_VERSION_FULL))
+			}
+		}
+	})
+endef

--
2.44.0

_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot

#6 lore raw

RFC PATCH 4/5 support/misc/cyclonedx.mk: support spdx license check

In reply to Thomas Perale via buildroot

To improve the tracking of the licenses of the components, the file support/misc/cyclonedx-spdx.mk that contains a definition of every approved SPDX licenses ID is included in this patch.

This file has been generated by the support/misc/cyclonedx-spdx.mk rule included support/misc/cyclonedx.mk. It will remain there to re-generate the file if it's updated.

Knowing if a license name is a valid SPDX ID or not, allows tools such as Dependency Track to directly show the license content of components with a known SPDX ID.

---
 support/misc/cyclonedx-spdx.mk | 617 +++++++++++++++++++++++++++++++++
 support/misc/cyclonedx.mk      |  35 +-
 2 files changed, 651 insertions(+), 1 deletion(-)
 create mode 100644 support/misc/cyclonedx-spdx.mk
support/misc/cyclonedx-spdx.mk new file +617 -0
new file mode 100644
index 0000000000..81c387fd23
@@ -0,0 +1,617 @@
+# List of approved SPDX license
+# See https://raw.githubusercontent.com/CycloneDX/specification/1.5/schema/spdx.schema.json
+define spdx
+0BSD
+AAL
+Abstyles
+AdaCore-doc
+Adobe-2006
+Adobe-Glyph
+ADSL
+AFL-1.1
+AFL-1.2
+AFL-2.0
+AFL-2.1
+AFL-3.0
+Afmparse
+AGPL-1.0
+AGPL-1.0-only
+AGPL-1.0-or-later
+AGPL-3.0
+AGPL-3.0-only
+AGPL-3.0-or-later
+Aladdin
+AMDPLPA
+AML
+AMPAS
+ANTLR-PD
+ANTLR-PD-fallback
+Apache-1.0
+Apache-1.1
+Apache-2.0
+APAFML
+APL-1.0
+App-s2p
+APSL-1.0
+APSL-1.1
+APSL-1.2
+APSL-2.0
+Arphic-1999
+Artistic-1.0
+Artistic-1.0-cl8
+Artistic-1.0-Perl
+Artistic-2.0
+ASWF-Digital-Assets-1.0
+ASWF-Digital-Assets-1.1
+Baekmuk
+Bahyph
+Barr
+Beerware
+Bitstream-Charter
+Bitstream-Vera
+BitTorrent-1.0
+BitTorrent-1.1
+blessing
+BlueOak-1.0.0
+Boehm-GC
+Borceux
+Brian-Gladman-3-Clause
+BSD-1-Clause
+BSD-2-Clause
+BSD-2-Clause-FreeBSD
+BSD-2-Clause-NetBSD
+BSD-2-Clause-Patent
+BSD-2-Clause-Views
+BSD-3-Clause
+BSD-3-Clause-Attribution
+BSD-3-Clause-Clear
+BSD-3-Clause-LBNL
+BSD-3-Clause-Modification
+BSD-3-Clause-No-Military-License
+BSD-3-Clause-No-Nuclear-License
+BSD-3-Clause-No-Nuclear-License-2014
+BSD-3-Clause-No-Nuclear-Warranty
+BSD-3-Clause-Open-MPI
+BSD-4-Clause
+BSD-4-Clause-Shortened
+BSD-4-Clause-UC
+BSD-4.3RENO
+BSD-4.3TAHOE
+BSD-Advertising-Acknowledgement
+BSD-Attribution-HPND-disclaimer
+BSD-Protection
+BSD-Source-Code
+BSL-1.0
+BUSL-1.1
+bzip2-1.0.5
+bzip2-1.0.6
+C-UDA-1.0
+CAL-1.0
+CAL-1.0-Combined-Work-Exception
+Caldera
+CATOSL-1.1
+CC-BY-1.0
+CC-BY-2.0
+CC-BY-2.5
+CC-BY-2.5-AU
+CC-BY-3.0
+CC-BY-3.0-AT
+CC-BY-3.0-DE
+CC-BY-3.0-IGO
+CC-BY-3.0-NL
+CC-BY-3.0-US
+CC-BY-4.0
+CC-BY-NC-1.0
+CC-BY-NC-2.0
+CC-BY-NC-2.5
+CC-BY-NC-3.0
+CC-BY-NC-3.0-DE
+CC-BY-NC-4.0
+CC-BY-NC-ND-1.0
+CC-BY-NC-ND-2.0
+CC-BY-NC-ND-2.5
+CC-BY-NC-ND-3.0
+CC-BY-NC-ND-3.0-DE
+CC-BY-NC-ND-3.0-IGO
+CC-BY-NC-ND-4.0
+CC-BY-NC-SA-1.0
+CC-BY-NC-SA-2.0
+CC-BY-NC-SA-2.0-DE
+CC-BY-NC-SA-2.0-FR
+CC-BY-NC-SA-2.0-UK
+CC-BY-NC-SA-2.5
+CC-BY-NC-SA-3.0
+CC-BY-NC-SA-3.0-DE
+CC-BY-NC-SA-3.0-IGO
+CC-BY-NC-SA-4.0
+CC-BY-ND-1.0
+CC-BY-ND-2.0
+CC-BY-ND-2.5
+CC-BY-ND-3.0
+CC-BY-ND-3.0-DE
+CC-BY-ND-4.0
+CC-BY-SA-1.0
+CC-BY-SA-2.0
+CC-BY-SA-2.0-UK
+CC-BY-SA-2.1-JP
+CC-BY-SA-2.5
+CC-BY-SA-3.0
+CC-BY-SA-3.0-AT
+CC-BY-SA-3.0-DE
+CC-BY-SA-3.0-IGO
+CC-BY-SA-4.0
+CC-PDDC
+CC0-1.0
+CDDL-1.0
+CDDL-1.1
+CDL-1.0
+CDLA-Permissive-1.0
+CDLA-Permissive-2.0
+CDLA-Sharing-1.0
+CECILL-1.0
+CECILL-1.1
+CECILL-2.0
+CECILL-2.1
+CECILL-B
+CECILL-C
+CERN-OHL-1.1
+CERN-OHL-1.2
+CERN-OHL-P-2.0
+CERN-OHL-S-2.0
+CERN-OHL-W-2.0
+CFITSIO
+checkmk
+ClArtistic
+Clips
+CMU-Mach
+CNRI-Jython
+CNRI-Python
+CNRI-Python-GPL-Compatible
+COIL-1.0
+Community-Spec-1.0
+Condor-1.1
+copyleft-next-0.3.0
+copyleft-next-0.3.1
+Cornell-Lossless-JPEG
+CPAL-1.0
+CPL-1.0
+CPOL-1.02
+Crossword
+CrystalStacker
+CUA-OPL-1.0
+Cube
+curl
+D-FSL-1.0
+diffmark
+DL-DE-BY-2.0
+DOC
+Dotseqn
+DRL-1.0
+DSDP
+dtoa
+dvipdfm
+ECL-1.0
+ECL-2.0
+eCos-2.0
+EFL-1.0
+EFL-2.0
+eGenix
+Elastic-2.0
+Entessa
+EPICS
+EPL-1.0
+EPL-2.0
+ErlPL-1.1
+etalab-2.0
+EUDatagrid
+EUPL-1.0
+EUPL-1.1
+EUPL-1.2
+Eurosym
+Fair
+FDK-AAC
+Frameworx-1.0
+FreeBSD-DOC
+FreeImage
+FSFAP
+FSFUL
+FSFULLR
+FSFULLRWD
+FTL
+GD
+GFDL-1.1
+GFDL-1.1-invariants-only
+GFDL-1.1-invariants-or-later
+GFDL-1.1-no-invariants-only
+GFDL-1.1-no-invariants-or-later
+GFDL-1.1-only
+GFDL-1.1-or-later
+GFDL-1.2
+GFDL-1.2-invariants-only
+GFDL-1.2-invariants-or-later
+GFDL-1.2-no-invariants-only
+GFDL-1.2-no-invariants-or-later
+GFDL-1.2-only
+GFDL-1.2-or-later
+GFDL-1.3
+GFDL-1.3-invariants-only
+GFDL-1.3-invariants-or-later
+GFDL-1.3-no-invariants-only
+GFDL-1.3-no-invariants-or-later
+GFDL-1.3-only
+GFDL-1.3-or-later
+Giftware
+GL2PS
+Glide
+Glulxe
+GLWTPL
+gnuplot
+GPL-1.0
+GPL-1.0+
+GPL-1.0-only
+GPL-1.0-or-later
+GPL-2.0
+GPL-2.0+
+GPL-2.0-only
+GPL-2.0-or-later
+GPL-2.0-with-autoconf-exception
+GPL-2.0-with-bison-exception
+GPL-2.0-with-classpath-exception
+GPL-2.0-with-font-exception
+GPL-2.0-with-GCC-exception
+GPL-3.0
+GPL-3.0+
+GPL-3.0-only
+GPL-3.0-or-later
+GPL-3.0-with-autoconf-exception
+GPL-3.0-with-GCC-exception
+Graphics-Gems
+gSOAP-1.3b
+HaskellReport
+Hippocratic-2.1
+HP-1986
+HPND
+HPND-export-US
+HPND-Markus-Kuhn
+HPND-sell-variant
+HPND-sell-variant-MIT-disclaimer
+HTMLTIDY
+IBM-pibs
+ICU
+IEC-Code-Components-EULA
+IJG
+IJG-short
+ImageMagick
+iMatix
+Imlib2
+Info-ZIP
+Inner-Net-2.0
+Intel
+Intel-ACPI
+Interbase-1.0
+IPA
+IPL-1.0
+ISC
+Jam
+JasPer-2.0
+JPL-image
+JPNIC
+JSON
+Kazlib
+Knuth-CTAN
+LAL-1.2
+LAL-1.3
+Latex2e
+Latex2e-translated-notice
+Leptonica
+LGPL-2.0
+LGPL-2.0+
+LGPL-2.0-only
+LGPL-2.0-or-later
+LGPL-2.1
+LGPL-2.1+
+LGPL-2.1-only
+LGPL-2.1-or-later
+LGPL-3.0
+LGPL-3.0+
+LGPL-3.0-only
+LGPL-3.0-or-later
+LGPLLR
+Libpng
+libpng-2.0
+libselinux-1.0
+libtiff
+libutil-David-Nugent
+LiLiQ-P-1.1
+LiLiQ-R-1.1
+LiLiQ-Rplus-1.1
+Linux-man-pages-1-para
+Linux-man-pages-copyleft
+Linux-man-pages-copyleft-2-para
+Linux-man-pages-copyleft-var
+Linux-OpenIB
+LOOP
+LPL-1.0
+LPL-1.02
+LPPL-1.0
+LPPL-1.1
+LPPL-1.2
+LPPL-1.3a
+LPPL-1.3c
+LZMA-SDK-9.11-to-9.20
+LZMA-SDK-9.22
+MakeIndex
+Martin-Birgmeier
+metamail
+Minpack
+MirOS
+MIT
+MIT-0
+MIT-advertising
+MIT-CMU
+MIT-enna
+MIT-feh
+MIT-Festival
+MIT-Modern-Variant
+MIT-open-group
+MIT-Wu
+MITNFA
+Motosoto
+mpi-permissive
+mpich2
+MPL-1.0
+MPL-1.1
+MPL-2.0
+MPL-2.0-no-copyleft-exception
+mplus
+MS-LPL
+MS-PL
+MS-RL
+MTLL
+MulanPSL-1.0
+MulanPSL-2.0
+Multics
+Mup
+NAIST-2003
+NASA-1.3
+Naumen
+NBPL-1.0
+NCGL-UK-2.0
+NCSA
+Net-SNMP
+NetCDF
+Newsletr
+NGPL
+NICTA-1.0
+NIST-PD
+NIST-PD-fallback
+NIST-Software
+NLOD-1.0
+NLOD-2.0
+NLPL
+Nokia
+NOSL
+Noweb
+NPL-1.0
+NPL-1.1
+NPOSL-3.0
+NRL
+NTP
+NTP-0
+Nunit
+O-UDA-1.0
+OCCT-PL
+OCLC-2.0
+ODbL-1.0
+ODC-By-1.0
+OFFIS
+OFL-1.0
+OFL-1.0-no-RFN
+OFL-1.0-RFN
+OFL-1.1
+OFL-1.1-no-RFN
+OFL-1.1-RFN
+OGC-1.0
+OGDL-Taiwan-1.0
+OGL-Canada-2.0
+OGL-UK-1.0
+OGL-UK-2.0
+OGL-UK-3.0
+OGTSL
+OLDAP-1.1
+OLDAP-1.2
+OLDAP-1.3
+OLDAP-1.4
+OLDAP-2.0
+OLDAP-2.0.1
+OLDAP-2.1
+OLDAP-2.2
+OLDAP-2.2.1
+OLDAP-2.2.2
+OLDAP-2.3
+OLDAP-2.4
+OLDAP-2.5
+OLDAP-2.6
+OLDAP-2.7
+OLDAP-2.8
+OLFL-1.3
+OML
+OpenPBS-2.3
+OpenSSL
+OPL-1.0
+OPL-UK-3.0
+OPUBL-1.0
+OSET-PL-2.1
+OSL-1.0
+OSL-1.1
+OSL-2.0
+OSL-2.1
+OSL-3.0
+Parity-6.0.0
+Parity-7.0.0
+PDDL-1.0
+PHP-3.0
+PHP-3.01
+Plexus
+PolyForm-Noncommercial-1.0.0
+PolyForm-Small-Business-1.0.0
+PostgreSQL
+PSF-2.0
+psfrag
+psutils
+Python-2.0
+Python-2.0.1
+Qhull
+QPL-1.0
+QPL-1.0-INRIA-2004
+Rdisc
+RHeCos-1.1
+RPL-1.1
+RPL-1.5
+RPSL-1.0
+RSA-MD
+RSCPL
+Ruby
+SAX-PD
+Saxpath
+SCEA
+SchemeReport
+Sendmail
+Sendmail-8.23
+SGI-B-1.0
+SGI-B-1.1
+SGI-B-2.0
+SGP4
+SHL-0.5
+SHL-0.51
+SimPL-2.0
+SISSL
+SISSL-1.2
+Sleepycat
+SMLNJ
+SMPPL
+SNIA
+snprintf
+Spencer-86
+Spencer-94
+Spencer-99
+SPL-1.0
+SSH-OpenSSH
+SSH-short
+SSPL-1.0
+StandardML-NJ
+SugarCRM-1.1.3
+SunPro
+SWL
+Symlinks
+TAPR-OHL-1.0
+TCL
+TCP-wrappers
+TermReadKey
+TMate
+TORQUE-1.1
+TOSL
+TPDL
+TPL-1.0
+TTWL
+TU-Berlin-1.0
+TU-Berlin-2.0
+UCAR
+UCL-1.0
+Unicode-DFS-2015
+Unicode-DFS-2016
+Unicode-TOU
+UnixCrypt
+Unlicense
+UPL-1.0
+Vim
+VOSTROM
+VSL-1.0
+W3C
+W3C-19980720
+W3C-20150513
+w3m
+Watcom-1.0
+Widget-Workshop
+Wsuipa
+WTFPL
+wxWindows
+X11
+X11-distribute-modifications-variant
+Xdebug-1.03
+Xerox
+Xfig
+XFree86-1.1
+xinetd
+xlock
+Xnet
+xpp
+XSkat
+YPL-1.0
+YPL-1.1
+Zed
+Zend-2.0
+Zimbra-1.3
+Zimbra-1.4
+Zlib
+zlib-acknowledgement
+ZPL-1.1
+ZPL-2.0
+ZPL-2.1
+389-exception
+Asterisk-exception
+Autoconf-exception-2.0
+Autoconf-exception-3.0
+Autoconf-exception-generic
+Autoconf-exception-macro
+Bison-exception-2.2
+Bootloader-exception
+Classpath-exception-2.0
+CLISP-exception-2.0
+cryptsetup-OpenSSL-exception
+DigiRule-FOSS-exception
+eCos-exception-2.0
+Fawkes-Runtime-exception
+FLTK-exception
+Font-exception-2.0
+freertos-exception-2.0
+GCC-exception-2.0
+GCC-exception-3.1
+GNAT-exception
+gnu-javamail-exception
+GPL-3.0-interface-exception
+GPL-3.0-linking-exception
+GPL-3.0-linking-source-exception
+GPL-CC-1.0
+GStreamer-exception-2005
+GStreamer-exception-2008
+i2p-gpl-java-exception
+KiCad-libraries-exception
+LGPL-3.0-linking-exception
+libpri-OpenH323-exception
+Libtool-exception
+Linux-syscall-note
+LLGPL
+LLVM-exception
+LZMA-exception
+mif-exception
+Nokia-Qt-exception-1.1
+OCaml-LGPL-linking-exception
+OCCT-exception-1.0
+OpenJDK-assembly-exception-1.0
+openvpn-openssl-exception
+PS-or-PDF-font-exception-20170817
+QPL-1.0-INRIA-2004-exception
+Qt-GPL-exception-1.0
+Qt-LGPL-exception-1.1
+Qwt-exception-1.0
+SHL-2.0
+SHL-2.1
+SWI-exception
+Swift-exception
+u-boot-exception-2.0
+Universal-FOSS-exception-1.0
+vsftpd-openssl-exception
+WxWindows-exception-3.1
+x11vnc-openssl-exception
+endef
support/misc/cyclonedx.mk +34 -1
index 3906a3b60a..1d7199c92c 100644
@@ -7,6 +7,8 @@
 #
 ################################################################################
 
+include support/misc/cyclonedx-spdx.mk
+
 # Note: to avoid conflict with <pkg>_VERSION `_SPEC` is added
 CYCLONEDX_VERSION_SPEC = 1.5
 
@@ -24,6 +26,22 @@ CYCLONEDX_VERSION_SPEC = 1.5
 # Turns "Public%20Domain,%20GPL-2.0" into "Public%20Domain GPL-2.0"
 _cyclonedx-licenses-as-list = $(subst $(comma)%20,$(space),$(1))
 
+# _cyclonedx-license-attribute -- according to CycloneDX spec correct SPDX
+#                                 licenses must use the 'id' key while
+#                                 other use the 'name' key.
+#                                 If a SPDX license is followed by parenthesis
+#                                 to describe its scope it will be threated as
+#                                 a non SPDX license.
+#
+# $(1): a license name with space encoded. Since all official SPDX license names
+#       are a single word (no spaces), it's not an issue to keep them url-encoded.
+define _cyclonedx-license-attribute
+	$(if $(filter $(spdx),$(1)), \
+		"id", \
+		"name" \
+	)
+endef
+
 # _cyclonedx-license -- create an entry of a cyclonedx component license list
 #
 # For more information on license object see
@@ -33,7 +51,8 @@ _cyclonedx-licenses-as-list = $(subst $(comma)%20,$(space),$(1))
 define _cyclonedx-license
 	{
 		"license": {
-			"name": $(call mk-json-str,$(1))
+			$(call _cyclonedx-license-attribute,$(1)):
+				$(call mk-json-str,$(1))
 		}
 	},
 endef
@@ -195,3 +214,17 @@ define cyclonedx-json
 		}
 	})
 endef
+
+# Use this rule to update the `cyclonedx-spdx.mk` file. The rule will
+# override the cyclonedx-spdx.mk file with a variable called 'spdx' that
+# contains the list of the SPDX license supported by CycloneDX spec.
+.PHONY: support/misc/cyclonedx-spdx.mk
+support/misc/cyclonedx-spdx.mk:
+	$(WGET) -O - https://raw.githubusercontent.com/CycloneDX/specification/$(CYCLONEDX_VERSION_SPEC)/schema/spdx.schema.json | \
+		$(JQ) jq -r '.enum[]' | { \
+			echo '# List of approved SPDX license'; \
+			echo '# See https://raw.githubusercontent.com/CycloneDX/specification/1.5/schema/spdx.schema.json'; \
+			echo 'define spdx'; \
+			cat; \
+			echo 'endef'; \
+		} > $@

--
2.44.0

_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot

#7 lore raw

RFC PATCH 5/5 Makefile: add command to generate SBOM in CycloneDX format

In reply to Thomas Perale via buildroot

This patch adds a new command to generate a JSON SBOM in the CycloneDX format based on the packages selected in the dot config.

Usage:

make cyclonedx

---
 Makefile | 13 +++++++++++++
 1 file changed, 13 insertions(+)
Makefile +13 -0
index 91973cca60..43344673c0 100644
@@ -155,6 +155,8 @@ MAKEOVERRIDES :=
 
 # Include some helper macros and variables
 include support/misc/utils.mk
+# Include CycloneDX SBOM support
+include support/misc/cyclonedx.mk
 
 # Set variables related to in-tree or out-of-tree build.
 # Here, both $(O) and $(CURDIR) are absolute canonical paths.
@@ -924,6 +926,16 @@ show-info:
 		) \
 	)
 
+.PHONY: cyclonedx
+cyclonedx:
+	@:
+	$(info $(call cyclonedx-json, \
+			$(sort $(foreach p,$(PACKAGES) $(TARGETS_ROOTFS), \
+				$(p) $($(call UPPERCASE,$(p))_FINAL_RECURSIVE_DEPENDENCIES)) \
+			) \
+		) \
+	)
+
 .PHONY: pkg-stats
 pkg-stats:
 	@cd "$(CONFIG_DIR)" ; \
@@ -1185,6 +1197,7 @@ help:
 	@echo '  source                 - download all sources needed for offline-build'
 	@echo '  external-deps          - list external packages used'
 	@echo '  legal-info             - generate info about license compliance'
+	@echo '  cyclonedx              - generate info about packages, as a CycloneDX formatted json blurb'
 	@echo '  show-info              - generate info about packages, as a JSON blurb'
 	@echo '  pkg-stats              - generate info about packages as JSON and HTML'
 	@echo '  printvars              - dump internal variables selected with VARS=...'

--
2.44.0

_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot

#8 lore raw
In reply to Thomas Perale via buildroot

Hi Thomas,

Great that you're taking up this topic. I'm highly interested because I'm currently generating my sbom "by hand" using a small python script which converts the output of show-info. But yours is already better as it adds the dependency info and also fills the purl field (to the best of its knowledge).

Did you see the work of Robert (i added him in CC)? He submitted a script to the list some time ago which can run on the manifest.csv [0]. Maybe there are some synergies.

First feedback:
I created my SBOM using your patchset and put it into our dependency track instance. Looks promising.

One thing that I noticed:
when running make cyclonedx I first get the output of wget which retrieves the cyclonedx schema. So I have to cut it from the output to have proper json. Maybe you want to silence this in some way.

I'll keep playing around and will give you further feedback if I'm noticing things.

Regards,
Michael

[0] https://github.com/CycloneDX/cyclonedx-buildroot

On Thursday, April 04, 2024 14:43 CEST, Thomas Perale via buildroot <[email protected]> wrote:

_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot

#9 lore raw

Hi Michael,

Thank you for taking the time checking my changes.

Quoted 2 lines

Did you see the work of Robert (i added him in CC)? He submitted a script to the list some time ago which can run on the manifest.csv [0]. Maybe there are some synergies.

I saw the work of Robert and used it as a reference to write the first iterations of this patch series but started working on a change for different reasons:

- Provide a solution that doesn't rely on python and external library to generate the SBOM. And a Makefile implementation just make sense to access all the package variable.
- Generating something based on show-info and legal-info just add more commands to run.

I would highly appreciate any contribution/reviews from Robert.

Quoted 3 lines

One thing that I noticed:
when running make cyclonedx I first get the output of wget which retrieves the cyclonedx schema. So I have to cut it from the output to have proper json. Maybe you want to silence this in some way.

Did you mean the cyclondx-spdx file from the support/misc/cyclonedx-spdx.mk rule ? It should not be ran by default and don't see why it's run ..

Regards,
PERALE Thomas

Quoted 15 lines · Michael Nosthoff

Hi Thomas,

Great that you're taking up this topic. I'm highly interested because I'm currently generating my sbom "by hand" using a small python script which converts the output of show-info. But yours is already better as it adds the dependency info and also fills the purl field (to the best of its knowledge).

Did you see the work of Robert (i added him in CC)? He submitted a script to the list some time ago which can run on the manifest.csv [0]. Maybe there are some synergies.

First feedback:
I created my SBOM using your patchset and put it into our dependency track instance. Looks promising.

One thing that I noticed:
when running make cyclonedx I first get the output of wget which retrieves the cyclonedx schema. So I have to cut it from the output to have proper json. Maybe you want to silence this in some way.

Quoted 9 lines

I'll keep playing around and will give you further feedback if I'm noticing things.

Regards,
Michael

[0] https://github.com/CycloneDX/cyclonedx-buildroot

On Thursday, April 04, 2024 14:43 CEST, Thomas Perale via buildroot <[email protected]> wrote:

_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot

#10 lore raw
In reply to Thomas Perale via buildroot

Hello Thomas,

Quoted 8 lines · On Thu, 4 Apr 2024 14:43:24 +0200 Thomas Perale <[email protected]>…

This RFC patch series propose to add support for the CycloneDX SBOM format.

There is a growing need to generate SBOM from buildroot configurations. Right now, there are different solutions available for buildroot users show-info, legal-info and pkg-stats. They all generate similar information (show-info showing more) but in a format that is specific to buildroot.

Thanks a lot for your work on this, this is really useful.

My initial reaction is whether it is really the right solution to have "make" spit out the CycloneDX format, or whether we should use "make show-info", and then process the JSON output using some Python script to generate the CycloneDX SBoM. Some of the mangling needed to generate the CycloneDX stuff is a bit tricky to write in make, and having a utility Python script doing that work based on a JSON input would I believe be simpler and easier to extend. Have you explored this idea?

Of course, if "make show-info" doesn't provide enough information in the generated JSON blurb, we can always extend that with more information.

Quoted 6 lines

This is a first sketch and I hope to gather comments on functionality the community want me to include. I already have a todo list of feature I plan to work on:

- [ ] Find a solution to handle versioning. The "version" property should be incremented every SBOM generation.

Perhaps easier to handle this with the Python script approach proposed above.

Best regards,

Thomas

Signature · 7 lines
Thomas Petazzoni, co-owner and CEO, Bootlin
Embedded Linux and Kernel engineering and training
https://bootlin.com
_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot
#11 lore raw

Thomas², All,

On 2024-04-07 23:15 +0200, Thomas Petazzoni via buildroot spake thusly:

Quoted 22 lines
Quoted 8 lines · On Thu, 4 Apr 2024 14:43:24 +0200 Thomas Perale <[email protected]>…

This RFC patch series propose to add support for the CycloneDX SBOM format.

There is a growing need to generate SBOM from buildroot configurations. Right now, there are different solutions available for buildroot users show-info, legal-info and pkg-stats. They all generate similar information (show-info showing more) but in a format that is specific to buildroot.

Thanks a lot for your work on this, this is really useful.

My initial reaction is whether it is really the right solution to have "make" spit out the CycloneDX format, or whether we should use "make show-info", and then process the JSON output using some Python script to generate the CycloneDX SBoM. Some of the mangling needed to generate the CycloneDX stuff is a bit tricky to write in make, and having a utility Python script doing that work based on a JSON input would I believe be simpler and easier to extend. Have you explored this idea?

Quoted 3 lines

Of course, if "make show-info" doesn't provide enough information in the generated JSON blurb, we can always extend that with more information.

This was also my reaction: when I introduced show-info, the goal was to expose details about the current configuration, so that it could be digested by various scripts to generate the required SBOM formats.

I would prefer that we do get a support/script/cyclonedx script (python or whatever) that gets show-info output and generates CycloneDX SBOM.

Quoted 9 lines
Quoted 6 lines

This is a first sketch and I hope to gather comments on functionality the community want me to include. I already have a todo list of feature I plan to work on:

- [ ] Find a solution to handle versioning. The "version" property should be incremented every SBOM generation.

Perhaps easier to handle this with the Python script approach proposed above.

Also I don't see how that versioning works.

If it gets incremented everytime the SBOM is generated, then (with the current state of this series):

git clone blablabla/buildroot
make -C buildroot foo_defconfig
make -C buildroot cyclonedx
make -C buildroot cyclonedx

would yield two different versions, although nothing changed in-between.

Also:

git clone blablabla/buildroot
make -C buildroot foo_defconfig
make -C buildroot
make -C buildroot cyclonedx
rm -rf buildroot
git clone blablabla/buildroot
make -C buildroot foo_defconfig
make -C buildroot
make -C buildroot cyclonedx

would have to yield two different versions, but where would the "latest" version be stored? Certainly not in the user's home directory.

And arguably, if two different people build the cyclonedx SBOM for the same project, on two different machines (with an otherwise identical environment), at different time, then how do they know the version used by the other? This also applies to "two CI pipelines" instead of "two users".

Regards,
Yann E. MORIN.

Signature · 10 lines
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot
#12 lore raw
Quoted 29 lines · Yann E. MORIN

Thomas², All,

On 2024-04-07 23:15 +0200, Thomas Petazzoni via buildroot spake thusly:

Quoted 22 lines
Quoted 8 lines · On Thu, 4 Apr 2024 14:43:24 +0200 Thomas Perale <[email protected]>…

This RFC patch series propose to add support for the CycloneDX SBOM format.

There is a growing need to generate SBOM from buildroot configurations. Right now, there are different solutions available for buildroot users show-info, legal-info and pkg-stats. They all generate similar information (show-info showing more) but in a format that is specific to buildroot.

Thanks a lot for your work on this, this is really useful.

My initial reaction is whether it is really the right solution to have "make" spit out the CycloneDX format, or whether we should use "make show-info", and then process the JSON output using some Python script to generate the CycloneDX SBoM. Some of the mangling needed to generate the CycloneDX stuff is a bit tricky to write in make, and having a utility Python script doing that work based on a JSON input would I believe be simpler and easier to extend. Have you explored this idea?

Quoted 3 lines

Of course, if "make show-info" doesn't provide enough information in the generated JSON blurb, we can always extend that with more information.

Quoted 6 lines

This was also my reaction: when I introduced show-info, the goal was to expose details about the current configuration, so that it could be digested by various scripts to generate the required SBOM formats.

I would prefer that we do get a support/script/cyclonedx script (python or whatever) that gets show-info output and generates CycloneDX SBOM.

This was (of course) on my instigation.

My premise is that we should (in the longer term) _always_ generate the
CycloneDX SBoM as part of the world target, so without needing a separate "make cyclonedx" step. I think we need to kind of force people to generate SBoMs, so there is no excuse not to have it.

This has a number of implications: it has to be quite lightweight (because it
is called for every build), so that basically limits it so make and shell (I didn't like the idea of "always build host-python"). It should also be possible to generate it without internet connection on condition that a 'make source' was called before (that's why the license list in support/misc/cyclonedx-spdx.mk is stored in the git repo and not downloaded while building). On the other hand, the cyclonedx generation can depend on the presence of downloads and build artifacts, if that is needed.

On that background, with the addition of cyclonedx, we now have 5 things that
do somewhat similar things: cyclonedx, legal-info, show-info, pkg-stats, and printvars. So a second goal is to try to unify all that in a single mechanism. And because cyclonedx is a standard, it makes sense to me to try to use that as the primary mechanism, and make the others derivations from it.

- For show-info: as long as the same information is available in the cyclonedx file, we can simply use the cyclonedx file and deprecate show-info (possibly with a conversion script, which is allowed to be written in python, but might even be possible to write with jq, to convert to the legacy format).
- For legal-info, it's quite obvious that cyclonedx should basically replace that - though legal-info currently still does a number of additional steps (e.g. collect the actual source) that cyclonedx doesn't cover. I also think the manifest.csv file makes a lot of sense. I'm not sure how to solve that, but I can imagine that there could still be an explicit legal-info target that consumes the cyclonedx file and does some additional steps.
- For pkg-stats, it turns out that DependencyTrack already covers a large number of features that we have in there (CVEs, obviously, but also upstream version). Some very buildroot-specific things are not in there (e.g. the package's infra) but it could be possible to add those to DependencyTrack.
- For printvars I have no solution, I guess it will have to stay.

So, why is it not a Python script based on show-info? Because I want the
CycloneDX file to be _always_ generated, which means no Python. And processing JSON in shell is horror, then it's better to just generate it twice.

Quoted 11 lines
Quoted 9 lines
Quoted 6 lines

This is a first sketch and I hope to gather comments on functionality the community want me to include. I already have a todo list of feature I plan to work on:

- [ ] Find a solution to handle versioning. The "version" property should be incremented every SBOM generation.

Perhaps easier to handle this with the Python script approach proposed above.

Also I don't see how that versioning works.

In my understanding, the version should be updated every time you edit a
CycloneDX file. So we always generate version 1. It is updated by tools that _process_ CycloneDX, e.g. to annotate with additional information.

Quoted 9 lines

If it gets incremented everytime the SBOM is generated, then (with the current state of this series):

git clone blablabla/buildroot
make -C buildroot foo_defconfig
make -C buildroot cyclonedx
make -C buildroot cyclonedx

would yield two different versions, although nothing changed in-between.

So this would generate the same version 1 both times.

Quoted 12 lines

Also:

git clone blablabla/buildroot
make -C buildroot foo_defconfig
make -C buildroot
make -C buildroot cyclonedx
rm -rf buildroot
git clone blablabla/buildroot
make -C buildroot foo_defconfig
make -C buildroot
make -C buildroot cyclonedx

Quoted 2 lines

would have to yield two different versions, but where would the "latest" version be stored? Certainly not in the user's home directory.

IMHO the cyclonedx file should be bit-for-bit reproducible, so this must
_definitely_ generate the same version.

Regards,
Arnout

Quoted 9 lines

And arguably, if two different people build the cyclonedx SBOM for the same project, on two different machines (with an otherwise identical environment), at different time, then how do they know the version used by the other? This also applies to "two CI pipelines" instead of "two users".

Regards,
Yann E. MORIN.

_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot

#13 lore raw

Arnout, All,

On 2024-04-09 14:17 +0200, Arnout Vandecappelle spake thusly:

Quoted 6 lines · Yann E. MORIN
Quoted 5 lines

On 2024-04-07 23:15 +0200, Thomas Petazzoni via buildroot spake thusly:

Quoted 4 lines
Quoted 2 lines · On Thu, 4 Apr 2024 14:43:24 +0200 Thomas Perale <[email protected]>…

This RFC patch series propose to add support for the CycloneDX SBOM format.

[--SNIP--]

Quoted 5 lines
Quoted 5 lines

On 2024-04-07 23:15 +0200, Thomas Petazzoni via buildroot spake thusly:

Quoted 4 lines

My initial reaction is whether it is really the right solution to have "make" spit out the CycloneDX format, or whether we should use "make show-info", and then process the JSON output using some Python script to generate the CycloneDX SBoM.

[--SNIP--]

Quoted 2 lines · Yann E. MORIN
Quoted 1 line

This was also my reaction: when I introduced show-info,

[--SNIP--]

On 2024-04-09 14:17 +0200, Arnout Vandecappelle spake thusly:

Quoted 4 lines

My premise is that we should (in the longer term) _always_ generate the
CycloneDX SBoM as part of the world target, so without needing a separate "make cyclonedx" step. I think we need to kind of force people to generate SBoMs, so there is no excuse not to have it.

An SBOM is totally worthless when generated on the developers machines, as the build environment there is uncontrollable; it is useless during development as well [0]. An SBOM only ever makes sense when generated in a delivery pipeline, like in a CI/CD pipeline.

[0] except in the rare cases of those developpers explicitly looking at or working on the SBOM: evaluating the SBOM, fixing bugs in the SBOM generation tooling, adding SBOM-related metadata...

Quoted 9 lines

This has a number of implications: it has to be quite lightweight (because
it is called for every build), so that basically limits it so make and shell (I didn't like the idea of "always build host-python"). It should also be possible to generate it without internet connection on condition that a 'make source' was called before (that's why the license list in support/misc/cyclonedx-spdx.mk is stored in the git repo and not downloaded while building). On the other hand, the cyclonedx generation can depend on the presence of downloads and build artifacts, if that is needed.

Quoted 3 lines

On that background, with the addition of cyclonedx, we now have 5 things
that do somewhat similar things: cyclonedx, legal-info, show-info, pkg-stats, and printvars.

Forget about printvars: it is unusable and legacy. We have show-vars instead, which does provide a reliable extract of the internal variables (except for those that need a $(@) or $(@D) or such, because there is no actual target in that case, and thus it is not possible to properly extract the internal state of variables).

We should now drop printvars.

Quoted 4 lines

So a second goal is to try to unify all that in a
single mechanism. And because cyclonedx is a standard, it makes sense to me to try to use that as the primary mechanism, and make the others derivations from it.

The nice things about standards, is that there are so many to pick from. ;-)

SPDX is also an SBOM standard, which predates cyclonedx by 6 years. It is more mature (older!), so why don't we settle on that instead?

Quoted 5 lines

- For show-info: as long as the same information is available in the cyclonedx file, we can simply use the cyclonedx file and deprecate show-info (possibly with a conversion script, which is allowed to be written in python, but might even be possible to write with jq, to convert to the legacy format).

I strongly disagree. show-info has other usages than generating an SBOM, and contains a lot of info that does not belong in a SBOM. For example, as you noticed, the package infra is mising in cyclonedx, and there is no reason to add it there, while it has its place in show-info.

Another example, is that (AFAICS) there is no dedicated way to describe reverse depdndencies in cylonedx, except by injected an arbitrary property, which means that it does not really fit... show-info provides that.

More generally: there is information that belongs to show-info, as it is usefull for generating various reports, that have absolutely no meaning in an SBOM, and so there is no reason to inject those in cyclonedx just because something else may need it. That's what show-info is for.

And as Thomas said, if show-info is missing data, we can easily add it.

Quoted 6 lines

- For legal-info, it's quite obvious that cyclonedx should basically replace that - though legal-info currently still does a number of additional steps (e.g. collect the actual source) that cyclonedx doesn't cover. I also think the manifest.csv file makes a lot of sense. I'm not sure how to solve that, but I can imagine that there could still be an explicit legal-info target that consumes the cyclonedx file and does some additional steps.

Or the other way around: cyclonedx consumes the output of legal-info.

Quoted 5 lines

- For pkg-stats, it turns out that DependencyTrack already covers a large number of features that we have in there (CVEs, obviously, but also upstream version). Some very buildroot-specific things are not in there (e.g. the package's infra) but it could be possible to add those to DependencyTrack.
- For printvars I have no solution, I guess it will have to stay.

See above: we should drop it in favour of show-vars. But yes, this wil have to stay, indeed.

Quoted 4 lines

So, why is it not a Python script based on show-info? Because I want the
CycloneDX file to be _always_ generated, which means no Python. And processing JSON in shell is horror, then it's better to just generate it twice.

As I said above: it does not make sense to always generate it, because it is only relevant when it is generated in a controlled environment, like a delivery pipeline for a delivery tag (or such), and not during development on developers machines.

[--SNIP--]

Quoted 6 lines
Quoted 3 lines
Quoted 2 lines
Quoted 2 lines

- [ ] Find a solution to handle versioning. The "version" property should be incremented every SBOM generation.

Also I don't see how that versioning works.

In my understanding, the version should be updated every time you edit a
CycloneDX file. So we always generate version 1. It is updated by tools that _process_ CycloneDX, e.g. to annotate with additional information.

Ah, so there are tools that have to run on the cyclonedx data after it has been generated, because they need to inject data we have no way to know in Buildroot?

This means that those tools can very well consume show-info to generate the final cyclonedx, to begin with...

Quoted 2 lines

IMHO the cyclonedx file should be bit-for-bit reproducible, so this must
_definitely_ generate the same version.

I do agree that it should be reproducible, but then I don't see how this articulates around the "version property should be incremented every SBOM generation."

Note that, if the version really has to be incremented every time the cyclonedx data is generated, we could just use the UNIX timestamp, with milli- or microsecond precision, which is in practice going to be monotonically incrementing. It's not going to be a sequential +1 every time, though...

Regards,
Yann E. MORIN.

Signature · 10 lines
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot
#14 lore raw
Quoted 28 lines · Yann E. MORIN

Arnout, All,

On 2024-04-09 14:17 +0200, Arnout Vandecappelle spake thusly:

Quoted 6 lines · Yann E. MORIN
Quoted 5 lines

On 2024-04-07 23:15 +0200, Thomas Petazzoni via buildroot spake thusly:

Quoted 4 lines
Quoted 2 lines · On Thu, 4 Apr 2024 14:43:24 +0200 Thomas Perale <[email protected]>…

This RFC patch series propose to add support for the CycloneDX SBOM format.

[--SNIP--]

Quoted 5 lines
Quoted 5 lines

On 2024-04-07 23:15 +0200, Thomas Petazzoni via buildroot spake thusly:

Quoted 4 lines

My initial reaction is whether it is really the right solution to have "make" spit out the CycloneDX format, or whether we should use "make show-info", and then process the JSON output using some Python script to generate the CycloneDX SBoM.

[--SNIP--]

Quoted 2 lines · Yann E. MORIN
Quoted 1 line

This was also my reaction: when I introduced show-info,

[--SNIP--]

On 2024-04-09 14:17 +0200, Arnout Vandecappelle spake thusly:

Quoted 4 lines

My premise is that we should (in the longer term) _always_ generate the
CycloneDX SBoM as part of the world target, so without needing a separate "make cyclonedx" step. I think we need to kind of force people to generate SBoMs, so there is no excuse not to have it.

Quoted 4 lines

An SBOM is totally worthless when generated on the developers machines, as the build environment there is uncontrollable; it is useless during development as well [0]. An SBOM only ever makes sense when generated in a delivery pipeline, like in a CI/CD pipeline.

Huh? It's equally worthless as the images created by Buildroot then. Those are
also uncontrollable. Of course, you have no guarantee that the SBoM corresponds to the image that gets shipped to customers, but that's not really relevant. You can still use the SBoM to see which packages get installed, which patches are applied, which CVEs apply, which licenses are used, etc.

And anyway, I don't think it's very relevant if this SBoM generation happens
on a developer machine or in CI. In either case, I don't think we want to rely on a host-installed Python version.

Quoted 3 lines

[0] except in the rare cases of those developpers explicitly looking at or working on the SBOM: evaluating the SBOM, fixing bugs in the SBOM generation tooling, adding SBOM-related metadata...

So you're saying that for developers who do not use the SBoM, it's not
relevant to generate them locally. That's true of course. Not very relevant IMHO though.

Quoted 13 lines
Quoted 9 lines

This has a number of implications: it has to be quite lightweight (because
it is called for every build), so that basically limits it so make and shell (I didn't like the idea of "always build host-python"). It should also be possible to generate it without internet connection on condition that a 'make source' was called before (that's why the license list in support/misc/cyclonedx-spdx.mk is stored in the git repo and not downloaded while building). On the other hand, the cyclonedx generation can depend on the presence of downloads and build artifacts, if that is needed.

Quoted 3 lines

On that background, with the addition of cyclonedx, we now have 5 things
that do somewhat similar things: cyclonedx, legal-info, show-info, pkg-stats, and printvars.

Quoted 7 lines

Forget about printvars: it is unusable and legacy. We have show-vars instead, which does provide a reliable extract of the internal variables (except for those that need a $(@) or $(@D) or such, because there is no actual target in that case, and thus it is not possible to properly extract the internal state of variables).

We should now drop printvars.

OMG, I completely forgot about show-vars! But indeed, we still need to keep
show-vars. And we should drop printvars.

Quoted 10 lines
Quoted 4 lines

So a second goal is to try to unify all that in a
single mechanism. And because cyclonedx is a standard, it makes sense to me to try to use that as the primary mechanism, and make the others derivations from it.

The nice things about standards, is that there are so many to pick from. ;-)

SPDX is also an SBOM standard, which predates cyclonedx by 6 years. It is more mature (older!), so why don't we settle on that instead?

Because SPDX is horribly messy and complicated.

Quoted 10 lines
Quoted 5 lines

- For show-info: as long as the same information is available in the cyclonedx file, we can simply use the cyclonedx file and deprecate show-info (possibly with a conversion script, which is allowed to be written in python, but might even be possible to write with jq, to convert to the legacy format).

I strongly disagree. show-info has other usages than generating an SBOM, and contains a lot of info that does not belong in a SBOM. For example, as you noticed, the package infra is mising in cyclonedx, and there is no reason to add it there, while it has its place in show-info.

Excellent example :-). The SBoM JSON (or show-info BTW) is not really usable
for a human as is. So you'll want a tool to visualize it. You can make your own tool (pkg-stats), or you can use something that exist already (DependencyTrack). In the latter case, you want to make sure that that tool displays all relevant information - so the SBoM needs to contain that information. Fortunately, CycloneDX allows to add additional properties. Now we just need to extend DependencyTrack to display that information. Thomas Perale is looking at that.

Quoted 4 lines

Another example, is that (AFAICS) there is no dedicated way to describe reverse depdndencies in cylonedx, except by injected an arbitrary property, which means that it does not really fit... show-info provides that.

True.

Quoted 3 lines

More generally: there is information that belongs to show-info, as it is usefull for generating various reports, that have absolutely no meaning in an SBOM,

What kind of information would that be? show-info and an SBoM are pretty much
the same thing: a list of packages and their properties.

Quoted 2 lines

and so there is no reason to inject those in cyclonedx just because something else may need it. That's what show-info is for

Anyway, for me replacing show-info is not the most important thing, it's more
a nice-to-have. What I don't want is to generate CycloneDX by post-processing the show-info output, because it turns out that that is still fairly complicated and it's just easier to directly generate CycloneDX from make.

Quoted 10 lines

And as Thomas said, if show-info is missing data, we can easily add it.

Quoted 6 lines

- For legal-info, it's quite obvious that cyclonedx should basically replace that - though legal-info currently still does a number of additional steps (e.g. collect the actual source) that cyclonedx doesn't cover. I also think the manifest.csv file makes a lot of sense. I'm not sure how to solve that, but I can imagine that there could still be an explicit legal-info target that consumes the cyclonedx file and does some additional steps.

Or the other way around: cyclonedx consumes the output of legal-info.

Again, it's going to be much easier to generate everything from make rather
than post-processing the output of legal-info and show-info.

Quoted 22 lines
Quoted 5 lines

- For pkg-stats, it turns out that DependencyTrack already covers a large number of features that we have in there (CVEs, obviously, but also upstream version). Some very buildroot-specific things are not in there (e.g. the package's infra) but it could be possible to add those to DependencyTrack.
- For printvars I have no solution, I guess it will have to stay.

See above: we should drop it in favour of show-vars. But yes, this wil have to stay, indeed.

Quoted 4 lines

So, why is it not a Python script based on show-info? Because I want the
CycloneDX file to be _always_ generated, which means no Python. And processing JSON in shell is horror, then it's better to just generate it twice.

As I said above: it does not make sense to always generate it, because it is only relevant when it is generated in a controlled environment, like a delivery pipeline for a delivery tag (or such), and not during development on developers machines.

[--SNIP--]

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

- [ ] Find a solution to handle versioning. The "version" property should be incremented every SBOM generation.

Quoted 8 lines
Quoted 4 lines
Quoted 1 line

Also I don't see how that versioning works.

In my understanding, the version should be updated every time you edit a
CycloneDX file. So we always generate version 1. It is updated by tools that _process_ CycloneDX, e.g. to annotate with additional information.

Ah, so there are tools that have to run on the cyclonedx data after it has been generated, because they need to inject data we have no way to know in Buildroot?

I don't know if those tools actually exist. I'm just saying that the version
field is supposed to be updated by such tools.

Quoted 2 lines

This means that those tools can very well consume show-info to generate the final cyclonedx, to begin with...

I am _very_ sure that no such tool exists :-)

Quoted 6 lines
Quoted 2 lines

IMHO the cyclonedx file should be bit-for-bit reproducible, so this must
_definitely_ generate the same version.

I do agree that it should be reproducible, but then I don't see how this articulates around the "version property should be incremented every SBOM generation."

The statement "version property should be incremented every SBOM generation."
from Thomas Perale is simply wrong (at least, AFAIU). It has to be incremented every time the SBoM is updated.

There is also a UUID (serialNumber) field defined [1]. That one is supposed to
be re-generated every time the SBoM is generated (but it can use any kind of UUID generation mechanism, e.g. random or time based). I don't like that very much though, because that makes the SBoM not reproducible.

Regards,
Arnout

[1] https://cyclonedx.org/docs/1.5/json/#serialNumber

Quoted 9 lines

Note that, if the version really has to be incremented every time the cyclonedx data is generated, we could just use the UNIX timestamp, with milli- or microsecond precision, which is in practice going to be monotonically incrementing. It's not going to be a sequential +1 every time, though...

Regards,
Yann E. MORIN.

_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot

#15 lore raw

Hello,

Quoted 10 lines · On Wed, 10 Apr 2024 21:26:09 +0200 Arnout Vandecappelle <[email protected]> wrote:
Quoted 4 lines

An SBOM is totally worthless when generated on the developers machines, as the build environment there is uncontrollable; it is useless during development as well [0]. An SBOM only ever makes sense when generated in a delivery pipeline, like in a CI/CD pipeline.

Huh? It's equally worthless as the images created by Buildroot then. Those are
also uncontrollable. Of course, you have no guarantee that the SBoM corresponds to the image that gets shipped to customers, but that's not really relevant. You can still use the SBoM to see which packages get installed, which patches are applied, which CVEs apply, which licenses are used, etc.

Yes, I agree the argument from Yann here is a bit dubious, but do we care?

Quoted 3 lines

And anyway, I don't think it's very relevant if this SBoM generation happens
on a developer machine or in CI. In either case, I don't think we want to rely on a host-installed Python version.

Why?

Quoted 4 lines
Quoted 1 line

We should now drop printvars.

OMG, I completely forgot about show-vars! But indeed, we still need to keep
show-vars. And we should drop printvars.

We don't care about backward compatibility?

Quoted 6 lines
Quoted 4 lines

Another example, is that (AFAICS) there is no dedicated way to describe reverse depdndencies in cylonedx, except by injected an arbitrary property, which means that it does not really fit... show-info provides that.

True.

So, how do you propose that CycloneDX replaces show-info, if CycloneDX cannot represent all information that is emitted today in show-info?

Quoted 6 lines
Quoted 3 lines

More generally: there is information that belongs to show-info, as it is usefull for generating various reports, that have absolutely no meaning in an SBOM,

What kind of information would that be? show-info and an SBoM are pretty much
the same thing: a list of packages and their properties.

Well, see above?

Quoted 6 lines
Quoted 2 lines

and so there is no reason to inject those in cyclonedx just because something else may need it. That's what show-info is for

Anyway, for me replacing show-info is not the most important thing, it's more
a nice-to-have. What I don't want is to generate CycloneDX by post-processing the show-info output, because it turns out that that is still fairly complicated and it's just easier to directly generate CycloneDX from make.

Why is it "fairly complicated"? It sounds to me that post-processing show-info into a simple and nice Python script to generate CycloneDX is not at all "fairly complicated" but rather "fairly simpler" than doing it in make.

Even from a pure architecture point of view, it makes a lot of sense to have a Buildroot-specific show-info emitted a JSON blurb with as much information as we want (we don't need to comply with any standard), and *then* have post-processing tool consume this Buildroot-specific JSON blurb and turn it into CycloneDX, SPDX, whatever people want, by using the relevant information provided in the Buildroot-specific JSON blurb.

So I'm sorry, but your argumentation is not convincing at all :-/

Quoted 9 lines
Quoted 9 lines

And as Thomas said, if show-info is missing data, we can easily add it.

Quoted 6 lines

- For legal-info, it's quite obvious that cyclonedx should basically replace that - though legal-info currently still does a number of additional steps (e.g. collect the actual source) that cyclonedx doesn't cover. I also think the manifest.csv file makes a lot of sense. I'm not sure how to solve that, but I can imagine that there could still be an explicit legal-info target that consumes the cyclonedx file and does some additional steps.

Quoted 4 lines
Quoted 1 line

Or the other way around: cyclonedx consumes the output of legal-info.

Again, it's going to be much easier to generate everything from make rather
than post-processing the output of legal-info and show-info.

Do you have some real argument to back this affirmation? Python code is typically much more readable than make code, and it's easier to do text manipulation in Python than in make.

Best regards,

Thomas

Signature · 7 lines
Thomas Petazzoni, co-owner and CEO, Bootlin
Embedded Linux and Kernel engineering and training
https://bootlin.com
_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot
#16 lore raw

Thomas, All,

On 2024-04-10 22:10 +0200, Thomas Petazzoni spake thusly:

Quoted 6 lines
Quoted 4 lines · On Wed, 10 Apr 2024 21:26:09 +0200 Arnout Vandecappelle <[email protected]> wrote:
Quoted 4 lines

An SBOM is totally worthless when generated on the developers machines, as the build environment there is uncontrollable; it is useless during development as well [0]. An SBOM only ever makes sense when generated in a delivery pipeline, like in a CI/CD pipeline.

Quoted 7 lines
Quoted 5 lines

Huh? It's equally worthless as the images created by Buildroot then. Those are
also uncontrollable. Of course, you have no guarantee that the SBoM corresponds to the image that gets shipped to customers, but that's not really relevant. You can still use the SBoM to see which packages get installed, which patches are applied, which CVEs apply, which licenses are used, etc.

Yes, I agree the argument from Yann here is a bit dubious, but do we care?

What I meant here, is that the environment on a developper's machine is unknown, so there is no way one can produce an SBOm there, that is meaningful. If the developper has an override-srcdir or a local.mk with arbitrary stuff, then this is not reproducible, and this should not be used to tag a release.

Where an SBOM is usefull, is when it is generated for a release, so that it can traval along it. A release should be done in a constrained, reproducible environment, like a CD pipeline.

Quoted 4 lines
Quoted 3 lines

And anyway, I don't think it's very relevant if this SBoM generation happens
on a developer machine or in CI. In either case, I don't think we want to rely on a host-installed Python version.

Why?

Indeed, why? As far as I see, such a script would only need modules from the stdlib (json, maybe subprocess, re, and similar). We'd just need to state that Buildroot requires a python3.8+ on the host, which to be honest would hugely help in simpifying our other scripts...

We indeed currently do not require such a python to be present, as that's not part of our requirements.

Quoted 4 lines
Quoted 3 lines
Quoted 1 line

We should now drop printvars.

OMG, I completely forgot about show-vars! But indeed, we still need to keep
show-vars. And we should drop printvars.

We don't care about backward compatibility?

Yes, but printvars is riddled with issues, and show-vars as been around for a while already...

Quoted 10 lines
Quoted 6 lines
Quoted 2 lines

and so there is no reason to inject those in cyclonedx just because something else may need it. That's what show-info is for

Anyway, for me replacing show-info is not the most important thing, it's more
a nice-to-have. What I don't want is to generate CycloneDX by post-processing the show-info output, because it turns out that that is still fairly complicated and it's just easier to directly generate CycloneDX from make.

Why is it "fairly complicated"? It sounds to me that post-processing show-info into a simple and nice Python script to generate CycloneDX is not at all "fairly complicated" but rather "fairly simpler" than doing it in make.

Here 's even a little patch that would allow to write everything in python, and generate the thign as part of the standard build process (which I would not suggest, but for the sake of the argument):

commit ccf4722ae0a9755aa73c9c1eaa2a6935b77d7a7c
Author: Yann E. MORIN <[email protected]>
AuthorDate: Wed Apr 10 22:25:11 2024 +0200
Commit: Yann E. MORIN <[email protected]>
CommitDate: Wed Apr 10 22:42:51 2024 +0200

Makefile: cyclonedx from show-info ezpz

diff --git a/Makefile b/Makefile
index d1caec63b5..344f72244a 100644
--- a/Makefile
+++ b/Makefile
@@ -497,7 +497,7 @@ export BASE_DIR
#
################################################################################

-all: world
+all: world sbom-all

# Include legacy before the other things, because package .mk files # may rely on it.
@@ -918,20 +918,37 @@ check-dependencies:
@cd "$(CONFIG_DIR)"; \
$(TOPDIR)/support/scripts/graph-depends -C

+SHOW_INFO = \
+ $(call clean-json, \
+ { $(foreach p, \
+ $(sort $(foreach i,$(PACKAGES) $(TARGETS_ROOTFS), \
+ $(i) \
+ $($(call UPPERCASE,$(i))_FINAL_RECURSIVE_DEPENDENCIES) \
+ ) \
+ ), \
+ $(call json-info,$(call UPPERCASE,$(p)))$(comma) \
+ ) } \
+ ) \
+
.PHONY: show-info
show-info:
@:
- $(info $(call clean-json, \
- { $(foreach p, \
- $(sort $(foreach i,$(PACKAGES) $(TARGETS_ROOTFS), \
- $(i) \
- $($(call UPPERCASE,$(i))_FINAL_RECURSIVE_DEPENDENCIES) \
- ) \
- ), \
- $(call json-info,$(call UPPERCASE,$(p)))$(comma) \
- ) } \
- ) \
- )
+ # Use $(foreach...) like for show-vars, just in case
+ $(foreach i,$(SHOW_INFO),$(info $(i)))
+
+sbom-all: sbom-buildroot sbom-cyclonedx +
+.PHONY: $(O)/info.json
+$(O)/info.json:
+ $(file > $(@),$(SHOW_INFO))
+
+.PHONY: sbom-buildroot
+sbom-buildroot: $(O)/sbom.json
+ @:
+
+.PHONY: sbom-cyclonedx
+sbom-cyclonedx: $(O)/sbom.json world legal-info
+ $(TOPDIR)/support/scripts/cyclonedx --show-info=$(<) --legal-info=$(LEGAL_INFO_DIR)

.PHONY: pkg-stats
pkg-stats:

Also: badly tab-mangled, but that's just for the example, not meant to be applied. Totally untested, although I already had the SHOW_INFO variable implemented a while back already, and IIRC that was working.

Regards,
Yann E. MORIN.

Signature · 10 lines
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot