Skip to contentSkip to conversation

spdx: Add software file externalRef support

RFC PATCH In review Fabio Berton 2 participants 4 messages Nov 10, 2025 – Nov 17, 2025

lore
Share & tools
lore
#0 lore raw

cover spdx: Add software file externalRef support

Hi all,

When starting to test SPDX 3.0 in our projects, we noticed that it would be necessary to have more information for files fetched via the 'file://' protocol, such as the full path of the file or a URL with git information.

Our first idea was to use 'downloadLocation', but what I understand is that this is a package property, and files fetched from the layer are 'software_File' type. Looking at the SPDX spec, it appears we could use the 'ExternalRef' for this purpose.

The idea is to have two options to add this information: one to add the full path of a file, and another to add the git information 'git+https://host/repo@commit#path/to/file'. The information is added as an 'externalRef' and can be configured using these types:
https://spdx.github.io/spdx-spec/v3.0.1/model/Core/Vocabularies/ExternalRefType/

When using the 'path' option, something like this is added:

```
"externalRef": [
          {
            "type": "ExternalRef",
            "externalRefType": "sourceArtifact",
           "locator": [
              "/home/user/src/openembedded-core/meta/recipes-core/busybox/files/syslog"
            ]
          }
        ],
```

This option is non-reproducible, if the build path changes, the SPDX will be different.

And with the 'git' option:

```
"externalRef": [
          {
            "type": "ExternalRef",
            "externalRefType": "sourceArtifact",
            "locator": [
              "git+https://git.openembedded.org/openembedded-core@ac5d9579a0db63b54bbebb5015de2ae860a462bf#meta/recipes-core/busybox/files/syslog"
            ]
          }
        ],
```

The implementation is not completely finished, but since there is already a thread on this subject,
https://lists.openembedded.org/g/openembedded-core/topic/thoughts_on_spdx_for_files/116135395,
I wanted to share my work and get opinions on how to improve this implementation.

My questions are:

Is the 'externalRef' the right way to add the information in the spdx file?

I'm using the 'choices' type, but this only works when inheriting typecheck.bbclass, and this bbclass is not inherited when using OE-Core with 'nodistro'. Can this 'choices' type be used here?

I still need to find a way to cache Git layer information to avoid calling the 'oe.buildcfg' function every time. Maybe it would be possible to use something like this:
https://git.openembedded.org/openembedded-core/tree/meta/classes/metadata_scm.bbclass
to get information at parsing time. However, this information is only needed when using SPDX_FILE_LOCATION with the git option, and for all layers. Any idea here?

For the git option, we need to get a git remote, but there can be more than one remote per layer, so we need a way to configure these remotes. In this first implementation, I'm assuming that all layers use the same remote, and the remote name can be configured, which fits our current use case.

Should I add a variable like 'SPDX_FILE_LOCATION_GIT_REMOTE_<layername> = "remote_name"' to set a specific remote for each layer? Would setting the git remote be sufficient to cover most cases?

Any feedback or suggestions would be appreciated.

Best regards,
Fabio

Fabio Berton (1):
spdx: Add software file externalRef support

 meta/classes/create-spdx-3.0.bbclass | 24 +++++++
 meta/lib/oe/sbom30.py                | 14 ++++-
 meta/lib/oe/spdx30_tasks.py          | 93 ++++++++++++++++++++++++++++
 3 files changed, 130 insertions(+), 1 deletion(-)
Signature · 1 line
2.51.1
#1 lore raw

RFC PATCH 1/1 spdx: Add software file externalRef support

From: Fabio Berton <[email protected]>

Add support for including external references in SPDX 3.0 file elements, allowing files fetched via file:// protocol to reference their source location via absolute path or git URL with commit hash.

Change add the following variables:

- SPDX_FILE_LOCATION variable with three options:
* 'none' (default) - disables external references
* 'path' - adds absolute file path as external reference
* 'git' - generates git URL with commit hash in format git+https://host/repo@commit#path/to/file

- SPDX_FILE_LOCATION_REF_TYPE control the external reference type (defaults to 'sourceArtifact')

- SPDX_FILE_LOCATION_GIT_REMOTE to specify which git remote to use when constructing git URLs (defaults to 'origin')

The git URL generation requires files to be in a git repository with a configured remote.

---
 meta/classes/create-spdx-3.0.bbclass | 24 +++++++
 meta/lib/oe/sbom30.py                | 14 ++++-
 meta/lib/oe/spdx30_tasks.py          | 93 ++++++++++++++++++++++++++++
 3 files changed, 130 insertions(+), 1 deletion(-)
meta/classes/create-spdx-3.0.bbclass +24 -0
index a6d2d44e34..efdd1feb59 100644
@@ -122,6 +122,30 @@ SPDX_PACKAGE_URL[doc] = "Provides a place for the SPDX data creator to record \
 the package URL string (in accordance with the Package URL specification) for \
 a software Package."
 
+SPDX_FILE_LOCATION ?= "none"
+SPDX_FILE_LOCATION[type] = "choice"
+SPDX_FILE_LOCATION[choices] = "none path git"
+# TODO: Type validation only works when inheriting typecheck bbclass
+
+# Available options are listed here:
+# https://spdx.github.io/spdx-spec/v3.0.1/model/Core/Vocabularies/ExternalRefType
+SPDX_FILE_LOCATION_REF_TYPE ?= "sourceArtifact"
+
+# Use 'origin' remote as default, this option is only valid when
+# SPDX_FILE_LOCATION is set to 'git'
+SPDX_FILE_LOCATION_GIT_REMOTE ?= "origin"
+
+SPDX_FILE_LOCATION[doc] = "Controls whether and how to add external references \
+    to source files in the SPDX SBOM. Valid options are: \
+    'none' (default) - disables external references \
+    'path' - adds the absolute file path as an external reference \
+    'git' - generates Git URLs with commit hash in the format \
+    'git+https://host/repo@commit#path/to/file' using the configured remote \
+    (see SPDX_FILE_LOCATION_GIT_REMOTE). When using 'git', files must be in a \
+    git repository with a configured remote. The external reference type is \
+    controlled by SPDX_FILE_LOCATION_REF_TYPE (default 'sourceArtifact'). \
+    NOTE: Using 'path' may result in non-reproducible SPDX output."
+
 IMAGE_CLASSES:append = " create-spdx-image-3.0"
 SDK_CLASSES += "create-spdx-sdk-3.0"
 
meta/lib/oe/sbom30.py +13 -1
index 227ac51877..276661b81b 100644
@@ -4,6 +4,7 @@
 # SPDX-License-Identifier: GPL-2.0-only
 #
 
+from collections import namedtuple
 from pathlib import Path
 
 import oe.spdx30
@@ -15,6 +16,9 @@ import os
 import oe.spdx_common
 from datetime import datetime, timezone
 
+# Named tuple for external reference information
+ExternalRef = namedtuple("ExternalRef", ["ref_type", "locator"])
+
 OE_SPDX_BASE = "https://rdf.openembedded.org/spdx/3.0/"
 
 VEX_VERSION = "1.0.0"
@@ -600,7 +604,7 @@ class ObjectSet(oe.spdx30.SHACLObjectSet):
         )
         spdx_file.extension.append(OELicenseScannedExtension())
 
-    def new_file(self, _id, name, path, *, purposes=[]):
+    def new_file(self, _id, name, path, *, purposes=[], external_ref=None):
         sha256_hash = bb.utils.sha256_file(path)
 
         for f in self.by_sha256_hash.get(sha256_hash, []):
@@ -641,6 +645,14 @@ class ObjectSet(oe.spdx30.SHACLObjectSet):
             spdx_file.software_primaryPurpose = purposes[0]
             spdx_file.software_additionalPurpose = purposes[1:]
 
+        if external_ref:
+            spdx_file.externalRef.append(
+                oe.spdx30.ExternalRef(
+                    externalRefType=external_ref.ref_type,
+                    locator=[external_ref.locator],
+                )
+            )
+
         spdx_file.verifiedUsing.append(
             oe.spdx30.Hash(
                 algorithm=oe.spdx30.HashAlgorithm.sha256,
meta/lib/oe/spdx30_tasks.py +93 -0

122 lines not on this page · raw message

Signature · 1 line
2.51.1
#2 lore raw
In reply to Fabio Berton

Hello Fabio,

thanks for your comments and patch!

Quoted 4 lines · Fabio Berton

Our first idea was to use 'downloadLocation', but what I understand is that this is a package property, and files fetched from the layer are 'software_File' type. Looking at the SPDX spec, it appears we could use the 'ExternalRef' for this purpose.

I'm not to familiar with the SPDX spec yet, but adding individual files entries as ExternalRef instead of downloadLocation to a recipes spdx sounds reasonable.

I think in the long term adding a SPDXRef-Layer-xyz entry per layer with a downloadLocation pointing to the subpath of the layer inside a git repo. I'm not quite shure if it would be possible to formulate a dependency on a file contained within a different SPDXRef, e.g.

```
SPDXRef-Layer-xyz:recipes-core/base-files/base-files/fstab
```

or if we'd have to create a SPDXRef Item for each file within a layer in order to reference it properly. That would make it even more verbose.

The approach of having a layer as an independent SPDXRef would mean getting the git revision etc. for that layer would run only once per build and not per file:// entry in SRC_URI.

Quoted 3 lines

The idea is to have two options to add this information: one to add the full path of a file, and another to add the git information

IMO the full path to the file is unneeded information, if the file is solely available locally a NOASSERTION would be appropriate.

Quoted 4 lines

Should I add a variable like 'SPDX_FILE_LOCATION_GIT_REMOTE_<layername> = "remote_name"' to set a specific remote for each layer? Would setting the git remote be sufficient to cover most cases?

In my experimentation I removed the per-layer setting again because tracking the vardeps for the do_create_spdx get's more complicated with per-layer variables.

Quoted 1 line

Sincerely
Daniel Wagenknecht

#3 lore raw
Quoted 18 lines · Daniel Wagenknecht

Hello Fabio,

thanks for your comments and patch!

Quoted 4 lines · Fabio Berton

Our first idea was to use 'downloadLocation', but what I understand is that this is a package property, and files fetched from the layer are 'software_File' type. Looking at the SPDX spec, it appears we could use the 'ExternalRef' for this purpose.

I'm not to familiar with the SPDX spec yet, but adding individual files entries as ExternalRef instead of downloadLocation to a recipes spdx sounds reasonable.

I think in the long term adding a SPDXRef-Layer-xyz entry per layer with a downloadLocation pointing to the subpath of the layer inside a git repo. I'm not quite shure if it would be possible to formulate a dependency on a file contained within a different SPDXRef, e.g.

Quoted 6 lines
```
SPDXRef-Layer-xyz:recipes-core/base-files/base-files/fstab
```

or if we'd have to create a SPDXRef Item for each file within a layer in order to reference it properly. That would make it even more verbose.

Hi Daniel,

Yes, we should have a way to get Git information at parser time to avoid calling for every file://. But I don't know exactly how to do this, because if we need to add a variable in all layers, and of course, we can't do this, we still need a fallback if the variable doesn't exist. In my case, we don't use OE-Core from https://git.openembedded.org/, we have all layers in an internal infrastructure, so we need to change all variables to point to our fork. My idea to use functions from 'oe.buildcfg' is to get Git information from the layer and not from variables, it doesn't matter if it's a fork or not. But I didn't cover the case where different remotes are used. I know that when using repo to manage Git repositories, it's common to use different remotes, e.g., https://github.com/Freescale/fsl-community-bsp-platform/blob/scarthgap/default.xml. Honestly, I don't know if adding a variable to set the "downloadLocation" will be better or not.

Quoted 10 lines

The approach of having a layer as an independent SPDXRef would mean getting the git revision etc. for that layer would run only once per build and not per file:// entry in SRC_URI.

Quoted 3 lines

The idea is to have two options to add this information: one to add the full path of a file, and another to add the git information

IMO the full path to the file is unneeded information, if the file is solely available locally a NOASSERTION would be appropriate.

The 'path' option is to not use the 'Git' information, e.g., when using a tarball and not a Git repo. The 'locator' will be '/home/user/src/openembedded-core/meta/recipes-core/busybox/files/syslog' instead of 'git+[https://git.openembedded.org/openembedded-core@ac5d9579a0db63b54bbebb5015de2ae860a462bf#meta/recipes-core/busybox/files/syslog](https://git.openembedded.org/openembedded-core@ac5d9579a0db63b54bbebb5015de2ae860a462bf#meta/recipes-core/busybox/files/syslog)'

Quoted 8 lines
Quoted 5 lines

Should I add a variable like 'SPDX_FILE_LOCATION_GIT_REMOTE_<layername> = "remote_name"' to set a specific remote for each layer? Would setting the git remote be sufficient to cover most cases?

In my experimentation I removed the per-layer setting again because tracking the vardeps for the do_create_spdx get's more complicated with per-layer variables.

Uhmm, good point, I didn't think about vardeps.

Quoted 3 lines
Quoted 1 line

Sincerely
Daniel Wagenknecht