utils/generate-cyclonedx: fixup scp-style git sites
PATCH 1/1 utils/generate-cyclonedx: fixup scp-style git sites
Commit e8c54ffb3d ("utils/generate-cyclonedx: generate vcs externalReferences for source repos") added externalReferences to the source code of packages.
This unfortunately causes issues with packages (in br2-external) fetching from git using the scp-like syntax, E.G.:
FOO_SITE_METHOD = git
FOO_SITE = [email protected]:<project>/<repo>.git
Which ends up in the SBOM as:
[
{
"type": "vcs",
"url": "[email protected]:<project>/<repo>.git",
"comment": "git repository" }
]
This (correctly) causes Dependency track to reject the SBOM import with:
{
"status": 400,
"title": "The uploaded BOM is invalid",
"detail": "Schema validation failed",
"errors": [
"$.components[2].externalReferences[0].url: does not match the iri-reference pattern must be a valid RFC 3987 IRI-reference", "$.components[2].externalReferences[0].url: does not match the iri-reference pattern must be a valid RFC 3987 IRI-reference", "$.components[2].externalReferences[0].url: does not match the regex pattern ^urn:cdx:[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}/[1-9][0-9]*$",
]
}
The CycloneDX spec indeed requires a URI:
The URI (URL or URN) to the external reference. External references are URIs and therefore can accept any URL scheme including https (RFC-7230), mailto (RFC-2368), tel (RFC-3966), and dns (RFC-4501)
The user@host:project/repo.git is a git-specific shorthand for a git-over-ssh URL. From man git-clone:
Git supports ssh, git, http, and https protocols (in addition, ftp and ftps can be used for fetching, but this is inefficient and deprecated; do not use them).
The native transport (i.e. git:// URL) does no authentication and should be used with caution on unsecured networks.
The following syntaxes may be used with them:
• ssh://[user@]host.xz[:port]/path/to/repo.git/ • git://host.xz[:port]/path/to/repo.git/ • http[s]://host.xz[:port]/path/to/repo.git/ • ftp[s]://host.xz[:port]/path/to/repo.git/
An alternative scp-like syntax may also be used with the ssh protocol:
• [user@]host.xz:path/to/repo.git/
So convert the scp-like syntax to ssh:// URLs in parse_uris() for spec compliance.
--- utils/generate-cyclonedx | 6 ++++++ 1 file changed, 6 insertions(+)
utils/generate-cyclonedx +6 -0
@@ -280,6 +280,12 @@ def parse_uris(uris: list[str]) -> Iterator[tuple[list[str], str]]: scheme, _, stripped_uri = uri.partition("+") if stripped_uri: parsed = urllib.parse.urlparse(stripped_uri) + + # convert scp-style host:path site to ssh:// uri + if scheme == "git" and not parsed.scheme: + host, _, path = stripped_uri.partition(":") + stripped_uri = "ssh://" + host + "/" + path + if parsed.hostname != "sources.buildroot.net": yield scheme.split("|"), stripped_uri
Signature · 6 lines
2.47.3 _______________________________________________ buildroot mailing list [email protected] https://lists.buildroot.org/mailman/listinfo/buildroot
Hi Peter,
In reply of:
Quoted 57 lines
Commit
e8c54ffb3d("utils/generate-cyclonedx: generate vcs externalReferences for source repos") added externalReferences to the source code of packages.This unfortunately causes issues with packages (in br2-external) fetching from git using the scp-like syntax, E.G.:
FOO_SITE_METHOD= git
FOO_SITE= [email protected]:<project>/<repo>.gitWhich ends up in the SBOM as:
[
{ "type": "vcs", "url": "[email protected]:<project>/<repo>.git", "comment": "git repository"}
]This (correctly) causes Dependency track to reject the SBOM import with:
{
"status": 400,
"title": "The uploaded BOM is invalid",
"detail": "Schema validation failed",
"errors": [
"$.components[2].externalReferences[0].url: does not match the iri-reference pattern must be a valid RFC 3987 IRI-reference", "$.components[2].externalReferences[0].url: does not match the iri-reference pattern must be a valid RFC 3987 IRI-reference", "$.components[2].externalReferences[0].url: does not match the regex pattern ^urn:cdx:[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}/[1-9][0-9]*$",
]
}The CycloneDX spec indeed requires a URI:
The URI (URL or URN) to the external reference. External references are URIs and therefore can accept any URL scheme including https (RFC-7230), mailto (RFC-2368), tel (RFC-3966), and dns (RFC-4501)
The user@host:project/repo.git is a git-specific shorthand for a git-over-ssh URL. From man git-clone:
Git supports ssh, git, http, and https protocols (in addition, ftp and ftps can be used for fetching, but this is inefficient and deprecated; do not use them).
The native transport (i.e.
git://URL) does no authentication and should be used with caution on unsecured networks.The following syntaxes may be used with them:
• ssh://[user@]host.xz[:port]/path/to/repo.git/ • git://host.xz[:port]/path/to/repo.git/ • http[s]://host.xz[:port]/path/to/repo.git/ • ftp[s]://host.xz[:port]/path/to/repo.git/
Quoted 8 lines
An alternative scp-like syntax may also be used with the ssh protocol:
• [user@]host.xz:path/to/repo.git/
So convert the scp-like syntax to ssh:// URLs in
parse_uris()for spec compliance.Signed-off-by Peter Korsgaard <[email protected]>
I see that "downloads" section from "show-info" are not tested on the generate-cyclonedx tests yet. I will add this later.
Thanks for the detailled description !
Quoted 28 lines
--- utils/generate-cyclonedx | 6 ++++++ 1 file changed, 6 insertions(+)utils/generate-cyclonedx +6
-0@@ -280,6 +280,12 @@ def parse_uris(uris: list[str]) -> Iterator[tuple[list[str], str]]: scheme, _, stripped_uri = uri.partition("+") if stripped_uri: parsed = urllib.parse.urlparse(stripped_uri) + + # convert scp-style host:path site to ssh:// uri + if scheme == "git" and not parsed.scheme: + host, _, path = stripped_uri.partition(":") + stripped_uri = "ssh://" + host + "/" + path + if parsed.hostname != "sources.buildroot.net": yield scheme.split("|"), stripped_uriSignature · 6 lines
2.47.3 _______________________________________________ buildroot mailing list [email protected] https://lists.buildroot.org/mailman/listinfo/buildroot
_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot
Quoted 59 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
"Peter" == Peter Korsgaard <[email protected]> writes:
Commit
e8c54ffb3d("utils/generate-cyclonedx: generate vcs externalReferences for source repos") added externalReferences to the source code of packages.This unfortunately causes issues with packages (in br2-external) fetching from git using the scp-like syntax, E.G.:
FOO_SITE_METHOD= git
FOO_SITE= [email protected]:<project>/<repo>.gitWhich ends up in the SBOM as:
[
{ "type": "vcs", "url": "[email protected]:<project>/<repo>.git", "comment": "git repository"}
]This (correctly) causes Dependency track to reject the SBOM import with:
{
"status": 400,
"title": "The uploaded BOM is invalid",
"detail": "Schema validation failed",
"errors": [
"$.components[2].externalReferences[0].url: does not match the iri-reference pattern must be a valid RFC 3987 IRI-reference", "$.components[2].externalReferences[0].url: does not match the iri-reference pattern must be a valid RFC 3987 IRI-reference", "$.components[2].externalReferences[0].url: does not match the regex pattern ^urn:cdx:[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}/[1-9][0-9]*$",
]
}The CycloneDX spec indeed requires a URI:
The URI (URL or URN) to the external reference. External references are URIs and therefore can accept any URL scheme including https (RFC-7230), mailto (RFC-2368), tel (RFC-3966), and dns (RFC-4501)
The user@host:project/repo.git is a git-specific shorthand for a git-over-ssh URL. From man git-clone:
Git supports ssh, git, http, and https protocols (in addition, ftp and ftps can be used for fetching, but this is inefficient and deprecated; do not use them).
The native transport (i.e.
git://URL) does no authentication and should be used with caution on unsecured networks.The following syntaxes may be used with them:
• ssh://[user@]host.xz[:port]/path/to/repo.git/ • git://host.xz[:port]/path/to/repo.git/ • http[s]://host.xz[:port]/path/to/repo.git/ • ftp[s]://host.xz[:port]/path/to/repo.git/
Quoted 8 lines
An alternative scp-like syntax may also be used with the ssh protocol:
• [user@]host.xz:path/to/repo.git/
So convert the scp-like syntax to ssh:// URLs in
parse_uris()for spec compliance.Signed-off-by Peter Korsgaard <[email protected]>
Committed, thanks.
Signature · 5 lines
Bye, Peter Korsgaard _______________________________________________ buildroot mailing list [email protected] https://lists.buildroot.org/mailman/listinfo/buildroot
Thomas Perale via buildroot
AppliedIn reply of:
Quoted 57 lines
Commit
e8c54ffb3d("utils/generate-cyclonedx: generate vcs externalReferences for source repos") added externalReferences to the source code of packages.This unfortunately causes issues with packages (in br2-external) fetching from git using the scp-like syntax, E.G.:
FOO_SITE_METHOD= git
FOO_SITE= [email protected]:<project>/<repo>.gitWhich ends up in the SBOM as:
[
{ "type": "vcs", "url": "[email protected]:<project>/<repo>.git", "comment": "git repository"}
]This (correctly) causes Dependency track to reject the SBOM import with:
{
"status": 400,
"title": "The uploaded BOM is invalid",
"detail": "Schema validation failed",
"errors": [
"$.components[2].externalReferences[0].url: does not match the iri-reference pattern must be a valid RFC 3987 IRI-reference", "$.components[2].externalReferences[0].url: does not match the iri-reference pattern must be a valid RFC 3987 IRI-reference", "$.components[2].externalReferences[0].url: does not match the regex pattern ^urn:cdx:[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}/[1-9][0-9]*$",
]
}The CycloneDX spec indeed requires a URI:
The URI (URL or URN) to the external reference. External references are URIs and therefore can accept any URL scheme including https (RFC-7230), mailto (RFC-2368), tel (RFC-3966), and dns (RFC-4501)
The user@host:project/repo.git is a git-specific shorthand for a git-over-ssh URL. From man git-clone:
Git supports ssh, git, http, and https protocols (in addition, ftp and ftps can be used for fetching, but this is inefficient and deprecated; do not use them).
The native transport (i.e.
git://URL) does no authentication and should be used with caution on unsecured networks.The following syntaxes may be used with them:
• ssh://[user@]host.xz[:port]/path/to/repo.git/ • git://host.xz[:port]/path/to/repo.git/ • http[s]://host.xz[:port]/path/to/repo.git/ • ftp[s]://host.xz[:port]/path/to/repo.git/
Quoted 8 lines
An alternative scp-like syntax may also be used with the ssh protocol:
• [user@]host.xz:path/to/repo.git/
So convert the scp-like syntax to ssh:// URLs in
parse_uris()for spec compliance.Signed-off-by Peter Korsgaard <[email protected]>
Applied to 2025.02.x & 2026.05.x. Thanks
Quoted 28 lines
--- utils/generate-cyclonedx | 6 ++++++ 1 file changed, 6 insertions(+)utils/generate-cyclonedx +6
-0@@ -280,6 +280,12 @@ def parse_uris(uris: list[str]) -> Iterator[tuple[list[str], str]]: scheme, _, stripped_uri = uri.partition("+") if stripped_uri: parsed = urllib.parse.urlparse(stripped_uri) + + # convert scp-style host:path site to ssh:// uri + if scheme == "git" and not parsed.scheme: + host, _, path = stripped_uri.partition(":") + stripped_uri = "ssh://" + host + "/" + path + if parsed.hostname != "sources.buildroot.net": yield scheme.split("|"), stripped_uriSignature · 6 lines
2.47.3 _______________________________________________ buildroot mailing list [email protected] https://lists.buildroot.org/mailman/listinfo/buildroot
_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot