pull-spdx-license.py: add the SPDX licenses exceptions
PATCH 1/1 pull-spdx-license.py: add the SPDX licenses exceptions
Fetch and extract the SPDX exception licenses as the normal licenses are handled.
[YOCTO #16294]
--- scripts/pull-spdx-licenses.py | 22 ++++++++++++---------- 1 file changed, 12 insertions(+), 10 deletions(-)
scripts/pull-spdx-licenses.py +12 -10
@@ -55,20 +55,22 @@ def main(): data = json.load(response) version = data["tag_name"] - print(f"Pulling SPDX license list version {version}") - req = urllib.request.Request( - f"https://raw.githubusercontent.com/spdx/license-list-data/{version}/json/licenses.json" - ) - with urllib.request.urlopen(req) as response: - spdx_licenses = json.load(response) + spdx_licenses = {} + for jsonfile in ["licenses.json", "exceptions.json"]: + req = urllib.request.Request( + f"https://raw.githubusercontent.com/spdx/license-list-data/{version}/json/{jsonfile}" + ) + with urllib.request.urlopen(req) as response: + spdx_licenses.update(json.load(response)) + print(len(spdx_licenses)) with (TOP_DIR / "meta" / "files" / "spdx-licenses.json").open("w") as f: json.dump(spdx_licenses, f, sort_keys=True, indent=2) - total_count = len(spdx_licenses["licenses"]) + total_count = len(spdx_licenses["licenses"] + spdx_licenses["exceptions"]) updated = 0 - for idx, lic in enumerate(spdx_licenses["licenses"]): - lic_id = lic["licenseId"] + for idx, lic in enumerate(spdx_licenses["licenses"] + spdx_licenses["exceptions"]): + lic_id = lic.get("licenseId") or lic.get("licenseExceptionId") print(f"[{idx + 1} of {total_count}] ", end="")
@@ -88,7 +90,7 @@ def main(): continue with dest_license_file.open("w") as f: - f.write(lic_data["licenseText"]) + f.write(lic_data.get("licenseText") or lic_data.get("licenseExceptionText")) updated += 1 print("done")
Signature · 1 line
2.47.3
Quoted 3 lines · Jan Vermaete via lists.openembedded.org
Fetch and extract the SPDX exception licenses as the normal licenses are handled.
Thanks, but we can't actually use the license exceptions this way. The problem is that exceptions are only valid after an spdx WITH statement (e.g. "GPL-3.0 AND OpenSSL-exception" is not valid), which our license expressions do not support. Also, it's better to keep the licenses and exceptions in separate files so that it's easier to distinguish them when parsing.
Regardless, I have been working on converting all our license expressions to valid SPDX expressions, but it got sidetracked by some SBoM work. Part of that was somewhat similar patch to fetch the exceptions. If you would like to take a look, the branch is contrib/jpew/spdx-licenses. I hope to get back to working on that as soon as the SBoM stuff is sorted out.
Quoted 76 lines
[YOCTO #16294]
Signed-off-by Jan Vermaete <[email protected]>--- scripts/pull-spdx-licenses.py | 22 ++++++++++++---------- 1 file changed, 12 insertions(+), 10 deletions(-)scripts/pull-spdx-licenses.py +0
-3@@ -55,20 +55,22 @@ def main(): data = json.load(response) version = data["tag_name"] - print(f"Pulling SPDX license list version {version}") - req = urllib.request.Request( - f"https://raw.githubusercontent.com/spdx/license-list-data/{version}/json/licenses.json
"scripts/pull-spdx-licenses.py +4
-3- ) - with urllib.request.urlopen(req) as response: - spdx_licenses = json.load(response) + spdx_licenses = {} + for jsonfile in ["licenses.json", "exceptions.json"]: + req = urllib.request.Request( + f"https://raw.githubusercontent.com/spdx/license-list-data/{version}/json/{jsonfile}
"scripts/pull-spdx-licenses.py +4
-0+ ) + with urllib.request.urlopen(req) as response: + spdx_licenses.update(json.load(response)) + print(len(spdx_licenses)) with (TOP_DIR / "meta" / "files" / "spdx-licenses.json").open("w") asf:
json.dump(spdx_licenses, f, sort_keys=True, indent=2)scripts/pull-spdx-licenses.py +1
-1- total_count = len(spdx_licenses["licenses"]) + total_count = len(spdx_licenses["licenses"] +spdx_licenses["exceptions"])
updated = 0scripts/pull-spdx-licenses.py +1
-2- for idx, lic in enumerate(spdx_licenses["licenses"]): - lic_id = lic["licenseId"] + for idx, lic in enumerate(spdx_licenses["licenses"] +spdx_licenses["exceptions"]):
scripts/pull-spdx-licenses.py +2
-2+ lic_id = lic.get("licenseId") or lic.get("licenseExceptionId") print(f"[{idx + 1} of {total_count}] ", end="")@@ -88,7 +90,7 @@ def main(): continue with dest_license_file.open("w") as f: - f.write(lic_data["licenseText"]) + f.write(lic_data.get("licenseText") or lic_data.get("licenseExceptionText")) updated += 1 print("done") --2.47.3
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group. View/Reply Online (#239697):
https://lists.openembedded.org/g/openembedded-core/message/239697
Mute This Topic: https://lists.openembedded.org/mt/120005594/3616693
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [ [email protected]]
-=-=-=-=-=-=-=-=-=-=-=-
Quoted 3 lines · Joshua Watt via lists.openembedded.org
Thanks, but we can't actually use the license exceptions this way. The problem is that exceptions are only valid after an spdx
WITHstatement (e.g. "GPL-3.0 AND OpenSSL-exception" is not valid), which our license expressions do not support. Also, it's better to keep the licenses and exceptions in separate files so that it's easier to distinguish them when parsing.Regardless, I have been working on converting all our license expressions to valid SPDX expressions, but it got sidetracked by some SBoM work. Part of that was somewhat similar patch to fetch the exceptions. If you would like to take a look, the branch is contrib/jpew/spdx-licenses. I hope to get back to working on that as soon as the SBoM stuff is sorted out.
A related issue is that lack of support for SPDX exceptions holds up correctly specifying fmt's license terms (which is 'MIT WITH fmt-exception' if I understood you right):
https://spdx.org/licenses/fmt-exception.html
https://github.com/fmtlib/fmt/blob/main/doc/LICENSE-exception
The current recipe says simply 'MIT'.
Alex
Quoted 15 lines · Alexander Kanavin
Quoted 1 line · Joshua Watt via lists.openembedded.org
Thanks, but we can't actually use the license exceptions this way. The
problem is that exceptions are only valid after an spdx
WITHstatement (e.g. "GPL-3.0 AND OpenSSL-exception" is not valid), which our license expressions do not support. Also, it's better to keep the licenses and exceptions in separate files so that it's easier to distinguish them when parsing.Quoted 2 lines
Regardless, I have been working on converting all our license
expressions to valid SPDX expressions, but it got sidetracked by some SBoM work. Part of that was somewhat similar patch to fetch the exceptions. If you would like to take a look, the branch is contrib/jpew/spdx-licenses. I hope to get back to working on that as soon as the SBoM stuff is sorted out.
Quoted 9 lines
A related issue is that lack of support for SPDX exceptions holds up correctly specifying fmt's license terms (which is 'MIT WITH fmt-exception' if I understood you right):
https://spdx.org/licenses/fmt-exception.html
https://github.com/fmtlib/fmt/blob/main/doc/LICENSE-exceptionThe current recipe says simply 'MIT'.
Correct. In the short term, you can make a new "OE" license that is "MIT-with-fmt-exception" to capture this (we have examples of this already, see Apache-2.0-with-LLVM-exception), and then once my changes are in you can change to the correct SPDX expression "MIT WITH fmt-exception"
Quoted 3 lines
Alex