scripts: mandate use of SPDX-License-Identifier tags in new files
PATCH v3 1/3 scripts: mandate that new files have SPDX-License-Identifier
In reply to Daniel P. BerrangéGoing forward we want all newly created source files to have an SPDX-License-Identifier tag present.
Initially mandate this for C, Python, Perl, Shell source files, as well as JSON (QAPI) and Makefiles, while encouraging users to consider it for other file types.
--- scripts/checkpatch.pl | 30 ++++++++++++++++++++++++++++++ 1 file changed, 30 insertions(+)
scripts/checkpatch.pl +30 -0
@@ -1378,6 +1378,8 @@ sub process { my $in_imported_file = 0; my $in_no_imported_file = 0; my $non_utf8_charset = 0; + my $expect_spdx = 0; + my $expect_spdx_file; our @report = (); our $cnt_lines = 0;
@@ -1615,6 +1617,34 @@ sub process { WARN("added, moved or deleted file(s), does MAINTAINERS need updating?\n" . $herecurr); } +# All new files should have a SPDX-License-Identifier tag + if ($line =~ /^new file mode\s*\d+\s*$/) { + if ($expect_spdx) { + if ($expect_spdx_file =~ + /\.(c|h|py|pl|sh|json|inc|Makefile)$/) { + # source code files MUST have SPDX license declared + ERROR("New file '$expect_spdx_file' requires " . + "'SPDX-License-Identifer'"); + } else { + # Other files MAY have SPDX license if appropriate + WARNING("Does new file '$expect_spdx_file' need " . + "'SPDX-License-Identifer'?"); + } + } + $expect_spdx = 1; + $expect_spdx_file = undef; + } elsif ($expect_spdx) { + $expect_spdx_file = $realfile unless + defined $expect_spdx_file; + + # SPDX tags may occurr in comments which were + # stripped from '$line', so use '$rawline' + if ($rawline =~ /SPDX-License-Identifier/) { + $expect_spdx = 0; + $expect_spdx_file = undef; + } + } + # Check for wrappage within a valid hunk of the file if ($realcnt != 0 && $line !~ m{^(?:\+|-| |\\ No newline|$)}) { ERROR("patch seems to be corrupt (line wrapped?)\n" .
Signature · 1 line
2.47.1
PATCH v3 2/3 scripts: validate SPDX license choices
In reply to Daniel P. BerrangéWe expect all new code to be contributed with the "GPL-2.0-or-later" license tag. Divergence is permitted if the new file is derived from pre-existing code under a different license, whether from elsewhere in QEMU codebase, or outside.
Issue a warning if the declared license is not "GPL-2.0-or-later", and an error if the license is not one of the handful of the expected licenses to prevent unintended proliferation. The warning asks users to explain their unusual choice of license in the commit message.
--- scripts/checkpatch.pl | 69 +++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 69 insertions(+)
scripts/checkpatch.pl +69 -0
@@ -1353,6 +1353,70 @@ sub checkfilename { } } +sub checkspdx { + my ($file, $expr) = @_; + + # Imported Linux headers probably have SPDX tags, but if they + # don't we're not requiring contributors to fix this, as these + # files are not expected to be modified locally in QEMU. + # Also don't accidentally detect own checking code. + if ($file =~ m,include/standard-headers, || + $file =~ m,linux-headers, || + $file =~ m,checkpatch.pl,) { + return; + } + + my $origexpr = $expr; + + # Flatten sub-expressions + $expr =~ s/\(|\)/ /g; + $expr =~ s/OR|AND/ /g; + + # Merge WITH exceptions to the license + $expr =~ s/\s+WITH\s+/-WITH-/g; + + # Cull more leading/trailing whitespace + $expr =~ s/^\s*//g; + $expr =~ s/\s*$//g; + + my @bits = split / +/, $expr; + + my $prefer = "GPL-2.0-or-later"; + my @valid = qw( + GPL-2.0-only + LGPL-2.1-only + LGPL-2.1-or-later + BSD-2-Clause + BSD-3-Clause + MIT + ); + + my $nonpreferred = 0; + my @unknown = (); + foreach my $bit (@bits) { + if ($bit eq $prefer) { + next; + } + if (grep /^$bit$/, @valid) { + $nonpreferred = 1; + } else { + push @unknown, $bit; + } + } + if (@unknown) { + ERROR("Saw unacceptable licenses '" . join(',', @unknown) . + "', valid choices for QEMU are:\n" . join("\n", $prefer, @valid)); + } + + if ($nonpreferred) { + WARN("Saw acceptable license '$origexpr' but note '$prefer' is " . + "preferred for new files unless the code is derived from a " . + "source file with an existing declared license that must be " . + "retained. Please explain the license choice in the commit " . + "message."); + } +} + sub process { my $filename = shift;
@@ -1645,6 +1709,11 @@ sub process { } } +# Check SPDX-License-Identifier references a permitted license + if ($rawline =~ m,SPDX-License-Identifier: (.*?)(\*/)?\s*$,) { + &checkspdx($realfile, $1); + } + # Check for wrappage within a valid hunk of the file if ($realcnt != 0 && $line !~ m{^(?:\+|-| |\\ No newline|$)}) { ERROR("patch seems to be corrupt (line wrapped?)\n" .
Signature · 1 line
2.47.1
PATCH v3 3/3 scripts: forbid use of arbitrary SPDX tags besides license identifiers
In reply to Daniel P. BerrangéWhile SPDX-License-Identifier is a well known SPDX tag, there are a great many more besides that[1]. These are mostly focused on making machine readable metadata available to the 'reuse' tool and similar. They cover concepts like author names, copyright owners, and much more. It is even possible to define source file line groups and apply different SPDX tags to regions of code within a file.
At this time we're only interested in adopting SPDX for recording the file global licensing info, so detect & reject any other SPDX metadata. If we want to explicitly collect extra data in SPDX format, we can evaluate each data item on its merits when someone wants to propose it at a later date.
[1] https://spdx.github.io/spdx-spec/v2.2.2/file-tags/
https://spdx.github.io/spdx-spec/v2.2.2/file-information/
--- scripts/checkpatch.pl | 12 ++++++++++++ 1 file changed, 12 insertions(+)
scripts/checkpatch.pl +12 -0
@@ -1714,6 +1714,18 @@ sub process { &checkspdx($realfile, $1); } + if ($rawline =~ m,(SPDX-[a-zA-Z0-9-_]+):,) { + my $tag = $1; + my @permitted = qw( + SPDX-License-Identifier + ); + + unless (grep { /^$tag$/ } @permitted) { + ERROR("Tag $tag not permitted in QEMU code, valid " . + "choices are: " . join(", ", @permitted)); + } + } + # Check for wrappage within a valid hunk of the file if ($realcnt != 0 && $line !~ m{^(?:\+|-| |\\ No newline|$)}) { ERROR("patch seems to be corrupt (line wrapped?)\n" .
Signature · 1 line
2.47.1
v3 cover scripts: mandate use of SPDX-License-Identifier tags in new files
One of the items raised at the QEMU maintainers meeting at KVM Forum 2024 was adoption of SPDX-License-Identifier for licensing of newly contributed source files, for which there were no dissenting voices.
Thus, this series proposes a way to put this into action by extending checkpatch.pl to mandate SPDX-License-Identifier in all new files.
Furthermore, anytime it sees SPDX-License-Identifier in any patch, whether a new file or pre-existing, it validates the declared license name. If it is not one of the commonly used QEMU licenses (the GPL variants, MIT, & a few BSD variants), it will report an error. To encourage sticking with GPL-2.0-or-later by default, it will issue a warning even if it is one of the common licenses, encouraging the contributor to double check their choice. This will reduce accidental license proliferation.
Finally, I've seen a few other random SPDX tags such as:
* SPDX-FileCopyrightText - replacing "Copyright ..."
* SPDX-FileContributor - replacing "Authors: ..."
* SPDX-URL - a link to the link license text
* SPDX-sourceInfo - arbitrary free form text about the file
These may or may not be worth considering in QEMU, but this series discourages their usage by raising an error in checkpatch for now.
If we feel we want to adopt any of these, I think it should be through a concious decision applied universally. Inconsistent & adhoc usage of other SPDX tags by a subset of contributors feels like it doesn't seem to give a clear win, and could even be a net loss through making practices inconsistent across the code.
Changed in v3:
* Add missing accepted license LGPL-2.1-only
* Drop LGPL-2.0-only & LGPL-2.0-or-later as acceptable
* Fix typo in commit message
Changed in v2:
* Tweaks to the commit messages
* Expand the message warning about non GPL-2.0-or-later usage, to request an explanation in the commit message for the unusual choice.
Daniel P. Berrangé (3):
scripts: mandate that new files have SPDX-License-Identifier
scripts: validate SPDX license choices
scripts: forbid use of arbitrary SPDX tags besides license identifiers
scripts/checkpatch.pl | 111 ++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 111 insertions(+)
Signature · 1 line
2.47.1
Re: [PATCH v3 2/3] scripts: validate SPDX license choices
In reply to Daniel P. BerrangéQuoted 6 lines · Daniel P. Berrangé
We expect all new code to be contributed with the "GPL-2.0-or-later" license tag. Divergence is permitted if the new file is derived from pre-existing code under a different license, whether from elsewhere in QEMU codebase, or outside.
Quoted 7 lines
Issue a warning if the declared license is not "GPL-2.0-or-later", and an error if the license is not one of the handful of the expected licenses to prevent unintended proliferation. The warning asks users to explain their unusual choice of license in the commit message.
Signed-off-by Daniel P. Berrangé <[email protected]>
thanks
-- PMM