devtools: add new SPDX license compliance checker
PATCH v2 1/1 devtools: add new SPDX license compliance checker
Simple script to look for drivers and scripts that are missing requires SPDX header.
--- devtools/spdx-check.sh | 23 +++++++++++++++++++++++ 1 file changed, 23 insertions(+) create mode 100755 devtools/spdx-check.sh
devtools/spdx-check.sh new file +23 -0
@@ -0,0 +1,23 @@ +#! /bin/sh +# SPDX-License-Identifier: BSD-3-Clause +# Copyright (c) 2019 Microsoft Corporation +# +# Produce a list of files without SPDX license identifiers + +echo "Files without SPDX License" +echo "--------------------------" + +git grep -L SPDX-License-Identifier -- \ + ':^.git*' ':^.ci/*' ':^.travis.yml' \ + ':^README' ':^MAINTAINERS' ':^VERSION' ':^ABI_VERSION' \ + ':^*/Kbuild' ':^*/README' \ + ':^license/' ':^doc/' ':^config/' ':^buildtools/' \ + ':^devtools/cocci/' \ + ':^*.def' ':^*.map' ':^*.ini' ':^*.data' ':^*.cfg' ':^*.txt' + +echo +echo "Files with redundant BSD boilerplate" +echo "------------------------------------" + +git grep -l SPDX-License-Identifier | \ + xargs grep -l 'Redistribution'
Signature · 1 line
2.20.1
Quoted 4 lines · On Wed, 29 Jan 2020 07:59:07 -0800 Stephen Hemminger…
Simple script to look for drivers and scripts that are missing requires SPDX header.
Signed-off-by Stephen Hemminger <[email protected]>
Ping. This should have been merged by now.
PATCH v3 1/1 devtools: add new SPDX license compliance checker
In reply to Stephen HemmingerSimple script to look for drivers and scripts that are missing requires SPDX header.
---
v3 - pickup more places with boilerplate text
avoid false positive for cocci scripts or abignore
devtools/spdx-check.sh | 24 ++++++++++++++++++++++++ 1 file changed, 24 insertions(+) create mode 100755 devtools/spdx-check.sh
devtools/spdx-check.sh new file +24 -0
@@ -0,0 +1,24 @@ +#! /bin/sh +# SPDX-License-Identifier: BSD-3-Clause +# Copyright (c) 2019 Microsoft Corporation +# +# Produce a list of files with incorrect license +# information + +echo "Files without SPDX License" +echo "--------------------------" + +git grep -L SPDX-License-Identifier -- \ + ':^.git*' ':^.ci/*' ':^.travis.yml' \ + ':^README' ':^MAINTAINERS' ':^VERSION' ':^ABI_VERSION' \ + ':^*/Kbuild' ':^*/README' \ + ':^license/' ':^doc/' ':^config/' ':^buildtools/' \ + ':^*.cocci' ':^*.abignore' \ + ':^*.def' ':^*.map' ':^*.ini' ':^*.data' ':^*.cfg' ':^*.txt' + +echo +echo "Files with additional license text" +echo "----------------------------------" + +git grep -l Redistribution -- \ + ':^license/' ':^/devtools/spdx-check.sh'
Signature · 1 line
2.20.1
07/02/2020 18:52, Stephen Hemminger:
Quoted 6 lines
Simple script to look for drivers and scripts that are missing requires SPDX header.
Signed-off-by Stephen Hemminger <[email protected]>--- devtools/spdx-check.sh | 24 ++++++++++++++++++++++++
For consistency with other scripts, it should be named check-spdx.sh
Please add the new file in MAINTAINERS file in the section "Developers and Maintainers Tools".
I think it should be mentioned also in the contributors guide.
devtools/spdx-check.sh line 3
Quoted 6 lines
devtools/spdx-check.sh new file +3
-0@@ -0,0 +1,24 @@ +#! /bin/sh +# SPDX-License-Identifier: BSD-3-Clause +# Copyright (c) 2019 Microsoft Corporation
While talking about license, are you sure the (c) is required?
devtools/spdx-check.sh line 6
Quoted 3 lines
devtools/spdx-check.sh new file +3
-0+# +# Produce a list of files with incorrect license +# information
In order to add this script in a CI, it should return an error code. Based on the error code, the CI can report it as success, warning or failure to patchwork.
devtools/spdx-check.sh line 17
Quoted 11 lines
devtools/spdx-check.sh new file +11
-0+ +echo "Files without SPDX License" +echo "--------------------------" + +git grep -L SPDX-License-Identifier -- \ + ':^.git*' ':^.ci/*' ':^.travis.yml' \ + ':^README' ':^MAINTAINERS' ':^VERSION' ':^ABI_VERSION' \ + ':^*/Kbuild' ':^*/README' \ + ':^license/' ':^doc/' ':^config/' ':^buildtools/' \ + ':^*.cocci' ':^*.abignore' \ + ':^*.def' ':^*.map' ':^*.ini' ':^*.data' ':^*.cfg' ':^*.txt'
This should be considered a critical error.
devtools/spdx-check.sh line 24
Quoted 7 lines
devtools/spdx-check.sh new file +7
-0+ +echo +echo "Files with additional license text" +echo "----------------------------------" + +git grep -l Redistribution -- \ + ':^license/' ':^/devtools/spdx-check.sh'
This should be considered as a warning.
07/02/2020 18:52, Stephen Hemminger:
Quoted 4 lines
Simple script to look for drivers and scripts that are missing requires SPDX header.
Signed-off-by Stephen Hemminger <[email protected]>
[...]
devtools/spdx-check.sh line 17
Quoted 7 lines
devtools/spdx-check.sh new file +7
-0+git grep -L SPDX-License-Identifier -- \ + ':^.git*' ':^.ci/*' ':^.travis.yml' \ + ':^README' ':^MAINTAINERS' ':^VERSION' ':^ABI_VERSION' \ + ':^*/Kbuild' ':^*/README' \ + ':^license/' ':^doc/' ':^config/' ':^buildtools/' \ + ':^*.cocci' ':^*.abignore' \ + ':^*.def' ':^*.map' ':^*.ini' ':^*.data' ':^*.cfg' ':^*.txt'
I think we should plan to add SPDX tag to the doc (rst and svg files).
PATCH v4 1/1 devtools: add new SPDX license compliance checker
In reply to Stephen HemmingerSimple script to look for drivers and scripts that are missing requires SPDX header.
Update the contribution guidelines to indicate that SPDX license identfier is required for this project.
---
v4 - add MAINTAINERS entry
update coding style document
change name of script
MAINTAINERS | 1 + devtools/check-spdx-tag.sh | 77 ++++++++++++++++++++++++ doc/guides/contributing/coding_style.rst | 9 ++- 3 files changed, 85 insertions(+), 2 deletions(-) create mode 100755 devtools/check-spdx-tag.sh
MAINTAINERS +1 -0
@@ -96,6 +96,7 @@ F: devtools/check-maintainers.sh F: devtools/check-forbidden-tokens.awk F: devtools/check-git-log.sh F: devtools/check-includes.sh +F: devtools/check-spdx-tag.sh F: devtools/check-symbol-maps.sh F: devtools/checkpatches.sh F: devtools/get-maintainer.sh
devtools/check-spdx-tag.sh new file +77 -0
77 lines not on this page · raw message
doc/guides/contributing/coding_style.rst +7 -2
@@ -54,8 +54,13 @@ To document a public API, a doxygen-like format must be used: refer to :ref:`dox License Header ~~~~~~~~~~~~~~ -Each file should begin with a special comment containing the appropriate copyright and license for the file. -Generally this is the BSD License, except for code for Linux Kernel modules. +Each file must begin with a special comment containing the +`Software Package Data Exchange (SPDX) License Identfier <https://spdx.org/using-spdx-license-identifier>`_. + +Generally this is the BSD License, except for code granted special exceptions. +The SPDX licences identifier is sufficient, a file should not contain +an additional text version of the license (boilerplate). + After any copyright header, a blank line should be left before any other contents, e.g. include statements in a C file. C Preprocessor Directives
Signature · 1 line
2.20.1
Quoted 12 lines · On Mon, 24 Feb 2020 13:01:30 -0800 Stephen Hemminger…
Simple script to look for drivers and scripts that are missing requires SPDX header.
Update the contribution guidelines to indicate that SPDX license identfier is required for this project.
Signed-off-by Stephen Hemminger <[email protected]>---
v4 - addMAINTAINERSentry
update coding style document
change name of script
Quoted 5 lines
MAINTAINERS | 1 + devtools/check-spdx-tag.sh | 77 ++++++++++++++++++++++++ doc/guides/contributing/coding_style.rst | 9 ++- 3 files changed, 85 insertions(+), 2 deletions(-) create mode 100755 devtools/check-spdx-tag.sh
Still waiting for this script all requested changes were done months ago.
Current output is:
$ ./devtools/check-spdx-tag.sh -v Files without SPDX License --------------------------
Files with redundant license text
---------------------------------app/test/test_timer_racecond.clib/librte_ethdev/rte_ethdev_pci.hlib/librte_ethdev/rte_ethdev_vdev.h
total: 0 errors, 3 warnings
Quoted 12 lines · On Mon, 24 Feb 2020 13:01:30 -0800 Stephen Hemminger…
Simple script to look for drivers and scripts that are missing requires SPDX header.
Update the contribution guidelines to indicate that SPDX license identfier is required for this project.
Signed-off-by Stephen Hemminger <[email protected]>---
v4 - addMAINTAINERSentry
update coding style document
change name of script
Quoted 6 lines
MAINTAINERS | 1 + devtools/check-spdx-tag.sh | 77 ++++++++++++++++++++++++ doc/guides/contributing/coding_style.rst | 9 ++- 3 files changed, 85 insertions(+), 2 deletions(-) create mode 100755 devtools/check-spdx-tag.sh
Going through the patch backlog.
Why is this not merged yet?
Why is not in DPDK 20.05?
11/06/2020 20:46, Stephen Hemminger:
Quoted 9 lines
Quoted 6 lines · On Mon, 24 Feb 2020 13:01:30 -0800 Stephen Hemminger…
Simple script to look for drivers and scripts that are missing requires SPDX header.
Update the contribution guidelines to indicate that SPDX license identfier is required for this project.
Quoted 6 lines
Quoted 1 line
Signed-off-by Stephen Hemminger <[email protected]>Going through the patch backlog.
Why is this not merged yet?
Why is not in DPDK 20.05?
I would like to see some reviews on such patch
which looks like a policy.
24/02/2020 22:01, Stephen Hemminger:
Quoted 1 line
+tmpfile=$(mktemp)
Please check how other temp files are created in other scripts for consitency.
devtools/check-spdx-tag.sh line 21
Quoted 5 lines
devtools/check-spdx-tag.sh new file +5
-0+ git grep -L SPDX-License-Identifier -- \ + ':^.git*' ':^.ci/*' ':^.travis.yml' \ + ':^README' ':^MAINTAINERS' ':^VERSION' ':^ABI_VERSION' \ + ':^*/Kbuild' ':^*/README' \ + ':^license/' ':^doc/' ':^config/' ':^buildtools/' \
I think doc/ should be part of the license check,
same for buildtools/.
devtools/check-spdx-tag.sh line 29
Quoted 8 lines
devtools/check-spdx-tag.sh new file +8
-0+ ':^*.cocci' ':^*.abignore' \ + ':^*.def' ':^*.map' ':^*.ini' ':^*.data' ':^*.cfg' ':^*.txt' \ + > $tmpfile + + errors=0 + while read -r line + do $quiet || echo $line + errors=$((errors + 1))
I'm surprised this works for you.
In general, "while" creates a subshell which makes impossible updating a variable.
I recommend using "for" with IFS=$'\n'.
Quoted 6 lines
devtools/check-spdx-tag.sh new file +6
-0+ done < $tmpfile +} + +check_boilerplate() { + if $verbose ; then + echo
devtools/check-spdx-tag.sh line 45
Quoted 10 lines
devtools/check-spdx-tag.sh new file +10
-0+ echo "Files with redundant license text" + echo "---------------------------------" + fi + + git grep -l Redistribution -- \ + ':^license/' ':^/devtools/check-spdx-tag.sh' | + while read line + do $quiet || echo $line + warnings=$((warnings + 1)) + done
Same comment about "while" subshell.
devtools/check-spdx-tag.sh line 50
Quoted 5 lines
devtools/check-spdx-tag.sh new file +5
-0+ + warnings=0 + while read -r line + do $quiet || echo $line + warnings=$((errors + 1))
Here too
Quoted 2 lines
(quoted diff) +2
-0+ done < $tmpfile +}
[...]
Quoted 2 lines
+Each file must begin with a special comment containing the
+Software Package Data Exchange (SPDX) License Identfier <https://spdx.org/using-spdx-license-identifier>_.
Typo: Identifier
doc/guides/contributing/coding_style.rst To document a public API, a doxygen-like format must be used: refer to :ref:`dox · line 60
Quoted 2 lines
doc/guides/contributing/coding_style.rst +2
-0+ +Generally this is the BSD License, except for code granted special exceptions.
Is a verb missing?
Quoted 2 lines
doc/guides/contributing/coding_style.rst +2
-0+The SPDX licences identifier is sufficient, a file should not contain +an additional text version of the license (boilerplate).
Hi Thomas, Stephen,
Quoted 15 lines · Thomas Monjalon
24/02/2020 22:01, Stephen Hemminger:
Quoted 1 line
+tmpfile=$(mktemp)
Please check how other temp files are created in other scripts for consitency.
devtools/check-spdx-tag.sh line 21
Quoted 5 lines
devtools/check-spdx-tag.sh new file +5
-0+ git grep -L SPDX-License-Identifier -- \ + ':^.git*' ':^.ci/*' ':^.travis.yml' \ + ':^README' ':^MAINTAINERS' ':^VERSION' ':^ABI_VERSION' \ + ':^*/Kbuild' ':^*/README' \ + ':^license/' ':^doc/' ':^config/' ':^buildtools/' \I think doc/ should be part of the license check,
same for buildtools/.
Quoted 14 lines
devtools/check-spdx-tag.sh line 29
Quoted 8 lines
devtools/check-spdx-tag.sh new file +8
-0+ ':^*.cocci' ':^*.abignore' \ + ':^*.def' ':^*.map' ':^*.ini' ':^*.data' ':^*.cfg' ':^*.txt' \ + > $tmpfile + + errors=0 + while read -r line + do $quiet || echo $line + errors=$((errors + 1))I'm surprised this works for you.
In general, "while" creates a subshell which makes impossible updating a variable.
I recommend using "for" with IFS=$'\n'.
No it should work, while will only spawn a subshell if you use a pipe with it. Ex:
err=0; while true; do err=$((err + 1)); done # $err changes err=0; yes | while read -r y; do err=$((err + 1)); done # $err is always 0
Using for could be an issue, as you are then limited by the number of parameters allowed. Additionally, the script is written for POSIX shell, using $'\n' is forbidden and the POSIX equivalent is very ugly:
IFS="$(printf '%b_' '\n')"; IFS="${IFS%_}" # protect trailing \nQuoted 5 lines
Quoted 5 lines
devtools/check-spdx-tag.sh new file +5
-0+ done < $tmpfile +} + +check_boilerplate() { + if $verbose ; then
Quoted 8 lines
devtools/check-spdx-tag.sh line 42
Quoted 8 lines
devtools/check-spdx-tag.sh new file +8
-0+ echo + echo "Files with redundant license text" + echo "---------------------------------" + fi + + git grep -l Redistribution -- \ + ':^license/' ':^/devtools/check-spdx-tag.sh' | + while read line
Missing a -r to read here: https://www.shellcheck.net/wiki/SC2162
Generally, best to use shellcheck when writing a script, especially if using /bin/sh.
Quoted 33 lines
devtools/check-spdx-tag.sh line 45
Quoted 3 lines
devtools/check-spdx-tag.sh new file +3
-0+ do $quiet || echo $line + warnings=$((warnings + 1)) + doneSame comment about "while" subshell.
devtools/check-spdx-tag.sh line 50
Quoted 5 lines
devtools/check-spdx-tag.sh new file +5
-0+ + warnings=0 + while read -r line + do $quiet || echo $line + warnings=$((errors + 1))Here too
Quoted 2 lines
(quoted diff) +2
-0+ done < $tmpfile +}[...]
Quoted 2 lines
+Each file must begin with a special comment containing the
+Software Package Data Exchange (SPDX) License Identfier <https://spdx.org/using-spdx-license-identifier>_.Typo: Identifier
doc/guides/contributing/coding_style.rst To document a public API, a doxygen-like format must be used: refer to :ref:`dox · line 60
Quoted 2 lines
doc/guides/contributing/coding_style.rst +2
-0+ +Generally this is the BSD License, except for code granted special exceptions.Is a verb missing?
Quoted 2 lines
doc/guides/contributing/coding_style.rst +2
-0+The SPDX licences identifier is sufficient, a file should not contain +an additional text version of the license (boilerplate).
Signature · 1 line
Gaëtan
Quoted 65 lines · On Thu, 11 Jun 2020 23:39:55 +0200 Thomas Monjalon <[email protected]> wrote:
24/02/2020 22:01, Stephen Hemminger:
Quoted 1 line
+tmpfile=$(mktemp)
Please check how other temp files are created in other scripts for consitency.
devtools/check-spdx-tag.sh line 21
Quoted 5 lines
devtools/check-spdx-tag.sh new file +5
-0+ git grep -L SPDX-License-Identifier -- \ + ':^.git*' ':^.ci/*' ':^.travis.yml' \ + ':^README' ':^MAINTAINERS' ':^VERSION' ':^ABI_VERSION' \ + ':^*/Kbuild' ':^*/README' \ + ':^license/' ':^doc/' ':^config/' ':^buildtools/' \I think doc/ should be part of the license check,
same for buildtools/.devtools/check-spdx-tag.sh line 29
Quoted 8 lines
devtools/check-spdx-tag.sh new file +8
-0+ ':^*.cocci' ':^*.abignore' \ + ':^*.def' ':^*.map' ':^*.ini' ':^*.data' ':^*.cfg' ':^*.txt' \ + > $tmpfile + + errors=0 + while read -r line + do $quiet || echo $line + errors=$((errors + 1))I'm surprised this works for you.
In general, "while" creates a subshell which makes impossible updating a variable.
I recommend using "for" with IFS=$'\n'.Quoted 6 lines
devtools/check-spdx-tag.sh new file +6
-0+ done < $tmpfile +} + +check_boilerplate() { + if $verbose ; then + echodevtools/check-spdx-tag.sh line 45
Quoted 10 lines
devtools/check-spdx-tag.sh new file +10
-0+ echo "Files with redundant license text" + echo "---------------------------------" + fi + + git grep -l Redistribution -- \ + ':^license/' ':^/devtools/check-spdx-tag.sh' | + while read line + do $quiet || echo $line + warnings=$((warnings + 1)) + doneSame comment about "while" subshell.
devtools/check-spdx-tag.sh line 50
Quoted 5 lines
devtools/check-spdx-tag.sh new file +5
-0+ + warnings=0 + while read -r line + do $quiet || echo $line + warnings=$((errors + 1))Here too
Quoted 2 lines
(quoted diff) +2
-0+ done < $tmpfile +}[...]
Quoted 2 lines
+Each file must begin with a special comment containing the
+Software Package Data Exchange (SPDX) License Identfier <https://spdx.org/using-spdx-license-identifier>_.Typo: Identifier
Quoted 7 lines
doc/guides/contributing/coding_style.rst To document a public API, a doxygen-like format must be used: refer to :ref:`dox · line 60
Quoted 2 lines
doc/guides/contributing/coding_style.rst +2
-0+ +Generally this is the BSD License, except for code granted special exceptions.Is a verb missing?
doc/guides/contributing/coding_style.rst To document a public API, a doxygen-like format must be used: refer to :ref:`dox · line 62
Quoted 2 lines
doc/guides/contributing/coding_style.rst +2
-0+The SPDX licences identifier is sufficient, a file should not contain +an additional text version of the license (boilerplate).
Thanks for the feedback.
Text processing is simpler in python, will rewrite in next version?
12/06/2020 16:53, Stephen Hemminger:
Quoted 73 lines
Quoted 70 lines · On Thu, 11 Jun 2020 23:39:55 +0200 Thomas Monjalon <[email protected]> wrote:
24/02/2020 22:01, Stephen Hemminger:
Quoted 1 line
+tmpfile=$(mktemp)
Please check how other temp files are created in other scripts for consitency.
devtools/check-spdx-tag.sh line 21
Quoted 5 lines
devtools/check-spdx-tag.sh new file +5
-0+ git grep -L SPDX-License-Identifier -- \ + ':^.git*' ':^.ci/*' ':^.travis.yml' \ + ':^README' ':^MAINTAINERS' ':^VERSION' ':^ABI_VERSION' \ + ':^*/Kbuild' ':^*/README' \ + ':^license/' ':^doc/' ':^config/' ':^buildtools/' \I think doc/ should be part of the license check,
same for buildtools/.devtools/check-spdx-tag.sh line 29
Quoted 8 lines
devtools/check-spdx-tag.sh new file +8
-0+ ':^*.cocci' ':^*.abignore' \ + ':^*.def' ':^*.map' ':^*.ini' ':^*.data' ':^*.cfg' ':^*.txt' \ + > $tmpfile + + errors=0 + while read -r line + do $quiet || echo $line + errors=$((errors + 1))I'm surprised this works for you.
In general, "while" creates a subshell which makes impossible updating a variable.
I recommend using "for" with IFS=$'\n'.Quoted 6 lines
devtools/check-spdx-tag.sh new file +6
-0+ done < $tmpfile +} + +check_boilerplate() { + if $verbose ; then + echodevtools/check-spdx-tag.sh line 45
Quoted 10 lines
devtools/check-spdx-tag.sh new file +10
-0+ echo "Files with redundant license text" + echo "---------------------------------" + fi + + git grep -l Redistribution -- \ + ':^license/' ':^/devtools/check-spdx-tag.sh' | + while read line + do $quiet || echo $line + warnings=$((warnings + 1)) + doneSame comment about "while" subshell.
devtools/check-spdx-tag.sh line 50
Quoted 5 lines
devtools/check-spdx-tag.sh new file +5
-0+ + warnings=0 + while read -r line + do $quiet || echo $line + warnings=$((errors + 1))Here too
Quoted 2 lines
(quoted diff) +2
-0+ done < $tmpfile +}[...]
Quoted 2 lines
+Each file must begin with a special comment containing the
+Software Package Data Exchange (SPDX) License Identfier <https://spdx.org/using-spdx-license-identifier>_.Typo: Identifier
doc/guides/contributing/coding_style.rst To document a public API, a doxygen-like format must be used: refer to :ref:`dox · line 60
Quoted 2 lines
doc/guides/contributing/coding_style.rst +2
-0+ +Generally this is the BSD License, except for code granted special exceptions.Is a verb missing?
Quoted 5 lines
Quoted 2 lines
doc/guides/contributing/coding_style.rst To document a public API, a doxygen-like format must be used: refer to :ref:`dox · line 62
Quoted 2 lines
doc/guides/contributing/coding_style.rst +2
-0+The SPDX licences identifier is sufficient, a file should not contain +an additional text version of the license (boilerplate).Thanks for the feedback.
Text processing is simpler in python, will rewrite in next version?
I don't see a need for rewrite.
I think addressing the comments is simpler.
Quoted 50 lines · Stephen Hemminger
Simple script to look for drivers and scripts that are missing requires SPDX header.
Update the contribution guidelines to indicate that SPDX license identfier is required for this project.
Signed-off-by Stephen Hemminger <[email protected]>---
v4 - addMAINTAINERSentry
update coding style document
change name of scriptMAINTAINERS | 1 + devtools/check-spdx-tag.sh | 77 ++++++++++++++++++++++++ doc/guides/contributing/coding_style.rst | 9 ++- 3 files changed, 85 insertions(+), 2 deletions(-) create mode 100755 devtools/check-spdx-tag.shMAINTAINERS +1
-0@@ -96,6 +96,7 @@ F: devtools/check-maintainers.sh F: devtools/check-forbidden-tokens.awk F: devtools/check-git-log.sh F: devtools/check-includes.sh +F: devtools/check-spdx-tag.sh F: devtools/check-symbol-maps.sh F: devtools/checkpatches.sh F: devtools/get-maintainer.shdevtools/check-spdx-tag.sh new file +14
-0@@ -0,0 +1,77 @@ +#! /bin/sh +# SPDX-License-Identifier: BSD-3-Clause +# Copyright (c) 2019 Microsoft Corporation +# +# Produce a list of files with incorrect license tags + +print_usage () { + echo "usage: $(basename $0) [-q] [-v]" + exit 1 +} + +check_spdx() { + if $verbose; then + echo "Files without SPDX License"
devtools/check-spdx-tag.sh line 24
Quoted 10 lines
devtools/check-spdx-tag.sh new file +10
-0+ echo "--------------------------" + fi + git grep -L SPDX-License-Identifier -- \ + ':^.git*' ':^.ci/*' ':^.travis.yml' \ + ':^README' ':^MAINTAINERS' ':^VERSION' ':^ABI_VERSION' \ + ':^*/Kbuild' ':^*/README' \ + ':^license/' ':^doc/' ':^config/' ':^buildtools/' \ + ':^*.cocci' ':^*.abignore' \ + ':^*.def' ':^*.map' ':^*.ini' ':^*.data' ':^*.cfg' ':^*.txt' \ + > $tmpfile
I find it easier to maintain an exclude list by setting a variable and generating the relevant parameters:
excludes='.git* .ci/* .travis.yml */Kbuild */README'
exclude_opt=""
set -f
for pattern in $excludes; do
exclude_opt="$exclude_opt ':^${pattern}'"
done
set +f
printf "\"%s\"\n" "$exclude_opt"However I recognize that means dealing with contrarian globbing issues in shells, so it comes at a price. But I find changing the exclude list much easier that way.
Quoted 76 lines
devtools/check-spdx-tag.sh new file +53
-0+ + errors=0 + while read -r line + do $quiet || echo $line + errors=$((errors + 1)) + done < $tmpfile +} + +check_boilerplate() { + if $verbose ; then + echo + echo "Files with redundant license text" + echo "---------------------------------" + fi + + git grep -l Redistribution -- \ + ':^license/' ':^/devtools/check-spdx-tag.sh' | + while read line + do $quiet || echo $line + warnings=$((warnings + 1)) + done + + warnings=0 + while read -r line + do $quiet || echo $line + warnings=$((errors + 1)) + done < $tmpfile +} + +quiet=false +verbose=false + +while getopts qvh ARG ; do + case $ARG in + q ) quiet=true ;; + v ) verbose=true ;; + h ) print_usage ; exit 0 ;; + ? ) print_usage ; exit 1 ;; + esac +done +shift $(($OPTIND - 1)) + +tmpfile=$(mktemp) +trap 'rm -f -- "$tmpfile"' INT TERM HUP EXIT + +check_spdx +$quiet || echo + +check_boilerplate + +$quiet || echo +echo "total: $errors errors, $warnings warnings" +exit $errorsdoc/guides/contributing/coding_style.rst +7
-2@@ -54,8 +54,13 @@ To document a public API, a doxygen-like format must be used: refer to :ref:`dox License Header ~~~~~~~~~~~~~~ -Each file should begin with a special comment containing the appropriate copyright and license for the file. -Generally this is the BSD License, except for code for Linux Kernel modules. +Each file must begin with a special comment containing the +`Software Package Data Exchange (SPDX) License Identfier <https://spdx.org/using-spdx-license-identifier>`_. + +Generally this is the BSD License, except for code granted special exceptions. +The SPDX licences identifier is sufficient, a file should not contain +an additional text version of the license (boilerplate). + After any copyright header, a blank line should be left before any other contents, e.g. include statements in a C file. C Preprocessor DirectivesSignature · 1 line
2.20.1
Signature · 1 line
Gaëtan
Quoted 74 lines · On Fri, 12 Jun 2020 11:05:27 +0200 Gaëtan Rivet <[email protected]> wrote:
Quoted 50 lines · Stephen Hemminger
Simple script to look for drivers and scripts that are missing requires SPDX header.
Update the contribution guidelines to indicate that SPDX license identfier is required for this project.
Signed-off-by Stephen Hemminger <[email protected]>---
v4 - addMAINTAINERSentry
update coding style document
change name of scriptMAINTAINERS | 1 + devtools/check-spdx-tag.sh | 77 ++++++++++++++++++++++++ doc/guides/contributing/coding_style.rst | 9 ++- 3 files changed, 85 insertions(+), 2 deletions(-) create mode 100755 devtools/check-spdx-tag.shMAINTAINERS +1
-0@@ -96,6 +96,7 @@ F: devtools/check-maintainers.sh F: devtools/check-forbidden-tokens.awk F: devtools/check-git-log.sh F: devtools/check-includes.sh +F: devtools/check-spdx-tag.sh F: devtools/check-symbol-maps.sh F: devtools/checkpatches.sh F: devtools/get-maintainer.shdevtools/check-spdx-tag.sh new file +14
-0@@ -0,0 +1,77 @@ +#! /bin/sh +# SPDX-License-Identifier: BSD-3-Clause +# Copyright (c) 2019 Microsoft Corporation +# +# Produce a list of files with incorrect license tags + +print_usage () { + echo "usage: $(basename $0) [-q] [-v]" + exit 1 +} + +check_spdx() { + if $verbose; then + echo "Files without SPDX License"devtools/check-spdx-tag.sh line 24
Quoted 10 lines
devtools/check-spdx-tag.sh new file +10
-0+ echo "--------------------------" + fi + git grep -L SPDX-License-Identifier -- \ + ':^.git*' ':^.ci/*' ':^.travis.yml' \ + ':^README' ':^MAINTAINERS' ':^VERSION' ':^ABI_VERSION' \ + ':^*/Kbuild' ':^*/README' \ + ':^license/' ':^doc/' ':^config/' ':^buildtools/' \ + ':^*.cocci' ':^*.abignore' \ + ':^*.def' ':^*.map' ':^*.ini' ':^*.data' ':^*.cfg' ':^*.txt' \ + > $tmpfileI find it easier to maintain an exclude list by setting a variable and generating the relevant parameters:
excludes='.git* .ci/* .travis.yml */Kbuild */README' exclude_opt="" set -f for pattern in $excludes; do exclude_opt="$exclude_opt ':^${pattern}'" done set +f printf "\"%s\"\n" "$exclude_opt"
Quoted 2 lines
However I recognize that means dealing with contrarian globbing issues in shells, so it comes at a price. But I find changing the exclude list much easier that way.
This is no easier to maintain. Doing more string stuff in shell is not worth it. If gets to be too much trouble, will consider moving to python3.
PATCH 1/1 devtools: add new SPDX license compliance checker
In reply to Stephen HemmingerSimple script to look for drivers and scripts that are missing requires SPDX header.
---
v5 - fix issue in counting warnings
update Copyright date
MAINTAINERS | 1 + devtools/check-spdx-tag.sh | 73 ++++++++++++++++++++++++ doc/guides/contributing/coding_style.rst | 9 ++- 3 files changed, 81 insertions(+), 2 deletions(-) create mode 100755 devtools/check-spdx-tag.sh
MAINTAINERS +1 -0
@@ -96,6 +96,7 @@ F: devtools/check-maintainers.sh F: devtools/check-forbidden-tokens.awk F: devtools/check-git-log.sh F: devtools/check-includes.sh +F: devtools/check-spdx-tag.sh F: devtools/check-symbol-maps.sh F: devtools/checkpatches.sh F: devtools/get-maintainer.sh
devtools/check-spdx-tag.sh new file +73 -0
73 lines not on this page · raw message
doc/guides/contributing/coding_style.rst +7 -2
@@ -54,8 +54,13 @@ To document a public API, a doxygen-like format must be used: refer to :ref:`dox License Header ~~~~~~~~~~~~~~ -Each file should begin with a special comment containing the appropriate copyright and license for the file. -Generally this is the BSD License, except for code for Linux Kernel modules. +Each file must begin with a special comment containing the +`Software Package Data Exchange (SPDX) License Identfier <https://spdx.org/using-spdx-license-identifier>`_. + +Generally this is the BSD License, except for code granted special exceptions. +The SPDX licences identifier is sufficient, a file should not contain +an additional text version of the license (boilerplate). + After any copyright header, a blank line should be left before any other contents, e.g. include statements in a C file. C Preprocessor Directives
Signature · 1 line
2.20.1
PATCH v5 1/1 devtools: add new SPDX license compliance checker
In reply to Stephen HemmingerSimple script to look for drivers and scripts that are missing requires SPDX header.
---
v5 - address review comments
simplify the display logic
scan doc and buildtools as well
MAINTAINERS | 1 + devtools/check-spdx-tag.sh | 70 ++++++++++++++++++++++++ doc/guides/contributing/coding_style.rst | 9 ++- 3 files changed, 78 insertions(+), 2 deletions(-) create mode 100755 devtools/check-spdx-tag.sh
MAINTAINERS +1 -0
@@ -90,6 +90,7 @@ F: devtools/check-maintainers.sh F: devtools/check-forbidden-tokens.awk F: devtools/check-git-log.sh F: devtools/check-includes.sh +F: devtools/check-spdx-tag.sh F: devtools/check-symbol-maps.sh F: devtools/checkpatches.sh F: devtools/get-maintainer.sh
devtools/check-spdx-tag.sh new file +70 -0
70 lines not on this page · raw message
doc/guides/contributing/coding_style.rst +7 -2
@@ -54,8 +54,13 @@ To document a public API, a doxygen-like format must be used: refer to :ref:`dox License Header ~~~~~~~~~~~~~~ -Each file should begin with a special comment containing the appropriate copyright and license for the file. -Generally this is the BSD License, except for code for Linux Kernel modules. +Each file must begin with a special comment containing the +`Software Package Data Exchange (SPDX) License Identfier <https://spdx.org/using-spdx-license-identifier>`_. + +Generally this is the BSD License, except for code granted special exceptions. +The SPDX licences identifier is sufficient, a file should not contain +an additional text version of the license (boilerplate). + After any copyright header, a blank line should be left before any other contents, e.g. include statements in a C file. C Preprocessor Directives
Signature · 1 line
2.27.0
Quoted 4 lines · On Tue, 14 Jul 2020 16:21:01 -0700 Stephen Hemminger…
Simple script to look for drivers and scripts that are missing requires SPDX header.
Signed-off-by Stephen Hemminger <[email protected]>
Preprocessor Directives
Because of the long delay in merging this tool new files have come in that are not following the required guidelines!
$ ./devtools/check-spdx-tag.sh -v Files without SPDX License -------------------------- doc/guides/custom.css doc/guides/linux_gsg/nic_perf_intel_platform.rst doc/guides/prog_guide/build-sdk-meson.rst drivers/net/bnxt/tf_core/cfa_resource_types.h drivers/net/qede/base/ecore_hsi_func_common.h lib/librte_eal/windows/eal_hugepages.c
Files with redundant license text
---------------------------------app/test/test_timer_racecond.clib/librte_ethdev/rte_ethdev_pci.hlib/librte_ethdev/rte_ethdev_vdev.h
total: 6 errors, 3 warnings
15/07/2020 01:25, Stephen Hemminger:
Quoted 11 lines
Quoted 4 lines · On Tue, 14 Jul 2020 16:21:01 -0700 Stephen Hemminger…
Simple script to look for drivers and scripts that are missing requires SPDX header.
Signed-off-by Stephen Hemminger <[email protected]>Preprocessor Directives
Because of the long delay in merging this tool new files have come in that are not following the required guidelines!
Probably it would have helped if you Cc'ed some persons known to be interested in this topic. Getting reviews is harder recently.
Quoted 4 lines · On Tue, 14 Jul 2020 16:21:01 -0700 Stephen Hemminger…
Simple script to look for drivers and scripts that are missing requires SPDX header.
Signed-off-by Stephen Hemminger <[email protected]>
All comments are addressed should be merged by now.
Thomas Monjalon
Applied15/07/2020 01:21, Stephen Hemminger:
Quoted 4 lines
Simple script to look for drivers and scripts that are missing requires SPDX header.
Signed-off-by Stephen Hemminger <[email protected]>
[...]
Quoted 19 lines
devtools/check-spdx-tag.sh new file +19
-0+#! /bin/sh +# SPDX-License-Identifier: BSD-3-Clause +# Copyright 2020 Microsoft Corporation +# +# Produce a list of files with incorrect license tags + +errors=0 +warnings=0 +quiet=false +verbose=false + +print_usage () { + echo "usage: $(basename $0) [-q] [-v]" + exit 1 +} + +check_spdx() { + if $verbose; then + echo "Files without SPDX License"
devtools/check-spdx-tag.sh line 29
Quoted 10 lines
devtools/check-spdx-tag.sh new file +10
-0+ echo "--------------------------" + fi + git grep -L SPDX-License-Identifier -- \ + ':^.git*' ':^.ci/*' ':^.travis.yml' \ + ':^README' ':^MAINTAINERS' ':^VERSION' ':^ABI_VERSION' \ + ':^*/Kbuild' ':^*/README' \ + ':^license/' ':^config/' ':^buildtools/' \ + ':^*.cocci' ':^*.abignore' \ + ':^*.def' ':^*.map' ':^*.ini' ':^*.data' ':^*.cfg' ':^*.txt' \ + ':^*.svg' ':^*.png'\
I don't agree with this list of files.
But I guess we can start with that and be more strict in future.
Quoted 21 lines
devtools/check-spdx-tag.sh new file +21
-0+ > $tmpfile + + errors=$(wc -l < $tmpfile) + $quiet || cat $tmpfile +} + +check_boilerplate() { + if $verbose ; then + echo + echo "Files with redundant license text" + echo "---------------------------------" + fi + + git grep -l Redistribution -- \ + ':^license/' ':^/devtools/check-spdx-tag.sh' > $tmpfile + + warnings=$(wc -l <$tmpfile) + $quiet || cat $tmpfile +} + +while getopts qvh ARG ; do
devtools/check-spdx-tag.sh line 60
Quoted 10 lines
devtools/check-spdx-tag.sh new file +10
-0+ case $ARG in + q ) quiet=true ;; + v ) verbose=true ;; + h ) print_usage ; exit 0 ;; + ? ) print_usage ; exit 1 ;; + esac +done +shift $(($OPTIND - 1)) + +tmpfile=$(mktemp)
Should be mktemp -t dpdk.checkspdx.XXXXXX
to keep namespace of our temp files. Will fix.
Quoted 1 line
+trap 'rm -f -- "$tmpfile"' INT TERM HUP EXIT
Why catching HUP signal?
Applied, thanks
Quoted 74 lines · On Fri, 31 Jul 2020 00:06:23 +0200 Thomas Monjalon <[email protected]> wrote:
15/07/2020 01:21, Stephen Hemminger:
Quoted 4 lines
Simple script to look for drivers and scripts that are missing requires SPDX header.
Signed-off-by Stephen Hemminger <[email protected]>[...]
Quoted 19 lines
devtools/check-spdx-tag.sh new file +19
-0+#! /bin/sh +# SPDX-License-Identifier: BSD-3-Clause +# Copyright 2020 Microsoft Corporation +# +# Produce a list of files with incorrect license tags + +errors=0 +warnings=0 +quiet=false +verbose=false + +print_usage () { + echo "usage: $(basename $0) [-q] [-v]" + exit 1 +} + +check_spdx() { + if $verbose; then + echo "Files without SPDX License"devtools/check-spdx-tag.sh line 29
Quoted 10 lines
devtools/check-spdx-tag.sh new file +10
-0+ echo "--------------------------" + fi + git grep -L SPDX-License-Identifier -- \ + ':^.git*' ':^.ci/*' ':^.travis.yml' \ + ':^README' ':^MAINTAINERS' ':^VERSION' ':^ABI_VERSION' \ + ':^*/Kbuild' ':^*/README' \ + ':^license/' ':^config/' ':^buildtools/' \ + ':^*.cocci' ':^*.abignore' \ + ':^*.def' ':^*.map' ':^*.ini' ':^*.data' ':^*.cfg' ':^*.txt' \ + ':^*.svg' ':^*.png'\I don't agree with this list of files.
But I guess we can start with that and be more strict in future.Quoted 21 lines
devtools/check-spdx-tag.sh new file +21
-0+ > $tmpfile + + errors=$(wc -l < $tmpfile) + $quiet || cat $tmpfile +} + +check_boilerplate() { + if $verbose ; then + echo + echo "Files with redundant license text" + echo "---------------------------------" + fi + + git grep -l Redistribution -- \ + ':^license/' ':^/devtools/check-spdx-tag.sh' > $tmpfile + + warnings=$(wc -l <$tmpfile) + $quiet || cat $tmpfile +} + +while getopts qvh ARG ; dodevtools/check-spdx-tag.sh line 60
Quoted 10 lines
devtools/check-spdx-tag.sh new file +10
-0+ case $ARG in + q ) quiet=true ;; + v ) verbose=true ;; + h ) print_usage ; exit 0 ;; + ? ) print_usage ; exit 1 ;; + esac +done +shift $(($OPTIND - 1)) + +tmpfile=$(mktemp)Should be mktemp -t dpdk.checkspdx.XXXXXX
to keep namespace of our temp files. Will fix.
Quoted 3 lines
Quoted 1 line
+trap 'rm -f -- "$tmpfile"' INT TERM HUP EXIT
Why catching HUP signal?
General practice to have a script cleanup if user logs out. Back in the old days, connections were lost sometimes :-)_
Quoted 30 lines · On Fri, 31 Jul 2020 00:06:23 +0200 Thomas Monjalon <[email protected]> wrote:
15/07/2020 01:21, Stephen Hemminger:
Quoted 4 lines
Simple script to look for drivers and scripts that are missing requires SPDX header.
Signed-off-by Stephen Hemminger <[email protected]>[...]
Quoted 24 lines
devtools/check-spdx-tag.sh new file +24
-0+#! /bin/sh +# SPDX-License-Identifier: BSD-3-Clause +# Copyright 2020 Microsoft Corporation +# +# Produce a list of files with incorrect license tags + +errors=0 +warnings=0 +quiet=false +verbose=false + +print_usage () { + echo "usage: $(basename $0) [-q] [-v]" + exit 1 +} + +check_spdx() { + if $verbose; then + echo "Files without SPDX License" + echo "--------------------------" + fi + git grep -L SPDX-License-Identifier -- \ + ':^.git*' ':^.ci/*' ':^.travis.yml' \ + ':^README' ':^MAINTAINERS' ':^VERSION' ':^ABI_VERSION' \
Quoted 8 lines
devtools/check-spdx-tag.sh line 29
Quoted 5 lines
devtools/check-spdx-tag.sh new file +5
-0+ ':^*/Kbuild' ':^*/README' \ + ':^license/' ':^config/' ':^buildtools/' \ + ':^*.cocci' ':^*.abignore' \ + ':^*.def' ':^*.map' ':^*.ini' ':^*.data' ':^*.cfg' ':^*.txt' \ + ':^*.svg' ':^*.png'\I don't agree with this list of files.
But I guess we can start with that and be more strict in future.
I suppose README, MAINTAINERS, .ini and .txt files could be removed from the exception list.
Quoted 42 lines · Stephen Hemminger
Quoted 30 lines · On Fri, 31 Jul 2020 00:06:23 +0200 Thomas Monjalon <[email protected]> wrote:
15/07/2020 01:21, Stephen Hemminger:
Quoted 4 lines
Simple script to look for drivers and scripts that are missing requires SPDX header.
Signed-off-by Stephen Hemminger <[email protected]>[...]
Quoted 24 lines
devtools/check-spdx-tag.sh new file +24
-0+#! /bin/sh +# SPDX-License-Identifier: BSD-3-Clause +# Copyright 2020 Microsoft Corporation +# +# Produce a list of files with incorrect license tags + +errors=0 +warnings=0 +quiet=false +verbose=false + +print_usage () { + echo "usage: $(basename $0) [-q] [-v]" + exit 1 +} + +check_spdx() { + if $verbose; then + echo "Files without SPDX License" + echo "--------------------------" + fi + git grep -L SPDX-License-Identifier -- \ + ':^.git*' ':^.ci/*' ':^.travis.yml' \ + ':^README' ':^MAINTAINERS' ':^VERSION' ':^ABI_VERSION' \Quoted 8 lines
devtools/check-spdx-tag.sh line 29
Quoted 5 lines
devtools/check-spdx-tag.sh new file +5
-0+ ':^*/Kbuild' ':^*/README' \ + ':^license/' ':^config/' ':^buildtools/' \ + ':^*.cocci' ':^*.abignore' \ + ':^*.def' ':^*.map' ':^*.ini' ':^*.data' ':^*.cfg' ':^*.txt' \ + ':^*.svg' ':^*.png'\I don't agree with this list of files.
But I guess we can start with that and be more strict in future.
Quoted 1 line
I suppose README,
MAINTAINERS, .ini and .txt files could be removed from the exception list.
We also want to scan the files in config and buildtools.