license.bbclass parsing changes rebased sans debug
v3 cover license.bbclass parsing changes rebased sans debug
Using python abstract syntax trees as a parser and an ast visitor class, I've begun the first steps to being able to put some sanity to license selection.
One note. License modifiers have traditionally been limited to the "or greater" modifier "+". I've added another modifier "with exceptions" for licenses that have no generic ie common-licenses/GPL-2,0-with-GCC-exception
The following changes since commit 62e3f0ff7b27d8beb21b4b226051c3db654ea4e4:
License Field Cleanup: Non-standard field names (2011-05-27 18:27:22 +0100)
are available in the git repository at:
git://git.pokylinux.org/poky-contrib eflanagan/common-licenses
http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=eflanagan/common-licenses
Beth Flanagan (1):
license.bbclass: Sane Parsing of licenses
meta/classes/license.bbclass | 152 +++++++++++++++++++++++++++++++----------- 1 files changed, 114 insertions(+), 38 deletions(-)
Richard Purdie
AppliedQuoted 9 lines · Elizabeth Flanagan
Using python abstract syntax trees as a parser and an ast visitor class, I've begun the first steps to being able to put some sanity to license selection.
One note. License modifiers have traditionally been limited to the "or greater" modifier "+". I've added another modifier "with exceptions" for licenses that have no generic ie common-licenses/GPL-2,0-with-GCC-exception
The following changes since commit
62e3f0ff7b27d8beb21b4b226051c3db654ea4e4:
Quoted 8 lines
License Field Cleanup: Non-standard field names (2011-05-27 18:27:22 +0100)are available in the git repository at:
git://git.pokylinux.org/poky-contribeflanagan/common-licenses
http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=eflanagan/common-licensesBeth Flanagan (1):
license.bbclass: Sane Parsing of licenses
Merged to master, thanks.
Richard
Op 28 mei 2011, om 00:13 heeft Elizabeth Flanagan het volgende geschreven:
Quoted 7 lines
Using python abstract syntax trees as a parser and an ast visitor class, I've begun the first steps to being able to put some sanity to license selection.
One note. License modifiers have traditionally been limited to the "or greater" modifier "+". I've added another modifier "with exceptions" for licenses that have no generic ie common-licenses/GPL-2,0-with-GCC-exception
I get this now:
WARNING: /OE/tentacle/sources/openembedded-core/meta/files/common-licenses
WARNING: We need to use a SPDXLICENSEMAP for GPLv2
Is that really a warning, should that be a note or even complete hidden from the console?
regards,
Koen
Op 28 mei 2011, om 00:13 heeft Elizabeth Flanagan het volgende geschreven:
Quoted 7 lines
Using python abstract syntax trees as a parser and an ast visitor class, I've begun the first steps to being able to put some sanity to license selection.
One note. License modifiers have traditionally been limited to the "or greater" modifier "+". I've added another modifier "with exceptions" for licenses that have no generic ie common-licenses/GPL-2,0-with-GCC-exception
When building shadow I get:
| WARNING: /home/koen/angstrom-core/sources/openembedded-core/meta/files/common-licenses | ERROR: Error executing a python function in /home/koen/angstrom-core/sources/openembedded-core/meta/recipes-extended/shadow/shadow_4.1.4.3.bb: | RuntimeError: maximum recursion depth exceeded while calling a Python object | | ERROR: The stack trace of python calls that resulted in this exception/failure was: | ERROR: File "do_populate_lic", line 121, in <module> | ERROR: | ERROR: File "do_populate_lic", line 118, in do_populate_lic | ERROR: | ERROR: File "/usr/lib64/python2.7/ast.py", line 243, in visit | ERROR: return visitor(node) | ERROR: | ERROR: File "do_populate_lic", line 13, in generic_visit
and a few hundred more lines, I had to change the scrollback buffer from 512 lines to unlimited to catch this :) I can send the full log on request.
Could you send me the log. That's odd and shouldn't have happened (the license is only two licenses...)
-b
Quoted 33 lines · Koen Kooi
Op 28 mei 2011, om 00:13 heeft Elizabeth Flanagan het volgende geschreven:
Quoted 7 lines
Using python abstract syntax trees as a parser and an ast visitor class, I've begun the first steps to being able to put some sanity to license selection.
One note. License modifiers have traditionally been limited to the "or greater" modifier "+". I've added another modifier "with exceptions" for licenses that have no generic ie common-licenses/GPL-2,0-with-GCC-exception
When building shadow I get:
| WARNING: /home/koen/angstrom-core/sources/openembedded-core/meta/files/common-licenses | ERROR: Error executing a python function in /home/koen/angstrom-core/sources/openembedded-core/meta/recipes-extended/shadow/shadow_4.1.4.3.bb: | RuntimeError: maximum recursion depth exceeded while calling a Python object | | ERROR: The stack trace of python calls that resulted in this exception/failure was: | ERROR: File "do_populate_lic", line 121, in<module> | ERROR: | ERROR: File "do_populate_lic", line 118, in do_populate_lic | ERROR: | ERROR: File "/usr/lib64/python2.7/ast.py", line 243, in visit | ERROR: return visitor(node) | ERROR: | ERROR: File "do_populate_lic", line 13, in generic_visitand a few hundred more lines, I had to change the scrollback buffer from 512 lines to unlimited to catch this :) I can send the full log on request.
_______________________________________________
Openembedded-core mailing list
[email protected]
http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
Signature · 4 lines
--------------- Elizabeth Flanagan Yocto Project Release Engineer
Here you go :)
Op 28 mei 2011, om 15:21 heeft Elizabeth Flanagan het volgende geschreven:
Quoted 44 lines
Could you send me the log. That's odd and shouldn't have happened (the license is only two licenses...)
-b
Quoted 33 lines · Koen Kooi
Op 28 mei 2011, om 00:13 heeft Elizabeth Flanagan het volgende geschreven:
Quoted 7 lines
Using python abstract syntax trees as a parser and an ast visitor class, I've begun the first steps to being able to put some sanity to license selection.
One note. License modifiers have traditionally been limited to the "or greater" modifier "+". I've added another modifier "with exceptions" for licenses that have no generic ie common-licenses/GPL-2,0-with-GCC-exception
When building shadow I get:
| WARNING: /home/koen/angstrom-core/sources/openembedded-core/meta/files/common-licenses | ERROR: Error executing a python function in /home/koen/angstrom-core/sources/openembedded-core/meta/recipes-extended/shadow/shadow_4.1.4.3.bb: | RuntimeError: maximum recursion depth exceeded while calling a Python object | | ERROR: The stack trace of python calls that resulted in this exception/failure was: | ERROR: File "do_populate_lic", line 121, in<module> | ERROR: | ERROR: File "do_populate_lic", line 118, in do_populate_lic | ERROR: | ERROR: File "/usr/lib64/python2.7/ast.py", line 243, in visit | ERROR: return visitor(node) | ERROR: | ERROR: File "do_populate_lic", line 13, in generic_visitand a few hundred more lines, I had to change the scrollback buffer from 512 lines to unlimited to catch this :) I can send the full log on request.
_______________________________________________
Openembedded-core mailing list
[email protected]
http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-coreSignature · 4 lines
--------------- Elizabeth Flanagan Yocto Project Release Engineer
Quoted 4 lines · Koen Kooi
Op 28 mei 2011, om 00:13 heeft Elizabeth Flanagan het volgende geschreven:
Quoted 1 line
Quoted 8 lines
Quoted 6 lines
Using python abstract syntax trees as a parser and an ast visitor class, I've begun the first steps to being able to put some sanity to license selection.
One note. License modifiers have traditionally been limited to the "or greater" modifier "+". I've added another modifier "with exceptions" for licenses that have no generic ie common-licenses/GPL-2,0-with-GCC-exception
When building shadow I get:
and when building elfutils I get
ERROR: (file: 'do_populate_lic', lineno: 116, function: do_populate_lic)
ERROR: Function 'do_populate_lic' failed
ERROR: Logfile of failure stored in: /home/kraj/work/slugos/build/tmp-slugos-eglibc/work/armv5te-oe-linux-gnueabi/elfutils-0.148-r2/temp/log.do_populate_lic.31784 Log data follows:
| WARNING: /home/kraj/work/slugos/openembedded-core/meta/files/common-licenses | ERROR: Error executing a python function in /home/kraj/work/slugos/openembedded-core/meta/recipes-devtools/elfutils/elfutils_0.148.bb: | File "<unknown>", line 1 | ( 'GPLv2' & 'Exception' ) | ^ | IndentationError: unexpected indent | | ERROR: The stack trace of python calls that resulted in this exception/failure was: | ERROR: File "do_populate_lic", line 121, in <module> | ERROR: | ERROR: File "do_populate_lic", line 116, in do_populate_lic | ERROR: | ERROR: File "/usr/lib/python2.7/ast.py", line 37, in parse | ERROR: return compile(source, filename, mode, PyCF_ONLY_AST) | ERROR: | ERROR: The code that was being executed was: | ERROR: 0117: v = LicenseVisitor() | ERROR: 0118: v.visit(node) | ERROR: 0119: | ERROR: 0120: | ERROR: *** 0121:do_populate_lic(d) | ERROR: 0122: | ERROR: (file: 'do_populate_lic', lineno: 121, function: <module>) | ERROR: 0112: clean_licenses += "'" + x + "'" | ERROR: 0113: else: | ERROR: 0114: clean_licenses += " " + x + " " | ERROR: 0115: | ERROR: *** 0116: node = ast.parse(clean_licenses) | ERROR: 0117: v = LicenseVisitor() | ERROR: 0118: v.visit(node) | ERROR: 0119: | ERROR: 0120: | ERROR: (file: 'do_populate_lic', lineno: 116, function: do_populate_lic) | ERROR: Function 'do_populate_lic' failed
NOTE: package elfutils-0.148-r2: task do_populate_lic: Failed
Quoted 20 lines
| WARNING: /home/koen/angstrom-core/sources/openembedded-core/meta/files/common-licenses | ERROR: Error executing a python function in /home/koen/angstrom-core/sources/openembedded-core/meta/recipes-extended/shadow/shadow_4.1.4.3.bb: | RuntimeError: maximum recursion depth exceeded while calling a Python object | | ERROR: The stack trace of python calls that resulted in this exception/failure was: | ERROR: File "do_populate_lic", line 121, in<module> | ERROR: | ERROR: File "do_populate_lic", line 118, in do_populate_lic | ERROR: | ERROR: File "/usr/lib64/python2.7/ast.py", line 243, in visit | ERROR: return visitor(node) | ERROR: | ERROR: File "do_populate_lic", line 13, in generic_visitand a few hundred more lines, I had to change the scrollback buffer from 512 lines to unlimited to catch this :) I can send the full log on request.
_______________________________________________
Openembedded-core mailing list
[email protected]
http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
In this case, elfutils should probably be changes as well as license.bbclass. As do_populate_lic is looking for valid python to make a syntax tree from, this will fail. Solution:
1. Fix license.bbclass to check LICENSE and massage it a bit more.
2. Fix elfutils to be GPLv2* (unless it's the GCC exception. I haven't read the license yet).
I'll poke around this today.
-b
Quoted 89 lines · Khem Raj
Quoted 4 lines · Koen Kooi
Op 28 mei 2011, om 00:13 heeft Elizabeth Flanagan het volgende geschreven:
Quoted 1 line
Quoted 8 lines
Quoted 6 lines
Using python abstract syntax trees as a parser and an ast visitor class, I've begun the first steps to being able to put some sanity to license selection.
One note. License modifiers have traditionally been limited to the "or greater" modifier "+". I've added another modifier "with exceptions" for licenses that have no generic ie common-licenses/GPL-2,0-with-GCC-exception
When building shadow I get:
and when building elfutils I get
ERROR: (file: '
do_populate_lic', lineno: 116, function:do_populate_lic)
ERROR: Function 'do_populate_lic' failed
ERROR: Logfile of failure stored in:/home/kraj/work/slugos/build/tmp-slugos-eglibc/work/armv5te-oe-linux-gnueabi/elfutils-0.148-r2/temp/log.do_populate_lic.31784Log data follows:
| WARNING:/home/kraj/work/slugos/openembedded-core/meta/files/common-licenses
| ERROR: Error executing a python function in/home/kraj/work/slugos/openembedded-core/meta/recipes-devtools/elfutils/elfutils_0.148.bb:| File "<unknown>", line 1 | ( 'GPLv2'& 'Exception' ) | ^ | IndentationError: unexpected indent | | ERROR: The stack trace of python calls that resulted in this exception/failure was: | ERROR: File "do_populate_lic", line 121, in<module> | ERROR: | ERROR: File "do_populate_lic", line 116, in do_populate_lic | ERROR: | ERROR: File "/usr/lib/python2.7/ast.py", line 37, in parse | ERROR: return compile(source, filename, mode, PyCF_ONLY_AST) | ERROR: | ERROR: The code that was being executed was: | ERROR: 0117: v = LicenseVisitor() | ERROR: 0118: v.visit(node) | ERROR: 0119: | ERROR: 0120: | ERROR: *** 0121:do_populate_lic(d) | ERROR: 0122: | ERROR: (file: 'do_populate_lic', lineno: 121, function:<module>) | ERROR: 0112: clean_licenses += "'" + x + "'" | ERROR: 0113: else: | ERROR: 0114: clean_licenses += " " + x + " " | ERROR: 0115: | ERROR: *** 0116: node = ast.parse(clean_licenses) | ERROR: 0117: v = LicenseVisitor() | ERROR: 0118: v.visit(node) | ERROR: 0119: | ERROR: 0120: | ERROR: (file: 'do_populate_lic', lineno: 116, function: do_populate_lic) | ERROR: Function 'do_populate_lic' failedNOTE: package elfutils-0.148-r2: task
do_populate_lic: FailedQuoted 15 lines
| WARNING: /home/koen/angstrom-core/sources/openembedded-core/meta/files/common-licenses | ERROR: Error executing a python function in /home/koen/angstrom-core/sources/openembedded-core/meta/recipes-extended/shadow/shadow_4.1.4.3.bb: | RuntimeError: maximum recursion depth exceeded while calling a Python object | | ERROR: The stack trace of python calls that resulted in this exception/failure was: | ERROR: File "do_populate_lic", line 121, in<module> | ERROR: | ERROR: File "do_populate_lic", line 118, in do_populate_lic | ERROR: | ERROR: File "/usr/lib64/python2.7/ast.py", line 243, in visit | ERROR: return visitor(node) | ERROR: | ERROR: File "do_populate_lic", line 13, in generic_visitQuoted 5 lines
and a few hundred more lines, I had to change the scrollback buffer from 512 lines to unlimited to catch this :) I can send the full log on request.
_______________________________________________
Openembedded-core mailing list
[email protected]
http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core_______________________________________________
Openembedded-core mailing list
[email protected]
http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
Signature · 4 lines
--------------- Elizabeth Flanagan Yocto Project Release Engineer
Quoted 7 lines · Elizabeth Flanagan
In this case, elfutils should probably be changes as well as license.bbclass. As
do_populate_licis looking for valid python to make a syntax tree from, this will fail. Solution:1. Fix license.bbclass to check LICENSE and massage it a bit more.
2. Fix elfutils to be GPLv2* (unless it's the GCC exception. I haven't read the license yet).
It does not like () around license values so simple solution is to replace
-LICENSE = "(GPLv2 & Exception)"
+LICENSE = "GPLv2 & Exception"
Generally its preferable to fix eflutils first before license class in this case to keep the tree bisect'able.
Quoted 112 lines
I'll poke around this today.
-b
Quoted 105 lines · Khem Raj
Quoted 10 lines · Koen Kooi
Op 28 mei 2011, om 00:13 heeft Elizabeth Flanagan het volgende geschreven:
Quoted 6 lines
Using python abstract syntax trees as a parser and an ast visitor class, I've
begun the first steps to being able to put some sanity to license selection.Quoted 7 lines
Quoted 5 lines
One note. License modifiers have traditionally been limited to the "or greater"
modifier "+". I've added another modifier "with exceptions" for licenses that
have no generic ie common-licenses/GPL-2,0-with-GCC-exceptionWhen building shadow I get:
and when building elfutils I get
ERROR: (file: '
do_populate_lic', lineno: 116, function:do_populate_lic)
ERROR: Function 'do_populate_lic' failed
ERROR: Logfile of failure stored in:/home/kraj/work/slugos/build/tmp-slugos-eglibc/work/armv5te-oe-linux-gnueabi/elfutils-0.148-r2/temp/log.do_populate_lic.31784Log data follows:
| WARNING:/home/kraj/work/slugos/openembedded-core/meta/files/common-licenses
| ERROR: Error executing a python function in/home/kraj/work/slugos/openembedded-core/meta/recipes-devtools/elfutils/elfutils_0.148.bb:| File "<unknown>", line 1 | ( 'GPLv2'& 'Exception' ) | ^ | IndentationError: unexpected indent | | ERROR: The stack trace of python calls that resulted in this exception/failure was: | ERROR: File "do_populate_lic", line 121, in<module> | ERROR: | ERROR: File "do_populate_lic", line 116, in do_populate_lic | ERROR: | ERROR: File "/usr/lib/python2.7/ast.py", line 37, in parse | ERROR: return compile(source, filename, mode, PyCF_ONLY_AST) | ERROR: | ERROR: The code that was being executed was: | ERROR: 0117: v = LicenseVisitor() | ERROR: 0118: v.visit(node) | ERROR: 0119: | ERROR: 0120: | ERROR: *** 0121:do_populate_lic(d) | ERROR: 0122: | ERROR: (file: 'do_populate_lic', lineno: 121, function:<module>) | ERROR: 0112: clean_licenses += "'" + x + "'" | ERROR: 0113: else: | ERROR: 0114: clean_licenses += " " + x + " " | ERROR: 0115: | ERROR: *** 0116: node = ast.parse(clean_licenses) | ERROR: 0117: v = LicenseVisitor() | ERROR: 0118: v.visit(node) | ERROR: 0119: | ERROR: 0120: | ERROR: (file: 'do_populate_lic', lineno: 116, function:
do_populate_lic)
| ERROR: Function 'do_populate_lic' failed
NOTE: package elfutils-0.148-r2: taskdo_populate_lic: FailedQuoted 21 lines
| WARNING:
/home/koen/angstrom-core/sources/openembedded-core/meta/files/common-licenses| ERROR: Error executing a python function in
/home/koen/angstrom-core/sources/openembedded-core/meta/recipes-extended/shadow/shadow_4.1.4.3.bb:| RuntimeError: maximum recursion depth exceeded while calling a Python object
|
| ERROR: The stack trace of python calls that resulted in this exception/failure was:| ERROR: File "do_populate_lic", line 121, in<module> | ERROR: | ERROR: File "do_populate_lic", line 118, in do_populate_lic | ERROR: | ERROR: File "/usr/lib64/python2.7/ast.py", line 243, in visit | ERROR: return visitor(node) | ERROR: | ERROR: File "do_populate_lic", line 13, in generic_visitQuoted 7 lines
and a few hundred more lines, I had to change the scrollback buffer from 512 lines to unlimited to catch this :) I can send the full log on request.
_______________________________________________
Openembedded-core mailing list [email protected]
http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core_______________________________________________
Openembedded-core mailing list
[email protected]
http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
Quoted 14 lines · Khem Raj
Quoted 7 lines · Elizabeth Flanagan
In this case, elfutils should probably be changes as well as license.bbclass. As
do_populate_licis looking for valid python to make a syntax tree from, this will fail. Solution:1. Fix license.bbclass to check LICENSE and massage it a bit more.
2. Fix elfutils to be GPLv2* (unless it's the GCC exception. I haven't read the license yet).It does not like () around license values so simple solution is to replace
(quoted diff) +1
-1-LICENSE = "(GPLv2& Exception)" +LICENSE = "GPLv2& Exception"
Quoted 3 lines
Generally its preferable to fix eflutils first before license class in this case to keep the tree bisect'able.
Actually, the issue is my .replace("(", " ( "). It makes the string " ( 'GPLv2' & 'Exception' ) " which tosses an error because of the indented string. I have a fix I'm putting a pull request in (in about 10 minutes) for both this and the issue Koen found.
-b
---------------
Elizabeth Flanagan
Yocto Project
Release Engineer
On Fri, May 27, 2011 at 22:13, Elizabeth Flanagan
<[email protected]> wrote:
...
Quoted 2 lines
Beth Flanagan (1):
license.bbclass: Sane Parsing of licenses
...
The change itself was OK and really an improvement but please pay some attention to whitespaces. It is nice to avoid useless whitespaces into new code.
Signature · 3 lines
Otavio Salvador O.S. Systems E-mail: [email protected] http://www.ossystems.com.br Mobile: +55 53 9981-7854 http://projetos.ossystems.com.br