bitbake: event: Inject empty lines to make code match lineno in filename
v3 cover bitbake: event: Inject empty lines to make code match lineno in filename
From: Robert Yang <[email protected]>
* V3: Inject empty lines to make code match lineno in filename as suggested by RP.
// Robert
The following changes since commit 6ce42d73a31bf9b2fc362fa4acad11de97ac96ec:
scripts: Drop shell sstate-cache-management (2023-12-28 10:57:49 +0000)
are available in the Git repository at:
https://github.com/robertlinux/yocto rbt/event
https://github.com/robertlinux/yocto/tree/rbt/event
Robert Yang (1):
bitbake: event: Inject empty lines to make code match lineno in
filename
bitbake/lib/bb/event.py | 2 ++ 1 file changed, 2 insertions(+)
Signature · 1 line
2.42.0
PATCH v3 1/1 bitbake: event: Inject empty lines to make code match lineno in filename
From: Robert Yang <[email protected]>
So that we can get the correct error messages.
* In python 3.10.9, the error message was:
ERROR: Unable to register event handler 'defaultbase_eventhandler':
File "/path/to/poky/meta/classes-global/base.bbclass", line 4
# SPDX-License-Identifier: MIT
^^^^^
SyntaxError: invalid syntax
This is hard to debug since the error line number 4 is incorrect, but nothing is wrong with the code in line 4.
* Now the error message and lineno is correct:
ERROR: Unable to register event handler 'defaultbase_eventhandler':
File "/path/to/poky/meta/classes-global/base.bbclass", line 256
an error line
^^^^^
SyntaxError: invalid syntax
And no impact on parsing time:
* Before:
$ rm -fr cache tmp; time bitbake -p real 0m27.254s
* Now:
$ rm -fr cache tmp; time bitbake -p real 0m27.200s
--- bitbake/lib/bb/event.py | 2 ++ 1 file changed, 2 insertions(+)
bitbake/lib/bb/event.py +2 -0
@@ -257,6 +257,8 @@ def register(name, handler, mask=None, filename=None, lineno=None, data=None): # handle string containing python code if isinstance(handler, str): tmp = "def %s(e, d):\n%s" % (name, handler) + # Inject empty lines to make code match lineno in filename + tmp = "\n" * (lineno-1) + tmp try: code = bb.methodpool.compile_cache(tmp) if not code:
Signature · 1 line
2.42.0
Quoted 40 lines · Robert Yang via lists.openembedded.org
From: Robert Yang <[email protected]>
So that we can get the correct error messages.
* In python 3.10.9, the error message was:
ERROR: Unable to register event handler 'defaultbase_eventhandler':
File "/path/to/poky/meta/classes-global/base.bbclass", line 4
# SPDX-License-Identifier: MIT
^^^^^
SyntaxError: invalid syntaxThis is hard to debug since the error line number 4 is incorrect, but nothing is wrong with the code in line 4.
* Now the error message and lineno is correct:
ERROR: Unable to register event handler 'defaultbase_eventhandler':
File "/path/to/poky/meta/classes-global/base.bbclass", line 256
an error line
^^^^^
SyntaxError: invalid syntaxAnd no impact on parsing time:
* Before:$ rm -fr cache tmp; time bitbake -p real 0m27.254s* Now:
$ rm -fr cache tmp; time bitbake -p real 0m27.200sSigned-off-by Robert Yang <[email protected]>--- bitbake/lib/bb/event.py | 2 ++ 1 file changed, 2 insertions(+)bitbake/lib/bb/event.py +0
-0
bitbake/lib/bb/event.py def register(name, handler, mask=None, filename=None, lineno=None, data=None): · line 264
Quoted 8 lines
bitbake/lib/bb/event.py +2
-0@@ -257,6 +257,8 @@ def register(name, handler, mask=None, filename=None, lineno=None, data=None): # handle string containing python code if isinstance(handler, str): tmp = "def %s(e, d):\n%s" % (name, handler) + # Inject empty lines to make code match lineno in filename + tmp = "\n" * (lineno-1) + tmp try: code = bb.methodpool.compile_cache(tmp) if not code:
Looks good but don't you need remove the ast line number adjustment a few lines later as well?
Should we put some tests into bitbake-selftest for this?
Cheers,
Richard
Quoted 37 lines · Richard Purdie
Quoted 35 lines · Robert Yang via lists.openembedded.org
From: Robert Yang <[email protected]>
So that we can get the correct error messages.
* In python 3.10.9, the error message was:
ERROR: Unable to register event handler 'defaultbase_eventhandler':
File "/path/to/poky/meta/classes-global/base.bbclass", line 4
# SPDX-License-Identifier: MIT
^^^^^
SyntaxError: invalid syntaxThis is hard to debug since the error line number 4 is incorrect, but nothing is wrong with the code in line 4.
* Now the error message and lineno is correct:
ERROR: Unable to register event handler 'defaultbase_eventhandler':
File "/path/to/poky/meta/classes-global/base.bbclass", line 256
an error line
^^^^^
SyntaxError: invalid syntaxAnd no impact on parsing time:
* Before:$ rm -fr cache tmp; time bitbake -p real 0m27.254s* Now:
$ rm -fr cache tmp; time bitbake -p real 0m27.200sSigned-off-by Robert Yang <[email protected]>--- bitbake/lib/bb/event.py | 2 ++ 1 file changed, 2 insertions(+)
Quoted 16 lines
bitbake/lib/bb/event.py def register(name, handler, mask=None, filename=None, lineno=None, data=None): · line 264
Quoted 13 lines
bitbake/lib/bb/event.py +2
-0@@ -257,6 +257,8 @@ def register(name, handler, mask=None, filename=None, lineno=None, data=None): # handle string containing python code if isinstance(handler, str): tmp = "def %s(e, d):\n%s" % (name, handler) + # Inject empty lines to make code match lineno in filename + tmp = "\n" * (lineno-1) + tmp try: code = bb.methodpool.compile_cache(tmp) if not code:Looks good but don't you need remove the ast line number adjustment a few lines later as well?
Yes, you're right, the code has incremented the lineno by '\n', so the following line isn't needed:
ast.increment_lineno(code, lineno-1)
Now the patch is: (Check lineno is not None to fix bitbake-selftest error)
bitbake/lib/bb/event.py +3 -2
@@ -257,14 +257,15 @@ def register(name, handler, mask=None, filename=None, lineno=None, data=None): # handle string containing python code if isinstance(handler, str): tmp = "def %s(e, d):\n%s" % (name, handler) + # Inject empty lines to make code match lineno in filename + if lineno is not None: + tmp = "\n" * (lineno-1) + tmp try: code = bb.methodpool.compile_cache(tmp) if not code: if filename is None: filename = "%s(e, d)" % name code = compile(tmp, filename, "exec", ast.PyCF_ONLY_AST) - if lineno is not None: - ast.increment_lineno(code, lineno-1) code = compile(code, filename, "exec") bb.methodpool.compile_cache_add(tmp, code) except SyntaxError:
I will send a V4 add with a testcase for it.
// Robert
Quoted 6 lines
Should we put some tests into bitbake-selftest for this?
Cheers,
Richard
This causes bitbake-selftest failures:
======================================================================
ERROR: test_register_from_string (bb.tests.event.EventHandlingTest)
Test register method receiving code in string
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/pokybuild/yocto-worker/oe-selftest-debian/build/bitbake/lib/bb/tests/event.py", line 121, in test_register_from_string
result = bb.event.register("string_handler", " return True")
File "/home/pokybuild/yocto-worker/oe-selftest-debian/build/bitbake/lib/bb/event.py", line 261, in register
tmp = "\n" * (lineno-1) + tmp
TypeError: unsupported operand type(s) for -: 'NoneType' and 'int'
https://autobuilder.yoctoproject.org/typhoon/#/builders/80/builds/6202/steps/11/logs/stdio
Quoted 61 lines · Robert Yang via lists.openembedded.org
From: Robert Yang <[email protected]>
So that we can get the correct error messages.
* In python 3.10.9, the error message was:
ERROR: Unable to register event handler 'defaultbase_eventhandler':
File "/path/to/poky/meta/classes-global/base.bbclass", line 4
# SPDX-License-Identifier: MIT
^^^^^
SyntaxError: invalid syntaxThis is hard to debug since the error line number 4 is incorrect, but nothing is wrong with the code in line 4.
* Now the error message and lineno is correct:
ERROR: Unable to register event handler 'defaultbase_eventhandler':
File "/path/to/poky/meta/classes-global/base.bbclass", line 256
an error line
^^^^^
SyntaxError: invalid syntaxAnd no impact on parsing time:
* Before:$ rm -fr cache tmp; time bitbake -p real 0m27.254s* Now:
$ rm -fr cache tmp; time bitbake -p real 0m27.200sSigned-off-by Robert Yang <[email protected]>--- bitbake/lib/bb/event.py | 2 ++ 1 file changed, 2 insertions(+)bitbake/lib/bb/event.py +2
-0@@ -257,6 +257,8 @@ def register(name, handler, mask=None, filename=None, lineno=None, data=None): # handle string containing python code if isinstance(handler, str): tmp = "def %s(e, d):\n%s" % (name, handler) + # Inject empty lines to make code match lineno in filename + tmp = "\n" * (lineno-1) + tmp try: code = bb.methodpool.compile_cache(tmp) if not code:Signature · 11 lines
2.42.0 -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#15717): https://lists.openembedded.org/g/bitbake-devel/message/15717 Mute This Topic: https://lists.openembedded.org/mt/103413642/3617179 Group Owner: [email protected] Unsubscribe: https://lists.openembedded.org/g/bitbake-devel/unsub [[email protected]] -=-=-=-=-=-=-=-=-=-=-=-
Signature · 3 lines
Alexandre Belloni, co-owner and COO, Bootlin Embedded Linux and Kernel engineering https://bootlin.com