classes: cve-check: Get shared database lock
PATCH 1/1 classes: cve-check: Get shared database lock
The CVE check database needs to have a shared lock acquired on it before it is accessed. This to prevent cve-update-db-native from deleting the database file out from underneath it.
[YOCTO #14899]
--- meta/classes/cve-check.bbclass | 23 ++++++++++++----------- 1 file changed, 12 insertions(+), 11 deletions(-)
meta/classes/cve-check.bbclass +12 -11
@@ -145,17 +145,18 @@ python do_cve_check () { """ from oe.cve_check import get_patched_cves - if os.path.exists(d.getVar("CVE_CHECK_DB_FILE")): - try: - patched_cves = get_patched_cves(d) - except FileNotFoundError: - bb.fatal("Failure in searching patches") - ignored, patched, unpatched, status = check_cves(d, patched_cves) - if patched or unpatched or (d.getVar("CVE_CHECK_COVERAGE") == "1" and status): - cve_data = get_cve_info(d, patched + unpatched + ignored) - cve_write_data(d, patched, unpatched, ignored, cve_data, status) - else: - bb.note("No CVE database found, skipping CVE check") + with bb.utils.fileslocked([d.getVar("CVE_CHECK_DB_FILE_LOCK")], shared=True): + if os.path.exists(d.getVar("CVE_CHECK_DB_FILE")): + try: + patched_cves = get_patched_cves(d) + except FileNotFoundError: + bb.fatal("Failure in searching patches") + ignored, patched, unpatched, status = check_cves(d, patched_cves) + if patched or unpatched or (d.getVar("CVE_CHECK_COVERAGE") == "1" and status): + cve_data = get_cve_info(d, patched + unpatched + ignored) + cve_write_data(d, patched, unpatched, ignored, cve_data, status) + else: + bb.note("No CVE database found, skipping CVE check") }
Signature · 1 line
2.33.0
meta/classes/cve-check.bbclass python do_cve_check () { · line 159
Quoted 13 lines · Joshua Watt
The CVE check database needs to have a shared lock acquired on it before it is accessed. This to prevent cve-update-db-native from deleting the database file out from underneath it.
[YOCTO #14899]
Signed-off-by Joshua Watt <[email protected]>meta/classes/cve-check.bbclass +4
-0+ cve_data = get_cve_info(d, patched + unpatched + ignored) + cve_write_data(d, patched, unpatched, ignored, cve_data, status) + else: + bb.note("No CVE database found, skipping CVE check")
With this commit in kirkstone-nut, we're getting an error with meta-zephyr builds:
RROR: zephyr-philosophers-3.1.0+gitAUTOINC+2ddd73feaf_5f86244bad-r0do_cve_check: Error executing a python function in exec_func_python()
autogenerated:
The stack trace of python calls that resulted in this exception/failure was:
File: 'exec_func_python() autogenerated', lineno: 2, function: <module>
0001:
*** 0002:do_cve_check(d)
0003:
File: '/tmp/workspace.4jc1Y12y3W/oe-core/meta/classes/cve-check.bbclass',
lineno: 142, function: do_cve_check
0138: Check recipe for patched and unpatched CVEs
0139: """
0140: from oe.cve_check import get_patched_cves
0141:
*** 0142: with bb.utils.fileslocked([d.getVar("CVE_CHECK_DB_FILE_LOCK")],
shared=True):
0143: if os.path.exists(d.getVar("CVE_CHECK_DB_FILE")):
0144: try:
0145: patched_cves = get_patched_cves(d)
0146: except FileNotFoundError:
File: '/usr/lib/python3.8/contextlib.py', lineno: 240, function: helper
0236: <cleanup>
0237: """
0238: @wraps(func)
0239: def helper(*args, **kwds):
*** 0240: return _GeneratorContextManager(func, args, kwds)
0241: return helper
0242:
0243:
0244:def asynccontextmanager(func):
File: '/usr/lib/python3.8/contextlib.py', lineno: 83, function: __init__
0079:class _GeneratorContextManagerBase:
0080: """Shared functionality for @contextmanager and @asynccontextmanager."""
0081:
0082: def __init__(self, func, args, kwds):
*** 0083: self.gen = func(*args, **kwds)
0084: self.func, self.args, self.kwds = func, args, kwds
0085: # Issue 19330: ensure context manager instances have good docstrings
0086: doc = getattr(func, "__doc__", None)
0087: if doc is None:
Exception: TypeError: fileslocked() got an unexpected keyword argument 'shared'
ERROR: Logfile of failure stored in:/tmp/workspace.4jc1Y12y3W/build/tmp-newlib/work/i586-yocto-elf/zephyr-philosophers/3.1.0+gitAUTOINC+2ddd73feaf_5f86244bad-r0/temp/log.do_cve_check.433603
NOTE: recipe zephyr-philosophers-3.1.0+gitAUTOINC+2ddd73feaf_5f86244bad-r0:
task do_cve_check: Failed
ERROR: Task (/tmp/workspace.4jc1Y12y3W/oe-core/../meta-zephyr/meta-zephyr-core/recipes-kernel/zephyr-kernel/zephyr-philosophers.bb:do_cve_check) failed with exit code '1'
Kind regards,
Marta
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 58 lines · Marta Rybczynska via lists.openembedded.org
meta/classes/cve-check.bbclass python do_cve_check () { · line 159
Quoted 13 lines · Joshua Watt
The CVE check database needs to have a shared lock acquired on it before it is accessed. This to prevent cve-update-db-native from deleting the database file out from underneath it.
[YOCTO #14899]
Signed-off-by Joshua Watt <[email protected]>meta/classes/cve-check.bbclass +4
-0+ cve_data = get_cve_info(d, patched + unpatched + ignored) + cve_write_data(d, patched, unpatched, ignored, cve_data, status) + else: + bb.note("No CVE database found, skipping CVE check")With this commit in kirkstone-nut, we're getting an error with meta-zephyr builds:
RROR: zephyr-philosophers-3.1.0+gitAUTOINC+2ddd73feaf_5f86244bad-r0
do_cve_check: Error executing a python function inexec_func_python()autogenerated:
The stack trace of python calls that resulted in this exception/failure was:
File: 'exec_func_python()autogenerated', lineno: 2, function: <module> 0001:
*** 0002:do_cve_check(d)
0003:
File: '/tmp/workspace.4jc1Y12y3W/oe-core/meta/classes/cve-check.bbclass',
lineno: 142, function:do_cve_check
0138: Check recipe for patched and unpatched CVEs
0139: """
0140: from oe.cve_check importget_patched_cves
0141:
*** 0142: withbb.utils.fileslocked([d.getVar("CVE_CHECK_DB_FILE_LOCK")], shared=True):
0143: if os.path.exists(d.getVar("CVE_CHECK_DB_FILE")):
0144: try:
0145:patched_cves=get_patched_cves(d)
0146: except FileNotFoundError:
File: '/usr/lib/python3.8/contextlib.py', lineno: 240, function: helper 0236: <cleanup>
0237: """
0238: @wraps(func)
0239: defhelper(*args, **kwds):
*** 0240: return_GeneratorContextManager(func, args, kwds)
0241: return helper
0242:
0243:
0244:def asynccontextmanager(func):
File: '/usr/lib/python3.8/contextlib.py', lineno: 83, function:__init__0079:class_GeneratorContextManagerBase:
0080: """Shared functionality for @contextmanager and @asynccontextmanager.""" 0081:
0082: def__init__(self, func, args, kwds):
*** 0083: self.gen =func(*args, **kwds)
0084: self.func, self.args, self.kwds = func, args, kwds
0085: # Issue 19330: ensure context manager instances have good docstrings 0086: doc =getattr(func, "__doc__", None)
Quoted 9 lines
0087: if doc is None:
Exception: TypeError:fileslocked()got an unexpected keyword argument 'shared'
ERROR: Logfile of failure stored in:/tmp/workspace.4jc1Y12y3W/build/tmp-newlib/work/i586-yocto-elf/zephyr-philosophers/3.1.0+gitAUTOINC+2ddd73feaf_5f86244bad-r0/temp/log.do_cve_check.433603
NOTE: recipe zephyr-philosophers-3.1.0+gitAUTOINC+2ddd73feaf_5f86244bad-r0:
taskdo_cve_check: Failed
ERROR: Task (/tmp/workspace.4jc1Y12y3W/oe-core/../meta-zephyr/meta-zephyr-core/recipes-kernel/zephyr-kernel/zephyr-philosophers.bb:do_cve_check) failed with exit code '1'
This is a mismatch with bitbake, because one commit hasn't reached 2.0. Steve, should I be using
https://git.openembedded.org/bitbake-contrib/log/?h=stable/2.0-nut for testing?
Kind regards,
Marta
Quoted 71 lines · Marta Rybczynska
Quoted 58 lines · Marta Rybczynska via lists.openembedded.org
meta/classes/cve-check.bbclass python do_cve_check () { · line 159
Quoted 13 lines · Joshua Watt
The CVE check database needs to have a shared lock acquired on it before it is accessed. This to prevent cve-update-db-native from deleting the database file out from underneath it.
[YOCTO #14899]
Signed-off-by Joshua Watt <[email protected]>meta/classes/cve-check.bbclass +4
-0+ cve_data = get_cve_info(d, patched + unpatched + ignored) + cve_write_data(d, patched, unpatched, ignored, cve_data, status) + else: + bb.note("No CVE database found, skipping CVE check")With this commit in kirkstone-nut, we're getting an error with meta-zephyr builds:
RROR: zephyr-philosophers-3.1.0+gitAUTOINC+2ddd73feaf_5f86244bad-r0
do_cve_check: Error executing a python function inexec_func_python()autogenerated:
The stack trace of python calls that resulted in this exception/failure was:
File: 'exec_func_python()autogenerated', lineno: 2, function: <module> 0001:
*** 0002:do_cve_check(d)
0003:
File: '/tmp/workspace.4jc1Y12y3W/oe-core/meta/classes/cve-check.bbclass',
lineno: 142, function:do_cve_check
0138: Check recipe for patched and unpatched CVEs
0139: """
0140: from oe.cve_check importget_patched_cves
0141:
*** 0142: withbb.utils.fileslocked([d.getVar("CVE_CHECK_DB_FILE_LOCK")], shared=True):
0143: if os.path.exists(d.getVar("CVE_CHECK_DB_FILE")):
0144: try:
0145:patched_cves=get_patched_cves(d)
0146: except FileNotFoundError:
File: '/usr/lib/python3.8/contextlib.py', lineno: 240, function: helper 0236: <cleanup>
0237: """
0238: @wraps(func)
0239: defhelper(*args, **kwds):
*** 0240: return_GeneratorContextManager(func, args, kwds)
0241: return helper
0242:
0243:
0244:def asynccontextmanager(func):
File: '/usr/lib/python3.8/contextlib.py', lineno: 83, function:__init__0079:class_GeneratorContextManagerBase:
0080: """Shared functionality for @contextmanager and @asynccontextmanager.""" 0081:
0082: def__init__(self, func, args, kwds):
*** 0083: self.gen =func(*args, **kwds)
0084: self.func, self.args, self.kwds = func, args, kwds
0085: # Issue 19330: ensure context manager instances have good docstrings 0086: doc =getattr(func, "__doc__", None)Quoted 9 lines
0087: if doc is None:
Exception: TypeError:fileslocked()got an unexpected keyword argument 'shared'
ERROR: Logfile of failure stored in:/tmp/workspace.4jc1Y12y3W/build/tmp-newlib/work/i586-yocto-elf/zephyr-philosophers/3.1.0+gitAUTOINC+2ddd73feaf_5f86244bad-r0/temp/log.do_cve_check.433603
NOTE: recipe zephyr-philosophers-3.1.0+gitAUTOINC+2ddd73feaf_5f86244bad-r0:
taskdo_cve_check: Failed
ERROR: Task (/tmp/workspace.4jc1Y12y3W/oe-core/../meta-zephyr/meta-zephyr-core/recipes-kernel/zephyr-kernel/zephyr-philosophers.bb:do_cve_check) failed with exit code '1'
Quoted 4 lines
This is a mismatch with bitbake, because one commit hasn't reached 2.0. Steve, should I be using
https://git.openembedded.org/bitbake-contrib/log/?h=stable/2.0-nut for testing?
I've gone ahead and backported that bitbake change to make things easier for people.
Cheers,
Richard
Quoted 72 lines · Marta Rybczynska
Quoted 58 lines · Marta Rybczynska via lists.openembedded.org
meta/classes/cve-check.bbclass python do_cve_check () { · line 159
Quoted 13 lines · Joshua Watt
The CVE check database needs to have a shared lock acquired on it before it is accessed. This to prevent cve-update-db-native from deleting the database file out from underneath it.
[YOCTO #14899]
Signed-off-by Joshua Watt <[email protected]>meta/classes/cve-check.bbclass +4
-0+ cve_data = get_cve_info(d, patched + unpatched + ignored) + cve_write_data(d, patched, unpatched, ignored, cve_data, status) + else: + bb.note("No CVE database found, skipping CVE check")With this commit in kirkstone-nut, we're getting an error with meta-zephyr builds:
RROR: zephyr-philosophers-3.1.0+gitAUTOINC+2ddd73feaf_5f86244bad-r0
do_cve_check: Error executing a python function inexec_func_python()autogenerated:
The stack trace of python calls that resulted in this exception/failure was:
File: 'exec_func_python()autogenerated', lineno: 2, function: <module> 0001:
*** 0002:do_cve_check(d)
0003:
File: '/tmp/workspace.4jc1Y12y3W/oe-core/meta/classes/cve-check.bbclass',
lineno: 142, function:do_cve_check
0138: Check recipe for patched and unpatched CVEs
0139: """
0140: from oe.cve_check importget_patched_cves
0141:
*** 0142: withbb.utils.fileslocked([d.getVar("CVE_CHECK_DB_FILE_LOCK")], shared=True):
0143: if os.path.exists(d.getVar("CVE_CHECK_DB_FILE")):
0144: try:
0145:patched_cves=get_patched_cves(d)
0146: except FileNotFoundError:
File: '/usr/lib/python3.8/contextlib.py', lineno: 240, function: helper 0236: <cleanup>
0237: """
0238: @wraps(func)
0239: defhelper(*args, **kwds):
*** 0240: return_GeneratorContextManager(func, args, kwds)
0241: return helper
0242:
0243:
0244:def asynccontextmanager(func):
File: '/usr/lib/python3.8/contextlib.py', lineno: 83, function:__init__0079:class_GeneratorContextManagerBase:
0080: """Shared functionality for @contextmanager and @asynccontextmanager.""" 0081:
0082: def__init__(self, func, args, kwds):
*** 0083: self.gen =func(*args, **kwds)
0084: self.func, self.args, self.kwds = func, args, kwds
0085: # Issue 19330: ensure context manager instances have good docstrings 0086: doc =getattr(func, "__doc__", None)Quoted 9 lines
0087: if doc is None:
Exception: TypeError:fileslocked()got an unexpected keyword argument 'shared'
ERROR: Logfile of failure stored in:/tmp/workspace.4jc1Y12y3W/build/tmp-newlib/work/i586-yocto-elf/zephyr-philosophers/3.1.0+gitAUTOINC+2ddd73feaf_5f86244bad-r0/temp/log.do_cve_check.433603
NOTE: recipe zephyr-philosophers-3.1.0+gitAUTOINC+2ddd73feaf_5f86244bad-r0:
taskdo_cve_check: Failed
ERROR: Task (/tmp/workspace.4jc1Y12y3W/oe-core/../meta-zephyr/meta-zephyr-core/recipes-kernel/zephyr-kernel/zephyr-philosophers.bb:do_cve_check) failed with exit code '1'
Quoted 4 lines
This is a mismatch with bitbake, because one commit hasn't reached 2.0. Steve, should I be using
https://git.openembedded.org/bitbake-contrib/log/?h=stable/2.0-nut for testing?
If you are going to try to use oe-core stable/kirkstone-nut then, yes, you should also use bitbake stable/2.0-nut since occasionally changes in core must be paired with a bitbake change.
However . . .
I highly discourage anyone from using the -nut branches! They are guaranteed to be rebased (and broken!), sometimes several times a day. So be prepared for pain!
Patches move to stable/kirkstone-next (and stable2.0-next) a day after I post a patchset for review. At this point they have passed all testing and are unlikely to change unless the community finds an issue during the review period. So it would be safer to use the stable/*-next branch (but I still discourage it since it is possible it will change)
Steve
Quoted 8 lines
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#170239): https://lists.openembedded.org/g/openembedded-core/message/170239
Mute This Topic: https://lists.openembedded.org/mt/93352038/3617601
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-