Skip to contentSkip to conversation

cve-check: Fix report generation

PATCH In review Marta Rybczynska 4 participants 5 messages May 17, 2022 – May 18, 2022

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 cve-check: Fix report generation

The addition of summary output caused two issues: error when building an image and the fact that JSON output was generated even when CVE_CHECK_FORMAT_JSON.

When generating an image it caused an error like:
ERROR: core-image-minimal-1.0-r0 do_rootfs: Error executing a python function in exec_func_python() autogenerated:

The stack trace of python calls that resulted in this exception/failure was:
File: 'exec_func_python() autogenerated', lineno: 2, function: <module>

       0001:
   *** 0002:cve_check_write_rootfs_manifest(d)
       0003:
  File: '/home/alexk/poky/meta/classes/cve-check.bbclass', lineno: 213, function: cve_check_write_rootfs_manifest
       0209:
       0210:        link_path = os.path.join(deploy_dir, "%s.json" % link_name)
       0211:        manifest_path = d.getVar("CVE_CHECK_MANIFEST_JSON")
       0212:        bb.note("Generating JSON CVE manifest")
   *** 0213:        generate_json_report(json_summary_name, json_summary_link_name)
       0214:        bb.plain("Image CVE JSON report stored in: %s" % link_path)
       0215:}
       0216:
       0217:ROOTFS_POSTPROCESS_COMMAND:prepend = "${@'cve_check_write_rootfs_manifest; ' if d.getVar('CVE_CHECK_CREATE_MANIFEST') == '1' else ''}"

Exception: NameError: name 'json_summary_name' is not defined

The fix is to pass the d variable to the pure python function generate_json_report to get correct values of variables and add conditions for the JSON output where needed.

In addition clarify the message presenting the summary JSON file, which isn't related to an image.

Uses partial fixes from Alex Kiernan, Ernst Sjöstrand (ernstp), and Davide Gardenal.

Fixes f2987891d315 ("cve-check: add JSON format to summary output")
---
 meta/classes/cve-check.bbclass | 18 ++++++++++--------
 1 file changed, 10 insertions(+), 8 deletions(-)
meta/classes/cve-check.bbclass +10 -8
index 24ddb865ea..7cd98ae462 100644
@@ -79,7 +79,7 @@ CVE_CHECK_LAYER_INCLUDELIST ??= ""
 # set to "alphabetical" for version using single alphabetical character as increment release
 CVE_VERSION_SUFFIX ??= ""
 
-def generate_json_report(out_path, link_path):
+def generate_json_report(d, out_path, link_path):
     if os.path.exists(d.getVar("CVE_CHECK_SUMMARY_INDEX_PATH")):
         import json
         from oe.cve_check import cve_check_merge_jsons
@@ -127,10 +127,11 @@ python cve_save_summary_handler () {
                     os.remove(cvefile_link)
                 os.symlink(os.path.basename(cve_summary_file), cvefile_link)
 
+    if d.getVar("CVE_CHECK_FORMAT_JSON") == "1":
         json_summary_link_name = os.path.join(cvelogpath, d.getVar("CVE_CHECK_SUMMARY_FILE_NAME_JSON"))
         json_summary_name = os.path.join(cvelogpath, "%s-%s.json" % (cve_summary_name, timestamp))
-        generate_json_report(json_summary_name, json_summary_link_name)
-        bb.plain("CVE report summary created at: %s" % json_summary_link_name)
+        generate_json_report(d, json_summary_name, json_summary_link_name)
+        bb.plain("Complete CVE JSON report summary created at: %s" % json_summary_link_name)
 }
 
 addhandler cve_save_summary_handler
@@ -207,11 +208,12 @@ python cve_check_write_rootfs_manifest () {
                 os.symlink(os.path.basename(manifest_name), manifest_link)
             bb.plain("Image CVE report stored in: %s" % manifest_name)
 
-        link_path = os.path.join(deploy_dir, "%s.json" % link_name)
-        manifest_path = d.getVar("CVE_CHECK_MANIFEST_JSON")
-        bb.note("Generating JSON CVE manifest")
-        generate_json_report(json_summary_name, json_summary_link_name)
-        bb.plain("Image CVE JSON report stored in: %s" % link_path)
+        if d.getVar("CVE_CHECK_FORMAT_JSON") == "1":
+            link_path = os.path.join(deploy_dir, "%s.json" % link_name)
+            manifest_path = d.getVar("CVE_CHECK_MANIFEST_JSON")
+            bb.note("Generating JSON CVE manifest")
+            generate_json_report(d, manifest_path, link_path)
+            bb.plain("Image CVE JSON report stored in: %s" % link_path)
 }
 
 ROOTFS_POSTPROCESS_COMMAND:prepend = "${@'cve_check_write_rootfs_manifest; ' if d.getVar('CVE_CHECK_CREATE_MANIFEST') == '1' else ''}"
Signature · 1 line
2.33.0
Reviewed-by on v1 1/1
#1 lore raw

Den tis 17 maj 2022 kl 08:01 skrev Marta Rybczynska <[email protected]>:

Quoted 119 lines

The addition of summary output caused two issues: error when building an image and the fact that JSON output was generated even when CVE_CHECK_FORMAT_JSON.

When generating an image it caused an error like:
ERROR: core-image-minimal-1.0-r0 do_rootfs: Error executing a python function in exec_func_python() autogenerated:

The stack trace of python calls that resulted in this exception/failure
was:
File: 'exec_func_python() autogenerated', lineno: 2, function: <module>

       0001:
   *** 0002:cve_check_write_rootfs_manifest(d)
       0003:
  File: '/home/alexk/poky/meta/classes/cve-check.bbclass', lineno: 213,
function: cve_check_write_rootfs_manifest
       0209:
       0210:        link_path = os.path.join(deploy_dir, "%s.json" %
link_name)
       0211:        manifest_path = d.getVar("CVE_CHECK_MANIFEST_JSON")
       0212:        bb.note("Generating JSON CVE manifest")
   *** 0213:        generate_json_report(json_summary_name,
json_summary_link_name)
       0214:        bb.plain("Image CVE JSON report stored in: %s" %
link_path)
       0215:}
       0216:
       0217:ROOTFS_POSTPROCESS_COMMAND:prepend =

"${@'cve_check_write_rootfs_manifest; ' if d.getVar('CVE_CHECK_CREATE_MANIFEST') == '1' else ''}"
Exception: NameError: name 'json_summary_name' is not defined

The fix is to pass the d variable to the pure python function generate_json_report
to get correct values of variables and add conditions for the JSON output where needed.

In addition clarify the message presenting the summary JSON file, which isn't related to an image.

Uses partial fixes from Alex Kiernan, Ernst Sjöstrand (ernstp), and Davide Gardenal.

Fixes f2987891d315 ("cve-check: add JSON format to summary output")
---
 meta/classes/cve-check.bbclass | 18 ++++++++++--------
 1 file changed, 10 insertions(+), 8 deletions(-)
meta/classes/cve-check.bbclass +0 -0
index 24ddb865ea..7cd98ae462 100644
@@ -79,7 +79,7 @@ CVE_CHECK_LAYER_INCLUDELIST ??= ""
 # set to "alphabetical" for version using single alphabetical character

as increment release
CVE_VERSION_SUFFIX ??= ""

meta/classes/cve-check.bbclass +2 -3
-def generate_json_report(out_path, link_path):
+def generate_json_report(d, out_path, link_path):
     if os.path.exists(d.getVar("CVE_CHECK_SUMMARY_INDEX_PATH")):
         import json
         from oe.cve_check import cve_check_merge_jsons
@@ -127,10 +127,11 @@ python cve_save_summary_handler () {
                     os.remove(cvefile_link)
                 os.symlink(os.path.basename(cve_summary_file),
 cvefile_link)
 
+    if d.getVar("CVE_CHECK_FORMAT_JSON") == "1":
         json_summary_link_name = os.path.join(cvelogpath,
 d.getVar("CVE_CHECK_SUMMARY_FILE_NAME_JSON"))
         json_summary_name = os.path.join(cvelogpath, "%s-%s.json" %
 (cve_summary_name, timestamp))
-        generate_json_report(json_summary_name, json_summary_link_name)
-        bb.plain("CVE report summary created at: %s" %

json_summary_link_name)

meta/classes/cve-check.bbclass +2 -0
+        generate_json_report(d, json_summary_name, json_summary_link_name)
+        bb.plain("Complete CVE JSON report summary created at: %s" %

json_summary_link_name)
}

addhandler cve_save_summary_handler

meta/classes/cve-check.bbclass +6 -5
@@ -207,11 +208,12 @@ python cve_check_write_rootfs_manifest () {
                 os.symlink(os.path.basename(manifest_name), manifest_link)
             bb.plain("Image CVE report stored in: %s" % manifest_name)
 
-        link_path = os.path.join(deploy_dir, "%s.json" % link_name)
-        manifest_path = d.getVar("CVE_CHECK_MANIFEST_JSON")
-        bb.note("Generating JSON CVE manifest")
-        generate_json_report(json_summary_name, json_summary_link_name)
-        bb.plain("Image CVE JSON report stored in: %s" % link_path)
+        if d.getVar("CVE_CHECK_FORMAT_JSON") == "1":
+            link_path = os.path.join(deploy_dir, "%s.json" % link_name)
+            manifest_path = d.getVar("CVE_CHECK_MANIFEST_JSON")
+            bb.note("Generating JSON CVE manifest")
+            generate_json_report(d, manifest_path, link_path)
+            bb.plain("Image CVE JSON report stored in: %s" % link_path)
 }
 
 ROOTFS_POSTPROCESS_COMMAND:prepend =

"${@'cve_check_write_rootfs_manifest; ' if d.getVar('CVE_CHECK_CREATE_MANIFEST') == '1' else ''}" --
2.33.0

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group. View/Reply Online (#165707):
https://lists.openembedded.org/g/openembedded-core/message/165707
Mute This Topic: https://lists.openembedded.org/mt/91158052/4947266
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [ [email protected]]
-=-=-=-=-=-=-=-=-=-=-=-

Tested-by
#2 lore raw
Quoted 103 lines · Ernst Sjöstrand

Den tis 17 maj 2022 kl 08:01 skrev Marta Rybczynska <[email protected]>:

Quoted 99 lines

The addition of summary output caused two issues: error when building an image and the fact that JSON output was generated even when CVE_CHECK_FORMAT_JSON.

When generating an image it caused an error like:
ERROR: core-image-minimal-1.0-r0 do_rootfs: Error executing a python function in exec_func_python() autogenerated:

The stack trace of python calls that resulted in this exception/failure was:
File: 'exec_func_python() autogenerated', lineno: 2, function: <module>

       0001:
   *** 0002:cve_check_write_rootfs_manifest(d)
       0003:
  File: '/home/alexk/poky/meta/classes/cve-check.bbclass', lineno: 213, function: cve_check_write_rootfs_manifest
       0209:
       0210:        link_path = os.path.join(deploy_dir, "%s.json" % link_name)
       0211:        manifest_path = d.getVar("CVE_CHECK_MANIFEST_JSON")
       0212:        bb.note("Generating JSON CVE manifest")
   *** 0213:        generate_json_report(json_summary_name, json_summary_link_name)
       0214:        bb.plain("Image CVE JSON report stored in: %s" % link_path)
       0215:}
       0216:
       0217:ROOTFS_POSTPROCESS_COMMAND:prepend = "${@'cve_check_write_rootfs_manifest; ' if d.getVar('CVE_CHECK_CREATE_MANIFEST') == '1' else ''}"

Exception: NameError: name 'json_summary_name' is not defined

The fix is to pass the d variable to the pure python function generate_json_report to get correct values of variables and add conditions for the JSON output where needed.

In addition clarify the message presenting the summary JSON file, which isn't related to an image.

Uses partial fixes from Alex Kiernan, Ernst Sjöstrand (ernstp), and Davide Gardenal.

Fixes f2987891d315 ("cve-check: add JSON format to summary output")
---
 meta/classes/cve-check.bbclass | 18 ++++++++++--------
 1 file changed, 10 insertions(+), 8 deletions(-)
meta/classes/cve-check.bbclass +10 -9
index 24ddb865ea..7cd98ae462 100644
@@ -79,7 +79,7 @@ CVE_CHECK_LAYER_INCLUDELIST ??= ""
 # set to "alphabetical" for version using single alphabetical character as increment release
 CVE_VERSION_SUFFIX ??= ""
 
-def generate_json_report(out_path, link_path):
+def generate_json_report(d, out_path, link_path):
     if os.path.exists(d.getVar("CVE_CHECK_SUMMARY_INDEX_PATH")):
         import json
         from oe.cve_check import cve_check_merge_jsons
@@ -127,10 +127,11 @@ python cve_save_summary_handler () {
                     os.remove(cvefile_link)
                 os.symlink(os.path.basename(cve_summary_file), cvefile_link)
 
+    if d.getVar("CVE_CHECK_FORMAT_JSON") == "1":
         json_summary_link_name = os.path.join(cvelogpath, d.getVar("CVE_CHECK_SUMMARY_FILE_NAME_JSON"))
         json_summary_name = os.path.join(cvelogpath, "%s-%s.json" % (cve_summary_name, timestamp))
-        generate_json_report(json_summary_name, json_summary_link_name)
-        bb.plain("CVE report summary created at: %s" % json_summary_link_name)
+        generate_json_report(d, json_summary_name, json_summary_link_name)
+        bb.plain("Complete CVE JSON report summary created at: %s" % json_summary_link_name)
 }
 
 addhandler cve_save_summary_handler
@@ -207,11 +208,12 @@ python cve_check_write_rootfs_manifest () {
                 os.symlink(os.path.basename(manifest_name), manifest_link)
             bb.plain("Image CVE report stored in: %s" % manifest_name)
 
-        link_path = os.path.join(deploy_dir, "%s.json" % link_name)
-        manifest_path = d.getVar("CVE_CHECK_MANIFEST_JSON")
-        bb.note("Generating JSON CVE manifest")
-        generate_json_report(json_summary_name, json_summary_link_name)
-        bb.plain("Image CVE JSON report stored in: %s" % link_path)
+        if d.getVar("CVE_CHECK_FORMAT_JSON") == "1":
+            link_path = os.path.join(deploy_dir, "%s.json" % link_name)
+            manifest_path = d.getVar("CVE_CHECK_MANIFEST_JSON")
+            bb.note("Generating JSON CVE manifest")
+            generate_json_report(d, manifest_path, link_path)
+            bb.plain("Image CVE JSON report stored in: %s" % link_path)
 }
 
 ROOTFS_POSTPROCESS_COMMAND:prepend = "${@'cve_check_write_rootfs_manifest; ' if d.getVar('CVE_CHECK_CREATE_MANIFEST') == '1' else ''}"
--

2.33.0

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#165707): https://lists.openembedded.org/g/openembedded-core/message/165707
Mute This Topic: https://lists.openembedded.org/mt/91158052/4947266
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-

Signature · 1 line
Alex Kiernan
#3 lore raw
In reply to Marta Rybczynska
Quoted 7 lines · Marta Rybczynska

The addition of summary output caused two issues: error when building an image and the fact that JSON output was generated even when CVE_CHECK_FORMAT_JSON.

When generating an image it caused an error like:
ERROR: core-image-minimal-1.0-r0 do_rootfs: Error executing a python function in exec_func_python() autogenerated:

Hello Steve,
Would you pick this one for dunfell too? The bug affects dunfell too ;( Or do you prefer me to do a resend
a well-applying patch?

Kind regards,
Marta

#4 lore raw
Quoted 7 lines · Marta Rybczynska
Quoted 5 lines · Marta Rybczynska

The addition of summary output caused two issues: error when building an image and the fact that JSON output was generated even when CVE_CHECK_FORMAT_JSON.

Quoted 7 lines
Quoted 2 lines

When generating an image it caused an error like:
ERROR: core-image-minimal-1.0-r0 do_rootfs: Error executing a python function in exec_func_python() autogenerated:

Hello Steve,
Would you pick this one for dunfell too? The bug affects dunfell too ;( Or do you prefer me to do a resend
a well-applying patch?

I was able to cherry-pick it with no problem, so no need to send a patch.

Thanks!

Steve