Patch review
cover Patch review
Please review this set of patches for dunfell and have comments back by end of day Monday.
Passed a-full on autobuilder:
https://autobuilder.yoctoproject.org/typhoon/#/builders/83/builds/3587
The following changes since commit 8e81d38048c953d0823abf04d5b2506cd988f0bb:
build-appliance-image: Update to dunfell head revision (2022-04-25 15:58:54 +0100)
are available in the Git repository at:
git://git.openembedded.org/openembedded-core-contrib stable/dunfell-nut
http://cgit.openembedded.org/openembedded-core-contrib/log/?h=stable/dunfell-nut
Dmitry Baryshkov (1):
linux-firmware: correct license for ar3k firmware
Marta Rybczynska (1):
cve-check: add json format
Richard Purdie (1):
perf-build-test/report: Drop phantomjs and html email reports support
Ross Burton (1):
boost: don't specify gcc version
Steve Sakoman (1):
scripts/contrib/oe-build-perf-report-email.py: remove obsolete check
for phantomjs and optipng
sana kazi (1):
tiff: Fix CVE-2022-0891
meta/classes/cve-check.bbclass | 144 +++++++++++- meta/lib/oe/cve_check.py | 16 ++ .../linux-firmware/linux-firmware_20220411.bb | 4 +- .../libtiff/files/CVE-2022-0891.patch | 217 ++++++++++++++++++ meta/recipes-multimedia/libtiff/tiff_4.1.0.bb | 1 + meta/recipes-support/boost/boost.inc | 2 +- scripts/contrib/build-perf-test-wrapper.sh | 15 +- scripts/contrib/oe-build-perf-report-email.py | 167 +------------- 8 files changed, 388 insertions(+), 178 deletions(-) create mode 100644 meta/recipes-multimedia/libtiff/files/CVE-2022-0891.patch
Signature · 1 line
2.25.1
PATCH 1/6 tiff: Fix CVE-2022-0891
From: sana kazi <[email protected]>
Fix CVE-2022-0891 for tiff
--- .../libtiff/files/CVE-2022-0891.patch | 217 ++++++++++++++++++ meta/recipes-multimedia/libtiff/tiff_4.1.0.bb | 1 + 2 files changed, 218 insertions(+) create mode 100644 meta/recipes-multimedia/libtiff/files/CVE-2022-0891.patch
meta/recipes-multimedia/libtiff/files/CVE-2022-0891.patch new file +217 -0
@@ -0,0 +1,217 @@ +From 232282fd8f9c21eefe8d2d2b96cdbbb172fe7b7c Mon Sep 17 00:00:00 2001 +From: Su Laus <[email protected]> +Date: Tue, 8 Mar 2022 17:02:44 +0000 +Subject: [PATCH] tiffcrop: fix issue #380 and #382 heap buffer overflow in + extractImageSection + +CVE: CVE-2022-0891 +Upstream-Status: Backport [https://sources.debian.org/src/tiff/4.1.0+git191117-2%7Edeb10u4/debian/patches/CVE-2022-0891.patch/] +Comment: No change in any hunk +Signed-off-by: Sana Kazi <[email protected]> +--- + tools/tiffcrop.c | 92 +++++++++++++++++++----------------------------- + 1 file changed, 36 insertions(+), 56 deletions(-) + +diff --git a/tools/tiffcrop.c b/tools/tiffcrop.c +index f2e5474a..e62bcc71 100644 +--- a/tools/tiffcrop.c ++++ b/tools/tiffcrop.c +@@ -105,8 +105,8 @@ + * of messages to monitor progess without enabling dump logs. + */ + +-static char tiffcrop_version_id[] = "2.4"; +-static char tiffcrop_rev_date[] = "12-13-2010"; ++static char tiffcrop_version_id[] = "2.4.1"; ++static char tiffcrop_rev_date[] = "03-03-2010"; + + #include "tif_config.h" + #include "tiffiop.h" +@@ -6670,10 +6670,10 @@ extractImageSection(struct image_data *image, struct pageseg *section, + #ifdef DEVELMODE + uint32 img_length; + #endif +- uint32 j, shift1, shift2, trailing_bits; ++ uint32 j, shift1, trailing_bits; + uint32 row, first_row, last_row, first_col, last_col; + uint32 src_offset, dst_offset, row_offset, col_offset; +- uint32 offset1, offset2, full_bytes; ++ uint32 offset1, full_bytes; + uint32 sect_width; + #ifdef DEVELMODE + uint32 sect_length; +@@ -6683,7 +6683,6 @@ extractImageSection(struct image_data *image, struct pageseg *section, + #ifdef DEVELMODE + int k; + unsigned char bitset; +- static char *bitarray = NULL; + #endif + + img_width = image->width; +@@ -6701,17 +6700,12 @@ extractImageSection(struct image_data *image, struct pageseg *section, + dst_offset = 0; + + #ifdef DEVELMODE +- if (bitarray == NULL) +- { +- if ((bitarray = (char *)malloc(img_width)) == NULL) +- { +- TIFFError ("", "DEBUG: Unable to allocate debugging bitarray"); +- return (-1); +- } +- } ++ char bitarray[39]; + #endif + +- /* rows, columns, width, length are expressed in pixels */ ++ /* rows, columns, width, length are expressed in pixels ++ * first_row, last_row, .. are index into image array starting at 0 to width-1, ++ * last_col shall be also extracted. */ + first_row = section->y1; + last_row = section->y2; + first_col = section->x1; +@@ -6721,9 +6715,14 @@ extractImageSection(struct image_data *image, struct pageseg *section, + #ifdef DEVELMODE + sect_length = last_row - first_row + 1; + #endif +- img_rowsize = ((img_width * bps + 7) / 8) * spp; +- full_bytes = (sect_width * spp * bps) / 8; /* number of COMPLETE bytes per row in section */ +- trailing_bits = (sect_width * bps) % 8; ++ /* The read function loadImage() used copy separate plane data into a buffer as interleaved ++ * samples rather than separate planes so the same logic works to extract regions ++ * regardless of the way the data are organized in the input file. ++ * Furthermore, bytes and bits are arranged in buffer according to COMPRESSION=1 and FILLORDER=1 ++ */ ++ img_rowsize = (((img_width * spp * bps) + 7) / 8); /* row size in full bytes of source image */ ++ full_bytes = (sect_width * spp * bps) / 8; /* number of COMPLETE bytes per row in section */ ++ trailing_bits = (sect_width * spp * bps) % 8; /* trailing bits within the last byte of destination buffer */ + + #ifdef DEVELMODE + TIFFError ("", "First row: %d, last row: %d, First col: %d, last col: %d\n", +@@ -6736,10 +6735,9 @@ extractImageSection(struct image_data *image, struct pageseg *section, + + if ((bps % 8) == 0) + { +- col_offset = first_col * spp * bps / 8; ++ col_offset = (first_col * spp * bps) / 8; + for (row = first_row; row <= last_row; row++) + { +- /* row_offset = row * img_width * spp * bps / 8; */ + row_offset = row * img_rowsize; + src_offset = row_offset + col_offset; + +@@ -6752,14 +6750,12 @@ extractImageSection(struct image_data *image, struct pageseg *section, + } + else + { /* bps != 8 */ +- shift1 = spp * ((first_col * bps) % 8); +- shift2 = spp * ((last_col * bps) % 8); ++ shift1 = ((first_col * spp * bps) % 8); /* shift1 = bits to skip in the first byte of source buffer*/ + for (row = first_row; row <= last_row; row++) + { + /* pull out the first byte */ + row_offset = row * img_rowsize; +- offset1 = row_offset + (first_col * bps / 8); +- offset2 = row_offset + (last_col * bps / 8); ++ offset1 = row_offset + ((first_col * spp * bps) / 8); /* offset1 = offset into source of byte with first bits to be extracted */ + + #ifdef DEVELMODE + for (j = 0, k = 7; j < 8; j++, k--) +@@ -6771,12 +6767,12 @@ extractImageSection(struct image_data *image, struct pageseg *section, + sprintf(&bitarray[9], " "); + for (j = 10, k = 7; j < 18; j++, k--) + { +- bitset = *(src_buff + offset2) & (((unsigned char)1 << k)) ? 1 : 0; ++ bitset = *(src_buff + offset1 + full_bytes) & (((unsigned char)1 << k)) ? 1 : 0; + sprintf(&bitarray[j], (bitset) ? "1" : "0"); + } + bitarray[18] = '\0'; +- TIFFError ("", "Row: %3d Offset1: %d, Shift1: %d, Offset2: %d, Shift2: %d\n", +- row, offset1, shift1, offset2, shift2); ++ TIFFError ("", "Row: %3d Offset1: %"PRIu32", Shift1: %"PRIu32", Offset2: %"PRIu32", Trailing_bits: %"PRIu32"\n", ++ row, offset1, shift1, offset1+full_bytes, trailing_bits); + #endif + + bytebuff1 = bytebuff2 = 0; +@@ -6800,11 +6796,12 @@ extractImageSection(struct image_data *image, struct pageseg *section, + + if (trailing_bits != 0) + { +- bytebuff2 = src_buff[offset2] & ((unsigned char)255 << (7 - shift2)); ++ /* Only copy higher bits of samples and mask lower bits of not wanted column samples to zero */ ++ bytebuff2 = src_buff[offset1 + full_bytes] & ((unsigned char)255 << (8 - trailing_bits)); + sect_buff[dst_offset] = bytebuff2; + #ifdef DEVELMODE + TIFFError ("", " Trailing bits src offset: %8d, Dst offset: %8d\n", +- offset2, dst_offset); ++ offset1 + full_bytes, dst_offset); + for (j = 30, k = 7; j < 38; j++, k--) + { + bitset = *(sect_buff + dst_offset) & (((unsigned char)1 << k)) ? 1 : 0; +@@ -6823,8 +6820,10 @@ extractImageSection(struct image_data *image, struct pageseg *section, + #endif + for (j = 0; j <= full_bytes; j++) + { +- bytebuff1 = src_buff[offset1 + j] & ((unsigned char)255 >> shift1); +- bytebuff2 = src_buff[offset1 + j + 1] & ((unsigned char)255 << (7 - shift1)); ++ /* Skip the first shift1 bits and shift the source up by shift1 bits before save to destination.*/ ++ /* Attention: src_buff size needs to be some bytes larger than image size, because could read behind image here. */ ++ bytebuff1 = src_buff[offset1 + j] & ((unsigned char)255 >> shift1); ++ bytebuff2 = src_buff[offset1 + j + 1] & ((unsigned char)255 << (8 - shift1)); + sect_buff[dst_offset + j] = (bytebuff1 << shift1) | (bytebuff2 >> (8 - shift1)); + } + #ifdef DEVELMODE +@@ -6840,36 +6839,17 @@ extractImageSection(struct image_data *image, struct pageseg *section, + #endif + dst_offset += full_bytes; + ++ /* Copy the trailing_bits for the last byte in the destination buffer. ++ Could come from one ore two bytes of the source buffer. */ + if (trailing_bits != 0) + { + #ifdef DEVELMODE +- TIFFError ("", " Trailing bits src offset: %8d, Dst offset: %8d\n", offset1 + full_bytes, dst_offset); +-#endif +- if (shift2 > shift1) +- { +- bytebuff1 = src_buff[offset1 + full_bytes] & ((unsigned char)255 << (7 - shift2)); +- bytebuff2 = bytebuff1 & ((unsigned char)255 << shift1); +- sect_buff[dst_offset] = bytebuff2; +-#ifdef DEVELMODE +- TIFFError ("", " Shift2 > Shift1\n"); ++ TIFFError("", " Trailing bits %4"PRIu32" src offset: %8"PRIu32", Dst offset: %8"PRIu32"\n", trailing_bits, offset1 + full_bytes, dst_offset); + #endif ++ /* More than necessary bits are already copied into last destination buffer, ++ * only masking of last byte in destination buffer is necessary.*/ ++ sect_buff[dst_offset] &= ((uint8_t)0xFF << (8 - trailing_bits)); + } +- else +- { +- if (shift2 < shift1) +- { +- bytebuff2 = ((unsigned char)255 << (shift1 - shift2 - 1)); +- sect_buff[dst_offset] &= bytebuff2; +-#ifdef DEVELMODE +- TIFFError ("", " Shift2 < Shift1\n"); +-#endif +- } +-#ifdef DEVELMODE +- else +- TIFFError ("", " Shift2 == Shift1\n"); +-#endif +- } +- } + #ifdef DEVELMODE + sprintf(&bitarray[28], " "); + sprintf(&bitarray[29], " "); +@@ -7022,7 +7002,7 @@ writeImageSections(TIFF *in, TIFF *out, struct image_data *image, + width = sections[i].x2 - sections[i].x1 + 1; + length = sections[i].y2 - sections[i].y1 + 1; + sectsize = (uint32) +- ceil((width * image->bps + 7) / (double)8) * image->spp * length; ++ ceil((width * image->bps * image->spp + 7) / (double)8) * length; + /* allocate a buffer if we don't have one already */ + if (createImageSection(sectsize, sect_buff_ptr)) + { +-- +GitLab
meta/recipes-multimedia/libtiff/tiff_4.1.0.bb +1 -0
@@ -18,6 +18,7 @@ SRC_URI = "http://download.osgeo.org/libtiff/tiff-${PV}.tar.gz \ file://0001-tiffset-fix-global-buffer-overflow-for-ASCII-tags-wh.patch \ file://561599c99f987dc32ae110370cfdd7df7975586b.patch \ file://eecb0712f4c3a5b449f70c57988260a667ddbdef.patch \ + file://CVE-2022-0891.patch \ " SRC_URI[md5sum] = "2165e7aba557463acc0664e71a3ed424" SRC_URI[sha256sum] = "5d29f32517dadb6dbcd1255ea5bbc93a2b54b94fbf83653b4d65c7d6775b8634"
Signature · 1 line
2.25.1
PATCH 2/6 boost: don't specify gcc version
In reply to Steve SakomanFrom: Ross Burton <[email protected]>
There's no need to specify an ancient GCC version here as Boost will probe it.
--- meta/recipes-support/boost/boost.inc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
meta/recipes-support/boost/boost.inc +1 -1
@@ -165,7 +165,7 @@ do_configure() { # D2194:Fixing the failure of "error: duplicate initialization of gcc with the following parameters" during compilation. rm -f ${WORKDIR}/user-config.jam - echo 'using gcc : 4.3.1 : ${CXX} : <cflags>"${CFLAGS}" <cxxflags>"${CXXFLAGS}" <linkflags>"${LDFLAGS}" ;' >> ${WORKDIR}/user-config.jam + echo 'using gcc : : ${CXX} : <cflags>"${CFLAGS}" <cxxflags>"${CXXFLAGS}" <linkflags>"${LDFLAGS}" ;' >> ${WORKDIR}/user-config.jam # If we want Python then we need to tell Boost *exactly* where to find it if ${@bb.utils.contains('BOOST_LIBS', 'python', 'true', 'false', d)}; then
Signature · 1 line
2.25.1
PATCH 3/6 linux-firmware: correct license for ar3k firmware
In reply to Steve SakomanFrom: Dmitry Baryshkov <[email protected]>
According to the WHENCE file, some a3k firmware files are licensed under the special ar3k license, while others are licensed under the more generic Atheros license. Document this by adding extending the LICENSE_${PN}-ar3k and depending on both of them.
--- meta/recipes-kernel/linux-firmware/linux-firmware_20220411.bb | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
meta/recipes-kernel/linux-firmware/linux-firmware_20220411.bb +2 -2
@@ -352,7 +352,7 @@ FILES_${PN}-carl9170 = " \ RDEPENDS_${PN}-carl9170 += "${PN}-gplv2-license" # For QualCommAthos -LICENSE_${PN}-ar3k = "Firmware-qualcommAthos_ar3k" +LICENSE_${PN}-ar3k = "Firmware-qualcommAthos_ar3k & Firmware-atheros_firmware" LICENSE_${PN}-ar3k-license = "Firmware-qualcommAthos_ar3k" LICENSE_${PN}-ath10k = "Firmware-qualcommAthos_ath10k" LICENSE_${PN}-ath10k-license = "Firmware-qualcommAthos_ath10k"
@@ -376,7 +376,7 @@ FILES_${PN}-qca = " \ ${nonarch_base_libdir}/firmware/qca \ " -RDEPENDS_${PN}-ar3k += "${PN}-ar3k-license" +RDEPENDS_${PN}-ar3k += "${PN}-ar3k-license ${PN}-atheros-license" RDEPENDS_${PN}-ath10k += "${PN}-ath10k-license" RDEPENDS_${PN}-ath11k += "${PN}-ath10k-license" RDEPENDS_${PN}-qca += "${PN}-ath10k-license"
Signature · 1 line
2.25.1
PATCH 4/6 cve-check: add json format
In reply to Steve SakomanFrom: Marta Rybczynska <[email protected]>
Backport to dunfell from master df567de36ae5964bee433ebb97e8bf702034994a
Add an option to output the CVE check in a JSON-based format. This format is easier to parse in software than the original text-based one and allows post-processing by other tools.
Output formats are now handed by CVE_CHECK_FORMAT_TEXT and CVE_CHECK_FORMAT_JSON. The text format is enabled by default to maintain compatibility, while the JSON format is disabled by default.
The JSON output format gets generated in a similar way to the text format with the exception of the manifest: appending to JSON arrays requires parsing the file. Because of that we first write JSON fragments and then assemble them in one pass at the end.
--- meta/classes/cve-check.bbclass | 144 ++++++++++++++++++++++++++++++++- meta/lib/oe/cve_check.py | 16 ++++ 2 files changed, 159 insertions(+), 1 deletion(-)
meta/classes/cve-check.bbclass +143 -1
@@ -34,15 +34,27 @@ CVE_CHECK_TMP_FILE ?= "${TMPDIR}/cve_check" CVE_CHECK_SUMMARY_DIR ?= "${LOG_DIR}/cve" CVE_CHECK_SUMMARY_FILE_NAME ?= "cve-summary" CVE_CHECK_SUMMARY_FILE ?= "${CVE_CHECK_SUMMARY_DIR}/${CVE_CHECK_SUMMARY_FILE_NAME}" +CVE_CHECK_SUMMARY_FILE_NAME_JSON = "cve-summary.json" +CVE_CHECK_SUMMARY_INDEX_PATH = "${CVE_CHECK_SUMMARY_DIR}/cve-summary-index.txt" + +CVE_CHECK_LOG_JSON ?= "${T}/cve.json" CVE_CHECK_DIR ??= "${DEPLOY_DIR}/cve" CVE_CHECK_RECIPE_FILE ?= "${CVE_CHECK_DIR}/${PN}" +CVE_CHECK_RECIPE_FILE_JSON ?= "${CVE_CHECK_DIR}/${PN}_cve.json" CVE_CHECK_MANIFEST ?= "${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.cve" +CVE_CHECK_MANIFEST_JSON ?= "${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.json" CVE_CHECK_COPY_FILES ??= "1" CVE_CHECK_CREATE_MANIFEST ??= "1" CVE_CHECK_REPORT_PATCHED ??= "1" +# Provide text output +CVE_CHECK_FORMAT_TEXT ??= "1" + +# Provide JSON output - disabled by default for backward compatibility +CVE_CHECK_FORMAT_JSON ??= "0" + # Whitelist for packages (PN) CVE_CHECK_PN_WHITELIST ?= ""
@@ -118,6 +130,7 @@ python cve_check_cleanup () { Delete the file used to gather all the CVE information. """ bb.utils.remove(e.data.getVar("CVE_CHECK_TMP_FILE")) + bb.utils.remove(e.data.getVar("CVE_CHECK_SUMMARY_INDEX_PATH")) } addhandler cve_check_cleanup
@@ -129,11 +142,15 @@ python cve_check_write_rootfs_manifest () { """ import shutil + from oe.cve_check import cve_check_merge_jsons if d.getVar("CVE_CHECK_COPY_FILES") == "1": deploy_file = d.getVar("CVE_CHECK_RECIPE_FILE") if os.path.exists(deploy_file): bb.utils.remove(deploy_file) + deploy_file_json = d.getVar("CVE_CHECK_RECIPE_FILE_JSON") + if os.path.exists(deploy_file_json): + bb.utils.remove(deploy_file_json) if os.path.exists(d.getVar("CVE_CHECK_TMP_FILE")): bb.note("Writing rootfs CVE manifest")
@@ -152,6 +169,26 @@ python cve_check_write_rootfs_manifest () { os.remove(manifest_link) os.symlink(os.path.basename(manifest_name), manifest_link) bb.plain("Image CVE report stored in: %s" % manifest_name) + + if os.path.exists(d.getVar("CVE_CHECK_SUMMARY_INDEX_PATH")): + import json + bb.note("Generating JSON CVE manifest") + deploy_dir = d.getVar("DEPLOY_DIR_IMAGE") + link_name = d.getVar("IMAGE_LINK_NAME") + manifest_name = d.getVar("CVE_CHECK_MANIFEST_JSON") + index_file = d.getVar("CVE_CHECK_SUMMARY_INDEX_PATH") + manifest = {"version":"1", "package": []} + with open(index_file) as f: + filename = f.readline() + while filename: + with open(filename.rstrip()) as j: + data = json.load(j) + cve_check_merge_jsons(manifest, data) + filename = f.readline() + + with open(manifest_name, "w") as f: + json.dump(manifest, f, indent=2) + bb.plain("Image CVE report stored in: %s" % manifest_name) } ROOTFS_POSTPROCESS_COMMAND_prepend = "${@'cve_check_write_rootfs_manifest; ' if d.getVar('CVE_CHECK_CREATE_MANIFEST') == '1' else ''}"
@@ -337,7 +374,7 @@ def get_cve_info(d, cves): conn.close() return cve_data -def cve_write_data(d, patched, unpatched, whitelisted, cve_data): +def cve_write_data_text(d, patched, unpatched, whitelisted, cve_data): """ Write CVE information in WORKDIR; and to CVE_CHECK_DIR, and CVE manifest if enabled.
@@ -403,3 +440,108 @@ def cve_write_data(d, patched, unpatched, whitelisted, cve_data): with open(d.getVar("CVE_CHECK_TMP_FILE"), "a") as f: f.write("%s" % write_string) + +def cve_check_write_json_output(d, output, direct_file, deploy_file, manifest_file): + """ + Write CVE information in the JSON format: to WORKDIR; and to + CVE_CHECK_DIR, if CVE manifest if enabled, write fragment + files that will be assembled at the end in cve_check_write_rootfs_manifest. + """ + + import json + + write_string = json.dumps(output, indent=2) + with open(direct_file, "w") as f: + bb.note("Writing file %s with CVE information" % direct_file) + f.write(write_string) + + if d.getVar("CVE_CHECK_COPY_FILES") == "1": + bb.utils.mkdirhier(os.path.dirname(deploy_file)) + with open(deploy_file, "w") as f: + f.write(write_string) + + if d.getVar("CVE_CHECK_CREATE_MANIFEST") == "1": + cvelogpath = d.getVar("CVE_CHECK_SUMMARY_DIR") + index_path = d.getVar("CVE_CHECK_SUMMARY_INDEX_PATH") + bb.utils.mkdirhier(cvelogpath) + fragment_file = os.path.basename(deploy_file) + fragment_path = os.path.join(cvelogpath, fragment_file) + with open(fragment_path, "w") as f: + f.write(write_string) + with open(index_path, "a+") as f: + f.write("%s\n" % fragment_path) + +def cve_write_data_json(d, patched, unpatched, ignored, cve_data): + """ + Prepare CVE data for the JSON format, then write it. + """ + + output = {"version":"1", "package": []} + nvd_link = "https://nvd.nist.gov/vuln/detail/" + + fdir_name = d.getVar("FILE_DIRNAME") + layer = fdir_name.split("/")[-3] + + include_layers = d.getVar("CVE_CHECK_LAYER_INCLUDELIST").split() + exclude_layers = d.getVar("CVE_CHECK_LAYER_EXCLUDELIST").split() + + if exclude_layers and layer in exclude_layers: + return + + if include_layers and layer not in include_layers: + return + + unpatched_cves = [] + + package_version = "%s%s" % (d.getVar("EXTENDPE"), d.getVar("PV")) + package_data = { + "name" : d.getVar("PN"), + "layer" : layer, + "version" : package_version + } + cve_list = [] + + for cve in sorted(cve_data): + is_patched = cve in patched + status = "Unpatched" + if is_patched and (d.getVar("CVE_CHECK_REPORT_PATCHED") != "1"): + continue + if cve in ignored: + status = "Ignored" + elif is_patched: + status = "Patched" + else: + # default value of status is Unpatched + unpatched_cves.append(cve) + + issue_link = "%s%s" % (nvd_link, cve) + + cve_item = { + "id" : cve, + "summary" : cve_data[cve]["summary"], + "scorev2" : cve_data[cve]["scorev2"], + "scorev3" : cve_data[cve]["scorev3"], + "vector" : cve_data[cve]["vector"], + "status" : status, + "link": issue_link + } + cve_list.append(cve_item) + + package_data["issue"] = cve_list + output["package"].append(package_data) + + direct_file = d.getVar("CVE_CHECK_LOG_JSON") + deploy_file = d.getVar("CVE_CHECK_RECIPE_FILE_JSON") + manifest_file = d.getVar("CVE_CHECK_SUMMARY_FILE_NAME_JSON") + + cve_check_write_json_output(d, output, direct_file, deploy_file, manifest_file) + +def cve_write_data(d, patched, unpatched, ignored, cve_data): + """ + Write CVE data in each enabled format. + """ + + if d.getVar("CVE_CHECK_FORMAT_TEXT") == "1": + cve_write_data_text(d, patched, unpatched, ignored, cve_data) + if d.getVar("CVE_CHECK_FORMAT_JSON") == "1": + cve_write_data_json(d, patched, unpatched, ignored, cve_data)
meta/lib/oe/cve_check.py +16 -0
@@ -63,3 +63,19 @@ def _cmpkey(release, patch_l, pre_l, pre_v): else: _pre = float(pre_v) if pre_v else float('-inf') return _release, _patch, _pre + +def cve_check_merge_jsons(output, data): + """ + Merge the data in the "package" property to the main data file + output + """ + if output["version"] != data["version"]: + bb.error("Version mismatch when merging JSON outputs") + return + + for product in output["package"]: + if product["name"] == data["package"][0]["name"]: + bb.error("Error adding the same package twice") + return + + output["package"].append(data["package"][0])
Signature · 1 line
2.25.1
PATCH 5/6 perf-build-test/report: Drop phantomjs and html email reports support
In reply to Steve SakomanFrom: Richard Purdie <[email protected]>
phantomjs isn't reliable and we've moved to sharing the reports via a webserver. Update the scripts to more match those being used in the autobuilder helper where the html email support was removed.
--- scripts/contrib/build-perf-test-wrapper.sh | 15 +- scripts/contrib/oe-build-perf-report-email.py | 154 +----------------- 2 files changed, 8 insertions(+), 161 deletions(-)
scripts/contrib/build-perf-test-wrapper.sh +2 -13
@@ -87,21 +87,10 @@ if [ $# -ne 0 ]; then exit 1 fi -if [ -n "$email_to" ]; then - if ! [ -x "$(command -v phantomjs)" ]; then - echo "ERROR: Sending email needs phantomjs." - exit 1 - fi - if ! [ -x "$(command -v optipng)" ]; then - echo "ERROR: Sending email needs optipng." - exit 1 - fi -fi - # Open a file descriptor for flock and acquire lock LOCK_FILE="/tmp/oe-build-perf-test-wrapper.lock" if ! exec 3> "$LOCK_FILE"; then - echo "ERROR: Unable to open lock file" + echo "ERROR: Unable to open loemack file" exit 1 fi if ! flock -n 3; then
@@ -226,7 +215,7 @@ if [ -n "$results_repo" ]; then if [ -n "$email_to" ]; then echo "Emailing test report" os_name=`get_os_release_var PRETTY_NAME` - "$script_dir"/oe-build-perf-report-email.py --to "$email_to" --subject "Build Perf Test Report for $os_name" --text $report_txt --html $report_html "${OE_BUILD_PERF_REPORT_EMAIL_EXTRA_ARGS[@]}" + "$script_dir"/oe-build-perf-report-email.py --to "$email_to" --subject "Build Perf Test Report for $os_name" --text $report_txt "${OE_BUILD_PERF_REPORT_EMAIL_EXTRA_ARGS[@]}" fi # Upload report files, unless we're on detached head
scripts/contrib/oe-build-perf-report-email.py +6 -148
195 lines not on this page · raw message
Signature · 1 line
2.25.1
PATCH 6/6 scripts/contrib/oe-build-perf-report-email.py: remove obsolete check for phantomjs and optipng
In reply to Steve SakomanUse of those tools was removed in b5c131006e3fad0a15e6cdf81f71dc1e96647028 perf-build-test/report: Drop phantomjs and html email reports support
--- scripts/contrib/oe-build-perf-report-email.py | 13 ------------- 1 file changed, 13 deletions(-)
scripts/contrib/oe-build-perf-report-email.py +0 -13
@@ -27,17 +27,6 @@ logging.basicConfig(level=logging.INFO, format="%(levelname)s: %(message)s") log = logging.getLogger('oe-build-perf-report') -def check_utils(): - """Check that all needed utils are installed in the system""" - missing = [] - for cmd in ('phantomjs', 'optipng'): - if not shutil.which(cmd): - missing.append(cmd) - if missing: - log.error("The following tools are missing: %s", ' '.join(missing)) - sys.exit(1) - - def parse_args(argv): """Parse command line arguments""" description = """Email build perf test report"""
@@ -101,8 +90,6 @@ def main(argv=None): if args.debug: log.setLevel(logging.DEBUG) - check_utils() - if args.outdir: outdir = args.outdir if not os.path.exists(outdir):
Signature · 1 line
2.25.1
A message that is not in the archive (sent off-list or not yet received): [email protected]
Re: [OE-core][dunfell 6/6] scripts/contrib/oe-build-perf-report-email.py: remove obsolete check for phantomjs and optipng
This patch is in master-next, but not yet master. I decided to run the dunfell review process in parallel with master. I will wait till the patch hits master before sending a pull request (and will add the cherry-pick notation)
Steve
Quoted 52 lines · Steve Sakoman via lists.openembedded.org
Use of those tools was removed in
b5c131006e3fad0a15e6cdf81f71dc1e96647028perf-build-test/report: Drop phantomjs and html email reports supportSigned-off-by Steve Sakoman <[email protected]>--- scripts/contrib/oe-build-perf-report-email.py | 13 ------------- 1 file changed, 13 deletions(-)scripts/contrib/oe-build-perf-report-email.py +0
-14@@ -27,17 +27,6 @@ logging.basicConfig(level=logging.INFO, format="%(levelname)s: %(message)s") log = logging.getLogger('oe-build-perf-report') -def check_utils(): - """Check that all needed utils are installed in the system""" - missing = [] - for cmd in ('phantomjs', 'optipng'): - if not shutil.which(cmd): - missing.append(cmd) - if missing: - log.error("The following tools are missing: %s", ' '.join(missing)) - sys.exit(1) - - def parse_args(argv): """Parse command line arguments""" description = """Email build perf test report"""@@ -101,8 +90,6 @@ def main(argv=None): if args.debug: log.setLevel(logging.DEBUG) - check_utils() - if args.outdir: outdir = args.outdir if not os.path.exists(outdir): --2.25.1
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#164984): https://lists.openembedded.org/g/openembedded-core/message/164984
Mute This Topic: https://lists.openembedded.org/mt/90764072/3620601
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-