Patch review
cover Patch review
Please review this set of changes for nanbield and have comments back by end of day Wednesday, November 22
Mostly passed a-full:
https://autobuilder.yoctoproject.org/typhoon/#/builders/83/builds/6216
with the exception of known autobbuilder intermittent issues:
https://bugzilla.yoctoproject.org/show_bug.cgi?id=15005
https://bugzilla.yoctoproject.org/show_bug.cgi?id=15166
https://bugzilla.yoctoproject.org/show_bug.cgi?id=15286
These intermittent issues are making life painful for your Yocto maintainers, any help in resolving them would bring smiles back to their faces :-)
The following changes since commit 28e6fde4627ffd053dde8a8d44441a40dafd545c:
layer.conf: Switch layer to nanbield series only (2023-11-09 17:09:43 +0000)
are available in the Git repository at:
https://git.openembedded.org/openembedded-core-contrib stable/nanbield-nut
https://git.openembedded.org/openembedded-core-contrib/log/?h=stable/nanbield-nut
Alejandro Hernandez Samaniego (1):
baremetal-helloworld: Pull in fix for race condition on x86-64
Alex Stewart (1):
libsndfile1: fix CVE-2022-33065
Joshua Watt (1):
goarch: Move Go architecture mapping to a library
Julien Stephan (1):
oeqa/selftest/devtool: abort if a local workspace already exist
Khem Raj (2):
llvm: Upgrade to 17.0.3
kernel.bbclass: Use strip utility used for kernel build in do_package
Max Krummenacher (1):
Revert "bin_package.bbclass: Inhibit the default dependencies"
Peter Kjellerstedt (1):
bb-matrix-plot.sh: Show underscores correctly in labels
Peter Marko (1):
openssl: Upgrade 3.1.3 -> 3.1.4
Richard Purdie (1):
base: Ensure recipes using mercurial-native have certificates
Ross Burton (1):
patchtest: remove unused imports
Rouven Czerwinski (1):
glib-2.0: Remove unnecessary assignement
Trevor Gamblin (16):
patchtest: test regardless of mergeability
patchtest: skip merge test if not targeting master
patchtest: fix lic_files_chksum test regex
patchtest-send-results: improve subject line
patchtest: disable merge test
patchtest-send-results: check max line length, simplify responses patchtest/selftest: add XSKIP, update test files
patchtest: simplify test directory structure
patchtest: reduce checksum test output length
patchtest: shorten test result outputs
patchtest-send-results: send results to submitter
patchtest-send-results: add In-Reply-To
patchtest: make pylint tests compatible with 3.x
patchtest: remove test for CVE tag in mbox
patchtest-send-results: fix sender parsing
patchtest: rework license checksum tests
Wang Mingyu (2):
ell: upgrade 0.58 -> 0.59
libsdl2: upgrade 2.28.3 -> 2.28.4
William Lyu (1):
perl: fix intermittent test failure
Xiangyu Chen (1):
linux-yocto: make sure the pahole-native available before
do_kernel_configme
luca fancellu (1):
oeqa/ssh: Handle SSHCall timeout error code
meta/classes-global/base.bbclass | 2 +-
meta/classes-recipe/bin_package.bbclass | 3 -
meta/classes-recipe/goarch.bbclass | 29 +-
meta/classes-recipe/kernel.bbclass | 2 +-
meta/lib/oe/__init__.py | 2 +-
meta/lib/oe/go.py | 34 +
meta/lib/oeqa/core/target/ssh.py | 17 +-
meta/lib/oeqa/runtime/cases/ssh.py | 3 +
meta/lib/oeqa/selftest/cases/devtool.py | 3 +
meta/lib/patchtest/data.py | 1 -
meta/lib/patchtest/repo.py | 1 -
...E.test_cve_presence_in_commit_message.fail | 72 --
...E.test_cve_presence_in_commit_message.pass | 74 --
...fail => TestMbox.test_author_valid.1.fail} | 0
...pass => TestMbox.test_author_valid.1.pass} | 0
...fail => TestMbox.test_author_valid.2.fail} | 0
...pass => TestMbox.test_author_valid.2.pass} | 0
... TestMbox.test_bugzilla_entry_format.fail} | 0
... TestMbox.test_bugzilla_entry_format.pass} | 0
...estMbox.test_commit_message_presence.fail} | 0
...estMbox.test_commit_message_presence.pass} | 0
....fail => TestMbox.test_mbox_format.1.fail} | 0
....fail => TestMbox.test_mbox_format.2.fail} | 0
...at.pass => TestMbox.test_mbox_format.pass} | 0
...TestMbox.test_series_merge_on_head.1.skip} | 0
...TestMbox.test_series_merge_on_head.2.skip} | 0
...ail => TestMbox.test_shortlog_format.fail} | 0
...ass => TestMbox.test_shortlog_format.pass} | 0
...ail => TestMbox.test_shortlog_length.fail} | 2 +-
...ass => TestMbox.test_shortlog_length.pass} | 0
...stMbox.test_signed_off_by_presence.1.fail} | 0
...stMbox.test_signed_off_by_presence.2.fail} | 0
...TestMbox.test_signed_off_by_presence.pass} | 0
..._files_chksum_modified_not_mentioned.fail} | 0
..._files_chksum_modified_not_mentioned.pass} | 0
...adata.test_lic_files_chksum_presence.fail} | 0
...adata.test_lic_files_chksum_presence.pass} | 0
...TestMetadata.test_src_uri_left_files.fail} | 0
...TestMetadata.test_src_uri_left_files.pass} | 0
...> TestMetadata.test_summary_presence.fail} | 0
...> TestMetadata.test_summary_presence.pass} | 0
...ail => TestPatch.test_cve_tag_format.fail} | 0
...ass => TestPatch.test_cve_tag_format.pass} | 0
...estPatch.test_signed_off_by_presence.fail} | 0
...estPatch.test_signed_off_by_presence.pass} | 0
meta/lib/patchtest/selftest/selftest | 11 +-
...{test_mbox_mailinglist.py => test_mbox.py} | 114 ++-
meta/lib/patchtest/tests/test_mbox_author.py | 29 -
.../lib/patchtest/tests/test_mbox_bugzilla.py | 20 -
meta/lib/patchtest/tests/test_mbox_cve.py | 39 -
.../patchtest/tests/test_mbox_description.py | 15 -
meta/lib/patchtest/tests/test_mbox_format.py | 14 -
meta/lib/patchtest/tests/test_mbox_merge.py | 24 -
.../lib/patchtest/tests/test_mbox_shortlog.py | 39 -
.../tests/test_mbox_signed_off_by.py | 27 -
meta/lib/patchtest/tests/test_metadata.py | 180 +++++
.../tests/test_metadata_lic_files_chksum.py | 80 --
.../patchtest/tests/test_metadata_license.py | 55 --
.../tests/test_metadata_max_length.py | 25 -
.../patchtest/tests/test_metadata_src_uri.py | 74 --
.../patchtest/tests/test_metadata_summary.py | 31 -
...patch_upstream_status.py => test_patch.py} | 56 +-
meta/lib/patchtest/tests/test_patch_cve.py | 38 -
.../tests/test_patch_signed_off_by.py | 41 -
.../lib/patchtest/tests/test_python_pylint.py | 19 +-
meta/lib/patchtest/utils.py | 1 -
.../{openssl_3.1.3.bb => openssl_3.1.4.bb} | 2 +-
.../ell/{ell_0.58.bb => ell_0.59.bb} | 2 +-
meta/recipes-core/glib-2.0/glib.inc | 5 -
meta/recipes-devtools/llvm/llvm_git.bb | 4 +-
...ent-failure-of-test-t-op-sigsystem.t.patch | 77 ++
meta/recipes-devtools/perl/perl_5.38.0.bb | 1 +
.../baremetal-helloworld_git.bb | 2 +-
.../{libsdl2_2.28.3.bb => libsdl2_2.28.4.bb} | 2 +-
meta/recipes-kernel/linux/linux-yocto.inc | 5 +-
.../libsndfile1/cve-2022-33065.patch | 739 ++++++++++++++++++
.../libsndfile/libsndfile1_1.2.2.bb | 1 +
scripts/contrib/bb-perf/bb-matrix-plot.sh | 4 +-
scripts/patchtest | 1 -
scripts/patchtest-get-branch | 1 -
scripts/patchtest-send-results | 70 +-
81 files changed, 1290 insertions(+), 803 deletions(-)
create mode 100644 meta/lib/oe/go.py
delete mode 100644 meta/lib/patchtest/selftest/files/CVE.test_cve_presence_in_commit_message.fail
delete mode 100644 meta/lib/patchtest/selftest/files/CVE.test_cve_presence_in_commit_message.pass
rename meta/lib/patchtest/selftest/files/{Author.test_author_valid.1.fail => TestMbox.test_author_valid.1.fail} (100%)
rename meta/lib/patchtest/selftest/files/{Author.test_author_valid.1.pass => TestMbox.test_author_valid.1.pass} (100%)
rename meta/lib/patchtest/selftest/files/{Author.test_author_valid.2.fail => TestMbox.test_author_valid.2.fail} (100%)
rename meta/lib/patchtest/selftest/files/{Author.test_author_valid.2.pass => TestMbox.test_author_valid.2.pass} (100%)
rename meta/lib/patchtest/selftest/files/{Bugzilla.test_bugzilla_entry_format.fail => TestMbox.test_bugzilla_entry_format.fail} (100%)
rename meta/lib/patchtest/selftest/files/{Bugzilla.test_bugzilla_entry_format.pass => TestMbox.test_bugzilla_entry_format.pass} (100%)
rename meta/lib/patchtest/selftest/files/{CommitMessage.test_commit_message_presence.fail => TestMbox.test_commit_message_presence.fail} (100%)
rename meta/lib/patchtest/selftest/files/{CommitMessage.test_commit_message_presence.pass => TestMbox.test_commit_message_presence.pass} (100%)
rename meta/lib/patchtest/selftest/files/{MboxFormat.test_mbox_format.1.fail => TestMbox.test_mbox_format.1.fail} (100%)
rename meta/lib/patchtest/selftest/files/{MboxFormat.test_mbox_format.2.fail => TestMbox.test_mbox_format.2.fail} (100%)
rename meta/lib/patchtest/selftest/files/{MboxFormat.test_mbox_format.pass => TestMbox.test_mbox_format.pass} (100%)
rename meta/lib/patchtest/selftest/files/{Merge.test_series_merge_on_head.pass => TestMbox.test_series_merge_on_head.1.skip} (100%)
rename meta/lib/patchtest/selftest/files/{Merge.test_series_merge_on_head.fail => TestMbox.test_series_merge_on_head.2.skip} (100%)
rename meta/lib/patchtest/selftest/files/{Shortlog.test_shortlog_format.fail => TestMbox.test_shortlog_format.fail} (100%)
rename meta/lib/patchtest/selftest/files/{CVE.test_cve_tag_format.pass => TestMbox.test_shortlog_format.pass} (100%)
rename meta/lib/patchtest/selftest/files/{Shortlog.test_shortlog_length.fail => TestMbox.test_shortlog_length.fail} (97%)
rename meta/lib/patchtest/selftest/files/{Shortlog.test_shortlog_format.pass => TestMbox.test_shortlog_length.pass} (100%)
rename meta/lib/patchtest/selftest/files/{SignedOffBy.test_signed_off_by_presence.1.fail => TestMbox.test_signed_off_by_presence.1.fail} (100%)
rename meta/lib/patchtest/selftest/files/{SignedOffBy.test_signed_off_by_presence.2.fail => TestMbox.test_signed_off_by_presence.2.fail} (100%)
rename meta/lib/patchtest/selftest/files/{PatchSignedOffBy.test_signed_off_by_presence.pass => TestMbox.test_signed_off_by_presence.pass} (100%)
rename meta/lib/patchtest/selftest/files/{LicFilesChkSum.test_lic_files_chksum_modified_not_mentioned.fail => TestMetadata.test_lic_files_chksum_modified_not_mentioned.fail} (100%)
rename meta/lib/patchtest/selftest/files/{LicFilesChkSum.test_lic_files_chksum_modified_not_mentioned.pass => TestMetadata.test_lic_files_chksum_modified_not_mentioned.pass} (100%)
rename meta/lib/patchtest/selftest/files/{LicFilesChkSum.test_lic_files_chksum_presence.fail => TestMetadata.test_lic_files_chksum_presence.fail} (100%)
rename meta/lib/patchtest/selftest/files/{LicFilesChkSum.test_lic_files_chksum_presence.pass => TestMetadata.test_lic_files_chksum_presence.pass} (100%)
rename meta/lib/patchtest/selftest/files/{SrcUri.test_src_uri_left_files.fail => TestMetadata.test_src_uri_left_files.fail} (100%)
rename meta/lib/patchtest/selftest/files/{SrcUri.test_src_uri_left_files.pass => TestMetadata.test_src_uri_left_files.pass} (100%)
rename meta/lib/patchtest/selftest/files/{Summary.test_summary_presence.fail => TestMetadata.test_summary_presence.fail} (100%)
rename meta/lib/patchtest/selftest/files/{Summary.test_summary_presence.pass => TestMetadata.test_summary_presence.pass} (100%)
rename meta/lib/patchtest/selftest/files/{CVE.test_cve_tag_format.fail => TestPatch.test_cve_tag_format.fail} (100%)
rename meta/lib/patchtest/selftest/files/{Shortlog.test_shortlog_length.pass => TestPatch.test_cve_tag_format.pass} (100%)
rename meta/lib/patchtest/selftest/files/{PatchSignedOffBy.test_signed_off_by_presence.fail => TestPatch.test_signed_off_by_presence.fail} (100%)
rename meta/lib/patchtest/selftest/files/{SignedOffBy.test_signed_off_by_presence.pass => TestPatch.test_signed_off_by_presence.pass} (100%)
rename meta/lib/patchtest/tests/{test_mbox_mailinglist.py => test_mbox.py} (40%)
delete mode 100644 meta/lib/patchtest/tests/test_mbox_author.py
delete mode 100644 meta/lib/patchtest/tests/test_mbox_bugzilla.py
delete mode 100644 meta/lib/patchtest/tests/test_mbox_cve.py
delete mode 100644 meta/lib/patchtest/tests/test_mbox_description.py
delete mode 100644 meta/lib/patchtest/tests/test_mbox_format.py
delete mode 100644 meta/lib/patchtest/tests/test_mbox_merge.py
delete mode 100644 meta/lib/patchtest/tests/test_mbox_shortlog.py
delete mode 100644 meta/lib/patchtest/tests/test_mbox_signed_off_by.py
create mode 100644 meta/lib/patchtest/tests/test_metadata.py
delete mode 100644 meta/lib/patchtest/tests/test_metadata_lic_files_chksum.py
delete mode 100644 meta/lib/patchtest/tests/test_metadata_license.py
delete mode 100644 meta/lib/patchtest/tests/test_metadata_max_length.py
delete mode 100644 meta/lib/patchtest/tests/test_metadata_src_uri.py
delete mode 100644 meta/lib/patchtest/tests/test_metadata_summary.py
rename meta/lib/patchtest/tests/{test_patch_upstream_status.py => test_patch.py} (55%)
delete mode 100644 meta/lib/patchtest/tests/test_patch_cve.py
delete mode 100644 meta/lib/patchtest/tests/test_patch_signed_off_by.py
rename meta/recipes-connectivity/openssl/{openssl_3.1.3.bb => openssl_3.1.4.bb} (99%)
rename meta/recipes-core/ell/{ell_0.58.bb => ell_0.59.bb} (89%)
create mode 100644 meta/recipes-devtools/perl/files/0001-Fix-intermittent-failure-of-test-t-op-sigsystem.t.patch
rename meta/recipes-graphics/libsdl2/{libsdl2_2.28.3.bb => libsdl2_2.28.4.bb} (97%)
create mode 100644 meta/recipes-multimedia/libsndfile/libsndfile1/cve-2022-33065.patchSignature · 1 line
2.34.1
PATCH 1/33 libsndfile1: fix CVE-2022-33065
From: Alex Stewart <[email protected]>
--- .../libsndfile1/cve-2022-33065.patch | 739 ++++++++++++++++++ .../libsndfile/libsndfile1_1.2.2.bb | 1 + 2 files changed, 740 insertions(+) create mode 100644 meta/recipes-multimedia/libsndfile/libsndfile1/cve-2022-33065.patch
meta/recipes-multimedia/libsndfile/libsndfile1/cve-2022-33065.patch new file +739 -0
@@ -0,0 +1,739 @@ +From c7ce5b0ebeeb58934825077d1324960aa0747718 Mon Sep 17 00:00:00 2001 +From: Alex Stewart <[email protected]> +Date: Tue, 10 Oct 2023 16:10:34 -0400 +Subject: [PATCH] mat4/mat5: fix int overflow in dataend calculation + +The clang sanitizer warns of a possible signed integer overflow when +calculating the `dataend` value in `mat4_read_header()`. + +``` +src/mat4.c:323:41: runtime error: signed integer overflow: 205 * -100663296 cannot be represented in type 'int' +SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior src/mat4.c:323:41 in +src/mat4.c:323:48: runtime error: signed integer overflow: 838860800 * 4 cannot be represented in type 'int' +SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior src/mat4.c:323:48 in +``` + +Cast the offending `rows` and `cols` ints to `sf_count_t` (the type of +`dataend` before performing the calculation, to avoid the issue. + +CVE: CVE-2022-33065 +Fixes: https://github.com/libsndfile/libsndfile/issues/789 +Fixes: https://github.com/libsndfile/libsndfile/issues/833 + +Upstream-Status: Backport [9a829113c88a51e57c1e46473e90609e4b7df151] + +Signed-off-by: Alex Stewart <[email protected]> +--- + src/mat4.c | 2 +- + 1 file changed, 1 insertion(+), 1 deletion(-) + +diff --git a/src/mat4.c b/src/mat4.c +index 0b1b414b..575683ba 100644 +--- a/src/mat4.c ++++ b/src/mat4.c +@@ -320,7 +320,7 @@ mat4_read_header (SF_PRIVATE *psf) + psf->filelength - psf->dataoffset, psf->sf.channels * psf->sf.frames * psf->bytewidth) ; + } + else if ((psf->filelength - psf->dataoffset) > psf->sf.channels * psf->sf.frames * psf->bytewidth) +- psf->dataend = psf->dataoffset + rows * cols * psf->bytewidth ; ++ psf->dataend = psf->dataoffset + (sf_count_t) rows * (sf_count_t) cols * psf->bytewidth ; + + psf->datalength = psf->filelength - psf->dataoffset - psf->dataend ; + +From 842303f984b2081481e74cb84a9a24ecbe3dec1a Mon Sep 17 00:00:00 2001 +From: Alex Stewart <[email protected]> +Date: Wed, 11 Oct 2023 16:36:02 -0400 +Subject: [PATCH] au: avoid int overflow while calculating data_end + +At several points in au_read_header(), we calculate the functional end +of the data segment by adding the (int)au_fmt.dataoffset and the +(int)au_fmt.datasize. This can overflow the implicit int_32 return value +and cause undefined behavior. + +Instead, precalculate the value and assign it to a 64-bit +(sf_count_t)data_end variable. + +CVE: CVE-2022-33065 +Fixes: https://github.com/libsndfile/libsndfile/issues/833 + +Signed-off-by: Alex Stewart <[email protected]> +--- + src/au.c | 10 ++++++---- + 1 file changed, 6 insertions(+), 4 deletions(-) + +diff --git a/src/au.c b/src/au.c +index 62bd691d..f68f2587 100644 +--- a/src/au.c ++++ b/src/au.c +@@ -291,6 +291,7 @@ static int + au_read_header (SF_PRIVATE *psf) + { AU_FMT au_fmt ; + int marker, dword ; ++ sf_count_t data_end ; + + memset (&au_fmt, 0, sizeof (au_fmt)) ; + psf_binheader_readf (psf, "pm", 0, &marker) ; +@@ -317,14 +318,15 @@ au_read_header (SF_PRIVATE *psf) + return SFE_AU_EMBED_BAD_LEN ; + } ; + ++ data_end = (sf_count_t) au_fmt.dataoffset + (sf_count_t) au_fmt.datasize ; + if (psf->fileoffset > 0) +- { psf->filelength = au_fmt.dataoffset + au_fmt.datasize ; ++ { psf->filelength = data_end ; + psf_log_printf (psf, " Data Size : %d\n", au_fmt.datasize) ; + } +- else if (au_fmt.datasize == -1 || au_fmt.dataoffset + au_fmt.datasize == psf->filelength) ++ else if (au_fmt.datasize == -1 || data_end == psf->filelength) + psf_log_printf (psf, " Data Size : %d\n", au_fmt.datasize) ; +- else if (au_fmt.dataoffset + au_fmt.datasize < psf->filelength) +- { psf->filelength = au_fmt.dataoffset + au_fmt.datasize ; ++ else if (data_end < psf->filelength) ++ { psf->filelength = data_end ; + psf_log_printf (psf, " Data Size : %d\n", au_fmt.datasize) ; + } + else +From 0754d3380a54e3fbdde0f684b88955c80c79f58f Mon Sep 17 00:00:00 2001 +From: Alex Stewart <[email protected]> +Date: Wed, 11 Oct 2023 16:46:29 -0400 +Subject: [PATCH] avr: fix int overflow in avr_read_header() + +Pre-cast hdr.frames to sf_count_t, to provide the calculation with +enough numeric space to avoid an int-overflow. + +CVE: CVE-2022-33065 +Fixes: https://github.com/libsndfile/libsndfile/issues/833 + +Signed-off-by: Alex Stewart <[email protected]> +--- + src/avr.c | 2 +- + 1 file changed, 1 insertion(+), 1 deletion(-) + +diff --git a/src/avr.c b/src/avr.c +index 6c78ff69..1bc1ffc9 100644 +--- a/src/avr.c ++++ b/src/avr.c +@@ -162,7 +162,7 @@ avr_read_header (SF_PRIVATE *psf) + psf->endian = SF_ENDIAN_BIG ; + + psf->dataoffset = AVR_HDR_SIZE ; +- psf->datalength = hdr.frames * (hdr.rez / 8) ; ++ psf->datalength = (sf_count_t) hdr.frames * (hdr.rez / 8) ; + + if (psf->fileoffset > 0) + psf->filelength = AVR_HDR_SIZE + psf->datalength ; +From 6ac31a68a614e2bba4a05b54e5558d6270c98376 Mon Sep 17 00:00:00 2001 +From: Alex Stewart <[email protected]> +Date: Wed, 11 Oct 2023 16:54:21 -0400 +Subject: [PATCH] sds: fix int overflow warning in sample calculations + +The sds_*byte_read() functions compose their uint_32 sample buffers by +shifting 7bit samples into a 32bit wide buffer, and adding them +together. Because the 7bit samples are stored in 32bit ints, code +fuzzers become concerned that the addition operation can overflow and +cause undefined behavior. + +Instead, bitwise-OR the bytes together - which should accomplish the +same arithmetic operation, without risking an int-overflow. + +CVE: CVE-2022-33065 +Fixes: https://github.com/libsndfile/libsndfile/issues/833 + +Signed-off-by: Alex Stewart <[email protected]> + +Do the same for the 3byte and 4byte read functions. +--- + src/sds.c | 6 +++--- + 1 file changed, 3 insertions(+), 3 deletions(-) + +diff --git a/src/sds.c b/src/sds.c +index 6bc76171..2a0f164c 100644 +--- a/src/sds.c ++++ b/src/sds.c +@@ -454,7 +454,7 @@ sds_2byte_read (SF_PRIVATE *psf, SDS_PRIVATE *psds) + + ucptr = psds->read_data + 5 ; + for (k = 0 ; k < 120 ; k += 2) +- { sample = arith_shift_left (ucptr [k], 25) + arith_shift_left (ucptr [k + 1], 18) ; ++ { sample = arith_shift_left (ucptr [k], 25) | arith_shift_left (ucptr [k + 1], 18) ; + psds->read_samples [k / 2] = (int) (sample - 0x80000000) ; + } ; + +@@ -498,7 +498,7 @@ sds_3byte_read (SF_PRIVATE *psf, SDS_PRIVATE *psds) + + ucptr = psds->read_data + 5 ; + for (k = 0 ; k < 120 ; k += 3) +- { sample = (((uint32_t) ucptr [k]) << 25) + (ucptr [k + 1] << 18) + (ucptr [k + 2] << 11) ; ++ { sample = (((uint32_t) ucptr [k]) << 25) | (ucptr [k + 1] << 18) | (ucptr [k + 2] << 11) ; + psds->read_samples [k / 3] = (int) (sample - 0x80000000) ; + } ; + +@@ -542,7 +542,7 @@ sds_4byte_read (SF_PRIVATE *psf, SDS_PRIVATE *psds) + + ucptr = psds->read_data + 5 ; + for (k = 0 ; k < 120 ; k += 4) +- { sample = (((uint32_t) ucptr [k]) << 25) + (ucptr [k + 1] << 18) + (ucptr [k + 2] << 11) + (ucptr [k + 3] << 4) ; ++ { sample = (((uint32_t) ucptr [k]) << 25) | (ucptr [k + 1] << 18) | (ucptr [k + 2] << 11) | (ucptr [k + 3] << 4) ; + psds->read_samples [k / 4] = (int) (sample - 0x80000000) ; + } ; + +From 96428e1dd4998f1cd47df24f8fe9b0da35d7b947 Mon Sep 17 00:00:00 2001 +From: Alex Stewart <[email protected]> +Date: Wed, 11 Oct 2023 17:26:51 -0400 +Subject: [PATCH] aiff: fix int overflow when counting header elements + +aiff_read_basc_chunk() tries to count the AIFF header size by keeping +track of the bytes returned by psf_binheader_readf(). Though improbable, +it is technically possible for these added bytes to exceed the int-sized +`count` accumulator. + +Use a 64-bit sf_count_t type for `count`, to ensure that it always has +enough numeric space. + +CVE: CVE-2022-33065 +Fixes: https://github.com/libsndfile/libsndfile/issues/833 + +Signed-off-by: Alex Stewart <[email protected]> +--- + src/aiff.c | 2 +- + 1 file changed, 1 insertion(+), 1 deletion(-) + +diff --git a/src/aiff.c b/src/aiff.c +index a2bda8f4..6b244302 100644 +--- a/src/aiff.c ++++ b/src/aiff.c +@@ -1702,7 +1702,7 @@ static int + aiff_read_basc_chunk (SF_PRIVATE * psf, int datasize) + { const char * type_str ; + basc_CHUNK bc ; +- int count ; ++ sf_count_t count ; + + count = psf_binheader_readf (psf, "E442", &bc.version, &bc.numBeats, &bc.rootNote) ; + count += psf_binheader_readf (psf, "E222", &bc.scaleType, &bc.sigNumerator, &bc.sigDenominator) ; +From b352c350d35bf978e4d3a32e5d9df1f2284445f4 Mon Sep 17 00:00:00 2001 +From: Alex Stewart <[email protected]> +Date: Wed, 11 Oct 2023 17:43:02 -0400 +Subject: [PATCH] ircam: fix int overflow in ircam_read_header() + +When reading the IRCAM header, it is possible for the calculated +blockwidth to exceed the bounds of a signed int32. + +Use a 64bit sf_count_t to store the blockwidth. + +CVE: CVE-2022-33065 +Fixes: https://github.com/libsndfile/libsndfile/issues/833 + +Signed-off-by: Alex Stewart <[email protected]> +--- + src/common.h | 2 +- + src/ircam.c | 10 +++++----- + 2 files changed, 6 insertions(+), 6 deletions(-) + +diff --git a/src/common.h b/src/common.h +index d92eabde..5369cb67 100644 +--- a/src/common.h ++++ b/src/common.h +@@ -439,7 +439,7 @@ typedef struct sf_private_tag + sf_count_t datalength ; /* Length in bytes of the audio data. */ + sf_count_t dataend ; /* Offset to file tailer. */ + +- int blockwidth ; /* Size in bytes of one set of interleaved samples. */ ++ sf_count_t blockwidth ; /* Size in bytes of one set of interleaved samples. */ + int bytewidth ; /* Size in bytes of one sample (one channel). */ + + void *dither ; +diff --git a/src/ircam.c b/src/ircam.c +index 8e7cdba8..3d73ba44 100644 +--- a/src/ircam.c ++++ b/src/ircam.c +@@ -171,35 +171,35 @@ ircam_read_header (SF_PRIVATE *psf) + switch (encoding) + { case IRCAM_PCM_16 : + psf->bytewidth = 2 ; +- psf->blockwidth = psf->sf.channels * psf->bytewidth ; ++ psf->blockwidth = (sf_count_t) psf->sf.channels * psf->bytewidth ; + + psf->sf.format = SF_FORMAT_IRCAM | SF_FORMAT_PCM_16 ; + break ; + + case IRCAM_PCM_32 : + psf->bytewidth = 4 ; +- psf->blockwidth = psf->sf.channels * psf->bytewidth ; ++ psf->blockwidth = (sf_count_t) psf->sf.channels * psf->bytewidth ; + + psf->sf.format = SF_FORMAT_IRCAM | SF_FORMAT_PCM_32 ; + break ; + + case IRCAM_FLOAT : + psf->bytewidth = 4 ; +- psf->blockwidth = psf->sf.channels * psf->bytewidth ; ++ psf->blockwidth = (sf_count_t) psf->sf.channels * psf->bytewidth ; + + psf->sf.format = SF_FORMAT_IRCAM | SF_FORMAT_FLOAT ; + break ; + + case IRCAM_ALAW : + psf->bytewidth = 1 ; +- psf->blockwidth = psf->sf.channels * psf->bytewidth ; ++ psf->blockwidth = (sf_count_t) psf->sf.channels * psf->bytewidth ; + + psf->sf.format = SF_FORMAT_IRCAM | SF_FORMAT_ALAW ; + break ; + + case IRCAM_ULAW : + psf->bytewidth = 1 ; +- psf->blockwidth = psf->sf.channels * psf->bytewidth ; ++ psf->blockwidth = (sf_count_t) psf->sf.channels * psf->bytewidth ; + + psf->sf.format = SF_FORMAT_IRCAM | SF_FORMAT_ULAW ; + break ; +From 3bcd291e57867f88f558fa6f80990e84311df78c Mon Sep 17 00:00:00 2001 +From: Alex Stewart <[email protected]> +Date: Wed, 11 Oct 2023 16:12:22 -0400 +Subject: [PATCH] mat4/mat5: fix int overflow when calculating blockwidth + +Pre-cast the components of the blockwidth calculation to sf_count_t to +avoid overflowing integers during calculation. + +CVE: CVE-2022-33065 +Fixes: https://github.com/libsndfile/libsndfile/issues/833 + +Signed-off-by: Alex Stewart <[email protected]> +--- + src/mat4.c | 2 +- + src/mat5.c | 2 +- + 2 files changed, 2 insertions(+), 2 deletions(-) + +diff --git a/src/mat4.c b/src/mat4.c +index 575683ba..9f046f0c 100644 +--- a/src/mat4.c ++++ b/src/mat4.c +@@ -104,7 +104,7 @@ mat4_open (SF_PRIVATE *psf) + + psf->container_close = mat4_close ; + +- psf->blockwidth = psf->bytewidth * psf->sf.channels ; ++ psf->blockwidth = (sf_count_t) psf->bytewidth * psf->sf.channels ; + + switch (subformat) + { case SF_FORMAT_PCM_16 : +diff --git a/src/mat5.c b/src/mat5.c +index da5a6eca..20f0ea64 100644 +--- a/src/mat5.c ++++ b/src/mat5.c +@@ -114,7 +114,7 @@ mat5_open (SF_PRIVATE *psf) + + psf->container_close = mat5_close ; + +- psf->blockwidth = psf->bytewidth * psf->sf.channels ; ++ psf->blockwidth = (sf_count_t) psf->bytewidth * psf->sf.channels ; + + switch (subformat) + { case SF_FORMAT_PCM_U8 : +From c177e292d47ef73b1d3c1bb391320299a0ed2ff9 Mon Sep 17 00:00:00 2001 +From: Alex Stewart <[email protected]> +Date: Mon, 16 Oct 2023 12:37:47 -0400 +Subject: [PATCH] common: fix int overflow in psf_binheader_readf() + +The psf_binheader_readf() function attempts to count and return the +number of bytes traversed in the header. During this accumulation, it is +possible to overflow the int-sized byte_count variable. + +Avoid this overflow by checking that the accumulated bytes do not exceed +INT_MAX and throwing an error if they do. This implies that files with +multi-gigabyte headers threaten to produce this error, but I imagine +those files don't really exist - and this error is better than the +undefined behavior which would have resulted previously. + +CVE: CVE-2022-33065 +Fixes: https://github.com/libsndfile/libsndfile/issues/833 + +Signed-off-by: Alex Stewart <[email protected]> +--- + src/common.c | 36 ++++++++++++++++++++++++------------ + 1 file changed, 24 insertions(+), 12 deletions(-) + +diff --git a/src/common.c b/src/common.c +index 1c3d951d..7f6cceca 100644 +--- a/src/common.c ++++ b/src/common.c +@@ -18,6 +18,7 @@ + + #include <config.h> + ++#include <limits.h> + #include <stdarg.h> + #include <string.h> + #if HAVE_UNISTD_H +@@ -990,6 +991,7 @@ psf_binheader_readf (SF_PRIVATE *psf, char const *format, ...) + double *doubleptr ; + char c ; + int byte_count = 0, count = 0 ; ++ int read_bytes = 0 ; + + if (! format) + return psf_ftell (psf) ; +@@ -998,6 +1000,7 @@ psf_binheader_readf (SF_PRIVATE *psf, char const *format, ...) + + while ((c = *format++)) + { ++ read_bytes = 0 ; + if (psf->header.indx + 16 >= psf->header.len && psf_bump_header_allocation (psf, 16)) + break ; + +@@ -1014,7 +1017,7 @@ psf_binheader_readf (SF_PRIVATE *psf, char const *format, ...) + intptr = va_arg (argptr, unsigned int*) ; + *intptr = 0 ; + ucptr = (unsigned char*) intptr ; +- byte_count += header_read (psf, ucptr, sizeof (int)) ; ++ read_bytes = header_read (psf, ucptr, sizeof (int)) ; + *intptr = GET_MARKER (ucptr) ; + break ; + +@@ -1022,7 +1025,7 @@ psf_binheader_readf (SF_PRIVATE *psf, char const *format, ...) + intptr = va_arg (argptr, unsigned int*) ; + *intptr = 0 ; + ucptr = (unsigned char*) intptr ; +- byte_count += header_read (psf, sixteen_bytes, sizeof (sixteen_bytes)) ; ++ read_bytes = header_read (psf, sixteen_bytes, sizeof (sixteen_bytes)) ; + { int k ; + intdata = 0 ; + for (k = 0 ; k < 16 ; k++) +@@ -1034,14 +1037,14 @@ psf_binheader_readf (SF_PRIVATE *psf, char const *format, ...) + case '1' : + charptr = va_arg (argptr, char*) ; + *charptr = 0 ; +- byte_count += header_read (psf, charptr, sizeof (char)) ; ++ read_bytes = header_read (psf, charptr, sizeof (char)) ; + break ; + + case '2' : /* 2 byte value with the current endian-ness */ + shortptr = va_arg (argptr, unsigned short*) ; + *shortptr = 0 ; + ucptr = (unsigned char*) shortptr ; +- byte_count += header_read (psf, ucptr, sizeof (short)) ; ++ read_bytes = header_read (psf, ucptr, sizeof (short)) ; + if (psf->rwf_endian == SF_ENDIAN_BIG) + *shortptr = GET_BE_SHORT (ucptr) ; + else +@@ -1051,7 +1054,7 @@ psf_binheader_readf (SF_PRIVATE *psf, char const *format, ...) + case '3' : /* 3 byte value with the current endian-ness */ + intptr = va_arg (argptr, unsigned int*) ; + *intptr = 0 ; +- byte_count += header_read (psf, sixteen_bytes, 3) ; ++ read_bytes = header_read (psf, sixteen_bytes, 3) ; + if (psf->rwf_endian == SF_ENDIAN_BIG) + *intptr = GET_BE_3BYTE (sixteen_bytes) ; + else +@@ -1062,7 +1065,7 @@ psf_binheader_readf (SF_PRIVATE *psf, char const *format, ...) + intptr = va_arg (argptr, unsigned int*) ; + *intptr = 0 ; + ucptr = (unsigned char*) intptr ; +- byte_count += header_read (psf, ucptr, sizeof (int)) ; ++ read_bytes = header_read (psf, ucptr, sizeof (int)) ; + if (psf->rwf_endian == SF_ENDIAN_BIG) + *intptr = psf_get_be32 (ucptr, 0) ; + else +@@ -1072,7 +1075,7 @@ psf_binheader_readf (SF_PRIVATE *psf, char const *format, ...) + case '8' : /* 8 byte value with the current endian-ness */ + countptr = va_arg (argptr, sf_count_t *) ; + *countptr = 0 ; +- byte_count += header_read (psf, sixteen_bytes, 8) ; ++ read_bytes = header_read (psf, sixteen_bytes, 8) ; + if (psf->rwf_endian == SF_ENDIAN_BIG) + countdata = psf_get_be64 (sixteen_bytes, 0) ; + else +@@ -1083,7 +1086,7 @@ psf_binheader_readf (SF_PRIVATE *psf, char const *format, ...) + case 'f' : /* Float conversion */ + floatptr = va_arg (argptr, float *) ; + *floatptr = 0.0 ; +- byte_count += header_read (psf, floatptr, sizeof (float)) ; ++ read_bytes = header_read (psf, floatptr, sizeof (float)) ; + if (psf->rwf_endian == SF_ENDIAN_BIG) + *floatptr = float32_be_read ((unsigned char*) floatptr) ; + else +@@ -1093,7 +1096,7 @@ psf_binheader_readf (SF_PRIVATE *psf, char const *format, ...) + case 'd' : /* double conversion */ + doubleptr = va_arg (argptr, double *) ; + *doubleptr = 0.0 ; +- byte_count += header_read (psf, doubleptr, sizeof (double)) ; ++ read_bytes = header_read (psf, doubleptr, sizeof (double)) ; + if (psf->rwf_endian == SF_ENDIAN_BIG) + *doubleptr = double64_be_read ((unsigned char*) doubleptr) ; + else +@@ -1117,7 +1120,7 @@ psf_binheader_readf (SF_PRIVATE *psf, char const *format, ...) + charptr = va_arg (argptr, char*) ; + count = va_arg (argptr, size_t) ; + memset (charptr, 0, count) ; +- byte_count += header_read (psf, charptr, count) ; ++ read_bytes = header_read (psf, charptr, count) ; + break ; + + case 'G' : +@@ -1128,7 +1131,7 @@ psf_binheader_readf (SF_PRIVATE *psf, char const *format, ...) + if (psf->header.indx + count >= psf->header.len && psf_bump_header_allocation (psf, count)) + break ; + +- byte_count += header_gets (psf, charptr, count) ; ++ read_bytes = header_gets (psf, charptr, count) ; + break ; + + case 'z' : +@@ -1152,7 +1155,7 @@ psf_binheader_readf (SF_PRIVATE *psf, char const *format, ...) + case 'j' : /* Seek to position from current position. */ + count = va_arg (argptr, size_t) ; + header_seek (psf, count, SEEK_CUR) ; +- byte_count += count ; ++ read_bytes = count ; + break ; + + case '!' : /* Clear buffer, forcing re-read. */ +@@ -1164,8 +1167,17 @@ psf_binheader_readf (SF_PRIVATE *psf, char const *format, ...) + psf->error = SFE_INTERNAL ; + break ; + } ; ++ ++ if (read_bytes > 0 && byte_count > (INT_MAX - read_bytes)) ++ { psf_log_printf (psf, "Header size exceeds INT_MAX. Aborting.", c) ; ++ psf->error = SFE_INTERNAL ; ++ break ; ++ } else ++ { byte_count += read_bytes ; + } ; + ++ } ; /*end while*/ ++ + va_end (argptr) ; + + return byte_count ; +From a23d563386e7c8d93dcdbe7d5b1d63cad6009116 Mon Sep 17 00:00:00 2001 +From: Alex Stewart <[email protected]> +Date: Thu, 19 Oct 2023 14:07:19 -0400 +Subject: [PATCH] nms_adpcm: fix int overflow in signal estimate + +It is possible (though functionally incorrect) for the signal estimate +calculation in nms_adpcm_update() to overflow the int value of s_e, +resulting in undefined behavior. + +Since adpcm state signal values are never practically larger than +16 bits, use smaller numeric sizes throughout the file to avoid the +overflow. + +CVE: CVE-2022-33065 +Fixes: https://github.com/libsndfile/libsndfile/issues/833 + +Authored-by: Arthur Taylor <[email protected]> +Signed-off-by: Alex Stewart <[email protected]> +Rebased-by: Alex Stewart <[email protected]> +--- + src/nms_adpcm.c | 85 ++++++++++++++++++++++++------------------------- + 1 file changed, 42 insertions(+), 43 deletions(-) + +diff --git a/src/nms_adpcm.c b/src/nms_adpcm.c +index 96d6ad26..460ea077 100644 +--- a/src/nms_adpcm.c ++++ b/src/nms_adpcm.c +@@ -48,36 +48,36 @@ + /* Variable names from ITU G.726 spec */ + struct nms_adpcm_state + { /* Log of the step size multiplier. Operated on by codewords. */ +- int yl ; ++ short yl ; + + /* Quantizer step size multiplier. Generated from yl. */ +- int y ; ++ short y ; + +- /* Coefficents of the pole predictor */ +- int a [2] ; ++ /* Coefficients of the pole predictor */ ++ short a [2] ; + +- /* Coefficents of the zero predictor */ +- int b [6] ; ++ /* Coefficients of the zero predictor */ ++ short b [6] ; + + /* Previous quantized deltas (multiplied by 2^14) */ +- int d_q [7] ; ++ short d_q [7] ; + + /* d_q [x] + s_ez [x], used by the pole-predictor for signs only. */ +- int p [3] ; ++ short p [3] ; + + /* Previous reconstructed signal values. */ +- int s_r [2] ; ++ short s_r [2] ; + + /* Zero predictor components of the signal estimate. */ +- int s_ez ; ++ short s_ez ; + + /* Signal estimate, (including s_ez). */ +- int s_e ; ++ short s_e ; + + /* The most recent codeword (enc:generated, dec:inputted) */ +- int Ik ; ++ char Ik ; + +- int parity ; ++ char parity ; + + /* + ** Offset into code tables for the bitrate. +@@ -109,7 +109,7 @@ typedef struct + } NMS_ADPCM_PRIVATE ; + + /* Pre-computed exponential interval used in the antilog approximation. */ +-static unsigned int table_expn [] = ++static unsigned short table_expn [] = + { 0x4000, 0x4167, 0x42d5, 0x444c, 0x45cb, 0x4752, 0x48e2, 0x4a7a, + 0x4c1b, 0x4dc7, 0x4f7a, 0x5138, 0x52ff, 0x54d1, 0x56ac, 0x5892, + 0x5a82, 0x5c7e, 0x5e84, 0x6096, 0x62b4, 0x64dd, 0x6712, 0x6954, +@@ -117,21 +117,21 @@ static unsigned int table_expn [] = + } ; + + /* Table mapping codewords to scale factor deltas. */ +-static int table_scale_factor_step [] = ++static short table_scale_factor_step [] = + { 0x0, 0x0, 0x0, 0x0, 0x4b0, 0x0, 0x0, 0x0, /* 2-bit */ + -0x3c, 0x0, 0x90, 0x0, 0x2ee, 0x0, 0x898, 0x0, /* 3-bit */ + -0x30, 0x12, 0x6b, 0xc8, 0x188, 0x2e0, 0x551, 0x1150, /* 4-bit */ + } ; + + /* Table mapping codewords to quantized delta interval steps. */ +-static unsigned int table_step [] = ++static unsigned short table_step [] = + { 0x73F, 0, 0, 0, 0x1829, 0, 0, 0, /* 2-bit */ + 0x3EB, 0, 0xC18, 0, 0x1581, 0, 0x226E, 0, /* 3-bit */ + 0x20C, 0x635, 0xA83, 0xF12, 0x1418, 0x19E3, 0x211A, 0x2BBA, /* 4-bit */ + } ; + + /* Binary search lookup table for quantizing using table_step. */ +-static int table_step_search [] = ++static short table_step_search [] = + { 0, 0x1F6D, 0, -0x1F6D, 0, 0, 0, 0, /* 2-bit */ + 0x1008, 0x1192, 0, -0x219A, 0x1656, -0x1656, 0, 0, /* 3-bit */ + 0x872, 0x1277, -0x8E6, -0x232B, 0xD06, -0x17D7, -0x11D3, 0, /* 4-bit */ +@@ -179,23 +179,23 @@ static sf_count_t nms_adpcm_seek (SF_PRIVATE *psf, int mode, sf_count_t offset) + ** Maps [1,20480] to [1,1024] in an exponential relationship. This is + ** approximately ret = b^exp where b = e^(ln(1024)/ln(20480)) ~= 1.0003385 + */ +-static inline int +-nms_adpcm_antilog (int exp) +-{ int ret ; ++static inline short ++nms_adpcm_antilog (short exp) ++{ int_fast32_t r ; + +- ret = 0x1000 ; +- ret += (((exp & 0x3f) * 0x166b) >> 12) ; +- ret *= table_expn [(exp & 0x7c0) >> 6] ; +- ret >>= (26 - (exp >> 11)) ; ++ r = 0x1000 ; ++ r += (((int_fast32_t) (exp & 0x3f) * 0x166b) >> 12) ; ++ r *= table_expn [(exp & 0x7c0) >> 6] ; ++ r >>= (26 - (exp >> 11)) ; + +- return ret ; ++ return (short) r ; + } /* nms_adpcm_antilog */ + + static void + nms_adpcm_update (struct nms_adpcm_state *s) + { /* Variable names from ITU G.726 spec */ +- int a1ul ; +- int fa1 ; ++ short a1ul, fa1 ; ++ int_fast32_t se ; + int i ; + + /* Decay and Modify the scale factor in the log domain based on the codeword. */ +@@ -222,7 +222,7 @@ nms_adpcm_update (struct nms_adpcm_state *s) + else if (fa1 > 256) + fa1 = 256 ; + +- s->a [0] = (0xff * s->a [0]) >> 8 ; ++ s->a [0] = (s->a [0] * 0xff) >> 8 ; + if (s->p [0] != 0 && s->p [1] != 0 && ((s->p [0] ^ s->p [1]) < 0)) + s->a [0] -= 192 ; + else +@@ -230,7 +230,7 @@ nms_adpcm_update (struct nms_adpcm_state *s) + fa1 = -fa1 ; + } + +- s->a [1] = fa1 + ((0xfe * s->a [1]) >> 8) ; ++ s->a [1] = fa1 + ((s->a [1] * 0xfe) >> 8) ; + if (s->p [0] != 0 && s->p [2] != 0 && ((s->p [0] ^ s->p [2]) < 0)) + s->a [1] -= 128 ; + else +@@ -250,19 +250,18 @@ nms_adpcm_update (struct nms_adpcm_state *s) + s->a [0] = a1ul ; + } ; + +- /* Compute the zero predictor estimate. Rotate past deltas too. */ +- s->s_ez = 0 ; ++ /* Compute the zero predictor estimate and rotate past deltas. */ ++ se = 0 ; + for (i = 5 ; i >= 0 ; i--) +- { s->s_ez += s->d_q [i] * s->b [i] ; ++ { se += (int_fast32_t) s->d_q [i] * s->b [i] ; + s->d_q [i + 1] = s->d_q [i] ; + } ; ++ s->s_ez = se >> 14 ; + +- /* Compute the signal estimate. */ +- s->s_e = s->a [0] * s->s_r [0] + s->a [1] * s->s_r [1] + s->s_ez ; +- +- /* Return to scale */ +- s->s_ez >>= 14 ; +- s->s_e >>= 14 ; ++ /* Complete the signal estimate. */ ++ se += (int_fast32_t) s->a [0] * s->s_r [0] ; ++ se += (int_fast32_t) s->a [1] * s->s_r [1] ; ++ s->s_e = se >> 14 ; + + /* Rotate members to prepare for next iteration. */ + s->s_r [1] = s->s_r [0] ; +@@ -274,7 +273,7 @@ nms_adpcm_update (struct nms_adpcm_state *s) + static int16_t + nms_adpcm_reconstruct_sample (struct nms_adpcm_state *s, uint8_t I) + { /* Variable names from ITU G.726 spec */ +- int dqx ; ++ int_fast32_t dqx ; + + /* + ** The ordering of the 12-bit right-shift is a precision loss. It agrees +@@ -308,17 +307,17 @@ nms_adpcm_codec_init (struct nms_adpcm_state *s, enum nms_enc_type type) + /* + ** nms_adpcm_encode_sample() + ** +-** Encode a linear 16-bit pcm sample into a 2,3, or 4 bit NMS-ADPCM codeword ++** Encode a linear 16-bit pcm sample into a 2, 3, or 4 bit NMS-ADPCM codeword + ** using and updating the predictor state. + */ + static uint8_t + nms_adpcm_encode_sample (struct nms_adpcm_state *s, int16_t sl) + { /* Variable names from ITU G.726 spec */ +- int d ; ++ int_fast32_t d ; + uint8_t I ; + + /* Down scale the sample from 16 => ~14 bits. */ +- sl = (sl * 0x1fdf) / 0x7fff ; ++ sl = ((int_fast32_t) sl * 0x1fdf) / 0x7fff ; + + /* Compute estimate, and delta from actual value */ + nms_adpcm_update (s) ; +@@ -407,7 +406,7 @@ nms_adpcm_encode_sample (struct nms_adpcm_state *s, int16_t sl) + */ + static int16_t + nms_adpcm_decode_sample (struct nms_adpcm_state *s, uint8_t I) +-{ int sl ; ++{ int_fast32_t sl ; + + nms_adpcm_update (s) ; + sl = nms_adpcm_reconstruct_sample (s, I) ;
meta/recipes-multimedia/libsndfile/libsndfile1_1.2.2.bb +1 -0
@@ -9,6 +9,7 @@ LICENSE = "LGPL-2.1-only" SRC_URI = "${GITHUB_BASE_URI}/download/${PV}/libsndfile-${PV}.tar.xz \ file://noopus.patch \ + file://cve-2022-33065.patch \ " GITHUB_BASE_URI = "https://github.com/libsndfile/libsndfile/releases/"
Signature · 1 line
2.34.1
PATCH 2/33 ell: upgrade 0.58 -> 0.59
In reply to Steve SakomanFrom: Wang Mingyu <[email protected]>
Changelog:
Fix issue with symbol visibility.
---
meta/recipes-core/ell/{ell_0.58.bb => ell_0.59.bb} | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
rename meta/recipes-core/ell/{ell_0.58.bb => ell_0.59.bb} (89%)meta/recipes-core/ell/ell_0.59.bb renamed from meta/recipes-core/ell/ell_0.58.bb +1 -1
@@ -15,7 +15,7 @@ DEPENDS = "dbus" inherit autotools pkgconfig SRC_URI = "https://mirrors.edge.kernel.org/pub/linux/libs/${BPN}/${BPN}-${PV}.tar.xz" -SRC_URI[sha256sum] = "531a980589c8954ff12a3110b4d958fa75a74c88ddcc3e2ace4317e76a7c1e9b" +SRC_URI[sha256sum] = "370dc2b7c73cd57856017180a2a70a15ca1b0183bfd453b3cffe2d707c37da3d" do_configure:prepend () { mkdir -p ${S}/build-aux
Signature · 1 line
2.34.1
PATCH 3/33 libsdl2: upgrade 2.28.3 -> 2.28.4
In reply to Steve SakomanFrom: Wang Mingyu <[email protected]>
This is a stable bugfix release, with the following changes:
Enable clipping for zero sized rectangles in the SDL renderer Notify X11 clipboard managers when the clipboard changes Fixed sensor timestamps for third-party PS5 controllers Added detection for Logitech and Simagic racing wheels
---
.../libsdl2/{libsdl2_2.28.3.bb => libsdl2_2.28.4.bb} | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
rename meta/recipes-graphics/libsdl2/{libsdl2_2.28.3.bb => libsdl2_2.28.4.bb} (97%)meta/recipes-graphics/libsdl2/libsdl2_2.28.4.bb renamed from meta/recipes-graphics/libsdl2/libsdl2_2.28.3.bb +1 -1
@@ -25,7 +25,7 @@ SRC_URI = "http://www.libsdl.org/release/SDL2-${PV}.tar.gz" S = "${WORKDIR}/SDL2-${PV}" -SRC_URI[sha256sum] = "7acb8679652701a2504d734e2ba7543ec1a83e310498ddd22fd44bf965eb5518" +SRC_URI[sha256sum] = "888b8c39f36ae2035d023d1b14ab0191eb1d26403c3cf4d4d5ede30e66a4942c" inherit cmake lib_package binconfig-disabled pkgconfig upstream-version-is-even
Signature · 1 line
2.34.1
PATCH 4/33 llvm: Upgrade to 17.0.3
In reply to Steve SakomanFrom: Khem Raj <[email protected]>
Brings following fixes
* 888437e1b600 [asan] Ensure __asan_register_elf_globals is called in COMDAT asan.module_ctor (#67745)
* 2e00f4ca4e91 [clang-format][doc] Update the Linux kernel coding style URL
* aeb83c3783a6 [clang-format] Fix a serious bug in git-clang-format (#65723)
* 268faa377aee [LSan] Mark create_thread_leak.cpp as UNSUPPORTED: darwin.
* 491a91e8eea2 [PowerPC] Use zext instead of anyext in custom and combine (#68784)
* 8ce6b65c89ad [PowerPC] Add test for #68783 (NFC)
* 7a23a5d43c67 [clang-format] Fix a bug in RemoveParentheses: ReturnStatement (#67911)
* be4016e52779 [X86] Fix logic for optimizing movmsk(bitcast(shuffle(x))); PR67287
* 496b174053bd [X86] Add tests for incorrectly optimizing out shuffle used in movmsk; PR67287
* f50c6382c716 [clang] [MinGW] Explicitly always pass the -fno-use-init-array (#68571)
* d10b731adcc8 [LVI][CVP] Treat undef like a full range (#68190)
* 37b79e779f44 [X86] combineConcatVectorOps - only concatenate single-use subops
* 5a13ce2d6020 Bump version to 17.0.3
* e7b3b94cf500 [clang] Correct behavior of LLVM_UNREACHABLE_OPTIMIZE=OFF for Release builds (#68284)
* f0a687d821c1 [LLD] [COFF] Fix handling of comdat .drectve sections (#68116)
* 8a8ade49ff49 workflows/release-binaries: Use more cores to avoid the 6 hour timeout (#67874)
* 1090b91a2840 [AArch64] Disable loop alignment for Windows targets (#67894)
* 69c8c96691c7 [Sema] Use underlying type of scoped enum for -Wformat diagnostics (#67378)
* b2417f51dbbd (tag: llvmorg-17.0.2) Fix release/export.sh to export runtimes tarball, too (#67404)
* 23988a1d82d5 [libc++] Fix std::pair's pair-like constructor's incorrect assumption (#66585)
* 33e14ecd6aac [CodeGen] Don't treat thread local globals as large data (#67764)
* 03f797b51df6 [workflow] Fix abi checker in llvm-tests. Same fix as in 99fb0af80d16b0ff886f032441392219e1cac452 (#67957)
* f6cf58eed973 [clang] [MinGW] Tolerate mingw specific linker options during compilation (#67891)
* b338a2830a2c [LLD] [COFF] Restore the current dir as the first entry in the search path (#67857)
* 6a5be8e95b43 [LLD] [COFF] Clarify -print-search-path for the empty string element (#67856)
* 71be0aafe357 [NFC] clang-format lld/COFF/Driver.cpp and lld/Common/Filesystem.cpp
* 0a2d7dae6ef2 [compiler-rt] Reinstate removal of CRT choice flags from CMAKE_*_FLAGS* (#67935)
* 098e653a5bed [MemCpyOpt] Merge alias metadatas when replacing arguments (#67539)
* 78d201ebc3e2 [MemCpyOpt] Add test for #67539 (NFC)
* e718f3240a57 [DependencyScanningFilesystem] Make sure the local/shared cache filename lookups use only absolute paths (#66122)
* 45066b9fbc7b [Sema] Fix fixit cast printing inside macros (#66853)
* 87ec1f460d0e Work around two more instances of __noinline__ conflicts. (#66138)
* 9da5b7a93bca [lldb] Fix building LLDB standlone without framework
* c056d720b534 [lldb][NFCI] Change logic to find clang resource dir in standalone builds
* cb23434f9e63 [XCOFF] Do not generate the special .ref for zero-length sections (#66805)
* 1b55dc9d94c3 Fix buildbot failure caused by D157623
* 28d81a2bfa0a [lld][COFF] Remove incorrect flag from EHcont table
* b7eba056b93c workflows/release-tasks: Setup FileCheck and not for release-lit (#66799)
* 9678f11b057c [StackColoring] Handle fixed object index
* 49e9ee190080 [StackColoring] Handle SEH catch object stack slots conservatively
* 17123a60b87c [X86] Add test for #66984 (NFC)
* 2839aa915066 [SimpleLoopUnswitch] Fix exponential unswitch
* 773f136d6faa [SimpleLoopUnswitch] Fix reversed branch during condition injection
* 4362f3e4cf48 [clang] Include expected-no-diagnostics in newly-added test (NFC)
* 5f1fcc43e592 [clang] Bail out when handling union access with virtual inheritance
* 178cf5bc8732 [clang][Diagnostics] Fix wrong line number display (#65238)
* 25a150b830f6 Revert "[InlineCost] Check for conflicting target attributes early"
--- meta/recipes-devtools/llvm/llvm_git.bb | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
meta/recipes-devtools/llvm/llvm_git.bb +2 -2
@@ -14,14 +14,14 @@ RDEPENDS:${PN}:append:class-target = " ncurses-terminfo" inherit cmake pkgconfig -PV = "17.0.2" +PV = "17.0.3" MAJOR_VERSION = "${@oe.utils.trim_version("${PV}", 1)}" LLVM_RELEASE = "${PV}" BRANCH = "release/${MAJOR_VERSION}.x" -SRCREV = "481358974fb0f732e33d503c224492a543f4d7bd" +SRCREV = "888437e1b60011b8a375dd30928ec925b448da57" SRC_URI = "git://github.com/llvm/llvm-project.git;branch=${BRANCH};protocol=https \ file://0007-llvm-allow-env-override-of-exe-path.patch;striplevel=2 \ file://0001-AsmMatcherEmitter-sort-ClassInfo-lists-by-name-as-we.patch;striplevel=2 \
Signature · 1 line
2.34.1
PATCH 5/33 openssl: Upgrade 3.1.3 -> 3.1.4
In reply to Steve SakomanFrom: Peter Marko <[email protected]>
Major changes between OpenSSL 3.1.3 and OpenSSL 3.1.4 [24 Oct 2023]
* Mitigate incorrect resize handling for symmetric cipher keys and IVs. (CVE-2023-5363)
---
.../openssl/{openssl_3.1.3.bb => openssl_3.1.4.bb} | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
rename meta/recipes-connectivity/openssl/{openssl_3.1.3.bb => openssl_3.1.4.bb} (99%)meta/recipes-connectivity/openssl/openssl_3.1.4.bb renamed from meta/recipes-connectivity/openssl/openssl_3.1.3.bb +1 -1
@@ -18,7 +18,7 @@ SRC_URI:append:class-nativesdk = " \ file://environment.d-openssl.sh \ " -SRC_URI[sha256sum] = "f0316a2ebd89e7f2352976445458689f80302093788c466692fb2a188b2eacf6" +SRC_URI[sha256sum] = "840af5366ab9b522bde525826be3ef0fb0af81c6a9ebd84caa600fea1731eee3" inherit lib_package multilib_header multilib_script ptest perlnative manpages MULTILIB_SCRIPTS = "${PN}-bin:${bindir}/c_rehash"
Signature · 1 line
2.34.1
PATCH 6/33 goarch: Move Go architecture mapping to a library
In reply to Steve SakomanFrom: Joshua Watt <[email protected]>
Other spaces uses the Go architecture definitions as their own (for example, container arches are defined to be Go arches). To make it easier for other places to use this mapping, move the code that does the translation of OpenEmbedded arches to Go arches to a library.
--- meta/classes-recipe/goarch.bbclass | 29 +++---------------------- meta/lib/oe/__init__.py | 2 +- meta/lib/oe/go.py | 34 ++++++++++++++++++++++++++++++ 3 files changed, 38 insertions(+), 27 deletions(-) create mode 100644 meta/lib/oe/go.py
meta/classes-recipe/goarch.bbclass +3 -26
@@ -68,33 +68,10 @@ SECURITY_NOPIE_CFLAGS ??= "" CCACHE_DISABLE ?= "1" def go_map_arch(a, d): - import re - if re.match('i.86', a): - return '386' - elif a == 'x86_64': - return 'amd64' - elif re.match('arm.*', a): - return 'arm' - elif re.match('aarch64.*', a): - return 'arm64' - elif re.match('mips64el.*', a): - return 'mips64le' - elif re.match('mips64.*', a): - return 'mips64' - elif a == 'mips': - return 'mips' - elif a == 'mipsel': - return 'mipsle' - elif re.match('p(pc|owerpc)(64le)', a): - return 'ppc64le' - elif re.match('p(pc|owerpc)(64)', a): - return 'ppc64' - elif a == 'riscv64': - return 'riscv64' - elif a == 'loongarch64': - return 'loong64' - else: + arch = oe.go.map_arch(a) + if not arch: raise bb.parse.SkipRecipe("Unsupported CPU architecture: %s" % a) + return arch def go_map_arm(a, d): if a.startswith("arm"):
meta/lib/oe/__init__.py +1 -1
@@ -9,4 +9,4 @@ __path__ = extend_path(__path__, __name__) BBIMPORTS = ["data", "path", "utils", "types", "package", "packagedata", \ "packagegroup", "sstatesig", "lsb", "cachedpath", "license", \ - "qa", "reproducible", "rust", "buildcfg"] + "qa", "reproducible", "rust", "buildcfg", "go"]
meta/lib/oe/go.py new file +34 -0
@@ -0,0 +1,34 @@ +# +# Copyright OpenEmbedded Contributors +# +# SPDX-License-Identifier: MIT +# + +import re + +def map_arch(a): + if re.match('i.86', a): + return '386' + elif a == 'x86_64': + return 'amd64' + elif re.match('arm.*', a): + return 'arm' + elif re.match('aarch64.*', a): + return 'arm64' + elif re.match('mips64el.*', a): + return 'mips64le' + elif re.match('mips64.*', a): + return 'mips64' + elif a == 'mips': + return 'mips' + elif a == 'mipsel': + return 'mipsle' + elif re.match('p(pc|owerpc)(64le)', a): + return 'ppc64le' + elif re.match('p(pc|owerpc)(64)', a): + return 'ppc64' + elif a == 'riscv64': + return 'riscv64' + elif a == 'loongarch64': + return 'loong64' + return ''
Signature · 1 line
2.34.1
PATCH 7/33 perl: fix intermittent test failure
In reply to Steve SakomanFrom: William Lyu <[email protected]>
Fixes [YOCTO #15136]
This fix addresses the intermittent failure of the Perl ptest t/op/sigsystem.t.
--- ...ent-failure-of-test-t-op-sigsystem.t.patch | 77 +++++++++++++++++++ meta/recipes-devtools/perl/perl_5.38.0.bb | 1 + 2 files changed, 78 insertions(+) create mode 100644 meta/recipes-devtools/perl/files/0001-Fix-intermittent-failure-of-test-t-op-sigsystem.t.patch
meta/recipes-devtools/perl/files/0001-Fix-intermittent-failure-of-test-t-op-sigsystem.t.patch new file +77 -0
@@ -0,0 +1,77 @@ +From 75d974a58c461b3b5d35280e497810e46abae4ca Mon Sep 17 00:00:00 2001 +From: William Lyu <[email protected]> +Date: Wed, 4 Oct 2023 08:58:41 -0400 +Subject: [PATCH] Fix intermittent failure of test t/op/sigsystem.t + +[Perl issue #21546] -- https://github.com/Perl/perl5/issues/21546 + +This fix addresses the intermittent failure of the test +t/op/sigsystem.t by improving its robustness. Before the fix, this +test waits a hard-coded amount of time in the parent process for the +child process to exit, and the child process may not be able to exit +soon enough. With this fix, the parent process in this test polls for +whether the SIGCHLD handler reaped the child process for at most 25 +seconds. + +Upstream-Status: Backport [commit ID: 75d974a] + +Signed-off-by: William Lyu <[email protected]> +Signed-off-by: Randy MacLeod <[email protected]> +Reported-by: Alexandre Belloni <[email protected]> + +Committer: William Lyu is now a Perl author. +--- + AUTHORS | 1 + + t/op/sigsystem.t | 17 ++++++++++++++--- + 2 files changed, 15 insertions(+), 3 deletions(-) + +diff --git a/AUTHORS b/AUTHORS +index 21948bfdc7..527dd992fd 100644 +--- a/AUTHORS ++++ b/AUTHORS +@@ -1443,6 +1443,7 @@ Wayne Scott <[email protected]> + Wayne Thompson <[email protected]> + Wilfredo Sánchez <[email protected]> + William J. Middleton <[email protected]> ++William Lyu <[email protected]> + William Mann <[email protected]> + William Middleton <[email protected]> + William R Ward <[email protected]> +diff --git a/t/op/sigsystem.t b/t/op/sigsystem.t +index 25da854902..831feefb0f 100644 +--- a/t/op/sigsystem.t ++++ b/t/op/sigsystem.t +@@ -37,7 +37,15 @@ SKIP: { + test_system('with reaper'); + + note("Waiting briefly for SIGCHLD..."); +- Time::HiRes::sleep(0.500); ++ ++ # Wait at most 50 * 0.500 = 25.0 seconds for the child process to be ++ # reaped. If the child process exits and gets reaped early, this polling ++ # loop will exit early. ++ ++ for (1..50) { ++ last if @pids; ++ Time::HiRes::sleep(0.500); ++ } + + ok(@pids == 1, 'Reaped only one process'); + ok($pids[0] == $pid, "Reaped the right process.") or diag(Dumper(\@pids)); +@@ -50,8 +58,11 @@ sub test_system { + my $got_zeroes = 0; + + # This test is looking for a race between system()'s waitpid() and a +- # signal handler. Looping a few times increases the chances of +- # catching the error. ++ # signal handler. The system() call is expected to not interfere with the ++ # SIGCHLD signal handler. In particular, the wait() called within system() ++ # is expected to reap the child process forked by system() before the ++ # SIGCHLD signal handler is called. ++ # Looping a few times increases the chances of catching the error. + + for (1..$expected_zeroes) { + $got_zeroes++ unless system(TRUE); +-- +2.25.1 +
meta/recipes-devtools/perl/perl_5.38.0.bb +1 -0
@@ -17,6 +17,7 @@ SRC_URI = "https://www.cpan.org/src/5.0/perl-${PV}.tar.gz;name=perl \ file://0002-Constant-Fix-up-shebang.patch \ file://determinism.patch \ file://0001-cpan-Sys-Syslog-Makefile.PL-Fix-_PATH_LOG-for-determ.patch \ + file://0001-Fix-intermittent-failure-of-test-t-op-sigsystem.t.patch \ " SRC_URI:append:class-native = " \ file://perl-configpm-switch.patch \
Signature · 1 line
2.34.1
PATCH 8/33 Revert "bin_package.bbclass: Inhibit the default dependencies"
In reply to Steve SakomanFrom: Max Krummenacher <[email protected]>
This reverts commit d1d09bd4d7be88f0e341d5fccbfbefeb98d4b727.
The commit not only removes the dependencies on the cross compiler but also does not depend on e.g. virtual/${TARGET_PREFIX}compilerlibs and virtual/libc which in turn makes the file-rdeps qa check fail if installing binaries linked against e.g. libc or libstdc++.
--- meta/classes-recipe/bin_package.bbclass | 3 --- 1 file changed, 3 deletions(-)
meta/classes-recipe/bin_package.bbclass +0 -3
@@ -20,9 +20,6 @@ # they would be in ${WORKDIR}. # -# Nothing is being built so there is no need for the cross-compiler. -INHIBIT_DEFAULT_DEPS = "1" - # Skip the unwanted steps do_configure[noexec] = "1" do_compile[noexec] = "1"
Signature · 1 line
2.34.1
PATCH 9/33 oeqa/ssh: Handle SSHCall timeout error code
In reply to Steve SakomanFrom: luca fancellu <[email protected]>
The current code in ssh.py is terminating the ssh process that does not finish its computation in a given timeout (when timeout is passed), the SSHCall function is returning the process error code.
The Openssl ssh before version 8.6_p1 is returning 0 when it is terminated, from commit 8a9520836e71830f4fccca066dba73fea3d16bda onwards (version >= 8.6_p1) ssh is returning 255 instead.
So for version of ssh older than 8.6_p1 when the SSHCall time out, the return code will be 0, meaning success, which is wrong.
Fix this issue checking if the process has timeout (hence it's been terminated) and checking if the returned code is 0, in that case set it to 255 to advertise that an error occurred.
Add a test case excercising the timeout in the SSHTest, test_ssh test function.
--- meta/lib/oeqa/core/target/ssh.py | 17 ++++++++++++++++- meta/lib/oeqa/runtime/cases/ssh.py | 3 +++ 2 files changed, 19 insertions(+), 1 deletion(-)
meta/lib/oeqa/core/target/ssh.py +16 -1
@@ -232,11 +232,12 @@ def SSHCall(command, logger, timeout=None, **opts): output_raw = b'' starttime = time.time() process = subprocess.Popen(command, **options) + has_timeout = False if timeout: endtime = starttime + timeout eof = False os.set_blocking(process.stdout.fileno(), False) - while time.time() < endtime and not eof: + while not has_timeout and not eof: try: logger.debug('Waiting for process output: time: %s, endtime: %s' % (time.time(), endtime)) if select.select([process.stdout], [], [], 5)[0] != []:
@@ -257,6 +258,10 @@ def SSHCall(command, logger, timeout=None, **opts): logger.debug('BlockingIOError') continue + if time.time() >= endtime: + logger.debug('SSHCall has timeout! Time: %s, endtime: %s' % (time.time(), endtime)) + has_timeout = True + process.stdout.close() # process hasn't returned yet
@@ -293,6 +298,16 @@ def SSHCall(command, logger, timeout=None, **opts): pass process.wait() + if has_timeout: + # Version of openssh before 8.6_p1 returns error code 0 when killed + # by a signal, when the timeout occurs we will receive a 0 error + # code because the process is been terminated and it's wrong because + # that value means success, but the process timed out. + # Afterwards, from version 8.6_p1 onwards, the returned code is 255. + # Fix this behaviour by checking the return code + if process.returncode == 0: + process.returncode = 255 + options = { "stdout": subprocess.PIPE, "stderr": subprocess.STDOUT,
meta/lib/oeqa/runtime/cases/ssh.py +3 -0
@@ -13,6 +13,9 @@ class SSHTest(OERuntimeTestCase): @OETestDepends(['ping.PingTest.test_ping']) @OEHasPackage(['dropbear', 'openssh-sshd']) def test_ssh(self): + (status, output) = self.target.run('sleep 20', timeout=2) + msg='run() timed out but return code was zero.' + self.assertNotEqual(status, 0, msg=msg) (status, output) = self.target.run('uname -a') self.assertEqual(status, 0, msg='SSH Test failed: %s' % output) (status, output) = self.target.run('cat /etc/controllerimage')
Signature · 1 line
2.34.1
PATCH 10/33 bb-matrix-plot.sh: Show underscores correctly in labels
In reply to Steve SakomanFrom: Peter Kjellerstedt <[email protected]>
Underscores previously caused the next character in the label to be printed using subscript due to the enhanced string support in gnuplot.
--- scripts/contrib/bb-perf/bb-matrix-plot.sh | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
scripts/contrib/bb-perf/bb-matrix-plot.sh +2 -2
@@ -16,8 +16,8 @@ # Setup the defaults DATFILE="bb-matrix.dat" -XLABEL="BB_NUMBER_THREADS" -YLABEL="PARALLEL_MAKE" +XLABEL="BB\\\\_NUMBER\\\\_THREADS" +YLABEL="PARALLEL\\\\_MAKE" FIELD=3 DEF_TITLE="Elapsed Time (seconds)" PM3D_FRAGMENT="unset surface; set pm3d at s hidden3d 100"
Signature · 1 line
2.34.1
PATCH 11/33 kernel.bbclass: Use strip utility used for kernel build in do_package
In reply to Steve SakomanFrom: Khem Raj <[email protected]>
os.environ does not pass this down to runstrip() function and in strip_execs() its using STRIP bitbake variable to find the strip utility to use. Since there might be a trailing whitespace in KERNEL_STRIP remove that otherwise python is not able to launch it.
e.g.
FileNotFoundError: [Errno 2] No such file or directory: 'riscv64-yoe-linux-strip '
This is more evident when STRIP and KERNEL_STRIP are different utilities e.g. when using clang as default toolchain but using gcc+binutils only for kernel build.
--- meta/classes-recipe/kernel.bbclass | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
meta/classes-recipe/kernel.bbclass +1 -1
@@ -336,7 +336,7 @@ kernel_do_transform_bundled_initramfs() { do_transform_bundled_initramfs[dirs] = "${B}" python do_package:prepend () { - os.environ['STRIP'] = d.getVar('KERNEL_STRIP') + d.setVar('STRIP', d.getVar('KERNEL_STRIP').strip()) } python do_devshell:prepend () {
Signature · 1 line
2.34.1
Re: [OE-core][nanbield 11/33] kernel.bbclass: Use strip utility used for kernel build in do_package
This requires additional fix from:
https://git.openembedded.org/openembedded-core/commit/?id=41f019afc41f800b622c46a6d7cf1beffc97716a
Cheers,
Quoted 54 lines · Steve Sakoman
From: Khem Raj <[email protected]>
os.environ does not pass this down to
runstrip()function and instrip_execs()its using STRIP bitbake variable to find the strip utility to use. Since there might be a trailing whitespace inKERNEL_STRIPremove that otherwise python is not able to launch it.
e.g.FileNotFoundError: [Errno 2] No such file or directory:
'riscv64-yoe-linux-strip 'This is more evident when STRIP and
KERNEL_STRIPare different utilities e.g. when using clang as default toolchain but using gcc+binutils only for kernel build.Signed-off-by Khem Raj <[email protected]>Cc Bruce Ashfield <[email protected]>Signed-off-by Luca Ceresoli <[email protected]>Signed-off-by Richard Purdie <[email protected]>(cherry picked from commit 77497dbdca92ab4d6386a071bc281c42a7e8a14b)Signed-off-by Steve Sakoman <[email protected]>--- meta/classes-recipe/kernel.bbclass | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)meta/classes-recipe/kernel.bbclass +1
-2@@ -336,7 +336,7 @@ kernel_do_transform_bundled_initramfs() { do_transform_bundled_initramfs[dirs] = "${B}" python do_package:prepend () { - os.environ['STRIP'] = d.getVar('KERNEL_STRIP') + d.setVar('STRIP', d.getVar('KERNEL_STRIP').strip()) } python do_devshell:prepend () { --2.34.1
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group. View/Reply Online (#190900):
https://lists.openembedded.org/g/openembedded-core/message/190900
Mute This Topic: https://lists.openembedded.org/mt/102710661/3617156
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [ [email protected]]
-=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core][nanbield 11/33] kernel.bbclass: Use strip utility used for kernel build in do_package
Got it! Thanks for reviewing.
Steve
Quoted 57 lines · Martin Jansa
This requires additional fix from:
https://git.openembedded.org/openembedded-core/commit/?id=41f019afc41f800b622c46a6d7cf1beffc97716aCheers,
Quoted 50 lines · Steve Sakoman
From: Khem Raj <[email protected]>
os.environ does not pass this down to
runstrip()function and instrip_execs()its using STRIP bitbake variable to find the strip utility to use. Since there might be a trailing whitespace inKERNEL_STRIPremove that otherwise python is not able to launch it.
e.g.FileNotFoundError: [Errno 2] No such file or directory: 'riscv64-yoe-linux-strip '
This is more evident when STRIP and
KERNEL_STRIPare different utilities e.g. when using clang as default toolchain but using gcc+binutils only for kernel build.Signed-off-by Khem Raj <[email protected]>Cc Bruce Ashfield <[email protected]>Signed-off-by Luca Ceresoli <[email protected]>Signed-off-by Richard Purdie <[email protected]>(cherry picked from commit 77497dbdca92ab4d6386a071bc281c42a7e8a14b)Signed-off-by Steve Sakoman <[email protected]>--- meta/classes-recipe/kernel.bbclass | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)meta/classes-recipe/kernel.bbclass +1
-2@@ -336,7 +336,7 @@ kernel_do_transform_bundled_initramfs() { do_transform_bundled_initramfs[dirs] = "${B}" python do_package:prepend () { - os.environ['STRIP'] = d.getVar('KERNEL_STRIP') + d.setVar('STRIP', d.getVar('KERNEL_STRIP').strip()) } python do_devshell:prepend () { --2.34.1
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#190900): https://lists.openembedded.org/g/openembedded-core/message/190900
Mute This Topic: https://lists.openembedded.org/mt/102710661/3617156
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-
PATCH 12/33 oeqa/selftest/devtool: abort if a local workspace already exist
In reply to Steve SakomanFrom: Julien Stephan <[email protected]>
if user run devtool selftests with a local workspacelayer the tests fail with various error such as:
- devtool.DevtoolAddTests.test_devtool_add just hangs
- devtool.DevtoolModifyTests.* fail with the following error:
ERROR: Found duplicated BBFILE_COLLECTIONS 'workspacelayer', check bblayers.conf or layer.conf to fix it. Found duplicated BBFILE_COLLECTIONS 'workspacelayer', check bblayers.conf or layer.conf to fix it.
Check if a workspacelayer exists, warn the user and abort the tests
--- meta/lib/oeqa/selftest/cases/devtool.py | 3 +++ 1 file changed, 3 insertions(+)
meta/lib/oeqa/selftest/cases/devtool.py +3 -0
@@ -27,6 +27,9 @@ def setUpModule(): corecopydir = os.path.join(templayerdir, 'core-copy') bblayers_conf = os.path.join(os.environ['BUILDDIR'], 'conf', 'bblayers.conf') edited_layers = [] + # make sure user doesn't have a local workspace + result = runCmd('bitbake-layers show-layers') + assert "workspacelayer" not in result.output, "Devtool test suite cannot be run with a local workspace directory" # We need to take a copy of the meta layer so we can modify it and not # have any races against other tests that might be running in parallel
Signature · 1 line
2.34.1
PATCH 13/33 linux-yocto: make sure the pahole-native available before do_kernel_configme
In reply to Steve SakomanFrom: Xiangyu Chen <[email protected]>
When using debug-btf.scc in a clean workspace, the CONFIG_MODULE_ALLOW_BTF_MISMATCH cannot apply to kernel until clean the kernel code(bitbake linux-yocto -c cleanall) and rebuild.
After tracking the code, some options depend on CONFIG_PAHOLE_VERSION, it was generated by scripts/pahole-version.sh in kernel, but during do_kernel_configme step, the pahole-native is not available in sysroot-native, so need to wait pahole-native install to sysroot-native before do_kernel_configme.
--- meta/recipes-kernel/linux/linux-yocto.inc | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-)
meta/recipes-kernel/linux/linux-yocto.inc +4 -1
@@ -65,7 +65,10 @@ KERNEL_DEBUG ?= "" DEPENDS += '${@bb.utils.contains_any("ARCH", [ "x86", "arm64", "powerpc" ], "elfutils-native", "", d)}' DEPENDS += "openssl-native util-linux-native" DEPENDS += "gmp-native libmpc-native" -DEPENDS += '${@bb.utils.contains("KERNEL_DEBUG", "True", "pahole-native", "", d)}' + +# Some options depend on CONFIG_PAHOLE_VERSION, so need to make pahole-native available before do_kernel_configme +do_kernel_configme[depends] += '${@bb.utils.contains("KERNEL_DEBUG", "True", "pahole-native:do_populate_sysroot", "", d)}' + EXTRA_OEMAKE += '${@bb.utils.contains("KERNEL_DEBUG", "True", "", "PAHOLE=false", d)}' do_devshell:prepend() {
Signature · 1 line
2.34.1
PATCH 14/33 base: Ensure recipes using mercurial-native have certificates
In reply to Steve SakomanFrom: Richard Purdie <[email protected]>
If you try and fetch using mercurial-native, you see certificate errors since it is configured to find ones in the sysroot, not the system. Add the missing dependency so that mercurial recipes using the native tool work.
Found trying to make mirroring for old meta-oe stable branches work.
--- meta/classes-global/base.bbclass | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
meta/classes-global/base.bbclass +1 -1
@@ -634,7 +634,7 @@ python () { # Mercurial packages should DEPEND on mercurial-native elif uri.scheme == "hg": d.appendVar("EXTRANATIVEPATH", ' python3-native ') - d.appendVarFlag('do_fetch', 'depends', ' mercurial-native:do_populate_sysroot') + d.appendVarFlag('do_fetch', 'depends', ' mercurial-native:do_populate_sysroot ca-certificates-native:do_populate_sysroot') # OSC packages should DEPEND on osc-native elif uri.scheme == "osc":
Signature · 1 line
2.34.1
PATCH 15/33 glib-2.0: Remove unnecessary assignement
In reply to Steve SakomanFrom: Rouven Czerwinski <[email protected]>
FILES:${PN}-utils is += extended and than replaced completely later, remove the first extension.
--- meta/recipes-core/glib-2.0/glib.inc | 5 ----- 1 file changed, 5 deletions(-)
meta/recipes-core/glib-2.0/glib.inc +0 -5
@@ -57,11 +57,6 @@ FILES:${PN} = "${libdir}/lib*${SOLIBS} \ ${datadir}/glib-2.0/dtds \ ${datadir}/glib-2.0/schemas" -FILES:${PN}-utils += "${bindir}/glib-genmarshal \ - ${bindir}/glib-gettextize \ - ${bindir}/glib-mkenums \ - ${bindir}/glib-compile-resources" - FILES:${PN}-dev += "${libdir}/glib-2.0/include \ ${libdir}/gio/modules/lib*${SOLIBSDEV} \ ${libdir}/gio/modules/*.la \
Signature · 1 line
2.34.1
PATCH 16/33 baremetal-helloworld: Pull in fix for race condition on x86-64
In reply to Steve SakomanFrom: Alejandro Hernandez Samaniego <[email protected]>
It was previously discovered that there was a race condition during the Makefile execution between the assemble and compile targets, the previous fix attempted to serialize the build targets, but the fix was missing for x86-64.
Pull in latest commit from upstream to fix this issue on x86-64.
[YOCTO #15146]
--- .../baremetal-example/baremetal-helloworld_git.bb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
meta/recipes-extended/baremetal-example/baremetal-helloworld_git.bb +1 -1
@@ -4,7 +4,7 @@ DESCRIPTION = "These are introductory examples to showcase the use of QEMU to ru LICENSE = "MIT" LIC_FILES_CHKSUM = "file://LICENSE;md5=39346640a23c701e4f459e05f56f4449" -SRCREV = "fc7c43d138185028b6ac14c83f6492fce26eca95" +SRCREV = "db2bf750eaef7fc0832e13ada8291343bbcc3afe" PV = "0.1+git" SRC_URI = "git://github.com/ahcbb6/baremetal-helloqemu.git;protocol=https;branch=master"
Signature · 1 line
2.34.1
PATCH 17/33 patchtest: remove unused imports
In reply to Steve SakomanFrom: Ross Burton <[email protected]>
--- meta/lib/patchtest/data.py | 1 - meta/lib/patchtest/repo.py | 1 - meta/lib/patchtest/tests/test_mbox_cve.py | 1 - meta/lib/patchtest/tests/test_mbox_mailinglist.py | 1 - meta/lib/patchtest/tests/test_metadata_src_uri.py | 1 - meta/lib/patchtest/tests/test_patch_cve.py | 1 - meta/lib/patchtest/tests/test_patch_upstream_status.py | 1 - meta/lib/patchtest/utils.py | 1 - scripts/patchtest | 1 - scripts/patchtest-get-branch | 1 - 10 files changed, 10 deletions(-)
meta/lib/patchtest/data.py +0 -1
@@ -16,7 +16,6 @@ import os import argparse import collections -import tempfile import logging logger=logging.getLogger('patchtest')
meta/lib/patchtest/repo.py +0 -1
@@ -11,7 +11,6 @@ import os import utils import logging -import json from patch import PatchTestPatch logger = logging.getLogger('patchtest')
meta/lib/patchtest/tests/test_mbox_cve.py +0 -1
@@ -6,7 +6,6 @@ # import base -import os import parse_cve_tags import pyparsing
meta/lib/patchtest/tests/test_mbox_mailinglist.py +0 -1
@@ -4,7 +4,6 @@ # # SPDX-License-Identifier: GPL-2.0-only -import subprocess import collections import base import pyparsing
meta/lib/patchtest/tests/test_metadata_src_uri.py +0 -1
@@ -4,7 +4,6 @@ # # SPDX-License-Identifier: GPL-2.0-only -import subprocess import base import os import pyparsing
meta/lib/patchtest/tests/test_patch_cve.py +0 -1
@@ -6,7 +6,6 @@ # import base -import os import pyparsing class CVE(base.Base):
meta/lib/patchtest/tests/test_patch_upstream_status.py +0 -1
@@ -7,7 +7,6 @@ import base import parse_upstream_status import pyparsing -import os class PatchUpstreamStatus(base.Base):
meta/lib/patchtest/utils.py +0 -1
@@ -11,7 +11,6 @@ import os import subprocess import logging -import sys import re import mailbox
scripts/patchtest +0 -1
@@ -12,7 +12,6 @@ import sys import os import unittest -import fileinput import logging import traceback import json
scripts/patchtest-get-branch +0 -1
@@ -15,7 +15,6 @@ import mailbox import argparse import re import git -import sys re_prefix = re.compile("(\[.*\])", re.DOTALL)
Signature · 1 line
2.34.1
PATCH 18/33 patchtest: test regardless of mergeability
In reply to Steve SakomanFrom: Trevor Gamblin <[email protected]>
--- meta/lib/patchtest/tests/test_metadata_lic_files_chksum.py | 5 ----- meta/lib/patchtest/tests/test_metadata_license.py | 5 ----- meta/lib/patchtest/tests/test_metadata_summary.py | 5 ----- meta/lib/patchtest/tests/test_python_pylint.py | 2 -- 4 files changed, 17 deletions(-)
meta/lib/patchtest/tests/test_metadata_lic_files_chksum.py +0 -5
@@ -15,11 +15,6 @@ class LicFilesChkSum(base.Metadata): lictag = 'License-Update' lictag_re = pyparsing.Regex("^%s:" % lictag) - def setUp(self): - # these tests just make sense on patches that can be merged - if not PatchTestInput.repo.canbemerged: - self.skip('Patch cannot be merged') - def test_lic_files_chksum_presence(self): if not self.added: self.skip('No added recipes, skipping test')
meta/lib/patchtest/tests/test_metadata_license.py +0 -5
@@ -12,11 +12,6 @@ class License(base.Metadata): metadata = 'LICENSE' invalid_license = 'PATCHTESTINVALID' - def setUp(self): - # these tests just make sense on patches that can be merged - if not PatchTestInput.repo.canbemerged: - self.skip('Patch cannot be merged') - def test_license_presence(self): if not self.added: self.skip('No added recipes, skipping test')
meta/lib/patchtest/tests/test_metadata_summary.py +0 -5
@@ -10,11 +10,6 @@ from data import PatchTestInput class Summary(base.Metadata): metadata = 'SUMMARY' - def setUp(self): - # these tests just make sense on patches that can be merged - if not PatchTestInput.repo.canbemerged: - self.skip('Patch cannot be merged') - def test_summary_presence(self): if not self.added: self.skip('No added recipes, skipping test')
meta/lib/patchtest/tests/test_python_pylint.py +0 -2
@@ -26,8 +26,6 @@ class PyLint(base.Base): def setUp(self): if self.unidiff_parse_error: self.skip('Python-unidiff parse error') - if not PatchTestInput.repo.canbemerged: - self.skip('Patch cannot be merged, no reason to execute the test method') if not PyLint.pythonpatches: self.skip('No python related patches, skipping test')
Signature · 1 line
2.34.1
PATCH 19/33 patchtest: skip merge test if not targeting master
In reply to Steve SakomanFrom: Trevor Gamblin <[email protected]>
Avoid testing mergeability of a patch when not targeting master, so that patches tested via other means (e.g. maintainer branches and AB runs) don't get unnecessarily reviewed an extra time.
--- meta/lib/patchtest/tests/test_mbox_merge.py | 2 ++ 1 file changed, 2 insertions(+)
meta/lib/patchtest/tests/test_mbox_merge.py +2 -0
@@ -18,6 +18,8 @@ def headlog(): class Merge(base.Base): def test_series_merge_on_head(self): + if PatchTestInput.repo.branch != "master": + self.skip("Skipping merge test since patch is not intended for master branch. Target detected is %s" % PatchTestInput.repo.branch) if not PatchTestInput.repo.ismerged: commithash, author, date, shortlog = headlog() self.fail('Series does not apply on top of target branch. Rebase your series and ensure the target is correct',
Signature · 1 line
2.34.1
PATCH 20/33 patchtest: fix lic_files_chksum test regex
In reply to Steve SakomanFrom: Trevor Gamblin <[email protected]>
the test_lic_files_chksum_modified_not_mentioned test in patchtest wasn't picking up on 'License-Update:' tags correctly. Use pyparsing's AtLineStart class to simplify the regex setup and search.
--- meta/lib/patchtest/tests/test_metadata_lic_files_chksum.py | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-)
meta/lib/patchtest/tests/test_metadata_lic_files_chksum.py +2 -3
@@ -12,8 +12,7 @@ class LicFilesChkSum(base.Metadata): metadata = 'LIC_FILES_CHKSUM' license = 'LICENSE' closed = 'CLOSED' - lictag = 'License-Update' - lictag_re = pyparsing.Regex("^%s:" % lictag) + lictag_re = pyparsing.AtLineStart("License-Update:") def test_lic_files_chksum_presence(self): if not self.added:
@@ -71,5 +70,5 @@ class LicFilesChkSum(base.Metadata): if self.lictag_re.search_string(commit.commit_message): break else: - self.fail('LIC_FILES_CHKSUM changed on target %s but there is no "%s" tag in commit message. Include it with a brief description' % (pn, self.lictag), + self.fail('LIC_FILES_CHKSUM changed on target %s but there is no "License-Update:" tag in commit message. Include it with a brief description' % pn, data=[('Current checksum', pretest), ('New checksum', test)])
Signature · 1 line
2.34.1
PATCH 21/33 patchtest-send-results: improve subject line
In reply to Steve SakomanFrom: Trevor Gamblin <[email protected]>
Pull the actual email's subject line from the .mbox file and use that in patchtest's test results response, so that it's clearer which patch it is replying to.
--- scripts/patchtest-send-results | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-)
scripts/patchtest-send-results +6 -1
@@ -53,7 +53,12 @@ with open(result_file, "r") as f: testresult = f.read() reply_contents = greeting + testresult + suggestions -subject_line = f"Patchtest results for {result_basename}" + +# we know these patch files will only contain a single patch, so only +# worry about the first element for getting the subject +mbox = mailbox.mbox(args.patch) +mbox_subject = mbox[0]['subject'] +subject_line = f"Patchtest results for {mbox_subject}" if "FAIL" in testresult: ses_client = boto3.client('ses', region_name='us-west-2')
Signature · 1 line
2.34.1
PATCH 22/33 patchtest: disable merge test
In reply to Steve SakomanFrom: Trevor Gamblin <[email protected]>
Disable the merge-on-head test until patchtest properly handles merging of series subsets and accounts for patches that are rapidly merged (i.e. before patchtest is run).
--- meta/lib/patchtest/tests/test_mbox_merge.py | 1 + 1 file changed, 1 insertion(+)
meta/lib/patchtest/tests/test_mbox_merge.py +1 -0
@@ -18,6 +18,7 @@ def headlog(): class Merge(base.Base): def test_series_merge_on_head(self): + self.skip("Merge test is disabled for now") if PatchTestInput.repo.branch != "master": self.skip("Skipping merge test since patch is not intended for master branch. Target detected is %s" % PatchTestInput.repo.branch) if not PatchTestInput.repo.ismerged:
Signature · 1 line
2.34.1
PATCH 23/33 patchtest-send-results: check max line length, simplify responses
In reply to Steve SakomanFrom: Trevor Gamblin <[email protected]>
Check that the maximum line length of the testresult file is less than 220 characters, to help guard against malicious changes being sent in email responses. If any line exceeds this length, replace the normal testresults used in the response with a line stating that tests failed, but the results could not be processed. Also clean up the respone substrings slightly to go along with the change.
--- scripts/patchtest-send-results | 17 +++++++++-------- 1 file changed, 9 insertions(+), 8 deletions(-)
scripts/patchtest-send-results +9 -8
@@ -25,12 +25,8 @@ more information:\n\n---\n""" suggestions = """\n---\n\nPlease address the issues identified and submit a new revision of the patch, or alternatively, reply to this -email with an explanation of why the patch format should be accepted. -Note that patchtest may report failures in the merge-on-head test for -patches that are part of a series if they rely on changes from -preceeding entries. - -If you believe these results are due to an error in patchtest, please +email with an explanation of why the patch format should be accepted. If +you believe these results are due to an error in patchtest, please submit a bug at https://bugzilla.yoctoproject.org/ (use the 'Patchtest' category under 'Yocto Project Subprojects'). Thank you!"""
@@ -52,8 +48,6 @@ testresult = None with open(result_file, "r") as f: testresult = f.read() -reply_contents = greeting + testresult + suggestions - # we know these patch files will only contain a single patch, so only # worry about the first element for getting the subject mbox = mailbox.mbox(args.patch)
@@ -61,6 +55,13 @@ mbox_subject = mbox[0]['subject'] subject_line = f"Patchtest results for {mbox_subject}" if "FAIL" in testresult: + reply_contents = None + if len(max(open(result_file, 'r'), key=len)) > 220: + warning = "Tests failed for the patch, but the results log could not be processed due to excessive result line length." + reply_contents = greeting + warning + suggestions + else: + reply_contents = greeting + testresult + suggestions + ses_client = boto3.client('ses', region_name='us-west-2') response = ses_client.send_email( Source='[email protected]',
Signature · 1 line
2.34.1
PATCH 24/33 patchtest/selftest: add XSKIP, update test files
In reply to Steve SakomanFrom: Trevor Gamblin <[email protected]>
Since we are skipping the merge test, two of the selftests now report SKIP instead of XPASS/XFAIL as expected. Adjust the two files to have the right endings for XSKIP, and add the category so that it can be used for more extensive testing in the future.
---
...ad.pass => Merge.test_series_merge_on_head.1.skip} | 0
...ad.fail => Merge.test_series_merge_on_head.2.skip} | 0
meta/lib/patchtest/selftest/selftest | 11 ++++++++---
3 files changed, 8 insertions(+), 3 deletions(-)
rename meta/lib/patchtest/selftest/files/{Merge.test_series_merge_on_head.pass => Merge.test_series_merge_on_head.1.skip} (100%)
rename meta/lib/patchtest/selftest/files/{Merge.test_series_merge_on_head.fail => Merge.test_series_merge_on_head.2.skip} (100%)meta/lib/patchtest/selftest/files/Merge.test_series_merge_on_head.1.skip renamed from meta/lib/patchtest/selftest/files/Merge.test_series_merge_on_head.pass +0 -0
meta/lib/patchtest/selftest/files/Merge.test_series_merge_on_head.2.skip renamed from meta/lib/patchtest/selftest/files/Merge.test_series_merge_on_head.fail +0 -0
meta/lib/patchtest/selftest/selftest +8 -3
@@ -18,14 +18,15 @@ parentdir = os.path.dirname(topdir) # path to the repo root repodir = os.path.dirname(os.path.dirname(parentdir)) -def print_results(passcount, skipcount, failcount, xpasscount, xfailcount, errorcount): - total = passcount + skipcount + failcount + xpasscount + xfailcount + errorcount +def print_results(passcount, failcount, skipcount, xpasscount, xfailcount, xskipcount, errorcount): + total = passcount + skipcount + failcount + xpasscount + xfailcount + xskipcount + errorcount print("============================================================================") print("Testsuite summary for %s" % os.path.basename(topdir)) print("============================================================================") print("# TOTAL: %s" % str(total)) print("# XPASS: %s" % str(xpasscount)) print("# XFAIL: %s" % str(xfailcount)) + print("# XSKIP: %s" % str(xskipcount)) print("# PASS: %s" % str(passcount)) print("# FAIL: %s" % str(failcount)) print("# SKIP: %s" % str(skipcount))
@@ -48,6 +49,7 @@ if __name__ == '__main__': skipcount = 0 xpasscount = 0 xfailcount = 0 + xskipcount = 0 errorcount = 0 results = None
@@ -71,6 +73,9 @@ if __name__ == '__main__': elif expected_result.upper() == "PASS" and result.upper() == "PASS": xpasscount = xpasscount + 1 print("XPASS: %s (file: %s)" % (testid.strip("."), os.path.basename(patch))) + elif expected_result.upper() == "SKIP" and result.upper() == "SKIP": + xskipcount = xskipcount + 1 + print("XSKIP: %s (file: %s)" % (testid.strip("."), os.path.basename(patch))) else: print("%s: %s (%s)" % (result.upper(), testid.strip("."), os.path.basename(patch))) if result.upper() == "PASS":
@@ -86,4 +91,4 @@ if __name__ == '__main__': else: print ("No test for=%s" % patch) - print_results(passcount, skipcount, failcount, xpasscount, xfailcount, errorcount) + print_results(passcount, failcount, skipcount, xpasscount, xfailcount, xskipcount, errorcount)
Signature · 1 line
2.34.1
PATCH 25/33 patchtest: simplify test directory structure
In reply to Steve SakomanFrom: Trevor Gamblin <[email protected]>
Consolidate the various mbox tests into a new TestMbox class, metadata tests into TestMetadata, and patch tests into TestPatch. Also update the selftest filenames to match the changes. The test contents are not significantly changed (other than to reference the new class names). While this doesn't improve overall readability, it does result in more obvious categorization, and more importantly reduces the number of calls to setup tinfoil in the tests, resulting in a roughly 25% reduction in runtime.
Before:
[tgamblin@megalith poky]$ time ./meta/lib/patchtest/selftest/selftest
XPASS: PatchSignedOffBy.test_signed_off_by_presence (file: PatchSignedOffBy.test_signed_off_by_presence.pass)
XFAIL: Shortlog.test_shortlog_format (file: Shortlog.test_shortlog_format.fail)
XFAIL: MboxFormat.test_mbox_format (file: MboxFormat.test_mbox_format.1.fail)
XPASS: Shortlog.test_shortlog_length (file: Shortlog.test_shortlog_length.pass)
XFAIL: CommitMessage.test_commit_message_presence (file: CommitMessage.test_commit_message_presence.fail)
XFAIL: SrcUri.test_src_uri_left_files (file: SrcUri.test_src_uri_left_files.fail)
XPASS: Author.test_author_valid (file: Author.test_author_valid.1.pass)
XFAIL: LicFilesChkSum.test_lic_files_chksum_modified_not_mentioned (file: LicFilesChkSum.test_lic_files_chksum_modified_not_mentioned.fail)
XPASS: CVE.test_cve_tag_format (file: CVE.test_cve_tag_format.pass)
XPASS: CVE.test_cve_presence_in_commit_message (file: CVE.test_cve_presence_in_commit_message.pass)
XFAIL: CVE.test_cve_tag_format (file: CVE.test_cve_tag_format.fail)
XFAIL: Author.test_author_valid (file: Author.test_author_valid.1.fail)
XFAIL: LicFilesChkSum.test_lic_files_chksum_presence (file: LicFilesChkSum.test_lic_files_chksum_presence.fail)
XSKIP: Merge.test_series_merge_on_head (file: Merge.test_series_merge_on_head.2.skip)
XPASS: MboxFormat.test_mbox_format (file: MboxFormat.test_mbox_format.pass)
XFAIL: SignedOffBy.test_signed_off_by_presence (file: SignedOffBy.test_signed_off_by_presence.1.fail)
XPASS: Shortlog.test_shortlog_format (file: Shortlog.test_shortlog_format.pass)
XFAIL: SignedOffBy.test_signed_off_by_presence (file: SignedOffBy.test_signed_off_by_presence.2.fail)
XFAIL: MboxFormat.test_mbox_format (file: MboxFormat.test_mbox_format.2.fail)
XFAIL: Summary.test_summary_presence (file: Summary.test_summary_presence.fail)
XPASS: Author.test_author_valid (file: Author.test_author_valid.2.pass)
XSKIP: Merge.test_series_merge_on_head (file: Merge.test_series_merge_on_head.1.skip)
XPASS: Bugzilla.test_bugzilla_entry_format (file: Bugzilla.test_bugzilla_entry_format.pass)
XFAIL: CVE.test_cve_presence_in_commit_message (file: CVE.test_cve_presence_in_commit_message.fail)
XPASS: SignedOffBy.test_signed_off_by_presence (file: SignedOffBy.test_signed_off_by_presence.pass)
XPASS: LicFilesChkSum.test_lic_files_chksum_presence (file: LicFilesChkSum.test_lic_files_chksum_presence.pass)
XPASS: CommitMessage.test_commit_message_presence (file: CommitMessage.test_commit_message_presence.pass)
XPASS: Summary.test_summary_presence (file: Summary.test_summary_presence.pass)
XPASS: LicFilesChkSum.test_lic_files_chksum_modified_not_mentioned (file: LicFilesChkSum.test_lic_files_chksum_modified_not_mentioned.pass)
XFAIL: Shortlog.test_shortlog_length (file: Shortlog.test_shortlog_length.fail)
XFAIL: PatchSignedOffBy.test_signed_off_by_presence (file: PatchSignedOffBy.test_signed_off_by_presence.fail)
XFAIL: Bugzilla.test_bugzilla_entry_format (file: Bugzilla.test_bugzilla_entry_format.fail)
XPASS: SrcUri.test_src_uri_left_files (file: SrcUri.test_src_uri_left_files.pass)
XFAIL: Author.test_author_valid (file: Author.test_author_valid.2.fail)
============================================================================
Testsuite summary for patchtest
============================================================================
============================================================================
real 24m14.386s user 1m13.599s sys 0m21.477s
After:
[tgamblin@megalith poky]$ time ./meta/lib/patchtest/selftest/selftest
XFAIL: TestMbox.test_bugzilla_entry_format (file: TestMbox.test_bugzilla_entry_format.fail)
XPASS: TestMetadata.test_summary_presence (file: TestMetadata.test_summary_presence.pass)
XFAIL: TestMbox.test_mbox_format (file: TestMbox.test_mbox_format.1.fail)
XFAIL: TestMetadata.test_src_uri_left_files (file: TestMetadata.test_src_uri_left_files.fail)
XSKIP: TestMbox.test_series_merge_on_head (file: TestMbox.test_series_merge_on_head.2.skip)
XPASS: TestMbox.test_commit_message_presence (file: TestMbox.test_commit_message_presence.pass)
XFAIL: TestMbox.test_commit_message_presence (file: TestMbox.test_commit_message_presence.fail)
XPASS: TestMbox.test_signed_off_by_presence (file: TestMbox.test_signed_off_by_presence.pass)
XFAIL: TestPatch.test_cve_tag_format (file: TestPatch.test_cve_tag_format.fail)
XFAIL: TestMbox.test_author_valid (file: TestMbox.test_author_valid.1.fail)
XFAIL: TestMbox.test_shortlog_length (file: TestMbox.test_shortlog_length.fail)
XPASS: TestMbox.test_mbox_format (file: TestMbox.test_mbox_format.pass)
XFAIL: TestPatch.test_signed_off_by_presence (file: TestPatch.test_signed_off_by_presence.fail)
XFAIL: TestMbox.test_shortlog_format (file: TestMbox.test_shortlog_format.fail)
XFAIL: TestMbox.test_mbox_format (file: TestMbox.test_mbox_format.2.fail)
XPASS: TestPatch.test_cve_tag_format (file: TestPatch.test_cve_tag_format.pass)
XSKIP: TestMbox.test_series_merge_on_head (file: TestMbox.test_series_merge_on_head.1.skip)
XPASS: TestMbox.test_author_valid (file: TestMbox.test_author_valid.2.pass)
XPASS: TestMetadata.test_lic_files_chksum_modified_not_mentioned (file: TestMetadata.test_lic_files_chksum_modified_not_mentioned.pass)
XPASS: TestMbox.test_bugzilla_entry_format (file: TestMbox.test_bugzilla_entry_format.pass)
XPASS: TestMetadata.test_src_uri_left_files (file: TestMetadata.test_src_uri_left_files.pass)
XPASS: TestMetadata.test_lic_files_chksum_presence (file: TestMetadata.test_lic_files_chksum_presence.pass)
XPASS: TestMbox.test_cve_presence_in_commit_message (file: TestMbox.test_cve_presence_in_commit_message.pass)
XFAIL: TestMbox.test_signed_off_by_presence (file: TestMbox.test_signed_off_by_presence.2.fail)
XFAIL: TestMbox.test_author_valid (file: TestMbox.test_author_valid.2.fail)
XFAIL: TestMetadata.test_lic_files_chksum_presence (file: TestMetadata.test_lic_files_chksum_presence.fail)
XPASS: TestMbox.test_shortlog_format (file: TestMbox.test_shortlog_format.pass)
XPASS: TestMbox.test_author_valid (file: TestMbox.test_author_valid.1.pass)
XPASS: TestPatch.test_signed_off_by_presence (file: TestPatch.test_signed_off_by_presence.pass)
XFAIL: TestMetadata.test_lic_files_chksum_modified_not_mentioned (file: TestMetadata.test_lic_files_chksum_modified_not_mentioned.fail)
XPASS: TestMbox.test_shortlog_length (file: TestMbox.test_shortlog_length.pass)
XFAIL: TestMbox.test_signed_off_by_presence (file: TestMbox.test_signed_off_by_presence.1.fail)
XFAIL: TestMbox.test_cve_presence_in_commit_message (file: TestMbox.test_cve_presence_in_commit_message.fail)
XFAIL: TestMetadata.test_summary_presence (file: TestMetadata.test_summary_presence.fail)
============================================================================
Testsuite summary for patchtest
============================================================================
============================================================================
real 18m39.749s user 0m41.857s sys 0m14.708s
---
...fail => TestMbox.test_author_valid.1.fail} | 0
...pass => TestMbox.test_author_valid.1.pass} | 0
...fail => TestMbox.test_author_valid.2.fail} | 0
...pass => TestMbox.test_author_valid.2.pass} | 0
... TestMbox.test_bugzilla_entry_format.fail} | 0
... TestMbox.test_bugzilla_entry_format.pass} | 0
...estMbox.test_commit_message_presence.fail} | 0
...estMbox.test_commit_message_presence.pass} | 0
....test_cve_presence_in_commit_message.fail} | 0
....test_cve_presence_in_commit_message.pass} | 0
....fail => TestMbox.test_mbox_format.1.fail} | 0
....fail => TestMbox.test_mbox_format.2.fail} | 0
...at.pass => TestMbox.test_mbox_format.pass} | 0
...TestMbox.test_series_merge_on_head.1.skip} | 0
...TestMbox.test_series_merge_on_head.2.skip} | 0
...ail => TestMbox.test_shortlog_format.fail} | 0
...ass => TestMbox.test_shortlog_format.pass} | 0
...ail => TestMbox.test_shortlog_length.fail} | 2 +-
...ass => TestMbox.test_shortlog_length.pass} | 0
...stMbox.test_signed_off_by_presence.1.fail} | 0
...stMbox.test_signed_off_by_presence.2.fail} | 0
...TestMbox.test_signed_off_by_presence.pass} | 0
..._files_chksum_modified_not_mentioned.fail} | 0
..._files_chksum_modified_not_mentioned.pass} | 0
...adata.test_lic_files_chksum_presence.fail} | 0
...adata.test_lic_files_chksum_presence.pass} | 0
...TestMetadata.test_src_uri_left_files.fail} | 0
...TestMetadata.test_src_uri_left_files.pass} | 0
...> TestMetadata.test_summary_presence.fail} | 0
...> TestMetadata.test_summary_presence.pass} | 0
...ail => TestPatch.test_cve_tag_format.fail} | 0
...ass => TestPatch.test_cve_tag_format.pass} | 0
...estPatch.test_signed_off_by_presence.fail} | 0
...estPatch.test_signed_off_by_presence.pass} | 0
meta/lib/patchtest/tests/test_mbox.py | 183 ++++++++++++++++
meta/lib/patchtest/tests/test_mbox_author.py | 29 ---
.../lib/patchtest/tests/test_mbox_bugzilla.py | 20 --
meta/lib/patchtest/tests/test_mbox_cve.py | 38 ----
.../patchtest/tests/test_mbox_description.py | 15 --
meta/lib/patchtest/tests/test_mbox_format.py | 14 --
.../patchtest/tests/test_mbox_mailinglist.py | 62 ------
meta/lib/patchtest/tests/test_mbox_merge.py | 27 ---
.../lib/patchtest/tests/test_mbox_shortlog.py | 39 ----
.../tests/test_mbox_signed_off_by.py | 27 ---
meta/lib/patchtest/tests/test_metadata.py | 204 ++++++++++++++++++
.../tests/test_metadata_lic_files_chksum.py | 74 -------
.../patchtest/tests/test_metadata_license.py | 50 -----
.../tests/test_metadata_max_length.py | 25 ---
.../patchtest/tests/test_metadata_src_uri.py | 73 -------
.../patchtest/tests/test_metadata_summary.py | 26 ---
...patch_upstream_status.py => test_patch.py} | 53 ++++-
meta/lib/patchtest/tests/test_patch_cve.py | 37 ----
.../tests/test_patch_signed_off_by.py | 41 ----
53 files changed, 436 insertions(+), 603 deletions(-)
rename meta/lib/patchtest/selftest/files/{Author.test_author_valid.1.fail => TestMbox.test_author_valid.1.fail} (100%)
rename meta/lib/patchtest/selftest/files/{Author.test_author_valid.1.pass => TestMbox.test_author_valid.1.pass} (100%)
rename meta/lib/patchtest/selftest/files/{Author.test_author_valid.2.fail => TestMbox.test_author_valid.2.fail} (100%)
rename meta/lib/patchtest/selftest/files/{Author.test_author_valid.2.pass => TestMbox.test_author_valid.2.pass} (100%)
rename meta/lib/patchtest/selftest/files/{Bugzilla.test_bugzilla_entry_format.fail => TestMbox.test_bugzilla_entry_format.fail} (100%)
rename meta/lib/patchtest/selftest/files/{Bugzilla.test_bugzilla_entry_format.pass => TestMbox.test_bugzilla_entry_format.pass} (100%)
rename meta/lib/patchtest/selftest/files/{CommitMessage.test_commit_message_presence.fail => TestMbox.test_commit_message_presence.fail} (100%)
rename meta/lib/patchtest/selftest/files/{CommitMessage.test_commit_message_presence.pass => TestMbox.test_commit_message_presence.pass} (100%)
rename meta/lib/patchtest/selftest/files/{CVE.test_cve_presence_in_commit_message.fail => TestMbox.test_cve_presence_in_commit_message.fail} (100%)
rename meta/lib/patchtest/selftest/files/{CVE.test_cve_presence_in_commit_message.pass => TestMbox.test_cve_presence_in_commit_message.pass} (100%)
rename meta/lib/patchtest/selftest/files/{MboxFormat.test_mbox_format.1.fail => TestMbox.test_mbox_format.1.fail} (100%)
rename meta/lib/patchtest/selftest/files/{MboxFormat.test_mbox_format.2.fail => TestMbox.test_mbox_format.2.fail} (100%)
rename meta/lib/patchtest/selftest/files/{MboxFormat.test_mbox_format.pass => TestMbox.test_mbox_format.pass} (100%)
rename meta/lib/patchtest/selftest/files/{Merge.test_series_merge_on_head.1.skip => TestMbox.test_series_merge_on_head.1.skip} (100%)
rename meta/lib/patchtest/selftest/files/{Merge.test_series_merge_on_head.2.skip => TestMbox.test_series_merge_on_head.2.skip} (100%)
rename meta/lib/patchtest/selftest/files/{Shortlog.test_shortlog_format.fail => TestMbox.test_shortlog_format.fail} (100%)
rename meta/lib/patchtest/selftest/files/{CVE.test_cve_tag_format.pass => TestMbox.test_shortlog_format.pass} (100%)
rename meta/lib/patchtest/selftest/files/{Shortlog.test_shortlog_length.fail => TestMbox.test_shortlog_length.fail} (97%)
rename meta/lib/patchtest/selftest/files/{Shortlog.test_shortlog_format.pass => TestMbox.test_shortlog_length.pass} (100%)
rename meta/lib/patchtest/selftest/files/{SignedOffBy.test_signed_off_by_presence.1.fail => TestMbox.test_signed_off_by_presence.1.fail} (100%)
rename meta/lib/patchtest/selftest/files/{SignedOffBy.test_signed_off_by_presence.2.fail => TestMbox.test_signed_off_by_presence.2.fail} (100%)
rename meta/lib/patchtest/selftest/files/{PatchSignedOffBy.test_signed_off_by_presence.pass => TestMbox.test_signed_off_by_presence.pass} (100%)
rename meta/lib/patchtest/selftest/files/{LicFilesChkSum.test_lic_files_chksum_modified_not_mentioned.fail => TestMetadata.test_lic_files_chksum_modified_not_mentioned.fail} (100%)
rename meta/lib/patchtest/selftest/files/{LicFilesChkSum.test_lic_files_chksum_modified_not_mentioned.pass => TestMetadata.test_lic_files_chksum_modified_not_mentioned.pass} (100%)
rename meta/lib/patchtest/selftest/files/{LicFilesChkSum.test_lic_files_chksum_presence.fail => TestMetadata.test_lic_files_chksum_presence.fail} (100%)
rename meta/lib/patchtest/selftest/files/{LicFilesChkSum.test_lic_files_chksum_presence.pass => TestMetadata.test_lic_files_chksum_presence.pass} (100%)
rename meta/lib/patchtest/selftest/files/{SrcUri.test_src_uri_left_files.fail => TestMetadata.test_src_uri_left_files.fail} (100%)
rename meta/lib/patchtest/selftest/files/{SrcUri.test_src_uri_left_files.pass => TestMetadata.test_src_uri_left_files.pass} (100%)
rename meta/lib/patchtest/selftest/files/{Summary.test_summary_presence.fail => TestMetadata.test_summary_presence.fail} (100%)
rename meta/lib/patchtest/selftest/files/{Summary.test_summary_presence.pass => TestMetadata.test_summary_presence.pass} (100%)
rename meta/lib/patchtest/selftest/files/{CVE.test_cve_tag_format.fail => TestPatch.test_cve_tag_format.fail} (100%)
rename meta/lib/patchtest/selftest/files/{Shortlog.test_shortlog_length.pass => TestPatch.test_cve_tag_format.pass} (100%)
rename meta/lib/patchtest/selftest/files/{PatchSignedOffBy.test_signed_off_by_presence.fail => TestPatch.test_signed_off_by_presence.fail} (100%)
rename meta/lib/patchtest/selftest/files/{SignedOffBy.test_signed_off_by_presence.pass => TestPatch.test_signed_off_by_presence.pass} (100%)
create mode 100644 meta/lib/patchtest/tests/test_mbox.py
delete mode 100644 meta/lib/patchtest/tests/test_mbox_author.py
delete mode 100644 meta/lib/patchtest/tests/test_mbox_bugzilla.py
delete mode 100644 meta/lib/patchtest/tests/test_mbox_cve.py
delete mode 100644 meta/lib/patchtest/tests/test_mbox_description.py
delete mode 100644 meta/lib/patchtest/tests/test_mbox_format.py
delete mode 100644 meta/lib/patchtest/tests/test_mbox_mailinglist.py
delete mode 100644 meta/lib/patchtest/tests/test_mbox_merge.py
delete mode 100644 meta/lib/patchtest/tests/test_mbox_shortlog.py
delete mode 100644 meta/lib/patchtest/tests/test_mbox_signed_off_by.py
create mode 100644 meta/lib/patchtest/tests/test_metadata.py
delete mode 100644 meta/lib/patchtest/tests/test_metadata_lic_files_chksum.py
delete mode 100644 meta/lib/patchtest/tests/test_metadata_license.py
delete mode 100644 meta/lib/patchtest/tests/test_metadata_max_length.py
delete mode 100644 meta/lib/patchtest/tests/test_metadata_src_uri.py
delete mode 100644 meta/lib/patchtest/tests/test_metadata_summary.py
rename meta/lib/patchtest/tests/{test_patch_upstream_status.py => test_patch.py} (56%)
delete mode 100644 meta/lib/patchtest/tests/test_patch_cve.py
delete mode 100644 meta/lib/patchtest/tests/test_patch_signed_off_by.pymeta/lib/patchtest/selftest/files/TestMbox.test_author_valid.1.fail renamed from meta/lib/patchtest/selftest/files/Author.test_author_valid.1.fail +0 -0
meta/lib/patchtest/selftest/files/TestMbox.test_author_valid.1.pass renamed from meta/lib/patchtest/selftest/files/Author.test_author_valid.1.pass +0 -0
meta/lib/patchtest/selftest/files/TestMbox.test_author_valid.2.fail renamed from meta/lib/patchtest/selftest/files/Author.test_author_valid.2.fail +0 -0
meta/lib/patchtest/selftest/files/TestMbox.test_author_valid.2.pass renamed from meta/lib/patchtest/selftest/files/Author.test_author_valid.2.pass +0 -0
meta/lib/patchtest/selftest/files/TestMbox.test_bugzilla_entry_format.fail renamed from meta/lib/patchtest/selftest/files/Bugzilla.test_bugzilla_entry_format.fail +0 -0
meta/lib/patchtest/selftest/files/TestMbox.test_bugzilla_entry_format.pass renamed from meta/lib/patchtest/selftest/files/Bugzilla.test_bugzilla_entry_format.pass +0 -0
meta/lib/patchtest/selftest/files/TestMbox.test_commit_message_presence.fail renamed from meta/lib/patchtest/selftest/files/CommitMessage.test_commit_message_presence.fail +0 -0
meta/lib/patchtest/selftest/files/TestMbox.test_commit_message_presence.pass renamed from meta/lib/patchtest/selftest/files/CommitMessage.test_commit_message_presence.pass +0 -0
meta/lib/patchtest/selftest/files/TestMbox.test_cve_presence_in_commit_message.fail renamed from meta/lib/patchtest/selftest/files/CVE.test_cve_presence_in_commit_message.fail +0 -0
meta/lib/patchtest/selftest/files/TestMbox.test_cve_presence_in_commit_message.pass renamed from meta/lib/patchtest/selftest/files/CVE.test_cve_presence_in_commit_message.pass +0 -0
meta/lib/patchtest/selftest/files/TestMbox.test_mbox_format.1.fail renamed from meta/lib/patchtest/selftest/files/MboxFormat.test_mbox_format.1.fail +0 -0
meta/lib/patchtest/selftest/files/TestMbox.test_mbox_format.2.fail renamed from meta/lib/patchtest/selftest/files/MboxFormat.test_mbox_format.2.fail +0 -0
meta/lib/patchtest/selftest/files/TestMbox.test_mbox_format.pass renamed from meta/lib/patchtest/selftest/files/MboxFormat.test_mbox_format.pass +0 -0
meta/lib/patchtest/selftest/files/TestMbox.test_series_merge_on_head.1.skip renamed from meta/lib/patchtest/selftest/files/Merge.test_series_merge_on_head.1.skip +0 -0
meta/lib/patchtest/selftest/files/TestMbox.test_series_merge_on_head.2.skip renamed from meta/lib/patchtest/selftest/files/Merge.test_series_merge_on_head.2.skip +0 -0
meta/lib/patchtest/selftest/files/TestMbox.test_shortlog_format.fail renamed from meta/lib/patchtest/selftest/files/Shortlog.test_shortlog_format.fail +0 -0
meta/lib/patchtest/selftest/files/TestMbox.test_shortlog_format.pass renamed from meta/lib/patchtest/selftest/files/CVE.test_cve_tag_format.pass +0 -0
meta/lib/patchtest/selftest/files/TestMbox.test_shortlog_length.fail renamed from meta/lib/patchtest/selftest/files/Shortlog.test_shortlog_length.fail +1 -1
@@ -56,7 +56,7 @@ index 547587bef4..76975a6729 100644 -SRC_URI = "file://helloworld.c" +SRC_URI = "file://helloworld.c \ -+ file://CVE-1234-56789.patch \ ++ file://0001-Fix-CVE-1234-56789.patch \ + " S = "${WORKDIR}"
meta/lib/patchtest/selftest/files/TestMbox.test_shortlog_length.pass renamed from meta/lib/patchtest/selftest/files/Shortlog.test_shortlog_format.pass +0 -0
meta/lib/patchtest/selftest/files/TestMbox.test_signed_off_by_presence.1.fail renamed from meta/lib/patchtest/selftest/files/SignedOffBy.test_signed_off_by_presence.1.fail +0 -0
meta/lib/patchtest/selftest/files/TestMbox.test_signed_off_by_presence.2.fail renamed from meta/lib/patchtest/selftest/files/SignedOffBy.test_signed_off_by_presence.2.fail +0 -0
meta/lib/patchtest/selftest/files/TestMbox.test_signed_off_by_presence.pass renamed from meta/lib/patchtest/selftest/files/PatchSignedOffBy.test_signed_off_by_presence.pass +0 -0
meta/lib/patchtest/selftest/files/TestMetadata.test_lic_files_chksum_modified_not_mentioned.fail renamed from meta/lib/patchtest/selftest/files/LicFilesChkSum.test_lic_files_chksum_modified_not_mentioned.fail +0 -0
meta/lib/patchtest/selftest/files/TestMetadata.test_lic_files_chksum_modified_not_mentioned.pass renamed from meta/lib/patchtest/selftest/files/LicFilesChkSum.test_lic_files_chksum_modified_not_mentioned.pass +0 -0
meta/lib/patchtest/selftest/files/TestMetadata.test_lic_files_chksum_presence.fail renamed from meta/lib/patchtest/selftest/files/LicFilesChkSum.test_lic_files_chksum_presence.fail +0 -0
meta/lib/patchtest/selftest/files/TestMetadata.test_lic_files_chksum_presence.pass renamed from meta/lib/patchtest/selftest/files/LicFilesChkSum.test_lic_files_chksum_presence.pass +0 -0
meta/lib/patchtest/selftest/files/TestMetadata.test_src_uri_left_files.fail renamed from meta/lib/patchtest/selftest/files/SrcUri.test_src_uri_left_files.fail +0 -0
meta/lib/patchtest/selftest/files/TestMetadata.test_src_uri_left_files.pass renamed from meta/lib/patchtest/selftest/files/SrcUri.test_src_uri_left_files.pass +0 -0
meta/lib/patchtest/selftest/files/TestMetadata.test_summary_presence.fail renamed from meta/lib/patchtest/selftest/files/Summary.test_summary_presence.fail +0 -0
meta/lib/patchtest/selftest/files/TestMetadata.test_summary_presence.pass renamed from meta/lib/patchtest/selftest/files/Summary.test_summary_presence.pass +0 -0
meta/lib/patchtest/selftest/files/TestPatch.test_cve_tag_format.fail renamed from meta/lib/patchtest/selftest/files/CVE.test_cve_tag_format.fail +0 -0
meta/lib/patchtest/selftest/files/TestPatch.test_cve_tag_format.pass renamed from meta/lib/patchtest/selftest/files/Shortlog.test_shortlog_length.pass +0 -0
meta/lib/patchtest/selftest/files/TestPatch.test_signed_off_by_presence.fail renamed from meta/lib/patchtest/selftest/files/PatchSignedOffBy.test_signed_off_by_presence.fail +0 -0
meta/lib/patchtest/selftest/files/TestPatch.test_signed_off_by_presence.pass renamed from meta/lib/patchtest/selftest/files/SignedOffBy.test_signed_off_by_presence.pass +0 -0
meta/lib/patchtest/tests/test_mbox.py new file +183 -0
183 lines not on this page · raw message
meta/lib/patchtest/tests/test_mbox_author.py deleted +0 -29
@@ -1,29 +0,0 @@ -# Checks related to the patch's author -# -# Copyright (C) 2016 Intel Corporation -# -# SPDX-License-Identifier: GPL-2.0-only - -import base -import pyparsing - -class Author(base.Base): - - auh_email = '[email protected]' - - invalids = [pyparsing.Regex("^Upgrade Helper.+"), - pyparsing.Regex(auh_email), - pyparsing.Regex("uh@not\.set"), - pyparsing.Regex("\S+@example\.com")] - - - def test_author_valid(self): - for commit in self.commits: - for invalid in self.invalids: - if invalid.search_string(commit.author): - self.fail('Invalid author %s. Resend the series with a valid patch author' % commit.author, commit=commit) - - def test_non_auh_upgrade(self): - for commit in self.commits: - if self.auh_email in commit.payload: - self.fail('Invalid author %s. Resend the series with a valid patch author' % self.auh_email, commit=commit)
meta/lib/patchtest/tests/test_mbox_bugzilla.py deleted +0 -20
@@ -1,20 +0,0 @@ -# Checks related to the patch's bugzilla tag -# -# Copyright (C) 2016 Intel Corporation -# -# SPDX-License-Identifier: GPL-2.0-only - -import pyparsing -import base - -class Bugzilla(base.Base): - rexp_detect = pyparsing.Regex('\[\s?YOCTO.*\]') - rexp_validation = pyparsing.Regex('\[(\s?YOCTO\s?#\s?(\d+)\s?,?)+\]') - - def test_bugzilla_entry_format(self): - for commit in Bugzilla.commits: - if not self.rexp_detect.search_string(commit.commit_message): - self.skip("No bug ID found") - elif not self.rexp_validation.search_string(commit.commit_message): - self.fail('Bugzilla issue ID is not correctly formatted - specify it with format: "[YOCTO #<bugzilla ID>]"', commit=commit) -
meta/lib/patchtest/tests/test_mbox_cve.py deleted +0 -38
@@ -1,38 +0,0 @@ -# Checks related to the patch's CVE lines -# -# Copyright (C) 2016 Intel Corporation -# -# SPDX-License-Identifier: GPL-2.0-only -# - -import base -import parse_cve_tags -import pyparsing - -class CVE(base.Base): - - revert_shortlog_regex = pyparsing.Regex('Revert\s+".*"') - prog = parse_cve_tags.cve_tag - patch_prog = parse_cve_tags.patch_cve_tag - - def setUp(self): - if self.unidiff_parse_error: - self.skip('Parse error %s' % self.unidiff_parse_error) - - # we are just interested in series that introduce CVE patches, thus discard other - # possibilities: modification to current CVEs, patch directly introduced into the - # recipe, upgrades already including the CVE, etc. - new_patches = [p for p in self.patchset if p.path.endswith('.patch') and p.is_added_file] - if not new_patches: - self.skip('No new patches introduced') - - def test_cve_presence_in_commit_message(self): - for commit in CVE.commits: - # skip those patches that revert older commits, these do not required the tag presence - if self.revert_shortlog_regex.search_string(commit.shortlog): - continue - if not self.patch_prog.search_string(commit.payload): - self.skip("No CVE tag in added patch, so not needed in mbox") - elif not self.prog.search_string(commit.payload): - self.fail('Missing or incorrectly formatted CVE tag in mbox. Correct or include the CVE tag in the mbox with format: "CVE: CVE-YYYY-XXXX"', - commit=commit)
meta/lib/patchtest/tests/test_mbox_description.py deleted +0 -15
@@ -1,15 +0,0 @@ -# Checks related to the patch's commit_message -# -# Copyright (C) 2016 Intel Corporation -# -# SPDX-License-Identifier: GPL-2.0-only - -import base - -class CommitMessage(base.Base): - - def test_commit_message_presence(self): - for commit in CommitMessage.commits: - if not commit.commit_message.strip(): - self.fail('Mbox is missing a descriptive commit message. Please include a commit message on your patch explaining the change', commit=commit) -
meta/lib/patchtest/tests/test_mbox_format.py deleted +0 -14
@@ -1,14 +0,0 @@ -# Checks correct parsing of mboxes -# -# Copyright (C) 2016 Intel Corporation -# -# SPDX-License-Identifier: GPL-2.0-only - -import base - -class MboxFormat(base.Base): - - def test_mbox_format(self): - if self.unidiff_parse_error: - self.fail('Series cannot be parsed correctly due to malformed diff lines. Create the series again using git-format-patch and ensure it can be applied using git am', - data=[('Diff line',self.unidiff_parse_error)])
meta/lib/patchtest/tests/test_mbox_mailinglist.py deleted +0 -62
62 lines not on this page · raw message
meta/lib/patchtest/tests/test_mbox_merge.py deleted +0 -27
@@ -1,27 +0,0 @@ -# Check if mbox was merged by patchtest -# -# Copyright (C) 2016 Intel Corporation -# -# SPDX-License-Identifier: GPL-2.0-only - -import subprocess -import base -from data import PatchTestInput - -def headlog(): - output = subprocess.check_output( - "cd %s; git log --pretty='%%h#%%aN#%%cD:#%%s' -1" % PatchTestInput.repodir, - universal_newlines=True, - shell=True - ) - return output.split('#') - -class Merge(base.Base): - def test_series_merge_on_head(self): - self.skip("Merge test is disabled for now") - if PatchTestInput.repo.branch != "master": - self.skip("Skipping merge test since patch is not intended for master branch. Target detected is %s" % PatchTestInput.repo.branch) - if not PatchTestInput.repo.ismerged: - commithash, author, date, shortlog = headlog() - self.fail('Series does not apply on top of target branch. Rebase your series and ensure the target is correct', - data=[('Targeted branch', '%s (currently at %s)' % (PatchTestInput.repo.branch, commithash))])
meta/lib/patchtest/tests/test_mbox_shortlog.py deleted +0 -39
@@ -1,39 +0,0 @@ -# Checks related to the patch's summary -# -# Copyright (C) 2016 Intel Corporation -# -# SPDX-License-Identifier: GPL-2.0-only - -import base -import parse_shortlog -import pyparsing - -maxlength = 90 - -class Shortlog(base.Base): - - def test_shortlog_format(self): - for commit in Shortlog.commits: - shortlog = commit.shortlog - if not shortlog.strip(): - self.skip('Empty shortlog, no reason to execute shortlog format test') - else: - # no reason to re-check on revert shortlogs - if shortlog.startswith('Revert "'): - continue - try: - parse_shortlog.shortlog.parseString(shortlog) - except pyparsing.ParseException as pe: - self.fail('Commit shortlog (first line of commit message) should follow the format "<target>: <summary>"', - commit=commit) - - def test_shortlog_length(self): - for commit in Shortlog.commits: - # no reason to re-check on revert shortlogs - shortlog = commit.shortlog - if shortlog.startswith('Revert "'): - continue - l = len(shortlog) - if l > maxlength: - self.fail('Edit shortlog so that it is %d characters or less (currently %d characters)' % (maxlength, l), - commit=commit)
meta/lib/patchtest/tests/test_mbox_signed_off_by.py deleted +0 -27
@@ -1,27 +0,0 @@ -# Checks related to the patch's signed-off-by lines -# -# Copyright (C) 2016 Intel Corporation -# -# SPDX-License-Identifier: GPL-2.0-only - -import base -import parse_signed_off_by -import pyparsing - -class SignedOffBy(base.Base): - - revert_shortlog_regex = pyparsing.Regex('Revert\s+".*"') - - @classmethod - def setUpClassLocal(cls): - # match self.mark with no '+' preceding it - cls.prog = parse_signed_off_by.signed_off_by - - def test_signed_off_by_presence(self): - for commit in SignedOffBy.commits: - # skip those patches that revert older commits, these do not required the tag presence - if self.revert_shortlog_regex.search_string(commit.shortlog): - continue - if not SignedOffBy.prog.search_string(commit.payload): - self.fail('Mbox is missing Signed-off-by. Add it manually or with "git commit --amend -s"', - commit=commit)
meta/lib/patchtest/tests/test_metadata.py new file +204 -0
204 lines not on this page · raw message
meta/lib/patchtest/tests/test_metadata_lic_files_chksum.py deleted +0 -74
74 lines not on this page · raw message
meta/lib/patchtest/tests/test_metadata_license.py deleted +0 -50
50 lines not on this page · raw message
meta/lib/patchtest/tests/test_metadata_max_length.py deleted +0 -25
@@ -1,25 +0,0 @@ -# Checks related to patch line lengths -# -# Copyright (C) 2016 Intel Corporation -# -# SPDX-License-Identifier: GPL-2.0-only - -import base -import pyparsing - -class MaxLength(base.Base): - add_mark = pyparsing.Regex('\+ ') - max_length = 200 - - def test_max_line_length(self): - for patch in self.patchset: - # for the moment, we are just interested in metadata - if patch.path.endswith('.patch'): - continue - payload = str(patch) - for line in payload.splitlines(): - if self.add_mark.search_string(line): - current_line_length = len(line[1:]) - if current_line_length > self.max_length: - self.fail('Patch line too long (current length %s, maximum is %s)' % (current_line_length, self.max_length), - data=[('Patch', patch.path), ('Line', '%s ...' % line[0:80])])
meta/lib/patchtest/tests/test_metadata_src_uri.py deleted +0 -73
73 lines not on this page · raw message
meta/lib/patchtest/tests/test_metadata_summary.py deleted +0 -26
@@ -1,26 +0,0 @@ -# Checks related to the patch's summary metadata variable -# -# Copyright (C) 2016 Intel Corporation -# -# SPDX-License-Identifier: GPL-2.0-only - -import base -from data import PatchTestInput - -class Summary(base.Metadata): - metadata = 'SUMMARY' - - def test_summary_presence(self): - if not self.added: - self.skip('No added recipes, skipping test') - - for pn in self.added: - # we are not interested in images - if 'core-image' in pn: - continue - rd = self.tinfoil.parse_recipe(pn) - summary = rd.getVar(self.metadata) - - # "${PN} version ${PN}-${PR}" is the default, so fail if default - if summary.startswith('%s version' % pn): - self.fail('%s is missing in newly added recipe' % self.metadata)
meta/lib/patchtest/tests/test_patch.py renamed from meta/lib/patchtest/tests/test_patch_upstream_status.py +48 -5
83 lines not on this page · raw message
meta/lib/patchtest/tests/test_patch_cve.py deleted +0 -37
@@ -1,37 +0,0 @@ -# Checks related to the patch's CVE lines -# -# Copyright (C) 2016 Intel Corporation -# -# SPDX-License-Identifier: GPL-2.0-only -# - -import base -import pyparsing - -class CVE(base.Base): - - re_cve_pattern = pyparsing.Regex("CVE\-\d{4}\-\d+") - re_cve_payload_tag = pyparsing.Regex("\+CVE:(\s+CVE\-\d{4}\-\d+)+") - - def setUp(self): - if self.unidiff_parse_error: - self.skip('Parse error %s' % self.unidiff_parse_error) - - # we are just interested in series that introduce CVE patches, thus discard other - # possibilities: modification to current CVEs, patch directly introduced into the - # recipe, upgrades already including the CVE, etc. - new_cves = [p for p in self.patchset if p.path.endswith('.patch') and p.is_added_file] - if not new_cves: - self.skip('No new CVE patches introduced') - - def test_cve_tag_format(self): - for commit in CVE.commits: - if self.re_cve_pattern.search_string(commit.shortlog) or self.re_cve_pattern.search_string(commit.commit_message): - tag_found = False - for line in commit.payload.splitlines(): - if self.re_cve_payload_tag.search_string(line): - tag_found = True - break - if not tag_found: - self.fail('Missing or incorrectly formatted CVE tag in patch file. Correct or include the CVE tag in the patch with format: "CVE: CVE-YYYY-XXXX"', - commit=commit)
meta/lib/patchtest/tests/test_patch_signed_off_by.py deleted +0 -41
41 lines not on this page · raw message
Signature · 1 line
2.34.1
PATCH 26/33 patchtest: reduce checksum test output length
In reply to Steve SakomanFrom: Trevor Gamblin <[email protected]>
The test_lic_files_chksum_modified_not_mentioned test in TestMetadata is outputting very long lines that fail the maximum length check when sending email results, preventing the actual errors from being displayed. Reduce the length of the failure message by rewording and removing redundant information.
--- meta/lib/patchtest/tests/test_metadata.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
meta/lib/patchtest/tests/test_metadata.py +1 -1
@@ -116,7 +116,7 @@ class TestMetadata(base.Metadata): if self.lictag_re.search_string(commit.commit_message): break else: - self.fail('LIC_FILES_CHKSUM changed on target %s but there is no "License-Update:" tag in commit message. Include it with a brief description' % pn, + self.fail('LIC_FILES_CHKSUM changed without "License-Update:" tag and description in commit message', data=[('Current checksum', pretest), ('New checksum', test)]) def test_max_line_length(self):
Signature · 1 line
2.34.1
PATCH 27/33 patchtest: shorten test result outputs
In reply to Steve SakomanFrom: Trevor Gamblin <[email protected]>
Some test result lines in TestMbox and TestPatch are still too long to avoid being flagged by the mailer script. Clean them up by removing redundant information, so that they are all under the length limit of 220 characters.
--- meta/lib/patchtest/tests/test_mbox.py | 14 +++++++------- meta/lib/patchtest/tests/test_patch.py | 4 ++-- 2 files changed, 9 insertions(+), 9 deletions(-)
meta/lib/patchtest/tests/test_mbox.py +7 -7
@@ -100,7 +100,7 @@ class TestMbox(base.Base): self.skip("Skipping merge test since patch is not intended for master branch. Target detected is %s" % PatchTestInput.repo.branch) if not PatchTestInput.repo.ismerged: commithash, author, date, shortlog = headlog() - self.fail('Series does not apply on top of target branch. Rebase your series and ensure the target is correct', + self.fail('Series does not apply on top of target branch %s' % PatchTestInput.repo.branch, data=[('Targeted branch', '%s (currently at %s)' % (PatchTestInput.repo.branch, commithash))]) def test_target_mailing_list(self):
@@ -114,7 +114,7 @@ class TestMbox(base.Base): for commit in TestMbox.commits: match = project_regex.search_string(commit.subject) if match: - self.fail('Series sent to the wrong mailing list. Check the project\'s README (%s) and send the patch to the indicated list' % match.group('project'), + self.fail('Series sent to the wrong mailing list or some patches from the series correspond to different mailing lists', commit=commit) for patch in self.patchset:
@@ -122,7 +122,7 @@ class TestMbox(base.Base): base_path = folders[0] for project in [self.bitbake, self.doc, self.oe, self.poky]: if base_path in project.paths: - self.fail('Series sent to the wrong mailing list or some patches from the series correspond to different mailing lists. Send the series again to the correct mailing list (ML)', + self.fail('Series sent to the wrong mailing list or some patches from the series correspond to different mailing lists', data=[('Suggested ML', '%s [%s]' % (project.listemail, project.gitrepo)), ('Patch\'s path:', patch.path)])
@@ -130,18 +130,18 @@ class TestMbox(base.Base): if base_path.startswith('scripts'): for poky_file in self.poky_scripts: if patch.path.startswith(poky_file): - self.fail('Series sent to the wrong mailing list or some patches from the series correspond to different mailing lists. Send the series again to the correct mailing list (ML)', + self.fail('Series sent to the wrong mailing list or some patches from the series correspond to different mailing lists', data=[('Suggested ML', '%s [%s]' % (self.poky.listemail, self.poky.gitrepo)),('Patch\'s path:', patch.path)]) def test_mbox_format(self): if self.unidiff_parse_error: - self.fail('Series cannot be parsed correctly due to malformed diff lines. Create the series again using git-format-patch and ensure it can be applied using git am', + self.fail('Series has malformed diff lines. Create the series again using git-format-patch and ensure it applies using git am', data=[('Diff line',self.unidiff_parse_error)]) def test_commit_message_presence(self): for commit in TestMbox.commits: if not commit.commit_message.strip(): - self.fail('Mbox is missing a descriptive commit message. Please include a commit message on your patch explaining the change', commit=commit) + self.fail('Please include a commit message on your patch explaining the change', commit=commit) def test_cve_presence_in_commit_message(self): if self.unidiff_parse_error:
@@ -161,7 +161,7 @@ class TestMbox(base.Base): if not self.patch_prog.search_string(commit.payload): self.skip("No CVE tag in added patch, so not needed in mbox") elif not self.prog.search_string(commit.payload): - self.fail('Missing or incorrectly formatted CVE tag in mbox. Correct or include the CVE tag in the mbox with format: "CVE: CVE-YYYY-XXXX"', + self.fail('A CVE tag should be provided in the commit message with format: "CVE: CVE-YYYY-XXXX"', commit=commit) def test_bugzilla_entry_format(self):
meta/lib/patchtest/tests/test_patch.py +2 -2
@@ -50,7 +50,7 @@ class TestPatch(base.Base): for newpatch in TestPatch.newpatches: payload = newpatch.__str__() if not self.upstream_status_regex.search_string(payload): - self.fail('Added patch file is missing Upstream-Status in the header. Add Upstream-Status: <Valid status> to the header', + self.fail('Added patch file is missing Upstream-Status: <Valid status> in the commit message', data=[('Standard format', self.standard_format), ('Valid status', self.valid_status)]) for line in payload.splitlines(): if self.patchmetadata_regex.match(line):
@@ -87,7 +87,7 @@ class TestPatch(base.Base): if TestPatch.prog.search_string(payload): break else: - self.fail('A patch file has been added, but does not have a Signed-off-by tag. Sign off the added patch file (%s)' % newpatch.path) + self.fail('A patch file has been added without a Signed-off-by tag. Sign off the added patch file (%s)' % newpatch.path) def test_cve_tag_format(self): for commit in TestPatch.commits:
Signature · 1 line
2.34.1
PATCH 28/33 patchtest-send-results: send results to submitter
In reply to Steve SakomanFrom: Trevor Gamblin <[email protected]>
Modify patchtest-send-results so that it extracts the submitter's email address and responds to them with the patch testresults. Also make a minor adjustment to the suggestions provided with each email and include a link to the Patchtest wiki page for additional clarification on specific failures.
--- scripts/patchtest-send-results | 19 +++++++++++++------ 1 file changed, 13 insertions(+), 6 deletions(-)
scripts/patchtest-send-results +13 -6
@@ -17,6 +17,7 @@ import boto3 import configparser import mailbox import os +import re import sys greeting = """Thank you for your submission. Patchtest identified one
@@ -25,10 +26,12 @@ more information:\n\n---\n""" suggestions = """\n---\n\nPlease address the issues identified and submit a new revision of the patch, or alternatively, reply to this -email with an explanation of why the patch format should be accepted. If -you believe these results are due to an error in patchtest, please -submit a bug at https://bugzilla.yoctoproject.org/ (use the 'Patchtest' -category under 'Yocto Project Subprojects'). Thank you!""" +email with an explanation of why the patch should be accepted. If you +believe these results are due to an error in patchtest, please submit a +bug at https://bugzilla.yoctoproject.org/ (use the 'Patchtest' category +under 'Yocto Project Subprojects'). For more information on specific +failures, see: https://wiki.yoctoproject.org/wiki/Patchtest. Thank +you!""" parser = argparse.ArgumentParser(description="Send patchtest results to a submitter for a given patch") parser.add_argument("-p", "--patch", dest="patch", required=True, help="The patch file to summarize")
@@ -54,6 +57,10 @@ mbox = mailbox.mbox(args.patch) mbox_subject = mbox[0]['subject'] subject_line = f"Patchtest results for {mbox_subject}" +# extract the submitter email address and use it as the reply address +# for the results +reply_address = re.findall("<(.*)>", mbox[0]['from']) + if "FAIL" in testresult: reply_contents = None if len(max(open(result_file, 'r'), key=len)) > 220:
@@ -66,9 +73,9 @@ if "FAIL" in testresult: response = ses_client.send_email( Source='[email protected]', Destination={ - 'ToAddresses': ['[email protected]'], + 'CcAddresses': ['[email protected]'], }, - ReplyToAddresses=['[email protected]'], + ReplyToAddresses=reply_address, Message={ 'Subject': { 'Data': subject_line,
Signature · 1 line
2.34.1
PATCH 29/33 patchtest-send-results: add In-Reply-To
In reply to Steve SakomanFrom: Trevor Gamblin <[email protected]>
Rework the script for sending results to use send_raw_email and specify the 'In-Reply-To' field so that patchtest replies to the emails, rather than sending them standalone to the submitter and mailing list.
--- scripts/patchtest-send-results | 39 ++++++++++++++++++---------------- 1 file changed, 21 insertions(+), 18 deletions(-)
scripts/patchtest-send-results +21 -18
@@ -59,7 +59,16 @@ subject_line = f"Patchtest results for {mbox_subject}" # extract the submitter email address and use it as the reply address # for the results -reply_address = re.findall("<(.*)>", mbox[0]['from']) +reply_address = re.findall("<(.*)>", mbox[0]['from'])[0] + +# extract the message ID and use that as the in-reply-to address +in_reply_to = re.findall("<(.*)>", mbox[0]['Message-ID'])[0] + +# the address the results email is sent from +from_address = "[email protected]" + +# mailing list to CC +cc_address = "[email protected]" if "FAIL" in testresult: reply_contents = None
@@ -70,24 +79,18 @@ if "FAIL" in testresult: reply_contents = greeting + testresult + suggestions ses_client = boto3.client('ses', region_name='us-west-2') - response = ses_client.send_email( - Source='[email protected]', - Destination={ - 'CcAddresses': ['[email protected]'], + raw_data = 'From: ' + from_address + '\nTo: ' + reply_address + \ + '\nCC: ' + cc_address + '\nSubject:' + subject_line + \ + '\nIn-Reply-To:' + in_reply_to + \ + '\nMIME-Version: 1.0" + \ + "\nContent-type: Multipart/Mixed;boundary="NextPart"\n\n--NextPart\nContent-Type: text/plain\n\n' + \ + reply_contents + '\n\n--NextPart' + response = ses_client.send_raw_email( + Source="[email protected]", + RawMessage={ + "Data": raw_data, }, - ReplyToAddresses=reply_address, - Message={ - 'Subject': { - 'Data': subject_line, - 'Charset': 'utf-8' - }, - 'Body': { - 'Text': { - 'Data': reply_contents, - 'Charset': 'utf-8' - } - } - } ) + else: print(f"No failures identified for {args.patch}.")
Signature · 1 line
2.34.1
PATCH 30/33 patchtest: make pylint tests compatible with 3.x
In reply to Steve SakomanFrom: Trevor Gamblin <[email protected]>
pylint 3.x has removed epylint, which is now a separate module. To avoid adding another recipe or using outdated modules, modify the test_python_pylint tests so that they use the standard pylint API.
--- meta/lib/patchtest/tests/test_python_pylint.py | 17 ++++++++++++----- 1 file changed, 12 insertions(+), 5 deletions(-)
meta/lib/patchtest/tests/test_python_pylint.py +12 -5
@@ -5,8 +5,11 @@ # SPDX-License-Identifier: GPL-2.0-only import base +from io import StringIO from data import PatchTestInput -import pylint.epylint as lint +from pylint.reporters.text import TextReporter +import pylint.lint as lint + class PyLint(base.Base): pythonpatches = []
@@ -32,8 +35,10 @@ class PyLint(base.Base): def pretest_pylint(self): for pythonpatch in self.pythonpatches: if pythonpatch.is_modified_file: - (pylint_stdout, pylint_stderr) = lint.py_run(command_options = pythonpatch.path + self.pylint_options, return_std=True) - for line in pylint_stdout.readlines(): + pylint_output = StringIO() + reporter = TextReporter(pylint_output) + lint.Run([self.pylint_options, pythonpatch.path], reporter=reporter, exit=False) + for line in pylint_output.readlines(): if not '*' in line: if line.strip(): self.pylint_pretest[line.strip().split(' ',1)[0]] = line.strip().split(' ',1)[1]
@@ -46,8 +51,10 @@ class PyLint(base.Base): path = pythonpatch.target_file[2:] else: path = pythonpatch.path - (pylint_stdout, pylint_stderr) = lint.py_run(command_options = path + self.pylint_options, return_std=True) - for line in pylint_stdout.readlines(): + pylint_output = StringIO() + reporter = TextReporter(pylint_output) + lint.Run([self.pylint_options, pythonpatch.path], reporter=reporter, exit=False) + for line in pylint_output.readlines(): if not '*' in line: if line.strip(): self.pylint_test[line.strip().split(' ',1)[0]] = line.strip().split(' ',1)[1]
Signature · 1 line
2.34.1
PATCH 31/33 patchtest: remove test for CVE tag in mbox
In reply to Steve SakomanFrom: Trevor Gamblin <[email protected]>
After patchtest went live it was determined that testing for a CVE tag in the mbox commit message is unnecessary, since it will already be in the shortlog and in any carried patches. Remove the test and the associated selftest files so that its absence isn't flagged in future test results.
--- ...x.test_cve_presence_in_commit_message.fail | 72 ------------------ ...x.test_cve_presence_in_commit_message.pass | 74 ------------------- meta/lib/patchtest/tests/test_mbox.py | 24 ------ 3 files changed, 170 deletions(-) delete mode 100644 meta/lib/patchtest/selftest/files/TestMbox.test_cve_presence_in_commit_message.fail delete mode 100644 meta/lib/patchtest/selftest/files/TestMbox.test_cve_presence_in_commit_message.pass
meta/lib/patchtest/selftest/files/TestMbox.test_cve_presence_in_commit_message.fail deleted +0 -72
@@ -1,72 +0,0 @@ -From 14d72f6973270f78455a8628143f2cff90e8f41e Mon Sep 17 00:00:00 2001 -From: Trevor Gamblin <[email protected]> -Date: Tue, 29 Aug 2023 14:12:27 -0400 -Subject: [PATCH] selftest-hello: fix CVE-1234-56789 - -This patch should fail the test for CVE presence in the mbox commit message. - -Signed-off-by: Trevor Gamblin <[email protected]> ---- - .../selftest-hello/files/CVE-1234-56789.patch | 27 +++++++++++++++++++ - .../selftest-hello/selftest-hello_1.0.bb | 6 +++-- - 2 files changed, 31 insertions(+), 2 deletions(-) - create mode 100644 meta-selftest/recipes-test/selftest-hello/files/CVE-1234-56789.patch - -diff --git a/meta-selftest/recipes-test/selftest-hello/files/CVE-1234-56789.patch b/meta-selftest/recipes-test/selftest-hello/files/CVE-1234-56789.patch -new file mode 100644 -index 0000000000..869cfb6fe5 ---- /dev/null -+++ b/meta-selftest/recipes-test/selftest-hello/files/CVE-1234-56789.patch -@@ -0,0 +1,27 @@ -+From b26a31186e6ee2eb1f506d5f2f9394d327a0df2f Mon Sep 17 00:00:00 2001 -+From: Trevor Gamblin <[email protected]> -+Date: Tue, 29 Aug 2023 14:08:20 -0400 -+Subject: [PATCH] Fix CVE-NOT-REAL -+ -+CVE: CVE-1234-56789 -+Upstream-Status: Backport(http://example.com/example) -+ -+Signed-off-by: Trevor Gamblin <[email protected]> -+--- -+ strlen.c | 1 + -+ 1 file changed, 1 insertion(+) -+ -+diff --git a/strlen.c b/strlen.c -+index 1788f38..83d7918 100644 -+--- a/strlen.c -++++ b/strlen.c -+@@ -8,6 +8,7 @@ int main() { -+ -+ printf("%d\n", str_len(string1)); -+ printf("%d\n", str_len(string2)); -++ printf("CVE FIXED!!!\n"); -+ -+ return 0; -+ } -+-- -+2.41.0 -diff --git a/meta-selftest/recipes-test/selftest-hello/selftest-hello_1.0.bb b/meta-selftest/recipes-test/selftest-hello/selftest-hello_1.0.bb -index 547587bef4..76975a6729 100644 ---- a/meta-selftest/recipes-test/selftest-hello/selftest-hello_1.0.bb -+++ b/meta-selftest/recipes-test/selftest-hello/selftest-hello_1.0.bb -@@ -3,7 +3,9 @@ SECTION = "examples" - LICENSE = "MIT" - LIC_FILES_CHKSUM = "file://${COMMON_LICENSE_DIR}/MIT;md5=0835ade698e0bcf8506ecda2f7b4f302" - --SRC_URI = "file://helloworld.c" -+SRC_URI = "file://helloworld.c \ -+ file://CVE-1234-56789.patch \ -+ " - - S = "${WORKDIR}" - -@@ -16,4 +18,4 @@ do_install() { - install -m 0755 helloworld ${D}${bindir} - } - --BBCLASSEXTEND = "native nativesdk" -\ No newline at end of file -+BBCLASSEXTEND = "native nativesdk" --- -2.41.0 -
meta/lib/patchtest/selftest/files/TestMbox.test_cve_presence_in_commit_message.pass deleted +0 -74
74 lines not on this page · raw message
meta/lib/patchtest/tests/test_mbox.py +0 -24
@@ -6,7 +6,6 @@ import base import collections -import parse_cve_tags import parse_shortlog import parse_signed_off_by import pyparsing
@@ -33,8 +32,6 @@ class TestMbox(base.Base): rexp_detect = pyparsing.Regex('\[\s?YOCTO.*\]') rexp_validation = pyparsing.Regex('\[(\s?YOCTO\s?#\s?(\d+)\s?,?)+\]') revert_shortlog_regex = pyparsing.Regex('Revert\s+".*"') - prog = parse_cve_tags.cve_tag - patch_prog = parse_cve_tags.patch_cve_tag signoff_prog = parse_signed_off_by.signed_off_by revert_shortlog_regex = pyparsing.Regex('Revert\s+".*"') maxlength = 90
@@ -143,27 +140,6 @@ class TestMbox(base.Base): if not commit.commit_message.strip(): self.fail('Please include a commit message on your patch explaining the change', commit=commit) - def test_cve_presence_in_commit_message(self): - if self.unidiff_parse_error: - self.skip('Parse error %s' % self.unidiff_parse_error) - - # we are just interested in series that introduce CVE patches, thus discard other - # possibilities: modification to current CVEs, patch directly introduced into the - # recipe, upgrades already including the CVE, etc. - new_patches = [p for p in self.patchset if p.path.endswith('.patch') and p.is_added_file] - if not new_patches: - self.skip('No new patches introduced') - - for commit in TestMbox.commits: - # skip those patches that revert older commits, these do not required the tag presence - if self.revert_shortlog_regex.search_string(commit.shortlog): - continue - if not self.patch_prog.search_string(commit.payload): - self.skip("No CVE tag in added patch, so not needed in mbox") - elif not self.prog.search_string(commit.payload): - self.fail('A CVE tag should be provided in the commit message with format: "CVE: CVE-YYYY-XXXX"', - commit=commit) - def test_bugzilla_entry_format(self): for commit in TestMbox.commits: if not self.rexp_detect.search_string(commit.commit_message):
Signature · 1 line
2.34.1
PATCH 32/33 patchtest-send-results: fix sender parsing
In reply to Steve SakomanFrom: Trevor Gamblin <[email protected]>
Not all mbox 'from' fields will contain angle brackets, so the re.findall invocation used for getting a reply_address may fail. Use a simpler reference to the field to get the sender's email address.
--- scripts/patchtest-send-results | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
scripts/patchtest-send-results +1 -1
@@ -59,7 +59,7 @@ subject_line = f"Patchtest results for {mbox_subject}" # extract the submitter email address and use it as the reply address # for the results -reply_address = re.findall("<(.*)>", mbox[0]['from'])[0] +reply_address = mbox[0]['from'] # extract the message ID and use that as the in-reply-to address in_reply_to = re.findall("<(.*)>", mbox[0]['Message-ID'])[0]
Signature · 1 line
2.34.1
PATCH 33/33 patchtest: rework license checksum tests
In reply to Steve SakomanFrom: Trevor Gamblin <[email protected]>
Remove the pretest_lic_files_chksum_modified_not_mentioned test entirely and use pyparsing in test_lic_files_chksum_modified_not_mentioned to scan the patches for lines starting with either "+LIC_FILES_CHKSUM" or "-LIC_FILES_CHKSUM". If either is found but no "License-Update" tag is present in the commit, fail the test.
--- meta/lib/patchtest/tests/test_metadata.py | 40 +++++------------------ 1 file changed, 8 insertions(+), 32 deletions(-)
meta/lib/patchtest/tests/test_metadata.py +8 -32
@@ -16,6 +16,8 @@ class TestMetadata(base.Metadata): license_var = 'LICENSE' closed = 'CLOSED' lictag_re = pyparsing.AtLineStart("License-Update:") + lic_chksum_added = pyparsing.AtLineStart("+" + metadata_chksum) + lic_chksum_removed = pyparsing.AtLineStart("-" + metadata_chksum) add_mark = pyparsing.Regex('\+ ') max_length = 200 metadata_src_uri = 'SRC_URI'
@@ -76,48 +78,22 @@ class TestMetadata(base.Metadata): if not lic_files_chksum: self.fail('%s is missing in newly added recipe' % self.metadata_chksum) - def pretest_lic_files_chksum_modified_not_mentioned(self): - if not self.modified: - self.skip('No modified recipes, skipping pretest') - # get the proper metadata values - for pn in self.modified: - rd = self.tinfoil.parse_recipe(pn) - pathname = rd.getVar('FILE') - # we are not interested in images - if '/images/' in pathname: - continue - PatchTestDataStore['%s-%s-%s' % (self.shortid(),self.metadata_chksum,pn)] = rd.getVar(self.metadata_chksum) - def test_lic_files_chksum_modified_not_mentioned(self): if not self.modified: self.skip('No modified recipes, skipping test') - # get the proper metadata values - for pn in self.modified: - rd = self.tinfoil.parse_recipe(pn) - pathname = rd.getVar('FILE') - # we are not interested in images - if '/images/' in pathname: + for patch in self.patchset: + # for the moment, we are just interested in metadata + if patch.path.endswith('.patch'): continue - PatchTestDataStore['%s-%s-%s' % (self.shortid(),self.metadata_chksum,pn)] = rd.getVar(self.metadata_chksum) - # compare if there were changes between pre-merge and merge - for pn in self.modified: - pretest = PatchTestDataStore['pre%s-%s-%s' % (self.shortid(),self.metadata_chksum, pn)] - test = PatchTestDataStore['%s-%s-%s' % (self.shortid(),self.metadata_chksum, pn)] - - # TODO: this is workaround to avoid false-positives when pretest metadata is empty (not reason found yet) - # For more info, check bug 12284 - if not pretest: - return - - if pretest != test: + payload = str(patch) + if (self.lic_chksum_added.search_string(payload) or self.lic_chksum_removed.search_string(payload)): # if any patch on the series contain reference on the metadata, fail for commit in self.commits: if self.lictag_re.search_string(commit.commit_message): break else: - self.fail('LIC_FILES_CHKSUM changed without "License-Update:" tag and description in commit message', - data=[('Current checksum', pretest), ('New checksum', test)]) + self.fail('LIC_FILES_CHKSUM changed without "License-Update:" tag and description in commit message') def test_max_line_length(self): for patch in self.patchset:
Signature · 1 line
2.34.1