libsoup: add ptest support
PATCH 1/1 libsoup: add ptest support
Build the installed-tests via the meson tests/installed_tests options (enabled through a new 'tests' PACKAGECONFIG when ptest is in DISTRO_FEATURES) and run them under ptest with gnome-desktop-testing-runner.
header-parsing-test is excluded as it aborts in do_param_list_tests() with the current glib; the rest of the suite passes.
Tested on qemux86-64 with ptest-runner: 34 tests, all pass.
--- .../distro/include/ptest-packagelists.inc | 1 + .../recipes-support/libsoup/libsoup/run-ptest | 3 +++ meta/recipes-support/libsoup/libsoup_3.6.6.bb | 22 +++++++++++++++---- 3 files changed, 22 insertions(+), 4 deletions(-) create mode 100644 meta/recipes-support/libsoup/libsoup/run-ptest
meta/conf/distro/include/ptest-packagelists.inc +1 -0
@@ -138,6 +138,7 @@ PTESTS_SLOW = "\ libmodule-build-perl \ libpcap \ libpng \ + libsoup \ ${@bb.utils.contains('DISTRO_FEATURES', 'seccomp', 'libseccomp', '',d)} \ lttng-tools \ lz4 \
meta/recipes-support/libsoup/libsoup/run-ptest new file +3 -0
@@ -0,0 +1,3 @@ +#! /bin/sh + +gnome-desktop-testing-runner libsoup-3.0
meta/recipes-support/libsoup/libsoup_3.6.6.bb +18 -4
@@ -9,7 +9,7 @@ LIC_FILES_CHKSUM = "file://COPYING;md5=5f30f0716dfdd0d91eb439ebec522ec2" DEPENDS = "glib-2.0 glib-2.0-native libxml2 sqlite3 libpsl nghttp2" -inherit gettext gnomebase upstream-version-is-even gobject-introspection gi-docgen vala +inherit gettext gnomebase upstream-version-is-even gobject-introspection gi-docgen vala ptest-gnome SRC_URI[archive.sha256sum] = "51ed0ae06f9d5a40f401ff459e2e5f652f9a510b7730e1359ee66d14d4872740"
@@ -21,6 +21,7 @@ SRC_URI += "file://CVE-2025-32049-1.patch \ file://CVE-2026-5119.patch \ file://CVE-2026-2708.patch \ file://CVE-2026-4271.patch \ + file://run-ptest \ " PROVIDES = "libsoup-3.0"
@@ -28,11 +29,12 @@ PROVIDES = "libsoup-3.0" GIR_MESON_ENABLE_FLAG = 'enabled' GIR_MESON_DISABLE_FLAG = 'disabled' -PACKAGECONFIG ??= "" +PACKAGECONFIG ??= "${@bb.utils.contains('PTEST_ENABLED', '1', 'tests', '', d)}" PACKAGECONFIG[brotli] = "-Dbrotli=enabled,-Dbrotli=disabled,brotli" PACKAGECONFIG[gssapi] = "-Dgssapi=enabled,-Dgssapi=disabled,krb5" PACKAGECONFIG[ntlm] = "-Dntlm=enabled,-Dntlm=disabled" PACKAGECONFIG[sysprof] = "-Dsysprof=enabled,-Dsysprof=disabled,sysprof" +PACKAGECONFIG[tests] = "-Dtests=true -Dinstalled_tests=true,-Dtests=false -Dinstalled_tests=false" # Tell libsoup where the target ntlm_auth is installed do_write_config:append:class-target() {
@@ -44,13 +46,21 @@ EOF EXTRA_OEMESON:append:class-target = " --cross-file ${WORKDIR}/soup.cross" EXTRA_OEMESON += "-Dtls_check=false" -# Disable the test suites -EXTRA_OEMESON += "-Dtests=false -Dautobahn=disabled -Dpkcs11_tests=disabled" +# tests are controlled via the 'tests' PACKAGECONFIG (enabled for ptest) +EXTRA_OEMESON += "-Dautobahn=disabled -Dpkcs11_tests=disabled" GIDOCGEN_MESON_OPTION = 'docs' GIDOCGEN_MESON_ENABLE_FLAG = 'enabled' GIDOCGEN_MESON_DISABLE_FLAG = 'disabled' +# header-parsing-test aborts in do_param_list_tests() with the current +# glib (soup_header_parse_semi_param_list() returns a single entry for a +# multi-parameter header). Drop its installed-test metadata so the rest of +# the suite can run under ptest; the test binary is left in place. +do_install_ptest:append() { + rm -f ${D}${datadir}/installed-tests/libsoup-3.0/header-parsing-test.test +} + # When built without gnome support, libsoup will contain only one shared lib # and will therefore become subject to renaming by debian.bbclass. Prevent # renaming in order to keep the package name consistent regardless of whether
@@ -60,6 +70,10 @@ DEBIAN_NOAUTONAME:${PN} = "1" # glib-networking is needed for SSL, proxies, etc. RRECOMMENDS:${PN} = "glib-networking" +# The TLS-based tests (ssl, server-auth, websocket, hsts) need a glib TLS +# backend at runtime. +RDEPENDS:${PN}-ptest += "glib-networking" + BBCLASSEXTEND = "native nativesdk" CVE_STATUS[CVE-2026-1467] = "fixed-version: fixed in 3.6.6"
Signature · 1 line
2.43.0
Quoted 5 lines · Pratik Farkase
Build the installed-tests via the meson tests/installed_tests options (enabled through a new 'tests' PACKAGECONFIG when ptest is in
DISTRO_FEATURES) and run them under ptest with gnome-desktop-testing-runner.
Quoted 7 lines
header-parsing-test is excluded as it aborts in
do_param_list_tests()with the current glib; the rest of the suite passes.Tested on qemux86-64 with ptest-runner: 34 tests, all pass.
Signed-off-by Pratik Farkase <[email protected]>---
Hi Pratik,
We have an upgrade to libsoup 3.8.0 that was merged a few days ago. I could try to merge the conflicts, but as the recipe did evolve a bit, it would mean the merged patch would be really different from what you sent. And we should probably avoid that.
Can you rebase on top of master and send a new version please?
Thanks,
Mathieu
Signature · 3 lines
Mathieu Dubois-Briand, Bootlin Embedded Linux and Kernel engineering https://bootlin.com
Hi Mathieu,
Thanks for the heads-up. I've rebased on top of master and reworked the patch for the 3.8.0 recipe. As the recipe changed quite a bit with the upgrade, I have sent it as a fresh patch rather than a v2.
Best Regards,
Pratik
________________________________________
From: Mathieu Dubois-Briand <[email protected]>
Sent: Monday, October 12, 2026 7:15 PM
To: Pratik Farkase; [email protected]
Cc: [email protected]
Subject: Re: [OE-core][PATCH] libsoup: add ptest support
Quoted 5 lines · Pratik Farkase
Build the installed-tests via the meson tests/installed_tests options (enabled through a new 'tests' PACKAGECONFIG when ptest is in
DISTRO_FEATURES) and run them under ptest with gnome-desktop-testing-runner.
Quoted 7 lines
header-parsing-test is excluded as it aborts in
do_param_list_tests()with the current glib; the rest of the suite passes.Tested on qemux86-64 with ptest-runner: 34 tests, all pass.
Signed-off-by Pratik Farkase <[email protected]>---
Hi Pratik,
We have an upgrade to libsoup 3.8.0 that was merged a few days ago. I could try to merge the conflicts, but as the recipe did evolve a bit, it would mean the merged patch would be really different from what you sent. And we should probably avoid that.
Can you rebase on top of master and send a new version please?
Thanks,
Mathieu
--
Mathieu Dubois-Briand, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com
PATCH v2 1/1 libsoup: add ptest support
In reply to Pratik FarkaseBuild the installed-tests via the meson tests/installed_tests options (enabled through a new 'tests' PACKAGECONFIG when ptest is in DISTRO_FEATURES) and run them under ptest with gnome-desktop-testing-runner.
Tested on qemux86-64 with ptest-runner: 36 tests, all pass.
---
Notes:
Changes in v2:
- Rebased onto master after the libsoup 3.6.6 -> 3.8.0 upgrade.
- Keep logger-test and http2-body-stream-test; instead of excluding them, raise QB_MEM for libsoup in core-image-ptest.bb so the large-allocation cases do not hit the OOM killer. Only header-parsing-test remains excluded (a glib parameter-list parsing assertion failure).
.../distro/include/ptest-packagelists.inc | 1 + meta/recipes-core/images/core-image-ptest.bb | 1 + .../recipes-support/libsoup/libsoup/run-ptest | 3 +++ meta/recipes-support/libsoup/libsoup_3.8.0.bb | 24 +++++++++++++++---- 4 files changed, 25 insertions(+), 4 deletions(-) create mode 100644 meta/recipes-support/libsoup/libsoup/run-ptest
meta/conf/distro/include/ptest-packagelists.inc +1 -0
@@ -138,6 +138,7 @@ PTESTS_SLOW = "\ libmodule-build-perl \ libpcap \ libpng \ + libsoup \ ${@bb.utils.contains('DISTRO_FEATURES', 'seccomp', 'libseccomp', '',d)} \ lttng-tools \ lz4 \
meta/recipes-core/images/core-image-ptest.bb +1 -0
@@ -49,6 +49,7 @@ QB_MEM:virtclass-mcextend-tcl = "-m 5100" QB_MEM:virtclass-mcextend-go = "-m 4096" QB_MEM:virtclass-mcextend-gnutls = "-m 1536" QB_MEM:virtclass-mcextend-zstd = "-m 4096" +QB_MEM:virtclass-mcextend-libsoup = "-m 4096" TEST_SUITES = "ping ssh parselogs ptest"
meta/recipes-support/libsoup/libsoup/run-ptest new file +3 -0
@@ -0,0 +1,3 @@ +#! /bin/sh + +gnome-desktop-testing-runner libsoup-3.0
meta/recipes-support/libsoup/libsoup_3.8.0.bb +20 -4
@@ -9,20 +9,23 @@ LIC_FILES_CHKSUM = "file://COPYING;md5=5f30f0716dfdd0d91eb439ebec522ec2" DEPENDS = "glib-2.0 glib-2.0-native libxml2 sqlite3 libpsl nghttp2" -inherit gettext gnomebase upstream-version-is-even gobject-introspection gi-docgen vala +inherit gettext gnomebase upstream-version-is-even gobject-introspection gi-docgen vala ptest-gnome SRC_URI[archive.sha256sum] = "bbf08fa3e03a88c31a3d27a0d87cb422e9490f2d08e149211103df6d638a2238" +SRC_URI += "file://run-ptest" + PROVIDES = "libsoup-3.0" GIR_MESON_ENABLE_FLAG = 'enabled' GIR_MESON_DISABLE_FLAG = 'disabled' -PACKAGECONFIG ??= "" +PACKAGECONFIG ??= "${@bb.utils.contains('PTEST_ENABLED', '1', 'tests', '', d)}" PACKAGECONFIG[brotli] = "-Dbrotli=enabled,-Dbrotli=disabled,brotli" PACKAGECONFIG[gssapi] = "-Dgssapi=enabled,-Dgssapi=disabled,krb5" PACKAGECONFIG[ntlm] = "-Dntlm=enabled,-Dntlm=disabled" PACKAGECONFIG[sysprof] = "-Dsysprof=enabled,-Dsysprof=disabled,sysprof" +PACKAGECONFIG[tests] = "-Dtests=true -Dinstalled_tests=true,-Dtests=false -Dinstalled_tests=false" # Tell libsoup where the target ntlm_auth is installed do_write_config:append:class-target() {
@@ -34,13 +37,22 @@ EOF EXTRA_OEMESON:append:class-target = " --cross-file ${WORKDIR}/soup.cross" EXTRA_OEMESON += "-Dtls_check=false" -# Disable the test suites -EXTRA_OEMESON += "-Dtests=false -Dautobahn=disabled -Dpkcs11_tests=disabled" +# tests are controlled via the 'tests' PACKAGECONFIG (enabled for ptest) +EXTRA_OEMESON += "-Dautobahn=disabled -Dpkcs11_tests=disabled" GIDOCGEN_MESON_OPTION = 'docs' GIDOCGEN_MESON_ENABLE_FLAG = 'enabled' GIDOCGEN_MESON_DISABLE_FLAG = 'disabled' +# header-parsing-test aborts in do_param_list_tests(): a g_hash_table_size() +# assertion on a parsed parameter list does not hold (returns one entry +# instead of three) with the current glib. The memory-hungry logger and +# http2-body-stream tests are handled by raising QB_MEM for libsoup in +# core-image-ptest.bb. +do_install_ptest:append() { + rm -f ${D}${datadir}/installed-tests/libsoup-3.0/header-parsing-test.test +} + # When built without gnome support, libsoup will contain only one shared lib # and will therefore become subject to renaming by debian.bbclass. Prevent # renaming in order to keep the package name consistent regardless of whether
@@ -50,6 +62,10 @@ DEBIAN_NOAUTONAME:${PN} = "1" # glib-networking is needed for SSL, proxies, etc. RRECOMMENDS:${PN} = "glib-networking" +# The TLS-based tests (ssl, server-auth, websocket, hsts) need a glib TLS +# backend at runtime. +RDEPENDS:${PN}-ptest += "glib-networking" + BBCLASSEXTEND = "native nativesdk" CVE_STATUS[CVE-2026-1467] = "fixed-version: fixed in 3.6.6"
Signature · 1 line
2.43.0
Quoted 9 lines · Pratik Farkase
Build the installed-tests via the meson tests/installed_tests options (enabled through a new 'tests' PACKAGECONFIG when ptest is in
DISTRO_FEATURES) and run them under ptest with gnome-desktop-testing-runner.Tested on qemux86-64 with ptest-runner: 36 tests, all pass.
Signed-off-by Pratik Farkase <[email protected]>---
Hi Pratik,
Thanks for the new version. It looks like we are still getting OOM issues:
WARNING: core-image-ptest-libsoup-1.0-r0 do_testimage: There were failing ptests.
Traceback (most recent call last):
File "/srv/pokybuild/yocto-worker/qemux86-64-musl-ptest/build/layers/openembedded-core/meta/lib/oeqa/core/decorator/__init__.py", line 35, in wrapped_f
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/srv/pokybuild/yocto-worker/qemux86-64-musl-ptest/build/layers/openembedded-core/meta/lib/oeqa/core/decorator/__init__.py", line 35, in wrapped_f
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/srv/pokybuild/yocto-worker/qemux86-64-musl-ptest/build/layers/openembedded-core/meta/lib/oeqa/core/decorator/__init__.py", line 35, in wrapped_f
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/srv/pokybuild/yocto-worker/qemux86-64-musl-ptest/build/layers/openembedded-core/meta/lib/oeqa/runtime/cases/ptest.py", line 27, in test_ptestrunner_expectfail
self.do_ptestrunner()
File "/srv/pokybuild/yocto-worker/qemux86-64-musl-ptest/build/layers/openembedded-core/meta/lib/oeqa/runtime/cases/ptest.py", line 121, in do_ptestrunner
self.fail(failmsg)
AssertionError: ERROR: Processes were killed by the OOM Killer:
[ 38.188495] Out of memory: Killed process 563 (logger-test) total-vm:4209544kB, anon-rss:3997540kB, file-rss:36kB, shmem-rss:0kB, UID:0 pgtables:7904kB oom_score_adj:0
Failed ptests:
{'libsoup': ['libsoup-3.0/logger-test.test']}
https://autobuilder.yoctoproject.org/valkyrie/#/builders/73/builds/4743
https://autobuilder.yoctoproject.org/valkyrie/#/builders/110/builds/1364
https://autobuilder.yoctoproject.org/valkyrie/#/builders/109/builds/1379
Can you have a look at the issue?
Thanks,
Mathieu
Signature · 3 lines
Mathieu Dubois-Briand, Bootlin Embedded Linux and Kernel engineering https://bootlin.com
Let's skip those if 4gigs don't suffice. sent a v3.
Best Regards,
Pratik
________________________________________
From: Mathieu Dubois-Briand <[email protected]>
Sent: Wednesday, October 14, 2026 1:48 PM
To: Pratik Farkase; [email protected]
Cc: [email protected]
Subject: Re: [OE-core][PATCH v2] libsoup: add ptest support
Quoted 9 lines · Pratik Farkase
Build the installed-tests via the meson tests/installed_tests options (enabled through a new 'tests' PACKAGECONFIG when ptest is in
DISTRO_FEATURES) and run them under ptest with gnome-desktop-testing-runner.Tested on qemux86-64 with ptest-runner: 36 tests, all pass.
Signed-off-by Pratik Farkase <[email protected]>---
Hi Pratik,
Thanks for the new version. It looks like we are still getting OOM issues:
WARNING: core-image-ptest-libsoup-1.0-r0 do_testimage: There were failing ptests.
Traceback (most recent call last):
File "/srv/pokybuild/yocto-worker/qemux86-64-musl-ptest/build/layers/openembedded-core/meta/lib/oeqa/core/decorator/__init__.py", line 35, in wrapped_f
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/srv/pokybuild/yocto-worker/qemux86-64-musl-ptest/build/layers/openembedded-core/meta/lib/oeqa/core/decorator/__init__.py", line 35, in wrapped_f
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/srv/pokybuild/yocto-worker/qemux86-64-musl-ptest/build/layers/openembedded-core/meta/lib/oeqa/core/decorator/__init__.py", line 35, in wrapped_f
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/srv/pokybuild/yocto-worker/qemux86-64-musl-ptest/build/layers/openembedded-core/meta/lib/oeqa/runtime/cases/ptest.py", line 27, in test_ptestrunner_expectfail
self.do_ptestrunner()
File "/srv/pokybuild/yocto-worker/qemux86-64-musl-ptest/build/layers/openembedded-core/meta/lib/oeqa/runtime/cases/ptest.py", line 121, in do_ptestrunner
self.fail(failmsg)
AssertionError: ERROR: Processes were killed by the OOM Killer:
[ 38.188495] Out of memory: Killed process 563 (logger-test) total-vm:4209544kB, anon-rss:3997540kB, file-rss:36kB, shmem-rss:0kB, UID:0 pgtables:7904kB oom_score_adj:0
Failed ptests:
{'libsoup': ['libsoup-3.0/logger-test.test']}
https://autobuilder.yoctoproject.org/valkyrie/#/builders/73/builds/4743
https://autobuilder.yoctoproject.org/valkyrie/#/builders/110/builds/1364
https://autobuilder.yoctoproject.org/valkyrie/#/builders/109/builds/1379
Can you have a look at the issue?
Thanks,
Mathieu
--
Mathieu Dubois-Briand, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com
PATCH v3 1/1 libsoup: add ptest support
In reply to Pratik FarkaseBuild the installed-tests via the meson tests/installed_tests options (enabled through a new 'tests' PACKAGECONFIG when ptest is in DISTRO_FEATURES) and run them under ptest with gnome-desktop-testing-runner.
Tested on qemux86-64 with ptest-runner.
---
Notes:
Changes in v3:
- Drop the QB_MEM bump approach from v2: 4 GiB was still not enough and logger-test was OOM-killed (~4 GiB RSS) on the autobuilder. Exclude logger-test and http2-body-stream-test instead, alongside the existing header-parsing-test exclusion.
Changes in v2:
- Rebased onto master after the libsoup 3.6.6 -> 3.8.0 upgrade.
.../distro/include/ptest-packagelists.inc | 1 + .../recipes-support/libsoup/libsoup/run-ptest | 3 ++ meta/recipes-support/libsoup/libsoup_3.8.0.bb | 28 ++++++++++++++++--- 3 files changed, 28 insertions(+), 4 deletions(-) create mode 100644 meta/recipes-support/libsoup/libsoup/run-ptest
meta/conf/distro/include/ptest-packagelists.inc +1 -0
@@ -138,6 +138,7 @@ PTESTS_SLOW = "\ libmodule-build-perl \ libpcap \ libpng \ + libsoup \ ${@bb.utils.contains('DISTRO_FEATURES', 'seccomp', 'libseccomp', '',d)} \ lttng-tools \ lz4 \
meta/recipes-support/libsoup/libsoup/run-ptest new file +3 -0
@@ -0,0 +1,3 @@ +#! /bin/sh + +gnome-desktop-testing-runner libsoup-3.0
meta/recipes-support/libsoup/libsoup_3.8.0.bb +24 -4
@@ -9,20 +9,23 @@ LIC_FILES_CHKSUM = "file://COPYING;md5=5f30f0716dfdd0d91eb439ebec522ec2" DEPENDS = "glib-2.0 glib-2.0-native libxml2 sqlite3 libpsl nghttp2" -inherit gettext gnomebase upstream-version-is-even gobject-introspection gi-docgen vala +inherit gettext gnomebase upstream-version-is-even gobject-introspection gi-docgen vala ptest-gnome SRC_URI[archive.sha256sum] = "bbf08fa3e03a88c31a3d27a0d87cb422e9490f2d08e149211103df6d638a2238" +SRC_URI += "file://run-ptest" + PROVIDES = "libsoup-3.0" GIR_MESON_ENABLE_FLAG = 'enabled' GIR_MESON_DISABLE_FLAG = 'disabled' -PACKAGECONFIG ??= "" +PACKAGECONFIG ??= "${@bb.utils.contains('PTEST_ENABLED', '1', 'tests', '', d)}" PACKAGECONFIG[brotli] = "-Dbrotli=enabled,-Dbrotli=disabled,brotli" PACKAGECONFIG[gssapi] = "-Dgssapi=enabled,-Dgssapi=disabled,krb5" PACKAGECONFIG[ntlm] = "-Dntlm=enabled,-Dntlm=disabled" PACKAGECONFIG[sysprof] = "-Dsysprof=enabled,-Dsysprof=disabled,sysprof" +PACKAGECONFIG[tests] = "-Dtests=true -Dinstalled_tests=true,-Dtests=false -Dinstalled_tests=false" # Tell libsoup where the target ntlm_auth is installed do_write_config:append:class-target() {
@@ -34,13 +37,26 @@ EOF EXTRA_OEMESON:append:class-target = " --cross-file ${WORKDIR}/soup.cross" EXTRA_OEMESON += "-Dtls_check=false" -# Disable the test suites -EXTRA_OEMESON += "-Dtests=false -Dautobahn=disabled -Dpkcs11_tests=disabled" +# tests are controlled via the 'tests' PACKAGECONFIG (enabled for ptest) +EXTRA_OEMESON += "-Dautobahn=disabled -Dpkcs11_tests=disabled" GIDOCGEN_MESON_OPTION = 'docs' GIDOCGEN_MESON_ENABLE_FLAG = 'enabled' GIDOCGEN_MESON_DISABLE_FLAG = 'disabled' +# These tests abort under ptest and leave the binaries in place: +# - header-parsing-test: a g_hash_table_size() assertion in +# do_param_list_tests() does not hold (returns one entry instead of three) +# with the current glib. +# - logger-test (long-invalid-body-length) and http2-body-stream-test +# (large_data) exercise multi-GiB allocations and are killed by the OOM +# killer on the memory-constrained ptest machines. +do_install_ptest:append() { + rm -f ${D}${datadir}/installed-tests/libsoup-3.0/header-parsing-test.test + rm -f ${D}${datadir}/installed-tests/libsoup-3.0/logger-test.test + rm -f ${D}${datadir}/installed-tests/libsoup-3.0/http2-body-stream-test.test +} + # When built without gnome support, libsoup will contain only one shared lib # and will therefore become subject to renaming by debian.bbclass. Prevent # renaming in order to keep the package name consistent regardless of whether
@@ -50,6 +66,10 @@ DEBIAN_NOAUTONAME:${PN} = "1" # glib-networking is needed for SSL, proxies, etc. RRECOMMENDS:${PN} = "glib-networking" +# The TLS-based tests (ssl, server-auth, websocket, hsts) need a glib TLS +# backend at runtime. +RDEPENDS:${PN}-ptest += "glib-networking" + BBCLASSEXTEND = "native nativesdk" CVE_STATUS[CVE-2026-1467] = "fixed-version: fixed in 3.6.6"
Signature · 1 line
2.43.0
Quoted 9 lines · Pratik Farkase
Build the installed-tests via the meson tests/installed_tests options (enabled through a new 'tests' PACKAGECONFIG when ptest is in
DISTRO_FEATURES) and run them under ptest with gnome-desktop-testing-runner.Tested on qemux86-64 with ptest-runner.
Signed-off-by Pratik Farkase <[email protected]>---
Hi Pratik,
Thanks, it's much better now. Still, I do note some intermittent failures.
Failed ptests:
{'libsoup': ['libsoup-3.0/multithread-test.test']}
https://autobuilder.yoctoproject.org/valkyrie/#/builders/61/builds/4725
Failed ptests:
{'libsoup': ['libsoup-3.0/http2-test.test']}
https://autobuilder.yoctoproject.org/valkyrie/#/builders/56/builds/26031
I believe we can keep it in testing for a bit of time and see which test fails and how often, then decide if it gets merged or not. But we will probably need a bit more work in there.
Thanks,
Mathieu
Signature · 3 lines
Mathieu Dubois-Briand, Bootlin Embedded Linux and Kernel engineering https://bootlin.com
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 24 lines · Mathieu Dubois-Briand via lists.openembedded.org
Quoted 9 lines · Pratik Farkase
Build the installed-tests via the meson tests/installed_tests options (enabled through a new 'tests' PACKAGECONFIG when ptest is in
DISTRO_FEATURES) and run them under ptest with gnome-desktop-testing-runner.Tested on qemux86-64 with ptest-runner.
Signed-off-by Pratik Farkase <[email protected]>---
Hi Pratik,
Thanks, it's much better now. Still, I do note some intermittent failures.
Failed ptests:
{'libsoup': ['libsoup-3.0/multithread-test.test']}
https://autobuilder.yoctoproject.org/valkyrie/#/builders/61/builds/4725Failed ptests:
{'libsoup': ['libsoup-3.0/http2-test.test']}
https://autobuilder.yoctoproject.org/valkyrie/#/builders/56/builds/26031
Quoted 6 lines
I believe we can keep it in testing for a bit of time and see which test fails and how often, then decide if it gets merged or not. But we will probably need a bit more work in there.
Thanks,
Mathieu
Hi Pratik,
So after a few days, here is the situation:
{'libsoup': ['libsoup-3.0/multithread-test.test']} 2 occurrences, 1 qemuarm64-musl-ptest, 1 qemuarm64-ptest
https://autobuilder.yoctoproject.org/valkyrie/#/builders/109/builds/1400
https://autobuilder.yoctoproject.org/valkyrie/#/builders/61/builds/4725
{'libsoup': ['libsoup-3.0/http2-test.test']}
8 occurences, all on qemuriscv64-ptest
https://autobuilder.yoctoproject.org/valkyrie/#/builders/56/builds/2603
https://autobuilder.yoctoproject.org/valkyrie/#/builders/56/builds/2614
https://autobuilder.yoctoproject.org/valkyrie/#/builders/56/builds/2615
https://autobuilder.yoctoproject.org/valkyrie/#/builders/56/builds/2616
https://autobuilder.yoctoproject.org/valkyrie/#/builders/56/builds/2618
https://autobuilder.yoctoproject.org/valkyrie/#/builders/56/builds/2621
https://autobuilder.yoctoproject.org/valkyrie/#/builders/56/builds/2625
https://autobuilder.yoctoproject.org/valkyrie/#/builders/56/builds/2626
So I believe this will need a bit more work to make these tests a bit more stable, I'm sorry.
Thanks,
Mathieu
Signature · 3 lines
Mathieu Dubois-Briand, Bootlin Embedded Linux and Kernel engineering https://bootlin.com
Hi Mathieu,
It's quite interesting we are catching these flaky tests now rather than more re-work later. Tests are expected to fail to some extent for riscv64 ARCH due to slower TCG emulation (growing area). Let's keep testing more on the autobuilder. I have sent a v4 correcting the failures from v3.
Best Regards,
Pratik
________________________________________
From: Mathieu Dubois-Briand <[email protected]>
Sent: Tuesday, October 20, 2026 9:36 AM
To: Pratik Farkase; [email protected]
Cc: [email protected]
Subject: Re: [OE-core][PATCH v3] libsoup: add ptest support
Quoted 24 lines · Mathieu Dubois-Briand via lists.openembedded.org
Quoted 9 lines · Pratik Farkase
Build the installed-tests via the meson tests/installed_tests options (enabled through a new 'tests' PACKAGECONFIG when ptest is in
DISTRO_FEATURES) and run them under ptest with gnome-desktop-testing-runner.Tested on qemux86-64 with ptest-runner.
Signed-off-by Pratik Farkase <[email protected]>---
Hi Pratik,
Thanks, it's much better now. Still, I do note some intermittent failures.
Failed ptests:
{'libsoup': ['libsoup-3.0/multithread-test.test']}
https://autobuilder.yoctoproject.org/valkyrie/#/builders/61/builds/4725Failed ptests:
{'libsoup': ['libsoup-3.0/http2-test.test']}
https://autobuilder.yoctoproject.org/valkyrie/#/builders/56/builds/26031
Quoted 6 lines
I believe we can keep it in testing for a bit of time and see which test fails and how often, then decide if it gets merged or not. But we will probably need a bit more work in there.
Thanks,
Mathieu
Hi Pratik,
So after a few days, here is the situation:
{'libsoup': ['libsoup-3.0/multithread-test.test']} 2 occurrences, 1 qemuarm64-musl-ptest, 1 qemuarm64-ptest
https://autobuilder.yoctoproject.org/valkyrie/#/builders/109/builds/1400
https://autobuilder.yoctoproject.org/valkyrie/#/builders/61/builds/4725
{'libsoup': ['libsoup-3.0/http2-test.test']}
8 occurences, all on qemuriscv64-ptest
https://autobuilder.yoctoproject.org/valkyrie/#/builders/56/builds/2603
https://autobuilder.yoctoproject.org/valkyrie/#/builders/56/builds/2614
https://autobuilder.yoctoproject.org/valkyrie/#/builders/56/builds/2615
https://autobuilder.yoctoproject.org/valkyrie/#/builders/56/builds/2616
https://autobuilder.yoctoproject.org/valkyrie/#/builders/56/builds/2618
https://autobuilder.yoctoproject.org/valkyrie/#/builders/56/builds/2621
https://autobuilder.yoctoproject.org/valkyrie/#/builders/56/builds/2625
https://autobuilder.yoctoproject.org/valkyrie/#/builders/56/builds/2626
So I believe this will need a bit more work to make these tests a bit more stable, I'm sorry.
Thanks,
Mathieu
--
Mathieu Dubois-Briand, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com
PATCH v4 1/1 libsoup: add ptest support
In reply to Pratik FarkaseBuild the installed-tests via the meson tests/installed_tests options (enabled through a new 'tests' PACKAGECONFIG when ptest is in DISTRO_FEATURES) and run them under ptest with gnome-desktop-testing-runner.
Some tests are excluded via PTEST_SKIP_TESTS:
- header-parsing-test hits a glib parameter-list parsing assertion.
- logger-test and http2-body-stream-test allocate multiple GiB and are OOM-killed on the ptest machines.
- http2-test (riscv64) and multithread-test (aarch64) intermittently abort during teardown on the slower emulated targets.
Tested on qemux86-64 with ptest-runner.
---
Notes:
Changes in v4:
- Skip http2-test on riscv64 and multithread-test on aarch64: both intermittently abort (SIGABRT) during teardown on the slower emulated targets (seen on the autobuilder over several days).
Changes in v3:
- Drop the QB_MEM bump; exclude logger-test and http2-body-stream-test instead, as 4 GiB was still OOM-killed on the autobuilder.
Changes in v2:
- Rebased onto master after the libsoup 3.6.6 -> 3.8.0 upgrade.
.../distro/include/ptest-packagelists.inc | 1 + .../recipes-support/libsoup/libsoup/run-ptest | 3 ++ meta/recipes-support/libsoup/libsoup_3.8.0.bb | 37 +++++++++++++++++-- 3 files changed, 37 insertions(+), 4 deletions(-) create mode 100644 meta/recipes-support/libsoup/libsoup/run-ptest
meta/conf/distro/include/ptest-packagelists.inc +1 -0
@@ -138,6 +138,7 @@ PTESTS_SLOW = "\ libmodule-build-perl \ libpcap \ libpng \ + libsoup \ ${@bb.utils.contains('DISTRO_FEATURES', 'seccomp', 'libseccomp', '',d)} \ lttng-tools \ lz4 \
meta/recipes-support/libsoup/libsoup/run-ptest new file +3 -0
@@ -0,0 +1,3 @@ +#! /bin/sh + +gnome-desktop-testing-runner libsoup-3.0
meta/recipes-support/libsoup/libsoup_3.8.0.bb +33 -4
@@ -9,20 +9,23 @@ LIC_FILES_CHKSUM = "file://COPYING;md5=5f30f0716dfdd0d91eb439ebec522ec2" DEPENDS = "glib-2.0 glib-2.0-native libxml2 sqlite3 libpsl nghttp2" -inherit gettext gnomebase upstream-version-is-even gobject-introspection gi-docgen vala +inherit gettext gnomebase upstream-version-is-even gobject-introspection gi-docgen vala ptest-gnome SRC_URI[archive.sha256sum] = "bbf08fa3e03a88c31a3d27a0d87cb422e9490f2d08e149211103df6d638a2238" +SRC_URI += "file://run-ptest" + PROVIDES = "libsoup-3.0" GIR_MESON_ENABLE_FLAG = 'enabled' GIR_MESON_DISABLE_FLAG = 'disabled' -PACKAGECONFIG ??= "" +PACKAGECONFIG ??= "${@bb.utils.contains('PTEST_ENABLED', '1', 'tests', '', d)}" PACKAGECONFIG[brotli] = "-Dbrotli=enabled,-Dbrotli=disabled,brotli" PACKAGECONFIG[gssapi] = "-Dgssapi=enabled,-Dgssapi=disabled,krb5" PACKAGECONFIG[ntlm] = "-Dntlm=enabled,-Dntlm=disabled" PACKAGECONFIG[sysprof] = "-Dsysprof=enabled,-Dsysprof=disabled,sysprof" +PACKAGECONFIG[tests] = "-Dtests=true -Dinstalled_tests=true,-Dtests=false -Dinstalled_tests=false" # Tell libsoup where the target ntlm_auth is installed do_write_config:append:class-target() {
@@ -34,13 +37,35 @@ EOF EXTRA_OEMESON:append:class-target = " --cross-file ${WORKDIR}/soup.cross" EXTRA_OEMESON += "-Dtls_check=false" -# Disable the test suites -EXTRA_OEMESON += "-Dtests=false -Dautobahn=disabled -Dpkcs11_tests=disabled" +# tests are controlled via the 'tests' PACKAGECONFIG (enabled for ptest) +EXTRA_OEMESON += "-Dautobahn=disabled -Dpkcs11_tests=disabled" GIDOCGEN_MESON_OPTION = 'docs' GIDOCGEN_MESON_ENABLE_FLAG = 'enabled' GIDOCGEN_MESON_DISABLE_FLAG = 'disabled' +# These tests abort under ptest and leave the binaries in place: +# - header-parsing-test: a g_hash_table_size() assertion in +# do_param_list_tests() does not hold (returns one entry instead of three) +# with the current glib. +# - logger-test (long-invalid-body-length) and http2-body-stream-test +# (large_data) exercise multi-GiB allocations and are killed by the OOM +# killer on the memory-constrained ptest machines. +PTEST_SKIP_TESTS = "header-parsing-test logger-test http2-body-stream-test" + +# These tests intermittently abort (SIGABRT) during teardown under the slower +# emulated targets, so skip them there: +# - http2-test on qemuriscv64. +# - multithread-test on qemuarm64 (glibc and musl). +PTEST_SKIP_TESTS:append:riscv64 = " http2-test" +PTEST_SKIP_TESTS:append:aarch64 = " multithread-test" + +do_install_ptest:append() { + for t in ${PTEST_SKIP_TESTS}; do + rm -f ${D}${datadir}/installed-tests/libsoup-3.0/$t.test + done +} + # When built without gnome support, libsoup will contain only one shared lib # and will therefore become subject to renaming by debian.bbclass. Prevent # renaming in order to keep the package name consistent regardless of whether
@@ -50,6 +75,10 @@ DEBIAN_NOAUTONAME:${PN} = "1" # glib-networking is needed for SSL, proxies, etc. RRECOMMENDS:${PN} = "glib-networking" +# The TLS-based tests (ssl, server-auth, websocket, hsts) need a glib TLS +# backend at runtime. +RDEPENDS:${PN}-ptest += "glib-networking" + BBCLASSEXTEND = "native nativesdk" CVE_STATUS[CVE-2026-1467] = "fixed-version: fixed in 3.6.6"
Signature · 1 line
2.43.0
meta/recipes-support/libsoup/libsoup_3.8.0.bb EOF · line 50
Quoted 5 lines · Pratik Farkase via lists.openembedded.org
meta/recipes-support/libsoup/libsoup_3.8.0.bb +4
-0+# These tests abort under ptest and leave the binaries in place: +# - header-parsing-test: a g_hash_table_size() assertion in +# do_param_list_tests() does not hold (returns one entry instead of three) +# with the current glib.
This feels like something that should be fixed. Is there an upstream ticket or fix for this? If not, can you file a bug?
Thanks,
Ross
Not at the moment Ross, I will file it.
Best Regards,
Pratik
________________________________________
From: Ross Burton <[email protected]>
Sent: Thursday, October 22, 2026 2:34 PM
To: Pratik Farkase
Cc: [email protected]; [email protected]
Subject: Re: [OE-core][PATCH v4] libsoup: add ptest support
meta/recipes-support/libsoup/libsoup_3.8.0.bb EOF · line 50
Quoted 5 lines · Pratik Farkase via lists.openembedded.org
meta/recipes-support/libsoup/libsoup_3.8.0.bb +4
-0+# These tests abort under ptest and leave the binaries in place: +# - header-parsing-test: a g_hash_table_size() assertion in +# do_param_list_tests() does not hold (returns one entry instead of three) +# with the current glib.
This feels like something that should be fixed. Is there an upstream ticket or fix for this? If not, can you file a bug?
Thanks,
Ross