Skip to contentSkip to conversation

perl: Add option to prevent COMPLEMENTARY_GLOB from ignoring perl-modules

PATCH In review [email protected] 4 participants 12 messages Jan 16, 2024 – Jan 22, 2024 1 message not in the archive

lore
Share & tools
lore
#0 lore raw

PATCH 1/2 perl: Add option to prevent COMPLEMENTARY_GLOB from ignoring perl-modules

From: William Lyu <[email protected]>

When perl-modules is added via COMPLEMENTARY_GLOB, all perl-module-* packages recommended by perl-modules are ignored due to the defined behavior of COMPLEMENTARY_GLOB.

This patch creates a new package perl-modules-force-all that RDEPENDS on the same set of packages perl-modules recommends. If another recipe always needs the packages recommended by perl-modules, it can add perl-modules-force-all to RDEPENDS to make sure that the recommended packages by perl-modules are not ignored by COMPLEMENTARY_GLOB.

---
 meta/recipes-devtools/perl/perl_5.38.2.bb | 22 ++++++++++++++++++++++
 1 file changed, 22 insertions(+)
meta/recipes-devtools/perl/perl_5.38.2.bb +22 -0
index a9d684cfc5..f75c5aa971 100644
@@ -311,6 +311,14 @@ ALTERNATIVE_LINK_NAME[Thread.3] = "${mandir}/man3/Thread.3"
 ALLOW_EMPTY:${PN}-modules = "1"
 PACKAGES += "${PN}-modules "
 
+# When perl-modules is added through COMPLEMENTARY_GLOB, as it is recommending
+# packages, all its recommendations will be ignored. So, we create a
+# perl-modules-force-all package depending on the same set of packages as listed
+# in perl-modules to provide a way to guarantee COMPLEMENTARY_GLOB not removing
+# these packages.
+ALLOW_EMPTY:${PN}-modules-force-all = "1"
+PACKAGES += "${PN}-modules-force-all "
+
 PACKAGESPLITFUNCS =+ "split_perl_packages"
 
 python split_perl_packages () {
@@ -328,6 +336,20 @@ python split_perl_packages () {
     packages = filter(lambda p: 'perl-module-' in p, d.getVar('PACKAGES').split())
     d.setVar(d.expand("RRECOMMENDS:${PN}-modules"), ' '.join(packages))
 
+    # perl-modules-force-all provides a way to prevent COMPLEMENTARY_GLOB
+    # from removing the packages specified in perl-modules.
+    # perl-modules-force-all contains the same set of packages as perl-modules.
+    d.setVar(
+        d.expand("RDEPENDS:${PN}-modules-force-all"),
+        d.getVar(d.expand("RRECOMMENDS:${PN}-modules"))
+    )
+
+    # As perl-modules-force-all depends on all other, potentially non-existent,
+    # perl packages, we allow these packages to be empty to prevent build
+    # errors.
+    for pkg in d.getVar(d.expand("RDEPENDS:${PN}-modules-force-all")).split():
+        d.setVar(d.expand("ALLOW_EMPTY:" + pkg), "1")
+
     # Read the pre-generated dependency file, and use it to set module dependecies
     for line in open(d.expand("${WORKDIR}") + '/perl-rdepends.txt').readlines():
         splitline = line.split()
Signature · 1 line
2.43.0
#1 lore raw

PATCH 2/2 openssl: Fix ptest dependencies on perl-module-* being ignored by COMPLEMENTARY_GLOB

From: William Lyu <[email protected]>

Change the openssl-ptest RDEPENDS from perl-modules to perl-modules-force-all which RDEPENDS on the same set of packages perl-module-* recommended by perl-modules. So, when openssl-ptest is added via COMPLEMENTARY_GLOB "ptest-pkgs", its dependencies on these perl-module-* packages still get added.

---
 meta/recipes-connectivity/openssl/openssl_3.2.0.bb | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
meta/recipes-connectivity/openssl/openssl_3.2.0.bb +1 -1
index d041d2d214..7076bd3a7b 100644
@@ -247,7 +247,7 @@ CONFFILES:openssl-conf = "${sysconfdir}/ssl/openssl.cnf"
 
 RRECOMMENDS:libcrypto += "openssl-conf ${PN}-ossl-module-legacy"
 RDEPENDS:${PN}-misc = "perl"
-RDEPENDS:${PN}-ptest += "openssl-bin perl perl-modules bash sed"
+RDEPENDS:${PN}-ptest += "openssl-bin perl perl-modules-force-all bash sed"
 
 RDEPENDS:${PN}-bin += "openssl-conf"
 
Signature · 1 line
2.43.0
#2 lore raw

Re: [OE-core] [PATCH 2/2] openssl: Fix ptest dependencies on perl-module-* being ignored by COMPLEMENTARY_GLOB

How about simple depending on specific module packages? I don’t think OpenSSL truly needs all modules.

Alex

Quoted 43 lines · William via lists.openembedded.org

From: William Lyu <[email protected]>

Change the openssl-ptest RDEPENDS from perl-modules to perl-modules-force-all which RDEPENDS on the same set of packages perl-module-* recommended by perl-modules. So, when openssl-ptest is added via COMPLEMENTARY_GLOB "ptest-pkgs", its dependencies on these perl-module-* packages still get added.

---
 meta/recipes-connectivity/openssl/openssl_3.2.0.bb | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
meta/recipes-connectivity/openssl/openssl_3.2.0.bb +0 -0
index d041d2d214..7076bd3a7b 100644
@@ -247,7 +247,7 @@ CONFFILES:openssl-conf =

"${sysconfdir}/ssl/openssl.cnf"

RRECOMMENDS:libcrypto += "openssl-conf ${PN}-ossl-module-legacy" RDEPENDS:${PN}-misc = "perl"

meta/recipes-connectivity/openssl/openssl_3.2.0.bb +1 -2
-RDEPENDS:${PN}-ptest += "openssl-bin perl perl-modules bash sed"
+RDEPENDS:${PN}-ptest += "openssl-bin perl perl-modules-force-all bash sed"
 
 RDEPENDS:${PN}-bin += "openssl-conf"
 
--

2.43.0

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group. View/Reply Online (#193866):
https://lists.openembedded.org/g/openembedded-core/message/193866
Mute This Topic: https://lists.openembedded.org/mt/103766390/1686489
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [ [email protected]]
-=-=-=-=-=-=-=-=-=-=-=-

#3 lore raw

Re: [OE-core] [PATCH 2/2] openssl: Fix ptest dependencies on perl-module-* being ignored by COMPLEMENTARY_GLOB

Quoted 3 lines · Alexander Kanavin

How about simple depending on specific module packages? I don’t think OpenSSL truly needs all modules.

Agreed. I don’t see a compelling reason to add the perl-modules-force-all complication. This will add confusion in my opinion.

Iterate the tests and add the required perl-module-* sub-packages one by one (or by inspection if the test cases)

In almost all cases, RDEPENDS on perl-modules is a short cut and should be avoided in favor of being explicit.

Quoted 52 lines

Alex

Quoted 27 lines · William via lists.openembedded.org

From: William Lyu <[email protected]>

Change the openssl-ptest RDEPENDS from perl-modules to perl-modules-force-all which RDEPENDS on the same set of packages perl-module-* recommended by perl-modules. So, when openssl-ptest is added via COMPLEMENTARY_GLOB "ptest-pkgs", its dependencies on these perl-module-* packages still get added.

---
 meta/recipes-connectivity/openssl/openssl_3.2.0.bb | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
meta/recipes-connectivity/openssl/openssl_3.2.0.bb +0 -0
index d041d2d214..7076bd3a7b 100644
@@ -247,7 +247,7 @@ CONFFILES:openssl-conf =

"${sysconfdir}/ssl/openssl.cnf"

RRECOMMENDS:libcrypto += "openssl-conf ${PN}-ossl-module-legacy" RDEPENDS:${PN}-misc = "perl"

meta/recipes-connectivity/openssl/openssl_3.2.0.bb +1 -1
-RDEPENDS:${PN}-ptest += "openssl-bin perl perl-modules bash sed"
+RDEPENDS:${PN}-ptest += "openssl-bin perl perl-modules-force-all bash

sed"

Quoted 9 lines

RDEPENDS:${PN}-bin += "openssl-conf"

--
2.43.0

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group. View/Reply Online (#193868):
https://lists.openembedded.org/g/openembedded-core/message/193868
Mute This Topic: https://lists.openembedded.org/mt/103766390/924729
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [ [email protected]]
-=-=-=-=-=-=-=-=-=-=-=-

#4 lore raw

Re: [OE-core] [PATCH 2/2] openssl: Fix ptest dependencies on perl-module-* being ignored by COMPLEMENTARY_GLOB

Can we change the perl recipe to simply use 'RDEPENDS' for perl-modules? I don't see what benefit the 'RRECOMMENDS' could give us.

Regards,
Qi

Quoted 78 lines · Tim Orling

On Tue, Jan 16, 2024 at 9:21 AM Alexander Kanavin <[email protected]> wrote:

How about simple depending on specific module packages? I don’t think OpenSSL truly needs all modules.

Agreed. I don’t see a compelling reason to add the perl-modules-force-all complication. This will add confusion in my opinion.

Iterate the tests and add the required perl-module-* sub-packages one by one (or by inspection if the test cases)

In almost all cases, RDEPENDS on perl-modules is a short cut and should be avoided in favor of being explicit.

    Alex

    On Tue 16. Jan 2024 at 17.56, Lyu, William via
    lists.openembedded.org
    <https://urldefense.com/v3/__http://lists.openembedded.org__;!!AjveYdw8EvQ!cxKJn1SS8SQ9ODf0H1IfuN2qyCFPWHDlbuuYvuan7dIN9Nds2MS9DozRKwvNf7Di3p7lYRZ7IPV-BvhkVBVlxg$>
    <[email protected]> wrote:

        From: William Lyu <[email protected]>

        Change the openssl-ptest RDEPENDS from perl-modules to
        perl-modules-force-all which RDEPENDS on the same set of packages
        perl-module-* recommended by perl-modules. So, when
        openssl-ptest is
        added via COMPLEMENTARY_GLOB "ptest-pkgs", its dependencies on
        these
        perl-module-* packages still get added.

        Signed-off-by: William Lyu <[email protected]>
        ---
         meta/recipes-connectivity/openssl/openssl_3.2.0.bb
        <https://urldefense.com/v3/__http://openssl_3.2.0.bb__;!!AjveYdw8EvQ!cxKJn1SS8SQ9ODf0H1IfuN2qyCFPWHDlbuuYvuan7dIN9Nds2MS9DozRKwvNf7Di3p7lYRZ7IPV-BvgVPdEvRA$>
        | 2 +-
         1 file changed, 1 insertion(+), 1 deletion(-)

        diff --git
        a/meta/recipes-connectivity/openssl/openssl_3.2.0.bb
        <https://urldefense.com/v3/__http://openssl_3.2.0.bb__;!!AjveYdw8EvQ!cxKJn1SS8SQ9ODf0H1IfuN2qyCFPWHDlbuuYvuan7dIN9Nds2MS9DozRKwvNf7Di3p7lYRZ7IPV-BvgVPdEvRA$>
        b/meta/recipes-connectivity/openssl/openssl_3.2.0.bb
        <https://urldefense.com/v3/__http://openssl_3.2.0.bb__;!!AjveYdw8EvQ!cxKJn1SS8SQ9ODf0H1IfuN2qyCFPWHDlbuuYvuan7dIN9Nds2MS9DozRKwvNf7Di3p7lYRZ7IPV-BvgVPdEvRA$>
        index d041d2d214..7076bd3a7b 100644
        --- a/meta/recipes-connectivity/openssl/openssl_3.2.0.bb
        <https://urldefense.com/v3/__http://openssl_3.2.0.bb__;!!AjveYdw8EvQ!cxKJn1SS8SQ9ODf0H1IfuN2qyCFPWHDlbuuYvuan7dIN9Nds2MS9DozRKwvNf7Di3p7lYRZ7IPV-BvgVPdEvRA$>
        +++ b/meta/recipes-connectivity/openssl/openssl_3.2.0.bb
        <https://urldefense.com/v3/__http://openssl_3.2.0.bb__;!!AjveYdw8EvQ!cxKJn1SS8SQ9ODf0H1IfuN2qyCFPWHDlbuuYvuan7dIN9Nds2MS9DozRKwvNf7Di3p7lYRZ7IPV-BvgVPdEvRA$>
        @@ -247,7 +247,7 @@ CONFFILES:openssl-conf =
        "${sysconfdir}/ssl/openssl.cnf"

         RRECOMMENDS:libcrypto += "openssl-conf ${PN}-ossl-module-legacy"
         RDEPENDS:${PN}-misc = "perl"
        -RDEPENDS:${PN}-ptest += "openssl-bin perl perl-modules bash sed"
        +RDEPENDS:${PN}-ptest += "openssl-bin perl
        perl-modules-force-all bash sed"

         RDEPENDS:${PN}-bin += "openssl-conf"

        -- 
        2.43.0

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#193872):https://lists.openembedded.org/g/openembedded-core/message/193872 Mute This Topic:https://lists.openembedded.org/mt/103766390/7304865
Group Owner:[email protected] Unsubscribe:https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-

#5 lore raw

Re: [PATCH 2/2] openssl: Fix ptest dependencies on perl-module-* being ignored by COMPLEMENTARY_GLOB

Qi, according to the comments in the perl_5.38.2.bb recipe, perl-modules recommends "all the other perl packages" which might include optional packages. I assume leaving optional packages as "runtime recommended" (RRECOMMENDS) - aka. soft dependencies - can reduce the possibility of conflicts. The following patch changed the package_manager to ignore RRECOMMENDS to avoid conflict:

https://lists.openembedded.org/g/openembedded-core/message/167303

A message that is not in the archive (sent off-list or not yet received): [email protected]

#7 lore raw

Re: [OE-core] [PATCH 2/2] openssl: Fix ptest dependencies on perl-module-* being ignored by COMPLEMENTARY_GLOB

Quoted 12 lines · Tim Orling via lists.openembedded.org
Quoted 3 lines · Alexander Kanavin

How about simple depending on specific module packages? I don’t think OpenSSL truly needs all modules.

Agreed. I don’t see a compelling reason to add the perl-modules-force-all complication. This will add confusion in my opinion.

Quoted 7 lines

Iterate the tests and add the required perl-module-* sub-packages one by one (or by inspection if the test cases)

In almost all cases, RDEPENDS on perl-modules is a short cut and should be avoided in favor of being explicit.

This inspired me to sweep the dust off of

https://git.yoctoproject.org/poky-contrib/log/?h=timo/perl-deps_13376

This detects (but doesn't yet know how to properly handle) the RDEPENDS for openssl-ptest. See attached snippet of the log.

It is too greedy so some dependencies are not going to be needed.

Quoted 44 lines
Quoted 34 lines

Alex

Quoted 29 lines · William via lists.openembedded.org

From: William Lyu <[email protected]>

Change the openssl-ptest RDEPENDS from perl-modules to perl-modules-force-all which RDEPENDS on the same set of packages perl-module-* recommended by perl-modules. So, when openssl-ptest is added via COMPLEMENTARY_GLOB "ptest-pkgs", its dependencies on these perl-module-* packages still get added.

---
 meta/recipes-connectivity/openssl/openssl_3.2.0.bb | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
meta/recipes-connectivity/openssl/openssl_3.2.0.bb +0 -0
index d041d2d214..7076bd3a7b 100644
@@ -247,7 +247,7 @@ CONFFILES:openssl-conf =

"${sysconfdir}/ssl/openssl.cnf"

RRECOMMENDS:libcrypto += "openssl-conf ${PN}-ossl-module-legacy" RDEPENDS:${PN}-misc = "perl"

meta/recipes-connectivity/openssl/openssl_3.2.0.bb +1 -1
-RDEPENDS:${PN}-ptest += "openssl-bin perl perl-modules bash sed"
+RDEPENDS:${PN}-ptest += "openssl-bin perl perl-modules-force-all bash

sed"

RDEPENDS:${PN}-bin += "openssl-conf"

Quoted 10 lines
Quoted 7 lines

--
2.43.0

Quoted 11 lines

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group. View/Reply Online (#193872):
https://lists.openembedded.org/g/openembedded-core/message/193872
Mute This Topic: https://lists.openembedded.org/mt/103766390/924729
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [ [email protected]]
-=-=-=-=-=-=-=-=-=-=-=-

DEBUG: openssl-3.2.0-r0 do_perldeps: len(unresolved_rdeps[openssl-ptest]) = 82
DEBUG: openssl-3.2.0-r0 do_perldeps: unresolved_rdeps[openssl-ptest]:
DEBUG: openssl-3.2.0-r0 do_perldeps: perl(TLSProxy::ServerHello)
DEBUG: openssl-3.2.0-r0 do_perldeps: perl(OpenSSL::Util)
DEBUG: openssl-3.2.0-r0 do_perldeps: perl(Errno)
DEBUG: openssl-3.2.0-r0 do_perldeps: perl(Getopt::Std)
DEBUG: openssl-3.2.0-r0 do_perldeps: perl >= 0:5.008
DEBUG: openssl-3.2.0-r0 do_perldeps: perl(parent)
DEBUG: openssl-3.2.0-r0 do_perldeps: perl(FindBin)
DEBUG: openssl-3.2.0-r0 do_perldeps: perl(platform::mingw)
DEBUG: openssl-3.2.0-r0 do_perldeps: perl(List::Util)
DEBUG: openssl-3.2.0-r0 do_perldeps: perl(warnings)
DEBUG: openssl-3.2.0-r0 do_perldeps: perl(File::Basename)
DEBUG: openssl-3.2.0-r0 do_perldeps: perl(IO::Socket)
DEBUG: openssl-3.2.0-r0 do_perldeps: perl(OpenSSL::Glob)
DEBUG: openssl-3.2.0-r0 do_perldeps: perl(Time::Local)
DEBUG: openssl-3.2.0-r0 do_perldeps: perl >= 1:5.10.0
DEBUG: openssl-3.2.0-r0 do_perldeps: perl(OpenSSL::Template)
DEBUG: openssl-3.2.0-r0 do_perldeps: perl(Test::Pod) >= 1.41
DEBUG: openssl-3.2.0-r0 do_perldeps: perl(TLSProxy::NewSessionTicket)
DEBUG: openssl-3.2.0-r0 do_perldeps: perl(File::Glob)
DEBUG: openssl-3.2.0-r0 do_perldeps: perl(Carp)
DEBUG: openssl-3.2.0-r0 do_perldeps: perl(Text::Diff)
DEBUG: openssl-3.2.0-r0 do_perldeps: perl(Text::Template) >= 1.46
DEBUG: openssl-3.2.0-r0 do_perldeps: perl(OpenSSL::Test::Simple)
DEBUG: openssl-3.2.0-r0 do_perldeps: perl(Pod::Html)
DEBUG: openssl-3.2.0-r0 do_perldeps: perl(Getopt::Long)
DEBUG: openssl-3.2.0-r0 do_perldeps: perl(File::Path) >= 2.00
DEBUG: openssl-3.2.0-r0 do_perldeps: perl(ssltests_base)
DEBUG: openssl-3.2.0-r0 do_perldeps: perl(TLSProxy::CertificateRequest)
DEBUG: openssl-3.2.0-r0 do_perldeps: perl(platform::BASE)
DEBUG: openssl-3.2.0-r0 do_perldeps: perl(TLSProxy::Alert)
DEBUG: openssl-3.2.0-r0 do_perldeps: perl(OpenSSL::Test)
DEBUG: openssl-3.2.0-r0 do_perldeps: perl(Test::More) >= 0.96
DEBUG: openssl-3.2.0-r0 do_perldeps: perl(TLSProxy::ServerKeyExchange)
DEBUG: openssl-3.2.0-r0 do_perldeps: perl(platform)
DEBUG: openssl-3.2.0-r0 do_perldeps: perl(Config)
DEBUG: openssl-3.2.0-r0 do_perldeps: perl(File::Temp)
DEBUG: openssl-3.2.0-r0 do_perldeps: perl(platform::Windows)
DEBUG: openssl-3.2.0-r0 do_perldeps: perl(Data::Dumper)
DEBUG: openssl-3.2.0-r0 do_perldeps: perl(OpenSSL::fallback)
DEBUG: openssl-3.2.0-r0 do_perldeps: perl(OpenSSL::Ordinals)
DEBUG: openssl-3.2.0-r0 do_perldeps: perl(Math::BigInt)
DEBUG: openssl-3.2.0-r0 do_perldeps: perl(Pod::Usage)
DEBUG: openssl-3.2.0-r0 do_perldeps: perl(Encode)
DEBUG: openssl-3.2.0-r0 do_perldeps: perl(Cwd)
DEBUG: openssl-3.2.0-r0 do_perldeps: perl(strict)
DEBUG: openssl-3.2.0-r0 do_perldeps: perl(platform::Windows::MSVC)
DEBUG: openssl-3.2.0-r0 do_perldeps: perl(platform::Unix)
DEBUG: openssl-3.2.0-r0 do_perldeps: perl(utf8)
DEBUG: openssl-3.2.0-r0 do_perldeps: perl(Exporter)
DEBUG: openssl-3.2.0-r0 do_perldeps: perl(TLSProxy::EncryptedExtensions)
DEBUG: openssl-3.2.0-r0 do_perldeps: perl(constant)
DEBUG: openssl-3.2.0-r0 do_perldeps: perl(Fcntl)
DEBUG: openssl-3.2.0-r0 do_perldeps: perl(base)
DEBUG: openssl-3.2.0-r0 do_perldeps: perl(File::Copy)
DEBUG: openssl-3.2.0-r0 do_perldeps: perl(POSIX)
DEBUG: openssl-3.2.0-r0 do_perldeps: perl(checkhandshake)
DEBUG: openssl-3.2.0-r0 do_perldeps: perl(TLSProxy::Message)
DEBUG: openssl-3.2.0-r0 do_perldeps: perl(Storable)
DEBUG: openssl-3.2.0-r0 do_perldeps: perl(OpenSSL::Config::Query)
DEBUG: openssl-3.2.0-r0 do_perldeps: perl(TLSProxy::CertificateVerify)
DEBUG: openssl-3.2.0-r0 do_perldeps: perl(vars)
DEBUG: openssl-3.2.0-r0 do_perldeps: perl(File::Compare)
DEBUG: openssl-3.2.0-r0 do_perldeps: perl(configdata)
DEBUG: openssl-3.2.0-r0 do_perldeps: perl(IPC::Open2)
DEBUG: openssl-3.2.0-r0 do_perldeps: perl(FileHandle)
DEBUG: openssl-3.2.0-r0 do_perldeps: perl(Scalar::Util)
DEBUG: openssl-3.2.0-r0 do_perldeps: perl(OpenSSL::Test::Utils)
DEBUG: openssl-3.2.0-r0 do_perldeps: perl(MIME::Base64)
DEBUG: openssl-3.2.0-r0 do_perldeps: perl(TLSProxy::Certificate)
DEBUG: openssl-3.2.0-r0 do_perldeps: perl(gentemplate)
DEBUG: openssl-3.2.0-r0 do_perldeps: perl(lib)
DEBUG: openssl-3.2.0-r0 do_perldeps: perl(Digest::SHA)
DEBUG: openssl-3.2.0-r0 do_perldeps: perl(IO::Select)
DEBUG: openssl-3.2.0-r0 do_perldeps: perl(feature)
DEBUG: openssl-3.2.0-r0 do_perldeps: perl(File::Spec::Functions)
DEBUG: openssl-3.2.0-r0 do_perldeps: perl(TLSProxy::Proxy)
DEBUG: openssl-3.2.0-r0 do_perldeps: perl(Test::Harness)
DEBUG: openssl-3.2.0-r0 do_perldeps: perl(TLSProxy::Record)
DEBUG: openssl-3.2.0-r0 do_perldeps: perl(OpenSSL::ParseC)
DEBUG: openssl-3.2.0-r0 do_perldeps: perl(IPC::Cmd)
DEBUG: openssl-3.2.0-r0 do_perldeps: perl(TLSProxy::ClientHello)
DEBUG: openssl-3.2.0-r0 do_perldeps: perl(File::Spec)
WARNING: openssl-3.2.0-r0 do_perldeps: [openssl-ptest] Possible missing runtime dependencies detected, but they could not be resolved "perl(TLSProxy::ServerHello) perl(OpenSSL::Util) perl(Errno) perl(Getopt::Std) perl >= 0:5.008 perl(parent) perl(FindBin) perl(platform::mingw) perl(List::Util) perl(warnings) perl(File::Basename) perl(IO::Socket) perl(OpenSSL::Glob) perl(Time::Local) perl >= 1:5.10.0 perl(OpenSSL::Template) perl(Test::Pod) >= 1.41 perl(TLSProxy::NewSessionTicket) perl(File::Glob) perl(Carp) perl(Text::Diff) perl(Text::Template) >= 1.46 perl(OpenSSL::Test::Simple) perl(Pod::Html) perl(Getopt::Long) perl(File::Path) >= 2.00 perl(ssltests_base) perl(TLSProxy::CertificateRequest) perl(platform::BASE) perl(TLSProxy::Alert) perl(OpenSSL::Test) perl(Test::More) >= 0.96 perl(TLSProxy::ServerKeyExchange) perl(platform) perl(Config) perl(File::Temp) perl(platform::Windows) perl(Data::Dumper) perl(OpenSSL::fallback) perl(OpenSSL::Ordinals) perl(Math::BigInt) perl(Pod::Usage) perl(Encode) perl(Cwd) perl(strict) perl(platform::Windows::MSVC) perl(platform::Unix) perl(utf8) perl(Exporter) perl(TLSProxy::EncryptedExtensions) perl(constant) perl(Fcntl) perl(base) perl(File::Copy) perl(POSIX) perl(checkhandshake) perl(TLSProxy::Message) perl(Storable) perl(OpenSSL::Config::Query) perl(TLSProxy::CertificateVerify) perl(vars) perl(File::Compare) perl(configdata) perl(IPC::Open2) perl(FileHandle) perl(Scalar::Util) perl(OpenSSL::Test::Utils) perl(MIME::Base64) perl(TLSProxy::Certificate) perl(gentemplate) perl(lib) perl(Digest::SHA) perl(IO::Select) perl(feature) perl(File::Spec::Functions) perl(TLSProxy::Proxy) perl(Test::Harness) perl(TLSProxy::Record) perl(OpenSSL::ParseC) perl(IPC::Cmd) perl(TLSProxy::ClientHello) perl(File::Spec)"

#8 lore raw
In reply to [email protected]

If this is only for fixing OpenSSL ptest, then it’s not needed perhaps, once OpenSSL recipe is fixed?

Alex

Quoted 83 lines · William via lists.openembedded.org

From: William Lyu <[email protected]>

When perl-modules is added via COMPLEMENTARY_GLOB, all perl-module-* packages recommended by perl-modules are ignored due to the defined behavior of COMPLEMENTARY_GLOB.

This patch creates a new package perl-modules-force-all that RDEPENDS on the same set of packages perl-modules recommends. If another recipe always needs the packages recommended by perl-modules, it can add perl-modules-force-all to RDEPENDS to make sure that the recommended packages by perl-modules are not ignored by COMPLEMENTARY_GLOB.

---
 meta/recipes-devtools/perl/perl_5.38.2.bb | 22 ++++++++++++++++++++++
 1 file changed, 22 insertions(+)
meta/recipes-devtools/perl/perl_5.38.2.bb +0 -0
index a9d684cfc5..f75c5aa971 100644
@@ -311,6 +311,14 @@ ALTERNATIVE_LINK_NAME[Thread.3] =

"${mandir}/man3/Thread.3"
ALLOW_EMPTY:${PN}-modules = "1"
PACKAGES += "${PN}-modules "

+# When perl-modules is added through COMPLEMENTARY_GLOB, as it is recommending

meta/recipes-devtools/perl/perl_5.38.2.bb +2 -0
+# packages, all its recommendations will be ignored. So, we create a
+# perl-modules-force-all package depending on the same set of packages as

listed
+# in perl-modules to provide a way to guarantee COMPLEMENTARY_GLOB not removing

meta/recipes-devtools/perl/perl_5.38.2.bb +7 -0
+# these packages.
+ALLOW_EMPTY:${PN}-modules-force-all = "1"
+PACKAGES += "${PN}-modules-force-all "
+
 PACKAGESPLITFUNCS =+ "split_perl_packages"
 
 python split_perl_packages () {
@@ -328,6 +336,20 @@ python split_perl_packages () {
     packages = filter(lambda p: 'perl-module-' in p,
 d.getVar('PACKAGES').split())
     d.setVar(d.expand("RRECOMMENDS:${PN}-modules"), ' '.join(packages))
 
+    # perl-modules-force-all provides a way to prevent COMPLEMENTARY_GLOB
+    # from removing the packages specified in perl-modules.
+    # perl-modules-force-all contains the same set of packages as

perl-modules.

meta/recipes-devtools/perl/perl_5.38.2.bb +6 -0
+    d.setVar(
+        d.expand("RDEPENDS:${PN}-modules-force-all"),
+        d.getVar(d.expand("RRECOMMENDS:${PN}-modules"))
+    )
+
+    # As perl-modules-force-all depends on all other, potentially

non-existent,

meta/recipes-devtools/perl/perl_5.38.2.bb +3 -0
+    # perl packages, we allow these packages to be empty to prevent build
+    # errors.
+    for pkg in

d.getVar(d.expand("RDEPENDS:${PN}-modules-force-all")).split():

meta/recipes-devtools/perl/perl_5.38.2.bb +2 -0
+        d.setVar(d.expand("ALLOW_EMPTY:" + pkg), "1")
+
     # Read the pre-generated dependency file, and use it to set module

dependecies

     for line in open(d.expand("${WORKDIR}") +

'/perl-rdepends.txt').readlines():
splitline = line.split()
--
2.43.0

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group. View/Reply Online (#193867):
https://lists.openembedded.org/g/openembedded-core/message/193867
Mute This Topic: https://lists.openembedded.org/mt/103766391/1686489
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [ [email protected]]
-=-=-=-=-=-=-=-=-=-=-=-

#9 lore raw

I have iteratively added the following 45 modules, and I still not at the point of covering all perl-module-* dependencies for openssl-test. Should I continue on with this effort? Using "perl-module-force-all" is a lot more convenient without conflicting with any existing usage of "perl-modules" - even including the ones that depends on "perl-modules" having the property of being a soft dependency ("perl-modules" RRECOMMENDS all of its packages).

William

List of perl-module-* I have iteratively added up to this point:

perl-module-base
perl-module-carp
perl-module-config
perl-module-constant
perl-module-cwd
perl-module-data-dumper
perl-module-digest-sha
perl-module-encode
perl-module-errno
perl-module-exporter
perl-module-fcntl
perl-module-feature
perl-module-file-basename
perl-module-file-compare
perl-module-file-copy
perl-module-file-glob
perl-module-filehandle
perl-module-file-path
perl-module-file-spec
perl-module-file-spec-functions
perl-module-file-temp
perl-module-findbin
perl-module-getopt-long
perl-module-getopt-std
perl-module-io-select
perl-module-io-socket
perl-module-ipc-cmd
perl-module-ipc-open2
perl-module-lib
perl-module-list-util
perl-module-math-bigint
perl-module-mime-base64
perl-module-parent
perl-module-pod-html
perl-module-pod-usage
perl-module-posix
perl-module-scalar-util
perl-module-storable
perl-module-strict
perl-module-test-harness
perl-module-test-more
perl-module-time-local
perl-module-utf8
perl-module-vars
perl-module-warnings

#10 lore raw

How are you making the list? I think this might be somewhat sped up, if you first grep ${WORKDIR}/packages-split/openssl-ptest/usr/lib/openssl/ptest/test/ (and util/perl/OpenSSL/ as well) for "^use " and transform the output lines into 'perl-module-something'. Then try it on the target in qemu to see whether something is missing.

Alex

Quoted 61 lines · William via lists.openembedded.org

I have iteratively added the following 45 modules, and I still not at the point of covering all perl-module-* dependencies for openssl-test. Should I continue on with this effort? Using "perl-module-force-all" is a lot more convenient without conflicting with any existing usage of "perl-modules" - even including the ones that depends on "perl-modules" having the property of being a soft dependency ("perl-modules" RRECOMMENDS all of its packages).

William

List of perl-module-* I have iteratively added up to this point:

perl-module-base
perl-module-carp
perl-module-config
perl-module-constant
perl-module-cwd
perl-module-data-dumper
perl-module-digest-sha
perl-module-encode
perl-module-errno
perl-module-exporter
perl-module-fcntl
perl-module-feature
perl-module-file-basename
perl-module-file-compare
perl-module-file-copy
perl-module-file-glob
perl-module-filehandle
perl-module-file-path
perl-module-file-spec
perl-module-file-spec-functions
perl-module-file-temp
perl-module-findbin
perl-module-getopt-long
perl-module-getopt-std
perl-module-io-select
perl-module-io-socket
perl-module-ipc-cmd
perl-module-ipc-open2
perl-module-lib
perl-module-list-util
perl-module-math-bigint
perl-module-mime-base64
perl-module-parent
perl-module-pod-html
perl-module-pod-usage
perl-module-posix
perl-module-scalar-util
perl-module-storable
perl-module-strict
perl-module-test-harness
perl-module-test-more
perl-module-time-local
perl-module-utf8
perl-module-vars
perl-module-warnings

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#193949): https://lists.openembedded.org/g/openembedded-core/message/193949
Mute This Topic: https://lists.openembedded.org/mt/103766391/1686489
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-

#11 lore raw

Following your suggestions and iteratively adding packages based on error message, I am finally able to come up with quite a long list of perl-module-* packages that fix the dependencies of openssl-ptest.

However, I am worried about if the list I came up with is truly comprehensive in the sense that this list covers all dependencies for other architectures and other OSes. I see that a lot of openssl-ptest subtests are marked as "skipped". I am not sure if these dependencies will be sufficient if these subtests are not skipped. Also, I am not sure if the packages I added are arch or OS specific.

The following is the list of perl-module-* I found.

This first portion is identified by following your suggestion of using grep "^\s*use". perl-module-extutils-makemaker \
perl-module-file-find \
perl-module-if \
perl-module-pod-checker \
perl-module-tap-harness \
perl-module-tap-parser-aggregator \
perl-module-tap-parser-source \
perl-module-tap-parser-sourcehandler-perl \
perl-module-text-parsewords \
perl-module-carp \
perl-module-cwd \
perl-module-data-dumper \
perl-module-digest-sha \
perl-module-encode \
perl-module-errno \
perl-module-exporter \
perl-module-fcntl \
perl-module-feature \
perl-module-file-basename \
perl-module-file-compare \
perl-module-file-copy \
perl-module-file-glob \
perl-module-filehandle \
perl-module-file-path \
perl-module-file-spec \
perl-module-file-spec-functions \
perl-module-file-temp \
perl-module-findbin \
perl-module-getopt-long \
perl-module-getopt-std \
perl-module-io-select \
perl-module-io-socket \
perl-module-ipc-cmd \
perl-module-ipc-open2 \
perl-module-lib \
perl-module-list-util \
perl-module-math-bigint \
perl-module-mime-base64 \
perl-module-pod-html \
perl-module-posix \
perl-module-scalar-util \
perl-module-storable \
perl-module-test-harness \
perl-module-test-more \
perl-module-time-local \
perl-module-utf8 \
perl-module-vars \
\

The following are added iteratively based on error reports from actually running the test. perl-module-pod-text \
perl-module-encode-encoding \
perl-module-metadata \
perl-module-locale \
\

The following are added due to internal dependencies between different TAP modules. I added a few iteratively, and there are still some errors. So, I just grepped and added all tap modules. perl-module-tap-base \
perl-module-tap-formatter-base \
perl-module-tap-formatter-color \
perl-module-tap-formatter-console \
perl-module-tap-formatter-console-parallelsession \
perl-module-tap-formatter-console-session \
perl-module-tap-formatter-file \
perl-module-tap-formatter-file-session \
perl-module-tap-formatter-session \
perl-module-tap-harness \
perl-module-tap-harness-env \
perl-module-tap-object \
perl-module-tap-parser \
perl-module-tap-parser-aggregator \
perl-module-tap-parser-grammar \
perl-module-tap-parser-iterator \
perl-module-tap-parser-iterator-array \
perl-module-tap-parser-iteratorfactory \
perl-module-tap-parser-iterator-process \
perl-module-tap-parser-iterator-stream \
perl-module-tap-parser-multiplexer \
perl-module-tap-parser-result \
perl-module-tap-parser-result-bailout \
perl-module-tap-parser-result-comment \
perl-module-tap-parser-resultfactory \
perl-module-tap-parser-result-plan \
perl-module-tap-parser-result-pragma \
perl-module-tap-parser-result-test \
perl-module-tap-parser-result-unknown \
perl-module-tap-parser-result-version \
perl-module-tap-parser-result-yaml \
perl-module-tap-parser-scheduler \
perl-module-tap-parser-scheduler-job \
perl-module-tap-parser-scheduler-spinner \
perl-module-tap-parser-source \
perl-module-tap-parser-sourcehandler \
perl-module-tap-parser-sourcehandler-executable \
perl-module-tap-parser-sourcehandler-file \
perl-module-tap-parser-sourcehandler-handle \
perl-module-tap-parser-sourcehandler-perl \
perl-module-tap-parser-sourcehandler-rawtap \
perl-module-tap-parser-yamlish-reader \
perl-module-tap-parser-yamlish-writer \
\

Only perl-module-extutils-mm-unix was requested. However, I noticed that the suffix has "unix", so I assumed that this module is OS dependant. In case openssl-ptest needs to be built for other OSes, I added all perl-module-extutils-mm*. perl-module-extutils-mm \
perl-module-extutils-mm-aix \
perl-module-extutils-mm-any \
perl-module-extutils-mm-beos \
perl-module-extutils-mm-cygwin \
perl-module-extutils-mm-darwin \
perl-module-extutils-mm-dos \
perl-module-extutils-mm-macos \
perl-module-extutils-mm-nw5 \
perl-module-extutils-mm-os2 \
perl-module-extutils-mm-os390 \
perl-module-extutils-mm-qnx \
perl-module-extutils-mm-unix \
perl-module-extutils-mm-uwin \
perl-module-extutils-mm-vms \
perl-module-extutils-mm-vos \
perl-module-extutils-mm-win32 \
perl-module-extutils-mm-win95 \

===== APPENDIX: Commands I Used to Identify the Potential List of Packages Needed ===== These are details and can be ignored.

Under "{build directory}/tmp/work/qemux86_64-poky-linux/core-image-minimal/1.0/rootfs/usr/lib/openssl/ptest", I executed the following command to convert all imported modules using "use" statements into "perl-module-*":

$ grep -r -n -I "^\s*use " | sed -E 's/.*:[0-9]+://' | sed -E 's/;/\n/g' | sed -E 's/^ *use *//' | sed -E 's/ /\n/g' | awk '!/^\s*$/' | tr '[:upper:]' '[:lower:]' | sed -E 's/::/-/g' | sed -E 's/^/perl-module-/' | sort | uniq

I saved the above list into a file "PARSED_USE". Then, I listed all directories under "tmp/work/core2-64-poky-linux/perl/5.38.2/packages-split" and saved this list as a list of valid perl-module-* into a file "VALID_MODULE".

I used the following command to determine which line in file "PARSED_USE" is a valid perl-module-* based on "VALID_MODULE" using the following command:

$ cat PARSED_USE VALID_MODULE | sort | uniq -c | sort -nk1,1 | grep -E '^ *2'
#12 lore raw

Thank you for doing this deep level investigation. I think this pretty much shows that listing all modules in a way that can be maintained isn't feasible (the idea was that maybe there's only a few of them needed for ptest).

Which brings us back to the question: why is perl-modules using RRECOMMENDS and not RDEPENDS? I went through git history, and perl recipe had RRECOMMENDS from the very first mass-import commit in 2005, so the answer is likely no one knows or remembers.

I also went and checked python: in there python3-modules is using RDEPENDS.

So I think the easiest way out is to simply replace RRECOMMENDS with RDEPENDS in perl as well. Can you make a patch?

Alex

Quoted 153 lines · William via lists.openembedded.org

Following your suggestions and iteratively adding packages based on error message, I am finally able to come up with quite a long list of perl-module-* packages that fix the dependencies of openssl-ptest.

However, I am worried about if the list I came up with is truly comprehensive in the sense that this list covers all dependencies for other architectures and other OSes. I see that a lot of openssl-ptest subtests are marked as "skipped". I am not sure if these dependencies will be sufficient if these subtests are not skipped. Also, I am not sure if the packages I added are arch or OS specific.

The following is the list of perl-module-* I found.

This first portion is identified by following your suggestion of using grep "^\s*use".

    perl-module-extutils-makemaker \
    perl-module-file-find \
    perl-module-if \
    perl-module-pod-checker \
    perl-module-tap-harness \
    perl-module-tap-parser-aggregator \
    perl-module-tap-parser-source \
    perl-module-tap-parser-sourcehandler-perl \
    perl-module-text-parsewords \
    perl-module-carp \
    perl-module-cwd \
    perl-module-data-dumper \
    perl-module-digest-sha \
    perl-module-encode \
    perl-module-errno \
    perl-module-exporter \
    perl-module-fcntl \
    perl-module-feature \
    perl-module-file-basename \
    perl-module-file-compare \
    perl-module-file-copy \
    perl-module-file-glob \
    perl-module-filehandle \
    perl-module-file-path \
    perl-module-file-spec \
    perl-module-file-spec-functions \
    perl-module-file-temp \
    perl-module-findbin \
    perl-module-getopt-long \
    perl-module-getopt-std \
    perl-module-io-select \
    perl-module-io-socket \
    perl-module-ipc-cmd \
    perl-module-ipc-open2 \
    perl-module-lib \
    perl-module-list-util \
    perl-module-math-bigint \
    perl-module-mime-base64 \
    perl-module-pod-html \
    perl-module-posix \
    perl-module-scalar-util \
    perl-module-storable \
    perl-module-test-harness \
    perl-module-test-more \
    perl-module-time-local \
    perl-module-utf8 \
    perl-module-vars \
    \

The following are added iteratively based on error reports from actually running the test.

    perl-module-pod-text \
    perl-module-encode-encoding \
    perl-module-metadata \
    perl-module-locale \
    \

The following are added due to internal dependencies between different TAP modules. I added a few iteratively, and there are still some errors. So, I just grepped and added all tap modules.

    perl-module-tap-base \
    perl-module-tap-formatter-base \
    perl-module-tap-formatter-color \
    perl-module-tap-formatter-console \
    perl-module-tap-formatter-console-parallelsession \
    perl-module-tap-formatter-console-session \
    perl-module-tap-formatter-file \
    perl-module-tap-formatter-file-session \
    perl-module-tap-formatter-session \
    perl-module-tap-harness \
    perl-module-tap-harness-env \
    perl-module-tap-object \
    perl-module-tap-parser \
    perl-module-tap-parser-aggregator \
    perl-module-tap-parser-grammar \
    perl-module-tap-parser-iterator \
    perl-module-tap-parser-iterator-array \
    perl-module-tap-parser-iteratorfactory \
    perl-module-tap-parser-iterator-process \
    perl-module-tap-parser-iterator-stream \
    perl-module-tap-parser-multiplexer \
    perl-module-tap-parser-result \
    perl-module-tap-parser-result-bailout \
    perl-module-tap-parser-result-comment \
    perl-module-tap-parser-resultfactory \
    perl-module-tap-parser-result-plan \
    perl-module-tap-parser-result-pragma \
    perl-module-tap-parser-result-test \
    perl-module-tap-parser-result-unknown \
    perl-module-tap-parser-result-version \
    perl-module-tap-parser-result-yaml \
    perl-module-tap-parser-scheduler \
    perl-module-tap-parser-scheduler-job \
    perl-module-tap-parser-scheduler-spinner \
    perl-module-tap-parser-source \
    perl-module-tap-parser-sourcehandler \
    perl-module-tap-parser-sourcehandler-executable \
    perl-module-tap-parser-sourcehandler-file \
    perl-module-tap-parser-sourcehandler-handle \
    perl-module-tap-parser-sourcehandler-perl \
    perl-module-tap-parser-sourcehandler-rawtap \
    perl-module-tap-parser-yamlish-reader \
    perl-module-tap-parser-yamlish-writer \
    \

Only perl-module-extutils-mm-unix was requested. However, I noticed that the suffix has "unix", so I assumed that this module is OS dependant. In case openssl-ptest needs to be built for other OSes, I added all perl-module-extutils-mm*.

    perl-module-extutils-mm \
    perl-module-extutils-mm-aix \
    perl-module-extutils-mm-any \
    perl-module-extutils-mm-beos \
    perl-module-extutils-mm-cygwin \
    perl-module-extutils-mm-darwin \
    perl-module-extutils-mm-dos \
    perl-module-extutils-mm-macos \
    perl-module-extutils-mm-nw5 \
    perl-module-extutils-mm-os2 \
    perl-module-extutils-mm-os390 \
    perl-module-extutils-mm-qnx \
    perl-module-extutils-mm-unix \
    perl-module-extutils-mm-uwin \
    perl-module-extutils-mm-vms \
    perl-module-extutils-mm-vos \
    perl-module-extutils-mm-win32 \
    perl-module-extutils-mm-win95 \

===== APPENDIX: Commands I Used to Identify the Potential List of Packages Needed ===== These are details and can be ignored.

Under "{build directory}/tmp/work/qemux86_64-poky-linux/core-image-minimal/1.0/rootfs/usr/lib/openssl/ptest", I executed the following command to convert all imported modules using "use" statements into "perl-module-*":

$ grep -r -n -I "^\s*use " | sed -E 's/.*:[0-9]+://' | sed -E 's/;/\n/g' | sed -E 's/^ *use *//' | sed -E 's/ /\n/g' | awk '!/^\s*$/' | tr '[:upper:]' '[:lower:]' | sed -E 's/::/-/g' | sed -E 's/^/perl-module-/' | sort | uniq

I saved the above list into a file "PARSED_USE". Then, I listed all directories under "tmp/work/core2-64-poky-linux/perl/5.38.2/packages-split" and saved this list as a list of valid perl-module-* into a file "VALID_MODULE".

I used the following command to determine which line in file "PARSED_USE" is a valid perl-module-* based on "VALID_MODULE" using the following command:

$ cat PARSED_USE VALID_MODULE | sort | uniq -c | sort -nk1,1 | grep -E '^ *2'

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#194062): https://lists.openembedded.org/g/openembedded-core/message/194062
Mute This Topic: https://lists.openembedded.org/mt/103766391/1686489
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-