Skip to contentSkip to conversation

procps: split into binary subpackages

PATCH Superseded Sinan Kaya 3 participants 8 messages Dec 1, 2020

v2 of 3 Superseded Latest is v3 →

lore
Share & tools
lore

Sinan Kaya

author
#0 lore raw

PATCH v2 1/1 procps: split into binary subpackages

This change allows you to pull individual binaries into the target without including the entire suite.

---
 meta/recipes-extended/procps/procps_3.3.16.bb | 51 +++++++++++++++++--
 1 file changed, 48 insertions(+), 3 deletions(-)
meta/recipes-extended/procps/procps_3.3.16.bb +48 -3
index 2810ebd285a..1b62ae33f5c 100644
@@ -50,15 +50,13 @@ do_install_append () {
 
 CONFFILES_${PN} = "${sysconfdir}/sysctl.conf"
 
-bindir_progs = "free pkill pmap pgrep pwdx skill snice top uptime w"
+bindir_progs = "free pkill pmap pgrep pwdx skill slabtop snice tload top uptime vmstat w"
 base_bindir_progs += "kill pidof ps watch"
 base_sbindir_progs += "sysctl"
 
 ALTERNATIVE_PRIORITY = "200"
 ALTERNATIVE_PRIORITY[pidof] = "150"
 
-ALTERNATIVE_${PN} = "${bindir_progs} ${base_bindir_progs} ${base_sbindir_progs}"
-
 ALTERNATIVE_${PN}-doc = "kill.1 uptime.1"
 ALTERNATIVE_LINK_NAME[kill.1] = "${mandir}/man1/kill.1"
 ALTERNATIVE_LINK_NAME[uptime.1] = "${mandir}/man1/uptime.1"
@@ -69,8 +67,55 @@ python __anonymous() {
 
     for prog in d.getVar('base_sbindir_progs').split():
         d.setVarFlag('ALTERNATIVE_LINK_NAME', prog, '%s/%s' % (d.getVar('base_sbindir'), prog))
+
+    for prog in d.getVar('bindir_progs').split():
+        d.setVarFlag('ALTERNATIVE_LINK_NAME', prog, '%s/%s' % (d.getVar('bindir'), prog))
 }
 
 # 'ps' isn't suitable for use as a security tool so whitelist this CVE.
 # https://bugzilla.redhat.com/show_bug.cgi?id=1575473#c3
 CVE_CHECK_WHITELIST += "CVE-2018-1121"
+
+python __anonymous () {
+    packages = []
+    for prog in d.getVar('bindir_progs').split():
+        pkg = "procps-%s" % prog.replace("_", "-")
+        packages.append(pkg)
+        d.setVar("RDEPENDS_%s" % pkg, "procps-lib")
+        d.setVar("ALTERNATIVE_%s" % pkg, prog.replace("_", "-"))
+        if not d.getVar("FILES_%s" % pkg):
+                d.setVar("FILES_%s" % pkg, "${bindir}/%s*" % prog)
+    for prog in d.getVar('base_sbindir_progs').split():
+        pkg = "procps-%s" % prog.replace("_", "-")
+        packages.append(pkg)
+        d.setVar("RDEPENDS_%s" % pkg, "procps-lib")
+        d.setVar("ALTERNATIVE_%s" % pkg, prog.replace("_", "-"))
+        if not d.getVar("FILES_%s" % pkg):
+                d.setVar("FILES_%s" % pkg, "${base_sbindir}/%s*" % prog)
+    for prog in d.getVar('base_bindir_progs').split():
+        pkg = "procps-%s" % prog.replace("_", "-")
+        packages.append(pkg)
+        d.setVar("RDEPENDS_%s" % pkg, "procps-lib")
+        d.setVar("ALTERNATIVE_%s" % pkg, prog.replace("_", "-"))
+        if not d.getVar("FILES_%s" % pkg):
+                d.setVar("FILES_%s" % pkg, "${base_bindir}/%s*" % prog)
+    pkg = "procps-man"
+    packages.append(pkg)
+    if not d.getVar("FILES_%s" % pkg):
+            d.setVar("FILES_%s" % pkg, "${mandir}/*")
+    pkg = "procps-lib"
+    packages.append(pkg)
+    if not d.getVar("FILES_%s" % pkg):
+            d.setVar("FILES_%s" % pkg, "${libdir}/*")
+    pkg = "procps-sysctl"
+    d.appendVar("FILES_%s" % pkg, " ${sysconfdir}/sysctl.conf")
+
+    d.setVar("PROCPS_PACKAGES", " ".join(packages))
+}
+# "procps" is a metapackage which pulls in all procps binaries
+PACKAGES += "${PROCPS_PACKAGES}"
+PROVIDES += "${PROCPS_PACKAGES}"
+FILES_${PN} = ""
+ALLOW_EMPTY_${PN} = "1"
+RRECOMMENDS_${PN} += "${PROCPS_PACKAGES}"
+
Signature · 1 line
2.17.1
#1 lore raw
Quoted 74 lines · Sinan Kaya

This change allows you to pull individual binaries into the target without including the entire suite.

---
 meta/recipes-extended/procps/procps_3.3.16.bb | 51 +++++++++++++++++--
 1 file changed, 48 insertions(+), 3 deletions(-)
meta/recipes-extended/procps/procps_3.3.16.bb +37 -3
index 2810ebd285a..1b62ae33f5c 100644
@@ -50,15 +50,13 @@ do_install_append () {
 
 CONFFILES_${PN} = "${sysconfdir}/sysctl.conf"
 
-bindir_progs = "free pkill pmap pgrep pwdx skill snice top uptime w"
+bindir_progs = "free pkill pmap pgrep pwdx skill slabtop snice tload top uptime vmstat w"
 base_bindir_progs += "kill pidof ps watch"
 base_sbindir_progs += "sysctl"
 
 ALTERNATIVE_PRIORITY = "200"
 ALTERNATIVE_PRIORITY[pidof] = "150"
 
-ALTERNATIVE_${PN} = "${bindir_progs} ${base_bindir_progs} ${base_sbindir_progs}"
-
 ALTERNATIVE_${PN}-doc = "kill.1 uptime.1"
 ALTERNATIVE_LINK_NAME[kill.1] = "${mandir}/man1/kill.1"
 ALTERNATIVE_LINK_NAME[uptime.1] = "${mandir}/man1/uptime.1"
@@ -69,8 +67,55 @@ python __anonymous() {
 
     for prog in d.getVar('base_sbindir_progs').split():
         d.setVarFlag('ALTERNATIVE_LINK_NAME', prog, '%s/%s' % (d.getVar('base_sbindir'), prog))
+
+    for prog in d.getVar('bindir_progs').split():
+        d.setVarFlag('ALTERNATIVE_LINK_NAME', prog, '%s/%s' % (d.getVar('bindir'), prog))
 }
 
 # 'ps' isn't suitable for use as a security tool so whitelist this CVE.
 # https://bugzilla.redhat.com/show_bug.cgi?id=1575473#c3
 CVE_CHECK_WHITELIST += "CVE-2018-1121"
+
+python __anonymous () {
+    packages = []
+    for prog in d.getVar('bindir_progs').split():
+        pkg = "procps-%s" % prog.replace("_", "-")
+        packages.append(pkg)
+        d.setVar("RDEPENDS_%s" % pkg, "procps-lib")
+        d.setVar("ALTERNATIVE_%s" % pkg, prog.replace("_", "-"))
+        if not d.getVar("FILES_%s" % pkg):
+                d.setVar("FILES_%s" % pkg, "${bindir}/%s*" % prog)
+    for prog in d.getVar('base_sbindir_progs').split():
+        pkg = "procps-%s" % prog.replace("_", "-")
+        packages.append(pkg)
+        d.setVar("RDEPENDS_%s" % pkg, "procps-lib")
+        d.setVar("ALTERNATIVE_%s" % pkg, prog.replace("_", "-"))
+        if not d.getVar("FILES_%s" % pkg):
+                d.setVar("FILES_%s" % pkg, "${base_sbindir}/%s*" % prog)
+    for prog in d.getVar('base_bindir_progs').split():
+        pkg = "procps-%s" % prog.replace("_", "-")
+        packages.append(pkg)
+        d.setVar("RDEPENDS_%s" % pkg, "procps-lib")
+        d.setVar("ALTERNATIVE_%s" % pkg, prog.replace("_", "-"))
+        if not d.getVar("FILES_%s" % pkg):
+                d.setVar("FILES_%s" % pkg, "${base_bindir}/%s*" % prog)
+    pkg = "procps-man"
+    packages.append(pkg)
+    if not d.getVar("FILES_%s" % pkg):
+            d.setVar("FILES_%s" % pkg, "${mandir}/*")
+    pkg = "procps-lib"
+    packages.append(pkg)
+    if not d.getVar("FILES_%s" % pkg):
+            d.setVar("FILES_%s" % pkg, "${libdir}/*")
+    pkg = "procps-sysctl"

meta/recipes-extended/procps/procps_3.3.16.bb python __anonymous() { · line 120

Quoted 10 lines
meta/recipes-extended/procps/procps_3.3.16.bb +10 -0
+    d.appendVar("FILES_%s" % pkg, " ${sysconfdir}/sysctl.conf")
+
+    d.setVar("PROCPS_PACKAGES", " ".join(packages))
+}
+# "procps" is a metapackage which pulls in all procps binaries
+PACKAGES += "${PROCPS_PACKAGES}"
+PROVIDES += "${PROCPS_PACKAGES}"
+FILES_${PN} = ""
+ALLOW_EMPTY_${PN} = "1"
+RRECOMMENDS_${PN} += "${PROCPS_PACKAGES}"

I am starting to get a little worried about the direction these patches are heading in. How much of the system are we going to split into individual package per binaries?

At some point the packages stop becoming useful and you have to consider whather package management is the best way to build these rootfs since I doubt you're enabling package management in the end result anyway given the size gains you're aiming for?

I'd like to understand and discuss the end goal here before we merge too many large chunks of custom python like the above?

An alternative may be to create a library function which can be optionally enabled for recipes as this code and the other code you sent does look similar, at least superficially...

Cheers,

Richard

#2 lore raw

Hello Richard,

Em ter., 1 de dez. de 2020 às 07:37, Richard Purdie <[email protected]> escreveu:

Quoted 1 line · Sinan Kaya

...

Quoted 3 lines

I am starting to get a little worried about the direction these patches are heading in. How much of the system are we going to split into individual package per binaries?

I am wondering why this is a concern for you? If we keep the old package rdepends on the new ones I see no problem in allowing this granular packaging.

Signature · 3 lines
Otavio Salvador                             O.S. Systems
http://www.ossystems.com.br        http://code.ossystems.com.br
Mobile: +55 (53) 9 9981-7854          Mobile: +1 (347) 903-9750
#3 lore raw
Quoted 6 lines · Otavio Salvador

Hello Richard,

Em ter., 1 de dez. de 2020 às 07:37, Richard Purdie <[email protected]> escreveu:

Quoted 1 line · Sinan Kaya

...

Quoted 8 lines
Quoted 4 lines

I am starting to get a little worried about the direction these patches
are heading in. How much of the system are we going to split into individual package per binaries?

I am wondering why this is a concern for you? If we keep the old package rdepends on the new ones I see no problem in allowing this granular packaging.

Taking this to a conclusion its heading towards, most recipes generating more than one binary would end up with this splitting code. I don't like having large blocks of python in each recipe and heading that way means we should probably change approach somehow.

My worry is that simpler recipes are easier to maintain, test and upgrade.

Cheers,

Richard

#4 lore raw

Em ter., 1 de dez. de 2020 às 11:15, Richard Purdie <[email protected]> escreveu:

Quoted 16 lines · Otavio Salvador
Quoted 6 lines

Hello Richard,

Em ter., 1 de dez. de 2020 às 07:37, Richard Purdie <[email protected]> escreveu:

Quoted 1 line · Sinan Kaya

...

Quoted 8 lines
Quoted 4 lines

I am starting to get a little worried about the direction these patches
are heading in. How much of the system are we going to split into individual package per binaries?

I am wondering why this is a concern for you? If we keep the old package rdepends on the new ones I see no problem in allowing this granular packaging.

Quoted 7 lines

Taking this to a conclusion its heading towards, most recipes generating more than one binary would end up with this splitting code. I don't like having large blocks of python in each recipe and heading that way means we should probably change approach somehow.

My worry is that simpler recipes are easier to maintain, test and upgrade.

Maybe Sinan could try to rework this and move the python code to a class reducing code duplication?

Signature · 3 lines
Otavio Salvador                             O.S. Systems
http://www.ossystems.com.br        http://code.ossystems.com.br
Mobile: +55 (53) 9 9981-7854          Mobile: +1 (347) 903-9750
author
#5 lore raw
Quoted 12 lines · Otavio Salvador
Quoted 12 lines
Quoted 7 lines
Quoted 4 lines

I am starting to get a little worried about the direction these patches
are heading in. How much of the system are we going to split into individual package per binaries?

I am wondering why this is a concern for you? If we keep the old package rdepends on the new ones I see no problem in allowing this granular packaging.

Taking this to a conclusion its heading towards, most recipes generating more than one binary would end up with this splitting code. I don't like having large blocks of python in each recipe and heading that way means we should probably change approach somehow.

Quoted 4 lines
Quoted 2 lines

My worry is that simpler recipes are easier to maintain, test and upgrade.

Maybe Sinan could try to rework this and move the python code to a class reducing code duplication?

The problem I'm trying to solve is I only need ps file out of this entire package. Everything else in this package is useless for me. I'm sure no-one wants dead code in their system especially if they are size constrained.

Ideal solution would be to have --with/without-foo support upstream that we can configure with PACKAGECONFIG.

I'm happy to look at other options if there is an alternative.

#6 lore raw
Quoted 23 lines · Sinan Kaya
Quoted 12 lines · Otavio Salvador
Quoted 12 lines
Quoted 7 lines
Quoted 4 lines

I am starting to get a little worried about the direction these patches
are heading in. How much of the system are we going to split into individual package per binaries?

I am wondering why this is a concern for you? If we keep the old package rdepends on the new ones I see no problem in allowing this granular packaging.

Taking this to a conclusion its heading towards, most recipes generating more than one binary would end up with this splitting code. I don't like having large blocks of python in each recipe and heading that way means we should probably change approach somehow.

Quoted 4 lines
Quoted 2 lines

My worry is that simpler recipes are easier to maintain, test and upgrade.

Maybe Sinan could try to rework this and move the python code to a class reducing code duplication?

The problem I'm trying to solve is I only need ps file out of this entire package. Everything else in this package is useless for me. I'm sure no-one wants dead code in their system especially if they are size constrained.

Quoted 4 lines

Ideal solution would be to have --with/without-foo support upstream that we can configure with PACKAGECONFIG.

I'm happy to look at other options if there is an alternative.

The question is how many other packages are you going to need just one thing out of?

You might want ps, someone else might want something different. We've had this problem a lot with util-linux and e2fsprogs so we split those in the end as the patches were getting painful.

If this is a widespread issue we might want to have a generic class, kind of like lib_package but for splitting the binaries into individual packages.

I just need an idea of how big this problem is going to be as I don't want to iterate over half the recipes in oe-core over the next 9 months! :)

Cheers,

Richard

author
#7 lore raw
Quoted 36 lines · Richard Purdie
Quoted 23 lines · Sinan Kaya
Quoted 12 lines · Otavio Salvador
Quoted 12 lines
Quoted 7 lines
Quoted 4 lines

I am starting to get a little worried about the direction these patches
are heading in. How much of the system are we going to split into individual package per binaries?

I am wondering why this is a concern for you? If we keep the old package rdepends on the new ones I see no problem in allowing this granular packaging.

Taking this to a conclusion its heading towards, most recipes generating more than one binary would end up with this splitting code. I don't like having large blocks of python in each recipe and heading that way means we should probably change approach somehow.

Quoted 4 lines
Quoted 2 lines

My worry is that simpler recipes are easier to maintain, test and upgrade.

Maybe Sinan could try to rework this and move the python code to a class reducing code duplication?

The problem I'm trying to solve is I only need ps file out of this entire package. Everything else in this package is useless for me. I'm sure no-one wants dead code in their system especially if they are size constrained.

Quoted 4 lines

Ideal solution would be to have --with/without-foo support upstream that we can configure with PACKAGECONFIG.

I'm happy to look at other options if there is an alternative.

The question is how many other packages are you going to need just one thing out of?

You might want ps, someone else might want something different. We've had this problem a lot with util-linux and e2fsprogs so we split those in the end as the patches were getting painful.

Quoted 7 lines

If this is a widespread issue we might want to have a generic class, kind of like lib_package but for splitting the binaries into individual packages.

I just need an idea of how big this problem is going to be as I don't want to iterate over half the recipes in oe-core over the next 9 months! :)

I see. In terms of my project requirements, the only package that I'm planning to go after like this is sudo.

We should not open the path for subpackages across the board. We need to prefer PACKAGECONFIG as the right option.