Skip to contentSkip to conversation

systemd split networkd package

PATCH Superseded Adrian Freihofer 2 participants 7 messages Dec 11, 2024 – Dec 12, 2024

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore
#0 lore raw

cover systemd split networkd package

First of all, this patch series fixes a bug in systemd.bbclass that forces service files with an Also= relationship to be in the same package. This may often be the case, but as long as the RDEPENDS between the packages are set correctly, it is no problem to distribute the service files over several packages.

If service files referenced with Also= are no longer automatically added to the list of FILES, build errors caused by unpacked files are expected. A bitbake world has shown that there are only two recipes in oe-core that use automatic packing: busybox and dropbear. There are 2 patches that fix this.

Also= relation are probably most often used for sockets. Consequently the systemd.bbclass should also allow to split socket files into separate packages. However, that's proably practically not used and would cause tons of packaging errors. Therefore the automatic packaging feature for sockets is not changed.

The latest patch splits systemd-networkd into its own package. This adds a little extra complexity, but splitting systemd into packages is a good step towards making Yocto more usable in binary form.

Adrian Freihofer (5):
busybox: add klogd service to FILES
dropbear: add dropbearkey.service to FILES systemd.bbclass: refactor adding files systemd.bbclass: do not automatically add Also services
systemd: split networkd into its own package

 meta/classes-recipe/systemd.bbclass           | 28 ++++++++-----------
 meta/recipes-core/busybox/busybox.inc         |  9 +++++-
 .../recipes-core/dropbear/dropbear_2024.86.bb |  2 ++
 meta/recipes-core/systemd/systemd_256.8.bb    | 27 ++++++++++++++++--
 4 files changed, 46 insertions(+), 20 deletions(-)
Signature · 1 line
2.47.0
#1 lore raw

PATCH 1/5 busybox: add klogd service to FILES

From: Adrian Freihofer <[email protected]>

busybox syslog.service file contains:
Also=busybox-klogd.service

That's why busybox-klog.service got automatically added to FILES:busybox-syslog. Since this incorrect auto adding feature gets dropped from systemd.bbclass this needs to be added explicitly.

---
 meta/recipes-core/busybox/busybox.inc | 9 ++++++++-
 1 file changed, 8 insertions(+), 1 deletion(-)
meta/recipes-core/busybox/busybox.inc +8 -1
index 67a9111e5bd..f778cff0f4a 100644
@@ -24,7 +24,14 @@ EXTRA_OEMAKE = "CC='${CC}' LD='${CCLD}' V=1 ARCH=${TARGET_ARCH} CROSS_COMPILE=${
 PACKAGES =+ "${PN}-httpd ${PN}-udhcpd ${PN}-udhcpc ${PN}-syslog ${PN}-mdev ${PN}-hwclock"
 
 FILES:${PN}-httpd = "${sysconfdir}/init.d/busybox-httpd /srv/www"
-FILES:${PN}-syslog = "${sysconfdir}/init.d/syslog* ${sysconfdir}/syslog-startup.conf* ${sysconfdir}/syslog.conf* ${systemd_system_unitdir}/syslog.service ${sysconfdir}/default/busybox-syslog"
+FILES:${PN}-syslog = "\
+    ${sysconfdir}/init.d/syslog* \
+    ${sysconfdir}/syslog-startup.conf* \
+    ${sysconfdir}/syslog.conf* \
+    ${systemd_system_unitdir}/syslog.service \
+    ${systemd_system_unitdir}/busybox-klogd.service \
+    ${sysconfdir}/default/busybox-syslog \
+"
 FILES:${PN}-mdev = "${sysconfdir}/init.d/mdev ${sysconfdir}/mdev.conf ${sysconfdir}/mdev/*"
 FILES:${PN}-udhcpd = "${sysconfdir}/init.d/busybox-udhcpd"
 FILES:${PN}-udhcpc = "${sysconfdir}/udhcpc.d ${datadir}/udhcpc"
Signature · 1 line
2.47.0
#2 lore raw

PATCH 2/5 dropbear: add dropbearkey.service to FILES

In reply to Adrian Freihofer

From: Adrian Freihofer <[email protected]>

dropbear.socket file contains:
Also=dropbearkey.service

That's why dropbearkey.service got automatically added to FILES. Since this incorrect auto adding feature gets dropped from systemd.bbclass this needs to be added explicitly.

---
 meta/recipes-core/dropbear/dropbear_2024.86.bb | 2 ++
 1 file changed, 2 insertions(+)
meta/recipes-core/dropbear/dropbear_2024.86.bb +2 -0
index e3f4d17896c..be246a0ccd5 100644
@@ -129,3 +129,5 @@ pkg_postrm:${PN} () {
 }
 
 CONFFILES:${PN} = "${sysconfdir}/default/dropbear"
+
+FILES:${PN} += "${systemd_system_unitdir}/dropbearkey.service"
Signature · 1 line
2.47.0
#3 lore raw

PATCH 3/5 systemd.bbclass: refactor adding files

In reply to Adrian Freihofer

From: Adrian Freihofer <[email protected]>

The keys variable was intended as an array of keys. But it looks like this has not been used for more than 10 years now. Adding files automatically to packages needs probably anyway very specific code rather than a generic loop. Lets simplify this a bit. Using python code should also not be slower for these usually small files.

---
 meta/classes-recipe/systemd.bbclass | 28 ++++++++++++----------------
 1 file changed, 12 insertions(+), 16 deletions(-)
meta/classes-recipe/systemd.bbclass +12 -16
index 7324af8555d..be77da48125 100644
@@ -124,29 +124,26 @@ python systemd_populate_packages() {
         return appended
 
     # Add systemd files to FILES:*-systemd, parse for Also= and follow recursive
-    def systemd_add_files_and_parse(pkg_systemd, path, service, keys):
+    def systemd_add_files_and_parse(pkg_systemd, path, service):
         # avoid infinite recursion
         if systemd_append_file(pkg_systemd, oe.path.join(path, service)):
             fullpath = oe.path.join(d.getVar("D"), path, service)
             if service.find('.service') != -1:
                 # for *.service add *@.service
                 service_base = service.replace('.service', '')
-                systemd_add_files_and_parse(pkg_systemd, path, service_base + '@.service', keys)
+                systemd_add_files_and_parse(pkg_systemd, path, service_base + '@.service')
             if service.find('.socket') != -1:
                 # for *.socket add *.service and *@.service
                 service_base = service.replace('.socket', '')
-                systemd_add_files_and_parse(pkg_systemd, path, service_base + '.service', keys)
-                systemd_add_files_and_parse(pkg_systemd, path, service_base + '@.service', keys)
-            for key in keys.split():
-                # recurse all dependencies found in keys ('Also';'Conflicts';..) and add to files
-                cmd = "grep %s %s | sed 's,%s=,,g' | tr ',' '\\n'" % (key, shlex.quote(fullpath), key)
-                pipe = os.popen(cmd, 'r')
-                line = pipe.readline()
-                while line:
-                    line = line.replace('\n', '')
-                    systemd_add_files_and_parse(pkg_systemd, path, line, keys)
-                    line = pipe.readline()
-                pipe.close()
+                systemd_add_files_and_parse(pkg_systemd, path, service_base + '.service')
+                systemd_add_files_and_parse(pkg_systemd, path, service_base + '@.service')
+            # Add all units which have an Also= referring a unit in this package to this package as well.
+            with open(fullpath, 'r') as unit_f:
+                for line in unit_f:
+                    if line.startswith('Also'):
+                        also_unit = line.split('=', 1)[1].strip()
+                        bb.warn("also: %s" % also_unit)
+                        systemd_add_files_and_parse(pkg_systemd, path, also_unit)
 
     # Check service-files and call systemd_add_files_and_parse for each entry
     def systemd_check_services():
@@ -155,7 +152,6 @@ python systemd_populate_packages() {
         searchpaths.append(d.getVar("systemd_user_unitdir"))
         systemd_packages = d.getVar('SYSTEMD_PACKAGES')
 
-        keys = 'Also'
         # scan for all in SYSTEMD_SERVICE[]
         for pkg_systemd in systemd_packages.split():
             for service in get_package_var(d, 'SYSTEMD_SERVICE', pkg_systemd).split():
@@ -179,7 +175,7 @@ python systemd_populate_packages() {
                             break
 
                 if path_found != '':
-                    systemd_add_files_and_parse(pkg_systemd, path_found, service, keys)
+                    systemd_add_files_and_parse(pkg_systemd, path_found, service)
                 else:
                     bb.fatal("Didn't find service unit '{0}', specified in SYSTEMD_SERVICE:{1}. {2}".format(
                         service, pkg_systemd, "Also looked for service unit '{0}'.".format(base) if base is not None else ""))
Signature · 1 line
2.47.0
#4 lore raw

PATCH 4/5 systemd.bbclass: do not automatically add Also services

In reply to Adrian Freihofer

From: Adrian Freihofer <[email protected]>

Starting a systemd service “Also” does not mean that both services must be in the same package. However, the systemd.bbclass enforces this.

Example:
a.service:
[Install]
Also=b.service
If a.service is packed in package A, b.service is automatically packed into package A as well. This happens even if b.service is explicitly added to package B using FILES and SYSTEMD_SERVICE variables.

The automatic packing of socket files with the corresponding service files is probably a widely used feature of systemd.bbclass. This bahavior does not change.

Adding regular service files to a package just because it is another service in the same package that "Also" uses the service is a bug that this commit fixes.

---
 meta/classes-recipe/systemd.bbclass | 14 +++++++-------
 1 file changed, 7 insertions(+), 7 deletions(-)
meta/classes-recipe/systemd.bbclass +7 -7
index be77da48125..4b4470b7b34 100644
@@ -132,18 +132,18 @@ python systemd_populate_packages() {
                 # for *.service add *@.service
                 service_base = service.replace('.service', '')
                 systemd_add_files_and_parse(pkg_systemd, path, service_base + '@.service')
+                # Add the socket unit which is referred by the Also= in this service file to the same package.
+                with open(fullpath, 'r') as unit_f:
+                    for line in unit_f:
+                        if line.startswith('Also'):
+                            also_unit = line.split('=', 1)[1].strip()
+                            if also_unit.find('.socket') != -1:
+                                systemd_add_files_and_parse(pkg_systemd, path, also_unit)
             if service.find('.socket') != -1:
                 # for *.socket add *.service and *@.service
                 service_base = service.replace('.socket', '')
                 systemd_add_files_and_parse(pkg_systemd, path, service_base + '.service')
                 systemd_add_files_and_parse(pkg_systemd, path, service_base + '@.service')
-            # Add all units which have an Also= referring a unit in this package to this package as well.
-            with open(fullpath, 'r') as unit_f:
-                for line in unit_f:
-                    if line.startswith('Also'):
-                        also_unit = line.split('=', 1)[1].strip()
-                        bb.warn("also: %s" % also_unit)
-                        systemd_add_files_and_parse(pkg_systemd, path, also_unit)
 
     # Check service-files and call systemd_add_files_and_parse for each entry
     def systemd_check_services():
Signature · 1 line
2.47.0
#5 lore raw

PATCH 5/5 systemd: split networkd into its own package

In reply to Adrian Freihofer

From: Adrian Freihofer <[email protected]>

Support of images with different network management implementations without having to recompile systemd and other components.

Fedora does this as well since systemd version 246.6-2:
https://src.fedoraproject.org/rpms/systemd/blob/rawhide/f/systemd.spec#_578
This proves that it is technically possible and officially supported by systemd.

---
 meta/recipes-core/systemd/systemd_256.8.bb | 27 +++++++++++++++++++---
 1 file changed, 24 insertions(+), 3 deletions(-)
meta/recipes-core/systemd/systemd_256.8.bb +24 -3
index c1915d7e941..ca43c8d2ddb 100644
@@ -412,6 +412,7 @@ PACKAGE_BEFORE_PN = "\
     ${PN}-journal-remote \
     ${PN}-kernel-install \
     ${PN}-mime \
+    ${PN}-networkd \
     ${PN}-rpm-macros \
     ${PN}-udev-rules \
     ${PN}-vconsole-setup \
@@ -440,6 +441,7 @@ SYSTEMD_PACKAGES = "${@bb.utils.contains('PACKAGECONFIG', 'binfmt', '${PN}-binfm
                     ${@bb.utils.contains('PACKAGECONFIG', 'microhttpd', '${PN}-journal-gatewayd', '', d)} \
                     ${@bb.utils.contains('PACKAGECONFIG', 'microhttpd', '${PN}-journal-remote', '', d)} \
                     ${@bb.utils.contains('PACKAGECONFIG', 'journal-upload', '${PN}-journal-upload', '', d)} \
+                    ${@bb.utils.contains('PACKAGECONFIG', 'networkd', '${PN}-networkd', '', d)} \
 "
 SYSTEMD_SERVICE:${PN}-binfmt = "systemd-binfmt.service"
 
@@ -453,7 +455,7 @@ GROUPADD_PARAM:${PN} = "-r systemd-journal;"
 GROUPADD_PARAM:udev = "-r render"
 GROUPADD_PARAM:${PN} += "${@bb.utils.contains('PACKAGECONFIG', 'polkit_hostnamed_fallback', '-r systemd-hostname;', '', d)}"
 USERADD_PARAM:${PN} += "${@bb.utils.contains('PACKAGECONFIG', 'coredump', '--system -d / -M --shell /sbin/nologin systemd-coredump;', '', d)}"
-USERADD_PARAM:${PN} += "${@bb.utils.contains('PACKAGECONFIG', 'networkd', '--system -d / -M --shell /sbin/nologin systemd-network;', '', d)}"
+USERADD_PARAM:${PN}-networkd = "--system -d / -M --shell /sbin/nologin systemd-network"
 USERADD_PARAM:${PN} += "${@bb.utils.contains('PACKAGECONFIG', 'polkit', '--system --no-create-home --user-group --home-dir ${datadir}/polkit-1 polkitd;', '', d)}"
 USERADD_PARAM:${PN} += "${@bb.utils.contains('PACKAGECONFIG', 'resolved', '--system -d / -M --shell /sbin/nologin systemd-resolve;', '', d)}"
 USERADD_PARAM:${PN} += "${@bb.utils.contains('PACKAGECONFIG', 'timesyncd', '--system -d / -M --shell /sbin/nologin systemd-timesync;', '', d)}"
@@ -620,6 +622,27 @@ FILES:${PN}-extra-utils = "\
 FILES:${PN}-mime = "${MIMEDIR}"
 RRECOMMENDS:${PN} += "${PN}-mime"
 
+FILES:${PN}-networkd = "\
+    ${bindir}/networkctl \
+    ${datadir}/dbus-1/system-services/org.freedesktop.network1.service \
+    ${datadir}/dbus-1/system.d/org.freedesktop.network1.conf \
+    ${datadir}/polkit-1/actions/org.freedesktop.network1.policy \
+    ${nonarch_libdir}/sysusers.d/systemd-network.conf \
+    ${nonarch_libdir}/tmpfiles.d/systemd-network.conf \
+    ${sysconfdir}/systemd/networkd.conf \
+    ${systemd_system_unitdir}/systemd-networkd* \
+    ${systemd_unitdir}/network/*.network \
+    ${systemd_unitdir}/network/*.network.example \
+    ${systemd_unitdir}/networkd.conf \
+    ${systemd_unitdir}/systemd-networkd* \
+"
+# systemd-networkd-persistent-storage.service BindsTo=systemd-networkd.service
+# systemd-networkd.service has Also=systemd-networkd-wait-online.service
+SYSTEMD_SERVICE:${PN}-networkd = "systemd-networkd.service"
+CONFFILES:${PN}-networkd = "${sysconfdir}/systemd/networkd.conf"
+RDEPENDS:${PN}-networkd += "${PN}"
+RRECOMMENDS:${PN} += "${@bb.utils.contains('PACKAGECONFIG', 'networkd', '${PN}-networkd', '', d)}"
+
 FILES:${PN}-udev-rules = "\
                         ${nonarch_libdir}/udev/rules.d/70-uaccess.rules \
                         ${nonarch_libdir}/udev/rules.d/71-seat.rules \
@@ -630,7 +653,6 @@ FILES:${PN}-udev-rules = "\
 CONFFILES:${PN} = "${sysconfdir}/systemd/coredump.conf \
 	${sysconfdir}/systemd/journald.conf \
 	${sysconfdir}/systemd/logind.conf \
-	${sysconfdir}/systemd/networkd.conf \
 	${sysconfdir}/systemd/pstore.conf \
 	${sysconfdir}/systemd/resolved.conf \
 	${sysconfdir}/systemd/sleep.conf \
@@ -698,7 +720,6 @@ FILES:${PN} = " ${base_bindir}/* \
                 ${nonarch_libdir}/modprobe.d/README \
                 ${datadir}/dbus-1/system.d/org.freedesktop.timedate1.conf \
                 ${datadir}/dbus-1/system.d/org.freedesktop.locale1.conf \
-                ${datadir}/dbus-1/system.d/org.freedesktop.network1.conf \
                 ${datadir}/dbus-1/system.d/org.freedesktop.resolve1.conf \
                 ${datadir}/dbus-1/system.d/org.freedesktop.systemd1.conf \
                 ${@bb.utils.contains('PACKAGECONFIG', 'polkit_hostnamed_fallback', '${datadir}/dbus-1/system.d/org.freedesktop.hostname1_no_polkit.conf', '', d)} \
Signature · 1 line
2.47.0
#6 lore raw

Re: [OE-core] [PATCH 5/5] systemd: split networkd into its own package

Quoted 5 lines · Adrian Freihofer via lists.openembedded.org

From: Adrian Freihofer <[email protected]>

Support of images with different network management implementations without having to recompile systemd and other components.

Quoted 7 lines

Fedora does this as well since systemd version 246.6-2:
https://src.fedoraproject.org/rpms/systemd/blob/rawhide/f/systemd.spec#_578
This proves that it is technically possible and officially supported by systemd.

---

Hi Adrian,

I believe this patch or another one from this series is responsible of some warnings we can see on the autobuilder:

stdio: WARNING: core-image-full-cmdline-1.0-r0 do_rootfs: User systemd-network has never been defined
stdio: WARNING: core-image-sato-1.0-r0 do_rootfs: User systemd-network has never been defined
stdio: WARNING: core-image-sato-sdk-1.0-r0 do_rootfs: User systemd-network has never been defined

https://valkyrie.yoctoproject.org/#/builders/10/builds/638/steps/11/logs/warnings
https://valkyrie.yoctoproject.org/#/builders/64/builds/559/steps/12/logs/warnings
https://valkyrie.yoctoproject.org/#/builders/13/builds/561/steps/12/logs/warnings

Can you look at this issue please ?

Signature · 3 lines
Mathieu Dubois-Briand, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com